honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove XPackExtension in favor of SecurityExtensions (elastic/x-pack-elasticsearch#3734) 

This change removes the XPackExtension mechanism in favor of
SecurityExtension that can be loaded via SPI and doesn't need
another (duplicate) plugin infrastructure

Original commit: elastic/x-pack-elasticsearch@f39e62a040
This commit is contained in:
Simon Willnauer 2018-01-26 16:14:11 +01:00 committed by GitHub
parent 7bec4af206
commit 730e7075ab
37 changed files with 54 additions and 2530 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
docs/en/release-notes/xpack-breaking.asciidoc
View File

@ -31,6 +31,10 @@ Security::
mappings, get field mappings and field capabilities API are now only the ones
that the user is authorized to access in case field level security is enabled.
* The legacy `XPackExtension` extension mechanism has been removed and replaced
with an SPI based extension mechanism that is installed and built as an elasticsearch
plugin.
See also:
* <<breaking-changes-7.0,{es} Breaking Changes>>

30
docs/en/security/authentication/custom-realm.asciidoc
View File

@ -3,7 +3,8 @@
If you are using an authentication system that is not supported out-of-the-box
by {security}, you can create a custom realm to interact with it to authenticate
users. You implement a custom realm as an {xpack} extension.
users. You implement a custom realm as an SPI loaded security extension
as part of an ordinary elasticsearch plugin.
[[implementing-custom-realm]]
==== Implementing a Custom Realm
@ -25,7 +26,7 @@ To create a custom realm, you need to:
To package your custom realm as a plugin:
. Implement an extension class for your realm that extends
`org.elasticsearch.xpack.extensions.XPackExtension`. There you need to
`org.elasticsearch.xpack.core.security.SecurityExtension`. There you need to
override one or more of the following methods:
+
[source,java]
@ -54,29 +55,18 @@ in certain authentication failure events.
[source,java]
----------------------------------------------------
@Override
public Collection<String> getRestHeaders() {
...
}
----------------------------------------------------
+
The `getRestHeaders` method returns a collection of header names that should be
copied from the request into the `ThreadContext` where they can be accessed by
the realm.
+
[source,java]
----------------------------------------------------
@Override
public List<String> getSettingsFilter() {
...
}
----------------------------------------------------
+
The `getSettingsFilter` method returns a list of setting names that should be
filtered from the settings APIs as they may contain sensitive credentials.
The `Plugin#getSettingsFilter` method returns a list of setting names that should be
filtered from the settings APIs as they may contain sensitive credentials. Note this method is not
part of the `SecurityExtension` interface, it's available as part of the elasticsearch plugin main class.
. Create a build configuration file for the plugin; Gradle is our recommendation.
. Create a `x-pack-extension-descriptor.properties` descriptor file for the
extension.
. Create a `META-INF/services/org.elasticsearch.xpack.core.security.SecurityExtension` descriptor file for the
extension that contains the fully qualified class name of your `org.elasticsearch.xpack.core.security.SecurityExtension` implementation
. Bundle all in a single zip file.
[[using-custom-realm]]
@ -85,12 +75,12 @@ filtered from the settings APIs as they may contain sensitive credentials.
To use a custom realm:
. Install the realm extension on each node in the cluster. You run
`bin/x-pack/extension` with the `install` sub-command and specify the URL
`bin/elasticsearch-plugin` with the `install` sub-command and specify the URL
pointing to the zip file that contains the extension. For example:
+
[source,shell]
----------------------------------------
bin/x-pack/extension install file:///<path>/my-realm-1.0.zip
bin/elasticsearch-plugin install file:///<path>/my-realm-1.0.zip
----------------------------------------
. Add a realm configuration of the appropriate realm type to `elasticsearch.yml`

20
docs/en/security/authorization/custom-roles-provider.asciidoc
View File

@ -3,7 +3,8 @@
If you need to retrieve user roles from a system not supported out-of-the-box
by {security}, you can create a custom roles provider to retrieve and resolve
roles. You implement a custom roles provider as an {xpack} extension.
roles. You implement a custom roles provider as an SPI loaded security extension
as part of an ordinary elasticsearch plugin.
[[implementing-custom-roles-provider]]
==== Implementing a Custom Roles Provider
@ -22,8 +23,8 @@ To create a custom roles provider:
To package your custom roles provider as a plugin:
. Implement an extension class for your roles provider that extends
`org.elasticsearch.xpack.core.extensions.XPackExtension`. There you need to
. Implement an extension class for your roles provider that implements
`org.elasticsearch.xpack.core.security.SecurityExtension`. There you need to
override one or more of the following methods:
+
[source,java]
@ -51,12 +52,13 @@ public List<String> getSettingsFilter() {
}
----------------------------------------------------
+
The `getSettingsFilter` method returns a list of setting names that should be
filtered from the settings APIs as they may contain sensitive credentials.
The `Plugin#getSettingsFilter` method returns a list of setting names that should be
filtered from the settings APIs as they may contain sensitive credentials. Note this method is not
part of the `SecurityExtension` interface, it's available as part of the elasticsearch plugin main class.
. Create a build configuration file for the plugin; Gradle is our recommendation.
. Create a `x-pack-extension-descriptor.properties` descriptor file for the
extension.
. Create a `META-INF/services/org.elasticsearch.xpack.core.security.SecurityExtension` descriptor file for the
extension that contains the fully qualified class name of your `org.elasticsearch.xpack.core.security.SecurityExtension` implementation
. Bundle all in a single zip file.
[[using-custom-roles-provider]]
@ -65,12 +67,12 @@ filtered from the settings APIs as they may contain sensitive credentials.
To use a custom roles provider:
. Install the roles provider extension on each node in the cluster. You run
`bin/x-pack/extension` with the `install` sub-command and specify the URL
`bin/elasticsearch-plugin` with the `install` sub-command and specify the URL
pointing to the zip file that contains the extension. For example:
+
[source,shell]
----------------------------------------
bin/x-pack/extension install file:///<path>/my-roles-provider-1.0.zip
bin/elasticsearch-plugin install file:///<path>/my-roles-provider-1.0.zip
----------------------------------------
. Add any configuration parameters for any of the custom roles provider implementations

7
plugin/core/src/main/java/org/elasticsearch/xpack/core/XPackPlugin.java
View File

@ -43,7 +43,6 @@ import org.elasticsearch.xpack.core.action.TransportXPackInfoAction;
import org.elasticsearch.xpack.core.action.TransportXPackUsageAction;
import org.elasticsearch.xpack.core.action.XPackInfoAction;
import org.elasticsearch.xpack.core.action.XPackUsageAction;
import org.elasticsearch.xpack.core.extensions.XPackExtension;
import org.elasticsearch.xpack.core.rest.action.RestXPackInfoAction;
import org.elasticsearch.xpack.core.rest.action.RestXPackUsageAction;
import org.elasticsearch.xpack.core.ssl.SSLConfigurationReloader;
@ -59,7 +58,6 @@ import java.security.PrivilegedAction;
import java.time.Clock;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.function.Supplier;
@ -118,11 +116,6 @@ public class XPackPlugin extends XPackClientPlugin implements ScriptPlugin, Exte
this.licensing = new Licensing(settings);
}
// For tests only
public Collection<Class<? extends XPackExtension>> getExtensions() {
return Collections.emptyList();
}
// overridable by tests
protected Clock getClock() {
return Clock.systemUTC();

359
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/InstallXPackExtensionCommand.java
View File

@ -1,359 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import joptsimple.OptionSet;
import joptsimple.OptionSpec;
import org.apache.lucene.util.IOUtils;
import org.elasticsearch.SpecialPermission;
import org.elasticsearch.bootstrap.JarHell;
import org.elasticsearch.cli.ExitCodes;
import org.elasticsearch.cli.EnvironmentAwareCommand;
import org.elasticsearch.cli.Terminal;
import org.elasticsearch.cli.UserException;
import org.elasticsearch.common.SuppressForbidden;
import org.elasticsearch.common.io.FileSystemUtils;
import org.elasticsearch.env.Environment;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.URL;
import java.net.URLDecoder;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.Arrays;
import java.util.HashSet;
import java.util.List;
import java.util.ArrayList;
import java.util.Comparator;
import java.util.Collections;
import java.util.Set;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
import java.security.Policy;
import java.security.PermissionCollection;
import java.security.Permission;
import java.security.NoSuchAlgorithmException;
import java.security.Permissions;
import java.security.PrivilegedAction;
import java.security.AccessController;
import java.security.UnresolvedPermission;
import java.security.URIParameter;
import static org.elasticsearch.cli.Terminal.Verbosity.VERBOSE;
import static org.elasticsearch.xpack.core.XPackPlugin.resolveXPackExtensionsFile;
/**
* A command for the extension cli to install an extension into x-pack.
*
* The install command takes a URL to an extension zip.
*
* Extensions are packaged as zip files. Each packaged extension must contain an
* extension properties file. See {@link XPackExtensionInfo}.
* <p>
* The installation process first extracts the extensions files into a temporary
* directory in order to verify the extension satisfies the following requirements:
* <ul>
* <li>The property file exists and contains valid metadata. See {@link XPackExtensionInfo#readFromProperties(Path)}</li>
* <li>Jar hell does not exist, either between the extension's own jars or with the parent classloader (elasticsearch + x-pack)</li>
* <li>If the extension contains extra security permissions, the policy file is validated</li>
* </ul>
*/
final class InstallXPackExtensionCommand extends EnvironmentAwareCommand {
private final OptionSpec<Void> batchOption;
private final OptionSpec<String> arguments;
InstallXPackExtensionCommand() {
super("Install an extension");
this.batchOption = parser.acceptsAll(Arrays.asList("b", "batch"),
"Enable batch mode explicitly, automatic confirmation of security permission");
this.arguments = parser.nonOptions("extension id");
}
@Override
protected void execute(Terminal terminal, OptionSet options, Environment env) throws Exception {
// TODO: in jopt-simple 5.0 we can enforce a min/max number of positional args
List<String> args = arguments.values(options);
if (args.size() != 1) {
throw new UserException(ExitCodes.USAGE, "Must supply a single extension id argument");
}
String extensionURL = args.get(0);
boolean isBatch = options.has(batchOption) || System.console() == null;
execute(terminal, extensionURL, isBatch, env);
}
// pkg private for testing
void execute(Terminal terminal, String extensionId, boolean isBatch, Environment env) throws Exception {
if (Files.exists(resolveXPackExtensionsFile(env)) == false) {
terminal.println("xpack extensions directory [" + resolveXPackExtensionsFile(env) + "] does not exist. Creating...");
Files.createDirectories(resolveXPackExtensionsFile(env));
}
Path extensionZip = download(terminal, extensionId, env.tmpFile());
Path extractedZip = unzip(extensionZip, resolveXPackExtensionsFile(env));
install(terminal, extractedZip, env, isBatch);
}
/** Downloads the extension and returns the file it was downloaded to. */
@SuppressForbidden(reason = "We use openStream to download extensions")
private Path download(Terminal terminal, String extensionURL, Path tmpDir) throws Exception {
terminal.println("-> Downloading " + URLDecoder.decode(extensionURL, "UTF-8"));
URL url = new URL(extensionURL);
Path zip = Files.createTempFile(tmpDir, null, ".zip");
try (InputStream in = url.openStream()) {
// must overwrite since creating the temp file above actually created the file
Files.copy(in, zip, StandardCopyOption.REPLACE_EXISTING);
}
return zip;
}
private Path unzip(Path zip, Path extensionDir) throws IOException, UserException {
// unzip extension to a staging temp dir
Path target = Files.createTempDirectory(extensionDir, ".installing-");
Files.createDirectories(target);
// TODO: we should wrap this in a try/catch and try deleting the target dir on failure?
try (ZipInputStream zipInput = new ZipInputStream(Files.newInputStream(zip))) {
ZipEntry entry;
byte[] buffer = new byte[8192];
while ((entry = zipInput.getNextEntry()) != null) {
Path targetFile = target.resolve(entry.getName());
// TODO: handle name being an absolute path
// be on the safe side: do not rely on that directories are always extracted
// before their children (although this makes sense, but is it guaranteed?)
Files.createDirectories(targetFile.getParent());
if (entry.isDirectory() == false) {
try (OutputStream out = Files.newOutputStream(targetFile)) {
int len;
while((len = zipInput.read(buffer)) >= 0) {
out.write(buffer, 0, len);
}
}
}
zipInput.closeEntry();
}
}
Files.delete(zip);
return target;
}
/** Load information about the extension, and verify it can be installed with no errors. */
private XPackExtensionInfo verify(Terminal terminal, Path extensionRoot, Environment env, boolean isBatch) throws Exception {
// read and validate the extension descriptor
XPackExtensionInfo info = XPackExtensionInfo.readFromProperties(extensionRoot);
terminal.println(VERBOSE, info.toString());
// check for jar hell before any copying
jarHellCheck(extensionRoot);
// read optional security policy (extra permissions)
// if it exists, confirm or warn the user
Path policy = extensionRoot.resolve(XPackExtensionInfo.XPACK_EXTENSION_POLICY);
if (Files.exists(policy)) {
readPolicy(policy, terminal, env, isBatch);
}
return info;
}
/** check a candidate extension for jar hell before installing it */
private void jarHellCheck(Path candidate) throws Exception {
// create list of current jars in classpath
// including the x-pack jars (see $ES_CLASSPATH in bin/extension script)
final Set<URL> jars = new HashSet<>(JarHell.parseClassPath());
// add extension jars to the list
Path extensionJars[] = FileSystemUtils.files(candidate, "*.jar");
for (Path jar : extensionJars) {
jars.add(jar.toUri().toURL());
}
// TODO: no jars should be an error
// TODO: verify the classname exists in one of the jars!
// check combined (current classpath + new jars to-be-added)
JarHell.checkJarHell(jars);
}
/**
* Installs the extension from {@code tmpRoot} into the extensions dir.
*/
private void install(Terminal terminal, Path tmpRoot, Environment env, boolean isBatch) throws Exception {
List<Path> deleteOnFailure = new ArrayList<>();
deleteOnFailure.add(tmpRoot);
try {
XPackExtensionInfo info = verify(terminal, tmpRoot, env, isBatch);
final Path destination = resolveXPackExtensionsFile(env).resolve(info.getName());
if (Files.exists(destination)) {
throw new UserException(ExitCodes.USAGE,
"extension directory " + destination.toAbsolutePath() +
" already exists. To update the extension, uninstall it first using 'remove " +
info.getName() + "' command");
}
Files.move(tmpRoot, destination, StandardCopyOption.ATOMIC_MOVE);
terminal.println("-> Installed " + info.getName());
} catch (Exception installProblem) {
try {
IOUtils.rm(deleteOnFailure.toArray(new Path[0]));
} catch (IOException exceptionWhileRemovingFiles) {
installProblem.addSuppressed(exceptionWhileRemovingFiles);
}
throw installProblem;
}
}
/** Format permission type, name, and actions into a string */
static String formatPermission(Permission permission) {
StringBuilder sb = new StringBuilder();
String clazz = null;
if (permission instanceof UnresolvedPermission) {
clazz = ((UnresolvedPermission) permission).getUnresolvedType();
} else {
clazz = permission.getClass().getName();
}
sb.append(clazz);
String name = null;
if (permission instanceof UnresolvedPermission) {
name = ((UnresolvedPermission) permission).getUnresolvedName();
} else {
name = permission.getName();
}
if (name != null && name.length() > 0) {
sb.append(' ');
sb.append(name);
}
String actions = null;
if (permission instanceof UnresolvedPermission) {
actions = ((UnresolvedPermission) permission).getUnresolvedActions();
} else {
actions = permission.getActions();
}
if (actions != null && actions.length() > 0) {
sb.append(' ');
sb.append(actions);
}
return sb.toString();
}
/**
* Parses extension policy into a set of permissions
*/
static PermissionCollection parsePermissions(Path file, Path tmpDir) throws IOException {
// create a zero byte file for "comparison"
// this is necessary because the default policy impl automatically grants two permissions:
// 1. permission to exitVM (which we ignore)
// 2. read permission to the code itself (e.g. jar file of the code)
Path emptyPolicyFile = Files.createTempFile(tmpDir, "empty", "tmp");
final Policy emptyPolicy;
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
sm.checkPermission(new SpecialPermission());
}
emptyPolicy =
AccessController.doPrivileged((PrivilegedAction<Policy>) () -> {
try {
return Policy.getInstance("JavaPolicy", new URIParameter(emptyPolicyFile.toUri()));
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
});
IOUtils.rm(emptyPolicyFile);
// parse the extension's policy file into a set of permissions
final Policy policy =
AccessController.doPrivileged((PrivilegedAction<Policy>) () -> {
try {
return Policy.getInstance("JavaPolicy", new URIParameter(file.toUri()));
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
});
PermissionCollection permissions = policy.getPermissions(XPackExtensionSecurity.class.getProtectionDomain());
// this method is supported with the specific implementation we use, but just check for safety.
if (permissions == Policy.UNSUPPORTED_EMPTY_COLLECTION) {
throw new UnsupportedOperationException("JavaPolicy implementation does not support retrieving permissions");
}
PermissionCollection actualPermissions = new Permissions();
for (Permission permission : Collections.list(permissions.elements())) {
if (!emptyPolicy.implies(XPackExtensionSecurity.class.getProtectionDomain(), permission)) {
actualPermissions.add(permission);
}
}
actualPermissions.setReadOnly();
return actualPermissions;
}
/**
* Reads extension policy, prints/confirms exceptions
*/
static void readPolicy(Path file, Terminal terminal, Environment environment, boolean batch) throws IOException {
PermissionCollection permissions = parsePermissions(file, environment.tmpFile());
List<Permission> requested = Collections.list(permissions.elements());
if (requested.isEmpty()) {
terminal.println(Terminal.Verbosity.VERBOSE, "extension has a policy file with no additional permissions");
return;
}
// sort permissions in a reasonable order
Collections.sort(requested, new Comparator<Permission>() {
@Override
public int compare(Permission o1, Permission o2) {
int cmp = o1.getClass().getName().compareTo(o2.getClass().getName());
if (cmp == 0) {
String name1 = o1.getName();
String name2 = o2.getName();
if (name1 == null) {
name1 = "";
}
if (name2 == null) {
name2 = "";
}
cmp = name1.compareTo(name2);
if (cmp == 0) {
String actions1 = o1.getActions();
String actions2 = o2.getActions();
if (actions1 == null) {
actions1 = "";
}
if (actions2 == null) {
actions2 = "";
}
cmp = actions1.compareTo(actions2);
}
}
return cmp;
}
});
terminal.println(Terminal.Verbosity.NORMAL, "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@");
terminal.println(Terminal.Verbosity.NORMAL, "@ WARNING: x-pack extension requires additional permissions @");
terminal.println(Terminal.Verbosity.NORMAL, "@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@");
// print all permissions:
for (Permission permission : requested) {
terminal.println(Terminal.Verbosity.NORMAL, "* " + formatPermission(permission));
}
terminal.println(Terminal.Verbosity.NORMAL, "See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html");
terminal.println(Terminal.Verbosity.NORMAL, "for descriptions of what these permissions allow and the associated risks.");
if (!batch) {
terminal.println(Terminal.Verbosity.NORMAL, "");
String text = terminal.readText("Continue with installation? [y/N]");
if (!text.equalsIgnoreCase("y")) {
throw new RuntimeException("installation aborted by user");
}
}
}
}

53
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/ListXPackExtensionCommand.java
View File

@ -1,53 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import joptsimple.OptionSet;
import org.elasticsearch.cli.EnvironmentAwareCommand;
import org.elasticsearch.cli.Terminal;
import org.elasticsearch.env.Environment;
import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import static org.elasticsearch.cli.Terminal.Verbosity.VERBOSE;
import static org.elasticsearch.xpack.core.XPackPlugin.resolveXPackExtensionsFile;
/**
* A command for the extension cli to list extensions installed in x-pack.
*/
class ListXPackExtensionCommand extends EnvironmentAwareCommand {
ListXPackExtensionCommand() {
super("Lists installed x-pack extensions");
}
@Override
protected void execute(Terminal terminal, OptionSet options, Environment env) throws Exception {
if (Files.exists(resolveXPackExtensionsFile(env)) == false) {
throw new IOException("Extensions directory missing: " + resolveXPackExtensionsFile(env));
}
terminal.println(VERBOSE, "XPack Extensions directory: " + resolveXPackExtensionsFile(env));
final List<Path> extensions = new ArrayList<>();
try (DirectoryStream<Path> paths = Files.newDirectoryStream(resolveXPackExtensionsFile(env))) {
for (Path extension : paths) {
extensions.add(extension);
}
}
Collections.sort(extensions);
for (final Path extension : extensions) {
terminal.println(extension.getFileName().toString());
XPackExtensionInfo info = XPackExtensionInfo.readFromProperties(extension);
terminal.println(VERBOSE, info.toString());
}
}
}

68
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/RemoveXPackExtensionCommand.java
View File

@ -1,68 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import joptsimple.OptionSet;
import joptsimple.OptionSpec;
import org.apache.lucene.util.IOUtils;
import org.elasticsearch.cli.ExitCodes;
import org.elasticsearch.cli.EnvironmentAwareCommand;
import org.elasticsearch.cli.Terminal;
import org.elasticsearch.cli.UserException;
import org.elasticsearch.common.Strings;
import org.elasticsearch.env.Environment;
import org.elasticsearch.xpack.core.XPackPlugin;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.StandardCopyOption;
import java.util.ArrayList;
import java.util.List;
import static org.elasticsearch.cli.Terminal.Verbosity.VERBOSE;
/**
* A command for the extension cli to remove an extension from x-pack.
*/
class RemoveXPackExtensionCommand extends EnvironmentAwareCommand {
private final OptionSpec<String> arguments;
RemoveXPackExtensionCommand() {
super("Removes an extension from x-pack");
this.arguments = parser.nonOptions("extension name");
}
@Override
protected void execute(Terminal terminal, OptionSet options, Environment env) throws Exception {
// TODO: in jopt-simple 5.0 we can enforce a min/max number of positional args
List<String> args = arguments.values(options);
if (args.size() != 1) {
throw new UserException(ExitCodes.USAGE, "Must supply a single extension id argument");
}
execute(terminal, args.get(0), env);
}
// pkg private for testing
void execute(Terminal terminal, String extensionName, Environment env) throws Exception {
terminal.println("-> Removing " + Strings.coalesceToEmpty(extensionName) + "...");
Path extensionDir = XPackPlugin.resolveXPackExtensionsFile(env).resolve(extensionName);
if (Files.exists(extensionDir) == false) {
throw new UserException(ExitCodes.USAGE,
"Extension " + extensionName + " not found. Run 'bin/x-pack/extension list' to get list of installed extensions.");
}
List<Path> extensionPaths = new ArrayList<>();
terminal.println(VERBOSE, "Removing: " + extensionDir);
Path tmpExtensionDir = XPackPlugin.resolveXPackExtensionsFile(env).resolve(".removing-" + extensionName);
Files.move(extensionDir, tmpExtensionDir, StandardCopyOption.ATOMIC_MOVE);
extensionPaths.add(tmpExtensionDir);
IOUtils.rm(extensionPaths.toArray(new Path[extensionPaths.size()]));
}
}

95
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/XPackExtension.java
View File

@ -1,95 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.common.settings.Setting;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.watcher.ResourceWatcherService;
import org.elasticsearch.xpack.core.security.authc.AuthenticationFailureHandler;
import org.elasticsearch.xpack.core.security.authc.Realm;
import org.elasticsearch.xpack.core.security.authc.RealmConfig;
import org.elasticsearch.xpack.security.SecurityExtension;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* An extension point allowing to plug in custom functionality in x-pack authentication module.
* @deprecated use {@link SecurityExtension} via SPI instead
*/
@Deprecated
public abstract class XPackExtension implements SecurityExtension {
/**
* The name of the plugin.
*/
public abstract String name();
/**
* The description of the plugin.
*/
public abstract String description();
/**
* Returns headers which should be copied from REST requests to internal cluster requests.
*/
public Collection<String> getRestHeaders() {
return Collections.emptyList();
}
/**
* Returns authentication realm implementations added by this extension.
*
* The key of the returned {@link Map} is the type name of the realm, and the value
* is a {@link Realm.Factory} which will construct
* that realm for use in authentication when that realm type is configured.
*
* @param resourceWatcherService Use to watch configuration files for changes
*/
public Map<String, Realm.Factory> getRealms(ResourceWatcherService resourceWatcherService) {
return Collections.emptyMap();
}
/**
* Returns the set of {@link Setting settings} that may be configured for the each type of realm.
*
* Each <em>setting key</em> must be unqualified and is in the same format as will be provided via {@link RealmConfig#settings()}.
* If a given realm-type is not present in the returned map, then it will be treated as if it supported <em>all</em> possible settings.
*
* The life-cycle of an extension dictates that this method will be called before {@link #getRealms(ResourceWatcherService)}
*/
public Map<String, Set<Setting<?>>> getRealmSettings() { return Collections.emptyMap(); }
/**
* Returns a handler for authentication failures, or null to use the default handler.
*
* Only one installed extension may have an authentication failure handler. If more than
* one extension returns a non-null handler, an error is raised.
*/
public AuthenticationFailureHandler getAuthenticationFailureHandler() {
return null;
}
/**
* Returns a list of settings that should be filtered from API calls. In most cases,
* these settings are sensitive such as passwords.
*
* The value should be the full name of the setting or a wildcard that matches the
* desired setting.
* @deprecated use {@link Plugin#getSettingsFilter()} ()} via SPI extension instead
*/
@Deprecated
public List<String> getSettingsFilter() {
return Collections.emptyList();
}
@Override
public String toString() {
return name();
}
}

28
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/XPackExtensionCli.java
View File

@ -1,28 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.cli.LoggingAwareMultiCommand;
import org.elasticsearch.cli.MultiCommand;
import org.elasticsearch.cli.Terminal;
/**
* A cli tool for adding, removing and listing extensions for x-pack.
*/
public class XPackExtensionCli extends LoggingAwareMultiCommand {
private XPackExtensionCli() {
super("A tool for managing installed x-pack extensions");
subcommands.put("list", new ListXPackExtensionCommand());
subcommands.put("install", new InstallXPackExtensionCommand());
subcommands.put("remove", new RemoveXPackExtensionCommand());
}
public static void main(String[] args) throws Exception {
exit(new XPackExtensionCli().main(args, Terminal.DEFAULT));
}
}

125
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/XPackExtensionInfo.java
View File

@ -1,125 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.Version;
import org.elasticsearch.bootstrap.JarHell;
import java.io.IOException;
import java.io.InputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Properties;
public class XPackExtensionInfo {
public static final String XPACK_EXTENSION_PROPERTIES = "x-pack-extension-descriptor.properties";
public static final String XPACK_EXTENSION_POLICY = "x-pack-extension-security.policy";
private String name;
private String description;
private String version;
private String classname;
public XPackExtensionInfo() {
}
/**
* Information about extensions
*
* @param name Its name
* @param description Its description
* @param version Version number
*/
XPackExtensionInfo(String name, String description, String version, String classname) {
this.name = name;
this.description = description;
this.version = version;
this.classname = classname;
}
/** reads (and validates) extension metadata descriptor file */
public static XPackExtensionInfo readFromProperties(Path dir) throws IOException {
Path descriptor = dir.resolve(XPACK_EXTENSION_PROPERTIES);
Properties props = new Properties();
try (InputStream stream = Files.newInputStream(descriptor)) {
props.load(stream);
}
String name = props.getProperty("name");
if (name == null || name.isEmpty()) {
throw new IllegalArgumentException("Property [name] is missing in [" + descriptor + "]");
}
String description = props.getProperty("description");
if (description == null) {
throw new IllegalArgumentException("Property [description] is missing for extension [" + name + "]");
}
String version = props.getProperty("version");
if (version == null) {
throw new IllegalArgumentException("Property [version] is missing for extension [" + name + "]");
}
String xpackVersionString = props.getProperty("xpack.version");
if (xpackVersionString == null) {
throw new IllegalArgumentException("Property [xpack.version] is missing for extension [" + name + "]");
}
Version xpackVersion = Version.fromString(xpackVersionString);
if (xpackVersion.equals(Version.CURRENT) == false) {
throw new IllegalArgumentException("extension [" + name + "] is incompatible with Elasticsearch [" +
Version.CURRENT.toString() + "]. Was designed for version [" + xpackVersionString + "]");
}
String javaVersionString = props.getProperty("java.version");
if (javaVersionString == null) {
throw new IllegalArgumentException("Property [java.version] is missing for extension [" + name + "]");
}
JarHell.checkVersionFormat(javaVersionString);
JarHell.checkJavaVersion(name, javaVersionString);
String classname = props.getProperty("classname");
if (classname == null) {
throw new IllegalArgumentException("Property [classname] is missing for extension [" + name + "]");
}
return new XPackExtensionInfo(name, description, version, classname);
}
/**
* @return Extension's name
*/
public String getName() {
return name;
}
/**
* @return Extension's description if any
*/
public String getDescription() {
return description;
}
/**
* @return extension's classname
*/
public String getClassname() {
return classname;
}
/**
* @return Version number for the extension
*/
public String getVersion() {
return version;
}
@Override
public String toString() {
final StringBuilder information = new StringBuilder()
.append("- XPack Extension information:\n")
.append("Name: ").append(name).append("\n")
.append("Description: ").append(description).append("\n")
.append("Version: ").append(version).append("\n")
.append(" * Classname: ").append(classname);
return information.toString();
}
}

67
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/XPackExtensionPolicy.java
View File

@ -1,67 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.common.SuppressForbidden;
import java.net.URL;
import java.security.Policy;
import java.security.ProtectionDomain;
import java.security.CodeSource;
import java.security.Permission;
import java.security.SecurityPermission;
import java.util.Map;
final class XPackExtensionPolicy extends Policy {
static final Permission SET_POLICY_PERMISSION = new SecurityPermission("setPolicy");
static final Permission GET_POLICY_PERMISSION = new SecurityPermission("getPolicy");
static final Permission CREATE_POLICY_PERMISSION = new SecurityPermission("createPolicy.JavaPolicy");
// the base policy (es + plugins)
final Policy basePolicy;
// policy extensions
final Map<String, Policy> extensions;
// xpack code source location
final URL xpackURL;
/**
*
* @param basePolicy The base policy
* @param extensions Extra code source extension's policy
*/
XPackExtensionPolicy(Policy basePolicy, Map<String, Policy> extensions) {
this.basePolicy = basePolicy;
this.extensions = extensions;
xpackURL = XPackExtensionPolicy.class.getProtectionDomain().getCodeSource().getLocation();
}
private boolean isPolicyPermission(Permission permission) {
return GET_POLICY_PERMISSION.equals(permission) ||
CREATE_POLICY_PERMISSION.equals(permission) ||
SET_POLICY_PERMISSION.equals(permission);
}
@Override @SuppressForbidden(reason = "fast equals check is desired")
public boolean implies(ProtectionDomain domain, Permission permission) {
CodeSource codeSource = domain.getCodeSource();
if (codeSource != null && codeSource.getLocation() != null) {
if (codeSource.getLocation().equals(xpackURL) &&
isPolicyPermission(permission)) {
// forbid to get, create and set java policy in xpack codesource
// it is only granted at startup in order to let xpack add the extensions policy
// and make this policy the default.
return false;
}
// check for an additional extension permission: extension policy is
// only consulted for its codesources.
Policy extension = extensions.get(codeSource.getLocation().getFile());
if (extension != null && extension.implies(domain, permission)) {
return true;
}
}
return basePolicy.implies(domain, permission);
}
}

145
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/XPackExtensionSecurity.java
View File

@ -1,145 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.SpecialPermission;
import org.elasticsearch.common.SuppressForbidden;
import org.elasticsearch.common.io.PathUtils;
import org.elasticsearch.xpack.core.extensions.XPackExtensionInfo;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Map;
import java.util.HashMap;
import java.util.List;
import java.util.ArrayList;
import java.util.Collections;
import java.security.Policy;
import java.security.PrivilegedAction;
import java.security.AccessController;
import java.security.URIParameter;
import java.security.NoSuchAlgorithmException;
final class XPackExtensionSecurity {
private XPackExtensionSecurity() {}
/**
* Initializes the XPackExtensionPolicy
* Can only happen once!
*
* @param extsDirectory the directory where the extensions are installed
*/
static void configure(Path extsDirectory) throws IOException {
Map<String, Policy> map = getExtensionsPermissions(extsDirectory);
if (map.size() > 0) {
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
sm.checkPermission(new SpecialPermission());
}
AccessController.doPrivileged((PrivilegedAction<Void>) () -> {
Policy newPolicy = new XPackExtensionPolicy(Policy.getPolicy(), map);
Policy.setPolicy(newPolicy);
return null;
});
}
}
/**
* Sets properties (codebase URLs) for policy files.
* we look for matching extensions and set URLs to fit
*/
@SuppressForbidden(reason = "proper use of URL")
static Map<String, Policy> getExtensionsPermissions(Path extsDirectory) throws IOException {
Map<String, Policy> map = new HashMap<>();
// collect up lists of extensions
List<Path> extensionPaths = new ArrayList<>();
if (Files.exists(extsDirectory)) {
try (DirectoryStream<Path> stream = Files.newDirectoryStream(extsDirectory)) {
for (Path extension : stream) {
extensionPaths.add(extension);
}
}
}
// now process each one
for (Path extension : extensionPaths) {
Path policyFile = extension.resolve(XPackExtensionInfo.XPACK_EXTENSION_POLICY);
if (Files.exists(policyFile)) {
// first get a list of URLs for the extension's jars:
// we resolve symlinks so map is keyed on the normalize codebase name
List<URL> codebases = new ArrayList<>();
try (DirectoryStream<Path> jarStream = Files.newDirectoryStream(extension, "*.jar")) {
for (Path jar : jarStream) {
codebases.add(jar.toRealPath().toUri().toURL());
}
}
// parse the extension's policy file into a set of permissions
Policy policy = readPolicy(policyFile.toUri().toURL(), codebases.toArray(new URL[codebases.size()]));
// consult this policy for each of the extension's jars:
for (URL url : codebases) {
if (map.put(url.getFile(), policy) != null) {
// just be paranoid ok?
throw new IllegalStateException("per-extension permissions already granted for jar file: " + url);
}
}
}
}
return Collections.unmodifiableMap(map);
}
/**
* Reads and returns the specified {@code policyFile}.
* <p>
* Resources (e.g. jar files and directories) listed in {@code codebases} location
* will be provided to the policy file via a system property of the short name:
* e.g. <code>${codebase.joda-convert-1.2.jar}</code> would map to full URL.
*/
@SuppressForbidden(reason = "accesses fully qualified URLs to configure security")
static Policy readPolicy(URL policyFile, URL codebases[]) throws IOException {
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
sm.checkPermission(new SpecialPermission());
}
try {
try {
// set codebase properties
for (URL url : codebases) {
String shortName = PathUtils.get(url.toURI()).getFileName().toString();
AccessController.doPrivileged((PrivilegedAction<Void>) () -> {
System.setProperty("codebase." + shortName, url.toString());
return null;
});
}
URIParameter uri = new URIParameter(policyFile.toURI());
return AccessController.doPrivileged((PrivilegedAction<Policy>) () -> {
try {
return Policy.getInstance("JavaPolicy", uri);
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
});
} finally {
// clear codebase properties
for (URL url : codebases) {
String shortName = PathUtils.get(url.toURI()).getFileName().toString();
AccessController.doPrivileged((PrivilegedAction<Void>) () -> {
System.clearProperty("codebase." + shortName);
return null;
});
}
}
} catch (URISyntaxException e) {
throw new IllegalArgumentException("unable to parse policy file `" + policyFile + "`", e);
}
}
}

190
plugin/core/src/main/java/org/elasticsearch/xpack/core/extensions/XPackExtensionsService.java
View File

@ -1,190 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import java.io.IOException;
import java.net.URL;
import java.net.URLClassLoader;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;
import org.apache.logging.log4j.Logger;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.bootstrap.JarHell;
import org.elasticsearch.common.collect.Tuple;
import org.elasticsearch.common.io.FileSystemUtils;
import org.elasticsearch.common.logging.Loggers;
import org.elasticsearch.common.settings.Settings;
import static org.elasticsearch.common.io.FileSystemUtils.isAccessibleDirectory;
public class XPackExtensionsService {
private final Settings settings;
/**
* We keep around a list of extensions
*/
private final List<Tuple<XPackExtensionInfo, XPackExtension>> extensions;
/**
* Constructs a new XPackExtensionsService
*
* @param settings The settings of the system
* @param extsDirectory The directory extensions exist in, or null if extensions should not be loaded from the filesystem
* @param classpathExtensions Extensions that exist in the classpath which should be loaded
*/
public XPackExtensionsService(Settings settings, Path extsDirectory,
Collection<Class<? extends XPackExtension>> classpathExtensions) {
try {
XPackExtensionSecurity.configure(extsDirectory);
} catch (Exception e) {
throw new IllegalStateException("Unable to configure extension policy", e);
}
this.settings = settings;
List<Tuple<XPackExtensionInfo, XPackExtension>> extensionsLoaded = new ArrayList<>();
// first we load extensions that are on the classpath. this is for tests
for (Class<? extends XPackExtension> extClass : classpathExtensions) {
XPackExtension ext = loadExtension(extClass, settings);
XPackExtensionInfo extInfo = new XPackExtensionInfo(ext.name(), ext.description(), "NA", extClass.getName());
extensionsLoaded.add(new Tuple<>(extInfo, ext));
}
// now, find all the ones that are in plugins/xpack/extensions
if (extsDirectory != null) {
try {
List<Bundle> bundles = getExtensionBundles(extsDirectory);
List<Tuple<XPackExtensionInfo, XPackExtension>> loaded = loadBundles(bundles);
extensionsLoaded.addAll(loaded);
} catch (IOException ex) {
throw new IllegalStateException("Unable to initialize extensions", ex);
}
}
extensions = Collections.unmodifiableList(extensionsLoaded);
}
public List<XPackExtension> getExtensions() {
return extensions.stream().map(Tuple::v2).collect(Collectors.toList());
}
// a "bundle" is a an extension in a single classloader.
static class Bundle {
XPackExtensionInfo info;
List<URL> urls = new ArrayList<>();
}
static List<Bundle> getExtensionBundles(Path extsDirectory) throws IOException {
Logger logger = Loggers.getLogger(XPackExtensionsService.class);
// TODO: remove this leniency, but tests bogusly rely on it
if (!isAccessibleDirectory(extsDirectory, logger)) {
return Collections.emptyList();
}
List<Bundle> bundles = new ArrayList<>();
try (DirectoryStream<Path> stream = Files.newDirectoryStream(extsDirectory)) {
for (Path extension : stream) {
if (FileSystemUtils.isHidden(extension)) {
logger.trace("--- skip hidden extension file[{}]", extension.toAbsolutePath());
continue;
}
logger.trace("--- adding extension [{}]", extension.toAbsolutePath());
final XPackExtensionInfo info;
try {
info = XPackExtensionInfo.readFromProperties(extension);
} catch (IOException e) {
throw new IllegalStateException("Could not load extension descriptor for existing extension ["
+ extension.getFileName() + "]. Was the extension built before 2.0?", e);
}
List<URL> urls = new ArrayList<>();
try (DirectoryStream<Path> jarStream = Files.newDirectoryStream(extension, "*.jar")) {
for (Path jar : jarStream) {
// normalize with toRealPath to get symlinks out of our hair
urls.add(jar.toRealPath().toUri().toURL());
}
}
final Bundle bundle = new Bundle();
bundles.add(bundle);
bundle.info = info;
bundle.urls.addAll(urls);
}
}
return bundles;
}
private List<Tuple<XPackExtensionInfo, XPackExtension>> loadBundles(List<Bundle> bundles) {
List<Tuple<XPackExtensionInfo, XPackExtension>> exts = new ArrayList<>();
for (Bundle bundle : bundles) {
// jar-hell check the bundle against the parent classloader and the x-pack classloader
// pluginmanager does it, but we do it again, in case lusers mess with jar files manually
try {
final Set<URL> jars = new LinkedHashSet<>();
// add the parent jars to the list
jars.addAll(JarHell.parseClassPath());
// add the x-pack jars to the list
ClassLoader xpackLoader = getClass().getClassLoader();
// this class is loaded from the isolated x-pack plugin's classloader
if (xpackLoader instanceof URLClassLoader) {
jars.addAll(Arrays.asList(((URLClassLoader) xpackLoader).getURLs()));
}
jars.addAll(bundle.urls);
JarHell.checkJarHell(jars);
} catch (Exception e) {
throw new IllegalStateException("failed to load bundle " + bundle.urls + " due to jar hell", e);
}
// create a child to load the extension in this bundle
ClassLoader loader = URLClassLoader.newInstance(bundle.urls.toArray(new URL[0]), getClass().getClassLoader());
final Class<? extends XPackExtension> extClass = loadExtensionClass(bundle.info.getClassname(), loader);
final XPackExtension ext = loadExtension(extClass, settings);
exts.add(new Tuple<>(bundle.info, ext));
}
return Collections.unmodifiableList(exts);
}
private Class<? extends XPackExtension> loadExtensionClass(String className, ClassLoader loader) {
try {
return loader.loadClass(className).asSubclass(XPackExtension.class);
} catch (ClassNotFoundException e) {
throw new ElasticsearchException("Could not find extension class [" + className + "]", e);
}
}
private XPackExtension loadExtension(Class<? extends XPackExtension> extClass, Settings settings) {
try {
try {
return extClass.getConstructor(Settings.class).newInstance(settings);
} catch (NoSuchMethodException e) {
try {
return extClass.getConstructor().newInstance();
} catch (NoSuchMethodException e1) {
throw new ElasticsearchException("No constructor for [" + extClass + "]. An extension class must " +
"have either an empty default constructor or a single argument constructor accepting a " +
"Settings instance");
}
}
} catch (Exception e) {
throw new ElasticsearchException("Failed to load extension class [" + extClass.getName() + "]", e);
}
}
}

2
plugin/core/src/main/java/org/elasticsearch/xpack/security/SecurityExtension.java → plugin/core/src/main/java/org/elasticsearch/xpack/core/security/SecurityExtension.java
View File

@ -3,7 +3,7 @@
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.security;
package org.elasticsearch.xpack.core.security;
import org.apache.lucene.util.SPIClassIterator;
import org.elasticsearch.action.ActionListener;

14
plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/RealmSettings.java
View File

@ -9,7 +9,7 @@ import org.elasticsearch.common.settings.AbstractScopedSettings;
import org.elasticsearch.common.settings.SecureSetting;
import org.elasticsearch.common.settings.Setting;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.xpack.core.extensions.XPackExtension;
import org.elasticsearch.xpack.core.security.SecurityExtension;
import java.util.Collection;
import java.util.HashMap;
@ -41,7 +41,7 @@ import static org.elasticsearch.xpack.core.security.SecurityField.setting;
* </p>
* <p>
* The allowable settings for each realm-type are determined by calls to {@link InternalRealmsSettings#getSettings()} and
* {@link XPackExtension#getRealmSettings()}
* {@link org.elasticsearch.xpack.core.security.SecurityExtension#getRealmSettings()}
*/
public class RealmSettings {
@ -54,11 +54,11 @@ public class RealmSettings {
/**
* Add the {@link Setting} configuration for <em>all</em> realms to the provided list.
*/
public static void addSettings(List<Setting<?>> settingsList, List<XPackExtension> extensions) {
public static void addSettings(List<Setting<?>> settingsList, List<SecurityExtension> extensions) {
settingsList.add(getGroupSetting(extensions));
}
public static Collection<String> getSettingsFilter(List<XPackExtension> extensions) {
public static Collection<String> getSettingsFilter(List<SecurityExtension> extensions) {
return getSettingsByRealm(extensions).values().stream()
.flatMap(Collection::stream)
.filter(Setting::isFiltered)
@ -107,11 +107,11 @@ public class RealmSettings {
return PREFIX + name + "." + subKey;
}
private static Setting<Settings> getGroupSetting(List<XPackExtension> extensions) {
private static Setting<Settings> getGroupSetting(List<SecurityExtension> extensions) {
return Setting.groupSetting(PREFIX, getSettingsValidator(extensions), Setting.Property.NodeScope);
}
private static Consumer<Settings> getSettingsValidator(List<XPackExtension> extensions) {
private static Consumer<Settings> getSettingsValidator(List<SecurityExtension> extensions) {
final Map<String, Set<Setting<?>>> childSettings = getSettingsByRealm(extensions);
childSettings.forEach(RealmSettings::verify);
return validator(childSettings);
@ -121,7 +121,7 @@ public class RealmSettings {
* @return A map from <em>realm-type</em> to a collection of <code>Setting</code> objects.
* @see InternalRealmsSettings#getSettings()
*/
private static Map<String, Set<Setting<?>>> getSettingsByRealm(List<XPackExtension> extensions) {
private static Map<String, Set<Setting<?>>> getSettingsByRealm(List<SecurityExtension> extensions) {
final Map<String, Set<Setting<?>>> settingsByRealm = new HashMap<>(InternalRealmsSettings.getSettings());
if (extensions != null) {
extensions.forEach(ext -> {

178
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/InstallXPackExtensionCommandTests.java
View File

@ -1,178 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.apache.lucene.util.LuceneTestCase;
import org.elasticsearch.Version;
import org.elasticsearch.cli.MockTerminal;
import org.elasticsearch.cli.UserException;
import org.elasticsearch.common.io.FileSystemUtils;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.env.Environment;
import org.elasticsearch.env.TestEnvironment;
import org.elasticsearch.test.ESTestCase;
import org.junit.Before;
import java.io.IOException;
import java.io.InputStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.nio.file.DirectoryStream;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.StandardCopyOption;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;
@LuceneTestCase.SuppressFileSystems("*")
public class InstallXPackExtensionCommandTests extends ESTestCase {
Path home;
Environment env;
@Before
public void setUp() throws Exception {
super.setUp();
home = createTempDir();
Files.createDirectories(home.resolve("org/elasticsearch/xpack/extensions").resolve("xpack").resolve("extensions"));
env = TestEnvironment.newEnvironment(Settings.builder().put("path.home", home.toString()).build());
}
/**
* creates a fake jar file with empty class files
*/
static void writeJar(Path jar, String... classes) throws IOException {
try (ZipOutputStream stream = new ZipOutputStream(Files.newOutputStream(jar))) {
for (String clazz : classes) {
stream.putNextEntry(new ZipEntry(clazz + ".class")); // no package names, just support simple classes
}
}
}
static String writeZip(Path structure) throws IOException {
Path zip = createTempDir().resolve(structure.getFileName() + ".zip");
try (ZipOutputStream stream = new ZipOutputStream(Files.newOutputStream(zip))) {
Files.walkFileTree(structure, new SimpleFileVisitor<Path>() {
@Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
String target = structure.relativize(file).toString();
stream.putNextEntry(new ZipEntry(target));
Files.copy(file, stream);
return FileVisitResult.CONTINUE;
}
});
}
return zip.toUri().toURL().toString();
}
/**
* creates an extension .zip and returns the url for testing
*/
static String createExtension(String name, Path structure) throws IOException {
XPackExtensionTestUtil.writeProperties(structure,
"description", "fake desc",
"name", name,
"version", "1.0",
"xpack.version", Version.CURRENT.toString(),
"java.version", System.getProperty("java.specification.version"),
"classname", "FakeExtension");
writeJar(structure.resolve("extension.jar"), "FakeExtension");
return writeZip(structure);
}
static MockTerminal installExtension(String extensionUrl, Path home) throws Exception {
Environment env = TestEnvironment.newEnvironment(Settings.builder().put("path.home", home).build());
MockTerminal terminal = new MockTerminal();
new InstallXPackExtensionCommand().execute(terminal, extensionUrl, true, env);
return terminal;
}
void assertExtension(String name, Environment env) throws IOException {
Path got = env.pluginsFile().resolve("x-pack").resolve("x-pack-security").resolve("extensions").resolve(name);
assertTrue("dir " + name + " exists", Files.exists(got));
assertTrue("jar was copied", Files.exists(got.resolve("extension.jar")));
assertInstallCleaned(env);
}
void assertInstallCleaned(Environment env) throws IOException {
try (DirectoryStream<Path> stream = Files.newDirectoryStream(env.pluginsFile().resolve("x-pack").
resolve("x-pack-security").resolve("extensions"))) {
for (Path file : stream) {
if (file.getFileName().toString().startsWith(".installing")) {
fail("Installation dir still exists, " + file);
}
}
}
}
public void testSomethingWorks() throws Exception {
Path extDir = createTempDir();
String extZip = createExtension("fake", extDir);
installExtension(extZip, home);
assertExtension("fake", env);
}
public void testSpaceInUrl() throws Exception {
Path extDir = createTempDir();
String extZip = createExtension("fake", extDir);
Path extZipWithSpaces = createTempFile("foo bar", ".zip");
try (InputStream in = FileSystemUtils.openFileURLStream(new URL(extZip))) {
Files.copy(in, extZipWithSpaces, StandardCopyOption.REPLACE_EXISTING);
}
installExtension(extZipWithSpaces.toUri().toURL().toString(), home);
assertExtension("fake", env);
}
public void testMalformedUrlNotMaven() throws Exception {
// has two colons, so it appears similar to maven coordinates
MalformedURLException e = expectThrows(MalformedURLException.class, () -> {
installExtension("://host:1234", home);
});
assertTrue(e.getMessage(), e.getMessage().contains("no protocol"));
}
public void testJarHell() throws Exception {
Path extDir = createTempDir();
writeJar(extDir.resolve("other.jar"), "FakeExtension");
String extZip = createExtension("fake", extDir); // adds extension.jar with FakeExtension
IllegalStateException e = expectThrows(IllegalStateException.class, () -> installExtension(extZip, home));
assertTrue(e.getMessage(), e.getMessage().contains("jar hell"));
assertInstallCleaned(env);
}
public void testIsolatedExtension() throws Exception {
// these both share the same FakeExtension class
Path extDir1 = createTempDir();
String extZip1 = createExtension("fake1", extDir1);
installExtension(extZip1, home);
Path extDir2 = createTempDir();
String extZip2 = createExtension("fake2", extDir2);
installExtension(extZip2, home);
assertExtension("fake1", env);
assertExtension("fake2", env);
}
public void testExistingExtension() throws Exception {
String extZip = createExtension("fake", createTempDir());
installExtension(extZip, home);
UserException e = expectThrows(UserException.class, () -> installExtension(extZip, home));
assertTrue(e.getMessage(), e.getMessage().contains("already exists"));
assertInstallCleaned(env);
}
public void testMissingDescriptor() throws Exception {
Path extDir = createTempDir();
Files.createFile(extDir.resolve("fake.yml"));
String extZip = writeZip(extDir);
NoSuchFileException e = expectThrows(NoSuchFileException.class, () -> installExtension(extZip, home));
assertTrue(e.getMessage(), e.getMessage().contains("x-pack-extension-descriptor.properties"));
assertInstallCleaned(env);
}
}

172
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/ListXPackExtensionCommandTests.java
View File

@ -1,172 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.apache.lucene.util.LuceneTestCase;
import org.elasticsearch.Version;
import org.elasticsearch.cli.ExitCodes;
import org.elasticsearch.cli.MockTerminal;
import org.elasticsearch.cli.UserException;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.env.Environment;
import org.elasticsearch.env.TestEnvironment;
import org.elasticsearch.test.ESTestCase;
import org.junit.Before;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
import java.util.Arrays;
import java.util.Map;
import java.util.stream.Collectors;
@LuceneTestCase.SuppressFileSystems("*")
public class ListXPackExtensionCommandTests extends ESTestCase {
private Path home;
private Environment env;
@Before
public void setUp() throws Exception {
super.setUp();
home = createTempDir();
Settings settings = Settings.builder()
.put("path.home", home)
.build();
env = TestEnvironment.newEnvironment(settings);
Files.createDirectories(extensionsFile(env));
}
private static class MockListXPackExtensionCommand extends ListXPackExtensionCommand {
private final Environment env;
private MockListXPackExtensionCommand(final Environment env) {
this.env = env;
}
@Override
protected Environment createEnv(Map<String, String> settings) throws UserException {
return env;
}
@Override
protected boolean addShutdownHook() {
return false;
}
}
static String buildMultiline(String... args){
return Arrays.asList(args).stream().collect(Collectors.joining("\n", "", "\n"));
}
static void buildFakeExtension(Environment env, String description, String name, String className) throws IOException {
XPackExtensionTestUtil.writeProperties(extensionsFile(env).resolve(name),
"description", description,
"name", name,
"version", "1.0",
"xpack.version", Version.CURRENT.toString(),
"java.version", System.getProperty("java.specification.version"),
"classname", className);
}
static Path extensionsFile(final Environment env) throws IOException {
return env.pluginsFile().resolve("x-pack").resolve("x-pack-security").resolve("extensions");
}
static MockTerminal listExtensions(Path home, Environment env) throws Exception {
MockTerminal terminal = new MockTerminal();
int status = new MockListXPackExtensionCommand(env).main(new String[] { "-Epath.home=" + home }, terminal);
assertEquals(ExitCodes.OK, status);
return terminal;
}
static MockTerminal listExtensions(Path home, Environment env, String[] args) throws Exception {
String[] argsAndHome = new String[args.length + 1];
System.arraycopy(args, 0, argsAndHome, 0, args.length);
argsAndHome[args.length] = "-Epath.home=" + home;
MockTerminal terminal = new MockTerminal();
int status = new MockListXPackExtensionCommand(env).main(argsAndHome, terminal);
assertEquals(ExitCodes.OK, status);
return terminal;
}
public void testExtensionsDirMissing() throws Exception {
Files.delete(extensionsFile(env));
IOException e = expectThrows(IOException.class, () -> listExtensions(home, env));
assertTrue(e.getMessage(), e.getMessage().contains("Extensions directory missing"));
}
public void testNoExtensions() throws Exception {
MockTerminal terminal = listExtensions(home, env);
assertTrue(terminal.getOutput(), terminal.getOutput().isEmpty());
}
public void testNoExtensionsVerbose() throws Exception {
String[] params = { "-v" };
MockTerminal terminal = listExtensions(home, env, params);
assertEquals(terminal.getOutput(), buildMultiline("XPack Extensions directory: " + extensionsFile(env)));
}
public void testOneExtension() throws Exception {
buildFakeExtension(env, "", "fake", "org.fake");
MockTerminal terminal = listExtensions(home, env);
assertEquals(terminal.getOutput(), buildMultiline("fake"));
}
public void testTwoExtensions() throws Exception {
buildFakeExtension(env, "", "fake1", "org.fake1");
buildFakeExtension(env, "", "fake2", "org.fake2");
MockTerminal terminal = listExtensions(home, env);
assertEquals(terminal.getOutput(), buildMultiline("fake1", "fake2"));
}
public void testExtensionWithVerbose() throws Exception {
buildFakeExtension(env, "fake desc", "fake_extension", "org.fake");
String[] params = { "-v" };
MockTerminal terminal = listExtensions(home, env, params);
assertEquals(terminal.getOutput(), buildMultiline("XPack Extensions directory: " + extensionsFile(env),
"fake_extension", "- XPack Extension information:", "Name: fake_extension",
"Description: fake desc", "Version: 1.0", " * Classname: org.fake"));
}
public void testExtensionWithVerboseMultipleExtensions() throws Exception {
buildFakeExtension(env, "fake desc 1", "fake_extension1", "org.fake");
buildFakeExtension(env, "fake desc 2", "fake_extension2", "org.fake2");
String[] params = { "-v" };
MockTerminal terminal = listExtensions(home, env, params);
assertEquals(terminal.getOutput(), buildMultiline("XPack Extensions directory: " + extensionsFile(env),
"fake_extension1", "- XPack Extension information:", "Name: fake_extension1",
"Description: fake desc 1", "Version: 1.0", " * Classname: org.fake",
"fake_extension2", "- XPack Extension information:", "Name: fake_extension2",
"Description: fake desc 2", "Version: 1.0", " * Classname: org.fake2"));
}
public void testExtensionWithoutVerboseMultipleExtensions() throws Exception {
buildFakeExtension(env, "fake desc 1", "fake_extension1", "org.fake");
buildFakeExtension(env, "fake desc 2", "fake_extension2", "org.fake2");
MockTerminal terminal = listExtensions(home, env, new String[0]);
String output = terminal.getOutput();
assertEquals(output, buildMultiline("fake_extension1", "fake_extension2"));
}
public void testExtensionWithoutDescriptorFile() throws Exception{
Files.createDirectories(extensionsFile(env).resolve("fake1"));
NoSuchFileException e = expectThrows(NoSuchFileException.class, () -> listExtensions(home, env));
assertEquals(e.getFile(),
extensionsFile(env).resolve("fake1").resolve(XPackExtensionInfo.XPACK_EXTENSION_PROPERTIES).toString());
}
public void testExtensionWithWrongDescriptorFile() throws Exception{
XPackExtensionTestUtil.writeProperties(extensionsFile(env).resolve("fake1"),
"description", "fake desc");
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> listExtensions(home, env));
assertEquals(e.getMessage(), "Property [name] is missing in [" +
extensionsFile(env).resolve("fake1").resolve(XPackExtensionInfo.XPACK_EXTENSION_PROPERTIES).toString() + "]");
}
}

76
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/RemoveXPackExtensionCommandTests.java
View File

@ -1,76 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.apache.lucene.util.LuceneTestCase;
import org.elasticsearch.cli.MockTerminal;
import org.elasticsearch.cli.UserException;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.env.Environment;
import org.elasticsearch.env.TestEnvironment;
import org.elasticsearch.test.ESTestCase;
import org.junit.Before;
import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
@LuceneTestCase.SuppressFileSystems("*")
public class RemoveXPackExtensionCommandTests extends ESTestCase {
private Path home;
private Environment env;
@Before
public void setUp() throws Exception {
super.setUp();
home = createTempDir();
env = TestEnvironment.newEnvironment(Settings.builder().put("path.home", home.toString()).build());
}
Path createExtensionDir(Environment env) throws IOException {
Path path = env.pluginsFile().resolve("x-pack").resolve("x-pack-security").resolve("extensions");
return Files.createDirectories(path);
}
static MockTerminal removeExtension(String name, Path home) throws Exception {
Environment env = TestEnvironment.newEnvironment(Settings.builder().put("path.home", home).build());
MockTerminal terminal = new MockTerminal();
new RemoveXPackExtensionCommand().execute(terminal, name, env);
return terminal;
}
static void assertRemoveCleaned(Path extDir) throws IOException {
try (DirectoryStream<Path> stream = Files.newDirectoryStream(extDir)) {
for (Path file : stream) {
if (file.getFileName().toString().startsWith(".removing")) {
fail("Removal dir still exists, " + file);
}
}
}
}
public void testMissing() throws Exception {
Path extDir = createExtensionDir(env);
UserException e = expectThrows(UserException.class, () -> removeExtension("dne", home));
assertTrue(e.getMessage(), e.getMessage().contains("Extension dne not found"));
assertRemoveCleaned(extDir);
}
public void testBasic() throws Exception {
Path extDir = createExtensionDir(env);
Files.createDirectory(extDir.resolve("fake"));
Files.createFile(extDir.resolve("fake").resolve("extension.jar"));
Files.createDirectory(extDir.resolve("fake").resolve("subdir"));
Files.createDirectory(extDir.resolve("other"));
removeExtension("fake", home);
assertFalse(Files.exists(extDir.resolve("fake")));
assertTrue(Files.exists(extDir.resolve("other")));
assertRemoveCleaned(extDir);
}
}

161
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/XPackExtensionInfoTests.java
View File

@ -1,161 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.Version;
import org.elasticsearch.test.ESTestCase;
import java.nio.file.Path;
public class XPackExtensionInfoTests extends ESTestCase {
public void testReadFromProperties() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", "my_extension",
"version", "1.0",
"xpack.version", Version.CURRENT.toString(),
"java.version", System.getProperty("java.specification.version"),
"classname", "FakeExtension");
XPackExtensionInfo info = XPackExtensionInfo.readFromProperties(extensionDir);
assertEquals("my_extension", info.getName());
assertEquals("fake desc", info.getDescription());
assertEquals("1.0", info.getVersion());
assertEquals("FakeExtension", info.getClassname());
}
public void testReadFromPropertiesNameMissing() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir);
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("Property [name] is missing in"));
XPackExtensionTestUtil.writeProperties(extensionDir, "name", "");
IllegalArgumentException e1 = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e1.getMessage().contains("Property [name] is missing in"));
}
public void testReadFromPropertiesDescriptionMissing() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir, "name", "fake-extension");
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("[description] is missing"));
}
public void testReadFromPropertiesVersionMissing() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir, "description", "fake desc", "name", "fake-extension");
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("[version] is missing"));
}
public void testReadFromPropertiesElasticsearchVersionMissing() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", "my_extension",
"version", "1.0");
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("[xpack.version] is missing"));
}
public void testReadFromPropertiesJavaVersionMissing() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", "my_extension",
"xpack.version", Version.CURRENT.toString(),
"version", "1.0");
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("[java.version] is missing"));
}
public void testReadFromPropertiesJavaVersionIncompatible() throws Exception {
String extensionName = "fake-extension";
Path extensionDir = createTempDir().resolve(extensionName);
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", extensionName,
"xpack.version", Version.CURRENT.toString(),
"java.version", "1000000.0",
"classname", "FakeExtension",
"version", "1.0");
IllegalStateException e = expectThrows(IllegalStateException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage(), e.getMessage().contains(extensionName + " requires Java"));
}
public void testReadFromPropertiesBadJavaVersionFormat() throws Exception {
String extensionName = "fake-extension";
Path extensionDir = createTempDir().resolve(extensionName);
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", extensionName,
"xpack.version", Version.CURRENT.toString(),
"java.version", "1.7.0_80",
"classname", "FakeExtension",
"version", "1.0");
IllegalStateException e = expectThrows(IllegalStateException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage(),
e.getMessage().equals("version string must be a sequence of nonnegative decimal " +
"integers separated by \".\"'s and may have leading zeros but was 1.7.0_80"));
}
public void testReadFromPropertiesBogusElasticsearchVersion() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"version", "1.0",
"name", "my_extension",
"xpack.version", "bogus");
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("version needs to contain major, minor, and revision"));
}
public void testReadFromPropertiesOldElasticsearchVersion() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", "my_extension",
"version", "1.0",
"xpack.version", Version.V_5_0_0.toString());
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("Was designed for version [5.0.0]"));
}
public void testReadFromPropertiesJvmMissingClassname() throws Exception {
Path extensionDir = createTempDir().resolve("fake-extension");
XPackExtensionTestUtil.writeProperties(extensionDir,
"description", "fake desc",
"name", "my_extension",
"version", "1.0",
"xpack.version", Version.CURRENT.toString(),
"java.version", System.getProperty("java.specification.version"));
IllegalArgumentException e = expectThrows(IllegalArgumentException.class, () -> {
XPackExtensionInfo.readFromProperties(extensionDir);
});
assertTrue(e.getMessage().contains("Property [classname] is missing"));
}
}

59
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/XPackExtensionSecurityTests.java
View File

@ -1,59 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.test.ESTestCase;
import java.nio.file.Path;
import java.security.Permission;
import java.security.PermissionCollection;
import java.security.Permissions;
import java.util.Collections;
import java.util.List;
public class XPackExtensionSecurityTests extends ESTestCase {
/** Test that we can parse the set of permissions correctly for a simple policy */
public void testParsePermissions() throws Exception {
Path scratch = createTempDir();
Path testFile = this.getDataPath("security/simple-x-pack-extension-security.policy");
Permissions expected = new Permissions();
expected.add(new RuntimePermission("queuePrintJob"));
PermissionCollection actual = InstallXPackExtensionCommand.parsePermissions(testFile, scratch);
assertEquals(expected, actual);
}
/** Test that we can parse the set of permissions correctly for a complex policy */
public void testParseTwoPermissions() throws Exception {
Path scratch = createTempDir();
Path testFile = this.getDataPath("security/complex-x-pack-extension-security.policy");
Permissions expected = new Permissions();
expected.add(new RuntimePermission("getClassLoader"));
expected.add(new RuntimePermission("closeClassLoader"));
PermissionCollection actual = InstallXPackExtensionCommand.parsePermissions(testFile, scratch);
assertEquals(expected, actual);
}
/** Test that we can format some simple permissions properly */
public void testFormatSimplePermission() throws Exception {
assertEquals("java.lang.RuntimePermission queuePrintJob",
InstallXPackExtensionCommand.formatPermission(new RuntimePermission("queuePrintJob")));
}
/** Test that we can format an unresolved permission properly */
public void testFormatUnresolvedPermission() throws Exception {
Path scratch = createTempDir();
Path testFile = this.getDataPath("security/unresolved-x-pack-extension-security.policy");
PermissionCollection actual = InstallXPackExtensionCommand.parsePermissions(testFile, scratch);
List<Permission> permissions = Collections.list(actual.elements());
assertEquals(1, permissions.size());
assertEquals("org.fake.FakePermission fakeName", InstallXPackExtensionCommand.formatPermission(permissions.get(0)));
}
/** no guaranteed equals on these classes, we assert they contain the same set */
private void assertEquals(PermissionCollection expected, PermissionCollection actual) {
assertEquals(asSet(Collections.list(expected.elements())), asSet(Collections.list(actual.elements())));
}
}

31
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/XPackExtensionTestUtil.java
View File

@ -1,31 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Properties;
/** Utility methods for testing extensions */
public class XPackExtensionTestUtil {
/** convenience method to write a plugin properties file */
public static void writeProperties(Path pluginDir, String... stringProps) throws IOException {
assert stringProps.length % 2 == 0;
Files.createDirectories(pluginDir);
Path propertiesFile = pluginDir.resolve(XPackExtensionInfo.XPACK_EXTENSION_PROPERTIES);
Properties properties = new Properties();
for (int i = 0; i < stringProps.length; i += 2) {
properties.put(stringProps[i], stringProps[i + 1]);
}
try (OutputStream out = Files.newOutputStream(propertiesFile)) {
properties.store(out, "");
}
}
}

23
plugin/core/src/test/java/org/elasticsearch/xpack/core/extensions/XPackExtensionsServiceTests.java
View File

@ -1,23 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.core.extensions;
import org.elasticsearch.test.ESTestCase;
import java.nio.file.Files;
import java.nio.file.Path;
public class XPackExtensionsServiceTests extends ESTestCase {
public void testExistingPluginMissingDescriptor() throws Exception {
Path extensionsDir = createTempDir();
Files.createDirectory(extensionsDir.resolve("extension-missing-descriptor"));
IllegalStateException e = expectThrows(IllegalStateException.class, () -> {
XPackExtensionsService.getExtensionBundles(extensionsDir);
});
assertTrue(e.getMessage(),
e.getMessage().contains("Could not load extension descriptor for existing extension"));
}
}

44
plugin/security/src/main/java/org/elasticsearch/xpack/security/Security.java
View File

@ -23,7 +23,6 @@ import org.elasticsearch.cluster.node.DiscoveryNode;
import org.elasticsearch.cluster.node.DiscoveryNodes;
import org.elasticsearch.cluster.service.ClusterService;
import org.elasticsearch.common.Booleans;
import org.elasticsearch.common.Nullable;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.inject.Module;
import org.elasticsearch.common.inject.util.Providers;
@ -79,9 +78,8 @@ import org.elasticsearch.transport.TransportRequestHandler;
import org.elasticsearch.watcher.ResourceWatcherService;
import org.elasticsearch.xpack.core.XPackPlugin;
import org.elasticsearch.xpack.core.XPackSettings;
import org.elasticsearch.xpack.core.extensions.XPackExtension;
import org.elasticsearch.xpack.core.extensions.XPackExtensionsService;
import org.elasticsearch.xpack.core.security.SecurityContext;
import org.elasticsearch.xpack.core.security.SecurityExtension;
import org.elasticsearch.xpack.core.security.SecurityField;
import org.elasticsearch.xpack.core.security.SecuritySettings;
import org.elasticsearch.xpack.core.security.action.realm.ClearRealmCacheAction;
@ -228,12 +226,10 @@ import java.util.function.Function;
import java.util.function.Predicate;
import java.util.function.Supplier;
import java.util.function.UnaryOperator;
import java.util.stream.Collectors;
import static java.util.Collections.emptyList;
import static java.util.Collections.singletonList;
import static org.elasticsearch.cluster.metadata.IndexMetaData.INDEX_FORMAT_SETTING;
import static org.elasticsearch.xpack.core.XPackPlugin.resolveXPackExtensionsFile;
import static org.elasticsearch.xpack.core.XPackSettings.HTTP_SSL_ENABLED;
import static org.elasticsearch.xpack.core.security.SecurityLifecycleServiceField.SECURITY_TEMPLATE_NAME;
import static org.elasticsearch.xpack.security.SecurityLifecycleService.SECURITY_INDEX_NAME;
@ -266,17 +262,16 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
private final SetOnce<TokenService> tokenService = new SetOnce<>();
private final SetOnce<SecurityActionFilter> securityActionFilter = new SetOnce<>();
private final List<BootstrapCheck> bootstrapChecks;
private final XPackExtensionsService extensionsService;
private final List<SecurityExtension> securityExtensions = new ArrayList<>();
public Security(Settings settings, final Path configPath) {
this(settings, configPath, Collections.emptyList());
}
Security(Settings settings, final Path configPath, List<SecurityExtension> extensions) {
this.settings = settings;
this.transportClientMode = XPackPlugin.transportClientMode(settings);
this.env = transportClientMode ? null : new Environment(settings, configPath);
this.extensionsService = transportClientMode ? null : new XPackExtensionsService(settings, resolveXPackExtensionsFile(env),
Collections.emptyList());
this.enabled = XPackSettings.SECURITY_ENABLED.get(settings);
if (enabled && transportClientMode == false) {
validateAutoCreateIndex(settings);
@ -295,6 +290,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
} else {
this.bootstrapChecks = Collections.emptyList();
}
this.securityExtensions.addAll(extensions);
}
@Override
@ -353,8 +349,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
NamedXContentRegistry xContentRegistry, Environment environment,
NodeEnvironment nodeEnvironment, NamedWriteableRegistry namedWriteableRegistry) {
try {
return createComponents(client, threadPool, clusterService, resourceWatcherService,
extensionsService.getExtensions().stream().collect(Collectors.toList()));
return createComponents(client, threadPool, clusterService, resourceWatcherService);
} catch (final Exception e) {
throw new IllegalStateException("security initialization failed", e);
}
@ -362,8 +357,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
// pkg private for testing - tests want to pass in their set of extensions hence we are not using the extension service directly
Collection<Object> createComponents(Client client, ThreadPool threadPool, ClusterService clusterService,
ResourceWatcherService resourceWatcherService,
List<SecurityExtension> extensions) throws Exception {
ResourceWatcherService resourceWatcherService) throws Exception {
if (enabled == false) {
return Collections.emptyList();
}
@ -417,9 +411,6 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
Map<String, Realm.Factory> realmFactories = new HashMap<>(InternalRealms.getFactories(threadPool, resourceWatcherService,
getSslService(), nativeUsersStore, nativeRoleMappingStore, securityLifecycleService));
for (SecurityExtension extension : securityExtensions) {
extensions.add(extension);
}
for (SecurityExtension extension : extensions) {
Map<String, Realm.Factory> newRealms = extension.getRealms(resourceWatcherService);
for (Map.Entry<String, Realm.Factory> entry : newRealms.entrySet()) {
if (realmFactories.put(entry.getKey(), entry.getValue()) != null) {
@ -435,7 +426,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
AuthenticationFailureHandler failureHandler = null;
String extensionName = null;
for (SecurityExtension extension : extensions) {
for (SecurityExtension extension : securityExtensions) {
AuthenticationFailureHandler extensionFailureHandler = extension.getAuthenticationFailureHandler();
if (extensionFailureHandler != null && failureHandler != null) {
throw new IllegalStateException("Extensions [" + extensionName + "] and [" + extension.toString() + "] " +
@ -459,7 +450,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
final NativeRolesStore nativeRolesStore = new NativeRolesStore(settings, client, getLicenseState(), securityLifecycleService);
final ReservedRolesStore reservedRolesStore = new ReservedRolesStore();
List<BiConsumer<Set<String>, ActionListener<Set<RoleDescriptor>>>> rolesProviders = new ArrayList<>();
for (SecurityExtension extension : extensions) {
for (SecurityExtension extension : securityExtensions) {
rolesProviders.addAll(extension.getRolesProviders(settings, resourceWatcherService));
}
final CompositeRolesStore allRolesStore = new CompositeRolesStore(settings, fileRolesStore, nativeRolesStore,
@ -542,13 +533,13 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
@Override
public List<Setting<?>> getSettings() {
return getSettings(transportClientMode, extensionsService);
return getSettings(transportClientMode, securityExtensions);
}
/**
* Get the {@link Setting setting configuration} for all security components, including those defined in extensions.
*/
public static List<Setting<?>> getSettings(boolean transportClientMode, @Nullable XPackExtensionsService extensionsService) {
public static List<Setting<?>> getSettings(boolean transportClientMode, List<SecurityExtension> securityExtensions) {
List<Setting<?>> settingsList = new ArrayList<>();
if (transportClientMode) {
@ -567,7 +558,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
// authentication settings
AnonymousUser.addSettings(settingsList);
RealmSettings.addSettings(settingsList, extensionsService == null ? null : extensionsService.getExtensions());
RealmSettings.addSettings(settingsList, securityExtensions);
NativeRolesStore.addSettings(settingsList);
ReservedRealm.addSettings(settingsList);
AuthenticationService.addSettings(settingsList);
@ -596,8 +587,6 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
if (AuthenticationServiceField.RUN_AS_ENABLED.get(settings)) {
headers.add(AuthenticationServiceField.RUN_AS_USER_HEADER);
}
headers.addAll(extensionsService.getExtensions().stream()
.flatMap(e -> e.getRestHeaders().stream()).collect(Collectors.toList()));
return headers;
}
@ -605,12 +594,7 @@ public class Security extends Plugin implements ActionPlugin, IngestPlugin, Netw
public List<String> getSettingsFilter() {
List<String> asArray = settings.getAsList(SecurityField.setting("hide_settings"));
ArrayList<String> settingsFilter = new ArrayList<>(asArray);
if (transportClientMode == false) {
settingsFilter.addAll(RealmSettings.getSettingsFilter(extensionsService.getExtensions()));
for (XPackExtension extension : extensionsService.getExtensions()) {
settingsFilter.addAll(extension.getSettingsFilter());
}
}
settingsFilter.addAll(RealmSettings.getSettingsFilter(securityExtensions));
// hide settings where we don't define them - they are part of a group...
settingsFilter.add("transport.profiles.*." + SecurityField.setting("*"));
return settingsFilter;

6
plugin/security/src/test/java/org/elasticsearch/xpack/security/SecurityTests.java
View File

@ -31,6 +31,7 @@ import org.elasticsearch.test.ESTestCase;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.watcher.ResourceWatcherService;
import org.elasticsearch.xpack.core.XPackSettings;
import org.elasticsearch.xpack.core.security.SecurityExtension;
import org.elasticsearch.xpack.core.security.SecurityField;
import org.elasticsearch.xpack.core.security.authc.Realm;
import org.elasticsearch.xpack.core.security.authc.file.FileRealmSettings;
@ -94,7 +95,7 @@ public class SecurityTests extends ESTestCase {
Environment env = TestEnvironment.newEnvironment(settings);
licenseState = new TestUtils.UpdatableLicenseState();
SSLService sslService = new SSLService(settings, env);
security = new Security(settings, null) {
security = new Security(settings, null, Arrays.asList(extensions)) {
@Override
protected XPackLicenseState getLicenseState() {
return licenseState;
@ -118,8 +119,7 @@ public class SecurityTests extends ESTestCase {
Client client = mock(Client.class);
when(client.threadPool()).thenReturn(threadPool);
when(client.settings()).thenReturn(settings);
return security.createComponents(client, threadPool, clusterService, mock(ResourceWatcherService.class),
Arrays.asList(extensions));
return security.createComponents(client, threadPool, clusterService, mock(ResourceWatcherService.class));
}
private static <T> T findComponent(Class<T> type, Collection<Object> components) {

6
plugin/security/src/test/java/org/elasticsearch/xpack/security/authc/RealmSettingsTests.java
View File

@ -12,9 +12,9 @@ import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.test.ESTestCase;
import org.elasticsearch.test.SecuritySettingsSource;
import org.elasticsearch.xpack.core.XPackSettings;
import org.elasticsearch.xpack.core.extensions.XPackExtension;
import org.elasticsearch.xpack.core.security.authc.RealmSettings;
import org.elasticsearch.xpack.core.security.authc.support.Hasher;
import org.elasticsearch.xpack.core.security.SecurityExtension;
import java.util.ArrayList;
import java.util.Arrays;
@ -45,7 +45,7 @@ public class RealmSettingsTests extends ESTestCase {
* This test exists because (in 5.x), we want to be backwards compatible and accept custom realms that
* have not been updated to explicitly declare their settings.
*
* @see XPackExtension#getRealmSettings()
* @see org.elasticsearch.xpack.core.security.SecurityExtension#getRealmSettings()
*/
public void testRealmWithUnknownTypeAcceptsAllSettings() throws Exception {
final Settings.Builder settings = baseSettings("tam", true)
@ -322,7 +322,7 @@ public class RealmSettingsTests extends ESTestCase {
private Setting<?> group() {
final List<Setting<?>> list = new ArrayList<>();
final List<XPackExtension> noExtensions = Collections.emptyList();
final List<SecurityExtension> noExtensions = Collections.emptyList();
RealmSettings.addSettings(list, noExtensions);
assertThat(list, hasSize(1));
return list.get(0);

85
qa/security-example-extension/build.gradle
View File

@ -1,85 +0,0 @@
import org.elasticsearch.gradle.MavenFilteringHack
import org.elasticsearch.gradle.VersionProperties
apply plugin: 'elasticsearch.build'
dependencies {
provided "org.elasticsearch:elasticsearch:${versions.elasticsearch}"
provided project(path: ':x-pack-elasticsearch:plugin:core', configuration: 'runtime')
testCompile "org.elasticsearch.test:framework:${project.versions.elasticsearch}"
testCompile project(path: ':x-pack-elasticsearch:transport-client', configuration: 'runtime')
}
Map generateSubstitutions() {
def stringSnap = { version ->
if (version.endsWith("-SNAPSHOT")) {
return version.substring(0, version.length() - 9)
}
return version
}
return [
'version': stringSnap(version),
'xpack.version': stringSnap(VersionProperties.elasticsearch),
'java.version': targetCompatibility as String
]
}
String outputDir = "generated-resources/${project.name}"
task copyXPackPluginProps(type: Copy) {
from project(':x-pack-elasticsearch:plugin:core').file('src/main/plugin-metadata')
from project(':x-pack-elasticsearch:plugin:core').tasks.pluginProperties
into outputDir
}
project.sourceSets.test.output.dir(outputDir, builtBy: copyXPackPluginProps)
processResources {
MavenFilteringHack.filter(it, generateSubstitutions())
}
task buildZip(type:Zip, dependsOn: [jar]) {
from 'build/resources/main/x-pack-extension-descriptor.properties'
from 'build/resources/main/x-pack-extension-security.policy'
from project.jar
}
task integTest(type: org.elasticsearch.gradle.test.RestIntegTestTask) {
mustRunAfter precommit
}
integTestRunner {
systemProperty 'tests.security.manager', 'false'
}
integTestCluster {
dependsOn buildZip
plugin ':x-pack-elasticsearch:plugin'
setting 'xpack.security.authc.realms.custom.order', '0'
setting 'xpack.security.authc.realms.custom.type', 'custom'
setting 'xpack.security.authc.realms.custom.filtered_setting', 'should be filtered'
setting 'xpack.security.authc.realms.esusers.order', '1'
setting 'xpack.security.authc.realms.esusers.type', 'file'
setting 'xpack.security.authc.realms.native.type', 'native'
setting 'xpack.security.authc.realms.native.order', '2'
setting 'xpack.ml.enabled', 'false'
// This is important, so that all the modules are available too.
// There are index templates that use token filters that are in analysis-module and
// processors are being used that are in ingest-common module.
distribution = 'zip'
setupCommand 'setupDummyUser',
'bin/x-pack/users', 'useradd', 'test_user', '-p', 'x-pack-test-password', '-r', 'superuser'
setupCommand 'installExtension',
'bin/x-pack/extension', 'install', 'file:' + buildZip.archivePath
waitCondition = { node, ant ->
File tmpFile = new File(node.cwd, 'wait.success')
ant.get(src: "http://${node.httpUri()}/_cluster/health?wait_for_nodes=>=${numNodes}&wait_for_status=yellow",
dest: tmpFile.toString(),
username: 'test_user',
password: 'x-pack-test-password',
ignoreerrors: true,
retries: 10)
return tmpFile.exists()
}
}
check.dependsOn integTest

86
qa/security-example-extension/src/main/java/org/elasticsearch/example/ExampleExtension.java
View File

@ -1,86 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.example.realm.CustomAuthenticationFailureHandler;
import org.elasticsearch.example.realm.CustomRealm;
import org.elasticsearch.example.role.CustomInMemoryRolesProvider;
import org.elasticsearch.watcher.ResourceWatcherService;
import org.elasticsearch.xpack.core.extensions.XPackExtension;
import org.elasticsearch.xpack.core.security.authc.AuthenticationFailureHandler;
import org.elasticsearch.xpack.core.security.authc.Realm;
import org.elasticsearch.xpack.core.security.authz.RoleDescriptor;
import java.security.AccessController;
import java.security.PrivilegedAction;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.function.BiConsumer;
import static org.elasticsearch.example.role.CustomInMemoryRolesProvider.ROLE_A;
import static org.elasticsearch.example.role.CustomInMemoryRolesProvider.ROLE_B;
/**
* An example x-pack extension for testing custom realms and custom role providers.
*/
public class ExampleExtension extends XPackExtension {
static {
// check that the extension's policy works.
AccessController.doPrivileged((PrivilegedAction<Void>) () -> {
System.getSecurityManager().checkPrintJobAccess();
return null;
});
}
@Override
public String name() {
return "custom realm example";
}
@Override
public String description() {
return "a very basic implementation of a custom realm to validate it works";
}
@Override
public Map<String, Realm.Factory> getRealms(ResourceWatcherService resourceWatcherService) {
return Collections.singletonMap(CustomRealm.TYPE, CustomRealm::new);
}
@Override
public AuthenticationFailureHandler getAuthenticationFailureHandler() {
return new CustomAuthenticationFailureHandler();
}
@Override
public Collection<String> getRestHeaders() {
return Arrays.asList(CustomRealm.USER_HEADER, CustomRealm.PW_HEADER);
}
@Override
public List<String> getSettingsFilter() {
return Collections.singletonList("xpack.security.authc.realms.*.filtered_setting");
}
@Override
public List<BiConsumer<Set<String>, ActionListener<Set<RoleDescriptor>>>>
getRolesProviders(Settings settings, ResourceWatcherService resourceWatcherService) {
CustomInMemoryRolesProvider rp1 = new CustomInMemoryRolesProvider(settings, Collections.singletonMap(ROLE_A, "read"));
Map<String, String> roles = new HashMap<>();
roles.put(ROLE_A, "all");
roles.put(ROLE_B, "all");
CustomInMemoryRolesProvider rp2 = new CustomInMemoryRolesProvider(settings, roles);
return Arrays.asList(rp1, rp2);
}
}

50
qa/security-example-extension/src/main/java/org/elasticsearch/example/realm/CustomAuthenticationFailureHandler.java
View File

@ -1,50 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example.realm;
import org.elasticsearch.ElasticsearchSecurityException;
import org.elasticsearch.common.util.concurrent.ThreadContext;
import org.elasticsearch.rest.RestRequest;
import org.elasticsearch.xpack.core.security.authc.AuthenticationToken;
import org.elasticsearch.xpack.core.security.authc.DefaultAuthenticationFailureHandler;
import org.elasticsearch.transport.TransportMessage;
public class CustomAuthenticationFailureHandler extends DefaultAuthenticationFailureHandler {
@Override
public ElasticsearchSecurityException failedAuthentication(RestRequest request, AuthenticationToken token,
ThreadContext context) {
ElasticsearchSecurityException e = super.failedAuthentication(request, token, context);
// set a custom header
e.addHeader("WWW-Authenticate", "custom-challenge");
return e;
}
@Override
public ElasticsearchSecurityException failedAuthentication(TransportMessage message, AuthenticationToken token, String action,
ThreadContext context) {
ElasticsearchSecurityException e = super.failedAuthentication(message, token, action, context);
// set a custom header
e.addHeader("WWW-Authenticate", "custom-challenge");
return e;
}
@Override
public ElasticsearchSecurityException missingToken(RestRequest request, ThreadContext context) {
ElasticsearchSecurityException e = super.missingToken(request, context);
// set a custom header
e.addHeader("WWW-Authenticate", "custom-challenge");
return e;
}
@Override
public ElasticsearchSecurityException missingToken(TransportMessage message, String action, ThreadContext context) {
ElasticsearchSecurityException e = super.missingToken(message, action, context);
// set a custom header
e.addHeader("WWW-Authenticate", "custom-challenge");
return e;
}
}

70
qa/security-example-extension/src/main/java/org/elasticsearch/example/realm/CustomRealm.java
View File

@ -1,70 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example.realm;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.common.settings.SecureString;
import org.elasticsearch.common.util.concurrent.ThreadContext;
import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
import org.elasticsearch.xpack.core.security.authc.AuthenticationToken;
import org.elasticsearch.xpack.core.security.authc.Realm;
import org.elasticsearch.xpack.core.security.authc.RealmConfig;
import org.elasticsearch.xpack.core.security.authc.support.CharArrays;
import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
import org.elasticsearch.xpack.core.security.user.User;
public class CustomRealm extends Realm {
public static final String TYPE = "custom";
public static final String USER_HEADER = "User";
public static final String PW_HEADER = "Password";
public static final String KNOWN_USER = "custom_user";
public static final SecureString KNOWN_PW = new SecureString("x-pack-test-password".toCharArray());
static final String[] ROLES = new String[] { "superuser" };
public CustomRealm(RealmConfig config) {
super(TYPE, config);
}
@Override
public boolean supports(AuthenticationToken token) {
return token instanceof UsernamePasswordToken;
}
@Override
public UsernamePasswordToken token(ThreadContext threadContext) {
String user = threadContext.getHeader(USER_HEADER);
if (user != null) {
String password = threadContext.getHeader(PW_HEADER);
if (password != null) {
return new UsernamePasswordToken(user, new SecureString(password.toCharArray()));
}
}
return null;
}
@Override
public void authenticate(AuthenticationToken authToken, ActionListener<AuthenticationResult> listener) {
UsernamePasswordToken token = (UsernamePasswordToken)authToken;
final String actualUser = token.principal();
if (KNOWN_USER.equals(actualUser)) {
if (CharArrays.constantTimeEquals(token.credentials().getChars(), KNOWN_PW.getChars())) {
listener.onResponse(AuthenticationResult.success(new User(actualUser, ROLES)));
} else {
listener.onResponse(AuthenticationResult.unsuccessful("Invalid password for user " + actualUser, null));
}
} else {
listener.onResponse(AuthenticationResult.notHandled());
}
}
@Override
public void lookupUser(String username, ActionListener<User> listener) {
listener.onResponse(null);
}
}

57
qa/security-example-extension/src/main/java/org/elasticsearch/example/role/CustomInMemoryRolesProvider.java
View File

@ -1,57 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example.role;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.common.component.AbstractComponent;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.xpack.core.security.authz.RoleDescriptor;
import java.util.HashSet;
import java.util.Map;
import java.util.Set;
import java.util.function.BiConsumer;
/**
* A custom roles provider implementation for testing that serves
* static roles from memory.
*/
public class CustomInMemoryRolesProvider
extends AbstractComponent
implements BiConsumer<Set<String>, ActionListener<Set<RoleDescriptor>>> {
public static final String INDEX = "foo";
public static final String ROLE_A = "roleA";
public static final String ROLE_B = "roleB";
private final Map<String, String> rolePermissionSettings;
public CustomInMemoryRolesProvider(Settings settings, Map<String, String> rolePermissionSettings) {
super(settings);
this.rolePermissionSettings = rolePermissionSettings;
}
@Override
public void accept(Set<String> roles, ActionListener<Set<RoleDescriptor>> listener) {
Set<RoleDescriptor> roleDescriptors = new HashSet<>();
for (String role : roles) {
if (rolePermissionSettings.containsKey(role)) {
roleDescriptors.add(
new RoleDescriptor(role, new String[] { "all" },
new RoleDescriptor.IndicesPrivileges[] {
RoleDescriptor.IndicesPrivileges.builder()
.privileges(rolePermissionSettings.get(role))
.indices(INDEX)
.grantedFields("*")
.build()
}, null)
);
}
}
listener.onResponse(roleDescriptors);
}
}

6
qa/security-example-extension/src/main/resources/x-pack-extension-descriptor.properties
View File

@ -1,6 +0,0 @@
description=Custom Extension
version=${version}
name=exampleextension
classname=org.elasticsearch.example.ExampleExtension
java.version=${java.version}
xpack.version=${xpack.version}

3
qa/security-example-extension/src/main/resources/x-pack-extension-security.policy
View File

@ -1,3 +0,0 @@
grant {
permission java.lang.RuntimePermission "queuePrintJob";
};

121
qa/security-example-extension/src/test/java/org/elasticsearch/example/realm/CustomRealmIT.java
View File

@ -1,121 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example.realm;
import org.apache.http.message.BasicHeader;
import org.elasticsearch.action.admin.cluster.health.ClusterHealthResponse;
import org.elasticsearch.action.admin.cluster.node.info.NodeInfo;
import org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.ResponseException;
import org.elasticsearch.client.transport.NoNodeAvailableException;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.network.NetworkModule;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.common.util.concurrent.ThreadContext;
import org.elasticsearch.env.Environment;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.test.ESIntegTestCase;
import org.elasticsearch.xpack.core.XPackClientPlugin;
import org.elasticsearch.xpack.client.PreBuiltXPackTransportClient;
import java.util.Collection;
import java.util.Collections;
import java.util.List;
import static org.hamcrest.Matchers.is;
/**
* Integration test to test authentication with the custom realm
*/
public class CustomRealmIT extends ESIntegTestCase {
@Override
protected Settings externalClusterClientSettings() {
return Settings.builder()
.put(ThreadContext.PREFIX + "." + CustomRealm.USER_HEADER, CustomRealm.KNOWN_USER)
.put(ThreadContext.PREFIX + "." + CustomRealm.PW_HEADER, CustomRealm.KNOWN_PW.toString())
.put(NetworkModule.TRANSPORT_TYPE_KEY, "security4")
.build();
}
@Override
protected Collection<Class<? extends Plugin>> transportClientPlugins() {
return Collections.singleton(XPackClientPlugin.class);
}
public void testHttpConnectionWithNoAuthentication() throws Exception {
try {
getRestClient().performRequest("GET", "/");
fail("request should have failed");
} catch(ResponseException e) {
Response response = e.getResponse();
assertThat(response.getStatusLine().getStatusCode(), is(401));
String value = response.getHeader("WWW-Authenticate");
assertThat(value, is("custom-challenge"));
}
}
public void testHttpAuthentication() throws Exception {
Response response = getRestClient().performRequest("GET", "/",
new BasicHeader(CustomRealm.USER_HEADER, CustomRealm.KNOWN_USER),
new BasicHeader(CustomRealm.PW_HEADER, CustomRealm.KNOWN_PW.toString()));
assertThat(response.getStatusLine().getStatusCode(), is(200));
}
public void testTransportClient() throws Exception {
NodesInfoResponse nodeInfos = client().admin().cluster().prepareNodesInfo().get();
List<NodeInfo> nodes = nodeInfos.getNodes();
assertTrue(nodes.isEmpty() == false);
TransportAddress publishAddress = randomFrom(nodes).getTransport().address().publishAddress();
String clusterName = nodeInfos.getClusterName().value();
Settings settings = Settings.builder()
.put("cluster.name", clusterName)
.put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toAbsolutePath().toString())
.put(ThreadContext.PREFIX + "." + CustomRealm.USER_HEADER, CustomRealm.KNOWN_USER)
.put(ThreadContext.PREFIX + "." + CustomRealm.PW_HEADER, CustomRealm.KNOWN_PW.toString())
.build();
try (TransportClient client = new PreBuiltXPackTransportClient(settings)) {
client.addTransportAddress(publishAddress);
ClusterHealthResponse response = client.admin().cluster().prepareHealth().execute().actionGet();
assertThat(response.isTimedOut(), is(false));
}
}
public void testTransportClientWrongAuthentication() throws Exception {
NodesInfoResponse nodeInfos = client().admin().cluster().prepareNodesInfo().get();
List<NodeInfo> nodes = nodeInfos.getNodes();
assertTrue(nodes.isEmpty() == false);
TransportAddress publishAddress = randomFrom(nodes).getTransport().address().publishAddress();
String clusterName = nodeInfos.getClusterName().value();
Settings settings = Settings.builder()
.put("cluster.name", clusterName)
.put(Environment.PATH_HOME_SETTING.getKey(), createTempDir().toAbsolutePath().toString())
.put(ThreadContext.PREFIX + "." + CustomRealm.USER_HEADER, CustomRealm.KNOWN_USER + randomAlphaOfLength(1))
.put(ThreadContext.PREFIX + "." + CustomRealm.PW_HEADER, CustomRealm.KNOWN_PW.toString())
.build();
try (TransportClient client = new PreBuiltXPackTransportClient(settings)) {
client.addTransportAddress(publishAddress);
client.admin().cluster().prepareHealth().execute().actionGet();
fail("authentication failure should have resulted in a NoNodesAvailableException");
} catch (NoNodeAvailableException e) {
// expected
}
}
public void testSettingsFiltering() throws Exception {
NodesInfoResponse nodeInfos = client().admin().cluster().prepareNodesInfo().clear().setSettings(true).get();
for(NodeInfo info : nodeInfos.getNodes()) {
Settings settings = info.getSettings();
assertNotNull(settings);
assertNull(settings.get("xpack.security.authc.realms.custom.filtered_setting"));
assertEquals(CustomRealm.TYPE, settings.get("xpack.security.authc.realms.custom.type"));
}
}
}

46
qa/security-example-extension/src/test/java/org/elasticsearch/example/realm/CustomRealmTests.java
View File

@ -1,46 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example.realm;
import org.elasticsearch.action.support.PlainActionFuture;
import org.elasticsearch.common.settings.SecureString;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.util.concurrent.ThreadContext;
import org.elasticsearch.env.TestEnvironment;
import org.elasticsearch.test.ESTestCase;
import org.elasticsearch.xpack.core.security.authc.AuthenticationResult;
import org.elasticsearch.xpack.core.security.authc.RealmConfig;
import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
import org.elasticsearch.xpack.core.security.user.User;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.notNullValue;
public class CustomRealmTests extends ESTestCase {
public void testAuthenticate() {
Settings globalSettings = Settings.builder().put("path.home", createTempDir()).build();
CustomRealm realm = new CustomRealm(new RealmConfig("test", Settings.EMPTY, globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
SecureString password = CustomRealm.KNOWN_PW.clone();
UsernamePasswordToken token = new UsernamePasswordToken(CustomRealm.KNOWN_USER, password);
PlainActionFuture<AuthenticationResult> plainActionFuture = new PlainActionFuture<>();
realm.authenticate(token, plainActionFuture);
User user = plainActionFuture.actionGet().getUser();
assertThat(user, notNullValue());
assertThat(user.roles(), equalTo(CustomRealm.ROLES));
assertThat(user.principal(), equalTo(CustomRealm.KNOWN_USER));
}
public void testAuthenticateBadUser() {
Settings globalSettings = Settings.builder().put("path.home", createTempDir()).build();
CustomRealm realm = new CustomRealm(new RealmConfig("test", Settings.EMPTY, globalSettings, TestEnvironment.newEnvironment(globalSettings), new ThreadContext(globalSettings)));
SecureString password = CustomRealm.KNOWN_PW.clone();
UsernamePasswordToken token = new UsernamePasswordToken(CustomRealm.KNOWN_USER + "1", password);
PlainActionFuture<AuthenticationResult> plainActionFuture = new PlainActionFuture<>();
realm.authenticate(token, plainActionFuture);
final AuthenticationResult result = plainActionFuture.actionGet();
assertThat(result.getStatus(), equalTo(AuthenticationResult.Status.CONTINUE));
}
}

95
qa/security-example-extension/src/test/java/org/elasticsearch/example/role/CustomRolesProviderIT.java
View File

@ -1,95 +0,0 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.example.role;
import org.apache.http.message.BasicHeader;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.ResponseException;
import org.elasticsearch.common.network.NetworkModule;
import org.elasticsearch.common.settings.SecureString;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.util.concurrent.ThreadContext;
import org.elasticsearch.example.realm.CustomRealm;
import org.elasticsearch.plugins.Plugin;
import org.elasticsearch.test.ESIntegTestCase;
import org.elasticsearch.xpack.core.XPackClientPlugin;
import org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken;
import org.elasticsearch.xpack.core.security.client.SecurityClient;
import java.util.Collection;
import java.util.Collections;
import static org.elasticsearch.example.role.CustomInMemoryRolesProvider.INDEX;
import static org.elasticsearch.example.role.CustomInMemoryRolesProvider.ROLE_A;
import static org.elasticsearch.example.role.CustomInMemoryRolesProvider.ROLE_B;
import static org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken.basicAuthHeaderValue;
import static org.hamcrest.Matchers.is;
/**
* Integration test for custom roles providers.
*/
public class CustomRolesProviderIT extends ESIntegTestCase {
private static final String TEST_USER = "test_user";
private static final String TEST_PWD = "change_me";
@Override
protected Settings externalClusterClientSettings() {
return Settings.builder()
.put(ThreadContext.PREFIX + "." + CustomRealm.USER_HEADER, CustomRealm.KNOWN_USER)
.put(ThreadContext.PREFIX + "." + CustomRealm.PW_HEADER, CustomRealm.KNOWN_PW.toString())
.put(NetworkModule.TRANSPORT_TYPE_KEY, "security4")
.build();
}
@Override
protected Collection<Class<? extends Plugin>> transportClientPlugins() {
return Collections.singleton(XPackClientPlugin.class);
}
public void setupTestUser(String role) {
SecurityClient securityClient = new SecurityClient(client());
securityClient.preparePutUser(TEST_USER, TEST_PWD.toCharArray(), role).get();
}
public void testAuthorizedCustomRoleSucceeds() throws Exception {
setupTestUser(ROLE_B);
// roleB has all permissions on index "foo", so creating "foo" should succeed
Response response = getRestClient().performRequest("PUT", "/" + INDEX, authHeader());
assertThat(response.getStatusLine().getStatusCode(), is(200));
}
public void testFirstResolvedRoleTakesPrecedence() throws Exception {
// the first custom roles provider has set ROLE_A to only have read permission on the index,
// the second custom roles provider has set ROLE_A to have all permissions, but since
// the first custom role provider appears first in order, it should take precedence and deny
// permission to create the index
setupTestUser(ROLE_A);
// roleB has all permissions on index "foo", so creating "foo" should succeed
try {
getRestClient().performRequest("PUT", "/" + INDEX, authHeader());
fail(ROLE_A + " should not be authorized to create index " + INDEX);
} catch (ResponseException e) {
assertThat(e.getResponse().getStatusLine().getStatusCode(), is(403));
}
}
public void testUnresolvedRoleDoesntSucceed() throws Exception {
setupTestUser("unknown");
// roleB has all permissions on index "foo", so creating "foo" should succeed
try {
getRestClient().performRequest("PUT", "/" + INDEX, authHeader());
fail(ROLE_A + " should not be authorized to create index " + INDEX);
} catch (ResponseException e) {
assertThat(e.getResponse().getStatusLine().getStatusCode(), is(403));
}
}
private BasicHeader authHeader() {
return new BasicHeader(UsernamePasswordToken.BASIC_AUTH_HEADER,
basicAuthHeaderValue(TEST_USER, new SecureString(TEST_PWD.toCharArray())));
}
}

2
qa/security-example-spi-extension/src/main/java/org/elasticsearch/example/ExampleSecurityExtension.java
View File

@ -14,7 +14,7 @@ import org.elasticsearch.watcher.ResourceWatcherService;
import org.elasticsearch.xpack.core.security.authc.AuthenticationFailureHandler;
import org.elasticsearch.xpack.core.security.authc.Realm;
import org.elasticsearch.xpack.core.security.authz.RoleDescriptor;
import org.elasticsearch.xpack.security.SecurityExtension;
import org.elasticsearch.xpack.core.security.SecurityExtension;
import java.security.AccessController;
import java.security.PrivilegedAction;

0
qa/security-example-spi-extension/src/main/resources/META-INF/services/org.elasticsearch.xpack.security.SecurityExtension → qa/security-example-spi-extension/src/main/resources/META-INF/services/org.elasticsearch.xpack.core.security.SecurityExtension
View File