From 08ff3914fd85cf184c9f0a5a5a03456082297afe Mon Sep 17 00:00:00 2001 From: David Kyle Date: Thu, 28 Dec 2017 10:50:20 +0000 Subject: [PATCH 1/6] [DOCS] Correct over field usage for freq rare function (elastic/x-pack-elasticsearch#3435) Original commit: elastic/x-pack-elasticsearch@2b315610b93d327e7f5473c8db9f1edc6f79e695 --- docs/en/ml/functions/rare.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/ml/functions/rare.asciidoc b/docs/en/ml/functions/rare.asciidoc index 1973a279a6b..2485605557c 100644 --- a/docs/en/ml/functions/rare.asciidoc +++ b/docs/en/ml/functions/rare.asciidoc @@ -96,7 +96,7 @@ values occur. This function supports the following properties: * `by_field_name` (required) -* `over_field_name` (optional) +* `over_field_name` (required) * `partition_field_name` (optional) For more information about those properties, see From 4d392dec96b51b8dee76021ee95fccf23cce2dac Mon Sep 17 00:00:00 2001 From: Michael Basnight Date: Thu, 28 Dec 2017 07:58:56 -0500 Subject: [PATCH 2/6] Increase gradle heap to 2048m The BWC tests on 6.x need to test both 6.1 and 5.6, so the theory here is that they just need more memories. This bumps it to ensure the tests dont OOM. This bumps it from the previous 1536m, which was already bumped from the default of 1024m. Original commit: elastic/x-pack-elasticsearch@cf5700ef47eeefca4d916097e5296c019e14a5b2 --- gradle.properties | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gradle.properties b/gradle.properties index 2511c740bb5..2cd952fcbd9 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,2 +1,2 @@ org.gradle.daemon=false -org.gradle.jvmargs=-Xmx1536m +org.gradle.jvmargs=-Xmx2048m From ce366a731d6ddb6e5b794e96d78a8b8cd0bff4ea Mon Sep 17 00:00:00 2001 From: Jason Tedor Date: Sun, 31 Dec 2017 09:55:27 -0500 Subject: [PATCH 3/6] Remove JDK 9 flag on CI builds The special JDK 9 flags are no longer needed after the JDK maintainers decided at the last minute to permit illegal access by default. Relates elastic/x-pack-elasticsearch#3250 Original commit: elastic/x-pack-elasticsearch@a4cfcd8d1b3fec09123e49fa1e3da878887880a4 --- dev-tools/ci | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/dev-tools/ci b/dev-tools/ci index b501cb30216..5762ddf2aba 100755 --- a/dev-tools/ci +++ b/dev-tools/ci @@ -67,16 +67,6 @@ case $key in "-Dtests.jvm.argline=-Dbuild.snapshot=false" ) ;; - jdk9) - GRADLE_CLI_ARGS=( - "-Pxpack.kibana.build=false" - "--info" - "check" - "-Dtests.network=true" - "-Dtests.badapples=true" - -Dtests.jvm.argline="--add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.nio.file=ALL-UNNAMED --add-opens=java.base/java.security.cert=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/javax.net.ssl=ALL-UNNAMED" - ) - ;; *) echo "Unsupported cli argument $1. Allowed arguments are packagingTest or check. No argument defaults to check." exit 1;; From e9abf1808902987c6cf0a1d13a62a25d3047dc3c Mon Sep 17 00:00:00 2001 From: Alexander Reelsen Date: Tue, 2 Jan 2018 17:38:52 +0100 Subject: [PATCH 4/6] Tests: Increase logging to see requests This test case fails on non-reproducibly on CI, so this adds request/response logging to make sure all the requests were actually successful. Relates elastic/x-pack-elasticsearch#3340 Original commit: elastic/x-pack-elasticsearch@9e6c359f8c8c87a4ce5cbd634691d2e11f9e3afb --- .../elasticsearch/smoketest/MonitoringWithWatcherRestIT.java | 2 ++ 1 file changed, 2 insertions(+) diff --git a/qa/smoke-test-monitoring-with-watcher/src/test/java/org/elasticsearch/smoketest/MonitoringWithWatcherRestIT.java b/qa/smoke-test-monitoring-with-watcher/src/test/java/org/elasticsearch/smoketest/MonitoringWithWatcherRestIT.java index 2b1b15dc150..8a373e02dc9 100644 --- a/qa/smoke-test-monitoring-with-watcher/src/test/java/org/elasticsearch/smoketest/MonitoringWithWatcherRestIT.java +++ b/qa/smoke-test-monitoring-with-watcher/src/test/java/org/elasticsearch/smoketest/MonitoringWithWatcherRestIT.java @@ -9,6 +9,7 @@ import org.apache.http.entity.ContentType; import org.apache.http.entity.StringEntity; import org.elasticsearch.client.Response; import org.elasticsearch.common.xcontent.XContentType; +import org.elasticsearch.test.junit.annotations.TestLogging; import org.elasticsearch.test.rest.ESRestTestCase; import org.elasticsearch.test.rest.yaml.ObjectPath; import org.elasticsearch.xpack.watcher.actions.ActionBuilders; @@ -25,6 +26,7 @@ import static org.elasticsearch.xpack.watcher.input.InputBuilders.simpleInput; import static org.elasticsearch.xpack.watcher.trigger.schedule.IntervalSchedule.Interval.Unit.MINUTES; import static org.hamcrest.Matchers.is; +@TestLogging("org.elasticsearch.client:TRACE") public class MonitoringWithWatcherRestIT extends ESRestTestCase { @After From 2ad6a3a53820c9d841afbe665f8bc8a498a5bb42 Mon Sep 17 00:00:00 2001 From: Tim Vernum Date: Wed, 3 Jan 2018 20:08:33 +1000 Subject: [PATCH 5/6] Cleanup the handling for bootstrap passwords (elastic/x-pack-elasticsearch#3470) Minor refactoring on the reserved realm: - Removed some duplicated code - Added in some additional assertions - Extended some testing - Removed use of the obsolete "allow_default_passwords" from the test. Original commit: elastic/x-pack-elasticsearch@584171d2bdb8e663c1a1174e554c39343761226e --- .../authc/esnative/ReservedRealm.java | 29 +++-- .../authc/esnative/ReservedRealmTests.java | 110 +++++++++++------- 2 files changed, 82 insertions(+), 57 deletions(-) diff --git a/plugin/src/main/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealm.java b/plugin/src/main/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealm.java index a43ead0ec88..43dfa5d3837 100644 --- a/plugin/src/main/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealm.java +++ b/plugin/src/main/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealm.java @@ -5,6 +5,12 @@ */ package org.elasticsearch.xpack.security.authc.esnative; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + import org.apache.logging.log4j.message.ParameterizedMessage; import org.apache.logging.log4j.util.Supplier; import org.elasticsearch.Version; @@ -32,12 +38,6 @@ import org.elasticsearch.xpack.security.user.KibanaUser; import org.elasticsearch.xpack.security.user.LogstashSystemUser; import org.elasticsearch.xpack.security.user.User; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collection; -import java.util.Collections; -import java.util.List; - /** * A realm for predefined users. These users can only be modified in terms of changing their passwords; no other modifications are allowed. * This realm is always enabled. @@ -97,6 +97,7 @@ public class ReservedRealm extends CachingUsernamePasswordRealm { } } finally { assert userInfo.passwordHash != DISABLED_DEFAULT_USER_INFO.passwordHash : "default user info must be cloned"; + assert userInfo.passwordHash != ENABLED_DEFAULT_USER_INFO.passwordHash : "default user info must be cloned"; assert userInfo.passwordHash != bootstrapUserInfo.passwordHash : "bootstrap user info must be cloned"; Arrays.fill(userInfo.passwordHash, (char) 0); } @@ -184,15 +185,11 @@ public class ReservedRealm extends CachingUsernamePasswordRealm { logger.debug("Marking user [{}] as disabled because the security mapping is not at the required version", username); listener.onResponse(DISABLED_DEFAULT_USER_INFO.deepClone()); } else if (securityLifecycleService.isSecurityIndexExisting() == false) { - listener.onResponse(bootstrapUserInfo.deepClone()); + listener.onResponse(getDefaultUserInfo(username)); } else { nativeUsersStore.getReservedUserInfo(username, ActionListener.wrap((userInfo) -> { if (userInfo == null) { - if (ElasticUser.NAME.equals(username)) { - listener.onResponse(bootstrapUserInfo.deepClone()); - } else { - listener.onResponse(ENABLED_DEFAULT_USER_INFO.deepClone()); - } + listener.onResponse(getDefaultUserInfo(username)); } else { listener.onResponse(userInfo); } @@ -204,6 +201,14 @@ public class ReservedRealm extends CachingUsernamePasswordRealm { } } + private ReservedUserInfo getDefaultUserInfo(String username) { + if (ElasticUser.NAME.equals(username)) { + return bootstrapUserInfo.deepClone(); + } else { + return ENABLED_DEFAULT_USER_INFO.deepClone(); + } + } + private boolean userIsDefinedForCurrentSecurityMapping(String username) { final Version requiredVersion = getDefinedVersion(username); return securityLifecycleService.checkSecurityMappingVersion(requiredVersion::onOrBefore); diff --git a/plugin/src/test/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealmTests.java b/plugin/src/test/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealmTests.java index 894a71b5cbd..70fad13fac0 100644 --- a/plugin/src/test/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealmTests.java +++ b/plugin/src/test/java/org/elasticsearch/xpack/security/authc/esnative/ReservedRealmTests.java @@ -5,6 +5,13 @@ */ package org.elasticsearch.xpack.security.authc.esnative; +import java.util.Collection; +import java.util.Collections; +import java.util.Map; +import java.util.Map.Entry; +import java.util.concurrent.ExecutionException; +import java.util.function.Predicate; + import org.elasticsearch.ElasticsearchSecurityException; import org.elasticsearch.Version; import org.elasticsearch.action.ActionListener; @@ -26,16 +33,10 @@ import org.elasticsearch.xpack.security.user.ElasticUser; import org.elasticsearch.xpack.security.user.KibanaUser; import org.elasticsearch.xpack.security.user.LogstashSystemUser; import org.elasticsearch.xpack.security.user.User; +import org.elasticsearch.xpack.security.user.UsernamesField; import org.junit.Before; import org.mockito.ArgumentCaptor; -import java.util.Collection; -import java.util.Collections; -import java.util.Map; -import java.util.Map.Entry; -import java.util.concurrent.ExecutionException; -import java.util.function.Predicate; - import static org.hamcrest.Matchers.contains; import static org.hamcrest.Matchers.containsInAnyOrder; import static org.hamcrest.Matchers.containsString; @@ -58,7 +59,6 @@ import static org.mockito.Mockito.when; public class ReservedRealmTests extends ESTestCase { private static final SecureString EMPTY_PASSWORD = new SecureString("".toCharArray()); - public static final String ACCEPT_DEFAULT_PASSWORDS = ReservedRealm.ACCEPT_DEFAULT_PASSWORD_SETTING.getKey(); private NativeUsersStore usersStore; private SecurityLifecycleService securityLifecycleService; @@ -71,33 +71,16 @@ public class ReservedRealmTests extends ESTestCase { mockGetAllReservedUserInfo(usersStore, Collections.emptyMap()); } - public void testDisableDefaultPasswordAuthentication() throws Throwable { - final User expected = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true)); + public void testReservedUserEmptyPasswordAuthenticationFails() throws Throwable { + final String principal = randomFrom(UsernamesField.ELASTIC_NAME, UsernamesField.KIBANA_NAME, UsernamesField.LOGSTASH_NAME); - final Environment environment = mock(Environment.class); - final AnonymousUser anonymousUser = new AnonymousUser(Settings.EMPTY); - final Settings settings = Settings.builder().put(ACCEPT_DEFAULT_PASSWORDS, false).build(); - final ReservedRealm reservedRealm = new ReservedRealm(environment, settings, usersStore, anonymousUser, - securityLifecycleService, new ThreadContext(Settings.EMPTY)); - - PlainActionFuture listener = new PlainActionFuture<>(); - reservedRealm.doAuthenticate(new UsernamePasswordToken(expected.principal(), EMPTY_PASSWORD), listener); - assertFailedAuthentication(listener, expected.principal()); - } - - public void testElasticEmptyPasswordAuthenticationFails() throws Throwable { - final User expected = new ElasticUser(true); - final String principal = expected.principal(); - - Settings settings = Settings.builder().put(ACCEPT_DEFAULT_PASSWORDS, true).build(); - final ReservedRealm reservedRealm = - new ReservedRealm(mock(Environment.class), settings, usersStore, - new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); PlainActionFuture listener = new PlainActionFuture<>(); - reservedRealm.doAuthenticate(new UsernamePasswordToken(principal, EMPTY_PASSWORD), listener); - assertFailedAuthentication(listener, expected.principal()); + reservedRealm.doAuthenticate(new UsernamePasswordToken(principal, EMPTY_PASSWORD), listener); + assertFailedAuthentication(listener, principal); } public void testAuthenticationDisabled() throws Throwable { @@ -107,8 +90,8 @@ public class ReservedRealmTests extends ESTestCase { when(securityLifecycleService.isSecurityIndexExisting()).thenReturn(true); } final ReservedRealm reservedRealm = - new ReservedRealm(mock(Environment.class), settings, usersStore, - new AnonymousUser(settings), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + new ReservedRealm(mock(Environment.class), settings, usersStore, + new AnonymousUser(settings), securityLifecycleService, new ThreadContext(Settings.EMPTY)); final User expected = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true)); final String principal = expected.principal(); @@ -129,9 +112,8 @@ public class ReservedRealmTests extends ESTestCase { } private void verifySuccessfulAuthentication(boolean enabled) throws Exception { - final Settings settings = Settings.builder().put(ACCEPT_DEFAULT_PASSWORDS, randomBoolean()).build(); - final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), settings, usersStore, - new AnonymousUser(settings), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); final User expectedUser = randomFrom(new ElasticUser(enabled), new KibanaUser(enabled), new LogstashSystemUser(enabled)); final String principal = expectedUser.principal(); final SecureString newPassword = new SecureString("foobar".toCharArray()); @@ -171,8 +153,8 @@ public class ReservedRealmTests extends ESTestCase { public void testLookup() throws Exception { final ReservedRealm reservedRealm = - new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, - new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true)); final String principal = expectedUser.principal(); @@ -196,8 +178,8 @@ public class ReservedRealmTests extends ESTestCase { public void testLookupDisabled() throws Exception { Settings settings = Settings.builder().put(XPackSettings.RESERVED_REALM_ENABLED_SETTING.getKey(), false).build(); final ReservedRealm reservedRealm = - new ReservedRealm(mock(Environment.class), settings, usersStore, new AnonymousUser(settings), - securityLifecycleService, new ThreadContext(Settings.EMPTY)); + new ReservedRealm(mock(Environment.class), settings, usersStore, new AnonymousUser(settings), + securityLifecycleService, new ThreadContext(Settings.EMPTY)); final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true)); final String principal = expectedUser.principal(); @@ -210,8 +192,8 @@ public class ReservedRealmTests extends ESTestCase { public void testLookupThrows() throws Exception { final ReservedRealm reservedRealm = - new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, - new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); final User expectedUser = randomFrom(new ElasticUser(true), new KibanaUser(true), new LogstashSystemUser(true)); final String principal = expectedUser.principal(); when(securityLifecycleService.isSecurityIndexExisting()).thenReturn(true); @@ -258,7 +240,7 @@ public class ReservedRealmTests extends ESTestCase { public void testGetUsers() { final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, - new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); PlainActionFuture> userFuture = new PlainActionFuture<>(); reservedRealm.users(userFuture); assertThat(userFuture.actionGet(), containsInAnyOrder(new ElasticUser(true), new KibanaUser(true), @@ -273,7 +255,7 @@ public class ReservedRealmTests extends ESTestCase { .build(); final AnonymousUser anonymousUser = new AnonymousUser(settings); final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), settings, usersStore, anonymousUser, - securityLifecycleService, new ThreadContext(Settings.EMPTY)); + securityLifecycleService, new ThreadContext(Settings.EMPTY)); PlainActionFuture> userFuture = new PlainActionFuture<>(); reservedRealm.users(userFuture); if (anonymousEnabled) { @@ -290,7 +272,7 @@ public class ReservedRealmTests extends ESTestCase { ReservedUserInfo userInfo = new ReservedUserInfo(hash, true, false); mockGetAllReservedUserInfo(usersStore, Collections.singletonMap("elastic", userInfo)); final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), Settings.EMPTY, usersStore, - new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); if (randomBoolean()) { PlainActionFuture future = new PlainActionFuture<>(); @@ -379,6 +361,44 @@ public class ReservedRealmTests extends ESTestCase { assertThat(result.getStatus(), is(AuthenticationResult.Status.SUCCESS)); } + public void testNonElasticUsersCannotUseBootstrapPasswordWhenSecurityIndexExists() throws Exception { + final MockSecureSettings mockSecureSettings = new MockSecureSettings(); + final String password = randomAlphaOfLengthBetween(8, 24); + mockSecureSettings.setString("bootstrap.password", password); + Settings settings = Settings.builder().setSecureSettings(mockSecureSettings).build(); + when(securityLifecycleService.isSecurityIndexExisting()).thenReturn(true); + + final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), settings, usersStore, + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + PlainActionFuture listener = new PlainActionFuture<>(); + + final String principal = randomFrom(KibanaUser.NAME, LogstashSystemUser.NAME); + doAnswer((i) -> { + ActionListener callback = (ActionListener) i.getArguments()[1]; + callback.onResponse(null); + return null; + }).when(usersStore).getReservedUserInfo(eq(principal), any(ActionListener.class)); + reservedRealm.doAuthenticate(new UsernamePasswordToken(principal, mockSecureSettings.getString("bootstrap.password")), listener); + final AuthenticationResult result = listener.get(); + assertThat(result.getStatus(), is(AuthenticationResult.Status.TERMINATE)); + } + + public void testNonElasticUsersCannotUseBootstrapPasswordWhenSecurityIndexDoesNotExists() throws Exception { + final MockSecureSettings mockSecureSettings = new MockSecureSettings(); + final String password = randomAlphaOfLengthBetween(8, 24); + mockSecureSettings.setString("bootstrap.password", password); + Settings settings = Settings.builder().setSecureSettings(mockSecureSettings).build(); + when(securityLifecycleService.isSecurityIndexExisting()).thenReturn(false); + + final ReservedRealm reservedRealm = new ReservedRealm(mock(Environment.class), settings, usersStore, + new AnonymousUser(Settings.EMPTY), securityLifecycleService, new ThreadContext(Settings.EMPTY)); + PlainActionFuture listener = new PlainActionFuture<>(); + + final String principal = randomFrom(KibanaUser.NAME, LogstashSystemUser.NAME); + reservedRealm.doAuthenticate(new UsernamePasswordToken(principal, mockSecureSettings.getString("bootstrap.password")), listener); + final AuthenticationResult result = listener.get(); + assertThat(result.getStatus(), is(AuthenticationResult.Status.TERMINATE)); + } /* * NativeUserStore#getAllReservedUserInfo is pkg private we can't mock it otherwise From 1cae1f8e93deb1b0e1b1f6b579b7b8be5ca2f0b5 Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Wed, 3 Jan 2018 09:28:49 -0800 Subject: [PATCH 6/6] [DOCS] Added certutil command (elastic/x-pack-elasticsearch#3294) * [DOCS] Added certutil command * [DOCS] Added examples to certutil command * [DOCS] Address certutil feedback Original commit: elastic/x-pack-elasticsearch@a9df7bcda2f52de136492cbd26491c3698b2141c --- docs/en/commands/certgen.asciidoc | 4 +- docs/en/commands/certutil.asciidoc | 279 +++++++++++++++++++++++++++++ docs/en/commands/index.asciidoc | 2 + 3 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 docs/en/commands/certutil.asciidoc diff --git a/docs/en/commands/certgen.asciidoc b/docs/en/commands/certgen.asciidoc index f0aabee30f8..7b93ecdc090 100644 --- a/docs/en/commands/certgen.asciidoc +++ b/docs/en/commands/certgen.asciidoc @@ -2,6 +2,8 @@ [[certgen]] == certgen +deprecated[6.1,Replaced by <>. ] + The `certgen` command simplifies the creation of certificate authorities (CA), certificate signing requests (CSR), and signed certificates for use with the Elastic Stack. @@ -56,7 +58,7 @@ signed certificates must be in PEM format to work with {security}. using an existing CA certificate, which is provided in the `` argument. This parameter cannot be used with the `-csr` parameter. -`--csr`:: Specifies to operation in certificate signing request mode. +`--csr`:: Specifies to operate in certificate signing request mode. `--days `:: Specifies an integer value that represents the number of days the generated keys diff --git a/docs/en/commands/certutil.asciidoc b/docs/en/commands/certutil.asciidoc new file mode 100644 index 00000000000..605d4a2d9fe --- /dev/null +++ b/docs/en/commands/certutil.asciidoc @@ -0,0 +1,279 @@ +[role="xpack"] +[[certutil]] +== certutil + +The `certutil` command simplifies the creation of certificates for use with +Transport Layer Security (TLS) in the Elastic Stack. + +[float] +=== Synopsis + +[source,shell] +-------------------------------------------------- +bin/x-pack/certutil +( +(ca [--ca-dn ] [--days ] [--pem]) + +| (cert ([--ca ] | [--ca-cert --ca-key ]) +[--ca-dn ] [--ca-pass ] [--days ] +[--dns ] [--in ] [--ip ] +[--keep-ca-key] [--multiple] [--name ] [--pem]) + +| (csr [--dns ] [--in ] [--ip ] +[--name ]) + +[-E ] [--keysize ] [--out ] +[--pass ] +) +[-h, --help] ([-s, --silent] | [-v, --verbose]) +-------------------------------------------------- + +[float] +=== Description + +You can specify one of the following modes: `ca`, `cert`, `csr`. The `certutil` +command also supports a silent mode of operation to enable easier batch +operations. + +[float] +[[certutil-ca]] +==== CA mode + +The `ca` mode generates a new certificate authority (CA). By default, it +produces a single PKCS#12 output file, which holds the CA certificate and the +private key for the CA. If you specify the `--pem` parameter, the command +generates a zip file, which contains the certificate and private key in PEM +format. + +You can subsequently use these files as input for the `cert` mode of the command. + +[float] +[[certutil-cert]] +==== CERT mode + +The `cert` mode generates X.509 certificates and private keys. By default, it +produces a single certificate and key for use on a single instance. + +To generate certificates and keys for multiple instances, specify the +`--multiple` parameter, which prompts you for details about each instance. +Alternatively, you can use the `--in` parameter to specify a YAML file that +contains details about the instances. + +An instance is any piece of the Elastic Stack that requires a TLS or SSL +certificate. Depending on your configuration, {es}, Logstash, {kib}, and Beats +might all require a certificate and private key. The minimum required +information for an instance is its name, which is used as the common name for +the certificate. The instance name can be a hostname value or a full +distinguished name. If the instance name would result in an invalid file or +directory name, you must also specify a file name in the `--name` command +parameter or in the `filename` field in an input YAML file. + +You can optionally provide IP addresses or DNS names for each instance. If +neither IP addresses nor DNS names are specified, the Elastic stack products +cannot perform hostname verification and you might need to configure the +`verfication_mode` security setting to `certificate` only. For more information +about this setting, see <>. + +All certificates that are generated by this command are signed by a CA. You can +provide your own CA with the `--ca` or `--ca-cert` parameters. Otherwise, the +command automatically generates a new CA for you. For more information about +generating a CA, see the <>. + +By default, the `cert` mode produces a single PKCS#12 output file which holds +the instance certificate, the instance private key, and the CA certificate. If +you specify the `--pem` parameter, the command generates PEM formatted +certificates and keys and packages them into a zip file. Likewise if you chose +to generate output for multiple instances, the command produces a zip file. + +[float] +[[certutil-csr]] +==== CSR mode + +The `csr` mode generates certificate signing requests (CSRs) that you can send +to a trusted certificate authority to obtain signed certificates. The signed +certificates must be in PEM or PKCS#12 format to work with {security}. + +By default, the command produces a single CSR for a single instance. + +To generate CSRs for multiple instances, specify the `--multiple` parameter, +which prompts you for details about each instance. Alternatively, you can use +the `--in` parameter to specify a YAML file that contains details about the +instances. + +The `cert` mode produces a single zip file which contains the CSRs and the +private keys for each instance. Each CSR is provided as a standard PEM +encoding of a PKCS#10 CSR. Each key is provided as a PEM encoding of an RSA +private key. + +[float] +=== Parameters + +`ca`:: Specifies to generate a new local certificate authority (CA). This +parameter cannot be used with the `csr` or `cert` parameters. + +`cert`:: Specifies to generate new X.509 certificates and keys. +This parameter cannot be used with the `csr` or `ca` parameters. + +`csr`:: Specifies to generate certificate signing requests. This parameter +cannot be used with the `ca` or `cert` parameters. + +`--ca `:: Specifies the path to an existing CA key pair +(in PKCS#12 format). This parameter cannot be used with the `ca` or `csr` parameters. + +`--ca-cert `:: Specifies the path to an existing CA certificate (in +PEM format). You must also specify the `--ca-key` parameter. The `--ca-cert` +parameter cannot be used with the `ca` or `csr` parameters. + +`--ca-dn `:: Defines the _Distinguished Name_ (DN) that is used for the +generated CA certificate. The default value is +`CN=Elastic Certificate Tool Autogenerated CA`. This parameter cannot be used +with the `csr` parameter. + +`--ca-key `:: Specifies the path to an existing CA private key (in +PEM format). You must also specify the `--ca-cert` parameter. The `--ca-key` +parameter cannot be used with the `ca` or `csr` parameters. + +`--ca-pass `:: Specifies the password for an existing CA private key +or the generated CA private key. This parameter cannot be used with the `ca` or +`csr` parameters. + +`--days `:: Specifies an integer value that represents the number of days the +generated certificates are valid. The default value is `1095`. This parameter +cannot be used with the `csr` parameter. + +`--dns `:: Specifies a comma-separated list of DNS names. This +parameter cannot be used with the `ca` parameter. + +`-E `:: Configures a setting. + +`-h, --help`:: Returns all of the command parameters. + +`--in `:: Specifies the file that is used to run in silent mode. The +input file must be a YAML file. This parameter cannot be used with the `ca` +parameter. + +`--ip `:: Specifies a comma-separated list of IP addresses. This +parameter cannot be used with the `ca` parameter. + +`--keep-ca-key`:: When running in `cert` mode with an automatically-generated +CA, specifies to retain the CA private key for future use. + +`--keysize `:: +Defines the number of bits that are used in generated RSA keys. The default +value is `2048`. + +`--multiple`:: +Specifies to generate files for multiple instances. This parameter cannot be +used with the `ca` parameter. + +`--name `:: +Specifies the name of the generated certificate. This parameter cannot be used +with the `ca` parameter. + +`--out `:: Specifies a path for the output files. + +`--pass `:: Specifies the password for the generated private keys. + +`--pem`:: Generates certificates and keys in PEM format instead of PKCS#12. This +parameter cannot be used with the `csr` parameter. + +`-s, --silent`:: Shows minimal output. + +`-v, --verbose`:: Shows verbose output. + +[float] +=== Examples + +The following command generates a CA certificate and private key in PKCS#12 +format: + +[source, sh] +-------------------------------------------------- +bin/x-pack/certutil ca +-------------------------------------------------- + +You are prompted for an output filename and a password. Alternatively, you can +specify the `--out` and `--pass` parameters. + +You can then generate X.509 certificates and private keys by using the new +CA. For example: + +[source, sh] +-------------------------------------------------- +bin/x-pack/certutil cert --ca elastic-stack-ca.p12 +-------------------------------------------------- + +You are prompted for the CA password and for an output filename and password. +Alternatively, you can specify the `--ca-pass`, `--out`, and `--pass` parameters. + +By default, this command generates a file called `elastic-certificates.p12`, +which you can copy to the relevant configuration directory for each Elastic +product that you want to configure. For more information, see +{xpack-ref}/ssl-tls.html[Setting Up TLS on a Cluster]. + +[float] +[[certutil-silent]] +==== Using `certutil` in Silent Mode + +To use the silent mode of operation, you must create a YAML file that contains +information about the instances. It must match the following format: + +[source, yaml] +-------------------------------------------------- +instances: + - name: "node1" <1> + ip: <2> + - "192.0.2.1" + dns: <3> + - "node1.mydomain.com" + - name: "node2" + ip: + - "192.0.2.2" + - "198.51.100.1" + - name: "node3" + - name: "node4" + dns: + - "node4.mydomain.com" + - "node4.internal" + - name: "CN=node5,OU=IT,DC=mydomain,DC=com" + filename: "node5" <4> +-------------------------------------------------- +<1> The name of the instance. This can be a simple string value or can be a +Distinguished Name (DN). This is the only required field. +<2> An optional array of strings that represent IP Addresses for this instance. +Both IPv4 and IPv6 values are allowed. The values are added as Subject +Alternative Names. +<3> An optional array of strings that represent DNS names for this instance. +The values are added as Subject Alternative Names. +<4> The filename to use for this instance. This name is used as the name of the +directory that contains the instance's files in the output. It is also used in +the names of the files within the directory. This filename should not have an +extension. Note: If the `name` provided for the instance does not represent a +valid filename, then the `filename` field must be present. + +When your YAML file is ready, you can use the `certutil` command to generate +certificates or certificate signing requests. Simply use the `--in` parameter to +specify the location of the file. For example: + +[source, sh] +-------------------------------------------------- +bin/x-pack/certutil cert --silent --in instances.yml --out test1.zip --pass testpassword +-------------------------------------------------- + +This command generates a compressed `test1.zip` file. After you decompress the +output file, there is a directory for each instance that was listed in the +`instances.yml` file. Each instance directory contains a single PKCS#12 (`.p12`) +file, which contains the instance certificate, instance private key, and CA +certificate. + +You an also use the YAML file to generate certificate signing requests. For +example: + +[source, sh] +-------------------------------------------------- +bin/x-pack/certutil csr --silent --in instances.yml --out test2.zip --pass testpassword +-------------------------------------------------- + +This command generates a compressed file, which contains a directory for each +instance. Each instance directory contains a certificate signing request +(`*.csr` file) and private key (`*.key` file). diff --git a/docs/en/commands/index.asciidoc b/docs/en/commands/index.asciidoc index 2bdc5ec6645..e839a54935f 100644 --- a/docs/en/commands/index.asciidoc +++ b/docs/en/commands/index.asciidoc @@ -8,6 +8,7 @@ {xpack} includes commands that help you configure security: * <> +* <> * <> * <> * <> @@ -16,6 +17,7 @@ -- include::certgen.asciidoc[] +include::certutil.asciidoc[] include::migrate-tool.asciidoc[] include::setup-passwords.asciidoc[] include::syskeygen.asciidoc[]