6198 Commits

Author SHA1 Message Date
Costin Leau
04c9427596 Pass jdbc fetch size to the scroller
Original commit: elastic/x-pack-elasticsearch@2309a43980
2017-07-21 00:11:01 +03:00
Costin Leau
76b429bfe2 Introduce SqlSettings for handling per-client configuration
To avoid leaking client information across the entire code-base, client
settings like TimeZone or pagination are stored in
SqlSession>SqlSettings which are available as a ThreadLocal (during
analysis) so that components that need them, can pick them up.

Since ES internally uses Joda, the date/time functionality relies on Joda,
whenever possible to match the behavior.

Original commit: elastic/x-pack-elasticsearch@20f41e2bb3
2017-07-20 19:28:04 +03:00
Costin Leau
8acacc4f7d Add random fetchSize
Original commit: elastic/x-pack-elasticsearch@1f8197b04f
2017-07-18 23:15:40 +03:00
Costin Leau
d010f181bc Update packages
Original commit: elastic/x-pack-elasticsearch@76c5ff3286
2017-07-18 18:52:46 +03:00
Costin Leau
448b5d8faf Merge remote-tracking branch 'remotes/upstream/master' into feature/sql
Original commit: elastic/x-pack-elasticsearch@154da6e26e
2017-07-18 18:43:31 +03:00
Costin Leau
f33ae72712 Minor polishing
Make JDBC and CLI consistent with the rest of the plugin
Replace ResultPage with an interface

Original commit: elastic/x-pack-elasticsearch@c62249cc2e
2017-07-18 18:41:23 +03:00
Adrien Grand
44c9bba39c Remove randomDynamicTemplates usage.
It has been removed it core.

Original commit: elastic/x-pack-elasticsearch@ba9fd16ed6
2017-07-17 16:55:39 +02:00
Tim Brooks
1abc40c645 Rename elastic bootstrap password setting (elastic/x-pack-elasticsearch#2009)
This is related to elastic/x-pack-elasticsearch#1991.

Original commit: elastic/x-pack-elasticsearch@b265211e4a
2017-07-14 16:41:42 -05:00
jaymode
1a6e6411cf Test: use the same settings object to construct tribe settings
This changes the SecurityTribeIT tests to use the same settings object when creating the settings
for the tribe node. Previously two different objects were being created and we would read regular
settings from one and secure settings from another. This causes problems since randomization means
that there may be settings added on the first call that do not get added on the second call. One
example is the randomization of when to only use a keystore or to use both a keystore and a
truststore. On the first call, we would add settings for both a keystore and a truststore but on
the second call only the keystore settings were added. This lead to failures as we would not be
able to open a password protected truststore since the password was never added to the secure
settings.

relates elastic/x-pack-elasticsearch#2005

Original commit: elastic/x-pack-elasticsearch@bbdb3ec662
2017-07-14 14:12:32 -06:00
Drew Raines
49407c1069 [DOCS] Fix typo
Original commit: elastic/x-pack-elasticsearch@1aaaa88404
2017-07-14 14:31:53 -05:00
David Roberts
8365038584 [TEST] Mute failing test: SSLTrustRestrictionsTests testRestrictionsAreReloaded
See elastic/x-pack-elasticsearch#2007

Original commit: elastic/x-pack-elasticsearch@1deb4a7162
2017-07-14 16:30:47 +01:00
Jay Modi
6b4468ea5c Clear security caches on security index health changes (elastic/x-pack-elasticsearch#1957)
This change clears the caches in the native realm and the composite roles store when there is a
a change in the health of the security index that necessitates this. When the security index goes
to a red state, the caches are left in tact as this allows for management operations to be
performed for a limited amount of time. When the index transitions out of the red state or exists
when it didn't exist before, the caches will be cleared so that we remove any stale values.

relates elastic/x-pack-elasticsearch#1789

Original commit: elastic/x-pack-elasticsearch@914959ea6b
2017-07-14 09:28:28 -06:00
Tim Brooks
8ab167cccb Fix ReservedRealm test for failed authentication
Original commit: elastic/x-pack-elasticsearch@5759d9268c
2017-07-14 09:28:44 -05:00
Yannick Welsch
dbbec0d37e Let primary own its replication group
Companion commit for elastic/x-pack-elasticsearch#25692

Original commit: elastic/x-pack-elasticsearch@ed93c56f07
2017-07-14 13:51:43 +02:00
David Roberts
81ec1a7ba5 [TEST] Mute failing test
See elastic/x-pack-elasticsearch#2003

Original commit: elastic/x-pack-elasticsearch@5f3611b1d3
2017-07-14 09:49:30 +01:00
Tim Vernum
1686add7ce The configured role-mapping file must be valid. (elastic/x-pack-elasticsearch#1940)
This adds a bootstrap-check that makes it an error to configure a role mapping file that doesn't exist or cannot be parsed.

We are still lenient on dynamic reload because
(a) killing a running node is quite drastic
(b) file writes aren't atomic, so we might be picking up a file that is half way through being written (etc).

If you rely on the default role mapping filename, then it doesn't need to exist (because you might be using the role mapping API instead) but if it does exist it has to parse successfully

Original commit: elastic/x-pack-elasticsearch@5424dea4c4
2017-07-14 15:04:26 +10:00
Tim Vernum
d57e38fbed [LDAP] Support explicit "dn" attribute in group search (elastic/x-pack-elasticsearch#1995)
The default for group_search.attribute is to search by DN, but explicitly setting that to dn
wouldn't work because the DN is returned in a special value in the result, and not in the attributes list.

This change detects when user_attribute is set to dn and treats it the same way as the default value.

Original commit: elastic/x-pack-elasticsearch@1933410a0b
2017-07-14 13:12:28 +10:00
Tim Vernum
44a104cb26 [TESTS] Fix SSLTrustRestriction IntegrationTest
- Changes the reloading test to always trust the "trusted" cert so that the health-ping works
- Adds some more logging in case we get new failures

Original commit: elastic/x-pack-elasticsearch@993bf9c721
2017-07-14 13:10:29 +10:00
Tim Brooks
6d04eacdec Require elastic password be bootstrapped (elastic/x-pack-elasticsearch#1962)
This is related to elastic/x-pack-elasticsearch#1217. This commit requires that the elastic password
be bootstrapped for the user to be authenticated. As a result it removes
the special "setup" mode that allowed the user to be authenticated from
localhost.

Additionally, this commit updates the tests to work with this
functionality.

Original commit: elastic/x-pack-elasticsearch@d0d5d697a7
2017-07-13 19:59:50 -05:00
Igor Motov
4de6d9ebe5 Upgrade API: upgrade assistance shouldn't throw 404 on an empty cluster (elastic/x-pack-elasticsearch#1997)
When a user asks for upgrade information for all indices and there are no indices in the cluster, upgrade assistance should just return an empty response indicating that no indices require upgrade or reindexing. This commit also reverts the temporary fix in WatchBackwardsCompatibilityIT tests that was added as a workaround for this issue.

Original commit: elastic/x-pack-elasticsearch@2ea9707867
2017-07-13 17:01:50 -04:00
Igor Motov
dd11fc3d0a Upgrade API: fix parent task propagation for upgrade (elastic/x-pack-elasticsearch#1986)
Ensures that parent task is propagated to child operations to ensure that reindex operation can be cancelled if needed.

Original commit: elastic/x-pack-elasticsearch@fa40b5a951
2017-07-13 16:25:38 -04:00
jaymode
20c06578f6 Do not add domain to the bind user when it is a DN
This change fixes the creation of the bind DN string for active directory realms so that they do
not add the `@domain` suffix to the bind DN, when it is a actual DN value.

Original commit: elastic/x-pack-elasticsearch@bd04c07e16
2017-07-13 12:15:08 -06:00
Chris Earle
f4b9dff71a [Monitoring] Support new Kibana mappings for Detecting Cloud (elastic/x-pack-elasticsearch#1989)
This adds the Cloud detection mappings so that phone home can take advantage of their existence.

It also sets the system load fields to use `half_floats`.

Original commit: elastic/x-pack-elasticsearch@75f7992d38
2017-07-13 13:59:40 -04:00
Chris Earle
924ff446bf [Monitoring] Stop checking for old alerts in upgraded version (elastic/x-pack-elasticsearch#1992)
This stops checking for older alerts now that we support emailing based on state changes. This only applies to 6.0 because the upgrade _can_ still happen usefully in 5.6 without the noise.

Original commit: elastic/x-pack-elasticsearch@9d73c64daa
2017-07-13 13:24:32 -04:00
Jay Modi
6fdad6039f Allow the Active Directory UPN authenticator to work with suffixes (elastic/x-pack-elasticsearch#1958)
The active directory user principal name format typically takes the form user@domain, which is what
the current implementation expects. However, active directory also allows the definition of other
suffixes that are not actual domains. A user can still authenticate using this user principal name
but the behavior of our realm would cause it to fail as it parsed the suffix as a domain and used it
as the search base for the user. Instead, we should use the default user search base and only look
for entries that have this exact user principal name. In a scenario where a realm is configured for
multiple domains in the same forest, the search base should be the base for the entire forest.

relates elastic/x-pack-elasticsearch#1744

Original commit: elastic/x-pack-elasticsearch@de00c4817e
2017-07-13 10:08:22 -06:00
Tim Vernum
c753ddf7f2 Enable TLS trust restrictions by subject alternative name (elastic/x-pack-elasticsearch#1919)
Optional restrictions can be applied on top of an existing certificate trust scheme (PEM CAs, JKS TrustStore etc).
The restrictions reduce the set of certificate that would be otherwise trusted.

In this commit, the only supported restrictions are to filter by the certificate's SubjectAlternativeNames that are tagged as "other name" with an object-identifier of "cn"

Original commit: elastic/x-pack-elasticsearch@c6105a47df
2017-07-13 23:45:00 +10:00
Alexander Reelsen
eb118b365c Tests: Ignore 404 errors in watcher bwc tests
Original commit: elastic/x-pack-elasticsearch@8e583cf293
2017-07-13 13:41:51 +02:00
Colin Goodheart-Smithe
8aec1d4737 [DOCS] Remove reference to field stats in security limitations
Original commit: elastic/x-pack-elasticsearch@9ca673ea36
2017-07-13 12:00:16 +01:00
Colin Goodheart-Smithe
6abe6e5b54 Changes to support removal of Field stats API (elastic/x-pack-elasticsearch#1953)
* Changes to support removal of Field stats API

* iter

Original commit: elastic/x-pack-elasticsearch@bac97b8701
2017-07-13 11:56:43 +01:00
Alexander Reelsen
32bc0cd5d5 Tests: Add logging and error tracing to watcher bwc tests
Original commit: elastic/x-pack-elasticsearch@4a4b65bcb7
2017-07-13 11:08:48 +02:00
Luca Cavanna
56cfaf8cb9 Rename client artifacts (elastic/x-pack-elasticsearch#1985)
Adapt to upstream rename of rest and sniffer artifacts

Original commit: elastic/x-pack-elasticsearch@f43368b3fb
2017-07-13 09:44:53 +02:00
Ryan Ernst
df41b8342d Convert put_template uses to put_script in rest tests
Original commit: elastic/x-pack-elasticsearch@5f5f52b955
2017-07-13 00:01:56 -07:00
Tim Vernum
e4c8851a24 Convert Realm.authenticate to provide a richer result (elastic/x-pack-elasticsearch#1932)
This allows for messages to be returned, and distinguishes between 4 different results:
- I have authenticated the user
- I don't know how to authenticate that user. Try another realm.
- I tried to authenticate the user, but failed. Try another realm.
- I tried to authenticate the user, but failed. Fail the authentication attempt.

Original commit: elastic/x-pack-elasticsearch@f796949cfb
2017-07-13 14:24:08 +10:00
Tim Vernum
a36121a725 [DOCS] [Security] Templates do not use bind_dn (elastic/x-pack-elasticsearch#1979)
Document that user_dn_template mode for LDAP authentication does not support bind_dn

Original commit: elastic/x-pack-elasticsearch@eef72615a8
2017-07-13 14:23:23 +10:00
Nik Everett
cf29dea577 Huge protocol refactor
Too big. Sorry. Some good things though:
1. Share some code between CLI and JDBC. Probably a good thing
at this point, better as we go on, I think.
2. Add round trip tests for all of proto.
3. Remove the `data` member from `QueryInitResponse` and
`QueryPageResponse` so we response serialization is consistent with
everything else.

Original commit: elastic/x-pack-elasticsearch@c6940a32ed
2017-07-12 17:32:13 -04:00
Simon Willnauer
84ee21ed26 Followup for elastic/elasticsearch#25658 (elastic/x-pack-elasticsearch#1984)
This is the xpack side fo elastic/elasticsearch#25658 which is mainly refactorings
of a ctor and added tests.

Original commit: elastic/x-pack-elasticsearch@d8e2a2a057
2017-07-12 22:19:34 +02:00
Jay Modi
e686d8a3bf Add active directory bind user and user lookup support (elastic/x-pack-elasticsearch#1956)
This commit adds support for a bind user when using the active directory realm. The addition of a
bind user also enables support for the user lookup mechanism, which is necessary to support the run
as functionality that we provide.

relates elastic/x-pack-elasticsearch#179

Original commit: elastic/x-pack-elasticsearch@40b07b3422
2017-07-12 14:01:39 -06:00
Brandon Kobel
ef25568b2a Removing indices privileges from the reporting_user (elastic/x-pack-elasticsearch#1806)
* Removing indices privileges from the reporting_user

* Updating tests

* Adding comment

Original commit: elastic/x-pack-elasticsearch@ba7688b1dd
2017-07-12 15:56:35 -04:00
Alexander Reelsen
e64cf23b13 Watcher: Start watcher on master node only with mixed versions (elastic/x-pack-elasticsearch#1983)
When there are data or master nodes in the cluster, that are older
than ES 6.0 alpha3, then watcher will only start on the master node.

Changed all transport actions to be master node actions, as there is
already a method to decide to run locally, which we can piggyback on.

Original commit: elastic/x-pack-elasticsearch@65cecb6d69
2017-07-12 20:58:47 +02:00
Tim Sullivan
51adbf0f51 [Monitoring] Update template version to 6.0.0 beta1 version (elastic/x-pack-elasticsearch#1948)
* [Monitoring] Update template version to 6000003

* update LAST_UPDATED_VERSION

* use the 6.0.0-beta1 version constant

* use the 6.0.0-beta1 version value (6000026)

* fix cluster uuid reference in terms filters

Original commit: elastic/x-pack-elasticsearch@40c822d396
2017-07-12 09:19:54 -07:00
David Roberts
176786de54 [ML] Improve mapping clash error message (elastic/x-pack-elasticsearch#1968)
relates elastic/x-pack-elasticsearch#1751

Original commit: elastic/x-pack-elasticsearch@01c221bf42
2017-07-12 15:58:37 +01:00
Jack Conradson
32dbfba0c2 Disallow lang to specified in requests where a stored script is used. (elastic/x-pack-elasticsearch#1949)
Requests that execute a stored script will no longer be allowed to specify the lang of the script. This information is stored in the cluster state making only an id necessary to execute against. Putting a stored script will still require a lang.

Original commit: elastic/x-pack-elasticsearch@926a7b2d86
2017-07-12 07:56:08 -07:00
Igor Motov
41f30b0ae4 Upgrade API: add common upgrade check method (elastic/x-pack-elasticsearch#1970)
Moves internal index format check to a central location.

Original commit: elastic/x-pack-elasticsearch@0869baac51
2017-07-12 08:46:59 -04:00
Hendrik Muhs
f5cfb98ea2 do not update mapping if it is newer than the current version (elastic/x-pack-elasticsearch#1971)
In case of mixed version in a cluster the update could potentially downgrade the mappings. This CL changes the check to avoid this situation.

Original commit: elastic/x-pack-elasticsearch@9ecd5df70e
2017-07-12 09:19:06 +02:00
Chris Earle
753690bf5c [TEST][Monitoring] Use specific LAST_UPDATED constant to avoid issues when they're not the same (elastic/x-pack-elasticsearch#1976)
Original commit: elastic/x-pack-elasticsearch@a41bff564a
2017-07-11 20:47:36 -04:00
Chris Earle
44eda0962e [Monitoring] Use ctx.metadata.xpack not just ctx.metadata (elastic/x-pack-elasticsearch#1974)
This tweaks a recent fix so that it uses the right metadata variable.

Original commit: elastic/x-pack-elasticsearch@af9d9b0e41
2017-07-11 18:04:38 -04:00
Jason Tedor
5ed683538d Name the log tail thread and mark it as a daemon
This commit provides an explicit name for the log tail thread, otherwise
the thread shows up with a generic name like Thread-12 in stack
dumps. While the stack trace provides enough information to conclude
what this thread is doing, it is better to be more explicit about the
purpose of this thread so that that can be discerned directly from the
name of the thread.

Additionally, we mark this thread as a daemon thread. Since this thread
is created by the main thread during node initialization it defaults to
being a user thread. Since the JVM only exits when the only threads
running are not user threads, if this thread were to somehow block then
it could prevent the server JVM from exiting. As such, this thread
should be marked as a daemon thread.

Relates elastic/x-pack-elasticsearch#1969

Original commit: elastic/x-pack-elasticsearch@b3761c7d91
2017-07-11 16:05:07 -04:00
Tim Brooks
c22f618a1a Modify password tool to pull password from keystore (elastic/x-pack-elasticsearch#1951)
This is related to elastic/x-pack-elasticsearch#1217. This commit modifies the password setup tool
to pull the password for the `elastic` user from the keystore.

Original commit: elastic/x-pack-elasticsearch@fbc71f393d
2017-07-11 12:50:53 -05:00
Igor Motov
9030c3ae77 Upgrade API: remove kibana upgraders (elastic/x-pack-elasticsearch#1963)
Also removes handling of extra parameters in the upgrade api

Original commit: elastic/x-pack-elasticsearch@9edb825da7
2017-07-11 11:29:31 -04:00
Jay Modi
03ed2bbbd0 Add setting for the LDAP user search filter and deprecate user attribute (elastic/x-pack-elasticsearch#1959)
This commit adds a setting to allow changing the user search filter. Previously the filter was a
simple equality filter that mapped a given attribute to the value of the username. The default
behavior remains the same with this change but provides additional flexibility to users to who may
need more advanced LDAP searches. The user attribute setting has been deprecated due to the overlap
with the new filter setting.

relates elastic/x-pack-elasticsearch#1861

Original commit: elastic/x-pack-elasticsearch@e9d797e81c
2017-07-11 09:27:24 -06:00