Commit Graph

43106 Commits

Author SHA1 Message Date
Ioannis Kakavas 89c515175b Update version for authenticate API responses
This commits changes the serialization version from V_7_0_0 to
v_6_6_0 for the authenticate API response now that the work to add
the realm info in the response has been backported to 6.x in
b515ec7c9b9074dfa2f5fd28bac68fd8a482209e

Relates #35648
2018-11-29 13:31:52 +02:00
Christoph Büscher c412ecd4a2 Muting ClusterClientITT#testClusterHealthYellowClusterLevel 2018-11-29 11:43:52 +01:00
Alan Woodward a646f85a99
Ensure TokenFilters only produce single tokens when parsing synonyms (#34331)
A number of tokenfilters can produce multiple tokens at the same position.  This
is a problem when using token chains to parse synonym files, as the SynonymMap
requires that there are no stacked tokens in its input.

This commit ensures that when used to parse synonyms, these tokenfilters either produce
a single version of their input token, or that they throw an error when mappings are 
generated.  In indexes created in elasticsearch 6.x deprecation warnings are emitted in place 
of the error. 

* asciifolding and cjk_bigram produce only the folded or bigrammed token
* decompounders, synonyms and keyword_repeat are skipped
* n-grams, word-delimiter-filter, multiplexer, fingerprint and phonetic throw errors

Fixes #34298
2018-11-29 10:35:38 +00:00
Alpar Torok c63d0af913 Prevent random build failures
Looks like some odd race condition causes failed builds by attempting to
run the task that should be disabled.
Disable the task explicitly untill we figure it out.
2018-11-29 11:30:09 +02:00
Martijn van Groningen 1390f366d4
[CCR] Only auto follow indices when all primary shards have started (#35814)
This change adds an extra check that verifies that all primary shards
have been started of an index that is about to be auto followed.

If not all primary shards have been started for an index
then the next auto follow run will try to follow to auto follow
this index again.

Closes #35480
2018-11-29 09:46:09 +01:00
Ryan Ernst fbdfec4305
Docs: Add note about oss repositories for deb/rpm (#35973)
This commit adds a note about configring the yum/apt repositories for
oss only packages.

closes #35960
2018-11-29 00:22:10 -08:00
Tanguy Leroux 0967620641
ActiveShardCount should not fail when closing the index (#35936)
The ActiveShardCount is used by cluster state observers to wait for a 
given number of shards to be active before returning to the caller. The 
current implementation does not work when an index is closed while an 
observer is waiting on shards to be active. In this case, a NPE is thrown 
and the observer is never notified that the shards won't become active.

This commit fixes the ActiveShardCount.enoughShardsActive() so that it 
does not fail when an index is closed, similarly to what is done when an 
index is deleted.
2018-11-29 09:08:30 +01:00
Alpar Torok 45db829039
Replace fixtures with docker-compose (#35651)
Creates a new plugin to manage docker-compose based test fixtures. 
Convert the smb-fixture as a first example.
2018-11-29 09:43:16 +02:00
Tim Vernum 609f742e5f
Fix kerberos setting registration (#35986)
In #30241 Realm settings were changed, but the Kerberos realm settings
were not registered correctly. This change fixes the registration of
those Kerberos settings.

Also adds a new integration test that ensures every internal realm can
be configured in a test cluster.

Also fixes the QA test for kerberos.

Resolves: #35942
2018-11-29 18:06:52 +11:00
Martijn van Groningen 6d01170ca1
[HLRC] Added support for CCR Delete Auto Follow Pattern API (#35981)
This change also adds documentation for the Delete Auto Follow Pattern API.

Relates to #33824
2018-11-29 08:06:31 +01:00
Ignacio Vera 93ed8b7d61
HLRC: Add delete user action (#35294)
* HLRC: Add delete user action

It adds delete user action to the high level rest client.

Relates #29827
2018-11-29 07:52:56 +01:00
Alpar Torok e0a678f0c4
Remove version.qualified from MainResponse (#35412)
The fully qualified version will be returned as `version.number`
2018-11-29 08:41:39 +02:00
Ryan Ernst afd42df15f
Core: Deguice RepositoriesService (#36016)
This commit moves the RepositoriesService to be created outside of
guice.
2018-11-28 21:20:44 -08:00
Jason Tedor 465233787e
Move creation of temporary directory to Java (#36002)
In the long run we want to move all of startup to a Java program. This
will simplify our startup scripts and make maintenance of startup less
dependent on the underlying platform that we run on. This commit moves
the creation of the temporary directory off of system-dependent commands
and onto a simple Java program.
2018-11-28 16:55:59 -05:00
ik 0916a1fb64 LLREST: Add PreferHasAttributeNodeSelector (#36005)
`PreferHasAttributeNodeSelector` works like exactly like
`HasAttributeNodeSelector` but if not nodes match the attribute
then it will not filter the list of nodes.
2018-11-28 16:29:05 -05:00
Lisa Cawley 906421cdad
[DOCS] Adds higher-level monitoring configuration page (#35926) 2018-11-28 12:36:29 -08:00
Gordon Brown c26af3b0a2
Deprecate X-Pack centric Migration endpoints (#35976)
This commit is part of our plan to deprecate and remove the use of
_xpack in the REST API routes.
2018-11-28 13:19:33 -07:00
Jim Ferenczi 9ca3a06475
Remove custom QueryBuilder#analyzeGraphPhrase (#35983)
Now that https://issues.apache.org/jira/browse/LUCENE-8479 is fixed
we can remove the custom implementation of QueryBuilder#analyzeGraphPhrase
in the match QueryBuilder.
2018-11-28 20:15:27 +01:00
Luca Cavanna 4b85769d24
Increase InternalHistogramTests coverage (#36004)
In `InternalHistogramTests` we were randomizing different values but `minDocCount` was hardcoded to `1`. It's important to test other values, especially `0` as it's the default. To make this possible, the test needed some adapting in the way buckets are randomly generated: all aggs need to share the same `interval`, `minDocCount` and `emptyBucketInfo`. Also assertions need to take into account that more (or less) buckets are expected depending on `minDocCount`.

This was originated by #35921 and its need to test adding empty buckets as part of the reduce phase.

Also relates to #26856 as one more key comparison needed to use `Double.compare` to properly handle `NaN` values, which was triggered by the increased test coverage.
2018-11-28 20:06:40 +01:00
Benjamin Trent 513e1ed095
ML: Adding XContentObjectTransformer class (#35957)
* ML: Adding XContentObjectTransformer class

* adding license headers

* Adding custom deprecation handler, and test for checking parsing failures

* forwarding deprection logs to LoggingDeprecationHandler
2018-11-28 11:43:57 -06:00
Lisa Cawley a72430e1ef
[DOCS] Adds TLS warning to rolling upgrades (#35841) 2018-11-28 09:38:58 -08:00
Simon Willnauer daef498cd9 [DOCS] Fix HLRC reference to unfreeze_index 2018-11-28 16:55:55 +01:00
Simon Willnauer 89e4ac8fcc
Add high-level REST client API for `_freeze` and `_unfreeze` (#35723)
This change adds support for `_freeze` and `_unfreeze` to the HLRC

Relates to #34352
2018-11-28 15:42:12 +01:00
Nik Everett 0588dad80b
Tasks: Only require task permissions (#35667)
Right now using the `GET /_tasks/<taskid>` API and causing a task to opt
in to saving its result after being completed requires permissions on
the `.tasks` index. When we built this we thought that that was fine,
but we've since moved towards not leaking details like "persisting task
results after the task is completed is done by saving them into an index
named `.tasks`." A more modern way of doing this would be to save the
tasks into the index "under the hood" and to have APIs to manage the
saved tasks. This is the first step down that road: it drops the
requirement to have permissions to interact with the `.tasks` index when
fetching task statuses and when persisting statuses beyond the lifetime
of the task.

In particular, this moves the concept of the "origin" of an action into
a more prominent place in the Elasticsearch server. The origin of an
action is ignored by the server, but the security plugin uses the origin
to make requests on behalf of a user in such a way that the user need
not have permissions to perform these actions. It *can* be made to be
fairly precise. More specifically, we can create an internal user just
for the tasks API that just has permission to interact with the `.tasks`
index. This change doesn't do that, instead, it uses the ubiquitus
"xpack" user which has most permissions because it is simpler. Adding
the tasks user is something I'd like to get to in a follow up change.

Instead, the majority of this change is about moving the "origin"
concept from the security portion of x-pack into the server. This should
allow any code to use the origin. To keep the change managable I've also
opted to deprecate rather than remove the "origin" helpers in the
security code. Removing them is almost entirely mechanical and I'd like
to that in a follow up as well.

Relates to #35573
2018-11-28 09:28:27 -05:00
Christoph Büscher 51a7dc54ec
Fix custom AUTO issue with Fuzziness#toXContent (#35807)
Currently when a Fuzziness instance with custom AUTO distance values gets
written to XContent, the customized lower and upper distance values are ommited
and can consequently not be parsed back. This changes this to write the String
including the optional custom values when writing to XContent and fixes the
tests that should have caught this in the first place, e.g. by adding the custom
low and high distance values to the equality check.
2018-11-28 15:07:11 +01:00
Andrey Ershov 0b45fb98b9
[Zen2] Generate coordinationMetaData with different configs (#35991)
This PR fixes test failure, which is caused by equal randomly generated lastAcceptedConfiguration and lastCommittedConfguration.
2018-11-28 14:49:07 +01:00
Marios Trivyzas a2338bb116
SQL: Fix issue with wrong data type for scripted Grouping keys (#35969)
When the grouping key of a GROUP BY is a painless script (functions are
involved), the data type of the key was incorrect in certain cases
(Boolean, IP, Date). This resulted in returning wrong data type for this
columns in the query results. E.g.:

```
SELECT COUNT(*), a > 10 AS a FROM t GROUP BY a
```

Fixes: #35662
2018-11-28 14:29:10 +01:00
Jason Tedor a3186e4a32
Deprecate X-Pack centric license endpoints (#35959)
This commit is part of our plan to deprecate and ultimately remove the
use of _xpack in the REST APIs.
2018-11-28 08:24:35 -05:00
Jason Tedor 78ac12d952
Remove deprecated Graph endpoints (#35956)
We had some endpoints in Graph that are deprecated for removal in
7.0.0. This commit removes these deprecated endpoints.
2018-11-28 08:24:15 -05:00
Andrei Stefan aabff7318e
SQL: DATABASE() and USER() system functions (#35946) 2018-11-28 15:20:04 +02:00
Yannick Welsch 5f0c036183 Disable testDeleteCreateInOneBulk on Zen2
This test needs adaptation to run with Zen2
2018-11-28 13:34:48 +01:00
Hendrik Muhs 8e86340d1d
set line lengths for comments equal to code (#35984)
eclipse (default) wraps code comments at 80 characters, this change sets line length for code comments equal to code == 140.
2018-11-28 13:34:20 +01:00
Peter Dyson 1f25a0bd31 [Docs] Add example for updating meta field (#35893) 2018-11-28 12:04:57 +01:00
Christoph Büscher 33865211db
[Docs] Emphazise suggest behaviour with missing query part (#35393)
Add a short extra sentence that explains that a missing query part in a search
request containing a "suggest" section will mean only suggestions are returned.

Closes #31640
2018-11-28 12:01:27 +01:00
Vladimir Dolzhenko f0c5f0c099 [HLRC] XPack ML info action (#35777)
Relates to #29827
2018-11-28 10:58:20 +00:00
Costin Leau 95346926ef
SQL: Lock down JDBC driver (#35798)
Move classes under the same package to avoid internal classes being
 exposed to the outside. Remove public visibility outside 3 classes:
 EsDriver, EsDataSource and EsTypes.
The driver only has one package, namely org.elasticsearch.xpack.sql.jdbc
Use Es prefix for classes to ease name conflict and indicate their
 destination

Fix #35437
2018-11-28 12:48:26 +02:00
Christoph Büscher 2f547bac65
Remove deprecated methods from QueryStringQueryBuilder (#35912)
This change removes the deprecated useDisMax() and useAllFields() methods from
the QueryStringQueryBuilder and related tests. The disMax parameter has already
been a no-op since 6.0 and also the useAllFields has been deprecated since 6.0
and there is a direct replacement via defaultField.
2018-11-28 11:09:03 +01:00
Ioannis Kakavas d948edfd69 [TESTS] Mute SmokeTestWatcherWithSecurityIT tests
Issue tracked in
https://github.com/elastic/elasticsearch/issues/35361
2018-11-28 11:34:41 +02:00
Ioannis Kakavas fc7e7e7d81 [TESTS] Mute SNI tests on FIPS 140 JVMs
These tests use a TrustAllConfig TrustManager as they set
verification_mode to none, that can't be used in a FIPS 140 JVM.
2018-11-28 09:35:36 +02:00
Jason Tedor c42d9d91c9
Deprecate X-Pack centric SQL endpoints (#35964)
This commit is part of our plan to deprecate and ultimately remove the
use of _xpack in the REST APIs.
2018-11-27 22:16:21 -05:00
Yogesh Gaikwad e50e0f997a
[Kerberos] Add support for Kerberos V5 Oid (#35764)
Clients can use the Kerberos V5 security mechanism and when it
used this to establish security context it failed to do so as
Elasticsearch server only accepted Spengo mechanism.
This commit adds support to accept Kerberos V5 credentials
over spnego.

Closes #34763
2018-11-28 13:29:43 +11:00
Michael Basnight 19ed17195f
Docs: DRY up indices docs (#35971)
This commit DRYs up the indices folder as well as fixing a few minor
mishaps that were in the docs.
2018-11-27 19:40:49 -06:00
Jason Tedor b84f1f6a3a
Deprecate X-Pack centric rollup endpoints (#35962)
This commit is part of our plan to deprecate and ultimately remove the
use of _xpack in the REST APIs.
2018-11-27 20:34:17 -05:00
Albert Zaharovits 36819f78ef
DOCS Audit event attributes in new format (#35510)
Accounts for the `Structured Audit Entries` in the format
documentation.
2018-11-28 01:24:03 +02:00
Ryan Ernst 383713d4e9
Scripting: Actually add joda time back to whitelist (#35965)
This commit actually loads the joda whitelist, which was missed in
2018-11-27 15:18:00 -08:00
Tal Levy a0b319a123 [DOCS] fix HLRC ILM doc misreferenced tag 2018-11-27 14:28:25 -08:00
Ioannis Kakavas 580b5baf21
Add realm information for Authenticate API (#35648)
- Add the authentication realm and lookup realm name and type in the response for the _authenticate API
- The authentication realm is set as the lookup realm too (instead of setting the lookup realm to null or empty ) when no lookup realm is used.
2018-11-27 23:35:42 +02:00
Tal Levy fe603e9163
[ILM] add HLRC docs to remove-policy-from-index (#35759)
This primarily introduces documentation for the HLRC
remove-policy-from-index action.
2018-11-27 13:28:58 -08:00
Zachary Tong 1a1092d08f [Rollup] Update serialization version after backport 2018-11-27 16:02:48 -05:00
Zachary Tong 48fa251812
[Rollup] Add more diagnostic stats to job (#35471)
* [Rollup] Add more diagnostic stats to job

To help debug future performance issues, this adds the
 min/max/avg/count/total latencies (in milliseconds) for search
and bulk phase.  This latency is the total service time including
transfer between nodes, not just the `took` time.

It also adds the count of search/bulk failures encountered during
runtime.  This information is also in the log, but a runtime counter
will help expose problems faster

* review cleanup

* Remove dead ParseFields
2018-11-27 15:46:10 -05:00