Commit Graph

39855 Commits

Author SHA1 Message Date
Ryan Ernst 49387bf2d7
Core: Combine doExecute methods in TransportAction (#31517)
TransportAction currently contains 2 doExecute methods, one which takes
a the task, and one that does not. The latter is what some subclasses
implement, while the first one just calls the latter, dropping the given
task. This commit combines these methods, in favor of just always
assuming a task is present.
2018-06-25 09:41:22 +01:00
Vladimir Dolzhenko 2ba50dbf50
IndexShard should not return null stats (#31528)
IndexShard should not return null stats - empty stats or AlreadyCloseException if it's closed is better
2018-06-25 09:41:21 +01:00
Tal Levy 1d11fdaad8 Merge branch 'master' into index-lifecycle 2018-06-22 10:11:15 -07:00
Vladimir Dolzhenko 7313a987f4
fix repository update with the same settings but different type (#31458)
fix repository update with the same settings but different type
2018-06-22 17:44:13 +02:00
Armin Braun 3c42bfad4e
Fix Mockito trying to mock IOException that isn't thrown by method (#31433) (#31527) 2018-06-22 17:24:27 +02:00
Luca Cavanna 16e4e7a7cf
Node selector per client rather than per request (#31471)
We have made node selectors configurable per request, but all 
of other language clients don't allow for that.
A good reason not to do so, is that having a different node selector 
per request breaks round-robin. This commit makes NodeSelector 
configurable only at client initialization. It also improves the docs 
on this matter, important given that a single node selector can still 
affect round-robin.
2018-06-22 17:15:29 +02:00
Ryan Ernst 59e7c6411a
Core: Combine messageRecieved methods in TransportRequestHandler (#31519)
TransportRequestHandler currently contains 2 messageReceived methods,
one which takes a Task, and one that does not. The first just delegates
to the second. This commit changes all existing implementors of
TransportRequestHandler to implement the version which takes Task, thus
allowing the class to be a functional interface, and eliminating the
need to throw exceptions when a task needs to be ensured.
2018-06-22 07:36:03 -07:00
Adrien Grand f023e95ae0
Upgrade to Lucene 7.4.0. (#31529)
This moves Elasticsearch from a recent 7.4.0 snapshot to the GA release.
2018-06-22 16:17:17 +02:00
Dimitris Athanasiou c6cbc99f9c
[ML] Add ML filter update API (#31437)
This adds an api to allow updating a filter:

POST _xpack/ml/filters/{filter_id}/_update

The request body may have:

- description: setting a new description
- add_items: a list of the items to add
- remove_items: a list of the items to remove

This commit also changes the PUT filter api to
error when the filter_id is already used. As
now there is an api for updating filters, the
put api should only be used to create new ones.

Also, updating a filter results into a notification
message auditing the change for every job that is
using that filter.
2018-06-22 15:13:31 +01:00
Yannick Welsch f22f91c57a
Allow multiple unicast host providers (#31509)
Introduces support for multiple host providers, which allows the settings based hosts resolver to be
treated just as any other UnicastHostsProvider. Also introduces the notion of a HostsResolver so
that plugins such as FileBasedDiscovery do not need to create their own thread pool for resolving
hosts, making it easier to add new similar kind of plugins.
2018-06-22 15:31:23 +02:00
Adrien Grand 8ae2049889
Avoid deprecation warning when running the ML datafeed extractor. (#31463)
In #29639 we added a `format` option to doc-value fields and deprecated usage
of doc-value fields without a format so that we could migrate doc-value fields
to use the format that comes with the mappings by default. However I missed to
fix the machine-learning datafeed extractor.
2018-06-22 13:46:48 +02:00
Colin Goodheart-Smithe e6b427c8e4
Fixes compile error due to client interface change 2018-06-22 10:03:43 +01:00
Colin Goodheart-Smithe d062556754
Merge branch 'master' into index-lifecycle 2018-06-22 10:00:35 +01:00
Sohaib Iftikhar eade161894 REST high-level client: add simulate pipeline API (#31158)
relates to #27205
2018-06-22 09:59:04 +02:00
Luca Cavanna 0352d88621
Get Mapping API to honour allow_no_indices and ignore_unavailable (#31507)
Get Mapping currently throws index not found exception (and returns
404 status code) from the REST layer whenever an index was specified
and no indices have been returned. We should not have this logic in the
REST layer though as only our index resolver should decide whether we
need to throw exceptions or not based on provided indices and corresponding
indices options.

Closes #31485
2018-06-22 09:57:32 +02:00
Yogesh Gaikwad 009ae48cba
[PkiRealm] Invalidate cache on role mappings change (#31510)
PkiRealm caches successful authentications and provides ways to
invalidate the cache. But in some scenario's the cache was not being
invalidated on role mapping change.
PkiRealm does not inform role mapper to be notified for cache
refresh on role mapping updates.
The logic in `TransportClearRealmCacheAction#nodeOperation`
which gets invoked for refreshing cache on realms, considers null or
empty realm names in the request as clear cache on all realms. When
LDAP realm is not present then it clears cache for all realms so it
works fine, but when LDAP realm is configured then role mapper
sends a request with LDAP realm names and so the cache is cleared
only for those realms.

This commit resolves the issue by registering PkiRealm with role
mapper for cache refresh. PkiRealm implements CachingRealm and as it
does not extend CachingUsernamePasswordRealm, have modified the
interface method `refreshRealmOnChange` to accept CachingRealm.
2018-06-22 17:47:20 +10:00
Yogesh Gaikwad 724438a0b0
[Security] Check auth scheme case insensitively (#31490)
According to RFC 7617, the Basic authentication scheme name
should not be case sensitive.
Case insensitive comparisons are also applicable for the bearer
tokens where Bearer authentication scheme is used as per
RFC 6750 and RFC 7235

Some Http clients may send authentication scheme names in
different case types for eg. Basic, basic, BASIC, BEARER etc.,
so the lack of case-insensitive check is an issue when these
clients try to authenticate with elasticsearch.

This commit adds case-insensitive checks for Basic and Bearer
authentication schemes.

Closes #31486
2018-06-22 10:15:38 +10:00
Julie Tibshirani 3b7225e9d1
In NumberFieldType equals and hashCode, make sure that NumberType is taken into account. (#31514) 2018-06-21 16:53:20 -07:00
Costin Leau 99f503e3be [DOCS] Fix REST tests in SQL docs
Fixed a search & replace gone awry
Tweaked the docs a bit
2018-06-22 00:16:03 +03:00
Lisa Cawley 438591566f
[DOCS] Add code snippet testing in more ML APIs (#31339) 2018-06-21 11:32:11 -07:00
Ryan Ernst 4f9332ee16
Core: Remove ThreadPool from base TransportAction (#31492)
Most transport actions don't need the node ThreadPool. This commit
removes the ThreadPool as a super constructor parameter for
TransportAction. The actions that do need the thread pool then have a
member added to keep it from their own constructor.
2018-06-21 11:25:26 -07:00
lcawl 60204af0cb [DOCS] Remove fixed file from build.gradle 2018-06-21 11:13:19 -07:00
Nhat Nguyen 048a92bf39
Rename createNewTranslog to fileBasedRecovery (#31508)
We renamed `createNewTranslog` to `fileBasedRecovery` in the
RecoveryTarget but did not do this for RecoverySourceHandler.
This commit makes sure that we a consistent parameter in both
recovery source and target.
2018-06-21 13:50:46 -04:00
Nik Everett 6f3e97f2b7 Test: Skip assertion on windows
Windows doesn't provide consistent exception messages when it can't
connect so skip the exception message assertion on windows.

Closes #31457
2018-06-21 13:24:39 -04:00
Lisa Cawley bd06563e78
[DOCS] Creates field and document level security overview (#30937) 2018-06-21 10:08:50 -07:00
Costin Leau 872418ff94 [DOCS] Significantly improve SQL docs
Introduce SQL commands
Move reserved keywords into an appendix
Add section on security
Introduce concepts section
2018-06-21 20:03:08 +03:00
Tal Levy c9eec7eadc Merge branch 'master' into index-lifecycle 2018-06-21 09:56:45 -07:00
Tal Levy 901aafb946
add APIs to Maintenance Mode in ILM (#31410)
- POST _xpack/index_lifecycle/_stop
   - issues a request to be placed into STOPPED mode (maintenance mode).
     This is not immediate, since we must first verify that
     it is safe to go from STOPPING -> STOPPED.
- POST _xpack/index_lifecycle/_start
   - issues a request to be placed back into RUNNING mode (immediately)
- GET _xpack/index_lifecycle/_status
   - get back the current mode our lifecycle management is in

- update task was hardened to support uninstalled metadata
- if no metadata is installed, the start/stop actions will install metadata
  and proceed to try and change it (default start mode is RUNNING)
- rename MAINTENANCE -> STOPPED, MAINTENANCE_REQUESTED -> STOPPING, NORMAL -> RUNNING

follow-up to #31164.
2018-06-21 09:56:02 -07:00
Lisa Cawley 68ec958873
[DOCS] Move migration APIs to docs (#31473) 2018-06-21 08:19:23 -07:00
Ryan Ernst 0a324b9943
Core: Convert TransportAction.execute uses to client calls (#31487)
This commit converts some of the existing calls to
TransportAction.execute to use the equivalent client method for the
desired action.
2018-06-21 07:59:55 -07:00
Tal Levy fc2f982542
inherit [index.lifecycle.date] from rolled-over time (#30853)
Indices that are rolled over preserve their origin index.lifecycle.date
as the time the index was created. Although this is also fine, it would
make more sense to start the timer for moving to the warm phase from the
time it was rolled over so that we capture the time elapsed from the
latest data, as opposed to when the index was created.

This commit adds an extra step within the RolloverAction that
extracts the index.creation_date of the newly created index from the RolloverStep
and sets that as the index.lifecycle.date of the rolled-over index that is
waiting for its next phase.
2018-06-21 07:18:14 -07:00
Yannick Welsch da69ab28c7
Return transport addresses from UnicastHostsProvider (#31426)
With #20695 we removed local transport and there is just TransportAddress now. The
UnicastHostsProvider currently returns DiscoveryNode instances, where, during pinging, we're
actually only making use of the TransportAddress to establish a first connection to the possible new 
node. To simplify the interface, we can just return a list of transport addresses instead, which
means that it's not necessary anymore to create fake node objects in each plugin just to return the
address information.
2018-06-21 16:00:26 +02:00
Colin Goodheart-Smithe 31976060d9
Adds ability to update a policy (#31361)
* Adds ability to update a policy as long as no indexes are in the shrink
action

* Address review comments
2018-06-21 12:18:59 +01:00
Colin Goodheart-Smithe dec6d49bd2
Adds an API to remove ILM from an index completely (#31358)
* Adds an API to remove ILM from an index completely

The removal will only happen if the index is not in the shrink action

* Fixes compile issues
2018-06-21 09:18:06 +01:00
Tim Brooks 86423f9563
Ensure local addresses aren't null (#31440)
Currently we set local addresses on the creation time of a NioChannel.
However, this may return null as the local address may not have been
set yet. An example is the local address has not been set on a client
channel as the connection process is not yet complete.

This PR modifies the getter to set the local field if it is currently null.
2018-06-20 19:50:14 -06:00
Ryan Ernst 00283a61e1
Remove unused generic type for client execute method (#31444)
This commit removes the request builder generic type for AbstractClient
as it was unused.
2018-06-20 16:26:26 -07:00
Tim Brooks 9ab1325953
Introduce http and tcp server channels (#31446)
Historically in TcpTransport server channels were represented by the
same channel interface as socket channels. This was necessary as
TcpTransport was parameterized by the channel type. This commit
introduces TcpServerChannel and HttpServerChannel classes. Additionally,
it adds the implementations for the various transports. This allows
server channels to have unique functionality and not implement the
methods they do not support (such as send and getRemoteAddress).

Additionally, with the introduction of HttpServerChannel this commit
extracts some of the storing and closing channel work to the abstract
http server transport.
2018-06-20 16:34:56 -06:00
Tal Levy 4c55dbce1a Merge branch 'master' into index-lifecycle 2018-06-20 13:29:47 -07:00
Tal Levy 8bfb9aadd9
lower rollover-info version bound to 6.4 (#31414)
This feature is supported in 6.4, this commit
reflects that
2018-06-20 13:00:36 -07:00
Tal Levy 4f850273b3
extend is-write-index serialization support to 6.4 (#31415) 2018-06-20 13:00:18 -07:00
Costin Leau e202923225 [DOCS] Fix JDBC Maven client group/artifact ID 2018-06-20 21:42:15 +03:00
David Kyle 6ebe6e32cf
[DOCS] Omit shard failures assertion for incompatible responses (#31430)
Filter out the assertion for _cat and _xpack/ml/datafeed APIs
2018-06-20 16:57:51 +01:00
Tal Levy 8d9b98569f Merge branch 'master' into index-lifecycle 2018-06-20 08:30:49 -07:00
Lisa Cawley f012de0f00
[DOCS] Move licensing APIs to docs (#31445) 2018-06-20 08:17:11 -07:00
Daniel Mitterdorfer 90d62e6e4d
Add Delete Snapshot High Level REST API
With this commit we add the delete snapshot API to the Java high level
REST client.

Relates #27205
Relates #31393
2018-06-20 16:42:50 +02:00
Nhat Nguyen db1b97fd85
Remove QueryCachingPolicy#ALWAYS_CACHE (#31451)
The QueryCachingPolicy#ALWAYS_CACHE was deprecated in Lucene-7.4 and
will be removed in Lucene-8.0. This change replaces it with QueryCachingPolicy.
This also makes INDEX_QUERY_CACHE_EVERYTHING_SETTING visible in testing only.
2018-06-20 10:34:08 -04:00
Jonathan Pool 297e99c4c2 [Docs] Extend Homebrew installation instructions (#28902)
Adding a note about proceeding after a successful homebrew installation.
2018-06-20 14:20:51 +02:00
Daniel Mitterdorfer 2aefb72891
Choose JVM options ergonomically
With this commit we add the possibility to define further JVM options (and
system properties) based on the current environment. As a proof of concept, it
chooses Netty's allocator ergonomically based on the maximum defined heap size.
We switch to the unpooled allocator at 1GB heap size (value determined
experimentally, see #30684 for more details). We are also explicit about the
choice of the allocator in either case.

Relates #30684
2018-06-20 13:12:56 +02:00
Peter Dyson e7a7b9689d [Docs] Mention ip_range datatypes on ip type page (#31416)
A link to the ip_range datatype page provides a way for newer users to know
it exists if they land directly on the ip datatype page first via a search.
2018-06-20 13:04:03 +02:00
Colin Goodheart-Smithe c5b69c87c6
Merge branch 'master' into index-lifecycle 2018-06-20 10:31:45 +01:00