Commit Graph

1514 Commits

Author SHA1 Message Date
Nhat Nguyen 2cffd85f7f Merge branch 'master' into ccr
* master:
  Remove RestGetAllAliasesAction (#31308)
  Temporary fix for broken build
  Reenable Checkstyle's unused import rule (#31270)
  Remove remaining unused imports before merging #31270
  Fix non-REST doc snippet
  [DOC] Extend SQL docs
  Immediately flush channel after writing to buffer (#31301)
  [DOCS] Shortens ML API intros
  Use quotes in the call invocation (#31249)
  move security ingest processors to a sub ingest directory (#31306)
  Add 5.6.11 version constant.
  Fix version detection.
  SQL: Whitelist SQL utility class for better scripting (#30681)
  [Docs] All Rollup docs experimental, agg limitations, clarify DeleteJob (#31299)
  CCS: don't proxy requests for already connected node (#31273)
  Mute ScriptedMetricAggregatorTests testSelfReferencingAggStateAfterMap
  [test] opensuse packaging turn up debug logging
  Add unreleased version 6.3.1
  Removes experimental tag from scripted_metric aggregation (#31298)
  [Rollup] Metric config parser must use builder so validation runs (#31159)
  [ML] Check licence when datafeeds use cross cluster search  (#31247)
  Add notion of internal index settings (#31286)
  Test: Remove broken yml test feature (#31255)
  REST hl client: cluster health to default to cluster level (#31268)
  [ML] Update test thresholds to account for changes to memory control (#31289)
  Log warnings when cluster state publication failed to some nodes (#31233)
  Fix AntFixture waiting condition (#31272)
  Ignore numeric shard count if waiting for ALL (#31265)
  [ML] Implement new rules design (#31110)
  index_prefixes back-compat should test 6.3 (#30951)
  Core: Remove plain execute method on TransportAction (#30998)
  Update checkstyle to 8.10.1 (#31269)
  Set analyzer version in PreBuiltAnalyzerProviderFactory (#31202)
  Modify pipelining handlers to require full requests (#31280)
  Revert upgrade to Netty 4.1.25.Final (#31282)
  Use armored input stream for reading public key (#31229)
  Fix Netty 4 Server Transport tests. Again.
  REST hl client: adjust wait_for_active_shards param in cluster health (#31266)
  REST high-level Client: remove deprecated API methods (#31200)
  [DOCS] Mark SQL feature as experimental
  [DOCS] Updates machine learning custom URL screenshots (#31222)
  Fix naming conventions check for XPackTestCase
  Fix security Netty 4 transport tests
  Fix race in clear scroll (#31259)
  [DOCS] Clarify audit index settings when remote indexing (#30923)
  Delete typos in SAML docs (#31199)
  REST high-level client: add Cluster Health API (#29331)
  [ML][TEST] Mute tests using rules (#31204)
  Support RequestedAuthnContext (#31238)
  SyncedFlushResponse to implement ToXContentObject (#31155)
  Add Get Aliases API to the high-level REST client (#28799)
  Remove some line length supressions (#31209)
  Validate xContentType in PutWatchRequest. (#31088)
  [INGEST] Interrupt the current thread if evaluation grok expressions take too long (#31024)
  Suppress extras FS on caching directory tests
  Revert "[DOCS] Added 6.3 info & updated the upgrade table. (#30940)"
  Revert "Fix snippets in upgrade docs"
  Fix snippets in upgrade docs
  [DOCS] Added 6.3 info & updated the upgrade table. (#30940)
  LLClient: Support host selection (#30523)
  Upgrade to Netty 4.1.25.Final (#31232)
  Enable custom credentials for core REST tests (#31235)
  Move ESIndexLevelReplicationTestCase to test framework (#31243)
  Encapsulate Translog in Engine (#31220)
  HLRest: Add get index templates API (#31161)
  Remove all unused imports and fix CRLF (#31207)
  [Tests] Fix self-referencing tests
  [TEST] Fix testRecoveryAfterPrimaryPromotion
  [Docs] Remove mention pattern files in Grok processor (#31170)
  Use stronger write-once semantics for Azure repository (#30437)
  Don't swallow exceptions on replication (#31179)
  Limit the number of concurrent requests per node (#31206)
  Call ensureNoSelfReferences() on _agg state variable after scripted metric agg script executions (#31044)
  Move java version checker back to its own jar (#30708)
  [test] add fix for rare virtualbox error (#31212)
2018-06-14 10:47:36 -04:00
Tanguy Leroux 4d7447cb5e
Reenable Checkstyle's unused import rule (#31270) 2018-06-14 09:52:46 +02:00
Nik Everett 77bb93557e
Test: Remove broken yml test feature (#31255)
The `requires_replica` yaml test feature hasn't worked for years. This
is what happens if you try to use it:
```
   > Throwable #1: java.lang.NullPointerException
   >    at __randomizedtesting.SeedInfo.seed([E6602FB306244B12:6E341069A8D826EA]:0)
   >    at org.elasticsearch.test.rest.yaml.Features.areAllSupported(Features.java:58)
   >    at org.elasticsearch.test.rest.yaml.section.SkipSection.skip(SkipSection.java:144)
   >    at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:321)
```

None of our tests use it.
2018-06-13 09:33:06 -04:00
Ryan Ernst a65b18f19d Core: Remove plain execute method on TransportAction (#30998)
TransportAction has many variants of execute. One of those variants
executes by returning a future, which is then often blocked on by
calling get(). This commit removes this variant of execute, instead
using a helper method for tests that want to block, or having tests
pass in a PlainActionFuture directly as a listener.

Co-authored-by: Simon Willnauer <simonw@apache.org>
2018-06-13 09:58:13 +02:00
Van0SS d5e8a5cd69 REST high-level client: add Cluster Health API (#29331)
Relates to #27205
2018-06-12 13:34:06 +02:00
olcbean 7d7ead95b2 Add Get Aliases API to the high-level REST client (#28799)
Given the weirdness of the response returned by the get alias API, we went for a client specific response, which allows us to hold the error message, exception and status returned as part of the response together with aliases. See #30536 .

Relates to #27205
2018-06-12 10:26:17 +02:00
Nik Everett 0d9b78834f
LLClient: Support host selection (#30523)
Allows users of the Low Level REST client to specify which hosts a
request should be run on. They implement the  `NodeSelector` interface
or reuse a built in selector like `NOT_MASTER_ONLY` to chose which nodes
are valid. Using it looks like:
```
Request request = new Request("POST", "/foo/_search");
RequestOptions options = request.getOptions().toBuilder();
options.setNodeSelector(NodeSelector.NOT_MASTER_ONLY);
request.setOptions(options);
...
```

This introduces a new `Node` object which contains a `HttpHost` and the
metadata about the host. At this point that metadata is just `version`
and `roles` but I plan to add node attributes in a followup. The
canonical way to **get** this metadata is to use the `Sniffer` to pull
the information from the Elasticsearch cluster.

I've marked this as "breaking-java" because it breaks custom
implementations of `HostsSniffer` by renaming the interface to
`NodesSniffer` and by changing it from returning a `List<HttpHost>` to a
`List<Node>`. It *shouldn't* break anyone else though.

Because we expect to find it useful, this also implements `host_selector`
support to `do` statements in the yaml tests. Using it looks a little
like:

```
---
"example test":
  - skip:
      features: host_selector
  - do:
      host_selector:
        version: " - 7.0.0" # same syntax as skip
      apiname:
        something: true
```

The `do` section parses the `version` string into a host selector that
uses the same version comparison logic as the `skip` section. When the
`do` section is executed it passed the off to the `RestClient`, using
the `ElasticsearchHostsSniffer` to sniff the required metadata.

The idea is to use this in mixed version tests to target a specific
version of Elasticsearch so we can be sure about the deprecation
logging though we don't currently have any examples that need it. We do,
however, have at least one open pull request that requires something
like this to properly test it.

Closes #21888
2018-06-11 17:07:27 -04:00
Nhat Nguyen dda56fc0fc
Move ESIndexLevelReplicationTestCase to test framework (#31243)
Other components might benefit from the testing infra provided by
ESIndexLevelReplicationTestCase. This commit moves it to the test
framework.
2018-06-11 12:47:38 -04:00
Lee Hinman c064b507df
Encapsulate Translog in Engine (#31220)
This removes the abstract `getTranslog` method in `Engine`, instead leaving it
to the abstract implementations of the other methods that use the translog. This
allows future Engines not to have a Translog, as instead they must implement the
methods that use the translog pieces to return necessary values.
2018-06-11 09:44:50 -06:00
Tanguy Leroux bf58660482
Remove all unused imports and fix CRLF (#31207)
The X-Pack opening and the recent other refactorings left a lot of 
unused imports in the codebase. This commit removes them all.
2018-06-11 15:12:12 +02:00
Nhat Nguyen 9feff9809b CCR side of #31213
Relates #31213
2018-06-09 20:04:45 -04:00
Nhat Nguyen afe02a3cac Merge branch 'master' into ccr
* master:
  Move default location of dependencies report (#31228)
  Remove dependencies report task dependencies (#31227)
  Add recognition of MPL 2.0 (#31226)
  Fix unknown licenses (#31223)
  Remove version from license file name for GCS SDK (#31221)
  Fully encapsulate LocalCheckpointTracker inside of the engine (#31213)
  [DOCS] Added 'fail_on_unsupported_field' param to MLT. Closes #28008 (#31160)
  Add licenses for transport-nio (#31218)
  Remove DocumentFieldMappers#simpleMatchToFullName. (#31041)
  Allow to trim all ops above a certain seq# with a term lower than X, post backport fix (#31211)
  Compliant SAML Response destination check (#31175)
  Remove DocumentFieldMappers#smartNameFieldMapper, as it is no longer needed. (#31018)
  Remove extraneous references to 'tokenized' in the mapper code. (#31010)
  Allow to trim all ops above a certain seq# with a term lower than X (#30176)
  SQL: Make a single JDBC driver jar (#31012)
  Enhance license detection for various licenses (#31198)
  [DOCS] Add note about long-lived idle connections (#30990)
  Move number of language analyzers to analysis-common module (#31143)
  Default max concurrent search req. numNodes * 5 (#31171)
  flush job to ensure all results have been written (#31187)
2018-06-09 19:53:21 -04:00
Lee Hinman bdb0fb2555
Fully encapsulate LocalCheckpointTracker inside of the engine (#31213)
* Fully encapsulate LocalCheckpointTracker inside of the engine

This makes the Engine interface not expose the `LocalCheckpointTracker`, instead
exposing the pieces needed (like retrieving the local checkpoint) as individual
methods.
2018-06-08 17:19:41 -06:00
Julie Tibshirani 00b0e10063
Remove DocumentFieldMappers#simpleMatchToFullName. (#31041)
* Remove DocumentFieldMappers#simpleMatchToFullName, as it is duplicative of MapperService#simpleMatchToIndexNames.
* Rename MapperService#simpleMatchToIndexNames -> simpleMatchToFullName for consistency.
* Simplify EsIntegTestCase#assertConcreteMappingsOnAll to accept concrete fields instead of wildcard patterns.
2018-06-08 13:53:35 -07:00
Jason Tedor 64b4cdeda6
Merge remote-tracking branch 'elastic/master' into ccr
* elastic/master: (53 commits)
  Painless: Restructure/Clean Up of Spec Documentation (#31013)
  Update ignore_unmapped serialization after backport
  Add back dropped substitution on merge
  high level REST api: cancel task (#30745)
  Enable engine factory to be pluggable (#31183)
  Remove vestiges of animal sniffer (#31178)
  Rename elasticsearch-nio to nio (#31186)
  Rename elasticsearch-core to core (#31185)
  Move cli sub-project out of server to libs (#31184)
  [DOCS] Fixes broken link in auditing settings
  QA: Better seed nodes for rolling restart
  [DOCS] Moves ML content to stack-docs
  [DOCS] Clarifies recommendation for audit index output type (#31146)
  Add nio-transport as option for http smoke tests (#31162)
  QA: Set better node names on rolling restart tests
  Add support for ignore_unmapped to geo sort (#31153)
  Share common parser in some AcknowledgedResponses (#31169)
  Fix random failure on SearchQueryIT#testTermExpansionExceptionOnSpanFailure
  Remove reference to multiple fields with one name (#31127)
  Remove BlobContainer.move() method (#31100)
  ...
2018-06-07 23:33:42 -04:00
Jason Tedor e481b860a1
Enable engine factory to be pluggable (#31183)
This commit enables the engine factory to be pluggable based on index
settings used when creating the index service for an index.
2018-06-07 17:01:06 -04:00
Tanguy Leroux b5f05f676c
Remove BlobContainer.move() method (#31100)
closes #30680
2018-06-07 10:48:31 +02:00
Simon Willnauer 5c6711b8a4
Use a `_recovery_source` if source is omitted or modified (#31106)
Today if a user omits the `_source` entirely or modifies the source
on indexing we have no chance to re-create the document after it has
been added. This is an issue for CCR and recovery based on soft deletes
which we are going to make the default. This change adds an additional
recovery source if the source is disabled or modified that is only kept
around until the document leaves the retention policy window.

This change adds a merge policy that efficiently removes this extra source
on merge for all document that are live and not in the retention policy window
anymore.
2018-06-07 07:39:28 +02:00
Tim Brooks 67e73b4df4
Combine accepting selector and socket selector (#31115)
This is related to #27260. This commit combines the AcceptingSelector
and SocketSelector classes into a single NioSelector. This change
allows the same selector to handle both server and socket channels. This
is valuable as we do not necessarily want a dedicated thread running for
accepting channels.

With this change, this commit removes the configuration for dedicated
accepting selectors for the normal transport class. The accepting
workload for new node connections is likely low, meaning that there is
no need to dedicate a thread to this process.
2018-06-06 11:59:54 -06:00
Jason Tedor bf1152fcc6
Use follower primary term when applying operations (#31113)
The primary shard copy on the following has authority of the replication
operations that occur on the following side in cross-cluster
replication. Yet today we are using the primary term directly from the
operations on the leader side. Instead we should be replacing the
primary term on the following side with the primary term of the primary
on the following side. This commit does this by copying the translog
operations with the corrected primary term. This ensures that we use
this primary term while applying the operations on the primary, and when
replicating them across to the replica (where the replica request was
carrying the primary term of the primary shard copy on the follower).
2018-06-06 11:03:57 -04:00
Yannick Welsch 1dca00deb9
Remove extra checks from HdfsBlobContainer (#31126)
This commit saves one network roundtrip when reading or deleting files from an HDFS repository.
2018-06-06 16:38:37 +02:00
Nhat Nguyen 755a25a41e
Replace exact numDocs by soft-del count in SegmentInfo (#31086)
This PR adapts/utilizes recent enhancements in Lucene-7.4:

- Replaces exactNumDocs by the soft-deletes count in SegmentCommitInfo.
This enhancement allows us to back out changes introduced in #30228.

- Always configure the soft-deletes field in IWC
2018-06-05 14:11:19 -04:00
Nhat Nguyen 2b5c0d77e9 Merge branch 'master' into ccr
* master:
  Removing erroneous repeat
  Adapt bwc versions after backporting #30983 to 6.4
  [Tests] Muting RatedRequestsTests#testXContentParsingIsNotLenient
  TEST:  Retry synced-flush if ongoing ops on primary (#30978)
  Fix docs build.
  Only auto-update license signature if all nodes ready (#30859)
  Add BlobContainer.writeBlobAtomic() (#30902)
  Add a doc value format to binary fields. (#30860)
2018-06-05 10:58:35 -04:00
Tanguy Leroux 9531b7bbcb
Add BlobContainer.writeBlobAtomic() (#30902)
This commit adds a new writeBlobAtomic() method to the BlobContainer
interface that can be implemented by repository implementations which
support atomic writes operations.

When the BlobContainer implementation does not provide a specific 
implementation of writeBlobAtomic(), then the writeBlob() method is used.

Related to #30680
2018-06-05 13:00:43 +02:00
Nhat Nguyen 646e1c01b0 Merge branch 'master' into ccr
* master:
  Add get mappings support to high-level rest client (#30889)
  Fix index prefixes to work with span_multi (#31066)
  [DOCS] Removes redundant authorization pages
  [DOCS] Re-adds custom realm
  Change ObjectParser exception (#31030)
  Upgrade to Lucene-7.4.0-snapshot-0a7c3f462f (#31073)
2018-06-04 16:32:43 -04:00
Nhat Nguyen d9a0a02e89 Merge branch 'master' into ccr
* master:
  Match phrase queries against non-indexed fields should throw an exception (#31060)
  In the internal highlighter APIs, use the field type as opposed to the mapper. (#31039)
  [DOCS] Removes duplicated authentication pages
  Enable customizing REST tests blacklist (#31074)
  Make sure KeywordFieldMapper#clone preserves split_queries_on_whitespace. (#31049)
  [DOCS] Moves machine learning overview to stack-docs
  [ML] Add secondary sort to ML events (#31063)
  [Rollup] Specialize validation exception for easier management (#30339)
  Adapt bwc versions after backporting #31045 to 6.3
  Remove usage of explicit type in docs (#29667)
  Share common readFrom/writeTo code in AcknowledgeResponse (#30983)
  Adapt bwc versions after backporting #31045 to 6.x
  Mute MatchPhrase*QueryBuilderTests
  [Docs] Fix typo in watcher conditions documentation (#30989)
  Remove wrong link in index phrases doc
  Move pipeline APIs to ingest namespace (#31027)
  [DOCS] Fixes accounting setting names (#30863)
  [DOCS] Rewords _field_names documentation (#31029)
  Index phrases (#30450)
  Remove leftover debugging from PTCMDT
  Fix PTCMDT#testMinVersionSerialization
  Make Persistent Tasks implementations version and feature aware (#31045)
2018-06-04 14:32:28 -04:00
Christoph Büscher 3f87c79500
Change ObjectParser exception (#31030)
ObjectParser should throw XContentParseExceptions, not IAE. A dedicated parsing
exception can includes the place where the error occurred.

Closes #30605
2018-06-04 20:20:37 +02:00
Jason Tedor be55da18c2
Enable customizing REST tests blacklist (#31074)
This commit enables adding additional REST tests to the blacklist for
builds that already define tests.rest.blacklist.
2018-06-04 13:35:49 -04:00
Boaz Leskes a7ceefe93f
Make Persistent Tasks implementations version and feature aware (#31045)
With #31020 we introduced the ability for transport clients to indicate what features they support
in order to make sure we don't serialize object to them they don't support. This PR adapts the
serialization logic of persistent tasks to be aware of those features and not serialize tasks that
aren't supported. 

Also, a version check is added for the future where we may add new tasks implementations and
need to be able to indicate they shouldn't be serialized both to nodes and clients.

As the implementation relies on the interface of `PersistentTaskParams`, these are no longer
optional. That's acceptable as all current implementation have them and we plan to make
`PersistentTaskParams` more central in the future.

Relates to #30731
2018-06-03 21:51:08 +02:00
Nhat Nguyen 5fd759b693 Merge branch 'master' into ccr
* master:
  Adapt transport tests for the extra byte introduced in #31020
2018-06-02 08:23:19 -04:00
Boaz Leskes 65d3f0efca Adapt transport tests for the extra byte introduced in #31020
We now serialize a feature array, which takes an extra byte when empty.
2018-06-02 13:04:40 +02:00
Nhat Nguyen a527095b8d Merge branch 'master' into ccr
* master:
  Avoid randomization bug in FeatureAwareTests
  Adjust BWC version on client features
  Add TRACE, CONNECT, and PATCH http methods (#31035)
  Adjust BWC version on client features
  [DOCS] Make geoshape docs less memory hungry (#31014)
  Fix handling of percent-encoded spaces in Windows batch files (#31034)
  [Docs] Fix a typo in Create Index naming limitation (#30891)
  Introduce client feature tracking (#31020)
  Ensure that index_prefixes settings cannot be changed (#30967)
  REST high-level client: add delete ingest pipeline API (#30865)
  [ML][TEST] Fix bucket count assertion in all tests in ModelPlotsIT (#31026)
  Allow rollup job creation only if cluster is x-pack ready (#30963)
  Fix interoperability with < 6.3 transport clients (#30971)
  Add an option to split keyword field on whitespace at query time (#30691)
  [Tests] Fix alias names in PutIndexTemplateRequestTests (#30960)
  REST high-level client: add get ingest pipeline API (#30847)
  Cross Cluster Search: preserve remote status code (#30976)
  High-level client: list tasks failure to not lose nodeId (#31001)
  [DOCS] Fixes links (#31011)
  Watcher: Give test a little more time
  Reuse expiration date of trial licenses (#30950)
  Remove unused query methods from MappedFieldType. (#30987)
  Transport client: Don't validate node in handshake (#30737)
  [DOCS] Clarify not all PKCS12 usable as truststores (#30750)
  HLRest: Allow caller to set per request options (#30490)
  Remove version read/write logic in Verify Response (#30879)
  [DOCS] Update readme for testing x-pack code snippets (#30696)
  Ensure intended key is selected in SamlAuthenticatorTests (#30993)
  Core: Remove RequestBuilder from Action (#30966)
2018-06-01 23:09:29 -04:00
Jason Tedor 4522b57e07
Introduce client feature tracking (#31020)
This commit introduces the ability for a client to communicate to the
server features that it can support and for these features to be used in
influencing the decisions that the server makes when communicating with
the client. To this end we carry the features from the client to the
underlying stream as we carry the version of the client today. This
enables us to enhance the logic where we make protocol decisions on the
basis of the version on the stream to also make protocol decisions on
the basis of the features on the stream. With such functionality, the
client can communicate to the server if it is a transport client, or if
it has, for example, X-Pack installed. This enables us to support
rolling upgrades from the OSS distribution to the default distribution
without breaking client connectivity as we can now elect to serialize
customs in the cluster state depending on whether or not the client
reports to us using the feature capabilities that it can under these
customs. This means that we would avoid sending a client pieces of the
cluster state that it can not understand. However, we want to take care
and always send the full cluster state during node-to-node communication
as otherwise we would end up with different understanding of what is in
the cluster state across nodes depending on which features they reported
to have. This is why when deciding whether or not to write out a custom
we always send the custom if the client is not a transport client and
otherwise do not send the custom if the client is transport client that
does not report to have the feature required by the custom.

Co-authored-by: Yannick Welsch <yannick@welsch.lu>
2018-06-01 11:45:35 -04:00
Jim Ferenczi 0791f93dbd
Add an option to split keyword field on whitespace at query time (#30691)
This change adds an option named `split_queries_on_whitespace` to the `keyword`
field type. When set to true full text queries (`match`, `multi_match`, `query_string`, ...) that target the field will split the input on whitespace to build the query terms. Defaults to `false`.
Closes #30393
2018-06-01 09:47:03 +02:00
Nik Everett b225f5e5c6
HLRest: Allow caller to set per request options (#30490)
This modifies the high level rest client to allow calling code to
customize per request options for the bulk API. You do the actual
customization by passing a `RequestOptions` object to the API call
which is set on the `Request` that is generated by the high level
client. It also makes the `RequestOptions` a thing in the low level
rest client. For now that just means you use it to customize the
headers and the `httpAsyncResponseConsumerFactory` and we'll add
node selectors and per request timeouts in a follow up.

I only implemented this on the bulk API because it is the first one
in the list alphabetically and I wanted to keep the change small
enough to review. I'll convert the remaining APIs in a followup.
2018-05-31 13:59:52 -04:00
Ryan Ernst 46e8d97813
Core: Remove RequestBuilder from Action (#30966)
This commit removes the RequestBuilder generic type from Action. It was
needed to be used by the newRequest method, which in turn was used by
client.prepareExecute. Both of these methods are now removed, along with
the existing users of prepareExecute constructing the appropriate
builder directly.
2018-05-31 16:15:00 +02:00
Martijn van Groningen 56472d6505
Merge remote-tracking branch 'es/master' into ccr
* es/master: (24 commits)
  Add missing_bucket option in the composite agg (#29465)
  Rename index_prefix to index_prefixes (#30932)
  Rename methods in PersistentTasksService (#30837)
  [DOCS] Fix watcher file location
  Update the version checks around range bucket keys, now that the change was backported.
  Use dedicated ML APIs in tests (#30941)
  [DOCS] Remove reference to platinum Docker image (#30916)
  Minor clean-up in InternalRange. (#30886)
  stable filemode for zip distributions (#30854)
  [DOCS] Adds missing TLS settings for auditing (#30822)
  [test] packaging: use shell when running commands (#30852)
  Fix location of AbstractHttpServerTransport (#30888)
  [test] packaging test logging for suse distros
  Moved keyword tokenizer to analysis-common module (#30642)
  Upgrade to Lucene-7.4-snapshot-1cbadda4d3 (#30928)
  Limit the scope of BouncyCastle dependency (#30358)
  [DOCS] Reset edit links (#30909)
  Fix IndexTemplateMetaData parsing from xContent (#30917)
  Remove log traces in AzureStorageServiceImpl and fix test (#30924)
  Deprecate accepting malformed requests in stored script API (#28939)
  ...
2018-05-30 09:55:34 +02:00
Martijn van Groningen 544822c78b
Moved keyword tokenizer to analysis-common module (#30642)
Relates to #23658
2018-05-29 19:22:28 +02:00
Nhat Nguyen 03e3bd28c9 Merge branch 'master' into ccr
* master:
  silence InstallPluginCommandTests, see https://github.com/elastic/elasticsearch/issues/30900
  Remove left-over comment
  Fix double semicolon in import statement
  [TEST] Fix minor random bug from #30794
  Include size of snapshot in snapshot metadata #18543, bwc clean up (#30890)
  Enabling testing against an external cluster (#30885)
  Add public key header/footer (#30877)
  SQL: Remove the last remaining server dependencies from jdbc (#30771)
  Include size of snapshot in snapshot metadata (#29602)
  Do not serialize basic license exp in x-pack info (#30848)
  Change BWC version for VerifyRepositoryResponse (#30796)
  [DOCS] Document index name limitations (#30826)
  Harmonize include_defaults tests (#30700)
2018-05-28 12:16:20 -04:00
Vladimir Dolzhenko 81eb8ba0f0
Include size of snapshot in snapshot metadata (#29602)
Include size of snapshot in snapshot metadata

Adds difference of number of files (and file sizes) between prev and current snapshot. Total number/size reflects total number/size of files in snapshot.

Closes #18543
2018-05-25 21:04:50 +02:00
Martijn van Groningen e6dd4144ce
Merge remote-tracking branch 'es/master' into ccr
* es/master:
  Move score script context from SearchScript to its own class (#30816)
  Fix bad version check writing Repository nodes (#30846)
  [docs] explainer for java packaging tests (#30825)
  Remove Throwable usage from transport modules (#30845)
  REST high-level client: add put ingest pipeline API (#30793)
  Update the version checks around ip_range bucket keys, now that the change was backported.
  Mute IndexMasterFailoverIT.testMasterFailoverDuringIndexingWithMappingChanges
  Use geohash cell instead of just a corner in geo_bounding_box (#30698)
  Limit user to single concurrent auth per realm (#30794)
  [Tests] Move templated _rank_eval tests (#30679)
  Security: fix dynamic mapping updates with aliases (#30787)
  Ensure that ip_range aggregations always return bucket keys. (#30701)
  Use remote client in TransportFieldCapsAction (#30838)
  Move Watcher versioning setting to meta field (#30832)
  [Docs] Explain incomplete dates in range queries (#30689)
  Move persistent task registrations to core (#30755)
  Decouple ClusterStateTaskListener & ClusterApplier (#30809)
  Send client headers from TransportClient (#30803)
  Packaging: Ensure upgrade_is_oss flag file is always deleted (#30732)
  Force stable file modes for built packages (#30823)
2018-05-25 11:07:45 +02:00
Martijn van Groningen ae2f021f1c
Move score script context from SearchScript to its own class (#30816) 2018-05-25 07:17:50 +02:00
Tim Brooks e8b70273c1
Remove Throwable usage from transport modules (#30845)
Currently nio and netty modules use the CompletableFuture class for
managing listeners. This is unfortunate as that class accepts
Throwable. This commit adds a class CompletableContext that wraps
the CompletableFuture but does not accept Throwable. This allows the
modification of netty and nio logic to no longer handle Throwable.
2018-05-24 17:33:29 -06:00
David Turner ff0b6c795a
Decouple ClusterStateTaskListener & ClusterApplier (#30809)
Today, the `ClusterApplier` and `MasterService` both use the
`ClusterStateTaskListener` interface to notify their callers when asynchronous
activities have completed. However, this is not wholly appropriate: none of the
callers into the `ClusterApplier` care about the `ClusterState` arguments that
they receive.  This change introduces a dedicated ClusterApplyListener
interface for callers into the `ClusterApplier`, to distinguish these listeners
from the real `ClusterStateTaskListener`s that are waiting for responses from
the `MasterService`.
2018-05-24 09:05:09 +01:00
Nhat Nguyen 3fb2c45fa7 Merge branch 'master' into ccr
* master:
  [DOCS] Fixes typos in security settings
  Fix GeoShapeQueryBuilder serialization after backport
  [DOCS] Splits auditing.asciidoc into smaller files
  Reintroduce mandatory http pipelining support (#30820)
  Painless: Types Section Clean Up (#30283)
  Add support for indexed shape routing in geo_shape query (#30760)
  [test] java tests for archive packaging (#30734)
  Revert "Make http pipelining support mandatory (#30695)" (#30813)
  [DOCS] Fix more edit URLs in Stack Overview (#30704)
  Use correct cluster state version for node fault detection (#30810)
  Change serialization version of doc-value fields.
  [DOCS] Fixes broken link for native realm
  [DOCS] Clarified audit.index.client.hosts (#30797)
  [TEST] Don't expect acks when isolating nodes
  Add a `format` option to `docvalue_fields`. (#29639)
  Fixes UpdateSettingsRequestStreamableTests mutate bug
  Mustes {p0=snapshot.get_repository/10_basic/*} YAML test
  Revert "Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled"
  Only allow x-pack metadata if all nodes are ready (#30743)
  Mutes MachineLearningTests.testNoAttributes_givenSameAndMlEnabled
  Use original settings on full-cluster restart (#30780)
  Only ack cluster state updates successfully applied on all nodes (#30672)
  Expose Lucene's FeatureField. (#30618)
  Fix a grammatical error in the 'search types' documentation.
  Remove http pipelining from integration test case (#30788)
2018-05-23 23:05:14 -04:00
Tim Brooks d7040ad7b4
Reintroduce mandatory http pipelining support (#30820)
This commit reintroduces 31251c9 and 63a5799. These commits introduced a
memory leak and were reverted. This commit brings those commits back
and fixes the memory leak by removing unnecessary retain method calls.
2018-05-23 14:38:52 -06:00
Colin Goodheart-Smithe 4fd0a3e492 Revert "Make http pipelining support mandatory (#30695)" (#30813)
This reverts commit 31251c9 introduced in #30695.

We suspect this commit is causing the OOME's reported in #30811 and we will use this PR to test this assertion.
2018-05-23 10:54:46 -06:00
Yannick Welsch 30b004f582
Use original settings on full-cluster restart (#30780)
When doing a node restart using the test framework, the restarted node does not only use the
settings provided to the original node, but also additional settings provided by plugin extensions,
which does not correspond to the settings that a node would have on a true restart.
2018-05-23 09:02:01 +02:00
Tim Brooks 63a5799526
Remove http pipelining from integration test case (#30788)
This is related to #29500. We are removing the ability to disable http
pipelining. This PR removes the references to disabling pipelining in
the integration test case.
2018-05-22 17:18:05 -06:00
Nhat Nguyen fb48b029e8 Merge branch 'master' into ccr
* master:
  QA: Add xpack tests to rolling upgrade (#30795)
  Modify state of VerifyRepositoryResponse for bwc (#30762)
  Reduce CLI scripts to one-liners on Windows (#30772)
  Simplify number of shards setting (#30783)
  Replace Request#setHeaders with addHeader (#30588)
  [TEST] remove endless wait in RestClientTests (#30776)
  [Docs] Fix script-fields snippet execution (#30693)
  Upgrade to Lucene-7.4.0-snapshot-cc2ee23050 (#30778)
  [DOCS] Add SAML configuration information (#30548)
  [DOCS] Remove X-Pack references from SQL CLI (#30694)
  Make http pipelining support mandatory (#30695)
  [Docs] Fix typo in circuit breaker docs (#29659)
  [Feature] Adding a char_group tokenizer (#24186)
  [Docs] Fix broken cross link in documentation
  Test: wait for netty threads in a JUnit ClassRule (#30763)
  Increase the maximum number of filters that may be in the cache. (#30655)
  [Security] Include an empty json object in an json array when FLS filters out all fields (#30709)
  [TEST] Wait for CS to be fully applied in testDeleteCreateInOneBulk
  Add more yaml tests for get alias API (#29513)
  Ignore empty completion input (#30713)
  [DOCS] fixed incorrect default
  [ML] Filter undefined job groups from update calendar actions (#30757)
  Fix docs failure on language analyzers (#30722)
  [Docs] Fix inconsistencies in snapshot/restore doc (#30480)
  Enable installing plugins from snapshots.elastic.co (#30765)
  Remove fedora 26, add 28 (#30683)
  Accept Gradle build scan agreement (#30645)
  Remove logging from elasticsearch-nio jar (#30761)
  Add Delete Repository High Level REST API (#30666)
2018-05-22 17:24:00 -04:00