Commit Graph

28049 Commits

Author SHA1 Message Date
Lee Hinman d2c69701e6 Fix SocketPermission in test framework for alpha2 bump
It was using the wrong version, which can cause errors like

```
  1> java.security.AccessControlException: access denied ("java.net.SocketPermission" "[0:0:0:0:0:0:0:1]:34221" "connect,resolve")
  1> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_111]
  1> 	at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0_111]
  1> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_111]
  1> 	at java.lang.SecurityManager.checkConnect(SecurityManager.java:1051) ~[?:1.8.0_111]
  1> 	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:625) ~[?:?]
  1> 	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processSessionRequests(DefaultConnectingIOReactor.java:273) ~[httpcore-nio-4.4.5.jar:4.4.5]
  1> 	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:139) ~[httpcore-nio-4.4.5.jar:4.4.5]
  1> 	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:348) ~[httpcore-nio-4.4.5.jar:4.4.5]
  1> 	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192) ~[httpasyncclient-4.1.2.jar:4.1.2]
1> at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64) ~[httpasyncclient-4.1.2.jar:4.1.2]
```

When running tests
2017-05-10 11:27:57 -06:00
Christoph Büscher ae5919e5df Tests: Fix VersionTests after version bump 2017-05-10 18:18:22 +02:00
javanna 2e602d6641 [TEST] reset subAggregationsSupplier in AggregationsTests to empty sub-aggs once the maxDepth is reached
If we don't do this, although we don't set the subAggsSupplier again, we will reuse the one set in the previous run, hence we can end up in situations where we keep on generating sub aggregations till a StackOverflowError gets thrown.
2017-05-10 17:51:58 +02:00
Christoph Büscher fbc8345db5 Tests: Fix VersionUtilsTests after version bump 2017-05-10 17:36:12 +02:00
Christoph Büscher 64ecf4f4bf Bumping version to v6.0.0-alpha2 2017-05-10 17:12:56 +02:00
David Roberts d611ab4855 Avoid race when shutting down controller processes (#24579)
This commit terminates any controller processes plugins might have after
the node has been closed.  This gives the plugins a chance to shut down their
controllers gracefully.

Previously there was a race condition where controller processes could be shut
down gracefully and terminated by two threads running in parallel, leading to
non-deterministic outcomes.

Additionally, controller processes that failed to shut down gracefully were
not forcibly terminated when running as a Windows service; there was a reliance
on the plugin to shut down its controller gracefully in this situation.
This commit also fixes this problem.
2017-05-10 14:59:14 +01:00
Tanguy Leroux 3201e22710 Fix merging conflicts 2017-05-10 14:05:43 +02:00
Clinton Gormley 2486086980 Deprecate the Tribe node
The Tribe node is deprecated in favour of Cross Cluster Search and will be removed in 7.0.
2017-05-10 14:05:12 +02:00
Tanguy Leroux bf718a686f Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/InternalTermsTestCase.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalExtendedStatsTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalMaxTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalStatsTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/avg/InternalAvgTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/cardinality/InternalCardinalityTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/geobounds/InternalGeoBoundsTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/geocentroid/InternalGeoCentroidTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/min/InternalMinTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/InternalPercentilesTestCase.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/hdr/InternalHDRPercentilesRanksTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/InternalTDigestPercentilesRanksTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/sum/InternalSumTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/valuecount/InternalValueCountTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/InternalSimpleValueTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/bucketmetrics/percentile/InternalPercentilesBucketTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/derivative/InternalDerivativeTests.java
#	test/framework/src/main/java/org/elasticsearch/test/InternalAggregationTestCase.java
2017-05-10 13:40:39 +02:00
Tanguy Leroux 3e4406f044 [Test] Fix wrong import 2017-05-10 12:31:28 +02:00
Tanguy Leroux 2fe53be0db [Test] Add unit tests for Range aggregations (#24569)
Related to #22278
2017-05-10 12:22:52 +02:00
Martijn van Groningen 0ff5933a55
Rewrite multi search template api to delegate to multi search api instead of to search template api.
The max concurrent searches logic is complex and we shouldn't duplicate that in multi search template api,
so we should template each individual template search request and then delegate to multi search api.
2017-05-10 11:47:53 +02:00
Tanguy Leroux 7c97296d77 [Test] Reference Terms aggregations in AggregationsTests 2017-05-10 11:37:20 +02:00
Tanguy Leroux 2178491387 Fix checkstyle violation 2017-05-10 11:34:34 +02:00
Martijn van Groningen 760e5fce77
Rewrite multi search template api to delegate to multi search api instead of to search template api.
The max concurrent searches logic is complex and we shouldn't duplicate that in multi search template api,
so we should template each individual template search request and then delegate to multi search api.
2017-05-10 11:23:24 +02:00
Isabel Drost-Fromm bd559d96d4
This adds max_concurrent_searches to multi-search-template endpoint.
Closes #20912
2017-05-10 11:23:24 +02:00
Martijn van Groningen 51c74ce547
Added unit tests for InternalMatrixStats.
Also moved InternalAggregationTestCase to test-framework module in order to make use of it from other modules than core.

Relates to #22278
2017-05-10 11:06:18 +02:00
Matt Weber b24326271e Add ICUCollationFieldMapper (#24126)
Adds a new "icu_collation" field type that exposes lucene's
ICUCollationDocValuesField.  ICUCollationDocValuesField is the replacement
for ICUCollationKeyFilter which has been deprecated since Lucene 5.
2017-05-10 10:35:11 +02:00
Tanguy Leroux 3c66ac06ae Add parsing for String/Long/Double Terms aggregations (#24521) 2017-05-10 10:17:25 +02:00
Ryan Ernst 3f1ef488cd fix checkstyle after script engine rename 2017-05-10 01:04:02 -07:00
Ryan Ernst 9ca7d28552 Scripting: Remove "service" from ScriptEngine interface name (#24574)
This commit renames ScriptEngineService to ScriptEngine.  It is often
confusing because we have the ScriptService, and then
ScriptEngineService implementations, but the latter are not services as
we see in other places in elasticsearch.
2017-05-10 00:47:33 -07:00
Christoph Büscher 01b976071f Merge branch 'master' into feature/client_aggs_parsing 2017-05-10 09:40:26 +02:00
Ryan Ernst 5242d3aeb0 Fix ids query builder test to use the same logic for matchnodocs as the
builder itself
2017-05-09 16:53:19 -07:00
Ryan Ernst ebd3e5f73f Scripting: Deprecate file script settings (#24555)
File scripts have 2 related settings: the path of file scripts, and
whether they can be dynamically reloaded. This commit deprecates those
settings.

relates #21798
2017-05-09 16:14:57 -07:00
Tal Levy 945b3cd4ca fix ingest pipeline timestamp timezone format check to be even more generic 2017-05-09 15:31:32 -07:00
Ryan Ernst 53f6d9412e Fix ids query test when none or ALL type is used
See #24460
2017-05-09 14:52:28 -07:00
Tal Levy d2416a3a40 modify ingest timestamp format check to remove timezone 2017-05-09 14:33:43 -07:00
Jason Tedor b6906702cc Remove global checkpoint constraint
A constraint on the global checkpoint was inadvertently committed from
the inlining global checkpoint work. Namely, the constraint prevents the
global checkpoint from advancing to no ops performed, a situation that
can occur when shards are started but empty.
2017-05-09 17:17:09 -04:00
Jason Tedor 3f54ae6264 Fix handling of spaces in plugin script on Windows
A previous fix for handling spaces in the Windows scripts should also be
applied to the plugin script. This commit does that.

Closes #24750
2017-05-09 15:37:43 -04:00
javanna 37dc165e7f [TEST] add test to verify that no aggs have been forgotten in AggregationsTests 2017-05-09 21:23:00 +02:00
Suhas Karanth 09c5fbfd00 Docs: Correct description of example (#24541)
Copy and paste error.
2017-05-09 15:18:43 -04:00
Jason Tedor 8f873620ee Inline global checkpoints
Today we rely on background syncs to relay the global checkpoint under
the mandate of the primary to its replicas. This means that the global
checkpoint on a replica can lag far behind the primary. The commit moves
to inlining global checkpoints with replication requests. When a
replication operation is performed, the primary will send the latest
global checkpoint inline with the replica requests. This keeps the
replicas closer in-sync with the primary.

However, consider a replication request that is not followed by another
replication request for an indefinite period of time. When the replicas
respond to the primary with their local checkpoint, the primary will
advance its global checkpoint. During this indefinite period of time,
the replicas will not be notified of the advanced global
checkpoint. This necessitates a need for another sync. To achieve this,
we perform a global checkpoint sync when a shard falls idle.

Relates #24513
2017-05-09 15:08:11 -04:00
Luca Cavanna c5bdbecc64 [TEST] Add test for Aggregations#fromXContent (#24524)
AggregationsTests#testFromXContent verifies that parsing of aggregations works by combining multiple aggs at the same level, and also adding sub-aggregations to multi bucket and single bucket aggs, up to a maximum depth of 5.
2017-05-09 21:08:10 +02:00
Nik Everett bb06d8ec4f Allow plugins to build pre-configured token filters (#24223)
This changes the way we register pre-configured token filters so that
plugins can declare them and starts to move all of the pre-configured
token filters out of core. It doesn't finish the job because doing
so would make the change unreviewably large. So this PR includes
a shim that keeps the "old" way of registering pre-configured token
filters around.

The Lowercase token filter is special because there is a "special"
interaction between it and the lowercase tokenizer. I'm not sure
exactly what to do about it so for now I'm leaving it alone with
the intent of figuring out what to do with it in a followup.

This also renames these pre-configured token filters from
"pre-built" to "pre-configured" because that seemed like a more
descriptive name.

This is a part of #23658
2017-05-09 14:50:49 -04:00
Christoph Büscher 1959fd6a1a Merge branch 'master' into feature/client_aggs_parsing 2017-05-09 19:54:36 +02:00
Nik Everett 428390865c Tests: Give stats tests long to wait for listener
This test waited 10 seconds for a refresh listener to appear in
the stats. It turns out that in our NFS testing infrastructure this can
take a lot longer than 10 seconds. The error reported here:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+nfs/257/consoleFull
has it taking something like 15 seconds. This bumps the timeout
to a solid minute.

Closes #24417
2017-05-09 13:33:33 -04:00
Jim Ferenczi b6c714ccc8 Fix BWC for query_and_fetch 2017-05-09 18:52:53 +02:00
Clinton Gormley b8dede06b5 Fixed prerelease URLs for RPM and Deb repos in docs 2017-05-09 18:33:23 +02:00
Jim Ferenczi a404f0aca7 Cleanup BWC for single shard optimization
The BWC layer is needed only for pre 5.3 indices.
2017-05-09 16:54:44 +02:00
Adrien Grand a72eaa8e0f Identify documents by their `_id`. (#24460)
Now that indices have a single type by default, we can move to the next step
and identify documents using their `_id` rather than the `_uid`.

One notable change in this commit is that I made deletions implicitly create
types. This helps with the live version map in the case that documents are
deleted before the first type is introduced. Otherwise there would be no way
to differenciate `DELETE index/foo/1` followed by `PUT index/foo/1` from
`DELETE index/bar/1` followed by `PUT index/foo/1`, even though those are
different if versioning is involved.
2017-05-09 16:33:52 +02:00
Jason Tedor f222748506 Increase compilation limit in ingest tests
These tests are running into the script compilation limit. This commit
increases the limit for these tests.
2017-05-09 09:42:02 -04:00
Clinton Gormley 41b10554fd Mark 6.0.0-alpha1 as prerelease 2017-05-09 14:02:11 +02:00
Clinton Gormley d77757fc1b Updated release notes for 6.0.0-alpha1 2017-05-09 14:01:32 +02:00
Jim Ferenczi 4df636b5ef Fix single shard scroll within a cluster with nodes in version >= 5.3 and <= 5.3 (#24512)
If a node in version >= 5.3 acts as a coordinating node during a scroll request that targets a single shard, the scroll may return the same documents over and over iff the targeted shard is hosted by a node with a version <= 5.3.
The nodes in this version will advance the scroll only if the search_type has been set to `query_and_fetch` though this search type has been removed in 5.3.
This change handles this situation by adding the removed search_type in the request that targets a node in version <= 5.3.
2017-05-09 09:14:17 +02:00
Tal Levy 423b0f5e3d add option for _ingest.timestamp to use new ZonedDateTime (#24030)
Previously, Mustache would call `toString` on the `_ingest.timestamp`
field and return a date format that did not match Elasticsearch's
defaults for date-mapping parsing. The new ZonedDateTime class in Java 8
happens to do format itself in the same way ES is expecting.

This commit adds support for a feature flag that enables the usage of this new date format 
that has more native behavior.

Fixes #23168.

This new fix can be found in the form of a cluster setting called
`ingest.new_date_format`. By default, in 5.x, the existing behavior
will remain the same. One will set this property to `true` in order to
take advantage of this update for ingest-pipeline convenience.
2017-05-08 15:06:39 -07:00
joachimdraeger fec1802e2f Fixes inefficient loading of snapshot repository data (#24510)
This commit fixes inefficient (worst case exponential) loading of 
snapshot repository data when checking for incompatible snapshots,
that was introduced in #22267.  When getting snapshot information,
getRepositoryData() was called on every snapshot, so if there are
a large number of snapshots in the repository and _all snapshots
were requested, the performance degraded exponentially.  This
commit fixes the issue by only calling getRepositoryData once and
using the data from it in all subsequent calls to get snapshot 
information.

Closes #24509
2017-05-08 17:43:01 -04:00
Ryan Ernst bd3717a0f8 Scripting: Deprecate file scripts (#24552)
File scripts will be removed in 6.0. This commit adds a deprecation
warning for 5.5 when the first file script is loaded.
2017-05-08 14:06:45 -07:00
Jason Tedor 0944577ee8 Remove commented code from ESILRTC
This commit removes some commented out code from
ESIndexLevelReplicationTestCase.java that was inadvertently committed.
2017-05-08 16:20:58 -04:00
Jason Tedor 1de5bb8cd5 Ensure test replicas have valid recovery state
When starting a new replication group in an index level replication test
case, a started replica would not have a valid recovery state. This
violates simple assumptions as replicas always have to have recovered
before being started. This commit causes this to be the case that this
assumption is not violated too.
2017-05-08 16:19:47 -04:00
Jason Tedor 89b305c09e Add global checkpoint assertion in index shard
We previously removed this assertion because it could be violated in
races. This commit adds this assertion back with sampling done more
carefully to avoid failures solely due to race conditions.
2017-05-08 13:37:28 -04:00