Commit Graph

25294 Commits

Author SHA1 Message Date
David Pilato 2842e2752a Updated dynamic mapper section
Backport of #21574 in master (6.0)
2016-11-16 09:52:08 +01:00
Christoph Büscher 4ccd8e79c1 Docs: Clarify date_histogram bucket sizes for DST time zones
Added a warning note that clarifies bucket sizes diverging from the intended
`interval` size when using a time zone that has DST changes.

Closes #18805
2016-11-16 09:40:07 +01:00
Boaz Leskes db87837c72 disable backwards-5.0 tests as there is no BWC layer in the seq# related components 2016-11-16 08:04:08 +00:00
Nik Everett e66261eee9 Handle release of 5.0.1
Adds a version constant for it, bwc indices, and a vagrant upgrade-from
version. Also bumps the "upgrade from" version for the backwards-5.0
test and adds `skip`s for tests that don't fail against 5.0 so we skip
them during the backwards testing.

Finally, this skips the "Shrink index via API" test because it fails
consistently for me. Inconsistently for CI, but consistently for me.
I'll work on making it consistent tomorrow.
2016-11-15 19:31:28 -05:00
Jason Tedor cca22088ac Fix skip reason for stats API parameters test
This commit corrects the reasons that the stats API parameters tests are
skipped against 5.0 nodes.
2016-11-15 17:52:45 -05:00
Jason Tedor 6a8a3e77ed Reduce skip version for stats API parameter tests
This commit reduces the skip version for the stats API unrecognized
parameter tests now that the logic for unrecognized parameters has been
backported to 5.x.
2016-11-15 17:43:19 -05:00
Jason Tedor 17b0041aaf Strict level parsing for indices stats
A previous commit added strict level parsing for the node stats API, but
that commit missed adding the same for the indices stats API. This
commit rectifies this miss.

Relates #21577
2016-11-15 16:26:37 -05:00
Yannick Welsch 40e0162e61 Remove cluster update task when task times out (#21578)
Fixes an issue where the cluster service does not remove an update task from its internal data structures that are used for batching cluster state updates.

* review comments

* checkstyle
2016-11-15 21:38:58 +01:00
Lee Hinman 17a2fffc9b [DOCS] Mention "all-fields" mode doesn't search across nested documents 2016-11-15 11:02:43 -07:00
Boaz Leskes d99d02ecc3 InternalTestCluster: when restarting a node we should validate the cluster is formed via the node we just restarted
This is to deal with potential delays in processing the fact that node was node is restarted.
2016-11-15 17:58:08 +00:00
Clinton Gormley a4e88bb64a Fixed bad asciidoc in boolean mapping docs 2016-11-15 17:50:23 +00:00
Clinton Gormley cfabc95f59 Fixed bad asciidoc ID in node stats 2016-11-15 17:39:15 +00:00
Lee Hinman 96122aa518 Be strict when parsing values searching for booleans (#21555)
This changes only the query parsing behavior to be strict when searching on
boolean values. We continue to accept the variety of values during index time,
but searches will only be parsed using `"true"` or `"false"`.

Resolves #21545
2016-11-15 10:36:57 -07:00
Boaz Leskes 9171407906 remove an unneeded assert busy 2016-11-15 17:36:06 +00:00
Christoph Büscher cd4634bdc6 Fix time zone rounding edge case for DST overlaps
When using TimeUnitRounding with a DAY_OF_MONTH unit, failing tests in #20833
uncovered an issue when the DST shift happenes just one hour after midnight
local time and sets back the clock to midnight, leading to an overlap.
Previously this would lead to two different rounding values, depending on
whether a date before or after the transition was rounded. This change detects
this special case and correct for it by using the previous rounding date for
both cases.

Closes #20833
2016-11-15 18:23:47 +01:00
Bryan Karlovitz f03723a812 Minor style change to getting-started.asciidoc (#21571)
* In line 525, change the phrase "... with as little network roundtrips as possible." to "... with as few network roundtrips as possible."
2016-11-15 12:18:49 -05:00
Jason Tedor f5ac0e5076 Remove lenient stats parsing
Today when parsing a stats request, Elasticsearch silently ignores
incorrect metrics. This commit removes lenient parsing of stats requests
for the nodes stats and indices stats APIs.

Relates #21417
2016-11-15 12:17:26 -05:00
Boaz Leskes 2c0338fa87 Merge remote-tracking branch 'upstream/master' into feature/seq_no 2016-11-15 17:09:08 +00:00
Lee Hinman 568a7ea5f1 Fix incorrect instructions for disabling deprecation logging (#21569)
We log deprecation events at "WARN", so setting it to `info` means the events
are still logged. It must be set to `error` in order to disable the logging.
2016-11-15 09:57:26 -07:00
Nik Everett 7dcff27aea Update docs for scripted metric agg
Now that the default language is painless the examples didn't work at
all. This fixes them.

Closes #21536
2016-11-15 11:47:17 -05:00
Tanguy Leroux 6cafe688b3 [TEST] Ensure file permission for /etc/sudoers.d/elasticsearch_vars
Ubuntu 12.04 checks the file permission for /etc/sudoers.d/elasticsearch_vars is mode 0440. This commit adds a `chmod` before the file is used by the `sudo` command.
2016-11-15 17:42:57 +01:00
Adrien Grand 6db683a4bd Fix recurring doc test failures with the cat API. (#21561)
This failure is due to the fact that we sort on store size, which is cached. So
it might happen that the store size that is taken into account is not the right
one, which makes the indices sorted in the wrong order. This changes the doc
example to sort on the number of docs instead.

Closes #21062
2016-11-15 16:00:44 +01:00
Adrien Grand df4482fdc8 Do not cache the QueryShardContext in PercolatorFieldMapper: it is cheap to create. 2016-11-15 15:45:18 +01:00
Tanguy Leroux aa871f839a [Tests] Do not pass all env vars to sudo (#21562)
In #21348 the command executed to run the packaging tests has been changed to "sudo -E bats ...", forcing all environment variables from the vagrant user to be passed to the `sudo` command. This breaks a test on opensuse-13 (the one where it checks that elasticsearch cannot be started when `java` is not found) because all the PATH from the user is passed to the sudo command.

This commit restores the previous behavior while allowing only necessary testing environment variables to be passed using a /etc/sudoers.d file.
2016-11-15 15:19:34 +01:00
Boaz Leskes d6c2b4f7c5 Adapt InternalTestCluster to auto adjust `minimum_master_nodes` (#21458)
#20960 removed `LocalDiscovery` and we now use `ZenDiscovery` in all our tests. To keep cluster forming fast, we are using a `MockZenPing` implementation which uses static maps to return instant results making master election fast. Currently, we don't set `minimum_master_nodes` causing the occasional split brain when starting multiple nodes concurrently and their pinging is so fast that it misses the fact that one of the node has elected it self master. To solve this, `InternalTestCluster` is modified to behave like a true cluster and manage and set `minimum_master_nodes` correctly with every change to the number of nodes.

Tests that want to manage the settings themselves can opt out using a new `autoMinMasterNodes` parameter to the `ClusterScope` annotation. 

Having `min_master_nodes` set means the started node may need to wait for other nodes to be started as well. To combat this, we set `discovery.initial_state_timeout` to `0` and wait for the cluster to form once all node have been started. Also, because a node may wait and ping while other nodes are started, `MockZenPing` is adapted to wait rather than busy-ping.
2016-11-15 13:42:26 +00:00
Jason Tedor ee722d738a Fix internal engine sequence number test bug
This commit fixes a test bug in internal engine tests, and adds some
additional assertions.
2016-11-15 08:34:54 -05:00
Simon Willnauer 66fbb0dbc2 Don't fail in `afterExecute` if context is already closed (#21563)
We run an assert on an potentially closed thread context. this should
not bubble up the `IllegalStateException`.
2016-11-15 13:55:50 +01:00
Adrien Grand 54809065a6 Make PercolatorFieldMapper get a QueryShardContext lazily. 2016-11-15 12:02:40 +01:00
Boaz Leskes c9f49039d3 Merge remote-tracking branch 'upstream/master' into feature/seq_no 2016-11-15 10:14:47 +00:00
Simon Willnauer 4c53e55d34 [TEST] Add master_timeout to shrink call to fail faster 2016-11-15 10:59:59 +01:00
Sumit Gupta e53405f4f3 Update geohashgrid-aggregation.asciidoc (#21530) 2016-11-15 10:49:02 +01:00
Simon Willnauer 200a2850a9 [TEST] Don't stop MockAppender some nodes might concurrently use it 2016-11-15 10:48:39 +01:00
Boaz Leskes 6d9af2fff4 Uncommitted mapping updates should not efect existing indices (#21306)
When processing a mapping updates, the master current creates an `IndexService` and uses its mapper service to do the hard work. However, if the master is also a data node and it already has an instance of `IndexService`, we currently reuse the the `MapperService` of that instance. Sadly, since mapping updates are change the in memory objects, this means that a mapping change that can rejected later on during cluster state publishing will leave a side effect on the index in question, bypassing the cluster state safety mechanism.

This commit removes this optimization and replaces the `IndexService` creation with a direct creation of a `MapperService`. 

Also, this fixes an issue multiple from multiple shards for the same field caused unneeded cluster state publishing as the current code always created a new cluster state.

This were discovered while researching #21189
2016-11-15 10:47:34 +01:00
Adrien Grand ad94bea0bb Remove XPointValues. (#21541)
This class had been added to address a bug in PointValues, which has been fixed
since then.
2016-11-15 10:11:41 +01:00
Yannick Welsch 64a7a960d9 Use pre-JDK9 style FilePermissions on JDK9 (#21540)
JDK9 removed pathname canonicalization when constructing FilePermission objects, which breaks some of the FilePermissions added by Elasticsearch. This commit adds the system property jdk.io.permissionsUseCanonicalPath which makes JDK9 behave like JDK8 w.r.t. FilePermission objects (see #21534).
2016-11-15 09:31:32 +01:00
Martijn van Groningen 8a3a885058 inner_hits: Skip adding a parent field to nested documents.
Otherwise an empty string get added as _parent field.

Closes #21503
2016-11-15 07:32:28 +01:00
Ryan Ernst c7bd4f3454 Tests: Add TestZenDiscovery and replace uses of MockZenPing with it (#21488)
This changes adds a test discovery (which internally uses the existing
mock zenping by default). Having the mock the test framework selects be a discovery
greatly simplifies discovery setup (no more weird callback to a Node
method).
2016-11-14 21:46:10 -08:00
Ryan Ernst 2cdf3eeae0 Remove generics from ActionRequest (#21556)
closes #21368
2016-11-14 16:31:17 -08:00
Ryan Ernst d14c470b89 Remove generics from ActionRequest
closes #21368
2016-11-14 15:32:01 -08:00
Jason Tedor 48579cccab Add socket permissions for tribe nodes
Today when a node starts, we create dynamic socket permissions based on
the configured HTTP ports and transport ports. If no ports are
configured, we use the default port ranges. When a tribe node starts, a
tribe node creates an internal node client for connecting to each remote
cluster. If neither an explicit HTTP port nor transport ports were
specified, the default port ranges are large enough for the tribe node
and its internal node clients. If an explicit HTTP port or transport
port was specified for the tribe node, then socket permissions for those
ports will be created, but not for the internal node clients. Whether
the internal node clients have explicit ports specified, or attempt to
bind within the default range, socket permissions for these will not
have been created and the internal node clients will hit a permissions
issue when attempting to bind. This commit addresses this issue by also
accounting for tribe nodes when creating the dynamic socket
permissions. Additionally, we add our first real integration test for
tribe nodes.

Relates #21546
2016-11-14 15:09:45 -05:00
Jay Modi 87d76c3ff8 assert blocking calls are not made on the cluster state update thread
This commit adds an assertion to ensure that we do not introduce blocking calls in code
that is called in a ClusterStateListener or another part of the cluster state update process.
2016-11-14 14:30:01 -05:00
Jason Tedor 795c121852 Remove explicit task paths in tribe node test
This commit removes some explicit paths in the tribe node smoke test so
that these are easier to maintain (in the case of reorganization).
2016-11-14 13:52:57 -05:00
Yannick Welsch 6f69f54216 Use pre-JDK9 style FilePermissions on JDK9 when running from start scripts 2016-11-14 19:52:35 +01:00
Jason Tedor 9fb54f4ef8 Remove unnecessary hash map copy in o.e.b.Security
This commit removes an unnecessary copying of the tribe node group
settings in o.e.b.Security.
2016-11-14 13:49:16 -05:00
Jason Tedor a12f09317d Fallback to settings if transport profile is empty
If the transport profile does not contain a TCP port range, we fallback
to the top-level settings.
2016-11-14 13:48:12 -05:00
Jason Tedor 491a945ac8 Add socket permissions for tribe nodes
Today when a node starts, we create dynamic socket permissions based on
the configured HTTP ports and transport ports. If no ports are
configured, we use the default port ranges. When a tribe node starts, a
tribe node creates an internal node client for connecting to each remote
cluster. If neither an explicit HTTP port nor transport ports were
specified, the default port ranges are large enough for the tribe node
and its internal node clients. If an explicit HTTP port or transport
port was specified for the tribe node, then socket permissions for those
ports will be created, but not for the internal node clients. Whether
the internal node clients have explicit ports specified, or attempt to
bind within the default range, socket permissions for these will not
have been created and the internal node clients will hit a permissions
issue when attempting to bind. This commit addresses this issue by also
accounting for tribe nodes when creating the dynamic socket
permissions. Additionally, we add our first real integration test for
tribe nodes.
2016-11-14 11:58:44 -05:00
Simon Willnauer 1d8c8529ed Remove `IndexTemplateAlreadyExistsException` and `IndexShardAlreadyExistsException` (#21539)
Both exception can be replaced with java built-in exception, IAE and ISE respectively.
This should be back ported partially to 5.x which the transport layer code should be preserved.

Relates to #21494
2016-11-14 17:09:57 +01:00
Simon Willnauer 26375256ff Enable 5.x to 6.x BWC tests (#21537)
This commit enables real BWC testing against a 5.1 snapshot. All
REST tests plus rolling upgrade test now run against a mixed version
cross major version cluster.
2016-11-14 17:03:57 +01:00
Yannick Welsch d3e97ce6cd Fix line length in TCPTransportTests
Makes checkstyle happy
2016-11-14 16:55:14 +01:00
Yannick Welsch d42f7eec61 Check valid cluster service state transitions (#21538)
This commit adds assertions to check whether the cluster service state transitions in a way that we expect it to.

Relates to #21379.
2016-11-14 16:49:25 +01:00