Commit Graph

30414 Commits

Author SHA1 Message Date
Jason Tedor 1fa701c18d
Fix using relative custom config path
Previously a user could set a custom config path to a relative directory
using ES_PATH_CONF. In a previous change related to enabling GC logging
by default, we forced the working directory for Elasticsearch to be
ES_HOME. This had the impact of causing all relative paths to be
relative to ES_HOME, against the intent of the user. This commit
addresses this by making ES_PATH_CONF absolute before we switch the
working directory to ES_HOME.

Relates #28700
2018-02-16 06:08:54 -05:00
Colin Goodheart-Smithe d58226be8f
[TEST] Renables bwc tests 2018-02-16 10:17:55 +00:00
Andy Bristol 70b279dbbc [TEST] AwaitsFix testTriggerUpdatesConcurrently 2018-02-15 16:34:24 -08:00
Cladis 3234fb1369 Grammar: "by geographically" → "geographically" (#28595) 2018-02-15 16:12:58 -08:00
Clinton Gormley f54cb567b6 Fixed es_release_notes.pl script to work with the new github labels
It also now ignores docs, test, non-issue, refactoring, and build
2018-02-15 20:31:19 +01:00
Lee Hinman d90a440bf7
Add XContentHelper shim for move to passing in deprecation handler (#28684)
In order to allow us to gradually move to passing the deprecation handler is, we
need a shim that contains both the non-passed and passed version.

Relates to #28504
2018-02-15 11:01:01 -07:00
Jason Tedor 3e846ab251
Handle throws on tasks submitted to thread pools
When we submit a task to a thread pool for asynchronous execution, we
are returned a future. Since we submitted to go asynchronous, these
futures are not inspected for failure (we would have to block a thread
to do that). While we have on failure handlers for exceptions that are
thrown during execution, we do not handle throwables that are not
exceptions and these end up silently lost. This commit adds a check
after the runnable returns that inspects the status of the future. If an
unhandled throwable occurred during execution, this throwable is
propogated out where it will land in the uncaught exception handler.

Relates #28667
2018-02-15 11:59:12 -05:00
javanna 3e17185010 REST high-level client: fix _cluster/settings endpoint
The recent addition of the _cluster/settings API was merged together with another change that added encoding of the different URL parts. That broke the cluster PUT settings API straight-away. This commit fixes this problem. The '/' that's part of the /_cluster/settings endpoint should not be encoded.
2018-02-15 17:40:43 +01:00
Luca Cavanna ebe5e8e635
REST high-level client: encode path parts (#28663)
The REST high-level client supports now encoding of path parts, so that for instance documents with valid ids, but containing characters that need to be encoded as part of urls (`#` etc.), are properly supported. We also make sure that each path part can contain `/` by encoding them properly too.

Closes #28625
2018-02-15 17:22:45 +01:00
olcbean 02fc16f10e Add Cluster Put Settings API to the high level REST client (#28633)
Relates to #27205
2018-02-15 17:21:45 +01:00
Paul Schwarz 81eda1834b Improve wording
"... as less as possible" -> "... as little as possible"
2018-02-15 15:31:00 +00:00
olcbean 658ca5e10b Add a note to the docs that _cat api `help` option cannot be used if an optional url param is used (#28686) 2018-02-15 15:56:01 +01:00
Jason Tedor 671e7e2f00
Lift error finding utility to exceptions helpers
We have code used in the networking layer to search for errors buried in
other exceptions. This code will be useful in other locations so with
this commit we move it to our exceptions helpers.

Relates #28691
2018-02-15 09:48:52 -05:00
Kuaaaly 3e07c6ff54 Change "tweet" type to "_doc" (#28690)
Elasticsearch 6.x indices do not allow multiple types index. Instead, they use "_doc" as default if created internally (Elasticsearch), or "doc" default if sent by Logstash.
2018-02-15 15:41:39 +01:00
Alex Moros Marco abe1e05ba4 [Docs] Add missing word in nested.asciidoc (#28507) 2018-02-15 14:56:02 +01:00
Boaz Leskes beb55d148a
Simplify the Translog constructor by always expecting an existing translog (#28676)
Currently the Translog constructor is capable both of opening an existing translog and creating a
new one (deleting existing files). This PR separates these two into separate code paths. The
constructors opens files and a dedicated static methods creates an empty translog.
2018-02-15 09:24:09 +01:00
Ke Li fc406c9a5a Upgrade t-digest to 3.2 (#28295) (#28305) 2018-02-15 08:23:20 +00:00
Jason Tedor cd54c96d56 Add comment explaining lazy declared versions
A recent change moved computing declared versions from using reflection
which occurred repeatedly to a lazily-initialized holder so that
declared versions are computed exactly once. This commit adds a comment
explaining the motivation for this change.
2018-02-14 23:15:59 -05:00
Nhat Nguyen 452bfc0d83 Backported synced-flush PR to v5.6.8 and v6.2.2
Relates #28464
2018-02-14 14:48:29 -05:00
Lee Hinman b59b1cf59d
Move more XContent.createParser calls to non-deprecated version (#28672)
* Move more XContent.createParser calls to non-deprecated version

Part 2

This moves more of the callers to pass in the DeprecationHandler.

Relates to #28504

* Use parser's deprecation handler where appropriate

* Use logging handler in test that uses deprecated field on purpose
2018-02-14 11:24:48 -07:00
Lee Hinman 7c1f5f5054
Move more XContent.createParser calls to non-deprecated version (#28670)
* Move more XContent.createParser calls to non-deprecated version

This moves more of the callers to pass in the DeprecationHandler.

Relates to #28504

* Use parser's deprecation handler where available
2018-02-14 09:01:40 -07:00
Ryan Ernst 53c38cc8fe
Build: Group archive and package distribution projects (#28673)
This commit adds intermediate gradle projects for archive based
distributions (zip, tar) and package based distributions (rpm, deb). The
grouping allows the common distribution build file to be considerably
shorter and clearly separated from the common zip/tar and rpm/deb
configuration.
2018-02-13 22:49:53 -08:00
Mayya Sharipova 34e95e5d50 [DOCS] Add supported token filters
Update normalizers.asciidoc with the list of supported token filters

Closes #28605
2018-02-13 14:10:25 -08:00
Tal Levy 6c7d12c34c [TEST] bump timeout in testFetchShardsSkipUnavailable to 5s
in response to #28668.
2018-02-13 13:09:43 -08:00
Michael Basnight 04fe9a9ab5
Relax remote check for bwc project checkouts (#28666)
The remote check previously validated both the remote name and the
repository as well, meaning that if someone passed in a repository that
was not a github URL, it would fail. This meant that it was not possible
to fully test bwc out with multiple branches without first pushing to a
remote. Removing the full check allows a user to pass in the origin
remote as its remote, which is already added as a file based remote to
each bwc snapshot build. This will allow changes to be made locally
across all bwc branches, tested, and then pushed simultaneously.
2018-02-13 14:54:11 -06:00
Lee Hinman 7c201a64b5 [TEST] Synchronize searcher list in IndexShardTests
It's possible to check the list size, then attempt to remove a searcher and
throw an IndexOutOfBoundsException due to multiple threads.

Resolves #27651
2018-02-13 12:16:04 -07:00
Andy Bristol 53d1cde7bc
[TEST] packaging: function to collect debug info (#28608)
[TEST] packaging: function to collect debug info

Sometimes when packaging tests fail in CI the test logs aren't enough to
tell what went wrong. This routine helps collect more info about the
state of the es installation at failure time
2018-02-13 10:51:13 -08:00
Scott Somerville a138e0e225 Compute declared versions in a static block
This method is called often enough (when computing minimum compatibility
versions) that the reflection and sort can be seen while profiling. This
commit addresses this issue by computing the declared versions exactly
once.

Relates #28661
2018-02-13 13:24:19 -05:00
Ryan Ernst 9a5199fae3
Docs: Remove references to elasticsearch directory in plugins (#28647)
This directory was removed from plugins in #28589, but docs still
referenced it. This commit cleans up the plugin author docs to no longer
refer to it.
2018-02-13 09:15:57 -08:00
Michael Basnight 3cd4da792c
Remove snapshot conditional for bwc snapshots (#28657)
The build.snapshot flag used by the main build was being propagated down
into the bwc snapshot builds, which is not correct. The bwc subprojects
are always meant to be snapshot builds, or null if they do not
exist. Marking these builds as non snapshots threw the release off as it
was looking for -SNAPSHOT builds.

Relates #28641
2018-02-13 10:35:20 -06:00
Catalin Ursachi ee00523737 Removed redundant JSON object from Put Mapping docs (#28514) 2018-02-13 17:20:35 +01:00
Ali El broudi 974ad680f5 Update threadpool.asciidoc target_response_time (#28655)
Update doc for "target_reponse~~_rate~~_time" param
source => https://github.com/elastic/elasticsearch/blob/6.2/server/src/main/java/org/elasticsearch/threadpool/AutoQueueAdjustingExecutorBuilder.java#L65
2018-02-13 08:30:16 -07:00
Jason Tedor 9e23b7ed72
Add startup logging for standalone tests
We need to investigate why startup is taking so long in CI for
standalone tests. This commit adds logging for bootstrap and network
code that is executed before the node starts initializing in case this
is the source of the trouble.

Relates #28659
2018-02-13 10:14:42 -05:00
Jason Tedor fea1b20529 Revert "Increase the REST test timeout. (#28640)"
This reverts commit db87294a00.
2018-02-13 10:13:32 -05:00
Jim Ferenczi 3b9f530839
Inc store reference before refresh (#28656)
If a tragic even happens while we are refreshing a searcher/reader the engine can open new files on a store that is already closed
For instance the following CI job failed because a merge was concurrently called on a failing shard:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+oracle-java10-periodic/84
This change increments the ref count of the store during a refresh in order to postpone the closing after a tragic event.
2018-02-13 15:38:02 +01:00
Jim Ferenczi 813d8e1f7e Fix meta plugin installation that contains plugins with dependencies
When installing a meta plugin we check the dependency of each sub plugin during the installation.
Though if the extended plugin is part of the meta plugin the installation fails because we only check for plugins that are
already installed. This change is a workaround that extracts all plugins (even those that are not fully installed yet) when the dependency check
is made during the installation. Note that this is how the plugin installation worked before https://github.com/elastic/elasticsearch/pull/28581.
2018-02-13 11:43:34 +01:00
Adrien Grand db87294a00
Increase the REST test timeout. (#28640)
By default we wait for 30 seconds for nodes to start. Looking at CI,
even for passing builds, the startup time is frequently over 20 seconds
(grep for `#wait (Thread[Task worker for ':',5,main]) completed` in any
console log), which is close to the threshold. This commit sets a
default timeout of 120 seconds instead, so that REST tests are less
likely to hit a timeout.
2018-02-13 11:05:26 +01:00
Clinton Gormley bf6378e426 Fixed docs rendering of plugin-descriptor.properties in plugin author docs 2018-02-13 10:35:39 +01:00
Andrew Anderson 54a9249992
Fixed typo in search for wrong type (#28645) 2018-02-13 02:47:01 -05:00
Ryan Ernst ea381969be
Plugins: Separate plugin semantic validation from properties format validation (#28581)
This commit moves the semantic validation (like which version a plugin
was built for or which java version it is compatible with) from reading
a plugin descriptor, leaving the checks on the format of the descriptor
intact.

relates #28540
2018-02-12 21:30:11 -08:00
Michael Basnight 920dff7053
Add released major logic to version utils (#28644)
Version Utils did not previously have logic that removed the last majors
minor snapshot if there was a next bugfix and maintenance bugfix
release. This adds the logic and fixes some broken assumptions in tests
as well.

relates #28505
2018-02-12 18:33:45 -06:00
Robin Neatherway 8d0f976024 Removed unused local variable from ScriptClassInfo (#28617)
Following [8999104](8999104b14)
  the local variable `argumentNames` is no longer used, so removed.
2018-02-12 15:58:09 -08:00
Robin Neatherway 282974215c MetaDataIndexAliasesService wrong get type (#28614)
A get of the wrong type would always have returned null so these
indices would have been inserted into the map repeatedly.
2018-02-12 15:55:17 -08:00
Robin Neatherway 3174b2cbfa NXYSignificanceHeuristic equality update (#28616)
* NXYSignificanceHeuristic.java: implementation of equality would have
  failed with a ClassCastException when comparing to another type.
  Replaced with the Eclipse generated form.
2018-02-12 15:51:57 -08:00
Robin Neatherway 68b7a5c281 Fix DeadlockAnalyzer printer (#28615)
Remove `if` block that was always true.
2018-02-12 15:28:56 -08:00
Rachel Johnson 617044e5fe Update search.asciidoc (#28646)
[DOCS] Corrected typo - singe to single.
2018-02-12 15:02:13 -08:00
Ryan Ernst 65f1dd424e
Plugins: Remove intermediate "elasticsearch" directory within plugin zips (#28589)
This commit removes the extra layer of all plugin files existing under
"elasticsearch" within plugin zips. This simplifies building plugin zips
and removes the need for special logic of modules vs plugins.
2018-02-12 14:27:30 -08:00
Michael Basnight 4e0c1463d5
Fix build.snapshot bug in version collection (#28641)
The build.snapshot was mistakenly passed in to every snapshot version,
so when release tests were run, these versions were mistaken as released
entities and could not be found in maven, because they do not
exist. This fix removes that bug in logic, and always makes them proper
snapshots. This has a benefit of cleaning up the VersionUtilsTests
because they no longer rely on different sets of versions to check
against, which was also a bug.
2018-02-12 14:56:07 -06:00
Nhat Nguyen 9eb9ce3843
Require translogUUID when reading global checkpoint (#28587)
Today we use the persisted global checkpoint to calculate the starting 
seqno in peer-recovery. However we do not check whether the translog 
actually belongs to the existing Lucene index when reading the global
checkpoint. In some rare cases if the translog does not match the Lucene
index, that recovering replica won't be able to complete its recovery.
This can happen as follows.

1. Replica executes a file-based recovery
2. Index files are copied to replica but crashed before finishing the recovery
3. Replica starts recovery again with seq-based as the copied commit is safe
4. Replica fails to open engine because translog and Lucene index are not matched
5. Replica won't be able to recover from primary

This commit enforces the translogUUID requirement when reading the 
global checkpoint directly from the checkpoint file.

Relates #28435
2018-02-12 13:23:32 -05:00
David Turner c15e5d480c
Improve logging in ClusterFormationTasks#configureWaitTask (#28637)
We suspect a build failure might be due to a startup timeout, but there is 
insufficient information in the logs. This change enhances the information
available so we will be better-informed next time.
2018-02-12 18:46:25 +01:00