Commit Graph

23880 Commits

Author SHA1 Message Date
Ali Beyad 456ea56527 Cleans up the BlobContainer interface by removing the (#19727)
writeBlob method takes a BytesReference in favor of just
the writeBlob method that takes an InputStream.

Closes #18528
2016-08-02 09:21:43 -04:00
javanna 772c16702e [DOCS] replace {es.version} in maven coordinates with current version 2016-08-02 15:17:21 +02:00
Ali Beyad a21dd80f1b Documentation changes for wait_for_active_shards (#19581)
Documentation changes and migration doc changes for introducing 
wait_for_active_shards and removing write consistency level.

Closes #19581
2016-08-02 09:15:01 -04:00
Ali Beyad 3d2a105825 Merge pull request #19454 from abeyad/remove-write-consistency-level
Removes write consistency level across replication action APIs in favor of wait_for_active_shards
2016-08-02 09:01:11 -04:00
Daniel Mitterdorfer b99a482992 Improve client benchmark (#19736)
* Allow to run client benchmark as an uberjar
* Busy wait to avoid accidental skew on low target throughput rates
* Trigger and wait for full GC to happen between trials
* Add missing SuppressForbidden to allow System.gc in client benchmark
2016-08-02 14:17:51 +02:00
Daniel Mitterdorfer 88cbfbaabd Ban dangerous methods of java.lang.Thread (#19677)
Ban dangerous methods of java.lang.Thread

With this commit we disallow usage of the method Thread.stop().
2016-08-02 14:11:42 +02:00
Daniel Mitterdorfer 419e9e090e Document and enforce cancellation policy of CancellableThreads (#19712)
With this commit we add documentation and additional checks to
enforce the cancellation policy of CancellableThreads (which is
disallow `Thread#interrupt()` on any of the threads managed by
it).
2016-08-02 08:46:38 +02:00
Ali Beyad 4923da93c8 Refactors wait_for_active_shards index settings tests 2016-08-01 19:14:37 -04:00
Nicholas Knize 63f33e0f1e Serialize doc counts in Matrix-Stats module
This fixes a bug in the RunningStats class for the matrix stats aggregation module. doc counts were not being searlized which means they were only computed the first time the aggregation was computed. This was causing incorrect results when the aggregation was pulled from cache.
2016-08-01 18:05:10 -05:00
Ali Beyad a7e68e36cf Fixes header level for migration doc entry 2016-08-01 16:15:12 -04:00
Lee Hinman f9fd64fc78 Revert to older exception message
If the uuidBytes and ref are converted to utf8, it's possible they can
trip an assertion related to valid UTF-8/UTF-16 ranges, so display them
as hex, not as strings.
2016-08-01 11:51:39 -06:00
Ali Beyad 6a7d005081 Makes the index.write.wait_for_active_shards setting index-level and
dynamically updatable for both index creation and write operations.
2016-08-01 13:37:05 -04:00
Ali Beyad 4a51ea8c8e Before, transport replication actions implemented a checkWriteConsistency()
method to determine if a write consistency check should be performed
before proceeding with the action.  This commit removes this method from
the transport replication actions in favor of setting the ActiveShardCount
on the request, with setting the value to ActiveShardCount.NONE if the
transport action's checkWriteConsistency() method returned false.
2016-08-01 13:35:30 -04:00
Ali Beyad d93f7d6085 Refactors ActiveShardCount 2016-08-01 13:35:29 -04:00
Ali Beyad 25d8eca62d Removes the notion of write consistency level across all APIs in
favor of waiting for active shard copy count (wait_for_active_shards).
2016-08-01 13:35:29 -04:00
Ali Beyad 9f88a8194a Merge pull request #19706 from elastic/enhancement/snapshot-blob-handling
More resilient blob handling in snapshot repositories
2016-08-01 12:03:53 -04:00
Tanguy Leroux 386902903e [TEST] Kill remaining lang-groovy messy tests
After #13834 many tests that used Groovy scripts (for good or bad reason) in their tests have been moved in the lang-groovy module and the issue #13837 has been created to track these messy tests in order to clean them up.

The work started with #19280, #19302 and #19336 and this PR moves the remaining messy tests back in core, removes the dependency on Groovy, changes the scripts in order to use the mocked script engine, and change the tests to integration tests.

It also moves IndexLookupIT test back (even if it has good chance to be removed soon) and fixes its tests.

It also changes AbstractQueryTestCase to use custom script plugins in tests.

closes #13837
2016-08-01 16:59:47 +02:00
Adrien Grand a4cb63b98c Remove `_missing_` from the docs.
It is removed in 5.0, see #15153.
2016-08-01 16:57:37 +02:00
Ali Beyad 401edeb0d8 AzureBlobContainer's deleteBlob method now throws a NoSuchFileException
instead of a vanilla IOException when the blob doesn't exist, in order
to conform to the BlobContainer's interface contract.
2016-08-01 10:50:02 -04:00
Alexander Lin 9ac6389e43 Rename operation to result and reworking responses
* Rename operation to result and reworking responses
* Rename DocWriteResponse.Operation enum to DocWriteResponse.Result

These are just easier to interpret names.

Closes #19664
2016-08-01 10:42:58 -04:00
Isabel Drost-Fromm 24b4b892c0 Merge pull request #19718 from elastic/revert-18519-docs/add_console_to_search
Revert "Add console to docs for inner hits, explain, and friends"
2016-08-01 14:11:35 +02:00
Isabel Drost-Fromm 672ffb6e4d Revert "Add console to docs for inner hits, explain, and friends" 2016-08-01 14:09:54 +02:00
Nik Everett 12fd4ed8f8 Add description to org.elasticsearch.tasks package (#19700)
Yet more readable docs!
2016-08-01 07:43:32 -04:00
Nik Everett aefc36bfaa Add descriptions for o.e.search.suggest packages (#19699)
Let's have readable javadoc!
2016-08-01 07:43:13 -04:00
Boaz Leskes 7c6527ed09 make election stop not be a failure (#19705)
During our master elections, nodes "vote" for a master being issuing a join request to it. Since this is done in an async fashion, joins may arrive before the master itself has realized it had won the election. Therefore we start accumulating node joins on every node at election start (we don't know the result yet). When the election finish nodes that did not become the master (i.e., joined another node which won the election) need to potentially process and fail any incoming join request they may have received during the election. This is currently achieved by always issuing a cluster state update task that is doomed to fail, even if no pending joins are actually there. That aspect results in confusing (debug) log messages, making it seems like something is wrong. For example (note that `NotMasterException`)

```
[2016-07-30 22:25:53,040][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-process-pending-joins [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400} elected]]: execute
[2016-07-30 22:25:53,041][DEBUG][transport                ] [node_t1] connected to node [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400}]
[2016-07-30 22:25:53,045][DEBUG][cluster.service          ] [node_t1] cluster state update task [zen-disco-process-pending-joins [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400} elected]] failed
NotMasterException[Node [{node_t1}{eAQts270TiGFpoCDE-0PQQ}{or5bsv2ET220su78DLJk5g}{127.0.0.1}{127.0.0.1:9401}] not master for join request]
[2016-07-30 22:25:53,048][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-process-pending-joins [{node_t0}{4SqBTyYNQ82J9c75Cs7jtg}{kutaNSYbTZCSybvqczgWCA}{127.0.0.1}{127.0.0.1:9400} elected]]: took [7ms] no change in cluster_state
```

This commit cleans up  the logic a bit to only use failure where there are actual joins that are failed. The result is cleaner logs as well:

```
[2016-07-30 22:23:12,880][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-election-stop [{node_t0}{jMR5HCpOQnOM4pGeFkUjng}{B5WIZQAdQk2cWbjGZ21mvQ}{127.0.0.1}{127.0.0.1:9400} elected]]: execute
[2016-07-30 22:23:12,881][DEBUG][cluster.service          ] [node_t1] processing [zen-disco-election-stop [{node_t0}{jMR5HCpOQnOM4pGeFkUjng}{B5WIZQAdQk2cWbjGZ21mvQ}{127.0.0.1}{127.0.0.1:9400} elected]]: took [0s] no change in cluster_state
[2016-07-30 22:23:12,881][DEBUG][transport                ] [node_t1] connected to node [{node_t0}{jMR5HCpOQnOM4pGeFkUjng}{B5WIZQAdQk2cWbjGZ21mvQ}{127.0.0.1}{127.0.0.1:9400}]
```
2016-08-01 13:08:50 +02:00
Isabel Drost-Fromm 5104437e4f Merge pull request #18519 from MaineC/docs/add_console_to_search
Add console to docs for inner hits, explain, and friends
2016-08-01 12:10:09 +02:00
Tanguy Leroux 64ca834722 Remove unnecessary indices refresh and wait for green status in REST tests 2016-08-01 11:09:56 +02:00
Tanguy Leroux 737db98bd7 /_cat/shards should support wilcards for indices
closes #19634
2016-08-01 11:09:48 +02:00
Christoph Büscher 87a4995bed Merge pull request #19665 from cbuescher/missing-field-MultiMatchQuery
`multi_match` query should produce MatchNoDocs query on unknown field
2016-08-01 10:59:52 +02:00
Tanguy Leroux e642187642 Remove unnecessary indices refresh from cluster.state/20_filtering.yaml file 2016-08-01 09:22:12 +02:00
Tanguy Leroux 7d4f557aa3 Allow routing table to be filtered by index pattern
Before this commit when an index pattern is used to filter the cluster state, only indices metadata are populated and routing table is just empty. This commit aligns the behavior of the filtering of cluster state's routing table with the filtering of cluster state's metadata so that coherent data are returned for both routing table & metadata when index pattern is requested.
2016-08-01 09:22:12 +02:00
Glen Smith bf51247ec0 Add note to put mapping API docs on index creation
This commit clarifies that the put mapping API can also apply when
creating an index.

Closes #19703
2016-07-31 15:58:24 -04:00
chengpohi 8aa1eb6aa4 Fix EquivalenceIT#testRandomRanges failed with -Dtest.seed A4648847991E5C27
Set double value to double type mapping in EquivalenceIT.

Closes #19697
2016-07-31 12:49:28 -04:00
Ali Beyad ce8881513d Merge pull request #19421 from abeyad/snapshot-uuids-in-blob-names
Snapshot UUIDs in blob names
2016-07-31 00:20:37 -04:00
Ali Beyad 0f335ac873 Removes legacy format in RepositoryData 2016-07-30 18:46:58 -04:00
Nik Everett 303c9faca5 Squash o.e.rest.action.admin.cluster
In an effort to reduce the number of tiny packages we have in the
code base this moves all the files that were in subdirectories of
`org.elasticsearch.rest.action.admin.cluster` into
`org.elasticsearch.rest.action.admin.cluster`.

Also fixes line length in these packages.
2016-07-29 20:31:24 -04:00
Michael McCandless 71166c020a Merge pull request #19554 from mikemccand/negative_usable_space
Guard against negative result from FileStore.getUsableSpace when picking data path for a new shard
2016-07-29 20:26:30 -04:00
Nik Everett 6f24866902 Reindex: Only ask for _version we need it
`_reindex` only needs the `_version` if the `dest` has
`"version_type": "external"`. So it shouldn't ask for it unless it does.

`_update_by_query` and `_delete_by_query` always need the `_version`.

Closes #19135
2016-07-29 17:13:27 -04:00
Mike McCandless 59181c8a66 use mockito instead 2016-07-29 17:13:01 -04:00
Nik Everett bdebd02d8c Only write forced_refresh if we forced a refresh
Otherwise it just adds noise to the response.

Closes #19629
2016-07-29 15:00:30 -04:00
Christoph Büscher 0d7c289f4c Adressing review comments 2016-07-29 20:28:17 +02:00
Ryan Ernst efa37b8b7d Merge pull request #19667 from rjernst/client_jar_id
Build: Fix client jars for plugins/modules to have the correct artifactId
2016-07-29 11:27:28 -07:00
Alexander Lin 119026b4fb Remove isCreated and isFound from the Java API
This is cleanup work from #19566, where @nik9000 suggested trying to nuke the isCreated and isFound methods. I've combined nuking the two methods with removing UpdateHelper.Operation in favor of DocWriteResponse.Operation here.

Closes #19631.
2016-07-29 14:21:43 -04:00
Nik Everett c9790a1257 Use fewer threads when reindexing-from-remote
Reindex from remote uses the Elasticsearch client which uses apache
httpasyncclient which spins up 5 thread by default, 1 as a dispatcher
and 4 more to handle IO. This changes Reindex's usage so it only spins
up two thread - 1 dispatcher and one to handle io. It also renames the
threads to "es-client-$taskid-$thread_number". That way if we see any
thread sticking around we can trace it back to the task.
2016-07-29 14:13:10 -04:00
Christoph Büscher 4450039cf6 Try catching potential null query results and convert to MatchNoDocsQuery 2016-07-29 18:29:48 +02:00
Martijn van Groningen a91bb29585 ingest: Made the response format of the get pipeline api match with the response format of the index template api
Closes #19585
2016-07-29 17:58:30 +02:00
Alexander Reelsen 30b497622b Dependencies: Upgrade to netty 4.1.4 (#19689)
Removes our internal snapshot repo we used until 4.1.4 was released.
2016-07-29 17:53:12 +02:00
Mike McCandless 37e0e63a65 add defense to selectNewPathForShard 2016-07-29 11:51:33 -04:00
Ryan Ernst 05edb0367d Add client jars pom generation, and make transport client depend on
transport client plugin jars
2016-07-29 07:41:39 -07:00
Nik Everett ad028f3f9c Squash o.e.rest.action.admin.indices
In an effort to reduce the number of tiny packages we have in the
code base this moves all the files that were in subdirectories of
`org.elasticsearch.rest.action.admin.indices` into
`org.elasticsearch.rest.action.admin.indices`.

It also adds a `package-info.java` file explaining what the files in
the package *do*.

Also fixes line length in these packages. It makes a single non-checkstyle
change: implementing `ToXContent` on `GetIndexTemplatesResponse`. I did
this because it was the right thing to do and it fixed a line length
violation.
2016-07-29 10:08:03 -04:00