Commit Graph

107 Commits

Author SHA1 Message Date
Shay Banon 341e980c52 [TEST] Use the correct renamed setting
and make the default value setting private
2015-06-18 20:40:07 +02:00
Simon Willnauer 1b2a3d0af6 Add @Repeat to forbidden APIs
@Repeat should not be committed just like @Seed.
Use -Pdev to run annotated methods.
2015-06-18 20:34:02 +02:00
Shay Banon 6d2ca116b9 [TEST] assertBusy on hasUnassigned
on fast machines, node leave might not move shards to unassigned right away, wait for it
2015-06-18 20:30:25 +02:00
Shay Banon 256f3d21ce Reset registeredNextDelaySetting on reroute
Need to reset the registered setting in order to make sure the nex round will capture the right delay interval

also randomize setting and name the setting properly

closes #11759
2015-06-18 20:20:49 +02:00
Jack Conradson 917aeb7278 Scripting: Allow executable expression scripts for aggregations
Added several classes to support expressions being used for numerical
calculations in aggregations.  Expressions will still not compile
when used with mapping and update script contexts.

Closes #11596
Closes #11689
2015-06-18 10:57:38 -07:00
Simon Willnauer 87946ba3b2 Allow security rule for advanced SSL configutation
if plugins need to install SSL factories etc. we have to allow
to `setFactory` in the security policy.
2015-06-18 16:50:35 +02:00
Shay Banon 792a545633 Optional Delayed Allocation on Node leave
Allow to set delayed allocation timeout on unassigned shards when a node leaves the cluster. This allows to wait for the node to come back for a specific period in order to try and assign the shards back to it to reduce shards movements and unnecessary relocations.

The setting is an index level setting under `index.unassigned.node_left.delayed_timeout` and defaults to 0 (== no delayed allocation). We might want to change the default, but lets do it in a different change to come up with the best value for it. The setting can be updated dynamically.

When shards are delayed, a log message with "info" level will notify how many shards are being delayed.

An implementation note, we really only need to care about delaying allocation on unassigned replica shards. If the primary shard is unassigned, anyhow we are going to wait for a copy of it, so really the only case to delay allocation is for replicas.

close #11712
2015-06-18 16:06:02 +02:00
Simon Willnauer 30cc984565 [TEST] fix TermsAggregationSearchBenchmark 2015-06-18 15:06:49 +02:00
Simon Willnauer 5808102d19 Upgrade AWS dependency to 1.10.0
This also apgrades JodaTime to 2.8.0.
2015-06-18 14:04:38 +02:00
Simon Willnauer 85dccdb8ab Use UTC as timeszone for index creation date 2015-06-18 11:15:40 +02:00
Simon Willnauer 2a63249441 Add DateTime ctors without timezone to forbidden APIs
Using DateTime with default timezone is asking for trouble and should
be added to forbidden APIs
2015-06-18 10:43:45 +02:00
Isabel Drost-Fromm 2fa56899d6 Merge pull request #11696 from MaineC/feature/simple-query-string-addboost
Add support for query boost to SimpleQueryStringBuilder.

Relates to #10217
2015-06-18 10:10:57 +02:00
Shay Banon cc97a4159e remove 1.7 version check 2015-06-18 09:34:39 +02:00
David Pilato 6a466b9a00 Replace Iterators#emptyIterator by JDK one
`com.google.common.collect.Iterators#emptyIterator()` is marked as deprecated and will be removed in May 2016. We should use JDK7 `Collections#emptyIterator()`
2015-06-18 08:47:31 +02:00
Igor Motov b2000a48a4 Snapshot/Restore: Add snapshot name validation logic to all snapshot operation
Make sure snapshot name validation occurs earlier in all snapshot operations.
2015-06-17 19:13:35 -04:00
Igor Motov 36da42c93b Show human readable Elasticsearch version that created index and date when index was created
By setting human parameter to true, it's now possible to see human readable versions of Elasticsearch that created and updated the index as well as the date when the index was created.

Closes #11484
2015-06-17 18:52:01 -04:00
Alex Ksikes 3f6dae1a73 More Like This: renamed `ignore_like` to `unlike`
This changes the parameter name `ignore_like` to the more user friendly name
`unlike`. This later feature generates a query from the terms in `A` but not
from the terms in `B`. This translates to a result set which is like `A` but
unlike `B`. We could have further negatively boosted any documents that have
some `B`, but these documents already do not receive any contribution from
having `B`, and would therefore negatively compete with documents having `A`.

Closes #11117
2015-06-17 17:18:50 -05:00
Simon Willnauer 90f9b5f60d Allow to opt-out of loading packaged REST tests
this is really just a workaround for plugins to run their own
REST tests instead of the core ones. It opts out of the rest test
loading from the core jar file and tries to load from the classpath instead.
Eventually we need to fix this infrastrucutre to move away from parameterized
tests such that subclasses can override behavior.

Closes #11721
2015-06-17 20:37:59 +02:00
Clinton Gormley 05d512f417 Packaging: Add LICENSE, NOTICE, and sha1 files and tests for all core dependencies
Added a licenses/ directory to core which contains a sha1 file for each JAR
dependency, and one or more LICENSE files and one NOTICE file for each
project.

Also adds dev-tools/src/main/resources/license-check/check_license_and_sha.pl
which checks that the licenses/ dir is up to date during a mvn verify,
and which can be used to update the sha1 files when upgrading dependencies.

Closes #2794
Closes #10684
Closes #11705
2015-06-17 18:06:00 +02:00
Michael McCandless 6fd893e10d Balance new shard allocations more evenly across multiple path.data
This change adds a simplistic heuristic to try to balance new shard
allocations across multiple data paths on one node so that e.g. if
there are two path.data and both have roughly the same free space, if
10 shards are suddenly allocated, we will put 5 on one path and 5 on
the other (vs 10 on a single path today).

Closes #11185

Closes #11122
2015-06-17 10:28:38 -04:00
Simon Willnauer 0434ecfb03 Merge pull request #11464 from nirmalc/nodes-preference
Search `preference` based on node specification
2015-06-17 12:33:51 +02:00
Isabel Drost-Fromm 5e9106ae5e Fixing review comments. 2015-06-17 11:18:32 +02:00
Isabel Drost-Fromm 907cdc61f7 Add boost to SimpleQueryStringBuilder.
As per discussion in #11274 this adds support for query boosting
to the SimpleQueryStringQuery.
2015-06-17 10:49:52 +02:00
Adrien Grand 17fac6dad5 Merge pull request #11568 from jpountz/remove/rivers
Rivers removal.
2015-06-17 08:20:48 +02:00
Adrien Grand 6e15e08f5e Merge pull request #11693 from jpountz/fix/simplify_timestamp_dv
Simplify doc values handling for _timestamp.
2015-06-17 08:19:33 +02:00
Nirmal Chidambaram 72a9d34eb8 5925 - Allow node specification in preference
-Allow node selector api's with new preference
ONLY_NODES ( selector apis like https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster.html)

-Update documentation
2015-06-16 11:49:12 -05:00
Adrien Grand 891689ed1e Merge pull request #11697 from jpountz/fix/synchronous_fielddata_removal
Reclaim memory from fielddata synchronously when an explicit clear is performed.
2015-06-16 17:36:29 +02:00
Shay Banon 4e0d65453f Add Unassigned meta data
Unassigned meta includes additional information as to why a shard is unassigned, this is especially handy when a shard moves to unassigned due to node leaving or shard failure.

The additional data is provided as part of the cluster state, and as part of `_cat/shards` API.

The additional meta includes the timestamp that the shard has moved to unassigned, allowing us in the future to build functionality such as delay allocation due to node leaving until a copy of the shard is found.
closes #11653
2015-06-16 17:01:48 +02:00
Tanguy Leroux 3b4ca8e399 Create PID_DIR in init.d script
Since the /var/run/elasticsearch directory is cleaned when the operating system starts, the init.d script must ensure that the PID_DIR is correctly created.

Closes #11594
2015-06-16 16:16:29 +02:00
Adrien Grand 28e88e5bc0 Reclaim memory from fielddata synchronously when an explicit clear is performed.
The cache we are using for fielddata reclaims memory lazily/asynchronously. While this
helps with throughput this is an issue when a clear operation is issued manually since
memory is not reclaimed immediately. Since our clear methods already perform in linear
time anyway, this commit changes the fielddata cache to reclaim memory synchronously
when a clear command is issued manually. However, it remains lazy in case cache entries
are invalidated because segments or readers are closed, which is important since such
events happen all the time.

Close #11695
2015-06-16 12:08:39 +02:00
javanna 0a526be344 Query DSL: fix support for _name in some queries
Some of our Java api builders had wrong logic when it comes to serializing the query in json format, resulting in missing fields like _name. Also, regexp parser was ignoring the _name field.

Closes #11694
2015-06-16 12:04:51 +02:00
Adrien Grand 2bb923a06d Simplify doc values handling for _timestamp.
`_timestamp` uses NumericDocValues instead of SortedNumericDocValues like other
numeric fields since it is guaranteed to be single-valued. However, we don't
need a different fielddata impl for it since DocValues.getSortedNumeric already
falls back to NUMERIC doc values if SORTED_NUMERIC are not available.
2015-06-16 10:52:06 +02:00
Adrien Grand cdd13253de Tests: Add test case from #11692 2015-06-16 10:34:51 +02:00
David Pilato 49cd9f3be4 Merge pull request #11676 from dadoonet/maven/clean-pom
[maven] clean pom.xml

In Maven parent project, in dependency management, we should only declare which versions of 3rd party jars we want to use but not force any scope.
It makes then more obvious in modules what is exactly the scope of any dependency.

For example, one could imagine importing `jimfs` as a `compile` dependency in another module/plugin with:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
</dependency>
```

But it won't work as expected as the default maven `scope` should be `compile` but here it's `test` as defined in the parent project.

So, if you want to use this lib for tests, you should simply define:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
   <scope>test</scope>
</dependency>
```

We also remove `maven-s3-wagon` from gce plugin as it's not used.
2015-06-16 09:12:03 +02:00
Ryan Ernst 98a5dbb10a Merge pull request #11686 from rjernst/remove/smart-object-wrapper
Remove SmartNameObjectMapper
2015-06-16 00:00:03 -07:00
Ryan Ernst a23c5b8bbc Make conditional clearer 2015-06-15 23:59:09 -07:00
Ryan Ernst 6e75f9fc93 fix field type tests 2015-06-15 17:03:32 -07:00
Ryan Ernst 7ab0009e41 Merge pull request #11644 from rjernst/refactor/field-type-equality
Add equals/hashcode to fieldtypes
2015-06-15 15:42:53 -07:00
Ryan Ernst 5601cea083 Mappings: Remove SmartNameObjectMapper
This was previously a container for an ObjectMapper, along with the
DocumentMapper that ObjectMapper came from. However, there was
only one use of needing the associated DocumentMapper, and that
wasn't actually used.
2015-06-15 15:40:01 -07:00
Boaz Leskes 2fb8df0985 Test: NetworkPartitionTests should make sure MockTransportService is used. 2015-06-15 23:40:52 +02:00
Ryan Ernst a08f51e13b Address PR comments 2015-06-15 10:55:37 -07:00
David Pilato bd5c7d0ea2 [maven] clean pom.xml
In Maven parent project, in dependency management, we should only declare which versions of 3rd party jars we want to use but not force any scope.
It makes then more obvious in modules what is exactly the scope of any dependency.

For example, one could imagine importing `jimfs` as a `compile` dependency in another module/plugin with:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
</dependency>
```

But it won't work as expected as the default maven `scope` should be `compile` but here it's `test` as defined in the parent project.

So, if you want to use this lib for tests, you should simply define:

```xml
<dependency>
   <groupId>com.google.jimfs</groupId>
   <artifactId>jimfs</artifactId>
   <scope>test</scope>
</dependency>
```

We also remove `maven-s3-wagon` from gce plugin as it's not used.
2015-06-15 17:08:15 +02:00
Adrien Grand e817393c8d Merge pull request #11669 from jpountz/remove/esoteric_fielddata_formats
Remove non-default fielddata formats.
2015-06-15 16:19:48 +02:00
Britta Weber 997550f5a2 Merge pull request #11595 from brwe/netowrk-partition-test
[TEST] remove node from nodes list if disruption is removed from node
2015-06-15 16:09:01 +02:00
Adrien Grand 14c9c239bc Remove non-default fielddata formats.
Now that doc values are the default for fielddata, specialized in-memory
formats are becoming an esoteric option. This commit removes such formats:
 - `fst` on string fields,
 - `compressed` on geo points.

I also removed documentation and tests that the fielddata cache is shared if
you change the format, since this is only true for in-memory fielddata formats
(given that for doc values, the caching is done directly in Lucene).
2015-06-15 14:05:23 +02:00
Adrien Grand 36cab80cc8 Remove SimpleLuceneTests.testAddDocAfterPrepareCommit.
It is unnecessary to test features we are not using.
2015-06-15 11:26:28 +02:00
Adrien Grand ff279275b3 Upgrade to Lucene 5.2.1. 2015-06-15 10:51:05 +02:00
Simon Willnauer 9210aeacea Restore interrupt status when InterruptedException is caught 2015-06-15 09:40:28 +02:00
Shay Banon 87f0d09ace Add version 1.7.0
it was added in 1.x, but not in master
2015-06-14 23:43:45 +02:00
Simon Willnauer 02e540384d [TEST] Use NoMergePolicy in EngineTest#testVerboseSegments
These tests rely on that there are no merges but since MP is now
random this can break the test.
2015-06-14 12:20:28 +02:00