11258 Commits

Author SHA1 Message Date
Clinton Gormley
abc7de96ae Docs: Updated version annotations in master 2015-04-09 14:50:11 +02:00
David Pilato
88ee7a5dca Deprecate rivers
* In code, we mark `River`, `AbstractRiverComponent`, `RiverComponent` and `RiverName` classes as deprecated
* We log that information when a cluster is still using it
* We add this information in the plugins list as well
2015-04-09 14:29:16 +02:00
Adrien Grand
fae124103a Merge pull request #10420 from jpountz/feature/numeric_resolution
Mappings: Bring back numeric_resolution.

Close #10420
2015-04-09 12:28:33 +02:00
Adrien Grand
e25db222ee Merge pull request #10418 from jpountz/enhancement/speed_up_aggs_include_exclude
Aggregations: Speed up include/exclude in terms aggregations with regexps.

Close #10418
2015-04-09 12:16:37 +02:00
Adrien Grand
aecd9ac515 Aggregations: Speed up include/exclude in terms aggregations with regexps.
Today we check every regular expression eagerly against every possible term.
This can be very slow if you have lots of unique terms, and even the bottleneck
if your query is selective.

This commit switches to Lucene regular expressions instead of Java (not exactly
the same syntax yet most existing regular expressions should keep working) and
uses the same logic as RegExpQuery to intersect the regular expression with the
terms dictionary. I wrote a quick benchmark (in the PR) to make sure it made
things faster and the same request that took 750ms on master now takes 74ms with
this change.

Close #7526
2015-04-09 12:12:56 +02:00
Adrien Grand
2a844fc457 Merge pull request #10411 from jpountz/fix/multi_level_breadth_first
Aggregations: Fix multi-level breadth-first aggregations.

Close #10411
2015-04-09 12:09:48 +02:00
Adrien Grand
6b16b32174 Aggregations: Fix multi-level breadth-first aggregations.
The refactoring in #9544 introduced a regression that broke multi-level
aggregations using breadth-first. This was due to sub-aggregators creating
deferred collectors before their parent aggregator and then the parent
aggregator trying to collect sub aggregators directly instead of going through
the deferred wrapper.

This commit fixes the issue but we should try to simplify all the pre/post
collection logic that we have.

Also `breadth_first` is now automatically ignored if the sub aggregators need
scores (just like we ignore `execution_mode` when the value does not make sense
like using ordinals on a script).

Close #9823
2015-04-09 12:06:08 +02:00
Simon Willnauer
17c06f06ba [TEST] issue additional flush 2015-04-09 11:56:36 +02:00
Michael McCandless
c9afeccac4 merged 2015-04-09 05:01:53 -04:00
Michael McCandless
9981d69e06 Core: add trace logging for the commitTranslog=false case in InternalEngine.flush 2015-04-09 05:00:40 -04:00
Simon Willnauer
953ae63d2b [TEST] beast slow index to fail with trace logging 2015-04-09 10:47:00 +02:00
Simon Willnauer
a243b3f924 [LOGGING] Use static logger name in Engine.java
To ensure subclasses like MockInternalEngine which is in a different
package (test.engine) are logging under the same logger name this commit
moves to a static logger class to determin the logger name. This way
all subclasses of engine will log under `index.engine` which also plays
nicely with `@TestLogging` where log messages sometimes disappeared since
they were enabled for the `index.engine` package but not for `test.engine`
2015-04-09 10:07:50 +02:00
Tanguy Leroux
e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6f5cb081c1e919c0136c13a3c837b72 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Ryan Ernst
a5bfe332e5 Tests: increase logging for lucene commit during flush in InternalEngine 2015-04-09 00:25:05 -07:00
Ryan Ernst
c821b8d3b4 Tests: remove static bwc delete index timeout, so that the slow delete
failure can repro with additional logging
2015-04-08 23:29:37 -07:00
Simon Willnauer
b52d24a031 [TEST] increase logging for pulling snapshot from InternalEngine 2015-04-09 06:41:11 +02:00
Ryan Ernst
b343d688c5 Tests: Increase delete timeout for static bwc tests to 1 minute 2015-04-08 21:24:36 -07:00
Spencer Alger
0cd1848e88 Indexed scripts/templates: fix yaml test indentation 2015-04-08 15:36:23 -07:00
Ryan Ernst
ab395c1267 Tests: allow up to 30s to delete indexes in old static tests
When deleting an index, the tests run checkindex, which requires a
flush.  This can be very slow on virtualized hardware..
2015-04-08 13:57:22 -07:00
Ryan Ernst
178f650552 Tests: Fix static bwc replicas at 1 2015-04-08 09:02:44 -07:00
Simon Willnauer
f687377e2f [ENGINE] sync translog before closing engine
If the translog is buffered we must make sure everything is synced to disk
before we rollback the writer otherwise we open a window for potential dataloss due
to stupid errors preventing the translog from being closed.
2015-04-08 17:57:33 +02:00
Ryan Ernst
6df978e76e Test: increasing replicas timeout to avoid slowness on virtualized
hardware (aka jenkins)
2015-04-08 08:44:39 -07:00
javanna
acabf2d55a Cluster state REST api: print routing_nodes out only when requested through specific flag
For bacwards compatibility reasons routing_nodes were previously printed out when routing_table was requested, together with the actual routing_table. Now they are printed out only when requests through `routing_nodes` flag.

Relates to #10412
Closes #10486
2015-04-08 16:10:36 +02:00
Boaz Leskes
174d141512 Test: increase recovery concurrency in OldIndexBackwardsCompatibilityTests.testOldIndices
Also added some logging to help pin point slowness
2015-04-08 15:44:45 +02:00
Leonardo Menezes
5fd9aee16e Cluster state REST api: routing_nodes as an independent metric option
Cluster state api returns both routing_table and routing_nodes sections whenever routing_table is requested. That is pretty much the same info, just grouped differently. This commit allows to differentiate between the two. Yet, routing_table still returns both for bw comp reasons.

Closes #10352
Closes #10412
2015-04-08 15:35:11 +02:00
marko asplund
5585175173 Docs: fix typos in example JSON data
Closes #10479
2015-04-08 13:40:35 +02:00
javanna
d9aebf4906 Scripting: remove deprecated methods from ScriptService
Removed the following methods from `ScriptService`, which don't require the `ScriptContext` argument:

```
public CompiledScript compile(String lang,  String script, ScriptType scriptType)

public ExecutableScript executable(String lang, String script, ScriptType scriptType, Map<String, Object> vars)

public SearchScript search(SearchLookup lookup, String lang, String script, ScriptType scriptType, @Nullable Map<String, Object> vars)
```

Also removed the ScriptContext.Standard.GENERIC_PLUGIN enum value, as it was used only for backwards compatibility.

 Plugins that make use of scripts should declare their own script contexts through `ScriptModule#registerScriptContext` and use them when compiling/executing scripts.

Closes #10476
2015-04-08 12:20:03 +02:00
javanna
7bd7ea8f13 Scripting: allow plugins to define custom operations that they use scripts for
Plugins can now define multiple operations/contexts that they use scripts for. Fine-grained settings can then be used to enable/disable scripts based on each single registered context.

Also added a new generic category called `plugin`, which will be used as a default when the context is not specified. This allows us to restore backwards compatibility for plugins on `ScriptService` by restoring the old methods that don't require the script context and making them internally use the `plugin` context, as they can only be called from plugins.

Closes #10347
Closes #10419
2015-04-08 11:57:00 +02:00
blackorzar
5d4a4fc3ff Docs: Include groovy community
This change includes the a new community: groovy to point to the official client.

Closes #10470
2015-04-08 11:46:05 +02:00
javanna
df875707ec Indexed scripts/templates: return response body when script is not found
Align get indexed scripts and get search template apis to our get api, which returns a response body when the document is not found, with a found boolean flag. Also, return metadata info all the time too.

Closes #7325
Closes #10396
2015-04-08 10:44:11 +02:00
Adrien Grand
85e4c21b0a Tests: Disable the query cache for parent/child unit tests. 2015-04-08 09:58:59 +02:00
Adrien Grand
717dbb0599 Tests: Fix bug in SimpleSortTests.
This tests adds a mappings with {"fielddata": {"format": "doc_values"}} but the
default mapping has {"doc_values": false} so when the document mapper parsing
logic merges both we have {"doc_values": false,"fielddata": {"format": "doc_values"}}
and {"doc_values": false} wins, so the test is not using doc values while it
thought it would.
2015-04-08 09:45:50 +02:00
Boaz Leskes
80e86e5719 Transport: shortcut local execution
In several places in the code we need to notify a node it needs to do something (typically the master). When that node is the local node, we have an optimization in serveral places that runs the execution code immediately instead of sending the request through the wire to itself. This is a shame as we need to implement the same pattern again and again. On top of that we may forget (see note bellow) to do so and we might have to write some craft if the code need to run under another thread pool.

This commit folds the optimization in the TrasnportService, shortcutting wire serliazition if the target node is local.

Note: this was discovered by #10247 which tries to import a dangling index quickly after the cluster forms. When sending an import dangling request to master, the code didn't take into account that fact that the local node may master. If this happens quickly enough, one would get a NodeNotConnected exception causing the dangling indices not to be imported. This will succeed after 10s where InternalClusterService.ReconnectToNodes runs and actively connects the local node to itself (which is not needed), potentially after another cluster state update.

Closes #10350
2015-04-08 09:10:47 +02:00
Adrien Grand
a24d898b3a Merge pull request #10468 from jpountz/upgrade/5.1.0-snapshot-1671894
Upgrade to lucene-5.1.0-snapshot-1671894.
2015-04-07 20:07:18 +02:00
Adrien Grand
89b97c1c8d Upgrade to lucene-5.1.0-snapshot-1671894.
This includes a fix for LUCENE-6406.
2015-04-07 20:04:54 +02:00
jaymode
1edf883bae Netty: make exceptionCaught method protected
The exceptionCaught method had default access, which imposes a requirement
for subclasses that need to override this method to be in a specific package. This
change simply makes the method protected, which removes the package requirement.
2015-04-07 13:09:47 -04:00
Adrien Grand
22472b7b57 Merge pull request #10389 from jpountz/enhancement/per_segment_apis
Internal: Make APIs work per-segment like Lucene's Collector.

Close #10389
2015-04-07 18:16:58 +02:00
Stefan
f7e6d79569 [DOCS] update versions in java api module.xml
The filenames are updated to fit to the current elasticsearch version
2015-04-07 18:10:53 +02:00
Adrien Grand
967a4e2a86 Internal: Make APIs work per-segment like Lucene's Collector.
We still have a lot of APIs that use setNextReader in order to change the
current segment that should be considered. This commit moves such APIs to
getLeafXXX() instead to be more in-line with Lucene 5's collector API.

I also renamed setDocId to setDocument to be more in-line with the doc values
APIs.

Close #10389
2015-04-07 17:56:01 +02:00
Adrien Grand
884c40abac Merge pull request #9165 from jpountz/enhancement/reduce_search_pool_size
Internal: reduce the size of the search thread pool.
2015-04-07 17:49:50 +02:00
Ryan Ernst
e575c4ce53 Tests: Add --all flag to create-bwc script to regenerate all indexes
It is currently a pain to regenerate all the bwc indexes when making
a change to the generation script.  This makes it a single command.
2015-04-07 08:32:34 -07:00
Adrien Grand
eb52febf0c Merge pull request #10393 from jpountz/fix/112_snapshot
Internal: Fix snapshot status of version 1.1.2.
2015-04-07 17:28:40 +02:00
Shay Banon
af19ec3006 Remove content thread safe from REST layer
there is no need for this anymore, for some time, since in netty now we rely on copying over the buffer and reusing it
closes #10429
2015-04-07 18:11:48 +03:00
Adrien Grand
ca6f6d4283 Tests: work-around for LUCENE-6406. 2015-04-07 16:57:20 +02:00
Simon Willnauer
c2f9987a15 [TEST] expect simpletext CFS format in test 2015-04-07 15:36:45 +02:00
Simon Willnauer
f1abf0aa10 Update Lucene version constants for 1.4.5 and above to 4.10.4 2015-04-07 15:26:58 +02:00
Isabel Drost-Fromm
60bb65c4d9 Docs: Note on shard vs. index level doc frequencies.
Relates to #10154 and #10150

Adds link to additional information on how document frequencies are treated across shards to the cutoff_frequency parameter documentation.

Closes #10451
2015-04-07 14:28:01 +02:00
joelbourbon
3c52bc1098 Docs: Missing 1 escape character in example
Closes #10446
2015-04-07 14:10:17 +02:00
Adrien Grand
37f3351b58 Tests: Speed up CopyOnWriteHash(Map|Set)Tests
These tests create artificial hash collisions in order to make sure that they
can be resolved correctly. But this also makes the tests very slow if there
are too many collisions because insertions/deletions become linear in such
cases. The tests have been modified to not do too many iterations when
collisions are likely.

Close #10442
2015-04-07 13:34:08 +02:00
Robert Muir
1d56502e52 Upgrade to Lucene 5.1 r1671277.
Closes #10435.

Squashed commit of the following:

commit aa1935c790b2731fc2bbc7de6142b09e3fe8bd4a
Author: Ryan Ernst <ryan@iernst.net>
Date:   Mon Apr 6 13:44:40 2015 -0700

    fix index lookup

commit bb6373595ff62ffc56fdf0cba3ac9c0ebe679946
Merge: 916962b eb3a170
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 14:24:38 2015 -0400

    Merge branch 'lucene_r1671277' of github.com:elasticsearch/elasticsearch into lucene_r1671277

commit 916962b82d192a53add471b4cc4a1396bc30eb0e
Merge: 197b3a2 21f72fe
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Apr 6 07:09:41 2015 -0400

    Merge branch 'master' into lucene_r1671277

commit eb3a1703f7932ddd0cf3e83bec0e86131d255407
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 11:06:03 2015 -0700

    re-enable index lookup tests

commit 80d65d5eab39062dd8364687da74ddbb87ebcb76
Author: Ryan Ernst <ryan@iernst.net>
Date:   Sat Apr 4 10:39:52 2015 -0700

    update pom to point to new snapshot repo

commit 197b3a21ac2c2d70c9f740fe53e58632a22d1aad
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:51:22 2015 -0400

    fix postingsenum usage

commit 0e2b7a00cd07d068f755c51185ac521aa1eb0326
Author: Robert Muir <rmuir@apache.org>
Date:   Sat Apr 4 12:21:23 2015 -0400

    upgrade to lucene r1671277 (have not yet run tests or looked at postings changes)
2015-04-07 06:09:35 -04:00