Commit Graph

11276 Commits

Author SHA1 Message Date
Robert Muir b936ec9a25 allow reflection of MXBean for file descriptor stats 2015-04-10 11:28:30 -04:00
Adrien Grand 0b413bd670 Merge pull request #10532 from jpountz/deprecate/limit_filter
Search: deprecate the limit filter.

Close #10532
2015-04-10 17:19:43 +02:00
Adrien Grand 5b3cc2f07c Search: deprecate the limit filter.
This is really a Collector instead of a filter. This commit deprecates the
`limit` filter, makes it a no-op and recommends to use the `terminate_after`
parameter instead that we introduced in the meantime.
2015-04-10 17:18:50 +02:00
Adrien Grand bad6656f02 Merge pull request #10391 from jpountz/fix/remove_flt
Queries: Remove fuzzy-like-this support.

Close #10391
2015-04-10 17:17:00 +02:00
Adrien Grand 919589b908 Queries: Remove fuzzy-like-this support.
The fuzzy-like-this query builds very expensive queries and only serves esoteric
use-cases.
2015-04-10 17:16:02 +02:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Ryan Ernst 224c43564b Tests: Add multi data path testing to static bwc tests
This randomly chooses to run bwc indexes on single or multi data paths.

closes #10519
2015-04-09 14:31:31 -07:00
Ryan Ernst 9d0764e5df Tests: increasing timeout for replicas in static bwc tests, to account
for reallySlowJenkinsWhenIndexHasLotsOfSegments
2015-04-09 13:31:08 -07:00
Michael McCandless 78612cf0a8 fix indent 2015-04-09 14:47:48 -04:00
Michael McCandless 54b702db07 Tests: don't fsync so often
Most tests don't "really" need to fsync, and this is costly (makes
tests slower, wears out our SSDs).

This change makes it uncommon to actually fsync when Lucene asks for
it.  It's just a workaround (in MockDirectoryHelper) until we can
cutover Elasticseach to use MockFileSystem like Lucene.

Closes #10516
2015-04-09 14:31:42 -04:00
Michael McCandless 2f5cbf5f9d Core: log path.data filesystem details to INFO level
This change logs total space, free space, usable free space, an
estimate of whether the IO system spins (e.g., SSD or not), the mount
point and filesystem type, on node startup.

It produces log output like this:

[2015-04-09 12:09:30,244][INFO ][env                      ] [node_t0] node data locations details:
  -> /l/es.logspins/target/J0/data/TEST-haswell-CHILD_VM=[0]-CLUSTER_SEED=[2926863498862121027]-HASH=[AFC194B1B384B]/nodes/0, free_space [260.6gb], usable_space [256.3gb], total_space [465gb], spins? [no], mount [/ (/dev/mapper/haswell--vg-root)], type [btrfs]

Closes #10502
2015-04-09 13:11:49 -04:00
Simon Willnauer d470bdbff6 Only flush for checkindex if we have uncommitted changes
Today we force a flush before check index to ensure we have an index
to check on. Yet if the index is large and the FS is slow this can have
significant impact on the index deletion performance. This commit introduces
a check if there are any uncommitted changes in order to skip the additional commit.

Closes #10505
2015-04-09 18:25:57 +02:00
javanna 5367e04fbc [TEST] add 1.5.1 index for bw comp tests 2015-04-09 17:03:35 +02:00
javanna 4608f868bf add 1.5.2 snapshot version and and make 1.5.1 a released version 2015-04-09 16:15:05 +02:00
Colin Goodheart-Smithe fcc09f62b9 Aggregations: removed aggregations from ReduceContext
ReduceContext contains the list of aggregations to reduce but these aggregations are set as null half of the time. This change makes the reduce(ReduceContext) method changed to reduce(List<InternalAggregation>, ReduceContext) and ReduceContext now only holds the BigArrays and Script services.
2015-04-09 14:58:17 +01:00
Simon Willnauer 3b41299273 Add missing hashCode method to RecoveryState#File 2015-04-09 15:07:28 +02:00
Clinton Gormley abc7de96ae Docs: Updated version annotations in master 2015-04-09 14:50:11 +02:00
David Pilato 88ee7a5dca Deprecate rivers
* In code, we mark `River`, `AbstractRiverComponent`, `RiverComponent` and `RiverName` classes as deprecated
* We log that information when a cluster is still using it
* We add this information in the plugins list as well
2015-04-09 14:29:16 +02:00
Adrien Grand fae124103a Merge pull request #10420 from jpountz/feature/numeric_resolution
Mappings: Bring back numeric_resolution.

Close #10420
2015-04-09 12:28:33 +02:00
Adrien Grand e25db222ee Merge pull request #10418 from jpountz/enhancement/speed_up_aggs_include_exclude
Aggregations: Speed up include/exclude in terms aggregations with regexps.

Close #10418
2015-04-09 12:16:37 +02:00
Adrien Grand aecd9ac515 Aggregations: Speed up include/exclude in terms aggregations with regexps.
Today we check every regular expression eagerly against every possible term.
This can be very slow if you have lots of unique terms, and even the bottleneck
if your query is selective.

This commit switches to Lucene regular expressions instead of Java (not exactly
the same syntax yet most existing regular expressions should keep working) and
uses the same logic as RegExpQuery to intersect the regular expression with the
terms dictionary. I wrote a quick benchmark (in the PR) to make sure it made
things faster and the same request that took 750ms on master now takes 74ms with
this change.

Close #7526
2015-04-09 12:12:56 +02:00
Adrien Grand 2a844fc457 Merge pull request #10411 from jpountz/fix/multi_level_breadth_first
Aggregations: Fix multi-level breadth-first aggregations.

Close #10411
2015-04-09 12:09:48 +02:00
Adrien Grand 6b16b32174 Aggregations: Fix multi-level breadth-first aggregations.
The refactoring in #9544 introduced a regression that broke multi-level
aggregations using breadth-first. This was due to sub-aggregators creating
deferred collectors before their parent aggregator and then the parent
aggregator trying to collect sub aggregators directly instead of going through
the deferred wrapper.

This commit fixes the issue but we should try to simplify all the pre/post
collection logic that we have.

Also `breadth_first` is now automatically ignored if the sub aggregators need
scores (just like we ignore `execution_mode` when the value does not make sense
like using ordinals on a script).

Close #9823
2015-04-09 12:06:08 +02:00
Simon Willnauer 17c06f06ba [TEST] issue additional flush 2015-04-09 11:56:36 +02:00
Michael McCandless c9afeccac4 merged 2015-04-09 05:01:53 -04:00
Michael McCandless 9981d69e06 Core: add trace logging for the commitTranslog=false case in InternalEngine.flush 2015-04-09 05:00:40 -04:00
Simon Willnauer 953ae63d2b [TEST] beast slow index to fail with trace logging 2015-04-09 10:47:00 +02:00
Simon Willnauer a243b3f924 [LOGGING] Use static logger name in Engine.java
To ensure subclasses like MockInternalEngine which is in a different
package (test.engine) are logging under the same logger name this commit
moves to a static logger class to determin the logger name. This way
all subclasses of engine will log under `index.engine` which also plays
nicely with `@TestLogging` where log messages sometimes disappeared since
they were enabled for the `index.engine` package but not for `test.engine`
2015-04-09 10:07:50 +02:00
Tanguy Leroux e60f61d0a9 Fix maven-resources-plugin warnings
Commit 168238dab6 declared multiple maven-resources-plugin usages instead of declaring multiple executions for the same plugin... resulting to Maven warnings.

Closes #10433
2015-04-09 10:05:10 +02:00
Ryan Ernst a5bfe332e5 Tests: increase logging for lucene commit during flush in InternalEngine 2015-04-09 00:25:05 -07:00
Ryan Ernst c821b8d3b4 Tests: remove static bwc delete index timeout, so that the slow delete
failure can repro with additional logging
2015-04-08 23:29:37 -07:00
Simon Willnauer b52d24a031 [TEST] increase logging for pulling snapshot from InternalEngine 2015-04-09 06:41:11 +02:00
Ryan Ernst b343d688c5 Tests: Increase delete timeout for static bwc tests to 1 minute 2015-04-08 21:24:36 -07:00
Spencer Alger 0cd1848e88 Indexed scripts/templates: fix yaml test indentation 2015-04-08 15:36:23 -07:00
Ryan Ernst ab395c1267 Tests: allow up to 30s to delete indexes in old static tests
When deleting an index, the tests run checkindex, which requires a
flush.  This can be very slow on virtualized hardware..
2015-04-08 13:57:22 -07:00
Ryan Ernst 178f650552 Tests: Fix static bwc replicas at 1 2015-04-08 09:02:44 -07:00
Simon Willnauer f687377e2f [ENGINE] sync translog before closing engine
If the translog is buffered we must make sure everything is synced to disk
before we rollback the writer otherwise we open a window for potential dataloss due
to stupid errors preventing the translog from being closed.
2015-04-08 17:57:33 +02:00
Ryan Ernst 6df978e76e Test: increasing replicas timeout to avoid slowness on virtualized
hardware (aka jenkins)
2015-04-08 08:44:39 -07:00
javanna acabf2d55a Cluster state REST api: print routing_nodes out only when requested through specific flag
For bacwards compatibility reasons routing_nodes were previously printed out when routing_table was requested, together with the actual routing_table. Now they are printed out only when requests through `routing_nodes` flag.

Relates to #10412
Closes #10486
2015-04-08 16:10:36 +02:00
Boaz Leskes 174d141512 Test: increase recovery concurrency in OldIndexBackwardsCompatibilityTests.testOldIndices
Also added some logging to help pin point slowness
2015-04-08 15:44:45 +02:00
Leonardo Menezes 5fd9aee16e Cluster state REST api: routing_nodes as an independent metric option
Cluster state api returns both routing_table and routing_nodes sections whenever routing_table is requested. That is pretty much the same info, just grouped differently. This commit allows to differentiate between the two. Yet, routing_table still returns both for bw comp reasons.

Closes #10352
Closes #10412
2015-04-08 15:35:11 +02:00
marko asplund 5585175173 Docs: fix typos in example JSON data
Closes #10479
2015-04-08 13:40:35 +02:00
javanna d9aebf4906 Scripting: remove deprecated methods from ScriptService
Removed the following methods from `ScriptService`, which don't require the `ScriptContext` argument:

```
public CompiledScript compile(String lang,  String script, ScriptType scriptType)

public ExecutableScript executable(String lang, String script, ScriptType scriptType, Map<String, Object> vars)

public SearchScript search(SearchLookup lookup, String lang, String script, ScriptType scriptType, @Nullable Map<String, Object> vars)
```

Also removed the ScriptContext.Standard.GENERIC_PLUGIN enum value, as it was used only for backwards compatibility.

 Plugins that make use of scripts should declare their own script contexts through `ScriptModule#registerScriptContext` and use them when compiling/executing scripts.

Closes #10476
2015-04-08 12:20:03 +02:00
javanna 7bd7ea8f13 Scripting: allow plugins to define custom operations that they use scripts for
Plugins can now define multiple operations/contexts that they use scripts for. Fine-grained settings can then be used to enable/disable scripts based on each single registered context.

Also added a new generic category called `plugin`, which will be used as a default when the context is not specified. This allows us to restore backwards compatibility for plugins on `ScriptService` by restoring the old methods that don't require the script context and making them internally use the `plugin` context, as they can only be called from plugins.

Closes #10347
Closes #10419
2015-04-08 11:57:00 +02:00
blackorzar 5d4a4fc3ff Docs: Include groovy community
This change includes the a new community: groovy to point to the official client.

Closes #10470
2015-04-08 11:46:05 +02:00
javanna df875707ec Indexed scripts/templates: return response body when script is not found
Align get indexed scripts and get search template apis to our get api, which returns a response body when the document is not found, with a found boolean flag. Also, return metadata info all the time too.

Closes #7325
Closes #10396
2015-04-08 10:44:11 +02:00
Adrien Grand 85e4c21b0a Tests: Disable the query cache for parent/child unit tests. 2015-04-08 09:58:59 +02:00
Adrien Grand 717dbb0599 Tests: Fix bug in SimpleSortTests.
This tests adds a mappings with {"fielddata": {"format": "doc_values"}} but the
default mapping has {"doc_values": false} so when the document mapper parsing
logic merges both we have {"doc_values": false,"fielddata": {"format": "doc_values"}}
and {"doc_values": false} wins, so the test is not using doc values while it
thought it would.
2015-04-08 09:45:50 +02:00