Commit Graph

11711 Commits

Author SHA1 Message Date
Adrien Grand c2d3478a47 Merge pull request #10727 from jpountz/upgrade/lucene-5.2-snapshot-1675363
Upgrade to lucene-5.2-snapshot-1675363.
2015-04-22 17:26:05 +02:00
Adrien Grand dd679a3a38 Upgrade to lucene-5.2-snapshot-1675363.
This snapshot contains in particular LUCENE-6446 (refactored explanation API)
and LUCENE-6448 (better equals/hashcode for filters).
2015-04-22 17:23:17 +02:00
Robert Muir d7eba0df59 Merge pull request #10726 from rmuir/nuke_nested_classes
Don't use nested classes for Rest tests.
2015-04-22 11:05:10 -04:00
Colin Goodheart-Smithe dcf91ff02f Temporarily disabled gap policy randomisation in MovAvgTests 2015-04-22 16:01:23 +01:00
Jun Ohtani 7f08ab301d Fix RestSearchScrollAction to also accept source parameter instead of the request body 2015-04-22 23:55:15 +09:00
Robert Muir ec07a41514 Don't use nested classes for Rest tests.
Nested classes have the advantage of organizing the hack in a way
where its easy to see what is happening overall, but they have
the downside of class names with $ in them.

These names work just fine, but can require shell escaping
or other annoyances, which is the last thing you want if
you are trying to just reproduce.
2015-04-22 10:40:30 -04:00
Boaz Leskes 1118965aa5 Test: move to ensure yellow in indices.stats/12_level.yaml 2015-04-22 16:10:28 +02:00
Boaz Leskes 91ff3f6963 Test: add ensure green to indices.stats/12_level.yaml
Also changed the stash logger to not log all stashed values under debug (it does trace now) but do dump the stash content upon failure (under info as a XContent)
2015-04-22 15:56:45 +02:00
Colin Goodheart-Smithe 77e2f644e3 Derivative tests for gaps in metrics 2015-04-22 14:50:49 +01:00
Simon Willnauer 6f1b398b33 Fix RestAnalyzeAction to also accept source parameter instead of the request body 2015-04-22 15:30:04 +02:00
Robert Muir 9406d83441 fix typo 2015-04-22 08:46:14 -04:00
markharwood 05c3d05cff Query enhancement: single value numeric queries shouldn't be handled by NumericRangeQuery and should use a TermQuery wrapped in a ConstantScoreQuery instead.
Equally, single value filters should use TermFilters rather than NumericRangeFilters

Closes #10646
2015-04-22 12:10:51 +01:00
Clinton Gormley a60571c597 Docs: Removed some unused callout from the scroll docs 2015-04-22 12:49:06 +02:00
Michael McCandless 4d2bc25b1f Make NodeEnvironment.getFileStore a bit more defensive
This improves the NodeEnvironment code that walks through all mount
points looking for the one matching the file store for a specified
path, to make it a bit more defensive.  We currently rely on this to
log the correct file system type of the path.data paths.

Closes #10696
2015-04-22 06:24:42 -04:00
Isabel Drost-Fromm f857f9e47c Merge pull request #10526 from MaineC/bug-fix/10397-update-templates-not-recompiled
Closes #10397: Fix updating indexed search templates.
2015-04-22 11:15:12 +02:00
Colin Goodheart-Smithe f6934e0410 unit test for derivative of metric agg with gaps 2015-04-22 10:06:22 +01:00
Isabel Drost-Fromm a9d540a859 Fix updating templates.
Closes #10397

When putting new templates to an index they are added to the cache
of compiled templates as a side effect of the validate method. When
updating templates they are also validated but the scripts that are
already in the cache never get updated.

As per comments on PR #10526 adding more tests around updating scripts
and templates.
2015-04-22 11:00:13 +02:00
Jun Ohtani 0955c127c0 Rest: Add json in request body to scroll, clear scroll, and analyze API
Change analyze.asciidoc and scroll.asciidoc
Add json support to Analyze and Scroll, and clear scrollAPI
Add rest-api-spec/test

Closes #5866
2015-04-22 17:53:20 +09:00
Simon Willnauer 18ede79ed5 [TEST] Make StoreTest extraFS proof 2015-04-22 09:36:01 +02:00
Boaz Leskes a1ba339517 Stats: add CommitStats to supply information about the current commit point
Extends ShardStats with commit specific information. We currently expose commit id, generation and the user data map.

The information is also retrievable via the Rest API by using `GET _stats?level=shards`

Closes #10687
2015-04-22 09:33:14 +02:00
Simon Willnauer 1ae87ca4a2 Fix download link in README.textile 2015-04-22 09:24:11 +02:00
Simon Willnauer b53e8fa986 [TEST] make sure extraFS files are not in the metadata 2015-04-22 09:20:45 +02:00
Boaz Leskes 65eb4210b1 Test: mute testConcurrentDynamicMapping 2015-04-22 09:12:44 +02:00
Robert Muir 270cb9f349 enable securitymanager 2015-04-22 03:04:50 -04:00
Robert Muir 730314fec1 Execute tests with $JAVA_HOME. 2015-04-21 20:27:27 -04:00
Robert Muir b7ab378996 Merge pull request #10709 from rmuir/remove_this_insanity
Fix JVM isolation in tests.
2015-04-21 19:14:25 -04:00
Robert Muir 69718916df actually remove this line rather than comment it out. tsts pass 2015-04-21 19:04:56 -04:00
Robert Muir 9d6b1382e7 Fix JVM isolation in tests.
Currently security manager would allow for one JVM to muck
with the files (read, write, AND delete) of another JVM.

This is unnecessary.
2015-04-21 19:02:14 -04:00
Robert Muir 22b23f6e44 unbreak the COMPILE 2015-04-21 18:43:14 -04:00
Ryan Ernst 25410f880c Mappings: Simplified mapper lookups
We no longer support overriding field index names, but the lookup
data structures still optimize for this use case. This complicates
the work for #8871.  Instead, we can use a lookup structure
by making the legacy case slower.

This change simplifies the field mappers lookup to only
store a single map, keyed by the field's full name. It also
changes a lot of tests to decrease the uses of the older api
(looking up by index name where the index name is different
than the field name).

closes #10705
2015-04-21 15:00:28 -07:00
Nicholas Knize 453217fd7a [GEO] Prioritize tree_level and precision parameters over default distance_error_pct
If a user explicitly defined the tree_level or precision parameter in a geo_shape mapping their specification was always overridden by the default_error_pct parameter (even though our docs say this parameter is a 'hint'). This lead to unexpected accuracy problems in the results of a geo_shape filter. (example provided in issue #9691)

This simple patch fixes the unexpected behavior by setting the default distance_error_pct parameter to zero when the tree_level or precision parameters are provided by the user. Under the covers the quadtree will now use the tree level defined by the user. The docs will be updated to alert the user to exercise caution with these parameters.  Specifying a precision of "1m" for an index using large complex shapes can quickly lead to OOM issues.

closes #9691
2015-04-21 14:42:10 -05:00
Nicholas Knize 0205fc7ac2 [GEO] Fix OOM for high precision exotic shapes
This is currently submitted as a patch in LUCENE-6422. It removes unnecessary transient memory usage for QuadPrefixTree and, for 1.6.0+ shape indexes adds a new compact bit encoded representation for each quadcell. This is the heart of numerous false positive matches, OOM exceptions, and all around poor shape indexing performance. The compact bit representation will also allows for encoding 3D shapes in future enhancements.
2015-04-21 14:02:27 -05:00
Simon Willnauer 982da25f6e [TEST] Share code for mock engines
Today we have duplicated logic in the MockInternal and MockShadowEngine
since they need to subclass the actual engine. This commit shares the most of
the code making it easier to add mock engines in the future.
2015-04-21 20:15:04 +02:00
Colin Goodheart-Smithe 89d424e074 Derivative can now access multi-value metric aggregations 2015-04-21 16:00:02 +01:00
Colin Goodheart-Smithe bd28c9c44e Documentation for the max_bucket reducer 2015-04-21 15:06:20 +01:00
Colin Goodheart-Smithe be647a89d3 Documentation for the derivative reducer 2015-04-21 15:06:20 +01:00
Colin Goodheart-Smithe 0f4b7f3b5c Added section for reducer aggregations in the main aggregation docs page 2015-04-21 15:06:19 +01:00
Adrien Grand daa25d1b20 Merge pull request #10531 from jpountz/upgrade/lucene-5.1-queries
Search: Replace deprecated filters with equivalent queries.
2015-04-21 15:40:21 +02:00
Adrien Grand d7abb12100 Replace deprecated filters with equivalent queries.
In Lucene 5.1 lots of filters got deprecated in favour of equivalent queries.
Additionally, random-access to filters is now replaced with approximations on
scorers. This commit
 - replaces the deprecated NumericRangeFilter, PrefixFilter, TermFilter and
   TermsFilter with NumericRangeQuery, PrefixQuery, TermQuery and TermsQuery,
   wrapped in a QueryWrapperFilter
 - replaces XBooleanFilter, AndFilter and OrFilter with a BooleanQuery in a
   QueryWrapperFilter
 - removes DocIdSets.isBroken: the new two-phase iteration API will now help
   execute slow filters efficiently
 - replaces FilterCachingPolicy with QueryCachingPolicy

Close #8960
2015-04-21 15:32:43 +02:00
Robert Muir b31e590421 Merge pull request #10699 from rmuir/lucene_1675100
Upgrade to Lucene 5.2 r1675100
2015-04-21 09:22:50 -04:00
Robert Muir d1c3ec6291 Upgrade to Lucene 5.2 r1675100
This upgrade is for https://issues.apache.org/jira/browse/LUCENE-6442

It should improve test reproducibility, especially if you are on a mac
and want to reproduce a jenkins failure that happened on linux.
2015-04-21 09:06:44 -04:00
markharwood 63db34f649 New feature - Sampler aggregation used to limit any nested aggregations' processing to a sample of the top-scoring documents.
Optionally, a “diversify” setting can limit the number of collected matches that share a common value such as an "author".

Closes #8108
2015-04-21 10:22:05 +01:00
Adrien Grand ac74247c6c Merge pull request #10634 from jpountz/fix/validate_mappings_on_master
Mappings: Validate dynamic mappings updates on the master node.
2015-04-21 11:19:58 +02:00
Adrien Grand 1adf232bb2 Mappings: Validate dynamic mappings updates on the master node.
This commit changes dynamic mappings updates so that they are synchronous on the
entire cluster and their validity is checked by the master node. There are some
important consequences of this commit:
 - a failing index request on a non-existing type does not implicitely create
   the type anymore
 - dynamic mappings updates cannot create inconsistent mappings on different
   shards
 - indexing requests that introduce new fields might induce latency spikes
   because of the overhead to update the mappings on the master node

Close #8688
2015-04-21 11:08:21 +02:00
Adrien Grand ca03e40605 Internal: Ensure that explanation descriptions are not null on serialization.
As requested on #10399
2015-04-21 11:08:21 +02:00
Martijn van Groningen 24c75dec2d Merge pull request #10694 from martijnvg/matched_queries/cleanup
matched queries: Remove redundant and broken code
2015-04-21 10:55:49 +02:00
Adrien Grand cc9957fe0e Merge pull request #10689 from jpountz/enhancement/check_expl_desc
Internal: Ensure that explanation descriptions are not null on serialization.

Close #10399
2015-04-21 09:50:48 +02:00
Martijn van Groningen bb1cd65c0d matched queries: Remove redundant and broken code
Because the fetch phase now has nested doc, the logic that deals with detecting if a named nested query/filter matches with a hit can be removed.

Closes #10661
2015-04-21 08:52:51 +02:00
Adrien Grand f4d5914511 Docs: Warn about the fact that min_doc_count=0 might return terms that only belong to different types. 2015-04-21 00:57:57 +02:00
Adrien Grand b69e7e4d35 Internal: Ensure that explanation descriptions are not null on serialization.
As requested on #10399
2015-04-20 23:58:28 +02:00