Commit Graph

11314 Commits

Author SHA1 Message Date
Boaz Leskes 3dd37c06ff Test: testTranslogReplayWithFailure didn't allow for MockDirectoryWrapper being optional 2015-04-14 20:36:32 +02:00
Alex Ksikes d339ee4005 Term Vectors: terms filtering
This adds a new feature to the Term Vectors API which allows for filtering of
terms based on their tf-idf scores. With `dfs` option on, this could be useful
for finding out a good characteric vector of a document or a set of documents.
The parameters are similar to the ones used in the MLT Query.

Closes #9561
2015-04-14 19:11:09 +02:00
Colin Goodheart-Smithe 82df50a039 Aggregations: Fix _as_string output to only show when format specified
Closes #10284
2015-04-14 16:30:18 +01:00
Adrien Grand 22720a1abd Merge pull request #10562 from jpountz/upgrade/lucene-5.2.0-snapshot-1673124
Upgrade to lucene-5.2.0-snapshot-1673124.
2015-04-14 16:57:09 +02:00
Simon Willnauer 210f6c7b33 [TRANSLOG] Remove useless random translog directory selection
Using ThreadLocalRandom only prevents reproducibilty but doesn't buy us
anything. In production different datapaths won't have the same since
anyway or at least with a low likelyhood.
2015-04-14 15:57:09 +02:00
Clinton Gormley 906d0ee369 Merge pull request #10552 from clintongormley/resiliency_1.5
Doc: Updates to resiliency page for 1.5.0/1
2015-04-14 15:31:26 +02:00
Clinton Gormley 7c147b2db5 Doc: Updates to resiliency page for 1.5.0/1 2015-04-14 15:30:33 +02:00
Christoph Büscher eac922b73b Merge pull request #10592 from cbuescher/remove-matchall-norms
Remove unused normsField from MatchAllQuery
2015-04-14 13:11:05 +02:00
Christoph Büscher c169e792b5 Remove unused normsField from MatchAllQuery 2015-04-14 12:39:05 +02:00
Michael McCandless 8925615eea also log all mounts in node path summary log line; use JmxFsProbe also in non-summarized case 2015-04-14 06:22:24 -04:00
Michael McCandless 68f93e53a8 Include spins and file system type in "fs" nodes stats
Also log a one-line summary at default INFO level logging of the
spins, type for all paths in the node's path.data.

Closes #10527
2015-04-14 04:58:35 -04:00
Simon Willnauer 0fcd31b6dc [CORE] Move back to single EngineConfig
We need to preserve settings (yet transient) even though the engine is not yet
started. This commit moves back to a single EngineConfig to simplify IndexShard
and settings state.

Closes #10584
2015-04-14 10:43:23 +02:00
Simon Willnauer 67b48da15f [BUILD] Fix m2.repository path permission in tests.policy 2015-04-14 10:40:31 +02:00
Simon Willnauer b756477fb4 [CORE] Rename START phase into VERIFY_INDEX
we really only optionally run checkindex in this phase and moving
the engine start into translog is move cleaner code wise.

Closes #10570
2015-04-14 09:53:02 +02:00
Simon Willnauer fe411a9295 [BUILD] Restrict read permission to project.basedir/target if security manager is used 2015-04-14 09:35:40 +02:00
Boaz Leskes 773d64c4f2 Test: mute testIndexBufferSizeUpdateAfterCreationRemoval
see #10584
2015-04-14 08:36:29 +02:00
Jun Ohtani deeea217e1 Analysis: fix ignoring tokenizer settings in SynonymTokenFilterFactory
add test for synonym with tokenizer settings
2015-04-14 08:37:04 +09:00
Robert Muir 131431aa54 nuke duplicate copy-paste in AllTermQuery 2015-04-13 18:57:02 -04:00
Shay Banon 41343eca95 Improve exception handling in transport local execution
Local execution of transport messages failures can create a more detailed remote transport exceptions. Also, when failing to handle an exception, the error should be logged, and not call the handler again with another exception
closes #10554
2015-04-13 15:45:55 -07:00
Shay Banon e2a05bb43f Cleanup JVM info and stats
Remove reflection since its not needed with Java 7, remove lastGcInfo since its not used, and move to only provide getters
closes #10553
2015-04-13 15:24:14 -07:00
Lee Hinman fcbe3f92b7 [CORE] Log sending translog operation batches to nodes 2015-04-13 14:59:32 -06:00
Alex Ksikes c347dfe91c Validate API: support for verbose explanation of succesfully validated queries
This commit adds a `rewrite` parameter to the validate API in order to shown
how the given query is re-written into primitive queries. For example, an MLT
query is re-written into a disjunction of the selected terms. Other use cases
include `fuzzy`, `common_terms`, or `match` query especially with a
`cutoff_frequency` parameter. Note that the explanation is only given for a
single randomly chosen shard only, so the output may vary from one shard to
another.

Relates #1412
Closes #10147
2015-04-13 19:17:58 +02:00
Simon Willnauer c13e604697 [BUILD] Restrict read permission to project.basedir
This prevents reads from anywhere outside of the elasticsearch
clone when running tests with security manager enabled.
2015-04-13 16:44:31 +02:00
Simon Willnauer b1c9dfc2e8 [RECOVERY] Integrate translog recovery into Engine / InternalEngine
Today the engine writes the transaction log itself as well as manages
all the commit / translog mapping internally. Yet, if an engine is closed
and reopend it doesn't replay it's translog or does anything to be consistent
with it's latest state again.
This change moves the transaction log replay code into the Engine / InternalEngine
and adds unittests for replaying and consistency.

Closes #10452
2015-04-13 16:42:45 +02:00
Clinton Gormley ab3fa78ae0 Docs: Reverte migration docs mentioning parent removal from update request
Relates to #9612
2015-04-13 16:35:21 +02:00
Boaz Leskes 8e302f1172 Recovery: allow to recover into a folder containing a corrupted shard
At the moment, we are very strict when handling data folders containing corrupted shards and will fail any recovery attempt into it. Typically this wouldn't be a problem as the shard will be assigned to another node (which we try first anyway when a shard fails). However, it has been proven to be too strict for smaller clusters which may not have an extra node available (either because of allocation filtering, disk space issues etc.). This commit changes the behavior to force a full recovery. Once all the new files are verified we remove the old corrupted data and start the shard.

This also fixes a small issue where the shard state file wasn't deleted on an engine failure (we had a protection against deleting the state file on an active shard, but in this case the shard is still active but will be removed). The state deletion is also moved to before the failure handlers are called, to avoid race conditions when calling the master (it will potentially try to read it when allocating the shard)

Closes #10558
2015-04-13 15:07:14 +02:00
Benoit Delbosc 1b35854768 Docs: Fix simple_query_string example
The "&" is not part of the simple_query_string DSL

Closes #10563
2015-04-13 14:46:47 +02:00
David Pursehouse 48472940c7 Update links in README file
Change-Id: Ife01695355db77c91246efd4f34a25998a3ca01f
2015-04-13 13:46:56 +02:00
Adrien Grand 45afa47a6f Upgrade to lucene-5.2.0-snapshot-1673124. 2015-04-13 10:43:58 +02:00
Simon Willnauer d738b3d1e2 [BUILD] Use SuppressFrobidden annotation instead of class level excludes
Forbidden APIs 1.8 allows excludes based on annotations which can now be on methods etc. for more find grained control.

Closes #10560
2015-04-13 10:08:50 +02:00
Uwe Schindler df118214c4 Update forbiddenapis to version 1.8 2015-04-13 01:27:56 +02:00
Britta Weber 4934def035 [TEST] better test start and end messages
print when cleaning up also for SingleNodeTests
distinguish between after suite and after test cleanup
2015-04-11 15:02:40 +02:00
Michael McCandless 3cc3390f17 don't return negative percentage when max < 0 2015-04-11 06:02:59 -04:00
Nicholas Knize 90e1775a2b [GEO] Correct ShapeBuilder coordinate parser to ignore values in 3rd+ dimension
ShapeBuilder's coordinate parser expected 2 double values for every coordinate array. If > 2 doubles were provided the parser terminated parsing of the coordinate array. This resulted in an invalid Shape state leaving LineStrings, LinearRings, and Polygons with a single coordinate. An incorrect parse exception was thrown. This corrects the parser to ignore those values in the 3rd+ dimension, correctly parsing the rest of the coordinate array.

Unit tests have been updated to verify the fix.

closes #10510
2015-04-10 15:33:21 -05:00
Nicholas Knize 754856289e [GEO] Add merge conflicts to GeoShapeFieldMapper
Prevents the user from changing strategies, tree, tree_level or precision. distance_error_pct changes are allowed as they do not compromise the integrity of the index. A separate issue is open for allowing users to change tree_level or precision.
2015-04-10 14:33:17 -05:00
Nicholas Knize a8a35d7c29 [GEO] Fix hole intersection at tangential coordinate
OGC SFA 2.1.10 assertion 3 allows interior boundaries to touch exterior boundaries provided they intersect at a single point. Issue #9511 provides an example where a valid shape is incorrectly interpreted as invalid (a false violation of assertion 3).  When the intersecting point appears as the first and last coordinate of the interior boundary in a polygon, the ShapeBuilder incorrectly counted this as multiple intersecting vertices. The fix required a little more than just a logic check. Passing the duplicate vertices resulted in a connected component in the edge graph causing an invalid self crossing polygon. This required additional logic to the edge assignment in order to correctly segment the connected components. Finally, an additional hole validation has been added along with proper unit tests for testing valid and invalid conditions (including dateline crossing polys).

closes #9511
2015-04-10 11:44:40 -05:00
David Pursehouse 97a9b4ec2f Fix m2e Eclipse Integration
Update Eclipse core prefs

Eclipse Luna overwrites the prefs file, putting all the settings
in alphabetical order and removing comments. This causes the prefs
files to be modified in the git workspace.

Update the file with the version generated by Eclipse to prevent it
from being modified every time.

No settings values are modified by this change.

This also  adds another plugin to the lifecycle mapping in the pom.xml which was missed in https://github.com/elastic/elasticsearch/pull/10524.
2015-04-10 16:48:04 +01:00
Adrien Grand ab8926bc6a Docs: fix build. 2015-04-10 17:38:36 +02:00
Robert Muir b936ec9a25 allow reflection of MXBean for file descriptor stats 2015-04-10 11:28:30 -04:00
Adrien Grand 0b413bd670 Merge pull request #10532 from jpountz/deprecate/limit_filter
Search: deprecate the limit filter.

Close #10532
2015-04-10 17:19:43 +02:00
Adrien Grand 5b3cc2f07c Search: deprecate the limit filter.
This is really a Collector instead of a filter. This commit deprecates the
`limit` filter, makes it a no-op and recommends to use the `terminate_after`
parameter instead that we introduced in the meantime.
2015-04-10 17:18:50 +02:00
Adrien Grand bad6656f02 Merge pull request #10391 from jpountz/fix/remove_flt
Queries: Remove fuzzy-like-this support.

Close #10391
2015-04-10 17:17:00 +02:00
Adrien Grand 919589b908 Queries: Remove fuzzy-like-this support.
The fuzzy-like-this query builds very expensive queries and only serves esoteric
use-cases.
2015-04-10 17:16:02 +02:00
Simon Willnauer bdc4c7f84c [BUILD] Add sigar binaries when running unittests
the sigar binaries are not available when running tests today. This
commit adds the path to the test run.
2015-04-10 14:37:18 +02:00
Colin Goodheart-Smithe 7e353e0768 removed erroneous tab character 2015-04-10 10:37:26 +01:00
Colin Goodheart-Smithe 4a09a98561 Fix to pom.xml to allow eclipse maven integration using m2e 2015-04-10 08:34:01 +01:00
Ryan Ernst 224c43564b Tests: Add multi data path testing to static bwc tests
This randomly chooses to run bwc indexes on single or multi data paths.

closes #10519
2015-04-09 14:31:31 -07:00
Ryan Ernst 9d0764e5df Tests: increasing timeout for replicas in static bwc tests, to account
for reallySlowJenkinsWhenIndexHasLotsOfSegments
2015-04-09 13:31:08 -07:00
Michael McCandless 78612cf0a8 fix indent 2015-04-09 14:47:48 -04:00
Michael McCandless 54b702db07 Tests: don't fsync so often
Most tests don't "really" need to fsync, and this is costly (makes
tests slower, wears out our SSDs).

This change makes it uncommon to actually fsync when Lucene asks for
it.  It's just a workaround (in MockDirectoryHelper) until we can
cutover Elasticseach to use MockFileSystem like Lucene.

Closes #10516
2015-04-09 14:31:42 -04:00