Commit Graph

8316 Commits

Author SHA1 Message Date
Adrien Grand 17a32fca03 [TEST] Random dynamic templates.
This change randomly indexes the _id field and randomizes field data formats
and loading.

Close #5834
2014-05-06 11:07:43 +02:00
Igal 76463ee2b2 [DOCS] Update service.bat
corrected typo, Exiting instead of Existing

Close #5885
2014-05-06 10:28:32 +02:00
Ali Bozorgkhan f1af845795 [DOCS] Fixed a typo
Close #5963
2014-05-06 10:28:13 +02:00
Igal 20b05b56c4 [DOCS] Update client.asciidoc
Should be classpath rather than classloader.

Close #5965
2014-05-06 10:28:13 +02:00
Audrey 52d2f2d229 [DOCS] Update phrase-suggest.asciidoc
Grammatical error

Close #5993
2014-05-06 10:28:13 +02:00
Adrien Grand fc78dd2f13 [DOC] Fix default values for filter cache size and field data circuit breaker.
Relates to #5990
2014-05-06 10:13:05 +02:00
mikemccand 07563379dc fix docs for merging and throttling 2014-05-05 16:22:00 -04:00
Clinton Gormley 7a9aad30f4 [DOCS] Changed score_type to score_mode for has_child/parent queries 2014-05-05 18:30:12 +02:00
Clinton Gormley 5937cefd85 Added char_filters param to indices.analyze spec 2014-05-05 18:07:50 +02:00
Alexander Reelsen d356881664 [REST] Missing scroll id now returns 404
A bad/non-existing scroll ID used to return a 200, however a 404 might be more useful.
Also, this PR returns the right Exception (SearchContextMissingException) in the Java API.

Additionally: Added StatusToXContent interface and RestStatusToXContentListener listener, so
the appropriate RestStatus can be returned

Closes #5729
2014-05-05 17:37:26 +02:00
Shay Banon fad5e2d0e1 Remove operation threading from broadcast actions
Similar to search removal, the operation threading options are not really ued, and the default should always be used. This also considerably simplifies the code.
A side affect is that we can now remove the ShardIterator#firstOrNull method, which can cause for sneaky bugs to occur.
closes #6044
2014-05-05 17:09:36 +02:00
Alexander Reelsen 799bb2491c Analyze API: Default analyzer accidentally removed stopwords
The analyze API used the standard analyzer from lucene and therefore removed
stopwords instead of using the elasticsearch default analyzer.

Closes #5974
2014-05-05 15:55:33 +02:00
Alexander Reelsen d4fcf23057 Cluster State API: Remove index template filtering
The possibility of filtering for index templates in the cluster state API
had been introduced before there was a dedicated index templates API. This
commit removes this support from the cluster state API, as it was not really
clean, requiring you to specify the metadata and the index templates.

Closes #4954
2014-05-05 14:54:14 +02:00
Shay Banon 76e3e54c4a only test on openjdk 7 on travis, no need to tax it 2014-05-05 13:38:33 +02:00
Shay Banon 4827ad0c8a remove jdk6 from travis options for our java 7 supported branches 2014-05-05 13:26:51 +02:00
Shay Banon 7ce8306bc5 Remove search operation threading option
Search operation threading is an option that is not really used, and current non default implementations are flawed. Handling it also creates quite the complexity in the search handling codebase...
This is a breaking change, but one that is actually a good one, since I haven't seen/heard anybody use it, and if its used, its problematic...
closes #6042
2014-05-05 11:39:16 +02:00
Benjamin Devèze cea2d21c50 Fix bug in PropertyPlaceholder and add unit tests.
Close #6034
2014-05-05 10:21:18 +02:00
Adrien Grand 727e6172e3 Restore read/write visibility is PlainShardsIterator.
Change #5561 introduced a potential bug in that iterations that are performed
on a thread are might not be visible to other threads due to the removal of the
`volatile` keyword.

Close #6039
2014-05-05 10:05:44 +02:00
Shay Banon 342a32fb16 Search might not return on thread pool rejection
When a thread pool rejects the execution on the local node, the search might not return.
This happens due to the fact that we move to the next shard only *within* the execution on the thread pool in the start method. If it fails to submit the task to the thread pool, it will go through the fail shard logic, but without "counting" the current shard itself. When this happens, the relevant shard will then execute more times than intended, causing the total opes counter to skew, and for example, if on another shard the search is successful, the total ops will be incremented *beyond* the expectedTotalOps, causing the check on == as the exit condition to never happen.
The fix here makes sure that the shard iterator properly progresses even in the case of rejections, and also includes improvement to when cleaning a context is sent in case of failures (which were exposed by the test).
Though the change fixes the problem, we should work on simplifying the code path considerably, the first suggestion as a followup is to remove the support for operation threading (also in broadcast), and move the local optimization execution to SearchService, this will simplify the code in different search action considerably, and will allow to remove the problematic #firstOrNull method on the shard iterator.
The second suggestion is to move the optimization of local execution to the TransportService, so all actions will not have to explicitly do the mentioned optimization.
fixes #4887
2014-05-05 09:24:53 +02:00
javanna e96e634d10 [TEST] fixed _cat/thread_pool REST tests with local transport, in case the transport port is not available and gets returned as '-'
Re-enabled REST tests suite

Closes #6033
2014-05-04 22:10:03 +02:00
mikemccand 6bc3a744a1 Fix StackOverflowException for long suggestion strings
Changed getFiniteStrings to use an iterative implementation instead of
recursive, so we don't use a Java stack-frame per character for each
suggestion at build & query time.
2014-05-04 13:35:05 -04:00
Shay Banon c9f1792c81 Change default filter cache to 10% and circuit breaker to 60%
The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM.
The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs.
 The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure.
 closes #5990
2014-05-04 15:38:16 +02:00
Adrien Grand 01eb01cb70 [TEST] Disable REST tests until #6033 is fixed. 2014-05-04 11:58:30 +02:00
Zachary Tong f4c5cde8af [TEST] Replace folded blocks with literal blocks
The regex tests are formatted with blocks for readability.  Previously,
they were formatted using folded style blocks (e.g. using `>`). Folded
blocks convert newlines into spaces.  This is problematic for our regex,
since comments can only be terminated with a newline.

Effectively, anything after a comment will be commented out, making many
of the regex "silently pass".

This commit replaces them with scalar-style blocks (e.g. using `|`), which
treats newlines as significant, and thus correctly terminates comments
inside the regex.

Also fixes a regex test (`cat.thread_pool/10_basic.yaml`) that started
to fail after the block was fixed.  The test was missing a `\s+` before
the closing newline.
2014-05-02 18:30:48 -04:00
gabriel-tessier 48930c2950 [DOC] Fix typo in function score query documentation. 2014-05-02 23:44:56 +02:00
Boaz Leskes 694bf287d6 Do not start a recovery process if the primary shard is currently allocated on a node which is not part of the cluster state
If a source node disconnect during recover, the target node will respond by canceling the recovery. Typically the master will respond by removing the disconnected node from the cluster state, promoting another shard to become primary. This is sent it to all nodes and the target node will start recovering from the new primary. However, if the drop of a node caused the node count to go bellow min_master_node, the master will step down and will not promote shard immediately. When a new master is elected we may publish a new cluster state (who's point is to notify of a new master) which is not yet updated. This caused the node to start a recovery to a non existent node. Before we aborted the recovery without cleaning up the shard, causing subsequent correct cluster states to be ignored. We should not start the recovery process but wait for another cluster state to come in.

Closes #6024
2014-05-02 23:30:24 +02:00
Alex Ksikes b55d8ed2e3 Fix behavior on default boost factor for More Like This.
A boost terms factor of 1.0 is not the same as no boosting of terms.
The desired behavior is to deactivate boosting by default. If the user
specifies any value other than 0, then boosting is activated.

Closes #6021
2014-05-02 16:59:09 +02:00
Mansur Ashraf d5f90e9803 [DOCS] Added Twitter Storehaus client
Added Twitter Storehaus client
2014-05-02 12:08:05 +02:00
Holger Hoffstätte f5c9bf6f0f Update JNA to latest version
Updating to this version allows to configure a special JNA directory,
in case the /tmp directory is mounted with the noexec option, as JNA
extracts some data and tries to execute parts of it.

Also updated documentation to clarify mlockall and memory settings as well
as pointing to the new jna.tmpdir system property.

Closes #5493
2014-05-02 11:52:57 +02:00
Britta Weber 2e44040388 function_score parser throws exception if both functions:[] and single function given
In addition, add a special warning if the misplaced function is a "boost_factor"
function to avoid confusion of "boost" and "boost_function".

closes #5995
2014-05-02 10:53:33 +02:00
Shay Banon a557ee8daf Support empty properties array in mappings
closes #5887
2014-05-01 12:18:39 -04:00
Boaz Leskes 42a112f50b debug log of receiving a cluster state from another master could be erroneously logged
Added trace logging to MinimumMasterNodesTests.multipleNodesShutdownNonMasterNodes
2014-05-01 13:15:08 +02:00
Martijn van Groningen 9493824a0e [TEST] (RecoveryPercolatorTests) Don't stop the master node and always use the client of the master node 2014-05-01 14:06:34 +07:00
Martijn van Groningen 61093f1bd1 [TEST] Replace execute().actionGet() with get() 2014-05-01 14:06:34 +07:00
Shay Banon 23f200bc0e Use non analyzed token stream optimization everywhere
In the string type, we have an optimization to reuse the StringTokenStream on a thread local when a non analyzed field is used (instead of creating it each time). We should use this across the board on all places where we create a field with a String.
Also, move to a specific XStringField, that we can reuse StringTokenStream instead of copying it.
closes #6001
2014-04-30 17:18:15 -04:00
Martijn van Groningen 12f43fbbc0 Fixed license headers. 2014-05-01 00:33:17 +07:00
Martijn van Groningen 013b319415 Added `reverse_nested` aggregation.
The `reverse_nested` aggregation allows to aggregate on properties outside of the nested scope of a `nested` aggregation.

Closes #5507
2014-05-01 00:23:05 +07:00
Martijn van Groningen 5a0070071a Use collectExistingBucket in GlobalOrdinalsSignificantTermsAggregator.WithHash.
Relates to #5955.
2014-04-30 23:24:33 +07:00
Matt Weber 2663d04a96 Run tests through forbidden-apis. 2014-04-30 17:48:33 +02:00
Adrien Grand 34fb5e48e2 Use collectExistingBucket in GlobalOrdinalsStringTermsAggregator.WithHash.
Relates to #5955.
2014-04-30 15:34:01 +02:00
Boaz Leskes 870bd90f54 ThreadPool.EstimatedTimeThread should be set on initialization
Some tests run before the thread is started and thus use 0 as a the current time, which later on leads to big time jumps and thus failures.
Ex. InternalEngineTests.testVersioningReplicaConflict2
2014-04-30 11:47:47 +02:00
Adrien Grand b2db7c8222 Improve the way sub-aggregations are collected.
Sub-aggregations are currently collected directly, by just forwarding the
doc ID and bucket ordinal to them. This change adds the new BucketCollector
abstract class that Aggregator extends, so that we have more flexibility to
add implicit filters or buffering between an aggregator and its sub
aggregators.

Close #5975
2014-04-30 08:47:25 +02:00
Adrien Grand 2eeaa56d95 Fix setting of readerGen in BytesRefOrdValComparator on nested documents.
Sorting was broken on nested documents because the `missing(slot)` method
didn't correctly set the segment ordinal (readerGen), causing term ordinals to
be compared across segments.

Close #5986
2014-04-30 08:21:26 +02:00
Shay Banon 2076194d8f Upgrade to Jackson 2.3.3
fixes the long value bug as well...
2014-04-29 20:13:43 -04:00
Shay Banon 34302a7cc5 disable using CBOR in randomized test infra
due to a bug in CBOR handling long values (test case to verify it is included), disalbe using CBOR in our tests till it gets fixed
2014-04-29 19:11:12 -04:00
Spencer Alger 9038db7bfc [REST-SPEC] update to update test, to check for es-js error messages 2014-04-29 14:18:20 -07:00
Martijn van Groningen dce127bcdf Added global ordinals based implementations for significant terms aggregator.
Closes #5970
2014-04-30 01:36:02 +07:00
Shay Banon a4ef418e6e Range/Term query/filter on dates fail to handle numbers properly
When providing a number (milliseconds since epoch, UTC), range and term query/filter don't handle it correctly and convert it to a string, that is then first tried to parse as a date
closes #5969
2014-04-29 14:25:05 -04:00
mikemccand fb53784e3b add thread name to logger message from IndexWriter's infoStream 2014-04-29 10:50:36 -04:00
Adrien Grand 6ec01c13e5 Fix computation of the missing ord (leftover of the ordinals change). 2014-04-29 16:29:01 +02:00