Commit Graph

9223 Commits

Author SHA1 Message Date
Simon Willnauer 0676869e6d [ENGINE] Wait until engine is started up when acquireing searcher
Today we have a small window where a searcher can be acquired but the
engine is in the state of starting up. This causes a NPE triggering a
shard failure if we are fast enough. This commit fixes this situation
gracefully.

Closes 
2014-08-26 14:07:04 +02:00
Britta Weber d7b8d1728e _all: report conflict on merge and throw exception on doc_values
- _all field was never merged when mapping was updated and no conflict reported
- _all accepted doc_values format although it is always tokenized

relates to 
closes 
2014-08-26 12:14:31 +02:00
mikemccand 075bd66713 Core: use Java's built-in ConcurrentHashMap
It's risky to have our own snapshot of Java 8's ConcurrentHashMap:
unless we keep the sources in sync over time (and OpenJDK's version
had already diverged), then we won't get bug/performance fixes.  Users
can choose to upgrade to Java 8 to see the improvements of CHM.

Closes 

Closes 
2014-08-26 06:11:05 -04:00
Alexander Reelsen f2aa4a38bc Docs: Added link to clarify meaning of filtering in fielddata context 2014-08-26 12:00:06 +02:00
Adrien Grand b43c2ced93 [TESTS] Temporary disable field data cache randomization. 2014-08-25 23:12:09 +02:00
mikemccand 783a9cbb18 Stats: add segments.index_writer_max_memory to see index writer's max RAM usage before buffered documents must be written to a new segment
Closes 

Closes 
2014-08-25 14:43:09 -04:00
Nik Everett 74287865b2 [Internal] discovery.id.seed is ignored
Closes , Closes 
2014-08-25 17:32:07 +02:00
javanna 3917ffc0ff [TEST] Explicitly clean up actions to be intercepted in IndicesRequestTests before asserting on collected requests
This helps making sure that no further requests are collected once we start asserting on them
2014-08-25 17:24:53 +02:00
Lee Hinman 1f7be7931b [TEST] fix issue clearing fielddata breaker introduced in 6950c38a04 2014-08-25 16:25:02 +02:00
Clinton Gormley ce0b2ade9c Test: Fixed the regex in cat.indices/10_basic.yaml
And renamed the file from .yml to .yaml
2014-08-25 16:13:16 +02:00
Adrien Grand 2a67b129e2 [TESTS] Temporarily disable FileBasedMappingsTests. 2014-08-25 12:39:36 +02:00
markharwood 570c679420 Context suggester: infinite loop in GeolocationContextMapping
Close 
2014-08-25 11:56:39 +02:00
Simon Willnauer 6950c38a04 Tests: Improve test coverage.
Close 
2014-08-25 11:56:38 +02:00
Alexander Reelsen 49f0f0bb5d Test: Fixed pluggable transport module test to support transportclient
Also made sure, that only a change of requests is tested for and not
an initial value, which might not be set in case of a node client.
2014-08-25 10:36:04 +02:00
Lee Hinman c5c11b1837 [DOCS] Document NOT as a simple_query_string flag
Fixes 
2014-08-25 10:27:24 +02:00
Martijn van Groningen bd0b68080b Nested: If the `_type` field isn't indexed nested docs must be filtered out. 2014-08-25 00:09:21 +02:00
Martijn van Groningen d471abe4d3 [TEST] Agg may not be a instance of StringTerms, but UnmappedTerms, so use common Terms class instead 2014-08-25 00:07:19 +02:00
Simon Willnauer 24e3c41afa [TEST] use more verbose assertion in IndicesRequestTests 2014-08-24 21:23:46 +02:00
Boaz Leskes e16a461317 [Tests] testNodeVersionIsUpdated stopped but didn't close it's node 2014-08-24 19:34:02 +02:00
Boaz Leskes 562fe1ddaf [Tests] NoMasterNodeTests make timeout checks less sensitive
Also remove catching of MasterNotDiscoveredException in bulk operation it is only set on a per item basis
2014-08-23 22:04:22 +02:00
javanna 00fc54c2ae Internal: made original indices optional for broadcast delete and delete by query shard requests
Shard requests like broadcast delete and delete by query, that needs to be executed on primary and all replicas, get read and written out to the transport on the same node. That means that if we add some field version checks are not enough to maintain bw comp since a newer node that holds the primary might receive the request from an older node, that didn't provide the field. Yet, when writing the request out again to a newer node that holds the replica, we do try and serialize the field although it's missing. The newer fields just needs to be set to optional in these cases, in addition to the version checks.

Re-enabled testDeleteByQuery and testDeleteRoutingRequired bw comp tests since this was the cause of their failures.

Closes 
2014-08-23 17:01:33 +02:00
Simon Willnauer 5f188d29fa [TEST] use CFS consistently to not trigger single segment merge without force flag 2014-08-23 16:41:46 +02:00
Boaz Leskes 06fb9ff761 [Tests] verifyThreadNames should account for new threads of shared cluster
The verifyThreadNames starts a node and checks that all new threads on the JVM are properly named. The current test uses the name of the new node which sometimes fails because our shared cluster spawns a new thread which is properly named but for not for the new name.

The commits relaxes the requirement of the test and on verify the threads are properly named (but not necessarily of the new node)
2014-08-23 14:45:08 +02:00
Simon Willnauer 45f062792c [TEST] use a default host name if localAddress is not available
Closes 
2014-08-23 13:47:11 +02:00
Simon Willnauer 805f042293 Add toString() method to Segment.java for debugging purposes 2014-08-23 11:17:14 +02:00
Simon Willnauer fdf1998f39 [ENGINE] Force optimize was not passed to shard request
The force flag to trigger optimiz calls of a single segment for upgrading
etc. was never passed on to the shard request.

Closes 
2014-08-22 15:39:04 +02:00
Alex Ksikes e78694ae82 More Like This Query: defaults to all possible fields for items
Items with no specified field now defaults to all the possible fields from the
document source. Previously, we had required 'fields' to be specified either
as a top level parameter or for each item. The default behavior is now similar
to the MLT API.

Closes 
2014-08-22 15:07:22 +02:00
Adrien Grand a1a9aadab5 [DOCS] Document the contracts of the RootMapper API.
Close 
2014-08-22 14:44:28 +02:00
javanna f4168a6382 Internal: move index templates api back to indices category and make put template and create index implement IndicesRequest
Closes 
2014-08-22 10:18:36 +02:00
javanna 9a14b3ce6f [TEST] copied delete bw comp tests to usual intergration tests
Added AwaitsFix to testDeletebyQuery and testDeleteRoutingRequired while checking if they fail as usual integration tests.
2014-08-22 10:12:17 +02:00
Martijn van Groningen 0196377190 [TEST] Muted tests 2014-08-22 09:45:56 +02:00
Alexander Reelsen e478f24b9e Dependencies: Upgrade to Apache HttpComponents client 4.3.5
Closes 
2014-08-22 09:44:13 +02:00
Simon Willnauer a9f42e7176 [BUILD] Randomize JVM heapsize between 512 and 1024m 2014-08-22 09:40:09 +02:00
Simon Willnauer 3b51342515 Use empty BytesRef if we read from <= 1.4.0 2014-08-21 22:13:06 +02:00
Shay Banon ffcc78ca04 Add back string op type to IndexRequest
This was removed by accident I think, and it breaks backward comp. on the Java API in minor 1.3 version
closes 
2014-08-21 12:04:09 -07:00
Greg Marzouka 887bd0a9c2 [DOCS] Update to the new {index}/_mapping/{type} format 2014-08-21 13:56:11 -04:00
Igor Motov 80887e8113 [TEST] Trigger random flushes while snapshot is created 2014-08-21 12:48:38 -04:00
Simon Willnauer 058a02b7aa [STORE] Improve recovery / snapshot restoring file identity handling
This commit changes the way how files are selected for retransmission
on recovery / restore. Today this happens on a per-file basis where the
rather weak checksum and the file length in bytes is compared to check if
a file is identical. This is prone to fail in the case of a checksum collision
which can happen under certain circumstances.
The changes in this commit move the identity comparsion to a per-commit / per-segment
level where files are only treated as identical iff all the other files in the
commit / segment are the same. This "all or nothing" strategy is reducing the chance for
a collision dramatically since we also use a strong hash to identify commits / segments
based on the content of the ".si" / "segments.N" file.

Closes 
2014-08-21 18:00:41 +02:00
Simon Willnauer 4c1bc3ae4f [STORE] Remove unnecessary deduplication 2014-08-21 17:50:04 +02:00
Simon Willnauer 4d3f761d3d [STORE] Ignore segments.gen on metadata snapshots
The segments.gen file is optional and might even change while we
read it. It's safer to just ignore that file in the snapshot instead.
2014-08-21 17:50:04 +02:00
Shay Banon 4af1a29057 [TEST] filter out keep alive timer thread name
Keep-Alive-Timer is an internal Java thread that might be started, make sure to filter it out
2014-08-21 08:40:43 -07:00
Shay Banon 39a64cf4dd [TEST] only reset clients on nightly tests
resetting the clients on each test (in after test) makes the tests running, especially in network mode, much slower, since transport client needs to be created each time when randmized to be used. Also, on OSX, the excessive connections causes bind exceptions eventually which makes running the network tests much harder on it.
closes 
2014-08-21 07:34:26 -07:00
Britta Weber ab9e33e38d _ttl: Report conflict when trying to disable _ttl
_ttl could never be disabled once it was enabled.
But when trying to, no conflict was reported.

relates to  and 

closes 
2014-08-21 16:16:08 +02:00
Britta Weber 2a32cf338e Docs: _ttl ignores all parameters except for enabled and default.
Also, add a line about that it can never be disabled after it was enabled.
2014-08-21 16:16:02 +02:00
Simon Willnauer 99ef3408fb [STORE] Allow to get metadata from arbitrary commit points
Today we always use the latest commit point to return the metadata from
the store. This might cause problems for snapshot and restore since in
contrast to recovery it won't prevent concurrent flushes (lucene commits).
This can lead to all kinds of interesting effects if we are snapshotting
while flushing. This change uses the IndexCommit to open the metadata snapshot
from the store which is consistent with what we snapshot.

Closes 
2014-08-21 16:09:12 +02:00
Britta Weber 5bfd75e489 Test: write heap dump per default
The heap dump is always interesting in case a test causes an OOM.
Per default is is written to target/JX/pidXYZ.hprof
2014-08-21 15:59:32 +02:00
Britta Weber 2f9eaae863 Testing doc: describe -Dtests.heap.size and -Dtests.jvm.argline 2014-08-21 15:59:31 +02:00
Colin Goodheart-Smithe 8550b9e84b Aggregations: Fixes pre and post offset serialisation for histogram aggs
Changes the serialisation of pre and post offset to use Long instead of VLong so that negative values are supported.  This actually only showed up in the case where minDocCount=0 as the rounding is only serialised in this case.

Closes 
2014-08-21 14:19:53 +01:00
javanna f956920acc Internal: make sure that multi_search request hands over its context and headers to its corresponding search requests
Closes 
2014-08-21 15:09:27 +02:00
javanna b6cdaff30c Internal: make sure that multi_percolate request hands over its context and headers to its corresponding shard requests
Closes 
2014-08-21 13:45:11 +02:00