Commit Graph

13644 Commits

Author SHA1 Message Date
Simon Willnauer b3f1a5da4a Merge pull request #11225 from s1monw/remove_flush_needed
[ENGINE] Remove flushNeeded in favor of IW#hasUncommittedChanges()
2015-05-19 14:53:09 +02:00
Simon Willnauer 2439faaf5b [TEST] Add more engine unittests for sync IDs 2015-05-19 14:43:03 +02:00
Shay Banon 643c2049a2 [TEST] add await fix for #11226 2015-05-19 14:40:02 +02:00
Simon Willnauer d6e7817666 [ENGINE] Remove flushNeeded in favor of IW#hasUncommittedChanges()
We maintain a boolean that is obsolete since IW maintains the same
property that we can just piggyback on.
2015-05-19 14:10:07 +02:00
Shay Banon a1a16276da [TEST] Add a corrupted replica test verifying its still allocated
Add a test that verifies that even though all replicas are corrupted on all available nodes, and listing of shard stores faield, it still get allocated and properly recovered from the primary shard
2015-05-19 13:18:43 +02:00
Simon Willnauer 488be75d19 Add some words about the purpose of a seal etc. 2015-05-19 12:26:08 +02:00
Simon Willnauer 9d2852f0ab Merge branch 'master' into feature/synced_flush
Conflicts:
	src/main/java/org/elasticsearch/index/engine/InternalEngine.java
	src/main/java/org/elasticsearch/index/shard/IndexShard.java
	src/main/java/org/elasticsearch/indices/recovery/RecoverySourceHandler.java
	src/test/java/org/elasticsearch/index/engine/InternalEngineTests.java
2015-05-19 12:16:22 +02:00
Simon Willnauer 55f99dfce6 Make SyncedFlushService fully asynchronous
Some requests in the SyncedFlushService were sill blocking on network
calls which made calling this service error prone if done on a network
thread. This commit makes this service fully async based on ActionListener.
2015-05-19 12:01:31 +02:00
Martijn van Groningen 84b24e130e inner hits: Fix search hit field and term resolution 2015-05-19 11:25:52 +02:00
Adrien Grand 17d65a5f90 Tests: Remove RelocationTests.testRelocationWithBusyClusterUpdateThread.
It is not relevant anymore with synchronous mapping updates.
2015-05-19 10:37:45 +02:00
Adrien Grand df5ffb1418 Merge pull request #11218 from jpountz/upgrade/lucene-5.2.0-snapshot-1680200
Upgrade to lucene-5.2.0-snapshot-1680200.
2015-05-19 10:34:32 +02:00
Simon Willnauer d817793f9b [TEST] Add Unittest for Lucene#getNumDocs 2015-05-19 10:32:40 +02:00
Adrien Grand 437910e882 Upgrade to lucene-5.2.0-snapshot-1680200. 2015-05-19 10:28:35 +02:00
Simon Willnauer 09283cf7d3 minor cleanups 2015-05-19 10:23:18 +02:00
Adrien Grand 4dc4566168 Merge pull request #11203 from jpountz/fix/remove_ignore_conflicts
Mappings: Remove the `ignore_conflicts` option.
2015-05-19 10:03:02 +02:00
Britta Weber 8a3d504efb skip everything between file checks and translog prepare if sync ids coincide
also throw exception instead of assert if num docs no coincide
2015-05-19 09:30:12 +02:00
Britta Weber 8ef734908c assert num docs in recovery if sync id was used 2015-05-19 09:30:12 +02:00
Adrien Grand 3ca3c6c09c Merge pull request #11209 from jpountz/fix/documentmapper_refreshsource_private
Mappings: Make DocumentMapper.refreshSource() private.
2015-05-19 08:49:52 +02:00
Adrien Grand f0d6dbbc36 Merge pull request #11210 from nik9000/detect_noop_null
detect_noop now understands null as a valid value
2015-05-19 08:38:49 +02:00
Ryan Ernst e95097c33c Mappings: Remove SmartNameFieldMappers
This change simplifies the users of the mapper service to no
longer have access to multiple fields for a single name. While
FieldMappersLookup still stores and gives access to multiple
fields, the current users of SmartNameFieldMappers already all
assumed a single field. The arbitrary selection of that field
when multiple exist is now isolated to the mapper service.
2015-05-18 22:41:56 -07:00
Robert Muir 0105b65160 Merge pull request #11212 from rmuir/java.io.tmpdir.early
Ensure java.io.tmpdir is created earlier in tests.
2015-05-18 20:50:37 -04:00
Robert Muir e89693408f Ensure java.io.tmpdir is created earlier in tests.
When testing, each jvm gets its own tmpdir set, so it may not exist
at all. A Lucene test rule ensures its created, but some tests (I
am looking at you rest tests) do a bunch of file stuff in static {},
in that case because its a parameterized test. And if you try to
extend it, it will fail if security manager is disabled...

Currently we ensure(java.io.tmpdir) very early when tests are running under
security manager, but otherwise we don't and it won't happen until the
test rule fires. So just do it early always.
2015-05-18 16:37:20 -04:00
Simon Willnauer 6dbe7d4b55 Make commitID byte[] final 2015-05-18 21:55:23 +02:00
Simon Willnauer 75b62e12a6 Merge pull request #11193 from s1monw/remove_commit_without_translog
[ENGINE] Remove the ability to flush without flushing the translog
2015-05-18 21:48:07 +02:00
Lee Hinman 8742d477c4 Fix compilation for latest ES versions
This fixes a change that recently landed in 1.x and master that prevents
the plugin from being compiled.
2015-05-18 13:43:22 -06:00
Nik Everett 400abfceaf detect_noop now understands null as a valid value
If the source contrains a null value for a field then detect_noop should
consider setting it to null again to be a noop.

Closes #11208
2015-05-18 15:14:01 -04:00
Simon Willnauer 4b8ed1b0bf [TRANSLOG] Return null from read() if location generation is outdated
This behavior has changed been changed rescently to throw an IAE if
the translog we try to read from is already outdated. This is not
the expected behavior and this commit adds back the `old` way returning
`null` instead. The InternalEngine implementation will then go and ask the
lucene index for the document instead.
2015-05-18 20:14:39 +02:00
Alexander Reelsen 13e5c19dcf Test: Prevent automatic refresh in ContextAndHeaderTransportTests
In some cases due to calling checking `rarely()` the `indexRandom()` method
can potentially flush, which creates flush requests, that miss a certain
header in this test and allow the test to fail.

In addition unused configuration code for this test has been removed.
2015-05-18 18:22:48 +02:00
Adrien Grand 2999f53e86 Tests: Mark RelocationTests.testRelocationWithBusyClusterUpdateThread as @AwaitsFix. 2015-05-18 18:05:26 +02:00
Simon Willnauer 91d0580b0d Merge pull request #11200 from s1monw/cleanup_translog
Some smallish translog cleanups
2015-05-18 17:41:31 +02:00
Simon Willnauer a21e4449b7 Added additional assertion that the view to remove was actually registered. 2015-05-18 17:14:06 +02:00
Simon Willnauer bc810e1320 remove NoopOutputStream 2015-05-18 17:05:36 +02:00
Simon Willnauer 651c067fae Cleanup translog operation serialization
We used to double write the translog operation which is not needed except
of for recovery. This commit cuts over to a big-array based temporary serialiation
and removes the crazy double writing.
2015-05-18 17:05:36 +02:00
Simon Willnauer 60b66a7235 [TRANSLOG] Fold Translog.View into it's only implementation 2015-05-18 17:05:36 +02:00
Adrien Grand a75cfb42de Mappings: Make DocumentMapper.refreshSource() private.
This method should not be public, we should refresh the source automatically
when we change mappings.
2015-05-18 17:05:02 +02:00
Simon Willnauer 8aef499aaa review comments 2015-05-18 17:00:47 +02:00
Simon Willnauer 470e875445 Decrement reference even if IndexShard#postRecovery barfs
This can cause a reference leak if we call `IndexShard#postRecovery`
on an already closed shard.
2015-05-18 16:58:41 +02:00
Simon Willnauer bba1528fa4 [TEST] Enable MockFS on InternalEngineTests and TranslogTests 2015-05-18 16:58:19 +02:00
Simon Willnauer 7d1eb6e900 [ENGINE] Close Translog if initial commit fails 2015-05-18 16:52:49 +02:00
Adrien Grand a2a41c0115 Merge pull request #11207 from jpountz/fix/recovery_no_mapping_update
Recovery: No need to send mappings to the master node on phase 2.
2015-05-18 16:50:03 +02:00
Adrien Grand 33adbe6082 Recovery: No need to send mappings to the master node on phase 2.
Now that mapping updates are synchronous, it is not necessary to send mappings
to the master node during the recovery process anymore: they will already be on
the master node since we ensure mappings are on the master node before indexing.
2015-05-18 16:38:13 +02:00
Adrien Grand 2c241e8a36 Mappings: Remove the `ignore_conflicts` option.
Mappings conflicts should not be ignored. If I read the history correctly, this
option was added when a mapping update to an existing field was considered a
conflict, even if the new mapping was exactly the same. Now that mapping updates
are smart enough to detect conflicting options, we don't need an option to
ignore conflicts.
2015-05-18 15:28:23 +02:00
Alexander Reelsen 8bdfd21b70 Internal: Propagate headers & contexts to sub-requests
Whenever a query parser (or any other component) issues another
request as part of a request, the headers and the context has to
be supplied as well.

In order to do this, the `SearchContext` has to have those headers
available, which in turn means, the shard level request needs to
copy those from the original `SearchRequest`

This commit introduces two new interface to supply the needed methods
to work with context and headers.

Closes #10979
2015-05-18 14:50:16 +02:00
Simon Willnauer 98640310c2 Remove dead code / unused class 2015-05-18 14:32:36 +02:00
Britta Weber 64956d2d87 add check at end of each test to see if documents are the same 2015-05-18 12:19:20 +02:00
Britta Weber f039c5de7a add rest statuses but need to discuss if they are the right ones 2015-05-18 11:06:36 +02:00
Simon Willnauer 7892c7c869 remove flushing before running checkindex 2015-05-18 11:03:11 +02:00
Adrien Grand 61a31dd57f Tests: Add information about shards that are not closed when the filter cache gets closed. 2015-05-18 10:42:45 +02:00
Britta Weber c4f9a41581 more comments and formatting 2015-05-18 10:41:13 +02:00
Simon Willnauer 6c56cedf9a Prevent MockStore where commits should be prevented on close 2015-05-18 10:12:15 +02:00