Commit Graph

12198 Commits

Author SHA1 Message Date
Britta Weber 8a3d504efb skip everything between file checks and translog prepare if sync ids coincide
also throw exception instead of assert if num docs no coincide
2015-05-19 09:30:12 +02:00
Britta Weber 8ef734908c assert num docs in recovery if sync id was used 2015-05-19 09:30:12 +02:00
Adrien Grand 3ca3c6c09c Merge pull request #11209 from jpountz/fix/documentmapper_refreshsource_private
Mappings: Make DocumentMapper.refreshSource() private.
2015-05-19 08:49:52 +02:00
Adrien Grand f0d6dbbc36 Merge pull request #11210 from nik9000/detect_noop_null
detect_noop now understands null as a valid value
2015-05-19 08:38:49 +02:00
Ryan Ernst e95097c33c Mappings: Remove SmartNameFieldMappers
This change simplifies the users of the mapper service to no
longer have access to multiple fields for a single name. While
FieldMappersLookup still stores and gives access to multiple
fields, the current users of SmartNameFieldMappers already all
assumed a single field. The arbitrary selection of that field
when multiple exist is now isolated to the mapper service.
2015-05-18 22:41:56 -07:00
Robert Muir 0105b65160 Merge pull request #11212 from rmuir/java.io.tmpdir.early
Ensure java.io.tmpdir is created earlier in tests.
2015-05-18 20:50:37 -04:00
Robert Muir e89693408f Ensure java.io.tmpdir is created earlier in tests.
When testing, each jvm gets its own tmpdir set, so it may not exist
at all. A Lucene test rule ensures its created, but some tests (I
am looking at you rest tests) do a bunch of file stuff in static {},
in that case because its a parameterized test. And if you try to
extend it, it will fail if security manager is disabled...

Currently we ensure(java.io.tmpdir) very early when tests are running under
security manager, but otherwise we don't and it won't happen until the
test rule fires. So just do it early always.
2015-05-18 16:37:20 -04:00
Simon Willnauer 6dbe7d4b55 Make commitID byte[] final 2015-05-18 21:55:23 +02:00
Simon Willnauer 75b62e12a6 Merge pull request #11193 from s1monw/remove_commit_without_translog
[ENGINE] Remove the ability to flush without flushing the translog
2015-05-18 21:48:07 +02:00
Nik Everett 400abfceaf detect_noop now understands null as a valid value
If the source contrains a null value for a field then detect_noop should
consider setting it to null again to be a noop.

Closes #11208
2015-05-18 15:14:01 -04:00
Simon Willnauer 4b8ed1b0bf [TRANSLOG] Return null from read() if location generation is outdated
This behavior has changed been changed rescently to throw an IAE if
the translog we try to read from is already outdated. This is not
the expected behavior and this commit adds back the `old` way returning
`null` instead. The InternalEngine implementation will then go and ask the
lucene index for the document instead.
2015-05-18 20:14:39 +02:00
Alexander Reelsen 13e5c19dcf Test: Prevent automatic refresh in ContextAndHeaderTransportTests
In some cases due to calling checking `rarely()` the `indexRandom()` method
can potentially flush, which creates flush requests, that miss a certain
header in this test and allow the test to fail.

In addition unused configuration code for this test has been removed.
2015-05-18 18:22:48 +02:00
Adrien Grand 2999f53e86 Tests: Mark RelocationTests.testRelocationWithBusyClusterUpdateThread as @AwaitsFix. 2015-05-18 18:05:26 +02:00
Simon Willnauer 91d0580b0d Merge pull request #11200 from s1monw/cleanup_translog
Some smallish translog cleanups
2015-05-18 17:41:31 +02:00
Simon Willnauer a21e4449b7 Added additional assertion that the view to remove was actually registered. 2015-05-18 17:14:06 +02:00
Simon Willnauer bc810e1320 remove NoopOutputStream 2015-05-18 17:05:36 +02:00
Simon Willnauer 651c067fae Cleanup translog operation serialization
We used to double write the translog operation which is not needed except
of for recovery. This commit cuts over to a big-array based temporary serialiation
and removes the crazy double writing.
2015-05-18 17:05:36 +02:00
Simon Willnauer 60b66a7235 [TRANSLOG] Fold Translog.View into it's only implementation 2015-05-18 17:05:36 +02:00
Adrien Grand a75cfb42de Mappings: Make DocumentMapper.refreshSource() private.
This method should not be public, we should refresh the source automatically
when we change mappings.
2015-05-18 17:05:02 +02:00
Simon Willnauer 8aef499aaa review comments 2015-05-18 17:00:47 +02:00
Simon Willnauer 470e875445 Decrement reference even if IndexShard#postRecovery barfs
This can cause a reference leak if we call `IndexShard#postRecovery`
on an already closed shard.
2015-05-18 16:58:41 +02:00
Simon Willnauer bba1528fa4 [TEST] Enable MockFS on InternalEngineTests and TranslogTests 2015-05-18 16:58:19 +02:00
Simon Willnauer 7d1eb6e900 [ENGINE] Close Translog if initial commit fails 2015-05-18 16:52:49 +02:00
Adrien Grand a2a41c0115 Merge pull request #11207 from jpountz/fix/recovery_no_mapping_update
Recovery: No need to send mappings to the master node on phase 2.
2015-05-18 16:50:03 +02:00
Adrien Grand 33adbe6082 Recovery: No need to send mappings to the master node on phase 2.
Now that mapping updates are synchronous, it is not necessary to send mappings
to the master node during the recovery process anymore: they will already be on
the master node since we ensure mappings are on the master node before indexing.
2015-05-18 16:38:13 +02:00
Adrien Grand 2c241e8a36 Mappings: Remove the `ignore_conflicts` option.
Mappings conflicts should not be ignored. If I read the history correctly, this
option was added when a mapping update to an existing field was considered a
conflict, even if the new mapping was exactly the same. Now that mapping updates
are smart enough to detect conflicting options, we don't need an option to
ignore conflicts.
2015-05-18 15:28:23 +02:00
Alexander Reelsen 8bdfd21b70 Internal: Propagate headers & contexts to sub-requests
Whenever a query parser (or any other component) issues another
request as part of a request, the headers and the context has to
be supplied as well.

In order to do this, the `SearchContext` has to have those headers
available, which in turn means, the shard level request needs to
copy those from the original `SearchRequest`

This commit introduces two new interface to supply the needed methods
to work with context and headers.

Closes #10979
2015-05-18 14:50:16 +02:00
Simon Willnauer 98640310c2 Remove dead code / unused class 2015-05-18 14:32:36 +02:00
Britta Weber 64956d2d87 add check at end of each test to see if documents are the same 2015-05-18 12:19:20 +02:00
Britta Weber f039c5de7a add rest statuses but need to discuss if they are the right ones 2015-05-18 11:06:36 +02:00
Simon Willnauer 7892c7c869 remove flushing before running checkindex 2015-05-18 11:03:11 +02:00
Adrien Grand 61a31dd57f Tests: Add information about shards that are not closed when the filter cache gets closed. 2015-05-18 10:42:45 +02:00
Britta Weber c4f9a41581 more comments and formatting 2015-05-18 10:41:13 +02:00
Simon Willnauer 6c56cedf9a Prevent MockStore where commits should be prevented on close 2015-05-18 10:12:15 +02:00
Simon Willnauer 03e5149994 [ENGINE] Remove the ability to flush without flushing the translog
This is a leftover from the times where we failed a flush when
recoveries are ongoing. This code is really not needed anymore and
we can luckily flush the translog all the time as well.
2015-05-18 09:56:25 +02:00
Simon Willnauer 8b8ba9a44f Implement toXContent on ShardOpertionFailureException
ShardOperationFailureException implementations alread provide structured
exception support but it's not yet exposed on the interface. This change
allows nice rendering of structured REST exceptions also if searches fail on
only a subset of the shards etc.

Closes #11017
2015-05-18 09:54:04 +02:00
Simon Willnauer f7696ec149 Merge pull request #11143 from elastic/feature/translog_checkpoints
Add translog checkpoints to prevent translog corruption

Closes to #10933
Relates to #11011
2015-05-18 09:42:51 +02:00
Adrien Grand b16e47c5fb Tests: Mark IndexCacheableQueryTests.testCache as @AwaitsFix. 2015-05-18 08:38:10 +02:00
Britta Weber 6ed472f873 comments for SyncedFlushService 2015-05-17 22:12:31 +02:00
Britta Weber 9ca17f834b inherit from BroadcastOperationRequest 2015-05-17 21:42:27 +02:00
Britta Weber 771763ae58 add error for inactive primary 2015-05-17 21:34:35 +02:00
Britta Weber a0315b5cb8 remove FAILED suffix 2015-05-17 21:29:31 +02:00
Britta Weber fcea10113c remove impl details from method name 2015-05-17 21:29:31 +02:00
Britta Weber 5ac07481eb assign variables 2015-05-17 21:29:31 +02:00
Britta Weber 018ad0bd76 final 2015-05-17 21:29:30 +02:00
Britta Weber 81e3d5cdcb use imutable object for commit id 2015-05-17 21:29:24 +02:00
Simon Willnauer 54ae0a557d [TEST] use correct index name created for this test 2015-05-17 20:26:13 +02:00
Simon Willnauer f9f0e99eae Merge branch 'master' into feature/translog_checkpoints
Conflicts:
	src/main/java/org/elasticsearch/index/engine/InternalEngine.java
2015-05-17 20:11:19 +02:00
Igor Motov 1038933832 Snapshot/Restore: Fix cluster state task name for update snapshot task
This commit fixes the name of the upated_snapshot task from something like "update_snapshot [org.elasticsearch.cluster.metadata.SnapshotMetaData$Entry@de00bc50]" to a more readable "update_snapshot [test-snap]"
2015-05-15 19:54:35 -04:00
Lee Hinman 22ae561bfd [GATEWAY] When using `recover_on_any_node`, respect Deciders
When in a shared filesystem environment and recovering the primary to
any node. We should respect the allocation deciders if possible (still
force-allocting to another node if there aren't any "YES" decisions).

The AllocationDeciders should take precedence over the shard state
version when force-allocating an unassigned primary shard.
2015-05-15 16:08:03 -06:00