Commit Graph

12016 Commits

Author SHA1 Message Date
Simon Willnauer b3f1a5da4a Merge pull request #11225 from s1monw/remove_flush_needed
[ENGINE] Remove flushNeeded in favor of IW#hasUncommittedChanges()
2015-05-19 14:53:09 +02:00
Shay Banon 643c2049a2 [TEST] add await fix for #11226 2015-05-19 14:40:02 +02:00
Simon Willnauer d6e7817666 [ENGINE] Remove flushNeeded in favor of IW#hasUncommittedChanges()
We maintain a boolean that is obsolete since IW maintains the same
property that we can just piggyback on.
2015-05-19 14:10:07 +02:00
Shay Banon a1a16276da [TEST] Add a corrupted replica test verifying its still allocated
Add a test that verifies that even though all replicas are corrupted on all available nodes, and listing of shard stores faield, it still get allocated and properly recovered from the primary shard
2015-05-19 13:18:43 +02:00
Martijn van Groningen 84b24e130e inner hits: Fix search hit field and term resolution 2015-05-19 11:25:52 +02:00
Adrien Grand 17d65a5f90 Tests: Remove RelocationTests.testRelocationWithBusyClusterUpdateThread.
It is not relevant anymore with synchronous mapping updates.
2015-05-19 10:37:45 +02:00
Adrien Grand df5ffb1418 Merge pull request #11218 from jpountz/upgrade/lucene-5.2.0-snapshot-1680200
Upgrade to lucene-5.2.0-snapshot-1680200.
2015-05-19 10:34:32 +02:00
Adrien Grand 437910e882 Upgrade to lucene-5.2.0-snapshot-1680200. 2015-05-19 10:28:35 +02:00
Adrien Grand 4dc4566168 Merge pull request #11203 from jpountz/fix/remove_ignore_conflicts
Mappings: Remove the `ignore_conflicts` option.
2015-05-19 10:03:02 +02:00
Adrien Grand 3ca3c6c09c Merge pull request #11209 from jpountz/fix/documentmapper_refreshsource_private
Mappings: Make DocumentMapper.refreshSource() private.
2015-05-19 08:49:52 +02:00
Adrien Grand f0d6dbbc36 Merge pull request #11210 from nik9000/detect_noop_null
detect_noop now understands null as a valid value
2015-05-19 08:38:49 +02:00
Robert Muir 0105b65160 Merge pull request #11212 from rmuir/java.io.tmpdir.early
Ensure java.io.tmpdir is created earlier in tests.
2015-05-18 20:50:37 -04:00
Robert Muir e89693408f Ensure java.io.tmpdir is created earlier in tests.
When testing, each jvm gets its own tmpdir set, so it may not exist
at all. A Lucene test rule ensures its created, but some tests (I
am looking at you rest tests) do a bunch of file stuff in static {},
in that case because its a parameterized test. And if you try to
extend it, it will fail if security manager is disabled...

Currently we ensure(java.io.tmpdir) very early when tests are running under
security manager, but otherwise we don't and it won't happen until the
test rule fires. So just do it early always.
2015-05-18 16:37:20 -04:00
Simon Willnauer 75b62e12a6 Merge pull request #11193 from s1monw/remove_commit_without_translog
[ENGINE] Remove the ability to flush without flushing the translog
2015-05-18 21:48:07 +02:00
Nik Everett 400abfceaf detect_noop now understands null as a valid value
If the source contrains a null value for a field then detect_noop should
consider setting it to null again to be a noop.

Closes #11208
2015-05-18 15:14:01 -04:00
Simon Willnauer 4b8ed1b0bf [TRANSLOG] Return null from read() if location generation is outdated
This behavior has changed been changed rescently to throw an IAE if
the translog we try to read from is already outdated. This is not
the expected behavior and this commit adds back the `old` way returning
`null` instead. The InternalEngine implementation will then go and ask the
lucene index for the document instead.
2015-05-18 20:14:39 +02:00
Alexander Reelsen 13e5c19dcf Test: Prevent automatic refresh in ContextAndHeaderTransportTests
In some cases due to calling checking `rarely()` the `indexRandom()` method
can potentially flush, which creates flush requests, that miss a certain
header in this test and allow the test to fail.

In addition unused configuration code for this test has been removed.
2015-05-18 18:22:48 +02:00
Adrien Grand 2999f53e86 Tests: Mark RelocationTests.testRelocationWithBusyClusterUpdateThread as @AwaitsFix. 2015-05-18 18:05:26 +02:00
Simon Willnauer 91d0580b0d Merge pull request #11200 from s1monw/cleanup_translog
Some smallish translog cleanups
2015-05-18 17:41:31 +02:00
Simon Willnauer a21e4449b7 Added additional assertion that the view to remove was actually registered. 2015-05-18 17:14:06 +02:00
Simon Willnauer bc810e1320 remove NoopOutputStream 2015-05-18 17:05:36 +02:00
Simon Willnauer 651c067fae Cleanup translog operation serialization
We used to double write the translog operation which is not needed except
of for recovery. This commit cuts over to a big-array based temporary serialiation
and removes the crazy double writing.
2015-05-18 17:05:36 +02:00
Simon Willnauer 60b66a7235 [TRANSLOG] Fold Translog.View into it's only implementation 2015-05-18 17:05:36 +02:00
Adrien Grand a75cfb42de Mappings: Make DocumentMapper.refreshSource() private.
This method should not be public, we should refresh the source automatically
when we change mappings.
2015-05-18 17:05:02 +02:00
Simon Willnauer 8aef499aaa review comments 2015-05-18 17:00:47 +02:00
Simon Willnauer 470e875445 Decrement reference even if IndexShard#postRecovery barfs
This can cause a reference leak if we call `IndexShard#postRecovery`
on an already closed shard.
2015-05-18 16:58:41 +02:00
Simon Willnauer bba1528fa4 [TEST] Enable MockFS on InternalEngineTests and TranslogTests 2015-05-18 16:58:19 +02:00
Simon Willnauer 7d1eb6e900 [ENGINE] Close Translog if initial commit fails 2015-05-18 16:52:49 +02:00
Adrien Grand a2a41c0115 Merge pull request #11207 from jpountz/fix/recovery_no_mapping_update
Recovery: No need to send mappings to the master node on phase 2.
2015-05-18 16:50:03 +02:00
Adrien Grand 33adbe6082 Recovery: No need to send mappings to the master node on phase 2.
Now that mapping updates are synchronous, it is not necessary to send mappings
to the master node during the recovery process anymore: they will already be on
the master node since we ensure mappings are on the master node before indexing.
2015-05-18 16:38:13 +02:00
Adrien Grand 2c241e8a36 Mappings: Remove the `ignore_conflicts` option.
Mappings conflicts should not be ignored. If I read the history correctly, this
option was added when a mapping update to an existing field was considered a
conflict, even if the new mapping was exactly the same. Now that mapping updates
are smart enough to detect conflicting options, we don't need an option to
ignore conflicts.
2015-05-18 15:28:23 +02:00
Alexander Reelsen 8bdfd21b70 Internal: Propagate headers & contexts to sub-requests
Whenever a query parser (or any other component) issues another
request as part of a request, the headers and the context has to
be supplied as well.

In order to do this, the `SearchContext` has to have those headers
available, which in turn means, the shard level request needs to
copy those from the original `SearchRequest`

This commit introduces two new interface to supply the needed methods
to work with context and headers.

Closes #10979
2015-05-18 14:50:16 +02:00
Simon Willnauer 98640310c2 Remove dead code / unused class 2015-05-18 14:32:36 +02:00
Simon Willnauer 7892c7c869 remove flushing before running checkindex 2015-05-18 11:03:11 +02:00
Adrien Grand 61a31dd57f Tests: Add information about shards that are not closed when the filter cache gets closed. 2015-05-18 10:42:45 +02:00
Simon Willnauer 6c56cedf9a Prevent MockStore where commits should be prevented on close 2015-05-18 10:12:15 +02:00
Simon Willnauer 03e5149994 [ENGINE] Remove the ability to flush without flushing the translog
This is a leftover from the times where we failed a flush when
recoveries are ongoing. This code is really not needed anymore and
we can luckily flush the translog all the time as well.
2015-05-18 09:56:25 +02:00
Simon Willnauer 8b8ba9a44f Implement toXContent on ShardOpertionFailureException
ShardOperationFailureException implementations alread provide structured
exception support but it's not yet exposed on the interface. This change
allows nice rendering of structured REST exceptions also if searches fail on
only a subset of the shards etc.

Closes #11017
2015-05-18 09:54:04 +02:00
Simon Willnauer f7696ec149 Merge pull request #11143 from elastic/feature/translog_checkpoints
Add translog checkpoints to prevent translog corruption

Closes to #10933
Relates to #11011
2015-05-18 09:42:51 +02:00
Adrien Grand b16e47c5fb Tests: Mark IndexCacheableQueryTests.testCache as @AwaitsFix. 2015-05-18 08:38:10 +02:00
Simon Willnauer 54ae0a557d [TEST] use correct index name created for this test 2015-05-17 20:26:13 +02:00
Simon Willnauer f9f0e99eae Merge branch 'master' into feature/translog_checkpoints
Conflicts:
	src/main/java/org/elasticsearch/index/engine/InternalEngine.java
2015-05-17 20:11:19 +02:00
Igor Motov 1038933832 Snapshot/Restore: Fix cluster state task name for update snapshot task
This commit fixes the name of the upated_snapshot task from something like "update_snapshot [org.elasticsearch.cluster.metadata.SnapshotMetaData$Entry@de00bc50]" to a more readable "update_snapshot [test-snap]"
2015-05-15 19:54:35 -04:00
Lee Hinman 22ae561bfd [GATEWAY] When using `recover_on_any_node`, respect Deciders
When in a shared filesystem environment and recovering the primary to
any node. We should respect the allocation deciders if possible (still
force-allocting to another node if there aren't any "YES" decisions).

The AllocationDeciders should take precedence over the shard state
version when force-allocating an unassigned primary shard.
2015-05-15 16:08:03 -06:00
Robert Muir f241236af0 Merge pull request #11194 from rmuir/mockito
allow mockito to be used in tests
2015-05-15 16:52:44 -04:00
Robert Muir b2a0ed3878 allow mockito to be used in tests 2015-05-15 16:33:24 -04:00
Robert Muir ae1d7e4ceb Merge branch 'master' of github.com:rmuir/elasticsearch 2015-05-15 16:33:08 -04:00
Robert Muir fc73540aac include rest tests in test-framework.jar
Closes #11192 which I accidentally already closed.

Squashed commit of the following:

commit f23faccddc2a77a880841da4c89c494edaa2aa46
Author: Robert Muir <rmuir@apache.org>
Date:   Fri May 15 16:04:55 2015 -0400

    Simplify this FileUtils even more: its either from the filesystem, or the classpath,
    not both. Its already trying 4 different combinations of crazy paths for either of these anyway.

commit c7016c8a2b5a6043e2ded4b48b160821ba196974
Author: Robert Muir <rmuir@apache.org>
Date:   Fri May 15 14:21:37 2015 -0400

    include rest tests in test-framework jar
2015-05-15 16:16:24 -04:00
Robert Muir 97ffb2b4fd include rest tests in test-framework.jar
Closes #11192 which I accidentally already closed.

Squashed commit of the following:

commit f23faccddc2a77a880841da4c89c494edaa2aa46
Author: Robert Muir <rmuir@apache.org>
Date:   Fri May 15 16:04:55 2015 -0400

    Simplify this FileUtils even more: its either from the filesystem, or the classpath,
    not both. Its already trying 4 different combinations of crazy paths for either of these anyway.

commit c7016c8a2b5a6043e2ded4b48b160821ba196974
Author: Robert Muir <rmuir@apache.org>
Date:   Fri May 15 14:21:37 2015 -0400

    include rest tests in test-framework jar
2015-05-15 16:15:23 -04:00
Simon Willnauer 217fd4443a Make SearchFactory static class in InternalEngine
Now that lucene provides a way to identify if the warming reader is
the first initial opened reader we can detach this class from the
enclosing and make it static. This is important since it might access
not fully initialized members of the enclosing class since it's initialized
and used during constructor invocation.
2015-05-15 22:02:00 +02:00