Commit Graph

504 Commits

Author SHA1 Message Date
Shay Banon cd7096dfc2 Replace primaryPostAllocated flag and use UnassignedInfo
There is no need to maintain additional state as to if a primary was allocated post api creation on the index routing table, we hold all this information already in the UnassignedInfo class.
closes #12374
2015-07-21 17:00:07 +02:00
Alexander Reelsen d3e454780f PluginManager: Fix bin/plugin calls in scripts/bats test
The release and smoke test python scripts used to install
plugins in the old fashion.

Also the BATS testing suite installed/removed plugins in that
way. Here the marvel tests have been removed, as marvel currently
does not work with the master branch.

In addition documentation has been updated as well, where it was
still missing.
2015-07-21 16:17:59 +02:00
Boaz Leskes 5def4193e9 Discovery: make sure NodeJoinController.ElectionCallback is always called from the update cluster state thread
This is important for correct handling of the joining thread. This causes assertions to trip in our test runs. See http://build-us-00.elastic.co/job/es_g1gc_master_metal/11653/ as an example

Closes #12372
2015-07-21 16:11:37 +02:00
Boaz Leskes 7adf9929ba Remove left over no commit from TransportReplicationAction
It asks to double check thread pool rejection. I did and don't see problems with it.
2015-07-21 15:54:36 +02:00
Alexander Reelsen 999d5ab801 CliTool: Various PluginManager fixes
The new plugin manager parser was not called correctly in the scripts.
In addition the plugin manager now creates a plugins/ directory in case
it does not exist.

Also the integration tests called the plugin manager in the deprecated way.
2015-07-21 15:31:57 +02:00
Alexander Reelsen 2f54b89a23 CLITool: Port PluginManager to use CLITool
In order to unify the handling and reuse the CLITool infrastructure
the plugin manager should make use of this as well.

This obsolets the -i and --install options but requires the user
to use `install` as the first argument of the CLI.

This is basically just a port of the existing functionality, which
is also the reason why this is not a refactoring of the plugin manager,
which will come in a separate commit.
2015-07-21 14:15:39 +02:00
Martijn van Groningen 0cf9c3268c Remove custom execute local logic in TransportSingleShardAction and TransportInstanceSingleOperationAction and rely on transport service to execute locally. (forking thread etc.)
Change TransportInstanceSingleOperationAction to have shardActionHandler to, so we can execute locally without endless spinning.
2015-07-21 13:45:42 +02:00
Alexander Reelsen 95f56d71bd Logging: Corrected potential confusing log message 2015-07-21 11:30:26 +02:00
Ryan Ernst e52a316609 Test: fix tests referring to _index 2015-07-21 01:24:49 -07:00
Ryan Ernst ededbf0c8b Merge pull request #12352 from rjernst/fix/index-name-in-mapper-service
Remove index name from mapping parser
2015-07-21 00:31:38 -07:00
Ryan Ernst 8cd03cce5e Merge branch 'master' into fix/12329 2015-07-21 00:29:34 -07:00
Boaz Leskes bb1318512b Test: move IndicesShardStoreRequestTests to the right package 2015-07-21 09:01:50 +02:00
Boaz Leskes 29c99ff23d Logging: shard store shouldn't log the entire cluster state under trace, but just its version.
If we want the entire CS, we can put the cluster.service on trace
2015-07-21 08:58:50 +02:00
Ryan Ernst 1c99626b84 Mappings: Remove ability to configure _index
The `_index` field is now a completely virtual field thanks
to #12027. It is no longer necessary to index the actual value
of the index name.

closes #12329
2015-07-20 23:54:35 -07:00
Ryan Ernst abcf269539 Remove old DocumentMapper.parse method 2015-07-20 22:37:44 -07:00
Areek Zillur ae750bca69 [TEST] add missing ensureYellow to IndicesShardStoreRequestTests.testBasic 2015-07-20 23:52:07 -04:00
Ryan Ernst 2ad682e8b4 Mappings: Remove index name from mapping parser
The index name was passed along through many levels of mapping parsing,
just so that it could be used for _index. However, the index name
is really metadata that should exist alongside things like type and
id in SourceToParse.

This change moves index name to SourceToParse, and eliminates it from the
DocumentMapperParser.
2015-07-20 17:50:07 -07:00
Martijn van Groningen 4e2bdb67b7 test: enable get field mapping tests 2015-07-20 23:38:47 +02:00
Areek Zillur e964b0c2ab [TEST] use official API to read cluster state and enable debug logging 2015-07-20 17:12:30 -04:00
Boaz Leskes 66e71da7bd Test: testTimeoutCleanup use assertBusy while checking timer queue 2015-07-20 22:38:53 +02:00
Jason Tedor 34155ff102 Merge pull request #12346 from jasontedor/fix/12345
Use time with nanosecond resolution calculated at the executing node to measure the time that contexts are held open
2015-07-20 15:54:54 -04:00
Boaz Leskes 068fae218f Test: testShardActiveElseWhere shouldn't force cluster state publishing
The test is supposed to fool a node, making it think all it's shards are allocated else where. The test verifies the node doesn't delete its shards. The current test submitted this "fake" CS to the master, causing it to published which in turn caused confusion on other nodes and test failures.

 See http://build-us-00.elastic.co/job/es_core_master_oracle_6/990/testReport/junit/org.elasticsearch.indices.store/IndicesStoreIntegrationTests/testShardActiveElseWhere/
2015-07-20 21:16:16 +02:00
Zachary Tong 9951db9c64 [TEST] Better floating point comparisons 2015-07-20 14:46:24 -04:00
Jason Tedor 3af763eb88 Use time with nanosecond resolution calculated at the executing node to measure the time that contexts are held open
Closes #12345
2015-07-20 12:42:13 -04:00
Martijn van Groningen 4c7bb17521 test: increased logging 2015-07-20 17:35:39 +02:00
Robert Muir cd9d1c617f Merge pull request #12344 from rmuir/2.1.16
Update randomizedtesting to 2.1.16
2015-07-20 11:13:57 -04:00
Robert Muir a0a6299840 Update randomizedtesting to 2.1.16 2015-07-20 11:02:50 -04:00
Shay Banon 32445bbc3a Simplify handling of ignored unassigned shards
Fold ignored unassigned to a UnassignedShards and have simpler handling of them. Also remove the trapy way of adding an ignored unassigned shards today directly to the list, and have dedicated methods for it.

This change also removes the useless moving of unassigned shards to the end, since anyhow we first, sort those unassigned shards, and second, we now have persistent "store exceptions" that should not cause "dead letter" shard allocation.
2015-07-20 15:43:31 +02:00
Shay Banon ca3e0c6d49 Initial Refactor Gateway Allocator
Break it into more manageable code by separating allocation primaries and allocating replicas. Start adding basic unit tests for primary shard allocator.
2015-07-20 14:04:24 +02:00
Boaz Leskes 64750fa8bb Test: firstAllocationFailureTwoNodes used hard coded node reference
The test starts two nodes and two shards and expects the primary shard to be allocated on the first node. Very rarely this is not the case.
2015-07-20 12:23:12 +02:00
Boaz Leskes 916466a9be add await fix firstAllocationFailureTwoNodes 2015-07-20 12:04:37 +02:00
Jason Tedor 964a849016 Merge pull request #12331 from jasontedor/feature/12330
Add scroll stats to cat API
2015-07-20 05:23:04 -04:00
Boaz Leskes e40eee557f ThreadPools: schedule a timeout check after adding command to queue
Our thread pools have support for timeout on a task. To support this, a special background task is schedule to run at timeout. That background task fires and check if the main task is still in the executor queue and then cancels it if needed. Currently we schedule this background task before adding the main task to the queue. If the timeout is very small (in tests we often use numbers like 2 ms)  the background task can fire before the main one is added to the queue causing the timeout to be missed.

See http://build-us-00.elastic.co/job/es_g1gc_master_metal/11780/testReport/junit/org.elasticsearch.cluster/ClusterServiceTests/testTimeoutUpdateTask/

Closes #12319
2015-07-20 10:51:03 +02:00
Boaz Leskes 275848eb9b Allocation: use recently added allocation ids for shard started/failed
On top of that:
 1) A relocation target shards' allocation id is changed to include the allocation id of the source shard under relocatingId (similar to shard routing semantics)
 2) The logic around state change for finalize shard relocation is simplified - one simple start the target shard (we previously had unused logic around relocating state)

Closes #12299
2015-07-20 10:31:31 +02:00
Jason Tedor c4778b8e78 Remove unused imports 2015-07-19 14:12:42 -04:00
Jason Tedor 8637784a2a Add scroll stats to cat API
Adds scroll stats at the node, shard and index levels to the cat API.

Closes #12330
2015-07-19 13:41:46 -04:00
Robert Muir 6dc5d402fe Merge branch 'master' into maven_run 2015-07-17 15:00:37 -04:00
Tanguy Leroux 468a2b1167 Update license SHA1 file for Netty 3.10.3.Final 2015-07-17 17:51:08 +02:00
Nicholas Knize bc9a4707db [GEO] Update ShapeBuilder and GeoPolygonQueryParser to accept unclosed GeoJSON
While the GeoJSON spec does say a polygon is represented as an array of LinearRings (where a LinearRing is defined as a 'closed' array of points), the coerce parameter provides users with flexibility to have ES automatically close polygons. This addresses situations like those integrated with twitter (where GeoJSON polygons are not closed) such that our users do not have to write extra code to close the polygon. This code change adds the optional coerce parameter to the GeoShapeFieldMapper.

closes #11131
2015-07-17 10:41:27 -05:00
Adrien Grand c483a221b7 Regenerate 1.6.1 and 1.7.1 backward indices to include the `bool` field. 2015-07-17 15:34:26 +02:00
Jason Tedor e379d20ae6 Merge pull request #12211 from jasontedor/features/12149
Add global search timeout setting
2015-07-17 09:23:16 -04:00
Martijn van Groningen ba16dec342 test: added 1.6.1 and 1.7.0 bwcompat indices 2015-07-17 14:29:19 +02:00
Martijn van Groningen 1fc8b429f3 added 1.7.1 and 1.6.2 versions 2015-07-17 13:00:00 +02:00
Boaz Leskes 1b37b48bb0 Tests: removed comment from SearchWhileRelocatingTests 2015-07-17 11:32:36 +02:00
Robert Muir 48bc49c093 Remove broken `exec` build target, replace with something better.
Currently this target is "yet another way" to run elasticsearch,
which we can't maintain. It also has the problem that it doesnt
ensure its running on the latest source code, doesn't configure
any scratch space properly, won't work with securitymanager, list
goes on.

Even if we made it work, it would break every day, since its untested.

Instead, `mvn package -Drun -DskipTests` will run packaging, and then
startup bin/elasticsearch (like integration tests, but in foreground).

It also enables debugger socket on port 8000, for people that like
IDE debuggers and not system.out.println.

Its a little slower to get started because of all the shading/RPM/DEB
building going on in `package` but that is just what it is right now
until that stuff is moved out.
2015-07-17 00:03:33 -04:00
Areek Zillur b7107663d1 [TEST] remove redundant import 2015-07-17 00:01:53 -04:00
Areek Zillur 98fd8d94fe [TEST] take into account when master node is stopped 2015-07-17 00:00:47 -04:00
Ryan Ernst b420c96fda quiet flaky test 2015-07-16 19:27:23 -07:00
Robert Muir b01c17959e fix REPRODUCE WITH for integration tests 2015-07-16 21:45:20 -04:00
Robert Muir 0ed28738ac Merge pull request #12302 from rmuir/junit4_integ
Use junit4 for running integration tests, too
2015-07-16 21:23:10 -04:00