709 Commits

Author SHA1 Message Date
Shay Banon
cd7096dfc2 Replace primaryPostAllocated flag and use UnassignedInfo
There is no need to maintain additional state as to if a primary was allocated post api creation on the index routing table, we hold all this information already in the UnassignedInfo class.
closes #12374
2015-07-21 17:00:07 +02:00
Alexander Reelsen
d3e454780f PluginManager: Fix bin/plugin calls in scripts/bats test
The release and smoke test python scripts used to install
plugins in the old fashion.

Also the BATS testing suite installed/removed plugins in that
way. Here the marvel tests have been removed, as marvel currently
does not work with the master branch.

In addition documentation has been updated as well, where it was
still missing.
2015-07-21 16:17:59 +02:00
Boaz Leskes
5def4193e9 Discovery: make sure NodeJoinController.ElectionCallback is always called from the update cluster state thread
This is important for correct handling of the joining thread. This causes assertions to trip in our test runs. See http://build-us-00.elastic.co/job/es_g1gc_master_metal/11653/ as an example

Closes #12372
2015-07-21 16:11:37 +02:00
Boaz Leskes
7adf9929ba Remove left over no commit from TransportReplicationAction
It asks to double check thread pool rejection. I did and don't see problems with it.
2015-07-21 15:54:36 +02:00
Adrien Grand
3fd7a36f91 Remove the dependecy on IndexFielddataService from MapperService.
This dependency was used in order for mapping updates that change the fielddata
format to take effect immediately. And the way it worked was by clearing the
cache of fielddata instances that were already loaded. However, we do not need
to cache the already loaded (logical) fielddata instances, they are cheap to
regenerate. Note that the fielddata _caches_ are still kept around so that we
don't keep on rebuilding costly (physical) fielddata values.
2015-07-21 15:39:58 +02:00
Alexander Reelsen
999d5ab801 CliTool: Various PluginManager fixes
The new plugin manager parser was not called correctly in the scripts.
In addition the plugin manager now creates a plugins/ directory in case
it does not exist.

Also the integration tests called the plugin manager in the deprecated way.
2015-07-21 15:31:57 +02:00
Alexander Reelsen
2f54b89a23 CLITool: Port PluginManager to use CLITool
In order to unify the handling and reuse the CLITool infrastructure
the plugin manager should make use of this as well.

This obsolets the -i and --install options but requires the user
to use `install` as the first argument of the CLI.

This is basically just a port of the existing functionality, which
is also the reason why this is not a refactoring of the plugin manager,
which will come in a separate commit.
2015-07-21 14:15:39 +02:00
Ruflin
3be98927a0 Fix cidr mask conversion issue for 0.0.0.0/0 and add tests 2015-07-21 13:53:16 +02:00
Martijn van Groningen
0cf9c3268c Remove custom execute local logic in TransportSingleShardAction and TransportInstanceSingleOperationAction and rely on transport service to execute locally. (forking thread etc.)
Change TransportInstanceSingleOperationAction to have shardActionHandler to, so we can execute locally without endless spinning.
2015-07-21 13:45:42 +02:00
Alexander Reelsen
95f56d71bd Logging: Corrected potential confusing log message 2015-07-21 11:30:26 +02:00
Ryan Ernst
e52a316609 Test: fix tests referring to _index 2015-07-21 01:24:49 -07:00
Ryan Ernst
ededbf0c8b Merge pull request #12352 from rjernst/fix/index-name-in-mapper-service
Remove index name from mapping parser
2015-07-21 00:31:38 -07:00
Ryan Ernst
8cd03cce5e Merge branch 'master' into fix/12329 2015-07-21 00:29:34 -07:00
Boaz Leskes
bb1318512b Test: move IndicesShardStoreRequestTests to the right package 2015-07-21 09:01:50 +02:00
Boaz Leskes
29c99ff23d Logging: shard store shouldn't log the entire cluster state under trace, but just its version.
If we want the entire CS, we can put the cluster.service on trace
2015-07-21 08:58:50 +02:00
Ryan Ernst
1c99626b84 Mappings: Remove ability to configure _index
The `_index` field is now a completely virtual field thanks
to #12027. It is no longer necessary to index the actual value
of the index name.

closes #12329
2015-07-20 23:54:35 -07:00
Ryan Ernst
abcf269539 Remove old DocumentMapper.parse method 2015-07-20 22:37:44 -07:00
Areek Zillur
ae750bca69 [TEST] add missing ensureYellow to IndicesShardStoreRequestTests.testBasic 2015-07-20 23:52:07 -04:00
Ryan Ernst
2ad682e8b4 Mappings: Remove index name from mapping parser
The index name was passed along through many levels of mapping parsing,
just so that it could be used for _index. However, the index name
is really metadata that should exist alongside things like type and
id in SourceToParse.

This change moves index name to SourceToParse, and eliminates it from the
DocumentMapperParser.
2015-07-20 17:50:07 -07:00
Robert Muir
052cf1446f Remove Environment.homeFile()
Today we grant read+write+delete access to any files underneath the home.
But we have to remove this, if we want to have improved security of files
underneath elasticsearch.
2015-07-20 20:27:36 -04:00
Martijn van Groningen
4e2bdb67b7 test: enable get field mapping tests 2015-07-20 23:38:47 +02:00
Areek Zillur
e964b0c2ab [TEST] use official API to read cluster state and enable debug logging 2015-07-20 17:12:30 -04:00
Boaz Leskes
66e71da7bd Test: testTimeoutCleanup use assertBusy while checking timer queue 2015-07-20 22:38:53 +02:00
Jason Tedor
34155ff102 Merge pull request #12346 from jasontedor/fix/12345
Use time with nanosecond resolution calculated at the executing node to measure the time that contexts are held open
2015-07-20 15:54:54 -04:00
Boaz Leskes
068fae218f Test: testShardActiveElseWhere shouldn't force cluster state publishing
The test is supposed to fool a node, making it think all it's shards are allocated else where. The test verifies the node doesn't delete its shards. The current test submitted this "fake" CS to the master, causing it to published which in turn caused confusion on other nodes and test failures.

 See http://build-us-00.elastic.co/job/es_core_master_oracle_6/990/testReport/junit/org.elasticsearch.indices.store/IndicesStoreIntegrationTests/testShardActiveElseWhere/
2015-07-20 21:16:16 +02:00
Zachary Tong
9951db9c64 [TEST] Better floating point comparisons 2015-07-20 14:46:24 -04:00
Jason Tedor
3af763eb88 Use time with nanosecond resolution calculated at the executing node to measure the time that contexts are held open
Closes #12345
2015-07-20 12:42:13 -04:00
Martijn van Groningen
4c7bb17521 test: increased logging 2015-07-20 17:35:39 +02:00
Robert Muir
cd9d1c617f Merge pull request #12344 from rmuir/2.1.16
Update randomizedtesting to 2.1.16
2015-07-20 11:13:57 -04:00
Robert Muir
a0a6299840 Update randomizedtesting to 2.1.16 2015-07-20 11:02:50 -04:00
Shay Banon
32445bbc3a Simplify handling of ignored unassigned shards
Fold ignored unassigned to a UnassignedShards and have simpler handling of them. Also remove the trapy way of adding an ignored unassigned shards today directly to the list, and have dedicated methods for it.

This change also removes the useless moving of unassigned shards to the end, since anyhow we first, sort those unassigned shards, and second, we now have persistent "store exceptions" that should not cause "dead letter" shard allocation.
2015-07-20 15:43:31 +02:00
Shay Banon
ca3e0c6d49 Initial Refactor Gateway Allocator
Break it into more manageable code by separating allocation primaries and allocating replicas. Start adding basic unit tests for primary shard allocator.
2015-07-20 14:04:24 +02:00
Boaz Leskes
64750fa8bb Test: firstAllocationFailureTwoNodes used hard coded node reference
The test starts two nodes and two shards and expects the primary shard to be allocated on the first node. Very rarely this is not the case.
2015-07-20 12:23:12 +02:00
Boaz Leskes
916466a9be add await fix firstAllocationFailureTwoNodes 2015-07-20 12:04:37 +02:00
Jason Tedor
964a849016 Merge pull request #12331 from jasontedor/feature/12330
Add scroll stats to cat API
2015-07-20 05:23:04 -04:00
Boaz Leskes
e40eee557f ThreadPools: schedule a timeout check after adding command to queue
Our thread pools have support for timeout on a task. To support this, a special background task is schedule to run at timeout. That background task fires and check if the main task is still in the executor queue and then cancels it if needed. Currently we schedule this background task before adding the main task to the queue. If the timeout is very small (in tests we often use numbers like 2 ms)  the background task can fire before the main one is added to the queue causing the timeout to be missed.

See http://build-us-00.elastic.co/job/es_g1gc_master_metal/11780/testReport/junit/org.elasticsearch.cluster/ClusterServiceTests/testTimeoutUpdateTask/

Closes #12319
2015-07-20 10:51:03 +02:00
Boaz Leskes
275848eb9b Allocation: use recently added allocation ids for shard started/failed
On top of that:
 1) A relocation target shards' allocation id is changed to include the allocation id of the source shard under relocatingId (similar to shard routing semantics)
 2) The logic around state change for finalize shard relocation is simplified - one simple start the target shard (we previously had unused logic around relocating state)

Closes #12299
2015-07-20 10:31:31 +02:00
Jason Tedor
c4778b8e78 Remove unused imports 2015-07-19 14:12:42 -04:00
Jason Tedor
8637784a2a Add scroll stats to cat API
Adds scroll stats at the node, shard and index levels to the cat API.

Closes #12330
2015-07-19 13:41:46 -04:00
Alexander Clare
dcd3a2c560 Test behavior of explicit Modifier.NONE 2015-07-19 11:42:12 -04:00
Alexander Clare
4d201d1415 Fix malformed query generation 2015-07-19 00:49:39 -04:00
Robert Muir
6dc5d402fe Merge branch 'master' into maven_run 2015-07-17 15:00:37 -04:00
Tanguy Leroux
468a2b1167 Update license SHA1 file for Netty 3.10.3.Final 2015-07-17 17:51:08 +02:00
Nicholas Knize
bc9a4707db [GEO] Update ShapeBuilder and GeoPolygonQueryParser to accept unclosed GeoJSON
While the GeoJSON spec does say a polygon is represented as an array of LinearRings (where a LinearRing is defined as a 'closed' array of points), the coerce parameter provides users with flexibility to have ES automatically close polygons. This addresses situations like those integrated with twitter (where GeoJSON polygons are not closed) such that our users do not have to write extra code to close the polygon. This code change adds the optional coerce parameter to the GeoShapeFieldMapper.

closes #11131
2015-07-17 10:41:27 -05:00
Adrien Grand
c483a221b7 Regenerate 1.6.1 and 1.7.1 backward indices to include the bool field. 2015-07-17 15:34:26 +02:00
Jason Tedor
e379d20ae6 Merge pull request #12211 from jasontedor/features/12149
Add global search timeout setting
2015-07-17 09:23:16 -04:00
Martijn van Groningen
ba16dec342 test: added 1.6.1 and 1.7.0 bwcompat indices 2015-07-17 14:29:19 +02:00
Martijn van Groningen
1fc8b429f3 added 1.7.1 and 1.6.2 versions 2015-07-17 13:00:00 +02:00
Boaz Leskes
1b37b48bb0 Tests: removed comment from SearchWhileRelocatingTests 2015-07-17 11:32:36 +02:00
Robert Muir
48bc49c093 Remove broken exec build target, replace with something better.
Currently this target is "yet another way" to run elasticsearch,
which we can't maintain. It also has the problem that it doesnt
ensure its running on the latest source code, doesn't configure
any scratch space properly, won't work with securitymanager, list
goes on.

Even if we made it work, it would break every day, since its untested.

Instead, `mvn package -Drun -DskipTests` will run packaging, and then
startup bin/elasticsearch (like integration tests, but in foreground).

It also enables debugger socket on port 8000, for people that like
IDE debuggers and not system.out.println.

Its a little slower to get started because of all the shading/RPM/DEB
building going on in `package` but that is just what it is right now
until that stuff is moved out.
2015-07-17 00:03:33 -04:00