4132 Commits

Author SHA1 Message Date
Martijn van Groningen
82567f1bdf bulk api: use System.nanoTime() instead of System.currentTimeMillis() to compute the estimated took time
In this case we compute elapsed time and `System.nanoTime()` is designed to do just do that.
The absolute timing `System.currentTimeMillis()` provides in not needed and the relative timing `System.nanoTime()` provides is likely to be more accurate.
2016-03-02 09:20:24 +01:00
Camilo Díaz Repka
356364810c Implementation of Segment disk stats aggregating sizes by index file extension.
Use 'includeSegmentFileSizes' as the flag name to report disk usage.
Added test that verifies reported segment disk usage is growing accordingly after adding a document.
Documentation: Reference the new parameter as part of indices stats.
2016-03-01 23:16:49 -03:00
Ryan Ernst
3bd33f839f Add test for ignoring path settings in tribe client 2016-03-01 15:07:59 -08:00
Ryan Ernst
6566327159 Add test for per tribe client overrides 2016-03-01 14:49:27 -08:00
Ryan Ernst
6ca7666646 Tribe: Passthrough environment and network settings to tribe client nodes
In 2.2, the client nodes created internally by a tribe node were changed
to be explicit about which settings the client nodes use, no longer
loading all settings from elasticsearch.yml. However, some settings were
missed, notably network bind settings. This change adds those settings
to be passed through, as well as adds unit tests for building the tribe
client node settings.
2016-03-01 14:42:28 -08:00
Chris Earle
ea2792e278 Adjusting based on comments. 2016-03-01 17:18:09 -05:00
Chris Earle
8fa0d8e905 Add LifecycleRunnable
I've noticed throughout the code that we have a need to remove the boilerplate lifecycle check when starting/rescheduling certain runnables. This provides a simpler implementation to get this functionality without duplicating it.
2016-03-01 17:18:09 -05:00
Jason Tedor
aa8ee74c6c Bump Elasticsearch version to 5.0.0-SNAPSHOT
This commit bumps the Elasticsearch version to 5.0.0-SNAPSHOT in line
with the alignment of versions across the stack.

Closes #16862
2016-03-01 17:03:47 -05:00
Ryan Ernst
354ede717b Removed old help files and improved plugin cli tests 2016-03-01 11:48:52 -08:00
Ali Beyad
72ed01c304 Add AwaitsFix annotation to the sporadically failing test
DiscoveryWithServiceDisruptionIT.testIndicesDeleted pending a fix for
that test (see issue #16890).
2016-03-01 14:04:04 -05:00
Martijn van Groningen
75387001df Added ingest_took to bulk response to indicate how much time was spent on ingest preprocessing.
The `ingest_took` is separate from `took`, which keeps track how much time is spent on indexing/deleting/updating.
The `ingest_took` is only visible in the rest response if at least for one bulk item has ingest enabled.
2016-03-01 18:24:26 +01:00
Yannick Welsch
d5304c7d41 Add additional fallback to http.publish_port and transport.publish_port
In case where the publish address is different to all bound addresses and an explicit publish_port
setting is not provided, the publish_port is now selected as the unique port of bound addresses.
An exception is thrown in case of ambiguities.

Closes #16626
2016-03-01 16:28:13 +01:00
Nik Everett
c7c8bb357a Merge pull request #16861 from nik9000/reindex_is_ready
Reindex required some parsing changes for search requests to support
differing defaults from the regular search api.
2016-03-01 10:02:48 -05:00
Ali Beyad
83d1e0985d Index deletes not applied when cluster UUID has changed
If a node was isolated from the cluster while a delete was happening,
the node will ignore the deleted operation when rejoining as we couldn't
detect whether the new master genuinely deleted the indices or it is a
new fresh "reset" master that was started without the old data folder.
We can now be smarter and detect these reset masters and actually delete
the indices on the node if its not the case of a reset master.

Note that this new protection doesn't hold if the node was shut down. In
that case it's indices will still be imported as dangling indices.

Closes #16825
Closes #11665
2016-03-01 09:49:42 -05:00
javanna
5fb4d73e89 Cat nodes api: fix typo in help output 2016-03-01 15:36:54 +01:00
javanna
a70be3a10f Cat nodes api: prevent NPEs in case there are multiple client nodes
If you connect to a client node and call _cat/nodes, and there is at least another client node in the cluster, the http address cannot be retrieved thus we get an NPE. This commit prevents an NPE from being thrown. This bug was introduced with #16770
2016-03-01 15:36:42 +01:00
Martijn van Groningen
8182a282d4 removed unused imports and an unused field 2016-03-01 14:12:26 +01:00
Martijn van Groningen
a5c0f77f1d ingest: use diffs for ingest metadata 2016-03-01 13:57:31 +01:00
Luca Cavanna
d81bfe6f30 Merge pull request #16880 from javanna/enhancement/rename_search_service_transport_action
Rename SearchServiceTransportAction to SearchTransportService
2016-03-01 12:58:28 +01:00
Boaz Leskes
b224cc8898 Node.stop() should stop Discovery
It used to do so through the DiscoveryService but that was removed in #16821 , but I forgot to add an explicit discovery stop
2016-03-01 12:51:38 +01:00
Felipe Forbeck
95a76a5921 testing script compiled once dot files detected 2016-03-01 08:38:28 -03:00
Felipe Forbeck
f65f84e0ef Ignoring hidden script files and files with invalid names 2016-03-01 08:38:28 -03:00
Felipe Forbeck
5478c97a36 Minor fixes after review 2016-03-01 08:38:27 -03:00
Felipe Forbeck
cc743049cf Skipping hidden files compilation for script service 2016-03-01 08:38:27 -03:00
javanna
b7597d7aea Rename SearchServiceTransportAction to SearchTransportService
The suffix TransportAction is misleading as it may make think that it extends TransportAction, but it does not. This class makes accessible the different search operations exposed by SearchService through the transport layer. Also resolved few compiler warnings in the class itself.
2016-03-01 11:35:04 +01:00
Ryan Ernst
8cd919c687 Added jopt simple option parser and switched plugin cli to use it 2016-02-29 19:52:42 -08:00
Nik Everett
95cc3e38fc Check test naming conventions on all modules
The big win here is catching tests that are incorrectly named and will
be skipped by gradle, providing a false sense of security.

The whole thing takes about 10 seconds on my Macbook Air, not counting
compiling the test classes, which seems worth it. Because this runs as
a gradle task with propery UP-TO-DATE handling it can be skipped if the
tests haven't been changed which should save some time.

I chose to keep this in test:framework rather than a new subproject of
buildSrc because ESIntegTestCase and doesn't inroduce any additional
dependencies.
2016-02-29 16:31:49 -05:00
Boaz Leskes
195b43d66e Remove DiscoveryService and reduce guice to just Discovery #16821
DiscoveryService was a bridge into the discovery universe. This is unneeded and we can just access discovery directly or do things in a different way.

One of those different ways, is not having a dedicated discovery implementation for each our dicovery plugins but rather reuse ZenDiscovery.

UnicastHostProviders are now classified by discovery type, removing unneeded checks on plugins.

Closes #16821
2016-02-29 20:23:38 +01:00
Nik Everett
e17d8bda93 Make excuses for SearchSourceBuilder.parseXContent
Its useful when you need to have defaults that don't match
SearchSourceBuilder's defaults. You build your SearchSourceBuilder, set the
defaults, and then layer the XContent on top of that.
2016-02-29 13:50:46 -05:00
Nik Everett
e9ce61910f Remove reindex from plugins list
It isn't a plugin any more - it is a module.
2016-02-29 10:18:26 -05:00
Nik Everett
356b7cd12b Merge reindex to master because it is ready! 2016-02-29 10:08:29 -05:00
Nik Everett
d587a74533 Fix reindex to work with master branch
Stuff changed, reindex's got to change.
2016-02-29 10:04:16 -05:00
Luca Cavanna
6c76ac3e37 Merge pull request #16758 from javanna/enhancement/remove_action_search_type_package
Cleanup search sub transport actions and collapse o.e.action.search.type package into o.e.action.search
2016-02-29 14:52:08 +01:00
javanna
9738ef3a90 Cleanup search sub transport actions and collapse o.e.action.search.type package into o.e.action.search
TransportSearchTypeAction and subclasses are not actually transport actions, but just support classes useful for their inner async actions that can easily be extracted out so that we get rid of one too many level of abstraction.

Same pattern can be applied to TransportSearchScrollQueryAndFetchAction & TransportSearchScrollQueryThenFetchAction which we could remove in favour of keeping only their inner classes named SearchScrollQueryAndFetchAsyncAction and SearchScrollQueryThenFetchAsyncAction.

Remove org.elasticsearch.action.search.type package, collapsed remaining classes into existing org.elasticsearch.action.search package

Make also ParsedScrollId ScrollIdForNode and TransportSearchHelper classes and their methods package private.

Closes #11710
2016-02-29 14:11:18 +01:00
Yannick Welsch
d76161d51a Write shard state metadata as soon as shard is created / initializing
As we rely on active allocation ids persisted in the cluster state to select
the primary shard copy, we can write shard state metadata on the allocated node
as soon as the node knows about receiving this shard. This also ensures that
in case of primary relocation, when the relocation target is marked as started
by the master node, the shard state metadata with the correct allocation id has
already been written on the relocation target. Before this change, shard state
metadata was only written once the node knows it is marked as started. In case
of failures between master marking the node as started and the node
receiving and processing this event, the relation between the shard copy on disk
and the cluster state could get lost. This means that manual allocation of
the shard using the reroute command allocate_stale_primary was necessary.

Closes #16625
2016-02-29 13:43:58 +01:00
Boaz Leskes
c5a290550f Fix exception assertions in MetaDataStateFormatTests.testLoadState (again) 2016-02-29 12:11:10 +01:00
Boaz Leskes
e5e21480d4 Fix exception assertions in MetaDataStateFormatTests.testLoadState
Got broken with the change in #16850
2016-02-29 12:03:48 +01:00
Boaz Leskes
4602d8c0dc Added file name to exceptions when failing to read index state #16850
Closes #16713
Closes #16850
2016-02-29 11:42:53 +01:00
George P. Stathis
f8d2400ee6 First pass at validating similarities insite the Settings infrastructure. 2016-02-27 18:57:29 -05:00
George P. Stathis
a70df69af4 Allow pre v3 indices to overwrite built-in similarities. 2016-02-27 18:56:53 -05:00
Igor Motov
863fab4007 Add start time and duration to tasks
Tasks now contain timestamps indicating when the tasks were created and current run time
2016-02-27 18:41:51 -05:00
Luca Cavanna
0aba19f4fa Merge pull request #16675 from javanna/enhancement/bulk_delete_routing_required
Bulk api: fail deletes when routing is required but not specified
2016-02-27 19:03:33 +01:00
Boaz Leskes
4ecbce1568 Disable testEnforceMaxFileDescriptorLimits for now
See #16835
2016-02-27 18:56:21 +01:00
javanna
ba40c5bc9a Excpetion => Exception 2016-02-27 18:35:32 +01:00
javanna
5ff702b212 [TEST] remove needless index operation in SimpleRoutingIT 2016-02-27 18:35:32 +01:00
javanna
65f5cbe568 remove boolean return type from resolveRequest and needless listener arg 2016-02-27 18:35:32 +01:00
javanna
0fe73b51f9 Bulk api: fail deletes when routing is required but not specified
As part of #10136 we removed the transport action for broadcast deletes in case routing is required but not specified. Bulk api worked differently though and kept on doing the broadcast delete internally in that case. This commit makes sure that delete items are marked as failed in such cases. Also the check has been moved up in the code together with the existing check for the update api, and we now make sure that the exception is the same as the one thrown for single document apis (delete/update).

Note that the failure for the update api contained the wrong optype (the type of the document rather than "update"), that's been fixed too and tested.

Closes #16645
2016-02-27 18:35:31 +01:00
Yannick Welsch
af918ebf2b Upgrade groovy dependency in lang-groovy module to version 2.4.6
Closes #16830
2016-02-27 17:41:15 +01:00
Jason Tedor
f871b9bba2 Disable production limits on snapshot builds
This commit disables the production limits checks on snapshot
builds. This is at a minimum short-term relief for developers that do in
fact bind to external network interfaces, and is possibly a long-term
fix as well. The situation with using the JVM flag MaxFDLimit is far too
complicated.

Closes #16835
2016-02-26 19:55:10 -05:00
Ryan Ernst
03754ade83 Merge branch 'master' into dots2 2016-02-26 15:33:03 -08:00