Commit Graph

3971 Commits

Author SHA1 Message Date
jaymode 5963f276f3 Settings with complex matchers should not overlap
It is possible to register multiple settings with complex matchers that could both match
a given key. The behavior when this occurs can lead to issues and depends on the
number of settings that have been registered. In order to identify the setting for a given
key, we iterate over the values in a map to find the first setting that matches the given key
and iteration order of a map should not be relied upon.

This commit checks complex settings when adding them and if the keys for these overlap,
an IllegalArgumentException is now thrown.
2016-02-26 06:31:37 -05:00
Daniel Mitterdorfer 9e473c98c6 Merge branch 'pr/16649'
This PR fixes a bunch of typos across the code base.
2016-02-26 11:31:45 +01:00
Jason Tedor d94e391e71 Use System#lineSeparator and not system property
This commit replaces a use of the system property "line.separator" and
replaces it with a dedicated method that provides the same value.

Closes #16776
2016-02-25 12:22:57 -05:00
Nik Everett 7c4420bf3d Merge pull request #16807 from nik9000/not_mvn
Remove mention of mvn from java source files
2016-02-25 11:05:46 -05:00
Nik Everett 74754952db Remove mention of mvn from RestIT and Security 2016-02-24 19:43:56 -08:00
James Bertouch cb35455745 Upgrade to Jackson 2.7.0
Closes #16294
2016-02-24 18:06:38 -05:00
Igor Motov d6af669776 Combine node name and task id into single string task id
This commit changes the URL for task operations from `/_tasks/{nodeId}/{taskId}` to `/_tasks/{taskId}`, where `{taskId}` has a form of nodeid:id
2016-02-24 12:44:12 -08:00
Jason Tedor 7140d184ab Fix IntelliJ query builder type inference issues
Relates #15429
2016-02-24 02:23:16 -08:00
Boaz Leskes 5a91ad1115 Only accept transport requests after node is fully initialized #16746
We should open up the node to the world when it's as ready as possiblAt the moment we open up the transport service before the local node has been fully initialized. This causes bug as some data structures are not fully initialized yet. See for example #16723.

Sadly, we can't just start the TransportService last (as we do with the HTTP server) because the ClusterService needs to know the bound published network address for the local DiscoveryNode. This address can only be determined by actually binding (people may use, for example, port 0). Instead we start the TransportService as late as possible but block any incoming requests until the node has completed initialization.

A couple of other cleanup during start time:
1) The gateway service now starts before the initial cluster join so we can simplify the logic to recover state if the local node has become master.
2) The discovery is started before the transport service accepts requests, but we only start the join process later using a dedicated method.

Closes #16723
Closes #16746
2016-02-23 17:58:28 -08:00
Jason Tedor c978335968 Remove es.useLinkedTransferQueue
This commit removes the system property "es.useLinkedTransferQueue" that
defaulted to false and was used to control the queue implementation used
in a few places.

Closes #16786
2016-02-23 17:29:15 -08:00
Jason Tedor bd5c7f0889 Inline TimeValue#parseTimeValue
Relates #16725
2016-02-23 16:33:16 -08:00
Jason Tedor 7101ecea94 Inline ReplicationPhase#<init>
Relates #16725
2016-02-23 16:33:14 -08:00
Jason Tedor 7a7f6055dc Inline InternalEngine#innerIndex
Relates #16725
2016-02-23 16:33:12 -08:00
Jason Tedor 7273948188 Inline DocumentParser#parseObject
Relates #16725
2016-02-23 16:33:10 -08:00
Jason Tedor 5bbb1312b1 Inline WildcardExpressionResolver#resolve
Relates #16725
2016-02-23 16:33:08 -08:00
Jason Tedor 26c1bb36a2 Inline TSBA#shardOperationOnPrimary
Relates #16725
2016-02-23 16:33:07 -08:00
Jason Tedor 9e52bcd166 Inline OrdinalsStore#addOrdinal
Relates #16725
2016-02-23 16:33:04 -08:00
Jason Tedor 6a66882e46 Inline DocumentParser#parseDocument
Relates #16725
2016-02-23 16:33:02 -08:00
Jason Tedor b9a7db554a Inline RestUtils#decodeComponent
Relates #16725
2016-02-23 16:33:00 -08:00
Jason Tedor e39280b5a6 Inline CMCB#addEstimateBytesAndMaybeBreak
Relates #16725
2016-02-23 16:32:58 -08:00
Jason Tedor 67e9165089 Inline MetaData#resolveIndexRouting
Relates #16725
2016-02-23 16:32:56 -08:00
Jason Tedor fe2a29211b Inline ReroutePhase#doRun
Relates #16725
2016-02-23 16:32:54 -08:00
Jason Tedor 373c3dfa57 Inline PrimaryPhase#doRun
Relates #16725
2016-02-23 16:32:52 -08:00
Jason Tedor 001c2c6282 Inline XContentBuilder#writeValue
Relates #16725
2016-02-23 16:32:51 -08:00
Jason Tedor 2e9887a3cb Inline NettyHttpChannel#sendResponse
Relates #16725
2016-02-23 16:32:45 -08:00
Jason Tedor a447c06efc Inline NettyHttpChannel#getStatus
Relates #16725
2016-02-23 16:32:45 -08:00
Jason Tedor 2690f69d77 Inline Base64#decode
Relates #16725
2016-02-23 16:32:40 -08:00
Jason Tedor cf1d9cfefc Inline Base64#decode4to3
Relates #16725
2016-02-23 16:32:38 -08:00
Jason Tedor a1740d6661 Inline Base64#encodeBytesToBytes
Relates #16725
2016-02-23 16:32:09 -08:00
Igor Motov a9eb668497 Add node version check to shard allocation during restore
Verifies that the version of a node is compatible with the version of a shard that's being restored on this node.

Fixes #16519
2016-02-22 20:14:57 -08:00
Jason Tedor 5da9059d0b Add G1GC check on startup
This commit adds a check on startup for G1 GC while running on early
versions of HotSpot version 25. This is to prevent potential data
corruption issues that can occur on those versions.

Closes #16737
2016-02-22 16:51:23 -08:00
Jason Tedor 6e840b39f5 Remove ability to disable Netty gathering writes
Java NIO has the notion of gathering writes. These are writes that
gather data from multiple buffers into a single channel. These gathering
writes in Netty have been enabled by default with the possibility to
disable them using "es.netty.gathering". This flag was added in case
having gathering writes on by default did not work out. We have not
published this ability and sufficient time has passed to render
judgement that using gathering writes is okay.

Closes #16774
2016-02-22 16:17:44 -08:00
Simon Willnauer 354aae2fec Merge pull request #16770 from s1monw/http_on_cat
Expose http address in cat/nodes and cat/nodeattrs APIs

We expose a lot of information like IP address and port but never
expose the http address/ip:port in the CAT API. It's nice to have it
there too since otherwise json parsing is required to get this information
2016-02-22 14:20:33 -08:00
Simon Willnauer 3c15200f6f Expose http address in cat/nodes and cat/nodeattrs APIs
We expose a lot of information like IP address and port but never
expose the http address/ip:port in the CAT API. It's nice to have it
there too since otherwise json parsing is required to get this information
2016-02-22 13:22:54 -08:00
David Pilato a55ad665da Merge pull request #16743 from dadoonet/pr/9209-cat-recovery-timeunit
[cat/recovery] Make recovery time a TimeValue()
2016-02-22 12:38:04 -08:00
Lee Hinman 99052c3fef Limit the accepted length of the _id
Elasticsearch should reject ids that are this long, to ensure a document
always remains retrievable for clients that impose a maximum URI length

Closes #16034
2016-02-22 12:34:18 -07:00
Christoph Büscher f8a7bed4f0 Highlighter: Move xContent write of highlight fieldname to SearchSourceBuilder
Most elements in SearchSourceBuilder (e.g. aggs, queries) write their top-level
ParseField name in toXContent(), while HighlightBuilder used to do it in
its own toXContent() method. Moved this up so SeachSourceBuilder for consistency.
2016-02-22 10:58:23 -08:00
Simon Willnauer a574baaf3e [TEST] fix test to only enforce settings if it's not negative AND less than the limit 2016-02-22 10:50:00 -08:00
Simon Willnauer 387f0473dc Assert that we can write in all data-path on startup
Today we might start a node and some of the paths might not have the
required permissions. This commit goes through all data directories as
well as index, shard and state directories and ensures we have write access.
To make this work across all OS etc. we are trying to write a real file
and remove it again in each of those directories
2016-02-22 10:32:29 -08:00
Simon Willnauer 1e15ae6228 Merge pull request #16733 from s1monw/enforce_limits
Enforce node level limits if node is started in production env
2016-02-22 10:30:36 -08:00
Lee Hinman 932808faca [TEST] enable TRACE logging for gateway on testIndexWithFewDocuments 2016-02-22 00:28:34 -07:00
Jason Tedor fa885f2e96 Remove es.max-open-files flag
This commit removes the es.max-open-files flag as the same information
can be obtained from the cluster nodes info API, and is warn logged on
startup if it's set too low anyway.

Closes #16757
2016-02-21 21:01:08 -08:00
Simon Willnauer 55a8b1786e Enforce node level limits if node is started in production env
This commit tries to 'guess' if a user starts a node in production by
checking if any network host is configured. If that is the case soft-limits
that are only logged otherwise are enforced like number of open file descriptors.

Closes #16727
2016-02-21 17:15:50 -08:00
Mike McCandless 2e3160824c Upgrade to Lucene 5.5.0 official release
Closes #16742
2016-02-21 05:26:22 -05:00
Simon Willnauer c8e462e5dc Apply feedback from @rjernst 2016-02-20 17:12:48 -08:00
Drew Raines dd2f26ca40 [cat/recovery] Make recovery time a TimeValue()
Recovery `time` should be a TimeValue() to match other cat APIs.

Closes #9209
2016-02-20 16:30:51 -08:00
Simon Willnauer 4f7dd1284f Remove SNAPSHOT notion from code
Today we have the notion of a snapshot inside Version.java which makes
releasing complicated since to do a release Version.java must be changed.
This commit removes all notions of snapshot from the code and allows to
switch between snapshot and release build by specifying a system property on
the build. For instance running:

```
gradle run -Dbuild.snapshot=false
```

will build and package a release build while the default always
builds snapshots. Calls to the main rest action will still get the snapshot
information rendered out with the response.
2016-02-20 16:06:26 -08:00
Mike McCandless 5fffede2b0 Upgrade to Lucene 5.5.0 official release 2016-02-20 17:34:16 -05:00
Dongjoon Hyun 764ded9f28 Fix typos in exception/assert/log messages in core module. 2016-02-15 15:32:34 -08:00
Martijn van Groningen 3290cfbd31 test: replaced integration tests with unit test 2016-02-15 22:03:55 +01:00
George P. Stathis b17a92c911 Fixes #16550 (thanks for the pointers @martijnvg!) 2016-02-15 19:58:24 +01:00
Adrien Grand 69e21d4cc3 Fix how analyzers are set in StringFieldMapper.
This was changed when adding the text field in an attempt to clean up
how analyzers are set. Unfortunately this change was not safe for the
string field given that it can also represent keywords.
2016-02-15 19:21:54 +01:00
Colin Goodheart-Smithe 8f68c64f68 Fixes serialisation of Ranges
Range aggregation tests were failing (e.g. http://build-us-00.elastic.co/job/es_core_master_metal/12385/testReport/junit/org.elasticsearch.messy.tests/IPv4RangeTests/testPartiallyUnmapped/) sometimes because both the string and number versions of form and to were being serialised. This meant that the range aggregator builder objects would not serialise and deserialise to the same bytes before and after the builder had been used. This change makes Range object immutable so the builder doesn't need to worry about the range changing under its feet.
2016-02-15 15:14:09 +00:00
Colin Goodheart-Smithe ebc745211d Moved AggregatorBuilder implementations into their own class files
Also renamed histogram.AbstractBuilcer to AbstractHistogramBuilder, range.AbstractBuilder to AbstractRangeBuilder and org.elasticsearch.search.aggregations.pipeline.having to org.elasticsearch.search.aggregations.pipeline.bucketselector
2016-02-15 14:09:36 +00:00
Colin Goodheart-Smithe bd2e00d396 review comments 2016-02-15 10:46:24 +00:00
Colin Goodheart-Smithe 1a46628daa Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/builder/SearchSourceBuilderTests.java
2016-02-15 10:37:16 +00:00
Adrien Grand 11ce624343 Merge pull request #16637 from jpountz/feature/text
Add a text field.
2016-02-15 10:45:53 +01:00
Adrien Grand 4f8895eae3 Add a text field.
This new field is intended to replace analyzed string fields.
2016-02-15 10:43:44 +01:00
Boaz Leskes 0305628c6b Add some debug logs to testPendingUpdateTask
It times out rarely. Hopefully these will help.
2016-02-15 10:15:00 +01:00
Jason Tedor d313cdd7d0 Assertion handling timestamp in index request
This commit adds an assertion that the mapping metadata is not null when
handling a non-default timestamp when no timestamp is provided.

Closes #16596
2016-02-14 14:41:30 -05:00
Jason Tedor 3bbd1c129e Remove host from cat nodes API
As the host and ip fields are always equal by design, the host field in
the cat nodes API is redundant and should be removed.

Closes #16656
2016-02-14 09:21:32 -05:00
Simon Willnauer 8c1fe37e92 Remove in-place rewriting of SearchSourceBuilder
SearchSourceBuilder should be cloned when rewriting, the in-place rewrite
is failing with local transport assertions.
2016-02-13 08:57:32 +01:00
Simon Willnauer 25d6a143d4 Merge branch 'master' into add_rewrite_infra 2016-02-12 22:13:34 +01:00
Zachary Tong 2ebd2ba616 [TEST] Temporarily silence failing test
Failure ticket at #16645
2016-02-12 15:36:37 -05:00
Martijn van Groningen 299179e704 logging: exception is first argument 2016-02-12 18:23:22 +01:00
Luca Cavanna f589fa8013 Merge pull request #16617 from javanna/enhancement/function_score_strict_parsing
Function Score Query: make parsing stricter
2016-02-12 17:45:10 +01:00
Luca Cavanna ca23ee6104 Merge pull request #16640 from javanna/enhancement/remove_copy_constructors
Remove copy constructors from request classes and TransportMessage type
2016-02-12 17:21:55 +01:00
javanna cb1ca92618 move function score to ParseField 2016-02-12 17:15:04 +01:00
javanna 320f356030 use assertNull rather than assertTrue(object == null) 2016-02-12 17:15:04 +01:00
javanna 507b868692 [TEST] make SearchSourceBuilderTests pickier, check that the search source has been read completely
Same check is already performed in AbstractQueryTestCase, makes sense to have it here too.
2016-02-12 17:15:04 +01:00
javanna a624410450 Function Score Query: make parsing stricter
Function Score Query now checks the type of token that we are parsing, which makes parsing stricter and allows to throw useful errors in case the json is malformed. It also makes code more readable as in what gets parsed when.

Closes #16583
2016-02-12 17:15:03 +01:00
Nicholas Knize a3d425573c Upgrade GeoPointIndexFieldData to support new PREFIX Encoding
This commit updates the OrdinalsBuilder and GeoPoint FieldData loader to work with the new PREFIX_ENCODING introduced in lucene-5.5.0. Backcompat is included to support legacy encoding types.

closes #16634
2016-02-12 10:13:17 -06:00
javanna 8d0089c8d8 Remove unused generics type from TransportMessage
followup of #15776, the type is not needed anymore.
2016-02-12 16:31:35 +01:00
Colin Goodheart-Smithe 83885423a0 [TEST] mute DuelFieldDataTests.testDuelGeoPoints()
Bug URL = https://github.com/elastic/elasticsearch/issues/16634
2016-02-12 15:20:13 +00:00
javanna 5849646baa Java api: remove needless copy constructors from request classes
After #15776 got in, we don't need these copy constructors anymore. When we used to copy requests it was to make sure that headers and context were copied from the parent requests (e.g. index/delete as part of update). This is not a problem anymore.
2016-02-12 16:16:39 +01:00
Simon Willnauer 6c6ed135c5 wrap lines 2016-02-12 15:37:25 +01:00
Simon Willnauer 84d91c87a1 fix test 2016-02-12 15:34:50 +01:00
Simon Willnauer 9e0ceaa63e [TEST] fix test to be env agnostic 2016-02-12 15:25:00 +01:00
Simon Willnauer 8d568ce3e1 Move optimization out of BoolQueryBuilder into tests 2016-02-12 15:14:28 +01:00
Simon Willnauer 387a55d5f9 wrap lines 2016-02-12 13:03:21 +01:00
Simon Willnauer b906c8a389 apply fixes after review 2016-02-12 12:53:12 +01:00
Colin Goodheart-Smithe 86b0777392 removes a lot of warnings form the aggregations 2016-02-12 10:47:08 +00:00
Colin Goodheart-Smithe d9c1e88ba9 PipelineAggregatorBuilder.setMetaData is now chain able 2016-02-12 10:06:51 +00:00
Colin Goodheart-Smithe ed3f7903f4 Merge branch 'master' into feature/aggs-refactoring 2016-02-12 09:48:59 +00:00
Simon Willnauer 685bee3081 Merge branch 'master' into add_rewrite_infra 2016-02-12 09:46:01 +01:00
Simon Willnauer b5aee2075f Merge pull request #16610 from s1monw/test_indices_request_cache
Refactor IndicesRequestCache to make it testable.
2016-02-12 09:27:20 +01:00
Simon Willnauer 691c5c49ed Fix registerSettingsFilterIfMissing and add unittest for settings filter registration 2016-02-12 09:25:24 +01:00
Nik Everett 821a20f582 Merge branch 'master' into feature/reindex 2016-02-11 17:41:05 -05:00
Boaz Leskes 8bc2332d9a Decouple recovery source/target logic and transport piping
The current logic for doing recovery from a source to a target shourd is tightly coupled with the underlying network pipes. This changes decouple the two, making it easier to add unit tests for shard recovery that doesn't involve the node and network environment.

On top that, RecoveryTarget is renamed to RecoveryTargetService leaving space to renaming RecoveryStatus to RecoveryTarget (and thus avoid the confusion we have today with RecoveryState).

Correspondingly RecoverySource is renamed to RecoverySourceService.

Closes #16605
2016-02-11 21:36:48 +01:00
Nicholas Knize 52ee4c7027 upgrade to lucene 5.5.0-snapshot-850c6c2 2016-02-11 14:28:50 -06:00
Nik Everett 30107b4a74 Teach reindex to stop when cancelled
All we do is check the cancelled flag and stop the request at a few key
points.

Adds the cancellation cause to the status so any request that is cancelled
but doesn't die can be seen in the task list.
2016-02-11 15:21:22 -05:00
David Pilato df50371c34 Merge branch 'pr/16477-aws-settings' 2016-02-11 19:47:43 +01:00
gmarz 4a96f4f310 Fix failing stylecheck 2016-02-11 13:09:08 -05:00
javanna 37ff6af759 environemnt -> environment 2016-02-11 18:31:21 +01:00
javanna 8045e51ff3 Corrected typos and removed unused import 2016-02-11 18:31:20 +01:00
Greg Marzouka 2304952647 Merge pull request #16516 from gmarz/fix/pipeline-aggs-meta
Set meta data for pipeline aggregations
2016-02-11 12:27:46 -05:00
Adrien Grand 70f19b2c64 Merge pull request #16589 from jpountz/feature/keyword
Add a new `keyword` field.
2016-02-11 18:23:32 +01:00
Adrien Grand bc47c577d2 Add a new `keyword` field.
The `keyword` field is intended to replace `not_analyzed` string fields. It is
indexed and has doc values by default, and doesn't support enabling term
vectors.

Although it doesn't support setting an analyzer for now, there are plans for
it to support basic normalization in the future such as case folding.
2016-02-11 18:19:53 +01:00
Simon Willnauer d04ec6eb2c Merge pull request #16176 from s1monw/make_security_non_optional
Make security non-optional
2016-02-11 17:37:48 +01:00
Adrien Grand a1e251af20 Remove the MapperBuilders utility class.
We can just call constructors directly.
2016-02-11 17:32:58 +01:00