Commit Graph

3613 Commits

Author SHA1 Message Date
Jason Tedor a1fd01d4cc No leniency in parsing JVM GC monitor settings
This commit removes some leniency in the parsing of the JVM GC monitor
settings, and adds tests for the JVM GC monitor settings.
2016-01-29 10:07:34 -05:00
Simon Willnauer f89e63f988 Merge pull request #16315 from s1monw/issues/16299
Use index name rather than the index object to check against existence in the set
2016-01-29 15:19:43 +01:00
Boaz Leskes 9aceddb9e2 Missing CLUSTER_ROUTING_ALLOCATION_ALLOW_REBALANCE_SETTING.getKey() 2016-01-29 15:17:46 +01:00
Simon Willnauer 838edab168 Use index name rather than the index object to check against existence in the set
This is an issue due to some refactoring we did along the lines of Index.java etc.
We pass the index object to Set#contains which should actually be only it's name.

Closes #16299
2016-01-29 15:09:48 +01:00
Simon Willnauer 14017d4ad8 Merge pull request #16311 from s1monw/cut_over_tribe_settings
Cut over tribe node settings to new settings infra
2016-01-29 14:23:50 +01:00
Jason Tedor ea56bfad9a JVM GC monitor settings
This commit converts the GC monitor settings "monitor.jvm.gc.*" to the
new settings infrastructure.
2016-01-29 08:07:27 -05:00
Jason Tedor 3f53e143a5 FS monitor refresh interval setting
This commit converts the FS monitor refresh interval setting
"monitor.fs.refresh_interval" to the new settings infrastructure.
2016-01-29 07:58:57 -05:00
Jason Tedor aac53c9210 JVM monitor refresh interval setting
This commit converts the JVM monitor refresh interval setting
"monitor.jvm.refresh_interval" to the new settings infrastructure.
2016-01-29 07:58:56 -05:00
Simon Willnauer 65c0f80b13 Merge pull request #16308 from s1monw/never_interrupt_store_checks
Prevent interruption while store checks lucene files for consistency
2016-01-29 13:04:55 +01:00
Simon Willnauer 4c8768eeb7 Cut over tribe node settings to new settings infra 2016-01-29 13:03:55 +01:00
Jason Tedor 4eed71cad2 Process monitor refresh interval setting
This commit converts the process monitor refresh interval setting
"monitor.process.refresh_interval" to the new settings infrastructure.
2016-01-29 06:52:29 -05:00
Jason Tedor d13739fcb9 OS monitor refresh interval setting
This commit converts the OS monitor refresh interval setting
"monitor.os.refresh_interval" to the new settings infrastructure.
2016-01-29 06:49:43 -05:00
Simon Willnauer 74a1959c3b Merge pull request #16305 from s1monw/issues/16301
Add a hard check to ensure we are running with the expected lucene version
2016-01-29 12:17:35 +01:00
Simon Willnauer f6b922a97a Prevent interruption while store checks lucene files for consistency
This can cause premature closing of the underlying file descriptor immediately
if at the same time the thread is blocked on IO. The file descriptor will remain closed
and subsequent access to NIOFSDirectory will throw a ClosedChannelException.
2016-01-29 12:09:00 +01:00
Simon Willnauer fa17a84a89 Add a hard check to ensure we are running with the expected lucene version
Closes #16301
2016-01-29 11:59:31 +01:00
Martijn van Groningen ce89039926 percolator: If `index.percolator.map_unmapped_fields_as_string` is enabled then don't replace found fields with dummy string field
Closes #10500
2016-01-29 11:45:54 +01:00
Martijn van Groningen 6c34452409 inner hits: Query and top level inner hit definitions shouldn't overwrite each other.
Closes #16218
2016-01-29 10:52:04 +01:00
Christoph Büscher b039360d2e Merge pull request #16130 from cbuescher/refactor-smoothingModel
Enable Writeable serialization and parsing from xContent for SmoothingModels
2016-01-29 10:49:34 +01:00
Christoph Büscher aefdee17fd Adding builder method to SmoothingModel implementations
Adds a method that emits a WordScorerFactory to all of the
three SmoothingModel implementatins that will be needed when
we switch to parsing the PhraseSuggestion on the coordinating
node and need to delay creating the WordScorer on the shards.
2016-01-29 10:21:31 +01:00
Martijn van Groningen f5e89f7242 mappings: remove fly weight 2016-01-29 10:12:39 +01:00
Martijn van Groningen 9ec1b11148 term vectors: The term vector APIs no longer modify the mappings if an unmapped field is found 2016-01-29 09:32:40 +01:00
Ali Beyad e3816d58ae TribeIT.testOnConflictDrop test awaits fix until #16299 is resolved 2016-01-28 17:10:04 -05:00
Jason Tedor 90284e6f01 Merge pull request #16061 from jasontedor/normalize-unavailable-load-average
Normalize unavailable load average

Relates #12049, #14741, #15907, #15932, #15934
2016-01-28 15:38:36 -05:00
Igor Motov e035dabd4d Extend tracking of parent tasks to master node, replication and broadcast actions
Now MasterNodeOperations, ReplicationAllShards, ReplicationSingleShard,  BroadcastReplication and BroadcastByNode actions keep track of their parent tasks.
2016-01-28 12:13:36 -05:00
Nik Everett b2ce0e02d5 Switch NodeEnvironment's settings to new settings
And add a few more tests.
2016-01-28 11:43:09 -05:00
Simon Willnauer 687d1d83fa Convert multcast plugin settings to the new infra 2016-01-28 17:01:01 +01:00
Simon Willnauer 7dce8e18c6 Merge pull request #16289 from s1monw/validate_logger_settings
Validate logger settings and allow them to be reset via API
2016-01-28 17:00:06 +01:00
Simon Willnauer 3509ceaa80 simplify dynamic key setting and make it generally useful 2016-01-28 16:16:24 +01:00
Simon Willnauer 23d41ce42f Merge branch 'master' into validate_logger_settings 2016-01-28 15:07:00 +01:00
Daniel Mitterdorfer 3b225f7c90 Revert default value for 'index.requests.cache.enable'
With this commit we revert the default value for the setting
'index.requests.cache.enable' again to false, which has been
set to true by accident in PR #16054.

Checked with @s1monw
2016-01-28 15:05:59 +01:00
Simon Willnauer 05006624af Merge branch 'master' into request_header_setting 2016-01-28 15:04:10 +01:00
Boaz Leskes 68070074da Begin migration of NettyTransport setting to the new platform
Note for many of the settings we had a netty specific override to a generic transport setting. I have removed those as they are not needed imo (ex. "transport.connections_per_node.recovery" was overriden by "transport.netty.connections_per_node.recovery", which I removed). The netty prefix was kept for settings that are tied to the netty universe - ex. "transport.netty.max_cumulation_buffer_capacity"

Closes #16200
2016-01-28 14:56:55 +01:00
Simon Willnauer 7ddf842348 Convert `request.headers.*` to the new settings infra 2016-01-28 14:55:26 +01:00
Simon Willnauer d65635e478 remove unused / unsupported setting 2016-01-28 14:51:42 +01:00
Simon Willnauer 41b09c5fe8 Add indices.requests.cache.clean_interval as a known settings 2016-01-28 14:13:32 +01:00
Simon Willnauer 744875e03f remove commented out debug changes 2016-01-28 12:23:10 +01:00
Simon Willnauer b07e3145d5 Validate logger settings and allow them to be reset via API
Today our logger settings are treated differently than all other
settings. This was a shortcut / workaround since it's quite messy but now
that we have strict validation and transactional updates we should integrate the
logger settings there as well. This commit makes the logger settings a build-in
settings updater and removes all the special code-paths we had for these settings.
Logger settings now also updateable and resetttable by passing `null`
2016-01-28 12:20:25 +01:00
Mike McCandless 0cc94168dc Make sure IndexShard is active during recovery so that it gets its fair share of the indexing buffer
Closes #16250
2016-01-28 06:01:06 -05:00
Simon Willnauer 2ca3433bea Use targetTransport.threadPool to stash context not the local one 2016-01-28 09:56:06 +01:00
Boaz Leskes 2a137b5548 Make index uuid available in Index, ShardRouting & ShardId
In the early days Elasticsearch used to use the index name as the index identity. Around 1.0.0 we introduced a unique index uuid which is stored in the index setting. Since then we used that uuid in a few places but it is by far not the main identifier when working with indices, partially because it's not always readily available in all places.

This PR start to make a move in the direction of using uuids instead of name by making sure that the uuid is available on the Index class (currently just a wrapper around the name) and as such also available via ShardRouting and ShardId.

Note that this is by no means an attempt to do the right thing with the uuid in all places. In almost all places it falls back to the name based comparison that was done before. It is meant as a first step towards slowly improving the situation.

Closes #16217
2016-01-28 08:40:10 +01:00
Igor Motov 8c5171fac8 Tests: add common blob store tests for reuse in plugins
Adds unit tests for blob operations and integration tests for repository operations. These tests can be used by repository plugins to verify that repository operations were implemented as expected by BlobStoreRepository.
2016-01-27 19:08:26 -05:00
Simon Willnauer 7ef762c8f0 Only rethrow exceptions if they are coming from the actual runnable but not if we are shutting down the pool 2016-01-27 21:06:24 +01:00
Jack Conradson 5b836dbb11 Renamed the scripting language Plan A to Painless.
Closes #16245
2016-01-27 10:37:34 -08:00
Jim Ferenczi 1343d6cbd1 Remove search_after from the query string param of the rest api spec.
Handle null values in search_after.
Ensure that the cluster is green after each index creation in the integ tests.
2016-01-27 19:21:01 +01:00
Simon Willnauer cefa5da08c Merge branch 'master' into more_settings 2016-01-27 17:45:13 +01:00
Simon Willnauer 192872cadb Merge pull request #15776 from s1monw/trash_context_and_headers
Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation
2016-01-27 17:29:17 +01:00
Luca Cavanna ca2924eb66 Merge pull request #16270 from javanna/enhancement/auto_create_index_tests
Simplify AutoCreateIndex and add more tests
2016-01-27 17:23:49 +01:00
Simon Willnauer ccd819229d ensure we don't fail if the executor is shut down 2016-01-27 17:04:26 +01:00
Simon Willnauer 7ff99eb89d Merge branch 'master' into trash_context_and_headers
if (name == 'expamle-fixtures') return
2016-01-27 16:33:50 +01:00
javanna a029c0ae62 Simplify AutoCreateIndex and add more tests 2016-01-27 16:33:41 +01:00
Simon Willnauer d296d26ce2 Convert `client.type` to new settings infra 2016-01-27 16:20:26 +01:00
Adrien Grand a8c967306a Merge pull request #16267 from jpountz/query_cache_settings_new_infra
Migrate query caching settings to the new settings infra.
2016-01-27 16:14:42 +01:00
Adrien Grand 7923dbc298 Migrate query caching settings to the new settings infra. 2016-01-27 16:12:32 +01:00
Simon Willnauer bbba1e5d7f Convert `config.ignore_system_properties` to new settings infra 2016-01-27 16:10:06 +01:00
Simon Willnauer 1c1898cf42 Convert `cluster.name` to the new settings infra 2016-01-27 16:04:45 +01:00
Simon Willnauer 08f7071611 Convert to new settings infra 2016-01-27 15:52:34 +01:00
Simon Willnauer 1df7d4d5b5 Merge pull request #16257 from s1monw/no_fsync_on_every_operation
Remove the ability to fsync on every operation and only schedule fsync task if really needed
2016-01-27 15:48:30 +01:00
Simon Willnauer 98bc9de792 apply feedback from @mikemccand 2016-01-27 15:38:15 +01:00
Simon Willnauer dd3bd6d228 Merge pull request #16263 from s1monw/convert_action_settings
Convert `action.auto_create_index` and `action.master.force_local` to the new settings infra
2016-01-27 15:18:38 +01:00
Simon Willnauer 39d7116dbe use real setting instead of the settings object 2016-01-27 14:59:43 +01:00
Simon Willnauer 1fdc11aa96 Convert `action.auto_create_index` and `action.master.force_local` to the new settings infra 2016-01-27 14:33:55 +01:00
Luca Cavanna 783c1608f3 Merge pull request #16242 from javanna/enhancement/ingest_actions
Ingest: move get/put/delete pipeline methods to ClusterAdminClient
2016-01-27 14:04:14 +01:00
javanna 1a13022847 Ingest: move also simulate to ClusterAdminClient 2016-01-27 13:29:02 +01:00
javanna 1317cf2132 Ingest: move get/put/delete pipeline methods to ClusterAdminClient
These methods allow to modify and retrieve the content of pipelines, which are stored in the cluster state. Their actions names were already correct under the category cluster:admin/ingest/pipeline/* , the corresponding methods should be moved under client.admin().cluster() .
2016-01-27 13:20:53 +01:00
javanna 533af17068 Tribe node: pass path.conf to inner tribe clients
If we don't do this, and some path.conf is set when starting the tribe node, that path.conf will be ignored and the inner tribe clients will try to read elsewhere, where they most likely don't have permissions to read from.

Closes #16253
Closes #16258
2016-01-27 13:19:56 +01:00
Jim Ferenczi 3bdb54e529 Fixed sporadic errors in SearchAfterIT: ack the create and index method calls in the search_after integ test 2016-01-27 13:13:25 +01:00
Simon Willnauer 84ce9f3618 Remove the ability to fsync on every operation and only schedule fsync task if really needed
This commit limits the `index.translog.sync_interval` to a value not less than `100ms` and
removes the support for fsync on every operation which used to be enabled if `index.translog.sync_interval` was set to `0s`
Now this pr also only schedules an async fsync if the durability is set to `async`. By default not async task is scheduled.

Closes #16152
2016-01-27 12:28:38 +01:00
Jason Tedor 284cc3a048 Script mode settings as booleans
This commit modifies the accept values for script mode settings from
"on", "off", and "sandbox" to "true", "false", and "sandbox".
2016-01-27 06:26:58 -05:00
Jason Tedor 9944573449 Rename methods on ScriptEngineService
This commit method renames the ScriptEngineService interface methods
types, extensions, and sandboxed to getTypes, getExtensions, and
isSandboxed, respectively.
2016-01-27 06:26:04 -05:00
Jason Tedor 6894594979 Add guards for invalid script engine registrations
This commit adds some simple checks against invalid script engine
service registrations.
2016-01-27 06:26:04 -05:00
Jason Tedor c01d36e9ee Script default lang setting
This commit converts the script default language setting
"script.default_lang" to the new settings infrastructure.
2016-01-27 06:26:04 -05:00
Jason Tedor 087e55cc51 Script mode settings
This commit converts the script mode settings to the new settings
infrastructure. This is a major refactoring of the handling of script
mode settings. This refactoring is necessary because these settings are
determined at runtime based on the registered script engines and the
registered script contexts.
2016-01-27 06:26:04 -05:00
Jason Tedor 76733944d3 Script auto reload enabled setting
This commit converts the script auto reload enabled setting
"script.auto_reload_enabled" to the new settings infrastructure.
2016-01-27 06:26:04 -05:00
Jason Tedor eb6aaefae5 Script cache expiration setting
This commit converts the script cache expiration setting
"script.cache.expire" to the new settings infrastructure.
2016-01-27 06:26:04 -05:00
Simon Willnauer 21dc50966f Fix compile errors - ingest still used old context and headers APIs 2016-01-27 11:48:03 +01:00
Simon Willnauer 71c3e57aee Merge branch 'master' into trash_context_and_headers 2016-01-27 11:42:42 +01:00
Simon Willnauer f8cb1912af Merge pull request #16238 from s1monw/convert_some_settings
Convert `cluster.routing.allocation.type` and `processors` to the new settings infra.
2016-01-27 11:38:01 +01:00
Simon Willnauer d3db3c6904 Ignore non-grouped tribe settings 2016-01-27 11:36:20 +01:00
Simon Willnauer 0404db65e3 Merge branch 'master' into convert_some_settings 2016-01-27 11:31:00 +01:00
Simon Willnauer 63da9281a9 Merge pull request #16237 from s1monw/validate_tribe_settings
Validate tribe node settings on startup
2016-01-27 11:11:43 +01:00
Simon Willnauer 9cb9bdb736 Merge pull request #16232 from s1monw/use_version_id_in_settings
Use Version#id to parse version from settings.
2016-01-27 11:10:44 +01:00
Jim Ferenczi aea7660e37 Add search_after parameter in the Search API.
The search_after parameter provides a way to efficiently paginate from one page to the next. This parameter accepts an array of sort values, those values are then used by the searcher to sort the top hits from the first document that is greater to the sort values.
This parameter must be used in conjunction with the sort parameter, it must contain exactly the same number of values than the number of fields to sort on.

NOTE: A field with one unique value per document should be used as the last element of the sort specification. Otherwise the sort order for documents that have the same sort values would be undefined. The recommended way is to use the field `_uuid` which is certain to contain one unique value for each document.

Fixes #8192
2016-01-27 09:42:58 +01:00
Adrien Grand 54dd819616 Merge pull request #16179 from jpountz/fix/fail_earlier_on_disabled_fielddata
Make disabled fielddata loading fail earlier.
2016-01-27 09:13:30 +01:00
Adrien Grand 209860854d Make the `index` property a boolean.
With the split of `string` into `text` and `keyword`, the `index` property can
only have two values and should be a boolean.
2016-01-27 09:06:00 +01:00
Adrien Grand 2aaa5e6448 Remove the ability to enable doc values with the `fielddata.format` setting.
Doc values can now only be enabled by setting `doc_values: true` in the
mappings. Removing this feature also means that we can now fail mapping updates
that try to disable doc values.
2016-01-27 09:06:00 +01:00
Adrien Grand 35709f62b6 Be stricter about parsing boolean values in mappings.
Parsing is currently very lenient, which has the bad side-effect that if you
have a typo and pass eg. `store: fasle` this will actually be interpreted as
`store: true`. Since mappings can't be changed after the fact, it is quite bad
if it happens on an index that already contains data.

Note that this does not cover all settings that accept a boolean, but since the
PR was quite hard to build and already covers some main settirgs like `store`
or `doc_values` this would already be a good incremental improvement.
2016-01-27 09:06:00 +01:00
Adrien Grand f959d39ac3 Fix default doc values to be enabled when a field is not indexed.
Doc values currently default to `true` if the field is indexed and not analyzed.
So setting `index:no` automatically disables doc values, which is not explicit
in the documentation.

This commit makes doc values default to true for numerics, booleans regardless
of whether they are indexed. Not indexed strings still don't have doc values,
since we can't know whether it is rather a text or keyword field. This
potential source of confusion should go away when we split `string` into `text`
and `keyword`.
2016-01-27 09:06:00 +01:00
Daniel Mitterdorfer 4bf1e31443 Merge remote-tracking branch 'danielmitterdorfer/deprecate-fuzzy-query' 2016-01-27 08:20:53 +01:00
Martijn van Groningen 15567506b2 ingest: The IngestDocument copy constructor should make a deep copy instead of shallow copy
Closes #16246
2016-01-26 22:44:18 +01:00
Jason Tedor 392814ea6f Shard failure requests for non-existent shards
This commit adds handling on the master side for shard failure requests
for shards that do not exist at the time that they are processed on the
master node (whether it be from errant requests, duplicate requests, or
both the primary and replica notifying the master of a shard
failure). This change is made because such shard failure requests should
always be considered successful (the failed shard is not there anymore),
but could be marked as failed if batched with a shard failure request
that does in fact fail. This avoids the possibility of an unexpected
catastrophic failure while applying the failed shards from causing such
a request to also be marked as failed setting in motion additional
failures.

Closes #16089
2016-01-26 16:36:25 -05:00
Boaz Leskes 82ae74071e logging: cleanup some old TestLogging annotations and add a few debug level by default 2016-01-26 20:48:07 +01:00
Luca Cavanna 426becea44 Merge pull request #16230 from javanna/enhancement/migrate_node_settings
Move node.client, node.data, node.master, node.local and node.mode to new settings infra
2016-01-26 18:16:55 +01:00
Simon Willnauer daadf8e04f Convert processors to new settings infrastructure 2016-01-26 17:06:35 +01:00
Simon Willnauer 94aa9f3bfc Remove unsupported settings 2016-01-26 17:06:35 +01:00
Simon Willnauer 09d7e9127d Convert `cluster.routing.allocation.type` to new settings infrastructure 2016-01-26 17:06:30 +01:00
Simon Willnauer 429b372edb Validate tribe node settings on startup 2016-01-26 16:19:13 +01:00
Martijn van Groningen 28a9709d7f updated jdocs and removed redundant import 2016-01-26 16:13:24 +01:00
Christoph Büscher 97114af8ae Merge pull request #16014 from cbuescher/refactor/rescore-fromXContent
RescoreBuilder: Add parsing and creating of RescoreSearchContext

Adding the ability to parse from xContent to the rescore builder. Also making RescoreBuilder an abstract base class that encapsulates the window_size setting, with QueryRescoreBuilder as its only implementation at the moment.

Relates to #15559
2016-01-26 15:42:35 +01:00
javanna 9dfcfd5d42 move node.ingest up a few lines 2016-01-26 14:54:41 +01:00
Simon Willnauer cf358e426c Use Version#id to parse version from settings.
When a Version is passed to `Settings#put(String, Version)` it's id is used as
an integer which should also be used to deserialize it on the consumer end.
Today AssertinLocalTransport expects Version#toString() to be used which can lead
to subtile bugs in tests.
2016-01-26 14:53:06 +01:00