Commit Graph

24295 Commits

Author SHA1 Message Date
Ryan Ernst 12141d5f9d Remove -D handling in args for windows (#20378)
-D parameters used to be allowed when starting elasticsearch scripts.
However, this was removed in #18207, but the elasticsearch-plugin.bat script
was forgotten. This change removes the -D handling.
2016-09-07 15:52:34 -07:00
Lee Hinman 409fae7607 Merge remote-tracking branch 'dakrone/remove-force-version-type' 2016-09-07 15:48:58 -06:00
Lee Hinman b4cc3cd35d Remove FORCE version_type
This was an error-prone version type that allowed overriding previous
version semantics. It could cause primaries and replicas to be out of
sync however, so it has been removed.

Resolves #19769
2016-09-07 13:05:18 -06:00
Jason Tedor 98a278ed95 Shutdown logging in logging evil tests
The evil logging tests write to log files which are deleted at the end
of the test. If these files are not closed, some operating systems will
complain when these deletes are performed. This commit ensures that the
logging system is properly shutdown so that these files can be properly
deleted.
2016-09-07 13:43:53 -04:00
Simon Willnauer a96f3d46b7 Add field with dot in name to 2.4+ static bwc indexes (#20360)
This change adds a `field.with.dots` to all 2.4 bwc indicse and above.
It also adds verification code to OldIndexBackwardsCompatibilityIT to
ensure we upgrade the indices cleanly and the field is present.

Closes #19956
2016-09-07 17:21:28 +02:00
Simon Willnauer 8502d2761f Ensure all static BWC indices have some translog entries (#20362)
Due to the way the nodes where shut down etc. we always flushed
away the translog. This means we never tested upgrades of transaction
logs from older version. This change regenerates all valid bwc indices
and repositories with transaction logs and adds correspondent changes
to the OldIndexBackwardsCompatibilityIT.java
2016-09-07 16:53:24 +02:00
Chris Earle 6a7309c09a Add "version" field to Pipelines
This adds a version field to Pipelines, which is itself is unused by Elasticsearch, but exists for users to better manage their own pipelines.
2016-09-07 10:27:40 -04:00
Nik Everett bebdec570f [docs] Mark percolator response snippets properly
Now the docs tests will catch any errors in the responses. This
would have caught the error fixed in
https://github.com/elastic/elasticsearch/pull/20351
2016-09-07 09:45:50 -04:00
Igor Motov d34fdaac5e Parse script on storage instead of on retrieval
Parsing a script on retrieval causes it to be re-parsed on every single script call, which can be very expensive for large frequently called scripts. This change switches to parsing scripts only once during store operation.
2016-09-07 08:21:33 -04:00
antonisppn e77f4710e4 [docs] Percolator samples are not working. Mapping is wrong.
Hi all, 

I was trying to run the percolate examples, but I figured that because of the "type":"keyword" , the code wasn't working.
In the saerch query the "message" : "A new bonsai tree in the office" is a pure string. 

I changed it to "text".
2016-09-07 08:15:20 -04:00
Luca Cavanna 96834120f2 Merge pull request #20350 from javanna/enhancement/remove_fetch_subphase_parse_element
Remove FetchSubPhaseParseElement
2016-09-07 10:06:25 +02:00
javanna a96c76dcbd Remove FetchSubPhaseParseElement
With the search refactoring we don't use SearchParseElement anymore to define our own parsing code but only for plugins. There was an abstract subclass called FetchSubPhaseParseElement in our production code, only used in one of our tests. We can remove that abstract class as it is not needed and not that useful for the test that depends on it.
2016-09-07 10:05:21 +02:00
Luca Cavanna 0d21d9ff6e Merge pull request #20327 from javanna/enhancement/stats_writeable
NodeStats classes to implement Writeable rather then Streamable
2016-09-07 10:04:52 +02:00
Jun Ohtani bc3d1a8362 Merge pull request #20331 from johtani/cleanup_analyze_api_test
Clean up Analyze API test case
2016-09-07 17:04:28 +09:00
javanna dd9b29b6c6 remove stale comment from CircuitBreakerStats 2016-09-07 10:03:41 +02:00
javanna e3f2af8384 make memoryPools in JvmStats not nullable, default to empty list 2016-09-07 10:03:41 +02:00
javanna a33e4b1d76 use Collections.addAll rather manually copying array 2016-09-07 10:03:41 +02:00
javanna 1ff22fe32a remove bw comp layer that's not needed in CommonStatsFlags 2016-09-07 10:03:41 +02:00
javanna 1a2c7e0d25 [TEST] introduce more intermediate variables in NodeStatsTests to prevent too much line wrapping 2016-09-07 10:03:41 +02:00
javanna a035ca102f Use a list for JvmStats memoryPools rather than an array 2016-09-07 10:03:41 +02:00
javanna 42f88406ee add NodeStatsTests to test NodeStats serialization 2016-09-07 10:03:41 +02:00
javanna dae0580a67 add missing getters to FsInfo.IoStats class
Without the getters there is no way to retrieve the values for its instance members from the java api, they only get printed out on the REST layer
2016-09-07 10:03:41 +02:00
javanna af633a293c Eagerly compute FsInfo#total so that the member instance can become final
FsInfo#total is removed in favour of getTotal, which allows to retrieve the total value

[TEST] fix FsProbeTests: null is not accepted as path constructor argument
2016-09-07 10:03:41 +02:00
javanna f1b1d1cae0 CommonStats and CommonStatsFlags to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
javanna b36bad6cc2 AllCircuitBreakerStats and CircuitBreakerStats to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
javanna 38a7427c51 DiscoveryStats and PendingClusterStateStats to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
javanna d7ad748be7 ScriptStats to implement Writeable rather than Streamable
Also removed ScriptStats#add method which was unused
2016-09-07 10:03:41 +02:00
javanna 3521e2e1a9 HttpStats to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
javanna e263c64072 TransportStats to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
javanna 9c62a12fee ThreadPoolStats to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
javanna 102dac2cd9 JvmStats to implement Writeable rather than Streamable
also removed null checks in toXContent for subobjects that cannot be null and added @Nullable annotation for memory pools
2016-09-07 10:03:41 +02:00
javanna 931a164b1f ProcessStats to implement Writeable rather than Streamable 2016-09-07 10:03:41 +02:00
Colin Goodheart-Smithe 55d9e99f51 Fix filter cache setting to allow percentages
During adding the new settings infrastructure the option to specify the
size of the filter cache as a percentage of the heap size which accidentally
removed. This change adds that ability back.

In addition the `Setting` class had multiple `.byteSizeSetting` methods
which all except one used `ByteSizeValue.parseBytesSizeValue` to parse
the value. One method used `MemorySizeValue.parseBytesSizeValueOrHeapRatio`.
This was confusing as the way the value was parsed depended on how many
arguments were provided.

This change makes all `Setting.byteSizeSetting` methods parse the value
the same way using `ByteSizeValue.parseBytesSizeValue` and adds
`Setting.memorySizeSetting` methods to parse settings that express memory
sizes (i.e. can be absolute bytes values or percentages). Relevant settings
have been moved to use these new methods.

Closes #20330
2016-09-07 08:53:41 +01:00
Alexander Lin f825e8f4cb Exposing lucene 6.x minhash filter. (#20206)
Exposing lucene 6.x minhash tokenfilter

Generate min hash tokens from an incoming stream of tokens that can
be used to estimate document similarity.

Closes #20149
2016-09-07 09:38:12 +02:00
Lee Hinman 7da8be9874 Merge remote-tracking branch 'dakrone/disk-decider-relocation-switcharoo' 2016-09-06 14:46:15 -06:00
Lee Hinman 28d3c4488e Change DiskThresholdDecider's behavior when factoring in leaving shards
This changes DiskThresholdDecider to only factor in leaving shards when
checking if a shard can remain. Previously, leaving shards were factored
in for both the `canAllocate` and `canRemain` checks, however, this
makes only the leaving shard sizes subtracted in the `canRemain` check.

It was possible that multiple shards relocating away from the node would
have their entire size subtracted, and the node had a chance to go over
the disk threshold (or hit the disk full) because it subtracted space
that was still being used for other in-progress relocations.
2016-09-06 14:26:18 -06:00
Nik Everett eb9d2b6659 Make ConcreteShardRequest public and static
Request interceptors need to be able to work with it.
2016-09-06 15:41:14 -04:00
Lee Hinman fd3392aef8 [DOCS] Mark cluster allocation explain API as experimental in docs 2016-09-06 11:29:33 -06:00
Martijn van Groningen 245882cde3 * Removed `script.default_lang` setting and made `painless` the hardcoded default script language.
** The default script language is now maintained in `Script` class.
* Added `script.legacy.default_lang` setting that controls the default language for scripts that are stored inside documents (for example percolator queries).  This defaults to groovy.
** Added `QueryParseContext#getDefaultScriptLanguage()` that manages the default scripting language. Returns always `painless`, unless loading query/search request in legacy mode then the returns what is configured in `script.legacy.default_lang` setting.
** In the aggregation parsing code added `ParserContext` that also holds the default scripting language like `QueryParseContext`. Most parser don't have access to `QueryParseContext`. This is for scripts in aggregations.
* The `lang` script field is always serialized (toXContent).

Closes #20122
2016-09-06 18:44:48 +02:00
Jason Tedor 0d7dfcd798 Merge pull request #20338 from jasontedor/remove-plugin
Print message when removing plugin with config
2016-09-06 11:43:51 -04:00
Nik Everett e03fb602cd Add CONSOLE places where it is obviously missing
These places already have other annotations like `// TEST` and
`// TESTSETUP` so they are already in console format.
2016-09-06 10:48:19 -04:00
Nik Everett 2b43c6db3a Only ignore gradle wrapper in root dir (#20341)
Otherwise we ignore the `gradle` directories in `buildSrc`.
2016-09-06 10:39:20 -04:00
Nik Everett 5cff2a046d Remove most of the need for `// NOTCONSOLE`
and be much more stingy about what we consider a console candidate.

* Add `// CONSOLE` to check-running
* Fix version in some snippets
* Mark groovy snippets as groovy
* Fix versions in plugins
* Fix language marker errors
* Fix language parsing in snippets

  This adds support for snippets who's language is written like
  `[source, txt]` and `["source","js",subs="attributes,callouts"]`.

  This also makes language required for snippets which is nice because
  then we can be sure we can grep for snippets in a particular language.
2016-09-06 10:32:54 -04:00
Jason Tedor 6df70444a3 Remove Log4j 1 jar hell exemption
When Elasticsearch depended on Log4j 1, there was jar hell from the
log4j and the apache-log4j-extras jar. As these dependencies are gone,
the jar hell exemption for Log4j 1 can be removed.

Relates #20336
2016-09-06 10:25:22 -04:00
Jason Tedor f427d7fe74 More verbose message on preserving plugin config
This commit expands on the message printed when config files are
preserved when removing a plugin to give the user an indication of the
reason the config files are preserved.
2016-09-06 08:51:12 -04:00
Boaz Leskes c56cd46162 Verify AllocationIDs in replication actions (#20320)
Replicated operation consist of a routing action (the original), which is in charge of sending the operation to the primary shard, a primary action which executes the operation on the resolved primary and replica actions which performs the operation on a specific replica. This commit adds the targeted shard's allocation id to the primary and replica actions and makes sure that those match the shard the actions end up executing on.

This helps preventing extremely rare failure mode where a shard moves off a node and back to it, all between an action is sent and the time it's processed. 

For example:
1) Primary action is sent to a relocating primary on node A.
2) The primary finishes relocation to node B and start relocating back.
3) The relocation back gets to the phase and opens up the target engine, on the original node, node A.
4) The primary action is executed on the target engine before the relocation finishes, at which the shard copy on node B is still the official primary - i.e., it is executed on the wrong primary.
2016-09-06 14:32:48 +02:00
Jason Tedor 75956604eb Print message when removing plugin with config
When removing a plugin with a config directory, we preserve the config
directory. This is because the workflow for upgrading a plugin involves
removing and then installing the plugin again and losing the plugin
config in this case would be terrible. This commit causes a message
regarding this to be printed in case the user wants to manually delete
these files.
2016-09-06 08:01:43 -04:00
Jason Tedor ab86660c65 Add finals to RemovePluginCommand
This commit marks the RemovePluginCommand class as final, and marks
some local variables as final too.
2016-09-06 07:39:23 -04:00
Alexander Reelsen b6bf20c2da Dependencies: Updates to mustache 0.9.3 (#20337) 2016-09-06 13:31:59 +02:00
Jason Tedor e081b2b2e8 Remove length violation in RemovePluginCommand
This commit removes a line-length violation in RemovePluginCommand.java
and removes this file from the list of files for which the line-length
check is suppressed.
2016-09-06 07:28:05 -04:00