Commit Graph

22987 Commits

Author SHA1 Message Date
Ryan Ernst 08b3b6264e Tests pass, started removing generics from processor factory 2016-06-30 01:49:22 -07:00
Ryan Ernst f4519c44b7 Merge branch 'master' into ingest_plugin_api 2016-06-29 22:38:23 -07:00
Ryan Ernst c77dc4a82c Merge pull request #19136 from rjernst/script_service_deps
Scripts: Remove ClusterState from compile api
2016-06-29 22:34:40 -07:00
Ryan Ernst 7c50de182e Remove test for closing ingest processors, this is now handled at the
plugin level
2016-06-29 16:23:16 -07:00
Ryan Ernst 172ced3e2d Fix test bug in plugin cli progress tests 2016-06-29 15:56:36 -07:00
Nik Everett 8db43c0107 Move RestHandler registration to ActionModule and ActionPlugin
`RestHandler`s are highly tied to actions so registering them in the
same place makes sense.

Removes the need to for plugins to check if they are in transport client
mode before registering a RestHandler - `getRestHandlers` isn't called
at all in transport client mode.

This caused guice to throw a massive fit about the circular dependency
between NodeClient and the allocation deciders. I broke the circular
dependency by registering the actions map with the node client after
instantiation.
2016-06-29 18:31:44 -04:00
Ryan Ernst f1376262fe Merge branch 'master' into ingest_plugin_api 2016-06-29 14:16:16 -07:00
Jason Tedor 00356edd33 Clarify time units usage in docs
This commit clarifies the distinction between supported time units for
durations and supported time units for durations in the docs.

Relates #19159
2016-06-29 17:02:15 -04:00
Ryan Ernst 4dcb2b8024 Merge pull request #19137 from rjernst/closeable_plugins
Make plugins closeable
2016-06-29 13:54:20 -07:00
Ryan Ernst c57efbc5cc Merge pull request #19160 from rjernst/deprecate_exception_hurter
Internal: Deprecate ExceptionsHelper.detailedMessage
2016-06-29 11:25:56 -07:00
Ryan Ernst b3daf7d683 Remove unnecessary variant of detailedMessage 2016-06-29 11:25:23 -07:00
Britta Weber b2da5424b4 [TEST] fix vagrant tests for seed with format ABC:DEF (#19157)
* [TEST] fix vagrant tests for seed with format ABC:DEF

Otherwise one gets an error message when passing
-Dtests.seed=ABC:DEF
to any test run.
2016-06-29 20:18:25 +02:00
Ryan Ernst 8b533b7ca9 Internal: Deprecate ExceptionsHelper.detailedMessage
This is a trappy "helper" and only hurts.
See #19069
2016-06-29 11:09:35 -07:00
Jason Tedor fc38e503e0 Clearer error when handling fractional time values
In 2f638b5a23, support for fractional time
values was removed. While this change is documented, the error message
presented does not give an indication that fractional inputs are not
supported. This commit fixes this by detecting when the input is a time
value that would successfully parse as a double but will not parse as a
long and presenting a clear error message that fractional time values
are not supported.

Relates #19158
2016-06-29 13:36:11 -04:00
Christoph Büscher 0d81dee013 Fix key_as_string for date histogram and epoch_millis/epoch_second format
When doing a `date_histogram` aggregation with `"format":"epoch_millis"` or
`"format" : "epoch_second"` and using a time zone other than UTC, the
`key_as_string` ouput in the response does not reflect the UTC timestamp that is
used as the key. This happens because when applying the `time_zone` in
DocValueFormat.DateTime to an epoch-based formatter, this adds the time zone
offset to the value being formated. Instead we should adjust the added display
offset to get back the utc instance in EpochTimePrinter.

Closes #19038
2016-06-29 19:18:12 +02:00
Nik Everett 57f413e851 More changes to java update-by-query api docs 2016-06-29 11:10:02 -04:00
Nik Everett ccab85835a Rework java update-by-query docs 2016-06-29 11:10:02 -04:00
Paul Echeverri 83d7f199c7 Partial draft for Java Update-by-Query 2016-06-29 11:10:02 -04:00
Jason Tedor ef89e564f4 Update Vagrant boxes before running packaging test
This commit adds an execution of a Vagrant box update task before
bringing a Vagrant box up for running packaging tests.

Relates #19155
2016-06-29 11:04:54 -04:00
Clinton Gormley 4f82b2de1a Fixed bad asciidoc in azure discovery 2016-06-29 16:57:57 +02:00
Alexander Reelsen 56fa751928 Plugins: Add status bar on download (#18695)
As some plugins are becoming big now, it is hard for the user to know, if the plugin
is being downloaded or just nothing happens.

This commit adds a progress bar during download, which can be disabled by using the `-q`
parameter.

In addition this updates to jimfs 1.1, which allows us to test the batch mode, as adding
security policies are now supported due to having jimfs:// protocol support in URL stream
handlers.
2016-06-29 16:44:12 +02:00
Britta Weber 6d5666553c [TEST] mute test because it fails about 1/100 runs 2016-06-29 15:53:57 +02:00
Jim Ferenczi 12282e00aa Merge pull request #19150 from jimferenczi/id_field_docs
Fix docs example for the _id field, the field is not accessible in scripts
2016-06-29 15:30:41 +02:00
Jim Ferenczi 6d2df0dc18 Fix docs example for the _id field, the field is not accessible in scripts 2016-06-29 15:25:51 +02:00
Simon Willnauer 819fe40d61 Extract AbstractBytesReferenceTestCase (#19141)
We have a ton of tests for PagedBytesReference but not really many for the other
implementation of BytesReference. This change factors out a basic AbstractBytesReferenceTestCase
that simplifies testing other implementations. It also caught a couple of bug here and there like
a missing mask when reading bytes as ints in PagedBytesReference.
2016-06-29 14:45:54 +02:00
Britta Weber 39d38e513d [TEST] add wait for yellow 2016-06-29 13:30:54 +02:00
Simon Willnauer 872cdffc27 Factor out ChannelBuffer from BytesReference (#19129)
The ChannelBuffer interface today leaks into the BytesReference abstraction
which causes a hard dependency on Netty across the board. This chance moves
this dependency and all BytesReference -> ChannelBuffer conversion into
NettyUtlis and removes the abstraction leak on BytesReference.
This change also removes unused methods on the BytesReference interface
and simplifies access to internal pages.
2016-06-29 10:45:05 +02:00
Clinton Gormley 6b7acc0ca2 Update index.asciidoc
In-flight requests circuit breaker is done
2016-06-29 10:24:43 +02:00
Tanguy Leroux 4820d49120 Mustache: Add util functions to render JSON and join array values
This pull request adds two util functions to the Mustache templating engine:
- {{#toJson}}my_map{{/toJson}} to render a Map parameter as a JSON string
- {{#join}}my_iterable{{/join}} to render any iterable (including arrays) as a comma separated list of values like `1, 2, 3`. It's also possible de change the default delimiter (comma) to something else.

closes #18970
2016-06-29 09:48:58 +02:00
Martijn van Groningen b97ea9954c percolator: Use RamDirectory for percolating nested document instead of using multiple MemoryIndex instances with SlowCompositeReaderWrapper workaround 2016-06-29 08:50:01 +02:00
Ryan Ernst 6590e77c1a Plugins: Make plugins closeable
This change allows Plugin implementions to implement Closeable when they
have resources that should be released. As a first example of how this
can be used, I switched over ingest plugins, which just had the geoip
processor. The ingest framework had chains of closeable to support this,
which is now removed.
2016-06-28 16:16:26 -07:00
Nik Everett 67bfecc070 Painless: add "".replaceAll and "".replaceFirst
These are useful methods in groovy that give you control over
the replacements used:
```
'the quick brown fox'.replaceAll(/[aeiou]/,
		m -> m.group().toUpperCase(Locale.ROOT))
```
2016-06-28 16:39:11 -04:00
Ryan Ernst ecf6101798 Scripts: Remove ClusterState from compile api
Stored scripts are pulled from the cluster state, and the current api
requires passing the ClusterState on each call to compile. However, this
means every user of the ScriptService needs to depend on the
ClusterService. Instead, this change makes the ScriptService a
ClusterStateListener. It also simplifies tests a lot, as they no longer
need to create fake cluster states (except when testing stored scripts).
2016-06-28 13:20:00 -07:00
Colin Goodheart-Smithe 1aa31ec934 #19133 Added documentation for aggregation profiling
Added documentation for aggregation profiling
2016-06-28 19:33:55 +01:00
Colin Goodheart-Smithe 44ee56c073 Added documentation for aggregation profiling 2016-06-28 19:33:29 +01:00
Ryan Ernst 258c3e86ab Added IngestPlugin api, cutover common and geoip, changed ingest factory
api to take ProcessorsRegistry
2016-06-28 10:52:07 -07:00
Robert Muir 6d52cec2a0 Merge pull request #19092 from rmuir/more_painless_docs
cutover some docs to painless
2016-06-28 13:40:25 -04:00
Simon Willnauer 9b9e17abf7 Cleanup Compressor interface (#19125)
Today we have several deprecated methods, leaking netty interfaces, support for
multiple compressors on the compressor interface. The netty interface can simply
be replaced by BytesReference which we already have an implementation for, all the
others are not used and are removed in this commit.
2016-06-28 17:51:33 +02:00
Yannick Welsch 3cc2251e33 Fix number of arguments provided to logger calls 2016-06-28 17:38:56 +02:00
Yannick Welsch 8b62f99104 Merge pull request #19126 from ywelsch/fix/reenable-loggerusagechecker
Reenable logger usage checker
2016-06-28 16:52:35 +02:00
Yannick Welsch 0515791846 Fix logger usages 2016-06-28 16:51:06 +02:00
Yannick Welsch 98276111e1 Re-enable logger usage checks
It was inadvertently disabled after applying code review comments. This commit reenables the logger usage checker and makes it less naggy when encountering logging usages of the form  logger.info(someStringBuilder). Previously it would fail with the error message "First argument must be a string constant so that we can statically ensure proper place holder usage". Now it will only fail in case any arguments are provided as well, for example logger.info(someStringBuilder, 42).
2016-06-28 16:48:05 +02:00
Boaz Leskes 2512594d9e Testing infra - stablize data folder usage and clean up (#19111)
The plan for persistent node ids ( #17811 ) is to tie the node identity to a file stored in it's data folders. As such it becomes important that nodes in our testing infra have better affinity with their data folders and that their data folders are not cleaned underneath them. The first is important because we fix the random seed used for node id generation (for reproducibility) and allowing the same node to use two different data folders causes two separate nodes to have the same id, which prevents the cluster from forming. The second is important, for example, where a full cluster restart / single node restart need to maintain node identity and wiping the data folders at the wrong moment prevents this.

Concretely this commit does the following:
1) Remove previous attempts to have data folder per role using a prefix. This wasn't effective as it was using the data paths settings which are only used for part of the runs. An attempt to completely separate the paths via the home dir failed due to assumptions made by index custom path about node data folder ordinal uniqueness (see #19076)
2) Change full cluster restarts to start up nodes in the same order their were first created in, only randomly swapping nodes with the same roles.
3) Change test cluster reset methods to first shutdown the unneeded nodes and then re-start the shared nodes that were shut down, so they'll reclaim their data folders.
4) Improve data folder wiping logic and make sure it wipes only folders of "offline" nodes.
5) Add some very basic tests
2016-06-28 16:38:56 +02:00
Jim Ferenczi 6d069078d3 Fixed tests that assumed that broken settings can be updated 2016-06-28 16:14:57 +02:00
Jim Ferenczi f74e03d79b Merge pull request #19122 from jimferenczi/validate_settings_bis
Validates new dynamic settings from the current state
2016-06-28 15:36:09 +02:00
Jim Ferenczi ef0e3db0de Validates new dynamic settings from the current state
Thanks to https://github.com/elastic/elasticsearch/pull/19088 the settings are now validated against dynamic updaters on the master.
Though only the new settings are applied to the IndexService created for the validation.
Because of this we cannot check the transition from one value to another in a dynamic updaters.
This change creates the IndexService from the current settings and validates that the new dynamic settings
can replace the current settings.
This change also removes the validation of dynamic settings when an index is opened.
The validation should have occurred when the settings have been updated.
2016-06-28 15:35:04 +02:00
Nik Everett fa4844c3f4 Pull actions from plugins
Instead of implementing onModule(ActionModule) to register actions,
this has plugins implement ActionPlugin to declare actions. This is
yet another step in cleaning up the plugin infrastructure.

While I was in there I switched AutoCreateIndex and DestructiveOperations
to be eagerly constructed which makes them easier to use when
de-guice-ing the code base.
2016-06-28 08:36:24 -04:00
Clinton Gormley fc9fa3afaf Added release notes for 5.0.0-alpha4 2016-06-28 12:26:03 +02:00
Colin Goodheart-Smithe 26e6a522c7 [TEST] Fixed bounds calculation for extended bounds in histogram agg empty buckets test 2016-06-28 09:37:34 +01:00
David Pilato 802c3876fd Merge remote-tracking branch 'origin/master' 2016-06-28 10:13:19 +02:00