Commit Graph

3785 Commits

Author SHA1 Message Date
Tanguy Leroux 0e7faf1005 Enable Checkstyle RedundantModifier 2016-07-04 15:22:12 +02:00
Jason Tedor 3343ceeae4 Do not catch throwable
Today throughout the codebase, catch throwable is used with reckless
abandon. This is dangerous because the throwable could be a fatal
virtual machine error resulting from an internal error in the JVM, or an
out of memory error or a stack overflow error that leaves the virtual
machine in an unstable and unpredictable state. This commit removes
catch throwable from the codebase and removes the temptation to use it
by modifying listener APIs to receive instances of Exception instead of
the top-level Throwable.

Relates #19231
2016-07-04 08:41:06 -04:00
Jim Ferenczi afe99fcdcd Restore reverted change now that alpha4 is out:
Rename `fields` to `stored_fields` and add `docvalue_fields`

`stored_fields` parameter will no longer try to retrieve fields from the _source but will only return stored fields.
`fields` will throw an exception if the user uses it.
Add `docvalue_fields` as an adjunct to `fielddata_fields` which is deprecated. `docvalue_fields` will try to load the value from the docvalue and fallback to fielddata cache if docvalues are not enabled on that field.

Closes #18943
2016-07-04 10:39:49 +02:00
Tanguy Leroux c8a9b28d1b Rename Search Template REST tests 2016-07-04 10:07:45 +02:00
Ryan Ernst e5caadc4f3 Merge branch 'master' into ingest_plugin_api 2016-07-01 12:35:26 -07:00
Ryan Ernst 65c9b0b588 Merge branch 'master' into ingest_plugin_api 2016-07-01 09:26:17 -07:00
Tanguy Leroux 8c40b2b54e Fix order of modifiers 2016-07-01 16:57:14 +02:00
Simon Willnauer 5c8164a561 Clean up BytesReference (#19196)
BytesReference should be a really simple interface, yet it has a gazillion
ways to achieve the same this. Methods like `#hasArray`, `#toBytesArray`, `#copyBytesArray`
`#toBytesRef` `#bytes` are all really duplicates. This change simplifies the interface
dramatically and makes implementations of it much simpler. All array access has been removed
and is streamlined through a single `#toBytesRef` method. Utility methods to materialize a
compact byte array has been added too for convenience.
2016-07-01 16:09:31 +02:00
Christoph Büscher 42662fd695 Adapt to changes in master 2016-07-01 15:03:17 +02:00
Christoph Büscher 3680069628 Merge branch 'master' into feature/rank-eval 2016-07-01 13:59:56 +02:00
Ryan Ernst 8275ab497b Merge pull request #19170 from rjernst/rest_handler_client
Changed rest handler interface to take NodeClient
2016-06-30 11:00:09 -07:00
Nik Everett f5a269b029 Start migration away from aggregation streams
We'll migrate to NamedWriteable so we can share code with the rest
of the system. So we can work on this in multiple pull requests without
breaking Elasticsearch in between the commits this change supports
*both* old style `InternalAggregations.stream` serialization and
`NamedWriteable` style serialization. As such it creates about a
half dozen `// NORELEASE` comments that will have to be removed
once the migration is complete.

This also introduces a boolean `transportClient` flag to `SearchModule`
which is used to skip inappropriate registrations for for the
transport client while still registering the things it needs. In
this case that means that the `InternalAggregation` subclasses are
registered with the `NamedWriteableRegistry` but the `AggregationBuilder`
subclasses are not.

Finally, this moves aggregation registration from guice configuration
time to `SearchModule` construction time. This will make it simpler to
work with in the future as we further clean up Elasticsearch's
extension points.
2016-06-30 12:57:34 -04:00
Boaz Leskes 09ca6d6ed2 Add a BridgePartition to be used by testAckedIndexing (#19172)
We have long worked to capture different partitioning scenarios in our testing infra. This PR adds a new variant, inspired by the Jepsen blogs, which was forgotten far - namely a partition where one node can still see and be seen by all other nodes. It also updates the resiliency page to better reflect all the work that was done in this area.
2016-06-30 17:58:12 +02:00
Ryan Ernst c762e7aa15 Merge branch 'master' into rest_handler_client 2016-06-30 08:16:25 -07:00
Ryan Ernst 0732004ae8 Merge pull request #19177 from rjernst/ingest_factory_generic
Remove generics from ingest Processor.Factory
2016-06-30 08:08:26 -07:00
Christoph Büscher be19d13d14 Adapting to changes in Plugin api 2016-06-30 16:18:05 +02:00
Isabel Drost-Fromm 6d4673fd58 Initial commit for Module to compute metrics on queries
This is an initial squashed commit of the work on a new feature for query metrics
proposed in #18798.
2016-06-30 16:15:08 +02:00
Tanguy Leroux dc53ce929d Document Update/Delete-By-Query with version number zero
Update-By-Query and Delete-By-Query use internal versioning to update/delete documents. But documents can have a version number equal to zero using the external versioning... making the UBQ/DBQ request fail because zero is not a valid version number and they only support internal versioning for now. Sequence numbers might help to solve this issue in the future.
2016-06-30 15:45:14 +02:00
Nik Everett d57b780bb4 Remote TransportRethrottleAction from RestRethrottleAction
Just use the client to call it.
2016-06-30 09:36:31 -04:00
Nik Everett e359be7632 Don't inject TransportPercolateAction into RestPercolateAction
Instead use the client. This will help us build the actions more
easily in the future.
2016-06-30 09:36:31 -04:00
Martijn van Groningen 4c2d6cf538 percolator: removed unused code 2016-06-30 14:43:28 +02:00
Ryan Ernst e4f265eb3a Ingest: Remove generics from Processor.Factory
The factory for ingest processor is generic, but that is only for the
return type of the create mehtod. However, the actual consumer of the
factories only cares about Processor, so generics are not needed.

This change removes the generic type from the factory. It also removes
AbstractProcessorFactory which only existed in order pull the optional
tag from config. This functionality is moved to the caller of the
factories in ConfigurationUtil, and the create method now takes the tag.
This allows the covariant return of the implementation to work with
tests not needing casts.
2016-06-30 02:33:54 -07:00
Ryan Ernst 08b3b6264e Tests pass, started removing generics from processor factory 2016-06-30 01:49:22 -07:00
Ryan Ernst f4519c44b7 Merge branch 'master' into ingest_plugin_api 2016-06-29 22:38:23 -07:00
Ryan Ernst c77dc4a82c Merge pull request #19136 from rjernst/script_service_deps
Scripts: Remove ClusterState from compile api
2016-06-29 22:34:40 -07:00
Ryan Ernst 865b951b7d Internal: Changed rest handler interface to take NodeClient
Previously all rest handlers would take Client in their injected ctor.
However, it was only to hold the client around for runtime. Instead,
this can be done just once in the HttpService which handles rest
requests, and passed along through the handleRequest method. It also
should always be a NodeClient, and other types of Clients (eg a
TransportClient) would not work anyways (and some handlers can be
simplified in follow ups like reindex by taking NodeClient).
2016-06-29 18:02:18 -07:00
Nik Everett 8db43c0107 Move RestHandler registration to ActionModule and ActionPlugin
`RestHandler`s are highly tied to actions so registering them in the
same place makes sense.

Removes the need to for plugins to check if they are in transport client
mode before registering a RestHandler - `getRestHandlers` isn't called
at all in transport client mode.

This caused guice to throw a massive fit about the circular dependency
between NodeClient and the allocation deciders. I broke the circular
dependency by registering the actions map with the node client after
instantiation.
2016-06-29 18:31:44 -04:00
Ryan Ernst f1376262fe Merge branch 'master' into ingest_plugin_api 2016-06-29 14:16:16 -07:00
Tanguy Leroux 4820d49120 Mustache: Add util functions to render JSON and join array values
This pull request adds two util functions to the Mustache templating engine:
- {{#toJson}}my_map{{/toJson}} to render a Map parameter as a JSON string
- {{#join}}my_iterable{{/join}} to render any iterable (including arrays) as a comma separated list of values like `1, 2, 3`. It's also possible de change the default delimiter (comma) to something else.

closes #18970
2016-06-29 09:48:58 +02:00
Martijn van Groningen b97ea9954c percolator: Use RamDirectory for percolating nested document instead of using multiple MemoryIndex instances with SlowCompositeReaderWrapper workaround 2016-06-29 08:50:01 +02:00
Nik Everett 67bfecc070 Painless: add "".replaceAll and "".replaceFirst
These are useful methods in groovy that give you control over
the replacements used:
```
'the quick brown fox'.replaceAll(/[aeiou]/,
		m -> m.group().toUpperCase(Locale.ROOT))
```
2016-06-28 16:39:11 -04:00
Ryan Ernst ecf6101798 Scripts: Remove ClusterState from compile api
Stored scripts are pulled from the cluster state, and the current api
requires passing the ClusterState on each call to compile. However, this
means every user of the ScriptService needs to depend on the
ClusterService. Instead, this change makes the ScriptService a
ClusterStateListener. It also simplifies tests a lot, as they no longer
need to create fake cluster states (except when testing stored scripts).
2016-06-28 13:20:00 -07:00
Ryan Ernst 258c3e86ab Added IngestPlugin api, cutover common and geoip, changed ingest factory
api to take ProcessorsRegistry
2016-06-28 10:52:07 -07:00
Robert Muir 6d52cec2a0 Merge pull request #19092 from rmuir/more_painless_docs
cutover some docs to painless
2016-06-28 13:40:25 -04:00
Yannick Welsch 0515791846 Fix logger usages 2016-06-28 16:51:06 +02:00
Nik Everett fa4844c3f4 Pull actions from plugins
Instead of implementing onModule(ActionModule) to register actions,
this has plugins implement ActionPlugin to declare actions. This is
yet another step in cleaning up the plugin infrastructure.

While I was in there I switched AutoCreateIndex and DestructiveOperations
to be eagerly constructed which makes them easier to use when
de-guice-ing the code base.
2016-06-28 08:36:24 -04:00
Colin Goodheart-Smithe 26e6a522c7 [TEST] Fixed bounds calculation for extended bounds in histogram agg empty buckets test 2016-06-28 09:37:34 +01:00
Alexander Reelsen ab8ff8909b Tests: Rename task.get to tasks.get
The task.get action got renamed to tasks.get, some tests
did not change this.

Relates #19107
2016-06-28 09:13:19 +02:00
Ryan Ernst 33ccc5aead Merge branch 'master' into mapper_plugin_api 2016-06-27 11:19:59 -07:00
Jim Ferenczi eb1e231a63 Revert "Rename `fields` to `stored_fields` and add `docvalue_fields`"
This reverts commit 2f46f53dc8.
2016-06-27 17:20:32 +02:00
Robert Muir 6fc1a22977 cutover some docs to painless 2016-06-27 09:55:16 -04:00
Colin Goodheart-Smithe 108ba23073 Pass resolved extended bounds to unmapped histogram aggregator
Previous to this change the unresolved extended bounds was passed into the histogram aggregator which meant extendedbounds.min and extendedbounds.max was passed through as null. This had two effects on the histogram aggregator:

1. If the histogram aggregator was unmapped across all shards, the reduce phase would not add buckets for the extended bounds and the response would contain zero buckets
2. If the histogram aggregator was not unmapped in some shards, the reduce phase might sometimes chose to reduce based on the unmapped shard response and therefore the extended bounds would be ignored.

This change resolves the extended bounds in the unmapped case and solves the above two issues.

Closes #19009
2016-06-27 14:07:37 +01:00
Martijn van Groningen d3cd58eb2f Merges PR #18957
This commit fixes several NPEs caused by implicitly performing a get request for a document that exists with its _source disabled and then trying to access the source. Instead of causing an NPE the following queries will throw an exception with a "source disabled" message (similar behavior as if the document does not exist).:
- GeoShape query for pre-indexed shape (throws IllegalArgumentException)
- Percolate query for an existing document (throws IllegalArgumentException)

A Terms query with a lookup will ignore the document if the source does not exist (same as if the document does not exist).

GET and HEAD requests for the document _source will return a 404 if the source is disabled (even if the document exists).
2016-06-27 09:37:28 +02:00
Martijn van Groningen 9a0ce62550 percolator: Add support for the synonym query. 2016-06-27 07:42:44 +02:00
Nik Everett 71b95fb63c Switch analysis from push to pull
Instead of plugins calling `registerTokenizer` to extend the analyzer
they now instead have to implement `AnalysisPlugin` and override
`getTokenizer`. This lines up extending plugins in with extending
scripts. This allows `AnalysisModule` to construct the `AnalysisRegistry`
immediately as part of its constructor which makes testing anslysis
much simpler.

This also moves the default analysis configuration into `AnalysisModule`
which is how search is setup.

Like `ScriptModule`, `AnalysisModule` no longer extends `AbstractModule`.
Instead it is only responsible for building `AnslysisRegistry`. We still
bind `AnalysisRegistry` but we only do so in `Node`. This is means it
is available at module construction time so we slowly remove the need to
bind it in guice.
2016-06-26 07:15:42 -04:00
Alex Benusovich 3ca909dfea Fix NPEs due to disabled source
This commit fixes several NPEs caused by implicitly performing a get request for a document that exists with its _source disabled and then trying to access the source. Instead of causing an NPE the following queries will throw an exception with a "source disabled" message (similar behavior as if the document does not exist).:
- GeoShape query for pre-indexed shape (throws IllegalArgumentException)
- Percolate query for an existing document (throws IllegalArgumentException)

A Terms query with a lookup will ignore the document if the source does not exist (same as if the document does not exist).

GET and HEAD requests for the document _source will return a 404 if the source is disabled (even if the document exists).
2016-06-24 22:03:03 -07:00
Ryan Ernst 6995bde710 Merge branch 'master' into mapper_plugin_api 2016-06-24 11:15:06 -07:00
Robert Muir 0b2baa7f63 Merge pull request #19065 from rmuir/help_painless_docs
Bring painless docs closer to reality
2016-06-24 12:52:30 -04:00
Robert Muir e6819648cc fix the primitive case of instanceof 2016-06-24 12:44:57 -04:00
Robert Muir 001a060c84 Bring painless docs closer to reality 2016-06-24 12:06:41 -04:00
Martijn van Groningen 599a548998 percolator: Don't verify candidate matches with MemoryIndex that are verified matches
If we don't care about scoring then for certain candidate matches we can be certain, that if they are a candidate match,
then they will always match. So verifying these queries with the MemoryIndex can be skipped.
2016-06-24 15:46:55 +02:00
Robert Muir f35cd0e30a Merge pull request #19022 from rmuir/loopCounter
Fix disabled loop counter
2016-06-23 07:45:06 -04:00
Tanguy Leroux 04da1bda0d Move templates out of the Search API, into lang-mustache module
This commit moves template support out of the Search API to its own dedicated Search Template API in the lang-mustache module. It provides a new SearchTemplateAction that can be used to render templates before it gets delegated to the usual Search API. The current REST endpoint are identical, but the Render Search Template endpoint now uses the same Search Template API with a new "simulate" option. When this option is enabled, the Search Template API only renders template and returns immediatly, without executing the search.

Closes #17906
2016-06-23 09:30:53 +02:00
Jim Ferenczi 2f46f53dc8 Rename `fields` to `stored_fields` and add `docvalue_fields`
`stored_fields` parameter will no longer try to retrieve fields from the _source but will only return stored fields.
`fields` will throw an exception if the user uses it.
Add `docvalue_fields` as an adjunct to `fielddata_fields` which is deprecated. `docvalue_fields` will try to load the value from the docvalue and fallback to fielddata cache if docvalues are not enabled on that field.

Closes #18943
2016-06-22 17:38:30 +02:00
Robert Muir f8c55a5e7b painless: fix disabled loop counter 2016-06-22 08:40:20 -04:00
Adrien Grand db9af54ec0 Remove `_timestamp` and `_ttl` on 5.x indices. #18980
This removes the ability to use `_timestamp` and `_ttl` on indices created on
or after 5.0.

Closes #18280
2016-06-22 08:35:54 +02:00
Ryan Ernst e817b5daa3 Plugins: Remove guice from Mapper plugins
This changes adds a MapperPlugin interface which allows pull style
retrieval of mappers and metadata mappers added by plugins. For now, I
have kept the MapperRegistry, but this should be removed in the future
as it is just a silly container for 2 maps which could themselves be
passed around.
2016-06-21 22:50:39 -07:00
Nik Everett 8925400f67 Remove guice from ScriptService
Makes ScriptModule just a plain class that manages building the
ScriptSettings and ScriptService from plugins. When we *need*
to bind ScriptService with guice we bind it in a lambda.
2016-06-21 16:45:45 -04:00
Tal Levy 28fd684eef Fix ignore_failure behavior in _simulate?verbose (#18987)
- fix it so that processors with the `ignore_failure` option do not
record their exception in the response
- add more tests to make empty `on_failure`. This now throws an
  exception
2016-06-21 13:29:53 -07:00
Jack Conradson 0b4fc85367 Added some brief docs. 2016-06-21 12:56:54 -07:00
Jack Conradson 346b1802ee Quick fixes for using built in method writes. 2016-06-21 12:53:04 -07:00
Jack Conradson 553214d771 Merge branch 'master' into init2 2016-06-21 12:33:42 -07:00
Jack Conradson d2c823e4cc Add initializers to Painless for arrays, lists, and maps. 2016-06-21 12:32:10 -07:00
Nik Everett 5f0292cb81 Fetch result when wait_for_completion
This makes this sequence:
```
curl -XDELETE localhost:9200/source,dest?pretty
for i in $( seq 1 100 ); do
  curl -XPOST localhost:9200/source/test -d'{"test": "test"}'; echo
done
curl localhost:9200/_refresh?pretty

curl -XPOST 'localhost:9200/_reindex?pretty&wait_for_completion=false' -d'{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest"
  }
}'

curl 'localhost:9200/_tasks/Jsyd6d9wSRW-O-NiiKbPcQ:237?wait_for_completion&pretty'
```

Return task *AND* the response to the user.

This also renames "result" to "response" in the persisted task info
to line it up with how we name the objects in Elasticsearch.
2016-06-21 14:18:53 -04:00
Adrien Grand 8078c205f9 Revert "Remove `_timestamp` and `_ttl` on 5.x indices. #18980"
This reverts commit 969e953645.
Docs are failing because of the removed functionality. I will
fix the docs before pushing it again.
2016-06-21 19:19:49 +02:00
Robert Muir f70211dedb Merge pull request #19003 from rmuir/augmentation
painless: add augmentation
2016-06-21 13:15:41 -04:00
Robert Muir 1b9695a9aa beef up tests so we ensure you still get good errors in these cases 2016-06-21 12:15:59 -04:00
Robert Muir f78ef232dc fix bogus comment 2016-06-21 12:05:10 -04:00
Adrien Grand 969e953645 Remove `_timestamp` and `_ttl` on 5.x indices. #18980
This removes the ability to use `_timestamp` and `_ttl` on indices created on
or after 5.0.

Closes #18280
2016-06-21 18:04:58 +02:00
Robert Muir 42d60f9f28 maps n lists 2016-06-21 11:25:43 -04:00
Martijn van Groningen 0488372c20 removed unused imports 2016-06-21 15:51:20 +02:00
Jim Ferenczi 881afcba60 Fixed tests that failed now that BM25 is the default similarity. 2016-06-21 15:42:42 +02:00
Robert Muir 80734c75b5 get things started 2016-06-21 08:35:12 -04:00
Martijn van Groningen 82f7bfad98 ingest: merged o.e.ingest.core with o.e.ingest and in ingest-common module added o.e.ingest.common package
and moved all code to that package.
2016-06-21 09:24:00 +02:00
Robert Muir 1b7d35e4a7 Merge pull request #18983 from rmuir/lambda_types
Infer lambda arguments/return type
2016-06-20 17:22:04 -04:00
Robert Muir 1cc0264827 Infer lambda arguments/return type 2016-06-20 14:54:45 -04:00
Robert Muir fea120b073 Merge branch 'master' into explicit_casts 2016-06-20 13:39:13 -04:00
Robert Muir 006829e89b add simple arguments test 2016-06-20 13:33:51 -04:00
Robert Muir 09305a0f98 Merge pull request #18954 from rmuir/lambda_captures
Painless: add lambda captures
2016-06-20 13:05:55 -04:00
Robert Muir 26a73c39bf throw error if the tree is totally malformed 2016-06-20 13:01:59 -04:00
Robert Muir 9510a8f39e add a few more tests 2016-06-20 10:46:23 -04:00
Robert Muir 28b1b149ab remove unused import 2016-06-20 08:32:53 -04:00
Robert Muir 9111ed3e2c add docs 2016-06-20 08:24:41 -04:00
Robert Muir cd1a7b441c Improve error messages for lambdas when the number of arguments is wrong 2016-06-20 07:57:00 -04:00
Robert Muir 4d78be5b9e remove arity restriction (as def call incorporates all lambdas and all their captures) 2016-06-20 05:37:31 -04:00
Simon Willnauer 7fea5bd8e7 Remove obsolete Modules that can simply be inlined in node creation 2016-06-20 11:28:14 +02:00
Robert Muir b53d735602 Function/Lambda parameters do not need to be lenient... 2016-06-20 05:05:34 -04:00
Tanguy Leroux 98951b1203 Compile each Groovy script in its own classloader
closes #18572
2016-06-20 08:17:09 +02:00
Uwe Schindler 5475e18ad0 Update forbiddenapis to 2.2 and fix painless tests 2016-06-19 20:40:38 +02:00
Robert Muir 8d9fa7e0b5 Fix explicit casts and improve tests. 2016-06-19 03:19:45 -04:00
Robert Muir a14ba1e5b2 Painless: add lambda captures 2016-06-18 10:23:35 -04:00
Robert Muir b0efca6473 Merge pull request #18951 from rmuir/refactor_variables
Refactor painless variables handling
2016-06-18 08:21:14 -04:00
Simon Willnauer 5c98818f48 fix compilation 2016-06-18 10:54:12 +02:00
Robert Muir e8826708c1 Refactor variables 2016-06-17 17:40:52 -04:00
Jeff Evans e9f2548ee0 Include script field even if it value is null
Include script field even if it value is null.

Closes #16408.
2016-06-17 16:41:25 -04:00
Nik Everett 1e16c22d03 Painless: move semicolon hack into lexer
Perviously we used token level lookbehind in the parser. That worked,
but only if the parser didn't have any ambiguity *at all*. Since the
parser has ambiguity it didn't work everywhere. In particular it failed
when parsing blocks in lambdas like `a -> {int b = a + 2; b * b}`.

This moves the hack from the parser into the lexer. There we can use
token lookbehind (same trick) to *insert* semicolons into the token
stream. This works much better for antlr because antlr's prediction
code can work with real tokens.

Also, the lexer is simpler than the parser, so if there is a place
to introduce a hack, that is a better place.
2016-06-17 16:18:41 -04:00
Areek Zillur 9356a6090f Merge branch 'master' into enhancement/rollover_api 2016-06-17 11:35:57 -04:00
Jim Ferenczi fb2a48d0f0 Revert "Remove support for sorting terms aggregation by ascending count"
This is delayed after alpha4 since Kibana relies on it.
2016-06-17 17:14:01 +02:00
Simon Willnauer bdb6dcea3a Cleanup ClusterService dependencies and detached from Guice (#18941)
This change removes some unnecessary dependencies from ClusterService
and cleans up ClusterName creation. ClusterService is now not created
by guice anymore.
2016-06-17 17:07:19 +02:00
Areek Zillur 545ffa7801 Merge branch 'master' into enhancement/rollover_api 2016-06-17 10:33:11 -04:00
Jim Ferenczi 755721953b Remove support for sorting terms aggregation by ascending count
closes #17614
2016-06-17 15:06:49 +02:00
Adrien Grand 712e387058 Rename PipelineAggregatorBuilder to PipelineAggregationBuilder.
This is a follow-up to #18377.
2016-06-17 14:35:49 +02:00
Adrien Grand 600cbb6ab0 Upgrade to Lucene 6.1.0. #18926 2016-06-17 09:03:00 +02:00
Uwe Schindler a7aedbe0a1 Fix compound assignment with string concats. in Java 9 there is no stringbuilder on stack! This closes #18929 2016-06-17 00:30:09 +02:00
Robert Muir d741e65da1 Merge pull request #18932 from rmuir/painless_debug_exception
improve Debugger to print code even if it hits exception
2016-06-16 17:47:14 -04:00
Robert Muir 2a3184604e improve Debugger to print code even if it hits exception 2016-06-16 17:34:50 -04:00
Areek Zillur 6adffa6b7b Merge branch 'master' into enhancement/rollover_api 2016-06-16 17:27:32 -04:00
Ryan Ernst 1600e56801 Merge branch 'master' into plugin_name_api 2016-06-16 13:49:48 -07:00
Ryan Ernst 8196cf01e3 Merge branch 'master' into plugin_name_api 2016-06-16 13:49:28 -07:00
Nik Everett 13d16fbf41 Painless: Disable java-9 indy string thing
It is breaking some of the doc tests. Also add a unit test that
shows the failure.

Relates to #18929
2016-06-16 15:06:01 -04:00
Nik Everett b665d8a187 Painless: Add flag support to regexes
Painless: Add support for //m
Painless: Add support for //s
Painless: Add support for //i
Painless: Add support for //u
Painless: Add support for //U
Painless: Add support for //l
  This means "literal" and is exposed for completeness sake with
  the java api.
Painless: Add support for //c
  c enables Java's CANON_EQ (canonical equivalence) flag which makes
  unicode characters that are canonically equal match. Java's javadoc
  gives "a\u030A" being equal to "\u00E5". That is that the "a" code
  point followed by the "combining ring above" code point is equal to
  the "a with combining ring above" code point.
Update docs and add multi-flag test
Whitelist most of the Pattern class.
2016-06-16 15:00:31 -04:00
Robert Muir 251001e435 add more simple tests 2016-06-16 12:40:30 -04:00
Robert Muir fdd1f152a7 improve lambda syntax (allow single expression) 2016-06-16 11:02:06 -04:00
Robert Muir ccad99fb5c Merge pull request #18911 from rmuir/noncapturing_lambdas
non-capturing lambda support
2016-06-16 10:31:54 -04:00
Simon Willnauer b22c526b34 Cut over settings registration to a pull model (#18890)
Today we have a push model for registering basically anything. All our extension points
are defined on modules which we pass in to plugins. This is harder to maintain and adds
unnecessary dependencies on the modules itself. This change moves towards a pull model
where the plugin offers a getter kind of method to get the extensions. This will also
help in the future if we need to pass dependencies to the extension points which can
easily be defined on the method as arguments if a pull model is used.
2016-06-16 15:52:58 +02:00
Nik Everett 8d3ef742db Painless: =~ and ==~ operators
Adds support for the find operator (=~) and the match operator (==~)
to painless's regexes. Also whitelists most of the Matcher class and
documents regex support in painless.

The find operator (=~) returns a boolean that is the result of building
a matcher on the lhs with the Pattern on the RHS and calling `find` on
it. Use it like this:

```
if (ctx._source.last =~ /b/)
```

The match operator (==~) returns boolean like find but instead of calling
`find` on the Matcher it calls `matches`.

```
if (ctx._source.last ==~ /[^aeiou].*[aeiou]/)
```

Finally, if you want the actual matcher you do:

```
Matcher m = /[aeiou]/.matcher(ctx._source.last)
```
2016-06-16 08:42:33 -04:00
Uwe Schindler 4293030fd9 painless: Remove stale comment and remove declared exception 2016-06-16 11:46:11 +02:00
Uwe Schindler f2d6219426 painless: remove useless dropArguments and throws statement in megamorphic cache; add tests 2016-06-16 11:08:10 +02:00
Simon Willnauer 18ff051ad5 Simplify ScriptModule and script registration (#18903)
Registering a script engine or native scripts still uses Guice today
and is much more complicated than needed. This change moves to a pull
based model where script plugins have to implement a dedicated interface
`ScriptPlugin` and defines simple getter returning instances rather than
classes.
2016-06-16 09:35:13 +02:00
Robert Muir ec7215e5f2 get non-capturing lambdas working 2016-06-16 01:25:43 -04:00
Robert Muir f7b7204721 Merge pull request #18907 from rmuir/fix_horrible_capture
Fix horrible capture
2016-06-15 20:54:15 -04:00
Robert Muir 60176afdde clean up a bit more 2016-06-15 20:30:16 -04:00
Ryan Ernst a4503c2aed Plugins: Remove name() and description() from api
In 2.0 we added plugin descriptors which require defining a name and
description for the plugin. However, we still have name() and
description() which must be overriden from the Plugin class. This still
exists for classpath plugins. But classpath plugins are mainly for
tests, and even then, referring to classpath plugins with their class is
a better idea. This change removes name() and description(), replacing
the name for classpath plugins with the full class name.
2016-06-15 17:12:22 -07:00
Robert Muir 54797b7d09 don't let megamorphic cache "capture" arbitrary arguments.
pass the shit we need as bootstrap params
2016-06-15 19:33:59 -04:00
Tal Levy a26260fb72 new ScriptProcessor for Ingest (#18193)
add new ScriptProcessor for executing ES Scripts within pipelines
2016-06-15 14:57:18 -07:00
Robert Muir 154d750e4b Merge pull request #18899 from rmuir/more_def_cleanup
fix bugs in operators and more improvements for the dynamic case
2016-06-15 17:06:46 -04:00
Uwe Schindler ae6ce1c42e Small refactoring and cleanup to separate the polymorphic and megamorphic case 2016-06-15 22:27:30 +02:00
Uwe Schindler 6e83f41e94 Merge branch 'more_def_cleanup' of https://github.com/rmuir/elasticsearch into more_def_cleanup 2016-06-15 21:47:28 +02:00
Uwe Schindler 5b11b15300 fix typo 2016-06-15 21:46:58 +02:00
Robert Muir 786c3b0fa8 speed up compound assignments 2016-06-15 14:38:37 -04:00
Robert Muir a4036b8069 add compound assignment tests 2016-06-15 14:02:19 -04:00
Uwe Schindler 949e99bde3 Merge branch 'more_def_cleanup' of https://github.com/rmuir/elasticsearch into more_def_cleanup 2016-06-15 19:48:53 +02:00
Uwe Schindler 8d3c411298 Remove unneeded type adaption and add error message on violation 2016-06-15 19:48:38 +02:00
Robert Muir e3f34730ae and compound assignment 2016-06-15 13:37:10 -04:00
Robert Muir 27f8b6e6db get compound assignment working 2016-06-15 13:30:21 -04:00
Uwe Schindler ca2e0e1660 Some small cleanups around the lookup 2016-06-15 19:07:13 +02:00
Uwe Schindler 767bb49da8 Fix bug with first param (receiver) on the cache lookup 2016-06-15 19:00:22 +02:00
Uwe Schindler 283410b7ac Merge branch 'more_def_cleanup' of https://github.com/rmuir/elasticsearch into more_def_cleanup 2016-06-15 18:37:07 +02:00
Uwe Schindler 04ab42cdf5 Use rethrow puzzler in the ClassValue 2016-06-15 18:36:45 +02:00
Robert Muir d4db8c9c82 some cleanup 2016-06-15 12:32:38 -04:00
Uwe Schindler 6487940a79 Move type changes up to static part 2016-06-15 18:26:11 +02:00
Uwe Schindler a473298f44 Add megamorphic cache 2016-06-15 18:21:42 +02:00
Simon Willnauer 7df5d05c62 Simplify SubFetchPhase interface (#18881)
This interface used to have dedicated methods to prevent calling execute
methods. These methods are unnecessary as the checks can simply be
done inside the execute methods itself. This simplifies the interface
as well as its usage.
2016-06-15 15:49:11 +02:00
Robert Muir b71f42a627 split MIC from PIC 2016-06-15 09:07:11 -04:00
Simon Willnauer 429dd3a876 Simplify FetchSubPhase registration and detach it from Guice (#18862)
this commit removes FetchSubPhrase registration by class to registration
by instance. No Guice binding needed anymore.
2016-06-15 09:13:02 +02:00
Robert Muir 2b1ebc55a9 remove null guards from +, if not needed.
improve DefBootstrapTests
2016-06-14 23:52:21 -04:00
Robert Muir 5e4b2f2815 clean up compound assignment tests, fix shift bug 2016-06-14 20:40:16 -04:00
Robert Muir cd4f4594d6 fix perf bug with xor promotion 2016-06-14 19:19:50 -04:00
Jack Conradson 5cef06fa3e Add a test. 2016-06-14 14:28:47 -07:00
Jack Conradson 36f7632232 Correct type checking during casting related to Object. 2016-06-14 14:12:23 -07:00
Robert Muir b69e8aadbd refactor tests 2016-06-14 15:07:56 -04:00
Nik Everett e392e0b1df Create get task API that falls back to the .tasks index
This adds a get task API that supports GET /_tasks/${taskId} and
removes that responsibility from the list tasks API. The get task
API supports wait_for_complation just as the list tasks API does
but doesn't support any of the list task API's filters. In exchange,
it supports falling back to the .results index when the task isn't
running any more. Like any good GET API it 404s when it doesn't
find the task.

Then we change reindex, update-by-query, and delete-by-query to
persist the task result when wait_for_completion=false. The leads
to the neat behavior that, once you start a reindex with
wait_for_completion=false, you can fetch the result of the task by
using the get task API and see the result when it has finished.

Also rename the .results index to .tasks.
2016-06-14 13:37:34 -04:00
Robert Muir e4dc469e58 Merge pull request #18849 from rmuir/give_indy_ops_types
Remove casts and boxing for dynamic math
2016-06-14 13:01:04 -04:00
Robert Muir 79f0ab2fc6 rename this variable 2016-06-14 12:51:29 -04:00
Colin Goodheart-Smithe d7e3f9e4eb #18854 Remove size 0 options in aggregations
Remove size 0 options in aggregations
2016-06-14 15:32:42 +01:00
Nik Everett 49c09f1527 Test: Fix error message check for Windows
Betrayal! CRLF strikes again. Weakened the test from equals to
containsString.
2016-06-14 09:56:49 -04:00
Colin Goodheart-Smithe bec621d46f changes from review 2016-06-14 13:45:03 +01:00
Colin Goodheart-Smithe cfd3356ee3 Remove size 0 options in aggregations
This removes the ability to set `size: 0` in the `terms`, `significant_terms` and `geohash_grid` aggregations for the reasons described in https://github.com/elastic/elasticsearch/issues/18838

Closes #18838
2016-06-14 13:07:02 +01:00
Robert Muir a12937d548 painless: remove casts and boxing for dynamic math 2016-06-14 00:49:13 -04:00
Robert Muir 7ecb4ca4e9 Refactor def math 2016-06-13 18:56:28 -04:00
Nik Everett 6617b53422 Painless: Add support for /regex/
Adds `/regex/` as a regex constructor. A couple of fun points:
1. This makes generic the idea of arbitrary stuff adding a constant.
Both SFunction and LRegex create a statically initialized constant.
Both go through Locals to do this because they LRegex isn't directly
iterable from SScript.
2. Differentiating `/` as-in-division from `/` as-in-start-of-regex
is hard. See:
http://www-archive.mozilla.org/js/language/js20-2002-04/rationale/syntax.html#regular-expressions
The javascript folks have a way, way tougher time of it then we do
because they have semicolon insertion. We have the much simpler
delimiter rules. Even with our simpler life we still have to add
a hack to get lexing `/regex/` to work properly. I chose to add
token-level lookbehind because it seems to be a pretty contained hack.
I considered and rejected lexer modes, a lexer member variable,
having the parser set variables on the lexer (this is a fairly common
solution for js, I believe), and moving regex parsing to the parser
level.
3. I've only added a very small subset of java.util.regex to the
whitelist because it is the subset I needed to test LRegex sanely.
More deserves to be added, and maybe more regex syntax like `=~` and
`==~`. Those can probably be added without too much pain.
2016-06-13 18:11:45 -04:00
Martijn van Groningen f132959c38 Merge pull request #18845 from martijnvg/percolator_dismax_query
Add percolator query extraction support for dismax query
2016-06-13 20:27:50 +02:00
Martijn van Groningen b8309b143f percolator: add query extraction support for dismax query 2016-06-13 20:14:35 +02:00
Robert Muir f295754498 Merge pull request #18831 from rmuir/moreRefs
painless: array constructor references
2016-06-13 13:36:26 -04:00
Robert Muir c6e144f669 remove stray tab, fix comment 2016-06-13 13:35:11 -04:00
Martijn van Groningen 8287e13616 test: fixed test bug 2016-06-13 15:58:36 +02:00
Martijn van Groningen 5e3f2ce7aa test: Added extra tests for percolating with nested documents. 2016-06-13 15:29:32 +02:00
Simon Willnauer 7379b17e61 Revert "Make random UUIDs reproducible in tests"
This reverts commit a25b8ee1bf.
2016-06-13 11:14:30 +02:00
Simon Willnauer f1d5fd72c8 Revert "Mark field in ReindexSameIndexTests as final"
This reverts commit 6d8692576e.
2016-06-13 11:14:30 +02:00
Robert Muir faee2323ab add array ctor refs 2016-06-12 12:53:37 -04:00
Robert Muir 5be9211761 cleanup grammar 2016-06-12 11:25:52 -04:00
Uwe Schindler 60cc152caf Remove unused field 2016-06-11 22:34:17 +02:00
Robert Muir 5c352a063a need not be public, should not be public 2016-06-11 16:03:45 -04:00
Robert Muir 07baefac32 Add comments 2016-06-11 16:01:50 -04:00
Uwe Schindler fd9b0f9087 Remove some hardcoded strings, fix exception handling (remove rethrow), add some utility methods around the "handle$" fields 2016-06-11 21:53:15 +02:00
Robert Muir 454881eeb4 test interface default methods 2016-06-11 15:51:32 -04:00
Uwe Schindler c08b45a277 More cleanup 2016-06-11 21:20:39 +02:00
Robert Muir 65352face2 fix too long line 2016-06-11 15:04:49 -04:00
Robert Muir 592042601f def case working 2016-06-11 15:02:43 -04:00
Uwe Schindler 876f0a613e Unfuck the MethodWriter nesting. All MethodWriters should only live separately from each other, no nesting 2016-06-11 21:00:06 +02:00
Robert Muir e2f999b025 make this static final 2016-06-11 14:33:48 -04:00
Robert Muir 8d5a70200f write refs for functions 2016-06-11 14:29:15 -04:00
Robert Muir ab457be9d8 remove unnecessary semicolon and return 2016-06-11 13:48:04 -04:00
Robert Muir 0bfb166eeb merge master 2016-06-11 13:23:17 -04:00
Nik Everett 4a265d0279 Painless: Add } as a delimiter. Kindof.
Add `}` is statement delimiter but only in places where it is
otherwise a valid part of the syntax, specificall the end of a block.
We do this by matching but not consuming it. Antlr 4 doesn't have
syntax for this so we have to kind of hack it together by actually
matching the `}` and then seeking backwards in the token stream to
"unmatch" it. This looks reasonably efficient. Not perfect, but way
better than the alternatives.

I tried and rejected a few options:
1. Actually consuming the `}` and piping a boolean all through the
grammar from the last statement in a block to the delimiter. This
ended up being a rather large change and made the grammar way more
complicated.
2. Adding a semantic predicate to delimiter that just does the
lookahead. This doesn't work out well because it doesn't work (I
never figured out why) and because it generates an *amazing*
`adaptivePredict` which makes a super huge DFA. It looks super
inefficient.

Closes #18821
2016-06-11 12:51:25 -04:00
Jack Conradson e293000d8d Merge branch 'master' into stubby 2016-06-10 16:22:17 -07:00
Jack Conradson bd94c62adf Added a stub node for lambda expressions. 2016-06-10 16:18:36 -07:00
Nik Everett 387155559e Make TimeValue Writeable instead of Streamable
Writeable is better for immutable objects like TimeValue.

Switch to writeZLong which takes up less space than the original
writeLong in the majority of cases. Since we expect negative
TimeValues we shouldn't use
writeVLong.
2016-06-10 18:24:16 -04:00
Robert Muir 16d17351f7 don't use loadLocal, add new node to package-info 2016-06-10 17:37:57 -04:00
Robert Muir 9e0a70c4b2 Merge branch 'master' into capturingReferences 2016-06-10 17:10:56 -04:00
Jack Conradson f98be6fa06 Add functions to Painless.
Closes #18810
2016-06-10 13:59:59 -07:00
Robert Muir edb0fd3b88 painless: add capturing method references 2016-06-10 14:24:58 -04:00
Jason Tedor 6d8692576e Mark field in ReindexSameIndexTests as final
This commit restores a final modifier on the field
AutoCreateIndex#AUTO_CREATE_INDEX that was inadvertently removed in
a25b8ee1bf.
2016-06-10 10:20:45 -04:00
Adrien Grand 44c653f5a8 Upgrade to lucene-6.1.0-snapshot-3a57bea. 2016-06-10 16:18:12 +02:00
Jason Tedor a25b8ee1bf Make random UUIDs reproducible in tests
Today we use a random source of UUIDs for assigning allocation IDs,
cluster IDs, etc. Yet, the source of randomness for this is not
reproducible in tests. Since allocation IDs end up as keys in hash maps,
this means allocation decisions and not reproducible in tests and this
leads to non-reproducible test failures. This commit modifies the
behavior of random UUIDs so that they are reproducible under tests. The
behavior for production code is not changed, we still use a true source
of secure randomness but under tests we just use a reproducible source
of non-secure randomness.

It is important to note that there is a test,
UUIDTests#testThreadedRandomUUID that relies on the UUIDs being truly
random. Thus, we have to modify the setup for this test to use a true
source of randomness. Thus, this is one test that will never be
reproducible but it is intentionally so.

Relates #18808
2016-06-10 10:18:06 -04:00
Robert Muir 6202b63ba7 Merge pull request #18789 from rmuir/java9_localdate_you_just_had_to_do_this_didnt_you
painless: exempt LocalDate from bridge method checks on java 9
2016-06-08 17:38:28 -04:00
Robert Muir 9826b64e1f Exempt LocalDate from bridge method checks on java 9
The painless whitelist has a lot of self-checking, in this case, it checks
for missing covariant overrides. It fails on java 9, because LocalDate.getEra()
now returns IsoEra instead of Era: https://bugs.openjdk.java.net/browse/JDK-8072746

To our checker, it thinks we were lazy with whitelisting :)

This means painless works on java 9 again
2016-06-08 15:47:09 -04:00
Nik Everett bd1af34506 [reindex] Extract runnable to inner class
Makes it more readable
2016-06-08 14:00:15 -04:00
Nik Everett 2437313e4e Remove extra logging
The test shouldn't be failing any more.
2016-06-08 13:52:43 -04:00
Nik Everett 5b94c4a25b Fix a race condition in reindex's rethrottle
If you rethrottled the request while is was performing a scroll
request then it wouldn't properly pick up the rethrottle for that
batch. This was causing test failure and might cause issues for
users. The work around is simple though: just issue the rethrottle
again with a slightly faster throttle than the first time.

Caught by:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-os-compatibility/os=centos/525/console
2016-06-08 13:52:43 -04:00
Nik Everett 4b21157906 Remove setRefresh
It has been replaced with `setRefreshPolicy` which has support for
waiting until refresh with `setRefreshPolicy(WAIT_FOR)`.

Related to #1063
2016-06-08 13:50:59 -04:00
Jack Conradson be168f528b Merge pull request #18771 from jdconrad/methods
Add Method to Get New MethodWriters
2016-06-07 11:47:29 -07:00
Jack Conradson f9b45107c4 Added method to allow creation of new methods on-the-fly. 2016-06-07 11:24:33 -07:00
Robert Muir 90f2aab9a9 Merge pull request #18748 from rmuir/painless_method_references
painless: method reference support
2016-06-07 14:22:49 -04:00
Robert Muir f23e8c3722 more cleanups 2016-06-07 12:42:19 -04:00
Robert Muir 89224dbc3c switch FunctionRef over to methodhandles 2016-06-07 11:56:44 -04:00
Martijn van Groningen f611f1c99e ingest: Move processors from core to ingest-common module.
Folded grok processor into ingest-common module.

The rest tests have been moved to ingest-common module as well, because these tests don't run in the rest-api-spec module but in the distribution:integ-test-zip module
and adding a test plugin there felt just wrong to me. I think this is ok. I left a tiny ingest rest test behind in that tests with an empty pipeline.

Removed messy tests, these tests were already covered in the rest tests

Added ingest test plugin in test infra so that each module testing integration with ingest doesn't need write its own plugin

Moved reindex ingest tests to qa module

Closes #18490
2016-06-07 17:32:52 +02:00
Robert Muir 32965f043a convert FunctionRef/Def usage to methodhandles. 2016-06-07 10:23:49 -04:00
Robert Muir 3108c399f0 use painless types in FunctionRef 2016-06-07 09:49:54 -04:00
Robert Muir 39550354a7 compute functional interface stuff in Definition 2016-06-07 09:41:03 -04:00
Uwe Schindler 3238868cc4 Make checkstyle happy (add Lookup import, line length) 2016-06-07 13:06:23 +02:00
Uwe Schindler cac111ff73 Don't hide LambdaConversionException and behave like real javac compiled code when a conversion fails. This works anyways, because fallback is allowed to throw any Throwable 2016-06-07 13:00:36 +02:00
Uwe Schindler 6913be2ec2 Pass through the lookup given by invokedynamic to the LambdaMetaFactory. Without it real lambdas won't work, as their implementations are private to script class 2016-06-07 12:32:59 +02:00
Robert Muir 950b093fc7 Merge branch 'master' into painless_method_references 2016-06-06 22:39:28 -04:00
Jason Tedor da74323141 Register thread pool settings
This commit refactors the handling of thread pool settings so that the
individual settings can be registered rather than registering the top
level group. With this refactoring, individual plugins must now register
their own settings for custom thread pools that they need, but a
dedicated API is provided for this in the thread pool module. This
commit also renames the prefix on the thread pool settings from
"threadpool" to "thread_pool". This enables a hard break on the settings
so that:
 - some of the settings can be given more sensible names (e.g., the max
   number of threads in a scaling thread pool is now named "max" instead
   of "size")
 - change the soft limit on the number of threads in the bulk and
   indexing thread pools to a hard limit
 - the settings names for custom plugins for thread pools can be
   prefixed (e.g., "xpack.watcher.thread_pool.size")
 - remove dynamic thread pool settings

Relates #18674
2016-06-06 22:09:12 -04:00
Jack Conradson b3804c47f7 Cleaned up SEach node. 2016-06-06 18:21:08 -07:00
Robert Muir 231268c89d remove outdated TODO 2016-06-06 21:04:59 -04:00
Jack Conradson 6dace47c1f Reverted S-node design change. 2016-06-06 17:59:39 -07:00
Robert Muir 8db9a971e5 enable test 2016-06-06 20:30:42 -04:00
Robert Muir 2852e82ab9 add def iteration over arrays 2016-06-06 20:07:32 -04:00
Robert Muir d331f8768e implement dynamic case for iterables 2016-06-06 19:41:37 -04:00
Jack Conradson aeaf39f36d Added a for each stub node for a target type of def. 2016-06-06 16:07:12 -07:00
Jack Conradson a088d367f7 More comments. 2016-06-06 15:58:56 -07:00
Jack Conradson f897342296 Updated package info documentation. 2016-06-06 15:34:51 -07:00
Jack Conradson 9a78f6955b Added foreach for array types. 2016-06-06 15:25:09 -07:00
Robert Muir 23d7cdd043 ban slow methods in MethodHandle 2016-06-06 17:58:00 -04:00
Robert Muir b32f6abb26 remove unnecessary caching 2016-06-06 17:29:03 -04:00
Jack Conradson a4ffaa6e7a Working iterable foreach with tests. 2016-06-06 13:53:31 -07:00
Areek Zillur d96fe20e3a add named writable registry glue 2016-06-06 16:11:46 -04:00
Jack Conradson b2e648dfde Implementation of iterable. 2016-06-06 12:33:25 -07:00
Jack Conradson 260b0fd40f Stubbed foreach node. 2016-06-06 10:49:40 -07:00
Jack Conradson 0c9174decc Merge branch 'master' into each 2016-06-06 09:23:01 -07:00
Nik Everett d8056c8213 Add support for waiting until a refresh occurs
This adds support for setting the refresh request parameter to
`wait_for` in the `index`, `delete`, `update`, and `bulk` APIs. When
`refresh=wait_for` is set those APIs will not return until their
results have been made visible to search by a refresh.

Also it adds a `forced_refresh` field to the response of `index`,
`delete`, `update`, and to each item in a bulk response. This will
be true for requests with `?refresh` or `?refresh=true` and will be
true for some requests (see below) with `refresh=wait_for` but ought
to otherwise always be false.

`refresh=wait_for` is implemented as a list of
`Tuple<Translog.Location, Consumer<Boolean>>`s in the new `RefreshListeners`
class that is managed by `IndexShard`. The dynamic, index scoped
`index.max_refresh_listeners` setting controls a maximum number of
listeners allowed in any shard. If more than that many listeners
accumulate in the engine then a refresh will be forced, the thread that
adds the listener will be blocked until the refresh completes, and then the
listener will be called with a `forcedRefresh` flag so it knows that it was
the "straw that broke the camel's back". These listeners are only used by
`refresh=wait_for` and that flag manifests itself as `forced_refresh` being
`true` in the response.

About half of this change comes from piping async-ness down to the appropriate
layer in a way that is compatible with the ongoing with with sequence ids.

Closes #1063

You can look up the winding story of all the commits here:
https://github.com/elastic/elasticsearch/pull/17986

Here are the commit messages in case they are intersting to you:
commit 59a753b89109828d2b8f0de05cb104fc663cf95e
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Jun 6 10:18:23 2016 -0400

    Replace a method reference with implementing an interface

    Saves a single allocation and forces more commonality
    between the WriteResults.

commit 31f7861a85b457fb7378a6f27fa0a0c171538f68
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Jun 6 10:07:55 2016 -0400

    Revert "Replace static method that takes consumer with delegate class that takes an interface"

    This reverts commit 777e23a6592c75db0081a53458cc760f4db69507.

commit 777e23a6592c75db0081a53458cc760f4db69507
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Jun 6 09:29:35 2016 -0400

    Replace static method that takes consumer with delegate class that takes an interface

    Same number of allocations, much less code duplication.

commit 9b49a480ca9587a0a16ebe941662849f38289644
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Jun 6 08:25:38 2016 -0400

    Patch from boaz

commit c2bc36524fda119fd0514415127e8901d94409c8
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 14:46:27 2016 -0400

    Fix docs

    After updating to master we are actually testing them.

commit 03975ac056e44954eb0a371149d410dcf303e212
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 14:20:11 2016 -0400

    Cleanup after merge from master

commit 9c9a1deb002c5bebb2a997c89fa12b3d7978e02e
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 14:09:14 2016 -0400

    Breaking changes notes

commit 1c3e64ae06c07a85f7af80534fab88279adb30b4
Merge: 9e63ad6 f67e580
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 14:00:05 2016 -0400

    Merge branch 'master' into block_until_refresh2

commit 9e63ad6de52d0b28f0b6d7203721baf1ebf6f56b
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 13:21:27 2016 -0400

    Test for TransportWriteAction

commit 522ecb59d39b3c9e8df0d3b8df34b9e7aeaf0ce9
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 10:30:18 2016 -0400

    Document deprecation

commit 0cd67b947f58867e704a1f0e66928a6fb5a11f11
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 10:26:23 2016 -0400

    Deprecate setRefresh(boolean)

    Users should use `setRefresh(RefreshPolicy)` instead.

commit aeb1be3f2c501990b33fb1f8230d496035f498ef
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 10:12:27 2016 -0400

    Remove checkstyle suppression

    It is fixed

commit 00d09a9caa638b6f90f4896b5502dd98d8fad56e
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 10:08:28 2016 -0400

    Improve comment

commit 788164b898a6ee2878a273961230122b7386c3c9
Author: Nik Everett <nik9000@gmail.com>
Date:   Thu Jun 2 10:01:01 2016 -0400

    S/ReplicatedWriteResponse/WriteResponse/

    Now it lines up with WriteRequest.

commit b74cf3fe778352b140355afcaa08d3d4412d749d
Author: Nik Everett <nik9000@gmail.com>
Date:   Wed Jun 1 18:27:52 2016 -0400

    Preserve `?refresh` behavior

    `?refresh` means the same things as `?refresh=true`.

commit 30f972bdaeaaa0de6fe67746cdb8628aa86f5a8c
Author: Nik Everett <nik9000@gmail.com>
Date:   Wed Jun 1 17:39:05 2016 -0400

    Handle hanging documents

    If a document is added to the index during a refresh we weren't properly
    firing its refresh listener. This happened because the way we detect
    whether a refresh makes something visible or not is imperfect. It is
    ok because it always errs on the side of thinking that something isn't
    yet visible.

    So when a document arrives during a refresh the refresh listeners
    won't think it made it into a refresh when, often, it does. The way
    we work around this is by telling Elasticsearch that it ought to
    trigger a refresh if there are any pending refresh listeners even
    if there aren't pending documents to update. Lucene short circuits
    the refresh so it doesn't take that much effort, but the refresh
    listeners still get the signal that a refresh has come in and they
    still pick up the change and notify the listener.

    This means that the time that a listener can wait is actually slightly
    longer than the refresh interval.

commit d523b5702b60c7ba309fb0dcf3cd3a4798f11960
Author: Nik Everett <nik9000@gmail.com>
Date:   Wed Jun 1 14:34:01 2016 -0400

    Explain Integer.MAX_VALUE

commit 4ffb7c0e954343cc1c04b3d7be2ebad66d3a016b
Author: Nik Everett <nik9000@gmail.com>
Date:   Wed Jun 1 14:27:39 2016 -0400

    Fire all refresh listeners in a single thread

    Rather than queueing a runnable each.

commit 19606ec3bbe612095df45eba734c5b7eb2709c01
Author: Nik Everett <nik9000@gmail.com>
Date:   Wed Jun 1 14:09:52 2016 -0400

    Assert translog ordering

commit 6bb4e5c75e850f4a42518f06fbc955f7ec76d245
Author: Nik Everett <nik9000@gmail.com>
Date:   Wed Jun 1 13:17:44 2016 -0400

    Support null RefreshListeners in InternalEngine

    Just skip using it.

commit 74be1480d6e44af2b354ff9ea47c234d4870b6c2
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 18:02:03 2016 -0400

    Move funny ShardInfo hack for bulk into bulk

    This should make it easier to understand because it is closer to where it
    matters....

commit 2b771f8dabd488e056cfdc9989608d18264ddfb0
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 17:39:46 2016 -0400

    Pull listener out into an inner class with javadoc and stuff

commit 058481ad72019c0492b03a7a4ac32a48673697d3
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 17:33:42 2016 -0400

    Fix javadoc links

commit d2123b1cabf29bce8ff561d4a4c1c1d5b42bccad
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 17:28:09 2016 -0400

    Make more stuff final

commit 8453fc4f7850f6a02fb5971c17a942a3e3fd9f7b
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 17:26:48 2016 -0400

    Javadoc

commit fb16d2fc7016c1e8e1621d481e8781c7ef43326c
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 16:14:48 2016 -0400

    Rewrite refresh docs

commit 5797d1b1c4d233c0db918c0d08c21731ddccd05e
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 15:02:34 2016 -0400

    Fix forced_refresh flag

    It wasn't being set.

commit 43ce50a1de250a9e073a2ca6cbf55c1b4c74b11b
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 14:02:56 2016 -0400

    Delay translog sync and flush until after refresh

    The sync might have occurred for us during the refresh so we
    have less work to do. Maybe.

commit bb2739202e084703baf02cfa58f09517598cf14e
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 13:08:08 2016 -0400

    Remove duplication in WritePrimaryResult and WriteReplicaResult

commit 2f579f89b4867a880396f2e7fcffc508449ff2de
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 12:19:05 2016 -0400

    Clean up registration of RefreshListeners

commit 87ab6e60ca5ba945bf0fba84784b2bbe53506abf
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 31 11:28:30 2016 -0400

    Shorten lock time in RefreshListeners

    Also use null to represent no listeners rather than an empty list.
    This saves allocating a new ArrayList every refresh cycle on every
    index.

commit 0d49d9c5720dadfb67da3fa760397bf6d874601c
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 10:46:18 2016 -0400

    Flip relationship between RefreshListeners and Engine

    Now RefreshListeners comes to Engine from EngineConfig.

commit b2704b8a39382953f8f91a9743e894ee289f7514
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 09:37:58 2016 -0400

    Remove unused imports

    Maybe I added them?

commit 04343a22647f19304d9dc716b3fac9b183227f63
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 09:37:52 2016 -0400

    Javadoc

commit da1e765678890a02d61d8a29aa433274beb5e00c
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 09:26:35 2016 -0400

    Reply with non-null

    Also move the fsync and flush to before the refresh listener stuff.

commit 5d8eecd0d904b497844b4c81c46477bd6178ed3a
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 08:58:47 2016 -0400

    Remove funky synchronization in AsyncReplicaAction

commit 1ec71eea0f4e1228ae1497d982307be818ef4b65
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 08:01:14 2016 -0400

    s/LinkedTransferQueue/ArrayList/

commit 7da36a4ceed2ccf7955138c3b005237fa41efcb4
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 07:46:38 2016 -0400

    More cleanup for RefreshListeners

commit 957e9b77007c32ee75dde152c6622bab065d5993
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 24 07:34:13 2016 -0400

    /Consumer<Runnable>/Executor/

commit 4d8bf5d4a70dcc56150c8d8d14165cd23d308b3c
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 23 22:20:42 2016 -0400

    explain

commit 15d948a348089bb2937eec5ac4e96f3ec67dbe32
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 23 22:17:59 2016 -0400

    Better....

commit dc28951d02973fc03b4d51913b5f96de14b75607
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 23 21:09:20 2016 -0400

    Javadocs and compromises

commit 8eebaa89c0a1ee74982fbe0d56d1485ca2ae09db
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 23 20:52:49 2016 -0400

    Take boaz's changes to their logic conclusion and unbreak important stuff like bulk

commit 7056b96ea412f275005b93e3570bcff895859ed5
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 23 15:49:32 2016 -0400

    Patch from boaz

commit 87be7eaed09a274cc6a99d1a3da81d2d7bf9dd64
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 23 15:49:13 2016 -0400

    Revert "Move async parts of replica operation outside of the lock"

    This reverts commit 13807ad10b6f5ecd39f98c9f20874f9f352c5bc2.

commit 13807ad10b6f5ecd39f98c9f20874f9f352c5bc2
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri May 20 22:53:15 2016 -0400

    Move async parts of replica operation outside of the lock

commit b8cadcef565908b276484f7f5f988fd58b38d8b6
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri May 20 16:17:20 2016 -0400

    Docs

commit 91149e0580233bf79c2273b419fe9374ca746648
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri May 20 15:17:40 2016 -0400

    Finally!

commit 1ff50c2faf56665d221f00a18d9ac88745904bf5
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri May 20 15:01:53 2016 -0400

    Remove Translog#lastWriteLocation

    I wasn't being careful enough with locks so it wasn't right anyway.
    Instead this builds a synthetic Tranlog.Location when you call
    getWriteLocation with much more relaxed equality guarantees. Rather
    than being equal to the last Translog.Location returned it is
    simply guaranteed to be greater than the last translog returned
    and less than the next.

commit 55596ea68b5484490c3637fbad0d95564236478b
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri May 20 14:40:06 2016 -0400

    Remove listener from shardOperationOnPrimary

    Create instead asyncShardOperationOnPrimary which is called after
    all of the replica operations are started to handle any async
    operations.

commit 3322e26211bf681b37132274ee158ae330afc28b
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 17 17:20:02 2016 -0400

    Increase default maximum number of listeners to 1000

commit 88171a8322a424e624d48960fb4c98dd43e4d671
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 17 16:40:57 2016 -0400

    Rename test

commit 179c27c4f829f2c6ded65967652cf85adaf2ae52
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 17 16:35:27 2016 -0400

    Move refresh listeners into their own class

    They still live at the IndexShard level but they live on their
    own in RefreshListeners which interacts with IndexShard using a
    couple of callbacks and a registration method. This lets us test
    the listeners without standing up an entire IndexShard. We still
    test the listeners against an InternalEngine, because the interplay
    between InternalEngine, Translog, and RefreshListeners is complex
    and important to get right.

commit d8926d5fc1d24b4da8ccff7e0f0907b98c583c41
Author: Nik Everett <nik9000@gmail.com>
Date:   Tue May 17 11:02:38 2016 -0400

    Move refresh listeners into IndexShard

commit df91cde398eb720143a85a8c6fa19bdc3a74e07d
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 16 16:01:03 2016 -0400

    unused import

commit 066da45b08148b266e4173166662fc1b3f66ed53
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 16 15:54:11 2016 -0400

    Remove RefreshListener interface

    Just pass a Translog.Location and a Consumer<Boolean> when registering.

commit b971d6d3301c7522b2e7eb90d5d8dd96a77fa625
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 16 14:41:06 2016 -0400

    Docs for setForcedRefresh

commit 6c43be821eaf61141d3ec520f988aad3a96a3941
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon May 16 14:34:39 2016 -0400

    Rename refresh setter and getter

commit e61b7391f91263a4c4d6107bfbc2a828bbcc805c
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 22:48:09 2016 -0400

    Trigger listeners even when there is no refresh

    Each refresh gives us an opportunity to pick up any listeners we may
    have left behind.

commit 0c9b0477085c021f503db775640d25668e02f635
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 20:30:06 2016 -0400

    REST

commit 8250343240de7e63118c663a230a7a314807a754
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 19:34:22 2016 -0400

    Switch to estimated count

    We don't need a linear time count of the number of listeners - a volatile
    variable is good enough to guess. It probably undercounts more than it
    overcounts but it isn't a huge problem.

commit bd531167fe54f1bde6f6d4ddb0a8de5a7bcc18a2
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 18:21:02 2016 -0400

    Don't try and set forced refresh on bulk items without a response

    NullPointerExceptions are bad. If the entire request fails then the user
    has worse problems then "did these force a refresh".

commit bcfded11515af5e0b3c3e36f3c2f73f5cd26512e
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 18:14:20 2016 -0400

    Replace LinkedList and synchronized with LinkedTransferQueue

commit 8a80cc70a76375a7593745884cb987535b37ca80
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 17:38:24 2016 -0400

    Support for update

commit 1f36966742f851b7328015151ef6fc8f95299af2
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 15:46:06 2016 -0400

    Cleanup translog tests

commit 8d121bf35eb265b8a0aee9710afeb1b054a113d4
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 15:40:53 2016 -0400

    Cleanup listener implementation

    Much more testing too!

commit 2058f4a808762c4588309f21b13b677245832f2c
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 11:45:55 2016 -0400

    Pass back information about whether we refreshed

commit e445cb0cb91ebdbcfdbf566696edb2bf1c84a882
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 11:03:31 2016 -0400

    Javadoc

commit 611cbeeaeb458f4b428bfc43a1ee6652adf4baff
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 11:01:40 2016 -0400

    Move ReplicationResponse

    now it is in the same package as its request

commit 9919758b644fd73895fb88cd6a4909a8387eb2e2
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 11:00:14 2016 -0400

    Oh boy that wasn't working

commit 247cb483c4459dea8e95e0e3bd2e4bf8d452c598
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 10:29:37 2016 -0400

    Basic block_until_refresh exposed to java client

    and basic "is it plugged in" style tests.

commit 46c855c9971cb2b748206d2afa6a2d88724be3ba
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 10:11:10 2016 -0400

    Move test to own class

commit a5ffd892d0a352ae7e9757f2640fc2a1fa656bf2
Author: Nik Everett <nik9000@gmail.com>
Date:   Mon Apr 25 07:44:25 2016 -0400

    WIP

commit 213bebb6ece11b85d17e44af9a54fc2e5e332d39
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 21:35:52 2016 -0400

    Add refresh listeners

commit a2bc7f30e6d4857a1224ef5a89909b36c8f33731
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 21:11:55 2016 -0400

    Return last written location from refresh

commit 85033a87551da89f36a23d4dfd5016db218e08ee
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 20:28:21 2016 -0400

    Never reply to replica actions while you have the operation lock

    This last thing was causing periodic test failures because we were
    replying while we had the operation lock. Now, we probably could get
    away with that in most cases but the tests don't like it and it isn't
    a good idea to do network io while you have a lock anyway. So this
    prevents it.

commit 1f25cf35e796835b3827b8a4110e09e5de61784c
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 19:56:18 2016 -0400

    Cleanup

commit 52c5f7c3f04710901f503334239a611c0e21c85a
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 19:33:00 2016 -0400

    Add a listener to shard operations

commit 5b142dc331214c8eef90587144f4b3f959f9eced
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 18:03:52 2016 -0400

    Cleanup

commit 3d22b2d7ceb473db339259452a7c4f117ce86069
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 17:59:55 2016 -0400

    Push the listener into shardOperationOnPrimary

commit 34b378943b8185451acf6350f661c0ad33b5836d
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 17:48:47 2016 -0400

    Doc

commit b42b8da968d42cc7414020c7b199606a5dcce50a
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 17:45:40 2016 -0400

    Don't finish early if the primary finishes early

    We use a "fake" pending shard that we resolve when the replicas have
    all started.

commit 0fc045b56e1e02a48c30383ac50a281d5af7e0b6
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 17:30:06 2016 -0400

    Make performOnPrimary asyncS

    Instead of returning Tuple<Response, ReplicaRequest> it returns
    ReplicaRequest and takes a ActionListener<Response> as an argument.
    We call the listener immediately to preserve backwards compatibility
    for now.

commit 80119b9a26ede96a865af45904c3ac69d5b19b59
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 16:51:53 2016 -0400

    Factor out common code in shardOperationOnPrimary

commit 0642083676702618f900fa842c08802a04c1a53e
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 16:32:29 2016 -0400

    Factor out common code from shardOperationOnReplica

commit 8bdc415fedaaa9f2d0c555590a13ec4699a7c3f7
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 16:23:28 2016 -0400

    Create ReplicatedMutationRequest

    Superclass for index, delete, and bulkShard requests.

commit 0f8fa846a2822c4293df32fed18c9b99660b39ff
Author: Nik Everett <nik9000@gmail.com>
Date:   Fri Apr 22 16:10:30 2016 -0400

    Create TransportReplicatedMutationAction

    It is the superclass of replication actions that mutate data: index, delete,
    and shardBulk. shardFlush and shardRefresh are replication actions but they
    do not extend TransportReplicatedMutationAction because they don't change
    the data, only shuffle it around.
2016-06-06 11:37:53 -04:00
Robert Muir 881e4058ee don't do a no-op filter, that was just for testing 2016-06-06 09:39:39 -04:00
Robert Muir 6dbf7ab1ea initial messy impl of painless method references 2016-06-06 09:27:19 -04:00
Tanguy Leroux a1172d816c Implement ctx.op = "delete" on _update_by_query and _reindex
closes #18043
2016-06-06 11:11:29 +02:00
Uwe Schindler c0a3a20071 painless: Add support for the new Java 9 MethodHandles#arrayLength() factory (see https://bugs.openjdk.java.net/browse/JDK-8156915) 2016-06-03 23:35:11 +02:00
Robert Muir ad118eb1e5 Merge pull request #18711 from rmuir/painless_compile_exceptions
Improve painless compile-time exceptions
2016-06-02 20:43:01 -04:00
Jack Conradson 9a0d0d7cff Fixed a grammar mistake in a comment. 2016-06-02 15:30:11 -07:00
Robert Muir b5393ce55e Improve painless compile-time exceptions 2016-06-02 18:23:47 -04:00
Nik Everett f82ab787a5 Add test rethrottle test case for delete-by-query
and remove some type parameters that we don't need that were getting
in the way.
2016-06-02 15:04:18 -04:00
Christoph Büscher 359f45988f Handle empty query bodies at parse time and remove EmptyQueryBuilder
Currently we support empty query clauses like the filter in

"constant_score" : {  "filter" : { } }

How these clauses are handled depends on the surrounding query.
They later are either ignored, converted to match all or no documents or
passed up further in the query hierarchy. During parsing these claues are
currently represented as EmptyQueryBuilders. When not handled anywhere else,
these special cases need to be checked for on the shard when building the
lucene query.

This is trappy, so this PR changes the parsing of compound queries. Instead
of returning QueryBuilder, the core query parsing method
QueryShardContext#parseInnerQueryBuilder() now return an Optional which can
be empty in the case of empty query clauses. This has the advantage of forcing
callers to deal with this sooner or later. When encountering empty Optionals,
compound query builders now have the choice to ignore them, pass them on or
rewrite to a different query, depending on context.
2016-06-02 11:25:56 +02:00
Nicholas Knize 2d57bbdacd remove unrecognized javadoc tag from matrix aggregation module 2016-06-01 17:30:20 -05:00
Nicholas Knize 54575e55ca set ValuesSourceConfig fields as private 2016-06-01 16:39:42 -05:00
Nicholas Knize 90b8f5d0d8 Adding MultiValuesSource support classes and documentation to matrix stats agg module 2016-06-01 16:39:42 -05:00
Nicholas Knize f449666c59 New Matrix Stats Aggregation module
This commit adds a new aggs-matrix-stats module. The module presents a new class of aggregations called Matrix Aggregations. Matrix aggregations work on multiple fields and produce a matrix as output. The first matrix aggregation provided by the module is matrix_stats aggregation. This aggregation computes the following statistics over a set of fields:

* Covariance
* Correlation

For completeness (and interpretation purposes) the following per-field statistics are also provided:

* sample count
* population mean
* population variance
* population skewness
* population kurtosis
2016-06-01 16:39:42 -05:00
Nik Everett 1b66d4a97f Add more logging to reindex rethrottle
The tests are failing in CI and we can't track down the cause. This should
help!
2016-06-01 12:54:49 -04:00
Adrien Grand d182e171a4 Upgrade to Lucene 6.0.1. 2016-06-01 10:31:10 +02:00
Adrien Grand a78c7d9911 AggregatorBuilder and PipelineAggregatorBuilder do not need generics. #18368
Similar reasoning as #18133 but for the aggs API. One important change is that
I moved the base PipelineAggregatorBuilder class to the o.e.s.aggregations
package instead of o.e.s.aggregations.pipeline so that the create method does
not need to be public.
2016-06-01 10:19:30 +02:00
Jack Conradson b6e1429760 Merge branch 'master' into each 2016-05-31 09:36:04 -07:00
Robert Muir 2d1eb89aef improve date api for expressions/painless fields 2016-05-31 09:32:33 -04:00
Jack Conradson d1f4e2ef04 Merge branch 'master' into each 2016-05-30 16:22:14 -07:00
Simon Willnauer f74a78940c Improve phrase suggest test speed (#18633)
There is no reason to read the entire marvel hero file to test the features,
it might take several seconds to do so which is unnecessary.
This commit also splits SearchSuggestTests into core and modules/mustache 
also add @Nighlty to forbidden API to make sure we don't use it since they won't run in CI these days.
2016-05-30 17:22:03 +02:00
Robert Muir 7c77c62dbc Merge pull request #18621 from rmuir/java.time
add java.time packages to painless whitelist
2016-05-28 06:51:09 -04:00
Jack Conradson f5be0982d9 Added initial infrastructure to allow for each to be made. 2016-05-27 17:41:31 -07:00
Jack Conradson 2d8030f09f Added foreach in grammar. 2016-05-27 13:50:38 -07:00
Robert Muir aa53285448 add java.time packages to painless whitelist 2016-05-27 15:07:27 -04:00
Martijn van Groningen fbd3f8df2b percolator: Don't bail query term extraction when a must clause contains an unsupported query, if there is at least one other must clause in a boolean query that can be extracted. 2016-05-27 19:57:50 +02:00
Jason Tedor 41710f1028 Upgrade joda-time to 2.9.4
This commit upgrades joda-time to version 2.9.4 to integrate a bug fix
there into Elasticsearch.

Relates #18609
2016-05-27 08:51:19 -04:00
Robert Muir 3f06d9f3b8 Merge pull request #18600 from rmuir/new_script_exception
replace ScriptException with a better one
2016-05-26 17:51:34 -04:00
Jack Conradson cde83e363a Add an update script equivalent test. 2016-05-26 13:16:22 -07:00
Robert Muir f037807117 replace ScriptException with a better one 2016-05-26 11:43:29 -04:00
Jack Conradson f1ccc88f75 Merge pull request #18578 from jdconrad/ref
Add Function Reference Stub to Painless
2016-05-26 08:32:10 -07:00
Tal Levy edfbdf2748 add ability to specify multiple grok patterns (#18074)
- now you can specify a list of grok patterns to match your field with
and the first one to successfully match wins.
- only non-null captures will be inserted into your matched document.

Fixes #17903.
2016-05-25 12:20:39 -07:00
Jack Conradson eefcafd831 Add method reference stub to Painless. 2016-05-25 10:34:21 -07:00
Ryan Ernst a916830ab4 Merge pull request #18560 from rjernst/score_tests
Tests: Make painless score tests unit tests
2016-05-25 08:44:13 -07:00
Tanguy Leroux bdee8c2632 Disable XContent auto closing of object and arrays 2016-05-25 16:46:09 +02:00
Tanguy Leroux bf41ac8bf2 Makes DeleteByQueryRequest implements CompositeIndicesRequest 2016-05-25 09:36:07 +02:00
Ryan Ernst 82ae65bbb5 Tests: Make score tests unit tests
This change simply removes the need for single node tests from score
tests.
2016-05-24 19:53:50 -07:00
Nik Everett 5e81270509 Add retry test case for delete-by-query
Tests that we retry failed searches, scrolls, and bulks.
2016-05-24 12:02:55 -04:00
Martijn van Groningen 27cc2fe4dc Moved the percolator from core to its own module
Significant changes:
* AbstractQueryTestCase has moved to the test framework module, in order for query builder tests in modules and plugins
* Added support to AbstractQueryTestCase to register plugins
* Lift the restriction that only one percolator could be added per index. This validation existed in MapperService, but because the percolator moved to a module it could no longer exist there. Instead of bringing it back it was removed. This validation existed since the percolator cache only supported one percolator query per document, since the percolator cache has been removed this restriction could removed as well.
* While moving percolator tests to the new module, also removed a couple of tests for the deprecated percolate and mpercolate api. These APIs are now sugar  APIs for bwc and rediect to the searvh and msearvh APIs. Some tests were still testing as if percolate and mpercolate API did the percolation, but this no longer the case and these tests could be removed.
2016-05-24 11:01:57 +02:00
Robert Muir a5d0ccf50b painless: add trap tests 2016-05-23 23:36:40 -04:00
Robert Muir 0a9c036c3e Merge branch 'master' into painless_more_whitelisting 2016-05-23 20:35:44 -04:00
Ryan Ernst f6074d383b Merge pull request #18532 from rjernst/less_assert_busy
Tests: Remove unnecessary Callable variant of assertBusy
2016-05-23 17:11:54 -07:00
Jack Conradson 1424f09c02 Remove Painless grammar ambiguities.
Closes #18531
2016-05-23 16:39:52 -07:00
Robert Muir a56a9cb818 improve painless whitelist coverage of java api 2016-05-23 19:35:28 -04:00
Ryan Ernst c7b45b2cc7 Tests: Remove unnecessary Callable variant of assertBusy
The assertBusy method currently has both a Runnable and Callable
version. This has caused confusion with type inference and lambdas
sometimes, in particular with java 9. This change removes the callable
version as nothing was actually using it.
2016-05-23 16:17:43 -07:00
Nik Everett 62ac719a94 Rerwite RetryTests to very carefully block the executors
This reproduces every time. No more randomness! Hurray!
2016-05-23 14:38:40 -04:00
Nik Everett b7817a6306 [reindex] Retry the retry test if it didn't cause retries
The retry test has failed a couple of times in CI because it wasn't able
to cause any retries. Putting it in a bash `while` loop shows that it
eventually does fail that way. The seed "4F6477A9C999CA20" seems especially
good at failing to get retries. It doesn't fail all the time, but more
than most.

This adds a retry to each test case, retrying a maximum of 10 times or
until it causes the retries. I've seen it fail to get retries 7 times
in a row but not go beyond that. Retrying doesn't seem to really hurt
the test runtime all that much. Most of the time is in the startup
cost.

Failing CI build that triggered this:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+periodic/852/console
2016-05-23 14:38:40 -04:00
Adrien Grand 459916f5dd Remove custom Base64 implementation. #18413
This replaces o.e.common.Base64 with java.util.Base64.
2016-05-23 11:32:42 +02:00
Tanguy Leroux b0b503035a Update reindex cancel tests 2016-05-23 10:31:14 +02:00
Ryan Ernst 37d36f2f4c Merge branch 'master' into java9 2016-05-21 14:19:58 -07:00
Ryan Ernst 1d40c4bbc1 Make java9 work again
This change makes ES compile with java9 again, build 118.
* There are a handful of changes due to failure to determine types during compile.
* The attachment plugins which use tika needed to have tika upgraded in order to pickup fixes there for java 9.
* azure discovery and s3 repository indirectly depend on jaxb, which is no longer in the default modules. They now add a jaxb dependency externally, and make JarHell allow for this package.
2016-05-21 09:41:51 -07:00
Lee Hinman fdfd2a2f18 Remove ScriptMode class in favor of boolean true/false
This removes the ScriptMode class entirely, which was an enum with two
options (ON and OFF) which essentially boiled down to true and false.
Now the boolean values are used instead.
2016-05-20 15:01:30 -06:00
Robert Muir a2ff002e2b Merge pull request #18463 from rmuir/whitelist_cleanup
painless: steps at definition cleanup
2016-05-20 14:26:41 -04:00
Jack Conradson b156438957 Proxy box method to use valueOf. 2016-05-20 11:22:18 -07:00
Jack Conradson 0e24ed49e6 Use the build-in valueOf method instead of the custom one. 2016-05-20 11:13:43 -07:00
Jack Conradson d836194095 Fixed tests and added a comment to the box method. 2016-05-20 11:04:09 -07:00
Jack Conradson 2b793c1e06 Fix boxing. 2016-05-20 10:38:06 -07:00
Robert Muir c0eb813e10 Merge branch 'clean' into whitelist_cleanup 2016-05-20 10:59:13 -04:00
Robert Muir ce9b36590a simplify tree node use of compiler settings 2016-05-20 09:36:09 -04:00
Martijn van Groningen 80fee8666f percolator: Removed percolator cache
Before 5.0 for it was required that the percolator queries were cached in jvm heap as Lucene queries for two reasons:
1) Performance. The percolator evaluated all percolator queries all the time. There was no pre-selecting queries that are likely to match like we have today.
2) Updates made to percolator queries were visible in realtime, Today these changes are visible in near realtime. So updating no longer requires the percolator to have the queries in jvm heap.

So having the percolator queries in jvm heap via the percolator cache is now less attractive. Especially when there are many percolator queries then these queries can consume many GBs of jvm heap.
Removing the percolator cache does make the percolate query slower compared to how the execution time in 5.0.0-alpha1 and alpha2, but it is still faster compared to 2.x and before.
2016-05-20 14:52:16 +02:00
Robert Muir b3c71c1e3a improve style for these constants 2016-05-20 08:42:29 -04:00
Uwe Schindler 7bc91c00d9 More elegant way to force class initialization 2016-05-20 14:12:15 +02:00
Uwe Schindler 6205f79554 Remove Definitions's copy-ctor; fix RuntimeClass to be unmodifiable
Please note: The maps inside the pirvate singleton instance of Defininition are no longer unmodifiable, but nothing from the outside can modify it! All private :-)
2016-05-20 14:03:46 +02:00
Uwe Schindler 4ffa92c7c1 Remove the Definition instance passed around everywhere 2016-05-20 12:35:08 +02:00
Uwe Schindler 91f4bba042 Make Definition's public API completely static.
TODO: Remove Definition arguments everywhere and hide INSTANCE field!
2016-05-20 11:22:03 +02:00
Uwe Schindler f5fc60ac67 Add utility type constant 2016-05-20 11:05:40 +02:00
Jack Conradson 4e454439bb Fixed bugs in comparison with Def. Fixed may tests.n 2016-05-20 02:04:38 -07:00
Uwe Schindler 87ab39613d more constants 2016-05-20 10:56:50 +02:00
Uwe Schindler 82aed08301 Restore constants as static final (first step) 2016-05-20 10:48:41 +02:00
Jack Conradson 64e2ef5807 Removed transforms. No user-facing boxing. 2016-05-19 23:38:14 -07:00
Jack Conradson 3381f50e07 Merge branch 'master' into clean 2016-05-19 18:52:10 -07:00
Jack Conradson 3962553551 Removed user-facing boxing. 2016-05-19 18:51:55 -07:00
Uwe Schindler cfe555d216 move unused @SuppressWarnings one level up 2016-05-20 00:46:05 +02:00
Nik Everett 223cb6a7f0 [reindex] Mark test awaits fix because it is unstable
Fix coming.
2016-05-19 18:03:36 -04:00
Robert Muir afa9ebad28 dce 2016-05-19 16:03:11 -04:00
Robert Muir 0ee726ed0d nuke eclipse warnings 2016-05-19 15:35:26 -04:00
Robert Muir a569130368 fix typo 2016-05-19 15:28:59 -04:00
Robert Muir b9c7dbcfbd nuke overflow detection as we cannot guarantee for def. simplify life :) 2016-05-19 15:25:31 -04:00
Tanguy Leroux a01ecb20ea Port Delete By Query to Reindex infrastructure
closes #16883
2016-05-19 16:07:50 +02:00
Robert Muir 915ab16176 First steps at whitelist cleanup 2016-05-19 09:29:48 -04:00
Ali Beyad fc6df23fea Rename AggregatorBuilder and all of its subclasses to
AggregationBuilder, in keeping consistent with the Java APIs.

Closes #18377
Closes #18367
2016-05-19 09:25:29 -04:00
Robert Muir 1022123f59 Merge pull request #18400 from uschindler/painless_java9StringConcats
Use Java 9 Indy String Concats, if available
2016-05-18 08:00:20 -04:00
Uwe Schindler ea9026e6d8 painless: fix checkstyle 2016-05-18 00:31:12 +02:00
Uwe Schindler b418d01117 painless: Improve string concat tests; add state machine check after MethodWriter 2016-05-18 00:26:47 +02:00
Jack Conradson 3042a77d2b Merge pull request #18425 from jdconrad/cast
Made def variable casting consistent with invokedynamic rules
2016-05-17 13:29:49 -07:00
Jack Conradson f055ad641a Made def variable casting consistent with invokedynamic rules. 2016-05-17 13:16:49 -07:00
polyfractal 978c1e3e36 [TEST] Add missing sort processor to test
Also fixes the naming of the sort REST test
to follow the numbering convention
2016-05-17 14:52:17 -04:00
Nik Everett cfb06954ba [reindex] Add assertBusy to test
It has timing issues.
2016-05-17 14:05:35 -04:00
Nik Everett fe4823eae0 Reindex should retry on search failures
This uses the same backoff policy we use for bulk and just retries until
the request isn't rejected.

Instead of `{"retries": 12}` in the response to count retries this now
looks like `{"retries": {"bulk": 12", "search": 1}`.

Closes #18059
2016-05-17 13:58:45 -04:00
Uwe Schindler b27b0483d5 Merge branch 'master' into painless_java9StringConcats
# Conflicts:
#	modules/lang-painless/src/test/java/org/elasticsearch/painless/StringTests.java
2016-05-17 18:47:33 +02:00
Uwe Schindler 5622e9c9d3 painless: use Java 9 Indy String Concats, if available 2016-05-17 17:44:46 +02:00
Robert Muir fa5f099486 make semicolons only optional at the end 2016-05-17 10:55:40 -04:00
Robert Muir 92339c4763 painless: add method overloading based on arity
Closes #18385

Squashed commit of the following:

commit b2819df4d392d69b86e5c96d358eb03424e67e02
Author: Robert Muir <rmuir@apache.org>
Date:   Tue May 17 09:15:47 2016 -0400

    add note about tuple

commit 85fcac6a0d0674da24535121eab23e2c407d683f
Author: Robert Muir <rmuir@apache.org>
Date:   Mon May 16 23:39:25 2016 -0400

    painless: add method overloading based on arity
2016-05-17 09:17:43 -04:00
Uwe Schindler 75f27df376 painless: Refactor WriterUtils to extends ASM GeneratorAdapter as "MethodWriter" for easy usage (and more coming later) 2016-05-17 02:25:05 +02:00
Jack Conradson 3fd3d367ef Moved ADefLink to be a marker interface instead. 2016-05-16 15:48:29 -07:00
Jack Conradson 3f23186712 Merge branch 'master' into types 2016-05-16 14:22:59 -07:00
Jack Conradson 95a1b5b2a3 Fixed a casting bug in Painless. 2016-05-16 14:18:05 -07:00
Robert Muir e69305ae21 Merge pull request #18372 from rmuir/painless_whitelist_expansion
painless whitelist expansion
2016-05-16 17:08:49 -04:00
Robert Muir 8d4c1befe5 Merge pull request #18364 from rmuir/nukeRunAsFloat
Remove LeafSearchScript.runAsFloat(): Nothing calls it.
2016-05-16 17:08:25 -04:00
Jack Conradson 1d394868d4 Merge branch 'master' into types 2016-05-16 12:43:24 -07:00
Jack Conradson 43344eee82 Add more def tests including checking bytecode for specific
optimizations.
2016-05-16 12:42:25 -07:00
Jack Conradson f664fa5f33 Merge pull request #18350 from uschindler/painless_isAssignableFrom
Use isAssignableFrom instead of relying on ClassCastException
2016-05-16 12:41:09 -07:00
Jack Conradson a2c2628a78 Merge pull request #18359 from uschindler/painless_removeStoreBoxing
Remove boxing when loading and storing values in "def" fields/arrays, remove boxing onsimple method calls of "def" methods
2016-05-16 12:11:37 -07:00
Uwe Schindler d6cbbdeb98 painless: Some reformatting in EChain to make it clear if "this" or inner "expression" is affected 2016-05-16 21:04:03 +02:00
Uwe Schindler b05ac87a5d painless: Add Jack's test 2016-05-16 20:37:51 +02:00
Uwe Schindler 3a5ef68ad0 painless: Fix issue with dup and cast 2016-05-16 20:31:27 +02:00
Robert Muir 6ac2877c32 Merge pull request #18371 from uschindler/painless_debuggerCleanup
painless: Small cleanup of Debugger class to use StringWriter
2016-05-16 09:58:50 -04:00
Robert Muir 2fc7ca46e1 expand whitelist for basic types 2016-05-16 09:56:29 -04:00
Uwe Schindler 07e4f416e6 painless: Small cleanup of Debugger class to use StringWriter 2016-05-16 15:52:14 +02:00
Nik Everett f569576c5b Switch default batch size for reindex to 1000 2016-05-16 08:19:29 -04:00
Robert Muir 8edf213492 Remove LeafSearchScript.runAsFloat(): Nothing calls it. 2016-05-15 22:59:28 -04:00
Uwe Schindler 65aca4f71c painless: make compound statement like a[1]++ work with def. There was also a bug with the size of LDefArray: fixed 2016-05-15 23:00:32 +02:00
Uwe Schindler d221cd14d2 painless: Also remove boxing for reads and method calls 2016-05-15 19:49:25 +02:00
Uwe Schindler 604bcd9320 painless: Make field stores not box; use GeneratorAdapter.invokeDynmaic for consistency with other method calls 2016-05-15 18:31:53 +02:00
Uwe Schindler 8195ef9e7a painless: Transform more try-catch-ClassCastException to isAssignableFrom 2016-05-15 01:06:29 +02:00
Uwe Schindler 7211017813 painless: Some cleanups 2016-05-14 17:23:29 +02:00
Uwe Schindler c78bf8c867 painless: Use isAssignableFrom instead of relying on ClassCastException 2016-05-14 16:04:18 +02:00
Uwe Schindler c8bd521ed3 painless: restore accidentally removed test 2016-05-14 12:00:30 +02:00
Uwe Schindler ba9dae820a Add test for wrong array index 2016-05-13 23:22:41 +02:00
Uwe Schindler 1a76329d89 Take return type from "after" field. 2016-05-13 23:17:22 +02:00
Uwe Schindler 1b10a14e8d painless: build descriptor of array and field load/store in code; fix array index to adapt type not DEF 2016-05-13 23:04:04 +02:00
Robert Muir 2028691e66 painless: improve exception stacktraces
closes #18319
2016-05-13 15:40:45 -04:00
Uwe Schindler 9731d5d314 painless: Rename the dynamic call site factory to DefBootstrap and make the inner class very short (PIC = Polymorphic Inline Cache) 2016-05-13 19:58:05 +02:00
Lee Hinman 9bcdafedda Allow only a single extension for a scripting engine
Previously multiple extensions could be provided, however, this can lead
to confusion with on-disk scripts (ie, "foo.js" and "foo.javascript")
having different content. Only a single extension is now supported.

The only language currently supporting multiple extensions was the
Javascript engine ("js" and "javascript"). It now only supports the
`.js` extension.

Relates to #10598
2016-05-13 09:54:31 -06:00
Lee Hinman d5b75491dc Merge remote-tracking branch 'dakrone/remove-script-sandbox' 2016-05-13 09:50:39 -06:00
Nik Everett 0a300320cd [test] Set logging to 11 on reindex test
It has failures we can't explain and we need logs to be able to do
anything useful with the failures:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+g1gc/359/consoleFull
2016-05-13 11:48:01 -04:00
Lee Hinman efff3918d8 Remove support for mulitple languages per scripting engine 2016-05-13 09:24:31 -06:00
Lee Hinman a4060f7436 Remove vestiges of script engine sandboxing
This removes all the mentions of the sandbox from the script engine
services and permissions model. This means that the following settings
are no longer supported:

```yaml
script.inline: sandbox
script.stored: sandbox
```

Instead, only a `true` or `false` value can be specified.

Since this would otherwise break the default-allow parameter for
languages like expressions, painless, and mustache, all script engines
have been updated to have individual settings, for instance:

```yaml
script.engine.groovy.inline: true
```

Would enable all inline scripts for groovy. (they can still be
overridden on a per-operation basis).

Expressions, Painless, and Mustache all default to `true` for inline,
file, and stored scripts to preserve the old scripting behavior.

Resolves #17114
2016-05-13 09:24:31 -06:00
Adrien Grand 638da06c1d Add back support for `ip` range aggregations. #17859
This commit adds support for range aggregations on `ip` fields. However it will
only work on 5.x indices.

Closes #17700
2016-05-13 17:22:01 +02:00
Robert Muir 520697eb14 Merge pull request #18323 from uschindler/dynamiccallsite_cleanup
painless: cleanup of DynamicCallSite
2016-05-13 11:19:27 -04:00
Uwe Schindler a1d0fb5008 painless: cleanup of DynamicCallSite to have all implementation encapsulated in inner class, outer class only has bootstrap method 2016-05-13 15:50:55 +02:00
Daniel Mitterdorfer ddbfda2c68 Exclude specific transport actions from request size limit check
We add support to explicitly exclude specific transport actions
from the request size limit check.

We also exclude the following request types currently:

*MasterPingRequest
* PingRequest
2016-05-13 14:21:24 +02:00
Jack Conradson 2e25837deb Made line number available in painless nodes for debugging. 2016-05-12 10:32:06 -07:00
Jack Conradson 28164ccf2e Decouple the ANTLR AST from Painless.
Closes #18286
2016-05-12 00:46:48 -07:00
Robert Muir 113e239e0f Merge pull request #18288 from rmuir/painless_debug
painless: add debugging tool to tests
2016-05-12 01:16:32 -04:00
Jack Conradson 8871fddba7 Merge pull request #18282 from jdconrad/longs
Long priority over Float
2016-05-11 21:08:28 -07:00
Robert Muir 930a215b5c add debugger 2016-05-11 23:56:19 -04:00
Robert Muir 25dd64250b painless: remove input, support params instead 2016-05-11 21:32:10 -04:00
Robert Muir c5532d3df0 add a rest test for this that seems to work, fix the documentation. thanks @s1monw 2016-05-11 16:07:08 -04:00
Robert Muir 6b4e47bf96 this makes aggregations per-document _value fast (bypass hash put, hash get, etc) for painless.
but i have no clue how to test it, it seems this feature never worked via REST?

Should we drop the feature instead?
2016-05-11 15:39:00 -04:00
Jack Conradson 20af482dd7 Removed extra branches. Only check for long first where we already know the
second type.
2016-05-11 10:22:40 -07:00
Jack Conradson 2764b47d22 Give long priority over float in Def math calcs. Promotion of types
still works correclty.
2016-05-11 09:32:30 -07:00
Robert Muir d790c76636 painless: _score as double, not float 2016-05-11 11:19:18 -04:00
Tanguy Leroux 6d288dec11 Clean up tests in Reindex module 2016-05-11 09:54:52 +02:00
Robert Muir 7689a1af28 Add 'ctx' keyword to painless. 2016-05-11 02:31:19 -04:00
Jack Conradson d8b373cac6 Fixed an issue related to bracket shortcuts. 2016-05-10 23:11:13 -07:00
Robert Muir 632bb4c720 add reserved word tests for 'doc', too 2016-05-11 00:17:19 -04:00
Robert Muir 389ef462c4 special access to doc 2016-05-10 23:43:57 -04:00
Robert Muir abb015bcb9 add tests for reserved words. ban stores against _score 2016-05-10 22:55:06 -04:00
Robert Muir a63285aaf9 painless: retrieve _score directly from Scorer 2016-05-10 20:53:08 -04:00
Robert Muir 1d80542199 Merge pull request #18247 from rmuir/needsScore
painless: implement needsScore() correctly.
2016-05-10 19:21:09 -04:00
Robert Muir ddc2c1f682 Merge pull request #18234 from rmuir/more_indy_type_data
painless: use better typing for dynamic method calls
2016-05-10 19:17:13 -04:00
Robert Muir 1c2d3b10ef some cleanups suggested by @uschindler and break some long lines 2016-05-10 12:31:02 -04:00
Robert Muir eb1b2cf111 add test and cleanup 2016-05-10 11:34:33 -04:00
Robert Muir 7a47d33a9b Merge branch 'master' into needsScore 2016-05-10 11:23:50 -04:00
Robert Muir 546aed8390 hack 2016-05-10 11:23:24 -04:00
Uwe Schindler 243c9e77fa painless: Add synthetic length property as alias to Lists, so they can really be used like arrays 2016-05-10 15:01:49 +02:00
Uwe Schindler 2e613f49b3 painless: Array load/store and length with invokedynamic 2016-05-10 12:41:27 +02:00
Robert Muir 3aeb66885b painless: use better typing for dynamic method calls 2016-05-10 05:44:16 -04:00
Robert Muir ba2fe156e8 Switch over dynamic method calls, loads and stores to invokedynamic.
Remove performance hack for accessing a document's fields, its not needed.
Add support for accessing is-getter methods like List.isEmpty() as .empty

Closes #18201
2016-05-09 21:44:32 -04:00
Tanguy Leroux 8c52e8814b Remove ReindexResponse in favor of BulkIndexByScrollResponse 2016-05-09 17:03:16 +02:00
Chris Earle 5be79ed02c Add Failure Details to every NodesResponse
Most of the current implementations of BaseNodesResponse (plural Nodes) ignore FailedNodeExceptions.

- This adds a helper function to do the grouping to TransportNodesAction
- Requires a non-null array of FailedNodeExceptions within the BaseNodesResponse constructor
- Reads/writes the array to output
- Also adds StreamInput and StreamOutput methods for generically reading and writing arrays
2016-05-06 14:59:43 -04:00
Adrien Grand 7d8708716e QueryBuilder does not need generics. #18133
QueryBuilder has generics, but those are never used: all call sites use
`QueryBuilder<?>`. Only `AbstractQueryBuilder` needs generics so that the base
class can contain a default implementation for setters that returns `this`.
2016-05-06 08:38:20 +02:00
Robert Muir e3ce6c9048 Painless: add fielddata accessors (.value/.values/.distance()/etc)
This gives better coverage and consistency with the scripting APIs, by
whitelisting the primary search scripting API classes and using them instead
of only Map and List methods.

For example, accessing fields can now be done with `.value` instead of `.0`
because `getValue()` is whitelisted. For now, access to a document's fields in
this way (loads) are fast-pathed in the code, to avoid dynamic overhead.

Access to geo fields and geo distance functions is now supported.

TODO: date support (e.g. whitelist ReadableDateTime methods as a start)
TODO: improve docs (like expressions and groovy have for document's fields)
TODO: remove fast-path hack

Closes #18169

Squashed commit of the following:

commit ec9f24b2424891a7429bb4c0a03f9868cba0a213
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:59:37 2016 -0400

    cutover to <Def> instead of <Object> here

commit 9edb1550438acd209733bc36f0d2e0aecf190ecb
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:03:02 2016 -0400

    add fast-path for docvalues field loads

commit f8e38c0932fccc0cfa217516130ad61522e59fe5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 16:47:31 2016 -0400

    Painless: add fielddata accessors (.value/.values/.distance()/etc)
2016-05-05 18:38:41 -04:00
Jack Conradson 2cae575f53 Added single-quoted strings.
Closes #18150
2016-05-05 09:26:02 -07:00
Robert Muir 59c135b58d make internal Def methods private and add basic javadocs 2016-05-05 03:47:56 -04:00
Robert Muir 928e2b904d painless: optimize/simplify dynamic field and method access 2016-05-05 03:42:14 -04:00
Jason Tedor 784c9e5fb9 Introduce node handshake
This commit introduces a handshake when initiating a light
connection. During this handshake, node information, cluster name, and
version are received from the target node of the connection. This
information can be used to immediately validate that the target node is
a member of the same cluster, and used to set the version on the
stream. This will allow us to extend APIs that are used during initial
cluster recovery without a major version change.

Relates #15971
2016-05-04 20:06:47 -04:00