Commit Graph

3596 Commits

Author SHA1 Message Date
Martijn van Groningen 245882cde3 * Removed `script.default_lang` setting and made `painless` the hardcoded default script language.
** The default script language is now maintained in `Script` class.
* Added `script.legacy.default_lang` setting that controls the default language for scripts that are stored inside documents (for example percolator queries).  This defaults to groovy.
** Added `QueryParseContext#getDefaultScriptLanguage()` that manages the default scripting language. Returns always `painless`, unless loading query/search request in legacy mode then the returns what is configured in `script.legacy.default_lang` setting.
** In the aggregation parsing code added `ParserContext` that also holds the default scripting language like `QueryParseContext`. Most parser don't have access to `QueryParseContext`. This is for scripts in aggregations.
* The `lang` script field is always serialized (toXContent).

Closes #20122
2016-09-06 18:44:48 +02:00
Alexander Reelsen b6bf20c2da Dependencies: Updates to mustache 0.9.3 (#20337) 2016-09-06 13:31:59 +02:00
Martijn van Groningen 6f6d17dc9c ingest: Add `dot_expander` processor that can turn fields with dots in the field name into object fields. 2016-09-05 07:28:38 +02:00
javanna 536d13ff11 ProcessInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
Jack Conradson d0f9ab06f6 Removed extraneous import. 2016-09-01 16:29:08 -07:00
Jack Conradson 71d8ee5eac Merge branch 'master' into deprecate 2016-09-01 08:51:29 -07:00
Martijn van Groningen a110498ad8 settings: Make `action.auto_create_index` setting a dynamic cluster setting.
Closes #7513
2016-09-01 12:33:30 +02:00
Jason Tedor d9064f454e Fix additional exception logging calls
This commit modifies a pair of exception logging calls to use
parameterized messages from Log4j.
2016-08-31 23:14:13 -04:00
Jack Conradson 1cdfc3ccfe Merge branch 'master' into deprecate 2016-08-31 15:58:14 -07:00
Jack Conradson 3b3baa6e6c Made deprecation of Groovy, Javascript, and Python more explicit. 2016-08-31 15:56:31 -07:00
Jason Tedor 76ab02e002 Merge branch 'master' into log4j2
* master:
  Avoid NPE in LoggingListener
  Randomly use Netty 3 plugin in some tests
  Skip smoke test client on JDK 9
  Revert "Don't allow XContentBuilder#writeValue(TimeValue)"
  [docs] Remove coming in 2.0.0
  Don't allow XContentBuilder#writeValue(TimeValue)
  [doc] Remove leftover from CONSOLE conversion
  Parameter improvements to Cluster Health API wait for shards (#20223)
  Add 2.4.0 to packaging tests list
  Docs: clarify scale is applied at origin+offest (#20242)
2016-08-31 16:37:55 -04:00
Jason Tedor 54083f7d6e Randomly use Netty 3 plugin in some tests
When Netty 4 was introduced, it was not the default network
implementation. Some tests were constructed to randomly use Netty 4
instead of the default network implementation. When Netty 4 was made the
default implementation, these tests were not updated. Thus, these tests
are randomly choosing between the default network implementation (Netty
4) and Netty 4. This commit updates these tests to reverse the role of
Netty 3 and Netty 4 so that the randomization is choosing between Netty
3 and the default (again, now Netty 4).

Relates #20265
2016-08-31 15:41:39 -04:00
Jason Tedor e166459bbe Merge branch 'master' into log4j2
* master:
  Increase visibility of deprecation logger
  Skip transport client plugin installed on JDK 9
  Explicitly disable Netty key set replacement
  percolator: Fail indexing percolator queries containing either a has_child or has_parent query.
  Make it possible for Ingest Processors to access AnalysisRegistry
  Allow RestClient to send array-based headers
  Silence rest util tests until the bogusness can be simplified
  Remove unknown HttpContext-based test as it fails unpredictably on different JVMs
  Tests: Improve rest suite names and generated test names for docs tests
  Add support for a RestClient base path
2016-08-31 10:59:27 -04:00
Martijn van Groningen 3fcb95b814 percolator: Fail indexing percolator queries containing either a has_child or has_parent query.
Closes #2960
2016-08-31 07:46:17 +02:00
Jason Tedor abf8a1a3f0 Avoid allocating log parameterized messages
This commit modifies the call sites that allocate a parameterized
message to use a supplier so that allocations are avoided unless the log
level is fine enough to emit the corresponding log message.
2016-08-30 18:17:09 -04:00
Jason Tedor 7da0cdec42 Introduce Log4j 2
This commit introduces Log4j 2 to the stack.
2016-08-30 13:31:24 -04:00
Jack Conradson c8f57df5cf Merge branch 'master' into deprecate 2016-08-30 09:11:18 -07:00
Jack Conradson 7930233527 Deprecate Groovy, Python, and Javascript scripts. 2016-08-30 09:06:18 -07:00
Jason Tedor 0df92a8da8 Upgrade to Netty 4.1.5
This commit upgrades the Netty dependencies from version 4.1.4 to
version 4.1.5. This upgrade brings several bug fixes including the
removal of a obnoxious and scary-looking log message when unsafe is
explicitly disabled.

Relates #20222
2016-08-30 05:46:40 -04:00
Martijn van Groningen 1925813e09 ingest: Fix rename processor change rename leaf fields into branch fields
Instead of get, set and remove we do get, remove and then set to avoid type conflicts in IngestDocument.
If the set still fails we try to restore the original field in ingest document.

Closes #19892
2016-08-30 07:38:01 +02:00
Tanguy Leroux 9727f123b9 Rename Netty TCP transports thread factories from http_* to transport_*
Netty3/4 TcpTransport implementations are creating thread factories with a "http_server" thread prefix whereas it should start with "transport_server" and let the "http_server" prefix for the HttpServerTransport implementations.
2016-08-29 13:49:52 +02:00
Jun Ohtani 2a00c9dc46 Merge pull request #19860 from johtani/fix/validate_empty_field_name
Validate blank field name
2016-08-29 11:52:18 +09:00
Martijn van Groningen 48926b4d66 ingest: don't render template twice for append processor 2016-08-26 18:07:32 +02:00
Chris Earle bd0b06440e Add "Async" to the end of each Async RestClient method
This makes it much harder to accidentally miss the Response.
2016-08-26 10:51:33 -04:00
Jun Ohtani 450f47d5b5 Validate blank field name
add validation and validate only 5.0+
Add tests before 5.0

Closes #19251
2016-08-26 20:10:33 +09:00
Jason Tedor bc136a90d5 Add network types to cluster stats
The network types in use on a cluster can be useful information to have,
so this commit adds aggregate metrics for the network types in use in a
cluster to the cluster stats.

Relates #20144
2016-08-25 21:08:05 -04:00
Jason Tedor 5a48ad661d Address race condition in HTTP pipeline tests
The Netty 4 HTTP server pipeline tests contains two different test
cases. The general idea behind these tests is to submit some requests to
a Netty 4 HTTP server, one test with pipelining enabled and another test
with pipelining disabled. These requests are submitted to two endpoints,
one with a path like /{id} and another with a path like /slow with a
query string parameter sleep. This parameter tells the request handler
how long to sleep for before replying. The idea is that in the case of
the pipelining enabled tests, the requests should come back exactly in
the order submitted, even with some of the requests hitting the slow
endpoint with random sleep durations; this is the guarantee that
pipelining provides. And in the case of the pipelining disabled tests,
requests were randombly submitted to /{id} and /slow with sleep
parameters starting at 600ms and increasing by 100ms for each slow
request constructed. We would expect the requests to come back with the
all the responses to the /{id} requests first because these requests
will execute instantaneously, and then the responses to the /slow
requests. Further, it was expected that the slow requests would come
back ordered by the length of the sleep, the thinking being that 100ms
should be enough of a difference between each request that we would
avoid any race conditions. Sadly, this is not the case, the threads do
sometimes hit race conditions.

This commit modifies the HTTP server pipelining tests to address this
race condition. The modification is that the query string parameter on
the /slow endpoint is removed in favor of just submitting requests to
the path /slow/{id}, where id just used a marker to distinguish each
request. The server chooses a random sleep of at least 500ms for each
request on the slow path. The assertion here then is that the /{id}
responses arrive first, then then /slow responses. We can not make an
assertion on the order of the responses, but we can assert that we did
see every expected response.

Relates #19845
2016-08-25 14:34:11 -04:00
Jack Conradson 0fdadf4737 Merge branch 'master' into break 2016-08-25 09:26:04 -07:00
Michael McCandless 1fe3e36934 Merge pull request #20147 from mikemccand/lucene_620_upgrade
Upgrade to Lucene 6.2.0
2016-08-25 06:03:34 -04:00
Jack Conradson 3deea3dbde Made for/each break tests more robust in Painless. 2016-08-24 15:17:18 -07:00
Mike McCandless 0ccfe69789 Upgrade to Lucene 6.2.0 2016-08-24 17:26:28 -04:00
Jack Conradson c60885b5d4 Fix break bug in for/foreach loops. 2016-08-24 14:25:54 -07:00
Igor Motov b36fbc4452 Add support for parameters to the script ingest processor
The script processor should support `params` to be consistent with all other script consumers.
2016-08-24 16:49:48 -04:00
Jim Ferenczi 4682fc34ae Add the ability to disable the retrieval of the stored fields entirely
This change adds a special field named _none_ that allows to disable the retrieval of the stored fields in a search request or in a TopHitsAggregation.

To completely disable stored fields retrieval (including disabling metadata fields retrieval such as _id or _type) use _none_ like this:

````
POST _search
{
   "stored_fields": "_none_"
}
````
2016-08-24 16:40:08 +02:00
Clinton Gormley abc025e18b Fixed the reindex_rethrottle REST tests
The API was renamed from reindex.rethrottle to reindex_rethrottle
2016-08-24 14:55:02 +02:00
Jack Conradson 131e370a16 Make Painless the default scripting language.
Closes #20017
2016-08-22 17:38:02 -07:00
Nik Everett 312a7d45ba Wait for task to start in reindex test
`RethrottleTests#testReindex` fail in CI:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+multijob-intake/1274/console

I was unable to reproduce it locally but it *looks* like a race to start
the task. So I've added a wait for it to start just in case.
2016-08-17 12:08:55 -04:00
Nik Everett 39d8f5f123 Reindex tests should expect the right failure
Reindex intentionally tries to fail the search operation to make sure
that the exception flows back. The exception message changed so we
should catch the appropriate exception.
2016-08-17 10:25:38 -04:00
Nik Everett 34bbd27f84 Fix _update_by_query's ingest pipeline support
It wasn't being serialized so it wasn't working with the transport
client.
2016-08-16 16:03:18 -04:00
Nik Everett 862843ec90 Suppress failing test
This test was failing in the presence of transport clients. This turns
off transport clients while I fix the test so it doesn't fail for
everyone in the mean time.
2016-08-16 15:12:40 -04:00
Ryan Ernst 743d9fd008 Merge branch 'master' into search_parser 2016-08-16 11:28:59 -07:00
Nik Everett fdd50612ae Fix reindex under the transport client
The big change here is cleaning up the `TaskListResponse` so it doesn't
have a breaky `toString` implementation. That was causing the reindex
tests to break.

Also removed `NetworkModule#registerTaskStatus` which is part of the
Plugin API. Use `Plugin#getNamedWriteables` instead.
2016-08-16 12:15:15 -04:00
Nik Everett 46bf8baf2e Switch aggregation registration for push to pull
Adds `getAggregations` to `SearchPlugin` which can be used to register
aggregations.

Fixup MockNode which wasn't createing MockBigArrays.
2016-08-16 09:08:36 -04:00
Ryan Ernst 7fde410586 Internal: Consolidate search parser registries
Parsing a search request is currently split up among a number of
classes, using multiple public static methods, which take multiple
regstries of elements that may appear in the search request like query
parsers and aggregations. This change begins consolidating all this code
by collapsing the registries normally used for parsing search requests
into a single SearchRequestParsers class. It is also made available to
plugin services to enable templating of search requests.  Eventually all
of the actual parsing logic should move to the class, and the registries
should be hidden, but for now they are at least co-located to reduce the
number of objects that must be passed around.
2016-08-16 01:59:24 -07:00
Nik Everett 1452ab4b9f Squash the rest of o.e.rest.action
Squashes all the subpackages of `org.elasticsearch.rest.action` down to
the following:
* `o.e.rest.action.admin` - Administrative actions
* `o.e.rest.action.cat` - Actions that make tables for `grep`ing
* `o.e.rest.action.document` - Actions that act on documents
* `o.e.rest.action.ingest` - Actions that act on ingest pipelines
* `o.e.rest.action.search` - Actions that search

I'm tempted to merge `search` into `document` but the `document`
package feels fairly complete as is and `Suggest` isn't actually always
about documents either....

I'm also tempted to merge `ingest` into `admin.cluster` because the
latter contains the actions for dealing with stored scripts.

I've moved the `o.e.rest.action.support` into `o.e.rest.action`.

I've also added `package-info.java`s to all packges in `o.e.rest`. I
figure if the package is too small to deserve a `package-info.java` file
then it is too small to deserve to be a package....

Also fixes checkstyle in all moved classes.
2016-08-15 21:06:32 -04:00
Tal Levy 84bf24b1e9 remove ability to set field value in script-processor configuration (#19981) 2016-08-15 10:57:39 -07:00
Igor Motov 10a766704e Rename Task Persistence into Storing Task Results
The term persisted task was used to indicate that a task should store its results upon its completion. We would like to use this term to indicate that a task can survive restart of nodes instead. This commit removes usages of the term "persist" when it means store results.
2016-08-15 10:02:43 -04:00
Nik Everett cf6e1a4362 Move all FetchSubPhases to `o.e.search.fetch.subphase`
As the most complicated `FetchSubPhase` highlighting gets its own package
(`o.e.seach.fetch.subphase.highlight`. No other `FetchSubPhase`s get their
own package. Instead they all reside together in `o.e.search.fetch.subphase`.

Add package descriptions to `o.e.search.fetch` and subpackages.
2016-08-12 18:21:15 -04:00
Lee Hinman f832eaa6d2 Merge remote-tracking branch 'dakrone/handle-groovy-errors-better' 2016-08-12 13:07:13 -06:00
Lee Hinman aab8c1f032 Catch AssertionError and NoClassDefFoundError in groovy scripts
Previously, we only caught subclasses of Exception, however, there are
some cases when Errors are thrown instead of Exceptions. These two cases
are `assert` and when a class cannot be found.

Without this change, the exception would bubble up to the
`uncaughtExceptionHandler`, which would in turn, exit the JVM
(related: #19923).

A note of difference between regular Java asserts and Groovy asserts,
from http://docs.groovy-lang.org/docs/latest/html/documentation/core-testing-guide.html

"Another important difference from Java is that in Groovy assertions are
enabled by default. It has been a language design decision to remove the
possibility to deactivate assertions."

In the event that a user uses an assert such as:

```groovy
def bar=false; assert bar, "message";
```

The GroovyScriptEngineService throws a NoClassDefFoundError being unable
to find the `java.lang.StringBuffer` class. It is *highly* recommended
that any Groovy scripting user switch to using regular exceptions rather
than unconfiguration Groovy assertions.

Resolves #19806
2016-08-12 13:06:49 -06:00