3679 Commits

Author SHA1 Message Date
Isabel Drost-Fromm
d34a117422 Use roundtrip helper for rank eval spec tests. 2016-09-07 13:03:18 +02:00
Isabel Drost-Fromm
ba9956a468 Remove FromXContentBuilder from RatedDocumentKey 2016-09-07 12:52:30 +02:00
Isabel Drost-Fromm
9d8bb720dc Remove leftover FromXContentBuilder reference 2016-09-07 12:50:04 +02:00
Isabel Drost-Fromm
2af0218bdd Rename test class to match renamed implementation
Used to be QuerySpec, is now RatedRequest; changing name
of test accordingly.
2016-09-07 12:12:06 +02:00
Isabel Drost-Fromm
0d25fc4925 Re-use roundtrip helper for QuerySpecTests 2016-09-07 12:11:07 +02:00
Isabel Drost-Fromm
b3a4a89151 Replace magic number in test with random number. 2016-09-07 12:06:39 +02:00
Isabel Drost-Fromm
c8a3b3c32f Move xcontent roundtrip method to helper class. 2016-09-07 12:02:00 +02:00
Isabel Drost-Fromm
efbef20361 Remove call to getClass from hashCode implementations. 2016-09-07 11:53:39 +02:00
Isabel Drost-Fromm
333b769871 Remove usage of FromXContentBuilder 2016-09-07 11:48:50 +02:00
Isabel Drost-Fromm
6bc646ac84 Make constructor enforce non-optional position argument. 2016-09-07 11:21:50 +02:00
Isabel Drost-Fromm
b8652b1224 Test wouldn't compile w/o these annotations. 2016-09-07 11:21:29 +02:00
Isabel Drost-Fromm
c4cfb9d7b5 Merge branch 'feature/rank-eval' into feature/rank-eval-roundtrip-testing 2016-09-07 11:05:56 +02:00
Christoph Büscher
dab771a0b4 Merge branch 'master' into feature/rank-eval 2016-09-07 09:57:43 +02:00
Martijn van Groningen
245882cde3 * Removed script.default_lang setting and made painless the hardcoded default script language.
** The default script language is now maintained in `Script` class.
* Added `script.legacy.default_lang` setting that controls the default language for scripts that are stored inside documents (for example percolator queries).  This defaults to groovy.
** Added `QueryParseContext#getDefaultScriptLanguage()` that manages the default scripting language. Returns always `painless`, unless loading query/search request in legacy mode then the returns what is configured in `script.legacy.default_lang` setting.
** In the aggregation parsing code added `ParserContext` that also holds the default scripting language like `QueryParseContext`. Most parser don't have access to `QueryParseContext`. This is for scripts in aggregations.
* The `lang` script field is always serialized (toXContent).

Closes #20122
2016-09-06 18:44:48 +02:00
Alexander Reelsen
b6bf20c2da Dependencies: Updates to mustache 0.9.3 (#20337) 2016-09-06 13:31:59 +02:00
Christoph Büscher
0b92d524a7 Add threshold for document ratings for PrecisionAtN and ReciprocalRank
PrecisionAtN and ReciprocalRank are binary evaluation metrics by default that
only distiguish between relevant/irrelevant search results. So far we assumed
that relevant documents are labaled with 1 (irrelevant docs with 0) in the
evaluation request, but this is cumbersome if the ratings are provided on a
larger integer scale and would need to get mapped to a 0/1 value.

This change introduces a threshold parameter on the PrecisionAtN and
ReciprocalRank metric than can be used to set the threshold from which on a
document is considered "relevant". It defaults to 1, so in case of 0/1 ratings
the threshold doesn't have to be set and only ratings with value 0 are
considered to be irrelevant.
2016-09-05 15:34:47 +02:00
Christoph Büscher
8910864a18 Merge branch 'master' into feature/rank-eval 2016-09-05 15:22:04 +02:00
Martijn van Groningen
6f6d17dc9c ingest: Add dot_expander processor that can turn fields with dots in the field name into object fields. 2016-09-05 07:28:38 +02:00
Christoph Büscher
8daec109fc Adding additional json object level to PrecisionAtN rendering 2016-09-02 18:03:58 +02:00
Christoph Büscher
11116db337 Merge branch 'master' into feature/rank-eval 2016-09-02 10:49:33 +02:00
javanna
536d13ff11 ProcessInfo to implement Writeable rather than Streamable 2016-09-02 10:23:05 +02:00
Jack Conradson
d0f9ab06f6 Removed extraneous import. 2016-09-01 16:29:08 -07:00
Jack Conradson
71d8ee5eac Merge branch 'master' into deprecate 2016-09-01 08:51:29 -07:00
Christoph Büscher
1c02e48a8f Merge branch 'master' into feature/rank-eval
Conflicts:
	test/framework/src/main/java/org/elasticsearch/test/TestCluster.java
2016-09-01 13:06:21 +02:00
Martijn van Groningen
a110498ad8 settings: Make action.auto_create_index setting a dynamic cluster setting.
Closes #7513
2016-09-01 12:33:30 +02:00
Jason Tedor
d9064f454e Fix additional exception logging calls
This commit modifies a pair of exception logging calls to use
parameterized messages from Log4j.
2016-08-31 23:14:13 -04:00
Jack Conradson
1cdfc3ccfe Merge branch 'master' into deprecate 2016-08-31 15:58:14 -07:00
Jack Conradson
3b3baa6e6c Made deprecation of Groovy, Javascript, and Python more explicit. 2016-08-31 15:56:31 -07:00
Jason Tedor
76ab02e002 Merge branch 'master' into log4j2
* master:
  Avoid NPE in LoggingListener
  Randomly use Netty 3 plugin in some tests
  Skip smoke test client on JDK 9
  Revert "Don't allow XContentBuilder#writeValue(TimeValue)"
  [docs] Remove coming in 2.0.0
  Don't allow XContentBuilder#writeValue(TimeValue)
  [doc] Remove leftover from CONSOLE conversion
  Parameter improvements to Cluster Health API wait for shards (#20223)
  Add 2.4.0 to packaging tests list
  Docs: clarify scale is applied at origin+offest (#20242)
2016-08-31 16:37:55 -04:00
Jason Tedor
54083f7d6e Randomly use Netty 3 plugin in some tests
When Netty 4 was introduced, it was not the default network
implementation. Some tests were constructed to randomly use Netty 4
instead of the default network implementation. When Netty 4 was made the
default implementation, these tests were not updated. Thus, these tests
are randomly choosing between the default network implementation (Netty
4) and Netty 4. This commit updates these tests to reverse the role of
Netty 3 and Netty 4 so that the randomization is choosing between Netty
3 and the default (again, now Netty 4).

Relates #20265
2016-08-31 15:41:39 -04:00
Jason Tedor
e166459bbe Merge branch 'master' into log4j2
* master:
  Increase visibility of deprecation logger
  Skip transport client plugin installed on JDK 9
  Explicitly disable Netty key set replacement
  percolator: Fail indexing percolator queries containing either a has_child or has_parent query.
  Make it possible for Ingest Processors to access AnalysisRegistry
  Allow RestClient to send array-based headers
  Silence rest util tests until the bogusness can be simplified
  Remove unknown HttpContext-based test as it fails unpredictably on different JVMs
  Tests: Improve rest suite names and generated test names for docs tests
  Add support for a RestClient base path
2016-08-31 10:59:27 -04:00
Martijn van Groningen
3fcb95b814 percolator: Fail indexing percolator queries containing either a has_child or has_parent query.
Closes #2960
2016-08-31 07:46:17 +02:00
Jason Tedor
abf8a1a3f0 Avoid allocating log parameterized messages
This commit modifies the call sites that allocate a parameterized
message to use a supplier so that allocations are avoided unless the log
level is fine enough to emit the corresponding log message.
2016-08-30 18:17:09 -04:00
Jason Tedor
7da0cdec42 Introduce Log4j 2
This commit introduces Log4j 2 to the stack.
2016-08-30 13:31:24 -04:00
Jack Conradson
c8f57df5cf Merge branch 'master' into deprecate 2016-08-30 09:11:18 -07:00
Jack Conradson
7930233527 Deprecate Groovy, Python, and Javascript scripts. 2016-08-30 09:06:18 -07:00
Isabel Drost-Fromm
8f71c29c16 Merge branch 'master' into feature/rank-eval 2016-08-30 14:20:43 +02:00
Jason Tedor
0df92a8da8 Upgrade to Netty 4.1.5
This commit upgrades the Netty dependencies from version 4.1.4 to
version 4.1.5. This upgrade brings several bug fixes including the
removal of a obnoxious and scary-looking log message when unsafe is
explicitly disabled.

Relates #20222
2016-08-30 05:46:40 -04:00
Martijn van Groningen
1925813e09 ingest: Fix rename processor change rename leaf fields into branch fields
Instead of get, set and remove we do get, remove and then set to avoid type conflicts in IngestDocument.
If the set still fails we try to restore the original field in ingest document.

Closes #19892
2016-08-30 07:38:01 +02:00
Tanguy Leroux
9727f123b9 Rename Netty TCP transports thread factories from http_* to transport_*
Netty3/4 TcpTransport implementations are creating thread factories with a "http_server" thread prefix whereas it should start with "transport_server" and let the "http_server" prefix for the HttpServerTransport implementations.
2016-08-29 13:49:52 +02:00
Jun Ohtani
2a00c9dc46 Merge pull request #19860 from johtani/fix/validate_empty_field_name
Validate blank field name
2016-08-29 11:52:18 +09:00
Martijn van Groningen
48926b4d66 ingest: don't render template twice for append processor 2016-08-26 18:07:32 +02:00
Chris Earle
bd0b06440e Add "Async" to the end of each Async RestClient method
This makes it much harder to accidentally miss the Response.
2016-08-26 10:51:33 -04:00
Jun Ohtani
450f47d5b5 Validate blank field name
add validation and validate only 5.0+
Add tests before 5.0

Closes #19251
2016-08-26 20:10:33 +09:00
Jason Tedor
bc136a90d5 Add network types to cluster stats
The network types in use on a cluster can be useful information to have,
so this commit adds aggregate metrics for the network types in use in a
cluster to the cluster stats.

Relates #20144
2016-08-25 21:08:05 -04:00
Jason Tedor
5a48ad661d Address race condition in HTTP pipeline tests
The Netty 4 HTTP server pipeline tests contains two different test
cases. The general idea behind these tests is to submit some requests to
a Netty 4 HTTP server, one test with pipelining enabled and another test
with pipelining disabled. These requests are submitted to two endpoints,
one with a path like /{id} and another with a path like /slow with a
query string parameter sleep. This parameter tells the request handler
how long to sleep for before replying. The idea is that in the case of
the pipelining enabled tests, the requests should come back exactly in
the order submitted, even with some of the requests hitting the slow
endpoint with random sleep durations; this is the guarantee that
pipelining provides. And in the case of the pipelining disabled tests,
requests were randombly submitted to /{id} and /slow with sleep
parameters starting at 600ms and increasing by 100ms for each slow
request constructed. We would expect the requests to come back with the
all the responses to the /{id} requests first because these requests
will execute instantaneously, and then the responses to the /slow
requests. Further, it was expected that the slow requests would come
back ordered by the length of the sleep, the thinking being that 100ms
should be enough of a difference between each request that we would
avoid any race conditions. Sadly, this is not the case, the threads do
sometimes hit race conditions.

This commit modifies the HTTP server pipelining tests to address this
race condition. The modification is that the query string parameter on
the /slow endpoint is removed in favor of just submitting requests to
the path /slow/{id}, where id just used a marker to distinguish each
request. The server chooses a random sleep of at least 500ms for each
request on the slow path. The assertion here then is that the /{id}
responses arrive first, then then /slow responses. We can not make an
assertion on the order of the responses, but we can assert that we did
see every expected response.

Relates #19845
2016-08-25 14:34:11 -04:00
Jack Conradson
0fdadf4737 Merge branch 'master' into break 2016-08-25 09:26:04 -07:00
Isabel Drost-Fromm
1d69a943be Merge branch 'feature/rank-eval' into feature/rank-eval-roundtrip-testing 2016-08-25 12:14:34 +02:00
Isabel Drost-Fromm
518e09d028 Merge branch 'master' into feature/rank-eval 2016-08-25 12:13:59 +02:00
Isabel Drost-Fromm
f18e23eb51 Fix test errors, add roundtrip testing to RankEvalSpec
This adds roundtrip testing to RankEvalSpec, fixes issues
introduced with the previous roundtrip tests, splits
xcontent generation/parsing from actually checking the resulting
objects to deal with e.g. all evaluation metrics needing
some extra treatment.

Renames QuerySpec to RatedRequest, renames newly introduced xcontent
generation helper to conform with naming conventions.

Fixes several lines that were too long, adds missing types where
needed.
2016-08-25 12:09:55 +02:00