Commit Graph

3719 Commits

Author SHA1 Message Date
Jack Conradson 1424f09c02 Remove Painless grammar ambiguities.
Closes #18531
2016-05-23 16:39:52 -07:00
Robert Muir a56a9cb818 improve painless whitelist coverage of java api 2016-05-23 19:35:28 -04:00
Ryan Ernst c7b45b2cc7 Tests: Remove unnecessary Callable variant of assertBusy
The assertBusy method currently has both a Runnable and Callable
version. This has caused confusion with type inference and lambdas
sometimes, in particular with java 9. This change removes the callable
version as nothing was actually using it.
2016-05-23 16:17:43 -07:00
Nik Everett 62ac719a94 Rerwite RetryTests to very carefully block the executors
This reproduces every time. No more randomness! Hurray!
2016-05-23 14:38:40 -04:00
Nik Everett b7817a6306 [reindex] Retry the retry test if it didn't cause retries
The retry test has failed a couple of times in CI because it wasn't able
to cause any retries. Putting it in a bash `while` loop shows that it
eventually does fail that way. The seed "4F6477A9C999CA20" seems especially
good at failing to get retries. It doesn't fail all the time, but more
than most.

This adds a retry to each test case, retrying a maximum of 10 times or
until it causes the retries. I've seen it fail to get retries 7 times
in a row but not go beyond that. Retrying doesn't seem to really hurt
the test runtime all that much. Most of the time is in the startup
cost.

Failing CI build that triggered this:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+periodic/852/console
2016-05-23 14:38:40 -04:00
Adrien Grand 459916f5dd Remove custom Base64 implementation. #18413
This replaces o.e.common.Base64 with java.util.Base64.
2016-05-23 11:32:42 +02:00
Tanguy Leroux b0b503035a Update reindex cancel tests 2016-05-23 10:31:14 +02:00
Jason Tedor 6eb96e5fd8 Fix line-length violations in ABBSAT.java
This commit fixes line-length violations in
AsyncBulkByScrollActionTests.java.
2016-05-21 21:26:28 -04:00
Jason Tedor ad7229fe72 Merge branch 'master' into feature/seq_no
* master: (158 commits)
  Document the hack
  Refactor property placeholder use of env. vars
  Force java9 log4j hack in testing
  Fix log4j buggy java version detection
  Make java9 work again
  Don't mkdir directly in deb init script
  Fix env. var placeholder test so it's reproducible
  Remove ScriptMode class in favor of boolean true/false
  [rest api spec] fix doc urls
  Netty request/response tracer should wait for send
  Filter client/server VM options from jvm.options
  [rest api spec] fix url for reindex api docs
  Remove use of a Fields class in snapshot responses that contains x-content keys, in favor of declaring/using the keys directly.
  Limit retries of failed allocations per index (#18467)
  Proxy box method to use valueOf.
  Use the build-in valueOf method instead of the custom one.
  Fixed tests and added a comment to the box method.
  Fix boxing.
  Do not decode path when sending error
  Fix race condition in snapshot initialization
  ...
2016-05-21 21:04:43 -04:00
Ryan Ernst 37d36f2f4c Merge branch 'master' into java9 2016-05-21 14:19:58 -07:00
Ryan Ernst 1d40c4bbc1 Make java9 work again
This change makes ES compile with java9 again, build 118.
* There are a handful of changes due to failure to determine types during compile.
* The attachment plugins which use tika needed to have tika upgraded in order to pickup fixes there for java 9.
* azure discovery and s3 repository indirectly depend on jaxb, which is no longer in the default modules. They now add a jaxb dependency externally, and make JarHell allow for this package.
2016-05-21 09:41:51 -07:00
Lee Hinman fdfd2a2f18 Remove ScriptMode class in favor of boolean true/false
This removes the ScriptMode class entirely, which was an enum with two
options (ON and OFF) which essentially boiled down to true and false.
Now the boolean values are used instead.
2016-05-20 15:01:30 -06:00
Robert Muir a2ff002e2b Merge pull request #18463 from rmuir/whitelist_cleanup
painless: steps at definition cleanup
2016-05-20 14:26:41 -04:00
Jack Conradson b156438957 Proxy box method to use valueOf. 2016-05-20 11:22:18 -07:00
Jack Conradson 0e24ed49e6 Use the build-in valueOf method instead of the custom one. 2016-05-20 11:13:43 -07:00
Jack Conradson d836194095 Fixed tests and added a comment to the box method. 2016-05-20 11:04:09 -07:00
Jack Conradson 2b793c1e06 Fix boxing. 2016-05-20 10:38:06 -07:00
Robert Muir c0eb813e10 Merge branch 'clean' into whitelist_cleanup 2016-05-20 10:59:13 -04:00
Robert Muir ce9b36590a simplify tree node use of compiler settings 2016-05-20 09:36:09 -04:00
Martijn van Groningen 80fee8666f percolator: Removed percolator cache
Before 5.0 for it was required that the percolator queries were cached in jvm heap as Lucene queries for two reasons:
1) Performance. The percolator evaluated all percolator queries all the time. There was no pre-selecting queries that are likely to match like we have today.
2) Updates made to percolator queries were visible in realtime, Today these changes are visible in near realtime. So updating no longer requires the percolator to have the queries in jvm heap.

So having the percolator queries in jvm heap via the percolator cache is now less attractive. Especially when there are many percolator queries then these queries can consume many GBs of jvm heap.
Removing the percolator cache does make the percolate query slower compared to how the execution time in 5.0.0-alpha1 and alpha2, but it is still faster compared to 2.x and before.
2016-05-20 14:52:16 +02:00
Robert Muir b3c71c1e3a improve style for these constants 2016-05-20 08:42:29 -04:00
Uwe Schindler 7bc91c00d9 More elegant way to force class initialization 2016-05-20 14:12:15 +02:00
Uwe Schindler 6205f79554 Remove Definitions's copy-ctor; fix RuntimeClass to be unmodifiable
Please note: The maps inside the pirvate singleton instance of Defininition are no longer unmodifiable, but nothing from the outside can modify it! All private :-)
2016-05-20 14:03:46 +02:00
Uwe Schindler 4ffa92c7c1 Remove the Definition instance passed around everywhere 2016-05-20 12:35:08 +02:00
Uwe Schindler 91f4bba042 Make Definition's public API completely static.
TODO: Remove Definition arguments everywhere and hide INSTANCE field!
2016-05-20 11:22:03 +02:00
Uwe Schindler f5fc60ac67 Add utility type constant 2016-05-20 11:05:40 +02:00
Jack Conradson 4e454439bb Fixed bugs in comparison with Def. Fixed may tests.n 2016-05-20 02:04:38 -07:00
Uwe Schindler 87ab39613d more constants 2016-05-20 10:56:50 +02:00
Uwe Schindler 82aed08301 Restore constants as static final (first step) 2016-05-20 10:48:41 +02:00
Jack Conradson 64e2ef5807 Removed transforms. No user-facing boxing. 2016-05-19 23:38:14 -07:00
Jack Conradson 3381f50e07 Merge branch 'master' into clean 2016-05-19 18:52:10 -07:00
Jack Conradson 3962553551 Removed user-facing boxing. 2016-05-19 18:51:55 -07:00
Uwe Schindler cfe555d216 move unused @SuppressWarnings one level up 2016-05-20 00:46:05 +02:00
Nik Everett 223cb6a7f0 [reindex] Mark test awaits fix because it is unstable
Fix coming.
2016-05-19 18:03:36 -04:00
Robert Muir afa9ebad28 dce 2016-05-19 16:03:11 -04:00
Robert Muir 0ee726ed0d nuke eclipse warnings 2016-05-19 15:35:26 -04:00
Robert Muir a569130368 fix typo 2016-05-19 15:28:59 -04:00
Robert Muir b9c7dbcfbd nuke overflow detection as we cannot guarantee for def. simplify life :) 2016-05-19 15:25:31 -04:00
Tanguy Leroux a01ecb20ea Port Delete By Query to Reindex infrastructure
closes #16883
2016-05-19 16:07:50 +02:00
Robert Muir 915ab16176 First steps at whitelist cleanup 2016-05-19 09:29:48 -04:00
Ali Beyad fc6df23fea Rename AggregatorBuilder and all of its subclasses to
AggregationBuilder, in keeping consistent with the Java APIs.

Closes #18377
Closes #18367
2016-05-19 09:25:29 -04:00
Robert Muir 1022123f59 Merge pull request #18400 from uschindler/painless_java9StringConcats
Use Java 9 Indy String Concats, if available
2016-05-18 08:00:20 -04:00
Uwe Schindler ea9026e6d8 painless: fix checkstyle 2016-05-18 00:31:12 +02:00
Uwe Schindler b418d01117 painless: Improve string concat tests; add state machine check after MethodWriter 2016-05-18 00:26:47 +02:00
Jack Conradson 3042a77d2b Merge pull request #18425 from jdconrad/cast
Made def variable casting consistent with invokedynamic rules
2016-05-17 13:29:49 -07:00
Jack Conradson f055ad641a Made def variable casting consistent with invokedynamic rules. 2016-05-17 13:16:49 -07:00
polyfractal 978c1e3e36 [TEST] Add missing sort processor to test
Also fixes the naming of the sort REST test
to follow the numbering convention
2016-05-17 14:52:17 -04:00
Nik Everett cfb06954ba [reindex] Add assertBusy to test
It has timing issues.
2016-05-17 14:05:35 -04:00
Nik Everett fe4823eae0 Reindex should retry on search failures
This uses the same backoff policy we use for bulk and just retries until
the request isn't rejected.

Instead of `{"retries": 12}` in the response to count retries this now
looks like `{"retries": {"bulk": 12", "search": 1}`.

Closes #18059
2016-05-17 13:58:45 -04:00
Uwe Schindler b27b0483d5 Merge branch 'master' into painless_java9StringConcats
# Conflicts:
#	modules/lang-painless/src/test/java/org/elasticsearch/painless/StringTests.java
2016-05-17 18:47:33 +02:00
Uwe Schindler 5622e9c9d3 painless: use Java 9 Indy String Concats, if available 2016-05-17 17:44:46 +02:00
Robert Muir fa5f099486 make semicolons only optional at the end 2016-05-17 10:55:40 -04:00
Robert Muir 92339c4763 painless: add method overloading based on arity
Closes #18385

Squashed commit of the following:

commit b2819df4d392d69b86e5c96d358eb03424e67e02
Author: Robert Muir <rmuir@apache.org>
Date:   Tue May 17 09:15:47 2016 -0400

    add note about tuple

commit 85fcac6a0d0674da24535121eab23e2c407d683f
Author: Robert Muir <rmuir@apache.org>
Date:   Mon May 16 23:39:25 2016 -0400

    painless: add method overloading based on arity
2016-05-17 09:17:43 -04:00
Uwe Schindler 75f27df376 painless: Refactor WriterUtils to extends ASM GeneratorAdapter as "MethodWriter" for easy usage (and more coming later) 2016-05-17 02:25:05 +02:00
Jack Conradson 3fd3d367ef Moved ADefLink to be a marker interface instead. 2016-05-16 15:48:29 -07:00
Jack Conradson 3f23186712 Merge branch 'master' into types 2016-05-16 14:22:59 -07:00
Jack Conradson 95a1b5b2a3 Fixed a casting bug in Painless. 2016-05-16 14:18:05 -07:00
Robert Muir e69305ae21 Merge pull request #18372 from rmuir/painless_whitelist_expansion
painless whitelist expansion
2016-05-16 17:08:49 -04:00
Robert Muir 8d4c1befe5 Merge pull request #18364 from rmuir/nukeRunAsFloat
Remove LeafSearchScript.runAsFloat(): Nothing calls it.
2016-05-16 17:08:25 -04:00
Jack Conradson 1d394868d4 Merge branch 'master' into types 2016-05-16 12:43:24 -07:00
Jack Conradson 43344eee82 Add more def tests including checking bytecode for specific
optimizations.
2016-05-16 12:42:25 -07:00
Jack Conradson f664fa5f33 Merge pull request #18350 from uschindler/painless_isAssignableFrom
Use isAssignableFrom instead of relying on ClassCastException
2016-05-16 12:41:09 -07:00
Jack Conradson a2c2628a78 Merge pull request #18359 from uschindler/painless_removeStoreBoxing
Remove boxing when loading and storing values in "def" fields/arrays, remove boxing onsimple method calls of "def" methods
2016-05-16 12:11:37 -07:00
Uwe Schindler d6cbbdeb98 painless: Some reformatting in EChain to make it clear if "this" or inner "expression" is affected 2016-05-16 21:04:03 +02:00
Uwe Schindler b05ac87a5d painless: Add Jack's test 2016-05-16 20:37:51 +02:00
Uwe Schindler 3a5ef68ad0 painless: Fix issue with dup and cast 2016-05-16 20:31:27 +02:00
Robert Muir 6ac2877c32 Merge pull request #18371 from uschindler/painless_debuggerCleanup
painless: Small cleanup of Debugger class to use StringWriter
2016-05-16 09:58:50 -04:00
Robert Muir 2fc7ca46e1 expand whitelist for basic types 2016-05-16 09:56:29 -04:00
Uwe Schindler 07e4f416e6 painless: Small cleanup of Debugger class to use StringWriter 2016-05-16 15:52:14 +02:00
Nik Everett f569576c5b Switch default batch size for reindex to 1000 2016-05-16 08:19:29 -04:00
Robert Muir 8edf213492 Remove LeafSearchScript.runAsFloat(): Nothing calls it. 2016-05-15 22:59:28 -04:00
Uwe Schindler 65aca4f71c painless: make compound statement like a[1]++ work with def. There was also a bug with the size of LDefArray: fixed 2016-05-15 23:00:32 +02:00
Uwe Schindler d221cd14d2 painless: Also remove boxing for reads and method calls 2016-05-15 19:49:25 +02:00
Uwe Schindler 604bcd9320 painless: Make field stores not box; use GeneratorAdapter.invokeDynmaic for consistency with other method calls 2016-05-15 18:31:53 +02:00
Jason Tedor 15d3d74444 Merge branch 'master' into feature/seq_no
* master: (904 commits)
  Removes unused methods in the o/e/common/Strings class.
  Add note regarding thread stack size on Windows
  painless: restore accidentally removed test
  Documented fuzzy_transpositions in match query
  Add not-null precondition check in BulkRequest
  Build: Make run task you full zip distribution
  Build: More pom generation improvements
  Add test for wrong array index
  Take return type from "after" field.
  painless: build descriptor of array and field load/store in code; fix array index to adapt type not DEF
  Build: Add developer info to generated pom files
  painless: improve exception stacktraces
  painless: Rename the dynamic call site factory to DefBootstrap and make the inner class very short (PIC = Polymorphic Inline Cache)
  Remove dead code.
  Avoid race while retiring executors
  Allow only a single extension for a scripting engine
  Adding REST tests to ensure key_as_string behavior stays consistent
  [test] Set logging to 11 on reindex test
  [TEST] increase logger level until we know what is going on
  Don't allow `fuzziness` for `multi_match` types cross_fields, phrase and phrase_prefix
  ...
2016-05-14 20:23:59 -04:00
Uwe Schindler 8195ef9e7a painless: Transform more try-catch-ClassCastException to isAssignableFrom 2016-05-15 01:06:29 +02:00
Uwe Schindler 7211017813 painless: Some cleanups 2016-05-14 17:23:29 +02:00
Uwe Schindler c78bf8c867 painless: Use isAssignableFrom instead of relying on ClassCastException 2016-05-14 16:04:18 +02:00
Uwe Schindler c8bd521ed3 painless: restore accidentally removed test 2016-05-14 12:00:30 +02:00
Uwe Schindler ba9dae820a Add test for wrong array index 2016-05-13 23:22:41 +02:00
Uwe Schindler 1a76329d89 Take return type from "after" field. 2016-05-13 23:17:22 +02:00
Uwe Schindler 1b10a14e8d painless: build descriptor of array and field load/store in code; fix array index to adapt type not DEF 2016-05-13 23:04:04 +02:00
Robert Muir 2028691e66 painless: improve exception stacktraces
closes #18319
2016-05-13 15:40:45 -04:00
Uwe Schindler 9731d5d314 painless: Rename the dynamic call site factory to DefBootstrap and make the inner class very short (PIC = Polymorphic Inline Cache) 2016-05-13 19:58:05 +02:00
Lee Hinman 9bcdafedda Allow only a single extension for a scripting engine
Previously multiple extensions could be provided, however, this can lead
to confusion with on-disk scripts (ie, "foo.js" and "foo.javascript")
having different content. Only a single extension is now supported.

The only language currently supporting multiple extensions was the
Javascript engine ("js" and "javascript"). It now only supports the
`.js` extension.

Relates to #10598
2016-05-13 09:54:31 -06:00
Lee Hinman d5b75491dc Merge remote-tracking branch 'dakrone/remove-script-sandbox' 2016-05-13 09:50:39 -06:00
Nik Everett 0a300320cd [test] Set logging to 11 on reindex test
It has failures we can't explain and we need logs to be able to do
anything useful with the failures:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+master+g1gc/359/consoleFull
2016-05-13 11:48:01 -04:00
Lee Hinman efff3918d8 Remove support for mulitple languages per scripting engine 2016-05-13 09:24:31 -06:00
Lee Hinman a4060f7436 Remove vestiges of script engine sandboxing
This removes all the mentions of the sandbox from the script engine
services and permissions model. This means that the following settings
are no longer supported:

```yaml
script.inline: sandbox
script.stored: sandbox
```

Instead, only a `true` or `false` value can be specified.

Since this would otherwise break the default-allow parameter for
languages like expressions, painless, and mustache, all script engines
have been updated to have individual settings, for instance:

```yaml
script.engine.groovy.inline: true
```

Would enable all inline scripts for groovy. (they can still be
overridden on a per-operation basis).

Expressions, Painless, and Mustache all default to `true` for inline,
file, and stored scripts to preserve the old scripting behavior.

Resolves #17114
2016-05-13 09:24:31 -06:00
Adrien Grand 638da06c1d Add back support for `ip` range aggregations. #17859
This commit adds support for range aggregations on `ip` fields. However it will
only work on 5.x indices.

Closes #17700
2016-05-13 17:22:01 +02:00
Robert Muir 520697eb14 Merge pull request #18323 from uschindler/dynamiccallsite_cleanup
painless: cleanup of DynamicCallSite
2016-05-13 11:19:27 -04:00
Uwe Schindler a1d0fb5008 painless: cleanup of DynamicCallSite to have all implementation encapsulated in inner class, outer class only has bootstrap method 2016-05-13 15:50:55 +02:00
Daniel Mitterdorfer ddbfda2c68 Exclude specific transport actions from request size limit check
We add support to explicitly exclude specific transport actions
from the request size limit check.

We also exclude the following request types currently:

*MasterPingRequest
* PingRequest
2016-05-13 14:21:24 +02:00
Jack Conradson 2e25837deb Made line number available in painless nodes for debugging. 2016-05-12 10:32:06 -07:00
Jack Conradson 28164ccf2e Decouple the ANTLR AST from Painless.
Closes #18286
2016-05-12 00:46:48 -07:00
Robert Muir 113e239e0f Merge pull request #18288 from rmuir/painless_debug
painless: add debugging tool to tests
2016-05-12 01:16:32 -04:00
Jack Conradson 8871fddba7 Merge pull request #18282 from jdconrad/longs
Long priority over Float
2016-05-11 21:08:28 -07:00
Robert Muir 930a215b5c add debugger 2016-05-11 23:56:19 -04:00
Robert Muir 25dd64250b painless: remove input, support params instead 2016-05-11 21:32:10 -04:00
Robert Muir c5532d3df0 add a rest test for this that seems to work, fix the documentation. thanks @s1monw 2016-05-11 16:07:08 -04:00
Robert Muir 6b4e47bf96 this makes aggregations per-document _value fast (bypass hash put, hash get, etc) for painless.
but i have no clue how to test it, it seems this feature never worked via REST?

Should we drop the feature instead?
2016-05-11 15:39:00 -04:00
Jack Conradson 20af482dd7 Removed extra branches. Only check for long first where we already know the
second type.
2016-05-11 10:22:40 -07:00
Jack Conradson 2764b47d22 Give long priority over float in Def math calcs. Promotion of types
still works correclty.
2016-05-11 09:32:30 -07:00
Robert Muir d790c76636 painless: _score as double, not float 2016-05-11 11:19:18 -04:00
Tanguy Leroux 6d288dec11 Clean up tests in Reindex module 2016-05-11 09:54:52 +02:00
Robert Muir 7689a1af28 Add 'ctx' keyword to painless. 2016-05-11 02:31:19 -04:00
Jack Conradson d8b373cac6 Fixed an issue related to bracket shortcuts. 2016-05-10 23:11:13 -07:00
Robert Muir 632bb4c720 add reserved word tests for 'doc', too 2016-05-11 00:17:19 -04:00
Robert Muir 389ef462c4 special access to doc 2016-05-10 23:43:57 -04:00
Robert Muir abb015bcb9 add tests for reserved words. ban stores against _score 2016-05-10 22:55:06 -04:00
Robert Muir a63285aaf9 painless: retrieve _score directly from Scorer 2016-05-10 20:53:08 -04:00
Robert Muir 1d80542199 Merge pull request #18247 from rmuir/needsScore
painless: implement needsScore() correctly.
2016-05-10 19:21:09 -04:00
Robert Muir ddc2c1f682 Merge pull request #18234 from rmuir/more_indy_type_data
painless: use better typing for dynamic method calls
2016-05-10 19:17:13 -04:00
Robert Muir 1c2d3b10ef some cleanups suggested by @uschindler and break some long lines 2016-05-10 12:31:02 -04:00
Robert Muir eb1b2cf111 add test and cleanup 2016-05-10 11:34:33 -04:00
Robert Muir 7a47d33a9b Merge branch 'master' into needsScore 2016-05-10 11:23:50 -04:00
Robert Muir 546aed8390 hack 2016-05-10 11:23:24 -04:00
Uwe Schindler 243c9e77fa painless: Add synthetic length property as alias to Lists, so they can really be used like arrays 2016-05-10 15:01:49 +02:00
Uwe Schindler 2e613f49b3 painless: Array load/store and length with invokedynamic 2016-05-10 12:41:27 +02:00
Robert Muir 3aeb66885b painless: use better typing for dynamic method calls 2016-05-10 05:44:16 -04:00
Robert Muir ba2fe156e8 Switch over dynamic method calls, loads and stores to invokedynamic.
Remove performance hack for accessing a document's fields, its not needed.
Add support for accessing is-getter methods like List.isEmpty() as .empty

Closes #18201
2016-05-09 21:44:32 -04:00
Tanguy Leroux 8c52e8814b Remove ReindexResponse in favor of BulkIndexByScrollResponse 2016-05-09 17:03:16 +02:00
Chris Earle 5be79ed02c Add Failure Details to every NodesResponse
Most of the current implementations of BaseNodesResponse (plural Nodes) ignore FailedNodeExceptions.

- This adds a helper function to do the grouping to TransportNodesAction
- Requires a non-null array of FailedNodeExceptions within the BaseNodesResponse constructor
- Reads/writes the array to output
- Also adds StreamInput and StreamOutput methods for generically reading and writing arrays
2016-05-06 14:59:43 -04:00
Adrien Grand 7d8708716e QueryBuilder does not need generics. #18133
QueryBuilder has generics, but those are never used: all call sites use
`QueryBuilder<?>`. Only `AbstractQueryBuilder` needs generics so that the base
class can contain a default implementation for setters that returns `this`.
2016-05-06 08:38:20 +02:00
Robert Muir e3ce6c9048 Painless: add fielddata accessors (.value/.values/.distance()/etc)
This gives better coverage and consistency with the scripting APIs, by
whitelisting the primary search scripting API classes and using them instead
of only Map and List methods.

For example, accessing fields can now be done with `.value` instead of `.0`
because `getValue()` is whitelisted. For now, access to a document's fields in
this way (loads) are fast-pathed in the code, to avoid dynamic overhead.

Access to geo fields and geo distance functions is now supported.

TODO: date support (e.g. whitelist ReadableDateTime methods as a start)
TODO: improve docs (like expressions and groovy have for document's fields)
TODO: remove fast-path hack

Closes #18169

Squashed commit of the following:

commit ec9f24b2424891a7429bb4c0a03f9868cba0a213
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:59:37 2016 -0400

    cutover to <Def> instead of <Object> here

commit 9edb1550438acd209733bc36f0d2e0aecf190ecb
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 17:03:02 2016 -0400

    add fast-path for docvalues field loads

commit f8e38c0932fccc0cfa217516130ad61522e59fe5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu May 5 16:47:31 2016 -0400

    Painless: add fielddata accessors (.value/.values/.distance()/etc)
2016-05-05 18:38:41 -04:00
Jack Conradson 2cae575f53 Added single-quoted strings.
Closes #18150
2016-05-05 09:26:02 -07:00
Robert Muir 59c135b58d make internal Def methods private and add basic javadocs 2016-05-05 03:47:56 -04:00
Robert Muir 928e2b904d painless: optimize/simplify dynamic field and method access 2016-05-05 03:42:14 -04:00
Jason Tedor 784c9e5fb9 Introduce node handshake
This commit introduces a handshake when initiating a light
connection. During this handshake, node information, cluster name, and
version are received from the target node of the connection. This
information can be used to immediately validate that the target node is
a member of the same cluster, and used to set the version on the
stream. This will allow us to extend APIs that are used during initial
cluster recovery without a major version change.

Relates #15971
2016-05-04 20:06:47 -04:00
Nik Everett 230697c202 [reindex] Switch throttle to Float.POSITIVE_INFITINTY/"unlimited"
All other values are errors.

Add java test for throttling. We had a REST test but it only ran against
one node so it didn't catch serialization errors.

Add Simple round trip test for rethrottle request
2016-05-04 16:14:32 -04:00
Christoph Büscher ca21aa0cb5 Make reset() in QueryShardContext private
The query shard reset() method resets some internal state in the
query shard context, like clearing query names, the filter flag
or named queries. The problem with this method being public is
that it currently (miss?) used for modifying an existing context
for recursive invocatiob, but the contexts that have been reseted
that way cannot be properly set back to their previous state.

This PR is a step towards removing reset() entirely by first making
it only be used internally in QueryShardContext. In places where
reset() was used we can either create new QueryShardContexts or
modify the existing context because it is discarded afterwards anyway.
2016-05-03 18:56:16 +02:00
Robert Muir fff82db681 Add tests/doc for boolean fields with expressions 2016-05-02 18:13:03 -04:00
Robert Muir 693c1f6671 Support geo_point fields in lucene expressions.
Closes #18096
2016-05-02 17:49:21 -04:00
Robert Muir 28409e4509 Add support for .empty to expressions, and some docs improvements
Closes #18077
2016-05-02 09:07:25 -04:00
Martijn van Groningen 7aca1389e2 ingest: Add `date_index_name` processor.
Closes #17814
2016-04-29 17:20:48 +02:00
Ryan Ernst d12a4bb51d Merge pull request #17933 from rjernst/camelcase4
Remove camelCase support
2016-04-22 13:46:43 -07:00
Nik Everett cc1a55423c Reindex: properly mark things as child tasks
Do this by creating a Client subclass that automatically assigns the
parentTask to all requests that come through it. Code that doesn't want
to set the parentTask can call `unwrap` on the Client to get the inner
client instance that doesn't set the parentTask. Reindex uses this for
its ClearScrollRequest so that the request will run properly after the
reindex request has been canceled.
2016-04-22 14:00:11 -04:00
Ryan Ernst 55388590c1 Remove camelCase support
Now that the current uses of magical camelCase support have been
deprecated, we can remove these in master (sans remaining issues like
BulkRequest). This change removes camel case support from ParseField,
query types, analysis, and settings lookup.

see #8988
2016-04-22 09:18:10 -07:00
Nik Everett 51621f9d75 Remove ChildTaskRequest and always pass parentTaskId when building a task
Passing parentTaskId forces the caller to handle the parentTaskId.
2016-04-22 11:26:18 -04:00
Nik Everett ffeb5e38fc Remove parent-less task methods
Callers should explicitly handle parents - either using EMPTY_TASK_ID when
a parent isn't possible or piping parents from the TransportRequest when
possible.
2016-04-22 11:26:18 -04:00
Martijn van Groningen c5ad2e2865 Changed indexed scripts to be stored in the cluster state instead of the `.scripts` index.
Also added max script size soft limit for stored scripts.

Closes #16651
2016-04-22 13:42:55 +02:00
Isabel Drost-Fromm 233fe86ee4 Makes Script type writeable
Used to be Streamable. Left-over of the PROTOTYPE related refactoring by
@nik9000

Closes #17753
2016-04-21 12:40:29 +02:00
Christoph Büscher e06e122f9f Wrap xcontent parser creation in try-with-resource statement where possible 2016-04-18 16:13:56 +02:00
Christoph Büscher cdb36a2b0c Merge pull request #17417
Clean up QueryParseContext and don't hold it inside QueryRewrite/ShardContext
2016-04-18 15:13:53 +02:00
Nik Everett d3b1306069 Reindex: never report negative throttled_until
Just clamp the value at 0. It isn't useful to tell the user "this
thread should have woken 5ms ago".

Closes #17783
2016-04-15 16:53:23 -04:00
Christoph Büscher fbd558382d Clean up QueryParseContext and don't hold it inside QueryRewrite/ShardContext
This change cleans up a few things in QueryParseContext and QueryShardContext
that make it hard to reason about the state of these context objects and are
thus error prone and should be simplified.

Currently the parser that used in QueryParseContext can be set and reset any
time from the outside, which makes reasoning about it hard. This change makes
the parser a mandatory constructor argument removes ability to later set a
different ParseFieldMatcher. If none is provided at construction time, the
one set inside the parser is used. If a ParseFieldMatcher is specified at
construction time, it is implicitely set on the parser that is beeing used.
The ParseFieldMatcher is only kept inside the parser.

Also currently the QueryShardContext historically holds an inner QueryParseContext
(in the super class QueryRewriteContext), that is mainly used to hold the parser
and parseFieldMatcher. For that reason, the parser can also be reset, which leads
to the same problems as above. This change removes the QueryParseContext from
QueryRewriteContext and removes the ability to reset or retrieve it from the
QueryShardContext. Instead, `QueryRewriteContext#newParseContext(parser)` can be
used to create new parse contexts with the given parser from a shard context
when needed.
2016-04-15 17:13:01 +02:00
Christoph Büscher de036d63d8 Rename context.parseFieldMatcher() to context.getParseFieldMatcher 2016-04-15 15:15:32 +02:00
Christoph Büscher 15c59d07b3 Remove ParseFieldMatcher from AbstractXContentParser
Currently we are able to set a ParseFieldMatcher on XContentParsers,
mainly to conveniently carry it around to be available where the
actual parsing happens. This was just recently introduced together
with ObjectParser so that ObjectParser can make use of deprecation
logging and throwing errors while parsing.

This however is trappy because we create parsers in so many places in
the code and it is easy to forget setting the right ParseFieldMatcher.
Instead we should hold the ParseFieldMatcher only in the parse contexts
(e.g. QueryParseContext).

This PR removes the ParseFieldMatcher from XContentParser. ObjectParser
can still make use of it because we can make the otherwise unbounded
`context` type to extend an interface that makes sure contexts used in
ObjectParser can supply a ParseFieldMatcher. Contexts in ObjectParser
are now no longer optional, but it is sufficient to pass in a small
lambda expression in places where no other context is available.

Relates to #17417
2016-04-15 15:14:46 +02:00
Adrien Grand d84c643f58 Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.

Notes about how the change works:
 - Numeric mappers have been split into a legacy version that is essentially
   the current mapper, and a new version that uses points, eg.
   LegacyDateFieldMapper and DateFieldMapper.
 - Since new and old fields have the same names, the decision about which one
   to use is made based on the index creation version.
 - If you try to force using a legacy field on a new index or a field that uses
   points on an old index, you will get an exception.
 - IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
   in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
   and sortable).
 - The internal MappedFieldType that is stored by the new mappers does not have
   any of the points-related properties set. Instead, it keeps setting the index
   options when parsing the `index` property of mappings and does
   `if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
   when parsing documents.

Known issues that won't fix:
 - You can't use numeric fields in significant terms aggregations anymore since
   this requires document frequencies, which points do not record.
 - Term queries on numeric fields will now return constant scores instead of
   giving better scores to the rare values.

Known issues that we could work around (in follow-up PRs, this one is too large
already):
 - Range queries on `ip` addresses only work if both the lower and upper bounds
   are inclusive (exclusive bounds are not exposed in Lucene). We could either
   decide to implement it, or drop range support entirely and tell users to
   query subnets using the CIDR notation instead.
 - Since IP addresses now use a different representation for doc values,
   aggregations will fail when running a terms aggregation on an ip field on a
   list of indices that contains both pre-5.0 and 5.0 indices.
 - The ip range aggregation does not work on the new ip field. We need to either
   implement range aggs for SORTED_SET doc values or drop support for ip ranges
   and tell users to use filters instead. #17700

Closes #16751
Closes #17007
Closes #11513
2016-04-14 17:56:23 +02:00
Christoph Büscher e15e7f7e6e Remove parser argument from methods where we already pass in a parse context
When we pass down both parser and QueryParseContext to a method, we cannot
make sure that the parser contained in the context and the parser that is
parsed as an argument have the same state. This removes the parser argument
from methods where we currently have both the parser and the parse context
as arguments and instead retrieves the parse from the context inside the
method.
2016-04-14 16:18:05 +02:00
Martijn van Groningen 2928fd6ef3 Cleanup query builder for inner hits construction.
* Inner hits can now only be provided and prepared via setter in the nested, has_child and has_parent query.
* Also made `score_mode` a required constructor parameter.
* Moved has_child's min_child/max_children validation from doToQuery(...) to a setter.
2016-04-14 14:43:21 +02:00
Nik Everett cca3154c43 Rename isSourceEmpty to hasSource
And add a test case for {} to reindex.
2016-04-13 08:19:58 -04:00
Nik Everett c2e745bf3b reindex: Guard against user disabling fields 2016-04-13 08:19:58 -04:00
Nik Everett 0f9804b0e2 reindex: gracefully handle when _source is disabled
Closes #17666
2016-04-13 08:19:58 -04:00
Adrien Grand 013acf9179 Remove MappedFieldType.value. #17557
This commit removes `MappedFieldType.value` and simplifies
`MappedFieldType.valueforSearch`. `valueforSearch` was used to post-process
values that come for stored fields (eg. to convert a long back to a string
representation of a date in the case of a date field) and also values that
are extracted from the source but only in the case of GET calls: it would
not be called when performing source filtering on search requests.

`valueforSearch` is now only called for stored fields, since values that are
extracted from the source should already be formatted as expected.
2016-04-12 09:12:56 +02:00
Adrien Grand a14db8e17e Remove MappedFieldType.useTermQueryWithQueryString() and isNumeric(). #17599
In both cases, what elasticsearch is really interested in is whether the field
is an analyzed string field. So it can just check `tokenized()` instead.
2016-04-12 08:45:28 +02:00
Adrien Grand 496c7fbd84 Upgrade Lucene 6 Release
* upgrades numerics to new Point format
* updates geo api changes
  * adds GeoPointDistanceRangeQuery as XGeoPointDistanceRangeQuery
  * cuts over to ES GeoHashUtils
2016-04-11 16:50:04 -05:00
Adrien Grand 0eb1a816c8 Allow the query cache to be disabled. #16268
This replaces the internal `index.queries.cache.type` setting with
a new `index.queries.cache.enabled` setting, which is documented.

Closes #15802
2016-04-11 18:06:16 +02:00
Adrien Grand 42526ac28e Remove Settings.settingsBuilder.
We have both `Settings.settingsBuilder` and `Settings.builder` that do exactly
the same thing, so we should keep only one. I kept `Settings.builder` since it
has my preference but also it is the one that we use in examples of the Java API.
2016-04-08 18:10:02 +02:00
Adrien Grand bef38a4d12 Fix test bug. 2016-04-07 09:49:27 +02:00
Adrien Grand e1bfe23c22 ExtendedStatsAggregator should also pass sigma to emtpy aggs. #17388
Because sigma is also used at reduce time, it should be passed to empty aggs.
Otherwise it causes bugs when an empty aggregation is used to perform reduction
is it would assume a sigma of zero.

Closes #17362
2016-04-07 09:34:11 +02:00
Nik Everett 16c12afabe Rework ScoreFunctionBuilder registration to remove PROTOTYPEs
This removes PROTOTYPEs from ScoreFunctionsBuilders. To do so we rework
registration so it doesn't need PROTOTYPEs and lines up with the recent
changes to query registration.
2016-04-06 13:04:11 -04:00
Nik Everett 2b6866d26b Fix references to the removed parsers
Mostly stuff is just in the builder now.
2016-04-06 11:15:22 -04:00
Adrien Grand 4c4bbb3e45 Replace FieldStatsProvider with a method on MappedFieldType. #17334
FieldStatsProvider had to perform instanceof calls to properly handle dates or
ip addresses. By moving the logic to MappedFieldType, each field type can check
whether all values are within bounds its way.

Note that this commit only keeps rewriting support for dates, which are the only
field for which the rewriting mechanism is likely to help (because of time-based
indices).
2016-04-01 10:28:58 +02:00
Nik Everett 14d37baa4b [reindex] Don't get rejected
BulkByScrollTaskTest#testDelayAndRethrottle was getting rejected exceptions
every once in a while. This was reproducible ~20% of the time for me. I
added a CyclicBarrier to prevent the test from shutting down the thread pool
before the threads get finished.
2016-03-31 14:50:14 -04:00
Nik Everett 0c762fca35 Fix test mistake 2016-03-31 12:27:35 -04:00
Nik Everett 7f794e7b77 Test for invalid scroll_size 2016-03-31 12:21:32 -04:00
Nik Everett 30a1862339 Remove PROTOTYPE from BulkItemResponse.Failure
Closes #17086
2016-03-31 09:10:36 -04:00
javanna 32b6e529f4 Merge branch 'master' into enhancement/discovery_node_one_getter 2016-03-31 10:49:26 +02:00
Jack Conradson a37e53c50f Painless clean up including fixing _score issues and improving type
error messages.

Closes #17428
2016-03-30 16:40:17 -07:00
Nik Everett 78ab6c5b7f [reindex] Dynamic throttle!
This allows the user to update the reindex throttle on the fly, with changes
that speed up the throttling being applied immediately and changes that
slow down the throttling being applied during the next batch. This means
that if a user throttles reindex in such a way that it tries to sleep for
16 years and then realizes that they've done something wrong then they
can change the throttle and reindex will wake up again. We don't apply
slow downs immediately so we never get in danger of losing the scan context.

Also, if reindex is canceled while it is sleeping (how it honor throttling)
then it'll immediately wake up and cancel itself.
2016-03-30 16:40:42 -04:00
javanna b9f9b2e3ee Merge branch 'master' into enhancement/discovery_node_one_getter 2016-03-30 17:22:40 +02:00
Nik Everett 1c16d63a9a Merge pull request #17394 from camilojd/refactor/replace-getrandom
Refactor: replace all ocurrences of ESTestCase.getRandom() with LuceneTestCase.random()
2016-03-30 08:58:21 -04:00
javanna a8bbdff3bc Remove DiscoveryNode#name in favour of existing DiscoveryNode#getName 2016-03-30 14:47:36 +02:00
Adrien Grand 068c788ec8 Disable fielddata on text fields by defaults. #17386
`text` fields will have fielddata disabled by default. Fielddata can still be
enabled on an existing index by setting `fielddata=true` in the mappings.
2016-03-30 14:35:32 +02:00
Camilo Diaz Repka 7be11a36cd Refactor: replace all ocurrences of ESTestCase.getRandom() for random().
Remove getRandom().
2016-03-29 23:18:05 -04:00
Nik Everett df08854c60 Remove PROTOTYPEs from suggesters
Also stops using guice for suggesters at all and lots of checkstyle.
2016-03-29 17:55:01 -04:00
javanna 061f09d9a4 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 20:19:33 +02:00
Tal Levy 16e888fac3 Merge pull request #17260 from talevy/fix-regex-exceptions
Handle regex parsing errors in Gsub and Grok Processors
2016-03-29 08:12:26 -07:00
Clinton Gormley 3087d2b882 Fixed bad YAML in reindex REST test: 50_routing.yaml 2016-03-29 15:03:09 +02:00
Clinton Gormley 52daed0732 Update-by-query rest tests: fixed bad yaml and deleted a client-dependent test 2016-03-29 14:58:29 +02:00
Colin Goodheart-Smithe ff3fd99074 Prevents exception being raised when ordering by an aggregation which wasn't collected
If a terms aggregation was ordered by a metric nested in a single bucket aggregator which did not collect any documents (e.g. a filters aggregation which did not match in that term bucket) an ArrayOutOfBoundsException would be thrown when the ordering code tried to retrieve the value for the metric. This fix fixes all numeric metric aggregators so they return their default value when a bucket ordinal is requested which was not collected.

Closes #17225
2016-03-29 13:28:03 +01:00
javanna 8fc9dbbb99 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 14:27:04 +02:00
Clinton Gormley 5f24581de3 The reindex body is now required, which changes the exception thrown by the REST test 2016-03-29 14:09:59 +02:00
Clinton Gormley b87beeb05f Rename update-by-query REST tests to update_by_query 2016-03-29 13:13:49 +02:00
Clinton Gormley 97606850e8 Renamed update-by-query REST spec to update_by_query 2016-03-29 11:45:20 +02:00
javanna de5cbda8e7 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 10:48:47 +02:00
Nik Everett 0e6141e675 Replace is_true: took with took >= 0
This prevents tests from failing on machines that can finish the request
less than half a millisecond.
2016-03-28 13:03:48 -04:00
Boaz Leskes 91021e3019 merge from master 2016-03-25 15:50:48 +01:00
javanna 27d4994aff Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-24 18:10:11 +01:00
Nik Everett 93ab4cfc99 Stop using PROTOTYPE in NamedWriteableRegistry
readFrom is confusing because it requires an instance of the type that it
is reading but it doesn't modify it. But we also have (deprecated) methods
named readFrom that *do* modify the instance. The "right" way to implement
the non-modifying readFrom is to delegate to a constructor that takes a
StreamInput so that the read object can be immutable. Now that we have
`@FunctionalInterface`s it is fairly easy to register things by referring
directly to the constructor.

This change modifying NamedWriteableRegistry so that it does that. It keeps
supporting `registerPrototype` which registers objects to be read by
readFrom but deprecates it and delegates it to a new `register` method
that allows passing a simple functional interface. It also cuts Task.Status
subclasses over to using that method.

The start of #17085
2016-03-24 11:26:44 -04:00
Nik Everett 48aaebf23d [reindex] Wait for headers
The test was checking that we'd set the headers properly but in some cases
the request had yet to come in because it was running on another thread.
Now we wait for the headers to show up before failing the test.

Closes #17299
2016-03-24 09:55:49 -04:00
Jim Ferenczi da42f199bd Enforce isolated mode for all plugins
This commit removes the isolated option, each plugin have its own classloader.
2016-03-24 09:17:33 +01:00
Nik Everett aaa4d57fff [reindex] Don't attempt to refresh on noop
If the user asks for a refresh but their reindex or update-by-query
operation touched no indexes we should just skip the resfresh call
entirely. Without this commit we refresh *all* indexes which is totally
wrong.

Closes #17296
2016-03-23 18:12:40 -04:00
Areek Zillur ed49ec437f remove suggest transport action 2016-03-23 16:37:56 -04:00
javanna 030453d320 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-23 11:25:34 +01:00
Adrien Grand e50eeeaffb Refactor fielddata mappings. #17148
The fielddata settings in mappings have been refatored so that:
 - text and string have a `fielddata` (boolean) setting that tells whether it
   is ok to load in-memory fielddata. It is true by default for now but the
   plan is to make it default to false for text fields.
 - text and string have a `fielddata_frequency_filter` which contains the same
   thing as `fielddata.filter.frequency` used to (but validated at parsing time
   instead of being unchecked settings)
 - regex fielddata filtering is not supported anymore and will be dropped from
   mappings automatically on upgrade.
 - text, string and _parent fields have an `eager_global_ordinals` (boolean)
   setting that tells whether to load global ordinals eagerly on refresh.
 - in-memory fielddata is not supported on keyword fields anymore at all.
 - the `fielddata` setting is not supported on other fields that text and string
   and will be dropped when upgrading if specified.
2016-03-23 09:48:13 +01:00
Tal Levy 534caa8927 Handle regex parsing errors in Gsub and Grok Processors
Currently, both Gsub and Grok parse regex strings during
Pipeline creation. Thrown parsing exceptions were leaking out, this
commit wraps those exceptions in ElasticsearchParseExceptions.
2016-03-22 15:06:29 -07:00
Boaz Leskes 7c8cdf4a71 merged from master 2016-03-22 19:21:28 +01:00
Nik Everett da96b6e41d [reindex] Add thottling support
The throttle is applied when starting the next scroll request so that its
timeout can include the throttle time.
2016-03-22 12:34:14 -04:00
Boaz Leskes 39ae16bc4c merge from master 2016-03-22 11:46:26 +01:00
javanna eebd0cfccd Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-22 10:34:40 +01:00
Simon Willnauer 47f0e6e8f4 [TEST] Disable InternalClusterInfoService in messy tests, it sends IndicesStatsRequest periodically which messes with the messy test 2016-03-22 10:12:03 +01:00
javanna bf390a935e Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-21 17:18:23 +01:00
Boaz Leskes 2d1152ebac Remove ClusterService interface, in favor of it's only production instance #17183
We current have a ClusterService interface, implemented by InternalClusterService and a couple of test classes. Since the decoupling of the transport service and the cluster service, one can construct a ClusterService fairly easily, so we don't need this extra indirection.

Closes #17183
2016-03-21 13:55:10 +01:00
Martijn van Groningen e3b7e5d75a percolator: Replace percolate api with the new percolator query
Also replaced the PercolatorQueryRegistry with the new PercolatorQueryCache.

The PercolatorFieldMapper stores the rewritten form of each percolator query's xcontext
in a binary doc values field. This make sure that the query rewrite happens only during
indexing (some queries for example fetch shapes, terms in remote indices) and
the speed up the loading of the queries in the percolator query cache.

Because the percolator now works inside the search infrastructure a number of features
(sorting fields, pagination, fetch features) are available out of the box.

The following feature requests are automatically implemented via this refactoring:

Closes #10741
Closes #7297
Closes #13176
Closes #13978
Closes #11264
Closes #10741
Closes #4317
2016-03-21 12:21:50 +01:00
Boaz Leskes 858610d0d1 merge from master 2016-03-19 13:57:40 +01:00
Ryan Ernst f71f0d6010 Revert "Build: Switch to maven-publish plugin"
This reverts commit a90a2b34fc.
2016-03-18 17:22:25 -07:00
Ryan Ernst 6af4c43c4f Merge pull request #17128 from rjernst/maven_publish
Build: Switch to maven-publish plugin
2016-03-17 11:53:50 -07:00
Jack Conradson 800c844ebd Split up Analyzer and Writer into multiple pieces.
Closes #17158
2016-03-17 10:15:13 -07:00
Simon Willnauer e91a141233 Prevent index level setting from being configured on a node level
Today we allow to set all kinds of index level settings on the node level which
is error prone and difficult to get right in a consistent manner.
For instance if some analyzers are setup in a yaml config file some nodes might
not have these analyzers and then index creation fails.

Nevertheless, this change allows some selected settings to be specified on a node level
for instance:
 * `index.codec` which is used in a hot/cold node architecture and it's value is really per node or per index
 * `index.store.fs.fs_lock` which is also dependent on the filesystem a node uses

All other index level setting must be specified on the index level. For existing clusters the index must be closed
and all settings must be updated via the API on each of the indices.

Closes #16799
2016-03-17 14:42:18 +01:00
Christoph Büscher 6ddf9ae92f Merge branch 'master' into feature-suggest-refactoring 2016-03-16 15:27:02 +01:00
Nik Everett 7197172047 [reindex] Properly register status
Without this commit fetching the status of a reindex from a node that isn't
coordinating the reindex will fail. This commit properly registers reindex's
status so this doesn't happen. To do so it moves all task status registration
into NetworkModule and creates a method to register other statuses which the
reindex plugin calls.
2016-03-16 07:40:49 -04:00
Christoph Büscher 39667b5793 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0/java.asciidoc
2016-03-16 12:06:42 +01:00
Ryan Ernst a90a2b34fc Build: Switch to maven-publish plugin
The build currently uses the old maven support in gradle. This commit
switches to use the newer maven-publish plugin. This will allow future
changes, for example, easily publishing to artifactory.

An additional part of this change makes publishing of build-tools part
of the normal publishing, instead of requiring a separate upload step
from within buildSrc. That also sets us up for a follow up to enable
precomit checks on the buildSrc code itself.
2016-03-15 19:16:37 -07:00
Jason Tedor 618441aea3 Merge pull request #17088 from jasontedor/simplify-bootstrap-settings
Bootstrap does not set system properties
2016-03-15 19:25:16 -04:00
Jason Tedor 66ba044ec5 Use setting in integration test cluster config 2016-03-15 17:45:17 -04:00
Yannick Welsch 7fb07a9a53 Merge pull request #17112 from ywelsch/enhance/forbid-sysout-for-tests
Forbid test sources to use System.out.println and Throwable.printStackTrace
2016-03-15 16:36:34 +01:00
Yannick Welsch f5e6db4090 Remove System.out.println and Throwable.printStackTrace from tests 2016-03-15 15:40:37 +01:00
Christoph Büscher b4b874f0d8 Merge branch 'master' into feature-suggest-refactoring 2016-03-15 12:11:39 +01:00
Christoph Büscher 1f1f6861b7 Merge branch 'master' into sort-serialization-scriptsort
Conflicts:
	core/src/test/java/org/elasticsearch/search/sort/AbstractSortTestCase.java
2016-03-15 11:03:28 +01:00
Simon Willnauer 345e988bbc Merge pull request #17072 from s1monw/add_backwards_rest_tests
Add infrastructure to run REST tests on a multi-version cluster

This change adds the infrastructure to run the rest tests on a multi-node
cluster that users 2 different minor versions of elasticsearch. It doesn't implement
any dedicated BWC tests but rather leverages the existing REST tests.

Since we don't have a real version to test against, the tests uses the current version
until the first minor / RC is released to ensure the infrastructure works.

Given the amount of problems this change already found I think it's worth having this run with our test suite by default. The structure of this infra will likely change over time but for now it's a step into the right direction. We will likely want to split it up into integTests and integBwcTests etc. so each plugin can have it's own bwc tests but that's left for future refactoring.
2016-03-15 09:17:43 +01:00
Ali Beyad f527a034a3 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-03-14 17:25:02 -04:00
Jack Conradson dfec4547ea Added one minor comment for expressions tests. 2016-03-14 13:19:52 -07:00
Simon Willnauer bd96075f7f Merge branch 'master' into add_backwards_rest_tests 2016-03-14 21:18:42 +01:00
Jack Conradson 7665c4fe6f Merge branch 'master' into pr/17091 2016-03-14 12:16:21 -07:00
Christoph Büscher 40f3501d7f Merge branch 'master' into feature-suggest-refactoring 2016-03-14 14:57:57 +01:00
Christoph Büscher f0074668db Merge branch 'master' into sort-serialization-scriptsort 2016-03-14 12:15:53 +01:00
Christoph Büscher 97638c95fc Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	docs/reference/migration/migrate_5_0.asciidoc
2016-03-14 11:13:47 +01:00
Simon Willnauer 31740e279f Resolve index names to Index instances early
Today index names are often resolved lazily, only when they are really
needed. This can be problematic especially when it gets to mapping updates
etc. when a node sends a mapping update to the master but while the request
is in-flight the index changes for whatever reason we would still apply the update
since we use the name of the index to identify the index in the clusterstate.
The problem is that index names can be reused which happens in practice and sometimes
even in a automated way rendering this problem as realistic.
In this change we resolve the index including it's UUID as early as possible in places
where changes to the clusterstate are possible. For instance mapping updates on a node use a
concrete index rather than it's name and the master will fail the mapping update iff
the index can't be found by it's <name, uuid> tuple.

Closes #17048
2016-03-14 11:08:48 +01:00
Alexander Kazakov 8e6b2b3909 Check that _value is used in aggregations script before setting value to specialValue #14262 2016-03-14 12:04:06 +03:00
Adrien Grand 5596e31068 Upgrade to lucene-6.0.0-f0aa4fc. #17075 2016-03-14 07:58:52 +01:00
Jason Tedor 8a05c2a2be Bootstrap does not set system properties
Today, certain bootstrap properties are set and read via system
properties. This action-at-distance way of managing these properties is
rather confusing, and completely unnecessary. But another problem exists
with setting these as system properties. Namely, these system properties
are interpreted as Elasticsearch settings, not all of which are
registered. This leads to Elasticsearch failing to startup if any of
these special properties are set. Instead, these properties should be
kept as local as possible, and passed around as method parameters where
needed. This eliminates the action-at-distance way of handling these
properties, and eliminates the need to register these non-setting
properties. This commit does exactly that.

Additionally, today we use the "-D" command line flag to set the
properties, but this is confusing because "-D" is a special flag to the
JVM for setting system properties. This creates confusion because some
"-D" properties should be passed via arguments to the JVM (so via
ES_JAVA_OPTS), and some should be passed as arguments to
Elasticsearch. This commit changes the "-D" flag for Elasticsearch
settings to "-E".
2016-03-13 20:09:15 -04:00
Simon Willnauer 121e7c8ca4 Add infrastructure to run REST tests on a multi-version cluster
This change adds the infrastructure to run the rest tests on a multi-node
cluster that users 2 different minor versions of elasticsearch. It doesn't implement
any dedicated BWC tests but rather leverages the existing REST tests.

Since we don't have a real version to test against, the tests uses the current version
until the first minor / RC is released to ensure the infrastructure works.

Relates to #14406
Closes #17072
2016-03-13 10:52:39 +01:00
Ryan Ernst 5bd7da5659 Addressed PR feedback
* Fix tests still referring to -E
* add comment about missing classes
* rename writer constant
2016-03-11 11:46:23 -08:00
Ryan Ernst 591fb8f028 Merge branch 'master' into cli-parsing 2016-03-11 10:45:05 -08:00
Ali Beyad 31dcb3e18b Merge pull request #16873 from abeyad/suggester-wiring-refactoring
Change internal representation of suggesters
2016-03-11 12:34:35 -05:00
Christoph Büscher bbcbba1bf5 Fixing some tests and compile problems in reindex module 2016-03-11 17:58:13 +01:00
Christoph Büscher 5107388fe9 Added enum for script sort type 2016-03-11 17:32:39 +01:00
Christoph Büscher daeffb149c Merge branch 'master' into feature-suggest-refactoring 2016-03-11 10:37:28 +01:00
Yannick Welsch 04e55ecf6b Make logging message String constant to allow static checks 2016-03-11 10:30:59 +01:00
Yannick Welsch 718876a941 Fix wrong placeholder usage in logging statements 2016-03-11 10:30:59 +01:00
Nik Everett ebc12690bc [reindex] Move refresh tests to unit test
The refresh tests were failing rarely due to refreshes happening
automatically on indexes with -1 refresh intervals. This commit moves
the refresh test into a unit test where we can check if it was attempted
so we never get false failures from background refreshes.

It also stopped refresh from being run if the reindex request was canceled.
2016-03-10 17:48:22 -05:00
Nik Everett b2eec96045 [reindex] Make search failure cause rest failure
Indexing failures have caused the reindex http request to fail for a while
now. Both search and indexing failures cause it to abort. But search
failures didn't cause a non-200 response code from the http api. This
fixes that.

Also slips in a fix to some infrequently failing rest tests.

Closes #16037
2016-03-10 13:47:49 -05:00
Nik Everett b8d931d23c [reindex] Timeout if sub-requests timeout
Sadly, it isn't easy to simulate a timeout during an integration test, you
just have to cause one. Groovy's sleep should do the job.
2016-03-10 13:05:23 -05:00
Boaz Leskes 838c7ddd82 fix indexing compilation issue 2016-03-10 12:12:14 +01:00
Christoph Büscher 69c83b3459 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/action/suggest/TransportSuggestAction.java
        core/src/main/java/org/elasticsearch/search/suggest/SuggestParseElement.java
	core/src/main/java/org/elasticsearch/search/suggest/SuggestionSearchContext.java
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggester.java
2016-03-10 11:44:36 +01:00
Nik Everett 38241a5d8b [reindex] Implement CompositeIndicesRequest
Implements CompositeIndicesRequest on UpdateByQueryRequest and
ReindexRequest so that plugins can reason about the request. In both cases
this implementation is imperfect but useful because instead of listing
all requests that make up the request it instead attempts to make dummy
requests that represent the requests that it will later make.
2016-03-09 16:29:23 -05:00
Simon Willnauer 016bc3ea2b Merge pull request #17001 from s1monw/cut_over_to_uuid
Use index UUID to lookup indices on IndicesService

Today we use the index name to lookup index instances on the IndicesService
which applied to search requests but also to index deletion etc. This commit
moves the interface to expect an Index instance which is a tuple
and looks up the index by uuid rather than by name. This prevents accidental modification
of the wrong index if and index is recreated or searching from the wrong index in such a case.
Accessing an index that has the same name but different UUID will now result in an IndexNotFoundException.
2016-03-09 19:44:36 +01:00
Simon Willnauer e72dac91b3 Use index UUID to lookup indices on IndicesService
Today we use the index name to lookup index instances on the IndicesService
which applied to search reqeusts but also to index deletion etc. This commit
moves the interface to expcet and `Index` instance which is a <name, uuid> tuple
and looks up the index by uuid rather than by name. This prevents accidential modificaiton
of the wrong index if and index is recreated or searching from the _wrong_ index in such a case.
Accessing an index that has the same name but different UUID will now result in an IndexNotFoundException.

Closes #17001
2016-03-09 19:42:15 +01:00
Ryan Ernst 1dafead2eb Fix precommit 2016-03-08 22:55:24 -08:00
Nik Everett 6d0efae713 Teach list tasks api to wait for tasks to finish
_wait_for_completion defaults to false. If set to true then the API will
wait for all the tasks that it finds to stop running before returning. You
can use the timeout parameter to prevent it from waiting forever. If you
don't set a timeout parameter it'll default to 30 seconds.

Also adds a log message to rest tests if any tasks overrun the test. This
is just a log (instead of failing the test) because lots of tasks are run
by the cluster on its own and they shouldn't cause the test to fail. Things
like fetching disk usage from the other nodes, for example.

Switches the request to getter/setter style methods as we're going that
way in the Elasticsearch code base. Reindex is all getter/setter style.

Closes #16906
2016-03-08 11:53:57 -05:00
Christoph Büscher 1264f37a1b Merge branch 'master' into feature-suggest-refactoring 2016-03-08 11:10:08 +01:00
Christoph Büscher 5ff413074a Adding tests for `time_zone` parameter for date range aggregation 2016-03-07 15:38:24 +01:00
Christoph Büscher 7ec5075a87 Merge branch 'master' into feature-suggest-refactoring 2016-03-07 10:45:43 +01:00
Adrien Grand c69fc008d4 Merge pull request #16972 from alexshadow007/fix-16812
Build empty extended stats aggregation if no docs collected for bucket
2016-03-07 10:44:20 +01:00
Robert Muir 54018a5d37 upgrade to lucene 6.0.0-snapshot-bea235f
Closes #16964

Squashed commit of the following:

commit a23f9d2d29220991aa498214530753d7a5a148c6
Merge: eec9c4e 0b0a251
Author: Robert Muir <rmuir@apache.org>
Date:   Mon Mar 7 04:12:02 2016 -0500

    Merge branch 'master' into lucene6

commit eec9c4e5cd11e9c3e0b426f04894bb2a6dae4f21
Merge: bc67205 675d940
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 13:45:00 2016 -0500

    Merge branch 'master' into lucene6

commit bc67205bdfe1526eae277ab7856fc050ecbdb7b2
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 09:56:31 2016 -0500

    fix test bug

commit a60723b007ff12d97b1810cef473bd7b553a0327
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:35:35 2016 +0100

    Fix SimpleValidateQueryIT to put braces around boosted terms

commit ae3a49d7ba7ced448d2a5262e5d8ec98671a9090
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:27:25 2016 +0100

    fix multimatchquery

commit ae23fdb88a8f6d3fb7ba60fd1aaf3fd72d899aa5
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 15:20:49 2016 +0100

    Rewrite DecayFunctionScoreIT to be independent of the similarity used

    This test relied a lot on the term scoring and compared scores
    that are dependent on the similarity. This commit changes the base query
    to be a predictable constant score query.

commit 366c2d518c35d31251033f1b6f6a93f6e2ae327d
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 14:06:14 2016 +0100

    Fix scoring in tests due to changes to idf calculation.

    Lucene 6 uses a different default similarity as well as a different
    way to calculate IDF. In contrast to older version lucene 6 uses docCount per field
    to calculate the IDF not the # of docs in the index to overcome the sparse field
    cases.

commit dac99fd64ac2fa71b8d8d106fe68825e574c49f8
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 08:21:57 2016 -0500

    don't hardcoded expected termquery score

commit 6e9f340ba49ab10eed512df86d52a121aa775b0f
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 08:04:45 2016 -0500

    suppress deprecation warning until migrated to points

commit 3ac8908424b3fdad44a90a4f7bdb3eff7efd077d
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:21:43 2016 -0500

    Remove invalid test: all commits have IDs, and its illegal to do this.

commit c12976288124ad1a26467e7e848fb810548e7eab
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:06:14 2016 -0500

    don't test with unsupported back compat

commit 18bbfe76128570bc70883bf91ff4c44c82d27817
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 07:02:18 2016 -0500

    remove now invalid lucene 4 backcompat test

commit 7e730e572886f0ef2d3faba712e4256216ff01ec
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:58:52 2016 -0500

    remove now invalid lucene 4 backwards test

commit 244d2ab6868ba5ac9e0bcde3c2833743751a25ec
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:47:23 2016 -0500

    use 6.0 codec

commit 5f64d4a431a6fdaa1234adca23f154c2a1de8284
Author: Robert Muir <rmuir@apache.org>
Date:   Fri Mar 4 06:43:08 2016 -0500

    compile, javadocs, forbidden-apis, etc

commit 1f273cd62a7fe9ca8f8944acbbfc5cbdd3d81ccb
Merge: cd33921 29e3443
Author: Simon Willnauer <simonw@apache.org>
Date:   Fri Mar 4 10:45:29 2016 +0100

    Merge branch 'master' into lucene6

commit cd33921ac742ef9fb351012eff35f3c7dbda7264
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:58:37 2016 -0500

    fix hunspell dictionary loading

commit c7fdbd837b01f7defe9cb1c24e2ec65604b0dc96
Merge: 4d4190f d8948ba
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:41:53 2016 -0500

    Merge branch 'master' into lucene6

commit 4d4190fd82601aaafac6b8254ccb3edf218faa34
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:39:14 2016 -0500

    remove nocommit

commit 77ca69e288b1a41aa9595c921ed166c272a00ea8
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:38:24 2016 -0500

    clean up numericutils vs legacynumericutils

commit a466d696fbaad04b647ffbc0857a9439b583d0bf
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:32:43 2016 -0500

    upgrade spatial4j

commit 5412c747a8cfe638bacedbc8233163cb75cc3dc5
Author: Robert Muir <rmuir@apache.org>
Date:   Thu Mar 3 23:19:28 2016 -0500

    move to 6.0.0-snapshot-8eada27

commit b32bfe924626b87e540692375ece09e7c2edb189
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:30:09 2016 +0100

    Fix some test compile errors.

commit 6ccde35e9840b03c68d1a2cd47c7923a06edf64a
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:25:51 2016 +0100

    Current Lucene version is 6.0.0.

commit f62e1015d931b4cc04c778298a8fa1ba65e97ad9
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 11:20:48 2016 +0100

    Fix compile errors in NGramTokenFilterFactory.

commit 6837c6eabf96075f743649da9b9b52dd39611c58
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:50:59 2016 +0100

    Fix the edge ngram tokenizer/filter.

commit ccd7f070de5efcdfbeb34b9555c65c4990bf1ba6
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:42:44 2016 +0100

    The missing value is now accessible through a getter.

commit bd3b77f9b28e5b05daa3d49683a9922a6baf2963
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:41:51 2016 +0100

    Remove IndexCacheableQuery.

commit 05f3091c347aeae80eeb16349ac51d2b53cf86f7
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:39:43 2016 +0100

    Fix compilation of function_score queries.

commit 81cda79a2431ac78f56b0cc5a5765387f662d801
Author: Adrien Grand <jpountz@gmail.com>
Date:   Thu Mar 3 10:35:02 2016 +0100

    Fix compile errors in BlendedTermQuery.

commit 70994ce8dd1eca0b995870974a38e20f26f96a7b
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 23:33:03 2016 -0500

    add bug ID

commit 29d4f1a71f36f646b5a6060bed3db019564a279d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 21:02:32 2016 -0500

    easy .store changes

commit 5e1a1e6fd665fa455e88d3a8987362fad5f44bb1
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 20:47:24 2016 -0500

    cleanups mostly around boosting

commit 333a669ec6c305ada5645d13ed1da0e19ec1d053
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 20:27:56 2016 -0500

    more simple fixes

commit bd5cd98a1e089c866b6b4a5e159400b110140ce6
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 19:49:38 2016 -0500

    more easy fixes and removal of ancient cruft

commit a68f419ee47da5f9c9ce5b372f01d707e902474c
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 19:35:02 2016 -0500

    cutover numerics

commit 4ca5dc1fa47dd5892db00899032133318fff3116
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:34:18 2016 -0500

    fix some constants

commit 88710a17817086e477c6c021ec346d0534b7fb88
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:14:25 2016 -0500

    Add spatial-extras jar as a core dependency

commit c8cd6726583e5ce3f546ed355d4eca037164a30d
Author: Robert Muir <rmuir@apache.org>
Date:   Wed Mar 2 18:03:33 2016 -0500

    update to lucene 6 jars
2016-03-07 04:12:23 -05:00
Alexander Kazakov 3674774b73 Build empty extended stats aggregation if no docs are collected for bucket #16812 2016-03-05 22:50:54 +03:00
javanna da19922489 [TEST] adapt test infra to node.client setting removal
No changes were really needed in our test infra as it didn't use `node.client`. Yet it didn't take into account ingest nodes, what we used to call client nodes in InternalTestCluster were actually ingest only nodes, which now become coordinating only nodes.
Also renamed some method to get rid of the node client terminology as much as possible in favour or coordinating only node.
2016-03-05 10:55:19 +01:00
Nik Everett 4d6cb34417 [reindex] Add ingest support 2016-03-04 10:05:13 -05:00
Christoph Büscher 30a788d87c Suggestions: Make field name mandatory ctor argument
The field name is a required argument for all suggesters, but
it was specified via a field() setter in SuggestionBuilder so far.
This changes field name to being a mandatory constructor argument
and lets suggestion builders throw an error if field name is missing
or the empty string.
2016-03-03 21:35:53 +01:00
Lee Hinman 6adbbff97c Fix organization rename in all files in project
Basically a query-replace of "https://github.com/elasticsearch/" with "https://github.com/elastic/"
2016-03-03 12:04:13 -07:00
Clinton Gormley 529f7cb42c Fixed bad YAML in msearch test 2016-03-03 14:45:02 +01:00
Christoph Büscher ef4db5c1e4 Merge branch 'master' into feature-suggest-refactoring 2016-03-03 12:18:03 +01:00
Adrien Grand 2b545df372 Fix modules/reindex to not use the string field anymore. 2016-03-03 11:11:00 +01:00
Adrien Grand eef19be072 Deprecate string in favor of text/keyword. #16877
This commit removes the ability to use string fields on indices created on or
after 5.0. Dynamic mappings now generate text fields by default for strings
but there are plans to also add a sub keyword field (in a future PR).

Most of the changes in this commit are just about replacing string with
keyword or text. Some tests have been removed because they existed because of
corner cases of string mappings like setting ignore-above on a text field or
enabling term vectors on a keyword field which are now impossible.

The plan is to remove strings entirely in 6.0.
2016-03-03 10:20:56 +01:00
Nik Everett 18e5bb83c5 Disable problematic reindex test
This should get the builds back to normal while we wait on #16914 or
something like it to fix the test properly.
2016-03-02 13:02:29 -05:00
Nik Everett 942eb70956 Revert "Silence reindex's rest tests"
This reverts commit aa0ef84f5a.
2016-03-02 09:17:06 -05:00
Nik Everett aa0ef84f5a Silence reindex's rest tests
They are failing sporadically in CI.
2016-03-02 08:32:30 -05:00
Christoph Büscher be8ed737bc Suggest: Move name of suggestion to SuggestBuilder
Currently each suggestion keeps track of its own name. This has
the disadvantage of having to pass down the parsed name property
in the suggestions fromXContent() and the serialization methods
as an argument, since we need it in the ctor.

This change moves the naming of the suggestions to the surrounding
SuggestBuilder and by this eliminates the need for passind down
the names in the parsing and serialization methods. By making
`name` a required argument in SuggestBuilder#addSuggestion() we
also make sure it is always set and prevent using the same name twice,
which wasn't possible before.
2016-03-01 17:34:47 +01:00
Christoph Büscher aecf51cb42 Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/rest/action/search/RestSearchAction.java
2016-03-01 17:20:15 +01:00
Nik Everett c7c8bb357a Merge pull request #16861 from nik9000/reindex_is_ready
Reindex required some parsing changes for search requests to support
differing defaults from the regular search api.
2016-03-01 10:02:48 -05:00
javanna b7597d7aea Rename SearchServiceTransportAction to SearchTransportService
The suffix TransportAction is misleading as it may make think that it extends TransportAction, but it does not. This class makes accessible the different search operations exposed by SearchService through the transport layer. Also resolved few compiler warnings in the class itself.
2016-03-01 11:35:04 +01:00
Nik Everett aeed7ee218 Reindex: rename source to searchRequest
This makes the code easier to read for those familiar with the
Elasticsearch code base.
2016-02-29 14:57:16 -05:00
Nik Everett 16934d50b6 Reindex isn't a plugin
Remove another mention of it being a plugin.
2016-02-29 10:21:10 -05:00
Nik Everett 356b7cd12b Merge reindex to master because it is ready! 2016-02-29 10:08:29 -05:00
Nik Everett d587a74533 Fix reindex to work with master branch
Stuff changed, reindex's got to change.
2016-02-29 10:04:16 -05:00
Christoph Büscher 379bd94c2a Merge branch 'master' into feature-suggest-refactoring 2016-02-29 14:06:30 +01:00
Yannick Welsch ab4e671bfd Remove suppressAccessChecks permission for Groovy script plugin
Closes #16527
2016-02-27 18:58:00 +01:00
Yannick Welsch af918ebf2b Upgrade groovy dependency in lang-groovy module to version 2.4.6
Closes #16830
2016-02-27 17:41:15 +01:00
Nik Everett c38119bae9 Merge branch 'master' into feature/reindex 2016-02-26 16:59:54 -05:00
Christoph Büscher d888b5c267 Merge branch 'master' into feature-suggest-refactoring 2016-02-22 11:22:28 -08:00
Jack Conradson 7986770e5f Moved Painless from a plugin to a module.
Closes #16755
2016-02-21 16:50:54 -08:00
Mike McCandless 5fffede2b0 Upgrade to Lucene 5.5.0 official release 2016-02-20 17:34:16 -05:00
Ali Beyad 5320d538f1 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-02-15 11:03:33 -05:00
Colin Goodheart-Smithe ebc745211d Moved AggregatorBuilder implementations into their own class files
Also renamed histogram.AbstractBuilcer to AbstractHistogramBuilder, range.AbstractBuilder to AbstractRangeBuilder and org.elasticsearch.search.aggregations.pipeline.having to org.elasticsearch.search.aggregations.pipeline.bucketselector
2016-02-15 14:09:36 +00:00
Colin Goodheart-Smithe 1a46628daa Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/builder/SearchSourceBuilderTests.java
2016-02-15 10:37:16 +00:00
Jason Tedor 1fb1ffbac6 Class permission for Groovy references
This commit adds a class permission for groovy.lang.Reference so they
can be used in scripts.

Closes #16660
2016-02-14 09:45:40 -05:00
Ali Beyad beea413820 Merge remote-tracking branch 'upstream/master' into feature-suggest-refactoring 2016-02-12 11:54:44 -05:00
Simon Willnauer 297fb24112 apply fixes after review 2016-02-12 12:53:18 +01:00
Colin Goodheart-Smithe ed3f7903f4 Merge branch 'master' into feature/aggs-refactoring 2016-02-12 09:48:59 +00:00
Simon Willnauer 685bee3081 Merge branch 'master' into add_rewrite_infra 2016-02-12 09:46:01 +01:00
Ali Beyad eed557742f Refactors building query specific objects from the term suggestion
builder and merges changes between the suggestion builders to context
object implementations.
2016-02-11 18:30:58 -05:00
Nik Everett 821a20f582 Merge branch 'master' into feature/reindex 2016-02-11 17:41:05 -05:00
Nik Everett 18808b7576 Move reindex from a plugin to a module 2016-02-11 17:39:49 -05:00
Nicholas Knize 52ee4c7027 upgrade to lucene 5.5.0-snapshot-850c6c2 2016-02-11 14:28:50 -06:00
Adrien Grand bc47c577d2 Add a new `keyword` field.
The `keyword` field is intended to replace `not_analyzed` string fields. It is
indexed and has doc values by default, and doesn't support enabling term
vectors.

Although it doesn't support setting an analyzer for now, there are plans for
it to support basic normalization in the future such as case folding.
2016-02-11 18:19:53 +01:00
Simon Willnauer 8bdc7362a6 Add infrastructure to rewrite query buidlers
QueryBuilders today do all their heavy lifting in toQuery() which
can be too late for several operations. For instance if we want to fetch geo shapes
on the coordinating node we need to do all this before we create the actual lucene query
which happens on the shard itself. Also optimizations for request caching need to be done
to the query builder rather than the query which then in-turn needs to be serialized again.
This commit adds the basic infrastructure for query rewriting and moves the heavy lifting into
the rewrite method for the following queries:

 * `WrapperQueryBuilder`
 * `GeoShapeQueryBuilder`
 * `TermsQueryBuilder`
 * `TemplateQueryBuilder`

Other queries like `MoreLikeThisQueryBuilder` still need to be fixed / converted. The nice
sideeffect of this is that queries like template queries will now also match the request cache
if their non-template equivalent has been cached befoore. In the future this will allow to
add optimizataion like rewriting time-based queries into primitives like `match_all_docs` or `match_no_docs`
based on the currents shards bounds. This is especially appealing for indices that are read-only ie. never change.
2016-02-11 10:45:49 +01:00
Christoph Büscher 9e0f6e3f9c Adding method to build SuggestionContext to PhraseSuggestionBuilder
This adds a build method for the SuggestionContext to the PhraseSuggestionBuilder
and another one that creates the SuggestionSearchContext to the top level
SuggestBuilder. Also adding tests that make sure the current way of parsing
xContent to a SuggestionContext is reflected in the output the builders create.
2016-02-10 18:57:27 +01:00
Colin Goodheart-Smithe 1f760bd1bd Merge branch 'master' into feature/aggs-refactoring 2016-02-10 12:16:26 +00:00
Christoph Büscher 421ed1228b Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggestionBuilder.java
2016-02-10 11:37:51 +01:00
Colin Goodheart-Smithe 54e2f31189 Parse Aggregations on Coordinating node 2016-02-10 10:00:16 +00:00
Yannick Welsch 380393a5b7 Add permission to access sun.reflect.MethodAccessorImpl from Groovy scripts
Groovy uses reflection to invoke closures. These reflective calls are optimized by the JVM after "sun.reflect.inflationThreshold" number of invocations.
After inflation, access to sun.reflect.MethodAccessorImpl is required from the security manager.

Closes #16536
2016-02-09 16:47:38 +01:00
Christoph Büscher fab3b5568f Merge branch 'master' into feature-suggest-refactoring
Conflicts:
	core/src/main/java/org/elasticsearch/search/suggest/phrase/PhraseSuggestionContext.java
2016-02-09 11:35:13 +01:00
Colin Goodheart-Smithe 02ecfd6279 Merge branch 'master' into feature/aggs-refactoring 2016-02-08 15:41:02 +00:00
Colin Goodheart-Smithe a0b004acb5 Split AggregatorFactory into AggregatorBuilder and AggregatorFactory 2016-02-08 14:27:34 +00:00
Isabel Drost-Fromm 0b3cab6da3 Move sorting tests w/o scripting back to core
In preparation of #16127 this moves sorting tests back to core
that don't actually need scripting or groovy to work.
2016-02-08 14:45:33 +01:00
Christoph Büscher 2dec129038 Merge branch 'master' into feature-suggest-refactoring 2016-02-08 09:47:06 +01:00
Simon Willnauer b185209b63 Make IndicesWarmer a private class of IndexService
There is no need for IndicesWarmer to be a global accessible class. All it needs
access to is inside IndexService. It also doesn't need to be mutable once it's not a per node
instance. This commit move IndicesWarmer to IndexWarmer and makes the default impls like field data and
norms warming an impl detail. Also the IndexShard doesn't depend on this class anymore, instead it accepts
an Engine.Warmer as a ctor argument which delegates to the actual warmer from the index.
2016-02-05 15:24:17 +01:00
Colin Goodheart-Smithe 19d9c588e1 more line length fixes 2016-02-05 10:41:06 +00:00
Simon Willnauer 8685a003d9 fix another test 2016-02-05 11:17:53 +01:00
Colin Goodheart-Smithe f06f17f328 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamInput.java
#	core/src/main/java/org/elasticsearch/common/io/stream/StreamOutput.java
2016-02-05 09:22:14 +00:00
Christoph Büscher 225adcfaeb Merge branch 'master' into feature-suggest-refactoring 2016-02-04 23:50:57 +01:00
Martijn van Groningen 7a6adfd93a ingest: Added foreach processor.
This processor is useful when all elements of a json array need to be processed in the same way.
This avoids that a processor needs to be defined for each element in an array.
Also it is very likely that it is unknown how many elements are inside an json array.
2016-02-04 23:44:01 +01:00
Ryan Ernst fe8c9d394d Build: Do not publish modules
Modules are an internal implementation detail of our build, and there is
no need to publish them with gradle. This change disables publishing of
all modules.
2016-02-04 13:03:03 -08:00
Christoph Büscher c0006426a7 Merge branch 'master' into feature-suggest-refactoring 2016-02-04 11:01:24 +01:00
Simon Willnauer cc520b20e1 Merge branch 'master' into fix_settings_filter 2016-02-04 09:28:41 +01:00
Nik Everett a2f07679fd Add task status
Implements a simple task status for superclasses of ReplicationRequest to
show how you can do use the status.
2016-02-03 18:21:42 -05:00
Simon Willnauer e02d2e004e Rewrite SettingsFilter to be immutable
This change rewrites the entire settings filtering mechanism to be immutable.
All filters must be registered up-front in the SettingsModule. Filters that are comma-sparated are
not allowed anymore and check on registration.
This commit also adds settings filtering to the default settings recently added to ensure we don't render
filtered settings.
2016-02-03 20:05:55 +01:00
Ali Beyad 96d7fc858f Fixed a compiler error with the SuggestSearchTests 2016-02-03 12:42:03 -05:00
Ali Beyad 882cd5c6ee Refactored the term suggestion builder for the query refactoring effort
Added the term suggestion builder's serialization/deserialization and
equals/hashCode methods.
2016-02-03 12:00:03 -05:00
Colin Goodheart-Smithe 5d9d91b761 Merge branch 'master' into feature/aggs-refactoring 2016-02-03 14:45:16 +00:00
Isabel Drost-Fromm 7d787ad1d1 Merge pull request #16411 from MaineC/test-fix/geo-sort-messy
Move non-groovy geo tests from messy tests to core
2016-02-03 14:52:01 +01:00
Isabel Drost-Fromm f99570b0f4 Split messy geo distance/ geo distance sorting tests.
This splits the geo distance and geo distance sorting tests marked messy:
Test cases that don't really need Groovy support are moved back to the
core test suite closer to the code they actually test.

Relates to #15178
2016-02-03 13:54:07 +01:00
Simon Willnauer a77344d742 Merge branch 'master' into make_settings_strict 2016-02-03 13:21:49 +01:00
Simon Willnauer 4a4e523357 Merge branch 'master' into make_settings_strict 2016-02-03 11:34:12 +01:00
Christoph Büscher b7e3323ded Merge branch 'master' into feature-suggest-refactoring 2016-02-03 09:51:23 +01:00
Robert Muir d5dc05f69e Upgrade to lucene 5.5.0-snapshot-1725675 2016-02-02 22:53:39 -05:00
Tal Levy 3191fc7347 Merge pull request #16355 from talevy/fix_ingest_exception
revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation
2016-02-02 14:11:24 -08:00
Tal Levy 0a1580eefa revert PipelineFactoryError handling with throwing ElasticsearchParseException in ingest pipeline creation 2016-02-02 14:08:22 -08:00
Martijn van Groningen 64ac037a82 Added an ingest qa that tests processor real world like configurations.
Renamed `ingest-with-mustache` to `smoke-test-ingest-with-all-dependencies`
Also renamed `ingest-disabled` to `smoke-test-ingest-disabled` so that the name is more inline with other qa smoke test modules.
2016-02-02 22:37:24 +01:00
Yannick Welsch 8cca0395ef Revert "Make GeoDistanceSortBuilder serializable"
This reverts commit c108a4ce6d.
2016-02-02 14:04:45 +01:00
Yannick Welsch 7ea2e99f91 Revert "Indentation fix for messy SimpleSortTest"
This reverts commit 26f77eb70d.
2016-02-02 14:03:40 +01:00
Isabel Drost-Fromm 26f77eb70d Indentation fix for messy SimpleSortTest 2016-02-02 13:46:25 +01:00
Simon Willnauer 818a9eefb2 Make settings validation strict
This commit enableds strict settings validation on node startup. All settings
passed to elasticsearch either through system properties, yaml files or any other
way to pass settings must be registered and valid. Settings that are unknown ie. due to
typos or due to deprecation or removal will cause the node to NOT start up. Plugins
have to declare all their settings on the `SettingsModule#registerSetting` and settings for
plugins that are not installed must be removed.

This commit also removes the ability to specify the nodes name via `-Des.name` or just `name` in the
configuration files. The node name must be prefixed with the node prexif like `node.name: Boom`. Left over
usage of `name` will also cause startup to fail.
2016-02-02 11:32:44 +01:00
Isabel Drost-Fromm c108a4ce6d Make GeoDistanceSortBuilder serializable
Adds to GeoDistanceSortBuilder:

* equals
* hashcode
* writeto/readfrom
* moves xcontent parsing logic over
* adds roundtrip tests
* fixes roundtrip test for xcontent by keeping points just as geopoints not geohashes internally
* fixes xcontent parsing of ignore_malformed if coerce is set/unset
* adds exception to sortMode setter to avoid setting invalid sort modes

Relates to #15178
2016-02-02 11:24:32 +01:00
Colin Goodheart-Smithe a241983c9d Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/aggregations/AggregationsBinaryIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/FilterIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/FiltersIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/GeoDistanceIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/NestedIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/SamplerIT.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/DerivativeIT.java
#	modules/lang-groovy/src/test/java/org/elasticsearch/messy/tests/BucketSelectorTests.java
#	modules/lang-groovy/src/test/java/org/elasticsearch/messy/tests/TDigestPercentileRanksTests.java
2016-02-01 13:38:33 +00:00
Christoph Büscher 7cae28f96a Merge branch 'master' into feature-suggest-refactoring 2016-02-01 13:38:34 +01:00
Christoph Büscher 61c435e6a9 PhraseSuggestionBuilde: Refactor DirectCandidateGenerator
As a prerequisite for refactoring the whole PhraseSuggestionBuilder
to be able to be parsed and streamed from the coordinating node, the
DirectCandidateGenerator must implement Writeable, be able to parse
a new instance (fromXContent()) and later when transported to the
shard to generate a PhraseSuggestionContext.DirectCandidateGenerator.
Also adding equals/hashCode and tests and moving DirectCandidateGenerator
to its own DirectCandidateGeneratorBuilder class.
2016-02-01 12:22:48 +01:00
Jason Tedor 105411060c Uppercase ells ('L') in long literals
This commit removes and forbids the use of lowercase ells ('l') in long
literals because they are often hard to distinguish from the digit
representing one ('1').

Closes #16329
2016-01-30 22:16:02 -05:00
Tal Levy fca442f4d1 Introduce Pipeline Factory Error Responses in Node Ingest
When there is an exception thrown during pipeline creation within
Rest calls (in put pipeline, and simulate) We now return a structured
error response to the user with details around which processor's
configuration is the cause of the issue, or which configuration property
is misconfigured, etc.
2016-01-29 13:37:27 -08:00
Colin Goodheart-Smithe 859f9e69b7 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/action/search/SearchRequestBuilder.java
#	core/src/main/java/org/elasticsearch/percolator/PercolateContext.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/significant/heuristics/ScriptHeuristic.java
#	core/src/main/java/org/elasticsearch/search/internal/DefaultSearchContext.java
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/significant/SignificanceHeuristicTests.java
2016-01-29 11:58:19 +00:00
Christoph Büscher 75e93c2eca Merge branch 'master' into feature-suggest-refactoring 2016-01-29 10:52:54 +01:00
Igor Motov e035dabd4d Extend tracking of parent tasks to master node, replication and broadcast actions
Now MasterNodeOperations, ReplicationAllShards, ReplicationSingleShard,  BroadcastReplication and BroadcastByNode actions keep track of their parent tasks.
2016-01-28 12:13:36 -05:00
Christoph Büscher 09c8a21e83 Merge branch 'master' into feature-suggest-refactoring 2016-01-28 10:46:40 +01:00
Boaz Leskes 2a137b5548 Make index uuid available in Index, ShardRouting & ShardId
In the early days Elasticsearch used to use the index name as the index identity. Around 1.0.0 we introduced a unique index uuid which is stored in the index setting. Since then we used that uuid in a few places but it is by far not the main identifier when working with indices, partially because it's not always readily available in all places.

This PR start to make a move in the direction of using uuids instead of name by making sure that the uuid is available on the Index class (currently just a wrapper around the name) and as such also available via ShardRouting and ShardId.

Note that this is by no means an attempt to do the right thing with the uuid in all places. In almost all places it falls back to the name based comparison that was done before. It is meant as a first step towards slowly improving the situation.

Closes #16217
2016-01-28 08:40:10 +01:00
Christoph Büscher c00c0fa020 Initial refactoring for phrase suggester
Adding initial serialization methods (readFrom, writeTo) to the
PhraseSuggestionBuilder, also adding the base test framework for
serialiazation testing, equals and hashCode. Moving SuggestionBuilder
out of the global SuggestBuilder for better readability.
2016-01-27 17:16:37 +01:00
Simon Willnauer 7ff99eb89d Merge branch 'master' into trash_context_and_headers
if (name == 'expamle-fixtures') return
2016-01-27 16:33:50 +01:00
Colin Goodheart-Smithe 187009c12c Merge branch 'master' into feature/aggs-refactoring 2016-01-27 14:54:12 +00:00
Jason Tedor 284cc3a048 Script mode settings as booleans
This commit modifies the accept values for script mode settings from
"on", "off", and "sandbox" to "true", "false", and "sandbox".
2016-01-27 06:26:58 -05:00
Jason Tedor 9944573449 Rename methods on ScriptEngineService
This commit method renames the ScriptEngineService interface methods
types, extensions, and sandboxed to getTypes, getExtensions, and
isSandboxed, respectively.
2016-01-27 06:26:04 -05:00
Jason Tedor 087e55cc51 Script mode settings
This commit converts the script mode settings to the new settings
infrastructure. This is a major refactoring of the handling of script
mode settings. This refactoring is necessary because these settings are
determined at runtime based on the registered script engines and the
registered script contexts.
2016-01-27 06:26:04 -05:00
Simon Willnauer 71c3e57aee Merge branch 'master' into trash_context_and_headers 2016-01-27 11:42:42 +01:00
Adrien Grand 2aaa5e6448 Remove the ability to enable doc values with the `fielddata.format` setting.
Doc values can now only be enabled by setting `doc_values: true` in the
mappings. Removing this feature also means that we can now fail mapping updates
that try to disable doc values.
2016-01-27 09:06:00 +01:00
Adrien Grand 35709f62b6 Be stricter about parsing boolean values in mappings.
Parsing is currently very lenient, which has the bad side-effect that if you
have a typo and pass eg. `store: fasle` this will actually be interpreted as
`store: true`. Since mappings can't be changed after the fact, it is quite bad
if it happens on an index that already contains data.

Note that this does not cover all settings that accept a boolean, but since the
PR was quite hard to build and already covers some main settirgs like `store`
or `doc_values` this would already be a good incremental improvement.
2016-01-27 09:06:00 +01:00
Boaz Leskes 82ae74071e logging: cleanup some old TestLogging annotations and add a few debug level by default 2016-01-26 20:48:07 +01:00
Jason Tedor bdddea2dd0 Security permissions for Groovy closures
This commit adds some permissions that Groovy needs to use closures.

Closes #16196
2016-01-26 12:33:07 -05:00
Colin Goodheart-Smithe 11bafa18e1 Removes Aggregation Builders in place of AggregatorFactory implementations 2016-01-26 15:13:43 +00:00
Colin Goodheart-Smithe 3b35754f59 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/test/java/org/elasticsearch/percolator/PercolateDocumentParserTests.java
2016-01-26 13:17:53 +00:00
javanna 9bb4caa6f1 adapt to upstream changes, jvm flag doesn't get returned anymore 2016-01-25 18:01:19 +01:00
javanna 36d98478bf Merge branch 'master' into feature/ingest 2016-01-25 18:01:09 +01:00
Colin Goodheart-Smithe d63b8a25f8 Merge branch 'master' into feature/aggs-refactoring 2016-01-25 10:42:59 +00:00
Colin Goodheart-Smithe cd8320b171 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filter/FilterAggregator.java
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/filters/FiltersAggregator.java
#	core/src/main/java/org/elasticsearch/search/SearchModule.java
2016-01-25 10:42:20 +00:00
Alexander Reelsen 3a0839de9a Scripting: Allow to get size of array in mustache
This adds support for returning the size of an array
or an collection, in addition to access fields via
`array.0` you can specify `array.size` to get its size.
2016-01-25 07:58:10 +01:00
markharwood d371ef35f4 Settings - change over to o.e.common.settings.Setting for http settings:
http.cors.allow-credentials
http.cors.enabled
http.detailed_errors.enabled
http.enabled
http.pipelining
http.cors.max-age
2016-01-22 15:13:11 +00:00
Daniel Mitterdorfer e9bb3d31a3 Convert "path.*" and "pidfile" to new settings infra 2016-01-22 15:14:13 +01:00
Ryan Ernst df24019261 Merge pull request #16038 from rjernst/remove_site_plugin
Plugins: Remove site plugins
2016-01-21 12:32:22 -08:00
Tal Levy 3a6c2d008e rename processor_tag to tag 2016-01-21 09:05:42 -08:00
Martijn van Groningen 20f634952c mustache: Improve the mustache script engine
* Added a `content_type` option at compile time to decide how variable values are encoded. Possible values are `application/json` and `plain/text`. Defaults to `application/json`.
* Added support for variable placeholders to lookup values from specific slots in arrays/lists.
2016-01-21 18:04:06 +01:00
Tal Levy c55f9bbd9b Merge pull request #16132 from talevy/holdmytag
[Ingest] add an AbstractProcessor to help hold the re-used processorTag variable
2016-01-21 07:30:29 -08:00
Simon Willnauer 142547271e Merge branch 'master' into trash_context_and_headers 2016-01-21 14:42:15 +01:00
Martijn van Groningen 44465c94f8 Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-21 11:46:27 +01:00
Tal Levy f1204cb1ab add an AbstractProcessor to help hold the re-used processorTag variable 2016-01-20 14:34:26 -08:00
Robert Muir 6e7e3a2274 Update lucene to r1725675
Adds DFI (divergence from independence) provider.
Fixes test bugs passing invalid values for BM25 parameters.
2016-01-20 03:32:51 -05:00
Martijn van Groningen e2e207687d Fixes due to changes in master branch. 2016-01-19 22:03:26 +01:00
Martijn van Groningen 602a0f183e Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-19 22:01:38 +01:00
Simon Willnauer 40783b5d32 delete dead code 2016-01-19 21:06:23 +01:00
Simon Willnauer 3d0cedbabb Merge branch 'master' into trash_context_and_headers 2016-01-19 20:47:43 +01:00
Nik Everett e1e73d9914 Create default for ExecutableScript#unwrap
Tons of scripts just return the variable they are passed and that is the
most intuitive behavior so that may as well be the default implementation.
2016-01-19 13:17:11 -05:00
Simon Willnauer fbfa9f4925 Merge branch 'master' into new_index_settings 2016-01-19 10:13:48 +01:00
Nik Everett 53cce3c399 Merge pull request #16071 from nik9000/xlint_lang_groovy
Remove Xlint from lang-groovy and discovery-azure
2016-01-18 21:06:11 -05:00
Ryan Ernst ef4f0a8699 Test: Make rest test framework accept http directly for the test cluster
The rest test framework, because it used to be tightly integrated with
ESIntegTestCase, currently expects the addresses for the test cluster to
be passed using the transport protocol port. However, it only uses this
to then find the http address.

This change makes ESRestTestCase extend from ESTestCase instead of
ESIntegTestCase, and changes the sysprop used to tests.rest.cluster,
which now takes the http address.

closes #15459
2016-01-18 16:44:14 -08:00
Nik Everett bd1324ccc9 Remove Xlint from lang-groovy and discovery-azure
discovery-azure didn't actually need it.

This removes all non-default Xlints from modules.
2016-01-18 18:24:44 -05:00
Nik Everett 63090f5d4b Merge pull request #15761 from nik9000/more_remove_inject
Don't use guice for QueryParsers
2016-01-18 17:39:53 -05:00
Nik Everett 093600cd63 Cleanup ContextAndHeaderTransportTests
We have two similar tests with the same name, ContextAndHeaderTransportTests.
They shared lots of common code so I extracted much of it into
ActionRecordingPlugin, a plugin which records all action requests for later
inspection.

I also removed all the warnings from both tests. That made lang-mustache
compile cleanly without any custom -Xlint so I removed those. To remove
the warnings I had to add type parameters to ActionFilter which seemed
like a good idea anyway.
2016-01-18 16:14:58 -05:00
Nik Everett 15c43b64a1 Don't use guice for QueryParsers
1. Gets guice out of the business of building ScoreFunctionParsers and
QueryParsers.
2. Moves QueryParser registration to SearchModule
3. Moves NamedWriteableRegistry construction out of guice and into Node and
TransportClient.
4. Moves shape registration into SearchModule so now all named writeable
registration is done in the SearchModule.

This is breaking for plugin authors. Instead of declaring new QueryParser
like:

```java
public void onModule(IndicesModule module) {
  module.registerQueryParser(NewQueryParser.class);
}
```

you do it like:

```java
public void onModule(SearchModule module) {
  module.registerQueryParser(NewQueryParser::new);
}
```

The QueryParser's argument no longer come from @Inject, now they come from
the declaration in the plugin. The above example is for a no-arg QueryParser.
Most of the QueryParsers in Elasticsearch are no-arg.

ScoreFunctionParsers have a similar but slightly different change. This:

```java
public void onModule(SearchModule module) {
  module.registerFunctionScoreParser(NewFunctionScoreParser.class);
}
```

becomes

```java
public void onModule(SearchModule module) {
  module.registerFunctionScoreParser(new NewFunctionScoreParser());
}
```

Since all known ScoreFunctionParsers have no arg constructors its simpler to
just build them at registration time rather than specify a supplier that is
used to build them later.
2016-01-18 15:02:47 -05:00
Simon Willnauer c2e8ba8970 Register index.version_created on unittest that set it 2016-01-18 15:04:35 +01:00
Simon Willnauer 7925e2ef84 convert IndexModule settings 2016-01-18 09:23:35 +01:00
Simon Willnauer a8eedd0457 fix all kinds of crazy test failures 2016-01-18 09:23:34 +01:00
Simon Willnauer 2eecb9c02d add tests for AutoExpandReplicas 2016-01-18 09:23:33 +01:00
Nik Everett 742e1da857 Squash more raw type errors
Its a long string to pull
2016-01-17 17:07:32 -05:00
Ryan Ernst 3b78267c71 Plugins: Remove site plugins
Site plugins used to be used for things like kibana and marvel, but
there is no longer a need since kibana (and marvel as a kibana plugin)
uses node.js. This change removes site plugins, as well as the flag for
jvm plugins. Now all plugins are jvm plugins.
2016-01-16 22:45:37 -08:00
Tal Levy 9f48df9736 Add on_failure support for verbose _simulate execution and introduce optional processor_tag to Processors 2016-01-15 14:56:20 -08:00
javanna 050585e89f remove BiFunction<Environment, TemplateService, Processor.Factory> in favour of Function<TemplateService, Processor.Factory>
the environment is now available through NodeModule#getNode#getEnvironment and can be retrieved during onModule(NodeModule), no need for this indirection anymore using the BiFunction
2016-01-15 15:12:53 +01:00
Martijn van Groningen 21cc0b2316 Cleanup ingest initialization code.
* Folded IngestModule into NodeModule
* Renamed IngestBootstrapper to IngestService
* Let NodeService construct IngestService and removed the Guice annotations
* Let IngestService implement Closable
2016-01-15 13:35:06 +01:00
javanna 169b3c75c5 move grok patterns loading to IngestGrokPlugin class out of the GrokProcessor.Factory 2016-01-14 16:24:07 +01:00
Martijn van Groningen f3883343cb Move the pipeline configuration from the dedicated index to the cluster state.
Closes #15842
2016-01-13 22:59:36 +01:00
Simon Willnauer ce42ae4cf3 Merge pull request #15922 from s1monw/feature/ingest
Review feedback and several cleanups
2016-01-13 12:04:04 +01:00
Simon Willnauer 574d1b35b3 Replace ContextAndHeaders with a ThreadPool based ThreadLocal implementation
ContextAndHeaders has a massive impact on the core infrastructure since it has to
be manually passed on to all relevant places across threads/network calls etc. For the same reason
it's also very error prone and easily forgotten on potentially relevant APIs.

The new ThreadContext is associated with a ThreadPool (node or transport client) and ensures that
headers and context registered on a current thread are inherited to new threads spawned, send across
the network to be deserialized on the receiver end as well as restored on the response handling thread
once the response is received.
2016-01-13 11:53:32 +01:00
Tal Levy 4bf12a7dca Merge pull request #15910 from talevy/ingest-fix-grok-nocaptures
[Ingest] fix NPE when Grok matches expression, but no captures defined.
2016-01-12 12:02:55 -08:00
Tal Levy b39a64e49f accept grok matches with no captures instead of NPE 2016-01-12 12:01:55 -08:00
Colin Goodheart-Smithe f429cb08d7 Merge branch 'master' into feature/aggs-refactoring 2016-01-12 12:59:52 +00:00
Simon Willnauer 4d38a47eb5 Review feedback and several cleanups 2016-01-12 12:06:14 +01:00
Martijn van Groningen 7bdd2583aa Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-12 01:01:30 +01:00
Colin Goodheart-Smithe 9d7d89d04d Merge branch 'master' into feature/aggs-refactoring 2016-01-11 12:50:31 +00:00
Martijn van Groningen 1637fe9e0b Moved the grok processor to its own module, so that it will available out-of-the-box, while its dependencies are isolated 2016-01-08 21:59:23 +01:00
Nik Everett 81a7607256 Remove -Xlint:-deprecation from plugins
Instead we suppress warnings about using deprecated stuff near the usage
site with a comment about why its ok.
2016-01-07 20:44:46 -05:00
Colin Goodheart-Smithe 3b7d1b47f7 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/percolator/PercolatorService.java
#	core/src/main/java/org/elasticsearch/percolator/QueryCollector.java
2016-01-07 12:42:02 +00:00
Nik Everett 0786c506dc Remove a few more Xlint skips 2016-01-06 23:28:13 -05:00
Martijn van Groningen 2d6adf6428 Percolator refactoring:
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.

Closes #12664
Closes #13646
2016-01-06 16:08:10 +01:00
Colin Goodheart-Smithe 2c33f78192 Merge branch 'master' into feature/aggs-refactoring
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/aggregations/bucket/children/ChildrenParser.java
#	core/src/main/java/org/elasticsearch/search/aggregations/support/ValuesSourceParser.java
#	test/framework/src/main/java/org/elasticsearch/test/TestSearchContext.java
2016-01-06 09:35:53 +00:00
Igor Motov a89dba27c2 Task Management: Add framework for registering and communicating with tasks
Adds task manager class and enables all activities to register with the task manager. Currently, the immutable Transport*Activity class represents activity itself shared across all requests. This PR adds and an additional structure Task that keeps track of currently running requests and can be used to communicate with these requests using TransportTaskAction.

Related to #15117
2016-01-05 12:24:43 -05:00
Simon Willnauer ea2a8150d3 Merge pull request #15692 from s1monw/issues/15675
Fix pharse suggest collate example to use correct script context/syntax
2016-01-05 09:27:18 +01:00
Adrien Grand 1a47226d9a Merge pull request #15663 from jpountz/remove/mapping_backcompat
Remove mapping backward compatibilit with pre-2.0.
2016-01-04 10:05:39 +01:00
Adrien Grand 6d3c9b074c Remove support for the `multi_field` type.
It is officially unsupported since version 1.0.
2015-12-30 12:03:15 +01:00
Simon Willnauer 3b06cd8d92 Fix pharse suggest collate example to use correct script context/syntax
Closes #15675
2015-12-29 09:22:36 +01:00
Robert Muir 180ab2493e Improve thirdPartyAudit check, round 3 2015-12-28 22:38:55 -05:00
Adrien Grand af122f4151 Remove mapping backward compatibilit with pre-2.0.
This removes the backward compatibility layer with pre-2.0 indices, notably
the extraction of _id, _routing or _timestamp from the source document when a
path is defined.
2015-12-24 13:47:37 +01:00
Adrien Grand d8d8666877 Remove `index_name` back compat.
Since 2.0 we enforce that fields have the same full and index names. So in 3.x
we can remove the ability to have different names on the same field.
2015-12-23 14:55:26 +01:00
Adrien Grand cf52e96c42 Upgrade to lucene-5.5.0-snapshot-1721183.
Some files that implement or use the Scorer API had to be changed because of
https://issues.apache.org/jira/browse/LUCENE-6919.
2015-12-21 17:02:08 +01:00
Colin Goodheart-Smithe 97c2f7b037 Aggregations Refactor: Refactor Histogram and Date Histogram Aggregation 2015-12-21 09:34:05 +00:00
Ryan Ernst 4ea19995cf Remove wildcard imports 2015-12-18 12:43:47 -08:00
Colin Goodheart-Smithe 15588a4991 Aggregations: Run pipeline aggregations for empty buckets added in the Range Aggregation
Closes #15471
2015-12-18 14:42:05 +00:00
Adrien Grand 6ea16671f4 Simplify the Text API.
We have the Text API, which is essentially a wrapper around a String and a
BytesReference and then we have 3 implementations depending on whether the
String view should be cached, the BytesReference view should be cached, or both
should be cached.

This commit merges everything into a single Text that is essentially the old
StringAndBytesText impl.

Long term we should look into whether this API has any performance benefit or
if we could just use plain strings. This would greatly simplify all our other
APIs that currently use Text.
2015-12-17 17:22:38 +01:00
Robert Muir a7cc91e868 Merge pull request #15501 from rmuir/sheisty_classes
thirdPartyAudit round 2
2015-12-17 03:44:27 -05:00
Robert Muir 6692e42d9a thirdPartyAudit round 2
This fixes the `lenient` parameter to be `missingClasses`. I will remove this boolean and we can handle them via the normal whitelist.
It also adds a check for sheisty classes (jar hell with the jdk).
This is inspired by the lucene "sheisty" classes check, but it has false positives. This check is more evil, it validates every class file against the extension classloader as a resource, to see if it exists there. If so: jar hell.

This jar hell is a problem for several reasons:

1. causes insanely-hard-to-debug problems (like bugs in forbidden-apis)
2. hides problems (like internal api access)
3. the code you think is executing, is not really executing
4. security permissions are not what you think they are
5. brings in unnecessary dependencies
6. its jar hell

The more difficult problems are stuff like jython, where these classes are simply 'uberjared' directly in, so you cant just fix them by removing a bogus dependency. And there is a legit reason for them to do that, they want to support java 1.4.
2015-12-17 02:35:00 -05:00
Jack Conradson 4523eaec88 Added plumbing for compile time script parameters.
Closes #15464
2015-12-16 18:29:21 -08:00
Robert Muir 4f9d4103f2 Merge pull request #15491 from rmuir/forbidden_third_party
Add gradle thirdPartyAudit to precommit tasks
2015-12-16 18:56:50 -05:00
Robert Muir 42138007db add some more comments about internal api usage 2015-12-16 18:56:02 -05:00
Robert Muir ee79d46583 Add gradle thirdPartyAudit to precommit tasks 2015-12-16 16:38:16 -05:00
Ryan Ernst a2b8f4b90a Merge pull request #15434 from rjernst/http_type
Expose http.type setting, and collapse al(most all) modules relating to transport/http
2015-12-16 11:54:30 -08:00
Adrien Grand 5d5c6591aa Validate that fields are defined only once.
There are two ways that a field can be defined twice:
 - by reusing the name of a meta mapper in the root object (`_id`, `_routing`,
   etc.)
 - by defining a sub-field both explicitly in the mapping and through the code
   in a field mapper (like ExternalMapper does)

This commit adds new checks in order to make sure this never happens.

Close #15057
2015-12-15 10:37:18 +01:00
Ryan Ernst 60d35c81af Plugins: Expose http.type setting, and collapse al(most all) modules relating to transport/http
This change adds back the http.type setting. It also cleans up all the
transport related guice code to be consolidated within the
NetworkModule (as transport and http related stuff is what and how ES
exposes over the network). The setter methods previously used by some
plugins to override eg the TransportService or HttpServerTransport are
removed, and those plugins should now register a custom implementation
of the class with a name and set that using the appropriate config
setting. Note that I think ActionModule should also be moved into here,
to sit along side the rest actions, but I left that for a followup.

closes #14148
2015-12-14 22:01:04 -08:00
Jason Tedor 11cbc08e45 Fix compilation in o.e.m.t.SimpleSortTests 2015-12-11 11:46:50 -05:00
Jason Tedor 3383c24be0 Remove and forbid use of Collections#shuffle(List) and Random#<init>()
This commit removes and now forbids all uses of
Collections#shuffle(List) and Random#<init>() across the codebase. The
rationale for removing and forbidding these methods is to increase test
reproducibility. As these methods use non-reproducible seeds, production
code and tests that rely on these methods contribute to
non-reproducbility of tests.

Instead of Collections#shuffle(List) the method
Collections#shuffle(List, Random) can be used. All that is required then
is a reproducible source of randomness. Consequently, the utility class
Randomness has been added to assist in creating reproducible sources of
randomness.

Instead of Random#<init>(), Random#<init>(long) with a reproducible seed
or the aforementioned Randomess class can be used.

Closes #15287
2015-12-11 11:16:38 -05:00
Robert Muir 1e8f9558a0 Remove now-dead code in expressions (fixed in https://issues.apache.org/jira/browse/LUCENE-6920) 2015-12-10 14:50:32 -05:00
Robert Muir 2741888498 Remove RuntimePermission("accessDeclaredMembers")
Upgrades lucene to 5.5.0-1719088, randomizedtesting to 2.3.2, and securemock to 1.2
2015-12-10 14:26:55 -05:00
Robert Muir e454fadc22 Merge branch 'master' into shave_mustache 2015-12-10 07:58:24 -05:00
Areek Zillur 025e9818e7 Refactors TransportReplicationAction to decouple request routing and shard operation logic 2015-12-10 01:32:07 -05:00
Robert Muir 61266dee19 mark messy rest test as messy and add to messy tests lists 2015-12-09 19:21:35 -05:00
Ryan Ernst 74dc5bf20a Fix template query parser tests to register mustache script engine 2015-12-08 21:33:48 -08:00
Ryan Ernst b7fb0824be Fix render search template tests 2015-12-08 21:33:48 -08:00
Robert Muir a6e1655fe9 fix integ tests 2015-12-09 00:30:32 -05:00
Robert Muir 17436d4332 get gradle check passing for lang-mustache 2015-12-09 00:10:37 -05:00
Robert Muir 3632ddf497 split out mustache methods so groovy works again 2015-12-09 00:00:40 -05:00
Ryan Ernst 907d3cf878 move template query builder tests back to core (using mockscript) 2015-12-08 20:26:45 -08:00
Robert Muir 2e299860e5 refactor mustache to lang-mustache plugin. No rest tests yet. 2015-12-08 22:45:21 -05:00
Robert Muir 3c419c2186 do expressions consistently with other engines 2015-12-05 22:08:40 -05:00
Robert Muir 2169a123a5 Filter classes loaded by scripts
Since 2.2 we run all scripts with minimal privileges, similar to applets in your browser.
The problem is, they have unrestricted access to other things they can muck with (ES, JDK, whatever).
So they can still easily do tons of bad things

This PR restricts what classes scripts can load via the classloader mechanism, to make life more difficult.
The "standard" list was populated from the old list used for the groovy sandbox: though
a few more were needed for tests to pass (java.lang.String, java.util.Iterator, nothing scary there).

Additionally, each scripting engine typically needs permissions to some runtime stuff.
That is the downside of this "good old classloader" approach, but I like the transparency and simplicity,
and I don't want to waste my time with any feature provided by the engine itself for this, I don't trust them.

This is not perfect and the engines are not perfect but you gotta start somewhere. For expert users that
need to tweak the permissions, we already support that via the standard java security configuration files, the
specification is simple, supports wildcards, etc (though we do not use them ourselves).
2015-12-05 21:46:52 -05:00
Robert Muir 46377778a9 Merge branch 'master' into getClassLoader 2015-12-04 15:58:36 -05:00
Robert Muir 7160c5ec15 list modules separately in pluginservice 2015-12-04 01:13:17 -05:00
Ryan Ernst a8e9403204 added gradle checks for modules configuration, and ability to add
modules to integ test cluster
2015-12-03 20:53:06 -08:00
Ryan Ernst 0a4a81afaf Added modules, distributions now include them (just plugins installed in
a diff dir)
2015-12-03 14:18:26 -08:00
Shay Banon bf0caafa04 remove old modules and gradle 2011-12-06 01:01:04 +02:00
Shay Banon a8fd2d48b8 first cleanup phase, move to single src 2011-12-06 00:59:23 +02:00
Shay Banon 9995d27841 Analysis: Add arabic, brazilian, czech to stemmer token filter language options, closes #1519. 2011-12-04 16:36:42 +02:00
Shay Banon 720954d8a6 Version missing a space, closes #1517. 2011-12-04 12:48:29 +02:00
Shay Banon 032e215f25 search missing from stats when using clear and search params, closes #1516. 2011-12-02 12:32:51 +02:00
Shay Banon be282cc4c8 Improve highlighting perf (a bit) by reusing some constructs across hits, closes #1513. 2011-11-30 19:15:31 +02:00
Shay Banon c93ddd9b61 check against hte fieldName... 2011-11-30 13:34:21 +02:00
Shay Banon 6c552b4187 have DocSet implement Bits interface 2011-11-29 23:53:38 +02:00
Shay Banon a21c0829c7 Query DSL: Bool filter does not take should clauses properly into account, closes #1511. 2011-11-29 22:39:07 +02:00
Shay Banon 76307a5e73 cleanup 2011-11-29 21:28:07 +02:00
Shay Banon e9d2f44d49 Query DSL: Bool filter does not take should clauses properly into account, closes #1511. 2011-11-29 21:27:36 +02:00
Shay Banon de49a313c5 Query DSL: Bool filter does not take should clauses properly into account, closes #1511. 2011-11-29 21:23:41 +02:00
Shay Banon 9bb9ce4e84 add 0.18.6 2011-11-29 13:55:11 +02:00
Shay Banon ae4ae598e3 make node closed exception serializable 2011-11-29 10:18:13 +02:00
Shay Banon b2fa6b7a94 When _source is disabled, don't return it in realtime get fetching the document from the transaction log, closes #1509. 2011-11-29 09:39:46 +02:00
Shay Banon e9fe8ec469 sync with latest Lucene block join, init parentDoc to -1, which makes the assert commented out valid again 2011-11-28 23:47:31 +02:00
Shay Banon 6b894d9f53 add the option to get mapping as a parsed map, also do some internal refactoring to share the code that parses into a map 2011-11-28 20:19:02 +02:00
Njal Karevoll b1707d219f set "http" on NodeInfo instances if it is passed in the constructor 2011-11-28 17:08:28 +02:00
Shay Banon d5aa7574f3 Upgrade to netty 3.2.7, closes #1506. 2011-11-28 14:48:49 +02:00
Shay Banon 4be7866bd9 Registering a percolate query with additional "object" level metadata can fail, closes #1505. 2011-11-27 18:06:18 +02:00
Shay Banon 32f1edf6f7 Analysis: Add language setting to lowercase filter, supporting greek and turkish, closes #1503. 2011-11-27 13:39:05 +02:00
George Chatzigeorgiou a1a856e4b9 Support Greek stemming
While the GreekAnalyzer supports stemming, custom analyzers for the greek language cannot be build because there is no GreekStemmer (although lucene has one).
2011-11-27 13:22:58 +02:00
Shay Banon f18ad903a9 Upgrade to Lucene 3.5, closes #1502. 2011-11-27 12:57:26 +02:00
Shay Banon fd5d754fe6 better failure messages when dynamic mapping is disabled 2011-11-25 09:40:44 +02:00
Shay Banon a7803855a1 add another path trie test 2011-11-25 00:21:01 +02:00
Shay Banon 3f835eb780 even though it can't happen (offset is 0 when loading from the index), use properly the field to get binary values with offset and length 2011-11-24 21:44:18 +02:00
Shay Banon 21988a0ca7 support compressed percolator index 2011-11-24 21:32:56 +02:00
Shay Banon ac2c2fb48d enable unsafe optimization in lzf for 0.19 2011-11-24 20:12:51 +02:00
Shay Banon 03c2e5ea52 improve how decoding is done on the transport layer, embedding FrameDecoder into the message handler, and reducing allocation of buffers and better guess into allocating cumalation buffers 2011-11-24 20:03:25 +02:00
Shay Banon f0efb8cdea Using _parent:123 in a query string query fails to fetch docs, closes #1497. 2011-11-24 13:59:10 +02:00
Shay Banon 81965d0ea9 Support using _id:1234, or using term query/filter on _id even when _id is not indexed, closes #1496. 2011-11-24 12:31:50 +02:00
Shay Banon daa4444e22 DocumentMapper.java wrong order on build rootMappers, closes #1493. 2011-11-24 09:58:05 +02:00
Shay Banon 2c0662e18e Query DSL: indices query to allow to set a `no_match_query`, closes #1492. 2011-11-23 19:01:14 +02:00
Shay Banon f47b77199d Multi field mapper with more than one extra mapping can cause endless re-sync'ing of mapping between nodes, closes #1487. 2011-11-23 09:04:28 +02:00
Shay Banon 05f98634d8 IndicesQueryBuilder generates the wrong query name, closes #1485. 2011-11-22 15:57:54 +02:00
Shay Banon f28c11b31c By default, set http.compression to false, closes #1482. 2011-11-21 19:50:24 +02:00
Shay Banon bff980c797 Allow empty Strings to be null for Number's and don't autodetect empty string fields as string types, closes #1473. 2011-11-21 18:02:12 +02:00
Shay Banon fbb03c611a add a sleep to fix test (need to think of a better fix) 2011-11-21 10:44:29 +02:00
Shay Banon b10094b109 support boost on block join query (nested) 2011-11-21 10:43:58 +02:00
Shay Banon 90af54dad5 make sure bulk item failure on primary shards will not execute on a replica shard 2011-11-20 15:09:13 +02:00
Shay Banon bb8ff3814e Binary field compression causes wrong _source decoding, closes #1475. 2011-11-20 12:59:22 +02:00
Jeremie BORDIER 6403a42e31 * Try to extract value from String using FieldMapper when reading from the transaction log. 2011-11-17 18:54:41 +02:00
Shay Banon 4d607bdd4a handle valueForString in binary case 2011-11-17 18:19:16 +02:00
Shay Banon c8641588ec clean code, script service no longer used in get action 2011-11-17 15:48:05 +02:00
Shay Banon 835dc33d45 make sure we use the default index analyzer for _all field, even though it works well without it now because the _analyzer field comes before it and sets the context.analyzer 2011-11-17 15:37:51 +02:00
Shay Banon b0880e114e add another instanceof check on Date and not direct check 2011-11-17 14:27:40 +02:00
Shay Banon 09fe743790 add another instanceof check on Date and not direct check 2011-11-17 14:27:16 +02:00
Shay Banon 118acb3992 add 0.18.5 version 2011-11-16 22:00:13 +02:00
Shay Banon b0fc243138 add 0.18.4 version 2011-11-16 15:23:58 +02:00
Shay Banon 72d2fd0da0 Inner queries not resolved correctly in has_child filter when searching directly against the parent type (in the URI for example), closes #1471. 2011-11-16 14:15:29 +02:00
Shay Banon b5a0ae2fbe better toString for custom score query 2011-11-15 22:20:15 +02:00
Shay Banon e52ec518f3 NPE when highlighting 2011-11-15 18:11:03 +02:00
Shay Banon afb690e233 refactor sub fetch phase to also allow for hits level execution 2011-11-15 18:09:45 +02:00
Shay Banon 537d9c5db0 FVH should use the parsed query, and not a possible rewritten query 2011-11-15 16:45:05 +02:00
Shay Banon 8f392af207 upgrade to mvel 2.1 Beta7 2011-11-15 16:23:13 +02:00
Shay Banon 8d0ac1787b upgrade to jackson 1.9.2 2011-11-15 15:53:18 +02:00
Shay Banon a6a2a264c3 Highlighting on term vector enabled field should not highlight filters by default, closes #1469. 2011-11-15 13:42:15 +02:00
Shay Banon b8fe9999e0 XContentBuilder to handle extended classes of java.util.Date, closes #1465. 2011-11-15 11:34:32 +02:00
Shay Banon abaf4b5196 Indices Stats API: Providing groups as part of the HTTP API does not return stats for those groups, closes #1468. 2011-11-15 10:31:55 +02:00
Igor Motov 0d714d580f Add counts of currently executing get operations 2011-11-15 10:21:10 +02:00
Shay Banon 9bf42fed1b Nested facet execution can fail with ArrayIndexOutOfBounds, closes #1467. 2011-11-15 09:23:56 +02:00
Shay Banon dc2824fe06 apply fix to unsafe decoder in lzf, still use vanilla for now 2011-11-15 08:46:05 +02:00
Shay Banon 73ba30b18a Date Range: Inclusive upper range does not round up properly, closes #1463. 2011-11-14 17:16:42 +02:00
Shay Banon ed281fbfd5 Index shard search slow log (query and fetch), closes #1462. 2011-11-14 14:10:39 +02:00
Shay Banon b43c8fe128 get should report time in millis, but return in nanoseconds 2011-11-14 10:40:05 +02:00
Igor Motov 62d67e6c2d Add counts of currently executing index, delete, query and fetch operations 2011-11-14 10:24:50 +02:00
Shay Banon eefbe52580 disable optimized decoder by default 2011-11-14 10:10:22 +02:00
Shay Banon eb1b0b150f Using root object level mapping (_size, _source) can cause reparsing of the mapping on startup, close #1458. 2011-11-13 20:56:16 +02:00
Shay Banon 3da6e0a5b0 Percolate / Analyzer API can hang if it fails to execute, closes #1457. 2011-11-13 19:54:13 +02:00
Jeremie BORDIER 28c9595af3 * Add support for LZF compression on binary fields. 2011-11-13 14:32:53 +02:00
Shay Banon 4bbf29834e upgrade to lzf compress 0.9 2011-11-13 14:06:09 +02:00
Shay Banon 27a7b0680c fix top_terms_boost parsing for multi term rewrite 2011-11-13 12:51:10 +02:00
Igor Motov 9cc0b7a28b Fix processing of regex patterns in large terms facet requests 2011-11-13 12:49:12 +02:00
Shay Banon a3978402e5 add the search request to the search context 2011-11-13 12:48:17 +02:00
Shay Banon f4adff652c Blank routing should be ignored, closes #1450. 2011-11-13 12:04:02 +02:00
Shay Banon da98aab13f also support _percolate for bulk item requests 2011-11-13 10:19:05 +02:00
Shay Banon 21c4530aaf Search: Search requests execute by mistake on the networking http IO thread, causing other http operations to hang, closes #1455. 2011-11-11 22:22:21 +02:00
Shay Banon 99d31cc8c8 better handlign of open channels 2011-11-11 22:22:15 +02:00
Shay Banon 93f1d50c18 configure the scheduled TP to not wait for delayed execution, so we shutodown faster 2011-11-10 21:52:42 +02:00
Shay Banon 68c8cac852 AbstractCompoundWordTokenFilterFactory should store the dictionary as a CharArraySet instance, not a Set, closes #1449. 2011-11-10 20:12:35 +02:00
Shay Banon 15c798fb4c fix minimum_should_match in query_string builder 2011-11-10 12:13:31 +02:00
Shay Banon c4542c8349 increase search stats rounds to make sure the stats are higher than 0 2011-11-10 12:01:39 +02:00
Shay Banon 62bc0836c1 improve boolean filter 2011-11-10 12:00:43 +02:00
Igor Motov ed8c289dec Log warning when application of an updated cluster state fails with an exception 2011-11-10 10:12:58 +02:00
Igor Motov e7baf30bd2 Fix reporting of total indexing stats 2011-11-10 10:07:10 +02:00
Igor Motov eafa2b0c9a Fix possible NPE in TransportNodesListGatewayStartedShards operation 2011-11-10 10:04:11 +02:00
Shay Banon c5ebe6e86f Mapping: Improve applying guessed types on dynamic templates, closes #1446. 2011-11-09 09:02:20 +02:00
Shay Banon 75f522adaa data.path locations are unevenly filled (0.18.2), closes #1436. 2011-11-02 23:45:35 +02:00
Igor Motov 3c2dd959cb Aliases should be processed when index routing changes 2011-11-01 20:05:46 +02:00
Shay Banon ca7247e1b1 add more details logging on info of data path 2011-10-28 07:45:17 +02:00
Shay Banon 65a525bb15 add 0.18.3 2011-10-27 20:42:43 +02:00
Shay Banon 723d2e8b15 Search: Fetching fields that end up extracted from source might fail, closes #1431. 2011-10-27 20:28:38 +02:00
Shay Banon 8922d5b073 Merge Scheduler: Allow to set index.merge.scheduler.max_merge_count, closes #1430. 2011-10-27 02:14:47 +02:00
Shay Banon efffcf2b26 add 0.18.2 option 2011-10-27 01:43:08 +02:00
Shay Banon 99e2bb270c External data location configuration fails (path.data setting), closes #1429. 2011-10-27 01:30:33 +02:00
Shay Banon 49a97f6fcf move to 0.19.0 snap 2011-10-26 21:04:10 +02:00
Shay Banon 3544b07330 release 0.18.0 2011-10-26 20:58:24 +02:00
Shay Banon 0cde90fcb1 minimum_number_should_match in a query_string, closes #1420. 2011-10-26 01:58:12 +02:00
Shay Banon b652c655d2 Highlighting broken with custom_filters_score query, closes #1425. 2011-10-26 00:58:52 +02:00
Shay Banon 2778a6756c indexing option ignored in index_stats, closes #1423. 2011-10-25 20:20:43 +02:00
Shay Banon f9d0e82389 if source is not stored, treat it as an empty map in things like scripts or non stored field extraction 2011-10-25 20:18:53 +02:00
Shay Banon 8d08a66e66 make flags public just so people can hack it if needed 2011-10-25 19:01:12 +02:00
Shay Banon 6f39fce41f First indexing fails when indexing a type with an attachment field mapped using a dynamic template, closes #1418. 2011-10-25 01:40:27 +02:00
Shay Banon 3f4eea469c better failure message if a type parser is not found with dynamic tempaltes 2011-10-25 01:10:49 +02:00
Shay Banon 3210785bf4 upgrade to jackson 1.9.1 2011-10-25 00:30:05 +02:00
Shay Banon fd23d16eaa Mapping: Allow to include / exclude specific parts of it when storing it, closes #1421. 2011-10-25 00:01:58 +02:00
Shay Banon dec8dc5bec Bool Filter fails with ArrayIndexOutOfBoundsException, closes #1419. 2011-10-24 00:30:30 +02:00
Shay Banon d8f426a166 remove unused class 2011-10-23 23:41:18 +02:00
Shay Banon a360cc4c4a on multiple data locations, write the state files of local gateway on all locations 2011-10-23 22:57:37 +02:00
Shay Banon b2b608f9dc Query DSL: Indices query type, closes #1416. 2011-10-23 01:35:09 +02:00
Shay Banon 721f315210 handle also illegal state case to see if OOM happened within the index writer 2011-10-21 20:18:57 +02:00
Hiroyuki Ikezoe 6444335f06 Use CJKAnalyzer instead of ChineseAnalyzer
Signed-off-by: Hiroyuki Ikezoe <hiikezoe@gnome.org>
2011-10-21 20:04:32 +02:00
Shay Banon 932215d6fa rename check setting to check on startup 2011-10-20 16:17:38 +02:00
Shay Banon ff977926a9 add equals support to index metadata 2011-10-20 00:51:24 +02:00
Lukas Vlcek 54273dd1ee Added tests for synonym parsers. 2011-10-19 23:38:15 +02:00
Lukáš Vlček f7747d7ff9 Adding Wordnet synonym format 2011-10-19 23:38:14 +02:00
Shay Banon 0f2b875df9 better failure message when failing to get a node lock 2011-10-19 23:12:01 +02:00
Shay Banon 9396735925 Script sorting doesn't accept parameters, closes #1382. 2011-10-18 22:31:20 +02:00
Shay Banon 724611f249 don't clear on both obtain and release for the search cache 2011-10-18 19:21:06 +02:00
Shay Banon 4ffbdbeaea better failure message for sorting when mapping is not found 2011-10-18 05:54:40 +02:00
Shay Banon b113eb18fe builder based search source is safe now 2011-10-18 05:53:53 +02:00
Shay Banon 184a38439f Search: A failed search request might get overrun when trying another shard, closes #1403. 2011-10-18 05:31:12 +02:00
Shay Banon 673655cc7b When searching against an index/type, use the type information to derive different search aspects, closes #1391. 2011-10-18 01:26:53 +02:00
Shay Banon 6a146e7ad0 When searching against an index/type, use the type information to derive different search aspects, closes #1391. 2011-10-18 00:38:29 +02:00
Shay Banon 325064c7aa add static close methods to main class entry point 2011-10-17 20:25:41 +02:00
Shay Banon bfb4666b3d remove casting 2011-10-17 00:43:25 +02:00
Shay Banon bcd582b8c1 add internal cluster reroute api 2011-10-16 19:02:32 +02:00
Shay Banon 1047cebabe replication base classes to allow for different implementation of the primary request and the replica request 2011-10-16 17:25:31 +02:00
Shay Banon 72ad722480 Array out-of-bounds exception with bool filter, closes #1400. 2011-10-15 15:35:14 +02:00
Erick Tryzelaar b2769b1086 multi_get ids shortcut should grab custom fields. 2011-10-15 11:56:55 +02:00
Shay Banon 6b497589ce rest _count endpoint to also allow for no query provided (defaults to match_all) 2011-10-14 17:53:27 +02:00
Shay Banon a76737ceaf also ignore index missing failure when refreshing, as it might happen (we refresh on shards that might have not been allocated yet) 2011-10-14 17:50:02 +02:00
Shay Banon dee4b42b16 Exception: TokenStream implementation classes or at least their incrementToken() implementation must be final, closes #1397. 2011-10-14 16:56:53 +02:00
Shay Banon 24f1f0ff96 improve refreshing logic to resync mappings on upgrade, reduce the amount of cluster events processing requires if the even if fired from several nodes / sources 2011-10-14 15:20:38 +02:00
Shay Banon ffc74260ac Synonym filter fails to properly load synonyms (since 0.17.7), closes #1392. 2011-10-13 00:18:01 +02:00
Shay Banon 19ff93b09a all field mapper should be "included in root object" 2011-10-12 22:50:16 +02:00
Shay Banon 0367103423 Text Query does not use search_analyzer on field that is prefixed with the type name, closes #1390. 2011-10-12 22:14:03 +02:00
Shay Banon 1012f6a3ea Root mapper analyzer/index_analyzer/search_analyzer with unregistered analyzer name won't fail, closes #1389. 2011-10-12 21:41:27 +02:00
Shay Banon bb9d8fe342 Search / Get Preference: Add _only_node:[node_id] option, closes #1388. 2011-10-12 21:11:12 +02:00
Igor Motov bd87f8de3a Fix NPE in HighlightField serialization. Fixes #1380 2011-10-08 20:54:56 +02:00
Shay Banon 4bbe1b111a count doesn't work for empty query (discrepancy with search API), closes #1377. 2011-10-06 21:57:19 +02:00
Shay Banon 768c6d2922 fix buffer recycler in lzf 2011-10-06 20:40:43 +02:00
Shay Banon 9c6dfc1508 now that the change to refresh can execute on not yet active shards, we need to ignore illegal shard state failures (expected...) 2011-10-05 23:35:07 +02:00
Shay Banon d69baa3e04 when writing _river status, use consistency of one 2011-10-05 19:12:21 +02:00
Shay Banon 8f55e8835f when getting the _meta doc from _river index (per river), make sure the callback listener is threaded 2011-10-05 17:29:11 +02:00
Shay Banon 111c472a0b upgrade to jackson 1.9.0 2011-10-05 13:32:33 +02:00
Shay Banon 63844ddd43 REST Bulk API: Allow to execute _bulk against /{index}/_bulk and /{index}/{type}/_bulk endpoints, closes #1375. 2011-10-05 13:03:30 +02:00