Commit Graph

4905 Commits

Author SHA1 Message Date
Alpar Torok 59536966c2
Add a new "contains" feature (#34738)
The contains syntax was added in #30874 but the skips were not properly
put in place.
The java runner has the feature so the tests will run as part of the
build, but language clients will be able to support it at their own
pace.
2018-10-25 08:50:50 +03:00
Ryan Ernst 687dc1eb11
Scripting: Remove SearchScript (#34730)
This commit removes the last non context based script class.
2018-10-24 15:03:38 -07:00
Jack Conradson 1b085252c3
[Painless] Add instance bindings (#34410)
This change adds instance bindings to Painless. This binding allows a whitelisted
 method to be called on an instance instantiated prior to script compilation. 
Whitelisting must be done in code as there is no practical way to instantiate a 
useful instance from a text file (see the tests for an example). Since an 
instance can be shared by multiple scripts, each method called must be 
thread-safe.
2018-10-24 12:57:28 -07:00
Andrey Atapin 5f588180f9 Improve IndexNotFoundException's default error message (#34649)
This commit adds the index name to the error message when an index is not found.
2018-10-24 12:53:31 -07:00
Luca Cavanna d51bc05dce
[TEST] Improve validation of do sections (#34734)
We throw parsing exception when an unknown array is found, but we don't when an unknown top-level field is found. This commit makes sure that unsupported top-level fields are not ignored in a do section.

Closes #34651
2018-10-24 21:27:07 +02:00
Nik Everett e0d7808148 REST: No strict warning testing for head body test
Our tests that HEAD requests don't have a body was hitting a warning.
For now we'll run that test without "strict" warnings mode enabled.
2018-10-24 11:17:38 -04:00
Ryan Ernst 8da1c9626a
Scripting: Add back params._source access in scripted metric aggs (#34777)
Access to special variables _source and _fields were accidentally
removed in recent refactorings. This commit adds them back, along with a
test.

closes #33884
2018-10-23 18:07:53 -07:00
Zachary Tong 299d044bfc
Collapse pipeline aggs into single package (#34658)
- Restrict visibility of Aggregators and Factories
- Move PipelineAggregatorBuilders up a level so it is consistent with
AggregatorBuilders
- Checkstyle line length fixes for a few classes
- Minor odds/ends (swapping to method references, formatting, etc)
2018-10-23 16:01:01 -04:00
Jake Landis 89dc07bdd9
ingest: better support for conditionals with simulate?verbose (#34155)
This commit introduces two corrections to the way simulate?verbose
handles conditionals on processors.

1) Prior to this change when executing simulate?verbose for
processors with conditionals that evaluate to false, that processor
would still be displayed in the result set. What was displayed was
correct, such that no changes to the document occurred. However, if the
conditional evaluates to false, the processor should not even be
displayed.

2) Prior to this change when executing simulate?verbose for
pipeline processors with conditionals, the individual steps would no
longer be displayed. Commit e37e5df addressed the issue, but
failed account for a conditional on the pipeline processor. Since
a pipeline processor can introduce cycles and is effectively a
single processor that encapsulates multiple other processors that
are potentially guarded by a single conditional, special handling is
needed to for pipeline and conditional pipeline processors.
2018-10-23 11:33:48 -05:00
Armin Braun 8e155b8430
INGEST: Rename Pipeline Processor Param. (#34733)
* `name` is more readable/ergnomic than having `pipeline` twice
2018-10-23 13:43:26 +02:00
Ryan Ernst 222652dfce
Scripting: Convert script fields to use script context (#34164)
This commit removes the use of SearchScript for script fields and adds
a new FieldScript.
2018-10-20 16:33:49 -07:00
Jason Tedor 48ab6d7c92
Switch old reindex tests to use use JDK 8 (#34646)
This commit switches the old reindex tests to use JDK 8. This enables us
to not require that JDK 7 be installed, easing the burden on developers.
2018-10-19 15:11:48 -04:00
Christoph Büscher c1c447a4cf
Check stemmer language setting early (#34601)
Currently the StemmerTokenFilterFactory checks the validity of the language
setting only when the first TokenStream is processed. Instead we should throw an
error earlier at mapping creation time. This change adds a check to the
StemmerTokenFilterFactory constructor that checks for a valid `language` setting
by trying to create a new TokenStream from an empty input stream. This will
throw errors about wrong language settings early on.

Closes #34170
2018-10-19 12:59:23 +02:00
Ryan Ernst 8734540345
Ensure map keys cannot be self referencing (#34569)
This commit improves self reference checking to map keys, as well as
adds it to ingest script processing.
2018-10-17 15:16:13 -07:00
Nik Everett 1452d55155
Expressions: Wrap at 140 columns (#34493)
Apply our standard line length to all of the java files in the
`lang-expression` plugin.
2018-10-17 16:12:21 -04:00
Armin Braun 3954d041a0
SCRIPTING: Move sort Context to its Own Class (#33717)
* SCRIPTING: Move sort Context to its own Class
2018-10-17 10:02:44 +01:00
Armin Braun ebca27371c
SCRIPTING: Move Aggregation Script Context to its own class (#33820)
* SCRIPTING: Move Aggregation Script Context to its own class
2018-10-15 17:28:05 +01:00
Armin Braun 511526250b
MINOR: Remove Deadcode in ExpressionTermSetQuery (#34442) 2018-10-15 17:26:08 +01:00
Jack Conradson a7c4dbbb0c
[Painless] Add a Map for java names to classes for use in the custom classloader (#34424)
This fixes a bug that wasn't including the class used for a static import 
where only the static import is whitelisted.
2018-10-12 15:34:48 -07:00
Armin Braun 7ba320d5b2
NETWORKING: Upgrade Netty to 4.1.30 (#34417)
* closes #34411
2018-10-12 16:13:28 +01:00
Jack Conradson 4270085360
[Painless] Allow statically imported methods without whitelist class. (#34370)
This removes the extraneous check to see if the class for a statically imported 
method is already whitelisted, so a statically imported method can be whitelisted
independently.
2018-10-10 10:00:45 -07:00
Nik Everett 06993e0c35
Logging: Make ESLoggerFactory package private (#34199)
Since all calls to `ESLoggerFactory` outside of the logging package were
deprecated, it seemed like it'd simplify things to migrate all of the
deprecated calls and declare `ESLoggerFactory` to be package private.
This does that.
2018-10-06 09:54:08 -04:00
eray daf88335d7 Add max_children limit to nested sort (#33587)
Add an option to `nested` sort to limit the number of children to visit when picking the sort value
of the root document. 

Closes #33592
2018-10-05 12:02:47 +02:00
Armin Braun 732ab06ee4
MINOR: Remove Dead Code from Netty4Transport (#34134)
* None of these methods are used
2018-10-05 10:53:03 +02:00
Armin Braun 3ccfc3de58
SCRIPTING: Terms set query expression (#33856)
* SCRIPTING: Add Expr. Compile for TermSetQuery Ctx.

* Follow up to #33602 adding the ability to compile TermsSetQuery
scripts with the expressions engine in the same way we support
SearchScript in Expressions
   * Duplicated the code here for now to make the change less complex,
 the only difference to SearchScript is that `_score` and `_value` are not handled for TermsSetQuery
* remove redundant check
2018-10-04 16:03:57 +02:00
Kazuhiro Sera d45fe43a68 Fix a variety of typos and misspelled words (#32792) 2018-10-03 18:11:38 +01:00
Christoph Büscher 5183ea3d68
Use OptionalInt instead of Optional<Integer> (#34220)
Optionals containing boxed primitive types are prohibitively costly because they
have two level of boxing. For Optional<Integer> the analogous OptionalInt can be
used to avoid the boxing of the contained int value.
2018-10-02 15:58:07 +02:00
Alan Woodward d25e3ef065 [CI] Use a single shard for synonym REST tests, to ensure score ordering
is stable
2018-10-01 09:47:22 +01:00
Ryan Ernst 47cbae9b26
Scripting: Remove ExecutableScript (#34154)
This commit removes the legacy ExecutableScript, which was no longer
used except in tests. All uses have previously been converted to script
contexts.
2018-09-28 17:13:08 -07:00
Jack Conradson bc84c8ecd6
Painless: Remove caching of Painless scripts in GenericElasticsearchScript (#34116)
When a script uses the special-cased SearchScript or ExecutableScript, Painless is 
internally caching the GenericElasticsearchScript that is generated via compile as part of 
the newInstance method for the factories. This breaks the model for class bindings as the 
class binding expects to not have any stored information for each instance generated via 
newInstance from the factory. This change fixes the issue by creating a new instance of 
the GenericElasticsearchScript each time newInstance is called against the factory.
2018-09-28 09:11:41 -07:00
Alan Woodward f243d75f59
Remove special-casing of Synonym filters in AnalysisRegistry (#34034)
The synonym filters no longer need access to the AnalysisRegistry in their
constructors, so we can remove the special-case code and move them to the
common analysis module.

This commit means that synonyms are no longer available for `server` integration tests,
so several of these are either rewritten or migrated to the common analysis module
as rest-spec-api tests
2018-09-28 09:02:47 +01:00
Ryan Ernst a2c941806b
Tests: Add support for custom contexts to mock scripts (#34100)
This commit adds the ability to plug in compilation of custom contexts
in mock script engine. This is needed for testing plugins which add
custom contexts like watcher.
2018-09-27 12:23:59 -07:00
Alan Woodward 1cc53b5988
Bad regex in CORS settings should throw a nicer error (#34035)
Currently a bad regex in CORS settings throws a PatternSyntaxException, which
then bubbles up through the bootstrap code, meaning users have to parse a
stack trace to work out where the problem is.  We should instead catch this
exception and rethrow with a more useful error message.
2018-09-27 10:02:29 +01:00
Ryan Ernst 85e4ef3429
Scripting: Reflect factory signatures in painless classloader (#34088)
It is sometimes desirable to pass a class into a script constructor that
will not actually be exposed in the script whitelist. This commit uses
reflection when creating the compiler to find all the classes of the
factory method signature, and make the classloader that wraps lookup
also expose these classes.
2018-09-26 13:22:48 -07:00
Jack Conradson 4fbe84edf6
Painless: Cleanup Cache (#33963)
* Adds equals/hashcode methods to the Painless* objects within the lookup package.
* Changes the caches to use the Painless* objects as keys as well as values. This forces 
future changes to taken into account the appropriate values for caching.
* Deletes the existing caching objects in favor of Painless* objects. This removes a pair of 
bugs that were not taking into account subtle corner cases related to augmented methods 
and caching.
* Uses the Painless* objects to check for equivalency in existing Painless* objects that 
may have already been added to a PainlessClass. This removes a bug related to return 
not being appropriately checked when adding methods.
* Cleans up several error messages.
2018-09-26 09:33:08 -07:00
Christoph Büscher ba3ceeaccf
Clean up "unused variable" warnings (#31876)
This change cleans up "unused variable" warnings. There are several cases were we 
most likely want to suppress the warnings (especially in the client documentation test
where the snippets contain many unused variables). In a lot of cases the unused
variables can just be deleted though.
2018-09-26 14:09:32 +02:00
Ryan Ernst be8475955e
Scripting: Use ParameterMap for deprecated ctx var in update scripts (#34065)
This commit removes the sysprop controlling whether ctx is in params for
update scripts and replaces it with use of the new ParameterMap, which
outputs a deprecation warning whenever params.ctx is used.
2018-09-25 22:08:02 -07:00
Armin Braun 0ba1855740
INGEST: Tests for Drop Processor (#33430)
* INGEST: Tests for Drop Processor

* UT for behavior of dropped callback
and drop processor
   * Moved drop processor to `server`
project to enable this test
* Simple IT
* Relates #32278
2018-09-25 19:29:22 +02:00
Armin Braun 2ad06f6e67
NETWORKING: Upgrade to Netty 4.1.29 (#33984) 2018-09-25 08:46:38 +02:00
Tim Brooks 78e483e8d8
Introduce abstract security transport testcase (#33878)
This commit introduces an AbstractSimpleSecurityTransportTestCase for
security transports. This classes provides transport tests that are
specific for security transports. Additionally, it fixes the tests referenced in
#33285.
2018-09-24 09:44:44 -06:00
Christoph Büscher b654d986d7
Add OneStatementPerLineCheck to Checkstyle rules (#33682)
This change adds the OneStatementPerLineCheck to our checkstyle precommit
checks. This rule restricts the number of statements per line to one. The
resoning behind this is that it is very difficult to read multiple statements on
one line. People seem to mostly use it in short lambdas and switch statements in
our code base, but just going through the changes already uncovered some actual
problems in randomization in test code, so I think its worth it.
2018-09-21 11:52:31 +02:00
Alan Woodward b33c18d316
Move SoraniNormalizationFilterFactory to the common analysis plugin (#33892)
Follow up to #25715
2018-09-20 17:31:41 +01:00
Nik Everett f963c29876
Logging: Drop Settings from some logger lookups (#33859)
Drops `Settings` from some of the methods to lookup loggers and
deprecates another logger lookup that takes `Settings` because
`Settings` is no longer required to build a logger.
2018-09-20 10:42:48 -04:00
Jake Landis e37e5dfc04
ingest: support simulate with verbose for pipeline processor (#33839)
* ingest: support simulate with verbose for pipeline processor

This change better supports the use of simulate?verbose with the
pipeline processor. Prior to this change any pipeline processors
executed with simulate?verbose would not show all intermediate 
processors for the inner pipelines.

This changes also moves the PipelineProcess and TrackingResultProcessor
classes to enable instance checks and to avoid overly public classes.
As well this updates the error message for when cycles are detected
in pipelines calling other pipelines.
2018-09-20 08:33:07 -05:00
Jack Conradson 3df285d9f0
Painless: Rename *Binding classes to *ClassBinding. (#33865)
With the upcoming instance bindings, the singular *Binding name isn't descriptive enough 
with multiple binding types. This renames the existing *Binding classes to *ClassBinding. 
Mechanical change (with some error messages changed from binding to class binding by 
hand).
2018-09-19 12:19:44 -07:00
Alan Woodward 5107949402
Allow TokenFilterFactories to rewrite themselves against their preceding chain (#33702)
We currently special-case SynonymFilterFactory and SynonymGraphFilterFactory, which need to 
know their predecessors in the analysis chain in order to correctly analyze their synonym lists. This
special-casing doesn't work with Referring filter factories, such as the Multiplexer or Conditional
filters. We also have a number of filters (eg the Multiplexer) that will break synonyms when they
appear before them in a chain, because they produce multiple tokens at the same position.

This commit adds two methods to the TokenFilterFactory interface.

* `getChainAwareTokenFilterFactory()` allows a filter factory to rewrite itself against its preceding
  filter chain, or to resolve references to other filters. It replaces `ReferringFilterFactory` and
  `CustomAnalyzerProvider.checkAndApplySynonymFilter`, and by default returns `this`.
* `getSynonymFilter()` defines whether or not a filter should be applied when building a synonym
  list `Analyzer`. By default it returns `true`.

Fixes #33609
2018-09-19 15:52:14 +01:00
Armin Braun c6462057a1
MINOR: Remove Some Dead Code in Scripting (#33800)
* The is default check method is not used in ScriptType
* The removed vars on ExpressionSearchScript are unused
2018-09-18 20:43:31 +02:00
Jim Ferenczi 241c74efb2
upgrade to a new snapshot of Lucene 8 (7d0a7782fa) (#33812) 2018-09-18 18:16:40 +02:00
Armin Braun 87cedef3cf
NETWORKING:Def CName in Http Publish Addr to True (#33631)
* Follow up to #32806 setting the setting to true for 7.x
2018-09-18 10:29:02 +02:00
Armin Braun 615f494c77
MINOR: Drop Redundant Ctx. Check in ScriptService (#33782)
* MINOR: Drop Redundant Ctx. Check in ScriptService

* This check is completely redundant, the expression script
engine will throw anyway (and with a similar message) for
those contexts that it cannot compile. Moreover, the update context
is not the only context that is not suported by the expression engine
at this point so handling the update context separately here makes
no sense.
2018-09-18 07:25:22 +02:00