Commit Graph

4892 Commits

Author SHA1 Message Date
Ryan Ernst 8734540345
Ensure map keys cannot be self referencing (#34569)
This commit improves self reference checking to map keys, as well as
adds it to ingest script processing.
2018-10-17 15:16:13 -07:00
Nik Everett 1452d55155
Expressions: Wrap at 140 columns (#34493)
Apply our standard line length to all of the java files in the
`lang-expression` plugin.
2018-10-17 16:12:21 -04:00
Armin Braun 3954d041a0
SCRIPTING: Move sort Context to its Own Class (#33717)
* SCRIPTING: Move sort Context to its own Class
2018-10-17 10:02:44 +01:00
Armin Braun ebca27371c
SCRIPTING: Move Aggregation Script Context to its own class (#33820)
* SCRIPTING: Move Aggregation Script Context to its own class
2018-10-15 17:28:05 +01:00
Armin Braun 511526250b
MINOR: Remove Deadcode in ExpressionTermSetQuery (#34442) 2018-10-15 17:26:08 +01:00
Jack Conradson a7c4dbbb0c
[Painless] Add a Map for java names to classes for use in the custom classloader (#34424)
This fixes a bug that wasn't including the class used for a static import 
where only the static import is whitelisted.
2018-10-12 15:34:48 -07:00
Armin Braun 7ba320d5b2
NETWORKING: Upgrade Netty to 4.1.30 (#34417)
* closes #34411
2018-10-12 16:13:28 +01:00
Jack Conradson 4270085360
[Painless] Allow statically imported methods without whitelist class. (#34370)
This removes the extraneous check to see if the class for a statically imported 
method is already whitelisted, so a statically imported method can be whitelisted
independently.
2018-10-10 10:00:45 -07:00
Nik Everett 06993e0c35
Logging: Make ESLoggerFactory package private (#34199)
Since all calls to `ESLoggerFactory` outside of the logging package were
deprecated, it seemed like it'd simplify things to migrate all of the
deprecated calls and declare `ESLoggerFactory` to be package private.
This does that.
2018-10-06 09:54:08 -04:00
eray daf88335d7 Add max_children limit to nested sort (#33587)
Add an option to `nested` sort to limit the number of children to visit when picking the sort value
of the root document. 

Closes #33592
2018-10-05 12:02:47 +02:00
Armin Braun 732ab06ee4
MINOR: Remove Dead Code from Netty4Transport (#34134)
* None of these methods are used
2018-10-05 10:53:03 +02:00
Armin Braun 3ccfc3de58
SCRIPTING: Terms set query expression (#33856)
* SCRIPTING: Add Expr. Compile for TermSetQuery Ctx.

* Follow up to #33602 adding the ability to compile TermsSetQuery
scripts with the expressions engine in the same way we support
SearchScript in Expressions
   * Duplicated the code here for now to make the change less complex,
 the only difference to SearchScript is that `_score` and `_value` are not handled for TermsSetQuery
* remove redundant check
2018-10-04 16:03:57 +02:00
Kazuhiro Sera d45fe43a68 Fix a variety of typos and misspelled words (#32792) 2018-10-03 18:11:38 +01:00
Christoph Büscher 5183ea3d68
Use OptionalInt instead of Optional<Integer> (#34220)
Optionals containing boxed primitive types are prohibitively costly because they
have two level of boxing. For Optional<Integer> the analogous OptionalInt can be
used to avoid the boxing of the contained int value.
2018-10-02 15:58:07 +02:00
Alan Woodward d25e3ef065 [CI] Use a single shard for synonym REST tests, to ensure score ordering
is stable
2018-10-01 09:47:22 +01:00
Ryan Ernst 47cbae9b26
Scripting: Remove ExecutableScript (#34154)
This commit removes the legacy ExecutableScript, which was no longer
used except in tests. All uses have previously been converted to script
contexts.
2018-09-28 17:13:08 -07:00
Jack Conradson bc84c8ecd6
Painless: Remove caching of Painless scripts in GenericElasticsearchScript (#34116)
When a script uses the special-cased SearchScript or ExecutableScript, Painless is 
internally caching the GenericElasticsearchScript that is generated via compile as part of 
the newInstance method for the factories. This breaks the model for class bindings as the 
class binding expects to not have any stored information for each instance generated via 
newInstance from the factory. This change fixes the issue by creating a new instance of 
the GenericElasticsearchScript each time newInstance is called against the factory.
2018-09-28 09:11:41 -07:00
Alan Woodward f243d75f59
Remove special-casing of Synonym filters in AnalysisRegistry (#34034)
The synonym filters no longer need access to the AnalysisRegistry in their
constructors, so we can remove the special-case code and move them to the
common analysis module.

This commit means that synonyms are no longer available for `server` integration tests,
so several of these are either rewritten or migrated to the common analysis module
as rest-spec-api tests
2018-09-28 09:02:47 +01:00
Ryan Ernst a2c941806b
Tests: Add support for custom contexts to mock scripts (#34100)
This commit adds the ability to plug in compilation of custom contexts
in mock script engine. This is needed for testing plugins which add
custom contexts like watcher.
2018-09-27 12:23:59 -07:00
Alan Woodward 1cc53b5988
Bad regex in CORS settings should throw a nicer error (#34035)
Currently a bad regex in CORS settings throws a PatternSyntaxException, which
then bubbles up through the bootstrap code, meaning users have to parse a
stack trace to work out where the problem is.  We should instead catch this
exception and rethrow with a more useful error message.
2018-09-27 10:02:29 +01:00
Ryan Ernst 85e4ef3429
Scripting: Reflect factory signatures in painless classloader (#34088)
It is sometimes desirable to pass a class into a script constructor that
will not actually be exposed in the script whitelist. This commit uses
reflection when creating the compiler to find all the classes of the
factory method signature, and make the classloader that wraps lookup
also expose these classes.
2018-09-26 13:22:48 -07:00
Jack Conradson 4fbe84edf6
Painless: Cleanup Cache (#33963)
* Adds equals/hashcode methods to the Painless* objects within the lookup package.
* Changes the caches to use the Painless* objects as keys as well as values. This forces 
future changes to taken into account the appropriate values for caching.
* Deletes the existing caching objects in favor of Painless* objects. This removes a pair of 
bugs that were not taking into account subtle corner cases related to augmented methods 
and caching.
* Uses the Painless* objects to check for equivalency in existing Painless* objects that 
may have already been added to a PainlessClass. This removes a bug related to return 
not being appropriately checked when adding methods.
* Cleans up several error messages.
2018-09-26 09:33:08 -07:00
Christoph Büscher ba3ceeaccf
Clean up "unused variable" warnings (#31876)
This change cleans up "unused variable" warnings. There are several cases were we 
most likely want to suppress the warnings (especially in the client documentation test
where the snippets contain many unused variables). In a lot of cases the unused
variables can just be deleted though.
2018-09-26 14:09:32 +02:00
Ryan Ernst be8475955e
Scripting: Use ParameterMap for deprecated ctx var in update scripts (#34065)
This commit removes the sysprop controlling whether ctx is in params for
update scripts and replaces it with use of the new ParameterMap, which
outputs a deprecation warning whenever params.ctx is used.
2018-09-25 22:08:02 -07:00
Armin Braun 0ba1855740
INGEST: Tests for Drop Processor (#33430)
* INGEST: Tests for Drop Processor

* UT for behavior of dropped callback
and drop processor
   * Moved drop processor to `server`
project to enable this test
* Simple IT
* Relates #32278
2018-09-25 19:29:22 +02:00
Armin Braun 2ad06f6e67
NETWORKING: Upgrade to Netty 4.1.29 (#33984) 2018-09-25 08:46:38 +02:00
Tim Brooks 78e483e8d8
Introduce abstract security transport testcase (#33878)
This commit introduces an AbstractSimpleSecurityTransportTestCase for
security transports. This classes provides transport tests that are
specific for security transports. Additionally, it fixes the tests referenced in
#33285.
2018-09-24 09:44:44 -06:00
Christoph Büscher b654d986d7
Add OneStatementPerLineCheck to Checkstyle rules (#33682)
This change adds the OneStatementPerLineCheck to our checkstyle precommit
checks. This rule restricts the number of statements per line to one. The
resoning behind this is that it is very difficult to read multiple statements on
one line. People seem to mostly use it in short lambdas and switch statements in
our code base, but just going through the changes already uncovered some actual
problems in randomization in test code, so I think its worth it.
2018-09-21 11:52:31 +02:00
Alan Woodward b33c18d316
Move SoraniNormalizationFilterFactory to the common analysis plugin (#33892)
Follow up to #25715
2018-09-20 17:31:41 +01:00
Nik Everett f963c29876
Logging: Drop Settings from some logger lookups (#33859)
Drops `Settings` from some of the methods to lookup loggers and
deprecates another logger lookup that takes `Settings` because
`Settings` is no longer required to build a logger.
2018-09-20 10:42:48 -04:00
Jake Landis e37e5dfc04
ingest: support simulate with verbose for pipeline processor (#33839)
* ingest: support simulate with verbose for pipeline processor

This change better supports the use of simulate?verbose with the
pipeline processor. Prior to this change any pipeline processors
executed with simulate?verbose would not show all intermediate 
processors for the inner pipelines.

This changes also moves the PipelineProcess and TrackingResultProcessor
classes to enable instance checks and to avoid overly public classes.
As well this updates the error message for when cycles are detected
in pipelines calling other pipelines.
2018-09-20 08:33:07 -05:00
Jack Conradson 3df285d9f0
Painless: Rename *Binding classes to *ClassBinding. (#33865)
With the upcoming instance bindings, the singular *Binding name isn't descriptive enough 
with multiple binding types. This renames the existing *Binding classes to *ClassBinding. 
Mechanical change (with some error messages changed from binding to class binding by 
hand).
2018-09-19 12:19:44 -07:00
Alan Woodward 5107949402
Allow TokenFilterFactories to rewrite themselves against their preceding chain (#33702)
We currently special-case SynonymFilterFactory and SynonymGraphFilterFactory, which need to 
know their predecessors in the analysis chain in order to correctly analyze their synonym lists. This
special-casing doesn't work with Referring filter factories, such as the Multiplexer or Conditional
filters. We also have a number of filters (eg the Multiplexer) that will break synonyms when they
appear before them in a chain, because they produce multiple tokens at the same position.

This commit adds two methods to the TokenFilterFactory interface.

* `getChainAwareTokenFilterFactory()` allows a filter factory to rewrite itself against its preceding
  filter chain, or to resolve references to other filters. It replaces `ReferringFilterFactory` and
  `CustomAnalyzerProvider.checkAndApplySynonymFilter`, and by default returns `this`.
* `getSynonymFilter()` defines whether or not a filter should be applied when building a synonym
  list `Analyzer`. By default it returns `true`.

Fixes #33609
2018-09-19 15:52:14 +01:00
Armin Braun c6462057a1
MINOR: Remove Some Dead Code in Scripting (#33800)
* The is default check method is not used in ScriptType
* The removed vars on ExpressionSearchScript are unused
2018-09-18 20:43:31 +02:00
Jim Ferenczi 241c74efb2
upgrade to a new snapshot of Lucene 8 (7d0a7782fa) (#33812) 2018-09-18 18:16:40 +02:00
Armin Braun 87cedef3cf
NETWORKING:Def CName in Http Publish Addr to True (#33631)
* Follow up to #32806 setting the setting to true for 7.x
2018-09-18 10:29:02 +02:00
Armin Braun 615f494c77
MINOR: Drop Redundant Ctx. Check in ScriptService (#33782)
* MINOR: Drop Redundant Ctx. Check in ScriptService

* This check is completely redundant, the expression script
engine will throw anyway (and with a similar message) for
those contexts that it cannot compile. Moreover, the update context
is not the only context that is not suported by the expression engine
at this point so handling the update context separately here makes
no sense.
2018-09-18 07:25:22 +02:00
Or Bin a5bad4d92c Docs: Fixed a grammatical mistake: 'a HTTP ...' -> 'an HTTP ...' (#33744)
Fixed a grammatical mistake: 'a HTTP ...' -> 'an HTTP ...'

Closes #33728
2018-09-17 15:35:54 -04:00
Ryan Ernst 3046656ab1
Scripting: Rework joda time backcompat (#33486)
This commit switches the joda time backcompat in scripting to use
augmentation over ZonedDateTime. The augmentation methods provide
compatibility with the missing methods between joda's DateTime and
java's ZonedDateTime. Due to getDayOfWeek returning an enum in the java
API, ZonedDateTime is wrapped so that the method can return int like the
joda time does. The java time api version is renamed to
getDayOfWeekEnum, which will be kept through 7.x for compatibility while
users switch back to getDayOfWeek once joda compatibility is removed.
2018-09-16 19:18:00 -07:00
Christoph Büscher c9131983f5
[Docs] Minor fix in `has_child` javadoc comment (#33674)
The min and max constants are accidentaly the wrong way around.
2018-09-14 09:41:20 +02:00
Armin Braun 94cdf0ceba
NETWORKING: http.publish_host Should Contain CNAME (#32806)
* NETWORKING: http.publish_host Should Contain CNAME

* Closes #22029
2018-09-12 06:15:36 +02:00
Alan Woodward f598297f55
Add predicate_token_filter (#33431)
This allows users to filter out tokens from a TokenStream using painless scripts, 
instead of having to write specialised Java code and packaging it up into a plugin.

The commit also refactors the AnalysisPredicateScript.Token class so that it wraps
and makes read-only an AttributeSource.
2018-09-11 09:16:39 +01:00
Alan Woodward 39c3234c2f
Upgrade to latest Lucene snapshot (#33505)
* LeafCollector.setScorer() now takes a Scorable
* Scorers may not have null Weights
* IndexWriter.getFlushingBytes() reports how much memory is being used by IW threads writing to disk
2018-09-10 20:51:55 +01:00
Tanguy Leroux 079d130d8c
[Test] Remove duplicate method in TestShardRouting (#32815) 2018-09-10 18:29:00 +02:00
Tanguy Leroux 80c4661d0c
Fix typos (#33499) 2018-09-10 14:20:04 +02:00
Jack Conradson facec187bb
Painless: Add Imported Static Method (#33440)
Allow static methods to be imported in Painless and called using just the method name.
2018-09-07 18:16:07 -07:00
Jim Ferenczi 79cd6385fe
Collapse package structure for metrics aggs (#33463)
This change collapses all metrics aggregations classes into a single package `org.elasticsearch.aggregations.metrics`.
It also restricts the visibility of some classes (aggregators and factories) that should not be used outside of the package.

Relates #22868
2018-09-07 10:58:06 +02:00
Jim Ferenczi 7ad71f906a
Upgrade to a Lucene 8 snapshot (#33310)
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
2018-09-06 14:42:06 +02:00
Alan Woodward e134f9b5f3
Fix generics in ScriptPlugin#getContexts() (#33426)
Changes the return value from List<ScriptContext> to List<ScriptContext<?>> to remove raw-types warnings.
2018-09-06 09:04:22 +01:00
Tim Brooks b697f485bb
Introduce `TransportLogger` for common logging (#32725)
Historically we have had a ESLoggingHandler in the netty module that
logs low-level connection operations. This class just extends the netty
logging handler with some (broken) message deserialization. This commit
fixes this message serialization and moves the class to server.

This new logger logs inbound and outbound messages. Eventually, we
should move other event logging to this class (connect, close, flush).
That way we will have consistent logging regards of which transport is
loaded.

Resolves #27306 on master. Older branches will need a different fix.
2018-09-05 16:12:37 -06:00