Commit Graph

519 Commits

Author SHA1 Message Date
Christoph Büscher b654d986d7
Add OneStatementPerLineCheck to Checkstyle rules (#33682)
This change adds the OneStatementPerLineCheck to our checkstyle precommit
checks. This rule restricts the number of statements per line to one. The
resoning behind this is that it is very difficult to read multiple statements on
one line. People seem to mostly use it in short lambdas and switch statements in
our code base, but just going through the changes already uncovered some actual
problems in randomization in test code, so I think its worth it.
2018-09-21 11:52:31 +02:00
Jack Conradson 3df285d9f0
Painless: Rename *Binding classes to *ClassBinding. (#33865)
With the upcoming instance bindings, the singular *Binding name isn't descriptive enough 
with multiple binding types. This renames the existing *Binding classes to *ClassBinding. 
Mechanical change (with some error messages changed from binding to class binding by 
hand).
2018-09-19 12:19:44 -07:00
Armin Braun 87cedef3cf
NETWORKING:Def CName in Http Publish Addr to True (#33631)
* Follow up to #32806 setting the setting to true for 7.x
2018-09-18 10:29:02 +02:00
Ryan Ernst 3046656ab1
Scripting: Rework joda time backcompat (#33486)
This commit switches the joda time backcompat in scripting to use
augmentation over ZonedDateTime. The augmentation methods provide
compatibility with the missing methods between joda's DateTime and
java's ZonedDateTime. Due to getDayOfWeek returning an enum in the java
API, ZonedDateTime is wrapped so that the method can return int like the
joda time does. The java time api version is renamed to
getDayOfWeekEnum, which will be kept through 7.x for compatibility while
users switch back to getDayOfWeek once joda compatibility is removed.
2018-09-16 19:18:00 -07:00
Armin Braun 94cdf0ceba
NETWORKING: http.publish_host Should Contain CNAME (#32806)
* NETWORKING: http.publish_host Should Contain CNAME

* Closes #22029
2018-09-12 06:15:36 +02:00
Alan Woodward 39c3234c2f
Upgrade to latest Lucene snapshot (#33505)
* LeafCollector.setScorer() now takes a Scorable
* Scorers may not have null Weights
* IndexWriter.getFlushingBytes() reports how much memory is being used by IW threads writing to disk
2018-09-10 20:51:55 +01:00
Jack Conradson facec187bb
Painless: Add Imported Static Method (#33440)
Allow static methods to be imported in Painless and called using just the method name.
2018-09-07 18:16:07 -07:00
Jim Ferenczi 7ad71f906a
Upgrade to a Lucene 8 snapshot (#33310)
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
2018-09-06 14:42:06 +02:00
Alan Woodward e134f9b5f3
Fix generics in ScriptPlugin#getContexts() (#33426)
Changes the return value from List<ScriptContext> to List<ScriptContext<?>> to remove raw-types warnings.
2018-09-06 09:04:22 +01:00
Jack Conradson bbbb11a698
Painless: Fix Bindings Bug (#33274)
When the change was made to the format for in the whitelist for bindings, parameters from
both the constructor and the method were combined into a single list instead of separate 
lists. The check for method parameters was being executed from the start of the combined 
list rather than the correct position. The tests for bindings used a constructor and a method 
that only used the int types so this was not caught. The test has been changed to also use 
a double type and this issue is fixed.
2018-08-30 14:33:32 -07:00
Jack Conradson b52818ec6f
Painless: Add Bindings (#33042)
Add bindings that allow some specialized methods to store permanent state between script executions.
2018-08-29 09:24:56 -07:00
Alexander Reelsen 48b388ce82
Core: Add java time xcontent serializers (#33120)
This ensures that the java time class exposed by painless have proper
serialization/string representations.

Closes #31853
2018-08-29 10:00:16 +02:00
Jack Conradson a381749aac
Painless: Fix Semicolon Regression (#33212)
Trailers (statements following something like an if statement) that don't use brackets currently require a semicolon even if they're the last statement. This is a regression caused by (#29566) and noted by (#33193). This change fixes the regression and adds a test for the broken case.
2018-08-28 12:40:41 -07:00
Alpar Torok 4b34b3f4aa
Set forbidden APIs target compatibility to compiler java version (#32935)
Set forbidden apis target compatibility to compiler version

Fix outstanding deprecation
2018-08-20 09:27:02 +03:00
Jack Conradson d9fd74bcdc
Painless: Special Case def (#32871)
This removes def from the classes map in PainlessLookup and instead always special 
cases it. This prevents potential calls against the def type that shouldn't be made and 
forces all cases of def throughout Painless code to be special cased.
2018-08-16 08:03:21 -07:00
Jack Conradson ab7e7508f5
Painless: Change fqn_only to no_import (#32817)
This changes the whitelist parameter fqn_only to no_import when specifying that a 
whitelisted class must have the fully-qualified-name instead of a shortcut name. This more 
closely correlates with Java imports, hence the rename.
2018-08-14 11:17:15 -07:00
Jack Conradson 47d2bcc3c4
Painless: Clean Up Whitelist Names (#32791)
Renames variables in the whitelists to match the current naming scheme. Mechanical change.
2018-08-10 14:46:31 -07:00
Jack Conradson 7a1bbbc581
Painless: Clean up PainlessCast (#32754)
Renames variables and methods in PainlessCast to go along with the new naming 
scheme. Mechanical change.
2018-08-10 09:55:38 -07:00
Jack Conradson 293c8a2b24
Painless: Add an Ingest Script Processor Example (#32302)
This commit adds two pieces. The first is a small set of documentation providing 
instructions on how to get setup to run context examples. This will require a download 
similar to how Kibana works for some of the examples. The second is an ingest processor 
example using the downloaded data. More examples will follow as ideally one per PR. 
This also adds a set of tests to individually test each script as a unit test.
2018-08-09 14:24:55 -07:00
Armin Braun 79375d35bb
Scripting: Replace Update Context (#32096)
* SCRIPTING: Move Update Scripts to their own context
* Added system property for backwards compatibility of change to `ctx.params`
2018-08-09 14:32:36 +02:00
Jack Conradson 9b00f095b9
Painless: Move More Logic to PainlessLookup (#32689)
This moves some run-time lookups for methods and fields to the PainlessLookup.
2018-08-08 16:25:14 -07:00
Jack Conradson 0b7fb4e7b9
Painless: Clean up FunctionRef (#32644)
This change consolidates all the logic for generating a FunctionReference (renamed from 
FunctionRef) from several arbitrary constructors to a single static function that is used at 
both compile-time and run-time. This increases long-term maintainability as it is much 
easier to follow when and how a function reference is being generated. It moves most of 
the duplicated logic out of the ECapturingFuncRef, EFuncRef and ELambda nodes and 
Def as well.
2018-08-07 12:26:57 -07:00
Jack Conradson 6ca24e13af
Painless: Use LocalMethod Map For Lookup at Runtime (#32599)
This modifies Def to use a Map<String, LocalMethod> to look up user-defined methods at runtime 
instead of writing constant methodhandles to do the reverse lookup. This creates a consistency 
between how LocalMethods are looked up at compile-time and run-time. This consistency will allow 
this code to be more maintainable moving forward. This will also allow FunctionReference to be 
cleaned up in a follow up PR.
2018-08-03 15:22:30 -07:00
Jack Conradson b938960602
Painless: Move Some Lookup Logic to PainlessLookup (#32565)
Renames existing methods in PainlessLookup. Adds lookupPainlessClass, 
lookupPainlessMethod, and lookupPainlessField to PainlessLookup. This consolidates 
the logic necessary to look these things up into a single place and begins the clean up of 
some of the nodes that were looking each of these things up individually. This also has 
the added benefit of improved consistency in error messaging.
2018-08-02 12:33:25 -07:00
Jack Conradson 2985920134
Painless: Clean Up PainlessField (#32525)
Updates PainlessField variable names to current naming scheme and removes 
extraneous variables.
2018-08-01 09:28:18 -07:00
Ryan Ernst 478f6d6cf1
Scripting: Conditionally use java time api in scripting (#31441)
This commit adds a boolean system property, `es.scripting.use_java_time`,
which controls the concrete return type used by doc values within
scripts. The return type of accessing doc values for a date field is
changed to Object, essentially duck typing the type to allow
co-existence during the transition from joda time to java time.
2018-08-01 08:58:49 -07:00
Jack Conradson 09e38f2f59
Painless: Clean up PainlessMethod (#32476)
Renames and removes variables from PainlessMethod to follow the new naming 
convention. Generates methodtypes at compile-time instead of using a method at run-
time. Moves write method to MethodWriter.
2018-07-31 16:25:53 -07:00
Ryan Ernst 2ed9782a67
Scripting: Fix painless compiler loader to know about context classes (#32385)
This commit fixes the painless compiler classloader to know about the
classes from the script context. This fixes an issue when a custom
context is used from a plugin which caused a ClassNotFoundException for
the script class and its factory classes.
2018-07-31 08:28:03 -07:00
Jack Conradson c69e62d96f
Painless: Add PainlessConstructor (#32447)
PainlessMethod was being used as both a method and a constructor, and while there are 
similarities, there are also some major differences. This allows the reflection objects to be 
stored reducing the number of other pieces of data stored in a PainlessMethod as they are 
now redundant. This temporarily increases some of the code in FunctionRef and 
PainlessDocGenerator as they now differentiate between constructors and methods, BUT 
is also makes the code more maintainable because there aren't checks in several places 
anymore to differentiate.
2018-07-30 14:46:24 -07:00
Jack Conradson e9e1095596
Painless: Add method type to method. (#32441)
MethodType can be computed at compile-time rather than run-time. This removes the 
method that collects MethodType at run-time from a PainlessMethod since is it no longer 
necessary.
2018-07-27 14:23:37 -07:00
Jack Conradson df579f8bce
Painless: Clean Up PainlessClass Variables (#32380)
Removes the variables name, clazz, and type as they are unnecessary. Renames 
staticMembers -> staticFields, members -> fields, getters -> getterMethodHandles, and 
setters -> setterMethodHandles.
2018-07-26 09:02:06 -07:00
Jack Conradson 853aa0afb4
Painless: Decouple PainlessLookupBuilder and Whitelists (#32346)
Implements a static function in PainlessLookupBuilder that contains all the logic related
to Whitelist.  PainlessLookupBuilder is available for use in loading from methods beyond
Whitelist now.
2018-07-25 10:52:01 -07:00
Jack Conradson 1690451a9f
Painless: Update More Methods to New Naming Scheme (#32305)
This finishes the updating the methods in the PainlessLookupBuilder to the new naming scheme. Mechanical change. Methods include the ones used for copying members in the inheritance hierarchy, calculating shortcuts, and setting the functional interface.
2018-07-24 13:08:05 -07:00
Jack Conradson d3c4904fa3
Painless: Clean up add methods in PainlessLookup (#32258)
This is largely mechanical change that cleans up the addConstructor, addMethod, and 
addFields methods in PainlessLookup. Changes include renamed variables, better error 
messages, and some minor code movement to make it more maintainable long term.
2018-07-23 09:12:30 -07:00
Christoph Büscher ff87b7aba4
Remove unnecessary warning supressions (#32250) 2018-07-23 11:31:04 +02:00
Armin Braun e21692e387
INGEST: Make a few Processors callable by Painless (#32170)
* INGEST: Make a few Processors callable by Painless
* Extracted a few stateless String processors as well as the json processor to static methods and whitelisted them in Painless
* provide whitelist from processors plugin
2018-07-20 21:10:35 +02:00
Jack Conradson c7a41c501a
Painless: Simplify Naming in Lookup Package (#32177)
This removes some extraneous naming syntax and makes clear the meaning of certain
naming conventions without ambiguities (stricter) within the lookup package. Purely 
mechanical change. Note this does not cover a large portion of the 
PainlessLookupBuilder and PainlessLookup yet as there are several more follow up PRs for these incoming.
2018-07-19 16:35:03 -07:00
Jack Conradson 605dc49c48
Painless: Fix caching bug and clean up addPainlessClass. (#32142)
This change cleans up the addPainlessClass methods by doing the following things:

* Rename many variable names to match the new conventions described in the JavaDocs 
for PainlessLookup
* Decouples Whitelist.Class from adding a PainlessClass directly
* Adds a second version of addPainlessClass that is intended for use to add future 
defaults in a follow PR

This change also fixes the method and field caches by storing Classes instead of Strings 
since it would technically be possible now that the whitelists are extendable to have 
different Classes with the same name. It was convenient to add this change together 
since some of the new constants are shared.

Note the changes are largely mechanical again where all the code behavior should 
remain the same.
2018-07-18 09:29:52 -07:00
Martijn van Groningen 53ab470264
use before instead of onOrBefore 2018-07-18 13:33:57 +02:00
Martijn van Groningen 1924f5d07c
Add more contexts to painless execute api (#30511)
This change adds two contexts the execute scripts against:

* SEARCH_SCRIPT: Allows to run scripts in a search script context.
This context is used in `function_score` query's script function,
script fields, script sorting and `terms_set` query.

* FILTER_SCRIPT: Allows to run scripts in a filter script context.
This context is used in the `script` query.

In both contexts a index name needs to be specified and a sample document.
The document is needed to create an in-memory index that the script can
access via the `doc[...]` and other notations. The index name is needed
because a mapping is needed to index the document.

Examples:

```
POST /_scripts/painless/_execute
{
  "script": {
    "source": "doc['field'].value.length()"
  },
  "context" : {
    "search_script": {
      "document": {
        "field": "four"
      },
      "index": "my-index"
    }
  }
}
```

Returns:

```
{
  "result": 4
}
```

POST /_scripts/painless/_execute
{
  "script": {
    "source": "doc['field'].value.length() <= params.max_length",
    "params": {
      "max_length": 4
    }
  },
  "context" : {
    "filter_script": {
      "document": {
        "field": "four"
      },
      "index": "my-index"
    }
  }
}

Returns:

```
{
  "result": true
}
```

Also changed PainlessExecuteAction.TransportAction to use TransportSingleShardAction
instead of HandledAction, because now in case score or filter contexts are used
the request needs to be redirected to a node that has an active IndexService
for the index being referenced (a node with a shard copy for that index).
2018-07-18 12:42:07 +02:00
Christoph Büscher ef5e8d8d8a Fix Java 11 javadoc compile problem
Java 11 complains with a "type arguments not allowed here" error when types are
used in javadoc links it seems. Simply removing it.
2018-07-18 10:36:31 +02:00
Jack Conradson 03c16cd0e3
Painless: Add PainlessClassBuilder (#32141)
Several pieces of data in PainlessClass cannot be passed in at the time the 
PainlessClass is created so it must be "frozen" after all the data is collected. This means 
PainlessClass is currently serving two functions as both a builder and a set of data. This 
separates the two pieces into clearly distinct values.

This change also removes the PainlessMethodKey in favor of a simple String. The goal is 
to have the painless method key be completely internal to the PainlessLookup eventually 
and this simplifies the way there. Note that this was added since PainlessClass and 
PainlessClassBuilder were already being changed instead of a follow up PR.
2018-07-17 13:54:49 -07:00
Jack Conradson 1c63eb1081
Painless: Fix Bug with Duplicate PainlessClasses (#32110)
When building the PainlessMethods and PainlessFields they stored a reference to a 
PainlessClass. This reference was prior to "freezing" the PainlessClass so the data was 
both incomplete and mutable. This has been replaced with a target java class instead 
since the PainlessClass is accessible through a java class now and it requires no special 
modifications to get around a chicken and egg issue.
2018-07-17 10:33:38 -07:00
Jack Conradson 15740d6229
Painless: Move and Rename Several Methods in the lookup package (#32105) 2018-07-16 16:13:48 -07:00
Jack Conradson 2a1a28f19c
Painless: Separate PainlessLookup into PainlessLookup and PainlessLookupBuilder (#32054) 2018-07-16 11:15:29 -07:00
Armin Braun b1479bbed8
Scripting: Remove dead code from painless module (#32064) 2018-07-16 18:43:00 +02:00
Christoph Büscher ca4c4f736a
Remove unused params from SSource and Walker (#31935)
The "source" field in SSource seems unused. If removed, it can also be removed
from the ctor, which in turn makes is possible to delete the sourceText in the
Walker class.
2018-07-16 10:54:23 +02:00
Armin Braun b65c586cef
Cleanup Duplication in `PainlessScriptEngine` (#31991)
* Cleanup Duplication in `PainlessScriptEngine`
* Extract duplicate building of compiler settings to method
* Remove dead method params + dead constant in `ScriptProcessor`
2018-07-14 13:37:59 +02:00
Mayya Sharipova 5481fbc249
Handle missing values in painless (#30975)
* Handle missing values in painless

Throw an exception for `doc['field'].value`
if this document is missing a value for the `field`.

For 7.0:
This is the default behaviour from 7.0

For 6.x:
To enable this behavior from 6.x, a user can set a jvm.option:
 `-Des.script.exception_for_missing_value=true` on a node.
If a user does not enable this behavior, a deprecation warning is logged on start up.

Closes #29286
2018-07-09 11:59:49 -04:00
Jack Conradson d9a92011bc
Painless: Restructure Definition/Whitelist (#31879)
Create lookup package
rename Definition to PainlessLookup and move to lookup package
rename Definition.Method to PainlessMethod
rename Definition.MethodKey to PainlessMethod
rename Definition.Field to PainlessField
rename Definition.Struct to PainlessClass
rename Definition.Cast to PainlessCast
rename Whitelist.Struct to WhitelistClass
rename Whitelist.Constructor to WhitelistConstructor
rename Whitelist.Method to WhitelistMethod
rename Whitelist.Field to WhitelistField
2018-07-08 12:00:23 -07:00
Alpar Torok cf2295b408
Add JDK11 support and enable in CI (#31644)
* Upgrade bouncycastle

Required to fix
`bcprov-jdk15on-1.55.jar; invalid manifest format `
on jdk 11

* Downgrade bouncycastle to avoid invalid manifest

* Add checksum for new jars

* Update tika permissions for jdk 11

* Mute test failing on jdk 11

* Add JDK11 to CI

* Thread#stop(Throwable) was removed

http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html

* Disable failing tests #31456

* Temprorarily disable doc tests

To see if there are other failures on JDK11

* Only blacklist specific doc tests

* Disable only failing tests in ingest attachment plugin

* Mute failing HDFS tests #31498

* Mute failing lang-painless tests #31500

* Fix backwards compatability builds

Fix JAVA version to 10 for ES 6.3

* Add 6.x to bwx -> java10

* Prefix out and err from buildBwcVersion for readability

```
> Task :distribution:bwc:next-bugfix-snapshot:buildBwcVersion
  [bwc] :buildSrc:compileJava
  [bwc] WARNING: An illegal reflective access operation has occurred
  [bwc] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/home/alpar/.gradle/wrapper/dists/gradle-4.5-all/cg9lyzfg3iwv6fa00os9gcgj4/gradle-4.5/lib/groovy-all-2.4.12.jar) to method java.lang.Object.finalize()
  [bwc] WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
  [bwc] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
  [bwc] WARNING: All illegal access operations will be denied in a future release
  [bwc] :buildSrc:compileGroovy
  [bwc] :buildSrc:writeVersionProperties
  [bwc] :buildSrc:processResources
  [bwc] :buildSrc:classes
  [bwc] :buildSrc:jar

```

* Also set RUNTIME_JAVA_HOME for bwcBuild

So that we can make sure it's not too new for the build to understand.

* Align bouncycastle dependency

* fix painles array tets

closes #31500

* Update jar checksums

* Keep 8/10 runtime/compile untill consensus builds on 11

* Only skip failing tests if running on Java 11

* Failures are dependent of compile java version not runtime

* Condition doc test exceptions on compiler java version as well

* Disable hdfs tests based on runtime java

* Set runtime java to minimum supported for bwc

* PR review

* Add comment with ticket for forbidden apis
2018-07-05 03:24:01 +00:00
Jack Conradson a02e5ee740
Painless: Complete Removal of Painless Type (#31699)
This completes the removal of Painless Type. The new data structures in the definition are a map of names (String) to Java Classes and a map of Java Classes to Painless Structs. The names to Java Classes map can contain a 2 to 1 ratio of names to classes depending on whether or not a short (imported) name is used. The Java Classes to Painless Structs is 1 to 1 always where the Java Class name must match the Painless Struct name. This should lead a significantly simpler type system in Painless moving forward since the Painless Type only held redundant information since Painless does not support generics.
2018-07-03 13:31:56 -07:00
Alpar Torok 08b8d11e30
Add support for switching distribution for all integration tests (#30874)
* remove left-over comment

* make sure of the property for plugins

* skip installing modules if these exist in the distribution

* Log the distrbution being ran

* Don't allow running with integ-tests-zip passed externally

* top level x-pack/qa can't run with oss distro

* Add support for matching objects in lists

Makes it possible to have a key that points to a list and assert that a
certain object is present in the list. All keys have to be present and
values have to match. The objects in the source list may have additional
fields.

example:
```
  match:  { 'nodes.$master.plugins': { name: ingest-attachment }  }
```

* Update plugin and module tests to work with other distributions

Some of the tests expected that the integration tests will always be ran
with  the `integ-test-zip` distribution so that there will be no other
plugins loaded.

With this change, we check for the presence of the plugin without
assuming exclusivity.

* Allow modules to run on other distros as well

To match the behavior of tets.distributions

* Add and use a new `contains` assertion

Replaces the  previus changes that caused `match` to do a partial match.

* Implement PR review comments
2018-06-26 06:49:03 -07:00
Jonathan Little 8e4768890a Migrate scripted metric aggregation scripts to ScriptContext design (#30111)
* Migrate scripted metric aggregation scripts to ScriptContext design #29328

* Rename new script context container class and add clarifying comments to remaining references to params._agg(s)

* Misc cleanup: make mock metric agg script inner classes static

* Move _score to an accessor rather than an arg for scripted metric agg scripts

This causes the score to be evaluated only when it's used.

* Documentation changes for params._agg -> agg

* Migration doc addition for scripted metric aggs _agg object change

* Rename "agg" Scripted Metric Aggregation script context variable to "state"

* Rename a private base class from ...Agg to ...State that I missed in my last commit

* Clean up imports after merge
2018-06-25 12:01:33 +01:00
Ryan Ernst 7a150ec06d
Core: Combine doExecute methods in TransportAction (#31517)
TransportAction currently contains 2 doExecute methods, one which takes
a the task, and one that does not. The latter is what some subclasses
implement, while the first one just calls the latter, dropping the given
task. This commit combines these methods, in favor of just always
assuming a task is present.
2018-06-22 15:03:01 -07:00
Ryan Ernst 4f9332ee16
Core: Remove ThreadPool from base TransportAction (#31492)
Most transport actions don't need the node ThreadPool. This commit
removes the ThreadPool as a super constructor parameter for
TransportAction. The actions that do need the thread pool then have a
member added to keep it from their own constructor.
2018-06-21 11:25:26 -07:00
Ryan Ernst 401800d958
Core: Remove index name resolver from base TransportAction (#31002)
Most transport actions don't need to resolve index names. This commit
removes the index name resolver as a super constructor parameter for
TransportAction. The actions that do need the resolver then have a
member added to keep the resolver from their own constructor.
2018-06-19 17:06:09 -07:00
Ryan Ernst e67aa96c81
Core: Combine Action and GenericAction (#31405)
Since #30966, Action no longer has anything but a call to the
GenericAction super constructor. This commit renames GenericAction
into Action, thus eliminating the Action class. Additionally, this
commit removes the Request generic parameter of the class, since
it was unused.
2018-06-18 23:53:04 +02:00
Jack Conradson 0324103737
Painless: Fix bug for static method calls on interfaces (#31348)
Static method calls on interfaces were not being called correctly
which was causing JVM crashes.  This change fixes the issue.
2018-06-14 18:30:37 -07:00
Tanguy Leroux bf58660482
Remove all unused imports and fix CRLF (#31207)
The X-Pack opening and the recent other refactorings left a lot of 
unused imports in the codebase. This commit removes them all.
2018-06-11 15:12:12 +02:00
Tanguy Leroux a1916658a9 [Tests] Fix self-referencing tests
This commit adapts some test after #31044 has been merged.
2018-06-11 12:45:27 +02:00
Ryan Ernst 46e8d97813
Core: Remove RequestBuilder from Action (#30966)
This commit removes the RequestBuilder generic type from Action. It was
needed to be used by the newRequest method, which in turn was used by
client.prepareExecute. Both of these methods are now removed, along with
the existing users of prepareExecute constructing the appropriate
builder directly.
2018-05-31 16:15:00 +02:00
Christoph Büscher 1ea9f11b03
Change ScriptException status to 400 (bad request) (#30861)
Currently failures to compile a script usually lead to a ScriptException, which
inherits the 500 INTERNAL_SERVER_ERROR from ElasticsearchException if it does
not contain another root cause. Instead, this should be a 400 Bad Request error.
This PR changes this more generally for script compilation errors by changing 
ScriptException to return 400 (bad request) as status code.

Closes #12315
2018-05-30 14:00:07 +02:00
Ryan Ernst 34180f2285
Scripting: Remove getDate methods from ScriptDocValues (#30690)
The getDate() and getDates() existed prior to 5.x on long fields in
scripting. In 5.x, a new Date type for ScriptDocValues was added. The
getDate() and getDates() methods were left on long fields and added to date
fields to ease the transition. This commit removes those methods for
7.0.
2018-05-18 21:26:26 -07:00
Zachary Tong d120fb222c [TEST] Adjust version skips for movavg/movfn tests
Since the MovFn PR was backported to 6.x, we can adjust
the version skip numbers in master to correctly
match 6.3.99 instead of 6.4.0
2018-05-17 18:07:52 +00:00
Zachary Tong df853c49c0
Add a MovingFunction pipeline aggregation, deprecate MovingAvg agg (#29594)
This pipeline aggregation gives the user the ability to script functions that "move" across a window
of data, instead of single data points.  It is the scripted version of MovingAvg pipeline agg.

Through custom script contexts, we expose a number of convenience methods:

 - MovingFunctions.max()
 - MovingFunctions.min()
 - MovingFunctions.sum()
 - MovingFunctions.unweightedAvg()
 - MovingFunctions.linearWeightedAvg()
 - MovingFunctions.ewma()
 - MovingFunctions.holt()
 - MovingFunctions.holtWinters()
 - MovingFunctions.stdDev()

The user can also define any arbitrary logic via their own scripting, or combine with the above methods.
2018-05-16 10:57:00 -04:00
Nik Everett 8401eac425 Test: Switch painless test to 1 shard
We *think* that #28600 is caused by warnings not being collected during
one of the fan out phases of search but we're not 100% sure how this is
happening. This commit drops the number of shards used for the test to 1
so there *isn't* a fan out phase. If this makes the issue go away we'll
have more information.
2018-04-27 15:01:42 -04:00
Martijn van Groningen 8afa7c174f
Added painless execute api. (#29164)
Added an api that allows to execute an arbitrary script and a result to be returned.

```
POST /_scripts/painless/_execute
{
  "script": {
    "source": "params.var1 / params.var2",
    "params": {
      "var1": 1,
      "var2": 1
    }
  }
}
```

Relates to #27875
2018-04-19 09:33:34 +02:00
Jack Conradson da9a6899ff
Painless: modify grammar to allow more statement delimiters (#29566)
This allows the grammar to determine when and what delimiters statements will use by
splitting up the statements into regular statements and delimited statements, those that do
not require a delimiter versus those that do. This allows consumers of the statements to
determine what delimiters the statements will use so that in certain cases semicolons are
not necessary like when there's a closing right bracket.

This change removes the need for semicolon insertion in the lexer, simplifying the existing
lexer quite a bit. It also ensures that there isn't a need to track semicolons being inserted
into places that aren't necessary such as array initializers.
2018-04-18 10:32:42 -07:00
Jack Conradson 782e41a67e
Painless: Remove extraneous INLINE constant. (#29340) 2018-04-02 21:34:01 -07:00
Robin Neatherway 6dadce4761 Painless: Correct ClassToName string conversion (#28997)
A typo of 'dimensions' rather than 'dimension' caused an infinite loop.
2018-03-13 13:16:48 -07:00
Jason Tedor 5904d936fa
Copy Lucene IOUtils (#29012)
As we have factored Elasticsearch into smaller libraries, we have ended
up in a situation that some of the dependencies of Elasticsearch are not
available to code that depends on these smaller libraries but not server
Elasticsearch. This is a good thing, this was one of the goals of
separating Elasticsearch into smaller libraries, to shed some of the
dependencies from other components of the system. However, this now
means that simple utility methods from Lucene that we rely on are no
longer available everywhere. This commit copies IOUtils (with some small
formatting changes for our codebase) into the fold so that other
components of the system can rely on these methods where they no longer
depend on Lucene.
2018-03-13 12:49:33 -04:00
Robin Neatherway 8d0f976024 Removed unused local variable from ScriptClassInfo (#28617)
Following [8999104](8999104b14)
  the local variable `argumentNames` is no longer used, so removed.
2018-02-12 15:58:09 -08:00
Jack Conradson 5c1d3aa2f0
Painless: Fixes a null pointer exception in certain cases of for loop usage (#28506)
The initializer and afterthought were not having their types
appropriately cast which is necessary with expressions which in turn
caused values to be popped off the stack that were null.
2018-02-05 11:57:21 -08:00
Nik Everett 5003ef18ac
Scripts: Fix security for deprecation warning (#28485)
If you call `getDates()` on a long or date type field add a deprecation
warning to the response and log something to the deprecation logger.
This *mostly* worked just fine but if the deprecation logger happens to
roll then the roll will be performed with the script's permissions
rather than the permissions of the server. And scripts don't have
permissions to, say, open files. So the rolling failed. This fixes that
by wrapping the call the deprecation logger in `doPriviledged`.

This is a strange `doPrivileged` call because it doens't check
Elasticsearch's `SpecialPermission`. `SpecialPermission` is a permission
that no-script code has and that scripts never have. Usually all
`doPrivileged` calls check `SpecialPermission` to make sure that they
are not accidentally acting on behalf of a script. But in this case we
are *intentionally* acting on behalf of a script.

Closes #28408
2018-02-03 14:56:08 -05:00
Jack Conradson 90c74a7e09
Remove RuntimeClass from Painless Definition in favor of just Painless Struct. (#28486) 2018-02-02 10:26:02 -08:00
Jack Conradson df1c696e1d
Remove Painless Type From Locals, Variables, Params, and ScriptInfo (#28471) 2018-02-01 12:46:20 -08:00
Jack Conradson e281d57d82
Remove Painless Type From Painless Method/Field (#28466) 2018-01-31 16:27:52 -08:00
Jack Conradson fa8e62d48f
Removes Painless Type in favor of Java Class from the expression nodes related to function references and lambdas. (#28433) 2018-01-30 08:18:14 -08:00
Jack Conradson 1d01bcf421
Remove Painless Type in favor of Java Class in FunctionRef. (#28429) 2018-01-29 16:43:36 -08:00
Jack Conradson f13da9f534
Remove Painless Type from e-nodes in favor of Java Class (#28364) 2018-01-29 12:44:50 -08:00
Jack Conradson a57a0ae78b
Remove Painless Type from MethodWriter in favor of Java Class. (#28346) 2018-01-24 11:02:46 -08:00
Jack Conradson b945006938
Completely remove Painless Type from AnalyzerCaster in favor of Java Class. (#28329)
Second part in a series of PR's to remove Painless Type in favor of Java Class. This completely removes the Painless Type dependency from AnalyzerCaster. Both casting and promotion are now based on Java Class exclusively. This also allows AnalyzerCaster to be decoupled from Definition and make cast checks be static calls again.
2018-01-23 12:38:31 -08:00
Simon Willnauer 4d3f7a7695
Ensure we protect Collections obtained from scripts from self-referencing (#28335)
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
2018-01-23 16:57:26 +01:00
Jack Conradson ef5c041819
Painless: Replace Painless Type with Java Class during Casts (#27847)
This is the first step in a series to replace Painless Type with Java Class for any casting done during compilation. There should be no behavioural change.
2018-01-22 13:01:13 -08:00
Ryan Ernst ba9c9e08e7
Painless: Add spi jar that will be published for extending whitelists (#28302)
In order to build a plugin that extends the painless whitelist, the spi
classes must be available to the plugin at compile time. This commit
moves the spi classes into a separate jar which will be published. Any
plugin authors whiching to extend painless through spi would then add a
compileOnly dependency on this jar.
2018-01-18 19:16:26 -08:00
Ryan Ernst 18463e7e9f
Painless: Add whitelist extensions (#28161)
This commit adds a PainlessExtension which may be plugged in via SPI to
add additional classes, methods and members to the painless whitelist on
a per context basis. An example plugin adding and using a whitelist is
also added.
2018-01-15 11:28:31 -08:00
Jack Conradson 5d795afddb
Painless: Add public member read/write access test. (#28156) 2018-01-09 15:06:51 -08:00
Jack Conradson 1d1dcd4ae7
Painless: Add a simple cache for whitelist methods and fields. (#28142)
With support for multiple contexts we are adding some caching to the whitelist to keep the memory footprint for definitions from exploding.
2018-01-08 17:54:45 -08:00
Jack Conradson b5377d294f
Painless: Modify Loader to Load Classes Directly from Definition (#28088) 2018-01-05 13:06:36 -08:00
Ryan Ernst d36ec18029
Plugins: Add plugin extension capabilities (#27881)
This commit adds the infrastructure to plugin building and loading to
allow one plugin to extend another. That is, one plugin may extend
another by the "parent" plugin allowing itself to be extended through
java SPI. When all plugins extending a plugin are finished loading, the
"parent" plugin has a callback (through the ExtensiblePlugin interface)
allowing it to reload SPI.

This commit also adds an example plugin which uses as-yet implemented
extensibility (adding to the painless whitelist).
2018-01-03 11:12:43 -08:00
Jack Conradson 1de927c80d
Painless: Clean Up Painless Cast Object (#27794)
Added static methods to make creating Painless casts obvious as to what is
being boxed/unboxed.
2017-12-14 09:08:10 -08:00
Jack Conradson 8188d9f7e5
Painless: Only allow Painless type names to be the same as the equivalent Java class. (#27264)
Also adds a parameter called only_fqn to the whitelist to enforce that a painless type must be specified as the fully-qualifed java class name.
2017-12-11 16:37:35 -08:00
Robin Neatherway 057efea893 Correct two equality checks on incomparable types (#27688) 2017-12-07 14:18:11 +01:00
Jack Conradson 2d927fabab
Painless: Fix errors allowing void to be assigned to def. (#27460) 2017-11-28 13:44:52 -08:00
Jack Conradson 9e42b77f7e
Painless: Fix variable scoping issue in lambdas not including captured variables. (#27571) 2017-11-28 13:30:13 -08:00
Adrien Grand 996990ad1f
Upgrade to lucene-7.2.0-snapshot-8c94404. (#27496)
The main highlight of this new snapshot is that it introduces the opportunity
for queries to opt out of caching. In case a query opts out of caching, not only
will it never be cached, but also no compound query that wraps it will be
cached.
2017-11-28 14:52:42 +01:00
Jack Conradson abaede2373
Upgrade Painless from ANTLR 4.5.1-1 to ANTLR 4.5.3. (#27153) 2017-10-27 11:07:49 -07:00
Jack Conradson dda5d1af29 Allow for the Painless Definition to have multiple instances (#27096) 2017-10-26 08:33:55 -07:00
Nik Everett 4a06dd919a Painless: add tests for cached boxing (#24163)
We had a TODO about adding tests around cached boxing. In #24077
I tracked down the uncached boxing tests and saw the TODO. Cached
boxing testing is a fairly small extension to that work.
2017-10-10 10:34:03 -04:00