Commit Graph

621 Commits

Author SHA1 Message Date
Mayya Sharipova 5481fbc249
Handle missing values in painless (#30975)
* Handle missing values in painless

Throw an exception for `doc['field'].value`
if this document is missing a value for the `field`.

For 7.0:
This is the default behaviour from 7.0

For 6.x:
To enable this behavior from 6.x, a user can set a jvm.option:
 `-Des.script.exception_for_missing_value=true` on a node.
If a user does not enable this behavior, a deprecation warning is logged on start up.

Closes #29286
2018-07-09 11:59:49 -04:00
Jack Conradson d9a92011bc
Painless: Restructure Definition/Whitelist (#31879)
Create lookup package
rename Definition to PainlessLookup and move to lookup package
rename Definition.Method to PainlessMethod
rename Definition.MethodKey to PainlessMethod
rename Definition.Field to PainlessField
rename Definition.Struct to PainlessClass
rename Definition.Cast to PainlessCast
rename Whitelist.Struct to WhitelistClass
rename Whitelist.Constructor to WhitelistConstructor
rename Whitelist.Method to WhitelistMethod
rename Whitelist.Field to WhitelistField
2018-07-08 12:00:23 -07:00
Alpar Torok cf2295b408
Add JDK11 support and enable in CI (#31644)
* Upgrade bouncycastle

Required to fix
`bcprov-jdk15on-1.55.jar; invalid manifest format `
on jdk 11

* Downgrade bouncycastle to avoid invalid manifest

* Add checksum for new jars

* Update tika permissions for jdk 11

* Mute test failing on jdk 11

* Add JDK11 to CI

* Thread#stop(Throwable) was removed

http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-June/053536.html

* Disable failing tests #31456

* Temprorarily disable doc tests

To see if there are other failures on JDK11

* Only blacklist specific doc tests

* Disable only failing tests in ingest attachment plugin

* Mute failing HDFS tests #31498

* Mute failing lang-painless tests #31500

* Fix backwards compatability builds

Fix JAVA version to 10 for ES 6.3

* Add 6.x to bwx -> java10

* Prefix out and err from buildBwcVersion for readability

```
> Task :distribution:bwc:next-bugfix-snapshot:buildBwcVersion
  [bwc] :buildSrc:compileJava
  [bwc] WARNING: An illegal reflective access operation has occurred
  [bwc] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/home/alpar/.gradle/wrapper/dists/gradle-4.5-all/cg9lyzfg3iwv6fa00os9gcgj4/gradle-4.5/lib/groovy-all-2.4.12.jar) to method java.lang.Object.finalize()
  [bwc] WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass
  [bwc] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
  [bwc] WARNING: All illegal access operations will be denied in a future release
  [bwc] :buildSrc:compileGroovy
  [bwc] :buildSrc:writeVersionProperties
  [bwc] :buildSrc:processResources
  [bwc] :buildSrc:classes
  [bwc] :buildSrc:jar

```

* Also set RUNTIME_JAVA_HOME for bwcBuild

So that we can make sure it's not too new for the build to understand.

* Align bouncycastle dependency

* fix painles array tets

closes #31500

* Update jar checksums

* Keep 8/10 runtime/compile untill consensus builds on 11

* Only skip failing tests if running on Java 11

* Failures are dependent of compile java version not runtime

* Condition doc test exceptions on compiler java version as well

* Disable hdfs tests based on runtime java

* Set runtime java to minimum supported for bwc

* PR review

* Add comment with ticket for forbidden apis
2018-07-05 03:24:01 +00:00
Jack Conradson a02e5ee740
Painless: Complete Removal of Painless Type (#31699)
This completes the removal of Painless Type. The new data structures in the definition are a map of names (String) to Java Classes and a map of Java Classes to Painless Structs. The names to Java Classes map can contain a 2 to 1 ratio of names to classes depending on whether or not a short (imported) name is used. The Java Classes to Painless Structs is 1 to 1 always where the Java Class name must match the Painless Struct name. This should lead a significantly simpler type system in Painless moving forward since the Painless Type only held redundant information since Painless does not support generics.
2018-07-03 13:31:56 -07:00
Alpar Torok 08b8d11e30
Add support for switching distribution for all integration tests (#30874)
* remove left-over comment

* make sure of the property for plugins

* skip installing modules if these exist in the distribution

* Log the distrbution being ran

* Don't allow running with integ-tests-zip passed externally

* top level x-pack/qa can't run with oss distro

* Add support for matching objects in lists

Makes it possible to have a key that points to a list and assert that a
certain object is present in the list. All keys have to be present and
values have to match. The objects in the source list may have additional
fields.

example:
```
  match:  { 'nodes.$master.plugins': { name: ingest-attachment }  }
```

* Update plugin and module tests to work with other distributions

Some of the tests expected that the integration tests will always be ran
with  the `integ-test-zip` distribution so that there will be no other
plugins loaded.

With this change, we check for the presence of the plugin without
assuming exclusivity.

* Allow modules to run on other distros as well

To match the behavior of tets.distributions

* Add and use a new `contains` assertion

Replaces the  previus changes that caused `match` to do a partial match.

* Implement PR review comments
2018-06-26 06:49:03 -07:00
Jonathan Little 8e4768890a Migrate scripted metric aggregation scripts to ScriptContext design (#30111)
* Migrate scripted metric aggregation scripts to ScriptContext design #29328

* Rename new script context container class and add clarifying comments to remaining references to params._agg(s)

* Misc cleanup: make mock metric agg script inner classes static

* Move _score to an accessor rather than an arg for scripted metric agg scripts

This causes the score to be evaluated only when it's used.

* Documentation changes for params._agg -> agg

* Migration doc addition for scripted metric aggs _agg object change

* Rename "agg" Scripted Metric Aggregation script context variable to "state"

* Rename a private base class from ...Agg to ...State that I missed in my last commit

* Clean up imports after merge
2018-06-25 12:01:33 +01:00
Ryan Ernst 7a150ec06d
Core: Combine doExecute methods in TransportAction (#31517)
TransportAction currently contains 2 doExecute methods, one which takes
a the task, and one that does not. The latter is what some subclasses
implement, while the first one just calls the latter, dropping the given
task. This commit combines these methods, in favor of just always
assuming a task is present.
2018-06-22 15:03:01 -07:00
Ryan Ernst 4f9332ee16
Core: Remove ThreadPool from base TransportAction (#31492)
Most transport actions don't need the node ThreadPool. This commit
removes the ThreadPool as a super constructor parameter for
TransportAction. The actions that do need the thread pool then have a
member added to keep it from their own constructor.
2018-06-21 11:25:26 -07:00
Ryan Ernst 401800d958
Core: Remove index name resolver from base TransportAction (#31002)
Most transport actions don't need to resolve index names. This commit
removes the index name resolver as a super constructor parameter for
TransportAction. The actions that do need the resolver then have a
member added to keep the resolver from their own constructor.
2018-06-19 17:06:09 -07:00
Ryan Ernst e67aa96c81
Core: Combine Action and GenericAction (#31405)
Since #30966, Action no longer has anything but a call to the
GenericAction super constructor. This commit renames GenericAction
into Action, thus eliminating the Action class. Additionally, this
commit removes the Request generic parameter of the class, since
it was unused.
2018-06-18 23:53:04 +02:00
Jack Conradson 0324103737
Painless: Fix bug for static method calls on interfaces (#31348)
Static method calls on interfaces were not being called correctly
which was causing JVM crashes.  This change fixes the issue.
2018-06-14 18:30:37 -07:00
Tanguy Leroux bf58660482
Remove all unused imports and fix CRLF (#31207)
The X-Pack opening and the recent other refactorings left a lot of 
unused imports in the codebase. This commit removes them all.
2018-06-11 15:12:12 +02:00
Tanguy Leroux a1916658a9 [Tests] Fix self-referencing tests
This commit adapts some test after #31044 has been merged.
2018-06-11 12:45:27 +02:00
Ryan Ernst 46e8d97813
Core: Remove RequestBuilder from Action (#30966)
This commit removes the RequestBuilder generic type from Action. It was
needed to be used by the newRequest method, which in turn was used by
client.prepareExecute. Both of these methods are now removed, along with
the existing users of prepareExecute constructing the appropriate
builder directly.
2018-05-31 16:15:00 +02:00
Christoph Büscher 1ea9f11b03
Change ScriptException status to 400 (bad request) (#30861)
Currently failures to compile a script usually lead to a ScriptException, which
inherits the 500 INTERNAL_SERVER_ERROR from ElasticsearchException if it does
not contain another root cause. Instead, this should be a 400 Bad Request error.
This PR changes this more generally for script compilation errors by changing 
ScriptException to return 400 (bad request) as status code.

Closes #12315
2018-05-30 14:00:07 +02:00
Ryan Ernst 34180f2285
Scripting: Remove getDate methods from ScriptDocValues (#30690)
The getDate() and getDates() existed prior to 5.x on long fields in
scripting. In 5.x, a new Date type for ScriptDocValues was added. The
getDate() and getDates() methods were left on long fields and added to date
fields to ease the transition. This commit removes those methods for
7.0.
2018-05-18 21:26:26 -07:00
Zachary Tong d120fb222c [TEST] Adjust version skips for movavg/movfn tests
Since the MovFn PR was backported to 6.x, we can adjust
the version skip numbers in master to correctly
match 6.3.99 instead of 6.4.0
2018-05-17 18:07:52 +00:00
Zachary Tong df853c49c0
Add a MovingFunction pipeline aggregation, deprecate MovingAvg agg (#29594)
This pipeline aggregation gives the user the ability to script functions that "move" across a window
of data, instead of single data points.  It is the scripted version of MovingAvg pipeline agg.

Through custom script contexts, we expose a number of convenience methods:

 - MovingFunctions.max()
 - MovingFunctions.min()
 - MovingFunctions.sum()
 - MovingFunctions.unweightedAvg()
 - MovingFunctions.linearWeightedAvg()
 - MovingFunctions.ewma()
 - MovingFunctions.holt()
 - MovingFunctions.holtWinters()
 - MovingFunctions.stdDev()

The user can also define any arbitrary logic via their own scripting, or combine with the above methods.
2018-05-16 10:57:00 -04:00
Nik Everett 8401eac425 Test: Switch painless test to 1 shard
We *think* that #28600 is caused by warnings not being collected during
one of the fan out phases of search but we're not 100% sure how this is
happening. This commit drops the number of shards used for the test to 1
so there *isn't* a fan out phase. If this makes the issue go away we'll
have more information.
2018-04-27 15:01:42 -04:00
Martijn van Groningen 8afa7c174f
Added painless execute api. (#29164)
Added an api that allows to execute an arbitrary script and a result to be returned.

```
POST /_scripts/painless/_execute
{
  "script": {
    "source": "params.var1 / params.var2",
    "params": {
      "var1": 1,
      "var2": 1
    }
  }
}
```

Relates to #27875
2018-04-19 09:33:34 +02:00
Jack Conradson da9a6899ff
Painless: modify grammar to allow more statement delimiters (#29566)
This allows the grammar to determine when and what delimiters statements will use by
splitting up the statements into regular statements and delimited statements, those that do
not require a delimiter versus those that do. This allows consumers of the statements to
determine what delimiters the statements will use so that in certain cases semicolons are
not necessary like when there's a closing right bracket.

This change removes the need for semicolon insertion in the lexer, simplifying the existing
lexer quite a bit. It also ensures that there isn't a need to track semicolons being inserted
into places that aren't necessary such as array initializers.
2018-04-18 10:32:42 -07:00
Jack Conradson 782e41a67e
Painless: Remove extraneous INLINE constant. (#29340) 2018-04-02 21:34:01 -07:00
Robin Neatherway 6dadce4761 Painless: Correct ClassToName string conversion (#28997)
A typo of 'dimensions' rather than 'dimension' caused an infinite loop.
2018-03-13 13:16:48 -07:00
Jason Tedor 5904d936fa
Copy Lucene IOUtils (#29012)
As we have factored Elasticsearch into smaller libraries, we have ended
up in a situation that some of the dependencies of Elasticsearch are not
available to code that depends on these smaller libraries but not server
Elasticsearch. This is a good thing, this was one of the goals of
separating Elasticsearch into smaller libraries, to shed some of the
dependencies from other components of the system. However, this now
means that simple utility methods from Lucene that we rely on are no
longer available everywhere. This commit copies IOUtils (with some small
formatting changes for our codebase) into the fold so that other
components of the system can rely on these methods where they no longer
depend on Lucene.
2018-03-13 12:49:33 -04:00
Robin Neatherway 8d0f976024 Removed unused local variable from ScriptClassInfo (#28617)
Following [8999104](8999104b14)
  the local variable `argumentNames` is no longer used, so removed.
2018-02-12 15:58:09 -08:00
Jack Conradson 5c1d3aa2f0
Painless: Fixes a null pointer exception in certain cases of for loop usage (#28506)
The initializer and afterthought were not having their types
appropriately cast which is necessary with expressions which in turn
caused values to be popped off the stack that were null.
2018-02-05 11:57:21 -08:00
Nik Everett 5003ef18ac
Scripts: Fix security for deprecation warning (#28485)
If you call `getDates()` on a long or date type field add a deprecation
warning to the response and log something to the deprecation logger.
This *mostly* worked just fine but if the deprecation logger happens to
roll then the roll will be performed with the script's permissions
rather than the permissions of the server. And scripts don't have
permissions to, say, open files. So the rolling failed. This fixes that
by wrapping the call the deprecation logger in `doPriviledged`.

This is a strange `doPrivileged` call because it doens't check
Elasticsearch's `SpecialPermission`. `SpecialPermission` is a permission
that no-script code has and that scripts never have. Usually all
`doPrivileged` calls check `SpecialPermission` to make sure that they
are not accidentally acting on behalf of a script. But in this case we
are *intentionally* acting on behalf of a script.

Closes #28408
2018-02-03 14:56:08 -05:00
Jack Conradson 90c74a7e09
Remove RuntimeClass from Painless Definition in favor of just Painless Struct. (#28486) 2018-02-02 10:26:02 -08:00
Jack Conradson df1c696e1d
Remove Painless Type From Locals, Variables, Params, and ScriptInfo (#28471) 2018-02-01 12:46:20 -08:00
Jack Conradson e281d57d82
Remove Painless Type From Painless Method/Field (#28466) 2018-01-31 16:27:52 -08:00
Jack Conradson fa8e62d48f
Removes Painless Type in favor of Java Class from the expression nodes related to function references and lambdas. (#28433) 2018-01-30 08:18:14 -08:00
Jack Conradson 1d01bcf421
Remove Painless Type in favor of Java Class in FunctionRef. (#28429) 2018-01-29 16:43:36 -08:00
Jack Conradson f13da9f534
Remove Painless Type from e-nodes in favor of Java Class (#28364) 2018-01-29 12:44:50 -08:00
Jack Conradson a57a0ae78b
Remove Painless Type from MethodWriter in favor of Java Class. (#28346) 2018-01-24 11:02:46 -08:00
Jack Conradson b945006938
Completely remove Painless Type from AnalyzerCaster in favor of Java Class. (#28329)
Second part in a series of PR's to remove Painless Type in favor of Java Class. This completely removes the Painless Type dependency from AnalyzerCaster. Both casting and promotion are now based on Java Class exclusively. This also allows AnalyzerCaster to be decoupled from Definition and make cast checks be static calls again.
2018-01-23 12:38:31 -08:00
Simon Willnauer 4d3f7a7695
Ensure we protect Collections obtained from scripts from self-referencing (#28335)
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
2018-01-23 16:57:26 +01:00
Jack Conradson ef5c041819
Painless: Replace Painless Type with Java Class during Casts (#27847)
This is the first step in a series to replace Painless Type with Java Class for any casting done during compilation. There should be no behavioural change.
2018-01-22 13:01:13 -08:00
Ryan Ernst ba9c9e08e7
Painless: Add spi jar that will be published for extending whitelists (#28302)
In order to build a plugin that extends the painless whitelist, the spi
classes must be available to the plugin at compile time. This commit
moves the spi classes into a separate jar which will be published. Any
plugin authors whiching to extend painless through spi would then add a
compileOnly dependency on this jar.
2018-01-18 19:16:26 -08:00
Ryan Ernst 18463e7e9f
Painless: Add whitelist extensions (#28161)
This commit adds a PainlessExtension which may be plugged in via SPI to
add additional classes, methods and members to the painless whitelist on
a per context basis. An example plugin adding and using a whitelist is
also added.
2018-01-15 11:28:31 -08:00
Jack Conradson 5d795afddb
Painless: Add public member read/write access test. (#28156) 2018-01-09 15:06:51 -08:00
Jack Conradson 1d1dcd4ae7
Painless: Add a simple cache for whitelist methods and fields. (#28142)
With support for multiple contexts we are adding some caching to the whitelist to keep the memory footprint for definitions from exploding.
2018-01-08 17:54:45 -08:00
Jack Conradson b5377d294f
Painless: Modify Loader to Load Classes Directly from Definition (#28088) 2018-01-05 13:06:36 -08:00
Ryan Ernst d36ec18029
Plugins: Add plugin extension capabilities (#27881)
This commit adds the infrastructure to plugin building and loading to
allow one plugin to extend another. That is, one plugin may extend
another by the "parent" plugin allowing itself to be extended through
java SPI. When all plugins extending a plugin are finished loading, the
"parent" plugin has a callback (through the ExtensiblePlugin interface)
allowing it to reload SPI.

This commit also adds an example plugin which uses as-yet implemented
extensibility (adding to the painless whitelist).
2018-01-03 11:12:43 -08:00
Jack Conradson 1de927c80d
Painless: Clean Up Painless Cast Object (#27794)
Added static methods to make creating Painless casts obvious as to what is
being boxed/unboxed.
2017-12-14 09:08:10 -08:00
Jack Conradson 8188d9f7e5
Painless: Only allow Painless type names to be the same as the equivalent Java class. (#27264)
Also adds a parameter called only_fqn to the whitelist to enforce that a painless type must be specified as the fully-qualifed java class name.
2017-12-11 16:37:35 -08:00
Robin Neatherway 057efea893 Correct two equality checks on incomparable types (#27688) 2017-12-07 14:18:11 +01:00
Jack Conradson 2d927fabab
Painless: Fix errors allowing void to be assigned to def. (#27460) 2017-11-28 13:44:52 -08:00
Jack Conradson 9e42b77f7e
Painless: Fix variable scoping issue in lambdas not including captured variables. (#27571) 2017-11-28 13:30:13 -08:00
Adrien Grand 996990ad1f
Upgrade to lucene-7.2.0-snapshot-8c94404. (#27496)
The main highlight of this new snapshot is that it introduces the opportunity
for queries to opt out of caching. In case a query opts out of caching, not only
will it never be cached, but also no compound query that wraps it will be
cached.
2017-11-28 14:52:42 +01:00
Jack Conradson abaede2373
Upgrade Painless from ANTLR 4.5.1-1 to ANTLR 4.5.3. (#27153) 2017-10-27 11:07:49 -07:00
Jack Conradson dda5d1af29 Allow for the Painless Definition to have multiple instances (#27096) 2017-10-26 08:33:55 -07:00
Nik Everett 4a06dd919a Painless: add tests for cached boxing (#24163)
We had a TODO about adding tests around cached boxing. In #24077
I tracked down the uncached boxing tests and saw the TODO. Cached
boxing testing is a fairly small extension to that work.
2017-10-10 10:34:03 -04:00
Nik Everett 38989191e7 Use LF line endings in Painless generated files (#26822)
Right now if you run `gradle regen` on Windows you'll get `CRLF` line
endings on all the ANTLR generated files because we run
```
    ant.fixcrlf(srcdir: outputPath) {
      patternset(includes: 'Painless*.java')
    }
```

The docs for fixcrlf say that the default line endings that it
corrects to is based on the OS:
https://ant.apache.org/manual/Tasks/fixcrlf.html

This change locks it to `LF`.
2017-10-06 16:44:03 -04:00
Jack Conradson c3746b268c Separate Painless Whitelist Loading from the Painless Definition (#26540)
Adds several small whitelist data structures and a new Whitelist class to separate the idea of loading a whitelist from the actual Painless Definition class. This is the first step of many in allowing users to define custom whitelists per context. Also supports the idea of loading multiple whitelists from different sources for a single context.
2017-09-18 15:51:07 -07:00
Michael Basnight f385e0cf26 Add bad_request to the rest-api-spec catch params (#26539)
This adds another request to the catch params. It also makes sure that
the generic request param does not allow 400 either.
2017-09-14 14:24:03 -05:00
Adrien Grand 93da7720ff Move non-core mappers to a module. (#26549)
Today we have all non-plugin mappers in core. I'd like to start moving those
that neither map to json datatypes nor are very frequently used like `date` or
`ip` to a module.

This commit creates a new module called `mappers-extra` and moves the
`scaled_float` and `token_count` mappers to it. I'd like to eventually move
`range` fields there but it's more complicated due to their intimate
relationship with range queries.

Relates #10368
2017-09-13 17:58:53 +02:00
Alexander Reelsen 80d0a32f8e ScriptService: Replace max compilation per minute setting with max compilation rate (#26399)
The current script service has a script compilation limit for a one
minute window. This is set to a small default value of 15. Instead of
increasing that default value, this commit introduces a new setting 
that allows to configure a rate per time unit, so that the script service can deal with bursts better.

The new setting is named `script.max_compilations_rate`,
requires a nonnegative number and a positive time value.

The default is `75/5m`, which is equivalent to the existing 15 per minute.
2017-09-01 10:15:27 +02:00
Yannick Welsch 0dbebd9177 Set classpath on Painless apiJavadoc task
so it also works on JDK 9
2017-08-23 14:45:40 +09:30
Jack Conradson 23858789f0 Remove Sort enum from Painless Definition (#26179)
This is step toward making Definition instanceable which is necessary for custom whitelists in different contexts.
2017-08-17 12:02:11 -07:00
Nik Everett b840fa3117 Fix some links in Painless method reference
Links to inner classes were using `$` in urls instead of `.`, causing
them to 404.

Also fixes the doc generation code to generate docs into the correct
directory. We moved the docs but never updated the generation code.
2017-08-17 10:50:38 -04:00
desmorto 292dd8f992 (refactor) some opportunities to use diamond operator (#25585)
* (refactor) some opportunities to use diamond operator

* Update ExceptionRetryIT.java

update typo
2017-08-15 16:36:42 -06:00
Adrien Grand f0cba4fce5 Add a scripted similarity. (#25831)
The goal of this similarity is to help users who would like to keep the
functionality of the `tf-idf` similarity that we want to remove, or to allow
for specific usec-cases (disabling idf, disabling tf, disabling length norm,
etc.) to not have to build a custom plugin and familiarize with the low-level
Lucene API.
2017-08-08 08:55:12 +02:00
Adrien Grand 53c829b6bc Painless: allow doubles to be casted to longs. (#25936)
Running `(long) someDoubleValue` currently throws a `ClassCastException` while
eg. `(int) someDoubleValue` is accepted.
2017-08-01 16:22:55 +02:00
Simon Willnauer b72c71083c Cleanup IndexFieldData visibility (#25900)
Today we expose `IndexFieldDataService` outside of IndexService to do maintenance
or lookup field data in different ways. Yet, we have a streamlined way to access IndexFieldData
via `QueryShardContext` that should encapsulate all access to it. This also ensures that we control all other functionality like cache clearing etc.

This change also removes the `recycler` option from `ClearIndicesCacheRequest` this option is a no-op and should have been removed long ago.
2017-07-26 20:03:42 +02:00
Simon Willnauer 634ce90dc0 Respect cluster alias in `_index` aggs and queries (#25885)
Today when we aggregate on the `_index` field the cross cluster search
alias is not taken into account. Neither is it respected when we search
on the field. This change adds support for cluster alias when the cluster
alias is present on the `_index` field.

Closes #25606
2017-07-26 09:16:52 +02:00
Jack Conradson 9f7463e796 remove lang url parameter from stored script requests (#25779)
Also has updates to ScriptMetaData for allowing the old namespace format to be loaded all the way back through 5.0; however, it will throw an exception if two scripts share the same id but different languages.
2017-07-20 08:51:08 -07:00
Jack Conradson 96b62409a8 Update Painless to Allow Augmentation from Any Class (#25360)
Custom whitelists in Painless will need to allow classes to be augmented beyond the currently hard-coded Augmentation class tied to Painless directly. This change allows any class to specify an augmentation on a Painless struct using an appropriate static method. Changes to loading the whitelist have also been created to allow for this specification of a different class for augmentation.
2017-06-22 12:16:46 -07:00
Jack Conradson 50db8cb351 Add needs methods for specific variables to Painless script context factories. (#25267) 2017-06-15 17:00:33 -07:00
Ryan Ernst caf7792db1 Scripting: Rename SearchScript.needsScores to needs_score (#25235)
This commit renames the needsScores method so as to make it
automatically generatable, based on the name of the `_score` variable
which is available in search scripts. It also adds documentation to
ScriptContext to explain the naming and signature of such methods.
2017-06-14 22:01:19 -07:00
Jack Conradson a4471f51e4 Support script context stateful factory in Painless. (#25233) 2017-06-14 16:44:41 -07:00
Ryan Ernst a03b6c2fa5 Scripting: Change keys for inline/stored scripts to source/id (#25127)
This commit adds back "id" as the key within a script to specify a
stored script (which with file scripts now gone is no longer ambiguous).
It also adds "source" as a replacement for "code". This is in an attempt
to normalize how scripts are specified across both put stored scripts and script usages, including search template requests. This also deprecates the old inline/stored keys.
2017-06-09 08:29:25 -07:00
Jack Conradson d187fa78fd Generate Painless Factory for Creating Script Instances (#25120) 2017-06-07 16:06:11 -07:00
Jack Conradson 8999104b14 Change ScriptContexts to use needs instead of uses$. (#25036) 2017-06-02 14:50:51 -07:00
Jack Conradson a926ace2e1 Update Painless to Use New Script Contexts (#25015)
*  All public methods starting with get will be added as local variables
to the execute method.
* The execute method on a ScriptContext must be both public and
abstract.  This method will be implemented by the Painless compiler.
* A static list of parameter names for the execute method must be
provided since the names will be eliminated at runtime.
* The uses$ methods will still be implemented as before.
* A single constructor may be provided by the ScriptContext.  This
constructor will be overridden by the Painless compiler to include the
exact same arguments.  This allows instances of a Painless script to
potentially contain state.  If a constructor is not provided it is
assumed the default constructor with no arguments will be used.
2017-06-02 13:36:45 -07:00
Ryan Ernst 8d88b94372 Scripting: Add optional context parameter to put stored script requests (#25014)
This commit adds an optional `context` url parameter to the put stored
script request. When a context is specified, the script is compiled
against that context before storing, as a validation the script will
work when used in that context.
2017-06-01 17:53:48 -07:00
Ryan Ernst 7c1211d2ed Scripting: Add StatefulFactoryType as optional intermediate factory in script contexts (#24974)
ScriptContexts currently understand a FactoryType that can produce
instances of the script InstanceType. However, for search scripts, this
does not work as we have the concept of LeafSearchScript that is created
per lucene segment. This commit effectively renames the existing
SearchScript class into SearchScript.LeafFactory, which is a new,
optional, class that can be defined within a ScriptContext.
LeafSearchScript is effectively renamed back into SearchScript. This
change allows the model of stateless factory -> stateful factory ->
script instance to continue, but in a generic way that any script
context may take advantage of.

relates #20426
2017-05-30 16:32:14 -07:00
Jack Conradson 04daac2243 Make Painless Compiler Use an Instance Per Context (#24972)
Allows for easier management of compilation of individual interfaces on a per script context basis.
2017-05-30 14:30:48 -07:00
Jack Conradson 5bcae914d9 Make PainlessScript an interface (#24966)
Allows more flexibility for the specified script context interface if we want to allow script contexts to specify an abstract class instead.
2017-05-30 09:03:46 -07:00
Ryan Ernst 74e031e842 Scripting: Rename CompiledType to FactoryType in ScriptContext (#24897)
This commit renames the concept of the "compiled type" to a "factory
type", along with all implementations of this class to be named Factory.
This brings it inline with the classes purpose.
2017-05-26 00:02:54 -07:00
Ryan Ernst 8eab1fefa1 Scripting: Make contexts available to ScriptEngine construction (#24896)
This commit adds collection of all contexts to the parameters of
getScriptEngine. This will allow script engines like painless to
precache extra information about the contexts.
2017-05-25 16:55:47 -07:00
Ryan Ernst 8aaea51a0a Scripting: Move context definitions to instance type classes (#24883)
This is a simple refactoring to move the context definitions into the
type that they use. While we have multiple context names for the same
class at the moment, this will eventually become one ScriptContext per
instance type, so the pattern of a static member on the interface called
CONTEXT can be used. This commit also moves the consolidated list of
contexts provided by core ES into ScriptModule.
2017-05-25 12:18:45 -07:00
Ryan Ernst 7d03cff820 Scripting: Make ScriptEngine.compile generic on the script context (#24873)
This commit changes the compile method of ScriptEngine to be generic in
the same way it is on ScriptService. This moves the shim of handling the
two existing context classes into each script engine, so that each
engine can be worked on independently to convert to real handling of
contexts.
2017-05-24 20:06:32 -07:00
Ryan Ernst 1daacd97b0 Scripting: Add instance and compiled classes to script contexts (#24868)
This commit modifies the compile method of ScriptService to be context
aware. The ScriptContext is now a generic class which contains both the
instance type and compiled type for a script. Instance type may be
stateful (for example, pre loading field information for the index a
script will execute on, like in expressions), while the compiled type is
stateless and used to construct instance type instances. This change is
only a first step to cutover ScriptService to the new paradigm. It only
converts callers to the script service, and has a small shim to wrap
compilation from the script engines to support the current two fixed
instance types, SearchScript and ExecutableScript.
2017-05-24 14:29:02 -07:00
Ryan Ernst 0ddd219423 Scripting: Add default implementation of close() for ScriptEngine (#24851)
Since groovy was removed, we no longer have any ScriptEngines with
resources to release. We may want to keep the option open for a script
engine to close resources, but this would not be common. This commit
adds a default implementation to ScriptEngine for `close()` to reduce
the boiler plate that must be added for a ScriptEngine implementation.
2017-05-24 13:19:27 -07:00
Ryan Ernst 52d504bb5f Scripting: Simplify ScriptContext (#24818)
As we work towards contexts implying the return type of compilation, we
first need ScriptContext to not be an enum. This commit removes the
Standard enum and Plugin subclass of ScriptContext.
2017-05-22 13:11:15 -07:00
Ryan Ernst 2de748859f Scripting: Remove "inline script enabled" on script engines (#24815)
ScriptEngine implementations have an overridable method to indicate they
are safe to use as inline scripts. Since groovy was removed fro 6.0,
there are no longer any implementations which used the default false
value. Furthermore, the value was not actually read anywhere. This
commit removes the method. The ScriptEngineRegistry was also no longer
necessary as it only was used to build a map from language to engine.
2017-05-20 12:01:25 -07:00
Ryan Ernst 463fe2f4d4 Scripting: Remove file scripts (#24627)
This commit removes file scripts, which were deprecated in 5.5.

closes #21798
2017-05-17 14:42:25 -07:00
Ryan Ernst 2a65bed243 Tests: Change rest test extension from .yaml to .yml (#24659)
This commit renames all rest test files to use the .yml extension
instead of .yaml. This way the extension used within all of
elasticsearch for yaml is consistent.
2017-05-16 17:24:35 -07:00
Koen De Groote f185b69e04 Replace manual copying an array or collection with static methods calls (#24657) 2017-05-15 09:35:48 +02:00
Koen De Groote 878ae8eb3c Size lists in advance when known
When constructing an array list, if we know the size of the list in
advance (because we are adding objects to it derived from another list),
we should size the array list to the appropriate capacity in advance (to
avoid resizing allocations). This commit does this in various places.

Relates #24439
2017-05-12 10:36:13 -04:00
Uwe Schindler f7c50f5f71 Painless: Optimize instance creation in LambdaBootstrap (#24618)
Optimize instance creation in LambdaBootstrap to allow Hotspot's escape analysis, preventing us from creating many instances stressing GC
2017-05-11 09:10:27 -07:00
Ryan Ernst 9ca7d28552 Scripting: Remove "service" from ScriptEngine interface name (#24574)
This commit renames ScriptEngineService to ScriptEngine.  It is often
confusing because we have the ScriptService, and then
ScriptEngineService implementations, but the latter are not services as
we see in other places in elasticsearch.
2017-05-10 00:47:33 -07:00
Koen De Groote 13c17c75b5 Remove unneeded empty string concatentation
This commit removes concatenation by empty string in places where it
is simply not needed to obtain a string representation.

Relates #24411
2017-05-06 00:28:53 -04:00
Nik Everett 9bc7e210a0 Test: Move flag to painless tests (#24494)
The `-XX:-OmitStackTraceInFastThrow` flag is only required by Painless's
tests so we'll only set it there. This is much simpler.
2017-05-04 13:11:09 -04:00
Nik Everett 3b47355e56 Try not to lose stacktraces (#24426)
This adds `-XX:-OmitStackTraceInFastThrow` to the JVM arguments
which *should* prevent the JVM from omitting stack traces on
common exception sites. Even though these sites are common, we'd
still like the exceptions to debug them.

This also adds the flag when running tests and adapts some tests
that had workarounds for the absense of the flag.

Closes #24376
2017-05-02 11:34:12 -04:00
Uwe Schindler 62fa7081b0 Painless: Add tests to check for existence and correct detection of the special Java 9 optimizations: Indified String concat and MethodHandles#ArrayLengthHelper() (#24405) 2017-05-02 08:08:51 -07:00
Uwe Schindler e88d54bf0a Painless: Fix method references to ctor with the new LambdaBootstrap and cleanup code (#24406)
* Fix wrong delegation to constructors when compiling lambdas with method references to ctors. Also remove the get$lambda factory.
* Cleanup code and remove unneeded transformations between binary and internal class names (uses ASM Type class instead)
* Cleanup Exception handling
* Simplification by moving the type adaption to the outside
* Remove STATIC access flag from our Lambda class (not required and also officially not allowed)
* Move the lambda counter to the classloader, so we have a per-script lambda ID
* Change Codesource of generated lambdas to be consistent
2017-05-01 16:15:13 -07:00
Nik Everett 5fbc86e2aa Allow painless to load stored fields (#24290)
We document that painless can load stored fields but it can't
because the classes that make that work aren't whitelisted.
2017-04-24 14:22:39 -04:00
Jack Conradson 30cc33e2e5 Fix Painless Lambdas for Java 9 (#24070)
Replaces LambdaMetaFactory with LambdaBootstrap, a custom solution for lambdas in Painless using a design similar to LambdaMetaFactory, but allows for custom adaptation of types which recent changes to LambdaMetaFactory no longer allowed.
2017-04-24 09:58:02 -07:00
Nik Everett db0a5e4263 Painless: more testing for script_stack (#24168)
`script_stack` is super useful when debugging Painless scripts
because it skips all the "weird" stuff involved that obfuscates
where the actual error is. It skips Painless's internals and
call site bootstrapping.

It works fine, but it didn't have many tests. This converts a
test that we had for line numbers into a test for the
`script_stack`. The line numbers test was an indirect test
for `script_stack`.
2017-04-18 22:52:59 -04:00
Ryan Ernst 212f24aa27 Tests: Clean up rest test file handling (#21392)
This change simplifies how the rest test runner finds test files and
removes all leniency.  Previously multiple prefixes and suffixes would
be tried, and tests could exist inside or outside of the classpath,
although outside of the classpath never quite worked. Now only classpath
tests are supported, and only one resource prefix is supported,
`/rest-api-spec/tests`.

closes #20240
2017-04-18 15:07:08 -07:00
Nik Everett 0b15fde27a Start on custom whitelists for Painless (#23563)
We'd like to be able to support context-sensitive whitelists in
Painless but we can't now because the whitelist is a static thing.
This begins to de-static the whitelist, in particular removing
the static keyword from most of the methods on `Definition` and
plumbing the static instance into the appropriate spots as though
it weren't static. Once we de-static all the methods we should be
able to fairly simply build context-sensitive whitelists.

The only "fun" bit of this is that I added another layer in the
chain of methods that bootstraps `def` calls. Instead of running
`invokedynamic` directly on `DefBootstrap` we now `invokedynamic`
`$bootstrapDef` on the script itself loads the `Definition` that
the script was compiled against and then calls `DefBootstrap`.

I chose to put `Definition` into `Locals` so I didn't have to
change the signature of all the `analyze` methods. I could have
do it another way, but that seems ok for now.
2017-04-18 10:39:42 -04:00
Nik Everett 25119a7e78 Harden painless test against "fun" caching (#24077)
The JVM caches `Integer` objects. This is known. A test in Painless
was relying on the JVM not caching the particular integer `1000`.
It turns out that when you provide `-XX:+AggressiveOpts` the JVM
*does* cache `1000`, causing the test to fail when that is
specified.

This replaces `1000` with a randomly selected integer that we test
to make sure *isn't* cached by the JVM. *Hopefully* this test is
good enough. It relies on the caching not changing in between when
we check that the value isn't cached and when we run the painless
code. The cache now is a simple array but there is nothing
preventing it from changing. If it does change in a way that thwarts
this test then the test fail fail again. At least when that happens
the next person can see the comment about how it is important
that the integer isn't cached and can follow that line of inquiry.

Closes #24041
2017-04-17 13:44:05 -04:00
Jason Tedor 653619079c Skip two Painless branch tests on Windows
This commit skips the two Painless tests
EqualsTests#testBranchEqualsDefAndPrimitive and
EqualsTests#testBranchNotEqualsDefAndPrimitive on Windows as the tests
are repeatedly failing there.
2017-04-11 06:19:42 -04:00
Jason Tedor 3136ed1490 Rename random ASCII helper methods
This commit renames the random ASCII helper methods in ESTestCase. This
is because this method ultimately uses the random ASCII methods from
randomized runner, but these methods actually only produce random
strings generated from [a-zA-Z].

Relates #23886
2017-04-04 11:04:18 -04:00
Ryan Ernst 4cb8a0100c Build: Rewrite antlr regeneration in gradle (#23733)
This change ports the regeneration of antlr parser/lexer into gradle
(but does still take advantage of ant calls where appropriate).
2017-03-24 09:44:53 -07:00
Ryan Ernst 8c53555b28 Tests: Use local clone build of 5.x with bwc tests (#22946)
The current rest backcompat tests, which run against a mixed cluster of
5.x and 6.0 nodes, depend on snapshot builds of 5.x. However, this has
the potential for inconsistency that results in CI failures, and happens
quite often, whenever some backcompat logic is added to 5.x, but the bwc
test on master fails because the 5.x code has not yet been published as
a snapshot.

This change creates a git clone of the 5.x branch,
builds the zip distribution, and ties that into gradle substitutions for
the 5.x version.
2017-03-23 22:32:13 -07:00
Nik Everett 257a7d77ed Painless: Fix regex lexer and error messages (#23634)
Without this change, if write a script with multiple regexes
*sometimes* the lexer will decide to look at them like one
big regex and then some trailing garbage. Like this discuss post:
https://discuss.elastic.co/t/error-with-the-split-function-in-painless-script/79021

```
def val = /\\\\/.split(ctx._source.event_data.param17);
if (val[2] =~ /\\./) {
  def val2 = /\\./.split(val[2]);
  ctx._source['user_crash'] = val2[0]
} else {
  ctx._source['user_crash'] = val[2]
}
```

The error message you get from the lexer is `lexer_no_viable_alt_exception`
right after the *second* regex.

With this change each regex is just a single regex like it ought to be.

As a bonus, while looking into this issue I found that the error
reporting for regexes wasn't very nice. If you specify an invalid
pattern then you get an error marker on the start of the pattern
with the JVM's regex error message which attempts to point you to the
location in the regex but is totally unreadable in the JSON response.

This change fixes the location to point to the appropriate spot
inside the pattern and removes the portion of the JVM's error message
that doesn't render well. It is no longer needed now that we point
users to the appropriate spot in the pattern.
2017-03-22 15:56:17 -04:00
Jason Tedor 8e09eca9a6 Mute Painless lambda tests on JDK 9
This commit mutes a ton of Painless lambda tests on JDK 9. This commit
did not attempt to discover exactly which tests are failing, but instead
just blanket muted all tests in LambdaTests, FunctionRefTests, and
AugmentationTests.

Relates #23473
2017-03-02 22:36:26 -05:00
Luca Cavanna cc65a94fd4 [TEST] improve yaml test sections parsing (#23407)
Throw error when skip or do sections are malformed, such as they don't start with the proper token (START_OBJECT). That signals bad indentation, which would be ignored otherwise. Thanks (or due to) our pull parsing code, we were still able to properly parse the sections, yet other runners weren't able to.

Closes #21980

* [TEST] fix indentation in matrix_stats yaml tests

* [TEST] fix indentation in painless yaml test

* [TEST] fix indentation in analysis yaml tests

* [TEST] fix indentation in generated docs yaml tests

* [TEST] fix indentation in multi_cluster_search yaml tests
2017-03-02 12:43:20 +01:00
javanna dad025a6ad [TEST] move test for binary field to specific test file that sets Content-Type header explicitly 2017-02-27 12:27:03 +01:00
Ryan Ernst 175bda64a0 Build: Rework integ test setup and shutdown to ensure stop runs when desired (#23304)
Gradle's finalizedBy on tasks only ensures one task runs after another,
but not immediately after. This is problematic for our integration tests
since it allows multiple project's integ test clusters to be
simultaneously. While this has not been a problem thus far (gradle 2.13
happened to keep the finalizedBy tasks close enough that no clusters
were running in parallel), with gradle 3.3 the task graph generation has
changed, and numerous clusters may be running simultaneously, causing
memory pressure, and thus generally slower tests, or even failure if the
system has a limited amount of memory (eg in a vagrant host).

This commit reworks how integ tests are configured. It adds an
`integTestCluster` extension to gradle which is equivalent to the current
`integTest.cluster` and moves the rest test runner task to
`integTestRunner`.  The `integTest` task is then just a dummy task,
which depends on the cluster runner task, as well as the cluster stop
task. This means running `integTest` in one project will both run the
rest tests, and shut down the cluster, before running `integTest` in
another project.
2017-02-22 12:43:15 -08:00
Nik Everett 38d25a0369 Fix Painless's implementation of interfaces returning primitives (#23298)
Fixes Painless to properly implement scripts that return primitives
and void. Adds some simple tests that we emit sane opcodes and some
other tests that we implement primitives as expected.

Mostly this is just a fix following up from #22983 but there is one
thing I did really worth talking about, I think. So, before this script
Painless scripts could only ever return Object and they did would always
return null for paths that didn't return any values. Now that they
can return primitives the question is "what should Painless return
from paths that don't return any values?" And I answered that with
"whatever the JLS default value is". So 0/0L/0f/0d/false.
2017-02-21 17:10:55 -05:00
Nik Everett 9105672969 Allow painless to implement more interfaces (#22983)
Generalizes three previously hard coded things in painless into
generic concepts:

1. The "main method" is no longer hardcoded to:
```
public abstract Object execute(Map<String, Object> params,
        Scorer scorer, LeafDocLookup doc, Object value);
```
Instead Painless's compiler takes an interface and implements it. It looks like:
```
public interface SomeScript {
    // Argument names we expose to Painless scripts
    String[] ARGUMENTS = new String[] {"a", "b"};
    // Method implemented by Painless script. Must be named execute but can have any parameters or return any value.
    Object execute(String a, int b);
    // Is the "a" argument used by the script?
    boolean uses$a();
}
SomeScript script = scriptEngine.compile(SomeScript.class, null, "the_script_here", emptyMap());
Object result = script.execute("a", 1);
```

`PainlessScriptEngine` now compiles all scripts to the new
`GenericElasticsearchScript` interface by default for compatibility
with the rest of Elasticsearch until it is able to use this new
ability.

2. `_score` and `ctx` are no longer hardcoded to be extracted from
`#score` and `params` respectively. Instead Painless's default
implementation of Elasticsearch scripts uses the `uses$_score` and
`uses$ctx` methods to determine if it is used and gives them
dummy values if they are not used.

3. Throwing the `ScriptException` is now handled by the Painless
script itself. That way Painless doesn't have to leak the metadata
that is required to build the fancy stack trace. And all painless scripts
get the fancy stack trace.
2017-02-21 14:08:57 -05:00
Jack Conradson fac2d954e3 Fix certain bad casts in Painless due to boxing/unboxing. (#23282) 2017-02-21 10:23:27 -08:00
Simon Willnauer aef0665ddb Detach SearchPhases from AbstractSearchAsyncAction (#23118)
Today all search phases are inner classes of AbstractSearchAsyncAction or one of it's
subclasses. This makes unit testing of these classes practically impossible. This commit
Extracts `DfsQueryPhase` and `FetchSearchPhase` or of the code that composes the actual
query execution types and moves most of the fan-out and collect code into an `InitialSearchPhase`
class that can be used to build initial search phases (phases that retry on shards). This will
make modification to these classes simpler and allows to easily compose or add new search phases
down the road if additional roundtrips are required.
2017-02-14 12:34:25 +01:00
Nik Everett 0d6e622242 Make dates be ReadableDateTimes in scripts (#22948)
Instead of longs. If you want millis since epoch you can call doc.date_field.value.millis.

Relates to #22875
2017-02-06 16:44:56 -05:00
Nik Everett b0c9759441 Painless: Don't allow casting from void to def (#22969)
Painless can cast anything into the magic type `def` but it
really shouldn't try to cast **nothing** into `def`. That causes
the byte code generation library to freak out a little.

Closes #22908
2017-02-03 16:38:47 -05:00
Jason Tedor 9a0b216c36 Upgrade checkstyle to version 7.5
This commit upgrades the checkstyle configuration from version 5.9 to
version 7.5, the latest version as of today. The main enhancement
obtained via this upgrade is better detection of redundant modifiers.

Relates #22960
2017-02-03 09:46:44 -05:00
Nik Everett 73bf29072f Painless: Fix def invoked qualified method refs (#22918)
We were incorrectly resolving qualified method references at run
time when invoked on `def`. This lead to errors like
`The struct with name [org] has not been defined.` when attempting

```
doc.date.dates.stream().map(
  org.joda.time.ReadableDateTime::centuryOfEra
).collect(Collectors.toList())
```
2017-02-02 10:15:03 -05:00
Nik Everett dacc150934 Expose multi-valued dates to scripts and document painless's date functions (#22875)
Implemented by wrapping an array of reused `ModuleDateTime`s that
we grow when needed. The `ModuleDateTime`s are reused when we
move to the next document.

Also improves the error message returned when attempting to modify
the `ScriptdocValues`, removes a couple of allocations, and documents
that the date functions are available in Painless.

Relates to #22162
2017-02-01 21:57:07 -05:00
Jack Conradson 3d2626c4c6 Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.

The new behavior is the following:

When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.

Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].

When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.

Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.

When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 13:27:02 -08:00
Nik Everett 8a2d424d68 Generate reference links for painless API (#22775)
Adds "Appending B. Painless API Reference", a reference of all classes
and methods available from Painless. Removes links to java packages
because they contain methods that we don't expose and don't contain
methods that we do expose (the ones in Augmentation). Instead this
generates a list of every class and every exposed method using the same
type information available to the
interpreter/compiler/whatever-we-call-it. From there you can jump to
the relevant docs.

Right now you build all the asciidoc files by running
```
gradle generatePainlessApi
```

These files are expected to be committed because we build the docs
without running `gradle`.

Also changes the output of `Debug.explain` so that it is easy to
search for the class in the generated reference documentation.

You can also run it in an IDE safely if you pass the path to the
directory in which to generate the docs as the first parameter. It'll
blow away the entire directory an recreate it from scratch so be careful.

And then you can build the docs by running something like:
```
../docs/build_docs.pl --out ../built_docs/ --doc docs/reference/index.asciidoc --open
```

That is, if you have checked out https://github.com/elastic/docs in
`../docs`. Wait a minute or two and your browser will pop open in with
all of Elasticsearch's reference documentation. If you go to
`http://localhost:8000/painless-api-reference.html` you can see this
list. Or you can get there by following the links to `Modules` and
`Scripting` and `Painless` and then clicking the link in the paragraphs
below titled `Appendix B. Painless API Reference`.

I like having these in asciidoc because we can deep link to them from the
rest of the guide with constructs like
`<<painless-api-reference-Object-hashCode-0>>` and
`<<painless-api-reference->>` and we get link checking. Then the only
brittle link maintenance bit is the link generation for javadoc. Which
sucks. But I think it is important that we link to the methods directly
so they are easy to find.

Relates to #22720
2017-01-26 10:39:19 -05:00
Luca Cavanna 47c0e13a3b Stop returning "es." internal exception headers as http response headers (#22703)
move "es." internal headers to separate metadata set in ElasticsearchException and stop returning them as response headers

Closes #17593

* [TEST] remove ESExceptionTests, move its methods to ElasticsearchExceptionTests or ExceptionSerializationTests
2017-01-24 16:12:45 +01:00
Nik Everett 28cfc533e2 Generate javadoc jar for painless's public API (#22704)
The simplest way to do that is to move the public API into a
new package and generate javadoc for that package.
2017-01-23 17:16:20 -05:00
Tim Brooks a4ac29c005 Add single static instance of SpecialPermission (#22726)
This commit adds a SpecialPermission constant and uses that constant
opposed to introducing new instances everywhere.

Additionally, this commit introduces a single static method to check that
the current code has permission. This avoids all the duplicated access
blocks that exist currently.
2017-01-21 12:03:52 -06:00
Nik Everett 22f1c9fa0f Remove @header we no longer need 2017-01-19 11:44:13 -05:00
Nik Everett bb83c283bb Make lexer abstract 2017-01-19 11:41:50 -05:00
Nik Everett dbb4a2ca6c Move lexer hacks to EnhancedPainlessLexer
This "feels" nicer. Less classes at least.
2017-01-19 11:23:16 -05:00
Nik Everett e2da6a8ee5 Improve painless's javadocs
Hopefully useful references.
2017-01-19 11:04:08 -05:00
Nik Everett 3ce41a0e15 Painless: Add augmentation to string for base 64 (#22665)
We don't want to expose `String#getBytes` which is required for
`Base64.getEncoder.encode` to work because we're worried about
character sets. This adds `encodeBase64` and `decodeBase64`
methods to `String` in Painless that are duals of one another
such that:
`someString == someString.encodeBase64().decodeBase64()`.

Both methods work with the UTF-8 encoding of the string.

Closes #22648
2017-01-19 09:31:45 -05:00
Nik Everett baed02bbe2 Whitelist some ScriptDocValues in painless (#22600)
Without this whitelist painless can't use ip or binary doc values.

Closes #22584
2017-01-12 15:26:09 -05:00
Jack Conradson 0c694b3d19 Update loop counter to be higher (1000000) instead of (10000). 2017-01-11 09:22:24 -08:00
Nik Everett f24ca5188a Fix some issues with painless's strings (#22393)
1. Escape sequences we're working. For example `\\` is now correctly
interpreted as `\` instead of `\\`. Same with `\'` being `'` and
`\"` being `"`.
2. `'` delimited strings weren't allowed to contain `"`s but it looked
like they were intended to support it. Now they do.
3. Improves the error message when the script contains an invalid
escape sequence inside a string to include a list of the valid
escape sequences.

Closes #22372
2017-01-06 11:35:22 -05:00
Nik Everett f5f2149ff2 Remove much ceremony from parsing client yaml test suites (#22311)
* Remove a checked exception, replacing it with `ParsingException`.
* Remove all Parser classes for the yaml sections, replacing them with static methods.
* Remove `ClientYamlTestFragmentParser`. Isn't used any more.
* Remove `ClientYamlTestSuiteParseContext`, replacing it with some static utility methods.

I did not rewrite the parsers using `ObjectParser` because I don't think it is worth it right now.
2016-12-22 11:00:34 -05:00
Jack Conradson 0ecdef026d Test fix for def equals test in Painless. (#21945)
Closes #21801
2016-12-02 14:41:13 -08:00
Tanguy Leroux 28dc02f01a [Test] Mute EqualsTests..testBranch(Not)EqualsDefAndPrimitive
It fails regurlarly and it is tracked by https://github.com/elastic/elasticsearch/issues/21801
2016-11-25 17:21:59 +01:00
Ryan Ernst c3ec8e22b8 Wrap VerifyError in ScriptException (#21769)
If a bug occurs in painless compilation (not from a user, but from the
painless infrastructure), a VerifyError may be thrown when compiling the
broken generated class. This commit wraps VerifyErrors in
ScriptException so that useful information is returned to the user,
which can be passed on to the ES team for analysis.
2016-11-23 14:45:21 -08:00
Jack Conradson ba2d772668 Fix a VerifyError bug in Painless (#21765)
This bug would cause a VerifyError when scripts using the === operator
were comparing a def type against a primitive type since the primitive
type wasn't being appropriately boxed.
2016-11-23 13:57:14 -08:00
Nik Everett 434fa4bd26 Docs and tests for painless lack of boxing for ?: and ?. (#21756)
NOTE: The result of `?.` and `?:` can't be assigned to primitives. So
`int[] someArray = null; int l = someArray?.length` and
`int s = params.size ?: 100` don't work. Do
`def someArray = null; def l = someArray?.length` and
`def s = params.size ?: 100` instead.

Relates to #21748
2016-11-23 14:33:32 -05:00
Nik Everett dbdcf9e95c Move painless yaml tests into painless dir
They were in a directory named "plan_a", the old name for painless.
2016-11-22 20:27:14 -05:00
Nik Everett 457c2d8fb0 Add Debug.explain to painless
You can use `Debug.explain(someObject)` in painless to throw an
`Error` that can't be caught by painless code and contains an
object's class. This is useful because painless's sandbox doesn't
allow you to call `someObject.getClass()`.

Closes #20263
2016-11-22 12:46:02 -05:00
Nik Everett f5c8c746e6 Implement toString in painless's AST
This should make debugging painless' analysis and code generation a
little easier.

The `toString` implementations mirror the AST somewhat, and look like
`(SSource (SReturn (ENumeric 1)))`.
2016-11-21 16:24:10 -05:00
Nik Everett ae468441dc Implement the ?: operator in painless (#21506)
Implements a null coalescing operator in painless that looks like `?:`. This form was chosen to emulate Groovy's `?:` operator. It is different in that it only coalesces null values, instead of Groovy's `?:` operator which coalesces all falsy values. I believe that makes it the same as Kotlin's `?:` operator. In other languages this operator looks like `??` (C#) and `COALESCE` (SQL) and `:-` (bash).

This operator is lazy, meaning the right hand side is only evaluated at all if the left hand side is null.
2016-11-18 13:54:26 -05:00
Jack Conradson ced433e9a8 Fix reserved variable availability in lambdas in Painless 2016-11-17 13:39:08 -08:00
Nik Everett 2a328034ef Support decimal constants with trailing [dD] in painless (#21412)
This adds support to painless for decimal constants with trailing `d` or
`D` to make it compatible with Java. It already supported integer
constants with a trailing `d` or `D` but this adds tests for it.

Closes #21116
2016-11-12 11:08:39 -05:00
Nik Everett a26b5a113c In painless suggest a long constant if int won't do (#21415)
In painless we prefer explicit types over implicit ones whereas
groovy is the other way around. Take this groovy code:

```
> 86400000.class
java.lang.Integer
> 864000000000.class
java.lang.Long
```

Painless accepts `86400000` just fine because that is a valid `int`
in the jvm. It rejects `864000000000` as an invlid `int` constant
because, in painless as in java, `long` constants always end in `L`
or `l`.

To ease the transition from groovy to painless, this changes the
compilation error returned from these invalid constants from:

```
Invalid int constant [864000000000].
```

to

```
Invalid int constant [864000000000]. If you want a long constant then change it to [864000000000L].
```

Inspired by #21313
2016-11-12 11:08:18 -05:00
Nik Everett d03b8e4abb Implement reading from null safe dereferences
Null safe dereferences make handling null or missing values shorter.
Compare without:
```
if (ctx._source.missing != null && ctx._source.missing.foo != null) {
  ctx._source.foo_length = ctx.source.missing.foo.length()
}
```

To with:
```
Integer length = ctx._source.missing?.foo?.length();
if (length != null) {
  ctx._source.foo_length = length
}
```

Combining this with the as of yet unimplemented elvis operator allows
for very concise defaults for nulls:
```
ctx._source.foo_length = ctx._source.missing?.foo?.length() ?: 0;
```

Since you have to start somewhere, we started with null safe dereferenes.

Anyway, this is a feature borrowed from groovy. Groovy allows writing to
null values like:
```
def v = null
v?.field = 'cat'
```
And the writes are simply ignored. Painless doesn't support this at this
point because it'd be complex to implement and maybe not all that useful.

There is no runtime cost for this feature if it is not used. When it is
used we implement it fairly efficiently, adding a jump rather than a
temporary variable.

This should also work fairly well with doc values.
2016-11-09 07:20:11 -05:00
Ryan Ernst 7a2c984bcc Test: Remove multi process support from rest test runner (#21391)
At one point in the past when moving out the rest tests from core to
their own subproject, we had multiple test classes which evenly split up
the tests to run. However, we simplified this and went back to a single
test runner to have better reproduceability in tests. This change
removes the remnants of that multiplexing support.
2016-11-07 15:07:34 -08:00
Nik Everett 24d5f31a54 Make painless's assertion about out of bound less brittle
Instead of asserting that the message is shaped a certain way we
cause the exception and catch it and assert that the messages are
the same. This is the way to go because the exception message from
the jvm is both local and jvm dependent.

This is the CI failure that found this:
https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+5.x+java9-periodic/515/consoleFull
2016-11-02 12:38:51 -04:00
Nik Everett 1bbd3c5400 Fix painless's out of bounds assertions in java 9
Java 9's exception message when lists have an out of bounds index
is much better than java 8 but the painless code asserted on the
java 8 message. Now it'll accept either.

I'm tempted to weaken the assertion but I like asserting that the
message is readable.
2016-10-29 22:21:57 -04:00
Nik Everett 3a7a218e8f Support negative array ofsets in painless
Adds support for indexing into lists and arrays with negative
indexes meaning "counting from the back". So for if
`x = ["cat", "dog", "chicken"]` then `x[-1] == "chicken"`.

This adds an extra branch to every array and list access but
some performance testing makes it look like the branch predictor
successfully predicts the branch every time so there isn't a
in execution time for this feature when the index is positive.
When the index is negative performance testing showed the runtime
is the same as writing `x[x.length - 1]`, again, presumably thanks
to the branch predictor.

Those performance metrics were calculated for lists and arrays but
`def`s get roughly the same treatment though instead of inlining
the test they need to make a invoke dynamic so we don't screw up
maps.

Closes #20870
2016-10-29 16:12:40 -04:00
Jack Conradson 512a77a633 Refactor ScriptType to be a top-level class. 2016-10-26 10:21:22 -07:00
Jack Conradson ceaae47d38 Remove more equivalents of the now method from the Painless whitelist. 2016-10-20 10:35:26 -07:00
Ryan Ernst 3d3dd7185d Add support for booleans in scripts (#20950)
* Scripting: Add support for booleans in scripts

Since 2.0, booleans have been represented as numeric fields (longs).
However, in scripts, this is odd, since you expect doing a comparison
against a boolean to work. While languages like groovy will auto convert
between booleans and longs, painless does not.

This changes the doc values accessor for boolean fields in scripts to
return Boolean objects instead of Long objects.

closes #20949

* Make Booleans final and remove wrapping of `this` for getValues()
2016-10-17 11:11:42 -07:00
Tanguy Leroux e4c7d8183e XContentBuilder: Avoid building self-referencing objects (#20550)
Some objects like maps, iterables or arrays of objects can self-reference themselves. This is mostly due to a bug in code but the XContentBuilder should be able to detect such situations and throws an IllegalArgumentException instead of building objects over and over until a stackoverflow occurs.

closes #20540
closes #19475
2016-10-11 11:41:54 +02:00
Simon Willnauer 37ca38df3d Expose `ctx._now` in update scripts (#20835)
Update scripts might want to update the documents `_timestamp` but need a notion of `now()`.
Painless doesn't support any notion of now() since it would make scripts non-pure functions. Yet,
in the update case this is a valid value and we can pass it with the context together to allow the
script to record the timestamp the document was updated.

Relates to #17895
2016-10-10 21:14:14 +02:00
Jack Conradson ba88d9af57 Remove all date 'now' methods from the Painless whitelist to ensure
Painless scripts are pure functions.
2016-10-05 09:47:20 -07:00
Jack Conradson 5755dd256d Fix String concatentation bug. 2016-09-21 15:49:56 -07:00
Nik Everett 69bf08f6c6 Disable regexes by default in painless
Adds a new node level, non-dynamic setting, `script.painless.regex.enabled`
can be used to enable regexes.

Closes #20397
2016-09-12 14:09:43 -04:00
Jack Conradson 3deea3dbde Made for/each break tests more robust in Painless. 2016-08-24 15:17:18 -07:00
Jack Conradson c60885b5d4 Fix break bug in for/foreach loops. 2016-08-24 14:25:54 -07:00
Jack Conradson f38d64039b Catch OutOfMemory and StackOverflow errors in Painless since it's safe
to do so.
2016-08-10 18:09:45 -07:00
Lee Hinman 5849c488b5 Merge remote-tracking branch 'dakrone/compliation-breaker' 2016-08-09 11:57:26 -06:00
Lee Hinman 2be52eff09 Circuit break the number of inline scripts compiled per minute
When compiling many dynamically changing scripts, parameterized
scripts (<https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting-using.html#prefer-params>)
should be preferred. This enforces a limit to the number of scripts that
can be compiled within a minute. A new dynamic setting is added -
`script.max_compilations_per_minute`, which defaults to 15.

If more dynamic scripts are sent, a user will get the following
exception:

```json
{
  "error" : {
    "root_cause" : [
      {
        "type" : "circuit_breaking_exception",
        "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
        "bytes_wanted" : 0,
        "bytes_limit" : 0
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "i",
        "node" : "a5V1eXcZRYiIk8lecjZ4Jw",
        "reason" : {
          "type" : "general_script_exception",
          "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
          "caused_by" : {
            "type" : "circuit_breaking_exception",
            "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
            "bytes_wanted" : 0,
            "bytes_limit" : 0
          }
        }
      }
    ],
    "caused_by" : {
      "type" : "general_script_exception",
      "reason" : "Failed to compile inline script [\"aaaaaaaaaaaaaaaa\"] using lang [painless]",
      "caused_by" : {
        "type" : "circuit_breaking_exception",
        "reason" : "[script] Too many dynamic script compilations within one minute, max: [15/min]; please use on-disk, indexed, or scripts with parameters instead",
        "bytes_wanted" : 0,
        "bytes_limit" : 0
      }
    }
  },
  "status" : 500
}
```

This also fixes a bug in `ScriptService` where requests being executed
concurrently on a single node could cause a script to be compiled
multiple times (many in the case of a powerful node with many shards)
due to no synchronization between checking the cache and compiling the
script. There is now synchronization so that a script being compiled
will only be compiled once regardless of the number of concurrent
searches on a node.

Relates to #19396
2016-08-09 10:26:27 -06:00
Nicholas Knize 2d590af593 Deprecate GeoDistance enumerators and remove geo distance script helpers
GeoDistance is implemented using a crazy enum that causes issues with the scripting modules. This commit moves all distance calculations to arcDistance and planeDistance static methods in GeoUtils. It also removes unnecessary distance helper methods from ScriptDocValues.GeoPoints.
2016-08-05 18:42:06 -05:00
Jack Conradson 9d87314105 Refactored Painless link nodes into expression nodes to simplify
load/store operations of variable/method chains.

Closes #19459
2016-07-27 14:01:08 -07:00
Nik Everett 9270e8b22b Rename client yaml test infrastructure
This makes it obvious that these tests are for running the client yaml
suites. Now that there are other ways of running tests using the REST
client against a running cluster we can't go on calling the shared
client yaml tests "REST tests". They are rest tests, but they aren't
**the** rest tests.
2016-07-26 13:53:44 -04:00
Nik Everett a95d4f4ee7 Add Location header and improve REST testing
This adds a header that looks like `Location: /test/test/1` to the
response for the index/create/update API. The requirement for the header
comes from https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

https://tools.ietf.org/html/rfc7231#section-7.1.2 claims that relative
URIs are OK. So we use an absolute path which should resolve to the
appropriate location.

Closes #19079

This makes large changes to our rest test infrastructure, allowing us
to write junit tests that test a running cluster via the rest client.
It does this by splitting ESRestTestCase into two classes:
* ESRestTestCase is the superclass of all tests that use the rest client
to interact with a running cluster.
* ESClientYamlSuiteTestCase is the superclass of all tests that use the
rest client to run the yaml tests. These tests are shared across all
official clients, thus the `ClientYamlSuite` part of the name.
2016-07-25 17:02:40 -04:00
Martijn van Groningen 2c3165d080 Removed deprecated 1.x script and template syntax
Closes #13729
2016-07-13 15:07:36 +02:00
Tanguy Leroux 0e7faf1005 Enable Checkstyle RedundantModifier 2016-07-04 15:22:12 +02:00
Tanguy Leroux 8c40b2b54e Fix order of modifiers 2016-07-01 16:57:14 +02:00
Nik Everett 67bfecc070 Painless: add "".replaceAll and "".replaceFirst
These are useful methods in groovy that give you control over
the replacements used:
```
'the quick brown fox'.replaceAll(/[aeiou]/,
		m -> m.group().toUpperCase(Locale.ROOT))
```
2016-06-28 16:39:11 -04:00
Robert Muir 6fc1a22977 cutover some docs to painless 2016-06-27 09:55:16 -04:00
Robert Muir 0b2baa7f63 Merge pull request #19065 from rmuir/help_painless_docs
Bring painless docs closer to reality
2016-06-24 12:52:30 -04:00
Robert Muir e6819648cc fix the primitive case of instanceof 2016-06-24 12:44:57 -04:00
Robert Muir 001a060c84 Bring painless docs closer to reality 2016-06-24 12:06:41 -04:00
Robert Muir f8c55a5e7b painless: fix disabled loop counter 2016-06-22 08:40:20 -04:00
Jack Conradson 0b4fc85367 Added some brief docs. 2016-06-21 12:56:54 -07:00
Jack Conradson 346b1802ee Quick fixes for using built in method writes. 2016-06-21 12:53:04 -07:00
Jack Conradson 553214d771 Merge branch 'master' into init2 2016-06-21 12:33:42 -07:00
Jack Conradson d2c823e4cc Add initializers to Painless for arrays, lists, and maps. 2016-06-21 12:32:10 -07:00
Robert Muir 1b9695a9aa beef up tests so we ensure you still get good errors in these cases 2016-06-21 12:15:59 -04:00
Robert Muir f78ef232dc fix bogus comment 2016-06-21 12:05:10 -04:00
Robert Muir 42d60f9f28 maps n lists 2016-06-21 11:25:43 -04:00
Robert Muir 80734c75b5 get things started 2016-06-21 08:35:12 -04:00
Robert Muir 1b7d35e4a7 Merge pull request #18983 from rmuir/lambda_types
Infer lambda arguments/return type
2016-06-20 17:22:04 -04:00
Robert Muir 1cc0264827 Infer lambda arguments/return type 2016-06-20 14:54:45 -04:00
Robert Muir fea120b073 Merge branch 'master' into explicit_casts 2016-06-20 13:39:13 -04:00
Robert Muir 006829e89b add simple arguments test 2016-06-20 13:33:51 -04:00
Robert Muir 09305a0f98 Merge pull request #18954 from rmuir/lambda_captures
Painless: add lambda captures
2016-06-20 13:05:55 -04:00
Robert Muir 26a73c39bf throw error if the tree is totally malformed 2016-06-20 13:01:59 -04:00
Robert Muir 9510a8f39e add a few more tests 2016-06-20 10:46:23 -04:00
Robert Muir 28b1b149ab remove unused import 2016-06-20 08:32:53 -04:00
Robert Muir 9111ed3e2c add docs 2016-06-20 08:24:41 -04:00
Robert Muir cd1a7b441c Improve error messages for lambdas when the number of arguments is wrong 2016-06-20 07:57:00 -04:00
Robert Muir 4d78be5b9e remove arity restriction (as def call incorporates all lambdas and all their captures) 2016-06-20 05:37:31 -04:00
Robert Muir b53d735602 Function/Lambda parameters do not need to be lenient... 2016-06-20 05:05:34 -04:00
Tanguy Leroux 98951b1203 Compile each Groovy script in its own classloader
closes #18572
2016-06-20 08:17:09 +02:00
Uwe Schindler 5475e18ad0 Update forbiddenapis to 2.2 and fix painless tests 2016-06-19 20:40:38 +02:00
Robert Muir 8d9fa7e0b5 Fix explicit casts and improve tests. 2016-06-19 03:19:45 -04:00
Robert Muir a14ba1e5b2 Painless: add lambda captures 2016-06-18 10:23:35 -04:00
Robert Muir e8826708c1 Refactor variables 2016-06-17 17:40:52 -04:00
Nik Everett 1e16c22d03 Painless: move semicolon hack into lexer
Perviously we used token level lookbehind in the parser. That worked,
but only if the parser didn't have any ambiguity *at all*. Since the
parser has ambiguity it didn't work everywhere. In particular it failed
when parsing blocks in lambdas like `a -> {int b = a + 2; b * b}`.

This moves the hack from the parser into the lexer. There we can use
token lookbehind (same trick) to *insert* semicolons into the token
stream. This works much better for antlr because antlr's prediction
code can work with real tokens.

Also, the lexer is simpler than the parser, so if there is a place
to introduce a hack, that is a better place.
2016-06-17 16:18:41 -04:00
Uwe Schindler a7aedbe0a1 Fix compound assignment with string concats. in Java 9 there is no stringbuilder on stack! This closes #18929 2016-06-17 00:30:09 +02:00
Robert Muir d741e65da1 Merge pull request #18932 from rmuir/painless_debug_exception
improve Debugger to print code even if it hits exception
2016-06-16 17:47:14 -04:00
Robert Muir 2a3184604e improve Debugger to print code even if it hits exception 2016-06-16 17:34:50 -04:00
Ryan Ernst 1600e56801 Merge branch 'master' into plugin_name_api 2016-06-16 13:49:48 -07:00
Ryan Ernst 8196cf01e3 Merge branch 'master' into plugin_name_api 2016-06-16 13:49:28 -07:00
Nik Everett 13d16fbf41 Painless: Disable java-9 indy string thing
It is breaking some of the doc tests. Also add a unit test that
shows the failure.

Relates to #18929
2016-06-16 15:06:01 -04:00
Nik Everett b665d8a187 Painless: Add flag support to regexes
Painless: Add support for //m
Painless: Add support for //s
Painless: Add support for //i
Painless: Add support for //u
Painless: Add support for //U
Painless: Add support for //l
  This means "literal" and is exposed for completeness sake with
  the java api.
Painless: Add support for //c
  c enables Java's CANON_EQ (canonical equivalence) flag which makes
  unicode characters that are canonically equal match. Java's javadoc
  gives "a\u030A" being equal to "\u00E5". That is that the "a" code
  point followed by the "combining ring above" code point is equal to
  the "a with combining ring above" code point.
Update docs and add multi-flag test
Whitelist most of the Pattern class.
2016-06-16 15:00:31 -04:00
Robert Muir 251001e435 add more simple tests 2016-06-16 12:40:30 -04:00
Robert Muir fdd1f152a7 improve lambda syntax (allow single expression) 2016-06-16 11:02:06 -04:00
Robert Muir ccad99fb5c Merge pull request #18911 from rmuir/noncapturing_lambdas
non-capturing lambda support
2016-06-16 10:31:54 -04:00
Nik Everett 8d3ef742db Painless: =~ and ==~ operators
Adds support for the find operator (=~) and the match operator (==~)
to painless's regexes. Also whitelists most of the Matcher class and
documents regex support in painless.

The find operator (=~) returns a boolean that is the result of building
a matcher on the lhs with the Pattern on the RHS and calling `find` on
it. Use it like this:

```
if (ctx._source.last =~ /b/)
```

The match operator (==~) returns boolean like find but instead of calling
`find` on the Matcher it calls `matches`.

```
if (ctx._source.last ==~ /[^aeiou].*[aeiou]/)
```

Finally, if you want the actual matcher you do:

```
Matcher m = /[aeiou]/.matcher(ctx._source.last)
```
2016-06-16 08:42:33 -04:00
Uwe Schindler 4293030fd9 painless: Remove stale comment and remove declared exception 2016-06-16 11:46:11 +02:00
Uwe Schindler f2d6219426 painless: remove useless dropArguments and throws statement in megamorphic cache; add tests 2016-06-16 11:08:10 +02:00
Simon Willnauer 18ff051ad5 Simplify ScriptModule and script registration (#18903)
Registering a script engine or native scripts still uses Guice today
and is much more complicated than needed. This change moves to a pull
based model where script plugins have to implement a dedicated interface
`ScriptPlugin` and defines simple getter returning instances rather than
classes.
2016-06-16 09:35:13 +02:00
Robert Muir ec7215e5f2 get non-capturing lambdas working 2016-06-16 01:25:43 -04:00
Robert Muir 60176afdde clean up a bit more 2016-06-15 20:30:16 -04:00
Ryan Ernst a4503c2aed Plugins: Remove name() and description() from api
In 2.0 we added plugin descriptors which require defining a name and
description for the plugin. However, we still have name() and
description() which must be overriden from the Plugin class. This still
exists for classpath plugins. But classpath plugins are mainly for
tests, and even then, referring to classpath plugins with their class is
a better idea. This change removes name() and description(), replacing
the name for classpath plugins with the full class name.
2016-06-15 17:12:22 -07:00
Robert Muir 54797b7d09 don't let megamorphic cache "capture" arbitrary arguments.
pass the shit we need as bootstrap params
2016-06-15 19:33:59 -04:00
Uwe Schindler ae6ce1c42e Small refactoring and cleanup to separate the polymorphic and megamorphic case 2016-06-15 22:27:30 +02:00
Uwe Schindler 6e83f41e94 Merge branch 'more_def_cleanup' of https://github.com/rmuir/elasticsearch into more_def_cleanup 2016-06-15 21:47:28 +02:00
Uwe Schindler 5b11b15300 fix typo 2016-06-15 21:46:58 +02:00
Robert Muir 786c3b0fa8 speed up compound assignments 2016-06-15 14:38:37 -04:00
Robert Muir a4036b8069 add compound assignment tests 2016-06-15 14:02:19 -04:00
Uwe Schindler 949e99bde3 Merge branch 'more_def_cleanup' of https://github.com/rmuir/elasticsearch into more_def_cleanup 2016-06-15 19:48:53 +02:00
Uwe Schindler 8d3c411298 Remove unneeded type adaption and add error message on violation 2016-06-15 19:48:38 +02:00
Robert Muir e3f34730ae and compound assignment 2016-06-15 13:37:10 -04:00
Robert Muir 27f8b6e6db get compound assignment working 2016-06-15 13:30:21 -04:00
Uwe Schindler ca2e0e1660 Some small cleanups around the lookup 2016-06-15 19:07:13 +02:00
Uwe Schindler 767bb49da8 Fix bug with first param (receiver) on the cache lookup 2016-06-15 19:00:22 +02:00
Uwe Schindler 283410b7ac Merge branch 'more_def_cleanup' of https://github.com/rmuir/elasticsearch into more_def_cleanup 2016-06-15 18:37:07 +02:00
Uwe Schindler 04ab42cdf5 Use rethrow puzzler in the ClassValue 2016-06-15 18:36:45 +02:00
Robert Muir d4db8c9c82 some cleanup 2016-06-15 12:32:38 -04:00
Uwe Schindler 6487940a79 Move type changes up to static part 2016-06-15 18:26:11 +02:00
Uwe Schindler a473298f44 Add megamorphic cache 2016-06-15 18:21:42 +02:00
Robert Muir b71f42a627 split MIC from PIC 2016-06-15 09:07:11 -04:00
Robert Muir 2b1ebc55a9 remove null guards from +, if not needed.
improve DefBootstrapTests
2016-06-14 23:52:21 -04:00
Robert Muir 5e4b2f2815 clean up compound assignment tests, fix shift bug 2016-06-14 20:40:16 -04:00
Robert Muir cd4f4594d6 fix perf bug with xor promotion 2016-06-14 19:19:50 -04:00
Jack Conradson 5cef06fa3e Add a test. 2016-06-14 14:28:47 -07:00
Jack Conradson 36f7632232 Correct type checking during casting related to Object. 2016-06-14 14:12:23 -07:00
Robert Muir b69e8aadbd refactor tests 2016-06-14 15:07:56 -04:00
Robert Muir e4dc469e58 Merge pull request #18849 from rmuir/give_indy_ops_types
Remove casts and boxing for dynamic math
2016-06-14 13:01:04 -04:00
Robert Muir 79f0ab2fc6 rename this variable 2016-06-14 12:51:29 -04:00
Nik Everett 49c09f1527 Test: Fix error message check for Windows
Betrayal! CRLF strikes again. Weakened the test from equals to
containsString.
2016-06-14 09:56:49 -04:00
Robert Muir a12937d548 painless: remove casts and boxing for dynamic math 2016-06-14 00:49:13 -04:00
Robert Muir 7ecb4ca4e9 Refactor def math 2016-06-13 18:56:28 -04:00