Commit Graph

711 Commits

Author SHA1 Message Date
Marios Trivyzas ea6f0e39bc
[Tests] Update skip version for YAML tests (#52310)
Update skip versions upper boundary to match the release
or intented release version of the feature/fix.
2020-02-13 15:36:31 +01:00
Jay Modi 3edadfefd0 RestHandlers declare handled routes (#52123)
This commit changes how RestHandlers are registered with the
RestController so that a RestHandler no longer needs to register itself
with the RestController. Instead the RestHandler interface has new
methods which when called provide information about the routes
(method and path combinations) that are handled by the handler
including any deprecated and/or replaced combinations.

This change also makes the publication of RestHandlers safe since they
no longer publish a reference to themselves within their constructors.

Closes #51622

Co-authored-by: Jason Tedor <jason@tedor.me>

Backport of #51950
2020-02-09 22:48:32 -07:00
Przemko Robakowski 8cf47aca7e
[7.x] Improve Painless compilation performance for nested conditionals (#52056) (#52074)
* Improve Painless compilation performance for nested conditionals (#52056)

This PR changes how conditional expression is handled in `PainlessParser`
in a way that avoids the need for backtracking, which led to exponential
compilation times in case of nested conditionals.

The test was added ensures that we can compile deeply nested conditionals.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* Fix Map.of in Java8

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-02-07 21:13:25 +01:00
Mark Vieira bc7aff917e
Mute LangPainlessClientYamlTestSuiteIT context API tests (#51939) (#52011) 2020-02-06 11:04:15 -08:00
Ryan Ernst 21224caeaf Remove comparison to true for booleans (#51723)
While we use `== false` as a more visible form of boolean negation
(instead of `!`), the true case is implied and the true value does not
need to explicitly checked. This commit converts cases that have slipped
into the code checking for `== true`.
2020-01-31 16:35:43 -08:00
Stuart Tettemer 41c15b438d
Scripting: Add char position of script errors (#51069) (#51266)
Add the character position of a scripting error to error responses.

The contents of the `position` field are experimental and subject to
change.  Currently, `offset` refers to the character location where the
error was encountered, `start` and `end` define a range of characters
that contain the error.

eg.
```
{
  "error": {
    "root_cause": [
      {
        "type": "script_exception",
        "reason": "runtime error",
        "script_stack": [
          "y = x;",
          "     ^---- HERE"
        ],
        "script": "def x = new ArrayList(); Map y = x;",
        "lang": "painless",
        "position": {
          "offset": 33,
          "start": 29,
          "end": 35
        }
      }
```

Refs: #50993
2020-01-21 13:45:59 -07:00
Henning Andersen 125feecabc
Guess root cause support unwrap (#50525) (#50742)
ElasticsearchException.guessRootCauses would return wrapper exception if
inner exception was not an ElasticsearchException. Fixed to never return
wrapper exceptions.

At least following APIs change root_cause.0.type as a result:

_update with bad script
_index with bad pipeline

Relates #50417
2020-01-08 19:09:14 +01:00
Stuart Tettemer f212994c16
[TEST] Unknown scripting annotations raise error (#50343) (#50346)
Ensure that unknown annotations, such as typo'd `@nondeterministic`, 
will raise an exception.
2019-12-19 16:22:22 -07:00
Stuart Tettemer 689df1f28f
Scripting: ScriptFactory not required by compile (#50344) (#50392)
Avoid backwards incompatible changes for 8.x and 7.6 by removing type
restriction on compile and Factory.  Factories may optionally implement
ScriptFactory.  If so, then they can indicate determinism and thus
cacheability.

**Backport**

Relates: #49466
2019-12-19 12:50:25 -07:00
Stuart Tettemer 06a24f09cf
Scripting: Cache script results if deterministic (#50106) (#50329)
Cache results from queries that use scripts if they use only
deterministic API calls.  Nondeterministic API calls are marked in the
whitelist with the `@nondeterministic` annotation.  Examples are
`Math.random()` and `new Date()`.

Refs: #49466
2019-12-18 13:00:42 -07:00
Jack Conradson eb20db8a1c Update Painless AST Catch Node (#50044)
This makes two changes to the catch node:

1. Use SDeclaration to replace independent variable usage.
2. Use a DType to set a "minimum" exception type - this allows us to require 
users to continue using Exception as "minimum" type for catch blocks, but 
for us to internally catch Error/Throwable. This is a required step to 
removing custom try/catch blocks from SClass.
2019-12-10 12:56:34 -08:00
Stuart Tettemer 17cda5b2c0
Scripting: Groundwork for caching script results (#49895) (#49944)
In order to cache script results in the query shard cache, we need to
check if scripts are deterministic.  This change adds a default method
to the script factories, `isResultDeterministic() -> false` which is
used by the `QueryShardContext`.

Script results were never cached and that does not change here.  Future
changes will implement this method based on whether the results of the
scripts are deterministic or not and therefore cacheable.

Refs: #49466

**Backport**
2019-12-06 15:08:05 -07:00
Jack Conradson cd3744c0b7 Add nodes to handle types (#49785)
This PR adds 3 nodes to handle types defined by a front-end creating a 
Painless AST. These types are decided with data immutability in mind - 
hence the reason for more than a single node.
2019-12-05 17:09:19 -08:00
Jack Conradson 687c6648d9 Minor Painless Clean Up (#49844)
This cleans up two minor things.
- Cleans up style of == false
- Pulls maxLoopCounter into a member variable instead of accessing
CompilerSettings multiple times in the SFunction node
2019-12-05 12:20:07 -08:00
Stuart Tettemer 426c7a5e8f
Scripting: add available languages & contexts API (#49652) (#49815)
Adds `GET /_script_language` to support Kibana dynamic scripting
language selection.

Response contains whether `inline` and/or `stored` scripts are
enabled as determined by the `script.allowed_types` settings.

For each scripting language registered, such as `painless`,
`expression`, `mustache` or custom, available contexts for the language
are included as determined by the `script.allowed_contexts` setting.

Response format:
```
{
  "types_allowed": [
    "inline",
    "stored"
  ],
  "language_contexts": [
    {
      "language": "expression",
      "contexts": [
        "aggregation_selector",
        "aggs"
        ...
      ]
    },
    {
      "language": "painless",
      "contexts": [
        "aggregation_selector",
        "aggs",
        "aggs_combine",
        ...
      ]
    }
...
  ]
}
```

Fixes: #49463 

**Backport**
2019-12-04 16:18:22 -07:00
Jack Conradson dbf6183469 Remove extraneous pass (#49797)
This removes the storeSettings pass where nodes in the AST could store 
information they needed out of CompilerSettings for use during later 
passes. CompilerSettings is part of ScriptRoot which is available during the 
analysis pass making the storeSettings pass redundant.
2019-12-04 12:18:04 -08:00
Jack Conradson a780ec14f0 Painless: Upgrade ASM to 7.2 (#49263)
This upgrades Painless to use the latest ASM libraries providing support up 
to Java 14. Note the library is not published with the latest versions in an 
"all" package, so we pick up each lib independently that's required. There 
were some changes to the getType method that require descriptors to be 
used in place of internal class names.
2019-11-20 07:09:47 -08:00
Rory Hunter c46a0e8708
Apply 2-space indent to all gradle scripts (#49071)
Backport of #48849. Update `.editorconfig` to make the Java settings the
default for all files, and then apply a 2-space indent to all `*.gradle`
files. Then reformat all the files.
2019-11-14 11:01:23 +00:00
Jack Conradson 155ecd0a76 Change Painless regex node to use SField instead of Globals (#47944)
* Change Painless regex node to use SField instead of Globals

* Use reflection instead of ASM to specify modifiers

* Remove synthetic from SField
2019-10-15 07:47:16 -07:00
Jack Conradson 076d3073b5 Move binding member field generation to Painless semantic pass (#47739)
This adds an SField node that operates similarly to SFunction as a top level 
node meant only for use in an SClass node. Member fields are generated 
for both class bindings and instance bindings using the new SField node 
during the semantic pass, and information is no longer passed through 
Globals for this during the write pass.
2019-10-09 10:24:53 -07:00
Alpar Torok 36d018c909 Convert RunTask to use testclusers, remove ClusterFormationTasks (#47572)
* Convert RunTask to use testclusers, remove ClusterFormationTasks

This PR adds a new RunTask and a way for it to start a
testclusters cluster out of band and block on it to replace
the old RunTask that used ClusterFormationTasks.

With this we can now remove ClusterFormationTasks.
2019-10-08 14:43:29 +03:00
Jack Conradson 833ed30f0d Modify Painless AST to add synthetic functions during semantic pass (#47611)
This has ELambda and ENewArrayFunctionRef add their generated synthetic
methods to the SClass node during the semantic pass and removes this 
data from the write pass. This is the first step to remove "Globals" (mutable 
state) from the write pass.
2019-10-07 07:48:51 -07:00
Jack Conradson e3aab1295e Add a ScriptRoot to consolidate global data necessary for multiple passes (#47532)
This PR is to get plumbing in for a ScriptRoot class that will consolidate 
several pieces of state required by potentially multiple passes including 
PainlessLookup, CompilerSettings, FunctionTable, the root class node, and a 
synthetic counter. It's possible more may be added to this as we move 
forward and slowly make the the nodes have less mutable state.
2019-10-04 08:37:19 -07:00
Ryan Ernst f32692208e
Add explanations to script score queries (#46693) (#47548)
While function scores using scripts do allow explanations, they are only
creatable with an expert plugin. This commit improves the situation for
the newer script score query by adding the ability to set the
explanation from the script itself.

To set the explanation, a user would check for `explanation != null` to
indicate an explanation is needed, and then call
`explanation.set("some description")`.
2019-10-03 21:05:05 -07:00
Jack Conradson 8f1a80a43d Move Painless local methods to a dedicated FunctionTable (#46889)
This moves the way Painless maintains function headers for use
across compilation into its own class - FunctionTable. This
allows us to store a dedicated object for function lookup at
runtime for the def type instead of a loose Map of functions.
2019-09-30 09:06:40 -07:00
Jack Conradson d09965a6dd Add ClassWriter to Painless writing pass (#47140)
This the first part of a series to allow nodes to write all of their appropriate
pieces to the class. Currently, nodes must add their bindings, constants, and
functions to main SClass node for delayed writing. This instead adds a
Painless version of ClassWriter to the write pass. The Painless ClassWriter
contains an appropriate ClassVisitor that can be accessed in any node
during the process along with access to the clinit method, and finally a
shortcut for creating new MethodWriter. The next step will be removing the
delayed writing in SClass, and instead, delegate all writing responsibilities to
the nodes.
2019-09-27 11:04:15 -07:00
Jack Conradson 9b4f377474 Change Painless function node to use a block instead of raw statements (#46884)
This change improves the node structure of SFunction. SFunction now uses 
an SBlock instead of a List of AStatments reducing code duplication and 
gives a future target for symbol table scoping.
2019-09-26 10:33:35 -07:00
Jack Conradson a1af2fe96a Rename Painless node SSource to SClass (#46984)
Mechanical renaming of SSource node to SClass to better align with the names of what other nodes generate.
2019-09-24 07:35:10 -07:00
Jim Ferenczi 23bf310c84 Replace the SearchContext with QueryShardContext when building aggregator factories (#46527)
This commit replaces the `SearchContext` with the `QueryShardContext` when building aggregator factories. Aggregator factories are part of the `SearchContext` so they shouldn't require a `SearchContext` to create them.
The main changes here are the signatures of `AggregationBuilder#build` that now takes a `QueryShardContext` and `AggregatorFactory#createInternal` that passes the `SearchContext` to build the `Aggregator`.

Relates #46523
2019-09-11 16:43:30 +02:00
Jim Ferenczi 425b1a77e8 Add more context to QueryShardContext (#46584)
This change adds an IndexSearcher and the node's BigArrays in the QueryShardContext.
It's a spin off of #46527 as this change is required to allow aggregation builder to solely use the
query shard context.

Relates #46523
2019-09-11 12:24:51 +02:00
Mayya Sharipova 2c5f9b558b Fix highlighting for script_score query (#46507) 2019-09-10 08:26:47 -04:00
Jack Conradson 45ad01ab1c Fix bugs in Painless SCatch node (#45880)
This fixes two bugs:
- A recently introduced bug where an NPE will be thrown if a catch block is 
empty.
- A long-time bug where an NPE will be thrown if multiple catch blocks in a 
row are empty for the same try block.
2019-08-23 08:08:02 -07:00
Jack Conradson a1b88ca009 Move regex error to node (#45813) 2019-08-22 07:12:54 -07:00
Andrey Ershov dbc90653dc transport.publish_address should contain CNAME (#45626)
This commit adds CNAME reporting for transport.publish_address same way
it's done for http.publish_address.

Relates #32806
Relates #39970

(cherry picked from commit e0a2558a4c3a6b6fbfc6cd17ed34a6f6ef7b15a9)
2019-08-16 17:42:00 +02:00
Luca Cavanna c31cddf27e
Update the schema for the REST API specification (#42346)
* Update the REST API specification

This patch updates the REST API spefication in JSON files to better encode deprecated entities,
to improve specification of URL paths, and to open up the schema for future extensions.

Notably, it changes the `paths` from a list of strings to a list of objects, where each
particular object encodes all the information for this particular path: the `parts` and the `methods`.

Among the benefits of this approach is eg. encoding the difference between using the `PUT` and `POST`
methods in the Index API, to either use a specific document ID, or let Elasticsearch generate one.

Also `documentation` becomes an object that supports an `url` and also a `description` which is a
new field.

* Adapt YAML runner to new REST API specification format

The logic for choosing the path to use when running tests has been
simplified, as a consequence of the path parts being listed under each
path in the spec. The special case for create and index has been removed.

Also the parsing code has been hardened so that errors are thrown earlier
when the structure of the spec differs from what expected, and their
error messages should be more helpful.
2019-08-16 14:40:00 +02:00
Armin Braun de58353722
Lower Painless Static Memory Footprint (#45487) (#45619)
* Painless generates a ton of duplicate strings and empty `Hashmap` instances wrapped as unmodifiable
* This change brings down the static footprint of Painless on an idle node by 20MB (after running the PMC benchmark against said node)
   * Since we were looking into ways of optimizing for smaller node sizes I think this is a worthwhile optimization
2019-08-15 19:41:45 +02:00
Jack Conradson 7f550f2b29 Complete decoupling ANTLR AST from Painless AST (#45366)
This change removes the Reserved class used to track variables usages 
within the ANTLR grammar. That task is now performed by an existing pass 
"extractVariables" in the Painless AST. The Painless AST no longer has any 
dependencies on the ANTLR AST for state outside of the tree being built. 
This will simplify future refactoring and opens the possibility of alternate 
grammars.
2019-08-13 08:02:10 -07:00
Armin Braun a9e1402189
Remove Settings from BaseRestRequest Constructor (#45418) (#45429)
* Resolving the todo, cleaning up the unused `settings` parameter
* Cleaning up some other minor dead code in affected classes
2019-08-12 05:14:45 +02:00
Jack Conradson b716b840d3 Remove loop counter from Reserved in Painless AST. (#45298)
This change adds a compiler pass to give each node the chance to store 
settings necessary for analysis and writing. This removes the need to pass 
this in a somewhat convoluted way through an additional class called 
Reserved, and also removes the need to have the Walker set values for 
settings on reserved. This is next step in decoupling the Painless grammar 
from the Painless AST.
2019-08-08 09:34:51 -07:00
Jason Tedor bd59ee6c72
Fix clock used in update requests (#45262)
We accidentally switched to using the relative time provider here. This
commit fixes this by switching to the appropriate absolute clock.
2019-08-06 21:15:21 -04:00
Jack Conradson fc8a6fc9d0 Decouple Painless AST Lambda generation from the grammar (#45111)
This is the first step in decoupling the Painless AST from the grammar. The
Painless AST should be able to generate classes independently of how the
AST is generated from a grammar. (If I were to build a Painless AST by hand
in code this should be all that's necessary.) This change removes Lambda
name generation from the ANTLR grammar tree walker. It also removes
unnecessary node generation of new array function references from the
tree walker as well.
2019-08-06 10:08:19 -07:00
Zachary Tong 3df1c76f9b Allow pipeline aggs to select specific buckets from multi-bucket aggs (#44179)
This adjusts the `buckets_path` parser so that pipeline aggs can
select specific buckets (via their bucket keys) instead of fetching
the entire set of buckets.  This is useful for bucket_script in
particular, which might want specific buckets for calculations.

It's possible to workaround this with `filter` aggs, but the workaround
is hacky and probably less performant.

- Adjusts documentation
- Adds a barebones AggregatorTestCase for bucket_script
- Tweaks AggTestCase to use getMockScriptService() for reductions and
pipelines.  Previously pipelines could just pass in a script service
for testing, but this didnt work for regular aggs.  The new
getMockScriptService() method fixes that issue, but needs to be used
for pipelines too.  This had a knock-on effect of touching MovFn,
AvgBucket and ScriptedMetric
2019-08-05 12:18:40 -04:00
Jack Conradson 54552edaf6 Whitelist randomUUID in Painless (#45148)
This whitelists randomUUID with the understanding that it's possible for 
/dev/random to cause blocking on *nix systems. Users that need 
randomUUID should switch their random generator source to /dev/urandom 
if this is a concern for them.
2019-08-02 11:53:56 -07:00
Jack Conradson 5202d2624e Add several context examples for Painless date documentation (#44985) 2019-07-31 08:23:17 -07:00
Jack Conradson 1a21682ed0 Fix JodaCompatibleZonedDateTime casts in Painless (#44874)
This is a temporary fix during the Joda to Java datetime transition. This will 
implicitly cast a JodaCompatibleZonedDateTime to a ZonedDateTime for 
both def and static types. This is necessary to insulate users from needing 
to know about JodaCompatibleZonedDateTime explicitly.
2019-07-29 12:05:26 -07:00
Ryan Ernst 03dd22b56c Add missing ZonedDateTime methods for joda compat layer (#44829)
While joda no longer exists in the apis for 7.x, the compatibility layer
still exists with helper methods mimicking the behavior of joda for
ZonedDateTime objects returned for date fields in scripts. This layer
was originally intended to be removed in 7.0, but is now likely to exist
for the lifetime of 7.x.

This commit adds missing methods from ChronoZonedDateTime to the compat
class. These methods were not part of joda, but are needed to act like a
real ZonedDateTime.

relates #44411
2019-07-25 11:45:57 -07:00
Jason Tedor 1e9c505e95
Avoid dumping the heap in Painless tests (#44782)
Well, we have a test here that intentionally causes an OutOfMemoryError,
to ensure that Painless handles it (I still strongly disagree with doing
this). This causes two things to happen: an OutOfMemoryError to be
dumped to the console, and the heap to be dumped to disk. This makes it
look like we had an OutOfMemoryError while running tests, and the tests
did not fail properly. This commit changes the tests configuration so
that we suppress the heap dump, which also causes the OutOfMemoryError
to no longer be dumped to the console.
2019-07-24 16:04:19 +09:00
Ryan Ernst f193d14764
Convert remaining Action Response/Request to writeable.reader (#44528) (#44607)
This commit converts readFrom to ctor with StreamInput on the remaining
ActionResponse and ActionRequest classes.

relates #34389
2019-07-19 13:33:38 -07:00
Tal Levy 075a3f0e99
remove usage of ActionType#(String) (#44459) (#44526)
this commit removes usage of the deprecated
constructor with a single argument and no Writeable.Reader.

The purpose of this is to reduce the boilerplate necessary for
properly implementing a new action, as well as reducing the
chances of using the incorrect super constructor while classes
are being migrated to Writeable

relates #34389.
2019-07-17 20:28:11 -07:00
Ryan Ernst fb77d8f461 Removed writeTo from TransportResponse and ActionResponse (#44092)
The base classes for transport requests and responses currently
implement Streamable and Writeable. The writeTo method on these base
classes is implemented with an empty implementation. Not only does this
complicate subclasses to think they need to call super.writeTo, but it
also can lead to not implementing writeTo when it should have been
implemented, or extendiong one of these classes when not necessary,
since there is nothing to actually implement.

This commit removes the empty writeTo from these base classes, and fixes
subclasses to not call super and in some cases implement an empty
writeTo themselves.

relates #34389
2019-07-10 12:42:04 -07:00
Jack Conradson 81f60652d8 Add timezone documentation for Painless datetimes (#43768) 2019-07-01 21:31:13 -07:00
Ryan Ernst 3a2c698ce0
Rename Action to ActionType (#43778)
Action is a class that encapsulates meta information about an action
that allows it to be called remotely, specifically the action name and
response type. With recent refactoring, the action class can now be
constructed as a static constant, instead of needing to create a
subclass. This makes the old pattern of creating a singleton INSTANCE
both misnamed and lacking a common placement.

This commit renames Action to ActionType, thus allowing the old INSTANCE
naming pattern to be TYPE on the transport action itself. ActionType
also conveys that this class is also not the action itself, although
this change does not rename any concrete classes as those will be
removed organically as they are converted to TYPE constants.

relates #34389
2019-06-30 22:00:17 -07:00
Ryan Ernst 28ab77a023
Add StreamableResponseAction to aid in deprecation of Streamable (#43770)
The Action base class currently works for both Streamable and Writeable
response types. This commit intorduces StreamableResponseAction, for
which only the legacy Action implementions which provide newResponse()
will extend. This eliminates the need for overriding newResponse() with
an UnsupportedOperationException.

relates #34389
2019-06-28 21:40:00 -07:00
Stuart Tettemer 500205e8c5
Add painless method getByPath, get value from nested collections with dotted path (#43170) (#43606)
Given a nested structure composed of Lists and Maps, getByPath will return the value
keyed by path.  getByPath is a method on Lists and Maps.

The path is string Map keys and integer List indices separated by dot. An optional third
argument returns a default value if the path lookup fails due to a missing value.

Eg.
['key0': ['a', 'b'], 'key1': ['c', 'd']].getByPath('key1') = ['c', 'd']
['key0': ['a', 'b'], 'key1': ['c', 'd']].getByPath('key1.0') = 'c'
['key0': ['a', 'b'], 'key1': ['c', 'd']].getByPath('key2', 'x') = 'x'
[['key0': 'value0'], ['key1': 'value1']].getByPath('1.key1') = 'value1'

Throws IllegalArgumentException if an item cannot be found and a default is not given.
Throws NumberFormatException if a path element operating on a List is not an integer.

Fixes #42769
2019-06-26 09:06:34 -06:00
Jack Conradson 5eb044e635 Add annotations to Painless whitelist (#43239)
This change adds the ability to attach annotative information for
classes, methods, fields, static methods, class bindings, and
instance bindings during Painless whitelisting.

Annotations are specified as @annotation or optionally as
@annotation[parameter="argument",...].

Annotations open up the ability to specify whitelist objects as
having a short name (no_import -> @no_import) or deprecated.
2019-06-24 09:27:24 -07:00
Zachary Tong eaa9ee1f16 Set document on script when using Bytes.WithScript (#43390)
Long and Double ValuesSource set the current document on the script
before executing, but Bytes was missing this method call.  That meant
it was possible to generate an OutOfBoundsException when using
a "value" script (field + script) on keyword or other bytes
fields.

This adds in the method call, and a few yaml tests to verify correct
behavior.
2019-06-24 12:20:28 -04:00
Jack Conradson 7ef62a6cbc Remove old Painless doc generator (#43404)
This removes the previous Painless API Doc Generator prior to contexts 
existing. It has been replaced with the new doc generator that uses the 
documentation rest API.
2019-06-20 08:48:52 -07:00
Martijn van Groningen a4c45b5d70
Replace Streamable w/ Writeable in SingleShardRequest and subclasses (#43222) (#43364)
Backport of: https://github.com/elastic/elasticsearch/pull/43222

This commit replaces usages of Streamable with Writeable for the
SingleShardRequest / TransportSingleShardAction classes and subclasses of
these classes.

Note that where possible response fields were made final and default
constructors were removed.

Relates to #34389
2019-06-19 16:15:09 +02:00
Mayya Sharipova aa6248d4d7
Move dense_vector and sparse_vector to module (#43280) (#43333) 2019-06-18 11:56:04 -04:00
Martijn Laarman 8b1b9f8ab9
Introduce stability description to the REST API specification (#38413) (#43278)
* introduce state to the REST API specification

* change state over to stability

* CCR is no GA updated to stable

* SQL is now GA so marked as stable

* Introduce `internal` as state for API's, marks stable in terms of lifetime but unstable in terms of guarantees on its output format since it exposes internal representations

* make setting a wrong stability value, or not setting it at all an error that causes the YAML test suite to fail

* update spec files to be explicit about their stability state

* Document the fact that stability needs to be defined

Otherwise the YAML test runner will fail (with a nice exception message)

* address check style violations

* update rest spec unit tests to include stability

* found one more test spec file not declaring stability, made sure stability appears after documentation everywhere

* cluster.state is stable, mark response in some way to denote its a key value format that can be changed during minors

* mark data frame API's as beta

* remove internal and private as states for an API

* removed the wrong enum values in the Stability Enum in the previous commit

(cherry picked from commit 61c34bbd92f8f7e5f22fa411c6b682b0ebd8a99d)
2019-06-17 16:57:13 +02:00
Ryan Ernst 5be0fb32f8 Move painless context api spec to test local (#43122)
The painless context api is internal and currently meant only for use in
generating docs. This commit moves the spec file for the api so that it
is only used by the test for this api, and not externally by any clients
building from the public rest spec.
2019-06-12 08:19:45 -07:00
Jack Conradson f7d4ca7a14 Exclude internal SQL methods from Painless API Docs (#43116) 2019-06-11 15:42:54 -07:00
Mark Vieira e44b8b1e2e
[Backport] Remove dependency substitutions 7.x (#42866)
* Remove unnecessary usage of Gradle dependency substitution rules (#42773)

(cherry picked from commit 12d583dbf6f7d44f00aa365e34fc7e937c3c61f7)
2019-06-04 13:50:23 -07:00
Jack Conradson de72fe344c Add Basic Date Docs to Painless (#42544) 2019-06-03 13:39:03 -07:00
Zachary Tong 6ae6f57d39
[7.x Backport] Force selection of calendar or fixed intervals (#41906)
The date_histogram accepts an interval which can be either a calendar
interval (DST-aware, leap seconds, arbitrary length of months, etc) or
fixed interval (strict multiples of SI units). Unfortunately this is inferred
by first trying to parse as a calendar interval, then falling back to fixed
if that fails.

This leads to confusing arrangement where `1d` == calendar, but
`2d` == fixed.  And if you want a day of fixed time, you have to
specify `24h` (e.g. the next smallest unit).  This arrangement is very
error-prone for users.

This PR adds `calendar_interval` and `fixed_interval` parameters to any
code that uses intervals (date_histogram, rollup, composite, datafeed, etc).
Calendar only accepts calendar intervals, fixed accepts any combination of
units (meaning `1d` can be used to specify `24h` in fixed time), and both
are mutually exclusive.

The old interval behavior is deprecated and will throw a deprecation warning.
It is also mutually exclusive with the two new parameters. In the future the
old dual-purpose interval will be removed.

The change applies to both REST and java clients.
2019-05-20 12:07:29 -04:00
Tal Levy 5640197632
Refactor TransportSingleShardAction to serialize Writeable responses (#41985) (#42040)
Previously, TransportSingleShardAction required constructing a new
empty response object. This response object's Streamable readFrom
was used. As part of the migration to Writeable, the interface here
was updated to leverage Writeable.Reader.

relates to #34389.
2019-05-09 22:08:31 -07:00
Christian Mesh 99a50ac3b7 Add painless string split function (splitOnToken) (#39772)
Adds two String split functions to Painless that can be used without enabling regexes.
2019-05-09 15:16:11 -07:00
Jack Conradson 2c561481cd Add static section whitelist info to api docs generation (#41870)
This change adds imported methods, class bindings, and instance bindings to the documentation generation for the Painless Context APIs.
2019-05-08 11:15:38 -07:00
Jack Conradson a74ba7d5ba Task to generate Painless API's per context (#41233)
This adds a gradle task called generateContextDoc in the Painless module. The 
task will start a cluster, issue commands against the context rest api for 
Painless, and generate documentation for each API per context. Each context 
has a first page of classes sorted by package first and class name second, 
along with a page per package with each classes' constructors, methods, and 
fields. A link is generated for each constructor, method, and field to a JavaDoc 
page when possible.
2019-04-23 11:28:27 -07:00
Christoph Büscher 2980a6c70f Clarify some ToXContent implementations behaviour (#41000)
This change adds either ToXContentObject or ToXContentFragment to classes
directly implementing ToXContent currently. This helps in reasoning about
whether those implementations output full xcontent object or just fragments.

Relates to #16347
2019-04-15 09:42:08 +02:00
Mark Vieira 1287c7d91f
[Backport] Replace usages RandomizedTestingTask with built-in Gradle Test (#40978) (#40993)
* Replace usages RandomizedTestingTask with built-in Gradle Test (#40978)

This commit replaces the existing RandomizedTestingTask and supporting code with Gradle's built-in JUnit support via the Test task type. Additionally, the previous workaround to disable all tasks named "test" and create new unit testing tasks named "unitTest" has been removed such that the "test" task now runs unit tests as per the normal Gradle Java plugin conventions.

(cherry picked from commit 323f312bbc829a63056a79ebe45adced5099f6e6)

* Fix forking JVM runner

* Don't bump shadow plugin version
2019-04-09 11:52:50 -07:00
Jack Conradson fd51780de2 Move test classes to test root in Painless (#40873)
This moves several test classes that were part of the main root to the test root. 
These were part of the main root due to limitations prior to whitelist 
customization. Without whitelist customization these can be moved to a test 
context and removed from the base whitelists as they should not be user 
facing.
2019-04-05 15:16:04 -07:00
Jack Conradson a042b22d24 Move PainlessExecuteAction into action package (#40752) 2019-04-04 08:17:45 -07:00
Alpar Torok 25944c4317 convert modules to use testclusters (#40804)
* convert modules to use testclusters
* Eliminate PluginPropertiesTask and move logic in plugin where it belongs
2019-04-04 11:45:40 +03:00
Mayya Sharipova 24755209b4 Add randomScore function in script_score query (#40186)
To make script_score query to have the same features
as function_score query, we need to add randomScore
function.

This function produces different
random scores on different index shards.
It is also able to produce random scores
based on the internal Lucene Document Ids.
2019-03-28 13:23:47 -04:00
Jack Conradson 0be7780cb0 Add implicit this for class binding in Painless (#40285)
This change allows class bindings to add as their first argument, the base script 
class. The this reference to the base script class will be implicitly passed into a 
class binding as the first constructor argument upon initialization when 
specified as the first argument in whitelist entry for the class binding. This 
allows a class binding access to additional information added to the base script 
class such as more information about the current document or current shard. 
One extra requirement for this to work is the appropriate script base class 
must be whitelisted (should be empty).
2019-03-22 12:55:47 -07:00
Jack Conradson 6ea3272f41 Add double and Double standard casts tests to Painless (#40324) 2019-03-21 16:10:28 -07:00
Jack Conradson 5ec56d7d22 Add float and Float standard casting tests to Painless. (#40221) 2019-03-20 08:55:18 -07:00
Jack Conradson dcaabdfce8 Add Painless cast tests for long and Long (#40007) 2019-03-15 09:37:26 -07:00
Jack Conradson b57af6c401 Add a Painless Context REST API (#39382)
This PR adds an internal REST API for querying context information about 
Painless whitelists.

Commands include the following:
GET /_scripts/painless/_context -- retrieves a list of contexts
GET /_scripts/painless/_context?context=%name% retrieves all available 
information about the API for this specific context
2019-03-14 12:42:12 -07:00
Jack Conradson aeb0116355 Add Painless cast tests for int and Integer (#39813) 2019-03-12 12:03:36 -07:00
Jack Conradson ca78e44006 Fix Painless def [char] to String casts (#39759)
* Start to fix def char casts.

* Fix def char to String casts
2019-03-11 10:47:35 -07:00
Jack Conradson 31e6f6cf48 Add char tests and fix String to char cast (#39725)
This fixes a bug where a String to char cast in Painless could be done implicitly. 
It is now required that a String to char cast is explicit as documented in the 
existing specification. This also adds char and Character casting tests.
2019-03-11 10:43:50 -07:00
Martijn van Groningen b78a8a3e80
Use RestToXContentListener in painless execute action rest action. (#39638) 2019-03-05 08:55:32 +01:00
Jack Conradson 7b8ff2d7c5 Add tests for Painless casting from short and Short (#39587)
This adds tests for casting from short and Short to other standard types in 
Painless. This also corrects a few errors from byte and Byte cast tests.
2019-03-04 10:09:29 -08:00
Jack Conradson 687a66b580 Add byte and Byte to Painless standard cast tests (#39415) 2019-03-01 08:35:20 -08:00
Julie Tibshirani 3ce7d2c9b6
Make sure to reject mappings with type _doc when include_type_name is false. (#38270)
`CreateIndexRequest#source(Map<String, Object>, ... )`, which is used when
deserializing index creation requests, accidentally accepts mappings that are
nested twice under the type key (as described in the bug report #38266).

This in turn causes us to be too lenient in parsing typeless mappings. In
particular, we accept the following index creation request, even though it
should not contain the type key `_doc`:

```
PUT index?include_type_name=false
{
  "mappings": {
    "_doc": {
      "properties": { ... }
    }
  }
}
```

There is a similar issue for both 'put templates' and 'put mappings' requests
as well.

This PR makes the minimal changes to detect and reject these typed mappings in
requests. It does not address #38266 generally, or attempt a larger refactor
around types in these server-side requests, as I think this should be done at a
later time.
2019-02-05 10:52:32 -08:00
Jack Conradson 630889baec
Remove extraneous test from Painless lambda tests (#38111)
This test has been awaiting a fix that isn't currently relevant because incoming
lambda parameters are read-only. If this ever changes a new set of tests can
be added that are up-to-date.
2019-02-01 15:10:59 -08:00
Julie Tibshirani c2e9d13ebd
Default include_type_name to false in the yml test harness. (#38058)
This PR removes the temporary change we made to the yml test harness in #37285
to automatically set `include_type_name` to `true` in index creation requests
if it's not already specified. This is possible now that the vast majority of
index creation requests were updated to be typeless in #37611. A few additional
tests also needed updating here.

Additionally, this PR updates the test harness to set `include_type_name` to
`false` in index creation requests when communicating with 6.x nodes. This
mirrors the logic added in #37611 to allow for typeless document write requests
in test set-up code. With this update in place, we can remove many references
to `include_type_name: false` from the yml tests.
2019-02-01 11:44:13 -08:00
Jack Conradson e066a59c89
Fix Painless void return bug (#38046)
Painless now allows void functions and contexts to with a void return type to use
a return statement without a following expression.
2019-01-31 08:32:38 -08:00
Jack Conradson 36ee78d924
Add test coverage for Painless general casting of boolean and Boolean (#37780)
This adds test coverage for general casts in Painless between boolean and other types and Boolean and other types.
2019-01-30 11:01:45 -08:00
Colin Goodheart-Smithe 21e392e95e
Removes typed calls from YAML REST tests (#37611)
This PR attempts to remove all typed calls from our YAML REST tests. The PR adds include_type_name: false to create index requests that use a mapping and also to put mapping requests. It also removes _type from index requests where they haven't already been removed. The PR ignores tests named *_with_types.yml since this are specifically testing typed API behaviour.

The change also includes changing the test harness to add the type _doc to index, update, get and bulk requests that do not specify the document type when the test is running against a mixed 7.x/6.x cluster.
2019-01-30 16:32:58 +00:00
Mayya Sharipova fdb66039d4
Change `rational` to `saturation` in script_score (#37766)
This change of the function name is necessary for conformity
with feature queries.

Closes #37714
2019-01-23 14:28:20 -05:00
Michael Basnight d625b79df2
Add getZone to JodaCompatibleZonedDateTime (#37084)
The ZonedDateTime#getZone() was not accessible via the Joda shim. This
commit adds getZone() and exposes it through painless.
2019-01-09 22:09:34 -06:00
Nhat Nguyen 7580d9d925
Make SourceToParse immutable (#36971)
Today the routing of a SourceToParse is assigned in a separate step
after the object is created. We can easily forget to set the routing.
With this commit, the routing must be provided in the constructor of
SourceToParse.

Relates #36921
2018-12-24 14:06:50 -05:00
Jack Conradson c13a7bc04a
[Painless] Add String Casting Tests (#36945)
This adds additional standard casting tests for String as the original type. This also cleans up the error messages in the String to char cast method.
2018-12-21 13:42:07 -08:00
Jack Conradson be573ab5e7
[Painless] Casting Tests for Object and Number (#36804)
This adds more casting tests with the original type as Object and then Number. Covers the entire set of possible numeric cases for these two types.
2018-12-20 09:42:33 -08:00
Alan Woodward 344917efab
Add script filter to intervals (#36776)
This commit adds the ability to filter out intervals based on their start and end position, and internal
gaps:
```
POST _search
{
  "query": {
    "intervals" : {
      "my_text" : {
        "match" : {
          "query" : "hot porridge",
          "filter" : {
            "script" : {
              "source" : "interval.start > 10 && interval.end < 20 && interval.gaps == 0"
            }
          }
        }
      }
    }
  }
}
```
2018-12-19 11:12:18 +00:00
Alpar Torok e9ef5bdce8
Converting randomized testing to create a separate unitTest task instead of replacing the builtin test task (#36311)
- Create a separate unitTest task instead of Gradle's built in 
- convert all configuration to use the new task 
- the  built in task is now disabled
2018-12-19 08:25:20 +02:00
Jack Conradson 7de85f55e3
[Painless] Add tests for boxed return types (#36747)
Adds tests for each primitive/boxed and def type to be implicitly cast to an 
appropriate boxed return type from a method.
2018-12-18 10:14:48 -08:00
Jack Conradson a0e7e571e4
[Painless] Add boxed type to boxed type casts for method/return (#36571)
This adds implicit boxed type to boxed types casts for non-def types to create asymmetric casting relative to the def type when calling methods or returning values. This means that a user calling a method taking an Integer can call it with a Byte, Short, etc. legally which matches the way def works. This creates consistency in the casting model that did not previously exist.
2018-12-17 10:50:19 -08:00
Michael Basnight dae422fb2b
Update joda compat methods to use compat class (#36654)
The existing joda compat methods isEquals isAfter and isBefore all took
in a ZonedDateTime, but since all of the scripting is now using the new
JodaCompatZonedDateTime, these are changed to take that in instead.
2018-12-14 15:38:51 -06:00
Jeff Hajewski f1f3b28f5c Delete deprecated getValues from ScriptDocValues (#36183)
* Adds deprecation logging to ScriptDocValues#getValues.

First commit addressing issue #22919.

`ScriptDocValues#getValues` was added for backwards compatibility but no
longer needed. Scripts using the syntax `doc['foo'].values` when
`doc['foo']` is a list should be using `doc['foo']` instead.

* Fixes two build errors in #34279

* Removes unused import in ScriptDocValuesDatesTest
* Removes used of `.values` in example in diversified-sampler-aggregation.asciidoc

* Removes use of .values from painless test.

Part of #34279

* Updates tests to use `doc[foo]` syntax rather than `doc[foo].values`.

* Removes use of `getValues()` and replaces use of `doc[foo].values` with `doc[foo]`.

* Indentation fix.

* Remove unnecessary list construction at previous `getValues()` callsite in ScriptDocValues.GeoPoints.

* Update migration doc and add link to `getValue` in ScriptDocValues javadoc.

* Fix compile

* Fix javadoc issue

* Removes ScriptDocValues#getValues usage from painless whitelist.
2018-12-14 07:56:47 -05:00
Jack Conradson 8e988f6c06
[Painless] Add def to boxed type casts (#36506)
This adds casts for the def type to all standard boxed types. Prior to this certain 
casts such as def [long/Long] -> Double would fail which does not follow the 
goals of the Painless casting model to remove the need for explicit boxing. This 
also creates symmetry with the casts for the newly created bridge methods 
being called at run-time.
2018-12-11 14:06:38 -08:00
Jack Conradson 13b1f19772
[Painless] Add extensive tests for def to primitive casts (#36455)
This adds tests for each possible cast of def to a primitive type both implicit 
and explicit. This also fixes a minor bug where we were only checking the type 
of a def to be Number in some explicit casts. This does not work because it 
allows possible unintended casts from BigInteger and BigDecimal to primitive 
types since they both extend Number but are not included as part of the 
Painless casting model.
2018-12-11 08:03:08 -08:00
Ryan Ernst a0da390df2
Scripting: Switch watcher to use joda bwc time objects (#35966)
This commit converts the watcher execution context to use the joda
compat java time objects. It also again removes the joda methods from
the painless whitelist.
2018-12-10 17:29:25 -08:00
Jack Conradson 2df4bd1f81
[Painless] Generate Bridge Methods (#36097)
We use MethodHandles.asType to cast argument types into the appropriate parameter types for 
method calls when the target of the call is a def type at runtime. Currently, certain implicit casts 
using the def type are asymmetric. It is possible to cast Integer -> float as an argument to parameter, but not from int -> Float (boxed to primitive with upcasting is okay, but primitive to 
boxed with upcasting is not).

This PR introduces a solution to the issue by generating bridge methods for all whitelisted methods 
that have at least a single boxed type as an argument. The bridge method will conduct appropriate 
casts and then call the original method. This adds a bit of overhead for correctness. It should not be
used often as Painless avoids boxed types as much as possible.

Note that a large portion of this change is adding methods to do the appropriate def to boxed type 
casts and a few mechanical changes as well. The most important method for review is 
generateBridgeMethod in PainlessLookupBuilder.
2018-12-07 09:32:27 -08:00
Jim Ferenczi 18866c4c0b
Make hits.total an object in the search response (#35849)
This commit changes the format of the `hits.total` in the search response to be an object with
a `value` and a `relation`. The `value` indicates the number of hits that match the query and the
`relation` indicates whether the number is accurate (in which case the relation is equals to `eq`)
or a lower bound of the total (in which case it is equals to `gte`).
This change also adds a parameter called `rest_total_hits_as_int` that can be used in the
search APIs to opt out from this change (retrieve the total hits as a number in the rest response).
Note that currently all search responses are accurate (`track_total_hits: true`) or they don't contain
`hits.total` (`track_total_hits: true`). We'll add a way to get a lower bound of the total hits in a
follow up (to allow numbers to be passed to `track_total_hits`).

Relates #33028
2018-12-05 19:49:06 +01:00
Alan Woodward 73ceaad03a
Update to lucene-8.0.0-snapshot-c78429a554 (#36212)
Includes:

* A fix for a bug in Intervals.or() (https://issues.apache.org/jira/browse/LUCENE-8586)
* The ability to disable offset mangling in WordDelimiterGraphFilter
        (https://issues.apache.org/jira/browse/LUCENE-8509)
* BM25Similarity no longer multiplies scores by k1 + 1
2018-12-05 12:43:56 +00:00
Martijn van Groningen 364badd12e
Fix serialization bug in painless execute api request (#36075)
The `xContentType` was incorrectly serialized:
`xContentType.mediaTypeWithoutParameters()` was used to serialize, but
`xContentType.mediaType()` was used to de-serialize.

Also the serialization test class did not test `ContextSetup` well,
this was due to limitation in serialization test base class. Changed the
test class to manually test xcontent serialization, so that for both binary
and xcontent serialization tests, the `ContextSetup` is properly tested.

Closes #36050
2018-11-30 07:41:44 +01:00
Ryan Ernst 383713d4e9
Scripting: Actually add joda time back to whitelist (#35965)
This commit actually loads the joda whitelist, which was missed in
2018-11-27 15:18:00 -08:00
Jeff Hajewski 49087f16f5 Adds deprecation logging to ScriptDocValues#getValues. (#34279)
`ScriptDocValues#getValues` was added for backwards compatibility but no
longer needed. Scripts using the syntax `doc['foo'].values` when
`doc['foo']` is a list should be using `doc['foo']` instead.

Closes #22919
2018-11-27 14:30:13 -05:00
Ryan Ernst 72d7f236a5
Scripting: Add back joda to whitelist (#35915)
Watcher still exposes some dates as joda DateTime objects. This commit
adds back joda to the painless whitelist so they can still be accessed.

closes #35913
2018-11-26 16:33:05 -08:00
Sachin Araballi 5f656e4eab Fields are declared at the top of the class. (#35891)
Fields should be declared at the top of the class, before any method declarations, constructors, initializers or inner classes.
2018-11-26 22:05:28 +01:00
Jim Ferenczi 3ed3aab127
Fix rest test when numShards is greater than 1 (#35909) 2018-11-26 16:45:56 +01:00
Mayya Sharipova b6014d971c
Forbid negative scores in functon_score query (#35709)
* Forbid negative scores in functon_score query

- Throw an exception when scores are negative in field_value_factor
function
- Throw an exception when scores are negative in script_score
function

Relates to #33309
2018-11-22 06:08:48 -05:00
Mayya Sharipova 643bb20137
Add a new query type - ScriptScoreQuery (#34533)
* Add a new query type - ScriptScoreQuery

script_score query uses script to calculate document scores.
Added as a substitute for function_score with an intentation
to deprecate function_scoreq query.

```http
GET /_search
{
    "query": {
        "script_score" : {
            "query": {
                "match": { "message": "elasticsearch" }
            },
            "script" : {
              "source": "Math.log(2 + doc['likes'].value)"
            },
            "min_score" : 2
        }
    }
}
```

Add several functions to painless to be used inside script_score:

double rational(double, double)
double sigmoid(double, double, double)
double randomNotReproducible() 
double randomReproducible(String, int) 

double decayGeoLinear(String, String, String, double, GeoPoint)
double decayGeoExp(String, String, String, double, GeoPoint)
double decayGeoGauss(String, String, String, double, GeoPoint)

double decayNumericLinear(String, String, String, double, double)
double decayNumericExp(String, String, String, double, double)
double decayNumericGauss(String, String, String, double, double)

double decayDateLinear(String, String, String, double, JodaCompatibleZonedDateTime)
double decayDateExp(String, String, String, double, JodaCompatibleZonedDateTime)
double decayDateGauss(String, String, String, double, JodaCompatibleZonedDateTime)

Date functions only works on dates in  the default format and default time zone
2018-11-20 16:10:06 -05:00
Mayya Sharipova a8833b097b
Painless Context Doc: Add min should match example (#35423) 2018-11-15 15:10:49 -05:00
Mayya Sharipova 501c03e529
Painless Context Doc: Add filter context example (#35305) 2018-11-15 14:33:34 -05:00
Mayya Sharipova 7e144b318a
Painless Context Doc: Add field context example (#35130)
* Painless Context Doc: Add field context example

relates to #34829
2018-11-08 14:54:20 -05:00
Jack Conradson efe9beca7b
[Painless] Removes extraneous compile method (#35323)
With the removal of SearchScript (#34730), an extraneous compile method 
was left behind in PainlessScriptEngine. This change removes the method and 
updates the tests that depend on it to use the main compile method which 
gives better test coverage.
2018-11-07 07:52:12 -08:00
Alexander Reelsen 409050e8de
Refactor: Remove settings from transport action CTOR (#35208)
As settings are not used in the transport action constructor, this
removes the passing of the settings in all the transport actions.
2018-11-05 13:08:18 +01:00
Jack Conradson 44f08717ba
[Scripting] Make Max Script Length Setting Dynamic (#35184)
This changes the current script.max_size_in_bytes to be dynamic so it can be 
set through the cluster settings API. This setting is also applied to inline scripts 
in the compile method of ScriptService to prevent excessively long inline 
scripts from being compiled. The script length limit is removed from Painless as 
this is no longer necessary with the protection in compile.
2018-11-02 16:07:54 -07:00
Nik Everett e28509fbfe
Core: Less settings to AbstractComponent (#35140)
Stop passing `Settings` to `AbstractComponent`'s ctor. This allows us to
stop passing around `Settings` in a *ton* of places. While this change
touches many files, it touches them all in fairly small, mechanical
ways, doing a few things per file:
1. Drop the `super(settings);` line on everything that extends
`AbstractComponent`.
2. Drop the `settings` argument to the ctor if it is no longer used.
3. If the file doesn't use `logger` then drop `extends
AbstractComponent` from it.
4. Clean up all compilation failure caused by the `settings` removal
and drop any now unused `settings` isntances and method arguments.

I've intentionally *not* removed the `settings` argument from a few
files:
1. TransportAction
2. AbstractLifecycleComponent
3. BaseRestHandler

These files don't *need* `settings` either, but this change is large
enough as is.

Relates to #34488
2018-10-31 21:23:20 -04:00
Alpar Torok 59536966c2
Add a new "contains" feature (#34738)
The contains syntax was added in #30874 but the skips were not properly
put in place.
The java runner has the feature so the tests will run as part of the
build, but language clients will be able to support it at their own
pace.
2018-10-25 08:50:50 +03:00
Ryan Ernst 687dc1eb11
Scripting: Remove SearchScript (#34730)
This commit removes the last non context based script class.
2018-10-24 15:03:38 -07:00
Jack Conradson 1b085252c3
[Painless] Add instance bindings (#34410)
This change adds instance bindings to Painless. This binding allows a whitelisted
 method to be called on an instance instantiated prior to script compilation. 
Whitelisting must be done in code as there is no practical way to instantiate a 
useful instance from a text file (see the tests for an example). Since an 
instance can be shared by multiple scripts, each method called must be 
thread-safe.
2018-10-24 12:57:28 -07:00
Luca Cavanna d51bc05dce
[TEST] Improve validation of do sections (#34734)
We throw parsing exception when an unknown array is found, but we don't when an unknown top-level field is found. This commit makes sure that unsupported top-level fields are not ignored in a do section.

Closes #34651
2018-10-24 21:27:07 +02:00
Ryan Ernst 8da1c9626a
Scripting: Add back params._source access in scripted metric aggs (#34777)
Access to special variables _source and _fields were accidentally
removed in recent refactorings. This commit adds them back, along with a
test.

closes #33884
2018-10-23 18:07:53 -07:00
Zachary Tong 299d044bfc
Collapse pipeline aggs into single package (#34658)
- Restrict visibility of Aggregators and Factories
- Move PipelineAggregatorBuilders up a level so it is consistent with
AggregatorBuilders
- Checkstyle line length fixes for a few classes
- Minor odds/ends (swapping to method references, formatting, etc)
2018-10-23 16:01:01 -04:00
Ryan Ernst 222652dfce
Scripting: Convert script fields to use script context (#34164)
This commit removes the use of SearchScript for script fields and adds
a new FieldScript.
2018-10-20 16:33:49 -07:00
Armin Braun 3954d041a0
SCRIPTING: Move sort Context to its Own Class (#33717)
* SCRIPTING: Move sort Context to its own Class
2018-10-17 10:02:44 +01:00
Armin Braun ebca27371c
SCRIPTING: Move Aggregation Script Context to its own class (#33820)
* SCRIPTING: Move Aggregation Script Context to its own class
2018-10-15 17:28:05 +01:00
Jack Conradson a7c4dbbb0c
[Painless] Add a Map for java names to classes for use in the custom classloader (#34424)
This fixes a bug that wasn't including the class used for a static import 
where only the static import is whitelisted.
2018-10-12 15:34:48 -07:00
Jack Conradson 4270085360
[Painless] Allow statically imported methods without whitelist class. (#34370)
This removes the extraneous check to see if the class for a statically imported 
method is already whitelisted, so a statically imported method can be whitelisted
independently.
2018-10-10 10:00:45 -07:00
Nik Everett 06993e0c35
Logging: Make ESLoggerFactory package private (#34199)
Since all calls to `ESLoggerFactory` outside of the logging package were
deprecated, it seemed like it'd simplify things to migrate all of the
deprecated calls and declare `ESLoggerFactory` to be package private.
This does that.
2018-10-06 09:54:08 -04:00
Kazuhiro Sera d45fe43a68 Fix a variety of typos and misspelled words (#32792) 2018-10-03 18:11:38 +01:00
Ryan Ernst 47cbae9b26
Scripting: Remove ExecutableScript (#34154)
This commit removes the legacy ExecutableScript, which was no longer
used except in tests. All uses have previously been converted to script
contexts.
2018-09-28 17:13:08 -07:00
Jack Conradson bc84c8ecd6
Painless: Remove caching of Painless scripts in GenericElasticsearchScript (#34116)
When a script uses the special-cased SearchScript or ExecutableScript, Painless is 
internally caching the GenericElasticsearchScript that is generated via compile as part of 
the newInstance method for the factories. This breaks the model for class bindings as the 
class binding expects to not have any stored information for each instance generated via 
newInstance from the factory. This change fixes the issue by creating a new instance of 
the GenericElasticsearchScript each time newInstance is called against the factory.
2018-09-28 09:11:41 -07:00
Ryan Ernst 85e4ef3429
Scripting: Reflect factory signatures in painless classloader (#34088)
It is sometimes desirable to pass a class into a script constructor that
will not actually be exposed in the script whitelist. This commit uses
reflection when creating the compiler to find all the classes of the
factory method signature, and make the classloader that wraps lookup
also expose these classes.
2018-09-26 13:22:48 -07:00
Jack Conradson 4fbe84edf6
Painless: Cleanup Cache (#33963)
* Adds equals/hashcode methods to the Painless* objects within the lookup package.
* Changes the caches to use the Painless* objects as keys as well as values. This forces 
future changes to taken into account the appropriate values for caching.
* Deletes the existing caching objects in favor of Painless* objects. This removes a pair of 
bugs that were not taking into account subtle corner cases related to augmented methods 
and caching.
* Uses the Painless* objects to check for equivalency in existing Painless* objects that 
may have already been added to a PainlessClass. This removes a bug related to return 
not being appropriately checked when adding methods.
* Cleans up several error messages.
2018-09-26 09:33:08 -07:00
Christoph Büscher ba3ceeaccf
Clean up "unused variable" warnings (#31876)
This change cleans up "unused variable" warnings. There are several cases were we 
most likely want to suppress the warnings (especially in the client documentation test
where the snippets contain many unused variables). In a lot of cases the unused
variables can just be deleted though.
2018-09-26 14:09:32 +02:00
Christoph Büscher b654d986d7
Add OneStatementPerLineCheck to Checkstyle rules (#33682)
This change adds the OneStatementPerLineCheck to our checkstyle precommit
checks. This rule restricts the number of statements per line to one. The
resoning behind this is that it is very difficult to read multiple statements on
one line. People seem to mostly use it in short lambdas and switch statements in
our code base, but just going through the changes already uncovered some actual
problems in randomization in test code, so I think its worth it.
2018-09-21 11:52:31 +02:00
Jack Conradson 3df285d9f0
Painless: Rename *Binding classes to *ClassBinding. (#33865)
With the upcoming instance bindings, the singular *Binding name isn't descriptive enough 
with multiple binding types. This renames the existing *Binding classes to *ClassBinding. 
Mechanical change (with some error messages changed from binding to class binding by 
hand).
2018-09-19 12:19:44 -07:00
Armin Braun 87cedef3cf
NETWORKING:Def CName in Http Publish Addr to True (#33631)
* Follow up to #32806 setting the setting to true for 7.x
2018-09-18 10:29:02 +02:00
Ryan Ernst 3046656ab1
Scripting: Rework joda time backcompat (#33486)
This commit switches the joda time backcompat in scripting to use
augmentation over ZonedDateTime. The augmentation methods provide
compatibility with the missing methods between joda's DateTime and
java's ZonedDateTime. Due to getDayOfWeek returning an enum in the java
API, ZonedDateTime is wrapped so that the method can return int like the
joda time does. The java time api version is renamed to
getDayOfWeekEnum, which will be kept through 7.x for compatibility while
users switch back to getDayOfWeek once joda compatibility is removed.
2018-09-16 19:18:00 -07:00
Armin Braun 94cdf0ceba
NETWORKING: http.publish_host Should Contain CNAME (#32806)
* NETWORKING: http.publish_host Should Contain CNAME

* Closes #22029
2018-09-12 06:15:36 +02:00
Alan Woodward 39c3234c2f
Upgrade to latest Lucene snapshot (#33505)
* LeafCollector.setScorer() now takes a Scorable
* Scorers may not have null Weights
* IndexWriter.getFlushingBytes() reports how much memory is being used by IW threads writing to disk
2018-09-10 20:51:55 +01:00
Jack Conradson facec187bb
Painless: Add Imported Static Method (#33440)
Allow static methods to be imported in Painless and called using just the method name.
2018-09-07 18:16:07 -07:00
Jim Ferenczi 7ad71f906a
Upgrade to a Lucene 8 snapshot (#33310)
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
2018-09-06 14:42:06 +02:00