Commit Graph

263 Commits

Author SHA1 Message Date
Julie Tibshirani fba710469a
Refactor the REST actions to clarify what endpoints are deprecated. () 2018-12-20 18:06:41 -08:00
Julie Tibshirani 33152f648f
Fix some inconsistencies in the types deprecation code. ()
* Make sure to test conversion for both typed and typeless HLRC requests.
* Update a few more statements to deprecatedAndMaybeLog.
* Make sure Rest*SearchTemplateActionTests extend RestActionTestCase.
2018-12-12 10:38:02 -08:00
Julie Tibshirani b15d1aebcf
For msearch templates, make sure to use the right name for deprecation logging. () 2018-12-07 14:50:47 -08:00
Julie Tibshirani 51e1d40dca
Small improvements related to types deprecation. ()
* Make sure to use deprecatedAndMaybeLog for types deprecation messages.
* Introduce a common base class for Rest*Action tests.
2018-12-07 11:21:24 -08:00
Jim Ferenczi 18866c4c0b
Make hits.total an object in the search response ()
This commit changes the format of the `hits.total` in the search response to be an object with
a `value` and a `relation`. The `value` indicates the number of hits that match the query and the
`relation` indicates whether the number is accurate (in which case the relation is equals to `eq`)
or a lower bound of the total (in which case it is equals to `gte`).
This change also adds a parameter called `rest_total_hits_as_int` that can be used in the
search APIs to opt out from this change (retrieve the total hits as a number in the rest response).
Note that currently all search responses are accurate (`track_total_hits: true`) or they don't contain
`hits.total` (`track_total_hits: true`). We'll add a way to get a lower bound of the total hits in a
follow up (to allow numbers to be passed to `track_total_hits`).

Relates 
2018-12-05 19:49:06 +01:00
Jim Ferenczi 8a7f3f75f3
Add support for rest_total_hits_as_int ()
The support for rest_total_hits_as_int has already been merged to 6x
in  so this change adds this new option to master. The plan was
to add this new option as part of  but we've decided to wait a few
days before merging this breaking change so this commit just handles
the new option as a noop exactly like 6x for now. This will allow
users to migrate to this parameter before  is merged.

Relates 
2018-11-29 18:36:16 +01:00
Julie Tibshirani 25c416b12d
Deprecate types in search and multi search templates. ()
This PR adds deprecation warnings to the relevant `Rest*Action` classes, plus tests in `Rest*ActionTests`. No updates to REST tests, the Java HLRC, or documentation were necessary, since they didn't make use of types.
2018-11-27 10:19:19 -08:00
Alpar Torok 8a85b2eada
Remove build qualifier from server's Version ()
With this change, `Version` no longer carries information about the qualifier,
we still need a way to show the "display version" that does have both
qualifier and snapshot. This is now stored  by the build and red from `META-INF`.
2018-11-07 14:01:05 +02:00
Simon Willnauer 833e0f8ecf
Prevent throttled indices to be searched through wildcards by default ()
Today if a wildcard, date-math expression or alias expands/resolves
to an index that is search-throttled we still search it. This is likely
not the desired behavior since it can unexpectedly slow down searches
significantly.

This change adds a new indices option that allows `search`, `count`
and `msearch` to ignore throttled indices by default. Users can
force expansion to throttled indices by using `ignore_throttled=true`
on the rest request to expand also to throttled indices.

Relates to 
2018-11-06 09:45:30 +01:00
Alexander Reelsen 409050e8de
Refactor: Remove settings from transport action CTOR ()
As settings are not used in the transport action constructor, this
removes the passing of the settings in all the transport actions.
2018-11-05 13:08:18 +01:00
Alpar Torok 59536966c2
Add a new "contains" feature ()
The contains syntax was added in  but the skips were not properly
put in place.
The java runner has the feature so the tests will run as part of the
build, but language clients will be able to support it at their own
pace.
2018-10-25 08:50:50 +03:00
Andrey Atapin 5f588180f9 Improve IndexNotFoundException's default error message ()
This commit adds the index name to the error message when an index is not found.
2018-10-24 12:53:31 -07:00
Nik Everett 06993e0c35
Logging: Make ESLoggerFactory package private ()
Since all calls to `ESLoggerFactory` outside of the logging package were
deprecated, it seemed like it'd simplify things to migrate all of the
deprecated calls and declare `ESLoggerFactory` to be package private.
This does that.
2018-10-06 09:54:08 -04:00
Nik Everett 6c8f568808
Switch remaining LLREST usage to new style Requests ()
In  we added `Request` object flavored requests to the low level
REST client and in  we deprecated the old `performRequest`s. In a
long series of PRs I've changed all of the old style requests that I
could find with `grep`. In this PR I change all requests that I could
find by *removing* the deprecated methods. Since this is a non-trivial
change I do not include actually removing the deprecated requests. I'll
do that in a follow up. But this should be the last set of usage
removals before the actual deprecated method removal. Yay!
2018-08-28 14:20:14 -04:00
Jim Ferenczi f4e9729d64
Remove unsupported Version.V_5_* ()
This change removes the es 5x version constants and their usages.
2018-08-24 09:51:21 +02:00
Armin Braun ccf6126410
SCRIPTING: Remove unused MultiSearchTemplateRequestBuilder ()
* Ever since 46e8d97813 this class is unused
2018-07-14 09:03:35 +02:00
Christoph Büscher bd1c513422
Reduce more raw types warnings ()
Similar to .
2018-07-05 15:38:06 +02:00
Sohaib Iftikhar 40b822c878 Scripting: Remove support for deprecated StoredScript contexts ()
Removes support for storing scripts without the usual json around the
script. So You can no longer do:
```
POST _scripts/<templatename>
{
    "query": {
        "match": {
            "title": "{{query_string}}"
        }
    }
}
```

and must instead do:
```
POST _scripts/<templatename>
{
    "script": {
        "lang": "mustache",
        "source": {
            "query": {
                "match": {
                    "title": "{{query_string}}"
                }
            }
        }
    }
}
```

This improves error reporting when you attempt to store a script but don't
quite get the syntax right. Before, there was a good chance that we'd
think of it as a "raw" template and just store it. Now we won't do that.
Nice.
2018-07-05 09:30:08 -04:00
markharwood 09dd19a403
Add MultiSearchTemplate support to High Level Rest client ()
Add MultiSearchTemplate support to High Level Rest client.
Addresses part of 
2018-06-28 14:05:26 +01:00
Alpar Torok 08b8d11e30
Add support for switching distribution for all integration tests ()
* remove left-over comment

* make sure of the property for plugins

* skip installing modules if these exist in the distribution

* Log the distrbution being ran

* Don't allow running with integ-tests-zip passed externally

* top level x-pack/qa can't run with oss distro

* Add support for matching objects in lists

Makes it possible to have a key that points to a list and assert that a
certain object is present in the list. All keys have to be present and
values have to match. The objects in the source list may have additional
fields.

example:
```
  match:  { 'nodes.$master.plugins': { name: ingest-attachment }  }
```

* Update plugin and module tests to work with other distributions

Some of the tests expected that the integration tests will always be ran
with  the `integ-test-zip` distribution so that there will be no other
plugins loaded.

With this change, we check for the presence of the plugin without
assuming exclusivity.

* Allow modules to run on other distros as well

To match the behavior of tets.distributions

* Add and use a new `contains` assertion

Replaces the  previus changes that caused `match` to do a partial match.

* Implement PR review comments
2018-06-26 06:49:03 -07:00
Ryan Ernst 7a150ec06d
Core: Combine doExecute methods in TransportAction ()
TransportAction currently contains 2 doExecute methods, one which takes
a the task, and one that does not. The latter is what some subclasses
implement, while the first one just calls the latter, dropping the given
task. This commit combines these methods, in favor of just always
assuming a task is present.
2018-06-22 15:03:01 -07:00
Ryan Ernst 4f9332ee16
Core: Remove ThreadPool from base TransportAction ()
Most transport actions don't need the node ThreadPool. This commit
removes the ThreadPool as a super constructor parameter for
TransportAction. The actions that do need the thread pool then have a
member added to keep it from their own constructor.
2018-06-21 11:25:26 -07:00
Ryan Ernst 0a324b9943
Core: Convert TransportAction.execute uses to client calls ()
This commit converts some of the existing calls to
TransportAction.execute to use the equivalent client method for the
desired action.
2018-06-21 07:59:55 -07:00
Ryan Ernst 401800d958
Core: Remove index name resolver from base TransportAction ()
Most transport actions don't need to resolve index names. This commit
removes the index name resolver as a super constructor parameter for
TransportAction. The actions that do need the resolver then have a
member added to keep the resolver from their own constructor.
2018-06-19 17:06:09 -07:00
Ryan Ernst e67aa96c81
Core: Combine Action and GenericAction ()
Since , Action no longer has anything but a call to the
GenericAction super constructor. This commit renames GenericAction
into Action, thus eliminating the Action class. Additionally, this
commit removes the Request generic parameter of the class, since
it was unused.
2018-06-18 23:53:04 +02:00
Tanguy Leroux bf58660482
Remove all unused imports and fix CRLF ()
The X-Pack opening and the recent other refactorings left a lot of 
unused imports in the codebase. This commit removes them all.
2018-06-11 15:12:12 +02:00
rationull 85c26d682a Call ensureNoSelfReferences() on _agg state variable after scripted metric agg script executions ()
Previously this was called for the combine script only. This change checks for self references for
init, map, and reduce scripts as well, and adds unit test coverage for the init, map, and combine cases.
2018-06-11 08:39:05 +02:00
Ryan Ernst 46e8d97813
Core: Remove RequestBuilder from Action ()
This commit removes the RequestBuilder generic type from Action. It was
needed to be used by the newRequest method, which in turn was used by
client.prepareExecute. Both of these methods are now removed, along with
the existing users of prepareExecute constructing the appropriate
builder directly.
2018-05-31 16:15:00 +02:00
markharwood facbb2b2dc
Add “took” timing info to response for _msearch/template API ()
Add “took” timing info to response for _msearch/template API
Closes 
2018-05-30 17:57:28 +01:00
Christoph Büscher 1ea9f11b03
Change ScriptException status to 400 (bad request) ()
Currently failures to compile a script usually lead to a ScriptException, which
inherits the 500 INTERNAL_SERVER_ERROR from ElasticsearchException if it does
not contain another root cause. Instead, this should be a 400 Bad Request error.
This PR changes this more generally for script compilation errors by changing 
ScriptException to return 400 (bad request) as status code.

Closes 
2018-05-30 14:00:07 +02:00
Sohaib Iftikhar 3c918d799c Deprecate accepting malformed requests in stored script API ()
The stored scripts API today accepts malformed requests instead of throwing an exception.
This PR deprecates accepting malformed put stored script requests (requests not using the official script format).

Relates to 
2018-05-29 15:45:53 +02:00
Julie Tibshirani 4f9dd37169
Add support for search templates to the high-level REST client. () 2018-05-15 13:07:58 -07:00
Jim Ferenczi fb81e2cacf Fix template _msearch with extra tokens
This change removes the check for extra tokens when parsing a source generated by a templated
_msearch request. This was added unintentionally in  but the intent of this modification was to validate
simple _search request only.
2018-04-11 18:04:10 +02:00
Jim Ferenczi 1b6d5e531b
Fail _search request with trailing tokens ()
This change validates that the `_search` request does not have trailing
tokens after the main object and fails the request with a parsing exception otherwise.

Closes 
2018-04-11 13:10:22 +02:00
Lee Hinman a93c942927
Move ObjectParser into the x-content lib ()
* Move ObjectParser into the x-content lib

This moves `ObjectParser`, `AbstractObjectParser`, and
`ConstructingObjectParser` into the libs/x-content dependency. This decoupling
allows them to be used for parsing for projects that don't want to depend on the
entire Elasticsearch jar.

Relates to 
2018-04-06 09:41:14 -06:00
Lee Hinman 8e8fdc4f0e
Decouple XContentBuilder from BytesReference ()
* Decouple XContentBuilder from BytesReference

This commit removes all mentions of `BytesReference` from `XContentBuilder`.
This is needed so that we can completely decouple the XContent code and move it
into its own dependency.

While this change appears large, it is due to two main changes, moving
`.bytes()` and `.string()` out of XContentBuilder itself into static methods
`BytesReference.bytes` and `Strings.toString` respectively. The rest of the
change is code reacting to these changes (the majority of it in tests).

Relates to 
2018-03-14 13:47:57 -06:00
Lee Hinman 46a79127ed
Remove FastStringReader in favor of vanilla StringReader ()
This allows us to remove another dependency in the decoupling of the XContent
code. Rather than move this class over or decouple it, it can simply be removed.

Relates tangentially to 
2018-03-08 17:17:36 -07:00
Lee Hinman 7c1f5f5054
Move more XContent.createParser calls to non-deprecated version ()
* Move more XContent.createParser calls to non-deprecated version

This moves more of the callers to pass in the DeprecationHandler.

Relates to 

* Use parser's deprecation handler where available
2018-02-14 09:01:40 -07:00
Simon Willnauer 7957c9751c [TEST] fix test to use a dedicated index to ensure lucene docIDs are guaranteed 2018-01-26 14:52:39 +01:00
Simon Willnauer 4d3f7a7695
Ensure we protect Collections obtained from scripts from self-referencing ()
Self referencing maps can cause SOE if they are iterated ie. in their toString methods. This chance adds some protected to the usage of those collections.
2018-01-23 16:57:26 +01:00
Martijn van Groningen 4d78e1a9ad
Added msearch api to high level client 2017-12-05 10:17:47 +01:00
Simon Willnauer fadbe0de08
Automatically prepare indices for splitting ()
Today we require users to prepare their indices for split operations.
Yet, we can do this automatically when an index is created which would
make the split feature a much more appealing option since it doesn't have
any 3rd party prerequisites anymore.

This change automatically sets the number of routinng shards such that
an index is guaranteed to be able to split once into twice as many shards.
The number of routing shards is scaled towards the default shard limit per index
such that indices with a smaller amount of shards can be split more often than
larger ones. For instance an index with 1 or 2 shards can be split 10x
(until it approaches 1024 shards) while an index created with 128 shards can only
be split 3x by a factor of 2. Please note this is just a default value and users
can still prepare their indices with `index.number_of_routing_shards` for custom
splitting.

NOTE: this change has an impact on the document distribution since we are changing
the hash space. Documents are still uniformly distributed across all shards but since
we are artificually changing the number of buckets in the consistent hashign space
document might be hashed into different shards compared to previous versions.

This is a 7.0 only change.
2017-11-23 09:48:54 +01:00
Nik Everett eb754a71be Fix update_by_query's default size parameter ()
We were accidentally defaulting it to the scroll size.
Untwists some of the tricks that we play with parsing
so that the size is no longer scrambled.

Closes 
2017-09-25 16:25:27 -04:00
Christoph Büscher 22e200e79a Remove deprecated type and slop field in MatchQueryBuilder ()
The `type` field has been deprecated in 5.0 and can be removed. It has been
replaced by using the MatchPhraseQueryBuilder or the
MatchPhrasePrefixQueryBuilder. The `slop` field has also been deprecated and can
be removed, the phrase and phrase prefix query builders still provide this
parameter.
2017-09-20 14:24:30 +02:00
Tal Levy ed151d829d Migrate Search requests to use Writeable reading strategies ()
Migrates many SearchRequest objects to use Writeable conventions and rejects usage of `readFrom` in these new classes.
2017-08-30 11:00:33 -07:00
Luca Cavanna 14ba36977e [TEST] prevent yaml tests from using raw requests ()
Raw requests are supported only by the java yaml test runner and were introduced to test docs snippets. Some yaml tests ended up using them (see ) which causes failures for other language clients. This commit migrates those yaml tests to Java tests that send requests through the Java low-level REST client, and also moves the ability to send raw requests to a special client that's only available when testing docs snippets.

Closes 
2017-08-07 11:02:16 +02:00
Jack Conradson 9f7463e796 remove lang url parameter from stored script requests ()
Also has updates to ScriptMetaData for allowing the old namespace format to be loaded all the way back through 5.0; however, it will throw an exception if two scripts share the same id but different languages.
2017-07-20 08:51:08 -07:00
Ryan Ernst 072402463b Scripting: Remove search template actions ()
The dedicated search template put/get/delete actions are deprecated in
5.6. This commit removes them from 6.0.
2017-07-14 23:12:05 -07:00
Ryan Ernst 70b2897bdf Scripting: Deprecate stored search template apis ()
This commit deprecates the PUT, GET and DELETE search template apis.
Instead, the stored script api should be used.

closes 
2017-07-12 16:07:28 -07:00
Jack Conradson d2b4f7ac5a Disallow lang to be used with Stored Scripts ()
Requests that execute a stored script will no longer be allowed to specify the lang of the script. This information is stored in the cluster state making only an id necessary to execute against. Putting a stored script will still require a lang.
2017-07-12 07:55:57 -07:00