Commit Graph

42435 Commits

Author SHA1 Message Date
Colin Goodheart-Smithe 0b26f8b14c
Fixes NPE in multi node qa testt 2018-10-25 10:45:30 +01:00
Costin Leau b97546a5a9
SQL: Introduce ODBC mode, similar to JDBC (#34825)
Close #34720
2018-10-25 12:40:00 +03:00
Andrei Stefan a7e08f462f
SQL: handle X-Pack or X-Pack SQL not being available in a more graceful way (#34736)
Throw a different error message for a http response code of 400, but also when the error itself is of a specific type.
2018-10-25 12:14:49 +03:00
王振荟 d6dc62ef03 [Docs] Add explanation for code snippets line width (#34796)
Add "Line width for code snippets is 76 characters" to  Java Language Formatting Guidelines
2018-10-25 10:55:14 +02:00
Nhat Nguyen ff49e79d40 CCR: Rename follow-task parameters and stats (#34836)
* CCR: Rename follow parameters and stats

This commit renames the follow-task parameters and its stats.
Below are the changes:

## Params
- remote_cluster (unchanged)
- leader_index (unchanged)
- max_read_request_operation_count -> max_read_request_operation_count
- max_batch_size -> max_read_request_size
- max_write_request_operation_count (new)
- max_write_request_size (new)
- max_concurrent_read_batches -> max_outstanding_read_requests
- max_concurrent_write_batches -> max_outstanding_write_requests
- max_write_buffer_size (unchanged)
- max_write_buffer_count (unchanged)
- max_retry_delay (unchanged)
- poll_timeout -> read_poll_timeout

## Stats
- remote_cluster (unchanged)
- leader_index (unchanged)
- follower_index (unchanged)
- shard_id (unchanged)
- leader_global_checkpoint (unchanged)
- leader_max_seq_no (unchanged)
- follower_global_checkpoint (unchanged)
- follower_max_seq_no (unchanged)
- last_requested_seq_no (unchanged)
- number_of_concurrent_reads -> outstanding_read_requests
- number_of_concurrent_writes -> outstanding_write_requests
- buffer_size_in_bytes -> write_buffer_size_in_bytes (new)
- number_of_queued_writes -> write_buffer_operation_count
- mapping_version -> follower_mapping_version
- total_fetch_time_millis -> total_read_time_millis
- total_fetch_remote_time_millis -> total_read_remote_exec_time_millis
- number_of_successful_fetches -> successful_read_requests
- number_of_failed_fetches -> failed_read_requests
- operation_received -> operations_read
- total_transferred_bytes -> bytes_read
- total_index_time_millis -> total_write_time_millis [?]
- number_of_successful_bulk_operations -> successful_write_requests
- number_of_failed_bulk_operations -> failed_write_requests
- number_of_operations_indexed -> operations_written
- fetch_exception -> read_exceptions
- time_since_last_read_millis -> time_since_last_read_millis

* add test for max_write_request_(operation_count|size)
2018-10-25 10:36:15 +02:00
Alpar Torok 13ca942152
Switch build-tools to latest target version (#34746)
- we already require Java 11 to build, yet we target the minimum
supported version in build-tools ( currently 8 )
- this is because we have some checks that are executed in a new JVM
which could be running the minimum version.
- For everything else it would be nice to be able to use new features,
like the new process API.

With this change, we selectively compile the few classes that need an
older target version and move everything over to Java 10.
Unfortunately the current Gradle version does not support 11 as a target
version yet.
2018-10-25 10:03:23 +03:00
Martijn van Groningen bb807b147b
fixed test 2018-10-25 08:09:45 +02:00
Armin Braun e7ced94a65
NETWORKING: Add SSL Handler before other Handlers (#34636)
* NETWORKING: Add SSL Handler before other Handlers

* The only way to run into the issue in #33998 is for `Netty4MessageChannelHandler`
to be in the pipeline while the SslHandler is not. Adding the SslHandler before any
other handlers should ensure correct ordering here even when we handle upstream events
in our own thread pool
* Ensure that channels that were closed concurrently don't trip the assertion
* Closes #33998
2018-10-25 07:51:14 +02:00
Alpar Torok 59536966c2
Add a new "contains" feature (#34738)
The contains syntax was added in #30874 but the skips were not properly
put in place.
The java runner has the feature so the tests will run as part of the
build, but language clients will be able to support it at their own
pace.
2018-10-25 08:50:50 +03:00
Jason Tedor 153157e56d
Enable ingest attachment docs tests on JDK 11 (#34770)
These were disabled because ingest attachement was not playing well with
JDK 11. We have addressed that by upgrading the Tika dependency, yet
forgot to reenable these. This commit enables these tests again.
2018-10-24 23:17:45 -04:00
Ryan Ernst 687dc1eb11
Scripting: Remove SearchScript (#34730)
This commit removes the last non context based script class.
2018-10-24 15:03:38 -07:00
Mayya Sharipova d7afd7c123 Reduce the number of callouts to 15 in docs
relates to #33447
2018-10-24 17:57:35 -04:00
Martijn van Groningen 6fe0e62b7a
[CCR] Added write buffer size limit (#34797)
This limit is based on the size in bytes of the operations in the write buffer. If this limit is exceeded then no more read operations will be coordinated until the size in bytes of the write buffer has dropped below the configured write buffer size limit.

Renamed existing `max_write_buffer_size` to ``max_write_buffer_count` to indicate that limit is count based.

Closes #34705
2018-10-24 23:48:49 +02:00
Mark Tozzi d94406a68a
HLRC: Deactivate Watch API (#34192)
Relates to #29827
2018-10-24 17:01:22 -04:00
Mayya Sharipova 3211760e2c Correct term-vectors.asciidoc
relates to #33447
2018-10-24 16:35:20 -04:00
Mayya Sharipova 6cceba9b54 Correct CRUDDocumentationIT.java
relates to #33447
2018-10-24 15:59:58 -04:00
Jack Conradson 1b085252c3
[Painless] Add instance bindings (#34410)
This change adds instance bindings to Painless. This binding allows a whitelisted
 method to be called on an instance instantiated prior to script compilation. 
Whitelisting must be done in code as there is no practical way to instantiate a 
useful instance from a text file (see the tests for an example). Since an 
instance can be shared by multiple scripts, each method called must be 
thread-safe.
2018-10-24 12:57:28 -07:00
Andrey Atapin 5f588180f9 Improve IndexNotFoundException's default error message (#34649)
This commit adds the index name to the error message when an index is not found.
2018-10-24 12:53:31 -07:00
Stéphane Campinas 04f3e67c77 Remove redundant method from RestClearScrollAction (#34268)
The check for null argument is already done in `splitStringByCommaToArray`, hence it can be removed, which allows us to remove the whole splitScrollIds private method.
2018-10-24 21:31:29 +02:00
Luca Cavanna d51bc05dce
[TEST] Improve validation of do sections (#34734)
We throw parsing exception when an unknown array is found, but we don't when an unknown top-level field is found. This commit makes sure that unsupported top-level fields are not ignored in a do section.

Closes #34651
2018-10-24 21:27:07 +02:00
Benjamin Trent cd27b0b996
Revert "Rollup add default metrics to histo groups (#34534)" (#34815)
This reverts commit 4236358f5d.
2018-10-24 14:25:10 -05:00
Costin Leau 7570d69254
SQL: Introduce support for IP fields (#34758)
IP fields are recognized and can be used through-out the query

Close #32499
2018-10-24 22:03:11 +03:00
Marios Trivyzas 98cd7ca861
SQL: Fix queries with filter resulting in NO_MATCH (#34812)
Previously, `Mapper` was returning an incorrect plan which resulted in an
```
SQLFeatureNotSupportedException: Found 1 problem(s)
line 1:8: Unexecutable item
```

Queries with a `WHERE` and/or `HAVING` clause which results in NO_MATCH
are now handled correctly and return 0 rows.

Fixes: #34613

Co-authored-by: Costin Leau <costin.leau@gmail.com>
2018-10-24 20:36:30 +02:00
Mayya Sharipova bf4d90a5dc
HLRC API for _termvectors (#33447)
* HLRC API for _termvectors

relates to #27205
2018-10-24 14:27:22 -04:00
Colin Goodheart-Smithe e7fddb5c93
Adds usage data for ILM (#33377)
* Adds usage data for ILM

* Adds tests for IndexLifecycleFeatureSetUsage and friends

* Adds tests for IndexLifecycleFeatureSet

* Fixes merge errors

* Add number of indices managed to usage stats

Also adds more tests

* Addresses Review comments
2018-10-24 18:28:46 +01:00
Costin Leau f19565c3e0
SQL: Verifier allows aliases aggregates for sorting (#34773)
Improve Verifier rule that prevented grouping with aliases inside aggregates
to not be accepted for ordering.

Close #34607
2018-10-24 19:31:26 +03:00
markharwood f7a6fb288f
Fix compilation error peculiar to eclipse2018-09 (#34798)
Fix compilation error peculiar to eclipse2018-09.
Added generics to make eclipse compiler happy
2018-10-24 16:58:31 +01:00
Nhat Nguyen d73768f812
CCR: Do not follow if leader does not have soft-deletes (#34767)
We should not create a follower index and abort a follow request if the
leader does not have soft-deletes. Moreover, we also should not
auto-follow an index if it does not have soft-deletes.
2018-10-24 11:19:39 -04:00
Armin Braun 0a85997cb0
TESTS: Use File Based Discovery in REST Tests (#34560)
* For `6.5+` use file based discovery in REST tests
* Relates #33675
2018-10-24 17:18:52 +02:00
Nik Everett e0d7808148 REST: No strict warning testing for head body test
Our tests that HEAD requests don't have a body was hitting a warning.
For now we'll run that test without "strict" warnings mode enabled.
2018-10-24 11:17:38 -04:00
Alpar Torok 4370b44ef0
Rename testcluster extension to align with plugin name (#34751) 2018-10-24 18:15:25 +03:00
Colin Goodheart-Smithe c7fe87e43f
Removes Set Policy API in favour of setting index.lifecycle.name directly (#34304)
* Removes Set Policy API in favour of setting index.lifecycle.name
directly

* Reinstates matcher that will still be used

* Cleans up code after rebase

* Adds test to check changing policy with ndex settings works

* Fixes TimeseriesLifecycleActionsIT after API removal

* Fixes docs tests

* Fixes case on close where lifecycle service was never created
2018-10-24 16:14:59 +01:00
Shaunak Kashyap 0f1a5ecf61
Adding stack_monitoring_agent role (#34369)
* Adding stack_monitoring_agent role

* Fixing checkstyle issues

* Adding tests for new role

* Tighten up privileges around index templates

* s/stack_monitoring_user/remote_monitoring_collector/ + remote_monitoring_user

* Fixing checkstyle violation

* Fix test

* Removing unused field

* Adding missed code

* Fixing data type

* Update Integration Test for new builtin user
2018-10-24 07:20:46 -07:00
Alpar Torok 795d57b4f9
Auto configure all test tasks (#34666)
With this change, we apply the common test config automatically to all
newly created tasks instead of opting in specifically.

For plugin authors using the plugin externally this means that the
configuration will be applied to their RandomizedTestingTasks as well.

The purpose of the task is to simplify setup and make it easier to
change projects that use the `test` task but actually run integration
tests to use a task called `integTest` for clarity, but also because
we may want to configure and run them differently.
E.x. using different levels of concurrency.
2018-10-24 16:05:50 +03:00
Marios Trivyzas 4c73854da7
SQL: Implement null handling for `IN(v1, v2, ...)` (#34750)
Implemented null handling for both the value tested but also for
values inside the list of values tested against.

The null handling is implemented for local processors, painless scripts
and Lucene Terms queries making it available for `IN` expressions occuring
in `SELECT`, `WHERE` and `HAVING` clauses.

Closes: #34582
2018-10-24 14:42:40 +02:00
lipsill d5ad3de42e [test] Introduce strict deprecation mode for REST tests (#34338)
#33708 introduced a strict deprecation mode that makes a REST request
fail if there is a warning header in the response returned by
Elasticsearch (usually a deprecation message signaling that a feature
or a field has been deprecated).

This change adds the strict deprecation mode into the REST integration
tests, and makes the tests fail if a deprecated feature is used. Also
any test using a deprecated feature has been modified to pass the build.

The YAML integration tests already analyzed HTTP warnings so they do
not use this mode, keeping their "expected vs actual" behavior.
2018-10-24 08:21:24 -04:00
Colin Goodheart-Smithe 89404ced9b
Fixes SOTP docs test to restart ILM after 2018-10-24 13:05:41 +01:00
Martijn van Groningen 76240e6bbe
[CCR] Renamed leader_cluster to remote_cluster (#34776)
and also some occurrences of clusterAlias to remoteCluster.

Closes #34682
2018-10-24 13:39:36 +02:00
Boaz Leskes be907516ad
Change ShardFollowTask defaults (#34793)
Per #31717 this commit changes the defaults to the following:

Batch size of 5120 ops.
Maximum of 12 concurrent read requests.
Maximum of 9 concurrent write requests.

This is not necessarily our final values but it's good to have these as defaults for the purposes of initial testing.
2018-10-24 13:32:48 +02:00
hanbing0715 fe54f73434 Persist generated public keys unencrypted (#34626)
The changes introduced in cca1a2a mean that we should
not encrypt the public keys that might be generated by
the key-pair-generator when storing the file, as the code 
that would consume them assumes that they are not encrypted
2018-10-24 13:55:21 +03:00
Martijn van Groningen 18e004c3ea
[CCR] Removed unused static parse field. 2018-10-24 09:27:08 +02:00
Martijn van Groningen 18007a29b2
[CCR] Made leader cluster required in shard follow task.
Left over from #34580
2018-10-24 08:38:25 +02:00
Martijn van Groningen abf8cb6706
[CCR] Cleanup pause follow action (#34183)
* Change the `TransportPauseFollowAction` to extend from `TransportMasterNodeAction`
  instead of `HandledAction`, this removes a sync cluster state api call.
* Introduced `ResponseHandler` that removes duplicated code in `TransportPauseFollowAction` and
  `TransportResumeFollowAction`.
* Changed `PauseFollowAction.Request` to not use `readFrom()`.
2018-10-24 08:12:39 +02:00
Nhat Nguyen 52266d8b11
TEST: Clone replicas list when compute replication targets (#34728)
In #34407, we supposed to clone the list of replicas of ReplicationGroup
when computing replication targets, but somehow we missed it. If we
don't clone the list, a WriteReplicationAction may use an old
ReplicationTargets which consists replicas which are removed from the
current list of replicas

Relates #34407
Closes #33457
2018-10-23 21:08:34 -04:00
Ryan Ernst 8da1c9626a
Scripting: Add back params._source access in scripted metric aggs (#34777)
Access to special variables _source and _fields were accidentally
removed in recent refactorings. This commit adds them back, along with a
test.

closes #33884
2018-10-23 18:07:53 -07:00
Ryan Ernst 596b5cf108
Test: Fix last reference to SearchScript (#34731)
This was accidentally left over when converting to FieldScript.

closes #34683
2018-10-23 17:26:17 -07:00
Gordon Brown da20dfd81c
Add cluster-wide shard limit warnings (#34021)
In a future major version, we will be introducing a soft limit on the
number of shards in a cluster based on the number of nodes in the
cluster. This limit will be configurable, and checked on operations
which create or open shards and issue a warning if the operation would
take the cluster over the limit.

There is an option to enable strict enforcement of the limit, which
turns the warnings into errors.  In a future release, the option will be
removed and strict enforcement will be the default (and only) behavior.
2018-10-23 16:35:10 -06:00
Julie Tibshirani c5a0739381 Mute SettingsBasedHostProviderIT to avoid future test flakes. 2018-10-23 15:26:39 -07:00
Julie Tibshirani 7580f59c9f Mute FullClusterRestartIT#testSqlFailsOnIndexWithTwoTypes while we await a fix. 2018-10-23 15:14:08 -07:00
Gordon Brown 17adfeb20f
HLRC: Standardize access in *RequestConverters (#34768)
With the move to separate RequestConverters classes for each client,
some of the access restrictions on the new classes are more open than
the prior RequestConverters classes. This standardizes the
*RequestConverters classes as package-private, final, and with a private
constructor so that no instances of the can be inadvertently created.
2018-10-23 15:59:11 -06:00