Commit Graph

45351 Commits

Author SHA1 Message Date
Marios Trivyzas ddf17dfb1e
SQL: Small code improvements of Pipes & Processors (#40909)
- Remove superfluous methods that are already
defined in superclasses.
- Improve tests for null folding on conditionals

(cherry picked from commit 67f9404f5004362e569353d1e950ffe5d7a9ab6e)
2019-04-08 10:14:13 +02:00
Martijn van Groningen c29027d99e
Change HLRC CCR response tests to use AbstractResponseTestCase base class. (#40257)
This way the response classes are tested in a more realistic setting.

Relates to #39745
2019-04-08 07:09:28 +02:00
Tim Vernum 26c63e0115
Add test for HTTP and Transport TLS on basic license (#40932)
This adds a new security/qa test for TLS on a basic license.

It starts a 2 node cluster with a basic license, and TLS enabled
on both HTTP and Transport, and verifies the license type, x-pack
SSL usage and SSL certificates API.

It also upgrades the cluster to a trial license and performs that
same set of checks (to ensure that clusters with basic license
and TLS enabled can be upgraded to a higher feature license)

Backport of: #40714
2019-04-08 13:23:12 +10:00
Hendrik Muhs d5fcbf2f4a refactor onStart and onFinish to take runnables and executed them guarded by state (#40855)
refactor onStart and onFinish to take action listeners and execute them when indexer is in indexing state.
2019-04-07 21:46:37 +02:00
Jason Tedor 4163e59768
Mute failing IndexShard local history test
This test fails reliably with, so this commit mutes that test until a
fix is available.
2019-04-07 10:17:46 -04:00
Jason Tedor 6900399144
Be lenient when parsing build flavor and type on the wire (#40734)
Today we are strict when parsing build flavor and types off the
wire. This means that if a later version introduces a new build flavor
or type, an older version would not be able to parse what that new
version is sending. For a practical example of this, we recently added
the build type "docker", and this means that in a rolling upgrade
scenario older nodes would not be able to understand the build type that
the newer node is sending. This breaks clusters and is bad. We do not
normally think of adding a new enumeration value as being a
serialization breaking change, it is just not a lesson that we have
learned before. We should be lenient here though, so that we can add
future changes without running the risk of breaking ourselves
horribly. It is either that, or we have super-strict testing
infrastructure here yet still I fear the possibility of mistakes. This
commit changes the parsing of build flavor and build type so that we are
still strict at startup, yet we are lenient with values coming across
the wire. This will help avoid us breaking rolling upgrades, or clients
that are on an older version.
2019-04-06 17:24:16 -04:00
Jason Tedor e44e84ab42
Suppress lease background sync failures if stopping (#40902)
If the transport service is stopped, likely because we are shutting
down, and a retention lease background sync fires the logs will display
a warn message and stacktrace. Yet, this situaton is harmless and can
happen as a normal course of business when shutting down. This commit
suppresses the log messages in this case.
2019-04-06 10:18:52 -04:00
debadair a69ff8221f [DOCS] Added settings page for ILM. (#40880)
* [DOCS] Added settings page for ILM.

* [DOCS] Adding ILM settings file

* [DOCS] Moved the ILM settings to a separate section

* [DOCS] Linked to the rollover docs.

* [DOCS] Tweaked the "required" wording.
2019-04-05 16:39:18 -07:00
DeDe Morton b0ce3e0a10 [Docs] Remove extraneous text (#40914)
Removes text that was likely introduced by copy/paste error.
2019-04-05 15:32:25 -07:00
Jack Conradson fd51780de2 Move test classes to test root in Painless (#40873)
This moves several test classes that were part of the main root to the test root. 
These were part of the main root due to limitations prior to whitelist 
customization. Without whitelist customization these can be moved to a test 
context and removed from the base whitelists as they should not be user 
facing.
2019-04-05 15:16:04 -07:00
Jason Tedor ac58b9bded
Fix date index name processor default date_formats (#40915)
This commit is a correction of a doc bug in the docs for the ingest
date-index-name processor. The correct pattern is
yyyy-MM-dd'T'HH:mm:ss.SSSXX. This is due to the transition from Joda
time to Java time where Z does not mean the same thing between the two.
2019-04-05 17:45:57 -04:00
Ioannis Kakavas aea4e6596f
Source additional files correctly in elasticsearch-cli (#40890)
Since we only source additional sources from the same dir as our
cli scripts, resolve the path relevant to $ES_HOME
2019-04-05 17:42:34 -04:00
lcawl f4348843ba [DOCS] Adds placeholder for 7.0.0 release notes 2019-04-05 14:27:05 -07:00
Jason Tedor 2fd1689341
Allow AVX-512 on JDK 11+ (#40828)
We previously found a bug in the JVM where AVX-512 instructions could
crash the JVM to crash with a segmentation fault. This bug impacted JDK
9 and JDK 10, but was most prominent on JDK 10 because AVX-512 was
enabled there by default. In JDK 11, this bug is reported fixed so this
commit restricts the disabling of AVX-512 to JDK 10 only. Since we no
longer support JDK 10 for any versions that this commit will be
integrated into (7.1, 8.0), we simply remove the disabling of this flag
from the JVM options.
2019-04-05 16:40:31 -04:00
Jay Modi f34663282c
Update apache httpclient to version 4.5.8 (#40875)
This change updates our version of httpclient to version 4.5.8, which
contains the fix for HTTPCLIENT-1968, which is a bug where the client
started re-writing paths that contained encoded reserved characters
with their unreserved form.
2019-04-05 13:48:10 -06:00
DeDe Morton 360d8d40be [Docs] Change example to show col headers (#40822)
Command needs `?v` so user can see the column headers. Otherwise the instructions in the note about checking the init and relo columns don't make sense
2019-04-05 14:58:09 -04:00
David Turner 2ff19bc1b7
Use Writeable for TransportReplAction derivatives (#40905)
Relates #34389, backport of #40894.
2019-04-05 19:10:10 +01:00
Benjamin Trent a8dbb07546
[ML] Changes default destination index field mapping and adds scripted_metric agg (#40750) (#40846)
* [ML] Allowing destination index mappings to have dynamic types, adds script_metric agg

* Making dynamic|source mapping explicit
2019-04-05 11:34:20 -05:00
Colin Goodheart-Smithe 4452e8e10f Mutes GatewayIndexStateIT.testRecoverBrokenIndexMetadata 2019-04-05 10:53:52 -04:00
Brandon Kobel eb656244bf Add Kibana application privileges for monitoring and ml reserved roles (#40651)
* Add Kibana application privileges for monitoring and ml reserved roles

* Adding test for kibana-.kibana application explicitly

* Whoa there, fat fingered kibana and application...

* And I copied something from monitoring I shouldn't have...

* And actually doing what Yogesh recommended...
2019-04-05 07:44:29 -07:00
Benjamin Trent 665f0d81aa
[ML] refactoring start task a bit, removing unused code (#40798) (#40845) 2019-04-05 09:01:01 -05:00
David Turner 922a70ce32 Remove unused import
Relates #40863
2019-04-05 09:21:34 +01:00
David Turner d8956d2601 Remove test-only customisation from TransReplAct (#40863)
The `getIndexShard()` and `sendReplicaRequest()` methods in
TransportReplicationAction are effectively only used to customise some
behaviour in tests. However there are other ways to do this that do not cause
such an obstacle to separating the TransportReplicationAction into its two
halves (see #40706).

This commit removes these customisation points and injects the test-only
behaviour using other techniques.
2019-04-05 08:54:41 +01:00
Alexander Reelsen 669d72e47a Fix dense/sparse vector limit documentation (#40852)
The documentation stated a wrong limit of dense/sparse vector sizes.
This was changed in #40597 but the documentation was not fixed.
2019-04-05 09:35:30 +02:00
Tim Vernum 920024e06d
Remove dynamic objects from security index (#40715)
The security index had a few "object" types with

   "dynamic": true

However, this automatically creates a mapping for each field that is
created within those objects. This means that types are dynamically
inferred and "locked in" for future updates.

Instead we want "dynamic": false which will allow us to store a range
of fields in these nested objects and retrieve them from the source,
without creating mapping types for those fields.

Backport of: #40499
2019-04-05 18:28:09 +11:00
Martijn van Groningen 809a5f13a4
Make -try xlint warning disabled by default. (#40833)
Many gradle projects specifically use the -try exclude flag, because
there are many cases where auto-closeable resource ignore is never
referenced in body of corresponding try statement. Suppressing this
warning specifically in each case that it happens using
`@SuppressWarnings("try")` would be very verbose.

This change removes `-try` from any gradle project and adds it to the
build plugin. Also this change removes exclude flags from gradle projects
that is already specified in build plugin (for example -deprecation).

Relates to #40366
2019-04-05 08:02:26 +02:00
Nhat Nguyen 5a2eb07c0e Primary replica resync should not send ops without seqno (#40433)
Primary-replica resync in a mixed-cluster between 6.x and 5.6 can send
operations without sequence number to a replica which already processed
operations with sequence number. This leads to the failure of that
replica for we trip the sequence number assertion when writing resync
operations without sequence number to translog.
2019-04-04 21:54:31 -04:00
Deb Adair 8f72d72c4c [DOCS] Removed redundant (not quite right) information about upgrades. 2019-04-04 16:19:34 -07:00
Jim Ferenczi a15f55b2de Rollup ignores time_zone on date histogram (#40844)
When translating the original aggregation for the rollup indices,
the timezone of the date histogram is validated against the rollup
job but the value is not copied in the newly created date_histogram.
2019-04-04 21:16:50 +02:00
roy 075078e7e0 HLRC: fix uri encode bug when url path starts with '/' (#34436)
This commit sets the authority of a URI to blank such that it does not
misinterpret slashes in the path as the authority.

Closes #34433
2019-04-04 12:59:59 -05:00
Gordon Brown 5347dec55e
Allow ILM to stop if indices have nonexistent policies (#40820)
Prior to this PR, there is a bug in ILM which does not allow ILM to stop
if one or more indices have an index.lifecycle.name which refers to
a policy that does not exist - the operation_mode will be stuck as
STOPPING until either the policy is created or the nonexistent
policy is removed from those indices.

This change allows ILM to stop in this case and makes the logging more
clear as to why ILM is not stopping.
2019-04-04 11:46:21 -06:00
Przemyslaw Gomulka 65d25186d3
Bat scripts to work with JAVA_HOME with parentheses Backports (#40832)
backports :
Bat scripts to work with JAVA_HOME with parentheses (#39712)
Link to SYSTEM_JAVA_HOME on windows (#40806)
2019-04-04 18:42:05 +02:00
Michael Basnight fb5a0652a8 HLRC: Convert xpack methods to client side objects (#40705)
This commit fixes a problem with BWC that was brought up in #40511. A
newer version of the code was emitting a new value for an enum to an
older version, and the older version could not handle that. It caused
the response to error. The MainResponse is now relaxed, and will accept
whatever values the server expose, and holds most of them as Strings
instead of complex objects.

Fixes #40511
2019-04-04 11:06:44 -05:00
Nik Everett 9e8499d20b Docs: Pin two IDs in the rest client (#40785)
We generate two pages with "funny" names:
* _changing_the_client_8217_s_initialization_code.html
* _changing_the_application_8217_s_code.html

The leading `_` comes from us not specifying the name of the page. The
`8217` comes about because of the single quote character. This is a
funny name, but it is the name that we have so we shouldn't change it
without putting in a redirect.

We're looking at switching these docs from being built with the
no-longer-maintained AsciiDoc project to being built with the
actively-maintained Asciidoctor project. Asciidoctor Doesn't include the
`8217`s in the generated ids. That is *better*, but we don't really want
to change the pages. Ultimately we'd prefer none of our pages start with
`_`, but that is a problem for a different time.

Anyway, this pins the ids to their "funny" id so it won't change when we
switch to Asciidoctor. We'll remove it later, when we have more fine
control of our redirects.
2019-04-04 12:03:36 -04:00
James Rodewig b7497bf2df [DOCS] Remind users to include @ symbol when applying license file (#40688) 2019-04-04 11:50:20 -04:00
Colin Goodheart-Smithe 402f312c5e
Adds version 6.7.2 2019-04-04 16:35:39 +01:00
Taylor Scott 38f4dda2be Fix grammatical error in analysis.asciidoc (#40827) 2019-04-04 17:33:02 +02:00
Jack Conradson a042b22d24 Move PainlessExecuteAction into action package (#40752) 2019-04-04 08:17:45 -07:00
Adrien Grand 7c27e5f243 Revert "Mute failing test"
This reverts commit 1af2b2bfe6.
2019-04-04 15:55:09 +02:00
Nik Everett fce1bbc20e Docs: Drop inline callouts from painless book (#40805)
Drops the inline callouts from the painless reference book. These
callouts are incompatible with Asciidoctor and we'd very much like to
switch to Asciidoctor for building this book, partially because
Asciidoctor is actively developed and AsciiDoc is not, and partially
because it builds the book three times faster.
2019-04-04 09:53:56 -04:00
Nhat Nguyen 2756a3936b Reject illegal flush parameters (#40213)
This change rejects an illegal combination of flush parameters where
force is true, but wait_if_ongoing is false. This combination is trappy
and should be forbidden.

Closes #36342
2019-04-04 09:02:31 -04:00
Nhat Nguyen c4960ad736 Ensure flush happen before closing an index (#40184)
If there's an ongoing flush triggered by the translog flush threshold,
we may fail to execute a flush because waitIfOngoing is false by
default.

Relates to #36342
2019-04-04 09:02:31 -04:00
Nhat Nguyen e716b9ceee Ensure no scheduled refresh in testPendingRefreshWithIntervalChange
If a refresh, which is scheduled by the setting change, executes after
the index-2 operation and win the refresh race (i.e., maybeRefresh) with
the scheduledRefresh that we are going to check, then the latter will
return false.

Closes #39565
Relates #39462

PR #40387
2019-04-04 09:02:31 -04:00
James Rodewig 11f2da9d5e [DOCS] Rewrite query def (#40746) 2019-04-04 08:57:39 -04:00
James Rodewig 7c669026ac
[DOCS] Document index.load_fixed_bitset_filters_eagerly (#40780) (#40842) 2019-04-04 08:48:37 -04:00
Alpar Torok 1af2b2bfe6 Mute failing test
Tracked in #40838
2019-04-04 14:54:21 +03:00
Tim Vernum 9d34164ce6
Add build utility to check cluster health over ssl (#40713)
By default, in integ tests we wait for the standalone cluster to start
by using the ant Get task to retrieve the cluster health endpoint.
However the ant task has no facilities for customising the trusted
CAs for a https resource, so if the integ test cluster has TLS enabled
on the http interface (using a custom CA) we need a separate utility
for that purpose.

Backport of: #40573
2019-04-04 21:44:03 +11:00
David Roberts d16f86f7ab [ML] Add created_by info to usage stats (#40518)
This change adds information about which UI path
(if any) created ML anomaly detector jobs to the
stats returned by the _xpack/usage endpoint.

Counts for the following possibilities are expected:

* ml_module_apache_access
* ml_module_apm_transaction
* ml_module_auditbeat_process_docker
* ml_module_auditbeat_process_hosts
* ml_module_nginx_access
* ml_module_sample
* multi_metric_wizard
* population_wizard
* single_metric_wizard
* unknown

The "unknown" count is for jobs that do not have a
created_by setting in their custom_settings.

Closes #38403
2019-04-04 10:55:20 +01:00
Marios Trivyzas c8047c0644 SQL: [Docs] Small fixes for CURRENT_TIMESTAMP docs (#40792)
- Added square brackets for the optional argument of precision
- Fixed character to lower case after comma

(cherry picked from commit d2f6f3b9ce36875e2eb6145c50464b4d72f2b1df)
2019-04-04 11:45:20 +02:00
Marios Trivyzas f3c207d27b SQL: Implement CURRENT_TIME/CURTIME functions (#40662)
After `TIME` SQL data type is introduced, implement
`CURRENT_TIME/CURTIME` functions similarly to CURRENT_TIMESTAMP
that return the system's current time (only, without the date part).

Closes: #40468
(cherry picked from commit 9feede781409d0e264ce45951a25b28ff129b187)
2019-04-04 11:45:20 +02:00