Commit Graph

41065 Commits

Author SHA1 Message Date
JeffSaxeVA efdad7d5fc [DOCS] Add "remove a tag" script logic as an example (#32556)
It took me quite a while of online searching and experimenting to realize the function-call asymmetry in the Add versus Remove from a list, like the "tags" list! I realize we cannot give examples for every single thing the user wants to do in Painless, but this is such a common use case (removing a tag from a single doc, or from a set of docs with Update-By-Query) that I believe it ought to be demonstrated immediately after the "add a tag" example. We have an example of removing an entire document field, but not removing one element of a list (a multi-valued field).

Also, a minor grammar fix: I have added an apostrophe to the word "its" in the accompanying text of the example just above.
2018-08-17 10:56:06 +02:00
Alpar Torok 148a76f0c7 Fix failing BuildExamplePluginsIT test 2018-08-17 11:14:58 +03:00
Alpar Torok d16562eab5
RFC: Test that example plugins build stand-alone (#32235)
Add tests for build-tools to make sure example plugins build stand-alone using it.

This will catch issues such as referencing files from the buildSrc directly, breaking external uses of build-tools.
2018-08-17 09:41:39 +03:00
Jay Modi 1136a95837
Security: remove put privilege API (#32879)
This commit removes the put privilege API in favor of having a single API to
create and update privileges. If we see the need to have an API like this in
the future we can always add it back.
2018-08-16 21:16:06 -06:00
Julie Tibshirani cbf160a4e6
For filters aggs, make sure that rewrites preserve other_bucket. (#32921) 2018-08-16 17:36:58 -07:00
Jason Tedor 0876630b30
Guard against null in email admin watches (#32923)
The Kibana settings docs that these watches rely on can sometimes
contain no xpack settings. When this is the case, we will end up with a
null pointer exception in the script. We need to guard against in these
scripts so this commit does that.
2018-08-16 16:25:34 -04:00
Ed Savage 62559d2b3c
Re enable ml bwc tests (#32916)
[ML] Re-enabling BWC tests

Re-enable BWC tests for ML now that #32816 has been backported to 6.x
2018-08-16 18:23:26 +01:00
Ed Savage d604b3e3a1 Temporarily disabled ML BWC tests for backporting
https://github.com/elastic/elasticsearch/pull/32816
2018-08-16 17:18:51 +01:00
Jason Tedor b5a85362e3
Fix docs for fixed filename for heap dump path (#32882)
The docs here incorrectly state that it is okay for a heap dump file to
exist when heap dump path is configured to a fixed filename. This is
incorrect, the JVM will fail to write the heap dump if a heap dump file
already exists at the specified location (see the DumpWriter constructor
DumpWriter::DumpWriter(const char* path) in the JVM source).
2018-08-16 11:32:35 -04:00
Jack Conradson d9fd74bcdc
Painless: Special Case def (#32871)
This removes def from the classes map in PainlessLookup and instead always special 
cases it. This prevents potential calls against the def type that shouldn't be made and 
forces all cases of def throughout Painless code to be special cased.
2018-08-16 08:03:21 -07:00
Adrien Grand eaaf37a1f9 AwaitFix FullClusterRestartIT#testRollupIDSchemeAfterRestart. 2018-08-16 16:35:10 +02:00
Jim Ferenczi 3dd1677cdc [Test] Fix DuelScrollIT#testDuelIndexOrderQueryThenFetch
This commit disables the automatic `refresh_interval` in order to ensure
that index readers cannot differ between the normal and scroll search.
This issue is related to the 7.5 Lucene upgrade which contains a change that
makes single segment merge more likely to occur (max deletes percentage).

Closes #32682
2018-08-16 15:33:17 +02:00
Benjamin Trent aedc2c1c49
HLRC: adding machine learning delete job (#32820)
* HLRC: adding machine learning delete job

* Fixing whitespace

* Moving docs and tests around

* Unifying ml asciidoc file naming convention
2018-08-16 07:18:43 -05:00
Hazem Khaled b87f3062b7 [DOCS] Update WordPress plugins links (#32194) 2018-08-16 13:55:40 +02:00
Jason Tedor f8c7414ee8
Remove passphrase support from reload settings API (#32889)
We do not support passphrases on the secure settings storage (the
keystore). Yet, we added support for this in the API layer. This commit
removes this support so that we are not limited in our future options,
or have to make a breaking change.
2018-08-16 07:24:05 -04:00
Adrien Grand e35be01901 AwaitFix AckIT.
Relates #32767
2018-08-16 12:31:58 +02:00
Colin Goodheart-Smithe d80457ee2a
Mutes test in DuelScrollIT
Due to https://github.com/elastic/elasticsearch/issues/32682
2018-08-16 11:08:00 +01:00
Adrien Grand 039babddf5 CharArraysTests: Fix test bug. 2018-08-16 11:54:39 +02:00
David Kyle 7f6802cb51 [ML] Choose seconds to fix intermittent DatafeeedConfigTest failure 2018-08-16 10:48:56 +01:00
Jim Ferenczi 996ed73d73 Test: Fix unpredictive merges in DocumentSubsetReaderTests
The merge policy that was used could lead to unpredictive merges due to the
randomization of `setDeletesPctAllowed`.

Closes #32457
2018-08-16 11:44:57 +02:00
datosh e6bfba1d79 [DOCS] Clarify sentence in network-host.asciidoc (#32429) 2018-08-16 11:38:19 +02:00
markharwood 70d80a3d09
Docs enhancement: added reference to cluster-level setting `search.default_allow_partial_results` (#32810)
Closes #32809
2018-08-16 10:21:37 +01:00
debadair 93ada98eb5
[DOCS] Fixing cross doc link to Stack Overview security topic. 2018-08-15 15:09:41 -07:00
Jay Modi 1a45b27d8b
Move CharArrays to core lib (#32851)
This change cleans up some methods in the CharArrays class from x-pack, which
includes the unification of char[] to utf8 and utf8 to char[] conversions that
intentionally do not use strings. There was previously an implementation in
x-pack and in the reloading of secure settings. The method from the reloading
of secure settings was adopted as it handled more scenarios related to the
backing byte and char buffers that were used to perform the conversions. The
cleaned up class is moved into libs/core to allow it to be used by requests
that will be migrated to the high level rest client.

Relates #32332
2018-08-15 15:26:00 -06:00
Jason Tedor 4475f88c95
Merge branch 'master' into ccr
* master:
  Fix global checkpoint listeners test
  HLRC: adding machine learning open job (#32860)
  [ML] Add log structure finder functionality (#32788)
  INGEST: Add Configuration Except. Data to Metdata (#32322)
2018-08-15 16:07:28 -04:00
Jason Tedor 364ccc36d6
Fix global checkpoint listeners test
This commit fixes a global checkpoint listeners test wherein we were
expecting an executor to have been used even if there were no
listeners. This is silliness, so this commit adjusts the assertion to
verify that the executor never fires if there are no listeners, and
fires exactly once if there is one or more listeners.
2018-08-15 15:53:15 -04:00
Benjamin Trent f90f06ede1
HLRC: adding machine learning open job (#32860)
* HLRC: adding machine learning open job

HLRC: adding ML open job API call

* Addressing @droberts195 comments and fixing minor style issue
2018-08-15 13:19:41 -05:00
David Roberts 5ba04e23fc
[ML] Add log structure finder functionality (#32788)
This change adds a library to ML that can be used to deduce a log
file's structure given only a sample of the log file.

Eventually this will be used to add an endpoint to ML to make the
functionality available to end users, but this will follow in a
separate change.

The functionality is split into a library so that it can also be
used by a command line tool without requiring the command line
tool to include all server code.
2018-08-15 18:04:21 +01:00
Armin Braun 986c55b830
INGEST: Add Configuration Except. Data to Metdata (#32322)
* closes #27728
2018-08-15 19:02:19 +02:00
Jason Tedor aa147cca44
Merge remote-tracking branch 'elastic/master' into ccr
* elastic/master:
  Revert "cluster formation DSL - Gradle integration -  part 2 (#32028)" (#32876)
  cluster formation DSL - Gradle integration -  part 2 (#32028)
  Introduce global checkpoint listeners (#32696)
  Move connection profile into connection manager (#32858)
  [ML] Temporarily disabling rolling-upgrade tests
  Use generic AcknowledgedResponse instead of extended classes (#32859)
  [ML] Removing old per-partition normalization code (#32816)
  Use JDK 10 for 6.4 BWC builds (#32866)
  Removed flaky test. Looks like randomisation makes these assertions unreliable.
  [test] mute IndexShardTests.testDocStats
  Introduce the dissect library (#32297)
  Security: remove password hash bootstrap check (#32440)
  Move validation to server for put user requests (#32471)
  [ML] Add high level REST client docs for ML put job endpoint (#32843)
  Test: Fix forbidden uses in test framework (#32824)
  Painless: Change fqn_only to no_import (#32817)
  [test] mute testSearchWithSignificantTermsAgg
  Watcher: Remove unused hipchat render method (#32211)
  Watcher: Remove extraneous auth classes (#32300)
  Watcher: migrate PagerDuty v1 events API to v2 API (#32285)
2018-08-15 12:30:35 -04:00
Alpar Torok bb3eacf384
Revert "cluster formation DSL - Gradle integration - part 2 (#32028)" (#32876)
This reverts commit e680a64434.
2018-08-15 19:17:46 +03:00
Alpar Torok e680a64434
cluster formation DSL - Gradle integration - part 2 (#32028)
* Implement Version in java

- This allows to move all  all .java files from .groovy.
- Will prevent eclipse from tangling up in this setup
- make it possible to use Version from Java

* PR review comments

* Cluster formation plugin with reference counting

```
> Task :plugins:ingest-user-agent:listElasticSearchClusters
Starting cluster: myTestCluster
   * myTestCluster: /home/alpar/work/elastic/elasticsearch/plugins/ingest-user-agent/foo
Asked to unClaimAndStop myTestCluster, since cluster still has 1 claim it will not be stopped

> Task :plugins:ingest-user-agent:testme UP-TO-DATE
Stopping myTestCluster, since no of claims is 0
```

- Meant to auto manage the clusters lifecycle
- Add integration test for cluster formation

* Fix rebase

* Change to `useCluster` method on task
2018-08-15 19:16:10 +03:00
Jason Tedor 068d03f56b
Introduce global checkpoint listeners (#32696)
This commit introduces the ability for global checkpoint listeners to be
registered at the shard level. These listeners are notified when the
global checkpoint is updated, and also when the shard closes. To
encapsulate these listeners, we introduce a shard-level component that
handles synchronization of notification and modifications to the
collection of listeners.
2018-08-15 12:04:24 -04:00
Tim Brooks 2464b68613
Move connection profile into connection manager (#32858)
This is related to #31835. It moves the default connection profile into
the ConnectionManager class. The will allow us to have different
connection managers with different profiles.
2018-08-15 09:08:33 -06:00
Ed Savage 51cece1900 [ML] Temporarily disabling rolling-upgrade tests
BWC tests disabled while backporting https://github.com/elastic/elasticsearch/pull/32816
2018-08-15 15:48:50 +01:00
Lee Hinman 48281ac5bc
Use generic AcknowledgedResponse instead of extended classes (#32859)
This removes custom Response classes that extend `AcknowledgedResponse` and do nothing, these classes are not needed and we can directly use the non-abstract super-class instead.

While this appears to be a large PR, no code has actually changed, only class names have been changed and entire classes removed.
2018-08-15 08:06:14 -06:00
Ed Savage 8ce1ab3ed9
[ML] Removing old per-partition normalization code (#32816)
[ML] Removing old per-partition normalization code

Per-partition normalization is an old, undocumented feature that was
never used by clients. It has been superseded by per-partition maximum
scoring.

To maintain communication compatibility with nodes prior to 6.5 it is
necessary to maintain/cope with the old wire format
2018-08-15 13:13:32 +01:00
Jason Tedor 7ff37ae232
Use JDK 10 for 6.4 BWC builds (#32866)
The 6.4 release will officially be built with JDK 10. We should push JDK
10 down to the BWC builds for the 6.4 branch then too.
2018-08-15 08:01:15 -04:00
markharwood eb812c8b7d Removed flaky test. Looks like randomisation makes these assertions unreliable.
This test is superfluous - it was added to address #32770 but it later turned out there was an existing test that just required a fix to provide the missing test coverage.

Closes #32855
2018-08-15 10:06:39 +01:00
Andy Bristol a1cff86012 [test] mute IndexShardTests.testDocStats
For #32766
2018-08-14 18:21:59 -07:00
Jake Landis be62092060
Introduce the dissect library (#32297)
The dissect library will be used for the ingest node as an alternative
to Grok to split a string based on a pattern. Dissect differs from
Grok such that regular expressions are not used to split the string.
Note - Regular expressions are used during construction of the
objects, but not in the hot path.

A dissect pattern takes the form of: '%{a} %{b},%{c}' which is
composed of 3 keys (a,b,c) and two delimiters (space and comma).
This dissect pattern will match a string of the form: 'foo bar,baz'
and will result a key/value pairing of 'a=foo, b=bar, and c=baz'.
See the comments in DissectParser for a full explanation.

This commit does not include the ingest node processor that will consume
it. However, the consumption should be a trivial mapping between the
key/value pairing returned by the parser and the key/value pairing
needed for the IngestDocument.
2018-08-14 17:08:55 -07:00
Jay Modi ac5ef8c389
Security: remove password hash bootstrap check (#32440)
This change removes the PasswordHashingBootstrapCheck and replaces it
with validation on the setting itself. This ensures we always get a
valid value from the setting when it is used.
2018-08-14 13:34:22 -06:00
Jay Modi 7d8a64d703
Move validation to server for put user requests (#32471)
This change moves the validation for values of usernames and passwords
from the request to the transport action. This is done to prevent
the need to move more classes into protocol once we add this API to the
high level rest client. Additionally, this resolves an issue where
validation depends on settings and we always pass empty settings
instead of the actual settings.

Relates #32332
2018-08-14 13:18:07 -06:00
David Roberts c985f500f4
[ML] Add high level REST client docs for ML put job endpoint (#32843)
Relates #29827
Relates #32726
2018-08-14 19:52:54 +01:00
Ryan Ernst 0158b59a5a
Test: Fix forbidden uses in test framework (#32824)
This commit fixes existing uses of forbidden apis in the test framework
and re-enables the forbidden apis check. It was previously completely
disabled and had missed a rename of the forbidden apis signatures files.

closes #32772
2018-08-14 11:35:09 -07:00
Jack Conradson ab7e7508f5
Painless: Change fqn_only to no_import (#32817)
This changes the whitelist parameter fqn_only to no_import when specifying that a 
whitelisted class must have the fully-qualified-name instead of a shortcut name. This more 
closely correlates with Java imports, hence the rename.
2018-08-14 11:17:15 -07:00
Andy Bristol 84aa87f638 [test] mute testSearchWithSignificantTermsAgg
For #32855
2018-08-14 10:59:44 -07:00
Nhat Nguyen 6556186d9a Merge branch 'master' into ccr 2018-08-14 12:11:35 -04:00
Michael Basnight 6fde9e5c2f
Watcher: Remove unused hipchat render method (#32211)
The HipChatMessage#render is no longer used, and instead the
HipChatAccount#render is used in the ExecutableHipChatAction. Only a
test that validated the HttpProxy used this render method still. This
commit cleans it up.
2018-08-14 11:07:58 -05:00
Michael Basnight e1206a6272
Watcher: Remove extraneous auth classes (#32300)
The auth.basic package was an example of a single implementation
interface that leaked into many different classes. In order to clean
this up, the HttpAuth interface, factories, and Registries all were
removed and the single implementation, BasicAuth, was substituted in all
cases. This removes some dependenies between Auth and the Templates,
which can now use static methods on BasicAuth. BasicAuth was also moved
into the http package and all of the other classes were removed.
2018-08-14 11:06:46 -05:00