Commit Graph

39205 Commits

Author SHA1 Message Date
Nhat Nguyen 29f647e6f6 Mute testCorruptFileThenSnapshotAndRestore
Tracked at #30577
2018-05-19 08:28:15 -04:00
Ryan Ernst 34180f2285
Scripting: Remove getDate methods from ScriptDocValues (#30690)
The getDate() and getDates() existed prior to 5.x on long fields in
scripting. In 5.x, a new Date type for ScriptDocValues was added. The
getDate() and getDates() methods were left on long fields and added to date
fields to ease the transition. This commit removes those methods for
7.0.
2018-05-18 21:26:26 -07:00
Nhat Nguyen 67d8fc222d
Upgrade to Lucene-7.4.0-snapshot-59f2b7aec2 (#30726)
This snapshot resolves issues related to ShrinkIndexIT.
2018-05-18 18:21:39 -04:00
Christoph Büscher 994405a768
[Docs] Fix single page :docs:check invocation (#30725)
The docs/README shows how to run the :docs:check goal on a subset of pages,
however using the current example fails. Removing the masking character before
the first wildcard fixes the problem.
2018-05-18 23:33:41 +02:00
William Dearden 13c56ae444 Docs: Add uptasticsearch to list of clients (#30738)
It is a client for r.
2018-05-18 17:13:12 -04:00
lcawl 8180e4431f [DOCS] Removes out-dated x-pack/docs/en/index.asciidoc 2018-05-18 11:31:25 -07:00
Lisa Cawley 6846d2c94a
[DOCS] Removes redundant index.asciidoc files (#30707) 2018-05-18 11:05:40 -07:00
David Roberts 2b72adc8ac
[TEST] Reduce forecast overflow to disk test memory limit (#30727)
By default ML native processes are only allowed to use
30% of RAM, so the previous 2GB setting prevented the
test passing on VMs with only 4GB RAM.  This change
reduces the limit to 1200MB, which means it can now
pass on VMs with 4GB RAM.
2018-05-18 19:01:43 +01:00
Ryan Ernst b3f3a4312b
Plugins: Remove meta plugins (#30670)
Meta plugins existed only for a short time, in order to enable breaking
up x-pack into multiple plugins. However, now that x-pack is no longer
installed as a plugin, the need for them has disappeared. This commit
removes the meta plugins infrastructure.
2018-05-18 10:56:08 -07:00
Lisa Cawley e750462e0c
[DOCS] Moves X-Pack configurationg pages in table of contents (#30702) 2018-05-18 10:26:03 -07:00
Nhat Nguyen 95f52f07ce TEST: Add engine log to testCorruptFileThenSnapshotAndRestore
Relates #30577
2018-05-18 11:08:25 -04:00
Dimitris Athanasiou 6bb2a1da22
[ML][TEST] Fix bucket count assertion in ModelPlotsIT (#30717)
As the first record is random, there's a chance it will
be aligned on a bucket start. Thus we need to check the
bucket count is in [23, 24].

Closes #30715
2018-05-18 17:59:01 +03:00
Dimitris Athanasiou 1484a31be5
[ML][TEST] Make AutodetectMemoryLimitIT less fragile (#30716)
These tests aim to check the set model memory limit is
respected. Additionally, it was asserting counts of
partition, by, over fields in an attempt to check that
the used memory is spent meaningfully. However, this
made the tests fragile, as changes in the ml-cpp could
lead to CI failures.

This commit removes those assertions. We are working on
adding tests in ml-cpp that will compensate.
2018-05-18 17:57:20 +03:00
Jason Tedor d68c44b76c
Default copy settings to true and deprecate on the REST layer (#30598)
This commit defaults the copy_settings REST parameter to the shrink and
split APIs to true, and deprecates the parameter.
2018-05-18 10:12:08 -04:00
Martijn van Groningen 33cba44d0a
[Build] Add test admin when starting gradle run with trial license and
also add more documentation around gradle run task. (#30671)
2018-05-18 14:11:11 +02:00
Hendrik Muhs 6c313a9871 This implementation lazily (on 1st forecast request) checks for available
diskspace and creates a subfolder for storing data outside of Lucene
indexes, but as part of the ES data paths.

Details:
 - tmp storage is managed and does not allow allocation if disk space is
   below a threshold (5GB at the moment)
 - tmp storage is supposed to be managed by the native component but in
   case this fails cleanup is provided:
    - on job close
    - on process crash
    - after node crash, on restart
 - available space is re-checked for every forecast call (the native
   component has to check again before writing)

Note: The 1st path that has enough space is chosen on job open (job
close/reopen triggers a new search)
2018-05-18 14:04:09 +02:00
Alexander Reelsen b5a793b569
Tests: Fail if test watches could not be triggered (#30392)
Watcher tests now always fail hard when watches that were 
tried to be triggered in a test using the trigger() method, 
but could not because they were not found on any of the 
nodes in the cluster.
2018-05-18 08:51:43 +02:00
Hendrik Muhs d893041634
[ML] add version information in case of crash of native ML process (#30674)
This change adds version information in case a native ML process crashes, the version is important for choosing the right symbol files when analyzing the crash. Adding the version combines all necessary information on one line.

relates elastic/ml-cpp#94
2018-05-18 07:46:52 +02:00
Jason Tedor 443c7014ba
Make TransportClusterStateAction abide to our style (#30697)
I still do not like == false. However, I am so use to reading it that
today I read this line of code and could not understand how it could
possibly be doing the right thing. It was only when I finally noticed
the ! that the code made sense. This commit changes this code to be in
our style of == false. I still do not like == false.
2018-05-17 21:17:24 -04:00
tomcallahan 9be3fbd1b2
Change required version for Get Settings transport API changes to 6.4.0 (#30706)
Get Settings API changes have now been backported to version 6.4, and
therefore the latest version must send and expect the extra fields when
communicating with 6.4+ code.

Relates #29229 #30494
2018-05-17 20:29:08 -04:00
Lisa Cawley a8b4a98ee5
[DOCS] Fixes edit URLs for stack overview (#30583) 2018-05-17 14:20:03 -07:00
Ryan Ernst 30e0f33d16 Silence sleep based watcher test
see https://github.com/elastic/elasticsearch/issues/30699
2018-05-17 13:04:10 -07:00
Zachary Tong d120fb222c [TEST] Adjust version skips for movavg/movfn tests
Since the MovFn PR was backported to 6.x, we can adjust
the version skip numbers in master to correctly
match 6.3.99 instead of 6.4.0
2018-05-17 18:07:52 +00:00
lcawl 663295d635 [DOCS] Replace X-Pack terms with attributes 2018-05-17 09:57:11 -07:00
Dimitris Athanasiou 75665a2d3e
[ML] Clean left behind model state docs (#30659)
It is possible for state documents to be
left behind in the state index. This may be
because of bugs or uncontrollable scenarios.
In any case, those documents may take up quite
some disk space when they add up. This commit
adds a step in the expired data deletion that
is part of the daily maintenance service. The
new step searches for state documents that
do not belong to any of the current jobs and
deletes them.

Closes #30551
2018-05-17 17:51:26 +03:00
Mayya Sharipova 7c2fc26011 Correct typos
Relates to #28725
2018-05-17 10:34:42 -04:00
Piotr Prądzyński a0a8c4f186 filters agg docs duplicated 'bucket' word removal (#30677)
In one place word 'bucket' was duplicated.
2018-05-17 15:21:50 +01:00
Piotr Prądzyński cefbd29db3 top_hits doc example description update (#30676)
Example description does not fit example code.
2018-05-17 15:21:25 +01:00
Christoph Büscher 712473b558
[Docs] Replace InetSocketTransportAddress with TransportAdress (#30673)
The former class has been removed in 6.0, the documentation code
snippets should be updated accordingly.
2018-05-17 14:23:08 +02:00
David Roberts ef0daee850
[TEST] Account for increase in ML C++ memory usage (#30675)
Recent changes to the ML C++ have resulted in higher memory usage,
so fewer "by" fields can be analyzed in a given amount of model
memory.
2018-05-17 12:59:20 +01:00
Yannick Welsch b57d21bab1
User proper write-once semantics for GCS repository (#30438)
There's no need for an extra blobExists() call when writing a blob to the GCS service. GCS provides
an option (with stronger consistency guarantees) on the insert method that guarantees that the
blob that's uploaded does not already exist.

Relates to #19749
2018-05-17 13:58:10 +02:00
Christoph Büscher 399489b242 Remove bogus file accidentally added 2018-05-17 13:56:10 +02:00
Tim Vernum 9f824c4aa8
Add detailed assert message to IndexAuditUpgradeIT (#30669)
Print out the returned buckets if the size does not match the
expectation.
2018-05-17 21:36:13 +10:00
Ioannis Kakavas 35fa934971
Adjust fast forward for token expiration test (#30668)
Adjust fast forward for token expiration test

Adjusts the maximum fast forward time for token expiration tests
to be 5 seconds before actual token expiration so that the test
won't fail even when upperlimit is randomly selected.

Resolves: #30062
2018-05-17 14:10:49 +03:00
Mayya Sharipova 3dfa93ef7c
Improve explanation in rescore (#30629)
Currently in a rescore request if window_size is smaller than
the top N documents returned (N=size), explanation of scores could be incorrect
for documents that were a part of topN and not part of rescoring.
This PR corrects this, but saving in RescoreContext docIDs of documents
for which rescoring was applied, and adding rescoring explanation
only for these docIDs.

Closes #28725
2018-05-17 07:09:18 -04:00
Christoph Büscher b6340658f4
Deprecate `nGram` and `edgeNGram` names for ngram filters (#30209)
The camel case name `nGram` should be removed in favour of `ngram` and
similar for `edgeNGram` and `edge_ngram`. Before removal, we need to
deprecate the camel case names first. This change adds deprecation
warnings for indices with versions 6.4.0 and higher and logs deprecation
warnings.
2018-05-17 12:52:22 +02:00
Alexander Reelsen 11d776ecf0
Watcher: Fix watch history template for dynamic slack attachments (#30172)
The part of the history template responsible for slack attachments had a
dynamic mapping configured which could lead to problems, when a string
value looking like a date was configured in the value field of an
attachment.

This commit fixes the template by setting this field always to text.
This also requires a change in the template numbering to be sure this
will be applied properly when starting watcher.
2018-05-17 11:57:54 +02:00
Tanguy Leroux 2ac1f9fe89
Fix _cluster/state to always return cluster_uuid (#30656)
Since #30143, the Cluster State API should always returns the current
cluster_uuid in the response body, regardless of the metrics filters.

This is not exactly true as it is returned only if metadata metrics and
no specific indices are requested.

This commit fixes the behavior to always return the cluster_uuid and
add new test.
2018-05-17 10:58:25 +02:00
Tanguy Leroux 7915b5f7aa [Tests] Add debug information to CorruptedFileIT
This test failed but the cause is not obvious. This commit adds more
debug logging traces so that if it reproduces we could gather more
information.

Related #30577
2018-05-17 10:57:25 +02:00
Tim Vernum 8a89306aaf
Preserve REST client auth despite 401 response (#30558)
The default behaviour for Apache HTTP client is to mimic the standard
browser behaviour of clearing the authentication cache (for a given
host) if that host responds with 401.
This behaviour is appropriate in a interactive browser environment
where the user is given the opportunity to provide alternative
credentials, but it is not the preferred behaviour for the ES REST
client.

X-Pack may respond with a 401 status if a request is made before the
node/cluster has recovered sufficient state to know how to handle the
provided authentication credentials - for example the security index
need to be recovered before we can authenticate native users.
In these cases the correct behaviour is to retry with the same
credentials (rather than discarding those credentials).
2018-05-17 18:27:18 +10:00
Andy Bristol 890afadb37
[test] packaging: add windows boxes (#30402)
Adds windows server 2012r2 and 2016 vagrant boxes to packaging tests.
They can only be used if IDs for their images are specified, which are
passed to gradle and then to vagrant via env variables. Adds options
to the project property `vagrant.boxes` to choose between linux and
windows boxes.

Bats tests are run only on linux boxes, and portable packaging tests run
on all boxes. Platform tests are only run on linux boxes since they are
not being maintained.

For #26741
2018-05-16 16:42:08 -07:00
Ryan Ernst a4c9c2fa2a
Make xpack modules instead of a meta plugin (#30589)
This commit removes xpack from being a meta-plugin-as-a-module.
It also fixes a couple tests which were missing task dependencies, which
failed once the gradle execution order changed.
2018-05-16 15:35:57 -07:00
Nhat Nguyen e0ccd4b816 Mute ShrinkIndexIT
This is tracked at https://issues.apache.org/jira/browse/LUCENE-8318
2018-05-16 16:44:44 -04:00
Dimitris Athanasiou 01bdfcde6f
[ML] DeleteExpiredDataAction should use client with origin (#30646)
This is an admin action that should be allowed to operate on
ML indices with full permissions.
2018-05-16 23:35:23 +03:00
Shashwat Anand f0da3da6b0 Reindex: Fixed typo in assertion failure message (#30619)
Fix a typo in an assertion failure message.
2018-05-16 16:26:23 -04:00
lcawl 8ff9baeb82 [DOCS] Fixes list of unconverted snippets in build.gradle 2018-05-16 13:11:06 -07:00
lcawl 4f41018753 [DOCS] Reorganizes RBAC documentation 2018-05-16 12:22:52 -07:00
Igor Motov 7b6d0faec2
SQL: Remove dependency for server's version from JDBC driver (#30631)
Removes dependency for server's version from the JDBC driver code. This
should allow us to dramatically reduce driver's size by removing the
server dependency from the driver.

Relates #29856
2018-05-16 15:07:14 -04:00
jaymode d912fa4357
Test: increase search logging for LicensingTests
This commit increases the logging level around search to aid in
debugging failures in LicensingTests#testSecurityActionsByLicenseType
where we are seeing all shards failed error while trying to search the
security index.

See #30301
2018-05-16 11:59:49 -06:00
Lee Hinman a89073015d Adjust serialization version in IndicesOptions
The PR for the new format of serialization was backported, so this needs to be
adjusted since it can now speak to 6.4 nodes in the new way.
2018-05-16 10:19:05 -06:00