Commit Graph

41068 Commits

Author SHA1 Message Date
Nik Everett ebd5eb6dc2 ML: Fix build after HLRC change
I recently merged a HLRC change that passed the PR builds but didn't
compile after merging. Sad time. This fixes the compilation.
2018-09-04 11:10:44 -04:00
Jim Ferenczi dbc7102c86
Fix inner hits retrieval when stored fields are disabled (_none_) (#33018)
Now that types are unique per mapping we can retrieve the document mapper
without referencing the type. This fixes an NPE when stored fields are disabled.
For 6x we'll need a different fix since mappings can still have multiple types.

Relates #32941
2018-09-04 16:25:52 +02:00
Costin Leau 17c7f99343
SQL: Show/desc commands now support table ids (#33363)
Extend SHOW TABLES, DESCRIBE and SHOW COLUMNS to support table
identifiers not just SQL LIKE pattern.
This allows both Elasticsearch-style multi-index patterns and SQL LIKE.
To disambiguate between the two (as the " vs ' can be easy to miss),
the grammar now requires LIKE keyword as a prefix for all LIKE-like
patterns.

Also added some docs comparing the two types of patterns.

Fix #33294
2018-09-04 16:54:10 +03:00
Nhat Nguyen 16b53b5ab5 Mute testValidateFollowingIndexSettings
Tracked at #33379
2018-09-04 09:03:26 -04:00
Sohaib Iftikhar 761e8c461f HLRC: Add delete by query API (#32782)
Adds the delete-by-query API to the High Level REST Client.
2018-09-04 08:56:26 -04:00
Dimitris Athanasiou 1457b07a06
[ML] The sort field on get records should default to the record_score (#33358)
This is not changing the behaviour as when the sort field was set
to `influencer_score` the secondary sort would be used and that
was using the `record_score` at the highest priority.
2018-09-04 11:38:24 +01:00
David Roberts 84eaac79d7
[ML] Minor improvements to categorization Grok pattern creation (#33353)
1. The TOMCAT_DATESTAMP format needs to be checked before
   TIMESTAMP_ISO8601, otherwise TIMESTAMP_ISO8601 will
   match the start of the Tomcat datestamp.
2. Exclude more characters before and after numbers.  For
   example, in 1.2.3 we don't want to match 1.2 as a float.
2018-09-04 09:43:49 +01:00
Nikolay Vasiliev d9f394b099 [DOCS] fix a couple of typos (#33356) 2018-09-04 10:07:11 +02:00
Alpar Torok 7f7e8fd733
Disable assemble task instead of removing it (#33348) 2018-09-04 07:32:14 +03:00
Julie Tibshirani 78df00ff24
Simplify the return type of FieldMapper#parse. (#32654) 2018-09-04 01:15:19 +00:00
Benjamin Trent 767d8e0801
[ML] Delete forecast API (#31134) (#33218)
* Delete forecast API (#31134)
2018-09-03 19:06:18 -05:00
Jason Tedor 09bf4e5f00
Introduce private settings (#33327)
This commit introduces the formal notion of a private setting. This
enables us to register some settings that we had previously not
registered as fully-fledged settings to avoid them being exposed via
APIs such as the create index API. For example, we had hacks in the
codebase to allow index.version.created to be passed around inside of
settings objects, but was not registered as a setting so that if a user
tried to use the setting on any API then they would get an
exception. This prevented users from setting index.version.created on
index creation, or updating it via the index settings API. By
introducing private settings, we can continue to reject these attempts,
yet now we can represent these settings as actual settings. In this
change, we register index.version.created as an actual setting. We do
not cutover all settings that we had been treating as private in this
pull request, it is already quite large due to moving some tests around
to account for the fact that some tests need to be able to set the
index.version.created. This can be done in a follow-up change.
2018-09-03 19:17:57 -04:00
Christoph Büscher 79db16f9bb
[Docs] Add search timeout caveats (#33354)
Global search timeouts and timeouts specified in the search request body use the
same internal mechanism as search cancellation. Therefore the same caveats
apply, mostly around the responsiveness of the timeout which gets only checked
by a running search on segment boundaries by default.

Closes #31263
2018-09-03 20:56:05 +02:00
Armin Braun 1f046617bf
TESTS: Fix Race Condition in Temp Path Creation (#33352)
* TESTS: Fix Race Condition in Temp Path Creation

* Calling `createTempDir` concurrently here in
the `Follower`s causes collisions at times
which lead to `createEngine` throwing because
of unexpected files in the newly created temp
dir
   * Fixed by creating all temp dirs in the main test thread
* closes #33344
2018-09-03 19:55:59 +02:00
Nhat Nguyen 24d60c7f4b
Fix from_range in search_after in changes snapshot (#33335)
We can have multiple documents in Lucene with the same seq_no for
parent-child documents (or without rollback). In this case, the usage
"lastSeenSeqNo + 1" is an off-by-one error as it may miss some
documents. This error merely affects the `skippedOperations` contract.

See: https://github.com/elastic/elasticsearch/pull/33222#discussion_r213842257

Closes #33318
2018-09-03 11:58:49 -04:00
Armin Braun 42424aff21
TESTS+DISTR.: Fix testIndexCheckOnStartup Flake (#33349)
* Ignore all `RuntimeException` since random
file corruption triggers other RTE in addition
to the randomly caught one
* closes #33345
2018-09-03 17:06:12 +02:00
tony-dillon a9d2b1dde8 Null completion field should not throw IAE (#33268)
Ignore null value on the completion field

Closes #33200
2018-09-03 16:49:53 +02:00
Colin Goodheart-Smithe 0bf36253a9
Adds code to help with IndicesRequestCacheIT failures (#33313)
* Adds code to help with IndicesRequestCacheIT failures

Relates to #32827

* Adds comment

* Fixes test failure
2018-09-03 14:54:17 +01:00
David Kyle ccb2ad25cc
Prevent NPE parsing the stop datafeed request. (#33347)
The issue depends on the request parameters being passed in the request
body rather than as query parameters.
2018-09-03 13:35:04 +01:00
Dimitris Athanasiou 54fe7fb5a1
HLRC: Add ML get overall buckets API (#33297)
Relates #29827
2018-09-03 13:34:36 +01:00
Alexander Reelsen 246a7df8c2
Core: Fix epoch millis java time formatter (#33302)
The existing implemention could not deal with negative numbers as well
as +- 999 milliseconds around the epoch.

This commit uses Instant.ofEpochMilli() and parses the input to
a number instead of using a date formatter.
2018-09-03 13:13:19 +02:00
Christoph Büscher 978d1ed257
[Docs] Improve tuning for speed advice (#33315)
This change merges two sections in the "Tune for search speed" documentation
that recommend mapping numeric identifiers as keywords. Both sections contain
mostly the same advice, so they can be merged.

Closes #32733
2018-09-03 11:09:30 +02:00
Zachary Tong 90ce3a6224 [Rollup] Fix Caps Comparator to handle calendar/fixed time (#33336)
The comparator used TimeValue parsing, which meant it couldn't handle
calendar time.  This fixes the comparator to handle either (and potentially
mixed).  The mixing shouldn't be an issue since the validation code
upstream will prevent it, but was simplest to allow the comparator
to handle both.
2018-09-03 10:49:19 +02:00
Jim Ferenczi 9310d2eaf3 [CI] Mute IndexShardTests#testIndexCheckOnStartup fails #33345 2018-09-03 10:27:42 +02:00
Jim Ferenczi 2fa75b4438 [CI] Mute LuceneChangesSnapshotTests#testUpdateAndReadChangesConcurrently 2018-09-03 10:14:00 +02:00
Jim Ferenczi f0a61b6dec
Security for _field_names field should not override field statistics (#33261)
In Lucene 8 the statistics for a field (doc_count, sum_doc_count, ...) are
checked and invalid values (v < 0) are rejected. Though for the _field_names
field we hide the statistics of the field if security is enabled since
some terms (field names) may be filtered. However this statistics are never
used, this field is not used for ranking and cannot be used to generate
term vectors. For these reasons this commit restores the original statistics
for the field in order to be compliant with Lucene 8.
2018-09-03 09:36:39 +02:00
Jim Ferenczi 713c07e14d
Add early termination support to BucketCollector (#33279)
This commit adds the support to early terminate the collection of a leaf
in the aggregation framework. This change introduces a MultiBucketCollector which
handles CollectionTerminatedException exactly like the Lucene MultiCollector.
Any aggregator can now throw a CollectionTerminatedException without stopping
the collection of a sibling aggregator. This is useful for aggregators that
can infer their result without visiting all documents (e.g.: a min/max aggregation on a match_all query).
2018-09-03 09:34:35 +02:00
Alpar Torok 3c367a2c46
Fix extractjar task ci (#33272)
Remove tasks to check license and notice and add build integration test instead.

Closes #33201
2018-09-03 10:18:34 +03:00
Nhat Nguyen 3a1dad1050 Mute testFollowIndexAndCloseNode
Tracked at #33337
2018-09-02 19:17:51 -04:00
Nik Everett f8b7a4dbc8
Logging: Drop Settings from some logging ctors (#33332)
Drops `Settings` from some logging ctors now that they are no longer
needed. This should allow us to stop passing `Settings` around to quite
as many places.
2018-09-02 16:51:26 -04:00
Jason Tedor ea4eef8641
Merge branch 'master' into ccr
* master:
  HLREST: add update by query API (#32760)
2018-09-02 16:07:50 -04:00
Sohaib Iftikhar 389bf67275 HLREST: add update by query API (#32760)
Adds update by query to the high level rest client.
2018-09-02 15:15:00 -04:00
Nhat Nguyen 3197a6bbdd Merge branch 'master' into ccr
* master:
  HLRC: ML Flush job (#33187)
  HLRC: Adding ML Job stats (#33183)
  LLREST: Drop deprecated methods (#33223)
  Mute testSyncerOnClosingShard
  [DOCS] Moves machine learning APIs to docs folder (#31118)
2018-09-02 09:30:51 -04:00
Nhat Nguyen c6b011f8ea
TEST: Increase timeout testFollowIndexAndCloseNode (#33333)
This test fails several times due to timeout when asserting the number
of docs on the following and leading indices. This change reduces
the number of docs to index and increases the timeout.
2018-09-02 09:28:47 -04:00
Benjamin Trent 6770a456b8
HLRC: ML Flush job (#33187)
* HLRC: ML Flush job

* Fixing package, paths, and test

* Addressing comments
2018-09-01 16:01:23 -05:00
Benjamin Trent 19b14fa5ed
HLRC: Adding ML Job stats (#33183)
* HLRC: Adding pojos for get job stats

HLRC: Adding pojos for job stats request

* HLRC: Adding job stats pojos

* HLRC: ML job stats

* Minor syntax changes and adding license headers

* minor comment change

* Moving to client package, minor changes

* Addressing PR comments

* removing bad sleep

* addressing minor comment around test methods

* adding toplevel random fields for tests

* addressing minor review comments
2018-09-01 13:32:18 -05:00
Nik Everett f28cddf951
LLREST: Drop deprecated methods (#33223)
In #29623 we added `Request` object flavored requests to the low level
REST client and in #30315 we deprecated the old `performRequest`s. In a
long series of PRs I've changed all of the old style requests. This
drops the deprecated methods and will be released with 7.0.
2018-09-01 11:11:25 -04:00
Nhat Nguyen ce635f5f15 Mute testSyncerOnClosingShard
Tracked at #33330
2018-09-01 09:53:31 -04:00
Lisa Cawley b7a63f7e7d
[DOCS] Moves machine learning APIs to docs folder (#31118) 2018-08-31 16:49:24 -07:00
Nhat Nguyen b93507608a Merge branch 'master' into ccr
* master:
  Mute test watcher usage stats output
  [Rollup] Fix FullClusterRestart test
  Adjust soft-deletes version after backport into 6.5
  completely drop `index.shard.check_on_startup: fix` for 7.0 (#33194)
  Fix AwaitsFix issue number
  Mute SmokeTestWatcherWithSecurityIT testsi
  drop `index.shard.check_on_startup: fix` (#32279)
  tracked at
  [DOCS] Moves ml folder from x-pack/docs to docs (#33248)
  [DOCS] Move rollup APIs to docs (#31450)
  [DOCS] Rename X-Pack Commands section (#33005)
  TEST: Disable soft-deletes in ParentChildTestCase
  Fixes SecurityIntegTestCase so it always adds at least one alias (#33296)
  Fix pom for build-tools (#33300)
  Lazy evaluate java9home (#33301)
  SQL: test coverage for JdbcResultSet (#32813)
  Work around to be able to generate eclipse projects (#33295)
  Highlight that index_phrases only works if no slop is used (#33303)
  Different handling for security specific errors in the CLI. Fix for https://github.com/elastic/elasticsearch/issues/33230 (#33255)
  [ML] Refactor delimited file structure detection (#33233)
  SQL: Support multi-index format as table identifier (#33278)
  MINOR: Remove Dead Code from PathTrie (#33280)
  Enable forbiddenapis server java9 (#33245)
2018-08-31 19:03:04 -04:00
Nhat Nguyen ca94d052b8 Mute test watcher usage stats output
Tracked at #33326
2018-08-31 18:58:46 -04:00
Zachary Tong ebed8f2618 [Rollup] Fix FullClusterRestart test
We need to wait for the job to fully initialize and start before
we can attempt to stop it.  If we don't, it's possible for the stop
API to be called before the persistent task is fully loaded and it'll
throw an exception.

Closes #32773
2018-08-31 18:44:42 -04:00
Nhat Nguyen 08b9247ce2 Adjust soft-deletes version after backport into 6.5
Relates #33222
2018-08-31 16:50:08 -04:00
Vladimir Dolzhenko 00b272af32 completely drop `index.shard.check_on_startup: fix` for 7.0 (#33194)
Relates to #32279
2018-08-31 22:08:28 +02:00
Tim Brooks 6a77cb4211
Fix AwaitsFix issue number
In the previous commit where SmokeTestWatcherWithSecurityIT tests were
muted, I added the incorrect issue numbers. This commit fixes this. The
issue for the tests is #33320.
2018-08-31 13:37:22 -06:00
Tim Brooks 4f1ffb5cb1
Mute SmokeTestWatcherWithSecurityIT testsi
Tests from the SmokeTestWatcherWithSecurityIT suite have been failing
occasionally. This commit mutes all the tests. This is tracked in
2018-08-31 13:34:44 -06:00
Vladimir Dolzhenko 3d82a30fad
drop `index.shard.check_on_startup: fix` (#32279)
drop `index.shard.check_on_startup: fix`

Relates #31389
2018-08-31 21:29:06 +02:00
Tal Levy d4f2b5be7d tracked at
https://github.com/elastic/elasticsearch/issues/33320
and
https://github.com/elastic/elasticsearch/issues/30777
2018-08-31 12:03:49 -07:00
Lisa Cawley 874ebcb6d4
[DOCS] Moves ml folder from x-pack/docs to docs (#33248) 2018-08-31 11:56:26 -07:00
Lisa Cawley cdeadfc585
[DOCS] Move rollup APIs to docs (#31450) 2018-08-31 10:50:43 -07:00