Commit Graph

41322 Commits

Author SHA1 Message Date
Andrei Stefan 6fb7e49b22
SQL: TRUNCATE and ROUND functions (#33779)
* Added TRUNCATE function, modified ROUND to accept two parameters instead of one. Made the second parameter optional for both functions.
* Added documentation for both functions.
2018-09-20 15:29:53 +03:00
Marios Trivyzas 168a0b23fe
SQL: Fix ANTL4 Grammar ambiguities. (#33854)
Removed rules in the grammar that were superfluous, as they
are already "caught" other rules in the same context.

Also switched to exact ambig detection for debug mode

Fixes: #31885
2018-09-20 14:26:07 +02:00
Simon Willnauer 3522b9084b
Introduce a `search_throttled` threadpool (#33732)
Today all searches happen on the search threadpool which is the correct
behavior in almost any case. Yet, there are exceptions where for instance
searches searches should be passed through a single-thread
thread-pool to reduce impact on a node. This change adds a index-private setting that allows to mark an index as throttled for searches and forks off all non-stats searcher access to this thread-pool for indices that are marked as `index.search.throttled`
2018-09-20 13:43:11 +02:00
David Roberts f5a2ffc3f6
[DOCS][ML] Document the ML find_file_structure endpoint (#33723)
Relates #33471
Relates #33630
2018-09-20 12:17:09 +01:00
David Turner c041e94349
Test that transient settings beat persistent ones (#33818)
Transient settings override persistent settings, but in fact all of the tests
that run as part of `:server:test` and `:server:integTest` will pass if the
precedence is changed to be the other way round. This change adds a test that
verifies the precedence is as documented.
2018-09-20 11:17:19 +01:00
David Turner 6d3a7910f8
Remove no-op test (#33861)
This test was introduced when muting some other tests in #32498, but not
removed when the tests were unmuted in #32514.
2018-09-20 10:45:28 +01:00
Tim Vernum 8d50c10208 Mute ShrinkIndexIT.testCreateShrinkIndexToN on Windows
Relates: #33857
2018-09-20 18:21:15 +10:00
Daniel Mitterdorfer b1cc58e425
Allow to clear the fielddata cache per field
With this commit we clear the fielddata cache per field as it is
supposed to be. Previously we retrieved the proper field from the cache
but then cleared the entire cache anyway.

Closes #33798
Relates #33807
2018-09-20 08:59:53 +02:00
Martijn van Groningen 384ce58535
removed unused fields 2018-09-20 08:56:23 +02:00
Tim Vernum 3ca53437a2 Mute SimpleSecurityNetty4TransportTests.testRenegotiation
This test has failed 3 times in the last 3 days

Relates: #33772
2018-09-20 16:24:41 +10:00
Martijn van Groningen 44c7c4b166
[CCR] Add auto follow stats api (#33801)
GET /_ccr/auto_follow/stats

Returns:

```
{
   "number_of_successful_follow_indices": ...
   "number_of_failed_follow_indices": ...
   "number_of_failed_remote_cluster_state_requests": ...
   "recent_auto_follow_errors": [
      ...
   ]
}
```

Relates to #33007
2018-09-20 07:16:20 +02:00
Tim Vernum 1f1ebb4656 Add additional null check in _cat/shards
The target of the func lambda may be null (e.g. in a mixed cluster
where older nodes lack some of the values)

Relates: #33858 / 331caba
Closes #33877
2018-09-20 06:44:13 +02:00
Tim Vernum ff934e3dcd Mute broken test on MacOS
Seems to be triggered by 0cf0d73
See: https://github.com/elastic/elasticsearch/issues/33879
2018-09-20 14:06:40 +10:00
Nhat Nguyen 05bf9dc2e8
Add contains method to LocalCheckpointTracker (#33871)
This change adds "contains" method to LocalCheckpointTracker.
One of the use cases is to check if a given operation has been processed
in an engine or not by looking up its seq_no in LocalCheckpointTracker.

Relates #33656
2018-09-19 20:29:36 -04:00
Benjamin Trent 4767a016a5
Adding node_count to ML Usage (#33850) (#33863) 2018-09-19 13:35:09 -07:00
Jason Tedor 839a677557
Do not override named S3 client credentials (#33793)
In cases when mixed secure S3 client credentials and insecure S3 client
credentials were used (that is, those defined on the repository), we
were overriding the credentials from the repository using insecure
settings to all the repositories. This commit fixes this by not mixing
up repositories that use insecure settings with those that use secure
settings.
2018-09-19 16:18:54 -04:00
Nik Everett 26c4f1fb6c
Core: Default node.name to the hostname (#33677)
Changes the default of the `node.name` setting to the hostname of the
machine on which Elasticsearch is running. Previously it was the first 8
characters of the node id. This had the advantage of producing a unique
name even when the node name isn't configured but the disadvantage of
being unrecognizable and not being available until fairly late in the
startup process. Of particular interest is that it isn't available until
after logging is configured. This forces us to use a volatile read
whenever we add the node name to the log.

Using the hostname is available immediately on startup and is generally
recognizable but has the disadvantage of not being unique when run on
machines that don't set their hostname or when multiple elasticsearch
processes are run on the same host. I believe that, taken together, it
is better to default to the hostname.

1. Running multiple copies of Elasticsearch on the same node is a fairly
advanced feature. We do it all the as part of the elasticsearch build
for testing but we make sure to set the node name then.
2. That the node.name defaults to some flavor of "localhost" on an
unconfigured box feels like it isn't going to come up too much in
production. I expect most production deployments to at least set the
hostname.

As a bonus, production deployments need no longer set the node name in
most cases. At least in my experience most folks set it to the hostname
anyway.
2018-09-19 15:21:29 -04:00
Jack Conradson 3df285d9f0
Painless: Rename *Binding classes to *ClassBinding. (#33865)
With the upcoming instance bindings, the singular *Binding name isn't descriptive enough 
with multiple binding types. This renames the existing *Binding classes to *ClassBinding. 
Mechanical change (with some error messages changed from binding to class binding by 
hand).
2018-09-19 12:19:44 -07:00
Julie Tibshirani c764012347
Create a WatchStatus class for the high-level REST client. (#33527)
This class will be used in a few of the watcher responses ('get watch', 'ack watch', etc.), so it's being introduced first in its own PR.
2018-09-19 12:17:08 -07:00
Simon Willnauer a92dda2e7e
Move CompletionStats into the Engine (#33847)
By moving CompletionStats into the engine we can easily cache the stats for
read-only engines if necessary. It also moves the responsibiltiy out of IndexShard
which has quiet some complexity already.

Relates to #33835
2018-09-19 20:35:57 +02:00
Simon Willnauer 0fa5758bc6
Fix potential NPE in `_cat/shards/` with partial CommonStats (#33858)
Today if we fetch common stats from a shard we might get a partial response
if the shard is closed while we fetch the stats. This causes hard to track and
reproduce NPEs. This change streamlines null checking to ensure we only render
stats we actually received.
2018-09-19 20:34:54 +02:00
Nik Everett 3ede13a454
Test framework fall cleaning (#33423)
Wraps all lines in our test framework at 140 characters because that is
our standard line length and removes all of the checkstyle suppressions
for the test framework.

Drops most of `ModuleTestCase` because it isn't used and we're moving
away from using guice in the way that it wants to test anyway. Also
switches a few classes that extend it but don't use it to extend
`ESTestCase` instead.
2018-09-19 14:34:02 -04:00
Simon Willnauer 6ec12bef0d Add missing IndexShard#readAllowed()
This was lost in #33835
2018-09-19 17:07:13 +02:00
Alan Woodward 5107949402
Allow TokenFilterFactories to rewrite themselves against their preceding chain (#33702)
We currently special-case SynonymFilterFactory and SynonymGraphFilterFactory, which need to 
know their predecessors in the analysis chain in order to correctly analyze their synonym lists. This
special-casing doesn't work with Referring filter factories, such as the Multiplexer or Conditional
filters. We also have a number of filters (eg the Multiplexer) that will break synonyms when they
appear before them in a chain, because they produce multiple tokens at the same position.

This commit adds two methods to the TokenFilterFactory interface.

* `getChainAwareTokenFilterFactory()` allows a filter factory to rewrite itself against its preceding
  filter chain, or to resolve references to other filters. It replaces `ReferringFilterFactory` and
  `CustomAnalyzerProvider.checkAndApplySynonymFilter`, and by default returns `this`.
* `getSynonymFilter()` defines whether or not a filter should be applied when building a synonym
  list `Analyzer`. By default it returns `true`.

Fixes #33609
2018-09-19 15:52:14 +01:00
Benjamin Trent 4190a9f1e9
Delete custom index if the only contained job is deleted (#33788)
* Delete custom index if the only contained job is deleted
2018-09-19 07:42:26 -07:00
markharwood c118581617 Test fix - Graph connections could appear in different orders
Graph connections could appear in different orders based on insertion sequence

Closes #33686
2018-09-19 15:16:14 +01:00
Christoph Büscher 546e7361ed
[Tests] Nudge wait time in RemoteClusterServiceTests (#33853)
This test occasionally fails in `testCollectSearchShards` waiting on what seems
to be a search request to a remote cluster for one second. Given that the test
fails here very rarely I suspect maybe one second is very rarely not enough so
we could fix it by increasing the max wait time slightly.

Closes #33852
2018-09-19 15:58:35 +02:00
Shaunak Kashyap d78966c4c7
Fixing assertions in integration test (#33833) 2018-09-19 04:49:54 -07:00
Martijn van Groningen d9947c631a
[CCR] Rename idle_shard_retry_delay to poll_timout in auto follow patterns (#33821) 2018-09-19 13:13:20 +02:00
David Kyle 68c0a29578
HLRC: Delete ML calendar (#33775) 2018-09-19 10:20:21 +01:00
Simon Willnauer 0c77f45dc6
Move DocsStats into Engine (#33835)
By moving DocStats into the engine we can easily cache the stats for
read-only engines if necessary. It also moves the responsibility out of IndexShard
which has quiet some complexity already.
2018-09-19 11:03:11 +02:00
Christoph Büscher 6f3b3338ba
[Docs] Clarify accessing Date methods in painless (#33560)
The documentation currently tells users to use `doc['event_date'].value.getMillis` to access
milliseconds in a date. It turns out the way it works is `doc['event_date'].value.millis`. This 
change corrects this and gives a hint at how other date related methods work.
2018-09-19 10:34:53 +02:00
Vladimir Dolzhenko a3e8b831ee
add elasticsearch-shard tool (#32281)
Relates #31389
2018-09-19 10:28:22 +02:00
Simon Willnauer 251489d59a
Cut over to unwrap segment reader (#33843)
The fix in #33757 introduces some workaround since FilterCodecReader didn't
support unwrapping. This cuts over to a more elegant fix to access the readers
segment infos.
2018-09-19 10:18:03 +02:00
Marios Trivyzas d22b383b9c
SQL: Fix issue with options for QUERY() and MATCH(). (#33828)
Previously multiple comma separated lists of options where not
recognized correctly which resulted in only the last of them
to be taked into account, e.g.:

For the following query:
  SELECT * FROM test WHERE QUERY('search', 'default_field=foo', 'default_operator=and')"
only the `default_operator=and` was finally passed to the ES query.

Fixes: #32602
2018-09-19 10:16:24 +02:00
David Turner c9765d5fb9
Emphasize that filesystem-level backups don't work (#33102)
It is not obvious that a filesystem-level backup may capture an inconsistent
set of files that may fail on restore, or (worse) succeed having silently
discarded some data. This change spells the out, and reorganises the first page
or so of the snapshot/restore docs to make this warning fit more nicely.
2018-09-19 08:36:03 +01:00
Jim Ferenczi 61e1df0274
Use the global doc id to generate a random score (#33599)
This commit changes the random_score function to use the global docID of the document
rather than the segment docID to generate random scores. As a result documents that have
the same segment docID within the shard will generate different scores.
2018-09-19 09:28:38 +02:00
Adrien Grand c4261bab44
Add minimal sanity checks to custom/scripted similarities. (#33564)
Add minimal sanity checks to custom/scripted similarities.

Lucene 8 introduced more constraints on similarities, in particular:
 - scores must not be negative,
 - scores must not decrease when term freq increases,
 - scores must not increase when norm (interpreted as an unsigned long)
   increases.

We can't check every single case, but could at least run some sanity checks.

Relates #33309
2018-09-19 09:19:13 +02:00
Ignacio Vera 7f473b683d
Profiler: Don’t profile NEXTDOC for ConstantScoreQuery. (#33196)
* Profiler: Don’t profile NEXTDOC for ConstantScoreQuery.

A ConstantScore query will return the iterator of its inner query.
However, when profiling, the constant score query is wrapped separately
from its inner query, which distorts the times emitted by the profiler.
Return the iterator directly in such a case.

Closes #23430
2018-09-18 23:32:16 -07:00
Martijn van Groningen 013b64a07c
[CCR] Change FollowIndexAction.Request class to be more user friendly (#33810)
Instead of having one constructor that accepts all arguments, all parameters
should be provided via setters. Only leader and follower index are required
arguments. This makes using this class in tests and transport client easier.
2018-09-19 07:18:24 +02:00
Andrei Stefan c6e3231ef3
SQL: day and month name functions tests locale providers enforcement (#33653)
DAYNAME and MONTHNAME functions tests will be skipped if the right JVM parameter (-Djava.locale.providers=COMPAT) is not used in unit testing environment
2018-09-19 07:33:15 +03:00
Armin Braun 0cf0d73813
TESTS: Set SO_LINGER = 0 for MockNioTransport (#32560)
* TESTS: Set SO_LINGER = 0 for MockNioTransport

* Prevents lingering sockets in TIME_WAIT piling up during test runs and leading to port collisions that manifest as timeouts
* Fixes #32552
2018-09-19 06:05:36 +02:00
Ryan Ernst 99513b306e
Test: Relax jarhell gradle test (#33787)
Gradle can sometimes emit mixed log lines due to how we spawn things in
separate processes. This commit changes the jarhell integ test to only
look for the exception and message, instead of including the outer part
about the exception in thread main.

closes #33774
2018-09-18 13:11:46 -07:00
Martijn van Groningen 805a12361f
[CCR] Fail with a descriptive error if leader index does not exist (#33797)
Closes #33737
2018-09-18 21:47:02 +02:00
Zachary Tong f4cbbcf98b
Add ES version 6.4.2 (#33831)
Version and properties files
2018-09-18 15:25:20 -04:00
Armin Braun c6462057a1
MINOR: Remove Some Dead Code in Scripting (#33800)
* The is default check method is not used in ScriptType
* The removed vars on ExpressionSearchScript are unused
2018-09-18 20:43:31 +02:00
Simon Willnauer 9026c3ee92
Ensure realtime `_get` and `_termvectors` don't run on the network thread (#33814)
The change in #27500 introduces this regression that causes `_get` and `_term_vector`
actions to run on the network thread if the realtime flag is set.
This fixes the issue by delegating to the super method forking on the corresponding threadpool.
2018-09-18 19:53:42 +02:00
Simon Willnauer 98ccd94962
Factor out a ChannelActionListener (#33819)
We use similar / same concepts in SerachTransportService and HandledTransportAction but both
duplicate the efforts with slightly different implementation details. This streamlines
sending responses / exceptions back to a channel in an ActionListener with appropriate logging.
2018-09-18 19:53:26 +02:00
Jim Ferenczi 241c74efb2
upgrade to a new snapshot of Lucene 8 (7d0a7782fa) (#33812) 2018-09-18 18:16:40 +02:00
Nik Everett bc12a948b5
Checkstyle: Package declarations (#33784)
Make sure that all java files have a package declaration and that all of
the package declarations line up with the directory structure. This
would have caught the bug that I caused in
190ea9a6de and fixed in
b6d68bd805.
2018-09-18 12:04:35 -04:00