Commit Graph

30506 Commits

Author SHA1 Message Date
Sergey Galkin f057fc294a Rescore collapsed documents (#28521)
This change adds the ability to rescore collapsed documents.
2018-03-04 13:39:50 -08:00
Jim Ferenczi c26bd6046b
Fix (simple)_query_string to ignore removed terms (#28871)
This change ensures that we ignore terms removed from the analysis rather than returning a match_no_docs query for the part
that contain the stop word. For instance a query like "the AND fox" should ignore "the" if it is considered as a stop word instead of
adding a match_no_docs query.
This change also fixes the analysis of prefix terms that start with a stop word (e.g. `the*`). In such case if `analyze_wildcard` is true and `the`
is considered as a stop word this part of the query is rewritten into a match_no_docs query. Since it's a prefix query this change forces the prefix query
on `the` even if it is removed from the analysis.

Fixes #28855
Fixes #28856
2018-03-04 13:25:46 -08:00
Tim Roes 5689dc1182 [Docs] Fix typo in composite aggregation (#28891) 2018-03-04 11:47:24 -08:00
Simon Willnauer 5a1d9f33a0
Try if tombstone is eligable for pruning before locking on it's key (#28767)
Pruning tombstones is quite expensive since we have to walk though all
deletes in the live version map and acquire a lock on every value even though
it's impossible to prune it. This change does a pre-check if a delete is old enough
and if not it skips acquireing the lock.
2018-03-04 11:31:13 -08:00
Mayya Sharipova f53d159aa1
Limit analyzed text for highlighting (improvements) (#28808)
Increase the default limit of `index.highlight.max_analyzed_offset` to 1M instead of previous 10K.

Enhance an error message when offset increased to include field name, index name and doc_id.

Relates to https://github.com/elastic/kibana/issues/16764
2018-03-02 08:09:05 -08:00
Andrew Odendaal f207aacac0 Missing `timeout` parameter from the REST API spec JSON files (#28328)
Closes #28200
2018-03-02 14:47:06 +01:00
Jim Ferenczi 48a7425ae6
Clarifies how query_string splits textual part (#28798)
* Clarifies how the query_string splits textual part to build a query

Whitespaces are not considered as operators anymore in 6x but the documentation is not clear about it.
This commit changes the example in the documentation and adds a note regarding whitespaces and operators.

Closes #28719
2018-03-01 15:08:25 -08:00
Ivan Brusic 6fb8319ffc Update outdated java version reference (#28870)
It is only a comment, but can confuse those reading the code

Used 6.0 as an arbitrary elasticsearch.version value since it is version that required Java 8
2018-03-01 14:17:24 -08:00
olcbean 472acf7833 [DOCS] fix put_mapping snippet (#28814)
Add a java snippet to be run in an integration test
in order to guarantee that the snippet is correct

Closes #28778
2018-03-01 10:59:52 +01:00
olcbean 25834e2d26 REST api spec: remove a common param from nodes.usage.json (#28835)
"human" is a common parameter defined in `_common.json`
It should not be repeated again for a concrete api.

Closes #28226
2018-03-01 10:58:38 +01:00
olcbean ec0a1b36c9 REST api spec: remove unsupported parameter `parent_node` (#28841) 2018-03-01 10:57:25 +01:00
Luca Cavanna 184a8718d8
REST high-level client: add flush API (#28852)
Relates to #27205
2018-03-01 10:56:03 +01:00
Bojan Matić 742e9f5078 Document removal of forced versioning for update API (#28834) 2018-02-28 09:41:44 -08:00
Yu 95dea2408d Add Refresh API for RestHighLevelClient (#27799)
Relates to #27205
2018-02-28 11:49:14 +01:00
Jason Tedor 303ce30f77 Add note regarding quoting values for network.host
Values for the network.host setting can often contain a colon which is a
character that is considered special by YAML (these arise in IPv6
addresses and some of the special tags like ":ipv4"). As such, these
values need to be quoted or a YAML parser will be unhappy with
them. This commit adds a note to the docs regarding this.
2018-02-27 10:33:45 -08:00
FUJI Goro 2baa19ea64 [Docs] Specify function score logarithm modifiers (#28821)
The logarithm with base 10 is called "Common Logarithm".
2018-02-27 10:29:43 -08:00
olcbean c5821f9645 [docs] Line breaks for High-level REST Client (#28825)
Wrap code snippets for better display in the generated docs.
2018-02-26 14:03:03 -05:00
Jason Tedor fb073216b1 Move search concurrency and parallelism paragraphs
These paragraphs should be on the top-level search page for visibility
so this commit moves them, and puts them under a clear heading.
2018-02-26 07:47:57 -08:00
olcbean beb8b10556 Fix inconsistency in docs regarding single types (#28715)
This commit fixes some inconsistencies in the docs regarding single
types. The inconsistencies are between the verbiage and the relevant
snippets.
2018-02-26 07:08:37 -08:00
lzh3636 929dba8667 Fix log message when virtual lock not possible (#28829)
When virtual lock is not possible because JNA is unavailable, we log a
warning message. Yet, this log message refers to mlockall rather than
virtual lock, presumably because of a copy/paste error. This commit
fixes this issue.
2018-02-26 06:55:32 -08:00
Jason Tedor 57225652b3 Revert "Destroy test VMs on clean (#28818)"
This reverts commit f66234bdd5.
2018-02-25 21:49:36 -05:00
Jason Tedor f66234bdd5
Destroy test VMs on clean (#28818)
This commit adds a dependency on the destroy task to the clean task so
that the VMs are destroyed when the clean task is executed.
2018-02-25 17:02:38 -05:00
Jason Tedor c9ed49da10 Specify working directory for destroy task
This commit specifies that the working directory of the destroy task for
destroying test VMs is the root of the build. This is necessary in case
the build was run from a sub-directory, the Vagrant command would then
not be able to locate the Vagrantfile for the VMs in question.
2018-02-25 11:56:54 -08:00
Jason Tedor 22736848b0
Always destroy Vagrant boxes before tests (#28813)
Today we do not destroy Vagrant boxes before tests. This is because
constantly reprovisioning these boxes is time-consuming. Yet, not
destroying these boxes can lead to state being left around that impacts
subsequent test runs. To address this, we now always destroy these boxes
before tests and provide a flag to set if this is not desired while
iterating locally.
2018-02-25 09:15:56 -05:00
Ryan Ernst 88e44dd133 Fix rest test to not specify zip distribution
This was lefter from
https://github.com/elastic/elasticsearch/pull/28805, and effectively did
not allow changing the distribution from zip (exception in esplugin
which sets it later).
2018-02-24 23:12:13 -08:00
Nhat Nguyen f1a94de9e7
Replace log4j.Supplier by jdk’s in non-logging usage (#28812)
This commit replaces `org.apache.logging.log4j.util.Supplier` by
`java.util.function.Supplier` in non-logging code. These usages are
neither incorrect nor wrong but rather than accidental. I think our
intention was to use the JDK's Supplier in these places.
2018-02-24 14:28:32 -05:00
Ryan Ernst 3a64e6d121
Test: Remove specifying zip distribution in qa tests (#28805)
Applying the rest test gradle plugin already uses the zip distribution
by default, so specifying it explicitly is not necessary. These are
leftovers from before zip was the default for rest tests.
2018-02-23 13:45:38 -08:00
Ke Li a77273fc01 Reject regex search if regex string is too long (#28542)
* Reject regex search if regex string is too long (#28344)

* Add docs

* Introduce index level setting `index.max_regex_length`
 to control the maximum length of the regular expression

Closes #28344
2018-02-23 10:41:24 -08:00
Luca Cavanna cd3d9c9f80
[TEST] share code between streamable/writeable/xcontent base test classes (#28785)
Today we have two test base classes that have a lot in common when it comes to testing wire and xcontent serialization: `AbstractSerializingTestCase` and `AbstractXContentStreamableTestCase`. There are subtle differences though between the two, in the way they work, what can be overridden and features that they support (e.g. insertion of random fields).

This commit introduces a new base class called `AbstractWireTestCase` which holds all of the serialization test code in common between `Streamable` and `Writeable`. It has two minimal subclasses called `AbstractWireSerializingTestCase` and `AbstractStreamableTestCase` which are specialized for `Writeable` and `Streamable`.

This commit also introduces a new test class called `AbstractXContentTestCase` for all of the xContent testing, which holds a testFromXContent method for parsing and rendering to xContent. This one can be delegated to from the existing `AbstractStreamableXContentTestCase` and `AbstractSerializingTestCase` so that we avoid code duplicate as much as possible and all these base classes offer the same functionalities in the same way. Having this last base class decoupled from the serialization testing may also help with the REST high-level client testing, as there are some classes where it's hard to implement equals/hashcode and this makes it possible to override `assertEqualInstances` for custom equality comparisons (also this base class doesn't require implementing equals/hashcode as it doesn't test such methods.
2018-02-23 10:48:48 +01:00
Nicholas Knize 3728c50d85 [GEO] Fix points_only indexing failure for GeoShapeFieldMapper
This commit fixes a bug that was introduced in PR #27415 for 6.1
and 7.0 where a change to support MULTIPOINT shapes mucked up
indexing of standalone points.
2018-02-22 20:57:09 -06:00
Lee Hinman 5bb79558e7
Decouple XContentGenerator and JsonXContentGenerator from BytesReference (#28772)
This removes the link these two classes have with BytesReference, in favor of an
`InputStream` approach.

Relates to #28504
2018-02-22 14:22:37 -07:00
Ryan Ernst b29ba25c86 Build: Fix packages distributions inclusion of empty directories
This was accidentally broken in #28760.
2018-02-22 11:30:49 -08:00
jxy 497b3d7a20 Fix node ID reported by ThrottlingAllocationDecider (#28779)
Previously the message reported when `node_concurrent_outgoing_recoveries`
resulted in a `THROTTLE` decision included the reporting node's ID rather than
that of the primary. This commit fixes that.

Fixes #28777.
2018-02-22 19:23:15 +00:00
Tim Brooks 5a8ec9b762
Selectors operate on channel contexts (#28468)
This commit is related to #27260. Currently there is a weird
relationship between channel contexts and nio channels. The selectors
use the context for read and writing. But the selector operates directly
on the nio channel for registering, closing, and connecting.

This commit works on improving this relationship. The selector operates
directly on the context which wraps the low level java.nio.channels. The
NioChannel class is simply an API that is used to interact with the
channel (sending messages from outside the selector event loop,
scheduling a close, adding listeners, etc). The context is only used
internally by the channel to implement these apis and by the selector to
perform these operations.
2018-02-22 09:44:52 -07:00
Daniel Mitterdorfer d0831fdd3e
Reduce max javac memory to 512mb (#28783)
With this commit we reduce the maximum amount of memory that the javac
compiler can use from 1g to 512mb. While the build would succeed even
with 256mb, it influences compile time slightly negatively.

We have measured that the runtime overhead stays tolerable by running
the following command five times under repeatable conditions (i.e. we
execute `./gradlew clean`, then drop the caches and TRIM the disk):

```
./gradlew compileGroovy compileJava compileJava9Java\
 compileTestGroovy compileTestJava compileGroovy
```

The results in seconds (as reported by Gradle) are:

* 1gb: avg: 253s, min: 252s, max: 256s
* 512mb: avg: 257s, min: 256s, max: 259s
2018-02-22 16:36:01 +01:00
Tanguy Leroux a6a138905d
Use client settings in repository-gcs (#28575)
Similarly to what has been done for s3 and azure, this commit removes
the repository settings `application_name` and `connect/read_timeout`
in favor of client settings. It introduce a GoogleCloudStorageClientSettings
class (similar to S3ClientSettings) and a bunch of unit tests for that,
it aligns the documentation to be more coherent with the S3 one, it
documents the connect/read timeouts that were not documented at all and
also adds a new client setting that allows to define a custom endpoint.
2018-02-22 15:40:20 +01:00
Jason Tedor daf430c006 Revert "Disable BWC tests for build issues"
This reverts commit 45470945d2.
2018-02-22 06:36:27 -05:00
Luca Cavanna 1df711c5b7
Remove AcknowledgedRestListener in favour of RestToXContentListener (#28724)
This commit makes AcknowledgedResponse implement ToXContentObject, so that the response knows how to print its own content out to XContent, which allows us to remove AcknowledgedRestListener.
2018-02-22 09:13:30 +01:00
Ryan Ernst 014e90d903
Build: Consolidate archives and packages configuration (#28760)
This commit moves the distribution specific tasks into the respective
archives and packages builds. The collocation of common and distribution
specific tasks make it much easier to reason about what is expected in a
particular distribution.
2018-02-21 17:46:40 -08:00
Jason Tedor 3dfb4b8b18 Skip some plugins service tests on Windows
These tests need to be skipped. They cause plugins to be loaded which
causes a child classloader to be opened. We do not want to add the
permissions to be able to close a classloader solely for these tests,
and the JARs created in the test can not be deleted on Windows until the
classloader is closed. Since this will not happen before test teardown,
the test will fail on Windows. So, we skip these tests.
2018-02-21 15:22:27 -05:00
Luca Cavanna 8b4a298874
Migrate some *ResponseTests to AbstractStreamableXContentTestCase (#28749)
This allows us to save a bit of code, but also adds more coverage as it tests serialization which was missing in some of the existing tests. Also it requires implementing equals/hashcode and we get the corresponding tests for them for free from the base test class.
2018-02-21 20:04:12 +01:00
Jason Tedor 45470945d2 Disable BWC tests for build issues
This is a temporary disabling of BWC tests while waiting for some build
issues to be addressed.
2018-02-21 13:25:33 -05:00
Tim Brooks de2a0dfa6e
Ensure that azure stream has socket privileges (#28751)
This is related to #28662. It wraps the azure repository inputstream in
an inputstream that ensures `read` calls have socket permissions. This
is because the azure inputstream internally makes service calls.
2018-02-21 11:20:06 -07:00
Deb Adair 7715813bc9 [DOCS] Fixed broken link. 2018-02-21 10:04:09 -08:00
Lee Hinman d7eae4b90f
Pass InputStream when creating XContent parser (#28754)
* Pass InputStream when creating XContent parser

Rather than passing the raw `BytesReference` in when creating the xcontent
parser, this passes the StreamInput (which is an InputStream), this allows us to
decouple XContent from BytesReference.

This also removes the use of `commons.Booleans` so it doesn't require more
external commons classes.

Related to #28504

* Undo boolean removal

* Enhance deprecation javadoc
2018-02-21 11:03:25 -07:00
Deb Adair 260d68aad0 [DOCS] Changed to use transient setting to reenabled allocation. Closes #27677 2018-02-21 08:56:06 -08:00
Jason Tedor ed2bbc6e64
Delay path expansion on Windows
There is a bug in the for statement where we execute the JVM options
parser. The bug manfiests in the handling of paths with ) in the
name. The problem is this: we use a for statement to capture the output
of the JVM options parser. A for statement that executes a command
defers execution to cmd. There is this gem from the help:

  1.  If all of the following conditions are met, then quote characters
      on the command line are preserved:

      - no /S switch
      - exactly two quote characters
      - no special characters between the two quote characters,
        where special is one of: &<>()@^|
      - there are one or more whitespace characters between the
        two quote characters
      - the string between the two quote characters is the name
        of an executable file.

  2.  Otherwise, old behavior is to see if the first character is
      a quote character and if so, strip the leading character and
      remove the last quote character on the command line, preserving
      any text after the last quote character.

This means that the ) causes the quotes to be stripped which ruins
everything. This commit fixes this by delaying expansion of the paths.

Relates #28753
2018-02-21 10:58:49 -05:00
javanna e5bceee4a7 [TEST] replace randomAsciiAlphanumOfLengthBetween with randomAsciiLettersOfLengthBetween
This was wrongly changed from the corresponding deprecated method.
2018-02-21 16:36:04 +01:00
Tanguy Leroux 9a95be35cf
[Tests] Extract the testing logic for Google Cloud Storage (#28576)
This pull request extracts in a dedicated class the request/response 
logic that "emulates" a Google Cloud Storage service in our 
repository-gcs tests.

The idea behind this is to make the logic more reusable. The class 
MockHttpTransport has been renamed to MockStorage which now 
only takes care of instantiating a Storage client and does the low-level 
request/response plumbing needed by this client.

The "Google Cloud Storage" logic has been extracted from 
MockHttpTransport and put in a new GoogleCloudStorageTestServer 
that is now independent from the google client testing framework.
2018-02-21 13:20:35 +01:00
Martijn van Groningen 793cbc651a
Moved Grok helper code to a separate Gradle module and let ingest-common module depend on it. 2018-02-21 11:18:08 +01:00