Commit Graph

26543 Commits

Author SHA1 Message Date
Jim Ferenczi 1ba9770037 Fix comparaison of double in InternalTopHits
InternalTopHits uses "==" to compare hit scores and fails when score is NaN.
This commit changes the comparaison to always use Double.compare.

Relates #23253
2017-02-21 18:18:44 +01:00
Simon Willnauer 5e4ba4a60e Never reduce the same agg twice
Some randomization caused reduction of the same agg multiple times
which causes issues on some aggregations.

Relates to #23253
2017-02-21 17:55:44 +01:00
Simon Willnauer 489f38918d Fix incremental reduce randomization in base tests cases
We can and should randomly reduce down to a single result before
we passing the aggs to the final reduce. This commit changes the logic
to do that and ensures we don't trip the assertions the previous imple tripped.

Relates to #23253
2017-02-21 17:13:46 +01:00
Tanguy Leroux 3a0fc526bb UpdateRequest implements ToXContent (#23289)
This commit changes UpdateRequest so that it implements the ToXContentObject interface.
2017-02-21 15:20:15 +01:00
Jim Ferenczi cc865cbc96 Add unit tests for stats and extended stats aggregations (#23287)
Add tests for InternalStats, InternalExtendedStats and StatsAggregator/ExtendedStatsAggregator

Relates #22278
2017-02-21 15:14:54 +01:00
Daniel Mitterdorfer 0744a00001 Set network receive predictor size to 32kb (#23284)
Previously we calculated Netty' receive predictor size for HTTP and transport
traffic based on available memory and worker nodes. This resulted in a receive
predictor size between 64kb and 512kb. In our benchmarks this leads to increased
GC pressure.

With this commit we set Netty's receive predictor size to 32kb. This value is in
a sweet spot between heap memory waste (-> GC pressure) and effect on request
metrics (achieved throughput and latency numbers).

Closes #23185
2017-02-21 14:45:33 +01:00
Simon Willnauer f933f80902 First step towards incremental reduction of query responses (#23253)
Today all query results are buffered up until we received responses of
all shards. This can hold on to a significant amount of memory if the number of
shards is large. This commit adds a first step towards incrementally reducing
aggregations results if a, per search request, configurable amount of responses
are received. If enough query results have been received and buffered all so-far
received aggregation responses will be reduced and released to be GCed.
2017-02-21 13:02:48 +01:00
Tanguy Leroux 39ed76c58b Add parsing method to bulk response (#23234)
This commit adds the `fromXContent()` parsing method to BulkResponse.
2017-02-21 10:49:40 +01:00
Tanguy Leroux c88eb00b83 Add javadoc for DocWriteResponse.Builders (#23267) 2017-02-21 10:19:01 +01:00
Martin Scholz 24bf18b610 Upgrade HDRHistogram to 2.1.9 (#23254) 2017-02-21 08:50:26 +01:00
Martin Scholz 3e292d5245 Migrate TermsQuery to TermInSetQuery (#23229) 2017-02-21 08:49:43 +01:00
Jason Tedor 64bcb74f4b Add another content-type header in packaging tests
In the packaging tests we make some requests to Elasticsearch as part of
the tests. These requests were not setting the content-type header. This
commit addresses this.
2017-02-20 16:15:52 -05:00
Jason Tedor 49b6f670ad Send content-type header in packaging tests
In the packaging tests we make some requests to Elasticsearch as part of
the tests. These requests were not setting the content-type header. This
commit addresses this.
2017-02-20 15:56:09 -05:00
Jim Ferenczi 1ff5b318be Fix for IpRangeAggregatorTests#testRanges
Handle null from/to ranges.

Closes #23272
2017-02-20 21:16:14 +01:00
Jason Tedor 4c2bd5feab Introduce sequence-number-aware translog
Today, the relationship between Lucene and the translog is rather
simple: every document not in Lucene is guaranteed to be in the
translog. We need a stronger guarantee from the translog though, namely
that it can replay all operations after a certain sequence number. For
this to be possible, the translog has to made sequence-number aware. As
a first step, we introduce the min and max sequence numbers into the
translog so that each generation knows the possible range of operations
contained in the generation. This will enable future work to keep around
all generations containing operations after a certain sequence number
(e.g., the global checkpoint).

Relates #22822
2017-02-20 15:05:24 -05:00
Jason Tedor 15f5810774 Mark IP range aggregator test as awaits fix
This test reliably fails with the seed 4AC319F8A6B0329B.
2017-02-20 14:42:16 -05:00
Glen Smith a590a22ea3 Add note and link to 'tune for disk usage' (#23252)
* Add note and link to 'tune for disk usage'

* Changed formatting as suggested

Thanks, @ clintongormley!
2017-02-20 20:31:19 +01:00
Christoph Büscher ea7deace5d Adding fromXContent to Suggest and Suggestion class (#23226)
A follow up to #23202, this adds parsing from xContent and tests to the four Suggestion implementations
and the top level suggest element to be used later when parsing the entire SearchResponse.
2017-02-20 15:45:10 +01:00
Christoph Büscher ea9d51114c Tests: Add unit test for InternalChildren (#23261)
Relates to #22278
2017-02-20 14:02:56 +01:00
Jim Ferenczi 76d6b872dd Add unit tests for GeoBoundsAggregator/InternalGeoBounds (#23259)
* Add unit tests for GeoBoundsAggregator/InternalGeoBounds

Relates #22278
2017-02-20 12:04:30 +01:00
Jim Ferenczi 69b1463f7c Add unit tests for BinaryRangeAggregator/InternalBinaryRange (#23255)
* Add unit tests for BinaryRangeAggregator/InternalBinaryRange

Relates #22278
2017-02-20 11:55:48 +01:00
Tanguy Leroux 872412f645 [Tests] Cleans up DocWriteResponse parsing tests (#23233)
This commit cleans up some parsing tests added from the High Level Rest Client: IndexResponseTests, DeleteResponseTests, UpdateResponseTests, BulkItemResponseTests.

These tests are now more uniform with the others test-from-to-XContent tests we have, they now shuffle the XContent fields before parsing, the asserting method for parsed objects does not used a Map<String, Object> anymore, and buggy equals/hasCode methods in ShardInfo and ShardInfo.Failure have been removed.
2017-02-20 09:45:33 +01:00
Ryan Ernst 5db5dec0e7 Update bwc test to 5.4 snapshot 2017-02-17 14:58:20 -08:00
Nik Everett d1de9574ea Checkstyle: Fix link lengths in sampler aggregation 2017-02-17 15:03:57 -05:00
Jay Modi b234644035 Enforce Content-Type requirement on the rest layer and remove deprecated methods (#23146)
This commit enforces the requirement of Content-Type for the REST layer and removes the deprecated methods in transport
requests and their usages.

While doing this, it turns out that there are many places where *Entity classes are used from the apache http client
libraries and many of these usages did not specify the content type. The methods that do not specify a content type
explicitly have been added to forbidden apis to prevent more of these from entering our code base.

Relates #19388
2017-02-17 14:45:41 -05:00
Adrien Grand 3bd1d46fc7 Add unit tests for terms aggregation objects. (#23149)
Relates #22278
2017-02-17 18:01:40 +01:00
javanna 578853f264 Remove stale comment about setting routing before parent
Order does not matter anymore since we merged #15371
2017-02-17 17:10:53 +01:00
Yuhao Bi 576e698613 Minor fix of _cat output (#23211) (#23213)
One line was missing a trailing "\n"
2017-02-17 10:46:20 +01:00
Ryan Ernst 37f7a70e22 Tests: Remove java installation from vagrant provisioning (#23223)
The elastic images used for testing different systems now have java
installed in the base image. This commit removes the installation of
java, which should make the packagingTest runs more stable, as they will
not depend on flaky system repository mirrors.
2017-02-16 19:32:12 -08:00
Ryan Ernst f277bef5c0 Fix packaging test to only run lsb_release if it exists 2017-02-16 17:48:37 -08:00
Jason Tedor 21181d7865 Fix more missing markup in configuration docs
This commit fixes some missing markup in the configuration docs in the
logging configuration section.
2017-02-16 19:00:49 -05:00
Ryan Ernst fbf824f6a6 Fix plugin properties generation to be exact about the file to copy to
the output zip
2017-02-16 13:43:07 -08:00
Jason Tedor 00a8b8799f Fix control group pattern
The file /proc/self/cgroup lists the control groups to which the process
belongs. This file is a colon separated list of three fields:
 1. a hierarchy ID number
 2. a comma-separated list of hierarchies
 3. the pathname of the control group in the hierarchy

The regex pattern for this contains a bug for the second field. It
allows one or two entries in the comma-separated list, but not
more. This commit fixes the pattern to allow one or more entires in the
comma-separated list.

Relates #23219
2017-02-16 15:31:18 -05:00
Andreas Roussos 788c64848b [DOCS] Fixed various typos in the 'cat APIs' section (#23216) 2017-02-16 20:41:42 +01:00
Ryan Ernst efb6259e3a Tests: Re-enable debian-8 and fedora-24 packaging tests (#23217)
These images have been rebuilt to be preloaded with java 8 installed.
This change re-enables the systems. It also removes some redundancy in
the rpm checks I found while testing the new images, and fixes a
potential issue with generated resources in plugins where a stale dir
can cause junk to get into the distribution.
2017-02-16 11:26:46 -08:00
Ryan Ernst 164079bb93 Disable javadoc for java vesion checker 2017-02-16 09:56:11 -08:00
Jason Tedor 0e3e111921 Replace HTTP link with HTTPS link
This commit replaces an HTTP link in the configuration docs with an
HTTPS link.
2017-02-16 12:19:30 -05:00
Jason Tedor 69dc730341 Fix missing markup in configuration docs
This commit adds a missing backtick in the configuration docs.
2017-02-16 12:19:30 -05:00
Christoph Büscher 268d15ec4c Adding fromXContent to Suggestion.Entry and subclasses (#23202)
This adds parsing from xContent to Suggestion.Entry and its subclasses for Terms-, Phrase-
and CompletionSuggestion.Entry.
2017-02-16 17:59:55 +01:00
David Pilato 76675229c7 Merge branch 'fix/22077-ingest-attachment' 2017-02-16 15:49:04 +01:00
Jason Tedor c9cde11a5e Introduce Java version checker
Today when users start Elasticsearch with their Java configuration
pointing to a pre-Java 8 install, they encounter a cryptic message:

    Exception in thread "main" java.lang.UnsupportedClassVersionError:
    org/elasticsearch/bootstrap/Elasticsearch : Unsupported major.minor
    version 52.0

They often think that they have Java 8 installed but if their JAVA_HOME
or other configuration is causing them to start with a pre-Java 8
install, this error message does not help them.

We introduce a Java version checker that runs on Java 6 as part of the
startup scripts. If the Java version is pre-Java 8, we can display a
helpful error message to the user informing them of the Java version
that the runtime was started with. Otherwise, Elasticsearch starts as it
does today.
2017-02-16 09:06:49 -05:00
Ryan Ernst 60b823c756 Add version checker tool to distributions 2017-02-16 09:06:49 -05:00
Jason Tedor 2925a81cc9 Fix REST spec for exists
A previous change aligned the handling of the GET document and HEAD
document APIs. This commit aligns the specification for these two APIs
as well, and fixes a failing test.

Relates #23196
2017-02-16 08:56:38 -05:00
markharwood 1cd1ff6010 Test fix - faulty assumptions about when exceptions are thrown in relation to number of failing shards. (#23205)
Search exceptions are thrown only when all shards report failure. Fix changes assertion logic to reflect this.

Closes #23203
2017-02-16 13:48:17 +00:00
markwalkom ced99dde50 Update stop-analyzer.asciidoc (#23195)
Clarified where the stopwords file needs to live
2017-02-16 13:36:15 +01:00
Jason Tedor 3f07fcef64 Mark CrudIT#testExists as awaits fix
This commit marks the test CrudIT#testExists as awaiting a fix due to a
behavior change from aligning the GET and HEAD document APIs.
2017-02-15 22:56:52 -05:00
Lee Hinman 5443f7d625 Console-ify curl statements for allocation explain API docs (#23190)
* Console-ify curl statements for allocation explain API docs

Relates to #23001

* Fix tests

* Remove exclusion from build.gradle

* Call out index creation in prose

* Add console back and skip test
2017-02-15 17:18:07 -07:00
Nik Everett a6cf170384 Add content-type header to curl in getting-started
Relates to #23001
2017-02-15 18:14:25 -05:00
Ali Beyad 71739623d3 Consolify snapshot documentation (#23189)
This commit brings the snapshot documentation in conformity
with the CONSOLE format, and fixes the docs so that the documentation
tests can be run against them.
2017-02-15 18:13:27 -05:00
Ryan Ernst 6cdf4f3f72 Plugins: Include license and notice files in zip (#23191)
This commit adds the elasticsearch LICENSE.txt to all plugins that
released with elasticsearch, as well as a generated NOTICE.txt specific
to the dependencies of each plugin.
2017-02-15 11:23:12 -08:00