Commit Graph

26256 Commits

Author SHA1 Message Date
Luca Cavanna 47c0e13a3b Stop returning "es." internal exception headers as http response headers (#22703)
move "es." internal headers to separate metadata set in ElasticsearchException and stop returning them as response headers

Closes #17593

* [TEST] remove ESExceptionTests, move its methods to ElasticsearchExceptionTests or ExceptionSerializationTests
2017-01-24 16:12:45 +01:00
Clinton Gormley 12f5309041 Docs: Made cross-cluster search experimental 2017-01-24 12:44:01 +01:00
Jason Tedor bcffc6fa49 Add hack for Docker cgroups
Docker cgroups are mounted in the wrong place (i.e., inconsistently with
/proc/self/cgroup). This commit adds an undocumented hack for working
around, for now.

Relates #22757
2017-01-24 06:36:03 -05:00
Christoph Büscher 59aefe5a38 Include human readable responses in response parsing tests (#22717)
As a follow up to #22649, this changes the resent tests for parsing parts of search 
responses to randomly set the humanReadable() flag of the XContentBuilder that 
is used to render the responses. This should help to test that we can parse back 
thoses classes if the user specifies `?human=true` in the request url.
2017-01-24 11:17:58 +01:00
Yannick Welsch 12b6ff5233 Use Gradle-version specific source sets
Instead of using Gradle-version specific compilation options, use distinct source sets. This also allows compilation of buildSrc/build-tools under IDEs that
don't understand the version-specific compilation options.

Relates to #22669
2017-01-24 11:09:57 +01:00
Yannick Welsch 36198e0275 Make build Gradle 2.14 / 3.x compatible (#22669)
This changes build files so that building Elasticsearch works with both Gradle 2.13 as well as higher versions of Gradle (tested 2.14 and 3.3), enabling a smooth transition from Gradle 2.13 to 3.x.
2017-01-24 11:09:57 +01:00
David Pilato 8701f7a3ce Add missing mime4j library
In some cases (apparently with outlook files), mime4j library is needed.
We removed it in the past which can cause elasticsearch to crash when you are using ingest-attachment (and probably mapper-attachments as well in 2.x series) with a file which requires this library.

 Similar problem as the one reported at #22077.
2017-01-24 10:25:02 +01:00
Jim Ferenczi b0c2a5da30 Remove unused field in CollapseBuilder 2017-01-24 09:26:23 +01:00
Jim Ferenczi 868b12b548 Add BWC tests for field collapsing
Field collapsing is supported from version 5.3
2017-01-24 08:34:16 +01:00
Tim Brooks 7f20b93051 Use generic interfaces for checking socket access (#22753)
This commit replaces specialized functional interfaces in various
plugins with generic options. Instead of creating `StorageRunnable`
interfaces in every plugin we can just use `Runnable` or `CheckedRunnable`.
2017-01-23 16:34:24 -06:00
Nik Everett 28cfc533e2 Generate javadoc jar for painless's public API (#22704)
The simplest way to do that is to move the public API into a
new package and generate javadoc for that package.
2017-01-23 17:16:20 -05:00
Nik Everett 2e399e5505 Rename constant
It deserves a new name after the cleanup in #22749
2017-01-23 16:30:42 -05:00
javanna 8065531236 [TEST] add test to verify that the SMILE format works within the _bulk api 2017-01-23 19:40:24 +01:00
Nik Everett ee264c6957 Fix parsing for `max_determinized_states` (#22749)
There was a typo in the `ParseField` declaration. I know
we want to port these parsers to `ObjectParser` eventually
but I don't have the energy for that today and want to get
this fixed.

Closes #22722
2017-01-23 11:57:43 -05:00
William Webber f1a902865f Update span-multi-term-query.asciidoc (#22733)
"term" is not actually a multi-term query (perhaps confusion with "term range")
2017-01-23 17:33:40 +01:00
William Webber abaf728882 "from" => "gte", "to" => "lte" in bool example (#22735) 2017-01-23 17:29:00 +01:00
Jason Tedor d9a3ffcea0 Add note regarding custom jvm.options
When users need to specify a custom location for configuration files,
they also need to specify a custom location for the jvm.options file yet
our docs are absent in this regard. This commit adds a note to the
rolling upgrade docs explaining this situation.

Relates #22747
2017-01-23 10:39:11 -05:00
Jim Ferenczi e48bc2eed7 Add field collapsing for search request (#22337)
* Add top hits collapsing to search request

The field collapsing is done with a custom top docs collector that "collapse" search hits with same field value.
The distributed aspect is resolve using the two passes that the regular search uses. The first pass "collapse" the top hits, then the coordinating node merge/collapse the top hits from each shard.

```
GET _search
{
   "collapse": {
      "field": "category",
   }
}
```

This change also adds an ExpandCollapseSearchResponseListener that intercepts the search response and expands collapsed hits using the CollapseBuilder#innerHit} options.
The retrieval of each inner_hits is done by sending a query to all shards filtered by the collapse key.

```
GET _search
{
   "collapse": {
      "field": "category",
      "inner_hits": {
	"size": 2
      }
   }
}
```
2017-01-23 16:33:51 +01:00
Tanguy Leroux 11164b394b Add unit tests for ValueCountAggregator and InternalValueCount (#22741)
Adds unit tests for the value count aggregator.

Relates #22278
2017-01-23 16:24:55 +01:00
Nik Everett da8740128b Docs: CONSOLE-ify value_count aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the snippets
in the `value_count` docs and causes the build to execute the snippets
for testing.

Release #18160
2017-01-23 10:07:29 -05:00
markharwood 87495750ff Docs fix - Added missing link to new Adjacency-matrix agg 2017-01-23 10:18:30 +00:00
Simon Willnauer 27b5c2ad54 Pass `forceExecution` flag to transport interceptor (#22739)
To effectively allow a plugin to intercept a transport handler it needs
to know if the handler must be executed even if there is a rejection on the
thread pool in the case the wrapper forks a thread to execute the actual handler.
2017-01-23 11:04:27 +01:00
Alexander Reelsen 6159ca28ae Version: Add missing releases from 2.x in Version.java (#22594) 2017-01-23 09:53:21 +01:00
Nik Everett a99bddcc7e CONSOLE-ify filter aggregation docs
This adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the
snippet and causes the build to execute the snippet as a test.

Relates to #18160
2017-01-23 01:32:56 -05:00
Nik Everett 40e2645177 CONSOLE-ify date_range aggregation docs
This adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the
snippets in the docs for the `date_range` aggregation and tests
those snippets as part of the build.

Relates to #18160
2017-01-22 23:38:45 -05:00
Tim Brooks a4ac29c005 Add single static instance of SpecialPermission (#22726)
This commit adds a SpecialPermission constant and uses that constant
opposed to introducing new instances everywhere.

Additionally, this commit introduces a single static method to check that
the current code has permission. This avoids all the duplicated access
blocks that exist currently.
2017-01-21 12:03:52 -06:00
Simon Willnauer 3ad6d6ebcc Simplify InternalEngine#innerIndex (#22721)
Today `InternalEngine#innerIndex` is a pretty big method (> 150 SLoC). This
commit merged `#index` and `#innerIndex` and splits it up into smaller contained
methods.
2017-01-21 08:51:35 +01:00
Jim Ferenczi 8028578305 Upgrade to Lucene 6.4.0 (#22724)
* Upgrade to Lucene 6.4.0

`ValueSource`s are now converted to `DoubleValueSource`s using the Lucene adapter made for the migration to the new API in 6.4.0.
2017-01-21 04:48:01 +01:00
Igor Motov cfb415de7b Fix broken TaskInfo.toString()
Related to #22387
2017-01-20 20:57:42 -05:00
Tim Brooks d86f97c428 Add CheckedSupplier and CheckedRunnable to core (#22725)
Introduce CheckedSupplier and CheckedRunnable functional interfaces
into core. These offer a checked version of the Supplier and Runnable
interfaces for use with lambda apis.
2017-01-20 19:17:44 -06:00
Jason Tedor 8f6c074691 Revert "Make build Gradle 2.14 / 3.x compatible (#22669)"
This reverts commit 652cb7dbf7.

Relates #22727
2017-01-20 18:16:45 -05:00
Ali Beyad 3bf06d1440 Fixes retrieval of the latest snapshot index blob (#22700)
This commit ensures that the index.latest blob is first examined to
determine the latest index-N blob id, before attempting to list all
index-N blobs and picking the blob with the highest N.

It also fixes the MockRepository#move so that tests are able to handle
non-atomic moves.  This is done by adding a special setting to the
MockRepository that requires the test to specify if it can handle
non-atomic moves.  If so, then the MockRepository#move operation will be
non-atomic to allow testing for against such repositories.
2017-01-20 17:00:46 -06:00
Nik Everett f7524fbdef CONSOLE-ify date histogram docs
This adds the `VIEW IN SENSE` and `COPY AS CURL` links and has
the build automatically execute the snippets and verify that they
work.

Relates to #18160
2017-01-20 16:23:28 -05:00
Nik Everett c2a580304b CONSOLE-ify min and max aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the docs
and makes the build automatically test them.

Relates to #18160
2017-01-20 15:33:00 -05:00
Nik Everett 8c856eaa9f CONSOLE-ify global-aggregation.asciidoc
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the example
`global` aggregation. Also improves the example by adding a
non-`global` aggregation to compare it to.

Relates to #18160
2017-01-20 14:36:51 -05:00
Jim Ferenczi 4ec4bad908 Fix script score function that combines _score and weight (#22713)
The weight factor function does not check if the delegate score function needs to access the score of the query.
This results in a _score equals to 0 for all score function that set a weight.
This change modifies the WeightFactorFunction#needsScore to delegate the call to its underlying score function.

Fix #21483
2017-01-20 19:50:57 +01:00
Folusho Oladipo daf1f184d4 Corrected a plural verb to a singular one. (#22681) 2017-01-20 19:00:10 +01:00
Patryk Krawaczyński 42c0e8947f Fix duplicates from search.query (#22701)
search.query_current, search.query_time and search.query_total have wrong aliases.
2017-01-20 18:45:10 +01:00
Nik Everett 00707b0b0d Readd unconverted snippets mark for doc
I accidentally removed it when cleaning up something else.
2017-01-20 12:39:37 -05:00
Nik Everett 6265ef1c1b Deguice rest handlers (#22575)
There are presently 7 ctor args used in any rest handlers:
* `Settings`: Every handler uses it to initialize a logger and
  some other strange things.
* `RestController`: Every handler registers itself with it.
* `ClusterSettings`: Used by `RestClusterGetSettingsAction` to
  render the default values for cluster settings.
* `IndexScopedSettings`: Used by `RestGetSettingsAction` to get
  the default values for index settings.
* `SettingsFilter`: Used by a few handlers to filter returned
  settings so we don't expose stuff like passwords.
* `IndexNameExpressionResolver`: Used by `_cat/indices` to
  filter the list of indices.
* `Supplier<DiscoveryNodes>`: Used to fill enrich the response
  by handlers that list tasks.

We probably want to reduce these arguments over time but
switching construction away from guice gives us tighter
control over the list of available arguments.

These parameters are passed to plugins using
`ActionPlugin#initRestHandlers` which is expected to build and
return that handlers immediately. This felt simpler than
returning an reference to the ctors given all the different
possible args.

Breaks java plugins by moving rest handlers off of guice.
2017-01-20 11:48:51 -05:00
Nik Everett 025b1a0fc5 Remove unused scripts from docs build
These were added in 5.x to support testing the lang-python and
lang-js plugins which have since been removed.
2017-01-20 11:31:52 -05:00
Simon Willnauer 824beea89d Fix handling of document failure expcetion in InternalEngine (#22718)
Today we try to be smart and make a generic decision if an exception should
be treated as a document failure but in some cases concurrency in the index writer
make this decision very difficult since we don't have a consistent state in the case
another thread is currently failing the IndexWriter/InternalEngine due to a tragic event.

This change simplifies the exception handling and makes specific decisions about document failures
rather than using a generic heuristic. This prevent exceptions to be treated as document failures
that should have failed the engine but backed out of failing since since some other thread has
already taken over the failure procedure but didn't finish yet.
2017-01-20 16:55:00 +01:00
markharwood f01784205f New AdjacencyMatrix aggregation
Similar to the Filters aggregation but only supports "keyed" filter buckets and automatically "ANDs" pairs of filters to produce a form of adjacency matrix.
The intersection of buckets "A" and "B" is named "A&B" (the choice of separator is configurable). Empty intersection buckets are removed from the final results.

Closes #22169
2017-01-20 15:49:31 +00:00
Tim Brooks bc16162d21 Remove accept SocketPermissions from core (#22622)
This is related to #22116. Core no longer needs SocketPermission 
accept. This permission is relegated to the transport-netty4 module 
and (for tests) to the mocksocket jar.
2017-01-20 09:27:45 -06:00
Tanguy Leroux 239ed0c912 Add unit tests for DateHistogramAggregator (#22714)
Adds unit tests for the date histogram aggregator.

Relates #22278
2017-01-20 14:18:30 +01:00
Daniel Mitterdorfer 4073349267 Add example configuration for rotating GC log (#22715)
By default, the JVM GC log file grows without
limitation. This is inconvenient for a long running
process like Elasticsearch.

With this commit we add an example configuration
for a rotating GC log in `conig/jvm.options`.
2017-01-20 13:23:04 +01:00
Christoph Büscher 54105f3ddd Add parsing from xContent to ShardSearchFailure (#22699)
In preparation for being able to parse SearchResponse from its rest
representation, this adds fromXContent to ShardSearchFailure.
2017-01-20 12:49:54 +01:00
Yannick Welsch 1f0e0a2170 Close InputStream when receiving cluster state in PublishClusterStateAction (#22711)
Not closing the InputStream will leak native memory as the DeflateCompressor/Inflater won't be closed.
2017-01-20 12:26:07 +01:00
Boaz Leskes 5d806bf93e Index creation and setting update may not return deprecation logging (#22702)
Those services validate their setting before submitting an AckedClusterStateUpdateTask to the cluster state service. An acked cluster state may be completed by a networking thread when the last acks as received. As such it needs special care to make sure that thread context headers are handled correctly.
2017-01-20 10:14:13 +01:00
David Pilato fc4dc5ef21 Fix comment 2017-01-20 10:13:13 +01:00