Commit Graph

26474 Commits

Author SHA1 Message Date
Nik Everett f7524fbdef CONSOLE-ify date histogram docs
This adds the `VIEW IN SENSE` and `COPY AS CURL` links and has
the build automatically execute the snippets and verify that they
work.

Relates to #18160
2017-01-20 16:23:28 -05:00
Nik Everett c2a580304b CONSOLE-ify min and max aggregation docs
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the docs
and makes the build automatically test them.

Relates to #18160
2017-01-20 15:33:00 -05:00
Nik Everett 8c856eaa9f CONSOLE-ify global-aggregation.asciidoc
Adds the `VIEW IN CONSOLE` and `COPY AS CURL` links to the example
`global` aggregation. Also improves the example by adding a
non-`global` aggregation to compare it to.

Relates to #18160
2017-01-20 14:36:51 -05:00
Jim Ferenczi 4ec4bad908 Fix script score function that combines _score and weight (#22713)
The weight factor function does not check if the delegate score function needs to access the score of the query.
This results in a _score equals to 0 for all score function that set a weight.
This change modifies the WeightFactorFunction#needsScore to delegate the call to its underlying score function.

Fix #21483
2017-01-20 19:50:57 +01:00
Folusho Oladipo daf1f184d4 Corrected a plural verb to a singular one. (#22681) 2017-01-20 19:00:10 +01:00
Patryk Krawaczyński 42c0e8947f Fix duplicates from search.query (#22701)
search.query_current, search.query_time and search.query_total have wrong aliases.
2017-01-20 18:45:10 +01:00
Nik Everett 00707b0b0d Readd unconverted snippets mark for doc
I accidentally removed it when cleaning up something else.
2017-01-20 12:39:37 -05:00
Nik Everett 6265ef1c1b Deguice rest handlers (#22575)
There are presently 7 ctor args used in any rest handlers:
* `Settings`: Every handler uses it to initialize a logger and
  some other strange things.
* `RestController`: Every handler registers itself with it.
* `ClusterSettings`: Used by `RestClusterGetSettingsAction` to
  render the default values for cluster settings.
* `IndexScopedSettings`: Used by `RestGetSettingsAction` to get
  the default values for index settings.
* `SettingsFilter`: Used by a few handlers to filter returned
  settings so we don't expose stuff like passwords.
* `IndexNameExpressionResolver`: Used by `_cat/indices` to
  filter the list of indices.
* `Supplier<DiscoveryNodes>`: Used to fill enrich the response
  by handlers that list tasks.

We probably want to reduce these arguments over time but
switching construction away from guice gives us tighter
control over the list of available arguments.

These parameters are passed to plugins using
`ActionPlugin#initRestHandlers` which is expected to build and
return that handlers immediately. This felt simpler than
returning an reference to the ctors given all the different
possible args.

Breaks java plugins by moving rest handlers off of guice.
2017-01-20 11:48:51 -05:00
Nik Everett 025b1a0fc5 Remove unused scripts from docs build
These were added in 5.x to support testing the lang-python and
lang-js plugins which have since been removed.
2017-01-20 11:31:52 -05:00
Simon Willnauer 824beea89d Fix handling of document failure expcetion in InternalEngine (#22718)
Today we try to be smart and make a generic decision if an exception should
be treated as a document failure but in some cases concurrency in the index writer
make this decision very difficult since we don't have a consistent state in the case
another thread is currently failing the IndexWriter/InternalEngine due to a tragic event.

This change simplifies the exception handling and makes specific decisions about document failures
rather than using a generic heuristic. This prevent exceptions to be treated as document failures
that should have failed the engine but backed out of failing since since some other thread has
already taken over the failure procedure but didn't finish yet.
2017-01-20 16:55:00 +01:00
markharwood f01784205f New AdjacencyMatrix aggregation
Similar to the Filters aggregation but only supports "keyed" filter buckets and automatically "ANDs" pairs of filters to produce a form of adjacency matrix.
The intersection of buckets "A" and "B" is named "A&B" (the choice of separator is configurable). Empty intersection buckets are removed from the final results.

Closes #22169
2017-01-20 15:49:31 +00:00
Tim Brooks bc16162d21 Remove accept SocketPermissions from core (#22622)
This is related to #22116. Core no longer needs SocketPermission 
accept. This permission is relegated to the transport-netty4 module 
and (for tests) to the mocksocket jar.
2017-01-20 09:27:45 -06:00
Tanguy Leroux 239ed0c912 Add unit tests for DateHistogramAggregator (#22714)
Adds unit tests for the date histogram aggregator.

Relates #22278
2017-01-20 14:18:30 +01:00
Daniel Mitterdorfer 4073349267 Add example configuration for rotating GC log (#22715)
By default, the JVM GC log file grows without
limitation. This is inconvenient for a long running
process like Elasticsearch.

With this commit we add an example configuration
for a rotating GC log in `conig/jvm.options`.
2017-01-20 13:23:04 +01:00
Christoph Büscher 54105f3ddd Add parsing from xContent to ShardSearchFailure (#22699)
In preparation for being able to parse SearchResponse from its rest
representation, this adds fromXContent to ShardSearchFailure.
2017-01-20 12:49:54 +01:00
Yannick Welsch 1f0e0a2170 Close InputStream when receiving cluster state in PublishClusterStateAction (#22711)
Not closing the InputStream will leak native memory as the DeflateCompressor/Inflater won't be closed.
2017-01-20 12:26:07 +01:00
Boaz Leskes 5d806bf93e Index creation and setting update may not return deprecation logging (#22702)
Those services validate their setting before submitting an AckedClusterStateUpdateTask to the cluster state service. An acked cluster state may be completed by a networking thread when the last acks as received. As such it needs special care to make sure that thread context headers are handled correctly.
2017-01-20 10:14:13 +01:00
David Pilato fc4dc5ef21 Fix comment 2017-01-20 10:13:13 +01:00
David Pilato ad5b8def26 Merge branch 'pr/delete-from-xcontent' 2017-01-20 09:16:34 +01:00
Lee Hinman eb8a41ef94 Add missing serialization BWC for disk usage estimates
Relates to #22081
2017-01-19 15:37:06 -07:00
Nik Everett 296f4aac7a Mechanical changes to make PainlessLexer a bit more obvious (#22695)
Makes `PainlessLexer` abstract and adds the hacks that it needs as abstract methods implemented in `EnhancedPainlessLexer`. This feels a little cleaner than referencing the hacks statically.
2017-01-19 17:31:12 -05:00
Lee Hinman 4eb32e9d86 Expose disk usage estimates in nodes stats
This exposes the least and most used disk usage estimates within the "fs" nodes
stats output:

```json
GET /_nodes/stats/fs?pretty&human
{
  "nodes" : {
    "34fPVU0uQ_-wWitDzDXX_g" : {
      "fs" : {
        "timestamp" : 1481238723550,
        "total" : {
          "total" : "396.1gb",
          "total_in_bytes" : 425343254528,
          "free" : "140.6gb",
          "free_in_bytes" : 151068725248,
          "available" : "120.5gb",
          "available_in_bytes" : 129438912512
        },
        "least_usage_estimate" : {
          "path" : "/home/hinmanm/es/elasticsearch/distribution/build/cluster/run node0/elasticsearch-6.0.0-alpha1-SNAPSHOT/data/nodes/0",
          "total" : "396.1gb",
          "total_in_bytes" : 425343254528,
          "available" : "120.5gb",
          "available_in_bytes" : 129438633984,
          "used_disk_percent" : 69.56842912023208
        },
        "most_usage_estimate" : {
          "path" : "/home/hinmanm/es/elasticsearch/distribution/build/cluster/run node0/elasticsearch-6.0.0-alpha1-SNAPSHOT/data/nodes/0",
          "total" : "396.1gb",
          "total_in_bytes" : 425343254528,
          "available" : "120.5gb",
          "available_in_bytes" : 129438633984,
          "used_disk_percent" : 69.56842912023208
        },
        "data" : [{...}],
        "io_stats" : {...}
      }
    }
  }
}
```

Resolves #8686
Resolves #22081
2017-01-19 13:56:52 -07:00
Ryan Ernst c5b4bba30b S3 repository: Deprecate specifying credentials through env vars, sys props, and remove profile files (#22567)
* S3 repository: Deprecate specifying credentials through env vars and sys props

This is a follow up to #22479, where storing credentials secure way was
added.
2017-01-19 12:36:32 -08:00
Nik Everett 3ea37ec817 Fix Eclipse project generation
The new plugin causes Eclipse to go a bit crazy. So we skip it
for Eclipse.

Relates to #22646
2017-01-19 13:13:11 -05:00
Jason Tedor 9781b88a38 Fix deprecation logging for lenient booleans
This commit fixes an issue with deprecation logging for lenient
booleans. The underlying issue is that adding deprecation logging for
lenient booleans added a static deprecation logger to the Settings
class. However, the Settings class is initialized very early and in CLI
tools can be initialized before logging is initialized. This leads to
status logger error messages. Additionally, the deprecation logging for
a lot of the settings does not provide useful context (for example, in
the token filter factories, the deprecation logging only produces the
name of the setting, but gives no context which token filter factory it
comes from). This commit addresses both of these issues by changing the
call sites to push a deprecation logger through to the lenient boolean
parsing.

Relates #22696
2017-01-19 12:30:33 -05:00
Nik Everett 22f1c9fa0f Remove @header we no longer need 2017-01-19 11:44:13 -05:00
Nik Everett bb83c283bb Make lexer abstract 2017-01-19 11:41:50 -05:00
Yannick Welsch 881993de3a [Docs] Remove outdated info about enabling/disabling doc_values (#22694) 2017-01-19 17:33:40 +01:00
David Pilato 5be8bd76e2 Also test found field
And optimize imports
2017-01-19 17:28:31 +01:00
Nik Everett dbb4a2ca6c Move lexer hacks to EnhancedPainlessLexer
This "feels" nicer. Less classes at least.
2017-01-19 11:23:16 -05:00
Tim Brooks 3deae99a34 Fix incorrect args order passed to createAggregator
This commit fixes a compile issue where the arguments are passed to
createAggregator in the incorrect order.
2017-01-19 10:08:38 -06:00
Nik Everett e2da6a8ee5 Improve painless's javadocs
Hopefully useful references.
2017-01-19 11:04:08 -05:00
Tim Brooks a10aa8aade Add TestWithDependenciesPlugin to build (#22646)
This commit adds a MessyRestTestPlugin to the gradle build. It extends 
StandaloneRestTestPlugin. The main piece of functionality that it adds 
is to copy plugin-metadata from dependencies into the 
generated-resources for the current test source. This is necessary to 
ensure that permissions for dependencies are applied when running the 
tests.

A current limitation is that the permissions are applied differently 
than in the distribution sources. When permissions are granted to all 
depedencies for a module or plugin, the permissions are granted to all 
dependencies on the classpath for tests besides a few hardcoded 
exclusions:
- es core
- es test framework
- lucene test framework
- randomized runner
- junit library
2017-01-19 09:43:53 -06:00
Christoph Büscher e03554070c Add parsing from xContent to SearchProfileShardResults and nested classes (#22649)
In preparation for being able to parse SearchResponse from its rest representation
for the java rest client, this adds fromXContent to SearchProfileShardResults and its
nested classes.
2017-01-19 16:29:10 +01:00
Jim Ferenczi b781a4a176 Add unit tests for FiltersAggregator (#22678)
Adds unit tests for the `filters` aggregation.
This change also adds an helper to search and reduce any aggregator in a unit test.
This is done by dividing a single searcher in sub-searcher, one for each segment.

Relates #22278
2017-01-19 16:22:48 +01:00
David Pilato 0315dcc306 Use now common methods with index/update
Brought by #22229
2017-01-19 16:10:13 +01:00
Jim Ferenczi 3d54258de2 Don't register search response listener in transport clients
Small fix for https://github.com/elastic/elasticsearch/pull/22682
2017-01-19 16:08:24 +01:00
David Pilato 718a6b9be7 Add fromxcontent methods to delete response
This commit adds the parsing fromXContent() methods to the IndexResponse class.

It's a pale copy of what has been done in #22229.
2017-01-19 15:59:24 +01:00
Nicholas Knize b006636aaf unmute FieldStatsIntegrationIT.testGeoPointNotIndexed, fix already pushed 2017-01-19 08:44:00 -06:00
Nicholas Knize 88c78833f0 Mute FieldStatsIntegrationIT.testGeoPointNotIndexed, for now 2017-01-19 08:38:17 -06:00
Nik Everett 3ce41a0e15 Painless: Add augmentation to string for base 64 (#22665)
We don't want to expose `String#getBytes` which is required for
`Base64.getEncoder.encode` to work because we're worried about
character sets. This adds `encodeBase64` and `decodeBase64`
methods to `String` in Painless that are duals of one another
such that:
`someString == someString.encodeBase64().decodeBase64()`.

Both methods work with the UTF-8 encoding of the string.

Closes #22648
2017-01-19 09:31:45 -05:00
Jim Ferenczi d145d459ae Fix NPE on FieldStats with mixed cluster on version pre/post 5.2 (#22688)
* Fix NPE on FieldStats with mixed cluster on version pre/post 5.2

In 5.2 the FieldStats API can return null min/max values.
These values cannot be deserialized by a node with version pre 5.2 so if this node
is pick to coordinate a FieldStats request in a mixed cluster an NPE can be thrown.
This change prevents the NPE by removing the non serializable FieldStats object directly in the field stats shard request.
The filtered fields will not be present in the response when a node pre 5.2 acts as a coordinating node.
2017-01-19 14:20:07 +01:00
Tanguy Leroux 833284cae2 Add parsing methods for UpdateResponse (#22586)
This commit adds the fromXContent() method to the UpdateResponse class, so that it can be used with the high level rest client.
2017-01-19 12:49:45 +01:00
Jim Ferenczi 21dae1924f Add the ability to define search response listeners in plugins (#22682)
This change is a simple adaptation of https://github.com/elastic/elasticsearch/pull/19587 for the current state of master.
It allows to define search response listener in the form of `BiConsumer<SearchRequest, SearchResponse>`s in a search plugin.
2017-01-19 12:48:45 +01:00
Yannick Welsch 652cb7dbf7 Make build Gradle 2.14 / 3.x compatible (#22669)
This changes build files so that building Elasticsearch works with both Gradle 2.13 as well as higher versions of Gradle (tested 2.14 and 3.3), enabling a smooth transition from Gradle 2.13 to 3.x.
2017-01-19 09:56:54 +01:00
Daniel Mitterdorfer ce765f7ad2 Use a proper boolean in FieldStatsIntegrationIT#testGeoPointNotIndexed() 2017-01-19 08:33:08 +01:00
Daniel Mitterdorfer aece89d6a1 Make boolean conversion strict (#22200)
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.
2017-01-19 07:59:18 +01:00
Nik Everett ee5f8c4522 Consolidate some reindex utility classes (#22666)
Everything that extended `AbstractAsyncBulkByScrollAction` also
extended `AbstractAsyncBulkIndexByScrollAction` so this removes
`AbstractAsyncBulkIndexByScrollAction`, merging it into
`AbstractAsyncBulkByScrollAction`.
2017-01-18 16:58:39 -05:00
Nicholas Knize 51e80e7176 remove unnecessary text from exception message 2017-01-18 14:51:56 -06:00
Nicholas Knize 84e4f91253 Add geo_point to FieldStats
This commit adds a new GeoPoint class to FieldStats for computing field stats over geo_point field types.
2017-01-18 14:37:03 -06:00