Commit Graph

23272 Commits

Author SHA1 Message Date
Ryan Ernst 62397c0c3d Fix node service to still work when http is disabled 2016-07-01 12:55:41 -07:00
Ryan Ernst 6110dcc710 Fix unit tests for http server that no longer need node service 2016-07-01 12:45:43 -07:00
Ryan Ernst f6491047a0 Fix line lengths for ingest config utils 2016-07-01 12:36:44 -07:00
Ryan Ernst e5caadc4f3 Merge branch 'master' into ingest_plugin_api 2016-07-01 12:35:26 -07:00
Ryan Ernst a200292211 Merge pull request #19218 from rjernst/http_server_without_node_service
Internal: Remove cyclic dependency between HttpServer and NodeService
2016-07-01 12:32:52 -07:00
Nik Everett de71a9abb3 Migrate value_count, percentiles, and percentile_ranks aggregations to NamedWriteable
These are the first aggregations with multiple `InternalAggregation`s
backing the same `AggregationBuilder`. This required a change in the
register method's signature.
2016-07-01 14:48:08 -04:00
Nik Everett f30a70c51f Fix comment
I forgot a word....
2016-07-01 14:48:08 -04:00
Ali Beyad cb20776439 Includes the index UUID in the _cat/indices API and adds tests
for the _cat/indices functionality.

Closes #19204
Closes #19132
2016-07-01 14:45:55 -04:00
Nik Everett ff42d7cfc6 Add embedded stash key support to rest tests
This allowes embedding stash keys in string like `t${key}est`. This
allows simple string concatenation like acitons.

The test for this is in `ObjectPathTests` because `Stash` doesn't seem
to have a test on its own and it is simple enough to test embedded
stashes this way. And this is a way I expect them to be used eventually.
2016-07-01 14:11:11 -04:00
Ryan Ernst a206c7a149 Remove unnecessary transport level bwc 2016-07-01 10:16:56 -07:00
Ryan Ernst dbbfadeefa Internal: Remove cyclic dependency between HttpServer and NodeService
NodeService has an "service attributes" map, which is only
set by HttpServer on start/stop. But the only thing it puts in this map
is already available as part of the HttpServer info which is added to
node info requests. This change removes the attributes map and removes
the dependency in HttpServer on NodeService.
2016-07-01 09:50:54 -07:00
Ryan Ernst 76ba10bab6 Remove unnecessary optional injection of ScriptService into NodeService 2016-07-01 09:31:41 -07:00
Ryan Ernst 65c9b0b588 Merge branch 'master' into ingest_plugin_api 2016-07-01 09:26:17 -07:00
Tal Levy 01d7020ee3 Skip the execution of an empty pipeline (#19200)
main optimization: `sourceToMap` is not called, therefore avoiding creation of Map of Maps
Closes #19192.
2016-07-01 09:15:05 -07:00
Nik Everett b66fc308fc Add ingest-useragent to vagrant tests 2016-07-01 11:29:48 -04:00
Tanguy Leroux 5c7ca5cc0c Fix checkstyle violations 2016-07-01 17:24:43 +02:00
Tanguy Leroux 0a293fad29 Remove some unused code 2016-07-01 17:01:39 +02:00
Tanguy Leroux 8c40b2b54e Fix order of modifiers 2016-07-01 16:57:14 +02:00
Tanguy Leroux 93b42b8e69 Enable checkstyle ModifierOrder module 2016-07-01 16:57:14 +02:00
Christoph Wurm 368c3ccc54 Fix Factory test 2016-07-01 16:53:17 +02:00
Christoph Wurm 4c32b025d4 Fix Processor: Now implements Processor.Factory 2016-07-01 16:13:51 +02:00
Simon Willnauer 5c8164a561 Clean up BytesReference (#19196)
BytesReference should be a really simple interface, yet it has a gazillion
ways to achieve the same this. Methods like `#hasArray`, `#toBytesArray`, `#copyBytesArray`
`#toBytesRef` `#bytes` are all really duplicates. This change simplifies the interface
dramatically and makes implementations of it much simpler. All array access has been removed
and is streamlined through a single `#toBytesRef` method. Utility methods to materialize a
compact byte array has been added too for convenience.
2016-07-01 16:09:31 +02:00
Christoph Wurm 42addb5692 Add ingest-useragent plugin (#19074) 2016-07-01 15:49:43 +02:00
Nik Everett 27e320d5ce Migrate sum, min, and max aggs to NamedWriteable 2016-07-01 09:23:26 -04:00
Nik Everett 91b66e3cf4 Migration stats and extended stats to NamedWriteable
Migrates the `stats` and `extended_stats` aggregations and pipeline
aggregations from the special purpose aggregations streams to
`NamedWriteable`. These are the first pipeline aggregations so this
adds the infrastructure to support both streams and `NamedWriteable`s
for pipeline aggregations.
2016-07-01 09:13:15 -04:00
Leon Weidauer 1297a707da non-binary gender option in term aggr. example (#19188)
* non-binary gender option in term aggr. example

* replace gender with music genre for term aggregation docs
2016-07-01 14:59:03 +02:00
Luca Cavanna 4d67b1fe5e Merge pull request #19206 from javanna/fix/client_group_id
Build: set group for client and sniffer, disable publishing for client-test
2016-07-01 14:28:53 +02:00
javanna c4b87149c2 Build: set group for client and sniffer, disable publishing for client-test
Closes #19205
2016-07-01 12:09:34 +02:00
Luca Cavanna b5eee760f4 Merge pull request #19091 from javanna/test/nuke_response_body_assertion
Consolidate docs snippets testing in our REST test infra
2016-07-01 12:08:52 +02:00
javanna dd781d410a fix line length problems in all classes under o.e.test.rest package 2016-07-01 11:13:10 +02:00
javanna 0b5a549305 [TEST] remove special treatment for stashed $body in REST tests, instead always evaluate the stash through ObjectPath
When we introduced docs testing we added a special case for $body in Stash, so that the last stashed body could be evaluated, and expressions like "$body.took" could be extracted out of it. We can instead do that for any object in the stash, by simply wrapping the internal map in an ObjectPath instance. We can then drop the special stashResponse method and go back to using the ordinary stashValue too.

The downside of this change is that it adds a feature that may not be supported by other REST test runners, namely the evaluation of compouned paths from the stash. If we have "object" stashed as an object, it is now possible to extract directly each subobject of it as well e.g. "object.subobject.field1". None of the current REST tests rely on this, but our docs snippets tests do.
2016-07-01 11:13:10 +02:00
javanna 43b82ce244 [TEST] remove feature yaml from REST tests
The only runner that supported it was the java runner, we can use json format instead given that the default one with cat apis is text
2016-07-01 11:13:10 +02:00
javanna 60bafa5d78 [TEST] parse yaml responses too through ObjectPath rather than only json responses
No need to match against yaml responses via regexes in REST tests, yaml responses can be properly parsed via ObjectPath instead. Few REST tests need to be updated accordingly.
2016-07-01 11:13:10 +02:00
javanna 34f5c50a7f [TEST] eagerly parse response body at ObjectPath initialization and read content type from response headers
We are going to parse the body anyways whenever it's in json format as it is going to be stashed. It is not useful to lazily parse it anymore. Also this allows us to not rely on automatic detection of the xcontent type based on the content of the response, but rather read the content type from the response headers.
2016-07-01 11:13:10 +02:00
javanna d5df738538 [TEST] ObjectPath to support parsing yaml or json that have an array as root object
ObjectPath used a Map up until now for the internal representation of its navigable object. That works in most of the cases, but there could also be an array as root object, in which case a List needs to be used instead of a Map. This commit changes the internal representation of the object to Object which can either be a List or a Map. The change is minimal as ObjectPath already had the checks in place to verify the type of the object in the current position and navigate through it.

  Note: The new test added to ObjectPathTest uses yaml format explicitly as auto-detection of json format works only for a json object that starts with '{', not if the root object is actually an array and starts with '['.
2016-07-01 11:13:10 +02:00
javanna bbaa23bdfd [TEST] extend ObjectPathTests to support also yaml format 2016-07-01 11:13:10 +02:00
javanna 44dc801e90 [TEST] make JsonPath independent of data format, rename to ObjectPath
The internal representation of the object that JsonPath gives access to is a map. That is independent of the initial input format, which is json but could also be yaml etc.
This commit renames JsonPath to ObjectPath and adds a static method to create an ObjectPath from an XContent
2016-07-01 11:13:10 +02:00
javanna 76199ce497 [TEST] rename REST tests Stash methods to distinguish between retrieving a value and replacing values within a map
Stash#unstashMap -> replaceStashedValues
Stash#unstashValue -> getValue
2016-07-01 11:13:10 +02:00
javanna 62462f5d9b [TEST] replace ResponseBodyAssertion with existing MatchAssertion
We introduced a special response_body assertion to test our docs snippets. The match assertion does the same job though and can be reused and adapted where needed. ResponseBodyAssertion contains provides much better and accurate errors though, which can be now utilized in MatchAssertion so that many more REST tests can benefit from readable error messages.

 Each response body gets always stashed and can be retrieved for later evaluations already. Instead of providing the response body as strings that get parsed to json objects separately, then converted to maps as ResponseBodyAssertion did, we parse everything once, the json is part of the yaml test, which is supported. The only downside is that json comments cannot be used, rather yaml comments should be used (// C style vs # ). There were only two docs tests that were using comments in ingest-node.asciidoc where I went ahead and remove the comments which didn't seem that useful anyways.
2016-07-01 11:13:10 +02:00
javanna 598c36128e Revert "Raised IOException on deleteBlob (#18815)"
This reverts commit d24cc65cad as it seems to be causing test failures.
2016-07-01 11:00:32 +02:00
Tanguy Leroux 9bfc23e958 Add missing permission to repository-s3
Repository-S3 needs a special permission because of problems in AmazonS3Client: when no region is set on a AmazonS3Client instance, the AWS SDK loads all known partitions from a JSON file and uses a Jackson's ObjectMapper for that: this one, in version 2.5.3 with the default binding options, tries to suppress access checks of ctor/field/method and thus requires this special permission. AWS must be fixed to uses Jackson correctly and have the correct modifiers on binded classes.

This must be fixed in aws sdk (see https://github.com/aws/aws-sdk-java/issues/766) but in the meanwhile we have no choice.

closes #18539
2016-07-01 10:32:32 +02:00
gfyoung d24cc65cad Raised IOException on deleteBlob (#18815)
Raise IOException on deleteBlob if the blob doesn't exist

This commit raises an IOException on BlobContainer#deleteBlob
if the blob does not exist, in conformance with the BlobContainer
interface contract.  Each implementation of BlobContainer now
conforms to this contract (file system, S3, Azure, HDFS).  This 
commit also contains blob container tests for each of the 
repository implementations.

Closes #18530
2016-06-30 23:00:10 -04:00
Jason Tedor a24d302e73 Fix discovery-azure-classic plugin packaging test
This commit fixes the discovery-azure-classing packaging test by fixing
the expected name of the installed plugin.
2016-06-30 14:47:38 -04:00
Ryan Ernst 8275ab497b Merge pull request #19170 from rjernst/rest_handler_client
Changed rest handler interface to take NodeClient
2016-06-30 11:00:09 -07:00
Nik Everett f5a269b029 Start migration away from aggregation streams
We'll migrate to NamedWriteable so we can share code with the rest
of the system. So we can work on this in multiple pull requests without
breaking Elasticsearch in between the commits this change supports
*both* old style `InternalAggregations.stream` serialization and
`NamedWriteable` style serialization. As such it creates about a
half dozen `// NORELEASE` comments that will have to be removed
once the migration is complete.

This also introduces a boolean `transportClient` flag to `SearchModule`
which is used to skip inappropriate registrations for for the
transport client while still registering the things it needs. In
this case that means that the `InternalAggregation` subclasses are
registered with the `NamedWriteableRegistry` but the `AggregationBuilder`
subclasses are not.

Finally, this moves aggregation registration from guice configuration
time to `SearchModule` construction time. This will make it simpler to
work with in the future as we further clean up Elasticsearch's
extension points.
2016-06-30 12:57:34 -04:00
Clinton Gormley e1ab3f16fd Add link to alpha4 release notes 2016-06-30 18:32:15 +02:00
Boaz Leskes 09ca6d6ed2 Add a BridgePartition to be used by testAckedIndexing (#19172)
We have long worked to capture different partitioning scenarios in our testing infra. This PR adds a new variant, inspired by the Jepsen blogs, which was forgotten far - namely a partition where one node can still see and be seen by all other nodes. It also updates the resiliency page to better reflect all the work that was done in this area.
2016-06-30 17:58:12 +02:00
Jay Modi 5a7dfe78c2 Merge pull request #19190 from jaymode/rest_teardown
Add support for `teardown` section in REST tests
2016-06-30 11:51:11 -04:00
Ryan Ernst 04a4bcdca0 Add comment explaining bytes reference edge case 2016-06-30 08:47:55 -07:00
Ryan Ernst e079c83020 Fix test edge case for bytes reference 2016-06-30 08:45:54 -07:00