Commit Graph

868 Commits

Author SHA1 Message Date
Tanguy Leroux 83aa00b3f6 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-19 13:13:00 +02:00
Ryan Ernst 46530c1cba Build: Fix plugin installation for integ test to have unique configuration name (#24778) 2017-05-18 10:04:52 -07:00
Nik Everett 82d2c7a142 Remove vagrant testing versions (#24754)
Now that we generate the versions list from Versions.java we can
drop the list of versions maintained for vagrant testing. One nice
thing that the vagrant testing did was to check if the list of
versions was out of date. This moves that test to the core
project.
2017-05-18 09:33:13 -04:00
Ryan Ernst 0353bd1fb6 Test: Convert rolling upgrade test to have task per wire compat version (#24758)
This commit changes the rolling upgrade test to create a set of rest
test tasks per wire compat version. The most recent wire compat version
is always tested with the `integTest` task, and all versions can be
tested with `bwcTest`.
2017-05-18 01:14:24 -07:00
Tanguy Leroux eeef2e6c31 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-18 09:43:57 +02:00
Ryan Ernst 463fe2f4d4 Scripting: Remove file scripts (#24627)
This commit removes file scripts, which were deprecated in 5.5.

closes #21798
2017-05-17 14:42:25 -07:00
javanna ce7326eb88 Merge branch 'master' into feature/client_aggs_parsing 2017-05-17 17:59:00 +02:00
Ryan Ernst 2a65bed243 Tests: Change rest test extension from .yaml to .yml (#24659)
This commit renames all rest test files to use the .yml extension
instead of .yaml. This way the extension used within all of
elasticsearch for yaml is consistent.
2017-05-16 17:24:35 -07:00
Nik Everett c38b3360b6 Allow unstashing values into keys (#24685)
This is almost exclusively for docs test which frequently match the
entire response. This allow something like:
```
  - set: {nodes.$master.http.publish_address: host}
  - match:
      $body:
        {
          "nodes": {
            $host: {
              ... stuff in here ...
            }
          }
        }
```

This should make it possible for the docs tests to work with
unpredictable keys.
2017-05-16 11:16:12 -04:00
Tanguy Leroux 5fb04fa603 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-15 17:08:26 +02:00
Jason Tedor bd5aee8cfa Split disruption test suite
The disruption tests sit in a single test suite which causes these tests
to be single-threaded. We can split this test suite into multiple suites
(logically, of course) enabling them to be run in parallel reducing the
total run time of all integration tests in core. This commit splits the
discovery with service disruptions test suite into three suites
 - master disruptions
 - discovery disruptions
 - cluster disruptions

The last one could probably be better named, it is meant to represent
performing actions in the cluster (indexing, failing a shard, etc.)
while a disruption is taking place.

Relates #24662
2017-05-15 07:47:42 -04:00
Christoph Büscher 42e8d4b761 Merge branch 'master' into feature/client_aggs_parsing
Conflicts:
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/filter/InternalFilterTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/global/InternalGlobalTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/missing/InternalMissingTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/InternalNestedTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/nested/InternalReverseNestedTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/sampler/InternalSamplerTests.java
	modules/parent-join/src/test/java/org/elasticsearch/join/aggregations/InternalChildrenTests.java
	test/framework/src/main/java/org/elasticsearch/search/aggregations/InternalSingleBucketAggregationTestCase.java
2017-05-15 12:25:07 +02:00
Jim Ferenczi 279a18a527 Add parent-join module (#24638)
* Add parent-join module

This change adds a new module named `parent-join`.
The goal of this module is to provide a replacement for the `_parent` field but as a first step this change only moves the `has_child`, `has_parent` queries and the `children` aggregation to this module.
These queries and aggregations are no longer in core but they are deployed by default as a module.

Relates #20257
2017-05-12 15:58:06 +02:00
Christoph Büscher 744b1afcb2 Merge branch 'master' into feature/client_aggs_parsing
Conflicts:
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/histogram/InternalHistogramTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/DoubleTermsTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/LongTermsTests.java
	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/StringTermsTests.java
2017-05-12 10:55:05 +02:00
Simon Willnauer 1155615536 Move DeleteByQuery and Reindex requests into core (#24578)
This allows other plugins to use a client to call the functionality
that is in the core modules without duplicating the logic.
Plugins can now safely send the request and response classes via the
client even if the requests are executed locally. All relevant classes
are loaded by the core classloader such that plugins can share them.

This is re-adds this commit that was revered in 952feb58e4
2017-05-11 20:22:30 +02:00
Simon Willnauer 952feb58e4 Revert "Move DeleteByQuery and Reindex requests into core (#24578)"
This reverts commit 6ea2ae32b8.
2017-05-11 18:26:40 +02:00
Simon Willnauer 6ea2ae32b8 Move DeleteByQuery and Reindex requests into core (#24578)
This allows other plugins to use a client to call the functionality
that is in the core modules without duplicating the logic.
Plugins can now safely send the request and response classes via the
client even if the requests are executed locally. All relevant classes
are loaded by the core classloader such that plugins can share them.
2017-05-11 16:20:40 +02:00
Christoph Büscher 570390ac36 Merge branch 'master' into feature/client_aggs_parsing 2017-05-11 11:20:39 +02:00
Christoph Büscher 64ecf4f4bf Bumping version to v6.0.0-alpha2 2017-05-10 17:12:56 +02:00
Tanguy Leroux bf718a686f Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/aggregations/bucket/terms/InternalTermsTestCase.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalExtendedStatsTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalMaxTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalStatsTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/avg/InternalAvgTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/cardinality/InternalCardinalityTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/geobounds/InternalGeoBoundsTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/geocentroid/InternalGeoCentroidTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/min/InternalMinTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/InternalPercentilesTestCase.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/hdr/InternalHDRPercentilesRanksTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/tdigest/InternalTDigestPercentilesRanksTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/sum/InternalSumTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/valuecount/InternalValueCountTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/InternalSimpleValueTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/bucketmetrics/percentile/InternalPercentilesBucketTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/pipeline/derivative/InternalDerivativeTests.java
#	test/framework/src/main/java/org/elasticsearch/test/InternalAggregationTestCase.java
2017-05-10 13:40:39 +02:00
Ryan Ernst 3f1ef488cd fix checkstyle after script engine rename 2017-05-10 01:04:02 -07:00
Christoph Büscher 1959fd6a1a Merge branch 'master' into feature/client_aggs_parsing 2017-05-09 19:54:36 +02:00
Lee Hinman 1907c46689 Refactor UpdateHelper into unit-testable pieces
This starts breaking up the `UpdateHelper.prepare` method so that each piece can
be individually unit tested. No actual functionality has changed.

Note however, that I did add a TODO about `ctx.op` leniency, which I'd love to
remove as a separate PR if desired.
2017-05-08 09:00:47 -06:00
Jason Tedor 4e5537d9ed Add workaround so path.data can be set in run task
Previously you could set the system property tests.es.path.data and
start the run task with a custom data directory. A change in core to
detect duplicate settings broke this. That change should stay, but we
can work around this easily by only setting path.data to the data
directory if tests.es.path.data is not set.
2017-05-05 15:57:54 -04:00
Tanguy Leroux c7c524dd3f Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-05-05 21:54:37 +02:00
Jason Tedor 035494fa17 Remove obsolete JVM options from build
We start the test JVMs with various options. There are two that we
should remove, they do not make sense.
 - we require Java 8 yet there was a conditional build option for Java 7
 - we do not set MaxDirectMemorySize in our default JVM options, we
   should not in the test JVMs either

Relates #24501
2017-05-05 00:49:18 -04:00
Nik Everett 9bc7e210a0 Test: Move flag to painless tests (#24494)
The `-XX:-OmitStackTraceInFastThrow` flag is only required by Painless's
tests so we'll only set it there. This is much simpler.
2017-05-04 13:11:09 -04:00
James Baiera d928ae210d Add Vagrant based testing fixture (#24249) 2017-05-04 10:17:55 -04:00
Dimitrios Liappis ee6deb34d0 Tests: Switch to fedora-25 in test plugin
Use fedora-25 Vagrant box in VagrantTestPlugin, which was missing from
9a3ab3e800 causing packaging test
failures.

Additionally update TESTING.asciidoc
2017-05-04 14:29:35 +03:00
javanna 62c37339b8 Merge branch 'master' into feature/client_aggs_parsing 2017-05-03 22:04:06 +02:00
Jason Tedor 23801153c7 Fix JVM test argline (#24448)
* Fix JVM test argline

The argline was being overridden by '-XX:-OmitStackTraceInFastThrow'
which led to test failures that were expecting the JVM to be in a
certain state based on the value of tests.jvm.argline but they were not
since these arguments were never passed to the JVM. Additionally, we
need to respect the provided JVM argline if it is already provided with
a flag for OmitStackTraceInFastThrow. This commit fixes this by only
setting OmitStackTraceInFastThrow if it is not already set.

* Add comment

* Fix comment

* More elaborate comment

* Sigh
2017-05-02 19:59:07 -04:00
Simon Willnauer 2f9e9460d4 Move RemoteClusterService into TransportService (#24424)
TransportService and RemoteClusterService are closely coupled already today
and to simplify remote cluster integration down the road it can be a direct
dependency of TransportService. This change moves RemoteClusterService into
TransportService with the goal to make it a hidden implementation detail
of TransportService in followup changes.
2017-05-02 18:09:32 +02:00
Nik Everett 3b47355e56 Try not to lose stacktraces (#24426)
This adds `-XX:-OmitStackTraceInFastThrow` to the JVM arguments
which *should* prevent the JVM from omitting stack traces on
common exception sites. Even though these sites are common, we'd
still like the exceptions to debug them.

This also adds the flag when running tests and adapts some tests
that had workarounds for the absense of the flag.

Closes #24376
2017-05-02 11:34:12 -04:00
javanna 60866da4b7 Merge branch 'master' into feature/client_aggs_parsing 2017-05-02 12:48:30 +02:00
David Pilato 186b401fd8 Upgrade to JUnit 4.12 (#23877)
* Upgrade to JUnit 4.12
* Add permission to junit 4.12 and remove junit4-ant specific permission
2017-05-02 09:58:21 +02:00
Jason Tedor 9525c1c4fa Remove duplicates from checkstyle suppressions
This commit removes a few duplicates from the list of checkstyle
suppressions that accumulated while auto-generating the list of
suppressions based on the files that currently exceed the 140-column
limit.
2017-04-30 12:43:04 -04:00
Jason Tedor e339d894c5 Remove errant entry in checkstyle suppressions
The list of checkstyle suppressions was auto-generated based on the list
of files that currently violate the 140-column limit. An errant entry
was committed that did no harm, but this commit removes it.
2017-04-30 12:15:54 -04:00
Jason Tedor a0f4cde905 Adjust checkstyle suppressions to 140-column limit
We are back to having a 140-column limit. While at some point we will
apply auto-formatting tools to the code base, that is a down-the-road
thing and adjusting the suppressions now shaves minutes off the build
time.

Relates #24408
2017-04-30 12:11:57 -04:00
Christoph Büscher db07a34718 Merge branch 'master' into feature/client_aggs_parsing 2017-04-28 16:42:55 +02:00
Yannick Welsch 35f78d098a Separate publishing from applying cluster states (#24236)
Separates cluster state publishing from applying cluster states:

- ClusterService is split into two classes MasterService and ClusterApplierService. MasterService has the responsibility to calculate cluster state updates for actions that want to change the cluster state (create index, update shard routing table, etc.). ClusterApplierService has the responsibility to apply cluster states that have been successfully published and invokes the cluster state appliers and listeners.
- ClusterApplierService keeps track of the last applied state, but MasterService is stateless and uses the last cluster state that is provided by the discovery module to calculate the next prospective state. The ClusterService class is still kept around, which now just delegates actions to ClusterApplierService and MasterService.
- The discovery implementation is now responsible for managing the last cluster state that is used by the consensus layer and the master service. It also exposes the initial cluster state which is used by the ClusterApplierService. The discovery implementation is also responsible for adding the right cluster-level blocks to the initial state.
- NoneDiscovery has been renamed to TribeDiscovery as it is exclusively used by TribeService. It adds the tribe blocks to the initial state.
- ZenDiscovery is synchronized on state changes to the last cluster state that is used by the consensus layer and the master service, and does not submit cluster state update tasks anymore to make changes to the disco state (except when becoming master).

Control flow for cluster state updates is now as follows:

- State updates are sent to MasterService
- MasterService gets the latest committed cluster state from the discovery implementation and calculates the next cluster state to publish
- MasterService submits the new prospective cluster state to the discovery implementation for publishing
- Discovery implementation publishes cluster states to all nodes and, once the state is committed, asks the ClusterApplierService to apply the newly committed state.
- ClusterApplierService applies state to local node.
2017-04-28 09:34:31 +02:00
Christoph Büscher 1f221154a6 Merge branch 'master' into feature/client_aggs_parsing 2017-04-26 23:09:45 +02:00
Nik Everett fc97e25b56 Add task to look for tests in src/main (#24298)
Creates a new task `namingConventionsMain`, that runs on the
`buildSrc` and `test:framework` projects and fails the build if
any of the classes in the main artifacts are named like tests or
are non-abstract subclasses of ESTestCase.

It also fixes the three tests that would cause it to fail.
2017-04-25 21:11:47 -04:00
Tanguy Leroux 35946a13d0 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing
# Conflicts:
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/InternalPercentilesTestCase.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/percentiles/hdr/InternalHDRPercentilesTests.java
2017-04-20 13:35:21 +02:00
Nik Everett caf376c8af Start building analysis-common module (#23614)
Start moving built in analysis components into the new analysis-common
module. The goal of this project is:
1. Remove core's dependency on lucene-analyzers-common.jar which should
shrink the dependencies for transport client and high level rest client.
2. Prove that analysis plugins can do all the "built in" things by moving all
"built in" behavior to a plugin.
3. Force tests not to depend on any oddball analyzer behavior. If tests
need anything more than the standard analyzer they can use the mock
analyzer provided by Lucene's test infrastructure.
2017-04-19 18:51:34 -04:00
Ryan Ernst e82d8007e3 Build: Add jstack output when starting integ test cluster if timeout occurs (#24193)
This commit adds a call to jstack to see where each node is stuck when
starting up, if a timeout occurs. This also decreases the timeout back
to 30 seconds.
2017-04-19 10:50:47 -07:00
Tanguy Leroux 5717ac3cc6 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing
# Conflicts:
#	core/src/main/java/org/elasticsearch/search/DocValueFormat.java
#	core/src/test/java/org/elasticsearch/search/aggregations/InternalAggregationTestCase.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/InternalMaxTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/avg/InternalAvgTests.java
#	core/src/test/java/org/elasticsearch/search/aggregations/metrics/min/InternalMinTests.java
2017-04-19 10:12:11 +02:00
Tanguy Leroux e81bbc288a Remove Ubuntu 12.04 (#24161)
Ubuntu 12.04 will be EOL on April 28, 2017.
2017-04-19 09:39:42 +02:00
Ryan Ernst 8f666a74f8 Test: Fix plugin integ test to depend on bundling plugin (#24170)
This was broken in the recent refactoring to add dependsOn directly to
cluster configuration.
2017-04-18 17:26:40 -07:00
Adrien Grand 4632661bc7 Upgrade to a Lucene 7 snapshot (#24089)
We want to upgrade to Lucene 7 ahead of time in order to be able to check whether it causes any trouble to Elasticsearch before Lucene 7.0 gets released. From a user perspective, the main benefit of this upgrade is the enhanced support for sparse fields, whose resource consumption is now function of the number of docs that have a value rather than the total number of docs in the index.

Some notes about the change:
 - it includes the deprecation of the `disable_coord` parameter of the `bool` and `common_terms` queries: Lucene has removed support for coord factors
 - it includes the deprecation of the `index.similarity.base` expert setting, since it was only useful to configure coords and query norms, which have both been removed
 - two tests have been marked with `@AwaitsFix` because of #23966, which we intend to address after the merge
2017-04-18 15:17:21 +02:00
Tanguy Leroux 67a9696e55 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-04-18 14:55:58 +02:00
Ryan Ernst a8017ff020 Tests: Move cluster dependencies from runner to cluster (#24142)
After splitting integ tests into cluster configuration and the test
runner task, we still have dependencies of the test runner added as deps
of the cluster. This commit adds dependencies directly to the cluster,
so that the runner can have other dependencies independent of what is
needed for the cluster.
2017-04-17 16:02:46 -07:00
Yannick Welsch 7e581187a7 [TEST] Increase node startup timeouts
Tests are periodically failing on CI as the wait conditions are not met. This commit increases the timeouts.
2017-04-14 10:36:46 +02:00
Ryan Ernst 13489ff3a7 Test: Fix repro line for platformTest (#24095)
This was accidentally being added to packagingTest, which then had two
repro lines.
2017-04-13 14:17:11 -07:00
Tanguy Leroux 7f730c9489 Merge remote-tracking branch 'origin/master' into feature/client_aggs_parsing 2017-04-13 09:24:03 +02:00
Ryan Ernst fb3a281755 Build: Switch jna dependency to an elastic version (#24081)
This new version of jna is rebuilt from the official release of jna, but
with native libs linked against older glibc in order to support all
platforms elasticsearch supports.

closes #23640
2017-04-13 00:17:50 -07:00
Christoph Büscher cf89fb86b5 Merge branch 'master' into feature/client_aggs_parsing 2017-04-12 12:00:40 +02:00
Ryan Ernst 7cebc0fc93 Cleanup outdated comments for fixing up pom dependencies (#24056) 2017-04-11 23:15:30 -07:00
Nik Everett 76603d4413 Move line split in eclipse back to 140 characters
Now that we've moved checkstyle back to 140 we should move the
split.
2017-04-11 13:23:03 -04:00
Simon Willnauer 8cfb9e446c Move back to 140 chars line length (#24013)
This reverts the line limit change in #23623 - this PR doesn't touch the suppression file since we are moving towards automatic code formatting which makes it mainly obsolete.
2017-04-10 17:07:18 +02:00
javanna f538d7b8d6 Merge branch 'master' into feature/client_aggs_parsing 2017-04-10 14:46:57 +02:00
David Roberts 37aadb2adf Add the ability to include extra notices in a plugin's NOTICES file (#23898)
Adds the option for a plugin to specify extra directories containing notices
and licenses files to be incorporated into the overall notices file that is
generated for the plugin.

This can be useful, for example, where the plugin has a non-Java dependency
that itself incorporates many 3rd party components.
2017-04-10 12:37:42 +01:00
Luca Cavanna b73f87b0ea Make buffer limit configurable in HeapBufferedConsumerFactory (#23970)
The buffer limit should have been configurable already, but the factory constructor is package private so it is truly configurable only from the org.elasticsearch.client package. Also the HttpAsyncResponseConsumerFactory interface was package private, so it could only be implemented from the org.elasticsearch.client package.

Closes #23958
2017-04-10 12:27:42 +02:00
Jason Tedor 61c5976aee Upgrade to Log4j 2.8.2
This commit upgrades the Log4j dependencies from version 2.7 to version
2.8.2. This release includes a fix for a case where Log4j could lose
exceptions in the presence of a security manager.

Relates #23995
2017-04-09 07:19:16 -04:00
Ryan Ernst 776006bac5 Collapse repository gcs classes into a single java package (#23975)
This is a single reorge of the classes to simplify making them mostly
package protected.
2017-04-07 11:27:26 -07:00
javanna 67e087088d add class to suppressions list for line length check 2017-04-07 15:08:44 +02:00
Ryan Ernst 203f8433c2 Collapse packages in ec2 discovery plugin (#23909)
This commit collapses all the classes inside ec2 discovery to a single
package name.
2017-04-05 23:51:49 -07:00
Adrien Grand d5d0f140d6 The `filter` and `significant_terms` aggregations should parse the `filter` as a filter, not as a query. (#23797)
This is important for some queries like `bool`, which are parsed differently
depending on whether we want to get a query or a filter.
2017-04-05 16:46:21 +02:00
Ryan Ernst d31d2caf09 Collapse packages in repository-s3 (#23907)
This commit puts all the classes in the repository-s3 plugin into a
single package.  In addition to simplifying the plugin, it will make it
easier to test as things that should be package private will not be
difficult to use inside tests alone.
2017-04-04 15:15:25 -07:00
Jason Tedor 8c554215e0 Ban Boolean#getBoolean
The method Boolean#getBoolean is dangerous. It is too easy to mistakenly
invoke this method thinking that it is parsing a string as a
boolean. However, what it actually does is get a system property with
the specified string, and then attempts to use usual crappy boolean
parsing in the JDK to parse that system property as boolean with
complete leniency (it parses every input value into either true or
false); that is, this method amounts to invoking
Boolean#parseBoolean(String) on the result of
System#getProperty(String). Boo. This commit bans usage of this method.

Relates #23864
2017-04-01 17:02:19 -04:00
Ryan Ernst 1a5c36509f Build: Remove exclusions hack used for gradle 2.13 (#23809)
Now that we are on gradle 3.3, we can take advantage of a fix that was
made in 2.14 which properly handles disabling transitive dependencies in
pom generation.  As it was currently, we actually ended up generated two
exclusions sections in the generated pom. This is yet another example of
why we need validation on the pom files with our generation here, but I
leave that for another day because I still don't know a good way to do
it.
2017-03-29 21:55:48 -07:00
Lee Hinman c8081bde91 Further refactor and extend testing for `TransportShardBulkAction`
This moves `updateReplicaRequest` to `createPrimaryResponse` and separates the
translog updating to be a separate function so that the function purpose is more
easily understood (and testable).

It also separates the logic for `MappingUpdatePerformer` into two functions,
`updateMappingsIfNeeded` and `verifyMappings` so they don't do too much in a
single function. This allows finer-grained error testing for when a mapping
fails to parse or be applied.

Finally, it separates parsing and version validation for
`executeIndexRequestOnReplica` into a separate
method (`prepareIndexOperationOnReplica`) and adds a test for it.

Relates to #23359
2017-03-29 10:56:51 -06:00
Jason Tedor 6b461a1458 Remove line-length violations in SmokeTestClientIT
This commit removes the line-length violations in SmokeTestClientIT.java
and removes this file from the list of suppressions.
2017-03-27 19:59:48 -04:00
Jason Tedor defd0452e7 Modify permissions dialog for plugins
This commit modifies the handling of plugins that require special
permissions to cover a case that was not previously covered.

Relates #23742
2017-03-27 15:52:45 -04:00
Jim Ferenczi 0e95c90e9f Upgrade to Lucene 6.5.0 (#23750) 2017-03-27 15:57:54 +02:00
Ryan Ernst 8a0bc73435 Build: Use targetConfiguration to remove gradle deprecation warning (#23734)
In gradle 3.3 use of getConfiguration on a ModuleDependency was
deprecated. This commit changes it to use getTargetConfiguration.
2017-03-24 09:43:38 -07:00
Jason Tedor 2517cb3062 Fix line-length violations in gce/util/Access
This commit addresses all 100-column line-length violations in
gce/util/Access.java and removes this file from the suppressions list.
2017-03-22 21:34:15 -04:00
Ryan Ernst 25448e3046 Build: remove progress logger hack for gradle 2.13 (#23679)
We are now on minimum gradle 3.3, so we no longer need the groovy/gradle
hacks used to support both 2.13 and 2.14+.
2017-03-21 23:32:40 -07:00
Ryan Ernst ee802ad63c Test: Add dump of integ test cluster logs on failure (#23688)
This commit adds a build listener to the integ test runner which will
print out an excerpt of the logs from the integ test cluster if the test
run fails.  There are future improvements that can be made (for example,
to dump excerpts from dependent clusters like in tribe node tests), but
this is a start, and would help with simple rest tests failures that we
currently don't know what actually happened on the node.
2017-03-21 23:26:07 -07:00
Boaz Leskes 38a18db81e Revert "Revert "Build: Upgrade min gradle to 3.3 (#23544)""
This reverts commit f926a717ed.
2017-03-21 11:20:04 +01:00
Boaz Leskes f926a717ed Revert "Build: Upgrade min gradle to 3.3 (#23544)"
This reverts commit 5d5815416d.
2017-03-21 10:34:07 +01:00
Ryan Ernst 5d5815416d Build: Upgrade min gradle to 3.3 (#23544)
This will allow us to get rid of deprecation warnings that appear when
using 3.3, and also get rid of extra logic for 2.13 required because of
the progress logger.
2017-03-20 21:15:17 -07:00
Jason Tedor 7b17689458 Search took time should use a relative clock
Search took time uses an absolute clock to measure elapsed time, and
then tries to deal with the complexities of using an absolute clock for
this purpose. Instead, we should use a high-precision monotonic relative
clock that is designed exactly for measuring elapsed time. This commit
modifies the search infrastructure to use a relative clock for measuring
took time, but still provides an absolute clock for the components of
search that require a real clock (e.g., index name expression
resolution, etc.).

Relates #23662
2017-03-20 18:48:51 -04:00
Tim Brooks 15a5d1da7e Setup keystore during integration tests (#22966)
This commit creates a keystore and adds settings to it during the
cluster formation for integration tests. Users can define a
`keyStoreSetting` in build files for settings that need to be placed in
the keystore.
2017-03-20 10:56:17 -05:00
Jason Tedor 2eafe8310e Format RemovePluginCommand to 100-column limit
This commit formats RemovePluginCommand.java to the 100-column limit and
removes this file from the list of suppressions.
2017-03-19 22:50:13 -04:00
Jason Tedor 73097f55e7 Format EvilJNANativesTests to 100-column limit
This commit formats EvilJNANativesTests.java to the 100-column limit and
removes this file from the list of suppressions.
2017-03-18 11:13:59 -04:00
Jason Tedor 44d75db9e2 Upgrade from JNA 4.2.2 to JNA 4.4.0
This commit upgrades the JNA dependency from version 4.2.2 to version
4.4.0.

Relates #23636
2017-03-17 21:06:16 -04:00
Nik Everett f30f18285c Eclipse: move print margin to 100 columns
This comes "for free" when you run `gradle eclipse`.
2017-03-17 16:15:44 -04:00
Jason Tedor e8b2f9afd4 Migrate to max line length of 100
This commit moves the checkstyle rule of max line length from 140
characters to 100 characters. We whitelist all existing violations and
will address them in follow-ups.

Relates #23623
2017-03-17 11:19:58 -04:00
Yannick Welsch 5bd14424e6 Fix third-party audit task for Gradle 3.4 (#23612)
In Gradle 3.4, the buildSrc plugin seems to be packaged into a jar before it is accessed by the rest of the build and the signatures file for the third-party audit task cannot be accessed as
getClass().getResource('/forbidden/third-party-audit.txt') then points to a file entry in a JAR, which cannot be loaded directly as a File object. This commit changes the third-party audit task to pass the content of the signatures file as a String instead.
2017-03-17 02:05:46 -07:00
Ryan Ernst 9d4aff524c Build: Split output for jrunscript to stdout and stderr (#23553)
While trying to improve the failure output in #23547, the stderr was
also captured from jrunscript. This was under the assumption that stderr
is only written to in case of an error. However, with java 9, when
JAVA_TOOL_OPTIONS are set, they are output to stderr. And our CI sets
JAVA_TOOL_OPTIONS for some reason. This commit fixes the jrunscript call
to use a separate buffer for stderr.
2017-03-12 08:20:32 -07:00
Jason Tedor c51ef0b2ca Honor max concurrent searches in multi-search
A previous change to the multi-search request execution to avoid stack
overflows regressed on limiting the number of concurrent search requests
from a batched multi-search request. In particular, the replacement of
the tail-recursive call with a loop could asynchronously fire off all of
the remaining search requests in the batch while max concurrent search
requests are already executing. This commit attempts to address this
issue by taking a more careful approach to the initial problem of
recurisve calls. The cause of the initial problem was due to possibility
of individual requests completing on the same thread as invoked the
search action execution. This can happen, for example, in cases when an
individual request does not resolve to any shards. To address this
problem, when an individual request completes we check if it completed
on the same thread as fired off the request. In this case, we loop and
otherwise safely recurse. Sadly, there was a unit test to check that the
maximum number of concurrent search requests was not exceeded, but that
test was broken while modifying the test to reproduce a case that led to
the possibility of stack overflow. As such, we randomize whether or not
search actions execute on the same thread as the thread that invoked the
action.

Relates #23538
2017-03-12 00:45:40 -08:00
Ryan Ernst a2e838d286 Build: Give better output for java version introspection (#23547)
This commit improves the output when jrunscript fails to include the
full output of the command. It also makes the quoting that is needed for
windows only happen on windows (which worked on java 8, but for some
reason does not work with java 9)
2017-03-11 22:12:48 -08:00
Ryan Ernst 57c1d5f821 Build: Fix jrunscript command to quote correctly on windows (#23539)
As noted
in https://github.com/elastic/elasticsearch/issues/22898#issuecomment-277192425
2017-03-11 10:15:51 -08:00
Ryan Ernst 9488985775 Test: Upgrade randomized runner to 2.5.0 (#23513)
This commit upgrades to the newest version of randomized runner. There
is a new additional check that allows ensuring the working directory
for each child jvm is empty. By default, this check will fail the test
run. However, for elasticsearch, we default to wipe the directory. For
example, if you previously told the runner to not wipe the directory, in
order to investigate a failure, the wipe option will delete this data
upon re-running the test.
2017-03-09 11:56:43 -08:00
Ryan Ernst 188c2b3a26 Build: Fix notice generation to use configured notice file (#23472)
While the esplugin extension already had an input for the base notice
file of the plugin, the NoticeTask did not actually know how to use
that, and always used the base notice file from Elasticsearch.
2017-03-02 19:02:43 -08:00
Ryan Ernst 946b9794ad Fix notice generation to use configured notice file 2017-03-02 16:52:09 -08:00
Ryan Ernst 1768efd72d Build: Allow ignoring sha checks for dependencies (#23471)
This commit adds an `ignoreSha` configuration to the `dependencyLicense`
task, which allows to not check for a sha for a given dependency jar.
This is useful for locally built jars, which will constantly change.
2017-03-02 16:49:20 -08:00
Nik Everett a54daade33 Tests InternalSingleBucketAggregation subclasses (#23388)
Adds a common base class for testing subclasses of
`InternalSingleBucketAggregation`. They are so similar they
call into question the utility of having all of these classes.
We maybe could just use `InternalSingleBucketAggregation` in
all those cases.... But for now, let's test the classes!

Relates to #22278
2017-03-01 16:50:52 -05:00
Jason Tedor 640ab1ceab Set build hash to Unknown when unavailable
This commit changes the build hash to be the string "Unknown" when for
some reason this build hash is not available. This aligns the value with
the value we use when the hash is not available from the jar
manifest. This situation can occur when running tests from a worktree
which is not currently handled correctly by JGit, the upstream
dependency that is used to acquire the hash. This causes problems when
running tests locally because the warning header pattern expects a hash
or the string "Unknown". While the warning header pattern be changed to
allow "N/A" as well, it seemed more sensible to align the value here
with the value when the hash is not available from the jar manifest.

Relates #23421
2017-03-01 07:57:14 -05:00
Jim Ferenczi 5c84640126 Upgrade to lucene-6.5.0-snapshot-d00c5ca (#23385)
Lucene upgrade
2017-02-27 18:39:04 +01:00
Ryan Ernst 48280a9403 Tests: Add platformTest to vagrant (#23339)
This change adds a new test task, platformTest, which runs `gradle test
integTest` within a vagrant host. This will allow us to still test on
all the supported platforms, but be able to standardize on the tools
provided in the host system, for example, with a modern version of git
that can allow #22946.

In order to have sufficient memory and cpu to run the tests, the
vagrantfile has been updated to use 8GB and 4 cpus by default. This can
be customized with the `VAGRANT_MEMORY` and `VAGRANT_CPUS` environment
variables.  Also, to save time to show this can work, it currently uses
the same Vagrantfile the packaging tests do. There are a lot of cleanups
we can do to how the gradle-vagrant tasks work, including generating
Vagrantfile altogether, but I think this is fine for now as the same
machines that would run platformTest run packagingTest, and they are
relatively beefy machines, so the higher memory and cpu for them, with
either task, should not be an issue.
2017-02-27 09:21:28 -08:00
Ryan Ernst 37452dbe23 Tests: Fix reproduce line for packagingTest (#23365)
This commit fixes the reproduce line output when the vagrant packagingTest
fails. Before only the `gradle packagingTest` would be output, but the
seed and list of versions was swallowed by groovy with an ancillary
failure (due to the `+` being on the wrong line for a string
continuation).  With the new reproduce line, it is now output next to
the task right after failure, contains the actual task (specific to the
box that fails), and contains the seed. It also no longer contains the
upgrade versions list, as the seed is used to determine which of those
to use, and the same file would be read when testing a failure on a
particular git commit. Finally, this also ties bats test setup directly
to packagingTest, instead of to the vagrant up command.
2017-02-24 19:40:28 -08:00
Ryan Ernst de8049fd2a Tests: Ensure multi node integ tests wait on first node
When a rest integ test has multiple nodes, each node is supposed to not
start configuring itself until the first node has been started, so that
the unicast host information can be written. However, this was never
explicitly setup to occur, and we were just very lucky with the current
gradle version and stability of the code always produced a task graph
that had node0 starting first. With the recent refactorings to integ
tests, the order has changed. This commit fixes the ordering by adding
an explicit dependency between the first node and the other nodes.
2017-02-22 20:54:58 -08:00
Ryan Ernst 74ecd34fd7 Build: Change location in zip of license and notice inclusion for plugins (#23316)
This commit moves the LICENSE.txt and NOTICE.txt files for each plugin
to be alongside the other plugin files, inside the elasticsearch subdir.
This ensures those files are installed alongside the plugin.
2017-02-22 16:13:50 -08:00
Ryan Ernst 175bda64a0 Build: Rework integ test setup and shutdown to ensure stop runs when desired (#23304)
Gradle's finalizedBy on tasks only ensures one task runs after another,
but not immediately after. This is problematic for our integration tests
since it allows multiple project's integ test clusters to be
simultaneously. While this has not been a problem thus far (gradle 2.13
happened to keep the finalizedBy tasks close enough that no clusters
were running in parallel), with gradle 3.3 the task graph generation has
changed, and numerous clusters may be running simultaneously, causing
memory pressure, and thus generally slower tests, or even failure if the
system has a limited amount of memory (eg in a vagrant host).

This commit reworks how integ tests are configured. It adds an
`integTestCluster` extension to gradle which is equivalent to the current
`integTest.cluster` and moves the rest test runner task to
`integTestRunner`.  The `integTest` task is then just a dummy task,
which depends on the cluster runner task, as well as the cluster stop
task. This means running `integTest` in one project will both run the
rest tests, and shut down the cluster, before running `integTest` in
another project.
2017-02-22 12:43:15 -08:00
Simon Willnauer ce625ebdcc Expose `batched_reduce_size` via `_search` (#23288)
In #23253 we added an the ability to incrementally reduce search results.
This change exposes the parameter to control the batch since and therefore
the memory consumption of a large search request.
2017-02-21 18:36:59 +01:00
Nik Everett d1de9574ea Checkstyle: Fix link lengths in sampler aggregation 2017-02-17 15:03:57 -05:00
Jay Modi b234644035 Enforce Content-Type requirement on the rest layer and remove deprecated methods (#23146)
This commit enforces the requirement of Content-Type for the REST layer and removes the deprecated methods in transport
requests and their usages.

While doing this, it turns out that there are many places where *Entity classes are used from the apache http client
libraries and many of these usages did not specify the content type. The methods that do not specify a content type
explicitly have been added to forbidden apis to prevent more of these from entering our code base.

Relates #19388
2017-02-17 14:45:41 -05:00
Ryan Ernst fbf824f6a6 Fix plugin properties generation to be exact about the file to copy to
the output zip
2017-02-16 13:43:07 -08:00
Ryan Ernst efb6259e3a Tests: Re-enable debian-8 and fedora-24 packaging tests (#23217)
These images have been rebuilt to be preloaded with java 8 installed.
This change re-enables the systems. It also removes some redundancy in
the rpm checks I found while testing the new images, and fixes a
potential issue with generated resources in plugins where a stale dir
can cause junk to get into the distribution.
2017-02-16 11:26:46 -08:00
Ryan Ernst 6cdf4f3f72 Plugins: Include license and notice files in zip (#23191)
This commit adds the elasticsearch LICENSE.txt to all plugins that
released with elasticsearch, as well as a generated NOTICE.txt specific
to the dependencies of each plugin.
2017-02-15 11:23:12 -08:00
Ryan Ernst 8453cf0622 Build: Add notice file generation (#23170)
This change improves the notice file present in our distributions to
include notice and license files from each included dependency.

closes #22546
2017-02-15 09:40:16 -08:00
Ryan Ernst c4625f230f Build: Support mavenLocal flag in buildSrc (#23157)
When configuring which repositories to pull from, we currently add
mavenLocal() when the `repos.mavenLocal` flag is set. However, this is
only done in normal projects, but not the special buildSrc project. This
change adds that support. Note that this was not possible before gradle
2.13, as there was a bug which prevented sys props from reaching the
buildSrc project (https://issues.gradle.org/browse/GRADLE-2475).
However, we already require 2.13+.
2017-02-14 21:31:56 -08:00
Uwe Schindler 82613198fe Update to forbiddenapis 2.3 (improves Gradle configuration time) (#23154) 2017-02-13 14:30:08 -08:00
Adrien Grand 709cc9ba65 Upgrade to lucene-6.5.0-snapshot-f919485. (#23087) 2017-02-10 15:08:47 +01:00
Tim Brooks fcc568fd8d Add methods requiring connect to forbidden apis (#22964)
This is related to #22116. This commit adds calls that require
SocketPermission connect to forbidden APIs.

The following calls are now forbidden:

- java.net.URL#openStream()
- java.net.URLConnection#connect()
- java.net.URLConnection#getInputStream()
- java.net.Socket#connect(java.net.SocketAddress)
- java.net.Socket#connect(java.net.SocketAddress, int)
- java.nio.channels.SocketChannel#open(java.net.SocketAddress)
- java.nio.channels.SocketChannel#connect(java.net.SocketAddress)
2017-02-07 14:41:50 -06:00
Martijn van Groningen c8d470f190 Change `org.elasticsearch.bootstrap.JNAKernel32Library$SizeT` constructor's modifier to public.
Otherwise `NativeMappedConverter` can't construct this class.

Closes #22991
2017-02-06 15:16:16 +01:00
Adrien Grand c8496fc4f4 Upgrade to Lucene 6.4.1. (#22978) 2017-02-06 09:28:43 +01:00
Ryan Ernst 796087a5a9 Build: Disable fedora for packaging tests (#22974)
Now that debian is disabled, we are seeing similar failures with fedora
not able to install java. This commit temporarily disables fedora until
it is once again stable.
2017-02-03 18:33:53 -08:00
Lee Hinman 39e7c30912 Change certain replica failures not to fail the replica shard
This changes the way that replica failures are handled such that not all
failures will cause the replica shard to be failed or marked as stale.

In some cases such as refresh operations, or global checkpoint syncs, it is
"okay" for the operation to fail without the shard being failed (because no data
is out of sync). In these cases, instead of failing the shard we should simply
fail the operation, and, in the event it is a user-facing operation, return a
5xx response code including the shard-specific failures.

This was accomplished by having two forms of the `Replicas` proxy, one that is
for non-write operations that does not fail the shard, and one that is for write
operations that will fail the shard when an operation fails.

Relates to #10708
2017-02-03 14:39:46 -07:00
Ryan Ernst bbf44394a8 Build: Temporarily remove debian 8 for packaging tests (#22967)
Debian 8 has been having issues with the openjdk package dependencies
being broken. This comment comments out debian-8 from the boxes which
packaging tests will run on CI.
2017-02-03 12:57:21 -08:00
Tim Brooks f70188ac58 Remove connect SocketPermissions from core (#22797)
This is related to #22116. Core no longer needs `SocketPermission`
`connect`.

This permission is relegated to these modules/plugins:
- transport-netty4 module
- reindex module
- repository-url module
- discovery-azure-classic plugin
- discovery-ec2 plugin
- discovery-gce plugin
- repository-azure plugin
- repository-gcs plugin
- repository-hdfs plugin
- repository-s3 plugin

And for tests:
- mocksocket jar
- rest client
- httpcore-nio jar
- httpasyncclient jar
2017-02-03 09:39:56 -06:00
Jason Tedor 9a0b216c36 Upgrade checkstyle to version 7.5
This commit upgrades the checkstyle configuration from version 5.9 to
version 7.5, the latest version as of today. The main enhancement
obtained via this upgrade is better detection of redundant modifiers.

Relates #22960
2017-02-03 09:46:44 -05:00
Ryan Ernst be0236416b Build: Improve detection of older java versions (#22950)
This change switches to using jrunscript, instead of jjs, for detecting
version properties of java, which is available on java versions prior to 8.

closes #22898
2017-02-02 22:58:19 -08:00
Ryan Ernst 7f59bed87b Build: Add nicer error message for pre java 8 (#22911)
This commit adds clearer error message when trying to find the version of java, but
the jjs binary is not found.

closes #22898
2017-02-01 14:07:21 -08:00
Jack Conradson 3d2626c4c6 Change Namespace for Stored Script to Only Use Id (#22206)
Currently, stored scripts use a namespace of (lang, id) to be put, get, deleted, and executed. This is not necessary since the lang is stored with the stored script. A user should only have to specify an id to use a stored script. This change makes that possible while keeping backwards compatibility with the previous namespace of (lang, id). Anywhere the previous namespace is used will log deprecation warnings.

The new behavior is the following:

When a user specifies a stored script, that script will be stored under both the new namespace and old namespace.

Take for example script 'A' with lang 'L0' and data 'D0'. If we add script 'A' to the empty set, the scripts map will be ["A" -- D0, "A#L0" -- D0]. If a script 'A' with lang 'L1' and data 'D1' is then added, the scripts map will be ["A" -- D1, "A#L1" -- D1, "A#L0" -- D0].

When a user deletes a stored script, that script will be deleted from both the new namespace (if it exists) and the old namespace.

Take for example a scripts map with {"A" -- D1, "A#L1" -- D1, "A#L0" -- D0}. If a script is removed specified by an id 'A' and lang null then the scripts map will be {"A#L0" -- D0}. To remove the final script, the deprecated namespace must be used, so an id 'A' and lang 'L0' would need to be specified.

When a user gets/executes a stored script, if the new namespace is used then the script will be retrieved/executed using only 'id', and if the old namespace is used then the script will be retrieved/executed using 'id' and 'lang'
2017-01-31 13:27:02 -08:00
Jason Tedor 930282e161 Introduce sequence-number-based recovery
This commit introduces sequence-number-based recovery. When a replica
has fallen out of sync, rather than performing a file-based recovery we
first attempt to replay operations since the last local checkpoint on
the replica. To do this, at the start of recovery the replica tells the
primary what its local checkpoint is. The primary will then wait for all
operations between that local checkpoint and the current maximum
sequence number to complete; this is to ensure that there are no gaps in
the operations that will be replayed from the primary to the
replica. This is a best-effort attempt as we currently have no
guarantees on the primary that these operations will be available; if we
are not able to replay all operations in the desired range, we just
fallback to file-based recovery. Later work will strengthen the
guarantees.

Relates #22484
2017-01-27 08:16:38 -08:00
Christoph Büscher 33fd1a606b Remove unneeded checkstyle LineLength check suppression for more classes 2017-01-26 19:40:49 +01:00
Tim Brooks 719e75bb3f Add repository-url module and move URLRepository (#22752)
This is related to #22116. URLRepository requires SocketPermission
connect. This commit introduces a new module called "repository-url"
where URLRepository will reside. With the new module, permissions can
be removed from core.
2017-01-25 17:09:25 -06:00
Nik Everett d704a880e7 Add tests for top_hits aggregation (#22754)
Add unit tests for `TopHitsAggregator` and convert some snippets in
docs for `top_hits` aggregation to `// CONSOLE`.

Relates to #22278
Relates to #18160
2017-01-25 16:15:50 -05:00
Yannick Welsch e37a7d73a2 Remove leftover build files and simplify ProgressLogger usage
These files should have been removed in an earlier commit. This commit also simplifies usage of ProgressLoggerWrapper by using the Groovy delegation instead of using explicit delegation.
2017-01-24 20:44:14 +01:00
Luca Cavanna 47c0e13a3b Stop returning "es." internal exception headers as http response headers (#22703)
move "es." internal headers to separate metadata set in ElasticsearchException and stop returning them as response headers

Closes #17593

* [TEST] remove ESExceptionTests, move its methods to ElasticsearchExceptionTests or ExceptionSerializationTests
2017-01-24 16:12:45 +01:00
Yannick Welsch 12b6ff5233 Use Gradle-version specific source sets
Instead of using Gradle-version specific compilation options, use distinct source sets. This also allows compilation of buildSrc/build-tools under IDEs that
don't understand the version-specific compilation options.

Relates to #22669
2017-01-24 11:09:57 +01:00
Yannick Welsch 36198e0275 Make build Gradle 2.14 / 3.x compatible (#22669)
This changes build files so that building Elasticsearch works with both Gradle 2.13 as well as higher versions of Gradle (tested 2.14 and 3.3), enabling a smooth transition from Gradle 2.13 to 3.x.
2017-01-24 11:09:57 +01:00
Jim Ferenczi 8028578305 Upgrade to Lucene 6.4.0 (#22724)
* Upgrade to Lucene 6.4.0

`ValueSource`s are now converted to `DoubleValueSource`s using the Lucene adapter made for the migration to the new API in 6.4.0.
2017-01-21 04:48:01 +01:00
Jason Tedor 8f6c074691 Revert "Make build Gradle 2.14 / 3.x compatible (#22669)"
This reverts commit 652cb7dbf7.

Relates #22727
2017-01-20 18:16:45 -05:00
Ryan Ernst c5b4bba30b S3 repository: Deprecate specifying credentials through env vars, sys props, and remove profile files (#22567)
* S3 repository: Deprecate specifying credentials through env vars and sys props

This is a follow up to #22479, where storing credentials secure way was
added.
2017-01-19 12:36:32 -08:00
Nik Everett 3ea37ec817 Fix Eclipse project generation
The new plugin causes Eclipse to go a bit crazy. So we skip it
for Eclipse.

Relates to #22646
2017-01-19 13:13:11 -05:00
Tim Brooks a10aa8aade Add TestWithDependenciesPlugin to build (#22646)
This commit adds a MessyRestTestPlugin to the gradle build. It extends 
StandaloneRestTestPlugin. The main piece of functionality that it adds 
is to copy plugin-metadata from dependencies into the 
generated-resources for the current test source. This is necessary to 
ensure that permissions for dependencies are applied when running the 
tests.

A current limitation is that the permissions are applied differently 
than in the distribution sources. When permissions are granted to all 
depedencies for a module or plugin, the permissions are granted to all 
dependencies on the classpath for tests besides a few hardcoded 
exclusions:
- es core
- es test framework
- lucene test framework
- randomized runner
- junit library
2017-01-19 09:43:53 -06:00
Yannick Welsch 652cb7dbf7 Make build Gradle 2.14 / 3.x compatible (#22669)
This changes build files so that building Elasticsearch works with both Gradle 2.13 as well as higher versions of Gradle (tested 2.14 and 3.3), enabling a smooth transition from Gradle 2.13 to 3.x.
2017-01-19 09:56:54 +01:00
Daniel Mitterdorfer aece89d6a1 Make boolean conversion strict (#22200)
This PR removes all leniency in the conversion of Strings to booleans: "true"
is converted to the boolean value `true`, "false" is converted to the boolean
value `false`. Everything else raises an error.
2017-01-19 07:59:18 +01:00
Nik Everett 1fe74a6b4b Better error when can't auto create index (#22488)
Changes the error message when `action.auto_create_index` or
`index.mapper.dynamic` forbids automatic creation of an index
from `no such index` to one of:
* `no such index and [action.auto_create_index] is [false]`
* `no such index and [index.mapper.dynamic] is [false]`
* `no such index and [action.auto_create_index] contains [-<pattern>] which forbids automatic creation of the index`
* `no such index and [action.auto_create_index] ([all patterns]) doesn't match`

This should make it more clear *why* there is `no such index`.

Closes #22435
2017-01-18 15:18:32 -05:00
Nik Everett e71b26f480 Improve unit test coverage of aggs (#22668)
Add tests for `GlobalAggregator`, `MaxAggregator`, and `InternalMax`.

Relates to #22278
2017-01-18 10:33:45 -05:00
Simon Willnauer f30b1f82ee Remove HttpServer and HttpServerAdapter in favor of a simple dispatch method (#22636)
Today we have quite some abstractions that are essentially providing a simple
dispatch method to the plugins defining a `HttpServerTransport`. This commit
removes `HttpServer` and `HttpServerAdaptor` and introduces a simple `Dispatcher` functional
interface that delegate to `RestController` by default.

Relates to #18482
2017-01-16 21:06:08 +01:00
Alexander Reelsen f6ee6e420b Indexing: Add shard id to indexing operation listener (#22606)
The IndexingOperationListener interface did not provide any
information about the shard id when a document was indexed.

This commit adds the shard id as the first parameter to all methods
in the IndexingOperationListener.
2017-01-16 09:08:16 +01:00
Tanguy Leroux 3a3ce61186 Update Jackson to 2.8.6 (#22596)
closes #22266
2017-01-13 09:05:48 +01:00
Nik Everett 4df2e182bf Fix some typos in error messages in the build 2017-01-10 16:11:05 -05:00
Luca Cavanna ddb93946aa use ElasticsearchException#renderException in BytesRestResponse#convert (#22531) 2017-01-10 20:58:31 +01:00
Nik Everett d50f96e122 Remove InternalAggregation.Type (#22511)
It is no longer needed. It used to contain a lot of strings
used by serialization but those have since been removed. Now
it is just another thing to pass around that we don't really
need.
2017-01-10 11:57:19 -05:00
Tanguy Leroux dc81afd4f8 [TESTS] Create vagrant up task for all boxes (#22496)
Currently, such tasks are only created for default boxes (centos-7, ubuntu-1404) and not all boxes and this can be misleading for developers who want to debug testing scripts on non-default boxes.
2017-01-10 09:45:33 +01:00