1. tests don't have a bogus test dependency on zips anymore,
instead we handle this in pre-integration-test. This reduces
lots of confusion for e.g. mvn clean test.
2. refactor integ logic so that core/ and plugin/ share it.
previously they were duplicates but the above change simplifies life.
it also makes it easier for doing more interesting stuff
Today we have a intermediate hierarchy for shard and index exceptions
which makes it hard to introduce generic exceptions like ResourceNotFoundException
intoduced in this commit. This commit breaks up the hierarchy by adding index and shard
as a special internal header that gets rendered for every exception that fills that header.
This commit removes dedicated exceptions like `IndexMissingException` or
`IndexShardMissingException` in favour of `ResourceNotFoundException`
Modified ScriptEngineService to pass in a CompiledScript object
with newly added name and type member variables.
This can in turn be used to give better scripting error messages
with the type of script used and the name of the script.
Required slight modifications to the caching mechanism.
Note that this does not enforce good behavior in that plugins will
have to write exceptions that also output the name of the script
in order to be effective. There was no way to wrap the script
methods in a try/catch block properly further up the chain because
many have script-like objects passed back that can be run at a
later time.
closes#6653closes#11449
Changes in a nutshell:
* All expression logic is now encapsulated by ExpressionResolver interface.
* MetaData#convertFromWildcards() gets replaced by WildcardExpressionResolver.
* All of the indices expansion methods are being moved from MetaData class to the new IndexNameExpressionResolver class.
* All single index expansion optimisations are removed.
The logic for resolving a concrete index name from an expression has been moved from MetaData to IndexExpressionResolver. The logic has been cleaned up and simplified were was possible without breaking bwc.
Also the notion of aliasOrIndex has been changed to index expression.
The IndexNameExpressionResolver translates index name expressions into concrete indices. The list of index name expressions are first delegated to the known ExpressionResolverS. An ExpressionResolver is responsible for translating if possible an expression into another expression (possibly but not required this can be concrete indices or aliases) otherwise the expressions are left untouched. Concretely this means converting wildcard expressions into concrete indices or aliases, but in the future other implementations could convert expressions based on different rules.
To prevent many overloading of methods, DocumentRequest extends now from IndicesRequest. All implementation of DocumentRequest already did implement IndicesRequest indirectly.
Today everything is tight to having the next version as the latest.
In order to work towards 2.0.0.beta1 we need to fix all the usage of
2.0.0-SNAPSHOT to reflect the version we will release soon.
Usually we do this on the release branch but to simplify things I wanna
keep this on master for now and move to 2.1.0-SNAPSHOT on master once
we created a 2.0 branch.
Closes#12148
Reported in https://github.com/elastic/elasticsearch/issues/11647#issuecomment-118523861
> btw, I think you broke some plugins on Master, cloud-aws doesn't register s3 repos anymore.
```
org.elasticsearch.common.inject.CreationException: Guice creation errors:
1) Error injecting constructor, java.lang.NoClassDefFoundError: org/apache/http/protocol/HttpContext
at org.elasticsearch.repositories.s3.S3Repository.<init>(Unknown Source)
while locating org.elasticsearch.repositories.s3.S3Repository
while locating org.elasticsearch.repositories.Repository
1 error
at org.elasticsearch.common.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:344)
at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:178)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:140)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:69)
at org.elasticsearch.repositories.RepositoriesService.createRepositoryHolder(RepositoriesService.java:404)
at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:368)
at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:55)
at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:110)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:378)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:209)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:179)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/apache/http/protocol/HttpContext
at com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:129)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:432)
at com.amazonaws.services.s3.AmazonS3Client.<init>(AmazonS3Client.java:414)
at org.elasticsearch.cloud.aws.InternalAwsS3Service.getClient(InternalAwsS3Service.java:153)
at org.elasticsearch.cloud.aws.InternalAwsS3Service.client(InternalAwsS3Service.java:82)
at org.elasticsearch.repositories.s3.S3Repository.<init>(S3Repository.java:125)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
at org.elasticsearch.common.inject.FactoryProxy.get(FactoryProxy.java:54)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
... 13 more
Caused by: java.lang.ClassNotFoundException: org.apache.http.protocol.HttpContext
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 37 more
```
Closes#12034.
We can keep only unit tests in plugins instead of starting each time a local node and running tests against it.
Also follow up of #12091
Tests can use more than one JVM
We don't need anymore to set the number of jvm to run tests as we moved IT to Rest Tests
For all plugins but cloud plugins which will require another way for running integration tests.
Follow up for https://github.com/elastic/elasticsearch-analysis-kuromoji/issues/61
We don't shade anymore elasticsearch dependencies, so plugins might include jars in the distribution ZIP file which might not be needed anymore.
For example, `elasticsearch-cloud-aws` comes with:
```
Archive: cloud-aws/target/releases/elasticsearch-cloud-aws-2.0.0-SNAPSHOT.zip
Length Date Time Name
-------- ---- ---- ----
1920788 05-18-15 09:42 aws-java-sdk-ec2-1.9.34.jar
503963 05-18-15 09:42 aws-java-sdk-core-1.9.34.jar
232771 01-19-15 09:24 commons-codec-1.6.jar
915096 01-19-15 09:24 jackson-databind-2.3.2.jar
252288 05-18-15 09:42 aws-java-sdk-kms-1.9.34.jar
62050 01-19-15 09:24 commons-logging-1.1.3.jar
282269 10-31-14 13:19 httpcore-4.3.2.jar
35058 01-19-15 09:24 jackson-annotations-2.3.0.jar
229998 05-29-15 12:28 jackson-core-2.5.3.jar
589289 01-19-15 09:24 joda-time-2.7.jar
562858 05-18-15 09:42 aws-java-sdk-s3-1.9.34.jar
590533 10-31-14 13:19 httpclient-4.3.5.jar
44854 06-12-15 19:22 elasticsearch-cloud-aws-2.0.0-SNAPSHOT.jar
-------- -------
6221815 13 files
```
A lot of those files are already distributed with elasticsearch itself so classes are available within the classloader.
We mark all es core dependencies as provided in plugins.
We also remove `groupId` as already defined in parent pom.
And we remove non needed licenses files as some jars are not included anymore in plugins.
Closes#11647.
The change makes rest-spec-api a project in the same way as we build dev-tools. it packages the tests and api in a bundle using the maven-remote-resources-plugin and uses the same plugin in the plugins and core pom to unpack the rest-api-spec into the target directory and references the rest tests there in the test resources.
The main stimulus for this change is that for those using Eclipse the current build does not work. After running `mvn eclipse:eclipse` the Eclipse IDE errors because the rest-api-spec is outside of the project scope, meaning that every time the command is run (required whenever any dependencies change), the class path of all the projects has to be manually fixed.
With a recent change in core, we don't support anymore non explicit byte size units when setting values.
This commit fix that in azure repository. Otherwise, tests can't be executed.
The deleted counter is incremented even if the document is missing. Also, this commit ensures that the scroll id is cleared even if no documents are found by the scan request.
this is really just a workaround for plugins to run their own
REST tests instead of the core ones. It opts out of the rest test
loading from the core jar file and tries to load from the classpath instead.
Eventually we need to fix this infrastrucutre to move away from parameterized
tests such that subclasses can override behavior.
Closes#11721
The delete by query plugin adds support for deleting all of the documents (from one or more indices) which match the specified query. It is a replacement for the problematic delete-by-query functionality which has been removed from Elasticsearch core in 2.0. Internally, it uses the Scan/Scroll and Bulk APIs to delete documents in an efficient and safe manner. It is slower than the old delete-by-query functionality, but fixes the problems with the previous implementation.
Closes#7052
[build] mark elasticsearch as provided in plugins
When we build a plugin, we suppose it will be executed within elasticsearch server.
So we should mark it as `provided`.
If a java developer needs to embed the plugin and elasticsearch, it will make sense to declare both in its `pom.xml` file.
In Maven parent project, in dependency management, we should only declare which versions of 3rd party jars we want to use but not force any scope.
It makes then more obvious in modules what is exactly the scope of any dependency.
For example, one could imagine importing `jimfs` as a `compile` dependency in another module/plugin with:
```xml
<dependency>
<groupId>com.google.jimfs</groupId>
<artifactId>jimfs</artifactId>
</dependency>
```
But it won't work as expected as the default maven `scope` should be `compile` but here it's `test` as defined in the parent project.
So, if you want to use this lib for tests, you should simply define:
```xml
<dependency>
<groupId>com.google.jimfs</groupId>
<artifactId>jimfs</artifactId>
<scope>test</scope>
</dependency>
```
We also remove `maven-s3-wagon` from gce plugin as it's not used.
The script APIs have been deprecated long ago we can now remove them.
This commit still keeps the parsing code since it might be used in a
query that is still stuck in transaction log. This issue should be discussed
elsewhere.
Closes#11619
When we build a plugin, we suppose it will be executed within elasticsearch server.
So we should mark it as `provided`.
If a java developer needs to embed the plugin and elasticsearch, it will make sense to declare both in its `pom.xml` file.
Robert Muir
Simon Willnauer
uboness
Jack Conradson
Martijn van Groningen
David Pilato
Clinton Gormley
Tanguy Leroux
David Pursehouse
Colin Goodheart-Smithe
Boaz Leskes