Closes#14353
Squashed commit of the following:
commit edae0729f71ea3d3f9fa9c0d27c9effc042eb5a9
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 29 14:13:42 2015 -0400
update sha1 and simplify test
commit 635c4f245d66ad353a16267c810e02b725553fad
Author: Robert Muir <rmuir@apache.org>
Date: Thu Oct 29 07:01:26 2015 -0400
Add threadgroup isolation.
Code with `modifyThread` and `modifyThreadGroup` may only modify
its own threadgroup (or an ancestor of that). This enforces
what is intended by the ThreadGroup class.
This has two immediate implications:
1. Code without these permissions (scripts) may not create or mess with threads
2. ES application threads cannot mess with Java system threads
ES puts all application threads in one single group today, but in the future
this can be organized better, and we will have more isolation in the system.
* plugin authors can use full policy syntax, including codebase substitution
properties like core syntax.
* simplify test logic.
* move out test-framework permissions to separate file.
Closes#14311
This commit upgrades mustache.java to version 0.9.1. The primary motive
for this is because version 0.8.13 depends on Guava, but version 0.9.1
does not.
Relates #13224
This fixes license checks to apply to all java files under src/ as opposed to
only those in the org.elasticsearch package. It found some license headers that
had to be reformatted. I also added a missing license header to Nullability.java
however this has not be caught by the license checker since it ignores guice
files.
Relates to #13703
Java 8's javadoc defaults to very strict linting. It is very `-Wall -Werr`
style. And Elasticsearch's Javadocs do not pass and it'd be a huge and not
super useful effort to get them to pass the linting. So this disables it.
Closes#13336
graduate this from a hack for insecure plugins to something we can
live with for per-module/plugin permissions, it now works reasonably
in unit tests and with Intellij and Eclipse IDEs.
remove security warnings: we will deal with these issues in a secure
way, if we cannot, then the plugin shouldn't be in our core codebase.
The semantics of the `boost` parameter for `function_score` changed. This is
due to the fact that Lucene now requires that query boosts and top-level boosts
are applied the same way.
We have a handful of compiler warnings, mostly because of passing an
array to varargs methods. This change fixes these warnings and adds
-Werror so we don't get anymore of these warnings.
Note this does *not* enable deprecation or unchecked type warnings, so
these remain "hidden". We should work towards removing those as well,
but this is a first step.
Each time you run mvn test, you end up creating a `.logs` empty dir.
Instead of writing for each plugin in the `./logs` dir, I'd recommend writing this in `./target/heapdump`.
This dir is cleanup automatically when you run `mvn clean`.
We currently have a small number of test classes with the suffix "Test",
yet most use the suffix "Tests". This change renames all the "Test"
classes, so that we have a simple rule: "Non-inner classes ending with
Tests".
These are not actually tests, but command line applications that must be
run manually. This change removes the entire stresstest package. We can
add back individual tests that we find necessary, and make them real
tests (whether integ or not).
While the list of having exclusions is small, it shouldn't be necessary
at all. Base test cases should be suffixed with TestCase so they are not
picked up by the test class name pattern. This same rule works for
abstract classes as well.
This change renames abstract tests to use the TestCase suffix, adds a
check in naming convention tests, and removes the exclusion from our
test runner configuration. It also excludes inner classes (the only
exclude we should have IMO), so that we have no need to @Ignore the
inner test classes for naming convention tests.
The shaded version of elasticsearch was built at the very beginning to avoid dependency conflicts in a specific case where:
* People use elasticsearch from Java
* People needs to embed elasticsearch jar within their own application (as it's today the only way to get a `TransportClient`)
* People also embed in their application another (most of the time older) version of dependency we are using for elasticsearch, such as: Guava, Joda, Jackson...
This conflict issue can be solved within the projects themselves by either upgrade the dependency version and use the one provided by elasticsearch or by shading elasticsearch project and relocating some conflicting packages.
Example
-------
As an example, let's say you want to use within your project `Joda 2.1` but elasticsearch `2.0.0-beta1` provides `Joda 2.8`.
Let's say you also want to run all that with shield plugin.
Create a new maven project or module with:
```xml
<groupId>fr.pilato.elasticsearch.test</groupId>
<artifactId>es-shaded</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<elasticsearch.version>2.0.0-beta1</elasticsearch.version>
</properties>
<dependencies>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
<dependency>
<groupId>org.elasticsearch.plugin</groupId>
<artifactId>shield</artifactId>
<version>${elasticsearch.version}</version>
</dependency>
</dependencies>
```
And now shade and relocate all packages which conflicts with your own application:
```xml
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.4.1</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<relocations>
<relocation>
<pattern>org.joda</pattern>
<shadedPattern>fr.pilato.thirdparty.joda</shadedPattern>
</relocation>
</relocations>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
```
You can create now a shaded version of elasticsearch + shield by running `mvn clean install`.
In your project, you can now depend on:
```xml
<dependency>
<groupId>fr.pilato.elasticsearch.test</groupId>
<artifactId>es-shaded</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.1</version>
</dependency>
```
Build then your TransportClient as usual:
```java
TransportClient client = TransportClient.builder()
.settings(Settings.builder()
.put("path.home", ".")
.put("shield.user", "username:password")
.put("plugin.types", "org.elasticsearch.shield.ShieldPlugin")
)
.build();
client.addTransportAddress(new InetSocketTransportAddress(new InetSocketAddress("localhost", 9300)));
// Index some data
client.prepareIndex("test", "doc", "1").setSource("foo", "bar").setRefresh(true).get();
SearchResponse searchResponse = client.prepareSearch("test").get();
```
If you want to use your own version of Joda, then import for example `org.joda.time.DateTime`. If you want to access to the shaded version (not recommended though), import `fr.pilato.thirdparty.joda.time.DateTime`.
You can run a simple test to make sure that both classes can live together within the same JVM:
```java
CodeSource codeSource = new org.joda.time.DateTime().getClass().getProtectionDomain().getCodeSource();
System.out.println("unshaded = " + codeSource);
codeSource = new fr.pilato.thirdparty.joda.time.DateTime().getClass().getProtectionDomain().getCodeSource();
System.out.println("shaded = " + codeSource);
```
It will print:
```
unshaded = (file:/path/to/joda-time-2.1.jar <no signer certificates>)
shaded = (file:/path/to/es-shaded-1.0-SNAPSHOT.jar <no signer certificates>)
```
This PR also removes fully-loaded module.
By the way, the project can now build with Maven 3.3.3 so we can relax a bit our maven policy.
From a user perspective, the main benefit from this upgrade is that the new
Lucene53Codec has disk-based norms. The elasticsearch directory has been fixed
to load these norms through mmap instead of nio.
Other changes include the removal of `max_thread_states`, the fact that
PhraseQuery and BooleanQuery are now immutable, and that deleted docs are now
applied on top of the Scorer API.
This change introduces a couple of `AwaitsFix`s but I don't think it should
hold us from merging.
```xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-enforcer-plugin</artifactId>
<version>1.4.1</version>
</plugin>
```
Release Notes - Maven Enforcer - Version 1.4.1
========
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12317520&version=12330766
Bugs:
-----
* [MENFORCER-222] - RequireSameVersion rule regression between 1.3 and 1.4
* [MENFORCER-224] - Regression from 1.3.1 to 1.4 with bannedDependencies rule
* [MENFORCER-229] - Ban Distribution Management documentation example doesn't work
* [MENFORCER-237] - Resources Link to codehaus is wrong
Improvements:
------------
* [MENFORCER-223] - Upgrade mrm-maven-plugin to 1.0-beta-2
* [MENFORCER-227] - Document nullability with @Nonnull on EnforcerRule API
* [MENFORCER-233] - Upgrade maven-invoker-plugin to 2.0.0
* [MENFORCER-235] - Use maven-fluido-skin 1.4
* [MENFORCER-236] - Upgrade maven-assembly-plugin version from 2.4 to 2.5.5 in integration test
* [MENFORCER-238] - Upgrade plexus-utils to 3.0.22
- DateTimeZone data updated to version 2015f
- Fixed to handle JDK 8u60 [#288, #291]
Without this fix, formatting a time-zone will print "+00:00" instead of "GMT" for the GMT time-zone
- DateTimeZone data updated to version 2015e
Renaming the distribution's artifactIds to "elasticsearch" caused the eclipse
import process to balk. Fix those modules by moving their eclipse package to
"[groupId].[artifactId]".
Robert Muir
Adrien Grand
Nicholas Knize
Simon Willnauer
David Pilato
Jason Tedor
Uwe Schindler
Robert Muir
Tanguy Leroux
Colin Goodheart-Smithe
Nik Everett
Britta Weber
Britta Weber
Martijn Laarman
Ryan Ernst