From fbd8f6927380aae4557166fe43795f49b774eb34 Mon Sep 17 00:00:00 2001 From: David Pilato Date: Tue, 11 Aug 2015 15:22:58 +0200 Subject: [PATCH] Build of QA: Smoke Test Shaded Jar fails under maven 3.3.x Build fails with maven 3.3.1 and 3.3.3. To reproduce, install one of the 3.3.x versions of maven and run `mvn clean verify` in the root directory of the project. The build will fail in the QA: Smoke Test Shaded Jar module with the following error: ``` Started J0 PID(99979@flea.local). Suite: org.elasticsearch.shaded.test.ShadedIT 2> NOTE: reproduce with: ant test -Dtestcase=ShadedIT -Dtests.method=testJodaIsNotOnTheCP -Dtests.seed=2F4D23A7462CF921 -Dtests.locale= -Dtests.timezone=Asia/Baku -Dtests.asserts=true -Dtests.file.encoding=UTF-8 FAILURE 0.06s | ShadedIT.testJodaIsNotOnTheCP <<< > Throwable #1: junit.framework.AssertionFailedError: Expected an exception but the test passed: java.lang.ClassNotFoundException > at __randomizedtesting.SeedInfo.seed([2F4D23A7462CF921:3A9404F1F69FD80]:0) > at junit.framework.Assert.fail(Assert.java:57) > at java.lang.Thread.run(Thread.java:745) 2> NOTE: reproduce with: ant test -Dtestcase=ShadedIT -Dtests.method=testGuavaIsNotOnTheCP -Dtests.seed=2F4D23A7462CF921 -Dtests.locale= -Dtests.timezone=Asia/Baku -Dtests.asserts=true -Dtests.file.encoding=UTF-8 FAILURE 0.01s | ShadedIT.testGuavaIsNotOnTheCP <<< > Throwable #1: junit.framework.AssertionFailedError: Expected an exception but the test passed: java.lang.ClassNotFoundException > at __randomizedtesting.SeedInfo.seed([2F4D23A7462CF921:C2502FD54D83433D]:0) > at junit.framework.Assert.fail(Assert.java:57) > at java.lang.Thread.run(Thread.java:745) 2> NOTE: reproduce with: ant test -Dtestcase=ShadedIT -Dtests.method=testjsr166eIsNotOnTheCP -Dtests.seed=2F4D23A7462CF921 -Dtests.locale= -Dtests.timezone=Asia/Baku -Dtests.asserts=true -Dtests.file.encoding=UTF-8 FAILURE 0.01s | ShadedIT.testjsr166eIsNotOnTheCP <<< > Throwable #1: junit.framework.AssertionFailedError: Expected an exception but the test passed: java.lang.ClassNotFoundException > at __randomizedtesting.SeedInfo.seed([2F4D23A7462CF921:35593286F4269392]:0) > at junit.framework.Assert.fail(Assert.java:57) > at java.lang.Thread.run(Thread.java:745) 2> NOTE: leaving temporary files on disk at: /Users/Shared/Jenkins/Home/workspace/elasticsearch-master/qa/smoke-test-shaded/target/J0/temp/org.elasticsearch.shaded.test.ShadedIT_2F4D23A7462CF921-001 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=, timezone=Asia/Baku 2> NOTE: Mac OS X 10.10.4 x86_64/Oracle Corporation 1.8.0_25 (64-bit)/cpus=8,threads=1,free=482137936,total=514850816 2> NOTE: All tests run in this JVM: [ShadedIT] Completed [1/1] in 6.61s, 5 tests, 3 failures <<< FAILURES! Tests with failures: - org.elasticsearch.shaded.test.ShadedIT.testJodaIsNotOnTheCP - org.elasticsearch.shaded.test.ShadedIT.testGuavaIsNotOnTheCP - org.elasticsearch.shaded.test.ShadedIT.testjsr166eIsNotOnTheCP ``` Please note that build doesn't fail with maven 3.2.x and it doesn't fail if mvn command is executed inside the qa/smoke-test-shaded directory. Only when the build is started from the root directory the error above can be observed. The reason is because of the shaded version which depends on elasticsearch core. When Maven build the module only, then elasticsearch core is not added to the dependency tree. ```sh mvn dependency:tree -pl :smoke-test-shaded ``` ``` [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ smoke-test-shaded --- [INFO] org.elasticsearch.qa:smoke-test-shaded:jar:2.0.0-beta1-SNAPSHOT [INFO] +- org.elasticsearch.distribution.shaded:elasticsearch:jar:2.0.0-beta1-SNAPSHOT:compile [INFO] | +- org.apache.lucene:lucene-core:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-backward-codecs:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-analyzers-common:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queries:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-memory:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-highlighter:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queryparser:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-sandbox:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-suggest:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-misc:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-join:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-grouping:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-spatial:jar:5.2.1:compile [INFO] | \- com.spatial4j:spatial4j:jar:0.4.1:compile [INFO] +- org.hamcrest:hamcrest-all:jar:1.3:test [INFO] \- org.apache.lucene:lucene-test-framework:jar:5.2.1:test [INFO] +- org.apache.lucene:lucene-codecs:jar:5.2.1:test [INFO] +- com.carrotsearch.randomizedtesting:randomizedtesting-runner:jar:2.1.16:test [INFO] +- junit:junit:jar:4.11:test [INFO] \- org.apache.ant:ant:jar:1.8.2:test ``` But if shaded plugin is involved during the build, it modifies the `projectArtifactMap`: ```sh mvn dependency:tree -pl org.elasticsearch.distribution.shaded:elasticsearch,:smoke-test-shaded ``` ``` [INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ smoke-test-shaded --- [INFO] org.elasticsearch.qa:smoke-test-shaded:jar:2.0.0-beta1-SNAPSHOT [INFO] +- org.elasticsearch.distribution.shaded:elasticsearch:jar:2.0.0-beta1-SNAPSHOT:compile [INFO] | \- org.elasticsearch:elasticsearch:jar:2.0.0-beta1-SNAPSHOT:compile [INFO] | +- org.apache.lucene:lucene-backward-codecs:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-analyzers-common:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queries:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-memory:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-highlighter:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-queryparser:jar:5.2.1:compile [INFO] | | \- org.apache.lucene:lucene-sandbox:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-suggest:jar:5.2.1:compile [INFO] | | \- org.apache.lucene:lucene-misc:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-join:jar:5.2.1:compile [INFO] | | \- org.apache.lucene:lucene-grouping:jar:5.2.1:compile [INFO] | +- org.apache.lucene:lucene-spatial:jar:5.2.1:compile [INFO] | | \- com.spatial4j:spatial4j:jar:0.4.1:compile [INFO] | +- com.google.guava:guava:jar:18.0:compile [INFO] | +- com.carrotsearch:hppc:jar:0.7.1:compile [INFO] | +- joda-time:joda-time:jar:2.8:compile [INFO] | +- org.joda:joda-convert:jar:1.2:compile [INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.5.3:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.5.3:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.5.3:compile [INFO] | | \- org.yaml:snakeyaml:jar:1.12:compile [INFO] | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.5.3:compile [INFO] | +- io.netty:netty:jar:3.10.3.Final:compile [INFO] | +- com.ning:compress-lzf:jar:1.0.2:compile [INFO] | +- com.tdunning:t-digest:jar:3.0:compile [INFO] | +- org.hdrhistogram:HdrHistogram:jar:2.1.6:compile [INFO] | +- org.apache.commons:commons-lang3:jar:3.3.2:compile [INFO] | +- commons-cli:commons-cli:jar:1.3.1:compile [INFO] | \- com.twitter:jsr166e:jar:1.1.0:compile [INFO] +- org.hamcrest:hamcrest-all:jar:1.3:test [INFO] \- org.apache.lucene:lucene-test-framework:jar:5.2.1:test [INFO] +- org.apache.lucene:lucene-codecs:jar:5.2.1:test [INFO] +- org.apache.lucene:lucene-core:jar:5.2.1:compile [INFO] +- com.carrotsearch.randomizedtesting:randomizedtesting-runner:jar:2.1.16:test [INFO] +- junit:junit:jar:4.11:test [INFO] \- org.apache.ant:ant:jar:1.8.2:test ``` A fix could consist of fixing something on Maven side. Probably something changed in a recent version and introduced this "issue" but it might be not really an issue. More a fix. There are two workarounds: 1) exclude manually elasticsearch core from shaded version in smoke-test-shaded module and add manually each lucene lib needed by elasticsearch 2) add a new `elasticsearch-lucene` (lucene) POM module which simply declares all needed lucene libs in subprojects (such as the smoke tester one). I choose the later. Closes #12791. --- core/pom.xml | 42 +++--------------------- distribution/shaded/pom.xml | 1 + lucene/pom.xml | 62 ++++++++++++++++++++++++++++++++++++ pom.xml | 1 + qa/smoke-test-shaded/pom.xml | 14 +++++++- 5 files changed, 81 insertions(+), 39 deletions(-) create mode 100644 lucene/pom.xml diff --git a/core/pom.xml b/core/pom.xml index 8782a9ee95e..81b6498a8d7 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -44,44 +44,10 @@ - org.apache.lucene - lucene-core - - - org.apache.lucene - lucene-backward-codecs - - - org.apache.lucene - lucene-analyzers-common - - - org.apache.lucene - lucene-queries - - - org.apache.lucene - lucene-memory - - - org.apache.lucene - lucene-highlighter - - - org.apache.lucene - lucene-queryparser - - - org.apache.lucene - lucene-suggest - - - org.apache.lucene - lucene-join - - - org.apache.lucene - lucene-spatial + org.elasticsearch + elasticsearch-lucene + ${elasticsearch.version} + pom org.apache.lucene diff --git a/distribution/shaded/pom.xml b/distribution/shaded/pom.xml index 7024504a7ec..2302d1db2a6 100644 --- a/distribution/shaded/pom.xml +++ b/distribution/shaded/pom.xml @@ -77,6 +77,7 @@ ${project.build.directory}/dependency-reduced-pom.xml + org.elasticsearch:elasticsearch-lucene org.apache.lucene:* com.spatial4j:* diff --git a/lucene/pom.xml b/lucene/pom.xml new file mode 100644 index 00000000000..e423d9e9a70 --- /dev/null +++ b/lucene/pom.xml @@ -0,0 +1,62 @@ + + + + elasticsearch-parent + org.elasticsearch + 2.0.0-beta1-SNAPSHOT + + 4.0.0 + + elasticsearch-lucene + Lucene dependencies for elasticsearch + pom + + + true + + + + + org.apache.lucene + lucene-core + + + org.apache.lucene + lucene-backward-codecs + + + org.apache.lucene + lucene-analyzers-common + + + org.apache.lucene + lucene-queries + + + org.apache.lucene + lucene-memory + + + org.apache.lucene + lucene-highlighter + + + org.apache.lucene + lucene-queryparser + + + org.apache.lucene + lucene-suggest + + + org.apache.lucene + lucene-join + + + org.apache.lucene + lucene-spatial + + + diff --git a/pom.xml b/pom.xml index a66f5ba5b43..8fc826bc7e5 100644 --- a/pom.xml +++ b/pom.xml @@ -1487,6 +1487,7 @@ org.eclipse.jdt.ui.text.custom_code_templates= dev-tools rest-api-spec + lucene core distribution plugins diff --git a/qa/smoke-test-shaded/pom.xml b/qa/smoke-test-shaded/pom.xml index 0b968e13d55..d0bb0ede85f 100644 --- a/qa/smoke-test-shaded/pom.xml +++ b/qa/smoke-test-shaded/pom.xml @@ -23,7 +23,19 @@ org.elasticsearch.distribution.shaded elasticsearch - 2.0.0-SNAPSHOT + ${elasticsearch.version} + + + org.elasticsearch + elasticsearch + + + + + org.elasticsearch + elasticsearch-lucene + ${elasticsearch.version} + pom org.hamcrest