Commit Graph

1112 Commits

Author SHA1 Message Date
Nik Everett d3a4fe9a8e
Docs: Wrap expert script example to fit in docs (#34201)
This slightly reworks the expert script plugin example so it fits on the
page when the docs are rendered. The box in which it is rendered is not
very wide so it took a bit of twisting to make it readable.
2018-10-02 12:25:58 -04:00
David Turner 2aff005a69
Clean up TransportMasterNodeAction (#34076)
Mainly this fixes a warning by replacing the unchecked `new ActionListener`
with the checked `new ActionListener<Response>`, and it also fixes the line
length violations in this class.
2018-10-02 03:17:55 +01:00
Nik Everett ad3218b4ab
Checkstyle: Catch wide snippets (#34163)
We use wrap code in `// tag` and `//end` to include it in our docs. Our
current docs style wraps code snippets in a box that is only wide enough
for 76 characters and adds a horizontal scroll bar for wider snippets
which makes the snippet much harder to read. This adds a checkstyle check
that looks for java code that is included in the docs and is wider than
that 76 characters so all snippets fit into the box. It solves many of
the failures that this catches but suppresses many more. I will clean
those up in a follow up change.
2018-10-01 15:22:15 -04:00
Jason Tedor 128ea1b7b5
Add Ubuntu 18.04 to packaging tests (#34139)
We intend to start supporting Ubuntu 18.04 with the 6.5.0 release. This
commit adds Ubuntu 18.04 to the packaging tests.
2018-09-28 17:33:29 -04:00
Christoph Büscher b654d986d7
Add OneStatementPerLineCheck to Checkstyle rules (#33682)
This change adds the OneStatementPerLineCheck to our checkstyle precommit
checks. This rule restricts the number of statements per line to one. The
resoning behind this is that it is very difficult to read multiple statements on
one line. People seem to mostly use it in short lambdas and switch statements in
our code base, but just going through the changes already uncovered some actual
problems in randomization in test code, so I think its worth it.
2018-09-21 11:52:31 +02:00
Nik Everett 3ede13a454
Test framework fall cleaning (#33423)
Wraps all lines in our test framework at 140 characters because that is
our standard line length and removes all of the checkstyle suppressions
for the test framework.

Drops most of `ModuleTestCase` because it isn't used and we're moving
away from using guice in the way that it wants to test anyway. Also
switches a few classes that extend it but don't use it to extend
`ESTestCase` instead.
2018-09-19 14:34:02 -04:00
Ryan Ernst 99513b306e
Test: Relax jarhell gradle test (#33787)
Gradle can sometimes emit mixed log lines due to how we spawn things in
separate processes. This commit changes the jarhell integ test to only
look for the exception and message, instead of including the outer part
about the exception in thread main.

closes #33774
2018-09-18 13:11:46 -07:00
Jim Ferenczi 241c74efb2
upgrade to a new snapshot of Lucene 8 (7d0a7782fa) (#33812) 2018-09-18 18:16:40 +02:00
Nik Everett bc12a948b5
Checkstyle: Package declarations (#33784)
Make sure that all java files have a package declaration and that all of
the package declarations line up with the directory structure. This
would have caught the bug that I caused in
190ea9a6de and fixed in
b6d68bd805.
2018-09-18 12:04:35 -04:00
Armin Braun 87cedef3cf
NETWORKING:Def CName in Http Publish Addr to True (#33631)
* Follow up to #32806 setting the setting to true for 7.x
2018-09-18 10:29:02 +02:00
Ryan Ernst e686909768
Build: Change test task sanity check to be per project (#33544)
This commit changes the sanity check which ensures the test task was
properly replaced with randomized testing to have a per project check,
isntead of a global one. The previous global check assumed all test
tasks within the root project and below should be randomized testing,
but that is not the case for a multi project in which only one project
is an elasticsearch plugin. While the new check is not able to emit all
of the failed replacements in one error message, the efficacy of the
check remains.
2018-09-17 15:10:16 -07:00
Alpar Torok 5ca6f31205
Move precommit task implementation to java (#33407)
Replace precommit tasks that execute with Java implementations
2018-09-17 14:09:28 +03:00
Alpar Torok 896b3864a8
Favor running compile tasks before pre-commit (#33424) 2018-09-17 12:04:57 +03:00
Ryan Ernst 3046656ab1
Scripting: Rework joda time backcompat (#33486)
This commit switches the joda time backcompat in scripting to use
augmentation over ZonedDateTime. The augmentation methods provide
compatibility with the missing methods between joda's DateTime and
java's ZonedDateTime. Due to getDayOfWeek returning an enum in the java
API, ZonedDateTime is wrapped so that the method can return int like the
joda time does. The java time api version is renamed to
getDayOfWeekEnum, which will be kept through 7.x for compatibility while
users switch back to getDayOfWeek once joda compatibility is removed.
2018-09-16 19:18:00 -07:00
Vladimir Dolzhenko 141c6ef93e
upgrade randomizedrunner to 2.7.0 (#33623) 2018-09-12 17:18:34 +02:00
Armin Braun 94cdf0ceba
NETWORKING: http.publish_host Should Contain CNAME (#32806)
* NETWORKING: http.publish_host Should Contain CNAME

* Closes #22029
2018-09-12 06:15:36 +02:00
Alan Woodward 39c3234c2f
Upgrade to latest Lucene snapshot (#33505)
* LeafCollector.setScorer() now takes a Scorable
* Scorers may not have null Weights
* IndexWriter.getFlushingBytes() reports how much memory is being used by IW threads writing to disk
2018-09-10 20:51:55 +01:00
Tanguy Leroux 079d130d8c
[Test] Remove duplicate method in TestShardRouting (#32815) 2018-09-10 18:29:00 +02:00
Tanguy Leroux 80c4661d0c
Fix typos (#33499) 2018-09-10 14:20:04 +02:00
David Turner 284c45a6ff
Strengthen FilterRoutingTests (#33149)
Today the FilterRoutingTests take the belt-and-braces approach of excluding
some node attribute values and including some others. This means that we don't
really test that both inclusion and exclusion work correctly: as long as one of
them works as expected then the test will pass. This change improves these
tests by only using one approach at once, demonstrating that both do indeed
work, and adds tests for various other scenarios too.
2018-09-10 11:23:05 +02:00
Nik Everett 190ea9a6de
Logging: Configure the node name when we have it (#32983)
Change the logging infrastructure to handle when the node name isn't
available in `elasticsearch.yml`. In that case the node name is not
available until long after logging is configured. The biggest change is
that the node name logging no longer fixed at pattern build time.
Instead it is read from a `SetOnce` on every print. If it is unset it is
printed as `unknown` so we have something that fits in the pattern.
On normal startup we don't log anything until the node name is available
so we never see the `unknown`s.
2018-09-07 14:31:23 -04:00
Jim Ferenczi 79cd6385fe
Collapse package structure for metrics aggs (#33463)
This change collapses all metrics aggregations classes into a single package `org.elasticsearch.aggregations.metrics`.
It also restricts the visibility of some classes (aggregators and factories) that should not be used outside of the package.

Relates #22868
2018-09-07 10:58:06 +02:00
Jim Ferenczi 7ad71f906a
Upgrade to a Lucene 8 snapshot (#33310)
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
2018-09-06 14:42:06 +02:00
Nik Everett 74b87989d9
Build: Merge xpack checkstyle config into core (#33399)
Solves all of the xpack line length suppressions and then merges the
remainder of the xpack checkstyle_suppressions.xml file into the core
checkstyle_suppressions.xml file. At this point that just means the
antlr generated files for sql.

It also adds an exclusion to the line length tests for javadocs that
are just a URL. We have one such javadoc and breaking up the line would
make the link difficult to use.
2018-09-05 09:17:02 -04:00
Alpar Torok 7f7e8fd733
Disable assemble task instead of removing it (#33348) 2018-09-04 07:32:14 +03:00
Jason Tedor 09bf4e5f00
Introduce private settings (#33327)
This commit introduces the formal notion of a private setting. This
enables us to register some settings that we had previously not
registered as fully-fledged settings to avoid them being exposed via
APIs such as the create index API. For example, we had hacks in the
codebase to allow index.version.created to be passed around inside of
settings objects, but was not registered as a setting so that if a user
tried to use the setting on any API then they would get an
exception. This prevented users from setting index.version.created on
index creation, or updating it via the index settings API. By
introducing private settings, we can continue to reject these attempts,
yet now we can represent these settings as actual settings. In this
change, we register index.version.created as an actual setting. We do
not cutover all settings that we had been treating as private in this
pull request, it is already quite large due to moving some tests around
to account for the fact that some tests need to be able to set the
index.version.created. This can be done in a follow-up change.
2018-09-03 19:17:57 -04:00
Alpar Torok 3c367a2c46
Fix extractjar task ci (#33272)
Remove tasks to check license and notice and add build integration test instead.

Closes #33201
2018-09-03 10:18:34 +03:00
Alpar Torok db3d32ce91
Fix pom for build-tools (#33300)
Looks like `java-gradle-plugin` reconfigures the pom.
Stop using it since we don't publish to Gradle plugin portal.
2018-08-31 16:48:00 +03:00
Alpar Torok a84a20844b
Lazy evaluate java9home (#33301) 2018-08-31 16:36:57 +03:00
Alpar Torok 44ed5f6306
Enable forbiddenapis server java9 (#33245) 2018-08-31 09:31:55 +03:00
Alpar Torok 5cf6e0d4bc
Ignore module-info in jar hell checks (#33011)
* Ignore module-info in JarHell checks
* Add unit test
* integration test to test that jarhell is ran with precommit
2018-08-30 11:41:39 +03:00
Alpar Torok 9c541b8f72
Upgrade to latest Gradle 4.10 (#32801)
Upgrade to Gradle 4.10
2018-08-30 10:20:38 +03:00
Alpar Torok f097446066
Fix/30904 cluster formation part2 (#32877)
Gradle integration for the Cluster formation plugin with ref counting
2018-08-30 10:17:42 +03:00
Alpar Torok f29f0af7bc
Consider multi release jars when running third party audit (#33206)
Exclude classes meant for newer versions than what we are auditing against, those classes won't be found. There's no reason to exclude JDK classes from newer versions, with this PR, we will not extract them in the first place.
2018-08-29 09:53:04 +03:00
Jonathan Little 9d92a87ae6 Remove support for deprecated params._agg/_aggs for scripted metric aggregations (#32979) 2018-08-28 09:27:43 +01:00
Alpar Torok 2cc611604f
Run Third party audit with forbidden APIs CLI (part3/3) (#33052)
The new implementation is functional equivalent with the old, ant based one.
It parses task standard error to get the missing classes and violations in the same way.
I considered re-using ForbiddenApisCliTask but Gradle makes it hard to build inheritance with tasks that have task actions , since the order of the task actions can't be controlled.
This inheritance isn't dully desired either as the third party audit task is much more opinionated and we don't want to expose some of the configuration.
We could probably extract a common base class without any task actions, but probably more trouble than it's worth.

Closes #31715
2018-08-28 10:03:30 +03:00
Alpar Torok 71d5c66fd3
Fix plugin build test on Windows (#33078)
Fix plugin build test on Windows
2018-08-28 10:00:11 +03:00
Jay Modi 309fb22181
Build: forked compiler max memory matches jvmArgs (#33138)
This commit removes the setting of the fork options maximum memory size
in our build plugin and instead adds the value in the gradle.properties
file to be alongside the value set in jvmArgs.

This change is necessary when using parallel compilation as 512m is not
sufficient for parallel compilation on some machines.
2018-08-27 10:26:25 -06:00
Alpar Torok 974f839093
Fix forbiddenapis on java 11 (#33116)
Cap forbiddenapis to java version 10
2018-08-27 08:47:42 +03:00
Jim Ferenczi f4e9729d64
Remove unsupported Version.V_5_* (#32937)
This change removes the es 5x version constants and their usages.
2018-08-24 09:51:21 +02:00
Alpar Torok edd477a15e Fix the default pom file name (#33063)
Before this change the default was fixed at compile time and not picking
up changes in the build script.
2018-08-22 14:53:42 -04:00
Alpar Torok 82d10b484a
Run forbidden api checks with runtimeJavaVersion (#32947)
Run forbidden APIs checks with runtime hava version
2018-08-22 09:05:22 +03:00
Nik Everett 2c81d7f77e
Build: Rework shadow plugin configuration (#32409)
This reworks how we configure the `shadow` plugin in the build. The major
change is that we no longer bundle dependencies in the `compile` configuration,
instead we bundle dependencies in the new `bundle` configuration. This feels
more right because it is a little more "opt in" rather than "opt out" and the
name of the `bundle` configuration is a little more obvious.

As an neat side effect of this, the `runtimeElements` configuration used when
one project depends on another now contains exactly the dependencies needed
to run the project so you no longer need to reference projects that use the
shadow plugin like this:

```
testCompile project(path: ':client:rest-high-level', configuration: 'shadow')
```

You can instead use the much more normal:

```
testCompile "org.elasticsearch.client:elasticsearch-rest-high-level-client:${version}"
```
2018-08-21 20:03:28 -04:00
Alpar Torok e3700a9b8d
Only configure publishing if it's applied externally (#32351)
Only configure publishing if it's applied externally, reconfigure for hasClientJar
2018-08-20 13:46:21 +03:00
Alpar Torok 4b34b3f4aa
Set forbidden APIs target compatibility to compiler java version (#32935)
Set forbidden apis target compatibility to compiler version

Fix outstanding deprecation
2018-08-20 09:27:02 +03:00
Ioannis Kakavas e3aa68b0a9
[TEST] Run pre 6.4 nodes in non-FIPS JVMs (#32901)
Elasticsearch versions earlier than 6.4.0 cannot properly run in a
FIPS 140 JVM. This commit ensures that we use a non-FIPS JVM for
nodes that we spin up in BWC tests even when we're testing FIPS.
2018-08-17 18:23:13 +03:00
Jonathan Little a08127c072 Scripted metric aggregations: add deprecation warning and system property to control legacy params (#31597)
* Scripted metric aggregations: add deprecation warning and system property to control legacy params

Scripted metric aggregation params._agg/_aggs are replaced by state/states context variables. By default the old params are still present, and a deprecation warning is emitted when Scripted Metric Aggregations are used. A new system property can be used to disable the legacy params. This functionality will be removed in a future revision.

* Fix minor style issue and docs test failure

* Disable deprecated params._agg/_aggs in tests and revise tests to use state/states instead

* Add integration test covering deprecated scripted metrics aggs params._agg/_aggs access

* Disable deprecated params._agg/_aggs in docs integration tests and revise stored scripts to use state/states instead

* Revert unnecessary migrations doc change

A relevant note should be added in the changes destined for 7.0; this PR is going to be backported to 6.x.

* Replace deprecated _agg param bwc integration test with a couple of unit tests

* Fix compatibility test after merge

* Rename backwards compatibility system property per code review feedback

* Tweak deprecation warning text per review feedback
2018-08-17 13:11:18 +01:00
Alpar Torok d16562eab5
RFC: Test that example plugins build stand-alone (#32235)
Add tests for build-tools to make sure example plugins build stand-alone using it.

This will catch issues such as referencing files from the buildSrc directly, breaking external uses of build-tools.
2018-08-17 09:41:39 +03:00
Alpar Torok bb3eacf384
Revert "cluster formation DSL - Gradle integration - part 2 (#32028)" (#32876)
This reverts commit e680a64434.
2018-08-15 19:17:46 +03:00
Alpar Torok e680a64434
cluster formation DSL - Gradle integration - part 2 (#32028)
* Implement Version in java

- This allows to move all  all .java files from .groovy.
- Will prevent eclipse from tangling up in this setup
- make it possible to use Version from Java

* PR review comments

* Cluster formation plugin with reference counting

```
> Task :plugins:ingest-user-agent:listElasticSearchClusters
Starting cluster: myTestCluster
   * myTestCluster: /home/alpar/work/elastic/elasticsearch/plugins/ingest-user-agent/foo
Asked to unClaimAndStop myTestCluster, since cluster still has 1 claim it will not be stopped

> Task :plugins:ingest-user-agent:testme UP-TO-DATE
Stopping myTestCluster, since no of claims is 0
```

- Meant to auto manage the clusters lifecycle
- Add integration test for cluster formation

* Fix rebase

* Change to `useCluster` method on task
2018-08-15 19:16:10 +03:00