Commit Graph

284 Commits

Author SHA1 Message Date
Ryan Ernst 20c37efea2
Build: Replace provided configuration with compileOnly (#28564)
When elasticsearch was originally moved to gradle, the "provided" equivalent in maven had to be done through a plugin. Since then, gradle added the "compileOnly" configuration. This commit removes the provided plugin and replaces all uses with compileOnly.
2018-02-09 11:30:24 -08:00
Ryan Ernst 7e65cd8ca8
Build: Add warning when bwc tests are disabled (#28585)
The bwc tests can be disabled in order to facilitate commits necessary
to older branches to maintain backcompat. A check already exists to
ensure this flag is not left disabled too long (once a day by the
branchConsistency check). However, it can be surprising if you try
running bwc tests explicitly and they look like nothing is happening.
This commit adds a warning during configuration to ensure it is clear
the bwc tests are disabled and enforces a link to a PR which is in the
process of being backported.
2018-02-09 11:26:22 -08:00
Ryan Ernst 578773f11b Re-enable bwc tests after backport of #28556 2018-02-08 11:53:48 -08:00
Ryan Ernst a55eda626f
Plugins: Store elasticsearch and java versions in PluginInfo (#28556)
Plugin descriptors currently contain an elasticsearch version,
which the plugin was built against, and a java version, which the plugin
was built with. These versions are read and validated, but not stored.
This commit keeps them in PluginInfo so they can be used later.
While seeing the elasticsearch version is less interesting (since it is
enforced to match that of the running elasticsearc node), the java
version is interesting since we only validate the format, not the actual
version. This also makes PluginInfo have full parity with the plugin
properties file.
2018-02-08 08:31:39 -08:00
markharwood 998461c737
Test fix - reenable BWC tests and lower version checks now that PR 28440 for allowPartialSearchResults flag backported to 6.x (#28482)
Support for allowPartialSearchResults is now in 6.3 so changing master BWC checks accordingly
2018-02-01 19:20:21 +00:00
Jason Tedor f792606f1c Re-disable BWC tests after inadvertent re-enabling
These tests were disabled to facilitate backport of a PR which is not
yet complete. These tests were accidentally reenabled after a merge
conflict was resolved in the wrong direction. This commit addresses this
issue.
2018-01-31 19:14:31 -05:00
Jason Tedor 1b3d529bef Introduce secure security manager to project
This commit migrates SecureSM, our secure security manager
implementation, from its own repository to being a sub-project of
Elasticsearch.
2018-01-31 18:23:28 -05:00
markharwood 77d2dd203e
Search - add allow_partial_search_results flag with default setting false (#28440)
Adds allow_partial_search_results flag to search requests with default setting = true.
When false, will error if search either timeouts, has partial errors or has missing shards rather
than returning partial search results. A cluster-level setting provides a default for search requests with no flag.

Closes #27435
2018-01-31 15:51:29 +00:00
Tim Brooks 3895add2ca
Introduce elasticsearch-core jar (#28191)
This is related to #27933. It introduces a jar named elasticsearch-core
in the lib directory. This commit moves the JarHell class from server to
elasticsearch-core. Additionally, PathUtils and some of Loggers are
moved as JarHell depends on them.
2018-01-15 09:59:01 -07:00
Tim Brooks 99f88f15c5
Rename core module to server (#28180)
This is related to #27933. It renames the core module to server. This is
the first step towards introducing an elasticsearch-core jar.
2018-01-11 11:30:43 -07:00
Jason Tedor 480aeb7eb7
Introduce Gradle wrapper
We have agreed to introduce the Gradle wrapper to simplify workflows for
developers, and managing infrastructure (e.g., CI, release builds, etc.)
as well as consideration for the fact that other projects in our stack
use Gradle and do not necessarily want to be tied to our Gradle version.

Relates #28065
2018-01-04 16:36:13 -05:00
Ryan Ernst cb2334a78b Re-enable bwc tests after #27881 was backported 2018-01-04 09:58:38 -08:00
Ryan Ernst d36ec18029
Plugins: Add plugin extension capabilities (#27881)
This commit adds the infrastructure to plugin building and loading to
allow one plugin to extend another. That is, one plugin may extend
another by the "parent" plugin allowing itself to be extended through
java SPI. When all plugins extending a plugin are finished loading, the
"parent" plugin has a callback (through the ExtensiblePlugin interface)
allowing it to reload SPI.

This commit also adds an example plugin which uses as-yet implemented
extensibility (adding to the painless whitelist).
2018-01-03 11:12:43 -08:00
Tim Brooks 06b313025c
Add elasticsearch-nio jar for base nio classes (#27801)
This is related to #27802. This commit adds a jar called
elasticsearch-nio that contains the base nio classes that will be used
for the tcp nio transport and eventually the http nio transport.

The jar does not depend on elasticsearch:core, so all references to core
have been removed.
2017-12-20 16:29:16 -06:00
Christoph Büscher 5406a9f30d Add rank-eval module to transport client and HL client dependencies 2017-12-13 18:05:43 +01:00
David Turner 89ba8996c6 Consolidate version numbering semantics (#27397)
Fixes to the build system, particularly around BWC testing, and to make future
version bumps less painful.
2017-11-23 20:21:53 +00:00
Jim Ferenczi 6319424e4a
Move composite aggregation to core (#27474)
This change removes the module named aggs-composite and adds the `composite` aggs
as a core aggregation. This allows other plugins to use this new aggregation
and simplifies the integration in the HL rest client.
2017-11-21 13:31:01 +01:00
Michael Basnight cb3e8f4763
Move the CLI into its own subproject (#27114)
Projects the depend on the CLI currently depend on core. This should not
always be the case. The EnvironmentAwareCommand will remain in :core,
but the rest of the CLI components have been moved into their own
subproject of :core, :core:cli.
2017-11-18 21:42:57 -06:00
David Turner 9766b858d0
Prepare for bump to 6.0.1 on the master branch (#27391)
An assortment of fixes, particularly to version number calculations, in preparation for the bump to 6.0.1.
2017-11-16 18:38:54 +00:00
Jim Ferenczi 623367d793
Add composite aggregator (#26800)
* This change adds a module called `aggs-composite` that defines a new aggregation named `composite`.
The `composite` aggregation is a multi-buckets aggregation that creates composite buckets made of multiple sources.
The sources for each bucket can be defined as:
  * A `terms` source, values are extracted from a field or a script.
  * A `date_histogram` source, values are extracted from a date field and rounded to the provided interval.
This aggregation can be used to retrieve all buckets of a deeply nested aggregation by flattening the nested aggregation in composite buckets.
A composite buckets is composed of one value per source and is built for each document as the combinations of values in the provided sources.
For instance the following aggregation:

````
"test_agg": {
  "terms": {
    "field": "field1"
  },
  "aggs": {
    "nested_test_agg":
      "terms": {
        "field": "field2"
      }
  }
}
````
... which retrieves the top N terms for `field1` and for each top term in `field1` the top N terms for `field2`, can be replaced by a `composite` aggregation in order to retrieve **all** the combinations of `field1`, `field2` in the matching documents:

````
"composite_agg": {
  "composite": {
    "sources": [
      {
	"field1": {
          "terms": {
              "field": "field1"
            }
        }
      },
      {
	"field2": {
          "terms": {
            "field": "field2"
          }
        }
      },
    }
  }
````

The response of the aggregation looks like this:

````
"aggregations": {
  "composite_agg": {
    "buckets": [
      {
        "key": {
          "field1": "alabama",
          "field2": "almanach"
        },
        "doc_count": 100
      },
      {
        "key": {
          "field1": "alabama",
          "field2": "calendar"
        },
        "doc_count": 1
      },
      {
        "key": {
          "field1": "arizona",
          "field2": "calendar"
        },
        "doc_count": 1
      }
    ]
  }
}
````

By default this aggregation returns 10 buckets sorted in ascending order of the composite key.
Pagination can be achieved by providing `after` values, the values of the composite key to aggregate after.
For instance the following aggregation will aggregate all composite keys that sorts after `arizona, calendar`:

````
"composite_agg": {
  "composite": {
    "after": {"field1": "alabama", "field2": "calendar"},
    "size": 100,
    "sources": [
      {
	"field1": {
          "terms": {
            "field": "field1"
          }
        }
      },
      {
	"field2": {
          "terms": {
            "field": "field2"
          }
	}
      }
    }
  }
````

This aggregation is optimized for indices that set an index sorting that match the composite source definition.
For instance the aggregation above could run faster on indices that defines an index sorting like this:

````
"settings": {
  "index.sort.field": ["field1", "field2"]
}
````

In this case the `composite` aggregation can early terminate on each segment.
This aggregation also accepts multi-valued field but disables early termination for these fields even if index sorting matches the sources definition.
This is mandatory because index sorting picks only one value per document to perform the sort.
2017-11-16 15:13:36 +01:00
Boaz Leskes ace446f335 Update shrink's bwc version to 6.1.0 and enabled bwc tests 2017-11-07 15:35:46 +01:00
Boaz Leskes 2fc6c64c82 Disable bwc tests in preparation of backporting #26931 2017-11-07 10:58:45 +01:00
Yannick Welsch 31f43f5c34 Reenable BWC tests
Relates to #26682
2017-09-22 16:54:00 +02:00
Yannick Welsch df5c450e89 Add v6.1 BWC layer for adding wait_for_active_shards to index open command
This commit disables BWC tests while adding a v6.1 BWC layer for the PR #26682
2017-09-22 16:30:07 +02:00
Jason Tedor 954fb1c80d Reenable BWC tests after global checkpoint sync
This commit reenables the BWC tests after the introduction of the
post-operation and background global checkpoint sync.

Relates #26591
2017-09-21 15:44:36 -04:00
Jason Tedor f35d1de502 Introduce global checkpoint background sync
It is the exciting return of the global checkpoint background
sync. Long, long ago, in snapshot version far, far away we had and only
had a global checkpoint background sync. This sync would fire
periodically and send the global checkpoint from the primary shard to
the replicas so that they could update their local knowledge of the
global checkpoint. Later in time, as we sped ahead towards finalizing
the initial version of sequence IDs, we realized that we need the global
checkpoint updates to be inline. This means that on a replication
operation, the primary shard would piggy back the global checkpoint with
the replication operation to the replicas. The replicas would update
their local knowledge of the global checkpoint and reply with their
local checkpoint. However, this could allow the global checkpoint on the
primary to advance again and the replicas would fall behind in their
local knowledge of the global checkpoint. If another replication
operation never fired, then the replicas would be permanently behind. To
account for this, we added one more sync that would fire when the
primary shard fell idle. However, this has problems:
 - the shard idle timer defaults to five minutes, a long time to wait
   for the replicas to learn of the new global checkpoint
 - if a replica missed the sync, there was no follow-up sync to catch
   them up
 - there is an inherent race condition where the primary shard could
   fall idle mid-operation (after having sent the replication request to
   the replicas); in this case, there would never be a background sync
   after the operation completes
 - tying the global checkpoint sync to the idle timer was never natural

To fix this, we add two additional changes for the global checkpoint to
be synced to the replicas. The first is that we add a post-operation
sync that only fires if there are no operations in flight and there is a
lagging replica. This gives us a chance to sync the global checkpoint to
the replicas immediately after an operation so that they are always kept
up to date. The second is that we add back a global checkpoint
background sync that fires on a timer. This timer fires every thirty
seconds, and is not configurable (for simplicity). This background sync
is smarter than what we had previously in the sense that it only sends a
sync if the global checkpoint on at least one replica is lagging that of
the primary. When the timer fires, we can compare the global checkpoint
on the primary to its knowledge of the global checkpoint on the replicas
and only send a sync if there is a shard behind.

Relates #26591
2017-09-21 15:34:13 -04:00
Jason Tedor 52e80a9292 Reenable BWC tests after disabling for backport
This commit reenables the BWC tests after they were disabled for
backporting the change to track global checkpoints of shard copies on
the primary.

Relates #26666
2017-09-18 06:28:50 -04:00
Jason Tedor c238b79cf4 Add global checkpoint tracking on the primary
This commit adds local tracking of the global checkpoints on all shard
copies when a global checkpoint tracker is operating in primary
mode. With this, we relay the global checkpoint on a shard copy back to
the primary shard during replication operations. This serves as another
step towards adding a background sync of the global checkpoint to the
shard copies.

Relates #26666
2017-09-18 06:04:44 -04:00
Boaz Leskes a99803f89a enable bwc testing 2017-09-15 00:20:19 +03:00
Boaz Leskes 1ca0b5e9e4 Introduce a History UUID as a requirement for ops based recovery (#26577)
The new ops based recovery, introduce as part of  #10708, is based on the assumption that all operations below the global checkpoint known to the replica do not need to be synced with the primary. This is based on the guarantee that all ops below it are available on primary and they are equal. Under normal operations this guarantee holds. Sadly, it can be violated when a primary is restored from an old snapshot. At the point the restore primary can miss operations below the replica's global checkpoint, or even worse may have total different operations at the same spot. This PR introduces the notion of a history uuid to be able to capture the difference with the restored primary (in a follow up PR).

The History UUID is generated by a primary when it is first created and is synced to the replicas which are recovered via a file based recovery. The PR adds a requirement to ops based recovery to make sure that the history uuid of the source and the target are equal. Under normal operations, all shard copies will stay with that history uuid for the rest of the index lifetime and thus this is a noop. However, it gives us a place to guarantee we fall back to file base syncing in special events like a restore from snapshot (to be done as a follow up) and when someone calls the truncate translog command which can go wrong when combined with primary recovery (this is done in this PR).

We considered in the past to use the translog uuid for this function (i.e., sync it across copies) and thus avoid adding an extra identifier. This idea was rejected as it removes the ability to verify that a specific translog really belongs to a specific lucene index. We also feel that having a history uuid will serve us well in the future.
2017-09-14 21:25:02 +03:00
Ryan Ernst 8ba4ff3be0 Build: Move javadoc linking to root build.gradle (#26529)
Javadoc linking between projects currently relies on
projectSubstitutions. However, that is an extension variable that is not
part of BuildPlugin. This commit moves the javadoc linking into the root
build.gradle, alongside where projectSubstitutions are defined.
2017-09-11 15:43:34 -07:00
Jason Tedor 53848877c8 Reenable BWC tests after backport
This commit reenables the BWC tests after they were disabled for
backporting a change that had broad BWC implications.
2017-08-31 22:13:50 -04:00
Tim Vernum eb87df9ff9 Allow abort of bulk items before processing (#26434)
Adds support for bulk items to be aborted before they are processed by the TransportShardBulkAction.
This can be used by an ActionFilter to reject a subset of the items in a bulk action without rejecting the whole action (or all the items for a shard).
2017-08-31 21:23:14 +10:00
Ryan Ernst 35a2ee38e1 Build: Add git hashes used as build metadata (#26397)
This commit adds files to the build output called build_metadata which
contain key/value pairs of metadata associated with the build. The first
use of this metadata are the git hashes associated with bwc checkouts.
These metadata files will be picked up by CI intake jobs and stored
along with last-good-commit, and then passed back in throug the
BUILD_METADATA env var on periodic jobs.
2017-08-28 14:10:06 -07:00
Michael Basnight cfd14cd2b8 Revert shading for the low level rest client (#26367)
At current, we do not feel there is enough of a reason to shade the low
level rest client. It caused problems with commons logging and IDE's
during the brief time it was used. We did not know exactly how many
users will need this, and decided that leaving shading out until we
gather more information is best. Users can still shade the jar
themselves. For information and feeback, see issue #26366.

Closes #26328

This reverts commit 3a20922046.
This reverts commit 2c271f0f22.
This reverts commit 9d10dbea39.
This reverts commit e816ef89a2.
2017-08-25 14:13:12 -05:00
Ryan Ernst 661648b3aa Build: Allow build to configure which license/notice to embed in jars (#26373)
We currently add the apache license/notice for elasticsearch to any
plugin that uses our ES plugin gradle plugin. However, each plugin
should be able to use their own license. This commit adds a licenseFile
and noticeFile property to the root of project's using BuildPlugin,
which is added to jar files for that project.
2017-08-24 22:46:30 -07:00
Nik Everett 99ac7beb8e Teach the build about betas and rcs (#26066)
The build was ignoring suffixes like "beta1" and "rc1" on the version numbers which was causing the backwards compatibility packaging tests to fail because they expected to be upgrading from 6.0.0 even though they were actually upgrading from 6.0.0-beta1. This adds the suffixes to the information that the build scrapes from Version.java. It then uses those suffixes when it resolves artifacts build from the bwc branch and for testing.

Closes #26017
2017-08-10 14:30:00 -04:00
Ryan Ernst 072281d5aa Update version to 7.0.0-alpha1 (#25876)
This commit updates the version for master to 7.0.0-alpha1. It also adds
the 6.1 version constant, and fixes many tests, as well as marking some
as awaits fix.

Closes #25893
Closes #25870
2017-08-01 15:47:48 -04:00
Michael Basnight e816ef89a2 Shade external dependencies in the rest client jar
This commit removes all external dependencies from the rest client jar
and shades them in an 'org.elasticsearch.client' package within the jar
using shadowJar gradle plugin. All projects that depended on the
existing jar have been converted to using the 'org.elasticsearch.client'
package prefixes to interact with the rest client.

Closes #25208
2017-07-24 12:55:43 -05:00
Yannick Welsch 49279d26da Reenable BWC tests
after merging #25822 & #25824
2017-07-21 16:36:50 +02:00
Yannick Welsch a2624dfcef Move primary term from ReplicationRequest to ConcreteShardRequest (#25822)
Removes the primary term from the replication request and pushes it into the transport envelope. This makes it possible to remove the term from the ReplicationOperation universe. The primary term that is to be used for a replication operation is now determined in the reroute phase when the node decides to execute a primary action (and validated once the primary action gets to execute). This makes it possible to validate that the primary action was sent to the correct primary shard instance that it was meant to be sent to (currently we only validate primary actions using the allocation id, which can be reused for failed and reallocated primaries).
2017-07-21 15:57:42 +02:00
Luca Cavanna ec66d655b5 Rename client artifacts (#25693)
It was brought up that our current client artifacts have generic names like 'rest' that may cause conflicts with other artifacts.

This commit renames:

- rest -> elasticsearch-rest-client
- sniffer -> elasticsearch-rest-client-sniffer
- rest-high-level -> elasticsearch-rest-high-level-client

A couple of small changes are also preparing the high level client for its first release.

Closes #20248
2017-07-13 09:44:25 +02:00
Boaz Leskes 215bffb08b Enable bwc testing
#25512 & #25511 have been merged
2017-07-08 11:57:22 +02:00
Boaz Leskes b40b002c1d Disable bwc tests in preparation of merging #25512 & #25511 2017-07-08 09:25:43 +02:00
Nik Everett 21b1db2965 Remove assemble from build task when assemble removed
Removes the `assemble` task from the `build` task when we have
removed `assemble` from the project. We removed `assemble` from
projects that aren't published so our releases will be faster. But
That broke CI because CI builds with `gradle precommit build` and,
it turns out, that `build` includes `check` and `assemble`. With
this change CI will only run `check` for projects without an
`assemble`.
2017-06-16 17:19:14 -04:00
Nik Everett 7b358190d6 Remove assemble task when not used for publishing (#25228)
Removes the `assemble` task from projects that are not published.
This should speed up `gradle assemble` by skipping projects that
don't need to be built. Which is useful because `gradle assemble`
is how we cut releases.
2017-06-16 11:46:34 -04:00
Ryan Ernst 106e373412 Build: Add master flag for disabling bwc tests (#25230)
This commit adds a gradle project, set inside the root build.gradle,
which controls all our bwc tests. This allows for seamless (ie no errant
CI failures) backporting of behavior.
2017-06-14 22:01:49 -07:00
Ryan Ernst 160a049930 Build: Move verifyVersions to new branchConsistency task (#25009)
This commit adds a new `branchConsistency` task which will run in CI
once a day, instead of on every commit. This allows `verifyVersions` to
not break immediately once a new version is released in maven.
2017-06-01 10:29:51 -07:00
Nik Everett 6a167a7ba8 Build: improve verifyVersions error message (#25006)
The error message was confusing because it doesn't
include unreleased versions like CURRENT.
2017-06-01 11:55:23 -04:00
Nik Everett bc4df47a76 Rework bwc snapshot projects to build up to two bwc versions (#24870)
Removes the `distribution:bwc` project in favor of
`distribution:bwc-release-snapshot` and
`distribution:bwc-stable-snapshot`.
`distribution:bwc-release-snapshot` builds a snapshot of the
latest release branch (5.4 now) if needed for backwards
compatibility. `distribution:bwc-stable-snapshot` builds a
snapshot of the latest stable branch (5.x now) if needed for
backwards compatibility.
2017-05-29 10:22:32 -04:00
Nik Everett 5da8ce8318 Remove the need for _UNRELEASED suffix in versions (#24798)
Removes the need for the `_UNRELEASED` suffix on versions by detecting if a version should be unreleased or not based on the versions around it. This should make it simpler to automate the task of adding a new version label.
2017-05-26 18:36:32 -04:00
Jason Tedor 95ebdbaddb Add BWC packaging distributions
Some packaging tests depend on snapshot versions of packaging
distributions yet the build does not use a repository that includes such
distributions. While we could add such a repository, a better strategy
is to follow our approach for other BWC tests where we depend on a
locally-compiled archive distribution. This commit adds a local
compilation of packaging artifacts and substitutes these anywhere that
we would otherwise depend on a snapshot of these artifacts.

Relates #24861
2017-05-24 11:55:32 -04:00
Luca Cavanna 747fa721e4 Build: add client jar for aggs-matrix-stats (#24827)
This will be useful for the high level client to add support for the matrix stats aggregation, as we will ship with this jar by default like we do for parent-join-client which is aligned with distributing core with the modules already included.

Relates to #24796
2017-05-23 13:33:54 +02:00
Nik Everett 82d2c7a142 Remove vagrant testing versions (#24754)
Now that we generate the versions list from Versions.java we can
drop the list of versions maintained for vagrant testing. One nice
thing that the vagrant testing did was to check if the list of
versions was out of date. This moves that test to the core
project.
2017-05-18 09:33:13 -04:00
Ryan Ernst 0353bd1fb6 Test: Convert rolling upgrade test to have task per wire compat version (#24758)
This commit changes the rolling upgrade test to create a set of rest
test tasks per wire compat version. The most recent wire compat version
is always tested with the `integTest` task, and all versions can be
tested with `bwcTest`.
2017-05-18 01:14:24 -07:00
Ryan Ernst ff34434bba Build: Extract all ES versions into gradle properties (#24748)
This commit expands the logic for version extraction from Version.java
to include a list of all versions for backcompat purposes. The tests
using bwcVersion are converted to use this list, but those tests
(rolling upgrade and backwards-5.0) are still not randomized; that will
happen in another followup.
2017-05-17 12:58:37 -07:00
Jim Ferenczi 279a18a527 Add parent-join module (#24638)
* Add parent-join module

This change adds a new module named `parent-join`.
The goal of this module is to provide a replacement for the `_parent` field but as a first step this change only moves the `has_child`, `has_parent` queries and the `children` aggregation to this module.
These queries and aggregations are no longer in core but they are deployed by default as a module.

Relates #20257
2017-05-12 15:58:06 +02:00
Ryan Ernst b3466818ac Remove outdated comment 2017-03-29 14:37:32 -07:00
Ryan Ernst cc1addeac2 Build: Find bwc version during build (#23801)
We currently have the last minor version of the previous major hardcoded
in tests like rolling upgrade. This change programatically finds this
during gradle initialization by parsing versions from Version.java.
2017-03-29 12:11:38 -07:00
Ryan Ernst 8c53555b28 Tests: Use local clone build of 5.x with bwc tests (#22946)
The current rest backcompat tests, which run against a mixed cluster of
5.x and 6.0 nodes, depend on snapshot builds of 5.x. However, this has
the potential for inconsistency that results in CI failures, and happens
quite often, whenever some backcompat logic is added to 5.x, but the bwc
test on master fails because the 5.x code has not yet been published as
a snapshot.

This change creates a git clone of the 5.x branch,
builds the zip distribution, and ties that into gradle substitutions for
the 5.x version.
2017-03-23 22:32:13 -07:00
Ryan Ernst 621643a5c3 Build: Only add ASL license to pom for elasticsearch project (#22664)
The extra plugins that may be attached to the elasticsearch build
contain their own license. In the past, the ASL license elasticsearch
uses was avoided by specially checking for the gradle project prefix of
`:x-plugins`. However, since refactoring to the elasticsearch-extra dir
structure, this mechanism was broken. This change fixes the pom license
adding to only be applied to projects that fall under the root project
(ie elasticsearch).
2017-01-17 13:38:32 -08:00
Tim Brooks 864e3128e7 Add task to clean idea build directory. Make cleanIdea task invoke it. 2016-12-22 11:24:13 -06:00
Ryan Ernst a30f649013 Build: Apply license section in poms only to elasticsearch artifacts (#21757) 2016-11-24 00:03:43 -08:00
Tanguy Leroux b9bee8bca3 Remove transport-netty3-client mention (#21650)
Now netty3 is gone, this mention must also be removed.
2016-11-18 11:54:22 +01:00
Ryan Ernst 85b8f29415 Build: Remove old maven deploy support (#20403)
* Build: Remove old maven deploy support

This change removes the old maven deploy that we have in parallel to
maven-publish, and makes maven-publish fully work with publishing to
maven local. Using `gradle publishToMavenLocal` should be used to
publish to .m2.

Note that there is an unfortunate hack that means for
zip artifacts we must first create/publish a dummy pom file, and then
follow that with the real pom file. It would be nice to have the pom
file contains packaging=zip, but maven central then requires sources and
javadocs. But our zips are really just attached artifacts, so we already
set the packaging type to pom for our zip files. This change just works
around a limitation of the underlying maven publishing library which
silently skips attached artifacts when the packaging type is set to pom.

relates #20164
closes #20375

* Remove unnecessary extra spacing
2016-09-19 15:10:41 -07:00
Yannick Welsch 95ae58b531 Set up project substitution for client:transport
This ensures that projects depending on client:transport (such as client:benchmark) properly resolve this dependency.
2016-07-26 17:58:39 +02:00
Jason Tedor 2d1b0587dd Introduce Netty 4
This commit adds transport-netty4, a transport and HTTP implementation
based on Netty 4.

Relates #19526
2016-07-22 22:26:35 -04:00
Ryan Ernst 2fb3cdceff Build: Simplify plugin configuration for rest tests
This change removes the multiple ways that plugins can be added to the
integ test cluster. It also removes the use of the default
configuration, and instead adds a zip configuration to all plugins. This
will enable using project substitutions with plugins, which must be done
with the default configuration.
2016-07-15 14:34:21 -07:00
Nik Everett 0bf447c697 Group client projects under :client
:client ---------> :client:rest
:client-sniffer -> :client:sniffer
:client-test ----> :client:test

This lines the client up with how we do things like modules and
plugins.
2016-06-22 14:26:41 -04:00
javanna c2839c1577 [TEST] Add client-test module and make client tests use randomized runner directly
The lucene-test dependency caused issues with IDEs as they would always load the lucene 5 jar although they shouldn't have, which caused jarhell in es core tests.

If we depend directly on randomized runner we don't have this problem. It is luckily still compatible with java 1.7. This requires though adding a thin module that includes the base test class which can be shared between client and client-sniffer.
2016-06-22 19:37:08 +02:00
javanna ace3a7b146 Merge branch 'master' into feature/http_client 2016-06-15 11:44:46 +02:00
Nik Everett d0e4485d42 Move NamingConventionsCheck into buildSrc
This will let things that don't depend on :test:framework like the
client use it.

Also skip initializing the classes we check because we don't care
about their initialization behavior because we're not executing them.
This makes the naming conventions check pretty close to instant
from a "human eye" perspective.
2016-06-14 18:30:34 -04:00
javanna 044a97c740 move client sniffer to its own project
Create a new subproject called client-sniffer that contains the o.e.client.sniff package. Since it is going to go to a separate jar, due to its additional functionalities and dependency on jackson, it makes sense to have it as a separate project that depends on client. This way we make sure that client doesn't depend on it etc.
2016-06-03 16:01:07 +02:00
javanna 3890842fd4 add project substitution for org.elasticsearch:client so that we can add it as a dep in other projects 2016-06-03 16:01:07 +02:00
Ryan Ernst 9bfa613901 Build: Rework eclipse settings copy so it does not get automatically cleaned
Gradle has "rules" for certain task names, and clean is one of these.
When you run clean, it searches for any tasks named cleanX, and tries to
reverse engineer the X task. For eclipse, this means it finds
cleanEclipse, and internally runs it (but this does not show up as a
dependency of clean in tasks list!!). Since we added .settings as an
additional file to delete with cleanEclipse, this gets deleted when
running just "clean". It doesn't delete the other files because those
have their own clean methods, and dependencies are not followed in this
insanity. This change simply makes a separate task for cleaning eclipse
settings.
2016-05-31 14:20:21 -07:00
javanna 1a298c2e6f Revert "Revert "Update the minimum required version of gradle to 2.13""
This reverts commit 0b965167fd.
All the CI machines have been updated to gradle 2.13, we should be ok upgrading the required minimum version.
2016-05-27 14:40:38 +02:00
javanna 0b965167fd Revert "Update the minimum required version of gradle to 2.13"
This reverts commit 8c6cbf0305. We need to update all our CI machines first, then get this change back in.
2016-05-27 13:16:43 +02:00
javanna 8c6cbf0305 Update the minimum required version of gradle to 2.13
It came out with improvements around idea integration and language levels. This will make it possible to have the upcoming java client as a new project compiled against java 7 and have idea working on the right language level.
2016-05-27 12:26:06 +02:00
Ryan Ernst 6dcfcf0bf1 Build: Remove buildSrc as a project in IDEs
We have have a virtual build-tools project which pulls in all the same
IDE settings we have for other projects, so there is no longer a need
for hacking buildSrc into our IDEs.
2016-05-25 12:08:57 -07:00
Ryan Ernst ceff71fa98 Build: Use -release with java 9 instead of -source/-target
This uses the new -release flag for javac which does not require setting
the bootclasspath as -source/-target did.
2016-05-23 12:51:01 -07:00
Ryan Ernst 7cf758943e Fix git info lookup to use correct dir where git exists 2016-05-21 22:58:55 -07:00
Ryan Ernst 5aea24137c Build: More pom generation improvements
This adds a non empty description to all generated poms, as well as
fixing distributions so they actually have a generated pom.
2016-05-13 16:32:35 -07:00
Ryan Ernst 81a521fcb5 Build: Add developer info to generated pom files 2016-05-13 13:57:08 -07:00
Ryan Ernst e16af604bf Build: Add pom generation to assemble task
In preparation for a unified release process, we need to be able to
generate the pom files independently of trying to actually publish. This
change adds back the maven-publish plugin just for that purpose. The
nexus plugin still exists for now, so that we do not break snapshots,
but that can be removed at a later time once snapshots are happenign
through the unified tools. Note I also changed the dir jars are written
into so that all our artifacts are under build/distributions.
2016-05-05 17:57:44 -07:00
Gabriel Moskovicz 48d26d8886 Fix indentation 2016-04-28 11:08:21 -03:00
Gabriel Moskovicz efabaa7c19 Allow import of all projects in Eclipse on Windows 2016-04-28 11:02:11 -03:00
Nik Everett c966d14f8b [build] Don't resolve the revision hash unless you need it
This fixes the build for folks that build without git installed locally
and should speed up the general case because we aren't trying to resolve
git information when it isn't really needed.
2016-04-15 11:35:05 -04:00
Alexander Reelsen 95579ca95a Build: Allow for file based deploy, sign packages
This allows for a local file based deploy without needed nexus
auth information.

Also signing of packages has been added, either via gradle.properties
or using system properties as a fallback.

The property build.repository allows to configure another endpoint if no
snapshot build is done.

Fix creation of .asc file for tar.gz distribution

Closes #17405
2016-04-15 17:14:05 +02:00
Ryan Ernst ba2abbf37c Build: Exclude gradle and eclipse build dirs from intellij
Intellij has a model of "everything is a source dir unless you say
otherwise". This change fixes the intellij configuration to not think
the gradle or eclipse build dirs are source dirs.
2016-04-14 14:54:52 -07:00
Ryan Ernst f71f0d6010 Revert "Build: Switch to maven-publish plugin"
This reverts commit a90a2b34fc.
2016-03-18 17:22:25 -07:00
Ryan Ernst cf8c9ffabb Always set version, even for x-plugins 2016-03-17 12:21:31 -07:00
Ryan Ernst a90a2b34fc Build: Switch to maven-publish plugin
The build currently uses the old maven support in gradle. This commit
switches to use the newer maven-publish plugin. This will allow future
changes, for example, easily publishing to artifactory.

An additional part of this change makes publishing of build-tools part
of the normal publishing, instead of requiring a separate upload step
from within buildSrc. That also sets us up for a follow up to enable
precomit checks on the buildSrc code itself.
2016-03-15 19:16:37 -07:00
Yannick Welsch b17f4b40ba Check that number of placeholders in log message matches number of parameters 2016-03-11 10:30:59 +01:00
Nik Everett 15a9da4d84 Remove accidental println 2016-02-11 17:21:21 -05:00
Nik Everett 903d4d48ab Merge pull request #16300 from nik9000/eclipse_project_names
Name eclipse projects to gradle path
2016-02-11 12:53:10 -05:00
Nik Everett bb61cdb330 Name eclipse projects to gradle path
This groups like projects together which is nice. It creates two weirdly
named projects:

1. buildSrc - its still just called buildSrc and it doesn't match. I don't
see why we import it into Eclipse anyway. Its groovy and easier to just edit
in vim or whatever.

2. elasticsearch - this is the name of the root project. It's also not
particularly useful to import into eclipse but we've always named it this way
and the name ':' was even more confusing so we just kept the name.
2016-02-10 13:30:18 -05:00
Ryan Ernst b9342b305b Also use separte buildSrc build dir for intellij 2016-02-04 15:58:05 -08:00
Ryan Ernst 2fef5dee96 Build: use separate build directories for eclipse and intellij
This fixes the problem where doing a gradle clean from the command line
completely breaks the IDE.
2016-02-04 15:44:48 -08:00
Ryan Ernst 9f1dfdbaea Build: Add AntTask to simplify controlling logging when running ant from gradle
This new task allows setting code, similar to a doLast or doFirst,
except it is specifically geared at running ant (and thus called doAnt).
It adjusts the ant logging while running the ant so that the log
level/behavior can be tweaked, and automatically buffers based on gradle
logging level, and dumps the ant output upon failure.
2015-12-18 11:45:52 -08:00
Robert Muir 7e53076112 Merge pull request #15545 from rmuir/rat
add gradle licenseHeaders to precommit
2015-12-18 13:27:37 -05:00