This commit makes a number of improvements when importing the
Elasticsearch project into IntelliJ IDEA. Specifically:
- Contributing documentation has been updated to reflect that the
'idea' task should no long be used and Gradle project import is
instead the officially supported way of setting up the project.
- Attempts to run the 'idea' task will result in a failure with a
message directing folks to our CONTRIBUTING.md document.
- The project JDK is explicit set rather that using whatever JAVA_HOME
is.
- Gradle build operation delegation is disabled, and test execution is
configured to 'choose per test'.
- Gradle is configured to inherit the project JDK.
- Some code style conventions are automatically configured.
- File encoding is explicitly set to UTF-8.
- Parallel module compilation is enabled and deprecated feature
warnings are disabled.
- A remote debug run configuration using listen mode is created.
- JUnit runner is configured with required system properties.
- License headers are configured such that Apache 2 is the default
notice added to all source files with exception of source in /x-pack
which will use the Elastic license.
When building IntelliJ generates several source files related to the
benchmarks. This commit adds that path to the gitignore so these don't
get accidentally committed.
* Move periodic job to ES repo
This change kickstarts the process of moving CI job definitions to this
repo.
* Added a minimal readme to provide pointers to the documentation
* Update .ci/README.md
Co-Authored-By: Rory Hunter <pugnascotia@users.noreply.github.com>
* Update .ci/README.md
Co-Authored-By: Rory Hunter <pugnascotia@users.noreply.github.com>
* point to main repo
* PR review
* Add link to JJBB
Adds the `.vscode` directory to `.gitignore`. This didn't come up when
we had x-pack in a different repository than Elasticsearch because I'd
open the directory that contained both of them in VSCode. Now that they
are merged I tend to just open the Elasticsearch repository which causes
VSCode to create this directory that we don't need to commit.
Also removes leftover maven files from `.gitignore`.
We have agreed to introduce the Gradle wrapper to simplify workflows for
developers, and managing infrastructure (e.g., CI, release builds, etc.)
as well as consideration for the fact that other projects in our stack
use Gradle and do not necessarily want to be tied to our Gradle version.
Relates #28065
Sequence BWC logic consists of two elements:
1) Wire level BWC using stream versions.
2) A changed to the global checkpoint maintenance semantics.
For the sequence number infra to work with a mixed version clusters, we have to consider situation where the primary is on an old node and replicas are on new ones (i.e., the replicas will receive operations without seq#) and also the reverse (i.e., the primary sends operations to a replica but the replica can't process the seq# and respond with local checkpoint). An new primary with an old replica is a rare because we do not allow a replica to recover from a new primary. However, it can occur if the old primary failed and a new replica was promoted or during primary relocation where the source primary is treated as a replica until the master starts the target.
1) Old Primary & New Replica - this case is easy as is taken care of by the wire level BWC. All incoming requests will have their seq# set to `UNASSIGNED_SEQ_NO`, which doesn't confuse the local checkpoint logic (keeping it at `NO_OPS_PERFORMED`)
2) New Primary & Old replica - this one is trickier as the global checkpoint service currently takes all in sync replicas into consideration for the global checkpoint calculation. In order to deal with old replicas, we change the semantics to say all *new node* in sync replicas. That means the replicas on old nodes don't count for the global checkpointing. In this state the seq# infra is not fully operational (you can't search on it, because copies may miss it) but it is maintained on shards that can support it. The old replicas will have to go through a file based recovery at some point and will get the seq# information at that point. There is still an edge case where a new primary fails and an old replica takes over. I'lll discuss this one with @ywelsch as I prefer to avoid it completely.
This PR also re-enables the BWC tests which were disabled. As such it had to fix any BWC issue that had crept in. Most notably an issue with the removal of the `timestamp` field in #21670.
The commit also includes a fix for the default value of the seq number field in replicated write requests (it was 0 but should be -2), that surface some other minor bugs which are fixed as well.
Last - I added some debugging tools like more sane node names and forcing replication request to implement a `toString`
Applied (almost) the same rules we use to validate index names
to new alias names. The only rule not applies it "must be lowercase".
We have tests that don't follow that rule and I except there are lots
of examples of camelCase alias names in the wild. We can add that
validation but I'm not sure it is worth it.
Closes#20748
Adds an alias that starts with `#` to the BWC index and validates
that you can read from it and remove it. Starting with `#` isn't
allowed after 5.1.0/6.0.0 so we don't create the alias or check it
after those versions.
This adds a generated-resources dir that the plugin properties are
generated into. This must be outside of the build dir, since intellij
has build as "excluded".
closes#14392
This creates a module in qa called vagrant that can be run if you have
vagrant and virtualbox installed and will run the packaging tests in trusty
and centos-7.0. You can ask it to run tests in other linuxes. This is the full
list:
* precise aka Ubuntu 12.04
* trusty aka Ubuntu 14.04
* vivid aka Ubuntun 15.04
* wheezy aka Debian 7, the current debian oldstable distribution
* jessie aka Debian 8, the current debina stable distribution
* centos-6
* centos-7
* fedora-22
* oel-7
There is lots of documentation on how to do this in the TESTING.asciidoc.
Closes#12611
Rather than specify paths for the .gitignored files that Eclipse uses for
project management just ignore all files and directories that look like
those directories. That way we can add new subprojects and we won't need
add more .gitignore entries.
This commit adds a shell script which:
* move current elasticsearch core source in `/core`
* fetch `elasticsearch-parent` project in `/`
* fetch plugins in `/plugins`
* change `groupId` for all plugins to `org.elasticsearch.plugin` so versions won't conflict in maven central
* remove plugins/name/dev-tools dir which is not needed anymore
* remove plugins/name/.git dir
* remove plugins/name/LICENSE and plugins/name/CONTRIBUTING files
* clean `core/pom.xml` of useless settings that are inherited from parent project.
* `core/pom.xml` is adapted to change location of rest tests definition (`../`)
* change core name to `Elasticsearch Core`
* remove `plugins` dir from `.gitignore`
Plugins added:
* Analysis
* analysis-kuromoji
* analysis-smartcn
* analysis-stempel
* analysis-phonetic
* analysis-icu
* Mapper
* mapper-attachments
* Language
* lang-python
* lang-mvel
* lang-javascript
* Cloud
* cloud-gce
* cloud-azure
* cloud-aws
River plugins are ignored but might be added if we want to.
Todo:
* check and adapt our release tool. It now has to upload all submodules as well.
* adapt Jenkins jobs
This commit add a basic infrastructure as well as primitive tests
to ensure version backwards compatibility between the current
development trunk and an arbitrary previous version. The compatibility
tests are simple unit tests derived from a base class that starts
and manages nodes from a provided elasticsearch release package.
Use the following commandline executes all backwards compatiblity tests
in isolation:
```
mvn test -Dtests.bwc=true -Dtests.bwc.version=1.2.1 -Dtests.class=org.elasticsearch.bwcompat.*
```
These tests run basic checks like rolling upgrades and
routing/searching/get etc. against the specified version. The version
must be present in the `./backwards` folder as
`./backwards/elasticsearch-x.y.z`
Since plugins should never be committed to the core codebase and it is useful to be able to add plugins to the development environment adding plugins folder to the .gitignore file will stop it from appearing in the unstaged changes
The fact that Maven and Eclipse share the same build directories can trigger
race conditions when both are trying to build at the same time, eg. if you run
`mvn clean test` while Eclipse is up and running: Eclipse will notice that some
class files are missing and start compiling in parallel with Maven.
For better integration with the Lucene Test Framework and the
availabilty of RandomizedRunner / Randommized Testing this commit
moves over from TestNG to JUnit.
This commit also moves relevant places over to RandomzedRunner for
reproduceability and removes copied classes from the Lucene Test
Framework.
This enforces that settings are taken into account whichever mean is used to
import the project into Eclipse (manual import, m2e, mvn eclipse:eclipse, ...).
Mark Vieira
Gordon Brown
Alpar Torok
Nik Everett
Jason Tedor
Boaz Leskes
Andrzej Wisłowski
Ryan Ernst
Clinton Gormley
Colin Goodheart-Smithe
Robert Muir
David Pilato
Janmejay Singh
Simon Willnauer
Colin Goodheart-Smithe
Andrew Raines
Adrien Grand
Shay Banon
Peter
Nicolas Huray
Paul_Loy