Commit Graph

51 Commits

Author SHA1 Message Date
javanna d9d1e6a67a Scripting: add support for fine-grained settings
Allow to on/off scripting based on their source (where they get loaded from), the  operation that executes them and their language.

The settings cover the following combinations:

- mode: on, off, sandbox
- source: indexed, dynamic, file
- engine: groovy, expressions, mustache, etc
- operation: update, search, aggs, mapping

The following settings are supported for every engine:

script.engine.groovy.indexed.update:    sandbox/on/off
script.engine.groovy.indexed.search:    sandbox/on/off
script.engine.groovy.indexed.aggs:      sandbox/on/off
script.engine.groovy.indexed.mapping:   sandbox/on/off
script.engine.groovy.dynamic.update:    sandbox/on/off
script.engine.groovy.dynamic.search:    sandbox/on/off
script.engine.groovy.dynamic.aggs:      sandbox/on/off
script.engine.groovy.dynamic.mapping:   sandbox/on/off
script.engine.groovy.file.update:       sandbox/on/off
script.engine.groovy.file.search:       sandbox/on/off
script.engine.groovy.file.aggs:         sandbox/on/off
script.engine.groovy.file.mapping:      sandbox/on/off

For ease of use, the following more generic settings are supported too:

script.indexed: sandbox/on/off
script.dynamic: sandbox/on/off
script.file:    sandbox/on/off

script.update:  sandbox/on/off
script.search:  sandbox/on/off
script.aggs:    sandbox/on/off
script.mapping: sandbox/on/off

These will be used to calculate the more specific settings, using the stricter setting of each combination. Operation based settings have precedence over conflicting source based ones.

Note that the `mustache` engine is affected by generic settings applied to any language, while native scripts aren't as they are static by definition.

Also, the previous `script.disable_dynamic` setting can now be deprecated.

Closes 
Closes 
Closes 
2015-03-26 19:56:55 +01:00
Martijn van Groningen 6a2a4bf28d change elasticsearch.org into elastic.co 2015-03-23 15:13:28 +01:00
Martijn van Groningen 5b3c0143c8 Disable marvel as it may fail the tests, because it creates indices. 2015-03-23 15:13:28 +01:00
Martijn van Groningen 911f522a0e change url to use elastic organization 2015-03-23 13:13:26 +01:00
javanna 65b9a94baa Benchmark api: removed leftovers
Closes 
Closes 
2015-03-21 10:36:05 +01:00
Britta Weber fc5dcf189c [release script] Check for //NORELEASE in code before release
Lines in the code that should be removed before a release can be annotated with
//NORELEASE . This can be useful when debugging test failures. For example,
one might want to add additional logging that would be too verbose for production
and therfore should be removed before releasing.

closes 
2015-03-18 09:59:28 -07:00
Alexander Reelsen fe2b30230f Build: Check that all packages contain the same jar files
This should ensure, that RPM and DEB packages do not lack dependencies,
that have been included in the tar.gz and zip files
2015-02-19 10:58:15 +01:00
Simon Willnauer 0698af93c2 [BUILD] Add marvel and license to the release script 2015-02-16 09:46:12 +01:00
Ryan Ernst 65e41f2ec3 release tool: Improve output for open github issues 2015-02-04 14:34:33 -08:00
Veres Lajos 4059e4ac86 typo fixes - https://github.com/vlajos/misspell_fixer
Closes 
2014-11-08 18:55:57 +01:00
Lee Hinman 2371ab7528 Update Sonatype release link in build_release.py 2014-11-06 10:40:07 +01:00
Michael McCandless 34e513c1a0 release tool: must decode bytes to string 2014-10-02 08:58:21 -04:00
Michael McCandless a4e09d9a8a release tool: fix 'log exists' check to not create the log before it checks ;) 2014-10-02 07:35:38 -04:00
Michael McCandless 868778c5b6 Core: improve build_release.py
Don't insist on log file removal until after usage is printed.

Some simple Python code improvements (x.find(y) != -1 --> y in x)

Make sure the git area is "clean" (has no unpushed changes, has pulled
all changes, has no untracked files)

Add label color detail when creating next github version label.

Closes 
2014-10-02 06:31:45 -04:00
mikemccand 997b94b427 fix typo: sonartype -> Sonatype 2014-09-29 04:12:08 -04:00
Michael McCandless 5e9e2cf50c Core: try again to upgrade to Lucene 4.10.1-snapshot 2014-09-24 13:48:49 -04:00
Michael McCandless ab3be76644 Revert Lucene upgrade 2014-09-24 13:25:55 -04:00
Michael McCandless 15c75b1967 Core: upgrade to Lucene 4.10.1 snapshot
Lucene will soon release official 4.10.1, but by upgrading sooner we can 1) sidestep the false failures due to the 1.8.0_20 JVM hotspot bug (has caused a number of false failures in recent Jenkins tests), 2) make sure none of the Lucene changes in 4.10.1 are problematic.

Closes 
2014-09-24 13:13:07 -04:00
Simon Willnauer 0a1701d416 [BUILD] skip bwc version check if directory doesn't exists or is not a directory 2014-07-23 14:54:28 +02:00
Simon Willnauer b923f9556f [BUILD] Add BWC tests to release script
This commit adds the ability to run bwc tests during the release
process to ensure the current release is backwards compatible with
the latest installed previous version. Installed means available
in the configured bwc test path.

Closes 
2014-07-22 11:11:44 +02:00
Simon Willnauer 5575ba1a12 [BUILD] Check for tabs and nocommits in the code on validate
This commit adds checks for nocommit and tabs in the source code.
The task is executed during the validate phase and can be disabled via
`-Dvalidate.skip`
2014-06-12 11:11:23 +02:00
Simon Willnauer 93552fe8f4 Revert "[BUILD] Calculate the full path ahead of time"
Revert "[BUILD] Promote artifacts from strings to their own type"

This reverts commit dcd4ba0654eb6780235718092969c2f9e6b38775.
This reverts commit 00d7eb3c0a6eefdb5947d07b18cf071ba538d696.
2014-06-03 16:27:10 +02:00
Simon Willnauer a4de19efdd [BUILD] Calculate the full path ahead of time 2014-06-03 16:11:31 +02:00
Andrew Raines fc9c6e33c3 [BUILD] Promote artifacts from strings to their own type 2014-06-02 13:23:08 -05:00
Simon Willnauer 5b13ef55f5 [BUILD] Check for nocommit in source files before building release 2014-06-02 19:29:42 +02:00
Simon Willnauer 60ba9d1e69 [BUILD] pass java exe to plugin install command 2014-05-22 11:44:27 +02:00
javanna 4331021361 enable dynamic scripts in release node, otherwise REST tests will fail
Relates to 
2014-05-10 11:16:25 +02:00
javanna 6678da8c28 [TEST] randomly added node.bench=true to client node in test cluster and re-enabled REST benchmark tests based on number of bench nodes available
In our REST tests we already have support for features and skip sections that allow to skip tests if a feature is not supported.
We can then add a skip section based on the benchmark feature to the benchmark tests and execute them only when they are supported, knowing that they need at least a node with node.bench settings within the cluster. We can check that this requirement is met by calling the nodes info api.

This way we can dynamically decide whether to execute those tests or not and we don't need to have a node.bench around all the time. In fact, given that the REST tests use the GLOBAL cluster, we want to be able to randomize settings as much as possible and run tests against default settings as well. Also, this mechanism can be easily supported by the external cluster implementation that is used during the release process.

Introduced ability to disable benchmark nodes which is needed by BenchmarkNegativeTest.
2014-05-09 23:36:00 +02:00
Simon Willnauer eaf04fc828 [BUILD] Fail release if test have AwaitsFix annotations
Note this commit upgrades Forbidden APIs to version 1.5

Closes 
2014-04-17 16:45:46 +02:00
Simon Willnauer 27d4d76769 [BUILD] remove leftover print statement 2014-04-14 22:23:10 +02:00
Simon Willnauer 11bf13c363 Check for no open issues before build release 2014-04-14 18:54:04 +02:00
javanna 1ec4f8f04b [TEST] Replaced RestTestSuiteRunner with parametrized test that uses RandomizedRunner directly
ElasticsearchRestTests extends now ElasticsearchIntegrationTest and makes use of our ordinary test infrastructure, in particular all randomized aspects now come for free instead of having to maintain a separate (custom) tests runner

We previously parsed only the tests that needed to be run given the version of the cluster the tests are running against. This doesn't happen anymore as it didn't buy much and it would be harder to support as the tests get now parsed before the test cluster gets started. Thus all the tests are now parsed regardless of their skip sections, afterwards the ones that don't need to be run will be skipped through assume directives.

Fixed REST tests that rely on a specific number of shards as this change introduces also random number of shards and replicas (through randomIndexTemplate)

Closes 
2014-04-07 17:08:05 +02:00
Simon Willnauer 13109d3547 [Build] splitted release commits into version and feature flag commits
Elasticsearch is release from release-branches but the modifications
to the documentation must be cherry-picked into the current development
branch. To make this easier this commit splits the commits of the
Version and the documenation into seperate commits.
2014-03-25 09:16:34 +01:00
Simon Willnauer 8ceb98752d Move master to Java 1.7
Closes 
2014-02-27 15:12:02 +01:00
David Pilato a5001440bf Fix typo 2014-02-17 18:46:51 +01:00
Alexander Reelsen bcfba8bc46 Added note to release script to announce on ML 2014-01-16 09:43:56 +01:00
Simon Willnauer 99eae50f9c run mvn clean before smoketesting 2014-01-14 17:00:50 +01:00
Simon Willnauer 6829b67ff0 Fix Release tool to run smoke tests off tags
this commit allows to run the release tool for smoke
testing without being on the actually released branch.
This commit also added a list of plugins that will be installed
for smoke testing to see if the plugin startup mechanism works
correctly.
2014-01-14 16:31:25 +01:00
Simon Willnauer ed7daada2d Open files with encoding='utf-8' in release script 2014-01-10 10:14:08 +01:00
Simon Willnauer 10ec2e948a Fix ASL Header in source files to reflect s/ElasticSearch/Elasticsearch
This commit also removes the license to Shay Banon in favor of soley
Elasticsearch. Thanks Shay for this awesome product you took it far!

Closes 
2014-01-07 11:22:01 +01:00
Simon Willnauer 8d9af7e7a5 Start elasticsearch as deamon when running smoketests
In previous version daemonized was default now we need to set it explicitly.
2014-01-07 10:52:06 +01:00
Simon Willnauer af10b65fe1 Add smoketest support for published packages
the build_release.py tool now also downloads and verfyfies the
released packages from S3. It checks integrity based on the sha1
checksums and runs the smoketest against the specs in the current branch.
2013-12-23 10:34:48 +01:00
Simon Willnauer 7872ef401c Run REST test-suite against release packages during release process 2013-12-18 12:14:08 +01:00
Simon Willnauer 65b49bfbf6 Allow setting release log file via env 2013-12-02 16:16:38 +01:00
Simon Willnauer 098cbef6d0 use dedicated release log in order to build lucene & es releases in parallel 2013-11-13 12:41:23 +01:00
Simon Willnauer 71f5b16137 Make RPM building mandatory
If RPM tools are not installed the release tool now fails with an
appropriate message. The tool now also fails if any of the required
artifacts is not present.
2013-11-05 23:55:13 +01:00
Simon Willnauer 97c8fe6801 Add optional RPM building, signing and uploading to release script 2013-11-05 16:11:36 +01:00
Simon Willnauer f2840c5ccf Added doc file processing to update coming/added references
We use 'coming[x.y.z]' in our ref docs which needs to be updated
when doing a release to 'added[x.y.z]' This commit adds support
for replacing the references where applicable during preparing the release.
2013-11-04 21:23:22 +01:00
Simon Willnauer 555ececaa9 Improve top level documentation for building releases 2013-11-04 16:10:58 +01:00
Simon Willnauer 1f3c7be851 Fix typo in mvn command for running with multiple JVMs 2013-11-04 14:30:31 +01:00