Commit Graph

647 Commits

Author SHA1 Message Date
Igor Motov 8fe5d903b7 Add support for bulk delete operation in snapshot repository
Currently when we delete files belonging to deleted snapshots we issue one delete command to underlying snapshot store at a time. Some repositories can benefit from bulk deletes of multiple files.

Closes #12533
2015-08-04 14:52:28 -06:00
Robert Muir 6f9a067197 Change master branch back to 2.0-beta1 2015-08-04 15:38:21 -04:00
Simon Willnauer 8f6e75b158 Ignore EngineClosedException on IndexShard#sync
This method syncs the translog unless it's already synced. If the engine
is alreayd closed we are guaranteed to be synced already such that we can just
ignore this exception.

Closes #12603
2015-08-04 15:43:17 +02:00
Simon Willnauer 2b2657279d Use UTC instead of default timezone for creation date in CAT endpoint
this change was added recently which uses default timezone for the creation
date on CAT endpoints. We should be consistent and use UTC across the board.
This commit adds #getDefaultTimzone() to forbidden API and fixes the REST tests.

Relates to #11688
2015-08-04 14:38:23 +02:00
Simon Willnauer 6753f7f03e Cut over master to 2.0.0-SNAPSHOT 2015-08-04 10:54:12 +02:00
Tanguy Leroux a74684473e Move Streams.copyTo(String|Bytes)FromClasspath() into StreamsUtils
The Streams.copyTo(String|Bytes)FromClasspath() methods resolve resources using org.elasticsearch.io.Streams classloader. This is fine in elasticsearch core and when running tests but if used in a plugin this can lead to FileNotFoundExceptions at runtime because plugin are loaded in a dedicated classloader.
2015-08-04 10:11:18 +02:00
Adrien Grand caca13c878 Merge pull request #12599 from jpountz/fix/PrioritizedEsThreadPoolExecutor_concurrency
Fix concurrency issue in PrioritizedEsThreadPoolExecutor.
2015-08-04 10:00:46 +02:00
Robert Muir 64850c0539 Improve site-example integ test to test served contents.
Closes #12578
2015-08-03 23:47:36 -04:00
Robert Muir 45ad47581d tone down these stress tests 2015-08-03 22:47:24 -04:00
Robert Muir d7cf5cd6af NamingConventionTests should test subclasses of ESIntegTestCase end with IT
These are integration tests.
2015-08-03 21:34:53 -04:00
Robert Muir 2fe279d2a9 cleanup more abstract test class -> TestCase and integ -> IT 2015-08-03 21:07:21 -04:00
Ryan Ernst 1e12d03252 Tests: Rename base tests cases to use "TestCase" suffix
Most of the abstract base test classes we have were previously @Ignored.
However, there were also some other tests ignored. Having two ways to
quiet tests is confusing, and clearly it has caused some tests
to get lost in the fold.

This change moves all base test classes to use the "TestCase" suffix,
which is not picked up by the test class name pattern. It also removes
@Ignore from (almost) all tests, and adds it to forbidden apis.
And since we were renaming, I shorted base test class names to use
"ES" instead of "Elasticsearch". I type this a lot of types a day,
and I have heard others express a similar desire for a shorter name.

closes #10659
2015-08-03 17:43:00 -07:00
Ryan Ernst be941f0669 Tests: Remove uses of @Slow and @Integration
Now that integ tests are moved into `mvn verify`, we don't really have
a need for @Slow, and especially not @Integration. This removes
uses of the first, and completely removes uses of the latter.
2015-08-03 16:26:07 -07:00
Robert Muir 062feac80a Cut over some remaining integration tests to IT
A few subclasses got missed here
2015-08-03 19:08:01 -04:00
Ryan Ernst d349deb9b7 Merge pull request #12613 from rjernst/refactor/integ-tests
Rename integ tests to IT suffix
2015-08-03 14:57:26 -07:00
Ryan Ernst 622d8a830e Tests: Rename integ tests to IT suffix
This rename effectively moves all integration tests to be run with `mvn
verify`. `mvn test` now runs in about 2 mins. This is a follow up to
2015-08-03 14:42:34 -07:00
Robert Muir 9769477c9d Merge pull request #12609 from rmuir/sane_permissions
improve sanity of securitymanager file permissions
2015-08-03 17:39:30 -04:00
Robert Muir 2623ce7960 Improve permissions checking in security tests and fix crazy pidfile parent permissions 2015-08-03 16:58:25 -04:00
Robert Muir be6f9a0196 improve sanity of securitymanager file permissions
conf/ and plugins/ do not need read-write access: this prevents a lot
of bad possibilities.
2015-08-03 14:43:40 -04:00
Nik Everett c82717789c Merge pull request #11688 from szroland/master
Add option to `_cat/indices` to return index creation date #11524
2015-08-03 14:42:18 -04:00
Robert Muir e472cbed09 Merge branch 'master' into perms
Conflicts:
	core/src/test/java/org/elasticsearch/plugins/PluginManagerTests.java
2015-08-03 13:20:34 -04:00
Robert Muir 0e027d82c9 Merge pull request #12602 from rmuir/core_integ_tests
Add integration test phase for elasticsearch core/
2015-08-03 10:15:37 -04:00
Robert Muir 379ffdb8ee Add integration test phase for elasticsearch core/ 2015-08-03 09:43:11 -04:00
Britta Weber 26d51c299e Merge pull request #12400 from brwe/plug-fetch-sub-phases
Make fetch sub phases pluggable
2015-08-03 15:19:28 +02:00
Adrien Grand 1eaa9f9fda Fix concurrency issue in PrioritizedEsThreadPoolExecutor.
Tasks can be registered with a timeout, which runs as a task in a separate
threadpool. The idea is that the timeout runner cancels the main task when
the time is out, and the timeout runner is cancelled when the main task
starts executing. However, the following statement:

```java
                    timeoutFuture = timer.schedule(new Runnable() {
                        @Override
                        public void run() {
                            if (remove(TieBreakingPrioritizedRunnable.this)) {
                                runAndClean(timeoutCallback);
                            }
                        }
                    }, timeValue.nanos(), TimeUnit.NANOSECONDS);
```

is not atomic: the removal task is first started, and then the (volatile)
variable is assigned. As a consequence, there is a short window that allows
a timeout task to wait until the time is out even if the task is already
completed.

See http://build-us-00.elastic.co/job/es_core_17_centos/496/ for an example of
such a failure.
2015-08-03 14:11:49 +02:00
Robert Muir c5b91b9058 Merge pull request #12589 from rmuir/shading_cleanup
Fix shaded jar packaging
2015-08-03 08:10:50 -04:00
Adrien Grand 365a324fc5 Tests: Add more logging to SearchPreferenceTests so that we can know which preference is failing. 2015-08-03 12:55:15 +02:00
Colin Goodheart-Smithe ade3881152 Aggregations: Full path validation for pipeline aggregations
Previously only the first aggregation in a buckets_path was check to make sure the aggregation existed. Now the whole path is checked to ensure an aggregation exists at each element in the buckets_path

Closes #12360
2015-08-03 11:15:40 +01:00
Robert Muir a5a27a5cd9 Add jar hell check test for the shaded jar 2015-08-03 05:54:55 -04:00
Robert Muir 20ed7c1724 Merge pull request #12586 from rmuir/exit_value
Fix Bootstrap to not call System.exit
2015-08-03 05:27:08 -04:00
Martijn van Groningen 50c345238b fix ShardUtils#getElasticsearchDirectoryReader(...) to use FilterDirectoryReader#getDelegate() 2015-08-03 09:54:56 +02:00
szroland 742607ec9b using default timezone explicitly 2015-08-02 20:53:03 +02:00
szroland 35d947b524 Add option to `_cat/indices` to return index creation date #11524
Returning index creation date, both as a numeric millisecond value and
as a string. This implements #11524
2015-08-02 20:53:02 +02:00
Robert Muir 6f33ea4f62 move shading to distribution/shaded 2015-08-01 11:37:43 -04:00
Robert Muir d173b89d11 make this less syntactically confusing 2015-07-31 14:13:00 -04:00
Robert Muir 5f9edd26ea Fix Bootstrap to not call System.exit
Its not going to work: its blocked by security policy
and will just add a confusing SecurityException to the mix, and
bogusly give an exit status of 0 when in fact something bad happened.

Finally, if ES can't startup, it is a serious problem, there is
no sense in hiding the reason why: deliver the full stack trace.
2015-07-31 14:00:18 -04:00
Robert Muir 8d5b5ad862 restore my lost code changes and add measures to detect/prevent this in the future 2015-07-31 13:00:24 -04:00
Igor Motov 354504334f Create a directory during repository verification
The repository verification process should create a subdirectory to make sure we check permission of newly created directories in case elasticsearch processes on different nodes are running using different uids and creating blobs with incompatible permissions.

Closes #11611
2015-07-31 12:11:05 -04:00
Robert Muir f804ec3cd9 cleanup 2015-07-31 10:05:11 -04:00
Robert Muir fd8e92a18a don't represent site plugins with 'null' anymore 2015-07-31 09:41:39 -04:00
Lee Hinman 5a8356c86e Reroute shards when a node goes under disk watermarks
Previously we issued a reroute when a node went over the high watermark
in order to move shards away from the node. This change tracks nodes
that have previously been over the high or low watermarks and issues a
reroute when the node goes back underneath the watermark.

This allows shards that may be unassigned to be assigned back to a node
that was previously over the low watermark but no longer is.

Resolves #12422
2015-07-30 16:32:54 -06:00
Robert Muir ba63d57e82 don't lose stacktrace if a plugin somehow makes jar hell 2015-07-30 15:56:33 -04:00
Britta Weber 8a4957967b more review comments 2015-07-30 17:47:21 +02:00
Alexander Reelsen a8310d0575 Tests: Fix CLI help checking tests under windows
As windows has different line endings and this has
already been fixed by another test, the method has been
moved into CliToolTestCase.

In addition one test has been removed, as it was redundant.
2015-07-30 16:12:14 +02:00
Alexander Reelsen 69d7f1a78a Startup: Remove getopt parsing in shell script, use java CLITool
In order to ensure, we have the same experience across operating systems
and shells, this commit uses the java CLI parser instead of the shell
getopt parsing to parse arguments.

This also allows for support for paths, which contain spaces.

Also commons-cli depdency was upgraded to 1.3.1 and tests have been added.

Changes

* new exit code, OK_AND_EXIT, allowing to tell the caller to exit, as everything
  went as expected (e.g. when running a version output)

BWC breaking:

* execute() returns an ExitStatus instead of an integer, otherwise there is no
  possibility to signal by a command, if the JVM should be exited after a run.
  This affects plugins, that have command line tools
* -v used to be version, but is a verbose flag by default in the current CLI infra,
  must be -V or --version now
* -X has been removed - the current implementation was useless anyway, as
  it prefixed those properties with "es.". You should use
  ES_JAVA_OPTS/JAVA_OPTS for JVM configuration
2015-07-30 13:20:29 +02:00
Martijn van Groningen b3b34668e8 made `dateMathExpressionResolver` final 2015-07-30 11:34:10 +02:00
Adrien Grand 23fa535bfa Upgrade HDRHistogram to version 2.1.6.
This release fixes https://github.com/HdrHistogram/HdrHistogram/pull/68 which
we have been hitting in our CI tests, for instance:
http://build-us-00.elastic.co/job/es_core_master_metal/10567/
2015-07-30 09:55:12 +02:00
Dave Parfitt 2d2f21e760 Merge pull request #12534 from metadave/feature/cat_nodeattrs
Add _cat/nodeattrs API
2015-07-29 17:01:22 -04:00
Dave Parfitt f209809716 Add _cat/nodeattrs API
This provides a _cat/nodeattrs API call, which presents
custom node attributes in a denormalized table.

Closes #8000
2015-07-29 16:11:18 -04:00
Martijn van Groningen 0c53204b40 Added RealtimeRequest interface and let GetRequest, MultiGetRequest, TermVectorsRequest and MultiTermVectorsRequest implement it.
This is useful because now all requests that are realtime are now grouped under the same marker interface.
2015-07-29 21:47:11 +02:00