Commit Graph

5756 Commits

Author SHA1 Message Date
Simon Willnauer a834cc0e0f Shutdown indices service last
We do wait for shards to be closed in IndicesService for 30 second.
Yet, if somebody holds on to a store reference ie. an open scroll request
the 30 seconds time-out and node shutdown takes very long. We should
release all other resources first before we shutdown IndicesService.

Closes #8940
2014-12-15 22:43:37 +01:00
Ryan Ernst 37287284e6 Settings: Remove `mapping.date.round_ceil` setting for date math parsing
The setting `mapping.date.round_ceil` (and the undocumented setting
`index.mapping.date.parse_upper_inclusive`) affect how date ranges using
`lte` are parsed.  In #8556 the semantics of date rounding were
solidified, eliminating the need to have different parsing functions
whether the date is inclusive or exclusive.

This change removes these legacy settings and improves the tests
for the date math parser (now at 100% coverage!). It also removes the
unnecessary function `DateMathParser.parseTimeZone` for which
the existing `DateTimeZone.forID` handles all use cases.

Any user previously using these settings can refer to the changed
semantics and change their query accordingly. This is a breaking change
because even dates without datemath previously used the different
parsing functions depending on context.

closes #8598
closes #8889
2014-12-15 13:13:45 -08:00
Lee Hinman 8fbf45ef2b [TEST] Make parent breaker check less strict
In cases of heavy contention, it's possible for more than 2 threads
to race to a circuit breaking exception.

Essentially this means that if we have 3 threads all trying to add 3 and
simultaneously cause a circuit breaking exception (due to retry), when
adjusting after circuit breaking we can "rewind" past what this test
expects the child breaker to be at.

This adds leeway into the check, where it's okay to be within
NUM_THREADS from the parentLimit, because each thread should only add 1
to the breaker at a time.
2014-12-15 17:06:21 +01:00
Simon Willnauer 3bba45289e Remove unused code 2014-12-15 16:26:48 +01:00
Simon Willnauer 1247774ff1 Remove Gateway abstraction
We only have a single gatweway since es 1.3. There is no need to keep all
these abstractsion and nested packages. We can fold most of it into simpler
structures.
2014-12-15 15:53:02 +01:00
Lee Hinman a8fa650ee6 [CORE] Remove IndexEngine
IndexEngine was an abstraction where we had index-level engines (instead
of shard-level) that could store meta information about the index. It
was never actually used by Elasticsearch, and only there for plugins.

This removes it, because it is a confusing abstraction and not needed,
no plugins should be implementing their own IndexEngines.
2014-12-15 14:30:44 +01:00
Boaz Leskes d62bf5f67f Discovery: concurrent node failures can cause unneeded cluster state publishing
When a node fails (or closes), the master processes the network disconnect event and removes the node from the cluster state. If multiple nodes fail (or shut down) in rapid succession, we process the events and remove the nodes one by one. During this process, the intermediate cluster states may cause the node fault detection to signal the failure of nodes that are not yet removed from the cluster state. While this is fine, it currently causes unneeded reroutes and cluster state publishing, which can be cumbersome in big clusters.

Closes #8804
Closes #8933
2014-12-15 14:01:25 +01:00
Lee Hinman 9b18c44b67 Default _cat APIs to verbose
`?v=false` can be used if the headers are not desired.

Resolves #8922

Fixes #8927
2014-12-15 12:51:59 +00:00
Simon Willnauer e47b753617 [SEARCH] close active contexts on SearchService#close()
When we close a node all pending / active search requests need to be
cleared otherwise a node will wait up to 30 sec for shutdown sicne there
could be open scroll requests. This behavior was introduces in 1.5 such that
versions <= 1.4.x are not affected.

Closes #8940
2014-12-15 09:41:31 +01:00
Boaz Leskes a63a055f63 Test: missing {} from log command in indexRandom 2014-12-13 17:24:46 +01:00
Boaz Leskes 22da975e34 Test: reduce join timeout in testFullRollingRestart
Occasionally a the join thread successfully connected to a just closed node and which causes the subsequent join request to time out. It's default timeout 60s throws the test off when it waits for a cluster to form.
2014-12-13 13:05:04 +01:00
Michael McCandless ae11c4654b Core: use compound file by default for merged segments < 10% of index size
Change Elasticsearch to use Lucene's defaults, to reduce file descriptor count.

Closes #8934

Closes #8919
2014-12-12 15:51:37 -05:00
Britta Weber 60e805cc1f [TEST] use ensureYellow() 2014-12-12 18:07:14 +01:00
Britta Weber 185521be4b [TEST] wait for yellow before searching 2014-12-12 17:34:07 +01:00
Britta Weber 2dc9392a34 [TEST] get trace logs for search packages 2014-12-12 17:06:16 +01:00
Lee Hinman 6bf18056b0 [CORE] Remove explicit .cleanUp() on cache clear
Calling cache.cleanUp() is kind of like calling System.gc(), meaning
that we should never have (non-test) things that rely on this
functionality.

For the field data and filter cache, we already have a periodic process
that runs this .cleanUp(), so there is no need to block index
closing/clearing on it. Instead, we can clean the field data cache in
InternalTestCluster before we check the circuit breaker.

This can help tests that time out because cleaning the cache is taking
too long
2014-12-12 13:24:45 +01:00
Simon Willnauer 42d9a57d0c [TEST] Wait for yellow before verifying - sometimes the shard is not even started 2014-12-12 12:40:34 +01:00
Simon Willnauer b857a1f635 Remove circular dependency between IndicesService and IndicesStore 2014-12-12 12:25:25 +01:00
Simon Willnauer 498331d16f [TEST] Remove random templates for ConcurrentDynamicTemplateTests 2014-12-12 12:18:58 +01:00
Michael McCandless c9d63d3482 Core: don't block shard stats when phase 3 of recovery is running
Today, shard stats are blocked while phase 3 of recovery (replay xlog)
is running; this change removes the engine readLock from shard stats
so it's not blocked.

Closes #8910
2014-12-12 05:18:25 -05:00
Simon Willnauer 6dacf61dfc [TEST] Add test to ensure master is not prone to #8917 2014-12-12 10:59:11 +01:00
Simon Willnauer 7b82660ffc [TEST] Remove debug leftover 2014-12-12 09:29:33 +01:00
Simon Willnauer dac520170f [TEST] Close the node env after test is done 2014-12-11 21:24:15 +01:00
Simon Willnauer 3877dc618d Remove some Internal* abstractions
We have lots of boilerplate code that is unnecessarily abstracting
services ie InternalIndexShard and IndexShard or InternalIndexService and
IndexService. It's enough to have concrete classes for these core classes.

Closes #8904
2014-12-11 17:31:01 +01:00
Simon Willnauer 59534391da [GATEWAY] Cleanup LocalGatewayShardsState
This commit tries to cleanup LocalGatewayShardsState to be more efficient
and easier to understand.
2014-12-11 17:17:50 +01:00
Alexander Reelsen 544ef8cb17 Packaging: Add java7/8 java-package paths to debian init script
If you use the java-package tool to create java packages, those
paths also should be added to the debian init script.

Also updated the docs, that it is ok to install java8.

Closes #7383
2014-12-11 16:15:00 +01:00
Peter Fabian Mitchell b2bab05c29 HTTP: Add 'http.publish_port' setting to the HTTP module
This change adds a 'http.publish_port' setting to the HTTP module to configure
the port which HTTP clients should use when communicating with the node. This
is useful when running on a bridged network interface or when running behind
a proxy or firewall.

Closes #8807
Closes #8137
2014-12-11 16:10:07 +01:00
Simon Willnauer 123fb2f5db [SCRIPT] ScriptService can deadlock entire nodes if script index is recovering
we currently have operationThreaded set to false when indexing a script. This setting
means that if we are executing the operation locally that we don't spawn a new thread for
it althought incoming thread in this case is the network thread. Now sicne we are indexing here
the engine is currently recovering which sometimes locks the engine for finalization blocks on
a network call waiting for the recovery target to come back the internal lock in engine will never be
released since we are waiting with our network thread for it to be released.
2014-12-11 16:04:25 +01:00
Simon Willnauer ba881a9b58 [ENGINE] Remove engine related command classes
Todaqy we pass structs to the engine to call optimize / refresh and flush.
This commit cleans up this logic to reduce complexity in the engine.
2014-12-11 15:47:24 +01:00
Jun Ohtani 80bd69811d Mappings: Fix Get field mapping api with pretty flag
Closes #6552
2014-12-11 22:56:54 +09:00
Michael McCandless 084d25cdbd Test: create private store for these test cases 2014-12-11 05:25:44 -05:00
Robert Muir a2ffe494ae [core] add best_compression option for Lucene 5.0
Upgrades lucene to latest, and supports the BEST_COMPRESSION parameter
now supported (with backwards compatibility, etc) in Lucene.
This option uses deflate, tuned for highly compressible data.

index.codec::
The default value compresses stored data with LZ4 compression, but
this can be set to best_compression for a higher compression ratio,
at the expense of slower stored fields performance.

IMO its safest to implement as a named codec here, because ES already
has logic to handle this correctly, and because its unrealistic to have
a plethora of options to Lucene's default codec... we are practically
limited in Lucene to what we can support with back compat, so I don't
think we should overengineer this and add additional unnecessary plumbing.

See also:
https://issues.apache.org/jira/browse/LUCENE-5914
https://issues.apache.org/jira/browse/LUCENE-6089
https://issues.apache.org/jira/browse/LUCENE-6090
https://issues.apache.org/jira/browse/LUCENE-6100

Closes #8863
2014-12-10 22:13:09 -05:00
Nicholas Knize aa644e3ad7 [GEO] Fix for NPE enclosed in SearchParseException for a "geo_shape" filter or query
This fix adds better error handling for parsing multipoint, linestring, and polygon GeoJSONs.  Current logic throws a NPE when parsing a multipoint, linestring, or polygon that does not comply with the GeoJSON specification. That is, if a user provides a single coordinate instead of an array of coordinates, or array of linestrings, the ShapeParser throws a NPE wrapped in a SearchParseException instead of a more useful error message.

Closes #8432
2014-12-10 16:42:36 -06:00
Simon Willnauer f308049a90 [ENGINE] Fix updates dynamic settings in InternalEngineHolder
After the refactoring in #8784 some settings didn't get passed to the
actual engine and there exists a race if the settings are updated while
the engine is started such that the actual starting engine doesn't see
the latest settings. This commit fixes the concurrency issue as well as
adds tests to ensure the settings are reflected.
2014-12-10 23:07:13 +01:00
Simon Willnauer 788d7cb451 [TEST] Reset test logger to default level 2014-12-10 22:47:03 +01:00
Britta Weber 57b77c6907 [TEST] wait for yellow to avoid searching while relocating
After upgrading shard might start relocating again. If there are no
replicas the cluster state of a node might not be up to data for
a few miliseconds and direct a search request to a node that does not
have the shard anymore. This result in the following test failures:

1> java.lang.AssertionError: Count is 99 but 101 was expected.  Total shards: 13 Successful shards: 12 & 0 shard failures:
1>     __randomizedtesting.SeedInfo.seed([1932F73B458703CA:6F4FAD3DAC55591C]:0)
1>     [...org.junit.*]
1>     org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount(ElasticsearchAssertions.java:184)
1>     org.elasticsearch.bwcompat.BasicBackwardsCompatibilityTest.testIndexRollingUpgrade(BasicBackwardsCompatibilityTest.java:358)

Waiting for relocation finished should fix this.
2014-12-10 17:46:32 +01:00
Simon Willnauer 905dc90eec [CORE] Drop support for state written pre 0.90
Today we have several upgrade methods that can read state written
pre 0.90 or even pre 0.19. Version 2.0 should not support these state
formats. Users of these version should upgrade to a 1.x or 0.90.x version
first.

Closes #8850
2014-12-10 16:59:07 +01:00
Simon Willnauer e5a7eaff22 [TEST] Use private store for a test private engine
The test was using the same store as the suite level engine which caused
problems with write locks in some cases.

Closes #8880
2014-12-10 16:56:52 +01:00
Michael McCandless 624a4b87bf Core: put []'s around the values in the slow-merge log message 2014-12-10 09:22:28 -05:00
Simon Willnauer 3b585e5a5e Add more trace logging to gateway translog recovery 2014-12-10 11:10:32 +01:00
Simon Willnauer da11bf4c1d Use Lucene version string in main rest action
This commit removes the deprecated constant for the main
version and uses the real lucene version we are running instead.
Behind the scenes the same value was used and is now obsolet.
2014-12-10 10:40:58 +01:00
Michael McCandless de75517357 Core: also log byte size and doc count when we log slow merges
Closes #8855

Closes #8853
2014-12-09 18:16:07 -05:00
Simon Willnauer 127255f62e [TEST] Restore test logging level after test is done 2014-12-09 23:31:24 +01:00
javanna 796ebcb88b [TEST] LoggingListener to restore the initial logger levels after any modification
Modifications to LoggingListener pushed with #8820 caused the original logger levels not to be reset after modifications, as the new state was saved for restore instead of the previous one.

 Added unit tests for LoggingListener as well.

Closes #8845
2014-12-09 14:26:13 +01:00
Simon Willnauer 8ffe8e0259 [TEST] compare strings and strings - leftover from Path API refactoring 2014-12-09 14:22:15 +01:00
Robert Muir 39186edc86 Ban java.io.File in tests.
Restrict use of java.io.File to 5 methods (excluded), but otherwise ban.
This is a prerequisite to do any mocking here.

I don't try to do any heavy cleanup on these tests, I am not familiar with them.
So this is mostly a rote straightforward conversion.

Closes #8836
2014-12-09 05:57:48 -05:00
Simon Willnauer c0d50f2a80 [TEST] Beef up InternalEngineTest and remove bogus timeouts 2014-12-09 09:31:33 +01:00
Ryan Ernst 0c8f5ac129 Tests: Additional test for memory stats api. 2014-12-08 15:43:31 -08:00
Ryan Ernst fde32cc599 Stats: Add more fine grained memory stats from Lucene segment reader.
This is a start to exposing memory stats improvements from Lucene 5.0.
This adds the following categories of Lucene index pieces to index stats:
* Terms
* Stored fields
* Term Vectors
* Norms
* Doc values
2014-12-08 15:29:43 -08:00
Robert Muir 150c2203ac Add test that ES filterreader getCoreCacheKey() behaves correctly.
Closes #8831.
2014-12-08 17:58:27 -05:00
Robert Muir 380e47dbf2 fix cache key methods on leaf reader 2014-12-08 15:28:53 -05:00
Michael McCandless b0b96af746 Test: fix this test to work in IntelliJ 2014-12-08 15:11:06 -05:00
Simon Willnauer b28fc1afa5 [ENGINE] Add engine lifecycle store reference to EngineHolder
This commit add the engines reference to the store out of the actual
implementation into the hodler since the holder manages the actual lifcycle.
Engine internal references like per searcher or per recovery are kept inside
the actual implemenation since the have a different lifecycle.
2014-12-08 21:07:12 +01:00
tlrx 31a77185a6 Merge branch 'fix/plugins-loading' 2014-12-08 17:15:06 +01:00
tlrx 97ec8f94ae Plugins: Plugin failed to load since #8666
The method Path.endsWith(String s) doesn't work exactly the same way as String.endsWith() (see http://docs.oracle.com/javase/7/docs/api/java/nio/file/Path.html#endsWith(java.nio.file.Path)). This blocks the loading of plugins.
2014-12-08 17:13:39 +01:00
Simon Willnauer 84066128ed [TEST] Pass class level test logging to external nodes
This commit passes the test logging annotation from the class
level to the external nodes as well.

Closes #8552
2014-12-08 13:25:03 +01:00
Boaz Leskes 83bb65a020 Internal: allow InternalEngine to be stopped and started
Once the current engine is started you can only close it once. Once closed the engine cannot be started again. This commit adds a stop method which signals the engine to free it's resources but in a way that allows restarting.

This is done by introducing InternalEngineHolder which is a wrapper around InternalEngine. This allows to add the stop() method without adding complexity the engine implementation. InternalEngineHolder also serves an entry point for listeners (incoming and outgoing) to other ES components, which removes the needs add/remove them if the engine is stopped.

Closes #8784
2014-12-08 12:40:38 +01:00
Lee Hinman 83fa7bfaba [TEST] Add unit tests for DiskThresholdDecider settings 2014-12-08 12:14:09 +01:00
Simon Willnauer 8d7ce3c558 [STORE] Expose ShardId via LeafReader rather than Direcotry API
Today we try to fetch a shard Id for a given IndexReader / LeafReader
by walking it's tree until the lucene internal SegmentReader and then
casting the directory into a StoreDirecotory. This class is fully internal
to Elasticsearch and should not be exposed outside of the Store.

This commit makes StoreDirectory a private inner class and adds dedicated
ElasticsearchDirectoryReader / ElasticserachLeafReader exposing a ShardId
getter to obtain information about the shard the index / segment belogs to.

These classes can be used to expose other segment specific information in
the future more easily.
2014-12-08 12:10:28 +01:00
Simon Willnauer a43259eba4 Add more trace loggin to LocalIndexShardGateway.java 2014-12-08 12:07:16 +01:00
tlrx a046ee756d Scripting: Add explicit error message when script_score script returns NaN
When a scoring script returns not a number, the current message is confusing (IllegalArgumentException[docID must be >= 0 and < maxDoc=3 (got docID=2147483647)]). This commit adds the error message ScriptException[script score function returns a wrong score: NaN].

Closes #2426
2014-12-08 10:14:01 +01:00
Alexander Reelsen 7b64a5c2c8 Packaging: Check if proc file exists before calling sysctl
The packaged init scripts could return an error, if the file
/proc/sys/vm/max_map_count was not existing and we still called
sysctl.

This is primarly to prevent confusing error messages when elasticsearch
is started under virtualized environments without a proc file system.

Closes #4978
2014-12-08 09:56:14 +01:00
Boaz Leskes 2bc48a4806 Tests: move RecoverAfterNodesTests to org.elasticsearch.gateway.local and increase BLOCK_WAIT_TIMEOUT to 10s
The tests were still in org.elasticsearch.gateway.none but the none gateway was removed.
2014-12-07 22:10:11 +01:00
Simon Willnauer 3cdf266d4d [ALLOCATION] Speed-up disk-threshold decider
Instead of iterating all shards of all indices to get all relocating
shards for a given node we can just use the RoutingNode#shardsWithState
method and fetch all INITIALIZING / RELOCATING shards and check if they
are relocating. This operation is much faster and uses pre-build
data-structures.

Relates to #6372
2014-12-06 22:03:37 +01:00
Patrick Koenig 00275ac1d6 Clarify InternalIndexShard callbacks
This commit changes internal callback to be clear
about when they are called and also provide the
exception that was potentially thrown as a callback argument.

Closes #5945
2014-12-05 23:55:32 +01:00
Igor Motov 0b024ad2f3 Snapshot/Restore: switch to write once mode for snapshot metadata files
This commit removes creation of in-progress snapshot file and makes creation of the final snapshot file atomic.

Fixes #8696
2014-12-05 12:39:24 -05:00
Simon Willnauer 0bab17ffde [TEST] wait unitl all machines joined the cluster 2014-12-05 18:12:00 +01:00
Lee Hinman d32f1a8ad0 [TESTS] Log what the _default_ template is in ElasticsearchIntegrationTest 2014-12-05 14:45:24 +01:00
Lee Hinman caa5af4bf6 [TESTS] Use a _default_ template to load field data lazily
Previously it was possible for the field data clearing in this test to
take too long, causing the test to time out.

This also switches to using `scaledRandomIntBetween` for the number of
fields.
2014-12-05 14:45:17 +01:00
Thilo Fromm 3d03c39ae6 Packaging: Make ES user own plugins dir, remove on uninstall
This change will chown /usr/share/elasticsearch/plugins to the elasticsearch
user (the directory was formerly owned by root). This enables the ES user to
manage plugins.

Also, /usr/share/elasticsearch/plugins is now removed when the elasticsearch
package is un-installed. Previously it was left lying there.

Closes #8732

Signed-off-by: Thilo Fromm <github@thilo-fromm.de>
2014-12-05 14:25:15 +01:00
Thilo Fromm 98d18c4bd9 deb: add systemd service config for upcoming Jessie
This change adds a systemd service configuration file, and adds systemd logic
to installation and de-installation scripts. The upcoming Debian 8 "Jessie"
release will use systemd.

fixes #8943

Signed-off-by: Thilo Fromm <github@thilo-fromm.de>
2014-12-05 12:25:48 +01:00
Louis Sautier 6c2abcc754 Debian package: Prevent init script from returning when the service isn't actually started.
Close #6909
2014-12-05 10:26:31 +01:00
Boaz Leskes 72155311f0 Gateway: GatewayService should register cluster state listener before checking for current state
At the moment we may miss a state change and fail to recover on time.

Closes #8789
2014-12-05 10:22:20 +01:00
John Michael Luy 27724bef6a Typo in error message with too few points for geo polygon filter.
Closes #8767
2014-12-04 20:20:21 +01:00
Simon Willnauer b8687163c4 [TEST] produce valid symlinks in tests 2014-12-04 17:02:38 +01:00
Martijn van Groningen 7ac713aedc Core: surgically removed slow scroll, because master (2.0) requires full cluster restart coming from previous versions. 2014-12-04 15:56:03 +01:00
Simon Willnauer 8b5bc2643e [Store] Only fail recovery if files are inconsistent
the recovery diff can return file in the `different` category
since it's conservative if it can't tell if the files are the same.
Yet, the cleanup code only needs to ensure both ends of the recovery
are consistent. If we have a very old segments_N file no checksum is present
but for the delete files they might be such that the segments file passes
the consistency check but the .del file doesn't sicne it's in-fact the same
but this check was missing in the last commit.
2014-12-04 15:48:40 +01:00
javanna ad004072bb Internal: remove optional original indices
Original indices are optional in ShardDeleteByQueryRequest only for backwards compatibility, see #7406. We can remove this in master since 2.0 will require a full cluster restart.

Closes #8777
2014-12-04 14:25:46 +01:00
javanna 95c2d844a9 Internal: remove some more bwc code
Some more code that was left for backwards compatibility can now be removed on master since 2.0 will require a full cluster restart.

Closes #8778
2014-12-04 14:07:58 +01:00
javanna 5f329bd607 Internal: remove bwc leftover
This caused some test failures as the readFrom method was not in sync anymore with the writeTo
2014-12-04 13:38:20 +01:00
Simon Willnauer 219bb88bc2 Remove runtime version checks
This cleanup commmit removes a large protion of the versioned reads / writes
in the network protocol since master requires a full cluster restart.
2014-12-04 11:31:29 +01:00
Simon Willnauer f4052fd936 Factor out PID file creation and add tests
This commit factors out the PID file creation from bootstrap and adds
tests for error conditions etc. We also can't rely on DELETE_ON_CLOSE
since it might not even write the file depending on the OS and JVM implementation.
This impl uses a shutdown hook to best-effort remove the pid file if it was written.

Closes #8771
2014-12-04 11:12:16 +01:00
Simon Willnauer ab0e3a6db2 [CLIENT] Add internal liveness action
This commit adds a very lightweight action to the transport
serivce that allows to fetch clustername and the discovery node
from a node. This is used by transport clients to test liveness of
a node without using the nodesinfo API which can be blocking if management
threadpools are busy.

Closes #8763
2014-12-04 10:49:20 +01:00
javanna 171e718f88 [DOCS] Document ActionNamesTests 2014-12-03 16:28:36 +01:00
javanna 6ccb46ef37 [TEST] remove action names bwc layer
The bwc layer added with #7105 is not needed in master as a full cluster restart will be required, thus from 2.0 on the only supported action names are compliant to the defined conventions and don't need to be converted to the old format

Closes #8758
2014-12-03 16:18:43 +01:00
David Pilato d2a2d1bb53 java: QueryBuilders cleanup: remove deprecated
Related to #8667:

Some QueryBuilders have been deprecated in 1.x branches. We removed them in 2.0.

Removed
-------

* `textPhrase(...)`
* `textPhrasePrefix(...)`
* `textPhrasePrefixQuery(...)`
* `filtered(...)`
* `inQuery(...)`
* `commonTerms(...)`
* `queryString(...)`
* `simpleQueryString(...)`

Closes #8721.
2014-12-03 16:07:34 +01:00
Simon Willnauer d732077900 [TEST] Wait for yellow before running rescorer tests 2014-12-03 10:03:45 +01:00
Simon Willnauer 921e028e99 [PLUGIN] Don't use atomic move when installing plugin
the extract location might be on a different filesystem where
atomic move won't work. Yet this operation is not critical in terms
of visibility so there is no need to do this.
2014-12-03 09:48:49 +01:00
Simon Willnauer a436da7cdb [PLUGIN] don't set posix permission on filesystem that don't support it 2014-12-03 09:47:54 +01:00
javanna 36e12d39fd [TEST] guarantee REST tests execution order
REST tests are being shuffled before their execution. To guarantee their repeatability given the seed, their order needs to be always the same before the shuffling happens.

Closes #8745
2014-12-03 08:36:15 +01:00
Simon Willnauer 3dfff84043 Revert back APIs that resolve files from classpath to java.net.URL
The conversion to the Path API doesn't work if the path points
to a file inside a JAR like a config. These path must be read
while the ZIP filesystem is opened which can't be guaranteed across
the board. This commit reverts back the relevant changes to java.net.URL
and adds a util method to read UTF-8 Encoded files from URLs correctly.
2014-12-03 00:09:35 +01:00
Simon Willnauer a6510f9245 Add File.java to forbidden APIs
This commit cuts over all of core (not quite all tests) to java.nio.Path
It also adds the file class to the core forbidden APIs to prevent its usage.

This commit also resolves #8254 since we now consistently useing the NIO Path
API. The Changes in this commit allow for more information if IO operations fail
since the NIO API throws exceptions instead of boolean return values. The build-in
methods used in this commit are also more resillient to encodeing errors like
unmappable characters and throw exceptions if those chars are present in a file.

Closes #8254
Closes #8666
2014-12-02 21:29:26 +01:00
Simon Willnauer d8f16178d3 Override write(byte[] b, int off, int len) in FilterOutputStream for better performance
Closes #8748
2014-12-02 21:20:53 +01:00
Adrien Grand 943dbb8ab6 Fix resource description of VerifyingIndexOutput. 2014-12-02 19:26:17 +01:00
Simon Willnauer 48ec6599c2 [TEST] use private randomness in InternalTestCluster 2014-12-02 18:22:44 +01:00
Simon Willnauer e3ed471d30 [TEST] speed up tests by reducing the recovery retry by default 2014-12-02 17:43:06 +01:00
Adrien Grand eba2bd51c1 Core: Upgrade to lucene-5.0.0-snapshot-1642891.
The only required change was to add a description to our index outputs.

Close #8743
2014-12-02 17:38:53 +01:00
Martijn van Groningen a8a5391255 Removed 1.x version check 2014-12-02 14:56:12 +01:00
Martijn van Groningen 4c159f1140 Fixed forgotten delegation 2014-12-02 14:50:57 +01:00
Simon Willnauer 8736543c71 [RECOVERY] Ensure shards are identical after recovery
Today we don't check if the recovery target has all the
files that we expect there after the recovery. This commit
adds aditional safety to ensure all files are present with the
correct checksums on recovery finalization.

Closes #8723
2014-12-02 14:05:44 +01:00