Commit Graph

10303 Commits

Author SHA1 Message Date
javanna ecc56a57f5 [TEST] move LoggingConfigurationTests to common.logging.log4j
Make also LogConfigurator#ALLOWED_SUFFIXES package private so that it can be used in LoggingConfigurationTests, now that it's in the same package as the class that it tests.
Add few randomized aspects to LoggingConfigurationTests.
2014-11-19 11:34:35 +01:00
Mathias Fussenegger 2dcb1f503d Logging: restrict files loaded as logging configuration based on their suffix
Make sure that files such as logging.yml.rpmnew or logging.yml.bak are not loaded as logging configuration.

Only files that start with the "logging." prefix and end with ".yaml", ".yml", ".json" and ".properties" suffix get loaded.

Closes #7457
2014-11-19 11:34:03 +01:00
Igor Motov 314da4ec9e Test: don't enable unnecessary http transport in restoreIndexWithShardsMissingInLocalGateway test 2014-11-18 21:08:24 -05:00
Boaz Leskes 1e16375d04 Docs: Update execution hint docs for Significant terms agg
copied over the relevant pieces from the terms agg

Closes #8532
2014-11-18 20:54:26 +01:00
Simon Willnauer 7f27664ae0 [TEST] Revert accidential massive iterations 2014-11-18 17:26:31 +01:00
Simon Willnauer fdac110368 [TEST] add more debug output when engine / store are closed 2014-11-18 17:04:34 +01:00
Lee Hinman 7bd389de61 [TEST] Give tests for ctx._ttl more leeway
Fixes #8500
2014-11-18 17:01:36 +01:00
Simon Willnauer 734dc198ca [TEST] distributor direcotory is only used if multiple datapath are configured 2014-11-18 15:33:00 +01:00
Simon Willnauer 5c6fe2593e [CORE] Ban all useage of Future#cancel(true)
Interrupting a thread while blocking on an NIO Read / Write Operation
can cause a file to be closed due to the interrupts. This can have unpredictable
effects when files are open by index readers etc. we should prevent interruptions
across the board if possible.

Closes #8494
2014-11-18 14:14:09 +01:00
Simon Willnauer 119aa4af20 [STORE] Use DistributorDirectory only if there are more than one data direcotry
We don't need the overhead of DistributorDirectory if there is only a single directory
in the distributor.
2014-11-18 14:02:41 +01:00
Simon Willnauer c6c709eda2 [TEST] Register data.path for all nodes on close in InternalTestCluster
We need to register those data paths otherwise we might miss path that
need to get cleaned when using local gatway etc. which can otherwise
cause imports of dangeling indices.
2014-11-18 13:39:06 +01:00
Michael McCandless 2f40b464ad Test: force merge index in the end of IndexStatsTests.throttleStats
This works around slow IO (fsync) causing the test-framework cleanup
to timeout at 30 seconds when trying to delete the index.

Closes #8528
2014-11-18 07:25:16 -05:00
Igor Motov fd8a56dc0b Snapshot/Restore: keep the last legacy checksums file at the end of restore
This commit fixes the issue caused by restore process deleting all legacy checksum files at the end of restore process. Instead it keeps the latest version of the checksum intact. The issue manifests itself in losing checksum for all legacy files restored into post 1.3.0 cluster, which in turn causes unnecessary snapshotting of files that didn't change.

Fixes #8119
2014-11-17 19:50:53 -05:00
Igor Motov b0dde6ee4a Snapshot/Restore: restore of indices that are only partially available in the cluster
Fixes the issue with restoring of an index that had only some of its primary shards allocated before it was closed.

Fixes #8224
2014-11-17 18:47:58 -05:00
markharwood 6f79d67f81 Bulk indexing issue - missing parent routing causes NullPointerException.
Now each error is reported in bulk response rather than causing entire bulk to fail.
Added a Junit test but the use of TransportClient means the error is manifested differently to a REST based request - instead of a NullPointer the whole of the bulk request failed with a RoutingMissingException. Changed TransportBulkAction to catch this exception and treat it the same as the existing logic for a ElasticsearchParseException - the individual bulk request items are flagged and reported individually rather than failing the whole bulk request.

Closes #8365
2014-11-17 17:16:35 +00:00
Ken Wu d1c6d3b7b0 fix of Bulk update child doc, NPE error message when parent is not specified #8365
- Throw an RoutingMissingException instead of NPE
2014-11-17 17:13:57 +00:00
Andreas Kohn 66cf4ecc54 Docs: Fix typo in the javadoc for #tieBreaker()
Closes #8503
2014-11-17 16:57:15 +01:00
Adrien Grand e70b4d835b Search: Do not force the post-filter to be loaded into a BitSet.
Close #8488
2014-11-17 15:46:13 +01:00
Ryan Doyle a2c93304f8 Docs: Add Wireshark protocol dissection support
Hi guys, I've finished my Wireshark dissector for Elasticsearch. It is now merged into their master. See https://code.wireshark.org/review/#/c/4948/

Closes #8495
2014-11-17 13:07:01 +01:00
Martijn van Groningen 28f3ea1b8d Test: Let the random parent/child tests use the query and filter parsers instead of creating the queries and filters in a custom way.
By using the query and filter parsers we increase the test coverage and make the random parent/child tests simpler.
2014-11-17 10:05:46 +01:00
Boaz Leskes 461c20049f Test: CorruptedFileTest.testCorruptionOnNetworkLayer used node settings when creating an index
Test used `indices.recovery.concurrent_streams` when creating an index but this is a node setting. Moved it to the node settings and added similar settings to speed up concurrent recoveries.

Also fixed a misleading log message in ShardRecoveryHandler when logging a remove corruption
2014-11-16 23:54:36 +01:00
Simon Willnauer b0b7c917c3 Prevent double wrapping directories in MockDirectoryWrapper 2014-11-16 20:54:49 +01:00
Simon Willnauer e6908de04a [TEST] Close node env otherwise windows can't delete tmp files 2014-11-16 17:44:38 +01:00
Simon Willnauer 176b4eb1ed Fix synchronization - leftover from refactoring 2014-11-16 15:58:10 +01:00
Simon Willnauer 1c64a113de [CORE] Intorduce shards level locks to prevent concurrent shard modifications
Today it's possible that the data directory for a single shard is used by more than on
IndexShard->Store instances. While one shard is already closed but has a concurrent recovery
running and a new shard is creating it's engine files can conflict and data can potentially
be lost. We also remove shards data without checking if there are still users of the files
or if files are still open which can cause pending writes / flushes or the delete operation
to fail. If the latter is the case the index might be treated as a dangeling index and is brought
back to life at a later point in time.

This commit introduces a shard level lock that prevents modifications to the shard data
while it's still in use. Locks are created per shard and maintined in NodeEnvironment.java.
In contrast to most java concurrency primitives those locks are not reentrant.

This commit also adds infrastructure that checks if all shard locks are released after tests.
2014-11-16 14:24:29 +01:00
Boaz Leskes 37661aed60 Logging: BroadcastOperationAction - added trace logging for successful shard-level responses
In order to be able to trace the exact shards that participated in the operation.
2014-11-15 18:33:56 +01:00
Martijn van Groningen 983a108776 Percolator: Remove `index.percolator.allow_unmapped_fields` setting.
There should be no option to opt out from strict field resolution for percolator query parsing.

Closes #8439
2014-11-14 22:39:07 +01:00
Martijn van Groningen 5714b0a7ad Aggregations: Changed child filters to not require a random access based bitset in `nested` agg.
Also the nested agg now requires docs to be consumed / scored in order.

Closes #8454
2014-11-14 21:40:44 +01:00
Martijn van Groningen 284491d874 Core: In the bitset cache only eagerly load bitsets for parent nested object fields.
Don't eagerly cache parent type filters in bitset cache or nested object fields that are leafs.
Also let parent/child queries not rely on FixedBitSetFilter, but rather on regular Filter

Closes #8440
2014-11-14 21:00:14 +01:00
Nicholas Knize 0067a0cb7e Updating to throw IllegalArgument exception for null value coordinates. Tests included. 2014-11-14 10:28:30 -06:00
Nicholas Knize 49935659e4 Adding parse gates for valid GeoJSON coordinates. Includes unit tests. 2014-11-14 09:58:34 -06:00
Ken Wu 31fa4dc58b Configuration: Tab characters in YAML should throw an exception.
Throw an exception if there is a 'tab' character in the elasticsearch.yml file

Close #8259
2014-11-14 16:24:52 +01:00
Nicholas Knize 345c06e5e8 Correcting coordinate checks on LinearRing and LineString, updating test 2014-11-14 08:12:38 -06:00
nwarz e77f9720d2 Mapping: Fixes Merging of default analyzer
Fixed behaviour where two representations of the default index analyzer weren't being treated as equivalent. Added REST test to confirm fix.

Closes #2716
2014-11-14 09:58:26 +00:00
Olivier Favre 4d68d3d053 Provide more context variables in update scripts
In addition to `_source`, the following variables are available through
the `ctx` map: `_index`, `_type`, `_id`, `_version`, `_routing`,
`_parent`, `_timestamp`, `_ttl`.

Some of these fields are more useful still within the context of an
Update By Query, see #1607, #2230, #2231.
2014-11-14 10:14:39 +01:00
Alex Ksikes 936b4c63fc Term Vectors: Fix NPE with dfs and no tvs
Fixes a bug with dfs option for when term vectors are not stored and not generated.
2014-11-14 09:11:13 +01:00
Nicholas Knize c39ca479c7 [GEO] Fix for ArithmeticException[/ by zero] when parsing a "polygon" with one pair of coordinates
While this commit is primariy a fix for issue/8433 it adds more rigor to ShapeBuilder for parsing against the GeoJSON specification. Specifically, this adds LinearRing and LineString validity checks as defined in http://geojson.org/geojson-spec.html to ensure valid polygons are specified. The benefit of this fix is to provide a gate check at parse time to avoid any further processing if an invalid GeoJSON is provided.  More parse checks like these will be necessary going forward to ensure full compliance with the GeoJSON specification.

Closes #8433
2014-11-13 11:45:04 -06:00
Alexander Reelsen 9956e7721d Tests: Improve netty test behaviour
Based on some test failures, this commit fixes two minor things

* Bind ports only on so called ephemeral ports to prevent try to
  bind to ports where elasticsearch already runs on
* Remove @Network annotation as it was used in a wrong scope
2014-11-13 15:48:50 +01:00
Colin Goodheart-Smithe f56976c2b3 Removed bwc for GetIndexRequest serialisation 2014-11-13 14:20:42 +00:00
Vidar Kongsli 9e2f6bb5e5 Added quotes to allow spaces in installation path 2014-11-13 09:14:24 -05:00
Clinton Gormley 32fc657d71 Docs: Fixed a bad ref to docs-bulk-udp which no longer exists in master 2014-11-13 14:34:49 +01:00
Colin Goodheart-Smithe 353574d6af Indices API: Fix GET index API always running all features
Previous to this change all features (_alias,_mapping,_settings,_warmer) are run regardless of which features are actually requested. This change fixes the request object to resolve this bug
2014-11-13 13:22:46 +00:00
Clinton Gormley 6b05b229af Docs: Changed breaking docs in master to correspond with 1.x for easier merging 2014-11-13 13:50:57 +01:00
Colin Goodheart-Smithe 34b37ab7f0 [DOCS] Added documentation for log4j-extras dependency 2014-11-13 12:40:14 +00:00
Philip McMahon 4194a699c0 Logging: Add log4j-extras dependency
Close #7927
2014-11-13 12:39:30 +00:00
javanna c1428b5964 [DOCS] Expand logging documentation
Updated log4j link so it doesn't point to log4j 2.0 but version 1.2. Clarified which formats are supported and briefly explained what loggers and appenders are, plus added a link to the log4j docs.

Closes #5305
Closes #8455
2014-11-13 11:08:10 +01:00
Joel Taddei 7e72800c83 [DOCS] Corrected syntax error in search curl cmd
Closes #8447
2014-11-12 17:21:19 +01:00
Colin Goodheart-Smithe 972afe61a0 Mappers: Better validation of mapping JSON
Closes #7205
2014-11-12 14:32:25 +00:00
Mark Walkom bfd1bcd30a Updated threadpool documentation to elaborate/clarify what the pools are for and their values
Closes #8446
2014-11-12 22:33:38 +11:00
Martijn van Groningen 0b15859e33 Discovery: Removed unnecessary DiscoveryService reference from LocalDiscover, which causes circular Guice dependencies errors. 2014-11-12 11:25:41 +01:00