Commit Graph

10194 Commits

Author SHA1 Message Date
Martijn van Groningen 099b1a70d5 Core: Let the disk threshold decider take into account shards moving away from a node in order to determine if a shard can remain.
By taking this into account we can prevent that we move too many shards away than is necessary.

Closes #8538
Closes #8659
2014-11-26 10:14:02 +01:00
Simon Willnauer 716212c037 Raise REST test Timeout - LocalGW takes it't tall... 2014-11-26 09:12:34 +01:00
Igor Motov 7e0b75625b Core: clarify index removal log message 2014-11-25 21:50:07 -05:00
Chris Earle 08521a4066 Revert "Update to Jackson 2.4.3"
This reverts commit 7523d0b150.
2014-11-25 16:41:33 -05:00
Nicholas Knize 6692ac3b75 Adding unit test for even / odd boundary condition 2014-11-25 13:32:01 -06:00
Clément Tourrière 15db5b98d2 Fix for geohash neighbors when geohash length is even.
We don't have to set XLimit and YLimit depending on the level (even or odd), since semantics of x and y are already swapped on each level.
XLimit is always 7 and YLimit is always 3.

Close #8526
2014-11-25 13:31:56 -06:00
Chris Earle 7523d0b150 Update to Jackson 2.4.3
- Update pom to 2.4.3 from 2.4.2
- Enable the CBOR data header (aka tag) from the CBOR Generator to provide binary identification like the Smile format
- Check for the CBOR header and ensure that the data sent in represents a "major type" that is an object
- Cleans up `JsonVsCborTests` unused imports
2014-11-25 14:03:16 -05:00
Sebastian Ziebell 3a6c6f4b26 Docs: Adds documentation for indices.exists_template
Closes: #8657
2014-11-25 19:36:01 +01:00
Sebastian Ziebell 4dd228d02f Spec: Fixes URL links to documentation
Closes #8656
2014-11-25 19:23:59 +01:00
tristanbob 807f363d6d Added note that ES packages automatically change vm.max_map_count
Closes #8601
2014-11-25 18:25:46 +01:00
Adrien Grand d22645cbfc Scripts: Return new lists on calls to getValues.
Scripts currently share the same list across invocations to getValues. This
caused a bug in script fields where all documents coming from the same segment
would get the same values (basically, for the next document for which script
values have been requested). Scripts now return a fresh new list on every
invocation to `getValues`.

Close #8576
2014-11-25 17:39:26 +01:00
Nils Dijk 0f4ca09e54 Aggregations: fix rounding issues on DST switch.
Closes #8339.
2014-11-25 16:48:03 +01:00
Karel Minarik d33f1d4bd8 [SPEC] Removed the utilities for generating REST specs/code
The utilities have been migrated to the elasticsearch/elasticsearch-ruby repository.

See: https://github.com/elasticsearch/elasticsearch-ruby/tree/master/elasticsearch-api/utils

Related: elasticsearch/elasticsearch-ruby@dfac51d

Closes #8606
2014-11-25 15:33:44 +01:00
Simon Willnauer 9e7b15b8f3 [GATEWAY] Cut over MetaDataStateFormat to Path API
Closes #8609
2014-11-25 15:07:10 +01:00
Simon Willnauer 82868e9cf2 remove unnecessary clearScroll call - these contexts are released by delete index now 2014-11-25 14:53:38 +01:00
Martijn van Groningen 13d1bb5681 Parent/child: Fixed parent/child not being able to be used in alias filters.
Closes #8628
2014-11-25 14:46:29 +01:00
Simon Willnauer 0a4e041ee5 [RECOVERY] Throw IndexShardClosedException if shard is closed
Today we throw a generic ElasticsearchException when a recovery is cancled. This
causes verbose logging and send shard failures and additional unnecessary cluster state
events. We can just throw IndexShardClosedException which prevents the send shard failures
2014-11-25 14:35:06 +01:00
Matt Hughes afba977e80 Docs: Added swift openstack repository
Closes #8583
2014-11-25 13:49:15 +01:00
David Haney 2c429452e9 Typo: changed "5% or the real words" to "5% of the real words"
Closes #8582
2014-11-25 13:15:33 +01:00
Simon Willnauer 35b278fc68 [TRANSLOG] Cut over to Path API
This commit moves all the Translog related code over to the
NIO2 Path API. It also make transaction logs write once since it
never reuses a translog file.

Closes #8611
2014-11-25 12:43:57 +01:00
Lee Hinman 6749b2c306 [TEST] Reduce stringency of breaker assertions
While in a perfect world we should only ever have 2 circuit breaker
trips, it's possible to get a race condition between the child and the
parent breaker with many threads. Since multiple breaking exceptions are
not actually a bad thing, it's okay to relax the constraints in the
test.

The race conditions are due to no locking inside the breaker logic, to
ensure that it is as low overhead as possible. Even though no locking is
used, we use atomic counters internally to ensure that the "estimated"
numbers for the breakers are never out of sync (which this test still
checks with no leeway).
2014-11-25 11:53:29 +01:00
Colin Goodheart-Smithe c420a17f7d Aggregations: Added getProperty method to Aggregations
This allows arbitrary properties to be retrieved from an aggregation tree. The property is specified using the same syntax as the
order parameter in the terms aggregation. If a property path contians a multi-bucket aggregation the property values from each bucket will be returned in an array.
2014-11-25 10:07:42 +00:00
Michael McCandless 856b294441 Core: let Lucene kick off merges
Today, Elasticsearch has a separate merge thread pool checking once
per second (by default) if any merges are necessary, but this is no
longer necessary since we can and do now tell Lucene's
ConcurrentMergeScheduler never to "hard pause" threads when merges
fall behind, since we do our own index throttling.

This change goes back to letting Lucene launch merges as needed, and
removes these two expert settings:

  index.merge.force_async_merge
  index.merge.async_interval

Now merges kick off immediately instead of waiting up to 1 second
before running.

Closes #8643
2014-11-25 04:13:57 -05:00
unknown c4e2f63b17 Aggregations: Make size parsing consistent for strings.
Close #6061
2014-11-25 08:55:24 +01:00
Igor Motov 668dbe498a Snapshot/Restore: fix compilation issue for JDK 1.7 2014-11-24 21:17:03 -05:00
Igor Motov 1aff8631ed Snapshot/Restore: restore with wait_for_completion=true should wait for succesfully restored shards to get started
This commit ensures that restore operation with wait_for_completion=true doesn't return until all successfully restored shards are started. Before it was returning as soon as restore operation was over, which cause some shards to be unavailable immediately after restore completion.

Fixes #8340
2014-11-24 19:37:43 -05:00
Sebastian Ziebell e0d6e0a6d3 Spec: Fixes a few locations of the params array in JSON specification
A few locations of the `params` object in the JSON specification were not located in the `url` block.

Closes #8607
2014-11-24 20:43:33 +01:00
Adrien Grand d60500f22e Fielddata: Fix iterator over global ordinals.
Our iterator over global ordinals is currently incorrect since it does NOT
return -1 (NO_MORE_ORDS) when all ordinals have been consumed. This bug does
not strike immediately with elasticsearch since we always consume ordinals in
a random-access fashion. However it strikes when consuming ordinals through
Lucene helpers such as DocValues#docsWithField.

Close #8580
2014-11-24 19:42:53 +01:00
Martijn van Groningen 13b9e07522 Core: Fields defined in the `_default_` mapping of an index template should be picked up when an index alias filter is parsed if a new index is introduced when a document is indexed into an index that doesn't exist yet.
Closes #8473
2014-11-24 18:25:31 +01:00
Simon Willnauer 043b37d932 [GATEWAY] Remove _state directory if index has been deleted
Today we try to delete the index directory if all shard locks have been
acquired. Yet, if this fails due to still running recoveries etc. We might
re-import the index as dangeling which also can happen if the node is restarted.
In contrast to the shard direcotries we can safely delete the metastate which is used
to import dangling indices while leaving the shard directories untouched.
2014-11-24 16:00:45 +01:00
Reuben Sutton fda1576d55 Fix SearchRequest.templateParams so that it is a Map<String, Object> so that it can take more data-types than just strings, to support Arrays. 2014-11-24 14:46:48 +00:00
Nicholas Knize 08bbfac7eb Updating comments in BasePolygonBuilder to explain "pairwise" intersection 2014-11-24 08:34:10 -06:00
Nicholas Knize 82f5252c14 Removing IntersectionOrder.SENTINEL and adding Edge.MAX_COORDINATE for code readability. 2014-11-24 08:33:56 -06:00
Nicholas Knize fc955551d4 [GEO] Fix for geo_shape query with polygon from -180/90 to 180/-90
This fix adds a simple consistency check that intersection edges appear pairwise. Polygonal boundary tests were passing (false positive) on the Eastern side of the dateline simply due to the initial order (edge direction) of the intersection edges.  Polygons in the Eastern hemispehere (which were not being tested) were correctly failing inside of JTS due to an attempt to connect incorrect intersection edges (that is, edges that were not even intersections). While this patch fixes issue/8467 (and adds broader test coverage) it is not intented as a long term solution.  The mid term fix (in work) will refactor all geospatial computational geometry to use ENU / ECF coordinate systems for higher accuracy and eliminate brute force mercator checks and conversions.

Closes #8467
2014-11-24 08:31:36 -06:00
Martijn van Groningen 1d7cdd7d22 Applied PR, changed the way defaults are handled and updated the docs.
Closes #4452
2014-11-24 13:32:41 +01:00
Brusic 5a001e1357 Cleaner error handling. Pre without post (or vice versa) is not an error condition. Set appropriate defaults for pre/post. 2014-11-24 13:28:55 +01:00
Brusic 94074b4028 Use static int for easier to read code 2014-11-24 13:28:55 +01:00
Brusic 7c10b445d4 Expose dist/pre/post options for SpanNotQuery 2014-11-24 13:28:54 +01:00
Lee Hinman 45408844e7 Remove NoneGateway, NoneGatewayAllocator, & NoneGatewayModule
Always use the LocalGateway* equivalents

We already check in the LocalGateway whether a node is a client node, or
is not master-eligible, and skip writing the state there. This allows us
to remove this code that was previously used only for tribe nodes (which
are not master eligible anyway and wouldn't write state) and in
tests (which can shake more bugs out)
2014-11-24 12:22:05 +01:00
dw ad408eee85 Docs: Reword note regarding _source for accuracy
Previously it suggested _source was always present, when that is not the case.

Closes #8491
2014-11-24 12:19:44 +01:00
Laurent Broudoux feb465f26f Docs: Update plugins.asciidoc on river plugins section
Adding links to Amazon S3 and Google Drive river plugins

Closes #8544
2014-11-24 12:15:12 +01:00
Michael McCandless dfb6d6081c Core: upgrade to current Lucene 5.0.0 snapshot
Elasticsearch no longer unlocks the Lucene index on startup (this was
dangerous, and could possibly lead to corruption).

Added the new serbian_normalization TokenFilter from Lucene.

NoLockFactory is no longer supported (index.store.fs.fs_lock = none),
and if you have a typo in your fs_lock you'll now hit a StoreException
instead of silently using NoLockFactory.

Closes #8588
2014-11-24 05:08:42 -05:00
Uwe Schindler 866571f4d7 Switch to forbidden-apis 1.7 2014-11-24 09:58:44 +01:00
Adrien Grand 8346e92ebb Core: Fix script fields to be returned as a multivalued field when they produce a list.
This change is essentially the same as #3015 but on script fields.

Close #8592
2014-11-24 09:41:16 +01:00
mdzor bc52ccfd33 Docs: Update update-settings.asciidoc
Inconsistent indentation

Closes #8525
2014-11-23 14:45:56 +01:00
barbasa fd6c41bfbf Missing quote in the example 2014-11-23 14:03:58 +01:00
Alban Perillat-Merceroz 54466938da Fix error in documentation
Indexation does not fail if no timestamp provided when there is a default value defined in mapping.
2014-11-23 14:02:51 +01:00
dw bb81055c33 Docs: Remove reference to imaginary "no_docs_query"
No reference to it in the source code except this file.

Closes #8566
2014-11-23 13:56:33 +01:00
Mariam Hakobyan 4a1ab6543c Docs: Added new elasticsearch-river-kafka plugin to the documentation (uses latest version of Kafka, EL Bulk API, and supports concurrent requests)
Closes #8518
2014-11-23 12:28:45 +01:00
Simon Willnauer b6b3382a8b [STORE] Use Lucene checksums if segment version is >= 4.9.0
We started to use the lucene CRC32 checksums instead of the legacy Adler32
in `v1.3.0` which was the first version using lucene `4.9.0`. We can safely
assume that if the segment was written with this version that checksums
from lucene can be used even if the legacy checksum claims that it has a Adler32
for a given file / segment.

Closes #8587

Conflicts:
	src/main/java/org/elasticsearch/index/store/Store.java
	src/test/java/org/elasticsearch/index/store/StoreTest.java
2014-11-21 22:35:21 +01:00