5116 Commits

Author SHA1 Message Date
Christoph Büscher
ca21aa0cb5 Make reset() in QueryShardContext private
The query shard reset() method resets some internal state in the
query shard context, like clearing query names, the filter flag
or named queries. The problem with this method being public is
that it currently (miss?) used for modifying an existing context
for recursive invocatiob, but the contexts that have been reseted
that way cannot be properly set back to their previous state.

This PR is a step towards removing reset() entirely by first making
it only be used internally in QueryShardContext. In places where
reset() was used we can either create new QueryShardContexts or
modify the existing context because it is discarded afterwards anyway.
2016-05-03 18:56:16 +02:00
Jason Tedor
85d2fc0e38 Avoid string concatentation in IngestDocument.FieldPath
Today, the constructor for IngestDocument#FieldPath does a string
concatentation and two object allocation on every field path. This
commit removes these unnecessary operations.

Relates #18108
2016-05-03 10:05:44 -04:00
Yannick Welsch
98dba262cf Use object equality to compare versions in IndexSettings (#18103)
Fixes an issue where updating index metadata on a index from a version that it does not have in its static list of known versions fails.
2016-05-03 11:31:56 +02:00
Daniel Mitterdorfer
0a6f40c7f5 Enable HTTP compression by default with compression level 3
With this commit we compress HTTP responses provided the client
supports it (as indicated by the HTTP header 'Accept-Encoding').

We're also able to process compressed HTTP requests if needed.

The default compression level is lowered from 6 to 3 as benchmarks
have indicated that this reduces query latency with a negligible
increase in network traffic.

Closes #7309
2016-05-03 08:53:15 +02:00
Mike McCandless
0c6d6a5495 also compute completion stats for 2.x shards 2016-05-02 16:14:05 -04:00
Michael McCandless
16c4c251f5 Merge pull request #18094 from mikemccand/completion_stats_once
Don't try to compute completion stats on a reader after we already closed it

Conflicts:
	core/src/main/java/org/elasticsearch/index/shard/IndexShard.java
2016-05-02 14:49:15 -04:00
Jason Tedor
15f33e61b7 Kill redundant conditional in BootstrapCheck#check
This commit removes an unnecessary if statement in Bootstrap#check. The
removed if statement was duplicating the conditionals in the nested if
statements and was merely an artifact of an earlier refactoring.
2016-05-02 12:22:40 -04:00
Jason Tedor
226679c606 Mark MaxMapCountCheck#isSystemCheck as final
This commit marks the method MaxMapCountCheck#isSystemCheck as final as
this method should not be modifiable by inheriting classes (used in
tests).
2016-05-02 11:48:35 -04:00
Adrien Grand
8bdda49f90 Fail automatic string upgrade if the value of index is not recognized. #18082
Closes #18062
2016-05-02 17:28:43 +02:00
Jason Tedor
8e178c4f8e Add system bootstrap checks escape hatch
Today when running in production mode the bootstrap checks are
completely unforgiving. But there are cases where an end-user might not
have the ability to modify some of the system-level settings that cause
the bootstrap checks to trip (e.g., guest settings that are inherited
from a host and can not be modified). This commit adds a setting that
allows system-level bootstrap checks to be ignored for these
end-users. We classify certain bootstrap checks into system-level checks
and only those bootstrap checks will be ignored if this flag is
enabled. All other bootstrap checks are still subject to being enforced
if the user is in production mode. We will still log warnings for these
bootstrap checks because the end-user does still need to be made aware
that they are running in a configuration that is less-than-ideal from a
resiliency perspective.

Relates #18088
2016-05-02 10:59:50 -04:00
Isabel Drost-Fromm
372eceb854 Switch to using predicate for testing existing value 2016-05-02 15:41:05 +02:00
Isabel Drost-Fromm
4cf5385e4d Use plain random values for generating *SortBuilders 2016-05-02 13:50:00 +02:00
Isabel Drost-Fromm
cf1d0d5935 Merge branch 'master' into tests/switch_to_random_value_other_than_for_sort 2016-05-02 11:38:40 +02:00
Jason Tedor
0b41580656 Remove unused import from ScalingThreadPoolTests 2016-04-29 16:21:59 -04:00
Jason Tedor
c82e564886 Remove racy but unnecessary assertion
This commit removes a racy but unnecessary assertion in scaling thread
pool idle test. Namely, the main test thread can reach the removed
assertion before the last few threads in the thread pool have completed
their tasks and caused the completed tasks count on the underlying
executor to be updated. But this assertion is unnecessary. The main test
thread already waits on a latch that is only decremented immediately
before a task completes. This ensures that it was in fact the case that
every submitted task was executed.

Closes #18072
2016-04-29 16:21:22 -04:00
Christoph Büscher
262a814c8d Tests: Fix TermsQueryBuilderTests expectations when lookup returns no terms
When the termslookup (mocked in this case) doesn't return any terms, the
query used to rewrite to an empty boolean query. Now it rewrites to a
MatchNoDocsQuery. This changes the test expectation accordingly.

Closes #18071
2016-04-29 19:29:25 +02:00
Ali Beyad
a31b94e237 When checking if an index tombstone can be applied, use both the index
name and uuid because the cluster state may contain an active index of
the same name (but different uuid).

Closes #18058
Closes #18054
2016-04-29 11:40:31 -04:00
Martijn van Groningen
7aca1389e2 ingest: Add date_index_name processor.
Closes #17814
2016-04-29 17:20:48 +02:00
Jason Tedor
f349c4f135 Return null for "same" thread pool info
This commit modifes the EsThreadPoolTestCase#info helper method to
return null when info for the thread pool can not be found. This really
should only happen for the "same" thread pool, and so we also assert
that we only get to a place where there is no info if the thread pool
that info was requested for is in fact the "same" thread pool. Not
returning null here and instead throwing an exception would fail tests
that tried to lookup info on the "same" thread pool.
2016-04-29 10:29:02 -04:00
Martijn van Groningen
8c77399f1c test: changed test's expectation that a BooleanQuery is returned. A NoMatchDocsQuery is returned instead now. 2016-04-29 15:47:03 +02:00
David Pilato
2232a7cdf3 Merge branch 'pr/cat-size-time-units' 2016-04-29 15:09:14 +02:00
Jason Tedor
c25f8ad912 Avoid sliced locked contention in internal engine
Today we use a sliced lock strategy for acquiring locks to prevent
concurrent updates to the same document. The number of sliced locks is
computed as a linear function of the number of logical
processors. Unfortunately, the probability of a collision against a
sliced lock is prone to the birthday problem and grows faster than
expected. In fact, the mathematics works out such that for a fixed
target probability of collision, the number of lock slices should grow
like the square of the number of logical processors. This is
less-than-ideal, and we can do better anyway. This commit introduces a
strategy for avoiding lock contention within the internal
engine. Ideally, we would only have lock contention if there were
concurrent updates to the same document. We can get close to this ideal
world by associating a lock with the ID of each document. This
association can be held in a concurrent hash map. Now, the JDK
ConcurrentHashMap also uses a sliced lock internally, but it has several
strategies for avoiding taking the locks and these locks are only held
for a very short period of time. This implementation associates a
reference count with the lock that is associated with a document ID and
automatically removes the document ID from the concurrent hash map when
the reference count reaches zero.

Relates #18060
2016-04-29 08:05:23 -04:00
Martijn van Groningen
6c3beaa2eb Drop top level inner hits in favour of inner hits defined in the query dsl.
Fix a limitation that prevent from hierarchical inner hits be defined in query dsl.

Removed the nested_path, parent_child_type and query options from inner hits dsl. These options are only set by ES
upon parsing the has_child, has_parent and nested queries are using their respective query builders.

These options are still used internally, when these options are set a new private copy is created based on the
provided InnerHitBuilder and configuring either nested_path or parent_child_type and the inner query of the query builder
being used.

Closes #11118
2016-04-29 11:17:24 +02:00
Jason Tedor
0cf1d16187 Fix line-length violation in SetProcessor
This commit addresses a checkstyle line-length violation in
SetProcessor.java.
2016-04-28 23:41:16 -04:00
Tal Levy
6302fb65a3 add ability to disable ability to override values of existing fields in set processor 2016-04-28 13:50:19 -07:00
Lee Hinman
a0ba3c9a02 Merge remote-tracking branch 'dakrone/add-store-status-to-explain' 2016-04-28 13:47:55 -06:00
Lee Hinman
cb29e041c4 Handle the case where allocatedPostIndexCreate is true 2016-04-28 13:46:06 -06:00
Jay Modi
34b5839487 Merge pull request #17949 from jaymode/list_setting_exists
fix exists method for list settings when using numbered setting format
2016-04-28 12:25:48 -04:00
Martijn van Groningen
22dc06fb7d removed duplicated code 2016-04-28 16:42:12 +02:00
Isabel Drost-Fromm
a03b131319 Shorten nested test query builder generation. 2016-04-28 15:31:56 +02:00
Isabel Drost-Fromm
3490daa7ea Get rid of no longer needed missing generation method 2016-04-28 15:27:36 +02:00
Isabel Drost-Fromm
0c7e58de93 Get rid of duplicate random sort order generation 2016-04-28 15:14:15 +02:00
Isabel Drost-Fromm
8dc0610eaa Get rid of duplicated random ascii generation 2016-04-28 15:07:55 +02:00
Isabel Drost-Fromm
47fefdd273 Switch from separate sort_mode to more general randomValueOtherThan
... for sort tests only ...
2016-04-28 14:45:56 +02:00
Isabel Drost-Fromm
3ab006e422 Switch to new random value supplier for test. 2016-04-28 14:27:22 +02:00
Isabel Drost-Fromm
a19c426e0f Deprecate coerce/ignore_malformed for GeoBoundingBoxQuery 2016-04-28 14:10:59 +02:00
Mathias Fußenegger
a5ab648fae Handle indices=["_all"] when restoring a snapshot (#18025)
Setting indices=["_all"] when restoring a snapshot would throw an IndexNotFoundException instead of selecting all indices as in other APIs.
2016-04-28 14:10:01 +02:00
Isabel Drost-Fromm
3f743a30cf Deprecate coerce/ignore_malformed in GeoDistanceQueryBuilder 2016-04-28 14:06:27 +02:00
Isabel Drost-Fromm
3160798084 Deprecate coerce/ignore_malformed for GeoDistanceRangeQuery 2016-04-28 14:01:54 +02:00
Isabel Drost-Fromm
5306de3ce3 Deprecate coerce/ignore_malformed for GeoPolygonQueryBuilder
Includes update to parsing code, tests, migration docs and reference
docs.
2016-04-28 13:56:50 +02:00
Isabel Drost-Fromm
7bcc079d60 Adjust wording according to review. 2016-04-28 12:39:16 +02:00
Isabel Drost-Fromm
c34ba6fe1e Merge branch 'master' into enhancement/switch_boolean_to_match_no_docs_query 2016-04-28 12:35:43 +02:00
Isabel Drost-Fromm
78ff4f52d6 Introduces GeoValidationMethod to GeoDistanceSortBuilder
Previously like in other geo related query parsers we were using
a combination of two booleans for coerce and ignore_malformed
which was error prone and not very clear.

Switched to using GeoValidationMethod instead as we already do
e.g. in GeoBoundingBoxQueryBuilder.

Left support for both, coerce and ignore_malformed in the parser
but deprecated the two in favour of validation method.

Introduced the same deprecation in geo bounding box query builder.
2016-04-28 12:10:56 +02:00
Adrien Grand
de43bda3f3 iter 2016-04-28 09:39:08 +02:00
Adrien Grand
b77893b1d1 Fail queries on not indexed fields.
While returning no hits on fields that are not mapped may be fine, it is not
for fields that are mapped but not indexed (`index:false`). We should fail the
query in that case rather than returning no hits.
2016-04-28 09:37:04 +02:00
Nik Everett
1df4efccd5 Wrap settings at 140 columns
and remove the checkstyle suppressions.
2016-04-27 18:18:20 -04:00
Nik Everett
6cccd4462c Wrap Version at 140 columns
and remove its checkstyle_suppression line
2016-04-27 17:25:37 -04:00
Nik Everett
47577613cb Wrap tasks code at 140 columns
Switch something from an explicit toString to Strings.toString which
is the same thing but with more code reuse.

Also renamed a constant to be CONSTANT_CASE.
2016-04-27 17:05:09 -04:00
Nik Everett
3468be0075 [test] Remove BulkProcessorRetryIT's generic threadpool setting
It wasn't required to excercise the retries and it caused stability issues.
2016-04-27 16:21:39 -04:00
Nik Everett
d9afb8bfd6 [test] Wrap long lines in TasksIT 2016-04-27 15:32:56 -04:00