Commit Graph

21368 Commits

Author SHA1 Message Date
Jim Ferenczi 3dc21e780b Add missing index name to search slow log.
This commits adds the index name as part of the logging message.
Closes #17025
2016-04-18 12:13:10 +02:00
Daniel Mitterdorfer b84a192884 Honor invariant Suggest.suggestions != null 2016-04-18 10:18:23 +02:00
Daniel Mitterdorfer b9faa13274 Balance request sizes in NettyHttpRequestSizeLimitIT
This test should demonstrate that a single (larger)
request is processed but on of multiple large concurrent requests
is rejected. This test broke too early under some circumstances in
network mode as the limit is quite low.

With this commit we reduce the size of the individual large
requests but issue more concurrent ones thus increasing stability
of this test.
2016-04-18 09:31:48 +02:00
Sergii Golubev 5ce3eb96b0 tophits-aggregation.asciidoc: fix a typo 2016-04-18 09:23:39 +02:00
Nik Everett d3b1306069 Reindex: never report negative throttled_until
Just clamp the value at 0. It isn't useful to tell the user "this
thread should have woken 5ms ago".

Closes #17783
2016-04-15 16:53:23 -04:00
Jim Ferenczi 4575d79522 Merge pull request #17776 from jimferenczi/simple_query_string_wildcard
Apply the default operator on analyzed wildcard in simple_query_string builder
2016-04-15 22:22:55 +02:00
Jim Ferenczi b4e6b63088 Apply the default operator on analyzed wildcard in simple_query_string builder:
* This is a followup from https://github.com/elastic/elasticsearch/pull/17711 where we now apply the default operator on analyzed wildcard query in query_string builder.
2016-04-15 21:56:50 +02:00
Nik Everett 65803f8abd Cut significant_terms to registerAggregation
and remove its PROTOTYPE.

Relates to #17085
2016-04-15 14:23:43 -04:00
Nik Everett d223090d7d Cut top_hits aggregation to registerAggregation
and remove its PROTOTYPE.

Relates to #17085
2016-04-15 14:15:12 -04:00
Nik Everett c94302d246 Cut geo aggregations to registerAggregation
and remove their prototypes.

Relates to #17085
2016-04-15 14:01:48 -04:00
Michael McCandless b55368b39d Merge pull request #17778 from mikemccand/imc_registered_settings
Switch to registered Settings for all IndexingMemoryController settings
2016-04-15 13:14:12 -04:00
Nik Everett c966d14f8b [build] Don't resolve the revision hash unless you need it
This fixes the build for folks that build without git installed locally
and should speed up the general case because we aren't trying to resolve
git information when it isn't really needed.
2016-04-15 11:35:05 -04:00
Alexander Reelsen 95579ca95a Build: Allow for file based deploy, sign packages
This allows for a local file based deploy without needed nexus
auth information.

Also signing of packages has been added, either via gradle.properties
or using system properties as a fallback.

The property build.repository allows to configure another endpoint if no
snapshot build is done.

Fix creation of .asc file for tar.gz distribution

Closes #17405
2016-04-15 17:14:05 +02:00
Nik Everett b5a58ece41 Cut nested and reverse_nested aggregations to registerAggregation
and remove their PROTOTYPES.

Relates to #17085
2016-04-15 10:19:03 -04:00
Adrien Grand eb4ba38032 Add points to SegmentStats. #17775
This way points memory and disk usage will be reported in the stats API.

Closes #16974
2016-04-15 15:58:26 +02:00
Nik Everett 7f6a765a1e Cut the sampler and diversified_sampler aggregations to registerAggregation
and remove their PROTOTYPEs.

Relates to #17085
2016-04-15 09:53:56 -04:00
Mike McCandless a361f5b810 fix test method name 2016-04-15 09:47:08 -04:00
Mike McCandless 529087301a fix IndexingMemoryController to use existing Settings API support to parse % or byte size, and to enforce min/max values; don't try to share string constants for settings in tests 2016-04-15 09:42:29 -04:00
Christoph Büscher 068df5f8aa Merge pull request #17756 from cbuescher/no-pfm-in-parser
Remove ParseFieldMatcher from AbstractXContentParser
2016-04-15 15:21:27 +02:00
Christoph Büscher de036d63d8 Rename context.parseFieldMatcher() to context.getParseFieldMatcher 2016-04-15 15:15:32 +02:00
Christoph Büscher 15c59d07b3 Remove ParseFieldMatcher from AbstractXContentParser
Currently we are able to set a ParseFieldMatcher on XContentParsers,
mainly to conveniently carry it around to be available where the
actual parsing happens. This was just recently introduced together
with ObjectParser so that ObjectParser can make use of deprecation
logging and throwing errors while parsing.

This however is trappy because we create parsers in so many places in
the code and it is easy to forget setting the right ParseFieldMatcher.
Instead we should hold the ParseFieldMatcher only in the parse contexts
(e.g. QueryParseContext).

This PR removes the ParseFieldMatcher from XContentParser. ObjectParser
can still make use of it because we can make the otherwise unbounded
`context` type to extend an interface that makes sure contexts used in
ObjectParser can supply a ParseFieldMatcher. Contexts in ObjectParser
are now no longer optional, but it is sufficient to pass in a small
lambda expression in places where no other context is available.

Relates to #17417
2016-04-15 15:14:46 +02:00
Nik Everett e3c65408ec Cut terms aggregation to registerAggregation
and remove its PROTOTYPE. This is the first aggregation builder that
serializes its targetValueType so ValuesSourceAggregatorBuilder had to
grow support for that.

Relates to #17085
2016-04-15 09:12:29 -04:00
Nik Everett cf80b00507 Cut filters aggregation to registerAggregation
and remove its PROTOTYPE.
2016-04-15 09:04:46 -04:00
Nik Everett 135511134c Fix bad toXContent for derivative aggregation
I busted it in the last commit.
2016-04-15 08:51:22 -04:00
Nik Everett a36b6138d7 Cut avg aggregation to registerAggregation
and remove its PROTOTYPE

Relates to #17085
2016-04-15 08:37:22 -04:00
Nik Everett b10c1f1939 Cut derivative aggregation to registerPipelineAggregation
and remove its PROTOTYPE.
2016-04-15 08:28:18 -04:00
Nik Everett beafae8ea5 Cut range aggregations to registerAggregation
and remove their PROTOTYPES. Does not remove the PROTOTYPEs from their
Range implementations which will have to wait for another commit.
2016-04-15 08:14:05 -04:00
Mike McCandless 36e99c5f8d switch to registered Settings for all IndexingMemoryController settings 2016-04-15 06:10:30 -04:00
Colin Goodheart-Smithe d863cbaa07 [TEST] fix geo_bounding_box tests to work with unmapped fields 2016-04-15 10:10:43 +01:00
Daniel Mitterdorfer 3688629e11 Adjust line-length of transport related classes to coding standard 2016-04-15 10:12:24 +02:00
Ryan Ernst c3f4eb36e3 Merge pull request #17768 from rjernst/dots4
Mappings: Fix array parsing to remove its context when finished parsing
2016-04-15 00:22:55 -07:00
LeonardGC 0b8be7f894 Update field-mapping.asciidoc (#17670) 2016-04-15 09:22:38 +02:00
Jason Tedor f985cfc95a Add max map count check
This commit adds a bootstrap check on Linux for the max map count (max
virtual memory areas) available to the Elasticsearch process.

Relates #16944
2016-04-14 21:04:03 -04:00
Igor Motov f030796b0b Shard level tasks in Bulk Action lose reference to their parent tasks
During the bulk action a hierachy of tasks is getting created: bulk->bulk[s] (coord node) -> bulk[s] (primary shard node) -> bulk[s][p] and bulk[s][r]. Due to a bug the first bulk[s] task didn't have bulk's task id is set as a parent id.  This commit fixes this bug.
2016-04-14 19:30:07 -04:00
Ryan Ernst 24130e1f30 Merge pull request #17764 from rjernst/parse_doc_without_threadlocal
Internal: Remove threadlocal from document parser
2016-04-14 15:17:45 -07:00
Ryan Ernst 6d8ab56350 Merge pull request #17769 from rjernst/gradle_idea_excludes
Build: Exclude gradle and eclipse build dirs from intellij
2016-04-14 15:16:05 -07:00
Ryan Ernst ba2abbf37c Build: Exclude gradle and eclipse build dirs from intellij
Intellij has a model of "everything is a source dir unless you say
otherwise". This change fixes the intellij configuration to not think
the gradle or eclipse build dirs are source dirs.
2016-04-14 14:54:52 -07:00
Ryan Ernst 60df3450f4 Mappings: Fix array parsing to remove its context when finished parsing
Handling of the current path when parsing a document is very sensitive.
This fixes a subtle bug in array parsing, where the path that was added
by parsing an array would not be cleared. It also adds a hard state
check at the end of parsing to ensure we ended with a clean path.
2016-04-14 14:45:29 -07:00
Ryan Ernst c6dd17a2c6 Internal: Remove threadlocal from document parser
The doc parser uses a context object to store the state of parsing,
namely the existing mappings, new mappings, and the parsed document.
Currently this uses a threadlocal which is "reset" for each doc parsed.
However, the thread local doesn't actually save anything, since
resetting is constructing new objects. This change removes the thread
local, which also simplifies the mapper service as it now does not need
to be closeable.
2016-04-14 13:10:20 -07:00
bloublou 83944c5628 Typo correction heap_size.asciidoc (#17745)
* Typo correction Xms Xmx

Typo correction on "-Xms4000mb -Xmx4000mb"

* Change mb to m for Xms/Xmx
2016-04-14 20:37:37 +02:00
Jason Tedor ea26e86880 Actually restore lost Node#start no-op check
This commit really reverts the inadvertent removal of allowing duplicate
calls to Node#start to be a no-op (but was mistakenly restored to
Node#stop in ddfa3a661510f25c2ce431dfd6fb86ac11eb8888).
2016-04-14 14:26:04 -04:00
Jason Tedor f87c599ff1 Revert "Restore lost Node#start no-op check"
This reverts commit ddfa3a661510f25c2ce431dfd6fb86ac11eb8888.
2016-04-14 14:26:04 -04:00
jaymode f35cfc3715 Copy checkstyle config to allow running from a jar
The checkstyle configuration files were being accessed as resources within the project and
being converted from a URL to a File by gradle. This works when the build tools project is being
referenced as a local project. However, when using the published jar the URL points to a resource
in the jar file, that URL cannot be converted to a File object and causes the build to fail.

This change copies the files into a `checkstyle` directory in the project build folder and always uses
File objects pointing to the copied files.
2016-04-14 13:33:10 -04:00
Nik Everett 3197f7f7d8 Remove PROTOTYPE from filter aggregation
and cut it to registerAggregation

Relates to #17085
2016-04-14 12:37:40 -04:00
Adrien Grand 0c9aca94d4 Disallow fielddata loading on text fields that are not indexed. #17747
If a field is not indexed, we cannot load fielddata for it.
2016-04-14 18:11:11 +02:00
Colin Goodheart-Smithe 90d044309f Merge pull request #17710 from colings86/deprecate/indicesQuery
Deprecate Indices query
2016-04-14 17:09:42 +01:00
Adrien Grand d84c643f58 Use the new points API to index numeric fields. #17746
This makes all numeric fields including `date`, `ip` and `token_count` use
points instead of the inverted index as a lookup structure. This is expected
to perform worse for exact queries, but faster for range queries. It also
requires less storage.

Notes about how the change works:
 - Numeric mappers have been split into a legacy version that is essentially
   the current mapper, and a new version that uses points, eg.
   LegacyDateFieldMapper and DateFieldMapper.
 - Since new and old fields have the same names, the decision about which one
   to use is made based on the index creation version.
 - If you try to force using a legacy field on a new index or a field that uses
   points on an old index, you will get an exception.
 - IP addresses now support IPv6 via Lucene's InetAddressPoint and store them
   in SORTED_SET doc values using the same encoding (fixed length of 16 bytes
   and sortable).
 - The internal MappedFieldType that is stored by the new mappers does not have
   any of the points-related properties set. Instead, it keeps setting the index
   options when parsing the `index` property of mappings and does
   `if (fieldType.indexOptions() != IndexOptions.NONE) { // add point field }`
   when parsing documents.

Known issues that won't fix:
 - You can't use numeric fields in significant terms aggregations anymore since
   this requires document frequencies, which points do not record.
 - Term queries on numeric fields will now return constant scores instead of
   giving better scores to the rare values.

Known issues that we could work around (in follow-up PRs, this one is too large
already):
 - Range queries on `ip` addresses only work if both the lower and upper bounds
   are inclusive (exclusive bounds are not exposed in Lucene). We could either
   decide to implement it, or drop range support entirely and tell users to
   query subnets using the CIDR notation instead.
 - Since IP addresses now use a different representation for doc values,
   aggregations will fail when running a terms aggregation on an ip field on a
   list of indices that contains both pre-5.0 and 5.0 indices.
 - The ip range aggregation does not work on the new ip field. We need to either
   implement range aggs for SORTED_SET doc values or drop support for ip ranges
   and tell users to use filters instead. #17700

Closes #16751
Closes #17007
Closes #11513
2016-04-14 17:56:23 +02:00
Nik Everett 9080f51340 NORELEASE for getWriteableName in aggregations base class 2016-04-14 11:33:03 -04:00
Nik Everett f095e64825 Remove PROTOTYPE from histogram aggregations 2016-04-14 11:22:52 -04:00
Ali Beyad b87fd54ba9 Improvements to the IndicesService class
This commit contains the following improvements/fixes:
  1. Renaming method names and variables to better reflect the purpose
of the method and the semantics of the variable.
  2. For deleting indexes, replace the closed parameter passed to the
delete index/store methods with obtaining the index's state from the
IndexSettings that is already passed in.
  3. Added tests to the IndexWithShadowReplicaIT suite, some of which
show issues in the shadow replica delete process that are captured in
Github issue 17695.

Closes #17638
2016-04-14 11:14:02 -04:00