Commit Graph

4053 Commits

Author SHA1 Message Date
Robert Muir 260f5eb9ff Merge pull request #10913 from rmuir/spanspanspanspanspan
Add span within/containing queries.
2015-05-01 11:26:08 -04:00
Robert Muir aade6194b7 Add span within/containing queries.
Expose new span queries from https://issues.apache.org/jira/browse/LUCENE-6083

Within returns matches from 'little' that are enclosed inside of a match from 'big'.
Containing returns matches from 'big' that enclose matches from 'little'.
2015-04-30 23:31:31 -04:00
Igor Motov c165afb4d5 Logging: Add logging of slow cluster state tasks
Closes #10874
2015-04-30 19:13:49 -04:00
Jack Conradson aa968f6b65 Scripting: Add Field Methods
Added infrastructure to allow basic member methods in the expressions
language to be called.  The methods must have a signature with no arguments.  Also
added the following member methods for date fields (and it should be easy to add more)
* getYear
* getMonth
* getDayOfMonth
* getHourOfDay
* getMinutes
* getSeconds

Allow fields to be accessed without using the member variable [value].
(Note that both ways can be used to access fields for back-compat.)

closes #10890
2015-04-30 15:36:46 -07:00
Ryan Ernst a0451a37cc Upgrade lucene snapshot to r1677039 2015-04-30 13:43:19 -07:00
Adrien Grand 12c19508fb Tests: Fix test bug in aggregations' EquivalenceTests due to the change of the default min_doc_count. 2015-04-30 19:14:56 +02:00
Martijn van Groningen 7a6fe809d0 inner_hits: Don't use bitset cache for children filters.
Only parent filters should use bitset filter cache, to avoid memory being wasted.
Also in case of object fields inline the field name into the nested object,
instead of creating an additional (dummy) nested identity.

Closes #10662
Closes #10629
2015-04-30 16:56:43 +02:00
Adrien Grand e5be85d586 Aggs: Change the default `min_doc_count` to 0 on histograms.
The assumption is that gaps in histogram are generally undesirable, for instance
if you want to build a visualization from it. Additionally, we are building new
aggregations that require that there are no gaps to work correctly (eg.
derivatives).
2015-04-30 15:48:23 +02:00
Colin Goodheart-Smithe d16bf992a9 Aggregations: min_bucket aggregation
An aggregation to calculate the minimum value in a set of buckets.

Closes #9999
2015-04-30 13:34:21 +01:00
javanna 77ac4528fb Transport: read/writeGenericValue to support BytesRef
Add support for BytesRef to existing StreamInput#readGenericValue and StreamOutput#writeGenericValue

Closes #10878
2015-04-30 10:59:01 +02:00
Ryan Ernst 2fd387d378 fix dumb test copy/paste mistake 2015-04-29 23:33:43 -07:00
Ryan Ernst 79a1c38ed6 Mappings: Fix _field_names to not have doc values
When doc values were turned on a by default, most meta fields
had it explicitly disabled.  However, _field_names was missed.
This change forces doc values to be off always for _field_names
and removes the unnecessary support when creating index fields.

closes #10892
2015-04-29 23:12:16 -07:00
Igor Motov 8e5543dea0 Test: ignore cluster state differences on the nodes that disconnected from the master 2015-04-29 12:49:41 -04:00
Igor Motov bac135261c Test: make sure that tests are not affected by changing in address resolution settings 2015-04-29 12:49:40 -04:00
Igor Motov 38be1e8a1a Test: remove reference to the local node before comparing cluster states in ensureClusterStateConsistency 2015-04-29 12:49:40 -04:00
Simon Willnauer 9828e955f3 [TEST] enable host name resolving to gain consistent transport addresses in clusterstate 2015-04-29 12:49:40 -04:00
Simon Willnauer 58eed45ee5 [TEST] Move XContentTestUtils.java into o.e.test folder
Classes referenced by the Test base classes must be under this
package otherwise the test jar can't be used in a 3rd party application.
2015-04-29 12:49:40 -04:00
Igor Motov 478c253f89 Add support for cluster state diffs
Adds support for calculating and sending diffs instead of full cluster state of the most frequently changing elements - cluster state, meta data and routing table.

Closes #6295
2015-04-29 12:49:40 -04:00
David Pilato 3c3e9b63a7 fix: query string time zone not working
If you define exactly the same date range query using either `DATE+0200` notation or `DATE` and set `timezone: +0200`, elasticsearch gives back different results:

```
DELETE foo
PUT /foo
{
  "mapping": {
    "tweets": {
      "properties": {
        "tweet_date": {
          "type": "date"
        }
      }
    }
  }
}

POST /foo/tweets/1/
{
  "tweet_date": "2015-04-05T23:00:00+0000"
}
POST /foo/tweets/2/
{
  "tweet_date": "2015-04-06T00:00:00+0000"
}

GET /foo/tweets/_search?pretty
{
    "query": {
        "query_string": {
            "query": "tweet_date:[2015-04-06T00:00:00+0200 TO 2015-04-06T23:00:00+0200]"
        }
    }
}
GET /foo/tweets/_search?pretty
{
    "query": {
        "query_string": {
            "query": "tweet_date:[2015-04-06T00:00:00 TO 2015-04-06T23:00:00]",
            "time_zone": "+0200"
        }
    }
}
```

This PR fixes it and will also allow us to add the same feature to simple_query_string as well in another PR.

Closes #10477.

(cherry picked from commit 880f4a0)
2015-04-29 18:10:02 +02:00
Simon Willnauer d4463602f6 [TEST] Use a high shard delete timeout when clusterstates are delayed
`IndiceStore#indexCleanup` uses a disruption scheme to delay cluster state
processing. Yet, the delay is [1..2] seconds but tests are setting the shard
deletion timeout to 1 second to speed up tests. This can cause random not
reproducible failures in this test since the timeouts and delays are bascially
overlapping. This commit adds a longer timeout for this test to prevent these
problems.
2015-04-29 17:51:21 +02:00
Colin Goodheart-Smithe 88aa8934a3 Merge branch 'master' into feature/aggs_2_0 2015-04-29 16:10:19 +01:00
Britta Weber a202c2a434 Revert "Write state also on data nodes if not master eligible"
This reverts commit 4088dd38cb.
2015-04-29 17:07:07 +02:00
Colin Goodheart-Smithe a33e77ff96 Muted intermittently failing tests
To reproduce the failures use `-Dtests.seed=D9EF60095522804F`
2015-04-29 16:04:29 +01:00
Colin Goodheart-Smithe 57a8885964 Merge branch 'master' into feature/aggs_2_0
# Conflicts:
#	src/main/java/org/elasticsearch/index/query/CommonTermsQueryBuilder.java
#	src/main/java/org/elasticsearch/search/aggregations/AggregationModule.java
#	src/main/java/org/elasticsearch/search/aggregations/AggregatorFactories.java
#	src/main/java/org/elasticsearch/search/aggregations/AggregatorParsers.java
#	src/main/java/org/elasticsearch/search/aggregations/InternalMultiBucketAggregation.java
#	src/main/java/org/elasticsearch/search/aggregations/bucket/nested/NestedAggregator.java
#	src/main/java/org/elasticsearch/search/aggregations/metrics/InternalNumericMetricsAggregation.java
#	src/test/java/org/elasticsearch/search/aggregations/bucket/nested/NestedAggregatorTest.java
2015-04-29 15:49:41 +01:00
markharwood 528f6481ea Query enhancement: return positions of parse errors found in JSON
Extend SearchParseException and QueryParsingException to report position information in query JSON where errors were found. All query DSL parser classes that throw these exception types now pass the underlying position information (line and column number) at the point the error was found.

Closes #3303
2015-04-29 15:02:46 +01:00
Colin Goodheart-Smithe 3bb8ff2a92 fixed issue with eggs in percolation request for 1 shard 2015-04-29 14:55:20 +01:00
javanna 1d4df4b628 [TEST] remove source parameter validation from REST tests runner
source parameter is implicitly supported and doesn't need to be declared in rest spec. It is tested though, as every api that supports get with body can also get requests using POST with body or get with source query_string parameter.
2015-04-29 15:44:43 +02:00
Britta Weber 4088dd38cb Write state also on data nodes if not master eligible
When a node was a data node only then the index state was not written.
In case this node connected to a master that did not have the index
in the cluster state, for example because a master was restarted and
the data folder was lost, then the indices were not imported as dangling
but instead deleted.
This commit makes sure that index state for data nodes is also written
if they have at least one shard of this index allocated.

closes #8823
closes #9952
2015-04-29 15:27:21 +02:00
Simon Willnauer 94d8b20611 Add multi data.path to migration guide
this commit removes the obsolete settings for distributors and updates
the documentation on multiple data.path. It also adds an explain to the
migration guide.

Relates to #9498
Closes #10770
2015-04-29 11:51:37 +02:00
Simon Willnauer b4efb6c472 Remove unused / obsolete import and throws statements
This commit removes unused thorws statements when RuntimeExceptions are
mentioned in the throws statement. It also removes obsolete import statements
for java.lang.IllegalArgumentException and java.lang.IllegalStateException
2015-04-29 10:42:31 +02:00
Simon Willnauer 4f14af21c5 Remove ElasticsearchIAE and ElasticsearchISE
This commit removes ElasticsearchIAE and ElasticsearchISE in favor of
the JDKs IAE and ISE.

Closes #10794
2015-04-29 10:42:31 +02:00
Adrien Grand c23d550823 Merge pull request #10856 from jpountz/fix/remove_qparser_cache
Internal: Remove the query parser cache.
2015-04-29 09:45:51 +02:00
Simon Willnauer f18f623878 [TEST] Remove searchers from tracking map once they are closed 2015-04-29 09:33:20 +02:00
Ryan Ernst bf09e58cb3 Mappings: Remove includes and excludes from _source
Regardless of the outcome of #8142, we should at least enforce that
when _source is enabled, it is sufficient to reindex. This change
removes the excludes and includes settings, since these modify
the source, causing us to lose the ability to reindex some fields.

closes #10814
2015-04-28 15:03:51 -07:00
Ryan Ernst 99584deae8 Mappings: Consolidate document parsing logic
The code to parse a document was spread across 3 different classes,
and depended on traversing the ObjectMapper hiearchy. This change
consolidates all the doc parsing code into a new DocumentParser.
This should allow adding unit tests (future issue) for document
parsing so the logic can be simplified.  All code was copied
directly for this change with only minor modifications to make
it work within the new location.

closes #10802
2015-04-28 14:51:47 -07:00
Ryan Ernst f6d8b12796 Mappings: Explicitly disallow multi fields from using object or nested fields
Multi fields currently parse any field type passed in.  However, they
were only intended to support copying simple values from the outter
field. This change adds validation to ensure object and nested
fields are not used within multi fields.

closes #10745
2015-04-28 14:39:39 -07:00
Michael McCandless cf2fb4ed0f Remove core delete-by-query implementation, to be replaced with a plugin
The current implementation is dangerous: it unexpectedly refreshes,
which can quickly cause an unhealthy index (segment explosion).  It
can also delete different documents on primary vs replicas, causing
inconsistent replicas.

For 2.0 we will replace this with an optional plugin that does a
scan/scroll search and then issues bulk delete requests.

Closes #10859
2015-04-28 16:09:04 -04:00
Simon Willnauer c0d3205146 Fix more download URLs 2015-04-28 20:40:14 +02:00
Brian Murphy 3ad279b8fd Add `fairness` option to KeyedLock.
This change adds the ability of a `KeyedLock` to be `fair` this means that threads will aquire the lock in the order they ask for it.
2015-04-28 12:41:29 -04:00
Adrien Grand 9d890c472b Internal: Remove the query parser cache.
The original goal of this cache was to avoid parsing the same query several
times in case several shards are held on the same node. While this might
sound like a good idea, this would only help when parsing the query takes
non-negligible time compared to actually running the query, which should not
be the case.
2015-04-28 18:05:34 +02:00
Simon Willnauer 87cf1452d5 [REST] Render non-elasticsearch exception as root cause
if we don't have an ElasticsearchException as the wrapper of the
actual cause we don't render a root cause today. This commit adds
support for 3rd party exceptions as root causes.

Closes #10836
2015-04-28 17:46:31 +02:00
Lee Hinman 04f6067c66 Merge branch 'pr/10845' 2015-04-28 09:13:26 -06:00
Simon Willnauer bc3136a345 [TEST] Cleanup Direcotry and Searcher mock code
We deployed our own code to check if directories are closed etc an d
if serachers are still open. Yet, since we don't have a global cluster
anymore we can just use lucene's internal mechanism to do that. This commit
removes all special handling and usese LuceneTestCase.closeAfterSuite to
fail if certain resources are not closed

Closes #10853
2015-04-28 17:11:39 +02:00
Nik Everett cb89a14010 Add default to field_value_factor
field_value_factor now takes a default that is used if the document doesn't
have a value for that field. It looks like:
"field_value_factor": {
  "field": "popularity",
  "missing": 1
}

Closes #10841
2015-04-28 11:06:24 -04:00
Simon Willnauer 4f0492a780 [TEST] Run tests with 1 or 2 nodes by default
This commit adds support for running with only one node and sets the
maximum number of nodes to 3 by default. if run with test.nighly=true
at most 6 nodes are used. This gave a 20% speed improvement compared to
the previoulys minimum number of nodes of 3.
2015-04-28 14:38:53 +02:00
javanna b8caa52e7c [TEST] make CountRequestBuilder#toString and SearchRequestBuilder#toString tests non platform dependent 2015-04-28 14:25:00 +02:00
Boaz Leskes 54f2a91aff Test: testMinimumMasterNodes should use at least min_master_nodes unicast hosts 2015-04-28 13:09:00 +03:00
javanna c914134355 Scripting: remove groovy sandbox
Groovy sandboxing was disabled by default from 1.4.3 on though since we found out that it could be worked around, so it makes little sense to keep it and maintain it.

Closes #10156
Closes #10480
2015-04-28 11:27:50 +02:00
javanna 371bc5a6b3 Java api: implement toString in CountRequestBuilder
Similarly to what SearchRequestBuilder does, we print out a string representation of the query that the count request builder holds.

Closes #5555
2015-04-28 11:22:15 +02:00
javanna 78608536e2 Java api: SearchRequestBuilder#toString to print out the query without wiping the request source
Best effort to print out the search source depending on how it was set to the SearchRequestBuilder, don't call `internalBuilder() as that causes the content of the request to be wiped.

Closes #5576
2015-04-28 11:22:15 +02:00