Commit Graph

7266 Commits

Author SHA1 Message Date
Martijn van Groningen 244723d401 Added `fields` support to `geo_point` and `completion` field type.
Upgrading 0.90.x `multi_field` type that has a `geo_point` or `completion` field type as default field would otherwise fail.
Also it make sense to support these field types, because both support specifying the actual values as string.
2014-02-02 07:16:36 +01:00
Simon Willnauer 9cf8251a0d Add RamUsageEstimator#sizeOf(Object) to forbidden APIs
This method can be a performance trap since it traverse the
entire object tree that is referenced by the provided object.
See LUCENE-5373
2014-01-31 21:43:20 +01:00
Simon Willnauer 6e18a8945d Prevent usage of object based RamUsageEstimator in Completion090PostingsFormat 2014-01-31 21:16:38 +01:00
Simon Willnauer 3e3170278e Don't load CompetionTerms if lookupFactory is null
Closes #4970
Closes #4788
2014-01-31 21:16:38 +01:00
Igor Motov 90da268237 Remove support for boost in copy_to field
Currently, boosting on `copy_to` is misleading and does not work as originally specified in #4520. Instead of boosting just the terms from the origin field, it boosts the whole destination field.  If two fields copy_to a third field, one with a boost of 2 and another with a boost of 3, all the terms in the third field end up with a boost of 6.  This was not the intention.

  The alternative: to store the boost in a payload for every term, results in poor performance and inflexibility. Instead, users should either (1) query the common field AND the field that requires boosting, or (2) the multi_match query will soon be able to perform term-centric cross-field matching that will allow per-field boosting at query time (coming in 1.1).
2014-01-31 14:34:01 -05:00
Lee Hinman 5448477c54 Add explanations for all AllocationDeciders
Relates to #4380
Relates to #2483
2014-01-31 10:47:23 -07:00
Luca Cavanna d68d8fbf11 [TEST] clarified skip features usage 2014-01-31 18:03:24 +01:00
Luca Cavanna 33d5a722b3 [TEST] Added ability to skip REST test suite/sections based on their required features
As we have different runners for the REST tests we need a mechanism that allows us to add features to any of them without breaking all others builds.
The idea is to name a feature and temporarily use skip sections that mention the required new features, so that runners that don't support it will skip the test.

Added support for `features` field in skip section.
Added `Features` class that contains a static list of the features supported by the runner. If a feature mentioned in a skip section is not listed here, the test will be skipped.
2014-01-31 16:47:40 +01:00
Martijn van Groningen f60f72f19b Added missing licence header 2014-01-31 10:19:32 +01:00
Martijn van Groningen 0d9197a1d3 The binary field shouldn't be stored by default, because it is already available in the _source. 2014-01-31 10:15:51 +01:00
Martijn van Groningen b9707b395d Added benchmark for scroll searches. 2014-01-31 10:12:32 +01:00
Shay Banon ae3e8beca4 filtered query parses _name incorrectly
fixes #4960
2014-01-30 21:32:52 +01:00
Simon Willnauer dddfe23bed Randomized number of shards / replicas & add mapping for numeric fields 2014-01-30 20:42:31 +01:00
Mark Conlin cd9be4364f Improve boolean handling in snapshot/restore request parsing
Fixes #4949
2014-01-30 13:16:53 -05:00
Martijn van Groningen 7e1eed9814 The forceful no cache behaviour for range filter with now date match expression should only be active if no rounding has been specified for `now` in the date range range expression (for example: `now/d`).
Also the automatic now detection in range filters is overrideable by the `_cache` option.

 Closes #4947
 Relates to #4846
2014-01-30 15:51:33 +01:00
David Pilato abf9a86678 Add version to plugins
Plugin developpers can now add a version number to their es-plugin.properties file:

```properties
plugin=org.elasticsearch.test.integration.nodesinfo.TestPlugin
version=0.0.7-SNAPSHOT
```

Also, for site plugins, it's recommended to add a `es-plugin.properties` file in root site directory with `description` and `version` properties:

```properties
description=This is a description for a dummy test site plugin.
version=0.0.7-BOND-SITE
```

When running Nodes Info API, you will get information on versions:

```sh
$ curl 'http://localhost:9200/_nodes?plugin=true&pretty'
```

```javascript
{
  "ok" : true,
  "cluster_name" : "test-cluster-MacBook-Air-de-David.local",
  "nodes" : {
    "RHMsToxiRcCXwHiS6mEaFw" : {
      "name" : "node2",
      "transport_address" : "inet[/192.168.0.15:9301]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/192.168.0.15:9201]",
      "plugins" : [ {
        "name" : "dummy",
        "version" : "0.0.7-BOND-SITE",
        "description" : "This is a description for a dummy test site plugin.",
        "url" : "/_plugin/dummy/",
        "site" : true,
        "jvm" : false
      } ]
    },
    "IKiUOo-LSCq1Km1GUhBwPg" : {
      "name" : "node3",
      "transport_address" : "inet[/192.168.0.15:9302]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/192.168.0.15:9202]",
      "plugins" : [ {
        "name" : "test-plugin",
        "version" : "0.0.7-SNAPSHOT",
        "description" : "test-plugin description",
        "site" : false,
        "jvm" : true
      } ]
    },
    "H64dcSF2R_GNWh6XRCYZJA" : {
      "name" : "node1",
      "transport_address" : "inet[/192.168.0.15:9300]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/192.168.0.15:9200]",
      "plugins" : [ ]
    },
    "mGEZcYl8Tye0Rm5AACBhPA" : {
      "name" : "node4",
      "transport_address" : "inet[/192.168.0.15:9303]",
      "hostname" : "MacBook-Air-de-David.local",
      "version" : "0.90.0.Beta2-SNAPSHOT",
      "http_address" : "inet[/192.168.0.15:9203]",
      "plugins" : [ {
        "name" : "test-plugin",
        "version" : "0.0.7-SNAPSHOT",
        "description" : "test-plugin description",
        "site" : false,
        "jvm" : true
      }, {
        "name" : "test-no-version-plugin",
        "version" : "NA",
        "description" : "test-no-version-plugin description",
        "site" : false,
        "jvm" : true
      }, {
        "name" : "dummy",
        "version" : "NA",
        "description" : "No description found for dummy.",
        "url" : "/_plugin/dummy/",
        "site" : true,
        "jvm" : false
      } ]
    }
  }
}
```

Relative to #2668.
Closes #2784.
2014-01-30 14:13:46 +01:00
Alexander Reelsen 2c67ba8f1a REST Get Field Mapping API: Fix NPE if field not existent
When fixing #4738, a small issue leaked into the implementation.
The equals check in the RestAction only applied when the master node
returned the rest request, otherwise the object equality would not hold
due to being transferred over the wire and being deserialized into
another object (from and an equality point of view) than the
FieldMappingMetaData.NULL object - this could result in serialization
exceptions as an empty length bytes reference is used in toXContent.
2014-01-30 12:02:52 +01:00
uboness 79d1633dd6 fixed a bug where when executing on a single shard and sorting terms agg based on sub metric agg, the order (asc/desc) is not respected
- fixed tests for terms order

Closes #4951
2014-01-30 06:54:25 +01:00
Lee Hinman 4c06c533e8 Fix javadocs for DeleteResponse.isFound() 2014-01-29 22:29:05 -07:00
uboness d3f2173ef9 fixed date_/histogram aggregation documentation - added documentation for the `min_doc_count` setting
Closes #4944
2014-01-29 20:55:26 +01:00
Igor Motov 2755eecf65 Add throttling to snaphost and restore operations
Closes #4855
2014-01-29 10:33:59 -05:00
David Pilato 0541456b34 scroll REST API should support source parameter
As stated in documentation, we should support `?source=` parameter in `/_search/scroll` REST operations.

This is how to reproduce it:

```sh
curl -XDELETE "http://localhost:9200/test"
curl -XPOST "http://localhost:9200/test/type/1" -d'
{
    "foo": "bar"
}'

# This one works
curl -XPOST "http://localhost:9200/_search/scroll" -d "FAKESCROLLID"

# This one gives: {"error":"Failed to derive xcontent from org.elasticsearch.common.bytes.BytesArray@0"}
curl -XGET "http://localhost:9200/_search/scroll/?source=FAKESCROLLID"
```

Closes #4941.
2014-01-29 14:43:36 +01:00
Clinton Gormley c900ec2152 [TEST] Fixed whitespace in cluster.state/20_filtering.yaml 2014-01-29 13:33:29 +01:00
Igor Motov 2b80c752df Remove waiting for green in CopyToMapperIntegrationTests 2014-01-29 07:29:35 -05:00
Martijn van Groningen c82f27577b Added dedicated thread pool cat api, that can show all thread pool related statistic (size, rejected, queue etc.) for all thread pools (get, search, index etc.)
By default active, rejected and queue thread statistics are included for the index, bulk and search thread pool.
Other thread statistics of other thread pools can be included via the `h` query string parameter.

Closes #4907
2014-01-29 13:25:06 +01:00
Igor Motov a0e381ad3a Improve test stability 2014-01-29 07:14:27 -05:00
uboness 9f04e5fe38 fixed nested example response in docs
Closes #4935
2014-01-29 13:09:12 +01:00
Martijn van Groningen 2de0e60342 Wait for green after indexing doc, so that the mapping update is always after the index operation on replica has completed. (replica shard may needed to catch up) 2014-01-29 12:10:17 +01:00
mrsolo aadcfa7b51 Updated URLs to include 1.x and 1.0 snapshots
This file is used by elasticsearch client jerkin projects
2014-01-28 10:59:50 -08:00
David Pilato d621ab9958 Fix potential NPE when no source and no body
In recent changes, we added missing support for `source` parameter in some REST APIs:

* #4892 : mget
* #4900 : mpercolate
* #4901 : msearch
* #4902 : mtermvectors
* #4903 : percolate

```java
        BytesReference content = null;
        if (request.hasContent()) {
            content = request.content();
        } else {
            String source = request.param("source");
            if (source != null) {
                content = new BytesArray(source);
            }
        }
```

It's definitely better to have:

```java
        BytesReference content = request.content();
        if (!request.hasContent()) {
            String source = request.param("source");
            if (source != null) {
                content = new BytesArray(source);
            }
        }
```

That said, it could be nice to have a single method to manage it for various REST actions.

Closes #4924.
2014-01-28 18:15:27 +01:00
uboness dd389d1cc5 Made all multi-bucket aggs return consistent response format
Closes #4926
2014-01-28 17:46:57 +01:00
Simon Willnauer f9d1552282 Use #ensureSearchable() in GeoHashGridTests 2014-01-28 15:11:49 +01:00
Simon Willnauer 25f0bba30c Use num of actual threads if busiestThreads is larger
We currently use the number of hot threads that we are
interested in as the value for iterating over the actual
hot threads which can lead to AIOOB is the actual number
of threads is less than the given number.

Closes #4927
2014-01-28 14:23:42 +01:00
uboness 2ba2fb193d fixed compilation error on jdk7 2014-01-28 14:12:51 +01:00
Simon Willnauer b21b7ac40f Make test more reliable 2014-01-28 13:32:34 +01:00
uboness fc6bc4c477 cleanup of aggregations api
- add javadocs
- remove Iterable from all multi-bucket aggregations
- all single-bucket aggregations should have getDocCount() and getAggregations()
- all multi-bucket aggregations should have getBuckets() that returns Collection
- every multi-bucket aggregation should have these methods:
 - getBuckets() : Collection
 - getBucketByKey(String) : Bucket
 - getBucketByKey(Number) : Bucket (only for numeric buckets)
 - getBucketByKey(DateTime) : Bucket (only for date buckets)
 - getBucketByKey(GeoPoint) : Bucket (only for geohash_grid)
- every bucket in all multi-bucket aggregations should have these methods:
 - getKey() : String
 - getKeyAsText() : Text
 - getKeyAsNumber() : Number (if the key can be numeric value, eg. range & histograms)
 - getKeyAsGeoPoint() : GeoPoint (in case of the geohash_grid agg)

 Closes #4922
2014-01-28 13:28:04 +01:00
Luca Cavanna b61ca9932a [DOCS] Clarified docs for cluster.routing.allocation.same_shard.host cluster setting
Clarified also javadocs for SameShardAllocationDecider
2014-01-28 12:32:37 +01:00
Martijn van Groningen cb75830c68 Fixed failure for testDeletePercolatorType where the .percolator mapping hasn't propagated to master that was dynamically created via an index call, which made the delete mapping call fail. 2014-01-28 11:19:57 +01:00
Simon Willnauer b5a6a5a6ed Ignore internal errors if JVM can't find the memory pool 2014-01-28 11:11:41 +01:00
Benjamin Devèze 215df2e52f foreground mode is now the default, s/lets/let's/ 2014-01-28 11:07:20 +01:00
Simon Willnauer 91acca7836 Upgrade to Lucene 4.6.1
This upgrade includes a fix for RAM estimation on IndexReader
that allows to expose the amount of used bytes per segment now
as a setting in Elasticsearch. (LUCENE-5373)

Additionally this bugfix release contained a small fix for highlighting
that was already ported to Elasticsearch when reported (LUCENE-5361)

Closes #4897
2014-01-28 10:35:39 +01:00
David Pilato 1da5f3af67 Fix for #4902 2014-01-28 10:12:32 +01:00
Martijn van Groningen 6aaddb03f1 Changed p/c benchmarks to use random generated parent ids, index parent and its children in a non deterministic order and let query values match with multiple child / parent documents. 2014-01-28 10:11:00 +01:00
David Pilato 8706cf610a percolate REST API should support source parameter
As stated in documentation, we should support `?source=` parameter in percolate REST operations.

This is how to reproduce it:

```sh
curl -XDELETE "http://localhost:9200/test"

curl -XPUT "http://localhost:9200/test/.percolator/1" -d'
{
    "query" : {
        "match" : {
            "foo" : "bar"
        }
    }
}'

# This one works
curl -XPOST "http://localhost:9200/test/message/_percolate" -d '{
  "doc" : {
    "foo" : "bar is in foo"
  }
}'

# This one gives: BroadcastShardOperationFailedException[[test][2] ]; nested: PercolateException[failed to percolate]; nested: ElasticsearchIllegalArgumentException[Nothing to percolate];
curl -XGET "http://localhost:9200/test/message/_percolate?source=%7B%22doc%22%3A%7B%22foo%22%3A%22bar%20is%20in%20foo%22%7D%7D"
```

Closes #4903.
2014-01-28 09:43:28 +01:00
David Pilato 71b8876d83 mtermvectors REST API should support source parameter
As stated in documentation, we should support `?source=` parameter in msearch REST operations.

This is how to reproduce it:

```sh
curl -XDELETE "http://localhost:9200/test"

curl -XPOST "http://localhost:9200/test/type/1?refresh" -d'{
    "foo": "bar"
}'

# This one works
curl -XPOST "http://localhost:9200/test/type/_mtermvectors" -d'
{
    "ids" : ["1"]
}'

# This one gives: "ActionRequestValidationException[Validation Failed: 1: multi term vectors: no documents requested;]"
curl -XGET "http://localhost:9200/test/type/_mtermvectors?source=%7B%22ids%22%3A%5B%221%22%5D%7D"
```

Closes #4902.
2014-01-28 09:40:35 +01:00
David Pilato bf3d20eb05 msearch REST API should support source parameter
As stated in documentation, we should support `?source=` parameter in msearch REST operations.

This is how to reproduce it:

```sh
curl -XDELETE "http://localhost:9200/test"

curl -XPOST "http://localhost:9200/test/type/1?refresh" -d'{
    "foo": "bar"
}'

cat requests
{}
{"query" : {"match_all" : {}}}

# This one works
curl -XGET localhost:9200/_msearch --data-binary @requests

# This one gives: {"error":"Failed to derive xcontent from org.elasticsearch.common.bytes.BytesArray@0"}
curl -XGET "http://localhost:9200/test/type/_mget?source=%7B%7D%0A%7B%22query%22%3A%7B%22match_all%22%3A%7B%7D%7D%7D%0A"
```

Closes #4901.
2014-01-28 09:39:44 +01:00
David Pilato 03c02143dd mpercolate REST API should support source parameter
As stated in documentation, we should support `?source=` parameter in mpercolate REST operations.

This is how to reproduce it:

```sh
curl -XDELETE "http://localhost:9200/test"

curl -XPUT "http://localhost:9200/test/.percolator/1" -d'
{
    "query" : {
        "match" : {
            "foo" : "bar"
        }
    }
}'

# This one works
curl -XPOST "http://localhost:9200/test/message/_mpercolate" -d '
{"percolate" : {}}
{"doc" : {"foo" : "bar is in foo"}}
'

# This one gives: BroadcastShardOperationFailedException[[test][2] ]; nested: PercolateException[failed to percolate]; nested: ElasticsearchIllegalArgumentException[Nothing to percolate];
curl -XGET "http://localhost:9200/test/message/_mpercolate?source=%7B%22percolate%22%3A%7B%7D%7D%0A%7B%22doc%22%3A%7B%22foo%22%3A%22bar is in foo%22%7D%7D%0A"
```

Closes #4900.
2014-01-28 09:38:30 +01:00
Alexander Reelsen 5e58f4066e REST API: Consistent get field mapping response
If a get field mapping request is issued, and all but the field can be
found, the response should return an empty JSON object instead of a 404.

Closes #4738
2014-01-28 08:12:53 +01:00
markharwood d9699e02f4 Changed GeoEncodingTests to ensure accuracy always >1mm due to rounding errors with very small numbers 2014-01-27 17:05:19 +00:00
Luca Cavanna 95bf091dd6 [DOCS] unified index settings info and added warmers section in create index docs 2014-01-27 17:10:38 +01:00