Commit Graph

8323 Commits

Author SHA1 Message Date
mikemccand 4f7792e64b Tie-break suggestions from phrase suggester by term
If the score for two suggestions is the same, we now tie break by term; earlier terms (aaa) sort before later terms (zzz).

Closes #5978
2014-05-18 16:45:37 -04:00
Simon Willnauer f79b28375d Add missing coming tag
Relates to #6188
Relates to #5539
2014-05-18 10:54:17 +02:00
Simon Willnauer dab4596b13 Use default forceAnalyzeQueryString if no query builder is present
In the single field case no query builder is selected which causes NPE
when the query has only a numeric field.

Closes #6215
2014-05-18 10:20:31 +02:00
Boaz Leskes 1e5138889e Translog: remove unneeded Versions.readVersion & Versions.writeVersion
These calls were introduced in pr #6149 as a backward compatibility layer for the previous value of `Versions.MATCH_ANY`. This is not needed as the translog never contains these values. On top of that, the calls are not effective as the stream the translog used is effectively not versioned (versioining is done on an item by item basis)
2014-05-18 09:45:00 +02:00
Boaz Leskes 682acfcacd DeleteRequest.version was not initialized to `Versions.MATCH_ANY` 2014-05-18 09:45:00 +02:00
Simon Willnauer c7db8843b3 [TEST] Stabelize BenchmarkIntegrationTest#testAbortBenchmark 2014-05-17 23:33:49 +02:00
Alex Ksikes db991dc3a4 More Like This Query: Added searching for multiple items.
The syntax to specify one or more items is the same as for the Multi GET API.
If only one document is specified, the results returned are the same as when
using the More Like This API.

Relates #4075 Closes #5857
2014-05-17 19:14:56 +02:00
Igor Motov a3581959d7 [TESTS] Ignore SnapshotMissingException in snapshotWithStuckNodeTest
The retry mechanism in the transport layer might cause the delete snapshot request to be executed twice if the cluster master is closed while the request is executed. First time delete snapshot request is getting successfully executed on the old master and then it is retried on the newly elected master. When the new master tries to delete the snapshot - the snapshot no longer exists (since it was successfully deleted by the old master) and SnapshotMissingException is returned.
2014-05-17 11:18:11 -04:00
Igor Motov c20713530d Switch to shared thread pool for all snapshot repositories
Closes #6181
2014-05-16 19:03:15 -04:00
Igor Motov 7f5befd95e Add Partial snapshot state
Currently even if some shards of the snapshot are not snapshotted successfully, the snapshot is still marked as "SUCCESS". Users may miss the fact the there are shard failures present in the snapshot and think that snapshot was completed. This change adds a new snapshot state "PARTIAL" that provides a quick indication that the snapshot was only partially successful.

Closes #5792
2014-05-16 18:26:56 -04:00
Boaz Leskes 9f10547f4b Allow 0 as a valid external version
Until now all version types have officially required the version to be a positive long number. Despite of this has being documented, ES versions <=1.0 did not enforce it when using the `external` version type. As a result people have succesfully indexed documents with 0 as a version. In 1.1. we introduced validation checks on incoming version values and causing indexing request to fail if the version was set to 0. While this is strictly speaking OK, we effectively have a situation where data already indexed does not match the version invariant.

To be lenient and adhere to spirit of our data backward compatibility policy, we have decided to allow 0 as a valid external version type. This is somewhat complicated as 0 is also the internal value of `MATCH_ANY`, which indicates requests should succeed regardles off the current doc version. To keep things simple, this commit changes the internal value of `MATCH_ANY` to `-3` for all version types.

Since we're doing this in a minor release (and because versions are stored in the transaction log), the default `internal` version type still accepts 0 as a `MATCH_ANY` value. This is not a problem for other version types as `MATCH_ANY` doesn't make sense in that context.

Closes #5662
2014-05-16 22:10:16 +02:00
Clinton Gormley f510e25306 [DOCS] Renamed the "cat" chapters to be more searchable 2014-05-16 21:43:35 +02:00
Simon Willnauer bf22df7fd0 Remove SoftReferences from StreamInput/StreamOutput
We try to reuse character arrays and UTF8 writers with softreferences.
SoftReferences have negative impact on GC and should be avoided in
general. Yet in this case it can simply replaced with a per-stream
Bytes/CharsRef that is thread local and has the same lifetime as the
stream.
2014-05-16 20:58:42 +02:00
Simon Willnauer 11a3201a09 Use EnumSet rather than static mutable arrays
ClusterBlockLevel uses arrays but should use EnumSets instead
2014-05-16 20:54:01 +02:00
Simon Willnauer d65e9e9bea Add some finals where appropriate 2014-05-16 20:54:01 +02:00
Simon Willnauer c561900512 Use UTF-8 as string encoding 2014-05-16 20:54:01 +02:00
Clinton Gormley bfeb5a7120 added install instruction with apt
Closes #6206
2014-05-16 19:07:05 +02:00
David Pilato 0dbc83e7b0 [TEST] Do not filter gz files 2014-05-16 15:23:09 +02:00
Simon Willnauer d806b567e4 Remove dead code 2014-05-16 15:08:56 +02:00
Simon Willnauer eef505ed51 RecoveryID should not be a per JVM but per Node
Today the RecovyerID is taken from a static atomic long which
is essentially a per JVM ID. We run the tests within the same
JVM and that means we don't really simulate what happens in
production environments. Instead we should use a per node generated
ID.
2014-05-16 14:59:32 +02:00
Simon Willnauer 9a9cc0b8e4 Add simple example to XContentParser how to obtain an instance of it 2014-05-16 14:55:22 +02:00
David Pilato bd871f96c2 Check that a plugin is Lucene compatible with the current running node using `lucene` property in `es-plugin.properties` file.
* If plugin does not provide `lucene` property, we consider that the plugin is compatible.
* If plugin provides `lucene` property, we try to load related Enum org.apache.lucene.util.Version. If this fails, it means that the node is too "old" comparing to the Lucene version the plugin was built for.
* We compare then two first digits of current node lucene version against two first digits of plugin Lucene version. If not equal, it means that the plugin is too "old" for the current node.

Plugin developers who wants to launch plugin check only have to add a `lucene` property in `es-plugin.properties` file. If you are using maven to build your plugin, you can do it like this:

In `pom.xml`:

```xml
    <properties>
        <lucene.version>4.6.0</lucene.version>
    </properties>

    <build>
        <resources>
            <resource>
                <directory>src/main/resources</directory>
                <filtering>true</filtering>
            </resource>
        </resources>
    </build>
```

In `es-plugin.properties`, add:

```properties
lucene=${lucene.version}
```

BTW, if you don't already have it, you can add the plugin version as well:

```properties
version=${project.version}
```

You can disable that check using `plugins.check_lucene: false`.
2014-05-16 13:41:20 +02:00
Simon Willnauer 094908ac7f Randomize CMS settings in index template
This commit adds randomization for:
 * `index.merge.scheduler.max_thread_count`
 * `index.merge.scheduler.max_merge_count`

This commit also moves to use
EsExecutors#boundedNumberOfProcessors(Settings) to default
configure the default `max_thread_count` for better reproducibility

Closes #6194
2014-05-15 23:16:45 +02:00
Simon Willnauer 13f37b3800 Shade mustache into org.elasticsearch.common package
Previously we shared the jar but never rewrote the packages such
that the shading had no effect.

Closes #6192
2014-05-15 21:21:36 +02:00
javanna 7548b2edb7 Unified MetaData#concreteIndices methods into a single method that accepts indices (or aliases) and indices options
Added new internal flag to IndicesOptions that tells whether aliases can be resolved to multiple indices or not.

Cut over to new metaData#concreteIndices(IndicesOptions, String...) for all the api previously using MetaData#concreteIndices(String[], IndicesOptions) and removed old method, deprecation is not needed as it doesn't break client code.

Introduced constants for flags in IndicesOptions for more readability

Renamed MetaData#concreteIndex to concreteSingleIndex, left method as a shortcut although it calls the common concreteIndices that accepts IndicesOptions and multipleIndices
2014-05-15 20:53:05 +02:00
Boaz Leskes 1f28cd0ba8 When sending shard start/failed message due to a cluster state change, use the master indicated in the new state rather than current
This commit also adds extra protection in other cases against a master node being de-elected and thus being null.

Closes #6189
2014-05-15 18:42:26 +02:00
Boaz Leskes 84593f0d7c Added meta data and routing version to cluster state's pretty print 2014-05-15 15:55:11 +02:00
Boaz Leskes dc07ece790 Added some debug logs to the recovery process 2014-05-15 15:37:30 +02:00
Simon Willnauer e47de1f809 [TEST] Randomize number of available processors
We configure the threadpools according to the number of processors which is
different on every machine. Yet, we had some test failures related to this
and #6174 that only happened reproducibly on a node with 1 available processor.
This commit does:
  * sometimes randomize the number of available processors
  * if we don't randomize we should set the actual number of available processors
    in the settings on the test node
  * always print out the num of processors when a test fails to make sure we can
    reproduce the thread pool settings with the reproduce info line

Closes #6176
2014-05-15 12:24:53 +02:00
Simon Willnauer 53bfe44e19 Fix debug logging message for put template action 2014-05-15 11:13:30 +02:00
Andrew Selden fc0bed5236 Fix bug for BENCH thread pool size == 1
On small hardware, the BENCH thread pool can be set to size 1. This is
problematic as it means that while a benchmark is active, there are no
threads available to service administrative tasks such as listing and
aborting. This change fixes that by executing list and abort operations
on the GENERIC thread pool.

Closes #6174
2014-05-14 10:40:39 -07:00
Simon Willnauer 2c1c5c163f [TEST] Ensure all benchmarks are aborted on failure and latches are counted down 2014-05-14 16:40:34 +02:00
Clinton Gormley 8f0991c14f [DOCS] Rewrote the memory settings section on the configuration page 2014-05-14 16:02:59 +02:00
Simon Willnauer fc2ab0909e [TEST] Remove busy waiting from BenchmarkIntegrationTest
I think Chuck Norris is required to fix this at this point until we have an API
that can for instance pause a Benchmark. We basically wait for a query to be executed
and that query syncs on a latch with the test in a script :)

This commit also adds some more testing for benchmarks that run into errors.
2014-05-14 14:40:27 +02:00
David Pilato e0a95d9c19 Allow sorting on nested sub generated field
When you have a nested document and want to sort on its fields, it's perfectly doable on regular fields but not on "generated" sub fields.

Here is a SENSE recreation:

```
DELETE /tmp

PUT /tmp

PUT /tmp/doc/_mapping
{
  "properties": {
    "flat": {
      "type": "string",
      "index": "analyzed",
      "fields": {
        "sub": {
          "type": "string",
          "index": "not_analyzed"
        }
      }
    },
    "nested": {
      "type": "nested",
      "properties": {
        "foo": {
          "type": "string",
          "index": "analyzed",
          "fields": {
            "sub": {
              "type": "string",
              "index": "not_analyzed"
            }
          }
        }
      }
    }
  }
}

PUT /tmp/doc/1
{
  "flat":"bar",
  "nested":{
    "foo":"bar"
  }
}
```

When sorting on `flat.sub` sub field, everything is fine:

```
GET /tmp/doc/_search
{
  "sort": [
    {
      "flat.sub": {
        "order": "desc"
      }
    }
  ]
}

```

When sorting on `nested` field, everything is fine:

```
GET /tmp/doc/_search
{
  "sort": [
    {
      "nested.foo": {
        "order": "desc"
      }
    }
  ]
}

```

But when sorting on `nested.sub` field, sorting is incorrect:

```
GET /tmp/doc/_search
{
  "sort": [
    {
      "nested.foo.sub": {
        "order": "desc"
      }
    }
  ]
}

Closes #6150.
2014-05-14 14:13:44 +02:00
Britta Weber 08e57890f8 use shard_min_doc_count also in TermsAggregation
This was discussed in issue #6041 and #5998 .

closes #6143
2014-05-14 14:10:04 +02:00
Britta Weber d4a0eb818e refactor: make requiredSize, shardSize, minDocCount and shardMinDocCount a single parameter
Every class using these parameters has their own member where these four
are stored. This clutters the code. Because they mostly needed together
it might make sense to group them.
2014-05-14 14:10:02 +02:00
Britta Weber 8e3bcb5e2f refactor: unify terms and significant_terms parsing
Both need the requiredSize, shardSize, minDocCount and shardMinDocCount.
Parsing should not be duplicated.
2014-05-14 14:09:59 +02:00
Gaurav Arora e041b5992c Fix typo in docs 2014-05-14 12:36:35 +02:00
Clinton Gormley 31b876b0e2 Renamed "tire" to "retire" in community clients
Closes #5673
2014-05-14 12:19:45 +02:00
Clinton Gormley ff12585fea Improved wording in search-type.asciidoc
Closes #5951
2014-05-14 12:15:48 +02:00
Clinton Gormley 98dfc26fb1 Updated groovy docs to point to the new groovy repo 2014-05-14 12:08:02 +02:00
Clinton Gormley 2912e1cce3 Fixed typo in getting-started.asciidoc
Closes #6064
2014-05-14 12:03:12 +02:00
ericheiker 0eb7b5024d Update match-query.asciidoc 2014-05-14 11:59:12 +02:00
Mahesh Paolini-Subramanya c93e7f26c5 Type is the 'doc-type', not the word 'type' 2014-05-14 11:50:08 +02:00
Adrien Grand bfcebbb957 [TESTS] Fix test bug in PagedBytesReferenceTest. 2014-05-14 10:09:11 +02:00
Adrien Grand 265b386fa7 [TESTS] Fix test bugs for parent/child queries.
If you got a bad seed and tests.nightly=true, these tests would either call
Random#nextInt on `0` or trigger infinite loops.
2014-05-14 09:35:45 +02:00
Boaz Leskes 9daa72941a [Test] increase ping timeout to 400ms in MinimumMasterNodesTests.dynamicUpdateMinimumMasterNodes 2014-05-14 09:28:44 +02:00
Boaz Leskes fb501b22e1 [Test] SimpleNodesInfoTests.testNodesInfos didn't wait for cluster to form properly 2014-05-14 08:59:48 +02:00
Adrien Grand 9425472f61 [BUILD] Shade t-digest. 2014-05-14 00:42:48 +02:00