Commit Graph

9035 Commits

Author SHA1 Message Date
mikemccand a58d9a1dd0 Core: simultaneous create/delete against same id can cause silently inconsistent replica
If simultaneous create & delete operations arrive against the same id,
it's possible that primary and replica see those operations in
different orders, which may result in replica throwing
DocumentAlreadyExistsException when the primary didn't which would
lead to replica being inconsistent (missing a document that primary
had indexed).

This push fixes the issue, by never throwing DAEE from the replica on
create.

Closes #7146 #7142
2014-08-04 09:14:09 -04:00
javanna 8989d062cd MultiGet & MultiTermVector api: fail when using no routing and an alias to an index that has routing required (for that doc type)
Made sure that the routing required check is performed against the concrete index, added use of aliases to existing routing tests.

Taken the change to unify the failure message as well to this form: routing is required for [" + index + "]/[" + type + "]/[" + id + "]

Closes #7145
2014-08-04 14:19:19 +02:00
Shay Banon 5795e4fbd7 [TEST] better failure message when back location is missing 2014-08-04 12:50:37 +02:00
Martijn van Groningen c8cc59df57 Percolator should cache index field data instances.
Before the index reader used by the percolator didn't allow to register a CoreCloseListener, but now it does, making it safe to cache index field data cache entries.
Creating field data structures is relatively expensive and caching them can save a lot of noise if many queries are evaluated in a percolator call.

Closes #6806
Closes #7081
2014-08-04 10:23:34 +02:00
Alexander Reelsen 3c83c0f9d7 Dependencies: Randomized testing version bump 2014-08-04 09:29:16 +02:00
Britta Weber f84dc23b96 Docs: remove duplicate label 2014-08-04 08:43:44 +02:00
Britta Weber 5706858722 Add parameter to GET for checking if generated fields can be retrieved
Fields of type `token_count`, `murmur3`, `_all` and `_field_names` are generated only when indexing.
If a GET requests accesses the transaction log (because no refresh
between indexing and GET request) then these fields cannot be retrieved at all.
Before the behavior was so:

`_all, _field_names`: The field was siletly ignored
`murmur3, token_count`: `NumberFormatException` because GET tried to parse the values from the source.

In addition, if these fields were not stored, the same behavior occured if the fields were
retrieved with GET after a `refresh()` because here also the source was used to get the fields.

Now, GET accepts a parameter `ignore_errors_on_generated_fields` which has
the following effect:
- Throw exception with meaningful error message explaining the problem if set to false (default)
- Ignore the field if set to true
- Always ignore the field if it was not set to stored

This changes the behavior for `_all` and `_field_names` as now an Exception is thrown if a user
tries to GET them before a `refresh()`.

closes #6676
closes #6973
2014-08-04 08:15:34 +02:00
Britta Weber a3cefd919e significant terms: add google normalized distance, add chi square
closes #6858
2014-08-04 08:15:26 +02:00
uboness 5f0719fd50 Added CliToolTestCase's sub classes to the test jar 2014-08-03 22:18:33 +02:00
uboness 45714b1977 Added CliToolTestCase to the test jar 2014-08-03 21:10:25 +02:00
uboness b667bcdedf fixed platform independent line separator in CliToolTests 2014-08-02 21:02:22 +02:00
Shay Banon 95762e8126 Support "default" for tcpNoDelay and tcpKeepAlive
Allow to set the value default to network.tcp.no_delay and network.tcp.keep_alive so they won't be set at all, since on solaris, setting tcpNoDelay can actually cause failure
relates to #7115
2014-08-02 17:32:41 +02:00
uboness 5ccc7beaf4 Added a cli infrastructure
CliTool is a base class for command-line interface tools (such as the plugin manager and potentially others). It supports the following:
  - single or multi command tool
  - help printing infrastructure (based on help files)
  - consistent mechanism of parsing arguments (based on commons-cli lib)
  - separation of argument parsing and command execution (for easier unit testing)
  - terminal abstraction (will use System.console() when available)
2014-08-02 17:16:27 +02:00
Shay Banon 2d31349ab0 Fix missing break statement causing reroute serialization failure
closes #7135
2014-08-02 16:52:52 +02:00
Areek Zillur b81b240924 [Fix] CompletionMapper throws misleading error on null value
closes #6399
2014-08-01 15:25:05 -04:00
uboness 3c9c9f33e2 Aggregations Added Filters aggregation
A multi-bucket aggregation where multiple filters can be defined (each filter defines a bucket). The buckets will collect all the documents that match their associated filter.

This aggregation can be very useful when one wants to compare analytics between different criterias. It can also be accomplished using multiple definitions of the single filter aggregation, but here, the user will only need to define the sub-aggregations only once.

Closes #6118
2014-08-01 16:01:08 +01:00
Adrien Grand d9d5b35be9 Sort: Make `ignore_unmapped` work for cross-index queries.
Close #2255
2014-08-01 15:30:17 +02:00
Lee Hinman db7b6097cc [TEST] check breaker reset after parent trip instead of trip count 2014-08-01 15:18:38 +02:00
javanna d5b6de3295 Removed support for aliases as part of index settings
Now that we have explicit support for aliases when creating indices and as part of index templates, we may remove support for aliases (only names) as part of index settings. This is partially breaking as the following calls:

curl -XPUT localhost:9200/index -d '{
  "settings" : {
    "aliases" : [ "alias1"]
  }
}

and

curl -XPUT localhost:9200/index -d '{
  "settings" : {
    "index.aliases" : [ "alias1"]
  }
}

were previously supported and will need to be replaced with

curl -XPUT localhost:9200/index -d '{
  "aliases" : {
    "alias1": {}
  }
}

Closes #5545
2014-08-01 13:49:43 +02:00
Skye Book 0040ed4f6f Function score query: Add missing whitespace when throwing exception
DecayFunctionParser throws a parse exception with a string containing "scaleand origin", this fixes the spacing issue.
2014-08-01 12:43:26 +02:00
Stefan Antoni 8e862f15c1 [DOCS] fixed small typo in percolate.asciidoc 2014-08-01 12:38:35 +02:00
Britta Weber d6a18ab2ba Docs: add 1.4.0 label to many to many geo distance sort 2014-08-01 12:30:08 +02:00
Kurt Hurtado 66560acebb Update fielddata-fields.asciidoc 2014-08-01 09:20:19 +02:00
Adrien Grand a9d5c03924 Aggregations: Fix infinite loop in the histogram reduce logic.
The histogram reduce method can run into an infinite loop if the
Rounding.nextRoundingValue value is buggy, which happened to be the case for
DayTimeZoneRoundingFloor.

DayTimeZoneRoundingFloor is fixed, and the histogram reduce method has been
changed to fail instead of running into an infinite loop in case of a buffy
nextRoundingValue impl.

Close #6965
2014-08-01 08:58:10 +02:00
Adrien Grand 99b32901d2 Mappings: Fix `copy_to` behavior on nested documents.
Today, `copy_to` always copies a field to the current document, which is often
wrong in the case of nested documents. For example, if you have a nested field
called `n` which has a sub-field `n.source` whose content should be copied to
`target`, then the latter field should be created in the root document instead
of the nested one, since it doesn't have `n.` as a prefix. On the contrary, if
you configure the destination field to be `n.target`, then it should go to the
nested document.

Close #6701
2014-08-01 08:57:33 +02:00
Areek Zillur 1d581e6286 Search Exists API: Checks if any matching documents exist for a given query
Implements a new Exists API allowing users to do fast exists check on any matched documents for a given query.
This API should be faster then using the Count API as it will:
 - early terminate the search execution once any document is found to exist
 - return the response as soon as the first shard reports matched documents

closes #6995
2014-07-31 15:42:30 -04:00
David Pilato 85eb0ea0e7 Generate timestamp when path is null
Index process fails when having `_timestamp` enabled and `path` option is set.
It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message.

Reproduction:

```
DELETE test
PUT  test
{
    "mappings": {
        "test": {
            "_timestamp" : {
                "enabled" : "yes",
                "path" : "post_date"
            }
        }
    }
}
PUT test/test/1
{
  "foo": "bar"
}
```

You can define a default value for when timestamp is not provided
within the index request or in the `_source` document.

By default, the default value is `now` which means the date the document was processed by the indexing chain.

You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory:

```
{
    "tweet" : {
        "_timestamp" : {
            "enabled" : true,
            "default" : null
        }
    }
}
```

If you don't provide any timestamp value, indexation will fail.

You can also set the default value to any date respecting timestamp format:

```
{
    "tweet" : {
        "_timestamp" : {
            "enabled" : true,
            "format" : "YYYY-MM-dd",
            "default" : "1970-01-01"
        }
    }
}
```

If you don't provide any timestamp value, indexation will fail.

Closes #4718.
Closes #7036.
2014-07-31 19:48:22 +02:00
Brian Murphy 6b39aa615e [TEST] Reduce number of updates and wait after 1st NoNodeException. 2014-07-31 18:01:30 +01:00
Britta Weber fe86c8bc88 _geo_distance sort: allow many to many geo point distance
Add computation of disyance to many geo points. Example request:

```
{
  "sort": [
    {
      "_geo_distance": {
        "location": [
          {
            "lat":1.2,
            "lon":3
          },
          {
             "lat":1.2,
            "lon":3
          }
        ],
        "order": "desc",
        "unit": "km",
        "sort_mode": "max"
      }
    }
  ]
}
```

closes #3926
2014-07-31 17:33:45 +02:00
Shay Banon 739e977aa7 cluster block with auto create index bulk action can cause bulk execution to not return
when there is a cluster block (like no master yet discovered), the bulk action doesn't properly catch the exception of inner execute to notify the listener, causing the bulk operation to hang
closes #7086
2014-07-31 16:57:49 +02:00
Martijn van Groningen 5ea6267883 [TEST] All shards should be allocated before snapshotting & restoring 2014-07-31 16:22:24 +02:00
Martijn van Groningen 1c59ae1b99 [TEST] Increased logging and make use of prepareCreate and assertAcked 2014-07-31 16:22:24 +02:00
Simon Willnauer 196b56d47a [BUILD] Add profile that skips all validations
This commit adds a profile that skips all validation ie.
 - nocommit / tabs checking
 - forbidden API checks
 - license headers

It's not active by default but can easily be activated with
  `mvn -Pdev` or in the `~/.m2/settings.xml`

for reference see:
  http://maven.apache.org/guides/introduction/introduction-to-profiles.html
2014-07-31 15:22:11 +02:00
Clinton Gormley 4b0a89d4fb Update translog.asciidoc
Documented `index.gateway.local.sync`
2014-07-31 14:06:24 +02:00
Clinton Gormley 36e1c7928c Rewrote post-filter.asciidoc
Closes #5166
2014-07-31 12:56:11 +02:00
Shay Banon 521f8b28b5 [TEST] Concurrent percolation more randomized + safe use of rand 2014-07-31 11:56:36 +02:00
Nik Everett 34426eb8c2 Docs: Fix syntax on lang-analyzer
Some of the language analyzer documentation contained invalid json.

Closes #7098
2014-07-30 20:17:27 +02:00
Spencer Alger a848b658e8 [TEST] Rest API: cat.count regexp matchers now explicitely test for newline 2014-07-30 10:19:24 -07:00
Alex Ksikes e3b3b6c055 Term Vectors API: adds support for wildcards in selected fields
This could useful to generate all term vectors or a chosen set of them.

Closes #7061
2014-07-30 17:44:37 +02:00
Alexander Reelsen 2077d4be48 Packaging: Dont remove ancestors on deb removal
The used -p option could result in accidentally deleting more directories
than /var/lib/elasticsearch - so this option was removed

Note: This only happens if the directories are empty, but still isnt needed.

Relates #5770
2014-07-30 15:01:06 +02:00
Lee Hinman f9f8459c79 [TEST] randomize byte size and thread count in breaker tests 2014-07-30 12:02:16 +02:00
mikemccand dbcc4e9255 Test: finish all merges before test returns to side step 'Delete Index failed - not acked' failure 2014-07-29 11:54:45 -04:00
Simon Willnauer dc696e6fc8 [TEST] Pick random version from backwards directory
The upgrade tests required to specify the lower and the upper version
that it should run against. This commit adds support for random picks
if either the lower or the upper or both are not specified.
2014-07-29 14:29:56 +02:00
Simon Willnauer 873b491f4e [TEST] Make default flush opertion blocking 2014-07-29 13:48:50 +02:00
Simon Willnauer af5b2ae28a [TEST] wait for yellow after index was created in BWC tests 2014-07-29 13:48:42 +02:00
Adrien Grand d67e013e08 [TESTS] Don't create an unclosed threadpool in BigArraysTests. 2014-07-29 13:26:53 +02:00
Simon Willnauer 7343ae1911 [TEST] Add basic upgrade test script
This commit adds a basic full cluster restart upgrade script that tests
upgrades of elasticsearch across major version. In contrast to the BWC
tests we are running using the java test framework this test uses a python
script as well as the REST APIs to ensure upgrades across major version work and
indices are compatible.

The upgrade test starts 2 or more nodes of an old elasticserach version, indexes
a random number of documents into the running nodes and executes a full cluster restart.
After the nodes are recovered a small set of basic checks are executed to ensure all
documents are still searchable and field data can be loaded etc.

Closes #7074
2014-07-29 13:00:59 +02:00
Lee Hinman 5aa2d0cf61 Add support for the `_name` parameter to the simple_query_string query 2014-07-29 12:41:41 +02:00
gabriel-tessier eaac8141cc Docs: Fix typo in scripting.asciidoc
Replace the mvel by groovy in the forgotten place.
I add the previous change in this one.
Sorry for the spam!

Closes #7071
2014-07-29 12:30:09 +02:00
gabriel-tessier c2c2190d27 Docs: Fix typo in scripting.asciidoc
Closes #7070
2014-07-29 12:28:41 +02:00