Commit Graph

21059 Commits

Author SHA1 Message Date
javanna 0c70a9d5bd fix bug introduced with refactoring of DiscoveryNode constructors
Transport client was replacing the address of the nodes connecting to with the ones received from the liveness api rather keeping the original listed nodes. Written a test for that.
2016-03-29 18:36:09 +02:00
Igor Motov c356b30cff Update task management docs to reflect the latest changes in the interface
Brings docs in line with new list task syntax and adds task cancellation API docs.
2016-03-29 12:26:37 -04:00
Boaz Leskes 48b4f086e0 Replication operation that try to perform the primary phase on a replica should be retried
In extreme cases a local primary shard can be replaced with a replica while a replication request is in flight and the primary action is applied to the shard (via `acquirePrimaryOperationLock()).  #17044 changed the exception used in that method to something that isn't recognized as `TransportActions.isShardNotAvailableException`, causing the operation to fail immediately instead of retrying. This commit fixes this by check the primary flag before
acquiring the lock. This is safe to do as an IndexShard will never be demoted once a primary.

Closes #17358
2016-03-29 17:21:14 +02:00
Tal Levy 833fc8420f split long line in ConvertProcessorTests 2016-03-29 08:19:15 -07:00
Tal Levy 16e888fac3 Merge pull request #17260 from talevy/fix-regex-exceptions
Handle regex parsing errors in Gsub and Grok Processors
2016-03-29 08:12:26 -07:00
Tal Levy 9ac3887139 Merge pull request #17263 from talevy/auto-convert
add  type conversion support to ConvertProcessor
2016-03-29 07:57:57 -07:00
Tal Levy 2064fe3985 add type conversion support to ConvertProcessor 2016-03-29 07:56:53 -07:00
Martijn van Groningen 9d37f459b5 percolator: Make explain use the two phase iterator
So that we don't eveluate percolator queries that don't match.

Closes #17314
2016-03-29 16:26:31 +02:00
Martijn van Groningen 60793a848e test: make sure we don't flush during indexing the percolator queries 2016-03-29 16:24:49 +02:00
Clinton Gormley 798e4281fa Added experimental annotation to the update-by-query and reindex docs 2016-03-29 15:06:27 +02:00
Clinton Gormley 3087d2b882 Fixed bad YAML in reindex REST test: 50_routing.yaml 2016-03-29 15:03:09 +02:00
Clinton Gormley 52daed0732 Update-by-query rest tests: fixed bad yaml and deleted a client-dependent test 2016-03-29 14:58:29 +02:00
Isabel Drost-Fromm 778a447ef0 Do not fail if providing coordinates for same field name 2016-03-29 14:38:48 +02:00
Colin Goodheart-Smithe ff3fd99074 Prevents exception being raised when ordering by an aggregation which wasn't collected
If a terms aggregation was ordered by a metric nested in a single bucket aggregator which did not collect any documents (e.g. a filters aggregation which did not match in that term bucket) an ArrayOutOfBoundsException would be thrown when the ordering code tried to retrieve the value for the metric. This fix fixes all numeric metric aggregators so they return their default value when a bucket ordinal is requested which was not collected.

Closes #17225
2016-03-29 13:28:03 +01:00
javanna 8fc9dbbb99 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 14:27:04 +02:00
javanna f9a5e1a03a wrap EnumSet into unmodifiableSet directly, plus minor changes 2016-03-29 14:26:09 +02:00
Clinton Gormley 5f24581de3 The reindex body is now required, which changes the exception thrown by the REST test 2016-03-29 14:09:59 +02:00
Clinton Gormley 978b24327e Docs: Included Nodes Task API and tidied reindex/update-by-query 2016-03-29 13:51:11 +02:00
Isabel Drost-Fromm 46e4ea9d16 Refine dealing with removed reverse sort option.
For geo distance sort parsing: Disallow anything but
VALUE_STRING as geo hash, disallow resetting field
name for geo fields.

Also make error message for wrong lat/lon values more
verbose by including the affected field name.
2016-03-29 13:16:33 +02:00
Clinton Gormley b87beeb05f Rename update-by-query REST tests to update_by_query 2016-03-29 13:13:49 +02:00
javanna 36f446759f fix silly serialization mistake in DiscoveryNode 2016-03-29 12:44:31 +02:00
javanna f7becf1f53 Move Set<Role> to EnumSet<Role> in DiscoveryNode 2016-03-29 12:33:14 +02:00
Clinton Gormley 647437ce56 REST: The body is required in the reindex API 2016-03-29 11:45:20 +02:00
Clinton Gormley 579d976e90 The source parameter should not be defined in the delete-by-query REST spec 2016-03-29 11:45:20 +02:00
Clinton Gormley 97606850e8 Renamed update-by-query REST spec to update_by_query 2016-03-29 11:45:20 +02:00
Adrien Grand cb31e591f1 Fix test bug in TypeQueryBuilderTests. 2016-03-29 11:43:27 +02:00
javanna 5794455912 Use regular Map for attributes in DiscoveryNode, get rid of attributes getter in favour of regular getter 2016-03-29 11:42:44 +02:00
Isabel Drost-Fromm 5a913fcc69 Fix build errors after last merge. 2016-03-29 11:26:41 +02:00
javanna 30ac3dcf01 use ordinary incrementing loops in ClusterStatsNodes 2016-03-29 11:05:58 +02:00
Isabel Drost-Fromm 407e2cdcf9 Merge branch 'master' into deprecation/sort-option-reverse-removal
Conflicts:
	core/src/main/java/org/elasticsearch/search/sort/ScoreSortBuilder.java
	core/src/test/java/org/elasticsearch/search/sort/FieldSortBuilderTests.java
2016-03-29 11:04:02 +02:00
javanna 66a8e4efeb less streams more loops 2016-03-29 11:01:49 +02:00
javanna 8034e13365 adapt cluster allocation explain to DiscoveryNode changes 2016-03-29 10:53:42 +02:00
javanna de5cbda8e7 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 10:48:47 +02:00
Adrien Grand c7bdfb1126 Add comment why it is safe to check the number of nested fields in MapperService.merge. 2016-03-29 10:02:29 +02:00
Adrien Grand 0eedc784fe Automatically add a sub keyword field to string dynamic mappings. #17188
If you add a string field to a document, it will have the following default
mapping:

```
{
  "type": "text",
  "fields": {
    "keyword": {
      "type": "keyword",
      "ignore_above": 256
    }
  }
}
```
2016-03-29 09:49:16 +02:00
Adrien Grand 4bd27bc2a0 Type filters should not have a performance impact when there is a single type. #17350
Today, if you call /index/type/_search instead of /index/_search, elasticsearch
will automatically insert a type filter to only match documents of the given
type. This commit uses a new TypeQuery instead of a TermQuery for this filter,
which rewrites to a MatchAllDocsQuery when all documents of a shard match the
filtered type. This is helpful because BooleanQuery has a special rewrite rule
to remove MatchAllDocsQuery as FILTER clauses. So for instance if your query is
`+body:"quick fox" #_type:my_type`, it will be rewritten to
`+body:"quick fox" #*:*` which is then rewritten to `body:"quick fox"`.
2016-03-29 08:52:49 +02:00
Lee Hinman 3e9f8a4c59 Merge pull request #17360 from qwerty4030/master
Fixed broken links in CONTRIBUTING.md
2016-03-28 17:27:18 -06:00
Lee Hinman c63cb21745 Merge remote-tracking branch 'dakrone/allocation-explain' 2016-03-28 16:06:49 -06:00
Lee Hinman 80ab366de4 Add API to explain why a shard is or isn't assigned
This adds a new `/_cluster/allocation/explain` API that explains why a
shard can or cannot be allocated to nodes in the cluster. Additionally,
it will show where the master *desires* to put the shard, according to
the `ShardsAllocator`.

It looks like this:

```
GET /_cluster/allocation/explain?pretty
{
  "index": "only-foo",
  "shard": 0,
  "primary": false
}
```

Though, you can optionally send an empty body, which means "explain the
allocation for the first unassigned shard you find".

The output when a shard is unassigned looks like this:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : false
  },
  "assigned" : false,
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2016-03-22T20:04:23.620Z"
  },
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 0.06666675,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "NO",
      "weight" : -1.3833332,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 2.3166666,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

And when the shard *is* assigned, the output looks like:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : true
  },
  "assigned" : true,
  "assigned_node_id" : "Qc6VL8c5RWaw1qXZ0Rg57g",
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 1.4499999,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "CURRENTLY_ASSIGNED",
      "weight" : 0.0,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 3.6999998,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

Only "NO" decisions are returned by default, but all decisions can be
shown by specifying the `?include_yes_decisions=true` parameter in the
request.

Resolves #14593
2016-03-28 15:21:02 -06:00
spalger ce44bbfadf [docs] clarify where discovery.zen.minimum_master_node is required
https://github.com/elastic/elasticsearch/pull/17288 added a check to enforce that the `discovery.zen.minimum_master_nodes` configuration is set when nodes have the `host`, `port`, or `bind_host` set in either `transport` or general `network` configuration sections. This was documented incorrectly as "nodes that are bound to a non-loopback interface", which lead to confusion as I set `network.host: "localhost"` and the check was still failing.

This change updates the docs to detail the actual check. I think it also highlights how complex the check is and the need for a simpler solution.
2016-03-28 12:53:40 -07:00
Igor Motov 8a5c19b25f Handle RejectedExecution gracefully in TransportService during shutdown
Today we might run into a rejected execution exception when we shutdown the node while handling a transport exception. The exception is run in a seperate thread but that thread might not be able to execute due to the shutdown. Today we barf and fill the logs with large exception. This commit catches this exception and logs it as debug logging instead.

Extends changes made in 8652cd8
2016-03-28 15:12:50 -04:00
Areek Zillur a763599df7 Merge pull request #17367 from areek/cleanup/remove_dead_code
Remove dead code
2016-03-28 14:59:42 -04:00
Areek Zillur 976b8d2cee Remove dead code
This commit removes dead DeleteByQuery class from core.
2016-03-28 14:48:40 -04:00
Nik Everett 0e6141e675 Replace is_true: took with took >= 0
This prevents tests from failing on machines that can finish the request
less than half a millisecond.
2016-03-28 13:03:48 -04:00
qwerty4030 9008eadd0d Fixed broken links in CONTRIBUTING.md 2016-03-27 14:57:51 -07:00
Nik Everett 9402251eaf Remove PROTOTYPE from ShapeBuilders
Also cuts lots of tests over to expectThrows and fixes DistanceUnit's
serialization.
2016-03-26 13:58:33 -04:00
Igor Motov ee49081bc7 Take filterNodeIds into consideration while sending tasks actions requests to nodes
This commit fixes a bug that was causing the result of TransportTasksAction#filterNodeIds to be ignored and as a result the tasks actions were executed on all nodes.
2016-03-26 13:15:39 -04:00
Martijn van Groningen 444641ac55 test: cleanup imports and method rename 2016-03-26 15:13:56 +01:00
Nik Everett 8a89482555 Remove PROTOTYPE from SortBuilders 2016-03-25 22:08:50 -04:00
Martijn van Groningen 6cb82965bf percolator: Add query extract support for the blended term query and the common terms query. 2016-03-25 23:22:44 +01:00