Commit Graph

1007 Commits

Author SHA1 Message Date
Tanguy Leroux a01ecb20ea Port Delete By Query to Reindex infrastructure
closes #16883
2016-05-19 16:07:50 +02:00
markharwood a846ff93e9 Aggregations fix: support include/exclude strings formatted for IP and date fields in terms and significant_terms aggregations.
Closes #17705
2016-05-18 16:21:55 +01:00
Tanguy Leroux d7a31c8cf7 Add missing builder.endObject() in FsInfo
closes #18433
2016-05-18 15:19:30 +02:00
Christoph Büscher de321fb159 Disabling nodes.stats/30_discovery.yaml rest test
This is tracked in issue #18433, so temporarily disabling
the tests.
2016-05-18 13:02:55 +02:00
polyfractal 978c1e3e36 [TEST] Add missing sort processor to test
Also fixes the naming of the sort REST test
to follow the numbering convention
2016-05-17 14:52:17 -04:00
Zachary Tong 7c46b57ff2 Add a Sort ingest processor
Sorts an array of values in ascending or descending order. If all elements are numerics, they will be sorted numerically. If values are strings, or mixtures of strings/numbers, the elements will be sorted lexicographically.
2016-05-17 12:06:48 -04:00
Chris Earle 9303ffbf82 Adding REST tests to ensure key_as_string behavior stays consistent 2016-05-13 11:48:49 -04:00
Adrien Grand 61b1f4ad0b Fix xcontent rendering of ip terms aggs. #18003
Currently terms on an ip address try to put their binary representation in the
json response. With this commit, they would return a formatted ip address:

```
      "buckets": [
        {
          "key": "192.168.1.7",
          "doc_count": 1
        }
      ]
```
2016-05-13 14:59:36 +02:00
Lee Hinman 1c54033e92 Merge branch 'pr/18068' 2016-05-10 08:27:43 -06:00
Jim Ferenczi aef78ceb13 Do not return fieldstats information for fields that exist in the mapping
but not in the index.
2016-05-09 19:24:03 +02:00
Tanguy Leroux 0ff5652fff Add node name to Cat Recovery
closes #8041
2016-05-06 16:59:53 +02:00
Nik Everett 230697c202 [reindex] Switch throttle to Float.POSITIVE_INFITINTY/"unlimited"
All other values are errors.

Add java test for throttling. We had a REST test but it only ran against
one node so it didn't catch serialization errors.

Add Simple round trip test for rethrottle request
2016-05-04 16:14:32 -04:00
Martijn van Groningen 7aca1389e2 ingest: Add `date_index_name` processor.
Closes #17814
2016-04-29 17:20:48 +02:00
David Pilato 2232a7cdf3 Merge branch 'pr/cat-size-time-units' 2016-04-29 15:09:14 +02:00
Jim Ferenczi 573c4f3ed1 Extend field stats:
* Add isSearchable and isAggregatable (collapsed to true if any of the instances of that field are searchable or aggregatable).
* Accept wildcards in field names.
* Add a section named conflicts for fields with the same name but with incompatible types (instead of throwing an exception).
2016-04-27 16:51:53 +02:00
Alexander Kazakov a8a33a1a94 Row-centric output for _cat/fielddata 2016-04-27 13:29:02 +03:00
Martijn Laarman 98a37a1e54 Remove trailing / in rest spec for ingest.simulate (#17976)
The url that takes an id has a trailing forward slash, not really an
error but as its the only url in the whole spec that does this it
triggered my OCD :)
2016-04-26 11:03:17 +02:00
Nik Everett 1c2e84ba46 Fail request if rescore window > 10,000
The setting is named `index.max_rescore_window` and defaults to
`index.max_result_window` which defaults to 10,000.
2016-04-22 11:10:01 -04:00
Martijn van Groningen c5ad2e2865 Changed indexed scripts to be stored in the cluster state instead of the `.scripts` index.
Also added max script size soft limit for stored scripts.

Closes #16651
2016-04-22 13:42:55 +02:00
Martijn van Groningen dd2184ab25 ingest: Streamline option naming for several processors:
* `rename` processor, renamed `to` to `target_field`
* `date` processor, renamed `match_field` to `field` and renamed `match_formats` to `formats`
* `geoip` processor, renamed `source_field` to `field` and renamed `fields` to `properties`
* `attachment` processor, renamed `source_field` to `field` and renamed `fields` to `properties`

Closes #17835
2016-04-21 13:40:43 +02:00
Jun Ohtani 9eb242a5fe Analyze API : Rename filters/token_filters/char_filter to filter/token_filter/char_filter
Closes #15189
2016-04-21 18:05:11 +09:00
Martijn van Groningen 40c22fc654 percolator: removed .percolator type instead a field of type `percolator` should be configured before indexing percolator queries
* Added an extra `field` parameter to the `percolator` query to indicate what percolator field should be used. This must be an existing field in the mapping of type `percolator`.
* The `.percolator` type is now forbidden. (just like any type that starts with a `.`)

This only applies for new indices created on 5.0 and later. Indices created on previous versions the .percolator type is still allowed to exist.
The new `percolator` field type isn't active in such indices and the `PercolatorQueryCache` knows how to load queries from these legacy indices.
The `PercolatorQueryBuilder` will not enforce that the `field` parameter is of type `percolator`.
2016-04-19 11:20:31 +02:00
David Pilato 5e1f26c22a Add support for documented byte/size units and for micros as a time unit in _cat API
We advertise in our documentation that byte units are like `kb`, `mb`... But we actually only support the simple notation `k` or `m`.
This commit adds support for the documented form and keeps the non documented options to avoid any breaking change.

It also adds support for `micros`, `nanos` and `d` as a time unit in `_cat` API.

Remove the support for `b` as a SizeValue unit. Actually, for numbers, when using raw numbers without unit, there is no text to add/parse after the number. For example, you don't write `10` as `10b`. We support option like `size=` in `_cat` API which means that we want to display raw data without unit (singles).

Documentation updated accordingly.

Add test for the empty size option.

Fix missing TimeValues options for some cat APIs
2016-04-15 20:55:41 +02:00
jaymode 293ede3237 Enable installing the rest-api-spec artifact
Running `gradle install` on the rest-api-spec fails because there is no available install
task. This change applies the nexus plugin so that we can install and should also enable
publishing as part of the uploadArchives task.
2016-04-13 11:26:47 -04:00
Jun Ohtani 048d273408 Cat: cat health supports ts=0 option
If ts=0, cat health disable epoch and timestamp
Be Constant String timestamp and epoch
Move timestamp and epoch to Table
Add rest-api test and test

Closes #10109
2016-04-13 18:08:30 +09:00
Nik Everett 7908759949 Make aggregation registration more like query registration
Creates a class to hold behavior common to these `ParseFieldRegistry`s.

Also renames some weirdly named methods on StreamInput.
2016-04-12 14:50:44 -04:00
Igor Motov 81c59cae18 Add _cat/tasks
Adds new _cat endpoint that lists all tasks
2016-04-07 09:28:21 -06:00
Jimmy Jones f157dae053 Disallow unquoted field names, fix testcases using unquoted JSON 2016-04-06 14:37:15 -06:00
Spencer 7037670aeb [REST API] set correct default value
The correct default value for the `expand_wildcards` parameter to `indices.get_alias` is `all` as of all
f4d75f0212
2016-04-05 09:19:21 -07:00
Mpdreamz dd0c99bb65 index is a required url part 2016-04-04 14:14:44 +02:00
Mpdreamz 2944869ab9 Document task id's as string in the rest spec 2016-04-01 11:04:15 +02:00
Martijn Laarman dfe2c0ff0a Remove deprecated indices.get_aliases
This has been deprecated since the first release of Elasticsearch 1.0
2016-03-31 14:46:53 +02:00
Igor Motov e073b0c75d Add ability to group tasks by common parent
By default, tasks are grouped by node. However, task execution in elasticsearch can be quite complex and an individual task that runs on a coordinating node can have many subtasks running on other nodes in the cluster. This commit makes it possible to list task grouped by common parents instead of by node. When this option is enabled all subtask are grouped under the coordinating node task that started all subtasks in the group. To group tasks by common parents, use the following syntax:

 GET /tasks?group_by=parents
2016-03-30 17:50:27 -04:00
Nik Everett 78ab6c5b7f [reindex] Dynamic throttle!
This allows the user to update the reindex throttle on the fly, with changes
that speed up the throttling being applied immediately and changes that
slow down the throttling being applied during the next batch. This means
that if a user throttles reindex in such a way that it tries to sleep for
16 years and then realizes that they've done something wrong then they
can change the throttle and reindex will wake up again. We don't apply
slow downs immediately so we never get in danger of losing the scan context.

Also, if reindex is canceled while it is sleeping (how it honor throttling)
then it'll immediately wake up and cancel itself.
2016-03-30 16:40:42 -04:00
Isabel Drost-Fromm 132f96b6ba Merge pull request #17403 from elastic/docs/remove-rest-api-utils-reference
Remove reference to utils for generating REST docs
2016-03-30 21:18:41 +02:00
Adrien Grand 068c788ec8 Disable fielddata on text fields by defaults. #17386
`text` fields will have fielddata disabled by default. Fielddata can still be
enabled on an existing index by setting `fielddata=true` in the mappings.
2016-03-30 14:35:32 +02:00
Isabel Drost-Fromm 72d1ed65a4 Remove reference to utils for generating REST docs
This removes the reference to a no longer existing utils directory that used to be there for generating docs and tests from Java source.
2016-03-30 13:36:51 +02:00
javanna 8fc9dbbb99 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 14:27:04 +02:00
Clinton Gormley b87beeb05f Rename update-by-query REST tests to update_by_query 2016-03-29 13:13:49 +02:00
Clinton Gormley 647437ce56 REST: The body is required in the reindex API 2016-03-29 11:45:20 +02:00
Clinton Gormley 97606850e8 Renamed update-by-query REST spec to update_by_query 2016-03-29 11:45:20 +02:00
javanna de5cbda8e7 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-29 10:48:47 +02:00
Lee Hinman 80ab366de4 Add API to explain why a shard is or isn't assigned
This adds a new `/_cluster/allocation/explain` API that explains why a
shard can or cannot be allocated to nodes in the cluster. Additionally,
it will show where the master *desires* to put the shard, according to
the `ShardsAllocator`.

It looks like this:

```
GET /_cluster/allocation/explain?pretty
{
  "index": "only-foo",
  "shard": 0,
  "primary": false
}
```

Though, you can optionally send an empty body, which means "explain the
allocation for the first unassigned shard you find".

The output when a shard is unassigned looks like this:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : false
  },
  "assigned" : false,
  "unassigned_info" : {
    "reason" : "INDEX_CREATED",
    "at" : "2016-03-22T20:04:23.620Z"
  },
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 0.06666675,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "NO",
      "weight" : -1.3833332,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 2.3166666,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

And when the shard *is* assigned, the output looks like:

```
{
  "shard" : {
    "index" : "only-foo",
    "index_uuid" : "KnW0-zELRs6PK84l0r38ZA",
    "id" : 0,
    "primary" : true
  },
  "assigned" : true,
  "assigned_node_id" : "Qc6VL8c5RWaw1qXZ0Rg57g",
  "nodes" : {
    "V-Spi0AyRZ6ZvKbaI3691w" : {
      "node_name" : "Susan Storm",
      "node_attributes" : {
        "bar" : "baz"
      },
      "final_decision" : "NO",
      "weight" : 1.4499999,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    },
    "Qc6VL8c5RWaw1qXZ0Rg57g" : {
      "node_name" : "Slipstream",
      "node_attributes" : {
        "bar" : "baz",
        "foo" : "bar"
      },
      "final_decision" : "CURRENTLY_ASSIGNED",
      "weight" : 0.0,
      "decisions" : [ {
        "decider" : "same_shard",
        "decision" : "NO",
        "explanation" : "the shard cannot be allocated on the same node id [Qc6VL8c5RWaw1qXZ0Rg57g] on which it already exists"
      } ]
    },
    "PzdyMZGXQdGhqTJHF_hGgA" : {
      "node_name" : "The Symbiote",
      "node_attributes" : { },
      "final_decision" : "NO",
      "weight" : 3.6999998,
      "decisions" : [ {
        "decider" : "filter",
        "decision" : "NO",
        "explanation" : "node does not match index include filters [foo:\"bar\"]"
      } ]
    }
  }
}
```

Only "NO" decisions are returned by default, but all decisions can be
shown by specifying the `?include_yes_decisions=true` parameter in the
request.

Resolves #14593
2016-03-28 15:21:02 -06:00
Nik Everett 0e6141e675 Replace is_true: took with took >= 0
This prevents tests from failing on machines that can finish the request
less than half a millisecond.
2016-03-28 13:03:48 -04:00
javanna a685148268 [TEST] expand REST tests to check for roles in nodes info, nodes stats and tasks list response 2016-03-25 22:53:21 +01:00
javanna a9f4982c40 Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-25 20:16:40 +01:00
Clinton Gormley 30d78f4be0 In cat.snapshots, repository is required
Closes #17216
2016-03-25 14:23:52 +01:00
javanna 27d4994aff Merge branch 'master' into enhancement/remove_node_client_setting 2016-03-24 18:10:11 +01:00
Areek Zillur e16e113691 Remove suggest threadpool
In #17198, we removed suggest transport action, which
used the `suggest` threadpool to execute requests. Now
`suggest` threadpool is unused and suggest requests are
executed on the `search` threadpool.
2016-03-23 18:01:45 -04:00
Areek Zillur 91dd9b3301 Merge suggest stats into search stats 2016-03-23 16:37:56 -04:00