Commit Graph

903 Commits

Author SHA1 Message Date
Jim Ferenczi 1343d6cbd1 Remove search_after from the query string param of the rest api spec.
Handle null values in search_after.
Ensure that the cluster is green after each index creation in the integ tests.
2016-01-27 19:21:01 +01:00
javanna 8006e5cd15 [TEST] re-enable and merge cluster settings REST tests
We used to have a disabled test around cluster put settings as it left cluster settings behind without a way to remove them. That has been in fixed in the cluster put settings api, so the test can be re-enabled.
2016-01-27 17:37:42 +01:00
Jim Ferenczi aea7660e37 Add search_after parameter in the Search API.
The search_after parameter provides a way to efficiently paginate from one page to the next. This parameter accepts an array of sort values, those values are then used by the searcher to sort the top hits from the first document that is greater to the sort values.
This parameter must be used in conjunction with the sort parameter, it must contain exactly the same number of values than the number of fields to sort on.

NOTE: A field with one unique value per document should be used as the last element of the sort specification. Otherwise the sort order for documents that have the same sort values would be undefined. The recommended way is to use the field `_uuid` which is certain to contain one unique value for each document.

Fixes #8192
2016-01-27 09:42:58 +01:00
Tal Levy ff0e8272cb [ingest] update test to verify that documents are deep-copied between verbose results 2016-01-26 14:12:42 -08:00
Martijn van Groningen df0be87b18 Merge pull request #16049 from elastic/feature/ingest
Merge feature/ingest branch into master branch.

This adds the ingest feature to ES that allows to preprocess document before indexing on an ingest node.
By default a node is an ingest node. Documents are preprocessed via a pipeline. A pipeline consists
out of one or more processors Each processor makes one or more modifications to a document processed.
There are many types of processors available out-of-the-box that are designed to make a specific change to a document being processed. In a cluster many pipeline can be configured via dedicated pipeline APIs. An new option on the bulk
and index APIs allows to control what pipeline is picked for preprocessing. If no pipeline is specified then the ingest
feature is skipped and no preprocessing takes place.
2016-01-26 13:41:13 +01:00
Martijn van Groningen 8b02f214c4 percolator: The percolate api shouldn't add field mappings for unmapped fields inside the document being percolated to the mapping.
Closes #15751
2016-01-26 10:26:46 +01:00
javanna 36d98478bf Merge branch 'master' into feature/ingest 2016-01-25 18:01:09 +01:00
Ryan Ernst df24019261 Merge pull request #16038 from rjernst/remove_site_plugin
Plugins: Remove site plugins
2016-01-21 12:32:22 -08:00
Tal Levy 3a6c2d008e rename processor_tag to tag 2016-01-21 09:05:42 -08:00
Martijn van Groningen 602a0f183e Merge remote-tracking branch 'es/master' into feature/ingest 2016-01-19 22:01:38 +01:00
Tal Levy 4ef85eda36 add default separator test to dedot rest test 2016-01-18 09:25:36 -08:00
Simon Willnauer 9562fb76bc expose default settings via rest API 2016-01-18 12:48:47 +01:00
Simon Willnauer 13e5547537 Add REST tests for reset index settings and for listing defaults. 2016-01-18 10:02:37 +01:00
Simon Willnauer dc05669fd9 replace unsupported setting translog.disable_flush with a high value of translog.flush_threshold_size 2016-01-18 09:23:35 +01:00
Ryan Ernst 3b78267c71 Plugins: Remove site plugins
Site plugins used to be used for things like kibana and marvel, but
there is no longer a need since kibana (and marvel as a kibana plugin)
uses node.js. This change removes site plugins, as well as the flag for
jvm plugins. Now all plugins are jvm plugins.
2016-01-16 22:45:37 -08:00
Tal Levy 9f48df9736 Add on_failure support for verbose _simulate execution and introduce optional processor_tag to Processors 2016-01-15 14:56:20 -08:00
Tal Levy 1754eece66 introduce DeDotProcessor
fixes #15944.
2016-01-15 11:35:18 -08:00
javanna 9c06736dbd Merge branch 'master' into feature/ingest 2016-01-15 10:11:56 +01:00
javanna 07a82d0c09 make get alias expand to open and closed indices by default
This change affects get alias, get aliases as well as cat aliases. They all return closed indices too by default. get alias and get aliases also allow to return open indices only through the `expand_wildcards` option (set it to `open`).

Closes #14982
2016-01-14 10:40:31 +01:00
Martijn van Groningen f3883343cb Move the pipeline configuration from the dedicated index to the cluster state.
Closes #15842
2016-01-13 22:59:36 +01:00
javanna ea8065aa3d Merge branch 'master' into feature/ingest 2016-01-12 18:28:42 +01:00
Jason Tedor 1de2081ed3 Reintroduce five-minute and fifteen-minute load averages on Linux
This commit reintroduces the five-minute and fifteen-minute load stats
on Linux, and changes the format of the load_average field back to an
array.
2016-01-11 23:42:47 -05:00
javanna 90743d8db0 add REST test for bulk api integration with ingest 2016-01-11 19:04:34 +01:00
javanna ae69d46f92 move processors that have no deps to core, also move to core rest spec and tests and set node.inget to true by default 2016-01-08 10:39:39 +01:00
Adrien Grand 67d233cecd Remove warmers and the warmer API.
Warmers are now barely useful and will be removed in 3.0. Note that this only
removes the warmer API and query-based warmers. We still have warmers internally
for eg. global ordinals.

Close #15607
2016-01-07 09:57:07 +01:00
Martijn van Groningen 2d6adf6428 Percolator refactoring:
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.

Closes #12664
Closes #13646
2016-01-06 16:08:10 +01:00
Igor Motov a89dba27c2 Task Management: Add framework for registering and communicating with tasks
Adds task manager class and enables all activities to register with the task manager. Currently, the immutable Transport*Activity class represents activity itself shared across all requests. This PR adds and an additional structure Task that keeps track of currently running requests and can be used to communicate with these requests using TransportTaskAction.

Related to #15117
2016-01-05 12:24:43 -05:00
Adrien Grand 6d3c9b074c Remove support for the `multi_field` type.
It is officially unsupported since version 1.0.
2015-12-30 12:03:15 +01:00
Lee Hinman 482843e27b Fix build to run correctly on FreeBSD
This adds the required changes/checks so that the build can run on
FreeBSD.

There are a few things that differ between FreeBSD and Linux:

- CPU probes return -1 for CPU usage
- `hot_threads` cannot be supported on FreeBSD

From OpenJDK's `os_bsd.cpp`:

```c++
bool os::is_thread_cpu_time_supported() {
  #ifdef __APPLE__
  return true;
  #else
  return false;
  #endif
}
```

So this API now returns (for each FreeBSD node):

```
curl -s localhost:9200/_nodes/hot_threads
::: {Devil Hunter Gabriel}{q8OJnKCcQS6EB9fygU4R4g}{127.0.0.1}{127.0.0.1:9300}
   hot_threads is not supported on FreeBSD
```

- multicast fails in native `join` method - known bug:
  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=193246

Which causes:

```
1> Caused by: java.net.SocketException: Invalid argument
1>    at java.net.PlainDatagramSocketImpl.join(Native Method)
1>    at java.net.AbstractPlainDatagramSocketImpl.join(AbstractPlainDatagramSocketImpl.java:179)
1>    at java.net.MulticastSocket.joinGroup(MulticastSocket.java:323)
1>    at org.elasticsearch.plugin.discovery.multicast.MulticastChannel$Plain.buildMulticastSocket(MulticastChannel.java:309)
```

So these tests are skipped on FreeBSD.

Resolves #15562
2015-12-22 12:36:04 -07:00
Simon Willnauer 6ea266a89c Merge branch 'master' into settings_prototype 2015-12-15 16:33:01 +01:00
Jun Ohtani fab44398d9 Analysis: Add detail response support
add explain option
fix char_filter bug

Closes #11076 #15257
2015-12-10 23:10:51 +09:00
Robert Muir e454fadc22 Merge branch 'master' into shave_mustache 2015-12-10 07:58:24 -05:00
Yannick Welsch bef0bedba9 Add support to _aliases endpoint to specify multiple indices and aliases in one action
Closes #15305
2015-12-09 19:08:27 +01:00
Robert Muir a6e1655fe9 fix integ tests 2015-12-09 00:30:32 -05:00
Jim Ferenczi 23aeaa88b2 Fixes random failures of org.apache.elasticsearch.test.rest.RestIT
RestTable: ignores right padding for the last cell of a column.
2015-12-08 20:52:24 +01:00
Myll 73a3c326c9 _cat APIs: remove space at the end of a line
Fixes #9464
2015-12-08 15:03:59 +01:00
Simon Willnauer 8502926327 fuck you linefeed 2015-12-08 14:39:16 +01:00
Simon Willnauer 2e27ee393f add rest API to reset settings 2015-12-08 14:39:16 +01:00
Lee Hinman f709b7283f Remove `GET` option for /_forcemerge
POST should be used to indicate this is not just a retrieval operation.

Resolves #15165
2015-12-03 13:56:15 -07:00
Jim Ferenczi e182072b6f Merge pull request #15017 from jimferenczi/fields_option
Refuse to load fields from _source when using the `fields` option and support wildcards.
2015-11-30 11:01:21 +01:00
Jim Ferenczi 731833cfc6 Fixes #14489
Do not to load fields from _source when using the `fields` option.
  Non stored (non existing) fields are ignored by the fields visitor when using the `fields` option.

Fixes #10783
  Support * wildcard to retrieve stored fields when using the `fields` option.
  Supported pattern styles are "xxx*", "*xxx", "*xxx*" and "xxx*yyy".
2015-11-30 11:00:32 +01:00
Clinton Gormley 27dac8dc2c REST spec: Added the verbose flag to indices.segments
Relates to #9111
2015-11-30 07:41:29 +01:00
Jayson Minard 815c53e6b4 body attribute was at wrong nesting level 2015-11-26 14:34:02 -03:00
Lee Hinman a25b407aeb Add support for headers in REST tests
This adds support for arbitrary headers sent with each REST request, it
will allow us to test things like different xcontent-encoding (see
50_with_headers.yaml for what this looks like).

Headers are specified at the same level as `catch`, so a request would
look like:

```yaml
- do:
    headers:
      Content-Type: application/yaml
    get:
      index: test_1
      type:  _all
      id:    1
```
2015-11-24 08:25:02 -07:00
Martijn van Groningen 48771f1a76 field stats: Added `min_value_as_string` and `max_value_as_string` response elements for all number based fields. The existing `min_value` and `max_value` will return the values as numbers instead.
Closes #14404
2015-11-23 08:48:28 +01:00
Xu Zhang 2e6d72de27 Catch exception when reading corrupted snapshot.
Single corrupted snapshot file shouldn't prevent listing all other
snapshot in repository.
2015-11-18 21:43:46 -08:00
Jason Tedor 185027a0ff Update REST tests to reflect changes to cat nodes default response
This commit updates the cat nodes REST test to include the CPU percent
that was recently added to the default output of the cat nodes response.
2015-11-17 14:51:03 -05:00
Jason Tedor 95c4846e58 Fix race condition in cat shards test
This commit fixes a test bug in the cat shards REST test. In
particular, there was a race condition in the test that would cause the
test to sometimes fail. The race condition is that some of the shards
would go to state STARTED after the sync flush was issued. These shards
would (correctly) show up in the output as having state started but
without a sync_id. However, the expected output was written to only
look for shards that have state STARTED and a sync_id, or shards that
are still INITIALIZING or are UNASSIGNED and (of course) do not have a
sync_id. The best approach here is to just simplify the test.
2015-11-13 12:16:22 -05:00
Jason Tedor 99abb76c78 Fix cat shards test bug 2015-11-13 09:31:44 -05:00
Jason Tedor a9ab35a487 Add sync_id to cat shards API
This commit adds the ability to get the sync_id from the cat shards API.

Closes #14705
2015-11-13 05:13:08 -05:00