Commit Graph

6808 Commits

Author SHA1 Message Date
Alexander Reelsen 33878be1e8 Simplify indices stats API
Note: This breaks backward compatibility

* Removed clear/all parameters, now all stats are returned by default
* Made the metrics part of the URL
* Removed a lot of handlers
* Added shards/indices/cluster level paremeter to change response serialization
* Returning translog statistics in IndicesStats
* Added TranslogStats class
* Added IndexShard.translogStats() method to get the stats from concrete implementation
* Updated documentation

Closes #4054
2014-01-06 07:27:03 +01:00
Martijn van Groningen 3024cc24a6 Added missing indices options to rest spec and rest actions. 2014-01-05 23:32:44 +01:00
Martijn van Groningen 61535bd1b4 Renamed old setIgnoreIndices to setIndicesOptions. 2014-01-05 23:11:34 +01:00
Simon Willnauer 312d2348f7 use trace logging to print RoutingNodes in test 2014-01-04 23:18:21 +01:00
Lee Hinman 47607a69a1 Default the circuit breaker limit to 80% of the maximum JVM heap 2014-01-03 16:21:55 -07:00
Lee Hinman 5463f7953f Expose `simple_query_string` flags in `flags` parameter 2014-01-03 16:14:19 -07:00
Simon Willnauer 602c63d2aa pass on node seed to the node level settings in TestCluster 2014-01-03 21:48:43 +01:00
Igor Motov 49d0ced16c Fix potential infinite loop in double wildcard processing
Fixes #4610
2014-01-03 12:44:44 -05:00
Simon Willnauer 8fba11dd74 Fix typo in JavaDoc -- s/note/not 2014-01-03 17:55:53 +01:00
Alexander Reelsen 811b7d7d78 Do not start packages on installation
The reason to not start packages on installation is to allow to configure
them before starting up (setting heap, cluster.name etc)

Also the documentation was updated in order to show, which statements need
to be executed.
In addition, these statements are also printed out when the package is
installed, depending on whether chkconfig, system or update-rc.d is used.

Closes #3722
2014-01-03 17:40:27 +01:00
Martijn van Groningen f1bf585089 The `fields` option should always return an array for json document fields and single valued field for metadata fields.
Also the `fields` option can only be used to fetch leaf fields, trying to do fetch object fields will return in a client error.

Closes #4542
2014-01-03 17:29:12 +01:00
Andrew Raines fdfc7d7460 Add cache stats to cat/nodes.
Closes #4543.
2014-01-03 10:22:00 -06:00
David Pilato 0c7b494bb8 plugin manager: new `timeout` option
When testing plugin manager with real downloads, it could happen that the test run forever. Fortunately, test suite will be interrupted after 20 minutes, but it could be useful not to fail the whole test suite but only warn in that case.

By default, plugin manager still wait indefinitely but it can be modified using new `--timeout` option:

```sh
bin/plugin --install elasticsearch/kibana --timeout 30s

bin/plugin --install elasticsearch/kibana --timeout 1h
```

Closes #4603.
Closes #4600.
2014-01-03 16:48:18 +01:00
Martijn van Groningen 2cb5cfecec Fixed issue where the parentTypes set would be updated when a new parent type is being added or removed during a refresh, which would have lead to concurrency issues. 2014-01-03 16:37:29 +01:00
Simon Willnauer 911ef6a058 Pass correct number of expected shards to assertion 2014-01-03 15:51:33 +01:00
Simon Willnauer 5b5b2e6c85 Add NodeVersionAllocationDecider that prevent allocations that require forward compatibility.
Today during restart scenarios it is possible that we recover from a node that
has already been upgraded to version N+1. The node that we relocate to is
on version N and might not be able to read the index format from the node
we relocate from. This causes `IndexFormatToNewException` during
recovery but only after recovery has finished which can cause large
load spikes during the upgrade period.

Closes #4588
2014-01-03 15:51:33 +01:00
Martijn van Groningen d5c440cd2e Fixed SimpleIdCache#clear() to not invoke onRemoval twice, which can happen in rare cases. 2014-01-03 15:24:27 +01:00
Martijn van Groningen 48c63c137a IdCache shouldn't implement Iterable 2014-01-03 15:24:27 +01:00
Martijn van Groningen 38f038f899 Already loaded SimpleIdReaderCache should be reloaded when a new `_parent` has been introduced.
Closes #4595
Relates #4568
2014-01-03 15:24:27 +01:00
Simon Willnauer fbae6e940b Fix transient settings assertions in ElasticsearchIntegrationTest
We missed to fail if transient settings were modifed in the test since we
compared against persistent settings.
2014-01-03 15:06:03 +01:00
sam 87947cb006 Raise visibilty of `#types()` to public in request classes
`CountRequest` and `ValidateQueryRequest` have package
private accessors for `#types()` which is inconsitent with
other getters.
2014-01-03 14:15:15 +01:00
Simon Willnauer 65c4282bb9 Check if node is still present when collecting attribute shard routings
The node we need to lookup for attribute colelction might not be part
of the `DiscoveryNodes` anymore due to node failure or shutdown. This
commit adds a check and removes the shard from the iteration.

Closes #4589
2014-01-03 14:03:24 +01:00
Britta Weber 9f54e9782d rename _shard -> _index and also rename classes and variables
closes #4584
2014-01-03 14:00:23 +01:00
Florian Schilling 611dd0a396 Setup an accurate version of Haversine closes #4596 2014-01-03 17:41:36 +09:00
Shay Banon 2a73cf4f82 support aliases for columns in cat API
use it as an example in nodes for now for some columns, though we need to go over all the columns and properly name them and alias them
2014-01-03 00:41:26 +01:00
Lee Hinman a754224751 Add field data memory circuit breaker.
This adds the field data circuit breaker, which is used to estimate
the amount of memory required to load field data before loading it. It
then raises a CircuitBreakingException if the limit is exceeded.

It is configured with two parameters:

`indices.fielddata.cache.breaker.limit` - the maximum number of bytes
of field data to be loaded before circuit breaking. Defaults to
`indices.fielddata.cache.size` if set, unbounded otherwise.

`indices.fielddata.cache.breaker.overhead` - a contast for all field
data estimations to be multiplied with before aggregation. Defaults to
1.03.

Both settings can be configured dynamically using the cluster update
settings API.
2014-01-02 15:04:47 -07:00
Dawid Weiss 84565c2951 In the spirit of the soon-to-be New Year 2014? 2014-01-02 22:07:53 +01:00
Honza Král 8517d8954e [TEST] add name parameter to get_alias in update_alias tests
to avoid failure on older es versions since get_alias without name has been
only introduced in #4539
2014-01-02 20:04:24 +01:00
Honza Král 076a24af14 [TEST] split tests with parent to pre/post 1.0 in the yaml test suite
See #4506 for details
2014-01-02 20:04:24 +01:00
Honza Král d5efb54785 [TEST] Split delete by query tests pre-1.0 and post-1.0
See #4074 for details
2014-01-02 20:04:24 +01:00
Simon Willnauer edb3e5f0f4 s/similariry/similarity in AllFieldMapper 2014-01-02 17:53:43 +01:00
Simon Willnauer beaa9153a6 Simulate the entire toXContent instead of special caseing
Today we try to detect if we need to generate the mapping or not in
the all mapper. This is error prone since it misses conditions if not
explicitly added. We should rather similate the generation instead.

This commit also adds a random test to check if the settings
of the all field mapper are correctly applied.

Closes #4579
Closes #4581
2014-01-02 17:15:51 +01:00
Simon Willnauer 79f676e45e Term Vector settings should be treated like flags without propergation
today if a specific feature is disabled for term vectors with something
like 'store_term_vector_positions = false' term vectors might be disabeled
alltogether even if 'store_term_vectors=true' in the mapping. This depends on the
order of the values in the mapping since the more specific one might override
the less specific on.

Closes #4582
2014-01-02 17:15:51 +01:00
Shay Banon c12427d047 remove double check for null in value source 2014-01-02 17:03:24 +01:00
Martijn van Groningen aa548f5148 Remove GET `_aliases` api in favour for GET `_alias` api
Currently there are two get aliases apis that both have the same functionality, but have a different response structure. The reason for having 2 apis is historic.

The GET _alias api was added in 0.90.x and is more efficient since it only sends the needed alias data from the cluster state between the master node and the node that received the request. In the GET _aliases api the complete cluster state is send to the node that received the request and then the right information is filtered out and send back to the client.

The GET _aliases api should be removed in favour for the alias api

Closes to #4539
2014-01-02 13:56:11 +01:00
Alexander Reelsen 8d4be46e59 Made parsing of ByteSizeValue case independent
This allows to parse '12GB' as well as '12gb'

Closes #4442
2014-01-02 13:00:41 +01:00
Martijn van Groningen f4bf0d5112 Replaced `ignore_indices` with `ignore_unavailable`, `expand_wildcards` and `allow_no_indices`.
* `ignore_unavailable` - Controls whether to ignore if any specified indices are unavailable, this includes indices that don't exist or closed indices. Either `true` or `false` can be specified.
* `allow_no_indices` - Controls whether to fail if a wildcard indices expressions results into no concrete indices. Either `true` or `false` can be specified. For example if the wildcard expression `foo*` is specified and no indices are available that start with `foo` then depending on this setting the request will fail. This setting is also applicable when `_all`, `*` or no index has been specified.
* `expand_wildcards` - Controls to what kind of concrete indices wildcard indices expression expand to. If `open` is specified then the wildcard expression if expanded to only open indices and if `closed` is specified then the wildcard expression if expanded only to closed indices. Also both values (`open,closed`) can be specified to expand to all indices.

Closes to #4436
2014-01-02 12:19:45 +01:00
Alexander Reelsen 040719f337 Allow GetAliasRequest to retrieve all aliases
Results in less data being sent over the wire, as the Cat API does not
need to have the whole cluster state.

Also added matchers for hasKey() for immutable open map (I think we should
add more of those to have map style assertions).

Closes #4455
2014-01-02 12:06:29 +01:00
Britta Weber 1ede9a5730 make term statistics accessible in scripts
term statistics can be accessed via the _shard variable.

Below is a minimal example. See documentation on details.

```

DELETE paytest

PUT paytest
{
    "mappings": {
        "test": {
            "_all": {
                "auto_boost": true,
                "enabled": true
            },
            "properties": {
                "text": {
                    "index_analyzer": "fulltext_analyzer",
                    "store": "yes",
                    "type": "string"
                }
            }
        }
    },
    "settings": {
        "analysis": {
            "analyzer": {
                "fulltext_analyzer": {
                    "filter": [
                        "my_delimited_payload_filter"
                    ],
                    "tokenizer": "whitespace",
                    "type": "custom"
                }
            },
            "filter": {
                "my_delimited_payload_filter": {
                    "delimiter": "+",
                    "encoding": "float",
                    "type": "delimited_payload_filter"
                }
            }
        },
        "index": {
            "number_of_replicas": 0,
            "number_of_shards": 1
        }
    }
}

POST paytest/test/1
{
    "text": "the+1 quick+2 brown+3 fox+4 is quick+10"
}

POST paytest/test/2
{
    "text": "the+1 quick+2 red+3 fox+4"
}

POST paytest/_refresh

POST paytest/_search
{
    "script_fields": {
       "ttf": {
          "script": "_shard[\"text\"][\"quick\"].ttf()"
       }
    }
}

POST paytest/_search
{
    "script_fields": {
       "freq": {
          "script": "_shard[\"text\"][\"quick\"].freq()"
       }
    }
}
POST paytest/test/2/_termvector
POST paytest/_search
{
    "script_fields": {
       "payloads": {
          "script": "term = _shard[\"text\"].get(\"red\",_PAYLOADS);payloads = []; for(pos : term){payloads.add(pos.payloadAsFloat(-1));} return payloads;"
       }
    }
}

POST paytest/_search
{
   "script_fields": {
      "tv": {
         "script": "_shard[\"text\"][\"quick\"].freq()"
      }
   },
   "query": {
      "function_score": {
         "functions": [
            {
               "script_score": {
                  "script": "_shard[\"text\"][\"quick\"].freq()"
               }
            }
         ]
      }
   }
}

```

closes #3772
2014-01-02 11:17:33 +01:00
Britta Weber df9b8ae02e do not call score() twice 2014-01-02 11:16:55 +01:00
Martijn van Groningen a7bb28c0e7 Made single shards APIs fail if routing is configured to be required in the mapping.
This change make single shard requests fail when no routing is specified and routing has been configured to be required in the mapping. Thi

 Closes #4506
2014-01-02 10:47:53 +01:00
Simon Willnauer c78f517d36 Allow 'omit_norms' on the '_all' field
The '_all' field doesn't allow to omit norms. In certain scenarios
omitting the norm values makes a lot of sense to get senseable scoring.

Closes #3734
2014-01-02 10:27:53 +01:00
Martijn van Groningen bb01995722 Made APIs consistently accept a query in the request body's `query` field.
The following APIs now accept the query in a top level `query` field like:
* delete_by_query
* validate_query
* count

These APIs used to accept the query directly in the request body which was inconsistent with the search and explain APIs. For this reason t

Closes #4074
2014-01-02 10:06:01 +01:00
Alexander Reelsen dee325de79 Packaging: Increasing default for max mapped pages to 262144 2014-01-02 09:10:46 +01:00
Simon Willnauer e7a84d744a Add ability to run certain packages with assertions disabled
Test can be run with `-Dtests.assertion.disabled=org.elasticsearch`
to run the tests without assertions to make sure assertions
don't hide any assignements etc. that introduce bugs in production.
2013-12-30 19:36:02 +01:00
Shay Banon e6e1a3463a more cleanup of cat API, fix index lookup failure count/health 2013-12-30 16:12:15 +01:00
David Pilato b29f89f7f9 We run PluginManagerTests using only node client.
We also add some debug logs and fix `tests.network` (setting it to true was not working from jenkins)
2013-12-30 15:40:52 +01:00
Shay Banon 05c5804341 Expose filtered nodes on TransportClient
Expose the list of nodes that were filtered out with the TransportClient, for example, due to different cluster name. Relates to #4569
closes #4571
2013-12-30 15:27:50 +01:00
Shay Banon 95abbe2057 mark abstract class as abstract 2013-12-30 14:40:01 +01:00
Shay Banon debfb0e996 move helper class for allocation tests to base class 2013-12-30 14:23:34 +01:00