Commit Graph

316 Commits

Author SHA1 Message Date
Yannick Welsch d5b691b68e Extend reroute with an option to force assign stale primary shard copies
Closes 
2016-01-19 12:07:01 +01:00
Yannick Welsch a1b8dd2de9 Add per-index setting to limit number of nested fields
Closes 
2016-01-19 10:03:48 +01:00
Jason Tedor df598e8129 Modify load average formats
This commit modifies the load_average in the node stats API response
to be an object containing the one-minute, five-minute and
fifteen-minute load averages as fields (if those values are
available). Additionally, this commit modifies the cat nodes API
response to format the one-minute, five-minute and fifteen-minute load
averages as null if any of the respective values are not available.
2016-01-18 11:41:34 -05:00
Nik Everett a8ceb37d07 Merge pull request from nik9000/deprecate_collect_payloads
Deprecate span_near's collect_payloads
2016-01-12 09:56:19 -05:00
Michael McCandless b4a095d430 Merge pull request from mikemccand/fair_indexing_buffers
The indexing buffer on a node (default: 10% of the JVM heap) is now a "shared pool" across all shards on that node.  This way, shards doing intense indexing can use much more than other shards doing only light indexing, and only once the sum of all indexing buffers across all shards exceeds the node's indexing buffer will we ask shards to move recently indexed documents to segments on disk.
2016-01-12 04:59:10 -05:00
Jason Tedor d9fd6e2fe3 Merge pull request from jasontedor/load-average
Reintroduce five-minute and fifteen-minute load averages on Linux

Relates , relates 
2016-01-11 23:43:40 -05:00
Jason Tedor 1de2081ed3 Reintroduce five-minute and fifteen-minute load averages on Linux
This commit reintroduces the five-minute and fifteen-minute load stats
on Linux, and changes the format of the load_average field back to an
array.
2016-01-11 23:42:47 -05:00
Nik Everett ae0bf1b32f Deprecate span_near's collect_payloads 2016-01-11 22:11:38 -05:00
Michael McCandless 07e8370322 feedback 2016-01-11 17:16:28 -05:00
Clinton Gormley f782c8ac07 Merge pull request from sromocki/patch-1
Update crud.asciidoc, nouns mixed up
2016-01-11 20:01:16 +01:00
Adrien Grand 67d233cecd Remove warmers and the warmer API.
Warmers are now barely useful and will be removed in 3.0. Note that this only
removes the warmer API and query-based warmers. We still have warmers internally
for eg. global ordinals.

Close 
2016-01-07 09:57:07 +01:00
Martijn van Groningen 2d6adf6428 Percolator refactoring:
* Added percolator field mapper that extracts the query terms and indexes these terms with the percolator query.
* At percolate time these extracted terms are used to query percolator queries that are like to be evaluated. This can significantly cut down the time it takes to percolate. Whereas before all percolator queries were evaluated if they matches with the document being percolated.
* Changes made to percolator queries are no longer immediately visible, a refresh needs to happen before the changes are visible.
* By default the percolate api only returns upto 10 matches instead of returning all matching percolator queries.
* Made percolate more modular, so that it is easier to add unit tests.
* Added unit tests for the percolator.

Closes 
Closes 
2016-01-06 16:08:10 +01:00
Jim Ferenczi 992ffac509 Merge pull request from jimferenczi/classic_similarity
Renames `default` similarity into `classic`
2015-12-30 08:42:20 -08:00
Adrien Grand a7e1eeb13d Merge pull request from jpountz/deprecate/field_boost
Remove object notation for core types.
2015-12-30 09:58:01 +01:00
Adrien Grand bffbad3c19 Remove object notation for core types.
When specifying a string field, you can either do:

```
{
  "foo": "bar"
}
```

or

```
{
  "foo": {
    "value": "bar",
    "boost": 42
  }
}
```

The latter option is now removed.

Closes 
2015-12-28 19:49:18 +01:00
Simon Willnauer f5e4cd4616 Remove recovery threadpools and throttle outgoing recoveries on the master
Today we throttle recoveries only for incoming recoveries. Nodes that have a lot
of primaries can get overloaded due to too many recoveries. To still keep that at bay
we limit the number of threads that are sending files to the target to overcome this problem.

The right solution here is to also throttle the outgoing recoveries that are today unbounded on
the master and don't start the recovery until we have enough resources on both source and target nodes.

The concurrency aspects of the recovery source also added a lot of complexity and additional threadpools
that are hard to configure. This commit removes the concurrent streamns notion completely and sends files
in the thread that drives the recovery simplifying the recovery code considerably.
Outgoing recoveries are not throttled on the master via a allocation decider.
2015-12-22 14:59:43 +01:00
Simon Willnauer 3a5d3a3bb0 Remove deprecated query cache settings 2015-12-22 09:39:12 +01:00
Simon Willnauer fcfd98e9e8 Drop support for simple translog and hard-wire buffer to 8kb
Today we have two variants of translogs for indexing. We only recommend the buffered
one which also has a 20% advantage in indexing speed. This commit removes the option and defaults
to the buffered case. It also hard-wires the translog buffer to 8kb instead of 64kb. We used to
adjust that buffer based on if the shard is active or not, this code has also been removed and
instead we just keep an 8kb buffer arround.
2015-12-21 16:44:35 +01:00
Jim Ferenczi 81fd2169cf Renames "default" similarity into "classic".
Replaces deprecated DefaultSimilarity by ClassicSimilarity.
Fixes 
2015-12-21 16:22:53 +01:00
Simon Willnauer 613e8a9782 add note to migration guide 2015-12-21 15:20:32 +01:00
Yannick Welsch 3a442db9bd Allocate primary shards based on allocation ids
Closes 
2015-12-17 15:55:50 +01:00
Adrien Grand 9fd843748e Remove the `format` option of the `_source` field.
This option allows to force the xcontent type to use to store the `_source`
document. The default is to use the same format as the input format.

This commit makes this option ignored for 2.x indices and rejected for 3.0
indices.
2015-12-11 18:06:20 +01:00
Jim Ferenczi d78e24689b Merge pull request from jimferenczi/missing_query_removal
Remove the MissingQueryBuilder which was deprecated in 2.2.0.
2015-12-11 17:20:47 +01:00
Jim Ferenczi 437488ae64 Remove the MissingQueryBuilder which was deprecated in 2.2.0.
As a replacement use ExistsQueryBuilder inside a mustNot() clause.
So instead of using `new ExistsQueryBuilder(name)` now use:
`new BoolQueryBuilder().mustNot(new ExistsQueryBuilder(name))`.

Closes 
2015-12-10 12:55:05 +01:00
David Pilato 414fccb7d1 Merge branch 'fix/15268-proxy-auth' 2015-12-09 23:21:00 +01:00
Adrien Grand f2d8a35888 Dynamically map floating-point numbers as floats instead of doubles.
Close 
2015-12-08 18:19:47 +01:00
Britta Weber e0aa481bf5 Merge pull request from brwe/copy-to-in-multi-fields-exception
throw exception if a copy_to is within a multi field

Copy to within multi field is ignored from 2.0 on, see .
Instead of just ignoring it, we should throw an exception if this
is found in the mapping when a mapping is added. For already
existing indices we should at least log a warning.
We remove the copy_to in any case.

related to 
2015-12-08 14:41:07 +01:00
David Pilato 7dcb40bcac Add support for proxy authentication for s3 and ec2
When using S3 or EC2, it was possible to use a proxy to access EC2 or S3 API but username and password were not possible to be set.

This commit adds support for this. Also, to make all that consistent, proxy settings for both plugins have been renamed:

* from `cloud.aws.proxy_host` to `cloud.aws.proxy.host`
* from `cloud.aws.ec2.proxy_host` to `cloud.aws.ec2.proxy.host`
* from `cloud.aws.s3.proxy_host` to `cloud.aws.s3.proxy.host`
* from `cloud.aws.proxy_port` to `cloud.aws.proxy.port`
* from `cloud.aws.ec2.proxy_port` to `cloud.aws.ec2.proxy.port`
* from `cloud.aws.s3.proxy_port` to `cloud.aws.s3.proxy.port`

New settings are `proxy.username` and `proxy.password`.

```yml
cloud:
    aws:
        protocol: https
        proxy:
            host: proxy1.company.com
            port: 8083
            username: myself
            password: theBestPasswordEver!
```

You can also set different proxies for `ec2` and `s3`:

```yml
cloud:
    aws:
        s3:
            proxy:
                host: proxy1.company.com
                port: 8083
                username: myself1
                password: theBestPasswordEver1!
        ec2:
            proxy:
                host: proxy2.company.com
                port: 8083
                username: myself2
                password: theBestPasswordEver2!
```

Note that `password` is filtered with `SettingsFilter`.

We also fix a potential issue in S3 repository. We were supposed to accept key/secret either set under `cloud.aws` or `cloud.aws.s3` but the actual code never implemented that.

It was:

```java
account = settings.get("cloud.aws.access_key");
key = settings.get("cloud.aws.secret_key");
```

We replaced that by:

```java
String account = settings.get(CLOUD_S3.KEY, settings.get(CLOUD_AWS.KEY));
String key = settings.get(CLOUD_S3.SECRET, settings.get(CLOUD_AWS.SECRET));
```

Also, we extract all settings for S3 in `AwsS3Service` as it's already the case for `AwsEc2Service` class.

Closes .
2015-12-07 23:10:54 +01:00
Simon Willnauer 4e80a5e099 Merge pull request from s1monw/remove_ancient_settings
Remove ancient deprecated and alternative recovery settings
2015-12-07 21:15:49 +01:00
Simon Willnauer c1f7f8c03c add settings to migration guide 2015-12-07 09:54:23 +01:00
Lee Hinman f709b7283f Remove `GET` option for /_forcemerge
POST should be used to indicate this is not just a retrieval operation.

Resolves 
2015-12-03 13:56:15 -07:00
Clinton Gormley e84b61358f Fix version order for breaking changes docs 2015-12-03 14:24:19 +01:00
Jim Ferenczi 731833cfc6 Fixes
Do not to load fields from _source when using the `fields` option.
  Non stored (non existing) fields are ignored by the fields visitor when using the `fields` option.

Fixes 
  Support * wildcard to retrieve stored fields when using the `fields` option.
  Supported pattern styles are "xxx*", "*xxx", "*xxx*" and "xxx*yyy".
2015-11-30 11:00:32 +01:00
Martijn van Groningen 48771f1a76 field stats: Added `min_value_as_string` and `max_value_as_string` response elements for all number based fields. The existing `min_value` and `max_value` will return the values as numbers instead.
Closes 
2015-11-23 08:48:28 +01:00
Christoph Büscher 991ab030dc Adding notes about ShapeBuilder changes to migration doc. 2015-11-21 12:51:28 +01:00
David Pilato 052285ac88 Revert 3cacec4468
We actually want to keep the test when using deprecated setting in 3.0.
We will keep this setting deprecated as well so people will be able to update in a smoother way.

Also add the deprecating information to the migration documentation.
2015-11-18 19:14:26 +01:00
Clinton Gormley 062abf813b Update packaging.asciidoc
Remove mention of following symlinks under `path.scripts`

Closes 
2015-11-18 13:48:43 +01:00
Jason Tedor 6872d545ac Add system CPU percent to OS stats
This commit adds the system CPU percent reflecting the recent CPU usage
for the whole system.
2015-11-17 13:48:46 -05:00
Peter Lebedev 1201843d72 Update java.asciidoc
Add few more breaking changes.

Closes 
2015-11-17 12:27:44 +01:00
Shaunak Kashyap 636c88d09f Fixing setting name 2015-11-13 13:36:14 -08:00
Clinton Gormley bf3f9c778c Docs: Added plugin-descriptor.properties to breaking changes
Closes 
2015-11-09 12:17:44 +01:00
Clinton Gormley f0fda74083 Moved a breaking change to the correct section. Closes 2015-11-08 21:53:56 +01:00
Clinton Gormley 1849c645f3 Merge pull request from elastic/markw/typo-parentchild-releasenotes-14508
Fixed minor typo
2015-11-08 21:48:40 +01:00
Clinton Gormley 28015d9e03 Docs: Made the list of breaking changes in each version more obvious 2015-11-08 12:59:07 +01:00
Clinton Gormley 6fa799e694 Docs: Correct docs around CONF_DIR and -Des.path.conf 2015-11-05 14:05:52 +01:00
Mark Walkom 4f2154b0e0 Removed / from commands 2015-11-04 14:56:06 +11:00
Mark Walkom ab0cba4bbf Removed / from command 2015-11-04 14:54:40 +11:00
Mark Walkom 97a066829c Fixed minor typo 2015-11-04 14:46:37 +11:00
javanna ce6aa258a9 Remove support for query_binary and filter_binary
query_binary and filter_binary are unused at this point, as we only parse on the coordinating node and the java api only holds structured java objects for queries and filters, meaning they all implement Writeable and get natively serialized.

Relates to 
Closes 
2015-11-03 14:05:45 +01:00
Jason Tedor e3b8dc7121 Forbid changing thread pool types
This commit forbids the changing of thread pool types for any thread
pool. The motivation here is that these are expert settings with
little practical advantage.

Closes , relates , relates , relates 
2015-11-02 20:52:48 -05:00