Commit Graph

18649 Commits

Author SHA1 Message Date
Tal Levy d1356b0a6f Merge pull request #15763 from talevy/fix/simulate
[Ingest] Fix/simulate
2016-01-05 07:21:36 -08:00
Tal Levy 183386173c cleanup simulate test and add docs 2016-01-04 16:10:42 -08:00
Tal Levy 82d87e7149 Merge pull request #15647 from talevy/ingest_fail_processor
[Ingest] Fail Processor
2016-01-04 11:35:51 -08:00
Tal Levy 9dd54f2b4f Introduce a fail processor 2016-01-04 11:33:50 -08:00
Tal Levy af89b57a39 Merge pull request #15646 from talevy/error_metadata
[Ingest] add on_failure context to ingest metadata during executeOnFailure
2016-01-04 11:26:43 -08:00
Tal Levy f34ce9ddf4 add on_failure context to ingest metadata during executeOnFailure 2016-01-04 11:24:27 -08:00
Luca Cavanna d14f6dbd3b Merge pull request #15747 from javanna/enhancement/simulate_index_type_not_required
make index type and id optional in simulate api
2016-01-04 13:25:22 +01:00
javanna 8de31b3c64 make index type and id optional in simulate api
Default values are _index, _type and _id.

Closes #15711
2016-01-04 13:11:44 +01:00
Martijn van Groningen f2fce0edca Removed ingest runner main class, `gradle run --debug-jvm` should be used instead. 2015-12-29 13:07:23 +01:00
Martijn van Groningen 9fa3f469e9 fix test after merging in master branch 2015-12-29 13:00:27 +01:00
Martijn van Groningen 79161d77df Merge remote-tracking branch 'es/master' into feature/ingest 2015-12-29 12:56:09 +01:00
Daniel Mitterdorfer c813d21ffb Replace * import with explicit imports 2015-12-29 12:08:30 +01:00
David Pilato 6dd1eb0d17 Merge branch 'pr/14277-azure-timeout' 2015-12-29 12:00:57 +01:00
Daniel Mitterdorfer 46a4aa9704 Tighten assertions in BulkProcessorRetryIT
With this commit we check more precisely on the result of a bulk
request. It could either be ok, fail or be rejected due to resource
constraints. Previously, we have relied that by default we never
get rejected.

However, this is a valid condition even when retrying. With this
commit we check that we either retried often enough that we don't
get rejected *and* if we got rejected that we maxed out the number
of specified retries.
2015-12-29 11:43:41 +01:00
David Pilato 96b3166c6d Add timeout settings (default to 5 minutes)
By default, azure does not timeout. This commit adds support for a timeout settings which defaults to 5 minutes.
It's a timeout **per request** not a global timeout for a snapshot request.

It can be defined globally, per account or both. Defaults to `5m`.

```yml
cloud:
    azure:
        storage:
            timeout: 10s
            my_account1:
                account: your_azure_storage_account1
                key: your_azure_storage_key1
                default: true
            my_account2:
                account: your_azure_storage_account2
                key: your_azure_storage_key2
                timeout: 30s
```

In this example, timeout will be 10s for `my_account1` and 30s for `my_account2`.

Closes #14277.
2015-12-29 11:40:48 +01:00
David Pilato a49fe189b0 Support global `repositories.azure.` settings
All those repository settings can also be defined globally in `elasticsearch.yml` file using prefix `repositories.azure.`. For example:

```yml
repositories.azure:
    container: backup-container
    base_path: backups
    chunk_size: 32m
    compress": true
```

Closes #13776.
2015-12-29 10:43:01 +01:00
Simon Willnauer 60cbb2d7bc [TEST] Protect UpgradeIT from using too many replicas 2015-12-29 10:41:04 +01:00
Yannick Welsch c6182cbd37 Fail replica shards before primary shards
As failing primaries also fail associated replicas, we must fail replicas first so that their nodes are properly added to ignore list

Closes #15686
2015-12-28 22:37:18 +01:00
Simon Willnauer 387bdbd322 Don't limit recoveries in RoutingTableTests 2015-12-28 21:36:34 +01:00
Yannick Welsch 24ab66c694 Merge pull request #15678 from ywelsch/enhance/faster-balancer
Speed improvements for BalancedShardsAllocator
2015-12-28 19:38:11 +01:00
Yannick Welsch 7e134da7d3 Cache result of RoutingNodes.node(...) in ModelNode 2015-12-28 19:14:48 +01:00
Yannick Welsch eea791de15 Remove superfluous method numPrimaries 2015-12-28 19:14:48 +01:00
Yannick Welsch e855282258 Simplify numShards calculation 2015-12-28 19:14:48 +01:00
Yannick Welsch 1536d7fe37 For rebalancing an index, only consider nodes that currently have a shard of that index or where the index can be allocated
This allows to prune a large number of nodes in case of hot/warm setup
2015-12-28 19:14:48 +01:00
Yannick Welsch 207dfc457d Remove superfluous method 2015-12-28 19:14:48 +01:00
Yannick Welsch fc0a33be05 Eliminate adding/removing shard to simulate weight of added shard / removed shard
Removal of the pattern node.addShard() -> calculate weight -> node.removeShard() which is expensive as, beside map lookups, it invalidates caching of precomputed values in ModelNode and ModelIndex. Replaced by adding an additional parameter to the weight function which accounts for the added / removed shard.
2015-12-28 19:14:48 +01:00
Yannick Welsch 5bd31a6cca Fix typo in method name 2015-12-28 19:14:48 +01:00
Yannick Welsch 67905b384f Remove unused class 2015-12-28 19:14:48 +01:00
Yannick Welsch 1bb7ca8a85 Removed unused methods 2015-12-28 19:14:48 +01:00
Yannick Welsch 0fd19008b1 Precalculate avgShardsPerNode 2015-12-28 19:14:48 +01:00
Yannick Welsch 507bb11345 Split two-element array into proper variables 2015-12-28 19:14:48 +01:00
Adrien Grand c6cf84336f Merge pull request #15679 from jpountz/fix/text_parsing
Make text parsing less lenient.
2015-12-28 16:47:05 +01:00
Adrien Grand 5eb7555ffb Make text parsing less lenient.
It now requires that the parser is on a value.
2015-12-28 16:06:42 +01:00
Simon Willnauer ba755a554f Merge pull request #15372 from s1monw/trash_recovery_threads
Remove recovery threadpools and throttle outgoing recoveries on the master
2015-12-28 15:43:14 +01:00
Simon Willnauer dcb164db3b Merge pull request #15598 from lks21c/master
Put space into last comment line
2015-12-28 15:41:10 +01:00
Jason Tedor 84c4ab6c18 Correctly release threads from starting gate in o.e.c.ClusterServiceIT 2015-12-28 08:13:55 -05:00
Jason Tedor 35cc749c9a Correctly release threads from starting gate in o.e.c.c.CacheTests 2015-12-28 08:07:37 -05:00
Simon Willnauer e8daad66d1 apply more feedback from @dakrone 2015-12-28 12:52:38 +01:00
Simon Willnauer a20b07d581 Merge branch 'master' into trash_recovery_threads 2015-12-28 12:45:14 +01:00
Simon Willnauer 0a816cd343 Add tests for equals and hashCode and fix FiltersFunctionScoreQuery equals and hashCode impls
Relates to #15676
2015-12-28 12:34:34 +01:00
Adrien Grand baaeb0f578 Merge pull request #15664 from jpountz/remove/ParseContext_ignoredValue
Remove ParseContext.ignoredValue.
2015-12-28 12:05:26 +01:00
Martijn van Groningen 4a0ec0da26 Merge remote-tracking branch 'es/master' into feature/ingest 2015-12-24 15:34:20 +01:00
Adrien Grand 23f736a0c1 Remove ParseContext.ignoredValue.
This API is unused.
2015-12-24 15:28:09 +01:00
Martijn van Groningen 1936d6118d Added index template for the '.ingest' index and added logic that ensure the index template is installed.
An index template for the '.ingest' index is required because:
* We don't want arbitrary fields in pipeline documents, because that can turn into upgrade problems if we add more properties to the pipeline dsl.
* We know what are the usages are of the '.ingest' index, so we can optimize for that and prevent that this index is used for different purposes.

Closes #15001
2015-12-24 15:18:07 +01:00
socurites 485915bbe7 comma(,) was duplicated
deleted it.
2015-12-24 14:31:26 +01:00
socurites 25d23091e2 Edge NGram: "side" setting was depercated
Edge NGram: "side" setting was depercated
2015-12-24 14:26:24 +01:00
Adrien Grand e2fbdcfb4f Merge pull request #15633 from jpountz/fix/dynamic_mappings
Improve cross-type dynamic mapping updates.
2015-12-24 10:24:08 +01:00
Adrien Grand 3015eb3088 Improve cross-type dynamic mapping updates.
Today when dynamically mapping a field that is already defined in another type,
we use the regular dynamic mapping logic and try to copy some settings to avoid
introducing conflicts. However this is quite fragile as we don't deal with every
existing setting. This proposes a different approach that will just reuse the
shared field type.

Close #15568
2015-12-24 10:22:35 +01:00
Adrien Grand 79b14215aa Merge pull request #15636 from jpountz/feature/bool_fields
Add sub-fields support to `bool` fields.
2015-12-24 10:21:49 +01:00
Robert Muir d144ba24a5 Merge pull request #15588 from rmuir/hdfs2-only
merge current hdfs improvements to master
2015-12-23 18:17:22 -05:00