Commit Graph

18837 Commits

Author SHA1 Message Date
Robert Muir 180ab2493e Improve thirdPartyAudit check, round 3 2015-12-28 22:38:55 -05:00
Yannick Welsch c6182cbd37 Fail replica shards before primary shards
As failing primaries also fail associated replicas, we must fail replicas first so that their nodes are properly added to ignore list

Closes #15686
2015-12-28 22:37:18 +01:00
Simon Willnauer 387bdbd322 Don't limit recoveries in RoutingTableTests 2015-12-28 21:36:34 +01:00
Adrien Grand bffbad3c19 Remove object notation for core types.
When specifying a string field, you can either do:

```
{
  "foo": "bar"
}
```

or

```
{
  "foo": {
    "value": "bar",
    "boost": 42
  }
}
```

The latter option is now removed.

Closes #15388
2015-12-28 19:49:18 +01:00
Yannick Welsch 24ab66c694 Merge pull request #15678 from ywelsch/enhance/faster-balancer
Speed improvements for BalancedShardsAllocator
2015-12-28 19:38:11 +01:00
Yannick Welsch 7e134da7d3 Cache result of RoutingNodes.node(...) in ModelNode 2015-12-28 19:14:48 +01:00
Yannick Welsch eea791de15 Remove superfluous method numPrimaries 2015-12-28 19:14:48 +01:00
Yannick Welsch e855282258 Simplify numShards calculation 2015-12-28 19:14:48 +01:00
Yannick Welsch 1536d7fe37 For rebalancing an index, only consider nodes that currently have a shard of that index or where the index can be allocated
This allows to prune a large number of nodes in case of hot/warm setup
2015-12-28 19:14:48 +01:00
Yannick Welsch 207dfc457d Remove superfluous method 2015-12-28 19:14:48 +01:00
Yannick Welsch fc0a33be05 Eliminate adding/removing shard to simulate weight of added shard / removed shard
Removal of the pattern node.addShard() -> calculate weight -> node.removeShard() which is expensive as, beside map lookups, it invalidates caching of precomputed values in ModelNode and ModelIndex. Replaced by adding an additional parameter to the weight function which accounts for the added / removed shard.
2015-12-28 19:14:48 +01:00
Yannick Welsch 5bd31a6cca Fix typo in method name 2015-12-28 19:14:48 +01:00
Yannick Welsch 67905b384f Remove unused class 2015-12-28 19:14:48 +01:00
Yannick Welsch 1bb7ca8a85 Removed unused methods 2015-12-28 19:14:48 +01:00
Yannick Welsch 0fd19008b1 Precalculate avgShardsPerNode 2015-12-28 19:14:48 +01:00
Yannick Welsch 507bb11345 Split two-element array into proper variables 2015-12-28 19:14:48 +01:00
Adrien Grand c6cf84336f Merge pull request #15679 from jpountz/fix/text_parsing
Make text parsing less lenient.
2015-12-28 16:47:05 +01:00
Adrien Grand 5eb7555ffb Make text parsing less lenient.
It now requires that the parser is on a value.
2015-12-28 16:06:42 +01:00
Simon Willnauer ba755a554f Merge pull request #15372 from s1monw/trash_recovery_threads
Remove recovery threadpools and throttle outgoing recoveries on the master
2015-12-28 15:43:14 +01:00
Simon Willnauer dcb164db3b Merge pull request #15598 from lks21c/master
Put space into last comment line
2015-12-28 15:41:10 +01:00
Jason Tedor 84c4ab6c18 Correctly release threads from starting gate in o.e.c.ClusterServiceIT 2015-12-28 08:13:55 -05:00
Jason Tedor 35cc749c9a Correctly release threads from starting gate in o.e.c.c.CacheTests 2015-12-28 08:07:37 -05:00
Simon Willnauer e8daad66d1 apply more feedback from @dakrone 2015-12-28 12:52:38 +01:00
Simon Willnauer a20b07d581 Merge branch 'master' into trash_recovery_threads 2015-12-28 12:45:14 +01:00
Simon Willnauer 0a816cd343 Add tests for equals and hashCode and fix FiltersFunctionScoreQuery equals and hashCode impls
Relates to #15676
2015-12-28 12:34:34 +01:00
Adrien Grand baaeb0f578 Merge pull request #15664 from jpountz/remove/ParseContext_ignoredValue
Remove ParseContext.ignoredValue.
2015-12-28 12:05:26 +01:00
Martijn van Groningen 4a0ec0da26 Merge remote-tracking branch 'es/master' into feature/ingest 2015-12-24 15:34:20 +01:00
Adrien Grand 23f736a0c1 Remove ParseContext.ignoredValue.
This API is unused.
2015-12-24 15:28:09 +01:00
Martijn van Groningen 1936d6118d Added index template for the '.ingest' index and added logic that ensure the index template is installed.
An index template for the '.ingest' index is required because:
* We don't want arbitrary fields in pipeline documents, because that can turn into upgrade problems if we add more properties to the pipeline dsl.
* We know what are the usages are of the '.ingest' index, so we can optimize for that and prevent that this index is used for different purposes.

Closes #15001
2015-12-24 15:18:07 +01:00
socurites 485915bbe7 comma(,) was duplicated
deleted it.
2015-12-24 14:31:26 +01:00
socurites 25d23091e2 Edge NGram: "side" setting was depercated
Edge NGram: "side" setting was depercated
2015-12-24 14:26:24 +01:00
Adrien Grand af122f4151 Remove mapping backward compatibilit with pre-2.0.
This removes the backward compatibility layer with pre-2.0 indices, notably
the extraction of _id, _routing or _timestamp from the source document when a
path is defined.
2015-12-24 13:47:37 +01:00
Adrien Grand e2fbdcfb4f Merge pull request #15633 from jpountz/fix/dynamic_mappings
Improve cross-type dynamic mapping updates.
2015-12-24 10:24:08 +01:00
Adrien Grand 3015eb3088 Improve cross-type dynamic mapping updates.
Today when dynamically mapping a field that is already defined in another type,
we use the regular dynamic mapping logic and try to copy some settings to avoid
introducing conflicts. However this is quite fragile as we don't deal with every
existing setting. This proposes a different approach that will just reuse the
shared field type.

Close #15568
2015-12-24 10:22:35 +01:00
Adrien Grand 79b14215aa Merge pull request #15636 from jpountz/feature/bool_fields
Add sub-fields support to `bool` fields.
2015-12-24 10:21:49 +01:00
Robert Muir d144ba24a5 Merge pull request #15588 from rmuir/hdfs2-only
merge current hdfs improvements to master
2015-12-23 18:17:22 -05:00
Robert Muir f14a21639c add cleanups from simon 2015-12-23 18:15:33 -05:00
Jason Tedor 6ab922c5a0 Reduce runtime of CidrsTests#testValidCombinations
This commit reduces the running time of CidrsTests#testValidCombinations
by hoisting some costly repeated operations outside of the inner test
loop.
2015-12-23 18:02:41 -05:00
Adrien Grand f40b72633b Add sub-fields support to `bool` fields.
`bool` is our only core mapper that does not support sub fields.

Close #6587
2015-12-23 17:45:46 +01:00
Martijn van Groningen 767114adec Merge remote-tracking branch 'es/master' into feature/ingest 2015-12-23 15:16:20 +01:00
Adrien Grand 76fa9023b6 Merge pull request #15488 from jpountz/remove/index_name_back_compat
Remove `index_name` back compat.
2015-12-23 14:55:41 +01:00
Adrien Grand d8d8666877 Remove `index_name` back compat.
Since 2.0 we enforce that fields have the same full and index names. So in 3.x
we can remove the ability to have different names on the same field.
2015-12-23 14:55:26 +01:00
Adrien Grand b771daa3c7 Merge pull request #15602 from jpountz/enhancement/function_score_two_phase
FunctionScoreQuery should implement two-phase iteration.
2015-12-23 14:54:23 +01:00
Alexander Reelsen 1e2919b03d Smoke Tester: Install mapper attachment & repository hdfs plugin 2015-12-23 13:26:23 +01:00
Alexander Reelsen a0b946ccab Smoke Tester: Dont install groovy/expression plugins
Those two plugins are modules now and thus dont need to be installed
explicitely when testing.
2015-12-23 13:15:56 +01:00
Adrien Grand 07658f58a8 FunctionScoreQuery should implement two-phase iteration.
FunctionScoreQuery should do two things that it doesn't do today:
 - propagate the two-phase iterator from the wrapped scorer so that things are
   still executed efficiently eg. if a phrase or geo-distance query is wrapped
 - filter out docs that don't have a high enough score using two-phase
   iteration: this way the score is only checked when everything else matches

While doing these changes, I noticed that minScore was ignored when scores were
not needed and that explain did not take it into account, so I fixed these
issues as well.
2015-12-23 12:02:21 +01:00
Alexander Reelsen 9cd618244a Docs: Remove deprecated documentation flag in profile docs 2015-12-23 10:59:38 +01:00
Adrien Grand 56d2dd701e Fix SizeMappingTests failure. 2015-12-23 10:48:00 +01:00
Adrien Grand a2072fe927 Merge pull request #15539 from jpountz/fix/immutable_document_mapper
Make mapping updates more robust.
2015-12-23 09:55:42 +01:00
Adrien Grand f535c27024 Make mapping updates more robust.
This changes a couple of things:

Mappings are truly immutable. Before, each field mapper stored a
MappedFieldTypeReference that was shared across fields that have the same name
across types. This means that a mapping update could have the side-effect of
changing the field type in other types when updateAllTypes is true. This works
differently now: after a mapping update, a new copy of the mappings is created
in such a way that fields across different types have the same MappedFieldType.
See the new Mapper.updateFieldType API which replaces MappedFieldTypeReference.

DocumentMapper is now immutable and MapperService.merge has been refactored in
such a way that if an exception is thrown while eg. lookup structures are being
updated, then the whole mapping update will be aborted. As a consequence,
FieldTypeLookup's checkCompatibility has been folded into copyAndAddAll.

Synchronization was simplified: given that mappings are truly immutable, we
don't need the read/write lock so that no documents can be parsed while a
mapping update is being processed. Document parsing is not performed under a
lock anymore, and mapping merging uses a simple synchronized block.
2015-12-23 09:55:07 +01:00