Commit Graph

11041 Commits

Author SHA1 Message Date
Simon Willnauer 1d357cb48a Merge pull request #10157 from spinscale/1503-cleanup-remove-bytesarray-unsafe
Cleanup: Remove unsafe field in BytesStreamInput
2015-03-20 13:14:02 -07:00
Michael McCandless 34b397597c Core: increase default rate limiting for snapshot, restore and recovery to 40 MB/sec
This also fixes a possible issue that may cause over-throttling when
there are many small files being copied, which should be rare.

Closed #10185
2015-03-20 16:09:42 -04:00
Simon Willnauer 93fedcbb88 [RECOVERY] Wipe shard state before switching recovered files live
Today we leave the shard state behind even if a recovery is half finished
this causes in rare conditions shards to be recovered and promoted as
primaries that have never been fully recovered.

Closes #10053
2015-03-20 08:42:03 -07:00
Boaz Leskes 0f2d2d0495 Test: testCancelRecoveryAndResume - add network hook before bumping replicas 2015-03-20 16:24:36 +01:00
Boaz Leskes 7d64877326 remove left over recovery log 2015-03-20 16:10:26 +01:00
javanna 9ca74c8217 [DOCS] clarify no-master-block docs
Closes #9739
2015-03-20 15:58:16 +01:00
Boaz Leskes 8c69535580 Recovery: add throttle stats
This commit adds throttling statistics to the index stats API and to the recovery state.

Closes #10097
2015-03-20 10:13:40 +01:00
javanna 4348959f9d Delete api: remove broadcast delete if routing is missing when required
This commit changes the behaviour of the delete api when processing a delete request that refers to a type that has routing set to required in the mapping, and the routing is missing in the request. Up until now the delete api sent a broadcast delete request to all of the shards that belong to the index, making sure that the document could be found although the routing value wasn't specified. This was probably not the best choice: if the routing is set to required, an error should be thrown instead.

A `RoutingMissingException` gets now thrown instead, like it happens in the same situation with every other api (index, update, get etc.). Last but not least, this change allows to get rid of a couple of `TransportAction`s, `Request`s and `Response`s and simplify the codebase.

Closes #9123
Closes #10136
2015-03-20 09:19:43 +01:00
Simon Willnauer 780fe57a2c [TEST] Add more logging to TruncatedRecoveryTests 2015-03-19 23:17:48 -07:00
Lee Hinman f14e226e82 [TEST] Add additional logging to testCorruptTranslogFiles 2015-03-19 22:58:49 -06:00
Lee Hinman 25b09ede16 [TEST] Use less shards in testShadowReplicaNaturalRelocation
This makes the assertion a bit more flexible and removes the
`ensureGreen` in favor of `ensureYellow`, which is really all that is
needed to perform a search. On slow machines the relocations can take a
while and time out the `ensureGreen`.
2015-03-19 18:12:21 -06:00
Britta Weber f253542abb remove unused method. close() is neither needed nor called anywhere
closes #10175
2015-03-19 17:05:24 -07:00
Simon Willnauer 48388dbb6e apply feedback 2015-03-19 15:56:28 -07:00
Simon Willnauer d9be606c5e [LIFECYCLE] Add before/afterIndexShardDelete callback
This commit allows code to be executed before or after a shards content
is deleted from disk. This is only executed if the shard owns the
content ie. on a shard file system only a primary shard will execute
these operations.
2015-03-19 15:47:51 -07:00
Simon Willnauer a6897aa073 [GATEWAY] Move GatewayShardsState logic into IndexShard
The index shard should take care of shard state persistence since it has
all the information and the gateway concept has been removed in master.
2015-03-19 14:48:02 -07:00
Simon Willnauer 1168347b9d [REPLICATION] Remove `async` replication
Closes #10114
2015-03-19 14:44:21 -07:00
Boaz Leskes 955ff05a5e Transport: fix potential NPE in new tracer log if request timeout
Closes #9994
2015-03-19 21:29:34 +01:00
Clinton Gormley a8e5c6eeb0 Merge pull request #10169 from clintongormley/deprecate_thrift_memcached
Remove references to the thrift and memcached transport plugins
2015-03-19 20:56:25 +01:00
Clinton Gormley aa94ced0ae Remove references to the thrift and memcached transport plugins
as they are no longer supported

Closes #10166
2015-03-19 20:49:58 +01:00
olivier bourgain deade2eb71 return an HTTP code representing the error when a suggest request failed instead of 200 2015-03-19 13:00:31 -04:00
Clinton Gormley 6e9eea9b98 Deleted REST test pending shard header
Bug introduced in #10162
2015-03-19 17:57:25 +01:00
Clinton Gormley ef5bf9c21b Changed the release notes script to use the category labels instead of title prefix 2015-03-19 16:52:47 +01:00
Clinton Gormley 806e7b4763 Merge pull request #10162 from clintongormley/deprecate_async_master
Remove async replication from the docs and REST spec
2015-03-19 15:35:23 +01:00
Clinton Gormley 25369f0727 Remove async replication from the docs and REST spec
Relates to #10114
2015-03-19 15:34:12 +01:00
javanna f4691458d5 [DOCS] added note about dynamic scriptings and updated links in getting started page
Closes #10074
2015-03-19 15:22:39 +01:00
Alexander Reelsen 16efc50139 Cleanup: Remove unsafe field in BytesStreamInput
It is only used once (otherwise it is set to `false` anyway), so copying the array in that case makes more sense.
2015-03-19 13:05:59 +01:00
javanna ddcecc2bc2 [DOCS] added instructions on how to write parameterized tests
Closes #9423
2015-03-19 12:43:51 +01:00
Simon Willnauer 5d39bea91b [TEST] Add tests that triggers truncated files on recovery
Several issues where reported showing truncated files where footers
didn't match and checksums read past EOF. This test reproduces the issue
on the latest 1.4 branch but passes on all versions above.

Closes #10155
2015-03-19 01:03:47 -07:00
Nicholas Knize 9bd0c019f2 [TEST] Fix geo orientation integration test
Pass node name to getInstance method of internalCluster to ensure we return the indexService from a node containing the index.
2015-03-18 21:30:20 -05:00
Nicholas Knize 1a79e1c8cc [GEO] Fix orientation persistence
Fixing geo_shape field mapper to persist the orientation parameter. Also adding parsing and integration tests to ensure persistence across cluster restarts.
2015-03-18 18:55:49 -05:00
Christoph Büscher 5788289a6d [TEST] Remove 'gtelte' feature from yaml tests
Some yaml tests still used the 'gtelte' feature in their skip section.
Since all language clients support this, the feature skip can be
removed.
2015-03-18 15:31:17 -07:00
jaymode 105bdd486a [HTTP] add option to only return simple exception messages
Adds a setting to disable detailed error messages and full exception stack traces
in HTTP responses. When set to false, the error_trace request parameter will result
in a HTTP 400 response. When the error_trace parameter is not present, the message
of the first ElasticsearchException will be output and no nested exception messages
will be output.
2015-03-18 17:49:05 -04:00
Robert Muir 5f1ed47df6 Merge pull request #10143 from rmuir/bwcompat1
use bigger back compat indexes
2015-03-18 14:49:32 -04:00
Robert Muir 3a0700862a regenerate 0.90.x indexes without completion suggester 2015-03-18 14:29:41 -04:00
Robert Muir 087b107dd2 Create bigger back compat indexes. 2015-03-18 14:16:16 -04:00
Britta Weber dca0104061 [TEST] muste test, we have an issue for it and know what is going on 2015-03-18 10:17:38 -07:00
Britta Weber fc5dcf189c [release script] Check for //NORELEASE in code before release
Lines in the code that should be removed before a release can be annotated with
//NORELEASE . This can be useful when debugging test failures. For example,
one might want to add additional logging that would be too verbose for production
and therfore should be removed before releasing.

closes #10141
2015-03-18 09:59:28 -07:00
Michael McCandless a85b13773f Fix typo in UnavailableShardsException message
Closes #10140
2015-03-18 12:41:13 -04:00
javanna fdfdf594f9 [TEST] RepeatOnExceptionRule to rethrow unexpected exception
In case an exception was caught by the repeat rule, the retry mechanism would kick in only if the exception was the expected one. If not an NPE got thrown, while we should rather just bubble it up to the caller. This makes `NettyTransportMultiPortTests` run from a plane. An assumption would kick in to make sure that the test gets ignored but the `AssumptionViolationException` was caught and not properly re-thrown.
2015-03-18 14:08:46 +01:00
Simon Willnauer 56a2fb04ee [ALLOCATION] Early terminate if the cluster can't be rebalanced
By default we won't allow rebalance operation if no all shards are active.
if this is the case we don't need to worry about costly rebalance calculations at all.
2015-03-17 22:49:05 -07:00
Simon Willnauer 9ad1434a3b [TEST] Add test for restoring into a shadowreplica enabled index 2015-03-17 18:26:41 -07:00
Simon Willnauer 9554e509e3 [PLUGINS] Always send current ES version when downloading plugins
to enable download servers to send the correct plugin version for the
node that is installing it this PR sends the current version as a header
to the server.
2015-03-17 16:44:09 -07:00
Drew Raines e49aa2dee8 Merge pull request #9938 from drewr/issue/6297/cat/npe
_cat/nodes: Thread null handling through stats and info
2015-03-17 16:02:47 -07:00
Simon Willnauer ed7f6526b9 [TRANSPORT] Close all resources if doStart fails
Closes #9898
2015-03-17 14:40:37 -07:00
Joshua Rich db2caa54cd Small grammar fix. 2015-03-17 11:27:13 -07:00
Alexander Reelsen b8334513d7 Transport: Return useful error message on potential HTTP connect
In case a HTTP client connects to the transport protocol and issues a
HTTP method followed by a space, we can just try to be smart and return
a string back to the client to point the user to the fact that the wrong
port has been used.

Closes #2139
Closes #10108
2015-03-17 09:47:24 -07:00
Boaz Leskes b605184471 Recovery: add total operations to the `_recovery` API
This commit adds the current total number of translog operations to the recovery reporting API. We also expose the recovered / total percentage:

```
"translog": {
   "recovered": 536,
   "total": 986,
   "percent": "54.3%",
   "total_time": "2ms",
   "total_time_in_millis": 2
},
```

Closes #9368
Closes #10042
2015-03-17 07:31:29 -07:00
Simon Willnauer 73953142a7 [BUILD] fix snapshot URL 2015-03-16 22:59:49 -07:00
Martijn van Groningen ef3cb2d436 percolator: Take filters from index aliases into account when selecting queries to run on a document.
The filter from an indexed alias is as if you would filter on the metadata of a percolator query, but then the filter is defined in the index alias instead of the percolate request.

Closes #6241
2015-03-16 17:25:55 -07:00
Martijn van Groningen 4393939f5e inner_hits: Nested parent field should be resolved based on the parent inner hit definition, instead of the nested parent field in the mapping.
The behaviour is better in the case someone has multiple levels of nested object fields defined in the mapping and like to define a single inner_hits definition that is two or more levels deep.

If someone wants inner hits on a nested field that is 2 levels deep the following would need to be defined:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1" : {
            "inner_hits" : {
               "path" : {
                  "level2" : {
                     "query" : { .... }
                  }
               }
            }
        }
     }
  }
}
```

With this change the above can be defined as:

```
{
  ...
  "inner_hits" : {
     "path" : {
        "level1.level2" : {
            "query" : { .... }
        }
     }
  }
}
```

Closes #9251
2015-03-16 16:31:03 -07:00