This commit adds the current total number of translog operations to the recovery reporting API. We also expose the recovered / total percentage:
```
"translog": {
"recovered": 536,
"total": 986,
"percent": "54.3%",
"total_time": "2ms",
"total_time_in_millis": 2
},
```
Closes#9368Closes#10042
To support the `_recovery` API, the recovery process keeps track of current progress in a class called RecoveryState. This class currently have some issues, mostly around concurrency (see #6644 ). This PR cleans it up as well as other issues around it:
- Make the Index subsection API cleaner:
- remove redundant information - all calculation is done based on the underlying file map
- clearer definition of what is what: total files, vs reused files (local files that match the source) vs recovered files (copied over). % based progress is reported based on recovered files only.
- cleaned up json response to match other API (sadly this breaks the structure). We now properly report human values for dates and other units.
- Add more robust unit testing
- Detail flag was passed along as state (it's now a ToXContent param)
- State lookup during reporting is now always done via the IndexShard , no more fall backs to many other classes.
- Cleanup APIs around time and move the little computations to the state class as opposed to doing them out of the API
I also improved error messages out of the REST testing infra for things I run into.
Closes#6644Closes#9811
Add source_node and target_node fields to the recovery cat API. Also fixed and updated the documentation which was not complete concerning fields names.
Closes#8041
This fixes a stack overflow in the test for the _cat/recovery API.
The regular expression that tests the response body was modified to
handle large responses properly.
The regex tests are formatted with blocks for readability. Previously,
they were formatted using folded style blocks (e.g. using `>`). Folded
blocks convert newlines into spaces. This is problematic for our regex,
since comments can only be terminated with a newline.
Effectively, anything after a comment will be commented out, making many
of the regex "silently pass".
This commit replaces them with scalar-style blocks (e.g. using `|`), which
treats newlines as significant, and thus correctly terminates comments
inside the regex.
Also fixes a regex test (`cat.thread_pool/10_basic.yaml`) that started
to fail after the block was fixed. The test was missing a `\s+` before
the closing newline.
Adds a new API endpoint at /_recovery as well as to the Java API. The
recovery API allows one to see the recovery status of all shards in the
cluster. It will report on percent complete, recovery type, and which
files are copied.
Closes#4637
Boaz Leskes
Lee Hinman
tlrx
Clinton Gormley
Andrew Selden
Zachary Tong
javanna
Simon Willnauer