Added minimal docs for reindex api in java-api docs

Additionally:
* Included the existing update by query java api docs in java-api docs.
  (for some reason it was never included, it needed some tweaking and
  then it was good to go)
* moved delete-by-query / update-by-query code samples to java file so
  that we can verify that these samples at least compile.

Closes #24203
This commit is contained in:
Martijn van Groningen 2018-03-15 20:36:00 +01:00
parent 4897e0034b
commit 069a876542
No known key found for this signature in database
GPG Key ID: AB236F4FCF2AF12A
6 changed files with 255 additions and 117 deletions

View File

@ -7,12 +7,14 @@ This section describes the following CRUD APIs:
* <<java-docs-index>>
* <<java-docs-get>>
* <<java-docs-delete>>
* <<java-docs-delete-by-query>>
* <<java-docs-update>>
.Multi-document APIs
* <<java-docs-multi-get>>
* <<java-docs-bulk>>
* <<java-docs-reindex>>
* <<java-docs-update-by-query>>
* <<java-docs-delete-by-query>>
NOTE: All CRUD APIs are single-index APIs. The `index` parameter accepts a single
index name, or an `alias` which points to a single index.
@ -28,3 +30,7 @@ include::docs/update.asciidoc[]
include::docs/multi-get.asciidoc[]
include::docs/bulk.asciidoc[]
include::docs/update-by-query.asciidoc[]
include::docs/reindex.asciidoc[]

View File

@ -20,15 +20,9 @@ For more information on the delete operation, check out the
The delete by query API allows one to delete a given set of documents based on
the result of a query:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
BulkByScrollResponse response =
DeleteByQueryAction.INSTANCE.newRequestBuilder(client)
.filter(QueryBuilders.matchQuery("gender", "male")) <1>
.source("persons") <2>
.get(); <3>
long deleted = response.getDeleted(); <4>
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[delete-by-query-sync]
--------------------------------------------------
<1> query
<2> index
@ -38,21 +32,9 @@ long deleted = response.getDeleted(); <4>
As it can be a long running operation, if you wish to do it asynchronously, you can call `execute` instead of `get`
and provide a listener like:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
DeleteByQueryAction.INSTANCE.newRequestBuilder(client)
.filter(QueryBuilders.matchQuery("gender", "male")) <1>
.source("persons") <2>
.execute(new ActionListener<BulkByScrollResponse>() { <3>
@Override
public void onResponse(BulkByScrollResponse response) {
long deleted = response.getDeleted(); <4>
}
@Override
public void onFailure(Exception e) {
// Handle the exception
}
});
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[delete-by-query-async]
--------------------------------------------------
<1> query
<2> index

View File

@ -0,0 +1,11 @@
[[java-docs-reindex]]
=== Reindex API
See {ref}/docs-reindex.html[reindex API].
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[reindex1]
--------------------------------------------------
<1> Optionally a query can provided to filter what documents should be
re-indexed from the source to the target index.

View File

@ -1,18 +1,13 @@
[[docs-update-by-query]]
== Update By Query API
[[java-docs-update-by-query]]
=== Update By Query API
The simplest usage of `updateByQuery` updates each
document in an index without changing the source. This usage enables
<<picking-up-a-new-property,picking up a new property>> or another online
mapping change.
picking up a new property or another online mapping change.
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index").abortOnVersionConflict(false);
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query]
--------------------------------------------------
Calls to the `updateByQuery` API start by getting a snapshot of the index, indexing
@ -41,78 +36,50 @@ The `UpdateByQueryRequestBuilder` API supports filtering the updated documents,
limiting the total number of documents to update, and updating documents
with a script:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index")
.filter(termQuery("level", "awesome"))
.size(1000)
.script(new Script("ctx._source.awesome = 'absolutely'", ScriptType.INLINE, "painless", emptyMap()));
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-filter]
--------------------------------------------------
`UpdateByQueryRequestBuilder` also enables direct access to the query used
to select the documents. You can use this access to change the default scroll size or
otherwise modify the request for matching documents.
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index")
.source().setSize(500);
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-size]
--------------------------------------------------
You can also combine `size` with sorting to limit the documents updated:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index").size(100)
.source().addSort("cat", SortOrder.DESC);
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-sort]
--------------------------------------------------
In addition to changing the `_source` field for the document, you can use a
script to change the action, similar to the Update API:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index")
.script(new Script(
"if (ctx._source.awesome == 'absolutely) {"
+ " ctx.op='noop'
+ "} else if (ctx._source.awesome == 'lame') {"
+ " ctx.op='delete'"
+ "} else {"
+ "ctx._source.awesome = 'absolutely'}", ScriptType.INLINE, "painless", emptyMap()));
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-script]
--------------------------------------------------
As in the <<docs-update,Update API>>, you can set the value of `ctx.op` to change the
As in the <<java-docs-update,Update API>>, you can set the value of `ctx.op` to change the
operation that executes:
`noop`::
Set `ctx.op = "noop"` if your script doesn't make any
changes. The `updateByQuery` operaton then omits that document from the updates.
This behavior increments the `noop` counter in the
<<docs-update-by-query-response-body, response body>>.
This behavior increments the `noop` counter in the response body.
`delete`::
Set `ctx.op = "delete"` if your script decides that the document must be
deleted. The deletion will be reported in the `deleted` counter in the
<<docs-update-by-query-response-body, response body>>.
response body.
Setting `ctx.op` to any other value generates an error. Setting any
other field in `ctx` generates an error.
@ -123,79 +90,55 @@ from its original location.
You can also perform these operations on multiple indices and types at once, similar to the search API:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("foo", "bar").source().setTypes("a", "b");
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-multi-index]
--------------------------------------------------
If you provide a `routing` value then the process copies the routing value to the scroll query,
limiting the process to the shards that match that routing value:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source().setRouting("cat");
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-routing]
--------------------------------------------------
`updateByQuery` can also use the <<ingest>> feature by
`updateByQuery` can also use the ingest node by
specifying a `pipeline` like this:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.setPipeline("hurray");
BulkByScrollResponse response = updateByQuery.get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-pipeline]
--------------------------------------------------
[float]
[[docs-update-by-query-task-api]]
[[java-docs-update-by-query-task-api]]
=== Works with the Task API
You can fetch the status of all running update-by-query requests with the
<<tasks,Task API>>:
You can fetch the status of all running update-by-query requests with the Task API:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
ListTasksResponse tasksList = client.admin().cluster().prepareListTasks()
.setActions(UpdateByQueryAction.NAME).setDetailed(true).get();
for (TaskInfo info: tasksList.getTasks()) {
TaskId taskId = info.getTaskId();
BulkByScrollTask.Status status = (BulkByScrollTask.Status) info.getStatus();
// do stuff
}
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-list-tasks]
--------------------------------------------------
With the `TaskId` shown above you can look up the task directly:
// provide API Example
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
GetTaskResponse get = client.admin().cluster().prepareGetTask(taskId).get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-get-task]
--------------------------------------------------
[float]
[[docs-update-by-query-cancel-task-api]]
[[java-docs-update-by-query-cancel-task-api]]
=== Works with the Cancel Task API
Any Update By Query can be canceled using the <<tasks,Task Cancel API>>:
Any Update By Query can be canceled using the Task Cancel API:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
// Cancel all update-by-query requests
client.admin().cluster().prepareCancelTasks().setActions(UpdateByQueryAction.NAME).get().getTasks()
// Cancel a specific update-by-query request
client.admin().cluster().prepareCancelTasks().setTaskId(taskId).get().getTasks()
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-cancel-task]
--------------------------------------------------
Use the `list tasks` API to find the value of `taskId`.
@ -204,14 +147,14 @@ Cancelling a request is typically a very fast process but can take up to a few s
The task status API continues to list the task until the cancellation is complete.
[float]
[[docs-update-by-query-rethrottle]]
[[java-docs-update-by-query-rethrottle]]
=== Rethrottling
Use the `_rethrottle` API to change the value of `requests_per_second` on a running update:
[source,java]
["source","java",subs="attributes,callouts,macros"]
--------------------------------------------------
RethrottleAction.INSTANCE.newRequestBuilder(client).setTaskId(taskId).setRequestsPerSecond(2.0f).get();
include-tagged::{client-reindex-tests}/ReindexDocumentationIT.java[update-by-query-rethrottle]
--------------------------------------------------
Use the `list tasks` API to find the value of `taskId`.

View File

@ -132,6 +132,8 @@ and add it as a dependency. As an example, we will use the `slf4j-simple` logger
:client-tests: {docdir}/../../server/src/test/java/org/elasticsearch/client/documentation
:client-reindex-tests: {docdir}/../../modules/reindex/src/test/java/org/elasticsearch/client/documentation
include::client.asciidoc[]
include::docs.asciidoc[]

View File

@ -0,0 +1,194 @@
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
package org.elasticsearch.client.documentation;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.admin.cluster.node.tasks.get.GetTaskResponse;
import org.elasticsearch.action.admin.cluster.node.tasks.list.ListTasksResponse;
import org.elasticsearch.client.Client;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.reindex.BulkByScrollResponse;
import org.elasticsearch.index.reindex.BulkByScrollTask;
import org.elasticsearch.index.reindex.DeleteByQueryAction;
import org.elasticsearch.index.reindex.ReindexAction;
import org.elasticsearch.index.reindex.ReindexRequest;
import org.elasticsearch.index.reindex.ReindexRequestBuilder;
import org.elasticsearch.index.reindex.RethrottleAction;
import org.elasticsearch.index.reindex.UpdateByQueryAction;
import org.elasticsearch.index.reindex.UpdateByQueryRequestBuilder;
import org.elasticsearch.script.Script;
import org.elasticsearch.script.ScriptType;
import org.elasticsearch.search.sort.SortOrder;
import org.elasticsearch.tasks.TaskId;
import org.elasticsearch.tasks.TaskInfo;
import org.elasticsearch.test.ESIntegTestCase;
import java.util.Collections;
public class ReindexDocumentationIT extends ESIntegTestCase {
public void reindex() {
Client client = client();
// tag::reindex1
BulkByScrollResponse response = ReindexAction.INSTANCE.newRequestBuilder(client)
.destination("target_index")
.filter(QueryBuilders.matchQuery("category", "xzy")) // <1>
.get();
// end::reindex1
}
public void updateByQuery() {
Client client = client();
{
// tag::update-by-query
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index").abortOnVersionConflict(false);
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query
}
{
// tag::update-by-query-filter
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index")
.filter(QueryBuilders.termQuery("level", "awesome"))
.size(1000)
.script(new Script(ScriptType.INLINE, "ctx._source.awesome = 'absolutely'", "painless", Collections.emptyMap()));
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-filter
}
{
// tag::update-by-query-size
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index")
.source().setSize(500);
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-size
}
{
// tag::update-by-query-sort
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index").size(100)
.source().addSort("cat", SortOrder.DESC);
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-sort
}
{
// tag::update-by-query-script
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("source_index")
.script(new Script(
ScriptType.INLINE,
"if (ctx._source.awesome == 'absolutely) {"
+ " ctx.op='noop'"
+ "} else if (ctx._source.awesome == 'lame') {"
+ " ctx.op='delete'"
+ "} else {"
+ "ctx._source.awesome = 'absolutely'}",
"painless",
Collections.emptyMap()));
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-script
}
{
// tag::update-by-query-multi-index
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source("foo", "bar").source().setTypes("a", "b");
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-multi-index
}
{
// tag::update-by-query-routing
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.source().setRouting("cat");
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-routing
}
{
// tag::update-by-query-pipeline
UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client);
updateByQuery.setPipeline("hurray");
BulkByScrollResponse response = updateByQuery.get();
// end::update-by-query-pipeline
}
{
// tag::update-by-query-list-tasks
ListTasksResponse tasksList = client.admin().cluster().prepareListTasks()
.setActions(UpdateByQueryAction.NAME).setDetailed(true).get();
for (TaskInfo info: tasksList.getTasks()) {
TaskId taskId = info.getTaskId();
BulkByScrollTask.Status status = (BulkByScrollTask.Status) info.getStatus();
// do stuff
}
// end::update-by-query-list-tasks
}
{
TaskId taskId = null;
// tag::update-by-query-get-task
GetTaskResponse get = client.admin().cluster().prepareGetTask(taskId).get();
// end::update-by-query-get-task
}
{
TaskId taskId = null;
// tag::update-by-query-cancel-task
// Cancel all update-by-query requests
client.admin().cluster().prepareCancelTasks().setActions(UpdateByQueryAction.NAME).get().getTasks();
// Cancel a specific update-by-query request
client.admin().cluster().prepareCancelTasks().setTaskId(taskId).get().getTasks();
// end::update-by-query-cancel-task
}
{
TaskId taskId = null;
// tag::update-by-query-rethrottle
RethrottleAction.INSTANCE.newRequestBuilder(client)
.setTaskId(taskId)
.setRequestsPerSecond(2.0f)
.get();
// end::update-by-query-rethrottle
}
}
public void deleteByQuery() {
Client client = client();
// tag::delete-by-query-sync
BulkByScrollResponse response = DeleteByQueryAction.INSTANCE.newRequestBuilder(client)
.filter(QueryBuilders.matchQuery("gender", "male")) // <1>
.source("persons") // <2>
.get(); // <3>
long deleted = response.getDeleted(); // <4>
// end::delete-by-query-sync
// tag::delete-by-query-async
DeleteByQueryAction.INSTANCE.newRequestBuilder(client)
.filter(QueryBuilders.matchQuery("gender", "male")) // <1>
.source("persons") // <2>
.execute(new ActionListener<BulkByScrollResponse>() { // <3>
@Override
public void onResponse(BulkByScrollResponse response) {
long deleted = response.getDeleted(); // <4>
}
@Override
public void onFailure(Exception e) {
// Handle the exception
}
});
// end::delete-by-query-async
}
}