honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[ML] all multiple wildcard values for GET Calendars, Events, and DELETE forecasts (#62563) (#62629) 

This commit adjusts the following APIs so now they not only support an `_all` case, but wildcard patterned Ids as well.

- `GET _ml/calendars/<calendar_id>/events`
- `GET _ml/calendars/<calendar_id>`
- `GET _ml/anomaly_detectors/<job_id>/model_snapshots/<snapshot_id>`
- `DELETE _ml/anomaly_detectors/<job_id>/_forecast/<forecast_id>`
This commit is contained in:
Benjamin Trent 2020-09-18 11:06:07 -04:00 committed by GitHub
parent 43ace5f80d
commit 0f142c6afc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 452 additions and 140 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
docs/reference/ml/anomaly-detection/apis/delete-forecast.asciidoc
View File

@ -6,7 +6,7 @@
<titleabbrev>Delete forecast</titleabbrev>
++++
Deletes forecasts from a {ml} job.
Deletes forecasts from a {ml} job.
[[ml-delete-forecast-request]]
== {api-request-title}
@ -27,12 +27,12 @@ Deletes forecasts from a {ml} job.
[[ml-delete-forecast-desc]]
== {api-description-title}
By default, forecasts are retained for 14 days. You can specify a different
By default, forecasts are retained for 14 days. You can specify a different
retention period with the `expires_in` parameter in the
<<ml-forecast,forecast jobs API>>. The delete forecast API enables you to delete
one or more forecasts before they expire.
NOTE: When you delete a job, its associated forecasts are deleted.
NOTE: When you delete a job, its associated forecasts are deleted.
For more information, see
{ml-docs}/ml-overview.html#ml-forecasting[Forecasting the future].
@ -42,26 +42,26 @@ For more information, see
`<forecast_id>`::
(Optional, string) A comma-separated list of forecast identifiers. If you do not
specify this optional parameter or if you specify `_all`, the API deletes all
forecasts from the job.
specify this optional parameter or if you specify `_all` or `*` the API deletes all
forecasts from the job.
`<job_id>`::
(Required, string)
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
[[ml-delete-forecast-query-parms]]
== {api-query-parms-title}
`allow_no_forecasts`::
(Optional, boolean) Specifies whether an error occurs when there are no
forecasts. In particular, if this parameter is set to `false` and there are no
forecasts. In particular, if this parameter is set to `false` and there are no
forecasts associated with the job, attempts to delete all forecasts return an
error. The default value is `true`.
`timeout`::
(Optional, <<time-units, time units>>) Specifies the period of time to wait
for the completion of the delete operation. When this period of time elapses,
(Optional, <<time-units, time units>>) Specifies the period of time to wait
for the completion of the delete operation. When this period of time elapses,
the API fails and returns an error. The default value is `30s`.
[[ml-delete-forecast-example]]

6
docs/reference/ml/anomaly-detection/apis/get-calendar-event.asciidoc
View File

@ -25,8 +25,10 @@ Retrieves information about the scheduled events in calendars.
[[ml-get-calendar-event-desc]]
== {api-description-title}
You can get scheduled event information for a single calendar or for all
calendars by using `_all`.
You can get scheduled event information for multiple calendars in a single
API request by using a comma-separated list of ids or a wildcard expression.
You can get scheduled event information for all calendars by using `_all`,
by specifying `*` as the `<calendar_id>`, or by omitting the `<calendar_id>`.
For more information, see
{ml-docs}/ml-calendars.html[Calendars and scheduled events].

8
docs/reference/ml/anomaly-detection/apis/get-calendar.asciidoc
View File

@ -25,10 +25,12 @@ Retrieves configuration information for calendars.
[[ml-get-calendar-desc]]
== {api-description-title}
You can get information for a single calendar or for all calendars by using
`_all`.
You can get information for multiple calendars in a single API request by using a
comma-separated list of ids or a wildcard expression. You can get
information for all calendars by using `_all`, by specifying `*` as the
`<calendar_id>`, or by omitting the `<calendar_id>`.
For more information, see
For more information, see
{ml-docs}/ml-calendars.html[Calendars and scheduled events].
[[ml-get-calendar-path-parms]]

6
docs/reference/ml/anomaly-detection/apis/get-snapshot.asciidoc
View File

@ -34,8 +34,10 @@ include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=job-id-anomaly-detection]
include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=snapshot-id]
+
--
If you do not specify this optional parameter, the API returns information about
all model snapshots.
You can multiple snapshots for a single job in a single API request
by using a comma-separated list of `<snapshot_id>` or a wildcard expression.
You can get all snapshots for all calendars by using `_all`,
by specifying `*` as the `<snapshot_id>`, or by omitting the `<snapshot_id>`.
--
[[ml-get-snapshot-request-body]]

2
x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/ml/action/GetCalendarEventsAction.java
View File

@ -128,7 +128,7 @@ public class GetCalendarEventsAction extends ActionType<GetCalendarEventsAction.
if (jobId != null && Strings.isAllOrWildcard(calendarId) == false) {
e = ValidateActions.addValidationError("If " + Job.ID.getPreferredName() + " is used " +
Calendar.ID.getPreferredName() + " must be '" + GetCalendarsAction.Request.ALL + "'", e);
Calendar.ID.getPreferredName() + " must be '" + GetCalendarsAction.Request.ALL + "' or '*'", e);
}
return e;
}

2
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ml/action/GetCalendarEventsActionRequestTests.java
View File

@ -53,7 +53,7 @@ public class GetCalendarEventsActionRequestTests extends AbstractSerializingTest
ActionRequestValidationException validationException = request.validate();
assertNotNull(validationException);
assertEquals("Validation Failed: 1: If job_id is used calendar_id must be '_all';", validationException.getMessage());
assertEquals("Validation Failed: 1: If job_id is used calendar_id must be '_all' or '*';", validationException.getMessage());
request = new GetCalendarEventsAction.Request("_all");
request.setJobId("foo");

85
x-pack/plugin/ml/qa/native-multi-node-tests/src/javaRestTest/java/org/elasticsearch/xpack/ml/integration/ForecastIT.java
View File

@ -187,7 +187,7 @@ public class ForecastIT extends MlNativeAutodetectIntegTestCase {
equalTo("Cannot run forecast: Forecast cannot be executed as job requires data to have been processed and modeled"));
}
public void testMemoryStatus() throws Exception {
public void testMemoryStatus() {
Detector.Builder detector = new Detector.Builder("mean", "value");
detector.setByFieldName("clientIP");
@ -287,6 +287,74 @@ public class ForecastIT extends MlNativeAutodetectIntegTestCase {
}
public void testDeleteWildCard() throws Exception {
Detector.Builder detector = new Detector.Builder("mean", "value");
TimeValue bucketSpan = TimeValue.timeValueHours(1);
AnalysisConfig.Builder analysisConfig = new AnalysisConfig.Builder(Collections.singletonList(detector.build()));
analysisConfig.setBucketSpan(bucketSpan);
DataDescription.Builder dataDescription = new DataDescription.Builder();
dataDescription.setTimeFormat("epoch");
Job.Builder job = new Job.Builder("forecast-it-test-delete-wildcard");
job.setAnalysisConfig(analysisConfig);
job.setDataDescription(dataDescription);
registerJob(job);
putJob(job);
openJob(job.getId());
long now = Instant.now().getEpochSecond();
long timestamp = now - 50 * bucketSpan.seconds();
List<String> data = new ArrayList<>();
while (timestamp < now) {
data.add(createJsonRecord(createRecord(timestamp, 10.0)));
data.add(createJsonRecord(createRecord(timestamp, 30.0)));
timestamp += bucketSpan.seconds();
}
postData(job.getId(), data.stream().collect(Collectors.joining()));
flushJob(job.getId(), false);
String forecastIdDefaultDurationDefaultExpiry = forecast(job.getId(), null, null);
String forecastIdDuration1HourNoExpiry = forecast(job.getId(), TimeValue.timeValueHours(1), TimeValue.ZERO);
String forecastId2Duration1HourNoExpiry = forecast(job.getId(), TimeValue.timeValueHours(1), TimeValue.ZERO);
String forecastId2Duration1HourNoExpiry2 = forecast(job.getId(), TimeValue.timeValueHours(1), TimeValue.ZERO);
waitForecastToFinish(job.getId(), forecastIdDefaultDurationDefaultExpiry);
waitForecastToFinish(job.getId(), forecastIdDuration1HourNoExpiry);
waitForecastToFinish(job.getId(), forecastId2Duration1HourNoExpiry);
waitForecastToFinish(job.getId(), forecastId2Duration1HourNoExpiry2);
closeJob(job.getId());
assertNotNull(getForecastStats(job.getId(), forecastIdDefaultDurationDefaultExpiry));
assertNotNull(getForecastStats(job.getId(), forecastIdDuration1HourNoExpiry));
assertNotNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry));
assertNotNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry2));
{
DeleteForecastAction.Request request = new DeleteForecastAction.Request(job.getId(),
forecastIdDefaultDurationDefaultExpiry.substring(0, forecastIdDefaultDurationDefaultExpiry.length() - 2) + "*"
+ ","
+ forecastIdDuration1HourNoExpiry);
AcknowledgedResponse response = client().execute(DeleteForecastAction.INSTANCE, request).actionGet();
assertTrue(response.isAcknowledged());
assertNull(getForecastStats(job.getId(), forecastIdDefaultDurationDefaultExpiry));
assertNull(getForecastStats(job.getId(), forecastIdDuration1HourNoExpiry));
assertNotNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry));
assertNotNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry2));
}
{
DeleteForecastAction.Request request = new DeleteForecastAction.Request(job.getId(), "*");
AcknowledgedResponse response = client().execute(DeleteForecastAction.INSTANCE, request).actionGet();
assertTrue(response.isAcknowledged());
assertNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry));
assertNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry2));
}
}
public void testDelete() throws Exception {
Detector.Builder detector = new Detector.Builder("mean", "value");
@ -317,6 +385,8 @@ public class ForecastIT extends MlNativeAutodetectIntegTestCase {
flushJob(job.getId(), false);
String forecastIdDefaultDurationDefaultExpiry = forecast(job.getId(), null, null);
String forecastIdDuration1HourNoExpiry = forecast(job.getId(), TimeValue.timeValueHours(1), TimeValue.ZERO);
String forecastId2Duration1HourNoExpiry = forecast(job.getId(), TimeValue.timeValueHours(1), TimeValue.ZERO);
String forecastId2Duration1HourNoExpiry2 = forecast(job.getId(), TimeValue.timeValueHours(1), TimeValue.ZERO);
waitForecastToFinish(job.getId(), forecastIdDefaultDurationDefaultExpiry);
waitForecastToFinish(job.getId(), forecastIdDuration1HourNoExpiry);
closeJob(job.getId());
@ -333,13 +403,11 @@ public class ForecastIT extends MlNativeAutodetectIntegTestCase {
forecastIdDefaultDurationDefaultExpiry + "," + forecastIdDuration1HourNoExpiry);
AcknowledgedResponse response = client().execute(DeleteForecastAction.INSTANCE, request).actionGet();
assertTrue(response.isAcknowledged());
}
{
ForecastRequestStats forecastStats = getForecastStats(job.getId(), forecastIdDefaultDurationDefaultExpiry);
assertNull(forecastStats);
ForecastRequestStats otherStats = getForecastStats(job.getId(), forecastIdDuration1HourNoExpiry);
assertNull(otherStats);
assertNull(getForecastStats(job.getId(), forecastIdDefaultDurationDefaultExpiry));
assertNull(getForecastStats(job.getId(), forecastIdDuration1HourNoExpiry));
assertNotNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry));
assertNotNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry2));
}
{
@ -354,6 +422,9 @@ public class ForecastIT extends MlNativeAutodetectIntegTestCase {
DeleteForecastAction.Request request = new DeleteForecastAction.Request(job.getId(), Metadata.ALL);
AcknowledgedResponse response = client().execute(DeleteForecastAction.INSTANCE, request).actionGet();
assertTrue(response.isAcknowledged());
assertNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry));
assertNull(getForecastStats(job.getId(), forecastId2Duration1HourNoExpiry2));
}
{

187
x-pack/plugin/ml/src/internalClusterTest/java/org/elasticsearch/xpack/ml/integration/JobResultsProviderIT.java
View File

@ -16,6 +16,7 @@ import org.elasticsearch.action.admin.indices.mapping.get.GetMappingsResponse;
import org.elasticsearch.action.bulk.BulkRequestBuilder;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.support.PlainActionFuture;
import org.elasticsearch.action.support.WriteRequest;
import org.elasticsearch.client.OriginSettingClient;
import org.elasticsearch.cluster.metadata.AliasMetadata;
@ -37,6 +38,7 @@ import org.elasticsearch.common.xcontent.XContentBuilder;
import org.elasticsearch.common.xcontent.XContentFactory;
import org.elasticsearch.threadpool.ThreadPool;
import org.elasticsearch.xpack.core.ClientHelper;
import org.elasticsearch.xpack.core.action.util.PageParams;
import org.elasticsearch.xpack.core.action.util.QueryPage;
import org.elasticsearch.xpack.core.ml.MlMetaIndex;
import org.elasticsearch.xpack.core.ml.MlMetadata;
@ -72,6 +74,7 @@ import org.elasticsearch.xpack.ml.utils.persistence.ResultsPersisterService;
import org.junit.Before;
import java.io.IOException;
import java.time.Instant;
import java.time.ZonedDateTime;
import java.util.ArrayList;
import java.util.Arrays;
@ -255,21 +258,69 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
calendars.add(new Calendar("cat foo calendar", Arrays.asList("cat", "foo"), null));
indexCalendars(calendars);
List<Calendar> queryResult = getCalendars("ted");
List<Calendar> queryResult = getCalendars(CalendarQueryBuilder.builder().jobId("ted"));
assertThat(queryResult, is(empty()));
queryResult = getCalendars("foo");
queryResult = getCalendars(CalendarQueryBuilder.builder().jobId("foo"));
assertThat(queryResult, hasSize(3));
Long matchedCount = queryResult.stream().filter(
c -> c.getId().equals("foo calendar") || c.getId().equals("foo bar calendar") || c.getId().equals("cat foo calendar"))
.count();
assertEquals(Long.valueOf(3), matchedCount);
queryResult = getCalendars("bar");
queryResult = getCalendars(CalendarQueryBuilder.builder().jobId("bar"));
assertThat(queryResult, hasSize(1));
assertEquals("foo bar calendar", queryResult.get(0).getId());
}
public void testGetCalandarById() throws Exception {
List<Calendar> calendars = new ArrayList<>();
calendars.add(new Calendar("empty calendar", Collections.emptyList(), null));
calendars.add(new Calendar("foo calendar", Collections.singletonList("foo"), null));
calendars.add(new Calendar("foo bar calendar", Arrays.asList("foo", "bar"), null));
calendars.add(new Calendar("cat calendar", Collections.singletonList("cat"), null));
calendars.add(new Calendar("cat foo calendar", Arrays.asList("cat", "foo"), null));
indexCalendars(calendars);
List<Calendar> queryResult = getCalendars(CalendarQueryBuilder.builder()
.calendarIdTokens(new String[]{"foo*"})
.sort(true));
assertThat(queryResult, hasSize(2));
assertThat(queryResult.get(0).getId(), equalTo("foo bar calendar"));
assertThat(queryResult.get(1).getId(), equalTo("foo calendar"));
queryResult = getCalendars(CalendarQueryBuilder.builder()
.calendarIdTokens(new String[]{"foo calendar", "cat calendar"})
.sort(true));
assertThat(queryResult, hasSize(2));
assertThat(queryResult.get(0).getId(), equalTo("cat calendar"));
assertThat(queryResult.get(1).getId(), equalTo("foo calendar"));
}
public void testGetCalendarByIdAndPaging() throws Exception {
List<Calendar> calendars = new ArrayList<>();
calendars.add(new Calendar("empty calendar", Collections.emptyList(), null));
calendars.add(new Calendar("foo calendar", Collections.singletonList("foo"), null));
calendars.add(new Calendar("foo bar calendar", Arrays.asList("foo", "bar"), null));
calendars.add(new Calendar("cat calendar", Collections.singletonList("cat"), null));
calendars.add(new Calendar("cat foo calendar", Arrays.asList("cat", "foo"), null));
indexCalendars(calendars);
List<Calendar> queryResult = getCalendars(CalendarQueryBuilder.builder()
.calendarIdTokens(new String[]{"foo*"})
.pageParams(new PageParams(0, 1))
.sort(true));
assertThat(queryResult, hasSize(1));
assertThat(queryResult.get(0).getId(), equalTo("foo bar calendar"));
queryResult = getCalendars(CalendarQueryBuilder.builder()
.calendarIdTokens(new String[]{"foo calendar", "cat calendar"})
.sort(true)
.pageParams(new PageParams(1, 1)));
assertThat(queryResult, hasSize(1));
assertThat(queryResult.get(0).getId(), equalTo("foo calendar"));
}
public void testUpdateCalendar() throws Exception {
MlMetadata.Builder mlBuilder = new MlMetadata.Builder();
mlBuilder.putJob(createJob("foo").build(), false);
@ -319,7 +370,7 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
throw exceptionHolder.get();
}
List<Calendar> updatedCalendars = getCalendars(null);
List<Calendar> updatedCalendars = getCalendars(CalendarQueryBuilder.builder());
assertEquals(5, updatedCalendars.size());
for (Calendar cal: updatedCalendars) {
assertThat("bar", is(not(in(cal.getJobIds()))));
@ -341,7 +392,7 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
throw exceptionHolder.get();
}
updatedCalendars = getCalendars(null);
updatedCalendars = getCalendars(CalendarQueryBuilder.builder());
assertEquals(5, updatedCalendars.size());
for (Calendar cal: updatedCalendars) {
assertThat("bar", is(not(in(cal.getJobIds()))));
@ -384,16 +435,11 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
return aliasMetadataList.stream().map(AliasMetadata::alias).collect(Collectors.toSet());
}
private List<Calendar> getCalendars(String jobId) throws Exception {
private List<Calendar> getCalendars(CalendarQueryBuilder query) throws Exception {
CountDownLatch latch = new CountDownLatch(1);
AtomicReference<Exception> exceptionHolder = new AtomicReference<>();
AtomicReference<QueryPage<Calendar>> result = new AtomicReference<>();
CalendarQueryBuilder query = new CalendarQueryBuilder();
if (jobId != null) {
query.jobId(jobId);
}
jobProvider.calendars(query, ActionListener.wrap(
r -> {
result.set(r);
@ -455,7 +501,7 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
return calendarHolder.get();
}
public void testScheduledEvents() throws Exception {
public void testScheduledEventsForJobs() throws Exception {
Job.Builder jobA = createJob("job_a");
Job.Builder jobB = createJob("job_b");
Job.Builder jobC = createJob("job_c");
@ -504,6 +550,59 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
assertEquals(events.get(3), returnedEvents.get(1));
}
public void testScheduledEvents() throws Exception {
createJob("job_a");
createJob("job_b");
createJob("job_c");
String calendarAId = "maintenance_a";
List<Calendar> calendars = new ArrayList<>();
calendars.add(new Calendar(calendarAId, Collections.singletonList("job_a"), null));
ZonedDateTime now = ZonedDateTime.now();
List<ScheduledEvent> events = new ArrayList<>();
events.add(buildScheduledEvent("downtime", now.plusDays(1), now.plusDays(2), calendarAId));
events.add(buildScheduledEvent("downtime_AA", now.plusDays(8), now.plusDays(9), calendarAId));
events.add(buildScheduledEvent("downtime_AAA", now.plusDays(15), now.plusDays(16), calendarAId));
String calendarABId = "maintenance_a_and_b";
calendars.add(new Calendar(calendarABId, Arrays.asList("job_a", "job_b"), null));
events.add(buildScheduledEvent("downtime_AB", now.plusDays(12), now.plusDays(13), calendarABId));
indexCalendars(calendars);
indexScheduledEvents(events);
List<ScheduledEvent> returnedEvents = getScheduledEvents(new ScheduledEventsQueryBuilder());
assertEquals(4, returnedEvents.size());
assertEquals(events.get(0), returnedEvents.get(0));
assertEquals(events.get(1), returnedEvents.get(1));
assertEquals(events.get(3), returnedEvents.get(2));
assertEquals(events.get(2), returnedEvents.get(3));
returnedEvents = getScheduledEvents(ScheduledEventsQueryBuilder.builder().calendarIds(new String[]{"maintenance_a"}));
assertEquals(3, returnedEvents.size());
assertEquals(events.get(0), returnedEvents.get(0));
assertEquals(events.get(1), returnedEvents.get(1));
assertEquals(events.get(2), returnedEvents.get(2));
returnedEvents = getScheduledEvents(ScheduledEventsQueryBuilder.builder()
.calendarIds(new String[]{"maintenance_a", "maintenance_a_and_b"}));
assertEquals(4, returnedEvents.size());
assertEquals(events.get(0), returnedEvents.get(0));
assertEquals(events.get(1), returnedEvents.get(1));
assertEquals(events.get(3), returnedEvents.get(2));
assertEquals(events.get(2), returnedEvents.get(3));
returnedEvents = getScheduledEvents(ScheduledEventsQueryBuilder.builder()
.calendarIds(new String[]{"maintenance_a*"}));
assertEquals(4, returnedEvents.size());
assertEquals(events.get(0), returnedEvents.get(0));
assertEquals(events.get(1), returnedEvents.get(1));
assertEquals(events.get(3), returnedEvents.get(2));
assertEquals(events.get(2), returnedEvents.get(3));
}
public void testScheduledEventsForJob_withGroup() throws Exception {
String groupA = "group-a";
String groupB = "group-b";
@ -545,6 +644,49 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
.build();
}
public void testGetSnapshots() {
String jobId = "test_get_snapshots";
Job.Builder job = createJob(jobId);
indexModelSnapshot(new ModelSnapshot.Builder(jobId).setSnapshotId("snap_2")
.setTimestamp(Date.from(Instant.ofEpochMilli(10)))
.build());
indexModelSnapshot(new ModelSnapshot.Builder(jobId).setSnapshotId("snap_1")
.setTimestamp(Date.from(Instant.ofEpochMilli(11)))
.build());
indexModelSnapshot(new ModelSnapshot.Builder(jobId).setSnapshotId("other_snap")
.setTimestamp(Date.from(Instant.ofEpochMilli(12)))
.build());
client().admin().indices().prepareRefresh(AnomalyDetectorsIndex.jobStateIndexPattern(),
AnomalyDetectorsIndex.jobResultsAliasedName(jobId)).get();
PlainActionFuture<QueryPage<ModelSnapshot>> future = new PlainActionFuture<>();
jobProvider.modelSnapshots(jobId, 0, 4, "9", "15", "", false, "snap_2,snap_1", future::onResponse, future::onFailure);
List<ModelSnapshot> snapshots = future.actionGet().results();
assertThat(snapshots.get(0).getSnapshotId(), equalTo("snap_2"));
assertThat(snapshots.get(1).getSnapshotId(), equalTo("snap_1"));
future = new PlainActionFuture<>();
jobProvider.modelSnapshots(jobId, 0, 4, "9", "15", "", false, "snap_*", future::onResponse, future::onFailure);
snapshots = future.actionGet().results();
assertThat(snapshots.get(0).getSnapshotId(), equalTo("snap_2"));
assertThat(snapshots.get(1).getSnapshotId(), equalTo("snap_1"));
future = new PlainActionFuture<>();
jobProvider.modelSnapshots(jobId, 0, 4, "9", "15", "", false, "snap_*,other_snap", future::onResponse, future::onFailure);
snapshots = future.actionGet().results();
assertThat(snapshots.get(0).getSnapshotId(), equalTo("snap_2"));
assertThat(snapshots.get(1).getSnapshotId(), equalTo("snap_1"));
assertThat(snapshots.get(2).getSnapshotId(), equalTo("other_snap"));
future = new PlainActionFuture<>();
jobProvider.modelSnapshots(jobId, 0, 4, "9", "15", "", false, "*", future::onResponse, future::onFailure);
snapshots = future.actionGet().results();
assertThat(snapshots.get(0).getSnapshotId(), equalTo("snap_2"));
assertThat(snapshots.get(1).getSnapshotId(), equalTo("snap_1"));
assertThat(snapshots.get(2).getSnapshotId(), equalTo("other_snap"));
}
public void testGetAutodetectParams() throws Exception {
String jobId = "test_get_autodetect_params";
Job.Builder job = createJob(jobId, Arrays.asList("fruit", "tea"));
@ -664,6 +806,27 @@ public class JobResultsProviderIT extends MlSingleNodeTestCase {
return searchResultHolder.get().results();
}
private List<ScheduledEvent> getScheduledEvents(ScheduledEventsQueryBuilder query) throws Exception {
AtomicReference<Exception> errorHolder = new AtomicReference<>();
AtomicReference<QueryPage<ScheduledEvent>> searchResultHolder = new AtomicReference<>();
CountDownLatch latch = new CountDownLatch(1);
jobProvider.scheduledEvents(query, ActionListener.wrap(
params -> {
searchResultHolder.set(params);
latch.countDown();
}, e -> {
errorHolder.set(e);
latch.countDown();
}));
latch.await();
if (errorHolder.get() != null) {
throw errorHolder.get();
}
return searchResultHolder.get().results();
}
private Job.Builder createJob(String jobId) {
return createJob(jobId, Collections.emptyList());
}

23
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/action/TransportDeleteForecastAction.java
View File

@ -54,11 +54,11 @@ import org.elasticsearch.xpack.core.ml.job.results.ForecastRequestStats;
import org.elasticsearch.xpack.core.ml.job.results.ForecastRequestStats.ForecastRequestStatus;
import org.elasticsearch.xpack.core.ml.job.results.Result;
import org.elasticsearch.xpack.core.ml.utils.ExceptionsHelper;
import org.elasticsearch.xpack.ml.utils.QueryBuilderHelper;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.EnumSet;
import java.util.HashSet;
@ -96,22 +96,21 @@ public class TransportDeleteForecastAction extends HandledTransportAction<Delete
protected void doExecute(Task task, DeleteForecastAction.Request request, ActionListener<AcknowledgedResponse> listener) {
final String jobId = request.getJobId();
String forecastsExpression = request.getForecastId();
final String[] forecastIds = Strings.tokenizeToStringArray(forecastsExpression, ",");
final String forecastsExpression = request.getForecastId();
final String[] forecastIds = Strings.splitStringByCommaToArray(forecastsExpression);
ActionListener<SearchResponse> forecastStatsHandler = ActionListener.wrap(
searchResponse -> deleteForecasts(searchResponse, request, listener),
e -> listener.onFailure(new ElasticsearchException("An error occurred while searching forecasts to delete", e)));
SearchSourceBuilder source = new SearchSourceBuilder();
BoolQueryBuilder builder = QueryBuilders.boolQuery();
BoolQueryBuilder innerBool = QueryBuilders.boolQuery().must(
QueryBuilders.termQuery(Result.RESULT_TYPE.getPreferredName(), ForecastRequestStats.RESULT_TYPE_VALUE));
if (Strings.isAllOrWildcard(forecastIds) == false) {
innerBool.must(QueryBuilders.termsQuery(Forecast.FORECAST_ID.getPreferredName(), new HashSet<>(Arrays.asList(forecastIds))));
}
source.query(builder.filter(innerBool));
BoolQueryBuilder builder = QueryBuilders.boolQuery()
.filter(QueryBuilders.termQuery(Result.RESULT_TYPE.getPreferredName(), ForecastRequestStats.RESULT_TYPE_VALUE));
QueryBuilderHelper
.buildTokenFilterQuery(Forecast.FORECAST_ID.getPreferredName(), forecastIds)
.ifPresent(builder::filter);
source.query(builder);
SearchRequest searchRequest = new SearchRequest(AnomalyDetectorsIndex.jobResultsAliasedName(jobId));
searchRequest.source(source);
@ -143,7 +142,7 @@ public class TransportDeleteForecastAction extends HandledTransportAction<Delete
}
if (forecastsToDelete.isEmpty()) {
if (Strings.isAllOrWildcard(new String[]{request.getForecastId()}) &&
if (Strings.isAllOrWildcard(request.getForecastId()) &&
request.isAllowNoForecasts()) {
listener.onResponse(new AcknowledgedResponse(true));
} else {

3
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/action/TransportDeleteModelSnapshotAction.java
View File

@ -7,6 +7,7 @@ package org.elasticsearch.xpack.ml.action;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.message.ParameterizedMessage;
import org.elasticsearch.ResourceNotFoundException;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.support.ActionFilters;
@ -90,7 +91,7 @@ public class TransportDeleteModelSnapshotAction extends HandledTransportAction<D
deleteCandidate.getSnapshotId(), deleteCandidate.getDescription());
auditor.info(request.getJobId(), msg);
logger.debug("[{}] {}", request.getJobId(), msg);
logger.debug(() -> new ParameterizedMessage("[{}] {}", request.getJobId(), msg));
// We don't care about the bulk response, just that it succeeded
listener.onResponse(new AcknowledgedResponse(true));
}

30
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/action/TransportGetCalendarEventsAction.java
View File

@ -17,6 +17,7 @@ import org.elasticsearch.xpack.core.ml.action.GetCalendarEventsAction;
import org.elasticsearch.xpack.core.ml.calendars.ScheduledEvent;
import org.elasticsearch.xpack.core.ml.job.config.Job;
import org.elasticsearch.xpack.core.ml.utils.ExceptionsHelper;
import org.elasticsearch.xpack.ml.job.persistence.CalendarQueryBuilder;
import org.elasticsearch.xpack.ml.job.persistence.JobConfigProvider;
import org.elasticsearch.xpack.ml.job.persistence.JobResultsProvider;
import org.elasticsearch.xpack.ml.job.persistence.ScheduledEventsQueryBuilder;
@ -41,17 +42,15 @@ public class TransportGetCalendarEventsAction extends HandledTransportAction<Get
@Override
protected void doExecute(Task task, GetCalendarEventsAction.Request request,
ActionListener<GetCalendarEventsAction.Response> listener) {
final String[] calendarId = Strings.splitStringByCommaToArray(request.getCalendarId());
ActionListener<Boolean> calendarExistsListener = ActionListener.wrap(
r -> {
ScheduledEventsQueryBuilder query = new ScheduledEventsQueryBuilder()
.start(request.getStart())
.end(request.getEnd())
.from(request.getPageParams().getFrom())
.size(request.getPageParams().getSize());
if (Strings.isAllOrWildcard(request.getCalendarId()) == false) {
query.calendarIds(Collections.singletonList(request.getCalendarId()));
}
.start(request.getStart())
.end(request.getEnd())
.from(request.getPageParams().getFrom())
.size(request.getPageParams().getSize())
.calendarIds(calendarId);
ActionListener<QueryPage<ScheduledEvent>> eventsListener = ActionListener.wrap(
events -> {
@ -63,8 +62,8 @@ public class TransportGetCalendarEventsAction extends HandledTransportAction<Get
if (request.getJobId() != null) {
jobConfigProvider.getJob(request.getJobId(), ActionListener.wrap(
jobBuiler -> {
Job job = jobBuiler.build();
jobBuilder -> {
Job job = jobBuilder.build();
jobResultsProvider.scheduledEventsForJob(request.getJobId(), job.getGroups(), query, eventsListener);
},
@ -74,7 +73,10 @@ public class TransportGetCalendarEventsAction extends HandledTransportAction<Get
groupExists -> {
if (groupExists) {
jobResultsProvider.scheduledEventsForJob(
null, Collections.singletonList(request.getJobId()), query, eventsListener);
null,
Collections.singletonList(request.getJobId()),
query,
eventsListener);
} else {
listener.onFailure(ExceptionsHelper.missingJobException(request.getJobId()));
}
@ -89,16 +91,16 @@ public class TransportGetCalendarEventsAction extends HandledTransportAction<Get
},
listener::onFailure);
checkCalendarExists(request.getCalendarId(), calendarExistsListener);
checkCalendarExists(calendarId, calendarExistsListener);
}
private void checkCalendarExists(String calendarId, ActionListener<Boolean> listener) {
private void checkCalendarExists(String[] calendarId, ActionListener<Boolean> listener) {
if (Strings.isAllOrWildcard(calendarId)) {
listener.onResponse(true);
return;
}
jobResultsProvider.calendar(calendarId, ActionListener.wrap(
jobResultsProvider.calendars(CalendarQueryBuilder.builder().calendarIdTokens(calendarId), ActionListener.wrap(
c -> listener.onResponse(true),
listener::onFailure
));

36
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/action/TransportGetCalendarsAction.java
View File

@ -14,12 +14,9 @@ import org.elasticsearch.tasks.Task;
import org.elasticsearch.transport.TransportService;
import org.elasticsearch.xpack.core.ml.action.GetCalendarsAction;
import org.elasticsearch.xpack.core.action.util.PageParams;
import org.elasticsearch.xpack.core.action.util.QueryPage;
import org.elasticsearch.xpack.core.ml.calendars.Calendar;
import org.elasticsearch.xpack.ml.job.persistence.CalendarQueryBuilder;
import org.elasticsearch.xpack.ml.job.persistence.JobResultsProvider;
import java.util.Collections;
public class TransportGetCalendarsAction extends HandledTransportAction<GetCalendarsAction.Request, GetCalendarsAction.Response> {
@ -34,35 +31,18 @@ public class TransportGetCalendarsAction extends HandledTransportAction<GetCalen
@Override
protected void doExecute(Task task, GetCalendarsAction.Request request, ActionListener<GetCalendarsAction.Response> listener) {
final String calendarId = request.getCalendarId();
if (request.getCalendarId() != null && Strings.isAllOrWildcard(request.getCalendarId()) == false) {
getCalendar(calendarId, listener);
} else {
PageParams pageParams = request.getPageParams();
if (pageParams == null) {
pageParams = PageParams.defaultParams();
}
getCalendars(pageParams, listener);
final String[] calendarIds = Strings.splitStringByCommaToArray(request.getCalendarId());
PageParams pageParams = request.getPageParams();
if (pageParams == null) {
pageParams = PageParams.defaultParams();
}
getCalendars(calendarIds, pageParams, listener);
}
private void getCalendar(String calendarId, ActionListener<GetCalendarsAction.Response> listener) {
jobResultsProvider.calendar(calendarId, ActionListener.wrap(
calendar -> {
QueryPage<Calendar> page = new QueryPage<>(Collections.singletonList(calendar), 1, Calendar.RESULTS_FIELD);
listener.onResponse(new GetCalendarsAction.Response(page));
},
listener::onFailure
));
}
private void getCalendars(PageParams pageParams, ActionListener<GetCalendarsAction.Response> listener) {
CalendarQueryBuilder query = new CalendarQueryBuilder().pageParams(pageParams).sort(true);
private void getCalendars(String[] idTokens, PageParams pageParams, ActionListener<GetCalendarsAction.Response> listener) {
CalendarQueryBuilder query = new CalendarQueryBuilder().pageParams(pageParams).calendarIdTokens(idTokens).sort(true);
jobResultsProvider.calendars(query, ActionListener.wrap(
calendars -> {
listener.onResponse(new GetCalendarsAction.Response(calendars));
},
calendars -> listener.onResponse(new GetCalendarsAction.Response(calendars)),
listener::onFailure
));
}

16
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/action/TransportGetModelSnapshotsAction.java
View File

@ -7,6 +7,7 @@ package org.elasticsearch.xpack.ml.action;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.message.ParameterizedMessage;
import org.elasticsearch.action.ActionListener;
import org.elasticsearch.action.support.ActionFilters;
import org.elasticsearch.action.support.HandledTransportAction;
@ -40,10 +41,17 @@ public class TransportGetModelSnapshotsAction extends HandledTransportAction<Get
@Override
protected void doExecute(Task task, GetModelSnapshotsAction.Request request,
ActionListener<GetModelSnapshotsAction.Response> listener) {
logger.debug("Get model snapshots for job {} snapshot ID {}. from = {}, size = {}"
+ " start = '{}', end='{}', sort={} descending={}",
request.getJobId(), request.getSnapshotId(), request.getPageParams().getFrom(), request.getPageParams().getSize(),
request.getStart(), request.getEnd(), request.getSort(), request.getDescOrder());
logger.debug(
() -> new ParameterizedMessage(
"Get model snapshots for job {} snapshot ID {}. from = {}, size = {} start = '{}', end='{}', sort={} descending={}",
request.getJobId(),
request.getSnapshotId(),
request.getPageParams().getFrom(),
request.getPageParams().getSize(),
request.getStart(),
request.getEnd(),
request.getSort(),
request.getDescOrder()));
jobManager.jobExists(request.getJobId(), ActionListener.wrap(
ok -> {

35
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/job/persistence/CalendarQueryBuilder.java
View File

@ -5,13 +5,16 @@
*/
package org.elasticsearch.xpack.ml.job.persistence;
import org.elasticsearch.ResourceNotFoundException;
import org.elasticsearch.cluster.metadata.Metadata;
import org.elasticsearch.common.Strings;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermsQueryBuilder;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.xpack.core.action.util.PageParams;
import org.elasticsearch.xpack.core.ml.calendars.Calendar;
import org.elasticsearch.xpack.ml.utils.QueryBuilderHelper;
import java.util.ArrayList;
import java.util.Collections;
@ -23,6 +26,11 @@ public class CalendarQueryBuilder {
private String jobId;
private List<String> jobGroups = Collections.emptyList();
private boolean sort = false;
private String[] idTokens = new String[0];
public static CalendarQueryBuilder builder() {
return new CalendarQueryBuilder();
}
/**
* Page the query result
@ -49,6 +57,19 @@ public class CalendarQueryBuilder {
return this;
}
public CalendarQueryBuilder calendarIdTokens(String[] idTokens) {
this.idTokens = idTokens;
return this;
}
public boolean isForAllCalendars() {
return Strings.isAllOrWildcard(idTokens);
}
public Exception buildNotFoundException() {
return new ResourceNotFoundException("No calendar with id [" + Strings.arrayToCommaDelimitedString(idTokens) + "]");
}
/**
* Sort results by calendar_id
* @param sort Sort if true
@ -60,7 +81,8 @@ public class CalendarQueryBuilder {
}
public SearchSourceBuilder build() {
QueryBuilder qb;
BoolQueryBuilder qb = QueryBuilders.boolQuery()
.filter(QueryBuilders.termQuery(Calendar.TYPE.getPreferredName(), Calendar.CALENDAR_TYPE));
List<String> jobIdAndGroups = new ArrayList<>(jobGroups);
if (jobId != null) {
jobIdAndGroups.add(jobId);
@ -68,12 +90,11 @@ public class CalendarQueryBuilder {
if (jobIdAndGroups.isEmpty() == false) {
jobIdAndGroups.add(Metadata.ALL);
qb = new BoolQueryBuilder()
.filter(new TermsQueryBuilder(Calendar.TYPE.getPreferredName(), Calendar.CALENDAR_TYPE))
.filter(new TermsQueryBuilder(Calendar.JOB_IDS.getPreferredName(), jobIdAndGroups));
} else {
qb = new TermsQueryBuilder(Calendar.TYPE.getPreferredName(), Calendar.CALENDAR_TYPE);
qb.filter(new TermsQueryBuilder(Calendar.JOB_IDS.getPreferredName(), jobIdAndGroups));
}
QueryBuilderHelper
.buildTokenFilterQuery(Calendar.ID.getPreferredName(), idTokens)
.ifPresent(qb::filter);
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder().query(qb);

16
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/job/persistence/JobResultsProvider.java
View File

@ -1037,12 +1037,12 @@ public class JobResultsProvider {
String snapshotId,
Consumer<QueryPage<ModelSnapshot>> handler,
Consumer<Exception> errorHandler) {
ResultsFilterBuilder fb = new ResultsFilterBuilder();
if (snapshotId != null && !snapshotId.isEmpty()) {
fb.term(ModelSnapshotField.SNAPSHOT_ID.getPreferredName(), snapshotId);
}
String[] snapshotIds = Strings.splitStringByCommaToArray(snapshotId);
QueryBuilder qb = new ResultsFilterBuilder()
.resourceTokenFilters(ModelSnapshotField.SNAPSHOT_ID.getPreferredName(), snapshotIds)
.timeRange(Result.TIMESTAMP.getPreferredName(), startEpochMs, endEpochMs)
.build();
QueryBuilder qb = fb.timeRange(Result.TIMESTAMP.getPreferredName(), startEpochMs, endEpochMs).build();
modelSnapshots(jobId, from, size, sortField, sortDescending, qb, handler, errorHandler);
}
@ -1298,7 +1298,7 @@ public class JobResultsProvider {
handler.onResponse(new QueryPage<>(Collections.emptyList(), 0, ScheduledEvent.RESULTS_FIELD));
return;
}
List<String> calendarIds = calendars.results().stream().map(Calendar::getId).collect(Collectors.toList());
String[] calendarIds = calendars.results().stream().map(Calendar::getId).toArray(String[]::new);
queryBuilder.calendarIds(calendarIds);
scheduledEvents(queryBuilder, handler);
},
@ -1487,6 +1487,10 @@ public class JobResultsProvider {
List<Calendar> calendars = new ArrayList<>();
SearchHit[] hits = response.getHits().getHits();
try {
if (queryBuilder.isForAllCalendars() == false && hits.length == 0) {
listener.onFailure(queryBuilder.buildNotFoundException());
return;
}
for (SearchHit hit : hits) {
calendars.add(MlParserUtils.parse(hit, Calendar.LENIENT_PARSER).build());
}

6
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/job/persistence/ResultsFilterBuilder.java
View File

@ -12,6 +12,7 @@ import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.elasticsearch.xpack.core.ml.job.results.Result;
import org.elasticsearch.xpack.ml.utils.QueryBuilderHelper;
import java.util.ArrayList;
import java.util.List;
@ -88,6 +89,11 @@ public class ResultsFilterBuilder {
return this;
}
public ResultsFilterBuilder resourceTokenFilters(String fieldName, String[] tokens) {
QueryBuilderHelper.buildTokenFilterQuery(fieldName, tokens).ifPresent(this::addQuery);
return this;
}
public ResultsFilterBuilder resultType(String resultType) {
return term(Result.RESULT_TYPE.getPreferredName(), resultType);
}

46
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/job/persistence/ScheduledEventsQueryBuilder.java
View File

@ -6,16 +6,13 @@
package org.elasticsearch.xpack.ml.job.persistence;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.RangeQueryBuilder;
import org.elasticsearch.index.query.TermsQueryBuilder;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.xpack.core.ml.calendars.Calendar;
import org.elasticsearch.xpack.core.ml.calendars.ScheduledEvent;
import org.elasticsearch.xpack.ml.utils.QueryBuilderHelper;
import java.util.ArrayList;
import java.util.List;
/**
* Query builder for {@link ScheduledEvent}s
@ -27,11 +24,15 @@ public class ScheduledEventsQueryBuilder {
private Integer from = 0;
private Integer size = DEFAULT_SIZE;
private List<String> calendarIds;
private String[] calendarIds;
private String start;
private String end;
public ScheduledEventsQueryBuilder calendarIds(List<String> calendarIds) {
public static ScheduledEventsQueryBuilder builder() {
return new ScheduledEventsQueryBuilder();
}
public ScheduledEventsQueryBuilder calendarIds(String[] calendarIds) {
this.calendarIds = calendarIds;
return this;
}
@ -72,46 +73,31 @@ public class ScheduledEventsQueryBuilder {
}
public SearchSourceBuilder build() {
List<QueryBuilder> queries = new ArrayList<>();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery()
.filter(QueryBuilders.termQuery(ScheduledEvent.TYPE.getPreferredName(), ScheduledEvent.SCHEDULED_EVENT_TYPE));
if (start != null) {
RangeQueryBuilder startQuery = QueryBuilders.rangeQuery(ScheduledEvent.END_TIME.getPreferredName());
startQuery.gt(start);
queries.add(startQuery);
boolQueryBuilder.filter(startQuery);
}
if (end != null) {
RangeQueryBuilder endQuery = QueryBuilders.rangeQuery(ScheduledEvent.START_TIME.getPreferredName());
endQuery.lt(end);
queries.add(endQuery);
boolQueryBuilder.filter(endQuery);
}
if (calendarIds != null && calendarIds.isEmpty() == false) {
queries.add(new TermsQueryBuilder(Calendar.ID.getPreferredName(), calendarIds));
}
QueryBuilderHelper.buildTokenFilterQuery(Calendar.ID.getPreferredName(), calendarIds).ifPresent(boolQueryBuilder::filter);
QueryBuilder typeQuery = new TermsQueryBuilder(ScheduledEvent.TYPE.getPreferredName(), ScheduledEvent.SCHEDULED_EVENT_TYPE);
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.sort(ScheduledEvent.START_TIME.getPreferredName());
searchSourceBuilder.sort(ScheduledEvent.DESCRIPTION.getPreferredName());
SearchSourceBuilder searchSourceBuilder = SearchSourceBuilder.searchSource()
.sort(ScheduledEvent.START_TIME.getPreferredName())
.sort(ScheduledEvent.DESCRIPTION.getPreferredName())
.query(boolQueryBuilder);
if (from != null) {
searchSourceBuilder.from(from);
}
if (size != null) {
searchSourceBuilder.size(size);
}
if (queries.isEmpty()) {
searchSourceBuilder.query(typeQuery);
} else {
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
boolQueryBuilder.filter(typeQuery);
for (QueryBuilder query : queries) {
boolQueryBuilder.filter(query);
}
searchSourceBuilder.query(boolQueryBuilder);
}
return searchSourceBuilder;
}
}

61
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/utils/QueryBuilderHelper.java Normal file
View File

@ -0,0 +1,61 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
package org.elasticsearch.xpack.ml.utils;
import org.elasticsearch.common.Strings;
import org.elasticsearch.common.regex.Regex;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.TermsQueryBuilder;
import org.elasticsearch.index.query.WildcardQueryBuilder;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;
public final class QueryBuilderHelper {
private QueryBuilderHelper() { }
/**
* Helper function for adding OR type queries for a given identity field.
*
* The filter consists of should clauses (i.e. "or" boolean queries).
*
* - When a token is a wildcard token, a wildcard query is added
* - When a token is NOT a wildcard, a term query is added
*
* @param identityField The field to query for the tokens
* @param tokens A non-null collection of tokens. Can include wildcards
* @return An optional boolean query builder filled with "should" queries for the supplied tokens and identify field
*/
public static Optional<QueryBuilder> buildTokenFilterQuery(String identityField, String[] tokens) {
if (Strings.isAllOrWildcard(tokens)) {
return Optional.empty();
}
BoolQueryBuilder shouldQueries = new BoolQueryBuilder();
List<String> terms = new ArrayList<>();
for (String token : tokens) {
if (Regex.isSimpleMatchPattern(token)) {
shouldQueries.should(new WildcardQueryBuilder(identityField, token));
} else {
terms.add(token);
}
}
if (terms.isEmpty() == false) {
shouldQueries.should(new TermsQueryBuilder(identityField, terms));
}
if (shouldQueries.should().isEmpty()) {
return Optional.empty();
}
return Optional.of(shouldQueries);
}
}

4
x-pack/plugin/src/test/resources/rest-api-spec/test/ml/calendar_crud.yml
View File

@ -1,5 +1,9 @@
---
"Test calendar CRUD":
- do:
ml.get_calendars:
calendar_id: _all
- match: { count: 0 }
- do:
ml.put_job: