[Rollup] Add new capabilities endpoint for concrete rollup indices (#30401)

This introduces a new GetRollupIndexCaps API which allows the user to retrieve rollup capabilities of a specific rollup index (or index pattern). This is distinct from the existing RollupCaps endpoint.

- Multiple jobs can be stored in multiple indices and point to a single target data index pattern (logstash-*). The existing API finds capabilities/config of all jobs matching that data index pattern.
- One rollup index can hold data from multiple jobs, targeting multiple data index patterns. This new API finds the capabilities based on the concrete rollup indices.

x-pack/docs/en/rest-api/rollup/rollup-index-caps.asciidoc

@@ -0,0 +1,161 @@
+[role="xpack"]
+[[rollup-get-rollup-index-caps]]
+=== Get Rollup Index Capabilities
+++++
+<titleabbrev>Get Rollup Index Caps</titleabbrev>
+++++
+
+experimental[]
+
+This API returns the rollup capabilities of all jobs inside of a rollup index (e.g. the index where rollup data is stored).
+A single rollup index may store the data for multiple rollup jobs, and may have a variety of capabilities depending on those jobs.
+
+This API will allow you to determine:
+
+1. What jobs are stored in an index (or indices specified via a pattern)?
+2. What target indices were rolled up, what fields were used in those rollups and what aggregations can be performed on each job?
+
+==== Request
+
+`GET {index}/_xpack/rollup/data`
+
+//===== Description
+
+==== Path Parameters
+
+`index`::
+  (string) Index or index-pattern of concrete rollup indices to check for capabilities.
+
+
+
+==== Request Body
+
+There is no request body for the Get Jobs API.
+
+==== Authorization
+
+You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API.
+For more information, see
+{xpack-ref}/security-privileges.html[Security Privileges].
+
+==== Examples
+
+Imagine we have an index named `sensor-1` full of raw data.  We know that the data will grow over time, so there
+will be a `sensor-2`, `sensor-3`, etc.  Let's create a Rollup job, which stores it's data in `sensor_rollup`:
+
+[source,js]
+--------------------------------------------------
+PUT _xpack/rollup/job/sensor
+{
+    "index_pattern": "sensor-*",
+    "rollup_index": "sensor_rollup",
+    "cron": "*/30 * * * * ?",
+    "page_size" :1000,
+    "groups" : {
+      "date_histogram": {
+        "field": "timestamp",
+        "interval": "1h",
+        "delay": "7d"
+      },
+      "terms": {
+        "fields": ["node"]
+      }
+    },
+    "metrics": [
+        {
+            "field": "temperature",
+            "metrics": ["min", "max", "sum"]
+        },
+        {
+            "field": "voltage",
+            "metrics": ["avg"]
+        }
+    ]
+}
+--------------------------------------------------
+// CONSOLE
+// TEST[setup:sensor_index]
+
+If at a later date, we'd like to determine what jobs and capabilities were stored in the `sensor_rollup` index, we can use the Get Rollup
+Index API:
+
+[source,js]
+--------------------------------------------------
+GET /sensor_rollup/_xpack/rollup/data
+--------------------------------------------------
+// CONSOLE
+// TEST[continued]
+
+Note how we are requesting the concrete rollup index name (`sensor_rollup`) as the first part of the URL.
+This  will yield the following response:
+
+[source,js]
+----
+{
+  "sensor_rollup" : {
+    "rollup_jobs" : [
+      {
+        "job_id" : "sensor",
+        "rollup_index" : "sensor_rollup",
+        "index_pattern" : "sensor-*",
+        "fields" : {
+          "node" : [
+            {
+              "agg" : "terms"
+            }
+          ],
+          "temperature" : [
+            {
+              "agg" : "min"
+            },
+            {
+              "agg" : "max"
+            },
+            {
+              "agg" : "sum"
+            }
+          ],
+          "timestamp" : [
+            {
+              "agg" : "date_histogram",
+              "time_zone" : "UTC",
+              "interval" : "1h",
+              "delay": "7d"
+            }
+          ],
+          "voltage" : [
+            {
+              "agg" : "avg"
+            }
+          ]
+        }
+      }
+    ]
+  }
+}
+----
+// TESTRESPONSE
+
+
+The response that is returned contains information that is similar to the original Rollup configuration, but formatted
+differently.  First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data,
+the index pattern that the job was targeting.
+
+Next it shows a list of fields that contain data eligible for rollup searches.  Here we see four fields: `node`, `temperature`,
+`timestamp` and `voltage`.  Each of these fields list the aggregations that are possible.  For example, you can use a min, max
+or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`.
+
+Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index
+or index pattern.  Each of these jobs may have different configurations, so the API returns a list of all the various
+configurations available.
+
+
+Like other APIs that interact with indices, you can specify index patterns instead of explicit indices:
+
+[source,js]
+--------------------------------------------------
+GET /*_rollup/_xpack/rollup/data
+--------------------------------------------------
+// CONSOLE
+// TEST[continued]
+

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/action/GetRollupCapsAction.java

@@ -139,7 +139,7 @@ public class GetRollupCapsAction extends Action<GetRollupCapsAction.Response> {
         public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
             builder.startObject();
             for (Map.Entry<String, RollableIndexCaps> entry : jobs.entrySet()) {
-               entry.getValue().toXContent(builder, params);
+                entry.getValue().toXContent(builder, params);
             }
             builder.endObject();
             return builder;

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/rollup/action/GetRollupIndexCapsAction.java

@@ -0,0 +1,195 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.core.rollup.action;
+
+
+import org.elasticsearch.action.Action;
+import org.elasticsearch.action.ActionRequest;
+import org.elasticsearch.action.ActionRequestBuilder;
+import org.elasticsearch.action.ActionRequestValidationException;
+import org.elasticsearch.action.ActionResponse;
+import org.elasticsearch.action.IndicesRequest;
+import org.elasticsearch.action.support.IndicesOptions;
+import org.elasticsearch.client.ElasticsearchClient;
+import org.elasticsearch.common.ParseField;
+import org.elasticsearch.common.Strings;
+import org.elasticsearch.common.io.stream.StreamInput;
+import org.elasticsearch.common.io.stream.StreamOutput;
+import org.elasticsearch.common.io.stream.Writeable;
+import org.elasticsearch.common.xcontent.ToXContent;
+import org.elasticsearch.common.xcontent.ToXContentObject;
+import org.elasticsearch.common.xcontent.XContentBuilder;
+import org.elasticsearch.xpack.core.rollup.RollupField;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.Map;
+import java.util.Objects;
+
+public class GetRollupIndexCapsAction extends Action<GetRollupIndexCapsAction.Response> {
+
+    public static final GetRollupIndexCapsAction INSTANCE = new GetRollupIndexCapsAction();
+    public static final String NAME = "indices:data/read/xpack/rollup/get/index/caps";
+    public static final ParseField CONFIG = new ParseField("config");
+    public static final ParseField STATUS = new ParseField("status");
+    private static final ParseField INDICES_OPTIONS = new ParseField("indices_options");
+
+    private GetRollupIndexCapsAction() {
+        super(NAME);
+    }
+
+    @Override
+    public Response newResponse() {
+        return new Response();
+    }
+
+    public static class Request extends ActionRequest implements IndicesRequest.Replaceable, ToXContent {
+        private String[] indices;
+        private IndicesOptions options;
+
+        public Request(String[] indices) {
+            this(indices, IndicesOptions.STRICT_EXPAND_OPEN_FORBID_CLOSED);
+        }
+
+        public Request(String[] indices, IndicesOptions options) {
+            this.indices = indices;
+            this.options = options;
+        }
+
+        public Request() {}
+
+        @Override
+        public IndicesOptions indicesOptions() {
+            return options;
+        }
+
+        @Override
+        public String[] indices() {
+            return indices;
+        }
+
+        @Override
+        public IndicesRequest indices(String... indices) {
+            Objects.requireNonNull(indices, "indices must not be null");
+            for (String index : indices) {
+                Objects.requireNonNull(index, "index must not be null");
+            }
+            this.indices = indices;
+            return this;
+        }
+
+        @Override
+        public void readFrom(StreamInput in) throws IOException {
+            super.readFrom(in);
+            this.indices = in.readStringArray();
+            this.options = IndicesOptions.readIndicesOptions(in);
+        }
+
+        @Override
+        public void writeTo(StreamOutput out) throws IOException {
+            super.writeTo(out);
+            out.writeStringArray(indices);
+            options.writeIndicesOptions(out);
+        }
+
+        @Override
+        public ActionRequestValidationException validate() {
+            return null;
+        }
+
+        @Override
+        public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
+            builder.array(RollupField.ID.getPreferredName(), indices);
+            builder.field(INDICES_OPTIONS.getPreferredName(), options);
+            return builder;
+        }
+
+        @Override
+        public int hashCode() {
+            return Objects.hash(Arrays.hashCode(indices), options);
+        }
+
+        @Override
+        public boolean equals(Object obj) {
+            if (obj == null) {
+                return false;
+            }
+            if (getClass() != obj.getClass()) {
+                return false;
+            }
+            Request other = (Request) obj;
+            return Arrays.equals(indices, other.indices)
+                && Objects.equals(options, other.options);
+        }
+    }
+
+    public static class RequestBuilder extends ActionRequestBuilder<Request, Response> {
+
+        protected RequestBuilder(ElasticsearchClient client, GetRollupIndexCapsAction action) {
+            super(client, action, new Request());
+        }
+    }
+
+    public static class Response extends ActionResponse implements Writeable, ToXContentObject {
+
+        private Map<String, RollableIndexCaps> jobs = Collections.emptyMap();
+
+        public Response() {
+
+        }
+
+        public Response(Map<String, RollableIndexCaps> jobs) {
+            this.jobs = Objects.requireNonNull(jobs);
+        }
+
+        Response(StreamInput in) throws IOException {
+            jobs = in.readMap(StreamInput::readString, RollableIndexCaps::new);
+        }
+
+        public Map<String, RollableIndexCaps> getJobs() {
+            return jobs;
+        }
+
+        @Override
+        public void writeTo(StreamOutput out) throws IOException {
+            super.writeTo(out);
+            out.writeMap(jobs, StreamOutput::writeString, (out1, value) -> value.writeTo(out1));
+        }
+
+        @Override
+        public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException {
+            builder.startObject();
+            for (Map.Entry<String, RollableIndexCaps> entry : jobs.entrySet()) {
+                entry.getValue().toXContent(builder, params);
+            }
+            builder.endObject();
+            return builder;
+        }
+
+        @Override
+        public int hashCode() {
+            return Objects.hash(jobs);
+        }
+
+        @Override
+        public boolean equals(Object obj) {
+            if (obj == null) {
+                return false;
+            }
+            if (getClass() != obj.getClass()) {
+                return false;
+            }
+            Response other = (Response) obj;
+            return Objects.equals(jobs, other.jobs);
+        }
+
+        @Override
+        public final String toString() {
+            return Strings.toString(this);
+        }
+    }
+}

x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/Rollup.java

@@ -38,6 +38,7 @@ import org.elasticsearch.xpack.core.XPackSettings;
 import org.elasticsearch.xpack.core.rollup.RollupField;
 import org.elasticsearch.xpack.core.rollup.action.DeleteRollupJobAction;
 import org.elasticsearch.xpack.core.rollup.action.GetRollupCapsAction;
+import org.elasticsearch.xpack.core.rollup.action.GetRollupIndexCapsAction;
 import org.elasticsearch.xpack.core.rollup.action.GetRollupJobsAction;
 import org.elasticsearch.xpack.core.rollup.action.PutRollupJobAction;
 import org.elasticsearch.xpack.core.rollup.action.RollupSearchAction;
@@ -47,6 +48,7 @@ import org.elasticsearch.xpack.core.scheduler.SchedulerEngine;
 import org.elasticsearch.xpack.core.template.TemplateUtils;
 import org.elasticsearch.xpack.rollup.action.TransportDeleteRollupJobAction;
 import org.elasticsearch.xpack.rollup.action.TransportGetRollupCapsAction;
+import org.elasticsearch.xpack.rollup.action.TransportGetRollupIndexCapsAction;
 import org.elasticsearch.xpack.rollup.action.TransportGetRollupJobAction;
 import org.elasticsearch.xpack.rollup.action.TransportPutRollupJobAction;
 import org.elasticsearch.xpack.rollup.action.TransportRollupSearchAction;
@@ -55,6 +57,7 @@ import org.elasticsearch.xpack.rollup.action.TransportStopRollupAction;
 import org.elasticsearch.xpack.rollup.job.RollupJobTask;
 import org.elasticsearch.xpack.rollup.rest.RestDeleteRollupJobAction;
 import org.elasticsearch.xpack.rollup.rest.RestGetRollupCapsAction;
+import org.elasticsearch.xpack.rollup.rest.RestGetRollupIndexCapsAction;
 import org.elasticsearch.xpack.rollup.rest.RestGetRollupJobsAction;
 import org.elasticsearch.xpack.rollup.rest.RestPutRollupJobAction;
 import org.elasticsearch.xpack.rollup.rest.RestRollupSearchAction;
@@ -136,13 +139,14 @@ public class Rollup extends Plugin implements ActionPlugin, PersistentTaskPlugin
         }
 
         return Arrays.asList(
-                new RestRollupSearchAction(settings, restController),
-                new RestPutRollupJobAction(settings, restController),
-                new RestStartRollupJobAction(settings, restController),
-                new RestStopRollupJobAction(settings, restController),
-                new RestDeleteRollupJobAction(settings, restController),
-                new RestGetRollupJobsAction(settings, restController),
-                new RestGetRollupCapsAction(settings, restController)
+            new RestRollupSearchAction(settings, restController),
+            new RestPutRollupJobAction(settings, restController),
+            new RestStartRollupJobAction(settings, restController),
+            new RestStopRollupJobAction(settings, restController),
+            new RestDeleteRollupJobAction(settings, restController),
+            new RestGetRollupJobsAction(settings, restController),
+            new RestGetRollupCapsAction(settings, restController),
+            new RestGetRollupIndexCapsAction(settings, restController)
         );
 
     }
@@ -153,13 +157,14 @@ public class Rollup extends Plugin implements ActionPlugin, PersistentTaskPlugin
             return emptyList();
         }
         return Arrays.asList(
-                new ActionHandler<>(RollupSearchAction.INSTANCE, TransportRollupSearchAction.class),
-                new ActionHandler<>(PutRollupJobAction.INSTANCE, TransportPutRollupJobAction.class),
-                new ActionHandler<>(StartRollupJobAction.INSTANCE, TransportStartRollupAction.class),
-                new ActionHandler<>(StopRollupJobAction.INSTANCE, TransportStopRollupAction.class),
-                new ActionHandler<>(DeleteRollupJobAction.INSTANCE, TransportDeleteRollupJobAction.class),
-                new ActionHandler<>(GetRollupJobsAction.INSTANCE, TransportGetRollupJobAction.class),
-                new ActionHandler<>(GetRollupCapsAction.INSTANCE, TransportGetRollupCapsAction.class)
+            new ActionHandler<>(RollupSearchAction.INSTANCE, TransportRollupSearchAction.class),
+            new ActionHandler<>(PutRollupJobAction.INSTANCE, TransportPutRollupJobAction.class),
+            new ActionHandler<>(StartRollupJobAction.INSTANCE, TransportStartRollupAction.class),
+            new ActionHandler<>(StopRollupJobAction.INSTANCE, TransportStopRollupAction.class),
+            new ActionHandler<>(DeleteRollupJobAction.INSTANCE, TransportDeleteRollupJobAction.class),
+            new ActionHandler<>(GetRollupJobsAction.INSTANCE, TransportGetRollupJobAction.class),
+            new ActionHandler<>(GetRollupCapsAction.INSTANCE, TransportGetRollupCapsAction.class),
+            new ActionHandler<>(GetRollupIndexCapsAction.INSTANCE, TransportGetRollupIndexCapsAction.class)
         );
     }
 

x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/action/TransportGetRollupCapsAction.java

@@ -44,7 +44,6 @@ public class TransportGetRollupCapsAction extends HandledTransportAction<GetRoll
 
     @Override
     protected void doExecute(Task task, GetRollupCapsAction.Request request, ActionListener<GetRollupCapsAction.Response> listener) {
-
         Map<String, RollableIndexCaps> allCaps = getCaps(request.getIndexPattern(), clusterService.state().getMetaData().indices());
         listener.onResponse(new GetRollupCapsAction.Response(allCaps));
     }
@@ -67,7 +66,7 @@ public class TransportGetRollupCapsAction extends HandledTransportAction<GetRoll
 
                 jobCaps.forEach(jobCap -> {
                     String pattern = indexPattern.equals(MetaData.ALL)
-                            ? jobCap.getIndexPattern() : indexPattern;
+                        ? jobCap.getIndexPattern() : indexPattern;
 
                     // Do we already have an entry for this index pattern?
                     RollableIndexCaps indexCaps = allCaps.get(pattern);
@@ -98,11 +97,11 @@ public class TransportGetRollupCapsAction extends HandledTransportAction<GetRoll
         }
 
         RollupIndexCaps caps = RollupIndexCaps.parseMetadataXContent(
-                new BytesArray(rollupMapping.source().uncompressed()), indexName);
+            new BytesArray(rollupMapping.source().uncompressed()), indexName);
 
         if (caps.hasCaps()) {
             return Optional.of(caps);
         }
         return Optional.empty();
     }
-}
+}

x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/action/TransportGetRollupIndexCapsAction.java

@@ -0,0 +1,79 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.rollup.action;
+
+import org.elasticsearch.action.ActionListener;
+import org.elasticsearch.action.support.ActionFilters;
+import org.elasticsearch.action.support.HandledTransportAction;
+import org.elasticsearch.cluster.metadata.IndexMetaData;
+import org.elasticsearch.cluster.metadata.IndexNameExpressionResolver;
+import org.elasticsearch.cluster.service.ClusterService;
+import org.elasticsearch.common.collect.ImmutableOpenMap;
+import org.elasticsearch.common.inject.Inject;
+import org.elasticsearch.common.settings.Settings;
+import org.elasticsearch.tasks.Task;
+import org.elasticsearch.transport.TransportService;
+import org.elasticsearch.xpack.core.rollup.action.GetRollupIndexCapsAction;
+import org.elasticsearch.xpack.core.rollup.action.RollableIndexCaps;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Map;
+import java.util.TreeMap;
+import java.util.function.Supplier;
+import java.util.stream.StreamSupport;
+
+public class TransportGetRollupIndexCapsAction extends HandledTransportAction<GetRollupIndexCapsAction.Request,
+    GetRollupIndexCapsAction.Response> {
+
+    private final ClusterService clusterService;
+
+    @Inject
+    public TransportGetRollupIndexCapsAction(Settings settings, TransportService transportService,
+                                             ClusterService clusterService, ActionFilters actionFilters) {
+        super(settings, GetRollupIndexCapsAction.NAME, transportService, actionFilters,
+            (Supplier<GetRollupIndexCapsAction.Request>) GetRollupIndexCapsAction.Request::new);
+        this.clusterService = clusterService;
+    }
+
+    @Override
+    protected void doExecute(Task task, GetRollupIndexCapsAction.Request request,
+                             ActionListener<GetRollupIndexCapsAction.Response> listener) {
+
+        IndexNameExpressionResolver resolver = new IndexNameExpressionResolver(clusterService.getSettings());
+        String[] indices = resolver.concreteIndexNames(clusterService.state(),
+            request.indicesOptions(), request.indices());
+        Map<String, RollableIndexCaps> allCaps = getCapsByRollupIndex(Arrays.asList(indices),
+            clusterService.state().getMetaData().indices());
+        listener.onResponse(new GetRollupIndexCapsAction.Response(allCaps));
+    }
+
+    static Map<String, RollableIndexCaps> getCapsByRollupIndex(List<String> resolvedIndexNames,
+                                                               ImmutableOpenMap<String, IndexMetaData> indices) {
+        Map<String, RollableIndexCaps> allCaps = new TreeMap<>();
+
+        StreamSupport.stream(indices.spliterator(), false)
+            .filter(entry -> resolvedIndexNames.contains(entry.key))
+            .forEach(entry -> {
+                // Does this index have rollup metadata?
+                TransportGetRollupCapsAction.findRollupIndexCaps(entry.key, entry.value)
+                    .ifPresent(cap -> {
+                        cap.getJobCaps().forEach(jobCap -> {
+                            // Do we already have an entry for this index?
+                            RollableIndexCaps indexCaps = allCaps.get(jobCap.getRollupIndex());
+                            if (indexCaps == null) {
+                                indexCaps = new RollableIndexCaps(jobCap.getRollupIndex());
+                            }
+                            indexCaps.addJobCap(jobCap);
+                            allCaps.put(jobCap.getRollupIndex(), indexCaps);
+                        });
+                    });
+            });
+
+        return allCaps;
+    }
+
+}

x-pack/plugin/rollup/src/main/java/org/elasticsearch/xpack/rollup/rest/RestGetRollupIndexCapsAction.java

@@ -0,0 +1,38 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.rollup.rest;
+
+import org.elasticsearch.action.support.IndicesOptions;
+import org.elasticsearch.client.node.NodeClient;
+import org.elasticsearch.common.ParseField;
+import org.elasticsearch.common.settings.Settings;
+import org.elasticsearch.rest.BaseRestHandler;
+import org.elasticsearch.rest.RestController;
+import org.elasticsearch.rest.RestRequest;
+import org.elasticsearch.rest.action.RestToXContentListener;
+import org.elasticsearch.xpack.core.rollup.action.GetRollupIndexCapsAction;
+
+public class RestGetRollupIndexCapsAction extends BaseRestHandler {
+    public static final ParseField INDEX = new ParseField("index");
+
+    public RestGetRollupIndexCapsAction(Settings settings, RestController controller) {
+        super(settings);
+        controller.registerHandler(RestRequest.Method.GET, "/{index}/_xpack/rollup/data", this);
+    }
+
+    @Override
+    protected RestChannelConsumer prepareRequest(RestRequest restRequest, NodeClient client) {
+        String index = restRequest.param(INDEX.getPreferredName());
+        IndicesOptions options = IndicesOptions.fromRequest(restRequest, IndicesOptions.STRICT_EXPAND_OPEN_FORBID_CLOSED);
+        GetRollupIndexCapsAction.Request request = new GetRollupIndexCapsAction.Request(new String[]{index}, options);
+        return channel -> client.execute(GetRollupIndexCapsAction.INSTANCE, request, new RestToXContentListener<>(channel));
+    }
+
+    @Override
+    public String getName() {
+        return "rollup_get_caps_action";
+    }
+}

x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/action/GetRollupCapsActionRequestTests.java

@@ -11,11 +11,11 @@ import org.elasticsearch.cluster.metadata.MappingMetaData;
 import org.elasticsearch.cluster.metadata.MetaData;
 import org.elasticsearch.common.collect.ImmutableOpenMap;
 import org.elasticsearch.test.AbstractStreamableTestCase;
+import org.elasticsearch.xpack.core.rollup.ConfigTestHelpers;
 import org.elasticsearch.xpack.core.rollup.RollupField;
 import org.elasticsearch.xpack.core.rollup.action.GetRollupCapsAction;
 import org.elasticsearch.xpack.core.rollup.action.RollableIndexCaps;
 import org.elasticsearch.xpack.core.rollup.job.RollupJobConfig;
-import org.elasticsearch.xpack.core.rollup.ConfigTestHelpers;
 import org.mockito.Mockito;
 
 import java.io.IOException;

x-pack/plugin/rollup/src/test/java/org/elasticsearch/xpack/rollup/action/GetRollupIndexCapsActionRequestTests.java

@@ -0,0 +1,177 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the Elastic License;
+ * you may not use this file except in compliance with the Elastic License.
+ */
+package org.elasticsearch.xpack.rollup.action;
+
+
+import org.elasticsearch.cluster.metadata.IndexMetaData;
+import org.elasticsearch.cluster.metadata.MappingMetaData;
+import org.elasticsearch.cluster.metadata.MetaData;
+import org.elasticsearch.common.collect.ImmutableOpenMap;
+import org.elasticsearch.test.AbstractStreamableTestCase;
+import org.elasticsearch.xpack.core.rollup.ConfigTestHelpers;
+import org.elasticsearch.xpack.core.rollup.RollupField;
+import org.elasticsearch.xpack.core.rollup.action.GetRollupIndexCapsAction;
+import org.elasticsearch.xpack.core.rollup.action.RollableIndexCaps;
+import org.mockito.Mockito;
+
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.elasticsearch.xpack.rollup.action.TransportGetRollupIndexCapsAction.getCapsByRollupIndex;
+import static org.hamcrest.Matchers.equalTo;
+
+
+public class GetRollupIndexCapsActionRequestTests extends AbstractStreamableTestCase<GetRollupIndexCapsAction.Request> {
+
+    @Override
+    protected GetRollupIndexCapsAction.Request createTestInstance() {
+        if (randomBoolean()) {
+            return new GetRollupIndexCapsAction.Request(new String[]{MetaData.ALL});
+        }
+        return new GetRollupIndexCapsAction.Request(new String[]{randomAlphaOfLengthBetween(1, 20)});
+    }
+
+    @Override
+    protected GetRollupIndexCapsAction.Request createBlankInstance() {
+        return new GetRollupIndexCapsAction.Request();
+    }
+
+
+    public void testNoIndicesByRollup() {
+        ImmutableOpenMap<String, IndexMetaData> indices = new ImmutableOpenMap.Builder<String, IndexMetaData>().build();
+        Map<String, RollableIndexCaps> caps = getCapsByRollupIndex(Collections.singletonList("foo"), indices);
+        assertThat(caps.size(), equalTo(0));
+    }
+
+    public void testAllIndicesByRollupSingleRollup() throws IOException {
+        int num = randomIntBetween(1,5);
+        ImmutableOpenMap.Builder<String, IndexMetaData> indices = new ImmutableOpenMap.Builder<>(5);
+        int indexCounter = 0;
+        for (int j = 0; j < 5; j++) {
+
+            Map<String, Object> jobs = new HashMap<>(num);
+            for (int i = 0; i < num; i++) {
+                String jobName = randomAlphaOfLength(10);
+                String indexName = Integer.toString(indexCounter);
+                indexCounter += 1;
+                jobs.put(jobName, ConfigTestHelpers.getRollupJob(jobName).setRollupIndex("foo").build());
+            }
+
+            MappingMetaData mappingMeta = new MappingMetaData(RollupField.TYPE_NAME,
+                Collections.singletonMap(RollupField.TYPE_NAME,
+                    Collections.singletonMap("_meta",
+                        Collections.singletonMap(RollupField.ROLLUP_META, jobs))));
+
+            ImmutableOpenMap.Builder<String, MappingMetaData> mappings = ImmutableOpenMap.builder(1);
+            mappings.put(RollupField.TYPE_NAME, mappingMeta);
+            IndexMetaData meta = Mockito.mock(IndexMetaData.class);
+            Mockito.when(meta.getMappings()).thenReturn(mappings.build());
+            indices.put("foo", meta);
+        }
+
+        Map<String, RollableIndexCaps> caps = getCapsByRollupIndex(Collections.singletonList("foo"),
+            indices.build());
+        assertThat(caps.size(), equalTo(1));
+    }
+
+    public void testAllIndicesByRollupManyRollup() throws IOException {
+        ImmutableOpenMap.Builder<String, IndexMetaData> indices = new ImmutableOpenMap.Builder<>(5);
+        int indexCounter = 0;
+        for (int j = 0; j < 5; j++) {
+
+            Map<String, Object> jobs = new HashMap<>(1);
+            String jobName = randomAlphaOfLength(10);
+            String indexName = Integer.toString(indexCounter);
+            indexCounter += 1;
+            jobs.put(jobName, ConfigTestHelpers.getRollupJob(jobName)
+                .setIndexPattern(indexName)
+                .setRollupIndex("rollup_" + indexName).build());
+
+
+            MappingMetaData mappingMeta = new MappingMetaData(RollupField.TYPE_NAME,
+                Collections.singletonMap(RollupField.TYPE_NAME,
+                    Collections.singletonMap("_meta",
+                        Collections.singletonMap(RollupField.ROLLUP_META, jobs))));
+
+            ImmutableOpenMap.Builder<String, MappingMetaData> mappings = ImmutableOpenMap.builder(1);
+            mappings.put(RollupField.TYPE_NAME, mappingMeta);
+            IndexMetaData meta = Mockito.mock(IndexMetaData.class);
+            Mockito.when(meta.getMappings()).thenReturn(mappings.build());
+            indices.put("rollup_" + indexName, meta);
+        }
+
+        Map<String, RollableIndexCaps> caps = getCapsByRollupIndex(Arrays.asList(indices.keys().toArray(String.class)), indices.build());
+        assertThat(caps.size(), equalTo(5));
+    }
+
+
+    public void testOneIndexByRollupManyRollup() throws IOException {
+        ImmutableOpenMap.Builder<String, IndexMetaData> indices = new ImmutableOpenMap.Builder<>(5);
+        int indexCounter = 0;
+        for (int j = 0; j < 5; j++) {
+
+            Map<String, Object> jobs = new HashMap<>(1);
+            String jobName = randomAlphaOfLength(10);
+            String indexName = Integer.toString(indexCounter);
+            indexCounter += 1;
+            jobs.put(jobName, ConfigTestHelpers.getRollupJob(jobName)
+                .setIndexPattern("foo_" + indexName)
+                .setRollupIndex("rollup_" + indexName).build());
+
+            MappingMetaData mappingMeta = new MappingMetaData(RollupField.TYPE_NAME,
+                Collections.singletonMap(RollupField.TYPE_NAME,
+                    Collections.singletonMap("_meta",
+                        Collections.singletonMap(RollupField.ROLLUP_META, jobs))));
+
+            ImmutableOpenMap.Builder<String, MappingMetaData> mappings = ImmutableOpenMap.builder(1);
+            mappings.put(RollupField.TYPE_NAME, mappingMeta);
+            IndexMetaData meta = Mockito.mock(IndexMetaData.class);
+            Mockito.when(meta.getMappings()).thenReturn(mappings.build());
+            indices.put("rollup_" + indexName, meta);
+        }
+
+        Map<String, RollableIndexCaps> caps = getCapsByRollupIndex(Collections.singletonList("rollup_1"), indices.build());
+        assertThat(caps.size(), equalTo(1));
+        assertThat(caps.get("rollup_1").getIndexName(), equalTo("rollup_1"));
+        assertThat(caps.get("rollup_1").getJobCaps().size(), equalTo(1));
+    }
+
+    public void testOneIndexByRollupOneRollup() throws IOException {
+        ImmutableOpenMap.Builder<String, IndexMetaData> indices = new ImmutableOpenMap.Builder<>(5);
+        int indexCounter = 0;
+        for (int j = 0; j < 5; j++) {
+
+            Map<String, Object> jobs = new HashMap<>(1);
+            String jobName = randomAlphaOfLength(10);
+            String indexName = Integer.toString(indexCounter);
+            indexCounter += 1;
+            jobs.put(jobName, ConfigTestHelpers.getRollupJob(jobName)
+                .setIndexPattern("foo_" + indexName)
+                .setRollupIndex("rollup_foo").build());
+
+            MappingMetaData mappingMeta = new MappingMetaData(RollupField.TYPE_NAME,
+                Collections.singletonMap(RollupField.TYPE_NAME,
+                    Collections.singletonMap("_meta",
+                        Collections.singletonMap(RollupField.ROLLUP_META, jobs))));
+
+            ImmutableOpenMap.Builder<String, MappingMetaData> mappings = ImmutableOpenMap.builder(1);
+            mappings.put(RollupField.TYPE_NAME, mappingMeta);
+            IndexMetaData meta = Mockito.mock(IndexMetaData.class);
+            Mockito.when(meta.getMappings()).thenReturn(mappings.build());
+            indices.put("rollup_foo", meta);
+        }
+
+        Map<String, RollableIndexCaps> caps = getCapsByRollupIndex(Collections.singletonList("rollup_foo"), indices.build());
+        assertThat(caps.size(), equalTo(1));
+        assertThat(caps.get("rollup_foo").getIndexName(), equalTo("rollup_foo"));
+        assertThat(caps.get("rollup_foo").getJobCaps().size(), equalTo(1));
+    }
+}
+
+

x-pack/plugin/src/test/resources/rest-api-spec/api/xpack.rollup.get_rollup_index_caps.json

@@ -0,0 +1,17 @@
+{
+  "xpack.rollup.get_rollup_index_caps": {
+    "documentation": "",
+    "methods": [ "GET" ],
+    "url": {
+      "path": "/{index}/_xpack/rollup/data",
+      "paths": [ "/{index}/_xpack/rollup/data" ],
+      "parts": {
+        "index": {
+          "type": "string",
+          "required": true,
+          "description": "The rollup index or index pattern to obtain rollup capabilities from."
+        }
+      }
+    }
+  }
+}

x-pack/plugin/src/test/resources/rest-api-spec/test/rollup/get_rollup_caps.yml

@@ -22,6 +22,18 @@ setup:
                 type: date
               value_field:
                 type: integer
+
+  - do:
+      indices.create:
+        index: foo3
+        include_type_name: false
+        body:
+          mappings:
+            properties:
+              the_field:
+                type: date
+              value_field:
+                type: integer
   - do:
       headers:
         Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
@@ -228,4 +240,3 @@ setup:
                 - agg: "min"
                 - agg: "max"
                 - agg: "sum"
-

x-pack/plugin/src/test/resources/rest-api-spec/test/rollup/get_rollup_index_caps.yml

@@ -0,0 +1,363 @@
+setup:
+  - do:
+      indices.create:
+        index: foo
+        include_type_name: false
+        body:
+          mappings:
+            properties:
+              the_field:
+                type: date
+              value_field:
+                type: integer
+
+  - do:
+      indices.create:
+        index: foo2
+        include_type_name: false
+        body:
+          mappings:
+            properties:
+              the_field:
+                type: date
+              value_field:
+                type: integer
+
+  - do:
+      indices.create:
+        index: foo3
+        include_type_name: false
+        body:
+          mappings:
+            properties:
+              the_field:
+                type: date
+              value_field:
+                type: integer
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo
+        body:  >
+          {
+            "index_pattern": "foo",
+            "rollup_index": "foo_rollup",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+
+---
+"Verify one job caps by rollup index":
+
+  - do:
+      xpack.rollup.get_rollup_index_caps:
+        index: "foo_rollup"
+
+  - match:
+      foo_rollup:
+        rollup_jobs:
+          - job_id: "foo"
+            rollup_index: "foo_rollup"
+            index_pattern: "foo"
+            fields:
+              the_field:
+                - agg: "date_histogram"
+                  interval: "1h"
+                  time_zone: "UTC"
+              value_field:
+                - agg: "min"
+                - agg: "max"
+                - agg: "sum"
+
+---
+"Verify two job caps by rollup index":
+
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo2
+        body:  >
+          {
+            "index_pattern": "foo",
+            "rollup_index": "foo_rollup",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+  - do:
+      xpack.rollup.get_rollup_index_caps:
+        index: "foo_rollup"
+
+  - match:
+      foo_rollup:
+        rollup_jobs:
+          - job_id: "foo"
+            rollup_index: "foo_rollup"
+            index_pattern: "foo"
+            fields:
+              the_field:
+                - agg: "date_histogram"
+                  interval: "1h"
+                  time_zone: "UTC"
+              value_field:
+                - agg: "min"
+                - agg: "max"
+                - agg: "sum"
+          - job_id: "foo2"
+            rollup_index: "foo_rollup"
+            index_pattern: "foo"
+            fields:
+              the_field:
+                - agg: "date_histogram"
+                  interval: "1h"
+                  time_zone: "UTC"
+              value_field:
+                - agg: "min"
+                - agg: "max"
+                - agg: "sum"
+
+
+---
+"Verify two different job caps by rollup index":
+
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo2
+        body:  >
+          {
+            "index_pattern": "foo2",
+            "rollup_index": "foo_rollup2",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+  - do:
+      xpack.rollup.get_rollup_index_caps:
+        index: "foo_rollup"
+
+  - match:
+      foo_rollup:
+        rollup_jobs:
+          - job_id: "foo"
+            rollup_index: "foo_rollup"
+            index_pattern: "foo"
+            fields:
+              the_field:
+                - agg: "date_histogram"
+                  interval: "1h"
+                  time_zone: "UTC"
+              value_field:
+                - agg: "min"
+                - agg: "max"
+                - agg: "sum"
+
+---
+"Verify all job caps by rollup index":
+
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo2
+        body:  >
+          {
+            "index_pattern": "foo2",
+            "rollup_index": "foo_rollup",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo3
+        body:  >
+          {
+            "index_pattern": "foo3",
+            "rollup_index": "foo_rollup2",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+
+  - do:
+      xpack.rollup.get_rollup_index_caps:
+        index: "_all"
+
+  - match:
+      $body:
+        foo_rollup:
+          rollup_jobs:
+            - job_id: "foo"
+              rollup_index: "foo_rollup"
+              index_pattern: "foo"
+              fields:
+                the_field:
+                  - agg: "date_histogram"
+                    interval: "1h"
+                    time_zone: "UTC"
+                value_field:
+                  - agg: "min"
+                  - agg: "max"
+                  - agg: "sum"
+            - job_id: "foo2"
+              rollup_index: "foo_rollup"
+              index_pattern: "foo2"
+              fields:
+                the_field:
+                  - agg: "date_histogram"
+                    interval: "1h"
+                    time_zone: "UTC"
+                value_field:
+                  - agg: "min"
+                  - agg: "max"
+                  - agg: "sum"
+        foo_rollup2:
+          rollup_jobs:
+            - job_id: "foo3"
+              rollup_index: "foo_rollup2"
+              index_pattern: "foo3"
+              fields:
+                the_field:
+                  - agg: "date_histogram"
+                    interval: "1h"
+                    time_zone: "UTC"
+                value_field:
+                  - agg: "min"
+                  - agg: "max"
+                  - agg: "sum"
+
+---
+"Verify index pattern":
+
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo2
+        body:  >
+          {
+            "index_pattern": "foo2",
+            "rollup_index": "foo_rollup",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+  - do:
+      headers:
+        Authorization: "Basic eF9wYWNrX3Jlc3RfdXNlcjp4LXBhY2stdGVzdC1wYXNzd29yZA==" # run as x_pack_rest_user, i.e. the test setup superuser
+      xpack.rollup.put_job:
+        id: foo3
+        body:  >
+          {
+            "index_pattern": "foo3",
+            "rollup_index": "foo_rollup2",
+            "cron": "*/30 * * * * ?",
+            "page_size" :10,
+            "groups" : {
+              "date_histogram": {
+                "field": "the_field",
+                "interval": "1h"
+              }
+            },
+            "metrics": [
+              {
+                "field": "value_field",
+                "metrics": ["min", "max", "sum"]
+              }
+            ]
+          }
+
+  - do:
+      xpack.rollup.get_rollup_index_caps:
+        index: "*_rollup2"
+
+  - match:
+      $body:
+        foo_rollup2:
+          rollup_jobs:
+            - job_id: "foo3"
+              rollup_index: "foo_rollup2"
+              index_pattern: "foo3"
+              fields:
+                the_field:
+                  - agg: "date_histogram"
+                    interval: "1h"
+                    time_zone: "UTC"
+                value_field:
+                  - agg: "min"
+                  - agg: "max"
+                  - agg: "sum"
+