diff --git a/_config.yml b/_config.yml index 63d4cb50..bbabab57 100644 --- a/_config.yml +++ b/_config.yml @@ -48,6 +48,9 @@ collections: replication-plugin: permalink: /:collection/:path/ output: true + observability-plugins: + permalink: /:collection/:path/ + output: true monitoring-plugins: permalink: /:collection/:path/ output: true @@ -87,6 +90,9 @@ just_the_docs: replication-plugin: name: Replication plugin nav_fold: true + observability-plugins: + name: Observability plugins + nav_fold: true monitoring-plugins: name: Monitoring plugins nav_fold: true diff --git a/_observability-plugins/event-analytics.md b/_observability-plugins/event-analytics.md new file mode 100644 index 00000000..ed440f1f --- /dev/null +++ b/_observability-plugins/event-analytics.md @@ -0,0 +1,33 @@ +--- +layout: default +title: Event analytics +nav_order: 10 +--- + +# Event analytics + +Event analytics in observability is where you can use [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index) (PPL) queries to build and view different visualizations of your data. + +## Get started with event analytics + +To get started, choose **Observability** in OpenSearch Dashboards, and then choose **Event analytics**. If you want to start exploring without adding any of your own data, choose **Add sample Events Data**, and Dashboards adds some sample visualizations you can interact with. + +## Build a query + +To generate custom visualizations, you must first specify a PPL query. OpenSearch Dashboards then automatically creates a visualization based on the results of your query. + +For example, the following PPL query returns a count of how many host addresses are currently in your data. + +``` +source = opensearch_dashboards_sample_data_logs | fields host | stats count() +``` + +By default, Dashboards shows results from the last 15 minutes of your data. To see data from a different timeframe, use the date and time selector. + +For more information about building PPL queries, see [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index). + +## Save a visualization + +After Dashboards generates a visualization, you must save it if you want to return to it at a later time or if you want to add it to an [operational panel]({{site.url}}{{site.baseurl}}/observability-plugins/operational-panels). + +To save a visualization, expand the save dropdown menu next to **Run**, enter a name for your visualization, then choose **Save**. You can reopen any saved visualizations on the event analytics page. diff --git a/_observability-plugins/index.md b/_observability-plugins/index.md new file mode 100644 index 00000000..fe13291e --- /dev/null +++ b/_observability-plugins/index.md @@ -0,0 +1,22 @@ +--- +layout: default +title: About Observability +nav_order: 1 +has_children: false +redirect_from: + - /observability-plugins/ +--- + +# About Observability +OpenSearch Dashboards +{: .label .label-yellow :} + +The Observability plugins are a collection of plugins that let you visualize data-driven events by using Piped Processing Language to explore, discover, and query data stored in OpenSearch. + +Your experience of exploring data might differ, but if you're new to exploring data to create visualizations, we recommend trying a work flow like the following: + +1. Explore data over a certain timeframe using [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index). +1. Use [event analytics]({{site.url}}{{site.baseurl}}/observability-plugins/event-analytics) to turn data-driven events into visualizations. +1. Create [operational panels]({{site.url}}{{site.baseurl}}/observability-plugins/plugins/operational-panels) and add visualizations to compare data the way you like. +1. Use [trace analytics]({{site.url}}{{site.baseurl}}/observability-plugins/plugins/trace/index) to create traces and dive deep into your data. +1. Leverage [notebooks]({{site.url}}{{site.baseurl}}/observability-plugins/plugins/notebooks) to combine different visualizations and code blocks that you can share with team members. diff --git a/_dashboards/notebooks.md b/_observability-plugins/notebooks.md similarity index 100% rename from _dashboards/notebooks.md rename to _observability-plugins/notebooks.md diff --git a/_observability-plugins/operational-panels.md b/_observability-plugins/operational-panels.md new file mode 100644 index 00000000..0f47d182 --- /dev/null +++ b/_observability-plugins/operational-panels.md @@ -0,0 +1,25 @@ +--- +layout: default +title: Operational panels +nav_order: 30 +--- + +# Operational panels + +Operational panels in OpenSearch Dashboards are collections of visualizations generated using [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index) (PPL) queries. + +## Get started with operational panels + +If you want to start using operational panels without adding any data, expand the **Action** menu, choose **Add samples**, and Dashboards adds a set of operational panels with saved visualizations for you to explore. + +## Create an operational panel + +To create an operational panel and add visualizations: + +1. From the **Add Visualization** dropdown menu, choose **Select Existing Visualization** or **Create New Visualization**, which takes you to the [event analytics]({{site.url}}{{site.baseurl}}/observability-plugins/event-analytics) explorer, where you can use PPL to create visualizations. +1. If you're adding already existing visualizations, choose a visualization from the dropdown menu. +1. Choose **Add**. + +![Sample operational panel]({{site.url}}{{site.baseurl}}/images/operational-panel.png) + +To search for a particular visualization in your operation panels, use PPL queries to search for data you've already added to your panel. diff --git a/_search-plugins/ppl/commands.md b/_observability-plugins/ppl/commands.md similarity index 100% rename from _search-plugins/ppl/commands.md rename to _observability-plugins/ppl/commands.md diff --git a/_search-plugins/ppl/datatypes.md b/_observability-plugins/ppl/datatypes.md similarity index 100% rename from _search-plugins/ppl/datatypes.md rename to _observability-plugins/ppl/datatypes.md diff --git a/_search-plugins/ppl/endpoint.md b/_observability-plugins/ppl/endpoint.md similarity index 100% rename from _search-plugins/ppl/endpoint.md rename to _observability-plugins/ppl/endpoint.md diff --git a/_search-plugins/ppl/functions.md b/_observability-plugins/ppl/functions.md similarity index 100% rename from _search-plugins/ppl/functions.md rename to _observability-plugins/ppl/functions.md diff --git a/_search-plugins/ppl/identifiers.md b/_observability-plugins/ppl/identifiers.md similarity index 100% rename from _search-plugins/ppl/identifiers.md rename to _observability-plugins/ppl/identifiers.md diff --git a/_search-plugins/ppl/index.md b/_observability-plugins/ppl/index.md similarity index 95% rename from _search-plugins/ppl/index.md rename to _observability-plugins/ppl/index.md index 0eaa888f..e3c9724a 100644 --- a/_search-plugins/ppl/index.md +++ b/_observability-plugins/ppl/index.md @@ -1,7 +1,7 @@ --- layout: default title: Piped processing language -nav_order: 42 +nav_order: 40 has_children: true has_toc: false redirect_from: @@ -52,9 +52,9 @@ search source=accounts firstname | lastname | :--- | :--- | -Amber | Duke -Hattie | Bond -Nanette | Bates -Dale | Adams +Amber | Duke +Hattie | Bond +Nanette | Bates +Dale | Adams ![PPL query workbench]({{site.url}}{{site.baseurl}}/images/ppl.png) diff --git a/_search-plugins/ppl/protocol.md b/_observability-plugins/ppl/protocol.md similarity index 100% rename from _search-plugins/ppl/protocol.md rename to _observability-plugins/ppl/protocol.md diff --git a/_search-plugins/ppl/settings.md b/_observability-plugins/ppl/settings.md similarity index 100% rename from _search-plugins/ppl/settings.md rename to _observability-plugins/ppl/settings.md diff --git a/_monitoring-plugins/trace/data-prepper-reference.md b/_observability-plugins/trace/data-prepper-reference.md similarity index 99% rename from _monitoring-plugins/trace/data-prepper-reference.md rename to _observability-plugins/trace/data-prepper-reference.md index 9eca2e99..99a8f522 100644 --- a/_monitoring-plugins/trace/data-prepper-reference.md +++ b/_observability-plugins/trace/data-prepper-reference.md @@ -7,7 +7,7 @@ nav_order: 25 # Data Prepper configuration reference -This page lists all supported Data Prepper sources, buffers, preppers, and sinks, along with their associated options. For example configuration files, see [Data Prepper]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper/). +This page lists all supported Data Prepper sources, buffers, preppers, and sinks, along with their associated options. For example configuration files, see [Data Prepper]({{site.url}}{{site.baseurl}}/observability-plugins/trace/data-prepper/). ## Data Prepper server options diff --git a/_monitoring-plugins/trace/data-prepper.md b/_observability-plugins/trace/data-prepper.md similarity index 98% rename from _monitoring-plugins/trace/data-prepper.md rename to _observability-plugins/trace/data-prepper.md index 8909eb5d..7683033b 100644 --- a/_monitoring-plugins/trace/data-prepper.md +++ b/_observability-plugins/trace/data-prepper.md @@ -105,7 +105,7 @@ service-map-pipeline: trace_analytics_service_map: true ``` -To learn more, see the [Data Prepper configuration reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper-reference/). +To learn more, see the [Data Prepper configuration reference]({{site.url}}{{site.baseurl}}/observability-plugins/trace/data-prepper-reference/). ## Configure the Data Prepper server Data Prepper itself provides administrative HTTP endpoints such as `/list` to list pipelines and `/metrics/prometheus` to provide Prometheus-compatible metrics data. The port which serves these endpoints, as well as TLS configuration, is specified by a separate YAML file. Example: diff --git a/_monitoring-plugins/trace/get-started.md b/_observability-plugins/trace/get-started.md similarity index 91% rename from _monitoring-plugins/trace/get-started.md rename to _observability-plugins/trace/get-started.md index 03a175b4..4bc535a0 100644 --- a/_monitoring-plugins/trace/get-started.md +++ b/_observability-plugins/trace/get-started.md @@ -20,9 +20,9 @@ OpenSearch Trace Analytics consists of two components---Data Prepper and the Tra 1. The [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/) receives data from the application and formats it into OpenTelemetry data. -1. [Data Prepper]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster. +1. [Data Prepper]({{site.url}}{{site.baseurl}}/observability-plugins/trace/data-prepper/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster. -1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput. +1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugins/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput. ## Jaeger HotROD @@ -80,4 +80,4 @@ curl -X GET -u 'admin:admin' -k 'https://localhost:9200/otel-v1-apm-span-000001/ Navigate to `http://localhost:5601` in a web browser and choose **Trace Analytics**. You can see the results of your single click in the Jaeger HotROD web interface: the number of traces per API and HTTP method, latency trends, a color-coded map of the service architecture, and a list of trace IDs that you can use to drill down on individual operations. -If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/ta-dashboards/). +If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugins/trace/ta-dashboards/). diff --git a/_monitoring-plugins/trace/index.md b/_observability-plugins/trace/index.md similarity index 92% rename from _monitoring-plugins/trace/index.md rename to _observability-plugins/trace/index.md index 92123d88..ca8f72e7 100644 --- a/_monitoring-plugins/trace/index.md +++ b/_observability-plugins/trace/index.md @@ -1,11 +1,9 @@ --- layout: default title: Trace analytics -nav_order: 48 +nav_order: 60 has_children: true has_toc: false -redirect_from: - - /monitoring-plugins/trace/ --- # Trace Analytics diff --git a/_monitoring-plugins/trace/ta-dashboards.md b/_observability-plugins/trace/ta-dashboards.md similarity index 100% rename from _monitoring-plugins/trace/ta-dashboards.md rename to _observability-plugins/trace/ta-dashboards.md diff --git a/_opensearch/data-streams.md b/_opensearch/data-streams.md index ff06d6e1..fa736ac4 100644 --- a/_opensearch/data-streams.md +++ b/_opensearch/data-streams.md @@ -228,7 +228,7 @@ POST logs-redis/_rollover If you now perform a `GET` operation on the `logs-redis` data stream, you see that the generation ID is incremented from 1 to 2. -You can also set up an [Index State Management (ISM) policy]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/) to automate the rollover process for the data stream. +You can also set up an [Index State Management (ISM) policy]({{site.url}}{{site.baseurl}}/im-plugin/ism/policies/) to automate the rollover process for the data stream. The ISM policy is applied to the backing indices at the time of their creation. When you associate a policy to a data stream, it only affects the future backing indices of that data stream. You also don’t need to provide the `rollover_alias` setting, because the ISM policy infers this information from the backing index. @@ -262,4 +262,4 @@ You can use wildcards to delete more than one data stream. We recommend deleting data from a data stream using an ISM policy. -You can also use [asynchronous search]({{site.url}}{{site.baseurl}}/search-plugins/async/index/) and [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/search-plugins/ppl/index/) to query your data stream directly. You can also use the security plugin to define granular permissions on the data stream name. +You can also use [asynchronous search]({{site.url}}{{site.baseurl}}/search-plugins/async/index/) and [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index/) to query your data stream directly. You can also use the security plugin to define granular permissions on the data stream name. diff --git a/images/operational-panel.png b/images/operational-panel.png new file mode 100644 index 00000000..a25273ee Binary files /dev/null and b/images/operational-panel.png differ diff --git a/images/ta-dashboard.png b/images/ta-dashboard.png index 70065261..c3c66f59 100644 Binary files a/images/ta-dashboard.png and b/images/ta-dashboard.png differ diff --git a/images/ta-services.png b/images/ta-services.png index 2d7e6389..b00d3415 100644 Binary files a/images/ta-services.png and b/images/ta-services.png differ