Merge pull request #281 from opensearch-project/observability

Added observability plugin

_config.yml

@@ -48,6 +48,9 @@ collections:
   replication-plugin:
     permalink: /:collection/:path/
     output: true
+  observability-plugins:
+    permalink: /:collection/:path/
+    output: true
   monitoring-plugins:
     permalink: /:collection/:path/
     output: true
@@ -87,6 +90,9 @@ just_the_docs:
     replication-plugin:
       name: Replication plugin
       nav_fold: true
+    observability-plugins:
+      name: Observability plugins
+      nav_fold: true
     monitoring-plugins:
       name: Monitoring plugins
       nav_fold: true

_observability-plugins/event-analytics.md

@@ -0,0 +1,33 @@
+---
+layout: default
+title: Event analytics
+nav_order: 10
+---
+
+# Event analytics
+
+Event analytics in observability is where you can use [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index) (PPL) queries to build and view different visualizations of your data.
+
+## Get started with event analytics
+
+To get started, choose **Observability** in OpenSearch Dashboards, and then choose **Event analytics**. If you want to start exploring without adding any of your own data, choose **Add sample Events Data**, and Dashboards adds some sample visualizations you can interact with.
+
+## Build a query
+
+To generate custom visualizations, you must first specify a PPL query. OpenSearch Dashboards then automatically creates a visualization based on the results of your query.
+
+For example, the following PPL query returns a count of how many host addresses are currently in your data.
+
+```
+source = opensearch_dashboards_sample_data_logs | fields host | stats count()
+```
+
+By default, Dashboards shows results from the last 15 minutes of your data. To see data from a different timeframe, use the date and time selector.
+
+For more information about building PPL queries, see [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index).
+
+## Save a visualization
+
+After Dashboards generates a visualization, you must save it if you want to return to it at a later time or if you want to add it to an [operational panel]({{site.url}}{{site.baseurl}}/observability-plugins/operational-panels).
+
+To save a visualization, expand the save dropdown menu next to **Run**, enter a name for your visualization, then choose **Save**. You can reopen any saved visualizations on the event analytics page.

_observability-plugins/index.md

@@ -0,0 +1,26 @@
+---
+layout: default
+title: About Observability
+nav_order: 1
+has_children: false
+redirect_from:
+  - /observability-plugins/
+---
+
+# About Observability
+OpenSearch Dashboards
+{: .label .label-yellow :}
+
+The Observability plugins are a collection of plugins that let you visualize data-driven events by using Piped Processing Language to explore, discover, and query data stored in OpenSearch.
+
+Your experience of exploring data might differ, but if you're new to exploring data to create visualizations, we recommend trying a workflow like the following:
+
+1. Explore data over a certain timeframe using [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index).
+1. Use [event analytics]({{site.url}}{{site.baseurl}}/observability-plugins/event-analytics) to turn data-driven events into visualizations.
+  ![Sample Event Analytics View]({{site.url}}{{site.baseurl}}/images/event-analytics.png)
+1. Create [operational panels]({{site.url}}{{site.baseurl}}/observability-plugins/operational-panels) and add visualizations to compare data the way you like.
+  ![Sample Operational Panel View]({{site.url}}{{site.baseurl}}/images/operational-panel.png)
+1. Use [trace analytics]({{site.url}}{{site.baseurl}}/observability-plugins/trace/index) to create traces and dive deep into your data.
+  ![Sample Trace Analytics View]({{site.url}}{{site.baseurl}}/images/observability-trace.png)
+1. Leverage [notebooks]({{site.url}}{{site.baseurl}}/observability-plugins/notebooks) to combine different visualizations and code blocks that you can share with team members.
+  ![Sample Notebooks View]({{site.url}}{{site.baseurl}}/images/notebooks.png)

_observability-plugins/operational-panels.md

@@ -0,0 +1,25 @@
+---
+layout: default
+title: Operational panels
+nav_order: 30
+---
+
+# Operational panels
+
+Operational panels in OpenSearch Dashboards are collections of visualizations generated using [Piped Processing Language]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index) (PPL) queries.
+
+## Get started with operational panels
+
+If you want to start using operational panels without adding any data, expand the **Action** menu, choose **Add samples**, and Dashboards adds a set of operational panels with saved visualizations for you to explore.
+
+## Create an operational panel
+
+To create an operational panel and add visualizations:
+
+1. From the **Add Visualization** dropdown menu, choose **Select Existing Visualization** or **Create New Visualization**, which takes you to the [event analytics]({{site.url}}{{site.baseurl}}/observability-plugins/event-analytics) explorer, where you can use PPL to create visualizations.
+1. If you're adding already existing visualizations, choose a visualization from the dropdown menu.
+1. Choose **Add**.
+
+![Sample operational panel]({{site.url}}{{site.baseurl}}/images/operational-panel.png)
+
+To search for a particular visualization in your operation panels, use PPL queries to search for data you've already added to your panel.

_observability-plugins/ppl/commands.md

@@ -80,7 +80,7 @@ Field | Description | Type | Required | Default
 `int` |  Retain the specified number of duplicate events for each combination. The number must be greater than 0. If you do not specify a number, only the first occurring event is kept and all other duplicates are removed from the results. | `string` | No | 1
 `keepempty` | If true, keep the document if any field in the field list has a null value or a field missing. | `nested list of objects` | No | False
 `consecutive` | If true, remove only consecutive events with duplicate combinations of values. | `Boolean` | No | False
-`field-list` | Specify a comma-delimited field list. At least one field is required. | `string` or comma-separated list of strings | Yes | -
+`field-list` | Specify a comma-delimited field list. At least one field is required. | `String` or comma-separated list of strings | Yes | -
 
 *Example 1*: Dedup by one field
 

_observability-plugins/ppl/index.md

@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Piped processing language
-nav_order: 42
+nav_order: 40
 has_children: true
 has_toc: false
 redirect_from:

_observability-plugins/trace/data-prepper-reference.md

@@ -7,7 +7,7 @@ nav_order: 25
 
 # Data Prepper configuration reference
 
-This page lists all supported Data Prepper sources, buffers, preppers, and sinks, along with their associated options. For example configuration files, see [Data Prepper]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper/).
+This page lists all supported Data Prepper sources, buffers, preppers, and sinks, along with their associated options. For example configuration files, see [Data Prepper]({{site.url}}{{site.baseurl}}/observability-plugins/trace/data-prepper/).
 
 
 ## Data Prepper server options

_observability-plugins/trace/data-prepper.md

@@ -105,7 +105,7 @@ service-map-pipeline:
         trace_analytics_service_map: true
 ```
 
-To learn more, see the [Data Prepper configuration reference]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper-reference/).
+To learn more, see the [Data Prepper configuration reference]({{site.url}}{{site.baseurl}}/observability-plugins/trace/data-prepper-reference/).
 
 ## Configure the Data Prepper server
 Data Prepper itself provides administrative HTTP endpoints such as `/list` to list pipelines and `/metrics/prometheus` to provide Prometheus-compatible metrics data. The port which serves these endpoints, as well as TLS configuration, is specified by a separate YAML file. Example:

_observability-plugins/trace/get-started.md

@@ -20,9 +20,9 @@ OpenSearch Trace Analytics consists of two components---Data Prepper and the Tra
 
 1. The [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/) receives data from the application and formats it into OpenTelemetry data.
 
-1. [Data Prepper]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/data-prepper/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster.
+1. [Data Prepper]({{site.url}}{{site.baseurl}}/observability-plugins/trace/data-prepper/) processes the OpenTelemetry data, transforms it for use in OpenSearch, and indexes it on an OpenSearch cluster.
 
-1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput.
+1. The [Trace Analytics OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugins/trace/ta-dashboards/) displays the data in near real-time as a series of charts and tables, with an emphasis on service architecture, latency, error rate, and throughput.
 
 
 ## Jaeger HotROD
@@ -80,4 +80,4 @@ curl -X GET -u 'admin:admin' -k 'https://localhost:9200/otel-v1-apm-span-000001/
 
 Navigate to `http://localhost:5601` in a web browser and choose **Trace Analytics**. You can see the results of your single click in the Jaeger HotROD web interface: the number of traces per API and HTTP method, latency trends, a color-coded map of the service architecture, and a list of trace IDs that you can use to drill down on individual operations.
 
-If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/monitoring-plugins/trace/ta-dashboards/).
+If you don't see your trace, adjust the timeframe in OpenSearch Dashboards. For more information on using the plugin, see [OpenSearch Dashboards plugin]({{site.url}}{{site.baseurl}}/observability-plugins/trace/ta-dashboards/).

_observability-plugins/trace/index.md

@@ -1,11 +1,9 @@
 ---
 layout: default
 title: Trace analytics
-nav_order: 48
+nav_order: 60
 has_children: true
 has_toc: false
-redirect_from:
-  - /monitoring-plugins/trace/
 ---
 
 # Trace Analytics

_opensearch/data-streams.md

@@ -262,4 +262,4 @@ You can use wildcards to delete more than one data stream.
 
 We recommend deleting data from a data stream using an ISM policy.
 
-You can also use [asynchronous search]({{site.url}}{{site.baseurl}}/search-plugins/async/index/) and [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/search-plugins/ppl/index/) to query your data stream directly. You can also use the security plugin to define granular permissions on the data stream name.
+You can also use [asynchronous search]({{site.url}}{{site.baseurl}}/search-plugins/async/index/) and [SQL]({{site.url}}{{site.baseurl}}/search-plugins/sql/index/) and [PPL]({{site.url}}{{site.baseurl}}/observability-plugins/ppl/index/) to query your data stream directly. You can also use the security plugin to define granular permissions on the data stream name.