diff --git a/_config.yml b/_config.yml index 5d43186e..25a612a0 100644 --- a/_config.yml +++ b/_config.yml @@ -122,7 +122,7 @@ just_the_docs: name: Machine learning nav_fold: true tuning-your-cluster: - name: Tuning your cluster + name: Creating and tuning your cluster nav_fold: true monitoring-your-cluster: name: Monitoring your cluster diff --git a/_dashboards/index.md b/_dashboards/index.md new file mode 100644 index 00000000..0970f3d3 --- /dev/null +++ b/_dashboards/index.md @@ -0,0 +1,32 @@ +--- +layout: default +title: OpenSearch Dashboards +nav_order: 1 +has_children: false +--- + +# OpenSearch Dashboards + +OpenSearch Dashboards is the user interface that lets you visualize your OpenSearch data and run and scale your OpenSearch clusters. + +## Getting started + +| Concept | Description | +|---------|-------------| +| [OpenSearch Dashboards Quickstart]({{site.url}}{{site.baseurl}}/dashboards/quickstart-dashboards/) | Learn about the basic concepts and features of OpenSearch Dashboards. | +| [OpenSearch Playground](https://playground.opensearch.org/app/home#/) | Explore features in OpenSearch Dashboards without downloading or installing anything. | +| [Install and configure OpenSearch Dashboards]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/index/) | Get started with OpenSearch Dashboards. | +| [Create visualizations]({{site.url}}{{site.baseurl}}/dashboards/visualize/viz-index/) | Learn about visualizing data in OpenSearch Dashboards. | +| [Explore and query data]({{site.url}}{{site.baseurl}}/dashboards/discover/index-discover/) | Learn how to explore and query data in OpenSearch. | + +## Observability + +| Concept | Description | +|---------|-------------| +| [Observability in OpenSearch Dashboards]({{site.url}}{{site.baseurl}}//observing-your-data/index/) | Observe, monitor, and secure data and improve performance across tools and workflows. | + +## Dev Tools + +| Concept | Description | +|---------|-------------| +| [Dev Tools]({{site.url}}{{site.baseurl}}/dashboards/dev-tools/index-dev/) | Learn how to run OpenSearch queries in an integrated console. | diff --git a/_dashboards/quickstart-dashboards.md b/_dashboards/quickstart.md similarity index 98% rename from _dashboards/quickstart-dashboards.md rename to _dashboards/quickstart.md index 65945ac5..586bb485 100644 --- a/_dashboards/quickstart-dashboards.md +++ b/_dashboards/quickstart.md @@ -1,13 +1,13 @@ --- layout: default -title: Quickstart guide for OpenSearch Dashboards -nav_order: 10 +title: Quickstart guide +nav_order: 2 has_children: false redirect_from: - - /dashboards/index/ + - /dashboards/quickstart-dashboards/ --- -# Quickstart guide for OpenSearch Dashboards +# Quickstart guide This quickstart guide covers the core concepts that you need to understand to get started with OpenSearch Dashboards. You'll learn how to: diff --git a/_field-types/flat-object.md b/_field-types/flat-object.md index 7adc2599..1f76f6d1 100644 --- a/_field-types/flat-object.md +++ b/_field-types/flat-object.md @@ -9,7 +9,7 @@ grand_parent: Supported field types # Flat object field type -In OpenSearch, you don't have to specify a mapping before indexing documents. If you don't specify a mapping, OpenSearch uses [dynamic mapping]({{site.url}}{{site.baseurl}}/field-types/mappings#dynamic-mapping) to map every field and its subfields in the document automatically. When you ingest documents such as logs, you may not know every field's subfield name and type in advance. In this case, dynamically mapping all new subfields can quickly lead to a "mapping explosion," where the growing number of fields may degrade the performance of your cluster. +In OpenSearch, you don't have to specify a mapping before indexing documents. If you don't specify a mapping, OpenSearch uses [dynamic mapping]({{site.url}}{{site.baseurl}}/field-types/index#dynamic-mapping) to map every field and its subfields in the document automatically. When you ingest documents such as logs, you may not know every field's subfield name and type in advance. In this case, dynamically mapping all new subfields can quickly lead to a "mapping explosion," where the growing number of fields may degrade the performance of your cluster. The flat object field type solves this problem by treating the entire JSON object as a string. Subfields within the JSON object are accessible using standard dot path notation, but they are not indexed for fast lookup. diff --git a/_field-types/index.md b/_field-types/index.md index ee748870..f6c31b84 100644 --- a/_field-types/index.md +++ b/_field-types/index.md @@ -1,183 +1,157 @@ --- layout: default -title: Supported field types -nav_order: 80 -has_children: true -has_toc: false -redirect_from: - - /opensearch/supported-field-types/ - - //opensearch/supported-field-types/index/ +title: Mappings and field types +nav_order: 1 +redirect_from: + - /opensearch/mappings/ --- -# Supported field types +# Mappings and field types -You can specify data types for your fields when creating a mapping. The following table lists all data field types that OpenSearch supports. +You can define how documents and their fields are stored and indexed by creating a _mapping_. The mapping specifies the list of fields for a document. Every field in the document has a _field type_, which corresponds to the type of data the field contains. For example, you may want to specify that the `year` field should be of type `date`. To learn more, see [Supported field types]({{site.url}}{{site.baseurl}}/field-types/supported-field-types/index/). -Field data type | Description -:--- | :--- -[`alias`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/alias/) | An additional name for an existing field. -[`binary`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/binary/) | A binary value in Base64 encoding. -[Numeric]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/numeric/) | `byte`, `double`, `float`, `half_float`, `integer`, `long`, `scaled_float`, `short`. -[`boolean`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/boolean/) | A Boolean value. -[`date`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/) | A date value as a formatted string, a long value, or an integer. -[`ip`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/ip/) | An IP address in IPv4 or IPv6 format. -[Range]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/range/) | `integer_range`, `long_range`,`double_range`, `float_range`, `date_range`,`ip_range`. -[Object]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/object/) | `object`, `nested`, `flat_object`, `join`. -String | [`keyword`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/keyword/), [`text`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/text/), [`token_count`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/token-count/). -[Autocomplete]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/autocomplete/) | `completion`, `search_as_you_type`. -[Geographic]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/geographic/) | `geo_point`, `geo_shape`. -[Rank]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/rank/) | `rank_feature`, `rank_features`. -[`percolator`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/percolator/) | Specifies to treat this field as a query. +If you're just starting to build out your cluster and data, you may not know exactly how your data should be stored. In those cases, you can use dynamic mappings, which tell OpenSearch to dynamically add data and its fields. However, if you know exactly what types your data falls under and want to enforce that standard, then you can use explicit mappings. -## Arrays +For example, if you want to indicate that `year` should be of type `text` instead of an `integer`, and `age` should be an `integer`, you can do so with explicit mappings. By using dynamic mapping, OpenSearch might interpret both `year` and `age` as integers. -There is no dedicated array field type in OpenSearch. Instead, you can pass an array of values into any field. All values in the array must have the same field type. +This section provides an example for how to create an index mapping and how to add a document to it that will get ip_range validated. + +#### Table of contents +1. TOC +{:toc} + + +--- +## Dynamic mapping + +When you index a document, OpenSearch adds fields automatically with dynamic mapping. You can also explicitly add fields to an index mapping. + +#### Dynamic mapping types + +Type | Description +:--- | :--- +null | A `null` field can't be indexed or searched. When a field is set to null, OpenSearch behaves as if that field has no values. +boolean | OpenSearch accepts `true` and `false` as boolean values. An empty string is equal to `false.` +float | A single-precision 32-bit floating point number. +double | A double-precision 64-bit floating point number. +integer | A signed 32-bit number. +object | Objects are standard JSON objects, which can have fields and mappings of their own. For example, a `movies` object can have additional properties such as `title`, `year`, and `director`. +array | Arrays in OpenSearch can only store values of one type, such as an array of just integers or strings. Empty arrays are treated as though they are fields with no values. +text | A string sequence of characters that represent full-text values. +keyword | A string sequence of structured characters, such as an email address or ZIP code. +date detection string | Enabled by default, if new string fields match a date's format, then the string is processed as a `date` field. For example, `date: "2012/03/11"` is processed as a date. +numeric detection string | If disabled, OpenSearch may automatically process numeric values as strings when they should be processed as numbers. When enabled, OpenSearch can process strings into `long`, `integer`, `short`, `byte`, `double`, `float`, `half_float`, `scaled_float`, and `unsigned_long`. Default is disabled. + +## Explicit mapping + +If you know exactly what your field data types need to be, you can specify them in your request body when creating your index. ```json -PUT testindex1/_doc/1 -{ - "number": 1 -} - -PUT testindex1/_doc/2 -{ - "number": [1, 2, 3] -} -``` - -## Multifields - -Multifields are used to index the same field differently. Strings are often mapped as `text` for full-text queries and `keyword` for exact-value queries. - -Multifields can be created using the `fields` parameter. For example, you can map a book `title` to be of type `text` and keep a `title.raw` subfield of type `keyword`. - -```json -PUT books -{ - "mappings" : { - "properties" : { - "title" : { - "type" : "text", - "fields" : { - "raw" : { - "type" : "keyword" - } - } - } - } - } -} -``` - -## Null value - -Setting a field's value to `null`, an empty array or an array of `null` values makes this field equivalent to an empty field. Therefore, you cannot search for documents that have `null` in this field. - -To make a field searchable for `null` values, you can specify its `null_value` parameter in the index's mappings. Then, all `null` values passed to this field will be replaced with the specified `null_value`. - -The `null_value` parameter must be of the same type as the field. For example, if your field is a string, the `null_value` for this field must also be a string. -{: .note} - -### Example - -Create a mapping to replace `null` values in the `emergency_phone` field with the string "NONE": - -```json -PUT testindex +PUT sample-index1 { "mappings": { "properties": { - "name": { - "type": "keyword" - }, - "emergency_phone": { - "type": "keyword", - "null_value": "NONE" + "year": { "type" : "text" }, + "age": { "type" : "integer" }, + "director":{ "type" : "text" } + } + } +} +``` + +### Response +```json +{ + "acknowledged": true, + "shards_acknowledged": true, + "index": "sample-index1" +} +``` + +To add mappings to an existing index or data stream, you can send a request to the `_mapping` endpoint using the `PUT` or `POST` HTTP method: + +```json +POST sample-index1/_mapping +{ + "properties": { + "year": { "type" : "text" }, + "age": { "type" : "integer" }, + "director":{ "type" : "text" } + } +} +``` + +You cannot change the mapping of an existing field, you can only modify the field's mapping parameters. +{: .note} + +--- +## Mapping example usage + +The following example shows how to create a mapping to specify that OpenSearch should ignore any documents with malformed IP addresses that do not conform to the [`ip`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/ip/) data type. You accomplish this by setting the `ignore_malformed` parameter to `true`. + +### Create an index with an `ip` mapping + +To create an index, use a PUT request: + +```json +PUT /test-index +{ + "mappings" : { + "properties" : { + "ip_address" : { + "type" : "ip", + "ignore_malformed": true } } } } ``` -Index three documents into testindex. The `emergency_phone` fields of documents 1 and 3 contain `null`, while the `emergency_phone` field of document 2 has an empty array: +You can add a document that has a malformed IP address to your index: ```json -PUT testindex/_doc/1 +PUT /test-index/_doc/1 { - "name": "Akua Mansa", - "emergency_phone": null + "ip_address" : "malformed ip address" } ``` +This indexed IP address does not throw an error because `ignore_malformed` is set to true. + +You can query the index using the following request: + ```json -PUT testindex/_doc/2 -{ - "name": "Diego Ramirez", - "emergency_phone" : [] -} +GET /test-index/_search ``` -```json -PUT testindex/_doc/3 -{ - "name": "Jane Doe", - "emergency_phone": [null, null] -} -``` - -Search for people who do not have an emergency phone: - -```json -GET testindex/_search -{ - "query": { - "term": { - "emergency_phone": "NONE" - } - } -} -``` - -The response contains documents 1 and 3 but not document 2 because only explicit `null` values are replaced with the string "NONE": +The response shows that the `ip_address` field is ignored in the indexed document: ```json { - "took" : 1, - "timed_out" : false, - "_shards" : { - "total" : 1, - "successful" : 1, - "skipped" : 0, - "failed" : 0 + "took": 14, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 }, - "hits" : { - "total" : { - "value" : 2, - "relation" : "eq" + "hits": { + "total": { + "value": 1, + "relation": "eq" }, - "max_score" : 0.18232156, - "hits" : [ + "max_score": 1, + "hits": [ { - "_index" : "testindex", - "_type" : "_doc", - "_id" : "1", - "_score" : 0.18232156, - "_source" : { - "name" : "Akua Mansa", - "emergency_phone" : null - } - }, - { - "_index" : "testindex", - "_type" : "_doc", - "_id" : "3", - "_score" : 0.18232156, - "_source" : { - "name" : "Jane Doe", - "emergency_phone" : [ - null, - null - ] + "_index": "test-index", + "_id": "1", + "_score": 1, + "_ignored": [ + "ip_address" + ], + "_source": { + "ip_address": "malformed ip address" } } ] @@ -185,5 +159,60 @@ The response contains documents 1 and 3 but not document 2 because only explicit } ``` -The `_source` field still contains explicit `null` values because it is not affected by the `null_value`. -{: .note} \ No newline at end of file +## Get a mapping + +To get all mappings for one or more indexes, use the following request: + +```json +GET /_mapping +``` + +In the above request, `` may be an index name or a comma-separated list of index names. + +To get all mappings for all indexes, use the following request: + +```json +GET _mapping +``` + +To get a mapping for a specific field, provide the index name and the field name: + +```json +GET _mapping/field/ +GET //_mapping/field/ +``` + +Both `` and `` can be specified as one value or a comma-separated list. + +For example, the following request retrieves the mapping for the `year` and `age` fields in `sample-index1`: + +```json +GET sample-index1/_mapping/field/year,age +``` + +The response contains the specified fields: + +```json +{ + "sample-index1" : { + "mappings" : { + "year" : { + "full_name" : "year", + "mapping" : { + "year" : { + "type" : "text" + } + } + }, + "age" : { + "full_name" : "age", + "mapping" : { + "age" : { + "type" : "integer" + } + } + } + } + } +} +``` \ No newline at end of file diff --git a/_field-types/mappings.md b/_field-types/mappings.md deleted file mode 100644 index 8a4f2922..00000000 --- a/_field-types/mappings.md +++ /dev/null @@ -1,218 +0,0 @@ ---- -layout: default -title: Mapping -nav_order: 13 -redirect_from: - - /opensearch/mappings/ ---- - -# Mapping - -You can define how documents and their fields are stored and indexed by creating a mapping. - -If you're just starting to build out your cluster and data, you may not know exactly how your data should be stored. In those cases, you can use dynamic mappings, which tell OpenSearch to dynamically add data and its fields. However, if you know exactly what types your data falls under and want to enforce that standard, then you can use explicit mappings. - -For example, if you want to indicate that `year` should be of type `text` instead of an `integer`, and `age` should be an `integer`, you can do so with explicit mappings. Using dynamic mapping OpenSearch might interpret both `year` and `age` as integers. - -This section provides an example for how to create an index mapping, and how to add a document to it that will get ip_range validated. - -#### Table of contents -1. TOC -{:toc} - - ---- -## Dynamic mapping - -When you index a document, OpenSearch adds fields automatically with dynamic mapping. You can also explicitly add fields to an index mapping. - -#### Dynamic mapping types - -Type | Description -:--- | :--- -null | A `null` field can't be indexed or searched. When a field is set to null, OpenSearch behaves as if that field has no values. -boolean | OpenSearch accepts `true` and `false` as boolean values. An empty string is equal to `false.` -float | A single-precision 32-bit floating point number. -double | A double-precision 64-bit floating point number. -integer | A signed 32-bit number. -object | Objects are standard JSON objects, which can have fields and mappings of their own. For example, a `movies` object can have additional properties such as `title`, `year`, and `director`. -array | Arrays in OpenSearch can only store values of one type, such as an array of just integers or strings. Empty arrays are treated as though they are fields with no values. -text | A string sequence of characters that represent full-text values. -keyword | A string sequence of structured characters, such as an email address or ZIP code. -date detection string | Enabled by default, if new string fields match a date's format, then the string is processed as a `date` field. For example, `date: "2012/03/11"` is processed as a date. -numeric detection string | If disabled, OpenSearch may automatically process numeric values as strings when they should be processed as numbers. When enabled, OpenSearch can process strings into `long`, `integer`, `short`, `byte`, `double`, `float`, `half_float`, `scaled_float`, and `unsigned_long`. Default is disabled. - -## Explicit mapping - -If you know exactly what your field data types need to be, you can specify them in your request body when creating your index. - -```json -PUT sample-index1 -{ - "mappings": { - "properties": { - "year": { "type" : "text" }, - "age": { "type" : "integer" }, - "director":{ "type" : "text" } - } - } -} -``` - -### Response -```json -{ - "acknowledged": true, - "shards_acknowledged": true, - "index": "sample-index1" -} -``` - -To add mappings to an existing index or data stream, you can send a request to the `_mapping` endpoint using the `PUT` or `POST` HTTP method: - -```json -POST sample-index1/_mapping -{ - "properties": { - "year": { "type" : "text" }, - "age": { "type" : "integer" }, - "director":{ "type" : "text" } - } -} -``` - -You cannot change the mapping of an existing field, you can only modify the field's mapping parameters. -{: .note} - ---- -## Mapping example usage - -The following example shows how to create a mapping to specify that OpenSearch should ignore any documents with malformed IP addresses that do not conform to the [`ip`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/ip/) data type. You accomplish this by setting the `ignore_malformed` parameter to `true`. - -### Create an index with an `ip` mapping - -To create an index, use a PUT request: - -```json -PUT /test-index -{ - "mappings" : { - "properties" : { - "ip_address" : { - "type" : "ip", - "ignore_malformed": true - } - } - } -} -``` - -You can add a document that has a malformed IP address to your index: - -```json -PUT /test-index/_doc/1 -{ - "ip_address" : "malformed ip address" -} -``` - -This indexed IP address does not throw an error because `ignore_malformed` is set to true. - -You can query the index using the following request: - -```json -GET /test-index/_search -``` - -The response shows that the `ip_address` field is ignored in the indexed document: - -```json -{ - "took": 14, - "timed_out": false, - "_shards": { - "total": 1, - "successful": 1, - "skipped": 0, - "failed": 0 - }, - "hits": { - "total": { - "value": 1, - "relation": "eq" - }, - "max_score": 1, - "hits": [ - { - "_index": "test-index", - "_id": "1", - "_score": 1, - "_ignored": [ - "ip_address" - ], - "_source": { - "ip_address": "malformed ip address" - } - } - ] - } -} -``` - -## Get a mapping - -To get all mappings for one or more indexes, use the following request: - -```json -GET /_mapping -``` - -In the above request, `` may be an index name or a comma-separated list of index names. - -To get all mappings for all indexes, use the following request: - -```json -GET _mapping -``` - -To get a mapping for a specific field, provide the index name and the field name: - -```json -GET _mapping/field/ -GET //_mapping/field/ -``` - -Both `` and `` can be specified as one value or a comma-separated list. - -For example, the following request retrieves the mapping for the `year` and `age` fields in `sample-index1`: - -```json -GET sample-index1/_mapping/field/year,age -``` - -The response contains the specified fields: - -```json -{ - "sample-index1" : { - "mappings" : { - "year" : { - "full_name" : "year", - "mapping" : { - "year" : { - "type" : "text" - } - } - }, - "age" : { - "full_name" : "age", - "mapping" : { - "age" : { - "type" : "integer" - } - } - } - } - } -} -``` \ No newline at end of file diff --git a/_field-types/alias.md b/_field-types/supported-field-types/alias.md similarity index 99% rename from _field-types/alias.md rename to _field-types/supported-field-types/alias.md index 8cb4e6d2..29cc5888 100644 --- a/_field-types/alias.md +++ b/_field-types/supported-field-types/alias.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/alias/ + - /field-types/alias/ --- # Alias field type diff --git a/_field-types/autocomplete.md b/_field-types/supported-field-types/autocomplete.md similarity index 96% rename from _field-types/autocomplete.md rename to _field-types/supported-field-types/autocomplete.md index 006f0786..79b98aa1 100644 --- a/_field-types/autocomplete.md +++ b/_field-types/supported-field-types/autocomplete.md @@ -7,6 +7,7 @@ has_toc: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/autocomplete/ + - /field-types/autocomplete/ --- # Autocomplete field types diff --git a/_field-types/binary.md b/_field-types/supported-field-types/binary.md similarity index 98% rename from _field-types/binary.md rename to _field-types/supported-field-types/binary.md index 7786785b..7e253a03 100644 --- a/_field-types/binary.md +++ b/_field-types/supported-field-types/binary.md @@ -6,6 +6,7 @@ nav_order: 12 has_children: false redirect_from: - /opensearch/supported-field-types/binary/ + - /field-types/binary/ --- # Binary field type diff --git a/_field-types/boolean.md b/_field-types/supported-field-types/boolean.md similarity index 99% rename from _field-types/boolean.md rename to _field-types/supported-field-types/boolean.md index 4e5e795f..3993f312 100644 --- a/_field-types/boolean.md +++ b/_field-types/supported-field-types/boolean.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/boolean/ + - /field-types/boolean/ --- # Boolean field type diff --git a/_field-types/completion.md b/_field-types/supported-field-types/completion.md similarity index 99% rename from _field-types/completion.md rename to _field-types/supported-field-types/completion.md index 05fcbaa5..9214c258 100644 --- a/_field-types/completion.md +++ b/_field-types/supported-field-types/completion.md @@ -7,6 +7,7 @@ parent: Autocomplete field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/completion/ + - /field-types/completion/ --- # Completion field type diff --git a/_field-types/date.md b/_field-types/supported-field-types/date.md similarity index 99% rename from _field-types/date.md rename to _field-types/supported-field-types/date.md index 075eed36..ea093117 100644 --- a/_field-types/date.md +++ b/_field-types/supported-field-types/date.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/date/ + - /field-types/date/ --- # Date field type diff --git a/_field-types/geo-point.md b/_field-types/supported-field-types/geo-point.md similarity index 98% rename from _field-types/geo-point.md rename to _field-types/supported-field-types/geo-point.md index ee40a133..0912dc61 100644 --- a/_field-types/geo-point.md +++ b/_field-types/supported-field-types/geo-point.md @@ -7,6 +7,7 @@ parent: Geographic field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/geo-point/ + - /field-types/geo-point/ --- # Geopoint field type diff --git a/_field-types/geo-shape.md b/_field-types/supported-field-types/geo-shape.md similarity index 99% rename from _field-types/geo-shape.md rename to _field-types/supported-field-types/geo-shape.md index cf0506de..cbf63551 100644 --- a/_field-types/geo-shape.md +++ b/_field-types/supported-field-types/geo-shape.md @@ -7,6 +7,7 @@ parent: Geographic field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/geo-shape/ + - /field-types/geo-shape/ --- # Geoshape field type diff --git a/_field-types/geographic.md b/_field-types/supported-field-types/geographic.md similarity index 95% rename from _field-types/geographic.md rename to _field-types/supported-field-types/geographic.md index 34d1a54f..07d03820 100644 --- a/_field-types/geographic.md +++ b/_field-types/supported-field-types/geographic.md @@ -7,6 +7,7 @@ has_toc: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/geographic/ + - /field-types/geographic/ --- # Geographic field types diff --git a/_field-types/supported-field-types/index.md b/_field-types/supported-field-types/index.md new file mode 100644 index 00000000..230f6351 --- /dev/null +++ b/_field-types/supported-field-types/index.md @@ -0,0 +1,189 @@ +--- +layout: default +title: Supported field types +nav_order: 80 +has_children: true +has_toc: false +redirect_from: + - /opensearch/supported-field-types/ + - /opensearch/supported-field-types/index/ +--- + +# Supported field types + +You can specify data types for your fields when creating a mapping. The following table lists all data field types that OpenSearch supports. + +Field data type | Description +:--- | :--- +[`alias`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/alias/) | An additional name for an existing field. +[`binary`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/binary/) | A binary value in Base64 encoding. +[Numeric]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/numeric/) | `byte`, `double`, `float`, `half_float`, `integer`, `long`, `scaled_float`, `short`. +[`boolean`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/boolean/) | A Boolean value. +[`date`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/date/) | A date value as a formatted string, a long value, or an integer. +[`ip`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/ip/) | An IP address in IPv4 or IPv6 format. +[Range]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/range/) | `integer_range`, `long_range`,`double_range`, `float_range`, `date_range`,`ip_range`. +[Object]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/object/) | `object`, `nested`, `join`. +String | [`keyword`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/keyword/), [`text`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/text/), [`token_count`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/token-count/). +[Autocomplete]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/autocomplete/) | `completion`, `search_as_you_type`. +[Geographic]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/geographic/) | `geo_point`, `geo_shape`. +[Rank]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/rank/) | `rank_feature`, `rank_features`. +[`percolator`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/percolator/) | Specifies to treat this field as a query. + +## Arrays + +There is no dedicated array field type in OpenSearch. Instead, you can pass an array of values into any field. All values in the array must have the same field type. + +```json +PUT testindex1/_doc/1 +{ + "number": 1 +} + +PUT testindex1/_doc/2 +{ + "number": [1, 2, 3] +} +``` + +## Multifields + +Multifields are used to index the same field differently. Strings are often mapped as `text` for full-text queries and `keyword` for exact-value queries. + +Multifields can be created using the `fields` parameter. For example, you can map a book `title` to be of type `text` and keep a `title.raw` subfield of type `keyword`. + +```json +PUT books +{ + "mappings" : { + "properties" : { + "title" : { + "type" : "text", + "fields" : { + "raw" : { + "type" : "keyword" + } + } + } + } + } +} +``` + +## Null value + +Setting a field's value to `null`, an empty array, or an array of `null` values makes this field equivalent to an empty field. Therefore, you cannot search for documents that have `null` in this field. + +To make a field searchable for `null` values, you can specify its `null_value` parameter in the index's mappings. Then, all `null` values passed to this field will be replaced with the specified `null_value`. + +The `null_value` parameter must be of the same type as the field. For example, if your field is a string, the `null_value` for this field must also be a string. +{: .note} + +### Example + +Create a mapping to replace `null` values in the `emergency_phone` field with the string "NONE": + +```json +PUT testindex +{ + "mappings": { + "properties": { + "name": { + "type": "keyword" + }, + "emergency_phone": { + "type": "keyword", + "null_value": "NONE" + } + } + } +} +``` + +Index three documents into testindex. The `emergency_phone` fields of documents 1 and 3 contain `null`, while the `emergency_phone` field of document 2 has an empty array: + +```json +PUT testindex/_doc/1 +{ + "name": "Akua Mansa", + "emergency_phone": null +} +``` + +```json +PUT testindex/_doc/2 +{ + "name": "Diego Ramirez", + "emergency_phone" : [] +} +``` + +```json +PUT testindex/_doc/3 +{ + "name": "Jane Doe", + "emergency_phone": [null, null] +} +``` + +Search for people who do not have an emergency phone: + +```json +GET testindex/_search +{ + "query": { + "term": { + "emergency_phone": "NONE" + } + } +} +``` + +The response contains documents 1 and 3 but not document 2 because only explicit `null` values are replaced with the string "NONE": + +```json +{ + "took" : 1, + "timed_out" : false, + "_shards" : { + "total" : 1, + "successful" : 1, + "skipped" : 0, + "failed" : 0 + }, + "hits" : { + "total" : { + "value" : 2, + "relation" : "eq" + }, + "max_score" : 0.18232156, + "hits" : [ + { + "_index" : "testindex", + "_type" : "_doc", + "_id" : "1", + "_score" : 0.18232156, + "_source" : { + "name" : "Akua Mansa", + "emergency_phone" : null + } + }, + { + "_index" : "testindex", + "_type" : "_doc", + "_id" : "3", + "_score" : 0.18232156, + "_source" : { + "name" : "Jane Doe", + "emergency_phone" : [ + null, + null + ] + } + } + ] + } +} +``` + +The `_source` field still contains explicit `null` values because it is not affected by the `null_value`. +{: .note} \ No newline at end of file diff --git a/_field-types/ip.md b/_field-types/supported-field-types/ip.md similarity index 99% rename from _field-types/ip.md rename to _field-types/supported-field-types/ip.md index a3e1277b..1793bf31 100644 --- a/_field-types/ip.md +++ b/_field-types/supported-field-types/ip.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/ip/ + - /field-types/ip/ --- # IP address field type diff --git a/_field-types/join.md b/_field-types/supported-field-types/join.md similarity index 99% rename from _field-types/join.md rename to _field-types/supported-field-types/join.md index 806c7eb3..c83705f4 100644 --- a/_field-types/join.md +++ b/_field-types/supported-field-types/join.md @@ -7,6 +7,7 @@ parent: Object field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/join/ + - /field-types/join/ --- # Join field type diff --git a/_field-types/keyword.md b/_field-types/supported-field-types/keyword.md similarity index 99% rename from _field-types/keyword.md rename to _field-types/supported-field-types/keyword.md index 4624ebb8..22271ce1 100644 --- a/_field-types/keyword.md +++ b/_field-types/supported-field-types/keyword.md @@ -7,6 +7,7 @@ parent: String field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/keyword/ + - /field-types/keyword/ --- # Keyword field type diff --git a/_field-types/nested.md b/_field-types/supported-field-types/nested.md similarity index 99% rename from _field-types/nested.md rename to _field-types/supported-field-types/nested.md index a075d238..d09caf0e 100644 --- a/_field-types/nested.md +++ b/_field-types/supported-field-types/nested.md @@ -7,6 +7,7 @@ parent: Object field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/nested/ + - /field-types/nested/ --- # Nested field type diff --git a/_field-types/numeric.md b/_field-types/supported-field-types/numeric.md similarity index 99% rename from _field-types/numeric.md rename to _field-types/supported-field-types/numeric.md index e76b64ff..d31cad8f 100644 --- a/_field-types/numeric.md +++ b/_field-types/supported-field-types/numeric.md @@ -6,6 +6,7 @@ nav_order: 15 has_children: false redirect_from: - /opensearch/supported-field-types/numeric/ + - /field-types/numeric/ --- # Numeric field types diff --git a/_field-types/object-fields.md b/_field-types/supported-field-types/object-fields.md similarity index 88% rename from _field-types/object-fields.md rename to _field-types/supported-field-types/object-fields.md index b0c416f8..64869fc3 100644 --- a/_field-types/object-fields.md +++ b/_field-types/supported-field-types/object-fields.md @@ -5,6 +5,9 @@ nav_order: 40 has_children: true has_toc: false parent: Supported field types +redirect_from: + - /opensearch/supported-field-types/object-fields/ + - /field-types/object-fields/ --- # Object field types diff --git a/_field-types/object.md b/_field-types/supported-field-types/object.md similarity index 99% rename from _field-types/object.md rename to _field-types/supported-field-types/object.md index 7bb446e1..372a5c46 100644 --- a/_field-types/object.md +++ b/_field-types/supported-field-types/object.md @@ -7,6 +7,7 @@ parent: Object field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/object/ + - /field-types/object/ --- # Object field type diff --git a/_field-types/percolator.md b/_field-types/supported-field-types/percolator.md similarity index 99% rename from _field-types/percolator.md rename to _field-types/supported-field-types/percolator.md index e39f1cad..92325b61 100644 --- a/_field-types/percolator.md +++ b/_field-types/supported-field-types/percolator.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/percolator/ + - /field-types/percolator/ --- # Percolator field type diff --git a/_field-types/range.md b/_field-types/supported-field-types/range.md similarity index 99% rename from _field-types/range.md rename to _field-types/supported-field-types/range.md index 3815053f..af770678 100644 --- a/_field-types/range.md +++ b/_field-types/supported-field-types/range.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/range/ + - /field-types/range/ --- # Range field types diff --git a/_field-types/rank.md b/_field-types/supported-field-types/rank.md similarity index 99% rename from _field-types/rank.md rename to _field-types/supported-field-types/rank.md index 691fbbb7..c46467f8 100644 --- a/_field-types/rank.md +++ b/_field-types/supported-field-types/rank.md @@ -6,6 +6,7 @@ has_children: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/rank/ + - /field-types/rank/ --- # Rank field types diff --git a/_field-types/search-as-you-type.md b/_field-types/supported-field-types/search-as-you-type.md similarity index 99% rename from _field-types/search-as-you-type.md rename to _field-types/supported-field-types/search-as-you-type.md index fdef9c0c..b9141e6b 100644 --- a/_field-types/search-as-you-type.md +++ b/_field-types/supported-field-types/search-as-you-type.md @@ -7,6 +7,7 @@ parent: Autocomplete field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/search-as-you-type/ + - /field-types/search-as-you-type/ --- # Search-as-you-type field type diff --git a/_field-types/string.md b/_field-types/supported-field-types/string.md similarity index 96% rename from _field-types/string.md rename to _field-types/supported-field-types/string.md index 304fd434..21cee52d 100644 --- a/_field-types/string.md +++ b/_field-types/supported-field-types/string.md @@ -7,6 +7,7 @@ has_toc: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/string/ + - /field-types/string/ --- # String field types diff --git a/_field-types/text.md b/_field-types/supported-field-types/text.md similarity index 99% rename from _field-types/text.md rename to _field-types/supported-field-types/text.md index c68f678f..0a16f3aa 100644 --- a/_field-types/text.md +++ b/_field-types/supported-field-types/text.md @@ -7,6 +7,7 @@ parent: String field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/text/ + - /field-types/text/ --- # Text field type diff --git a/_field-types/token-count.md b/_field-types/supported-field-types/token-count.md similarity index 99% rename from _field-types/token-count.md rename to _field-types/supported-field-types/token-count.md index adc5b725..07982431 100644 --- a/_field-types/token-count.md +++ b/_field-types/supported-field-types/token-count.md @@ -7,6 +7,7 @@ parent: String field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/token-count/ + - /field-types/token-count/ --- # Token count field type diff --git a/_field-types/xy-point.md b/_field-types/supported-field-types/xy-point.md similarity index 99% rename from _field-types/xy-point.md rename to _field-types/supported-field-types/xy-point.md index b9edabab..57b6f647 100644 --- a/_field-types/xy-point.md +++ b/_field-types/supported-field-types/xy-point.md @@ -7,6 +7,7 @@ parent: Cartesian field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/xy-point/ + - /field-types/xy-point/ --- # xy point field type diff --git a/_field-types/xy-shape.md b/_field-types/supported-field-types/xy-shape.md similarity index 99% rename from _field-types/xy-shape.md rename to _field-types/supported-field-types/xy-shape.md index e6f95b73..f1c71912 100644 --- a/_field-types/xy-shape.md +++ b/_field-types/supported-field-types/xy-shape.md @@ -7,6 +7,7 @@ parent: Cartesian field types grand_parent: Supported field types redirect_from: - /opensearch/supported-field-types/xy-shape/ + - /field-types/xy-shape/ --- # xy shape field type diff --git a/_field-types/xy.md b/_field-types/supported-field-types/xy.md similarity index 98% rename from _field-types/xy.md rename to _field-types/supported-field-types/xy.md index 2ab07be5..a6f8a45a 100644 --- a/_field-types/xy.md +++ b/_field-types/supported-field-types/xy.md @@ -7,6 +7,7 @@ has_toc: false parent: Supported field types redirect_from: - /opensearch/supported-field-types/xy/ + - /field-types/xy/ --- # Cartesian field types diff --git a/_install-and-configure/index.md b/_install-and-configure/index.md new file mode 100644 index 00000000..9237d4d5 --- /dev/null +++ b/_install-and-configure/index.md @@ -0,0 +1,29 @@ +--- +layout: default +title: Install and upgrade OpenSearch +nav_order: 1 +has_children: false +has_toc: false +--- + +# Install and upgrade OpenSearch + +OpenSearch and OpenSearch Dashboards are available on any compatible host that supports Docker (such as Linux, MacOS, or Windows). Additionally, you can install both products on various Linux distributions and on Windows. + +[Download OpenSearch](https://opensearch.org/downloads.html) for your preferred platform and then choose one of the following installation guides. + +| OpenSearch | OpenSearch Dashboards | +| :--- | :--- | +| [Docker]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/docker/) | [Docker]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/docker/) | +| [Helm]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/helm/) | [Helm]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/helm/) | +| [Tarball]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/tar/) | [Tarball]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/tar/) | +| [RPM]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/rpm/) | [RPM]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/rpm/) | +| [Debian]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/debian/) | [Debian]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/debian/) | +| [Ansible playbook]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/ansible/) | | +| [Windows]({{site.url}}{{site.baseurl}}/install-and-configure/install-opensearch/windows/) | [Windows]({{site.url}}{{site.baseurl}}/install-and-configure/install-dashboards/windows/) | + +After you've installed OpenSearch, learn about [configuring]({{site.url}}{{site.baseurl}}/install-and-configure/configuration/) it for your deployment. + +For more information about upgrading your OpenSearch cluster, see the [upgrade guide]({{site.url}}{{site.baseurl}}/install-and-configure/upgrade-opensearch/index/). + +For plugin installation, see [Installing plugins]({{site.url}}{{site.baseurl}}/install-and-configure/plugins/). \ No newline at end of file diff --git a/_monitoring-your-cluster/index.md b/_monitoring-your-cluster/index.md new file mode 100644 index 00000000..92255382 --- /dev/null +++ b/_monitoring-your-cluster/index.md @@ -0,0 +1,17 @@ +--- +layout: default +title: Monitoring your cluster +nav_order: 1 +has_children: false +has_toc: false +--- + +# Monitoring your cluster + +OpenSearch provides several ways for you to monitor your cluster health and performance and automate common tasks: + +- The OpenSearch [logs]({{site.url}}{{site.baseurl}}/monitoring-your-cluster/logs/) include valuable information for monitoring cluster operations and troubleshooting issues. + +- [Performance analyzer]({{site.url}}{{site.baseurl}}/monitoring-your-cluster/pa/index/) is an agent and REST API that allows you to query numerous performance metrics for your cluster, including aggregations of those metrics. + +- OpenSearch [Job Scheduler]({{site.url}}{{site.baseurl}}/monitoring-your-cluster/job-scheduler/index/) plugin provides a framework that you can use to build schedules for common cluster management tasks. \ No newline at end of file diff --git a/_query-dsl/analyzers/text-analyzers.md b/_query-dsl/analyzers/text-analyzers.md index 0003740c..a535fc3b 100644 --- a/_query-dsl/analyzers/text-analyzers.md +++ b/_query-dsl/analyzers/text-analyzers.md @@ -6,6 +6,7 @@ has_children: true permalink: /analyzers/text-analyzers/ redirect_from: - /opensearch/query-dsl/text-analyzers/ + - /query-dsl/analyzers/text-analyzers/ --- diff --git a/_query-dsl/index.md b/_query-dsl/index.md new file mode 100644 index 00000000..94c0f35f --- /dev/null +++ b/_query-dsl/index.md @@ -0,0 +1,15 @@ +--- +layout: default +title: Query DSL, aggregations, and analyzers +nav_order: 1 +has_children: false +has_toc: false +--- + +# Query DSL, aggregations, and analyzers + +[Analyzers]({{site.url}}{{site.baseurl}}/analyzers/text-analyzers/) process text to make it searchable. OpenSearch provides various analyzers that let you customize the way text is split into terms and converted into a structured format. To search documents written in a different language, you can use one of the built-in [language analyzers]({{site.url}}{{site.baseurl}}/query-dsl/analyzers/language-analyzers/) for your language of choice. + +The most essential search function is using a query to return relevant documents. OpenSearch provides a search language called _query domain-specific language_ (DSL) that lets you build complex and targeted queries. Explore the [query DSL documentation]({{site.url}}{{site.baseurl}}/query-dsl/) to learn more about the different types of queries OpenSearch supports. + +[Aggregations]({{site.url}}{{site.baseurl}}/aggregations/) let you categorize your data and analyze it to extract statistics. Use cases for aggregations include analyzing data in real time and using OpenSearch Dashboards to create visualizations. \ No newline at end of file diff --git a/_search-plugins/index.md b/_search-plugins/index.md new file mode 100644 index 00000000..b366dbb0 --- /dev/null +++ b/_search-plugins/index.md @@ -0,0 +1,29 @@ +--- +layout: default +title: Search +nav_order: 1 +has_children: false +has_toc: false +--- + +# Search + +OpenSearch provides several features for customizing your search use cases and improving search relevance. In OpenSearch, you can: + +- Use [SQL and Piped Processing Language (PPL)]({{site.url}}{{site.baseurl}}/search-plugins/sql/) as alternatives to [query domain-specific language (DSL)]({{site.url}}{{site.baseurl}}/query-dsl/) to search data. + +- Run resource-intensive queries asynchronously with [asynchronous search]({{site.url}}{{site.baseurl}}/search-plugins/async/). + +- Search for k-nearest neighbors with [k-NN search]({{site.url}}{{site.baseurl}}/search-plugins/knn/). + +- Abstract OpenSearch queries into [search templates]({{site.url}}{{site.baseurl}}/search-plugins/search-template/). + +- Integrate machine learning (ML) language models into your search workloads with [neural search]({{site.url}}{{site.baseurl}}/search-plugins/neural-search/). + +- [Compare search results]({{site.url}}{{site.baseurl}}/search-plugins/search-relevance/) to tune search relevance. + +- Use a dataset that is fixed in time to paginate results with [Point in Time]({{site.url}}{{site.baseurl}}/search-plugins/point-in-time/). + +- [Paginate]({{site.url}}{{site.baseurl}}/search-plugins/searching-data/paginate) and [sort]({{site.url}}{{site.baseurl}}/search-plugins/searching-data/sort/) search results, [highlight]({{site.url}}{{site.baseurl}}/search-plugins/searching-data/highlight/) search terms, and use the [autocomplete]({{site.url}}{{site.baseurl}}/search-plugins/searching-data/autocomplete/) and [did-you-mean]({{site.url}}{{site.baseurl}}/search-plugins/searching-data/did-you-mean/) functionality. + +- Rewrite queries with [Querqy]({{site.url}}{{site.baseurl}}/search-plugins/querqy). diff --git a/_tuning-your-cluster/cluster.md b/_tuning-your-cluster/index.md similarity index 99% rename from _tuning-your-cluster/cluster.md rename to _tuning-your-cluster/index.md index 20871f4e..125aa4c5 100644 --- a/_tuning-your-cluster/cluster.md +++ b/_tuning-your-cluster/index.md @@ -1,9 +1,10 @@ --- layout: default title: Creating a cluster -nav_order: 8 +nav_order: 1 redirect_from: - /opensearch/cluster/ + - /tuning-your-cluster/cluster/ --- # Creating a cluster