Style guide fix - white space (#6023)
* Style guide fix - white space Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Update _data-prepper/pipelines/configuration/processors/parse-json.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com>
This commit is contained in:
parent
bce6a3a6bc
commit
a7da03343e
|
@ -56,7 +56,7 @@ Normalization | `arabic_normalization`: [ArabicNormalizer](https://lucene.apache
|
||||||
`stop` | [StopFilter](https://lucene.apache.org/core/8_7_0/core/org/apache/lucene/analysis/StopFilter.html) | Removes stop words from a token stream.
|
`stop` | [StopFilter](https://lucene.apache.org/core/8_7_0/core/org/apache/lucene/analysis/StopFilter.html) | Removes stop words from a token stream.
|
||||||
`synonym` | N/A | Supplies a synonym list for the analysis process. The synonym list is provided using a configuration file.
|
`synonym` | N/A | Supplies a synonym list for the analysis process. The synonym list is provided using a configuration file.
|
||||||
`synonym_graph` | N/A | Supplies a synonym list, including multiword synonyms, for the analysis process.
|
`synonym_graph` | N/A | Supplies a synonym list, including multiword synonyms, for the analysis process.
|
||||||
`trim` | [TrimFilter](https://lucene.apache.org/core/8_7_0/analyzers-common/org/apache/lucene/analysis/miscellaneous/TrimFilter.html) | Trims leading and trailing whitespace from each token in a stream.
|
`trim` | [TrimFilter](https://lucene.apache.org/core/8_7_0/analyzers-common/org/apache/lucene/analysis/miscellaneous/TrimFilter.html) | Trims leading and trailing white space from each token in a stream.
|
||||||
`truncate` | [TruncateTokenFilter](https://lucene.apache.org/core/8_7_0/analyzers-common/org/apache/lucene/analysis/miscellaneous/TruncateTokenFilter.html) | Truncates tokens whose length exceeds the specified character limit.
|
`truncate` | [TruncateTokenFilter](https://lucene.apache.org/core/8_7_0/analyzers-common/org/apache/lucene/analysis/miscellaneous/TruncateTokenFilter.html) | Truncates tokens whose length exceeds the specified character limit.
|
||||||
`unique` | N/A | Ensures each token is unique by removing duplicate tokens from a stream.
|
`unique` | N/A | Ensures each token is unique by removing duplicate tokens from a stream.
|
||||||
`uppercase` | [UpperCaseFilter](https://lucene.apache.org/core/8_7_0/analyzers-common/org/apache/lucene/analysis/core/LowerCaseFilter.html) | Converts tokens to uppercase.
|
`uppercase` | [UpperCaseFilter](https://lucene.apache.org/core/8_7_0/analyzers-common/org/apache/lucene/analysis/core/LowerCaseFilter.html) | Converts tokens to uppercase.
|
||||||
|
|
|
@ -211,7 +211,7 @@ After you run Data Prepper, the source is converted to the following format:
|
||||||
|
|
||||||
## trim_string
|
## trim_string
|
||||||
|
|
||||||
The `trim_string` processor removes whitespace from the beginning and end of a key.
|
The `trim_string` processor removes white space from the beginning and end of a key.
|
||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
|
@ -219,7 +219,7 @@ You can configure the `trim_string` processor with the following options.
|
||||||
|
|
||||||
Option | Required | Description
|
Option | Required | Description
|
||||||
:--- | :--- | :---
|
:--- | :--- | :---
|
||||||
`with_keys` | Yes | A list of keys from which to trim the whitespace. |
|
`with_keys` | Yes | A list of keys from which to trim the white space. |
|
||||||
|
|
||||||
### Usage
|
### Usage
|
||||||
|
|
||||||
|
|
|
@ -18,7 +18,7 @@ You can configure the `parse_json` processor with the following options.
|
||||||
| Option | Required | Type | Description |
|
| Option | Required | Type | Description |
|
||||||
| :--- | :--- | :--- | :--- |
|
| :--- | :--- | :--- | :--- |
|
||||||
| `source` | No | String | The field in the `event` that will be parsed. Default value is `message`. |
|
| `source` | No | String | The field in the `event` that will be parsed. Default value is `message`. |
|
||||||
| `destination` | No | String | The destination field of the parsed JSON. Defaults to the root of the `event`. Cannot be `""`, `/`, or any whitespace-only `string` because these are not valid `event` fields. |
|
| `destination` | No | String | The destination field of the parsed JSON. Defaults to the root of the `event`. Cannot be `""`, `/`, or any white-space-only `string` because these are not valid `event` fields. |
|
||||||
| `pointer` | No | String | A JSON pointer to the field to be parsed. There is no `pointer` by default, meaning the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid then the entire `source` data is parsed into the outgoing `event`. If the key that is pointed to already exists in the `event` and the `destination` is the root, then the pointer uses the entire path of the key. |
|
| `pointer` | No | String | A JSON pointer to the field to be parsed. There is no `pointer` by default, meaning the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid then the entire `source` data is parsed into the outgoing `event`. If the key that is pointed to already exists in the `event` and the `destination` is the root, then the pointer uses the entire path of the key. |
|
||||||
|
|
||||||
## Usage
|
## Usage
|
||||||
|
|
|
@ -8,11 +8,11 @@ nav_order: 120
|
||||||
|
|
||||||
# trim_string
|
# trim_string
|
||||||
|
|
||||||
The `trim_string` processor removes whitespace from the beginning and end of a key and is a [mutate string](https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/mutate-string-processors#mutate-string-processors) processor. The following table describes the option you can use to configure the `trim_string` processor.
|
The `trim_string` processor removes white space from the beginning and end of a key and is a [mutate string](https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/mutate-string-processors#mutate-string-processors) processor. The following table describes the option you can use to configure the `trim_string` processor.
|
||||||
|
|
||||||
Option | Required | Type | Description
|
Option | Required | Type | Description
|
||||||
:--- | :--- | :--- | :---
|
:--- | :--- | :--- | :---
|
||||||
with_keys | Yes | List | A list of keys to trim the whitespace from.
|
with_keys | Yes | List | A list of keys to trim the white space from.
|
||||||
|
|
||||||
<!---## Configuration
|
<!---## Configuration
|
||||||
|
|
||||||
|
|
|
@ -53,5 +53,5 @@ Parameter | Description
|
||||||
`norms` | A Boolean value that specifies whether the field length should be used when calculating relevance scores. Default is `false`.
|
`norms` | A Boolean value that specifies whether the field length should be used when calculating relevance scores. Default is `false`.
|
||||||
[`null_value`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/index#null-value) | A value to be used in place of `null`. Must be of the same type as the field. If this parameter is not specified, the field is treated as missing when its value is `null`. Default is `null`.
|
[`null_value`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/index#null-value) | A value to be used in place of `null`. Must be of the same type as the field. If this parameter is not specified, the field is treated as missing when its value is `null`. Default is `null`.
|
||||||
`similarity` | The ranking algorithm for calculating relevance scores. Default is `BM25`.
|
`similarity` | The ranking algorithm for calculating relevance scores. Default is `BM25`.
|
||||||
`split_queries_on_whitespace` | A Boolean value that specifies whether full-text queries should be split on whitespace. Default is `false`.
|
`split_queries_on_whitespace` | A Boolean value that specifies whether full-text queries should be split on white space. Default is `false`.
|
||||||
`store` | A Boolean value that specifies whether the field value should be stored and can be retrieved separately from the _source field. Default is `false`.
|
`store` | A Boolean value that specifies whether the field value should be stored and can be retrieved separately from the _source field. Default is `false`.
|
|
@ -199,7 +199,7 @@ POST /_ingest/pipeline/_simulate
|
||||||
{
|
{
|
||||||
"pipeline" :
|
"pipeline" :
|
||||||
{
|
{
|
||||||
"description": "Splits text on whitespace characters",
|
"description": "Splits text on white space characters",
|
||||||
"processors": [
|
"processors": [
|
||||||
{
|
{
|
||||||
"csv" : {
|
"csv" : {
|
||||||
|
|
|
@ -17,7 +17,7 @@ OpenSearch supports two types of identifiers: regular identifiers and delimited
|
||||||
|
|
||||||
A regular identifier is a string of characters that starts with an ASCII letter (lower or upper case).
|
A regular identifier is a string of characters that starts with an ASCII letter (lower or upper case).
|
||||||
The next character can either be a letter, digit, or underscore (_). It can't be a reserved keyword.
|
The next character can either be a letter, digit, or underscore (_). It can't be a reserved keyword.
|
||||||
Whitespace and other special characters are also not allowed.
|
White space and other special characters are also not allowed.
|
||||||
|
|
||||||
OpenSearch supports the following regular identifiers:
|
OpenSearch supports the following regular identifiers:
|
||||||
|
|
||||||
|
|
|
@ -69,7 +69,7 @@ multi_match('query'=query_expression[, 'fields'=field_expression][, option=<opti
|
||||||
multimatchquery('query'=query_expression[, 'fields'=field_expression][, option=<option_value>]*)
|
multimatchquery('query'=query_expression[, 'fields'=field_expression][, option=<option_value>]*)
|
||||||
```
|
```
|
||||||
|
|
||||||
The `fields` parameter is optional and can contain a single field or a comma-separated list (whitespace characters are not allowed). The weight for each field is optional and is specified after the field name. It should be delimited by the `caret` character -- `^` -- without whitespace.
|
The `fields` parameter is optional and can contain a single field or a comma-separated list (white space characters are not allowed). The weight for each field is optional and is specified after the field name. It should be delimited by the `caret` character -- `^` -- without white space.
|
||||||
|
|
||||||
### Example
|
### Example
|
||||||
|
|
||||||
|
@ -99,7 +99,7 @@ The `QUERY` function is a synonym for [`QUERY_STRING`]({{site.url}}{{site.baseur
|
||||||
query('query'=query_expression[, 'fields'=field_expression][, option=<option_value>]*)
|
query('query'=query_expression[, 'fields'=field_expression][, option=<option_value>]*)
|
||||||
```
|
```
|
||||||
|
|
||||||
The `fields` parameter is optional and can contain a single field or a comma-separated list (whitespace characters are not allowed). The weight for each field is optional and is specified after the field name. It should be delimited by the `caret` character -- `^` -- without whitespace.
|
The `fields` parameter is optional and can contain a single field or a comma-separated list (white space characters are not allowed). The weight for each field is optional and is specified after the field name. It should be delimited by the `caret` character -- `^` -- without white space.
|
||||||
|
|
||||||
### Example
|
### Example
|
||||||
|
|
||||||
|
|
|
@ -78,7 +78,7 @@ plugins.security.authcz.admin_dn:
|
||||||
- CN=kirk,OU=client,O=client,L=test,C=DE
|
- CN=kirk,OU=client,O=client,L=test,C=DE
|
||||||
```
|
```
|
||||||
|
|
||||||
You can't use node certificates as admin certificates. The two must be separate. Also, do not add whitespace between the parts of the DN.
|
You can't use node certificates as admin certificates. The two must be separate. Also, do not add white space between the parts of the DN.
|
||||||
{: .warning }
|
{: .warning }
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -41,7 +41,7 @@ openssl x509 -in node1.pem -text -noout
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
||||||
### Check for special characters and whitespace in DNs
|
### Check for special characters and white space in DNs
|
||||||
|
|
||||||
The Security plugin uses the [string representation of Distinguished Names (RFC1779)](https://www.ietf.org/rfc/rfc1779.txt) when validating node certificates.
|
The Security plugin uses the [string representation of Distinguished Names (RFC1779)](https://www.ietf.org/rfc/rfc1779.txt) when validating node certificates.
|
||||||
|
|
||||||
|
@ -52,7 +52,7 @@ plugins.security.nodes_dn:
|
||||||
- 'CN=node-0.example.com,OU=SSL,O=My\, Test,L=Test,C=DE'
|
- 'CN=node-0.example.com,OU=SSL,O=My\, Test,L=Test,C=DE'
|
||||||
```
|
```
|
||||||
|
|
||||||
You can have whitespace within a field, but not between fields.
|
You can have white space within a field, but not between fields.
|
||||||
|
|
||||||
#### Bad configuration
|
#### Bad configuration
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue