honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/docs/reference/mapping/fields/timestamp-field.asciidoc

132 lines
3.6 KiB
Plaintext
Raw Normal View History

Migrated documentation into the main repo
2013-08-28 19:24:34 -04:00
[[mapping-timestamp-field]]
=== `_timestamp`
The `_timestamp` field allows to automatically index the timestamp of a
document. It can be provided externally via the index request or in the
`_source`. If it is not provided externally it will be automatically set
Generate timestamp when path is null Index process fails when having `_timestamp` enabled and `path` option is set. It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message. Reproduction: ``` DELETE test PUT test { "mappings": { "test": { "_timestamp" : { "enabled" : "yes", "path" : "post_date" } } } } PUT test/test/1 { "foo": "bar" } ``` You can define a default value for when timestamp is not provided within the index request or in the `_source` document. By default, the default value is `now` which means the date the document was processed by the indexing chain. You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` If you don't provide any timestamp value, indexation will fail. You can also set the default value to any date respecting timestamp format: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "format" : "YYYY-MM-dd", "default" : "1970-01-01" } } } ``` If you don't provide any timestamp value, indexation will fail. Closes #4718. Closes #7036.
2014-07-25 09:14:31 -04:00
to a <<mapping-timestamp-field-default,default date>>.
Migrated documentation into the main repo
2013-08-28 19:24:34 -04:00
[float]
==== enabled
[DOCS] Multiple doc fixes Closes #5047
2014-03-07 08:21:45 -05:00
By default it is disabled. In order to enable it, the following mapping
Migrated documentation into the main repo
2013-08-28 19:24:34 -04:00
should be defined:
[source,js]
--------------------------------------------------
{
"tweet" : {
"_timestamp" : { "enabled" : true }
}
}
--------------------------------------------------
[float]
==== store / index
Mappings: Store _timestamp by default. Storing `_timestamp` by default means that under the default configuration, you would have all the information you need in order to reindex into a different index. Close #8139
2014-10-17 09:34:46 -04:00
By default the `_timestamp` field has `store` set to `true` and `index`
Migrated documentation into the main repo
2013-08-28 19:24:34 -04:00
set to `not_analyzed`. It can be queried as a standard date field.
[float]
==== path
The `_timestamp` value can be provided as an external value when
indexing. But, it can also be automatically extracted from the document
to index based on a `path`. For example, having the following mapping:
[source,js]
--------------------------------------------------
{
"tweet" : {
"_timestamp" : {
"enabled" : true,
"path" : "post_date"
}
}
}
--------------------------------------------------
Will cause `2009-11-15T14:12:12` to be used as the timestamp value for:
[source,js]
--------------------------------------------------
{
"message" : "You know, for Search",
"post_date" : "2009-11-15T14:12:12"
}
--------------------------------------------------
Docs: Fix typo in timestamp-field.asciidoc Closes #6661
2014-07-01 07:37:38 -04:00
Note, using `path` without explicit timestamp value provided requires an
Migrated documentation into the main repo
2013-08-28 19:24:34 -04:00
additional (though quite fast) parsing phase.
[float]
Generate timestamp when path is null Index process fails when having `_timestamp` enabled and `path` option is set. It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message. Reproduction: ``` DELETE test PUT test { "mappings": { "test": { "_timestamp" : { "enabled" : "yes", "path" : "post_date" } } } } PUT test/test/1 { "foo": "bar" } ``` You can define a default value for when timestamp is not provided within the index request or in the `_source` document. By default, the default value is `now` which means the date the document was processed by the indexing chain. You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` If you don't provide any timestamp value, indexation will fail. You can also set the default value to any date respecting timestamp format: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "format" : "YYYY-MM-dd", "default" : "1970-01-01" } } } ``` If you don't provide any timestamp value, indexation will fail. Closes #4718. Closes #7036.
2014-07-25 09:14:31 -04:00
[[mapping-timestamp-field-format]]
Migrated documentation into the main repo
2013-08-28 19:24:34 -04:00
==== format
You can define the <<mapping-date-format,date
format>> used to parse the provided timestamp value. For example:
[source,js]
--------------------------------------------------
{
"tweet" : {
"_timestamp" : {
"enabled" : true,
"path" : "post_date",
"format" : "YYYY-MM-dd"
}
}
}
--------------------------------------------------
Note, the default format is `dateOptionalTime`. The timestamp value will
first be parsed as a number and if it fails the format will be tried.
Generate timestamp when path is null Index process fails when having `_timestamp` enabled and `path` option is set. It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message. Reproduction: ``` DELETE test PUT test { "mappings": { "test": { "_timestamp" : { "enabled" : "yes", "path" : "post_date" } } } } PUT test/test/1 { "foo": "bar" } ``` You can define a default value for when timestamp is not provided within the index request or in the `_source` document. By default, the default value is `now` which means the date the document was processed by the indexing chain. You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` If you don't provide any timestamp value, indexation will fail. You can also set the default value to any date respecting timestamp format: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "format" : "YYYY-MM-dd", "default" : "1970-01-01" } } } ``` If you don't provide any timestamp value, indexation will fail. Closes #4718. Closes #7036.
2014-07-25 09:14:31 -04:00
[float]
[[mapping-timestamp-field-default]]
==== default
You can define a default value for when timestamp is not provided
within the index request or in the `_source` document.
By default, the default value is `now` which means the date the document was processed by the indexing chain.
[Mapper] Add `ignore_missing` option to `timestamp` Related to #9049. By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain. You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`): ```js { "tweet" : { "_timestamp" : { "enabled" : true, "ignore_missing" : false } } } ``` When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax. Let say you have defined this in elasticsearch 1.4.x: ```js DELETE test PUT test { "settings": { "number_of_shards": 1, "number_of_replicas": 0 } } PUT test/type/_mapping { "type" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` After migration, the mapping become: ```js { "test": { "mappings": { "type": { "_timestamp": { "enabled": true, "store": false, "ignore_missing": false }, "properties": {} } } } } ``` Closes #8882.
2014-12-30 11:32:50 -05:00
You can reject documents which do not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`):
Generate timestamp when path is null Index process fails when having `_timestamp` enabled and `path` option is set. It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message. Reproduction: ``` DELETE test PUT test { "mappings": { "test": { "_timestamp" : { "enabled" : "yes", "path" : "post_date" } } } } PUT test/test/1 { "foo": "bar" } ``` You can define a default value for when timestamp is not provided within the index request or in the `_source` document. By default, the default value is `now` which means the date the document was processed by the indexing chain. You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` If you don't provide any timestamp value, indexation will fail. You can also set the default value to any date respecting timestamp format: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "format" : "YYYY-MM-dd", "default" : "1970-01-01" } } } ``` If you don't provide any timestamp value, indexation will fail. Closes #4718. Closes #7036.
2014-07-25 09:14:31 -04:00
[source,js]
--------------------------------------------------
{
"tweet" : {
"_timestamp" : {
"enabled" : true,
[Mapper] Add `ignore_missing` option to `timestamp` Related to #9049. By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain. You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`): ```js { "tweet" : { "_timestamp" : { "enabled" : true, "ignore_missing" : false } } } ``` When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax. Let say you have defined this in elasticsearch 1.4.x: ```js DELETE test PUT test { "settings": { "number_of_shards": 1, "number_of_replicas": 0 } } PUT test/type/_mapping { "type" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` After migration, the mapping become: ```js { "test": { "mappings": { "type": { "_timestamp": { "enabled": true, "store": false, "ignore_missing": false }, "properties": {} } } } } ``` Closes #8882.
2014-12-30 11:32:50 -05:00
"ignore_missing" : false
Generate timestamp when path is null Index process fails when having `_timestamp` enabled and `path` option is set. It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message. Reproduction: ``` DELETE test PUT test { "mappings": { "test": { "_timestamp" : { "enabled" : "yes", "path" : "post_date" } } } } PUT test/test/1 { "foo": "bar" } ``` You can define a default value for when timestamp is not provided within the index request or in the `_source` document. By default, the default value is `now` which means the date the document was processed by the indexing chain. You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` If you don't provide any timestamp value, indexation will fail. You can also set the default value to any date respecting timestamp format: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "format" : "YYYY-MM-dd", "default" : "1970-01-01" } } } ``` If you don't provide any timestamp value, indexation will fail. Closes #4718. Closes #7036.
2014-07-25 09:14:31 -04:00
}
}
}
--------------------------------------------------
You can also set the default value to any date respecting <<mapping-timestamp-field-format,timestamp format>>:
[source,js]
--------------------------------------------------
{
"tweet" : {
"_timestamp" : {
"enabled" : true,
"format" : "YYYY-MM-dd",
"default" : "1970-01-01"
}
}
}
--------------------------------------------------
Fix error in documentation Indexation does not fail if no timestamp provided when there is a default value defined in mapping.
2014-11-21 20:23:38 -05:00
If you don't provide any timestamp value, _timestamp will be set to this default value.
Generate timestamp when path is null Index process fails when having `_timestamp` enabled and `path` option is set. It fails with a `TimestampParsingException[failed to parse timestamp [null]]` message. Reproduction: ``` DELETE test PUT test { "mappings": { "test": { "_timestamp" : { "enabled" : "yes", "path" : "post_date" } } } } PUT test/test/1 { "foo": "bar" } ``` You can define a default value for when timestamp is not provided within the index request or in the `_source` document. By default, the default value is `now` which means the date the document was processed by the indexing chain. You can disable that default value by setting `default` to `null`. It means that `timestamp` is mandatory: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` If you don't provide any timestamp value, indexation will fail. You can also set the default value to any date respecting timestamp format: ``` { "tweet" : { "_timestamp" : { "enabled" : true, "format" : "YYYY-MM-dd", "default" : "1970-01-01" } } } ``` If you don't provide any timestamp value, indexation will fail. Closes #4718. Closes #7036.
2014-07-25 09:14:31 -04:00
[Mapper] Add `ignore_missing` option to `timestamp` Related to #9049. By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain. You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`): ```js { "tweet" : { "_timestamp" : { "enabled" : true, "ignore_missing" : false } } } ``` When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax. Let say you have defined this in elasticsearch 1.4.x: ```js DELETE test PUT test { "settings": { "number_of_shards": 1, "number_of_replicas": 0 } } PUT test/type/_mapping { "type" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` After migration, the mapping become: ```js { "test": { "mappings": { "type": { "_timestamp": { "enabled": true, "store": false, "ignore_missing": false }, "properties": {} } } } } ``` Closes #8882.
2014-12-30 11:32:50 -05:00
coming[1.5.0]
In elasticsearch 1.4, we allowed setting explicitly `"default":null` which is not possible anymore
[Docs] fix missing space
2015-01-29 13:17:41 -05:00
as we added a new `ignore_missing` setting.
[Mapper] Add `ignore_missing` option to `timestamp` Related to #9049. By default, the default value for `timestamp` is `now` which means the date the document was processed by the indexing chain. You can now reject documents which not provide a `timestamp` value by setting `ignore_missing` to false (default to `true`): ```js { "tweet" : { "_timestamp" : { "enabled" : true, "ignore_missing" : false } } } ``` When you update the cluster to 1.5 or master, this index created with 1.4 we automatically migrate an index created with 1.4 to the 1.5 syntax. Let say you have defined this in elasticsearch 1.4.x: ```js DELETE test PUT test { "settings": { "number_of_shards": 1, "number_of_replicas": 0 } } PUT test/type/_mapping { "type" : { "_timestamp" : { "enabled" : true, "default" : null } } } ``` After migration, the mapping become: ```js { "test": { "mappings": { "type": { "_timestamp": { "enabled": true, "store": false, "ignore_missing": false }, "properties": {} } } } } ``` Closes #8882.
2014-12-30 11:32:50 -05:00
When reading an index created with elasticsearch 1.4 and using this, we automatically update it by
removing `"default": null` and setting `"ignore_missing": false`