[role="xpack"] [[data-streams]] = Data streams ++++ Data streams ++++ A data stream lets you store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams are well-suited for logs, events, metrics, and other continuously generated data. You can submit indexing and search requests directly to a data stream. The stream automatically routes the request to backing indices that store the stream's data. You can use <> to automate the management of these backing indices. For example, you can use {ilm-init} to automatically move older backing indices to less expensive hardware and delete unneeded indices. {ilm-init} can help you reduce costs and overhead as your data grows. [discrete] [[backing-indices]] == Backing indices A data stream consists of one or more <>, auto-generated backing indices. image::images/data-streams/data-streams-diagram.svg[align="center"] Each data stream requires a matching <>. The template contains the mappings and settings used to configure the stream's backing indices. Every document indexed to a data stream must contain a `@timestamp` field, mapped as a <> or <> field type. If the index template doesn't specify a mapping for the `@timestamp` field, {es} maps `@timestamp` as a `date` field with default options. The same index template can be used for multiple data streams. You cannot delete an index template in use by a data stream. [discrete] [[data-stream-read-requests]] == Read requests When you submit a read request to a data stream, the stream routes the request to all its backing indices. image::images/data-streams/data-streams-search-request.svg[align="center"] [discrete] [[data-stream-write-index]] == Write index The most recently created backing index is the data stream’s write index. The stream adds new documents to this index only. image::images/data-streams/data-streams-index-request.svg[align="center"] You cannot add new documents to other backing indices, even by sending requests directly to the index. You also cannot perform operations on a write index that may hinder indexing, such as: * <> * <> * <> * <> * <> * <> [discrete] [[data-streams-rollover]] == Rollover When you create a data stream, {es} automatically creates a backing index for the stream. This index also acts as the stream's first write index. A <> creates a new backing index that becomes the stream's new write index. We recommend using <> to automatically roll over data streams when the write index reaches a specified age or size. If needed, you can also <> a data stream. [discrete] [[data-streams-generation]] == Generation Each data stream tracks its generation: a six-digit, zero-padded integer that acts as a cumulative count of the stream's rollovers, starting at `000001`. When a backing index is created, the index is named using the following convention: [source,text] ---- .ds-- ---- Backing indices with a higher generation contain more recent data. For example, the `web-server-logs` data stream has a generation of `34`. The stream's most recent backing index is named `.ds-web-server-logs-000034`. Some operations, such as a <> or <>, can change a backing index's name. These name changes do not remove a backing index from its data stream. [discrete] [[data-streams-append-only]] == Append-only Data streams are designed for use cases where existing data is rarely, if ever, updated. You cannot send update or deletion requests for existing documents directly to a data stream. Instead, use the <> and <> APIs. If needed, you can <> by submitting requests directly to the document's backing index. TIP: If you frequently update or delete existing documents, use an <> and <> instead of a data stream. You can still use <> to manage indices for the alias. include::set-up-a-data-stream.asciidoc[] include::use-a-data-stream.asciidoc[] include::change-mappings-and-settings.asciidoc[]