[[data-streams-overview]] == Data streams overview ++++ Overview ++++ A data stream consists of one or more _backing indices_. Backing indices are <>, auto-generated indices used to store a stream's documents. image::images/data-streams/data-streams-diagram.svg[align="center"] The creation of a data stream requires a matching <>. This template acts as a blueprint for the stream's backing indices. It contains: * A name or wildcard (`*`) pattern for the data stream. * The data stream's _timestamp field_. This field must be mapped as a <> or <> field data type and must be included in every document indexed to the data stream. * The mappings and settings applied to each backing index when it's created. The same index template can be used to create multiple data streams. See <>. [discrete] [[data-streams-generation]] === Generation Each data stream tracks its _generation_: a six-digit, zero-padded integer that acts as a cumulative count of the data stream's backing indices. This count includes any deleted indices for the stream. The generation is incremented whenever a new backing index is added to the stream. When a backing index is created, the index is named using the following convention: [source,text] ---- .ds-- ---- .*Example* [%collapsible] ==== The `web_server_logs` data stream has a generation of `34`. The most recently created backing index for this data stream is named `.ds-web_server_logs-000034`. ==== Because the generation increments with each new backing index, backing indices with a higher generation contain more recent data. Backing indices with a lower generation contain older data. A backing index's name can change after its creation due to a <>, <>, or other operations. [discrete] [[data-stream-write-index]] === Write index When a read request is sent to a data stream, it routes the request to all its backing indices. For example, a search request sent to a data stream would query all its backing indices. image::images/data-streams/data-streams-search-request.svg[align="center"] However, the most recently created backing index is the data stream’s only _write index_. The data stream routes all indexing requests for new documents to this index. image::images/data-streams/data-streams-index-request.svg[align="center"] You cannot add new documents to a stream's other backing indices, even by sending requests directly to the index. This means you cannot submit the following requests directly to any backing index except the write index: * An <> request with an <> of `create`. The `op_type` parameter defaults to `create` when adding new documents. * A <> request using a `create` action Because it's the only index capable of ingesting new documents, you cannot perform operations on a write index that might hinder indexing. These prohibited operations include: * <> * <> * <> * <> * <> * <> [discrete] [[data-streams-rollover]] === Rollover When a data stream is created, one backing index is automatically created. Because this single index is also the most recently created backing index, it acts as the stream's write index. A <> creates a new backing index for a data stream. This new backing index becomes the stream's write index, replacing the current one, and increments the stream's generation. In most cases, we recommend using <> to automate rollovers for data streams. This lets you automatically roll over the current write index when it meets specified criteria, such as a maximum age or size. However, you can also use the <> to manually perform a rollover. See <>. [discrete] [[data-streams-append-only]] === Append-only For most time-series use cases, existing data is rarely, if ever, updated. Because of this, data streams are designed to be append-only. You can send <> directly to a data stream. However, you cannot send the following requests for existing documents directly to a data stream: * An <> request with an <> of `index`. The `op_type` parameter defaults to `index` for existing documents. * A <> request using the `delete`, `index`, or `update` action. * A <> request Instead, you can use the <> and <> APIs to update or delete existing documents in a data stream. See <>. Alternatively, you can update or delete a document by submitting requests to the backing index containing the document. See <>. TIP: If you frequently update or delete existing documents, we recommend using an <> and <> instead of a data stream. You can still use <> to manage indices for the alias.