doc for seal api and doc for syned flush in general

docs/reference/indices/seal.asciidoc

@@ -0,0 +1,74 @@
+[[indices-seal]]
+== Seal
+
+The seal API allows to flush one or more indices and adds a marker to primaries and replicas if there are no pending write operations.
+The seal marker is used during recovery after a node restarts. If a replica is allocated on a node which already has a shard copy with the same seal as the primary then no files will be copied during recovery.
+Sealing is a best effort operation. If write operations are ongoing while the sealing is in progress then writing the seal might fail on some copies.
+
+[source,js]
+--------------------------------------------------
+$ curl -XPOST 'http://localhost:9200/twitter/_seal'
+--------------------------------------------------
+
+The response contains details about for which shards a seal was written and the reason in case of failure.
+Response in case all copies of a shard successfully wrote the seal:
+
+[source,js]
+--------------------------------------------------
+{
+   "twitter": [
+      {
+         "shard_id": 0,
+         "responses": {
+            "5wjOIntuRqy9F_7JRrrLwA": "success",
+            "M2iCBe-nS5yaInE8volfSg": "success"
+         },
+         "message": "success"
+      }
+}
+--------------------------------------------------
+
+
+Response in case some copies of a shard failed:
+[source,js]
+--------------------------------------------------
+{
+   "twitter": [
+      {
+         "shard_id": 0,
+         "responses": {
+            "M2iCBe-nS5yaInE8volfSg": "pending operations",
+            "5wjOIntuRqy9F_7JRrrLwA": "success"
+         },
+         "message": "failed on some copies"
+      }
+}
+--------------------------------------------------
+
+
+Response in case all copies of a shard failed:
+[source,js]
+--------------------------------------------------
+{
+   "twitter": [
+      {
+         "shard_id": 0,
+         "message": "operation counter on primary is non zero [2]"
+      }
+}
+--------------------------------------------------
+
+
+[float]
+[[seal-multi-index]]
+=== Multi Index
+
+The seal API can be applied to more than one index with a single call,
+or even on `_all` the indices.
+
+[source,js]
+--------------------------------------------------
+$ curl -XPOST 'http://localhost:9200/kimchy,elasticsearch/_seal'
+
+$ curl -XPOST 'http://localhost:9200/_seal'
+--------------------------------------------------

src/main/java/org/elasticsearch/indices/SyncedFlushService.java

@@ -106,6 +106,31 @@ public class SyncedFlushService extends AbstractComponent {
         });
     }
 
+    /*
+    * Tries to flush all copies of a shard and write a sync id to it.
+    * After a synced flush two shard copies may only contain the same sync id if they contain the same documents.
+    * To ensure this, synced flush works in three steps:
+    * 1. Flush all shard copies and gather the commit points for each copy after the flush
+    * 2. Ensure that there are no ongoing indexing operations on the primary
+    * 3. Perform an additional flush on each shard copy that writes the sync id
+    *
+    * Step 3 is only executed on a shard if
+    * a) the shard has no uncommitted changes since the last flush
+    * b) the last flush was the one executed in 1 (use the collected commit id to verify this)
+    *
+    * This alone is not enough to ensure that all copies contain the same documents. Without step 2 a sync id would be written for inconsistent copies in the following scenario:
+    *
+    * Write operation has completed on a primary and is being sent to replicas. The write request does not reach the replicas until sync flush is finished.
+    * Step 1 is executed. After the flush the commit points on primary contains a write operation that the replica does not have.
+    * Step 3 will be executed on primary and replica as well because there are no uncommitted changes on primary (the first flush committed them) and there are no uncommitted
+    * changes on the replica (the write operation has not reached the replica yet).
+    *
+    * Step 2 detects this scenario and fails the whole synced flush if a write operation is ongoing on the primary.
+    *
+    * Synced flush is a best effort operation. The sync id may be written on all, some or none of the copies. 
+    *
+    * **/
+
     public void attemptSyncedFlush(ShardId shardId, ActionListener<SyncedFlushResult> actionListener) {
         try {
             final ClusterState state = clusterService.state();