From 959f60205a6191fc76195dbbf95922a6e4d53011 Mon Sep 17 00:00:00 2001 From: Igal Levy Date: Tue, 4 Mar 2014 11:21:30 -0800 Subject: [PATCH 1/4] updated info re deep storage --- docs/content/Cluster-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/Cluster-setup.md b/docs/content/Cluster-setup.md index e4ba0e564f1..f8d18995dab 100644 --- a/docs/content/Cluster-setup.md +++ b/docs/content/Cluster-setup.md @@ -70,7 +70,7 @@ The effective utilization of cores by Zookeeper, MySQL, and Master nodes is like Storage ------- -Indexed segments should be kept in a permanent store accessible by all nodes like AWS S3 or HDFS or equivalent. Currently Druid supports S3, but this will be extended soon. +Indexed segments should be kept in a permanent store accessible by all nodes. Druid supports AWS S3, HDFS, and "local" mountpoints of nonlocal storage (e.g. NFS). Members of the Druid community are adding additional deep-storage options, such as [Cassandra](http://planetcassandra.org/blog/post/cassandra-as-a-deep-storage-mechanism-for-druid-real-time-analytics-engine/). Local disk ("ephemeral" on AWS EC2) for caching is recommended over network mounted storage (example of mounted: AWS EBS, Elastic Block Store) in order to avoid network delays during times of heavy usage. If your data center is suitably provisioned for networked storage, perhaps with separate LAN/NICs just for storage, then mounted might work fine. From f9f9b30709c80f0a25547749f442143d437c5520 Mon Sep 17 00:00:00 2001 From: Igal Levy Date: Tue, 4 Mar 2014 11:40:35 -0800 Subject: [PATCH 2/4] fixed minor typo --- docs/content/Cluster-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/Cluster-setup.md b/docs/content/Cluster-setup.md index e3d13d03563..79ce2c37ea9 100644 --- a/docs/content/Cluster-setup.md +++ b/docs/content/Cluster-setup.md @@ -70,7 +70,7 @@ The effective utilization of cores by Zookeeper, MySQL, and Coordinator nodes is Storage ------- -Indexed segments should be kept in a permanent store accessible by all nodes like AWS S3 or HDFS or equivalent. Refer [Deep-Storage](deep-storage.html) for more details on supported storage types. +Indexed segments should be kept in a permanent store accessible by all nodes like AWS S3 or HDFS or equivalent. Refer to [Deep-Storage](deep-storage.html) for more details on supported storage types. Local disk ("ephemeral" on AWS EC2) for caching is recommended over network mounted storage (example of mounted: AWS EBS, Elastic Block Store) in order to avoid network delays during times of heavy usage. If your data center is suitably provisioned for networked storage, perhaps with separate LAN/NICs just for storage, then mounted might work fine. From a73b4afc4b1c225f8d2598bf34744ecc306354a2 Mon Sep 17 00:00:00 2001 From: Igal Levy Date: Tue, 4 Mar 2014 11:41:05 -0800 Subject: [PATCH 3/4] added note+link about Cassandra option --- docs/content/Deep-Storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/Deep-Storage.md b/docs/content/Deep-Storage.md index bf01571c6e9..4aca22c72d3 100644 --- a/docs/content/Deep-Storage.md +++ b/docs/content/Deep-Storage.md @@ -4,7 +4,7 @@ layout: doc_page # Deep Storage Deep storage is where segments are stored. It is a storage mechanism that Druid does not provide. This deep storage infrastructure defines the level of durability of your data, as long as Druid nodes can see this storage infrastructure and get at the segments stored on it, you will not lose data no matter how many Druid nodes you lose. If segments disappear from this storage layer, then you will lose whatever data those segments represented. -The currently supported types of deep storage follow. +The currently supported types of deep storage follow. Other deep-storage options, such as [Cassandra](http://planetcassandra.org/blog/post/cassandra -as-a-deep-storage-mechanism-for-druid-real-time-analytics-engine/), have been developed by members of the community. ## S3-compatible From 916bd30db3b7517df0a8fdb4ec9b7a6de958817a Mon Sep 17 00:00:00 2001 From: Igal Levy Date: Tue, 4 Mar 2014 11:44:06 -0800 Subject: [PATCH 4/4] fixed link to Cassandra article --- docs/content/Deep-Storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/content/Deep-Storage.md b/docs/content/Deep-Storage.md index 4aca22c72d3..b4c28098a6f 100644 --- a/docs/content/Deep-Storage.md +++ b/docs/content/Deep-Storage.md @@ -4,7 +4,7 @@ layout: doc_page # Deep Storage Deep storage is where segments are stored. It is a storage mechanism that Druid does not provide. This deep storage infrastructure defines the level of durability of your data, as long as Druid nodes can see this storage infrastructure and get at the segments stored on it, you will not lose data no matter how many Druid nodes you lose. If segments disappear from this storage layer, then you will lose whatever data those segments represented. -The currently supported types of deep storage follow. Other deep-storage options, such as [Cassandra](http://planetcassandra.org/blog/post/cassandra -as-a-deep-storage-mechanism-for-druid-real-time-analytics-engine/), have been developed by members of the community. +The currently supported types of deep storage follow. Other deep-storage options, such as [Cassandra](http://planetcassandra.org/blog/post/cassandra-as-a-deep-storage-mechanism-for-druid-real-time-analytics-engine/), have been developed by members of the community. ## S3-compatible