From 8d1aa2beb3f3d3b949f3e0be95f9b5102246da5f Mon Sep 17 00:00:00 2001 From: James Estes Date: Fri, 13 Feb 2015 12:53:28 -0700 Subject: [PATCH] Add common config dir to classpath in examples Also adds query examples in the Cluster tutorial --- docs/content/Tutorial:-All-About-Queries.md | 6 ++-- docs/content/Tutorial:-Loading-Batch-Data.md | 6 ++-- .../Tutorial:-Loading-Streaming-Data.md | 2 +- docs/content/Tutorial:-The-Druid-Cluster.md | 34 ++++++++++++++++--- 4 files changed, 36 insertions(+), 12 deletions(-) diff --git a/docs/content/Tutorial:-All-About-Queries.md b/docs/content/Tutorial:-All-About-Queries.md index d96d9fd7961..b4294684d61 100644 --- a/docs/content/Tutorial:-All-About-Queries.md +++ b/docs/content/Tutorial:-All-About-Queries.md @@ -19,19 +19,19 @@ Note: If Zookeeper and metadata storage aren't running, you'll have to start the To start a Coordinator node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/coordinator io.druid.cli.Main server coordinator +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/coordinator io.druid.cli.Main server coordinator ``` To start a Historical node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/historical io.druid.cli.Main server historical +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/historical io.druid.cli.Main server historical ``` To start a Broker node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/broker io.druid.cli.Main server broker +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/broker io.druid.cli.Main server broker ``` Querying Your Data diff --git a/docs/content/Tutorial:-Loading-Batch-Data.md b/docs/content/Tutorial:-Loading-Batch-Data.md index 8c1cc59edd8..01abb44da7c 100644 --- a/docs/content/Tutorial:-Loading-Batch-Data.md +++ b/docs/content/Tutorial:-Loading-Batch-Data.md @@ -69,19 +69,19 @@ Note: If Zookeeper and MySQL aren't running, you'll have to start them again as To start the Indexing Service: ```bash -java -Xmx2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*::config/overlord io.druid.cli.Main server overlord +java -Xmx2g -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*::config/_common:config/overlord io.druid.cli.Main server overlord ``` To start the Coordinator Node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/coordinator io.druid.cli.Main server coordinator +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/coordinator io.druid.cli.Main server coordinator ``` To start the Historical Node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/historical io.druid.cli.Main server historical +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/historical io.druid.cli.Main server historical ``` #### Index the Data diff --git a/docs/content/Tutorial:-Loading-Streaming-Data.md b/docs/content/Tutorial:-Loading-Streaming-Data.md index e1d3124aaf4..49b2392baf3 100644 --- a/docs/content/Tutorial:-Loading-Streaming-Data.md +++ b/docs/content/Tutorial:-Loading-Streaming-Data.md @@ -93,7 +93,7 @@ You should be comfortable starting Druid nodes at this point. If not, it may be 1. Real-time nodes can be started with: ```bash - java -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Ddruid.realtime.specFile=examples/indexing/wikipedia.spec -classpath lib/*:config/realtime io.druid.cli.Main server realtime + java -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Ddruid.realtime.specFile=examples/indexing/wikipedia.spec -classpath lib/*:config/_common:config/realtime io.druid.cli.Main server realtime ``` 2. A realtime.spec should already exist for the data source in the Druid tarball. You should be able to find it at: diff --git a/docs/content/Tutorial:-The-Druid-Cluster.md b/docs/content/Tutorial:-The-Druid-Cluster.md index 3f937e3dfbe..8457a099b62 100644 --- a/docs/content/Tutorial:-The-Druid-Cluster.md +++ b/docs/content/Tutorial:-The-Druid-Cluster.md @@ -169,7 +169,7 @@ druid.coordinator.startDelay=PT70s To start the coordinator node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/coordinator io.druid.cli.Main server coordinator +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/coordinator io.druid.cli.Main server coordinator ``` #### Start a Historical Node @@ -202,7 +202,7 @@ druid.server.maxSize=10000000000 To start the historical node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/historical io.druid.cli.Main server historical +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/historical io.druid.cli.Main server historical ``` #### Start a Broker Node @@ -234,7 +234,7 @@ druid.processing.numThreads=1 To start the broker node: ```bash -java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/broker io.druid.cli.Main server broker +java -Xmx256m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -classpath lib/*:config/_common:config/broker io.druid.cli.Main server broker ``` #### Start a Realtime Node @@ -280,7 +280,7 @@ Now we should be handing off segments every 6 minutes or so. To start the realtime node that was used in our first tutorial, you simply have to issue: ``` -java -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Ddruid.realtime.specFile=examples/wikipedia/wikipedia_realtime.spec -classpath lib/*:config/realtime io.druid.cli.Main server realtime +java -Xmx512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Ddruid.realtime.specFile=examples/wikipedia/wikipedia_realtime.spec -classpath lib/*:config/_common:config/realtime io.druid.cli.Main server realtime ``` The configurations are located in `config/realtime/runtime.properties` and should contain the following: @@ -301,7 +301,31 @@ druid.processing.numThreads=2 Once the real-time node starts up, it should begin ingesting data and handing that data off to the rest of the Druid cluster. You can use a web UI located at coordinator_ip:port to view the status of data being loaded. Once data is handed off from the real-time nodes to historical nodes, the historical nodes should begin serving segments. -At any point during ingestion, we can query for data. The queries should span across both real-time and historical nodes. For more information on querying, see this [link](Querying.html). +#### Query + +At any point during ingestion, we can query for data. For example: + +``` +curl -X POST 'http://localhost:8080/druid/v2/?pretty' -H 'content-type: application/json' -d@examples/wikipedia/query.body +``` + +This query will span across both realtime and historical nodes. If you're curious, you can query the historical node directly by sending the same query to the historical node's port: + +``` +curl -X POST 'http://localhost:8081/druid/v2/?pretty' -H 'content-type: application/json' -d@examples/wikipedia/query.body +``` + +This query may produce no results if the realtime node hasn't run long enough to hand off the segment (we configured it above to be 5 minutes). Query the realtime node directly by sending the same query to the realtime node's port: + +``` +curl -X POST 'http://localhost:8083/druid/v2/?pretty' -H 'content-type: application/json' -d@examples/wikipedia/query.body +``` + +The realtime query results will reflect the data that was recently indexed from wikipedia, and not handed off to the historical node yet. Once the historical node acknowledges it has loaded the segment, the realtime node will stop querying the segment. + +Querying the historical and realtime node directly is useful for understanding how the segment handling is working, but if you just want to run a query for all the data (realtime and historical), then send the query to the broker at port 8080 (which is what we did in the first example). The broker will send the query to the historical and realtime nodes and merge the results. + +For more information on querying, see this [link](Querying.html). Next Steps ----------