Commit Graph

547 Commits

Author SHA1 Message Date
fjy d4014c6a4e Merge pull request #675 from metamx/slimmer-column-cache
reduce object count overhead of column cache
2014-08-12 14:16:33 -06:00
Xavier Léauté 1c244c8be8 add column cache docs 2014-08-12 10:40:09 -07:00
fjy bd98e0dd29 prep next release 2014-08-09 19:33:31 -07:00
fjy 8998700760 prepare for next release 2014-08-08 16:19:45 -07:00
fjy 8e87cae990 prepare for next rel 2014-08-08 15:51:22 -07:00
fjy 35937132a7 prep next release 2014-08-08 14:35:22 -07:00
fjy bb44056810 prepare for next release 2014-08-08 11:38:17 -07:00
fjy 57a979ef19 prepare for next release 2014-08-06 13:31:27 -07:00
Gian Merlino 6e33cd4e0a Coordinator drop/remove related stuff.
- DruidCoordinatorCleanup should wait for availableSegments to populate
  before dropping any segments.
- Clarify that millisToWaitBeforeDeleting applies to "removing" rather than
  "dropping" segments.
- LoadRule shouldn't need to wait for the deletionWaitTime before dropping
  excess replicas.
2014-08-06 12:52:09 -07:00
Xavier Léauté 91fc5fc82d reformatting 2014-08-01 15:28:13 -07:00
Xavier Léauté e158aa30a7 add timeboundary docs 2014-08-01 10:44:45 -07:00
fjy 362be7a2c0 prepare for next release 2014-08-01 10:12:03 -07:00
fjy 046e8864b1 more docs 2014-08-01 10:10:23 -07:00
Xavier Léauté b44546ad8c query cancellation/timeout docs + missing items 2014-08-01 09:48:06 -07:00
fjy 90ebeaa8d3 fix broken approx histo doc 2014-08-01 09:42:52 -07:00
nishantmonu51 80a4f7c5b5 doc for IngestSegmentFirehose 2014-08-01 16:43:39 +05:30
fjy d202eae3b5 toc fix 2014-07-31 17:27:16 -07:00
fjy 611dcb9d1c new docs on logging
Conflicts:
	docs/content/toc.textile
2014-07-31 17:26:15 -07:00
fjy 17c600c691 some doc fixes 2014-07-31 17:25:30 -07:00
fjy e5ce9b4a2a prepare for next release 2014-07-31 14:05:56 -07:00
fjy 32bd7003ec minor doc fix 2014-07-31 12:58:19 -07:00
fjy 147f5754da add missing doc 2014-07-31 10:23:48 -07:00
fjy 625377fdd2 prep next rel 2014-07-29 22:00:11 -07:00
fjy 7803ca07e7 fix broken chat handler 2014-07-29 17:58:21 -07:00
fjy 89149b5fcb prepare for next release 2014-07-29 17:38:19 -07:00
fjy 0be5d27f2a another docs fix 2014-07-29 11:40:35 -07:00
fjy d9c9190e2f fix breakage in batch ingestion schema 2014-07-29 11:36:29 -07:00
Jasmine Hegman 1844bf4f08 Adding a section to detail supporting Kafka 8 2014-07-28 14:46:39 -07:00
Jasmine Hegman 61af84a4a1 Cleaning up code sample indentations 2014-07-28 14:44:59 -07:00
fjy 45e443bedb a few slight tweaks and bumping up curator version 2014-07-25 16:07:40 -07:00
fjy 7e0fa6fe06 Merge branch 'master' of github.com:metamx/druid 2014-07-25 10:50:47 -07:00
fjy bb5c5b3ce7 fix config with broker select and add docs 2014-07-25 10:50:41 -07:00
fjy 41770574f9 Merge pull request #646 from metamx/inverted-topn-metric-spec-docs
add docs
2014-07-25 10:30:38 -06:00
fjy d3b920769d a few more cleanups to docs 2014-07-25 09:26:44 -07:00
Xavier Léauté 9c36014645 remove css from docs 2014-07-24 14:41:45 -07:00
Gian Merlino baeb93772d Rename OrderBy to LimitSpec in docs. 2014-07-24 14:20:43 -07:00
nishantmonu51 27cda43d89 add docs
doc for inverted topN metric spec
2014-07-24 13:16:44 +05:30
fjy 387350abd8 fix toc entry 2014-07-23 13:41:14 -07:00
fjy ded83557dd Merge pull request #640 from metamx/disable-worker
An alternative way to disable middlemanagers based on worker version
2014-07-17 19:07:27 -06:00
fjy 83cb7931ac add more stuff to docs 2014-07-17 14:59:56 -07:00
fjy d765ff5e75 docs for updating 2014-07-17 14:53:26 -07:00
fjy bb5598df54 fix doc 2014-07-17 14:07:12 -07:00
fjy c6078ca841 address code review 2014-07-17 13:34:05 -07:00
fjy bdfeccd092 doc and timeout fix 2014-07-15 15:57:58 -07:00
fjy ce478be899 Remove redundant forking task runner config 2014-07-07 18:08:37 -06:00
fjy c646648319 fix examples start port 2014-07-02 10:07:04 -07:00
fjy dba7e972d8 refactor docs for histograms 2014-07-01 16:39:24 -07:00
Xavier Léauté 70ae6e1fcd Approximate Histogram Module 2014-07-01 15:10:23 -07:00
fjy 484b91db5a Merge pull request #617 from metamx/igalDruid
Igal druid
2014-06-30 19:58:38 -06:00
Gian Merlino 991ff34f38 More explicit example of jobProperties. 2014-06-25 20:24:51 -07:00
fjy ff00b545bc Merge pull request #623 from metamx/document-hadoop-jobProperties
Document how to set hadoop properties
2014-06-25 21:15:21 -06:00
Gian Merlino a7ef019b22 Document batch ingestion jobProperties. 2014-06-25 20:14:11 -07:00
Igal Levy 805e80c24d Merge branch 'master' into igalDruid 2014-06-20 17:16:41 -07:00
Igal Levy 262d43ae2e removed build section that didn't work (direct from whirr repo); added info on resource files needed to run test; tweaked whirr config section to match current requirements; changed cluster-start section to reflect fact that we're starting multiple instances; other minor edits 2014-06-20 17:03:21 -07:00
fjy 5afcb9cbd3 add docs 2014-06-19 16:55:02 -07:00
fjy 276e48e564 another doc fix 2014-06-19 13:33:16 -07:00
fjy fdb39d57a3 fix another doc problem 2014-06-19 13:10:08 -07:00
fjy 9aa018925d fix doc header 2014-06-19 12:59:58 -07:00
fjy f1759b450c fix Zk setup problem 2014-06-19 12:55:02 -07:00
Gian Merlino 3164228ecf Fix example middle manager configs. 2014-06-19 12:43:42 -07:00
Gian Merlino eab9381ba1 Docs for string-based EC2 userData. 2014-06-18 10:09:07 -07:00
Gian Merlino e9ef1e956c Docs for defaultHadoopCoordinates. 2014-06-18 09:50:37 -07:00
fjy 1110b34328 Merge pull request #586 from metamx/jisoo
modify whirr documentation
2014-06-06 16:14:48 -06:00
jisookim0513 11f07b5fbf revert changes to wikipedia_realtime_task.json, CliBroker.java, CliHistorical.java 2014-06-05 17:51:58 -07:00
jisookim0513 a0d51a8cb2 modify whirr documentation 2014-06-05 15:51:23 -07:00
fjy c8689507c8 prepare for next release 2014-06-05 10:01:25 -07:00
fjy e3941a2890 prepare for next rel 2014-06-04 20:19:50 -07:00
fjy 612624fb89 prepare for next release 2014-06-04 16:47:54 -07:00
fjy 8f657686c5 fix typos 2014-06-04 10:14:39 -07:00
fjy 22c62cc176 update performance FAQ 2014-06-04 10:09:17 -07:00
fjy 22bbe037a7 prepare for next release 2014-06-04 09:34:29 -07:00
fjy dd76fc49e2 fix doc mistake 2014-06-02 15:38:25 -07:00
fjy abe26a7f2e change default merge size to 500mb instead of 1g 2014-06-02 12:48:57 -07:00
fjy abac529116 prepare for next release 2014-06-02 10:49:54 -07:00
fjy 34d3eb3b5f fix minor link problem in docs 2014-06-02 10:21:00 -07:00
fjy 49f03b808d fix default value for max segments to merge 2014-06-02 10:20:13 -07:00
fjy 9adf80da7c prepare for next release 2014-05-29 17:24:02 -07:00
fjy 53638a74ec add more segment metadata docs 2014-05-29 12:53:13 -07:00
fjy fcc9bf4f26 document limitspec in groupby, fixes #394 2014-05-29 10:19:11 -07:00
fjy 7be93a770a make all firehoses work with tasks, add a lot more documentation about configuration 2014-05-28 16:33:59 -07:00
fjy 002fefb4cf Merge pull request #554 from jcollum/kafka-404-fix
changed the url for kafka wget (old one was 404)
2014-05-19 14:37:20 -06:00
Justin Collum d3d2534105 changed the url for kafka wget (old one was 404) 2014-05-19 10:55:22 -07:00
fjy b6fb4245aa Merge branch 'master' into new-schema
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
	pom.xml
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
2014-05-06 14:32:51 -07:00
fjy b1bd6eb0f2 Merge pull request #512 from metamx/cardinality
Cardinality aggregator
2014-05-05 17:10:24 -06:00
fjy 8af927e2a7 Merge pull request #516 from metamx/alphanumeric-topn
Alpha-numeric sorting for topN
2014-05-05 17:09:24 -06:00
fjy b8184821b5 update docs for next rel 2014-05-01 15:07:14 -07:00
fjy 268c9aa807 update docs version 2014-04-30 13:11:46 -07:00
fjy 3fd651c235 fix the rule again, update docs, and prepare for next release 2014-04-30 10:43:30 -07:00
fjy 0790303b4a update docs and prepare for next release 2014-04-30 10:23:35 -07:00
fjy ffd37bc7bd Merge pull request #521 from metamx/igalDruid
Igal druid
2014-04-29 18:06:30 -06:00
Igal Levy 7cd2f9a9bb fixed couple of typos 2014-04-29 10:00:28 -07:00
Igal Levy 77fb424034 fixed borked description of whirr.hardware-id 2014-04-29 09:56:00 -07:00
Xavier Léauté fc874f8bbb add docs 2014-04-28 09:51:58 -07:00
fjy 76e0a48527 Merge branch 'master' into new-schema
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
2014-04-25 14:03:28 -07:00
Xavier Léauté 42c4e955fa cosmetic changes 2014-04-25 13:26:11 -07:00
Xavier Léauté 14907645aa add cardinality aggregator docs 2014-04-25 13:23:43 -07:00
Xavier Léauté 55d316b1ff update docs version 2014-04-24 17:18:27 -07:00
fjy c4cefb667e prepare for next release 2014-04-24 11:45:25 -07:00
nishantmonu51 0595b001e5 add doc for numShards 2014-04-23 22:11:28 +05:30
fjy b653ea0530 prepare for next release 2014-04-22 17:07:11 -07:00
fjy c689043938 fix twitter tutorial doc 2014-04-22 15:31:14 -07:00
fjy 771f89f495 update docs 2014-04-22 15:27:19 -07:00
fjy c0ddfc0872 update versions 2014-04-21 22:32:53 -07:00
fjy 04009b4195 prepare for next release 2014-04-21 12:14:50 -07:00
Son fd0d745018 Fix typo in link back to The Druid Cluster page. 2014-04-19 14:23:40 +08:00
fjy 517d3d2682 prepare for next release 2014-04-18 17:42:47 -07:00
fjy c00fb1d08e Merge pull request #445 from metamx/hadoop-version-update
Hadoop version update
2014-04-17 19:16:44 -06:00
fjy ad60a4a5fd prepare for next release 2014-04-17 15:28:46 -07:00
fjy 88874ed56c Merge pull request #485 from metamx/igalDruid
Igal druid
2014-04-16 18:05:36 -06:00
fjy 795d506939 internet disconnected and last build failed 2014-04-16 11:32:32 -07:00
fjy 7f4e7d2b8f prepare for next release 2014-04-16 11:24:45 -07:00
Igal Levy f3125490ff Merge branch 'master' into igalDruid 2014-04-16 11:24:18 -07:00
Igal Levy 52e53db54e minor edit to Metrics Module section 2014-04-16 11:23:47 -07:00
Igal Levy 0629b1a044 Major edits including the following:
* Emitter module is logging
* readTimeout for http client module 15min
* Fixed broken link to ZooKeeper.html
* Clarified druid.processing.numThreads is #cores - 1 or 1 if #cores == 1
* aws storage section moved under deep storage
* Added more info on various monitors and how to specify them (Metrics Module section)
* Clarified info on:
    * storage tiers and added the property druid.server.priority (Storage Node Module section)
    * segment cache : where historical nodes store data in disk cache
    * druid.query.chunkPeriod
* Removed druid.db.connector.pollDuration
* Fixed definition of druid.selectors.indexing.serviceName and moved to discovery section
2014-04-16 10:57:52 -07:00
Igal Levy ee47937889 overhaul of config reference, first pass 2014-04-15 15:38:58 -07:00
Xavier Léauté ce250222bf update docs 2014-04-14 16:33:08 -07:00
fjy 7db756f819 prepare for next release 2014-04-08 16:25:59 -07:00
fjy a4df23f092 prepare for next release 2014-04-08 11:42:59 -07:00
fjy 4ef2bbbfc1 prepare for next release 2014-04-07 14:37:02 -07:00
fjy 2e9579a146 prepare for next release 2014-04-07 12:43:30 -07:00
Jon Chase 0036954261 docs: correcting path to index file 2014-04-07 09:14:21 -04:00
Igal Levy 4576c5077e switched operations and ingestion sections 2014-04-04 10:05:33 -07:00
Igal Levy 0b0b8a1d37 Merge branch 'master' into igalDruid 2014-04-04 10:04:22 -07:00
fjy d9062d91fa fix endpoint bugs and more docs 2014-04-03 17:01:33 -07:00
fjy b0bf08554a fix broken build 2014-04-02 22:22:42 -07:00
fjy e53d107472 revert broken cache 2014-04-02 22:12:22 -07:00
fjy ea9f31bbe0 learn to math and prepare for next release 2014-04-02 21:31:17 -07:00
fjy f185d215e1 fix jetty default thread pool size for jetty 9 and prepare for next release 2014-04-02 21:01:07 -07:00
fjy fc94b847c3 prepare for next release 2014-04-02 17:31:19 -07:00
Igal Levy ba6c289ccb minor typos fixed 2014-03-31 13:05:54 -07:00
Igal Levy adfcc3d16f Merge branch 'master' into igalDruid 2014-03-27 16:19:58 -07:00
Igal Levy 0f4e5cb125 added firehose and plumber sections, which were being referenced but were missing 2014-03-27 16:18:56 -07:00
Igal Levy 2245764b8f added link to info on rejection policy 2014-03-27 15:16:47 -07:00
Igal Levy effd30f991 added link to info on how tasks are posted 2014-03-27 15:12:50 -07:00
Igal Levy de771a6642 edits for readability 2014-03-27 15:09:42 -07:00
Igal Levy fa0f0a810c removed references to config elements that do not appear on the page 2014-03-27 15:00:28 -07:00
Igal Levy 525cbf5aab completed config info with rjection policy, segmentGranularity; some minor edits 2014-03-27 14:58:40 -07:00
nishantmonu51 298b9c9346 update default version in docs 2014-03-27 20:16:49 +05:30
fjy 183a133ee4 Merge pull request #442 from metamx/demo-paper
Writing a demonstration paper for a conference
2014-03-25 22:29:36 -06:00
fjy 15af9ac3c5 fix things 2014-03-25 21:26:48 -07:00
Igal Levy 24dbc58cec edited for flow and readability 2014-03-25 17:19:55 -07:00
Igal Levy 5ff77dad34 Merge branch 'master' into igalDruid 2014-03-25 16:56:24 -07:00
Igal Levy 7881f068c6 added task types that were missing from some of the JSON task configs 2014-03-25 16:53:06 -07:00
Igal Levy 8859aa675a fixed link text to realtime ingestion 2014-03-25 16:43:58 -07:00
Igal Levy d780b28a09 added link for more info on indexGranularity 2014-03-25 16:25:00 -07:00
Igal Levy 5eacd97239 fixed typo in task spec 2014-03-24 16:51:11 -07:00
Michael E. Driscoll 98cd83071a Update index.md
Fixed a few typos and grammatical issues.
2014-03-23 14:58:11 -07:00
fjy 55169175de prepare for next release 2014-03-21 11:05:54 -07:00
Igal Levy a64be537a5 updated task ID definition 2014-03-19 21:08:40 -07:00
Guillaume Simard 1b20188ec7 Merge 2014-03-18 17:12:49 -04:00
Igal Levy c4d06faa05 Merge branch 'master' into igalDruid 2014-03-18 11:00:10 -07:00
Igal Levy 04040ef75f Fixed:
- Bad link for granularitySpec changed to real definition.
- All heads set to logical levels (consistent with all others).
- Fixed bad link in definition for spatialDimensions.
2014-03-18 10:28:33 -07:00
fjy 804a4051fa prepare for next rel 2014-03-18 08:10:59 -07:00
fjy 1d1ec1d855 clean up some defaults and prepare for next release 2014-03-17 21:41:34 -07:00
fjy 2adcf07f5f Merge branch 'master' into new-schema
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DetermineHashedPartitionsJob.java
	indexing-service/src/main/java/io/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/test/java/io/druid/indexing/common/task/TaskSerdeTest.java
	processing/src/test/java/io/druid/segment/TestIndex.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
2014-03-17 10:59:31 -07:00
fjy f1da044cfa fix another breaking endpoint in druid coordinator 2014-03-14 15:15:08 -07:00
fjy f525361c3d fix broken tests and prepare for next release 2014-03-14 15:05:07 -07:00
fjy f0fdfbb2f7 fix broken constant post agg 2014-03-14 14:54:52 -07:00
fjy b3fb2fea9c prepare for next release 2014-03-14 10:57:43 -07:00
fjy 6844063a52 fix broken examples 2014-03-14 10:56:27 -07:00
fjy b4f1591260 clean up examples, finish paper 2014-03-13 18:52:08 -07:00
Igal Levy 71818288e2 syntax and grammar fixes; tightened up language and flow 2014-03-13 17:55:45 -07:00
Igal Levy a8820bd3a0 Merge branch 'master' into igalDruid 2014-03-13 17:44:43 -07:00
fjy ae2dc6506c fix broken granularity 2014-03-10 15:03:43 -07:00
fjy 243af486cf prepare for next release 2014-03-10 13:24:53 -07:00
fjy 228cfa95bd Merge pull request #371 from N3TWORK/subquery
Subquery functionality for Group By queries
2014-03-07 13:41:56 -07:00
Yuval Oren 4b63645802 Merge branch 'master' into subquery
Conflicts:
	processing/src/main/java/io/druid/query/BaseQuery.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryQueryToolChest.java
	processing/src/test/java/io/druid/query/QueryRunnerTestHelper.java
	processing/src/test/java/io/druid/query/topn/TopNQueryRunnerTest.java
2014-03-06 13:33:59 -08:00
fjy e1fac64298 prepare for next release 2014-03-06 12:53:05 -08:00
fjy 08138688e4 port hyperunique to open source 2014-03-05 14:19:38 -08:00
Igal Levy b6d9619db0 fixed markdown rendering issue (literal '<' are interpreted) 2014-03-04 16:14:19 -08:00
fjy 0e8e527662 Merge branch 'master' into new-schema 2014-03-04 14:42:16 -08:00
Igal Levy 916bd30db3 fixed link to Cassandra article 2014-03-04 11:44:06 -08:00
Igal Levy a73b4afc4b added note+link about Cassandra option 2014-03-04 11:41:05 -08:00
Igal Levy f9f9b30709 fixed minor typo 2014-03-04 11:40:35 -08:00
fjy 0feddc3831 Merge pull request #411 from metamx/small-fixes-docs-searchquery-limit
Small fixes
2014-03-04 10:30:27 -07:00
fjy 46b9ac78e7 Merge branch 'master' into new-schema
Conflicts:
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	pom.xml
	publications/whitepaper/druid.pdf
	publications/whitepaper/druid.tex
2014-03-03 14:48:15 -08:00
nishantmonu51 5603da189d fix #405 update cluster setup doc
rename old names to new ones
2014-02-28 13:02:05 +05:30
Igal Levy 95c491db0f fixed broken link 2014-02-26 17:37:52 -08:00
Igal Levy 0721220772 Merge branch 'master' into igalDruid 2014-02-26 17:36:48 -08:00
nishantmonu51 8af63005a6 refactor randomPartitionsSpec to hashedPartitionsSpec
refactor to a more appropriate name
2014-02-25 03:07:31 +05:30
Igal Levy 21299a6f81 refactored hadoop intro since the replacement page for instructions also wasn't right and there are quite a few available that are right 2014-02-21 10:43:52 -08:00
Igal Levy d3a1041e49 fixed broken url to other tutorial 2014-02-20 09:52:25 -08:00
Igal Levy 723f1427eb fixed broken url to hadoop setup instructions 2014-02-20 09:50:44 -08:00
Igal Levy 7aadef8e76 minor edits for clarity 2014-02-19 21:20:07 -08:00
fjy 20cac8c506 not compiling yet but close 2014-02-19 15:54:27 -08:00
Igal Levy 75ae4d434d added advice about restarting zk and mysql if not already running 2014-02-19 15:28:48 -08:00
fjy fb93448e9d prepare for next release because my last one failzored out of bad internetz 2014-02-14 15:22:27 -08:00
fjy 1a8f9f2e12 prepare for next release 2014-02-14 15:07:44 -08:00
fjy 3979eb270c Revert "Revert "Merge branch 'determine-partitions-improvements'""
This reverts commit 189b3e2b9b.
2014-02-14 12:58:56 -08:00
fjy 189b3e2b9b Revert "Merge branch 'determine-partitions-improvements'"
This reverts commit 7ad228ceb5, reversing
changes made to 9c55e2b779.
2014-02-14 12:47:34 -08:00
Igal Levy 5ee776db64 fixed broken link 2014-02-13 16:39:44 -08:00
Igal Levy 0714800904 Merge branch 'master' into igalDruid 2014-02-13 16:38:09 -08:00
fjy a986166b9c prepare for next release 2014-02-12 20:01:05 -08:00
Igal Levy 9313c7f511 minor tweaking to clarify instructions for csv and tsv 2014-02-12 18:59:35 -08:00
nishantmonu51 7ad228ceb5 Merge branch 'determine-partitions-improvements'
Conflicts:
	pom.xml
2014-02-12 10:51:26 +05:30
Gian Merlino 38e9a79560 Fix typo 2014-02-08 15:07:01 -08:00
fjy 2497bd2e85 prepare for next release 2014-02-07 13:04:34 -08:00
fjy c8b350270f prepare for next release 2014-02-07 11:54:56 -08:00
fjy 0fd95bc2b4 refactor out the configuration docs 2014-02-07 11:35:44 -08:00
nishantmonu51 48d0c37f98 documentation for random partition spec 2014-02-05 15:30:44 +05:30