Commit Graph

13874 Commits

Author SHA1 Message Date
Eric Tschetter 0f63cb4f00 1) Have IndexGeneratorJob write the descriptors for each of the segments it creates to a path in the temporary working directory (generally HDFS)
2) Have the DbUpdaterJob read descriptors from the temporary working directory instead of looking in the final segment output location (often the eventually consistent S3)
3) 1 and 2 Fixes #30
2012-11-20 15:30:50 -06:00
Eric Tschetter 9cbffa287f Merge branch 'master' of github.com:metamx/druid 2012-11-20 10:13:49 -06:00
Eric Tschetter a48b67c917 1) Remove -Djava.net.preferIPv4Stack=true from the install script, not sure why it's there. 2012-11-20 10:13:33 -06:00
cheddar 6107bb7b80 Merge pull request #32 from metamx/fjy
Fixes to indexing service to handle large merges better
2012-11-19 15:51:07 -08:00
Fangjin Yang 4131063049 Merge branch 'master' into fjy 2012-11-19 15:07:00 -08:00
Fangjin Yang b228c053e5 bug fix for too much data to indexer 2012-11-19 15:06:46 -08:00
Eric Tschetter c657969243 Merge branch 'master' of github.com:metamx/druid 2012-11-19 17:01:33 -06:00
Eric Tschetter 701cc9562b 1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up.
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29, or at least clarifies the behavior...)
2012-11-19 17:01:17 -06:00
cheddar cdda8c2b7c Merge pull request #31 from tralfamadude/master
doc/data_flow* diagrams added
2012-11-19 14:55:47 -08:00
Gian Merlino 16835a1f84 Master: Add segment limit to merge selection algo
If we attempt to merge too many segments at once, we risk creating an
oversized indexing task.
2012-11-19 14:46:45 -08:00
Paul Baclace b7d43a6622 doc/data_flow* diagrams added 2012-11-15 21:32:56 -08:00
Eric Tschetter aa073e4e73 1) Add verification of memory settings to initialization. (Fixes #21) 2012-11-14 15:37:11 -06:00
Fangjin Yang cf1c741f8a [maven-release-plugin] prepare for next development iteration 2012-11-13 15:10:12 -08:00
Fangjin Yang 06bd77fca7 [maven-release-plugin] prepare release druid-0.1.4 2012-11-13 15:10:00 -08:00
cheddar 06ed0e7448 Merge pull request #28 from metamx/fjy
Some modifications on how to register things in Druid
2012-11-13 15:02:14 -08:00
Fangjin Yang cd1a812fff construct registering nodes with lists 2012-11-13 15:00:55 -08:00
Fangjin Yang 0ef40171a8 nodes no longer inherit from interfaces but instead extend classes 2012-11-13 13:18:31 -08:00
Fangjin Yang 24564d73e1 register subtypes for reducer 2012-11-12 16:41:34 -08:00
Fangjin Yang 7eccd76e9a add missing headers 2012-11-12 16:34:48 -08:00
Fangjin Yang 0d697ed104 fix rename of this node 2012-11-12 16:17:51 -08:00
Fangjin Yang 57468d39ef reverting some of the last changes 2012-11-12 16:14:48 -08:00
Fangjin Yang c20dccd0f4 modifying the way registering serdes works to hopefully be a bit easier to use 2012-11-12 13:58:43 -08:00
cheddar c30f04e0d4 Merge pull request #27 from marshallpierce/master
Remove duplicate copies of servlet-api from dependency tree
2012-11-12 08:49:31 -08:00
Marshall Pierce 447dfc0cc6 Switch to using javax.servlet:servlet-api:2.5 instead of two different versions of jetty's servlet-api distribution 2012-11-09 17:31:03 -08:00
Fangjin Yang dd17992172 [maven-release-plugin] prepare for next development iteration 2012-11-09 15:50:24 -08:00
Fangjin Yang 47e71080db [maven-release-plugin] prepare release druid-0.1.3 2012-11-09 15:50:16 -08:00
Fangjin Yang e668660a9c fix strategy check in indexing coordinator 2012-11-09 15:48:26 -08:00
Fangjin Yang 5859599365 [maven-release-plugin] prepare for next development iteration 2012-11-09 13:41:04 -08:00
Fangjin Yang 56f2c15e21 [maven-release-plugin] prepare release druid-0.1.2 2012-11-09 13:40:57 -08:00
Fangjin Yang 2ef1136ab2 fix a bug where intervals were being returned as String instead of Interval objects by the compute nodes 2012-11-09 13:39:02 -08:00
Fangjin Yang 0b7a3f259a [maven-release-plugin] prepare for next development iteration 2012-11-09 10:10:32 -08:00
Fangjin Yang baba4e7b75 [maven-release-plugin] prepare release druid-0.1.1 2012-11-09 10:10:26 -08:00
Fangjin Yang 03e5d74087 [maven-release-plugin] prepare for next development iteration 2012-11-09 09:33:06 -08:00
Fangjin Yang 8aff8decad [maven-release-plugin] prepare release druid-0.1.0 2012-11-09 09:32:57 -08:00
Eric Tschetter 0a0e2a6cc1 1) Try to fix the dependency issues for running the HadoopDruidIndexer locally. 2012-11-08 17:06:02 -08:00
cheddar 81a33bcbb3 Merge pull request #26 from metamx/misc
Updates to druid indexer to be able to index complex metrics
2012-11-08 16:39:07 -08:00
Fangjin Yang 6da047b5fa fix backwards compatibility issues 2012-11-08 15:09:00 -08:00
Fangjin Yang 17fb93d049 fixes to init and more unit tests for timeseries 2012-11-08 14:43:02 -08:00
Fangjin Yang 5cf21e3c8a Merge branch 'master' into misc 2012-11-08 13:49:02 -08:00
cheddar 90b7fd9935 Merge pull request #25 from metamx/moarunittests
A basic timeseries query for druid and also more unit tests
2012-11-08 13:46:30 -08:00
Fangjin Yang 09e81be14c rename file 2012-11-08 13:42:58 -08:00
Fangjin Yang e8d1909883 rename file 2012-11-08 13:42:14 -08:00
Fangjin Yang ef6e65b8ca Merge branch 'moarunittests' into misc
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
2012-11-08 10:51:41 -08:00
Fangjin Yang 4c98b176e4 Merge branch 'master' into misc
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
2012-11-08 10:51:13 -08:00
Fangjin Yang 3ddfdfc702 add missing header 2012-11-08 10:45:21 -08:00
Fangjin Yang bff5a7d786 fix some merge issues 2012-11-08 10:43:39 -08:00
Fangjin Yang 8323feedb4 Merge branch 'master' into moarunittests
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
2012-11-08 10:37:05 -08:00
Fangjin Yang 88bb02963a fix compile breakage introduced by scaling strategies 2012-11-08 10:32:18 -08:00
cheddar f8a5d08dc3 Merge pull request #20 from metamx/autoscaling
Improvements to distributed indexing service and auto scaling
2012-11-08 10:11:33 -08:00
Fangjin Yang 7083821fbf no param for scaling provision and minor fix for initialization 2012-11-08 10:08:40 -08:00