Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,742 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

2742 Commits

Author SHA1 Message Date
Gian Merlino 77a3f3cbe0 Merge branch 'master' into determine-partitions 
Conflicts:
	indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
 2013-01-21 14:46:13 -08:00
Gian Merlino d9e6f1d954 DeterminePartitions follow-up 
HadoopDruidIndexerConfig:
- Add partitionsSpec (backwards compatible with targetPartitionSize and partitionDimension)
- Add assumeGrouped flag to partitionsSpec

DeterminePartitionsJob:
- Skip group-by job if assumeGrouped is set
- Clean up code a bit
 2013-01-21 14:38:35 -08:00
xvrl 068ca67dba fix cache not preserving timezone information 2013-01-21 14:38:04 -08:00
Nelson Ray 2d7113b263 huge simplification of balancing code 2013-01-21 14:28:25 -08:00
xvrl 40c0bcad29 simplify MapCache 2013-01-18 18:25:51 -08:00
cheddar 07131c51ed Merge pull request #53 from metamx/autoscaling 
Use a database to store configurations for indexer workers
 2013-01-18 17:08:52 -08:00
Fangjin Yang ac31afbce5 remove redundant index for primary key in config table 2013-01-18 16:45:23 -08:00
cheddar 8a8b8a59f9 Merge pull request #57 from metamx/yops-descriptor-dimensions 
YeOldePlumberSchool: Populate dimension metadata in segment descriptor
 2013-01-18 16:42:02 -08:00
xvrl e0c34c3b97 rename cacheBroker -> cache 2013-01-18 15:22:56 -08:00
xvrl 86ca8967ca rework code pulling from cache to be more readable 2013-01-18 15:17:10 -08:00
xvrl a70ae15585 replace Pair<String, ByteBuffer> with NamedKey 2013-01-18 15:17:05 -08:00
xvrl 9032ef521b fix interrupted thread 2013-01-18 15:17:05 -08:00
Gian Merlino 7166534666 YeOldePlumberSchool: Tweak for IndexIO changes 2013-01-17 16:03:18 -08:00
Gian Merlino 5ce53eb2ac Merge branch 'master' into yops-descriptor-dimensions 2013-01-17 16:02:32 -08:00
Fangjin Yang 38b2041ad9 key/value config table 2013-01-17 14:56:48 -08:00
Eric Tschetter 689ce4f8e1 1) Upgrade java-util dependency to include "ruby" time 2013-01-17 13:10:11 -06:00
xvrl 0bacb85a4a fix duplicate keys, shutdown gracefully and make sure we check all multiget keys in memcached benchmark 2013-01-16 19:18:14 -08:00
xvrl dcaa77a883 implement bulk get test 2013-01-16 19:15:43 -08:00
Eric Tschetter 5b1e03530c 1) Fix some bugs found by external test suite 2013-01-16 21:06:57 -06:00
Fangjin Yang 21613bc73b initial commit to hard delete segments 2013-01-16 17:31:01 -08:00
xvrl e2788187fb don't let timeout skew benchmark stats 2013-01-16 16:02:51 -08:00
Eric Tschetter c8cb96b006 1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir() 
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
 2013-01-16 17:10:33 -06:00
xvrl a2090411a3 modify cacheBroker interface to implement bulk-get 2013-01-16 14:49:50 -08:00
Gian Merlino 6fc350bfba YeOldePlumberSchool: Populate dimension metadata in segment descriptor 2013-01-16 11:30:24 -08:00
Gian Merlino 7b42ee6a6e Rework DeterminePartitionsJob in the hadoop indexer 
- Can handle non-rolled-up input (by grouping input rows using an additional MR stage)
- Can select its own partitioning dimension, if none is supplied
- Can detect and avoid oversized shards due to bad dimension value distribution
- Shares input parsing code with IndexGeneratorJob
 2013-01-16 08:15:01 -08:00
Eric Tschetter 538d00e75e Merge branch 'master' of github.com:metamx/druid 2013-01-16 10:02:01 -06:00
Eric Tschetter 8b31d8db9f 1) Adjust IndexMerger to create convert the indexes it creates from the old format to the new. This is done quite sub-optimally, but it will work for now... 2013-01-16 10:01:46 -06:00
Gian Merlino 616415cb7e UniformGranularitySpec: Only return bucketInterval for timestamps that legitimately 
overlap our input intervals
 2013-01-15 22:30:17 -08:00
Gian Merlino 86277d1114 StringInputRowParser: Treat dimensionExclusions case-insensitively 2013-01-15 22:30:17 -08:00
Fangjin Yang 3c0880fe28 Merge branch 'master' into autoscaling 2013-01-15 13:36:39 -08:00
Fangjin Yang 7e074e8158 fix pom breakage 2013-01-15 12:04:12 -08:00
Fangjin Yang 47ac36d3e4 Merge branch 'master' of github.com:metamx/druid into autoscaling 2013-01-15 12:02:32 -08:00
Nelson Ray e847faf02f change name of MAX_SEGMENTS_TO_MOVE since it's not a final 2013-01-15 11:29:46 -08:00
Nelson Ray ef9e077d9a remove getConfig method from DruidMaster 2013-01-14 16:01:29 -08:00
Eric Tschetter 998f0bf3c8 1) Better error message when aggregators want more memory than is available. 2013-01-14 18:00:51 -06:00
Fangjin Yang ae76097bc8 Merge branch 'master' into autoscaling 2013-01-14 14:55:29 -08:00
Fangjin Yang d1f4317af7 updates to autoscaling config based on code review comments 2013-01-14 14:55:04 -08:00
Eric Tschetter 135474e49e Merge branch 'master' of github.com:metamx/druid 2013-01-14 15:01:32 -06:00
Eric Tschetter 3de62b34b6 1) Should be on 0.2.0 with the recent API changes for StorageAdapters 2013-01-14 15:00:28 -06:00
Nelson Ray 5f40dc6d9c put BalancerCostAnalyzer into params 2013-01-10 10:27:43 -08:00
Fangjin Yang 4c2da93389 bug fixes with ec2 auto scaling 2013-01-09 14:51:35 -08:00
Nelson Ray 617907d85b move empty server holder list check outside of loop 2013-01-08 16:12:16 -08:00
Nelson Ray 4549c3a8c3 add some finals, reorganize 2013-01-08 09:07:43 -08:00
Fangjin Yang 3685839376 Merge branch 'master' into autoscaling 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
 2013-01-07 18:04:14 -08:00
Nelson Ray 2b4dbdde03 lots of changes 2013-01-07 17:28:53 -08:00
Nelson Ray 6fda5330fd finish making MAX_SEGMENTS_TO_MOVE a config item 2013-01-07 16:56:55 -08:00
xvrl 700ce24106 fix JVM optimizing out the benchmarking code 2013-01-07 16:54:35 -08:00
Nelson Ray ba82fe746a put MAX_SEGMENTS_TO_MOVE in the config 2013-01-07 16:28:27 -08:00
Fangjin Yang 79e9ff94a6 add missing headers 2013-01-07 14:07:47 -08:00
Fangjin Yang aff4a05ca3 db based configs for indexer workers 2013-01-07 13:41:26 -08:00