Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
325 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

325 Commits

Author SHA1 Message Date
cheddar ec034ddef4 Merge pull request #56 from metamx/determine-partitions 
Determine partitions better
 2013-01-24 12:57:39 -08:00
cheddar ad6f962000 Merge pull request #58 from metamx/cache-bulkget 
modify cacheBroker interface to implement bulk-get
 2013-01-24 09:57:43 -08:00
Eric Tschetter 9b6244ec15 Merge branch 'master' of github.com:metamx/druid 2013-01-23 18:37:03 -06:00
Eric Tschetter 67feee3cd6 1) Indexes don't always have an index.drd file anymore 2013-01-23 18:36:52 -06:00
Gian Merlino 77a3f3cbe0 Merge branch 'master' into determine-partitions 
Conflicts:
	indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
 2013-01-21 14:46:13 -08:00
Gian Merlino d9e6f1d954 DeterminePartitions follow-up 
HadoopDruidIndexerConfig:
- Add partitionsSpec (backwards compatible with targetPartitionSize and partitionDimension)
- Add assumeGrouped flag to partitionsSpec

DeterminePartitionsJob:
- Skip group-by job if assumeGrouped is set
- Clean up code a bit
 2013-01-21 14:38:35 -08:00
xvrl 40c0bcad29 simplify MapCache 2013-01-18 18:25:51 -08:00
cheddar 07131c51ed Merge pull request #53 from metamx/autoscaling 
Use a database to store configurations for indexer workers
 2013-01-18 17:08:52 -08:00
Fangjin Yang ac31afbce5 remove redundant index for primary key in config table 2013-01-18 16:45:23 -08:00
cheddar 8a8b8a59f9 Merge pull request #57 from metamx/yops-descriptor-dimensions 
YeOldePlumberSchool: Populate dimension metadata in segment descriptor
 2013-01-18 16:42:02 -08:00
xvrl e0c34c3b97 rename cacheBroker -> cache 2013-01-18 15:22:56 -08:00
xvrl 86ca8967ca rework code pulling from cache to be more readable 2013-01-18 15:17:10 -08:00
xvrl a70ae15585 replace Pair<String, ByteBuffer> with NamedKey 2013-01-18 15:17:05 -08:00
xvrl 9032ef521b fix interrupted thread 2013-01-18 15:17:05 -08:00
Gian Merlino 7166534666 YeOldePlumberSchool: Tweak for IndexIO changes 2013-01-17 16:03:18 -08:00
Gian Merlino 5ce53eb2ac Merge branch 'master' into yops-descriptor-dimensions 2013-01-17 16:02:32 -08:00
Fangjin Yang 38b2041ad9 key/value config table 2013-01-17 14:56:48 -08:00
Eric Tschetter 689ce4f8e1 1) Upgrade java-util dependency to include "ruby" time 2013-01-17 13:10:11 -06:00
xvrl 0bacb85a4a fix duplicate keys, shutdown gracefully and make sure we check all multiget keys in memcached benchmark 2013-01-16 19:18:14 -08:00
xvrl dcaa77a883 implement bulk get test 2013-01-16 19:15:43 -08:00
Eric Tschetter 5b1e03530c 1) Fix some bugs found by external test suite 2013-01-16 21:06:57 -06:00
xvrl e2788187fb don't let timeout skew benchmark stats 2013-01-16 16:02:51 -08:00
Eric Tschetter c8cb96b006 1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir() 
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
 2013-01-16 17:10:33 -06:00
xvrl a2090411a3 modify cacheBroker interface to implement bulk-get 2013-01-16 14:49:50 -08:00
Gian Merlino 6fc350bfba YeOldePlumberSchool: Populate dimension metadata in segment descriptor 2013-01-16 11:30:24 -08:00
Gian Merlino 7b42ee6a6e Rework DeterminePartitionsJob in the hadoop indexer 
- Can handle non-rolled-up input (by grouping input rows using an additional MR stage)
- Can select its own partitioning dimension, if none is supplied
- Can detect and avoid oversized shards due to bad dimension value distribution
- Shares input parsing code with IndexGeneratorJob
 2013-01-16 08:15:01 -08:00
Eric Tschetter 538d00e75e Merge branch 'master' of github.com:metamx/druid 2013-01-16 10:02:01 -06:00
Eric Tschetter 8b31d8db9f 1) Adjust IndexMerger to create convert the indexes it creates from the old format to the new. This is done quite sub-optimally, but it will work for now... 2013-01-16 10:01:46 -06:00
Gian Merlino 616415cb7e UniformGranularitySpec: Only return bucketInterval for timestamps that legitimately 
overlap our input intervals
 2013-01-15 22:30:17 -08:00
Gian Merlino 86277d1114 StringInputRowParser: Treat dimensionExclusions case-insensitively 2013-01-15 22:30:17 -08:00
Fangjin Yang 3c0880fe28 Merge branch 'master' into autoscaling 2013-01-15 13:36:39 -08:00
Fangjin Yang 7e074e8158 fix pom breakage 2013-01-15 12:04:12 -08:00
Fangjin Yang 47ac36d3e4 Merge branch 'master' of github.com:metamx/druid into autoscaling 2013-01-15 12:02:32 -08:00
Eric Tschetter 998f0bf3c8 1) Better error message when aggregators want more memory than is available. 2013-01-14 18:00:51 -06:00
Fangjin Yang ae76097bc8 Merge branch 'master' into autoscaling 2013-01-14 14:55:29 -08:00
Fangjin Yang d1f4317af7 updates to autoscaling config based on code review comments 2013-01-14 14:55:04 -08:00
Eric Tschetter 135474e49e Merge branch 'master' of github.com:metamx/druid 2013-01-14 15:01:32 -06:00
Eric Tschetter 3de62b34b6 1) Should be on 0.2.0 with the recent API changes for StorageAdapters 2013-01-14 15:00:28 -06:00
Fangjin Yang 4c2da93389 bug fixes with ec2 auto scaling 2013-01-09 14:51:35 -08:00
Fangjin Yang 3685839376 Merge branch 'master' into autoscaling 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
 2013-01-07 18:04:14 -08:00
xvrl 700ce24106 fix JVM optimizing out the benchmarking code 2013-01-07 16:54:35 -08:00
Fangjin Yang 79e9ff94a6 add missing headers 2013-01-07 14:07:47 -08:00
Fangjin Yang aff4a05ca3 db based configs for indexer workers 2013-01-07 13:41:26 -08:00
Eric Tschetter 4ef31de62d 1) Fix stray import, take 2... yay IntelliJ 2013-01-07 13:11:28 -08:00
Eric Tschetter b62bdf915b 1) Fix stray import... yay IntelliJ 2013-01-07 10:09:19 -08:00
Eric Tschetter 72db507cf8 1) Update java-util version to be a non-SNAPSHOT version 2013-01-06 23:37:52 -08:00
Eric Tschetter b184c46ae0 1) Introduce idea of value types and generic columns to the serialization format 
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
 2013-01-04 18:12:49 -06:00
Eric Tschetter 4f5087b386 Merge branch 'master' of github.com:metamx/druid 2013-01-04 11:28:35 -06:00
xvrl 9627fdcd1c [maven-release-plugin] prepare for next development iteration 2013-01-03 10:09:52 -08:00
xvrl f2c2d3f4cf [maven-release-plugin] prepare release druid-0.1.25 2013-01-03 10:09:43 -08:00