110 Commits

Author SHA1 Message Date
Xavier Léauté
0498df25df override metadata storage injection in CliHadoopIndexer 2014-11-07 13:44:56 -08:00
Xavier Léauté
50a191425c fix injection on MetadataStorageUpdaterJob 2014-11-07 11:11:14 -08:00
Xavier Léauté
20a9aef96a fix test 2014-11-06 17:27:05 -08:00
Xavier Léauté
9c06db021f rename db->metadata postgres->postgresql 2014-10-31 10:30:27 -07:00
jisookim0513
aa754b86e8 build success! 2014-10-24 11:28:42 -07:00
fjy
bef74104d9 merge with 0.7.x and resolve any conflicts 2014-10-23 17:24:06 -07:00
fjy
d76d57d95d update docs 2014-10-22 16:16:28 -07:00
jisookim0513
37979282fe enabled ansi-quote in mysql; insert statement should now work 2014-10-21 00:09:19 -07:00
jisookim0513
7d5c5f2083 fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider 2014-10-17 00:10:36 -07:00
nishantmonu51
41e88baeca Add test for bucket selection 2014-10-15 23:09:28 +05:30
nishantmonu51
f4a97aebbc fix rollup for hashed partitions
truncate timestamp while calculating the partitionNumber
2014-10-15 22:32:56 +05:30
jisookim0513
9d7b5d9b0f fixed javadoc; fixed pom files; deleted unnecessary class 2014-09-30 13:47:35 -07:00
jisookim0513
74565c9371 cleaned up the code 2014-09-27 13:10:01 -07:00
jisookim0513
aa887edb73 added two seperate modules for mysql and postgres 2014-09-27 13:08:53 -07:00
flow
2dd62979bb Fixed the issue of batch ingestion with indexing service to hdfs end up with the path of metadata in mysql missing "hdfs://host" prefix. The detail describe can be found here: https://groups.google.com/forum/#!topic/druid-development/ofvSxiPpCxI 2014-09-27 22:26:52 +08:00
jisookim0513
6a641621b2 finished merging into druid-0.7.x; derby not working (to be fixed) 2014-09-26 14:24:53 -07:00
jisookim0513
43cc6283d3 trying to revert files that have overwritten changes 2014-09-26 12:38:04 -07:00
jisookim0513
3bf39cc9f8 attempted to fix merge-conflicts 2014-09-24 15:55:42 -07:00
jisookim0513
273205f217 initial attempt for abstraction; druid cluster works with Derby as a default 2014-09-19 17:39:59 -07:00
Xavier Léauté
d501b052ea remove unused columnConfig 2014-09-15 13:02:47 -07:00
Xavier Léauté
e57e2d97ba make constants final 2014-09-15 12:53:40 -07:00
fjy
469ccbbe5e Merge branch 'master' into druid-0.7.x
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
2014-09-11 16:20:50 -07:00
Xavier Léauté
58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté
ac05836833 make Java 8 javadoc happy 2014-08-29 13:58:50 -07:00
nishantmonu51
4ce12470a1 Add way to skip determine partitions for index task
Add a way to skip determinePartitions for IndexTask by manually
specifying numShards.
2014-07-18 18:52:15 +05:30
nishantmonu51
fa43049240 review comments & pom changes 2014-07-10 11:48:46 +05:30
nishantmonu51
36fc85736c Add ShardSpec Lookup
Optimize choosing shardSpec for Hash Partitions
2014-07-08 18:01:31 +05:30
fjy
4c40e71e54 address cr 2014-06-19 14:48:46 -07:00
fjy
a870fe5cbe inject column config 2014-06-19 14:47:57 -07:00
Xavier Léauté
09346b0a3c make column cache configurable 2014-06-19 14:43:03 -07:00
fjy
a63cda3281 Merge branch 'master' into new-guava
Conflicts:
	server/src/main/java/io/druid/server/QueryResource.java
2014-06-13 10:08:10 -07:00
Gian Merlino
1ca7bf03b8 IndexGeneratorJob needs to respect isCombineText, too. 2014-06-04 17:54:31 -07:00
fjy
adc00f2bcf make combine text configurable 2014-06-04 16:24:56 -07:00
fjy
bb4105ed1a fix broken standalone hadoop ingestion 2014-06-04 09:23:46 -07:00
fjy
77ec4df797 update guava, java-util, and druid-api 2014-06-03 13:43:38 -07:00
fjy
4c13327297 more logging for determine hashed 2014-05-30 16:19:20 -07:00
fjy
7be93a770a make all firehoses work with tasks, add a lot more documentation about configuration 2014-05-28 16:33:59 -07:00
Deepak
7d92cf2b3b Update IndexGeneratorJob.java
CombineTextInputFormat instead of TextInputFormat combines multiple splits for a single mapper and reduces the strain on hadoop platform. It greatly improves job completion time as there are fewer number of mappers to bookkeep.
2014-05-22 15:08:12 +05:30
Deepak
de0a7b27e7 Update DetermineHashedPartitionsJob.java
CombineTextInputFormat instead of TextInputFormat combines multiple splits for a single mapper and reduces the strain on hadoop platform. It greatly improves job completion time as there are fewer number of mappers to bookkeep.
2014-05-22 15:06:56 +05:30
Xavier Léauté
9ec7c71e0f fix compilation error with updated druid-api 2014-05-19 14:06:23 -07:00
fjy
1100d2f2a1 rename configs to make a bit more sense 2014-05-06 14:52:50 -07:00
fjy
b6fb4245aa Merge branch 'master' into new-schema
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
	pom.xml
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
2014-05-06 14:32:51 -07:00
Gian Merlino
bdf9e74a3b Allow config-based overriding of hadoop job properties. 2014-05-06 09:11:31 -07:00
fjy
f9523274ac remove extra println 2014-05-01 15:06:51 -07:00
nishantmonu51
5137031304 use same logic for compression
Use same logic for compression across creating files, reading from
files, and checking file existence
2014-05-01 15:20:47 +05:30
nishantmonu51
728f1e8ee3 fix exists check with compression 2014-05-01 15:01:10 +05:30
nishantmonu51
01e84f10b7 add the checks again.
removing these checks breaks when there is no data for any interval
2014-05-01 14:35:09 +05:30
fjy
76e0a48527 Merge branch 'master' into new-schema
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
2014-04-25 14:03:28 -07:00
fjy
2d1f33e59f Merge pull request #500 from metamx/batch-ingestion-fixes
Batch ingestion fixes
2014-04-22 17:59:24 -06:00
nishantmonu51
357bbf5127 add all the shard specs 2014-04-23 05:23:11 +05:30