Apache/druid
1
0
Fork 0
mirror of https://github.com/apache/druid.git synced 2025-02-07 18:48:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,536 Commits 116 Branches 507 Tags
Commit Graph

142 Commits

Author SHA1 Message Date
nishantmonu51
9598a524a8 review comment - move index closure to finally 2014-08-12 14:58:55 +05:30
nishantmonu51
637bd35785 merge changes from druid-0.7.x 2014-07-31 16:07:22 +05:30
nishantmonu51
4ce12470a1 Add way to skip determine partitions for index task 
Add a way to skip determinePartitions for IndexTask by manually
specifying numShards.
 2014-07-18 18:52:15 +05:30
nishantmonu51
f5f05e3a9b Sync changes from branch new-ingestion PR #599 
Sync and Resolve Conflicts
 2014-07-11 16:15:10 +05:30
nishantmonu51
fa43049240 review comments & pom changes 2014-07-10 11:48:46 +05:30
nishantmonu51
36fc85736c Add ShardSpec Lookup 
Optimize choosing shardSpec for Hash Partitions
 2014-07-08 18:01:31 +05:30
fjy
4c40e71e54 address cr 2014-06-19 14:48:46 -07:00
fjy
a870fe5cbe inject column config 2014-06-19 14:47:57 -07:00
Xavier Léauté
09346b0a3c make column cache configurable 2014-06-19 14:43:03 -07:00
fjy
a63cda3281 Merge branch 'master' into new-guava 
Conflicts:
	server/src/main/java/io/druid/server/QueryResource.java
 2014-06-13 10:08:10 -07:00
nishantmonu51
a7e19ad892 configure buffer sizes 2014-06-12 19:32:37 +05:30
nishantmonu51
6265613bb9 Merge branch 'master' into offheap-incremental-index 2014-06-05 17:42:57 +05:30
nishantmonu51
01e8a713b6 unit tests passing with offheap-indexing 2014-06-05 17:42:53 +05:30
Gian Merlino
1ca7bf03b8 IndexGeneratorJob needs to respect isCombineText, too. 2014-06-04 17:54:31 -07:00
fjy
adc00f2bcf make combine text configurable 2014-06-04 16:24:56 -07:00
fjy
bb4105ed1a fix broken standalone hadoop ingestion 2014-06-04 09:23:46 -07:00
fjy
77ec4df797 update guava, java-util, and druid-api 2014-06-03 13:43:38 -07:00
fjy
4c13327297 more logging for determine hashed 2014-05-30 16:19:20 -07:00
fjy
7be93a770a make all firehoses work with tasks, add a lot more documentation about configuration 2014-05-28 16:33:59 -07:00
Deepak
7d92cf2b3b Update IndexGeneratorJob.java 
CombineTextInputFormat instead of TextInputFormat combines multiple splits for a single mapper and reduces the strain on hadoop platform. It greatly improves job completion time as there are fewer number of mappers to bookkeep.
 2014-05-22 15:08:12 +05:30
Deepak
de0a7b27e7 Update DetermineHashedPartitionsJob.java 
CombineTextInputFormat instead of TextInputFormat combines multiple splits for a single mapper and reduces the strain on hadoop platform. It greatly improves job completion time as there are fewer number of mappers to bookkeep.
 2014-05-22 15:06:56 +05:30
Xavier Léauté
9ec7c71e0f fix compilation error with updated druid-api 2014-05-19 14:06:23 -07:00
fjy
1100d2f2a1 rename configs to make a bit more sense 2014-05-06 14:52:50 -07:00
fjy
b6fb4245aa Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDriverConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfig.java
	indexing-hadoop/src/main/java/io/druid/indexer/HadoopDruidIndexerConfigBuilder.java
	pom.xml
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/firehose/EventReceiverFirehoseFactory.java
 2014-05-06 14:32:51 -07:00
Gian Merlino
bdf9e74a3b Allow config-based overriding of hadoop job properties. 2014-05-06 09:11:31 -07:00
fjy
f9523274ac remove extra println 2014-05-01 15:06:51 -07:00
nishantmonu51
5137031304 use same logic for compression 
Use same logic for compression across creating files, reading from
files, and checking file existence
 2014-05-01 15:20:47 +05:30
nishantmonu51
728f1e8ee3 fix exists check with compression 2014-05-01 15:01:10 +05:30
nishantmonu51
01e84f10b7 add the checks again. 
removing these checks breaks when there is no data for any interval
 2014-05-01 14:35:09 +05:30
fjy
76e0a48527 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DbUpdaterJob.java
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	indexing-service/src/main/java/io/druid/indexing/common/task/HadoopIndexTask.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumber.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-04-25 14:03:28 -07:00
fjy
2d1f33e59f Merge pull request #500 from metamx/batch-ingestion-fixes 
Batch ingestion fixes
 2014-04-22 17:59:24 -06:00
nishantmonu51
357bbf5127 add all the shard specs 2014-04-23 05:23:11 +05:30
nishantmonu51
625a5418d2 minor fix 2014-04-23 05:05:51 +05:30
nishantmonu51
1ca61237c1 review comments- use final variables 2014-04-23 03:33:28 +05:30
nishantmonu51
0d8c1ffe54 review comments and add partitioner 2014-04-23 03:30:30 +05:30
nishantmonu51
ea4a80e8d2 Add serde test for shardCount 2014-04-23 00:24:08 +05:30
nishantmonu51
e920cec5d0 remove unused import 2014-04-23 00:13:30 +05:30
nishantmonu51
0748eabe9b batch ingestion fixes 
1) Fix path when mapped output is compressed
2) Add number of reducers to the determine hashed partitions job
manually
3) Add a way to disable determine partitions and specify shardCount in
HashedPartitionsSpec
 2014-04-23 00:05:08 +05:30
Crystark
40a6804192 Support for postgresql 
I think it was the last request using 'end' missing the postgresql support.
 2014-04-07 17:37:03 +02:00
fjy
2adcf07f5f Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/main/java/io/druid/indexer/DetermineHashedPartitionsJob.java
	indexing-service/src/main/java/io/druid/indexing/common/task/RealtimeIndexTask.java
	indexing-service/src/test/java/io/druid/indexing/common/task/TaskSerdeTest.java
	processing/src/test/java/io/druid/segment/TestIndex.java
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
	server/src/main/java/io/druid/segment/realtime/plumber/RealtimePlumberSchool.java
 2014-03-17 10:59:31 -07:00
nishantmonu51
4ec1959c30 Use druid implementation of HyperLogLog 
remove dependency on clear spring analytics
 2014-03-07 00:06:40 +05:30
fjy
5db00afb37 clean up and default values 2014-03-04 14:38:27 -08:00
fjy
c4c4d80336 make local testing pass 2014-03-03 14:52:43 -08:00
fjy
46b9ac78e7 Merge branch 'master' into new-schema 
Conflicts:
	indexing-hadoop/src/test/java/io/druid/indexer/HadoopDruidIndexerConfigTest.java
	pom.xml
	publications/whitepaper/druid.pdf
	publications/whitepaper/druid.tex
 2014-03-03 14:48:15 -08:00
fjy
13c7f1c7b1 remove dead code 2014-02-27 15:52:19 -08:00
fjy
bf2ddda897 unit tests passing after more refactoring 2014-02-27 15:21:09 -08:00
nishantmonu51
5e0d418b4b fix determine partitions partitioner to work in local mode 2014-02-26 16:31:42 +05:30
nishantmonu51
1ed5254d5b improvements 
1) Number of reducers use 1 only when intervals are to be determined
2) Read only useful bytes from BytesWritable
 2014-02-26 02:51:45 +05:30
nishantmonu51
8af63005a6 refactor randomPartitionsSpec to hashedPartitionsSpec 
refactor to a more appropriate name
 2014-02-25 03:07:31 +05:30
fjy
5d2367f0fd unit tests pass at this point 2014-02-20 15:52:12 -08:00