Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,729 Commits 111 Branches 503 Tags 728 MiB
Commit Graph

244 Commits

Author SHA1 Message Date
nishantmonu51 e5fa8e0cb4 Fix NPE for union queries 
filter non-existing datasources for union queries.

handle review comment
 2015-02-10 23:21:58 +05:30
Fangjin Yang 25cf15824b Merge pull request #1085 from gianm/dsmrv-fix 
DataSourceMetadataResultValue fixes and JodaUtils adjustments.
 2015-02-03 17:51:33 -08:00
Gian Merlino 1298b4abd3 JodaUtils: Bring in MIN/MAX instants a bit. 
This prevents the difference overflowing a long.
 2015-02-03 17:34:33 -08:00
Eric Tschetter 42eba986ce Towards consistent null handling 
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
 2015-02-02 12:53:07 -08:00
Fangjin Yang 92e616de11 Merge pull request #1077 from metamx/remove-unused-imports 
remove unused imports
 2015-02-02 10:45:27 -08:00
nishantmonu51 ba932bb1f2 remove unused imports 2015-02-02 21:53:39 +05:30
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
Charles Allen 3d27747f7e Upgrade to log4j2 
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
 2015-01-20 12:35:18 -08:00
fjy 84cc32ba32 Add more logging for when ingestion may be throttled 2015-01-08 14:51:46 -08:00
fjy c0af7b2a5d fix broken ut 2014-12-15 18:29:15 -08:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
Xavier Léauté d19b895fb9 add tests for empty sequences 2014-11-13 14:49:13 -08:00
fjy 788a4a846b add encoding 2014-11-11 17:31:08 -08:00
Charles Allen a89b539b4f Merge pull request #823 from metamx/roaring 
Configurable bitmap indexes: roaring and concise
 2014-11-11 17:26:38 -08:00
fjy 1cc162727b address code review 2014-11-11 14:05:37 -08:00
Xavier Léauté 59f093882d fix Derby instantiation 2014-11-10 16:55:53 -08:00
fjy 8cbd21af90 fix defaults 2014-11-10 15:40:57 -08:00
fjy 98d05e63bf make derby instance only on coordinator 2014-11-10 15:34:49 -08:00
nishantmonu51 8bebb24fd5 Union Queries move merge to historical 2014-11-07 18:05:16 +05:30
Xavier Léauté 5bda4ee1dd global task entry type 2014-11-06 17:08:20 -08:00
Xavier Léauté ac753ed0f0 always use validation query 2014-11-06 17:08:01 -08:00
Xavier Léauté 10aa8a2d56 add explicit type names 2014-11-06 17:07:24 -08:00
Xavier Léauté 350bb09605 refactor sql storage to abstract task storage 2014-11-05 17:19:37 -08:00
Xavier Léauté 253c0a70b3 simplify interface + more tests 2014-11-04 15:03:13 -08:00
Xavier Léauté 696c9ba944 fix formatting 2014-10-31 13:32:18 -07:00
jisookim0513 a6016d70fc added deleted classes back 2014-10-31 13:10:09 -07:00
Xavier Léauté 9c06db021f rename db->metadata postgres->postgresql 2014-10-31 10:30:27 -07:00
Xavier Léauté fb4d41cedb make the injection gods happy 2014-10-30 21:16:36 -07:00
Xavier Léauté 377151beda better abstraction for metadatastorage 2014-10-30 18:23:35 -07:00
jisookim0513 37979282fe enabled ansi-quote in mysql; insert statement should now work 2014-10-21 00:09:19 -07:00
jisookim0513 7d5c5f2083 fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider 2014-10-17 00:10:36 -07:00
fjy c7b4d5b7b4 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
 2014-10-02 18:12:10 -07:00
jisookim0513 0e50852985 fixed MetadataTaskStorage and handler 2014-09-30 14:09:23 -07:00
jisookim0513 9d7b5d9b0f fixed javadoc; fixed pom files; deleted unnecessary class 2014-09-30 13:47:35 -07:00
nishantmonu51 91eecfe3e7 fix #730 - memory check throws exception if fails 
fix exception handling, do not catch ProvisionException
 2014-09-30 23:03:40 +05:30
jisookim0513 74565c9371 cleaned up the code 2014-09-27 13:10:01 -07:00
jisookim0513 aa887edb73 added two seperate modules for mysql and postgres 2014-09-27 13:08:53 -07:00
jisookim0513 6a641621b2 finished merging into druid-0.7.x; derby not working (to be fixed) 2014-09-26 14:24:53 -07:00
jisookim0513 43cc6283d3 trying to revert files that have overwritten changes 2014-09-26 12:38:04 -07:00
fjy eaf0a48b92 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-09-26 11:39:24 -07:00
jisookim0513 3bf39cc9f8 attempted to fix merge-conflicts 2014-09-24 15:55:42 -07:00
jisookim0513 273205f217 initial attempt for abstraction; druid cluster works with Derby as a default 2014-09-19 17:39:59 -07:00
Xavier Léauté cfa92e8217 fix incorrect nullable annotations 2014-09-15 12:13:52 -07:00
fjy 469ccbbe5e Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
 2014-09-11 16:20:50 -07:00
Xavier Léauté 88de45c695 ensure jackson injection is available on startup 2014-08-29 15:13:52 -07:00
Xavier Léauté 58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté ac05836833 make Java 8 javadoc happy 2014-08-29 13:58:50 -07:00
fjy 4c40e71e54 address cr 2014-06-19 14:48:46 -07:00
fjy 77ec4df797 update guava, java-util, and druid-api 2014-06-03 13:43:38 -07:00
Xavier Léauté 4de7d4e127 make timeline findOvershadowed a set 2014-05-19 20:24:47 -07:00
fjy 1b6e816c28 Merge pull request #544 from metamx/java-8-fix-tests 
Make tests pass with Java 8
 2014-05-19 18:11:52 -06:00
Xavier Léauté 3a17a60d4c more comprehensive merge sequence tests 2014-05-15 16:27:13 -07:00
Xavier Léauté b51f0f7837 update java-util and add more merge sequence tests 2014-05-15 16:19:12 -07:00
fjy 43853a1d9e Merge pull request #545 from metamx/union-query-fix 
Union query fix
 2014-05-15 17:17:46 -06:00
nishantmonu51 ddc36c89cc union query fix and a broken test 2014-05-15 23:10:40 +05:30
Xavier Léauté 8fb76138bd tests must not assume findOvershadowed ordering 2014-05-14 22:48:10 -07:00
Xavier Léauté 62c55eaf26 fix broken CombiningSequence.toYielder behavior 2014-05-14 17:08:15 -07:00
Xavier Léauté e0ff2aa0d6 make isPostgreSQL check URI instead of metadata 2014-03-27 12:13:10 -07:00
Xavier Léauté fa34f2ac59 more consistent placeholder references 2014-03-26 16:41:27 -07:00
Xavier Léauté e84128ebbe remove database specific code from ConfigManager 2014-03-26 14:29:37 -07:00
fjy aec6457a4d fix config manager 2014-03-26 09:42:33 -07:00
Gian Merlino 70db460f97 Blocking Executors and maxPendingPersists, oh my! 
- Execs.newBlockingSingleThreaded can now accept capacity = 0.
- Changed default maxPendingPersists from 2 to 0.
- Fixed serde of maxPendingPersists in RealtimeIndexTasks.
 2014-03-05 10:55:12 -08:00
Tugdual Saunier e40725d5f3 Added support for PostgreSQL on overlord nodes 2014-02-26 01:24:37 +01:00
Tugdual Saunier a835db2a3c Added support for task storage in PostgreSQL 2014-02-26 01:24:37 +01:00
Tugdual Saunier d089f05127 Improved support for PostgreSQL in config management 2014-02-17 18:11:15 +00:00
Tugdual Saunier 8fac0b0b7a Fixed automatic tables creation for PostgreSQL 2014-02-17 18:11:15 +00:00
Gian Merlino 53c5124b99 DbConnector: Indexes on tasks table 2014-02-05 15:59:14 -08:00
nishantmonu51 5c043803c6 review comments 2014-01-20 13:40:36 +05:30
nishantmonu51 874cdb1b5b review comments 2014-01-17 12:51:28 +05:30
nishantmonu51 3db33c3268 review comments 2014-01-17 12:51:10 +05:30
nishantmonu51 6f5e1afbff add blocking mechanism after 2 pending persist batches 2014-01-17 03:01:04 +05:30
fjy d49554185a fix broken default table names 2013-11-07 17:38:00 -08:00
fjy 560981acbf a ton of fixes to docs 2013-10-10 15:05:01 -07:00
cheddar c47fe202c7 Fix HadoopDruidIndexer to work with the new way of things 
There are multiple and sundry changes in here.

First, "HadoopDruidIndexer" has been split into two pieces, (1) CliHadoop which pulls the hadoop version and builds up the right classpath with the proper hadoop version to run the indexer and (2) CliInternalHadoopIndexer which actually runs the indexer.

In order to work around a bunch of jets3t version conflicts with Hadoop and Druid, I needed to extract the S3 deep storage stuff into its own module.  I then also moved the HDFS stuff into its own module so that I could eliminate the dependency on Hadoop for druid-server.

In doing these changes, I wanted to make the extensions buildable with only the druid-api jar, so a few other things had to move out of Druid and into druid-api.  They are all API-level things, however, so they really belong in druid-api instead.

Lastly, I removed the druid-realtime module and put it all in druid-server.
 2013-10-09 15:15:44 -05:00
fjy bc8db7daa5 1) make chat handler resource work again 
2) add more default configs
3) make examples work again
 2013-10-02 14:22:39 -07:00
fjy 87259321b6 port hadoop druid indexer to new guice framework 2013-09-26 11:04:42 -07:00
cheddar 5712b29c8c Fix issues with bindings and handling extensions 
The way the Guice bindings were setup previously, each process only had bindings
for the things it cared about.  This became problematic when adding extension modules
that bound everything that they could possibly need expecting that the processes would
only instantiate what they actually do need.  Guice tries to fail-fast and verifies that all
 bindings exist before it does anything, which is a problem because the extension bind
 some objects that don't necessarily have all of their dependencies bound in all processes.

The fix for this is to build a single Injector with all bindings in it and let each of the
 processes only load the things that they care about.  This also requires the use of
 Module overrides and other such interesting things, which are node done.

 In doing the fix, I also swapped out the way that the DataSegmentPusher/Puller stuff is bound, as well as made the Cassandra stuff fail if its settings are not provided.  This all of a sudden made all of the things require Cassandra's settings, so I migrated the Cassandra deep storage stuff into its own module.

 In doing these changes, I also discovered that some properties weren't properly converting for the ConvertProperties command (specifically, the properties related to data segment loading and pushing), so I fixed that.
 2013-09-20 17:45:01 -05:00
fjy 64f3821b4e throw exception for annotated members 2013-09-16 17:36:31 -07:00
cheddar 3e4a4f5566 1) Adjustments to allow for the addition of new Runnables via modules. Modules should implement CliCommandCreator and add their stuff there. 2013-09-11 17:41:20 -05:00
cheddar dadd9bc5ce 1) Rename MetricSelector(Factory) to ColumnSelector(Factory) 
2) Hard coded plugin as a first pass, won't actually work for anyone else.
 2013-09-04 11:19:46 -05:00
cheddar 3c39f90c89 1) Move Firehose interface and dependencies to druid-api 
2) Move DataSegment* interfaces and dependencies to druid-api
 2013-08-31 16:43:28 -05:00
cheddar 5ab671050e No more com.metamx.druid, it is now all io.druid! 2013-08-30 19:42:12 -05:00
cheddar bd0756e360 More stuff moved, things still compiling and tests still passing. Yay! 2013-08-30 18:58:35 -05:00
cheddar 56e2b956d0 OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.! 2013-08-30 18:21:04 -05:00
cheddar ca2c56d827 1) Remove PostAggregator and put it in druid-api 
2) Remove io.druid files from client
 2013-08-29 16:50:09 -05:00
cheddar cb90ed05b0 Revert the previous commit. After going down this path, I realized that extracting things enough to allow Queries to be extended without depending on Druid proper was going to lead down a very nasty path. So, I've decided against that. Extending queries will require a tight dependency on Druid proper. 2013-08-29 16:45:03 -05:00
cheddar 2a46086e20 1) Didn't remove the io.druid files from client. Remove those and make sure things compile 
2) Switch DefaultObjectMapper to CommonObjectMapper
3) Create new DefaultObjectMapper in client that has Query stuff registered on it by default
 2013-08-29 15:25:36 -05:00
cheddar 1f37e962f6 1) Delete the files that have moved to druid-api and get them through the druid-api dependency. Still need to fix Searchable on StorageAdapter, but other than that, everything should compile. 2013-08-28 16:18:40 -05:00
cheddar bfe96739a3 1) Move various "api" classes to io.druid packages and make sure things compile and stuff -- continued! Come classes didn't get committed? 2013-08-28 15:52:55 -05:00
cheddar 9c30ced5ea 1) Move various "api" classes to io.druid packages and make sure things compile and stuff 2013-08-28 15:51:02 -05:00
cheddar 5fa944dd26 Merge branch 'master' into guice 
Conflicts:
	client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
	client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
	client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
	client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
	indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
	indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
	indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
	pom.xml
	server/src/main/java/com/metamx/druid/http/MasterMain.java
	server/src/main/java/com/metamx/druid/http/MasterServletModule.java
	server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
	server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
	server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
 2013-08-27 14:27:32 -05:00
cheddar 269997dc94 1) ExecutorNode is working, except for the running of the task. Need to adjust it to be able to run a task and then everything will be wonderful 2013-08-26 18:08:41 -05:00
cheddar 55dbda2046 1) Worker appears to be running! It's also now known as the MiddleManager 2013-08-23 17:59:48 -05:00
cheddar b897c2cb22 1) IndexCoordinator appears to work as the CliOverlord now, yay! 2013-08-23 14:11:34 -05:00
Gian Merlino 9b9a2a9f53 Merge branch 'master' into num-shard-spec 
Conflicts:
	common/src/main/java/com/metamx/druid/partition/IntegerPartitionChunk.java
 2013-08-21 15:01:46 -07:00
fjy 54ad854471 Merge branch 'master' into rt-ut 2013-08-21 13:15:02 -07:00
fjy 88661b26a0 bug fix for RTR removing workers race condition and partition chunks not being sorted by chunk number 2013-08-21 11:14:54 -07:00
Gian Merlino 9c5fcc0984 Add missing license headers 2013-08-20 17:48:15 -07:00