3c39f90c89
1) Move Firehose interface and dependencies to druid-api
...
2) Move DataSegment* interfaces and dependencies to druid-api
2013-08-31 16:43:28 -05:00
5ab671050e
No more com.metamx.druid, it is now all io.druid!
2013-08-30 19:42:12 -05:00
bd0756e360
More stuff moved, things still compiling and tests still passing. Yay!
2013-08-30 18:58:35 -05:00
56e2b956d0
OMG!!! A lot of stuff has been moved. Modules have been created and destroyed, but everything is compiling and unit tests are passing, OMFG this is awesome.!
2013-08-30 18:21:04 -05:00
cb90ed05b0
Revert the previous commit. After going down this path, I realized that extracting things enough to allow Queries to be extended without depending on Druid proper was going to lead down a very nasty path. So, I've decided against that. Extending queries will require a tight dependency on Druid proper.
2013-08-29 16:45:03 -05:00
2a46086e20
1) Didn't remove the io.druid files from client. Remove those and make sure things compile
...
2) Switch DefaultObjectMapper to CommonObjectMapper
3) Create new DefaultObjectMapper in client that has Query stuff registered on it by default
2013-08-29 15:25:36 -05:00
9c30ced5ea
1) Move various "api" classes to io.druid packages and make sure things compile and stuff
2013-08-28 15:51:02 -05:00
ee1e73cfa1
1) Make it compile again after the merge
2013-08-27 14:36:01 -05:00
8097450d8c
Some things that didn't get committed with the merge for some reason!?
2013-08-27 14:29:03 -05:00
5fa944dd26
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/coordination/BatchDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/curator/announcement/Announcer.java
client/src/main/java/com/metamx/druid/query/filter/SelectorDimFilter.java
client/src/main/java/com/metamx/druid/query/search/SearchQueryQueryToolChest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/tasklogs/S3TaskLogs.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/ForkingTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunner.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/WorkerCuratorCoordinator.java
indexing-service/src/test/java/com/metamx/druid/indexing/coordinator/RemoteTaskRunnerTest.java
pom.xml
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/http/MasterServletModule.java
server/src/main/java/com/metamx/druid/master/DruidMasterConfig.java
server/src/test/java/com/metamx/druid/master/DruidMasterTest.java
server/src/test/java/com/metamx/druid/query/group/GroupByQueryRunnerTest.java
2013-08-27 14:27:32 -05:00
b897c2cb22
1) IndexCoordinator appears to work as the CliOverlord now, yay!
2013-08-23 14:11:34 -05:00
261ef7ce56
add some fixes
2013-08-22 10:56:50 -07:00
6a8c160740
update code according to code review
2013-08-22 10:46:05 -07:00
5b6273ac3d
fix null values in wikipedia decoder
2013-08-21 17:56:31 -07:00
71b19af61f
fix missing json property names
2013-08-21 17:38:30 -07:00
b4310d6cbf
Merge pull request #219 from metamx/wikipedia-firehose
...
Wikipedia firehose
2013-08-21 14:34:07 -07:00
7913b27066
reuse tmp file for geoip db
2013-08-21 14:12:39 -07:00
0f4a132807
address code review
2013-08-21 13:57:50 -07:00
ac32c6f27c
fix tests according to code review
2013-08-21 13:23:38 -07:00
9c145f5ce3
basic unit tets for realtime functinality
2013-08-19 15:00:10 -07:00
071e8cb2f3
updated javadocs
2013-08-15 11:07:55 -07:00
3f33ca15b0
refactor namespaces and add docs / license
2013-08-15 10:58:03 -07:00
dacee9ed20
Fix various InputRowParser serde issues
2013-08-14 16:51:02 -07:00
6735ae4ecd
add ip geo lookup
2013-08-14 14:29:44 -07:00
1cc1d0be5f
remove example code
2013-08-13 19:05:56 -07:00
c2305a82f9
Merge pull request #209 from zanox/kafka-protobuf
...
Kafka protobuf
2013-08-13 18:38:27 -07:00
8ccac2f13d
basic wikipedia firehose factory
2013-08-13 18:35:07 -07:00
85ee8bb267
port realtime to guice
2013-08-13 17:08:45 -07:00
1d1cb045f0
RealtimeManager: Handle FormattedExceptions at any stage of processing other than hasMore
2013-08-13 07:23:37 -07:00
3f32ef7bbc
Merge branch 'master' of github.com:metamx/druid into kafka-protobuf
2013-08-12 17:42:40 +02:00
ec53a30c6e
Rejigger exception handling in standalone realtime
...
- Throw FormattedExceptions on null or unparseable timestamps
- Throw away events (and increment thrownAway) on non-FormattedExceptions in the fire chief
2013-08-06 17:32:08 -07:00
2361e0112a
Make it all compile again...
2013-08-02 10:14:46 -07:00
9e78bb38f5
Merge branch 'master' into guice
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
client/src/main/java/com/metamx/druid/client/ServerInventoryView.java
client/src/main/java/com/metamx/druid/coordination/SingleDataSegmentAnnouncer.java
client/src/main/java/com/metamx/druid/initialization/CuratorDiscoveryConfig.java
client/src/main/java/com/metamx/druid/query/MetricsEmittingExecutorService.java
indexing-hadoop/src/test/java/com/metamx/druid/indexer/HadoopDruidIndexerConfigTest.java
indexing-service/src/main/java/com/metamx/druid/indexing/common/TaskToolbox.java
indexing-service/src/main/java/com/metamx/druid/indexing/coordinator/http/IndexerCoordinatorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/executor/ExecutorNode.java
indexing-service/src/main/java/com/metamx/druid/indexing/worker/http/WorkerNode.java
pom.xml
server/src/main/java/com/metamx/druid/coordination/ServerManager.java
server/src/main/java/com/metamx/druid/coordination/ZkCoordinator.java
server/src/main/java/com/metamx/druid/db/DatabaseRuleManager.java
server/src/main/java/com/metamx/druid/db/DatabaseSegmentManager.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
server/src/main/java/com/metamx/druid/http/MasterMain.java
server/src/main/java/com/metamx/druid/loading/SegmentLoaderConfig.java
server/src/main/java/com/metamx/druid/loading/SingleSegmentLoader.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-08-01 16:42:47 -07:00
73d5ff4b94
Merge branch 'master' of github.com:metamx/druid into HEAD
2013-08-01 18:00:29 +02:00
bba83ec532
introduced a ByteBufferInputRowParser to keep the old StringInputRowParser in places outside the KafkaFireHose
2013-07-29 11:59:08 +02:00
a6bc63732d
Merge pull request #199 from metamx/indexing-service-mods
...
Improve robustness and usability of RealtimeIndexTask and some code cleanup in the Indexing Service
2013-07-25 17:09:15 -07:00
20acd6f450
RealtimePlumberSchool: Persist and merge immediately upon finishJob
2013-07-25 14:38:07 -07:00
a909a31bcc
better select sinks
2013-07-24 19:25:32 -07:00
5c96d500d8
More controllable realtime shutdown
...
- Realtime plumber will start persisting segments shortly after finishJob is called, regardless
of rejection policy
- Add "none" rejection policy
2013-07-24 15:32:10 -07:00
0eebb0a149
Add RealtimeMetricsMonitor to RealtimeIndexTask
2013-07-24 14:39:59 -07:00
867dedfe1b
Remove unused class
2013-07-24 10:27:21 -07:00
71704f7ee8
Replace graceful/hard shutdown combo with simple exit
2013-07-24 10:27:21 -07:00
fc65648809
fix a bug introduced in postgres support that breaks inserts for mysql
2013-07-23 13:21:14 -07:00
1c34e3f364
1) Fix class-level javadoc on RabbitMQFirehoseFactory to reflect changes in config
2013-07-19 16:47:00 -07:00
527c6c81c5
refactored StringInputRowParser to handle a ByteBuffer, better wiring and configuration of KafkaFirehose
2013-07-19 14:12:53 +02:00
ade00e441f
Adding minor class-level JavaDoc.
2013-07-19 11:24:13 +00:00
af97b44a6f
merging pull request from cheddar - jacksonifying the RabbitMQ connection and config.
2013-07-19 11:16:50 +00:00
0edfbaffab
Errr, needed to make the ConnectionFactory a JacksonifiedConnectionFactory
2013-07-18 13:17:11 -07:00
807a52f963
Another take on the configuration
2013-07-18 13:16:42 -07:00
da658ac69a
Changes according to Eric's comments on the pull request.
...
- moved the RabbitMQ ConnectionFactory configuration to a method of its own to clean up the FirehoseFactory.connect() method.
- using Throwables.propagate*() methods in exception handling.
- removed TODOs and instead using the "keyword" FUTURE as well as adding some context to the comments themselves.
- cleaned up the exception handling in hasMore() a little and made it more readable.
2013-07-18 19:30:14 +00:00
cheddar
fjy
Xavier Léauté
cheddar
Gian Merlino
Jan Rudert
Stefán Freyr Stefánsson