40 Commits

Author SHA1 Message Date
Gian Merlino
8948c8d4ca Merge branch 'master' into event-push
Conflicts:
	client/src/main/java/com/metamx/druid/initialization/Initialization.java
	examples/src/main/java/druid/examples/twitter/TwitterSpritzerFirehoseFactory.java
	index-common/src/main/java/com/metamx/druid/indexer/data/StringInputRowParser.java
	merger/src/main/java/com/metamx/druid/merger/common/task/RealtimeIndexTask.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/executor/ExecutorNode.java
2013-05-02 16:34:14 +03:00
cheddar
0a3492c9f5 Move all the code for examples under the examples directory. Also add initial take on flights demo. 2013-04-30 18:27:38 -05:00
cheddar
9153cc94ac 1) Nicer logging
2) Announcer should actually delete its zk entries when it is told to unannounce something
3) Skip over complex columns that we don't have a proper deserializer for
2013-04-26 19:19:55 -05:00
Gian Merlino
341ee27419 Changes to allow pushing events directly to realtime tasks, and the creation
of single-segment realtime tasks.

Realtime:
- Move firehose stuff to com.metamx.druid.realtime.firehose package
- ClippedFirehoseFactory provides a time-clipped view
- TimedShutoffFirehoseFactory shuts off at a particular time

Indexer:
- Split StringInputRowParser into {String,Map}InputRowParser

Merger:
- Remove minTime in favor of clipped firehoses
- Add task executor servlet that can push events to EventReceivers
- Add EventReceivingFirehose that accepts events as an EventReceiver
2013-04-25 14:06:46 +03:00
xvrl
2afe2bdeae formatting 2013-04-16 09:33:50 -07:00
xvrl
c83631928c object column selectors 2013-04-16 09:33:50 -07:00
xvrl
0c68bd1b1d rename MetricSelectorFactory to ColumnSelectorFactory 2013-04-16 09:33:49 -07:00
René Scheibe
6c27cd2e8e cleanup - removed unused imports 2013-04-07 23:31:22 +02:00
René Scheibe
a6dc29cf85 cleanup - removed extra empty lines
- there should not be any adjacent empty lines
2013-04-07 14:32:09 +02:00
Eric Tschetter
a933438e4e 1) Fix bugs with VersionConverterTask
2) Fix bugs with NPEs on indexing
2013-03-15 13:48:55 -05:00
Eric Tschetter
a11a34f87e 1) Initial commit: Converter Task 2013-03-07 17:44:05 -06:00
Eric Tschetter
8513a5ab2a 1) Fix SimpleColumn to not produce NPEs when one of its parts is null. 2013-02-24 22:18:44 -08:00
Gian Merlino
548c901c0b Additional fix for columns with cardinality 0 2013-02-14 23:45:02 -08:00
Eric Tschetter
48538d0454 1) Fix bug in index converter when column has cardinality 0 2013-02-14 22:59:17 -06:00
Eric Tschetter
a0b159fed7 1) Move up to Jackson 2.1
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-12 20:58:17 -06:00
Eric Tschetter
34b4383934 1) Adjust DataSegment to have a binaryVersion field that indicates the expected binary version of the segment files
2) Eliminate race condition from RemoteTaskRunnerTest.testAlreadyExecutedTask()
2013-02-01 18:59:33 -06:00
Eric Tschetter
8af3ae09d6 1) Fix bug with loading old indexes that might have mixed case column names 2013-01-29 11:38:35 -06:00
Eric Tschetter
f2cce28297 1) Make SegmentMetadataQuery work 2013-01-29 11:29:26 -06:00
Gian Merlino
77a3f3cbe0 Merge branch 'master' into determine-partitions
Conflicts:
	indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
2013-01-21 14:46:13 -08:00
Eric Tschetter
5b1e03530c 1) Fix some bugs found by external test suite 2013-01-16 21:06:57 -06:00
Eric Tschetter
c8cb96b006 1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir()
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
2013-01-16 17:10:33 -06:00
Eric Tschetter
8b31d8db9f 1) Adjust IndexMerger to create convert the indexes it creates from the old format to the new. This is done quite sub-optimally, but it will work for now... 2013-01-16 10:01:46 -06:00
Gian Merlino
86277d1114 StringInputRowParser: Treat dimensionExclusions case-insensitively 2013-01-15 22:30:17 -08:00
Eric Tschetter
b184c46ae0 1) Introduce idea of value types and generic columns to the serialization format
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
2013-01-04 18:12:49 -06:00
Eric Tschetter
06b8e4230c 1) Fix PostAggregations with GroupBy (Fixes #24)
2) Add GroupByTimeseriesQueryRunnerTest which wraps a GroupByQueryRunner to look like a TimeseriesQueryRunner and leverages the tests there to help verify behavior (#8)
2012-11-21 12:51:43 -06:00
Eric Tschetter
701cc9562b 1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up.
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29, or at least clarifies the behavior...)
2012-11-19 17:01:17 -06:00
Fangjin Yang
57468d39ef reverting some of the last changes 2012-11-12 16:14:48 -08:00
Fangjin Yang
c20dccd0f4 modifying the way registering serdes works to hopefully be a bit easier to use 2012-11-12 13:58:43 -08:00
Eric Tschetter
0a0e2a6cc1 1) Try to fix the dependency issues for running the HadoopDruidIndexer locally. 2012-11-08 17:06:02 -08:00
Fangjin Yang
34cb352cf8 working indexer with registererers 2012-11-06 14:26:53 -08:00
Fangjin Yang
68e5adde33 register registererers in the config 2012-11-06 11:49:17 -08:00
Fangjin Yang
9fbee29eb4 change hadoop indexer to be node based 2012-11-05 18:19:04 -08:00
Fangjin Yang
51cd361fbe some changes to how handlers are registered 2012-11-05 13:39:16 -08:00
Fangjin Yang
b1b611735e Merge branch 'master' into autoscaling
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-02 15:57:16 -07:00
Eric Tschetter
f07b15db1f Undo massive import re-ordering 2012-11-01 18:38:27 -05:00
Eric Tschetter
8dd6f5c059 1) Refactor the BrokerMain into a BrokerMain and a BrokerNode to allow for extensions
2) Rename ServerMain to ComputeMain
3) Refactor ComputeMain to ComputeMain and ComputeNode to allow for extensions
4) Remove "TheSizeAdjuster", was old legacy stuff that's not relevant anymore
5) Fix bug with registering different IndexIO handlers
6) Adjust it so that when a query type is unknown, it returns an error message instead of just emitting an alert and returning nothing
7) Create super classes for the various *Node classes that allow for code sharing on pieces that are common
2012-11-01 18:20:38 -05:00
Fangjin Yang
d0f66c1fb3 bug fix for merging with empty strings 2012-10-31 10:28:24 -07:00
Ian Brandt
0d24df7628 Removed unused imports. 2012-10-30 22:22:46 -07:00
Eric Tschetter
27999caca0 1) Create LICENSE
2) Attach copyright and notice of license to files
2012-10-24 05:09:47 -04:00
Eric Tschetter
9d41599967 Initial commit of OSS Druid Code 2012-10-24 03:39:51 -04:00