fjy
1ec098c010
use num processing threads for realtime query execution
2013-09-13 13:54:41 -07:00
Xavier Léauté
cd14e7fbca
fix regex and assign random user ids
2013-09-05 13:43:25 -07:00
Xavier Léauté
5b6273ac3d
fix null values in wikipedia decoder
2013-08-21 17:56:31 -07:00
Xavier Léauté
71b19af61f
fix missing json property names
2013-08-21 17:38:30 -07:00
cheddar
b4310d6cbf
Merge pull request #219 from metamx/wikipedia-firehose
...
Wikipedia firehose
2013-08-21 14:34:07 -07:00
Xavier Léauté
7913b27066
reuse tmp file for geoip db
2013-08-21 14:12:39 -07:00
Xavier Léauté
0f4a132807
address code review
2013-08-21 13:57:50 -07:00
fjy
ac32c6f27c
fix tests according to code review
2013-08-21 13:23:38 -07:00
fjy
9c145f5ce3
basic unit tets for realtime functinality
2013-08-19 15:00:10 -07:00
Xavier Léauté
071e8cb2f3
updated javadocs
2013-08-15 11:07:55 -07:00
Xavier Léauté
3f33ca15b0
refactor namespaces and add docs / license
2013-08-15 10:58:03 -07:00
Gian Merlino
dacee9ed20
Fix various InputRowParser serde issues
2013-08-14 16:51:02 -07:00
Xavier Léauté
6735ae4ecd
add ip geo lookup
2013-08-14 14:29:44 -07:00
Xavier Léauté
1cc1d0be5f
remove example code
2013-08-13 19:05:56 -07:00
cheddar
c2305a82f9
Merge pull request #209 from zanox/kafka-protobuf
...
Kafka protobuf
2013-08-13 18:38:27 -07:00
Xavier Léauté
8ccac2f13d
basic wikipedia firehose factory
2013-08-13 18:35:07 -07:00
Gian Merlino
1d1cb045f0
RealtimeManager: Handle FormattedExceptions at any stage of processing other than hasMore
2013-08-13 07:23:37 -07:00
Jan Rudert
3f32ef7bbc
Merge branch 'master' of github.com:metamx/druid into kafka-protobuf
2013-08-12 17:42:40 +02:00
Gian Merlino
ec53a30c6e
Rejigger exception handling in standalone realtime
...
- Throw FormattedExceptions on null or unparseable timestamps
- Throw away events (and increment thrownAway) on non-FormattedExceptions in the fire chief
2013-08-06 17:32:08 -07:00
Jan Rudert
73d5ff4b94
Merge branch 'master' of github.com:metamx/druid into HEAD
2013-08-01 18:00:29 +02:00
Jan Rudert
bba83ec532
introduced a ByteBufferInputRowParser to keep the old StringInputRowParser in places outside the KafkaFireHose
2013-07-29 11:59:08 +02:00
cheddar
a6bc63732d
Merge pull request #199 from metamx/indexing-service-mods
...
Improve robustness and usability of RealtimeIndexTask and some code cleanup in the Indexing Service
2013-07-25 17:09:15 -07:00
Gian Merlino
20acd6f450
RealtimePlumberSchool: Persist and merge immediately upon finishJob
2013-07-25 14:38:07 -07:00
fjy
a909a31bcc
better select sinks
2013-07-24 19:25:32 -07:00
Gian Merlino
5c96d500d8
More controllable realtime shutdown
...
- Realtime plumber will start persisting segments shortly after finishJob is called, regardless
of rejection policy
- Add "none" rejection policy
2013-07-24 15:32:10 -07:00
Gian Merlino
0eebb0a149
Add RealtimeMetricsMonitor to RealtimeIndexTask
2013-07-24 14:39:59 -07:00
Gian Merlino
867dedfe1b
Remove unused class
2013-07-24 10:27:21 -07:00
Gian Merlino
71704f7ee8
Replace graceful/hard shutdown combo with simple exit
2013-07-24 10:27:21 -07:00
fjy
fc65648809
fix a bug introduced in postgres support that breaks inserts for mysql
2013-07-23 13:21:14 -07:00
cheddar
1c34e3f364
1) Fix class-level javadoc on RabbitMQFirehoseFactory to reflect changes in config
2013-07-19 16:47:00 -07:00
Jan Rudert
527c6c81c5
refactored StringInputRowParser to handle a ByteBuffer, better wiring and configuration of KafkaFirehose
2013-07-19 14:12:53 +02:00
Stefán Freyr Stefánsson
ade00e441f
Adding minor class-level JavaDoc.
2013-07-19 11:24:13 +00:00
Stefán Freyr Stefánsson
af97b44a6f
merging pull request from cheddar - jacksonifying the RabbitMQ connection and config.
2013-07-19 11:16:50 +00:00
cheddar
0edfbaffab
Errr, needed to make the ConnectionFactory a JacksonifiedConnectionFactory
2013-07-18 13:17:11 -07:00
cheddar
807a52f963
Another take on the configuration
2013-07-18 13:16:42 -07:00
Stefán Freyr Stefánsson
da658ac69a
Changes according to Eric's comments on the pull request.
...
- moved the RabbitMQ ConnectionFactory configuration to a method of its own to clean up the FirehoseFactory.connect() method.
- using Throwables.propagate*() methods in exception handling.
- removed TODOs and instead using the "keyword" FUTURE as well as adding some context to the comments themselves.
- cleaned up the exception handling in hasMore() a little and made it more readable.
2013-07-18 19:30:14 +00:00
Stefán Freyr Stefánsson
22ec7c53fc
Code cleanup and formatting, documentation and moving all configuration into the consumerProps.
2013-07-16 15:08:39 +00:00
Stefán Freyr Stefánsson
82d2623cc9
Minor sanity checks when checking for and processing a new message.
2013-07-15 15:25:59 +00:00
Stefán Freyr Stefánsson
13328a6b36
Implemented a better query.body for the test I'm using. Added a shutdown listener to the connection and channel to get a notification when they are closed. Question remains whether the connection should be re-established when that happens.
2013-07-15 09:50:58 +00:00
Stefán Freyr Stefánsson
4d998e4c27
Making more things configurable through the realtime.spec file.
2013-07-13 16:49:39 +00:00
Stefán Freyr Stefánsson
01b7507a5a
First version of a RabbitMQ Firehose implementation.
2013-07-13 16:03:46 +00:00
Jan Rudert
a8039107d8
no bytebuffer in the middle
2013-07-12 16:02:15 +02:00
Jan Rudert
89b0c84f3b
initial implementation of an protocol buffers firehose
2013-07-12 13:34:57 +02:00
cheddar
b42c5743c9
Merge pull request #177 from nebrera/master
...
Postgresql adaptation
2013-07-05 16:42:57 -07:00
cheddar
0712941825
1) Add check whether a Hydrant has already been persisted before persisting. Persisting happens synchronously on the same thread, but multiple persist requests can be queued up on that thread which means that subsequent ones would fail with an NPE. Fixes #178
2013-07-02 11:53:53 -07:00
Pablo Nebrera
e9ce6b596c
PostgresSQL integration with druid
2013-06-28 12:49:24 +02:00
fjy
15a4dae2b1
bunch of issue fixes
2013-06-26 10:45:54 -07:00
cheddar
9d80369c2b
Merge branch 'master' of github.com:metamx/druid
2013-06-19 11:16:13 -07:00
cheddar
369412262e
1) Fix truncation math when figuring out what Segments to persist for realtime plumber. Previously, Sinks could get persist-n-merged without actually being ready to be persist-n-merged. Fixes #168
2013-06-19 11:15:58 -07:00
kOD-dKOD
270a4993ce
Update RealtimePlumberSchool.java
...
Put a check to make sure that we're only looking for the folder those have number-only names, to avoid "merged" directory issue.
2013-06-11 12:05:17 +05:30
