Commit Graph

149 Commits

Author SHA1 Message Date
Eric Tschetter 1e0f2c2d92 1) Make log a bit more descriptive 2013-03-13 19:26:22 -05:00
Eric Tschetter fa66e1d2cc 1) SQL is hard 2013-03-13 16:29:34 -05:00
Gian Merlino 3fa46988f5 Realtime:
- MetadataUpdater now built from SegmentAnnouncer, SegmentPublisher instances.
- Sinks can take a version instead of always using interval.start. The realtime
  plumber selects a version using a VersioningPolicy.
- Plumbers gained a startJob method.
- Realtime plumbers gained an implementation for finishJob.
2013-03-11 11:14:51 -07:00
Nelson Ray a470f3fa6a change name of druid.indexer.configTable to druid.database.configTable to be consistent 2013-03-08 10:35:13 -08:00
Eric Tschetter f70f71243d 1) Stop polling in ConfigManager when stop is called
2) Remove WorkSetupManager in favor of just using ConfigManager
2013-03-07 17:45:57 -06:00
Eric Tschetter 1c3ef48f34 1) Adjust the Config stuff that WorkerSetupManager was using to be reusable
2) Use new ConfigManager to pull out a whitelist for the Master
2013-03-07 17:45:57 -06:00
Gian Merlino f862d9205d Merge branch 'master' into hadoop-index-task
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
2013-02-27 06:53:06 -08:00
Gian Merlino 14cf506c43 DefaultObjectMapper: Add GuavaModule 2013-02-25 17:05:01 -08:00
Eric Tschetter f8c54a72c2 1) Changes to allow for local storage 2013-02-21 15:47:01 -06:00
Eric Tschetter dc3459d3f9 1) Initial commit of refactorings on top of housejester's to simplify the zipping and simplify the creation of other methods of loading data 2013-02-21 15:47:01 -06:00
Deep Ganguli 60da9ffddb Merge branch 'master' into hadoop-index-task
Conflicts:
	common/src/main/java/com/metamx/druid/utils/JodaUtils.java
	indexer/src/main/java/com/metamx/druid/indexer/DbUpdaterJob.java
	indexer/src/main/java/com/metamx/druid/indexer/HadoopDruidIndexerConfig.java
	indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
2013-02-15 13:22:31 -08:00
Deep Ganguli e042c29173 Fixed typos in comments, changed prefix on s3Paths to s3n from s3://, cleaned
up umbrellaIntervals code in JodaUtils, modified the DbUpdater to push segment
meta-data to the database in batches.
2013-02-15 11:39:49 -08:00
xvrl 1b85eaad3e simplify query toolchest interface 2013-02-13 21:25:16 -08:00
Eric Tschetter c5b78e75cf 1) Move up to Jackson 2.1
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-13 18:51:32 -08:00
xvrl e10025c841 filter out unneeded segments for TimeBoundary query 2013-02-11 13:21:37 -08:00
Eric Tschetter 6708029043 1) Remove class level comments auto-created by IntelliJ 2013-02-11 13:13:52 -06:00
Jae Hyeon Bae 4f70bbc687 adding linear shard spec and ACL for s3 bucket 2013-02-04 10:27:09 -08:00
Eric Tschetter 7439a2e820 1) Make tests pass 2013-01-28 16:42:51 -06:00
Deep Ganguli fc07bc315e Added umbrellaInterval method, which takes an Iterable of intervals and returns
a single interval spanning the entire range of input intervals.
2013-01-23 18:59:51 -08:00
xvrl 55ae4c87dd timezone support in groupby query 2013-01-23 08:51:02 -08:00
Fangjin Yang ac31afbce5 remove redundant index for primary key in config table 2013-01-18 16:45:23 -08:00
Fangjin Yang 38b2041ad9 key/value config table 2013-01-17 14:56:48 -08:00
Fangjin Yang d1f4317af7 updates to autoscaling config based on code review comments 2013-01-14 14:55:04 -08:00
Fangjin Yang 4c2da93389 bug fixes with ec2 auto scaling 2013-01-09 14:51:35 -08:00
Fangjin Yang 3685839376 Merge branch 'master' into autoscaling
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
2013-01-07 18:04:14 -08:00
Fangjin Yang aff4a05ca3 db based configs for indexer workers 2013-01-07 13:41:26 -08:00
Eric Tschetter b184c46ae0 1) Introduce idea of value types and generic columns to the serialization format
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
2013-01-04 18:12:49 -06:00
xvrl 6525d818ad postAggregators now return field dependencies 2013-01-02 18:36:03 -08:00
xvrl db08640efc reenable daylight saving test, now that we use joda-time 2 2012-12-31 14:10:43 -08:00
Fangjin Yang 436828b67c post aggs verify that they have all the fields they need 2012-12-12 10:36:46 -08:00
Fangjin Yang 6827e48bd3 bug fixes for invalid query params 2012-12-11 15:09:16 -08:00
Fangjin Yang 1ebd7a8a8a fix wrong jquery version 2012-12-06 14:50:29 -08:00
Fangjin Yang 4a1abfc5dc gui for rule configuratin; introducing jquery ui to master console 2012-12-05 15:39:59 -08:00
Fangjin Yang 5822f4f5f7 refactor master to run rules before cleaning up; more master stats; general improvements 2012-12-03 14:43:04 -08:00
Fangjin Yang aa8fb7b9a3 abstract the assign and drop logic; cleanups everywhere; http endpoints for getting and assigning rules 2012-11-30 11:28:10 -08:00
Fangjin Yang 2e5e1ce989 first commit of tiers for compute nodes; working UT at this point 2012-11-28 17:37:08 -08:00
Eric Tschetter 228b223509 1) Fix bug with Sinks that causes Realtime it to overwrite the first chunk of data when creating a segment to hand off to compute nodes.
2) Fix bug with RealtimePlumberSchool that can cause it to miss some previously-persisted segments when directory naming changes
3) Have the various runnables in RealtimePlumberSchool name the threads to make it a bit easier to debug
2012-11-21 18:38:57 -06:00
Eric Tschetter 701cc9562b 1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up.
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29, or at least clarifies the behavior...)
2012-11-19 17:01:17 -06:00
Nelson Ray 217caa3a04 HistogramVisual: put min. and max. in quantile array 2012-11-06 16:55:35 -08:00
Nelson Ray 0a42f18cc3 HistogramVisual: add support for quantiles 2012-11-05 14:33:15 -08:00
Eric Tschetter 88a3fb842c 1) New attempt at making OpenJDK 6 happy 2012-11-02 14:24:43 -05:00
Eric Tschetter 38ed1d8b75 1) Revert failed attempt at making OpenJDK6 happy. 2012-11-02 14:24:09 -05:00
Eric Tschetter 63b2f502ac 1) Make TimelineEntry in VersionedIntervalTimeline a static class with its own generic parameters to hopefully make OpenJDK happier. 2012-11-02 11:35:23 -05:00
Eric Tschetter f07b15db1f Undo massive import re-ordering 2012-11-01 18:38:27 -05:00
Ian Brandt e24948319b Removed some dead/unused code. 2012-10-31 00:00:30 -07:00
Ian Brandt 0d24df7628 Removed unused imports. 2012-10-30 22:22:46 -07:00
xvrl a34c5a100c add buffered version of JavaScriptAggregator 2012-10-29 14:34:17 -07:00
Eric Tschetter 27999caca0 1) Create LICENSE
2) Attach copyright and notice of license to files
2012-10-24 05:09:47 -04:00
Eric Tschetter 9d41599967 Initial commit of OSS Druid Code 2012-10-24 03:39:51 -04:00