Gian Merlino
d8fbddb9d4
Merger: Service discovery for worker -> master communication
2013-02-26 11:36:07 -08:00
Gian Merlino
2427e81874
Merger: Feedback from code review
2013-02-25 17:27:02 -08:00
Gian Merlino
14cf506c43
DefaultObjectMapper: Add GuavaModule
2013-02-25 17:05:01 -08:00
Gian Merlino
3129e5451b
Merge branch 'master' into task-stuff
2013-02-25 12:33:45 -08:00
Gian Merlino
9964e7dfe4
[maven-release-plugin] prepare for next development iteration
2013-02-25 12:24:24 -08:00
Gian Merlino
99c4f9446c
[maven-release-plugin] prepare release druid-0.3.3
2013-02-25 12:24:15 -08:00
gianm
4811b50055
Merge pull request #94 from metamx/fix-dpj
...
Fix DeterminePartitionsJob ISE for dimensions not present in all rows
2013-02-25 12:21:48 -08:00
Gian Merlino
49700dffe9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2013-02-25 12:18:31 -08:00
Gian Merlino
7d7ce2b7fe
Fix DeterminePartitionsJob ISE for dimensions not present in all rows
2013-02-25 11:22:25 -08:00
Eric Tschetter
8513a5ab2a
1) Fix SimpleColumn to not produce NPEs when one of its parts is null.
2013-02-24 22:18:44 -08:00
Eric Tschetter
e0f6df1a5c
[maven-release-plugin] prepare for next development iteration
2013-02-21 20:39:50 -06:00
Eric Tschetter
846bc0e4f0
[maven-release-plugin] prepare release druid-0.3.2
2013-02-21 20:39:45 -06:00
Eric Tschetter
5e123988a0
1) Specify old Jackson version to resolve version conflicts between AWS SDK and curator
2013-02-21 20:37:51 -06:00
Eric Tschetter
699aadd2b1
[maven-release-plugin] prepare for next development iteration
2013-02-21 15:50:01 -06:00
Eric Tschetter
606a2e4b01
[maven-release-plugin] prepare release druid-0.3.1
2013-02-21 15:49:55 -06:00
Eric Tschetter
f8c54a72c2
1) Changes to allow for local storage
2013-02-21 15:47:01 -06:00
Eric Tschetter
dc3459d3f9
1) Initial commit of refactorings on top of housejester's to simplify the zipping and simplify the creation of other methods of loading data
2013-02-21 15:47:01 -06:00
James Estes
d1626576c0
Working toward making it easier to add new SegmentPullers.
...
1) Move the local cacheFile logic out of the S3 pullers into the SingleSegmentLoader
2) Make the S3SegmentPuller just pull down the file
3) Make the Loader do the unzip, ungzip, or rename
4) 2 and 3 make S3ZippedSegmentPuller not necessary (still there, just deprecated and empty)
4) Tweak the TaskToolbox so that the Pullers returned by getSegmentGetters behave the same as they did before
2013-02-21 15:47:01 -06:00
Gian Merlino
824e3c0eb2
[maven-release-plugin] prepare for next development iteration
2013-02-15 13:03:39 -08:00
Gian Merlino
6bbc992101
[maven-release-plugin] prepare release druid-0.3.0
2013-02-15 13:03:32 -08:00
Eric Tschetter
0d99cee3c3
1) Whitespace ftw!
2013-02-15 14:25:44 -06:00
cheddar
06ef83db69
Merge pull request #85 from metamx/speedup-timeboundary
...
Avoid hitting all segments for TimeBoundary queries
2013-02-15 11:43:42 -08:00
xvrl
05d347c304
Merge remote-tracking branch 'origin/master' into speedup-timeboundary
...
Conflicts:
client/pom.xml
common/pom.xml
druid-services/pom.xml
examples/pom.xml
examples/rand/pom.xml
examples/twitter/pom.xml
index-common/pom.xml
indexer/pom.xml
merger/pom.xml
pom.xml
realtime/pom.xml
server/pom.xml
2013-02-15 11:25:47 -08:00
xvrl
780410446c
even more simple
2013-02-15 11:19:13 -08:00
Eric Tschetter
5ee7c0c264
1) Remove stray import of java.util.List
2013-02-15 12:15:53 -06:00
cheddar
e1533b871e
Merge pull request #88 from metamx/8to9-empty-column
...
Additional fix for columns with cardinality 0
2013-02-15 10:15:05 -08:00
Gian Merlino
548c901c0b
Additional fix for columns with cardinality 0
2013-02-14 23:45:02 -08:00
Gian Merlino
3fed8ee0ce
[maven-release-plugin] prepare for next development iteration
2013-02-14 21:23:46 -08:00
Gian Merlino
ba7df5b709
[maven-release-plugin] prepare release druid-0.2.9
2013-02-14 21:23:41 -08:00
Gian Merlino
22d1f2a0c0
IndexMergerTest: Empty column test
2013-02-14 21:11:31 -08:00
Eric Tschetter
53e801ecb3
Merge branch 'master' of github.com:metamx/druid
2013-02-14 22:59:43 -06:00
Eric Tschetter
48538d0454
1) Fix bug in index converter when column has cardinality 0
2013-02-14 22:59:17 -06:00
Gian Merlino
152bf201ec
[maven-release-plugin] prepare for next development iteration
2013-02-14 16:27:20 -08:00
Gian Merlino
e56bc27f93
[maven-release-plugin] prepare release druid-0.2.8
2013-02-14 16:27:12 -08:00
Eric Tschetter
4f11eb5209
1) Fix alert sent from line DruidMasterBalancer:71 to not include all of the segments that a server is holding
2013-02-14 18:24:51 -06:00
Gian Merlino
dd44f5cbf0
Merge branch 'master' into task-stuff
2013-02-14 15:11:40 -08:00
Eric Tschetter
ee4236dac2
1) Update to mmx build of jackson 2.1 branch to get bug-fix for jackson-databind #167
2013-02-14 17:10:27 -06:00
Gian Merlino
ad1de9ba9d
Merge branch 'master' into task-stuff
2013-02-14 14:16:46 -08:00
Fangjin Yang
77fbe974d8
fix version conflicts of old jackson
2013-02-14 13:52:19 -08:00
Gian Merlino
de4fc98128
TaskMasterLifecycle: Stop leaderLifecycle if it dies while starting
2013-02-14 13:36:43 -08:00
Gian Merlino
18daa19fc9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskHolder.java
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/AbstractTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskContext.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-14 13:10:29 -08:00
Gian Merlino
ff325b1f65
Merger: Changes to support rolling locks and a task-action API
...
Generally, TaskStatus is no longer used as the sole way for tasks to
communicate status and metadata update requests back to the
coordinator. It is still used to communicate success or failure, but a
TaskActionClient is used for all other communication. In remote mode
this uses an http api.
TaskContext no longer exists, as its purpose is now served by
the TaskActionClient.
The TaskQueue and TaskConsumer are no longer in charge of committing
stuff for a task. Instead, it happens when an action is sent in. Tasks
are expected to ensure that this is OK by making their behavior
idempotent (since they may be retried after partially completing).
One new kind of request afforded to tasks is the ability to release
and acquire arbitrary locks. This will be useful for long-running tasks.
Short-lived tasks can still be given a lock automatically upon startup
by providing a "fixed interval".
New tables when using db storage:
- tasklocks, to support persistence of arbitrary locks.
- tasklogs, to log actions taken by tasks
2013-02-14 12:46:40 -08:00
xvrl
6998d604a2
fixes #80
2013-02-14 12:41:34 -08:00
xvrl
98de0382d5
Merge branch 'master' into speedup-timeboundary
2013-02-13 21:27:27 -08:00
xvrl
1b85eaad3e
simplify query toolchest interface
2013-02-13 21:25:16 -08:00
Gian Merlino
e3822f6ca7
DeterminePartitionsJob: Fix docs
2013-02-13 18:51:33 -08:00
Gian Merlino
ee193f0ca7
DeterminePartitionsJob: Select partition dimension to minimize segment size variance when cardinality is low
2013-02-13 18:51:32 -08:00
Eric Tschetter
c5b78e75cf
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-13 18:51:32 -08:00
Gian Merlino
ce1d90788d
DeterminePartitionsJob: Fix docs
2013-02-13 10:16:03 -08:00
cheddar
0eb65f69d4
Merge pull request #86 from metamx/dimpartition-by-variance
...
Select partition dimension by variance when cardinality is low
2013-02-13 09:54:34 -08:00
