6438401a32
split pick segment logic into its own method
2013-01-28 15:06:09 -08:00
1bcf443b49
Merge pull request #62 from metamx/groupby-tests
...
Actual groupby tests (tests fail because of group by bug)
2013-01-28 14:45:00 -08:00
673cffc3f2
Merge branch 'groupby-tests' of github.com:metamx/druid into groupby-tests
...
Conflicts:
common/src/main/java/com/metamx/druid/input/MapBasedRow.java
2013-01-28 16:44:18 -06:00
7439a2e820
1) Make tests pass
2013-01-28 16:42:51 -06:00
c0822325e7
proper groupby tests
2013-01-28 16:41:03 -06:00
5f18f368e3
Merge pull request #61 from metamx/fix-cache-timezone
...
Cache did not preserve timezone information
2013-01-28 14:39:00 -08:00
59dd9a8e18
Merge pull request #66 from metamx/queryservlet-alert-remote-addr
...
QueryServlet: Add "host" (remote address) to alerts
2013-01-28 14:38:29 -08:00
74057600f9
fix worker node still using db connection
2013-01-27 09:57:37 -08:00
4a1f5315a6
1) Make tests pass
2013-01-25 18:15:51 -06:00
1cff766e2a
RemoteTaskRunner: Only cleanup paths during bootstrapping if task is complete
2013-01-25 16:01:17 -08:00
efdff7b9f1
fix unit test bug
2013-01-25 14:33:05 -08:00
0f4746450b
RemoteTaskRunner: null check on workerData
2013-01-25 13:40:20 -08:00
bb68091cef
Index service: TaskConsumer tests
2013-01-25 13:15:55 -08:00
82d77a8b72
Index service: Fix kill task
2013-01-25 13:15:49 -08:00
553738e1d8
Merge branch 'master' into task-stuff
2013-01-25 11:34:18 -08:00
f3b04d3f5f
S3SegmentKiller: Add TODO note
2013-01-25 11:33:50 -08:00
e6a618ca76
Merge branch 'killsegments' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/LocalTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/exec/TaskConsumer.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
2013-01-25 11:30:10 -08:00
a14200d779
Index service support for early returns and choice of commit semantics.
...
Task:
- Add TaskCallback to run method (for early returns)
TaskStatus:
- Remove CONTINUED status
- Add segmentsNuked (placeholder for future deletion support)
- Add more builder methods
- Add validations to constructor
TaskStorage:
- Add TaskStorageQueryAdapter, a concrete class that wraps TaskStorages and
provides various read-only convenience methods
- Add getTask method for benefit of TaskStorageQueryAdapter
TaskQueue:
- Rename "done" to "notify"
- notify is responsible for deciding if we should commit
- Add optional commitRunnable to "notify", which gets called when it's time to commit
- Allow nextTasks and commits to run early (statusCode RUNNING)
- Move getStatus, collapseStatus functionality to TaskStorageQueryAdapter
2013-01-25 11:05:34 -08:00
37417cf22f
QueryServlet: Add "host" (remote address) to alerts
2013-01-25 11:03:44 -08:00
5e1cac6d9f
timezone test
2013-01-24 17:57:10 -08:00
cb845e6f09
Addresed Gian's code review
2013-01-24 17:54:06 -08:00
7f410f201d
updating amazon sdk version
2013-01-24 17:47:10 -08:00
ee7337fbb9
1) Adjust the Timeseries caching fixes to still store the long, but do the timezone adjustment on the way out.
...
2) Store a reference to the granularity object instead of getting it every time
2013-01-24 18:25:21 -06:00
ec034ddef4
Merge pull request #56 from metamx/determine-partitions
...
Determine partitions better
2013-01-24 12:57:39 -08:00
ad6f962000
Merge pull request #58 from metamx/cache-bulkget
...
modify cacheBroker interface to implement bulk-get
2013-01-24 09:57:43 -08:00
017d4779d6
Implemented Hadoop Index Task which takes as input a HadoopDruidIndexConfig and
...
generates index segments.
The HadoopIndexTask run method wraps a HadoopDruidIndexerJob run method. The
key modifications to the HadoopDruidIndexerJob are as follows:
- The UpDaterJobSpec field of the config that is used to set up the indexer job
is set to null. This ensures that the job does not push a list of published
segments to the database, in order to allow the indexing service to handle this
later.
- Set the version field of the config file based on the TaskContext. Also
changed config.setVersion method to take a string (as opposed to a Date) as
input, and propogated this change where necessary.
- Set the SegmentOutputDir field of the config file based on the TaskToolbox,
to allow the indexing service to handle where to write the segments too.
- Added a method to IndexGeneratorJob called getPublishedSegments, that simply
returns a list of published segments without publishing this list to the
database.
2013-01-23 19:27:14 -08:00
fc07bc315e
Added umbrellaInterval method, which takes an Iterable of intervals and returns
...
a single interval spanning the entire range of input intervals.
2013-01-23 18:59:51 -08:00
9b6244ec15
Merge branch 'master' of github.com:metamx/druid
2013-01-23 18:37:03 -06:00
67feee3cd6
1) Indexes don't always have an index.drd file anymore
2013-01-23 18:36:52 -06:00
55ae4c87dd
timezone support in groupby query
2013-01-23 08:51:02 -08:00
35058786d9
match query interval to granularity for this test
2013-01-23 08:50:43 -08:00
86a6d112e3
proper groupby tests
2013-01-22 16:54:14 -08:00
272d737517
cleaning up some interactions with RTR and workers
2013-01-22 16:21:38 -08:00
d7ea8e9afc
compare result timestamp based on millis + utcoffset
2013-01-21 17:01:41 -08:00
8f38b775ae
fix expected object type
2013-01-21 16:31:32 -08:00
f05c050c53
add test for timezone
2013-01-21 15:49:39 -08:00
94b72e8878
replace param BalancerCostAnalyzer getter with a factory
2013-01-21 15:32:29 -08:00
bab9ee8827
Merge branch 'master' into killsegments
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
2013-01-21 14:47:49 -08:00
77a3f3cbe0
Merge branch 'master' into determine-partitions
...
Conflicts:
indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
2013-01-21 14:46:13 -08:00
d9e6f1d954
DeterminePartitions follow-up
...
HadoopDruidIndexerConfig:
- Add partitionsSpec (backwards compatible with targetPartitionSize and partitionDimension)
- Add assumeGrouped flag to partitionsSpec
DeterminePartitionsJob:
- Skip group-by job if assumeGrouped is set
- Clean up code a bit
2013-01-21 14:38:35 -08:00
068ca67dba
fix cache not preserving timezone information
2013-01-21 14:38:04 -08:00
2d7113b263
huge simplification of balancing code
2013-01-21 14:28:25 -08:00
40c0bcad29
simplify MapCache
2013-01-18 18:25:51 -08:00
07131c51ed
Merge pull request #53 from metamx/autoscaling
...
Use a database to store configurations for indexer workers
2013-01-18 17:08:52 -08:00
ac31afbce5
remove redundant index for primary key in config table
2013-01-18 16:45:23 -08:00
8a8b8a59f9
Merge pull request #57 from metamx/yops-descriptor-dimensions
...
YeOldePlumberSchool: Populate dimension metadata in segment descriptor
2013-01-18 16:42:02 -08:00
e0c34c3b97
rename cacheBroker -> cache
2013-01-18 15:22:56 -08:00
86ca8967ca
rework code pulling from cache to be more readable
2013-01-18 15:17:10 -08:00
a70ae15585
replace Pair<String, ByteBuffer> with NamedKey
2013-01-18 15:17:05 -08:00
9032ef521b
fix interrupted thread
2013-01-18 15:17:05 -08:00
Nelson Ray
cheddar
Eric Tschetter
xvrl
Fangjin Yang
Gian Merlino
Deep Ganguli