2a94bd508d
Merge pull request #104 from metamx/javascript-dimfilters
...
support for Javascript DimFilters
2013-03-14 11:47:55 -07:00
4c165b4880
1) Better logging of master doing version checking
...
2) Exception out when the scv cannot find its indexer
2013-03-13 19:15:29 -05:00
d1161ae0ec
1) Master should create the config table if it's not there.
2013-03-08 12:41:46 -06:00
7a57609c5b
emit metrics corresponding to cost-based balancing
2013-03-07 16:31:57 -08:00
9f405c0b1e
fix constant segment balancing and separate out the differences between segment assignment and balancing
2013-03-07 16:31:57 -08:00
1c3ef48f34
1) Adjust the Config stuff that WorkerSetupManager was using to be reusable
...
2) Use new ConfigManager to pull out a whitelist for the Master
2013-03-07 17:45:57 -06:00
6864007c05
1) Remove Task parameter from other methods off of TaskToolbox that required it
...
2) Move getTaskDir() off of TaskConfig and onto TaskToolbox
3) Rename SegmentKiller interface to DataSegmentKiller
4) Change signature of "DataSegmentKiller.kill(Collection<DataSegment>) throws ServiceException" to just kill(DataSegment) throws SegmentLoadingException
5) Add various log messages
6) Update the version of the segment that has been converted
2013-03-07 17:44:06 -06:00
c9b411c0ca
1) Remove the need for TaskActions to require a Task as a constructor parameter
2013-03-07 17:44:05 -06:00
fd0fcc7319
Whitespace!
2013-03-07 16:18:31 -06:00
82aa6ed7dd
1) Adjust the SingleSegmentLoader to be able to use cache locations from previous versions as well without re-downloading
2013-03-07 15:55:23 -06:00
9d9efadf72
working indexer console
2013-03-06 10:38:40 -08:00
4cb3ca00bc
support for javascript dimfilters
2013-03-06 09:59:53 -08:00
92ab51e70c
fix bugs in balancing logic and removing auto-scaled node
2013-03-04 13:03:15 -08:00
f862d9205d
Merge branch 'master' into hadoop-index-task
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
2013-02-27 06:53:06 -08:00
d1634fe5ef
fix cost balancing so assignment/balancing doesn't get stuck
2013-02-26 18:58:16 -08:00
e67a8757ba
Merge remote-tracking branch 'origin/master' into new_balancer
...
Conflicts:
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-26 14:18:07 -08:00
d32a6284ce
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/worker/config/WorkerConfig.java
2013-02-26 13:03:56 -08:00
49700dffe9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2013-02-25 12:18:31 -08:00
f8c54a72c2
1) Changes to allow for local storage
2013-02-21 15:47:01 -06:00
dc3459d3f9
1) Initial commit of refactorings on top of housejester's to simplify the zipping and simplify the creation of other methods of loading data
2013-02-21 15:47:01 -06:00
d1626576c0
Working toward making it easier to add new SegmentPullers.
...
1) Move the local cacheFile logic out of the S3 pullers into the SingleSegmentLoader
2) Make the S3SegmentPuller just pull down the file
3) Make the Loader do the unzip, ungzip, or rename
4) 2 and 3 make S3ZippedSegmentPuller not necessary (still there, just deprecated and empty)
4) Tweak the TaskToolbox so that the Pullers returned by getSegmentGetters behave the same as they did before
2013-02-21 15:47:01 -06:00
60da9ffddb
Merge branch 'master' into hadoop-index-task
...
Conflicts:
common/src/main/java/com/metamx/druid/utils/JodaUtils.java
indexer/src/main/java/com/metamx/druid/indexer/DbUpdaterJob.java
indexer/src/main/java/com/metamx/druid/indexer/HadoopDruidIndexerConfig.java
indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
2013-02-15 13:22:31 -08:00
728e208f49
merge master
2013-02-15 11:39:22 -08:00
4f11eb5209
1) Fix alert sent from line DruidMasterBalancer:71 to not include all of the segments that a server is holding
2013-02-14 18:24:51 -06:00
bf0d9c601d
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/KillTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskMasterLifecycle.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunnerWorkQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorServletModule.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-14 15:42:07 -08:00
18daa19fc9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskHolder.java
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/AbstractTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskContext.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-14 13:10:29 -08:00
a616f86128
add DateTime import back
2013-02-14 09:09:22 -08:00
754b065c54
merge master
2013-02-14 09:08:25 -08:00
82f02a4fe4
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-13 16:34:14 -08:00
c7b4973700
bug fixes for RTR
2013-02-13 14:19:11 -08:00
429e23d3b2
change break to continue in BalancerCostAnalyzer to skip calculating cost when it's not necessary
2013-02-13 08:57:39 -08:00
a0b159fed7
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-12 20:58:17 -06:00
fcbac96193
revert change to findNewSegmentHome: now return single ServerHolder
2013-02-12 10:47:30 -08:00
b27a459f71
more refactoring and UT passing at this point
2013-02-11 18:13:01 -08:00
695b78b19c
make findNewSegmentHome return a priority queue so assignment can assign replicants to other servers
2013-02-11 15:28:17 -08:00
69d0d98df6
t Merge branch 'master' into indexing_refactor
2013-02-11 12:37:20 -08:00
a9176e738a
Merge branch 'master' into indexing_refactor
2013-02-11 12:36:49 -08:00
e12da51057
merge master
2013-02-11 11:28:50 -08:00
05168808c2
Merge pull request #77 from metacret/master
...
Adding LinearShardSpec and ACL on S3 object
2013-02-11 11:12:31 -08:00
07c005a0f7
Merge branch 'master' of github.com:metamx/druid
2013-02-11 10:56:39 -08:00
ac93f752b0
fix a case where a segment may get stuck in the terminate queue
2013-02-11 10:56:30 -08:00
111df9a90e
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-07 10:35:32 -08:00
6ff42f379c
fix cut-n-paste mistake in error message
2013-02-07 10:11:36 -08:00
267c797023
fix bug in throttle replicants
2013-02-06 18:40:29 -08:00
0d303bee96
reset lifetimes for the replication throttle
2013-02-06 17:41:08 -08:00
e2862fe1fb
Merge pull request #79 from metamx/bug_fixes
...
Throttle replicants that are created by rules
2013-02-06 16:50:04 -08:00
84baef0462
fix some bugs from last commit
2013-02-06 13:57:56 -08:00
97999c06c4
throttle replicants
2013-02-06 11:08:44 -08:00
4f70bbc687
adding linear shard spec and ACL for s3 bucket
2013-02-04 10:27:09 -08:00
34b4383934
1) Adjust DataSegment to have a binaryVersion field that indicates the expected binary version of the segment files
...
2) Eliminate race condition from RemoteTaskRunnerTest.testAlreadyExecutedTask()
2013-02-01 18:59:33 -06:00
cheddar
Eric Tschetter
Nelson Ray
Fangjin Yang
xvrl
Gian Merlino
James Estes
Deep Ganguli
fjy
Jae Hyeon Bae