Fangjin Yang
2e1e1753d5
[maven-release-plugin] prepare for next development iteration
2013-02-26 15:15:54 -08:00
Fangjin Yang
ab456cf3b1
[maven-release-plugin] prepare release druid-0.3.4
2013-02-26 15:15:36 -08:00
Nelson Ray
e67a8757ba
Merge remote-tracking branch 'origin/master' into new_balancer
...
Conflicts:
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-26 14:18:07 -08:00
Gian Merlino
d32a6284ce
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/worker/config/WorkerConfig.java
2013-02-26 13:03:56 -08:00
Gian Merlino
3129e5451b
Merge branch 'master' into task-stuff
2013-02-25 12:33:45 -08:00
Gian Merlino
9964e7dfe4
[maven-release-plugin] prepare for next development iteration
2013-02-25 12:24:24 -08:00
Gian Merlino
99c4f9446c
[maven-release-plugin] prepare release druid-0.3.3
2013-02-25 12:24:15 -08:00
Gian Merlino
49700dffe9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2013-02-25 12:18:31 -08:00
Eric Tschetter
e0f6df1a5c
[maven-release-plugin] prepare for next development iteration
2013-02-21 20:39:50 -06:00
Eric Tschetter
846bc0e4f0
[maven-release-plugin] prepare release druid-0.3.2
2013-02-21 20:39:45 -06:00
Eric Tschetter
699aadd2b1
[maven-release-plugin] prepare for next development iteration
2013-02-21 15:50:01 -06:00
Eric Tschetter
606a2e4b01
[maven-release-plugin] prepare release druid-0.3.1
2013-02-21 15:49:55 -06:00
Eric Tschetter
f8c54a72c2
1) Changes to allow for local storage
2013-02-21 15:47:01 -06:00
Eric Tschetter
dc3459d3f9
1) Initial commit of refactorings on top of housejester's to simplify the zipping and simplify the creation of other methods of loading data
2013-02-21 15:47:01 -06:00
James Estes
d1626576c0
Working toward making it easier to add new SegmentPullers.
...
1) Move the local cacheFile logic out of the S3 pullers into the SingleSegmentLoader
2) Make the S3SegmentPuller just pull down the file
3) Make the Loader do the unzip, ungzip, or rename
4) 2 and 3 make S3ZippedSegmentPuller not necessary (still there, just deprecated and empty)
4) Tweak the TaskToolbox so that the Pullers returned by getSegmentGetters behave the same as they did before
2013-02-21 15:47:01 -06:00
Deep Ganguli
60da9ffddb
Merge branch 'master' into hadoop-index-task
...
Conflicts:
common/src/main/java/com/metamx/druid/utils/JodaUtils.java
indexer/src/main/java/com/metamx/druid/indexer/DbUpdaterJob.java
indexer/src/main/java/com/metamx/druid/indexer/HadoopDruidIndexerConfig.java
indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
2013-02-15 13:22:31 -08:00
Gian Merlino
824e3c0eb2
[maven-release-plugin] prepare for next development iteration
2013-02-15 13:03:39 -08:00
Gian Merlino
6bbc992101
[maven-release-plugin] prepare release druid-0.3.0
2013-02-15 13:03:32 -08:00
Nelson Ray
728e208f49
merge master
2013-02-15 11:39:22 -08:00
xvrl
05d347c304
Merge remote-tracking branch 'origin/master' into speedup-timeboundary
...
Conflicts:
client/pom.xml
common/pom.xml
druid-services/pom.xml
examples/pom.xml
examples/rand/pom.xml
examples/twitter/pom.xml
index-common/pom.xml
indexer/pom.xml
merger/pom.xml
pom.xml
realtime/pom.xml
server/pom.xml
2013-02-15 11:25:47 -08:00
Eric Tschetter
5ee7c0c264
1) Remove stray import of java.util.List
2013-02-15 12:15:53 -06:00
Gian Merlino
548c901c0b
Additional fix for columns with cardinality 0
2013-02-14 23:45:02 -08:00
Gian Merlino
3fed8ee0ce
[maven-release-plugin] prepare for next development iteration
2013-02-14 21:23:46 -08:00
Gian Merlino
ba7df5b709
[maven-release-plugin] prepare release druid-0.2.9
2013-02-14 21:23:41 -08:00
Gian Merlino
22d1f2a0c0
IndexMergerTest: Empty column test
2013-02-14 21:11:31 -08:00
Gian Merlino
152bf201ec
[maven-release-plugin] prepare for next development iteration
2013-02-14 16:27:20 -08:00
Gian Merlino
e56bc27f93
[maven-release-plugin] prepare release druid-0.2.8
2013-02-14 16:27:12 -08:00
Eric Tschetter
4f11eb5209
1) Fix alert sent from line DruidMasterBalancer:71 to not include all of the segments that a server is holding
2013-02-14 18:24:51 -06:00
Fangjin Yang
bf0d9c601d
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/KillTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskMasterLifecycle.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunnerWorkQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorServletModule.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-14 15:42:07 -08:00
Gian Merlino
18daa19fc9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskHolder.java
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/AbstractTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskContext.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-14 13:10:29 -08:00
Nelson Ray
a616f86128
add DateTime import back
2013-02-14 09:09:22 -08:00
Nelson Ray
754b065c54
merge master
2013-02-14 09:08:25 -08:00
Eric Tschetter
c5b78e75cf
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-13 18:51:32 -08:00
Fangjin Yang
82f02a4fe4
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-13 16:34:14 -08:00
Fangjin Yang
c7b4973700
bug fixes for RTR
2013-02-13 14:19:11 -08:00
Nelson Ray
429e23d3b2
change break to continue in BalancerCostAnalyzer to skip calculating cost when it's not necessary
2013-02-13 08:57:39 -08:00
Eric Tschetter
a0b159fed7
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-12 20:58:17 -06:00
Nelson Ray
fcbac96193
revert change to findNewSegmentHome: now return single ServerHolder
2013-02-12 10:47:30 -08:00
xvrl
07b0b4611b
bump up to 0.3.0
2013-02-12 09:52:21 -08:00
Fangjin Yang
b27a459f71
more refactoring and UT passing at this point
2013-02-11 18:13:01 -08:00
xvrl
e9f546c4ca
add default implementations to QueryToolChest methods
2013-02-11 16:51:40 -08:00
Nelson Ray
695b78b19c
make findNewSegmentHome return a priority queue so assignment can assign replicants to other servers
2013-02-11 15:28:17 -08:00
xvrl
e10025c841
filter out unneeded segments for TimeBoundary query
2013-02-11 13:21:37 -08:00
Fangjin Yang
69d0d98df6
t Merge branch 'master' into indexing_refactor
2013-02-11 12:37:20 -08:00
Fangjin Yang
a9176e738a
Merge branch 'master' into indexing_refactor
2013-02-11 12:36:49 -08:00
Nelson Ray
1b9764fffb
add binary version to datasegments for balancer test
2013-02-11 11:36:45 -08:00
Nelson Ray
e12da51057
merge master
2013-02-11 11:28:50 -08:00
cheddar
05168808c2
Merge pull request #77 from metacret/master
...
Adding LinearShardSpec and ACL on S3 object
2013-02-11 11:12:31 -08:00
Fangjin Yang
e519f82e87
[maven-release-plugin] prepare for next development iteration
2013-02-11 10:58:51 -08:00
Fangjin Yang
73f7468a23
[maven-release-plugin] prepare release druid-0.2.7
2013-02-11 10:58:43 -08:00
Fangjin Yang
07c005a0f7
Merge branch 'master' of github.com:metamx/druid
2013-02-11 10:56:39 -08:00
Fangjin Yang
ac93f752b0
fix a case where a segment may get stuck in the terminate queue
2013-02-11 10:56:30 -08:00
Fangjin Yang
111df9a90e
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-07 10:35:32 -08:00
xvrl
6ff42f379c
fix cut-n-paste mistake in error message
2013-02-07 10:11:36 -08:00
Fangjin Yang
ec2b906fad
[maven-release-plugin] prepare for next development iteration
2013-02-06 18:43:42 -08:00
Fangjin Yang
22d9607169
[maven-release-plugin] prepare release druid-0.2.6
2013-02-06 18:43:34 -08:00
Fangjin Yang
267c797023
fix bug in throttle replicants
2013-02-06 18:40:29 -08:00
Fangjin Yang
e09c658352
[maven-release-plugin] prepare for next development iteration
2013-02-06 17:42:39 -08:00
Fangjin Yang
668353e895
[maven-release-plugin] prepare release druid-0.2.5
2013-02-06 17:42:33 -08:00
Fangjin Yang
0d303bee96
reset lifetimes for the replication throttle
2013-02-06 17:41:08 -08:00
Fangjin Yang
d023d219ae
[maven-release-plugin] prepare for next development iteration
2013-02-06 16:54:21 -08:00
Fangjin Yang
d0fdf4ccf0
[maven-release-plugin] prepare release druid-0.2.4
2013-02-06 16:54:16 -08:00
Fangjin Yang
7540dd58ca
fix broken UT
2013-02-06 16:52:24 -08:00
fjy
e2862fe1fb
Merge pull request #79 from metamx/bug_fixes
...
Throttle replicants that are created by rules
2013-02-06 16:50:04 -08:00
Fangjin Yang
84baef0462
fix some bugs from last commit
2013-02-06 13:57:56 -08:00
Fangjin Yang
97999c06c4
throttle replicants
2013-02-06 11:08:44 -08:00
Jae Hyeon Bae
4f70bbc687
adding linear shard spec and ACL for s3 bucket
2013-02-04 10:27:09 -08:00
Eric Tschetter
34b4383934
1) Adjust DataSegment to have a binaryVersion field that indicates the expected binary version of the segment files
...
2) Eliminate race condition from RemoteTaskRunnerTest.testAlreadyExecutedTask()
2013-02-01 18:59:33 -06:00
Eric Tschetter
c63eb2b77b
Merge branch 'master' of github.com:metamx/druid
2013-02-01 18:39:09 -06:00
xvrl
7ec3563e7b
[maven-release-plugin] prepare for next development iteration
2013-02-01 16:25:42 -08:00
xvrl
a239f62158
[maven-release-plugin] prepare release druid-0.2.3
2013-02-01 16:25:35 -08:00
Eric Tschetter
1a9edbebd3
1) whitespace
2013-02-01 16:21:26 -06:00
Eric Tschetter
5f2c3cec1b
Merge branch 'master' of github.com:metamx/druid
2013-02-01 16:19:26 -06:00
Eric Tschetter
7de5c7806e
1) Some whitespace changes
...
2) Fix up the alerting to actually include the segment and server for when the MasterBalancer has something stuck in the queue
2013-02-01 16:19:17 -06:00
Fangjin Yang
9e88197845
change endpoints for killing segments
2013-02-01 14:08:52 -08:00
Nelson Ray
1c9270add3
change format of line comment
2013-02-01 11:25:41 -08:00
Nelson Ray
0b61f96965
extra documentation
2013-02-01 11:25:05 -08:00
Fangjin Yang
f50c214514
Merge branch 'master' into task-stuff
2013-01-31 09:59:52 -08:00
Fangjin Yang
00e6ac1e10
[maven-release-plugin] prepare for next development iteration
2013-01-30 17:09:23 -08:00
Fangjin Yang
ef19de8270
[maven-release-plugin] prepare release druid-0.2.2
2013-01-30 17:09:18 -08:00
Fangjin Yang
fa733565e8
[maven-release-plugin] prepare for next development iteration
2013-01-30 16:28:28 -08:00
Fangjin Yang
0b116a8fe4
[maven-release-plugin] prepare release druid-0.2.1
2013-01-30 16:28:22 -08:00
Fangjin Yang
2a0de2090b
[maven-release-plugin] prepare for next development iteration
2013-01-30 13:47:15 -08:00
Fangjin Yang
049ca2bad4
[maven-release-plugin] prepare release druid-0.2.0
2013-01-30 13:47:08 -08:00
Nelson Ray
dc49cccf56
javadoc format
2013-01-29 18:32:52 -08:00
Nelson Ray
73f774e768
make masterbalancertest more stringent
2013-01-29 18:17:38 -08:00
Nelson Ray
a1ce4b294e
simplify balancer test
2013-01-29 18:16:49 -08:00
Nelson Ray
b205f7b3c4
use existing currentlyMovingSegments instead of creating a reundant segmentsBeingMoved
2013-01-29 18:16:34 -08:00
Nelson Ray
034d652569
make balancer tests take into account loadqueue status
2013-01-29 17:13:01 -08:00
cheddar
9a22e46eca
Merge pull request #71 from metamx/queryrunner-onerow-test
...
Bug fixes for incremental index storage adapter and one row indexes
2013-01-29 13:41:15 -08:00
cheddar
a7abd2ca86
Merge pull request #70 from metamx/timezone-tests
...
More time zone tests
2013-01-29 13:40:04 -08:00
Fangjin Yang
8fc10a61a0
bug fix for incremental index and 1 row
2013-01-29 13:23:57 -08:00
Fangjin Yang
41ca33c734
Merge branch 'master' of github.com:metamx/druid into task-stuff
2013-01-29 12:57:56 -08:00
Gian Merlino
61569f62be
TimeseriesQueryRunner tests with small incremental indexes
2013-01-29 12:54:52 -08:00
Eric Tschetter
da914b835e
1) Add caching for SegmentMetadataQuery
2013-01-29 13:04:57 -06:00
xvrl
27e09987c6
timezone test
2013-01-29 10:38:22 -08:00
Eric Tschetter
08d3253f3d
Merge branch 'master' of github.com:metamx/druid
2013-01-29 11:29:42 -06:00
Eric Tschetter
f2cce28297
1) Make SegmentMetadataQuery work
2013-01-29 11:29:26 -06:00
Nelson Ray
6438401a32
split pick segment logic into its own method
2013-01-28 15:06:09 -08:00
Eric Tschetter
7439a2e820
1) Make tests pass
2013-01-28 16:42:51 -06:00
xvrl
c0822325e7
proper groupby tests
2013-01-28 16:41:03 -06:00
cheddar
5f18f368e3
Merge pull request #61 from metamx/fix-cache-timezone
...
Cache did not preserve timezone information
2013-01-28 14:39:00 -08:00
Fangjin Yang
74057600f9
fix worker node still using db connection
2013-01-27 09:57:37 -08:00
Gian Merlino
f3b04d3f5f
S3SegmentKiller: Add TODO note
2013-01-25 11:33:50 -08:00
Gian Merlino
e6a618ca76
Merge branch 'killsegments' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/LocalTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/exec/TaskConsumer.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
2013-01-25 11:30:10 -08:00
xvrl
5e1cac6d9f
timezone test
2013-01-24 17:57:10 -08:00
Deep Ganguli
017d4779d6
Implemented Hadoop Index Task which takes as input a HadoopDruidIndexConfig and
...
generates index segments.
The HadoopIndexTask run method wraps a HadoopDruidIndexerJob run method. The
key modifications to the HadoopDruidIndexerJob are as follows:
- The UpDaterJobSpec field of the config that is used to set up the indexer job
is set to null. This ensures that the job does not push a list of published
segments to the database, in order to allow the indexing service to handle this
later.
- Set the version field of the config file based on the TaskContext. Also
changed config.setVersion method to take a string (as opposed to a Date) as
input, and propogated this change where necessary.
- Set the SegmentOutputDir field of the config file based on the TaskToolbox,
to allow the indexing service to handle where to write the segments too.
- Added a method to IndexGeneratorJob called getPublishedSegments, that simply
returns a list of published segments without publishing this list to the
database.
2013-01-23 19:27:14 -08:00
Eric Tschetter
67feee3cd6
1) Indexes don't always have an index.drd file anymore
2013-01-23 18:36:52 -06:00
xvrl
55ae4c87dd
timezone support in groupby query
2013-01-23 08:51:02 -08:00
xvrl
35058786d9
match query interval to granularity for this test
2013-01-23 08:50:43 -08:00
xvrl
86a6d112e3
proper groupby tests
2013-01-22 16:54:14 -08:00
Fangjin Yang
272d737517
cleaning up some interactions with RTR and workers
2013-01-22 16:21:38 -08:00
xvrl
f05c050c53
add test for timezone
2013-01-21 15:49:39 -08:00
Nelson Ray
94b72e8878
replace param BalancerCostAnalyzer getter with a factory
2013-01-21 15:32:29 -08:00
Fangjin Yang
bab9ee8827
Merge branch 'master' into killsegments
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
2013-01-21 14:47:49 -08:00
Nelson Ray
2d7113b263
huge simplification of balancing code
2013-01-21 14:28:25 -08:00
Eric Tschetter
5b1e03530c
1) Fix some bugs found by external test suite
2013-01-16 21:06:57 -06:00
Fangjin Yang
21613bc73b
initial commit to hard delete segments
2013-01-16 17:31:01 -08:00
Eric Tschetter
c8cb96b006
1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir()
...
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
2013-01-16 17:10:33 -06:00
Eric Tschetter
538d00e75e
Merge branch 'master' of github.com:metamx/druid
2013-01-16 10:02:01 -06:00
Eric Tschetter
8b31d8db9f
1) Adjust IndexMerger to create convert the indexes it creates from the old format to the new. This is done quite sub-optimally, but it will work for now...
2013-01-16 10:01:46 -06:00
Fangjin Yang
7e074e8158
fix pom breakage
2013-01-15 12:04:12 -08:00
Nelson Ray
e847faf02f
change name of MAX_SEGMENTS_TO_MOVE since it's not a final
2013-01-15 11:29:46 -08:00
Nelson Ray
ef9e077d9a
remove getConfig method from DruidMaster
2013-01-14 16:01:29 -08:00
Eric Tschetter
998f0bf3c8
1) Better error message when aggregators want more memory than is available.
2013-01-14 18:00:51 -06:00
Nelson Ray
5f40dc6d9c
put BalancerCostAnalyzer into params
2013-01-10 10:27:43 -08:00
Nelson Ray
617907d85b
move empty server holder list check outside of loop
2013-01-08 16:12:16 -08:00
Nelson Ray
4549c3a8c3
add some finals, reorganize
2013-01-08 09:07:43 -08:00
Nelson Ray
2b4dbdde03
lots of changes
2013-01-07 17:28:53 -08:00
Nelson Ray
6fda5330fd
finish making MAX_SEGMENTS_TO_MOVE a config item
2013-01-07 16:56:55 -08:00
Nelson Ray
ba82fe746a
put MAX_SEGMENTS_TO_MOVE in the config
2013-01-07 16:28:27 -08:00
Eric Tschetter
4ef31de62d
1) Fix stray import, take 2... yay IntelliJ
2013-01-07 13:11:28 -08:00
Nelson Ray
d2ff88e923
add more stats to be emitted
2013-01-07 13:09:18 -08:00
Nelson Ray
a7553776fa
make sampling of segments uniformly at random and add unit test
2013-01-07 12:51:51 -08:00
Eric Tschetter
b62bdf915b
1) Fix stray import... yay IntelliJ
2013-01-07 10:09:19 -08:00
Nelson Ray
2ea164acd9
get rid of extraneous analyzer.inits
2013-01-07 09:09:56 -08:00
Eric Tschetter
b184c46ae0
1) Introduce idea of value types and generic columns to the serialization format
...
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
2013-01-04 18:12:49 -06:00
Nelson Ray
402ddbf261
update
2013-01-04 11:20:06 -08:00
Nelson Ray
117cac2795
update
2013-01-03 14:12:10 -08:00
Nelson Ray
bb31fa7b68
update
2013-01-03 10:48:05 -08:00
xvrl
9627fdcd1c
[maven-release-plugin] prepare for next development iteration
2013-01-03 10:09:52 -08:00
xvrl
f2c2d3f4cf
[maven-release-plugin] prepare release druid-0.1.25
2013-01-03 10:09:43 -08:00
Nelson Ray
5441662cb8
Merge remote-tracking branch 'origin/master' into new_balancer
2013-01-03 09:10:48 -08:00
xvrl
a9b8d8f1ce
fix end of month bug in PeriodDropRule as well
2013-01-02 16:10:23 -08:00
xvrl
0543322c56
tested timestamps should be deterministic
2013-01-02 14:31:51 -08:00
xvrl
19c3fd3363
make reference timestamp explicit for time based rules
2013-01-02 14:27:34 -08:00
xvrl
2a188996cd
[maven-release-plugin] prepare for next development iteration
2012-12-31 14:56:10 -08:00
xvrl
6cdcc2becc
[maven-release-plugin] prepare release druid-0.1.24
2012-12-31 14:56:03 -08:00
xvrl
d0290b1975
fix flawed interval overlap rule test
...
fails around month end
2012-12-31 14:52:42 -08:00
Nelson Ray
72758e1345
update
2012-12-31 13:55:48 -08:00
Nelson Ray
3eda9dc5f2
working balancercostanalyzer
2012-12-31 11:08:37 -08:00
Nelson Ray
366216f9f3
initial cost analyzer commit
2012-12-28 16:30:21 -08:00
Gian Merlino
5c1b2360d6
- Add optional query/wait metric to MetricsEmittingQueryRunner
...
- Add MetricsEmittingExecutorService decorator, and use it on compute nodes
2012-12-20 16:11:08 -08:00
Fangjin Yang
5137af0750
[maven-release-plugin] prepare for next development iteration
2012-12-18 16:17:44 -08:00
Fangjin Yang
c579ba5a42
[maven-release-plugin] prepare release druid-0.1.23
2012-12-18 16:17:38 -08:00
Fangjin Yang
00ecf9d028
sorted db datasources for rules; smaller time boundary range
2012-12-18 16:16:03 -08:00
Fangjin Yang
090a5a105d
[maven-release-plugin] prepare for next development iteration
2012-12-18 14:19:09 -08:00
Fangjin Yang
fb8f4f1f41
[maven-release-plugin] prepare release druid-0.1.22
2012-12-18 14:19:02 -08:00
xvrl
2a7214089e
fix NPE in segmentMetadata query for null dimension values
2012-12-18 14:10:01 -08:00
Fangjin Yang
75d465aed0
[maven-release-plugin] prepare for next development iteration
2012-12-18 10:37:16 -08:00
Fangjin Yang
decc4e4c0e
[maven-release-plugin] prepare release druid-0.1.21
2012-12-18 10:37:08 -08:00
Fangjin Yang
e34944b1c5
[maven-release-plugin] prepare for next development iteration
2012-12-15 00:52:01 -08:00
Fangjin Yang
24f0207fd8
[maven-release-plugin] prepare release druid-0.1.20
2012-12-15 00:51:54 -08:00
Fangjin Yang
db556102db
fix typo
2012-12-15 00:50:28 -08:00
Fangjin Yang
f9386c3623
[maven-release-plugin] prepare for next development iteration
2012-12-15 00:44:58 -08:00
Fangjin Yang
4bcc8d27fa
[maven-release-plugin] prepare release druid-0.1.19
2012-12-15 00:44:50 -08:00
Fangjin Yang
585a812f6e
bug fixes again in load and drop
2012-12-15 00:43:16 -08:00
Fangjin Yang
5fff07a8b0
[maven-release-plugin] prepare for next development iteration
2012-12-14 22:56:49 -08:00
Fangjin Yang
2b2818dbaf
[maven-release-plugin] prepare release druid-0.1.18
2012-12-14 22:56:43 -08:00
Fangjin Yang
76807ae00e
fix overloading of segments
2012-12-14 22:55:12 -08:00
Fangjin Yang
b0e1f90e35
[maven-release-plugin] prepare for next development iteration
2012-12-14 22:07:57 -08:00
Fangjin Yang
ce8e02b5a3
[maven-release-plugin] prepare release druid-0.1.17
2012-12-14 22:07:50 -08:00
Fangjin Yang
884da334b2
fix for master getting stuck when assigning
2012-12-14 22:06:19 -08:00
Fangjin Yang
0821dce8c1
[maven-release-plugin] prepare for next development iteration
2012-12-14 21:29:20 -08:00
Fangjin Yang
a0d22a6646
[maven-release-plugin] prepare release druid-0.1.16
2012-12-14 21:29:14 -08:00
Fangjin Yang
6a11e99681
bug fixes for dropping segments and logging
2012-12-14 21:27:18 -08:00
Fangjin Yang
e37e24cc25
[maven-release-plugin] prepare for next development iteration
2012-12-14 17:19:59 -08:00
Fangjin Yang
4e78d8e734
[maven-release-plugin] prepare release druid-0.1.15
2012-12-14 17:19:53 -08:00
Fangjin Yang
74a977504e
bug fix for dropping segments in master
2012-12-14 17:06:03 -08:00
Eric Tschetter
b4a1bb4a00
[maven-release-plugin] prepare for next development iteration
2012-12-14 11:37:17 -06:00
Eric Tschetter
cb9b18c7c2
[maven-release-plugin] prepare release druid-0.1.14
2012-12-14 11:37:12 -06:00
Eric Tschetter
da137a4883
1) Optimize imports a bit
2012-12-14 11:34:23 -06:00
cheddar
7670c8483b
Merge pull request #45 from metamx/fjy
...
bug fixes for exceptions in rules
2012-12-14 09:32:37 -08:00
Fangjin Yang
18692a1214
better error emitting for mismatched rules
2012-12-14 09:25:39 -08:00
Fangjin Yang
8c7fd3bc49
more specific checks in rule test
2012-12-14 09:12:02 -08:00
Fangjin Yang
aa69ed5251
alert when no rules match; exceptions in rules no longer block
2012-12-14 09:05:25 -08:00
Eric Tschetter
29a64f50e2
[maven-release-plugin] prepare for next development iteration
2012-12-13 17:57:46 -06:00
Eric Tschetter
c82007973e
[maven-release-plugin] prepare release druid-0.1.13
2012-12-13 17:57:42 -06:00
Fangjin Yang
6729f166d9
[maven-release-plugin] prepare for next development iteration
2012-12-12 15:38:58 -08:00
Fangjin Yang
578cf94bc0
[maven-release-plugin] prepare release druid-0.1.12
2012-12-12 15:38:37 -08:00
xvrl
5dcf962eaa
register SegmentMetadataQueryRunnerFactory in ServerInit
2012-12-12 13:45:15 -08:00
xvrl
bfc9b63931
segment metadata query
2012-12-12 13:45:12 -08:00
Marshall Pierce
1a0ef0d5c3
Fix direct memory vm option name in warning message
2012-12-11 11:15:09 -08:00
Fangjin Yang
0105233fc3
[maven-release-plugin] prepare for next development iteration
2012-12-07 17:50:47 -08:00
Fangjin Yang
943fe5e6bf
[maven-release-plugin] prepare release druid-0.1.11
2012-12-07 17:50:41 -08:00
Fangjin Yang
4695fbfcd9
fix bug with comparing historical servers
2012-12-07 17:49:18 -08:00
Fangjin Yang
da4b31c28f
[maven-release-plugin] prepare for next development iteration
2012-12-07 15:18:19 -08:00
Fangjin Yang
6df32ac0b0
[maven-release-plugin] prepare release druid-0.1.10
2012-12-07 15:18:11 -08:00
Fangjin Yang
404f6322c6
bug fix for checking if a segment exists on a node
2012-12-07 15:16:34 -08:00
Fangjin Yang
9eb28254f8
[maven-release-plugin] prepare for next development iteration
2012-12-07 14:49:13 -08:00
Fangjin Yang
9ce536e040
[maven-release-plugin] prepare release druid-0.1.9
2012-12-07 14:49:04 -08:00
Fangjin Yang
5a511cbef0
bug fix for missing quotations creating default datasource
2012-12-07 14:47:38 -08:00
Fangjin Yang
35362b3e6b
[maven-release-plugin] prepare for next development iteration
2012-12-07 14:37:25 -08:00
Fangjin Yang
bd3c05dddb
[maven-release-plugin] prepare release druid-0.1.8
2012-12-07 14:37:18 -08:00
Fangjin Yang
f6519744e6
bug fix for using configurable default for datasource
2012-12-07 14:35:49 -08:00
Fangjin Yang
50a28f827e
[maven-release-plugin] prepare for next development iteration
2012-12-07 10:59:52 -08:00
Fangjin Yang
b59361065e
[maven-release-plugin] prepare release druid-0.1.7
2012-12-07 10:59:34 -08:00
Fangjin Yang
6028273332
defaults for rules to make startup a bit easier
2012-12-07 10:54:38 -08:00
Fangjin Yang
1ebd7a8a8a
fix wrong jquery version
2012-12-06 14:50:29 -08:00
Fangjin Yang
4a1abfc5dc
gui for rule configuratin; introducing jquery ui to master console
2012-12-05 15:39:59 -08:00
Fangjin Yang
5b8bd412c0
caching for rules; get rules from cache
2012-12-03 16:13:20 -08:00
Fangjin Yang
5822f4f5f7
refactor master to run rules before cleaning up; more master stats; general improvements
2012-12-03 14:43:04 -08:00
Fangjin Yang
91cd9e9536
fix broken UT
2012-11-30 13:17:14 -08:00
Fangjin Yang
aa8fb7b9a3
abstract the assign and drop logic; cleanups everywhere; http endpoints for getting and assigning rules
2012-11-30 11:28:10 -08:00
Fangjin Yang
2e5e1ce989
first commit of tiers for compute nodes; working UT at this point
2012-11-28 17:37:08 -08:00
Eric Tschetter
09a31e5995
Tests should build.
2012-11-28 12:35:24 -06:00
Eric Tschetter
b85c54d8a3
Merge branch 'master' of github.com:metamx/druid
2012-11-28 11:55:25 -06:00
Eric Tschetter
4de6ef5025
1) Remove the requirement on the segmentsPath zk property.
2012-11-28 11:54:59 -06:00
Gian Merlino
5902ccd0cb
DruidMasterSegmentMerger: Clearer method names and more tests
2012-11-27 15:46:03 -08:00
Eric Tschetter
0390e7fa5c
[maven-release-plugin] prepare for next development iteration
2012-11-21 22:13:15 -06:00
Eric Tschetter
93a9affa45
[maven-release-plugin] prepare release druid-0.1.6
2012-11-21 22:13:08 -06:00
Eric Tschetter
9c732e75a1
1) Add new SearchQueryDimFilter/SearchQueryFilter that will do a filter using the same queries that can be used for SearchQuery
2012-11-21 19:06:32 -06:00
Eric Tschetter
3a9591bda6
Merge branch 'master' of github.com:metamx/druid
2012-11-21 18:39:09 -06:00
Eric Tschetter
228b223509
1) Fix bug with Sinks that causes Realtime it to overwrite the first chunk of data when creating a segment to hand off to compute nodes.
...
2) Fix bug with RealtimePlumberSchool that can cause it to miss some previously-persisted segments when directory naming changes
3) Have the various runnables in RealtimePlumberSchool name the threads to make it a bit easier to debug
2012-11-21 18:38:57 -06:00
Gian Merlino
5e447937e2
[maven-release-plugin] prepare for next development iteration
2012-11-21 16:09:14 -08:00
Gian Merlino
32d46a3eec
[maven-release-plugin] prepare release druid-0.1.5
2012-11-21 16:09:08 -08:00
Eric Tschetter
06b8e4230c
1) Fix PostAggregations with GroupBy ( Fixes #24 )
...
2) Add GroupByTimeseriesQueryRunnerTest which wraps a GroupByQueryRunner to look like a TimeseriesQueryRunner and leverages the tests there to help verify behavior (#8 )
2012-11-21 12:51:43 -06:00
Eric Tschetter
0f63cb4f00
1) Have IndexGeneratorJob write the descriptors for each of the segments it creates to a path in the temporary working directory (generally HDFS)
...
2) Have the DbUpdaterJob read descriptors from the temporary working directory instead of looking in the final segment output location (often the eventually consistent S3)
3) 1 and 2 Fixes #30
2012-11-20 15:30:50 -06:00
Fangjin Yang
4131063049
Merge branch 'master' into fjy
2012-11-19 15:07:00 -08:00
Eric Tschetter
701cc9562b
1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up.
...
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29 , or at least clarifies the behavior...)
2012-11-19 17:01:17 -06:00
Gian Merlino
16835a1f84
Master: Add segment limit to merge selection algo
...
If we attempt to merge too many segments at once, we risk creating an
oversized indexing task.
2012-11-19 14:46:45 -08:00
Eric Tschetter
aa073e4e73
1) Add verification of memory settings to initialization. ( Fixes #21 )
2012-11-14 15:37:11 -06:00
Fangjin Yang
cf1c741f8a
[maven-release-plugin] prepare for next development iteration
2012-11-13 15:10:12 -08:00
Fangjin Yang
06bd77fca7
[maven-release-plugin] prepare release druid-0.1.4
2012-11-13 15:10:00 -08:00
Fangjin Yang
0d697ed104
fix rename of this node
2012-11-12 16:17:51 -08:00
Fangjin Yang
57468d39ef
reverting some of the last changes
2012-11-12 16:14:48 -08:00
Fangjin Yang
c20dccd0f4
modifying the way registering serdes works to hopefully be a bit easier to use
2012-11-12 13:58:43 -08:00
Marshall Pierce
447dfc0cc6
Switch to using javax.servlet:servlet-api:2.5 instead of two different versions of jetty's servlet-api distribution
2012-11-09 17:31:03 -08:00
Fangjin Yang
dd17992172
[maven-release-plugin] prepare for next development iteration
2012-11-09 15:50:24 -08:00
Fangjin Yang
47e71080db
[maven-release-plugin] prepare release druid-0.1.3
2012-11-09 15:50:16 -08:00
Fangjin Yang
5859599365
[maven-release-plugin] prepare for next development iteration
2012-11-09 13:41:04 -08:00
Fangjin Yang
56f2c15e21
[maven-release-plugin] prepare release druid-0.1.2
2012-11-09 13:40:57 -08:00
Fangjin Yang
0b7a3f259a
[maven-release-plugin] prepare for next development iteration
2012-11-09 10:10:32 -08:00
Fangjin Yang
baba4e7b75
[maven-release-plugin] prepare release druid-0.1.1
2012-11-09 10:10:26 -08:00
Fangjin Yang
03e5d74087
[maven-release-plugin] prepare for next development iteration
2012-11-09 09:33:06 -08:00
Fangjin Yang
8aff8decad
[maven-release-plugin] prepare release druid-0.1.0
2012-11-09 09:32:57 -08:00
Eric Tschetter
0a0e2a6cc1
1) Try to fix the dependency issues for running the HadoopDruidIndexer locally.
2012-11-08 17:06:02 -08:00
Fangjin Yang
09e81be14c
rename file
2012-11-08 13:42:58 -08:00
Fangjin Yang
3ddfdfc702
add missing header
2012-11-08 10:45:21 -08:00
Fangjin Yang
bff5a7d786
fix some merge issues
2012-11-08 10:43:39 -08:00
Fangjin Yang
8323feedb4
Merge branch 'master' into moarunittests
...
Conflicts:
client/src/main/java/com/metamx/druid/http/BrokerMain.java
2012-11-08 10:37:05 -08:00
Fangjin Yang
4d6cfdc774
Merge branch 'master' into autoscaling
2012-11-07 17:27:42 -08:00
Eric Tschetter
b1475219a9
Merge branch 'master' of git://github.com/tralfamadude/druid-1
...
Added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon.
2012-11-06 20:15:57 -08:00
Paul Baclace
cb02c6841b
removed recursive zpath listing from DruidSetup dump command.
2012-11-06 15:30:46 -08:00
Paul Baclace
86d2fb3550
added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon.
2012-11-06 00:16:27 -08:00
Fangjin Yang
499263386c
timeseries for druid + unit tests
2012-11-05 11:38:03 -08:00
Fangjin Yang
b1b611735e
Merge branch 'master' into autoscaling
...
Conflicts:
client/src/main/java/com/metamx/druid/http/BrokerMain.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-02 15:57:16 -07:00
Fangjin Yang
69b929fad5
Merge branch 'master' into moarunittests
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
2012-11-01 16:41:13 -07:00
Fangjin Yang
09cc2e3353
Merge branch 'master' into moarunittests
...
Conflicts:
client/src/main/java/com/metamx/druid/http/BrokerMain.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-01 16:39:28 -07:00
Eric Tschetter
f07b15db1f
Undo massive import re-ordering
2012-11-01 18:38:27 -05:00
Eric Tschetter
8dd6f5c059
1) Refactor the BrokerMain into a BrokerMain and a BrokerNode to allow for extensions
...
2) Rename ServerMain to ComputeMain
3) Refactor ComputeMain to ComputeMain and ComputeNode to allow for extensions
4) Remove "TheSizeAdjuster", was old legacy stuff that's not relevant anymore
5) Fix bug with registering different IndexIO handlers
6) Adjust it so that when a query type is unknown, it returns an error message instead of just emitting an alert and returning nothing
7) Create super classes for the various *Node classes that allow for code sharing on pieces that are common
2012-11-01 18:20:38 -05:00
Fangjin Yang
83a51a801e
cleanup redirects and reannounce workers on connection loss
2012-10-31 16:43:47 -07:00
Ian Brandt
e7ec073d75
Cleaned Maven dependency configuration. In particular the 'mvn dependency:analyze' report no longer warns of used but undeclared dependencies.
2012-10-31 16:10:07 -07:00
Fangjin Yang
11c64593ae
improvements to the indexing service and auto scaling
2012-10-31 14:35:09 -07:00
Ian Brandt
e8f02219bb
Streamlined the Maven POMs, in particular the plugin configurations. The 'mvn versions:display-plugin-updates' report is now warning free, and there are no more Maven warnings reported in Eclipse.
2012-10-31 12:58:16 -07:00
Fangjin Yang
d0f66c1fb3
bug fix for merging with empty strings
2012-10-31 10:28:24 -07:00
Ian Brandt
e24948319b
Removed some dead/unused code.
2012-10-31 00:00:30 -07:00
Ian Brandt
c5e2a88187
Resolved dead code warning by null checking the correct variable according to the associated log statement.
2012-10-30 23:59:31 -07:00
Ian Brandt
35f04b4df6
Fixed some minor potential resource leaks.
2012-10-30 23:26:21 -07:00
Ian Brandt
0d24df7628
Removed unused imports.
2012-10-30 22:22:46 -07:00
Trotter Cashion
2c341144bf
Allow blank mysql passwords
2012-10-26 10:46:30 -07:00
Matt Croydon
6a9d86e1ef
Removed unused import in IndexMerger that was causing build.sh to fail with Java 1.7.0_04-b21 on OS X 10.7.4.
2012-10-24 11:46:32 -05:00
Eric Tschetter
27999caca0
1) Create LICENSE
...
2) Attach copyright and notice of license to files
2012-10-24 05:09:47 -04:00
Eric Tschetter
9d41599967
Initial commit of OSS Druid Code
2012-10-24 03:39:51 -04:00