Brian O'Neill
79d802fede
Working data segment pushed for C*.
2013-05-06 17:54:12 -04:00
fjy
f7dfea056b
[maven-release-plugin] prepare for next development iteration
2013-05-06 14:30:11 -07:00
fjy
a4bd6bc843
[maven-release-plugin] prepare release druid-0.4.5
2013-05-06 14:30:03 -07:00
fjy
f34d73aaff
fix pom versions because of failed release
2013-05-06 14:27:22 -07:00
fjy
0eb7b248d9
[maven-release-plugin] prepare release druid-0.4.4
2013-05-06 14:24:16 -07:00
fjy
8902986069
Merge branch 'master' into spatial
2013-05-03 23:09:30 -07:00
fjy
344a2b5d24
numerous bug fixes and improvements according to code review
2013-05-03 23:09:05 -07:00
cheddar
de720e6e1a
[maven-release-plugin] prepare for next development iteration
2013-05-03 18:41:39 -05:00
cheddar
ba2b2fb39f
[maven-release-plugin] prepare release druid-0.4.3
2013-05-03 18:41:33 -05:00
fjy
3417c50e98
Merge branch 'master' into spatial
...
Conflicts:
client/src/main/java/com/metamx/druid/query/group/DefaultLimitSpec.java
client/src/main/java/com/metamx/druid/query/group/GroupByQueryQueryToolChest.java
client/src/main/java/com/metamx/druid/query/group/LimitSpec.java
pom.xml
2013-05-03 16:20:37 -07:00
cheddar
665f3ef0fd
1) Unit tests for order by and limit stuff for groupBy queries
...
2) Adjustments to order by and limit stuff in light of unit tests-n-stuff
2013-05-03 17:47:18 -05:00
fjy
d57141f46e
checking stuff in but stuff not really working yet
2013-05-03 10:35:26 -07:00
cheddar
c89f4cc85f
[maven-release-plugin] prepare for next development iteration
2013-05-03 12:17:05 -05:00
cheddar
4fef7c2c43
[maven-release-plugin] prepare release druid-0.4.2
2013-05-03 12:16:57 -05:00
Gian Merlino
af08ea7617
ServiceAnnouncer:
...
- Interface with two impls, Noop and Curator
- Used wherever fine services are advertised
Initialization:
- Don't automatically register with service discovery
- Add makeServiceAnnouncer, announceDefaultService methods
- Replace serviceInstance with makeServiceInstanceFactory
2013-05-03 14:29:17 +03:00
cheddar
5f184ea978
[maven-release-plugin] prepare for next development iteration
2013-05-02 17:13:28 -05:00
cheddar
df6fd5eca7
[maven-release-plugin] prepare release druid-0.4.1
2013-05-02 17:13:22 -05:00
Brian O'Neill
b8c8ed8d68
Formatting.
2013-05-01 15:17:15 -04:00
Brian O'Neill
bf01399a2a
Cassandra Data Segment Pusher, initial commit.
2013-05-01 15:11:50 -04:00
fjy
6d4c0850ca
Merge branch 'master' into spatial
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
2013-05-01 11:49:00 -07:00
fjy
63ccb24ef6
Merge branch 'master' into spatial
...
Conflicts:
pom.xml
server/src/main/java/com/metamx/druid/http/InfoResource.java
2013-05-01 10:12:04 -07:00
cheddar
0a3492c9f5
Move all the code for examples under the examples directory. Also add initial take on flights demo.
2013-04-30 18:27:38 -05:00
cheddar
8922adb1f2
1) Make the demos work again.
...
2) First step in restructuring the demos into a single repository
2013-04-30 18:13:55 -05:00
cheddar
60b279b0d3
1) Add support for storing segments in HDFS
2013-04-30 12:47:43 -05:00
cheddar
a665e8b3b7
whitespace!
2013-04-29 14:02:30 -05:00
cheddar
2630d75cd3
More explicit comments around Zookeeper race condition
2013-04-29 13:58:57 -05:00
cheddar
405eed7c60
Better name for ServerInventoryThingie
2013-04-29 11:48:42 -05:00
cheddar
99305b29c8
Fixorz unit tests!
2013-04-26 19:19:57 -05:00
cheddar
163a73bed1
More logging stuff
2013-04-26 19:19:57 -05:00
cheddar
de5c8c9d64
It's better when things compile
2013-04-26 19:19:56 -05:00
cheddar
1eb2dfe17e
1) Don't need a ServiceEmitter in ZkCoordinator anymore.
2013-04-26 19:19:56 -05:00
cheddar
38b5d94ffc
1) Imports...
2013-04-26 19:19:56 -05:00
cheddar
67ce1b6f26
1) Adjust all of the uses of zookeeper paths to be based on ZkPathsConfig
2013-04-26 19:19:56 -05:00
cheddar
3e94204c44
1) Switch from com.netflix.curator to org.apache.curator
2013-04-26 19:19:55 -05:00
cheddar
b8ba9138ff
1) Fix bug with Master not starting up without an indexer specified
...
2) Fix bug with PotentiallyGzippedCompressionProvider only catching ZipExceptions and not IOException (java 6 throws IO, java 7 throws Zip)
3) Create DruidServerMetadata and use that instead of DruidServer to represent the current server
2013-04-26 19:19:55 -05:00
cheddar
f71b941a1a
1) Refactor the announcement of segments to all exist inside the DataSegmentAnnouncer
...
2) Adjust the ExecutorNodes to expose the correct nodeType given the refactorings
2013-04-26 19:19:54 -05:00
cheddar
7370b0f2fc
1) Cleanup TODO comments
2013-04-26 19:19:54 -05:00
cheddar
57c31656d1
1) Eliminate PhoneBook
2013-04-26 19:19:53 -05:00
cheddar
dde50a0d87
1) Remove references to zkclient
...
2) Eradicate zkclient from the poms!
2013-04-26 19:19:53 -05:00
cheddar
43d630c098
1) Replace InventoryManagers with Curator-based Inventory Managers
...
2) Replace ZkSegmentAnnouncer with CuratorSegmentAnnouncer
3) Adjust LoadQueuePeons to operate using Curator
4) Remove ZkPhoneBook
Things left to do:
1) Unit tests
2) Remove references to zkclient library
3) Remove references to PhoneBook interface
2013-04-26 19:19:23 -05:00
fjy
e01a78515e
many more fixes and unit tests
2013-04-25 19:01:45 -07:00
fjy
13b617cfa9
fix some interval bugs
2013-04-23 16:53:57 -07:00
fjy
7184e528fd
fix some config bugs
2013-04-23 13:16:50 -07:00
fjy
70937af7bd
Merge branch 'master' of github.com:metamx/druid into spatial
...
Conflicts:
server/src/main/java/com/metamx/druid/index/v1/IndexStorageAdapter.java
server/src/main/java/com/metamx/druid/index/v1/QueryableIndexStorageAdapter.java
2013-04-23 11:40:44 -07:00
fjy
9b9c516e30
more tests
2013-04-22 18:39:41 -07:00
Gian Merlino
deefa870ec
[maven-release-plugin] prepare for next development iteration
2013-04-19 17:15:07 -07:00
Gian Merlino
5ea1a77a45
[maven-release-plugin] prepare release druid-0.4.0
2013-04-19 17:15:00 -07:00
fjy
6c4e844f97
semi working spatial
2013-04-19 16:56:56 -07:00
Fangjin Yang
71269d7e88
somehow tests are passing
2013-04-17 17:13:24 -07:00
xvrl
db2f3aa761
fix pom versions
2013-04-16 09:53:34 -07:00
xvrl
50998671ae
formatting
2013-04-16 09:33:50 -07:00
xvrl
4597d7f44e
remove IndexStorageAdapter
2013-04-16 09:33:50 -07:00
xvrl
2afe2bdeae
formatting
2013-04-16 09:33:50 -07:00
xvrl
c83631928c
object column selectors
2013-04-16 09:33:50 -07:00
xvrl
0c68bd1b1d
rename MetricSelectorFactory to ColumnSelectorFactory
2013-04-16 09:33:49 -07:00
Gian Merlino
a166344297
[maven-release-plugin] prepare for next development iteration
2013-04-15 18:55:53 -07:00
Gian Merlino
21c2aa5793
[maven-release-plugin] prepare release druid-0.3.38
2013-04-15 18:55:46 -07:00
Gian Merlino
d879c9a41e
[maven-release-plugin] prepare for next development iteration
2013-04-15 18:17:42 -07:00
Gian Merlino
c3d229df7f
[maven-release-plugin] prepare release druid-0.3.37
2013-04-15 18:17:36 -07:00
Gian Merlino
128b873022
[maven-release-plugin] prepare for next development iteration
2013-04-15 16:43:53 -07:00
Gian Merlino
eb80429177
[maven-release-plugin] prepare release druid-0.3.36
2013-04-15 16:43:47 -07:00
Gian Merlino
beff15f8d4
[maven-release-plugin] prepare for next development iteration
2013-04-15 15:03:26 -07:00
Gian Merlino
b71ae8ade3
[maven-release-plugin] prepare release druid-0.3.35
2013-04-15 15:03:20 -07:00
Fangjin Yang
cc929b2a07
slightly better configuration when talking to indexing service
2013-04-15 11:09:04 -07:00
Eric Tschetter
55648c47a7
1) Adjust the GroupByQuery to also be able to merge results on the local node. Fixes #116
...
2) Make the GroupByQuery operate in a multi-threaded fashion by default (this is configurable via druid.query.groupBy.singleThreaded). Fixes #96
3) Fix up some post aggregation computation stuff. I believe #72 is fixed
4) Fix case sensitivity issue with post aggregations on GroupBy queries
2013-04-09 19:13:08 -07:00
cheddar
a678f08b67
Merge pull request #118 from darxriggs/master
...
cleanup & refactoring
2013-04-09 18:56:59 -07:00
Fangjin Yang
6fe9707f96
[maven-release-plugin] prepare for next development iteration
2013-04-08 11:26:37 -07:00
Fangjin Yang
f0ab28be90
[maven-release-plugin] prepare release druid-0.3.34
2013-04-08 11:26:32 -07:00
Fangjin Yang
120b822c18
Merge branch 'master' of github.com:metamx/druid
...
Conflicts:
client/src/main/java/com/metamx/druid/QueryableNode.java
server/src/main/java/com/metamx/druid/http/ComputeNode.java
2013-04-08 11:24:17 -07:00
Fangjin Yang
925c104dd2
update emitter version and fix some NPEs
2013-04-08 11:23:08 -07:00
René Scheibe
6c27cd2e8e
cleanup - removed unused imports
2013-04-07 23:31:22 +02:00
René Scheibe
a6dc29cf85
cleanup - removed extra empty lines
...
- there should not be any adjacent empty lines
2013-04-07 14:32:09 +02:00
Nelson Ray
b1f2186c1f
use reservoir sampling in pickSegmentToMove to avoid IndexOutOfBoundsException when a segment gets dropped mid-run
2013-04-02 12:22:11 -07:00
René Scheibe
c9d541364e
fix cut-n-paste mistake in error message (issue #81 )
2013-04-02 18:42:30 +02:00
Fangjin Yang
8b100248a8
[maven-release-plugin] prepare for next development iteration
2013-04-01 17:17:50 -07:00
Fangjin Yang
bbb3991bc9
[maven-release-plugin] prepare release druid-0.3.33
2013-04-01 17:17:44 -07:00
Fangjin Yang
0730976c80
fix broken UT
2013-04-01 17:16:19 -07:00
Fangjin Yang
258f18471e
nodes will not try and download segments during startup
2013-04-01 16:55:03 -07:00
Fangjin Yang
e8c6936aca
[maven-release-plugin] prepare for next development iteration
2013-04-01 14:28:43 -07:00
Fangjin Yang
1f342dc0f2
[maven-release-plugin] prepare release druid-0.3.32
2013-04-01 14:28:36 -07:00
Fangjin Yang
b56ab64572
fix segment partition deletion bug
2013-04-01 14:26:02 -07:00
Fangjin Yang
30465b5c26
[maven-release-plugin] prepare for next development iteration
2013-03-28 19:48:46 -07:00
Fangjin Yang
1515dddfe0
[maven-release-plugin] prepare release druid-0.3.31
2013-03-28 19:48:39 -07:00
Fangjin Yang
04a46fb74f
[maven-release-plugin] prepare for next development iteration
2013-03-28 19:21:14 -07:00
Fangjin Yang
36601ee3aa
[maven-release-plugin] prepare release druid-0.3.30
2013-03-28 19:21:07 -07:00
Fangjin Yang
9e54e11827
[maven-release-plugin] prepare for next development iteration
2013-03-27 11:55:48 -07:00
Fangjin Yang
8fee044c68
[maven-release-plugin] prepare release druid-0.3.29
2013-03-27 11:55:38 -07:00
Fangjin Yang
5bdcc3613c
fix inconsistent druid http paths + fix master console bugs
2013-03-27 10:40:44 -07:00
Fangjin Yang
e785b87be0
[maven-release-plugin] prepare for next development iteration
2013-03-26 11:10:15 -07:00
Fangjin Yang
a97287fa50
[maven-release-plugin] prepare release druid-0.3.28
2013-03-26 11:10:04 -07:00
Fangjin Yang
a7d836a0dd
fix NPE for master balancing; introduce max number of workers for indexing service
2013-03-26 10:25:52 -07:00
Fangjin Yang
efd57dcf38
[maven-release-plugin] prepare for next development iteration
2013-03-25 13:17:50 -07:00
Fangjin Yang
3d7a0ea2d3
[maven-release-plugin] prepare release druid-0.3.27
2013-03-25 13:17:43 -07:00
fjy
05fdf08644
Merge pull request #107 from metamx/aggregator-close
...
add close method to aggregators
2013-03-25 13:09:19 -07:00
fjy
669f792668
Merge pull request #109 from metamx/fix-groupby-allgran-timestamp
...
fix output timestamps in groupby queries with granularity "all".
2013-03-25 12:58:32 -07:00
Nelson Ray
e57249bf15
refactor balancer metric emission code
2013-03-25 11:49:46 -07:00
Nelson Ray
a28de5fa88
fix per tier metrics emission for new balancer
2013-03-25 10:32:02 -07:00
xvrl
c420fe3b56
fix output timestamps in groupby queries with granularity "all".
2013-03-23 23:42:43 -07:00
xvrl
3687cba18e
close resources as we iterate over them
2013-03-22 17:42:10 -07:00
Eric Tschetter
35cfd1f186
[maven-release-plugin] prepare for next development iteration
2013-03-21 13:14:20 -05:00
Eric Tschetter
3860710f11
[maven-release-plugin] prepare release druid-0.3.26
2013-03-21 13:14:14 -05:00
xvrl
16e4cd4291
fix cursor overshooting granularity boundaries in IncrementalIndexStorageAdapter as well
2013-03-21 10:38:39 -07:00
xvrl
c0397aa67c
add test case for granularities not aligned with segment boundaries
2013-03-21 10:38:06 -07:00
xvrl
d722c99530
fix cursor overshooting granularity boundaries, due to incorrect gran.next() usage
2013-03-20 23:36:52 -07:00
xvrl
57f8810721
properly close rowiterators
2013-03-20 14:07:06 -07:00
xvrl
0b04114c12
more squigglies
2013-03-20 12:17:20 -07:00
xvrl
8eec41f934
squigglies ftw
2013-03-20 11:42:17 -07:00
Eric Tschetter
cc08d6029f
1) getDimensionCardinality() on QueryIndexStorageAdapter wasn't case insensitive.
2013-03-20 12:03:07 -05:00
xvrl
c68bd66945
add close method to aggregators
2013-03-19 16:03:48 -07:00
Eric Tschetter
f47319f118
[maven-release-plugin] prepare for next development iteration
2013-03-18 17:09:01 -05:00
Eric Tschetter
1066144b43
[maven-release-plugin] prepare release druid-0.3.25
2013-03-18 17:08:56 -05:00
Eric Tschetter
71aa8b8077
1) It's great when log lines don't generate exceptions
2013-03-18 17:06:51 -05:00
Eric Tschetter
57f52f71fc
1) Try checking for parent dirs first and creating if they aren't there when moving files from the legacy directory to the new one.
2013-03-18 17:04:11 -05:00
Eric Tschetter
cba404a65d
[maven-release-plugin] prepare for next development iteration
2013-03-18 13:02:40 -05:00
Eric Tschetter
29a0a7e6c6
[maven-release-plugin] prepare release druid-0.3.24
2013-03-18 13:02:33 -05:00
Fangjin Yang
b9de751b32
[maven-release-plugin] prepare for next development iteration
2013-03-15 14:32:31 -07:00
Fangjin Yang
57655f19c4
[maven-release-plugin] prepare release druid-0.3.23
2013-03-15 14:32:24 -07:00
Fangjin Yang
d316dfc8e1
Merge branch 'master' of github.com:metamx/druid
2013-03-15 14:29:19 -07:00
Fangjin Yang
6f771796b6
fix js bugs with indexer console
2013-03-15 14:28:55 -07:00
Eric Tschetter
7dacf952d6
[maven-release-plugin] prepare for next development iteration
2013-03-15 13:53:39 -05:00
Eric Tschetter
ff017fe72a
[maven-release-plugin] prepare release druid-0.3.22
2013-03-15 13:53:33 -05:00
Eric Tschetter
a933438e4e
1) Fix bugs with VersionConverterTask
...
2) Fix bugs with NPEs on indexing
2013-03-15 13:48:55 -05:00
Gian Merlino
f6752799bc
Merge branch 'master' into realtime-index-task
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/actions/SegmentInsertAction.java
2013-03-14 13:45:59 -07:00
cheddar
2a94bd508d
Merge pull request #104 from metamx/javascript-dimfilters
...
support for Javascript DimFilters
2013-03-14 11:47:55 -07:00
Gian Merlino
df8e4d4061
Merge branch 'master' into realtime-index-task
2013-03-13 22:43:01 -07:00
Eric Tschetter
4c165b4880
1) Better logging of master doing version checking
...
2) Exception out when the scv cannot find its indexer
2013-03-13 19:15:29 -05:00
Eric Tschetter
2bd34f1454
[maven-release-plugin] prepare for next development iteration
2013-03-13 18:32:00 -05:00
Eric Tschetter
b7960e202d
[maven-release-plugin] prepare release druid-0.3.21
2013-03-13 18:31:55 -05:00
Eric Tschetter
f1175389c4
[maven-release-plugin] prepare for next development iteration
2013-03-13 16:33:17 -05:00
Eric Tschetter
8aac482618
[maven-release-plugin] prepare release druid-0.3.20
2013-03-13 16:33:10 -05:00
Fangjin Yang
a49d0c5e4c
[maven-release-plugin] prepare for next development iteration
2013-03-11 17:20:16 -07:00
Fangjin Yang
97f828f6b1
[maven-release-plugin] prepare release druid-0.3.19
2013-03-11 17:20:09 -07:00
Gian Merlino
f8d9a3b19b
IndexGranularity: Fix increment for day, week
2013-03-11 11:08:44 -07:00
Eric Tschetter
d1161ae0ec
1) Master should create the config table if it's not there.
2013-03-08 12:41:46 -06:00
Nelson Ray
7a57609c5b
emit metrics corresponding to cost-based balancing
2013-03-07 16:31:57 -08:00
Nelson Ray
9f405c0b1e
fix constant segment balancing and separate out the differences between segment assignment and balancing
2013-03-07 16:31:57 -08:00
Eric Tschetter
1c3ef48f34
1) Adjust the Config stuff that WorkerSetupManager was using to be reusable
...
2) Use new ConfigManager to pull out a whitelist for the Master
2013-03-07 17:45:57 -06:00
Eric Tschetter
6864007c05
1) Remove Task parameter from other methods off of TaskToolbox that required it
...
2) Move getTaskDir() off of TaskConfig and onto TaskToolbox
3) Rename SegmentKiller interface to DataSegmentKiller
4) Change signature of "DataSegmentKiller.kill(Collection<DataSegment>) throws ServiceException" to just kill(DataSegment) throws SegmentLoadingException
5) Add various log messages
6) Update the version of the segment that has been converted
2013-03-07 17:44:06 -06:00
Eric Tschetter
c9b411c0ca
1) Remove the need for TaskActions to require a Task as a constructor parameter
2013-03-07 17:44:05 -06:00
Eric Tschetter
fd0fcc7319
Whitespace!
2013-03-07 16:18:31 -06:00
Eric Tschetter
82aa6ed7dd
1) Adjust the SingleSegmentLoader to be able to use cache locations from previous versions as well without re-downloading
2013-03-07 15:55:23 -06:00
Fangjin Yang
9d9efadf72
working indexer console
2013-03-06 10:38:40 -08:00
xvrl
4cb3ca00bc
support for javascript dimfilters
2013-03-06 09:59:53 -08:00
Fangjin Yang
552b365194
[maven-release-plugin] prepare for next development iteration
2013-03-04 15:11:37 -08:00
Fangjin Yang
fdb3911794
[maven-release-plugin] prepare release druid-0.3.18
2013-03-04 15:11:28 -08:00
Fangjin Yang
91f54a72ee
fix broken UT
2013-03-04 15:09:55 -08:00
Fangjin Yang
6977fe03bd
[maven-release-plugin] prepare for next development iteration
2013-03-04 13:08:26 -08:00
Fangjin Yang
d4f4e41320
[maven-release-plugin] prepare release druid-0.3.16
2013-03-04 13:08:09 -08:00
Fangjin Yang
92ab51e70c
fix bugs in balancing logic and removing auto-scaled node
2013-03-04 13:03:15 -08:00
Fangjin Yang
ee38d096c7
[maven-release-plugin] prepare for next development iteration
2013-03-02 09:29:04 -08:00
Fangjin Yang
a90d204ba2
[maven-release-plugin] prepare release druid-0.3.15
2013-03-02 09:28:51 -08:00
Fangjin Yang
879b2475d4
[maven-release-plugin] prepare for next development iteration
2013-03-01 19:53:13 -08:00
Fangjin Yang
e3ba6efb80
[maven-release-plugin] prepare release druid-0.3.14
2013-03-01 19:53:07 -08:00
Fangjin Yang
7f6747194d
[maven-release-plugin] prepare for next development iteration
2013-03-01 19:45:59 -08:00
Fangjin Yang
06a2a3a793
[maven-release-plugin] prepare release druid-0.3.13
2013-03-01 19:45:50 -08:00
Fangjin Yang
e0700bb161
[maven-release-plugin] prepare for next development iteration
2013-03-01 19:28:34 -08:00
Fangjin Yang
38c8ffb552
[maven-release-plugin] prepare release druid-0.3.12
2013-03-01 19:28:28 -08:00
Fangjin Yang
49e61d00fd
[maven-release-plugin] prepare for next development iteration
2013-03-01 19:05:46 -08:00
Fangjin Yang
7e316e3508
[maven-release-plugin] prepare release druid-0.3.11
2013-03-01 19:05:39 -08:00
Fangjin Yang
e2b4163e31
[maven-release-plugin] prepare for next development iteration
2013-03-01 18:24:09 -08:00
Fangjin Yang
023199702e
[maven-release-plugin] prepare release druid-0.3.10
2013-03-01 18:24:03 -08:00
Fangjin Yang
a574da55ed
fix broken UT
2013-03-01 18:22:30 -08:00
Gian Merlino
0092ad7efb
[maven-release-plugin] prepare for next development iteration
2013-03-01 16:57:47 -08:00
Gian Merlino
03ff72b7e2
[maven-release-plugin] prepare release druid-0.3.8
2013-03-01 16:57:38 -08:00
Gian Merlino
998ed0a8de
[maven-release-plugin] prepare for next development iteration
2013-03-01 14:43:20 -08:00
Gian Merlino
89d5ddf2f7
[maven-release-plugin] prepare release druid-0.3.7
2013-03-01 14:43:13 -08:00
Fangjin Yang
360c6a1df1
[maven-release-plugin] prepare for next development iteration
2013-03-01 13:55:07 -08:00
Fangjin Yang
a43dc44ebb
[maven-release-plugin] prepare release druid-0.3.6
2013-03-01 13:55:01 -08:00
Fangjin Yang
3264a689f1
[maven-release-plugin] prepare for next development iteration
2013-03-01 11:43:23 -08:00
Fangjin Yang
334fffdef5
[maven-release-plugin] prepare release druid-0.3.5
2013-03-01 11:43:16 -08:00
Gian Merlino
f862d9205d
Merge branch 'master' into hadoop-index-task
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
2013-02-27 06:53:06 -08:00
Nelson Ray
d1634fe5ef
fix cost balancing so assignment/balancing doesn't get stuck
2013-02-26 18:58:16 -08:00
Fangjin Yang
2e1e1753d5
[maven-release-plugin] prepare for next development iteration
2013-02-26 15:15:54 -08:00
Fangjin Yang
ab456cf3b1
[maven-release-plugin] prepare release druid-0.3.4
2013-02-26 15:15:36 -08:00
Nelson Ray
e67a8757ba
Merge remote-tracking branch 'origin/master' into new_balancer
...
Conflicts:
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-26 14:18:07 -08:00
Gian Merlino
d32a6284ce
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/worker/config/WorkerConfig.java
2013-02-26 13:03:56 -08:00
Gian Merlino
3129e5451b
Merge branch 'master' into task-stuff
2013-02-25 12:33:45 -08:00
Gian Merlino
9964e7dfe4
[maven-release-plugin] prepare for next development iteration
2013-02-25 12:24:24 -08:00
Gian Merlino
99c4f9446c
[maven-release-plugin] prepare release druid-0.3.3
2013-02-25 12:24:15 -08:00
Gian Merlino
49700dffe9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2013-02-25 12:18:31 -08:00
Eric Tschetter
e0f6df1a5c
[maven-release-plugin] prepare for next development iteration
2013-02-21 20:39:50 -06:00
Eric Tschetter
846bc0e4f0
[maven-release-plugin] prepare release druid-0.3.2
2013-02-21 20:39:45 -06:00
Eric Tschetter
699aadd2b1
[maven-release-plugin] prepare for next development iteration
2013-02-21 15:50:01 -06:00
Eric Tschetter
606a2e4b01
[maven-release-plugin] prepare release druid-0.3.1
2013-02-21 15:49:55 -06:00
Eric Tschetter
f8c54a72c2
1) Changes to allow for local storage
2013-02-21 15:47:01 -06:00
Eric Tschetter
dc3459d3f9
1) Initial commit of refactorings on top of housejester's to simplify the zipping and simplify the creation of other methods of loading data
2013-02-21 15:47:01 -06:00
James Estes
d1626576c0
Working toward making it easier to add new SegmentPullers.
...
1) Move the local cacheFile logic out of the S3 pullers into the SingleSegmentLoader
2) Make the S3SegmentPuller just pull down the file
3) Make the Loader do the unzip, ungzip, or rename
4) 2 and 3 make S3ZippedSegmentPuller not necessary (still there, just deprecated and empty)
4) Tweak the TaskToolbox so that the Pullers returned by getSegmentGetters behave the same as they did before
2013-02-21 15:47:01 -06:00
Deep Ganguli
60da9ffddb
Merge branch 'master' into hadoop-index-task
...
Conflicts:
common/src/main/java/com/metamx/druid/utils/JodaUtils.java
indexer/src/main/java/com/metamx/druid/indexer/DbUpdaterJob.java
indexer/src/main/java/com/metamx/druid/indexer/HadoopDruidIndexerConfig.java
indexer/src/main/java/com/metamx/druid/indexer/IndexGeneratorJob.java
2013-02-15 13:22:31 -08:00
Gian Merlino
824e3c0eb2
[maven-release-plugin] prepare for next development iteration
2013-02-15 13:03:39 -08:00
Gian Merlino
6bbc992101
[maven-release-plugin] prepare release druid-0.3.0
2013-02-15 13:03:32 -08:00
Nelson Ray
728e208f49
merge master
2013-02-15 11:39:22 -08:00
xvrl
05d347c304
Merge remote-tracking branch 'origin/master' into speedup-timeboundary
...
Conflicts:
client/pom.xml
common/pom.xml
druid-services/pom.xml
examples/pom.xml
examples/rand/pom.xml
examples/twitter/pom.xml
index-common/pom.xml
indexer/pom.xml
merger/pom.xml
pom.xml
realtime/pom.xml
server/pom.xml
2013-02-15 11:25:47 -08:00
Eric Tschetter
5ee7c0c264
1) Remove stray import of java.util.List
2013-02-15 12:15:53 -06:00
Gian Merlino
548c901c0b
Additional fix for columns with cardinality 0
2013-02-14 23:45:02 -08:00
Gian Merlino
3fed8ee0ce
[maven-release-plugin] prepare for next development iteration
2013-02-14 21:23:46 -08:00
Gian Merlino
ba7df5b709
[maven-release-plugin] prepare release druid-0.2.9
2013-02-14 21:23:41 -08:00
Gian Merlino
22d1f2a0c0
IndexMergerTest: Empty column test
2013-02-14 21:11:31 -08:00
Gian Merlino
152bf201ec
[maven-release-plugin] prepare for next development iteration
2013-02-14 16:27:20 -08:00
Gian Merlino
e56bc27f93
[maven-release-plugin] prepare release druid-0.2.8
2013-02-14 16:27:12 -08:00
Eric Tschetter
4f11eb5209
1) Fix alert sent from line DruidMasterBalancer:71 to not include all of the segments that a server is holding
2013-02-14 18:24:51 -06:00
Fangjin Yang
bf0d9c601d
Merge branch 'task-stuff' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/TaskToolbox.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/KillTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskMasterLifecycle.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskRunnerWorkQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorServletModule.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-14 15:42:07 -08:00
Gian Merlino
18daa19fc9
Merge branch 'master' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskHolder.java
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/AbstractTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskContext.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-14 13:10:29 -08:00
Nelson Ray
a616f86128
add DateTime import back
2013-02-14 09:09:22 -08:00
Nelson Ray
754b065c54
merge master
2013-02-14 09:08:25 -08:00
Eric Tschetter
c5b78e75cf
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-13 18:51:32 -08:00
Fangjin Yang
82f02a4fe4
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexDeterminePartitionsTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/Task.java
merger/src/main/java/com/metamx/druid/merger/common/task/V8toV9UpgradeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/MergerDBCoordinator.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
2013-02-13 16:34:14 -08:00
Fangjin Yang
c7b4973700
bug fixes for RTR
2013-02-13 14:19:11 -08:00
Nelson Ray
429e23d3b2
change break to continue in BalancerCostAnalyzer to skip calculating cost when it's not necessary
2013-02-13 08:57:39 -08:00
Eric Tschetter
a0b159fed7
1) Move up to Jackson 2.1
...
2) Some bugs were fixed, but they ended up getting mixed in with the Jackson upgrade :(
2013-02-12 20:58:17 -06:00
Nelson Ray
fcbac96193
revert change to findNewSegmentHome: now return single ServerHolder
2013-02-12 10:47:30 -08:00
xvrl
07b0b4611b
bump up to 0.3.0
2013-02-12 09:52:21 -08:00
Fangjin Yang
b27a459f71
more refactoring and UT passing at this point
2013-02-11 18:13:01 -08:00
xvrl
e9f546c4ca
add default implementations to QueryToolChest methods
2013-02-11 16:51:40 -08:00
Nelson Ray
695b78b19c
make findNewSegmentHome return a priority queue so assignment can assign replicants to other servers
2013-02-11 15:28:17 -08:00
xvrl
e10025c841
filter out unneeded segments for TimeBoundary query
2013-02-11 13:21:37 -08:00
Fangjin Yang
69d0d98df6
t Merge branch 'master' into indexing_refactor
2013-02-11 12:37:20 -08:00
Fangjin Yang
a9176e738a
Merge branch 'master' into indexing_refactor
2013-02-11 12:36:49 -08:00
Nelson Ray
1b9764fffb
add binary version to datasegments for balancer test
2013-02-11 11:36:45 -08:00
Nelson Ray
e12da51057
merge master
2013-02-11 11:28:50 -08:00
cheddar
05168808c2
Merge pull request #77 from metacret/master
...
Adding LinearShardSpec and ACL on S3 object
2013-02-11 11:12:31 -08:00
Fangjin Yang
e519f82e87
[maven-release-plugin] prepare for next development iteration
2013-02-11 10:58:51 -08:00
Fangjin Yang
73f7468a23
[maven-release-plugin] prepare release druid-0.2.7
2013-02-11 10:58:43 -08:00
Fangjin Yang
07c005a0f7
Merge branch 'master' of github.com:metamx/druid
2013-02-11 10:56:39 -08:00
Fangjin Yang
ac93f752b0
fix a case where a segment may get stuck in the terminate queue
2013-02-11 10:56:30 -08:00
Fangjin Yang
111df9a90e
Merge branch 'master' into indexing_refactor
...
Conflicts:
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
server/src/main/java/com/metamx/druid/master/DruidMaster.java
2013-02-07 10:35:32 -08:00
xvrl
6ff42f379c
fix cut-n-paste mistake in error message
2013-02-07 10:11:36 -08:00
Fangjin Yang
ec2b906fad
[maven-release-plugin] prepare for next development iteration
2013-02-06 18:43:42 -08:00
Fangjin Yang
22d9607169
[maven-release-plugin] prepare release druid-0.2.6
2013-02-06 18:43:34 -08:00
Fangjin Yang
267c797023
fix bug in throttle replicants
2013-02-06 18:40:29 -08:00
Fangjin Yang
e09c658352
[maven-release-plugin] prepare for next development iteration
2013-02-06 17:42:39 -08:00
Fangjin Yang
668353e895
[maven-release-plugin] prepare release druid-0.2.5
2013-02-06 17:42:33 -08:00
Fangjin Yang
0d303bee96
reset lifetimes for the replication throttle
2013-02-06 17:41:08 -08:00
Fangjin Yang
d023d219ae
[maven-release-plugin] prepare for next development iteration
2013-02-06 16:54:21 -08:00
Fangjin Yang
d0fdf4ccf0
[maven-release-plugin] prepare release druid-0.2.4
2013-02-06 16:54:16 -08:00
Fangjin Yang
7540dd58ca
fix broken UT
2013-02-06 16:52:24 -08:00
fjy
e2862fe1fb
Merge pull request #79 from metamx/bug_fixes
...
Throttle replicants that are created by rules
2013-02-06 16:50:04 -08:00
Fangjin Yang
84baef0462
fix some bugs from last commit
2013-02-06 13:57:56 -08:00
Fangjin Yang
97999c06c4
throttle replicants
2013-02-06 11:08:44 -08:00
Jae Hyeon Bae
4f70bbc687
adding linear shard spec and ACL for s3 bucket
2013-02-04 10:27:09 -08:00
Eric Tschetter
34b4383934
1) Adjust DataSegment to have a binaryVersion field that indicates the expected binary version of the segment files
...
2) Eliminate race condition from RemoteTaskRunnerTest.testAlreadyExecutedTask()
2013-02-01 18:59:33 -06:00
Eric Tschetter
c63eb2b77b
Merge branch 'master' of github.com:metamx/druid
2013-02-01 18:39:09 -06:00
xvrl
7ec3563e7b
[maven-release-plugin] prepare for next development iteration
2013-02-01 16:25:42 -08:00
xvrl
a239f62158
[maven-release-plugin] prepare release druid-0.2.3
2013-02-01 16:25:35 -08:00
Eric Tschetter
1a9edbebd3
1) whitespace
2013-02-01 16:21:26 -06:00
Eric Tschetter
5f2c3cec1b
Merge branch 'master' of github.com:metamx/druid
2013-02-01 16:19:26 -06:00
Eric Tschetter
7de5c7806e
1) Some whitespace changes
...
2) Fix up the alerting to actually include the segment and server for when the MasterBalancer has something stuck in the queue
2013-02-01 16:19:17 -06:00
Fangjin Yang
9e88197845
change endpoints for killing segments
2013-02-01 14:08:52 -08:00
Nelson Ray
1c9270add3
change format of line comment
2013-02-01 11:25:41 -08:00
Nelson Ray
0b61f96965
extra documentation
2013-02-01 11:25:05 -08:00
Fangjin Yang
f50c214514
Merge branch 'master' into task-stuff
2013-01-31 09:59:52 -08:00
Fangjin Yang
00e6ac1e10
[maven-release-plugin] prepare for next development iteration
2013-01-30 17:09:23 -08:00
Fangjin Yang
ef19de8270
[maven-release-plugin] prepare release druid-0.2.2
2013-01-30 17:09:18 -08:00
Fangjin Yang
fa733565e8
[maven-release-plugin] prepare for next development iteration
2013-01-30 16:28:28 -08:00
Fangjin Yang
0b116a8fe4
[maven-release-plugin] prepare release druid-0.2.1
2013-01-30 16:28:22 -08:00
Fangjin Yang
2a0de2090b
[maven-release-plugin] prepare for next development iteration
2013-01-30 13:47:15 -08:00
Fangjin Yang
049ca2bad4
[maven-release-plugin] prepare release druid-0.2.0
2013-01-30 13:47:08 -08:00
Nelson Ray
dc49cccf56
javadoc format
2013-01-29 18:32:52 -08:00
Nelson Ray
73f774e768
make masterbalancertest more stringent
2013-01-29 18:17:38 -08:00
Nelson Ray
a1ce4b294e
simplify balancer test
2013-01-29 18:16:49 -08:00
Nelson Ray
b205f7b3c4
use existing currentlyMovingSegments instead of creating a reundant segmentsBeingMoved
2013-01-29 18:16:34 -08:00
Nelson Ray
034d652569
make balancer tests take into account loadqueue status
2013-01-29 17:13:01 -08:00
cheddar
9a22e46eca
Merge pull request #71 from metamx/queryrunner-onerow-test
...
Bug fixes for incremental index storage adapter and one row indexes
2013-01-29 13:41:15 -08:00
cheddar
a7abd2ca86
Merge pull request #70 from metamx/timezone-tests
...
More time zone tests
2013-01-29 13:40:04 -08:00
Fangjin Yang
8fc10a61a0
bug fix for incremental index and 1 row
2013-01-29 13:23:57 -08:00
Fangjin Yang
41ca33c734
Merge branch 'master' of github.com:metamx/druid into task-stuff
2013-01-29 12:57:56 -08:00
Gian Merlino
61569f62be
TimeseriesQueryRunner tests with small incremental indexes
2013-01-29 12:54:52 -08:00
Eric Tschetter
da914b835e
1) Add caching for SegmentMetadataQuery
2013-01-29 13:04:57 -06:00
xvrl
27e09987c6
timezone test
2013-01-29 10:38:22 -08:00
Eric Tschetter
08d3253f3d
Merge branch 'master' of github.com:metamx/druid
2013-01-29 11:29:42 -06:00
Eric Tschetter
f2cce28297
1) Make SegmentMetadataQuery work
2013-01-29 11:29:26 -06:00
Nelson Ray
6438401a32
split pick segment logic into its own method
2013-01-28 15:06:09 -08:00
Eric Tschetter
7439a2e820
1) Make tests pass
2013-01-28 16:42:51 -06:00
xvrl
c0822325e7
proper groupby tests
2013-01-28 16:41:03 -06:00
cheddar
5f18f368e3
Merge pull request #61 from metamx/fix-cache-timezone
...
Cache did not preserve timezone information
2013-01-28 14:39:00 -08:00
Fangjin Yang
74057600f9
fix worker node still using db connection
2013-01-27 09:57:37 -08:00
Gian Merlino
f3b04d3f5f
S3SegmentKiller: Add TODO note
2013-01-25 11:33:50 -08:00
Gian Merlino
e6a618ca76
Merge branch 'killsegments' into task-stuff
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/common/TaskStatus.java
merger/src/main/java/com/metamx/druid/merger/common/task/DeleteTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexGeneratorTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/IndexTask.java
merger/src/main/java/com/metamx/druid/merger/common/task/MergeTask.java
merger/src/main/java/com/metamx/druid/merger/coordinator/LocalTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskQueue.java
merger/src/main/java/com/metamx/druid/merger/coordinator/exec/TaskConsumer.java
merger/src/main/java/com/metamx/druid/merger/worker/TaskMonitor.java
merger/src/test/java/com/metamx/druid/merger/coordinator/RemoteTaskRunnerTest.java
merger/src/test/java/com/metamx/druid/merger/coordinator/TaskQueueTest.java
2013-01-25 11:30:10 -08:00
xvrl
5e1cac6d9f
timezone test
2013-01-24 17:57:10 -08:00
Deep Ganguli
017d4779d6
Implemented Hadoop Index Task which takes as input a HadoopDruidIndexConfig and
...
generates index segments.
The HadoopIndexTask run method wraps a HadoopDruidIndexerJob run method. The
key modifications to the HadoopDruidIndexerJob are as follows:
- The UpDaterJobSpec field of the config that is used to set up the indexer job
is set to null. This ensures that the job does not push a list of published
segments to the database, in order to allow the indexing service to handle this
later.
- Set the version field of the config file based on the TaskContext. Also
changed config.setVersion method to take a string (as opposed to a Date) as
input, and propogated this change where necessary.
- Set the SegmentOutputDir field of the config file based on the TaskToolbox,
to allow the indexing service to handle where to write the segments too.
- Added a method to IndexGeneratorJob called getPublishedSegments, that simply
returns a list of published segments without publishing this list to the
database.
2013-01-23 19:27:14 -08:00
Eric Tschetter
67feee3cd6
1) Indexes don't always have an index.drd file anymore
2013-01-23 18:36:52 -06:00
xvrl
55ae4c87dd
timezone support in groupby query
2013-01-23 08:51:02 -08:00
xvrl
35058786d9
match query interval to granularity for this test
2013-01-23 08:50:43 -08:00
xvrl
86a6d112e3
proper groupby tests
2013-01-22 16:54:14 -08:00
Fangjin Yang
272d737517
cleaning up some interactions with RTR and workers
2013-01-22 16:21:38 -08:00
xvrl
f05c050c53
add test for timezone
2013-01-21 15:49:39 -08:00
Nelson Ray
94b72e8878
replace param BalancerCostAnalyzer getter with a factory
2013-01-21 15:32:29 -08:00
Fangjin Yang
bab9ee8827
Merge branch 'master' into killsegments
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
2013-01-21 14:47:49 -08:00
Nelson Ray
2d7113b263
huge simplification of balancing code
2013-01-21 14:28:25 -08:00
Eric Tschetter
5b1e03530c
1) Fix some bugs found by external test suite
2013-01-16 21:06:57 -06:00
Fangjin Yang
21613bc73b
initial commit to hard delete segments
2013-01-16 17:31:01 -08:00
Eric Tschetter
c8cb96b006
1) Remove vast majority of usages of IndexIO.mapDir() and deprecated it. IndexIO.loadIndex() is the new IndexIO.mapDir()
...
2) Fix bug with IndexMerger and null columns
3) Add QueryableIndexIndexableAdapter so that QueryableIndexes can be merged
4) Adjust twitter example to have multiple values for each hash tag
5) Adjusted GroupByQueryEngine to just drop dimensions that don't exist instead of throwing an NPE
2013-01-16 17:10:33 -06:00
Eric Tschetter
538d00e75e
Merge branch 'master' of github.com:metamx/druid
2013-01-16 10:02:01 -06:00
Eric Tschetter
8b31d8db9f
1) Adjust IndexMerger to create convert the indexes it creates from the old format to the new. This is done quite sub-optimally, but it will work for now...
2013-01-16 10:01:46 -06:00
Fangjin Yang
7e074e8158
fix pom breakage
2013-01-15 12:04:12 -08:00
Nelson Ray
e847faf02f
change name of MAX_SEGMENTS_TO_MOVE since it's not a final
2013-01-15 11:29:46 -08:00
Nelson Ray
ef9e077d9a
remove getConfig method from DruidMaster
2013-01-14 16:01:29 -08:00
Eric Tschetter
998f0bf3c8
1) Better error message when aggregators want more memory than is available.
2013-01-14 18:00:51 -06:00
Nelson Ray
5f40dc6d9c
put BalancerCostAnalyzer into params
2013-01-10 10:27:43 -08:00
Nelson Ray
617907d85b
move empty server holder list check outside of loop
2013-01-08 16:12:16 -08:00
Nelson Ray
4549c3a8c3
add some finals, reorganize
2013-01-08 09:07:43 -08:00
Nelson Ray
2b4dbdde03
lots of changes
2013-01-07 17:28:53 -08:00
Nelson Ray
6fda5330fd
finish making MAX_SEGMENTS_TO_MOVE a config item
2013-01-07 16:56:55 -08:00
Nelson Ray
ba82fe746a
put MAX_SEGMENTS_TO_MOVE in the config
2013-01-07 16:28:27 -08:00
Eric Tschetter
4ef31de62d
1) Fix stray import, take 2... yay IntelliJ
2013-01-07 13:11:28 -08:00
Nelson Ray
d2ff88e923
add more stats to be emitted
2013-01-07 13:09:18 -08:00
Nelson Ray
a7553776fa
make sampling of segments uniformly at random and add unit test
2013-01-07 12:51:51 -08:00
Eric Tschetter
b62bdf915b
1) Fix stray import... yay IntelliJ
2013-01-07 10:09:19 -08:00
Nelson Ray
2ea164acd9
get rid of extraneous analyzer.inits
2013-01-07 09:09:56 -08:00
Eric Tschetter
b184c46ae0
1) Introduce idea of value types and generic columns to the serialization format
...
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
2013-01-04 18:12:49 -06:00
Nelson Ray
402ddbf261
update
2013-01-04 11:20:06 -08:00
Nelson Ray
117cac2795
update
2013-01-03 14:12:10 -08:00
Nelson Ray
bb31fa7b68
update
2013-01-03 10:48:05 -08:00
xvrl
9627fdcd1c
[maven-release-plugin] prepare for next development iteration
2013-01-03 10:09:52 -08:00
xvrl
f2c2d3f4cf
[maven-release-plugin] prepare release druid-0.1.25
2013-01-03 10:09:43 -08:00
Nelson Ray
5441662cb8
Merge remote-tracking branch 'origin/master' into new_balancer
2013-01-03 09:10:48 -08:00
xvrl
a9b8d8f1ce
fix end of month bug in PeriodDropRule as well
2013-01-02 16:10:23 -08:00
xvrl
0543322c56
tested timestamps should be deterministic
2013-01-02 14:31:51 -08:00
xvrl
19c3fd3363
make reference timestamp explicit for time based rules
2013-01-02 14:27:34 -08:00
xvrl
2a188996cd
[maven-release-plugin] prepare for next development iteration
2012-12-31 14:56:10 -08:00
xvrl
6cdcc2becc
[maven-release-plugin] prepare release druid-0.1.24
2012-12-31 14:56:03 -08:00
xvrl
d0290b1975
fix flawed interval overlap rule test
...
fails around month end
2012-12-31 14:52:42 -08:00
Nelson Ray
72758e1345
update
2012-12-31 13:55:48 -08:00
Nelson Ray
3eda9dc5f2
working balancercostanalyzer
2012-12-31 11:08:37 -08:00
Nelson Ray
366216f9f3
initial cost analyzer commit
2012-12-28 16:30:21 -08:00
Gian Merlino
5c1b2360d6
- Add optional query/wait metric to MetricsEmittingQueryRunner
...
- Add MetricsEmittingExecutorService decorator, and use it on compute nodes
2012-12-20 16:11:08 -08:00
Fangjin Yang
5137af0750
[maven-release-plugin] prepare for next development iteration
2012-12-18 16:17:44 -08:00
Fangjin Yang
c579ba5a42
[maven-release-plugin] prepare release druid-0.1.23
2012-12-18 16:17:38 -08:00
Fangjin Yang
00ecf9d028
sorted db datasources for rules; smaller time boundary range
2012-12-18 16:16:03 -08:00
Fangjin Yang
090a5a105d
[maven-release-plugin] prepare for next development iteration
2012-12-18 14:19:09 -08:00
Fangjin Yang
fb8f4f1f41
[maven-release-plugin] prepare release druid-0.1.22
2012-12-18 14:19:02 -08:00
xvrl
2a7214089e
fix NPE in segmentMetadata query for null dimension values
2012-12-18 14:10:01 -08:00
Fangjin Yang
75d465aed0
[maven-release-plugin] prepare for next development iteration
2012-12-18 10:37:16 -08:00
Fangjin Yang
decc4e4c0e
[maven-release-plugin] prepare release druid-0.1.21
2012-12-18 10:37:08 -08:00
Fangjin Yang
e34944b1c5
[maven-release-plugin] prepare for next development iteration
2012-12-15 00:52:01 -08:00
Fangjin Yang
24f0207fd8
[maven-release-plugin] prepare release druid-0.1.20
2012-12-15 00:51:54 -08:00
Fangjin Yang
db556102db
fix typo
2012-12-15 00:50:28 -08:00
Fangjin Yang
f9386c3623
[maven-release-plugin] prepare for next development iteration
2012-12-15 00:44:58 -08:00
Fangjin Yang
4bcc8d27fa
[maven-release-plugin] prepare release druid-0.1.19
2012-12-15 00:44:50 -08:00
Fangjin Yang
585a812f6e
bug fixes again in load and drop
2012-12-15 00:43:16 -08:00
Fangjin Yang
5fff07a8b0
[maven-release-plugin] prepare for next development iteration
2012-12-14 22:56:49 -08:00
Fangjin Yang
2b2818dbaf
[maven-release-plugin] prepare release druid-0.1.18
2012-12-14 22:56:43 -08:00
Fangjin Yang
76807ae00e
fix overloading of segments
2012-12-14 22:55:12 -08:00
Fangjin Yang
b0e1f90e35
[maven-release-plugin] prepare for next development iteration
2012-12-14 22:07:57 -08:00
Fangjin Yang
ce8e02b5a3
[maven-release-plugin] prepare release druid-0.1.17
2012-12-14 22:07:50 -08:00
Fangjin Yang
884da334b2
fix for master getting stuck when assigning
2012-12-14 22:06:19 -08:00
Fangjin Yang
0821dce8c1
[maven-release-plugin] prepare for next development iteration
2012-12-14 21:29:20 -08:00
Fangjin Yang
a0d22a6646
[maven-release-plugin] prepare release druid-0.1.16
2012-12-14 21:29:14 -08:00
Fangjin Yang
6a11e99681
bug fixes for dropping segments and logging
2012-12-14 21:27:18 -08:00
Fangjin Yang
e37e24cc25
[maven-release-plugin] prepare for next development iteration
2012-12-14 17:19:59 -08:00
Fangjin Yang
4e78d8e734
[maven-release-plugin] prepare release druid-0.1.15
2012-12-14 17:19:53 -08:00
Fangjin Yang
74a977504e
bug fix for dropping segments in master
2012-12-14 17:06:03 -08:00
Eric Tschetter
b4a1bb4a00
[maven-release-plugin] prepare for next development iteration
2012-12-14 11:37:17 -06:00
Eric Tschetter
cb9b18c7c2
[maven-release-plugin] prepare release druid-0.1.14
2012-12-14 11:37:12 -06:00
Eric Tschetter
da137a4883
1) Optimize imports a bit
2012-12-14 11:34:23 -06:00
cheddar
7670c8483b
Merge pull request #45 from metamx/fjy
...
bug fixes for exceptions in rules
2012-12-14 09:32:37 -08:00
Fangjin Yang
18692a1214
better error emitting for mismatched rules
2012-12-14 09:25:39 -08:00
Fangjin Yang
8c7fd3bc49
more specific checks in rule test
2012-12-14 09:12:02 -08:00
Fangjin Yang
aa69ed5251
alert when no rules match; exceptions in rules no longer block
2012-12-14 09:05:25 -08:00
Eric Tschetter
29a64f50e2
[maven-release-plugin] prepare for next development iteration
2012-12-13 17:57:46 -06:00
Eric Tschetter
c82007973e
[maven-release-plugin] prepare release druid-0.1.13
2012-12-13 17:57:42 -06:00
Fangjin Yang
6729f166d9
[maven-release-plugin] prepare for next development iteration
2012-12-12 15:38:58 -08:00
Fangjin Yang
578cf94bc0
[maven-release-plugin] prepare release druid-0.1.12
2012-12-12 15:38:37 -08:00
xvrl
5dcf962eaa
register SegmentMetadataQueryRunnerFactory in ServerInit
2012-12-12 13:45:15 -08:00
xvrl
bfc9b63931
segment metadata query
2012-12-12 13:45:12 -08:00
Marshall Pierce
1a0ef0d5c3
Fix direct memory vm option name in warning message
2012-12-11 11:15:09 -08:00
Fangjin Yang
0105233fc3
[maven-release-plugin] prepare for next development iteration
2012-12-07 17:50:47 -08:00
Fangjin Yang
943fe5e6bf
[maven-release-plugin] prepare release druid-0.1.11
2012-12-07 17:50:41 -08:00
Fangjin Yang
4695fbfcd9
fix bug with comparing historical servers
2012-12-07 17:49:18 -08:00
Fangjin Yang
da4b31c28f
[maven-release-plugin] prepare for next development iteration
2012-12-07 15:18:19 -08:00
Fangjin Yang
6df32ac0b0
[maven-release-plugin] prepare release druid-0.1.10
2012-12-07 15:18:11 -08:00
Fangjin Yang
404f6322c6
bug fix for checking if a segment exists on a node
2012-12-07 15:16:34 -08:00
Fangjin Yang
9eb28254f8
[maven-release-plugin] prepare for next development iteration
2012-12-07 14:49:13 -08:00
Fangjin Yang
9ce536e040
[maven-release-plugin] prepare release druid-0.1.9
2012-12-07 14:49:04 -08:00
Fangjin Yang
5a511cbef0
bug fix for missing quotations creating default datasource
2012-12-07 14:47:38 -08:00
Fangjin Yang
35362b3e6b
[maven-release-plugin] prepare for next development iteration
2012-12-07 14:37:25 -08:00
Fangjin Yang
bd3c05dddb
[maven-release-plugin] prepare release druid-0.1.8
2012-12-07 14:37:18 -08:00
Fangjin Yang
f6519744e6
bug fix for using configurable default for datasource
2012-12-07 14:35:49 -08:00
Fangjin Yang
50a28f827e
[maven-release-plugin] prepare for next development iteration
2012-12-07 10:59:52 -08:00
Fangjin Yang
b59361065e
[maven-release-plugin] prepare release druid-0.1.7
2012-12-07 10:59:34 -08:00
Fangjin Yang
6028273332
defaults for rules to make startup a bit easier
2012-12-07 10:54:38 -08:00
Fangjin Yang
1ebd7a8a8a
fix wrong jquery version
2012-12-06 14:50:29 -08:00
Fangjin Yang
4a1abfc5dc
gui for rule configuratin; introducing jquery ui to master console
2012-12-05 15:39:59 -08:00
Fangjin Yang
5b8bd412c0
caching for rules; get rules from cache
2012-12-03 16:13:20 -08:00
Fangjin Yang
5822f4f5f7
refactor master to run rules before cleaning up; more master stats; general improvements
2012-12-03 14:43:04 -08:00
Fangjin Yang
91cd9e9536
fix broken UT
2012-11-30 13:17:14 -08:00
Fangjin Yang
aa8fb7b9a3
abstract the assign and drop logic; cleanups everywhere; http endpoints for getting and assigning rules
2012-11-30 11:28:10 -08:00
Fangjin Yang
2e5e1ce989
first commit of tiers for compute nodes; working UT at this point
2012-11-28 17:37:08 -08:00
Eric Tschetter
09a31e5995
Tests should build.
2012-11-28 12:35:24 -06:00
Eric Tschetter
b85c54d8a3
Merge branch 'master' of github.com:metamx/druid
2012-11-28 11:55:25 -06:00
Eric Tschetter
4de6ef5025
1) Remove the requirement on the segmentsPath zk property.
2012-11-28 11:54:59 -06:00
Gian Merlino
5902ccd0cb
DruidMasterSegmentMerger: Clearer method names and more tests
2012-11-27 15:46:03 -08:00
Eric Tschetter
0390e7fa5c
[maven-release-plugin] prepare for next development iteration
2012-11-21 22:13:15 -06:00
Eric Tschetter
93a9affa45
[maven-release-plugin] prepare release druid-0.1.6
2012-11-21 22:13:08 -06:00
Eric Tschetter
9c732e75a1
1) Add new SearchQueryDimFilter/SearchQueryFilter that will do a filter using the same queries that can be used for SearchQuery
2012-11-21 19:06:32 -06:00
Eric Tschetter
3a9591bda6
Merge branch 'master' of github.com:metamx/druid
2012-11-21 18:39:09 -06:00
Eric Tschetter
228b223509
1) Fix bug with Sinks that causes Realtime it to overwrite the first chunk of data when creating a segment to hand off to compute nodes.
...
2) Fix bug with RealtimePlumberSchool that can cause it to miss some previously-persisted segments when directory naming changes
3) Have the various runnables in RealtimePlumberSchool name the threads to make it a bit easier to debug
2012-11-21 18:38:57 -06:00
Gian Merlino
5e447937e2
[maven-release-plugin] prepare for next development iteration
2012-11-21 16:09:14 -08:00
Gian Merlino
32d46a3eec
[maven-release-plugin] prepare release druid-0.1.5
2012-11-21 16:09:08 -08:00
Eric Tschetter
06b8e4230c
1) Fix PostAggregations with GroupBy ( Fixes #24 )
...
2) Add GroupByTimeseriesQueryRunnerTest which wraps a GroupByQueryRunner to look like a TimeseriesQueryRunner and leverages the tests there to help verify behavior (#8 )
2012-11-21 12:51:43 -06:00
Eric Tschetter
0f63cb4f00
1) Have IndexGeneratorJob write the descriptors for each of the segments it creates to a path in the temporary working directory (generally HDFS)
...
2) Have the DbUpdaterJob read descriptors from the temporary working directory instead of looking in the final segment output location (often the eventually consistent S3)
3) 1 and 2 Fixes #30
2012-11-20 15:30:50 -06:00
Fangjin Yang
4131063049
Merge branch 'master' into fjy
2012-11-19 15:07:00 -08:00
Eric Tschetter
701cc9562b
1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up.
...
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29 , or at least clarifies the behavior...)
2012-11-19 17:01:17 -06:00
Gian Merlino
16835a1f84
Master: Add segment limit to merge selection algo
...
If we attempt to merge too many segments at once, we risk creating an
oversized indexing task.
2012-11-19 14:46:45 -08:00
Eric Tschetter
aa073e4e73
1) Add verification of memory settings to initialization. ( Fixes #21 )
2012-11-14 15:37:11 -06:00
Fangjin Yang
cf1c741f8a
[maven-release-plugin] prepare for next development iteration
2012-11-13 15:10:12 -08:00
Fangjin Yang
06bd77fca7
[maven-release-plugin] prepare release druid-0.1.4
2012-11-13 15:10:00 -08:00
Fangjin Yang
0d697ed104
fix rename of this node
2012-11-12 16:17:51 -08:00
Fangjin Yang
57468d39ef
reverting some of the last changes
2012-11-12 16:14:48 -08:00
Fangjin Yang
c20dccd0f4
modifying the way registering serdes works to hopefully be a bit easier to use
2012-11-12 13:58:43 -08:00
Marshall Pierce
447dfc0cc6
Switch to using javax.servlet:servlet-api:2.5 instead of two different versions of jetty's servlet-api distribution
2012-11-09 17:31:03 -08:00
Fangjin Yang
dd17992172
[maven-release-plugin] prepare for next development iteration
2012-11-09 15:50:24 -08:00
Fangjin Yang
47e71080db
[maven-release-plugin] prepare release druid-0.1.3
2012-11-09 15:50:16 -08:00
Fangjin Yang
5859599365
[maven-release-plugin] prepare for next development iteration
2012-11-09 13:41:04 -08:00
Fangjin Yang
56f2c15e21
[maven-release-plugin] prepare release druid-0.1.2
2012-11-09 13:40:57 -08:00
Fangjin Yang
0b7a3f259a
[maven-release-plugin] prepare for next development iteration
2012-11-09 10:10:32 -08:00
Fangjin Yang
baba4e7b75
[maven-release-plugin] prepare release druid-0.1.1
2012-11-09 10:10:26 -08:00
Fangjin Yang
03e5d74087
[maven-release-plugin] prepare for next development iteration
2012-11-09 09:33:06 -08:00
Fangjin Yang
8aff8decad
[maven-release-plugin] prepare release druid-0.1.0
2012-11-09 09:32:57 -08:00
Eric Tschetter
0a0e2a6cc1
1) Try to fix the dependency issues for running the HadoopDruidIndexer locally.
2012-11-08 17:06:02 -08:00
Fangjin Yang
09e81be14c
rename file
2012-11-08 13:42:58 -08:00
Fangjin Yang
3ddfdfc702
add missing header
2012-11-08 10:45:21 -08:00
Fangjin Yang
bff5a7d786
fix some merge issues
2012-11-08 10:43:39 -08:00
Fangjin Yang
8323feedb4
Merge branch 'master' into moarunittests
...
Conflicts:
client/src/main/java/com/metamx/druid/http/BrokerMain.java
2012-11-08 10:37:05 -08:00
Fangjin Yang
4d6cfdc774
Merge branch 'master' into autoscaling
2012-11-07 17:27:42 -08:00
Eric Tschetter
b1475219a9
Merge branch 'master' of git://github.com/tralfamadude/druid-1
...
Added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon.
2012-11-06 20:15:57 -08:00
Paul Baclace
cb02c6841b
removed recursive zpath listing from DruidSetup dump command.
2012-11-06 15:30:46 -08:00
Paul Baclace
86d2fb3550
added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon.
2012-11-06 00:16:27 -08:00
Fangjin Yang
499263386c
timeseries for druid + unit tests
2012-11-05 11:38:03 -08:00
Fangjin Yang
b1b611735e
Merge branch 'master' into autoscaling
...
Conflicts:
client/src/main/java/com/metamx/druid/http/BrokerMain.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-02 15:57:16 -07:00
Fangjin Yang
69b929fad5
Merge branch 'master' into moarunittests
...
Conflicts:
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
2012-11-01 16:41:13 -07:00
Fangjin Yang
09cc2e3353
Merge branch 'master' into moarunittests
...
Conflicts:
client/src/main/java/com/metamx/druid/http/BrokerMain.java
merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
2012-11-01 16:39:28 -07:00
Eric Tschetter
f07b15db1f
Undo massive import re-ordering
2012-11-01 18:38:27 -05:00
Eric Tschetter
8dd6f5c059
1) Refactor the BrokerMain into a BrokerMain and a BrokerNode to allow for extensions
...
2) Rename ServerMain to ComputeMain
3) Refactor ComputeMain to ComputeMain and ComputeNode to allow for extensions
4) Remove "TheSizeAdjuster", was old legacy stuff that's not relevant anymore
5) Fix bug with registering different IndexIO handlers
6) Adjust it so that when a query type is unknown, it returns an error message instead of just emitting an alert and returning nothing
7) Create super classes for the various *Node classes that allow for code sharing on pieces that are common
2012-11-01 18:20:38 -05:00
Fangjin Yang
83a51a801e
cleanup redirects and reannounce workers on connection loss
2012-10-31 16:43:47 -07:00
Ian Brandt
e7ec073d75
Cleaned Maven dependency configuration. In particular the 'mvn dependency:analyze' report no longer warns of used but undeclared dependencies.
2012-10-31 16:10:07 -07:00
Fangjin Yang
11c64593ae
improvements to the indexing service and auto scaling
2012-10-31 14:35:09 -07:00
Ian Brandt
e8f02219bb
Streamlined the Maven POMs, in particular the plugin configurations. The 'mvn versions:display-plugin-updates' report is now warning free, and there are no more Maven warnings reported in Eclipse.
2012-10-31 12:58:16 -07:00
Fangjin Yang
d0f66c1fb3
bug fix for merging with empty strings
2012-10-31 10:28:24 -07:00
Ian Brandt
e24948319b
Removed some dead/unused code.
2012-10-31 00:00:30 -07:00
Ian Brandt
c5e2a88187
Resolved dead code warning by null checking the correct variable according to the associated log statement.
2012-10-30 23:59:31 -07:00
Ian Brandt
35f04b4df6
Fixed some minor potential resource leaks.
2012-10-30 23:26:21 -07:00
Ian Brandt
0d24df7628
Removed unused imports.
2012-10-30 22:22:46 -07:00
Trotter Cashion
2c341144bf
Allow blank mysql passwords
2012-10-26 10:46:30 -07:00
Matt Croydon
6a9d86e1ef
Removed unused import in IndexMerger that was causing build.sh to fail with Java 1.7.0_04-b21 on OS X 10.7.4.
2012-10-24 11:46:32 -05:00
Eric Tschetter
27999caca0
1) Create LICENSE
...
2) Attach copyright and notice of license to files
2012-10-24 05:09:47 -04:00
Eric Tschetter
9d41599967
Initial commit of OSS Druid Code
2012-10-24 03:39:51 -04:00