Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
12,507 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

3994 Commits

Author SHA1 Message Date
Nelson Ray ba82fe746a put MAX_SEGMENTS_TO_MOVE in the config 2013-01-07 16:28:27 -08:00
Eric Tschetter 4ef31de62d 1) Fix stray import, take 2... yay IntelliJ 2013-01-07 13:11:28 -08:00
Nelson Ray d2ff88e923 add more stats to be emitted 2013-01-07 13:09:18 -08:00
Nelson Ray a7553776fa make sampling of segments uniformly at random and add unit test 2013-01-07 12:51:51 -08:00
Eric Tschetter b62bdf915b 1) Fix stray import... yay IntelliJ 2013-01-07 10:09:19 -08:00
Nelson Ray 2ea164acd9 get rid of extraneous analyzer.inits 2013-01-07 09:09:56 -08:00
Eric Tschetter b184c46ae0 1) Introduce idea of value types and generic columns to the serialization format 
2) Create SegmentLoader interface for the loading of segments!
3) Setup serialization of new format and conversion function inside IndexIO
4) Make some adjustments to the twitter demo to make it make a bit more sense based on some email feedback from someone kicking the tires.
 2013-01-04 18:12:49 -06:00
Nelson Ray 402ddbf261 update 2013-01-04 11:20:06 -08:00
Nelson Ray 117cac2795 update 2013-01-03 14:12:10 -08:00
Nelson Ray bb31fa7b68 update 2013-01-03 10:48:05 -08:00
xvrl 9627fdcd1c [maven-release-plugin] prepare for next development iteration 2013-01-03 10:09:52 -08:00
xvrl f2c2d3f4cf [maven-release-plugin] prepare release druid-0.1.25 2013-01-03 10:09:43 -08:00
Nelson Ray 5441662cb8 Merge remote-tracking branch 'origin/master' into new_balancer 2013-01-03 09:10:48 -08:00
xvrl a9b8d8f1ce fix end of month bug in PeriodDropRule as well 2013-01-02 16:10:23 -08:00
xvrl 0543322c56 tested timestamps should be deterministic 2013-01-02 14:31:51 -08:00
xvrl 19c3fd3363 make reference timestamp explicit for time based rules 2013-01-02 14:27:34 -08:00
xvrl 2a188996cd [maven-release-plugin] prepare for next development iteration 2012-12-31 14:56:10 -08:00
xvrl 6cdcc2becc [maven-release-plugin] prepare release druid-0.1.24 2012-12-31 14:56:03 -08:00
xvrl d0290b1975 fix flawed interval overlap rule test 
fails around month end
 2012-12-31 14:52:42 -08:00
Nelson Ray 72758e1345 update 2012-12-31 13:55:48 -08:00
Nelson Ray 3eda9dc5f2 working balancercostanalyzer 2012-12-31 11:08:37 -08:00
Nelson Ray 366216f9f3 initial cost analyzer commit 2012-12-28 16:30:21 -08:00
Gian Merlino 5c1b2360d6 - Add optional query/wait metric to MetricsEmittingQueryRunner 
- Add MetricsEmittingExecutorService decorator, and use it on compute nodes
 2012-12-20 16:11:08 -08:00
Fangjin Yang 5137af0750 [maven-release-plugin] prepare for next development iteration 2012-12-18 16:17:44 -08:00
Fangjin Yang c579ba5a42 [maven-release-plugin] prepare release druid-0.1.23 2012-12-18 16:17:38 -08:00
Fangjin Yang 00ecf9d028 sorted db datasources for rules; smaller time boundary range 2012-12-18 16:16:03 -08:00
Fangjin Yang 090a5a105d [maven-release-plugin] prepare for next development iteration 2012-12-18 14:19:09 -08:00
Fangjin Yang fb8f4f1f41 [maven-release-plugin] prepare release druid-0.1.22 2012-12-18 14:19:02 -08:00
xvrl 2a7214089e fix NPE in segmentMetadata query for null dimension values 2012-12-18 14:10:01 -08:00
Fangjin Yang 75d465aed0 [maven-release-plugin] prepare for next development iteration 2012-12-18 10:37:16 -08:00
Fangjin Yang decc4e4c0e [maven-release-plugin] prepare release druid-0.1.21 2012-12-18 10:37:08 -08:00
Fangjin Yang e34944b1c5 [maven-release-plugin] prepare for next development iteration 2012-12-15 00:52:01 -08:00
Fangjin Yang 24f0207fd8 [maven-release-plugin] prepare release druid-0.1.20 2012-12-15 00:51:54 -08:00
Fangjin Yang db556102db fix typo 2012-12-15 00:50:28 -08:00
Fangjin Yang f9386c3623 [maven-release-plugin] prepare for next development iteration 2012-12-15 00:44:58 -08:00
Fangjin Yang 4bcc8d27fa [maven-release-plugin] prepare release druid-0.1.19 2012-12-15 00:44:50 -08:00
Fangjin Yang 585a812f6e bug fixes again in load and drop 2012-12-15 00:43:16 -08:00
Fangjin Yang 5fff07a8b0 [maven-release-plugin] prepare for next development iteration 2012-12-14 22:56:49 -08:00
Fangjin Yang 2b2818dbaf [maven-release-plugin] prepare release druid-0.1.18 2012-12-14 22:56:43 -08:00
Fangjin Yang 76807ae00e fix overloading of segments 2012-12-14 22:55:12 -08:00
Fangjin Yang b0e1f90e35 [maven-release-plugin] prepare for next development iteration 2012-12-14 22:07:57 -08:00
Fangjin Yang ce8e02b5a3 [maven-release-plugin] prepare release druid-0.1.17 2012-12-14 22:07:50 -08:00
Fangjin Yang 884da334b2 fix for master getting stuck when assigning 2012-12-14 22:06:19 -08:00
Fangjin Yang 0821dce8c1 [maven-release-plugin] prepare for next development iteration 2012-12-14 21:29:20 -08:00
Fangjin Yang a0d22a6646 [maven-release-plugin] prepare release druid-0.1.16 2012-12-14 21:29:14 -08:00
Fangjin Yang 6a11e99681 bug fixes for dropping segments and logging 2012-12-14 21:27:18 -08:00
Fangjin Yang e37e24cc25 [maven-release-plugin] prepare for next development iteration 2012-12-14 17:19:59 -08:00
Fangjin Yang 4e78d8e734 [maven-release-plugin] prepare release druid-0.1.15 2012-12-14 17:19:53 -08:00
Fangjin Yang 74a977504e bug fix for dropping segments in master 2012-12-14 17:06:03 -08:00
Eric Tschetter b4a1bb4a00 [maven-release-plugin] prepare for next development iteration 2012-12-14 11:37:17 -06:00
Eric Tschetter cb9b18c7c2 [maven-release-plugin] prepare release druid-0.1.14 2012-12-14 11:37:12 -06:00
Eric Tschetter da137a4883 1) Optimize imports a bit 2012-12-14 11:34:23 -06:00
cheddar 7670c8483b Merge pull request #45 from metamx/fjy 
bug fixes for exceptions in rules
 2012-12-14 09:32:37 -08:00
Fangjin Yang 18692a1214 better error emitting for mismatched rules 2012-12-14 09:25:39 -08:00
Fangjin Yang 8c7fd3bc49 more specific checks in rule test 2012-12-14 09:12:02 -08:00
Fangjin Yang aa69ed5251 alert when no rules match; exceptions in rules no longer block 2012-12-14 09:05:25 -08:00
Eric Tschetter 29a64f50e2 [maven-release-plugin] prepare for next development iteration 2012-12-13 17:57:46 -06:00
Eric Tschetter c82007973e [maven-release-plugin] prepare release druid-0.1.13 2012-12-13 17:57:42 -06:00
Fangjin Yang 6729f166d9 [maven-release-plugin] prepare for next development iteration 2012-12-12 15:38:58 -08:00
Fangjin Yang 578cf94bc0 [maven-release-plugin] prepare release druid-0.1.12 2012-12-12 15:38:37 -08:00
xvrl 5dcf962eaa register SegmentMetadataQueryRunnerFactory in ServerInit 2012-12-12 13:45:15 -08:00
xvrl bfc9b63931 segment metadata query 2012-12-12 13:45:12 -08:00
Marshall Pierce 1a0ef0d5c3 Fix direct memory vm option name in warning message 2012-12-11 11:15:09 -08:00
Fangjin Yang 0105233fc3 [maven-release-plugin] prepare for next development iteration 2012-12-07 17:50:47 -08:00
Fangjin Yang 943fe5e6bf [maven-release-plugin] prepare release druid-0.1.11 2012-12-07 17:50:41 -08:00
Fangjin Yang 4695fbfcd9 fix bug with comparing historical servers 2012-12-07 17:49:18 -08:00
Fangjin Yang da4b31c28f [maven-release-plugin] prepare for next development iteration 2012-12-07 15:18:19 -08:00
Fangjin Yang 6df32ac0b0 [maven-release-plugin] prepare release druid-0.1.10 2012-12-07 15:18:11 -08:00
Fangjin Yang 404f6322c6 bug fix for checking if a segment exists on a node 2012-12-07 15:16:34 -08:00
Fangjin Yang 9eb28254f8 [maven-release-plugin] prepare for next development iteration 2012-12-07 14:49:13 -08:00
Fangjin Yang 9ce536e040 [maven-release-plugin] prepare release druid-0.1.9 2012-12-07 14:49:04 -08:00
Fangjin Yang 5a511cbef0 bug fix for missing quotations creating default datasource 2012-12-07 14:47:38 -08:00
Fangjin Yang 35362b3e6b [maven-release-plugin] prepare for next development iteration 2012-12-07 14:37:25 -08:00
Fangjin Yang bd3c05dddb [maven-release-plugin] prepare release druid-0.1.8 2012-12-07 14:37:18 -08:00
Fangjin Yang f6519744e6 bug fix for using configurable default for datasource 2012-12-07 14:35:49 -08:00
Fangjin Yang 50a28f827e [maven-release-plugin] prepare for next development iteration 2012-12-07 10:59:52 -08:00
Fangjin Yang b59361065e [maven-release-plugin] prepare release druid-0.1.7 2012-12-07 10:59:34 -08:00
Fangjin Yang 6028273332 defaults for rules to make startup a bit easier 2012-12-07 10:54:38 -08:00
Fangjin Yang 1ebd7a8a8a fix wrong jquery version 2012-12-06 14:50:29 -08:00
Fangjin Yang 4a1abfc5dc gui for rule configuratin; introducing jquery ui to master console 2012-12-05 15:39:59 -08:00
Fangjin Yang 5b8bd412c0 caching for rules; get rules from cache 2012-12-03 16:13:20 -08:00
Fangjin Yang 5822f4f5f7 refactor master to run rules before cleaning up; more master stats; general improvements 2012-12-03 14:43:04 -08:00
Fangjin Yang 91cd9e9536 fix broken UT 2012-11-30 13:17:14 -08:00
Fangjin Yang aa8fb7b9a3 abstract the assign and drop logic; cleanups everywhere; http endpoints for getting and assigning rules 2012-11-30 11:28:10 -08:00
Fangjin Yang 2e5e1ce989 first commit of tiers for compute nodes; working UT at this point 2012-11-28 17:37:08 -08:00
Eric Tschetter 09a31e5995 Tests should build. 2012-11-28 12:35:24 -06:00
Eric Tschetter b85c54d8a3 Merge branch 'master' of github.com:metamx/druid 2012-11-28 11:55:25 -06:00
Eric Tschetter 4de6ef5025 1) Remove the requirement on the segmentsPath zk property. 2012-11-28 11:54:59 -06:00
Gian Merlino 5902ccd0cb DruidMasterSegmentMerger: Clearer method names and more tests 2012-11-27 15:46:03 -08:00
Eric Tschetter 0390e7fa5c [maven-release-plugin] prepare for next development iteration 2012-11-21 22:13:15 -06:00
Eric Tschetter 93a9affa45 [maven-release-plugin] prepare release druid-0.1.6 2012-11-21 22:13:08 -06:00
Eric Tschetter 9c732e75a1 1) Add new SearchQueryDimFilter/SearchQueryFilter that will do a filter using the same queries that can be used for SearchQuery 2012-11-21 19:06:32 -06:00
Eric Tschetter 3a9591bda6 Merge branch 'master' of github.com:metamx/druid 2012-11-21 18:39:09 -06:00
Eric Tschetter 228b223509 1) Fix bug with Sinks that causes Realtime it to overwrite the first chunk of data when creating a segment to hand off to compute nodes. 
2) Fix bug with RealtimePlumberSchool that can cause it to miss some previously-persisted segments when directory naming changes
3) Have the various runnables in RealtimePlumberSchool name the threads to make it a bit easier to debug
 2012-11-21 18:38:57 -06:00
Gian Merlino 5e447937e2 [maven-release-plugin] prepare for next development iteration 2012-11-21 16:09:14 -08:00
Gian Merlino 32d46a3eec [maven-release-plugin] prepare release druid-0.1.5 2012-11-21 16:09:08 -08:00
Eric Tschetter 06b8e4230c 1) Fix PostAggregations with GroupBy (Fixes #24) 
2) Add GroupByTimeseriesQueryRunnerTest which wraps a GroupByQueryRunner to look like a TimeseriesQueryRunner and leverages the tests there to help verify behavior (#8)
 2012-11-21 12:51:43 -06:00
Eric Tschetter 0f63cb4f00 1) Have IndexGeneratorJob write the descriptors for each of the segments it creates to a path in the temporary working directory (generally HDFS) 
2) Have the DbUpdaterJob read descriptors from the temporary working directory instead of looking in the final segment output location (often the eventually consistent S3)
3) 1 and 2 Fixes #30
 2012-11-20 15:30:50 -06:00
Fangjin Yang 4131063049 Merge branch 'master' into fjy 2012-11-19 15:07:00 -08:00
Eric Tschetter 701cc9562b 1) Adjust the StorageAdapters to lowercase names of metrics and dimensions before looking them up. 
2) Add some docs to InputRow/Row to indicate that column names passed into the methods are *always* lowercase and that the rows need to act accordingly. (fixes #29, or at least clarifies the behavior...)
 2012-11-19 17:01:17 -06:00
Gian Merlino 16835a1f84 Master: Add segment limit to merge selection algo 
If we attempt to merge too many segments at once, we risk creating an
oversized indexing task.
 2012-11-19 14:46:45 -08:00
Eric Tschetter aa073e4e73 1) Add verification of memory settings to initialization. (Fixes #21) 2012-11-14 15:37:11 -06:00
Fangjin Yang cf1c741f8a [maven-release-plugin] prepare for next development iteration 2012-11-13 15:10:12 -08:00
Fangjin Yang 06bd77fca7 [maven-release-plugin] prepare release druid-0.1.4 2012-11-13 15:10:00 -08:00
Fangjin Yang 0d697ed104 fix rename of this node 2012-11-12 16:17:51 -08:00
Fangjin Yang 57468d39ef reverting some of the last changes 2012-11-12 16:14:48 -08:00
Fangjin Yang c20dccd0f4 modifying the way registering serdes works to hopefully be a bit easier to use 2012-11-12 13:58:43 -08:00
Marshall Pierce 447dfc0cc6 Switch to using javax.servlet:servlet-api:2.5 instead of two different versions of jetty's servlet-api distribution 2012-11-09 17:31:03 -08:00
Fangjin Yang dd17992172 [maven-release-plugin] prepare for next development iteration 2012-11-09 15:50:24 -08:00
Fangjin Yang 47e71080db [maven-release-plugin] prepare release druid-0.1.3 2012-11-09 15:50:16 -08:00
Fangjin Yang 5859599365 [maven-release-plugin] prepare for next development iteration 2012-11-09 13:41:04 -08:00
Fangjin Yang 56f2c15e21 [maven-release-plugin] prepare release druid-0.1.2 2012-11-09 13:40:57 -08:00
Fangjin Yang 0b7a3f259a [maven-release-plugin] prepare for next development iteration 2012-11-09 10:10:32 -08:00
Fangjin Yang baba4e7b75 [maven-release-plugin] prepare release druid-0.1.1 2012-11-09 10:10:26 -08:00
Fangjin Yang 03e5d74087 [maven-release-plugin] prepare for next development iteration 2012-11-09 09:33:06 -08:00
Fangjin Yang 8aff8decad [maven-release-plugin] prepare release druid-0.1.0 2012-11-09 09:32:57 -08:00
Eric Tschetter 0a0e2a6cc1 1) Try to fix the dependency issues for running the HadoopDruidIndexer locally. 2012-11-08 17:06:02 -08:00
Fangjin Yang 09e81be14c rename file 2012-11-08 13:42:58 -08:00
Fangjin Yang 3ddfdfc702 add missing header 2012-11-08 10:45:21 -08:00
Fangjin Yang bff5a7d786 fix some merge issues 2012-11-08 10:43:39 -08:00
Fangjin Yang 8323feedb4 Merge branch 'master' into moarunittests 
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
 2012-11-08 10:37:05 -08:00
Fangjin Yang 4d6cfdc774 Merge branch 'master' into autoscaling 2012-11-07 17:27:42 -08:00
Eric Tschetter b1475219a9 Merge branch 'master' of git://github.com/tralfamadude/druid-1 
Added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon.
 2012-11-06 20:15:57 -08:00
Paul Baclace cb02c6841b removed recursive zpath listing from DruidSetup dump command. 2012-11-06 15:30:46 -08:00
Paul Baclace 86d2fb3550 added DruidSetup.java and install/druid_setup.sh to run it for Druid ensemble setup (put properties to zk, create zk zpaths, prep db, and dump properties/paths in zk); property druid.zk.paths.base can establish a namespace for a druid ensemble allowing more than one ensemble to use the same zookeepers; this namespace is the base zpath prefix for paths with properties in the form druid.zk.paths.*Path which are normally set automatically now; if druid.zk.paths.*Path are explicitly set, then all of them must be set with absolute paths to ensure careful attention; ZkSetup is now deprecated, use DruidSetup instead; examples/twitter RealtimeStandaloneMain will exit properly now after a kill -15 (control-c or kill pid) instead of hanging on an unstopped daemon. 2012-11-06 00:16:27 -08:00
Fangjin Yang 499263386c timeseries for druid + unit tests 2012-11-05 11:38:03 -08:00
Fangjin Yang b1b611735e Merge branch 'master' into autoscaling 
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
 2012-11-02 15:57:16 -07:00
Fangjin Yang 69b929fad5 Merge branch 'master' into moarunittests 
Conflicts:
	merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
	merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
 2012-11-01 16:41:13 -07:00
Fangjin Yang 09cc2e3353 Merge branch 'master' into moarunittests 
Conflicts:
	client/src/main/java/com/metamx/druid/http/BrokerMain.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/RemoteTaskRunner.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/TaskInventoryManager.java
	merger/src/main/java/com/metamx/druid/merger/coordinator/http/IndexerCoordinatorNode.java
	merger/src/main/java/com/metamx/druid/merger/worker/Worker.java
	merger/src/main/java/com/metamx/druid/merger/worker/WorkerCuratorCoordinator.java
	merger/src/main/java/com/metamx/druid/merger/worker/http/WorkerNode.java
 2012-11-01 16:39:28 -07:00
Eric Tschetter f07b15db1f Undo massive import re-ordering 2012-11-01 18:38:27 -05:00
Eric Tschetter 8dd6f5c059 1) Refactor the BrokerMain into a BrokerMain and a BrokerNode to allow for extensions 
2) Rename ServerMain to ComputeMain
3) Refactor ComputeMain to ComputeMain and ComputeNode to allow for extensions
4) Remove "TheSizeAdjuster", was old legacy stuff that's not relevant anymore
5) Fix bug with registering different IndexIO handlers
6) Adjust it so that when a query type is unknown, it returns an error message instead of just emitting an alert and returning nothing
7) Create super classes for the various *Node classes that allow for code sharing on pieces that are common
 2012-11-01 18:20:38 -05:00
Fangjin Yang 83a51a801e cleanup redirects and reannounce workers on connection loss 2012-10-31 16:43:47 -07:00
Ian Brandt e7ec073d75 Cleaned Maven dependency configuration. In particular the 'mvn dependency:analyze' report no longer warns of used but undeclared dependencies. 2012-10-31 16:10:07 -07:00
Fangjin Yang 11c64593ae improvements to the indexing service and auto scaling 2012-10-31 14:35:09 -07:00
Ian Brandt e8f02219bb Streamlined the Maven POMs, in particular the plugin configurations. The 'mvn versions:display-plugin-updates' report is now warning free, and there are no more Maven warnings reported in Eclipse. 2012-10-31 12:58:16 -07:00
Fangjin Yang d0f66c1fb3 bug fix for merging with empty strings 2012-10-31 10:28:24 -07:00
Ian Brandt e24948319b Removed some dead/unused code. 2012-10-31 00:00:30 -07:00
Ian Brandt c5e2a88187 Resolved dead code warning by null checking the correct variable according to the associated log statement. 2012-10-30 23:59:31 -07:00
Ian Brandt 35f04b4df6 Fixed some minor potential resource leaks. 2012-10-30 23:26:21 -07:00
Ian Brandt 0d24df7628 Removed unused imports. 2012-10-30 22:22:46 -07:00
Trotter Cashion 2c341144bf Allow blank mysql passwords 2012-10-26 10:46:30 -07:00
Matt Croydon 6a9d86e1ef Removed unused import in IndexMerger that was causing build.sh to fail with Java 1.7.0_04-b21 on OS X 10.7.4. 2012-10-24 11:46:32 -05:00
Eric Tschetter 27999caca0 1) Create LICENSE 
2) Attach copyright and notice of license to files
 2012-10-24 05:09:47 -04:00
Eric Tschetter 9d41599967 Initial commit of OSS Druid Code 2012-10-24 03:39:51 -04:00