Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,441 Commits 111 Branches 503 Tags 726 MiB
Commit Graph

1239 Commits

Author SHA1 Message Date
Xavier Léauté 5ad5d7d18b Merge pull request #1379 from flowbehappy/fix-hadoop-ha 
bug fix: hdfs task log and indexing task not work properly with Hadoop HA
 2015-05-22 09:14:50 -04:00
flow 07659f30ab bug fix: hdfs task log and indexing task not work properly with Hadoop HA 2015-05-21 20:49:42 +08:00
Charles Allen 29ba05c04f Abstractify HadoopTask 
* Add `invokeForeignLoader` to commonize the way tasks are attempted to be launched in a foreign class loader
* Add `buildClassLoader` to accomplish the common tasks for hadoop jobs when building a ClassLoader
 2015-05-14 17:04:43 -07:00
fjy 7a6acf5c1b update pom to 0.8 2015-05-11 19:41:58 -06:00
Gian Merlino e69d82a2b4 Realtime: Delay firehose connection until job is started. 
Some firehoses (like the Kafka firehose) acquire input resources when they
connect, so it helps to delay this until after plumber.startJob() runs.
 2015-05-04 10:54:07 -07:00
Xavier Léauté 721505c017 Merge pull request #1208 from druid-io/rework-metrics 
Schemaless metrics + additional metrics for things we care about
 2015-04-27 15:04:54 -07:00
fjy 963e5765bf Schemaless metrics + additional metrics for things we care about 2015-04-27 13:39:40 -07:00
Charles Allen 633fdb029e Add option to ConvertSegmentTask to skip validation 
* Validation is enabled by default
 2015-04-27 08:37:55 -07:00
Charles Allen 29341f9837 Fix random unit test failure from NoopTask ID collision 2015-04-24 13:07:48 -07:00
Xavier Léauté f73f14ab91 Merge pull request #1297 from metamx/versionConverterTaskUpdates 
Update VersionConverterTask for IndexSpec and allowing Forced updates
 2015-04-20 16:44:35 -07:00
Charles Allen 7479ac9012 Update VersionConverterTask for IndexSepc and allowing Forced updates 2015-04-20 16:17:06 -07:00
fjy d260515a43 update druid-api version 2015-04-17 14:58:35 -07:00
Xavier Léauté ea5572d001 Merge pull request #1271 from metamx/strictErrorChecking 
Add stricter checking for potential coding errors
 2015-04-15 15:21:41 -07:00
Charles Allen abdeaa0746 Add stricter checking for potential coding errors 
Can use via `mvn clean compile test-compile -P strict'
 2015-04-15 14:52:25 -07:00
Xavier Léauté 3a3046ccf3 add support for dimension compression 
- compression for single-value dimensions using CompressedVSizeIntsIndexedSupplier
- makes dimension compression configurable via IndexSpec
- IndexSpec also enables configuring bitmap and metric compression
 2015-04-14 10:44:18 -07:00
fjy 195a3b8bb8 ignore rows with invalid interval 2015-04-06 16:08:40 -07:00
Fangjin Yang 208e307915 Merge pull request #1251 from metamx/uriSegmentLoaders 
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
 2015-03-30 17:43:51 -07:00
fjy aea7f9d192 [maven-release-plugin] prepare for next development iteration 2015-03-30 16:35:24 -07:00
fjy 060d7aef03 [maven-release-plugin] prepare release druid-0.7.1 2015-03-30 16:35:20 -07:00
Charles Allen 1c6cbea89c Revert "Revert "Overhaul of SegmentPullers to add consistency and retries"" 
This reverts commit f904bc7858.
 2015-03-30 13:40:04 -07:00
Fangjin Yang f904bc7858 Revert "Overhaul of SegmentPullers to add consistency and retries" 2015-03-30 13:15:50 -07:00
Charles Allen 6d407e8677 Add URI handling to SegmentPullers 
* Requires https://github.com/druid-io/druid-api/pull/37
* Requires https://github.com/metamx/java-util/pull/22
* Moves the puller logic to use a more standard workflow going through java-util helpers instead of re-writing the handlers for each impl
  * General workflow goes like this: 1) LoadSpec makes sure the correct Puller is called with the correct parameters. 2) The Puller sets up general information like how to make an InputStream, how to find a file name (for .gz files for example), and when to retry. 3) CompressionUtils does most of the heavy lifting when it can
 2015-03-30 12:33:23 -07:00
msprunck 942c17a2aa Remove timeline chunk count assumptions. 
* Replace with generic iterables
 2015-03-24 22:40:49 +01:00
fjy b389cfe404 [maven-release-plugin] prepare for next development iteration 2015-03-19 12:38:17 -07:00
fjy 60e7d543cc [maven-release-plugin] prepare release druid-0.7.1-rc1 2015-03-19 12:38:13 -07:00
Xavier Léauté 9d6b728054 Merge pull request #1215 from metamx/log-audit-IP-Address 
Add remote ip address in audit log.
 2015-03-17 13:59:31 -07:00
fjy bfe10bd156 This fixes arbitrary gran spec breaking 2015-03-17 12:19:43 -07:00
nishantmonu51 f9821d242f also log author ip address in audit log 2015-03-17 23:15:15 +05:30
Xavier Léauté ddfafa0711 randomize task ID to fix spurious test failure 2015-03-12 18:08:48 -07:00
Fangjin Yang a508c0955f Merge pull request #1195 from himanshug/task_storage_config_fix 
correctly parse recentlyFinishedThreshold from config
 2015-03-12 16:50:49 -07:00
nishantmonu51 3ec4a30ab5 initial commit 
review comments

more refactoring and cleaning of redundant code

add UT + docs + more refactoring

fixes + review comments

more cleanup

end points to fetch history

review comments

remove unnecessary changes

review comments rename header name

review comments + add test for MetadataRulesManager

review comments docs
 2015-03-12 22:50:29 +05:30
Himanshu Gupta 23545fc01c correctly parse recentlyFinishedThreshold from config 2015-03-12 09:46:57 -05:00
Xavier Léauté d3f5bddc5c Add ability to apply extraction functions to the time dimension 
- Moves DimExtractionFn under a more generic ExtractionFn interface to
  support extracting dimension values other than strings
- pushes down extractionFn to the storage adapter from query engine
- 'dimExtractionFn' parameter has been deprecated in favor of 'extractionFn'
- adds a TimeFormatExtractionFn, allowing to project the '__time' dimension
- JavascriptDimExtractionFn renamed to JavascriptExtractionFn, adding
  support for any dimension value types that map directly to Javascript
- update documentation for time column extraction and related changes
 2015-03-11 16:45:42 -07:00
Gian Merlino b00c243786 Need a null check for iamProfile. 2015-03-10 17:52:15 -07:00
Gian Merlino b810cdfe58 EC2AutoScaler: Allow setting "iamProfile". 2015-03-10 17:41:35 -07:00
Gian Merlino d102a89760 Fix license on EC2AutoScalerSerdeTest. 2015-03-10 17:31:30 -07:00
Gian Merlino 9235b45063 EC2AutoScaler: Support for setting subnetId. 2015-03-10 11:29:56 -07:00
Xavier Léauté 113d204b10 break up archive task actions, which was missed in #566a3a6112 2015-03-04 13:19:52 -08:00
Himanshu Gupta bd5cecdd44 UTs update for indexing service 2015-02-25 15:45:58 -08:00
Xavier Léauté b167dcf82c [maven-release-plugin] prepare for next development iteration 2015-02-23 14:28:06 -08:00
Xavier Léauté e81ac2ba43 [maven-release-plugin] prepare release druid-0.7.0 2015-02-23 14:27:58 -08:00
Fangjin Yang 25db9abb7f Merge pull request #1138 from metamx/better-default-hostname 
Better default hostname
 2015-02-18 17:37:34 -08:00
Xavier Léauté 53d2b961c5 default to canonical hostname instead of localhost 2015-02-18 16:44:48 -08:00
Xavier Léauté 78df7f6165 Move Druid release artifacts to Sonatype 
- Switch to using Druid parent POM
- Add required fields for Sonatype
- Common plugin versions and settings have been moved to the parent pom
- Cleanup artifacts and POMs for consistent formatting
- Remove org.hyperic.sigar dependency and update docs to reflect necessary jars to add at runtime when sigar is needed
 2015-02-13 14:26:31 -08:00
fjy d29740ed9f [maven-release-plugin] prepare for next development iteration 2015-02-12 16:16:00 -08:00
fjy 211fd15b7e [maven-release-plugin] prepare release druid-0.7.0-rc3 2015-02-12 16:15:56 -08:00
fjy 708759e1e0 Update http-client to 1.0.0 2015-02-10 13:36:47 -08:00
Charles Allen 79a3e8f59f Fix overriding base of IndexerZkConfig to be absolute instead of relative 
* Updated docs to clarify ZK config behavior
* Added unit tests for this case
 2015-02-04 13:04:06 -08:00
fjy 1f12c5b2f1 [maven-release-plugin] prepare for next development iteration 2015-02-03 12:06:49 -08:00
fjy e82d431be7 [maven-release-plugin] prepare release druid-0.7.0-rc2 2015-02-03 12:06:41 -08:00
Fangjin Yang 92e616de11 Merge pull request #1077 from metamx/remove-unused-imports 
remove unused imports
 2015-02-02 10:45:27 -08:00
nishantmonu51 ba932bb1f2 remove unused imports 2015-02-02 21:53:39 +05:30
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
Xavier Léauté a01a22dba1 Merge pull request #1074 from druid-io/overlord-leader 
Add an endpoint to return the overlord leader
 2015-01-30 13:44:49 -08:00
Xavier Léauté bd49528805 Merge pull request #1073 from druid-io/fix-statusPath 
Fix worker status path announcement with indexer zk config
 2015-01-30 12:51:21 -08:00
fjy 649f285feb Add an endpoint to return the overlord leader 2015-01-30 12:37:48 -08:00
fjy bc1405bee0 fix worker status path announcement with indexer zk config 2015-01-30 12:26:08 -08:00
Xavier Léauté 2c2771b90e Make dynamic worker selection actually work 2015-01-27 14:17:42 -08:00
nishantmonu51 0f3eac4705 fix dimension exclusion 2015-01-23 00:31:23 +05:30
fjy 1f94de22c6 [maven-release-plugin] prepare for next development iteration 2015-01-20 14:23:55 -08:00
fjy 17476edc31 [maven-release-plugin] prepare release druid-0.7.0-rc1 2015-01-20 14:23:51 -08:00
fjy 2d516fa591 Add a new equal distribution strategy for assigning tasks 2015-01-20 13:12:22 -08:00
Xavier Léauté cd9635ff5e Merge pull request #1034 from druid-io/minor-rename 
minor rename of things in hadoop ingestion config to match 0.6.x
 2015-01-15 15:46:13 -08:00
fjy ccddbf8747 minor rename of things in hadoop ingestion config to match 0.6.x 2015-01-15 14:04:55 -08:00
Fangjin Yang 5bfcc43377 Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate 
Update all String conversions to and from byte[] to use the java-util StringUtils functions
 2015-01-15 13:50:27 -08:00
Charles Allen 67757b6aea Change IndexerZkConfig to use @JacksonInject instead of just straight @Inject 
* Updated IndexerZkConfig to use no setters, and take all arguments from constructor instead
* Also added more unit tests
 2015-01-08 11:11:17 -08:00
Charles Allen f6fbb733b8 Added a few places where tests were using Object instead of Module 2015-01-05 13:47:25 -08:00
Charles Allen b1b5c9099e Update all String conversions to and from byte[] to use the java-util StringUtils functions 
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
 2015-01-05 11:22:32 -08:00
Charles Allen 65286a24e0 Change zk configs to use Jackson injection instead of Skife 
* Also added generic config testing class JsonConfigTesterBase
 2014-12-29 10:36:12 -08:00
Fangjin Yang af1185b58c Merge pull request #969 from metamx/fixRemoteLogViewing 
Remove try-with-resources for log stream in WokerResource
 2014-12-15 16:26:02 -07:00
Charles Allen 54068e8b1d Remove try-with-resources for log stream in WokerResource 2014-12-15 15:24:59 -08:00
fjy ac407fb6ba clean up defaults 2014-12-15 15:05:02 -08:00
fjy e872952390 fix working path default bug 2014-12-15 14:51:58 -08:00
Fangjin Yang b3fe91bb50 Merge pull request #830 from metamx/union-merge-on-historical 
Union merge on historical
 2014-12-15 13:36:47 -07:00
Charles Allen bed3e7e1d2 Merge pull request #966 from metamx/fix-tasklog-streaming 
fix task log streaming
 2014-12-14 09:31:41 -08:00
Xavier Léauté bd91a40491 fix task log streaming 2014-12-13 15:22:55 -08:00
Xavier Léauté 092dfe0309 fix IndexTaskTest tmp dir 
- Create local firehose files in a clean temp directory to avoid
firehose reading other random temp files that start with 'druid'
 2014-12-12 17:05:45 -08:00
fjy 123db3da4d fix another broken ut 2014-12-09 15:47:28 -08:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
Charles Allen a0f9f9877e Changed all "application/json" to MediaType.APPLICATION_JSON except for in druid.js 2014-12-08 14:21:49 -08:00
nishantmonu51 6e03a6245f Merge branch 'master' into onheap-incremental-index 2014-12-05 10:40:28 +05:30
Xavier Léauté 7cd45a6e1f IncrementalIndex throws exception if limit exceeded 
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
 2014-12-04 14:38:16 -08:00
Charles Allen 18234a2f00 Fix confusing error message in HadoopIndexTask 2014-12-04 10:57:57 -08:00
Gian Merlino 20a7239ffd Replace google-http-client imports with real guava imports. 2014-12-04 10:57:57 -08:00
Xavier Léauté 0c521e0a77 update joda-time and fix min/max instant 2014-12-04 10:57:56 -08:00
Fangjin Yang 27d4b2bdea Merge pull request #934 from metamx/fix-hadoop-metadata-injection 
metadata update handler injection not needed for indexing service
 2014-12-04 10:45:20 -07:00
Xavier Léauté 2e6c254937 metadata injection not needed for indexing service 2014-12-03 15:09:31 -08:00
Charles Allen 325a5c4abc Update ForkingTaskRunner to remove @Deprecated Files method usage 2014-12-03 13:18:33 -08:00
xvrl 2681da4420 Merge pull request #929 from metamx/google-cleanup 
Replace google-http-client imports with real guava imports.
 2014-12-03 11:50:19 -08:00
Charles Allen b6f71d3fd6 Fix confusing error message in HadoopIndexTask 2014-12-03 11:11:53 -08:00
Gian Merlino d388a8fe89 Replace google-http-client imports with real guava imports. 2014-12-03 10:52:57 -08:00
nishantmonu51 da8bd7836b Introduce buffer size 2014-12-03 16:28:22 +05:30
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
Xavier Léauté d23fd1e1ab make host+port more explicit 
- document the behavior for node host/port initialization
- throw exception if settings make no sense
- fixes announcement for nodes without host/port defaults
- makes code clearer as to when host vs. host+port are used
 2014-11-26 22:03:25 -08:00
Fangjin Yang 3ff569ef2d Merge pull request #879 from metamx/rtr-with-pref 
Rewrite autoscaling and enable easier configuration of worker selection and autoscaling behaviour
 2014-11-24 17:54:28 -07:00
fjy 3808411340 address some cr 2014-11-24 16:54:47 -08:00
fjy 13cae41f6c Merge branch 'master' into refactor-examples 2014-11-24 11:00:26 -08:00
fjy 9b701bbc76 a few more code review fixes 2014-11-24 10:54:29 -08:00
fjy 1aaea9a0d7 address code review 2014-11-24 10:52:30 -08:00
fjy 8ee4d12562 Refactor structure for examples and extensions 2014-11-21 14:45:24 -08:00
fjy 580e1172c1 move IndexTask to use hashed partition; fixes #815 2014-11-21 11:15:25 -08:00
fjy fdeab0c6af make Druid case sensitive 2014-11-19 14:27:31 -08:00
Fangjin Yang 109fdf0b34 Merge pull request #852 from metamx/druid-0.7.x-TaskLogStreamer 
(DO NOT MERGE YET) Update logging to https://github.com/druid-io/druid-api/pull/27
 2014-11-19 15:03:12 -07:00
Fangjin Yang 590d31799e Merge pull request #876 from metamx/remove-backwards-compatible 
Remove backwards compatible
 2014-11-19 14:33:14 -07:00
fjy 64719b15e0 rewrite autoscaling with tests 2014-11-18 15:41:06 -08:00
xvrl a96eaeb036 Merge pull request #882 from metamx/now_with_OPEN_SOURCE 
Added src jar build to maven poms and re-formatted to conform to style guidelines.
 2014-11-18 13:00:04 -08:00
Charles Allen dc66e1708e Added src jar build to maven poms and re-formatted to conform to style guidelines. 2014-11-18 09:05:30 -08:00
fjy c91310914b fix a few naming things 2014-11-17 16:05:18 -08:00
fjy 32600e10bb address code review 2014-11-17 15:55:22 -08:00
fjy 1af6b337f2 optionally choose what worker to send tasks to 2014-11-17 14:50:56 -08:00
Xavier Léauté d914afe1cd make defaultVersion configurable for non-jar testing 2014-11-17 13:54:32 -08:00
nishantmonu51 0c2d06475d merge from master 2014-11-17 19:19:18 +05:30
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
nishantmonu51 ad1dd161e7 formatting 2014-11-17 17:44:26 +05:30
Fangjin Yang d4ca805cb9 Merge pull request #806 from metamx/rtac-error-messages 
RemoteTaskActionClient: Better error messages for non-2xx responses.
 2014-11-13 11:57:50 -07:00
Charles Allen a89b539b4f Merge pull request #823 from metamx/roaring 
Configurable bitmap indexes: roaring and concise
 2014-11-11 17:26:38 -08:00
fjy 1cc162727b address code review 2014-11-11 14:05:37 -08:00
Xavier Léauté aeb194ad12 fix injection for real 2014-11-07 11:43:53 -08:00
Fangjin Yang 2336e6c167 Merge pull request #758 from metamx/jisoo-metadata 
make metadata storage pluggable
 2014-11-07 11:30:11 -07:00
nishantmonu51 fd8eb7742b handle union query on realtime node 2014-11-07 23:27:50 +05:30
Xavier Léauté 5bda4ee1dd global task entry type 2014-11-06 17:08:20 -08:00
Charles Allen d52530e0de Update logging to https://github.com/druid-io/druid-api/pull/27 2014-11-06 11:43:25 -08:00
Xavier Léauté 9bc20ef8bf prefer druid.curator.compress to druid.indexer.runner.compressZnodes 2014-11-06 11:28:51 -08:00
Xavier Léauté 350bb09605 refactor sql storage to abstract task storage 2014-11-05 17:19:37 -08:00
Xavier Léauté 1872b8f979 make it easier to test 2014-10-31 14:49:07 -07:00
Xavier Léauté 97a2f5af4a rename db->metadata 2014-10-31 10:54:33 -07:00
Xavier Léauté 9c06db021f rename db->metadata postgres->postgresql 2014-10-31 10:30:27 -07:00
Xavier Léauté fb4d41cedb make the injection gods happy 2014-10-30 21:16:36 -07:00
Xavier Léauté 377151beda better abstraction for metadatastorage 2014-10-30 18:23:35 -07:00
Xavier Léauté 3cc1b2e690 remove mysql dependencies + fix jdbc class loading 2014-10-29 17:30:43 -07:00
jisookim0513 aa754b86e8 build success! 2014-10-24 11:28:42 -07:00
fjy bef74104d9 merge with 0.7.x and resolve any conflicts 2014-10-23 17:24:06 -07:00
Gian Merlino ca0a4bd8a5 RemoteTaskActionClient: Better error messages for non-2xx responses. 2014-10-22 18:02:22 -07:00
fjy 3b29e77866 [maven-release-plugin] prepare for next development iteration 2014-10-22 16:25:32 -07:00
fjy dcab2997f2 [maven-release-plugin] prepare release druid-0.6.160 2014-10-22 16:25:27 -07:00
jisookim0513 02e79d6b15 attempted to solve merge-conflict; IncrementalIndex has unresolved classes after updates - needs to be fixed 2014-10-22 00:18:17 -07:00
jisookim0513 37979282fe enabled ansi-quote in mysql; insert statement should now work 2014-10-21 00:09:19 -07:00
jisookim0513 b8cbe2457a fixed variable name 2014-10-17 00:13:30 -07:00
jisookim0513 7d5c5f2083 fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider 2014-10-17 00:10:36 -07:00
nishantmonu51 f4a97aebbc fix rollup for hashed partitions 
truncate timestamp while calculating the partitionNumber
 2014-10-15 22:32:56 +05:30
nishantmonu51 bce388fb27 merge changes from 0.7.x branch 2014-10-14 18:46:02 +05:30
nishantmonu51 b5d66381f3 more cleanup 2014-10-14 18:32:40 +05:30
fjy a4c8f04409 [maven-release-plugin] prepare for next development iteration 2014-10-13 12:50:45 -07:00
fjy 7fd1747ffa [maven-release-plugin] prepare release druid-0.6.159 2014-10-13 12:50:41 -07:00
nishantmonu51 454acd3f5a remove backwards compatible code 
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
 2014-10-13 19:30:44 +05:30
Gian Merlino e1fedbe741 RemoteTaskRunner should respect worker version changes (fixes #787). 2014-10-12 11:27:44 -07:00
jisookim0513 521398267c fixed inconsistent variable names 2014-10-10 17:00:50 -07:00
fjy d0fbd5f729 [maven-release-plugin] prepare for next development iteration 2014-10-09 14:24:53 -07:00
fjy dada60a301 [maven-release-plugin] prepare release druid-0.6.158 2014-10-09 14:24:48 -07:00
fjy 8a4b32c482 [maven-release-plugin] prepare for next development iteration 2014-10-07 14:30:26 -07:00
fjy 96e29c2fdd [maven-release-plugin] prepare release druid-0.6.157 2014-10-07 14:30:20 -07:00
fjy c7b4d5b7b4 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
 2014-10-02 18:12:10 -07:00
fjy c3bea245a7 fix up some bugs 2014-09-30 17:20:52 -07:00
fjy a0782d4c54 fix compile 2014-09-30 16:56:44 -07:00
jisookim0513 0e50852985 fixed MetadataTaskStorage and handler 2014-09-30 14:09:23 -07:00
Gian Merlino 0781781b99 Merge pull request #766 from metamx/extend-rtr 
make the worker selection strategy in remotetaskrunner extendable
 2014-09-30 12:52:12 -07:00
fjy fab7caafff final code reviews 2014-09-30 12:32:14 -07:00
fjy 06757034f2 add default impl 2014-09-30 11:54:29 -07:00
fjy 2b2b028e5c why am i so bad at coding 2014-09-30 11:53:18 -07:00
fjy b1b9e0a267 i suck 2014-09-30 11:45:19 -07:00
fjy 575d51b0ce fix compilation error 2014-09-30 11:44:50 -07:00
fjy 4c23a5e9f6 address cr again 2014-09-30 11:40:29 -07:00
fjy 55db06ccb1 address cr 2014-09-30 10:29:02 -07:00
nishantmonu51 358ff915bb fix merge conflicts 2014-09-30 22:19:18 +05:30
nishantmonu51 2789536bed merge changes from druid-0.7.x 2014-09-30 22:05:49 +05:30
nishantmonu51 61c7fd2e6e make ingestOffheap tuneable 2014-09-30 15:30:02 +05:30
fjy 3f524baa67 Merge pull request #769 from metamx/task-logs-fixes 
TaskLogs fixes and cleanups.
 2014-09-29 17:37:00 -06:00
Gian Merlino 1e6ce8ac9a TaskLogs fixes and cleanups. 
- Fix negative offsets in FileTaskLogs and HdfsTaskLogs.
- Consolidate file offset code into LogUtils (currently used in two places).
- Clean up style for HdfsTaskLogs and related classes.
- Remove unused code in ForkingTaskRunner.
 2014-09-29 16:20:34 -07:00
fjy 4a09678739 make the selection strategy in rtr extendable 2014-09-29 14:24:02 -07:00
jisookim0513 74565c9371 cleaned up the code 2014-09-27 13:10:01 -07:00
jisookim0513 6a641621b2 finished merging into druid-0.7.x; derby not working (to be fixed) 2014-09-26 14:24:53 -07:00
jisookim0513 43cc6283d3 trying to revert files that have overwritten changes 2014-09-26 12:38:04 -07:00
fjy eaf0a48b92 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-09-26 11:39:24 -07:00
jisookim0513 a5d02b9030 second attempt to fix merge-conflicts 2014-09-24 15:59:00 -07:00
jisookim0513 3bf39cc9f8 attempted to fix merge-conflicts 2014-09-24 15:55:42 -07:00
Xavier Léauté 7cfe86b7bc [maven-release-plugin] prepare for next development iteration 2014-09-23 16:31:25 -07:00
Xavier Léauté 80166314d9 [maven-release-plugin] prepare release druid-0.6.156 2014-09-23 16:31:21 -07:00
fjy 4235e14fc6 [maven-release-plugin] prepare for next development iteration 2014-09-23 16:12:43 -07:00
fjy 576e9b8a84 [maven-release-plugin] prepare release druid-0.6.155 2014-09-23 16:12:39 -07:00
nishantmonu51 f51ab84386 merge changes from druid-0.7.x 2014-09-22 23:48:45 +05:30
jisookim0513 273205f217 initial attempt for abstraction; druid cluster works with Derby as a default 2014-09-19 17:39:59 -07:00
nishantmonu51 8eb6466487 revert buffer size and add back rowFlushBoundary 2014-09-19 23:06:04 +05:30
nishantmonu51 e6d93a3070 fix NPE 
fix NPE when the dimension of metric is not present one of the segments
to be reIndexed.
 2014-09-17 15:57:58 +05:30
fjy a17794a516 [maven-release-plugin] prepare for next development iteration 2014-09-16 15:22:55 -07:00
fjy f9c91ace3f [maven-release-plugin] prepare release druid-0.6.154 2014-09-16 15:22:49 -07:00
nishantmonu51 f006de8639 fix #732 
fix metric list discovery
 2014-09-16 22:12:36 +05:30
Xavier Léauté c8b8e3f6e9 negating compare is bad 2014-09-15 13:00:06 -07:00
Xavier Léauté 137ad50bf1 classes that should be static 2014-09-15 13:00:06 -07:00
Xavier Léauté e57e2d97ba make constants final 2014-09-15 12:53:40 -07:00
Xavier Léauté cfa92e8217 fix incorrect nullable annotations 2014-09-15 12:13:52 -07:00
fjy 469ccbbe5e Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
 2014-09-11 16:20:50 -07:00
Xavier Léauté 32e256a842 [maven-release-plugin] prepare for next development iteration 2014-09-10 17:24:15 -07:00
Xavier Léauté f7dc13d240 [maven-release-plugin] prepare release druid-0.6.153 2014-09-10 17:24:11 -07:00
fjy fec7b43fcb make making v9 segments something completely configurable 2014-09-10 15:28:30 -07:00
fjy 351afb8be7 allow legacy index generator 2014-09-09 17:04:35 -07:00
Xavier Léauté 22553b0b28 [maven-release-plugin] prepare for next development iteration 2014-09-05 11:07:22 -07:00
Xavier Léauté 570a7a07bf [maven-release-plugin] prepare release druid-0.6.152 2014-09-05 11:07:14 -07:00
fjy e24be03a38 [maven-release-plugin] prepare for next development iteration 2014-09-04 14:10:13 -07:00
fjy a933ea1866 [maven-release-plugin] prepare release druid-0.6.151 2014-09-04 14:10:09 -07:00
fjy f473e62f1f Merge pull request #714 from metamx/cleanup-javadocs 
Cleanup javadocs
 2014-09-04 12:28:10 -06:00
Xavier Léauté 508e982190 Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/main/java/io/druid/server/ClientQuerySegmentWalker.java
	services/pom.xml
 2014-08-30 22:42:36 -07:00
Gian Merlino 96d7b18163 [maven-release-plugin] prepare for next development iteration 2014-08-29 15:29:58 -07:00
Gian Merlino 860cf36ab1 [maven-release-plugin] prepare release druid-0.6.150 2014-08-29 15:29:54 -07:00
Xavier Léauté 58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté ac05836833 make Java 8 javadoc happy 2014-08-29 13:58:50 -07:00
Xavier Léauté daca178631 [maven-release-plugin] prepare for next development iteration 2014-08-29 10:43:19 -07:00
Xavier Léauté 0793a5aee6 [maven-release-plugin] prepare release druid-0.6.149 2014-08-29 10:43:15 -07:00
Gian Merlino 8bcb73a0eb Java programs are best run with a java command. 2014-08-28 18:10:01 -07:00
Gian Merlino 5564a67bfc File.pathSeparator is better than ":". 2014-08-28 18:09:33 -07:00
Gian Merlino 68aeafaacd Allow indexing tasks to specify extra classpaths. 
This could be used by Hadoop tasks to reference configs for different clusters, assuming
that the possible configs have been pre-distributed to middle managers.
 2014-08-28 18:00:26 -07:00
Xavier Léauté 1d6842c7f0 [maven-release-plugin] prepare for next development iteration 2014-08-27 12:22:53 -07:00
Xavier Léauté 575951f99a [maven-release-plugin] prepare release druid-0.6.148 2014-08-27 12:22:49 -07:00
fjy 00f7077f50 [maven-release-plugin] prepare for next development iteration 2014-08-22 08:28:56 -07:00
fjy df488563e4 [maven-release-plugin] prepare release druid-0.6.147 2014-08-22 08:28:51 -07:00
fjy df48b48e35 [maven-release-plugin] prepare for next development iteration 2014-08-21 12:50:33 -07:00
fjy 4f0b994037 [maven-release-plugin] prepare release druid-0.6.146 2014-08-21 12:50:30 -07:00
fjy d64879ccca more cleanup 2014-08-20 13:22:42 -07:00
fjy 1614f40f1a fix index task 2014-08-20 13:11:00 -07:00
fjy 92f26d9a1f cleanup rowflushboundary 2014-08-20 13:09:37 -07:00
nishantmonu51 fe105d52ee use bufferSize for IndexTask 2014-08-20 22:41:34 +05:30
nishantmonu51 33354cf7fe replace maxRowsInMemory with BufferSize 2014-08-20 20:59:44 +05:30
nishantmonu51 e525562767 review comments - cleanup ColumnSelectorFactory 2014-08-20 15:04:43 +05:30
nishantmonu51 60906c3244 Revert "make valueType configurable" 
This reverts commit 6f60a3f604.
 2014-08-20 11:55:26 +05:30
fjy 4fd5479559 fix typo 2014-08-19 12:34:10 -07:00
fjy 77e514688a Merge branch 'druid-0.7.x' into offheap-incremental-index 2014-08-18 11:14:19 -07:00
Xavier Léauté 1fd30ab588 default service/host/port for all nodes 2014-08-15 17:14:05 -07:00
fjy 51988be776 [maven-release-plugin] prepare for next development iteration 2014-08-14 14:59:00 -07:00
fjy 2830028092 [maven-release-plugin] prepare release druid-0.6.145 2014-08-14 14:58:57 -07:00
nishantmonu51 6f60a3f604 make valueType configurable 2014-08-13 14:37:57 +05:30
nishantmonu51 1b0a72751b Add support for LongColumn 2014-08-13 08:52:36 +05:30
fjy 4e3f4fbc22 [maven-release-plugin] prepare for next development iteration 2014-08-12 13:24:55 -07:00
fjy f3aafc246c [maven-release-plugin] prepare release druid-0.6.144 2014-08-12 13:24:50 -07:00
nishantmonu51 c6712739dc merge changes from druid-0.7.x 2014-08-12 15:47:42 +05:30
fjy 0cff63c7c6 [maven-release-plugin] prepare for next development iteration 2014-08-09 19:36:11 -07:00
fjy 724562b181 [maven-release-plugin] prepare release druid-0.6.143 2014-08-09 19:36:05 -07:00
fjy d58ae945a4 [maven-release-plugin] prepare for next development iteration 2014-08-08 16:22:38 -07:00
fjy d240479133 [maven-release-plugin] prepare release druid-0.6.142 2014-08-08 16:22:34 -07:00
fjy 199f2b1683 [maven-release-plugin] prepare for next development iteration 2014-08-08 15:53:53 -07:00
fjy 29a077d893 [maven-release-plugin] prepare release druid-0.6.141 2014-08-08 15:53:49 -07:00
fjy 958792db5c [maven-release-plugin] prepare for next development iteration 2014-08-08 15:16:38 -07:00
fjy 4482e1b7d8 [maven-release-plugin] prepare release druid-0.6.140 2014-08-08 15:16:34 -07:00
fjy ddcea18631 fix dependencies 2014-08-08 14:02:14 -07:00
fjy 74646f18f4 [maven-release-plugin] prepare for next development iteration 2014-08-08 11:40:52 -07:00
fjy 78c7140b52 [maven-release-plugin] prepare release druid-0.6.139 2014-08-08 11:40:46 -07:00
fjy 91ebe45b4e support both rejectionPolicy and rejectionPolicyFactory in serde 2014-08-07 10:06:27 -07:00
fjy 9f4dd7b33e [maven-release-plugin] prepare for next development iteration 2014-08-06 13:34:04 -07:00
fjy aa5cb91618 [maven-release-plugin] prepare release druid-0.6.138 2014-08-06 13:33:59 -07:00
fjy 65b8278add [maven-release-plugin] prepare for next development iteration 2014-08-01 10:14:34 -07:00
fjy 0ecbef6c5d [maven-release-plugin] prepare release druid-0.6.137 2014-08-01 10:14:30 -07:00
fjy bf8c24f88d [maven-release-plugin] prepare for next development iteration 2014-07-31 14:08:38 -07:00
fjy 66020fc2b6 [maven-release-plugin] prepare release druid-0.6.136 2014-07-31 14:08:34 -07:00
nishantmonu51 637bd35785 merge changes from druid-0.7.x 2014-07-31 16:07:22 +05:30
fjy a8c8e4cb51 [maven-release-plugin] prepare for next development iteration 2014-07-29 22:03:14 -07:00
fjy 25eda62a0c [maven-release-plugin] prepare release druid-0.6.135 2014-07-29 22:03:09 -07:00
fjy 0c898c58d1 [maven-release-plugin] prepare for next development iteration 2014-07-29 18:01:06 -07:00
fjy c8244d05d1 [maven-release-plugin] prepare release druid-0.6.134 2014-07-29 18:01:02 -07:00
fjy f7319620c4 [maven-release-plugin] prepare for next development iteration 2014-07-29 17:40:56 -07:00
fjy 540a146eaa [maven-release-plugin] prepare release druid-0.6.133 2014-07-29 17:40:52 -07:00
fjy b5ecf000ad [maven-release-plugin] prepare for next development iteration 2014-07-29 16:20:14 -07:00
fjy 877bd98667 [maven-release-plugin] prepare release druid-0.6.132 2014-07-29 16:20:10 -07:00
fjy e1d87ed5ea [maven-release-plugin] prepare for next development iteration 2014-07-29 14:13:41 -07:00
fjy c46965c52c [maven-release-plugin] prepare release druid-0.6.131 2014-07-29 14:13:37 -07:00
fjy 1de0f89718 [maven-release-plugin] prepare for next development iteration 2014-07-18 11:55:50 -07:00
fjy 39bff5c1c2 [maven-release-plugin] prepare release druid-0.6.130 2014-07-18 11:55:46 -07:00
Gian Merlino 09fcfc3b6d Fix race in RemoteTaskRunner that could lead to zombie tasks. 2014-07-18 11:41:50 -07:00
fjy 9e303527f2 [maven-release-plugin] prepare for next development iteration 2014-07-18 11:17:56 -07:00
fjy ea04022772 [maven-release-plugin] prepare release druid-0.6.129 2014-07-18 11:17:52 -07:00
nishantmonu51 4ce12470a1 Add way to skip determine partitions for index task 
Add a way to skip determinePartitions for IndexTask by manually
specifying numShards.
 2014-07-18 18:52:15 +05:30
fjy d8b8826c2e Merge branch 'cleanup-ingest' of github.com:metamx/druid into cleanup-ingest 
Conflicts:
	server/src/test/java/io/druid/realtime/firehose/CombiningFirehoseFactoryTest.java
 2014-07-17 20:26:11 -07:00
fjy 291f4c00ae Merge branch 'master' of github.com:metamx/druid into cleanup-ingest 2014-07-17 20:24:59 -07:00
nishantmonu51 0e0454a34c switch reingest task to noop & fix compilation 
switch back to noop task, its confusing to have a reinvest task that
does nothing.
fix compilation
 2014-07-18 06:50:58 +05:30
fjy ded83557dd Merge pull request #640 from metamx/disable-worker 
An alternative way to disable middlemanagers based on worker version
 2014-07-17 19:07:27 -06:00
fjy beac0be45b fix enabled endpoint 2014-07-17 18:04:36 -07:00
fjy c6078ca841 address code review 2014-07-17 13:34:05 -07:00
fjy ba978d8b79 some minor cleanups to ingest firehose 2014-07-17 13:05:59 -07:00
fjy bc650a1c80 Merge pull request #627 from metamx/druid-firehose 
Functionality to ingest a Druid segment and change the schema
 2014-07-17 13:41:16 -06:00
fjy 5197ea527a disable middlemanagers based on worker version 2014-07-17 12:35:45 -07:00
nishantmonu51 e59c9ebdbc minor fixes 
fix IndexOutOfBoundsException
fix ingestFirehose
 2014-07-17 17:24:57 +05:30
nishantmonu51 972c5dac31 improve memory usage and rename firehose 2014-07-14 21:17:53 +05:30
nishantmonu51 f5f05e3a9b Sync changes from branch new-ingestion PR #599 
Sync and Resolve Conflicts
 2014-07-11 16:15:10 +05:30
nishantmonu51 7168adcca7 Use Ingest task instead of Noop Task 2014-07-08 21:20:17 +05:30
fjy ce478be899 Remove redundant forking task runner config 2014-07-07 18:08:37 -06:00
nishantmonu51 97b58eb193 review comments 
1) Rename firehose to IngestSegment
2) fix segment overlapping, intervals
3) fix overshadow
 2014-07-04 12:31:33 +05:30
nishantmonu51 518ab473f3 improve port finding strategy for task runner 
1) Recycle free ports
2) Choose only ports that are free & not used by any other application
 2014-07-03 09:58:12 +05:30
nishantmonu51 1de390801f Druid Fireshose 
Add druidFirehose, can be used to reIndex filtered data from a
datasource
 2014-07-02 20:34:18 +05:30
fjy 4c40e71e54 address cr 2014-06-19 14:48:46 -07:00
fjy a870fe5cbe inject column config 2014-06-19 14:47:57 -07:00
Xavier Léauté 09346b0a3c make column cache configurable 2014-06-19 14:43:03 -07:00
fjy 0bc1915067 Merge pull request #578 from metamx/new-guava 
Update guava, java-util, and druid-api
 2014-06-18 14:23:32 -06:00
Gian Merlino 7c7d170e78 [maven-release-plugin] prepare for next development iteration 2014-06-18 10:53:40 -07:00
Gian Merlino a52a1d3929 [maven-release-plugin] prepare release druid-0.6.128 2014-06-18 10:53:36 -07:00
fjy b0517dc55d Merge pull request #610 from metamx/ec2-user-data 
More flexible EC2 user data classes.
 2014-06-18 11:36:13 -06:00
Gian Merlino 5c4b9a7d5b Indexing: Allow null userData. 2014-06-18 10:08:50 -07:00
Gian Merlino 9532beb05c [maven-release-plugin] prepare for next development iteration 2014-06-17 17:29:31 -07:00
Gian Merlino e3dcbde07d [maven-release-plugin] prepare release druid-0.6.127 2014-06-17 17:29:27 -07:00
Gian Merlino 65c1553a76 Fix busted TaskConfig. 2014-06-17 17:27:41 -07:00
Gian Merlino 9f16f0a955 More flexible EC2 user data classes. 2014-06-17 17:10:17 -07:00
Gian Merlino a658afe902 [maven-release-plugin] prepare for next development iteration 2014-06-17 13:01:01 -07:00
Gian Merlino 4e875b8084 [maven-release-plugin] prepare release druid-0.6.126 2014-06-17 13:00:54 -07:00
fjy 5227b43050 fix more test breakage 2014-06-17 10:35:01 -07:00