Commit Graph

5880 Commits

Author SHA1 Message Date
Charles Allen 69bbe21512 Add flush prior to close on output stream in QueryResource 2015-06-16 11:40:56 -07:00
Charles Allen 94a567732a Wipe FileContext off the face of the earth
* Fixes https://github.com/druid-io/druid/issues/1433
* Works arround https://issues.apache.org/jira/browse/HADOOP-10643
* Reverts to the prior method of renaming
2015-06-16 09:48:09 -07:00
Xavier Léauté 0de914823d Merge pull request #1435 from metamx/jitterWaitTimes
Add jitter to the retries for RemoteTaskActionClient
2015-06-15 22:24:19 -07:00
Charles Allen acc0a3fbf7 Add jitter to the retries for RemoteTaskActionClient 2015-06-12 17:43:25 -07:00
Xavier Léauté 6763e3780a Merge pull request #1417 from metamx/rtr-fixes
fix behaviour of middlemanagers around ZK disconnects , fixes #709
2015-06-10 14:42:49 -07:00
Xavier Léauté 9cf8662aeb Merge pull request #1427 from guobingkun/fix_issue_1016
Fix duplicate extension loading issue described in #1016
2015-06-10 14:42:24 -07:00
Bingkun Guo bdf4d541e3 Fix issue #1016 that if user specifies extension coordinates, the local extensions will be loaded twice,
which could cause Guice duplicate binding errors.
Add unit test to replicate duplicate extension issue. Update documents. Add an package accessible getter for loadersMap for testing only.
Notice that extensions explicitly specified in druid.extensions.coordinates have a higher priority than ones included in the classpath.
Extension modules that don't have a canonical class name will be ignored.
2015-06-10 15:48:03 -05:00
nishant e9afec4a2b fix task status issues on zk outages
docs

review comments

fix test

review comments

Review comments

fix compilation

fix typo
2015-06-11 00:49:52 +05:30
Xavier Léauté 78d468700b Merge pull request #1388 from metamx/fix-1360
fix race described in 1360
2015-06-10 11:59:36 -07:00
Xavier Léauté 7fd9dcff48 Merge pull request #1409 from metamx/convertTaskMerger
Converter to use TaskMerger not TaskMaker
2015-06-10 11:58:52 -07:00
Xavier Léauté f6b336ac3e Merge pull request #1432 from metamx/config-fix
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
2015-06-10 11:42:58 -07:00
Charles Allen 6230ac90ae Use IndexMerger for conversion 2015-06-10 11:34:58 -07:00
cheddar 0004ba249f Merge pull request #1351 from metamx/hadoopConvert
Add Hadoop Converter Job and task
2015-06-10 11:07:06 -07:00
nishant 963682d696 Add check for valid rowFlushBoundary configuration and fix tests 2015-06-10 21:38:34 +05:30
nishant 191b302f6a fix passing of config from IndexTuningConfig to RealtimeTuningConfig
- pass rowFlushboundary correctly instead of using default.
- fixes indexTask failing with
io.druid.segment.incremental.IndexSizeExceededException when
rowFlushboundary is set higher than
RealtimeTuningConfig.defaultMaxRowsInMemory

rename test method
2015-06-10 21:07:25 +05:30
Xavier Léauté e5937482bc Merge pull request #1431 from pjain1/coordinator_ut
Druid Coordinator test
2015-06-10 07:24:39 -07:00
nishant af9ea08041 fix race described in 1360
review comments

review comments

review comments

no need to remove

fix test

review comments
2015-06-10 12:19:12 +05:30
Charles Allen 056cab93ed Add Hadoop Converter Job and task
* Fixes https://github.com/druid-io/druid/issues/1363
* Add extra utils in JobHelper based on PR feedback
2015-06-09 14:47:38 -07:00
Parag Jain b72e6fd51a Druid Coordinator Unit test 2015-06-09 13:51:01 -05:00
Fangjin Yang 6ae4ecc7d4 Merge pull request #1428 from metamx/abstractifyHadoopConfigStuff
Abstractify hadoopy indexer configuration.
2015-06-08 11:26:34 -07:00
Charles Allen 2a76bdc60a Abstractify hadoopy indexer configuration.
* Moves many items to JobHelper
* Remove dependencies of these functions on HadoopDruidIndexerConfig in favor of more general items
* Changes functionalities of some of the path methods to always return a path with scheme
* Adds retry to uploads
* Change output loadSpec determining from using outputFS.getClass().getName() to using outputFS.getScheme()
2015-06-08 10:53:27 -07:00
cheddar 04e4aac03b Merge pull request #1430 from guobingkun/lazy_init_abstract_http_client_provider
Make AbstractHttpClientProvider contents lazily instantiated
2015-06-08 09:33:35 -07:00
Bingkun Guo 22607d19be Make AbstractHttpClientProvider contents lazily instantiated 2015-06-08 09:53:26 -05:00
Xavier Léauté 52f261eea7 Merge pull request #1429 from druid-io/fix-doc
rename servermonitor to historicalmetricsmonitor in docs
2015-06-05 17:33:06 -07:00
fjy afdd9304cf rename servermonitor to historicalmetricsmonitor in docs 2015-06-05 17:32:04 -07:00
Charles Allen b0085c8a8b Merge pull request #1426 from druid-io/more-links
Add more links for talks
2015-06-05 14:37:19 -07:00
fjy 6744eb2889 Add more links for talks 2015-06-05 14:35:56 -07:00
Fangjin Yang c6f36dcccb Merge pull request #1411 from metamx/more-benchmarks
More benchmarks for data compression
2015-06-04 22:01:39 -07:00
Xavier Léauté 395ba79f8b Merge pull request #1403 from metamx/mergerMakerTests
Improvements around resource handling in IndexMerger / IndexIO / QueryableIndex
2015-06-04 15:59:10 -07:00
Himanshu f1721fa912 Merge pull request #1410 from guobingkun/unit_test
Unit tests for LoadQueuePeon and SegmentLoaderLocalCacheManager
2015-06-04 17:58:13 -05:00
Bingkun Guo 1646f650b6 Unit tests for LoadQueuePeon and SegmentLoaderLocalCacheManager, JavaDoc for LoadPeonCallback 2015-06-04 17:28:22 -05:00
Charles Allen ed8eb5c991 Improvements around resource handling in IndexMerger / IndexIO / QueryableIndex
* Fix resource leak in `io.druid.segment.IndexIO.DefaultIndexIOHandler#validateTwoSegments(java.io.File, java.io.File)`
* Un-deprecate `close()` in `QueryableIndex` and make it inherit `Closeable`
* Fix resource leaks in various unit tests
* Add `CloserRule` for closing out resources
2015-06-04 14:18:27 -07:00
Himanshu f81a0d9210 Merge pull request #1425 from guobingkun/guice_fix
Anyways, @guobingkun , if possible,  can you please test proposed change and send another PR to fix #1016 .  but that doesn't necessarily hold this PR, so I'm merging this.
2015-06-04 14:43:26 -05:00
Himanshu f44f7f07de Merge pull request #1424 from pjain1/validate_rules
Check lower bound of replicant values for rules
2015-06-04 14:16:50 -05:00
Parag Jain 1030899cdf Check lower bound of replicant values for rules 2015-06-04 13:47:43 -05:00
nishant 37873eb4f0 more benchmarks + filter
fix test

Modify test to access random filtered rows

formatting

fix test

remove main method

Add newline

Add newline
2015-06-04 22:12:29 +05:30
Bingkun Guo dc26350480 Separate overwrite of Guice modules into 2 steps so that extension modules can overwrite cli modules 2015-06-04 09:57:56 -05:00
Himanshu 50ad0e6474 Merge pull request #1412 from pjain1/alphaNumericTopN_NPE_fix
NPE fix for TopN query with alphaNumericTopN metric spec
2015-06-04 09:49:31 -05:00
Parag Jain a7b09e857c NPE fix for alphaNumericTopN when pervious stop is not specified 2015-06-04 09:30:31 -05:00
Xavier Léauté 35e2fde18e Merge pull request #1386 from himanshug/aggregation_testing1
General class for testing any Aggregation Implementation
2015-06-03 23:43:36 -07:00
Fangjin Yang 7f5164db14 Merge pull request #1418 from metamx/reduce-test-runtime
reduce test runtime
2015-06-03 21:40:36 -07:00
nishant 21864a9407 reduce test runtime
fix check

fix test
2015-06-04 08:35:12 +05:30
Charles Allen ef9b67cce3 Merge pull request #1422 from metamx/fix-ec2-public-ip
fix public IP not working in EC2 autoscaling
2015-06-03 16:30:51 -07:00
Xavier Léauté 4ebdfea76f fix public IP not working in EC2 autoscaling 2015-06-03 16:05:59 -07:00
Himanshu c096c21b6a Merge pull request #1413 from metamx/TaskUtilIdCommonizer
Move TaskUtils.makeId to AbstractTask
2015-06-03 15:54:22 -05:00
Charles Allen 8289914f76 Make AbstractTask.makeId use AbstractTask.joinId
* Also remove TaskUtil
2015-06-03 13:24:20 -07:00
Xavier Léauté 92d7316ed8 Merge pull request #1414 from metamx/timeout2TIMEOUT
Replace "timeout" with QueryContextKeys.TIMEOUT
2015-06-02 17:11:09 -07:00
Charles Allen 1c4d42bc15 Replace "timeout" with QueryContextKeys.TIMEOUT 2015-06-02 14:49:21 -07:00
Xavier Léauté 224886ab52 Merge pull request #1370 from metamx/dimExtractionOptimizations
Add optimizations for ExtractionFn by enabling MANY_TO_ONE vs ONE_TO_ONE codepaths
2015-06-02 13:25:19 -07:00
Charles Allen f48db09e35 Add optimizations for ExtractionFn by enabling MANY_TO_ONE vs ONE_TO_ONE codepaths
* Also adds LookupExtractionFn and MapLookupExtractor which takes in an explicit mapping of renames
* Add injective to javascript extraction fn
2015-06-02 12:22:56 -07:00