Himanshu
34c8f4c3da
Merge pull request #1437 from metamx/limitUsedSegmentInterval
...
Add start and end bounds to getUsedSegmentsForInterval
2015-06-22 14:11:38 -05:00
Charles Allen
8d91c8d399
Add start and end bounds to getUsedSegmentsForInterval
...
* Add unit tests
* Add javadoc for methods in IndexerMetadataStorageCoordinator
2015-06-22 11:46:53 -07:00
Charles Allen
f0f453940f
Merge pull request #1456 from metamx/test-1120
...
test for #1120
2015-06-22 10:54:24 -07:00
ravishrathod
9213fd3801
updating doc for rabbitmq firehose
2015-06-22 02:40:11 -04:00
nishant
f9cdb0ad61
test for #1120
...
Make the changes described in #1120 to add test for the issue described
there.
2015-06-21 23:34:21 +05:30
fjy
9c74993559
fix protobuf impl and docs
2015-06-20 21:59:38 -07:00
Hao Xia
1931491c9f
A couple of hdfs related fixes
...
* Class loading issue with hdfs-storage extension
* Exception when using hdfs with non-fully qualified segment path
2015-06-19 17:22:20 -07:00
Charles Allen
b72a70522b
Add basic ConciseComplementBenchmark for testing concise complements
2015-06-19 16:47:41 -07:00
Xavier Léauté
f1951b253c
Fix bad distribution of cache keys across nodes
...
With the existing hash function some nodes could end up with 3 times the
number of keys as others. The following changes improve that to roughly
less than 5% differences across nodes.
- switch from fnv-1a to murmur3_128 hash
- increase repetitions for ketama algorithm
- test to analyze distribution
Also updates spymemcached for recent bugfixes
2015-06-19 15:30:35 -07:00
sahner
4ba34fe43d
minor documentation fixes in Tasks.md, index.md, indexing-service.md
2015-06-19 17:09:53 -05:00
nishant
fb4052d577
JavaScript Worker Select Strategy
...
this PR adds a JavaScriptWorkerSelectStrategy which allows defining
arbitrary logic for selecting workers to run task using a JavaScript
function.
This gives users full control to implement complex worker selection
strategies based on task attributes.
more tests and a complex javascript config
fix for java8 modify for nashorn compatibility
2015-06-20 02:01:34 +05:30
Xavier Léauté
0a5bb909a2
[maven-release-plugin] prepare for next development iteration
2015-06-18 17:35:19 -07:00
Xavier Léauté
59c6b2b279
[maven-release-plugin] prepare release druid-0.8.0-rc1
2015-06-18 17:35:14 -07:00
Xavier Léauté
d276d2c90b
Merge pull request #1444 from druid-io/logging-improvement
...
Separate bootstrap threads from loading threads on historical startup
2015-06-17 15:24:07 -07:00
Eric Tschetter
06c97b6d7d
Separate out the "bootstrapping" threads from the "loading" threads.
...
It is highly recommended to keep loading threads single threaded. There can be benefits to having multiple bootstrapping threads, though. Those bootstrapping threads shouldn't be kept alive for the whole life of the process, however.
Also, adjust some logging and do some code cleanup.
2015-06-17 14:58:28 -07:00
Xavier Léauté
f7a7daeff9
Merge pull request #1434 from metamx/fix1433
...
Wipe FileContext off the face of the earth
2015-06-16 21:42:40 -07:00
Fangjin Yang
0539e0073d
Merge pull request #1442 from guobingkun/broker_server_view_test
...
Unit test for BrokerServerView
2015-06-16 19:45:20 -07:00
Eric Tschetter
1aad7ce521
Add a bit more information to the mapping logging logic.
2015-06-16 19:33:38 -07:00
Bingkun Guo
446d01cc53
Unit test for BrokerServerView
2015-06-16 15:20:32 -05:00
Charles Allen
69bbe21512
Add flush prior to close on output stream in QueryResource
2015-06-16 11:40:56 -07:00
Charles Allen
94a567732a
Wipe FileContext off the face of the earth
...
* Fixes https://github.com/druid-io/druid/issues/1433
* Works arround https://issues.apache.org/jira/browse/HADOOP-10643
* Reverts to the prior method of renaming
2015-06-16 09:48:09 -07:00
Xavier Léauté
0de914823d
Merge pull request #1435 from metamx/jitterWaitTimes
...
Add jitter to the retries for RemoteTaskActionClient
2015-06-15 22:24:19 -07:00
Charles Allen
acc0a3fbf7
Add jitter to the retries for RemoteTaskActionClient
2015-06-12 17:43:25 -07:00
Xavier Léauté
6763e3780a
Merge pull request #1417 from metamx/rtr-fixes
...
fix behaviour of middlemanagers around ZK disconnects , fixes #709
2015-06-10 14:42:49 -07:00
Xavier Léauté
9cf8662aeb
Merge pull request #1427 from guobingkun/fix_issue_1016
...
Fix duplicate extension loading issue described in #1016
2015-06-10 14:42:24 -07:00
Bingkun Guo
bdf4d541e3
Fix issue #1016 that if user specifies extension coordinates, the local extensions will be loaded twice,
...
which could cause Guice duplicate binding errors.
Add unit test to replicate duplicate extension issue. Update documents. Add an package accessible getter for loadersMap for testing only.
Notice that extensions explicitly specified in druid.extensions.coordinates have a higher priority than ones included in the classpath.
Extension modules that don't have a canonical class name will be ignored.
2015-06-10 15:48:03 -05:00
nishant
e9afec4a2b
fix task status issues on zk outages
...
docs
review comments
fix test
review comments
Review comments
fix compilation
fix typo
2015-06-11 00:49:52 +05:30
Xavier Léauté
78d468700b
Merge pull request #1388 from metamx/fix-1360
...
fix race described in 1360
2015-06-10 11:59:36 -07:00
Xavier Léauté
7fd9dcff48
Merge pull request #1409 from metamx/convertTaskMerger
...
Converter to use TaskMerger not TaskMaker
2015-06-10 11:58:52 -07:00
Xavier Léauté
f6b336ac3e
Merge pull request #1432 from metamx/config-fix
...
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
2015-06-10 11:42:58 -07:00
Charles Allen
6230ac90ae
Use IndexMerger for conversion
2015-06-10 11:34:58 -07:00
cheddar
0004ba249f
Merge pull request #1351 from metamx/hadoopConvert
...
Add Hadoop Converter Job and task
2015-06-10 11:07:06 -07:00
nishant
963682d696
Add check for valid rowFlushBoundary configuration and fix tests
2015-06-10 21:38:34 +05:30
nishant
191b302f6a
fix passing of config from IndexTuningConfig to RealtimeTuningConfig
...
- pass rowFlushboundary correctly instead of using default.
- fixes indexTask failing with
io.druid.segment.incremental.IndexSizeExceededException when
rowFlushboundary is set higher than
RealtimeTuningConfig.defaultMaxRowsInMemory
rename test method
2015-06-10 21:07:25 +05:30
Xavier Léauté
e5937482bc
Merge pull request #1431 from pjain1/coordinator_ut
...
Druid Coordinator test
2015-06-10 07:24:39 -07:00
nishant
af9ea08041
fix race described in 1360
...
review comments
review comments
review comments
no need to remove
fix test
review comments
2015-06-10 12:19:12 +05:30
Charles Allen
056cab93ed
Add Hadoop Converter Job and task
...
* Fixes https://github.com/druid-io/druid/issues/1363
* Add extra utils in JobHelper based on PR feedback
2015-06-09 14:47:38 -07:00
Parag Jain
b72e6fd51a
Druid Coordinator Unit test
2015-06-09 13:51:01 -05:00
Fangjin Yang
6ae4ecc7d4
Merge pull request #1428 from metamx/abstractifyHadoopConfigStuff
...
Abstractify hadoopy indexer configuration.
2015-06-08 11:26:34 -07:00
Charles Allen
2a76bdc60a
Abstractify hadoopy indexer configuration.
...
* Moves many items to JobHelper
* Remove dependencies of these functions on HadoopDruidIndexerConfig in favor of more general items
* Changes functionalities of some of the path methods to always return a path with scheme
* Adds retry to uploads
* Change output loadSpec determining from using outputFS.getClass().getName() to using outputFS.getScheme()
2015-06-08 10:53:27 -07:00
cheddar
04e4aac03b
Merge pull request #1430 from guobingkun/lazy_init_abstract_http_client_provider
...
Make AbstractHttpClientProvider contents lazily instantiated
2015-06-08 09:33:35 -07:00
Bingkun Guo
22607d19be
Make AbstractHttpClientProvider contents lazily instantiated
2015-06-08 09:53:26 -05:00
Xavier Léauté
52f261eea7
Merge pull request #1429 from druid-io/fix-doc
...
rename servermonitor to historicalmetricsmonitor in docs
2015-06-05 17:33:06 -07:00
fjy
afdd9304cf
rename servermonitor to historicalmetricsmonitor in docs
2015-06-05 17:32:04 -07:00
Charles Allen
b0085c8a8b
Merge pull request #1426 from druid-io/more-links
...
Add more links for talks
2015-06-05 14:37:19 -07:00
fjy
6744eb2889
Add more links for talks
2015-06-05 14:35:56 -07:00
Fangjin Yang
c6f36dcccb
Merge pull request #1411 from metamx/more-benchmarks
...
More benchmarks for data compression
2015-06-04 22:01:39 -07:00
Xavier Léauté
395ba79f8b
Merge pull request #1403 from metamx/mergerMakerTests
...
Improvements around resource handling in IndexMerger / IndexIO / QueryableIndex
2015-06-04 15:59:10 -07:00
Himanshu
f1721fa912
Merge pull request #1410 from guobingkun/unit_test
...
Unit tests for LoadQueuePeon and SegmentLoaderLocalCacheManager
2015-06-04 17:58:13 -05:00
Bingkun Guo
1646f650b6
Unit tests for LoadQueuePeon and SegmentLoaderLocalCacheManager, JavaDoc for LoadPeonCallback
2015-06-04 17:28:22 -05:00