28fa1642b9
add node time metrics to DirectDruidClient
2015-06-26 17:57:44 -07:00
fbcac10e00
Remove metrics emitting from caching clustered client
2015-06-26 10:49:13 -07:00
e4e18b17b5
fix npe thrown from getEntry() and getStatus() SQLMetadataStorageActionHandler due to a non-exist entryId
2015-06-26 10:40:56 -05:00
0c85c8c60a
Merge pull request #1462 from pjain1/validate_rules
...
tieredReplicants can be null
2015-06-24 13:29:27 -07:00
ff738ace05
converting null tieredeplicants to default and not allowing empty ones
2015-06-24 14:55:12 -05:00
0c13fd8018
Merge pull request #1450 from metamx/cache-key-distribution
...
Fix bad distribution of cache keys across cache nodes
2015-06-22 17:58:15 -07:00
34c8f4c3da
Merge pull request #1437 from metamx/limitUsedSegmentInterval
...
Add start and end bounds to getUsedSegmentsForInterval
2015-06-22 14:11:38 -05:00
8d91c8d399
Add start and end bounds to getUsedSegmentsForInterval
...
* Add unit tests
* Add javadoc for methods in IndexerMetadataStorageCoordinator
2015-06-22 11:46:53 -07:00
f1951b253c
Fix bad distribution of cache keys across nodes
...
With the existing hash function some nodes could end up with 3 times the
number of keys as others. The following changes improve that to roughly
less than 5% differences across nodes.
- switch from fnv-1a to murmur3_128 hash
- increase repetitions for ketama algorithm
- test to analyze distribution
Also updates spymemcached for recent bugfixes
2015-06-19 15:30:35 -07:00
0a5bb909a2
[maven-release-plugin] prepare for next development iteration
2015-06-18 17:35:19 -07:00
59c6b2b279
[maven-release-plugin] prepare release druid-0.8.0-rc1
2015-06-18 17:35:14 -07:00
d276d2c90b
Merge pull request #1444 from druid-io/logging-improvement
...
Separate bootstrap threads from loading threads on historical startup
2015-06-17 15:24:07 -07:00
06c97b6d7d
Separate out the "bootstrapping" threads from the "loading" threads.
...
It is highly recommended to keep loading threads single threaded. There can be benefits to having multiple bootstrapping threads, though. Those bootstrapping threads shouldn't be kept alive for the whole life of the process, however.
Also, adjust some logging and do some code cleanup.
2015-06-17 14:58:28 -07:00
0539e0073d
Merge pull request #1442 from guobingkun/broker_server_view_test
...
Unit test for BrokerServerView
2015-06-16 19:45:20 -07:00
1aad7ce521
Add a bit more information to the mapping logging logic.
2015-06-16 19:33:38 -07:00
446d01cc53
Unit test for BrokerServerView
2015-06-16 15:20:32 -05:00
69bbe21512
Add flush prior to close on output stream in QueryResource
2015-06-16 11:40:56 -07:00
9cf8662aeb
Merge pull request #1427 from guobingkun/fix_issue_1016
...
Fix duplicate extension loading issue described in #1016
2015-06-10 14:42:24 -07:00
bdf4d541e3
Fix issue #1016 that if user specifies extension coordinates, the local extensions will be loaded twice,
...
which could cause Guice duplicate binding errors.
Add unit test to replicate duplicate extension issue. Update documents. Add an package accessible getter for loadersMap for testing only.
Notice that extensions explicitly specified in druid.extensions.coordinates have a higher priority than ones included in the classpath.
Extension modules that don't have a canonical class name will be ignored.
2015-06-10 15:48:03 -05:00
e5937482bc
Merge pull request #1431 from pjain1/coordinator_ut
...
Druid Coordinator test
2015-06-10 07:24:39 -07:00
b72e6fd51a
Druid Coordinator Unit test
2015-06-09 13:51:01 -05:00
22607d19be
Make AbstractHttpClientProvider contents lazily instantiated
2015-06-08 09:53:26 -05:00
1646f650b6
Unit tests for LoadQueuePeon and SegmentLoaderLocalCacheManager, JavaDoc for LoadPeonCallback
2015-06-04 17:28:22 -05:00
f81a0d9210
Merge pull request #1425 from guobingkun/guice_fix
...
Anyways, @guobingkun , if possible, can you please test proposed change and send another PR to fix #1016 . but that doesn't necessarily hold this PR, so I'm merging this.
2015-06-04 14:43:26 -05:00
f44f7f07de
Merge pull request #1424 from pjain1/validate_rules
...
Check lower bound of replicant values for rules
2015-06-04 14:16:50 -05:00
1030899cdf
Check lower bound of replicant values for rules
2015-06-04 13:47:43 -05:00
dc26350480
Separate overwrite of Guice modules into 2 steps so that extension modules can overwrite cli modules
2015-06-04 09:57:56 -05:00
21864a9407
reduce test runtime
...
fix check
fix test
2015-06-04 08:35:12 +05:30
1c4d42bc15
Replace "timeout" with QueryContextKeys.TIMEOUT
2015-06-02 14:49:21 -07:00
185c99deec
Add schema check for overlap between dimension and metric names
2015-05-28 22:18:08 -07:00
be2a35188e
Additional schema validations and better logs for common extensions
2015-05-27 16:25:02 -07:00
051c3ccede
Add futures to SQLMetadataSegmentManager and SQLMetadataRuleManager
2015-05-15 16:08:22 -07:00
3c3db7229c
Merge pull request #1355 from himanshug/long_max_min_aggregators
...
Long max/min aggregators
2015-05-13 12:08:11 -07:00
d0ec945129
adding aliases doubleMax and doubleMin for max and min respectively
...
renamed all [Max/Min]*.java to [DoubleMax/DoubleMin]*.java and created [Max/Min]AggregatorFactory.java which can be removed when we dont need the min/max aggregator type backward compatibility
2015-05-13 09:25:41 -05:00
7a6acf5c1b
update pom to 0.8
2015-05-11 19:41:58 -06:00
a2dc58cd2d
Merge pull request #1345 from pjain1/unit_test_warn_fix
...
fix warn msg and some unit tests
2015-05-08 08:06:20 -07:00
01448d264c
Fix warn msg and added some unit tests
2015-05-07 17:10:05 -05:00
7608bf5799
Merge pull request #1342 from guobingkun/unit_test
...
Fix a potential issue in DeterminePartitionsJob, and two unit tests, reformat code
2015-05-05 09:24:23 -07:00
1ee550dd91
Fix a potential issue in DeterminePartitionsJob by making HadoopDruidIndexerConfig non-static, and two unit tests for DeterminPartitionsJob and LocalDataSegmentKiller
2015-05-04 20:00:29 -07:00
e69d82a2b4
Realtime: Delay firehose connection until job is started.
...
Some firehoses (like the Kafka firehose) acquire input resources when they
connect, so it helps to delay this until after plumber.startJob() runs.
2015-05-04 10:54:07 -07:00
8eb441ec08
RealtimeManager: Include partitionNum in chief thread name.
2015-05-04 10:53:48 -07:00
27e25fe513
fix IRCFirehoseFactory json name/nick mismatch
...
* Now it is nick as per wikipedia example
2015-05-01 12:35:44 -07:00
15ed308a4b
Merge pull request #1322 from himanshug/fix_audit_tbl_idx
...
fixing the typo for audit table (type, created_date) index creation
2015-04-30 13:22:04 -07:00
a7f9bb1a1e
Additional coordinator logs to figure out when racey things might be happening
2015-04-30 08:26:52 -07:00
4aa9381f6a
fixing the typo for audit table (type, created_date) index creation
2015-04-29 17:09:35 -05:00
6e84caf3d5
Proxy Requests from coordinator to overlord
...
Add unit test and support https scheme
review comments
2015-04-29 20:58:26 +05:30
062a413ccd
Fix DruidNodeTest for ipv6 only hosts
2015-04-28 13:11:47 -07:00
721505c017
Merge pull request #1208 from druid-io/rework-metrics
...
Schemaless metrics + additional metrics for things we care about
2015-04-27 15:04:54 -07:00
963e5765bf
Schemaless metrics + additional metrics for things we care about
2015-04-27 13:39:40 -07:00
958b34e0c2
fix exception during timeboundary
...
Fix Exception during time boundary query where results can be
represented as integers
2015-04-23 20:32:35 +05:30
1addbe163a
Merge pull request #1295 from anubhgup/master
...
Fix type of values in sql update.
2015-04-17 20:26:17 -07:00
4c240da4e9
Fix type of values in sql update.
2015-04-17 18:44:49 -07:00
d260515a43
update druid-api version
2015-04-17 14:58:35 -07:00
ea5572d001
Merge pull request #1271 from metamx/strictErrorChecking
...
Add stricter checking for potential coding errors
2015-04-15 15:21:41 -07:00
abdeaa0746
Add stricter checking for potential coding errors
...
Can use via `mvn clean compile test-compile -P strict'
2015-04-15 14:52:25 -07:00
3a3046ccf3
add support for dimension compression
...
- compression for single-value dimensions using CompressedVSizeIntsIndexedSupplier
- makes dimension compression configurable via IndexSpec
- IndexSpec also enables configuring bitmap and metric compression
2015-04-14 10:44:18 -07:00
44e23a8267
Merge pull request #1224 from metamx/alert-improve
...
skip raising false alert when the coordinator looses leadership
2015-04-07 10:11:34 -07:00
208e307915
Merge pull request #1251 from metamx/uriSegmentLoaders
...
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
2015-03-30 17:43:51 -07:00
aea7f9d192
[maven-release-plugin] prepare for next development iteration
2015-03-30 16:35:24 -07:00
060d7aef03
[maven-release-plugin] prepare release druid-0.7.1
2015-03-30 16:35:20 -07:00
1c6cbea89c
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
...
This reverts commit f904bc7858
2015-03-30 13:40:04 -07:00
f904bc7858
Revert "Overhaul of SegmentPullers to add consistency and retries"
2015-03-30 13:15:50 -07:00
6d407e8677
Add URI handling to SegmentPullers
...
* Requires https://github.com/druid-io/druid-api/pull/37
* Requires https://github.com/metamx/java-util/pull/22
* Moves the puller logic to use a more standard workflow going through java-util helpers instead of re-writing the handlers for each impl
* General workflow goes like this: 1) LoadSpec makes sure the correct Puller is called with the correct parameters. 2) The Puller sets up general information like how to make an InputStream, how to find a file name (for .gz files for example), and when to retry. 3) CompressionUtils does most of the heavy lifting when it can
2015-03-30 12:33:23 -07:00
e5653f0752
Merge pull request #1190 from vigiglobe/master
...
Fix NPE when partionNumber 0 does not exist.
2015-03-26 13:25:39 -07:00
2c58e355fd
Merge pull request #1239 from metamx/log4j2forRealz
...
Exclude log4j from curator dependencies in favor of log4j-1.2-api
2015-03-26 13:21:13 -07:00
9cd6c08e96
Exclude log4j from curator dependencies in favor of log4j-1.2-api
2015-03-26 13:05:12 -07:00
47cc11829d
Merge pull request #1240 from himanshug/async_query_forwarding_servlet_router_fix
...
adding router numMaxThread configuration for ProxyServlet HttpClient executor
2015-03-26 10:15:44 -07:00
942c17a2aa
Remove timeline chunk count assumptions.
...
* Replace with generic iterables
2015-03-24 22:40:49 +01:00
b131a4fd3b
adding router numMaxThread configuration for ProxyServlet HttpClient executor
2015-03-23 21:58:01 -05:00
3b894f9fb9
add ut for TierSelectorStrategies
...
fix test
2015-03-24 08:18:28 +05:30
23528299c9
Merge pull request #1235 from druid-io/comments-update
...
Minor update to zk comments to make it more clear what is happening
2015-03-23 14:02:08 -07:00
a0ce1caaf8
minor update to zk comments to make it more clear what is happening
2015-03-23 11:47:43 -07:00
b5b32dbcb4
in test explicitly setting maxThreads for ProxyServlet to work around https://tickets.puppetlabs.com/browse/TK-152
2015-03-23 12:27:19 -05:00
b389cfe404
[maven-release-plugin] prepare for next development iteration
2015-03-19 12:38:17 -07:00
60e7d543cc
[maven-release-plugin] prepare release druid-0.7.1-rc1
2015-03-19 12:38:13 -07:00
21cd75bcf6
improve alert
...
skip doing false alert when the coordinator looses leadership
2015-03-18 09:42:22 +05:30
8d0dfd9af1
Merge pull request #1221 from metamx/AnnounceIdUUID
...
Change announcement ID to a UUID instead of Timestamp
2015-03-17 15:38:53 -07:00
79b1443cc3
Change announcement ID to a UUID instead of Timestamp
...
* Also add a UUIDUtils to the common package
2015-03-17 15:34:04 -07:00
f9821d242f
also log author ip address in audit log
2015-03-17 23:15:15 +05:30
526a386f50
Merge pull request #1193 from metamx/reduce-test-verbosity
...
move test output to file for cleaner build logs
2015-03-12 15:42:34 -07:00
3ec4a30ab5
initial commit
...
review comments
more refactoring and cleaning of redundant code
add UT + docs + more refactoring
fixes + review comments
more cleanup
end points to fetch history
review comments
remove unnecessary changes
review comments rename header name
review comments + add test for MetadataRulesManager
review comments docs
2015-03-12 22:50:29 +05:30
fc613771d2
move test output to file for cleaner build logs
...
- removes the need for special test log4j2.xml
2015-03-11 17:56:19 -07:00
9f242ed1ba
Merge pull request #1182 from druid-io/better-docs
...
Rework the druid docs and fix many mistakes
2015-03-10 14:23:38 -07:00
064d65a009
rework the druid docs and fix many mistakes
2015-03-10 14:19:19 -07:00
ef842b2eae
add test coverage
2015-03-09 14:32:13 -07:00
e161021902
Merge pull request #1171 from druid-io/fix-rt-task
...
Add more metrics and fail when realtime tasks fail to persist and merge
2015-03-06 16:42:23 -08:00
86a0a3cab8
add more metrics and fail when realtime tasks fail to persist and merge
2015-03-06 11:00:34 -08:00
956361dbbd
Refactoring: Moved BaseJettyServerInitializer.java to JettyServerInitUtils.java and changed it from abstract class to a utils class containing static methods
...
And, moved jetty stuff to its own package io.druid.server.initialization.jetty
2015-03-05 19:33:27 -06:00
a8648d8f3d
feature to add servlet filters in a druid node via extension modules
2015-03-03 22:35:02 -06:00
d8e199a3f5
Merge pull request #1161 from himanshug/zk_ood_updates
...
Fix to maintain correctness when out-of-order ZK updates are received
2015-03-03 14:42:31 -08:00
dda2a62ff5
on receiving ZK ADD/UPDATE events, get latest data from zookeeper instead of taking it from the event which might be stale due to event coming out of order etc
2015-03-03 16:21:56 -06:00
9578304d6f
Merge pull request #1130 from housejester/jde-building
...
Various improvements to the tutorial and building experience.
2015-03-02 14:48:32 -08:00
e8605c63a9
Merge pull request #1150 from himanshug/broker-parallel-chunk-process
...
interval chunk query runner now processes individual chunk in a threadpool
2015-03-02 13:50:23 -08:00
29039fd541
interval chunk query runner now processes individual chunk in a thread pool and prints metrics query/time per chunk
2015-03-02 15:45:09 -06:00
a76a24742b
Merge pull request #1149 from himanshug/broker-get-datasource-update
...
updating ClientInfoResource.getDatasource() with a full flag
2015-03-02 13:28:27 -08:00
a39f4f8bf4
updating ClientInfoResource.getDatasource() with a full flag to return all the served intervals for given data source along with the dimension and metric information
2015-03-02 14:18:52 -06:00
17743b94f3
Cleanup router servlet + tests for gzip proxying
...
- Uses method overrides instead of modified Jetty code, now that
ProxyServlet provides enough method hooks for proper overrides.
This means we may also benefit from any Jetty ProxyServlet fixes
- Adds test for async proxy servlet to make sure gzip encoding is
properly proxied.
- Router now proxies POST requests for requests that are not Druid
queries, by only treating /druid/v2/* endpoints as queries.
2015-03-02 11:36:15 -08:00
c21086b6a8
Merge pull request #1113 from metamx/fix1111
...
Make Realtime queries not submit to thread pools for the master merger
2015-02-27 11:13:25 -08:00
9cb5d9bd00
Various improvements to the tutorial and building experience.
...
- Improve geo ip download stability
- Retry default geo ip download if corrupt
- Clarify docs on building & where to get artifact
2015-02-25 18:49:16 -07:00
b633111c76
Merge pull request #1156 from druid-io/fix-log
...
Print better messages when there are problems with ingestion
2015-02-25 17:02:46 -08:00
Xavier Léauté
Charles Allen
Bingkun Guo
Xavier Léauté
Parag Jain
Fangjin Yang
Himanshu
Eric Tschetter
nishant
Gian Merlino
Anubhav Gupta
msprunck
cheddar
Himanshu Gupta
James Estes