Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,205 Commits 111 Branches 503 Tags 730 MiB
Commit Graph

2237 Commits

Author SHA1 Message Date
Fangjin Yang 91a79dbf95 Merge pull request #1031 from metamx/ingestmetadata-query 
DataSourceMetadata query
 2015-01-19 21:55:35 -08:00
Fangjin Yang 56bad2114a Merge pull request #1054 from metamx/increase-test-wait-time 
use waiting multiple when waiting in AnnouncerTest
 2015-01-19 11:03:35 -08:00
Xavier Léauté b5f0d119f1 use waiting multiple when waiting in AnnouncerTest 2015-01-19 10:19:13 -08:00
fjy 52f65b1273 Fix a small bug where an NPE can occur if a closeable is not present 2015-01-17 15:23:01 -08:00
fjy 38c155e143 Merge branch 'master' of https://github.com/druid-io/druid 2015-01-16 13:14:49 -08:00
Fangjin Yang b4041c13e5 Merge pull request #1029 from metamx/fixChainedExecutionQueryRunnerTest 
Address spurious test failures
 2015-01-16 13:08:32 -08:00
Xavier Léauté 6bc6e71b0d Add messages to catch spurious test failures 2015-01-15 16:49:43 -08:00
fjy f7ea308b25 fix minor error output problem 2015-01-15 15:57:45 -08:00
Fangjin Yang 5bfcc43377 Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate 
Update all String conversions to and from byte[] to use the java-util StringUtils functions
 2015-01-15 13:50:27 -08:00
nishantmonu51 c7452b75f6 Merge branch 'master' into ingestmetadata-query 2015-01-15 18:00:31 +05:30
fjy 021a7de759 Add retry capabilities for realtime logic 2015-01-13 14:30:49 -08:00
fjy 84cc32ba32 Add more logging for when ingestion may be throttled 2015-01-08 14:51:46 -08:00
Charles Allen c1295202ff Allow background caching run after the futures are finished 2015-01-07 13:23:34 -08:00
Charles Allen f6fbb733b8 Added a few places where tests were using Object instead of Module 2015-01-05 13:47:25 -08:00
Fangjin Yang 852e863425 Merge pull request #981 from druid-io/strictModuleTyping 
Use Module instead of generic Object in Guice related items
 2015-01-05 12:43:20 -08:00
Fangjin Yang ab0ddc0d7c Merge pull request #998 from drcrallen/zkConfigJackson 
Change zk configs to use Jackson injection instead of Skife
 2015-01-05 12:42:40 -08:00
Charles Allen b1b5c9099e Update all String conversions to and from byte[] to use the java-util StringUtils functions 
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
 2015-01-05 11:22:32 -08:00
Charles Allen 81fdc791e0 Add memory hinting and concurrency helpers in CachingClusteredClient 2015-01-02 15:54:26 -08:00
Charles Allen 65286a24e0 Change zk configs to use Jackson injection instead of Skife 
* Also added generic config testing class JsonConfigTesterBase
 2014-12-29 10:36:12 -08:00
Xavier Léauté 574ab3017c fix test timeout 2014-12-23 16:25:38 -08:00
Fangjin Yang bf4b52384c Merge pull request #988 from drcrallen/queryErrorFixer 
Fix NPE in QueryResource on bad query
 2014-12-22 23:16:59 -08:00
Charles Allen 458236c8fd Add simple unit tests for QueryResource 2014-12-22 21:53:58 -08:00
Charles Allen 4338af0e3f Fix NPE in QueryResource on bad query 2014-12-22 12:59:47 -08:00
Gian Merlino badc09e85c Disable checkGzExists in the default GzipFilter. 
It is unhelpful because we don't actually have any precomputed .gz resources,
and checking for them inside jars is expensive.
 2014-12-22 12:45:48 -05:00
Charles Allen 7c8d4a7433 Use Module instead of generic Object in Guice related items 2014-12-19 10:54:06 -08:00
Charles Allen a0b2a63d7f Fix deadlock in DirectDruidClient 2014-12-16 18:34:30 -08:00
fjy e9238161cd fix broken test 2014-12-15 18:12:44 -08:00
Charles Allen 242edd14e5 Add deprecated smile media type for backwards compatability during upgrades 2014-12-15 18:00:10 -08:00
Charles Allen a520835972 Merge pull request #957 from metamx/fix-gzip-encoding 
fix gzip compression
 2014-12-15 13:20:36 -08:00
Xavier Léauté e166260f80 more inventory logs 2014-12-15 13:19:47 -08:00
fjy 9c6e32fc3f address cr 2014-12-15 13:06:36 -08:00
fjy 61101bee6c more logs for CIM 2014-12-15 12:57:40 -08:00
Fangjin Yang b3fe91bb50 Merge pull request #830 from metamx/union-merge-on-historical 
Union merge on historical
 2014-12-15 13:36:47 -07:00
Fangjin Yang 31cda0b830 Merge pull request #958 from metamx/fix-router-service-discovery 
fix router discovering brokers with default names
 2014-12-15 13:35:39 -07:00
Xavier Léauté 77bf5da317 fix typo 2014-12-15 10:07:51 -08:00
Charles Allen 3c33118f3a Merge pull request #962 from metamx/add-broker-loadstatus 
add loadstatus endpoint for serverView status
 2014-12-12 17:13:48 -08:00
Charles Allen ca8300a461 Replace AppendableByteArrayInputStream in DirectDruidClient 
* Replace with SequenceInputStream fueled by an enumeration of ChannelBufferInputStream which directly wrap the response context ChannelBuffer
* Added zero-length byte array when DirectDruidClient is done. This prevents an odd race condition on `done`.
 2014-12-12 14:01:13 -08:00
Xavier Léauté 4b5282d224 add loadstatus endpoint for serverView status 2014-12-12 13:18:31 -08:00
Xavier Léauté 352480ac77 make method protected to prevent accidental usage 
- add documentation as to why
 2014-12-11 15:20:36 -08:00
Xavier Léauté 3abf348d2c fix router discovering brokers with default names 2014-12-11 12:16:44 -08:00
nishantmonu51 7ad03087c0 Merge branch 'master' into ingestmetadata-query 2014-12-11 16:54:38 +05:30
nishantmonu51 32b4f55b8a review comments refactoring 2014-12-11 16:33:14 +05:30
Xavier Léauté eb661d3130 /status replaces /health 2014-12-10 17:35:05 -08:00
Xavier Léauté 0de56efe5c fix gzip compression 
- fixes compression not working for POST requests
- router now forwards raw bytes instead of decompressing
- cleanded up router servlet initialization
- add test for gzip on get and post methods
- use Jersey annotation when possible in QueryResource
 2014-12-10 17:30:17 -08:00
Xavier Léauté 7878d391bd use first available port for JettyTest 2014-12-10 12:53:11 -08:00
Charles Allen 6f6cc463b0 Merge pull request #950 from metamx/fix-bysegment-serde 
Fix bySegment metrics de-serialization
 2014-12-10 12:42:40 -08:00
Xavier Léauté 5f3f4e0a60 prevent /status from redirect, fixes #953 2014-12-10 11:32:22 -08:00
Xavier Léauté 7853e801d5 fixes for bySegment queries and caching 
- prevent bySegment queries from getting cached
- avoid computing cache keys if not populating or using cache
- avoid computing cache data when not populating cache
 2014-12-10 10:57:48 -08:00
nishantmonu51 3763357f6e Ingest metadata query implementation 2014-12-10 19:44:00 +05:30
Xavier Léauté c44242f0af add comment about de-serialing bySegment 2014-12-09 17:02:24 -08:00
Xavier Léauté a7da9aed9f fix metrics de-serialization for bySegment results 2014-12-09 17:02:23 -08:00
fjy 7620dfbd1a fix broken ut 2014-12-09 15:43:07 -08:00
Fangjin Yang d6d3ec6846 Merge pull request #948 from metamx/ingestion-docs 
Redocumenting ingestion
 2014-12-09 15:30:03 -07:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
fjy 28b72a69ad redocumenting ingestion 2014-12-08 16:15:46 -08:00
Fangjin Yang 623443a145 Merge pull request #946 from metamx/MoreMediaTypes 
Changed all "application/json" to MediaType.APPLICATION_JSON  except for in druid.js
 2014-12-08 15:24:21 -07:00
Charles Allen a0f9f9877e Changed all "application/json" to MediaType.APPLICATION_JSON except for in druid.js 2014-12-08 14:21:49 -08:00
xvrl 1392e2731f Merge pull request #936 from metamx/cachingRunnerImprovements 
General Caching Query Runners cleanup (40% query time reduction for HLL)
 2014-12-08 14:07:52 -08:00
Charles Allen 7b65f0635d General Caching Query Runners cleanup 
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
 2014-12-08 13:29:32 -08:00
Fangjin Yang c48839a0a0 Merge pull request #945 from metamx/MediaTypeFixes 
Added checks for backwards compat on MediaTypes
 2014-12-08 14:11:58 -07:00
Charles Allen 43041ed4ba Added checks for backwards compat on MediaTypes 2014-12-08 13:11:00 -08:00
Fangjin Yang eb233fe50e Merge pull request #921 from metamx/onheap-incremental-index 
separate implementations of  incremental index
 2014-12-08 13:59:08 -07:00
Charles Allen 03f7fae3e8 Changed references to MediaType to pull from javax.ws.rs.core.MediaType or com.fasterxml.jackson.jaxrs.smile.MediaType 2014-12-08 12:39:36 -08:00
Xavier Léauté c88de97370 Merge branch 'master' into onheap-incremental-index 
Conflicts:
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
 2014-12-08 11:48:21 -08:00
Fangjin Yang 3a7f571aeb Merge pull request #942 from metamx/fix-tiered-selector-naming 
fix tiered selector service name usage
 2014-12-08 12:09:45 -07:00
nishantmonu51 ef86dec660 remove unused but required parameter 2014-12-08 19:25:46 +05:30
nishantmonu51 4883f5526b fix tiered selector service name usage 2014-12-08 19:22:27 +05:30
Fangjin Yang c255c16c1a Merge pull request #894 from zhaown/fix-query-realtime-with-multi-partition 
bugfix for #887
 2014-12-07 09:49:31 -07:00
Xavier Léauté 629f6bc759 moved methods to java-util 2014-12-05 15:53:23 -08:00
Weinan Zhao 3119770ce9 Reformatting using intellij_formatting.jar 2014-12-05 14:21:24 +08:00
nishantmonu51 6e03a6245f Merge branch 'master' into onheap-incremental-index 2014-12-05 10:40:28 +05:30
Xavier Léauté d4a1752c5d add code to release ByteBuffers 2014-12-04 16:18:55 -08:00
Xavier Léauté 7cd45a6e1f IncrementalIndex throws exception if limit exceeded 
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
 2014-12-04 14:38:16 -08:00
Gian Merlino 20a7239ffd Replace google-http-client imports with real guava imports. 2014-12-04 10:57:57 -08:00
Gian Merlino b48fc51ed1 SSL-capable ServiceEmitter. 2014-12-04 10:57:56 -08:00
Xavier Léauté 0c521e0a77 update joda-time and fix min/max instant 2014-12-04 10:57:56 -08:00
fjy 187126dd4b fix broken unit tests 2014-12-04 10:49:36 -08:00
Fangjin Yang 332e4a4d1c Merge pull request #902 from metamx/cache-config 
Commonalize the cache config and change default to disable cache
 2014-12-04 11:30:24 -07:00
xvrl 2681da4420 Merge pull request #929 from metamx/google-cleanup 
Replace google-http-client imports with real guava imports.
 2014-12-03 11:50:19 -08:00
Gian Merlino d388a8fe89 Replace google-http-client imports with real guava imports. 2014-12-03 10:52:57 -08:00
Gian Merlino 03ae70ae54 SSL-capable ServiceEmitter. 2014-12-03 10:43:24 -08:00
nishantmonu51 4dc0fdba8a consider mapped size in limit calculation & review comments 2014-12-03 23:47:30 +05:30
nishantmonu51 da8bd7836b Introduce buffer size 2014-12-03 16:28:22 +05:30
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
nishantmonu51 eac776f1a7 tests passing with on heap incremental index 2014-12-02 22:29:28 +05:30
Xavier Léauté 59542c41f8 fix port not set in DruidNode 2014-12-01 14:37:28 -08:00
Charles Allen 7ed5f995cb Modify InitializationTest to use a port of 0 instead of -1 2014-12-01 13:48:49 -08:00
Xavier Léauté 36c9211791 remove unused imports 2014-12-01 13:34:58 -08:00
Xavier Léauté 839e288ed5 request proper type from DBI instead of casting 2014-12-01 13:27:52 -08:00
Fangjin Yang 646e184ca6 Merge pull request #918 from metamx/fix-broker-timeouts 
fix query timeouts not being raised by broker
 2014-12-01 13:58:04 -07:00
Xavier Léauté 445af2dc5f use guava methods instead 2014-12-01 12:53:20 -08:00
Xavier Léauté eb5525f9b4 add support for ipv6 hostnames 2014-12-01 12:27:42 -08:00
Xavier Léauté d23fd1e1ab make host+port more explicit 
- document the behavior for node host/port initialization
- throw exception if settings make no sense
- fixes announcement for nodes without host/port defaults
- makes code clearer as to when host vs. host+port are used
 2014-11-26 22:03:25 -08:00
Xavier Léauté 4dccfda61d fix query timeouts not being raised by broker 2014-11-26 21:42:07 -08:00
Xavier Léauté 523f9c53cd update curator and fix test 
curator is more strict now
 2014-11-26 14:41:08 -08:00
Xavier Léauté d4bda82916 remove unused imports 2014-11-26 14:41:08 -08:00
nishantmonu51 76891d450e update spymemcached library 
update spymemcached library
for fixes mainly related to -
1) memory leaks on reconnects
2) better handling of network issues
 2014-11-26 19:33:52 +05:30
Xavier Léauté 9fb93a3a79 fix smile exception on error + content-type 2014-11-25 11:30:17 -08:00
Weinan Zhao bcc5f86723 Modify according to fjy's review, add reformatting according to eaclipse_formatting.xml 2014-11-26 00:14:40 +08:00
fjy 083a012aa2 Commonalize the cache config and change default to disable cache 2014-11-24 14:54:11 -08:00
xvrl 829c254bac Merge pull request #898 from metamx/refactor-examples 
Refactor structure for examples and extensions
 2014-11-24 14:02:34 -08:00
fjy f42b1a4181 fix broken ut 2014-11-24 11:42:24 -08:00
xvrl 37e5678870 Merge pull request #899 from metamx/fix-log 
fix crazy amounts of logging when coordinator loses leadership
 2014-11-24 11:01:44 -08:00
fjy 13cae41f6c Merge branch 'master' into refactor-examples 2014-11-24 11:00:26 -08:00
xvrl 9ced097abd Merge pull request #895 from metamx/fix-interval-retry 
A set of fixes to retry the query for missing segments in the timeline
 2014-11-24 10:23:02 -08:00
fjy f518c53008 address cr 2014-11-24 10:12:29 -08:00
fjy 10d45ab9cf address code review 2014-11-21 16:48:45 -08:00
fjy f9db3a1fc6 fix crazy amounts of logging when coordinator loses leadership 2014-11-21 15:46:14 -08:00
fjy 8ee4d12562 Refactor structure for examples and extensions 2014-11-21 14:45:24 -08:00
fjy 580e1172c1 move IndexTask to use hashed partition; fixes #815 2014-11-21 11:15:25 -08:00
Fangjin Yang 780e4a27f7 Merge pull request #868 from metamx/fix-bad-test 
fix non-determininistic reference counting segment test
 2014-11-20 15:25:57 -07:00
fjy e044598669 make wait timeout a bit longer 2014-11-20 14:25:49 -08:00
fjy 0d275c7a53 remove another wait 2014-11-20 14:22:16 -08:00
fjy 47f5c1bd0a fix retry interval is stupid 2014-11-20 12:50:56 -08:00
fjy 3d9d989a9f A set of fixes to retry the query for missing intervals in the timeline 2014-11-20 12:04:37 -08:00
Weinan Zhao 5096cbc5e4 fix bug for #887, that is cannot query multi partitions of one datasource on realtime node. 2014-11-20 16:36:57 +08:00
fjy fdeab0c6af make Druid case sensitive 2014-11-19 14:27:31 -08:00
Fangjin Yang 590d31799e Merge pull request #876 from metamx/remove-backwards-compatible 
Remove backwards compatible
 2014-11-19 14:33:14 -07:00
xvrl a96eaeb036 Merge pull request #882 from metamx/now_with_OPEN_SOURCE 
Added src jar build to maven poms and re-formatted to conform to style guidelines.
 2014-11-18 13:00:04 -08:00
Fangjin Yang 87ea6fd377 Merge pull request #878 from metamx/default-extension-version 
use Druid version as the default version for extensions
 2014-11-18 13:56:54 -07:00
Xavier Léauté a41f179cc0 fix output content-type 2014-11-18 11:08:44 -08:00
Charles Allen 8819a1b9d5 Add Smile annotation to BrokerServerView 2014-11-18 10:18:10 -08:00
Charles Allen dc66e1708e Added src jar build to maven poms and re-formatted to conform to style guidelines. 2014-11-18 09:05:30 -08:00
Xavier Léauté d914afe1cd make defaultVersion configurable for non-jar testing 2014-11-17 13:54:32 -08:00
Xavier Léauté 0ca23d7d0b use Druid version as the default version for extensions 2014-11-17 13:36:37 -08:00
nishantmonu51 0c2d06475d merge from master 2014-11-17 19:19:18 +05:30
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
Fangjin Yang 0b45942cff Merge pull request #867 from metamx/avoid-abandoned-sinks 
RealtimePlumber: Bail out of persist-n-merge of abandoned sinks (fixes #812)
 2014-11-14 18:35:00 -07:00
xvrl e1e171ad20 Merge pull request #865 from metamx/fix-retry-qr 
Fix a couple of bugs with retry query runner after testing it locally
 2014-11-14 13:33:51 -08:00
xvrl a4fc64ca3f Merge pull request #856 from metamx/druid-845 
Fix query by segment
 2014-11-14 13:10:54 -08:00
fjy bbc079b880 fix retry to actually return correct sequences 2014-11-14 12:10:04 -08:00
Xavier Léauté 07c4923bf8 add test for out of order merging 2014-11-14 11:17:50 -08:00
Xavier Léauté 6e8ac89737 refactor sequence merging method to simplify testing 2014-11-14 11:17:50 -08:00
Xavier Léauté 716cc2cff2 fix results ordering 2014-11-13 14:59:56 -08:00
fjy 313d81ca2f fix non-determininistic reference counting segment test 2014-11-13 13:39:56 -08:00
Gian Merlino bf22524d63 RealtimePlumber: Bail out of persist-n-merge of abandoned sinks (fixes #812). 2014-11-13 12:59:49 -08:00
fjy 0d6816a037 Fix a couple of bugs with retry query runner after testing it locally 2014-11-13 11:53:29 -08:00
Fangjin Yang 9d661d6d36 Merge pull request #854 from metamx/serverInventoryViewJson 
Better logging in JSON parse errors in ServerInventoryView
 2014-11-13 11:56:59 -07:00
Charles Allen 9a9238a801 Merge remote-tracking branch 'origin/master' into druid-845 2014-11-13 10:04:56 -08:00
Charles Allen 3cdcf8736f Changed downloadStartMarker in OmniSegmentLoader to simply use its .delete() method 2014-11-13 09:52:41 -08:00
Xavier Léauté ba442b9882 fix default FilteredServerView + batch annoucement 2014-11-12 23:17:16 -08:00
fjy 3ef21bfc66 fix leak with files not cleaning up correctly 2014-11-12 22:29:50 -08:00
Charles Allen 31fed7d329 Fix query by segment 
* Changed topN queries to use joda Interval instead of string values
* topN by segment now implements BySegmentResultValue<Result<TopNResultValue>> instead of BySegmentResultValue<TopNResultValue>
* Added a unit test which failed uner the prior implementation.
 2014-11-12 21:20:59 -08:00
Charles Allen a8103fb02d Better logging in JSON parse errors in ServerInventoryView 2014-11-12 16:52:15 -08:00
xvrl 10b7ca9fa9 Merge pull request #849 from metamx/default-coordinator-service-name 
Default service names
 2014-11-12 15:06:02 -08:00
Xavier Léauté 3a9def7fb1 add broker default and define constants 2014-11-12 14:53:53 -08:00
Xavier Léauté 4ac1aaf90e Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	processing/src/main/java/io/druid/segment/QueryableIndexStorageAdapter.java
 2014-11-12 14:08:51 -08:00
Xavier Léauté dbab127a6d default zk host to localhost 2014-11-12 13:56:07 -08:00
Fangjin Yang 58260165ad Merge pull request #842 from metamx/fix-timestamp-column-exclusion 
fix dimension exclusions for timestamp and aggs
 2014-11-12 09:44:50 -07:00
Charles Allen 847c01a045 Merge pull request #844 from metamx/derby-dependencies-fix 
specify versions in parent pom
 2014-11-12 08:28:55 -08:00