Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,106 Commits 111 Branches 503 Tags 721 MiB
Commit Graph

1673 Commits

Author SHA1 Message Date
Tarek Rached f47f0d51c7 removed unecessary logging 
fixed simplifyInterval logic
 2015-02-06 21:13:25 -08:00
Tarek Rached a0fc20fc09 better regex for durations 2015-02-06 17:02:31 -08:00
Tarek Rached dae91e5894 fixed links back to old console 2015-02-04 14:21:46 -08:00
Xavier Léauté ffb180e29d Merge pull request #1080 from metamx/fix-cost-balancer-strategy 
fix integer value overflow and add test
 2015-02-03 11:45:03 -08:00
nishantmonu51 6da6b9eeda fix integer value overflow and add test 2015-02-03 11:39:30 -08:00
Eric Tschetter 42eba986ce Towards consistent null handling 
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
 2015-02-02 12:53:07 -08:00
Fangjin Yang 92e616de11 Merge pull request #1077 from metamx/remove-unused-imports 
remove unused imports
 2015-02-02 10:45:27 -08:00
nishantmonu51 ba932bb1f2 remove unused imports 2015-02-02 21:53:39 +05:30
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
Fangjin Yang b94b24c46e Merge pull request #1072 from metamx/common-aws-config 
Common AWS module + consistent credential chain for both S3 and auto-scaling
 2015-01-30 14:41:24 -08:00
Fangjin Yang 14837d7808 Merge pull request #1075 from metamx/fix-test-timeout 
increase test timeouts on DruidClusterBridgeTest
 2015-01-30 14:31:32 -08:00
Xavier Léauté cd7e768640 increase test timeouts on DruidClusterBridgeTest 2015-01-30 13:18:35 -08:00
fjy bc1405bee0 fix worker status path announcement with indexer zk config 2015-01-30 12:26:08 -08:00
Xavier Léauté f00872c41b move common AWS related classes into a separate module 2015-01-29 13:55:49 -08:00
Fangjin Yang 2d96b62e6c Merge pull request #1066 from metamx/fix-background-caching-ordering 
fix background caching not preserving result order
 2015-01-28 06:09:17 +08:00
Xavier Léauté 59c1907322 add test for out-of-order background-caching execution 2015-01-27 13:38:08 -08:00
fjy cd9ba1f852 fix serde bug 2015-01-27 18:47:36 +08:00
Xavier Léauté b334d7973d fix background caching not preserving result order 2015-01-26 09:59:18 -08:00
Xavier Léauté 26c5c2193f fix BatchServerInventoryViewTest test timeout 2015-01-20 17:14:57 -08:00
Xavier Léauté 5a2030c49b Merge pull request #1059 from druid-io/remove-test 
Remove test rejection policy - it has no place anywhere
 2015-01-20 13:39:46 -08:00
fjy 1ff5829d6d remove test rejection policy 2015-01-20 13:22:30 -08:00
Charles Allen 3d27747f7e Upgrade to log4j2 
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
 2015-01-20 12:35:18 -08:00
Fangjin Yang 91a79dbf95 Merge pull request #1031 from metamx/ingestmetadata-query 
DataSourceMetadata query
 2015-01-19 21:55:35 -08:00
Fangjin Yang 56bad2114a Merge pull request #1054 from metamx/increase-test-wait-time 
use waiting multiple when waiting in AnnouncerTest
 2015-01-19 11:03:35 -08:00
Xavier Léauté b5f0d119f1 use waiting multiple when waiting in AnnouncerTest 2015-01-19 10:19:13 -08:00
fjy 52f65b1273 Fix a small bug where an NPE can occur if a closeable is not present 2015-01-17 15:23:01 -08:00
fjy 38c155e143 Merge branch 'master' of https://github.com/druid-io/druid 2015-01-16 13:14:49 -08:00
Fangjin Yang b4041c13e5 Merge pull request #1029 from metamx/fixChainedExecutionQueryRunnerTest 
Address spurious test failures
 2015-01-16 13:08:32 -08:00
Xavier Léauté 6bc6e71b0d Add messages to catch spurious test failures 2015-01-15 16:49:43 -08:00
fjy f7ea308b25 fix minor error output problem 2015-01-15 15:57:45 -08:00
Fangjin Yang 5bfcc43377 Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate 
Update all String conversions to and from byte[] to use the java-util StringUtils functions
 2015-01-15 13:50:27 -08:00
nishantmonu51 c7452b75f6 Merge branch 'master' into ingestmetadata-query 2015-01-15 18:00:31 +05:30
fjy 021a7de759 Add retry capabilities for realtime logic 2015-01-13 14:30:49 -08:00
fjy 84cc32ba32 Add more logging for when ingestion may be throttled 2015-01-08 14:51:46 -08:00
Charles Allen c1295202ff Allow background caching run after the futures are finished 2015-01-07 13:23:34 -08:00
Charles Allen f6fbb733b8 Added a few places where tests were using Object instead of Module 2015-01-05 13:47:25 -08:00
Fangjin Yang 852e863425 Merge pull request #981 from druid-io/strictModuleTyping 
Use Module instead of generic Object in Guice related items
 2015-01-05 12:43:20 -08:00
Fangjin Yang ab0ddc0d7c Merge pull request #998 from drcrallen/zkConfigJackson 
Change zk configs to use Jackson injection instead of Skife
 2015-01-05 12:42:40 -08:00
Charles Allen b1b5c9099e Update all String conversions to and from byte[] to use the java-util StringUtils functions 
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
 2015-01-05 11:22:32 -08:00
Charles Allen 81fdc791e0 Add memory hinting and concurrency helpers in CachingClusteredClient 2015-01-02 15:54:26 -08:00
Charles Allen 65286a24e0 Change zk configs to use Jackson injection instead of Skife 
* Also added generic config testing class JsonConfigTesterBase
 2014-12-29 10:36:12 -08:00
Xavier Léauté 574ab3017c fix test timeout 2014-12-23 16:25:38 -08:00
Fangjin Yang bf4b52384c Merge pull request #988 from drcrallen/queryErrorFixer 
Fix NPE in QueryResource on bad query
 2014-12-22 23:16:59 -08:00
Charles Allen 458236c8fd Add simple unit tests for QueryResource 2014-12-22 21:53:58 -08:00
Charles Allen 4338af0e3f Fix NPE in QueryResource on bad query 2014-12-22 12:59:47 -08:00
Gian Merlino badc09e85c Disable checkGzExists in the default GzipFilter. 
It is unhelpful because we don't actually have any precomputed .gz resources,
and checking for them inside jars is expensive.
 2014-12-22 12:45:48 -05:00
Charles Allen 7c8d4a7433 Use Module instead of generic Object in Guice related items 2014-12-19 10:54:06 -08:00
Charles Allen a0b2a63d7f Fix deadlock in DirectDruidClient 2014-12-16 18:34:30 -08:00
fjy e9238161cd fix broken test 2014-12-15 18:12:44 -08:00
Charles Allen 242edd14e5 Add deprecated smile media type for backwards compatability during upgrades 2014-12-15 18:00:10 -08:00
Charles Allen a520835972 Merge pull request #957 from metamx/fix-gzip-encoding 
fix gzip compression
 2014-12-15 13:20:36 -08:00
Xavier Léauté e166260f80 more inventory logs 2014-12-15 13:19:47 -08:00
fjy 9c6e32fc3f address cr 2014-12-15 13:06:36 -08:00
fjy 61101bee6c more logs for CIM 2014-12-15 12:57:40 -08:00
Fangjin Yang b3fe91bb50 Merge pull request #830 from metamx/union-merge-on-historical 
Union merge on historical
 2014-12-15 13:36:47 -07:00
Fangjin Yang 31cda0b830 Merge pull request #958 from metamx/fix-router-service-discovery 
fix router discovering brokers with default names
 2014-12-15 13:35:39 -07:00
Xavier Léauté 77bf5da317 fix typo 2014-12-15 10:07:51 -08:00
Charles Allen 3c33118f3a Merge pull request #962 from metamx/add-broker-loadstatus 
add loadstatus endpoint for serverView status
 2014-12-12 17:13:48 -08:00
Charles Allen ca8300a461 Replace AppendableByteArrayInputStream in DirectDruidClient 
* Replace with SequenceInputStream fueled by an enumeration of ChannelBufferInputStream which directly wrap the response context ChannelBuffer
* Added zero-length byte array when DirectDruidClient is done. This prevents an odd race condition on `done`.
 2014-12-12 14:01:13 -08:00
Xavier Léauté 4b5282d224 add loadstatus endpoint for serverView status 2014-12-12 13:18:31 -08:00
Xavier Léauté 352480ac77 make method protected to prevent accidental usage 
- add documentation as to why
 2014-12-11 15:20:36 -08:00
Xavier Léauté 3abf348d2c fix router discovering brokers with default names 2014-12-11 12:16:44 -08:00
nishantmonu51 7ad03087c0 Merge branch 'master' into ingestmetadata-query 2014-12-11 16:54:38 +05:30
nishantmonu51 32b4f55b8a review comments refactoring 2014-12-11 16:33:14 +05:30
Xavier Léauté eb661d3130 /status replaces /health 2014-12-10 17:35:05 -08:00
Xavier Léauté 0de56efe5c fix gzip compression 
- fixes compression not working for POST requests
- router now forwards raw bytes instead of decompressing
- cleanded up router servlet initialization
- add test for gzip on get and post methods
- use Jersey annotation when possible in QueryResource
 2014-12-10 17:30:17 -08:00
Xavier Léauté 7878d391bd use first available port for JettyTest 2014-12-10 12:53:11 -08:00
Charles Allen 6f6cc463b0 Merge pull request #950 from metamx/fix-bysegment-serde 
Fix bySegment metrics de-serialization
 2014-12-10 12:42:40 -08:00
Xavier Léauté 5f3f4e0a60 prevent /status from redirect, fixes #953 2014-12-10 11:32:22 -08:00
Xavier Léauté 7853e801d5 fixes for bySegment queries and caching 
- prevent bySegment queries from getting cached
- avoid computing cache keys if not populating or using cache
- avoid computing cache data when not populating cache
 2014-12-10 10:57:48 -08:00
nishantmonu51 3763357f6e Ingest metadata query implementation 2014-12-10 19:44:00 +05:30
Xavier Léauté c44242f0af add comment about de-serialing bySegment 2014-12-09 17:02:24 -08:00
Xavier Léauté a7da9aed9f fix metrics de-serialization for bySegment results 2014-12-09 17:02:23 -08:00
fjy 7620dfbd1a fix broken ut 2014-12-09 15:43:07 -08:00
Fangjin Yang d6d3ec6846 Merge pull request #948 from metamx/ingestion-docs 
Redocumenting ingestion
 2014-12-09 15:30:03 -07:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
fjy 28b72a69ad redocumenting ingestion 2014-12-08 16:15:46 -08:00
Fangjin Yang 623443a145 Merge pull request #946 from metamx/MoreMediaTypes 
Changed all "application/json" to MediaType.APPLICATION_JSON  except for in druid.js
 2014-12-08 15:24:21 -07:00
Charles Allen a0f9f9877e Changed all "application/json" to MediaType.APPLICATION_JSON except for in druid.js 2014-12-08 14:21:49 -08:00
xvrl 1392e2731f Merge pull request #936 from metamx/cachingRunnerImprovements 
General Caching Query Runners cleanup (40% query time reduction for HLL)
 2014-12-08 14:07:52 -08:00
Charles Allen 7b65f0635d General Caching Query Runners cleanup 
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
 2014-12-08 13:29:32 -08:00
Fangjin Yang c48839a0a0 Merge pull request #945 from metamx/MediaTypeFixes 
Added checks for backwards compat on MediaTypes
 2014-12-08 14:11:58 -07:00
Charles Allen 43041ed4ba Added checks for backwards compat on MediaTypes 2014-12-08 13:11:00 -08:00
Fangjin Yang eb233fe50e Merge pull request #921 from metamx/onheap-incremental-index 
separate implementations of  incremental index
 2014-12-08 13:59:08 -07:00
Charles Allen 03f7fae3e8 Changed references to MediaType to pull from javax.ws.rs.core.MediaType or com.fasterxml.jackson.jaxrs.smile.MediaType 2014-12-08 12:39:36 -08:00
Xavier Léauté c88de97370 Merge branch 'master' into onheap-incremental-index 
Conflicts:
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
 2014-12-08 11:48:21 -08:00
Fangjin Yang 3a7f571aeb Merge pull request #942 from metamx/fix-tiered-selector-naming 
fix tiered selector service name usage
 2014-12-08 12:09:45 -07:00
nishantmonu51 ef86dec660 remove unused but required parameter 2014-12-08 19:25:46 +05:30
nishantmonu51 4883f5526b fix tiered selector service name usage 2014-12-08 19:22:27 +05:30
Fangjin Yang c255c16c1a Merge pull request #894 from zhaown/fix-query-realtime-with-multi-partition 
bugfix for #887
 2014-12-07 09:49:31 -07:00
Xavier Léauté 629f6bc759 moved methods to java-util 2014-12-05 15:53:23 -08:00
Weinan Zhao 3119770ce9 Reformatting using intellij_formatting.jar 2014-12-05 14:21:24 +08:00
nishantmonu51 6e03a6245f Merge branch 'master' into onheap-incremental-index 2014-12-05 10:40:28 +05:30
Xavier Léauté d4a1752c5d add code to release ByteBuffers 2014-12-04 16:18:55 -08:00
Xavier Léauté 7cd45a6e1f IncrementalIndex throws exception if limit exceeded 
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
 2014-12-04 14:38:16 -08:00
Gian Merlino 20a7239ffd Replace google-http-client imports with real guava imports. 2014-12-04 10:57:57 -08:00
Gian Merlino b48fc51ed1 SSL-capable ServiceEmitter. 2014-12-04 10:57:56 -08:00
Xavier Léauté 0c521e0a77 update joda-time and fix min/max instant 2014-12-04 10:57:56 -08:00
fjy 187126dd4b fix broken unit tests 2014-12-04 10:49:36 -08:00
Fangjin Yang 332e4a4d1c Merge pull request #902 from metamx/cache-config 
Commonalize the cache config and change default to disable cache
 2014-12-04 11:30:24 -07:00
xvrl 2681da4420 Merge pull request #929 from metamx/google-cleanup 
Replace google-http-client imports with real guava imports.
 2014-12-03 11:50:19 -08:00
Gian Merlino d388a8fe89 Replace google-http-client imports with real guava imports. 2014-12-03 10:52:57 -08:00
Gian Merlino 03ae70ae54 SSL-capable ServiceEmitter. 2014-12-03 10:43:24 -08:00
nishantmonu51 4dc0fdba8a consider mapped size in limit calculation & review comments 2014-12-03 23:47:30 +05:30
nishantmonu51 da8bd7836b Introduce buffer size 2014-12-03 16:28:22 +05:30
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
nishantmonu51 eac776f1a7 tests passing with on heap incremental index 2014-12-02 22:29:28 +05:30
Xavier Léauté 59542c41f8 fix port not set in DruidNode 2014-12-01 14:37:28 -08:00
Charles Allen 7ed5f995cb Modify InitializationTest to use a port of 0 instead of -1 2014-12-01 13:48:49 -08:00
Xavier Léauté 36c9211791 remove unused imports 2014-12-01 13:34:58 -08:00
Xavier Léauté 839e288ed5 request proper type from DBI instead of casting 2014-12-01 13:27:52 -08:00
Fangjin Yang 646e184ca6 Merge pull request #918 from metamx/fix-broker-timeouts 
fix query timeouts not being raised by broker
 2014-12-01 13:58:04 -07:00
Xavier Léauté 445af2dc5f use guava methods instead 2014-12-01 12:53:20 -08:00
Xavier Léauté eb5525f9b4 add support for ipv6 hostnames 2014-12-01 12:27:42 -08:00
Xavier Léauté d23fd1e1ab make host+port more explicit 
- document the behavior for node host/port initialization
- throw exception if settings make no sense
- fixes announcement for nodes without host/port defaults
- makes code clearer as to when host vs. host+port are used
 2014-11-26 22:03:25 -08:00
Xavier Léauté 4dccfda61d fix query timeouts not being raised by broker 2014-11-26 21:42:07 -08:00
Xavier Léauté 523f9c53cd update curator and fix test 
curator is more strict now
 2014-11-26 14:41:08 -08:00
Xavier Léauté d4bda82916 remove unused imports 2014-11-26 14:41:08 -08:00
nishantmonu51 76891d450e update spymemcached library 
update spymemcached library
for fixes mainly related to -
1) memory leaks on reconnects
2) better handling of network issues
 2014-11-26 19:33:52 +05:30
Xavier Léauté 9fb93a3a79 fix smile exception on error + content-type 2014-11-25 11:30:17 -08:00
Weinan Zhao bcc5f86723 Modify according to fjy's review, add reformatting according to eaclipse_formatting.xml 2014-11-26 00:14:40 +08:00
fjy 083a012aa2 Commonalize the cache config and change default to disable cache 2014-11-24 14:54:11 -08:00
xvrl 829c254bac Merge pull request #898 from metamx/refactor-examples 
Refactor structure for examples and extensions
 2014-11-24 14:02:34 -08:00
fjy f42b1a4181 fix broken ut 2014-11-24 11:42:24 -08:00
xvrl 37e5678870 Merge pull request #899 from metamx/fix-log 
fix crazy amounts of logging when coordinator loses leadership
 2014-11-24 11:01:44 -08:00
fjy 13cae41f6c Merge branch 'master' into refactor-examples 2014-11-24 11:00:26 -08:00
xvrl 9ced097abd Merge pull request #895 from metamx/fix-interval-retry 
A set of fixes to retry the query for missing segments in the timeline
 2014-11-24 10:23:02 -08:00
fjy f518c53008 address cr 2014-11-24 10:12:29 -08:00
fjy 10d45ab9cf address code review 2014-11-21 16:48:45 -08:00
fjy f9db3a1fc6 fix crazy amounts of logging when coordinator loses leadership 2014-11-21 15:46:14 -08:00
fjy 8ee4d12562 Refactor structure for examples and extensions 2014-11-21 14:45:24 -08:00
fjy 580e1172c1 move IndexTask to use hashed partition; fixes #815 2014-11-21 11:15:25 -08:00
Fangjin Yang 780e4a27f7 Merge pull request #868 from metamx/fix-bad-test 
fix non-determininistic reference counting segment test
 2014-11-20 15:25:57 -07:00
fjy e044598669 make wait timeout a bit longer 2014-11-20 14:25:49 -08:00
fjy 0d275c7a53 remove another wait 2014-11-20 14:22:16 -08:00
fjy 47f5c1bd0a fix retry interval is stupid 2014-11-20 12:50:56 -08:00
fjy 3d9d989a9f A set of fixes to retry the query for missing intervals in the timeline 2014-11-20 12:04:37 -08:00
Weinan Zhao 5096cbc5e4 fix bug for #887, that is cannot query multi partitions of one datasource on realtime node. 2014-11-20 16:36:57 +08:00
fjy fdeab0c6af make Druid case sensitive 2014-11-19 14:27:31 -08:00
Fangjin Yang 590d31799e Merge pull request #876 from metamx/remove-backwards-compatible 
Remove backwards compatible
 2014-11-19 14:33:14 -07:00
Fangjin Yang 87ea6fd377 Merge pull request #878 from metamx/default-extension-version 
use Druid version as the default version for extensions
 2014-11-18 13:56:54 -07:00
Xavier Léauté a41f179cc0 fix output content-type 2014-11-18 11:08:44 -08:00
Charles Allen 8819a1b9d5 Add Smile annotation to BrokerServerView 2014-11-18 10:18:10 -08:00
Xavier Léauté d914afe1cd make defaultVersion configurable for non-jar testing 2014-11-17 13:54:32 -08:00
Xavier Léauté 0ca23d7d0b use Druid version as the default version for extensions 2014-11-17 13:36:37 -08:00
nishantmonu51 0c2d06475d merge from master 2014-11-17 19:19:18 +05:30
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
Fangjin Yang 0b45942cff Merge pull request #867 from metamx/avoid-abandoned-sinks 
RealtimePlumber: Bail out of persist-n-merge of abandoned sinks (fixes #812)
 2014-11-14 18:35:00 -07:00
xvrl e1e171ad20 Merge pull request #865 from metamx/fix-retry-qr 
Fix a couple of bugs with retry query runner after testing it locally
 2014-11-14 13:33:51 -08:00
xvrl a4fc64ca3f Merge pull request #856 from metamx/druid-845 
Fix query by segment
 2014-11-14 13:10:54 -08:00
fjy bbc079b880 fix retry to actually return correct sequences 2014-11-14 12:10:04 -08:00
Xavier Léauté 07c4923bf8 add test for out of order merging 2014-11-14 11:17:50 -08:00
Xavier Léauté 6e8ac89737 refactor sequence merging method to simplify testing 2014-11-14 11:17:50 -08:00
Xavier Léauté 716cc2cff2 fix results ordering 2014-11-13 14:59:56 -08:00
fjy 313d81ca2f fix non-determininistic reference counting segment test 2014-11-13 13:39:56 -08:00
Gian Merlino bf22524d63 RealtimePlumber: Bail out of persist-n-merge of abandoned sinks (fixes #812). 2014-11-13 12:59:49 -08:00
fjy 0d6816a037 Fix a couple of bugs with retry query runner after testing it locally 2014-11-13 11:53:29 -08:00
Fangjin Yang 9d661d6d36 Merge pull request #854 from metamx/serverInventoryViewJson 
Better logging in JSON parse errors in ServerInventoryView
 2014-11-13 11:56:59 -07:00
Charles Allen 9a9238a801 Merge remote-tracking branch 'origin/master' into druid-845 2014-11-13 10:04:56 -08:00
Charles Allen 3cdcf8736f Changed downloadStartMarker in OmniSegmentLoader to simply use its .delete() method 2014-11-13 09:52:41 -08:00
Xavier Léauté ba442b9882 fix default FilteredServerView + batch annoucement 2014-11-12 23:17:16 -08:00
fjy 3ef21bfc66 fix leak with files not cleaning up correctly 2014-11-12 22:29:50 -08:00
Charles Allen 31fed7d329 Fix query by segment 
* Changed topN queries to use joda Interval instead of string values
* topN by segment now implements BySegmentResultValue<Result<TopNResultValue>> instead of BySegmentResultValue<TopNResultValue>
* Added a unit test which failed uner the prior implementation.
 2014-11-12 21:20:59 -08:00
Charles Allen a8103fb02d Better logging in JSON parse errors in ServerInventoryView 2014-11-12 16:52:15 -08:00
xvrl 10b7ca9fa9 Merge pull request #849 from metamx/default-coordinator-service-name 
Default service names
 2014-11-12 15:06:02 -08:00
Xavier Léauté 3a9def7fb1 add broker default and define constants 2014-11-12 14:53:53 -08:00
Xavier Léauté 4ac1aaf90e Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	processing/src/main/java/io/druid/segment/QueryableIndexStorageAdapter.java
 2014-11-12 14:08:51 -08:00
Xavier Léauté dbab127a6d default zk host to localhost 2014-11-12 13:56:07 -08:00
Fangjin Yang 58260165ad Merge pull request #842 from metamx/fix-timestamp-column-exclusion 
fix dimension exclusions for timestamp and aggs
 2014-11-12 09:44:50 -07:00
Xavier Léauté 69146736a8 fix dimension exclusions for timestamp and aggs 
- also added tests
 2014-11-11 20:56:17 -08:00
Xavier Léauté 87b72e8822 fix non-deterministic test 2014-11-11 20:26:13 -08:00
Charles Allen a89b539b4f Merge pull request #823 from metamx/roaring 
Configurable bitmap indexes: roaring and concise
 2014-11-11 17:26:38 -08:00
Xavier Léauté 59f093882d fix Derby instantiation 2014-11-10 16:55:53 -08:00
fjy 8cbd21af90 fix defaults 2014-11-10 15:40:57 -08:00
fjy 98d05e63bf make derby instance only on coordinator 2014-11-10 15:34:49 -08:00
fjy 6188315293 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/query/search/SearchQueryRunnerTest.java
 2014-11-10 14:52:10 -08:00
nishantmonu51 42a61290a8 specify default service name for coordinator 2014-11-10 23:36:01 +05:30
fjy 3c21f62afd a whole bunch of fixes and some ugly conversion code 2014-11-07 16:03:22 -08:00
Xavier Léauté 0498df25df override metadata storage injection in CliHadoopIndexer 2014-11-07 13:44:56 -08:00
Fangjin Yang 2336e6c167 Merge pull request #758 from metamx/jisoo-metadata 
make metadata storage pluggable
 2014-11-07 11:30:11 -07:00
nishantmonu51 5be801356a fix query/wait emitted twice 2014-11-07 23:32:36 +05:30
nishantmonu51 fd8eb7742b handle union query on realtime node 2014-11-07 23:27:50 +05:30
nishantmonu51 8bebb24fd5 Union Queries move merge to historical 2014-11-07 18:05:16 +05:30
Xavier Léauté d8bab70647 provide proper defaults for indexing service name 2014-11-06 18:16:01 -08:00
Xavier Léauté a5f98be457 remove unused code 2014-11-06 17:18:40 -08:00
Xavier Léauté f69f3cfc5a run MetadataSegmentManagerTest against Derby 2014-11-06 17:08:54 -08:00
Xavier Léauté ac753ed0f0 always use validation query 2014-11-06 17:08:01 -08:00
Xavier Léauté 350bb09605 refactor sql storage to abstract task storage 2014-11-05 17:19:37 -08:00
Xavier Léauté 253c0a70b3 simplify interface + more tests 2014-11-04 15:03:13 -08:00
Xavier Léauté b453922319 add tests for task and task status storage 2014-11-04 14:35:39 -08:00
Xavier Léauté adb4fec264 properly serialize task timestamps, fixes #819 2014-11-04 11:06:34 -08:00
xvrl 2c62b9e6e3 Merge pull request #821 from metamx/minor-fix 
Some minor fixes in docs and change jetty defaults
 2014-11-04 10:18:57 -08:00
fjy c6d555e1f3 better default 2014-11-04 10:18:27 -08:00
fjy 42cf20c9f1 some minor fixes in docs and change jetty defaults 2014-11-04 10:08:50 -08:00
Xavier Léauté f56d026d7d fix realtime module + cleaner metadata abstraction 
- better metadata abstractions
- more db->metadata renaming
 2014-11-03 16:40:23 -08:00
Xavier Léauté 3468163dcd remove unused imports 2014-10-31 16:25:52 -07:00
Xavier Léauté cad4d22f35 better derby test database 2014-10-31 16:25:36 -07:00
Xavier Léauté 80c72eb486 minor test adjustments 2014-10-31 15:44:09 -07:00
Xavier Léauté 1872b8f979 make it easier to test 2014-10-31 14:49:07 -07:00
Xavier Léauté 0bd9ac1154 default to Derby + remove DBI injection 
- default to Derby, but still fail if the given
  metadata storage type does not exit
- use SQLMetadataConnector injection instead of DBI injection
  to simplify code
 2014-10-31 14:37:37 -07:00
jisookim0513 6d501008b8 moved MetadataConnector test to metadata package 2014-10-31 13:13:19 -07:00
jisookim0513 a6016d70fc added deleted classes back 2014-10-31 13:10:09 -07:00
jisookim0513 01f2b3482c merging 2014-10-31 12:52:53 -07:00
jisookim0513 df7566701d fixed syntax error in creating index statements; added a test for MetadataConnector 2014-10-31 12:52:15 -07:00
Xavier Léauté cbe6ea67ee more druid.db -> druid.metadata.storage and allow multiple storage extensions loaded at once 2014-10-31 12:03:04 -07:00
Xavier Léauté 97a2f5af4a rename db->metadata 2014-10-31 10:54:33 -07:00
Xavier Léauté 9c06db021f rename db->metadata postgres->postgresql 2014-10-31 10:30:27 -07:00
Xavier Léauté f77b810f36 fix imports 2014-10-31 10:01:58 -07:00
Xavier Léauté 9fea34be4b simplify connectors even further 2014-10-31 09:59:34 -07:00
Xavier Léauté df0af894f4 fix factory binding + remove unused imports 2014-10-30 22:35:37 -07:00
Xavier Léauté fb4d41cedb make the injection gods happy 2014-10-30 21:16:36 -07:00
Xavier Léauté 377151beda better abstraction for metadatastorage 2014-10-30 18:23:35 -07:00
Xavier Léauté 3cc1b2e690 remove mysql dependencies + fix jdbc class loading 2014-10-29 17:30:43 -07:00
Xavier Léauté c2727dd063 better type 2014-10-29 14:22:00 -07:00
Xavier Léauté 1f171a2b86 workaround for Derby requiring batch statements 
+ make sure payloads are always written as bytes
 2014-10-29 13:59:37 -07:00
Xavier Léauté 55d5f1f618 remove commented out code 2014-10-29 12:55:03 -07:00
Xavier Léauté 7be252599e reduce memory usage when pulling segments 2014-10-29 12:43:21 -07:00
Xavier Léauté 68b9436199 optimize imports 2014-10-29 12:07:21 -07:00
Xavier Léauté e8b4640875 remove duplicate code in table creation 2014-10-29 11:47:29 -07:00
Xavier Léauté 123296b767 add documentation 2014-10-29 11:31:25 -07:00
Xavier Léauté 54eb434a0b remove Derby specific code in favor of better generic JDBI code 2014-10-29 01:09:05 -07:00
Xavier Léauté ec70ecc5fd make tests pass with Java 7 as well 2014-10-28 23:17:30 -07:00
Xavier Léauté 92afd10947 simplify SQL metadata storage 2014-10-28 18:12:30 -07:00
Xavier Léauté 575919ec6c provide a little more involved example 2014-10-27 19:18:09 -07:00
jisookim0513 aa754b86e8 build success! 2014-10-24 11:28:42 -07:00
fjy bef74104d9 merge with 0.7.x and resolve any conflicts 2014-10-23 17:24:06 -07:00
jisookim0513 02e79d6b15 attempted to solve merge-conflict; IncrementalIndex has unresolved classes after updates - needs to be fixed 2014-10-22 00:18:17 -07:00
jisookim0513 197878ae75 changed SQLMetadataConnector to a concrete class 2014-10-21 23:39:58 -07:00
jisookim0513 a0d9944fa6 changed SQLMetadataConnector to a concrete class 2014-10-21 23:38:26 -07:00
Fangjin Yang 3d8f33d4ad Merge pull request #801 from metamx/fix-unordered-merging 
fix ordering for partitions with same start time, fixes #796
 2014-10-21 16:47:31 -06:00
Xavier Léauté 8735272ce3 fix file casing 2014-10-21 15:36:49 -07:00
Xavier Léauté fd7d3e5ace make it work with Java 8 2014-10-21 15:33:28 -07:00
Xavier Léauté 9d5e65913d fix serde 2014-10-21 15:05:08 -07:00
jisookim0513 37979282fe enabled ansi-quote in mysql; insert statement should now work 2014-10-21 00:09:19 -07:00
Xavier Léauté 23a59db566 fix ordering for partitions with same start time, fixes #796 
Partitions with same start time may produce results out of order,
           especially in the presence of data gaps.

Results for those segments must be re-ordered prior to being merged
 2014-10-20 17:22:23 -07:00
Xavier Léauté 38cb73e0cc javacript tiered broker selector strategy 2014-10-20 14:12:30 -07:00
jisookim0513 7d5c5f2083 fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider 2014-10-17 00:10:36 -07:00
nishantmonu51 f4a97aebbc fix rollup for hashed partitions 
truncate timestamp while calculating the partitionNumber
 2014-10-15 22:32:56 +05:30
nishantmonu51 bce388fb27 merge changes from 0.7.x branch 2014-10-14 18:46:02 +05:30
nishantmonu51 b5d66381f3 more cleanup 2014-10-14 18:32:40 +05:30
jisookim0513 76304d6b7b made modifications according to comments; need to fix some more 2014-10-13 23:56:09 -07:00
nishantmonu51 035b4c1af4 more cleanup 2014-10-14 00:57:15 +05:30
nishantmonu51 454acd3f5a remove backwards compatible code 
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
 2014-10-13 19:30:44 +05:30
jisookim0513 0dda8975e7 deleted DerbyMetadataRule/SegmentManagerProvider and replaced them with SQL ones 2014-10-10 16:54:33 -07:00
jisookim0513 66277ddf2b deleted dbSegmentPublisher 2014-10-10 12:41:35 -07:00
Xavier Léauté 290107ce7b remove unused jsonspec content type 2014-10-09 10:26:47 -07:00
Xavier Léauté 30183d3b4d update druid-api for json parsing improvements 2014-10-07 13:25:02 -07:00
fjy d4217f1588 Merge pull request #777 from metamx/initialized-endpoint 
Historical endpoints accessible while loading + new loadstatus endpoint
 2014-10-06 17:35:58 -06:00
Xavier Léauté 81641d7887 http accessible while loading + status endpoint 2014-10-06 13:32:27 -07:00
fjy c7b4d5b7b4 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
 2014-10-02 18:12:10 -07:00
Gian Merlino efa898422d Merge pull request #773 from metamx/cleanup-metrics 
Use multi-val dims for metrics
 2014-10-02 17:58:50 -07:00
fjy 816986f914 cleanup according to cr 2014-10-02 17:51:33 -07:00
fjy 75fb7986df Merge pull request #602 from metamx/offheap-incremental-index 
Offheap incremental index
 2014-10-01 16:20:24 -06:00
fjy 4fc0382d8a add more logging 2014-10-01 09:13:59 -07:00
nishantmonu51 ad75a21040 separate offheapIncrementalIndex implementation 2014-10-01 13:58:51 +05:30
fjy c3bea245a7 fix up some bugs 2014-09-30 17:20:52 -07:00
fjy 5b166134fb add docs 2014-09-30 14:59:02 -07:00
fjy 6d2747adcf make loading local files easier 2014-09-30 14:50:48 -07:00
fjy ee713f8e94 Merge pull request #727 from metamx/new-console-0.7.x 
New console 0.7.x
 2014-09-30 15:36:32 -06:00
jisookim0513 0e50852985 fixed MetadataTaskStorage and handler 2014-09-30 14:09:23 -07:00
jisookim0513 9d7b5d9b0f fixed javadoc; fixed pom files; deleted unnecessary class 2014-09-30 13:47:35 -07:00
fjy 0f96a09939 use multi-val dims for metrics 2014-09-30 13:36:01 -07:00
nishantmonu51 91eecfe3e7 fix #730 - memory check throws exception if fails 
fix exception handling, do not catch ProvisionException
 2014-09-30 23:03:40 +05:30
nishantmonu51 358ff915bb fix merge conflicts 2014-09-30 22:19:18 +05:30
nishantmonu51 2789536bed merge changes from druid-0.7.x 2014-09-30 22:05:49 +05:30
nishantmonu51 61c7fd2e6e make ingestOffheap tuneable 2014-09-30 15:30:02 +05:30
nishantmonu51 8cecb00aa9 upgrade jetty and workaround for half open connection 
Upgrade jetty and set acceptor priority delta to -1
 to give priority to servicing connections already accepted rather than
opening new connections.
 2014-09-30 14:22:16 +05:30
fjy 1a67ebe861 fix regression in test 2014-09-29 15:22:26 -07:00
nishantmonu51 adb4a65e0a Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-29 12:38:31 +05:30
jisookim0513 74565c9371 cleaned up the code 2014-09-27 13:10:01 -07:00
jisookim0513 aa887edb73 added two seperate modules for mysql and postgres 2014-09-27 13:08:53 -07:00
fjy fb1d6c177a fix dynamic config bug 2014-09-26 15:45:20 -07:00
jisookim0513 91bf8fadcf added MetadataSegmentManagerTest 2014-09-26 15:38:40 -07:00
jisookim0513 59b3e79c70 derby seems to be working..? 2014-09-26 14:36:17 -07:00
jisookim0513 6a641621b2 finished merging into druid-0.7.x; derby not working (to be fixed) 2014-09-26 14:24:53 -07:00
jisookim0513 43cc6283d3 trying to revert files that have overwritten changes 2014-09-26 12:38:04 -07:00
fjy 4244accab5 Merge branch 'druid-0.7.x' of github.com:metamx/druid into druid-0.7.x 2014-09-26 11:39:50 -07:00
fjy eaf0a48b92 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-09-26 11:39:24 -07:00
jisookim0513 a5d02b9030 second attempt to fix merge-conflicts 2014-09-24 15:59:00 -07:00
jisookim0513 3bf39cc9f8 attempted to fix merge-conflicts 2014-09-24 15:55:42 -07:00
Xavier Léauté 77c9afe92f try to make javadoc happy 2014-09-23 18:53:29 -07:00
Xavier Léauté d02fd97ea3 fix comment 2014-09-23 16:25:46 -07:00
Xavier Léauté 8603d2fd3e keep default behavior 2014-09-23 16:23:02 -07:00
Xavier Léauté 35fb210cfa add test for parallel loading 2014-09-23 16:06:46 -07:00
Xavier Léauté 05d4f71ddc fix background annoucing race condition 2014-09-23 15:04:48 -07:00
Xavier Léauté 12449481e3 parallelize segment loading 2014-09-22 16:33:25 -07:00
nishantmonu51 f51ab84386 merge changes from druid-0.7.x 2014-09-22 23:48:45 +05:30
nishantmonu51 443e5788fb make OffheapIncrementalIndex tuneable 2014-09-22 19:26:10 +05:30
jisookim0513 273205f217 initial attempt for abstraction; druid cluster works with Derby as a default 2014-09-19 17:39:59 -07:00
nishantmonu51 8eb6466487 revert buffer size and add back rowFlushBoundary 2014-09-19 23:06:04 +05:30
Xavier Léauté 32b6135f3d background announce segment cache 2014-09-18 12:09:20 -07:00
Xavier Léauté d9a0d403b3 default zk compressed + batch announcements in 0.7 2014-09-16 14:54:45 -07:00
fjy 8b387034a3 Merge pull request #741 from metamx/serverview-initialization 
initialize segment view before announcing broker
 2014-09-16 15:35:09 -06:00
fjy 1dbb6fd4b0 Merge pull request #736 from metamx/fix-synchronized 
fix synchronized on field getting updated
 2014-09-16 11:06:35 -06:00
Xavier Léauté 762485a0c9 remove unnecessary transient 2014-09-16 09:59:25 -07:00
Xavier Léauté f1e3518f37 initialize segment view before announcing broker 2014-09-16 00:07:31 -07:00
xvrl a73670ee7b Merge pull request #738 from metamx/dim-ex 
Have better dimension exclusions
 2014-09-15 17:14:29 -07:00
fjy 77dbac3739 fix import 2014-09-15 17:10:34 -07:00
fjy f2d6360d4d cleanup 2014-09-15 17:08:45 -07:00
fjy 34ae426ffa address cr 2014-09-15 17:08:02 -07:00