Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,266 Commits 113 Branches 506 Tags 732 MiB
Commit Graph

1074 Commits

Author SHA1 Message Date
Xavier Léauté e01ed16030 serde tests + equals/hashCode fixes for extraction functions 2015-03-11 16:48:28 -07:00
Xavier Léauté d3f5bddc5c Add ability to apply extraction functions to the time dimension 
- Moves DimExtractionFn under a more generic ExtractionFn interface to
  support extracting dimension values other than strings
- pushes down extractionFn to the storage adapter from query engine
- 'dimExtractionFn' parameter has been deprecated in favor of 'extractionFn'
- adds a TimeFormatExtractionFn, allowing to project the '__time' dimension
- JavascriptDimExtractionFn renamed to JavascriptExtractionFn, adding
  support for any dimension value types that map directly to Javascript
- update documentation for time column extraction and related changes
 2015-03-11 16:45:42 -07:00
Himanshu Gupta 55ebf0cfdf force eager the processing of segment metadata query on the processing threadpool by using ChainedExecutionQueryRunner in SegmentMetadataQueryRunnerFactory.mergeRunners(..) 2015-03-11 12:58:58 -05:00
Xavier Léauté 217e674063 Handling aggregators and post aggregators with duplicate names 
* add test for same-name groupBy hyperUniques post-agg
* add test for same-name post-agg in groupby with approx histogram
* Fixes https://github.com/druid-io/druid/issues/1045
* Throws an error if post aggs and aggs do not have unique names
* Add more groupBy tests for Having filters
 2015-03-10 17:10:43 -07:00
Fangjin Yang 0b467624ec Merge pull request #694 from druid-io/arithmetic-op-strategies 
normal division & configurable ordering for ArithmeticPostAggregator
 2015-03-10 13:48:27 -07:00
Fangjin Yang 2abdce1dc0 Merge pull request #1180 from metamx/logging-groupBy-NPE 
add null check early to catch root cause for groupBy NPE while running bySegment query
 2015-03-09 09:16:33 -07:00
nishantmonu51 6e935cca0a add null check early to catch root cause 2015-03-09 21:10:28 +05:30
Xavier Léauté 0d47c0c36d normal division and configurable ordering for ArithmeticPostAggregator 
Fixes #510
 2015-03-04 12:44:24 -08:00
Fangjin Yang d685e2ab04 Merge pull request #1165 from friedhardware/fix-NPerror-select 
Added null check for the pagingSpec on a Select Query.
 2015-03-02 14:17:06 -08:00
Fangjin Yang e8605c63a9 Merge pull request #1150 from himanshug/broker-parallel-chunk-process 
interval chunk query runner now processes individual chunk in a threadpool
 2015-03-02 13:50:23 -08:00
Himanshu Gupta 29039fd541 interval chunk query runner now processes individual chunk in a thread pool and prints metrics query/time per chunk 2015-03-02 15:45:09 -06:00
Joshua Schumacher e6130e0fdc Added null check for the pagingSpec on a Select Query. 2015-03-02 12:41:59 -08:00
Fangjin Yang 005f4da2c0 Merge pull request #1143 from metamx/update-rhino-1.7rc5 
Update Rhino to 1.7RC5
 2015-02-25 12:50:23 -08:00
Xavier Léauté b167dcf82c [maven-release-plugin] prepare for next development iteration 2015-02-23 14:28:06 -08:00
Xavier Léauté e81ac2ba43 [maven-release-plugin] prepare release druid-0.7.0 2015-02-23 14:27:58 -08:00
James Estes 562de6c621 Update docs and examples for log4j2 usage. 
- Put configs early in classpath in examples so log4j2.xml will get picked up properly
- Add an example log4j2.xml file.
- Update Logging doc.
 2015-02-19 11:40:56 -07:00
Xavier Léauté c4d721fffd update Rhino to 1.7RC5 2015-02-19 09:48:18 -08:00
Xavier Léauté 78df7f6165 Move Druid release artifacts to Sonatype 
- Switch to using Druid parent POM
- Add required fields for Sonatype
- Common plugin versions and settings have been moved to the parent pom
- Cleanup artifacts and POMs for consistent formatting
- Remove org.hyperic.sigar dependency and update docs to reflect necessary jars to add at runtime when sigar is needed
 2015-02-13 14:26:31 -08:00
fjy d29740ed9f [maven-release-plugin] prepare for next development iteration 2015-02-12 16:16:00 -08:00
fjy 211fd15b7e [maven-release-plugin] prepare release druid-0.7.0-rc3 2015-02-12 16:15:56 -08:00
Fangjin Yang 90bc62eb5c Merge pull request #1108 from metamx/improve-groupby-perf 
Improve groupby by removing conversion to case insensitive row
 2015-02-12 11:45:20 -08:00
nishantmonu51 15cf432b74 remove conversion to case insensitive row 
this is not required after death to casing in 0.7
 2015-02-11 19:40:36 +05:30
Xavier Léauté c5e99bf6ec Merge pull request #1105 from metamx/fixEmptyExtractionFilter 
Fix empty results on ExtractionFilter.
 2015-02-10 14:25:58 -08:00
Charles Allen b9cb311a52 Fix empty results on ExtractionFilter. 
* Now returns empty results rather than erroring out
* Added unit tests for multiples case
 2015-02-10 14:04:38 -08:00
fjy 708759e1e0 Update http-client to 1.0.0 2015-02-10 13:36:47 -08:00
Xavier Léauté a7dcaffb53 fix `__time` column selector for incremental index 
- also adds tests for selecting the time column
 2015-02-06 12:06:05 -08:00
Fangjin Yang 42e902b6e3 Merge pull request #1090 from metamx/alphanum-attribution 
update code attribution
 2015-02-04 15:51:34 -08:00
Xavier Léauté 0fbc6071c9 update code attribution 2015-02-04 15:28:44 -08:00
Fangjin Yang 25cf15824b Merge pull request #1085 from gianm/dsmrv-fix 
DataSourceMetadataResultValue fixes and JodaUtils adjustments.
 2015-02-03 17:51:33 -08:00
Gian Merlino 085ad8d345 Fix DataSourceMetadataResultValue serde. 2015-02-03 17:39:42 -08:00
fjy 1f12c5b2f1 [maven-release-plugin] prepare for next development iteration 2015-02-03 12:06:49 -08:00
fjy e82d431be7 [maven-release-plugin] prepare release druid-0.7.0-rc2 2015-02-03 12:06:41 -08:00
Xavier Léauté 4eff269536 Merge pull request #1079 from druid-io/cleanup-deps 
Remove non friendly dependencies from Druid
 2015-02-03 11:56:41 -08:00
fjy 3e5d338c8e Remove non friendly dependencies from Druid 2015-02-03 11:36:08 -08:00
Fangjin Yang 71b4c5fa86 Merge pull request #1076 from metamx/remove-threadlocals 
remove thread-locals in GenericIndexed in favor of wrapped objects
 2015-02-02 20:02:33 -08:00
Xavier Léauté cb2e300eba remove thread-locals in GenericIndexed in favor of wrapped objects to reduce GC pressure 2015-02-02 15:59:30 -08:00
Eric Tschetter 42eba986ce Towards consistent null handling 
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
 2015-02-02 12:53:07 -08:00
Fangjin Yang 92e616de11 Merge pull request #1077 from metamx/remove-unused-imports 
remove unused imports
 2015-02-02 10:45:27 -08:00
nishantmonu51 ba932bb1f2 remove unused imports 2015-02-02 21:53:39 +05:30
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
Xavier Léauté f24a89a22a fix NPE for topN over missing hyperUniques column 2015-01-27 16:12:41 -08:00
Charles Allen 226dd91a31 Add a hash map for storing groupBy partition index 
* Improves groupBy performance by approx 15%
 2015-01-26 08:42:02 -08:00
fjy 1f94de22c6 [maven-release-plugin] prepare for next development iteration 2015-01-20 14:23:55 -08:00
fjy 17476edc31 [maven-release-plugin] prepare release druid-0.7.0-rc1 2015-01-20 14:23:51 -08:00
Charles Allen 3d27747f7e Upgrade to log4j2 
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
 2015-01-20 12:35:18 -08:00
Fangjin Yang 91a79dbf95 Merge pull request #1031 from metamx/ingestmetadata-query 
DataSourceMetadata query
 2015-01-19 21:55:35 -08:00
Charles Allen 7bb038756c Account for very slow writer threads in IncrementalIndexTest 2015-01-17 13:02:59 -08:00
Fangjin Yang b4041c13e5 Merge pull request #1029 from metamx/fixChainedExecutionQueryRunnerTest 
Address spurious test failures
 2015-01-16 13:08:32 -08:00
Xavier Léauté 3b3aad78cb Merge pull request #1027 from metamx/concurrentOnHeapIncrementalIndexFix 
Fix concurrency issues in OnheapIncrementalIndex
 2015-01-16 12:54:42 -08:00
Charles Allen 197af967ef Fix concurrency issues in OnheapIncrementalIndex 
* Was encountering weird errors when fast writes were coming in while queries were happening.
* Added unit tests which tend to cause concurrency query problems
 2015-01-16 12:01:46 -08:00
Charles Allen ebafa2a786 Fix spurious test failures in ChainedExecutionQueryRunnerTest 2015-01-15 16:49:16 -08:00
Fangjin Yang 5bfcc43377 Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate 
Update all String conversions to and from byte[] to use the java-util StringUtils functions
 2015-01-15 13:50:27 -08:00
nishantmonu51 c7452b75f6 Merge branch 'master' into ingestmetadata-query 2015-01-15 18:00:31 +05:30
Xavier Léauté d5f4182de4 global test timeouts + fix test race condition 2015-01-07 23:36:57 -08:00
Fangjin Yang 852e863425 Merge pull request #981 from druid-io/strictModuleTyping 
Use Module instead of generic Object in Guice related items
 2015-01-05 12:43:20 -08:00
Charles Allen b1b5c9099e Update all String conversions to and from byte[] to use the java-util StringUtils functions 
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
 2015-01-05 11:22:32 -08:00
Xavier Léauté 3fc6cf918d add test for large chunks 2015-01-02 14:31:22 -08:00
Xavier Léauté f2f9cbeca8 throw error rather than returning garbage results 2015-01-02 14:29:21 -08:00
Xavier Léauté 071943a367 fix LZF compression with buffers exceeding LZF chunk size 2015-01-02 11:39:50 -08:00
Xavier Léauté f2439899e7 fix bitmap factory serde 2014-12-23 15:07:32 -08:00
Xavier Léauté 27a3169312 increase test timeouts 2014-12-19 17:09:43 -08:00
Charles Allen 971afab36f Lengthen CompressionStrategyTest::testKnownSizeConcurrency() to have 2m timeout on its test to account for shared Jenkins build lag 2014-12-19 12:53:20 -08:00
Charles Allen 7c8d4a7433 Use Module instead of generic Object in Guice related items 2014-12-19 10:54:06 -08:00
Fangjin Yang be507b8cb4 Merge pull request #943 from mrijke/partialdimextractfn-nullpointer 
Fix NullPointerException in PartialDimExtractionFn
 2014-12-16 12:29:27 -07:00
nishantmonu51 80e4b68ee7 review comments 2014-12-16 21:16:48 +05:30
Fangjin Yang b3fe91bb50 Merge pull request #830 from metamx/union-merge-on-historical 
Union merge on historical
 2014-12-15 13:36:47 -07:00
fjy 3cb7999eb9 i hate hadoop dependencies 2014-12-15 09:52:46 -08:00
nishantmonu51 a0d3579a92 add docs + fix tests 2014-12-11 17:58:01 +05:30
nishantmonu51 7ad03087c0 Merge branch 'master' into ingestmetadata-query 2014-12-11 16:54:38 +05:30
nishantmonu51 32b4f55b8a review comments refactoring 2014-12-11 16:33:14 +05:30
nishantmonu51 3763357f6e Ingest metadata query implementation 2014-12-10 19:44:00 +05:30
Fangjin Yang d6d3ec6846 Merge pull request #948 from metamx/ingestion-docs 
Redocumenting ingestion
 2014-12-09 15:30:03 -07:00
fjy 9596c11f42 address cr 2014-12-09 14:19:18 -08:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
xvrl 1392e2731f Merge pull request #936 from metamx/cachingRunnerImprovements 
General Caching Query Runners cleanup (40% query time reduction for HLL)
 2014-12-08 14:07:52 -08:00
Charles Allen 7b65f0635d General Caching Query Runners cleanup 
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
 2014-12-08 13:29:32 -08:00
Maarten Rijke 90670a9c7e Fix NullPointerException in PartialDimExtractionFn by explicity checking for dimValue == null, attempt 2 2014-12-08 22:26:35 +01:00
Maarten Rijke bd9bbf396c Fix NullPointerException in PartialDimExtractionFn by explicity checking for dimValue == null 2014-12-08 20:11:58 +01:00
Xavier Léauté ad23e49777 use fixed-size mapdb cache to avoid heap growing uncontrollably 2014-12-05 15:34:50 -08:00
Xavier Léauté 7cd45a6e1f IncrementalIndex throws exception if limit exceeded 
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
 2014-12-04 14:38:16 -08:00
Xavier Léauté c7dbe6116c write byte data as is in smile 2014-12-04 10:57:56 -08:00
Xavier Léauté c21a82a697 upgrade LZ4 to operate directly on ByteBuffers 2014-12-04 10:57:56 -08:00
Xavier Léauté 0c521e0a77 update joda-time and fix min/max instant 2014-12-04 10:57:56 -08:00
nishantmonu51 269a51964e fix size calculation 2014-12-04 17:22:24 +05:30
nishantmonu51 4dc0fdba8a consider mapped size in limit calculation & review comments 2014-12-03 23:47:30 +05:30
Charles Allen 529e7e0272 Merge pull request #927 from metamx/speedup-smile-bytes 
Improve Smile serde performance by writing binary data as is
 2014-12-03 10:02:08 -08:00
Charles Allen 0f5d5840da Merge pull request #924 from metamx/update-joda-time 
Update Joda-Time and fix min/max instant overflow
 2014-12-03 09:15:39 -08:00
nishantmonu51 da8bd7836b Introduce buffer size 2014-12-03 16:28:22 +05:30
Xavier Léauté 5fece517fa write byte data as is in smile 2014-12-03 00:01:01 -08:00
Xavier Léauté 18f50097a9 upgrade LZ4 to operate directly on ByteBuffers 2014-12-02 23:53:56 -08:00
fjy bc173d14fc a whole bunch of cleanup and fixes 2014-12-02 17:32:05 -08:00
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
nishantmonu51 b65933ffb8 make tests parameterised 2014-12-02 23:55:29 +05:30
nishantmonu51 6dc69c2f30 code cleanups & formatting 2014-12-02 22:44:33 +05:30
nishantmonu51 eac776f1a7 tests passing with on heap incremental index 2014-12-02 22:29:28 +05:30
Xavier Léauté 4eee7e69b9 fix cardinality aggregator caching 2014-11-26 15:00:37 -08:00
xvrl 5bc1be5ba0 Merge pull request #850 from metamx/druid-0.7.x-compressionstrategy 
Compression strategy changes
 2014-11-25 12:58:39 -08:00
Charles Allen c6043afa32 Removed empty function from CompressionStrategyTest 2014-11-25 12:57:06 -08:00
Charles Allen 6943db5251 Changed branching logic for LZFCompressor to return null only on error, and avoid checking in most circumstances 2014-11-25 12:53:11 -08:00
Charles Allen 9f945c2216 Removed lz4Fast from CompressedObjectStrategy for compression since it is not currently used 2014-11-24 16:11:03 -08:00