Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
6,324 Commits 113 Branches 504 Tags 724 MiB
Commit Graph

1272 Commits

Author SHA1 Message Date
Xavier Léauté fa6142e217 cleanup and remove unused imports 2015-11-11 12:25:21 -08:00
dclim fd0935ecb9 fix spatial dimension transformer to work with hadoop 2015-11-10 19:16:51 -07:00
Slim Bouguerra c511273efd adding in filter 2015-11-06 16:23:24 -06:00
Charles Allen 929b981710 Change DefaultObjectMapper to NOT overwrite final fields unless explicitly asked to 2015-11-05 18:10:13 -08:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Gian Merlino 8defe29270 Merge pull request #1901 from guobingkun/fix_typo_and_rename 
Fix metadata typo and rename default extension directory
 2015-11-03 14:02:11 -08:00
Bingkun Guo 962f65cc76 fix metadata typo and rename default extension directory 2015-11-03 14:50:42 -06:00
Fangjin Yang cec09a9967 Merge pull request #1804 from himanshug/objectify_index_creators 
static to non-static conversion for methods in Index[Merger/Maker/IO]
 2015-11-03 11:25:32 -08:00
Himanshu Gupta 8b67417ac8 make methods in Index[Merger,Maker,IO] non-static so that they can have 
appropriate ObjectMapper injected instead of creating one statically
 2015-11-02 23:24:26 -06:00
navis.ryu e03fc2032f changed equals/hashCode implementation 2015-11-02 17:21:35 +09:00
navis.ryu 69c86716d6 addressed comments 2015-11-02 14:23:13 +09:00
navis.ryu 032c3e986d Make 'search' filter have a case sensitive option(#1878) 2015-10-30 16:38:54 +09:00
Fangjin Yang 25a0eb7ed5 Merge pull request #1799 from dclim/nested-groupby-aggregator-fix 
Support multiple outer aggregators of same type and provide more help…
 2015-10-29 18:01:31 -07:00
Xavier Léauté 59872bd0cd Merge pull request #1809 from metamx/fifoPriorityExecutorService 
Make PrioritizedExecutorService optionally FIFO
 2015-10-27 15:19:32 -07:00
Charles Allen 060402a216 Merge pull request #1855 from himanshug/fix_having_specs 
fix [GreaterThan,LessThan,Equals] HavingSpecs
 2015-10-27 14:46:04 -07:00
Charles Allen ecdafa87c5 Make PrioritizedExecutorService optionally FIFO 2015-10-27 14:16:22 -07:00
Himanshu Gupta a71c7270b9 making [GreaterThan,LessThan,Equals] HavingSpecs more robust by carefully using long vs float for comparison 2015-10-27 13:15:13 -05:00
Fangjin Yang 5a082b2f5e Merge pull request #1824 from metamx/UniformGranularitySpecHashEquals 
Add hashCode and equals to UniformGranularitySpec
 2015-10-26 09:34:01 -07:00
Fangjin Yang 5f23703216 Merge pull request #1638 from guobingkun/remove_maven_client_code 
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
 2015-10-26 09:30:05 -07:00
Nishant 7cecc55045 Add segment merge time as a metric 
Add merge and persist cpu time

Fix typo

review comment

move cpu time measuring to VMUtils

review comments.
 2015-10-22 12:28:03 +05:30
Bingkun Guo 4914925d65 New extension loading mechanism 
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
 2015-10-21 14:22:36 -05:00
Xavier Léauté e4ac78e43d bump next snapshot to 0.9.0 2015-10-20 13:46:13 -07:00
dclim 46ecdfa757 add comment explaining logic 2015-10-15 16:04:06 -06:00
Xavier Léauté 4c2c7a2c37 update version to 0.8.3 2015-10-14 21:40:55 -07:00
Charles Allen f432b8e3f9 Add hashCode and equals to UniformGranularitySpec 
* Also add hashCode != 0 to AllGranularity and NoneGranularity
 2015-10-13 16:42:21 -07:00
Gian Merlino c9d6994040 Merge pull request #1821 from himanshug/storage_adapter_update 
cache max data timestamp in QueryableIndexStorageAdapter
 2015-10-13 10:52:43 -07:00
Himanshu Gupta 490de1f98a support multiple non-consecutive intervals in outer query of nested group-by 2015-10-13 10:16:06 -05:00
Himanshu Gupta fbba30eb60 cache max data timestamp in QueryableIndexStorageAdapter so that TimestampCheckingOffset 
does not have to get it per cursor.
 2015-10-12 15:34:22 -05:00
Charles Allen 8ed5d2c06a Add hashCode and equals to stock lookups 2015-10-12 10:29:39 -07:00
Himanshu Gupta 2737fd83f5 in the IndexSizeExceededException put maxRowCount to confirm if it is correctly picked up from configuration 2015-10-06 15:23:14 -05:00
Himanshu Gupta 8654732ef6 make IndexSizeExceededException constructor take formatString and arguments than just fixed String 
like ISE, IAE etc
 2015-10-06 13:44:22 -05:00
dclim f4e0a76820 Support multiple outer aggregators of same type and provide more helpful exception when the same inner aggregator is referenced by multiple types of outer aggregators 2015-10-01 15:15:12 -06:00
Gian Merlino 774765dc40 GroupByQueryRunnerTest for hyperUnique finalizing post aggregators 2015-10-01 00:09:29 -04:00
Gian Merlino e3bb93e8c7 Revert "Merge pull request #1781 from dclim/nested-groupby-multiple-same-aggregator-fix-v2" 
This reverts commit dae488b7c0, reversing
changes made to 397be4b897.
 2015-10-01 00:05:59 -04:00
dclim 8e20a1e1f3 Use DoubleSumAggregatorFactory instead of CountAggregatorFactory, add test for non-integers 2015-09-30 17:11:39 -06:00
David Lim 70ae5ca922 Fix failure in nested groupBy with multiple aggregators with same fieldName 
Version 2 - Throws an exception if an outer query references an
aggregator that doesn't exist in the inner query, and then uses the
inner query aggregator names to form the columns for the intermediate
incremental index.

Also deleted all the getRequiredColumns() methods which are no longer
being used.

We do something wacky by adding an aggregator factory for the post
aggregators when building the intermediate incremental index, otherwise
queries on post aggregate results fail because the data isn't in the
incremental index.

Closes #1419
 2015-09-30 15:43:11 -06:00
Charles Allen 8199ecf1a4 Merge pull request #1782 from jon-wei/smq_cachekey 
Add analysisTypes to SegmentMetadataQuery cache key
 2015-09-29 15:51:35 -07:00
jon-wei 41ff271339 Add analysisTypes to SegmentMetadataQuery cache key 2015-09-29 14:33:35 -07:00
Charles Allen 2d847ad654 Merge pull request #1730 from metamx/union-queries-fix 
fix #1727 - Union bySegment queries fix
 2015-09-29 12:23:25 -07:00
Nishant 573aa96bd6 fix #1727 - Union bySegment queries fix 
Fixes #1727.
revert to doing merging for results for union queries on broker.

revert unrelated changes

Add test for union query runner

Add test

remove unused imports

fix imports

fix renamed file

fix test

update docs.
 2015-09-29 23:32:36 +05:30
Gian Merlino 62d4ced4dd Separate ListColumnIncluderator cache key parts with nul bytes 2015-09-29 13:59:58 -04:00
jon-wei e6a6284ebd Allow SegmentMetadataQuery to skip cardinality and size calculations 2015-09-22 13:51:55 -07:00
Gian Merlino aaa8a88464 Merge pull request #1739 from jon-wei/segment_realtime 
Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node
 2015-09-17 18:36:53 -07:00
Charles Allen df4c2bab10 Soften concurrency requirements on IncrementalIndexTest 2015-09-17 15:51:07 -07:00
jon-wei 367c50d4ba Allow SegmentAnalyzer to read columns from StorageAdapter, allow SegmentMetadataQuery to query IncrementalIndexSegments on realtime node 2015-09-16 18:39:31 -07:00
Charles Allen 6e1eb3b7fe Add better concurrency testing to IncrementalIndexTest 2015-09-16 14:04:20 -07:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta 
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
 2015-09-16 12:36:25 -07:00
Fangjin Yang 8b071a7230 Merge pull request #1710 from metamx/incrementalIndexConcurrentTestLatching 
Add some basic latching to concurrency testing in IncrementalIndexTest
 2015-09-15 13:55:52 -07:00
jon-wei 193fb4fdfc Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups 2015-09-14 19:41:42 -07:00
Charles Allen bd605a097e Merge pull request #1731 from metamx/regex-extraction-npe 
fix NPE with regex extraction function
 2015-09-14 15:55:05 -07:00
Xavier Léauté 08a527d01a fix NPE with regex extraction function 2015-09-14 14:45:30 -07:00
Charles Allen e569f4b6a7 Add dimension extraction functionality to SearchQuery 
* Add IdentityExtractionFn
 2015-09-14 11:36:15 -07:00
Himanshu 5ff92664f8 Merge pull request #1696 from metamx/cpuTimeReporting 
Add CPU time to metrics for segment scanning.
 2015-09-14 10:53:55 -05:00
Fangjin Yang 34ef81572d Merge pull request #1700 from himanshug/update_agg_test_helper 
update indexing in the helper to use multiple persists and merge
 2015-09-14 06:56:29 -07:00
Charles Allen 8d3cdd8572 Don't check for sortedness if we already know GenericIndexedWriter isn't sorted 2015-09-11 16:32:09 -07:00
Charles Allen d6849805ea Add some basic latching to concurrency testing in IncrementalIndexTest 2015-09-10 10:06:51 -07:00
Himanshu Gupta 5da58e48e0 use Rule based TemporaryFolder for cleanup of temp directory/files 2015-09-09 11:10:33 -05:00
Himanshu Gupta 44911039c5 update indexing in the helper to use multiple persists and final merge to 
catch further issues in aggregator implementations
 2015-09-09 11:10:33 -05:00
Charles Allen fcf5cae81d Add CPU time to metrics for segment scanning. 2015-09-08 13:34:19 -07:00
cheddar 4f61b42f40 Merge pull request #1578 from b-slim/fix_extraction_filter_2 
Fix UT and documentation to the extraction filter
 2015-09-01 10:46:20 -07:00
Himanshu 04ff6cd355 Merge pull request #1685 from gianm/close-loudly 
Close output streams and channels loudly when creating segments.
 2015-08-28 23:32:22 -05:00
Gian Merlino 940e1aa3eb Replace funky imports with standard ones. 
1) Lots of Guava imports were not coming from the actual Guava
2) junit.framework.Assert should be org.junit.Assert
 2015-08-28 18:02:05 -07:00
Gian Merlino 7d6fa2ba50 Close output streams and channels loudly when creating segments. 2015-08-28 17:14:03 -07:00
Himanshu Gupta 2e0dd1d792 adding UTs and addressing review comments to 
firehoseV2 addition to Realtime[Manager|Plumber],
essential segment metadata persist support,
kafka-simple-consumer-firehose extension patch
 2015-08-27 20:50:46 -05:00
lvjq 2237a8cf0f kafka 8 simple consumer firehose 2015-08-27 20:50:46 -05:00
Charles Allen c1388a1685 Merge pull request #1632 from Hailei/fix-subquery-innerquery-demension 
Inner Query  should build on sub query
 2015-08-27 10:25:38 -07:00
Gian Merlino 2a866f49df Downgrade Jackson to 2.4.6. 2015-08-26 18:25:55 -07:00
Charles Allen 24aa762c79 Add test for #1632 2015-08-25 20:50:30 -07:00
Xavier Léauté 51f6a9a2c9 update jackson to 2.6.1 2015-08-25 16:07:01 -07:00
Himanshu Gupta c57c07f28a add ability for client code to provide InputStream of input data in addition to File 
It would be needed when input data file does not reside in the same jar
but you could still use getResourceAsStream() to read the data inside a file
 2015-08-20 00:54:58 -05:00
Xavier Léauté 3b2e41e42a update for next release 2015-08-18 17:16:46 -07:00
Slim Bouguerra 7549f02578 support the case filter value is null 2015-08-17 15:09:37 -05:00
zhanghailei 234a958817 Inner Query should build on sub query 2015-08-17 18:18:26 +08:00
Charles Allen db19d2d547 Revert "Update to guice 4.0" 2015-08-14 09:26:07 -07:00
Charles Allen be89105621 Merge pull request #1602 from metamx/more-code-cleanup 
Some perf Improvements in Broker
 2015-08-11 13:51:49 -07:00
Xavier Léauté fbdb841928 Merge pull request #1603 from metamx/optimize-lexicographic-topN 
Optimizations for LexicographicTopNs
 2015-08-11 13:35:34 -07:00
Nishant b8d8a8da9e Optimisations for LexicographicTopNs 
initial review for perf optimizations for lexicographic TopNs

fix compilation

create map with proper size

review comment

review comment

review comments
 2015-08-12 00:37:48 +05:30
Charles Allen 7e61216287 Update to guice 4.0 
- Mark a lot of `@Provides` methods as final since guice 4.0 disallows overriding them
 2015-08-10 13:57:18 -07:00
Slim Bouguerra f0bc362981 clean code if is not needed anymore 2015-08-07 12:38:41 -05:00
Slim Bouguerra 64d638a386 optimize makeMatcher 2015-08-06 17:04:36 -05:00
Nishant 1a46c4c71c avoid creating mergeSeqence when not required 2015-08-06 14:25:13 +05:30
Slim Bouguerra 83de5a4716 addressing reviewers comments 2015-08-03 09:03:28 -05:00
Slim Bouguerra dda0790a60 Fix extractionFilter by implementing make matcher 
Fix getBitmapIndex to consider the case were dim is null
Unit Test for exractionFn with empty result and null_column
UT for TopN queries with Extraction filter
refactor in Extractiuon fileter makematcher for realtime segment and clean code in b/processing/src/test/java/io/druid/query/groupby/GroupByQueryRunnerTest.java
fix to make sure that empty string are converted to null
 2015-08-03 09:02:17 -05:00
Himanshu Gupta d11d9b6c45 dont waste memory in storing all lines from input 
CharSource.readLines() reads all lines from input into a in-memory list
Since we need an iterator here, so this wastage can be easily prevented
 2015-07-20 21:59:38 -05:00
Fangjin Yang 0481c8ca26 Merge pull request #1406 from zhaown/fix-breaking-while-exceeding-max-intermediate-rows 
Fix breaking while exceeding max intermediate rows.
 2015-07-20 13:41:22 -07:00
Himanshu Gupta f7a92db332 generic byte[] serde for InputRow 2015-07-20 12:01:53 -05:00
Himanshu Gupta 0439e8ec23 adding serde methods for intermediate aggregation object to ComplexMetricSerde 
This provides the alternative to using ComplexMetricSerde.getObjectStrategy()
and using the serde methods from ObjectStrategy as that usage pattern is deprecated.
 2015-07-20 12:01:53 -05:00
zhaown 524b05f073 Fix breaking while exceeding max intermediate rows. 2015-07-19 10:41:53 +08:00
Fangjin Yang e21195f987 Merge pull request #1469 from guobingkun/table_config 
Inconsistent property names for "druid.metadata.storage.tables.xxx"
 2015-07-17 07:43:19 -07:00
Himanshu 19af3bc9bc Merge pull request #1535 from metamx/alphanum-docs-tests 
Update alphanumeric sort docs + more tests / examples
 2015-07-16 22:09:41 -05:00
Xavier Léauté 2c464ad936 correct reference in docs + more tests / examples 2015-07-16 19:50:05 -07:00
Xavier Léauté 9616c10b1d remove import static 2015-07-16 17:46:21 -07:00
Xavier Léauté c1308203b8 Merge pull request #1532 from metamx/fixTopNDimExtractionDoubleApply 
Fix TopN dimension extractions being applied twice
 2015-07-16 13:39:02 -07:00
Xavier Léauté 3a0793aaf9 Merge pull request #1533 from metamx/extraCheckGroupByDimExtraction 
Add more unit tests for group by
 2015-07-15 21:09:00 -07:00
Charles Allen 7d0b77c261 Add more unit tests for group by 2015-07-15 20:15:21 -07:00
Xavier Léauté a15a2c4047 fix histogram aggregator cache key 2015-07-15 17:33:36 -07:00
Charles Allen 9092c665b7 Fix TopN dimension extractions being applied twice 2015-07-15 16:58:15 -07:00
Charles Allen 456ad9ffba Merge pull request #1529 from metamx/update-versions 
inrement version
 2015-07-15 13:25:31 -07:00
Xavier Léauté 4cfb00bc8a inrement version 2015-07-15 13:09:05 -07:00
Charles Allen 5eadd395e2 Move lots of executor service creation to Execs 2015-07-14 15:38:49 -07:00