Commit Graph

7140 Commits

Author SHA1 Message Date
David Lim 7641f2628f add control and status endpoints to KafkaIndexTask (#2730) 2016-04-21 15:34:59 -07:00
Slim 984a518c9f Merge pull request #2734 from b-slim/LookupIntrospection2
[QTL][Lookup] adding introspection endpoint
2016-04-21 12:15:57 -05:00
Gian Merlino c74391e54c JavaScript: Ability to disable. (#2853)
Fixes #2852.
2016-04-21 09:43:15 -05:00
Xavier Léauté 5938d9085b Stream segments from database (#2859)
* Avoids fetching all segment records into heap by JDBC driver
* Set connection to read-only to help database optimize queries
* Update JDBC drivers (MySQL has fixes for streaming results)
2016-04-21 05:40:07 +08:00
Gian Merlino 7d3e55717d Reduce cost of various toFilter calls. (#2860)
These happen once per segment and so it's better if they don't do
as much work.
2016-04-21 04:28:46 +08:00
Xavier Léauté fc91120b54 Merge pull request #2857 from metamx/upgrade-zk
upgrade zookeeper client dependency to 3.4.8
2016-04-20 10:36:07 +05:30
Gian Merlino 59460b17cc Add Filters.matchPredicate helper, use it where appropriate. (#2851)
This approach simplifies code and is generally faster, due to skipping
unnecessary dictionary lookups (see #2850).
2016-04-19 15:54:32 -07:00
Xavier Léauté b2745befb7 remove obsolete comment (#2858) 2016-04-19 13:06:58 -07:00
Nishant f80a5dc4ef Avoid creating multiple NoneShardSpec objects (#2855)
* Avoid creating multiple NoneShardSpec objects
* deprecate NoneShardSpec constructor
2016-04-19 10:30:14 -07:00
Nishant dbf63f738f Add ability to filter segments for specific dataSources on broker without creating tiers (#2848)
* Add back FilteredServerView removed in a32906c7fd to reduce memory usage using watched tiers.

* Add functionality to specify "druid.broker.segment.watchedDataSources"
2016-04-19 10:10:06 -07:00
Gian Merlino 08c784fbf6 KafkaIndexTask: Use a separate sequence per Kafka partition in order to make (#2844)
segment creation deterministic.

This means that each segment will contain data from just one Kafka
partition. So, users will probably not want to have a super high number
of Kafka partitions...

Fixes #2703.
2016-04-18 22:29:52 -07:00
Jisoo Kim 7b65ca7889 refactor ClientQuerySegmentWalker (#2837)
* refactor ClientQuerySegmentWalker

* add header to FluentQueryRunnerBuilder

* refactor QueryRunnerTestHelper
2016-04-18 14:00:47 -07:00
Xavier Léauté 838768c632 upgrade curator, fixes #2829 (#2849) 2016-04-18 13:17:36 -07:00
Gian Merlino 7c0b1dde3a DimensionPredicateFilter: Skip unnecessary dictionary lookup. (#2850) 2016-04-18 12:38:25 -07:00
Gaurav Kumar f5822faca3 Fixed wrong parseSpec in Avro Hadoop Parser (#2846)
`parseSpec` should contain `format` instead of `type`. It was wrongly defaulting to `tsv`
2016-04-16 11:34:54 -07:00
binlijin c1e690288c Improve some log (#2807) 2016-04-15 09:34:26 -07:00
Nishant 632b21472b fix test failure (#2818)
formatting changes
2016-04-14 21:40:19 -07:00
Jonathan Wei b534f7203c Fix performance regression from #2753 in IndexMerger (#2841) 2016-04-14 21:39:41 -07:00
Jonathan Wei a26134575b Fix NPE in TopNLexicographicResultBuilder.addEntry() (#2835) 2016-04-13 17:27:16 -07:00
du00cs 639d0630b8 jackson conflict workaround in hadooop ingestio & parquet extension coordinate update (#2817) 2016-04-13 14:20:33 -07:00
Fangjin Yang 0c4a42bb6f change toc entry (#2834) 2016-04-13 13:45:07 -07:00
Gian Merlino e320d13385 Fix various broken links in the docs. (#2833) 2016-04-13 13:30:01 -07:00
Gian Merlino 725ee1401d Update tranquility version in the docs. (#2832) 2016-04-13 11:33:59 -07:00
Gian Merlino aa25cc1f68 Fix up Kafka tutorial (#2831)
1) Remove extraneous section
2) Remove -SNAPSHOT version
2016-04-13 11:33:45 -07:00
Fangjin Yang 22c2cd57fb update readme (#2830) 2016-04-13 11:33:31 -07:00
Fangjin Yang abd951df1a Document how to use roaring bitmaps (#2824)
* Document how to use roaring bitmaps

This fixes #2408.
While not all indexSpec properties are explained, it does explain how roaring bitmaps can be turned on.

* fix

* fix

* fix

* fix
2016-04-12 19:28:02 -07:00
michaelschiff db35dd7508 fix issue #2744. Check for null before combining metrics (#2774) 2016-04-12 14:46:31 -07:00
Charles Allen ed5377465a add AirBnB Caravel to list of libraries (#2719) 2016-04-12 12:53:50 -07:00
Sébastien Launay 37d2ab623e Merge pull request #2815 from slaunay/documentation/hadoop-classpath-issue-fix-with-configuration
Doc for mapreduce.job.user.classpath.first=true
2016-04-12 10:51:51 -07:00
Fangjin Yang bf4aa965fb Merge pull request #2777 from metamx/postgresql-upsert
support PostgreSQL >= 9.5 upsert capability
2016-04-12 10:24:08 -07:00
Fangjin Yang 886ee4e30d Merge pull request #2821 from metamx/review-comments-2784
handle review comments for PR 2784
2016-04-12 10:20:43 -07:00
Fangjin Yang b486eff6b7 Merge pull request #2805 from metamx/query-time-start
request log should reflect time the query was received
2016-04-12 09:44:42 -07:00
Nishant deb6ecf919 handle review comments for PR 2784
https://github.com/druid-io/druid/pull/2784#discussion_r59062021
2016-04-12 21:52:00 +05:30
Nishant 1bf1dd03a0 Merge pull request #2812 from mrijke/fix-missing-equals-hashcode-filters
Add missing equals/hashcode to JS, Regex and SearchQuery DimFilters
2016-04-12 12:00:23 +05:30
Fangjin Yang bd6bd34cd8 Merge pull request #2090 from himanshug/math_exp
math expression support
2016-04-11 21:36:17 -07:00
Navis Ryu 49ef4d96cb Merge pull request #2802 from navis/optimize_multiplepath_concat
Optimize adding lots of paths to pathspec
2016-04-11 23:35:28 -05:00
Charles Allen 21e406613c Merge pull request #2809 from metamx/fix2694
Fix test for snapshot taker to better check for lookup perist failure
2016-04-11 14:52:47 -07:00
Nishant 9440598685 fix integration-test (#2814) 2016-04-11 15:46:32 -05:00
Maarten Rijke de68d6b7c4 Add missing equals/hashcode to JS, Regex and SearchQuery DimFilters
This commits adds missing equals() and hashcode() methods to
 the JavascriptDimFilter, RegexDimFilter and the SearchQueryDimFilter.
2016-04-11 12:16:24 +02:00
Fangjin Yang 234125878a Merge pull request #2808 from metamx/moveLookupSaveStateConfigDocs
Move lookup config doc to proper location
2016-04-08 13:50:42 -06:00
Himanshu Gupta aa6a230c90 remove DruidSQL.g4, its failing with newer version of ANTLR, will bring it back and fix if needed later 2016-04-08 11:46:21 -05:00
Himanshu Gupta 308211cc18 math expression language with parser/lexer generated using ANTLR 2016-04-08 11:40:29 -05:00
Himanshu Gupta 36ccfbd20e math expression language with hand written parser/lexer 2016-04-08 11:40:29 -05:00
Charles Allen 2b99f717e4 Move lookup config doc to proper location 2016-04-08 08:15:38 -07:00
Xavier Léauté d4d1d615c1 request log should reflect time the query was received, as opposed to processed 2016-04-07 12:39:34 -07:00
Slim d5787dd3cd Merge pull request #2784 from metamx/lite-segment-announcements
Add ability to skip loadSpec,dimensions, metrics from DataSegment Announcements
2016-04-07 09:51:22 -05:00
Nishant edd74f2b67 Allow Lite DataSegment Announcements
separate config for each skipping dimensions, metrics and loadSpec

Add test

fix test comment

Add docs
2016-04-07 18:24:12 +05:30
Nishant bbb326decf Merge pull request #2799 from b-slim/fix_snapshot
MapLookupFactory need to be Ser/Desr ready.
2016-04-07 13:22:34 +05:30
Slim Bouguerra bf1eafc4e1 remove all the mock lookupFactory 2016-04-06 15:37:52 -05:00
Slim Bouguerra 59eb2490a0 MapLookupFactory need to be Ser/Desr. 2016-04-06 15:02:18 -05:00