Commit Graph

6956 Commits

Author SHA1 Message Date
Slim cf342d8d3c Merge pull request #2517 from b-slim/adding_lookup_snapshot_utility
[QTL][Lookup] lookup module with the snapshot utility
2016-03-17 11:39:47 -05:00
Slim Bouguerra 0c86b29ef0 lookup module with the snapshot utility 2016-03-17 09:20:41 -05:00
Nishant 60fc5d9ad5 Merge pull request #2672 from himanshug/upload_mysql
update upload.sh to upload mysql-metadata-storage tarball too
2016-03-16 14:25:03 -07:00
Himanshu Gupta e0a648e29a update upload.sh to upload mysql-metadata-storage tarball too 2016-03-16 15:29:58 -05:00
Nishant 011efbfca7 Merge pull request #2663 from pjain1/warn_predicate_firehose
do not silently ignore rows
2016-03-16 11:07:26 -07:00
Bingkun Guo 89110ffd3d Merge pull request #2671 from rasahner/IT_searchCount
add count field to expected returns for search queries
2016-03-16 10:37:46 -05:00
Robin 1205c572bb add count field to expected returns for search queries 2016-03-16 09:45:38 -05:00
Parag Jain 948b19a088 do not silently ingnore rows 2016-03-16 09:30:19 -05:00
Fangjin Yang 8cea85816d Merge pull request #2668 from navis/fix-document-selectquery
Document for search query was not updated properly (Fix for #2662)
2016-03-15 20:34:27 -07:00
Fangjin Yang 8c481e496b Merge pull request #2666 from dclim/caching-doc-fix
caching doc fix
2016-03-15 18:07:38 -07:00
navis.ryu 71ee9e2aac Document for search query is not updated properly (Fix for #2662) 2016-03-16 09:22:26 +09:00
dclim 553b677971 caching doc fix 2016-03-15 17:09:33 -06:00
Fangjin Yang b2b62a723f Merge pull request #2657 from gianm/ports-docs
Additional ports docs.
2016-03-15 13:23:07 -07:00
Charles Allen 32399bc711 Merge pull request #2588 from metamx/abstract-zkworker
Use ImmutableWorkerInfo instead of ZKWorker
2016-03-15 10:27:21 -07:00
Fangjin Yang ec949d76e3 Merge pull request #2655 from navis/hint-coordinator-client
Add hint message for missing `druid.selectors.coordinator.serviceName`
2016-03-14 20:57:40 -07:00
Himanshu ea3281ad78 Merge pull request #2645 from atomx/gs-scheme
Add gs:// hdfs support
2016-03-14 22:15:42 -05:00
Gian Merlino a938f0853b Additional ports docs. 2016-03-14 19:11:18 -07:00
Charles Allen 2ac8a22173 Merge pull request #2579 from metamx/closerIsCloser
Make CloserRule use guava's Closer
2016-03-14 17:18:19 -07:00
Jonathan Wei 5ec5ac92c6 Merge pull request #2382 from himanshug/broker_segment_tier_selection
at broker, if configured, only add segments from specific tiers to the timeline
2016-03-14 16:53:06 -07:00
navis.ryu 83e1d5d7bf Add hint message for missing `druid.selectors.coordinator.serviceName` 2016-03-15 08:39:07 +09:00
Charles Allen a64979463f Make CloserRule use guava's Closer 2016-03-14 15:01:24 -07:00
Fangjin Yang 06813b510a Merge pull request #2571 from himanshug/gp_by_avoid_sort
avoid sort while doing groupBy merging when possible
2016-03-14 14:46:51 -07:00
Fangjin Yang a41a70d370 Merge pull request #2651 from gianm/ports-docs
Docs on default ports.
2016-03-14 14:15:52 -07:00
Charles Allen 02805a74a1 Merge pull request #2648 from chtefi/master
Ignore case when testing for table existence
2016-03-14 13:57:53 -07:00
Fangjin Yang dbdbacaa18 Merge pull request #2260 from navis/cardinality-for-searchquery
Support cardinality for search query
2016-03-14 13:24:40 -07:00
Gian Merlino e51277b96c Docs on default ports. 2016-03-14 11:25:21 -07:00
Nishant 773d6fe86c Merge pull request #2646 from atomx/update-maxmind
Update com.maxmind.geoip2 to 2.6.0
2016-03-14 11:20:48 -07:00
Nishant 9cceff2274 Use ImmutableWorkerInfo instead of ZKWorker
review comments

add test for equals and hashcode
2016-03-14 11:17:15 -07:00
Himanshu d51a0a0cf4 Merge pull request #2220 from gianm/appenderator-kafka
Appenderators, DataSource metadata, KafkaIndexTask
2016-03-14 13:14:36 -05:00
rasahner 2861e854f0 Merge pull request #2540 from pjain1/remove_kill
Remove extra parameter from deleteDataSourceSpecificInterval endpoint and correct exception message for invalid interval
2016-03-14 11:16:23 -05:00
Slim 8cc3582e70 Merge pull request #2644 from metamx/optimize-timeboundary
optimize timeboundary for min or max bound
2016-03-13 13:16:24 -05:00
Stéphane Derosiaux 416cb03687 Ignore case when testing for table existence 2016-03-13 11:17:30 +01:00
Erik Dubbelboer 21b7572533 Update com.maxmind.geoip2 to 2.6.0
com.maxmind.geoip2 2.6.0 depends on com.google.http-client 1.15.0-rc (3 years old).
When trying to include other libraries in Druid that require an up to date version of com.google.http-client this causes a problem.
2016-03-12 09:44:00 +00:00
Erik Dubbelboer 375620cfb3 Add gs:// hdfs support
Used to access google cloud storage
2016-03-12 08:57:57 +00:00
navis.ryu be341bf4e3 Support cardinality for search query (Fix for #2260) 2016-03-12 09:51:01 +09:00
Nishant cf7f6da392 Merge pull request #2634 from gianm/stopGracefully-avoid-interrupt
ThreadPoolTaskRunner: Make graceful shutdown logs less scary.
2016-03-11 16:36:10 -08:00
Charles Allen a3f0048ea4 Merge pull request #2631 from gianm/plumbers-rpe
Better logging for ParseExceptions on index aggregation, and remove unnecessary exception handling.
2016-03-11 14:22:58 -08:00
Xavier Léauté 6f0d6ef0e9 optimize timeboundary for min or max bound 2016-03-11 14:11:47 -08:00
Fangjin Yang f381c6066e Merge pull request #2638 from gianm/attempt-fix-2637
WorkerTaskMonitor: stop() waits for mainLoop to exit.
2016-03-11 13:27:24 -08:00
Gian Merlino 79a95f7789 WorkerTaskMonitor: stop() waits for mainLoop to exit.
Fixes #2637.
2016-03-11 11:40:13 -08:00
Himanshu 61cd838d6b Merge pull request #2640 from guobingkun/fix_doc
fix broken link for Tasks
2016-03-11 12:55:04 -06:00
Bingkun Guo 96c981cd0a fix broken link for Tasks 2016-03-11 11:36:34 -06:00
Gian Merlino 05397a9b4f ThreadPoolTaskRunner: Make graceful shutdown logs less scary.
- It's okay to suppress InterruptedException during graceful shutdown, as
  tasks may use it to accelerate their own shutdown.
- It's okay to ignore return statuses during graceful shutdown (which may
  be FAILED!) because it actually doesn't matter what they are.
2016-03-11 07:49:29 -08:00
Fangjin Yang f4ab1c2e52 Merge pull request #2632 from gianm/examples-druid-provided
examples: Switch druid-server, druid-common to "provided".
2016-03-10 19:57:32 -08:00
Gian Merlino d63473e0d5 examples: Switch druid-server, druid-common to "provided". 2016-03-10 18:43:29 -08:00
Gian Merlino f22fb2c2cf KafkaIndexTask.
Reads a specific offset range from specific partitions, and can use dataSource metadata
transactions to guarantee exactly-once ingestion.

Each task has a finite lifecycle, so it is expected that some process will be supervising
existing tasks and creating new ones when needed.
2016-03-10 18:41:43 -08:00
Gian Merlino 187569e702 DataSource metadata.
Geared towards supporting transactional inserts of new segments. This involves an
interface "DataSourceMetadata" that allows combining of partially specified metadata
(useful for partitioned ingestion).

DataSource metadata is stored in a new "dataSource" table.
2016-03-10 17:41:50 -08:00
Gian Merlino 3d2214377d Appenderatoring.
Appenderators are a way of getting more control over the ingestion process
than a Plumber allows. The idea is that existing Plumbers could be implemented
using Appenderators, but you could also implement things that Plumbers can't do.

FiniteAppenderatorDrivers help simplify indexing a finite stream of data.

Also:
- Sink: Ability to consider itself "finished" vs "still writable".
- Sink: Ability to return the number of rows contained within the sink.
2016-03-10 17:41:50 -08:00
Gian Merlino 08284fea62 Publish test-jar for indexing-service. 2016-03-10 16:50:37 -08:00
Gian Merlino 92c828f904 Make SegmentHandoffNotifier Closeable. 2016-03-10 16:50:37 -08:00