Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
9,184 Commits 113 Branches 505 Tags 731 MiB
Commit Graph

3302 Commits

Author SHA1 Message Date
Bingkun Guo 06c220ae7b build number 11 2015-12-21 19:40:47 -06:00
Bingkun Guo 93a91e5633 build number 10 2015-12-21 18:21:15 -06:00
Bingkun Guo 471dd7e164 build number 9 2015-12-21 17:35:18 -06:00
Bingkun Guo 627c311923 build #8 2015-12-21 17:06:37 -06:00
Bingkun Guo 293f2ad65d build #7 2015-12-21 16:09:19 -06:00
Bingkun Guo 9ed843ea21 build #6 2015-12-21 15:41:23 -06:00
Bingkun Guo f8342ca240 build #5 2015-12-21 15:10:37 -06:00
Bingkun Guo 52261c5539 build #4 2015-12-21 14:41:38 -06:00
Bingkun Guo 47a8f2d3f1 build #3 2015-12-21 13:56:43 -06:00
Bingkun Guo 5cc56cf9e7 build #2 2015-12-21 13:05:38 -06:00
Bingkun Guo f12cfa99fa another effort on fixing the transient error for Broker/CoordinatorServerViewTest happend on Travis build 2015-12-21 12:31:36 -06:00
Charles Allen c2a2d19d62 Merge pull request #2126 from guobingkun/remove_single_announcer 
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
 2015-12-21 08:55:07 -08:00
navis.ryu 7b24d1f355 Fix sporadic fail of DruidCoordinatorTest#testCoordinatorRun 2015-12-21 17:00:52 +09:00
Bingkun Guo 951a4e9b35 Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer 2015-12-21 00:05:53 -06:00
Bingkun Guo 89b477970f DataSegmentFinder tool 
`insert-segment-to-db` is a tool that can insert segments into Druid metadata storage. It is intended to be used
to update the segment table in metadata storage after people manually migrate segments from one place to another.
It can also be used to insert missing segment into Druid, or even recover metadata storage by telling it where the
segments are stored.

Note: This tool expects users to have Druid cluster running in a "safe" mode, where there are no active tasks to interfere
the segments being inserted. Users can optionally bring down the cluster to make 100% sure nothing is interfering.
 2015-12-21 00:02:04 -06:00
navis.ryu ad3312171b Fix sporadic fail of io.druid.client.CoordinatorServerViewTest 2015-12-21 11:39:32 +09:00
Fangjin Yang 7019d3c421 Merge pull request #2107 from jon-wei/fix_smq 
More efficient SegmentMetadataQuery
 2015-12-18 16:40:47 -08:00
Fangjin Yang e52c9d2b3c Merge pull request #2095 from guobingkun/fix_client_merge_task_and_serde 
Fix getType() and Json serialization in ClientMergeQuery + serde tests
 2015-12-18 11:40:13 -08:00
jon-wei 356b07c6c3 More efficient SegmentMetadataQuery 2015-12-17 12:46:23 -08:00
Bingkun Guo 1e5aa2f3ac fix getType() and Json serialization in ClientMergeQuery and add serde tests 2015-12-15 12:08:43 -06:00
Gian Merlino e6c2db89da Merge pull request #1921 from jon-wei/flat_json 
Add docs and benchmark for JSON flattening parser
 2015-12-14 20:53:54 -08:00
Xavier Léauté a2d0bea223 Merge pull request #2074 from guobingkun/add_comment 
Add comment to BrokerServerViewTest
 2015-12-09 16:19:27 -08:00
jon-wei c53bf85d83 Add docs and benchmark for JSON flattening parser 2015-12-09 16:13:30 -08:00
Bingkun Guo 31502a920c Add comment to BrokerServerViewTest 
Explain why we need to do explicit checks on existences of zNodes before we create them. Address issue: https://github.com/druid-io/druid/issues/1512
 2015-12-09 14:01:07 -06:00
Fangjin Yang ce79d707dd Merge pull request #2072 from guobingkun/fix_transient_error_brokerserverviewtest 
An effort to fix transient error in BrokerServerViewTest
 2015-12-09 11:12:35 -08:00
Bingkun Guo 161c4ce288 An effort to fix transient error in BrokerServerViewTest 2015-12-09 10:50:13 -06:00
Bingkun Guo c4de26ef09 fix typo in InitializationTest.java 2015-12-09 00:11:03 -06:00
Charles Allen 5bc6d61a25 Fix InitializationTest 2015-12-08 17:37:56 -08:00
Xavier Léauté dcd15735cd Merge pull request #2015 from metamx/handoff-notifier-coordinator 
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoff information
 2015-12-08 14:06:39 -08:00
Nishant a32906c7fd Remove FilteredServerView 2015-12-09 01:54:12 +05:30
Nishant 9491e8de3b Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoffs 
- fixes #1970
- extracted out segment handoff callbacks in SegmentHandoffNotifier
which is responsible for tracking segment handoffs and doing callbacks
when handoff is complete.
- Coordinator now maintains a view of segments in the cluster, this
will affect the jam heap requirements for the overlord for large
clusters.
realtime index task and nodes now use HTTP end points exposed by the
coordinator to get serverView

review comment

fix realtime node guide injection

review comments

make test not rely on scheduled exec

fix compilation

fix import

review comment

introduce immutableSegmentLoadInfo

fix son reading

remove unnecessary logging
 2015-12-09 01:54:09 +05:30
Fangjin Yang 56fd48c0e4 Merge pull request #2056 from guobingkun/fix_server_selector2 
Bug fix: stale segment in ServerSelector
 2015-12-07 16:57:33 -08:00
Fangjin Yang ba980dcadb Merge pull request #2058 from cheddar/report-uncovered-segments 
Add "uncoveredIntervals" to responseContext
 2015-12-07 16:42:42 -08:00
Eric Tschetter a36185926e Add "uncoveredIntervals" to responseContext 
This change will cause the CachingClusteredClient to populate the "uncoveredIntervals"
key in the responseContext map.  The value will be any intervals that were requested in
the query but are not actually covered by the segments underlying the data source.

For unit testing, CachingClisteredClientTest is testing the caching behavior of the
object and it is pretty hard to adjust it to only test this new behavior, so I created
a new, parallel "CachingClusteredClientFunctionalityTest" to simplify testing just
basic functionality.
 2015-12-07 16:16:57 -08:00
Bingkun Guo 970b9cfcaa fix stale segment in ServerSelector 2015-12-07 14:04:34 -06:00
Fangjin Yang 00416e26ff Merge pull request #2012 from himanshug/add_comment 
adding more comments to why drops do not happen
 2015-12-07 11:50:06 -08:00
Himanshu Gupta 62ba9ade37 unifying license header in all java files 2015-12-05 22:16:23 -06:00
刘麒贇 3781aa380f Added an injection test for ZkCoordinator. 2015-12-05 11:56:22 +08:00
Himanshu Gupta 221fb95d07 add support for getting used segments for multiple interval in IndexerMetadataStorageCoordinator 2015-12-03 21:28:04 -06:00
Nishant 1eb8211346 Add datasource and taskId to metrics emitted by peons 
This PR adds the datasource and taskId to the jvm and sys metrics
emitted by the peons.

fix spelling

review comment

review comment
 2015-12-01 23:20:59 +05:30
Himanshu Gupta 7c91219130 adding more comments to why drops do not happen when there are no segments in db 2015-11-25 16:26:51 -06:00
Fangjin Yang 8e83d800d6 Merge pull request #1881 from gianm/restartable-tasks 
Restorable indexing tasks
 2015-11-23 21:14:37 -08:00
Gian Merlino 13af260102 Merge pull request #1957 from metamx/skip-incremental-segment 
Ability to skip Incremental Index during query using query context
 2015-11-23 17:32:57 -08:00
Gian Merlino 501dcb43fa Some changes that make it possible to restart tasks on the same hardware. 
This is done by killing and respawning the jvms rather than reconnecting to existing
jvms, for a couple reasons. One is that it lets you restore tasks after server reboots
too, and another is that it lets you upgrade all the software on a box at once by just
restarting everything.

The main changes are,

1) Add "canRestore" and "stopGracefully" methods to Tasks that say if a task can
   stop gracefully, and actually do a graceful stop. RealtimeIndexTask is the only
   one that currently implements this.

2) Add "stop" method to TaskRunners that attempts to do an orderly shutdown.
   ThreadPoolTaskRunner- call stopGracefully on restorable tasks, wait for exit
   ForkingTaskRunner- close output stream to restorable tasks, wait for exit
   RemoteTaskRunner- do nothing special, we actually don't want to shutdown

3) Add "restore" method to TaskRunners that attempts to bootstrap tasks from last run.
   Only ForkingTaskRunner does anything here. It maintains a "restore.json" file with
   a list of restorable tasks.

4) Have the CliPeon's ExecutorLifecycle lock the task base directory to avoid a restored
   task and a zombie old task from stomping on each other.
 2015-11-23 11:22:08 -08:00
Gian Merlino 36569094ce EventReceiverFirehose: Drain buffer when closed, until empty. 2015-11-20 19:12:59 -08:00
Fangjin Yang 21c84b5ff7 Merge pull request #1896 from gianm/allocate-segment 
SegmentAllocateAction (fixes #1515)
 2015-11-18 21:05:46 -08:00
Nishant 60f649dab1 Ability to skip Incremental Index during query using query context 
This PR adds adds the ability to skip incremental index when querying
results from realtime nodes. default behaviour is to include
incrementalIndex in queries.

review comment
 2015-11-19 00:29:41 +05:30
Charles Allen 8fcf2403e3 Merge pull request #1943 from metamx/realtime-caching 
Enable caching on intermediate realtime persists
 2015-11-17 15:06:43 -08:00
Charles Allen 8fa34ee671 Merge pull request #1973 from metamx/cachemonitor-optional-injection 
CacheMonitor - make cache injection optional
 2015-11-17 13:02:06 -08:00
Himanshu 9118b010c0 Merge pull request #1976 from noddi/feature/count-parameter-rules-history 
Add count parameter to rules history endpoint.
 2015-11-17 14:41:50 -06:00
Charles Allen dbe201aeed Merge pull request #1929 from pjain1/jetty_threads 
separate ingestion and query thread pool
 2015-11-17 12:14:25 -08:00
Parag Jain 6c498b7d4a separate ingestion and query thread pool 2015-11-17 13:42:41 -06:00
Bartosz Ługowski b0a444eafc Add count parameter to rules history endpoint. 2015-11-17 20:14:12 +01:00
Xavier Léauté d7eb2f717e enable query caching on intermediate realtime persists 2015-11-17 10:58:00 -08:00
Xavier Léauté 71376efd57 make cache injection optional 
allows enabling the CacheMonitor for all index tasks, even if some don't
bind any cache instance.
 2015-11-16 22:40:01 -08:00
Fangjin Yang 4f46d457f1 Merge pull request #1947 from noddi/feature/count-parameter-history-endpoints 
Add count parameter to history endpoints
 2015-11-12 10:23:44 -08:00
Fangjin Yang 465cbcf9a7 Merge pull request #1956 from metamx/remove-unused-imports 
Cleanup + remove unused imports
 2015-11-11 17:36:47 -08:00
Gian Merlino e4e5f0375b SegmentAllocateAction (fixes #1515) 
This is a feature meant to allow realtime tasks to work without being told upfront
what shardSpec they should use (so we can potentially publish a variable number
of segments per interval).

The idea is that there is a "pendingSegments" table in the metadata store that
tracks allocated segments. Each one has a segment id (the same segment id we know
and love) and is also part of a sequence.

The sequences are an idea from @cheddar that offers a way of doing replication.
If there are N tasks reading exactly the same data with exactly the same logic
(think Kafka tasks reading a fixed range of offsets) then you can place them
in the same sequence, and they will generate the same sequence of segments.
 2015-11-11 16:54:35 -08:00
Charles Allen dc4ae59aaf Merge pull request #1865 from noddi/bugfix/issue-560 
Fix #560 - datasource segments summary shouldn't include replication.
 2015-11-11 15:29:08 -08:00
Bartosz Ługowski 6e5d2c6745 Add count parameter to history endpoints. 2015-11-11 23:03:57 +01:00
Gian Merlino 8e743b70c6 SegmentIdentifier, like a pre-DataSegment. 2015-11-11 12:25:43 -08:00
Xavier Léauté e26f7fca36 cleanup cache interface 2015-11-11 12:25:22 -08:00
Xavier Léauté fa6142e217 cleanup and remove unused imports 2015-11-11 12:25:21 -08:00
Gian Merlino f2c271e500 Better error message when LocalDataSegmentPusher cannot create its directory. 2015-11-10 19:49:01 -08:00
Xavier Léauté cf779946ef Merge pull request #1791 from guobingkun/event_receiver_firehose_monitor 
EventReceiverFirehoseMonitor
 2015-11-10 11:09:42 -08:00
Xavier Léauté a57cbfd2c3 Merge pull request #1387 from metamx/enableShutdownLogging 
Add special handler to allow logger messages during shutdown
 2015-11-09 17:20:09 -08:00
Charles Allen 1df4baf489 Move Jackson Guice adapters into io.druid 
* Removes access to protected methods in com.fasterxml
* Eliminates druid-common's use of foreign package com.fasterxml
 2015-11-09 10:50:45 -08:00
Fangjin Yang fe6efb6367 Merge pull request #1932 from gianm/data-schema-exclude-metric-names 
DataSchema: Exclude metric names from dimension list.
 2015-11-07 15:22:33 -08:00
Bartosz Ługowski 8b5de492cc Fix #560 - datasource segments summary shouldn't include replication. 2015-11-07 21:43:47 +01:00
Gian Merlino 6b908a5061 DataSchema: Exclude metric names from dimension list. 
Otherwise we could end up with a metric and a dimension with the same column name.
 2015-11-07 12:25:36 -08:00
Xavier Léauté e9533db987 Merge pull request #1850 from metamx/friendlyBardCache 
Allow setting upper limit on the number of cache segments a broker will try to fetch.
 2015-11-06 10:25:49 -08:00
Xavier Léauté c896818241 Update curator to 2.9.1 
Lots of bugfixes since 2.8.0
- https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314425&version=12333324
- https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12314425&version=12332392
 2015-11-05 15:53:01 -08:00
fjy 4efc90dda3 fix zkcoordinator inject 2015-11-04 17:30:17 -08:00
fjy 8f231fd3e3 cleanup druid codebase 2015-11-04 13:59:53 -08:00
Xavier Léauté 3b3d88bd03 Merge pull request #1857 from noddi/feature/intervals-route 
Queryable intervals
 2015-11-03 13:19:04 -08:00
Himanshu Gupta 8b67417ac8 make methods in Index[Merger,Maker,IO] non-static so that they can have 
appropriate ObjectMapper injected instead of creating one statically
 2015-11-02 23:24:26 -06:00
Fangjin Yang f90ddfdb89 Merge pull request #1745 from himanshug/numbered_to_elastic 
making NumberedShardSpec elastic
 2015-10-30 16:01:03 -07:00
Bingkun Guo c3b6fcce9d Add EventReceiverFirehoseMonitor 
add an EventReceiverFirehoseMonitor so that we can monitor how many
events have been queued in the EventReceiverFirehose and get a sense
about whether the firehose is under too much pressure.
 2015-10-30 11:40:02 -05:00
Bartosz Ługowski 6de51e022d Coordinator - add intervals route. 2015-10-30 11:51:42 +01:00
Xavier Léauté 149333d77d forward cancellation request to all brokers, fixes #1802 2015-10-29 16:58:38 -07:00
Charles Allen dfce14ed17 Allow setting upper limit on the number of cache segments a broker will try to fetch. 2015-10-29 11:50:00 -07:00
Xavier Léauté 59872bd0cd Merge pull request #1809 from metamx/fifoPriorityExecutorService 
Make PrioritizedExecutorService optionally FIFO
 2015-10-27 15:19:32 -07:00
Fangjin Yang ea2267e08c Merge pull request #1868 from gianm/fix-announcements 
Historical and MiddleManager server announcements should not remove parents.
 2015-10-27 14:50:05 -07:00
Gian Merlino 7df7370935 Merge pull request #1862 from metamx/indexingServiceMMGone 
Add timeout to shutdown request to middle manager for indexing service
 2015-10-27 14:38:01 -07:00
Charles Allen 7a2ceef690 Add special handler to allow logger messages during shutdown 
* Adds a special PropertyChecker interface which is ONLY for setting string properties at the very start of psvm
 2015-10-27 14:33:36 -07:00
Charles Allen ecdafa87c5 Make PrioritizedExecutorService optionally FIFO 2015-10-27 14:16:22 -07:00
Charles Allen 67d58f6d8d Update CoordinatorRuleManager for http-client 1.0.4 2015-10-27 14:08:45 -07:00
Gian Merlino 4b92752deb Historical and MiddleManager server announcements should not remove parents. 
Removing parent paths causes watchers of the "announcements" path to get stuck
and stop seeing new updates.
 2015-10-27 08:06:11 -07:00
Fangjin Yang 5a082b2f5e Merge pull request #1824 from metamx/UniformGranularitySpecHashEquals 
Add hashCode and equals to UniformGranularitySpec
 2015-10-26 09:34:01 -07:00
Fangjin Yang 5f23703216 Merge pull request #1638 from guobingkun/remove_maven_client_code 
Remove Maven client at runtime + Provide a way to load Druid extensions through local file system
 2015-10-26 09:30:05 -07:00
Xavier Léauté 72c408cf2d Merge pull request #1770 from metamx/merge-time 
Add segment merge time as a metric
 2015-10-22 22:03:41 -07:00
Himanshu 853f95f64c Merge pull request #1845 from metamx/moreVerboseRuleAuditLogging 
Add more verbose logging to SQLMetadataRuleManager
 2015-10-22 16:12:46 -05:00
Nishant 7cecc55045 Add segment merge time as a metric 
Add merge and persist cpu time

Fix typo

review comment

move cpu time measuring to VMUtils

review comments.
 2015-10-22 12:28:03 +05:30
Charles Allen 6ceab03fc4 Add debug logging to start of segment manager polling 2015-10-21 16:17:49 -07:00
Charles Allen 1cad571354 Add more verbose logging to SQLMetadataRuleManager 2015-10-21 16:11:40 -07:00
Bingkun Guo 4914925d65 New extension loading mechanism 
1) Remove maven client from downloading extensions at runtime.
2) Provide a way to load Druid extensions and hadoop dependencies through file system.
3) Refactor pull-deps so that it can download extensions into extension directories.
4) Add documents on how to use this new extension loading mechanism.
5) Change the way how Druid tarball is generated. Now all the extensions + hadoop-client 2.3.0
are packaged within the Druid tarball.
 2015-10-21 14:22:36 -05:00
Xavier Léauté e4ac78e43d bump next snapshot to 0.9.0 2015-10-20 13:46:13 -07:00
Himanshu 0292db648d Merge pull request #1833 from metamx/improve-log 
Improve alert message in CachingClusteredClient - Add Datasource info.
 2015-10-20 15:40:57 -05:00
Nishant bf214f23b2 Improve message for alert - Add Datasource info. 2015-10-16 19:43:21 +05:30
Xavier Léauté 4c2c7a2c37 update version to 0.8.3 2015-10-14 21:40:55 -07:00
Charles Allen f432b8e3f9 Add hashCode and equals to UniformGranularitySpec 
* Also add hashCode != 0 to AllGranularity and NoneGranularity
 2015-10-13 16:42:21 -07:00
Charles Allen bf11723a52 Update usages of io.druid.client.selector.Server to build URL or URI directly instead of using String.format 2015-10-12 12:30:56 -07:00
Charles Allen e450877a78 Make ServerDiscoverySelector more IPv6 friendly 2015-10-08 10:03:50 -07:00
Charles Allen 693e14a14d Add unit tests for ipv6 in ServerDiscoverySelectorTest 2015-10-08 10:03:22 -07:00
Charles Allen 020a706ac4 Merge pull request #1800 from metamx/hybrid-cache 
Hybrid L1/L2 cache
 2015-10-06 15:18:07 -07:00
Xavier Léauté b11c0859e8 hybrid l1/l2 cache to combine local and remote cache 2015-10-06 14:32:08 -07:00
Fangjin Yang b5737a233a Merge pull request #1801 from guobingkun/clientInfoResource 
Fix ClientInfoResource
 2015-10-06 07:53:44 -07:00
Charles Allen 2bc4aed1e5 Add CPUTimeMetricQueryRunner to ClientQuerySegmentWalker 2015-10-05 10:09:22 -07:00
Bingkun Guo 57c8e56451 Fix ClientInfoResource 
Fix ClientInfoResource so that it doesn't return empty list for overshadowed segments
 2015-10-02 14:12:55 -05:00
Himanshu 166c4fcf46 Merge pull request #1795 from metamx/announcerTestTransactions 
Try and make AnnouncerTest a bit more predictable
 2015-10-02 13:04:33 -05:00
Charles Allen 7d635a2ce2 Try and make AnnouncerTest a bit more predictable 2015-10-02 09:19:40 -07:00
Xavier Léauté 5db4c5a089 move cache bindings into a dedicated module 2015-10-02 00:29:05 -04:00
Himanshu Gupta b9740a727b unit test for IngestSegmentFirehose 2015-10-01 16:06:04 -05:00
Charles Allen 2d847ad654 Merge pull request #1730 from metamx/union-queries-fix 
fix #1727 - Union bySegment queries fix
 2015-09-29 12:23:25 -07:00
Nishant 573aa96bd6 fix #1727 - Union bySegment queries fix 
Fixes #1727.
revert to doing merging for results for union queries on broker.

revert unrelated changes

Add test for union query runner

Add test

remove unused imports

fix imports

fix renamed file

fix test

update docs.
 2015-09-29 23:32:36 +05:30
Gian Merlino 348172203f OverlordRedirectInfo: Fix ability to detect that there is no leader. 2015-09-25 09:30:09 -07:00
Gian Merlino 906e99d631 RedirectFilter: User agents are more likely to preserve method on 307. 2015-09-25 08:54:23 -07:00
Gian Merlino 3aba401ee0 SQLMetadataConnector: Retry table creation, in case something goes wrong. 
Also rejigger table creation methods to not take a DBI. It's already available
inside the connector, and everyone was just using that one anyway.
 2015-09-24 21:39:36 -07:00
Charles Allen 4308aa93e2 Add meter metrics for memcached 2015-09-22 12:50:54 -07:00
Charles Allen 46ba60f10f Enable logging for memcached in factory 2015-09-22 11:13:47 -07:00
Xavier Léauté 1c86eac9e1 Merge pull request #1760 from metamx/memcached082Updates 
Update memcached client for better concurrency in metrics.
 2015-09-22 10:59:43 -07:00
Xavier Léauté 35caa753aa Merge pull request #1761 from gianm/local-path-for-hadoop 
LocalDataSegmentPusher: Fix for Hadoop + relative paths.
 2015-09-22 08:57:44 -07:00
Gian Merlino 4efbe649a3 LocalDataSegmentPusher: Fix for Hadoop + relative paths. 2015-09-22 07:44:12 -07:00
Charles Allen e6f07a832d Update memcached client for better concurrency in metrics. Also fixes another injection problem 2015-09-22 07:40:24 -07:00
Himanshu 34a8fbcd68 Merge pull request #1755 from metamx/update-druid-api 
update druid-api for timestamp parsing speedup
 2015-09-21 19:43:13 -05:00
Gian Merlino 490b1aea2a Merge pull request #1758 from metamx/fixMemcachedInjection 
Fix memcached cache provider injection and add test
 2015-09-21 15:36:29 -07:00
Charles Allen 37eb83a1d0 Fix memcached cache provider injection and add test 2015-09-21 15:00:22 -07:00
Xavier Léauté 8f489c1f66 fix type cast warnings 2015-09-21 09:57:29 -07:00
Xavier Léauté af86c0e6ea update druid-api + java-util for timstamp parsing speedup 2015-09-21 09:57:29 -07:00
Himanshu Gupta 117a6754b5 improving msging when indexing service is not found 2015-09-18 23:07:44 -05:00
Fangjin Yang 6605a60ef0 Merge pull request #1747 from jon-wei/realtime_corrupt_sink_fix 
Account for potential gaps in hydrants in sink initialization, hydrant swapping (e.g. h0, h1, h4)
 2015-09-18 15:22:50 -07:00
jon-wei 9f6bb03ef4 Account for potential gaps in hydrants in sink initialization, hydrant swapping (e.g. h0, h1, h4) 2015-09-18 14:53:37 -07:00
Xavier Léauté df4c8a3aa5 balance memcached cache across multiple connections 2015-09-18 09:52:16 -07:00
Himanshu Gupta a7b1cacbbf make NumberedShardSpec "elastic" and UTs 2015-09-17 08:38:48 -05:00
Gian Merlino 9705c5139b Merge pull request #1732 from jon-wei/segmentmeta 
Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups
 2015-09-16 12:36:25 -07:00
Himanshu Gupta 74f4572bd4 Lazily deserialize "parser" to InputRowParser in DataSchema 
so that user hadoop related InputRowParsers are created only when needed
this allows overlord to accept a HadoopIndexTask with a hadoopy InputRowParser
and not fail because hadoopy InputRowParser might need hadoop libraries
 2015-09-16 10:58:13 -05:00
Xavier Léauté b464da438c Merge pull request #1688 from metamx/moreMemcachedMetrics 
More memcached metrics
 2015-09-15 15:33:51 -07:00
jon-wei 193fb4fdfc Add support for a configurable default segment history period for segmentMetadata queries and GET /datasources/<datasourceName> lookups 2015-09-14 19:41:42 -07:00
Himanshu 5ff92664f8 Merge pull request #1696 from metamx/cpuTimeReporting 
Add CPU time to metrics for segment scanning.
 2015-09-14 10:53:55 -05:00
Himanshu Gupta ebdb612933 composing emitter module to use multiple emitters together 2015-09-09 16:45:50 -05:00
Charles Allen 5813856819 More memcached metrics 2015-09-08 13:34:58 -07:00
Charles Allen fcf5cae81d Add CPU time to metrics for segment scanning. 2015-09-08 13:34:19 -07:00
Gian Merlino 062a47fba4 Modify Plumbers in these ways, 
1) Persist using Committer instead of Runnable. (Although the metadata object
   is ignored in this patch)

2) Remove the getSink method.

3) Plumbers are now responsible for time-based and hydrant-full-based periodic
   committing. (FireChief, RealtimeIndexTask, and IndexTask used to do this)
 2015-09-03 11:13:06 -07:00
Nishant e79572b4bc Merge pull request #1637 from metamx/peonPlaysNiceOnShutdown 
Make jetty scheduler threads daemon thread
 2015-09-03 00:03:20 +05:30
Nishant 0096e6a0a0 Merge pull request #1658 from metamx/cleanupJDBCExtractionNamespaceTest 
Hopefully add better timeouts and ordering to JDBCExtractionNamespaceTest
 2015-09-02 23:49:49 +05:30
nishant 0835e12f2a add endpoint to fetch rule history for all datasources. 
review comments

Review comment

fix compilation
 2015-08-31 14:05:52 +05:30
Fangjin Yang dfe3be0f6d Merge pull request #1689 from metamx/lz4TranscoderUpdate 
Update LZ4Transcoder to match Compressed strategy factory type.
 2015-08-29 12:31:59 -07:00
Charles Allen 413c0d1c9e Update LZ4Transcoder to match Compressed strategy factory type. 2015-08-28 19:19:48 -07:00
Gian Merlino 940e1aa3eb Replace funky imports with standard ones. 
1) Lots of Guava imports were not coming from the actual Guava
2) junit.framework.Assert should be org.junit.Assert
 2015-08-28 18:02:05 -07:00
Himanshu ceaa49ec4f Merge pull request #1681 from gianm/ingest-segment-overlapping 
Fix overlapping segments in IngestSegmentFirehose, DatasourceInputFormat
 2015-08-28 11:12:51 -05:00
Gian Merlino e575ef2b20 Remove unused Indexer interface. 2015-08-28 08:40:47 -07:00
Gian Merlino 414a6fb477 Fix overlapping segments in IngestSegmentFirehose, DatasourceInputFormat. 
Fixes #1678. IngestSegmentFirehose (and its users) need to remember which
windows of which segments should actually be read, based on a timeline.
 2015-08-28 07:32:41 -07:00
Himanshu Gupta 2e0dd1d792 adding UTs and addressing review comments to 
firehoseV2 addition to Realtime[Manager|Plumber],
essential segment metadata persist support,
kafka-simple-consumer-firehose extension patch
 2015-08-27 20:50:46 -05:00
lvjq 2237a8cf0f kafka 8 simple consumer firehose 2015-08-27 20:50:46 -05:00
Charles Allen 10af233b37 Merge pull request #1622 from metamx/smile-eventreceiverfirehose 
Support for JSON Smile format for EventReceiverFirehoseFactory
 2015-08-27 10:42:25 -07:00
Charles Allen ac8e32b58e Hopefully add better timeouts and ordering to JDBCExtractionNamespaceTest 2015-08-26 23:05:51 -07:00
Gian Merlino 5b67ec6cda Merge pull request #1672 from metamx/derbyRule 
Migrate TestDerbyConnector to a JUnit @Rule
 2015-08-26 22:56:18 -07:00
Charles Allen e38cf54bc8 Migrate TestDerbyConnector to a JUnit @Rule 2015-08-26 21:47:40 -07:00
Xavier Léauté e9237611c5 Merge pull request #1668 from metamx/cachingClusteredClientCodeCleanup1 
Code cleanup for CachingClusteredClientTest
 2015-08-26 17:13:55 -07:00
Xavier Léauté c325b194c0 Merge pull request #1661 from pjain1/poll_rules 
Poll rules immediately after change
 2015-08-25 13:46:55 -07:00
Charles Allen 85aeda834c Code cleanup for CachingClusteredClientTest 2015-08-25 12:49:55 -07:00
Gian Merlino 2bf9a70bfa Consolidate SQL retrying by moving logic into the connectors. 
Also change boolean removeLock to void addLock in MetadataStorageActionHandler.
 2015-08-25 12:42:29 -07:00
Parag Jain 073c32f01b Poll rules immediately after change 2015-08-24 16:08:25 -05:00
Fangjin Yang e3964cd9f6 Merge pull request #1653 from metamx/fix-success-flag 
fix success flag emitted in router query metrics
 2015-08-21 07:56:57 -07:00
Nishant 6d85a62ed2 fix success flag emitted in router query metrics 2015-08-21 12:25:57 +05:30
Nishant e6b20db62d Support for JSON Smile format for EventReceiverFirehoseFactory 
Support for Smile format for EventReceiverFireshoseFactory.
would help in decreasing network load on middle managers ingesting high
data volumes.

review comments
 2015-08-20 19:52:45 +05:30
Fangjin Yang 6a0389297c Merge pull request #1634 from metamx/IrcFirehoseFixes 
Allow IrcFirehoseFactory to shutdown cleanly
 2015-08-18 21:06:34 -07:00
Xavier Léauté 3b2e41e42a update for next release 2015-08-18 17:16:46 -07:00
Charles Allen 13773865cf Make jetty scheduler threads daemon thread 2015-08-18 13:08:19 -07:00
Charles Allen cd2c3776f8 Allow IrcFirehoseFactory to shutdown cleanly 2015-08-17 13:24:41 -07:00
Himanshu Gupta 4d4aa8bfc6 refactor IngestSegmentFirehoseFactory so that IngestSegmentFirehose becomes reusable 
Conflicts:
	indexing-service/src/main/java/io/druid/indexing/firehose/IngestSegmentFirehoseFactory.java
 2015-08-14 14:44:22 -05:00
Charles Allen db19d2d547 Revert "Update to guice 4.0" 2015-08-14 09:26:07 -07:00
Charles Allen be89105621 Merge pull request #1602 from metamx/more-code-cleanup 
Some perf Improvements in Broker
 2015-08-11 13:51:49 -07:00
Charles Allen 7e61216287 Update to guice 4.0 
- Mark a lot of `@Provides` methods as final since guice 4.0 disallows overriding them
 2015-08-10 13:57:18 -07:00
Nishant b33c644da0 avoid unnecessary call to MultipleSpecificSegmentSpec.getIntervals() 
avoid unnecessary call to MultipleSpecificSegmentSpec.getIntervals()
profiling shows it took upto 6-12% of cpu time in
JodaUtils.condenseIntervals
 2015-08-06 13:51:17 +05:30
Charles Allen 9dd18de1a5 Merge pull request #1596 from metamx/console-artifact 
make console a maven dependency instead of filedump
 2015-08-04 15:28:45 -07:00
Xavier Léauté 07ee1acca2 make console a maven dependency instead of filedump 2015-08-04 09:49:34 -07:00
Fangjin Yang b05be0a104 Merge pull request #1593 from metamx/run-tests-offline 
Fix test so that it can run offline
 2015-08-04 09:23:40 -07:00
Nishant f490547faf Fix test so that it can run offline too 
Test was getting stuck for me when I tried to run it offline while
resolving the host. creating unresolved SocketAddress works around
this.
 2015-08-03 23:37:04 +05:30
Bingkun Guo 479b09c2dd fix transient error in BrokerServerViewTest 2015-07-31 13:33:55 -05:00
Fangjin Yang 22567946cf Merge pull request #1259 from metamx/queryTimeLookup 
Query Time Lookup
 2015-07-28 11:43:05 -10:00
Charles Allen ba59f8afc4 Merge pull request #1550 from himanshug/optionally_log_all_requests 
print *all* HTTP requests to log if configured
 2015-07-28 11:19:36 -07:00
Charles Allen 86ede702b1 Add namespaced lookups as extensions 
* Adds kafka, URI, and JDBC namespace defintions
* Add ability to explicitly rename using a "namespace" which is a particular data collection that is loaded on all realtime, historic nodes, and brokers. If any of these nodes has the namespace extension, ALL nodes have the namespace extension.
* Add namespace caching and populating (can be on heap or off heap)
* Add NamespaceExtractionCacheManager for handling caches
* Added ExtractionNamespace for handling metadata on the extraction namespaces
* Added ExtractionNamespaceUpdate for handling metadata related to updates
* Add extension which caches renames from a kafka stream (requires kafka8)
* Added README.md for the namespace kafka extension
* Added docs
* Added namespace/size, namespace/count, namespace/deltaTasksStarted metrics

Add static config for namespaces via `druid.query.extraction.namespace`
* This is a rebase of https://github.com/b-slim/druid/tree/static_config_only
 2015-07-28 11:14:14 -07:00
Himanshu Gupta 90b475975e Log all the requests to debug level in logger io.druid.jetty.RequestLog 2015-07-28 12:56:04 -05:00
nishant efb6e0649e simplify merging of results on broker. 
add benchmark

formatting and review comments

organize imports
 2015-07-27 11:46:11 -07:00
Xavier Léauté 6e8da5de86 Merge pull request #1441 from metamx/queryResourceFlusher 
Add flush prior to close on output stream in QueryResource
 2015-07-17 16:32:52 -07:00
Fangjin Yang e21195f987 Merge pull request #1469 from guobingkun/table_config 
Inconsistent property names for "druid.metadata.storage.tables.xxx"
 2015-07-17 07:43:19 -07:00
Charles Allen 456ad9ffba Merge pull request #1529 from metamx/update-versions 
inrement version
 2015-07-15 13:25:31 -07:00
Xavier Léauté 4cfb00bc8a inrement version 2015-07-15 13:09:05 -07:00
Fangjin Yang a4cb0332fb Merge pull request #1524 from metamx/peonShutdown 
Make CliPeon run shutdown hooks properly
 2015-07-15 10:09:33 -07:00
Charles Allen 92d5c250ab Remove comment 2015-07-14 16:20:36 -07:00
Charles Allen 5eadd395e2 Move lots of executor service creation to Execs 2015-07-14 15:38:49 -07:00
Charles Allen 2b87ad307d Make CliPeon run shutdown hooks properly 
* Also make jetty threads daemon
 2015-07-14 15:15:26 -07:00
Fangjin Yang da8e86146e Merge pull request #1505 from metamx/improve-test 
Modify test to check for multiple shards for same interval
 2015-07-14 08:50:47 -07:00
Fangjin Yang 3f7ba58227 Merge pull request #1504 from metamx/fix-1447 
fix for #1447
 2015-07-14 08:50:08 -07:00
Himanshu 1affdf7bec Merge pull request #1502 from samjhecht/poll-alerting 
add alert on errors polling for rules
 2015-07-13 18:37:37 -05:00
samjhecht fc438e5a72 add alert on errors polling for rules 
reset retrystarttime after alerting
 2015-07-13 09:03:51 -07:00
Himanshu 725086cc89 Merge pull request #1506 from gianm/realtime-plumber-nulls 
Consider null inputRows and parse errors as unparseable during realtime ingestion.
 2015-07-13 10:12:12 -05:00
Gian Merlino 9068bcd062 Consider null inputRows and parse errors as unparseable during realtime ingestion. 
Also, harmonize exception handling between the RealtimeIndexTask and the RealtimeManager.
Conditions other than null inputRows and parse errors bubble up in both.
 2015-07-11 20:40:03 -07:00
Himanshu e9b627bf55 Merge pull request #1476 from pjain1/improved_tests 
all the review comments have been taken care of, so merging.
 2015-07-09 16:25:30 -05:00
Nishant 1766341648 Modify test to check for multiple shards for same interval 
Modify test to check for multiple shards for same interval.
 2015-07-09 20:35:31 +05:30
Parag Jain 58cc3954ff improved coordinator and datasourceresource tests 2015-07-09 09:46:54 -05:00
Nishant 5fe27fe4ad fix for #1447 
fixes #1447
 2015-07-09 19:05:48 +05:30
Nishant 184b12bee8 fix groupBy caching to work with renamed aggregators 
Issue - while storing results in cache we store the event map which
contains aggregator names mapped to values. Now when someone fire same
query after renaming aggs, the cache key will be same but the event
will contain metric values mapped to older names which leads to wrong
results.
Fix - modify cache to not store raw event but the actual list of values
only.

review comments + fix dimension renaming

review comment
 2015-07-09 11:48:26 +05:30
Nishant fe36f7f852 emit alert for DB polling error 2015-07-07 19:51:36 +05:30
Fangjin Yang d846f3ced6 Merge pull request #1498 from rasahner/doc_localFirehose 
say explicitly that local firehose searches dirs recursively for files
 2015-07-05 15:34:36 -07:00
sahner acd20e8c00 say explicitly that local firehose searches directories recursively for files 2015-07-05 14:46:44 -05:00
Charles Allen f2919b98b1 Merge pull request #1471 from metamx/direct-client-metrics 
add query/node/time metrics to DirectDruidClient
 2015-06-30 22:06:44 -07:00
Bingkun Guo 4a0ae7d8d5 Fix inconsistent druid property names for "druid.metadata.storage.tables.xxx" between document and code 2015-06-29 10:12:30 -05:00
Himanshu 8172bdff1c Merge pull request #1468 from guobingkun/fix_npe_for_storage_action_handler 
Fix npe thrown from SQLMetadataStorageActionHandler
 2015-06-29 09:57:51 -05:00
Xavier Léauté 28fa1642b9 add node time metrics to DirectDruidClient 2015-06-26 17:57:44 -07:00
Charles Allen fbcac10e00 Remove metrics emitting from caching clustered client 2015-06-26 10:49:13 -07:00
Bingkun Guo e4e18b17b5 fix npe thrown from getEntry() and getStatus() SQLMetadataStorageActionHandler due to a non-exist entryId 2015-06-26 10:40:56 -05:00
Xavier Léauté 0c85c8c60a Merge pull request #1462 from pjain1/validate_rules 
tieredReplicants can be null
 2015-06-24 13:29:27 -07:00
Parag Jain ff738ace05 converting null tieredeplicants to default and not allowing empty ones 2015-06-24 14:55:12 -05:00
Fangjin Yang 0c13fd8018 Merge pull request #1450 from metamx/cache-key-distribution 
Fix bad distribution of cache keys across cache nodes
 2015-06-22 17:58:15 -07:00
Himanshu 34c8f4c3da Merge pull request #1437 from metamx/limitUsedSegmentInterval 
Add start and end bounds to getUsedSegmentsForInterval
 2015-06-22 14:11:38 -05:00
Charles Allen 8d91c8d399 Add start and end bounds to getUsedSegmentsForInterval 
* Add unit tests
* Add javadoc for methods in IndexerMetadataStorageCoordinator
 2015-06-22 11:46:53 -07:00
Xavier Léauté f1951b253c Fix bad distribution of cache keys across nodes 
With the existing hash function some nodes could end up with 3 times the
number of keys as others. The following changes improve that to roughly
less than 5% differences across nodes.

- switch from fnv-1a to murmur3_128 hash
- increase repetitions for ketama algorithm
- test to analyze distribution

Also updates spymemcached for recent bugfixes
 2015-06-19 15:30:35 -07:00
Xavier Léauté 0a5bb909a2 [maven-release-plugin] prepare for next development iteration 2015-06-18 17:35:19 -07:00
Xavier Léauté 59c6b2b279 [maven-release-plugin] prepare release druid-0.8.0-rc1 2015-06-18 17:35:14 -07:00
Xavier Léauté d276d2c90b Merge pull request #1444 from druid-io/logging-improvement 
Separate bootstrap threads from loading threads on historical startup
 2015-06-17 15:24:07 -07:00
Eric Tschetter 06c97b6d7d Separate out the "bootstrapping" threads from the "loading" threads. 
It is highly recommended to keep loading threads single threaded.  There can be benefits to having multiple bootstrapping threads, though.  Those bootstrapping threads shouldn't be kept alive for the whole life of the process, however.

Also, adjust some logging and do some code cleanup.
 2015-06-17 14:58:28 -07:00
Fangjin Yang 0539e0073d Merge pull request #1442 from guobingkun/broker_server_view_test 
Unit test for BrokerServerView
 2015-06-16 19:45:20 -07:00
Eric Tschetter 1aad7ce521 Add a bit more information to the mapping logging logic. 2015-06-16 19:33:38 -07:00
Bingkun Guo 446d01cc53 Unit test for BrokerServerView 2015-06-16 15:20:32 -05:00
Charles Allen 69bbe21512 Add flush prior to close on output stream in QueryResource 2015-06-16 11:40:56 -07:00
Xavier Léauté 9cf8662aeb Merge pull request #1427 from guobingkun/fix_issue_1016 
Fix duplicate extension loading issue described in #1016
 2015-06-10 14:42:24 -07:00
Bingkun Guo bdf4d541e3 Fix issue #1016 that if user specifies extension coordinates, the local extensions will be loaded twice, 
which could cause Guice duplicate binding errors.
Add unit test to replicate duplicate extension issue. Update documents. Add an package accessible getter for loadersMap for testing only.
Notice that extensions explicitly specified in druid.extensions.coordinates have a higher priority than ones included in the classpath.
Extension modules that don't have a canonical class name will be ignored.
 2015-06-10 15:48:03 -05:00
Xavier Léauté e5937482bc Merge pull request #1431 from pjain1/coordinator_ut 
Druid Coordinator test
 2015-06-10 07:24:39 -07:00
Parag Jain b72e6fd51a Druid Coordinator Unit test 2015-06-09 13:51:01 -05:00
Bingkun Guo 22607d19be Make AbstractHttpClientProvider contents lazily instantiated 2015-06-08 09:53:26 -05:00
Bingkun Guo 1646f650b6 Unit tests for LoadQueuePeon and SegmentLoaderLocalCacheManager, JavaDoc for LoadPeonCallback 2015-06-04 17:28:22 -05:00
Himanshu f81a0d9210 Merge pull request #1425 from guobingkun/guice_fix 
Anyways, @guobingkun , if possible,  can you please test proposed change and send another PR to fix #1016 .  but that doesn't necessarily hold this PR, so I'm merging this.
 2015-06-04 14:43:26 -05:00
Himanshu f44f7f07de Merge pull request #1424 from pjain1/validate_rules 
Check lower bound of replicant values for rules
 2015-06-04 14:16:50 -05:00
Parag Jain 1030899cdf Check lower bound of replicant values for rules 2015-06-04 13:47:43 -05:00
Bingkun Guo dc26350480 Separate overwrite of Guice modules into 2 steps so that extension modules can overwrite cli modules 2015-06-04 09:57:56 -05:00
nishant 21864a9407 reduce test runtime 
fix check

fix test
 2015-06-04 08:35:12 +05:30
Charles Allen 1c4d42bc15 Replace "timeout" with QueryContextKeys.TIMEOUT 2015-06-02 14:49:21 -07:00
fjy 185c99deec Add schema check for overlap between dimension and metric names 2015-05-28 22:18:08 -07:00
fjy be2a35188e Additional schema validations and better logs for common extensions 2015-05-27 16:25:02 -07:00
Charles Allen 051c3ccede Add futures to SQLMetadataSegmentManager and SQLMetadataRuleManager 2015-05-15 16:08:22 -07:00
Xavier Léauté 3c3db7229c Merge pull request #1355 from himanshug/long_max_min_aggregators 
Long max/min aggregators
 2015-05-13 12:08:11 -07:00
Himanshu Gupta d0ec945129 adding aliases doubleMax and doubleMin for max and min respectively 
renamed all [Max/Min]*.java to [DoubleMax/DoubleMin]*.java and created [Max/Min]AggregatorFactory.java which can be removed when we dont need the min/max aggregator type backward compatibility
 2015-05-13 09:25:41 -05:00
fjy 7a6acf5c1b update pom to 0.8 2015-05-11 19:41:58 -06:00
Fangjin Yang a2dc58cd2d Merge pull request #1345 from pjain1/unit_test_warn_fix 
fix warn msg and some unit tests
 2015-05-08 08:06:20 -07:00
Parag Jain 01448d264c Fix warn msg and added some unit tests 2015-05-07 17:10:05 -05:00
Fangjin Yang 7608bf5799 Merge pull request #1342 from guobingkun/unit_test 
Fix a potential issue in DeterminePartitionsJob, and two unit tests, reformat code
 2015-05-05 09:24:23 -07:00
Bingkun Guo 1ee550dd91 Fix a potential issue in DeterminePartitionsJob by making HadoopDruidIndexerConfig non-static, and two unit tests for DeterminPartitionsJob and LocalDataSegmentKiller 2015-05-04 20:00:29 -07:00
Gian Merlino e69d82a2b4 Realtime: Delay firehose connection until job is started. 
Some firehoses (like the Kafka firehose) acquire input resources when they
connect, so it helps to delay this until after plumber.startJob() runs.
 2015-05-04 10:54:07 -07:00
Gian Merlino 8eb441ec08 RealtimeManager: Include partitionNum in chief thread name. 2015-05-04 10:53:48 -07:00
Charles Allen 27e25fe513 fix IRCFirehoseFactory json name/nick mismatch 
* Now it is nick as per wikipedia example
 2015-05-01 12:35:44 -07:00
Fangjin Yang 15ed308a4b Merge pull request #1322 from himanshug/fix_audit_tbl_idx 
fixing the typo for audit table (type, created_date) index creation
 2015-04-30 13:22:04 -07:00
fjy a7f9bb1a1e Additional coordinator logs to figure out when racey things might be happening 2015-04-30 08:26:52 -07:00
Himanshu Gupta 4aa9381f6a fixing the typo for audit table (type, created_date) index creation 2015-04-29 17:09:35 -05:00
nishant 6e84caf3d5 Proxy Requests from coordinator to overlord 
Add unit test and support https scheme

review comments
 2015-04-29 20:58:26 +05:30
Charles Allen 062a413ccd Fix DruidNodeTest for ipv6 only hosts 2015-04-28 13:11:47 -07:00
Xavier Léauté 721505c017 Merge pull request #1208 from druid-io/rework-metrics 
Schemaless metrics + additional metrics for things we care about
 2015-04-27 15:04:54 -07:00
fjy 963e5765bf Schemaless metrics + additional metrics for things we care about 2015-04-27 13:39:40 -07:00
nishant 958b34e0c2 fix exception during timeboundary 
Fix Exception during time boundary query where results can be
represented as integers
 2015-04-23 20:32:35 +05:30
Fangjin Yang 1addbe163a Merge pull request #1295 from anubhgup/master 
Fix type of values in sql update.
 2015-04-17 20:26:17 -07:00
Anubhav Gupta 4c240da4e9 Fix type of values in sql update. 2015-04-17 18:44:49 -07:00
fjy d260515a43 update druid-api version 2015-04-17 14:58:35 -07:00
Xavier Léauté ea5572d001 Merge pull request #1271 from metamx/strictErrorChecking 
Add stricter checking for potential coding errors
 2015-04-15 15:21:41 -07:00
Charles Allen abdeaa0746 Add stricter checking for potential coding errors 
Can use via `mvn clean compile test-compile -P strict'
 2015-04-15 14:52:25 -07:00
Xavier Léauté 3a3046ccf3 add support for dimension compression 
- compression for single-value dimensions using CompressedVSizeIntsIndexedSupplier
- makes dimension compression configurable via IndexSpec
- IndexSpec also enables configuring bitmap and metric compression
 2015-04-14 10:44:18 -07:00
Xavier Léauté 44e23a8267 Merge pull request #1224 from metamx/alert-improve 
skip raising false alert when the coordinator looses leadership
 2015-04-07 10:11:34 -07:00
Fangjin Yang 208e307915 Merge pull request #1251 from metamx/uriSegmentLoaders 
Revert "Revert "Overhaul of SegmentPullers to add consistency and retries""
 2015-03-30 17:43:51 -07:00
fjy aea7f9d192 [maven-release-plugin] prepare for next development iteration 2015-03-30 16:35:24 -07:00
fjy 060d7aef03 [maven-release-plugin] prepare release druid-0.7.1 2015-03-30 16:35:20 -07:00
Charles Allen 1c6cbea89c Revert "Revert "Overhaul of SegmentPullers to add consistency and retries"" 
This reverts commit f904bc7858.
 2015-03-30 13:40:04 -07:00
Fangjin Yang f904bc7858 Revert "Overhaul of SegmentPullers to add consistency and retries" 2015-03-30 13:15:50 -07:00
Charles Allen 6d407e8677 Add URI handling to SegmentPullers 
* Requires https://github.com/druid-io/druid-api/pull/37
* Requires https://github.com/metamx/java-util/pull/22
* Moves the puller logic to use a more standard workflow going through java-util helpers instead of re-writing the handlers for each impl
  * General workflow goes like this: 1) LoadSpec makes sure the correct Puller is called with the correct parameters. 2) The Puller sets up general information like how to make an InputStream, how to find a file name (for .gz files for example), and when to retry. 3) CompressionUtils does most of the heavy lifting when it can
 2015-03-30 12:33:23 -07:00
Fangjin Yang e5653f0752 Merge pull request #1190 from vigiglobe/master 
Fix NPE when partionNumber 0 does not exist.
 2015-03-26 13:25:39 -07:00
Xavier Léauté 2c58e355fd Merge pull request #1239 from metamx/log4j2forRealz 
Exclude log4j from curator dependencies in favor of log4j-1.2-api
 2015-03-26 13:21:13 -07:00
Charles Allen 9cd6c08e96 Exclude log4j from curator dependencies in favor of log4j-1.2-api 2015-03-26 13:05:12 -07:00
Xavier Léauté 47cc11829d Merge pull request #1240 from himanshug/async_query_forwarding_servlet_router_fix 
adding router numMaxThread configuration for ProxyServlet HttpClient executor
 2015-03-26 10:15:44 -07:00
msprunck 942c17a2aa Remove timeline chunk count assumptions. 
* Replace with generic iterables
 2015-03-24 22:40:49 +01:00
Himanshu Gupta b131a4fd3b adding router numMaxThread configuration for ProxyServlet HttpClient executor 2015-03-23 21:58:01 -05:00
nishantmonu51 3b894f9fb9 add ut for TierSelectorStrategies 
fix test
 2015-03-24 08:18:28 +05:30
Xavier Léauté 23528299c9 Merge pull request #1235 from druid-io/comments-update 
Minor update to zk comments to make it more clear what is happening
 2015-03-23 14:02:08 -07:00
fjy a0ce1caaf8 minor update to zk comments to make it more clear what is happening 2015-03-23 11:47:43 -07:00
Himanshu Gupta b5b32dbcb4 in test explicitly setting maxThreads for ProxyServlet to work around https://tickets.puppetlabs.com/browse/TK-152 2015-03-23 12:27:19 -05:00
fjy b389cfe404 [maven-release-plugin] prepare for next development iteration 2015-03-19 12:38:17 -07:00
fjy 60e7d543cc [maven-release-plugin] prepare release druid-0.7.1-rc1 2015-03-19 12:38:13 -07:00
nishantmonu51 21cd75bcf6 improve alert 
skip doing false alert when the coordinator looses leadership
 2015-03-18 09:42:22 +05:30
Fangjin Yang 8d0dfd9af1 Merge pull request #1221 from metamx/AnnounceIdUUID 
Change announcement ID to a UUID instead of Timestamp
 2015-03-17 15:38:53 -07:00
Charles Allen 79b1443cc3 Change announcement ID to a UUID instead of Timestamp 
* Also add a UUIDUtils to the common package
 2015-03-17 15:34:04 -07:00
nishantmonu51 f9821d242f also log author ip address in audit log 2015-03-17 23:15:15 +05:30
cheddar 526a386f50 Merge pull request #1193 from metamx/reduce-test-verbosity 
move test output to file for cleaner build logs
 2015-03-12 15:42:34 -07:00
nishantmonu51 3ec4a30ab5 initial commit 
review comments

more refactoring and cleaning of redundant code

add UT + docs + more refactoring

fixes + review comments

more cleanup

end points to fetch history

review comments

remove unnecessary changes

review comments rename header name

review comments + add test for MetadataRulesManager

review comments docs
 2015-03-12 22:50:29 +05:30
Xavier Léauté fc613771d2 move test output to file for cleaner build logs 
- removes the need for special test log4j2.xml
 2015-03-11 17:56:19 -07:00
Xavier Léauté 9f242ed1ba Merge pull request #1182 from druid-io/better-docs 
Rework the druid docs and fix many mistakes
 2015-03-10 14:23:38 -07:00
fjy 064d65a009 rework the druid docs and fix many mistakes 2015-03-10 14:19:19 -07:00
Xavier Léauté ef842b2eae add test coverage 2015-03-09 14:32:13 -07:00
Xavier Léauté e161021902 Merge pull request #1171 from druid-io/fix-rt-task 
Add more metrics and fail when realtime tasks fail to persist and merge
 2015-03-06 16:42:23 -08:00
fjy 86a0a3cab8 add more metrics and fail when realtime tasks fail to persist and merge 2015-03-06 11:00:34 -08:00
Himanshu Gupta 956361dbbd Refactoring: Moved BaseJettyServerInitializer.java to JettyServerInitUtils.java and changed it from abstract class to a utils class containing static methods 
And, moved jetty stuff to its own package io.druid.server.initialization.jetty
 2015-03-05 19:33:27 -06:00
Himanshu Gupta a8648d8f3d feature to add servlet filters in a druid node via extension modules 2015-03-03 22:35:02 -06:00
Fangjin Yang d8e199a3f5 Merge pull request #1161 from himanshug/zk_ood_updates 
Fix to maintain correctness when out-of-order ZK updates are received
 2015-03-03 14:42:31 -08:00
Himanshu Gupta dda2a62ff5 on receiving ZK ADD/UPDATE events, get latest data from zookeeper instead of taking it from the event which might be stale due to event coming out of order etc 2015-03-03 16:21:56 -06:00
Fangjin Yang 9578304d6f Merge pull request #1130 from housejester/jde-building 
Various improvements to the tutorial and building experience.
 2015-03-02 14:48:32 -08:00
Fangjin Yang e8605c63a9 Merge pull request #1150 from himanshug/broker-parallel-chunk-process 
interval chunk query runner now processes individual chunk in a threadpool
 2015-03-02 13:50:23 -08:00
Himanshu Gupta 29039fd541 interval chunk query runner now processes individual chunk in a thread pool and prints metrics query/time per chunk 2015-03-02 15:45:09 -06:00
Fangjin Yang a76a24742b Merge pull request #1149 from himanshug/broker-get-datasource-update 
updating ClientInfoResource.getDatasource() with a full flag
 2015-03-02 13:28:27 -08:00
Himanshu Gupta a39f4f8bf4 updating ClientInfoResource.getDatasource() with a full flag to return all the served intervals for given data source along with the dimension and metric information 2015-03-02 14:18:52 -06:00
Xavier Léauté 17743b94f3 Cleanup router servlet + tests for gzip proxying 
- Uses method overrides instead of modified Jetty code, now that
  ProxyServlet provides enough method hooks for proper overrides.
  This means we may also benefit from any Jetty ProxyServlet fixes
- Adds test for async proxy servlet to make sure gzip encoding is
  properly proxied.
- Router now proxies POST requests for requests that are not Druid
  queries, by only treating /druid/v2/* endpoints as queries.
 2015-03-02 11:36:15 -08:00
Fangjin Yang c21086b6a8 Merge pull request #1113 from metamx/fix1111 
Make Realtime queries not submit to thread pools for the master merger
 2015-02-27 11:13:25 -08:00
James Estes 9cb5d9bd00 Various improvements to the tutorial and building experience. 
- Improve geo ip download stability
- Retry default geo ip download if corrupt
- Clarify docs on building & where to get artifact
 2015-02-25 18:49:16 -07:00
Xavier Léauté b633111c76 Merge pull request #1156 from druid-io/fix-log 
Print better messages when there are problems with ingestion
 2015-02-25 17:02:46 -08:00
fjy 9a3ed353dc Have a more meaningful message when events are dropped or unparseable 2015-02-25 16:59:46 -08:00
Himanshu Gupta 2192d0cb57 UTs for server 2015-02-25 15:45:58 -08:00
Xavier Léauté 69ba802abb Merge pull request #1109 from metamx/fixAnnouncerTest 
Make the Announcer closing and unannouncing atomic.
 2015-02-25 15:24:18 -08:00
Xavier Léauté aa9dbb773d Merge pull request #1153 from metamx/announcerTestForWaitingFix 
Use forWaiting() in AnnouncerTest
 2015-02-24 21:48:54 -08:00
Charles Allen 29a3522708 Use forWaiting() in AnnouncerTest 
* Helps reduce random failures in unit test on slow systems
 2015-02-24 08:35:07 -08:00
Xavier Léauté b167dcf82c [maven-release-plugin] prepare for next development iteration 2015-02-23 14:28:06 -08:00
Xavier Léauté e81ac2ba43 [maven-release-plugin] prepare release druid-0.7.0 2015-02-23 14:27:58 -08:00
Fangjin Yang 25db9abb7f Merge pull request #1138 from metamx/better-default-hostname 
Better default hostname
 2015-02-18 17:37:34 -08:00
Xavier Léauté 53d2b961c5 default to canonical hostname instead of localhost 2015-02-18 16:44:48 -08:00
fjy 3fab045b2d Go through and fix mistakes in tutorials and docs 2015-02-17 15:21:16 -08:00
cheddar 2a45501393 Merge pull request #1110 from metamx/move-to-sonatype 
Move Druid release artifacts to Sonatype
 2015-02-17 11:54:16 -08:00
extpvaan b42b41cca7 Bind value of column "partitioned" as boolean, as it is defined in the "create table" ddl. 2015-02-17 10:22:10 -08:00
Fangjin Yang 2593864c52 Merge pull request #1126 from gianm/md-insert-or-update-tx 
Use jdbi inTransaction for SQLMetadataConnector's insertOrUpdate.
 2015-02-16 08:23:05 -08:00
Gian Merlino e00de8c19f Use jdbi inTransaction for SQLMetadataConnector's insertOrUpdate. 
This fixes an issue where failed transactions were not properly rolled back.
 2015-02-16 07:42:43 -08:00
Xavier Léauté 78df7f6165 Move Druid release artifacts to Sonatype 
- Switch to using Druid parent POM
- Add required fields for Sonatype
- Common plugin versions and settings have been moved to the parent pom
- Cleanup artifacts and POMs for consistent formatting
- Remove org.hyperic.sigar dependency and update docs to reflect necessary jars to add at runtime when sigar is needed
 2015-02-13 14:26:31 -08:00
fjy d29740ed9f [maven-release-plugin] prepare for next development iteration 2015-02-12 16:16:00 -08:00
fjy 211fd15b7e [maven-release-plugin] prepare release druid-0.7.0-rc3 2015-02-12 16:15:56 -08:00
Fangjin Yang 90bc62eb5c Merge pull request #1108 from metamx/improve-groupby-perf 
Improve groupby by removing conversion to case insensitive row
 2015-02-12 11:45:20 -08:00
Charles Allen bd500fcf4b Make the Announcer closing and unannouncing atomic. 2015-02-11 19:47:04 -08:00
Charles Allen e116600c7f Make Realtime queries not submit to thread pools for the master merger 
Fix https://github.com/druid-io/druid/issues/1111
 2015-02-11 15:47:40 -08:00
nishantmonu51 15cf432b74 remove conversion to case insensitive row 
this is not required after death to casing in 0.7
 2015-02-11 19:40:36 +05:30
Charles Allen d51b37cfd5 Remove QueryResource's ObjectMapper::copy() in favor of using the injected ObjectMapper directly 
* Workaround for https://github.com/FasterXML/jackson-databind/issues/696
 2015-02-10 16:50:23 -08:00
Xavier Léauté 31e248736d Merge pull request #1106 from druid-io/update-http-client 
Update http client to 1.0.0
 2015-02-10 13:37:27 -08:00
fjy 708759e1e0 Update http-client to 1.0.0 2015-02-10 13:36:47 -08:00
Fangjin Yang 1418cbb0e9 Merge pull request #1103 from metamx/fix-request-time 
fix change in request/time meaning introduced by d2c729a, fixes #1102
 2015-02-09 17:23:05 -08:00
Xavier Léauté 2db3144f04 fix change in request/time meaning introduced by d2c729a, fixed #1102 2015-02-09 16:41:30 -08:00
Fangjin Yang 040e771da2 Merge pull request #1099 from metamx/console_cleanup 
console cleanup
 2015-02-08 09:06:05 -08:00
Tarek Rached f47f0d51c7 removed unecessary logging 
fixed simplifyInterval logic
 2015-02-06 21:13:25 -08:00
Tarek Rached a0fc20fc09 better regex for durations 2015-02-06 17:02:31 -08:00
Tarek Rached dae91e5894 fixed links back to old console 2015-02-04 14:21:46 -08:00
fjy 1f12c5b2f1 [maven-release-plugin] prepare for next development iteration 2015-02-03 12:06:49 -08:00
fjy e82d431be7 [maven-release-plugin] prepare release druid-0.7.0-rc2 2015-02-03 12:06:41 -08:00
Xavier Léauté 4eff269536 Merge pull request #1079 from druid-io/cleanup-deps 
Remove non friendly dependencies from Druid
 2015-02-03 11:56:41 -08:00
Xavier Léauté ffb180e29d Merge pull request #1080 from metamx/fix-cost-balancer-strategy 
fix integer value overflow and add test
 2015-02-03 11:45:03 -08:00
nishantmonu51 6da6b9eeda fix integer value overflow and add test 2015-02-03 11:39:30 -08:00
fjy 3e5d338c8e Remove non friendly dependencies from Druid 2015-02-03 11:36:08 -08:00
Eric Tschetter 42eba986ce Towards consistent null handling 
This commit also includes
1) the addition of a context parameter on timeseries queries that allows it to ignore empty buckets instead of generating results for them
2) A cleanup of an unused method on an interface
 2015-02-02 12:53:07 -08:00
Fangjin Yang 92e616de11 Merge pull request #1077 from metamx/remove-unused-imports 
remove unused imports
 2015-02-02 10:45:27 -08:00
nishantmonu51 ba932bb1f2 remove unused imports 2015-02-02 21:53:39 +05:30
fjy d05032b98a towards a community led druid 2015-01-31 20:57:36 -08:00
Fangjin Yang b94b24c46e Merge pull request #1072 from metamx/common-aws-config 
Common AWS module + consistent credential chain for both S3 and auto-scaling
 2015-01-30 14:41:24 -08:00
Fangjin Yang 14837d7808 Merge pull request #1075 from metamx/fix-test-timeout 
increase test timeouts on DruidClusterBridgeTest
 2015-01-30 14:31:32 -08:00
Xavier Léauté cd7e768640 increase test timeouts on DruidClusterBridgeTest 2015-01-30 13:18:35 -08:00
fjy bc1405bee0 fix worker status path announcement with indexer zk config 2015-01-30 12:26:08 -08:00
Xavier Léauté f00872c41b move common AWS related classes into a separate module 2015-01-29 13:55:49 -08:00
Fangjin Yang 2d96b62e6c Merge pull request #1066 from metamx/fix-background-caching-ordering 
fix background caching not preserving result order
 2015-01-28 06:09:17 +08:00
Xavier Léauté 59c1907322 add test for out-of-order background-caching execution 2015-01-27 13:38:08 -08:00
fjy cd9ba1f852 fix serde bug 2015-01-27 18:47:36 +08:00
Xavier Léauté b334d7973d fix background caching not preserving result order 2015-01-26 09:59:18 -08:00
Xavier Léauté 26c5c2193f fix BatchServerInventoryViewTest test timeout 2015-01-20 17:14:57 -08:00
fjy 1f94de22c6 [maven-release-plugin] prepare for next development iteration 2015-01-20 14:23:55 -08:00
fjy 17476edc31 [maven-release-plugin] prepare release druid-0.7.0-rc1 2015-01-20 14:23:51 -08:00
Xavier Léauté 5a2030c49b Merge pull request #1059 from druid-io/remove-test 
Remove test rejection policy - it has no place anywhere
 2015-01-20 13:39:46 -08:00
fjy 1ff5829d6d remove test rejection policy 2015-01-20 13:22:30 -08:00
Charles Allen 3d27747f7e Upgrade to log4j2 
Default behavior is as before.
Added documentation for how to enable synchronous logging for select chatty classes:
* io.druid.client.ServerInventoryView
* io.druid.client.BatchServerInventoryView
* io.druid.curator.inventory.CuratorInventoryManager
* com.metamx.http.client.pool.ChannelResourceFactory
 2015-01-20 12:35:18 -08:00
Fangjin Yang 91a79dbf95 Merge pull request #1031 from metamx/ingestmetadata-query 
DataSourceMetadata query
 2015-01-19 21:55:35 -08:00
Fangjin Yang 56bad2114a Merge pull request #1054 from metamx/increase-test-wait-time 
use waiting multiple when waiting in AnnouncerTest
 2015-01-19 11:03:35 -08:00
Xavier Léauté b5f0d119f1 use waiting multiple when waiting in AnnouncerTest 2015-01-19 10:19:13 -08:00
fjy 52f65b1273 Fix a small bug where an NPE can occur if a closeable is not present 2015-01-17 15:23:01 -08:00
fjy 38c155e143 Merge branch 'master' of https://github.com/druid-io/druid 2015-01-16 13:14:49 -08:00
Fangjin Yang b4041c13e5 Merge pull request #1029 from metamx/fixChainedExecutionQueryRunnerTest 
Address spurious test failures
 2015-01-16 13:08:32 -08:00
Xavier Léauté 6bc6e71b0d Add messages to catch spurious test failures 2015-01-15 16:49:43 -08:00
fjy f7ea308b25 fix minor error output problem 2015-01-15 15:57:45 -08:00
Fangjin Yang 5bfcc43377 Merge pull request #1008 from metamx/stringConversionJavaUtilUpdate 
Update all String conversions to and from byte[] to use the java-util StringUtils functions
 2015-01-15 13:50:27 -08:00
nishantmonu51 c7452b75f6 Merge branch 'master' into ingestmetadata-query 2015-01-15 18:00:31 +05:30
fjy 021a7de759 Add retry capabilities for realtime logic 2015-01-13 14:30:49 -08:00
fjy 84cc32ba32 Add more logging for when ingestion may be throttled 2015-01-08 14:51:46 -08:00
Charles Allen c1295202ff Allow background caching run after the futures are finished 2015-01-07 13:23:34 -08:00
Charles Allen f6fbb733b8 Added a few places where tests were using Object instead of Module 2015-01-05 13:47:25 -08:00
Fangjin Yang 852e863425 Merge pull request #981 from druid-io/strictModuleTyping 
Use Module instead of generic Object in Guice related items
 2015-01-05 12:43:20 -08:00
Fangjin Yang ab0ddc0d7c Merge pull request #998 from drcrallen/zkConfigJackson 
Change zk configs to use Jackson injection instead of Skife
 2015-01-05 12:42:40 -08:00
Charles Allen b1b5c9099e Update all String conversions to and from byte[] to use the java-util StringUtils functions 
* Speedup of GroupBy with javaScript filters by ~10%
* Requires https://github.com/metamx/java-util/pull/15
 2015-01-05 11:22:32 -08:00
Charles Allen 81fdc791e0 Add memory hinting and concurrency helpers in CachingClusteredClient 2015-01-02 15:54:26 -08:00
Charles Allen 65286a24e0 Change zk configs to use Jackson injection instead of Skife 
* Also added generic config testing class JsonConfigTesterBase
 2014-12-29 10:36:12 -08:00
Xavier Léauté 574ab3017c fix test timeout 2014-12-23 16:25:38 -08:00
Fangjin Yang bf4b52384c Merge pull request #988 from drcrallen/queryErrorFixer 
Fix NPE in QueryResource on bad query
 2014-12-22 23:16:59 -08:00
Charles Allen 458236c8fd Add simple unit tests for QueryResource 2014-12-22 21:53:58 -08:00
Charles Allen 4338af0e3f Fix NPE in QueryResource on bad query 2014-12-22 12:59:47 -08:00
Gian Merlino badc09e85c Disable checkGzExists in the default GzipFilter. 
It is unhelpful because we don't actually have any precomputed .gz resources,
and checking for them inside jars is expensive.
 2014-12-22 12:45:48 -05:00
Charles Allen 7c8d4a7433 Use Module instead of generic Object in Guice related items 2014-12-19 10:54:06 -08:00
Charles Allen a0b2a63d7f Fix deadlock in DirectDruidClient 2014-12-16 18:34:30 -08:00
fjy e9238161cd fix broken test 2014-12-15 18:12:44 -08:00
Charles Allen 242edd14e5 Add deprecated smile media type for backwards compatability during upgrades 2014-12-15 18:00:10 -08:00
Charles Allen a520835972 Merge pull request #957 from metamx/fix-gzip-encoding 
fix gzip compression
 2014-12-15 13:20:36 -08:00
Xavier Léauté e166260f80 more inventory logs 2014-12-15 13:19:47 -08:00
fjy 9c6e32fc3f address cr 2014-12-15 13:06:36 -08:00
fjy 61101bee6c more logs for CIM 2014-12-15 12:57:40 -08:00
Fangjin Yang b3fe91bb50 Merge pull request #830 from metamx/union-merge-on-historical 
Union merge on historical
 2014-12-15 13:36:47 -07:00
Fangjin Yang 31cda0b830 Merge pull request #958 from metamx/fix-router-service-discovery 
fix router discovering brokers with default names
 2014-12-15 13:35:39 -07:00
Xavier Léauté 77bf5da317 fix typo 2014-12-15 10:07:51 -08:00
Charles Allen 3c33118f3a Merge pull request #962 from metamx/add-broker-loadstatus 
add loadstatus endpoint for serverView status
 2014-12-12 17:13:48 -08:00
Charles Allen ca8300a461 Replace AppendableByteArrayInputStream in DirectDruidClient 
* Replace with SequenceInputStream fueled by an enumeration of ChannelBufferInputStream which directly wrap the response context ChannelBuffer
* Added zero-length byte array when DirectDruidClient is done. This prevents an odd race condition on `done`.
 2014-12-12 14:01:13 -08:00
Xavier Léauté 4b5282d224 add loadstatus endpoint for serverView status 2014-12-12 13:18:31 -08:00
Xavier Léauté 352480ac77 make method protected to prevent accidental usage 
- add documentation as to why
 2014-12-11 15:20:36 -08:00
Xavier Léauté 3abf348d2c fix router discovering brokers with default names 2014-12-11 12:16:44 -08:00
nishantmonu51 7ad03087c0 Merge branch 'master' into ingestmetadata-query 2014-12-11 16:54:38 +05:30
nishantmonu51 32b4f55b8a review comments refactoring 2014-12-11 16:33:14 +05:30
Xavier Léauté eb661d3130 /status replaces /health 2014-12-10 17:35:05 -08:00
Xavier Léauté 0de56efe5c fix gzip compression 
- fixes compression not working for POST requests
- router now forwards raw bytes instead of decompressing
- cleanded up router servlet initialization
- add test for gzip on get and post methods
- use Jersey annotation when possible in QueryResource
 2014-12-10 17:30:17 -08:00
Xavier Léauté 7878d391bd use first available port for JettyTest 2014-12-10 12:53:11 -08:00
Charles Allen 6f6cc463b0 Merge pull request #950 from metamx/fix-bysegment-serde 
Fix bySegment metrics de-serialization
 2014-12-10 12:42:40 -08:00
Xavier Léauté 5f3f4e0a60 prevent /status from redirect, fixes #953 2014-12-10 11:32:22 -08:00
Xavier Léauté 7853e801d5 fixes for bySegment queries and caching 
- prevent bySegment queries from getting cached
- avoid computing cache keys if not populating or using cache
- avoid computing cache data when not populating cache
 2014-12-10 10:57:48 -08:00
nishantmonu51 3763357f6e Ingest metadata query implementation 2014-12-10 19:44:00 +05:30
Xavier Léauté c44242f0af add comment about de-serialing bySegment 2014-12-09 17:02:24 -08:00
Xavier Léauté a7da9aed9f fix metrics de-serialization for bySegment results 2014-12-09 17:02:23 -08:00
fjy 7620dfbd1a fix broken ut 2014-12-09 15:43:07 -08:00
Fangjin Yang d6d3ec6846 Merge pull request #948 from metamx/ingestion-docs 
Redocumenting ingestion
 2014-12-09 15:30:03 -07:00
nishantmonu51 1a1b0e6f23 merge from master and review comments 2014-12-09 13:16:45 +05:30
fjy 28b72a69ad redocumenting ingestion 2014-12-08 16:15:46 -08:00
Fangjin Yang 623443a145 Merge pull request #946 from metamx/MoreMediaTypes 
Changed all "application/json" to MediaType.APPLICATION_JSON  except for in druid.js
 2014-12-08 15:24:21 -07:00
Charles Allen a0f9f9877e Changed all "application/json" to MediaType.APPLICATION_JSON except for in druid.js 2014-12-08 14:21:49 -08:00
xvrl 1392e2731f Merge pull request #936 from metamx/cachingRunnerImprovements 
General Caching Query Runners cleanup (40% query time reduction for HLL)
 2014-12-08 14:07:52 -08:00
Charles Allen 7b65f0635d General Caching Query Runners cleanup 
* Add type strictness to CachingClusteredClient.
* Add background caching to CachingClusteredClient. Gives between 0% and 5% query speed increase.
* Add @BackgroundCaching annotation for injected ExecutorService items
* Add `numBackgroundThreads' configuration options to CacheConfig (default 0 aka same thread legacy behavior)
* Add unit tests for CacheConfig
* Add an abstract caching query runner class, currently it doesn't do anything exceppt simply make the two caching queries distinct.
* Add caching to CachingQueryRunner. Gives up to a WHOPPING 40% reduction in query time on HLL queries
* Updated docs with more info on cache settings.
 2014-12-08 13:29:32 -08:00
Fangjin Yang c48839a0a0 Merge pull request #945 from metamx/MediaTypeFixes 
Added checks for backwards compat on MediaTypes
 2014-12-08 14:11:58 -07:00
Charles Allen 43041ed4ba Added checks for backwards compat on MediaTypes 2014-12-08 13:11:00 -08:00
Fangjin Yang eb233fe50e Merge pull request #921 from metamx/onheap-incremental-index 
separate implementations of  incremental index
 2014-12-08 13:59:08 -07:00
Charles Allen 03f7fae3e8 Changed references to MediaType to pull from javax.ws.rs.core.MediaType or com.fasterxml.jackson.jaxrs.smile.MediaType 2014-12-08 12:39:36 -08:00
Xavier Léauté c88de97370 Merge branch 'master' into onheap-incremental-index 
Conflicts:
	server/src/main/java/io/druid/segment/realtime/RealtimeManager.java
 2014-12-08 11:48:21 -08:00
Fangjin Yang 3a7f571aeb Merge pull request #942 from metamx/fix-tiered-selector-naming 
fix tiered selector service name usage
 2014-12-08 12:09:45 -07:00
nishantmonu51 ef86dec660 remove unused but required parameter 2014-12-08 19:25:46 +05:30
nishantmonu51 4883f5526b fix tiered selector service name usage 2014-12-08 19:22:27 +05:30
Fangjin Yang c255c16c1a Merge pull request #894 from zhaown/fix-query-realtime-with-multi-partition 
bugfix for #887
 2014-12-07 09:49:31 -07:00
Xavier Léauté 629f6bc759 moved methods to java-util 2014-12-05 15:53:23 -08:00
Weinan Zhao 3119770ce9 Reformatting using intellij_formatting.jar 2014-12-05 14:21:24 +08:00
nishantmonu51 6e03a6245f Merge branch 'master' into onheap-incremental-index 2014-12-05 10:40:28 +05:30
Xavier Léauté d4a1752c5d add code to release ByteBuffers 2014-12-04 16:18:55 -08:00
Xavier Léauté 7cd45a6e1f IncrementalIndex throws exception if limit exceeded 
- For now uses a hardcoded ratio of aggregator to timeanddim buffer sizes
- canAppendRow is a workaround for realtime index since the
Firehose currently does not have a way of rolling back the last event in
case of error
- canAppendRow needs a fudge factor; there is a race between checking
if we can add a row and actually adding a row, because of the way MapDB
reports its size.
 2014-12-04 14:38:16 -08:00
Gian Merlino 20a7239ffd Replace google-http-client imports with real guava imports. 2014-12-04 10:57:57 -08:00
Gian Merlino b48fc51ed1 SSL-capable ServiceEmitter. 2014-12-04 10:57:56 -08:00
Xavier Léauté 0c521e0a77 update joda-time and fix min/max instant 2014-12-04 10:57:56 -08:00
fjy 187126dd4b fix broken unit tests 2014-12-04 10:49:36 -08:00
Fangjin Yang 332e4a4d1c Merge pull request #902 from metamx/cache-config 
Commonalize the cache config and change default to disable cache
 2014-12-04 11:30:24 -07:00
xvrl 2681da4420 Merge pull request #929 from metamx/google-cleanup 
Replace google-http-client imports with real guava imports.
 2014-12-03 11:50:19 -08:00
Gian Merlino d388a8fe89 Replace google-http-client imports with real guava imports. 2014-12-03 10:52:57 -08:00
Gian Merlino 03ae70ae54 SSL-capable ServiceEmitter. 2014-12-03 10:43:24 -08:00
nishantmonu51 4dc0fdba8a consider mapped size in limit calculation & review comments 2014-12-03 23:47:30 +05:30
nishantmonu51 da8bd7836b Introduce buffer size 2014-12-03 16:28:22 +05:30
Xavier Léauté a79389a9e5 update joda-time and fix min/max instant 2014-12-02 17:27:22 -08:00
nishantmonu51 eac776f1a7 tests passing with on heap incremental index 2014-12-02 22:29:28 +05:30
Xavier Léauté 59542c41f8 fix port not set in DruidNode 2014-12-01 14:37:28 -08:00
Charles Allen 7ed5f995cb Modify InitializationTest to use a port of 0 instead of -1 2014-12-01 13:48:49 -08:00
Xavier Léauté 36c9211791 remove unused imports 2014-12-01 13:34:58 -08:00
Xavier Léauté 839e288ed5 request proper type from DBI instead of casting 2014-12-01 13:27:52 -08:00
Fangjin Yang 646e184ca6 Merge pull request #918 from metamx/fix-broker-timeouts 
fix query timeouts not being raised by broker
 2014-12-01 13:58:04 -07:00
Xavier Léauté 445af2dc5f use guava methods instead 2014-12-01 12:53:20 -08:00
Xavier Léauté eb5525f9b4 add support for ipv6 hostnames 2014-12-01 12:27:42 -08:00
Xavier Léauté d23fd1e1ab make host+port more explicit 
- document the behavior for node host/port initialization
- throw exception if settings make no sense
- fixes announcement for nodes without host/port defaults
- makes code clearer as to when host vs. host+port are used
 2014-11-26 22:03:25 -08:00
Xavier Léauté 4dccfda61d fix query timeouts not being raised by broker 2014-11-26 21:42:07 -08:00
Xavier Léauté 523f9c53cd update curator and fix test 
curator is more strict now
 2014-11-26 14:41:08 -08:00
Xavier Léauté d4bda82916 remove unused imports 2014-11-26 14:41:08 -08:00
nishantmonu51 76891d450e update spymemcached library 
update spymemcached library
for fixes mainly related to -
1) memory leaks on reconnects
2) better handling of network issues
 2014-11-26 19:33:52 +05:30
Xavier Léauté 9fb93a3a79 fix smile exception on error + content-type 2014-11-25 11:30:17 -08:00
Weinan Zhao bcc5f86723 Modify according to fjy's review, add reformatting according to eaclipse_formatting.xml 2014-11-26 00:14:40 +08:00
fjy 083a012aa2 Commonalize the cache config and change default to disable cache 2014-11-24 14:54:11 -08:00
xvrl 829c254bac Merge pull request #898 from metamx/refactor-examples 
Refactor structure for examples and extensions
 2014-11-24 14:02:34 -08:00
fjy f42b1a4181 fix broken ut 2014-11-24 11:42:24 -08:00
xvrl 37e5678870 Merge pull request #899 from metamx/fix-log 
fix crazy amounts of logging when coordinator loses leadership
 2014-11-24 11:01:44 -08:00
fjy 13cae41f6c Merge branch 'master' into refactor-examples 2014-11-24 11:00:26 -08:00
xvrl 9ced097abd Merge pull request #895 from metamx/fix-interval-retry 
A set of fixes to retry the query for missing segments in the timeline
 2014-11-24 10:23:02 -08:00
fjy f518c53008 address cr 2014-11-24 10:12:29 -08:00
fjy 10d45ab9cf address code review 2014-11-21 16:48:45 -08:00
fjy f9db3a1fc6 fix crazy amounts of logging when coordinator loses leadership 2014-11-21 15:46:14 -08:00
fjy 8ee4d12562 Refactor structure for examples and extensions 2014-11-21 14:45:24 -08:00
fjy 580e1172c1 move IndexTask to use hashed partition; fixes #815 2014-11-21 11:15:25 -08:00
Fangjin Yang 780e4a27f7 Merge pull request #868 from metamx/fix-bad-test 
fix non-determininistic reference counting segment test
 2014-11-20 15:25:57 -07:00
fjy e044598669 make wait timeout a bit longer 2014-11-20 14:25:49 -08:00
fjy 0d275c7a53 remove another wait 2014-11-20 14:22:16 -08:00
fjy 47f5c1bd0a fix retry interval is stupid 2014-11-20 12:50:56 -08:00
fjy 3d9d989a9f A set of fixes to retry the query for missing intervals in the timeline 2014-11-20 12:04:37 -08:00
Weinan Zhao 5096cbc5e4 fix bug for #887, that is cannot query multi partitions of one datasource on realtime node. 2014-11-20 16:36:57 +08:00
fjy fdeab0c6af make Druid case sensitive 2014-11-19 14:27:31 -08:00
Fangjin Yang 590d31799e Merge pull request #876 from metamx/remove-backwards-compatible 
Remove backwards compatible
 2014-11-19 14:33:14 -07:00
xvrl a96eaeb036 Merge pull request #882 from metamx/now_with_OPEN_SOURCE 
Added src jar build to maven poms and re-formatted to conform to style guidelines.
 2014-11-18 13:00:04 -08:00
Fangjin Yang 87ea6fd377 Merge pull request #878 from metamx/default-extension-version 
use Druid version as the default version for extensions
 2014-11-18 13:56:54 -07:00
Xavier Léauté a41f179cc0 fix output content-type 2014-11-18 11:08:44 -08:00
Charles Allen 8819a1b9d5 Add Smile annotation to BrokerServerView 2014-11-18 10:18:10 -08:00
Charles Allen dc66e1708e Added src jar build to maven poms and re-formatted to conform to style guidelines. 2014-11-18 09:05:30 -08:00
Xavier Léauté d914afe1cd make defaultVersion configurable for non-jar testing 2014-11-17 13:54:32 -08:00
Xavier Léauté 0ca23d7d0b use Druid version as the default version for extensions 2014-11-17 13:36:37 -08:00
nishantmonu51 0c2d06475d merge from master 2014-11-17 19:19:18 +05:30
nishantmonu51 cbffe3c648 merge from master and resolve conflicts 2014-11-17 18:07:08 +05:30
Fangjin Yang 0b45942cff Merge pull request #867 from metamx/avoid-abandoned-sinks 
RealtimePlumber: Bail out of persist-n-merge of abandoned sinks (fixes #812)
 2014-11-14 18:35:00 -07:00
xvrl e1e171ad20 Merge pull request #865 from metamx/fix-retry-qr 
Fix a couple of bugs with retry query runner after testing it locally
 2014-11-14 13:33:51 -08:00
xvrl a4fc64ca3f Merge pull request #856 from metamx/druid-845 
Fix query by segment
 2014-11-14 13:10:54 -08:00
fjy bbc079b880 fix retry to actually return correct sequences 2014-11-14 12:10:04 -08:00
Xavier Léauté 07c4923bf8 add test for out of order merging 2014-11-14 11:17:50 -08:00
Xavier Léauté 6e8ac89737 refactor sequence merging method to simplify testing 2014-11-14 11:17:50 -08:00
Xavier Léauté 716cc2cff2 fix results ordering 2014-11-13 14:59:56 -08:00
fjy 313d81ca2f fix non-determininistic reference counting segment test 2014-11-13 13:39:56 -08:00
Gian Merlino bf22524d63 RealtimePlumber: Bail out of persist-n-merge of abandoned sinks (fixes #812). 2014-11-13 12:59:49 -08:00
fjy 0d6816a037 Fix a couple of bugs with retry query runner after testing it locally 2014-11-13 11:53:29 -08:00
Fangjin Yang 9d661d6d36 Merge pull request #854 from metamx/serverInventoryViewJson 
Better logging in JSON parse errors in ServerInventoryView
 2014-11-13 11:56:59 -07:00
Charles Allen 9a9238a801 Merge remote-tracking branch 'origin/master' into druid-845 2014-11-13 10:04:56 -08:00
Charles Allen 3cdcf8736f Changed downloadStartMarker in OmniSegmentLoader to simply use its .delete() method 2014-11-13 09:52:41 -08:00
Xavier Léauté ba442b9882 fix default FilteredServerView + batch annoucement 2014-11-12 23:17:16 -08:00
fjy 3ef21bfc66 fix leak with files not cleaning up correctly 2014-11-12 22:29:50 -08:00
Charles Allen 31fed7d329 Fix query by segment 
* Changed topN queries to use joda Interval instead of string values
* topN by segment now implements BySegmentResultValue<Result<TopNResultValue>> instead of BySegmentResultValue<TopNResultValue>
* Added a unit test which failed uner the prior implementation.
 2014-11-12 21:20:59 -08:00
Charles Allen a8103fb02d Better logging in JSON parse errors in ServerInventoryView 2014-11-12 16:52:15 -08:00
xvrl 10b7ca9fa9 Merge pull request #849 from metamx/default-coordinator-service-name 
Default service names
 2014-11-12 15:06:02 -08:00
Xavier Léauté 3a9def7fb1 add broker default and define constants 2014-11-12 14:53:53 -08:00
Xavier Léauté 4ac1aaf90e Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	processing/src/main/java/io/druid/segment/QueryableIndexStorageAdapter.java
 2014-11-12 14:08:51 -08:00
Xavier Léauté dbab127a6d default zk host to localhost 2014-11-12 13:56:07 -08:00
Fangjin Yang 58260165ad Merge pull request #842 from metamx/fix-timestamp-column-exclusion 
fix dimension exclusions for timestamp and aggs
 2014-11-12 09:44:50 -07:00
Charles Allen 847c01a045 Merge pull request #844 from metamx/derby-dependencies-fix 
specify versions in parent pom
 2014-11-12 08:28:55 -08:00
nishantmonu51 c9bcdd5dd0 specify versions in parent pom 
- move version declarations to parent pom
- remove extra dependency
 2014-11-12 10:31:11 +05:30
Xavier Léauté 69146736a8 fix dimension exclusions for timestamp and aggs 
- also added tests
 2014-11-11 20:56:17 -08:00
Xavier Léauté 87b72e8822 fix non-deterministic test 2014-11-11 20:26:13 -08:00
Charles Allen a89b539b4f Merge pull request #823 from metamx/roaring 
Configurable bitmap indexes: roaring and concise
 2014-11-11 17:26:38 -08:00
Xavier Léauté 59f093882d fix Derby instantiation 2014-11-10 16:55:53 -08:00
fjy 8cbd21af90 fix defaults 2014-11-10 15:40:57 -08:00
fjy 98d05e63bf make derby instance only on coordinator 2014-11-10 15:34:49 -08:00
fjy 6188315293 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/query/search/SearchQueryRunnerTest.java
 2014-11-10 14:52:10 -08:00
nishantmonu51 42a61290a8 specify default service name for coordinator 2014-11-10 23:36:01 +05:30
fjy 3c21f62afd a whole bunch of fixes and some ugly conversion code 2014-11-07 16:03:22 -08:00
Xavier Léauté 0498df25df override metadata storage injection in CliHadoopIndexer 2014-11-07 13:44:56 -08:00
Fangjin Yang 2336e6c167 Merge pull request #758 from metamx/jisoo-metadata 
make metadata storage pluggable
 2014-11-07 11:30:11 -07:00
nishantmonu51 5be801356a fix query/wait emitted twice 2014-11-07 23:32:36 +05:30
nishantmonu51 fd8eb7742b handle union query on realtime node 2014-11-07 23:27:50 +05:30
nishantmonu51 8bebb24fd5 Union Queries move merge to historical 2014-11-07 18:05:16 +05:30
Xavier Léauté d8bab70647 provide proper defaults for indexing service name 2014-11-06 18:16:01 -08:00
Xavier Léauté a5f98be457 remove unused code 2014-11-06 17:18:40 -08:00
Xavier Léauté f69f3cfc5a run MetadataSegmentManagerTest against Derby 2014-11-06 17:08:54 -08:00
Xavier Léauté ac753ed0f0 always use validation query 2014-11-06 17:08:01 -08:00
Xavier Léauté 350bb09605 refactor sql storage to abstract task storage 2014-11-05 17:19:37 -08:00
Xavier Léauté 253c0a70b3 simplify interface + more tests 2014-11-04 15:03:13 -08:00
Xavier Léauté b453922319 add tests for task and task status storage 2014-11-04 14:35:39 -08:00
Xavier Léauté adb4fec264 properly serialize task timestamps, fixes #819 2014-11-04 11:06:34 -08:00
xvrl 2c62b9e6e3 Merge pull request #821 from metamx/minor-fix 
Some minor fixes in docs and change jetty defaults
 2014-11-04 10:18:57 -08:00
fjy c6d555e1f3 better default 2014-11-04 10:18:27 -08:00
fjy 42cf20c9f1 some minor fixes in docs and change jetty defaults 2014-11-04 10:08:50 -08:00
Xavier Léauté f56d026d7d fix realtime module + cleaner metadata abstraction 
- better metadata abstractions
- more db->metadata renaming
 2014-11-03 16:40:23 -08:00
Xavier Léauté 3468163dcd remove unused imports 2014-10-31 16:25:52 -07:00
Xavier Léauté cad4d22f35 better derby test database 2014-10-31 16:25:36 -07:00
Xavier Léauté 80c72eb486 minor test adjustments 2014-10-31 15:44:09 -07:00
Xavier Léauté 1872b8f979 make it easier to test 2014-10-31 14:49:07 -07:00
Xavier Léauté 0bd9ac1154 default to Derby + remove DBI injection 
- default to Derby, but still fail if the given
  metadata storage type does not exit
- use SQLMetadataConnector injection instead of DBI injection
  to simplify code
 2014-10-31 14:37:37 -07:00
jisookim0513 6d501008b8 moved MetadataConnector test to metadata package 2014-10-31 13:13:19 -07:00
jisookim0513 a6016d70fc added deleted classes back 2014-10-31 13:10:09 -07:00
jisookim0513 01f2b3482c merging 2014-10-31 12:52:53 -07:00
jisookim0513 df7566701d fixed syntax error in creating index statements; added a test for MetadataConnector 2014-10-31 12:52:15 -07:00
Xavier Léauté cbe6ea67ee more druid.db -> druid.metadata.storage and allow multiple storage extensions loaded at once 2014-10-31 12:03:04 -07:00
Xavier Léauté 97a2f5af4a rename db->metadata 2014-10-31 10:54:33 -07:00
Xavier Léauté 9c06db021f rename db->metadata postgres->postgresql 2014-10-31 10:30:27 -07:00
Xavier Léauté f77b810f36 fix imports 2014-10-31 10:01:58 -07:00
Xavier Léauté 9fea34be4b simplify connectors even further 2014-10-31 09:59:34 -07:00
Xavier Léauté df0af894f4 fix factory binding + remove unused imports 2014-10-30 22:35:37 -07:00
Xavier Léauté fb4d41cedb make the injection gods happy 2014-10-30 21:16:36 -07:00
Xavier Léauté 377151beda better abstraction for metadatastorage 2014-10-30 18:23:35 -07:00
Xavier Léauté 3cc1b2e690 remove mysql dependencies + fix jdbc class loading 2014-10-29 17:30:43 -07:00
Xavier Léauté c2727dd063 better type 2014-10-29 14:22:00 -07:00
Xavier Léauté 1f171a2b86 workaround for Derby requiring batch statements 
+ make sure payloads are always written as bytes
 2014-10-29 13:59:37 -07:00
Xavier Léauté 55d5f1f618 remove commented out code 2014-10-29 12:55:03 -07:00
Xavier Léauté 7be252599e reduce memory usage when pulling segments 2014-10-29 12:43:21 -07:00
Xavier Léauté 68b9436199 optimize imports 2014-10-29 12:07:21 -07:00
Xavier Léauté e8b4640875 remove duplicate code in table creation 2014-10-29 11:47:29 -07:00
Xavier Léauté 123296b767 add documentation 2014-10-29 11:31:25 -07:00
Xavier Léauté 54eb434a0b remove Derby specific code in favor of better generic JDBI code 2014-10-29 01:09:05 -07:00
Xavier Léauté ec70ecc5fd make tests pass with Java 7 as well 2014-10-28 23:17:30 -07:00
Xavier Léauté 92afd10947 simplify SQL metadata storage 2014-10-28 18:12:30 -07:00
Xavier Léauté 575919ec6c provide a little more involved example 2014-10-27 19:18:09 -07:00
jisookim0513 aa754b86e8 build success! 2014-10-24 11:28:42 -07:00
fjy bef74104d9 merge with 0.7.x and resolve any conflicts 2014-10-23 17:24:06 -07:00
fjy 3b29e77866 [maven-release-plugin] prepare for next development iteration 2014-10-22 16:25:32 -07:00
fjy dcab2997f2 [maven-release-plugin] prepare release druid-0.6.160 2014-10-22 16:25:27 -07:00
jisookim0513 02e79d6b15 attempted to solve merge-conflict; IncrementalIndex has unresolved classes after updates - needs to be fixed 2014-10-22 00:18:17 -07:00
jisookim0513 197878ae75 changed SQLMetadataConnector to a concrete class 2014-10-21 23:39:58 -07:00
jisookim0513 a0d9944fa6 changed SQLMetadataConnector to a concrete class 2014-10-21 23:38:26 -07:00
Fangjin Yang 3d8f33d4ad Merge pull request #801 from metamx/fix-unordered-merging 
fix ordering for partitions with same start time, fixes #796
 2014-10-21 16:47:31 -06:00
Xavier Léauté 8735272ce3 fix file casing 2014-10-21 15:36:49 -07:00
Xavier Léauté fd7d3e5ace make it work with Java 8 2014-10-21 15:33:28 -07:00
Xavier Léauté 9d5e65913d fix serde 2014-10-21 15:05:08 -07:00
jisookim0513 37979282fe enabled ansi-quote in mysql; insert statement should now work 2014-10-21 00:09:19 -07:00
Xavier Léauté 23a59db566 fix ordering for partitions with same start time, fixes #796 
Partitions with same start time may produce results out of order,
           especially in the presence of data gaps.

Results for those segments must be re-ordered prior to being merged
 2014-10-20 17:22:23 -07:00
Xavier Léauté 38cb73e0cc javacript tiered broker selector strategy 2014-10-20 14:12:30 -07:00
jisookim0513 7d5c5f2083 fixed createTable; fixed miscellaneous stuff; added DerbyMetadataRuleManagerProvider 2014-10-17 00:10:36 -07:00
nishantmonu51 f4a97aebbc fix rollup for hashed partitions 
truncate timestamp while calculating the partitionNumber
 2014-10-15 22:32:56 +05:30
nishantmonu51 bce388fb27 merge changes from 0.7.x branch 2014-10-14 18:46:02 +05:30
nishantmonu51 b5d66381f3 more cleanup 2014-10-14 18:32:40 +05:30
jisookim0513 76304d6b7b made modifications according to comments; need to fix some more 2014-10-13 23:56:09 -07:00
fjy a4c8f04409 [maven-release-plugin] prepare for next development iteration 2014-10-13 12:50:45 -07:00
fjy 7fd1747ffa [maven-release-plugin] prepare release druid-0.6.159 2014-10-13 12:50:41 -07:00
nishantmonu51 035b4c1af4 more cleanup 2014-10-14 00:57:15 +05:30
nishantmonu51 454acd3f5a remove backwards compatible code 
1) remove backwards compatible and deprecated code
2) make hashed partitions spec default
 2014-10-13 19:30:44 +05:30
jisookim0513 0dda8975e7 deleted DerbyMetadataRule/SegmentManagerProvider and replaced them with SQL ones 2014-10-10 16:54:33 -07:00
jisookim0513 66277ddf2b deleted dbSegmentPublisher 2014-10-10 12:41:35 -07:00
fjy d0fbd5f729 [maven-release-plugin] prepare for next development iteration 2014-10-09 14:24:53 -07:00
fjy dada60a301 [maven-release-plugin] prepare release druid-0.6.158 2014-10-09 14:24:48 -07:00
Xavier Léauté 290107ce7b remove unused jsonspec content type 2014-10-09 10:26:47 -07:00
fjy 8a4b32c482 [maven-release-plugin] prepare for next development iteration 2014-10-07 14:30:26 -07:00
fjy 96e29c2fdd [maven-release-plugin] prepare release druid-0.6.157 2014-10-07 14:30:20 -07:00
Xavier Léauté 8f7f497b55 jax-rs smile support 2014-10-07 14:19:59 -07:00
Xavier Léauté 30183d3b4d update druid-api for json parsing improvements 2014-10-07 13:25:02 -07:00
fjy d4217f1588 Merge pull request #777 from metamx/initialized-endpoint 
Historical endpoints accessible while loading + new loadstatus endpoint
 2014-10-06 17:35:58 -06:00
Xavier Léauté 81641d7887 http accessible while loading + status endpoint 2014-10-06 13:32:27 -07:00
fjy c7b4d5b7b4 Merge branch 'master' into druid-0.7.x 
Conflicts:
	processing/src/test/java/io/druid/segment/filter/SpatialFilterTest.java
 2014-10-02 18:12:10 -07:00
Gian Merlino efa898422d Merge pull request #773 from metamx/cleanup-metrics 
Use multi-val dims for metrics
 2014-10-02 17:58:50 -07:00
fjy 816986f914 cleanup according to cr 2014-10-02 17:51:33 -07:00
fjy 75fb7986df Merge pull request #602 from metamx/offheap-incremental-index 
Offheap incremental index
 2014-10-01 16:20:24 -06:00
fjy 4fc0382d8a add more logging 2014-10-01 09:13:59 -07:00
nishantmonu51 ad75a21040 separate offheapIncrementalIndex implementation 2014-10-01 13:58:51 +05:30
fjy c3bea245a7 fix up some bugs 2014-09-30 17:20:52 -07:00
fjy 5b166134fb add docs 2014-09-30 14:59:02 -07:00
fjy 6d2747adcf make loading local files easier 2014-09-30 14:50:48 -07:00
fjy ee713f8e94 Merge pull request #727 from metamx/new-console-0.7.x 
New console 0.7.x
 2014-09-30 15:36:32 -06:00
jisookim0513 0e50852985 fixed MetadataTaskStorage and handler 2014-09-30 14:09:23 -07:00
jisookim0513 9d7b5d9b0f fixed javadoc; fixed pom files; deleted unnecessary class 2014-09-30 13:47:35 -07:00
fjy 0f96a09939 use multi-val dims for metrics 2014-09-30 13:36:01 -07:00
nishantmonu51 91eecfe3e7 fix #730 - memory check throws exception if fails 
fix exception handling, do not catch ProvisionException
 2014-09-30 23:03:40 +05:30
nishantmonu51 358ff915bb fix merge conflicts 2014-09-30 22:19:18 +05:30
nishantmonu51 2789536bed merge changes from druid-0.7.x 2014-09-30 22:05:49 +05:30
nishantmonu51 61c7fd2e6e make ingestOffheap tuneable 2014-09-30 15:30:02 +05:30
nishantmonu51 8cecb00aa9 upgrade jetty and workaround for half open connection 
Upgrade jetty and set acceptor priority delta to -1
 to give priority to servicing connections already accepted rather than
opening new connections.
 2014-09-30 14:22:16 +05:30
fjy 1a67ebe861 fix regression in test 2014-09-29 15:22:26 -07:00
nishantmonu51 adb4a65e0a Merge branch 'offheap-incremental-index' into mapdb-branch 2014-09-29 12:38:31 +05:30
jisookim0513 74565c9371 cleaned up the code 2014-09-27 13:10:01 -07:00
jisookim0513 aa887edb73 added two seperate modules for mysql and postgres 2014-09-27 13:08:53 -07:00
fjy fb1d6c177a fix dynamic config bug 2014-09-26 15:45:20 -07:00
jisookim0513 91bf8fadcf added MetadataSegmentManagerTest 2014-09-26 15:38:40 -07:00
jisookim0513 59b3e79c70 derby seems to be working..? 2014-09-26 14:36:17 -07:00
jisookim0513 6a641621b2 finished merging into druid-0.7.x; derby not working (to be fixed) 2014-09-26 14:24:53 -07:00
jisookim0513 43cc6283d3 trying to revert files that have overwritten changes 2014-09-26 12:38:04 -07:00
fjy 4244accab5 Merge branch 'druid-0.7.x' of github.com:metamx/druid into druid-0.7.x 2014-09-26 11:39:50 -07:00
fjy eaf0a48b92 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-09-26 11:39:24 -07:00
jisookim0513 a5d02b9030 second attempt to fix merge-conflicts 2014-09-24 15:59:00 -07:00
jisookim0513 3bf39cc9f8 attempted to fix merge-conflicts 2014-09-24 15:55:42 -07:00
Xavier Léauté 77c9afe92f try to make javadoc happy 2014-09-23 18:53:29 -07:00
Xavier Léauté 7cfe86b7bc [maven-release-plugin] prepare for next development iteration 2014-09-23 16:31:25 -07:00
Xavier Léauté 80166314d9 [maven-release-plugin] prepare release druid-0.6.156 2014-09-23 16:31:21 -07:00
Xavier Léauté d02fd97ea3 fix comment 2014-09-23 16:25:46 -07:00
Xavier Léauté 8603d2fd3e keep default behavior 2014-09-23 16:23:02 -07:00
fjy 4235e14fc6 [maven-release-plugin] prepare for next development iteration 2014-09-23 16:12:43 -07:00
fjy 576e9b8a84 [maven-release-plugin] prepare release druid-0.6.155 2014-09-23 16:12:39 -07:00
Xavier Léauté 35fb210cfa add test for parallel loading 2014-09-23 16:06:46 -07:00
Xavier Léauté 05d4f71ddc fix background annoucing race condition 2014-09-23 15:04:48 -07:00
Xavier Léauté 12449481e3 parallelize segment loading 2014-09-22 16:33:25 -07:00
nishantmonu51 f51ab84386 merge changes from druid-0.7.x 2014-09-22 23:48:45 +05:30
nishantmonu51 443e5788fb make OffheapIncrementalIndex tuneable 2014-09-22 19:26:10 +05:30
jisookim0513 273205f217 initial attempt for abstraction; druid cluster works with Derby as a default 2014-09-19 17:39:59 -07:00
nishantmonu51 8eb6466487 revert buffer size and add back rowFlushBoundary 2014-09-19 23:06:04 +05:30
Xavier Léauté 32b6135f3d background announce segment cache 2014-09-18 12:09:20 -07:00
fjy a17794a516 [maven-release-plugin] prepare for next development iteration 2014-09-16 15:22:55 -07:00
fjy f9c91ace3f [maven-release-plugin] prepare release druid-0.6.154 2014-09-16 15:22:49 -07:00
Xavier Léauté d9a0d403b3 default zk compressed + batch announcements in 0.7 2014-09-16 14:54:45 -07:00
fjy 8b387034a3 Merge pull request #741 from metamx/serverview-initialization 
initialize segment view before announcing broker
 2014-09-16 15:35:09 -06:00
fjy 1dbb6fd4b0 Merge pull request #736 from metamx/fix-synchronized 
fix synchronized on field getting updated
 2014-09-16 11:06:35 -06:00
Xavier Léauté 762485a0c9 remove unnecessary transient 2014-09-16 09:59:25 -07:00
Xavier Léauté f1e3518f37 initialize segment view before announcing broker 2014-09-16 00:07:31 -07:00
xvrl a73670ee7b Merge pull request #738 from metamx/dim-ex 
Have better dimension exclusions
 2014-09-15 17:14:29 -07:00
fjy 77dbac3739 fix import 2014-09-15 17:10:34 -07:00
fjy f2d6360d4d cleanup 2014-09-15 17:08:45 -07:00
fjy 34ae426ffa address cr 2014-09-15 17:08:02 -07:00
fjy 1ab2662772 have better dimension exclusions 2014-09-15 16:52:07 -07:00
Xavier Léauté 02892233ac fix synchronized on field getting updated 2014-09-15 13:56:31 -07:00
fjy aa28bc0613 address cr 2014-09-15 13:14:23 -07:00
Xavier Léauté 0712c666cb null check for something that cannot be null 2014-09-15 13:00:06 -07:00
Xavier Léauté c8b8e3f6e9 negating compare is bad 2014-09-15 13:00:06 -07:00
Xavier Léauté 137ad50bf1 classes that should be static 2014-09-15 13:00:06 -07:00
Xavier Léauté e57e2d97ba make constants final 2014-09-15 12:53:40 -07:00
Xavier Léauté d4795ce927 fix missing charsets 2014-09-15 12:53:40 -07:00
Xavier Léauté cfa92e8217 fix incorrect nullable annotations 2014-09-15 12:13:52 -07:00
fjy cbf5076961 still propagate log 2014-09-12 11:43:42 -07:00
fjy e62ed40d86 make some logs not emit 2014-09-11 16:49:18 -07:00
fjy 469ccbbe5e Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	processing/src/main/java/io/druid/query/FinalizeResultsQueryRunner.java
	processing/src/main/java/io/druid/query/UnionQueryRunner.java
	processing/src/main/java/io/druid/query/groupby/GroupByQueryRunnerFactory.java
	processing/src/main/java/io/druid/query/topn/TopNQueryEngine.java
	processing/src/main/java/io/druid/query/topn/TopNQueryRunnerFactory.java
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/test/java/io/druid/server/initialization/JettyTest.java
	services/pom.xml
 2014-09-11 16:20:50 -07:00
Xavier Léauté 32e256a842 [maven-release-plugin] prepare for next development iteration 2014-09-10 17:24:15 -07:00
Xavier Léauté f7dc13d240 [maven-release-plugin] prepare release druid-0.6.153 2014-09-10 17:24:11 -07:00
fjy fec7b43fcb make making v9 segments something completely configurable 2014-09-10 15:28:30 -07:00
Xavier Léauté d6f6a59ae2 fix for missing context when embedding broker 2014-09-09 14:25:56 -07:00
Tarek Rached 609887887e updated env when we are local 2014-09-09 13:29:42 -07:00
Tarek Rached ae2a09ab5e added new console 2014-09-09 12:44:57 -07:00
Tarek Rached 47b52aaae1 moved old console to old-console 2014-09-09 12:44:30 -07:00
Xavier Léauté 22553b0b28 [maven-release-plugin] prepare for next development iteration 2014-09-05 11:07:22 -07:00
Xavier Léauté 570a7a07bf [maven-release-plugin] prepare release druid-0.6.152 2014-09-05 11:07:14 -07:00
fjy e24be03a38 [maven-release-plugin] prepare for next development iteration 2014-09-04 14:10:13 -07:00
fjy a933ea1866 [maven-release-plugin] prepare release druid-0.6.151 2014-09-04 14:10:09 -07:00
xvrl c4c54f7bc9 Merge pull request #721 from metamx/fix-union-query 
fix Union Query result merging
 2014-09-04 12:20:19 -07:00
nishantmonu51 0dec5826ed remove unused imports 2014-09-05 00:07:42 +05:30
nishantmonu51 90632ecf3a review comment - have separate methods 2014-09-05 00:03:36 +05:30
fjy f473e62f1f Merge pull request #714 from metamx/cleanup-javadocs 
Cleanup javadocs
 2014-09-04 12:28:10 -06:00
nishantmonu51 2772a78b3b review comments - move merging to tool chest 2014-09-04 11:31:04 +05:30
nishantmonu51 268a42bd79 fix Union Query result merging 2014-09-03 16:38:02 +05:30
Xavier Léauté 508e982190 Merge remote-tracking branch 'origin/master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	server/src/main/java/io/druid/server/ClientQuerySegmentWalker.java
	services/pom.xml
 2014-08-30 22:42:36 -07:00
Gian Merlino 96d7b18163 [maven-release-plugin] prepare for next development iteration 2014-08-29 15:29:58 -07:00
Gian Merlino 860cf36ab1 [maven-release-plugin] prepare release druid-0.6.150 2014-08-29 15:29:54 -07:00
Xavier Léauté 58ab759fc6 remove unused imports 2014-08-29 14:03:47 -07:00
Xavier Léauté ac05836833 make Java 8 javadoc happy 2014-08-29 13:58:50 -07:00
Xavier Léauté daca178631 [maven-release-plugin] prepare for next development iteration 2014-08-29 10:43:19 -07:00
Xavier Léauté 0793a5aee6 [maven-release-plugin] prepare release druid-0.6.149 2014-08-29 10:43:15 -07:00
Xavier Léauté b0b39d6ec0 fix maxTime caching for individual bounds 2014-08-28 17:14:25 -07:00
Xavier Léauté 1d6842c7f0 [maven-release-plugin] prepare for next development iteration 2014-08-27 12:22:53 -07:00
Xavier Léauté 575951f99a [maven-release-plugin] prepare release druid-0.6.148 2014-08-27 12:22:49 -07:00
Xavier Léauté 047f793cb7 query post-processing operators 
- Allows post-processing of query results
- Example timewarp post-processing operator
 2014-08-25 16:55:46 -07:00
Xavier Léauté 8b79749989 add tests for groupBy caching with HLL 2014-08-22 13:44:21 -07:00
fjy 00f7077f50 [maven-release-plugin] prepare for next development iteration 2014-08-22 08:28:56 -07:00
fjy df488563e4 [maven-release-plugin] prepare release druid-0.6.147 2014-08-22 08:28:51 -07:00
fjy df48b48e35 [maven-release-plugin] prepare for next development iteration 2014-08-21 12:50:33 -07:00
fjy 4f0b994037 [maven-release-plugin] prepare release druid-0.6.146 2014-08-21 12:50:30 -07:00
nishantmonu51 4641b3ac1b Add test for MessageTimeRejectionPolicy 
Add test for persist for MessageTimeRejectionPolicy
 2014-08-22 01:07:34 +05:30
Gian Merlino 92b0611afb Merge pull request #693 from metamx/handle-arithmetic-ex 
Handle arithmetic ex - additional commit for PR #692
 2014-08-20 23:24:06 -07:00
nishantmonu51 fe03476c53 truncate segmentGranularity 2014-08-21 11:40:30 +05:30
Gian Merlino 5970b96dc5 Merge pull request #692 from metamx/handle-arithmetic-ex 
Handle arithmetic exception for overlow
 2014-08-20 23:09:43 -07:00
nishantmonu51 8c9d639947 review comment - better way to handle it 2014-08-21 11:38:24 +05:30
nishantmonu51 87b83e7bdb better comment 2014-08-21 09:47:23 +05:30
nishantmonu51 c48c4ac0f6 handle arithmeticEx 2014-08-21 09:44:55 +05:30
nishantmonu51 67f4bbae74 fixes from review comments 
fix sync of aggs,
fix NPE in sink.isFull,
RealtimeTuningConfig lower the bufferSize to 256m
 2014-08-21 09:24:14 +05:30
nishantmonu51 79ff993b31 increase default buffer size to 512m 2014-08-20 22:15:06 +05:30
nishantmonu51 33354cf7fe replace maxRowsInMemory with BufferSize 2014-08-20 20:59:44 +05:30
nishantmonu51 60906c3244 Revert "make valueType configurable" 
This reverts commit 6f60a3f604.
 2014-08-20 11:55:26 +05:30
fjy a15d32da9b Merge branch 'druid-0.7.x' into offheap-incremental-index 2014-08-18 17:11:12 -07:00
Xavier Léauté 78054d7f05 fix test initialization 2014-08-18 13:37:47 -07:00
fjy 77e514688a Merge branch 'druid-0.7.x' into offheap-incremental-index 2014-08-18 11:14:19 -07:00
Xavier Léauté 1fd30ab588 default service/host/port for all nodes 2014-08-15 17:14:05 -07:00
fjy 51988be776 [maven-release-plugin] prepare for next development iteration 2014-08-14 14:59:00 -07:00
fjy 2830028092 [maven-release-plugin] prepare release druid-0.6.145 2014-08-14 14:58:57 -07:00
fjy 4f0f198a00 add more logging 2014-08-14 14:43:54 -07:00
fjy c36ffd1e55 address cr 2014-08-14 14:39:08 -07:00
fjy 6ba0a0df8b address cr 2014-08-14 13:56:36 -07:00
fjy 96d5d4a59a fix some rt cleanup and failure edge cases 2014-08-14 13:38:32 -07:00
nishantmonu51 6f60a3f604 make valueType configurable 2014-08-13 14:37:57 +05:30
nishantmonu51 1b0a72751b Add support for LongColumn 2014-08-13 08:52:36 +05:30
fjy 4e3f4fbc22 [maven-release-plugin] prepare for next development iteration 2014-08-12 13:24:55 -07:00
fjy f3aafc246c [maven-release-plugin] prepare release druid-0.6.144 2014-08-12 13:24:50 -07:00
nishantmonu51 c6712739dc merge changes from druid-0.7.x 2014-08-12 15:47:42 +05:30
fjy b740987381 Merge pull request #616 from metamx/jisoo 
add retry feature on broker side; modify QueryRunner inferface and tests
 2014-08-11 15:36:16 -06:00
fjy 2ecd55bae5 Merge branch 'druid-0.7.x' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-11 12:47:38 -07:00
fjy 70b86c5551 Merge branch 'master' into druid-0.7.x 
Conflicts:
	cassandra-storage/pom.xml
	common/pom.xml
	examples/pom.xml
	hdfs-storage/pom.xml
	histogram/pom.xml
	indexing-hadoop/pom.xml
	indexing-service/pom.xml
	kafka-eight/pom.xml
	kafka-seven/pom.xml
	pom.xml
	processing/pom.xml
	rabbitmq/pom.xml
	s3-extensions/pom.xml
	server/pom.xml
	services/pom.xml
 2014-08-11 12:35:47 -07:00
fjy 0cff63c7c6 [maven-release-plugin] prepare for next development iteration 2014-08-09 19:36:11 -07:00
fjy 724562b181 [maven-release-plugin] prepare release druid-0.6.143 2014-08-09 19:36:05 -07:00
fjy 0e1a8f6f02 concurrency my old foe, we meet again 2014-08-09 19:33:07 -07:00
fjy d58ae945a4 [maven-release-plugin] prepare for next development iteration 2014-08-08 16:22:38 -07:00
fjy d240479133 [maven-release-plugin] prepare release druid-0.6.142 2014-08-08 16:22:34 -07:00
fjy 8998700760 prepare for next release 2014-08-08 16:19:45 -07:00
fjy 199f2b1683 [maven-release-plugin] prepare for next development iteration 2014-08-08 15:53:53 -07:00
fjy 29a077d893 [maven-release-plugin] prepare release druid-0.6.141 2014-08-08 15:53:49 -07:00
fjy e5896f4863 fix yet another bug with LQP 2014-08-08 15:50:57 -07:00
fjy 958792db5c [maven-release-plugin] prepare for next development iteration 2014-08-08 15:16:38 -07:00
fjy 4482e1b7d8 [maven-release-plugin] prepare release druid-0.6.140 2014-08-08 15:16:34 -07:00
fjy 9a371f3490 Merge pull request #674 from metamx/fix-arbitrary-granularity 
fix missing queryGranularity in ArbitraryGranularitySpec
 2014-08-08 16:12:24 -06:00
Xavier Léauté 3826b1d976 fix missing queryGranularity in ArbitraryGranularitySpec 2014-08-08 15:11:35 -07:00
fjy 68827ff88b fix another bug in LQP with concurrency 2014-08-08 15:06:03 -07:00
fjy 48b99e43fc fix LQP register callback 2014-08-08 14:34:54 -07:00
fjy ddcea18631 fix dependencies 2014-08-08 14:02:14 -07:00
fjy c860de43cf fix dependencies 2014-08-08 13:46:25 -07:00
fjy f3970bb1d8 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
 2014-08-08 12:44:29 -07:00
fjy 74646f18f4 [maven-release-plugin] prepare for next development iteration 2014-08-08 11:40:52 -07:00
fjy 78c7140b52 [maven-release-plugin] prepare release druid-0.6.139 2014-08-08 11:40:46 -07:00
xvrl 038f2de496 Merge pull request #672 from metamx/router-strat 
Add more config to router strategies; fix bad test
 2014-08-08 11:37:35 -07:00
fjy 9803be7529 Merge pull request #673 from metamx/rt-plumber-merge-behavior 
Change merged-segment-removing behavior in RealtimePlumber.
 2014-08-08 12:24:23 -06:00
Gian Merlino 179170224d Fix conditional. 2014-08-08 11:20:36 -07:00
Gian Merlino f547049929 Change merged-segment-removing behavior in RealtimePlumber. 
Old: Merged segments are deleted upon successful push, or on IOException.
New: Merged segments are deleted on any Exception, but NOT successful push.

Deleting a merged segment means that the next merge-and-push run will try to
push it again. So we want that to happen if there was any sort of Exception.
We *don't* want it to happen if the merge-and-push was successful, since in
that case, we are just waiting for historical nodes to load the segment. It
might take a while, but there's no reason to re-push while waiting.
 2014-08-08 10:59:16 -07:00
fjy a855db2170 make things immutable 2014-08-08 10:53:10 -07:00
fjy d2c7a67a0c clean up some redundant code 2014-08-08 10:52:05 -07:00
fjy f1409f751a fix bad defaults 2014-08-08 10:46:36 -07:00
fjy d4cf2763ac fix test again 2014-08-08 10:45:59 -07:00
fjy 8877ee9714 fix test 2014-08-08 10:43:50 -07:00
fjy b3eb4cc5da fix defaults 2014-08-08 10:42:58 -07:00
fjy 627447c6f8 add more config to router strategies; fix bad test 2014-08-08 10:41:11 -07:00
fjy 08c7aabb34 fix deadlock condition that can exist in the coordinator 2014-08-07 15:55:25 -07:00
fjy 91ebe45b4e support both rejectionPolicy and rejectionPolicyFactory in serde 2014-08-07 10:06:27 -07:00
fjy 7624bb5ea3 Merge branch 'master' into new-init 
Conflicts:
	examples/config/historical/runtime.properties
	examples/config/overlord/runtime.properties
	examples/config/realtime/runtime.properties
	processing/src/main/java/io/druid/guice/PropertiesModule.java
	server/src/main/java/io/druid/initialization/Initialization.java
 2014-08-06 15:47:22 -07:00
Xavier Léauté 87fc354237 rename CleanupAvailable -> CleanupUnneeded 2014-08-06 14:29:00 -07:00
Xavier Léauté 70660ad295 separate cleanup of overshadowed vs. unavailable segments 2014-08-06 14:06:52 -07:00
fjy 9f4dd7b33e [maven-release-plugin] prepare for next development iteration 2014-08-06 13:34:04 -07:00
fjy aa5cb91618 [maven-release-plugin] prepare release druid-0.6.138 2014-08-06 13:33:59 -07:00
fjy 8b6f030ad5 Merge pull request #661 from metamx/drop-remove-stuff 
Coordinator drop/remove related stuff.
 2014-08-06 14:28:23 -06:00
Gian Merlino 6e33cd4e0a Coordinator drop/remove related stuff. 
- DruidCoordinatorCleanup should wait for availableSegments to populate
  before dropping any segments.
- Clarify that millisToWaitBeforeDeleting applies to "removing" rather than
  "dropping" segments.
- LoadRule shouldn't need to wait for the deletionWaitTime before dropping
  excess replicas.
 2014-08-06 12:52:09 -07:00
fjy 4d4c6b891e address cr 2014-08-05 15:58:42 -07:00
fjy 8097e04938 fix naming 2014-08-05 15:48:30 -07:00
fjy 3a78960e65 Fix how metrics are emitted from servermanager 2014-08-05 15:47:30 -07:00
fjy 66225d698f fix stupidness with spatial indexing 2014-08-05 15:07:56 -07:00
Gian Merlino c58a316437 Need more synchronizing in DruidDataSource. 2014-08-05 13:32:53 -07:00
jisookim0513 6f58cd1156 fix merge conflcts; change test files 2014-08-04 17:45:56 -07:00
fjy 290982433b Merge pull request #657 from metamx/only-uncached-query-metrics 
Only emit query metrics for uncached queries.
 2014-08-04 18:01:18 -06:00
Gian Merlino 6c8222788b Only emit query metrics for uncached queries. 2014-08-04 16:42:38 -07:00
fjy 65b8278add [maven-release-plugin] prepare for next development iteration 2014-08-01 10:14:34 -07:00
fjy 0ecbef6c5d [maven-release-plugin] prepare release druid-0.6.137 2014-08-01 10:14:30 -07:00
fjy 872a699142 another log fix 2014-07-31 17:23:40 -07:00
fjy bf8c24f88d [maven-release-plugin] prepare for next development iteration 2014-07-31 14:08:38 -07:00
fjy 66020fc2b6 [maven-release-plugin] prepare release druid-0.6.136 2014-07-31 14:08:34 -07:00
fjy 1c4d8c55dc finally track down and fix the node exists problem with not deleting noop requests 2014-07-31 13:56:29 -07:00
nishantmonu51 637bd35785 merge changes from druid-0.7.x 2014-07-31 16:07:22 +05:30
fjy a8c8e4cb51 [maven-release-plugin] prepare for next development iteration 2014-07-29 22:03:14 -07:00
fjy 25eda62a0c [maven-release-plugin] prepare release druid-0.6.135 2014-07-29 22:03:09 -07:00
fjy ecbc9eda48 revert breakage to chat handlers 2014-07-29 21:55:05 -07:00
fjy 3920300900 fix router bug 2014-07-29 21:44:07 -07:00
fjy 0c898c58d1 [maven-release-plugin] prepare for next development iteration 2014-07-29 18:01:06 -07:00
fjy c8244d05d1 [maven-release-plugin] prepare release druid-0.6.134 2014-07-29 18:01:02 -07:00
fjy f7319620c4 [maven-release-plugin] prepare for next development iteration 2014-07-29 17:40:56 -07:00
fjy 540a146eaa [maven-release-plugin] prepare release druid-0.6.133 2014-07-29 17:40:52 -07:00
fjy f577e1e74b fix guava version and firehose 2014-07-29 17:23:32 -07:00
fjy b5ecf000ad [maven-release-plugin] prepare for next development iteration 2014-07-29 16:20:14 -07:00
fjy 877bd98667 [maven-release-plugin] prepare release druid-0.6.132 2014-07-29 16:20:10 -07:00
fjy 43d4e5418a Merge pull request #653 from metamx/override-static-resource 
allow overriding coordinator static resource
 2014-07-29 17:15:01 -06:00
fjy 1586d60ec2 fix bugs in log printing 2014-07-29 16:14:29 -07:00
Xavier Léauté 05a09d2550 allow overriding coordinator static resource 2014-07-29 15:13:57 -07:00
fjy dc38677454 remove unneeded dependency 2014-07-29 14:23:50 -07:00
fjy e1d87ed5ea [maven-release-plugin] prepare for next development iteration 2014-07-29 14:13:41 -07:00
fjy c46965c52c [maven-release-plugin] prepare release druid-0.6.131 2014-07-29 14:13:37 -07:00
fjy 1ac84fcd00 some more logging cleanup for zk coordination 2014-07-29 14:07:19 -07:00
xvrl d21e377a20 Merge pull request #651 from metamx/rewrite-async 
rewrite router async logic
 2014-07-29 13:56:01 -07:00
Xavier Léauté 25a5a0fd33 fix unnecessary error messages 2014-07-29 13:54:36 -07:00
fjy 088e2c5af3 address cr 2014-07-29 13:36:28 -07:00
fjy b09517319d remove debug code 2014-07-29 13:23:12 -07:00
fjy 4a417cbcfc clean up some logging in Zk coordinator 2014-07-29 11:49:06 -07:00
fjy 41db4b1063 actually use the metrics emitting listener 2014-07-29 10:28:51 -07:00
fjy 2ad1bd3f44 add back metrics 2014-07-29 10:22:55 -07:00
fjy 382cde9cf2 less verbose logging for LQP 2014-07-29 09:51:53 -07:00
fjy 088c2386dc rewrite router async logic 2014-07-28 16:32:15 -07:00
fjy 82d623ada7 fix compilation error 2014-07-25 16:35:55 -07:00
Xavier Léauté 863929d550 unnecessary arguments 2014-07-25 16:26:44 -07:00
Xavier Léauté df59d2acb4 remove debug code 2014-07-25 16:25:47 -07:00
Xavier Léauté 2d4f1889e8 completely async request proxying + jetty update 2014-07-25 16:22:43 -07:00
fjy fc95a79abd add query priorities to metric emission 2014-07-25 16:10:55 -07:00
fjy 45e443bedb a few slight tweaks and bumping up curator version 2014-07-25 16:07:40 -07:00
fjy bb5c5b3ce7 fix config with broker select and add docs 2014-07-25 10:50:41 -07:00
Xavier Léauté 85f739797f remove debug println 2014-07-24 16:49:09 -07:00
Xavier Léauté 03efbb0e02 default query timeout to server idle timeout 2014-07-22 16:59:27 -07:00
Xavier Léauté 00f086665c default async timeout to server idle timeout 2014-07-22 16:55:21 -07:00
fjy 09e2f13ffc adding missing header 2014-07-22 10:12:47 -07:00
fjy 8f2fc595ee allow router to override selection based on config 2014-07-22 10:12:02 -07:00
fjy 1de0f89718 [maven-release-plugin] prepare for next development iteration 2014-07-18 11:55:50 -07:00
fjy 39bff5c1c2 [maven-release-plugin] prepare release druid-0.6.130 2014-07-18 11:55:46 -07:00
fjy 9e303527f2 [maven-release-plugin] prepare for next development iteration 2014-07-18 11:17:56 -07:00
fjy ea04022772 [maven-release-plugin] prepare release druid-0.6.129 2014-07-18 11:17:52 -07:00
nishantmonu51 4ce12470a1 Add way to skip determine partitions for index task 
Add a way to skip determinePartitions for IndexTask by manually
specifying numShards.
 2014-07-18 18:52:15 +05:30
fjy d8b8826c2e Merge branch 'cleanup-ingest' of github.com:metamx/druid into cleanup-ingest 
Conflicts:
	server/src/test/java/io/druid/realtime/firehose/CombiningFirehoseFactoryTest.java
 2014-07-17 20:26:11 -07:00
fjy 291f4c00ae Merge branch 'master' of github.com:metamx/druid into cleanup-ingest 2014-07-17 20:24:59 -07:00
nishantmonu51 0e0454a34c switch reingest task to noop & fix compilation 
switch back to noop task, its confusing to have a reinvest task that
does nothing.
fix compilation
 2014-07-18 06:50:58 +05:30
Xavier Léauté 95a09c04bf log errors in asyncForwardingServlet 2014-07-17 16:22:30 -07:00
fjy 465ac77b82 fix broken UT 2014-07-17 14:39:52 -07:00
fjy 76f096e667 Merge branch 'master' of github.com:metamx/druid into cleanup-ingest 2014-07-17 14:38:54 -07:00
fjy 8ef380b3f9 Merge pull request #638 from metamx/rejigger-cache 
Rejiggering the cache
 2014-07-17 15:19:23 -06:00
fjy ef325f15ff update pom and fix broken UT 2014-07-17 13:47:59 -07:00
fjy ba978d8b79 some minor cleanups to ingest firehose 2014-07-17 13:05:59 -07:00
fjy bc650a1c80 Merge pull request #627 from metamx/druid-firehose 
Functionality to ingest a Druid segment and change the schema
 2014-07-17 13:41:16 -06:00
fjy b70a6b1061 address code review comments 2014-07-16 13:52:06 -07:00
fjy de8cb55260 Merge branch 'master' into rejigger-cache 
Conflicts:
	pom.xml
 2014-07-16 13:37:20 -07:00
nishantmonu51 b74845e942 fix test 
committed it by mistake.
 2014-07-16 09:39:51 +05:30
fjy 27c4750780 fix infinite loop in test 2014-07-15 16:00:56 -07:00
fjy bdfeccd092 doc and timeout fix 2014-07-15 15:57:58 -07:00