04d3054353
Merge pull request #2303 from CHOIJAEHONG1/localfirehouse-basedir-npe
...
Throw an IAE when baseDir is null in LocalFireHose
2016-01-21 07:58:52 -08:00
dcb7830330
Merge pull request #984 from drcrallen/thread-priority-rebase
...
Use thread priorities. (aka set `nice` values for background-like tasks)
2016-01-21 15:02:34 +05:30
2e1d6aaf3d
Use thread priorities. (aka set `nice` values for background-like tasks)
...
* Defaults the thread priority to java.util.Thread.NORM_PRIORITY in io.druid.indexing.common.task.AbstractTask
* Each exec service has its own Task Factory which is assigned a priority for spawned task. Therefore each priority class has a unique exec service
* Added priority to tasks as taskPriority in the task context. <0 means low, 0 means take default, >0 means high. It is up to any particular implementation to determine how to handle these numbers
* Add options to ForkingTaskRunner
* Add "-XX:+UseThreadPriorities" default option
* Add "-XX:ThreadPriorityPolicy=42" default option
* AbstractTask - Removed unneded @JsonIgnore on priority
* Added priority to RealtimePlumber executors. All sub-executors (non query runners) get Thread.MIN_PRIORITY
* Add persistThreadPriority and mergeThreadPriority to realtime tuning config
2016-01-20 14:00:31 -08:00
61aca6f9cc
remove wrong checks
...
sink never have null hydrants and hydrants never have null adapters
2016-01-20 23:43:53 +05:30
7132428bba
throw IAE when baseDir is null in LocalFireHose
2016-01-21 01:27:32 +09:00
59ea186af7
fix reference counting for segments
2016-01-20 17:24:21 +05:30
747343e621
Preserve dimension order across indexes during ingestion
2016-01-19 13:34:11 -08:00
df2906a91c
Merge pull request #2290 from gianm/index-merger-v9-stuff
...
Respect buildV9Directly in PlumberSchools, so it works on standalone realtime.
2016-01-19 13:04:00 -08:00
0c31f007fc
Merge pull request #1728 from himanshug/aggregators_in_segment_metadata
...
Store AggregatorFactory[] in segment metadata
2016-01-19 12:55:49 -08:00
fe841fd961
Merge pull request #2118 from guobingkun/fix_segment_loading
...
Fix loading segment for historical
2016-01-19 14:25:48 -06:00
a99aef29a1
adding aggregators to segment metadata
2016-01-19 14:23:39 -06:00
1dcf22edb7
Respect buildV9Directly in PlumberSchools, so it works on standalone realtime nodes.
...
Also parameterize some tests to run with/without buildV9Directly:
- IndexGeneratorJobTest
- RealtimeIndexTaskTest
- RealtimePlumberSchoolTest
2016-01-19 12:15:06 -08:00
c4ad50f92c
Fix loading segment for historical
...
Historical will drop a segment that shouldn't be dropped in the following scenario:
Historical node tried to load segmentA, but failed with SegmentLoadingException,
then ZkCoordinator called removeSegment(segmentA, blah) to schedule a runnable that would drop segmentA by deleting its files. Now, before that runnable executed, another LOAD request was sent to this historical, this time historical actually succeeded on loading segmentA and announced it. But later on, the scheduled drop-of-segment runnable started executing and removed the segment files, while historical is still announcing segmentA.
2016-01-19 10:29:49 -06:00
164b0aad7a
removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class
...
instead of a Map to store segment metadata
2016-01-18 22:03:46 -06:00
637d2605e7
kill unwanted parent directories when a segment is deleted from LocalDataSegmentKiller
2016-01-18 21:51:04 -06:00
f6a1a4ae20
Merge pull request #2138 from KurtYoung/feature-build-v9
...
build v9 directly
2016-01-16 13:35:46 -06:00
82ff98c2bf
add config for build v9 directly and update docs
2016-01-16 11:26:34 +08:00
2e54553a8f
Merge pull request #1990 from himanshug/schedule_kill_task
...
support periodic hard delete of segments
2016-01-15 15:22:33 -06:00
7c65880e55
Merge pull request #2270 from rasahner/warnOfChatHandlerNoop
...
if chathandler is noop, log using warn instead of info
2016-01-15 13:29:27 -07:00
34cd8f8c72
Merge pull request #2258 from fjy/acl-zk
...
acl for zookeeper is added
2016-01-15 10:27:08 -07:00
a54c726726
Merge pull request #2266 from anubhgup/fix-announce
...
Fix loss in segment announcements when segments do not fit in zNode
2016-01-14 17:10:43 -08:00
dc1a62c3d9
Merge pull request #2248 from metamx/druidNodeHashEquals
...
Add hashCode and equals to DruidNode
2016-01-14 16:04:58 -08:00
6d09ab839f
Fix for loss in segment announcements when segments do not fit in the znodes during compress mode.
...
Added unit test (from Navis).
2016-01-14 14:44:23 -08:00
7361cd173f
if chathandler is noop, log using warn instead of info
2016-01-14 10:12:42 -06:00
1908d63162
acl for zookeeper is added
2016-01-13 14:56:05 -08:00
18479bb757
time-descending result of timeseries queries
2016-01-13 12:23:01 +09:00
eb2d251ac8
support periodic hard delete of segments
2016-01-12 16:55:05 -06:00
01a0715ee2
Merge pull request #2161 from metamx/query-metrics-timeout
...
Fix Query metrics for query timeout
2016-01-12 09:50:40 -06:00
ef3cddabe9
adding UTs for DruidCoordinatorConfig
2016-01-11 22:03:25 -06:00
b0e04a9162
Add hashCode and equals to DruidNode
2016-01-11 15:23:45 -08:00
ea623e43d2
Merge pull request #2240 from metamx/fix-load-rule
...
Fix loadRule when one of the tiers had no available servers
2016-01-11 10:05:31 -08:00
32bc2f776e
Fix loadRule when one of the tiers had no servers
...
When one of the tiers have no servers, LoadRule should ignore that tier
and continue to load/drop segments in other available tiers.
the bug also causes whacky behavior with LoadRule with non existent
tier where the segment balancer keeps on moving segments to other nodes
in existing tiers but the extra segment copies are never dropped
eventually leading to all the tiers getting full .
2016-01-11 15:53:14 +05:30
d255f4baac
Merge pull request #2234 from pjain1/emit_realtime_metrics
...
emit handoff count metrics
2016-01-08 14:24:16 -06:00
9dba0f67e7
emit handoff count metrics
2016-01-08 12:36:13 -06:00
15fc070232
Merge pull request #2213 from himanshug/fix_curtator_test_base
...
[wip] trying/finding fix for announcer test failures
2016-01-07 18:23:49 -08:00
1bfb4e3988
Emit query/time for failed and timeout queries
...
emit query/time metric also
add success flag
fix success flag for router metrics
review comments
formatting.
2016-01-07 19:41:54 +05:30
7ab810f3eb
do not ignore exceptions from curator cleanup in CuratorTestBase
2016-01-06 10:42:53 -06:00
14989f272d
Add metrics for ingest/bytes/received for EventReceiverFirehose
...
review comments
review comments
2016-01-05 20:06:09 +05:30
57d91d754d
Comment out buggy unit tests, fix #2185
2016-01-03 09:50:16 -08:00
8424b2b456
Fix announcer test bad check
2016-01-01 19:55:22 -08:00
fa5c3bb014
adding decorate(DimensionSelector) to DimensionSpec to enable support for arbitrary filtering/transformations to returned dimension values
2015-12-30 15:06:24 -06:00
3c107c5757
Merge pull request #2150 from himanshug/emit_query_bytes
...
emit query/bytes metric
2015-12-30 13:44:19 -06:00
7ffa706655
Merge pull request #2152 from metamx/add-taskId
...
Add taskId to realtimeMetrics
2015-12-29 10:33:40 -08:00
b1261035a7
Merge pull request #1861 from guobingkun/insert_segment_tool
...
insert-segment tool
2015-12-29 10:06:07 -08:00
38b0f1fbc2
fix transient failures in unit tests
2015-12-28 20:03:30 -08:00
d94821998f
Merge pull request #2153 from navis/fix-testfail-DruidCoordinatorTest
...
Another try to fix DruidCoordinatorTest#testCoordinatorRun
2015-12-27 17:03:46 -08:00
10c19fe9f8
Another try to fix DruidCoordinatorTest#testCoordinatorRun
2015-12-24 03:02:37 +09:00
2c3c4a3f8f
Another try to fix xxServerViewTests
2015-12-24 02:13:40 +09:00
508629916d
Merge pull request #2137 from guobingkun/fix_serverview_transient_error2
...
[Don't merge]Another effort on fixing the transient error for XXXServerViewTest
2015-12-23 07:33:54 -08:00
978a3fd8ae
Add taskId to realtimeMetrics
...
Add task Id to Realtime Metrics
2015-12-23 18:05:25 +05:30
Fangjin Yang
Nishant
Charles Allen
Jaehong Choi
jon-wei
Jonathan Wei
Himanshu
Gian Merlino
Bingkun Guo
Kurt Young
David Lim
Xavier Léauté
Anubhav Gupta
Robin
Nikita Geer
navis.ryu
Charles Allen
Parag Jain