e78a469fb7
UTs for ExtensionsConfig
2016-03-25 10:51:28 -05:00
004b00bb96
config to explicitly specify classpath for hadoop container during hadoop ingestion
2016-03-25 10:51:28 -05:00
713062053c
Filters: Add filter.toFilter method, use that instead of the instanceof chain in Filters.
...
I believe that the instanceof chain in Filters exists because in the past, Filter
and DimFilter were in different packages (DimFilter was in druid-client and Filter
was in druid-processing). And since druid-client didn't depend on druid-processing,
DimFilter couldn't have a toFilter method. But now it can.
2016-03-23 17:03:49 -07:00
f25b2ed6f8
Duplicate statement in ReservoirSegmentSamplerTest.java
2016-03-22 22:14:36 -04:00
826b371259
Merge pull request #2697 from guobingkun/remove_duplicate_version_converter
...
remove duplicated DruidCoordinatorVersionConverter
2016-03-22 15:48:09 -07:00
a6e9ff48ec
Merge pull request #2688 from pjain1/props_cli
...
do not inject properties directly in module
2016-03-22 15:27:19 -05:00
3778adf1f4
remove duplicated DruidCoordinatorVersionConverter
2016-03-22 14:45:52 -05:00
7b93195dc6
do not inject properties directly in module
2016-03-22 14:30:10 -05:00
00d7021291
Merge pull request #2607 from jon-wei/dim_schema
...
Support use of DimensionSchema class in DimensionsSpec
2016-03-22 11:53:46 -05:00
3220b109ad
Merge pull request #2570 from binlijin/single_dimension_partitioning
...
Single dimension hash-based partitioning
2016-03-22 11:51:06 -05:00
bce600f5d5
Single dimension hash-based partitioning
2016-03-22 13:15:33 +08:00
a59c9ee1b1
Support use of DimensionSchema class in DimensionsSpec
2016-03-21 13:12:04 -07:00
25967d0ed8
fix servlet startup sequence, fixes #2681
2016-03-18 15:06:15 -07:00
5da9a280b6
Query Time Lookup - Dynamic Configuration
2016-03-18 09:45:05 -07:00
45c413af7e
Merge pull request #2674 from metamx/fix-broadcast-lockup
...
separate HTTP client pool for cancellation requests
2016-03-17 15:23:42 -07:00
1718a7224b
separate HTTP pool for cancellation requests
...
* reduces contention between queries and cancellation requests
* more aggressive timeouts for cancellation requests
2016-03-17 12:11:18 -07:00
c716af5b04
Merge pull request #2678 from metamx/fixImports
...
Fix some google related imports
2016-03-17 11:53:16 -07:00
a52c6d3bee
Fix some google related imports
2016-03-17 11:03:29 -07:00
738dcd8cd9
Update version to 0.9.1-SNAPSHOT.
...
Fixes #2462
2016-03-17 10:34:20 -07:00
948b19a088
do not silently ingnore rows
2016-03-16 09:30:19 -05:00
ec949d76e3
Merge pull request #2655 from navis/hint-coordinator-client
...
Add hint message for missing `druid.selectors.coordinator.serviceName`
2016-03-14 20:57:40 -07:00
5ec5ac92c6
Merge pull request #2382 from himanshug/broker_segment_tier_selection
...
at broker, if configured, only add segments from specific tiers to the timeline
2016-03-14 16:53:06 -07:00
83e1d5d7bf
Add hint message for missing `druid.selectors.coordinator.serviceName`
2016-03-15 08:39:07 +09:00
06813b510a
Merge pull request #2571 from himanshug/gp_by_avoid_sort
...
avoid sort while doing groupBy merging when possible
2016-03-14 14:46:51 -07:00
773d6fe86c
Merge pull request #2646 from atomx/update-maxmind
...
Update com.maxmind.geoip2 to 2.6.0
2016-03-14 11:20:48 -07:00
d51a0a0cf4
Merge pull request #2220 from gianm/appenderator-kafka
...
Appenderators, DataSource metadata, KafkaIndexTask
2016-03-14 13:14:36 -05:00
2861e854f0
Merge pull request #2540 from pjain1/remove_kill
...
Remove extra parameter from deleteDataSourceSpecificInterval endpoint and correct exception message for invalid interval
2016-03-14 11:16:23 -05:00
21b7572533
Update com.maxmind.geoip2 to 2.6.0
...
com.maxmind.geoip2 2.6.0 depends on com.google.http-client 1.15.0-rc (3 years old).
When trying to include other libraries in Druid that require an up to date version of com.google.http-client this causes a problem.
2016-03-12 09:44:00 +00:00
187569e702
DataSource metadata.
...
Geared towards supporting transactional inserts of new segments. This involves an
interface "DataSourceMetadata" that allows combining of partially specified metadata
(useful for partitioned ingestion).
DataSource metadata is stored in a new "dataSource" table.
2016-03-10 17:41:50 -08:00
3d2214377d
Appenderatoring.
...
Appenderators are a way of getting more control over the ingestion process
than a Plumber allows. The idea is that existing Plumbers could be implemented
using Appenderators, but you could also implement things that Plumbers can't do.
FiniteAppenderatorDrivers help simplify indexing a finite stream of data.
Also:
- Sink: Ability to consider itself "finished" vs "still writable".
- Sink: Ability to return the number of rows contained within the sink.
2016-03-10 17:41:50 -08:00
92c828f904
Make SegmentHandoffNotifier Closeable.
2016-03-10 16:50:37 -08:00
ad5ffdf483
Nix Committers.supplierOf; Suppliers.ofInstance is good enough.
2016-03-10 16:50:37 -08:00
8a11161b20
Plumbers: Move plumber.add out of try/catch for ParseException.
...
The incremental indexes handle that now so it's not necessary.
Also, add debug logging and more detailed exceptions to the incremental
indexes for the case where there are parse exceptions during aggregation.
2016-03-10 16:39:26 -08:00
02dfd5cd80
update IncrementalIndex to support unsorted facts map that can be used in groupBy merging to improve performance
2016-03-10 16:11:48 -06:00
708bc674fa
Make specifying query context booleans more consistent.
...
Before, some needed to be strings and some needed to be real booleans. Now
they can all be either one.
2016-03-08 19:38:26 -08:00
9c2420a1bc
Merge pull request #2599 from himanshug/datasource_isolation
...
make coordinator db polling for list of segments more robust
2016-03-08 12:43:49 -08:00
e7018f524f
Merge pull request #2598 from himanshug/handoff_timeout
...
optional ability to configure handoff wait timeout on realtime tasks
2016-03-08 12:43:36 -08:00
c72438ead0
override metric name
2016-03-08 10:58:12 -06:00
1288784bde
in coordinator db polling for available segments, ignore corrupted entries in segments table so that coordinator continues to load new segments even if there are few corrupted segment entries
2016-03-07 15:13:10 -06:00
0402636598
configurable handoffConditionTimeout in realtime tasks for segment handoff wait
2016-03-05 10:14:54 -06:00
d06c1c5c85
Merge pull request #2583 from guobingkun/fix_multiple_specs_2
...
update querySegmentSpec when passing query to getQueryRunner
2016-03-02 18:05:34 -08:00
9e74772d6b
Merge pull request #2574 from gianm/allostuff
...
Make first few allocatePendingSegment retries quiet.
2016-03-02 16:16:53 -07:00
cfe2dbf1eb
Merge pull request #2580 from gianm/rtc-basePersist
...
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
2016-03-02 16:56:49 -06:00
4a58462fc7
update querySegmentSpec when passing query to getQueryRunner
...
After finding the FireChief for a specific partition, Druid will need to find the specific queryRunner for each segment being queried by passing the query to FireChief. Currently Druid is passing the original query that contains all the segments need to be queried, it's possible that fireChief.getQueryRunner(query) returns more than 1 queryRunner because query.getIntervals() is not specific to a single segment.
In this patch, for each segment being queried, Druid will update the query with its corresponding SpecificSegmentSpec.
2016-03-02 16:44:56 -06:00
e65e6a49a5
RealtimeTuningConfig: Use different default basePersistDirectory per instance.
2016-03-02 13:57:53 -08:00
004028b887
Make first few allocatePendingSegment retries quiet.
...
Some light retrying can happen during normal operation (SELECT -> INSERT races) and the
ensuing log messages would be scary for users.
2016-03-02 13:40:29 -08:00
612e327426
Merge pull request #2581 from gianm/fix-deadlock
...
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
2016-03-02 11:37:49 -08:00
7557eb2800
CliPeon: Fix deadlock on startup by eagerly creating ExecutorLifecycle, ChatHandlerResource.
...
See stack traces here, from current master: https://gist.github.com/gianm/bd9a66c826995f97fc8f
1. The thread "qtp925672150-62" holds the lock on InternalInjectorCreator.class,
used by Scopes.SINGLETON, and wants the lock on "handlers" in Lifecycle.addMaybeStartHandler
called by DiscoveryModule.getServiceAnnouncer.
2. The main thread holds the lock on "handlers" in Lifecycle.addMaybeStartHandler, which it
took because it's trying to add the ExecutorLifecycle to the lifecycle. main is trying
to get the InternalInjectorCreator.class lock because it's running ExecutorLifecycle.start,
which does some Jackson deserialization, and Jackson needs that lock in order to inject
stuff into the Task it's deserializing.
This patch eagerly instantiates ChatHandlerResource (which I believe is what's trying to
create the ServiceAnnouncer in the qtp925672150-62 jetty thread) and the ExecutorLifecycle.
2016-03-02 10:53:42 -08:00
102fc92120
SQLMetadataConnector: Fix overzealous retries that were preventing EntryExistsException from making it out.
2016-03-01 17:20:33 -08:00
9340cae985
Merge pull request #2457 from bjozet/docs/fixes
...
Default value for maxRowsInMemory
2016-03-01 07:43:26 -08:00
2462c82c0e
New defaults for maxRowsInMemory rowFlushBoundary
...
To bring consistency to docs and source this commit changes the default
values for maxRowsInMemory and rowFlushBoundary to 75000 after
discussion in PR https://github.com/druid-io/druid/pull/2457 .
The previous default was 500000 and it's lower now on the grounds that
it's better for a default to be somewhat less efficient, and work,
than to reach for the stars and possibly result in
"OutOfMemoryError: java heap space" errors.
2016-03-01 13:50:28 +01:00
4edcb1b861
Refactor FireChief + UTs for RealtimeManagerTest
...
Add tests that verify whether RealtimeManager is querying the correct FireChief for a specific partition
make FireChief static and package private, add latches in the UT
2016-02-29 14:41:10 -06:00
68631d89e9
Allow realtime nodes to have multiple shards of the same datasource
2016-02-29 12:30:25 -06:00
6b3c96c63a
better exception for invalid interval
2016-02-26 10:02:38 -06:00
29d29ba98d
Merge pull request #2263 from jon-wei/flex_dims3
...
Allow IncrementalIndex to store Long/Float dimensions
2016-02-25 17:23:02 -08:00
b331fb4a83
Fix parsing of druid.indexer.server.maxChatRequests.
2016-02-25 14:47:15 -08:00
b82b487f20
remove extra kill parameter
2016-02-24 17:16:18 -06:00
c17ce02467
Allow IncrementalIndex to store Long/Float dimensions
2016-02-24 13:51:57 -08:00
a3b37e9225
In persistAndMerge, increase the scope of try-catch block so that any exception while persisting hydrants is caught and consequently that sink is abandoned or the task will forever wait for handoff to happen.
2016-02-23 22:22:33 -06:00
6c9e1a28ad
Merge pull request #2519 from gianm/unparseable-handling
...
Better handling of ParseExceptions.
2016-02-24 04:46:29 +05:30
93540c0631
Merge pull request #2503 from gianm/jetty-qos
...
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
2016-02-23 10:35:53 -08:00
3534483433
Better handling of ParseExceptions.
...
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".
Behavior of the counters should now be:
- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).
If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).
Fixes #2510 .
2016-02-23 10:11:43 -08:00
0c984f9e32
Merge pull request #2109 from himanshug/segments_in_delta_ingestion
...
idempotent batch delta ingestion
2016-02-22 14:00:45 -08:00
3bdd757024
Merge pull request #1773 from b-slim/log_details
...
Adding downstream source when throwing QueryInterruptedException
2016-02-22 10:16:07 -08:00
21b0b8a07d
new coordinator endpoint to get list of used segment given a dataSource and list of intervals
2016-02-21 23:17:58 -06:00
77925cc061
adding downstream source of QueryInterruptedException
2016-02-20 13:05:14 -06:00
23c993c9e7
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
...
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
2016-02-19 13:36:09 -08:00
243ac5399b
Harmonize realtime indexing loop across the task and standalone nodes.
...
- Both now catch ParseExceptions on plumber.add (see https://groups.google.com/d/topic/druid-user/wmiRDvx2RvM/discussion )
- Standalone now treats IndexSizeExceededException as fatal (previously only the task did)
2016-02-19 07:34:15 -08:00
e0c049c0b0
Make startup properties logging optional.
...
Off by default, but enabled in the example config files. See also #2452 .
2016-02-12 14:12:16 -08:00
1430bc2c88
Merge pull request #2276 from harshjain2/feature-2021
...
Fix for issue 2021.
2016-02-10 17:04:45 -08:00
fa92b77f5a
Harmonize znode writing code in RTR and Worker.
...
- Throw most exceptions rather than suppressing them, which should help
detect problems. Continue suppressing exceptions that make sense to
suppress.
- Handle payload length checks consistently, and improve error message.
- Remove unused WorkerCuratorCoordinator.announceTaskAnnouncement method.
- Max znode length should be int, not long.
- Add tests.
2016-02-10 14:52:00 -08:00
a3eb863c8e
Fix for issue 2021
2016-02-10 22:19:12 +05:30
d1cb17d3f7
at broker - only add segments from specific tiers to the timeline
2016-02-09 22:33:22 -06:00
b40c342cd1
make Global stupid pool cache size configurable
2016-02-05 14:18:06 -06:00
9002548eeb
increase test time out and general clean up
2016-02-03 13:26:37 -06:00
5111fd52f2
Add check for log4j-core in Log4jShutterDownerModule
2016-02-02 15:56:48 -08:00
dc89cdd0f9
Merge pull request #2336 from himanshug/fix_2331
...
limit size of X-Druid-Response-Context header to 7K
2016-02-02 12:06:59 -06:00
c03918f89a
AsyncQueryForwardingServletTest#testDeleteBroadcast sometimes fails by port conflict
2016-01-29 19:28:58 +09:00
f6b4dbd697
bug fix and unit tests for DruidCoordinatorSegmentKiller
2016-01-28 14:10:17 -06:00
ab3edfa8fc
moving DruidCoordinatorSegmentKiller class out of DruidCoordinator
2016-01-28 14:03:56 -06:00
3880f54b87
Merge pull request #2332 from himanshug/configurable_partial
...
make populateUncoveredIntervals a configuration in query context
2016-01-28 10:34:35 +05:30
a7bde8f4da
limit size of X-Druid-Response-Context header to 7K due to https://github.com/druid-io/druid/issues/2331
2016-01-27 15:18:08 -06:00
5a3642bb93
Merge pull request #2247 from metamx/pedanticBuild
...
Enable strict building in travis
2016-01-27 10:27:03 -08:00
508734c8b0
Long constant reformatting in tests `l` --> `L`
2016-01-27 08:59:19 -08:00
fd6bf3fe22
Use interval comparator instead of bucketMonthComparator
...
fix when two segments have same interval
review comments
2016-01-27 17:35:43 +05:30
3719b6e3c8
make populateUncoveredIntervals a configuration in query context
2016-01-26 15:13:45 -06:00
41730b96d4
Fix for issue 2021
2016-01-25 02:48:22 +05:30
7a6109f0ca
Merge pull request #2321 from gianm/pending-index
...
Replace two-column index on pendingSegments table with one-column index.
2016-01-22 13:38:15 -06:00
0bd9bff075
Replace two-column index on pendingSegments table with one-column index.
...
Fixes #2319 .
2016-01-22 10:50:21 -08:00
1a5d4e714c
Adding custom mapper for json processing exception to return bad request instead of 500
2016-01-22 09:48:52 -08:00
04d3054353
Merge pull request #2303 from CHOIJAEHONG1/localfirehouse-basedir-npe
...
Throw an IAE when baseDir is null in LocalFireHose
2016-01-21 07:58:52 -08:00
dcb7830330
Merge pull request #984 from drcrallen/thread-priority-rebase
...
Use thread priorities. (aka set `nice` values for background-like tasks)
2016-01-21 15:02:34 +05:30
2e1d6aaf3d
Use thread priorities. (aka set `nice` values for background-like tasks)
...
* Defaults the thread priority to java.util.Thread.NORM_PRIORITY in io.druid.indexing.common.task.AbstractTask
* Each exec service has its own Task Factory which is assigned a priority for spawned task. Therefore each priority class has a unique exec service
* Added priority to tasks as taskPriority in the task context. <0 means low, 0 means take default, >0 means high. It is up to any particular implementation to determine how to handle these numbers
* Add options to ForkingTaskRunner
* Add "-XX:+UseThreadPriorities" default option
* Add "-XX:ThreadPriorityPolicy=42" default option
* AbstractTask - Removed unneded @JsonIgnore on priority
* Added priority to RealtimePlumber executors. All sub-executors (non query runners) get Thread.MIN_PRIORITY
* Add persistThreadPriority and mergeThreadPriority to realtime tuning config
2016-01-20 14:00:31 -08:00
61aca6f9cc
remove wrong checks
...
sink never have null hydrants and hydrants never have null adapters
2016-01-20 23:43:53 +05:30
7132428bba
throw IAE when baseDir is null in LocalFireHose
2016-01-21 01:27:32 +09:00
59ea186af7
fix reference counting for segments
2016-01-20 17:24:21 +05:30
747343e621
Preserve dimension order across indexes during ingestion
2016-01-19 13:34:11 -08:00
df2906a91c
Merge pull request #2290 from gianm/index-merger-v9-stuff
...
Respect buildV9Directly in PlumberSchools, so it works on standalone realtime.
2016-01-19 13:04:00 -08:00
0c31f007fc
Merge pull request #1728 from himanshug/aggregators_in_segment_metadata
...
Store AggregatorFactory[] in segment metadata
2016-01-19 12:55:49 -08:00
fe841fd961
Merge pull request #2118 from guobingkun/fix_segment_loading
...
Fix loading segment for historical
2016-01-19 14:25:48 -06:00
a99aef29a1
adding aggregators to segment metadata
2016-01-19 14:23:39 -06:00
1dcf22edb7
Respect buildV9Directly in PlumberSchools, so it works on standalone realtime nodes.
...
Also parameterize some tests to run with/without buildV9Directly:
- IndexGeneratorJobTest
- RealtimeIndexTaskTest
- RealtimePlumberSchoolTest
2016-01-19 12:15:06 -08:00
c4ad50f92c
Fix loading segment for historical
...
Historical will drop a segment that shouldn't be dropped in the following scenario:
Historical node tried to load segmentA, but failed with SegmentLoadingException,
then ZkCoordinator called removeSegment(segmentA, blah) to schedule a runnable that would drop segmentA by deleting its files. Now, before that runnable executed, another LOAD request was sent to this historical, this time historical actually succeeded on loading segmentA and announced it. But later on, the scheduled drop-of-segment runnable started executing and removed the segment files, while historical is still announcing segmentA.
2016-01-19 10:29:49 -06:00
164b0aad7a
removing Map<String,Object> segmentMetadata from methods in Index[Maker/Merger] and using Metadata class
...
instead of a Map to store segment metadata
2016-01-18 22:03:46 -06:00
637d2605e7
kill unwanted parent directories when a segment is deleted from LocalDataSegmentKiller
2016-01-18 21:51:04 -06:00
71f1cd5e34
Fix for issue 2021
2016-01-17 16:10:04 +05:30
f6a1a4ae20
Merge pull request #2138 from KurtYoung/feature-build-v9
...
build v9 directly
2016-01-16 13:35:46 -06:00
82ff98c2bf
add config for build v9 directly and update docs
2016-01-16 11:26:34 +08:00
2e54553a8f
Merge pull request #1990 from himanshug/schedule_kill_task
...
support periodic hard delete of segments
2016-01-15 15:22:33 -06:00
7c65880e55
Merge pull request #2270 from rasahner/warnOfChatHandlerNoop
...
if chathandler is noop, log using warn instead of info
2016-01-15 13:29:27 -07:00
6ec6835b5d
Fix for issue 2021.
2016-01-16 00:58:33 +05:30
34cd8f8c72
Merge pull request #2258 from fjy/acl-zk
...
acl for zookeeper is added
2016-01-15 10:27:08 -07:00
a54c726726
Merge pull request #2266 from anubhgup/fix-announce
...
Fix loss in segment announcements when segments do not fit in zNode
2016-01-14 17:10:43 -08:00
dc1a62c3d9
Merge pull request #2248 from metamx/druidNodeHashEquals
...
Add hashCode and equals to DruidNode
2016-01-14 16:04:58 -08:00
6d09ab839f
Fix for loss in segment announcements when segments do not fit in the znodes during compress mode.
...
Added unit test (from Navis).
2016-01-14 14:44:23 -08:00
7361cd173f
if chathandler is noop, log using warn instead of info
2016-01-14 10:12:42 -06:00
1908d63162
acl for zookeeper is added
2016-01-13 14:56:05 -08:00
18479bb757
time-descending result of timeseries queries
2016-01-13 12:23:01 +09:00
eb2d251ac8
support periodic hard delete of segments
2016-01-12 16:55:05 -06:00
01a0715ee2
Merge pull request #2161 from metamx/query-metrics-timeout
...
Fix Query metrics for query timeout
2016-01-12 09:50:40 -06:00
ef3cddabe9
adding UTs for DruidCoordinatorConfig
2016-01-11 22:03:25 -06:00
b0e04a9162
Add hashCode and equals to DruidNode
2016-01-11 15:23:45 -08:00
ea623e43d2
Merge pull request #2240 from metamx/fix-load-rule
...
Fix loadRule when one of the tiers had no available servers
2016-01-11 10:05:31 -08:00
32bc2f776e
Fix loadRule when one of the tiers had no servers
...
When one of the tiers have no servers, LoadRule should ignore that tier
and continue to load/drop segments in other available tiers.
the bug also causes whacky behavior with LoadRule with non existent
tier where the segment balancer keeps on moving segments to other nodes
in existing tiers but the extra segment copies are never dropped
eventually leading to all the tiers getting full .
2016-01-11 15:53:14 +05:30
d255f4baac
Merge pull request #2234 from pjain1/emit_realtime_metrics
...
emit handoff count metrics
2016-01-08 14:24:16 -06:00
9dba0f67e7
emit handoff count metrics
2016-01-08 12:36:13 -06:00
15fc070232
Merge pull request #2213 from himanshug/fix_curtator_test_base
...
[wip] trying/finding fix for announcer test failures
2016-01-07 18:23:49 -08:00
1bfb4e3988
Emit query/time for failed and timeout queries
...
emit query/time metric also
add success flag
fix success flag for router metrics
review comments
formatting.
2016-01-07 19:41:54 +05:30
7ab810f3eb
do not ignore exceptions from curator cleanup in CuratorTestBase
2016-01-06 10:42:53 -06:00
14989f272d
Add metrics for ingest/bytes/received for EventReceiverFirehose
...
review comments
review comments
2016-01-05 20:06:09 +05:30
57d91d754d
Comment out buggy unit tests, fix #2185
2016-01-03 09:50:16 -08:00
8424b2b456
Fix announcer test bad check
2016-01-01 19:55:22 -08:00
fa5c3bb014
adding decorate(DimensionSelector) to DimensionSpec to enable support for arbitrary filtering/transformations to returned dimension values
2015-12-30 15:06:24 -06:00
3c107c5757
Merge pull request #2150 from himanshug/emit_query_bytes
...
emit query/bytes metric
2015-12-30 13:44:19 -06:00
7ffa706655
Merge pull request #2152 from metamx/add-taskId
...
Add taskId to realtimeMetrics
2015-12-29 10:33:40 -08:00
b1261035a7
Merge pull request #1861 from guobingkun/insert_segment_tool
...
insert-segment tool
2015-12-29 10:06:07 -08:00
38b0f1fbc2
fix transient failures in unit tests
2015-12-28 20:03:30 -08:00
d94821998f
Merge pull request #2153 from navis/fix-testfail-DruidCoordinatorTest
...
Another try to fix DruidCoordinatorTest#testCoordinatorRun
2015-12-27 17:03:46 -08:00
10c19fe9f8
Another try to fix DruidCoordinatorTest#testCoordinatorRun
2015-12-24 03:02:37 +09:00
2c3c4a3f8f
Another try to fix xxServerViewTests
2015-12-24 02:13:40 +09:00
508629916d
Merge pull request #2137 from guobingkun/fix_serverview_transient_error2
...
[Don't merge]Another effort on fixing the transient error for XXXServerViewTest
2015-12-23 07:33:54 -08:00
978a3fd8ae
Add taskId to realtimeMetrics
...
Add task Id to Realtime Metrics
2015-12-23 18:05:25 +05:30
1a8546a682
emit query/bytes metric
2015-12-23 00:29:44 -06:00
89e2a9a6d2
build at Tue Dec 22 21:30:00 CST 2015
2015-12-22 21:30:00 -06:00
a110eafff7
build at Tue Dec 22 17:30:00 CST 2015
2015-12-22 17:30:00 -06:00
4b93528334
build at Tue Dec 22 17:00:01 CST 2015
2015-12-22 17:00:01 -06:00
b8d9860dbc
build at Tue Dec 22 16:00:00 CST 2015
2015-12-22 16:00:00 -06:00
ffdb9407e5
build at Tue Dec 22 15:30:00 CST 2015
2015-12-22 15:30:00 -06:00
6f54c1b60d
build at Tue Dec 22 15:00:00 CST 2015
2015-12-22 15:00:01 -06:00
4e61261c68
build at Tue Dec 22 14:30:00 CST 2015
2015-12-22 14:30:00 -06:00
4961b8c0bd
build at Tue Dec 22 14:00:00 CST 2015
2015-12-22 14:00:00 -06:00
0a635a11d6
build at Tue Dec 22 13:40:00 CST 2015
2015-12-22 13:40:00 -06:00
05c9e1b598
Reorder Before/After in JDBCExtractionNamespaceTest
...
* Fixes https://github.com/druid-io/druid/issues/2120
2015-12-22 11:39:46 -08:00
849ab2d958
build at
2015-12-22 13:08:29 -06:00
455980d659
address another node exists failure
2015-12-22 13:06:20 -06:00
2ffeda5d25
build number 13
2015-12-22 10:19:39 -06:00
7e64d5179f
Merge pull request #2139 from himanshug/emit_query_node_bytes
...
emit query/node/bytes metric
2015-12-22 08:07:04 -08:00
bc573484f3
build number 12
2015-12-22 00:52:58 -06:00
7b5fd76058
Merge pull request #2119 from navis/fix-sporadic-testfail2
...
Fix sporadic fail of DruidCoordinatorTest#testCoordinatorRun
2015-12-21 23:28:32 -06:00
b96f560255
emit query/node/bytes metric
2015-12-21 23:23:20 -06:00
06c220ae7b
build number 11
2015-12-21 19:40:47 -06:00
93a91e5633
build number 10
2015-12-21 18:21:15 -06:00
471dd7e164
build number 9
2015-12-21 17:35:18 -06:00
627c311923
build #8
2015-12-21 17:06:37 -06:00
293f2ad65d
build #7
2015-12-21 16:09:19 -06:00
9ed843ea21
build #6
2015-12-21 15:41:23 -06:00
f8342ca240
build #5
2015-12-21 15:10:37 -06:00
52261c5539
build #4
2015-12-21 14:41:38 -06:00
47a8f2d3f1
build #3
2015-12-21 13:56:43 -06:00
5cc56cf9e7
build #2
2015-12-21 13:05:38 -06:00
f12cfa99fa
another effort on fixing the transient error for Broker/CoordinatorServerViewTest happend on Travis build
2015-12-21 12:31:36 -06:00
c2a2d19d62
Merge pull request #2126 from guobingkun/remove_single_announcer
...
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
2015-12-21 08:55:07 -08:00
7b24d1f355
Fix sporadic fail of DruidCoordinatorTest#testCoordinatorRun
2015-12-21 17:00:52 +09:00
951a4e9b35
Remove SingleDataSegmentAnnouncer in favor of BatchDataSegmentAnnouncer
2015-12-21 00:05:53 -06:00
89b477970f
DataSegmentFinder tool
...
`insert-segment-to-db` is a tool that can insert segments into Druid metadata storage. It is intended to be used
to update the segment table in metadata storage after people manually migrate segments from one place to another.
It can also be used to insert missing segment into Druid, or even recover metadata storage by telling it where the
segments are stored.
Note: This tool expects users to have Druid cluster running in a "safe" mode, where there are no active tasks to interfere
the segments being inserted. Users can optionally bring down the cluster to make 100% sure nothing is interfering.
2015-12-21 00:02:04 -06:00
ad3312171b
Fix sporadic fail of io.druid.client.CoordinatorServerViewTest
2015-12-21 11:39:32 +09:00
7019d3c421
Merge pull request #2107 from jon-wei/fix_smq
...
More efficient SegmentMetadataQuery
2015-12-18 16:40:47 -08:00
e52c9d2b3c
Merge pull request #2095 from guobingkun/fix_client_merge_task_and_serde
...
Fix getType() and Json serialization in ClientMergeQuery + serde tests
2015-12-18 11:40:13 -08:00
356b07c6c3
More efficient SegmentMetadataQuery
2015-12-17 12:46:23 -08:00
1e5aa2f3ac
fix getType() and Json serialization in ClientMergeQuery and add serde tests
2015-12-15 12:08:43 -06:00
e6c2db89da
Merge pull request #1921 from jon-wei/flat_json
...
Add docs and benchmark for JSON flattening parser
2015-12-14 20:53:54 -08:00
a2d0bea223
Merge pull request #2074 from guobingkun/add_comment
...
Add comment to BrokerServerViewTest
2015-12-09 16:19:27 -08:00
c53bf85d83
Add docs and benchmark for JSON flattening parser
2015-12-09 16:13:30 -08:00
31502a920c
Add comment to BrokerServerViewTest
...
Explain why we need to do explicit checks on existences of zNodes before we create them. Address issue: https://github.com/druid-io/druid/issues/1512
2015-12-09 14:01:07 -06:00
ce79d707dd
Merge pull request #2072 from guobingkun/fix_transient_error_brokerserverviewtest
...
An effort to fix transient error in BrokerServerViewTest
2015-12-09 11:12:35 -08:00
161c4ce288
An effort to fix transient error in BrokerServerViewTest
2015-12-09 10:50:13 -06:00
c4de26ef09
fix typo in InitializationTest.java
2015-12-09 00:11:03 -06:00
5bc6d61a25
Fix InitializationTest
2015-12-08 17:37:56 -08:00
dcd15735cd
Merge pull request #2015 from metamx/handoff-notifier-coordinator
...
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoff information
2015-12-08 14:06:39 -08:00
a32906c7fd
Remove FilteredServerView
2015-12-09 01:54:12 +05:30
9491e8de3b
Remove ServerView from RealtimeIndexTasks and use coordinator http endpoint for handoffs
...
- fixes #1970
- extracted out segment handoff callbacks in SegmentHandoffNotifier
which is responsible for tracking segment handoffs and doing callbacks
when handoff is complete.
- Coordinator now maintains a view of segments in the cluster, this
will affect the jam heap requirements for the overlord for large
clusters.
realtime index task and nodes now use HTTP end points exposed by the
coordinator to get serverView
review comment
fix realtime node guide injection
review comments
make test not rely on scheduled exec
fix compilation
fix import
review comment
introduce immutableSegmentLoadInfo
fix son reading
remove unnecessary logging
2015-12-09 01:54:09 +05:30
56fd48c0e4
Merge pull request #2056 from guobingkun/fix_server_selector2
...
Bug fix: stale segment in ServerSelector
2015-12-07 16:57:33 -08:00
ba980dcadb
Merge pull request #2058 from cheddar/report-uncovered-segments
...
Add "uncoveredIntervals" to responseContext
2015-12-07 16:42:42 -08:00
a36185926e
Add "uncoveredIntervals" to responseContext
...
This change will cause the CachingClusteredClient to populate the "uncoveredIntervals"
key in the responseContext map. The value will be any intervals that were requested in
the query but are not actually covered by the segments underlying the data source.
For unit testing, CachingClisteredClientTest is testing the caching behavior of the
object and it is pretty hard to adjust it to only test this new behavior, so I created
a new, parallel "CachingClusteredClientFunctionalityTest" to simplify testing just
basic functionality.
2015-12-07 16:16:57 -08:00
970b9cfcaa
fix stale segment in ServerSelector
2015-12-07 14:04:34 -06:00
00416e26ff
Merge pull request #2012 from himanshug/add_comment
...
adding more comments to why drops do not happen
2015-12-07 11:50:06 -08:00
62ba9ade37
unifying license header in all java files
2015-12-05 22:16:23 -06:00
3781aa380f
Added an injection test for ZkCoordinator.
2015-12-05 11:56:22 +08:00
221fb95d07
add support for getting used segments for multiple interval in IndexerMetadataStorageCoordinator
2015-12-03 21:28:04 -06:00
1eb8211346
Add datasource and taskId to metrics emitted by peons
...
This PR adds the datasource and taskId to the jvm and sys metrics
emitted by the peons.
fix spelling
review comment
review comment
2015-12-01 23:20:59 +05:30
Himanshu Gupta
Gian Merlino
kilida
Fangjin Yang
Bingkun Guo
Parag Jain
binlijin
jon-wei
Xavier Léauté
Charles Allen
Jonathan Wei
navis.ryu
Nishant
rasahner
Erik Dubbelboer
Slim Bouguerra
David Lim
Björn Zettergren
Eric Tschetter
Harsh Jain
Xavier Léauté
Himank Chaudhary
Charles Allen
Jaehong Choi
Kurt Young
Anubhav Gupta
Robin
Nikita Geer
刘麒贇