b331fb4a83
Fix parsing of druid.indexer.server.maxChatRequests.
2016-02-25 14:47:15 -08:00
a8820d837b
More logging for JDBCExtractionNamespaceFunctionFactory
2016-02-25 09:48:56 -08:00
4c522a8159
Merge pull request #2543 from metamx/fixUnregisterChildPorts
...
UnRegister port in ForkingTaskRunner
2016-02-25 22:32:31 +05:30
3fa7a7ebfe
Limit PortFinder to 0xFFFF
2016-02-25 08:16:40 -08:00
187b788089
UnRegister port in ForkingTaskRunner
2016-02-25 08:04:25 -08:00
cf0bc905fb
Plumb task peon host/ports back out to the overlord.
...
- Add TaskLocation class
- Add registerListener to TaskRunner
- Add getLocation to TaskRunnerWorkItem
- Implement location tracking in existing TaskRunners
- Rework WorkerTaskMonitor to do management out of a single thread so it can
handle status and location updates more simply.
2016-02-24 15:13:10 -08:00
917e7e58f5
Merge pull request #2536 from jon-wei/regex_extraction_rename
...
Rename 'replaceMissingValues...' parameters in RegexExtractionFn
2016-02-24 14:41:06 -08:00
c17ce02467
Allow IncrementalIndex to store Long/Float dimensions
2016-02-24 13:51:57 -08:00
fd3782522c
Rename 'replaceMissingValues...' parameters in RegexExtractionFn
2016-02-24 13:12:56 -08:00
6ac32c5518
Merge pull request #2530 from himanshug/fix_plumber
...
In persistAndMerge, increase the scope of try-catch block so that any…
2016-02-24 22:41:32 +05:30
fb7eae34ed
Merge pull request #2249 from metamx/workerExpanded
...
Use Worker instead of ZkWorker whenever possible
2016-02-24 13:23:22 +05:30
a3b37e9225
In persistAndMerge, increase the scope of try-catch block so that any exception while persisting hydrants is caught and consequently that sink is abandoned or the task will forever wait for handoff to happen.
2016-02-23 22:22:33 -06:00
6c9e1a28ad
Merge pull request #2519 from gianm/unparseable-handling
...
Better handling of ParseExceptions.
2016-02-24 04:46:29 +05:30
ac13a5942a
Use Worker instead of ZkWorker whenver possible
...
* Moves last run task state information to Worker
* Makes WorkerTaskRunner a TaskRunner which has interfaces to help with getting information about a Worker
2016-02-23 15:02:03 -08:00
93540c0631
Merge pull request #2503 from gianm/jetty-qos
...
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
2016-02-23 10:35:53 -08:00
4f300cfe49
Merge pull request #2526 from druid-io/b-slim-patch-1
...
fix docs about sketches
2016-02-23 10:23:53 -08:00
3534483433
Better handling of ParseExceptions.
...
Two changes:
- Allow IncrementalIndex to suppress ParseExceptions on "aggregate".
- Add "reportParseExceptions" option to realtime tuning configs. By default this is "false".
Behavior of the counters should now be:
- processed: Number of rows indexed, including rows where some fields could be parsed and some could not.
- thrownAway: Number of rows thrown away due to rejection policy.
- unparseable: Number of rows thrown away due to being completely unparseable (no fields salvageable at all).
If "reportParseExceptions" is true then "unparseable" will always be zero (because a parse error would
cause an exception to be thrown). In addition, "processed" will only include fully parseable rows
(because even partial parse failures will cause exceptions to be thrown).
Fixes #2510 .
2016-02-23 10:11:43 -08:00
11c9a63ee3
Merge pull request #2522 from himanshug/doc_update
...
fix thetaSketchSetOp doc
2016-02-23 09:33:55 -08:00
86c4900347
fix thetaSketch post aggregator doc
2016-02-23 10:43:54 -06:00
c7cb5bff14
fix thetaSketchSetOp doc
2016-02-23 09:17:49 -06:00
0c984f9e32
Merge pull request #2109 from himanshug/segments_in_delta_ingestion
...
idempotent batch delta ingestion
2016-02-22 14:00:45 -08:00
79614bd159
Merge pull request #2514 from himanshug/doc_update
...
updating thetaSketchSetOp post agg documentation to reflect the possibility of nesting
2016-02-22 13:13:58 -06:00
3bdd757024
Merge pull request #1773 from b-slim/log_details
...
Adding downstream source when throwing QueryInterruptedException
2016-02-22 10:16:07 -08:00
f7679dd5a9
updating thetaSketchSetOp post agg documentation to reflect the possibility of nesting
2016-02-22 09:38:58 -06:00
21b0b8a07d
new coordinator endpoint to get list of used segment given a dataSource and list of intervals
2016-02-21 23:17:58 -06:00
09ffcae4ae
give user the option to specify the segments for dataSource inputSpec
2016-02-21 23:15:31 -06:00
77925cc061
adding downstream source of QueryInterruptedException
2016-02-20 13:05:14 -06:00
8ee81947cd
Merge pull request #2494 from himanshug/fix_timeseries
...
do not drop post-aggs in TimeseriesQueryToolChest.makePreComputeManipulatorFn
2016-02-20 10:37:32 -08:00
23c993c9e7
Add druid.indexer.server.maxChatRequests for QoS; deprecate separate ports.
...
- Add druid.indexer.server.maxChatRequests, which sets up a QoSFilter on the main Jetty server.
- Deprecate druid.indexer.runner.separateIngestionEndpoint
- Deprecate druid.indexer.server.chathandler.*
2016-02-19 13:36:09 -08:00
8a41f03377
Merge pull request #2496 from gianm/hll-finalizing-comparator
...
Add comparator to HyperUniquesFinalizingPostAggregator.
2016-02-19 13:03:43 -08:00
499288ff4b
Merge pull request #2509 from metamx/hadoopIsolatorTest
...
Add hadoop classloader isolation tests for HadoopTask
2016-02-19 14:23:22 -06:00
a3c29b91cc
Merge pull request #2505 from gianm/rt-exceptions
...
Harmonize realtime indexing loop across the task and standalone nodes.
2016-02-19 11:23:14 -08:00
9dff0e5dbd
Add hadoop classloader isolation tests for HadoopTask
2016-02-19 11:15:53 -08:00
44e925c98f
Merge pull request #2506 from metamx/failTravisFaster
...
Fail travis builds faster
2016-02-19 11:01:37 -08:00
ddf913d626
Merge pull request #2508 from gianm/ftr-shutdown-logging
...
ForkingTaskRunner: Better logging during orderly shutdown.
2016-02-19 10:02:24 -08:00
9acc2834ff
Merge pull request #2501 from metamx/forwardport2500
...
Make HadoopTasks's classloader a single one
2016-02-19 09:56:38 -08:00
c0c6cf77fa
ForkingTaskRunner: Better logging during orderly shutdown.
2016-02-19 09:17:16 -08:00
d25c46cb9f
Add comparator to HyperUniquesFinalizingPostAggregator.
...
This makes it possible to do groupBys with clauses like "HAVING uniques > 10".
Beforehand you couldn't do it with either an aggregator (because it returns
an HLLV1 which the havingSpec can't understand) or a finalized postaggregator
(because it didn't have a comparator).
Now you can at least do it with a finalizing postaggregator. Trying it with
the aggregator alone still doesn't work.
Added some topN and groupBy tests verifying the comparator, and added an
@Ignore test that should pass if havingSpecs are made work on the aggregator
directly.
2016-02-19 08:36:08 -08:00
dd3cf93430
Fail travis builds faster
2016-02-19 07:38:06 -08:00
243ac5399b
Harmonize realtime indexing loop across the task and standalone nodes.
...
- Both now catch ParseExceptions on plumber.add (see https://groups.google.com/d/topic/druid-user/wmiRDvx2RvM/discussion )
- Standalone now treats IndexSizeExceededException as fatal (previously only the task did)
2016-02-19 07:34:15 -08:00
4725522759
update middlemanager.conf
2016-02-18 21:47:50 -08:00
87752be740
Make HadoopTasks's classloader a single one
2016-02-18 20:58:09 -08:00
7e5d9a84f9
Merge pull request #2498 from guobingkun/fix_tarball
...
generate tarball by mvn package
2016-02-18 15:57:20 -08:00
9e4c908922
generate tarball by mvn package
2016-02-18 16:42:41 -06:00
5ba8893fa3
Merge pull request #2490 from guobingkun/better_printing
...
Integration tests: print expected results in json
2016-02-18 09:36:13 -06:00
72ab621c2a
Integration tests: print expected results in json format
2016-02-18 00:14:49 -06:00
11b0117422
do not drop post-aggs in timeseries query tool chest makePreComputeManipulatorFn like other query types
2016-02-17 20:51:35 -06:00
083f019a48
Merge pull request #2465 from druid-io/more-doc-fix
...
more doc fixes
2016-02-17 11:00:38 -08:00
1c4cfd5829
Merge pull request #2485 from rasahner/DOC_fixDockerLink
...
integration-tests/README: fix link to docker-install
2016-02-17 12:10:55 -06:00
661c3aac4f
Merge pull request #2483 from druid-io/b-slim-patch-1
...
clarification about monitoring period
2016-02-17 09:46:25 -08:00
Gian Merlino
Charles Allen
Nishant
jon-wei
Himanshu Gupta
Fangjin Yang
Slim
Slim Bouguerra
Bingkun Guo
pdeva
Parag Jain