Commit Graph

5854 Commits

Author SHA1 Message Date
Fangjin Yang 7b649f6993 Merge pull request #1305 from davrodpin/azure-storage-overall-improvements
Overall improvement on Azure Deep Storage extension.
2015-05-05 22:26:58 -07:00
David Rodrigues 11a76169b4 Overall improvement on Azure Deep Storage extension.
* Remove hard-coded azure path manipulation from the puller.
  * Fix segment size not being zero after uploading it do Azure.
  * Remove both index and desc files only on a success upload to Azure.
  * Add Azure container name to load spec.
      This patch would help future-proof azure deep-storage module and avoid
      having to introduce ugly backwards-compatibility fixes when we want to
      support multiple containers or moving data between containers.
2015-05-05 15:17:25 -07:00
Fangjin Yang 7c4054aaa3 Merge pull request #1343 from metamx/fix-npe
fix NPE
2015-05-05 11:02:44 -07:00
nishant 34be1e96fa fix NPE
review comments

Add test

fix test for java8
2015-05-05 23:11:13 +05:30
Fangjin Yang 7608bf5799 Merge pull request #1342 from guobingkun/unit_test
Fix a potential issue in DeterminePartitionsJob, and two unit tests, reformat code
2015-05-05 09:24:23 -07:00
Bingkun Guo 1ee550dd91 Fix a potential issue in DeterminePartitionsJob by making HadoopDruidIndexerConfig non-static, and two unit tests for DeterminPartitionsJob and LocalDataSegmentKiller 2015-05-04 20:00:29 -07:00
Fangjin Yang d7562fd4d1 Merge pull request #1337 from gianm/delay-firehose-connect
Realtime: Delay firehose connection until job is started.
2015-05-04 17:18:28 -07:00
Gian Merlino e69d82a2b4 Realtime: Delay firehose connection until job is started.
Some firehoses (like the Kafka firehose) acquire input resources when they
connect, so it helps to delay this until after plumber.startJob() runs.
2015-05-04 10:54:07 -07:00
Gian Merlino 8eb441ec08 RealtimeManager: Include partitionNum in chief thread name. 2015-05-04 10:53:48 -07:00
Neo 8f8400e24e fix handling of dimension having only null values
fixes #1211

fix value matcher

more improvements

more fixes for partial null column

fix handling of dimension having only null values

fixes #1211

fix value matcher

more improvements

more fixes for partial null column

review comment

IndexMaker speedups
* About 15% speedup

Conflicts:
	processing/src/main/java/io/druid/segment/IndexMaker.java

fix handling of dimension having only null values

fixes #1211

fix value matcher

more improvements

more fixes for partial null column

fix handling of dimension having only null values

fixes #1211

fix value matcher

more improvements

more fixes for partial null column

review comment

review comments

review comment

fix failing tests

review comment

fix compilation
2015-05-04 22:07:45 +05:30
Fangjin Yang 6be596c45e Merge pull request #1336 from metamx/fix-1330
fixes #1330
2015-05-04 09:06:32 -07:00
nishant 50158357ff fixes #1330
fixes #1330,
Avoid creating Period instance as creating a Period from Long.MAX_VALUE
throws arithmetic exception.
After this query metric will emit duration in seconds instead of
minutes.
2015-05-04 20:34:28 +05:30
Xavier Léauté 2ec87838d2 Merge pull request #1333 from himanshug/convert_props_ut
UTs for ConvertProperties
2015-05-03 15:11:53 -07:00
Himanshu Gupta eca69fd084 remove validation query config from docs as its not valid anymore 2015-05-03 08:41:35 -05:00
Himanshu Gupta aa10626b61 adding druid.db.tables.xxx renames and druid.db.connectory.*validation* from ConvertProperties 2015-05-03 08:41:15 -05:00
Himanshu Gupta 8722f4cc18 UTs for ConvertProperties 2015-05-03 08:40:17 -05:00
Xavier Léauté 592ba565e0 Merge pull request #1334 from himanshug/fix_default_indexing_service_name
fix doc for druid.[host,port,service] etc
2015-05-02 23:05:14 -07:00
Himanshu Gupta fd3b23072b fixing defaults for druid.[host,port,service] in docs etc 2015-05-02 15:44:03 -05:00
Himanshu Gupta ce7ad41aaa removing port number from druid.host in production-cluster-configuration as its gives the intuition that port name is mandatory to be specified which is not true anymore 2015-05-02 15:34:36 -05:00
Fangjin Yang f734471f22 Merge pull request #1331 from metamx/fixIrcFirehoseNameNick
fix IRCFirehoseFactory json name/nick mismatch
2015-05-01 15:35:08 -07:00
Charles Allen 27e25fe513 fix IRCFirehoseFactory json name/nick mismatch
* Now it is nick as per wikipedia example
2015-05-01 12:35:44 -07:00
Fangjin Yang 15ed308a4b Merge pull request #1322 from himanshug/fix_audit_tbl_idx
fixing the typo for audit table (type, created_date) index creation
2015-04-30 13:22:04 -07:00
Xavier Léauté a2bde3af1b Merge pull request #1327 from druid-io/more-coord-logs
Additional coordinator logs to figure out when racy things might be hap...
2015-04-30 10:51:09 -07:00
Fangjin Yang 85335227fb Merge pull request #1326 from himanshug/add_audit_tbl_config
add audit table name configuration doc
2015-04-30 10:25:20 -07:00
fjy a7f9bb1a1e Additional coordinator logs to figure out when racey things might be happening 2015-04-30 08:26:52 -07:00
Himanshu Gupta 3982301484 add audit table name configuration doc 2015-04-30 06:45:57 -05:00
Fangjin Yang 40121bc800 Merge pull request #1324 from metamx/fixMasterPropertiesTest
Partial revert of 467fcfae03
2015-04-29 22:15:21 -07:00
Charles Allen de16eb34af Partial revert of 467fcfae03 2015-04-29 17:20:51 -07:00
Xavier Léauté 8b16d70e19 Merge pull request #1320 from davrodpin/fix-deep-storage-doc
Remove duplicated Azure doc. section.
2015-04-29 16:09:37 -07:00
Xavier Léauté bfdb5dcb5d Merge pull request #1317 from metamx/overlord-endpoints
proxy /druid/indexer requests on the coordinator to the indexing overlord
2015-04-29 16:07:42 -07:00
Fangjin Yang 520ad68f84 Merge pull request #1323 from metamx/redoUpdater
Add options to the properties converter to update from 0.6.x to 0.7.x
2015-04-29 15:50:59 -07:00
Charles Allen e055a7e869 Add options to the properties converter to update from 0.6.x to 0.7.x
* Removed 0.5 --> 0.6 converter
2015-04-29 15:46:16 -07:00
Himanshu Gupta 4aa9381f6a fixing the typo for audit table (type, created_date) index creation 2015-04-29 17:09:35 -05:00
David Rodrigues 1dda3268d1 Remove duplicated Azure doc. section. 2015-04-29 11:42:57 -07:00
nishant 6e84caf3d5 Proxy Requests from coordinator to overlord
Add unit test and support https scheme

review comments
2015-04-29 20:58:26 +05:30
Xavier Léauté 95e0429eb1 Merge pull request #1318 from himanshug/broker_retries_cfg
adding druid.broker.retryPolicy.numTries config to documentation
2015-04-28 22:32:44 -07:00
Xavier Léauté e5d61f4e6e Merge pull request #1319 from metamx/fixipv6UnitTests
Fix DruidNodeTest for ipv6 only hosts
2015-04-28 15:41:11 -07:00
Charles Allen 062a413ccd Fix DruidNodeTest for ipv6 only hosts 2015-04-28 13:11:47 -07:00
Himanshu Gupta 42ed88e1cb adding druid.broker.retryPolicy.numTries config to documentation 2015-04-28 13:22:32 -05:00
Fangjin Yang 16f7b51d3c Merge pull request #1316 from himanshug/druid_server_maxSize
adding druid.server.maxSize config doc
2015-04-28 08:26:42 -07:00
Himanshu Gupta 0beaec5578 adding druid.server.maxSize config doc 2015-04-28 09:34:05 -05:00
Xavier Léauté 85902246ca Merge pull request #1314 from druid-io/update-server-metrics
Update server metrics to use schemaless design
2015-04-27 16:53:18 -07:00
fjy d6ef2c20df Update server metrics to use schemaless design 2015-04-27 15:06:25 -07:00
Xavier Léauté 721505c017 Merge pull request #1208 from druid-io/rework-metrics
Schemaless metrics + additional metrics for things we care about
2015-04-27 15:04:54 -07:00
fjy 963e5765bf Schemaless metrics + additional metrics for things we care about 2015-04-27 13:39:40 -07:00
Xavier Léauté 625ae830d9 Merge pull request #1313 from metamx/indexIOValidatorTSFix
Fix IndexIO segment validator to account for timestamp mismatches.
2015-04-27 13:01:39 -07:00
Charles Allen 27016c0289 Fix IndexIO segment validator to account for timestamp mismatches. 2015-04-27 12:42:16 -07:00
Xavier Léauté 962997f3f7 Merge pull request #1310 from metamx/optionalValidation
Optional validation on ConvertSegmentTask
2015-04-27 11:19:57 -07:00
Charles Allen 633fdb029e Add option to ConvertSegmentTask to skip validation
* Validation is enabled by default
2015-04-27 08:37:55 -07:00
Fangjin Yang 0f04798dda Merge pull request #1311 from metamx/staticS3FirehoseFactoryGZFix
Fix concatenated gzip files in StaticS3FirehoseFactory
2015-04-26 08:31:35 -07:00