Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,293 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

477 Commits

Author SHA1 Message Date
Xiao Chen 49de91446f HDFS-12933. Improve logging when DFSStripedOutputStream failed to write some blocks. Contributed by chencan. 
(cherry picked from commit e5c2fdddd6)
 2018-02-06 22:57:15 -08:00
Lei Xu 7b377f43d0 Preparing for 3.0.2 development 2018-02-06 10:50:37 -08:00
Yongjun Zhang 35247cca87 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16)

(cherry picked from commit 0524fac21e)
 2018-02-04 22:53:09 -08:00
Xiaoyu Yao 02790eb633 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 
(cherry picked from commit f4bcae10d306ef2b0083e1de17253ec732b5b5c1)
 2018-01-31 10:47:46 -08:00
Weiwei Yang 7f62d2a473 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:35:54 +08:00
Kihwal Lee 673200ac1e HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 
(cherry picked from commit fde95d463c)
 2018-01-29 17:25:30 -06:00
Arpit Agarwal 480b69c03c HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:23 -08:00
Inigo Goiri 49e285ded5 HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-17 12:57:14 -06:00
Yiqun Lin cb07172e21 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-17 12:50:06 -06:00
Manoj Govindassamy 710555e150 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-17 12:44:09 -06:00
Lei Xu 47773d68b2 HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 85067eaa8e352afeed9c094da3815100873c84e4)
 2018-01-04 10:33:34 -08:00
John Zhuge 9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232)
 2017-12-07 21:06:25 -08:00
Xiao Chen bf79246c5d HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 
(cherry picked from commit 56b1ff80dd)
 2017-12-05 20:48:46 -08:00
Xiao Chen 55fa509eb6 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 404eab4dc0)
 2017-12-04 10:49:58 -08:00
Kihwal Lee ee4525e071 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b)
 2017-11-29 15:12:47 -06:00
Kihwal Lee 1078d477c3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 738d1a206a)
 2017-11-22 13:02:45 -06:00
Andrew Wang 1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Xiao Chen 1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
 2017-11-02 21:27:43 -07:00
Weiwei Yang b2f717d967 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 
(cherry picked from commit 56b88b0670)
 2017-11-01 16:53:07 +08:00
Chris Douglas b5f66b005c Revert "HDFS-7878. API - expose a unique file identifier." 
This reverts commit 1ce0eb7d23.
 2017-10-31 13:56:41 -07:00
Wei-Chiu Chuang 1347e83c90 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c9)
 2017-10-31 12:52:58 -07:00
Chris Douglas 1ce0eb7d23 HDFS-7878. API - expose a unique file identifier. 
(cherry picked from commit d015e0bbd5)
 2017-10-31 09:46:39 -07:00
Chris Douglas 34512b2d7a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 
(cherry picked from commit d55a84951a)
 2017-10-27 15:36:18 -07:00
Arpit Agarwal 2510fc1801 HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by Xiaoyu Yao. 2017-10-27 07:49:18 -07:00
Xiao Chen e18d2c548b HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 
(cherry picked from commit e3f25f361720b544b464c5e4fbeb4f31e6c53c67)
 2017-10-25 14:11:38 -07:00
Lei Xu 6959db9c20 HDFS-12612. DFSStripedOutputStream.close will throw if called a second time with a failed streamer. (Lei (Eddy) Xu) 2017-10-17 15:53:07 -07:00
Lei Xu cdfcd384b2 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 
(cherry picked from commit 31ebccc962)
 2017-10-16 19:46:26 -07:00
Kai Zheng 8947c0b881 HDFS-12635. Unnecessary exception declaration of the CellBuffers constructor. Contributed by Huafeng Wang 2017-10-11 15:47:00 +08:00
Lei Xu 015eb628b8 HDFS-12606. When using native decoder, DFSStripedStream.close crashes JVM after being called multiple times. (Lei (Eddy) Xu) 
(cherry picked from commit 46644319e1)
 2017-10-09 10:09:49 -07:00
Andrew Wang 5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Daryn Sharp 61dc4bcd61 [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:27:32 -05:00
Andrew Wang 1792093bae HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 
(cherry picked from commit a12f09ba3c)
 2017-09-20 11:51:21 -07:00
Ming Ma 816933722a HDFS-12473. Change hosts JSON file format. 
(cherry picked from commit 230b85d586)
 2017-09-20 09:05:56 -07:00
Brahma Reddy Battula 1ce365e91e HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55)
 2017-09-19 11:27:41 +05:30
Kai Zheng 16a75b4ed3 HDFS-12479. Some misuses of lock in DFSStripedOutputStream. Contributed by Huafeng Wang 2017-09-19 17:46:39 +08:00
Lei Xu 2787766b41 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 
(cherry picked from commit fbe06b5880)
 2017-09-15 12:15:49 -07:00
Andrew Wang 61bc2c05af Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 7fcc73fc0d.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java

(cherry picked from commit 390c2b5df0)
 2017-09-14 15:12:42 -07:00
Arpit Agarwal 5f6abb2803 Revert "HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak." 
This reverts commit b4dc2fa247.
 2017-09-14 11:17:23 -07:00
Andrew Wang 4687d3d6f6 HDFS-12222. Document and test BlockLocation for erasure-coded files. Contributed by Huafeng Wang. 
(cherry picked from commit f4b6267465)
 2017-09-12 17:35:33 -07:00
Arpit Agarwal b4dc2fa247 HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak. 2017-09-09 09:16:51 -07:00
Andrew Wang 986e8fc553 HDFS-12218. Addendum. Rename split EC / replicated block metrics in BlockManager. 
(cherry picked from commit 4e50dc976a)
 2017-09-07 16:57:37 -07:00
Andrew Wang 4c1f8f3b26 HDFS-12218. Rename split EC / replicated block metrics in BlockManager. 
(cherry picked from commit 40c2f31f8d)
 2017-09-07 16:57:35 -07:00
Kai Zheng 7415a21ee2 HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. Contributed by Sammi Chen 2017-09-07 20:37:11 +08:00
Kai Zheng 3c71bf79f2 HDFS-12392. Writing striped file failed due to different cell size. Contributed by Sammi Chen 2017-09-06 13:34:53 +08:00
Kai Zheng b7739b468e HDFS-12388. A bad error message in DFSStripedOutputStream. Contributed by Huafeng Wang 
(cherry picked from commit 5dba54596a)
 2017-09-05 14:19:16 -07:00
Andrew Wang c333ae8b36 HDFS-11882. Precisely calculate acked length of striped block groups in updatePipeline. 
(cherry picked from commit ccd2ac60ec)
 2017-09-05 14:16:07 -07:00
Kai Zheng 7a96033b15 HDFS-11964. Decoding inputs should be correctly prepared in pread. Contributed by Takanobu Asanuma 2017-09-01 17:48:26 +08:00
Mingliang Liu 36f33a1efb HDFS-12380. Simplify dataQueue.wait condition logical operation in DataStreamer::run(). Contributed by liaoyuxiangqin 2017-08-31 23:18:48 -07:00
Rakesh Radhakrishnan 200b11368d HDFS-12258. ec -listPolicies should list all policies in system, no matter it's enabled or disabled. Contributed by Wei Zhou. 2017-08-30 12:58:56 +05:30
Andrew Wang f29a0fc288 HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou. 2017-08-25 14:14:23 -07:00