Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,273 Commits 326 Branches 371 Tags
Commit Graph

474 Commits

Author SHA1 Message Date
Xiaoyu Yao
02790eb633 HDFS-13061. SaslDataTransferClient#checkTrustAndSend should not trust a partially trusted channel. Contributed by Ajay Kumar. 
(cherry picked from commit f4bcae10d306ef2b0083e1de17253ec732b5b5c1)
 2018-01-31 10:47:46 -08:00
Weiwei Yang
7f62d2a473 HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 2018-01-31 07:35:54 +08:00
Kihwal Lee
673200ac1e HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah 
(cherry picked from commit fde95d463c3123b315b3d07cb5b7b7dc19f7cb73)
 2018-01-29 17:25:30 -06:00
Arpit Agarwal
480b69c03c HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:23 -08:00
Inigo Goiri
49e285ded5 HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-17 12:57:14 -06:00
Yiqun Lin
cb07172e21 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-17 12:50:06 -06:00
Manoj Govindassamy
710555e150 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-17 12:44:09 -06:00
Lei Xu
47773d68b2 HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 85067eaa8e352afeed9c094da3815100873c84e4)
 2018-01-04 10:33:34 -08:00
John Zhuge
9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232b01642b07697a289c773ea5b97217c)
 2017-12-07 21:06:25 -08:00
Xiao Chen
bf79246c5d HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 
(cherry picked from commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f)
 2017-12-05 20:48:46 -08:00
Xiao Chen
55fa509eb6 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 404eab4dc0582e0384b93664ea6ee77ccd5eeebc)
 2017-12-04 10:49:58 -08:00
Kihwal Lee
ee4525e071 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
 2017-11-29 15:12:47 -06:00
Kihwal Lee
1078d477c3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 738d1a206aba05f0b4be7d633b17db7fcd1c74bc)
 2017-11-22 13:02:45 -06:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Xiao Chen
1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
 2017-11-02 21:27:43 -07:00
Weiwei Yang
b2f717d967 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 
(cherry picked from commit 56b88b06705441f6f171eec7fb2fa77946ca204b)
 2017-11-01 16:53:07 +08:00
Chris Douglas
b5f66b005c Revert "HDFS-7878. API - expose a unique file identifier." 
This reverts commit 1ce0eb7d23156175db2d31f3fcbbe6529944c04b.
 2017-10-31 13:56:41 -07:00
Wei-Chiu Chuang
1347e83c90 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
 2017-10-31 12:52:58 -07:00
Chris Douglas
1ce0eb7d23 HDFS-7878. API - expose a unique file identifier. 
(cherry picked from commit d015e0bbd5416943cb4875274e67b7077c00e54b)
 2017-10-31 09:46:39 -07:00
Chris Douglas
34512b2d7a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 
(cherry picked from commit d55a84951abe87a31c17bd4b84cd309ed202e540)
 2017-10-27 15:36:18 -07:00
Arpit Agarwal
2510fc1801 HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by Xiaoyu Yao. 2017-10-27 07:49:18 -07:00
Xiao Chen
e18d2c548b HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 
(cherry picked from commit e3f25f361720b544b464c5e4fbeb4f31e6c53c67)
 2017-10-25 14:11:38 -07:00
Lei Xu
6959db9c20 HDFS-12612. DFSStripedOutputStream.close will throw if called a second time with a failed streamer. (Lei (Eddy) Xu) 2017-10-17 15:53:07 -07:00
Lei Xu
cdfcd384b2 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 
(cherry picked from commit 31ebccc96238136560f4210bdf6766fe18e0650c)
 2017-10-16 19:46:26 -07:00
Kai Zheng
8947c0b881 HDFS-12635. Unnecessary exception declaration of the CellBuffers constructor. Contributed by Huafeng Wang 2017-10-11 15:47:00 +08:00
Lei Xu
015eb628b8 HDFS-12606. When using native decoder, DFSStripedStream.close crashes JVM after being called multiple times. (Lei (Eddy) Xu) 
(cherry picked from commit 46644319e1b3295ddbc7597c060956bf46487d11)
 2017-10-09 10:09:49 -07:00
Andrew Wang
5a4f37019a Preparing for 3.0.0 GA development 2017-09-28 11:32:01 -07:00
Daryn Sharp
61dc4bcd61 [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:27:32 -05:00
Andrew Wang
1792093bae HDFS-12447. Rename AddECPolicyResponse to AddErasureCodingPolicyResponse. Contributed by SammiChen. 
(cherry picked from commit a12f09ba3c4a3aa4c4558090c5e1b7bcaebe3b94)
 2017-09-20 11:51:21 -07:00
Ming Ma
816933722a HDFS-12473. Change hosts JSON file format. 
(cherry picked from commit 230b85d5865b7e08fb7aaeab45295b5b966011ef)
 2017-09-20 09:05:56 -07:00
Brahma Reddy Battula
1ce365e91e HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:27:41 +05:30
Kai Zheng
16a75b4ed3 HDFS-12479. Some misuses of lock in DFSStripedOutputStream. Contributed by Huafeng Wang 2017-09-19 17:46:39 +08:00
Lei Xu
2787766b41 HDFS-12349. Improve log message when it could not alloc enough blocks for EC. (Lei (Eddy) Xu) 
(cherry picked from commit fbe06b58805aac4861fb27dfa273914b69e8bdc6)
 2017-09-15 12:15:49 -07:00
Andrew Wang
61bc2c05af Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 7fcc73fc0d248aae1edbd4e1514c5818f6198928.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHDFS.java

(cherry picked from commit 390c2b5df0c4e25a2156d8daefd4670efe82b191)
 2017-09-14 15:12:42 -07:00
Arpit Agarwal
5f6abb2803 Revert "HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak." 
This reverts commit b4dc2fa2474088e105d8ef0d4a87b1ff1fc2549a.
 2017-09-14 11:17:23 -07:00
Andrew Wang
4687d3d6f6 HDFS-12222. Document and test BlockLocation for erasure-coded files. Contributed by Huafeng Wang. 
(cherry picked from commit f4b6267465d139bfdaf75e25761672eaf61d8a11)
 2017-09-12 17:35:33 -07:00
Arpit Agarwal
b4dc2fa247 HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak. 2017-09-09 09:16:51 -07:00
Andrew Wang
986e8fc553 HDFS-12218. Addendum. Rename split EC / replicated block metrics in BlockManager. 
(cherry picked from commit 4e50dc976a92a9560630c87cfc4e4513916e5735)
 2017-09-07 16:57:37 -07:00
Andrew Wang
4c1f8f3b26 HDFS-12218. Rename split EC / replicated block metrics in BlockManager. 
(cherry picked from commit 40c2f31f8dd45bc94291535ad41ffe3cc30b5536)
 2017-09-07 16:57:35 -07:00
Kai Zheng
7415a21ee2 HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. Contributed by Sammi Chen 2017-09-07 20:37:11 +08:00
Kai Zheng
3c71bf79f2 HDFS-12392. Writing striped file failed due to different cell size. Contributed by Sammi Chen 2017-09-06 13:34:53 +08:00
Kai Zheng
b7739b468e HDFS-12388. A bad error message in DFSStripedOutputStream. Contributed by Huafeng Wang 
(cherry picked from commit 5dba54596a1587e0ba5f9f02f40483e597b0df64)
 2017-09-05 14:19:16 -07:00
Andrew Wang
c333ae8b36 HDFS-11882. Precisely calculate acked length of striped block groups in updatePipeline. 
(cherry picked from commit ccd2ac60ecc5fccce56debf21a068e663c1d5f11)
 2017-09-05 14:16:07 -07:00
Kai Zheng
7a96033b15 HDFS-11964. Decoding inputs should be correctly prepared in pread. Contributed by Takanobu Asanuma 2017-09-01 17:48:26 +08:00
Mingliang Liu
36f33a1efb HDFS-12380. Simplify dataQueue.wait condition logical operation in DataStreamer::run(). Contributed by liaoyuxiangqin 2017-08-31 23:18:48 -07:00
Rakesh Radhakrishnan
200b11368d HDFS-12258. ec -listPolicies should list all policies in system, no matter it's enabled or disabled. Contributed by Wei Zhou. 2017-08-30 12:58:56 +05:30
Andrew Wang
f29a0fc288 HDFS-12303. Change default EC cell size to 1MB for better performance. Contributed by Wei Zhou. 2017-08-25 14:14:23 -07:00
Kihwal Lee
8455d70756 HDFS-12299. Race Between update pipeline and DN Re-Registration 2017-08-25 14:49:29 -05:00
John Zhuge
3123895db0 HDFS-12318. Fix IOException condition for openInfo in DFSInputStream. Contributed by legend. 2017-08-24 00:26:37 -07:00
Xiao Chen
1000a2af04 HDFS-10899. Add functionality to re-encrypt EDEKs. 2017-08-23 17:06:16 -07:00