Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,553 Commits 326 Branches 371 Tags
Commit Graph

6851 Commits

Author SHA1 Message Date
Wei-Chiu Chuang
8062a7428d HDFS-12891. Do not invalidate blocks if toInvalidate is empty. Contributed by Zsolt Venczel. 
(cherry picked from commit 55fc2d6485702a99c6d4bb261a720d1f0498af2b)
 2017-12-11 16:43:42 -08:00
Inigo Goiri
6a43fbe724 HDFS-12875. RBF: Complete logic for -readonly option of dfsrouteradmin add command. Contributed by Inigo Goiri. 
(cherry picked from commit 5cd1056ad77a2ebb0466e7bc5933337b57f6fe30)
 2017-12-11 15:16:19 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
ba874d6787 HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 
(cherry picked from commit f19638333b11da6dcab9a964e73a49947b8390fd)
 2017-12-08 08:03:23 -08:00
Xiao Chen
1d1401ad38 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 
(cherry picked from commit 19e089420999dd9d97d981dcd0abd64b6166152d)
 2017-12-07 21:06:25 -08:00
John Zhuge
9942952dd8 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 
(cherry picked from commit bf6a660232b01642b07697a289c773ea5b97217c)
 2017-12-07 21:06:25 -08:00
Lei Xu
f3143d225a HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 67662e2ac9e68f32b725c8118cf2be79a662fca5)
 2017-12-07 11:20:19 -08:00
Akira Ajisaka
a8be4b559a
HDFS-12889. Addendum patch to add missing file. 
(cherry picked from commit 6cca5b3bcb440095f12d3eda88101fa250ac000a)
 2017-12-07 13:23:58 +09:00
Xiao Chen
bf79246c5d HDFS-12872. EC Checksum broken when BlockAccessToken is enabled. 
(cherry picked from commit 56b1ff80dd9fbcde8d21a604eff0babb3a16418f)
 2017-12-05 20:48:46 -08:00
Akira Ajisaka
c81057d936
HDFS-12889. Router UI is missing robots.txt file. Contributed by Bharat Viswanadham. 
(cherry picked from commit 44b06d34a537f8b558007cc92a5d1a8e59b5d86b)
 2017-12-06 11:41:40 +09:00
Xiao Chen
55fa509eb6 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 
(cherry picked from commit 404eab4dc0582e0384b93664ea6ee77ccd5eeebc)
 2017-12-04 10:49:58 -08:00
Chris Douglas
2399c96ee7 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 42307e3c3abbfe0b83d9a2581deba327435b910f)
 2017-12-02 23:32:44 -08:00
Wei-Chiu Chuang
83b62a8eed HDFS-12836. startTxId could be greater than endTxId when tailing in-progress edit log. Contributed by Chao Sun. 
(cherry picked from commit 0faf50624580b86b64a828cdbbb630ae8994e2cd)
 2017-12-01 12:03:04 -08:00
Chris Douglas
6266c52680 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak" 
This reverts commit 482fd5a880994f37fc3ad9e0cc2d127737b70aef.
 2017-12-01 11:19:17 -08:00
Chris Douglas
482fd5a880 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 
(cherry picked from commit 5304698dc8c5667c33e6ed9c4a827ef57172a723)
 2017-12-01 10:30:08 -08:00
Konstantin V Shvachko
def87db9cb HDFS-12638. Delete copy-on-truncate block along with the original block, when deleting a file being truncated. Contributed by Konstantin Shvachko. 
(cherry picked from commit 60fd0d7fd73198fd610e59d1a4cd007c5fcc7205)
 2017-11-30 18:27:16 -08:00
Kihwal Lee
ee4525e071 HDFS-11754. Make FsServerDefaults cache configurable. Contributed by Mikhail Erofeev. 
(cherry picked from commit 53509f295b5274059541565d7216bf98aa35347d)
 2017-11-29 15:12:47 -06:00
Wei Yan
4b1a2152eb HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d93ac22dc6fe1a05f18c1f266cc5e54)
 2017-11-29 09:59:17 -08:00
Konstantin V Shvachko
4cbd5ea42b HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24b3f22f609366740c9c4f449edc61ac)
 2017-11-28 17:18:21 -08:00
Yiqun Lin
72d0be964d HDFS-12858. Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 
(cherry picked from commit 94bed5047113fb148194380853ff01e92897a91f)
 2017-11-28 11:52:29 +08:00
Kihwal Lee
1078d477c3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 738d1a206aba05f0b4be7d633b17db7fcd1c74bc)
 2017-11-22 13:02:45 -06:00
Lei Xu
a5a86680a4 HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:23:10 -08:00
Inigo Goiri
3cd5224693 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e777947a85943504a207deb3cf3ffc047910)
 2017-11-17 10:08:04 -08:00
Weiwei Yang
f9f268cf52 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:23:13 +08:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Akira Ajisaka
9051d1b851
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong. 
(cherry picked from commit 3a3566e1d1ab5f78cfb734796b41802fe039196d)
 2017-11-09 15:18:32 +09:00
Brahma Reddy Battula
d25d3827d5 HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 
(cherry picked from commit 410d0319cf72b9b5f3807c522237f52121d98cd5)
 2017-11-08 14:49:10 +05:30
Weiwei Yang
19b1164fc6 HDFS-7060. Avoid taking locks when sending heartbeats from the DataNode. Contributed by Jiandan Yang. 2017-11-08 10:24:35 +08:00
Kihwal Lee
50b438b597 HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla. 
(cherry picked from commit 4d2dce40bbe5242953774e6a2fc0dc9111cf5ed0)
 2017-11-03 15:18:36 -05:00
Xiao Chen
71965747f4 HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang. 
(cherry picked from commit c8eb4beabad84686f869a2397f7188499cab12c9)
 2017-11-03 12:05:55 -07:00
Xiao Chen
621a50ebb1 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks. 
(cherry picked from commit e1187bad8e9b9abb54a55b5f9ab8373b0f64e6d7)
 2017-11-02 21:53:20 -07:00
Xiao Chen
1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
 2017-11-02 21:27:43 -07:00
Weiwei Yang
b2f717d967 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang. 
(cherry picked from commit 56b88b06705441f6f171eec7fb2fa77946ca204b)
 2017-11-01 16:53:07 +08:00
Yiqun Lin
5cada4564a
HDFS-12219. Javadoc for FSNamesystem#getMaxObjects is incorrect. Contributed by Erik Krogen. 
(cherry picked from commit 20304b91cc1513e3d82a01d36f4ee9c4c81b60e4)
 2017-11-01 16:43:44 +09:00
John Zhuge
0ec172b42f HDFS-12714. Hadoop 3 missing fix for HDFS-5169. Contributed by Joe McDonnell. 
(cherry picked from commit b8c8b5bc274211b29be125e5463662795a363f84)
 2017-10-31 22:46:26 -07:00
Lei Xu
9ee04006cb HDFS-12482. Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery. (lei) 2017-10-31 22:03:44 -07:00
Chris Douglas
b5f66b005c Revert "HDFS-7878. API - expose a unique file identifier." 
This reverts commit 1ce0eb7d23156175db2d31f3fcbbe6529944c04b.
 2017-10-31 13:56:41 -07:00
Wei-Chiu Chuang
1347e83c90 HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
 2017-10-31 12:52:58 -07:00
Manoj Govindassamy
166c75a0eb HDFS-12544. SnapshotDiff - support diff generation on any snapshot root descendant directory. 
(cherry picked from commit 075dd45a24398dcdcddd60da995f0dc152eee321)
 2017-10-31 12:50:54 -07:00
Manoj Govindassamy
1375c7205f HDFS-12573. Divide the total blocks metrics into replicated and erasure coded. Contributed by Takanobu Asanuma. 
(cherry picked from commit 78af6cdc5359404139665d81447f28d26b7bb43b)
 2017-10-31 12:50:21 -07:00
Manoj Govindassamy
014184127e HDFS-12614. FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured. 
(cherry picked from commit b406d8e3755d24ce72c443fd893a5672fd56babc)
 2017-10-31 12:49:41 -07:00
Wei-Chiu Chuang
579612c9d2 HDFS-12619. Do not catch and throw unchecked exceptions if IBRs fail to process. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4ab0c8f96a41c573cc1f1e71c18871d243f952b9)
 2017-10-31 12:49:08 -07:00
Akira Ajisaka
143dbf8497 HDFS-12622. Fix enumerate in HDFSErasureCoding.md. Contributed by Yiqun Lin. 
(cherry picked from commit dc63a6a52b7fdf076ab83a774b0378f77c1c0cd3)
 2017-10-31 12:48:29 -07:00
Andrew Wang
ae08fe518b Revert "HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham." 
This reverts commit f09725537e4aef28c3e192a7aacad206900aa7b5.
 2017-10-31 10:46:19 -07:00
Inigo Goiri
e259030066 HDFS-12699. TestMountTable fails with Java 7. Contributed by Inigo Goiri. 
(cherry picked from commit 982bd2a5bff4c490a5d62f2e4fd0d3755305349c)
 2017-10-31 10:23:06 -07:00
Arpit Agarwal
f09725537e HDFS-12499. dfs.namenode.shared.edits.dir property is currently namenode specific key. Contributed by Bharat Viswanadham. 2017-10-31 10:08:05 -07:00
Chris Douglas
1ce0eb7d23 HDFS-7878. API - expose a unique file identifier. 
(cherry picked from commit d015e0bbd5416943cb4875274e67b7077c00e54b)
 2017-10-31 09:46:39 -07:00
Chris Douglas
34512b2d7a HDFS-12582. Replace HdfsFileStatus constructor with a builder pattern. Contributed by Bharat Viswanadham 
(cherry picked from commit d55a84951abe87a31c17bd4b84cd309ed202e540)
 2017-10-27 15:36:18 -07:00
Arpit Agarwal
2510fc1801 HDFS-9914. Fix configurable WebhDFS connect/read timeout. Contributed by Xiaoyu Yao. 2017-10-27 07:49:18 -07:00
Junping Du
ce10c1ef05 Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf5189247fa38e7b0a188d568eba1b6c)
 2017-10-26 23:18:50 -07:00
Xiao Chen
e18d2c548b HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 
(cherry picked from commit e3f25f361720b544b464c5e4fbeb4f31e6c53c67)
 2017-10-25 14:11:38 -07:00