Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,303 Commits 326 Branches 371 Tags
Commit Graph

14303 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yiqun Lin
08cd911b69 HDFS-13194. CachePool permissions incorrectly checked. Contributed by Jianfei Jiang. 
(cherry picked from commit a9c14b11193adeaa31389578f4cb90fa79cad8c3)
 2018-02-28 10:25:49 +08:00
Inigo Goiri
2f18785f39 HDFS-13199. RBF: Fix the hdfs router page missing label icon issue. Contributed by maobaolong. 
(cherry picked from commit d86f301d464683f8d392dad50e83f50d823e862e)
 2018-02-27 17:42:28 -08:00
Yiqun Lin
1b0aa0e87a HDFS-13184. RBF: Improve the unit test TestRouterRPCClientRetries. Contributed by Yiqun Lin. 
(cherry picked from commit 1e85a995d1c7fe3dbf4b36a481feb5fdeeb6015b)
 2018-02-27 10:54:37 +08:00
Chris Douglas
762125b864 Backport HADOOP-13514 (surefire upgrade) to branch-2 2018-02-26 16:32:06 -08:00
Wei Yan
54803ebe49 HDFS-13187. RBF: Fix Routers information shown in the web UI. Contributed by Inigo Goiri and Wei Yan. 
(cherry picked from commit 6ce9f79cc9b2107e5953a39d05b22966aff0b7ff)
 2018-02-26 15:32:27 -08:00
Arpit Agarwal
0945f0eb23 HADOOP-15266. [branch-2] Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-26 13:45:51 -08:00
Kihwal Lee
4b43f2aa56 HDFS-12070. Failed block recovery leaves files open indefinitely and at risk for data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 451265a83d8798624ae2a144bc58fa41db826704)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockRecoveryWorker.java
 2018-02-26 10:58:07 -06:00
Arpit Agarwal
79af42f095 HDFS-12781. After Datanode down, In Namenode UI Datanode tab is throwing warning message.. Contributed by Brahma Reddy Battula. 2018-02-24 15:12:36 -08:00
Xuan Gong
a0b55392d4 MAPREDUCE-7027: HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails. Contributed by Gergely Novák 
(cherry picked from commit 68ce193efcb595f75d7addf751559c806a5aa399)
 2018-02-23 14:40:51 -08:00
Xiao Chen
80f7165374 HDFS-13164. File not closed if streamer fail with DSQuotaExceededException. 
(cherry picked from commit 51088d323359587dca7831f74c9d065c2fccc60d)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
 2018-02-23 14:00:33 -08:00
Gabor Bota
696e8e0299
HADOOP-15236. Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c36b4aa31ce25fbe5fa173bce36da2950d74a475)
(cherry picked from commit b8942cb3bf4a458c78654bc166447806523409b5)
 2018-02-23 13:58:08 +09:00
Jason Lowe
7a6aa7bf99 YARN-7945. Java Doc error in UnmanagedAMPoolManager for branch-2. Contributed by Botong Huang 2018-02-22 10:45:16 -06:00
Brahma Reddy Battula
7307a89ec1 HADOOP-14903. Add json-smart explicitly to pom.xml. Contributed by Brahma Reddy Battula and Ray Chiang 2018-02-22 18:52:48 +05:30
Ray Chiang
eeb731307f HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179aa094c21acf610439a8d69fe6420ab)
 2018-02-22 18:08:52 +05:30
Sunil G
f7e5e45b72 YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 
(cherry picked from commit bdd2a184d78379d99c802a43ebec7d2cef0bbaf7)
 2018-02-21 15:26:20 +05:30
Robert Kanter
14f5797eff HADOOP-15235. Authentication Tokens should use HMAC instead of MAC (rkanter) 
(cherry picked from commit 324e5a7cf2bdb6f93e7c6fd9023817528f243dcf)
 2018-02-20 17:25:12 -08:00
Xiao Chen
49ed7d7fc9 HDFS-11187. Optimize disk access for last partial chunk checksum of Finalized replica. Contributed by Gabor Bota, Wei-Chiu Chuang. 2018-02-19 19:15:02 -08:00
Yiqun Lin
a278ad6b91 HDFS-13119. RBF: Manage unavailable clusters. Contributed by Yiqun Lin. 2018-02-20 09:58:28 +08:00
Jason Lowe
51ad923f51 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 
(cherry picked from commit 85c611ad7dfa87846f708ba0131b4895a054752e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/CapacitySchedulerPage.java
 2018-02-19 15:44:56 -06:00
Rohith Sharma K S
e35dfc85de YARN-7937. Fix http method name in Cluster Application Timeout Update API example request. Contributed by Charan Hebri. 
(cherry picked from commit 87bdde69431c19a22d79a767071f6ea47e1ceb3d)
 2018-02-18 14:08:50 +05:30
Rohith Sharma K S
da0660ec84 YARN-7919. Refactor timelineservice-hbase module into submodules. Contributed by Haibo Chen. 2018-02-18 13:26:10 +05:30
Jason Lowe
b1d5e73c1f HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9e5c53f2fce7ccdeb4e1cd7d0f123b3)
 2018-02-16 15:00:10 -06:00
Eric Payne
904a6bf263 MAPREDUCE-7053: Timed out tasks can fail to produce thread dump. Contributed by Jason Lowe. 2018-02-16 08:47:42 -06:00
Kihwal Lee
fe044e69c3 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e56b0380147d42f4110ad03c2276c961)
 2018-02-15 15:38:49 -06:00
Jason Lowe
a87bccdc3f MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26e170a0338f93e228718da52e9196e4)
 2018-02-15 15:27:46 -06:00
Kihwal Lee
b725fd6924 xattr api cleanup 
(cherry picked from commit da59acd8ca9ab5b49b988ffca64e8cce91c5f741)
 2018-02-15 11:28:45 -06:00
Steve Loughran
07c7df4b26 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b79324b7f6231b879c0bfc608203b980)
 2018-02-15 16:29:44 +00:00
Steve Loughran
6f944452f7 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a31307f19551f87be4e81d4fb626de1)
 2018-02-15 14:26:36 +00:00
Kihwal Lee
56b950d777 Revert "HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)" 
This reverts commit 577a267198b1d745eee85d1ad9af64d4506bbd0a.
 2018-02-14 15:20:45 -06:00
Ray Chiang
577a267198 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179aa094c21acf610439a8d69fe6420ab)
 2018-02-14 00:07:09 +05:30
Jason Lowe
08feac4c83 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 2018-02-12 14:30:42 -06:00
Brahma Reddy Battula
24af0bc144 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
 2018-02-12 22:25:44 +05:30
Arpit Agarwal
82530c966a HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:02:42 -08:00
Yiqun Lin
d49074f0a2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 14:00:55 +08:00
Brahma Reddy Battula
9873eb63a7 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-08 18:33:11 +05:30
Yongjun Zhang
b7f7fb003e HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
(cherry picked from commit f491f717e9ee6b75ad5cfca48da9c6297e94a8f7)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
(cherry picked from commit 60d82ac61f2afa2a61c8eeabf20e45b5a89311d0)
 2018-02-07 15:35:41 -08:00
Billie Rinaldi
aa64a50746 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit a72dcb9cad7df7f7236092f5b86446c2ef4ea874.
 2018-02-07 08:58:28 -08:00
Kihwal Lee
34466c6b16 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de983a30bf0d9ee957bdb09f825f9d40a3)
 2018-02-06 15:36:32 -06:00
Billie Rinaldi
93128c1904 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362a1b35e5de20ee519f7c544ab1588e1)
 2018-02-06 12:04:52 -08:00
Jason Lowe
260f30f063 YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bcecadd80e28c81bc943b44abf13d20abae)
 2018-02-05 17:22:42 -06:00
Haibo Chen
c25c082961 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea9d4e38d7f8f9fd48035e342bb22f1c)
 2018-02-05 10:51:03 -08:00
Yongjun Zhang
6dfea05049 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16e81b4bd6fae443a997ef24d588a6e8)
 2018-02-02 23:33:10 -08:00
Yiqun Lin
7cd3770fee HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit b0627c891b0e90e29dab2bec64a01c2c2ffe4ed0)
 2018-02-02 17:38:32 +08:00
Inigo Goiri
7b8cc048cc HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204cebf4b7060696a07b1777723a6dfa7d9d)
 2018-02-01 10:38:39 -08:00
Yiqun Lin
e5cac88c82 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit dbb9dded33b3cff3b630e98300d30515a9d1eec4)
 2018-01-31 09:14:31 -08:00
Weiwei Yang
0c2c4c20cb HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 
(cherry picked from commit 2e7331ca264dd366b975f3c8e610cf84eb8cc155)
 2018-01-30 21:40:07 -08:00
Inigo Goiri
fffa7dc377 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca479dd53f614d75bab594043a9866e1)
 2018-01-30 18:28:21 -08:00
Sammi Chen
2816bd1f43 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 91184299c54df18540f841cca0efe0131d05b882)
 2018-01-30 16:25:02 +08:00
Kihwal Lee
eda786ea12 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 2018-01-29 17:52:04 -06:00
Jinjiang Ling
987a8972ac
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535aeb151c3f283cc41111b15d66990e5)
 2018-01-30 00:13:04 +09:00