Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,271 Commits 326 Branches 371 Tags
Commit Graph

14271 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arpit Agarwal
82530c966a HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:02:42 -08:00
Yiqun Lin
d49074f0a2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 14:00:55 +08:00
Brahma Reddy Battula
9873eb63a7 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-08 18:33:11 +05:30
Yongjun Zhang
b7f7fb003e HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
(cherry picked from commit f491f717e9ee6b75ad5cfca48da9c6297e94a8f7)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
(cherry picked from commit 60d82ac61f2afa2a61c8eeabf20e45b5a89311d0)
 2018-02-07 15:35:41 -08:00
Billie Rinaldi
aa64a50746 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit a72dcb9cad7df7f7236092f5b86446c2ef4ea874.
 2018-02-07 08:58:28 -08:00
Kihwal Lee
34466c6b16 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de983a30bf0d9ee957bdb09f825f9d40a3)
 2018-02-06 15:36:32 -06:00
Billie Rinaldi
93128c1904 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362a1b35e5de20ee519f7c544ab1588e1)
 2018-02-06 12:04:52 -08:00
Jason Lowe
260f30f063 YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bcecadd80e28c81bc943b44abf13d20abae)
 2018-02-05 17:22:42 -06:00
Haibo Chen
c25c082961 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea9d4e38d7f8f9fd48035e342bb22f1c)
 2018-02-05 10:51:03 -08:00
Yongjun Zhang
6dfea05049 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16e81b4bd6fae443a997ef24d588a6e8)
 2018-02-02 23:33:10 -08:00
Yiqun Lin
7cd3770fee HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit b0627c891b0e90e29dab2bec64a01c2c2ffe4ed0)
 2018-02-02 17:38:32 +08:00
Inigo Goiri
7b8cc048cc HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204cebf4b7060696a07b1777723a6dfa7d9d)
 2018-02-01 10:38:39 -08:00
Yiqun Lin
e5cac88c82 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit dbb9dded33b3cff3b630e98300d30515a9d1eec4)
 2018-01-31 09:14:31 -08:00
Weiwei Yang
0c2c4c20cb HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 
(cherry picked from commit 2e7331ca264dd366b975f3c8e610cf84eb8cc155)
 2018-01-30 21:40:07 -08:00
Inigo Goiri
fffa7dc377 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca479dd53f614d75bab594043a9866e1)
 2018-01-30 18:28:21 -08:00
Sammi Chen
2816bd1f43 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 91184299c54df18540f841cca0efe0131d05b882)
 2018-01-30 16:25:02 +08:00
Kihwal Lee
eda786ea12 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 2018-01-29 17:52:04 -06:00
Jinjiang Ling
987a8972ac
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535aeb151c3f283cc41111b15d66990e5)
 2018-01-30 00:13:04 +09:00
Xiao Chen
0922ea0aaf HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 
(cherry picked from commit b63dcd583f0b98e785831004f41bd7c7de8b3c18)
(cherry picked from commit 75be15cf021e31d167570b3038060ddce62e2120)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionFaultInjector.java
 2018-01-28 22:30:14 -08:00
Jian He
0497e09339 YARN-7765. Fixed an issue that kerberos tgt not found when NM posting timeline events. Contributed by Rohith Sharma K S 2018-01-28 13:24:49 -08:00
Steve Loughran
312c571694 HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-27 20:31:21 -08:00
Jason Lowe
e3a1647477 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 2018-01-26 15:46:01 -06:00
Arpit Agarwal
7be5a46a41 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:36 -08:00
Chris Douglas
9bd439e2c5 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
(cherry picked from commit 7f354877889b343878a8a09792d5cec8d2846a50)
 2018-01-26 09:15:53 -08:00
Haibo Chen
5f44bd3eba MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cc10852252c2d69294eabc68bd032cc630a53b18)
(cherry picked from commit e6c66baba97c34514e1d7491d5fe80867930f184)
 2018-01-25 17:23:08 -08:00
Jason Lowe
4d7bbee0a1 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:56:36 -06:00
Wei Yan
cefe4b7bb6 HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 12:58:16 -08:00
Sunil G
bed683d300 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968d2897ae006c5ef29eba74023f3483e)
 2018-01-25 14:34:31 +05:30
Yiqun Lin
9ce24d2d50 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff74494eb7fbbbba7f8bb4b4692d880d301)
 2018-01-25 15:54:20 +08:00
Jianfei Jiang
f4656e2150
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd0868864da994612f1dd1452ea38d31f5e85)
 2018-01-25 15:32:13 +09:00
Zhe Zhang
e9a338ff64 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4896a602dc383d5e266768392a9790c)
(cherry picked from commit 83a4f92a3f0c0134709bcb0a534e31b7c7056643)
 2018-01-24 13:10:35 -08:00
Jason Lowe
1ec82e5789 MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 
(cherry picked from commit cff9edd4b514bdcfe22cd49964e3707fb78ab876)
 2018-01-24 14:51:22 -06:00
Jian He
c611707e28 YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 11:19:46 -08:00
Sunil G
ce43e93f92 YARN-7806. Distributed Shell should use timeline async api's. Contributed by Rohith Sharma K S 
(cherry picked from commit 97607617ce01ee21af4fb9eb8df26fc6986a38a2)
 2018-01-24 16:57:29 +05:30
Rohith Sharma K S
f036609cea Revert "YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma" 
This reverts commit 41e10161b6227b994079c603e9a0a350f0d055ea.
 2018-01-24 15:05:53 +05:30
Inigo Goiri
1534936ac1 HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e6b42b227a22dfa7cc16edc7bdb58bb)
 2018-01-23 19:17:22 -08:00
Jian He
a80c2dbab5 YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang 2018-01-23 14:12:00 -08:00
Masahiro Tanaka
ab482ebe73
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d695ed94bd44c60f191cfbb5cdb32e2)
 2018-01-22 17:15:31 +09:00
Vrushali C
41e10161b6 YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma 2018-01-19 15:59:47 -08:00
Haibo Chen
a286acd7be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef9e82d31fe8ec59648b2a4a50c8869a)
(cherry picked from commit fc669778bec543427ea94abe12aff34ee1533b5c)
 2018-01-19 13:12:26 -08:00
Haibo Chen
895a0391fe YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759f773135f8a27dbaa9731196fac5233)
(cherry picked from commit 8793e45f53607315bf6c99902452ba6cf2e3eb72)
 2018-01-19 10:35:40 -08:00
Inigo Goiri
c6139332f9 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00e8cab59c3aa4d668e8bddd84544713)
 2018-01-19 09:52:29 -08:00
Yiqun Lin
65507053b4 HDFS-12973. RBF: Document global quota supporting in federation. Contributed by Yiqun Lin. 
(cherry picked from commit 9e4f52d32319828c153a3ea658520b946988ae31)
 2018-01-19 14:21:07 +08:00
Brahma Reddy Battula
8181367c62 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d055d85472f0c9371fefe9b56bfea1ed)
 2018-01-18 10:56:58 +05:30
Akira Ajisaka
b0c9b89281
HADOOP-15177. Update the release year to 2018. Contributed by Bharat Viswanadham. 
(cherry picked from commit cdaf92c9f57560219b8f915a19ad8603ddf2a505)
 2018-01-18 13:17:07 +09:00
Miklos Szegedi
86a2ac94b1 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 2018-01-17 12:56:10 -08:00
Jason Lowe
c228a7c707 MAPREDUCE-7029. FileOutputCommitter is slow on filesystems lacking recursive delete. Contributed by Karthik Palaniappan 2018-01-17 10:32:06 -06:00
Jason Lowe
fd09f72151 Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 896dc7c7801adaa4460fa6c19a4f452a6a6112d8.
 2018-01-17 09:47:43 -06:00
Vinayakumar B
b3eeea0b6e HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
 2018-01-17 14:22:29 +05:30
Sammi Chen
896dc7c780 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 55142849db02a9191db0dd6f4e1401ff19ec242a)
(cherry picked from commit 082a707bae4bb97444a34c00eecd62975807388d)
 2018-01-17 16:36:03 +08:00