Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,269 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

14269 Commits

Author SHA1 Message Date
Brahma Reddy Battula 9873eb63a7 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-08 18:33:11 +05:30
Yongjun Zhang b7f7fb003e HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
(cherry picked from commit f491f717e9)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
(cherry picked from commit 60d82ac61f)
 2018-02-07 15:35:41 -08:00
Billie Rinaldi aa64a50746 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit a72dcb9cad.
 2018-02-07 08:58:28 -08:00
Kihwal Lee 34466c6b16 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de98)
 2018-02-06 15:36:32 -06:00
Billie Rinaldi 93128c1904 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362)
 2018-02-06 12:04:52 -08:00
Jason Lowe 260f30f063 YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bceca)
 2018-02-05 17:22:42 -06:00
Haibo Chen c25c082961 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea)
 2018-02-05 10:51:03 -08:00
Yongjun Zhang 6dfea05049 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16)
 2018-02-02 23:33:10 -08:00
Yiqun Lin 7cd3770fee HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit b0627c891b)
 2018-02-02 17:38:32 +08:00
Inigo Goiri 7b8cc048cc HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204ceb)
 2018-02-01 10:38:39 -08:00
Yiqun Lin e5cac88c82 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit dbb9dded33)
 2018-01-31 09:14:31 -08:00
Weiwei Yang 0c2c4c20cb HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 
(cherry picked from commit 2e7331ca26)
 2018-01-30 21:40:07 -08:00
Inigo Goiri fffa7dc377 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca)
 2018-01-30 18:28:21 -08:00
Sammi Chen 2816bd1f43 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 91184299c5)
 2018-01-30 16:25:02 +08:00
Kihwal Lee eda786ea12 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 2018-01-29 17:52:04 -06:00
Jinjiang Ling 987a8972ac
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535)
 2018-01-30 00:13:04 +09:00
Xiao Chen 0922ea0aaf HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 
(cherry picked from commit b63dcd583f)
(cherry picked from commit 75be15cf02)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionFaultInjector.java
 2018-01-28 22:30:14 -08:00
Jian He 0497e09339 YARN-7765. Fixed an issue that kerberos tgt not found when NM posting timeline events. Contributed by Rohith Sharma K S 2018-01-28 13:24:49 -08:00
Steve Loughran 312c571694 HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-27 20:31:21 -08:00
Jason Lowe e3a1647477 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 2018-01-26 15:46:01 -06:00
Arpit Agarwal 7be5a46a41 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:36 -08:00
Chris Douglas 9bd439e2c5 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
(cherry picked from commit 7f35487788)
 2018-01-26 09:15:53 -08:00
Haibo Chen 5f44bd3eba MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cc10852252)
(cherry picked from commit e6c66baba9)
 2018-01-25 17:23:08 -08:00
Jason Lowe 4d7bbee0a1 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:56:36 -06:00
Wei Yan cefe4b7bb6 HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 12:58:16 -08:00
Sunil G bed683d300 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968)
 2018-01-25 14:34:31 +05:30
Yiqun Lin 9ce24d2d50 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff744)
 2018-01-25 15:54:20 +08:00
Jianfei Jiang f4656e2150
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd08688)
 2018-01-25 15:32:13 +09:00
Zhe Zhang e9a338ff64 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4)
(cherry picked from commit 83a4f92a3f)
 2018-01-24 13:10:35 -08:00
Jason Lowe 1ec82e5789 MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 
(cherry picked from commit cff9edd4b5)
 2018-01-24 14:51:22 -06:00
Jian He c611707e28 YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 11:19:46 -08:00
Sunil G ce43e93f92 YARN-7806. Distributed Shell should use timeline async api's. Contributed by Rohith Sharma K S 
(cherry picked from commit 97607617ce)
 2018-01-24 16:57:29 +05:30
Rohith Sharma K S f036609cea Revert "YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma" 
This reverts commit 41e10161b6.
 2018-01-24 15:05:53 +05:30
Inigo Goiri 1534936ac1 HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e)
 2018-01-23 19:17:22 -08:00
Jian He a80c2dbab5 YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang 2018-01-23 14:12:00 -08:00
Masahiro Tanaka ab482ebe73
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d)
 2018-01-22 17:15:31 +09:00
Vrushali C 41e10161b6 YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma 2018-01-19 15:59:47 -08:00
Haibo Chen a286acd7be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef)
(cherry picked from commit fc669778be)
 2018-01-19 13:12:26 -08:00
Haibo Chen 895a0391fe YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759)
(cherry picked from commit 8793e45f53)
 2018-01-19 10:35:40 -08:00
Inigo Goiri c6139332f9 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00)
 2018-01-19 09:52:29 -08:00
Yiqun Lin 65507053b4 HDFS-12973. RBF: Document global quota supporting in federation. Contributed by Yiqun Lin. 
(cherry picked from commit 9e4f52d323)
 2018-01-19 14:21:07 +08:00
Brahma Reddy Battula 8181367c62 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d0)
 2018-01-18 10:56:58 +05:30
Akira Ajisaka b0c9b89281
HADOOP-15177. Update the release year to 2018. Contributed by Bharat Viswanadham. 
(cherry picked from commit cdaf92c9f5)
 2018-01-18 13:17:07 +09:00
Miklos Szegedi 86a2ac94b1 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 2018-01-17 12:56:10 -08:00
Jason Lowe c228a7c707 MAPREDUCE-7029. FileOutputCommitter is slow on filesystems lacking recursive delete. Contributed by Karthik Palaniappan 2018-01-17 10:32:06 -06:00
Jason Lowe fd09f72151 Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 896dc7c780.
 2018-01-17 09:47:43 -06:00
Vinayakumar B b3eeea0b6e HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e1)
 2018-01-17 14:22:29 +05:30
Sammi Chen 896dc7c780 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 55142849db)
(cherry picked from commit 082a707bae)
 2018-01-17 16:36:03 +08:00
Brahma Reddy Battula 8e7ce0eb4c HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff)
 2018-01-16 16:29:13 +05:30
Rohith Sharma K S 8985fccbbc YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 
(cherry picked from commit d09058b2fd)
 2018-01-16 09:01:02 +05:30