Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,280 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

14280 Commits

Author SHA1 Message Date
Kihwal Lee fe044e69c3 HDFS-13112. Token expiration edits may cause log corruption or deadlock. Contributed by Daryn Sharp. 
(cherry picked from commit 47473952e5)
 2018-02-15 15:38:49 -06:00
Jason Lowe a87bccdc3f MAPREDUCE-7052. TestFixedLengthInputFormat#testFormatCompressedIn is flaky. Contributed by Peter Bacsko 
(cherry picked from commit a53d62ab26)
 2018-02-15 15:27:46 -06:00
Kihwal Lee b725fd6924 xattr api cleanup 
(cherry picked from commit da59acd8ca)
 2018-02-15 11:28:45 -06:00
Steve Loughran 07c7df4b26 HADOOP-13972. ADLS to support per-store configuration. 
Contributed by Sharad Sonker.

(cherry picked from commit 050f5287b7)
 2018-02-15 16:29:44 +00:00
Steve Loughran 6f944452f7 HADOOP-15090. Add ADL troubleshooting doc. 
Contributed by Steve Loughran.

(cherry picked from commit 58a2120e8a)
 2018-02-15 14:26:36 +00:00
Kihwal Lee 56b950d777 Revert "HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)" 
This reverts commit 577a267198.
 2018-02-14 15:20:45 -06:00
Ray Chiang 577a267198 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179)
 2018-02-14 00:07:09 +05:30
Jason Lowe 08feac4c83 MAPREDUCE-7048. Uber AM can crash due to unknown task in statusUpdate. Contributed by Peter Bacsko 2018-02-12 14:30:42 -06:00
Brahma Reddy Battula 24af0bc144 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351ca)
 2018-02-12 22:25:44 +05:30
Arpit Agarwal 82530c966a HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:02:42 -08:00
Yiqun Lin d49074f0a2 HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee)
 2018-02-09 14:00:55 +08:00
Brahma Reddy Battula 9873eb63a7 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 2018-02-08 18:33:11 +05:30
Yongjun Zhang b7f7fb003e HDFS-13115. In getNumUnderConstructionBlocks(), ignore the inodeIds for which the inodes have been deleted. Contributed by Yongjun Zhang. 
(cherry picked from commit f491f717e9)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/LeaseManager.java
(cherry picked from commit 60d82ac61f)
 2018-02-07 15:35:41 -08:00
Billie Rinaldi aa64a50746 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit a72dcb9cad.
 2018-02-07 08:58:28 -08:00
Kihwal Lee 34466c6b16 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de98)
 2018-02-06 15:36:32 -06:00
Billie Rinaldi 93128c1904 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362)
 2018-02-06 12:04:52 -08:00
Jason Lowe 260f30f063 YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bceca)
 2018-02-05 17:22:42 -06:00
Haibo Chen c25c082961 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea)
 2018-02-05 10:51:03 -08:00
Yongjun Zhang 6dfea05049 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16)
 2018-02-02 23:33:10 -08:00
Yiqun Lin 7cd3770fee HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit b0627c891b)
 2018-02-02 17:38:32 +08:00
Inigo Goiri 7b8cc048cc HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204ceb)
 2018-02-01 10:38:39 -08:00
Yiqun Lin e5cac88c82 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit dbb9dded33)
 2018-01-31 09:14:31 -08:00
Weiwei Yang 0c2c4c20cb HDFS-12528. Add an option to not disable short-circuit reads on failures. Contributed by Xiao Chen. 
(cherry picked from commit 2e7331ca26)
 2018-01-30 21:40:07 -08:00
Inigo Goiri fffa7dc377 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca)
 2018-01-30 18:28:21 -08:00
Sammi Chen 2816bd1f43 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302)
(cherry picked from commit 91184299c5)
 2018-01-30 16:25:02 +08:00
Kihwal Lee eda786ea12 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 2018-01-29 17:52:04 -06:00
Jinjiang Ling 987a8972ac
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535)
 2018-01-30 00:13:04 +09:00
Xiao Chen 0922ea0aaf HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 
(cherry picked from commit b63dcd583f)
(cherry picked from commit 75be15cf02)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionFaultInjector.java
 2018-01-28 22:30:14 -08:00
Jian He 0497e09339 YARN-7765. Fixed an issue that kerberos tgt not found when NM posting timeline events. Contributed by Rohith Sharma K S 2018-01-28 13:24:49 -08:00
Steve Loughran 312c571694 HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-27 20:31:21 -08:00
Jason Lowe e3a1647477 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 2018-01-26 15:46:01 -06:00
Arpit Agarwal 7be5a46a41 HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:36 -08:00
Chris Douglas 9bd439e2c5 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
(cherry picked from commit 7f35487788)
 2018-01-26 09:15:53 -08:00
Haibo Chen 5f44bd3eba MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cc10852252)
(cherry picked from commit e6c66baba9)
 2018-01-25 17:23:08 -08:00
Jason Lowe 4d7bbee0a1 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2018-01-25 17:56:36 -06:00
Wei Yan cefe4b7bb6 HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 12:58:16 -08:00
Sunil G bed683d300 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968)
 2018-01-25 14:34:31 +05:30
Yiqun Lin 9ce24d2d50 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff744)
 2018-01-25 15:54:20 +08:00
Jianfei Jiang f4656e2150
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd08688)
 2018-01-25 15:32:13 +09:00
Zhe Zhang e9a338ff64 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4)
(cherry picked from commit 83a4f92a3f)
 2018-01-24 13:10:35 -08:00
Jason Lowe 1ec82e5789 MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 
(cherry picked from commit cff9edd4b5)
 2018-01-24 14:51:22 -06:00
Jian He c611707e28 YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 11:19:46 -08:00
Sunil G ce43e93f92 YARN-7806. Distributed Shell should use timeline async api's. Contributed by Rohith Sharma K S 
(cherry picked from commit 97607617ce)
 2018-01-24 16:57:29 +05:30
Rohith Sharma K S f036609cea Revert "YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma" 
This reverts commit 41e10161b6.
 2018-01-24 15:05:53 +05:30
Inigo Goiri 1534936ac1 HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e)
 2018-01-23 19:17:22 -08:00
Jian He a80c2dbab5 YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang 2018-01-23 14:12:00 -08:00
Masahiro Tanaka ab482ebe73
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d)
 2018-01-22 17:15:31 +09:00
Vrushali C 41e10161b6 YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma 2018-01-19 15:59:47 -08:00
Haibo Chen a286acd7be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef)
(cherry picked from commit fc669778be)
 2018-01-19 13:12:26 -08:00
Haibo Chen 895a0391fe YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759)
(cherry picked from commit 8793e45f53)
 2018-01-19 10:35:40 -08:00