Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,242 Commits 326 Branches 371 Tags
Commit Graph

14242 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
ed512645b2 HDFS-8693. Addendum patch to execute the command using UGI. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 35c17351cab645dcc72e0d2ae1608507aa787ffb)
(cherry picked from commit 24af0bc144b7e89fc2ea8d2f49b2c8eeb5a2b725)
 2018-02-12 22:27:21 +05:30
Arpit Agarwal
41d434d08e HDFS-10453. ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster.. Contributed by He Xiaoqiao. 2018-02-12 07:12:02 -08:00
Yiqun Lin
050aa531ba HDFS-13099. RBF: Use the ZooKeeper as the default State Store. Contributed by Yiqun Lin. 
(cherry picked from commit 543f3abbee79d7ec70353f0cdda6397ee001324e)
 2018-02-09 14:03:13 +08:00
Xiaoyu Yao
77c6439c33 HDFS-13120. Snapshot diff could be corrupted after concat. Contributed by Xiaoyu Yao. 
(cherry picked from commit 8faf0b50d435039f69ea35f592856ca04d378809)
 2018-02-08 10:20:55 -08:00
Brahma Reddy Battula
3487f2b722 HDFS-12935. Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up. Contributed by Jianfei Jiang. 
(cherry picked from commit 9873eb63a7525301ab601a0ae65f7e615d1a6bce)
 2018-02-08 18:44:34 +05:30
Billie Rinaldi
52281fd893 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit 9229fb3523fbf1e98bd85a98530e3df9079d32b2.
 2018-02-07 09:57:46 -08:00
Kihwal Lee
39963d53df HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de983a30bf0d9ee957bdb09f825f9d40a3)
 2018-02-06 15:38:32 -06:00
Billie Rinaldi
43ea9a42d7 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 
(cherry picked from commit a196ee9362a1b35e5de20ee519f7c544ab1588e1)
 2018-02-06 12:06:25 -08:00
Jason Lowe
e73e820d2a YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 
(cherry picked from commit 60656bcecadd80e28c81bc943b44abf13d20abae)
 2018-02-05 17:29:49 -06:00
Haibo Chen
e78a701687 YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 
(cherry picked from commit a20e7105ea9d4e38d7f8f9fd48035e342bb22f1c)
(cherry picked from commit c25c082961a837b94016dcf0250bdc6d5f54e915)
 2018-02-05 10:51:39 -08:00
Yongjun Zhang
1ed6ae7b12 HDFS-13100. Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. Contributed by Aaron T. Myers and Yongjun Zhang. 
(cherry picked from commit 4e9a59ce16e81b4bd6fae443a997ef24d588a6e8)
 2018-02-02 23:45:28 -08:00
Yiqun Lin
024127e471 HDFS-13068. RBF: Add router admin option to manage safe mode. Contributed by Yiqun Lin. 
(cherry picked from commit 712e9381d2e3592953a7f9d66540864441842cbf)
 2018-02-02 17:37:14 +08:00
Inigo Goiri
6769c78cdd HDFS-13043. RBF: Expose the state of the Routers in the federation. Contributed by Inigo Goiri. 
(cherry picked from commit 6ca7204cebf4b7060696a07b1777723a6dfa7d9d)
 2018-02-01 10:39:25 -08:00
Inigo Goiri
38c3e17117 HDFS-13044. RBF: Add a safe mode for the Router. Contributed by Inigo Goiri. 
(cherry picked from commit 961dcb693d128a7416720d9bdf4ecf51a98c2dd1)
 2018-01-31 09:17:40 -08:00
Inigo Goiri
ffba476192 HDFS-13083. RBF: Fix doc error setting up client. Contributed by tartarus. 
(cherry picked from commit 5206b2c7ca479dd53f614d75bab594043a9866e1)
 2018-01-30 18:28:54 -08:00
Sammi Chen
e30abdaaae HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 91184299c54df18540f841cca0efe0131d05b882)
(cherry picked from commit 2816bd1f43bca3734a8eaae0b228aaa72b575792)
 2018-01-30 17:15:12 +08:00
Kihwal Lee
a15df67f65 HDFS-12574. Add CryptoInputStream to WebHdfsFileSystem read call. Contributed by Rushabh S Shah. 
(cherry picked from commit eda786ea12db6b3ca8d6b0565c4ebdeab28b3cf6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptionZones.java
 2018-01-29 17:57:54 -06:00
Jinjiang Ling
d70ca99594
YARN-7698. A misleading variable's name in ApplicationAttemptEventDispatcher 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3400d0c535aeb151c3f283cc41111b15d66990e5)
(cherry picked from commit 987a8972acf4201642881dc3dcda21f6cfcf3bc7)
 2018-01-30 00:13:26 +09:00
Steve Loughran
d9875b046f HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu

(cherry picked from commit 312c5716943b31f0f6363f6df02523f959eb981e)
 2018-01-27 21:00:25 -08:00
Jason Lowe
d69df0b596 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 
(cherry picked from commit e3a16474773248b116dad77c513e5e90a8235212)
 2018-01-26 15:51:34 -06:00
Chris Douglas
e062e2b08c MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
(cherry picked from commit 7f354877889b343878a8a09792d5cec8d2846a50)
(cherry picked from commit 9bd439e2c535b95ff0d2b5767b05a7ef43479298)
 2018-01-26 09:16:08 -08:00
Jason Lowe
98499bb098 YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 4d7bbee0a1dfc0b636cfbff0d0fd11f294985a31)
 2018-01-25 18:00:41 -06:00
Wei Yan
9951c2d04f HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 
(cherry picked from commit cefe4b7bb6322e7c89d7b129b41f5e1a2689ebcf)
 2018-01-25 12:59:36 -08:00
Sunil G
b354357733 YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968d2897ae006c5ef29eba74023f3483e)
 2018-01-25 15:55:45 +05:30
Yiqun Lin
44cfee9af4 HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff74494eb7fbbbba7f8bb4b4692d880d301)
 2018-01-25 15:55:24 +08:00
Jianfei Jiang
1b5f0e819e
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd0868864da994612f1dd1452ea38d31f5e85)
(cherry picked from commit f4656e215014fd3fbe4ac0336531f0b3c5df9396)
 2018-01-25 15:32:36 +09:00
Jian He
cbaf6d242d YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 16:18:36 -08:00
Zhe Zhang
258fb117d2 YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4896a602dc383d5e266768392a9790c)
(cherry picked from commit 83a4f92a3f0c0134709bcb0a534e31b7c7056643)
(cherry picked from commit e9a338ff644e4ac06f09762b30e90af8027377d7)
 2018-01-24 13:10:58 -08:00
Inigo Goiri
a43100adad HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e6b42b227a22dfa7cc16edc7bdb58bb)
 2018-01-23 19:17:47 -08:00
Jian He
6da08837da YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang 2018-01-23 14:07:41 -08:00
Masahiro Tanaka
93220f90ea
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d695ed94bd44c60f191cfbb5cdb32e2)
(cherry picked from commit ab482ebe73fbe24bf5242e705793e224f0f73500)
 2018-01-22 17:15:54 +09:00
Haibo Chen
c4ba8eb003 YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759f773135f8a27dbaa9731196fac5233)
(cherry picked from commit 8793e45f53607315bf6c99902452ba6cf2e3eb72)
(cherry picked from commit 895a0391fe4d7dc2b19bea6cc0528b7fd607f643)
 2018-01-19 10:38:18 -08:00
Inigo Goiri
8702882b54 HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00e8cab59c3aa4d668e8bddd84544713)
(cherry picked from commit c6139332f9d55ce176701e63426b158060c97d38)
 2018-01-19 09:53:22 -08:00
Jitendra Pandey
dd1de1ea37 HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 
(cherry picked from commit cb60c05c3aff762c39d7d81df9a7b6e071acb181)
 2018-01-18 20:18:19 -08:00
Miklos Szegedi
6e11ef7c44 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 2018-01-17 22:18:08 -08:00
Brahma Reddy Battula
3cdc392150 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d055d85472f0c9371fefe9b56bfea1ed)
 2018-01-18 11:02:38 +05:30
Akira Ajisaka
c614a39e9f
HADOOP-15177. Update the release year to 2018. Contributed by Bharat Viswanadham. 
(cherry picked from commit cdaf92c9f57560219b8f915a19ad8603ddf2a505)
(cherry picked from commit b0c9b89281c9264c70ae79b1e4830e0fa2411b4f)
 2018-01-18 13:17:42 +09:00
Jason Lowe
98835e14ae Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 622f6b65d684ce498a811784a229fb0386745711.
 2018-01-17 09:59:31 -06:00
Vinayakumar B
62b82d7f41 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
(cherry picked from commit b3eeea0b6e27f9e74c35d2a930b541617fc7c8f2)
 2018-01-17 14:23:33 +05:30
Sammi Chen
622f6b65d6 HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
(cherry picked from commit 55142849db02a9191db0dd6f4e1401ff19ec242a)
(cherry picked from commit 082a707bae4bb97444a34c00eecd62975807388d)
(cherry picked from commit 896dc7c7801adaa4460fa6c19a4f452a6a6112d8)
 2018-01-17 16:37:25 +08:00
Brahma Reddy Battula
28f69755f1 HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff7b5f350ec99bac9b0862009460b486)
 2018-01-16 16:33:34 +05:30
Sunil G
4e28aa4dcf YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:57:04 +05:30
Akira Ajisaka
fff7f77c96
YARN-7735. Fix typo in YARN documentation. Contributed by Takanobu Asanuma. 
(cherry picked from commit fbbbf59c82658e18dad7e0e256613187b5b75d0f)
(cherry picked from commit 68b7908aa828a070534e7853df56e74ed281af7e)
 2018-01-11 16:00:50 +09:00
Inigo Goiri
6dc0b885a4 HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 
(cherry picked from commit d9006d8a4e34eae78dfa1cf3be50eeb81c2fc11f)
 2018-01-09 19:02:45 -08:00
Haibo Chen
6602182a2b YARN-7716. metricsTimeStart and metricsTimeEnd should be all lower case in the doc. 
(cherry picked from commit f725b9e267c604fbcec09956e3a39caf04798809)
(cherry picked from commit cf816bc751e3b82be1f1931d6d0d5cbd77cb3e17)
(cherry picked from commit a58034cdfd592fa61a85c873b5249f9b6a3a825e)
 2018-01-09 09:05:25 -08:00
Jason Lowe
3701c399b3 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/RMAppAttemptImpl.java
 2018-01-09 11:00:10 -06:00
Jason Lowe
52097cf2f4 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit d2c2d8c913dfc12585394a1168d4a92934a4cfb4)
 2018-01-09 10:04:16 -06:00
bibinchundatt
7ee994a536 YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang. 
(cherry picked from commit b26e30ab1655a8cdf369862a5512db5c2a88ec46)
 2018-01-09 20:04:40 +05:30
Wei Yan
5e8029eff9 HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 
(cherry picked from commit 2ee0d64aceed876f57f09eb9efe1872b6de98d2e)
 2018-01-08 15:13:53 -08:00
Steve Loughran
a35267b47a HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt
(Backported to branch-2 via /HADOOP-15156)
 2018-01-08 15:12:41 +00:00