Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,460 Commits 326 Branches 371 Tags
Commit Graph

17460 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Steve Loughran
ec4f5f0f0d HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-26 15:23:08 -08:00
Jason Lowe
d7acc45ce4 MAPREDUCE-7020. Task timeout in uber mode can crash AM. Contributed by Peter Bacsko 
(cherry picked from commit 6eef3d7f1a1e5e3f27fb3bf7596663640d786181)
 2018-01-26 15:37:16 -06:00
Arpit Agarwal
480b69c03c HDFS-13054. Handling PathIsNotEmptyDirectoryException in DFSClient delete call. Contributed by Nanda kumar. 2018-01-26 11:45:23 -08:00
Chris Douglas
7f35487788 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
 2018-01-26 09:15:03 -08:00
Haibo Chen
e6c66baba9 MAPREDUCE-7041. MR should not try to clean up at first job attempt. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cc10852252c2d69294eabc68bd032cc630a53b18)
 2018-01-25 17:14:25 -08:00
Jason Lowe
217f408aaa YARN-7102. NM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit ff8378eb1b960c72d18a984c7e5d145b407ca11a)
 2018-01-25 17:51:51 -06:00
Wei Yan
c5c042135d HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 
(cherry picked from commit 0c139d5bcfbcd62fc69111ee6204926c57d57bf1)
 2018-01-25 12:52:45 -08:00
Sunil G
88bacad50f YARN-7728. Expose container preemptions related information in Capacity Scheduler queue metrics. Contributed by Eric Payne. 
(cherry picked from commit 82cc6f6968d2897ae006c5ef29eba74023f3483e)
 2018-01-25 14:33:42 +05:30
Yiqun Lin
52ead8b90a HDFS-13042. RBF: Heartbeat Router State. Contributed by Inigo Goiri. 
(cherry picked from commit 7721fff74494eb7fbbbba7f8bb4b4692d880d301)
 2018-01-25 15:53:09 +08:00
Jianfei Jiang
2a36b4b1ed
HDFS-13063. Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit eb2dd0868864da994612f1dd1452ea38d31f5e85)
 2018-01-25 15:31:45 +09:00
Zhe Zhang
83a4f92a3f YARN-7737. prelaunch.err file not found exception on container failure. Contributed by Keqiu Hu. 
(cherry picked from commit fa8cf4d1b4896a602dc383d5e266768392a9790c)
 2018-01-24 12:57:27 -08:00
Jason Lowe
5281c188df MAPREDUCE-7015. Possible race condition in JHS if the job is not loaded. Contributed by Peter Bacsko 
(cherry picked from commit cff9edd4b514bdcfe22cd49964e3707fb78ab876)
 2018-01-24 14:45:46 -06:00
Jian He
7598f73320 YARN-7801. AmFilterInitializer should addFilter after fill all parameters. Contributed by Wangda Tan 2018-01-24 11:20:23 -08:00
Sunil G
ecfdd01be7 YARN-7806. Distributed Shell should use timeline async api's. Contributed by Rohith Sharma K S 
(cherry picked from commit 97607617ce01ee21af4fb9eb8df26fc6986a38a2)
 2018-01-24 16:56:51 +05:30
Miklos Szegedi
bf795560fb YARN-7796. Container-executor fails with segfault on certain OS configurations. Contributed by Gergo Repas. 
(cherry picked from commit e7642a3e6f540b4b56367babfbaf35ee6b3c7675)
 2018-01-23 22:21:12 -08:00
Inigo Goiri
e4ad5f7f37 HDFS-12772. RBF: Federation Router State State Store internal API. Contributed by Inigo Goiri. 
(cherry picked from commit 95743c672e6b42b227a22dfa7cc16edc7bdb58bb)
 2018-01-23 19:16:27 -08:00
Jian He
92f789a131 YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang 2018-01-23 14:06:48 -08:00
Steve Loughran
1de0712578 HADOOP-15185. Update adls connector to use the current version of ADLS SDK. 
Contributed by Atul Sikaria.

(cherry picked from commit f63d13f10db708228482b53883863c873c89f69c)
 2018-01-23 11:37:33 -08:00
Hanisha Koneru
304642322f HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 
(cherry picked from commit 3fde0f1db599227773c0cd537b33312d368ad4d9)
 2018-01-22 16:26:23 -08:00
Masahiro Tanaka
dcb09c30b9
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d695ed94bd44c60f191cfbb5cdb32e2)
 2018-01-22 17:15:14 +09:00
Lei Xu
d33050b8cd HDFS-13039. StripedBlockReader.createBlockReader leaks socket on IOException. (Lei (Eddy) Xu) 
(cherry picked from commit 2ed9d61aadaffbf56ae0ed124520edab97646b49)
 2018-01-19 16:42:22 -08:00
Haibo Chen
fc669778be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef9e82d31fe8ec59648b2a4a50c8869a)
 2018-01-19 13:05:40 -08:00
Haibo Chen
8793e45f53 YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759f773135f8a27dbaa9731196fac5233)
 2018-01-19 10:29:33 -08:00
Inigo Goiri
6c10e0e12b HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00e8cab59c3aa4d668e8bddd84544713)
 2018-01-19 09:48:45 -08:00
Jitendra Pandey
169dacf1ee HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 
(cherry picked from commit d31c9d8c495794a803fb20729b5ed6b374e23eb4)
 2018-01-18 20:02:48 -08:00
Brahma Reddy Battula
e575406715 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d055d85472f0c9371fefe9b56bfea1ed)
 2018-01-18 10:56:12 +05:30
Akira Ajisaka
88e3ad31b5
HADOOP-15177. Update the release year to 2018. Contributed by Bharat Viswanadham. 
(cherry picked from commit cdaf92c9f57560219b8f915a19ad8603ddf2a505)
 2018-01-18 13:16:46 +09:00
Miklos Szegedi
7629353153 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 
(cherry picked from commit 41049ba5d129f0fd0953ed8fdeb12635f7546bb2)
 2018-01-17 13:47:44 -08:00
Vinayakumar B
2443692434 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
 2018-01-17 13:01:52 -06:00
Lei Xu
ce394be809 HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei) 
(cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)
 2018-01-17 12:59:53 -06:00
Inigo Goiri
49e285ded5 HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-17 12:57:14 -06:00
Yiqun Lin
cb07172e21 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-17 12:50:06 -06:00
Manoj Govindassamy
710555e150 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-17 12:44:09 -06:00
Jason Lowe
b1cc14e9cb Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 55142849db02a9191db0dd6f4e1401ff19ec242a.
 2018-01-17 09:53:24 -06:00
Sammi Chen
55142849db HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
 2018-01-17 16:12:23 +08:00
Robert Kanter
e54c65a326 YARN-7479. TestContainerManagerSecurity.testContainerManager[Simple] flaky in trunk (ajisakaa via rkanter) 
(cherry picked from commit 5ac109909a29fab30363b752b5215be7f5dc616b)
 2018-01-16 10:21:12 -08:00
Brahma Reddy Battula
d03115d028 HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff7b5f350ec99bac9b0862009460b486)
 2018-01-16 16:22:10 +05:30
Rohith Sharma K S
2a01652809 YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 
(cherry picked from commit d09058b2fd18803d12f0835fdf78aef5e0b99c90)
 2018-01-16 08:42:07 +05:30
Miklos Szegedi
75c0b19bea YARN-7705. Create the container log directory with correct sticky bit in C code. Contributed by Yufei Gu. 
(cherry picked from commit 2dcfc18)
 2018-01-12 17:55:58 -08:00
Wei-Chiu Chuang
b830040766 HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 
(cherry picked from commit d6c31a3e6b60c4b8af9ae4661f16614805654e59)
 2018-01-12 10:42:58 -08:00
Sunil G
f270818ed3 YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:42:06 +05:30
Miklos Szegedi
ed9c516bc8 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 
(cherry picked from commit bc285da)
 2018-01-11 10:28:08 -08:00
Akira Ajisaka
a13a773edb
YARN-7735. Fix typo in YARN documentation. Contributed by Takanobu Asanuma. 
(cherry picked from commit fbbbf59c82658e18dad7e0e256613187b5b75d0f)
 2018-01-11 15:59:55 +09:00
Miklos Szegedi
542cb8480c HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 
(cherry picked from commit 12d0645)
 2018-01-10 11:04:40 -08:00
Inigo Goiri
5ed1eff48b HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 
(cherry picked from commit d9006d8a4e34eae78dfa1cf3be50eeb81c2fc11f)
 2018-01-09 18:55:46 -08:00
Haibo Chen
7b32d6811c YARN-7665. Allow FS scheduler state dump to be turned on/off separately from FS debug log. (Wilfred Spiegelenburg via Haibo Chen) 
(cherry picked from commit 8ee7080e5da15d8841d5f7bbf72ca033905c9751)
 2018-01-09 16:53:10 -08:00
Lei Xu
92f63d04a1 HDFS-12994. TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 47563d86fe6ba1a2de934c9ed740d0aafbf72d4e)
 2018-01-09 12:26:30 -08:00
Haibo Chen
cf816bc751 YARN-7716. metricsTimeStart and metricsTimeEnd should be all lower case in the doc. 
(cherry picked from commit f725b9e267c604fbcec09956e3a39caf04798809)
 2018-01-09 08:59:04 -08:00
Jason Lowe
effce9b117 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)
 2018-01-09 10:52:21 -06:00
Jason Lowe
d2c2d8c913 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit ebff4decfe86571d203859928e28e6855862737b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
 2018-01-09 09:57:45 -06:00