Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,541 Commits 326 Branches 371 Tags
Commit Graph

17541 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Masahiro Tanaka
dcb09c30b9
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d695ed94bd44c60f191cfbb5cdb32e2)
 2018-01-22 17:15:14 +09:00
Lei Xu
d33050b8cd HDFS-13039. StripedBlockReader.createBlockReader leaks socket on IOException. (Lei (Eddy) Xu) 
(cherry picked from commit 2ed9d61aadaffbf56ae0ed124520edab97646b49)
 2018-01-19 16:42:22 -08:00
Haibo Chen
fc669778be MAPREDUCE-6984. MR AM to clean up temporary files from previous attempt in case of no recovery. (Gergo Repas via Haibo Chen) 
(cherry picked from commit cce71dceef9e82d31fe8ec59648b2a4a50c8869a)
 2018-01-19 13:05:40 -08:00
Haibo Chen
8793e45f53 YARN-5094. some YARN container events have timestamp of -1. 
(cherry picked from commit 4aca4ff759f773135f8a27dbaa9731196fac5233)
 2018-01-19 10:29:33 -08:00
Inigo Goiri
6c10e0e12b HDFS-13028. RBF: Fix spurious TestRouterRpc#testProxyGetStats. Contributed by Inigo Goiri. 
(cherry picked from commit 130f8bce00e8cab59c3aa4d668e8bddd84544713)
 2018-01-19 09:48:45 -08:00
Jitendra Pandey
169dacf1ee HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 
(cherry picked from commit d31c9d8c495794a803fb20729b5ed6b374e23eb4)
 2018-01-18 20:02:48 -08:00
Brahma Reddy Battula
e575406715 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d055d85472f0c9371fefe9b56bfea1ed)
 2018-01-18 10:56:12 +05:30
Akira Ajisaka
88e3ad31b5
HADOOP-15177. Update the release year to 2018. Contributed by Bharat Viswanadham. 
(cherry picked from commit cdaf92c9f57560219b8f915a19ad8603ddf2a505)
 2018-01-18 13:16:46 +09:00
Miklos Szegedi
7629353153 YARN-7758. Add an additional check to the validity of container and application ids passed to container-executor. Contributed by Yufei Gu. 
(cherry picked from commit 41049ba5d129f0fd0953ed8fdeb12635f7546bb2)
 2018-01-17 13:47:44 -08:00
Vinayakumar B
2443692434 HDFS-9049. Make Datanode Netty reverse proxy port to be configurable. Contributed by Vinayakumar B. 
(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
 2018-01-17 13:01:52 -06:00
Lei Xu
ce394be809 HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei) 
(cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)
 2018-01-17 12:59:53 -06:00
Inigo Goiri
49e285ded5 HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer. Contributed by Inigo Goiri. 2018-01-17 12:57:14 -06:00
Yiqun Lin
cb07172e21 HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under a given path. Contributed by Yiqun Lin. 2018-01-17 12:50:06 -06:00
Manoj Govindassamy
710555e150 HDFS-11847. Enhance dfsadmin listOpenFiles command to list files blocking datanode decommissioning. 2018-01-17 12:44:09 -06:00
Jason Lowe
b1cc14e9cb Revert "HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu)" 
This reverts commit 55142849db02a9191db0dd6f4e1401ff19ec242a.
 2018-01-17 09:53:24 -06:00
Sammi Chen
55142849db HADOOP-15027. AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance. (Contributed by Jinhu Wu) 
(cherry picked from commit 9195a6e302028ed3921d1016ac2fa5754f06ebf0)
 2018-01-17 16:12:23 +08:00
Robert Kanter
e54c65a326 YARN-7479. TestContainerManagerSecurity.testContainerManager[Simple] flaky in trunk (ajisakaa via rkanter) 
(cherry picked from commit 5ac109909a29fab30363b752b5215be7f5dc616b)
 2018-01-16 10:21:12 -08:00
Brahma Reddy Battula
d03115d028 HDFS-8693. refreshNamenodes does not support adding a new standby to a running DN. Contributed by Ajith S. 
(cherry picked from commit 880b9d24ff7b5f350ec99bac9b0862009460b486)
 2018-01-16 16:22:10 +05:30
Rohith Sharma K S
2a01652809 YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 
(cherry picked from commit d09058b2fd18803d12f0835fdf78aef5e0b99c90)
 2018-01-16 08:42:07 +05:30
Miklos Szegedi
75c0b19bea YARN-7705. Create the container log directory with correct sticky bit in C code. Contributed by Yufei Gu. 
(cherry picked from commit 2dcfc18)
 2018-01-12 17:55:58 -08:00
Wei-Chiu Chuang
b830040766 HDFS-11915. Sync rbw dir on the first hsync() to avoid file lost on power failure. Contributed by Vinayakumar B. 
(cherry picked from commit d6c31a3e6b60c4b8af9ae4661f16614805654e59)
 2018-01-12 10:42:58 -08:00
Sunil G
f270818ed3 YARN-7727. Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector. Contributed by Prabhu Joseph. 
(cherry picked from commit 128d773a2315fa6baaa3a52b13c53c77e741b69c)
 2018-01-12 15:42:06 +05:30
Miklos Szegedi
ed9c516bc8 YARN-7590. Improve container-executor validation check. Contributed by Eric Yang. 
(cherry picked from commit bc285da)
 2018-01-11 10:28:08 -08:00
Akira Ajisaka
a13a773edb
YARN-7735. Fix typo in YARN documentation. Contributed by Takanobu Asanuma. 
(cherry picked from commit fbbbf59c82658e18dad7e0e256613187b5b75d0f)
 2018-01-11 15:59:55 +09:00
Miklos Szegedi
542cb8480c HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 
(cherry picked from commit 12d0645)
 2018-01-10 11:04:40 -08:00
Inigo Goiri
5ed1eff48b HDFS-12802. RBF: Control MountTableResolver cache size. Contrubuted by Inigo Goiri. 
(cherry picked from commit d9006d8a4e34eae78dfa1cf3be50eeb81c2fc11f)
 2018-01-09 18:55:46 -08:00
Haibo Chen
7b32d6811c YARN-7665. Allow FS scheduler state dump to be turned on/off separately from FS debug log. (Wilfred Spiegelenburg via Haibo Chen) 
(cherry picked from commit 8ee7080e5da15d8841d5f7bbf72ca033905c9751)
 2018-01-09 16:53:10 -08:00
Lei Xu
92f63d04a1 HDFS-12994. TestReconstructStripedFile.testNNSendsErasureCodingTasks fails due to socket timeout. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 47563d86fe6ba1a2de934c9ed740d0aafbf72d4e)
 2018-01-09 12:26:30 -08:00
Haibo Chen
cf816bc751 YARN-7716. metricsTimeStart and metricsTimeEnd should be all lower case in the doc. 
(cherry picked from commit f725b9e267c604fbcec09956e3a39caf04798809)
 2018-01-09 08:59:04 -08:00
Jason Lowe
effce9b117 YARN-6948. Invalid event: ATTEMPT_ADDED at FINAL_SAVING. Contributed by lujie 
(cherry picked from commit 90c86d6e090d668414681fcb5eb02add5d419b33)
 2018-01-09 10:52:21 -06:00
Jason Lowe
d2c2d8c913 YARN-7663. RMAppImpl:Invalid event: START at KILLED. Contributed by lujie 
(cherry picked from commit ebff4decfe86571d203859928e28e6855862737b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
 2018-01-09 09:57:45 -06:00
bibinchundatt
c57e71b542 YARN-7508. NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode. Contributed by Tao Yang. 
(cherry picked from commit b26e30ab1655a8cdf369862a5512db5c2a88ec46)
 2018-01-09 20:01:42 +05:30
Kai Zheng
539fa11325 HADOOP-15104. AliyunOSS: change the default value of max error retry. Contributed by Jinhu Wu 
(cherry picked from commit ce04340ec73617daff74378056a95c5d0cc0a790)
 2018-01-09 15:41:12 +08:00
Wei Yan
cbecf554c9 HDFS-12945. Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods. Contributed by Wei Yan. 
(cherry picked from commit 2ee0d64aceed876f57f09eb9efe1872b6de98d2e)
 2018-01-08 14:56:22 -08:00
Rohith Sharma K S
281350674c YARN-7699. queueUsagePercentage is coming as INF for getApp REST api call. Contributed by Sunil G. 2018-01-08 15:52:03 +05:30
Inigo Goiri
53a7bd186d HDFS-12988. RBF: Mount table entries not properly updated in the local cache. Contributed by Inigo Goiri. 
(cherry picked from commit 83b513ac6d5448f3771b0b95f91e7aa7961ae2cc)
 2018-01-05 09:12:43 -08:00
Sunil G
a79abbc03e YARN-7619. Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user. Contributed by Eric Payne. 2018-01-05 14:50:17 +05:30
Lei Xu
47773d68b2 HDFS-12860. StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB. (Contributed by Lei (Eddy) Xu) 
(cherry picked from commit 85067eaa8e352afeed9c094da3815100873c84e4)
 2018-01-04 10:33:34 -08:00
Jason Lowe
c155d8e756 YARN-7678. Ability to enable logging of container memory stats. Contributed by Jim Brennan 
(cherry picked from commit d795661868e330ac55d6ac7b0ee62fb658e03ff7)
 2018-01-04 10:18:32 -06:00
Sean Mackrory
91b59b7c28 HDFS-12913. TestDNFencingWithReplication sometimes fails because of NameNode in standby state. Contributed by Zsolt Venczel. 
(cherry picked from commit 45a47198b43a96817679d5bf9e570d405555e3eb)
 2018-01-04 08:47:02 -07:00
Haibo Chen
889bebf8bb YARN-7602. NM should reference the singleton JvmMetrics instance. 
(cherry picked from commit 2f6c038be6c23522ea64fc4e415910fb72493eb2)
 2018-01-03 09:44:33 -08:00
Jason Lowe
2a870d91fe MAPREDUCE-7028. Concurrent task progress updates causing NPE in Application Master. Contributed by Gergo Repas 
(cherry picked from commit fe35103591ece0209f8345aba5544313e45a073c)
 2018-01-03 11:03:16 -06:00
Rohith Sharma K S
2c67985bd2 YARN-7692. Skip validating priority acls while recovering applications. Contributed by Sunil G. 
(cherry picked from commit c9bf813c9a6c018d14f2bef49ba086ec0e60c761)
 2018-01-03 18:21:47 +05:30
Arun Suresh
829917e9ba YARN-7691. Add Unit Tests for ContainersLauncher. (Sampada Dehankar via asuresh) 
(cherry picked from commit c0c7cce81d5609f6347bff67929d5026d5893d75)
 2018-01-02 22:03:56 -08:00
Miklos Szegedi
c3cff8eac6 HADOOP-15122. Lock down version of doxia-module-markdown plugin. Contributed by Marton Elek. 
(cherry picked from commit 4bb765ee2e9fa16bb18c70d8f62ab885d883f414)
 2017-12-29 17:53:32 -08:00
Xiao Chen
8c275c63ff HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 
(cherry picked from commit 81127616c571b7cd25686e60a1105f96ca0626b7)
 2017-12-29 13:41:43 -08:00
Lei Xu
8f9603cb9d HDFS-12915. Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy. (Contributed by Chris Douglas) 
(cherry picked from commit 6e3e1b8cde737e4c03b0f5279cab0239e7069a72)
 2017-12-29 12:23:33 -08:00
Arun Suresh
bd4dcc7772 YARN-7542. Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED. (Sampada Dehankar via asuresh) 
(cherry picked from commit a55884c68eb175f1c9f61771386c086bf1ee65a9)
 2017-12-28 22:21:52 -08:00
Xiao Chen
b7701495f4 HDFS-9023. When NN is not able to identify DN for replication, reason behind it can be logged. 
(cherry picked from commit 5bf7e594d7d54e5295fe4240c3d60c08d4755ab7)
 2017-12-28 11:55:14 -08:00
Steve Loughran
72a49503fe HADOOP-15086. NativeAzureFileSystem file rename is not atomic. 
Contributed by Thomas Marquardt

(cherry picked from commit 52babbb4a0e3c89f2025bf6e9a1b51a96e8f8fb0)
 2017-12-22 14:31:20 +00:00