Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,441 Commits 326 Branches 371 Tags
Commit Graph

15441 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
kevinzhao1661
2fe36b0038
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398949acfd0b6406abdaeb17456a25c37)
 2020-11-30 11:19:24 +09:00
Jim Brennan
6a58e6293f YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2020-11-17 17:36:22 +00:00
Ahmed Hussein
c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0fcaf73a3989000dbc60fa214a317da1)
(cherry picked from commit 23fe3bdab3410b8ed370fa6af9f2ed63817006ab)
(cherry picked from commit ac3e10af63c4a70e45b26acd50b01a93fa8fd605)
(cherry picked from commit 487eaa6dea14bba100583f30f196c0d51215bd7c)
 2020-11-12 10:55:35 -08:00
Akira Ajisaka
266c9ad66a
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811b271708c82bf4d5bedfc9d721366bf)
 2020-11-10 14:02:18 +09:00
Steve Loughran
894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr
9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea21e7e9286497ee17f89fd939d2eead)
 2020-11-06 15:07:34 -08:00
Eric Badger
7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d989770c4218c24e640f572bc7492f7d4b3)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang
fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f011905182dae2ddfbc68bb06341e2565424b9.
 2020-11-04 16:19:13 -08:00
Konstantin V Shvachko
a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4aa0d6cc9f9be530802e54610776a4d)
 2020-11-03 14:39:38 -08:00
Chen Liang
ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka
52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04cb8e5dab8410ef64daa0fe799e389fd)
(cherry picked from commit dd1634ec3b676d6c3da18af7da46734584ddd9f6)
(cherry picked from commit 1c7043b1ff89ba9c995cbf546a99bbbcccbe56b1)
(cherry picked from commit 166bee93af6fb68ee2b3fab4c355413cae9fc1d6)
 2020-11-03 11:24:08 -08:00
Jonathan Hung
6f436a6776 YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen 2020-10-28 10:45:34 -07:00
Wei-Chiu Chuang
e7e165a927 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-28 09:22:47 -07:00
Wei-Chiu Chuang
90ebbaa393 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
(cherry picked from commit 975d4b3d603632a5edacb138cf4a1ce92ebed02e)
 2020-10-24 13:38:58 -07:00
Akira Ajisaka
633e19cb49
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb4207af2f06f8ef57e83ff9759871d0cb)

 Conflicts:
	hadoop-project/pom.xml
 2020-10-19 11:04:28 +09:00
Eric Badger
c4b42fa1ae YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. 
Contributed by Jim Brennan.
 2020-10-16 19:29:04 +00:00
Konstantin V Shvachko
104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3cc13b0b92b9f42da1731c4ce35c9ded)
 2020-10-12 18:02:05 -07:00
Swaroopa Kadam
893fceadee
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:10:52 -07:00
Jim Brennan
4bb0124c69 YARN-9667. Container-executor.c duplicates messages to stdout. Contributed by Peter Bacsko 2020-10-12 15:41:33 +00:00
Eric Badger
3335fcfb05 YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by 
Ahmed Hussein.
 2020-10-09 20:41:58 +00:00
Adam Antal
1c0fe2eb20 YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan 2020-10-07 16:46:22 +02:00
Jim Brennan
0bf270d2ed YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne 
(cherry picked from commit ecf91638a8d1f546d90e92d076f32cb5681ce944)
 2020-10-06 18:46:08 +00:00
Wanqiang Ji
e8699d0bd5
MAPREDUCE-7289. Fix wrong comment in LongLong.java (#2338) 
(cherry picked from commit 143bdd41885a43c7bc6e11f0c0d6e5820768aa2c)
 2020-09-29 23:09:12 +09:00
zz
be421490da
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache. (#2319) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-22 11:57:36 -07:00
Masatake Iwasaki
7b0dbcb657 Publishing the bits for release 2.10.1 (addendum) 2020-09-22 11:51:53 +09:00
Masatake Iwasaki
0fd2ceb1b5 Publishing the bits for release 2.10.1 2020-09-22 11:48:18 +09:00
Wangda Tan
4885ea61fe Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
(cherry picked from commit ea18e70a74e811ffa48c7e18e68510dd37dda63d)
 2020-09-14 21:48:43 +09:00
Owen O'Malley
ca65409836 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e64c6edb11d852b91f2c816b1253eb3.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
(cherry picked from commit fa6a8b78d481d3b4d355e1bf078f30dd5e09850d)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationWithProxyUserFilter.java
 2020-09-14 21:48:17 +09:00
Masatake Iwasaki
f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Masatake Iwasaki
fdd8610ab2
HADOOP-17254. Upgrade hbase to 1.4.13 on branch-2.10. (#2290) 2020-09-11 08:54:30 +09:00
Wangda Tan
9a895a90d9 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
(cherry picked from commit b1833d9ba2c078582161da45ac392dd5c361dcdf)
(cherry picked from commit f9add5fdd9add4a5e6154de8ba6d9a17d123ac70)
 2020-09-10 14:40:05 -07:00
Masatake Iwasaki
7d8ee990cd
HADOOP-17253. Upgrade zookeeper to 3.4.14 on branch-2.10. (#2289) 2020-09-10 20:37:32 +09:00
Masatake Iwasaki
42ac46ff18
HADOOP-17249. Upgrade jackson-databind to 2.9.10.6 on branch-2.10. (#2279) 2020-09-09 09:55:11 +09:00
Masatake Iwasaki
0574de5f2f
HADOOP-17251. Upgrade netty-all to 4.1.50.Final on branch-2.10. (#2285) 2020-09-09 07:19:23 +09:00
Jitendra Pandey
43c6c54b8c HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 
(cherry picked from commit f29e55bf5e634b3be28c9e0e993f78877528b5c2)
 2020-09-08 22:35:01 +09:00
dependabot[bot]
7c838bf6ed Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a1567cac1bcf71e2e5f252cddffa0f97cc)
 2020-09-08 21:35:40 +09:00
Eric E Payne
e5bd8d2840 YARN-10177: Backport YARN-7307 to branch-2.10 Allow client/AM update supported resource types via YARN APIs 2020-09-04 18:23:08 +00:00
Steve Loughran
a75f2915f9 HADOOP-17199. S3A Directory Marker HADOOP-13230 backport 
This backports the listing-side changes of HADOOP-13230.

With this patch in, this branch of Hadoop is compatible with S3A clients
which do not delete directory markers when files are created underneath.

It does not allow this version to disable marker deletion; if the
fs.s3a.marker.retention option is changed to request this, a message
is printed at INFO and the policy remains at "delete"

The s3guard bucket-info command has been extended to support
probing for marker retention, as has the hasPathCapability method on
S3AFileSystem.

Read the documentation!

Change-Id: Ief23c39256b194abcc445231b5f7a018185ea9f6
 2020-09-03 18:29:30 +01:00
Masatake Iwasaki
ecf88b972f
YARN-10358. Fix findbugs warnings in hadoop-yarn-project on branch-2.10. (#2164) 2020-08-31 13:52:29 +09:00
Chen Liang
2b6b5476ba HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit f734455e5d76c75b7d3a0b751023f5bd02ba38d2)
 2020-08-28 17:48:38 -07:00
sguggilam
e645204733
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time (#2245) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-27 15:21:20 -07:00
Masatake Iwasaki
a9f82a11df
HADOOP-17202. Fix findbugs warnings in hadoop-tools on branch-2.10. (#2214) 2020-08-18 13:58:35 +09:00
Chao Sun
c7ab3195de
HADOOP-17205. Move personality file from Yetus to Hadoop repository (#2226) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fefacf2578e58cf7bf96d920f5b01dba66909ca2)
 2020-08-18 12:12:57 +09:00
prabhujoseph
30c95ddaff MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.

(cherry picked from commit a97f7776bd05f957a998b4883141855cad5afb31)
 2020-08-17 14:39:41 -07:00
prabhujoseph
f4f00d56a1 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.

(cherry picked from commit 6a4966f336c7b86a69d292b6157315e25963e527)
 2020-08-17 14:39:02 -07:00
Kihwal Lee
283def8bc5 MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 
(cherry picked from commit 4571351cccf6d4977469d3d623cf045b06a5f5f0)
 2020-08-13 18:54:31 -05:00
Akira Ajisaka
b87e080433
HADOOP-17204. Fix typo in Hadoop KMS document. Contributed by Xieming Li. 
(cherry picked from commit 141c62584bd253f0831d39557f54627f26cef7f7)
 2020-08-12 16:10:11 +09:00
Akira Ajisaka
2dffc1d8cb HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang. 
(cherry picked from commit 2ba9903932e3c99afb0e6abb7cd3c5d29a554bd9)
 2020-08-11 14:19:35 +09:00
Akira Ajisaka
7dedf344de MAPREDUCE-6881. Fix warnings from Spotbugs in hadoop-mapreduce. Contributed by Weiwei Yang. 
(cherry picked from commit 3ed3062fe3979ff55a411b730a8eee2b2c96d6b3)
 2020-08-11 11:19:01 +09:00
Eric E Payne
21788f9fd4 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan and Jim Brennan. 2020-08-10 20:52:44 +00:00