Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,384 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

15384 Commits

Author SHA1 Message Date
Wei-Chiu Chuang fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f0119051.
 2020-11-04 16:19:13 -08:00
Konstantin V Shvachko a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4)
 2020-11-03 14:39:38 -08:00
Chen Liang ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka 52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04c)
(cherry picked from commit dd1634ec3b)
(cherry picked from commit 1c7043b1ff)
(cherry picked from commit 166bee93af)
 2020-11-03 11:24:08 -08:00
Jonathan Hung 6f436a6776 YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen 2020-10-28 10:45:34 -07:00
Wei-Chiu Chuang e7e165a927 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-28 09:22:47 -07:00
Wei-Chiu Chuang 90ebbaa393 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837a)
(cherry picked from commit 975d4b3d60)
 2020-10-24 13:38:58 -07:00
Akira Ajisaka 633e19cb49
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb42)

 Conflicts:
	hadoop-project/pom.xml
 2020-10-19 11:04:28 +09:00
Eric Badger c4b42fa1ae YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. 
Contributed by Jim Brennan.
 2020-10-16 19:29:04 +00:00
Konstantin V Shvachko 104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3c)
 2020-10-12 18:02:05 -07:00
Swaroopa Kadam 893fceadee
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:10:52 -07:00
Jim Brennan 4bb0124c69 YARN-9667. Container-executor.c duplicates messages to stdout. Contributed by Peter Bacsko 2020-10-12 15:41:33 +00:00
Eric Badger 3335fcfb05 YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by 
Ahmed Hussein.
 2020-10-09 20:41:58 +00:00
Adam Antal 1c0fe2eb20 YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan 2020-10-07 16:46:22 +02:00
Jim Brennan 0bf270d2ed YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne 
(cherry picked from commit ecf91638a8)
 2020-10-06 18:46:08 +00:00
Wanqiang Ji e8699d0bd5
MAPREDUCE-7289. Fix wrong comment in LongLong.java (#2338) 
(cherry picked from commit 143bdd4188)
 2020-09-29 23:09:12 +09:00
zz be421490da
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache. (#2319) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-22 11:57:36 -07:00
Masatake Iwasaki 7b0dbcb657 Publishing the bits for release 2.10.1 (addendum) 2020-09-22 11:51:53 +09:00
Masatake Iwasaki 0fd2ceb1b5 Publishing the bits for release 2.10.1 2020-09-22 11:48:18 +09:00
Wangda Tan 4885ea61fe Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
(cherry picked from commit ea18e70a74)
 2020-09-14 21:48:43 +09:00
Owen O'Malley ca65409836 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
(cherry picked from commit fa6a8b78d4)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationWithProxyUserFilter.java
 2020-09-14 21:48:17 +09:00
Masatake Iwasaki f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Masatake Iwasaki fdd8610ab2
HADOOP-17254. Upgrade hbase to 1.4.13 on branch-2.10. (#2290) 2020-09-11 08:54:30 +09:00
Wangda Tan 9a895a90d9 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
(cherry picked from commit b1833d9ba2)
(cherry picked from commit f9add5fdd9)
 2020-09-10 14:40:05 -07:00
Masatake Iwasaki 7d8ee990cd
HADOOP-17253. Upgrade zookeeper to 3.4.14 on branch-2.10. (#2289) 2020-09-10 20:37:32 +09:00
Masatake Iwasaki 42ac46ff18
HADOOP-17249. Upgrade jackson-databind to 2.9.10.6 on branch-2.10. (#2279) 2020-09-09 09:55:11 +09:00
Masatake Iwasaki 0574de5f2f
HADOOP-17251. Upgrade netty-all to 4.1.50.Final on branch-2.10. (#2285) 2020-09-09 07:19:23 +09:00
Jitendra Pandey 43c6c54b8c HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 
(cherry picked from commit f29e55bf5e)
 2020-09-08 22:35:01 +09:00
dependabot[bot] 7c838bf6ed Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a156)
 2020-09-08 21:35:40 +09:00
Eric E Payne e5bd8d2840 YARN-10177: Backport YARN-7307 to branch-2.10 Allow client/AM update supported resource types via YARN APIs 2020-09-04 18:23:08 +00:00
Steve Loughran a75f2915f9 HADOOP-17199. S3A Directory Marker HADOOP-13230 backport 
This backports the listing-side changes of HADOOP-13230.

With this patch in, this branch of Hadoop is compatible with S3A clients
which do not delete directory markers when files are created underneath.

It does not allow this version to disable marker deletion; if the
fs.s3a.marker.retention option is changed to request this, a message
is printed at INFO and the policy remains at "delete"

The s3guard bucket-info command has been extended to support
probing for marker retention, as has the hasPathCapability method on
S3AFileSystem.

Read the documentation!

Change-Id: Ief23c39256b194abcc445231b5f7a018185ea9f6
 2020-09-03 18:29:30 +01:00
Masatake Iwasaki ecf88b972f
YARN-10358. Fix findbugs warnings in hadoop-yarn-project on branch-2.10. (#2164) 2020-08-31 13:52:29 +09:00
Chen Liang 2b6b5476ba HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit f734455e5d)
 2020-08-28 17:48:38 -07:00
sguggilam e645204733
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time (#2245) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-27 15:21:20 -07:00
Masatake Iwasaki a9f82a11df
HADOOP-17202. Fix findbugs warnings in hadoop-tools on branch-2.10. (#2214) 2020-08-18 13:58:35 +09:00
Chao Sun c7ab3195de
HADOOP-17205. Move personality file from Yetus to Hadoop repository (#2226) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fefacf2578)
 2020-08-18 12:12:57 +09:00
prabhujoseph 30c95ddaff MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.

(cherry picked from commit a97f7776bd)
 2020-08-17 14:39:41 -07:00
prabhujoseph f4f00d56a1 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.

(cherry picked from commit 6a4966f336)
 2020-08-17 14:39:02 -07:00
Kihwal Lee 283def8bc5 MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 
(cherry picked from commit 4571351ccc)
 2020-08-13 18:54:31 -05:00
Akira Ajisaka b87e080433
HADOOP-17204. Fix typo in Hadoop KMS document. Contributed by Xieming Li. 
(cherry picked from commit 141c62584b)
 2020-08-12 16:10:11 +09:00
Akira Ajisaka 2dffc1d8cb HADOOP-14400. Fix warnings from spotbugs in hadoop-tools. Contributed by Weiwei Yang. 
(cherry picked from commit 2ba9903932)
 2020-08-11 14:19:35 +09:00
Akira Ajisaka 7dedf344de MAPREDUCE-6881. Fix warnings from Spotbugs in hadoop-mapreduce. Contributed by Weiwei Yang. 
(cherry picked from commit 3ed3062fe3)
 2020-08-11 11:19:01 +09:00
Eric E Payne 21788f9fd4 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan and Jim Brennan. 2020-08-10 20:52:44 +00:00
Jonathan Hung 865828ae63 YARN-10251. Show extended resources on legacy RM UI. Contributed by Eric Payne 2020-08-07 17:45:04 -07:00
Eric Badger 120cd9eae6 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan. 2020-08-06 19:40:33 +00:00
Eric Badger 9bf554deae YARN-4575. ApplicationResourceUsageReport should return ALL reserved resource. 
Contributed by Bibin Chundatt and Eric Payne.

(cherry picked from commit 647be0c0f6)
 2020-08-05 23:20:52 +00:00
sguggilam 4eb1f81876
HADOOP-17164 UGI loginUserFromKeytab doesn't set the last login time (#2194) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-05 09:04:01 -07:00
Mingliang Liu 8d5821b9c3
HDFS-15499. Clean up httpfs/pom.xml to remove aws-java-sdk-s3 exclusion (#2188) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2020-08-04 23:27:46 -07:00
Akira Ajisaka cd596a0fdb
HADOOP-17185. Set MAVEN_OPTS in Jenkinsfile (#2195) 2020-08-05 15:22:09 +09:00
Akira Ajisaka 39c118c802
HADOOP-14056. Update maven-javadoc-plugin to 2.10.4. 
(cherry picked from commit fad766e1d2)
 2020-08-05 12:04:09 +09:00