Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,350 Commits 326 Branches 371 Tags
Commit Graph

15350 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
He Xiaoqiao
1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
He Xiaoqiao
45774daee8
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:51:01 +08:00
David Mollitor
16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350ba158c7872a39426e7a29c5e5e0bb48)
 2020-12-22 10:19:44 -08:00
Eric Badger
f239da08af YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api 
for nodes. Contributed by Jim Brennan.

(cherry picked from commit 4c5d88e230fa7adbb4a041c9a33bea2ef5a36fd3)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeResourceMonitorImpl.java
 2020-12-21 23:42:18 +00:00
He Xiaoqiao
2c45e93682 HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc001e28d9912e8d985d09bab75aedb87c)
(cherry picked from commit 3a860e876e799ef44fca34e3cb19a89dbd4f6744)
(cherry picked from commit 8d7600ae6b7fadba1b4695666cf9287be021d481)
(cherry picked from commit 1fe15113cf464015212e6ad7ecbca22dc1c50309)
 2020-12-15 14:08:20 -08:00
S O'Donnell
35c99be3da HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 2020-12-14 12:37:26 +00:00
Zhankun Tang
406957f5e8 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 
(cherry picked from commit fff4fbc9576d393a57489f3cd40770ec882f25dc)
 2020-12-10 18:05:30 +00:00
Jim Brennan
8af247df85 HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) Contributed by Ahmed Hussein 2020-12-08 20:32:31 +00:00
Yiqun Lin
8ddd85b0e6 HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu. 
(cherry picked from commit da1ea2530fa61c53a99770e10889023c474fb4ef)
 2020-12-07 18:56:15 +08:00
kevinzhao1661
2fe36b0038
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398949acfd0b6406abdaeb17456a25c37)
 2020-11-30 11:19:24 +09:00
Jim Brennan
6a58e6293f YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2020-11-17 17:36:22 +00:00
Ahmed Hussein
c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0fcaf73a3989000dbc60fa214a317da1)
(cherry picked from commit 23fe3bdab3410b8ed370fa6af9f2ed63817006ab)
(cherry picked from commit ac3e10af63c4a70e45b26acd50b01a93fa8fd605)
(cherry picked from commit 487eaa6dea14bba100583f30f196c0d51215bd7c)
 2020-11-12 10:55:35 -08:00
Akira Ajisaka
266c9ad66a
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811b271708c82bf4d5bedfc9d721366bf)
 2020-11-10 14:02:18 +09:00
Steve Loughran
894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr
9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea21e7e9286497ee17f89fd939d2eead)
 2020-11-06 15:07:34 -08:00
Eric Badger
7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d989770c4218c24e640f572bc7492f7d4b3)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang
fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f011905182dae2ddfbc68bb06341e2565424b9.
 2020-11-04 16:19:13 -08:00
Konstantin V Shvachko
a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4aa0d6cc9f9be530802e54610776a4d)
 2020-11-03 14:39:38 -08:00
Chen Liang
ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka
52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04cb8e5dab8410ef64daa0fe799e389fd)
(cherry picked from commit dd1634ec3b676d6c3da18af7da46734584ddd9f6)
(cherry picked from commit 1c7043b1ff89ba9c995cbf546a99bbbcccbe56b1)
(cherry picked from commit 166bee93af6fb68ee2b3fab4c355413cae9fc1d6)
 2020-11-03 11:24:08 -08:00
Jonathan Hung
6f436a6776 YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen 2020-10-28 10:45:34 -07:00
Wei-Chiu Chuang
e7e165a927 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-28 09:22:47 -07:00
Wei-Chiu Chuang
90ebbaa393 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
(cherry picked from commit 975d4b3d603632a5edacb138cf4a1ce92ebed02e)
 2020-10-24 13:38:58 -07:00
Akira Ajisaka
633e19cb49
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb4207af2f06f8ef57e83ff9759871d0cb)

 Conflicts:
	hadoop-project/pom.xml
 2020-10-19 11:04:28 +09:00
Eric Badger
c4b42fa1ae YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. 
Contributed by Jim Brennan.
 2020-10-16 19:29:04 +00:00
Konstantin V Shvachko
104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3cc13b0b92b9f42da1731c4ce35c9ded)
 2020-10-12 18:02:05 -07:00
Swaroopa Kadam
893fceadee
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:10:52 -07:00
Jim Brennan
4bb0124c69 YARN-9667. Container-executor.c duplicates messages to stdout. Contributed by Peter Bacsko 2020-10-12 15:41:33 +00:00
Eric Badger
3335fcfb05 YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by 
Ahmed Hussein.
 2020-10-09 20:41:58 +00:00
Adam Antal
1c0fe2eb20 YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan 2020-10-07 16:46:22 +02:00
Jim Brennan
0bf270d2ed YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne 
(cherry picked from commit ecf91638a8d1f546d90e92d076f32cb5681ce944)
 2020-10-06 18:46:08 +00:00
Wanqiang Ji
e8699d0bd5
MAPREDUCE-7289. Fix wrong comment in LongLong.java (#2338) 
(cherry picked from commit 143bdd41885a43c7bc6e11f0c0d6e5820768aa2c)
 2020-09-29 23:09:12 +09:00
zz
be421490da
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache. (#2319) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-22 11:57:36 -07:00
Masatake Iwasaki
7b0dbcb657 Publishing the bits for release 2.10.1 (addendum) 2020-09-22 11:51:53 +09:00
Masatake Iwasaki
0fd2ceb1b5 Publishing the bits for release 2.10.1 2020-09-22 11:48:18 +09:00
Wangda Tan
4885ea61fe Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
(cherry picked from commit ea18e70a74e811ffa48c7e18e68510dd37dda63d)
 2020-09-14 21:48:43 +09:00
Owen O'Malley
ca65409836 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e64c6edb11d852b91f2c816b1253eb3.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
(cherry picked from commit fa6a8b78d481d3b4d355e1bf078f30dd5e09850d)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationWithProxyUserFilter.java
 2020-09-14 21:48:17 +09:00
Masatake Iwasaki
f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Masatake Iwasaki
fdd8610ab2
HADOOP-17254. Upgrade hbase to 1.4.13 on branch-2.10. (#2290) 2020-09-11 08:54:30 +09:00
Wangda Tan
9a895a90d9 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
(cherry picked from commit b1833d9ba2c078582161da45ac392dd5c361dcdf)
(cherry picked from commit f9add5fdd9add4a5e6154de8ba6d9a17d123ac70)
 2020-09-10 14:40:05 -07:00
Masatake Iwasaki
7d8ee990cd
HADOOP-17253. Upgrade zookeeper to 3.4.14 on branch-2.10. (#2289) 2020-09-10 20:37:32 +09:00
Masatake Iwasaki
42ac46ff18
HADOOP-17249. Upgrade jackson-databind to 2.9.10.6 on branch-2.10. (#2279) 2020-09-09 09:55:11 +09:00
Masatake Iwasaki
0574de5f2f
HADOOP-17251. Upgrade netty-all to 4.1.50.Final on branch-2.10. (#2285) 2020-09-09 07:19:23 +09:00
Jitendra Pandey
43c6c54b8c HADOOP-14910. Upgrade netty-all jar to latest 4.0.x.Final. Contributed by Vinayakumar B. 
(cherry picked from commit f29e55bf5e634b3be28c9e0e993f78877528b5c2)
 2020-09-08 22:35:01 +09:00
dependabot[bot]
7c838bf6ed Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a1567cac1bcf71e2e5f252cddffa0f97cc)
 2020-09-08 21:35:40 +09:00
Eric E Payne
e5bd8d2840 YARN-10177: Backport YARN-7307 to branch-2.10 Allow client/AM update supported resource types via YARN APIs 2020-09-04 18:23:08 +00:00
Steve Loughran
a75f2915f9 HADOOP-17199. S3A Directory Marker HADOOP-13230 backport 
This backports the listing-side changes of HADOOP-13230.

With this patch in, this branch of Hadoop is compatible with S3A clients
which do not delete directory markers when files are created underneath.

It does not allow this version to disable marker deletion; if the
fs.s3a.marker.retention option is changed to request this, a message
is printed at INFO and the policy remains at "delete"

The s3guard bucket-info command has been extended to support
probing for marker retention, as has the hasPathCapability method on
S3AFileSystem.

Read the documentation!

Change-Id: Ief23c39256b194abcc445231b5f7a018185ea9f6
 2020-09-03 18:29:30 +01:00
Masatake Iwasaki
ecf88b972f
YARN-10358. Fix findbugs warnings in hadoop-yarn-project on branch-2.10. (#2164) 2020-08-31 13:52:29 +09:00
Chen Liang
2b6b5476ba HDFS-15290. NPE in HttpServer during NameNode startup. Contributed by Simbarashe Dzinamarira. 
(cherry picked from commit f734455e5d76c75b7d3a0b751023f5bd02ba38d2)
 2020-08-28 17:48:38 -07:00
sguggilam
e645204733
HADOOP-17159. Make UGI support forceful relogin from keytab ignoring the last login time (#2245) 
Contributed by Sandeep Guggilam.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Steve Loughran <stevel@apache.org>
 2020-08-27 15:21:20 -07:00