Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,360 Commits 326 Branches 371 Tags
Commit Graph

15360 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jim Brennan
c1a3e8158d [HDFS-10498] Intermittent test failure TestSnapshotFileLength.testSnapshotFileLength. Contributed by Jim Brennan. 
(cherry picked from commit 6184ab07f670c181d1fda99e76729e920fff287e)
 2021-01-28 17:46:06 +00:00
Akira Ajisaka
130d5af258
HADOOP-17478. Improve the description of hadoop.http.authentication.signature.secret.file (#2628) 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit d82009599a2e9f48050e0c41440b36c759ec068f)
 2021-01-24 22:59:54 +09:00
Takanobu Asanuma
3dcb06a1ba
HADOOP-16841. The description of hadoop.http.authentication.signature.secret.file contains outdated information. Contributed by Xieming Li. 
(cherry picked from commit 5cbc4c54611f062690c3bfcf78d09d37c78ffd12)
 2021-01-24 22:59:46 +09:00
Anton Kutuzov
0ed997abab HDFS-15632. AbstractContractDeleteTest should set recursive peremeter to true for recursive test cases. Contributed by Anton Kutuzov. 
(cherry picked from commit 91d4ba57c5b85379303ac8fb2a1a03ba10b07d4e)
 2021-01-22 18:19:51 -08:00
Wei-Chiu Chuang
e48b91c83c HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 
(cherry picked from commit 162228e8db937d4bdb9cf61d15ed555f1c96368f)
 2021-01-21 17:48:14 +00:00
Yiqun Lin
1b046fb26f HADOOP-13965. Groups should be consistent in using default group mapping class. Contributed by Yiqun Lin. 
(cherry picked from commit 383aa9c9f55983cdb1c225b2ed22f6f48616cb9c)
 2021-01-21 17:33:54 +00:00
Eric Badger
ea3c3d9022 YARN-10562. Follow up changes for YARN-9833. Contributed by Jim Brennan. 
(cherry picked from commit 768e2f42ba3f82802e2ebf39d44fe6c30037583c)
 2021-01-15 19:53:02 +00:00
Eric Badger
b80ff6ea57 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b73c0f52a7d5af2495355f4a0799344)
 2021-01-15 19:18:58 +00:00
Akira Ajisaka
db5b804238
HDFS-15717. Improve fsck logging. Contributed by Kihwal Lee and Ahmed Hussein (#2620) 
(cherry picked from commit be35fa186cc0c5c1f90028a9383851d47791bf04)
 2021-01-15 10:28:11 +09:00
Ahmed Hussein
4ddd41bd47
HADOOP-17438. Increase docker memory limit in Jenkins (#2560) 
Reviewed-by: Eric Badger <ebadger@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae894ecfcc46e40d9e2f535ac6acec50f8136fae)
 2021-01-12 15:37:21 +09:00
He Xiaoqiao
1efadde067
HDFS-15751. Add documentation for msync() API to filesystem.md. Contributed by Konstantin V Shvachko. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
 2021-01-03 16:38:47 +08:00
He Xiaoqiao
45774daee8
HADOOP-17445. Update the year to 2021. Contributed by He Xiaoqiao. 2020-12-24 20:51:01 +08:00
David Mollitor
16fd00d62d HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350ba158c7872a39426e7a29c5e5e0bb48)
 2020-12-22 10:19:44 -08:00
Eric Badger
f239da08af YARN-10540. Node page is broken in YARN UI1 and UI2 including RMWebService api 
for nodes. Contributed by Jim Brennan.

(cherry picked from commit 4c5d88e230fa7adbb4a041c9a33bea2ef5a36fd3)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeResourceMonitorImpl.java
 2020-12-21 23:42:18 +00:00
He Xiaoqiao
2c45e93682 HADOOP-17068. Client fails forever when namenode ipaddr changed. Contributed by Sean Chow. 
(cherry picked from commit fa14e4bc001e28d9912e8d985d09bab75aedb87c)
(cherry picked from commit 3a860e876e799ef44fca34e3cb19a89dbd4f6744)
(cherry picked from commit 8d7600ae6b7fadba1b4695666cf9287be021d481)
(cherry picked from commit 1fe15113cf464015212e6ad7ecbca22dc1c50309)
 2020-12-15 14:08:20 -08:00
S O'Donnell
35c99be3da HDFS-15725. Lease Recovery never completes for a committed block which the DNs never finalize. Contributed by Stephen O'Donnell 2020-12-14 12:37:26 +00:00
Zhankun Tang
406957f5e8 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 
(cherry picked from commit fff4fbc9576d393a57489f3cd40770ec882f25dc)
 2020-12-10 18:05:30 +00:00
Jim Brennan
8af247df85 HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy (#2528) Contributed by Ahmed Hussein 2020-12-08 20:32:31 +00:00
Yiqun Lin
8ddd85b0e6 HDFS-15660. StorageTypeProto is not compatiable between 3.x and 2.6. Contributed by Ryan Wu. 
(cherry picked from commit da1ea2530fa61c53a99770e10889023c474fb4ef)
 2020-12-07 18:56:15 +08:00
kevinzhao1661
2fe36b0038
YARN-10498. Fix typo in CapacityScheduler Markdown document (#2484) 
Reviewed-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4d2ae5b398949acfd0b6406abdaeb17456a25c37)
 2020-11-30 11:19:24 +09:00
Jim Brennan
6a58e6293f YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2020-11-17 17:36:22 +00:00
Ahmed Hussein
c46643549f HADOOP-17358. Improve excessive reloading of Configurations (#2436) 
Co-authored-by: ahussein <ahmed.hussein@verizonmedia.com>
(cherry picked from commit 71071e5c0fcaf73a3989000dbc60fa214a317da1)
(cherry picked from commit 23fe3bdab3410b8ed370fa6af9f2ed63817006ab)
(cherry picked from commit ac3e10af63c4a70e45b26acd50b01a93fa8fd605)
(cherry picked from commit 487eaa6dea14bba100583f30f196c0d51215bd7c)
 2020-11-12 10:55:35 -08:00
Akira Ajisaka
266c9ad66a
HADOOP-17352. Update PATCH_NAMING_RULE in the personality file. (#2433) 
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 34e2abd811b271708c82bf4d5bedfc9d721366bf)
 2020-11-10 14:02:18 +09:00
Steve Loughran
894614ee4e HADOOP-17340. TestLdapGroupsMapping failing -string mismatch in exception validation. (#2427). Contributed by Steve Loughran. 2020-11-07 17:06:17 +05:30
hchaverr
9160b0e8c8 HDFS-15623. Respect configured values of rpc.engine (#2403) Contributed by Hector Chaverri. 
(cherry picked from commit 6eacaffeea21e7e9286497ee17f89fd939d2eead)
 2020-11-06 15:07:34 -08:00
Eric Badger
7a830d2633 HADOOP-17342. Creating a token identifier should not do kerberos name 
resolution. Contributed by Jim Brennan.

(cherry picked from commit af389d989770c4218c24e640f572bc7492f7d4b3)
 2020-11-05 22:17:32 +00:00
Wei-Chiu Chuang
fc4c98dc49 Revert "HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291)" 
This reverts commit 52f011905182dae2ddfbc68bb06341e2565424b9.
 2020-11-04 16:19:13 -08:00
Konstantin V Shvachko
a9bf374380 HDFS-15665. Balancer logging improvements. Contributed by Konstantin V Shvachko. 
(cherry picked from commit d07dc7afb4aa0d6cc9f9be530802e54610776a4d)
 2020-11-03 14:39:38 -08:00
Chen Liang
ac82334041 HADOOP-17336. Backport HADOOP-16005 NativeAzureFileSystem does not support setXAttr and HADOOP-16785. Improve wasb and abfs resilience on double close() calls. followup to abfs close() fix to branch-2.10. Contributed by Sally Zuo. 2020-11-03 12:00:57 -08:00
Akira Ajisaka
52f0119051 HADOOP-17255. JavaKeyStoreProvider fails to create a new key if the keystore is HDFS. (#2291) 
Reviewed-by: Steve Loughran <stevel@cloudera.com>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7f5caca04cb8e5dab8410ef64daa0fe799e389fd)
(cherry picked from commit dd1634ec3b676d6c3da18af7da46734584ddd9f6)
(cherry picked from commit 1c7043b1ff89ba9c995cbf546a99bbbcccbe56b1)
(cherry picked from commit 166bee93af6fb68ee2b3fab4c355413cae9fc1d6)
 2020-11-03 11:24:08 -08:00
Jonathan Hung
6f436a6776 YARN-10467. ContainerIdPBImpl objects can be leaked in RMNodeImpl.completedContainers. Contributed by Haibo Chen 2020-10-28 10:45:34 -07:00
Wei-Chiu Chuang
e7e165a927 HDFS-15644. Failed volumes can cause DNs to stop block reporting. Contributed by Ahmed Hussein. 2020-10-28 09:22:47 -07:00
Wei-Chiu Chuang
90ebbaa393 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
(cherry picked from commit 975d4b3d603632a5edacb138cf4a1ce92ebed02e)
 2020-10-24 13:38:58 -07:00
Akira Ajisaka
633e19cb49
HADOOP-17309. Javadoc warnings and errors are ignored in the precommit jobs. (#2391) 
(cherry picked from commit 3a0922fb4207af2f06f8ef57e83ff9759871d0cb)

 Conflicts:
	hadoop-project/pom.xml
 2020-10-19 11:04:28 +09:00
Eric Badger
c4b42fa1ae YARN-10450. Add cpu and memory utilization per node and cluster-wide metrics. 
Contributed by Jim Brennan.
 2020-10-16 19:29:04 +00:00
Konstantin V Shvachko
104dd85ad8 HDFS-15567. [SBN Read] HDFS should expose msync() API to allow downstream applications call it explicitly. Contributed by Konstantin V Shvachko. 
(cherry picked from commit b3786d6c3cc13b0b92b9f42da1731c4ce35c9ded)
 2020-10-12 18:02:05 -07:00
Swaroopa Kadam
893fceadee
MAPREDUCE-7301: Expose Mini MR Cluster attribute for testing 
Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-10-12 11:10:52 -07:00
Jim Brennan
4bb0124c69 YARN-9667. Container-executor.c duplicates messages to stdout. Contributed by Peter Bacsko 2020-10-12 15:41:33 +00:00
Eric Badger
3335fcfb05 YARN-10455. TestNMProxy.testNMProxyRPCRetry is not consistent. Contributed by 
Ahmed Hussein.
 2020-10-09 20:41:58 +00:00
Adam Antal
1c0fe2eb20 YARN-10393. MR job live lock caused by completed state container leak in heartbeat between node manager and RM. Contributed by zhenzhao wang and Jim Brennan 2020-10-07 16:46:22 +02:00
Jim Brennan
0bf270d2ed YARN-10451. RM (v1) UI NodesPage can NPE when yarn.io/gpu resource type is defined. Contributed by Eric Payne 
(cherry picked from commit ecf91638a8d1f546d90e92d076f32cb5681ce944)
 2020-10-06 18:46:08 +00:00
Wanqiang Ji
e8699d0bd5
MAPREDUCE-7289. Fix wrong comment in LongLong.java (#2338) 
(cherry picked from commit 143bdd41885a43c7bc6e11f0c0d6e5820768aa2c)
 2020-09-29 23:09:12 +09:00
zz
be421490da
MAPREDUCE-7294. Only application master should upload resource to Yarn Shared Cache. (#2319) 
Contributed by Zhenzhao Wang <zhenzhaowang@gmail.com>

Signed-off-by: Mingliang Liu <liuml07@apache.org>
 2020-09-22 11:57:36 -07:00
Masatake Iwasaki
7b0dbcb657 Publishing the bits for release 2.10.1 (addendum) 2020-09-22 11:51:53 +09:00
Masatake Iwasaki
0fd2ceb1b5 Publishing the bits for release 2.10.1 2020-09-22 11:48:18 +09:00
Wangda Tan
4885ea61fe Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
(cherry picked from commit ea18e70a74e811ffa48c7e18e68510dd37dda63d)
 2020-09-14 21:48:43 +09:00
Owen O'Malley
ca65409836 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu." 
This reverts commit a847903b6e64c6edb11d852b91f2c816b1253eb3.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
(cherry picked from commit fa6a8b78d481d3b4d355e1bf078f30dd5e09850d)

 Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/http/TestHttpServerWithSpengo.java
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestAuthenticationWithProxyUserFilter.java
 2020-09-14 21:48:17 +09:00
Masatake Iwasaki
f4e0c14fe9 Preparing for 2.10.2 development 2020-09-13 14:33:36 +09:00
Masatake Iwasaki
fdd8610ab2
HADOOP-17254. Upgrade hbase to 1.4.13 on branch-2.10. (#2290) 2020-09-11 08:54:30 +09:00
Wangda Tan
9a895a90d9 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
(cherry picked from commit b1833d9ba2c078582161da45ac392dd5c361dcdf)
(cherry picked from commit f9add5fdd9add4a5e6154de8ba6d9a17d123ac70)
 2020-09-10 14:40:05 -07:00