Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,470 Commits 326 Branches 371 Tags
Commit Graph

15470 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
litao
feb5971f49 HDFS-16312. Fix typo for DataNodeVolumeMetrics and ProfilingFileIoEvents (#3637) 
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit 2a1a11c039518bab7e5b581ef23bc5a2e8c81544)
 2021-11-10 19:24:56 +09:00
litao
899c4802fc HDFS-16298. Improve error msg for BlockMissingException (#3615) 
Reviewed-by: Hui Fei <ferhui@apache.org>
(cherry picked from commit ea65fc26d80db478f7e49749065c69da7e241bf0)
 2021-11-10 19:19:31 +09:00
jianghuazhu
64cd82e472 HDFS-16301. Improve BenchmarkThroughput#SIZE naming standardization. (#3622) 
(cherry picked from commit b307f18a584b207782124bee0a377b2dbfc13df0)
 2021-11-10 11:46:50 +09:00
Zhendong Bai
eb6be306b4
HADOOP-17880. Build Hadoop on Centos 7 (#3535) 2021-11-08 17:45:12 +05:30
jianghuazhu
f7ec214568 HDFS-16294.Remove invalid DataNode#CONFIG_PROPERTY_SIMULATED. (#3605) 
(cherry picked from commit 72718a8dc1d6d16249c1073c31f1a078d4e118f6)
 2021-11-04 16:57:35 +09:00
Gautham B A
3dd888023b
HADOOP-17978. Exclude ASF license check for pkg-resolver JSON (#3583) 2021-10-25 16:13:26 -07:00
Ahmed Hussein
de120b16ad YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne) 2021-10-22 17:02:38 +00:00
Michael Kuchenbecker
79d1818879 HDFS-7612: Fix default cache directory in TestOfflineEditsViewer.testStored. Contributed by Michael Kuchenbecker (#3571) 
(cherry picked from commit a73ff6915ae3e0ced1b4c814a94845f51e655a0c)
 2021-10-21 13:54:05 -07:00
Szilard Nemeth
5db1b2d7fa HADOOP-17857. Check real user ACLs in addition to proxied user ACLs. Contributed by Eric Payne 
(cherry picked from commit 5428d36b56fab319ab68258139d6133ded9bbafc)
 2021-10-21 17:40:02 +00:00
Masatake Iwasaki
b4c8004db3
HADOOP-17965. Fix documentation build failure using JDK 7 on branch-2.10. (#3562) 2021-10-19 18:08:51 +09:00
Symious
b7bb43f6db
HDFS-16257. Set initialCapacity for guava cache to solve performance issue (#3524) 
* HDFS-16257. Set initialCapacity for guava cache to solve performance issue

* HDFS-16257. Add comments

Co-authored-by: Symious <yiyang0203@gmail.com>
 2021-10-18 19:58:00 +09:00
Xing Lin
2565ec5804 HADOOP-16532. Fix TestViewFsTrash to use the correct homeDir. Contributed by Xing Lin. (#3514) 
(cherry picked from commit 97c0f968792e1a45a1569a3184af7b114fc8c022)
 2021-10-13 14:59:38 -07:00
Weiwei Yang
5f2047d491 YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 
(cherry picked from commit 251f528814c4a4647cac0af6effb9a73135db180)
 2021-10-12 17:43:43 +00:00
Masatake Iwasaki
b8dc05efcd
HADOOP-17964. Increase Java heap size for running Maven in Dockerfile of branch-2.10. (#3545) 2021-10-13 00:53:03 +09:00
Weiwei Yang
bdd396b26d YARN-8546. Resource leak caused by a reserved container being released more than once under async scheduling. Contributed by Tao Yang. 
(cherry picked from commit 5be9f4a5d05c9cb99348719fe35626b1de3055db)
 2021-10-08 16:08:45 +00:00
Weiwei Yang
dc03afc7df YARN-8127. Resource leak when async scheduling is enabled. Contributed by Tao Yang. 
(cherry picked from commit 7eb783e2634d8c11fb646f1f2fdf597336325312)
 2021-10-04 20:16:40 +00:00
Wei-Chiu Chuang
6d11f7ae8c HDFS-16233. Do not use exception handler to implement copy-on-write for EnumCounters. (#3468) 
(cherry picked from commit 87632bbacf064e9b07564ba66ed1fbb1d42290d1)
(cherry picked from commit bb08de559a36fd56c7d4023c6dcac66da3fbacc5)
(cherry picked from commit 978ec57445f4178b31a6bfc383c6be032692c288)
(cherry picked from commit cd7c34f9b4005d27886f73e58bef88e706fcccf9)
 2021-09-24 09:15:48 -07:00
Eric Badger
008bd8afc3 YARN-10935. AM Total Queue Limit goes below per-user AM Limit if parent is full. Contributed by Eric Payne. 2021-09-23 17:12:45 +00:00
Rintaro Ikeda
25d0a97557 HADOOP-17926. Maven-eclipse-plugin is no longer needed since Eclipse can import Maven projects by itself. (#3465) 
(cherry picked from commit 962068d2d8143b97db1ed5e1f5e6c998d0960ded)
 2021-09-22 13:10:07 +00:00
Kihwal Lee
30f48c4d06 HDFS-16198. Short circuit read leaks Slot objects when InvalidToken exception is thrown (#3359) 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c4c5883d8bf1fdc330e1da4d93eba760fa70c0e8)
 2021-09-15 14:45:28 -05:00
wangzhaohui
79c1a93bd7 HDFS-16181. [SBN Read] Fix display of JournalNode metric RpcRequestCacheMissAmount (#3317) 
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>

(cherry picked from commit 232fd7cae170de8c6b52c14841a47dca8735c6d2)
(cherry picked from commit 2f73ac1c14dde86a5c0b86f9272cf98084478994)
(cherry picked from commit 928dbac0a627c5198b3d5884ef5a7dee3d975855)
(cherry picked from commit 9b63f601d3285c3885b2941c5874f225bfa9ab31)
 2021-09-15 10:10:33 -07:00
Surendra Singh Lilhore
edc138186f HDFS-14216. NullPointerException happens in NamenodeWebHdfs. Contributed by lujie. 
(cherry picked from commit 92b53c40f070bbfe65c736f6f3eca721b9d227f5)
(cherry picked from commit 2e939515dfbaf26ca466c8a755cedde0ce4e9c1a)
 2021-09-10 15:35:49 -07:00
Ahmed Hussein
7f32a31abf HDFS-16207. Remove NN logs stack trace for non-existent xattr query (#3375) 
Change-Id: Ibde523b20a6b8ac92991da52583e625a018d2ee6
 2021-09-09 21:34:04 +00:00
Masatake Iwasaki
ba86e01336 HADOOP-17897. Allow nested blocks in switch case in checkstyle settings. (#3394) 
(cherry picked from commit e183ec8998d0272884b73df7eb1a6da5adf1040a)
 2021-09-08 15:14:24 +09:00
Ahmed Hussein
42599f6c8f
HADOOP-17885. Upgrade JSON smart to 1.3.3 on branch-2.10 (#3370) 2021-09-02 16:40:06 -05:00
Ahmed Hussein
d636420e7c HADOOP-17886. Upgrade ant to 1.10.11 (#3371) 
(cherry picked from commit 051207375b30ffa126cf9c961ee97535c1dfabb1)
 2021-09-02 16:22:23 -05:00
Pranav Bheda
bc8069ac48
HADOOP-17223 update org.apache.httpcomponents:httpclient to 4.5.13 and httpcore to 4.4.13 (#2242) 
* update org.apache.httpcomponents:httpclient from 4.5.6 to 4.5.13
* update org.apache.httpcomponents:httpcore from 4.4.10 to 4.4.13

(cherry picked from commit be3edd05321a0cbe5e6d60536b4ab6ed34f2cef6)

This closes #3363
 2021-09-01 18:29:53 +09:00
Akira Ajisaka
126b314ae8
HADOOP-15584. Move httpcomponents versions in pom.xml. Contributed by Brandon Scheller. 
(cherry picked from commit 76d0365190af472f922a62aa24dd7288a743210c)
 2021-09-01 18:28:03 +09:00
Szilard Nemeth
b196130c29
YARN-10428. Zombie applications in the YARN queue using FAIR + sizebasedweight. Contributed by Guang Yang, Andras Gyori 
(cherry picked from commit 79a46599f76e470527ad94b0894dacb28db01465)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/policy/TestFairOrderingPolicy.java

(cherry picked from commit 7aea2e1b5c24cd6e2dffbe6942f0dadb6a45c48f)
 2021-09-01 13:16:30 +09:00
Abhishek Modi
b79cd11fcd YARN-9744. RollingLevelDBTimelineStore.getEntityByTime fails with NPE. Contributed by Prabhu Joseph. 
(cherry picked from commit b4097b96a39bad6214b01989e7f2fb37dad70793)
 2021-08-18 17:54:10 -05:00
Akira Ajisaka
bd7e20527e
HADOOP-17370. Upgrade commons-compress to 1.21 2021-08-08 11:37:11 +09:00
Akira Ajisaka
d07955ce50
Fix potential heap buffer overflow in hdfs.c. Contributed by Igor Chervatyuk. 
(cherry picked from commit 4972e7a246f4aab665fd04ce72d1848bc5da9d4e)
 2021-08-07 11:46:30 +09:00
Chris Douglas
f856092bdb MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 
(cherry picked from commit 782ba3bf9da52699b27405a3f147464975d1df99)
 2021-08-03 21:00:44 +00:00
hchaverr
2187c6d366 HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto. Contributed by Hector Sandoval Chaverri. (#3242) 
(cherry picked from commit 3c8a48e681babe4038e49205624e7be183b3466e)
 2021-07-28 16:04:46 -07:00
Viraj Jasani
5e65126e38 HADOOP-17813. Checkstyle - Allow line length: 100 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 98412ce2e1453308655d4f43e7f8c922960e6955)
 2021-07-22 15:33:09 -05:00
zhuqi-lucas
34acf9d4c8 YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger. 2021-07-21 15:35:45 +08:00
Abhishek Das
99b952fcb4 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260, #3218) 
(cherry picked from commit 1dd03cc4b573270dc960117c3b6c74bb78215caa)
 2021-07-20 18:26:24 -07:00
Jim Brennan
39c8297350 HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein. 2021-07-15 21:13:25 +00:00
Jim Brennan
577ed175f9 YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne. 2021-07-15 15:21:02 +00:00
Akira Ajisaka
992d2c5f4e
HADOOP-12665. Document hadoop.security.token.service.use_ip. (#3187) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit c81f82e21d137b6d45c20c481e7bbc6160d72f94)
 2021-07-12 10:19:06 +09:00
Artem Smotrakov
9241470dfd HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada573291266c4d6447862072fdf88af5)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 15:06:37 +09:00
Eric Payne
cef04b40b6 MAPREDUCE-7353: Mapreduce job fails when NM is stopped. Contributed by Bilwa S T (BilwaST) 
(cherry picked from commit 7581413156da396db218e36a966c5749589b31a7)
 2021-07-07 21:40:09 +00:00
Masatake Iwasaki
6df4bc1a8c
HADOOP-17775. Remove JavaScript package from Docker environment. (#3184) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-07 17:49:59 +09:00
Jim Brennan
f7bcc58e0f YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne. 2021-06-29 14:22:39 +00:00
Jim Brennan
51be95e234 YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T. 
(cherry picked from commit 7c7d02edbd6c17ee8ae2c4bf75e87adace059b76)
 2021-06-25 20:58:50 +00:00
Jim Brennan
7d231f4b98 HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false (#2443). Contributed by Ahmed Hussein. 2021-06-25 18:06:37 +00:00
Ahmed Hussein
b58be3a12d
HADOOP-17769. Upgrade JUnit to 4.13.2. fixes TestBlockRecovery (#3131). Contributed by Ahmed Hussein. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
Signed-off-by: Hui Fei <ferhui@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
 2021-06-23 20:05:31 +05:30
S O'Donnell
caa59d7a58 HDFS-15150. Introduce read write lock to Datanode. Contributed by Stephen O'Donnell and Ahmed Hussein 2021-06-22 16:24:19 +01:00
Kihwal Lee
ab6b5681e8 HDFS-15618. Improve datanode shutdown latency. Contributed by Ahmed Hussein. 2021-06-16 11:38:30 -05:00
Kihwal Lee
e7c7fb2896 HDFS-15963. Unreleased volume references cause an infinite loop. (#2941) Contributed by Shuyan Zhang. 
Reviewed-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 9f2db2c9fdcf1e61b39651a51f8223cfb8ce2e31)
 2021-06-15 19:57:21 -05:00