Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,664 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21664 Commits

Author SHA1 Message Date
Renukaprasad C fd1d138a62 HADOOP-17844. Upgrade JSON smart to 2.4.7 (#3299) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b90389ae98)

 Conflicts:
	LICENSE-binary

(cherry picked from commit d18c0a1d25)
 2021-08-14 20:28:26 +09:00
Brahma Reddy Battula 182167a9fb Preparing for 3.2.3 release. 2021-08-08 19:17:14 +05:30
Akira Ajisaka 840c6ef7fa
HADOOP-17370. Upgrade commons-compress to 1.21 (#3274) 
(cherry picked from commit 3565c9477d)

 Conflicts:
	LICENSE-binary
 2021-08-08 11:26:03 +09:00
Bryan Beaudreault dab829063d
HADOOP-17840: Backport HADOOP-17837 to branch-3.2 (#3275) 
Reviewed-by: Brahma Reddy Battula <brahma@apache.org>
 2021-08-07 09:49:46 +05:30
Akira Ajisaka 78c5fdd90d
Fix potential heap buffer overflow in hdfs.c. Contributed by Igor Chervatyuk. 
(cherry picked from commit 4972e7a246)
 2021-08-07 11:28:22 +09:00
wangzhaohui b98a8d259b
HDFS-16154. TestMiniJournalCluster failing intermittently because of not reseting UserGroupInformation completely (#3270) 
Co-authored-by: wangzhaohui8 <wangzhaohui8@jd.com>
Reviewed-by: Viraj Jasani <vjasani@apache.org>
(cherry picked from commit e85c44657c)
(cherry picked from commit 7068e18842)
 2021-08-06 18:25:28 +08:00
wangzhaohui 6b10154123 HDFS-16153. Avoid evaluation of LOG.debug statement in QuorumJournalManager (#3269). Contributed by wangzhaohui. 
(cherry picked from commit a73b64f86b)
 2021-08-06 09:52:51 +01:00
Viraj Jasani 2f5e527910 HADOOP-17808. Avoid excessive logging for interruption (ADDENDUM) (#3267) 
(cherry picked from commit 9fe1f24ec1)
 2021-08-06 09:54:34 +08:00
Akira Ajisaka c40abe10c0
YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3265) 
* YARN-10809. Missing dependency causing NoClassDefFoundError in TestHBaseTimelineStorageUtils (#3081)

Reviewed-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit ca27744770)

* Use mockito 1.x

Co-authored-by: Viraj Jasani <vjasani@apache.org>
 2021-08-05 10:05:10 +09:00
Abhishek Modi c5e2ec465b
YARN-9990. Testcase fails with Insufficient configured threads: required=16 < max=10. Contributed by Prabhu Joseph. 
(cherry picked from commit a2dadac790)
 2021-08-04 17:05:35 +09:00
Jim Brennan 1f9a315625
YARN-10337. Amendment to fix import as in HADOOP-17100 
(cherry picked from commit 53a1c7653f)
 2021-08-04 16:38:52 +09:00
Prabhu Joseph b9d0412fd5
YARN-10337. Fix failing testcase TestRMHATimelineCollectors. 
Contributed by Bilwa S T.

(cherry picked from commit 2bbd00dff4)
 2021-08-04 16:37:47 +09:00
Akira Ajisaka d830e84217
YARN-9875. Improve fair scheduler configuration store on HDFS. (#3262) 
Contributed by Prabhu Joseph

(cherry picked from commit 155864da00)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/conf/TestFSSchedulerConfigurationStore.java

Co-authored-by: Eric Yang <eyang@apache.org>
 2021-08-04 16:12:54 +09:00
Giovanni Matteo Fumarola 2f3c0b8dd1
HDFS-14189. Fix intermittent failure of TestNameNodeMetrics. Contributed by Ayush Saxena. 
(cherry picked from commit 1a08302897)
 2021-08-04 15:54:08 +09:00
Akira Ajisaka b838647b16
HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. (#3264) 
* HADOOP-16768. SnappyCompressor test cases wrongly assume that the compressed data is always smaller than the input data. (#2003)

(cherry picked from commit 328eae9a14)

* Add assertj-core dependency to hadoop-common for test
* Bump assertj-core version to 3.12.2 to use assertThat.hasSizeGreaterThan
 2021-08-04 15:46:00 +09:00
Akira Ajisaka e259d85a99
YARN-8992. Fair scheduler can delete a dynamic queue while an application attempt is being added to the queue. (Contributed by Wilfred Spiegelenburg) (#3257) 
(cherry picked from commit a41b648e98)

Co-authored-by: Haibo Chen <haibochen@apache.org>
 2021-08-04 10:17:51 +09:00
Siyao Meng fe59fe6744 HDFS-16055. Quota is not preserved in snapshot INode (#3078) 
(cherry picked from commit ebee2aed00)
(cherry picked from commit 72508e6430)
 2021-08-03 20:45:17 +01:00
Akira Ajisaka 786a43e729
YARN-8990. Fix fair scheduler race condition in app submit and queue cleanup. (Contributed by Wilfred Spiegelenburg) (#3254) 
(cherry picked from commit 524a7523c4)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java

Co-authored-by: Haibo Chen <haibochen@apache.org>
 2021-08-03 13:12:01 +09:00
Vrushali C 521c1085b4
YARN-9338 Timeline related testcases are failing. Contributed by Abhishek Modi. 
(cherry picked from commit 17a3e14d25)
 2021-08-03 11:42:28 +09:00
jenny 18ba3d6b1e
MAPREDUCE-7258. HistoryServerRest.html#Task_Counters_API, modify the jobTaskCounters's itemName from taskcounterGroup to taskCounterGroup (#1808) 
Co-authored-by: chenjuanni <chenjuanni@inspur.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e31169c864)
 2021-08-02 15:40:14 +09:00
Szilard Nemeth e4df745c13 YARN-6221. Entities missing from ATS when summary log file info got returned to the ATS before the domain log. Contributed by Xiaomin Zhang 2021-07-31 14:54:51 +02:00
Szilard Nemeth 7a8b6265c6 YARN-10789. RM HA startup can fail due to race conditions in ZKConfigurationStore. Contributed by Tarun Parimi 2021-07-29 19:22:57 +02:00
He Xiaoqiao f46a9e229e
HDFS-15175. Multiple CloseOp shared block instance causes the standby namenode to crash when rolling editlog. Contributed by Wan Chang. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Reviewed-by: Stephen O'Donnell <sodonnel@apache.org>
(cherry picked from commit 683feaa1d4)
 2021-07-29 11:26:09 +08:00
hchaverr 4f602cb17b HADOOP-17819. Add extensions to ProtobufRpcEngine RequestHeaderProto. Contributed by Hector Sandoval Chaverri. (#3242) 
(cherry picked from commit 3c8a48e681)
 2021-07-28 15:59:45 -07:00
9uapaw 1dc343726c YARN-9551. TestTimelineClientV2Impl.testSyncCall fails intermittent (#3212) 
Co-authored-by: Andras Gyori <gandras@cloudera.com>
(cherry picked from commit b4a524722a)
(cherry picked from commit f001595b47)
 2021-07-28 17:14:33 +02:00
Szilard Nemeth 6bf02a63e7 YARN-10813. Set default capacity of root for node labels. Contributed by Andras Gyori 2021-07-28 14:54:36 +02:00
Akira Ajisaka 7fe4f3cb89 HDFS-12920. HDFS default value change (with adding time unit) breaks old version MR tarball work with new version (3.0) of hadoop. (#3227) 
Revert "HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin"

This reverts commit b6d839a60c.

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 2f2f822488)
 2021-07-26 01:54:57 +09:00
Viraj Jasani 1df581a20c HADOOP-17808. ipc.Client to set interrupt flag after catching InterruptedException (#3219) 
(cherry picked from commit 3a52bfc5db)
 2021-07-23 12:57:36 +08:00
Viraj Jasani 1b1b3ceddc HADOOP-17813. Checkstyle - Allow line length: 100 
Signed-off-by: Sean Busbey <busbey@apache.org>
(cherry picked from commit 98412ce2e1)
 2021-07-22 15:09:12 -05:00
zhuqi-lucas cfe36ba762 YARN-10860. Make max container per heartbeat configs refreshable. Contributed by Eric Badger. 2021-07-22 10:13:55 +08:00
Masatake Iwasaki f3dac4ac23
YARN-10858. [UI2] YARN-10826 breaks Queue view. (#3213) 
(cherry picked from commit 17bf2fcbc5)
 2021-07-20 14:16:35 +09:00
He Xiaoqiao 913dbf2537
HDFS-15942. Increase Quota initialization threads. Contributed by Stephen O'Donnell. 
(cherry picked from commit 459df41997)
 2021-07-19 19:13:55 +08:00
Kihwal Lee 70c5e9e780 HDFS-16127. Improper pipeline close recovery causes a permanent write failure or data loss. Contributed by Kihwal Lee. 
(cherry picked from commit 47002719f2)
 2021-07-16 14:24:52 -05:00
crossfire 88f3ce1127
HDFS-15650. Make the socket timeout for computing checksum of striped blocks configurable (#2414) 
(cherry picked from commit 4bb25c810b)
(cherry picked from commit bd0b8a0bcf96fe66a93f59b74822f419cac7eb09)
 2021-07-16 11:01:03 +08:00
Jim Brennan 4fdeac1228 HDFS-16042. DatanodeAdminMonitor scan should be delay based. Contributed by Ahmed Hussein. 2021-07-15 20:59:09 +00:00
Jim Brennan 4a70248cb6 YARN-10456. RM PartitionQueueMetrics records are named QueueMetrics in Simon metrics registry. Contributed by Eric Payne. 
(cherry picked from commit 632f64cadb)
 2021-07-15 14:49:22 +00:00
Abhishek Das f9832031c2 HADOOP-17028. ViewFS should initialize mounted target filesystems lazily. Contributed by Abhishek Das (#2260) 
(cherry picked from commit 1dd03cc4b5)
 2021-07-13 18:25:03 -07:00
Akira Ajisaka a9505cf5d1
HADOOP-12665. Document hadoop.security.token.service.use_ip. (#3187) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
Reviewed-by: Chris Nauroth <cnauroth@apache.org>
(cherry picked from commit c81f82e21d)
 2021-07-12 10:18:44 +09:00
Artem Smotrakov 6c7f192f0f HADOOP-17793. Better token validation (#3189) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba325a8ada)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2021-07-10 14:25:08 +09:00
Akira Ajisaka aae60521a3 HADOOP-17794. Add a sample configuration to use ZKDelegationTokenSecretManager in Hadoop KMS (#3190) 
(cherry picked from commit 6ac5d8a1ed)
 2021-07-09 22:52:20 +09:00
S O'Donnell e2eacbd089 HDFS-15796. ConcurrentModificationException error happens on NameNode occasionally. Contributed by Daniel Ma 
(cherry picked from commit b6c06c4b76)
 2021-07-09 10:34:14 +01:00
bshashikant 844b8865c0 HDFS-16121. Iterative snapshot diff report can generate duplicate records for creates, deletes and Renames. (#3188) 
(cherry picked from commit 6450c1ea9a)
 2021-07-09 09:35:36 +01:00
Eric Payne ab7d2f1f3a MAPREDUCE-7353: Mapreduce job fails when NM is stopped. Contributed by Bilwa S T (BilwaST) 
(cherry picked from commit 7581413156)
 2021-07-07 21:14:47 +00:00
Masatake Iwasaki 11929ec0e7
HADOOP-17775. Remove JavaScript package from Docker environment. (#3183) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-07-07 17:49:41 +09:00
liangxs b7108cfcee HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout (#3080) 
(cherry picked from commit a5db6831bc)
 2021-07-07 09:54:16 +08:00
Viraj Jasani 4912e4c903
HDFS-16108. Fix incorrect log placeholders used in JournalNodeSyncer (#3169) 
(cherry picked from commit da575a53cf)
 2021-07-05 13:43:10 +09:00
Viraj Jasani 707d003eb6
HDFS-16092. Avoid creating LayoutFlags redundant objects (#3150) 
(cherry picked from commit 93a1685073)
(cherry picked from commit 8538881710)
 2021-06-29 18:14:02 +08:00
Jim Brennan c3c86b18cb YARN-10834. Intra-queue preemption: apps that don't use defined custom resource won't be preempted. Contributed by Eric Payne. 
(cherry picked from commit dc6f456e95)
 2021-06-28 15:28:38 +00:00
Jim Brennan 9c4ac62d68 YARN-10824. Title not set for JHS and NM webpages. Contributed by Bilwa S T. 
(cherry picked from commit 7c7d02edbd)
 2021-06-25 20:50:11 +00:00
Jim Brennan 9731dfbf0d HDFS-15659. MiniDFSCluster dfs.namenode.redundancy.considerLoad default to false (#2443). Contributed by Ahmed Hussein. 2021-06-25 17:57:15 +00:00