Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,898 Commits 326 Branches 371 Tags
Commit Graph

20898 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Inigo Goiri
917e53f066 HDFS-14460. DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured. Contributed by CR Hota. 
(cherry picked from commit 865c3289308327788f3bed355864c510deb40956)
 2019-10-01 09:53:36 -07:00
Surendra Singh Lilhore
a131611b4b HDFS-14175. EC: Native XOR decoder should reset the output buffer before using it. Contributed by Ayush Saxena. 
(cherry picked from commit dacc1a759e3ba3eca000cbacc6145b231253b174)
 2019-10-01 09:47:18 -07:00
Inigo Goiri
fc8a7a9e5b HDFS-14202. dfs.disk.balancer.max.disk.throughputInMBperSec property is not working as per set value. Contributed by Ranith Sardar. 
(cherry picked from commit 0e79a865822eed05f3f8433976b2cfef8f427f25)
 2019-10-01 09:35:04 -07:00
Inigo Goiri
06b2eceb76 HDFS-14418. Remove redundant super user priveledge checks from namenode. Contributed by Ayush Saxena. 
(cherry picked from commit be6c8014e66be919388269b70cb2966c35b8c578)
 2019-10-01 09:22:15 -07:00
Wei-Chiu Chuang
ab3fb28d47 HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad 
(cherry picked from commit cd2158456db8c89eeea64b72654a736ea8607e23)
 2019-10-01 09:02:48 -07:00
Kihwal Lee
ca18ae3bc1 HADOOP-15909. KeyProvider class should implement Closeable. Contributed 
by Kuhu Shukla.

(cherry picked from commit 33c009a45d95c9b582db25acd205d087e01e2085)
 2019-10-01 08:53:55 -07:00
Sunil G
312cfa994a YARN-9801. SchedConfCli does not work wiwith https mode. Contributed by Prabhu Joseph 
(cherry picked from commit 99cd7572f113cbbb8f5ef89af60af3734fe8cfa7)
 2019-10-01 20:07:18 +05:30
bibinchundatt
5cd6eb2a18 YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:04:58 +05:30
Sunil G
52f815d39d YARN-9864. Format CS Configuration present in Configuration Store. Contributeed by Prabhu Joseph 
(cherry picked from commit 137546a78a45adbfb5ef338c1c2984a97b3cecc2)
 2019-10-01 09:09:22 +05:30
Erik Krogen
f5fb7d8c18 HADOOP-16531. Log more timing information for slow RPCs. Contributed by Chen Zhang. 
(cherry picked from commit a23417533e1ee052893baf207ec636c4993c5994)
 2019-09-30 17:35:45 -07:00
Wei-Chiu Chuang
46e72775f5 HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko. 
(cherry picked from commit e872ceb810a343da7fce7185dca78d3b9aad9b7b)
 2019-09-30 17:25:02 -07:00
Zsombor Gegesy
f42c8a44ae HADOOP-15014. Addendum: KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0131bc265453051820e54908e70d39433c227ab)
 2019-09-30 17:16:10 -07:00
Zsombor Gegesy
85aff9facc HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 008766c119d9ed9d568f9458ed0c02136962da5b)
 2019-09-30 17:16:05 -07:00
Gopal V
79359e2091 HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock. 
Contributed by Gopal V.

Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4
(cherry picked from commit aebac6d2d2e612e400a7d73be67dafb47e239211)
 2019-09-30 17:05:29 -07:00
Konstantin V Shvachko
665360ec6f HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko. 2019-09-30 16:57:56 -07:00
Konstantin V Shvachko
3912a6fc7d Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." 
This reverts commit 0feba4396f6e96c332743a39f965de7995b67bde.
 2019-09-30 16:57:26 -07:00
Xiao Chen
bd5b59feea HADOOP-15849. Upgrade netty version to 3.10.6. 
(cherry picked from commit 8853fc8a55b07ecdc5ce8d85278b822e5675d97a)
 2019-10-01 06:54:20 +09:00
Ayush Saxena
b7ee3938e9 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-28 08:29:58 +05:30
Chen Liang
2d7b37ea2b HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang. 2019-09-26 15:55:10 -07:00
Jonathan Hung
f4f210d2e5 Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit d86a1acc866cbda845fb3896dc824baf12217383)
 2019-09-25 17:49:37 -07:00
Jonathan Hung
806c7b7dfb YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
 2019-09-24 13:51:54 -07:00
Ayush Saxena
a0db762206 HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-09-25 01:21:14 +05:30
Jonathan Hung
a1fa9a8a7f YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar 
(cherry picked from commit 3d78b1223d3fdc29d500803cefd2931b54f44928)
 2019-09-23 13:12:57 -07:00
Rohith Sharma K S
f9f0338104 Make upstream aware of 3.2.1 release. 2019-09-23 12:03:53 +05:30
Yuval Degani
e31594f20e HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a694564af65a1355c9d13275f44df51)
 2019-09-20 09:58:39 -07:00
Takanobu Asanuma
b2072444ab HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 
(cherry picked from commit 298f07abceb4bb854630c9058e1697d4ef3d2ae2)
 2019-09-20 13:57:40 +09:00
Kihwal Lee
7477f8d2e9 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce176287e863f567b333e0d408bf51ae6d)
 2019-09-19 08:24:39 -05:00
Chen Liang
4299bc0d3e HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang. 2019-09-18 16:48:58 -07:00
Kihwal Lee
e29f91bad3 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 1851d06eb3b70f39f3054a7c06f0ad2bc664aaec)
 2019-09-18 15:53:08 -05:00
Wei-Chiu Chuang
9c9920897d HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679b7fc35474a3d33dc40b0db89bb1d80)
 2019-09-18 10:30:25 -07:00
Wei-Chiu Chuang
b906f53b0d HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 5dd859a8a0d038ff04acbb96741a215d6a0e8c29)
 2019-09-18 09:36:10 -07:00
Anu Engineer
2e9482bd20 HDFS-14759. HDFS cat logs an info message. 
Contributed by Eric Badger.

(cherry picked from commit 8aaf5e1a14e577a7d8142bc7d49bb94014032afd)
 2019-09-18 15:23:28 +08:00
Sunil G
3e0025d877 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b73c0f52a7d5af2495355f4a0799344)
 2019-09-18 09:22:48 +05:30
Weiwei Yang
7ec229244a YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu. 
(cherry picked from commit f8c14326ee0026f2c57a4d061bff5fadec74b74d)
 2019-09-18 09:19:38 +08:00
Eric Yang
345ef049df YARN-9837. Fixed reading YARN Service JSON spec file larger than 128k. 
Contributed by Tarun Parimi

(cherry picked from commit eefe9bc85ccdabc2b7303969934dbce98f2b31b5)
 2019-09-17 13:22:53 -04:00
Ayush Saxena
4126e67e31 HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-09-14 10:12:17 +05:30
Yi Sheng
d39ebbfebc HADOOP-16555. Update commons-compress to 1.19. (#1425) Contributed by YiSheng Lien. 
(cherry picked from commit 1843c4688a3040c48f6834d166494cbf68b5e4fd)

Conflicts:
	LICENSE-binary
 2019-09-13 11:14:49 -07:00
Surendra Singh Lilhore
eb1ddcd04c HDFS-14699. Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold. Contributed by Zhao Yi Ming. 
(cherry picked from commit d1c303a49763029fffa5164295034af8e81e74a0)
 2019-09-12 19:25:50 +05:30
Takanobu Asanuma
8e27d6b4c9 HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li 
(cherry picked from commit c255333e20c9af6166db5931d70151011d540359)
 2019-09-11 17:03:23 +09:00
Jonathan Hung
1dbf87c9ff YARN-9824. Fall back to configured queue ordering policy class name 
(cherry picked from commit f8f8598ea5e86330fa5b553963f5632da286409d)
 2019-09-10 15:26:57 -07:00
bibinchundatt
e10050678d YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt. 
(cherry picked from commit a68d766e876631d7ee2e1a6504d4120ba628d178)
 2019-09-09 19:03:40 -07:00
Abhishek Modi
f6cc887f35 YARN-9821. NM hangs at serviceStop when ATSV2 Backend Hbase is Down. Contributed by Prabhu Joseph. 2019-09-09 15:44:45 +05:30
Jonathan Hung
1f0449ddfb YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph 2019-09-09 00:24:17 -07:00
Jonathan Hung
45220d1157 YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar 
(cherry picked from commit 43e389b9801e09741fdf78fef067b8ac60f691c8)
 2019-09-08 19:11:47 -07:00
Rohith Sharma K S
7d5bb2ebb7 Preparing for 3.2.2-SNAPSHOT development. 2019-09-07 08:52:08 +05:30
Wangda Tan
0e77347972 YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda) 
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
(cherry picked from commit 34b82e6da0a471010cdae613ba39487889d79369)
 2019-09-06 19:19:05 -07:00
Tao Yang
9ee257e353 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time(addendum). Contributed by Jonathan Hung. 2019-09-07 08:25:15 +08:00
Tao Yang
cfce39023d YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li. 2019-09-07 07:53:46 +08:00
Tao Yang
c002186ee1 YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph. 2019-09-07 07:40:04 +08:00
Jungtaek Lim (HeartSaVioR)
8cda5a6598
HADOOP-16255. Add ChecksumFs.rename(path, path, boolean) 
Contributed by Jungtaek Lim

Change-Id: If00a4d7d30456c08eb2b0f7e2b242197bc4ee05d
 2019-09-06 21:54:09 +01:00