Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,600 Commits 326 Branches 371 Tags
Commit Graph

19600 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Konstantin V Shvachko
8fc4e406ad HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko. 2019-09-30 17:24:15 -07:00
Konstantin V Shvachko
bece1a1f70 Revert "HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by He Xiaoqiao." 
This reverts commit 0feba4396f6e96c332743a39f965de7995b67bde.
 2019-09-30 17:24:15 -07:00
Zsombor Gegesy
db5b8358ca HADOOP-15014. Addendum: KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0131bc265453051820e54908e70d39433c227ab)
 2019-09-30 17:15:51 -07:00
Zsombor Gegesy
b59911d841 HADOOP-15014. KMS should log the IP address of the clients. Contributed by Zsombor Gegesy. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 008766c119d9ed9d568f9458ed0c02136962da5b)
 2019-09-30 17:15:43 -07:00
Gopal V
b9dc2c17de HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock. 
Contributed by Gopal V.

Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4
(cherry picked from commit aebac6d2d2e612e400a7d73be67dafb47e239211)
 2019-09-30 16:58:52 -07:00
Surendra Singh Lilhore
1030ac85bb HDFS-14235. Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon. Contributed by Ranith Sardar. 
(cherry picked from commit 41e18feda3f5ff924c87c4bed5b5cbbaecb19ae1)
(cherry picked from commit b93b127956508072904b44098fdc1c0dfc899606)
 2019-09-30 16:47:08 -07:00
Xiao Chen
5167a29adb HADOOP-15849. Upgrade netty version to 3.10.6. 
(cherry picked from commit 8853fc8a55b07ecdc5ce8d85278b822e5675d97a)
 2019-10-01 07:05:28 +09:00
Ayush Saxena
8cc702ad38 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-28 08:30:26 +05:30
Zhankun Tang
670d0657b4 Make upstream aware of 3.1.3 release. 2019-09-27 16:58:36 +08:00
Chen Liang
3d7730266a HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang. 2019-09-26 15:59:08 -07:00
Jonathan Hung
95ec7050b5 Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit d86a1acc866cbda845fb3896dc824baf12217383)
(cherry picked from commit f4f210d2e57e49999579efb7db039cc623798267)
 2019-09-25 17:49:52 -07:00
Jonathan Hung
783cbced1d YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)
 2019-09-24 13:52:14 -07:00
Ayush Saxena
5c058549b7 HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-09-25 01:22:11 +05:30
Jonathan Hung
6a1d2d56bd YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar 
(cherry picked from commit 3d78b1223d3fdc29d500803cefd2931b54f44928)
(cherry picked from commit a1fa9a8a7f79a1a711cd881b526724b502e03456)
 2019-09-23 13:13:09 -07:00
Yuval Degani
b091abc471 HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a694564af65a1355c9d13275f44df51)
(cherry picked from e31594f20ed43670a9ff1463a752d8b8e4b3b6f0)
 2019-09-20 09:58:57 -07:00
Takanobu Asanuma
6ef3204d54 HADOOP-16069. Support configure ZK_DTSM_ZK_KERBEROS_PRINCIPAL in ZKDelegationTokenSecretManager using principal with Schema /_HOST. Contributed by luhuachao. 
(cherry picked from commit 298f07abceb4bb854630c9058e1697d4ef3d2ae2)
 2019-09-20 13:58:19 +09:00
Kihwal Lee
56562b9117 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce176287e863f567b333e0d408bf51ae6d)
 2019-09-19 08:26:36 -05:00
Chen Liang
d25b3339ed HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang. 2019-09-18 17:20:05 -07:00
Kihwal Lee
2661c32c06 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 1851d06eb3b70f39f3054a7c06f0ad2bc664aaec)
 2019-09-18 16:45:29 -05:00
Wei-Chiu Chuang
6af1ecc8e4 HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679b7fc35474a3d33dc40b0db89bb1d80)
(cherry picked from 9c9920897de182c3da3ad33ff53e506242bbd362)
 2019-09-18 10:30:52 -07:00
Wei-Chiu Chuang
0fcbeed49a HDFS-14836. FileIoProvider should not increase FileIoErrors metric in datanode volume metric. Contributed by Aiphago. 
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
(cherry picked from commit 5dd859a8a0d038ff04acbb96741a215d6a0e8c29)
(cherry picked from commit b906f53b0d0e330d65a36295bfb8fe66d0a3995c)
 2019-09-18 09:36:55 -07:00
Anu Engineer
eb5c4ce432 HDFS-14759. HDFS cat logs an info message. 
Contributed by Eric Badger.

(cherry picked from commit 8aaf5e1a14e577a7d8142bc7d49bb94014032afd)
 2019-09-14 01:35:12 +09:00
Sunil G
45cf3de2e9 YARN-9833. Race condition when DirectoryCollection.checkDirs() runs during container launch. Contributed by Peter Bacsko. 
(cherry picked from commit c474e24c0b73c0f52a7d5af2495355f4a0799344)
 2019-09-18 09:23:46 +05:30
Weiwei Yang
1978317068 YARN-2255. YARN Audit logging not added to log4j.properties. Contributed by Aihua Xu. 
(cherry picked from commit f8c14326ee0026f2c57a4d061bff5fadec74b74d)
 2019-09-18 09:20:37 +08:00
Eric Yang
dae22c962d YARN-9837. Fixed reading YARN Service JSON spec file larger than 128k. 
Contributed by Tarun Parimi
 2019-09-17 13:17:13 -04:00
Ayush Saxena
cab81f7f5f HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-09-14 10:20:49 +05:30
Yi Sheng
7d7b415409 HADOOP-16555. Update commons-compress to 1.19. (#1425) Contributed by YiSheng Lien. 
(cherry picked from commit 1843c4688a3040c48f6834d166494cbf68b5e4fd)

Conflicts:
	LICENSE-binary
(cherry picked from commit d39ebbfebc9da982cadfbe92f3fb6677e744622d)
 2019-09-13 11:15:56 -07:00
Takanobu Asanuma
8d5460a045 HDFS-14838. RBF: Display RPC (instead of HTTP) Port Number in RBF web UI. Contributed by Xieming Li 
(cherry picked from commit c255333e20c9af6166db5931d70151011d540359)
 2019-09-11 17:11:13 +09:00
Jonathan Hung
d75693bd6e YARN-9824. Fall back to configured queue ordering policy class name 
(cherry picked from commit f8f8598ea5e86330fa5b553963f5632da286409d)
(cherry picked from commit 1dbf87c9ff33b5e2f6cbb0ce8efd348c4314a0e2)
 2019-09-10 15:31:58 -07:00
Jonathan Hung
80735a15a5 YARN-8541 (branch-3.1 addendum): RM startup failure on recovery after user deletion 2019-09-09 20:15:42 -07:00
bibinchundatt
aee8fb567b YARN-8948. PlacementRule interface should be for all YarnSchedulers. Contributed by Bibin A Chundatt. 
(cherry picked from commit a68d766e876631d7ee2e1a6504d4120ba628d178)
(cherry picked from commit e10050678d0bcd0e4384f2a684daf8c18e30ecb9)
 2019-09-09 20:00:46 -07:00
Wangda Tan
9e8ff94d16 YARN-8361. Change App Name Placement Rule to use App Name instead of App Id for configuration. (Zian Chen via wangda) 
Change-Id: I17e5021f8f611a9c5e3bd4b38f25e08585afc6b1
(cherry picked from commit a2e49f41a8bcc03ce0a85b294d0b86fee7e86f31)
 2019-09-09 20:00:33 -07:00
Wangda Tan
81d63d5ea1 YARN-8016. Refine PlacementRule interface and add a app-name queue mapping rule as an example. (Zian Chen via wangda) 
Change-Id: I35caf1480e0f76f5f3a53528af09312e39414bbb
(cherry picked from commit a90471b3e65326cc18ed31fe21aef654833b5883)
 2019-09-09 19:59:50 -07:00
Akira Ajisaka
244385d7eb
Revert "HADOOP-16061. Upgrade Yetus to 0.10.0" 
This reverts commit 565751e73901fb0e4017297480e8cc272d06f915.
 2019-09-09 18:58:38 +09:00
Jonathan Hung
0e88bcd8e6 YARN-9820. RM logs InvalidStateTransitionException when app is submitted. Contributed by Prabhu Joseph 2019-09-09 00:27:33 -07:00
Jonathan Hung
080fc6d943 YARN-9764. Print application submission context label in application summary. Contributed by Manoj Kumar 
(cherry picked from commit 43e389b9801e09741fdf78fef067b8ac60f691c8)
(cherry picked from commit 45220d115797663b8749980b78a61bafcb2344f1)
 2019-09-08 19:15:12 -07:00
Wangda Tan
0ee7d09138 YARN-9813. RM does not start on JDK11 when UIv2 is enabled. (Adam Antal/Eric Yang via wangda) 
Change-Id: I18b8edc930b2efa0652f59c246931ad0d46827f3
(cherry picked from commit 34b82e6da0a471010cdae613ba39487889d79369)
 2019-09-06 19:19:59 -07:00
Tao Yang
36c1caced5 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time(addendum). Contributed by Jonathan Hung. 2019-09-07 08:19:02 +08:00
Tao Yang
1f6f4a2457 YARN-9795. ClusterMetrics to include AM allocation delay. Contributed by Fengnan Li. 2019-09-07 07:54:30 +08:00
Tao Yang
3cc21350b5 YARN-9817. Fix failing testcases due to not initialized AsyncDispatcher - ArithmeticException: / by zero. Contributed by Prabhu Joseph. 2019-09-07 07:40:50 +08:00
Jonathan Hung
980a922481 YARN-9763. Print application tags in application summary. Contributed by Manoj Kumar 2019-09-06 10:52:37 -07:00
Jonathan Hung
11f6e3bc41 YARN-9761. Allow overriding application submissions based on server side configs. Contributed by Pralabh Kumar 2019-09-06 10:02:20 -07:00
Xiaoyu Yao
ed253c4412
HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun. 
(cherry picked from commit 62d71fbac3789c7d484bc76ced9ec7fa6ff94de1)
 2019-09-05 16:52:09 -07:00
Xiaoyu Yao
fbdfb36fdf
Revert "HDFS-14633. The StorageType quota and consume in QuotaFeature is not handled for rename. Contributed by Jinglun." 
This reverts commit 29e6a97faa81f6b143851b0a038706bccc223130.
 2019-09-05 16:52:09 -07:00
Billie Rinaldi
619bd1e876 YARN-9718. Fixed yarn.service.am.java.opts shell injection. Contributed by Eric Yang 
(cherry picked from commit 2e2e5401f297545181323b126a69eaa2239afb02)
 2019-09-05 14:19:05 -07:00
Jonathan Hung
37d1f8c81e YARN-9810. Add queue capacity/maxcapacity percentage metrics. Contributed by Shubham Gupta 
(cherry picked from commit 0ccf4b0fe16a8c879a560f2a612a3185eb2df72b)
(cherry picked from commit cb806988d72bde1f9837c9e0fb82a3a6c032542c)
 2019-09-05 14:06:09 -07:00
Tao Yang
d74c069427 YARN-8995. Log events info in AsyncDispatcher when event queue size cumulatively reaches a certain number every time. Contributed by zhuqi. 2019-09-05 16:54:55 +08:00
Ayush Saxena
8b702b2b4e HADOOP-16025. Update the year to 2019. 
Contributed by Ayush Saxena.

(cherry picked from commit cb26f154289ed065a967886b8eac04794907d643)
 2019-09-04 17:08:16 +08:00
Zhankun Tang
ef79d98788 Preparing for 3.1.4 development 2019-09-04 16:11:36 +08:00
Zhankun Tang
fff4fbc957 YARN-9785. Fix DominantResourceCalculator when one resource is zero. Contributed by Bibin A Chundatt, Sunil Govindan, Bilwa S T. 2019-09-04 12:05:29 +08:00