Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,081 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

15081 Commits

Author SHA1 Message Date
Jonathan Hung fab62a145d Add 2.10.0 release notes for HDFS-12943 
(cherry picked from commit ef9d12df24)
(cherry picked from commit 9ccfc2d6d3)
(cherry picked from commit f29481f046)
 2019-10-31 20:35:49 -07:00
Jonathan Hung ae3f5d66e4 YARN-9945. Fix javadoc in FederationProxyProviderUtil in branch-2 2019-10-31 15:30:23 -07:00
Duo Zhang a36dbe6f35 HADOOP-16598. Backport "HADOOP-16558 [COMMON+HDFS] use protobuf-maven-plugin to generate protobuf classes" to all active branches 
Signed-off-by: stack <stack@apache.org>
 2019-10-30 22:33:51 -07:00
Jonathan Hung 915b795358 Make upstream aware of 2.10.0 release 
(cherry picked from commit 7663db59c097c82eeed2df7a91168a4d7123c96b)
(cherry picked from commit 5d2ffcc7aa)
(cherry picked from commit 4d274f60bc)
 2019-10-30 21:01:40 -07:00
Eric Badger 128631c5ac YARN-9914. Use separate configs for free disk space checking for full and not-full disks. Contributed by Jim Brennan 
(cherry picked from commit eef34f2d87)
 2019-10-25 17:45:40 +00:00
Wei-Chiu Chuang a805d80ebe HDFS-13901. INode access time is ignored because of race between open and rename. Contributed by Jinglun. 2019-10-22 16:43:25 -07:00
Jonathan Hung 339ab2347d HDFS-14667. Backport [HDFS-14403] "Cost-based FairCallQueue" to branch-2. Contributed by Erik Krogen 2019-10-22 11:46:25 -07:00
Kihwal Lee b634bd225b HDFS-12749. DN may not send block report to NN after NN restart. Contributed by Xiaoqiao He. 
(cherry picked from commit c4e27ef773)
 2019-10-21 16:36:42 -05:00
Ayush Saxena c78eb006f9 HADOOP-16662. Remove unnecessary InnerNode check in NetworkTopology#add(). Contributed by Lisheng Sun. 2019-10-20 11:58:30 +05:30
Da Zhou 7f71473888
HADOOP-16640. WASB: Override getCanonicalServiceName() to return URI 
(cherry picked from commit 9a8edb0aed)
 2019-10-16 22:04:36 -07:00
Bilahari T H 6cb9ee32e1
HADOOP-16652. Backport of HADOOP-16587: Make ABFS AAD endpoints configurable 2019-10-16 15:39:39 -07:00
Jonathan Hung f555aa740b HADOOP-16655. Change cipher suite when fetching tomcat tarball for branch-2 2019-10-15 17:40:29 -07:00
Jonathan Hung 108127238b Preparing for 2.11.0 development 2019-10-15 10:01:45 -07:00
Haibo Chen 600d54fcd6 YARN-8750. Refactor TestQueueMetrics. (Contributed by Szilard Nemeth) 
(cherry picked from commit e60b797c88)
 2019-10-15 15:52:25 +00:00
Chen Liang 6566402a1b HDFS-14509. DN throws InvalidToken due to inequality of password when upgrade NN 2.x to 3.x. Contributed by Yuxuan Wang and Konstantin Shvachko. 2019-10-10 13:29:30 -07:00
Erik Krogen 200c52f78b HDFS-14245. [SBN read] Enable ObserverReadProxyProvider to work with non-ClientProtocol proxy types. Contributed by Erik Krogen. 
(cherry picked from 5847e00143)
(cherry picked from 6630c9b75d)
(cherry picked from 9fdb849e03)
 2019-10-10 09:09:50 -07:00
Erik Krogen 8337d1a489 HDFS-14162. [SBN read] Allow Balancer to work with Observer node. Add a new ProxyCombiner allowing for multiple related protocols to be combined. Allow AlignmentContext to be passed in NameNodeProxyFactory. Contributed by Erik Krogen. 2019-10-10 09:07:18 -07:00
Sneha Vijayarajan 27d7d09fa4
HADOOP-16630 : Backport of Hadoop-16548 : Disable Flush() over config 2019-10-09 16:50:06 -07:00
Sneha Vijayarajan 2ccb1683ba
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists 2019-10-09 16:23:25 -07:00
Jonathan Hung ab7181e9af YARN-9760. Support configuring application priorities on a workflow level. Contributed by Varun Saxena 2019-10-08 11:19:39 -07:00
Ayush Saxena ca777db3ca HDFS-14655. [SBN Read] Namenode crashes if one of The JN is down. Contributed by Ayush Saxena. 2019-10-08 13:30:30 +05:30
Shweta Yakkali 56e2ad2dce HDFS-14262. [SBN read] Make Log.WARN message in GlobalStateIdContext more informative. Contributed by Shweta Yakkali. 2019-10-03 16:43:27 -07:00
Ayush Saxena a3b9b955be HDFS-14660. [SBN Read] ObserverNameNode should throw StandbyException for requests not from ObserverProxyProvider. Contributed by Chao Sun. 2019-10-03 16:36:50 -07:00
Shweta Yakkali c87b1f6f5b HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-10-03 16:25:31 -07:00
Chen Liang 72b37e8833 HDFS-12979. StandbyNode should upload FsImage to ObserverNode after checkpointing. Contributed by Chen Liang. 2019-10-03 12:23:25 -07:00
Jonathan Hung 842b306b76 HADOOP-16588. Update commons-beanutils version to 1.9.4 in branch-2. Contributed by Wei-Chiu Chuang and Jonathan Hung. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-10-03 13:09:43 +09:00
Chen Liang ed8f1ae602 HDFS-14858. [SBN read] Allow configurably enable/disable AlignmentContext on NameNode. Contributed by Chen Liang. 2019-10-02 16:51:59 -07:00
bibinchundatt c57e6bc385 YARN-9858. Optimize RMContext getExclusiveEnforcedPartitions. Contributed by Jonathan Hung. 2019-10-01 16:13:53 +05:30
Konstantin V Shvachko dc2b838a8e HDFS-14305. Fix serial number calculation in BlockTokenSecretManager to avoid token key ID overlap between NameNodes. Contributed by Konstantin V Shvachko. 2019-09-30 18:04:16 -07:00
Masatake Iwasaki 9482da7053 HADOOP-16544. update io.netty in branch-2. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-10-01 05:24:32 +09:00
Ayush Saxena 66e60942e9 HDFS-14853. Addendum: NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-28 09:14:50 +05:30
Ayush Saxena 40becd37d0 HDFS-14853. NPE in DFSNetworkTopology#chooseRandomWithStorageType() when the excludedNode is not present. Contributed by Ranith Sardar. 2019-09-28 08:30:55 +05:30
Chen Liang 071d199774 HDFS-14785. [SBN read] Change client logging to be less aggressive. Contributed by Chen Liang. 2019-09-26 16:06:35 -07:00
Jonathan Hung a0e592db5a Addendum to YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit d86a1acc866cbda845fb3896dc824baf12217383)
(cherry picked from commit f4f210d2e5)
(cherry picked from commit 95ec7050b5)
 2019-09-25 17:50:05 -07:00
Jonathan Hung eedbf9d195 YARN-9730. Support forcing configured partitions to be exclusive based on app node label 
(cherry picked from commit 73a044a63822303f792183244e25432528ecfb1e)
(cherry picked from commit dd094d79023f6598e47146166aa8c213e03d41b7)
(cherry picked from commit 10bdcb6f1da3b86146efa479c0bbc8d1da505789)
 2019-09-24 13:51:11 -07:00
Jonathan Hung 90fbfbbe71 YARN-9762. Add submission context label to audit logs. Contributed by Manoj Kumar 
(cherry picked from commit 3d78b1223d)
(cherry picked from commit a1fa9a8a7f)
(cherry picked from commit 6a1d2d56bd)
 2019-09-23 13:13:24 -07:00
Erik Krogen 0050f4363e HADOOP-16581. Addendum: Remove use of Java 8 functionality. Contributed by Masatake Iwasaki. 2019-09-23 08:06:27 -07:00
Yuval Degani 44193ede8d HADOOP-16581. Revise ValueQueue to correctly replenish queues that go below the watermark (#1463) 
(cherry picked from dd0834696a)
(cherry picked from e31594f20e)
(cherry picked from b091abc471)
 2019-09-20 09:59:14 -07:00
Wangda Tan 727cbcb41d YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. (wangda) 
(cherry picked from commit df27824b80)
 2019-09-19 15:27:30 -07:00
Weiwei Yang c351230d0e YARN-7860. Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning. Contributed by Sunil G. 
(cherry picked from commit bff858e910)
 2019-09-19 20:25:31 +00:00
Wangda Tan e56f2d4993 YARN-7817. Add Resource reference to RM's NodeInfo object so REST API can get non memory/vcore resource usages. (Sunil G via wangda) 
Change-Id: Ia7ceeabd82046645ddeaf487c763288f36cfbdee
(cherry picked from commit e0cfb0a31a)
 2019-09-19 20:21:42 +00:00
Kihwal Lee ca93156cc5 HADOOP-16582. LocalFileSystem's mkdirs() does not work as expected under viewfs. Contributed by Kihwal Lee 
(cherry picked from commit d4205dce17)
 2019-09-19 08:27:23 -05:00
Chen Liang dc72b2e65a HDFS-14822. [SBN read] Revisit GlobalStateIdContext locking when getting server state id. Contributed by Chen Liang. 2019-09-18 17:22:49 -07:00
Kihwal Lee d4038e7a39 HDFS-13959. TestUpgradeDomainBlockPlacementPolicy is flaky. Contributed by Ayush Saxena. 
(cherry picked from commit 1851d06eb3)
 2019-09-18 16:51:21 -05:00
Wei-Chiu Chuang ac1924896a HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina. 
(cherry picked from 0f2dad6679)
(cherry picked from 9c9920897d)
(cherry picked from 6af1ecc8e4)
 2019-09-18 10:31:33 -07:00
He Xiaoqiao 6fe8261fa0 HDFS-14771. Backport HDFS-14617 to branch-2 (Improve fsimage load time by writing sub-sections to the fsimage index). Contributed by He Xiaoqiao and Stephen O'Donnel. 
Reviewed-by: Stephen O'Donnell <sodonnell@cloudera.com>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-09-17 09:34:54 -07:00
Ayush Saxena b9e832bf73 HDFS-14303. Addendum: check block directory logic not correct when there is only meta file, print no meaning warn log. Contributed by qiang Liu. 2019-09-14 10:30:46 +05:30
Yi Sheng 699297e0e6 HADOOP-16555. Update commons-compress to 1.19. (#1425) Contributed by YiSheng Lien. 
(cherry picked from commit 1843c4688a)

Conflicts:
	LICENSE-binary
(cherry picked from commit d39ebbfebc)
(cherry picked from commit 7d7b415409)
 2019-09-13 15:07:38 -07:00
Takanobu Asanuma 97491bfd7f HADOOP-15804. upgrade to commons-compress 1.18. Contributed by Akira Ajisaka. 
(cherry picked from commit 9bd18324c7)
(cherry picked from commit bad5d81ac1)
 2019-09-13 15:07:09 -07:00
Varun Saxena c9a46308a1 YARN-9825. Changes for initializing placement rules with ResourceScheduler in branch-2. Contributed by Jonathan Hung. 2019-09-13 17:03:47 +05:30