Commit Graph

19622 Commits

Author SHA1 Message Date
Akira Ajisaka 5853fa2ea2
HADOOP-16609. Add Jenkinsfile for all active branches ()
* Backport HADOOP-16035. Jenkinsfile for Hadoop
* Use Yetus 0.10.0

(cherry picked from commit ae8ca70fc0)
2019-12-25 17:25:14 +09:00
Chandra Sanivarapu e33651ace7
HDFS-15073. Replace curator-shaded guava import with the standard one ()
(cherry picked from commit d8cd7098b4)
2019-12-25 11:12:29 +09:00
Konstantin V Shvachko 1a22e4036c HDFS-15076. Fix tests that hold FSDirectory lock, without holding FSNamesystem lock. Contributed by Konstantin V Shvachko.
(cherry picked from commit b98ac2a3af)
2019-12-24 12:50:13 -08:00
Akira Ajisaka 01edc654df
YARN-10055. bower install fails. ()
(cherry picked from commit 34ff7dbaf5)
2019-12-23 20:27:06 +09:00
Eric Badger 0f01206e15 YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne
(cherry picked from commit 355ec33416)
2019-12-20 19:34:29 +00:00
Eric Badger ef00db4ce9 Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui."
This reverts commit 27555620fa.
2019-12-20 19:16:22 +00:00
Akira Ajisaka 39c6849d34
HADOOP-16771. Update checkstyle to 8.26 and maven-checkstyle-plugin to 3.1.0. Contributed by Andras Bokor.
(cherry picked from commit f777cd398f)
2019-12-20 13:12:46 +09:00
Inigo Goiri 27555620fa HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui.
(cherry picked from commit 52d7b745c6)
2019-12-19 09:47:30 -08:00
Jonathan Hung 750fb4c321 YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne
(cherry picked from commit 7b93575b92)
(cherry picked from commit 0707d0a0ae)
2019-12-18 13:25:30 -08:00
Jonathan Hung 406c35dd12 YARN-10039. Allow disabling app submission from REST endpoints 2019-12-18 10:58:32 -08:00
Shashikant Banerjee 280cd6dd1e HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee.
(cherry picked from commit fdd96e46d1)
2019-12-18 23:11:18 +05:30
Eric Badger 86120637a4 YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne.
(cherry picked from commit f47dcf2d4c)
2019-12-17 17:39:51 +00:00
Chen Liang ea0eeb8f1a HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang.
(cherry picked from commit 65c4660bcd)
(cherry picked from commit d4a6901c42)
2019-12-12 15:58:09 -08:00
He Xiaoqiao 974706c5fd HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
(cherry picked from commit 8ba4e13dcc)
2019-12-12 02:23:22 -08:00
pingsutw 70a0ef5d4a
HADOOP-16754. Fix docker failed to build yetus/hadoop
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-12-12 11:03:51 +09:00
Mingliang Liu a2336d35f6
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml ()
Contributed by Mingliang Liu
2019-12-11 11:54:56 -08:00
Erik Krogen b06965ae32 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen.
(cherry picked from c174d50b30)
(cherry picked from 3402c87353)
2019-12-11 09:41:11 -08:00
dependabot[bot] 935854e0ca
Bump nimbus-jose-jwt from 4.41.1 to 7.9 ()
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a156)
2019-12-09 10:23:00 +09:00
Jonathan Hung 9b4368a62f YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R
(cherry picked from commit 92bce918dc)
(cherry picked from commit 9228e3f0ad)
2019-12-08 16:38:21 -08:00
Sunil G 67cf1f94cd YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko.
(cherry picked from commit 002dcc4ebf)
(cherry picked from commit 69dc329acc)
2019-12-08 14:43:20 -08:00
Masatake Iwasaki 80a84ad8f5 HADOOP-16744. Fix building instruction to enable zstd. ()
(cherry picked from commit 18059acb6a)
2019-12-06 15:28:44 +09:00
Mingliang Liu 941a98eb28
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu
This closes 
2019-12-05 17:52:25 -08:00
Aravindan Vijayan e73fa45e5b HADOOP-16718. Allow disabling Server Name Indication (SNI) for Jetty. Contributed by Aravindan Vijayan.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Siyao Meng <smeng@cloudera.com>
(cherry picked from commit f1ab7f18c4)
(cherry picked from commit f0c1403ec3)
2019-12-03 15:48:25 -08:00
Erik Krogen 23f25418e8 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq.
(cherry picked from be77231452)
(cherry picked from c1ed247459)
2019-12-03 14:02:00 -08:00
Surendra Singh Lilhore 5eb21b83df HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-29 00:07:45 +05:30
prabhujoseph a97f7776bd MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure.
Contributed by Wilfred Spiegelenburg.
2019-11-28 14:34:52 +05:30
Sunil G 57c499ef19 YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo.
Contributed by Prabhu Joseph.
2019-11-27 23:01:16 +05:30
Szilard Nemeth 1172ebd6e8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo.
Contributed by Prabhu Joseph.
2019-11-27 22:30:10 +05:30
prabhujoseph 6a4966f336 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER.
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
2019-11-27 16:51:47 +05:30
Szilard Nemeth 80a97e6ac2 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:27:55 +01:00
HUAN-PING SU fb2df5234d
YARN-9966. Code duplication in UserGroupMappingPlacementRule ()
(cherry picked from commit f8e36e03b4)
2019-11-25 15:58:09 +09:00
Ayush Saxena 35f0478042 HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:47:59 +05:30
Szilard Nemeth df157211c9 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 13:00:18 +01:00
Tao Yang c95e772d5d YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:10:07 +08:00
Surendra Singh Lilhore 8551c04726 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:45:55 +05:30
Wei-Chiu Chuang 0807cb2485 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang.
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 9664b9c7a6)
(cherry picked from commit bd2f79cfd8)
2019-11-20 13:08:52 -08:00
Wanqiang Ji c7d0e0813a
HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin ()
(cherry picked from commit dfc61d8ea5)
(cherry picked from commit 4c79e2a02b)
2019-11-20 16:23:21 +09:00
He Xiaoqiao d101b74814 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd)
(cherry picked from commit 1d383436f09f8e490171c8f6af4605971ad2f904)
2019-11-19 12:22:54 -08:00
Szilard Nemeth 0a4e6b7b62 YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal 2019-11-19 21:22:01 +01:00
Siyao Meng df13f6d485 HADOOP-16703. Backport HADOOP-16152 to branch-3.1. Contributed by Siyao Meng.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-11-19 12:14:27 -08:00
Eric Yang 1669a5c1cb YARN-9983. Fixed typo in YARN Service overview.
Contributed by Denes Gerencser
2019-11-19 14:18:38 -05:00
Sunil G 9a51380208 YARN-9873. Version Number for each Scheduler Config Change. Contributed by Prabhu Joseph.
(cherry picked from commit be901f4962)
2019-11-19 17:17:45 +05:30
Abhishek Modi 96e07b7e97 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph.
(cherry picked from commit 751b5a1ac8)
2019-11-19 16:41:04 +05:30
Sunil G dab4cc6320 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph. 2019-11-19 16:19:19 +05:30
Sunil G 9b17fa7091 YARN-9950. Unset Ordering Policy of Leaf/Parent queue converted from Parent/Leaf queue respectively. Contributed by Prabhu Joseph.
(cherry picked from commit 51e7d1b37e)
2019-11-19 15:43:19 +05:30
Sunil G f24d1fac35 YARN-9984. FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node. Contributed by Wilfred Spiegelenburg.
(cherry picked from commit 215f2052fc)
2019-11-19 14:05:51 +05:30
Ayush Saxena b4f0c63a6a HDFS-14942. Change Log Level to debug in JournalNodeSyncer#syncWithJournalAtIndex. Contributed by Lisheng Sun.
(cherry picked from commit 9e287054a8)
(cherry picked from commit d16441460215155b25fa649a7ca733e579d97929)
2019-11-18 10:23:48 -08:00
Ayush Saxena 7d3c67b4a7 HADOOP-16712. Config ha.failover-controller.active-standby-elector.zk.op.retries is not in core-default.xml. Contributed by Xieming Li. 2019-11-18 19:59:18 +05:30
Prabhu Joseph d271d174ca YARN-9900. Revert Invalid Config and Refresh Support in SchedulerConfig Format.
Signed-off-by: prabhujoseph <pjoseph@cloudera.com>
2019-11-18 15:02:59 +05:30
Lukas Majercak 50d98a6568 HADOOP-16705. MBeanInfoBuilder puts unnecessary memory pressure on the system with a debug log.
MBeanInfoBuilder's get() method DEBUG logs all the MBeanAttributeInfo attributes that it gathered. This can have a high memory churn that can be easily avoided.

(cherry picked from commit c73334a924)
2019-11-15 13:29:29 -08:00