Commit Graph

17065 Commits

Author SHA1 Message Date
Subru Krishnan 7441d14f1c HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru).
(cherry picked from commit f2df6b8983)
2017-11-08 18:08:09 -08:00
Daniel Templeton 9bcc9e3f4d YARN-7458. TestContainerManagerSecurity is still flakey
(Contributed by Robert Kanter via Daniel Templeton)

Change-Id: Ibb1975ad086c3a33f8af0b4f8b9a13c3cdca3f7d
(cherry picked from commit 49b4c0b334)
2017-11-08 17:32:34 -08:00
Daniel Templeton d2b16e3e83 YARN-7166. Container REST endpoints should report resource types
Change-Id: If9c2fe58d4cf758bb6b6cf363dc01f35f8720987
(cherry picked from commit 0de10680b7)
2017-11-08 16:51:20 -08:00
Arun Suresh 91e726c07f YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh)
(cherry picked from commit a9c70b0e84)
2017-11-08 08:01:54 -08:00
Akira Ajisaka a536655c63
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling.
(cherry picked from commit e4c220ee4f)
2017-11-08 19:28:59 +09:00
Akira Ajisaka d77f17458c
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák.
(cherry picked from commit ffee10b68e)
2017-11-08 19:23:51 +09:00
Brahma Reddy Battula d25d3827d5 HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula
(cherry picked from commit 410d0319cf)
2017-11-08 14:49:10 +05:30
Weiwei Yang 19b1164fc6 HDFS-7060. Avoid taking locks when sending heartbeats from the DataNode. Contributed by Jiandan Yang. 2017-11-08 10:24:35 +08:00
Andrew Wang 47f2db2c7d HADOOP-15018. Update JAVA_HOME in create-release for Xenial Dockerfile.
(cherry picked from commit 51e882d5c9)
2017-11-07 16:39:06 -08:00
Daniel Templeton 4e7b5824b4 YARN-7401. Reduce lock contention in ClusterNodeTracker#getClusterCapacity()
(cherry picked from commit 8db9d61ac2)
2017-11-07 14:58:56 -08:00
Haibo Chen 91e95dc484 YARN-7360. TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic.
(cherry picked from commit 8f214dc4f8)
2017-11-06 15:54:14 -08:00
Wangda Tan df27824b80 YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. (wangda) 2017-11-06 15:26:49 -08:00
Wangda Tan 170b6a48c4 YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) 2017-11-06 10:01:10 -08:00
bibinchundatt e49509f024 Add containerId to Localizer failed logs. Contributed by Prabhu Joseph
(cherry picked from commit dcd99c4b9a)
2017-11-06 23:07:16 +05:30
Naganarasimha 8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph.
(cherry picked from commit 61bbdc511e)
2017-11-06 12:59:52 +05:30
Kihwal Lee 50b438b597 HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla.
(cherry picked from commit 4d2dce40bb)
2017-11-03 15:18:36 -05:00
Xiao Chen 71965747f4 HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang.
(cherry picked from commit c8eb4beabad84686f869a2397f7188499cab12c9)
2017-11-03 12:05:55 -07:00
Xiao Chen 621a50ebb1 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks.
(cherry picked from commit e1187bad8e9b9abb54a55b5f9ab8373b0f64e6d7)
2017-11-02 21:53:20 -07:00
Inigo Goiri 3e70fd9af0 YARN-7434. Router getApps REST invocation fails with multiple RMs. Contributed by Inigo Goiri.
(cherry picked from commit 6e2259264ad9525eeec2a14055d53236711659d7)
2017-11-02 21:30:20 -07:00
Xiao Chen 1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED.
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
2017-11-02 21:27:43 -07:00
Subru Krishnan 1b6de3fac5 YARN-7432. Fix DominantResourceFairnessPolicy serializable findbugs issues. (Daniel Templeton via Subru).
(cherry picked from commit ad0fff2b41)
2017-11-02 17:40:03 -07:00
Arun Suresh cbd81f305b HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh)
(cherry picked from commit 53c0fb7efe)
2017-11-02 17:17:27 -07:00
Eric Payne 3d923d66e8 YARN-7370: Preemption properties should be refreshable. Contrubted by Gergely Novák.
(cherry picked from commit e6ec02001f)
2017-11-02 12:48:07 -05:00
Wangda Tan fb442e9a7c YARN-7364. Queue dash board in new YARN UI has incorrect values. (Sunil G via wangda)
Change-Id: Ia5336f407147d2985a61b5f9f1e69b5b35f398a4
(cherry picked from commit 752f000e0f)
2017-11-02 09:50:57 -07:00
Jason Lowe b7dee1f060 YARN-7286. Add support for docker to have no capabilities. Contributed by Eric Badger
(cherry picked from commit d00b6f7c1f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/linux/runtime/TestDockerContainerRuntime.java
2017-11-02 09:47:26 -05:00
Akira Ajisaka 2ba71afcd5
MAPREDUCE-6999. Fix typo onf in DynamicInputChunk.java. Contributed by fang zhenyi.
(cherry picked from commit cc3f3eca40)
2017-11-02 18:32:57 +09:00
Sammi Chen 6b31a94b01 HADOOP-14997. Add hadoop-aliyun as dependency of hadoop-cloud-storage. Contributed by Genmao Yu 2017-11-02 17:08:53 +08:00
Akira Ajisaka 12f92e636e
MAPREDUCE-6983. Moving logging APIs over to slf4j in hadoop-mapreduce-client-core. Contributed by Jinjiang Ling.
(cherry picked from commit 178751ed8c)
2017-11-02 17:43:55 +09:00
Rohith Sharma K S 9efe9bd3d8 addendum patch for YARN-7289.
(cherry picked from commit 940ffe3f9c)
2017-11-02 14:02:48 +05:30
Eric Yang be7cfe57fc YARN-7412. Fix unit test for docker mount check on ubuntu. (Contributed by Eric Badger) 2017-11-01 18:45:24 -04:00
Junping Du 552f74e6b7 YARN-7400. Incorrect log preview displayed in jobhistory server ui. Contributed by Xuan Gong.
(cherry picked from commit d57dba9942)
2017-11-01 15:02:50 -07:00
Inigo Goiri 08c3efa4a0 YARN-7276. Federation Router Web Service fixes. Contributed by Inigo Goiri.
(cherry picked from commit 8be5707067)
2017-11-01 13:42:12 -07:00
Inigo Goiri 8d41875219 YARN-7276 addendum to add timeline service depencies. Contributed by Inigo Goiri.
(cherry picked from commit 70f1a9470c)
2017-11-01 13:28:50 -07:00
Weiwei Yang b2f717d967 HDFS-12744. More logs when short-circuit read is failed and disabled. Contributed by Weiwei Yang.
(cherry picked from commit 56b88b0670)
2017-11-01 16:53:07 +08:00
Yiqun Lin 5cada4564a
HDFS-12219. Javadoc for FSNamesystem#getMaxObjects is incorrect. Contributed by Erik Krogen.
(cherry picked from commit 20304b91cc)
2017-11-01 16:43:44 +09:00
John Zhuge 0ec172b42f HDFS-12714. Hadoop 3 missing fix for HDFS-5169. Contributed by Joe McDonnell.
(cherry picked from commit b8c8b5bc27)
2017-10-31 22:46:26 -07:00
Lei Xu 9ee04006cb HDFS-12482. Provide a configuration to adjust the weight of EC recovery tasks to adjust the speed of recovery. (lei) 2017-10-31 22:03:44 -07:00
Daniel Templeton c025c9ac62 YARN-7336. Unsafe cast from long to int Resource.hashCode() method
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit d64736d589)
2017-10-31 15:06:18 -07:00
Daniel Templeton 7689d4781a YARN-6927. Add support for individual resource types requests in MapReduce
(Contributed by Gergo Repas via Daniel Templeton)

(cherry picked from commit 9a7e810838)
2017-10-31 15:06:18 -07:00
Daniel Templeton 77401022a7 YARN-7374. Improve performance of DRF comparisons for resource types in fair scheduler
(cherry picked from commit 9711b78998)
2017-10-31 15:06:18 -07:00
Daniel Templeton ab93bf5b00 YARN-7397. Reduce lock contention in FairScheduler#getAppWeight()
(cherry picked from commit e62bbbca7a)
2017-10-31 15:06:18 -07:00
Daniel Templeton c818c85238 YARN-6984. DominantResourceCalculator.isAnyMajorResourceZero() should test all resources
(Contributed by Sunil G via Daniel Templeton)

(cherry picked from commit 679f99b145)
2017-10-31 15:06:18 -07:00
Daniel Templeton 68585156dc YARN-7270 addendum: Reapplied changes after YARN-3926 backports 2017-10-31 15:06:18 -07:00
Wangda Tan 6f41843906 YARN-7172. ResourceCalculator.fitsIn() should not take a cluster resource parameter. (Sen Zhao via wangda)
Change-Id: Icc3670c9381ce7591ca69ec12da5aa52d3612d34
(cherry picked from commit e81596d06d)
2017-10-31 15:06:18 -07:00
Yufei Gu f30f78a736 YARN-6612. Update fair scheduler policies to be aware of resource types. (Contributed by Daniel Templeton via Yufei Gu)
(cherry picked from commit 09b476e6da)
2017-10-31 15:06:18 -07:00
Sunil G 74030d808c YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan.
(cherry picked from commit da0b6a354b)
2017-10-31 15:06:17 -07:00
Daniel Templeton 0db9ddb728 YARN-7136. Additional Performance Improvement for Resource Profile Feature
(Contributed by Wangda Tan via Daniel Templeton)

(cherry picked from commit bf2b687412)
2017-10-31 15:06:17 -07:00
Wangda Tan 335e3a6334 YARN-7056. [Partial backport] Document Resource Profiles feature. (Sunil G via wangda)
Change-Id: I5f5d52cdf7aba4ac09684aab573e6916dbad183e
(cherry picked from commit 6b88cd1901)
2017-10-31 15:06:17 -07:00
Sunil G 4bf29910a8 YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R.
(cherry picked from commit 805095496d)
2017-10-31 15:06:17 -07:00
Wangda Tan 00fcc8593b YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda)
Change-Id: I88928a747ee3eec17bc76fb71e9aaa632d091f0f
(cherry picked from commit 53df3eac50)
2017-10-31 15:06:17 -07:00