Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,392 Commits 326 Branches 371 Tags
Commit Graph

17392 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Templeton
5536fa1e37 YARN-6953. Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit e094eb74b9e7d8c3c6f1990445d248b062cc230b)
 2017-11-15 10:06:11 -08:00
Daniel Templeton
4a70e75386 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 
(cherry picked from commit bfd588789a879b8583ea4abd59f4f5843c5ac285)
 2017-11-14 13:20:27 -08:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Andrew Wang
3f96ecf5c3 HADOOP-15037. Add site release notes for OrgQueue and resource types. 
(cherry picked from commit 8b125741659a825c71877bd1b1cb8f7e3ef26436)
 2017-11-13 18:49:50 -08:00
Junping Du
ef212b8550 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit e14f03dfbf078de63126a1e882261081b9ec6778)
 2017-11-13 15:28:21 -08:00
Wangda Tan
7b3cd10131 YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2017-11-13 14:45:40 -08:00
Arun Suresh
cb48eea922 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh) 
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
 2017-11-13 14:38:05 -08:00
Arun Suresh
f6de2b741e Addendum patch for Configuration fix. (Jason Lowe via asuresh) 
(cherry picked from commit b07e68b02a34d272114dda4194992a847928aef8)
 2017-11-13 14:05:41 -08:00
Daniel Templeton
c2f479c248 YARN-7369. Improve the resource types docs 2017-11-13 11:08:58 -08:00
Daniel Templeton
5b55a74bad YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton) 
(cherry picked from commit 2c6213a44280f5b3950167131293ff83f48ff56f)
 2017-11-13 11:03:56 -08:00
Xiaoyu Yao
f90f772384 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 
(cherry picked from commit 59d78a5088700350a5122c3a3ba5e76cd26d6a80)
 2017-11-12 19:44:18 -08:00
Anu Engineer
ba4355a62f HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru. 
(cherry picked from commit e1b32e0959dea5f5a40055157476f9320519a618)
 2017-11-12 19:44:18 -08:00
Subru Krishnan
37ae818660 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
(cherry picked from commit 3e26077848ed1d7461576116a9ae841d38aa3ef1)
 2017-11-12 09:54:55 -08:00
bibinchundatt
bfbf94197f YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang. 
(cherry picked from commit 2c2b7a3672e0744ce6a77a117cedefba04fed603)
 2017-11-11 10:38:47 +05:30
Chris Douglas
7f80c2f293 HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 
(cherry picked from commit 796a0d3a5c661f0c3b23af9c0db2d8f3db83c322)
 2017-11-10 17:45:15 -08:00
Xiao Chen
65d7968afd HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 
(cherry picked from commit 3c6adda291745c592741b87cd613214ae11887e4)
 2017-11-09 21:06:43 -08:00
Arun Suresh
72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 5eb7dbe9b31a45f57f2e1623aa1c9ce84a56c4d1)
 2017-11-09 15:16:55 -08:00
Sean Mackrory
abaf90792e YARN-7465. start-yarn.sh fails to start ResourceManagers unless running as root. 
(cherry picked from commit 1883a0024949f7946264a4d7649b03fd1881567a)
 2017-11-09 14:49:54 -07:00
Daniel Templeton
38f5d4cbba YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) 
Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b
(cherry picked from commit ba8136615ab66c450884614557eddc6509d63b7c)
 2017-11-09 13:39:08 -08:00
Yufei Gu
ddef7d05a4 YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak) 
(cherry picked from commit 99880d0a16727c770da053464da87960c5b02065)
 2017-11-09 13:38:57 -08:00
Konstantinos Karanasos
e55dc1700d YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos) 
(cherry picked from commit ac4d2b1081d8836a21bc70e77f4e6cd2071a9949)
 2017-11-09 13:07:10 -08:00
Robert Kanter
82abc7224f YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter) 
(cherry picked from commit a2c150a7369cc629bbfaa2dfa3a8495b6f9c42e2)
 2017-11-09 12:13:09 -08:00
Daniel Templeton
933a09e886 YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2017-11-09 11:59:16 -08:00
Wangda Tan
51d3693a9c YARN-7205. Log improvements for the ResourceUtils. (Sunil G via wangda) 
Change-Id: I0f5b7a7f68ec5d3e1d52211f83fdd089bc0bfd37
(cherry picked from commit 8bcc49e6771ca75f012211e27870a421b19233e7)
 2017-11-09 11:59:16 -08:00
Haibo Chen
91a7f8d246 YARN-7388. TestAMRestart should be scheduler agnostic. 
(cherry picked from commit a1382a18dff8a70aa25240d6fbba6e22832a7679)
 2017-11-09 10:50:49 -08:00
bibinchundatt
4370b70e63 YARN-7454. RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup. Contributed by Jason Lowe. 
(cherry picked from commit 0a72c2f56c37063609de72eef1f74632890c048a)
 2017-11-09 21:08:47 +05:30
Akira Ajisaka
9051d1b851
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong. 
(cherry picked from commit 3a3566e1d1ab5f78cfb734796b41802fe039196d)
 2017-11-09 15:18:32 +09:00
Subru Krishnan
7441d14f1c HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru). 
(cherry picked from commit f2df6b8983aace73ad27934bd9f7f4d766e0b25f)
 2017-11-08 18:08:09 -08:00
Daniel Templeton
9bcc9e3f4d YARN-7458. TestContainerManagerSecurity is still flakey 
(Contributed by Robert Kanter via Daniel Templeton)

Change-Id: Ibb1975ad086c3a33f8af0b4f8b9a13c3cdca3f7d
(cherry picked from commit 49b4c0b334e5472dbbf71b042a6a6b1d4b2ce3b7)
 2017-11-08 17:32:34 -08:00
Daniel Templeton
d2b16e3e83 YARN-7166. Container REST endpoints should report resource types 
Change-Id: If9c2fe58d4cf758bb6b6cf363dc01f35f8720987
(cherry picked from commit 0de10680b7e5a9dfc85173bcfd338fd3656aa57f)
 2017-11-08 16:51:20 -08:00
Arun Suresh
91e726c07f YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh) 
(cherry picked from commit a9c70b0e84dab0c41e480a0dc0cb1a22efdc64ee)
 2017-11-08 08:01:54 -08:00
Akira Ajisaka
a536655c63
MAPREDUCE-7001. Moving logging APIs over to slf4j in hadoop-mapreduce-client-shuffle. Contributed by Jinjiang Ling. 
(cherry picked from commit e4c220ee4fdc4550275bda0fa9468d7d87d143c3)
 2017-11-08 19:28:59 +09:00
Akira Ajisaka
d77f17458c
MAPREDUCE-6997. Moving logging APIs over to slf4j in hadoop-mapreduce-client-hs. Contributed by Gergely Novák. 
(cherry picked from commit ffee10b68ef1f2d75c9d0df9140c2a605f826724)
 2017-11-08 19:23:51 +09:00
Brahma Reddy Battula
d25d3827d5 HDFS-12788. Reset the upload button when file upload fails. Contributed by Brahma Reddy Battula 
(cherry picked from commit 410d0319cf72b9b5f3807c522237f52121d98cd5)
 2017-11-08 14:49:10 +05:30
Weiwei Yang
19b1164fc6 HDFS-7060. Avoid taking locks when sending heartbeats from the DataNode. Contributed by Jiandan Yang. 2017-11-08 10:24:35 +08:00
Andrew Wang
47f2db2c7d HADOOP-15018. Update JAVA_HOME in create-release for Xenial Dockerfile. 
(cherry picked from commit 51e882d5c9fd2f55cd9ac2eafd3e59eb4f239d9d)
 2017-11-07 16:39:06 -08:00
Daniel Templeton
4e7b5824b4 YARN-7401. Reduce lock contention in ClusterNodeTracker#getClusterCapacity() 
(cherry picked from commit 8db9d61ac2e04888cb228b29fe54b41c730cf0e6)
 2017-11-07 14:58:56 -08:00
Haibo Chen
91e95dc484 YARN-7360. TestRM.testNMTokenSentForNormalContainer() should be scheduler agnostic. 
(cherry picked from commit 8f214dc4f8423250947a3f0027f70b9ab402ab62)
 2017-11-06 15:54:14 -08:00
Wangda Tan
df27824b80 YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. (wangda) 2017-11-06 15:26:49 -08:00
Wangda Tan
170b6a48c4 YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) 2017-11-06 10:01:10 -08:00
bibinchundatt
e49509f024 Add containerId to Localizer failed logs. Contributed by Prabhu Joseph 
(cherry picked from commit dcd99c4b9a7a15649dc9823ac060ce23c6a31056)
 2017-11-06 23:07:16 +05:30
Naganarasimha
8e9369d387 MAPREDUCE-6975. Logging task counters. Contributed by Prabhu Joseph. 
(cherry picked from commit 61bbdc511e4e98c11a05d7c3bc01f3e971adbe13)
 2017-11-06 12:59:52 +05:30
Kihwal Lee
50b438b597 HDFS-12771. Add genstamp and block size to metasave Corrupt blocks list. Contributed by Kuhu Shukla. 
(cherry picked from commit 4d2dce40bbe5242953774e6a2fc0dc9111cf5ed0)
 2017-11-03 15:18:36 -05:00
Xiao Chen
71965747f4 HDFS-11467. Support ErasureCoding section in OIV XML/ReverseXML. Contributed by Huafeng Wang. 
(cherry picked from commit c8eb4beabad84686f869a2397f7188499cab12c9)
 2017-11-03 12:05:55 -07:00
Xiao Chen
621a50ebb1 HDFS-12725. BlockPlacementPolicyRackFaultTolerant fails with very uneven racks. 
(cherry picked from commit e1187bad8e9b9abb54a55b5f9ab8373b0f64e6d7)
 2017-11-02 21:53:20 -07:00
Inigo Goiri
3e70fd9af0 YARN-7434. Router getApps REST invocation fails with multiple RMs. Contributed by Inigo Goiri. 
(cherry picked from commit 6e2259264ad9525eeec2a14055d53236711659d7)
 2017-11-02 21:30:20 -07:00
Xiao Chen
1c96e0d693 HDFS-12682. ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED. 
(cherry picked from commit 4f4cea56b25d5acc14dd6d61dea70fd4a0c7a9d9)
 2017-11-02 21:27:43 -07:00
Subru Krishnan
1b6de3fac5 YARN-7432. Fix DominantResourceFairnessPolicy serializable findbugs issues. (Daniel Templeton via Subru). 
(cherry picked from commit ad0fff2b419e8fe8bb5c9a7e19b79dec16cbd96f)
 2017-11-02 17:40:03 -07:00
Arun Suresh
cbd81f305b HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh) 
(cherry picked from commit 53c0fb7efebfac4a79f5cce2dd42cf00411d51e7)
 2017-11-02 17:17:27 -07:00
Eric Payne
3d923d66e8 YARN-7370: Preemption properties should be refreshable. Contrubted by Gergely Novák. 
(cherry picked from commit e6ec02001fc4eed9eb51c8653d8f931135e49eda)
 2017-11-02 12:48:07 -05:00