Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,311 Commits 326 Branches 371 Tags
Commit Graph

17311 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arun Suresh
5884b8a80a YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh) 
(cherry picked from commit b46ca7e73b8bac3fdbff0b13afe009308078acf2)
 2017-11-22 19:19:52 -08:00
Yufei Gu
b85eb9bb53 YARN-7524. Remove unused FairSchedulerEventLog. (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 4cc9479dae2bfb7d14d29b55d103eea9fa35a586)
 2017-11-22 14:19:36 -08:00
Kihwal Lee
1078d477c3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 738d1a206aba05f0b4be7d633b17db7fcd1c74bc)
 2017-11-22 13:02:45 -06:00
Lei Xu
a5a86680a4 HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:23:10 -08:00
Chris Douglas
03f0b8b3f2 MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 
(cherry picked from commit 782ba3bf9da52699b27405a3f147464975d1df99)
 2017-11-21 20:42:59 -08:00
yufei
717634e45c YARN-7513. Remove the scheduler lock in FSAppAttempt.getWeight() (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 03c311eae3ad591630a452921172a4406dbda181)
 2017-11-21 10:35:06 -08:00
Haibo Chen
b419e8aa25 YARN-7531. ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType(). 2017-11-21 09:12:27 -08:00
Akira Ajisaka
4face7e015
HADOOP-15046. Document Apache Hadoop does not support Java 9 in BUILDING.txt. Contributed by Hanisha Koneru. 
(cherry picked from commit 0ed44f25653ad2d97e2726140a7f77a555c40471)
 2017-11-22 01:08:27 +09:00
Wangda Tan
adc5157a3c YARN-7527. Over-allocate node resource in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I51ae6c2ab7a3d1febdd7d8d0519b63a13295ac7d
(cherry picked from commit 0d781dd03b979d65de94978071b2faa55005b34a)
 2017-11-20 13:14:17 -08:00
bibinchundatt
fbcadfe3c4 YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit b5b81a4f086126974c75dab9b54aea20a0c1a48f)
 2017-11-18 20:01:01 +05:30
bibinchundatt
ae8d1b3c34 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit 96e6a993609d170b4d2c70b2dba8b4530093606f)
 2017-11-18 19:11:40 +05:30
Inigo Goiri
3cd5224693 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e777947a85943504a207deb3cf3ffc047910)
 2017-11-17 10:08:04 -08:00
Varun Vasudev
6681e7238b YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
 2017-11-17 12:05:19 +05:30
Haibo Chen
9bbe7eb226 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. (Yufei Gu via Haibo Chen) 
(cherry picked from commit 28d0fcbef40930ca5652c0e9a5d777910f3ad3c4)
 2017-11-16 10:49:40 -08:00
Sunil G
c3fb49667a YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:35:22 +05:30
Weiwei Yang
f9f268cf52 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:23:13 +08:00
Xiao Chen
a78dbd1571 HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 
(cherry picked from commit b1941b200d6b4fa6a7891421c0a1e212cad3d6eb)
 2017-11-15 16:44:48 -08:00
Jason Lowe
9780d58a63 YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:32:56 -06:00
Daniel Templeton
570bc80901 YARN-7414. FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight() 
(Contributed by Soumabrata Chakraborty via Daniel Templeton)

(cherry picked from commit b246c547490dd94271806ca4caf1e5f199f0fb09)
 2017-11-15 10:06:17 -08:00
Daniel Templeton
5536fa1e37 YARN-6953. Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit e094eb74b9e7d8c3c6f1990445d248b062cc230b)
 2017-11-15 10:06:11 -08:00
Daniel Templeton
4a70e75386 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 
(cherry picked from commit bfd588789a879b8583ea4abd59f4f5843c5ac285)
 2017-11-14 13:20:27 -08:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Andrew Wang
3f96ecf5c3 HADOOP-15037. Add site release notes for OrgQueue and resource types. 
(cherry picked from commit 8b125741659a825c71877bd1b1cb8f7e3ef26436)
 2017-11-13 18:49:50 -08:00
Junping Du
ef212b8550 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit e14f03dfbf078de63126a1e882261081b9ec6778)
 2017-11-13 15:28:21 -08:00
Wangda Tan
7b3cd10131 YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2017-11-13 14:45:40 -08:00
Arun Suresh
cb48eea922 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh) 
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
 2017-11-13 14:38:05 -08:00
Arun Suresh
f6de2b741e Addendum patch for Configuration fix. (Jason Lowe via asuresh) 
(cherry picked from commit b07e68b02a34d272114dda4194992a847928aef8)
 2017-11-13 14:05:41 -08:00
Daniel Templeton
c2f479c248 YARN-7369. Improve the resource types docs 2017-11-13 11:08:58 -08:00
Daniel Templeton
5b55a74bad YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton) 
(cherry picked from commit 2c6213a44280f5b3950167131293ff83f48ff56f)
 2017-11-13 11:03:56 -08:00
Xiaoyu Yao
f90f772384 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 
(cherry picked from commit 59d78a5088700350a5122c3a3ba5e76cd26d6a80)
 2017-11-12 19:44:18 -08:00
Anu Engineer
ba4355a62f HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru. 
(cherry picked from commit e1b32e0959dea5f5a40055157476f9320519a618)
 2017-11-12 19:44:18 -08:00
Subru Krishnan
37ae818660 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
(cherry picked from commit 3e26077848ed1d7461576116a9ae841d38aa3ef1)
 2017-11-12 09:54:55 -08:00
bibinchundatt
bfbf94197f YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang. 
(cherry picked from commit 2c2b7a3672e0744ce6a77a117cedefba04fed603)
 2017-11-11 10:38:47 +05:30
Chris Douglas
7f80c2f293 HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 
(cherry picked from commit 796a0d3a5c661f0c3b23af9c0db2d8f3db83c322)
 2017-11-10 17:45:15 -08:00
Xiao Chen
65d7968afd HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 
(cherry picked from commit 3c6adda291745c592741b87cd613214ae11887e4)
 2017-11-09 21:06:43 -08:00
Arun Suresh
72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 5eb7dbe9b31a45f57f2e1623aa1c9ce84a56c4d1)
 2017-11-09 15:16:55 -08:00
Sean Mackrory
abaf90792e YARN-7465. start-yarn.sh fails to start ResourceManagers unless running as root. 
(cherry picked from commit 1883a0024949f7946264a4d7649b03fd1881567a)
 2017-11-09 14:49:54 -07:00
Daniel Templeton
38f5d4cbba YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) 
Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b
(cherry picked from commit ba8136615ab66c450884614557eddc6509d63b7c)
 2017-11-09 13:39:08 -08:00
Yufei Gu
ddef7d05a4 YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak) 
(cherry picked from commit 99880d0a16727c770da053464da87960c5b02065)
 2017-11-09 13:38:57 -08:00
Konstantinos Karanasos
e55dc1700d YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos) 
(cherry picked from commit ac4d2b1081d8836a21bc70e77f4e6cd2071a9949)
 2017-11-09 13:07:10 -08:00
Robert Kanter
82abc7224f YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter) 
(cherry picked from commit a2c150a7369cc629bbfaa2dfa3a8495b6f9c42e2)
 2017-11-09 12:13:09 -08:00
Daniel Templeton
933a09e886 YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
(cherry picked from commit db82a41d94872cea4d0c1bb1336916cebc2faeec)
 2017-11-09 11:59:16 -08:00
Wangda Tan
51d3693a9c YARN-7205. Log improvements for the ResourceUtils. (Sunil G via wangda) 
Change-Id: I0f5b7a7f68ec5d3e1d52211f83fdd089bc0bfd37
(cherry picked from commit 8bcc49e6771ca75f012211e27870a421b19233e7)
 2017-11-09 11:59:16 -08:00
Haibo Chen
91a7f8d246 YARN-7388. TestAMRestart should be scheduler agnostic. 
(cherry picked from commit a1382a18dff8a70aa25240d6fbba6e22832a7679)
 2017-11-09 10:50:49 -08:00
bibinchundatt
4370b70e63 YARN-7454. RMAppAttemptMetrics#getAggregateResourceUsage can NPE due to double lookup. Contributed by Jason Lowe. 
(cherry picked from commit 0a72c2f56c37063609de72eef1f74632890c048a)
 2017-11-09 21:08:47 +05:30
Akira Ajisaka
9051d1b851
HDFS-12732. Correct spellings of ramdomly to randomly in log. Contributed by hu xiaodong. 
(cherry picked from commit 3a3566e1d1ab5f78cfb734796b41802fe039196d)
 2017-11-09 15:18:32 +09:00
Subru Krishnan
7441d14f1c HADOOP-15025. Ensure singleton for ResourceEstimatorService. (Rui Li via Subru). 
(cherry picked from commit f2df6b8983aace73ad27934bd9f7f4d766e0b25f)
 2017-11-08 18:08:09 -08:00
Daniel Templeton
9bcc9e3f4d YARN-7458. TestContainerManagerSecurity is still flakey 
(Contributed by Robert Kanter via Daniel Templeton)

Change-Id: Ibb1975ad086c3a33f8af0b4f8b9a13c3cdca3f7d
(cherry picked from commit 49b4c0b334e5472dbbf71b042a6a6b1d4b2ce3b7)
 2017-11-08 17:32:34 -08:00
Daniel Templeton
d2b16e3e83 YARN-7166. Container REST endpoints should report resource types 
Change-Id: If9c2fe58d4cf758bb6b6cf363dc01f35f8720987
(cherry picked from commit 0de10680b7e5a9dfc85173bcfd338fd3656aa57f)
 2017-11-08 16:51:20 -08:00
Arun Suresh
91e726c07f YARN-7453. Fix issue where RM fails to switch to active after first successful start. (Rohith Sharma K S via asuresh) 
(cherry picked from commit a9c70b0e84dab0c41e480a0dc0cb1a22efdc64ee)
 2017-11-08 08:01:54 -08:00