Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,120 Commits 326 Branches 371 Tags
Commit Graph

17120 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wei Yan
4b1a2152eb HDFS-12835. Fix the javadoc errors in Router-based federation. 
(cherry picked from commit 301641811d93ac22dc6fe1a05f18c1f266cc5e54)
 2017-11-29 09:59:17 -08:00
Konstantin V Shvachko
4cbd5ea42b HDFS-12832. INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit. Contribuited by Konstantin Shvachko. 
(cherry picked from commit d331762f24b3f22f609366740c9c4f449edc61ac)
 2017-11-28 17:18:21 -08:00
Jason Lowe
a4f1e3036d YARN-6647. RM can crash during transitionToStandby due to InterruptedException. Contributed by Bibin A Chundatt 
(cherry picked from commit a2c7a73e33045ce42cce19aacbe45c0421a61994)
 2017-11-28 11:19:43 -06:00
Steve Loughran
8bf74f325a HADOOP-15042. Azure PageBlobInputStream.skip() can return negative value when numberOfPagesRemaining is 0. 
Contributed by Rajesh Balamohan

(cherry picked from commit 0ea182d0faa35c726dcb37249d48786bfc8ca04c)
 2017-11-28 11:57:12 +00:00
Yiqun Lin
72d0be964d HDFS-12858. Add router admin commands usage in HDFS commands reference doc. Contributed by Yiqun Lin. 
(cherry picked from commit 94bed5047113fb148194380853ff01e92897a91f)
 2017-11-28 11:52:29 +08:00
Yufei Gu
c0ab3c8407 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 
(cherry picked from commit d8923cdbf1567aee10a54f144fef734d1465ebed)
 2017-11-27 14:33:13 -08:00
Yufei Gu
f335d509d3 YARN-7290. Method canContainerBePreempted can return true when it shouldn't. (Contributed by Steven Rand) 
(cherry picked from commit 2bde3aedf139368fc71f053d8dd6580b498ff46d)
 2017-11-24 23:39:03 -08:00
Wangda Tan
6e038165a6 YARN-7509. AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby. (Tao Yang via wangda) 
Change-Id: I7477fe355419fd4a0a6e2bdda7319abad4c4c748
(cherry picked from commit 834e91ee91d22d74866afbf6252107e969bf8370)
 2017-11-23 20:02:34 -08:00
Xiao Chen
4f71c27f2e HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 
(cherry picked from commit d162252d7a7223631ff66ba0210953296407e55f)
 2017-11-23 09:02:07 -08:00
Arun Suresh
5884b8a80a YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh) 
(cherry picked from commit b46ca7e73b8bac3fdbff0b13afe009308078acf2)
 2017-11-22 19:19:52 -08:00
Yufei Gu
b85eb9bb53 YARN-7524. Remove unused FairSchedulerEventLog. (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 4cc9479dae2bfb7d14d29b55d103eea9fa35a586)
 2017-11-22 14:19:36 -08:00
Kihwal Lee
1078d477c3 HDFS-12754. Lease renewal can hit a deadlock. Contributed by Kuhu Shukla. 
(cherry picked from commit 738d1a206aba05f0b4be7d633b17db7fcd1c74bc)
 2017-11-22 13:02:45 -06:00
Lei Xu
a5a86680a4 HDFS-12847. Regenerate editsStored and editsStored.xml in HDFS tests. Contributed by Lei (Eddy) Xu. 2017-11-22 10:23:10 -08:00
Chris Douglas
03f0b8b3f2 MAPREDUCE-7011. TestClientDistributedCacheManager::testDetermineCacheVisibilities assumes all parent dirs set other exec 
(cherry picked from commit 782ba3bf9da52699b27405a3f147464975d1df99)
 2017-11-21 20:42:59 -08:00
yufei
717634e45c YARN-7513. Remove the scheduler lock in FSAppAttempt.getWeight() (Contributed by Wilfred Spiegelenburg) 
(cherry picked from commit 03c311eae3ad591630a452921172a4406dbda181)
 2017-11-21 10:35:06 -08:00
Haibo Chen
b419e8aa25 YARN-7531. ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType(). 2017-11-21 09:12:27 -08:00
Akira Ajisaka
4face7e015
HADOOP-15046. Document Apache Hadoop does not support Java 9 in BUILDING.txt. Contributed by Hanisha Koneru. 
(cherry picked from commit 0ed44f25653ad2d97e2726140a7f77a555c40471)
 2017-11-22 01:08:27 +09:00
Wangda Tan
adc5157a3c YARN-7527. Over-allocate node resource in async-scheduling mode of CapacityScheduler. (Tao Yang via wangda) 
Change-Id: I51ae6c2ab7a3d1febdd7d8d0519b63a13295ac7d
(cherry picked from commit 0d781dd03b979d65de94978071b2faa55005b34a)
 2017-11-20 13:14:17 -08:00
bibinchundatt
fbcadfe3c4 YARN-7489. ConcurrentModificationException in RMAppImpl#getRMAppMetrics. Contributed by Tao Yang. 
(cherry picked from commit b5b81a4f086126974c75dab9b54aea20a0c1a48f)
 2017-11-18 20:01:01 +05:30
bibinchundatt
ae8d1b3c34 YARN-7525. Incorrect query parameters in cluster nodes REST API document. Contributed by Tao Yang. 
(cherry picked from commit 96e6a993609d170b4d2c70b2dba8b4530093606f)
 2017-11-18 19:11:40 +05:30
Inigo Goiri
3cd5224693 HDFS-12801. RBF: Set MountTableResolver as default file resolver. Contributed by Inigo Goiri. 
(cherry picked from commit e182e777947a85943504a207deb3cf3ffc047910)
 2017-11-17 10:08:04 -08:00
Varun Vasudev
6681e7238b YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 
(cherry picked from commit 5f0b238a118f3992bd149d8c02e6a1376dee96d7)
 2017-11-17 12:05:19 +05:30
Haibo Chen
9bbe7eb226 YARN-7390. All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. (Yufei Gu via Haibo Chen) 
(cherry picked from commit 28d0fcbef40930ca5652c0e9a5d777910f3ad3c4)
 2017-11-16 10:49:40 -08:00
Sunil G
c3fb49667a YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit. Contributed by Eric Payne. 
(cherry picked from commit 61ace174cdcbca9d22abce7aa0aa71148f37ad55)
 2017-11-16 22:35:22 +05:30
Weiwei Yang
f9f268cf52 HDFS-12814. Add blockId when warning slow mirror/disk in BlockReceiver. Contributed by Jiandan Yang. 2017-11-16 16:23:13 +08:00
Xiao Chen
a78dbd1571 HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 
(cherry picked from commit b1941b200d6b4fa6a7891421c0a1e212cad3d6eb)
 2017-11-15 16:44:48 -08:00
Jason Lowe
9780d58a63 YARN-7361. Improve the docker container runtime documentation. Contributed by Shane Kumpf 
(cherry picked from commit fac72eef23bb0a74a34f289dd6ef50ffa4303aa4)
 2017-11-15 12:32:56 -06:00
Daniel Templeton
570bc80901 YARN-7414. FairScheduler#getAppWeight() should be moved into FSAppAttempt#getWeight() 
(Contributed by Soumabrata Chakraborty via Daniel Templeton)

(cherry picked from commit b246c547490dd94271806ca4caf1e5f199f0fb09)
 2017-11-15 10:06:17 -08:00
Daniel Templeton
5536fa1e37 YARN-6953. Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() 
(Contributed by Manikandan R via Daniel Templeton)

(cherry picked from commit e094eb74b9e7d8c3c6f1990445d248b062cc230b)
 2017-11-15 10:06:11 -08:00
Daniel Templeton
4a70e75386 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 
(cherry picked from commit bfd588789a879b8583ea4abd59f4f5843c5ac285)
 2017-11-14 13:20:27 -08:00
Andrew Wang
1c8d334287 Preparing for 3.0.1 development 2017-11-14 11:17:18 -08:00
Andrew Wang
3f96ecf5c3 HADOOP-15037. Add site release notes for OrgQueue and resource types. 
(cherry picked from commit 8b125741659a825c71877bd1b1cb8f7e3ef26436)
 2017-11-13 18:49:50 -08:00
Junping Du
ef212b8550 YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi. 
(cherry picked from commit e14f03dfbf078de63126a1e882261081b9ec6778)
 2017-11-13 15:28:21 -08:00
Wangda Tan
7b3cd10131 YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2017-11-13 14:45:40 -08:00
Arun Suresh
cb48eea922 HADOOP-15036. Update LICENSE.txt for HADOOP-14840. (asuresh) 
(cherry picked from commit f871b7541a5375eb117eafb9a091e4f59401231f)
 2017-11-13 14:38:05 -08:00
Arun Suresh
f6de2b741e Addendum patch for Configuration fix. (Jason Lowe via asuresh) 
(cherry picked from commit b07e68b02a34d272114dda4194992a847928aef8)
 2017-11-13 14:05:41 -08:00
Daniel Templeton
c2f479c248 YARN-7369. Improve the resource types docs 2017-11-13 11:08:58 -08:00
Daniel Templeton
5b55a74bad YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton) 
(cherry picked from commit 2c6213a44280f5b3950167131293ff83f48ff56f)
 2017-11-13 11:03:56 -08:00
Xiaoyu Yao
f90f772384 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 
(cherry picked from commit 59d78a5088700350a5122c3a3ba5e76cd26d6a80)
 2017-11-12 19:44:18 -08:00
Anu Engineer
ba4355a62f HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru. 
(cherry picked from commit e1b32e0959dea5f5a40055157476f9320519a618)
 2017-11-12 19:44:18 -08:00
Subru Krishnan
37ae818660 YARN-7475. Fix Container log link in new YARN UI. (Sunil G via Subru). 
(cherry picked from commit 3c5b46c2edd69bb238d635ae61ff91656dec23df)
(cherry picked from commit 3e26077848ed1d7461576116a9ae841d38aa3ef1)
 2017-11-12 09:54:55 -08:00
bibinchundatt
bfbf94197f YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang. 
(cherry picked from commit 2c2b7a3672e0744ce6a77a117cedefba04fed603)
 2017-11-11 10:38:47 +05:30
Chris Douglas
7f80c2f293 HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 
(cherry picked from commit 796a0d3a5c661f0c3b23af9c0db2d8f3db83c322)
 2017-11-10 17:45:15 -08:00
Xiao Chen
65d7968afd HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 
(cherry picked from commit 3c6adda291745c592741b87cd613214ae11887e4)
 2017-11-09 21:06:43 -08:00
Arun Suresh
72d7bedac9 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 
(cherry picked from commit 5eb7dbe9b31a45f57f2e1623aa1c9ce84a56c4d1)
 2017-11-09 15:16:55 -08:00
Sean Mackrory
abaf90792e YARN-7465. start-yarn.sh fails to start ResourceManagers unless running as root. 
(cherry picked from commit 1883a0024949f7946264a4d7649b03fd1881567a)
 2017-11-09 14:49:54 -07:00
Daniel Templeton
38f5d4cbba YARN-7413. Support resource type in SLS (Contributed by Yufei Gu via Daniel Templeton) 
Change-Id: Ic0a897c123c5d2f57aae757ca6bcf1dad7b90d2b
(cherry picked from commit ba8136615ab66c450884614557eddc6509d63b7c)
 2017-11-09 13:39:08 -08:00
Yufei Gu
ddef7d05a4 YARN-6505. Define the strings used in SLS JSON input file format. (Contributed by Gergely Novak) 
(cherry picked from commit 99880d0a16727c770da053464da87960c5b02065)
 2017-11-09 13:38:57 -08:00
Konstantinos Karanasos
e55dc1700d YARN-7437. Rename PlacementSet and SchedulingPlacementSet. (Wangda Tan via kkaranasos) 
(cherry picked from commit ac4d2b1081d8836a21bc70e77f4e6cd2071a9949)
 2017-11-09 13:07:10 -08:00
Robert Kanter
82abc7224f YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter) 
(cherry picked from commit a2c150a7369cc629bbfaa2dfa3a8495b6f9c42e2)
 2017-11-09 12:13:09 -08:00