Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,555 Commits 326 Branches 371 Tags
Commit Graph

18555 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Wangda Tan
8ad82ea5b5 YARN-8506. Make GetApplicationsRequestPBImpl thread safe. (wangda) 
Change-Id: I2c006965375823c83036e7f45f7163d13c0bdf90
 2018-07-09 11:35:15 -07:00
Andrew Wang
242b5acdb3 HDFS-13719. Docs around dfs.image.transfer.timeout are misleading. Contributed by Kitti Nansi. 
(cherry picked from commit eecb5baaaaa54599aeae758abd4007e55e5b531f)
 2018-07-09 15:17:34 +02:00
Yiqun Lin
e994c4f8aa HDFS-13710. RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable. Contributed by yanghuafeng. 
(cherry picked from commit 43f7fe8aae0eca89cce4d67bfc4965fe8ce63e38)
 2018-07-09 15:07:47 +08:00
Rohith Sharma K S
785ed3f8ae YARN-8302. ATS v2 should handle HBase connection issue properly. Contributed by Billie Rinaldi. 
(cherry picked from commit ba683204498c97654be4727ab9e128c433a45498)
 2018-07-06 15:20:39 -07:00
Wei-Chiu Chuang
f5f4d0b7e7 HDFS-13121. NPE when request file descriptors when SC read. Contributed by Zsolt Venczel. 
(cherry picked from commit 0247cb6318507afe06816e337a19f396afc53efa)
 2018-07-06 15:01:10 -07:00
Xiaoyu Yao
ffe2019887 HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 629f7008b531bad469903e6a1f6f8cb4be691f36)
 2018-07-06 11:56:51 -07:00
Yiqun Lin
48addcdbbe HDFS-13528. RBF: If a directory exceeds quota limit then quota usage is not refreshed for other mount entries. Contributed by Dibyendu Karmakar. 
(cherry picked from commit 3b637155a47d2aa93284969a96208347a647083d)
 2018-07-04 15:30:17 +08:00
Sunil G
844538d669 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan. 
(cherry picked from commit 51654a3962bcd0482c0e1dd32765e9533e3f9158)
 2018-07-03 10:00:05 -07:00
Junping Du
3d6ba2dd4e yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 
(cherry picked from commit d9ba6f3656e8dc97d2813181e27d12e52dca4328)
 2018-07-03 22:36:24 +08:00
Andrew Wang
3b266abdad HDFS-13712. BlockReaderRemote.read() logging improvement. Contributed by Gergo Repas. 
(cherry picked from commit 344f324710522ffb27852c1a673c4f7d3d6eac4b)
 2018-07-03 11:07:57 +02:00
Bharat Viswanadham
0d7b811ee6 HDFS-13715:diskbalancer does not work if one of the blockpools are empty on a Federated cluster. Contributed by Bharat Viswanadham 
(cherry picked from commit 59a3038bc3d7913dca3de971026bc32cef536a2d)
 2018-07-02 21:48:09 -07:00
Inigo Goiri
177ccbc43f HDFS-13714. Fix TestNameNodePrunesMissingStorages test failures on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 7296b644f7c44400f03995668573a4cfd2ee552c)
 2018-07-02 16:37:12 -07:00
Sunil G
8ffc191f61 YARN-8415. TimelineWebServices.getEntity should throw ForbiddenException instead of 404 when ACL checks fail. Contributed by Suma Shivaprasad. 
(cherry picked from commit fa9ef15ecd6dc30fb260e1c342a2b51505d39b6b)
 2018-07-02 15:35:45 -07:00
Shane Kumpf
2f264764a4 YARN-8485. Priviledged container app launch is failing intermittently. Contributed by Eric Yang 
(cherry picked from commit 53e267fa7232add3c21174382d91b2607aa6becf)
 2018-07-02 16:23:23 -06:00
Eric Yang
0aa7272199 YARN-8465. Fixed docker container status for node manager restart. 
Contributed by Shane Kumpf
 2018-07-02 13:40:02 -04:00
Wei-Chiu Chuang
2ce892fcf5 HDFS-13635. Incorrect message when block is not found. Contributed by Gabor Bota. 
(cherry picked from commit fef20a446f7bf9f29e0f0ee690987fb6fc78a031)
 2018-07-02 09:43:44 -07:00
Andrew Wang
9f2d57697a HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon. 
(cherry picked from commit f51da9c4d1423c2ac92eb4f40e973264e7e968cc)
 2018-07-02 18:31:27 +02:00
Andrew Wang
2d2639beaa HDFS-13702. Remove HTrace hooks from DFSClient to reduce CPU usage. Contributed by Todd Lipcon. 
(cherry picked from commit 5d748bd056a32f2c6922514cd0c5b31d866a9919)
 2018-07-02 12:11:11 +02:00
Andrew Wang
0268b954e0 HDFS-13703. Avoid allocation of CorruptedBlocks hashmap when no corrupted blocks are hit. Contributed by Todd Lipcon. 
(cherry picked from commit 6ba99741086170b83c38d3e7e715d9e8046a1e00)
 2018-07-02 12:02:48 +02:00
Nanda kumar
e1cd5732a6 HADOOP-15574: Suppress build error if there are no docs after excluding private annotations. Contributed by Takanobu Asanuma. 
(cherry picked from commit d40121845e0e1ace6b349180ced53292353dbb7c)
 2018-07-01 17:16:48 +05:30
Eric E Payne
012baf4313 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 
(cherry picked from commit d36f6b9e93e4c30d24d0e837cb00bd24ffa8f274)
 2018-06-29 18:22:51 +00:00
Jason Lowe
dbb0a5f51f YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 
(cherry picked from commit 100470140d86eede0fa240a9aa93226f274ee4f5)
 2018-06-29 13:11:41 -05:00
Rohith Sharma K S
00406f583b YARN-8469. [UI2] URL needs to be trimmed to handle index.html redirection while accessing via knox. Contributed by Sunil Govindan. 
(cherry picked from commit a820738e347854aed7cdaa7758d9c0830df624e3)
 2018-06-29 10:39:30 -07:00
Sunil G
e7f2c9886a YARN-8455. Add basic ACL check for all ATS v2 REST APIs. Contributed by Rohith Sharma K S. 
(cherry picked from commit 469b29c0817b7bf1902c9195c4f8d031a909e1c9)
 2018-06-29 10:03:53 -07:00
Sunil G
122cb81b65 YARN-8379. Improve balancing resources in already satisfied queues by using Capacity Scheduler preemption. Contributed by Zian Chen. 
(cherry picked from commit 291194302cc1a875d6d94ea93cf1184a3f1fc2cc)
 2018-06-28 10:24:03 -07:00
Eric Yang
3ce3571a34 YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. 
Contributed by Chandni Singh
 2018-06-28 13:11:56 -04:00
Bharat Viswanadham
8fa530746f HDFS-13705:The native ISA-L library loading failure should be made warning rather than an error message. Contributed by Shashikant Banerjee. 
(cherry picked from commit d3fa83a44b01c85f39bfb4deaf2972912ac61ca3)
 2018-06-28 09:26:58 -07:00
Sunil G
3c68d4ce45 YARN-8401. [UI2] new ui is not accessible with out internet connection. Contributed by Bibin A Chundatt. 
(cherry picked from commit fbaff369e9b9022723a7b2c6f25e71122a8f8a15)
 2018-06-27 10:35:50 -07:00
Wangda Tan
ec37e02bc9 YARN-8464. Async scheduling thread could be interrupted when there are no NodeManagers in cluster. (Sunil G via wangda) 
Change-Id: I4f5f856373378685713e77752ba6cf0988a66065
(cherry picked from commit bedc4fe0799cf3b161100acc521fc62a97793427)
 2018-06-26 19:34:15 -07:00
Wangda Tan
8f226f4f1d YARN-8423. GPU does not get released even though the application gets killed. (Sunil G via wangda) 
Change-Id: I570db7d60f8c6c21762dd618a9207d1107c486a0
(cherry picked from commit ada8f63d0b3739d245300461387b0516dc92ccf9)
 2018-06-26 19:34:10 -07:00
Sunil G
aa8db00fc3 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-26 15:02:49 -07:00
Eric Yang
f6aa6a6981 YARN-8108. Added option to disable loading existing filters to prevent 
security filter from initialize twice.
            Contributed by Sunil Govindan

(cherry picked from commit b69ba0f3307a90500aeb0c5db9e582fcda60b501)
 2018-06-26 17:45:13 -04:00
Eric Yang
9e9530f505 YARN-8214. Change default RegistryDNS port. 
Contributed by Billie Rinaldi

(cherry picked from commit 3e586330eb2c7db0b884fe328b171fb27ce545fa)
 2018-06-26 14:26:26 -04:00
Rohith Sharma K S
e925de6483 YARN-8457. Compilation is broken with -Pyarn-ui. 
(cherry picked from commit 4ffe68a6f70ce01a5654da8991b4cdb35ae0bf1f)
 2018-06-25 10:39:26 -07:00
Inigo Goiri
cbe15066c8 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit abc3e4bad905efde5a4881e8a072c68f6e910ade)
 2018-06-25 09:51:01 -07:00
Weiwei Yang
bfdbc9dea2 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 
(Cherry picked from commit 440140cea6718229094a3d2b97b9b9bd28b95d9b)
 2018-06-25 09:28:57 +08:00
Eric Yang
3b3a85002d YARN-8326. Removed exit code file check for launched container. 
Contributed by Shane Kumpf

(cherry picked from commit 8a32bc39eb210fca8052c472601e24c2446b4cc2)
 2018-06-22 19:23:57 -04:00
Eric E Payne
1974dfa44d YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 
(cherry picked from commit 6432128622d64f3f9dd638b9c254c77cdf5408aa)
 2018-06-22 17:18:37 +00:00
Yiqun Lin
6b5b58a495 HDFS-13692. StorageInfoDefragmenter floods log when compacting StorageInfo TreeSet. Contributed by Bharat Viswanadham. 
(cherry picked from commit 30728aced4a6b05394b3fc8c613f39fade9cf3c2)
 2018-06-22 10:53:33 +08:00
Eric Yang
63ce762959 YARN-8445. Improved error message for duplicated service and component names. 
Contributed by Chandni Singh

(cherry picked from commit 9f15483c5d7c94251f4c84e0155449188f202779)
 2018-06-21 11:21:34 -04:00
Xiao Chen
e8655b8c72 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 
(cherry picked from commit 32f867a6a907c05a312657139d295a92756d98ef)
 2018-06-20 15:59:23 -07:00
Eric Yang
a31a966a20 HADOOP-15527. Improve delay check for stopping processes. 
Fixed script location.
               Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 2d87592fc6a56bfe77dd3c11953caea2b701c846)
 2018-06-19 13:40:10 -04:00
Eric Yang
68d179795f HADOOP-15527. Improve delay check for stopping processes. 
Contributed by Vinod Kumar Vavilapalli

(cherry picked from commit 2c87ec5affefeb1dc794c4eaae685a4e544f1841)
 2018-06-18 14:29:56 -04:00
Chris Douglas
7e655b642d HADOOP-15533. Make WASB listStatus messages consistent. Contributed by Esfandiar Manii 
(cherry picked from commit f34744603ee93e082e7ba148df1400af5ac7c30c)
 2018-06-17 23:12:59 -07:00
Surendra Singh Lilhore
40652e77e0 HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 
(cherry picked from commit 3905fdb793e6370243d05d0c3036ca69898fe3fb)
 2018-06-17 12:14:20 +05:30
Inigo Goiri
88e8a3e5bc HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:49:51 -07:00
Inigo Goiri
7c06be3912 HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 15:59:49 -07:00
Wei-Chiu Chuang
975d4b3d60 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
 2018-06-15 13:37:19 -07:00
Wangda Tan
332139e928 MAPREDUCE-7101. Add config parameter to allow JHS to alway scan user dir irrespective of modTime. (Thomas Marquardt via wangda) 
Change-Id: I8d8e71c44469fb48f1f5108620843e35e0100411
 2018-06-15 13:18:17 -07:00
Inigo Goiri
271ea7a3d9 HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit eebeb6033fd09791fcbff626f128a98e393f0a88)
 2018-06-15 13:08:24 -07:00