Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,954 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4782 Commits

Author SHA1 Message Date
Sunil G cadbc8b57f YARN-8561. [Submarine] Initial implementation: Training job submission and job history retrieval. Contributed by Wangda Tan. 2018-08-13 14:22:55 +05:30
Shane Kumpf e7951c69cb YARN-8520. Document best practice for user management. Contributed by Eric Yang 2018-08-10 14:32:03 -06:00
Weiwei Yang 0a71bf1452 YARN-8575. Avoid committing allocation proposal to unavailable nodes in async scheduling. Contributed by Tao Yang. 2018-08-10 14:37:45 +08:00
Weiwei Yang 08d5060605 YARN-8521. NPE in AllocationTagsManager when a container is removed more than once. Contributed by Weiwei Yang. 2018-08-10 08:32:02 +08:00
Robert Kanter b2517dd66b YARN-4946. RM should not consider an application as COMPLETED when log aggregation is not in a terminal state (snemeth via rkanter) 2018-08-09 14:58:04 -07:00
Wangda Tan 8244abb7ae YARN-8136. Add version attribute to site doc examples and quickstart. (Eric Yang via wangda) 
Change-Id: I4541b239f490ca0a6edf9698e0d3deaf83669151
 2018-08-09 11:04:02 -07:00
Wangda Tan 344c335a92 YARN-8588. Logging improvements for better debuggability. (Suma Shivaprasad via wangda) 
Change-Id: I66aa4b0ec031ae5ce0fae558e2f8cbcbbfebc442
 2018-08-09 11:04:02 -07:00
Weiwei Yang d352f167eb YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 2018-08-09 23:46:53 +08:00
Jason Lowe cd04e954d2 YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 2018-08-09 10:17:34 -05:00
Sunil G 00013d6ef7 YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-09 12:18:32 +05:30
Robert Kanter 8478732bb2 YARN-8568. Replace the deprecated zk-address property in the HA config example in ResourceManagerHA.md (bsteinbach via rkanter) 2018-08-08 15:08:55 -07:00
Wangda Tan 3214cd75ac Make 3.1.1 awared by other branches - adding missing files 2018-08-08 13:05:24 -07:00
Wangda Tan 49c687608b Make 3.1.1 awared by other branches 2018-08-08 13:02:12 -07:00
bibinchundatt ff06bd1be8 YARN-8601. Print ExecutionType in Container report CLI. Contributed by Bilwa S T. 2018-08-08 22:42:52 +05:30
Sunil G 5b898c176f Revert "YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB." 
This reverts commit 64901abdfa.
 2018-08-08 19:48:49 +05:30
Sunil G 64901abdfa YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-08 19:43:29 +05:30
Giovanni Matteo Fumarola d838179d8d YARN-8626. Create HomePolicyManager that sends all the requests to the home subcluster. Contributed by Inigo Goiri. 2018-08-07 15:33:16 -07:00
Wangda Tan 861095f761 YARN-8407. Container launch exception in AM log should be printed in ERROR level. (Yesha Vora via wangda) 
Change-Id: I154e873df1df3503a09c41d6b3874ca195af91d9
 2018-08-07 13:01:13 -07:00
Wangda Tan c0599151bb YARN-7089. Mark the log-aggregation-controller APIs as public. (Zian Chen via wangda) 
Change-Id: I37851bdc5935d623a27d0973a206c997258716eb
 2018-08-07 12:37:32 -07:00
Wangda Tan d4258fcad7 YARN-8629. Container cleanup fails while trying to delete Cgroups. (Suma Shivaprasad via wangda) 
Change-Id: I392ef4f8baa84d5d7b1f2e438c560b5426b6d4f2
 2018-08-07 12:36:55 -07:00
Eric Yang ca20e0d7e9 YARN-8624. Updated verbiage around entry point support. 
Contributed by Craig Condit
 2018-08-06 12:37:59 -04:00
Shane Kumpf bcfc9851f4 YARN-8287. Update documentation and yarn-default related to the Docker runtime. Contributed by Craig Condit 2018-08-03 18:20:49 -06:00
Sunil G 3426f4062a YARN-8615. [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu. Contributed by Akhil PB. 2018-08-03 19:28:16 +05:30
Sunil G 29417dbbf4 YARN-8620. [UI2] YARN Services UI new submission failures are not debuggable. Contributed by Akhil PB. 2018-08-03 19:12:34 +05:30
Rohith Sharma K S 022592ae79 YARN-8608. [UI2] No information available per application appAttempt about 'Total Outstanding Resource Requests'. Contributed by Akhil PB. 2018-08-03 15:30:39 +05:30
Rohith Sharma K S 33482d35e8 YARN-8603. [UI2] Latest run application should be listed first in the RM UI. Contributed by Akhil PB. 2018-08-03 15:30:39 +05:30
Haibo Chen 889df6f194 YARN-7948. Enable fair scheduler to refresh maximum allocation for multiple resource types. (Szilard Nemeth via Haibo Chen) 2018-08-02 10:10:12 -07:00
Sunil G 12a095a496 YARN-7159. Normalize unit of resource objects in RM to avoid unit conversion in critical path. Contributed by Manikandan R. 2018-08-02 22:29:21 +05:30
Jason Lowe 7526815e32 YARN-8263. DockerClient still touches hadoop.tmp.dir. Contributed by Craig Condit 2018-08-02 10:43:48 -05:00
Sunil G 5033d7da8f YARN-8318. [UI2] IP address in component page shows N/A. Contributed by Yesha Vora. 2018-08-02 20:09:24 +05:30
Sunil G 97870ec1f6 YARN-8592. [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome browser. Contributed by Akhil PB. 2018-08-02 16:10:54 +05:30
Sunil G 1ea81169ba YARN-8594. [UI2] Display current logged in user. Contributed by Akhil PB. 2018-08-02 12:41:06 +05:30
Sunil G 735b492556 YARN-8593. Add RM web service endpoint to get user information. Contributed by Akhil PB. 2018-08-02 08:34:09 +05:30
Eric Yang 23f394240e YARN-8610. Fixed initiate upgrade error message. 
Contributed by Chandni Singh
 2018-08-01 20:41:43 -04:00
Shane Kumpf 603a57476c YARN-8600. RegistryDNS hang when remote lookup does not reply. Contributed by Eric Yang 2018-08-01 12:22:01 -06:00
Billie Rinaldi 67c65da261 YARN-8403. Change the log level for fail to download resource from INFO to ERROR. Contributed by Eric Yang 2018-08-01 08:51:40 -07:00
Sunil G d920b9db77 YARN-8595. [UI2] Container diagnostic information is missing from container page. Contributed by Akhil PB. 2018-08-01 14:27:54 +05:30
Sunil G a48a0cc7fd YARN-8606. Opportunistic scheduling does not work post RM failover. Contributed by Bibin A Chundatt. 2018-08-01 12:17:18 +05:30
Wangda Tan 5cc8e99147 YARN-8522. Application fails with InvalidResourceRequestException. (Zian Chen via wangda) 
Change-Id: I34dd7fa49bd4d10580c4a78051033b1068d28f1e
 2018-07-31 22:34:53 -07:00
Sunil G 6310c0d17d YARN-8397. Potential thread leak in ActivitiesManager. Contributed by Rohith Sharma K S. 2018-08-01 08:33:30 +05:30
Eric Yang c7ebcd76bf YARN-8579. Recover NMToken of previous attempted component data. 
Contributed by Gour Saha
 2018-07-31 18:01:02 -04:00
Wangda Tan 4b540bbfcf YARN-8418. App local logs could leaked if log aggregation fails to initialize for the app. (Bibin A Chundatt via wangda) 
Change-Id: I29a23ca4b219b48c92e7975cd44cddb8b0e04104
 2018-07-31 12:08:00 -07:00
Haibo Chen 8aa93a575e YARN-8605. TestDominantResourceFairnessPolicy.testModWhileSorting is flaky. (Wilfred Spiegelenburg via Haibo Chen) 2018-07-31 11:32:40 -07:00
Inigo Goiri 9fea5c9ee7 YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek Modi. 2018-07-31 09:36:34 -07:00
Jonathan Hung 3e06a5dcea YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 17:44:18 -07:00
bibinchundatt 2b39ad2698 YARN-8584. Several typos in Log Aggregation related classes. Contributed by Szilard Nemeth. 2018-07-30 23:25:19 +05:30
Sunil G 63e08ec071 YARN-8591. [ATSv2] NPE while checking for entity acl in non-secure cluster. Contributed by Rohith Sharma K S. 2018-07-30 14:48:04 +05:30
bibinchundatt 3d586841ab YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2018-07-28 20:52:39 +05:30
Eric Yang ed9d60e888 YARN-8508. Release GPU resource for killed container. 
Contributed by Chandni Singh
 2018-07-27 19:33:58 -04:00
Inigo Goiri 79091cf76f YARN-8596. Allow SQLFederationStateStore to submit the same app in the same subcluster. Contributed by Giovanni Matteo Fumarola. 2018-07-27 15:23:57 -07:00
Robert Kanter 2cccf4061c YARN-8517. getContainer and getContainers ResourceManager REST API methods are not documented (snemeth via rkanter) 2018-07-27 14:35:03 -07:00
Robert Kanter fecbac499e YARN-8566. Add diagnostic message for unschedulable containers (snemeth via rkanter) 2018-07-27 14:32:34 -07:00
Billie Rinaldi b429f19d32 YARN-8571. Validate service principal format prior to launching yarn service. Contributed by Eric Yang 2018-07-27 11:30:19 -07:00
Eric Yang 8d3c068e59 YARN-8429. Improve diagnostic message when artifact is not set properly. 
Contributed by Gour Saha
 2018-07-26 20:02:13 -04:00
Eric Yang 40fad32824 YARN-8545. Return allocated resource to RM for failed container. 
Contributed by Chandni Singh
 2018-07-26 18:22:57 -04:00
Eric Yang f93ecf5c1e YARN-8330. Improved publishing ALLOCATED events to ATS. 
Contributed by Suma Shivaprasad
 2018-07-25 18:49:30 -04:00
Eric E Payne 9485c9aee6 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 2018-07-25 16:22:04 +00:00
Weiwei Yang 5be9f4a5d0 YARN-8546. Resource leak caused by a reserved container being released more than once under async scheduling. Contributed by Tao Yang. 2018-07-25 17:35:27 +08:00
Haibo Chen ea2c6c8c9a YARN-7133. Clean up lock-try order in fair scheduler. (Szilard Nemeth via Haibo Chen) 2018-07-24 12:46:59 -07:00
bibinchundatt e673dd1d4d YARN-8541. RM startup failure on recovery after user deletion. Contributed by Bibin A Chundatt. 2018-07-24 23:26:59 +05:30
Sunil G 35ce6eb1f5 YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 2018-07-24 22:20:17 +05:30
bibinchundatt ff7c2eda34 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 2018-07-24 16:17:20 +05:30
bibinchundatt 8461278833 YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 2018-07-24 13:09:17 +05:30
Eric Yang 8688a0c7f8 YARN-8380. Support bind propagation options for mounts in docker runtime. 
Contributed by Billie Rinaldi
 2018-07-23 20:12:04 -04:00
Haibo Chen 9d3c39e9dd YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 2018-07-23 11:07:24 -07:00
Eric Yang 84d7bf1eef YARN-8360. Improve YARN service restart policy and node manager auto restart policy. 
Contributed by Suma Shivaprasad
 2018-07-23 12:57:01 -04:00
Eric Yang 1622a4b810 YARN-8301. Added YARN service upgrade instructions. 
Contributed by Chandni Singh
 2018-07-20 19:46:35 -04:00
Haibo Chen 8a6bb8409c YARN-6964. Fair scheduler misuses Resources operations. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-20 10:46:28 -07:00
Weiwei Yang cbf2026483 YARN-8528. Final states in ContainerAllocation might be modified externally causing unexpected allocation results. Contributed by Xintong Song. 2018-07-20 22:34:06 +08:00
Haibo Chen e6873dfde0 YARN-7300. DiskValidator is not used in LocalDirAllocator. (Szilard Nemeth via Haibo Chen) 2018-07-19 16:27:11 -07:00
Haibo Chen f354f47f99 YARN-6995. Improve use of ResourceNotFoundException in resource types code. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-19 15:35:05 -07:00
Haibo Chen 2564884757 YARN-8436. FSParentQueue: Comparison method violates its general contract. (Wilfred Spiegelenburg via Haibo Chen) 2018-07-19 13:22:31 -07:00
Eric Yang 5836e0a46b YARN-8501. Reduce complexity of RMWebServices getApps method. 
Contributed by Szilard Nemeth
 2018-07-19 12:30:38 -04:00
Eric Yang 121865c3f9 YARN-8299. Added CLI and REST API for query container status. 
Contributed by Chandni Singh
 2018-07-16 17:41:23 -04:00
Eric Yang efb4e274e5 YARN-8538. Fixed memory leaks in container-executor and test cases. 
Contributed by Billie Rinaldi
 2018-07-16 17:38:49 -04:00
Wangda Tan 238ffff999 YARN-8524. Single parameter Resource / LightWeightResource constructor looks confusing. (Szilard Nemeth via wangda) 
Change-Id: I4ae97548b5b8d76a6bcebb2d3d70bf8e0be3c125
 2018-07-16 10:58:00 -07:00
Wangda Tan a2e49f41a8 YARN-8361. Change App Name Placement Rule to use App Name instead of App Id for configuration. (Zian Chen via wangda) 
Change-Id: I17e5021f8f611a9c5e3bd4b38f25e08585afc6b1
 2018-07-16 10:57:37 -07:00
Wangda Tan 752dcce5f4 YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda) 
Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9
 2018-07-16 10:54:41 -07:00
Eric E Payne 937ef39b3f YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 2018-07-16 16:24:21 +00:00
bibinchundatt 4523cc5637 YARN-8434. Update federation documentation of Nodemanager configurations. Contributed by Bibin A Chundatt. 2018-07-15 13:53:53 +05:30
Akira Ajisaka 88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Jason Lowe 17118f446c YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 2018-07-13 10:05:25 -05:00
Robert Kanter 1bc106a738 YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 2018-07-12 16:38:46 -07:00
bibinchundatt 52e1bc8539 YARN-8491. TestServiceCLI#testEnableFastLaunch fail when umask is 077. Contributed by K G Bakthavachalam. 2018-07-11 16:20:29 +05:30
Sunil G 7f1d3d0e9d YARN-8512. ATSv2 entities are not published to HBase from second attempt onwards. Contributed by Rohith Sharma K S. 2018-07-11 12:26:32 +05:30
Giovanni Matteo Fumarola 82ac3aa6d0 YARN-8502. Use path strings consistently for webservice endpoints in RMWebServices. Contributed by Szilard Nemeth. 2018-07-10 10:36:17 -07:00
Sunil G 705e2c1f7c YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 2018-07-10 20:11:47 +05:30
Giovanni Matteo Fumarola ea9b608237 YARN-7899. [AMRMProxy] Stateful FederationInterceptor for pending requests. Contributed by Botong Huang. 2018-07-09 12:27:36 -07:00
Wangda Tan 83cd84b70b YARN-8506. Make GetApplicationsRequestPBImpl thread safe. (wangda) 
Change-Id: If304567abb77a01b686d82c769bdf50728484163
 2018-07-09 11:30:08 -07:00
Robert Kanter 0838fe8337 Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 2018-07-09 10:37:20 -07:00
Rohith Sharma K S ba68320449 YARN-8302. ATS v2 should handle HBase connection issue properly. Contributed by Billie Rinaldi. 2018-07-06 15:19:01 -07:00
Sunil G e4bf38cf50 YARN-8492. ATSv2 HBase tests are failing with ClassNotFoundException. Contributed by Rohith Sharma K S. 2018-07-06 12:05:32 -07:00
Xiaoyu Yao 498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Robert Kanter a129e3e74e YARN-7451. Add missing tests to verify the presence of custom resources of RM apps and scheduler webservice endpoints (snemeth via rkanter) 2018-07-06 11:04:00 -07:00
Haibo Chen 9edc74f64a YARN-7556. Fair scheduler configuration should allow resource types in the minResources and maxResources properties. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-06 11:03:48 -07:00
Anu Engineer 39ad98903a Revert "Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk" 
This reverts commit c163d1797a, reversing
changes made to 0d9804dcef.
 2018-07-05 12:22:18 -07:00
Giovanni Matteo Fumarola c163d1797a Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk 2018-07-05 10:55:05 -07:00
Giovanni Matteo Fumarola 0d9804dcef YARN-8435. Fix NPE when the same client simultaneously contact for the first time Yarn Router. Contributed by Rang Jiaheng. 2018-07-05 10:54:31 -07:00
Robert Kanter 99febe7fd5 YARN-7451. Add missing tests to verify the presence of custom resources of RM apps and scheduler webservice endpoints (snemeth via rkanter) 2018-07-05 10:54:19 -07:00
Haibo Chen 1726247024 YARN-7556. Fair scheduler configuration should allow resource types in the minResources and maxResources properties. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-05 10:42:39 -07:00