Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,305 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

5085 Commits

Author SHA1 Message Date
Robert Kanter 8478732bb2 YARN-8568. Replace the deprecated zk-address property in the HA config example in ResourceManagerHA.md (bsteinbach via rkanter) 2018-08-08 15:08:55 -07:00
Wangda Tan 3214cd75ac Make 3.1.1 awared by other branches - adding missing files 2018-08-08 13:05:24 -07:00
Wangda Tan 49c687608b Make 3.1.1 awared by other branches 2018-08-08 13:02:12 -07:00
bibinchundatt ff06bd1be8 YARN-8601. Print ExecutionType in Container report CLI. Contributed by Bilwa S T. 2018-08-08 22:42:52 +05:30
Sunil G 5b898c176f Revert "YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB." 
This reverts commit 64901abdfa.
 2018-08-08 19:48:49 +05:30
Sunil G 64901abdfa YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-08 19:43:29 +05:30
Giovanni Matteo Fumarola d838179d8d YARN-8626. Create HomePolicyManager that sends all the requests to the home subcluster. Contributed by Inigo Goiri. 2018-08-07 15:33:16 -07:00
Wangda Tan 861095f761 YARN-8407. Container launch exception in AM log should be printed in ERROR level. (Yesha Vora via wangda) 
Change-Id: I154e873df1df3503a09c41d6b3874ca195af91d9
 2018-08-07 13:01:13 -07:00
Wangda Tan c0599151bb YARN-7089. Mark the log-aggregation-controller APIs as public. (Zian Chen via wangda) 
Change-Id: I37851bdc5935d623a27d0973a206c997258716eb
 2018-08-07 12:37:32 -07:00
Wangda Tan d4258fcad7 YARN-8629. Container cleanup fails while trying to delete Cgroups. (Suma Shivaprasad via wangda) 
Change-Id: I392ef4f8baa84d5d7b1f2e438c560b5426b6d4f2
 2018-08-07 12:36:55 -07:00
Eric Yang ca20e0d7e9 YARN-8624. Updated verbiage around entry point support. 
Contributed by Craig Condit
 2018-08-06 12:37:59 -04:00
Shane Kumpf bcfc9851f4 YARN-8287. Update documentation and yarn-default related to the Docker runtime. Contributed by Craig Condit 2018-08-03 18:20:49 -06:00
Sunil G 3426f4062a YARN-8615. [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu. Contributed by Akhil PB. 2018-08-03 19:28:16 +05:30
Sunil G 29417dbbf4 YARN-8620. [UI2] YARN Services UI new submission failures are not debuggable. Contributed by Akhil PB. 2018-08-03 19:12:34 +05:30
Rohith Sharma K S 022592ae79 YARN-8608. [UI2] No information available per application appAttempt about 'Total Outstanding Resource Requests'. Contributed by Akhil PB. 2018-08-03 15:30:39 +05:30
Rohith Sharma K S 33482d35e8 YARN-8603. [UI2] Latest run application should be listed first in the RM UI. Contributed by Akhil PB. 2018-08-03 15:30:39 +05:30
Haibo Chen 889df6f194 YARN-7948. Enable fair scheduler to refresh maximum allocation for multiple resource types. (Szilard Nemeth via Haibo Chen) 2018-08-02 10:10:12 -07:00
Sunil G 12a095a496 YARN-7159. Normalize unit of resource objects in RM to avoid unit conversion in critical path. Contributed by Manikandan R. 2018-08-02 22:29:21 +05:30
Jason Lowe 7526815e32 YARN-8263. DockerClient still touches hadoop.tmp.dir. Contributed by Craig Condit 2018-08-02 10:43:48 -05:00
Sunil G 5033d7da8f YARN-8318. [UI2] IP address in component page shows N/A. Contributed by Yesha Vora. 2018-08-02 20:09:24 +05:30
Sunil G 97870ec1f6 YARN-8592. [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS and Chrome browser. Contributed by Akhil PB. 2018-08-02 16:10:54 +05:30
Sunil G 1ea81169ba YARN-8594. [UI2] Display current logged in user. Contributed by Akhil PB. 2018-08-02 12:41:06 +05:30
Sunil G 735b492556 YARN-8593. Add RM web service endpoint to get user information. Contributed by Akhil PB. 2018-08-02 08:34:09 +05:30
Eric Yang 23f394240e YARN-8610. Fixed initiate upgrade error message. 
Contributed by Chandni Singh
 2018-08-01 20:41:43 -04:00
Shane Kumpf 603a57476c YARN-8600. RegistryDNS hang when remote lookup does not reply. Contributed by Eric Yang 2018-08-01 12:22:01 -06:00
Billie Rinaldi 67c65da261 YARN-8403. Change the log level for fail to download resource from INFO to ERROR. Contributed by Eric Yang 2018-08-01 08:51:40 -07:00
Sunil G d920b9db77 YARN-8595. [UI2] Container diagnostic information is missing from container page. Contributed by Akhil PB. 2018-08-01 14:27:54 +05:30
Sunil G a48a0cc7fd YARN-8606. Opportunistic scheduling does not work post RM failover. Contributed by Bibin A Chundatt. 2018-08-01 12:17:18 +05:30
Wangda Tan 5cc8e99147 YARN-8522. Application fails with InvalidResourceRequestException. (Zian Chen via wangda) 
Change-Id: I34dd7fa49bd4d10580c4a78051033b1068d28f1e
 2018-07-31 22:34:53 -07:00
Sunil G 6310c0d17d YARN-8397. Potential thread leak in ActivitiesManager. Contributed by Rohith Sharma K S. 2018-08-01 08:33:30 +05:30
Eric Yang c7ebcd76bf YARN-8579. Recover NMToken of previous attempted component data. 
Contributed by Gour Saha
 2018-07-31 18:01:02 -04:00
Wangda Tan 4b540bbfcf YARN-8418. App local logs could leaked if log aggregation fails to initialize for the app. (Bibin A Chundatt via wangda) 
Change-Id: I29a23ca4b219b48c92e7975cd44cddb8b0e04104
 2018-07-31 12:08:00 -07:00
Haibo Chen 8aa93a575e YARN-8605. TestDominantResourceFairnessPolicy.testModWhileSorting is flaky. (Wilfred Spiegelenburg via Haibo Chen) 2018-07-31 11:32:40 -07:00
Inigo Goiri 9fea5c9ee7 YARN-8175. Add support for Node Labels in SLS. Contributed by Abhishek Modi. 2018-07-31 09:36:34 -07:00
Jonathan Hung 3e06a5dcea YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 17:44:18 -07:00
bibinchundatt 2b39ad2698 YARN-8584. Several typos in Log Aggregation related classes. Contributed by Szilard Nemeth. 2018-07-30 23:25:19 +05:30
Sunil G 63e08ec071 YARN-8591. [ATSv2] NPE while checking for entity acl in non-secure cluster. Contributed by Rohith Sharma K S. 2018-07-30 14:48:04 +05:30
bibinchundatt 3d586841ab YARN-8558. NM recovery level db not cleaned up properly on container finish. Contributed by Bibin A Chundatt. 2018-07-28 20:52:39 +05:30
Eric Yang ed9d60e888 YARN-8508. Release GPU resource for killed container. 
Contributed by Chandni Singh
 2018-07-27 19:33:58 -04:00
Inigo Goiri 79091cf76f YARN-8596. Allow SQLFederationStateStore to submit the same app in the same subcluster. Contributed by Giovanni Matteo Fumarola. 2018-07-27 15:23:57 -07:00
Robert Kanter 2cccf4061c YARN-8517. getContainer and getContainers ResourceManager REST API methods are not documented (snemeth via rkanter) 2018-07-27 14:35:03 -07:00
Robert Kanter fecbac499e YARN-8566. Add diagnostic message for unschedulable containers (snemeth via rkanter) 2018-07-27 14:32:34 -07:00
Billie Rinaldi b429f19d32 YARN-8571. Validate service principal format prior to launching yarn service. Contributed by Eric Yang 2018-07-27 11:30:19 -07:00
Eric Yang 8d3c068e59 YARN-8429. Improve diagnostic message when artifact is not set properly. 
Contributed by Gour Saha
 2018-07-26 20:02:13 -04:00
Eric Yang 40fad32824 YARN-8545. Return allocated resource to RM for failed container. 
Contributed by Chandni Singh
 2018-07-26 18:22:57 -04:00
Eric Yang f93ecf5c1e YARN-8330. Improved publishing ALLOCATED events to ATS. 
Contributed by Suma Shivaprasad
 2018-07-25 18:49:30 -04:00
Eric E Payne 9485c9aee6 YARN-4606. CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps. Contributed by Manikandan R 2018-07-25 16:22:04 +00:00
Weiwei Yang 5be9f4a5d0 YARN-8546. Resource leak caused by a reserved container being released more than once under async scheduling. Contributed by Tao Yang. 2018-07-25 17:35:27 +08:00
Haibo Chen ea2c6c8c9a YARN-7133. Clean up lock-try order in fair scheduler. (Szilard Nemeth via Haibo Chen) 2018-07-24 12:46:59 -07:00
bibinchundatt e673dd1d4d YARN-8541. RM startup failure on recovery after user deletion. Contributed by Bibin A Chundatt. 2018-07-24 23:26:59 +05:30
Sunil G 35ce6eb1f5 YARN-7748. TestContainerResizing.testIncreaseContainerUnreservedWhenApplicationCompleted fails due to multiple container fail events. Contributed by Weiwei Yang. 2018-07-24 22:20:17 +05:30
bibinchundatt ff7c2eda34 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 2018-07-24 16:17:20 +05:30
bibinchundatt 8461278833 YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 2018-07-24 13:09:17 +05:30
Eric Yang 8688a0c7f8 YARN-8380. Support bind propagation options for mounts in docker runtime. 
Contributed by Billie Rinaldi
 2018-07-23 20:12:04 -04:00
Haibo Chen 9d3c39e9dd YARN-6966. NodeManager metrics may return wrong negative values when NM restart. (Szilard Nemeth via Haibo Chen) 2018-07-23 11:07:24 -07:00
Eric Yang 84d7bf1eef YARN-8360. Improve YARN service restart policy and node manager auto restart policy. 
Contributed by Suma Shivaprasad
 2018-07-23 12:57:01 -04:00
Eric Yang 1622a4b810 YARN-8301. Added YARN service upgrade instructions. 
Contributed by Chandni Singh
 2018-07-20 19:46:35 -04:00
Haibo Chen 8a6bb8409c YARN-6964. Fair scheduler misuses Resources operations. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-20 10:46:28 -07:00
Weiwei Yang cbf2026483 YARN-8528. Final states in ContainerAllocation might be modified externally causing unexpected allocation results. Contributed by Xintong Song. 2018-07-20 22:34:06 +08:00
Haibo Chen e6873dfde0 YARN-7300. DiskValidator is not used in LocalDirAllocator. (Szilard Nemeth via Haibo Chen) 2018-07-19 16:27:11 -07:00
Haibo Chen f354f47f99 YARN-6995. Improve use of ResourceNotFoundException in resource types code. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-19 15:35:05 -07:00
Haibo Chen 2564884757 YARN-8436. FSParentQueue: Comparison method violates its general contract. (Wilfred Spiegelenburg via Haibo Chen) 2018-07-19 13:22:31 -07:00
Eric Yang 5836e0a46b YARN-8501. Reduce complexity of RMWebServices getApps method. 
Contributed by Szilard Nemeth
 2018-07-19 12:30:38 -04:00
Eric Yang 121865c3f9 YARN-8299. Added CLI and REST API for query container status. 
Contributed by Chandni Singh
 2018-07-16 17:41:23 -04:00
Eric Yang efb4e274e5 YARN-8538. Fixed memory leaks in container-executor and test cases. 
Contributed by Billie Rinaldi
 2018-07-16 17:38:49 -04:00
Wangda Tan 238ffff999 YARN-8524. Single parameter Resource / LightWeightResource constructor looks confusing. (Szilard Nemeth via wangda) 
Change-Id: I4ae97548b5b8d76a6bcebb2d3d70bf8e0be3c125
 2018-07-16 10:58:00 -07:00
Wangda Tan a2e49f41a8 YARN-8361. Change App Name Placement Rule to use App Name instead of App Id for configuration. (Zian Chen via wangda) 
Change-Id: I17e5021f8f611a9c5e3bd4b38f25e08585afc6b1
 2018-07-16 10:57:37 -07:00
Wangda Tan 752dcce5f4 YARN-8511. When AM releases a container, RM removes allocation tags before it is released by NM. (Weiwei Yang via wangda) 
Change-Id: I6f9f409f2ef685b405cbff547dea9623bf3322d9
 2018-07-16 10:54:41 -07:00
Eric E Payne 937ef39b3f YARN-8421: when moving app, activeUsers is increased, even though app does not have outstanding request. Contributed by Kyungwan Nam 2018-07-16 16:24:21 +00:00
bibinchundatt 4523cc5637 YARN-8434. Update federation documentation of Nodemanager configurations. Contributed by Bibin A Chundatt. 2018-07-15 13:53:53 +05:30
Akira Ajisaka 88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Jason Lowe 17118f446c YARN-8515. container-executor can crash with SIGPIPE after nodemanager restart. Contributed by Jim Brennan 2018-07-13 10:05:25 -05:00
Robert Kanter 1bc106a738 YARN-8518. test-container-executor test_is_empty() is broken (Jim_Brennan via rkanter) 2018-07-12 16:38:46 -07:00
bibinchundatt 52e1bc8539 YARN-8491. TestServiceCLI#testEnableFastLaunch fail when umask is 077. Contributed by K G Bakthavachalam. 2018-07-11 16:20:29 +05:30
Sunil G 7f1d3d0e9d YARN-8512. ATSv2 entities are not published to HBase from second attempt onwards. Contributed by Rohith Sharma K S. 2018-07-11 12:26:32 +05:30
Giovanni Matteo Fumarola 82ac3aa6d0 YARN-8502. Use path strings consistently for webservice endpoints in RMWebServices. Contributed by Szilard Nemeth. 2018-07-10 10:36:17 -07:00
Sunil G 705e2c1f7c YARN-8473. Containers being launched as app tears down can leave containers in NEW state. Contributed by Jason Lowe. 2018-07-10 20:11:47 +05:30
Giovanni Matteo Fumarola ea9b608237 YARN-7899. [AMRMProxy] Stateful FederationInterceptor for pending requests. Contributed by Botong Huang. 2018-07-09 12:27:36 -07:00
Wangda Tan 83cd84b70b YARN-8506. Make GetApplicationsRequestPBImpl thread safe. (wangda) 
Change-Id: If304567abb77a01b686d82c769bdf50728484163
 2018-07-09 11:30:08 -07:00
Robert Kanter 0838fe8337 Only mount non-empty directories for cgroups (miklos.szegedi@cloudera.com via rkanter) 2018-07-09 10:37:20 -07:00
Rohith Sharma K S ba68320449 YARN-8302. ATS v2 should handle HBase connection issue properly. Contributed by Billie Rinaldi. 2018-07-06 15:19:01 -07:00
Sunil G e4bf38cf50 YARN-8492. ATSv2 HBase tests are failing with ClassNotFoundException. Contributed by Rohith Sharma K S. 2018-07-06 12:05:32 -07:00
Xiaoyu Yao 498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Robert Kanter a129e3e74e YARN-7451. Add missing tests to verify the presence of custom resources of RM apps and scheduler webservice endpoints (snemeth via rkanter) 2018-07-06 11:04:00 -07:00
Haibo Chen 9edc74f64a YARN-7556. Fair scheduler configuration should allow resource types in the minResources and maxResources properties. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-06 11:03:48 -07:00
Anu Engineer 39ad98903a Revert "Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk" 
This reverts commit c163d1797a, reversing
changes made to 0d9804dcef.
 2018-07-05 12:22:18 -07:00
Giovanni Matteo Fumarola c163d1797a Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk 2018-07-05 10:55:05 -07:00
Giovanni Matteo Fumarola 0d9804dcef YARN-8435. Fix NPE when the same client simultaneously contact for the first time Yarn Router. Contributed by Rang Jiaheng. 2018-07-05 10:54:31 -07:00
Robert Kanter 99febe7fd5 YARN-7451. Add missing tests to verify the presence of custom resources of RM apps and scheduler webservice endpoints (snemeth via rkanter) 2018-07-05 10:54:19 -07:00
Haibo Chen 1726247024 YARN-7556. Fair scheduler configuration should allow resource types in the minResources and maxResources properties. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-05 10:42:39 -07:00
Sunil G 51654a3962 YARN-8459. Improve Capacity Scheduler logs to debug invalid states. Contributed by Wangda Tan. 2018-07-03 09:59:20 -07:00
Junping Du d9ba6f3656 yarn.resourcemanager.fail-fast is used inconsistently. Contributed by Yuanbo Liu. 2018-07-03 14:46:44 +08:00
Sunil G fa9ef15ecd YARN-8415. TimelineWebServices.getEntity should throw ForbiddenException instead of 404 when ACL checks fail. Contributed by Suma Shivaprasad. 2018-07-02 15:34:47 -07:00
Shane Kumpf 53e267fa72 YARN-8485. Priviledged container app launch is failing intermittently. Contributed by Eric Yang 2018-07-02 16:18:32 -06:00
Eric Yang 5cc2541a16 YARN-8465. Fixed docker container status for node manager restart. 
Contributed by Shane Kumpf
 2018-07-02 13:37:51 -04:00
Giovanni Matteo Fumarola cdb084426b YARN-8481. AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters. Contributed by Botong Huang. 2018-06-29 11:47:30 -07:00
Jason Lowe 100470140d YARN-8451. Multiple NM heartbeat thread created when a slow NM resync with RM. Contributed by Botong Huang 2018-06-29 13:06:28 -05:00
Rohith Sharma K S a820738e34 YARN-8469. [UI2] URL needs to be trimmed to handle index.html redirection while accessing via knox. Contributed by Sunil Govindan. 2018-06-29 10:21:25 -07:00
Sunil G 469b29c081 YARN-8455. Add basic ACL check for all ATS v2 REST APIs. Contributed by Rohith Sharma K S. 2018-06-29 10:02:53 -07:00
Sunil G 291194302c YARN-8379. Improve balancing resources in already satisfied queues by using Capacity Scheduler preemption. Contributed by Zian Chen. 2018-06-28 10:23:31 -07:00
Eric Yang 384764cdea YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService. 
Contributed by Chandni Singh
 2018-06-28 12:39:49 -04:00
Sunil G fbaff369e9 YARN-8401. [UI2] new ui is not accessible with out internet connection. Contributed by Bibin A Chundatt. 2018-06-27 10:35:15 -07:00
Wangda Tan bedc4fe079 YARN-8464. Async scheduling thread could be interrupted when there are no NodeManagers in cluster. (Sunil G via wangda) 
Change-Id: I4f5f856373378685713e77752ba6cf0988a66065
 2018-06-26 19:27:17 -07:00
Wangda Tan ada8f63d0b YARN-8423. GPU does not get released even though the application gets killed. (Sunil G via wangda) 
Change-Id: I570db7d60f8c6c21762dd618a9207d1107c486a0
 2018-06-26 19:25:57 -07:00
Miklos Szegedi 62d83ca536 YARN-8461. Support strict memory control on individual container with elastic control memory mechanism. Contributed by Haibo Chen. 2018-06-26 15:21:35 -07:00
Eric Yang b69ba0f330 YARN-8108. Added option to disable loading existing filters to prevent 
security filter from initialize twice.
            Contributed by Sunil Govindan
 2018-06-26 17:34:57 -04:00
Eric Yang 3e586330eb YARN-8214. Change default RegistryDNS port. 
Contributed by Billie Rinaldi
 2018-06-26 14:25:15 -04:00
Miklos Szegedi 35ec9401e8 YARN-8438. TestContainer.testKillOnNew flaky on trunk. Contributed by Szilard Nemeth. 2018-06-25 15:47:54 -07:00
Rohith Sharma K S 4ffe68a6f7 YARN-8457. Compilation is broken with -Pyarn-ui. 2018-06-25 10:38:03 -07:00
Weiwei Yang 440140cea6 YARN-8443. Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers. Contributed by Tao Yang. 2018-06-25 09:15:31 +08:00
Eric Yang 8a32bc39eb YARN-8326. Removed exit code file check for launched container. 
Contributed by Shane Kumpf
 2018-06-22 19:12:48 -04:00
Yufei Gu 1cdce86d33 YARN-8184. Too many metrics if containerLocalizer/ResourceLocalizationService uses ReadWriteDiskValidator. Contributed by Yufei Gu 2018-06-22 14:03:55 -07:00
Inigo Goiri 99948565cb YARN-8412. Move ResourceRequest.clone logic everywhere into a proper API. Contributed by Botong Huang. 2018-06-21 18:24:10 -07:00
Eric Yang 9f15483c5d YARN-8445. Improved error message for duplicated service and component names. 
Contributed by Chandni Singh
 2018-06-21 11:20:24 -04:00
Miklos Szegedi 46f9058164 YARN-8441. Typo in CSQueueUtils local variable names: queueGuranteedResource. Contributed by Szilard Nemeth. 2018-06-20 11:58:18 -07:00
Miklos Szegedi 388fafa004 YARN-8442. Strange characters and missing spaces in FairScheduler documentation. Contributed by Szilard Nemeth. 2018-06-20 11:55:43 -07:00
Miklos Szegedi bbbc7cc426 YARN-7449. Split up class TestYarnClient to TestYarnClient and TestYarnClientImpl. Contributed by Szilard Nemeth. 2018-06-20 11:40:56 -07:00
Miklos Szegedi 55432b0981 YARN-8440. Typo in YarnConfiguration javadoc: "Miniumum request grant-able..". Contributed by Szilard Nemeth. 2018-06-20 11:36:12 -07:00
Miklos Szegedi 9a9e969570 YARN-8391. Investigate AllocationFileLoaderService.reloadListener locking issue. Contributed by Szilard Nemeth. 2018-06-20 10:59:33 -07:00
Haibo Chen 4939ffedb1 YARN-8437. Build oom-listener fails on older versions. (Miklos Szegedi via Haibo Chen) 2018-06-20 10:42:13 -07:00
Dedunu Dhananjaya f386e78a4b
YARN-7668. Remove unused variables from ContainerLocalizer 
This closes #364

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-06-18 16:45:50 -07:00
Inigo Goiri b272b71095 HADOOP-15537. Clean up ContainerLaunch and ContainerExecutor pre-HADOOP-15528. Contributed by Giovanni Matteo Fumarola. 2018-06-15 09:40:45 -07:00
Bharat Viswanadham 361ffb26be YARN-8426:Upgrade jquery-ui to 1.12.1 in YARN. Contributed by Sunil Govindan 2018-06-14 13:14:25 -07:00
Eric Yang 9591765040 YARN-8410. Fixed a bug in A record lookup by CNAME record. 
Contributed by Shane Kumpf
 2018-06-14 15:54:21 -04:00
Rohith Sharma K S 9119b3cf8f YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 2018-06-14 12:38:10 +05:30
Eric Yang 22994889dc YARN-8259. Improve privileged docker container liveliness checks. 
Contributed by Shane Kumpf
 2018-06-13 19:24:31 -04:00
Eric Yang 69b0596897 YARN-8411. Restart stopped system service during RM start. 
Contributed by Billie Rinaldi
 2018-06-13 19:05:52 -04:00
Sunil G 6307962b93 YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 2018-06-13 16:09:16 +05:30
Jitendra Pandey f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Weiwei Yang 29024a6203 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 2018-06-13 09:28:05 +08:00
Inigo Goiri 6e756e8a62 HADOOP-15529. ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows. Contributed by Giovanni Matteo Fumarola. 2018-06-12 10:24:34 -07:00
Haibo Chen 10d0e4be6e YARN-8325. Miscellaneous QueueManager code clean up. (Szilard Nemeth via Haibo Chen) 2018-06-12 10:11:30 -07:00
Haibo Chen 24a89825f0 YARN-6931. Make the aggregation interval in AppLevelTimelineCollector configurable. (Abhishek Modi via Haibo Chen) 2018-06-12 10:03:34 -07:00
Akira Ajisaka 652bcbb3e4
YARN-8363. Upgrade commons-lang version to 3.7 in hadoop-yarn-project. Contributed by Takanobu Asanuma. 2018-06-13 00:35:37 +09:00
Sunil G b3612dd90c YARN-8419. [UI2] User cannot submit a new service as submit button is always disabled. Contributed by Suma Shivaprasad. 2018-06-12 17:40:41 +05:30
Rohith Sharma K S 95303f50d3 YARN-8413. Flow activity page is failing with 'Timeline server failed with an error'. Contributed by Sunil Govindan. 2018-06-12 17:14:22 +05:30
Rohith Sharma K S 2df73dace0 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 2018-06-12 17:13:09 +05:30
Inigo Goiri 2b2f672022 YARN-8370. Some Node Manager tests fail on Windows due to improper path/file separator. Contributed by Anbang Hu. 2018-06-11 19:25:56 -07:00
Haibo Chen 180b3c960b YARN-8321. AllocationFileLoaderService.getAllocationFile() should be declared as VisibleForTest. (Szilard Nemeth via Haibo Chen) 2018-06-11 11:19:33 -07:00
Haibo Chen 676dcffff5 YARN-8322. Change log level when there is an IOException when the allocation file is loaded. (Szilard Nemeth via Haibo Chen) 2018-06-11 11:16:21 -07:00
Haibo Chen c190ac2be8 YARN-8323. FairScheduler.allocConf should be declared as volatile. (Szilard Nemeth via Haibo Chen) 2018-06-11 11:12:44 -07:00
Inigo Goiri 3b88fe25ba YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 2018-06-07 17:13:03 -07:00
Robert Kanter 351cf87c92 Disable mounting cgroups by default (miklos.szegedi@cloudera.com via rkanter) 2018-06-07 17:09:34 -07:00
Miklos Szegedi d5eca1a6a0 YARN-6677. Preempt opportunistic containers when root container cgroup goes over memory limit. Contributed by Haibo Chen. 2018-06-07 16:38:23 -07:00
Inigo Goiri 67fc70e09f YARN-8400. Fix typos in YARN Federation documentation page. Contributed by Giovanni Matteo Fumarola. 2018-06-07 16:10:15 -07:00
Rohith Sharma K S 377ea1bcdf YARN-8386. App log can not be viewed from Logs tab in secure cluster. Contributed by Sunil Govindan. 2018-06-07 21:31:15 +05:30
Rohith Sharma K S 58bc34f1e3 YARN-8399. NodeManager is giving 403 GSS exception post upgrade to 3.1 in secure mode. Contributed by Sunil Govindan. 2018-06-07 10:25:56 +05:30
Rohith Sharma K S 745f3a2441 YARN-8396. Click on an individual container continuously spins and doesn't load the page. Contributed by Sunil Govindan. 2018-06-05 19:28:24 +05:30
Eric Yang 5cf37418bd YARN-8365. Set DNS query type according to client request. 
Contributed by Shane Kumpf
 2018-06-04 19:06:28 -04:00
Robert Kanter 0cd145a443 YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 2018-06-04 15:32:03 -07:00
Haibo Chen 04cf699dd5 YARN-8388. TestCGroupElasticMemoryController.testNormalExit() hangs on Linux. (Miklos Szegedi via Haibo Chen) 2018-06-04 14:23:08 -07:00
Miklos Szegedi e2c172dc9f YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 2018-06-04 10:41:10 -07:00
Inigo Goiri 61fc7f73f2 YARN-8389. Improve the description of machine-list property in Federation docs. Contributed by Takanobu Asanuma. 2018-06-04 09:28:09 -07:00
Haibo Chen ba12f87dcb YARN-8390. Fix API incompatible changes in FairScheduler's AllocationFileLoaderService. (Gergo Repas via Haibo Chen) 2018-06-04 09:15:23 -07:00
Sunil G 9c4cbed8d1 YARN-8276. [UI2] After version field became mandatory, form-based submission of new YARN service doesn't work. Contributed by Gergely Novák. 2018-06-04 00:08:26 +05:30
Billie Rinaldi 31998643a5 YARN-8342. Enable untrusted docker image to run with launch command. Contributed by Eric Yang 2018-06-02 14:46:32 -07:00
Rohith Sharma K S 8261f9e571 More YARN pages need to honor yarn.resourcemanager.display.per-user-apps(addendum). Contributed by Sunil G. 2018-06-02 09:29:06 +05:30
Haibo Chen 4880d890ee YARN-8375. TestCGroupElasticMemoryController fails surefire build. (Miklos Szegedi via Haibo Chen) 2018-06-01 16:29:13 -07:00
Wangda Tan ff583d3fa3 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
 2018-06-01 14:49:18 -07:00
Wangda Tan 8956e5b8db YARN-8372. Distributed shell app master should not release containers when shutdown if keep-container is true. (Suma Shivaprasad via wangda) 
Change-Id: Ief04d1ca865621f348fba4ac85fa78bc47465904
 2018-06-01 14:49:18 -07:00
Wangda Tan 931f78718f YARN-7962. Race Condition When Stopping DelegationTokenRenewer causes RM crash during failover. (BELUGA BEHR via wangda) 
Change-Id: I617e2645f60a57080058ad5f06af860fb3f682c8
 2018-06-01 14:49:18 -07:00
Wangda Tan 3a6bd77550 YARN-8384. stdout.txt, stderr.txt logs of a launched docker container is coming with primary group of submit user instead of hadoop. (Eric Yang via wangda) 
Change-Id: Idbb2cd250f4f62dc32993e0d3ca3ec5684616baa
 2018-06-01 14:49:17 -07:00
Yufei Gu 6b21a599fb YARN-7340. Fix the missing time stamp in exception message in Class 
NoOverCommitPolicy. Contributed by Dinesh Chitlangia.
 2018-05-31 22:30:12 -07:00
Eric Yang 7dd26d5378 YARN-8308. Fixed YARN Service AM failure with HDFS token renewal. 
Contributed by Gour Saha
 2018-05-31 21:09:36 -04:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 6b74f5d7fc YARN-8197. Fixed AM IP Filter and Webapp proxy to redirect app tracking-URLs correctly when UI is secure. Contributed by Sunil Govindan. 2018-05-31 16:48:33 -07:00
Billie Rinaldi 6bc92e304f YARN-8333. Load balance YARN services using RegistryDNS multiple A records. Contributed by Eric Yang 2018-05-31 06:46:34 -07:00
Weiwei Yang 6468071f13 YARN-8367. Fix NPE in SingleConstraintAppPlacementAllocator when placement constraint in SchedulingRequest is null. Contributed by Weiwei Yang. 2018-05-31 20:46:39 +08:00
Billie Rinaldi 778a4a24be YARN-8350. NPE in service AM related to placement policy. Contributed by Gour Saha 2018-05-30 13:19:13 -07:00
Billie Rinaldi 96eefcc84a YARN-8368. yarn app start cli should print applicationId. Contributed by Rohith Sharma K S 2018-05-30 12:37:43 -07:00
Eric E Payne e44c0849d7 YARN-8377: Javadoc build failed in hadoop-yarn-server-nodemanager. Contributed by Takanobu Asanuma 2018-05-30 16:50:19 +00:00
Eric Yang 135941e00d YARN-8362. Bugfix logic in container retries in node manager. 
Contributed by Chandni Singh
 2018-05-29 16:56:58 -04:00
Jason Lowe 4827e9a908 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 2018-05-29 14:43:17 -05:00
Wangda Tan 17aa40f669 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
 2018-05-29 09:27:36 -07:00
Wangda Tan 3061bfcde5 YARN-8339. Service AM should localize static/archive resource types to container working directory instead of 'resources'. (Suma Shivaprasad via wangda) 
Change-Id: I9f8e8f621650347f6c2f9e3420edee9eb2f356a4
 2018-05-29 09:23:11 -07:00
Jason Lowe 31ab960f4f YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 2018-05-29 11:00:30 -05:00
Sunil G 7c343669ba YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 2018-05-28 16:32:53 +05:30
Rohith Sharma K S 88cbe57c06 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 2018-05-28 14:05:49 +05:30
Wangda Tan f24c842d52 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
 2018-05-25 21:53:20 -07:00
Eric E Payne 8d5509c681 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 2018-05-25 16:06:09 +00:00
Haibo Chen 86bc6425d4 YARN-8191. Fair scheduler: queue deletion without RM restart. (Gergo Repas via Haibo Chen) 2018-05-24 17:12:34 -07:00
Eric Yang d9852eb589 YARN-8357. Fixed NPE when YARN service is saved and not deployed. 
Contributed by Chandni Singh
 2018-05-24 16:32:13 -04:00
Eric Yang 7ff5a40218 YARN-8316. Improved diagnostic message for ATS unavailability for YARN Service. 
Contributed by Billie Rinaldi
 2018-05-24 16:26:02 -04:00
Shane Kumpf 1388de18ad YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:30:39 -06:00
Rohith Sharma K S c05b5d424b YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 2018-05-24 14:19:46 +05:30
Rohith Sharma K S 4cc0c9b0ba YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 2018-05-24 12:23:47 +05:30
Haibo Chen d996479954 YARN-4599. Set OOM control for memory cgroups. (Miklos Szegedi via Haibo Chen) 2018-05-23 16:35:37 -07:00
Inigo Goiri f09dc73001 YARN-8327. Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows. Contributed by Giovanni Matteo Fumarola. 2018-05-23 15:59:30 -07:00
Inigo Goiri d72615611c YARN-8348. Incorrect and missing AfterClass in HBase-tests to fix NPE failures. Contributed by Giovanni Matteo Fumarola. 2018-05-23 14:43:59 -07:00
Inigo Goiri e99e5bf104 YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 2018-05-23 14:15:26 -07:00
Inigo Goiri e30938af12 YARN-8336. Fix potential connection leak in SchedConfCLI and YarnWebServiceUtils. Contributed by Giovanni Matteo Fumarola. 2018-05-23 11:55:31 -07:00
Rohith Sharma K S f61e3e752e YARN-8297. Incorrect ATS Url used for Wire encrypted cluster.(addendum). Contributed by Sunil G. 2018-05-23 18:31:03 +05:30
Shane Kumpf 9837ca9cc7 YARN-8285. Remove unused environment variables from the Docker runtime. Contributed by Eric Badger 2018-05-23 06:43:44 -06:00
Miklos Szegedi 3e5f7ea986 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-22 18:10:33 -07:00
Robert Kanter b22f56c471 YARN-8273. Log aggregation does not warn if HDFS quota in target directory is exceeded (grepas via rkanter) 2018-05-22 14:24:38 -07:00
Wangda Tan 83f53e5c62 YARN-8332. Incorrect min/max allocation property name in resource types doc. (Weiwei Yang via wangda) 
Change-Id: If74f1ceed9c045a2cb2d6593741278b65ac44a9f
 2018-05-22 13:33:33 -07:00
Wangda Tan bd15d2396e YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
 2018-05-22 13:33:33 -07:00
Eric Yang 6176d2b35c YARN-7960. Added security flag no-new-privileges for YARN Docker integration. 
Contributed by Eric Badger
 2018-05-22 13:44:58 -04:00
Jason Lowe 5f11288e41 YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger 2018-05-22 09:27:08 -05:00
Eric E Payne 0b4c44bdee YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 2018-05-21 20:14:58 +00:00
Haibo Chen f48fec83d0 YARN-8248. Job hangs when a job requests a resource that its queue does not have. (Szilard Nemeth via Haibo Chen) 2018-05-21 08:10:41 -07:00