Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,930 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4977 Commits

Author SHA1 Message Date
Jonathan Hung 221494a75c YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b)
 2018-11-01 14:31:22 -07:00
Rohith Sharma K S d9a494b1e0 YARN-8950. Fix compilation issue due to dependency convergence error for hbase.profile=2.0. 
(cherry picked from commit 4ec4ec6971)
 2018-10-30 11:50:56 +05:30
Weiwei Yang 7dd8eafe33 YARN-8915. Update the doc about the default value of maximum-container-assignments for capacity scheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 78ea897b67)
 2018-10-29 12:06:52 +08:00
Weiwei Yang 70efe253f3 YARN-8944. TestContainerAllocation.testUserLimitAllocationMultipleContainers failure after YARN-8896. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 1d90a0dd23)
 2018-10-29 11:56:31 +08:00
Jason Lowe 3be72b7aa2 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9c)
 2018-10-23 12:55:48 -05:00
Rohith Sharma K S 3e3b088856 YARN-8826. Fix lingering timeline collector after serviceStop in TimelineCollectorManager. Contributed by Prabha Manepalli. 
(cherry picked from commit 0b62983c5a)
 2018-10-23 14:08:06 +05:30
Sunil G 0cb184d6e9 YARN-8868. Set HTTPOnly attribute to Cookie. Contributed by Chandni Singh. 
(cherry picked from commit 2202e00ba8)
 2018-10-23 09:58:05 +05:30
Sunil G 450f231a53 YARN-7502. Nodemanager restart docs should describe nodemanager supervised property. Contributed by Suma Shivaprasad. 
(cherry picked from commit e50334513c)
 2018-10-22 14:16:54 +05:30
Eric Yang e86efa8712 YARN-8910. Fixed misleading log statement when container max retries is infinite. 
Contributed by Chandni Singh

(cherry picked from commit 47ad98b2e1)
 2018-10-19 13:50:32 -04:00
Wangda Tan c11d2341af YARN-8908. Fix errors in yarn-default.xml related to GPU/FPGA. (Zhankun Tang via wangda) 
Change-Id: I46f511a9b51b65b069e15d5b827530110c41edab
(cherry picked from commit 071b9e19aa)
 2018-10-19 09:55:27 -07:00
Weiwei Yang beca90ece8 YARN-8907. Fix incorrect logging message in TestCapacityScheduler. Contributed by Zhankun Tang. 
(cherry picked from commit 13cc0f50ea)
 2018-10-19 10:02:46 +08:00
Wangda Tan d381f32219 YARN-6098. Add documentation for Delete Queue. (Suma Shivaprasad via wangda) 
Change-Id: I23bf7ca0acbe1168a7f805ceccdd62fe41bfeb35
(cherry picked from commit bfb88b10f4)
 2018-10-18 12:29:28 -07:00
Wangda Tan 46baafedf1 YARN-8896. Limit the maximum number of container assignments per heartbeat. (Zhankun Tang via wangda) 
Change-Id: I6e72f8362bd7f5c2a844cb9e3c4732492314e9f1
(cherry picked from commit 780be14f07)
 2018-10-18 12:29:19 -07:00
Eric Yang 3cb52958b2 YARN-8687. Update YARN service file type in documentation. 
Contributed by Suma Shivaprasad
 2018-10-18 12:04:27 -04:00
Eric Yang 7c40f9b37c YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. 
Contributed by Chandni Singh
 2018-10-16 18:58:28 -04:00
Wangda Tan 3e1894c376 YARN-8892. YARN UI2 doc changes to update security status (verified under security environment). (Sunil G via wangda) 
Change-Id: I8bc8622936861b8d6de3e42a0b75af86ad8a3961
(cherry picked from commit 538250db26)
 2018-10-16 14:36:36 -07:00
Sunil G 38e187384b YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang. 
(cherry picked from commit fa94d370b6)
 2018-10-15 21:23:07 +05:30
Sunil G 52d270ef8c YARN-8753. [UI2] Lost nodes representation missing from Nodemanagers Chart. Contributed by Yesha Vora. 
(cherry picked from commit 8a37983990)
 2018-10-11 16:16:26 +05:30
Sunil G fd9cd5f3c1 YARN-8666. [UI2] Remove application tab from YARN Queue Page. Contributed by Yesha Vora. 
(cherry picked from commit d3da741fa2)
 2018-10-11 16:07:30 +05:30
Weiwei Yang a0060cf8ee Revert "YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach." 
This reverts commit ce4a0898df.
 2018-10-10 21:41:00 +08:00
Weiwei Yang 3968ce1073 YARN-8858. CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. Contributed by Wangda Tan. 
(cherry picked from commit edce866489)
 2018-10-10 09:48:56 +08:00
Eric Yang 5813c1de56 YARN-8845. Removed unused hadoop.registry.rm reference. 
Contributed by Íñigo Goiri

(cherry picked from commit bf04f19456)
 2018-10-09 15:56:38 -04:00
Sunil G be0ce74c71 YARN-8853. [UI2] Application Attempts tab is not shown correctly when there are no attempts. Contributed by Akhil PB. 
(cherry picked from commit 6f11919beb)
 2018-10-09 22:06:21 +05:30
Weiwei Yang ce4a0898df YARN-8468. Enable the use of queue based maximum container allocation limit and implement it in FairScheduler. Contributed by Antal Bálint Steinbach. 2018-10-09 22:30:42 +08:00
Sunil G dd445e0368 YARN-7825. [UI2] Maintain constant horizontal application info bar for Application Attempt page. Contributed by Akhil PB. 
(cherry picked from commit 228dc192f1)
 2018-10-08 11:56:28 +05:30
Sunil G f3f0a9a2ca YARN-8797. [UI2] Improve error pages in new YARN UI. Contributed by Akhil PB. 
(cherry picked from commit 94d82f4162)
 2018-10-05 14:34:55 +05:30
Sunil G 6407f2a0ed YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB. 
(cherry picked from commit 751f626e50)
 2018-10-05 14:10:57 +05:30
Sunil G fae24b4747 Revert "YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB." 
This reverts commit dd70029255.
 2018-10-05 13:38:21 +05:30
Sunil G dd70029255 YARN-7957. [UI2] YARN service delete option disappears after stopping application. Contributed by Akhil PB. 
(cherry picked from commit 751f626e50)
 2018-10-05 12:29:33 +05:30
Wangda Tan 86a1ad4428 YARN-8844. TestNMProxy unit test is failing. (Eric Yang via wangda) 
Change-Id: I241fa8701b6f1dbcad87fd2e9a429e32e7aa40f5
(cherry picked from commit b3ac886933)
 2018-10-04 10:49:29 -07:00
Wangda Tan 30f50a0682 YARN-8758. Support getting PreemptionMessage when using AMRMClientAsyn. (Zian Chen via wangda) 
Change-Id: Ibf5d165f49957b582eeadeb41dc285c84d2f05e7
(cherry picked from commit 6926fd0ec6)
(cherry picked from commit abe4a8e5d8)
 2018-10-04 10:49:23 -07:00
Akira Ajisaka 58693c63db
YARN-8840. Add missing cleanupSSLConfig() call for TestTimelineClient test. Contributed by Aki Tanaka. 
(cherry picked from commit 39b35036ba)
 2018-10-04 09:53:22 +09:00
Shane Kumpf adbc010d0f YARN-8785. Improve the error message when a bind mount is not whitelisted. Contributed by Simon Prewo 
(cherry picked from commit 5edb9d3b97)
 2018-10-02 07:26:45 -06:00
Eric E Payne c306da08ec YARN-8774. Memory leak when CapacityScheduler allocates from reserved container with non-default label. Contributed by Tao Yang. 
(cherry picked from commit 8598b498bc)
 2018-09-28 15:34:23 +00:00
Vrushali C 5b72aa04e1 YARN-8270 Adding JMX Metrics for Timeline Collector and Reader. Contributed by Sushil Ks. 
(cherry picked from commit 90e2e493b3)
 2018-09-28 10:31:38 +05:30
Jason Lowe a56a345e07 YARN-8804. resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues. Contributed by Tao Yang 
(cherry picked from commit 6b988d821e)
 2018-09-26 16:15:48 -07:00
Rohith Sharma K S d473152e6a YARN-8824. App Nodelabel missed after RM restart for finished apps. Contributed by Bibin A Chundatt. 2018-09-26 20:36:34 +05:30
Akira Ajisaka 7dbdb7560d
YARN-8752. yarn-registry.md has wrong word ong-lived, it should be long-lived. Contributed by leiqiang. 
(cherry picked from commit 94b5c54e8a)
 2018-09-26 09:43:33 +09:00
Shane Kumpf eea10c7b20 YARN-8623. Update Docker examples to use image which exists. Contributed by Craig Condit 
(cherry picked from commit 26c94a0fd0)
 2018-09-25 06:42:20 -06:00
Rohith Sharma K S 47306cc2db YARN-8815. RM fails to recover finished unmanaged AM. Contributed by Bibin A Chundatt. 
(cherry picked from commit 50bc7746d7)
 2018-09-25 11:40:09 +05:30
Sunil G 0196ffaa22 YARN-8742. [UI2] Container logs on Application / Service pages on UI2 are not available many case, improve error messages in such cases. Contributed by Akhil PB. 
(cherry picked from commit 32a35dc94b)
 2018-09-24 12:21:15 +05:30
Sunil G 6699b9658e YARN-8628. [UI2] Few duplicated or inconsistent information displayed in UI2. Contributed by Akhil PB. 
(cherry picked from commit a2752779ac)
 2018-09-21 16:06:22 +05:30
Eric Yang e9315f6688 YARN-8801. Fixed header comments for docker utility functions. 
Contributed by Zian Chen
 2018-09-20 13:12:29 -04:00
Jason Lowe 3fb6787295 YARN-8784. DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk. Contributed by Eric Badger 
(cherry picked from commit 6b5838ed32)
 2018-09-19 16:49:21 -05:00
Weiwei Yang aaf0b119e5 YARN-8771. CapacityScheduler fails to unreserve when cluster resource contains empty resource type. Contributed by Tao Yang. 
(cherry picked from commit 0712537e79)
 2018-09-19 19:38:09 +08:00
Jason Lowe 3d77094cf2 YARN-8648. Container cgroups are leaked when using docker. Contributed by Jim Brennan 
(cherry picked from commit 2df0a8dcb3)
 2018-09-18 15:43:10 -05:00
Weiwei Yang 46eeba6234 YARN-8787. Fix broken list items in PlacementConstraints documentation. Contributed by Masahiro Tanaka. 
(cherry picked from commit 78a0d173e4)
 2018-09-18 17:21:21 +08:00
Sunil G 75d2d0fdb1 YARN-8652. [UI2] YARN UI2 breaks if getUserInfo REST API is not available in older versions. Contributed by Akhil PB. 
(cherry picked from commit bbeca0107e)
 2018-09-18 12:45:08 +05:30
Sunil G 8dccc4ef12 YARN-8726. [UI2] YARN UI2 is not accessible when config.env file failed to load. Contributed by Akhil PB. 
(cherry picked from commit 0cc6e03945)
 2018-09-18 12:32:30 +05:30
Weiwei Yang f0e5651cf4 YARN-8782. Fix exception message in Resource.throwExceptionWhenArrayOutOfBound. Contributed by Gergely Pollak. 
(cherry picked from commit 9a265fa673)
 2018-09-17 22:26:09 +08:00
Weiwei Yang 00a469138d YARN-8720. CapacityScheduler does not enforce max resource allocation check at queue level. Contributed by Tarun Parimi. 
(cherry picked from commit f1a893fdbc)
 2018-09-14 16:40:35 +08:00
Jason Lowe 88687213cc YARN-8680. YARN NM: Implement Iterable Abstraction for LocalResourceTracker state. Contributed by Pradeep Ambati 
(cherry picked from commit 250b50018e)
 2018-09-13 14:12:20 -05:00
Weiwei Yang a7b1d1e006 YARN-8729. Node status updater thread could be lost after it is restarted. Contributed by Tao Yang. 
(cherry picked from commit 39c1ea1ed4)
 2018-09-13 23:16:52 +08:00
Sunil G c879ca38de YARN-8630. ATSv2 REST APIs should honor filter-entity-list-by-user in non-secure cluster when ACls are enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit f4bda5e8e9)
 2018-09-13 17:48:01 +05:30
Eric E Payne b6bc0f409a YARN-8709: CS preemption monitor always fails since one under-served queue was deleted. Contributed by Tao Yang. 
(cherry picked from commit 987d8191ad)
 2018-09-10 20:02:39 +00:00
Eric Yang 0b97dc5869 YARN-8751. Reduce conditions that mark node manager as unhealthy. 
Contributed by Craig Condit

(cherry picked from commit 7d62334387)
 2018-09-07 20:32:11 -04:00
Giovanni Matteo Fumarola 7fed1a07ab HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang. 2018-09-07 16:37:03 -07:00
Shane Kumpf 2d68708a1d YARN-8638. Allow linux container runtimes to be pluggable. Contributed by Craig Condit 
(cherry picked from commit dffb7bfe6c)
 2018-09-05 06:55:25 -06:00
bibinchundatt e2e0fc26a2 YARN-8535. Fix DistributedShell unit tests. Contributed by Abhishek Modi. 
(cherry picked from commit eed8415dc1)
 2018-09-02 13:37:38 +05:30
Shane Kumpf b8618556ee YARN-8642. Add support for tmpfs mounts with the Docker runtime. Contributed by Craig Condit 
(cherry picked from commit 73625168c0)
 2018-08-29 07:11:38 -06:00
Weiwei Yang f164568b47 YARN-8723. Fix a typo in CS init error message when resource calculator is not correctly set. Contributed by Abhishek Modi. 
(cherry picked from commit 3fa4639421)
 2018-08-29 11:15:02 +08:00
Billie Rinaldi eefd780918 YARN-8675. Remove default hostname for docker containers when net=host. Contributed by Suma Shivaprasad 
(cherry picked from commit 05b2bbeb35)
 2018-08-27 11:42:09 -07:00
Weiwei Yang 271ed86ac0 YARN-8719. Typo correction for yarn configuration in OpportunisticContainers(federation) docs. Contributed by Y. SREENIVASULU REDDY. 
(cherry picked from commit e8b063f630)
 2018-08-28 01:07:05 +08:00
Haibo Chen e4282c077b YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 
(cherry picked from commit 93d47a0ed5)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMEmbeddedElector.java
 2018-08-24 13:24:08 -05:00
Inigo Goiri 246086984e YARN-7773. YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list'. Contributed by Yiran Wu. 
(cherry picked from commit 60e5c1b516)
 2018-08-24 07:10:02 +05:30
Jason Lowe 84654451fa YARN-8649. NPE in localizer hearbeat processing if a container is killed while localizing. Contributed by lujie 
(cherry picked from commit 585ebd873a)
 2018-08-23 09:37:43 -05:00
Rohith Sharma K S 63d5214332 YARN-8129. Improve error message for invalid value in fields attribute. Contributed by Abhishek Modi. 
(cherry picked from commit d3fef7a5c5)
 2018-08-21 12:11:12 +05:30
Wei-Chiu Chuang 0d155de159 HADOOP-14212. Addendum patch: Expose SecurityEnabled boolean field in JMX for other services besides NameNode. Contributed by Adam Antal. 2018-08-20 14:49:28 -07:00
Wei-Chiu Chuang 78fb14ba49 HADOOP-14212. Expose SecurityEnabled boolean field in JMX for other services besides NameNode. Contributed by Adam Antal. 2018-08-20 14:49:24 -07:00
Jason Lowe 44c4928b64 YARN-8242. YARN NM: OOM error while reading back the state store on recovery. Contributed by Pradeep Ambati and Kanwaljeet Sachdev 
(cherry picked from commit 65e7469712)
 2018-08-20 10:21:57 -05:00
Rohith Sharma K S a3d4a25bbf YARN-8679. [ATSv2] If HBase cluster is down for long time, high chances that NM ContainerManager dispatcher get blocked. Contributed by Wangda Tan. 
(cherry picked from commit 4aacbfff60)
 2018-08-18 11:04:09 +05:30
Rohith Sharma K S 3532bd5c8b YARN-8612. Fix NM Collector Service Port issue in YarnConfiguration. Contributed by Prabha Manepalli. 
(cherry picked from commit 1697a02306)
 2018-08-17 11:13:25 +05:30
Eric Yang 5237bdfb5a YARN-8667. Cleanup symlinks when container restarted by NM. 
Contributed by Chandni Singh

(cherry picked from commit d42806160e)
 2018-08-16 18:44:47 -04:00
Eric Yang 97c1934247 YARN-8474. Fixed ApiServiceClient kerberos negotiation. 
Contributed by Billie Rinaldi

(cherry picked from commit 8990eaf592)
 2018-08-16 12:50:52 -04:00
Jason Lowe 819a2a6f10 YARN-8656. container-executor should not write cgroup tasks files for docker containers. Contributed by Jim Brennan 
(cherry picked from commit cb21eaa026)
 2018-08-16 10:09:56 -05:00
Weiwei Yang 05547b1e0b YARN-8614. Fix few annotation typos in YarnConfiguration. Contributed by Sen Zhao. 
(cherry picked from commit 8dc07b4723)
 2018-08-15 12:38:55 +08:00
Jason Lowe 95cd6de5c6 YARN-8640. Restore previous state in container-executor after failure. Contributed by Jim Brennan 
(cherry picked from commit d1d129aa9d)
 2018-08-14 10:26:21 -05:00
Shane Kumpf ba6b48e19f YARN-8520. Document best practice for user management. Contributed by Eric Yang 
(cherry picked from commit e7951c69cb)
 2018-08-10 14:47:14 -06:00
Weiwei Yang 734bc42289 YARN-8575. Avoid committing allocation proposal to unavailable nodes in async scheduling. Contributed by Tao Yang. 
(cherry picked from commit 0a71bf1452)
 2018-08-10 15:10:27 +08:00
Weiwei Yang 991514f7c3 YARN-8521. NPE in AllocationTagsManager when a container is removed more than once. Contributed by Weiwei Yang. 
(cherry picked from commit 08d5060605)
 2018-08-10 08:44:53 +08:00
Wangda Tan beea18736b YARN-8136. Add version attribute to site doc examples and quickstart. (Eric Yang via wangda) 
Change-Id: I4541b239f490ca0a6edf9698e0d3deaf83669151
(cherry picked from commit 8244abb7ae)
 2018-08-09 12:04:33 -07:00
Wangda Tan 68279fcd65 YARN-8588. Logging improvements for better debuggability. (Suma Shivaprasad via wangda) 
Change-Id: I66aa4b0ec031ae5ce0fae558e2f8cbcbbfebc442
(cherry picked from commit 344c335a92)
 2018-08-09 12:04:25 -07:00
Weiwei Yang 0ee7e80047 YARN-8559. Expose mutable-conf scheduler's configuration in RM /scheduler-conf endpoint. Contributed by Weiwei Yang. 
(cherry picked from commit d352f167eb)
 2018-08-10 00:43:53 +08:00
Jason Lowe 3dd299a770 YARN-8331. Race condition in NM container launched after done. Contributed by Pradeep Ambati 
(cherry picked from commit cd04e954d2)
 2018-08-09 10:23:02 -05:00
Sunil G c6013b060d YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 
(cherry picked from commit 00013d6ef7)
 2018-08-09 12:35:21 +05:30
Bharat Viswanadham 5430aab133 YARN-8426:Upgrade jquery-ui to 1.12.1 in YARN. Contributed by Sunil Govindan 
(cherry picked from commit 361ffb26be)
 2018-08-09 12:27:50 +05:30
Robert Kanter 0ed91309d3 YARN-8568. Replace the deprecated zk-address property in the HA config example in ResourceManagerHA.md (bsteinbach via rkanter) 
(cherry picked from commit 8478732bb2)
 2018-08-08 15:10:52 -07:00
Wangda Tan f16c311486 Make 3.1.1 awared by other branches - adding missing files 
(cherry picked from commit 3214cd75ac)
 2018-08-08 13:06:14 -07:00
Wangda Tan 781620a44d Make 3.1.1 awared by other branches 
(cherry picked from commit 49c687608b)
 2018-08-08 13:04:05 -07:00
bibinchundatt b4f957e655 YARN-8601. Print ExecutionType in Container report CLI. Contributed by Bilwa S T. 
(cherry picked from commit ff06bd1be8)
 2018-08-08 22:46:16 +05:30
Wangda Tan f2768eaa38 YARN-8407. Container launch exception in AM log should be printed in ERROR level. (Yesha Vora via wangda) 
Change-Id: I154e873df1df3503a09c41d6b3874ca195af91d9
(cherry picked from commit 861095f761)
 2018-08-07 13:07:28 -07:00
Wangda Tan 450c791ecf YARN-8629. Container cleanup fails while trying to delete Cgroups. (Suma Shivaprasad via wangda) 
Change-Id: I392ef4f8baa84d5d7b1f2e438c560b5426b6d4f2
(cherry picked from commit d4258fcad7)
 2018-08-07 12:41:55 -07:00
Eric Yang 947dc34229 YARN-8624. Updated verbiage around entry point support. 
Contributed by Craig Condit
 2018-08-06 12:39:22 -04:00
Shane Kumpf 7773904974 YARN-8287. Update documentation and yarn-default related to the Docker runtime. Contributed by Craig Condit 
(cherry picked from commit bcfc9851f4)
 2018-08-03 18:23:37 -06:00
Sunil G d8fdcf8b42 YARN-8615. [UI2] Resource Usage tab shows only memory related info. No info available for vcores/gpu. Contributed by Akhil PB. 
(cherry picked from commit 3426f4062a)
 2018-08-03 19:36:19 +05:30
Sunil G 1bfce639e8 YARN-8620. [UI2] YARN Services UI new submission failures are not debuggable. Contributed by Akhil PB. 
(cherry picked from commit 29417dbbf4)
 2018-08-03 19:19:17 +05:30
Rohith Sharma K S 4816d10ddb YARN-8608. [UI2] No information available per application appAttempt about 'Total Outstanding Resource Requests'. Contributed by Akhil PB. 
(cherry picked from commit 022592ae79)
 2018-08-03 15:33:36 +05:30
Rohith Sharma K S 83ccb75b91 YARN-8603. [UI2] Latest run application should be listed first in the RM UI. Contributed by Akhil PB. 
(cherry picked from commit 33482d35e8)
 2018-08-03 15:33:24 +05:30
Jason Lowe 619019ccca YARN-8263. DockerClient still touches hadoop.tmp.dir. Contributed by Craig Condit 
(cherry picked from commit 7526815e32)
 2018-08-02 10:45:52 -05:00
Sunil G d76ab31961 YARN-8318. [UI2] IP address in component page shows N/A. Contributed by Yesha Vora. 
(cherry picked from commit 5033d7da8f)
 2018-08-02 20:10:08 +05:30