Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,793 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

920 Commits

Author SHA1 Message Date
Szilard Nemeth e8fa192f07 YARN-9217. Nodemanager will fail to start if GPU is misconfigured on the node or GPU drivers missing. Contributed by Peter Bacsko 2019-08-21 16:44:22 +02:00
Haibo Chen f51702d539 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 2019-08-06 13:52:30 -07:00
Szilard Nemeth 54ac80176e Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 2019-08-02 13:38:06 +02:00
Adam Antal cbcada804d YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently (#1121) 2019-08-02 13:04:34 +02:00
Szilard Nemeth a2a8be18cb YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 2019-07-03 13:45:00 +02:00
Weiwei Yang cbae241320 YARN-9623. Auto adjust max queue length of app activities to make sure activities on all nodes can be covered. Contributed by Tao Yang. 2019-06-28 23:24:53 +08:00
Sunil G e49162f4b3 YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-05-31 10:28:09 +05:30
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Igor Rudenko e2b79912c3 YARN-9469. Fix typo in YarnConfiguration. 2019-05-23 19:00:38 +02:00
Weiwei Yang 1a47c2b7ae YARN-9539.Improve cleanup process of app activities and make some conditions configurable. Contributed by Tao Yang. 2019-05-12 22:31:39 -07:00
Eric Yang accb811e57 YARN-6929. Improved partition algorithm for yarn remote-app-log-dir. 
Contributed by Prabhu Joseph
 2019-04-30 17:04:59 -04:00
Inigo Goiri 96e3027e46 YARN-2889. Limit the number of opportunistic container allocated per AM heartbeat. Contributed by Abhishek Modi. 2019-04-22 09:49:03 -07:00
Akira Ajisaka c65218ee8e YARN-8943. Upgrade JUnit from 4 to 5 in hadoop-yarn-api. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-04-16 12:13:49 +09:00
Eric Badger 254efc9358 YARN-9379. Can't specify docker runtime through environment. Contributed by caozhiqiang 2019-04-15 18:24:37 +00:00
Giovanni Matteo Fumarola cfec455c45 YARN-999. In case of long running tasks, reduce node resource should balloon out resource quickly by calling preemption API and suspending running task. Contributed by Inigo Goiri. 2019-04-09 10:59:43 -07:00
Vrushali C 22362c876d YARN-9335 [atsv2] Restrict the number of elements held in timeline collector when backend is unreachable for async calls. Contributed by Abhishesk Modi. 2019-04-05 12:06:51 -07:00
Eric Yang 09eabda314 YARN-9364. Remove commons-logging dependency from YARN. 
Contributed by Prabhu Joseph
 2019-03-18 19:58:42 -04:00
Eric Yang 2064ca015d YARN-9349. Changed logging to use slf4j api. 
Contributed by Prabhu Joseph
 2019-03-15 19:20:59 -04:00
Sunil G de15a66d78 YARN-9265. FPGA plugin fails to recognize Intel Processing Accelerator Card. Contributed by Peter Bacsko. 2019-03-08 17:39:22 +05:30
Eric Yang 7b42e0e32a YARN-7266. Fixed deadlock in Timeline Server thread initialization. 
Contributed by Prabhu Joseph
 2019-03-05 12:17:01 -05:00
Sunil G dcaca19871 YARN-9139. Simplify initializer code of GpuDiscoverer. Contributed by Szilard Nemeth. 2019-03-01 19:24:35 +05:30
Vrushali C 0ec962ac8f YARN-5336 Limit the flow name size & consider cleanup for hex chars. Contributed by Sushil Ks 2019-02-27 14:43:39 -08:00
Giovanni Matteo Fumarola 95372657fc YARN-9287. Consecutive StringBuilder append should be reuse. Contributed by Ayush Saxena. 2019-02-25 11:45:37 -08:00
Weiwei Yang 6c4ab0312b YARN-9258. Support to specify allocation tags without constraint in distributed shell CLI. Contributed by Prabhu Joseph. 2019-02-22 00:18:07 +08:00
Eric Yang 3dc2523266 YARN-9184. Add a system flag to allow update to latest docker images. 
Contributed by Zhaohui Xin
 2019-02-12 16:16:35 -05:00
Weiwei Yang 2b7f828d46 YARN-9252. Allocation Tag Namespace support in Distributed Shell. Contributed by Prabhu Joseph. 2019-02-11 14:42:21 +08:00
Rohith Sharma K S e3ec18b0c4 YARN-6735. Have a way to turn off container metrics from NMs. Contributed by Abhishek Modi. 2019-02-05 13:48:04 +05:30
Sunil G 604b2489a9 YARN-9206. RMServerUtils does not count SHUTDOWN as an accepted state. Contributed by Kuhu Shukla. 2019-02-04 12:48:28 +05:30
Eric E Payne 04105bbfdb YARN-6616: YARN AHS shows submitTime for jobs same as startTime. Contributed by Prabhu Joseph 2019-01-29 17:22:24 +00:00
Weiwei Yang 085f0e8ae7 YARN-9086. [CSI] Run csi-driver-adaptor as aux service. Contributed by Weiwei Yang. 2019-01-29 14:53:08 +08:00
Eric Yang 1ab69a9543 YARN-9221. Added flag to disable dynamic auxiliary service feature. 
Contributed by Billie Rinaldi
 2019-01-25 19:05:36 -05:00
Eric Yang a33ef4fd31 YARN-8867. Added resource localization status to YARN service status call. 
Contributed by Chandni Singh
 2019-01-24 18:43:21 -05:00
Weiwei Yang a4bd64e724 YARN-9116. Capacity Scheduler: implements queue level maximum-allocation inheritance. Contributed by Aihua Xu. 2019-01-24 17:53:43 +08:00
Weiwei Yang e3e076dc78 YARN-9205. When using custom resource type, application will fail to run due to the CapacityScheduler throws InvalidResourceRequestException(GREATER_THEN_MAX_ALLOCATION). Contributed by Zhankun Tang. 2019-01-23 14:14:49 +08:00
Vrushali C 6be39230a6 YARN-9150 Making TimelineSchemaCreator support different backends for Timeline Schema Creation in ATSv2. Contributed by Sushil Ks 2019-01-15 21:25:37 -08:00
Weiwei Yang 0921b706f7 YARN-9037. [CSI] Ignore volume resource in resource calculators based on tags. Contributed by Sunil Govindan. 2019-01-08 14:30:53 +08:00
Wangda Tan 0a01d49917 YARN-8822. Nvidia-docker v2 support for YARN GPU feature. (Charo Zhang via wangda) 
Change-Id: Id8af27134d3286a7a10d85eda9be25df9689d0e7
 2019-01-07 12:07:26 -08:00
Rohith Sharma K S 8c6978c3ba YARN-6149. Allow port range to be specified while starting NM Timeline collector manager. Contributed by Abhishek Modi. 2019-01-04 14:07:02 +05:30
Sunil G f4906ac019 YARN-9038. [CSI] Add ability to publish/unpublish volumes on node managers. Contributed by Weiwei Yang. 2019-01-04 12:10:00 +05:30
Eric Yang f82922dcfa YARN-5168. Added exposed port information for Docker container. 
Contributed by Xun Liu
 2018-12-21 19:44:07 -05:00
Weiwei Yang f659485ee8 YARN-8925. Updating distributed node attributes only when necessary. Contributed by Tao Yang. 2018-12-21 10:56:42 +08:00
Inigo Goiri 5df9fb16b9 YARN-9130. Add Bind_HOST configuration for Yarn Web Proxy. Contributed by Rong Tang. 2018-12-19 18:58:00 -08:00
Eric Yang c7a5a4435e YARN-9075. Add ability to register/remove auxiliary service without restart node manager. 
Contributed by Billie Rinaldi
 2018-12-18 17:05:51 -05:00
Vidura Mudalige 04c03476c9
YARN-9095. Removed Unused field from Resource: NUM_MANDATORY_RESOURCES 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-12-15 16:07:27 +09:00
Sunil G 5fb14e0635 YARN-8953. [CSI] CSI driver adaptor module support in NodeManager. Contributed by Weiwei Yang. 2018-11-20 08:02:11 +05:30
Billie Rinaldi 27ffec7ba7 YARN-8778. Add command line interface to invoke interactive docker shell. Contributed by Eric Yang 2018-11-19 17:59:12 -08:00
Sunil G 42f3a7082a YARN-8877. [CSI] Extend service spec to allow setting resource attributes. Contributed by Weiwei Yang. 2018-11-12 21:09:30 +05:30
Weiwei Yang f8c72d7b3a YARN-8880. Add configurations for pluggable plugin framework. Contributed by Zhankun Tang. 2018-11-08 12:23:00 +08:00
Weiwei Yang 482716e5a4 YARN-8976. Remove redundant modifiers in interface ApplicationConstants. Contributed by Zhankun Tang. 2018-11-07 10:48:07 +08:00
Billie Rinaldi d07e873b7d YARN-8569. Create an interface to provide cluster information to application. Contributed by Eric Yang 2018-10-26 17:57:05 -07:00
Haibo Chen 766b78ee07 YARN-8911. ContainerScheduler incorrectly uses percentage number as the cpu resource utlization. 2018-10-24 07:58:26 -07:00
Haibo Chen c2288ac45b YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter) 2018-10-16 13:36:26 -07:00
Sunil G 8e5365e277 YARN-8836. Add tags and attributes in resource definition. Contributed by Weiwei Yang. 2018-10-15 15:38:42 +05:30
Jason Lowe 9bb2801e8c YARN-4254. ApplicationAttempt stuck for ever due to UnknownHostException. Contributed by Bibin A Chundatt 2018-10-05 15:52:46 -05:00
Sunil G 58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Eric Yang b237a0dd44 YARN-6456. Added config to set default container runtimes. 
Contributed by Craig Condit
 2018-09-27 15:31:18 -04:00
Giovanni Matteo Fumarola 3090922805 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang. 2018-09-24 11:37:05 -07:00
Weiwei Yang 9a265fa673 YARN-8782. Fix exception message in Resource.throwExceptionWhenArrayOutOfBound. Contributed by Gergely Pollak. 2018-09-17 22:15:24 +08:00
Sunil G 33d8327cff YARN-8715. Make allocation tags in the placement spec optional for node-attributes. Contributed by Weiwei Yang. 2018-09-17 10:07:45 +05:30
Shane Kumpf 8e9afbfb66 YARN-8768. Javadoc error in node attributes. Contributed by Sunil Govindan. 2018-09-12 15:12:28 -06:00
bibinchundatt c44088ac19 YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha 67ae81f0e0 YARN-7863. Modify placement constraints to support node attributes. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha eb08543c7a YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:01 +05:30
bibinchundatt 8cf6a9a2bd YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha 5dc7d6e0f3 YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha 0a01b1350d YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Sunil G b9890d1f66 YARN-7875. Node Attribute store for storing and recovering attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
bibinchundatt 901e85238d YARN-8033. CLI Integration with NodeAttributesManagerImpl. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha 3b3b6efe21 YARN-7871. Node attributes reporting from NM to RM. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Sunil G ffcabd24c3 YARN-7856. Validate Node Attributes from NM. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Naganarasimha d312b5cf9f YARN-7757. Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
bibinchundatt 4458b2772f YARN-7840. Update PB for prefix support of node attributes. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha 1f42ce907a YARN-6855. [YARN-3409] CLI Proto Modifications to support Node Attributes. Contributed by Naganarasimha G R. 2018-09-12 16:00:59 +05:30
Eric Yang bf8a1750e9 YARN-8706. Updated docker container stop logic to avoid double kill. 
Contributed by Chandni Singh
 2018-09-07 20:18:09 -04:00
Giovanni Matteo Fumarola 3dc2988a37 YARN-8699. Add Yarnclient#yarnclusterMetrics API implementation in router. Contributed by Bibin A Chundatt. 2018-09-07 11:32:03 -07:00
Shane Kumpf dffb7bfe6c YARN-8638. Allow linux container runtimes to be pluggable. Contributed by Craig Condit 2018-09-05 06:47:54 -06:00
Shane Kumpf 73625168c0 YARN-8642. Add support for tmpfs mounts with the Docker runtime. Contributed by Craig Condit 2018-08-29 07:08:37 -06:00
Giovanni Matteo Fumarola e0f6ffdbad YARN-8581. [AMRMProxy] Add sub-cluster timeout in LocalityMulticastAMRMProxyPolicy. Contributed by Botong Huang. 2018-08-20 14:33:16 -07:00
Rohith Sharma K S 1697a02306 YARN-8612. Fix NM Collector Service Port issue in YarnConfiguration. Contributed by Prabha Manepalli. 2018-08-17 11:12:10 +05:30
Weiwei Yang 8dc07b4723 YARN-8614. Fix few annotation typos in YarnConfiguration. Contributed by Sen Zhao. 2018-08-15 12:15:54 +08:00
Sunil G 12a095a496 YARN-7159. Normalize unit of resource objects in RM to avoid unit conversion in critical path. Contributed by Manikandan R. 2018-08-02 22:29:21 +05:30
Jonathan Hung 3e06a5dcea YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 17:44:18 -07:00
Robert Kanter 2cccf4061c YARN-8517. getContainer and getContainers ResourceManager REST API methods are not documented (snemeth via rkanter) 2018-07-27 14:35:03 -07:00
bibinchundatt 8461278833 YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 2018-07-24 13:09:17 +05:30
Haibo Chen f354f47f99 YARN-6995. Improve use of ResourceNotFoundException in resource types code. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-19 15:35:05 -07:00
Wangda Tan 238ffff999 YARN-8524. Single parameter Resource / LightWeightResource constructor looks confusing. (Szilard Nemeth via wangda) 
Change-Id: I4ae97548b5b8d76a6bcebb2d3d70bf8e0be3c125
 2018-07-16 10:58:00 -07:00
Rohith Sharma K S ba68320449 YARN-8302. ATS v2 should handle HBase connection issue properly. Contributed by Billie Rinaldi. 2018-07-06 15:19:01 -07:00
Haibo Chen 9edc74f64a YARN-7556. Fair scheduler configuration should allow resource types in the minResources and maxResources properties. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-06 11:03:48 -07:00
Anu Engineer 39ad98903a Revert "Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk" 
This reverts commit c163d1797a, reversing
changes made to 0d9804dcef.
 2018-07-05 12:22:18 -07:00
Haibo Chen 1726247024 YARN-7556. Fair scheduler configuration should allow resource types in the minResources and maxResources properties. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-05 10:42:39 -07:00
Inigo Goiri 99948565cb YARN-8412. Move ResourceRequest.clone logic everywhere into a proper API. Contributed by Botong Huang. 2018-06-21 18:24:10 -07:00
Miklos Szegedi 55432b0981 YARN-8440. Typo in YarnConfiguration javadoc: "Miniumum request grant-able..". Contributed by Szilard Nemeth. 2018-06-20 11:36:12 -07:00
Haibo Chen 24a89825f0 YARN-6931. Make the aggregation interval in AppLevelTimelineCollector configurable. (Abhishek Modi via Haibo Chen) 2018-06-12 10:03:34 -07:00
Akira Ajisaka 652bcbb3e4
YARN-8363. Upgrade commons-lang version to 3.7 in hadoop-yarn-project. Contributed by Takanobu Asanuma. 2018-06-13 00:35:37 +09:00
Shane Kumpf 1388de18ad YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:30:39 -06:00
Rohith Sharma K S c05b5d424b YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 2018-05-24 14:19:46 +05:30
Haibo Chen d996479954 YARN-4599. Set OOM control for memory cgroups. (Miklos Szegedi via Haibo Chen) 2018-05-23 16:35:37 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 328f0847e3 YARN-8128. Documenting the per-node per-app file limit in YARN log aggregation. Contributed by Xuan Gong. 2018-05-17 12:47:10 -07:00
Haibo Chen e3b7d7ac16 YARN-7933. [atsv2 read acls] Add TimelineWriter#writeDomain. (Rohith Sharma K S via Haibo Chen) 2018-05-16 13:00:01 -07:00