Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,182 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4612 Commits

Author SHA1 Message Date
Zhe Zhang 0db6b8c93c YARN-7623. Fix the CapacityScheduler Queue configuration documentation. Contributed by Jonathan Hung. 
(cherry picked from commit 0b1c2b5fe1)
 2018-03-28 11:42:16 -07:00
Subru Krishnan a0091ec4b3 YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 
(cherry picked from commit 09999d7e01)
 2018-03-28 11:35:45 -07:00
Subru Krishnan ed49f59e18 Revert "YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru)." 
This reverts commit 85eebf1beb.
 2018-03-28 11:35:30 -07:00
Wangda Tan a2b4daab55 YARN-6629. NPE occurred when container allocation proposal is applied but its resource requests are removed before. (Tao Yang via wangda) 
Change-Id: I805880f90b3f6798ec96ed8e8e75755f390a9ad5
(cherry picked from commit 47f711eebc)
 2018-03-28 11:22:26 -07:00
Sunil G 6c904388f4 YARN-8075. DShell does not fail when we ask more GPUs than available even though AM throws 'InvalidResourceRequestException'. Contributed by Wangda Tan. 
(cherry picked from commit d1e378d02b)
 2018-03-28 08:03:59 +05:30
Subru Krishnan 85eebf1beb YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 
(cherry picked from commit 2a2ef15caf)
 2018-03-27 17:44:00 -07:00
Wangda Tan dc99611878 YARN-8072. RM log is getting flooded with MemoryPlacementConstraintManager info logs. (Zian Chen via wangda) 
Change-Id: I2567f4669ad868e13c4301fe44c4bde85bb92c37
(cherry picked from commit 24bc5e0e4d)
 2018-03-26 11:27:40 -07:00
Wangda Tan 62b9537332 YARN-8068. Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ (Sunil G via wangda) 
Change-Id: I7910bd1064a1b4dbbe2084080c060822ea6f3b48
(cherry picked from commit 9eef19b2ad)
 2018-03-26 11:27:32 -07:00
Wangda Tan 54da6a0d78 YARN-8062. yarn rmadmin -getGroups returns group from which the user has been removed. (Sunil G via wangda) 
Change-Id: I80ed63846502bf7751b890b6c6c6a7c0679e2b4a
(cherry picked from commit 5d381570f8)
 2018-03-26 11:27:21 -07:00
Wangda Tan c98dd3bf18 YARN-8070. Yarn Service API site doc broken due to unwanted character in YarnServiceAPI.md (Gour Saha via wangda) 
Change-Id: I22428b2f128d16e79ebbdeaf32869566963d0c6f
(cherry picked from commit 24f75e097a)
 2018-03-23 17:53:33 -07:00
Rohith Sharma K S 91791a8abd YARN-7986. ATSv2 REST API queries do not return results for uppercase application tags. Contributed by Charan Hebri. 
(cherry picked from commit 75fc05f369)
 2018-03-23 12:18:43 +05:30
Sunil G e10e6c9012 YARN-8063. DistributedShellTimelinePlugin wrongly check for entityId instead of entityType. Contributed by Rohith Sharma K S. 
(cherry picked from commit 22c5ddb7c4)
 2018-03-23 10:55:13 +05:30
Jason Lowe 5d4b2c31a1 YARN-8054. Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread. Contributed by Jonathan Eagles 
(cherry picked from commit 5aa7052e31)
 2018-03-21 17:20:36 -05:00
Rohith Sharma K S bbfe36d686 YARN-7581. HBase filters are not constructed correctly in ATSv2. Contributed by Habio Chen. 
(cherry picked from commit 29acea5000)
 2018-03-21 08:26:56 +05:30
Haibo Chen d8764f1c90 YARN-8053. Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 3ff6977d3e)
 2018-03-21 08:26:56 +05:30
Jason Lowe a4f0dbcb44 YARN-8027. Setting hostname of docker container breaks for --net=host in docker 1.13. Contributed by Jim Brennan 
(cherry picked from commit f480367af6)
 2018-03-19 17:44:06 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 0bd99621c5 Preparing for 3.1.0 development 2018-03-19 15:16:54 -07:00
Yufei Gu 49c747ab18 YARN-8039. Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer. Contributed by Miklos Szegedi. 2018-03-16 16:25:33 -07:00
Weiwei Yang 154cfb2b62 YARN-7636. Re-reservation count may overflow when cluster resource exhausted for a long time. contributed by Tao Yang. 2018-03-16 18:57:31 +08:00
Wangda Tan 4bf622043f YARN-7952. RM should be able to recover log aggregation status after restart/fail-over. (Xuan Gong via wangda) 
Change-Id: I725c9afe64831eda0aa6b0bebdbc79d2dd165707
 2018-03-15 13:26:45 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Eric Yang a82be7754d YARN-7999. Added file cache initializer for Linux container-executor. 
Contributed by Jason Lowe
 2018-03-14 12:21:57 -04:00
Rohith Sharma K S e6de10d0a6 YARN-8022. ResourceManager UI cluster/app/<app-id> page fails to render. Contributed by Tarun Parimi. 2018-03-14 09:42:20 +05:30
Wangda Tan b167d60763 YARN-7657. Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue. (Suma Shivaprasad via wangda) 
Change-Id: I32d566e8727840e43c0d66e39a77edef017e3a83
 2018-03-13 17:55:51 -07:00
Wangda Tan a5b27b3c67 YARN-5015. Support sliding window retry capability for container restart. (Chandni Singh via wangda) 
Change-Id: I07addd3e4ba8d98456ee2ff1d5c540a38fe61dea
 2018-03-13 17:55:17 -07:00
Miklos Szegedi a82d4a2e3a YARN-5764. NUMA awareness support for launching containers. Contributed by Devaraj K. 2018-03-13 12:36:57 -07:00
Yufei Gu ff31d8aefa YARN-8024. LOG in class MaxRunningAppsEnforcer is initialized with a faulty class. Contributed by Sen Zhao. 2018-03-12 16:35:26 -07:00
Billie Rinaldi e1f5251f3c YARN-7523. Introduce description and version field in Service record. Contributed by Chandni Singh 2018-03-10 07:49:10 -08:00
Wangda Tan ea18e70a74 Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
 2018-03-09 22:51:08 -08:00
Owen O'Malley 3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Billie Rinaldi 4f395063bb YARN-8000. Yarn Service component instance name shows up as component name in container record. Contributed by Chandni Singh 2018-03-09 08:50:28 -08:00
Sunil G 113f401f41 YARN-7944. [UI2] Remove master node link from headers of application pages. Contributed by Yesha Vora. 2018-03-08 23:53:36 +05:30
Weiwei Yang b451889e8e YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Tao Yang. 2018-03-08 18:13:36 +08:00
Wangda Tan 583f459431 YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: I16e081f21c5f80160564c49cc49d103bd8eb7e16
 2018-03-07 15:46:47 -08:00
Wangda Tan e718ac597f Revert "YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)" 
This reverts commit 4d53ef7eef.
 2018-03-07 15:46:47 -08:00
Wangda Tan 037d783483 YARN-7626. Allow regular expression matching in container-executor.cfg for devices and named docker volumes mount. (Zian Chen via wangda) 
Change-Id: If461277d4557922ab7e4dce9dd8dc5d0d5f22710
(cherry picked from commit 88f9138e12d2d5a1bd13f0915acef93037c1d086)
 2018-03-07 11:30:15 -08:00
Wangda Tan 4d53ef7eef YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: Ie16e34039d57df50128c73b37516ad0bc7c9590e
 2018-03-07 11:30:06 -08:00
Jason Lowe d69b31f7f7 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-03-07 09:20:34 -06:00
Rohith Sharma K S 55ba49dd07 YARN-7346. Add a profile to allow optional compilation for ATSv2 with HBase-2.0. Contributed by Haibo Chen and Rohith. 2018-03-06 11:55:21 +05:30
Takanobu Asanuma 745190ecdc
HADOOP-15287. JDK9 JavaDoc build fails due to one-character underscore identifiers in hadoop-yarn-common 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 18:15:53 -08:00
Sen Zhao 245751ffdc
YARN-7736. Fix itemization in YARN federation document 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 11:54:24 -08:00
Arun Suresh 1054b48c27 YARN-7972. Support inter-app placement constraints for allocation tags by application ID. (Weiwei Yang via asuresh) 2018-03-05 11:24:17 -08:00
Billie Rinaldi 628be58a4c YARN-7915. Trusted image log message repeated multiple times. Contributed by Shane Kumpf 2018-03-05 09:08:44 -08:00
Weiwei Yang e8c5be63f0 YARN-7995. Remove unnecessary boxings and unboxings from PlacementConstraintParser.java. Contributed by Sen Zhao. 2018-03-05 11:49:43 +08:00
Jason Lowe 83798f15f8 YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 2018-03-02 09:59:46 -06:00
Ray Chiang 923e1773e1 HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang) 2018-03-01 15:36:10 -08:00
Sunil G 22928c0c60 YARN-7637. GPU volume creation command fails when work preserving is disabled at NM. Contributed by Zian Chen. 2018-03-01 22:39:32 +05:30
Yufei Gu 6ca4e3017b YARN-5028. RMStateStore should trim down app state for completed applications(addendum). Contributed by Rohith Sharma K S. 2018-03-01 08:38:53 -08:00
Billie Rinaldi 5ed689e33a YARN-7958. ServiceMaster should only wait for recovery of containers with id that match the current application id. Contributed by Chandni Singh 2018-03-01 08:00:13 -08:00
Haibo Chen 10663b78c8 Revert "yarn-7346.07.patch" 
This reverts commit 5e37ca5bb4.
 2018-02-28 21:11:36 -08:00
Haibo Chen d1274c3b71 YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen) 2018-02-28 21:10:42 -08:00
Haibo Chen 5e37ca5bb4 yarn-7346.07.patch 2018-02-28 21:04:54 -08:00
Eric Yang 17f387e3c2 YARN-7985. Removed duplicate service name validation in Yarn Service. 
Contributed by Chandni Singh
 2018-02-28 20:48:55 -05:00
Billie Rinaldi 315f48e791 YARN-7955. Improve result of calling stop on an already stopped service. Contributed by Gour Saha 2018-02-28 15:01:56 -08:00
Billie Rinaldi 31009034bd YARN-7954. Set component status to STOPPED when YARN service is stopped. Contributed by Gour Saha 2018-02-28 11:57:24 -08:00
Billie Rinaldi 727c033997 YARN-7446. Remove --user flag when running privileged mode docker container. Contributed by Eric Yang 2018-02-27 14:33:57 -08:00
Wangda Tan eea0b069e7 YARN-7959. Add .vm extension to PlacementConstraints.md to ensure proper filtering. (Weiwei Yang via wangda) 
Change-Id: I34ec516994a5de7accaa5ef7044e443b340bd294
(cherry picked from commit 0d516b30769471d9114214b3a518e2146ff2f25e)
 2018-02-27 13:19:24 -08:00
Wangda Tan 5bbbbb6a1e YARN-7893. Document the FPGA isolation feature. (Zhankun Tang via wangda) 
Change-Id: I8723320377a4bd0a7db168670d183ef5543caa67
(cherry picked from commit c2af2f16718bd1d38ceb05e354f34d7bef89127d)
 2018-02-27 13:19:16 -08:00
Eric Yang b4f1ba1413 YARN-7963. Updated MockServiceAM unit test to prevent test hang. 
Contributed by Chandni Singh
 2018-02-26 18:49:01 -05:00
Konstantinos Karanasos e85188101c YARN-7921. Transform a PlacementConstraint to a string expression. Contributed by Weiwei Yang. 2018-02-26 12:15:16 -08:00
Arpit Agarwal 2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Jason Lowe 8e728f39c9 YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan 2018-02-23 15:46:35 -06:00
Sunil G d1cd573687 YARN-7949. [UI2] ArtifactsId should not be a compulsory field for new service. Contributed by Yesha Vora. 2018-02-23 16:50:02 +05:30
Sunil G cc683952d2 YARN-7675. [UI2] Support loading pre-2.8 version /scheduler REST response for queue page. Contributed by Gergely Novák. 2018-02-23 16:10:29 +05:30
Gabor Bota c36b4aa31c
HADOOP-15236. Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-23 13:55:18 +09:00
Carlo Curino 514794e1a5 YARN-7934. [GQ] Refactor preemption calculators to allow overriding for Federation Global Algos. (Contributed by curino) 2018-02-22 18:12:12 -08:00
Eric Yang 95904f6b3c YARN-7942. Add check for JAAS configuration for Yarn Service. 
Contributed by Billie Rinaldi
 2018-02-22 16:12:40 -05:00
Eric Yang 190969006d YARN-7836. Added error check for updating service components. 
(Contributed by Gour Saha)
 2018-02-22 16:08:30 -05:00
Jason Lowe 3132709b46 YARN-7916. Remove call to docker logs on failure in container-executor. Contributed by Shane Kumpf 2018-02-21 16:54:02 -06:00
Yufei Gu 92cbbfe79e YARN-5028. RMStateStore should trim down app state for completed applications. Contributed by Gergo Repas. 2018-02-21 11:42:51 -08:00
Sunil G bdd2a184d7 YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 2018-02-21 14:35:57 +05:30
Sunil G 86b227a1fb YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan. 2018-02-21 14:16:45 +05:30
Eric Yang 7280c5af82 YARN-7940. Fixed a bug in ServiceAM ZooKeeper initialization. 
(Contributed by Billie Rinaldi)
 2018-02-20 14:12:58 -05:00
Jason Lowe 9497215084 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:06:28 -06:00
Jason Lowe b9a429bb28 Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 8013475d44.
 2018-02-19 08:16:25 -06:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Rohith Sharma K S 87bdde6943 YARN-7937. Fix http method name in Cluster Application Timeout Update API example request. Contributed by Charan Hebri. 2018-02-18 14:01:23 +05:30
Rohith Sharma K S 9af30d46c6 YARN-7919. Refactor timelineservice-hbase module into submodules. Contributed by Haibo Chen. 2018-02-17 20:30:28 +05:30
Arun Suresh a1e56a6286 YARN-7918. Fix TestAMRMClientPlacementConstraints. (Gergely Novák via asuresh) 2018-02-17 03:24:55 -08:00
Wangda Tan ca1043ab90 YARN-7328. ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores. (lovekesh bansal via wangda) 
Change-Id: Ibb1faf0beefec079dae2208986976b2f64650672
 2018-02-17 12:28:08 +08:00
Sunil G aae629913c YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan. 2018-02-16 12:37:40 +05:30
Jason Lowe 8013475d44 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-02-15 17:09:00 -06:00
Konstantinos Karanasos 0b489e564c YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan. 2018-02-15 14:23:38 -08:00
Eric Payne bddfe42e2c Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit c5e6e3de1c.
 2018-02-14 14:29:20 -06:00
Wangda Tan 042ef2fa7b YARN-7789. Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator. (Zian Chen via wangda) 
Change-Id: I1f6a451fe16758def6f47c046a9b8a67ad7255af
 2018-02-14 23:11:10 +08:00
Eric Payne c5e6e3de1c YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 2018-02-13 10:11:02 -06:00
Jason Lowe 5a1db60ab1 YARN-7914. Fix exit code handling for short lived Docker containers. Contributed by Shane Kumpf 2018-02-12 15:50:10 -06:00
Jason Lowe 3414fd1243 YARN-7917. Fix failing test TestDockerContainerRuntime#testLaunchContainerWithDockerTokens. Contributed by Shane Kumpf 2018-02-12 09:27:43 -06:00
Wangda Tan d02e42cee4 YARN-7739. DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation. (wangda) 
Change-Id: I10cc9341237d9a2fc0f8c855efb98a36b91389e2
 2018-02-12 10:29:37 +08:00
Wangda Tan d4c98579e3 YARN-7697. NM goes down with OOM due to leak in log-aggregation. (Xuan Gong via wangda) 
Change-Id: Ie4fc7979d834e25f37a033c314f3efceeb8f4a9e
 2018-02-12 10:28:35 +08:00
Wangda Tan 789a185c16 YARN-5848. Remove unnecessary public/crossdomain.xml from YARN UIv2 sub project. (Sunil G via wangda) 
Change-Id: Ie295f88232192e6b520c335b0332383cc6a232c0
 2018-02-12 10:27:15 +08:00
Wangda Tan e795833d8c YARN-7906. Fix mvn site fails with error: Multiple sources of package comments found for package o.a.h.y.client.api.impl. (Akira Ajisaka via wangda) 
Change-Id: I20221d97446e97f208d587eacbc60448c11ffd48
 2018-02-12 10:25:22 +08:00
Weiwei Yang a08c048832 YARN-7838. Support AND/OR constraints in Distributed Shell. Contributed by Weiwei Yang. 2018-02-11 14:20:46 +08:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Eric Yang c97d5bceb2 YARN-7909. Add charset to YARN Service REST API. (Contributed by Eric Yang) 2018-02-09 20:01:25 -05:00
Jian He ddec08d7cc YARN-7827. Stop and Delete Yarn Service from RM UI fails with HTTP ERROR 404. Contributed by Sunil G 2018-02-08 21:32:40 -08:00
Yufei Gu 1bc03ddf97 YARN-7655. Avoid AM preemption caused by RRs for specific nodes or racks. Contributed by Steven Rand. 2018-02-08 12:32:43 -08:00
Jian He eb2449d539 YARN-5428. Allow for specifying the docker client configuration directory. Contributed by Shane Kumpf 2018-02-08 11:35:30 -08:00
Jason Lowe 456705a07c YARN-7815. Make the YARN mounts added to Docker containers more restrictive. Contributed by Shane Kumpf 2018-02-07 13:09:08 -06:00
Billie Rinaldi 266da25c04 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit e14f03dfbf.
 2018-02-07 07:17:00 -08:00