Commit Graph

1224 Commits

Author SHA1 Message Date
Sunil G 758b771943 YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin. 2017-09-12 09:19:12 -07:00
Wangda Tan df3855541a YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda)
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
2017-09-12 09:19:12 -07:00
Wangda Tan b1fe3a222e YARN-7042. Clean up unit tests after YARN-6610. (Daniel Templeton via wangda)
Change-Id: I8e40f704b6fcdd5b14faa9548a27986501044fa1
2017-09-12 09:19:12 -07:00
Sunil G a6989af1d9 YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan. 2017-09-12 09:19:12 -07:00
Sunil G a0030c8c6f YARN-6610. [YARN-3926] DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources. Contributed by Daniel Templeton. 2017-09-12 09:19:12 -07:00
Sunil G 2b51b262ab YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G. 2017-09-12 09:19:12 -07:00
Yufei Gu d5e9939ebb YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu) 2017-09-12 09:19:11 -07:00
Daniel Templeton 3aeaafecb8 YARN-6788. [YARN-3926] Improve performance of resource profile branch
(Contributed by Sunil Govindan via Daniel Templeton)
2017-09-12 09:19:11 -07:00
Sunil G 4e5632d28e YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup. Contributed by Yeliang Cang. 2017-09-12 09:19:11 -07:00
Sunil G ca2808d8bb YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G 58da54640e YARN-6445. [YARN-3926] Performance improvements in resource profile branch with respect to SLS. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G dae65f3bef YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Arun Suresh 6708ac3301 YARN-5587. Add support for resource profiles. (vvasudev via asuresh) 2017-09-12 09:19:10 -07:00
Varun Vasudev c2032e251e YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Varun Vasudev a9b4426302 YARN-5707. Add manager class for resource profiles. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Rohith Sharma K S 239c1824a0 YARN-5586. Update the Resources class to consider all resource types. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Rohith Sharma K S 9e4ba6aff5 YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Varun Vasudev 759114b006 YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Arun Suresh 7ba698997b YARN-4829. Add support for binary units in Resource class.(vvasudev via asuresh) 2017-09-12 09:19:10 -07:00
Varun Vasudev ceb12c59a3 YARN-4715. Add support to read resource types from a config file. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Wangda Tan 32c91223f1 YARN-4172. Extend DominantResourceCalculator to account for all resources. (Varun Vasudev via wangda) 2017-09-12 09:19:09 -07:00
Wangda Tan 1bbab7c157 YARN-4081. Add support for multiple resource types in the Resource class. (Varun Vasudev via wangda) 2017-09-12 09:19:09 -07:00
Junping Du 3fddabc2fe YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong. 2017-09-08 15:16:19 -07:00
Sunil G 56d93d2e39 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 07:15:17 +05:30
Jason Lowe b0b535d9d5 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:17:03 -05:00
Junping Du 22de9449f8 YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong. 2017-09-06 14:53:31 -07:00
Junping Du 63231a2a30 YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 2017-09-06 13:19:34 -07:00
Andrew Wang 0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Junping Du 91cc070d67 YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-08-31 16:41:43 -07:00
Daniel Templeton cc23514aba YARN-7115. Move BoundedAppender to org.hadoop.yarn.util pacakge
(Contributed by Jian He via Daniel Templeton)
2017-08-30 17:26:13 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Varun Saxena 16ba4f544f MAPREDUCE-6838. Addendum to fix code comment 2017-08-30 11:29:54 +05:30
Jian He 08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S e276c75ec1 YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena. 2017-08-30 11:29:54 +05:30
Rohith Sharma K S 7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Jian He ac7f52df83 YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena 2017-08-30 11:29:53 +05:30
Varun Saxena ff43b8db7a YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:52 +05:30
Sangjin Lee 580d884913 YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 2017-08-30 11:29:52 +05:30
Sangjin Lee 643a20a358 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis. 2017-08-30 11:29:51 +05:30
Varun Saxena 9bb5cc3d49 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-08-30 11:29:50 +05:30
Junping Du ad45d19998 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:16:23 -07:00
Junping Du c2cb7ea1ef YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 2017-08-24 13:36:49 -07:00
Arun Suresh 8410d862d3 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 2017-08-20 07:54:09 -07:00
Subru Krishnan bbbf0e2a41 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-11 13:58:45 -07:00
Daniel Templeton 47b145b9b4 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-08-08 10:33:26 -07:00
Junping Du f44b349b81 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-04 16:03:56 -07:00
Arun Suresh 35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Yufei Gu c617fe02b3 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:33:37 -07:00
Carlo Curino acda6b96a4 YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino)
(cherry picked from commit b6240b92ab)
2017-08-01 17:28:28 -07:00
Subru Krishnan 43a97174fe YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).
(cherry picked from commit 52daa6d971)
2017-08-01 17:28:26 -07:00
Subru Krishnan 859aa1f9d6 YARN-5531. UnmanagedAM pool manager for federating application across clusters. (Botong Huang via Subru).
(cherry picked from commit 73bb2102ce)
2017-08-01 17:28:24 -07:00
Subru Krishnan 7444406d6d YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru).
(cherry picked from commit 67846a5519)
2017-08-01 17:28:24 -07:00
Subru Krishnan 80e1904000 YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru).
(cherry picked from commit 4846069061)
2017-08-01 17:28:24 -07:00
Subru Krishnan 86b2bec56e YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru)
(cherry picked from commit 51aeb2ce0c)
2017-08-01 17:28:23 -07:00
Jian He bdfad4523f YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan
(cherry picked from commit 3307564a5f)
2017-08-01 17:28:20 -07:00
Jian He 5c84382397 YARN-3672. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan
(cherry picked from commit 5e976cd2b9)
2017-08-01 17:28:19 -07:00
Subru Krishnan d19b677301 YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui.
(cherry picked from commit d3dc461a93)
2017-08-01 17:28:18 -07:00
Akira Ajisaka 38c6fa5c7a
HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier. 2017-07-28 09:10:52 +09:00
Jason Lowe c8df3668ec YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 2017-07-20 11:03:04 -05:00
Arun Suresh 077fcf6a96 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 2017-07-19 12:26:40 -07:00
Xuan 9845bea59c YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong) 2017-07-12 15:40:45 -07:00
Jason Lowe 655110393b YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe 2017-07-12 15:52:56 -05:00
Naganarasimha 626d730bfc YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt. 2017-07-09 17:26:09 +05:30
Xuan 3be2659f83 YARN-6694. Add certain envs to the default yarn.nodemanager.env-whitelist. Contributed by Jian He 2017-06-29 20:10:35 -07:00
Andrew Wang af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Daniel Templeton 25d891a784 YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml
(Contributed by Lori Loberg via Daniel Templeton)
2017-06-28 13:17:58 -07:00
Sunil G ca13b224b2 YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 2017-06-22 23:50:57 -07:00
Naganarasimha 740204b292 YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt. 2017-06-23 07:52:41 +05:30
Naganarasimha ee89ac84e6 YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp 2017-06-19 22:42:45 +05:30
Naganarasimha 7582dedad1 YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang. 2017-06-18 19:53:42 +05:30
Daniel Templeton 036a24bab3 MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools.
(Contributed by LiXin Ge via Daniel Templeton)
2017-06-13 13:21:23 -07:00
Eric Payne 5578af8603 YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G. 2017-06-09 16:16:34 -05:00
Daniel Templeton a062374c39 YARN-6679. Reduce Resource instance overhead via non-PBImpl
(Contributed by Daryn Sharp via Daniel Templeton)
2017-06-08 10:08:08 -07:00
Haibo Chen 4b4a6524f2 YARN-6477. Dispatcher no longer needs the raw types suppression. (Maya Wexler via Haibo Chen) 2017-05-30 16:58:15 -07:00
Andrew Wang 16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Wangda Tan c583ab02c7 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
2017-05-22 14:26:13 -07:00
Robert Kanter 9855225a79 YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter) 2017-05-19 15:43:22 -07:00
Andrew Wang 543aac9f28 HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2. 2017-05-09 09:22:53 -07:00
Haibo Chen 1769b12a77 YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen) 2017-05-08 15:46:12 -07:00
Jason Lowe 424887ecb7 YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R 2017-05-08 17:14:37 -05:00
Naganarasimha 4b5bd73ac5 YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common. Contributed by Weiwei Yang 2017-05-01 16:31:16 +05:30
Daniel Templeton 2e52789edf YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton) 2017-04-28 13:30:28 -07:00
Junping Du 94607214ce YARN-6534. ResourceManager failed due to TimelineClient try to init SSLFactory even https is not enabled. Contributed by Rohith Sharma K S.
(cherry picked from commit 09e36ff59c05b8bd54d346d1103de0a7300875c6)
2017-04-27 13:11:32 -07:00
Haibo Chen 4f3ca0396a YARN-6510. Fix profs stat file warning caused by process names that includes parenthesis. (Wilfred Spiegelenburg via Haibo Chen) 2017-04-26 11:46:55 -07:00
Daniel Templeton dd43b895c2 YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded
(Contributed by Yufei Gu via Daniel Templeton)
2017-04-19 11:44:55 -07:00
Daniel Templeton 15acdbdba2 YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 10:31:14 -07:00
Sunil G 9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Junping Du 9d9087a67a YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen.
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
2017-04-12 13:43:18 -07:00
Jason Lowe 1b081ca27e YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:24:36 -05:00
Jason Lowe e8071aa249 YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:30:45 -05:00
Varun Saxena 7c2bc444b3 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 2017-03-31 02:02:57 +05:30
Daniel Templeton 640ba1d23f YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 2017-03-29 12:38:11 -07:00
Robert Kanter 9bae6720cb YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-27 17:23:13 -07:00
Wangda Tan cd014d57aa YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 2017-03-27 13:29:09 -07:00
Tsuyoshi Ozawa 84ddedc0b2 HADOOP-10101. Update guava dependency to the latest version. (ozawa) 2017-03-25 10:06:52 +09:00
Junping Du 221783b35b YARN-6294. ATS client should better handle Socket closed case. Contributed by Li Lu. 2017-03-15 11:48:17 -07:00
Junping Du 871dc420f8 YARN-6313. YARN logs cli should provide logs for a completed container even when application is still running. Contributed by Xuan Gong.
(cherry picked from commit b88f5e0f7858d1d89b79dfd325b767c34416052d)
2017-03-14 12:58:12 -07:00
Varun Saxena 229c7c9f89 YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-03-11 04:09:01 +05:30
Jason Lowe 9649c27864 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 2017-03-10 13:06:54 -06:00
Jason Lowe deb9f56946 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 2017-03-10 11:08:33 -06:00
Karthik Kambatla a96afae125 YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha) 2017-03-09 23:11:54 -08:00
Jian He 279d187f72 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:04 -08:00
Robert Kanter 6f6dfe0202 YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter) 2017-03-01 14:53:47 -08:00
Gera Shegalov 480b4dd574 YARN-1728. Regression test for guice-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera) 2017-02-28 09:27:22 -08:00
Xuan 53d372a255 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache
stores. Contributed by Li Lu
2017-02-24 10:59:35 -08:00
Karthik Kambatla 718ad9f6ee YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha) 2017-02-22 15:46:07 -08:00
Wangda Tan 1a6ca75f38 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-22 11:17:09 -08:00
Daniel Templeton c7a36e6130 YARN-6125. The application attempt's diagnostic message should have a maximum size
(Contributed by Andras Piros via Daniel Templeton)
2017-02-17 13:40:58 -08:00
Sangjin Lee 4fa1afdb88 YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 2017-02-16 11:41:04 -08:00
Karthik Kambatla 6c25dbcdc0 YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha) 2017-02-15 23:16:12 -08:00
Junping Du 0fc6f38379 Addendum patch for YARN-6174 - remove deleted file. 2017-02-15 09:07:49 -08:00
Junping Du ce2d5bfa5f YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong. 2017-02-15 09:05:14 -08:00
Arun Suresh aaf106fde3 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 2017-02-14 06:09:10 -08:00
Junping Du 464ff479ce YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 2017-02-13 06:12:54 -08:00
Sunil G 3a0a0a485a YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres Perez. 2017-02-10 15:48:46 +05:30
Junping Du eec52e158b YARN-6145. Improve log message on fail over. Contributed by Jian He. 2017-02-08 02:11:19 -08:00
Junping Du 9dbfab1284 YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong. 2017-02-07 02:13:14 -08:00
Junping Du cce35c3815 HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena. 2017-02-05 19:42:11 -08:00
Junping Du 327c9980aa YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong. 2017-02-02 00:41:18 -08:00
Junping Du 8528d85a68 YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong. 2017-01-24 15:26:53 -08:00
Sangjin Lee dc6ec97048 YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo. 2017-01-23 21:07:25 -08:00
Jason Lowe 69fa81679f YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 2017-01-23 17:12:51 +00:00
Andrew Wang 5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
bibinchundatt a9d946d4df YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer. 2017-01-17 14:16:00 +05:30
Daniel Templeton e6f13fe5d1 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 2017-01-11 16:00:01 -08:00
Junping Du 4db119b7b5 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 2017-01-10 18:20:27 -08:00
Wangda Tan 287d3d6804 YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 2017-01-09 08:40:39 -08:00
Naganarasimha a59df15757 YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-07 08:11:58 +05:30
Daniel Templeton 165d01a73e YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 2016-12-30 08:41:52 -08:00
Robert Kanter 9262797e86 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 2016-12-27 14:14:08 -08:00
Daniel Templeton e216e8e233 HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-27 13:03:16 -08:00
Arun Suresh ac1e5d4f77 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 2016-12-27 12:40:44 -08:00
Arun Suresh 2273a74c1f YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 2016-12-16 08:14:34 -08:00
Steve Loughran f66f61892a HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-12 13:58:27 +00:00
Robert Kanter 4b149a1e77 YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter) 2016-12-09 17:28:25 -08:00
Arun Suresh b0aace21b1 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 2016-12-09 16:41:25 -08:00
Akira Ajisaka 7d8e440eee Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran"
This reverts commit c265515725.
2016-12-09 12:49:27 +09:00
Steve Loughran c265515725 HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran 2016-12-08 17:57:15 +00:00
Sunil G 74d0066d33 YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S. 2016-12-08 15:53:56 +05:30
Robert Kanter 72fe546841 YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter) 2016-12-07 15:07:25 -08:00
Sunil ab923a53fc YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 2016-12-08 00:27:25 +05:30
Jian He 43ebff2e35 YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 2016-12-05 11:39:34 -08:00
Jason Lowe f304ccae3c YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria 2016-12-02 16:54:15 +00:00
Robert Kanter c87b3a448a MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter) 2016-12-01 17:29:38 -08:00
Daniel Templeton b3befc021b YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton) 2016-11-30 09:50:33 -08:00
Daniel Templeton 25f9872be6 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 09:40:49 -08:00
Akira Ajisaka 209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Sunil eb0a483ed0 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 2016-11-24 18:18:42 +05:30
Daniel Templeton 10468529a9 YARN-4752. Improved preemption in FairScheduler. (kasha)
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge

Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
2016-11-23 19:48:59 -10:00
Naganarasimha 3541ed8068 YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena 2016-11-23 14:12:23 +05:30
Naganarasimha 4667564162 YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena. 2016-11-23 08:49:48 +05:30
Akira Ajisaka 6f8074298d YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn. 2016-11-22 10:20:31 +09:00
Rohith Sharma K S d65603517e YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee. 2016-11-16 15:14:00 +05:30
Mingliang Liu 5af572b644 HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu 2016-11-15 10:57:00 -08:00
Arun Suresh 3219b7b4ac YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 2016-11-15 07:56:25 -08:00
Wangda Tan ede1a473f5 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 2016-11-11 13:34:56 -08:00
Eric Payne 93eeb13164 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 22:35:12 +00:00
Jian He bcc15c6290 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:08:05 -08:00
Arun Suresh 280357c29f YARN-5833. Addendum patch to include missing changes to yarn-default.xml 2016-11-09 07:15:11 -08:00
Andrew Wang e1c6ef2efa HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 2016-11-08 16:33:55 -08:00
Jason Lowe 3f93ac0733 YARN-5356. NodeManager should communicate physical resource capability to ResourceManager. Contributed by Inigo Goiri 2016-11-08 22:01:26 +00:00
Wangda Tan c00b5d1e51 YARN-5804. New UI2 is not able to launch with jetty 9 upgrade post HADOOP-10075. (Sunil G via wangda)
(cherry picked from commit ddebaa6b7ae1031899139d153d57df5387d60542)
2016-11-06 13:13:31 -08:00
Wangda Tan 3de0da2a76 YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan c85cc3b56e YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda)
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
2016-11-06 13:13:31 -08:00
Varun Saxena 07ab89e8bb YARN-5809. AsyncDispatcher possibly invokes multiple shutdown threads when handling exception (Jian He via Varun Saxena) 2016-11-01 04:37:03 +05:30
Daniel Templeton 773c60bd7b YARN-5800. Delete LinuxContainerExecutor comment from yarn-default.xml (Contributed by Jan Hentschel via Daniel Templeton) 2016-10-31 14:47:48 -07:00
Naganarasimha 3de130252d YARN-5794. Fix the asflicense warnings. Contributed by Yiqun Lin 2016-10-28 12:37:37 +05:30
Robert Kanter 5877f20f9c HADOOP-10075. Update jetty dependency to version 9 (rkanter) 2016-10-27 16:09:00 -07:00
Li Lu 221582c4ab YARN-4668. Reuse objectMapper instance in Yarn. (Yiqun Lin via gtcarrera9) 2016-10-27 15:20:17 -07:00
Varun Saxena dd4ed6a587 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:22:25 +05:30
Karthik Kambatla de6faae97c YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 13:35:47 -07:00
Xiao Chen c88c1dc50c YARN-5777. TestLogsCLI#testFetchApplictionLogsAsAnotherUser fails. Contributed by Akira Ajisaka. 2016-10-25 12:15:09 -07:00
Subru Krishnan 0a166b1347 YARN-5711. Propogate exceptions back to client when using hedging RM failover provider. 2016-10-24 18:59:51 -07:00
Robert Kanter 23d7d53a41 YARN-5679. TestAHSWebServices is failing (ajisakaa via rkanter) 2016-10-21 13:31:03 -07:00
Xuan b733a6f862 YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du. 2016-10-18 11:06:47 -07:00
Karthik Kambatla 6378845f9e YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha) 2016-10-11 21:42:08 -07:00
Jason Lowe ecb51b857a YARN-5551. Ignore file backed pages from memory computation when smaps is enabled. Contributed by Rajesh Balamohan 2016-10-11 15:12:43 +00:00
Chris Douglas d65b957776 YARN-5702. Refactor TestPBImplRecords for reuse in other YARN modules. Contributed by Subru Krishnan 2016-10-05 12:16:49 -07:00
Karthik Kambatla 736d33cddd YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha) 2016-10-03 14:35:57 -07:00
Naganarasimha 6e130c308c YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 02:02:26 -04:00
Subru Krishnan 3a3697deab YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru). 2016-09-30 19:41:43 -07:00
Jian He 2ae5a3a5bf YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 2016-09-29 22:00:31 +08:00
Varun Saxena 9b0fd01d2e YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-09-28 16:10:10 +05:30
Varun Vasudev bc2656f09f YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 2016-09-28 15:18:18 +05:30
Arun Suresh 4815d024c5 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 08:46:54 -07:00
Arun Suresh 2f163cd5cf Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh"
This reverts commit fe644bafe7.
2016-09-26 08:36:59 -07:00
Jian He fe644bafe7 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 22:41:16 +08:00
Varun Saxena b8a2d7b8fc YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:27:31 +05:30
Varun Saxena 3293a7d92d Revert "TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out"
This reverts commit 2e6ee95716.
2016-09-23 13:25:46 +05:30
Varun Saxena 2e6ee95716 TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out 2016-09-23 13:23:54 +05:30
Naganarasimha d0372dc613 YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-23 06:30:49 +05:30
Robert Kanter 5ffd4b7c1e YARN-4973. YarnWebParams next.fresh.interval should be next.refresh.interval (templedf via rkanter) 2016-09-22 16:45:34 -07:00
Wangda Tan 5a58bfee30 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 2016-09-20 17:20:50 -07:00
Naganarasimha 4174b9756c YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani 2016-09-17 10:35:39 +05:30
Naganarasimha ade7c2bc9c YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 2016-09-17 09:52:39 +05:30
Ray Chiang 378f624a39 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 11:14:35 -07:00
Arun Suresh d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Subru Krishnan b930dc3ec0 YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 2016-08-26 16:58:47 -07:00
Junping Du 9ef632f3b0 YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 2016-08-26 09:04:44 -07:00
Varun Vasudev bfb4d95059 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:30:18 +05:30
Junping Du 0da69c324d YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi.
(cherry picked from commit d464483bf7)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
2016-08-18 07:27:23 -07:00
Varun Saxena 913a895136 YARN=5526. DrainDispacher#serviceStop blocked if setDrainEventsOnStop invoked (sandflee via Varun Saxena) 2016-08-18 11:38:13 +05:30
Varun Saxena b427ce12bc YARN-5475. Fix test failure of TestAggregatedLogFormat#testReadAcontainerLogs1 (Jun Gong via Varun Saxena) 2016-08-16 20:24:53 +05:30
Ray Chiang aea3e65749 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 2016-08-10 19:23:29 -07:00
Arun Suresh 82c9e06101 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 2016-08-09 00:42:29 -07:00
Junping Du e605d47df0 YARN-5470. Differentiate exactly match with regex in yarn log CLI. Contributed by Xuan Gong. 2016-08-05 10:55:29 -07:00
Arun Suresh e5766b1dbe YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 2016-07-31 11:48:25 -07:00
Li Lu 7086fc72ee YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9) 2016-07-28 16:50:57 -07:00
Jason Lowe a1890c32c5 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 2016-07-28 21:47:13 +00:00
Junping Du 7f3c306e2e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 2016-07-28 06:35:24 -07:00
Subru Krishnan d2cbfd7de3 YARN-5441. Fixing minor Scheduler test case failures 2016-07-27 18:21:58 -07:00
Akira Ajisaka 54fe17a607 HADOOP-9427. Use JUnit assumptions to skip platform-specific tests. Contributed by Gergely Novák. 2016-07-27 19:41:09 +09:00
Arun Suresh 2d8d183b19 YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh) 2016-07-26 19:08:30 -07:00
Varun Vasudev 9ccf935b2c YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 2016-07-20 12:48:32 +05:30
Junping Du dc2f4b6ac8 YARN-5213. Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently. Contributed by Xuan Gong. 2016-07-19 07:17:58 -07:00
Andrew Wang da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Junping Du 7e5355c14e YARN-5339. Passing file to -out for YARN log CLI doesnt give warning or error code. Contributed by Xuan Gong. 2016-07-15 09:23:11 -07:00
Vinod Kumar Vavilapalli eb47163234 YARN-5200. Enhanced "yarn logs" to be able to get a list of containers whose logs are aggregated via a "show_container_log_info" option. Contributed by Xuan Gong. 2016-07-13 10:54:15 -07:00
Robert Kanter bf6f4a3b98 YARN-4366. Fix Lint Warnings in YARN Common (templedf via rkanter) 2016-07-12 15:12:25 -07:00
Jian He 819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Sangjin Lee 6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Varun Saxena 1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Li Lu 0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Li Lu c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee 39cce4e629 YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee) 2016-07-10 08:45:56 -07:00
Varun Saxena a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee 84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Sangjin Lee 9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Sangjin Lee 0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee 960af7d471 YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee) 2016-07-10 08:45:53 -07:00
Naganarasimha 6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Varun Saxena f5380d850e YARN-4622. TestDistributedShell fails for v2 test cases after modifications for 1.5. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:51 -07:00
Li Lu 34f02f07d5 Rebase to latest trunk 2016-07-10 08:45:51 -07:00
Sangjin Lee cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Li Lu 8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase.
Contributed by Zhan Zhang.
2016-07-10 08:45:49 -07:00
Xuan 2e2dbf59d1 YARN-4392. ApplicationCreatedEvent event time resets after RM
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
2016-07-10 08:45:49 -07:00
Li Lu b7d484aead YARN-4450. TestTimelineAuthenticationFilter and
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
2016-07-10 08:45:49 -07:00
Li Lu 89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no
impact when it's turned off. Contributed by Sangjin Lee.
2016-07-10 08:45:48 -07:00
Junping Du cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong.
(cherry picked from commit f910e4f639)
2016-07-10 08:45:48 -07:00
Vrushali 10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Junping Du 8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee.
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
2016-07-10 08:45:39 -07:00
Sangjin Lee 57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee)
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
2016-07-10 08:45:39 -07:00
Sangjin Lee e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee)
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
2016-07-10 08:45:39 -07:00
Sangjin Lee 22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee)
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
2016-07-10 08:45:38 -07:00
Zhijie Shen 42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du.
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
2016-07-10 08:45:37 -07:00
Sangjin Lee fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale)
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
2016-07-10 08:45:35 -07:00
Junping Du 2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen.
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
2016-07-10 08:45:34 -07:00
Junping Du 47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
2016-07-10 08:45:33 -07:00
Zhijie Shen 5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du 5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Junping Du 2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen 9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen 309592b9e5 YARN-3240. Added ".keep" file which was missed before.
(cherry picked from commit 6b4dc96414b3f15537670c67a0dccd048862f4f4)
2016-07-10 08:45:29 -07:00
Junping Du 4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
2016-07-10 08:45:29 -07:00
Zhijie Shen c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen.
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
2016-07-10 08:45:29 -07:00
Jian He 8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Wangda Tan 04f6ebb66a YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 10:36:48 -07:00
Xuan 6ab5aa1c1f YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong 2016-06-22 21:48:49 -07:00
Arun Suresh 99e5dd68d0 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 2016-06-22 19:04:54 -07:00
Tsuyoshi Ozawa 5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Junping Du d0162f2040 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 2016-06-19 17:44:54 -07:00
Junping Du c35fa4a0e5 YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 2016-06-17 08:24:24 -07:00
Junping Du 5dfc38ff57 YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-15 16:17:54 -07:00
Varun Vasudev 25064fb2fb YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong. 2016-06-15 12:42:44 +05:30
Wangda Tan c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan eec835ec17 YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 2016-06-13 21:57:33 -07:00
Arun Suresh 5143277958 YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh) 2016-06-12 09:42:38 -07:00
Wangda Tan 03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Vinod Kumar Vavilapalli 9378d9428f YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 2016-06-09 12:30:58 -07:00
Naganarasimha 1500a0a300 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 2016-06-09 05:41:09 +05:30
Wangda Tan 620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Ming Ma 4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Xuan 0bc05e40fa YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong 2016-06-01 13:44:21 -07:00
Varun Vasudev 86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev 42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Arun Suresh aa975bc781 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 2016-05-27 14:06:32 -07:00
Xuan bde819abbb YARN-5089. Improve "yarn log" command-line "logFiles" option to support
regex. Contributed by Xuan Gong
2016-05-26 23:49:20 -07:00
Rohith Sharma K S 0a544f8a3e YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt. 2016-05-27 10:44:35 +05:30
Arun Suresh 5b41b288d0 YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh) 2016-05-26 14:56:37 -07:00
Carlo Curino 013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Karthik Kambatla 77d5ce9069 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 2016-05-25 10:50:54 -07:00
Robert Kanter e07519b8db YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter) 2016-05-25 10:25:44 -07:00
Naganarasimha b4078bd17b YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 2016-05-24 08:06:53 +08:00
Varun Vasudev 0287c49107 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 2016-05-20 12:28:14 +05:30
Junping Du 22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
(cherry picked from commit 2703ec6871)
2016-05-19 14:15:21 -07:00
Varun Vasudev ef1757790d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 2016-05-18 22:44:33 +05:30
Andrew Wang 3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
This reverts commit 6b53802cba.
2016-05-12 15:32:45 -07:00
Andrew Wang 6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Vinod Kumar Vavilapalli 87f5e35133 YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 2016-05-09 22:41:18 -07:00
Jason Lowe d0da13229c YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-05-05 14:07:54 +00:00
Vinod Kumar Vavilapalli 9e37fe3b7a YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 2016-05-04 14:16:03 -07:00
Jason Lowe c6b4839168 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 2016-05-03 17:27:28 +00:00
Varun Vasudev 0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Jian He 4a8508501b YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 2016-04-28 21:54:11 -07:00
Jian He dd80042c42 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:27:25 -07:00
Varun Vasudev 66b07d8374 YARN-4966. Improve yarn logs to fetch container logs without specifying nodeId. Contributed by Xuan Gong. 2016-04-27 15:56:36 +05:30
Arun Suresh 341888a0aa YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 2016-04-26 20:12:12 -07:00