Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
26,621 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

980 Commits

Author SHA1 Message Date
Arun Suresh 38af237969 YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 8779a35742 YARN-7709. Remove SELF from TargetExpression type. (Konstantinos Karanasos via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh a5c1fc881e YARN-7696. Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh a52d11fb8c YARN-7613. Implement Basic algorithm for constraint based placement. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh f9af15d659 YARN-7612. Add Processor Framework for Rich Placement Constraints. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 06eb63e64b YARN-7669. API and interface modifications for placement constraint processor. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 69de9a1ba9 YARN-7448. [API] Add SchedulingRequest to the AllocateRequest. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Konstantinos Karanasos db928556c8 YARN-6595. [API] Add Placement Constraints at the application level. (Arun Suresh via kkaranasos) 2018-01-31 01:30:17 -08:00
Wangda Tan b57e8bc300 YARN-6594. [API] Introduce SchedulingRequest object. (Konstantinos Karanasos via wangda) 2018-01-31 01:30:17 -08:00
Wangda Tan 33a796d9b7 YARN-6593. [API] Introduce Placement Constraint object. (Konstantinos Karanasos via wangda) 
Change-Id: Id00edb7185fdf01cce6e40f920cac3585f8cbe9c
 2018-01-31 01:30:17 -08:00
Haibo Chen 649ef7ac33 YARN-7064. Use cgroup to get container resource utilization. (Miklos Szegedi via Haibo Chen) 2018-01-26 16:27:31 -08:00
Eric Yang 97fe3cc187 YARN-7729. Add support for setting Docker PID namespace mode. (Contributed by Billie Rinaldi) 2018-01-22 16:33:38 -05:00
Sunil G 06cceba1cb YARN-7738. CapacityScheduler: Support refresh maximum allocation for multiple resource types. Contributed by Wangda Tan. 2018-01-18 19:05:26 +05:30
Rohith Sharma K S d09058b2fd YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch. 2018-01-16 07:58:29 +05:30
Wangda Tan edcc3a95d5 YARN-7468. Provide means for container network policy control. (Xuan Gong via wangda) 
Change-Id: I73678c343f663412917758feef35d8308c216e76
 2018-01-12 16:14:10 -08:00
Eric Yang 3d65dbe032 YARN-5366. Improve signal handling and delete delay for Docker on Yarn. 
(Contributed by Shane Kumpf)
 2018-01-12 13:41:27 -05:00
Wangda Tan a81144daa0 YARN-7666. Introduce scheduler specific environment variable support in ApplicationSubmissionContext for better scheduling placement configurations. (Sunil G via wangda) 
Change-Id: I0fd826490f5160d47d42af2a9ac0bd8ec4e959dc
 2018-01-05 15:12:04 -08:00
Robert Kanter f8e7dd9b10 YARN-7557. It should be possible to specify resource types in the fair scheduler increment value (grepas via rkanter) 2018-01-05 11:15:06 -08:00
Varun Saxena c0aeb666a4 YARN-7662. [ATSv2] Define new set of configurations for reader and collectors to bind (Rohith Sharma K S via Varun Saxena) 2017-12-19 22:29:24 +05:30
Akira Ajisaka 001008958d
YARN-7664. Several javadoc errors. Contributed by Sean Mackrory. 2017-12-18 22:24:51 +09:00
Sunil G 2564b4d07f YARN-7119. Support multiple resource types in rmadmin updateNodeResource command. Contributed by Manikandan R. 2017-12-14 16:11:57 +05:30
Sunil G 10fc8d2a7d YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák. 2017-12-13 22:00:07 +05:30
Wangda Tan d52627a7cb YARN-7254. UI and metrics changes related to absolute resource configuration. (Sunil G via wangda) 2017-12-07 18:56:54 -08:00
Wangda Tan 0311cf0535 YARN-7381. Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled by default in yarn-default.xml. (Xuan Gong via wangda) 
Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
 2017-12-05 13:09:49 -08:00
Jason Lowe 25df505421 YARN-7589. TestPBImplRecords fails with NullPointerException. Contributed by Daniel Templeton 2017-12-01 15:37:36 -06:00
Wangda Tan 7225ec0ceb YARN-6507. Add support in NodeManager to isolate FPGA devices with CGroups. (Zhankun Tang via wangda) 
Change-Id: Ic9afd841805f1035423915a0b0add5f3ba96cf9d
 2017-12-01 10:50:49 -08:00
Daniel Templeton 8498d287cd YARN-7541. Node updates don't update the maximum cluster capability for resources other than CPU and memory 2017-11-29 11:11:36 -08:00
Yufei Gu d8923cdbf1 YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu) 2017-11-27 14:31:52 -08:00
Jian He fedabcad42 YARN-6168. Restarted RM may not inform AM about all existing containers. Contributed by Chandni Singh 2017-11-27 10:19:58 -08:00
Arun Suresh b46ca7e73b YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh) 2017-11-22 19:18:30 -08:00
Sunil G aab439593b YARN-7330. Add support to show GPU in UI including metrics. Contributed by Wangda Tan. 2017-11-23 07:54:20 +05:30
Haibo Chen 67bbbe1c0c YARN-7531. ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType(). 2017-11-21 09:09:16 -08:00
Subru Krishnan d5f66888b8 YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru). 2017-11-17 17:39:06 -08:00
Varun Vasudev 5f0b238a11 YARN-7430. Enable user re-mapping for Docker containers by default. Contributed by Eric Yang. 2017-11-17 12:04:47 +05:30
Daniel Templeton e094eb74b9 YARN-6953. Clean up ResourceUtils.setMinimumAllocationForMandatoryResources() and setMaximumAllocationForMandatoryResources() 
(Contributed by Manikandan R via Daniel Templeton)
 2017-11-15 10:03:29 -08:00
Jian He 5323b0048b YARN-7466. ResourceRequest has a different default for allocationRequestId than Container. Contributed by Chandni Singh 2017-11-13 15:37:39 -08:00
Daniel Templeton 2e512f016e YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton) 2017-11-13 11:03:30 -08:00
bibinchundatt 2c2b7a3672 YARN-7406. Moving logging APIs over to slf4j in hadoop-yarn-api. Contributed by Yeliang Cang. 2017-11-11 10:36:27 +05:30
Wangda Tan dd07038ffa YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda) 
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
 2017-11-09 14:51:15 -08:00
Daniel Templeton 462f6c490e YARN-7143. FileNotFound handling in ResourceUtils is inconsistent 
Change-Id: Ib1bb487e14a15edd2b5a42cf5078c5a2b295f069
 2017-11-09 11:58:49 -08:00
Jian He c723021579 YARN-6626. Embed REST API service into RM. Contributed by Eric Yang 2017-11-06 13:30:17 -08:00
Jian He 673c0db43c Revert "YARN-6626. Embed REST API service into RM. Contributed by Eric Yang" 
This reverts commit 63d1084e9781e0fee876916190b69f6242dd00e4.
 2017-11-06 13:30:17 -08:00
Jian He 9e677fa05c YARN-6626. Embed REST API service into RM. Contributed by Eric Yang 2017-11-06 13:30:17 -08:00
Sunil G 1700adc6f7 YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. Contributed by Wangda Tan. 2017-11-03 07:25:29 +05:30
Daniel Templeton d64736d589 YARN-7336. Unsafe cast from long to int Resource.hashCode() method 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-10-30 12:41:28 -07:00
Daniel Templeton 9a7e810838 YARN-6927. Add support for individual resource types requests in MapReduce 
(Contributed by Gergo Repas via Daniel Templeton)
 2017-10-30 11:04:22 -07:00
Daniel Templeton 9711b78998 YARN-7374. Improve performance of DRF comparisons for resource types in fair scheduler 2017-10-29 18:54:33 -07:00
Sunil G 9114d7a5a0 YARN-7224. Support GPU isolation for docker container. Contributed by Wangda Tan. 2017-10-29 11:08:44 +05:30
Jason Lowe 665bb147aa YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 2017-10-27 16:56:05 -05:00
Wangda Tan 36e158ae98 YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda) 
Change-Id: I14c5ea7252b7c17e86ab38f692b5f9d43196dbe0
 2017-10-26 20:15:19 -07:00
Robert Kanter b1de78619f YARN-7262. Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow (rkanter) 2017-10-26 17:47:32 -07:00
Subru Krishnan 25932da6d1 YARN-5516. Add REST API for supporting recurring reservations. (Sean Po via Subru). 2017-10-26 12:10:14 -07:00
Yufei Gu 7a27c2c367 YARN-7270. Fix unsafe casting from long to int for class Resource and 
its sub-classes. (Yufei)
 2017-10-13 12:41:59 -07:00
Wangda Tan 8bcc49e677 YARN-7205. Log improvements for the ResourceUtils. (Sunil G via wangda) 
Change-Id: I0f5b7a7f68ec5d3e1d52211f83fdd089bc0bfd37
 2017-10-11 15:25:28 -07:00
Sunil G fa5cfc68f3 YARN-6620. Add support in NodeManager to isolate GPU devices by using CGroups. Contributed by Wangda Tan. 2017-10-11 23:44:33 +05:30
Wangda Tan 5cfbad68c0 YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan) 
Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c
(cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)
 2017-10-09 20:36:57 -07:00
Jonathan Hung 09c5dfe937 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:11:23 -07:00
Wangda Tan ff39c0de20 YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) 
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
 2017-10-09 11:11:23 -07:00
Xuan 74ba6ffa0b YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung 2017-10-09 11:11:23 -07:00
Wangda Tan a4e6253046 YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) 
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
 2017-10-09 11:11:23 -07:00
Jonathan Hung ef686635c1 YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store 2017-10-09 11:11:23 -07:00
Haibo Chen c071aad5da YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen) 2017-10-05 15:50:13 -07:00
Arun Suresh b733348dde YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 2017-10-05 09:58:04 -07:00
Ray Chiang 67e12169b0 YARN-7219. Make AllocateRequestProto compatible with branch-2/branch-2.8. (rchiang) 2017-10-03 15:06:07 -07:00
Akira Ajisaka c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 2017-10-04 07:01:54 +09:00
Subru Krishnan d4d2fd1acd YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru). 2017-10-02 18:14:44 -07:00
Jason Lowe 85d81ae58e YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh 2017-09-28 14:10:15 -05:00
Jason Lowe bfd1a72ba8 YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger. 2017-09-21 17:41:34 -05:00
Arun Suresh 56ef5279c1 YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 2017-09-18 15:49:31 -07:00
Jason Lowe a4f9c7c924 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 2017-09-18 10:16:09 -05:00
Yufei Gu 09b476e6da YARN-6612. Update fair scheduler policies to be aware of resource types. (Contributed by Daniel Templeton via Yufei Gu) 2017-09-14 11:23:37 -07:00
Arun Suresh 4f8194430f YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh) 2017-09-14 08:51:26 -07:00
Arun Suresh 864fbacd45 YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh) 2017-09-14 08:48:39 -07:00
Rohith Sharma K S 5324388cf2 YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G. 2017-09-13 23:36:47 +05:30
Sunil G da0b6a354b YARN-7137. [YARN-3926] Move newly added APIs to unstable in YARN-3926 branch. Contributed by Wangda Tan. 2017-09-12 09:19:12 -07:00
Daniel Templeton bf2b687412 YARN-7136. Additional Performance Improvement for Resource Profile Feature 
(Contributed by Wangda Tan via Daniel Templeton)
 2017-09-12 09:19:12 -07:00
Sunil G 805095496d YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R. 2017-09-12 09:19:12 -07:00
Wangda Tan 53df3eac50 YARN-7093. Improve log message in ResourceUtils. (Sunil G via wangda) 
Change-Id: I88928a747ee3eec17bc76fb71e9aaa632d091f0f
 2017-09-12 09:19:12 -07:00
Wangda Tan e490602e9b YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda) 
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
 2017-09-12 09:19:12 -07:00
Sunil G a333ba54e3 YARN-7067. [YARN-3926] Optimize ResourceType information display in UI. Contributed by Wangda Tan. 2017-09-12 09:19:12 -07:00
Wangda Tan 39240b61a1 YARN-7043. Cleanup ResourceProfileManager. (wangda) 
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
 2017-09-12 09:19:12 -07:00
Sunil G 758b771943 YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin. 2017-09-12 09:19:12 -07:00
Wangda Tan df3855541a YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda) 
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
 2017-09-12 09:19:12 -07:00
Sunil G a6989af1d9 YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan. 2017-09-12 09:19:12 -07:00
Sunil G 2b51b262ab YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G. 2017-09-12 09:19:12 -07:00
Yufei Gu d5e9939ebb YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu) 2017-09-12 09:19:11 -07:00
Daniel Templeton 3aeaafecb8 YARN-6788. [YARN-3926] Improve performance of resource profile branch 
(Contributed by Sunil Govindan via Daniel Templeton)
 2017-09-12 09:19:11 -07:00
Sunil G ca2808d8bb YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G 58da54640e YARN-6445. [YARN-3926] Performance improvements in resource profile branch with respect to SLS. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G dae65f3bef YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Sunil G 7805deed48 YARN-5588. [YARN-3926] Add support for resource profiles in distributed shell. Contributed by Varun Vasudev. 2017-09-12 09:19:11 -07:00
Arun Suresh 6708ac3301 YARN-5587. Add support for resource profiles. (vvasudev via asuresh) 2017-09-12 09:19:10 -07:00
Varun Vasudev c2032e251e YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Varun Vasudev a9b4426302 YARN-5707. Add manager class for resource profiles. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Rohith Sharma K S 9e4ba6aff5 YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Varun Vasudev 759114b006 YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Arun Suresh 7ba698997b YARN-4829. Add support for binary units in Resource class.(vvasudev via asuresh) 2017-09-12 09:19:10 -07:00
Varun Vasudev ceb12c59a3 YARN-4715. Add support to read resource types from a config file. Contributed by Varun Vasudev. 2017-09-12 09:19:10 -07:00
Wangda Tan 1bbab7c157 YARN-4081. Add support for multiple resource types in the Resource class. (Varun Vasudev via wangda) 2017-09-12 09:19:09 -07:00
Rohith Sharma K S 4a83170be4 YARN-7140. CollectorInfo should have Public visibility. Contributed by Varun Saxena. 2017-09-08 20:23:16 +05:30
Sunil G 56d93d2e39 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 07:15:17 +05:30
Jason Lowe b0b535d9d5 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:17:03 -05:00
Subru Krishnan b6e7d13696 YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 2017-09-06 16:46:01 -07:00
Subru Krishnan e3345e985b Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem." 
This reverts commit 7996eca7dc.
 2017-09-06 16:39:23 -07:00
Subru Krishnan 7996eca7dc Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 2017-09-01 15:16:40 -07:00
Oleg Danilov c5281a85e1
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-09-01 21:33:31 +02:00
Andrew Wang 0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Jian He 08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Haibo Chen 9b08f365d7 YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen) 2017-08-30 11:29:54 +05:30
Rohith Sharma K S 32188d3295 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 2017-08-30 11:29:54 +05:30
Jason Lowe d5ff965fee YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C 2017-08-30 11:29:53 +05:30
Rohith Sharma K S 7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Sangjin Lee 580d884913 YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 2017-08-30 11:29:52 +05:30
Varun Saxena 02a9710a09 YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:51 +05:30
Sangjin Lee ecbc8d4816 YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen. 
(cherry picked from commit a5a55a54ab)
 2017-08-30 11:29:51 +05:30
Sangjin Lee 643a20a358 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis. 2017-08-30 11:29:51 +05:30
Sangjin Lee 05ff04439e YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S. 2017-08-30 11:29:51 +05:30
Varun Saxena 9bb5cc3d49 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-08-30 11:29:50 +05:30
Li Lu e72e6260da YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9) 2017-08-30 11:29:50 +05:30
Carlo Curino cc8893edc0 YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino) 2017-08-29 14:53:09 -07:00
Junping Du c2cb7ea1ef YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 2017-08-24 13:36:49 -07:00
Arun Suresh 8410d862d3 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 2017-08-20 07:54:09 -07:00
Subru Krishnan de462da04e YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru). 2017-08-16 11:43:24 -07:00
Carlo Curino cc59b5fb26 YARN-6896. Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution). (Contributed by Giovanni Matteo Fumarola via curino) 2017-08-11 15:58:01 -07:00
Subru Krishnan bbbf0e2a41 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-11 13:58:45 -07:00
Junping Du f44b349b81 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-04 16:03:56 -07:00
Arun Suresh 35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Carlo Curino acda6b96a4 YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit b6240b92ab)
 2017-08-01 17:28:28 -07:00
Subru Krishnan 86f05c3312 YARN-6866. Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk. (Botong Huang via Subru). 
(cherry picked from commit 40453879ec)
 2017-08-01 17:28:28 -07:00
Subru Krishnan 43a97174fe YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 52daa6d971)
 2017-08-01 17:28:26 -07:00
Subru Krishnan 7444406d6d YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 67846a5519)
 2017-08-01 17:28:24 -07:00
Subru Krishnan 80e1904000 YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 4846069061)
 2017-08-01 17:28:24 -07:00
Carlo Curino 5272af8c7e YARN-3663. Federation State and Policy Store (DBMS implementation). (Giovanni Matteo Fumarola via curino). 
(cherry picked from commit be99c1fe2e)
 2017-08-01 17:28:24 -07:00
Subru Krishnan 86b2bec56e YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru) 
(cherry picked from commit 51aeb2ce0c)
 2017-08-01 17:28:23 -07:00
Subru Krishnan 4128c9522d YARN-5676. Add a HashBasedRouterPolicy, and small policies and test refactoring. (Carlo Curino via Subru). 
(cherry picked from commit 575137f41c)
 2017-08-01 17:28:21 -07:00
Subru Krishnan 083a05bc14 YARN-5634. Simplify initialization/use of RouterPolicy via a RouterPolicyFacade. (Carlo Curino via Subru). 
(cherry picked from commit d7672ce2bd)
 2017-08-01 17:28:21 -07:00
Jian He 2797507d51 YARN-5601. Make the RM epoch base value configurable. Contributed by Subru Krishnan 
(cherry picked from commit 9ca2aba9cc)
 2017-08-01 17:28:20 -07:00
Jian He 94a24567d6 YARN-3671. Integrate Federation services with ResourceManager. Contributed by Subru Krishnan 
(cherry picked from commit 8573c286e2)
 2017-08-01 17:28:20 -07:00
Jian He bdfad4523f YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan 
(cherry picked from commit 3307564a5f)
 2017-08-01 17:28:20 -07:00
Jian He 5c84382397 YARN-3672. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan 
(cherry picked from commit 5e976cd2b9)
 2017-08-01 17:28:19 -07:00
Subru Krishnan d19b677301 YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui. 
(cherry picked from commit d3dc461a93)
 2017-08-01 17:28:18 -07:00
Jason Lowe 24853bf32a YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 2017-07-24 10:19:20 -05:00
Arun Suresh 077fcf6a96 YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh) 2017-07-19 12:26:40 -07:00
Xuan 9845bea59c YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong) 2017-07-12 15:40:45 -07:00
Arun Suresh 5496a34c0c YARN-6776. Refactor ApplicaitonMasterService to move actual processing logic to a separate class. (asuresh) 2017-07-10 14:34:58 -07:00
Andrew Wang af2773f609 Updating version for 3.0.0-beta1 development 2017-06-29 17:57:40 -07:00
Junping Du 4e3eebc943 YARN-5311. Document graceful decommission CLI and usage. Contributed by Elek, Marton. 2017-06-28 15:32:04 -07:00
Naganarasimha 740204b292 YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt. 2017-06-23 07:52:41 +05:30
Daniel Templeton a062374c39 YARN-6679. Reduce Resource instance overhead via non-PBImpl 
(Contributed by Daryn Sharp via Daniel Templeton)
 2017-06-08 10:08:08 -07:00
Andrew Wang 16ad896d5c Update maven version for 3.0.0-alpha4 development 2017-05-26 14:09:44 -07:00
Daniel Templeton d81372dfad YARN-6646. Modifier 'static' is redundant for inner enums 
(Contributed by ZhangBing Lin via Daniel Templeton)
 2017-05-26 12:05:48 -07:00
Robert Kanter 18c494a00c YARN-6447. Provide container sandbox policies for groups (gphillips via rkanter) 2017-05-16 18:02:39 -07:00
Jason Lowe 424887ecb7 YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R 2017-05-08 17:14:37 -05:00
Daniel Templeton 2e52789edf YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton) 2017-04-28 13:30:28 -07:00
Daniel Templeton 46940d92e2 YARN-6302. Fail the node if Linux Container Executor is not configured properly 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-04-19 12:23:49 -07:00
Sunil G 9ff25408bc YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu. 2017-04-19 20:39:09 +05:30
Daniel Templeton e9ac61cc0e YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 2017-04-10 14:56:42 -07:00
Varun Saxena 7c2bc444b3 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 2017-03-31 02:02:57 +05:30
Chris Douglas 7297e82cb4 HADOOP-14250. Correct spelling of 'separate' and variants. Contributed by Doris Gu 2017-03-30 10:17:11 -07:00
Daniel Templeton 640ba1d23f YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled 2017-03-29 12:38:11 -07:00
Robert Kanter 9bae6720cb YARN-6050. AMs can't be scheduled on racks or nodes (rkanter) 2017-03-27 17:23:13 -07:00
Robert Kanter 6f6dfe0202 YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter) 2017-03-01 14:53:47 -08:00
Xuan 53d372a255 YARN-6228: EntityGroupFSTimelineStore should allow configurable cache 
stores. Contributed by Li Lu
 2017-02-24 10:59:35 -08:00
Wangda Tan 1a6ca75f38 YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 2017-02-22 11:17:09 -08:00
Daniel Templeton c7a36e6130 YARN-6125. The application attempt's diagnostic message should have a maximum size 
(Contributed by Andras Piros via Daniel Templeton)
 2017-02-17 13:40:58 -08:00
Rohith Sharma K S a393e84c6f YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit b7ac85259c.
 2017-02-16 18:00:25 +05:30
Arun Suresh aaf106fde3 YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh) 2017-02-14 06:09:10 -08:00
Junping Du 464ff479ce YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong. 2017-02-13 06:12:54 -08:00
Jason Lowe 69fa81679f YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He 2017-01-23 17:12:51 +00:00
Andrew Wang 5d8b80ea9b Preparing for 3.0.0-alpha3 development 2017-01-19 15:50:07 -08:00
Daniel Templeton 695f110f93 YARN-6071. Fix incompatible API change on AM-RM protocol due to YARN-3866 (trunk only) 
(Contributed by Wangda Tan via Daniel Templeton)
 2017-01-17 12:55:47 -08:00
Varun Saxena db490eccce YARN-6074. FlowRunEntity does not deserialize long values correctly (Rohith Sharma K S via Varun Saxena) 2017-01-09 18:54:22 +05:30
Arun Suresh 85826f6ca5 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 
(cherry picked from commit 4985217de4)
 2017-01-08 09:04:20 -08:00
Daniel Templeton ac16400e1f YARN-6022. Revert changes of AbstractResourceRequest (Contributed by Wangda Tan via Daniel Templeton) 2017-01-07 08:24:30 -08:00
Wangda Tan 0a55bd841e YARN-5959. RM changes to support change of container ExecutionType. (Arun Suresh via wangda) 2017-01-05 10:31:05 -08:00
Daniel Templeton 165d01a73e YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 2016-12-30 08:41:52 -08:00
Wangda Tan 0840b4329b YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda) 2016-12-27 21:18:33 -08:00
Daniel Templeton e216e8e233 HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-27 13:03:16 -08:00
Arun Suresh ac1e5d4f77 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 2016-12-27 12:40:44 -08:00
Arun Suresh 2273a74c1f YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 2016-12-16 08:14:34 -08:00
Jian He a6410a542e YARN-5709. Cleanup leader election configs and pluggability. Contribtued by Karthik Kambatla 2016-12-09 17:00:37 -08:00
Arun Suresh b0aace21b1 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 2016-12-09 16:41:25 -08:00
Sangjin Lee a5a55a54ab YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen. 2016-12-08 12:31:12 -08:00
Sunil ab923a53fc YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S 2016-12-08 00:27:25 +05:30
Junping Du a7288da595 YARN-5184. Fix up incompatible changes introduced on ContainerStatus and NodeReport. Contributed by Sangjin Lee. 2016-12-06 14:36:41 -08:00
Karthik Kambatla c8d0a049b0 HADOOP-13859. TestConfigurationFieldsBase fails for fields that are DEFAULT values of skipped properties. (Haibo Chen via kasha) 2016-12-06 14:10:58 -08:00
Jian He 43ebff2e35 YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan 2016-12-05 11:39:34 -08:00
Daniel Templeton 25f9872be6 YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler 
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
 2016-11-29 09:40:49 -08:00
Akira Ajisaka 209e805430 HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki. 2016-11-28 14:34:57 +09:00
Sunil eb0a483ed0 YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S. 2016-11-24 18:18:42 +05:30
Akira Ajisaka 6f8074298d YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn. 2016-11-22 10:20:31 +09:00
Subru Krishnan 140b9939da YARN-5904. Reduce the number of default server threads for AMRMProxyService. 2016-11-17 18:26:25 -08:00
Arun Suresh 3219b7b4ac YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 2016-11-15 07:56:25 -08:00
Wangda Tan ede1a473f5 YARN-5552. Add Builder methods for common yarn API records. (Tao Jie via wangda) 2016-11-11 13:34:56 -08:00
Eric Payne 93eeb13164 YARN-4218. Metric for resource*time that was preempted. Contributed by Chang Li. 2016-11-10 22:35:12 +00:00
Jian He bcc15c6290 YARN-5611. Provide an API to update lifetime of an application. Contributed by Rohith Sharma K S 2016-11-09 16:08:05 -08:00
Andrew Wang e1c6ef2efa HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey. 2016-11-08 16:33:55 -08:00
Subru Krishnan 29e3b3417c YARN-5833. Add validation to ensure default ports are unique in Configuration. (Konstantinos Karanasos via Subru). 2016-11-08 14:38:18 -08:00
Wangda Tan 3de0da2a76 YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2016-11-06 13:13:31 -08:00
Wangda Tan c85cc3b56e YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2016-11-06 13:13:31 -08:00
Varun Saxena dd4ed6a587 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:22:25 +05:30
Varun Saxena 6cc7c43866 YARN-5710. Fix inconsistent naming in class ResourceRequest (Yufei Gu via Varun Saxena) 2016-10-27 13:55:17 +05:30
Karthik Kambatla de6faae97c YARN-5388. Deprecate and remove DockerContainerExecutor. (Daniel Templeton via kasha) 2016-10-25 13:35:47 -07:00
Xuan b733a6f862 YARN-5718. TimelineClient (and other places in YARN) shouldn't over-write HDFS client retry settings which could cause unexpected behavior. Contributed by Junping Du. 2016-10-18 11:06:47 -07:00
Karthik Kambatla 6378845f9e YARN-4464. Lower the default max applications stored in the RM and store. (Daniel Templeton via kasha) 2016-10-11 21:42:08 -07:00
Junping Du 459a4833a9 YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin. 2016-10-07 07:46:08 -07:00
Naganarasimha 6e130c308c YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 02:02:26 -04:00
Subru Krishnan 3a3697deab YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru). 2016-09-30 19:41:43 -07:00
Jian He 2ae5a3a5bf YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 2016-09-29 22:00:31 +08:00
Varun Saxena 9b0fd01d2e YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-09-28 16:10:10 +05:30
Varun Vasudev bc2656f09f YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 2016-09-28 15:18:18 +05:30
Arun Suresh 4815d024c5 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 08:46:54 -07:00
Arun Suresh 2f163cd5cf Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh" 
This reverts commit fe644bafe7.
 2016-09-26 08:36:59 -07:00
Jian He fe644bafe7 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 22:41:16 +08:00
Naganarasimha d0372dc613 YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-23 06:30:49 +05:30
Ray Chiang 378f624a39 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 11:14:35 -07:00
Arun Suresh d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Wangda Tan 19c743c1bb YARN-4889. Changes in AMRMClient for identifying resource-requests explicitly. (Arun Suresh via wangda) 2016-08-26 17:14:12 -07:00
Subru Krishnan b930dc3ec0 YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 2016-08-26 16:58:47 -07:00
Junping Du 9ef632f3b0 YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 2016-08-26 09:04:44 -07:00
Varun Vasudev bfb4d95059 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:30:18 +05:30
Wangda Tan 2da32a6ef9 YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 2016-08-19 16:31:16 -07:00
Junping Du 0da69c324d YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 
(cherry picked from commit d464483bf7)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
 2016-08-18 07:27:23 -07:00
Varun Saxena ffe1fff526 YARN-5514. Clarify DecommissionType.FORCEFUL comment (Vrushali C via Varun Saxena) 2016-08-16 14:05:41 +05:30
Ray Chiang aea3e65749 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 2016-08-10 19:23:29 -07:00
Arun Suresh 82c9e06101 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 2016-08-09 00:42:29 -07:00
Wangda Tan 3f100d76ff YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda) 2016-08-05 10:43:35 -07:00
Varun Saxena 4a26221021 YARN-5429. Fix return related javadoc warnings in yarn-api (Vrushali C via Varun Saxena) 2016-08-05 16:27:53 +05:30
Arun Suresh e5766b1dbe YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 2016-07-31 11:48:25 -07:00
Arun Suresh 2d8d183b19 YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh) 2016-07-26 19:08:30 -07:00
Andrew Wang da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Varun Vasudev 58e1850801 YARN-5298. Mount usercache and NM filecache directories into Docker container. Contributed by Sidharta Seethana. 2016-07-14 16:11:07 +05:30
Jian He 819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Sangjin Lee 6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Li Lu 0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Sangjin Lee 497d1c7f51 YARN-5111. YARN container system metrics are not aggregated to application (Naganarasimha G R via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee 8c8183e515 YARN-5093. created time shows 0 in most REST output (Varun Saxena via sjlee) 2016-07-10 08:45:59 -07:00
Li Lu c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee 39cce4e629 YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee 84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Sangjin Lee 9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Sangjin Lee 0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee 960af7d471 YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee) 2016-07-10 08:45:53 -07:00
Li Lu 9d40d9d34c YARN-4224. Support fetching entities by UID and change the REST 
interface to conform to current REST APIs' in YARN. (Varun Saxena via
gtcarrera9)
 2016-07-10 08:45:52 -07:00
Naganarasimha 6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Sangjin Lee cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Li Lu b7d484aead YARN-4450. TestTimelineAuthenticationFilter and 
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
 2016-07-10 08:45:49 -07:00
Li Lu 89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Sangjin Lee e3e857866d YARN-4179. [reader implementation] support flow activity queries based on time (Varun Saxena via sjlee) 2016-07-10 08:45:46 -07:00
Vrushali Channapattan d95dc89a02 YARN-4075 [reader REST API] implement support for querying for flows and flow runs (Varun Saxena via vrushali) 2016-07-10 08:45:44 -07:00
Vrushali 10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Sangjin Lee 102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Sangjin Lee 9e5155be36 YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee) 
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
 2016-07-10 08:45:41 -07:00
Zhijie Shen a9fab9b644 YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee. 
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
 2016-07-10 08:45:40 -07:00
Junping Du 8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee. 
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
 2016-07-10 08:45:39 -07:00
Sangjin Lee 57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee) 
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
 2016-07-10 08:45:39 -07:00
Sangjin Lee e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee) 
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
 2016-07-10 08:45:39 -07:00
Zhijie Shen 2d59bc4458 YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena. 
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
 2016-07-10 08:45:38 -07:00
Zhijie Shen f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Zhijie Shen 42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du. 
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
 2016-07-10 08:45:37 -07:00
Sangjin Lee fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale) 
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
 2016-07-10 08:45:35 -07:00
Junping Du 2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen. 
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
 2016-07-10 08:45:34 -07:00
Zhijie Shen 5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du 42e49399ce YARN-3374. Collector's web server should randomly bind an available port. Contributed by Zhijie Shen 
(cherry picked from commit 3aa898e734a1e4368ddf1d0bbd31f9b4de53ceba)
 2016-07-10 08:45:32 -07:00
Junping Du d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du 5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Junping Du 2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen 9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C. 
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
 2016-07-10 08:45:30 -07:00
Zhijie Shen 9d57c9c015 YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu. 
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
 2016-07-10 08:45:29 -07:00
Junping Du 4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen 
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
 2016-07-10 08:45:29 -07:00
Zhijie Shen c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen. 
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
 2016-07-10 08:45:29 -07:00
Jian He 8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Wangda Tan c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan eec835ec17 YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 2016-06-13 21:57:33 -07:00
Wangda Tan 03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Wangda Tan 620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Ming Ma 4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Varun Vasudev 86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev 42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Arun Suresh aa975bc781 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 2016-05-27 14:06:32 -07:00
Carlo Curino 013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Varun Vasudev 0287c49107 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 2016-05-20 12:28:14 +05:30
Junping Du 22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 
(cherry picked from commit 2703ec6871)
 2016-05-19 14:15:21 -07:00
Arun Suresh ccc93e7812 YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 2016-05-16 23:22:01 -07:00
Arun Suresh f0ac18d001 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 2016-05-13 13:38:36 -07:00
Andrew Wang 3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang 6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Sangjin Lee 0bbe01f8d5 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 2016-05-12 10:11:23 -07:00
Rohith Sharma K S b7ac85259c YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-12 15:17:49 +05:30
Jason Lowe d0da13229c YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-05-05 14:07:54 +00:00
Varun Vasudev 0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Jian He 4a8508501b YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 2016-04-28 21:54:11 -07:00
Jian He dd80042c42 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:27:25 -07:00
Arun Suresh 341888a0aa YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 2016-04-26 20:12:12 -07:00
Arun Suresh c282a08f38 YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
 2016-04-24 22:38:33 -07:00
Arun Suresh b2a654c5ee YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4f)
 2016-04-22 23:20:54 -07:00