c7101fe21b
YARN-7879. NM user is unable to access the application filecache due to permissions. Contributed by Jason Lowe.
2018-02-02 16:48:57 -08:00
d481344783
YARN-7822. Constraint satisfaction checker support for composite OR and AND constraints. (Weiwei Yang via asuresh)
2018-01-31 01:51:08 -08:00
8d1e2c6409
Merge branch 'YARN-6592' into trunk
2018-01-31 01:42:42 -08:00
7288b08330
YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi
2018-01-31 09:39:43 -06:00
c23980c4f2
YARN-7795. Fix jenkins issues of YARN-6592 branch. (Sunil G via asuresh)
2018-01-31 01:30:17 -08:00
38af237969
YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh)
2018-01-31 01:30:17 -08:00
8779a35742
YARN-7709. Remove SELF from TargetExpression type. (Konstantinos Karanasos via asuresh)
2018-01-31 01:30:17 -08:00
a5c1fc881e
YARN-7696. Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases. (asuresh)
2018-01-31 01:30:17 -08:00
a52d11fb8c
YARN-7613. Implement Basic algorithm for constraint based placement. (Panagiotis Garefalakis via asuresh)
2018-01-31 01:30:17 -08:00
f9af15d659
YARN-7612. Add Processor Framework for Rich Placement Constraints. (asuresh)
2018-01-31 01:30:17 -08:00
06eb63e64b
YARN-7669. API and interface modifications for placement constraint processor. (asuresh)
2018-01-31 01:30:17 -08:00
69de9a1ba9
YARN-7448. [API] Add SchedulingRequest to the AllocateRequest. (Panagiotis Garefalakis via asuresh)
2018-01-31 01:30:17 -08:00
db928556c8
YARN-6595. [API] Add Placement Constraints at the application level. (Arun Suresh via kkaranasos)
2018-01-31 01:30:17 -08:00
b57e8bc300
YARN-6594. [API] Introduce SchedulingRequest object. (Konstantinos Karanasos via wangda)
2018-01-31 01:30:17 -08:00
33a796d9b7
YARN-6593. [API] Introduce Placement Constraint object. (Konstantinos Karanasos via wangda)
...
Change-Id: Id00edb7185fdf01cce6e40f920cac3585f8cbe9c
2018-01-31 01:30:17 -08:00
901d15a30b
Revert "YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi"
...
This reverts commit 1b0f265db1
2018-01-30 08:34:39 -06:00
649ef7ac33
YARN-7064. Use cgroup to get container resource utilization. (Miklos Szegedi via Haibo Chen)
2018-01-26 16:27:31 -08:00
1b0f265db1
YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi
2018-01-26 13:25:20 -06:00
f3e33aeb80
Revert "YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma"
...
This reverts commit ec8f47e7fa
2018-01-24 15:04:15 +05:30
e307edcb47
YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He
2018-01-23 17:54:39 -08:00
39b999aba2
YARN-7782. Enable user re-mapping for Docker containers in yarn-default.xml. Contributed by Eric Yang
2018-01-23 14:03:53 -08:00
97fe3cc187
YARN-7729. Add support for setting Docker PID namespace mode. (Contributed by Billie Rinaldi)
2018-01-22 16:33:38 -05:00
ec8f47e7fa
YARN-7537 [Atsv2] load hbase configuration from filesystem rather than URL. Contributed by Rohith Sharma
2018-01-19 15:34:40 -08:00
d09058b2fd
YARN-6736. Consider writing to both ats v1 & v2 from RM for smoother upgrades. Contributed by Aaron Gresch.
2018-01-16 07:58:29 +05:30
3d65dbe032
YARN-5366. Improve signal handling and delete delay for Docker on Yarn.
...
(Contributed by Shane Kumpf)
2018-01-12 13:41:27 -05:00
a81144daa0
YARN-7666. Introduce scheduler specific environment variable support in ApplicationSubmissionContext for better scheduling placement configurations. (Sunil G via wangda)
...
Change-Id: I0fd826490f5160d47d42af2a9ac0bd8ec4e959dc
2018-01-05 15:12:04 -08:00
626b5103d4
YARN-7688. Miscellaneous Improvements To ProcfsBasedProcessTree. Contributed by BELUGA BEHR.
2018-01-02 17:39:50 -08:00
33ae2a4ae1
YARN-7687. ContainerLogAppender Improvements. Contributed by BELUGA BEHR.
2018-01-02 16:36:51 -08:00
c0aeb666a4
YARN-7662. [ATSv2] Define new set of configurations for reader and collectors to bind (Rohith Sharma K S via Varun Saxena)
2017-12-19 22:29:24 +05:30
631b5c2db7
YARN-5418. When partial log aggregation is enabled, display the list of aggregated files on the container log page. (Xuan Gong via wangda)
...
Change-Id: I1befb0bbaeb89fb315bafe3e2f3379663f8cf1ec
2017-12-15 15:38:36 -08:00
10fc8d2a7d
YARN-7383. Node resource is not parsed correctly for resource names containing dot. Contributed by Gergely Novák.
2017-12-13 22:00:07 +05:30
daa1cdd062
YARN-7564. Cleanup to fix checkstyle issues of YARN-5881 branch. Contributed by Sunil G.
2017-12-07 18:56:54 -08:00
7462c38277
YARN-7483. CapacityScheduler test cases cleanup post YARN-5881. (Sunil G via wangda)
...
Change-Id: I9741a6baf5cb7352d05636efb6c0b24790e7589a
2017-12-07 18:56:54 -08:00
034b312d9f
YARN-7411. Inter-Queue preemption's computeFixpointAllocation need to handle absolute resources while computing normalizedGuarantee. (Sunil G via wangda)
...
Change-Id: I41b1d7558c20fc4eb2050d40134175a2ef6330cb
2017-12-07 18:56:54 -08:00
d52627a7cb
YARN-7254. UI and metrics changes related to absolute resource configuration. (Sunil G via wangda)
2017-12-07 18:56:54 -08:00
5e798b1a0d
YARN-6471. Support to add min/max resource configuration for a queue. (Sunil G via wangda)
...
Change-Id: I9213f5297a6841fab5c573e85ee4c4e5f4a0b7ff
2017-12-07 18:56:54 -08:00
0311cf0535
YARN-7381. Enable the configuration: yarn.nodemanager.log-container-debug-info.enabled by default in yarn-default.xml. (Xuan Gong via wangda)
...
Change-Id: I1ed58dafad5cc276eea5c0b0813cf04f57d73a87
2017-12-05 13:09:49 -08:00
d8863fc16f
YARN-5594. Handle old RMDelegationToken format when recovering RM (rkanter)
2017-12-04 13:14:55 -08:00
7225ec0ceb
YARN-6507. Add support in NodeManager to isolate FPGA devices with CGroups. (Zhankun Tang via wangda)
...
Change-Id: Ic9afd841805f1035423915a0b0add5f3ba96cf9d
2017-12-01 10:50:49 -08:00
5cfaee2e6d
YARN-7495. Improve robustness of the AggregatedLogDeletionService. Contributed by Jonathan Eagles
2017-11-30 12:39:18 -06:00
d8923cdbf1
YARN-7363. ContainerLocalizer don't have a valid log4j config in case of Linux container executor. (Contributed by Yufei Gu)
2017-11-27 14:31:52 -08:00
fedabcad42
YARN-6168. Restarted RM may not inform AM about all existing containers. Contributed by Chandni Singh
2017-11-27 10:19:58 -08:00
b46ca7e73b
YARN-6483. Add nodes transitioning to DECOMMISSIONING state to the list of updated nodes returned to the AM. (Juan Rodriguez Hortala via asuresh)
2017-11-22 19:18:30 -08:00
67bbbe1c0c
YARN-7531. ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType().
2017-11-21 09:09:16 -08:00
d5f66888b8
YARN-6128. Add support for AMRMProxy HA. (Botong Huang via Subru).
2017-11-17 17:39:06 -08:00
0940e4f692
YARN-7218. Decouple YARN Services REST API namespace from RM. (Contributed by Eric Yang)
2017-11-17 12:28:12 -05:00
2e512f016e
YARN-7442. [YARN-7069] Limit format of resource type name (Contributed by Wangda Tan via Daniel Templeton)
2017-11-13 11:03:30 -08:00
dd07038ffa
YARN-6909. Use LightWeightedResource when number of resource types more than two. (Sunil G via wangda)
...
Change-Id: I90e021c5dea7abd9ec6bd73b2287c8adebe14595
2017-11-09 14:51:15 -08:00
a2c150a736
YARN-7386. Duplicate Strings in various places in Yarn memory (misha@cloudera.com via rkanter)
2017-11-09 12:12:52 -08:00
a55d0738f1
YARN-7371. Added allocateRequestId in NMContainerStatus for recovery. Contributed by Chandni Singh
2017-11-06 13:30:20 -08:00
c723021579
YARN-6626. Embed REST API service into RM. Contributed by Eric Yang
2017-11-06 13:30:17 -08:00
673c0db43c
Revert "YARN-6626. Embed REST API service into RM. Contributed by Eric Yang"
...
This reverts commit 63d1084e9781e0fee876916190b69f6242dd00e4.
2017-11-06 13:30:17 -08:00
9e677fa05c
YARN-6626. Embed REST API service into RM. Contributed by Eric Yang
2017-11-06 13:30:17 -08:00
bd96c4c235
Rebase onto latest trunk. minor conflicts
2017-11-06 13:30:13 -08:00
1888318c89
YARN-6903. Yarn-native-service framework core rewrite. Contributed by Jian He
2017-11-06 13:30:11 -08:00
1700adc6f7
YARN-7410. Cleanup FixedValueResource to avoid dependency to ResourceUtils. Contributed by Wangda Tan.
2017-11-03 07:25:29 +05:30
d00b6f7c1f
YARN-7286. Add support for docker to have no capabilities. Contributed by Eric Badger
2017-11-02 09:37:17 -05:00
d57dba9942
YARN-7400. Incorrect log preview displayed in jobhistory server ui. Contributed by Xuan Gong.
2017-11-01 14:48:16 -07:00
9114d7a5a0
YARN-7224. Support GPU isolation for docker container. Contributed by Wangda Tan.
2017-10-29 11:08:44 +05:30
36e158ae98
YARN-7307. Allow client/AM update supported resource types via YARN APIs. (Sunil G via wangda)
...
Change-Id: I14c5ea7252b7c17e86ab38f692b5f9d43196dbe0
2017-10-26 20:15:19 -07:00
b1de78619f
YARN-7262. Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow (rkanter)
2017-10-26 17:47:32 -07:00
25932da6d1
YARN-5516. Add REST API for supporting recurring reservations. (Sean Po via Subru).
2017-10-26 12:10:14 -07:00
679f99b145
YARN-6984. DominantResourceCalculator.isAnyMajorResourceZero() should test all resources
...
(Contributed by Sunil G via Daniel Templeton)
2017-10-24 10:08:26 -07:00
0799fde35e
YARN-7261. Add debug message for better download latency monitoring. (Yufei Gu)
2017-10-20 10:00:13 -07:00
298b174f66
YARN-7338. Support same origin policy for cross site scripting prevention. (Sunil G via wangda)
2017-10-19 14:44:42 -07:00
1fcbe7cf5f
YARN-7124. LogAggregationTFileController deletes/renames while file is open. Contributed by Jason Lowe.
2017-10-16 13:57:03 -07:00
7a27c2c367
YARN-7270. Fix unsafe casting from long to int for class Resource and
...
its sub-classes. (Yufei)
2017-10-13 12:41:59 -07:00
fa5cfc68f3
YARN-6620. Add support in NodeManager to isolate GPU devices by using CGroups. Contributed by Wangda Tan.
2017-10-11 23:44:33 +05:30
41351b05f3
YARN-7296. convertToProtoFormat(Resource r) is not setting for all resource types. Contributed by lovekesh bansal.
2017-10-10 12:02:18 +05:30
5cfbad68c0
YARN-7237. Cleanup usages of ResourceProfiles. (Wangda Tan)
...
Change-Id: I959c8531bd48231956068cb7d2312f3235549f9c
(cherry picked from commit 8536db5a7549fa44d9f9e444b41bd5978b5c6ae1)
2017-10-09 20:36:57 -07:00
09c5dfe937
YARN-7251. Misc changes to YARN-5734
2017-10-09 11:11:23 -07:00
ff39c0de20
YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda)
...
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
2017-10-09 11:11:23 -07:00
74ba6ffa0b
YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung
2017-10-09 11:11:23 -07:00
04f9e80bb2
YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao
2017-10-09 11:11:23 -07:00
916bdbd6be
YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong)
2017-10-09 11:11:23 -07:00
a4e6253046
YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda)
...
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
2017-10-09 11:11:23 -07:00
ef686635c1
YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store
2017-10-09 11:11:23 -07:00
c071aad5da
YARN-7009. TestNMClient.testNMClientNoCleanupOnStop is flaky by design. (Miklos Szegedi via Haibo Chen)
2017-10-05 15:50:13 -07:00
d0046bb5c6
YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe.
2017-10-06 00:07:53 +05:30
c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari.
2017-10-04 07:01:54 +09:00
280080fad0
YARN-7259. Add size-based rolling policy to LogAggregationIndexedFileController. (xgong via wangda)
...
Change-Id: Ifaf82c0aee6b73b9b6ebf103aa72e131e3942f31
2017-10-02 15:30:22 -07:00
ec2ae3060a
YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda)
...
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
2017-09-29 16:42:56 -07:00
85d81ae58e
YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh
2017-09-28 14:10:15 -05:00
28c4957fcc
YARN-7257. AggregatedLogsBlock reports a bad 'end' value as a bad
...
'start' value. Contributed by Jason Lowe
2017-09-27 15:46:18 -07:00
bfd1a72ba8
YARN-4266. Allow users to enter containers as UID:GID pair instead of by username. Contributed by luhuichun, Zhankun Tang, and Eric Badger.
2017-09-21 17:41:34 -05:00
8b336632ac
YARN-6771. Use classloader inside configuration class to make new
...
classes. Contributed by Jongyoul Lee.
2017-09-20 22:22:43 -07:00
56ef5279c1
YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh)
2017-09-18 15:49:31 -07:00
a4f9c7c924
YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh
2017-09-18 10:16:09 -05:00
e81596d06d
YARN-7172. ResourceCalculator.fitsIn() should not take a cluster resource parameter. (Sen Zhao via wangda)
...
Change-Id: Icc3670c9381ce7591ca69ec12da5aa52d3612d34
2017-09-17 21:20:43 -07:00
4f8194430f
YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh)
2017-09-14 08:51:26 -07:00
c92c1d521e
YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong.
2017-09-13 22:16:06 -07:00
5324388cf2
YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G.
2017-09-13 23:36:47 +05:30
bf2b687412
YARN-7136. Additional Performance Improvement for Resource Profile Feature
...
(Contributed by Wangda Tan via Daniel Templeton)
2017-09-12 09:19:12 -07:00
805095496d
YARN-6933. [YARN-3926] ResourceUtils.DISALLOWED_NAMES check is duplicated. Contributed by Manikandan R.
2017-09-12 09:19:12 -07:00
e490602e9b
YARN-7039. Fix javac and javadoc errors in YARN-3926 branch. (Sunil G via wangda)
...
Change-Id: I442bf6d838b3aba83f1f6779cf9dcf8596a2102d
2017-09-12 09:19:12 -07:00
39240b61a1
YARN-7043. Cleanup ResourceProfileManager. (wangda)
...
Change-Id: I463356f37bf1f6a3f1fc3c594c79916e8c0ab913
2017-09-12 09:19:12 -07:00
758b771943
YARN-6781. [YARN-3926] ResourceUtils#initializeResourcesMap takes an unnecessary Map parameter. Contributed by Yu-Tang Lin.
2017-09-12 09:19:12 -07:00
df3855541a
YARN-6789. Add Client API to get all supported resource types from RM. (Sunil G via wangda)
...
Change-Id: I366d8db6f6700acd087db5acb7a1be7e41b2b68d
2017-09-12 09:19:12 -07:00
b1fe3a222e
YARN-7042. Clean up unit tests after YARN-6610. (Daniel Templeton via wangda)
...
Change-Id: I8e40f704b6fcdd5b14faa9548a27986501044fa1
2017-09-12 09:19:12 -07:00
a6989af1d9
YARN-7030. [YARN-3926] Performance optimizations in Resource and ResourceUtils class. Contributed by Wangda Tan.
2017-09-12 09:19:12 -07:00
a0030c8c6f
YARN-6610. [YARN-3926] DominantResourceCalculator#getResourceAsValue dominant param is updated to handle multiple resources. Contributed by Daniel Templeton.
2017-09-12 09:19:12 -07:00
2b51b262ab
YARN-6892. [YARN-3926] Improve API implementation in Resources and DominantResourceCalculator class. Contributed by Sunil G.
2017-09-12 09:19:12 -07:00
d5e9939ebb
YARN-6994. [YARN-3926] Remove last uses of Long from resource types code. (Daniel Templeton via Yufei Gu)
2017-09-12 09:19:11 -07:00
3aeaafecb8
YARN-6788. [YARN-3926] Improve performance of resource profile branch
...
(Contributed by Sunil Govindan via Daniel Templeton)
2017-09-12 09:19:11 -07:00
4e5632d28e
YARN-6786. [YARN-3926] ResourcePBImpl imports cleanup. Contributed by Yeliang Cang.
2017-09-12 09:19:11 -07:00
ca2808d8bb
YARN-6761. Fix build for YARN-3926 branch. Contributed by Varun Vasudev.
2017-09-12 09:19:11 -07:00
58da54640e
YARN-6445. [YARN-3926] Performance improvements in resource profile branch with respect to SLS. Contributed by Varun Vasudev.
2017-09-12 09:19:11 -07:00
dae65f3bef
YARN-6232. Update resource usage and preempted resource calculations to take into account all resource types. Contributed by Varun Vasudev.
2017-09-12 09:19:11 -07:00
6708ac3301
YARN-5587. Add support for resource profiles. (vvasudev via asuresh)
2017-09-12 09:19:10 -07:00
c2032e251e
YARN-5708. Implement APIs to get resource profiles from the RM. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
a9b4426302
YARN-5707. Add manager class for resource profiles. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
239c1824a0
YARN-5586. Update the Resources class to consider all resource types. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
9e4ba6aff5
YARN-5242. Update DominantResourceCalculator to consider all resource types in calculations. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
759114b006
YARN-4830. Add support for resource types in the nodemanager. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
7ba698997b
YARN-4829. Add support for binary units in Resource class.(vvasudev via asuresh)
2017-09-12 09:19:10 -07:00
ceb12c59a3
YARN-4715. Add support to read resource types from a config file. Contributed by Varun Vasudev.
2017-09-12 09:19:10 -07:00
32c91223f1
YARN-4172. Extend DominantResourceCalculator to account for all resources. (Varun Vasudev via wangda)
2017-09-12 09:19:09 -07:00
1bbab7c157
YARN-4081. Add support for multiple resource types in the Resource class. (Varun Vasudev via wangda)
2017-09-12 09:19:09 -07:00
3fddabc2fe
YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong.
2017-09-08 15:16:19 -07:00
56d93d2e39
YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S.
2017-09-08 07:15:17 +05:30
b0b535d9d5
YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf
2017-09-07 16:17:03 -05:00
22de9449f8
YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong.
2017-09-06 14:53:31 -07:00
63231a2a30
YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong.
2017-09-06 13:19:34 -07:00
0d419c984f
Preparing for 3.1.0 development
2017-09-01 11:53:48 -07:00
91cc070d67
YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong.
2017-08-31 16:41:43 -07:00
cc23514aba
YARN-7115. Move BoundedAppender to org.hadoop.yarn.util pacakge
...
(Contributed by Jian He via Daniel Templeton)
2017-08-30 17:26:13 -07:00
3d00c8f394
Made fixes for whitespace errors and checstyle warnings before merge.
2017-08-30 11:29:54 +05:30
16ba4f544f
MAPREDUCE-6838. Addendum to fix code comment
2017-08-30 11:29:54 +05:30
08f40bcc7f
MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena
2017-08-30 11:29:54 +05:30
e276c75ec1
YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena.
2017-08-30 11:29:54 +05:30
7594d1de7b
YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena.
2017-08-30 11:29:53 +05:30
ac7f52df83
YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena
2017-08-30 11:29:53 +05:30
ff43b8db7a
YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena)
2017-08-30 11:29:52 +05:30
580d884913
YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C.
2017-08-30 11:29:52 +05:30
643a20a358
YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis.
2017-08-30 11:29:51 +05:30
9bb5cc3d49
YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena)
2017-08-30 11:29:50 +05:30
ad45d19998
YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang.
2017-08-29 15:16:23 -07:00
c2cb7ea1ef
YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong.
2017-08-24 13:36:49 -07:00
8410d862d3
YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh)
2017-08-20 07:54:09 -07:00
bbbf0e2a41
HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru).
2017-08-11 13:58:45 -07:00
47b145b9b4
YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-08-08 10:33:26 -07:00
f44b349b81
YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S.
2017-08-04 16:03:56 -07:00
35dc782923
YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh)
2017-08-03 21:15:40 -07:00
c617fe02b3
YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu)
2017-08-03 11:33:37 -07:00
acda6b96a4
YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino)
...
(cherry picked from commit b6240b92ab
2017-08-01 17:28:28 -07:00
43a97174fe
YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 52daa6d971
2017-08-01 17:28:26 -07:00
859aa1f9d6
YARN-5531. UnmanagedAM pool manager for federating application across clusters. (Botong Huang via Subru).
...
(cherry picked from commit 73bb2102ce
2017-08-01 17:28:24 -07:00
7444406d6d
YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 67846a5519
2017-08-01 17:28:24 -07:00
80e1904000
YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 4846069061
2017-08-01 17:28:24 -07:00
86b2bec56e
YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru)
...
(cherry picked from commit 51aeb2ce0c
2017-08-01 17:28:23 -07:00
bdfad4523f
YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan
...
(cherry picked from commit 3307564a5f
2017-08-01 17:28:20 -07:00
5c84382397
YARN-3672. Create Facade for Federation State and Policy Store. Contributed by Subru Krishnan
...
(cherry picked from commit 5e976cd2b9
2017-08-01 17:28:19 -07:00
d19b677301
YARN-5390. Federation Subcluster Resolver. Contributed by Ellen Hui.
...
(cherry picked from commit d3dc461a93
2017-08-01 17:28:18 -07:00
38c6fa5c7a
HADOOP-11875. [JDK9] Adding a second copy of Hamlet without _ as a one-character identifier.
2017-07-28 09:10:52 +09:00
c8df3668ec
YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling
2017-07-20 11:03:04 -05:00
077fcf6a96
YARN-6777. Support for ApplicationMasterService processing chain of interceptors. (asuresh)
2017-07-19 12:26:40 -07:00
9845bea59c
YARN-6689. PlacementRule should be configurable. (Jonathan Hung via xgong)
2017-07-12 15:40:45 -07:00
655110393b
YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe
2017-07-12 15:52:56 -05:00
626d730bfc
YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundatt.
2017-07-09 17:26:09 +05:30
3be2659f83
YARN-6694. Add certain envs to the default yarn.nodemanager.env-whitelist. Contributed by Jian He
2017-06-29 20:10:35 -07:00
af2773f609
Updating version for 3.0.0-beta1 development
2017-06-29 17:57:40 -07:00
25d891a784
YARN-6743. yarn.resourcemanager.zk-max-znode-size.bytes description needs spaces in yarn-default.xml
...
(Contributed by Lori Loberg via Daniel Templeton)
2017-06-28 13:17:58 -07:00
ca13b224b2
YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne.
2017-06-22 23:50:57 -07:00
740204b292
YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt.
2017-06-23 07:52:41 +05:30
ee89ac84e6
YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp
2017-06-19 22:42:45 +05:30
7582dedad1
YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang.
2017-06-18 19:53:42 +05:30
036a24bab3
MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver tools.
...
(Contributed by LiXin Ge via Daniel Templeton)
2017-06-13 13:21:23 -07:00
5578af8603
YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G.
2017-06-09 16:16:34 -05:00
a062374c39
YARN-6679. Reduce Resource instance overhead via non-PBImpl
...
(Contributed by Daryn Sharp via Daniel Templeton)
2017-06-08 10:08:08 -07:00
4b4a6524f2
YARN-6477. Dispatcher no longer needs the raw types suppression. (Maya Wexler via Haibo Chen)
2017-05-30 16:58:15 -07:00
16ad896d5c
Update maven version for 3.0.0-alpha4 development
2017-05-26 14:09:44 -07:00
c583ab02c7
YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Sunil G via wangda)
...
Change-Id: I9b19f69788068be05b3295247cdd7b972f8a573c
2017-05-22 14:26:13 -07:00
9855225a79
YARN-6602. Impersonation does not work if standby RM is contacted first (rkanter)
2017-05-19 15:43:22 -07:00
543aac9f28
HADOOP-14386. Rewind trunk from Guava 21.0 back to Guava 11.0.2.
2017-05-09 09:22:53 -07:00
1769b12a77
YARN-6457. Allow custom SSL configuration to be supplied in WebApps. (Sanjay M Pujare via Haibo Chen)
2017-05-08 15:46:12 -07:00
424887ecb7
YARN-3839. Quit throwing NMNotYetReadyException. Contributed by Manikandan R
2017-05-08 17:14:37 -05:00
4b5bd73ac5
YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common. Contributed by Weiwei Yang
2017-05-01 16:31:16 +05:30
2e52789edf
YARN-2962. ZKRMStateStore: Limit the number of znodes under a znode (Contributed by Varun Sexena via Daniel Templeton)
2017-04-28 13:30:28 -07:00
94607214ce
YARN-6534. ResourceManager failed due to TimelineClient try to init SSLFactory even https is not enabled. Contributed by Rohith Sharma K S.
...
(cherry picked from commit 09e36ff59c05b8bd54d346d1103de0a7300875c6)
2017-04-27 13:11:32 -07:00
4f3ca0396a
YARN-6510. Fix profs stat file warning caused by process names that includes parenthesis. (Wilfred Spiegelenburg via Haibo Chen)
2017-04-26 11:46:55 -07:00
dd43b895c2
YARN-6202. Configuration item Dispatcher.DISPATCHER_EXIT_ON_ERROR_KEY is disregarded
...
(Contributed by Yufei Gu via Daniel Templeton)
2017-04-19 11:44:55 -07:00
15acdbdba2
YARN-3427. Remove deprecated methods from ResourceCalculatorProcessTree
...
(Contributed by Miklos Szegedi via Daniel Templeton)
2017-04-19 10:31:14 -07:00
9ff25408bc
YARN-6164. Expose Queue Configurations per Node Label through YARN client api. Contributed by Benson Qiu.
2017-04-19 20:39:09 +05:30
9d9087a67a
YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen.
...
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
2017-04-12 13:43:18 -07:00
1b081ca27e
YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka
2017-04-06 16:24:36 -05:00
e8071aa249
YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang
2017-04-05 12:30:45 -05:00
7c2bc444b3
YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena)
2017-03-31 02:02:57 +05:30
640ba1d23f
YARN-5685. RM configuration allows all failover methods to disabled when automatic failover is enabled
2017-03-29 12:38:11 -07:00
9bae6720cb
YARN-6050. AMs can't be scheduled on racks or nodes (rkanter)
2017-03-27 17:23:13 -07:00
cd014d57aa
YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda)
2017-03-27 13:29:09 -07:00
84ddedc0b2
HADOOP-10101. Update guava dependency to the latest version. (ozawa)
2017-03-25 10:06:52 +09:00
221783b35b
YARN-6294. ATS client should better handle Socket closed case. Contributed by Li Lu.
2017-03-15 11:48:17 -07:00
871dc420f8
YARN-6313. YARN logs cli should provide logs for a completed container even when application is still running. Contributed by Xuan Gong.
...
(cherry picked from commit b88f5e0f7858d1d89b79dfd325b767c34416052d)
2017-03-14 12:58:12 -07:00
229c7c9f89
YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena)
2017-03-11 04:09:01 +05:30
9649c27864
YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger
2017-03-10 13:06:54 -06:00
deb9f56946
YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen
2017-03-10 11:08:33 -06:00
a96afae125
YARN-6264. AM not launched when a single vcore is available on the cluster. (Yufei Gu via kasha)
2017-03-09 23:11:54 -08:00
279d187f72
YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong
2017-03-03 15:24:04 -08:00
6f6dfe0202
YARN-5280. Allow YARN containers to run with Java Security Manager (gphillips via rkanter)
2017-03-01 14:53:47 -08:00
480b4dd574
YARN-1728. Regression test for guice-undecoded pathInfo in YARN WebApp. (Yuanbo Liu via gera)
2017-02-28 09:27:22 -08:00
53d372a255
YARN-6228: EntityGroupFSTimelineStore should allow configurable cache
...
stores. Contributed by Li Lu
2017-02-24 10:59:35 -08:00
718ad9f6ee
YARN-6210. FairScheduler: Node reservations can interfere with preemption. (kasha)
2017-02-22 15:46:07 -08:00
1a6ca75f38
YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda)
2017-02-22 11:17:09 -08:00
c7a36e6130
YARN-6125. The application attempt's diagnostic message should have a maximum size
...
(Contributed by Andras Piros via Daniel Templeton)
2017-02-17 13:40:58 -08:00
4fa1afdb88
YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R.
2017-02-16 11:41:04 -08:00
6c25dbcdc0
YARN-6163. FS Preemption is a trickle for severely starved applications. (kasha)
2017-02-15 23:16:12 -08:00
0fc6f38379
Addendum patch for YARN-6174 - remove deleted file.
2017-02-15 09:07:49 -08:00
ce2d5bfa5f
YARN-6174. Log files pattern should be same for both running and finished container. Contributed by Xuan Gong.
2017-02-15 09:05:14 -08:00
aaf106fde3
YARN-5966. AMRMClient changes to support ExecutionType update. (asuresh)
2017-02-14 06:09:10 -08:00
464ff479ce
YARN-6113. Re-direct NM Web Service to get container logs for finished applications. Contributed by Xuan Gong.
2017-02-13 06:12:54 -08:00
3a0a0a485a
YARN-6118. Add javadoc for Resources.isNone. Contributed by Andres Perez.
2017-02-10 15:48:46 +05:30
eec52e158b
YARN-6145. Improve log message on fail over. Contributed by Jian He.
2017-02-08 02:11:19 -08:00
9dbfab1284
YARN-6108. Improve AHS webservice to accept NM address as a parameter to get container logs. Contributed by Xuan Gong.
2017-02-07 02:13:14 -08:00
cce35c3815
HADOOP-12097. Allow port range to be specified while starting webapp. Contributed by Varun Saxena.
2017-02-05 19:42:11 -08:00
327c9980aa
YARN-6100. Improve YARN webservice to output aggregated container logs. Contributed by Xuan Gong.
2017-02-02 00:41:18 -08:00
8528d85a68
YARN-6099. Improve webservice to list aggregated log files. Contributed by Xuan Gong.
2017-01-24 15:26:53 -08:00
dc6ec97048
YARN-6117. SharedCacheManager does not start up. Contributed by Chris Trezzo.
2017-01-23 21:07:25 -08:00
69fa81679f
YARN-5910. Support for multi-cluster delegation tokens. Contributed by Jian He
2017-01-23 17:12:51 +00:00
5d8b80ea9b
Preparing for 3.0.0-alpha3 development
2017-01-19 15:50:07 -08:00
a9d946d4df
YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer.
2017-01-17 14:16:00 +05:30
e6f13fe5d1
YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton)
2017-01-11 16:00:01 -08:00
4db119b7b5
YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay.
2017-01-10 18:20:27 -08:00
287d3d6804
YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda)
2017-01-09 08:40:39 -08:00
a59df15757
YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S.
2017-01-07 08:11:58 +05:30
165d01a73e
YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton)
2016-12-30 08:41:52 -08:00
9262797e86
YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter)
2016-12-27 14:14:08 -08:00
e216e8e233
HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton)
2016-12-27 13:03:16 -08:00
ac1e5d4f77
YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh)
2016-12-27 12:40:44 -08:00
2273a74c1f
YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh)
2016-12-16 08:14:34 -08:00
f66f61892a
HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran
2016-12-12 13:58:27 +00:00
4b149a1e77
YARN-4457. Cleanup unchecked types for EventHandler (templedf via rkanter)
2016-12-09 17:28:25 -08:00
b0aace21b1
YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh)
2016-12-09 16:41:25 -08:00
7d8e440eee
Revert "HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran"
...
This reverts commit c265515725
2016-12-09 12:49:27 +09:00
c265515725
HADOOP-13852 hadoop build to allow hadoop version property to be explicitly set. Contriibuted by Steve Loughran
2016-12-08 17:57:15 +00:00
74d0066d33
YARN-5970. Validate application update timeout request parameters. Contributed by Rohith Sharma K S.
2016-12-08 15:53:56 +05:30
72fe546841
YARN-5963. Spelling errors in logging and exceptions for node manager, client, web-proxy, common, and app history code (gsohn via rkanter)
2016-12-07 15:07:25 -08:00
ab923a53fc
YARN-5965. Retrospect ApplicationReport#getApplicationTimeouts. Contributed by Rohith Sharma K S
2016-12-08 00:27:25 +05:30
43ebff2e35
YARN-5559. Analyse 2.8.0/3.0.0 jdiff reports and fix any issues. Contributed by Akira Ajisaka & Wangda Tan
2016-12-05 11:39:34 -08:00
f304ccae3c
YARN-5915. ATS 1.5 FileSystemTimelineWriter causes flush() to be called after every event write. Contributed by Atul Sikaria
2016-12-02 16:54:15 +00:00
c87b3a448a
MAPREDUCE-6787. Allow job_conf.xml to be downloadable on the job overview page in JHS (haibochen via rkanter)
2016-12-01 17:29:38 -08:00
b3befc021b
YARN-4997. Update fair scheduler to use pluggable auth provider (Contributed by Tao Jie via Daniel Templeton)
2016-11-30 09:50:33 -08:00
25f9872be6
YARN-5774. MR Job stuck in ACCEPTED status without any progress in Fair Scheduler
...
if set yarn.scheduler.minimum-allocation-mb to 0. (Contributed by Yufei Gu via Daniel Templeton)
2016-11-29 09:40:49 -08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
eb0a483ed0
YARN-4206. Add Application timeouts in Application report and CLI. Contributed by Rohith Sharma K S.
2016-11-24 18:18:42 +05:30
10468529a9
YARN-4752. Improved preemption in FairScheduler. (kasha)
...
Contains:
YARN-5605. Preempt containers (all on one node) to meet the requirement of starved applications
YARN-5821. Drop left-over preemption-related code and clean up method visibilities in the Schedulable hierarchy
YARN-5783. Verify identification of starved applications.
YARN-5819. Verify fairshare and minshare preemption
YARN-5885. Cleanup YARN-4752 branch for merge
Change-Id: Iee0962377d019dd64dc69a020725d2eaf360858c
2016-11-23 19:48:59 -10:00
3541ed8068
YARN-4330. MiniYARNCluster is showing multiple Failed to instantiate default resource calculator warning messages. Contributed by Varun Saxena
2016-11-23 14:12:23 +05:30
4667564162
YARN-5911. DrainDispatcher does not drain all events on stop even if setDrainEventsOnStop is true. Contributed by Varun Saxena.
2016-11-23 08:49:48 +05:30
6f8074298d
YARN-5713. Update jackson from 1.9.13 to 2.x in hadoop-yarn.
2016-11-22 10:20:31 +09:00
d65603517e
YARN-5375. invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures. Contributed by sandflee.
2016-11-16 15:14:00 +05:30
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
Miklos Szegedi
Arun Suresh
Jason Lowe
Konstantinos Karanasos
Wangda Tan
Haibo Chen
Rohith Sharma K S
Billie Rinaldi
Jian He
Eric Yang
Vrushali C
Varun Saxena
Sunil G
Robert Kanter
Yufei Gu
Subru Krishnan
Daniel Templeton
Junping Du
Subru Krishnan
Jonathan Hung
Xuan
Naganarasimha
Akira Ajisaka
Sangjin Lee
Varun Vasudev
Andrew Wang
Carlo Curino
Jason Lowe
Eric Payne
Haibo Chen
Tsuyoshi Ozawa
Karthik Kambatla
Gera Shegalov
bibinchundatt
Steve Loughran
Mingliang Liu