Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,033 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

4040 Commits

Author SHA1 Message Date
Sunil G 722ee84194 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:28:10 +05:30
Junping Du 3fddabc2fe YARN-7072. Add a new log aggregation file format controller. Contributed by Xuan Gong. 2017-09-08 15:16:19 -07:00
Haibo Chen 8edc60531f HADOOP-14847. Remove Guava Supplier and change to java Supplier in AMRMClient and AMRMClientAysnc. (Bharat Viswanadham via Haibo Chen) 2017-09-08 14:33:11 -07:00
Arun Suresh 1f53ae7972 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 2017-09-08 09:24:05 -07:00
Rohith Sharma K S 4a83170be4 YARN-7140. CollectorInfo should have Public visibility. Contributed by Varun Saxena. 2017-09-08 20:23:16 +05:30
Rohith Sharma K S 0bfb3a2566 YARN-7130. ATSv2 documentation changes post merge. Contributed by Varun Saxena. 2017-09-08 20:21:50 +05:30
Sunil G ab8368d2e0 YARN-7104. Improve Nodes Heatmap in new YARN UI with better color coding. Contributed by Da Ding. 2017-09-08 16:16:01 +05:30
Subru Krishnan fa6137501c YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 2017-09-07 19:07:17 -07:00
Sunil G 56d93d2e39 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 07:15:17 +05:30
Jason Lowe b0b535d9d5 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:17:03 -05:00
Wangda Tan f155ab7cfa YARN-7033. Add support for NM Recovery of assigned resources (e.g. GPU's, NUMA, FPGA's) to container. (Devaraj K and Wangda Tan) 
Change-Id: Iffd18bb95debe1c8cc55e30abc1d8f663e9d0e30
 2017-09-07 14:13:37 -07:00
Arun Suresh c41118a7f8 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh) 2017-09-07 10:23:12 -07:00
Sunil G 6f101e7df1 YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad. 2017-09-07 13:16:20 +05:30
Subru Krishnan b6e7d13696 YARN-5328. Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 2017-09-06 16:46:01 -07:00
Subru Krishnan e3345e985b Revert "Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem." 
This reverts commit 7996eca7dc.
 2017-09-06 16:39:23 -07:00
Junping Du 22de9449f8 YARN-7144. Log Aggregation controller should not swallow the exceptions when it calls closeWriter and closeReader. Contributed by Xuan Gong. 2017-09-06 14:53:31 -07:00
Jason Lowe 704267cb49 YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 2017-09-06 15:23:49 -05:00
Junping Du 63231a2a30 YARN-7148. TestLogsCLI fails in trunk and branch-2 and javadoc error. Contributed by Xuan Gong. 2017-09-06 13:19:34 -07:00
Sunil G ed162b7aa0 YARN-7022. Improve click interaction in queue topology in new YARN UI. Contributed by Abdullah Yousufi. 2017-09-05 08:06:43 +05:30
Varun Saxena ef87d34a0e YARN-7152. [ATSv2] Registering timeline client before AMRMClient service init throw exception (Rohith Sharma K S via Varun Saxena) 2017-09-04 20:11:02 +05:30
Sunil G 275980bb1e YARN-7075. Better styling for donut charts in new YARN UI. Contributed by Da Ding. 2017-09-02 08:08:59 +05:30
Subru Krishnan 7996eca7dc Plan/ResourceAllocation data structure enhancements required to support recurring reservations in ReservationSystem. 2017-09-01 15:16:40 -07:00
Oleg Danilov c5281a85e1
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2017-09-01 21:33:31 +02:00
Andrew Wang 0d419c984f Preparing for 3.1.0 development 2017-09-01 11:53:48 -07:00
Jason Lowe 99a7f5d451 YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 2017-09-01 11:09:14 -05:00
Sunil G a756704f5a YARN-7071. Add vcores and number of containers in new YARN UI node heat map. Contributed by Abdullah Yousufi. 2017-09-01 20:46:39 +05:30
bibinchundatt dcd0bedcc8 YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena 2017-09-01 11:29:16 +05:30
Allen Wittenauer 0adc3a0533
YARN-6721. container-executor should have stack checking 
Signed-off-by: Chris Douglas <cdouglas@apache.org>
 2017-08-31 19:39:31 -07:00
Allen Wittenauer 190410085b
HADOOP-14364. refresh changelog/release notes with newer Apache Yetus build 
Signed-off-by: Andrew Wang <wang@apache.org>
 2017-08-31 19:06:49 -07:00
Junping Du 91cc070d67 YARN-6877. Create an abstract log reader for extendability. Contributed by Xuan Gong. 2017-08-31 16:41:43 -07:00
Subru Krishnan bac4e8cca8 YARN-7095. Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). 2017-08-31 15:05:41 -07:00
Daniel Templeton d4417dae4f YARN-6780. ResourceWeights.toString() cleanup (Contributed by weiyuan via Daniel Templeton) 2017-08-31 09:35:01 -07:00
Sunil G f9e0cc8cdc YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 2017-08-31 18:42:01 +05:30
Allen Wittenauer 71bbb86d69
HADOOP-14670. Increase minimum cmake version for all platforms 
Signed-off-by: Chris Douglas <cdouglas@apache.org>
 2017-08-30 21:47:24 -07:00
Robert Kanter ce79f7be29 YARN-7094. Document the current known issue with server-side NM graceful decom (rkanter) 2017-08-30 18:04:55 -07:00
Daniel Templeton cc23514aba YARN-7115. Move BoundedAppender to org.hadoop.yarn.util pacakge 
(Contributed by Jian He via Daniel Templeton)
 2017-08-30 17:26:13 -07:00
Haibo Chen a20e7105ea YARN-6868. Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml. (Ray Chiang via Haibo Chen) 2017-08-30 09:14:59 -07:00
Varun Saxena 3d00c8f394 Made fixes for whitespace errors and checstyle warnings before merge. 2017-08-30 11:29:54 +05:30
Varun Saxena 512068a01f YARN-6905. Addendum to fix TestTimelineReaderWebServicesHBaseStorage due to missing FastNumberFormat 2017-08-30 11:29:54 +05:30
Varun Saxena 16ba4f544f MAPREDUCE-6838. Addendum to fix code comment 2017-08-30 11:29:54 +05:30
Jian He 08f40bcc7f MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Vrushali C bea3e4df76 YARN-6047 Documentation updates for TimelineService v2 (Contributed by Rohith Sharma) 2017-08-30 11:29:54 +05:30
Haibo Chen 9b08f365d7 YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen) 2017-08-30 11:29:54 +05:30
Varun Saxena b2efebdd07 YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:54 +05:30
Jian He 7fd6ae2479 YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena 2017-08-30 11:29:54 +05:30
Rohith Sharma K S e276c75ec1 YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena. 2017-08-30 11:29:54 +05:30
Rohith Sharma K S 32188d3295 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 2017-08-30 11:29:54 +05:30
Jian He b664569586 YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena 2017-08-30 11:29:53 +05:30
Jason Lowe d5ff965fee YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C 2017-08-30 11:29:53 +05:30
Varun Saxena 60765aff9b YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena) 2017-08-30 11:29:53 +05:30
Rohith Sharma K S 354be99dbf YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Rohith Sharma K S 7594d1de7b YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Varun Saxena 9f6540535d YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:53 +05:30
Rohith Sharma K S 3fb71b1393 YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C. 2017-08-30 11:29:53 +05:30
Rohith Sharma K S a990ff70c2 YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C. 2017-08-30 11:29:53 +05:30
Vrushali C 61136d03f2 YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C) 2017-08-30 11:29:53 +05:30
Rohith Sharma K S 70078e91e3 YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena. 2017-08-30 11:29:53 +05:30
Rohith Sharma K S 660413165a YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C. 2017-08-30 11:29:53 +05:30
Jian He ac7f52df83 YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena 2017-08-30 11:29:53 +05:30
Jian He d3f11e3f13 YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena 2017-08-30 11:29:53 +05:30
Jian He 879de51206 YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena 2017-08-30 11:29:53 +05:30
vrushali 24447b3626 YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C) 2017-08-30 11:29:52 +05:30
Varun Saxena f6a51dce18 YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena) 2017-08-30 11:29:52 +05:30
Haibo Chen a8f082a180 YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 47474fffac)
 2017-08-30 11:29:52 +05:30
Varun Saxena b87b72b40a YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena) 2017-08-30 11:29:52 +05:30
Varun Saxena 44999aabc2 YARN-6318. timeline service schema creator fails if executed from a remote machine (Sangjin Lee via Varun Saxena) 2017-08-30 11:29:52 +05:30
Varun Saxena 18b3a80df7 YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:52 +05:30
Varun Saxena c3bd8d6ad3 YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:52 +05:30
Varun Saxena 8bb2646595 YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:52 +05:30
Sangjin Lee cf30b3b914 YARN-6253. FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp. Contributed by Haibo Chen. 2017-08-30 11:29:52 +05:30
Varun Saxena ff43b8db7a YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:52 +05:30
Sangjin Lee ccb38c1991 YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen. 2017-08-30 11:29:52 +05:30
Sangjin Lee 580d884913 YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 2017-08-30 11:29:52 +05:30
Varun Saxena 5e0acee75e Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's 2017-08-30 11:29:52 +05:30
Varun Saxena 6f65cf27bb YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:51 +05:30
Varun Saxena 71847ed44d YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena) 2017-08-30 11:29:51 +05:30
Sangjin Lee 7f54ac48c6 YARN-5980. Update documentation for single node hbase deploy. Contributed by Vrushali C. 2017-08-30 11:29:51 +05:30
Varun Saxena 02a9710a09 YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena) 2017-08-30 11:29:51 +05:30
Sangjin Lee 2556c012e3 YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen. 2017-08-30 11:29:51 +05:30
Sangjin Lee ecbc8d4816 YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen. 
(cherry picked from commit a5a55a54ab)
 2017-08-30 11:29:51 +05:30
Sangjin Lee 4481561e4a YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu. 2017-08-30 11:29:51 +05:30
Sangjin Lee 092fead5d9 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 2017-08-30 11:29:51 +05:30
Varun Saxena c92a7ab31c YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena) 2017-08-30 11:29:51 +05:30
Varun Saxena 91d3443806 YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena) 2017-08-30 11:29:51 +05:30
Sangjin Lee 643a20a358 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis. 2017-08-30 11:29:51 +05:30
Sangjin Lee 05ff04439e YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S. 2017-08-30 11:29:51 +05:30
Varun Saxena 9bb5cc3d49 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-08-30 11:29:50 +05:30
Sangjin Lee 78b7e070d8 YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu. 2017-08-30 11:29:50 +05:30
Li Lu e72e6260da YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9) 2017-08-30 11:29:50 +05:30
Sangjin Lee 278bd6fcff YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee) 2017-08-30 11:29:50 +05:30
Rohith Sharma K S 4cae120c61 YARN-6982. Potential issue on setting AMContainerSpec#tokenConf to null before app is completed. Contributed by Manikandan R. 2017-08-30 10:45:11 +05:30
Jian He 8201ed8009 YARN-6756. ContainerRequest#executionTypeRequest causes NPE. Contributed by Jian He 2017-08-29 20:35:53 -07:00
Wangda Tan f59332b97b YARN-5219. When an export var command fails in launch_container.sh, the full container launch should fail. (Sunil G via wangda) 
Change-Id: Iaa6b978bb89482e9d1d77ba57f4adfdc48e39a3c
 2017-08-29 15:29:22 -07:00
Junping Du ad45d19998 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:16:23 -07:00
Carlo Curino cc8893edc0 YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino) 2017-08-29 14:53:09 -07:00
Jason Lowe 1f6dc4ee9a YARN-5816. TestDelegationTokenRenewer#testCancelWithMultipleAppSubmissions is still flakey. Contributed by Robert Kanter 2017-08-29 09:52:24 -05:00
Sunil G 7ddc7aa26e YARN-6386. Show decommissioning nodes in new YARN UI. Contributed by Elek Marton. 2017-08-29 16:51:14 +05:30
Junping Du 312b1fd9da YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 2017-08-28 18:26:51 -07:00
Wangda Tan 51881a863c YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I432f4523921666ac218fcf9936f826a0d8d635e0
 2017-08-28 11:37:35 -07:00
Yufei Gu d5bc958279 YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu) 2017-08-28 10:09:46 -07:00
Sunil G 02599bda04 YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 2017-08-28 12:22:56 +05:30
Li Lu 3bee7046c5 YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9) 2017-08-25 22:43:25 -07:00
Eric Payne e864f81471 YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 2017-08-25 15:37:54 -05:00
Jason Lowe 39a9dc8e4a YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 2017-08-25 15:31:29 -05:00
Arpit Agarwal 8b7cbe3840 HADOOP-14729. Upgrade JUnit 3 test cases to JUnit 4. Contributed by Ajay Kumar. 2017-08-25 09:55:46 -07:00
Jason Lowe 3a4e861169 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 2017-08-25 09:16:17 -05:00
Yufei Gu 9e2699ac2c YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu) 2017-08-24 17:52:41 -07:00
Junping Du c2cb7ea1ef YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 2017-08-24 13:36:49 -07:00
bibinchundatt de0cba700b YARN-7074. Fix NM state store update comment. Contributed by Botong Huang 2017-08-24 16:34:38 +05:30
Junping Du 652dd434d9 YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan. 2017-08-23 18:06:29 -07:00
Wangda Tan f49843a988 YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda) 
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
 2017-08-23 09:56:20 -07:00
Subru Krishnan 4249172e14 YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru). 2017-08-22 19:20:57 -07:00
Jason Lowe 657dd59cc8 YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 2017-08-22 13:16:24 -05:00
Jason Lowe 3efcd51c3b YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 2017-08-22 12:56:09 -05:00
Akira Ajisaka d5ff57a08f
YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang. 2017-08-22 17:14:12 +09:00
Arun Suresh ae8fb13b31 YARN-6923. Metrics for Federation Router. (Giovanni Matteo Fumarola via asuresh) 2017-08-21 22:50:24 -07:00
Arun Suresh 75abc9a8e2 YARN-5603. Metrics for Federation StateStore. (Ellen Hui via asuresh) 2017-08-21 22:43:08 -07:00
Arun Suresh 7a82d7bcea YARN-6979. [Addendum patch] Fixed classname and added javadocs. (Kartheek Muthyala via asuresh) 2017-08-20 10:24:05 -07:00
Arun Suresh 8410d862d3 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 2017-08-20 07:54:09 -07:00
Wangda Tan 436c2638f9 YARN-6852. Native code changes to support isolate GPU devices by using CGroups. (wangda) 
Change-Id: I4869cc4d8ad539539ccba4bea5a178cacdb741ab
 2017-08-18 18:26:36 -07:00
Yufei Gu 8991f0baec YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu) 2017-08-18 14:38:44 -07:00
bibinchundatt e05fa3451d YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su. 2017-08-18 20:28:50 +05:30
Jason Lowe ab1a8ae85f YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-17 15:50:14 -05:00
Sunil G f9a0e23381 YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad. 2017-08-17 15:07:15 +05:30
Subru Krishnan de462da04e YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru). 2017-08-16 11:43:24 -07:00
Jason Lowe 14553061be YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 2017-08-16 13:04:36 -05:00
Akira Ajisaka 588c190afd
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 2017-08-16 14:06:22 +09:00
Subru Krishnan 75dd866bfb HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 2017-08-15 16:53:59 -07:00
Nathan Roberts d265459024 YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts) 2017-08-15 15:52:48 -05:00
Sunil G dadb0c2225 YARN-5146. Support for Fair Scheduler in new YARN UI. Contributed by Abdullah Yousufi. 2017-08-15 21:58:44 +05:30
Arun Suresh 4d7be1d857 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh) 2017-08-14 19:46:17 -07:00
Jason Lowe 5558792894 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 2017-08-14 15:31:34 -05:00
Jason Lowe 3325ef653d YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 2017-08-14 14:40:08 -05:00
Daniel Templeton 6b09c32705 YARN-6881. LOG is unused in AllocationConfiguration (Contributed by weiyuan via Daniel Templeton) 2017-08-14 11:55:33 -07:00
Vrushali C 608a06cca5 YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen) 2017-08-14 11:41:11 -07:00
Subru Krishnan 18f3603bce YARN-6996. Change javax.cache library implementation from JSR107 to Apache Geronimo. (Ray Chiang via Subru). 2017-08-14 11:10:00 -07:00
Jian He e2f6299f6f YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-14 10:51:30 -07:00
Lei Xu ce797a1706 HDFS-12221. Replace xcerces in XmlEditsVisitor. (Ajay Kumar via lei) 2017-08-14 10:27:47 -07:00
bibinchundatt d8f74c3964 YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R. 2017-08-14 09:39:00 +05:30
Carlo Curino 28d97b79b6 YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino) 2017-08-11 16:58:04 -07:00
Carlo Curino cc59b5fb26 YARN-6896. Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution). (Contributed by Giovanni Matteo Fumarola via curino) 2017-08-11 15:58:01 -07:00
Daniel Templeton 0996acde6c YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator 
(Contributed by Larry Lo via Daniel Templeton)
 2017-08-11 14:50:46 -07:00
Daniel Templeton 65364defb4 YARN-6967. Limit application attempt's diagnostic message size thoroughly 
(Contributed by Chengbing Liu via Daniel Templeton)
 2017-08-11 14:28:55 -07:00
Daniel Templeton c7680d4cc4 YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces 
(Contributed by weiyuan via Daniel Templeton)
 2017-08-11 14:22:02 -07:00
Daniel Templeton 218588be77 YARN-6952. Enable scheduling monitor in FS (Contributed by Yufei Gu via Daniel Templeton) 2017-08-11 14:04:19 -07:00
Subru Krishnan bbbf0e2a41 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-11 13:58:45 -07:00
Karthik Kambatla 8c4b6d16a5 YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha) 2017-08-11 12:15:43 -07:00
Sunil G 8d953c2359 YARN-6631. Refactor loader.js in new Yarn UI. Contributed by Akhil P B. 2017-08-10 11:53:26 +05:30
Wangda Tan ec694145cf YARN-6033. Add support for sections in container-executor configuration file. (Varun Vasudev via wandga) 
Change-Id: Ibc6d2a959debe5d8ff2b51504149742449d1f1da
 2017-08-09 10:51:29 -07:00
Akira Ajisaka 63cfcb90ac
YARN-6958. Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice. Contributed by Yeliang Cang. 2017-08-09 23:58:22 +09:00
Akira Ajisaka 1a18d5e514
YARN-6515. Fix warnings from Spotbugs in hadoop-yarn-server-nodemanager. Contributed by Naganarasimha G R. 2017-08-09 21:56:43 +09:00
Subru Krishnan ad2a350662 YARN-6970. Add PoolInitializationException as retriable exception in FederationFacade. (Giovanni Matteo Fumarola via Subru). 2017-08-08 16:48:29 -07:00
Daniel Templeton e0c24145d2 YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code 
(Contributed by Angela Wang via Daniel Templeton)
 2017-08-08 13:35:22 -07:00
Wangda Tan 1794de3ea4 YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) 
Change-Id: If1b1827345f98f0a49cc7e39d1ba41fbeed5e911
 2017-08-08 12:56:29 -07:00
Jian He acf9bd8b1d YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:09:38 -07:00
Daniel Templeton 47b145b9b4 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path 
(Contributed by Miklos Szegedi via Daniel Templeton)
 2017-08-08 10:33:26 -07:00
Akira Ajisaka 98912950b6
YARN-6961. Remove commons-logging dependency from hadoop-yarn-server-applicationhistoryservice module. Contributed by Yeliang Cang. 2017-08-08 19:38:58 +09:00
Arun Suresh 8d3fd81980 YARN-6920. Fix resource leak that happens during container re-initialization. (asuresh) 2017-08-07 18:59:25 -07:00
Subru Krishnan c61f2c4198 YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru). 2017-08-07 16:58:29 -07:00
Wangda Tan adb84f34db YARN-4161. Capacity Scheduler : Assign single or multiple containers per heart beat driven by configuration. (Wei Yan via wangda) 
Change-Id: Ic441ae4e0bf72e7232411eb54243ec143d5fd0d3
 2017-08-07 11:32:21 -07:00
Akira Ajisaka b0fbf17965
YARN-6957. Moving logging APIs over to slf4j in hadoop-yarn-server-sharedcachemanager. Contributed by Yeliang Cang. 2017-08-07 19:25:40 +09:00
Akira Ajisaka 839e077faf
YARN-6873. Moving logging APIs over to slf4j in hadoop-yarn-server-applicationhistoryservice. Contributed by Yeliang Cang. 2017-08-07 18:56:00 +09:00
Sunil G 46b7054fa7 YARN-6951. Fix debug log when Resource Handler chain is enabled. Contributed by Yang Wang. 2017-08-07 13:15:46 +05:30
Junping Du f44b349b81 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-04 16:03:56 -07:00
Arun Suresh 35dc782923 YARN-5977. ContainerManagementProtocol changes to support change of container ExecutionType. (Kartheek Muthyala via asuresh) 2017-08-03 21:15:40 -07:00
Subru Krishnan 0542e6f86e YARN-6932. Fix TestFederationRMFailoverProxyProvider test case failure. 2017-08-03 14:18:03 -07:00
Yufei Gu c617fe02b3 YARN-6832. Tests use assertTrue(....equals(...)) instead of assertEquals()). (Daniel Templeton via Yufei Gu) 2017-08-03 11:33:37 -07:00
Haibo Chen 293c74a81b YARN-6673 Add cpu cgroup configurations for opportunistic containers. (Miklos Szegedi via Haibo Chen) 2017-08-03 09:56:51 -07:00
Haibo Chen c5d256c760 YARN-6674 Add memory cgroup settings for opportunistic containers. (Miklos Szegedi via Haibo Chen) 2017-08-03 09:52:35 -07:00
Sunil G f64cfeaf61 YARN-6678. Handle IllegalStateException in Async Scheduling mode of CapacityScheduler. Contributed by Tao Yang. 2017-08-03 19:27:10 +05:30
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Eric Payne 8ce8672b6b YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 2017-08-02 12:53:22 -05:00
Yufei Gu 45535f8afa YARN-6895. [FairScheduler] Preemption reservation may cause regular reservation leaks. (Miklos Szegedi via Yufei Gu) 2017-08-02 09:25:19 -07:00
Eric Payne 48899134d2 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 2017-08-02 10:59:33 -05:00
Jian He f9139ac8f6 YARN-6872. [Addendum patch] Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-02 00:03:25 -07:00
Carlo Curino 9625a030de YARN-6853. Add MySql Scripts for FederationStateStore. (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit 874ddbf0b5)
 2017-08-01 17:28:28 -07:00
Carlo Curino acda6b96a4 YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino) 
(cherry picked from commit b6240b92ab)
 2017-08-01 17:28:28 -07:00
Subru Krishnan 86f05c3312 YARN-6866. Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk. (Botong Huang via Subru). 
(cherry picked from commit 40453879ec)
 2017-08-01 17:28:28 -07:00
Subru Krishnan 8737c2a8e9 Bumping up hadoop-yarn-server-router module to 3.0.0-beta1 post rebase. 
(cherry picked from commit 69e3ed2680)
 2017-08-01 17:28:27 -07:00
Subru Krishnan 712e97d4cf Bumping up yarn-server-router (new) module pom to beta1 and fixing imports post rebase. 
(cherry picked from commit f1eff212c5)
 2017-08-01 17:28:27 -07:00
Subru Krishnan 3e1dc7ece0 YARN-6821. Move FederationStateStore SQL DDL files from test resource to sbin. 
(cherry picked from commit cd9db822f0)
 2017-08-01 17:28:27 -07:00
Subru Krishnan 825108d08a YARN-6815. FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not. 
(cherry picked from commit 8820693cd7)
 2017-08-01 17:28:27 -07:00
Subru Krishnan f427e4201f YARN-6807. Adding required missing configs to Federation configuration guide based on e2e testing. (Tanuj Nayak via Subru). 
(cherry picked from commit b4ac9d1b63)
 2017-08-01 17:28:26 -07:00
Subru Krishnan 43a97174fe YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 52daa6d971)
 2017-08-01 17:28:26 -07:00
Subru Krishnan 4cfec943b1 YARN-6724. Add ability to blacklist sub-clusters when invoking Routing policies. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit f8e5de5969)
 2017-08-01 17:28:26 -07:00
Arun Suresh 433ee44b58 YARN-6691. Update YARN daemon startup/shutdown scripts to include Router service. (Giovanni Matteo Fumarola via asuresh). 
(cherry picked from commit 790cba7ab9)
 2017-08-01 17:28:26 -07:00
Subru Krishnan 70b1a757f1 YARN-6511. Federation: transparently spanning application across multiple sub-clusters. (Botong Huang via Subru). 
(cherry picked from commit 8c988d235e)
 2017-08-01 17:28:26 -07:00
Subru Krishnan bed1832c93 YARN-3666. Federation Intercepting and propagating AM- home RM communications. (Botong Huang via Subru). 
(cherry picked from commit 2399eb8200)
 2017-08-01 17:28:25 -07:00
Subru Krishnan d6c29b2f5b YARN-6484. Documenting the YARN Federation feature. (Carlo Curino via Subru). 
(cherry picked from commit 749ca890e6)
 2017-08-01 17:28:25 -07:00
Subru Krishnan 42949a6e01 YARN-6666. Fix unit test failure in TestRouterClientRMService. (Botong Huang via Subru). 
(cherry picked from commit e750907d0a)
 2017-08-01 17:28:25 -07:00
Subru Krishnan f317e0cb48 Bumping up hadoop-yarn-server-router module to 3.0.0-alpha4 post rebase. 
(cherry picked from commit bd9c7b1bec)
 2017-08-01 17:28:25 -07:00
Subru Krishnan 859aa1f9d6 YARN-5531. UnmanagedAM pool manager for federating application across clusters. (Botong Huang via Subru). 
(cherry picked from commit 73bb2102ce)
 2017-08-01 17:28:24 -07:00
Subru Krishnan 7444406d6d YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 67846a5519)
 2017-08-01 17:28:24 -07:00
Subru Krishnan 80e1904000 YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 4846069061)
 2017-08-01 17:28:24 -07:00
Carlo Curino 5272af8c7e YARN-3663. Federation State and Policy Store (DBMS implementation). (Giovanni Matteo Fumarola via curino). 
(cherry picked from commit be99c1fe2e)
 2017-08-01 17:28:24 -07:00
Carlo Curino c4bbdfeb66 YARN-6203: Occasional test failure in TestWeightedRandomRouterPolicy (curino) 
(cherry picked from commit 121d55bd29)
 2017-08-01 17:28:23 -07:00
Subru Krishnan e1da8f0667 YARN-5602. Utils for Federation State and Policy Store. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit 326a2e6bde)
 2017-08-01 17:28:23 -07:00
Carlo Curino 1c64e1709b YARN-6370. Properly handle rack requests for non-active subclusters in LocalityMulticastAMRMProxyPolicy. (Contributed by Botong Huang via curino). 
(cherry picked from commit ce419881c3)
 2017-08-01 17:28:23 -07:00
Subru Krishnan 86b2bec56e YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru) 
(cherry picked from commit 51aeb2ce0c)
 2017-08-01 17:28:23 -07:00
Carlo Curino 8623644f45 YARN-6190. Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy. (Botong Huang via curino) 
(cherry picked from commit 5c486961cd)
 2017-08-01 17:28:22 -07:00