Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,579 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

364 Commits

Author SHA1 Message Date
Arun Suresh 2f163cd5cf Revert "YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh" 
This reverts commit fe644bafe7.
 2016-09-26 08:36:59 -07:00
Jian He fe644bafe7 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 22:41:16 +08:00
Arun Suresh d6d9cff21b YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 2016-08-30 15:52:29 -07:00
Junping Du 9ef632f3b0 YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 2016-08-26 09:04:44 -07:00
Varun Vasudev bfb4d95059 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:30:18 +05:30
Arun Suresh 82c9e06101 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 2016-08-09 00:42:29 -07:00
Wangda Tan 3f100d76ff YARN-4888. Changes in scheduler to identify resource-requests explicitly by allocation-id. (Subru Krishnan via wangda) 2016-08-05 10:43:35 -07:00
Arun Suresh e5766b1dbe YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 2016-07-31 11:48:25 -07:00
Subru Krishnan d2cbfd7de3 YARN-5441. Fixing minor Scheduler test case failures 2016-07-27 18:21:58 -07:00
Andrew Wang da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Jian He 819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Sangjin Lee 6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Varun Saxena 1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Sangjin Lee 84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Li Lu 89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Zhijie Shen f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Junping Du 47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen 
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
 2016-07-10 08:45:33 -07:00
Junping Du d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du 2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen 9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Arun Suresh 99e5dd68d0 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 2016-06-22 19:04:54 -07:00
Wangda Tan c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan 03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Varun Vasudev 42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Arun Suresh aa975bc781 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 2016-05-27 14:06:32 -07:00
Arun Suresh 5b41b288d0 YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh) 2016-05-26 14:56:37 -07:00
Arun Suresh 1597630681 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 2016-05-18 18:46:00 -07:00
Jian He 0c6726e20d Addendum patch for YARN-4832. Contributed by Junping Du 2016-05-17 14:50:30 -07:00
Jian He fa3bc3405d YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 2016-05-17 12:52:19 -07:00
Arun Suresh ccc93e7812 YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 2016-05-16 23:22:01 -07:00
Arun Suresh f0ac18d001 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 2016-05-13 13:38:36 -07:00
Andrew Wang 3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT." 
This reverts commit 6b53802cba.
 2016-05-12 15:32:45 -07:00
Andrew Wang 6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Junping Du 39f2bac38b YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 2016-05-11 09:28:35 -07:00
Junping Du e61d431275 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 2016-05-04 10:35:49 -07:00
Arun Suresh 341888a0aa YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 2016-04-26 20:12:12 -07:00
Arun Suresh c282a08f38 YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
 2016-04-24 22:38:33 -07:00
Karthik Kambatla c8172f5f14 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 2016-04-20 09:55:50 -07:00
Jian He 93bacda08b YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev 2016-04-06 16:13:47 -07:00
naganarasimha 5092c94195 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 2016-04-04 16:25:03 +05:30
Rohith Sharma K S 1e6f92977d YARN-4607. Pagination support for AppAttempt page TotalOutstandingResource Requests table. Contributed by Bibin A Chundatt 2016-04-04 08:09:29 +05:30
Karthik Kambatla c9a09d6926 YARN-4647. Make RegisterNodeManagerRequestPBImpl thread-safe. (kasha) 2016-01-29 08:12:54 -08:00
Wangda Tan 79c41b1d83 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 2015-12-16 13:18:19 -08:00
Xuan d8a45425eb YARN-4439. Clarify NMContainerStatus#toString method. Contributed by 
Jian He
 2015-12-14 15:41:28 -08:00
Jian He 9f77ccad73 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2015-12-03 12:48:50 -08:00
Jason Lowe 4ac6799d4a YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 2015-11-24 22:35:37 +00:00
Karthik Kambatla 52948bb20b YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 2015-11-24 13:47:17 +05:30
Jian He 8676a118a1 YARN-4349. Support CallerContext in YARN. Contributed by Wangda Tan 2015-11-23 17:19:48 -08:00
Jian He fcd7888029 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit 8fbea531d7.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:18:44 -08:00
Jian He 8fbea531d7 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 2015-11-09 10:43:45 -08:00
Wangda Tan 3cc73773eb YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 2015-10-26 13:07:39 -07:00
Jonathan Eagles f8adeb712d YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 2015-10-23 10:34:08 -05:00
Wangda Tan def374e666 YARN-4140. RM container allocation delayed incase of app submitted to Nodelabel partition. (Bibin A Chundatt via wangda) 2015-10-09 16:38:59 -07:00
Xuan 8f08532bde YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 2015-10-02 18:50:47 -07:00
Rohith Sharma K S a9aafad12b YARN-4044. Running applications information changes such as movequeue is not published to TimeLine server. (Sunil G via rohithsharmaks) 2015-09-24 12:13:22 +05:30
Jian He 89cab1ba5f YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 2015-09-23 13:29:38 -07:00
Jian He c3dc1af072 YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Jian He 5f5a968d65 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 2015-09-23 13:29:37 -07:00
Wangda Tan ae5308fe1d YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 2015-09-15 11:40:50 -07:00
Wangda Tan e2a0270217 YARN-4151. Fix findbugs errors in hadoop-yarn-server-common module. (Meng Ding via wangda) 2015-09-14 16:00:43 -07:00
Steve Loughran 7269906254 HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:26 +01:00
Jian He 6f72f1e600 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 2015-09-08 09:35:46 +08:00
Xuan 22dc5fc209 YARN-4028. AppBlock page key update and diagnostics value null on 
recovery. Contributed by Bibin A Chundatt
 2015-08-18 22:53:03 -07:00
Karthik Kambatla 13604bd5f1 YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 2015-08-16 15:08:53 -07:00
rohithsharmaks 1c12adb71f YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 2015-08-12 14:45:41 +05:30
Rohith Sharma K S b6265d39c5 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks) 2015-08-07 10:43:41 +05:30
Xuan 0306d902f5 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S
 2015-08-03 15:46:00 -07:00
Jonathan Eagles 3cd02b9522 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 2015-08-03 10:38:05 -05:00
Jonathan Eagles 7faae0e6fe YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 2015-07-17 11:02:11 -05:00
Karthik Kambatla 527c40e4d6 YARN-1012. Report NM aggregated container resource utilization in heartbeat. (Inigo Goiri via kasha) 2015-07-09 09:35:14 -07:00
Devaraj K 57f1a01eda YARN-3826. Race condition in ResourceTrackerService leads to wrong 
diagnostics messages. Contributed by Chengbing Liu.
 2015-06-25 16:13:59 +05:30
Zhijie Shen 8d0ef31632 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong. 2015-06-08 23:14:58 -07:00
Xuan 3e000a919f YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 2015-06-05 12:48:52 -07:00
Zhijie Shen 18dd01d6bf YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 2015-06-04 14:47:08 -07:00
Junping Du d7e7f6aa03 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 2015-06-04 04:59:27 -07:00
Zhijie Shen bc85959edd Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4a9ec1a824.
 2015-06-03 14:15:56 -07:00
Zhijie Shen dbc4f64937 YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G. 2015-06-03 11:51:41 -07:00
Xuan 9686261ecb YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
 2015-05-31 15:37:05 -07:00
Xuan da05361c3b Revert "YARN-2900. Application (Attempt and Container) Not Found in AHS results" 
This reverts commit 06f8e9caba.
 2015-05-31 00:26:04 -07:00
Xuan 06f8e9caba YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
 2015-05-30 21:56:19 -07:00
Zhijie Shen 4a9ec1a824 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 21:05:36 -07:00
Karthik Kambatla a8acdd65b3 YARN-3467. Expose allocatedMB, allocatedVCores, and runningContainers metrics on running Applications in RM Web UI. (Anubhav Dhoot via kasha) 2015-05-30 13:36:24 -07:00
Zhijie Shen 54504133f4 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 2015-05-27 16:52:22 -07:00
Vinayakumar B 55ed6558ba HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 2015-05-22 16:45:19 +05:30
Wangda Tan b37da52a1c YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 2015-05-19 16:34:17 -07:00
Junping Du 15ccd967ee YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. 2015-05-14 10:58:12 -07:00
Zhijie Shen fcd0702c10 YARN-3539. Updated timeline server documentation and marked REST APIs evolving. Contributed by Steve Loughran. 2015-05-12 21:13:22 -07:00
Devaraj K 241a72af0d YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:23:53 +05:30
Akira Ajisaka 918af8efff YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 2015-05-07 13:33:03 +09:00
Jian He 0d3188fd25 YARN-3301. Fixed the format issue of the new RM attempt web page. Contributed by Xuan Gong 2015-05-06 13:45:00 -07:00
Xuan b7dd3a4f04 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA
 2015-05-05 10:40:16 -07:00
Tsuyoshi Ozawa 4a3dabd94f YARN-3406. Display count of running containers in the RM's Web UI. Contributed by Ryu Kobayashi. 2015-04-25 07:17:11 +09:00
Xuan eee9facbba YARN-3511. Add errors and warnings page to ATS. Contributed by Varun Vasudev 2015-04-24 09:41:59 -07:00
Jian He 6779467ab6 YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 2015-04-17 15:55:34 -07:00
Junping Du 1db355a875 YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 2015-04-17 13:18:59 -07:00
Akira Ajisaka 1fa8075794 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 2015-04-16 10:04:09 -05:00
Jian He 1b89a3e173 YARN-3354. Add node label expression in ContainerTokenIdentifier to support RM recovery. Contributed by Wangda Tan 2015-04-15 13:57:06 -07:00
Jian He b2e6cf607f YARN-3404. Display queue name on application page. Contributed by Ryu Kobayashi 2015-04-15 13:52:50 -07:00
Jian He 0fefda645b YARN-3361. CapacityScheduler side changes to support non-exclusive node labels. Contributed by Wangda Tan 2015-04-14 11:45:58 -07:00
Junping Du 92431c9617 YARN-1376. NM need to notify the log aggregation status to RM through Node heartbeat. Contributed by Xuan Gong. 2015-04-10 08:56:18 -07:00
Xuan 19a4feaf6f YARN-3110. Few issues in ApplicationHistory web ui. Contributed by Naganarasimha G R 2015-04-07 08:22:39 -07:00
Wangda Tan bad070fe15 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 2015-04-02 17:23:20 -07:00
Xuan 4728bdfa15 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev
 2015-04-01 04:19:18 -07:00
Wangda Tan 2a945d24f7 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 2015-03-30 12:05:21 -07:00
Haohui Mai e4b8d9e72d HADOOP-10670. Allow AuthenticationFilters to load secret from signature secret files. Contributed by Kai Zheng. 2015-03-25 11:12:42 -07:00
Jian He 4e886eb9cb YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 2015-03-19 22:27:21 -07:00
Jian He 658097d6da YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 2015-03-17 21:30:23 -07:00
Tsuyoshi Ozawa 3da9a97cfb YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 2015-03-16 23:19:05 +09:00
Xuan 3ff1ba2a7b YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R
 2015-03-15 20:26:10 -07:00
Zhijie Shen 85f6d67fa7 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 2015-03-11 19:35:19 -07:00
Jian He c3003eba6f YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 2015-03-09 20:46:48 -07:00
Jian He 95bfd087dc YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 2015-03-05 21:20:09 -08:00
Tsuyoshi Ozawa d1c6accb6f HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-03-03 14:17:52 +09:00
Tsuyoshi Ozawa 9cedad11d8 Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)" 
This reverts commit 946456c6d8.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaByStorageTypeEntry.java
 2015-02-25 00:32:04 +09:00
Xuan e3d290244c YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY. 
Contributed by zhihai xu
 2015-02-21 16:18:40 -08:00
Tsuyoshi Ozawa 946456c6d8 HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-02-19 13:06:53 +09:00
Zhijie Shen e5b56e2af6 YARN-2543. Made resource usage be published to the timeline server too. Contributed by Naganarasimha G R. 2015-02-03 17:34:22 -08:00
Tsuyoshi Ozawa 3aab354e66 YARN-3082. Non thread safe access to systemCredentials in NodeHeartbeatResponse processing. Contributed by Anubhav Dhoot. 2015-01-23 16:04:18 +09:00
Zhijie Shen 43302f6f44 YARN-2815. Excluded transitive dependency of JLine in hadoop-yarn-server-common. Contributed by Ferdinand Xu. 2015-01-16 16:37:42 -08:00
Junping Du d468c9aaf1 YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 2014-12-23 20:04:33 -08:00
Junping Du 856b46cf47 Revert ca3381de61 and 20674255ad as missing JIRA number in commit message 2014-12-23 20:02:33 -08:00
Junping Du ca3381de61 Fix new findbugs warnings in rest of the hadoop-yarn components. (Contributed by Li Lu) 2014-12-23 18:57:16 -08:00
Karthik Kambatla a04143039e YARN-2236. [YARN-1492] Shared Cache uploader service on the Node Manager. (Chris Trezzo and Sanjin Lee via kasha) 2014-11-12 09:31:05 -08:00
Arun C. Murthy 43cd07b408 YARN-2830. Add backwords compatible ContainerId.newInstance constructor. Contributed by Jonathan Eagles. 2014-11-09 14:57:37 -08:00
Vinod Kumar Vavilapalli 6ce32f593b YARN-2789. Re-instated the NodeReport.newInstance private unstable API modified in YARN-2698 so that tests in YARN frameworks don't break. Contributed by Wangda Tan. 2014-10-31 15:32:21 -07:00
Karthik Kambatla 256697acd5 YARN-2186. [YARN-1492] Node Manager uploader service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 2014-10-31 13:00:42 -07:00
Vinod Kumar Vavilapalli e0233c16eb YARN-2698. Moved some node label APIs to be correctly placed in client protocol. Contributed by Wangda Tan. 2014-10-30 22:59:31 -07:00
Vinod Kumar Vavilapalli a16d022ca4 YARN-2704. Changed ResourceManager to optionally obtain tokens itself for the sake of localization and log-aggregation for long-running services. Contributed by Jian He. 2014-10-27 15:49:47 -07:00
Jian He c05b581a55 YARN-2591. Fixed AHSWebServices to return FORBIDDEN(403) if the request user doesn't have access to the history data. Contributed by Zhijie Shen 2014-10-27 10:45:53 -07:00
Karthik Kambatla c51e53d7aa YARN-2183. [YARN-1492] Cleaner service for cache manager. (Chris Trezzo and Sangjin Lee via kasha) 2014-10-25 10:31:06 -07:00
Zhijie Shen 0f3b6900be YARN-2209. Replaced AM resync/shutdown command with corresponding exceptions and made related MR changes. Contributed by Jian He. 2014-10-23 21:56:03 -07:00
Vinod Kumar Vavilapalli c0e034336c YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 2014-10-21 20:09:40 -07:00
Jian He 0af1a2b5bc YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA 2014-10-15 15:22:07 -07:00
Zhijie Shen 1220bb72d4 YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen. 2014-10-14 21:50:46 -07:00
Karthik Kambatla 4f426fe223 YARN-2180. [YARN-1492] In-memory backing store for cache manager. (Chris Trezzo via kasha) 2014-10-09 17:16:06 -07:00
Zhijie Shen 52bbe0f11b YARN-2630. Prevented previous AM container status from being acquired by the current restarted AM. Contributed by Jian He. 2014-10-01 15:38:11 -07:00
Karthik Kambatla 17d1202c35 YARN-2179. [YARN-1492] Initial cache manager structure and context. (Chris Trezzo via kasha) 2014-09-30 19:01:47 -07:00
Jian He 0a641496c7 YARN-1372. Ensure all completed containers are reported to the AMs across RM restart. Contributed by Anubhav Dhoot 2014-09-22 10:30:53 -07:00
junping_du 90a0c03f0a YARN-1250. Generic history service should support application-acls. (Contributed by Zhijie Shen) 2014-09-16 18:20:49 -07:00
Jian He 3122daa802 YARN-2229. Changed the integer field of ContainerId to be long type. Contributed by Tsuyoshi OZAWA 2014-09-12 10:33:33 -07:00
junping_du 6b8b1608e6 YARN-2033. Merging generic-history into the Timeline Store (Contributed by Zhijie Shen) 2014-09-12 10:04:51 +08:00
Jian He 83be3ad444 YARN-415. Capture aggregate memory allocation at the app-level for chargeback. Contributed by Eric Payne & Andrey Klochkov 2014-09-10 18:20:54 -07:00
Zhijie Shen a9023c2736 YARN-2397. Avoided loading two authentication filters for RM and TS web interfaces. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1618054 13f79535-47bb-0310-9956-ffa450edef68
 2014-08-14 21:17:20 +00:00
Zhijie Shen 1d6e178144 YARN-2347. Consolidated RMStateVersion and NMDBSchemaVersion into Version in yarn-server-common. Contributed by Junping Du. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1614838 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-31 09:27:43 +00:00
Zhijie Shen d6532d3a77 YARN-2247. Made RM web services authenticate users via kerberos and delegation token. Contributed by Varun Vasudev. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1613821 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-27 17:55:06 +00:00
Devarajulu K 2050e0dad6 YARN-1342. Recover container tokens upon nodemanager restart. Contributed by Jason Lowe. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1612995 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-24 05:02:00 +00:00
Junping Du 403ec8ea80 YARN-1341. Recover NMTokens upon nodemanager restart. (Contributed by Jason Lowe) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611512 13f79535-47bb-0310-9956-ffa450edef68
 2014-07-17 23:33:22 +00:00
Vinod Kumar Vavilapalli e285b98f0f YARN-2152. Added missing information into ContainerTokenIdentifier so that NodeManagers can report the same to RM when RM restarts. Contributed Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605205 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-24 21:43:22 +00:00
Jian He 95897ca14b YARN-1885. Fixed a bug that RM may not send application-clean-up signal to NMs where the completed applications previously ran in case of RM restart. Contributed by Wangda Tan 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1603028 13f79535-47bb-0310-9956-ffa450edef68
 2014-06-16 23:56:12 +00:00
Vinod Kumar Vavilapalli 23c325ad47 YARN-2115. Replaced RegisterNodeManagerRequest's ContainerStatus with a new NMContainerStatus which has more information that is needed for work-preserving RM-restart. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1598790 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-31 00:20:50 +00:00
Karthik Kambatla 4a19196a34 YARN-1987. Wrapper for leveldb DBIterator to aid in handling database exceptions. (Jason Lowe via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1593757 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-11 06:11:04 +00:00
Haohui Mai d777cb8996 HADOOP-10563. Remove the dependency of jsp in trunk. Contributed by Haohui Mai. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1592069 13f79535-47bb-0310-9956-ffa450edef68
 2014-05-02 22:26:02 +00:00
Jian He 71c2b159ab YARN-1521. Mark Idempotent/AtMostOnce annotations to the APIs in ApplicationClientProtcol, ResourceManagerAdministrationProtocol and ResourceTrackerProtocol so that they work in HA scenario. Contributed by Xuan Gong 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1581678 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-26 03:43:41 +00:00
Vinod Kumar Vavilapalli 8314674947 YARN-1685. Fixed few bugs related to handling of containers' log-URLs on ResourceManager and history-service. Contributed by Zhijie Shen. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578602 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 21:36:21 +00:00
Jonathan Turner Eagles 7944eab64a YARN-1136. Replace junit.framework.Assert with org.junit.Assert (Chen He via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1578539 13f79535-47bb-0310-9956-ffa450edef68
 2014-03-17 20:03:35 +00:00
Vinod Kumar Vavilapalli 990cffdcfa YARN-1553. Modified YARN and MR to stop using HttpConfig.isSecure() and 
instead rely on the http policy framework. And also fix some bugs related
to https handling in YARN web-apps. Contributed by Haohui Mai.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1568501 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-14 20:01:02 +00:00
Zhijie Shen ebe0c17a95 YARN-1461. Added tags for YARN applications and changed RM to handle them. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1564633 13f79535-47bb-0310-9956-ffa450edef68
 2014-02-05 04:55:00 +00:00
Vinod Kumar Vavilapalli cbee889711 YARN-321. Merging YARN-321 branch to trunk. 
svn merge ../branches/YARN-321


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1561452 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-26 04:51:10 +00:00
Vinod Kumar Vavilapalli 7c8b654ba5 YARN-888. Cleaned up POM files so that non-leaf modules don't include any dependencies and thus compact the dependency list for leaf modules. Contributed by Alejandro Abdelnur. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1557801 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-13 18:34:02 +00:00
Karthik Kambatla 2cddd21db9 YARN-1559. Race between ServerRMProxy and ClientRMProxy setting RMProxy#INSTANCE. (kasha and vinodkv via kasha) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1555970 13f79535-47bb-0310-9956-ffa450edef68
 2014-01-06 18:40:15 +00:00
Vinod Kumar Vavilapalli ca125153b3 YARN-1028. Added FailoverProxyProvider capability to ResourceManager to help with RM failover. Contributed by Karthik Kambatla. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1551739 13f79535-47bb-0310-9956-ffa450edef68
 2013-12-17 22:32:38 +00:00
Vinod Kumar Vavilapalli cfa783141f YARN-1210. Changed RM to start new app-attempts on RM restart only after ensuring that previous AM exited or after expiry time. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1543310 13f79535-47bb-0310-9956-ffa450edef68
 2013-11-19 05:17:20 +00:00
Daryn Sharp b858b812b7 HADOOP-9470. eliminate duplicate FQN tests in different Hadoop modules (Ivan A. Veselovsky via daryn) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1530667 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-09 15:09:37 +00:00
Bikas Saha cbab04727b YARN-1232. Configuration to support multiple RMs (Karthik Kambatla via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1529251 13f79535-47bb-0310-9956-ffa450edef68
 2013-10-04 18:40:18 +00:00
Jonathan Turner Eagles 5a15c392a1 YARN-819. ResourceManager and NodeManager should check for a minimum allowed version (Robert Parker via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1526660 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-26 19:55:26 +00:00
Arun Murthy 60a1a70197 YARN-1170. YARN & MapReduce proto definitions fixed to specify protobuf package as hadoop.yarn and hadoop.mapreduce respectively. Contibuted by Binglin Chang. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1523652 13f79535-47bb-0310-9956-ffa450edef68
 2013-09-16 14:04:04 +00:00
Siddharth Seth b776bd46ae YARN-1045. Improve toString implementation for PBImpls. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514185 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-15 07:20:14 +00:00
Alejandro Abdelnur 8451ab5c01 HADOOP-9872. Improve protoc version handling and detection. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1514068 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-14 22:15:04 +00:00
Alejandro Abdelnur 19b36f0900 HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1513281 13f79535-47bb-0310-9956-ffa450edef68
 2013-08-12 22:15:10 +00:00
Vinod Kumar Vavilapalli 817a654346 YARN-945. Removed setting of AMRMToken's service from ResourceManager and changed client libraries do it all the time and correctly. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1508232 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-29 23:17:39 +00:00
Alejandro Abdelnur f179afc68d YARN-937. Fix unmanaged AM in non-secure/secure setup post YARN-701. (tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1507706 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-27 20:04:38 +00:00
Vinod Kumar Vavilapalli 3ca3137179 YARN-926. Modified ContainerManagerProtcol APIs to take in requests for multiple containers. Contributed by Jian He. 
MAPREDUCE-5412. Update MR app to use multiple containers API of ContainerManager after YARN-926. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1506391 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-24 03:41:03 +00:00
Arun Murthy ac914f79bc YARN-918. Remove ApplicationAttemptId from RegisterApplicationMasterRequestProto. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1504735 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-19 00:57:40 +00:00
Bikas Saha 33b5a81509 YARN-513. Create common proxy client for communicating with RM (Xuan Gong & Jian He via bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1503933 13f79535-47bb-0310-9956-ffa450edef68
 2013-07-16 22:47:40 +00:00
Vinod Kumar Vavilapalli 243bcd367f YARN-694. Starting to use NMTokens to authenticate all communication with NodeManagers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1494369 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-18 23:19:49 +00:00
Vinod Kumar Vavilapalli 5d1b453b85 YARN-610. ClientToken is no longer set in the environment of the Containers. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493968 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 23:13:17 +00:00
Vinod Kumar Vavilapalli f5f8f3bca4 YARN-825. Fixed javadoc and annotations for yarn-common module. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493634 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 03:32:49 +00:00
Arun Murthy 77e134d13f YARN-387. Renamed YARN protocols for consistency. Contributed by Vinod K V. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493623 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-17 02:27:18 +00:00
Vinod Kumar Vavilapalli 769a0bd831 YARN-693. Modified RM to send NMTokens on allocate call so that AMs can then use them for authentication with NMs. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1493448 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-16 03:11:33 +00:00
Vinod Kumar Vavilapalli e9f753a11e YARN-792. Moved NodeHealthStatus from yarn.api.record to yarn.server.api.record. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492943 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 03:17:03 +00:00
Vinod Kumar Vavilapalli f0eb4bc342 YARN-692. Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. Contributed by Omkar Vinit Joshi. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492907 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-14 00:06:42 +00:00
Vinod Kumar Vavilapalli c0cd68d8fb YARN-773. Moved YarnRuntimeException from package api.yarn to api.yarn.exceptions. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492903 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-13 23:25:14 +00:00
Jonathan Turner Eagles b90755c296 YARN-427. Coverage fix for org.apache.hadoop.yarn.server.api.* (Aleksey Gorshkov via jeagles) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1492282 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-12 16:52:15 +00:00
Alejandro Abdelnur af8514eef2 YARN-686. Flatten NodeReport. (sandyr via tucu) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490827 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-07 20:59:23 +00:00
Bikas Saha 9fcfbf5f51 YARN-759. Create Command enum in AllocateResponse (bikas) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1490470 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-06 23:33:48 +00:00
Vinod Kumar Vavilapalli 66de4ccf60 YARN-724. Moved ProtoBase from api.records to api.records.impl.pb. Contributed by Jian He. 
MAPREDUCE-5303. Changed MR app after moving ProtoBase to package impl.pb via YARN-724. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489658 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 22:43:19 +00:00
Vinod Kumar Vavilapalli 978012b9b6 YARN-755. Renamed AllocateResponse.reboot to AllocateResponse.resync. Contributed by Bikas Saha. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489295 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 05:53:47 +00:00
Vinod Kumar Vavilapalli 28aabe0b2b YARN-756. Move Preemption* records to yarn.api where they really belong. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489290 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 05:03:51 +00:00
Vinod Kumar Vavilapalli a83fb61ac0 YARN-635. Renamed YarnRemoteException to YarnException. Contributed by Siddharth Seth. 
MAPREDUCE-5301. Updated MR code to work with YARN-635 changes of renaming YarnRemoteException to YarnException. Contributed by Siddharth Seth


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489283 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 04:05:50 +00:00
Vinod Kumar Vavilapalli d33534c4fb YARN-748. Moved BuilderUtils from yarn-common to yarn-server-common for eventual retirement. Contributed by Jian He. 
MAPREDUCE-5297. Updated MR App since BuilderUtils is no longer public after YARN-748. Contributed by Jian He.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1489257 13f79535-47bb-0310-9956-ffa450edef68
 2013-06-04 00:34:18 +00:00
Siddharth Seth b16c5638b5 YARN-719. Move RMIdentifier from Container to ContainerTokenIdentifier. Contributed by Vinod Kumar Vavilapalli. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1487741 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-30 04:59:05 +00:00
Vinod Kumar Vavilapalli 065747efab YARN-628. Fix the way YarnRemoteException is being unrolled to extract out the underlying exception. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1483207 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-16 06:58:34 +00:00
Vinod Kumar Vavilapalli 68148989bf YARN-590. Added an optional mesage to be returned by ResourceMaanger when RM asks an RM to shutdown/resync etc so that NMs can log this message locally for better debuggability. Contributed by Mayank Bansal. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481234 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 23:37:44 +00:00
Vinod Kumar Vavilapalli a35c7fd80b YARN-663. Changed ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException. Contributed by Xuan Gong. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481215 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:58:59 +00:00
Vinod Kumar Vavilapalli 2638bc67a4 YARN-634. Modified YarnRemoteException to be not backed by PB and introduced a separate SerializedException record. Contributed by Siddharth Seth. 
MAPREDUCE-5239. Updated MR App to reflect YarnRemoteException changes after YARN-634. Contributed by Siddharth Seth.


git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1481205 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-10 21:49:28 +00:00
Vinod Kumar Vavilapalli c0fa9d2231 YARN-645. Moved RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager where it really belongs. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1479022 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-04 01:12:17 +00:00
Vinod Kumar Vavilapalli fc275fed66 YARN-625. Move the utility method unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil. Contributed by Siddharth Seth. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478594 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-02 22:44:12 +00:00
Vinod Kumar Vavilapalli 46b2898a1a YARN-618. Modified RM_INVALID_IDENTIFIER to be -1 instead of zero. Contributed by Jian He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1478230 13f79535-47bb-0310-9956-ffa450edef68
 2013-05-01 23:23:20 +00:00
Vinod Kumar Vavilapalli 635f96e74e YARN-562. Missed files from previous commit. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1476038 13f79535-47bb-0310-9956-ffa450edef68
 2013-04-26 04:42:34 +00:00