Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,753 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

823 Commits

Author SHA1 Message Date
Varun Saxena f572448809 YARN-4388. Cleanup mapreduce.job.hdfs-servers from yarn-default.xml (Junping Du via Varun Saxena) 2016-10-28 02:24:24 +05:30
Subru Krishnan 81595a127f YARN-5711. Propogate exceptions back to client when using hedging RM failover provider. 
(cherry picked from commit 0a166b1347)
 2016-10-24 19:02:03 -07:00
Varun Saxena e9a58691ab YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2016-10-14 23:32:29 +05:30
Jason Lowe 28bd08798d YARN-5551. Ignore file backed pages from memory computation when smaps is enabled. Contributed by Rajesh Balamohan 
(cherry picked from commit ecb51b857a)
 2016-10-11 15:14:08 +00:00
Chris Douglas 539c3e4dc0 YARN-5702. Refactor TestPBImplRecords for reuse in other YARN modules. Contributed by Subru Krishnan 
(cherry picked from commit d65b957776)
 2016-10-05 12:19:21 -07:00
Karthik Kambatla 1578781fe9 YARN-4767. Network issues can cause persistent RM UI outage. (Daniel Templeton via kasha) 
(cherry picked from commit 736d33cddd)
 2016-10-03 14:36:14 -07:00
Naganarasimha 04f594dbea YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 
(cherry picked from commit 6e130c308c)
 2016-10-03 02:13:46 -04:00
Subru Krishnan 9f43f89899 YARN-5384. Expose priority in ReservationSystem submission APIs. (Sean Po via Subru). 
(cherry picked from commit 3a3697deab)
 2016-09-30 19:43:29 -07:00
Jian He 39f896e1a1 YARN-4205. Add a service for monitoring application life time out. Contributed by Rohith Sharma K S 
(cherry picked from commit 2ae5a3a5bf)
 2016-09-29 22:06:36 +08:00
Varun Vasudev 3f1d3e3177 YARN-5662. Provide an option to enable ContainerMonitor. Contributed by Jian He. 
(cherry picked from commit bc2656f09f)
 2016-09-28 15:27:09 +05:30
Arun Suresh 7484d0b1b9 YARN-5609. Expose upgrade and restart API in ContainerManagementProtocol. Contributed by Arun Suresh 2016-09-26 23:55:13 +08:00
Naganarasimha 0e4a5611ff YARN-3692. Allow REST API to set a user generated message when killing an application. Contributed by Rohith Sharma K S 2016-09-24 21:13:01 +05:30
Varun Saxena b58651d8d4 YARN-5539. TimelineClient failed to retry on java.net.SocketTimeoutException: Read timed out (Junping Du via Varun Saxena) 2016-09-23 13:30:56 +05:30
Robert Kanter 0e4f2a0edf YARN-4973. YarnWebParams next.fresh.interval should be next.refresh.interval (templedf via rkanter) 
(cherry picked from commit 5ffd4b7c1e)
 2016-09-22 16:46:27 -07:00
Wangda Tan b9b6396e09 YARN-4591. YARN Web UIs should provide a robots.txt. (Sidharta Seethana via wangda) 
(cherry picked from commit 5a58bfee30)
 2016-09-20 17:23:01 -07:00
Naganarasimha 20729ead91 YARN-5642. Typos in 9 log messages. Contributed by Mehran Hassani 2016-09-17 10:37:42 +05:30
Naganarasimha 656b520dc6 YARN-4232. TopCLI console support for HA mode. Contributed by Bibin A Chundatt 
(cherry picked from commit ade7c2bc9c)
 2016-09-17 09:54:29 +05:30
Ray Chiang 4ee1729cb0 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 14:57:05 -07:00
Arun Suresh b279f42d79 YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b)
 2016-08-30 16:33:31 -07:00
Subru Krishnan 3ad7cb3e1c YARN-5327. API changes required to support recurring reservations in the YARN ReservationSystem. (Sangeetha Abdu Jyothi via Subru). 
(cherry picked from commit b930dc3ec0)
 2016-08-26 17:01:37 -07:00
Junping Du af1502fcab YARN-5557. Add localize API to the ContainerManagementProtocol. Contributed by Jian He. 
(cherry picked from commit 9ef632f3b0)
 2016-08-26 09:05:33 -07:00
Varun Vasudev 49a97a76f5 YARN-5430. Return container's ip and host from NM ContainerStatus call. Contributed by Jian He. 2016-08-26 16:33:21 +05:30
Junping Du d464483bf7 YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 2016-08-18 07:23:29 -07:00
Varun Saxena 5fa7a14da9 YARN=5526. DrainDispacher#serviceStop blocked if setDrainEventsOnStop invoked (sandflee via Varun Saxena) 2016-08-18 12:23:34 +05:30
Varun Saxena 96e9f8bec9 YARN-5475. Fix test failure of TestAggregatedLogFormat#testReadAcontainerLogs1 (Jun Gong via Varun Saxena) 2016-08-16 20:28:02 +05:30
Ray Chiang 372f6f8456 YARN-5137. Make DiskChecker pluggable in NodeManager. (Yufei Gu via rchiang) 
(cherry picked from commit dbe9e70cc084220ea1f68da850cdb092281b5e96)
 2016-08-10 19:26:49 -07:00
Arun Suresh 5f7edb79d1 YARN-5457. Refactor DistributedScheduling framework to pull out common functionality. (asuresh) 
(cherry picked from commit 82c9e06101)
 2016-08-09 00:46:08 -07:00
Junping Du adcecd417b YARN-5470. Differentiate exactly match with regex in yarn log CLI. Contributed by Xuan Gong. 
(cherry picked from commit e605d47df0)
 2016-08-05 11:00:13 -07:00
Arun Suresh f197378f81 YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit e5766b1dbe)
 2016-07-31 11:49:09 -07:00
Li Lu 0b5d96abb5 YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit 7086fc72ee)
 2016-07-28 16:52:21 -07:00
Jason Lowe 2f854b8386 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 
(cherry picked from commit a1890c32c5)
 2016-07-28 21:48:40 +00:00
Junping Du 932bf79244 YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 
(cherry picked from commit 7f3c306e2e)
 2016-07-28 06:36:31 -07:00
Subru Krishnan 4ecfd05095 YARN-5441. Fixing minor Scheduler test case failures 
(cherry picked from commit d2cbfd7de3)
 2016-07-27 18:26:29 -07:00
Arun Suresh 0f6ff20652 YARN-5351. ResourceRequest should take ExecutionType into account during comparison. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2d8d183b19)
 2016-07-26 19:09:22 -07:00
Varun Vasudev 9e326f4eeb YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 
(cherry picked from commit 9ccf935b2c)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java
 2016-07-20 12:55:38 +05:30
Junping Du ad8180fc6c YARN-5213. Fix a bug in LogCLIHelpers which cause TestLogsCLI#testFetchApplictionLogs fails intermittently. Contributed by Xuan Gong. 
(cherry picked from commit dc2f4b6ac8)
 2016-07-19 07:19:17 -07:00
Junping Du e3bc4faa96 YARN-5339. Passing file to -out for YARN log CLI doesnt give warning or error code. Contributed by Xuan Gong. 
(cherry picked from commit 7e5355c14e)
 2016-07-15 09:24:03 -07:00
Vinod Kumar Vavilapalli 4a6cef8c91 YARN-5200. Enhanced "yarn logs" to be able to get a list of containers whose logs are aggregated via a "show_container_log_info" option. Contributed by Xuan Gong. 
(cherry picked from commit eb47163234)
 2016-07-13 10:55:18 -07:00
Robert Kanter 7aaf82d469 YARN-4366. Fix Lint Warnings in YARN Common (templedf via rkanter) 
(cherry picked from commit bf6f4a3b98)
 2016-07-12 15:13:00 -07:00
Jian He e94e6be772 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:37:52 -07:00
Jian He b7604a22e6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 
(cherry picked from commit 8a9d293dd6)
 2016-07-06 16:45:32 -07:00
Wangda Tan e246cf74c9 YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 16:42:35 -07:00
Xuan b7f18f2815 YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong 
(cherry picked from commit 6ab5aa1c1f)
 2016-06-22 21:50:09 -07:00
Arun Suresh e64d21b297 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 
(cherry picked from commit 99e5dd68d0)
 2016-06-22 19:05:28 -07:00
Junping Du ea4cab2330 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040)
 2016-06-19 17:45:57 -07:00
Junping Du 6afa044dda YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 
(cherry picked from commit c35fa4a0e5)
 2016-06-17 08:25:33 -07:00
Junping Du 6dd34baf3c YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 
(cherry picked from commit 5dfc38ff57)
 2016-06-15 16:20:07 -07:00
Varun Vasudev 75235149af YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong. 
(cherry picked from commit 25064fb2fb)
 2016-06-15 12:43:26 +05:30
Wangda Tan 863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Wangda Tan cd3bdbc70a YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 
(cherry picked from commit eec835ec17)
 2016-06-13 21:57:55 -07:00
Arun Suresh fccb641942 YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh) 
(cherry picked from commit 5143277958)
 2016-06-12 09:43:19 -07:00
Wangda Tan e91e9e596d YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 
(cherry picked from commit 03fc6b1bb0)
 2016-06-10 09:53:40 -07:00
Vinod Kumar Vavilapalli f6cd1bcf89 YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 
(cherry picked from commit 9378d9428f)
 2016-06-09 12:32:53 -07:00
Wangda Tan aef6e455b1 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-09 10:08:31 -07:00
Naganarasimha 27fc225558 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 
(cherry picked from commit 1500a0a300)
 2016-06-09 05:44:41 +05:30
Ming Ma 74e2b5efa2 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010)
 2016-06-06 14:34:47 -07:00
Arun Suresh 384a84828f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 
(cherry picked from commit dc26601d8f)
 2016-06-02 09:14:30 -07:00
Wangda Tan b63ebca9dd Addendum patch for YARN-4844, to supress few more deprecation warnings. 2016-06-01 14:37:20 -07:00
Xuan 04062593b4 YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong 
(cherry picked from commit 0bc05e40fa)
 2016-06-01 13:45:24 -07:00
Varun Vasudev d7bb28f15b YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 
(cherry picked from commit 86fb58b7dc)
 2016-05-29 21:33:43 +05:30
Varun Vasudev 4f36c3d214 YARN-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:55:39 +05:30
Arun Suresh 0b1472fbc0 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 
(cherry picked from commit aa975bc781)
 2016-05-27 14:07:15 -07:00
Xuan 38ae5954df YARN-5089. Improve "yarn log" command-line "logFiles" option to support 
regex. Contributed by Xuan Gong

(cherry picked from commit bde819abbb)
 2016-05-26 23:50:14 -07:00
Rohith Sharma K S ef3e0c6fd6 YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt. 
(cherry picked from commit 0a544f8a3e)
 2016-05-27 10:48:47 +05:30
Arun Suresh 44cbf5b7f5 YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh) 
(cherry picked from commit 5b41b288d0)
 2016-05-26 14:58:15 -07:00
Carlo Curino c656977961 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e)
 2016-05-25 16:58:19 -07:00
Karthik Kambatla 0ac8c098e8 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 
(cherry picked from commit 77d5ce9069)
 2016-05-25 10:51:09 -07:00
Robert Kanter 71e5b36763 YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter) 
(cherry picked from commit e07519b8db)
 2016-05-25 10:25:59 -07:00
Naganarasimha 155f25061e YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 
(cherry picked from commit b4078bd17b)
 2016-05-24 08:14:53 +08:00
Varun Vasudev e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 
(cherry picked from commit 0287c49107)
 2016-05-20 12:32:29 +05:30
Junping Du 2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Arun Suresh 1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa)
 2016-05-18 22:35:55 -07:00
Arun Suresh d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f38)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f14)
 2016-05-18 22:08:20 -07:00
Arun Suresh 68964bbaa2 YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4f)
(cherry picked from commit b2a654c5ee)
 2016-05-18 22:02:43 -07:00
Karthik Kambatla 2c6c701aff YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 
(cherry picked from commit fb00794368)
 2016-05-18 22:02:28 -07:00
Varun Vasudev 3c9392a31d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 
(cherry picked from commit ef1757790d)
 2016-05-18 22:45:06 +05:30
Vinod Kumar Vavilapalli e0d6a9632d YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 
(cherry picked from commit 87f5e35133)
 2016-05-09 22:47:58 -07:00
Jason Lowe ee86cef2fe YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit d0da13229c)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
 2016-05-05 14:33:01 +00:00
Vinod Kumar Vavilapalli 8262ef8318 YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 
(cherry picked from commit 9e37fe3b7a)
 2016-05-04 14:17:09 -07:00
Jason Lowe d8fcf02069 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b4839168)
 2016-05-03 22:14:50 +00:00
Varun Vasudev 6561e3b500 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 
(cherry picked from commit 0f25a1bb52)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java
 2016-04-29 16:22:06 +05:30
Jian He 6ba39a1597 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)
 2016-04-28 21:54:30 -07:00
Jian He a9707dceaf YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:28:03 -07:00
Varun Vasudev cf3e93ee73 YARN-4966. Improve yarn logs to fetch container logs without specifying nodeId. Contributed by Xuan Gong. 
(cherry picked from commit 66b07d8374)
 2016-04-27 15:57:17 +05:30
Chris Douglas 878924189d YARN-4976. Missing NullPointer check in ContainerLaunchContextPBImpl. Contributed by Giovanni Matteo Fumarola 
(cherry picked from commit 95a5046607)
 2016-04-21 10:46:35 -07:00
Vinod Kumar Vavilapalli 342bb438f2 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e)
 2016-04-18 11:49:00 -07:00
Xuan 94e4d349b0 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
 2016-04-16 19:40:11 -07:00
Naganarasimha 2213a40f30 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e5)
 2016-04-15 23:40:07 +05:30
Rohith Sharma K S d93cfefea5 YARN-4897. dataTables_wrapper change min height. Contributed by Bibin A Chundatt. 
(cherry picked from commit 4ee7f0a3e7)
 2016-04-12 10:28:18 +05:30
Jason Lowe f1a370ce8b Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 814ceeb489.
 2016-04-11 15:56:29 +00:00
Akira Ajisaka 8cf6630fc6 YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12)
 2016-04-11 14:55:37 +09:00
Varun Vasudev 4baee3eb67 YARN-4562. YARN WebApp ignores the configuration passed to it for keystore settings. Contributed by Sergey Shelukhin. 
(cherry picked from commit 07920acc16)
 2016-04-11 09:40:28 +05:30
Jason Lowe 814ceeb489 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit 1cbcd4a491)
 2016-04-05 13:41:18 +00:00
naganarasimha 3772602848 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195)
 2016-04-04 18:08:18 +05:30
Karthik Kambatla ecfd25dc65 YARN-4657. Javadoc comment is broken for Resources.multiplyByAndAddTo(). (Daniel Templeton via kasha) 
(cherry picked from commit 81d04cae41)
 2016-04-01 16:20:18 -07:00
Arun Suresh f56266fccc Missing file for YARN-4895. 
(cherry picked from commit 5686caa9fc)
 2016-04-01 15:58:38 -07:00
Varun Vasudev 10d8f8a39c Revert "YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki." 
This reverts commit 0064cba169.

(cherry picked from commit 3488c4f2c9)
 2016-04-01 12:21:49 +05:30
Sangjin Lee b91928cd26 YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 
(cherry picked from commit 6d67420dbc)
 2016-03-31 10:49:45 -07:00
Varun Vasudev dcad99892d YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki. 
(cherry picked from commit 0064cba169)
 2016-03-31 14:06:45 +05:30