Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,903 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

682 Commits

Author SHA1 Message Date
Ray Chiang fb9327edae YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang) 
(cherry picked from commit 0a5def155e)
 2016-07-01 15:55:56 -07:00
Arun Suresh e64d21b297 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 
(cherry picked from commit 99e5dd68d0)
 2016-06-22 19:05:28 -07:00
Sangjin Lee 2a79910c22 YARN-4958. The file localization process should allow for wildcards to reduce the application footprint in the state store (Daniel Templeton via sjlee) 
(cherry picked from commit 5107a967fa)
 2016-06-20 10:00:41 -07:00
Junping Du ea4cab2330 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040)
 2016-06-19 17:45:57 -07:00
Junping Du 6afa044dda YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 
(cherry picked from commit c35fa4a0e5)
 2016-06-17 08:25:33 -07:00
Junping Du 6dd34baf3c YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 
(cherry picked from commit 5dfc38ff57)
 2016-06-15 16:20:07 -07:00
Wangda Tan 863bfa4d6a YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:12:00 -07:00
Arun Suresh f3c7a99e33 YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 7dae2b3bc4)
 2016-06-12 10:06:18 -07:00
Vinod Kumar Vavilapalli f6cd1bcf89 YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 
(cherry picked from commit 9378d9428f)
 2016-06-09 12:32:53 -07:00
Naganarasimha 27fc225558 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 
(cherry picked from commit 1500a0a300)
 2016-06-09 05:44:41 +05:30
Arun Suresh c29fbdd8ac YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 3344ba70e0)
 2016-06-08 08:32:17 -07:00
Arun Suresh 154c7c343b YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 76f0800c21)
 2016-06-07 17:17:22 -07:00
Xuan 10f0c0475e YARN-5199. Close LogReader in in AHSWebServices#getStreamingOutput and 
FileInputStream in NMWebServices#getLogs. Contributed by Xuan Gong

(cherry picked from commit 58be55b6e0)
 2016-06-07 16:08:54 -07:00
Rohith Sharma K S a29f8bdc3d YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula. 
(cherry picked from commit bddea5fe5f)
 2016-06-07 11:23:02 +05:30
Ming Ma 74e2b5efa2 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010)
 2016-06-06 14:34:47 -07:00
Jian He 7f865bf438 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29)
 2016-06-03 11:11:20 -07:00
Arun Suresh 384a84828f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 
(cherry picked from commit dc26601d8f)
 2016-06-02 09:14:30 -07:00
Xuan 04062593b4 YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong 
(cherry picked from commit 0bc05e40fa)
 2016-06-01 13:45:24 -07:00
Varun Vasudev d7bb28f15b YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 
(cherry picked from commit 86fb58b7dc)
 2016-05-29 21:33:43 +05:30
Varun Vasudev 4f36c3d214 YARN-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:55:39 +05:30
Arun Suresh de28ca1e92 YARN-5117. QueuingContainerManager does not start GUARANTEED Container even if Resources are available. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 4fc09a897b)
 2016-05-27 17:50:34 -07:00
Arun Suresh 0b1472fbc0 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 
(cherry picked from commit aa975bc781)
 2016-05-27 14:07:15 -07:00
Carlo Curino c656977961 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e)
 2016-05-25 16:58:19 -07:00
Jason Lowe da77040793 YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e9)
 2016-05-25 21:29:27 +00:00
Robert Kanter 71e5b36763 YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter) 
(cherry picked from commit e07519b8db)
 2016-05-25 10:25:59 -07:00
Varun Vasudev 5361a91909 YARN-5115. Avoid setting CONTENT-DISPOSITION header in the container-logs web-service. Contributed by Xuan Gong. 
(cherry picked from commit c52c15dd2df5050c6c3ddbe5ee708404f1920721)
 2016-05-25 19:01:22 +05:30
Varun Vasudev 36f2ae0692 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
 2016-05-25 19:01:22 +05:30
Jason Lowe e8bd2a5be6 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf)
 2016-05-23 15:16:52 +00:00
Junping Du 867cd2f5a2 YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He. 2016-05-22 16:02:57 -07:00
Varun Vasudev e7f396fd47 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 
(cherry picked from commit 0287c49107)
 2016-05-20 12:32:29 +05:30
Junping Du 2703ec6871 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron. 2016-05-19 14:10:15 -07:00
Arun Suresh 15808717f3 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 1597630681)
 2016-05-18 23:20:54 -07:00
Arun Suresh bbcea3abf3 YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f45bc5a83e)
 2016-05-18 23:12:19 -07:00
Arun Suresh 69a9ce3494 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 
(cherry picked from commit f0ac18d001)
 2016-05-18 23:09:11 -07:00
Arun Suresh 005dae521c YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d971bf252f)
 2016-05-18 22:36:28 -07:00
Arun Suresh 1b4600abcf YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 
(cherry picked from commit 341888a0aa)
 2016-05-18 22:35:55 -07:00
Arun Suresh 307cda70db YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit d464f4d1c4)
 2016-05-18 22:09:30 -07:00
Arun Suresh d80d24aabf YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh) 
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
(cherry picked from commit c282a08f38)
 2016-05-18 22:08:55 -07:00
Karthik Kambatla b56fc51b70 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 
(cherry picked from commit c8172f5f14)
 2016-05-18 22:08:20 -07:00
Jian He 264c06a438 YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 2016-05-17 15:02:14 -07:00
Jason Lowe 575c056357 YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcd)
 2016-05-16 15:43:20 +00:00
Sangjin Lee d6d13ec670 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 
(cherry picked from commit 0bbe01f8d5)
 2016-05-12 10:14:24 -07:00
Varun Vasudev 38a3b86141 YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 
(cherry picked from commit 72b047715c)
 2016-05-05 13:02:38 +05:30
Wangda Tan 585299146a YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c)
 2016-05-04 11:39:25 -07:00
Junping Du 1ffb0c43d6 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
 2016-05-04 10:36:31 -07:00
Robert Kanter ac8fb579c6 Remove parent's env vars from child processes 2016-04-29 09:26:09 -07:00
Varun Vasudev 6561e3b500 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 
(cherry picked from commit 0f25a1bb52)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java
 2016-04-29 16:22:06 +05:30
Jian He 6ba39a1597 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)
 2016-04-28 21:54:30 -07:00
Karthik Kambatla 52bfa90fed YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha) 
(cherry picked from commit 1a3f1482e2)
 2016-04-26 06:18:27 -07:00
Jason Lowe ece01478c5 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 
(cherry picked from commit 3150ae8108)
 2016-04-14 19:19:46 +00:00
Robert Kanter e79a47670b YARN-4541. Change log message in LocalizedResource#handle() to DEBUG (rchiang via rkanter) 
(cherry picked from commit 0d9194df00)
 2016-04-13 17:45:36 -07:00
Naganarasimha 53c24e00e8 YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt. 
(cherry picked from commit 437e9d6475)
 2016-04-12 18:25:11 +05:30
Vinod Kumar Vavilapalli f1dcd40294 YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 
(cherry picked from commit 44bbc50d91)
 2016-04-11 12:12:30 -07:00
Akira Ajisaka 8cf6630fc6 YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12)
 2016-04-11 14:55:37 +09:00
Karthik Kambatla 2b97a50eec YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 
(cherry picked from commit e82f961a39)
 2016-04-07 17:30:54 -07:00
Varun Vasudev 8f9b97ccce YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He. 
(cherry picked from commit b41e65e5bc)
 2016-04-06 13:42:06 +05:30
Junping Du 0907ce8c93 YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 0005816743)
 2016-04-05 09:02:50 -07:00
naganarasimha 3772602848 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195)
 2016-04-04 18:08:18 +05:30
Jian He 3afc2caec8 YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage 2016-03-31 14:31:38 -07:00
Jason Lowe edf17fe8e5 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 
(cherry picked from commit 948b758070)
 2016-03-28 23:02:15 +00:00
Jian He bdc648ebe7 YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola 2016-03-27 20:22:49 -07:00
Eric Payne dd1e4107e5 YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger. 
(cherry picked from commit 92b7e0d413)
 2016-03-18 17:05:53 +00:00
Robert Kanter da24fde333 TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe)
 2016-03-15 10:05:28 -07:00
Vinod Kumar Vavilapalli da9f39b107 YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv) 
(cherry picked from commit b2661765a5)
 2016-03-07 11:11:29 -08:00
Jason Lowe 4eace7ab43 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 
(cherry picked from commit 059caf9989)
 2016-03-07 15:45:47 +00:00
Varun Vasudev 78919f8c34 YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e8)
 2016-03-07 16:19:27 +05:30
Varun Vasudev e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Haohui Mai c5db4ab0b4 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 14:10:18 -08:00
Jason Lowe bd0f5085e3 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c5)
 2016-02-29 15:26:26 +00:00
Ming Ma 1656bcec5f YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 
(cherry picked from commit 7f3139e54d)
 2016-02-26 08:43:14 -08:00
Robert Kanter 872b8d90a6 YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 
(cherry picked from commit d7fdec1e6b)
 2016-02-25 16:40:05 -08:00
Robert Kanter c2098d2470 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 
(cherry picked from commit 954dd57043)
 2016-02-24 15:00:48 -08:00
Jason Lowe acffe82353 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 
(cherry picked from commit d284e187b8)
 2016-02-23 20:51:01 +00:00
Varun Vasudev 2c218ca8a8 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d745)
 2016-02-23 12:32:16 +05:30
Varun Vasudev 3a5b39e931 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 
(cherry picked from commit fa00d3e205)
 2016-02-11 12:09:29 +05:30
Jason Lowe b12f16ab73 YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39)
 2016-02-03 17:22:12 +00:00
Varun Vasudev bbda446889 YARN-4649. Add additional logging to some NM state store operations. Contributed by Sidharta Seethana 
(cherry picked from commit 1cd55e0c17)
 2016-02-02 09:26:22 +05:30
Wangda Tan 9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:19:19 +08:00
Rohith Sharma K S b8d5df06f2 YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac68666803)
 2016-01-29 12:30:53 +05:30
Jason Lowe 29195c7668 YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 
(cherry picked from commit 61382ff8fa)
 2016-01-28 19:00:51 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 47b20d0c74 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 
(cherry picked from commit 2085e60a96)
 2016-01-25 16:22:47 -08:00
Jian He 2d236c9de2 YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:40 -08:00
Xuan 1d19557dda YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He

(cherry picked from commit 618bfd6ac2)
 2016-01-22 18:21:50 -08:00
Jason Lowe 635849fa45 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0)
 2016-01-22 21:55:59 +00:00
Varun Vasudev 85561207f9 YARN-4578. Directories that are mounted in docker containers need to be more restrictive/container-specific. Contributed by Sidharta Seethana. 
(cherry picked from commit b41a7e89d1)
 2016-01-22 14:44:22 +05:30
Wangda Tan 23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)
 2016-01-19 21:28:51 +08:00
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Colin Patrick Mccabe 48b8c99479 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810)
 2016-01-15 13:22:14 -08:00
Colin Patrick Mccabe cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Varun Vasudev 6220a024b1 YARN-4553. Add cgroups support for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 3ddb92bd30)
 2016-01-14 14:30:19 +05:30
Jason Lowe d164cdcda1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:07 +00:00
Akira Ajisaka 3905435d3d YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c4)
 2016-01-08 01:13:30 +09:00
Steve Loughran d9459ffa00 YARN-4550. Some tests in TestContainerLanch fails on non-english locale environment. (Takashi Ohnishi via stevel) 2016-01-07 14:30:07 +00:00
rohithsharmaks 4865642e7c YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae)
 2016-01-07 09:39:37 +05:30
Rohith Sharma K S ae25e283cf YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit. (tangshangwen via rohithsharmaks) 
(cherry picked from commit f9e36dea96)
 2016-01-04 10:04:27 +05:30
Gera Shegalov 8baf9369bb YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 
(cherry picked from commit 2c17b81569)
 2015-12-24 23:52:43 -08:00
Uma Mahesh fc9e130825 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878)
 2015-12-19 23:17:33 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) d9b9a75928 YARN-1856. Added cgroups based memory monitoring for containers as another alternative to custom memory-monitoring. Contributed by Varun Vasudev. 
(cherry picked from commit 4e7d32c0db)
 2015-12-17 12:13:56 -08:00
Wangda Tan 9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
 2015-12-16 13:28:56 -08:00
Jian He 72544db3cd YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 14:59:18 -08:00
Wangda Tan 9f979107c2 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb09)
 2015-12-14 11:13:51 -08:00
rohithsharmaks f626d18cc3 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 
(cherry picked from commit 15c3e7ffe3)
 2015-12-09 10:51:57 +05:30
Steve Loughran af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Jian He 23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)
 2015-12-03 12:49:22 -08:00
Junping Du e76ba91fa5 YARN-4408. Fix issue that NodeManager still reports negative running containers. Contributed by Robert Kanter. 
(cherry picked from commit 62e9348bc1)
 2015-12-03 06:37:31 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Tsuyoshi Ozawa d76b523b02 YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83)
 2015-11-26 01:10:41 +09:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a)
 2015-11-24 22:36:19 +00:00
Karthik Kambatla 49ed955c91 YARN-3980. Plumb resource-utilization info in node heartbeat through to the scheduler. (Inigo Goiri via kasha) 
(cherry picked from commit 52948bb20b)
 2015-11-24 13:52:29 +05:30
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Junping Du 1d107d8058 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b)
 2015-11-15 04:44:59 -08:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Jason Lowe c75d8b164f YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena 
(cherry picked from commit e2267de207)
 2015-10-29 16:39:55 +00:00
Akira Ajisaka d9156c1245 YARN-4300. [JDK8] Fix javadoc errors caused by wrong tags. (aajisaka) 
(cherry picked from commit 8a68630dd1)
 2015-10-27 10:28:30 +09:00
Wangda Tan 9bf1e74171 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 
(cherry picked from commit 6f606214e7)
 2015-10-26 16:38:39 -07:00
Rohith Sharma K S 33455fa415 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 5acdde4744)
 2015-10-26 15:45:22 +05:30
Tsuyoshi Ozawa e85ebf0825 YARN-3528. Tests with 12345 as hard-coded port break jenkins. Contributed by Brahma Reddy Battula. 
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java
 2015-10-26 17:06:46 +09:00
Allen Wittenauer 599653ad92 YARN-3724. Use POSIX nftw(3) instead of fts(3) (Alan Burlison via aw) 2015-10-25 21:43:51 -07:00
Jonathan Eagles 6db7bfbcfd YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 
(cherry picked from commit f8adeb712d)
 2015-10-23 10:43:59 -05:00
Varun Vasudev e4429de86f YARN-4267. Add additional logging to container launch implementations in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 151eab2a12)
 2015-10-19 20:56:43 +05:30
Varun Vasudev 196d549e2b YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit e39ae0e676)
 2015-10-19 20:21:48 +05:30
Steve Loughran 390f29ab99 YARN-4155. TestLogAggregationService.testLogAggregationServiceWithInterval failing. (Bibin A Chundatt via stevel) 2015-10-18 12:03:46 +01:00
Varun Vasudev 342870b70f YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 63020c54c1)
 2015-10-15 13:41:58 +05:30
Steve Loughran 0e4fb329ed HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:33 +01:00
Varun Vasudev 2dd9c475c4 YARN-4255. container-executor does not clean up docker operation command files. Contributed by Sidharta Seethana. 
(cherry picked from commit da1ee078f9)
 2015-10-14 15:27:18 +05:30
Varun Vasudev 0aeedfd572 YARN-4252. Log container-executor invocation details when exit code is non-zero. Contributed by Sidharta Seethana. 
(cherry picked from commit fb9c1519b3)
 2015-10-14 15:19:19 +05:30
Varun Vasudev a9b03b5afb YARN-4253. Standardize on using PrivilegedOperationExecutor for all invocations of container-executor in LinuxContainerExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit  8d59293089)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-10-14 15:04:22 +05:30
Steve Loughran 5eab51a24f HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:47:54 +01:00
Varun Vasudev 34b8142802 YARN-4017. container-executor overuses PATH_MAX. Contributed by Sidharta Seethana 
(cherry picked from commit 5b6bae0094)
 2015-10-13 12:50:28 +05:30
Jason Lowe 0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad)
 2015-10-09 14:19:41 +00:00
Jason Lowe ec74b0cd4e YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 
(cherry picked from commit 8d226225d0)
 2015-10-08 22:26:35 +00:00
Wangda Tan d24a3b9a3c YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 
(cherry picked from commit 30ac69c6bd)
 2015-10-05 15:48:40 -07:00
Vinod Kumar Vavilapalli b876babdc7 HADOOP-12441. Fixed shell-kill command behaviour to work correctly on some Linux distributions after HADOOP-12317. Contributed by Wangda Tan. 
(cherry picked from commit f1c19b9365)
 2015-10-05 10:57:55 -07:00
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde)
 2015-10-02 18:53:04 -07:00
Jason Lowe 3e3733437f YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 
(cherry picked from commit fdf02d1f26)
 2015-10-02 20:21:32 +00:00
Jason Lowe d645ee1d62 YARN-3727. For better error recovery, check if the directory exists before using it for localization. Contributed by Zhihai Xu 
(cherry picked from commit 854d25b0c3)
 2015-09-30 15:01:01 +00:00
Jian He d592aa115b YARN-3868. Recovery support for container resizing. Contributed by Meng Ding 
(cherry picked from commit c57eac5dfe)
 2015-09-24 10:24:13 -07:00
Jian He b6c594c6ea YARN-1644. RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing. Contributed by Meng Ding 
(cherry picked from commit c3dc1af072)
 2015-09-24 10:24:04 -07:00
Jian He b8955d81c5 YARN-1643. Make ContainersMonitor support changing monitoring size of an allocated container. Contributed by Meng Ding and Wangda Tan 
(cherry picked from commit c59ae4eeb1)
 2015-09-24 10:23:55 -07:00
Jian He fa0a554ae0 YARN-3867. ContainerImpl changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit 5f5a968d65)
 2015-09-24 10:23:46 -07:00
Jian He afe4afd0c9 YARN-1645. ContainerManager implementation to support container resizing. Contributed by Meng Ding & Wangda Tan 
(cherry picked from commit ffd820c27a)
 2015-09-24 10:23:35 -07:00
Jian He c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10)
 2015-09-24 10:23:25 -07:00
Rohith Sharma K S ace4d26936 YARN-4152. NodeManager crash with NPE when LogAggregationService#stopContainer called for absent container. (Bibin A Chundatt via rohithsharmaks) 2015-09-24 11:31:47 +05:30
Jason Lowe c829be5133 YARN-4095. Avoid sharing AllocatorPerContext object in LocalDirAllocator between ShuffleHandler and LocalDirsHandlerService. Contributed by Zhihai Xu 
(cherry picked from commit c890c51a91)
 2015-09-23 15:44:13 +00:00
Xuan f3c0a210b6 YARN-4149. yarn logs -am should provide an option to fetch all the log 
files. Contributed by Varun Vasudev

(cherry picked from commit 34ef1a092b)
 2015-09-15 14:37:59 -07:00
Jason Lowe 181bda0421 YARN-4158. Remove duplicate close for LogWriter in AppLogAggregatorImpl#uploadLogsForContainers. Contributed by Zhihai Xu 
(cherry picked from commit 8c1cdb17a0)
 2015-09-15 20:22:23 +00:00
Varun Vasudev a4288f1262 YARN-4136. LinuxContainerExecutor loses info when forwarding ResourceHandlerException. Contributed by Bibin A Chundatt. 
(cherry picked from commit 486d5cb803)
 2015-09-11 14:39:01 +05:30
Wangda Tan 5decd8f9f7 YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) 
(cherry picked from commit 77666105b4)
 2015-09-10 09:30:34 -07:00
Zhihai Xu 214338bd68 YARN-4096. App local logs are leaked if log aggregation fails to initialize for the app. Contributed by Jason Lowe. 
(cherry picked from commit 16b9037dc1)
 2015-09-08 12:33:29 -07:00
Jian He f23be93dd1 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 
(cherry picked from commit 6f72f1e600)
 2015-09-08 11:24:07 +08:00