Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,266 Commits 326 Branches 371 Tags
Commit Graph

628 Commits

Author SHA1 Message Date
Karthik Kambatla
2b97a50eec YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 
(cherry picked from commit e82f961a3925aadf9e53a009820a48ba9e4f78b6)
 2016-04-07 17:30:54 -07:00
Varun Vasudev
8f9b97ccce YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He. 
(cherry picked from commit b41e65e5bc9459b4d950a2c53860a223f1a0d2ec)
 2016-04-06 13:42:06 +05:30
Junping Du
0907ce8c93 YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 00058167431475c6e63c80207424f1d365569e3a)
 2016-04-05 09:02:50 -07:00
naganarasimha
3772602848 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195a63bd2c3e36d5a74b4c061cea1b847)
 2016-04-04 18:08:18 +05:30
Jian He
3afc2caec8 YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage 2016-03-31 14:31:38 -07:00
Jason Lowe
edf17fe8e5 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 
(cherry picked from commit 948b75807068c304ffe789e32f2b850c0d653e0a)
 2016-03-28 23:02:15 +00:00
Jian He
bdc648ebe7 YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola 2016-03-27 20:22:49 -07:00
Eric Payne
dd1e4107e5 YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger. 
(cherry picked from commit 92b7e0d41302b6b110927f99de5c2b4a4a93c5fd)
 2016-03-18 17:05:53 +00:00
Robert Kanter
da24fde333 TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe125a4f7bf38cc63ab8106c40a7a7ba)
 2016-03-15 10:05:28 -07:00
Vinod Kumar Vavilapalli
da9f39b107 YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv) 
(cherry picked from commit b2661765a5a48392a5691cee15904ed2de147b00)
 2016-03-07 11:11:29 -08:00
Jason Lowe
4eace7ab43 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 
(cherry picked from commit 059caf99891943d9587cac19b48e82efbed06b2d)
 2016-03-07 15:45:47 +00:00
Varun Vasudev
78919f8c34 YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e80c0def3fcb7604e0bd27c1c24d291e)
 2016-03-07 16:19:27 +05:30
Varun Vasudev
e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Haohui Mai
c5db4ab0b4 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 14:10:18 -08:00
Jason Lowe
bd0f5085e3 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c58209a8f78ff64e04e9112933489fb5)
 2016-02-29 15:26:26 +00:00
Ming Ma
1656bcec5f YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 
(cherry picked from commit 7f3139e54da2c496327446a5eac43f8421fc8839)
 2016-02-26 08:43:14 -08:00
Robert Kanter
872b8d90a6 YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 
(cherry picked from commit d7fdec1e6b465395d2faca6e404e329d20f6c3d8)
 2016-02-25 16:40:05 -08:00
Robert Kanter
c2098d2470 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 
(cherry picked from commit 954dd57043d2de4f962876c1b89753bfc7e4ce55)
 2016-02-24 15:00:48 -08:00
Jason Lowe
acffe82353 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 
(cherry picked from commit d284e187b8db43056236032ebc2114ee462c27f6)
 2016-02-23 20:51:01 +00:00
Varun Vasudev
2c218ca8a8 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d74565aad848b40a251ec02b597a3a20d4)
 2016-02-23 12:32:16 +05:30
Varun Vasudev
3a5b39e931 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 
(cherry picked from commit fa00d3e20560bee412b49e5792595749a247a8ab)
 2016-02-11 12:09:29 +05:30
Jason Lowe
b12f16ab73 YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39eda1c479389b99a5c121e640a1e0ad)
 2016-02-03 17:22:12 +00:00
Varun Vasudev
bbda446889 YARN-4649. Add additional logging to some NM state store operations. Contributed by Sidharta Seethana 
(cherry picked from commit 1cd55e0c171f7c4dec6f843931285557d59cd5ea)
 2016-02-02 09:26:22 +05:30
Wangda Tan
9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c63f343809907d06bf48a298035a611)
 2016-02-02 10:19:19 +08:00
Rohith Sharma K S
b8d5df06f2 YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac686668031ee9837deed3f3566f09f33c437870)
 2016-01-29 12:30:53 +05:30
Jason Lowe
29195c7668 YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 
(cherry picked from commit 61382ff8fabc76b3a51f227646573cdf367fea1a)
 2016-01-28 19:00:51 +00:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
47b20d0c74 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 
(cherry picked from commit 2085e60a9655b59aa2ba8917acdc511ab71ff6e4)
 2016-01-25 16:22:47 -08:00
Jian He
2d236c9de2 YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:40 -08:00
Xuan
1d19557dda YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He

(cherry picked from commit 618bfd6ac2a5b62d39e9bed80f75362bafc0ef28)
 2016-01-22 18:21:50 -08:00
Jason Lowe
635849fa45 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0e41646ba02391aea31481ead28aaf8)
 2016-01-22 21:55:59 +00:00
Varun Vasudev
85561207f9 YARN-4578. Directories that are mounted in docker containers need to be more restrictive/container-specific. Contributed by Sidharta Seethana. 
(cherry picked from commit b41a7e89d1eb8650975ac7092532ed9563ac60f2)
 2016-01-22 14:44:22 +05:30
Wangda Tan
23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac4bccf42d82686e146b02eb60d14736)
 2016-01-19 21:28:51 +08:00
Karthik Kambatla
ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1ad977636457a6cc96b6a4f9b903afc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Colin Patrick Mccabe
48b8c99479 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810888480d058512a326a5083e1bb4eaa)
 2016-01-15 13:22:14 -08:00
Colin Patrick Mccabe
cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77cb2fab80c54f9dfeb5d8b7139eca34)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Varun Vasudev
6220a024b1 YARN-4553. Add cgroups support for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 3ddb92bd30d4a267c5c35b410b21c6ea42a7238b)
 2016-01-14 14:30:19 +05:30
Jason Lowe
d164cdcda1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1c6d9fc78cd5013c860c1086d86176f)
 2016-01-12 15:57:07 +00:00
Akira Ajisaka
3905435d3d YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c44538c26fb9d0144f0b280be3593a38)
 2016-01-08 01:13:30 +09:00
Steve Loughran
d9459ffa00 YARN-4550. Some tests in TestContainerLanch fails on non-english locale environment. (Takashi Ohnishi via stevel) 2016-01-07 14:30:07 +00:00
rohithsharmaks
4865642e7c YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae0b351e0bf0b2ecec854dc72ab07935)
 2016-01-07 09:39:37 +05:30
Rohith Sharma K S
ae25e283cf YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit. (tangshangwen via rohithsharmaks) 
(cherry picked from commit f9e36dea96f592d09f159e521379e426e7f07ec9)
 2016-01-04 10:04:27 +05:30
Gera Shegalov
8baf9369bb YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 
(cherry picked from commit 2c17b81569c05b5f375d0ddad99d43cb4e3b23b1)
 2015-12-24 23:52:43 -08:00
Uma Mahesh
fc9e130825 YARN-4480. Clean up some inappropriate imports. (Kai Zheng via umamahesh) 
(cherry picked from commit 0f82b5d878a76b1626c9e07b2fbb55ce2a79232a)
 2015-12-19 23:17:33 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
d9b9a75928 YARN-1856. Added cgroups based memory monitoring for containers as another alternative to custom memory-monitoring. Contributed by Varun Vasudev. 
(cherry picked from commit 4e7d32c0db69882cde854ef581056142a997c005)
 2015-12-17 12:13:56 -08:00
Wangda Tan
9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83e981ae74cb8b58ffcf7907b7612ad4)
 2015-12-16 13:28:56 -08:00
Jian He
72544db3cd YARN-4402. TestNodeManagerShutdown And TestNodeManagerResync fails with bind exception. Contributed by Brahma Reddy Battula 2015-12-14 14:59:18 -08:00
Wangda Tan
9f979107c2 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb0963c89c0455d41223427165b9f9e537)
 2015-12-14 11:13:51 -08:00
rohithsharmaks
f626d18cc3 YARN-4431. Not necessary to do unRegisterNM() if NM get stop due to failed to connect to RM. (Junpin Du via rohithsharmaks) 
(cherry picked from commit 15c3e7ffe3d1c57ad36afd993f09fc47889c93bd)
 2015-12-09 10:51:57 +05:30
Steve Loughran
af14458de7 HADOOP-12321. Make JvmPauseMonitor an AbstractService. (Sunil G via Stevel) [includes HDFS-8947 MAPREDUCE-6462 and YARN-4072] 2015-12-06 17:42:56 +00:00
Jian He
23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad735f4843ce2c38355de9f434838d4507)
 2015-12-03 12:49:22 -08:00