Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,581 Commits 326 Branches 371 Tags
Commit Graph

2199 Commits

Author SHA1 Message Date
Xuan
85363ea4bd YARN-221. Addendum patch to compilation issue which is caused by missing 
AllContainerLogAggregationPolicy. Contributed by Xuan Gong

(cherry picked from commit b71c6006f579ac6f0755975a9b908b0062618b46)
 2015-08-23 16:50:02 -07:00
Xuan
205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a96d781e1c9982988b7de4aa5242d0c)
 2015-08-22 16:26:39 -07:00
Rohith Sharma K S
27ced8253b YARN-3986. getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface 
(cherry picked from commit 22de7c1dca1be63d523de833163ae51bfe638a79)
 2015-08-21 10:56:45 +05:30
Vinod Kumar Vavilapalli
9da81a3955 MAPREDUCE-6454. Fixed MapReduce to modify HADOOP_CLASSPATH to have distributed cache files so that child processes running hadoop scripts can access these files. Contributed by Junping Du. 2015-08-20 17:35:24 -07:00
Wangda Tan
6452b31760 YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 
(cherry picked from commit fc07464d1a48b0413da5e921614430e41263fdb7)
 2015-08-20 16:52:57 -07:00
Karthik Kambatla
b567aa2b4f YARN-4055. Report node resource utilization in heartbeat. (Inigo Goiri via kasha) 
(cherry picked from commit 13604bd5f119fc81b9942190dfa366afad61bc92)
 2015-08-20 16:52:33 -07:00
Xuan
fc6b176e70 YARN-4028. AppBlock page key update and diagnostics value null on 
recovery. Contributed by Bibin A Chundatt

(cherry picked from commit 22dc5fc20942a8d98333ed5470b5fb8c1d257f1a)
 2015-08-18 22:54:01 -07:00
Zhihai Xu
051a622b57 YARN-4057. If ContainersMonitor is not enabled, only print related log info one time. Contributed by Jun Gong. 
(cherry picked from commit 14215c8ef83d58b8443c52a3cb93e6d44fc87065)
 2015-08-18 11:47:41 -07:00
Zhihai Xu
8f2eb99d71 YARN-3857: Memory leak in ResourceManager with SIMPLE mode. Contributed by mujunchao. 
(cherry picked from commit 3a76a010b85176f2bcb85ed6f74c25dcb8acfe4d)
 2015-08-18 10:53:16 -07:00
Karthik Kambatla
d2ff763533 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 
(cherry picked from commit def12933b38efd5e47c5144b729c1a1496f09229)
 2015-08-16 06:26:21 -07:00
Jian He
b2e869f2a8 YARN-3987. Send AM container completed msg to NM once AM finishes. Contributed by sandflee 
(cherry picked from commit 0a030546e24c55662a603bb63c9029ad0ccf43fc)
 2015-08-13 16:23:30 -07:00
Jian He
703fa1b141 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfabcf9c6aae219051f65d3f6cb8feb87c)
 2015-08-13 16:13:28 -07:00
Jian He
4a5fdb7c23 YARN-4005. Completed container whose app is finished is possibly not removed from NMStateStore. Contributed by Jun Gong 
(cherry picked from commit 38aed1a94ed7b6da62e2445b5610bc02b1cddeeb)
 2015-08-13 14:46:34 -07:00
Robert Kanter
fbcfca6f91 YARN-4031. Add JvmPauseMonitor to ApplicationHistoryServer and WebAppProxyServer (djp via rkanter) 
(cherry picked from commit dc2340c60e33f903f8fd34958ec746c989016191)
 2015-08-12 15:48:14 -07:00
Jian He
65d22b3686 YARN-4026. Refactored ContainerAllocator to accept a list of priorites rather than a single priority. Contributed by Wangda Tan 
(cherry picked from commit e5003be907acef87c2770e3f2914953f62017b0e)
 2015-08-12 15:09:46 -07:00
rohithsharmaks
c5b20e0db6 YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 
(cherry picked from commit 1c12adb71f32f24ec525313ff8dfcd2b738fc8e2)
 2015-08-12 14:47:23 +05:30
Xuan
2ebdf5bfce YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa696b87e849dae40225dc59fb5ed114cb)
 2015-08-11 18:26:00 -07:00
Jian He
df97e3265b YARN-3887. Support changing Application priority during runtime. Contributed by Sunil G 
(cherry picked from commit fa1d84ae2739a1e76f58b9c96d1378f9453cc0d2)
 2015-08-10 20:52:18 -07:00
Wangda Tan
380d401da5 YARN-3873. PendingApplications in LeafQueue should also use OrderingPolicy. (Sunil G via wangda) 
(cherry picked from commit cf9d3c925608e8bc650d43975382ed3014081057)
 2015-08-10 14:58:19 -07:00
Wangda Tan
cdb61b5fb2 YARN-3966. Fix excessive loggings in CapacityScheduler. (Jian He via wangda) 
(cherry picked from commit 4bc42d76e7fa53cb268cab0f9fe1fd8d8dbb17fd)
 2015-08-07 09:53:42 -07:00
Rohith Sharma K S
a0da1ec010 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks) 
(cherry picked from commit b6265d39c5d0fda21dfe55273f193d8a6a20c6a8)
 2015-08-07 10:47:08 +05:30
Carlo Curino
8e23a7997f YARN-3974. Refactor the reservation system test cases to use parameterized base test. (subru via curino) 
(cherry picked from commit 8572a5a14b999a866fa64ce32ee20078ffefdb1e)
 2015-08-02 02:06:06 -07:00
Junping Du
7af531d2e3 YARN-4019. Add JvmPauseMonitor to ResourceManager and NodeManager. Contributed by Robert Kanter. 
(cherry picked from commit cfee02b3bdd1117370200c9d8ce216676cff8888)
 2015-08-06 06:50:40 -07:00
Arun Suresh
84ea7cd408 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh) 
(cherry picked from commit 154c9d2e421390c808dea3b2826cdaae492ea337)
 2015-08-05 23:15:33 -07:00
rohithsharmaks
dc76c4b035 YARN-3992. TestApplicationPriority.testApplicationPriorityAllocation fails intermittently. (Contributed by Sunil G) 
(cherry picked from commit df9e7280db58baddd02d6e23d3685efb8d5f1b97)
 2015-08-06 10:49:37 +05:30
Jian He
1466772827 YARN-3983. Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic. Contributed by Wangda Tan 
(cherry picked from commit ba2313d6145a1234777938a747187373f4cd58d9)
 2015-08-05 13:51:08 -07:00
Arun Suresh
707b96fa58 YARN-3736. Add RMStateStore apis to store and load accepted reservations for failover (adhoot via asuresh) 
(cherry picked from commit f271d377357ad680924d19f07e6c8315e7c89bae)
 2015-08-05 12:58:37 -07:00
Xuan
89ccfd9fcc YARN-4004. container-executor should print output of docker logs if the 
docker container exits with non-0 exit status. Contributed by Varun
Vasudev

(cherry picked from commit c3364ca8e75acfb911ab92e19f357b132f128123)
 2015-08-03 18:11:16 -07:00
Xuan
8126fde884 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S

(cherry picked from commit 0306d902f53582320aa5895ca9f5c31f64aaaff6)
 2015-08-03 15:47:21 -07:00
Jason Lowe
b264217bc3 YARN-3965. Add startup timestamp to nodemanager UI. Contributed by Hong Zhiguo 
(cherry picked from commit 469cfcd695da979e56c83d9303f9bc1f898c08ce)
 2015-08-03 15:55:04 +00:00
Jonathan Eagles
899df5bce0 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b95224e9d43fd63a4ef9ac5c44f113f710d)
 2015-08-03 10:40:04 -05:00
Jason Lowe
c31e3ba921 YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c035487e99df30ce80366446fe09bd6c)
 2015-07-31 17:38:49 +00:00
Zhihai Xu
0e2019fa30 YARN-433. When RM is catching up with node updates then it should not expire acquired containers. Contributed by Xuan Gong 
(cherry picked from commit ab80e277039a586f6d6259b2511ac413e29ea4f8)
 2015-07-30 22:03:04 -07:00
Wangda Tan
53a0653826 YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (Bibin A Chundatt via wangda) 
(cherry picked from commit 91b42e7d6e8169feb54dbe3715a3269eea6153a9)
 2015-07-30 10:01:01 -07:00
Wangda Tan
a28e23d049 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 
(cherry picked from commit 8acb30b0167e498991aef6993df7e36020428844)
 2015-07-30 09:43:45 -07:00
rohithsharmaks
9c00ae0009 YARN-3919. NPEs' while stopping service after exception during CommonNodeLabelsManager#start. (varun saxena via rohithsharmaks) 2015-07-29 23:44:15 +05:30
Karthik Kambatla
8659315062 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 
(cherry picked from commit 5205a330b387d2e133ee790b9fe7d5af3cd8bccc)
 2015-07-29 09:42:57 -07:00
Jason Lowe
6f50a6a456 YARN-3950. Add unique SHELL_ID environment variable to DistributedShell. Contributed by Robert Kanter 
(cherry picked from commit 2b2bd9214604bc2e14e41e08d30bf86f512151bd)
 2015-07-29 15:18:45 +00:00
Xuan
2ec008eb38 YARN-3982. container-executor parsing of container-executor.cfg broken 
in trunk and branch-2. Contributed by Varun Vasudev

(cherry picked from commit f17093421521efcbdc813f6f2b8411e45ecc7863)
 2015-07-27 23:47:31 -07:00
Jian He
7c123accda YARN-3846. RM Web UI queue filter is not working for sub queue. Contributed by Mohammad Shahid Khan 
(cherry picked from commit 3572ebd738aa5fa8b0906d75fb12cc6cbb991573)
 2015-07-27 17:12:24 -07:00
Varun Vasudev
9da487e0fd YARN-3853. Add docker container runtime support to LinuxContainterExecutor. Contributed by Sidharta Seethana. 
(cherry picked from commit 3e6fce91a471b4a5099de109582e7c6417e8a822)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2015-07-27 12:03:36 -07:00
Varun Vasudev
ec0f801f52 YARN-3852. Add docker container support to container-executor. Contributed by Abin Shahab. 
(cherry picked from commit f36835ff9b878fa20fe58a30f9d1e8c47702d6d2)
 2015-07-27 10:18:06 -07:00
Akira Ajisaka
0e7c88347e YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 
(cherry picked from commit 42d4e0ae99d162fde52902cb86e29f2c82a084c8)
 2015-07-27 11:45:41 +09:00
ccurino
26ea045814 YARN-3656. LowCost: A Cost-Based Placement Agent for YARN Reservations. (Jonathan Yaniv and Ishai Menache via curino) 
(cherry picked from commit 156f24ead00436faad5d4aeef327a546392cd265)
 2015-07-25 07:47:11 -07:00
Wangda Tan
4ba2e1ae95 YARN-3973. Recent changes to application priority management break reservation system from YARN-1051 (Carlo Curino via wangda) 
(cherry picked from commit a3bd7b4a59b3664273dc424f240356838213d4e7)
 2015-07-24 16:45:06 -07:00
Jason Lowe
2cf714123a YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a739bb13115167dc661b6a16b2ed2c04)
 2015-07-24 22:15:44 +00:00
Jian He
3233284e87 YARN-3026. Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp. Contributed by Wangda Tan 
(cherry picked from commit 83fe34ac0896cee0918bbfad7bd51231e4aec39b)
 2015-07-24 14:01:36 -07:00
Karthik Kambatla
d2ee02940a YARN-3957. FairScheduler NPE In FairSchedulerQueueInfo causing scheduler page to return 500. (Anubhav Dhoot via kasha) 
(cherry picked from commit d19d18775368f5aaa254881165acc1299837072b)
 2015-07-24 11:49:46 -07:00
Xuan
98ed4bca3b YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269221ec25834684477f434e19f0b66af)
 2015-07-24 10:17:05 -07:00
carlo curino
6a71a0a230 YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2add3f112907ff4e833e2f04947b53e)
 2015-07-23 19:36:54 -07:00