Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,186 Commits 326 Branches 371 Tags
Commit Graph

534 Commits

Author SHA1 Message Date
Eric Payne
71d91d53bf YARN-6618. TestNMLeveldbStateStoreService#testCompactionCycle can fail if compaction occurs more than once. Contributed by Jason Lowe. 
(cherry picked from commit 19482e0d0990feb719678d71971aa849095982fd)
 2017-05-19 11:36:18 -05:00
Varun Vasudev
23bbbce63e Sanitize arguments before launching Docker containers. 2017-05-18 12:03:52 +05:30
Varun Vasudev
51e65cc710 Validate docker image name before launching container. 2017-05-18 11:53:16 +05:30
Chris Douglas
9791ecc9d8 YARN-6577. Remove unused ContainerLocalization classes. Contributed by ZhangBing Lin 
(cherry picked from commit b23fcc86c670aa896151a2bd8878154d7bb45d13)
(cherry picked from commit 3d2afb209c9a296c05da4ddf62bfa69cd13b700d)
 2017-05-17 20:42:27 -07:00
Jason Lowe
6e41aec109 YARN-6288. Exceptions during aggregated log writes are mishandled. Contributed by Akira Ajisaka 2017-04-06 16:26:24 -05:00
Jason Lowe
ab231f5e1c YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:36:26 -05:00
Jason Lowe
6c8df1208a YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 
(cherry picked from commit 4a8e3045027036afebbcb80f23b7a2886e56c255)
 2017-03-17 14:08:14 -05:00
Jason Lowe
5d86ce7b0d YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 
(cherry picked from commit a16ba4296e163d5cb4caed129f2f1612a69a8d84)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/Container.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/MockContainer.java
 2017-03-16 09:36:34 -05:00
Chris Nauroth
cf6c143c0f YARN-5456. container-executor support for FreeBSD, NetBSD, and others if conf path is absolute. Contributed by Allen Wittenauer. 
(cherry picked from commit b913677365ad77ca7daa5741c04c14df1a0313cd)
(cherry picked from commit c5203e170ddd68fc8532792878c37d4762068472)
 2017-01-19 20:51:38 +05:30
Chris Nauroth
3668da2c18 YARN-5121. fix some container-executor portability issues. Contributed by Allen Wittenauer. 
(cherry picked from commit ef501b1a0b4c34a2cc43eb082d1c2364684cd7f1)
(cherry picked from commit 384803d09ac45886e74a0501f4b419a2b756c20c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-01-19 20:51:38 +05:30
Varun Saxena
fedcce2f4c YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce71d3019ab91b136b6d7646316954e7)
 2017-01-09 11:27:48 +05:30
Junping Du
689d19575f YARN-4990. Re-direction of a particular log file within in a container in NM UI does not redirect properly. Contributed by Xuan Gong. 
(cherry picked from commit 736f54b727c3f0ecc8fb9a594f2281c240c89cb8)
 2017-01-05 20:37:56 +00:00
Jason Lowe
2adce1b361 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit d0162f2040a4d98fbac51527dfc9447888cb32ab)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/util/WebAppUtils.java
 2017-01-05 20:28:26 +00:00
Jian He
ca66bedeff YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 
(cherry picked from commit 264c06a4381695002d218f688c85f448d6362ec8)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceTrackerService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNM.java
 2017-01-04 15:09:14 -08:00
Jason Lowe
74503ed6b4 YARN-5859. TestResourceLocalizationService#testParallelDownloadAttemptsForPublicResource sometimes fails. Contributed by Eric Badger 
(cherry picked from commit 009452bb6dbe5dffb0b304d67a2f360fe0eee1e2)
 2016-11-21 16:40:00 +00:00
Jason Lowe
cb0fccad19 YARN-5836. Malicious AM can kill containers of other apps running in any node its containers are running. Contributed by Botong Huang 
(cherry picked from commit 59bfcbf3579e45ddf96db3aafccf669c8e03648f)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/BaseContainerManagerTest.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/TestContainerManager.java
 2016-11-16 22:43:56 +00:00
Naganarasimha
5c13bb5008 YARN-4355. NPE while processing localizer heartbeat. Contributed by Varun Saxena & Jonathan Hung. 
(cherry picked from commit 7ffb9943b8838a3bb56684e0722db40d800743a2)
 2016-11-15 15:52:05 +05:30
Naganarasimha
59a3d9a576 Reverting it because of YARN-5287, blocker issue has been reported. 
Revert "YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang."

This reverts commit 16c8fd9dca75e8d4f36e63ef0940aed83f8ebecf.
 2016-11-13 17:43:01 +05:30
Jason Lowe
4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a41003d57d94eb1a5f486010815266046af)
 2016-11-01 20:24:44 +00:00
Jason Lowe
9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dca17bcd2e031864bc6ca07254c61b47)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe
83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd436a76585fb1f3ea93716795308cec)
 2016-10-28 15:50:22 +00:00
Jason Lowe
558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:57 +00:00
Jason Lowe
d21dc8aa6c YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d316b33d6a09bfd4e65e7e5384943bb1d)
 2016-10-27 20:45:18 +00:00
Varun Vasudev
d1841b4750 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-09 13:22:01 +05:30
Varun Vasudev
50409404bf YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e80c0def3fcb7604e0bd27c1c24d291e)
(cherry picked from commit 78919f8c341ec645cf9134991e3ae89a929b9184)
 2016-10-09 11:53:03 +05:30
Jason Lowe
515727f46f YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac686668031ee9837deed3f3566f09f33c437870)
 2016-10-03 16:06:17 +00:00
Jason Lowe
1ac8206a85 YARN-5630. NM fails to start after downgrade from 2.8 to 2.7. Contributed by Jason Lowe 
(cherry picked from commit e7933097354a246b080b46f1a4ca2ef0f39f3b38)
 2016-09-13 14:43:55 +00:00
Arun Suresh
979b29a03c YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b7b6141ed88359652cf22e8973c0661)
(cherry picked from commit b279f42d79175bef6529dc1ac4216198a3aaee4d)
 2016-08-31 20:06:49 -07:00
Junping Du
1a38dd9cee YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 00058167431475c6e63c80207424f1d365569e3a)
 2016-08-15 19:18:36 +00:00
Naganarasimha
16c8fd9dca YARN-5287. LinuxContainerExecutor fails to set proper permission. Contributed by Ying Zhang. 2016-08-09 17:34:48 +05:30
Jason Lowe
5883718eea YARN-4717. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe125a4f7bf38cc63ab8106c40a7a7ba)
 2016-08-03 19:33:58 +00:00
Jason Lowe
c972498b74 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 
(cherry picked from commit db646540f094077941b56ed681a4f3e5853f5b7f)
 2016-08-03 19:19:47 +00:00
Jason Lowe
9d08ca1ed7 YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae0b351e0bf0b2ecec854dc72ab07935)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-07-12 19:25:07 +00:00
Jian He
23eb3c7ceb YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:38:35 -07:00
Vinod Kumar Vavilapalli
4ea87cb38c YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 
(cherry picked from commit ce9c006430d13a28bc1ca57c5c70cc1b7cba1692)
 2016-07-05 17:12:37 -07:00
Junping Du
810470508b YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-16 07:18:36 -07:00
Wangda Tan
d838c6443d YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:21:41 -07:00
Junping Du
2be48e7d15 YARN-5199. Close LogReader in NMWebServices#getLogs. Contributed by Xuan Gong. 2016-06-09 12:29:25 -07:00
Xuan
11b4d1e486 Revert "YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong." 
This reverts commit 22ac37615a933f9cee8cf19ad0182586a037b690.
 2016-06-08 11:23:12 -07:00
Wangda Tan
19e578870d YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. (wangda) 2016-06-07 12:41:50 -07:00
Ming Ma
ec4f9a14f9 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 
(cherry picked from commit 4a1cedc010d3fa1d8ef3f2773ca12acadfee5ba5)
(cherry picked from commit 74e2b5efa26f27027fed212b4b2108f0e95587fb)
 2016-06-06 14:49:43 -07:00
Jian He
3c2bd19fa5 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 
(cherry picked from commit 99cc439e29794f8e61bebe03b2a7ca4b6743ec92)
 2016-06-03 11:11:49 -07:00
Wangda Tan
2f3e1d965d Revert "YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan." 
This reverts commit 457884737f75c796413ce860b1859a31cc5292ca.
 2016-05-31 22:16:53 -07:00
Varun Vasudev
457884737f YARn-4844. Add getMemoryLong/getVirtualCoreLong to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 20:57:56 +05:30
Carlo Curino
7982933c09 YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 
(cherry picked from commit 013532a95e63d7c53e601be530021d6d5a15ab7f)
(cherry picked from commit c656977961e2ba0f9dfd349ed59bf1d0d41c57f5)
 2016-05-25 17:02:22 -07:00
Jason Lowe
fe10caee8d YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 
(cherry picked from commit 1ba31fe9e906dbd093afd4b254216601967a4a7b)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2016-05-25 21:37:31 +00:00
Varun Vasudev
35456bb7c9 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit f440a9d8c4a177bc5062d21d4b4bc4d9b2944344)
(cherry picked from commit 36f2ae0692d73a865a5c0c520d1346b6d4498c25)
 2016-05-25 19:03:33 +05:30
Jason Lowe
8e404b4321 YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 
(cherry picked from commit d1df0266cf4e9ff0ec70813c156556ca4e74f791)
 2016-05-23 15:17:26 +00:00
Rohith Sharma K S
726c1f14b8 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 2016-05-19 10:50:32 +05:30
Jason Lowe
70faa87ccf YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcde2b31423438d6f1b8b8204d4ca05b3)
 2016-05-16 15:43:42 +00:00