Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,379 Commits 326 Branches 371 Tags
Commit Graph

2811 Commits

Author SHA1 Message Date
Eric Payne
a171cb3d81 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 2017-09-13 15:19:41 -05:00
Jason Lowe
2071f8338c YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb4909bf9968dd30067617f74543e07ffb)
 2017-09-06 15:26:13 -05:00
Oleg Danilov
867a4b7d78
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
(cherry picked from commit b616462eef5ee06a0d7f7d5f78b34a8069b80507)
 2017-09-01 21:36:45 +02:00
Jason Lowe
798ff2f52b YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451fb91ef956c926f891283f70f0ada1c)
 2017-09-01 11:11:28 -05:00
Junping Du
d665d85682 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 2017-08-31 18:10:33 -07:00
Junping Du
8724e36562 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:21:56 -07:00
Junping Du
79294b5f32 YARN-7083. Log aggregation deletes/renames while file is open. Contributed by Jason Lowe. 2017-08-29 13:18:49 -07:00
Junping Du
1dfb3850c8 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
(cherry picked from commit f252e1fe2887cc961cf3ad532bfa22d646775fbf)
 2017-08-28 18:31:36 -07:00
Wangda Tan
f40f4ce0e9 YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I26d719ad164062b688b49142dc7db40d888b882b
 2017-08-28 15:05:30 -07:00
Sunil G
bc7307788c YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:43:47 +05:30
Eric Payne
8eb39f775c YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
 2017-08-25 15:58:28 -05:00
Jason Lowe
dfdda969ad YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:33:00 -05:00
Jason Lowe
2d548759ad YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java
 2017-08-25 10:16:58 -05:00
Jason Lowe
ff9cbcbbf4 YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-22 16:32:45 -05:00
Jason Lowe
5cb8a92bdb YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
 2017-08-22 13:28:44 -05:00
Jason Lowe
0b64773678 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
 2017-08-22 13:02:52 -05:00
Jian He
3f735ad640 YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-18 13:08:54 -07:00
Jason Lowe
2810e6ab9c YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be0a341df3e628dcaf06717b4630b05e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java
 2017-08-16 13:14:25 -05:00
Akira Ajisaka
2b92c1be04
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
(cherry picked from commit a28a3dc8f2709738c28f70fc394f51d3b1266296)
 2017-08-16 14:07:23 +09:00
Jason Lowe
87c36d8b8b YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:41:54 -05:00
Jason Lowe
00e51a9cd3 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:48:48 -05:00
Naganarasimha
49d459a3ed YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
(cherry picked from commit a7cdd39234c3a236cd9b85853636d42dc8c850a4)
 2017-08-08 15:51:31 -07:00
Jian He
bc87a293d9 YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 2017-08-08 11:11:46 -07:00
Eric Payne
8943cf9d30 YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 13:07:39 -05:00
Eric Payne
d5150e3148 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
 2017-08-02 12:03:53 -05:00
Jian He
e84a3f43a1 YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 2017-08-02 00:09:25 -07:00
Wangda Tan
f2f09e2bb0 YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. (wangda) 
Change-Id: Ie8c5145f449582253dcf8c0f6f388e3660ab1c6b
 2017-08-01 15:24:19 -07:00
Jason Lowe
0020a8e7d9 YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles 
(cherry picked from commit b9640c02bf5072014299124cfd44880d7361a5a7)
 2017-07-31 16:39:05 -05:00
Akira Ajisaka
d57784cbda
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
(cherry picked from commit 98c3544e9407e3e1dfae9806d4a8181508952bed)
 2017-07-31 12:34:25 +09:00
Sunil G
e5ada740ff YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 2017-07-28 20:08:20 +05:30
Kihwal Lee
c586b7ab7d HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb52c36b74358f22db498424e82ec4a7)
(cherry picked from commit 2d6be7ea2311deb13cbad738b4c8fc68ad25ab2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestRMAdminCLI.java
 2017-07-26 14:15:39 -05:00
Jason Lowe
db0bc8fafa YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:25:04 -05:00
Jason Lowe
4fc070ac59 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c58209a8f78ff64e04e9112933489fb5)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:46:18 -05:00
Jason Lowe
293ac4c18b YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39eda1c479389b99a5c121e640a1e0ad)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:33:34 -05:00
Eric Payne
95be843851 YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
 2017-07-21 10:32:35 -05:00
Jason Lowe
6e45c543cb YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/impl/pb/TestApplicationClientProtocolRecords.java
 2017-07-20 11:11:41 -05:00
Wangda Tan
a39617df63 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) 
Change-Id: I5951f0997547de7d2e4a30b4ad87ab0a59b3066a
 2017-07-19 15:29:45 -07:00
Rohith Sharma K S
4daf5741e0 YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 
(cherry picked from commit e15c20edba1e9a23475ee6a4dfbadbdb8c1f668a)
 2017-07-17 14:52:06 -05:00
Jason Lowe
1edc16ee54 YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt 
(cherry picked from commit a5ae5ac50e97cf829c41dcf01655cd9bd4d36a00)
 2017-07-14 15:06:31 -05:00
Jason Lowe
8c69b040ac YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit a77fb561efcea03060674d37ebadc15069766b6f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2017-07-13 18:00:05 -05:00
Jason Lowe
9cff53fed9 YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles 
(cherry picked from commit 5f1ee72b0ebf0330417b7c0115083bc851923be4)
 2017-07-13 17:30:57 -05:00
Jason Lowe
4f1dfcc118 YARN-6797. TimelineWriter does not fully consume the POST response. Contributed by Jason Lowe 
(cherry picked from commit 655110393bf90a7dac5d85f831ea225071f5f3c2)
 2017-07-12 15:54:49 -05:00
Akira Ajisaka
ebb66d2038
YARN-6809. Fix typo in ResourceManagerHA.md. Contributed by Yeliang Cang. 
(cherry picked from commit b628d0da5196faa5abf4c95dc7e1bb4eb67f6707)
(cherry picked from commit 3419381e988b489d4fde677f2ab191a955724371)
 2017-07-12 19:39:29 +09:00
Naganarasimha
0f9f7bb0c6 YARN-6770. A small mistake in the example of TimelineClient. Contributed by Jinjiang Ling. 
(cherry picked from commit ba5b056e8a151fad9d7573c6e62cffc40cde7da3)
 2017-07-09 21:40:16 +05:30
Naganarasimha
ad70e929b2 YARN-6428. Queue AM limit is not honored in CS always. Contributed by Bibin A Chundutt. 2017-07-09 17:34:47 +05:30
Eric Payne
8949a9d541 YARN-2113. Add cross-user preemption within CapacityScheduler's leaf-queue. (Contributed by Sunil G) 2017-07-06 10:53:47 -05:00
Jason Lowe
cd895d0586 YARN-6708. Nodemanager container crash after ext3 folder limit. Contributed by Bibin A Chundatt 
(cherry picked from commit 7576a688ea84aed7206321b1f03594e43a5f216e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ContainerLocalizer.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestContainerLocalizer.java
 2017-07-06 09:53:13 -05:00
Naganarasimha
69480ba686 YARN-6681. Eliminate double-copy of child queues in canAssignToThisQueue. Contributed by Daryn Sharp. 2017-07-01 12:58:36 +05:30
bibinchundatt
f6fcbf426f YARN-6749. TestAppSchedulingInfo.testPriorityAccounting fails consistently. Contributed by Naganarasimha G R 2017-06-30 21:52:16 +05:30
Naganarasimha
29c68ccc6f YARN-6467. CSQueueMetrics needs to update the current metrics for default partition only. Contributed by Manikandan R. 2017-06-29 00:25:32 +05:30