Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,347 Commits 326 Branches 371 Tags
Commit Graph

12347 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Oleg Danilov
2f439e5fa7
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
(cherry picked from commit b616462eef5ee06a0d7f7d5f78b34a8069b80507)
(cherry picked from commit 867a4b7d78f81dde6f18e10e910f28ad96be037f)
 2017-09-01 21:37:41 +02:00
Junping Du
7e15cd8386 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 
(cherry picked from commit d665d856825a6aaf48632a1826e1ca10608f9c10)
 2017-08-31 18:11:04 -07:00
Junping Du
ab72dbb809 Set branch version to 2.8.2 before preparing RC. 2017-08-31 18:08:16 -07:00
Jitendra Pandey
a80cd4b897 HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R. 
(cherry picked from commit b82485d6fed2194bf0dc2eedfab7e226e30a7cf0)
(cherry picked from commit c0e294ffc82d611a64b352e3962bce9399ddbe5f)
 2017-08-31 16:23:37 -07:00
Haohui Mai
8e0e710754 HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 
(cherry picked from commit 6801b898d53cf603dea134fc8b492e16f65049fd)
 2017-08-31 16:01:14 -07:00
Haibo Chen
c53d351f34 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 
(cherry picked from commit 2fce0fa8d7c777a5374ee79e1a8e968b7319522a)
 2017-08-31 16:00:41 -07:00
Wangda Tan
4d5f2c402b YARN-5731. Preemption calculation is not accurate when reserved containers are present in queue. (wangda) 
Change-Id: Ie8c5145f449582253dcf8c0f6f388e3660ab1c6b
(cherry picked from commit f2f09e2bb09d3ffbce7a0af32fc225d47dcab5d1)
 2017-08-31 16:00:21 -07:00
Sunil G
fa632929f7 YARN-5892. Support user-specific minimum user limit percentage in Capacity Scheduler. Contributed by Eric Payne. 
(cherry picked from commit e5ada740ff5f9a45c1d6b004f40d3a5872840997)
(cherry picked from commit fc692616e965199992c6c320804bc05900fcee9b)
 2017-08-31 15:56:20 -07:00
John Zhuge
225c8cee68 HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 
(cherry picked from commit 7e6463d2fb5f9383d88baec290461868cf476e4c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java

(cherry picked from commit 832b0b82e269da0529a0584788ef6d6c803fd156)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-31 15:39:36 -07:00
Sunil G
8dba358a29 YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
(cherry picked from commit bc7307788c6a73873a7e058b42ce4b8820b26bd1)
 2017-08-31 14:29:27 -07:00
Eric Payne
615c661c30 YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
(cherry picked from commit 8eb39f775c09b255986a71eb375c13c0bbc74f6b)
 2017-08-31 14:29:09 -07:00
Jian He
e3e1bff301 YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du 
(cherry picked from commit bc87a293d91b6ca4f734300f8d91203393f2027a)
 2017-08-31 14:28:51 -07:00
Junping Du
c93e81f8eb HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9cfb0bcfb69cc0f1594120e7656f031)
(cherry picked from commit 7dd5bb949206fb635d875cd651292b451267a951)
(cherry picked from commit f9ebcebe74201b1c3997269927dfade50d89a201)
 2017-08-30 13:35:48 -07:00
Junping Du
12e35f4ef1 YARN-7083. Log aggregation deletes/renames while file is open. Contributed by Jason Lowe. 
(cherry picked from commit 79294b5f329317c30775f8b4dc61440f4b6c24cd)
 2017-08-29 13:20:05 -07:00
Jason Lowe
a766b72bae MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6042a837edb17bc1ae98a651d969a43)
 2017-08-29 13:01:37 -05:00
Wangda Tan
ef5f92a20b YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I26d719ad164062b688b49142dc7db40d888b882b
(cherry picked from commit f40f4ce0e93d6b45ab0c3910548b1a2e68857920)
 2017-08-29 09:01:49 -05:00
Junping Du
29588814b2 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
(cherry picked from commit f252e1fe2887cc961cf3ad532bfa22d646775fbf)
(cherry picked from commit 1dfb3850c880fefa2e7640c236939288d0e526f6)
 2017-08-28 18:32:29 -07:00
Brahma Reddy Battula
205a95c29b HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)(cherry picked from commit 9a168ae884f0075dea4930e07e1ced30a0d0fea6)
 2017-08-28 22:41:08 +08:00
Brahma Reddy Battula
f4208415ee HDFS-12364. [branch-2.8.2] Fix the Compile Error after HDFS-12299(addendum to HDFS-12299). Contributed by Jiandan Yang. 2017-08-28 22:27:40 +08:00
Jason Lowe
bc91e2310f YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:35:04 -05:00
Kihwal Lee
85e5fd94b3 HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396089972e23c533eca33c9cba231c45a)
 2017-08-25 15:05:20 -05:00
Arpit Agarwal
2a25ddc0b1 HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:03:42 -07:00
Arpit Agarwal
4b85d38f60 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 12:03:39 -07:00
Colin Patrick Mccabe
86db22f890 HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe) 
(cherry picked from commit f67149ab08bb49381def6c535ab4c4610e0a4221)
 2017-08-25 12:03:37 -07:00
Jason Lowe
310fa63cc1 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java

(cherry picked from commit 2d548759ad416b0647855eac5f5813467d4cbfd8)
 2017-08-25 10:17:20 -05:00
Jason Lowe
32d2417d3b HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 
(cherry picked from commit f6892f5821aedb33fdd6cd74d631fec2e67647be)
 2017-08-22 16:57:32 -05:00
Jason Lowe
0acf92210a YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 
(cherry picked from commit ff9cbcbbf47927db82c81660b413edd7c84c5022)
 2017-08-22 16:34:32 -05:00
Jason Lowe
b14228a9d1 YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
 2017-08-22 13:29:31 -05:00
Jason Lowe
3c295955a9 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
 2017-08-22 13:03:43 -05:00
Jason Lowe
864a0964f0 YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be0a341df3e628dcaf06717b4630b05e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java
 2017-08-16 13:17:02 -05:00
Ravi Prakash
155646c48c HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
(cherry picked from commit 895a35eac0422c1845ebe3c5bed1ac2bda1ac807)
(cherry picked from commit a611922de4e2fc82506872d68c4efcb963fb15e9)
 2017-08-15 15:47:08 -07:00
Jason Lowe
77e9d9ff48 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:45:47 -05:00
Jason Lowe
de271ac987 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:50:55 -05:00
Jian He
d1996152bd YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 
(cherry picked from commit e84a3f43a1b114d85a3c60151c821b1ca753f81a)
 2017-08-09 21:03:41 -07:00
Kihwal Lee
984a08895e HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:57:08 -05:00
Kihwal Lee
5758dad28f HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 
(cherry picked from commit 74bcc8d1f213ffd18056b7d0590d0cd346abaff9)
 2017-08-09 09:38:42 -05:00
Naganarasimha
68380262ff YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
(cherry picked from commit a7cdd39234c3a236cd9b85853636d42dc8c850a4)
(cherry picked from commit 49d459a3ed82169c02fca98dd7b54e3f09768881)
 2017-08-08 16:06:24 -07:00
Jason Lowe
a17faae509 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit ea67e1f266406477fec6eb4ff1ec1e1f47974166)
 2017-08-08 15:09:13 -05:00
Eric Payne
211c77a40b YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 13:20:47 -05:00
Eric Payne
82981738d5 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
(cherry picked from commit d5150e3148273a117e999814261a07b95def63c0)
 2017-08-02 12:09:32 -05:00
Junping Du
f4e42a728b HADOOP-14683. FileStatus.compareTo binary compatible issue. Contributed by Akira Ajisaka. 
(cherry picked from commit cf6794ebe93906c406a54acc5c8854dc600a9a7f)
(cherry picked from commit 60c10ea7b47f185997b215aea7a3f375ed713ac5)
(cherry picked from commit 168d8e0c04c0ba79aa31d04d498476334d6d61f6)
 2017-08-01 17:23:16 -07:00
Jason Lowe
64ff22de57 YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles 
(cherry picked from commit b9640c02bf5072014299124cfd44880d7361a5a7)
 2017-07-31 16:40:17 -05:00
Akira Ajisaka
222771c6ad
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
(cherry picked from commit 98c3544e9407e3e1dfae9806d4a8181508952bed)
(cherry picked from commit d57784cbda491f591e1ebd5d7a60e2b5fd23f087)
 2017-07-31 12:43:38 +09:00
Jason Lowe
db0bc8fafa YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:25:04 -05:00
Allen Wittenauer
02ef086824
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw) 
(cherry picked from commit d8c1fd1944160ad5b5d093731ae987d701802321)
(cherry picked from commit daa768534409f08a83759634b5c3966e32e6f170)
 2017-07-24 12:03:40 +09:00
Brahma Reddy Battula
a1421de70e HADOOP-13867. FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations. Contributed By Vinayakumar B. 
(cherry picked from commit 0ef796174ecb5383f79cfecfcbfc4f309d093cd7)
(cherry picked from commit 6f445408022f210e01f826499d447d4e7792b429)
 2017-07-23 16:09:29 +08:00
Brahma Reddy Battula
587d47cfec HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 
(cherry picked from commit 820496fbbc00ede0484bec5511c6b12913e97356)
 2017-07-23 16:09:10 +08:00
Jason Lowe
4fc070ac59 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c58209a8f78ff64e04e9112933489fb5)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:46:18 -05:00
Jason Lowe
293ac4c18b YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39eda1c479389b99a5c121e640a1e0ad)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:33:34 -05:00
Eric Payne
95be843851 YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
 2017-07-21 10:32:35 -05:00