Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,331 Commits 326 Branches 371 Tags
Commit Graph

12331 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Junping Du
29588814b2 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
(cherry picked from commit f252e1fe2887cc961cf3ad532bfa22d646775fbf)
(cherry picked from commit 1dfb3850c880fefa2e7640c236939288d0e526f6)
 2017-08-28 18:32:29 -07:00
Brahma Reddy Battula
205a95c29b HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit c4a85c694fae3f814ab4e7f3c172da1df0e0e353)(cherry picked from commit 9a168ae884f0075dea4930e07e1ced30a0d0fea6)
 2017-08-28 22:41:08 +08:00
Brahma Reddy Battula
f4208415ee HDFS-12364. [branch-2.8.2] Fix the Compile Error after HDFS-12299(addendum to HDFS-12299). Contributed by Jiandan Yang. 2017-08-28 22:27:40 +08:00
Jason Lowe
bc91e2310f YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:35:04 -05:00
Kihwal Lee
85e5fd94b3 HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396089972e23c533eca33c9cba231c45a)
 2017-08-25 15:05:20 -05:00
Arpit Agarwal
2a25ddc0b1 HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:03:42 -07:00
Arpit Agarwal
4b85d38f60 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 12:03:39 -07:00
Colin Patrick Mccabe
86db22f890 HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe) 
(cherry picked from commit f67149ab08bb49381def6c535ab4c4610e0a4221)
 2017-08-25 12:03:37 -07:00
Jason Lowe
310fa63cc1 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java

(cherry picked from commit 2d548759ad416b0647855eac5f5813467d4cbfd8)
 2017-08-25 10:17:20 -05:00
Jason Lowe
32d2417d3b HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 
(cherry picked from commit f6892f5821aedb33fdd6cd74d631fec2e67647be)
 2017-08-22 16:57:32 -05:00
Jason Lowe
0acf92210a YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 
(cherry picked from commit ff9cbcbbf47927db82c81660b413edd7c84c5022)
 2017-08-22 16:34:32 -05:00
Jason Lowe
b14228a9d1 YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
 2017-08-22 13:29:31 -05:00
Jason Lowe
3c295955a9 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
 2017-08-22 13:03:43 -05:00
Jason Lowe
864a0964f0 YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be0a341df3e628dcaf06717b4630b05e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java
 2017-08-16 13:17:02 -05:00
Ravi Prakash
155646c48c HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
(cherry picked from commit 895a35eac0422c1845ebe3c5bed1ac2bda1ac807)
(cherry picked from commit a611922de4e2fc82506872d68c4efcb963fb15e9)
 2017-08-15 15:47:08 -07:00
Jason Lowe
77e9d9ff48 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:45:47 -05:00
Jason Lowe
de271ac987 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:50:55 -05:00
Jian He
d1996152bd YARN-6872. Ensure apps could run given NodeLabels are disabled post RM switchover/restart. Contributed by Sunil G 
(cherry picked from commit e84a3f43a1b114d85a3c60151c821b1ca753f81a)
 2017-08-09 21:03:41 -07:00
Kihwal Lee
984a08895e HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:57:08 -05:00
Kihwal Lee
5758dad28f HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 
(cherry picked from commit 74bcc8d1f213ffd18056b7d0590d0cd346abaff9)
 2017-08-09 09:38:42 -05:00
Naganarasimha
68380262ff YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
(cherry picked from commit a7cdd39234c3a236cd9b85853636d42dc8c850a4)
(cherry picked from commit 49d459a3ed82169c02fca98dd7b54e3f09768881)
 2017-08-08 16:06:24 -07:00
Jason Lowe
a17faae509 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger 
(cherry picked from commit ea67e1f266406477fec6eb4ff1ec1e1f47974166)
 2017-08-08 15:09:13 -05:00
Eric Payne
211c77a40b YARN-5349. TestWorkPreservingRMRestart#testUAMRecoveryOnRMWorkPreservingRestart fail intermittently. Contributed by Jason Lowe. 
(cherry picked from commit 8ce8672b6b551dacb9467924fc70f88790f5891f)
 2017-08-02 13:20:47 -05:00
Eric Payne
82981738d5 YARN-6846. Nodemanager can fail to fully delete application local directories when applications are killed. Contributed by Jason Lowe. 
(cherry picked from commit 48899134d2a77935a821072b5388ab1b1b7b399c)
(cherry picked from commit d5150e3148273a117e999814261a07b95def63c0)
 2017-08-02 12:09:32 -05:00
Junping Du
f4e42a728b HADOOP-14683. FileStatus.compareTo binary compatible issue. Contributed by Akira Ajisaka. 
(cherry picked from commit cf6794ebe93906c406a54acc5c8854dc600a9a7f)
(cherry picked from commit 60c10ea7b47f185997b215aea7a3f375ed713ac5)
(cherry picked from commit 168d8e0c04c0ba79aa31d04d498476334d6d61f6)
 2017-08-01 17:23:16 -07:00
Jason Lowe
64ff22de57 YARN-6628. Unexpected jackson-core-2.2.3 dependency introduced. Contributed by Jonathan Eagles 
(cherry picked from commit b9640c02bf5072014299124cfd44880d7361a5a7)
 2017-07-31 16:40:17 -05:00
Akira Ajisaka
222771c6ad
YARN-5728. TestMiniYarnClusterNodeUtilization.testUpdateNodeUtilization timeout. 
(cherry picked from commit f8bed5e9a7d1ece127fb3da123bbfc26ada0016f)
(cherry picked from commit 98c3544e9407e3e1dfae9806d4a8181508952bed)
(cherry picked from commit d57784cbda491f591e1ebd5d7a60e2b5fd23f087)
 2017-07-31 12:43:38 +09:00
Jason Lowe
db0bc8fafa YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles 
(cherry picked from commit 24853bf32a045b8f029fb136edca2af03836c8d5)
 2017-07-24 10:25:04 -05:00
Allen Wittenauer
02ef086824
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw) 
(cherry picked from commit d8c1fd1944160ad5b5d093731ae987d701802321)
(cherry picked from commit daa768534409f08a83759634b5c3966e32e6f170)
 2017-07-24 12:03:40 +09:00
Brahma Reddy Battula
a1421de70e HADOOP-13867. FilterFileSystem should override rename(.., options) to take effect of Rename options called via FilterFileSystem implementations. Contributed By Vinayakumar B. 
(cherry picked from commit 0ef796174ecb5383f79cfecfcbfc4f309d093cd7)
(cherry picked from commit 6f445408022f210e01f826499d447d4e7792b429)
 2017-07-23 16:09:29 +08:00
Brahma Reddy Battula
587d47cfec HDFS-8312. Trash does not descent into child directories to check for permissions. Contributed By Weiwei Yang via Eric Yang. 
(cherry picked from commit 820496fbbc00ede0484bec5511c6b12913e97356)
 2017-07-23 16:09:10 +08:00
Jason Lowe
4fc070ac59 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c58209a8f78ff64e04e9112933489fb5)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:46:18 -05:00
Jason Lowe
293ac4c18b YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39eda1c479389b99a5c121e640a1e0ad)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2017-07-21 14:33:34 -05:00
Eric Payne
95be843851 YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestContainerManagerWithLCE.java
 2017-07-21 10:32:35 -05:00
Kihwal Lee
e229ffee64 HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 
(cherry picked from commit 8e3a992eccff26a7344c3f0e719898fa97706b8c)
 2017-07-21 09:22:28 -05:00
Brahma Reddy Battula
f65dc6ee95 HDFS-12177. NameNode exits due to setting BlockPlacementPolicy loglevel to Debug. Contributed by Jiandan Yang. 2017-07-21 21:13:20 +08:00
Konstantin V Shvachko
f3bf1a6eb9 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 
(cherry picked from commit 2a5a313539e211736fef12010918a60f9edad030)
 2017-07-20 16:09:56 -07:00
Kihwal Lee
5d58ba0e3d HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp. 
(cherry picked from commit b3269f7cc119cbc51b16118b918f345f970804bc)
 2017-07-20 16:22:11 -05:00
Jason Lowe
6e45c543cb YARN-6837. Null LocalResource visibility or resource type can crash the nodemanager. Contributed by Jinjiang Ling 
(cherry picked from commit c8df3668ecc37c2d58cad35520a762eaec3c8539)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/impl/pb/TestApplicationClientProtocolRecords.java
 2017-07-20 11:11:41 -05:00
Wangda Tan
a39617df63 YARN-6775. CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls. (Nathan Roberts via wangda) 
Change-Id: I5951f0997547de7d2e4a30b4ad87ab0a59b3066a
 2017-07-19 15:29:45 -07:00
Anu Engineer
6ed569df21 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 413b23eb04eee24275257ab462133e0818f87449)
(cherry picked from commit e0297ffbc89e9f037d5f6a8c5874ce8794656e0c)
 2017-07-19 10:46:21 -07:00
Rohith Sharma K S
4daf5741e0 YARN-5920. Fix deadlock in TestRMHA.testTransitionedToStandbyShouldNotHang. Contributed by Varun Saxena. 
(cherry picked from commit e15c20edba1e9a23475ee6a4dfbadbdb8c1f668a)
 2017-07-17 14:52:06 -05:00
Brahma Reddy Battula
6039a2a55f HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit b778887af59d96f1fac30cae14be1cabbdb74c8b)
 2017-07-15 11:12:56 +08:00
Xiao Chen
8b7d1df766 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:23:13 -07:00
Kihwal Lee
e28c74102d HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 
(cherry picked from commit e7d187a1b6a826edd5bd0f708184d48f3674d489)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPOfferService.java
 2017-07-14 16:17:44 -05:00
Kihwal Lee
561b72970f HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 
(cherry picked from commit a47820d72a4c4aba857fd7325fa9ae9a95fe8faa)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2017-07-14 15:50:18 -05:00
Jason Lowe
1edc16ee54 YARN-3260. AM attempt fail to register before RM processes launch event. Contributed by Bibin A Chundatt 
(cherry picked from commit a5ae5ac50e97cf829c41dcf01655cd9bd4d36a00)
 2017-07-14 15:06:31 -05:00
Ravi Prakash
d6228fbebd MAPREDUCE-6910. MapReduceTrackingUriPlugin can not return the right URI of history server with HTTPS. Contributed by Lantao Jin 
(cherry picked from commit 43f0503286eccbc6bb8ae77584b635bfd0c48e50)
(cherry picked from commit 756a06814355465c85b9d66f262ee875dd86dbb7)
 2017-07-13 22:57:44 -07:00
Jason Lowe
8c69b040ac YARN-6805. NPE in LinuxContainerExecutor due to null PrivilegedOperationException exit code. Contributed by Jason Lowe 
(cherry picked from commit a77fb561efcea03060674d37ebadc15069766b6f)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java
 2017-07-13 18:00:05 -05:00
Jason Lowe
9cff53fed9 YARN-6654. RollingLevelDBTimelineStore backwards incompatible after fst upgrade. Contributed by Jonathan Eagles 
(cherry picked from commit 5f1ee72b0ebf0330417b7c0115083bc851923be4)
 2017-07-13 17:30:57 -05:00