Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,903 Commits 326 Branches 371 Tags
Commit Graph

8903 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
carlo curino
b465c5b907 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 
(cherry picked from commit fc42fa8ae3bc9d6d055090a7bb5e6f0c5972fcff)
(cherry picked from commit bddc48ed862bd783ce0907aa9c7c9585a5c48481)
 2015-07-24 13:42:13 -07:00
Xuan
c65b6b760f YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269221ec25834684477f434e19f0b66af)
(cherry picked from commit 98ed4bca3be0ae635e0dfa1a9945c0f663c48145)
 2015-07-24 10:17:51 -07:00
carlo curino
76339bfb9f YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2add3f112907ff4e833e2f04947b53e)
(cherry picked from commit 6a71a0a230b337c8d62cd116a9742c54f00728e9)
 2015-07-24 08:54:59 -07:00
Akira Ajisaka
b919c1c044 HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a567147b62f463c2daa3d063ad40822b)
(cherry picked from commit b2a532f61c011dacbda1c9b173d1df126ff8809f)
 2015-07-24 18:38:44 +09:00
Jian He
65af0dffa5 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 
(cherry picked from commit 393fe71771e3ac6bc0efe59d9aaf19d3576411b3)
 2015-07-21 15:06:32 -07:00
Akira Ajisaka
3e793224f9 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df0448d9f7de91602646ecf5a51c52e4)
(cherry picked from commit b85bbca74565b18dfa6689c9545d07bff5d31f83)
 2015-07-21 11:42:21 +09:00
Arun Suresh
63e4ada517 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae78918e7d756d84920a9322187d61eed)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/RMContainerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2015-07-18 13:59:19 -07:00
Jonathan Eagles
009f5fbad6 YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 
(cherry picked from commit 7faae0e6fe027a3886d9f4e290b6a488a2c55b3a)
 2015-07-17 11:16:37 -05:00
Haohui Mai
2fe3136612 HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. 2015-07-16 15:25:32 -07:00
Tsuyoshi Ozawa
d9427a16b1 YARN-3170. YARN architecture document needs updating. Contirubted by Brahma Reddy Battula. 
(cherry picked from commit edcaae44c10b7e88e68fa97afd32e4da4a9d8df7)
 2015-07-15 15:43:49 +09:00
cnauroth
180d791541 HADOOP-12232. Upgrade Tomcat dependency to 6.0.44. Contributed by Chris Nauroth. 
(cherry picked from commit 6b56f05b9ea3ee45abd5d00b14fab93a353b1332)
(cherry picked from commit 89f484e4aa0a8db779a02999109dae212db1054c)
 2015-07-14 14:53:20 -07:00
Kihwal Lee
e263219829 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee 
(cherry picked from commit 59388a801514d6af64ef27fbf246d8054f1dcc74)
 2015-07-14 14:08:15 -05:00
Tsuyoshi Ozawa
0744a578e4 HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 4084eaf94347042d9b8cb1e86ba831036621dfdd)
 2015-07-14 19:15:56 +09:00
Jian He
9aa4411325 Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa47b4c79577096817acc00ad6421180c.
(cherry picked from commit 2466460d4cd13ad5837c044476b26e63082c1d37)
 2015-07-13 14:32:18 -07:00
Karthik Kambatla
8bb8006b71 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 
(cherry picked from commit aa067c6aa47b4c79577096817acc00ad6421180c)
(cherry picked from commit ccf18705f7ef88893ac8c269f172772c6c103fa0)
 2015-07-09 09:50:14 -07:00
Devaraj K
7e5f73a952 MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb431fcdd7ca4ce0e249f9d45f4968497b)
 2015-07-09 15:05:11 +05:30
Akira Ajisaka
148b4db7e0 YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d6325745e26d6acb0bbdb0972ce036297d66d5b0)
(cherry picked from commit d260478d3a8bc6bc7dce32733920bb4812c883bf)
 2015-07-08 15:43:21 +09:00
Vinayakumar B
a292e9800b HADOOP-12186. ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times (Contributed by zhihai xu) 
(cherry picked from commit 233cab89adb6bae21d7e171f2af516b92266242c)
(cherry picked from commit 91d2c599685ff359ce174f2b669dca6d764b39cb)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-07-07 18:10:29 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8ca27a34074b415f85599dd48b8fc50)
 2015-07-06 16:41:38 -07:00
Devaraj K
c9c7a00a82 MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.

(cherry picked from commit bff67dfe2f811654ffb1bbcbd87509c185f452b6)
 2015-07-06 13:48:44 +05:30
Wangda Tan
a6f6ba95ef YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) 2015-07-02 11:08:21 -07:00
Jason Lowe
055d9292a7 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 
(cherry picked from commit b5cdf78e8e6cd6c5c1fb7286207dac72be32c0d6)
 2015-07-01 21:15:30 +00:00
Yongjun Zhang
849b47525d HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang. 
(cherry picked from commit 43a1288fff5e323895fde66b9e3eb31a7412fcc2)
 2015-06-29 14:56:14 -07:00
Vinod Kumar Vavilapalli
a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli
2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5dba5858d6e9845f18c4baec16c91911d)
 2015-06-28 16:33:49 -07:00
Arpit Agarwal
f6550361f7 HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:58:07 -07:00
Jason Lowe
baf9e22284 YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949ad6f6e0dbdd248f2d257b05899f4332)
 2015-06-26 15:48:34 +00:00
Jason Lowe
41d9677740 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e6d9fe93784a9de2af0056bcc316d96)
 2015-06-24 16:38:40 +00:00
Jason Lowe
37b89deccf YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong 
(cherry picked from commit 2a20dd9b61ba3833460cbda0e8c3e8b6366fc3ab)
 2015-06-24 16:25:22 +00:00
Vinod Kumar Vavilapalli
a90e9b2f6f MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:31 -07:00
Arpit Agarwal
4a8db40f3b HDFS-8626. Add test case for branch-2.7 2015-06-23 10:03:44 -07:00
Karthik Kambatla
6b1a156e27 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)
(cherry picked from commit 9656ee4ee763dc48435ba8bb3ac9b10f6b306fe3)
 2015-06-22 17:49:55 -07:00
Colin Patrick Mccabe
86c01d90ba HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe) 
(cherry picked from commit 7b424f938c3c306795d574792b086d84e4f06425)
(cherry picked from commit cbd11681ce8a51d187d91748b67a708681e599de)

Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
 2015-06-22 14:46:57 -07:00
Arpit Agarwal
25b99e4816 HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal) 2015-06-20 13:08:18 -07:00
Allen Wittenauer
0adb6d8292 HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw) 
(cherry picked from commit 63613c79c1042ea3d7706ed6f7eccc8cf48ff6ea)
 2015-06-20 07:45:39 -07:00
Arpit Agarwal
dc4171cb66 HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru) 2015-06-19 18:30:53 -07:00
Yongjun Zhang
d06080bbb5 HADOOP-12103. Small refactoring of DelegationTokenAuthenticationFilter to allow code sharing. Contributed by Yongjun Zhang. 
(cherry picked from commit 49f5d20efe7af7cd7c45d93edad33997a695a746)
 2015-06-19 14:50:42 -07:00
Yongjun Zhang
8c03a079df HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang. 
(cherry picked from commit b42f1ec13d0a837f188cf424a8dd177720478cc9)
 2015-06-19 09:51:48 -07:00
Colin Patrick Mccabe
e5f9703419 HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) 
(cherry picked from commit 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)
(cherry picked from commit 2946e92f79a73cccf9533f31ad40471046496cb2)
 2015-06-18 11:39:54 -07:00
Xuan
c0a419b134 YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena 2015-06-17 16:27:01 -07:00
Kihwal Lee
6de4d52c31 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46bf00af408ed866b6577623cdec02de1)
 2015-06-16 15:44:38 -05:00
Xiaoyu Yao
fe35ba1df9 HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27cc33d311ab7a7248c3cf6303997ddd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:51:34 -07:00
Yongjun Zhang
f4ae56d605 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit b1d3ed02c16113f2f6bd74f56e3d293c0daf64e2)
 2015-06-15 21:51:21 -07:00
Tsuyoshi Ozawa
6c527f5514 YARN-3711. Documentation of ResourceManager HA should explain configurations about listen addresses. Contributed by Masatake Iwasaki. 
(cherry picked from commit e8c514373f2d258663497a33ffb3b231d0743b57)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-06-16 10:16:15 +09:00
Tsz-Wo Nicholas Sze
632cb542f6 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze
7a248f2e9e HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:13:56 -07:00
Arpit Agarwal
a1810d75f0 HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal) 2015-06-14 20:18:31 -07:00
Arpit Agarwal
1145d09e0a HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:42 -07:00
Yongjun Zhang
6dda57a817 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961410b6ac2b275cdcff4b95d75727e8d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit cd578fa289252b0ae1e27f7e7b097cceeb2c30ff)
 2015-06-14 12:04:16 -07:00
Haohui Mai
3651feb49d HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 19:14:48 -07:00