Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,472 Commits 326 Branches 371 Tags
Commit Graph

9472 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Karthik Kambatla
d610144d45 MAPREDUCE-6439. AM may fail instead of retrying if RM shuts down during the allocate call. (Anubhav Dhoot via kasha) 
(cherry picked from commit 8dfec7a1979e8f70f8355c096874921d368342ef)
(cherry picked from commit 20734320a18f12a28e6e4cd7b73dfcb5df869696)
 2015-08-15 00:56:22 -07:00
Vinayakumar B
4d94bc1d95 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
Moved to 2.6.1

(cherry picked from commit 84bf71295a5e52b2a7bb69440a885a25bc75f544)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 16:15:10 +05:30
Vinayakumar B
ca3215a71b HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru) 
Moved to 2.6.1

(cherry picked from commit fc508b41a90a193f06a0e28ee40862bf9ee0f0c7)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-08-14 15:57:40 +05:30
Vinayakumar B
b3546b6034 HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 295d678be8853a52c3ec3da43d9265478d6632b3)
(cherry picked from commit 80697e4f324948ec32b4cad3faccba55287be652)
 2015-08-14 15:50:11 +05:30
Vinayakumar B
d8d33055b0 HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full (zhaoyunjiong via cmccabe) 
Moved to 2.6.1

(cherry picked from commit 05ed69058f22ebeccc58faf0be491c269e950526)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt

(cherry picked from commit 796b94df1ea0269617417bb7889ed4035ed0f5a2)
 2015-08-14 12:56:25 +05:30
Vinayakumar B
90f3641728 HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang) 
Moved to 2.6.1

(cherry picked from commit 08bd4edf4092901273da0d73a5cc760fdc11052b)
 2015-08-14 12:38:59 +05:30
Vinayakumar B
8f4a09b607 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit e7aa81394dce61cc96d480e21204263a5f2ed153)
 2015-08-14 12:24:59 +05:30
Vinayakumar B
a046f7e570 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit d6050f06a3b7e049541b1cb4597c388abf00a5be)
 2015-08-14 12:10:58 +05:30
Vinayakumar B
b06d342749 HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo. 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit fa2641143c0d74c4fef122d79f27791e15d3b43f)
 2015-08-14 11:46:58 +05:30
Vinayakumar B
f40714f8d8 HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe) 
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit f2b4bc9b6a1bd3f9dbfc4e85c1b9bde238da3627)
 2015-08-14 11:38:28 +05:30
Vinayakumar B
5ba4065345 HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne. 
Moved CHANGES.TXT entry to 2.6.1

(cherry picked from commit d25cb8fe12d00faf3e8f3bfd23fd1b01981a340f)
(cherry picked from commit 2c0063d4206efffd9cd659269d7e5dc502e44f33)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-08-14 11:27:46 +05:30
Jian He
de1c4121a1 YARN-4047. ClientRMService getApplications has high scheduler lock contention. Contributed by Jason Lowe 
(cherry picked from commit 7a445fcfabcf9c6aae219051f65d3f6cb8feb87c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-08-13 16:15:44 -07:00
Xuan
617e294a63 YARN-3999. RM hangs on draing events. Contributed by Jian He 2015-08-11 18:27:23 -07:00
Vinod Kumar Vavilapalli
67ef0989bc Adding release 2.6.1 to CHANGES.txt 2015-08-11 13:25:36 -07:00
Jian He
c170f58a59 HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy Battula 
(cherry picked from commit f59612edd74d1bef2b60870c24c1f67c56b2b3cb)
 2015-08-05 16:14:35 -07:00
cnauroth
015d868367 HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth. 
(cherry picked from commit 3da0bedaec38714a62c3b7244e84bc4f981ddf45)
(cherry picked from commit b83d475252cc49f3c3289ca6a6d2d5e24dfa1ac9)
 2015-08-05 14:57:03 -07:00
Jonathan Eagles
dcffe43419 YARN-3978. Configurably turn off the saving of container info in Generic AHS (Eric Payne via jeagles) 
(cherry picked from commit 3cd02b95224e9d43fd63a4ef9ac5c44f113f710d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java
 2015-08-03 10:53:43 -05:00
Jason Lowe
e1ae722a7d YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected. Contributed by Bibin A Chundatt 
(cherry picked from commit 32e490b6c035487e99df30ce80366446fe09bd6c)
 2015-07-31 17:39:32 +00:00
Jason Lowe
e8410c0175 YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. Contributed by zhihai xu 
(cherry picked from commit ff9c13e0a739bb13115167dc661b6a16b2ed2c04)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/TestContainerLogsPage.java
 2015-07-24 22:27:52 +00:00
carlo curino
b465c5b907 YARN-3969. Updating CHANGES.txt to reflect the correct set of branches where this is committed 
(cherry picked from commit fc42fa8ae3bc9d6d055090a7bb5e6f0c5972fcff)
(cherry picked from commit bddc48ed862bd783ce0907aa9c7c9585a5c48481)
 2015-07-24 13:42:13 -07:00
Xuan
c65b6b760f YARN-3967. Fetch the application report from the AHS if the RM does not 
know about it. Contributed by Mit Desai

(cherry picked from commit fbd6063269221ec25834684477f434e19f0b66af)
(cherry picked from commit 98ed4bca3be0ae635e0dfa1a9945c0f663c48145)
 2015-07-24 10:17:51 -07:00
carlo curino
76339bfb9f YARN-3969. Allow jobs to be submitted to reservation that is active but does not have any allocations. (subru via curino) 
(cherry picked from commit 0fcb4a8cf2add3f112907ff4e833e2f04947b53e)
(cherry picked from commit 6a71a0a230b337c8d62cd116a9742c54f00728e9)
 2015-07-24 08:54:59 -07:00
Akira Ajisaka
b919c1c044 HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang. 
(cherry picked from commit 206d4933a567147b62f463c2daa3d063ad40822b)
(cherry picked from commit b2a532f61c011dacbda1c9b173d1df126ff8809f)
 2015-07-24 18:38:44 +09:00
Jian He
65af0dffa5 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 
(cherry picked from commit 393fe71771e3ac6bc0efe59d9aaf19d3576411b3)
 2015-07-21 15:06:32 -07:00
Akira Ajisaka
3e793224f9 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka) 
(cherry picked from commit 18a91fe4df0448d9f7de91602646ecf5a51c52e4)
(cherry picked from commit b85bbca74565b18dfa6689c9545d07bff5d31f83)
 2015-07-21 11:42:21 +09:00
Arun Suresh
63e4ada517 YARN-3535. Scheduler must re-request container resources when RMContainer transitions from ALLOCATED to KILLED (rohithsharma and peng.zhang via asuresh) 
(cherry picked from commit 9b272ccae78918e7d756d84920a9322187d61eed)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmcontainer/RMContainerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestAbstractYarnScheduler.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2015-07-18 13:59:19 -07:00
Jonathan Eagles
009f5fbad6 YARN-3905. Application History Server UI NPEs when accessing apps run after RM restart (Eric Payne via jeagles) 
(cherry picked from commit 7faae0e6fe027a3886d9f4e290b6a488a2c55b3a)
 2015-07-17 11:16:37 -05:00
Haohui Mai
2fe3136612 HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. 2015-07-16 15:25:32 -07:00
Tsuyoshi Ozawa
d9427a16b1 YARN-3170. YARN architecture document needs updating. Contirubted by Brahma Reddy Battula. 
(cherry picked from commit edcaae44c10b7e88e68fa97afd32e4da4a9d8df7)
 2015-07-15 15:43:49 +09:00
cnauroth
180d791541 HADOOP-12232. Upgrade Tomcat dependency to 6.0.44. Contributed by Chris Nauroth. 
(cherry picked from commit 6b56f05b9ea3ee45abd5d00b14fab93a353b1332)
(cherry picked from commit 89f484e4aa0a8db779a02999109dae212db1054c)
 2015-07-14 14:53:20 -07:00
Kihwal Lee
e263219829 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee 
(cherry picked from commit 59388a801514d6af64ef27fbf246d8054f1dcc74)
 2015-07-14 14:08:15 -05:00
Tsuyoshi Ozawa
0744a578e4 HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 4084eaf94347042d9b8cb1e86ba831036621dfdd)
 2015-07-14 19:15:56 +09:00
Jian He
9aa4411325 Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa47b4c79577096817acc00ad6421180c.
(cherry picked from commit 2466460d4cd13ad5837c044476b26e63082c1d37)
 2015-07-13 14:32:18 -07:00
Karthik Kambatla
8bb8006b71 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 
(cherry picked from commit aa067c6aa47b4c79577096817acc00ad6421180c)
(cherry picked from commit ccf18705f7ef88893ac8c269f172772c6c103fa0)
 2015-07-09 09:50:14 -07:00
Devaraj K
7e5f73a952 MAPREDUCE-6426. TestShuffleHandler#testGetMapOutputInfo is failing. 
Contributed by zhihai xu.

(cherry picked from commit fffb15bb431fcdd7ca4ce0e249f9d45f4968497b)
 2015-07-09 15:05:11 +05:30
Akira Ajisaka
148b4db7e0 YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d6325745e26d6acb0bbdb0972ce036297d66d5b0)
(cherry picked from commit d260478d3a8bc6bc7dce32733920bb4812c883bf)
 2015-07-08 15:43:21 +09:00
Vinayakumar B
a292e9800b HADOOP-12186. ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times (Contributed by zhihai xu) 
(cherry picked from commit 233cab89adb6bae21d7e171f2af516b92266242c)
(cherry picked from commit 91d2c599685ff359ce174f2b669dca6d764b39cb)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2015-07-07 18:10:29 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
dcafd3ccd2 Release process for 2.7.1: Set the release date for 2.7.1. 
(cherry picked from commit bf89ddb9b8ca27a34074b415f85599dd48b8fc50)
 2015-07-06 16:41:38 -07:00
Devaraj K
c9c7a00a82 MAPREDUCE-6425. ShuffleHandler passes wrong "base" parameter to 
getMapOutputInfo if mapId is not in the cache. Contributed by zhihai xu.

(cherry picked from commit bff67dfe2f811654ffb1bbcbd87509c185f452b6)
 2015-07-06 13:48:44 +05:30
Wangda Tan
a6f6ba95ef YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) 2015-07-02 11:08:21 -07:00
Jason Lowe
055d9292a7 YARN-3793. Several NPEs when deleting local files on NM recovery. Contributed by Varun Saxena 
(cherry picked from commit b5cdf78e8e6cd6c5c1fb7286207dac72be32c0d6)
 2015-07-01 21:15:30 +00:00
Yongjun Zhang
849b47525d HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang. 
(cherry picked from commit 43a1288fff5e323895fde66b9e3eb31a7412fcc2)
 2015-06-29 14:56:14 -07:00
Vinod Kumar Vavilapalli
a36f090c55 Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2 2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli
2e99675c72 Adding release 2.7.2 to CHANGES.txt. 
(cherry picked from commit aad6a7d5dba5858d6e9845f18c4baec16c91911d)
 2015-06-28 16:33:49 -07:00
Arpit Agarwal
f6550361f7 HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal) 2015-06-28 14:58:07 -07:00
Jason Lowe
baf9e22284 YARN-3850. NM fails to read files from full disks which can lead to container logs being lost and other issues. Contributed by Varun Saxena 
(cherry picked from commit 40b256949ad6f6e0dbdd248f2d257b05899f4332)
 2015-06-26 15:48:34 +00:00
Jason Lowe
41d9677740 YARN-3832. Resource Localization fails on a cluster due to existing cache directories. Contributed by Brahma Reddy Battula 
(cherry picked from commit 8d58512d6e6d9fe93784a9de2af0056bcc316d96)
 2015-06-24 16:38:40 +00:00
Jason Lowe
37b89deccf YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong 
(cherry picked from commit 2a20dd9b61ba3833460cbda0e8c3e8b6366fc3ab)
 2015-06-24 16:25:22 +00:00
Vinod Kumar Vavilapalli
a90e9b2f6f MAPREDUCE-6410. Fixed MapReduce JobHistory server to use the right (login) UGI to refresh log and cleaner settings. Contributed by Varun Saxena. 
(cherry picked from commit d481684c7c9293a94f54ef622a92753531c6acc7)
 2015-06-23 11:41:31 -07:00
Arpit Agarwal
4a8db40f3b HDFS-8626. Add test case for branch-2.7 2015-06-23 10:03:44 -07:00