Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,968 Commits 326 Branches 371 Tags
Commit Graph

7968 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Zhe Zhang
1b25855b9b HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
(cherry picked from commit ef3f3f6bb14cf44bef1778f1091d8ed8a4b764a3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-09 10:20:40 +08:00
Sangjin Lee
7f8c89c33a HADOOP-12773. HBase classes fail to load with client/job classloader enabled (sjlee) 
(cherry picked from commit 58acbf940a92ef8a761208a7a743175ee7b3377d)
 2016-02-08 14:00:55 -08:00
Vinayakumar B
ea517acfb6 HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/index.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/explorer.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/index.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/status.html
 2016-02-04 13:34:54 +05:30
Junping Du
89bfc884a5 Adding release 2.6.5 to CHANGES.txt. 2016-02-02 20:28:20 -08:00
Kihwal Lee
d8416aec3c HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd6218f46c33d7fb2f60cd52cb29e6d720)
(cherry picked from commit 4cf7f8441a288cefd44f126d60dae1998239892a)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend2.java
 2016-02-02 17:10:32 -08:00
Jason Lowe
fc01ec866a MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d9ac0b81ab8cc397c5d6d92bac986c6)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-02 19:33:32 +00:00
Junping Du
cd93382a59 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee. 2016-02-02 10:37:29 -08:00
Junping Du
30626ad35f HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 2016-02-02 10:29:31 -08:00
Junping Du
572b5872fe HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 2016-02-02 10:16:59 -08:00
Jason Lowe
b52279ca28 MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 
(cherry picked from commit 59a212b6e1265adfa9b55c71b65a22157dfccf77)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-01 16:12:49 +00:00
Jian He
ac43c56358 MAPREDUCE-6619. HADOOP_CLASSPATH is overwritten in MR container. Contributed by Junping Du 
(cherry picked from commit da18bbedaffad1f3ac9c078675583e7cfb68790f)
 2016-01-27 13:22:59 -08:00
Colin Patrick Mccabe
8ef73cd4ce HDFS-7694. FSDataInputStream should support "unbuffer" (cmccabe) 
(cherry picked from commit 6b39ad0865cb2a7960dd59d68178f0bf28865ce2)
(cherry picked from commit e35788aa5a41940651af0b73dfeaeca011556904)
 2016-01-27 08:46:38 -08:00
Junping Du
8b9241f93e Revert HADOOP-12715 which cause build failure on branch-2.6. 2016-01-27 08:14:30 -08:00
Walter Su
d75375a62b HADOOP-12715. TestValueQueue#testgetAtMostPolicyALL fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit 6eacdea0e475b4fff91cedce5005a7c11749cf64)
 2016-01-25 19:43:39 +08:00
Akira Ajisaka
ce235e3cba HADOOP-12736. TestTimedOutTestsListener#testThreadDumpAndDeadlocks sometimes times out. Contributed by Xiao Chen. 
(cherry picked from commit 643227927a7d7974655627d7e97aae42600692ae)
(cherry picked from commit 38ec290455a61a53f1f2c8fcc06f914de52fd5e2)
 2016-01-25 14:00:31 +09:00
Jason Lowe
4c8a65981a YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0e41646ba02391aea31481ead28aaf8)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-22 21:59:05 +00:00
Vinod Kumar Vavilapalli
a5b85634b4 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 863079bb874ba77918ca1c0741eae10e245995c8)
(cherry picked from commit 53aa3a4d1f2e02ab60fd8b4485286b57df5fcdf9)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
 2016-01-17 18:12:40 -08:00
Tsz-Wo Nicholas Sze
b05bb4984d HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2016-01-16 23:39:08 +08:00
Jason Lowe
cbc3288b55 MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 
(cherry picked from commit 296ecbd5ec9f7d2eed84217a3318b4a3bcd7bd99)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-01-15 17:04:34 +00:00
Junping Du
840535f561 YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b234efff2b0b646c31a4e6ff0a5118ef9)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit e09222306f65227d14fe1e656102d225b232409c)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-15 09:00:03 -08:00
Jason Lowe
9b26ee836c HADOOP-12706. TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp times out occasionally. Contributed by Sangjin Lee and Colin Patrick McCabe 
(cherry picked from commit cdf88952599a43b1ef5adda792bfb195c7529fad)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-14 23:53:02 +00:00
Jason Lowe
c5f9fd15d4 HADOOP-12107. long running apps may have a huge number of StatisticsData instances under FileSystem (Sangjin Lee via Ming Ma) 
(cherry picked from commit 8e1bdc17d9134e01115ae7c929503d8ac0325207)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-14 23:35:16 +00:00
Arpit Agarwal
f758dacb70 HDFS-9415. Document dfs.cluster.administrators and dfs.permissions.superusergroup. (Contributed by Xiaobing Zhou) 2016-01-14 14:19:04 -08:00
Junping Du
f5c50cd888 Add missing CHANGES.txt entry for HDFS-8767 for branch-2.6 2016-01-14 06:41:00 -08:00
Akira Ajisaka
ebd4f609e9 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 
(cherry picked from commit 7b0964f354e90968c2dac2f7acc17214732aed64)
(cherry picked from commit 6efee8907af3f21d8526927cf5e676d9922b686b)
 2016-01-14 13:16:38 +09:00
Jason Lowe
8c8b848b95 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1c6d9fc78cd5013c860c1086d86176f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 16:00:06 +00:00
Jason Lowe
9668c965f5 YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 15:28:31 +00:00
Jason Lowe
d43a651374 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 15:16:08 +00:00
Jason Lowe
16d6b9d3eb Move MR-5982 and MR-6492 to the appropriate section 2016-01-11 18:20:13 +00:00
Jason Lowe
de92dd5828 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
(cherry picked from commit fc10c9983901cd231f8ec959675d741512716b9e)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-01-11 17:20:56 +00:00
Jason Lowe
3d010b01b4 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 
(cherry picked from commit 7429438c859da06ed1ed34aec0ff9873bd33f98d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-01-11 16:51:35 +00:00
Rohith Sharma K S
58a6142c14 YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 2016-01-11 12:02:38 +05:30
Kihwal Lee
b91715bc83 Supplement to HDFS-9574. 2016-01-08 13:10:09 -06:00
Kihwal Lee
04b8a19f81 HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee. 
(cherry picked from commit b06e39de4fc4f9c35afb472eef0bba2adf91954f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-01-08 12:26:05 -06:00
Haohui Mai
9cb288e9fa HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. 
(cherry picked from commit 391d2d88f2ace4e95e82f4303644ba9d1dd0692a)
 2016-01-08 17:09:53 +09:00
Anubhav Dhoot
6b2abb7515 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967a660f356e953348cb6c34417f41055)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 22f2501476d987afb7bc19080a7a0db94ea72be6)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 7c9a368b45b0e38173521a94ab32dee8a2984bf8)
 2016-01-07 15:08:05 -08:00
Vinayakumar B
79da1283d6 HDFS-9600. do not check replication if the block is under construction (Contributed by Phil Yang) 
(cherry picked from commit 34cd7cd76505d01ec251e30837c94ab03319a0c1)
(cherry picked from commit aa710bd461b593b0f3d7d7ac41ca68e1aa3fa9d6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
 2016-01-07 11:52:27 +05:30
Tsuyoshi Ozawa
4be18edf2f YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83af6a48a8d8d0e37cdf1f813124f366)
(cherry picked from commit d76b523b020fcd76e9abdd661967ca4a931d9863)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-06 14:34:15 -08:00
Junping Du
cfbb035420 YARN-4354. Public resource localization fails with NPE. Contributed by Jason Lowe. 
(cherry picked from commit 855d52927b6115e2cfbd97a94d6c1a3ddf0e94bb)
(cherry picked from commit 1d107d805841e9b39db01aa236068a0ff1ca46d1)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/TestLocalResourcesTrackerImpl.java
 2016-01-06 14:15:55 -08:00
Junping Du
9da7b1fdd2 YARN-2902. Killing a container that is localizing can orphan resources in the DOWNLOADING state. Contributed by Varun Saxena. 2016-01-06 13:50:35 -08:00
Junping Du
74027c24c8 Addendum patch to fix build after porting YARN-4546. 2016-01-06 06:11:00 -08:00
Junping Du
c0ffe25a65 YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff7bb632df50e1c52de971cced56c6a3)
(cherry picked from commit 1cc001db4c3767072b5d065d161bc5c6d1c480d4)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
 2016-01-06 05:56:16 -08:00
Sangjin Lee
b2d02863a1 MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee) 
(cherry picked from commit f6f16118d38fcfe3b724f05fad752cb223f441ec)
(cherry picked from commit 28fd4c70cab27c2872dc6a63ddee223ccf53f21d)
 2016-01-05 16:04:28 -08:00
Sangjin Lee
51a2e6304a Preparing for 2.6.4 development 2016-01-05 15:40:31 -08:00
Karthik Kambatla
62e032a4da YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 
(cherry picked from commit 332b520a480994b7bd56c135f7941aad30b05e9c)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/TestFairScheduler.java
 2016-01-05 00:30:37 -08:00
Junping Du
236a6ba13e HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su. 2016-01-04 06:27:52 -08:00
Akira Ajisaka
0381556a42 Set release date for 2.6.3 
(cherry picked from commit db99e30f670cb5d73d5ec79671026eeeaf337d3d)
(cherry picked from commit 247ab92464beb55bad58b6e63eeac1a41767f432)
 2016-01-04 16:18:38 +09:00
Rohith Sharma K S
7af87093ad YARN-3893. Both RM in active state when Admin#transitionToActive failure from refeshAll() (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 7d6687fe76f6152a577ff2298c358dd30fce41fb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2016-01-04 11:44:28 +05:30
Akira Ajisaka
1823b3edd0 HADOOP-11252. RPC client does not time out by default. Contributed by Wilfred Spiegelenburg and Masatake Iwasaki. 
(cherry picked from commit 64ae85fd2ea91f46ab3b21f007befbeef8c3c947)
(cherry picked from commit d98cfe611f7cedc58cdecfb5a83829adb2521138)
 2016-01-04 14:38:35 +09:00
Akira Ajisaka
aadc864988 HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. Contributed by Akira AJISAKA. 2016-01-03 15:10:54 -08:00