Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,980 Commits 326 Branches 371 Tags
Commit Graph

7980 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sangjin Lee
84233f013d YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba06bbf0b18dca674d25a48218fed0a26)
 2016-02-24 09:37:28 -08:00
Jason Lowe
9e10dd6077 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 2016-02-23 20:54:42 +00:00
Varun Vasudev
d32100d7fc MAPREDUCE-6635. Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException. Contributed by Junping Du. 
(cherry picked from commit c6f2d761d5430eac6b9f07f137a7028de4e0660c)
(cherry picked from commit f1999fe2754cbf11b138fb048c7486cab9b02c97)

 Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2016-02-23 14:08:14 +05:30
Zhe Zhang
3fea7f0a38 HDFS-9688. Test the effect of nested encryption zones in HDFS downgrade. 
Change-Id: I2a802325a2bc82c97012d2c9fb388b292fe3af8f
 2016-02-22 15:11:00 -08:00
Zhe Zhang
44eef0ed7c HADOOP-12800. Copy docker directory from 2.8 to 2.7/2.6 repos to enable pre-commit Jenkins runs. (zhz) 
Change-Id: I893319c53e497420e2b20511f0d5c0803d8515ba
 2016-02-19 14:34:39 -08:00
Sangjin Lee
319790432d YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 
(cherry picked from commit 7de70680fe44967e2afc92ba4c92f8e7afa7b151)
 2016-02-17 20:57:09 -08:00
Vinayakumar B
89e90becfc HADOOP-12805. Annotate CanUnbuffer with @InterfaceAudience.Public (Contributed by Ted Yu) 
(cherry picked from commit 4b0e59fdc010be517dd67ddf35c962da204df547)
(cherry picked from commit e79a5e4460b5199eebb0d788fdc1eaa36d6b0ac4)
 2016-02-16 17:34:19 +05:30
Junping Du
43ac0b09e3 Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48944c38271399a6e452aa2c2a9f5b80)
(cherry picked from commit a28be9edc72cc4fe42fab107c17bcac6789af4e5)
 2016-02-13 01:51:27 -08:00
Masatake Iwasaki
39970794d7 HADOOP-12589. Fix intermittent test failure of TestCopyPreserveFlag (iwasakims) 
(cherry picked from commit 36ba5cac37dee6b60e607401ac06d263e4ac5d8f)
 2016-02-13 15:03:54 +09:00
Jason Lowe
1b013a4e83 MAPREDUCE-6191. Improve clearing stale state of Java serialization 
testcase.  (Sam Liu via Eric Yang)
(cherry picked from commit c379e102ddab1b83fd69e4492a40c9901fb50675)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 303c650d0dfcb10760cea589b8cef6ace8e36417)
 2016-02-10 16:49:25 +00:00
Arpit Agarwal
d833823daf HDFS-9752. Permanent write failures may happen to slow writers during datanode rolling upgrades. (Contributed by Walter Su) 2016-02-09 09:41:35 -08:00
Jason Lowe
f2e3935bd6 MAPREDUCE-6413. TestLocalJobSubmission is failing with unknown host. Contributed by zhihai xu 
(cherry picked from commit aa5b15b03be61ebb76a226e0de485d5228c8e3d0)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-09 16:38:59 +00:00
Zhe Zhang
1b25855b9b HDFS-9347. Invariant assumption in TestQuorumJournalManager.shutdown() is wrong. Contributed by Wei-Chiu Chuang. 
Change-Id: Idde8522e11c77f82b376e0c90a9bf33d163d5cb9
(cherry picked from commit ef3f3f6bb14cf44bef1778f1091d8ed8a4b764a3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2016-02-09 10:20:40 +08:00
Sangjin Lee
7f8c89c33a HADOOP-12773. HBase classes fail to load with client/job classloader enabled (sjlee) 
(cherry picked from commit 58acbf940a92ef8a761208a7a743175ee7b3377d)
 2016-02-08 14:00:55 -08:00
Vinayakumar B
ea517acfb6 HDFS-9629. Update the footer of Web UI to show year 2016 (Contributed by Xiao Chen and Vinayakumar B) 
(cherry picked from commit ddcd6954d03d3caadc8b69f7b886a3a17c3c9d0c)
(cherry picked from commit 8cdfdbf815bf89eee3184ec7cd00e1924c1ea316)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/index.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/explorer.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/index.html
	hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/status.html
 2016-02-04 13:34:54 +05:30
Junping Du
89bfc884a5 Adding release 2.6.5 to CHANGES.txt. 2016-02-02 20:28:20 -08:00
Kihwal Lee
d8416aec3c HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao. 
(cherry picked from commit c7c36cbd6218f46c33d7fb2f60cd52cb29e6d720)
(cherry picked from commit 4cf7f8441a288cefd44f126d60dae1998239892a)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend2.java
 2016-02-02 17:10:32 -08:00
Jason Lowe
fc01ec866a MAPREDUCE-6621. Memory Leak in JobClient#submitJobInternal(). Contributed by Xuan Gong 
(cherry picked from commit 43e669b22d9ac0b81ab8cc397c5d6d92bac986c6)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-02 19:33:32 +00:00
Junping Du
cd93382a59 HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee. 2016-02-02 10:37:29 -08:00
Junping Du
30626ad35f HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee. 2016-02-02 10:29:31 -08:00
Junping Du
572b5872fe HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 2016-02-02 10:16:59 -08:00
Jason Lowe
b52279ca28 MAPREDUCE-6618. YarnClientProtocolProvider leaking the YarnClient thread. Contributed by Xuan Gong 
(cherry picked from commit 59a212b6e1265adfa9b55c71b65a22157dfccf77)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-02-01 16:12:49 +00:00
Jian He
ac43c56358 MAPREDUCE-6619. HADOOP_CLASSPATH is overwritten in MR container. Contributed by Junping Du 
(cherry picked from commit da18bbedaffad1f3ac9c078675583e7cfb68790f)
 2016-01-27 13:22:59 -08:00
Colin Patrick Mccabe
8ef73cd4ce HDFS-7694. FSDataInputStream should support "unbuffer" (cmccabe) 
(cherry picked from commit 6b39ad0865cb2a7960dd59d68178f0bf28865ce2)
(cherry picked from commit e35788aa5a41940651af0b73dfeaeca011556904)
 2016-01-27 08:46:38 -08:00
Junping Du
8b9241f93e Revert HADOOP-12715 which cause build failure on branch-2.6. 2016-01-27 08:14:30 -08:00
Walter Su
d75375a62b HADOOP-12715. TestValueQueue#testgetAtMostPolicyALL fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit 6eacdea0e475b4fff91cedce5005a7c11749cf64)
 2016-01-25 19:43:39 +08:00
Akira Ajisaka
ce235e3cba HADOOP-12736. TestTimedOutTestsListener#testThreadDumpAndDeadlocks sometimes times out. Contributed by Xiao Chen. 
(cherry picked from commit 643227927a7d7974655627d7e97aae42600692ae)
(cherry picked from commit 38ec290455a61a53f1f2c8fcc06f914de52fd5e2)
 2016-01-25 14:00:31 +09:00
Jason Lowe
4c8a65981a YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0e41646ba02391aea31481ead28aaf8)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-22 21:59:05 +00:00
Vinod Kumar Vavilapalli
a5b85634b4 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 863079bb874ba77918ca1c0741eae10e245995c8)
(cherry picked from commit 53aa3a4d1f2e02ab60fd8b4485286b57df5fcdf9)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
 2016-01-17 18:12:40 -08:00
Tsz-Wo Nicholas Sze
b05bb4984d HDFS-9294. DFSClient deadlock when close file and failed to renew lease. Contributed by Brahma Reddy Battula 2016-01-16 23:39:08 +08:00
Jason Lowe
cbc3288b55 MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 
(cherry picked from commit 296ecbd5ec9f7d2eed84217a3318b4a3bcd7bd99)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-01-15 17:04:34 +00:00
Junping Du
840535f561 YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b234efff2b0b646c31a4e6ff0a5118ef9)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit e09222306f65227d14fe1e656102d225b232409c)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-15 09:00:03 -08:00
Jason Lowe
9b26ee836c HADOOP-12706. TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp times out occasionally. Contributed by Sangjin Lee and Colin Patrick McCabe 
(cherry picked from commit cdf88952599a43b1ef5adda792bfb195c7529fad)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-14 23:53:02 +00:00
Jason Lowe
c5f9fd15d4 HADOOP-12107. long running apps may have a huge number of StatisticsData instances under FileSystem (Sangjin Lee via Ming Ma) 
(cherry picked from commit 8e1bdc17d9134e01115ae7c929503d8ac0325207)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
 2016-01-14 23:35:16 +00:00
Arpit Agarwal
f758dacb70 HDFS-9415. Document dfs.cluster.administrators and dfs.permissions.superusergroup. (Contributed by Xiaobing Zhou) 2016-01-14 14:19:04 -08:00
Junping Du
f5c50cd888 Add missing CHANGES.txt entry for HDFS-8767 for branch-2.6 2016-01-14 06:41:00 -08:00
Akira Ajisaka
ebd4f609e9 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt. 
(cherry picked from commit 7b0964f354e90968c2dac2f7acc17214732aed64)
(cherry picked from commit 6efee8907af3f21d8526927cf5e676d9922b686b)
 2016-01-14 13:16:38 +09:00
Jason Lowe
8c8b848b95 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1c6d9fc78cd5013c860c1086d86176f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 16:00:06 +00:00
Jason Lowe
9668c965f5 YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 15:28:31 +00:00
Jason Lowe
d43a651374 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e58e1be8214dc1916a694a912075aa0a)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-12 15:16:08 +00:00
Jason Lowe
16d6b9d3eb Move MR-5982 and MR-6492 to the appropriate section 2016-01-11 18:20:13 +00:00
Jason Lowe
de92dd5828 MAPREDUCE-6492. AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask. Contributed by Bibin A Chundatt 
(cherry picked from commit ab11085b81353e1617875deb10f3c8e2a8b91a1e)
(cherry picked from commit fc10c9983901cd231f8ec959675d741512716b9e)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-01-11 17:20:56 +00:00
Jason Lowe
3d010b01b4 MAPREDUCE-5982. Task attempts that fail from the ASSIGNED state can disappear. Contributed by Chang Li 
(cherry picked from commit 7429438c859da06ed1ed34aec0ff9873bd33f98d)

Conflicts:

	hadoop-mapreduce-project/CHANGES.txt
 2016-01-11 16:51:35 +00:00
Rohith Sharma K S
58a6142c14 YARN-3849. Too much of preemption activity causing continuos killing of containers across queues. (Sunil G via wangda) 2016-01-11 12:02:38 +05:30
Kihwal Lee
b91715bc83 Supplement to HDFS-9574. 2016-01-08 13:10:09 -06:00
Kihwal Lee
04b8a19f81 HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee. 
(cherry picked from commit b06e39de4fc4f9c35afb472eef0bba2adf91954f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNodeFaultInjector.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-01-08 12:26:05 -06:00
Haohui Mai
9cb288e9fa HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. 
(cherry picked from commit 391d2d88f2ace4e95e82f4303644ba9d1dd0692a)
 2016-01-08 17:09:53 +09:00
Anubhav Dhoot
6b2abb7515 YARN-4180. AMLauncher does not retry on failures when talking to NM. (adhoot) 
(cherry picked from commit 9735afe967a660f356e953348cb6c34417f41055)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 22f2501476d987afb7bc19080a7a0db94ea72be6)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockRM.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestApplicationMasterLauncher.java

(cherry picked from commit 7c9a368b45b0e38173521a94ab32dee8a2984bf8)
 2016-01-07 15:08:05 -08:00
Vinayakumar B
79da1283d6 HDFS-9600. do not check replication if the block is under construction (Contributed by Phil Yang) 
(cherry picked from commit 34cd7cd76505d01ec251e30837c94ab03319a0c1)
(cherry picked from commit aa710bd461b593b0f3d7d7ac41ca68e1aa3fa9d6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
 2016-01-07 11:52:27 +05:30
Tsuyoshi Ozawa
4be18edf2f YARN-4380. TestResourceLocalizationService.testDownloadingResourcesOnContainerKill fails intermittently. Contributed by Varun Saxena. 
(cherry picked from commit 0656d2dc83af6a48a8d8d0e37cdf1f813124f366)
(cherry picked from commit d76b523b020fcd76e9abdd661967ca4a931d9863)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-06 14:34:15 -08:00