9455 Commits

Author SHA1 Message Date
Vinayakumar B
eb3dcc0739 HDFS-9661. Deadlock in DN.FsDatasetImpl between moveBlockAcrossStorage and createRbw (Contributed by ade)
(cherry picked from commit 14255786908f991fd2022480fe5575533a3dc7ce)
2016-01-20 21:11:39 +05:30
Tsz-Wo Nicholas Sze
2131b9820d In CHANGES.txt, move HDFS-9294 to Release 2.6.4. 2016-01-16 23:48:03 +08:00
Jason Lowe
296ecbd5ec MAPREDUCE-6554. MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory. Contributed by Bibin A Chundatt 2016-01-15 16:59:44 +00:00
Junping Du
f7d3870034 YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee.
(cherry picked from commit fc6d3a3b234efff2b0b646c31a4e6ff0a5118ef9)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
(cherry picked from commit e09222306f65227d14fe1e656102d225b232409c)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2016-01-15 09:02:51 -08:00
Jason Lowe
d01362f10f YARN-4121. Fix typos in capacity scheduler documentation. Contributed by Kai Sasaki.
(cherry picked from commit 435f935ba7d8abb1a35796b72d1de906ded80592)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2016-01-15 16:13:04 +00:00
Jason Lowe
dbed88237c HADOOP-12706. TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp times out occasionally. Contributed by Sangjin Lee and Colin Patrick McCabe
(cherry picked from commit cdf88952599a43b1ef5adda792bfb195c7529fad)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
2016-01-14 23:52:10 +00:00
Jason Lowe
4976aac46e HADOOP-12107. long running apps may have a huge number of StatisticsData instances under FileSystem (Sangjin Lee via Ming Ma)
(cherry picked from commit 8e1bdc17d9134e01115ae7c929503d8ac0325207)

Conflicts:

	hadoop-common-project/hadoop-common/CHANGES.txt
2016-01-14 23:34:29 +00:00
Arpit Agarwal
6cd5fcb634 HDFS-9415. Document dfs.cluster.administrators and dfs.permissions.superusergroup. (Contributed by Xiaobing Zhou) 2016-01-14 14:19:51 -08:00
Yongjun Zhang
7a550db931 HDFS-9648. TestStartup.testImageChecksum is broken by HDFS-9569's message change. (Wei-Chiu Chuang via Yongjun Zhang)
(cherry picked from commit 817cc1f02a60ef4e372171415058fdc76c0d2e39)
2016-01-14 12:45:42 -08:00
Kihwal Lee
a90dcb0865 Fix up CHANGES.txt 2016-01-14 09:58:35 -06:00
Junping Du
c2d1d69525 Fix yarn CHANGES.txt entry for 2.7.2 2016-01-14 06:17:42 -08:00
Junping Du
0e0272f120 Add missing CHANGES.txt entry for HDFS-8767.
(cherry picked from commit e50dd19a9c816c81c64fff8b958f4036dcb98822)

Conflicts:

	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit 62eabec549ef13ac7e6ed517ef628b5b01d0ab77)
2016-01-14 06:14:56 -08:00
Akira Ajisaka
b1ee278428 MAPREDUCE-6363. [NNBench] Lease mismatch error when running with multiple mappers. Contributed by Vlad Sharanhovich and Bibin A Chundatt.
(cherry picked from commit 7b0964f354e90968c2dac2f7acc17214732aed64)
(cherry picked from commit 6efee8907af3f21d8526927cf5e676d9922b686b)
2016-01-14 10:42:02 +09:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
f93f912969 MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause duplicate records (wilfreds via rkanter)
Putting CHANGES.txt entry under 2.6.3 after the backport.

(cherry picked from commit e3e09aa1e75905db1fe377bad04756eadf1fc632)
2016-01-13 15:08:13 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
faebc20e26 HADOOP-5323. Trash documentation should describe its directory structure and configurations. (Weiwei Yang via ozawa)
Moved entry from HDFS CHANGES.txt to that of common.

(cherry picked from commit cb73c753dab836105247945392ad4e43695a50bc)
2016-01-13 14:56:58 -08:00
Kihwal Lee
83a4791243 HDFS-9047. Retire libwebhdfs. 2016-01-13 11:01:35 -06:00
Yongjun Zhang
4c09a75f3e HDFS-9569. Log the name of the fsimage being loaded for better supportability. (Yongjun Zhang)
(cherry picked from commit 25051c3bd08efc12333a6acb51782cc7800403a4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
2016-01-12 09:40:56 -08:00
Jason Lowe
4b829c41ce YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li
(cherry picked from commit 13de8359a1c6d9fc78cd5013c860c1086d86176f)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2016-01-12 15:59:14 +00:00
Jason Lowe
a30b8ef59e YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti
(cherry picked from commit 62e583c7dcbb30d95d8b32a4978fbdb3b98d67cc)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2016-01-12 15:04:43 +00:00
Jason Lowe
fec437ac31 Add MR-5982 and MR-6492 to 2.6.4
(cherry picked from commit e5df366707e326d869f72db03b08228b33ff6e46)
2016-01-11 18:33:47 +00:00
Kihwal Lee
b06e39de4f HDFS-9574. Reduce client failures during datanode restart. Contributed by Kihwal Lee. 2016-01-08 11:31:49 -06:00
Vinayakumar B
5dc5bb2812 HDFS-9600. do not check replication if the block is under construction (Contributed by Phil Yang)
(cherry picked from commit 34cd7cd76505d01ec251e30837c94ab03319a0c1)
(cherry picked from commit aa710bd461b593b0f3d7d7ac41ca68e1aa3fa9d6)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
2016-01-07 11:50:12 +05:30
Junping Du
258ef9f9c3 YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe.
(cherry picked from commit c1462a67ff7bb632df50e1c52de971cced56c6a3)
(cherry picked from commit 1cc001db4c3767072b5d065d161bc5c6d1c480d4)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/TestSchedulerApplicationAttempt.java
2016-01-06 06:18:19 -08:00
Sangjin Lee
28fd4c70ca MAPREDUCE-6577. MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set (sjlee)
(cherry picked from commit f6f16118d38fcfe3b724f05fad752cb223f441ec)
2016-01-05 16:01:09 -08:00
Akira Ajisaka
68c68e187f Set release date for 2.6.3
(cherry picked from commit db99e30f670cb5d73d5ec79671026eeeaf337d3d)
(cherry picked from commit 247ab92464beb55bad58b6e63eeac1a41767f432)
2016-01-04 16:14:30 +09:00
Akira Ajisaka
3a1c98bba3 HADOOP-11252. RPC client does not time out by default. Contributed by Wilfred Spiegelenburg and Masatake Iwasaki.
(cherry picked from commit 64ae85fd2ea91f46ab3b21f007befbeef8c3c947)
(cherry picked from commit d98cfe611f7cedc58cdecfb5a83829adb2521138)
2016-01-04 14:35:54 +09:00
Kihwal Lee
cdf9f1c662 HDFS-7163. WebHdfsFileSystem should retry reads according to the configured retry policy. Contributed by Eric Payne. 2015-12-28 10:57:31 -06:00
Haohui Mai
e0e5a0fa82 HDFS-8914. Document HA support in the HDFS HdfsDesign.md. Contributed by Lars Francke.
(cherry picked from commit 049ccf71ce4457296974f12f5159540f8fcf1ede)
2015-12-21 21:44:46 +09:00
Akira Ajisaka
5ffad1551b HDFS-9505. HDFS Architecture documentation needs to be refreshed. Contributed by Masatake Iwasaki.
(cherry picked from commit fa544020f6f71ee993f047c9b986c047a25ed84c)
(cherry picked from commit 7526a2a005621e2d1d5eed6561fcd1b05ad20037)
2015-12-21 21:27:28 +09:00
Akira Ajisaka
e94a8aea57 MAPREDUCE-6583. Clarify confusing sentence in MapReduce tutorial document. Contributed by Kai Sasaki.
(cherry picked from commit 7995a6ea4dc524e5b17606359d09df72d771224a)
(cherry picked from commit 8607cb6074b40733d8990618a44c490f9f303ae3)
2015-12-21 00:19:11 +09:00
Junping Du
803fdc9f5e Creating 2.6.4 entries in CHANGES.txt files.
(cherry picked from commit 028894281bccd4e2f37eebff2e46409404ef64b7)

(cherry picked from commit 092c469059e32a3b45b88f24944ff31a22c4b394)
2015-12-18 14:24:51 +09:00
Akira Ajisaka
eed00804a7 HADOOP-12588. Fix intermittent test failure of TestGangliaMetrics. Contributed by Masatake Iwasaki.
(cherry picked from commit bd5e207432c54cc75fd8c040e232241678987e53)
(cherry picked from commit e06c291245902a6014770134443140eb78048240)
2015-12-18 13:13:30 +09:00
Kihwal Lee
03292b234e HDFS-9533. seen_txid in the shared edits directory is modified during bootstrapping. Contributed by Kihwal Lee.
(cherry picked from commit 5cb1e0118b173a95c1f7bdfae1e58d7833d61c26)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestBootstrapStandby.java
(cherry picked from commit 3c0ae5914ca673b055a9673b4c17c8aaa570c258)
2015-12-17 17:28:24 -06:00
cnauroth
56882e7b1c Revert "HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang."
This reverts commit 4d2ee51c52d371a10501a76f5bfc2d78a40e2083.
2015-12-17 13:49:47 -08:00
Kihwal Lee
4d2ee51c52 HDFS-9569. Log the name of the fsimage being loaded for better supportability. Contributed by Yongjun Zhang.
(cherry picked from commit eb6939cea0343840c62b930d4adb377f5eaf879f)
2015-12-17 10:33:15 -06:00
Junping Du
face83775b YARN-4452. NPE when submit Unmanaged application. Contributed by Naganarasimha G R.
(cherry picked from commit 50bd067e1d63d4c80dc1e7bf4024bfaf42cf4416)
(cherry picked from commit 1a2ef845b54166194b133e524ad9533cc259aed2)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/metrics/SystemMetricsPublisher.java
(cherry picked from commit 0f9dd488428ae65a17043f3a976c6ef778806328)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2015-12-17 06:07:50 -08:00
Plamen Jeliazkov
d97696e4c4 HDFS-9516. Truncate file fails with data dirs on multiple disks. Contributed by Plamen Jeliazkov. 2015-12-17 00:09:48 -08:00
Vinod Kumar Vavilapalli
cab5d86738 Revert "MAPREDUCE-6566. Add retry support to mapreduce CLI tool. Contributed by Varun Vasudev"
This reverts commit ba02e8f6d15cdfad4faac0ff3bad0ba34d305aea.
2015-12-16 15:17:01 -08:00
Vinod Kumar Vavilapalli
7b3f027ed4 HADOOP-12415. Fixed pom files to correctly include compile-time dependency on netty. (Tom Zeng via cos)
Addendum commit to include CHANGES.txt entry.

(cherry picked from commit ce16541c62728f69634111dc2ddb321690d3d29b)
2015-12-16 14:43:20 -08:00
Tom Zeng
6c62d422d6 HADOOP-12415. Add io.netty dependency to hadoop-nfs and to hadoop-hdfs, needed to build Bigtop successfuly, see BIGTOP-2049
Signed-off-by: Konstantin Boudnik <cos@boudnik.org>
(cherry picked from commit b4c6b511e705230dfe4580288addacb81f5c5c5f)
2015-12-16 14:27:30 -08:00
Zhihai Xu
e1a2f9e6e0 Update CHANGES.txt to move MAPREDUCE-6436 from YARN to MAPREDUCE
(cherry picked from commit 7092d47fc0b3b792dd31f967c01d460dc089f60b)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
	hadoop-yarn-project/CHANGES.txt
2015-12-15 22:27:32 -08:00
Jason Lowe
8ab642989c YARN-4439. Clarify NMContainerStatus#toString method. Contributed by Jian He 2015-12-15 18:06:13 +00:00
Jason Lowe
97a20ff2c6 Revert "YARN-4439. Clarify NMContainerStatus#toString method. Contributed by"
This reverts commit 1507d30bc126cac3f18a87de0936c2d0d0d1ccff.

Conflicts:

	hadoop-yarn-project/CHANGES.txt
2015-12-15 17:59:45 +00:00
Zhihai Xu
35a5b8929e MAPREDUCE-6436. JobHistory cache issue. Contributed by Kai Sasaki
(cherry picked from commit 5b7078d06921893200163a3d29c8901c3c0107cb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-15 01:24:03 -08:00
Xuan
1507d30bc1 YARN-4439. Clarify NMContainerStatus#toString method. Contributed by
Jian He

(cherry picked from commit d8a45425eba372cdebef3be50436b6ddf1c4e192)
(cherry picked from commit 992f65dd8b53ee31b6b7347e2cfe59a78d031268)
2015-12-14 15:42:38 -08:00
Kihwal Lee
a02312546a HDFS-9445. Datanode may deadlock while handling a bad volume. Contributed by Walter Su.
(cherry picked from commit a48301791e9564363bc2abad4e89e344b0d7a5ff)
2015-12-11 08:47:22 -06:00
Akira Ajisaka
02f666b324 HADOOP-12602. TestMetricsSystemImpl#testQSize occasionally fails. Contributed by Masatake Iwasaki.
(cherry picked from commit eee0cf4611b02171e8a043f1cc5f7dbad47fc3b4)
(cherry picked from commit 5047ee56af9edd2e6240286e2238b638a738f570)
2015-12-11 16:19:28 +09:00
Akira Ajisaka
c9be259f26 YARN-4434. NodeManager Disk Checker parameter documentation is not correct. Contributed by Weiwei Yang.
(cherry picked from commit 16dc65811d4e0e7ab61c61295149848fe53ddd89)
(cherry picked from commit 925bb94d87db7e916ed380d4512225e796f9e03d)
2015-12-09 16:26:58 +09:00
Wangda Tan
fd18377255 YARN-4424. Fix deadlock in RMAppImpl. (Jian he via wangda)
(cherry picked from commit 7e4715186d31ac889fba26d453feedcebb11fc70)

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 7013f9d6cda88e72a839b1c55757615b55101beb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
2015-12-08 14:31:31 -08:00
Vinod Kumar Vavilapalli
067d14f503 HDFS-9273. Moving to 2.6.3 CHANGES section to reflect the backport. 2015-12-08 11:36:15 -08:00