Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,149 Commits 326 Branches 371 Tags
Commit Graph

12149 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaobing Zhou
b9b6bd2932 HDFS-11628. Clarify the behavior of HDFS Mover in documentation. Contributed by Xiaobing Zhou. 2017-04-05 15:34:27 -07:00
Varun Saxena
bf2c63ce46 YARN-6437. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently (Jason Lowe via Varun Saxena) 
(cherry picked from commit 7d963c477ab997e0daad760dd3c568d7010fe511)
 2017-04-06 02:09:32 +05:30
Jason Lowe
ab231f5e1c YARN-6403. Invalid local resource request can raise NPE and make NM exit. Contributed by Tao Yang 2017-04-05 12:36:26 -05:00
Karthik Kambatla
9c1cf63a2d YARN-6436. TestSchedulingPolicy#testParseSchedulingPolicy timeout is too low. (Eric Badger via kasha) 
(cherry picked from commit 9cc04b4708879012da1f9ef690679d8d1e45c0cc)
 2017-04-05 10:04:36 -05:00
Varun Saxena
4fb72699ff YARN-6420. RM startup failure due to wrong order in nodelabel editlog (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit 19b89c4c7be0236aafded560b7bcad7e39d1c4ed)
 2017-04-05 09:43:36 +05:30
Mingliang Liu
e47c95ffbf HADOOP-14268. Fix markdown itemization in hadoop-aws documents. Contributed by Akira Ajisaka 2017-04-03 11:15:03 -07:00
Wei-Chiu Chuang
f9d3bb3b83 HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang. 
(cherry picked from commit bc7aff7cec07bbc3fed63a00c8f1584c34670998)
(cherry picked from commit 4f1f16f1a67dcb6db705a73bf7fcf3f0f9d7094c)
 2017-04-03 07:39:41 -07:00
Eric Payne
800d632caa YARN-6354. LeveldbRMStateStore can parse invalid keys when recovering reservations. Contributed by Jason Lowe 
(cherry picked from commit 318bfb01bc6793da09e32e9cc292eb63224b6ca2)
 2017-03-31 12:37:15 -05:00
Mingliang Liu
14e5a8ed28 HDFS-11592. Closing a file has a wasteful preconditions in NameNode. Contributed by Eric Badger 
(cherry picked from commit e5f0622a6f40706d360d45200c8f259c79046438)
 2017-03-30 15:47:21 -07:00
Jason Lowe
a351a30d9d MAPREDUCE-6850. Shuffle Handler keep-alive connections are closed from the server side. Contributed by Jonathan Eagles 
(cherry picked from commit c8bd5fc7a86f9890ceaa37a89491ab650e7e9a64)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/test/java/org/apache/hadoop/mapred/TestShuffleHandler.java
 2017-03-30 11:15:25 -05:00
Akira Ajisaka
6e005ad326
HADOOP-14256. [S3A DOC] Correct the format for "Seoul" example. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0d053eeb3002f348d34cf3aac5b43333af8ff10a)
(cherry picked from commit bb187e145a851e734b00a31d9df2efe31a8ac214)
 2017-03-30 18:13:55 +09:00
Zhe Zhang
0acf2bca9c MAPREDUCE-6873. MR Job Submission Fails if MR framework application path not on defaultFS. Contributed by Erik Krogen. 
(cherry picked from commit de7efd2687ccb74608249abf4c83c84a9345ec0a)
(cherry picked from commit 3f5dc51ec94af5a20a319dd9bc807fc9e4db4f6a)
 2017-03-29 22:19:50 -07:00
Varun Saxena
a2020024e2 YARN-6352. Header injections are possible in application proxy servlet (Naganarasimha G R via Varun Saxena) 
(cherry picked from commit 3fe7d36e72ec4167ad02e08a2414169385bad8c0)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java
 2017-03-30 04:06:54 +05:30
Mingliang Liu
ba3a726988 HADOOP-14247. FileContextMainOperationsBaseTest should clean up test root path. Contributed by Mingliang Liu 
(cherry picked from commit 13c766b62c48ae07849f7f1762ef6883239aef54)
 2017-03-29 11:23:46 -07:00
Andrew Wang
7b8f87eb78 HDFS-9705. Refine the behaviour of getFileChecksum when length = 0. Contributed by SammiChen and Kai Zheng. 
(cherry picked from commit 82b4a9c3d058bcf54051d3075ca14f30988f5cb4)
 2017-03-28 21:41:10 -07:00
Varun Saxena
5e751f3367 YARN-5368. Memory leak in timeline server (Jonathan Eagles via Varun Saxena) 
(cherry picked from commit 01aca54a22c8586d232a8f79fe9977aeb8d09b83)
 2017-03-29 02:01:53 +05:30
Jason Lowe
b1f0bde5dc YARN-6359. TestRM#testApplicationKillAtAcceptedState fails rarely due to race condition. Contributed by Robert Kanter 
(cherry picked from commit bea8f8190f036375981ef4ceda409f9199730740)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRM.java
 2017-03-28 09:54:55 -05:00
Masatake Iwasaki
ca3ede5fda HDFS-11486. Client close() should not fail fast if the last block is being decommissioned. Contributed by Wei-Chiu Chuang and Yiqun Lin. 2017-03-28 18:15:09 +09:00
John Zhuge
c543c0549d HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge. 
(cherry picked from commit 253e3e78abb9ebe0e8103d25538ff07875b33c84)
 2017-03-27 23:08:03 -07:00
Wangda Tan
b218676b15 YARN-6339. Improve performance for createAndGetApplicationReport. (Yunjiong Zhao via wangda) 
(cherry picked from commit cd014d57aa8b896da02b5bcadafbd404bca2bc12)
 2017-03-27 14:05:54 -07:00
Jonathan Eagles
ec5a65a0ca Configuration#get return value optimization (jeagles) 
(cherry picked from commit e41aac0cd41c6a4ee740afd09126bc48072880ab)
 2017-03-27 12:53:42 -05:00
Jonathan Eagles
4e40e9d4db Delay construction of PreCondition.check failure message in Configuration#set (jeagles) 
(cherry picked from commit 858d597be00ce006c1d1d4cd476729e4f1de8d25)
 2017-03-27 11:06:41 -05:00
Junping Du
19d479168e MAPREDUCE-6868. License check for jdiff output files should be ignored. Contributed by Akira Ajisaka. 
(cherry picked from commit 945b006fe85473c1bb87830dbbfc1df9051c7bb9)
(cherry picked from commit 0e8beb214d0635730a9e1126f06d67b1803617ff)
 2017-03-26 23:19:49 -07:00
Joe Mészáros
9909174677
MAPREDUCE-6866. Fix getNumMapTasks() documentation in JobConf. 
This closes #205

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e0a2eb71e3eb3efa865068089b649624132d33a1)
(cherry picked from commit 34a11b910896f9c543dab04e3a1ca88ea481b9f3)
 2017-03-27 13:53:08 +09:00
Junping Du
7ec1e6e4f0 Add 2.8.0 RELEASENOTEs.md and CHANGES.md. 
(cherry picked from commit d8e6d6df4c23aa11455c1775391b1c7883bec2ea)
(cherry picked from commit e796105a6fe17a9307f3936cd59763cb173462f7)
 2017-03-25 14:35:43 -07:00
Junping Du
070b9280af Set jdiff.stable.api to 2.8.0 
(cherry picked from commit e7b7c96a30b0fee3af22a7719eaaa6481e76a963)
(cherry picked from commit 43f23e50216d6e21c982f27c9eca7c87cc803b82)
 2017-03-25 14:34:55 -07:00
Junping Du
86800b30a9 Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release. 
(cherry picked from commit 6df029db36412b7219b64313dcbe1874dc1c8b0c)
(cherry picked from commit 3ad4d3736391f4e443d7fef79770a1fda2a535c2)
 2017-03-25 14:34:55 -07:00
Junping Du
a4fcf1c7ca Add HDFS jdiff for 2.8.0 release. 
(cherry picked from commit d174c06b01e1f743d3111b9b760a9824d8106b86)
(cherry picked from commit 2fefa1fa2ec8b5523c81e25b8db4a6e5928e8996)
 2017-03-25 14:34:55 -07:00
John Zhuge
d370d8ddb8 HADOOP-14230. TestAdlFileSystemContractLive fails to clean up. Contributed by John Zhuge. 
(cherry picked from commit d1b7439b48caa18d64a94be1ad5e4927ce573ab8)
 2017-03-24 13:41:34 -07:00
Andrew Wang
5130128a31 HADOOP-14211. FilterFs and ChRootedFs are too aggressive about enforcing 'authorityNeeded'. Contributed by Erik Krogen. 
(cherry picked from commit 0e556a5ba645570d381beca60114a1239b27d49f)
(cherry picked from commit 96fe940e59127dc7c3e4182c3ed450c3cd8d858e)
 2017-03-24 11:25:58 -07:00
John Zhuge
a92f654aa2 HADOOP-14195. CredentialProviderFactory$getProviders is not thread-safe. Contributed by Vihang Karajgaonkar. 
(cherry picked from commit 128015584d69492806fd1700c8f840d78aa9c729)
 2017-03-23 13:58:11 -07:00
Mingliang Liu
47d3c29e30 HADOOP-14214. DomainSocketWatcher::add()/delete() should not self interrupt while looping await(). Contributed by Mingliang Liu 
(cherry picked from commit d35e79abc2fee7153a6168e6088f100de59d8c81)
 2017-03-23 13:39:42 -07:00
Chris Douglas
5e794c74e5 HADOOP-14197. Fix ADLS doc for credential provider. Contributed by John Zhuge 
(cherry picked from commit c55195588f5853b16d34cb6389cc6f84acc3edea)
(cherry picked from commit 62b866572674e47f812be3d0e762bd5d488bfbdb)
 2017-03-23 12:17:52 -07:00
Chris Douglas
88252e85ad HADOOP-14196. Azure Data Lake doc is missing required config entry. Contributed by Atul Sikaria 
(cherry picked from commit 7d4884a57bcaf2ab1ad120c436f9c7e6739dbc2a)
(cherry picked from commit 2cea8293ef519ead772564750bd5e7f8b9d33da5)
 2017-03-23 12:13:42 -07:00
Mingliang Liu
45956131b9 HADOOP-14135. Remove URI parameter in AWSCredentialProvider constructors. Contributed by Mingliang Liu 
(cherry picked from commit 2e30aa72e01de7b5774fcb312406a393221e0908)
 2017-03-23 11:42:23 -07:00
John Zhuge
be58711e91 HDFS-11561. HttpFS doc errors. Contributed by Yuanbo Liu. 
(cherry picked from commit ed2d03f2a5318a88d0eced8476127f303eb1d1d9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/markdown/index.md
 2017-03-22 22:42:48 -07:00
Zhe Zhang
8ca9915d0f HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a888347f0fb9c51bb000afc986b64f98)
(cherry picked from commit f98f4bb6ca45e6f807ca747d673690232e39da52)
 2017-03-22 22:32:41 -07:00
John Zhuge
3ed867fef3 HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error. (John Zhuge via lei) 
(cherry picked from commit 15066a3e8ddacd36bf7be9fba0b10c350ae45668)
(cherry picked from commit 00dec84cc2d4513922f3f758b54b8e2db9ebd6c8)
 2017-03-21 09:14:17 -07:00
John Zhuge
3a8b120c78 HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge. 
(cherry picked from commit 84e2c11f969825e52b6bdb4696f48e4567875d41)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-03-21 08:37:18 -07:00
Steve Loughran
595501c9dd
HADOOP-14204 S3A multipart commit failing, 
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".

Contributed by Steve Loughran
 2017-03-21 13:13:16 +00:00
John Zhuge
e792f1671d HDFS-11132. Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files. Contributed by Vishwajeet Dusane 
(cherry picked from commit 19f373a46b2abb7a575f7884a9c7443b8ed67cd3)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
 2017-03-20 21:29:17 -07:00
Erik Krogen
c7b6e0dd6e HADOOP-14169. Implement listStatusIterator, listLocatedStatus for ViewFs. Contributed by Erik Krogen. 
(cherry picked from commit e1a99802fc5c2f2ad3ceb000378db1c0133069b3)
 2017-03-17 17:42:29 -07:00
Jason Lowe
6c8df1208a YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 
(cherry picked from commit 4a8e3045027036afebbcb80f23b7a2886e56c255)
 2017-03-17 14:08:14 -05:00
Kihwal Lee
66824b39fd HDFS-10601. Improve log message to include hostname when the NameNode is in safemode. Contributed by Kuhu Shukla. 
(cherry picked from commit ba62b50ebacd33b55eafc9db55a2fe5b4c80207a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-03-16 11:07:13 -05:00
Jason Lowe
5d86ce7b0d YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 
(cherry picked from commit a16ba4296e163d5cb4caed129f2f1612a69a8d84)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/Container.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/MockContainer.java
 2017-03-16 09:36:34 -05:00
Steve Loughran
72fbdf708e
HDFS-11431. hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider. 
Contributed by Steven Rand.

(cherry picked from commit b59dd4c79824280616641408b538e90f10eb49f8)
 2017-03-16 14:06:09 +00:00
Wei-Chiu Chuang
851ba7d9d1 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy. 
(cherry picked from commit 385d2cb777a0272ac20c62336c944fad295d5d12)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a1a6a8921c68f8b0f428b55152d05db)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-03-13 13:43:00 -07:00
John Zhuge
72fc7e0520 HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. 
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
(cherry picked from commit 5a40bafdaeec693e613aa02e79dbaaccfdab6f60)
 2017-03-13 09:15:09 -07:00
Yiqun Lin
1f2da39d05 HDFS-11512. Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum. Contributed by Eric Badger. 2017-03-13 18:29:57 +08:00
Mingliang Liu
b4d768123b HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 881ec4d97bd1db4582027aec3a4204156a4eda17)
 2017-03-10 13:23:17 -08:00