Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,114 Commits 326 Branches 371 Tags
Commit Graph

12114 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
John Zhuge
be58711e91 HDFS-11561. HttpFS doc errors. Contributed by Yuanbo Liu. 
(cherry picked from commit ed2d03f2a5318a88d0eced8476127f303eb1d1d9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-httpfs/src/site/markdown/index.md
 2017-03-22 22:42:48 -07:00
Zhe Zhang
8ca9915d0f HADOOP-9631. ViewFs should use underlying FileSystem's server side defaults. Contributed by Lohit Vijayarenu and Erik Krogen. 
(cherry picked from commit 59d69257a888347f0fb9c51bb000afc986b64f98)
(cherry picked from commit f98f4bb6ca45e6f807ca747d673690232e39da52)
 2017-03-22 22:32:41 -07:00
John Zhuge
3ed867fef3 HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error. (John Zhuge via lei) 
(cherry picked from commit 15066a3e8ddacd36bf7be9fba0b10c350ae45668)
(cherry picked from commit 00dec84cc2d4513922f3f758b54b8e2db9ebd6c8)
 2017-03-21 09:14:17 -07:00
John Zhuge
3a8b120c78 HADOOP-14205. No FileSystem for scheme: adl. Contributed by John Zhuge. 
(cherry picked from commit 84e2c11f969825e52b6bdb4696f48e4567875d41)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-03-21 08:37:18 -07:00
Steve Loughran
595501c9dd
HADOOP-14204 S3A multipart commit failing, 
"UnsupportedOperationException at java.util.Collections$UnmodifiableList.sort".

Contributed by Steve Loughran
 2017-03-21 13:13:16 +00:00
John Zhuge
e792f1671d HDFS-11132. Allow AccessControlException in contract tests when getFileStatus on subdirectory of existing files. Contributed by Vishwajeet Dusane 
(cherry picked from commit 19f373a46b2abb7a575f7884a9c7443b8ed67cd3)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java
 2017-03-20 21:29:17 -07:00
Erik Krogen
c7b6e0dd6e HADOOP-14169. Implement listStatusIterator, listLocatedStatus for ViewFs. Contributed by Erik Krogen. 
(cherry picked from commit e1a99802fc5c2f2ad3ceb000378db1c0133069b3)
 2017-03-17 17:42:29 -07:00
Jason Lowe
6c8df1208a YARN-6217. TestLocalCacheDirectoryManager test timeout is too aggressive. Contributed by Miklos Szegedi 
(cherry picked from commit 4a8e3045027036afebbcb80f23b7a2886e56c255)
 2017-03-17 14:08:14 -05:00
Kihwal Lee
66824b39fd HDFS-10601. Improve log message to include hostname when the NameNode is in safemode. Contributed by Kuhu Shukla. 
(cherry picked from commit ba62b50ebacd33b55eafc9db55a2fe5b4c80207a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-03-16 11:07:13 -05:00
Jason Lowe
5d86ce7b0d YARN-4051. ContainerKillEvent lost when container is still recovering and application finishes. Contributed by sandflee 
(cherry picked from commit a16ba4296e163d5cb4caed129f2f1612a69a8d84)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/Container.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/webapp/MockContainer.java
 2017-03-16 09:36:34 -05:00
Steve Loughran
72fbdf708e
HDFS-11431. hadoop-hdfs-client JAR does not include ConfiguredFailoverProxyProvider. 
Contributed by Steven Rand.

(cherry picked from commit b59dd4c79824280616641408b538e90f10eb49f8)
 2017-03-16 14:06:09 +00:00
Wei-Chiu Chuang
851ba7d9d1 HDFS-11499. Decommissioning stuck because of failing recovery. Contributed by Lukas Majercak and Manoj Govindassamy. 
(cherry picked from commit 385d2cb777a0272ac20c62336c944fad295d5d12)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 60be2e5d8a1a6a8921c68f8b0f428b55152d05db)

 Conflicts:
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2017-03-13 13:43:00 -07:00
John Zhuge
72fc7e0520 HADOOP-14173. Remove unused AdlConfKeys#ADL_EVENTS_TRACKING_SOURCE. Contributed by John Zhuge. 
Change-Id: I5dc6f885816b8834f718874542dfa373458b0333
(cherry picked from commit 5a40bafdaeec693e613aa02e79dbaaccfdab6f60)
 2017-03-13 09:15:09 -07:00
Yiqun Lin
1f2da39d05 HDFS-11512. Increase timeout on TestShortCircuitLocalRead#testSkipWithVerifyChecksum. Contributed by Eric Badger. 2017-03-13 18:29:57 +08:00
Mingliang Liu
b4d768123b HADOOP-14153. ADL module has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 881ec4d97bd1db4582027aec3a4204156a4eda17)
 2017-03-10 13:23:17 -08:00
Steve Loughran
1f92a9dbae HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran 
(cherry picked from commit e60c6543d57611039b0438d5dcb4cb19ee239bb6)
 2017-03-10 13:23:11 -08:00
Jason Lowe
dd7c688a94 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 
(cherry picked from commit 9649c27864a23ea156bae904368c1d3cf94c6e9d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/resource/TestResources.java
 2017-03-10 13:09:20 -06:00
Jason Lowe
5c14eb6f18 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 
(cherry picked from commit deb9f569465bb760e661e60a313dad1605635236)
 2017-03-10 11:09:39 -06:00
Mingliang Liu
026ae87236 HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 
(cherry picked from commit fd26783aaf3deea7a4e197439bd1075a6689681f)
 2017-03-10 00:22:16 -08:00
John Zhuge
843fa685a6 HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. 
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
(cherry picked from commit c5ee7fded46dcb1ac1ea4c1ada4949c50bc89afb)
 2017-03-09 19:54:19 -08:00
Mingliang Liu
518705b8e3 HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane 
(cherry picked from commit 924def78544a64449785f305cb6984c3559aea4d)
(cherry picked from commit acf20c8318270eaa8fdf6259ec7b07344e3bd7e5)
 2017-03-09 15:21:33 -08:00
Mingliang Liu
f0df481c76 HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane 
(cherry picked from commit f4329990250bed62efdebe3ce2bc740092cf9573)
(cherry picked from commit a14686680228d13386ccde7272e3b5bdabe7c792)
 2017-03-09 15:21:32 -08:00
Lei Xu
674b9cdab1 HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei) 
(cherry picked from commit 71c23c9fc94cfdf58de80effbc3f51c0925d0cfe)
(cherry picked from commit ed4388f903d34a2c09845d200e1e717397b217e1)
 2017-03-09 15:21:32 -08:00
Lei Xu
87abe01ab1 HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei) 
(cherry picked from commit ccf2d662443fc169835d66b715441529658b245c)
(cherry picked from commit d3a6124ffaa76f0aeaad1a3c58e005a65f507a10)
 2017-03-09 15:21:32 -08:00
Lei Xu
5eaec560d9 HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei) 
(cherry picked from commit e015b563197a475e354bf84fd27e7bbcc67e00a4)
(cherry picked from commit 01624f0c1b7eba9fe4b5778e1da9d38545af3722)
 2017-03-09 15:21:32 -08:00
Mingliang Liu
8a51e4f89d HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane 
(cherry picked from commit ef34bf2bb92a4e8def6617b185ae72db81450de8)
(cherry picked from commit 881b63499e10b808b3c58ca34e8d4ad5a6634ed5)
 2017-03-09 15:21:31 -08:00
Mingliang Liu
f1671e9519 HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane 
(cherry picked from commit 4113ec5fa5ca049ebaba039b1faf3911c6a34f7b)
(cherry picked from commit 7fd0556b2bcc4eb18c5301a580646a2d7d502b50)
 2017-03-09 15:21:31 -08:00
Chris Douglas
b4b4ca9199 HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane 
(cherry picked from commit edf149b9790a96563fe7bba289a040542c8ab8f2)
 2017-03-09 15:21:21 -08:00
Jason Lowe
ffc00c57d4 YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)
 2017-03-08 16:50:26 -06:00
John Zhuge
cdfa86cadd Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" 
This reverts commit 53a12fa721bb431f7d481aac7d245c93efb56153.

(cherry picked from commit 98142d2f722e82d57b0e2bae6276f7c17fd99598)

Change-Id: I3a7613a7f4f0e9c80bb6d3c40a53742f807fc689
 2017-03-08 11:47:36 -08:00
Jian He
2e8e51ad40 HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:49:42 -08:00
Akira Ajisaka
20fa7664d1
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri. 
(cherry picked from commit f597f4c43e0a6e2304b9bcaf727d6d8d15a365f9)
(cherry picked from commit d2d0872f4d109b09cdc3d8e9b163544e56469621)
 2017-03-07 15:17:42 +09:00
Akira Ajisaka
dd5d0103df
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f4cc7d925d078a7ce32e5800da4e429)
(cherry picked from commit 120f680318d766f44787fd0eec88270d61172523)
 2017-03-07 13:38:51 +09:00
Akira Ajisaka
8b5307c0e7
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989cac9acc1fa6f8d330fac32f772613325)
(cherry picked from commit aa1f944bba02dcb9a14d9d37cfc113845d437ffa)
 2017-03-07 13:13:52 +09:00
Mingliang Liu
ff706261be HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA 
(cherry picked from commit c571cda5c7d929477961dfff4176d7de4944d874)
 2017-03-06 17:01:26 -08:00
Andrew Wang
3f23d50c9a HDFS-11498. Make RestCsrfPreventionHandler and WebHdfsHandler compatible with Netty 4.0. 
(cherry picked from commit 5e74196ede9bfc20eb6d6fe3aa6a0e5c47a40fdd)
(cherry picked from commit 6e75c1e2f04be03410dd942df4dc3e94bf7b4b24)
 2017-03-06 15:04:34 -08:00
Xiao Chen
83f1e3b10a Revert "HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang." 
This reverts commit f70b9d92413940739ed6f6cd468e6d09702939d1.

(cherry picked from commit 67d2875f8fe83cdeb621856031eb0892a65ec946)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2017-03-06 14:59:07 -08:00
Andrew Wang
2bfa57488b HDFS-11441. Add escaping to error message in KMS web UI. Contributed by Aaron T. Myers. 
(cherry picked from commit ec839b94c0eb3f09e74f8a3b0bc9a08b3f5418b2)
(cherry picked from commit cf5a1390aac7189cffbe1b96accb6a626a2d4a8f)
 2017-03-06 10:47:39 -08:00
Daniel Templeton
87432ab13c HADOOP-14026. start-build-env.sh: invalid docker image name (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit 6757329c43c1d6586c135b5aa00f144dd0d8d2f1)
(cherry picked from commit c8b96296b26fab1b2637ee70676710c1560e96b5)
 2017-03-04 11:04:53 -08:00
Jian He
d13a5ddc16 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:40 -08:00
Jason Lowe
6c48440627 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc7603053e3ac1bc2464f9622995017db5245)
 2017-03-03 08:24:20 -06:00
Jason Lowe
daf3b5ee67 YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang 
(cherry picked from commit 05237636d3a359dc502e8fa2d66bfdcea274f86c)
 2017-03-03 08:02:31 -06:00
Akira Ajisaka
c6bf43e0be MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373f0dea56e5845f19990d469428489bc)
(cherry picked from commit e2fb9538f0e2855d757bcbd586c37278824700ea)
 2017-03-03 16:14:03 +09:00
Steve Loughran
2acc4d913e
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran

(cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)
 2017-03-02 21:43:05 +00:00
Brahma Reddy Battula
31bec2c16b HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula. 
(cherry picked from commit dcd03df9f9e0080d7e179060ffc8148336c31b3e)
 2017-03-01 10:49:36 +05:30
Zhe Zhang
265ddb20c3 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang. 
(cherry picked from commit d269b488a71a158d3ddcbdea96992abe29725c69)
(cherry picked from commit 5fb785e154b429beb0b2e392c39be91dae65bb5c)
 2017-02-28 09:04:07 -08:00
Akira Ajisaka
6833b4ea30 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882cc41939c512db221fd8ee59ecd52a8)
(cherry picked from commit 69f2ae449b28530d01b73a172f2be6c256dca800)
 2017-02-27 19:47:35 +09:00
Wei-Chiu Chuang
295e8685dd HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. 
(cherry picked from commit f3cdf29af4c67a1963f51f02bf88075bf6dce679)
 2017-02-25 21:15:07 -08:00
Steve Loughran
05ab09ca87
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. 
Contributed by Steve Loughran.
 2017-02-25 16:10:15 +00:00
Steve Loughran
ff87ca8441
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. 2017-02-24 10:35:21 +00:00