Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,149 Commits 326 Branches 371 Tags
Commit Graph

12149 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Steve Loughran
1f92a9dbae HADOOP-14113. Review ADL Docs. Contributed by Steve Loughran 
(cherry picked from commit e60c6543d57611039b0438d5dcb4cb19ee239bb6)
 2017-03-10 13:23:11 -08:00
Jason Lowe
dd7c688a94 YARN-6321. TestResources test timeouts are too aggressive. Contributed by Eric Badger 
(cherry picked from commit 9649c27864a23ea156bae904368c1d3cf94c6e9d)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/resource/TestResources.java
 2017-03-10 13:09:20 -06:00
Jason Lowe
5c14eb6f18 YARN-6310. OutputStreams in AggregatedLogFormat.LogWriter can be left open upon exceptions. Contributed by Haibo Chen 
(cherry picked from commit deb9f569465bb760e661e60a313dad1605635236)
 2017-03-10 11:09:39 -06:00
Mingliang Liu
026ae87236 HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 
(cherry picked from commit fd26783aaf3deea7a4e197439bd1075a6689681f)
 2017-03-10 00:22:16 -08:00
John Zhuge
843fa685a6 HADOOP-14123. Remove misplaced ADL service provider config file for FileSystem. Contributed by John Zhuge. 
Change-Id: Ic956e2eb8189625916442eaffdc69163d32f730e
(cherry picked from commit c5ee7fded46dcb1ac1ea4c1ada4949c50bc89afb)
 2017-03-09 19:54:19 -08:00
Mingliang Liu
518705b8e3 HADOOP-14017. User friendly name for ADLS user and group. Contributed by Vishwajeet Dusane 
(cherry picked from commit 924def78544a64449785f305cb6984c3559aea4d)
(cherry picked from commit acf20c8318270eaa8fdf6259ec7b07344e3bd7e5)
 2017-03-09 15:21:33 -08:00
Mingliang Liu
f0df481c76 HADOOP-14049. Honour AclBit flag associated to file/folder permission for Azure datalake account. Contributed by Vishwajeet Dusane 
(cherry picked from commit f4329990250bed62efdebe3ce2bc740092cf9573)
(cherry picked from commit a14686680228d13386ccde7272e3b5bdabe7c792)
 2017-03-09 15:21:32 -08:00
Lei Xu
674b9cdab1 HADOOP-13929. ADLS connector should not check in contract-test-options.xml. (John Zhuge via lei) 
(cherry picked from commit 71c23c9fc94cfdf58de80effbc3f51c0925d0cfe)
(cherry picked from commit ed4388f903d34a2c09845d200e1e717397b217e1)
 2017-03-09 15:21:32 -08:00
Lei Xu
87abe01ab1 HADOOP-13962. Update ADLS SDK to 2.1.4. (John Zhuge via lei) 
(cherry picked from commit ccf2d662443fc169835d66b715441529658b245c)
(cherry picked from commit d3a6124ffaa76f0aeaad1a3c58e005a65f507a10)
 2017-03-09 15:21:32 -08:00
Lei Xu
5eaec560d9 HADOOP-13956. Read ADLS credentials from Credential Provider. (John Zhuge via lei) 
(cherry picked from commit e015b563197a475e354bf84fd27e7bbcc67e00a4)
(cherry picked from commit 01624f0c1b7eba9fe4b5778e1da9d38545af3722)
 2017-03-09 15:21:32 -08:00
Mingliang Liu
8a51e4f89d HADOOP-13900. Remove snapshot version of SDK dependency from Azure Data Lake Store File System. Contributed by Vishwajeet Dusane 
(cherry picked from commit ef34bf2bb92a4e8def6617b185ae72db81450de8)
(cherry picked from commit 881b63499e10b808b3c58ca34e8d4ad5a6634ed5)
 2017-03-09 15:21:31 -08:00
Mingliang Liu
f1671e9519 HADOOP-13257. Improve Azure Data Lake contract tests. Contributed by Vishwajeet Dusane 
(cherry picked from commit 4113ec5fa5ca049ebaba039b1faf3911c6a34f7b)
(cherry picked from commit 7fd0556b2bcc4eb18c5301a580646a2d7d502b50)
 2017-03-09 15:21:31 -08:00
Chris Douglas
b4b4ca9199 HADOOP-13037. Refactor Azure Data Lake Store as an independent FileSystem. Contributed by Vishwajeet Dusane 
(cherry picked from commit edf149b9790a96563fe7bba289a040542c8ab8f2)
 2017-03-09 15:21:21 -08:00
Jason Lowe
ffc00c57d4 YARN-6165. Intra-queue preemption occurs even when preemption is turned off for a specific queue. Contributed by Eric Payne 
(cherry picked from commit d7762a55113a529abd6f4ecb8e6d9b0a84b56e08)
 2017-03-08 16:50:26 -06:00
John Zhuge
cdfa86cadd Revert "HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran" 
This reverts commit 53a12fa721bb431f7d481aac7d245c93efb56153.

(cherry picked from commit 98142d2f722e82d57b0e2bae6276f7c17fd99598)

Change-Id: I3a7613a7f4f0e9c80bb6d3c40a53742f807fc689
 2017-03-08 11:47:36 -08:00
Jian He
2e8e51ad40 HADOOP-14062. ApplicationMasterProtocolPBClientImpl.allocate fails with EOFException when RPC privacy is enabled. Contributed by Steven Rand 2017-03-08 10:49:42 -08:00
Akira Ajisaka
20fa7664d1
HADOOP-14087. S3A typo in pom.xml test exclusions. Contributed by Aaron Fabbri. 
(cherry picked from commit f597f4c43e0a6e2304b9bcaf727d6d8d15a365f9)
(cherry picked from commit d2d0872f4d109b09cdc3d8e9b163544e56469621)
 2017-03-07 15:17:42 +09:00
Akira Ajisaka
dd5d0103df
Treat encrypted files as private. Contributed by Daniel Templeton. 
(cherry picked from commit f01a69f84f4cc7d925d078a7ce32e5800da4e429)
(cherry picked from commit 120f680318d766f44787fd0eec88270d61172523)
 2017-03-07 13:38:51 +09:00
Akira Ajisaka
8b5307c0e7
MAPREDUCE-6855. Specify charset when create String in CredentialsTestJob. Contributed by Kai Sasaki. 
(cherry picked from commit 14413989cac9acc1fa6f8d330fac32f772613325)
(cherry picked from commit aa1f944bba02dcb9a14d9d37cfc113845d437ffa)
 2017-03-07 13:13:52 +09:00
Mingliang Liu
ff706261be HADOOP-14048. REDO operation of WASB#AtomicRename should create placeholder blob for destination folder. Contributed by NITIN VERMA 
(cherry picked from commit c571cda5c7d929477961dfff4176d7de4944d874)
 2017-03-06 17:01:26 -08:00
Andrew Wang
3f23d50c9a HDFS-11498. Make RestCsrfPreventionHandler and WebHdfsHandler compatible with Netty 4.0. 
(cherry picked from commit 5e74196ede9bfc20eb6d6fe3aa6a0e5c47a40fdd)
(cherry picked from commit 6e75c1e2f04be03410dd942df4dc3e94bf7b4b24)
 2017-03-06 15:04:34 -08:00
Xiao Chen
83f1e3b10a Revert "HDFS-8377. Support HTTP/2 in datanode. Contributed by Duo Zhang." 
This reverts commit f70b9d92413940739ed6f6cd468e6d09702939d1.

(cherry picked from commit 67d2875f8fe83cdeb621856031eb0892a65ec946)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2017-03-06 14:59:07 -08:00
Andrew Wang
2bfa57488b HDFS-11441. Add escaping to error message in KMS web UI. Contributed by Aaron T. Myers. 
(cherry picked from commit ec839b94c0eb3f09e74f8a3b0bc9a08b3f5418b2)
(cherry picked from commit cf5a1390aac7189cffbe1b96accb6a626a2d4a8f)
 2017-03-06 10:47:39 -08:00
Daniel Templeton
87432ab13c HADOOP-14026. start-build-env.sh: invalid docker image name (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit 6757329c43c1d6586c135b5aa00f144dd0d8d2f1)
(cherry picked from commit c8b96296b26fab1b2637ee70676710c1560e96b5)
 2017-03-04 11:04:53 -08:00
Jian He
d13a5ddc16 YARN-6270. WebUtils.getRMWebAppURLWithScheme() needs to honor RM HA setting. Contributed by Xuan Gong 2017-03-03 15:24:40 -08:00
Jason Lowe
6c48440627 YARN-6263. NMTokenSecretManagerInRM.createAndGetNMToken is not thread safe. Contributed by Haibo Chen 
(cherry picked from commit e58fc7603053e3ac1bc2464f9622995017db5245)
 2017-03-03 08:24:20 -06:00
Jason Lowe
daf3b5ee67 YARN-6274. Documentation refers to incorrect nodemanager health checker interval property. Contributed by WeiWei Yang 
(cherry picked from commit 05237636d3a359dc502e8fa2d66bfdcea274f86c)
 2017-03-03 08:02:31 -06:00
Akira Ajisaka
c6bf43e0be MAPREDUCE-6753. Variable in byte printed directly in mapreduce client. Contributed by Kai Sasaki. 
(cherry picked from commit 19a1fc6373f0dea56e5845f19990d469428489bc)
(cherry picked from commit e2fb9538f0e2855d757bcbd586c37278824700ea)
 2017-03-03 16:14:03 +09:00
Steve Loughran
2acc4d913e
HADOOP-14138. Remove S3A ref from META-INF service discovery, rely on existing core-default entry. 
Contributed by Steve Loughran

(cherry picked from commit 51fefbe38bf40373d2855163fa6444821b0f994c)
 2017-03-02 21:43:05 +00:00
Brahma Reddy Battula
31bec2c16b HDFS-11432. Federation : Support fully qualified path for Quota/Snapshot/cacheadmin/cryptoadmin commands. Contributed by Brahma Reddy Battula. 
(cherry picked from commit dcd03df9f9e0080d7e179060ffc8148336c31b3e)
 2017-03-01 10:49:36 +05:30
Zhe Zhang
265ddb20c3 HDFS-11466. Change dfs.namenode.write-lock-reporting-threshold-ms default from 1000ms to 5000ms. Contributed by Andrew Wang. 
(cherry picked from commit d269b488a71a158d3ddcbdea96992abe29725c69)
(cherry picked from commit 5fb785e154b429beb0b2e392c39be91dae65bb5c)
 2017-02-28 09:04:07 -08:00
Akira Ajisaka
6833b4ea30 MAPREDUCE-6841. Fix dead link in MapReduce tutorial document. Contributed by Victor Nee. 
(cherry picked from commit 4d33683882cc41939c512db221fd8ee59ecd52a8)
(cherry picked from commit 69f2ae449b28530d01b73a172f2be6c256dca800)
 2017-02-27 19:47:35 +09:00
Wei-Chiu Chuang
295e8685dd HDFS-8498. Blocks can be committed with wrong size. Contributed by Jing Zhao. 
(cherry picked from commit f3cdf29af4c67a1963f51f02bf88075bf6dce679)
 2017-02-25 21:15:07 -08:00
Steve Loughran
05ab09ca87
HADOOP-14028. S3A BlockOutputStreams doesn't delete temporary files in multipart uploads or handle part upload failures. 
Contributed by Steve Loughran.
 2017-02-25 16:10:15 +00:00
Steve Loughran
ff87ca8441
HADOOP-14114 S3A can no longer handle unencoded + in URIs. Contributed by Sean Mackrory. 2017-02-24 10:35:21 +00:00
Arpit Agarwal
6b6b3eba66 HADOOP-14100. Upgrade Jsch jar to latest version to fix vulnerability in old versions. Contributed by Vinayakumar B 2017-02-23 14:30:46 -08:00
Karthik Kambatla
1d156c8758 YARN-6175. FairScheduler: Negative vcore for resource needed to preempt. (Yufei Gu via kasha) 2017-02-23 14:29:30 -08:00
Mingliang Liu
b1ec790662 HADOOP-14091. AbstractFileSystem implementaion for 'wasbs' scheme. Contributed Varada Hemeswari. 
(cherry picked from commit 82607fce39151fc6ba5bced738088e2bc176dc77)
 2017-02-23 14:11:34 -08:00
Wangda Tan
920847edce YARN-6143. Fix incompatible issue caused by YARN-3583. (Sunil G via wangda) 
(cherry picked from commit 0566b30875dfc61dbbba4cde0d42e15b116e85b8)
 2017-02-22 11:21:57 -08:00
Steve Loughran
bb4b2f641c HADOOP-13826. S3A Deadlock in multipart copy due to thread pool limits. Contributed by Sean Mackrory. 
(cherry picked from commit e3a74e0369e6e2217d1280179b390227fe1b1684)
 2017-02-21 18:29:46 +00:00
Eric Payne
10d54eb978 HDFS-11404. Increase timeout on TestShortCircuitLocalRead.testDeprecatedGetBlockLocalPathInfoRpc. Contributed by Eric Badger 
(cherry picked from commit a07ddef10115fd0082832f1c338b2484507a8f49)
 2017-02-21 12:13:58 -06:00
Vinayakumar B
2b3e8b75c4 HDFS-11177. 'storagepolicies -getStoragePolicy' command should accept URI based path. (Contributed by Surendra Singh Lilhore) 
(cherry picked from commit 480405063063f564ae0cdb34e0757ac3990569aa)
(cherry picked from commit 00ca9f1792b8d2d572559b656ffd8ad7453ef319)
 2017-02-21 18:17:38 +05:30
Steve Loughran
8a05ea4aba HADOOP-14081. S3A: Consider avoiding array copy in S3ABlockOutputStream (ByteArrayBlock). Contributed by Rajesh Balamohan 2017-02-20 16:21:12 +00:00
Steve Loughran
b1c1f05b1d HADOOP-14092. Typo in hadoop-aws index.md. Contributed by John Zhuge 2017-02-18 18:16:19 +00:00
Mingliang Liu
3b4f840ccb HADOOP-14019. Fix some typos in the s3a docs. Contributed by Steve Loughran 
(cherry picked from commit bdad8b7b97d7f48119f016d68f32982d680c8796)
 2017-02-16 16:43:22 -08:00
Li Lu
83a5896180 YARN-6177. Yarn client should exit with an informative error message if an incompatible Jersey library is used at client. Contributed by Weiwei Yang. 
(cherry picked from commit 5d339c46f5b16b951afd82afd9e907b9aa2ded9a)
 2017-02-16 13:50:22 -08:00
Rohith Sharma K S
da944fe0f4 YARN-6200. Reverting since the same functionality achieved by YARN-1623. 
Revert "YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks)"

This reverts commit f102c4f6c8b5a38f73e41282215e9bc38240eed8.
 2017-02-16 17:54:37 +05:30
Wei-Chiu Chuang
4c47cb68e8 HDFS-11333. Print a user friendly error message when plugins are not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 859bd159ae554174200334b5eb1d7e8dbef958ad)
(cherry picked from commit a8531d5d52fb219c2adb5b6025cfc3fad264e8aa)
 2017-02-15 03:02:41 -08:00
Yongjun Zhang
5bb1350922 HDFS-11391. Numeric usernames do no work with WebHDFS FS write access. (Pierre Villard via Yongjun Zhang) 
(cherry picked from commit 8e53f2b9b08560bf4f8e81e697063277dbdc68f9)
 2017-02-14 14:14:58 -08:00
Xiao Chen
8f9aae0193 HDFS-11390. Add process name to httpfs process. Contributed by Weiwei Yang. 
(cherry picked from commit 92dbdbc490a527054388dc2fc6751aa8268856ef)
 2017-02-14 09:47:41 -08:00