Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,515 Commits 326 Branches 371 Tags
Commit Graph

11515 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
8657dece8b YARN-5389. TestYarnClient#testReservationDelete fails. Contributed by Sean Po 
(cherry picked from commit 4037b22b7cb2755df17a65b03f6c0b9f63a476cd)
 2016-08-25 15:11:47 +00:00
Wei-Chiu Chuang
bd729af8a8 HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-24 08:53:43 -07:00
Jason Lowe
00c8ab91ba MAPREDUCE-6761. Regression when handling providers - invalid configuration ServiceConfiguration causes Cluster initialization failure. Contributed by Peter Vary 
(cherry picked from commit 6fa9bf4407a0b09be8ce4e52d2fde89adba34f1a)
 2016-08-24 14:42:53 +00:00
Akira Ajisaka
f871aed87e HADOOP-13538. Deprecate getInstance and initialize methods with Path in TrashPolicy. Contributed by Yiqun Lin. 
(cherry picked from commit 092b4d5bfd02131d62723fc5673892305eb9fcef)
(cherry picked from commit 2321b73037fcf7ea9ee60c1a94838233c3df8908)
 2016-08-24 16:50:28 +09:00
Zhe Zhang
8bc33bf343 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. Branch-2 patch contributed by Wei-Chiu Chuang. 
(cherry picked from commit dfcbc12026b591745a7d7279f2b840152cb53a91)
 2016-08-23 14:46:47 -07:00
Masatake Iwasaki
80d8e79e26 HADOOP-13497. fix wrong command in CredentialProviderAPI.md. Contributed by Yuanbo Liu. 
(cherry picked from commit 8aae8d6bf03ade0607547ed461dc99a336a7e9d4)
 2016-08-24 00:10:37 +09:00
Jason Lowe
f57a26378e HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen 
(cherry picked from commit 126d165efd80e266a8309241f3cf059e358f5019)
 2016-08-23 14:49:27 +00:00
Jian He
5ae8b82f37 HADOOP-13524. mvn eclipse:eclipse generates .gitignore'able files. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit dd76238a3bafd58faa6f38f075505bef1012f150)
 2016-08-23 19:14:19 +08:00
Akira Ajisaka
dde3eaed62 HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin. 
(cherry picked from commit e90f3359de299ef5e3a54ca71070e3dfe1dbb98c)
(cherry picked from commit 94a844b4f61d4a47b2a5885e3db065a07ed29d97)
 2016-08-23 19:59:07 +09:00
Rohith Sharma K S
71a3b01d51 YARN-5544. TestNodeBlacklistingOnAMFailures fails on trunk. Contributed by Sunil G. 
(cherry picked from commit 0d5997d2b98eb89e72828dfcd78f02aa4e7e1e67)
 2016-08-23 14:40:05 +05:30
Akira Ajisaka
854af9cee1 MAPREDUCE-6587. Remove unused params in connection-related methods of Fetcher. Contributed by Yiqun Lin. 
(cherry picked from commit 8cc4a67059e37b2083cd5468b35a64a403a3e3ae)
(cherry picked from commit 41fe85f8688e6edd2a94ffd831d776e7c7a3614f)
 2016-08-23 17:12:32 +09:00
Xiao Chen
0f9bed050d HADOOP-13487. Hadoop KMS should load old delegation tokens from Zookeeper on startup. Contributed by Xiao Chen. 
(cherry picked from commit 2f21d53e3aaf97f425f954f52b50f36dddb6b886)
 2016-08-22 14:42:20 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
c0c4cc8ee8 HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan. 
(cherry picked from commit dc7a1c54f4447fb01980ae61d5d67e90c4f52f00)
 2016-08-22 11:37:14 -07:00
Varun Saxena
e714d82196 MAPREDUCE-6750. Fix TestHSAdminServer#testRefreshSuperUserGroups (Kihwal Lee via Varun Saxena) 2016-08-22 11:05:01 -05:00
Kihwal Lee
c9199095f8 HDFS-10738. Fix TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration test failure. Contributed by Rakesh R. 
(cherry picked from commit 0f701f433dd3be233bf53e856864c82349e8274e)
(cherry picked from commit 7f1879abe642219fce0328a8e96c983601d202b0)
 2016-08-22 11:04:03 -05:00
Mingliang Liu
9d304822f6 HADOOP-13527. Add Spark to CallerContext LimitedPrivate scope. (Contributed by Weiqing Yang) 
(cherry picked from commit 115ecb52a86b49aad3d058a6b4c1c7926b8b0a40)
 2016-08-21 09:42:06 -07:00
Jing Zhao
8628ad0790 HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal. 
(cherry picked from commit 0faee62a0c8c1b8fd83227babfd00fbc2b26bddf)
(cherry picked from commit fff0418458b23c3096384082f8f387b451efad09)
 2016-08-19 22:27:24 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
b2efb2040c HADOOP-13428. Fixed hadoop-common build files to generate jdiff by working around a jdiff-bug. Contributed by Wangda Tan. 
(cherry picked from commit 99603e902244f17b04cfd55122f47355d070b588)
 2016-08-19 19:11:04 -07:00
Wangda Tan
2e9d84d24c YARN-4685. Disable AM blacklisting by default to mitigate situations that application get hanged. (Rohith Sharma K S via wangda) 
(cherry picked from commit 2da32a6ef9edebd86ca9672d10ce35b5a46818cc)
(cherry picked from commit 3e511f39c220ef65708125a37e630c495c450ad1)
 2016-08-19 16:55:40 -07:00
Wangda Tan
9bf420b3bc YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-08-19 16:55:34 -07:00
Wangda Tan
3bf2e16f76 YARN-3388. Allocation in LeafQueue could get stuck because DRF calculator isn't well supported when computing user-limit. (Nathan Roberts via wangda) 
(cherry picked from commit 444b2ea7afebf9f6c3d356154b71abfd0ea95b23)
(cherry picked from commit c7d782d2f6e61103f7a009c48b11dcd64e07c2ce)
 2016-08-19 16:54:03 -07:00
Wangda Tan
e2bd490236 MAPREDUCE-6310. Add jdiff support to MapReduce. (Li Lu/vinodkv via wangda) 
(cherry picked from commit 3d937457ee60d617d71b36bb6a29344e771670b4)
(cherry picked from commit 65822e55fd99511ab0c39f802d3ec58290445f9d)
 2016-08-19 16:49:54 -07:00
Varun Saxena
6f1a9f1b6a MAPREDUCE-6763. Shuffle server listen queue is too small (Jason Lowe via Varun Saxena) 2016-08-20 04:08:55 +05:30
Kihwal Lee
4f3df8e22a HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee) 
(cherry picked from commit ae90d4dd908cf3f9e9ff26fa8e92f028057a9ca1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2016-08-19 15:32:11 -05:00
Chris Nauroth
dd6c149c4a HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran. 
(cherry picked from commit 763f0497bb996e331e40caed9ca0af966f5b3fac)
(cherry picked from commit 6dba38f08ba84109b01c8ce8d30c71354d089dbe)
 2016-08-19 12:55:57 -07:00
Jing Zhao
5d91f9fe35 HADOOP-13503. Improve SaslRpcClient failure logging. Contributed by Xiaobing Zhou. 
(cherry picked from commit 9fdd1ea72497f7a8aadbfedeba36688e0c16c52e)
 2016-08-18 15:00:17 -07:00
Mingliang Liu
ef38aa64c6 HADOOP-13512. ReloadingX509TrustManager should keep reloading in case of exception. (Contributed by Mingliang Liu) 
(cherry picked from commit 0f51eae0c085ded38216824377acf8122638c3a5)
 2016-08-18 14:22:58 -07:00
Varun Vasudev
5b49bcaee3 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382e1992ea2b8ccf3130edaaa751c644d)
(cherry picked from commit 99966b1912320326fb63be1ce6d4222d736ca365)
 2016-08-18 12:33:52 -07:00
Karthik Kambatla
42a88f9c58 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c42170ee3dab2f6fb09a94bbf72fc65)
(cherry picked from commit f0923819c35b8c499a9bcd0c6c78511f3a400fef)
 2016-08-18 12:30:39 -07:00
Chris Nauroth
1c4e039ba7 HADOOP-13513. Java 1.7 support for org.apache.hadoop.fs.azure testcases. Contributed by Tibor Kiss. 
(cherry picked from commit ae4db2544346370404826d5b55b2678f5f92fe1f)
(cherry picked from commit 804a40720bd5cf04415831b7bb33e98d11b2046e)
 2016-08-18 10:45:20 -07:00
Steve Loughran
c09797ce9f HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie 2016-08-18 14:35:26 +01:00
Arpit Agarwal
1fe08c919a HDFS-10682. Replace FsDatasetImpl object lock with a separate lock object. (Contributed by Chen Liang) 2016-08-17 16:22:07 -07:00
Xiao Chen
b89d79ca1d HDFS-10549. Correctly revoke file leases when closing files. Contributed by Yiqun Lin. 
(cherry picked from commit 85aacaadb5a3f8c78b191867c0bde09b3c4b3c3c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSStripedOutputStream.java

(cherry picked from commit a20b943cf951cf38eb4950177c826bbcf424aade)
 2016-08-17 16:00:33 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
d65024eddc Revert "HDFS-9395. Make HDFS audit logging consistant. Contributed by Kuhu Shukla." 
Reverting this on branch-2.* as it's an incompatible change.

This reverts commit 83f7f62be379045ad6933689b21b76c7086f919d.

(cherry picked from commit 2486c4c63a35fcef7338ea63f0d8aafa778cd05d)
 2016-08-17 13:34:35 -07:00
Mingliang Liu
00eb790622 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-16 17:41:51 -07:00
Chris Nauroth
ac779f2554 HADOOP-13324. s3a tests don't authenticate with S3 frankfurt (or other V4 auth only endpoints). Contributed by Steve Loughran. 
(cherry picked from commit 3808876c7397ea68906bc5cc18fdf690c9c42131)
(cherry picked from commit 555a57adf2fa5f2d50c1141da907ae4ca032db2c)
 2016-08-16 17:11:12 -07:00
Mingliang Liu
b6d92c61fe Revert "HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu)" 
This reverts commit 657064e59397feb92cec95094ac34c85e3040c50.
 2016-08-16 16:27:36 -07:00
Andrew Wang
8b6eeb2595 HADOOP-13494. ReconfigurableBase can log sensitive information. Contributed by Sean Mackrory. 
(cherry picked from commit 4b689e7a758a55cec2ca8398727feefc8ac21bfd)
(cherry picked from commit c91e6f28283d1f8ad4504a2338a01e284ced6b24)

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CommonConfigurationKeysPublic.java
 2016-08-16 15:14:06 -07:00
Kihwal Lee
b76f6e5de3 HDFS-10342. BlockManager#createLocatedBlocks should not check corrupt replicas if none are corrupt. Contributed by Kuhu Shukla. 
(cherry picked from commit b10c936020e2616609dcb3b2126e8c34328c10ca)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java

(cherry picked from commit 65a41696b2a22834f4592bcb49e40fffd2dbb96e)
 2016-08-16 09:39:33 -05:00
Mingliang Liu
54c9743263 HDFS-10641. TestBlockManager#testBlockReportQueueing fails intermittently. (Contributed by Daryn Sharp) 2016-08-15 20:46:48 -07:00
Mingliang Liu
657064e593 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-15 20:46:40 -07:00
Mingliang Liu
e9eaad0e67 HDFS-10747. o.a.h.hdfs.tools.DebugAdmin usage message is misleading. (Contributed by Mingliang Liu) 2016-08-15 20:46:34 -07:00
Mingliang Liu
0b934c375e HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:46:28 -07:00
Mingliang Liu
6471ec31bc HDFS-10724. Document caller context config keys. (Contributed by Mingliang Liu) 
(cherry picked from commit 4bcbef39f7ca07601092919a7f2bea531a2dfa07)
 2016-08-15 20:46:03 -07:00
Kihwal Lee
1a8280edde HDFS-10763. Open files can leak permanently due to inconsistent lease update. Contributed by Kihwal Lee. 
(cherry picked from commit 864f878d5912c82f3204f1582cfb7eb7c9f1a1da)
(cherry picked from commit e78db7d2a430983807750666fb72ebd5c97ce867)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-08-15 17:37:39 -05:00
Varun Saxena
6fed7b9c45 HADOOP-13333. testConf.xml ls comparators in wrong order (Vrushali C via Varun Saxena) 2016-08-16 03:14:24 +05:30
Jason Lowe
74831973cb YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5201a149218276469434df511697acc)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-08-15 19:45:45 +00:00
Junping Du
1a38dd9cee YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 
(cherry picked from commit 00058167431475c6e63c80207424f1d365569e3a)
 2016-08-15 19:18:36 +00:00
Kihwal Lee
579709fc22 HDFS-9696. Garbage snapshot records linger forever. Contributed by Kihwal Lee 
(cherry picked from commit 83e57e083f2cf6c0de8a46966c5492faeabd8f2a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java

(cherry picked from commit 4766a3153dd517ac832d4761c884ed88a83a6c09)
 2016-08-15 13:13:49 -05:00
Varun Saxena
ca139a3f87 YARN-5476. Non existent application reported as ACCEPTED by YarnClientImpl (Junping Du via Varun Saxena) 2016-08-13 00:27:17 +05:30