Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,576 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

11576 Commits

Author SHA1 Message Date
Kihwal Lee 58df27b873 HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199da)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java

(cherry picked from commit 77b61d1f4e)
 2016-08-04 09:45:03 -05:00
Kihwal Lee 804db5b277 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit c0166b0743)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2016-08-04 09:35:16 -05:00
Akira Ajisaka 59693ebbd5 MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c2781152)
(cherry picked from commit e54de94e5a)
 2016-08-04 20:18:37 +09:00
Kihwal Lee ceac628378 HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang. 
(cherry picked from commit 6f63566694)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java

(cherry picked from commit 1ca2ab4013)
 2016-08-03 15:28:13 -05:00
Jason Lowe 6fc68a379b HADOOP-10980. TestActiveStandbyElector fails occasionally in trunk. Contributed by Eric Badger 
(cherry picked from commit c82745432a)
 2016-08-03 20:20:24 +00:00
Jason Lowe eef24f5d93 YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger 
(cherry picked from commit db4a61dc61)
 2016-08-03 19:54:00 +00:00
Jing Zhao 5bdee64bde HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui. 
(cherry picked from commit f4ba5ff1d7)
(cherry picked from commit 180a26d9fb)
 2016-08-03 12:44:49 -07:00
Jason Lowe 5883718eea YARN-4717. TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe)
 2016-08-03 19:33:58 +00:00
Jason Lowe c972498b74 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 
(cherry picked from commit db646540f0)
 2016-08-03 19:19:47 +00:00
Jason Lowe aca7eea611 YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 2016-08-03 19:01:14 +00:00
Wangda Tan c322e749d6 YARN-5342. Improve non-exclusive node partition resource allocation in Capacity Scheduler. (Sunil G via wangda) 2016-08-03 11:29:12 -07:00
Kihwal Lee 33a87ffe10 HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai. 
(cherry picked from commit 58db263e93)
 2016-08-03 12:19:05 -05:00
Chris Nauroth 5c0db4e522 HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth. 
(cherry picked from commit 4ef1bc0bdb)
(cherry picked from commit 8ec3d702a3)
 2016-08-02 22:32:44 -07:00
Arpit Agarwal ad653de2d7 HDFS-9353. Code and comment mismatch in JavaKeyStoreProvider. (Andras Bokor) 2016-08-02 15:18:24 -07:00
Arpit Agarwal 745ba1160b HADOOP-13434. Add bash quoting to Shell class. (Owen O'Malley) 2016-08-02 13:43:50 -07:00
Kihwal Lee 288f9ccde2 HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger. 
(cherry picked from commit 5e5b8793fb)
 2016-08-02 15:43:11 -05:00
Chris Nauroth 06df0cd87f HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin. 
(cherry picked from commit 0458a2af6e)
(cherry picked from commit ec522a1907)
 2016-08-02 12:43:58 -07:00
Konstantin V Shvachko 9b3b7703b1 Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi." 
This reverts commit 9ab153d7ae.
 2016-08-01 22:48:09 -07:00
Akira Ajisaka f39ed9ea8e HADOOP-13444. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 5469e128d3)
 2016-08-02 10:18:52 +09:00
Daryn Sharp ecb143ed21 HDFS-10655. Fix path related byte array conversion bugs. (daryn) 
(cherry picked from commit 9f473cf903)
 2016-08-01 10:18:45 -05:00
Akira Ajisaka fa0b6ff97b MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a)
(cherry picked from commit 96d5607e7a)
 2016-07-30 22:16:43 +09:00
Yongjun Zhang cc066de22c HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 8d32bd884d)
 2016-07-29 23:47:46 -07:00
Akira Ajisaka 19062df92f MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d)
(cherry picked from commit 0fc838540b)
 2016-07-30 11:47:33 +09:00
Junping Du bba06f4526 YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter. 
(cherry picked from commit 95f2b98597)
(cherry picked from commit 47dd871f96)
 2016-07-29 10:31:17 -07:00
Akira Ajisaka e467f838bb HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin. 
(cherry picked from commit 204a2055b1)
(cherry picked from commit 0cff416c35)
 2016-07-29 15:41:17 +09:00
Xiaoyu Yao ac510e5cd0 HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru. 2016-07-28 16:16:44 -07:00
Jason Lowe 95fb2fe394 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S 
(cherry picked from commit a1890c32c5)
 2016-07-28 21:49:21 +00:00
Junping Du 08358c8d4e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu. 
(cherry picked from commit 7f3c306e2e)
(cherry picked from commit 932bf79244)
 2016-07-28 06:56:00 -07:00
Akira Ajisaka 960caf624d HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki. 
(cherry picked from commit 414fbfab41)
(cherry picked from commit ab49c2b399)
 2016-07-28 16:59:49 +09:00
Yongjun Zhang 71c49b1786 HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang) 
(cherry picked from commit eb7ff0c992)
 2016-07-27 15:49:56 -07:00
Brahma Reddy Battula 5dc29ab0f2 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 096d3fa25c)
 2016-07-27 10:31:29 +05:30
Chris Nauroth 244285488d HADOOP-13422. ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process. Contributed by Sergey Shelukhin. 
(cherry picked from commit 255ea45e50)
(cherry picked from commit df2ed6b2c4)
 2016-07-26 15:33:42 -07:00
Arpit Agarwal 45a6ee97b2 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:45:06 -07:00
Wei-Chiu Chuang 53bf632702 HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki. 
(cherry picked from commit 25f0a9b3ab)
 2016-07-26 08:31:00 -07:00
Steve Loughran 7921ad949f Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu." 
This reverts commit 32468b1ba5.
 2016-07-26 14:31:02 +01:00
Masatake Iwasaki 6b5d546014 HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin. 
(cherry picked from commit 59466b8c18)
 2016-07-26 19:52:23 +09:00
Brahma Reddy Battula 32468b1ba5 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu. 
(cherry picked from commit 7cac7655fd)
 2016-07-26 11:00:11 +05:30
Akira Ajisaka e377e94f64 Revert "HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg." 
This reverts commit 91f47ab5f9.
 2016-07-26 11:06:10 +09:00
Vinitha Reddy Gankidi 9ab153d7ae HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:59:42 -07:00
Jing Zhao 62800dc8b6 HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang. 
(cherry picked from commit 0cde9e12a7)
(cherry picked from commit 30c772fc59)
 2016-07-25 18:44:06 -07:00
Jason Lowe e098a732ef MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger 
(cherry picked from commit 703fdf86c6)
 2016-07-25 21:56:33 +00:00
Arpit Agarwal 00cbd3136e HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-25 09:51:59 -07:00
Jing Zhao 54be36a6d6 HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou. 
(cherry picked from commit 7149cdb3c2d9dd390cd8668883cbe5db94090e0a)
 2016-07-25 09:51:59 -07:00
Arpit Agarwal bbf711c030 HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-07-25 09:51:59 -07:00
Steve Loughran 71c8bdd50c HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran 2016-07-25 15:48:49 +01:00
Steve Loughran 5f9bffbb66 HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan 2016-07-25 14:43:45 +01:00
Steve Loughran 7c46303de3 HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong 2016-07-23 17:58:46 +01:00
Chris Nauroth 0456069aca HADOOP-13207. Specify FileSystem listStatus, listFiles and RemoteIterator. Contributed by Steve Loughran. 
(cherry picked from commit 5fda66e4a3)
(cherry picked from commit e6ba69b4db)
 2016-07-22 15:39:12 -07:00
Xiao Chen 8559442ed1 HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 132deb4cac)
 2016-07-22 15:04:21 -07:00
mattf-horton c96cb3fd48 HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley. 2016-07-21 16:01:05 -07:00