Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,919 Commits 326 Branches 371 Tags
Commit Graph

11919 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kihwal Lee
690ec789f0 HDFS-10343. BlockManager#createLocatedBlocks may return blocks on failed storages. Contributed by Kuhu Shukla. 
(cherry picked from commit 57369a678c4c51627fe9a654e697a906a6bef123)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
 2016-08-04 15:12:21 -05:00
Arpit Agarwal
4f74b5e793 HADOOP-13466. Add an AutoCloseableLock class. (Chen Liang) 2016-08-04 12:32:23 -07:00
Kihwal Lee
6ead6f3bb8 HDFS-10717. Fix findbugs warnings of hadoop-hdfs-client in branch-2. Contributed by Akira Ajisaka. 2016-08-04 12:34:54 -05:00
Arpit Agarwal
365a27099c HADOOP-13467. Shell#getSignalKillCommand should use the bash builtin on Linux. (Arpit Agarwal) 2016-08-04 10:08:23 -07:00
Kihwal Lee
3b2554f3b0 HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
 2016-08-04 11:55:52 -05:00
Tsz-Wo Nicholas Sze
5004b0571e In Balancer, the target task should be removed when its size < 0. Contributed by Yiqun Lin 2016-08-04 09:53:15 -07:00
Kihwal Lee
c7fe1f5ec5 HADOOP-13442. Optimize UGI group lookups. Contributed by Daryn Sharp. 
(cherry picked from commit 94225152399e6e89fa7b4cff6d17d33e544329a3)
 2016-08-04 10:48:17 -05:00
Kihwal Lee
77b61d1f4e HDFS-10662. Optimize UTF8 string/byte conversions. Contributed by Daryn Sharp. 
(cherry picked from commit 6ae39199dac6ac7be6802b31452552c76da16e24)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirWriteFileOp.java
 2016-08-04 09:43:56 -05:00
Kihwal Lee
c0166b0743 HDFS-10707. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 2016-08-04 09:28:30 -05:00
Akira Ajisaka
e54de94e5a MAPREDUCE-6730. Use StandardCharsets instead of String overload in TextOutputFormat. Contributed by Sahil Kang. 
This closes #114

(cherry picked from commit 70c278115249898132490a89a548fd936c09f54b)
 2016-08-04 20:18:13 +09:00
Varun Vasudev
19d894538d YARN-5459. Add support for docker rm. Contributed by Shane Kumpf. 
(cherry picked from commit 08e33381f5342590a68c65504c79c296b40fe5af)
 2016-08-04 14:18:01 +05:30
Xiao Chen
bb10750067 HADOOP-13443. KMS should check the type of underlying keyprovider of KeyProviderExtension before falling back to default. Contributed by Anthony Young-Garner. 
(cherry picked from commit 05db64913d8c620cf481f74219bac9b72cb142af)
 2016-08-04 16:08:25 +08:00
Wei-Chiu Chuang
5964d65a4d HADOOP-13458. LoadBalancingKMSClientProvider#doOp should log IOException stacktrace. Contributed by Chen Liang. 
(cherry picked from commit a1f6564d31c16582ea949e4448091f1cc56c7dfe)
 2016-08-03 14:27:33 -07:00
Kihwal Lee
1ca2ab4013 HDFS-10569. A bug causes OutOfIndex error in BlockListAsLongs. Contributed by Weiwei Yang. 
(cherry picked from commit 6f63566694f8cec64a469448a8fa00ce921ce367)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java
 2016-08-03 15:27:03 -05:00
Jason Lowe
5a34fa670e HADOOP-10980. TestActiveStandbyElector fails occasionally in trunk. Contributed by Eric Badger 
(cherry picked from commit c82745432a962c817a8a7db92bb830fb6af01e33)
 2016-08-03 20:19:52 +00:00
Jason Lowe
65a91c8527 YARN-5469. Increase timeout of TestAmFilter.testFilter. Contributed by Eric Badger 
(cherry picked from commit db4a61dc617954cf448a0d44cb4ac79f2abf9db3)
 2016-08-03 19:53:28 +00:00
Jing Zhao
180a26d9fb HDFS-10710. In BlockManager#rescanPostponedMisreplicatedBlocks(), postponed misreplicated block counts should be retrieved within the NN lock protection.Contributed by GAO Rui. 
(cherry picked from commit f4ba5ff1d70ef92d59851c09c4bd4b43d6c04971)
 2016-08-03 12:42:44 -07:00
Jason Lowe
1ea03411c2 YARN-5462. TestNodeStatusUpdater.testNodeStatusUpdaterRetryAndNMShutdown fails intermittently. Contributed by Eric Badger 
(cherry picked from commit db646540f094077941b56ed681a4f3e5853f5b7f)
 2016-08-03 19:18:50 +00:00
Kihwal Lee
9d62caa440 HADOOP-13483. Optimize IPC server protobuf decoding. Contributed by Daryn Sharp. 
(cherry picked from commit 580a8334963709e728ed677c815fb7fef9bca70e)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
 2016-08-03 14:12:13 -05:00
Jason Lowe
9a0ac56a5c YARN-4280. CapacityScheduler reservations may not prevent indefinite postponement on a busy cluster. Contributed by Kuhu Shukla 
(cherry picked from commit 4d92aefd35d4517d9435d81bafdec0d77905a7a0)
 2016-08-03 19:00:06 +00:00
Kihwal Lee
a5d12d9c1f HDFS-10674. Optimize creating a full path from an inode. Contributed by Daryn Sharp. 
(cherry picked from commit 22ef5286bc8511ddee9594b7cecc598bf41a850b)
 2016-08-03 13:15:28 -05:00
Kihwal Lee
c15959fa60 HDFS-742. A down DataNode makes Balancer to hang on repeatingly asking NameNode its partial block list. Contributed by Mit Desai. 
(cherry picked from commit 58db263e93daf08280e6a586a10cebd6122cf72a)
 2016-08-03 12:18:15 -05:00
Kihwal Lee
ab7072838f HDFS-10656. Optimize conversion of byte arrays back to path string. Contributed by Daryn Sharp. 
(cherry picked from commit bebf10d2455cad1aa8985553417d4d74a61150ee)
 2016-08-03 11:54:51 -05:00
Kihwal Lee
dd21d242eb HADOOP-13426. More efficiently build IPC responses. Contributed by Daryn Sharp. 
(cherry picked from commit 2d8227605fe22c1c05f31729edc8939013763c05)
 2016-08-03 09:33:04 -05:00
Varun Vasudev
1087745d79 YARN-5460. Change container runtime type logging in DelegatingLinuxContainerRuntime to debug. Contributed by Shane Kumpf. 
(cherry picked from commit d848184e905d70835293ceac7a01b5a0ca21cc0a)
 2016-08-03 12:03:31 +05:30
Chris Nauroth
8ec3d702a3 HADOOP-13459. hadoop-azure runs several test cases repeatedly, causing unnecessarily long running time. Contributed by Chris Nauroth. 
(cherry picked from commit 4ef1bc0bdb6b2c907431492e4c8395035fa96c84)
 2016-08-02 22:32:37 -07:00
Arpit Agarwal
9850cf07fa HDFS-9353. Code and comment mismatch in JavaKeyStoreProvider. (Andras Bokor) 2016-08-02 15:18:01 -07:00
Arpit Agarwal
158dd2d5a2 HADOOP-13434. Add bash quoting to Shell class. (Owen O'Malley) 2016-08-02 13:40:56 -07:00
Kihwal Lee
dc0a17e828 HDFS-5805. TestCheckpoint.testCheckpoint fails intermittently on branch-2. Contributed by Eric Badger. 
(cherry picked from commit 5e5b8793fba8e25aeba7a74878da4cf8e806f061)
 2016-08-02 15:39:03 -05:00
Chris Nauroth
ec522a1907 HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin. 
(cherry picked from commit 0458a2af6e925d023882714e8b7b0568eca7a775)
 2016-08-02 12:43:51 -07:00
Junping Du
7e213844a7 YARN-5226. Remove AHS enable check from LogsCLI#fetchAMContainerLogs. Contributed by Xuan Gong. 
(cherry picked from commit 3818393297c7b337e380e8111a55f2ac4745cb83)
 2016-08-02 10:42:31 -07:00
Kihwal Lee
41882bd5ff HADOOP-13429. Dispose of unnecessary SASL servers. Contributed by Daryn Sharp. 
(cherry picked from commit b3018e73ccae43484d9cb85eabae814eb7f050a6)
 2016-08-02 10:52:10 -05:00
Varun Vasudev
b63e6c8027 YARN-5458. Rename DockerStopCommandTest to TestDockerStopCommand. Contributed by Shane Kumpf. 
(cherry picked from commit 7fc70c6422da3602ad9d4364493f25454a1de50c)
 2016-08-02 17:02:17 +05:30
Varun Vasudev
4e5409a2bb YARN-5443. Add support for docker inspect command. Contributed by Shane Kumpf. 
(cherry picked from commit 2e7c2a13a853b8195bc4f51f6c3c1f61656c2b33)
 2016-08-02 17:01:59 +05:30
Gera Shegalov
7eeba35495 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472320fa7592ed1b08b623c55a27089c6)
 2016-08-01 23:41:35 -07:00
Konstantin V Shvachko
24574f0e16 Revert "HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi." 
This reverts commit 0dde4035c9303ea1531c8d61f81ad9de2d630aa3.
 2016-08-01 22:44:32 -07:00
Akira Ajisaka
5469e128d3 HADOOP-13444. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 2016-08-02 10:17:33 +09:00
Daryn Sharp
4ad2a73df6 HDFS-10655. Fix path related byte array conversion bugs. (daryn) 
(cherry picked from commit 9f473cf903e586c556154abd56b3a3d820c6b028)
 2016-08-01 10:17:24 -05:00
Varun Vasudev
7ecb1635b4 YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu. 
(cherry picked from commit 4455aa7c86a36b6e017e288479a7d4ced889faa0)
 2016-08-01 16:01:10 +05:30
Arun Suresh
f197378f81 YARN-5113. Refactoring and other clean-up for distributed scheduling. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit e5766b1dbee02ae0ef89618e172f3fb227af19e8)
 2016-07-31 11:49:09 -07:00
Akira Ajisaka
96d5607e7a MAPREDUCE-6729. Accurately compute the test execute time in DFSIO. Contributed by mingleizhang. 
This closes #112

(cherry picked from commit ce93595d7a18bd0561ae2c1983b51f6aba887179)
 2016-07-30 22:16:13 +09:00
Yongjun Zhang
f5bc6dd478 HDFS-10703. HA NameNode Web UI should show last checkpoint time. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit 8d32bd884d53948728b77de14531b3be196f4bc7)
 2016-07-29 23:39:38 -07:00
Akira Ajisaka
0fc838540b MAPREDUCE-6746. Replace org.apache.commons.io.Charsets with java.nio.charset.StandardCharsets. Contributed by Vincent Poon. 
(cherry picked from commit 3f0bffea2d35083c45acee79338e10b608374d8e)
 2016-07-30 11:47:15 +09:00
Junping Du
47dd871f96 YARN-5434. Add -client|server argument for graceful decommmission. Contributed by Robert Kanter. 
(cherry picked from commit 95f2b9859718eca12fb3167775cdd2dad25dde25)
 2016-07-29 10:27:11 -07:00
Akira Ajisaka
0cff416c35 HDFS-10691. FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException. Contributed by Yiqun Lin. 
(cherry picked from commit 204a2055b1b9270ae13ea03b7aeac62b65166efd)
 2016-07-29 15:40:59 +09:00
Xiao Chen
8fe4b2429a HADOOP-13381. KMS clients should use KMS Delegation Tokens from current UGI. Contributed by Xiao Chen. 
(cherry picked from commit 8ebf2e95d2053cb94c6ff87ca018811fe8276f2b)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2016-07-28 18:39:59 -07:00
Li Lu
0b5d96abb5 YARN-5436. Race in AsyncDispatcher can cause random test failures in Tez (probably YARN also). (Zhiyuan Yang via gtcarrera9) 
(cherry picked from commit 7086fc72eebc41fd174d91839ed703c014aac920)
 2016-07-28 16:52:21 -07:00
Xiao Chen
24d464a150 HDFS-9276. Failed to Update HDFS Delegation Token for long running application in HA mode. Contributed by Liangliang Gu and John Zhuge 
(cherry picked from commit d9aae22fdf2ab22ae8ce4a9d32ac71b3dde084d3)
 2016-07-28 16:41:31 -07:00
Xiaoyu Yao
f03ad5677a HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru. 2016-07-28 16:16:21 -07:00
Subru Krishnan
2f4df2b851 YARN-5203.Return ResourceRequest JAXB object in ResourceManager Cluster Applications REST API. Contributed by Ellen Hui. 
(cherry picked from commit 4e756d72719ec3c6d64a1e3daccbc0b8e8de998c)
 2016-07-28 16:04:30 -07:00