Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,315 Commits 326 Branches 371 Tags
Commit Graph

2484 Commits

Author SHA1 Message Date
Xiao Chen
bc5a9931d7 HADOOP-13698. Document caveat for KeyShell when underlying KeyProvider does not delete a key. 
(cherry picked from commit b84c4891f9eca8d56593e48e9df88be42e24220d)
 2016-10-11 17:05:20 -07:00
Wei-Chiu Chuang
773e2e6b6c HADOOP-13684. Snappy may complain Hadoop is built without snappy if libhadoop is not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4b32b1420d98ea23460d05ae94f2698109b3d6f7)
(cherry picked from commit 56184de510e27990c36726e2a648e8ce4b06504c)
 2016-10-11 13:22:41 -07:00
Mingliang Liu
def48f5228 HDFS-10985. o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions. Contributed by Mingliang Liu 
(cherry picked from commit de30f13a0ae22f0ffc647e4c1daf19ca9c01bb1a)
 2016-10-10 13:40:39 -07:00
Kai Zheng
583283db25 HADOOP-13641. Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentation. Contributed by Huafeng Wang 2016-10-10 11:45:07 +06:00
Steve Loughran
96cc056cb7 HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran. 2016-10-07 13:22:20 +01:00
Steve Loughran
d24933ad3a HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-10-07 13:12:04 +01:00
Chris Nauroth
e36b665f90 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 
(cherry picked from commit 1d330fbaf6b50802750aa461640773fb788ef884)
(cherry picked from commit a28ffd0fdeff64a12612e60f4ebc5e6311b4b112)
 2016-10-06 13:24:54 -07:00
Chris Nauroth
b9ce40e12c HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 
(cherry picked from commit 2d46c3f6b7d55b6a2f124d07fe26d37359615df4)
(cherry picked from commit ecccb114ae1b09526d11385df7085b6dd3376e2d)
 2016-10-06 10:57:15 -07:00
Brahma Reddy Battula
a4ee4f9e51 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 
(cherry picked from commit b90fc70d671481564e468550c770c925f25d7db0)
 2016-10-06 17:49:48 +05:30
Akira Ajisaka
c0699a1f55 HADOOP-13685. Document -safely option of rm command. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9a61b52fbefd2970af78bb283ae174d462d57a13)
(cherry picked from commit bdbb75e083218e11d8c8e34d38e4dc859b444c97)
 2016-10-05 20:41:51 +09:00
Xiao Chen
bf043e63f9 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157362661ccab10379c2d82e780382f83)
 2016-10-03 16:02:11 -07:00
Wei-Chiu Chuang
9a79c54867 HADOOP-12974. Create a CachingGetSpaceUsed implementation that uses df. Contributed by Elliott Clark. 
(cherry picked from commit 57aec2b46b0e46b73a1f49927e30e2c41138d535)
(cherry picked from commit a6863e5334569fa92e70cc4d84afb545f993c8d3)
 2016-09-30 13:29:59 -07:00
Chris Nauroth
1085b3ee45 Revert "HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin." 
This reverts commit 06df0cd87f5517e778cf0d97614035e126e2fb80.
 2016-09-29 13:58:35 -07:00
Kihwal Lee
d60b4648dc HADOOP-13537. Support external calls in the RPC call queue. Contributed by Daryn Sharp. 
(cherry picked from commit 236ac773c964fa21d6d5f1496023cd61818dd3b1)
(cherry picked from commit 92336ea37e64c48ca84dbb71d84e4d78aaa29438)
 2016-09-29 13:58:17 -05:00
Steve Loughran
1c8e388181 HADOOP-13663 Index out of range in SysInfoWindows. Contributed by Inigo Goiri 2016-09-29 11:35:54 +01:00
Kihwal Lee
7091753bd2 HADOOP-11780. Prevent IPC reader thread death. Contributed by Daryn Sharp. 
(cherry picked from commit e19b37ead23805c7ed45bdcbfa7fdc8898cde7b2)
 2016-09-28 10:03:24 -05:00
Brahma Reddy Battula
5f754e8638 HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 
(cherry picked from commit e9a34ae29c7390f3ffcbeee02dc5faa26fca482a)
 2016-09-28 11:16:40 +05:30
Mingliang Liu
ecb5f282ea HADOOP-13658. Replace config key literal strings with names I: hadoop common. Contributed by Chen Liang 
(cherry picked from commit 9a44a832a99eb967aa4e34338dfa75baf35f9845)
 2016-09-27 17:42:01 -07:00
Wei-Chiu Chuang
6ca5ffe4b5 HADOOP-13535. Add jetty6 acceptor startup issue workaround to branch-2. Contributed by Min Shen. 
(cherry picked from commit 23984e17870bcff6fffd8e1ca185fd37e2352b29)
 2016-09-23 14:28:00 -07:00
Brahma Reddy Battula
aa967869a5 HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document. 
(cherry picked from commit bbdf350ff9fb624fe736c1eb9271c5dcb4e14b06)
 2016-09-23 19:35:34 +05:30
Mingliang Liu
c7bd564b79 HADOOP-13601. Fix a log message typo in AbstractDelegationTokenSecretManager. Contributed by Mehran Hassani. 
(cherry picked from commit e80386d69d5fb6a08aa3366e42d2518747af569f)
 2016-09-20 13:24:02 -07:00
Xiao Chen
900dc70a01 HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 
(cherry picked from commit ea839bd48e4478fc7b6d0a69e0eaeae2de5e0f0d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

(cherry picked from commit 20585ade1d9d7fbd8b9df8b1e3db5a74fd4441a6)
 2016-09-19 16:38:00 -07:00
Xiao Chen
a7f1dc8aa8 HADOOP-13558. UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket. Contributed by Xiao Chen. 
(cherry picked from commit 680be58aac03a9ffab6b07c8fde9602ddb9dc858)
(cherry picked from commit d157733082697e67be56f516606a0127f5830c58)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
 2016-09-19 12:52:58 -07:00
Akira Ajisaka
06d82ffb76 HADOOP-13616. Broken code snippet area in Hadoop Benchmarking. Contributed by Kai Sasaki. 
(cherry picked from commit 7cad7b704644f260688eb0d4d92413f3a8630f25)
(cherry picked from commit e485ace0dd30c684d3432c49ccc20496270f7aea)
 2016-09-15 23:43:57 +09:00
Steve Loughran
6a7ce4ee52 HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran 2016-09-14 15:44:20 +01:00
Jing Zhao
26597aa59b HADOOP-13546. Override equals and hashCode of the default retry policy to avoid connection leakage. Contributed by Xiaobing Zhou. 
(cherry picked from commit 08d8e0ba259f01465a83d8db09466dfd46b7ec81)
(cherry picked from commit 55600b38a81b822f2bdc5d71dbfb1b7336c2c024)
 2016-09-13 11:15:51 -07:00
Yongjun Zhang
b96e2cbd20 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit d2466ec3e01b5ef2a0bde738232c5ad6d2d956eb)
 2016-09-12 22:13:17 -07:00
Arpit Agarwal
ed35f99045 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:34:28 -07:00
Arpit Agarwal
04f620c4d0 HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang. 2016-09-10 18:01:37 -07:00
Kihwal Lee
4b9845bc53 HADOOP-10940. RPC client does no bounds checking of responses. Contributed by Daryn Sharp. 
(cherry picked from commit d4d076876a8d0002bd3a73491d8459d11cb4896c)
(cherry picked from commit d2349902081c32010e10ce67bdc33d8cb9423fa6)
 2016-09-09 10:48:05 -05:00
Kihwal Lee
56340ac029 HADOOP-13549. Eliminate intermediate buffer for server-side PB encoding. Contributed by Daryn Sharp. 
(cherry picked from commit 39d1b1d747b1e325792b897b3264272f32b756a9)
 2016-09-06 10:19:34 -05:00
Kihwal Lee
20eb5386d9 HADOOP-13547. Optimize IPC client protobuf decoding. Contributed by Daryn Sharp. 
(cherry picked from commit 28ea4122f087581f5a7b1bea753f253af5ea0cc7)
 2016-09-02 11:14:39 -05:00
Ray Chiang
2dd809eada Revert "YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang)" 
This reverts commit 9ad3a7a43b35dd4bb9d10bd322fbb956d3b01ff2.
 2016-09-01 15:10:37 -07:00
Kihwal Lee
af26fba62e HADOOP-13473. Tracing in IPC Server is broken. Contributed by Daryn Sharp. 
(cherry picked from commit caf800d5290d8618003b764afb0b3ef8d9a5a0a8)
 2016-09-01 16:19:10 -05:00
Kihwal Lee
2ff6b13e69 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 
(cherry picked from commit 76cd81f4b656f0d40a4b2122e15f04ea53d8020b)
 2016-09-01 16:11:45 -05:00
Kihwal Lee
a4292fe618 HADOOP-13483. Optimize IPC server protobuf decoding. Contributed by Daryn Sharp. 
(cherry picked from commit 580a8334963709e728ed677c815fb7fef9bca70e)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/ProtobufRpcEngine.java
 2016-09-01 16:00:55 -05:00
Kihwal Lee
983d47ecb3 HADOOP-13429. Dispose of unnecessary SASL servers. Contributed by Daryn Sharp. 
(cherry picked from commit b3018e73ccae43484d9cb85eabae814eb7f050a6)
 2016-09-01 15:59:53 -05:00
Kihwal Lee
afc8da0d86 HADOOP-13426. More efficiently build IPC responses. Contributed by Daryn Sharp. 
(cherry picked from commit 2d8227605fe22c1c05f31729edc8939013763c05)
 2016-09-01 15:59:40 -05:00
Mingliang Liu
22e182f295 HADOOP-13375. o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky. (Contributed by Weiwei Yang) 
(cherry picked from commit dcd21d083ab2a66fc3ca3bfda03887461698b7b1)
(cherry picked from commit 4d709be8c9db236bc7f473e7f31ecd8385211c01)
 2016-09-01 11:14:57 -07:00
Ray Chiang
9ad3a7a43b YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang) 
(cherry picked from commit 05ede003868871addc30162e9707c3dc14ed6b7a)
(cherry picked from commit 3b19bcb8f0f70606a8080bc46fa64cd34f6280d7)
 2016-08-29 16:15:39 -07:00
Mingliang Liu
250ddf155f HADOOP-13559. Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes. (Contributed by Aaron Fabbri) 
(cherry picked from commit 6fcb04c1780ac3dca5b986f1bcd558fffccb3eb9)
 2016-08-29 13:05:34 -07:00
Jason Lowe
b1979bec51 HADOOP-13552. RetryInvocationHandler logs all remote exceptions. Contributed by Jason Lowe 
(cherry picked from commit 92d8f371553b88e5b3a9d3354e93f75d60d81368)
 2016-08-29 15:58:34 +00:00
Wei-Chiu Chuang
bd729af8a8 HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-24 08:53:43 -07:00
Akira Ajisaka
f871aed87e HADOOP-13538. Deprecate getInstance and initialize methods with Path in TrashPolicy. Contributed by Yiqun Lin. 
(cherry picked from commit 092b4d5bfd02131d62723fc5673892305eb9fcef)
(cherry picked from commit 2321b73037fcf7ea9ee60c1a94838233c3df8908)
 2016-08-24 16:50:28 +09:00
Zhe Zhang
8bc33bf343 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. Branch-2 patch contributed by Wei-Chiu Chuang. 
(cherry picked from commit dfcbc12026b591745a7d7279f2b840152cb53a91)
 2016-08-23 14:46:47 -07:00
Masatake Iwasaki
80d8e79e26 HADOOP-13497. fix wrong command in CredentialProviderAPI.md. Contributed by Yuanbo Liu. 
(cherry picked from commit 8aae8d6bf03ade0607547ed461dc99a336a7e9d4)
 2016-08-24 00:10:37 +09:00
Xiao Chen
0f9bed050d HADOOP-13487. Hadoop KMS should load old delegation tokens from Zookeeper on startup. Contributed by Xiao Chen. 
(cherry picked from commit 2f21d53e3aaf97f425f954f52b50f36dddb6b886)
 2016-08-22 14:42:20 -07:00
Mingliang Liu
9d304822f6 HADOOP-13527. Add Spark to CallerContext LimitedPrivate scope. (Contributed by Weiqing Yang) 
(cherry picked from commit 115ecb52a86b49aad3d058a6b4c1c7926b8b0a40)
 2016-08-21 09:42:06 -07:00
Kihwal Lee
4f3df8e22a HDFS-7959. WebHdfs logging is missing on Datanode (Kihwal Lee via sjlee) 
(cherry picked from commit ae90d4dd908cf3f9e9ff26fa8e92f028057a9ca1)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/webhdfs/WebHdfsHandler.java
 2016-08-19 15:32:11 -05:00
Chris Nauroth
dd6c149c4a HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran. 
(cherry picked from commit 763f0497bb996e331e40caed9ca0af966f5b3fac)
(cherry picked from commit 6dba38f08ba84109b01c8ce8d30c71354d089dbe)
 2016-08-19 12:55:57 -07:00