Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,467 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3273 Commits

Author SHA1 Message Date
Xiaoyu Yao 0205ad5fa8 HDFS-10757. KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used. Contributed by Xiaoyu Yao. 
(cherry picked from commit be72372248)
 2016-10-21 14:25:29 -07:00
Chris Douglas 25f4327f0b HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du 
(cherry picked from commit ae8bccd509)
 2016-10-21 13:01:08 -07:00
Steve Loughran 385c1daa46 HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-21 19:11:31 +01:00
Brahma Reddy Battula 2ab149f0e4 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc03)
 2016-10-21 16:19:18 +05:30
Steve Loughran f0af3dee25 Revert "HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran." 
This reverts commit 42f8a1d6eb.
 2016-10-20 22:48:26 +01:00
Anu Engineer 1f384b617d HADOOP-13737. Cleanup DiskChecker interface. Contributed by Arpit Agarwal. 2016-10-20 13:40:01 -07:00
Anu Engineer 42f8a1d6eb HADOOP-13716. Add LambdaTestUtils class for tests; fix eventual consistency problem in contract test setup. Contributed by Steve Loughran. 2016-10-20 12:47:57 -07:00
Steve Loughran ab36519b6f Revert "HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula" 
This reverts commit bf6379fdd2.
 2016-10-20 19:32:10 +01:00
Brahma Reddy Battula bf6379fdd2 HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula 
(cherry picked from commit f872c6bc03)
 2016-10-20 21:48:23 +05:30
Steve Loughran bc176961e6 HADOOP-13560. S3ABlockOutputStream to support huge (many GB) file writes. Contributed by Steve Loughran 2016-10-18 19:33:38 +01:00
Akira Ajisaka b2618685fb HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini. 
(cherry picked from commit bedfec0c10)
 2016-10-18 15:03:33 +09:00
Andrew Wang 4ed7cf3b36 HADOOP-13724. Fix a few typos in site markdown documents. Contributed by Ding Fei. 
(cherry picked from commit 987ee51141)
 2016-10-17 13:32:39 -07:00
Allen Wittenauer fbdb23d2af HADOOP-11559. Add links to RackAwareness and InterfaceClassification to site index (Masatake Iwasaki via aw) 
(cherry picked from commit 7eeca90daa)

 Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-project/src/site/site.xml
 2016-10-17 13:32:28 -07:00
Eric Yang 7993fb5b81 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)

    (cherry picked from commit dc308e98b9)
    With Addendum patch
 2016-10-17 07:57:45 -07:00
Eric Yang a4356bb7a5 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)

    (cherry picked from commit dc308e98b9)
    With Addendum patch
 2016-10-16 20:32:48 -07:00
Eric Yang 439422fff9 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)

(cherry picked from commit dc308e98b9)
With Addendum patch
 2016-10-15 21:23:21 -07:00
Masatake Iwasaki a13a607e20 HADOOP-13419. Fix javadoc warnings by JDK8 in hadoop-common package. Contributed by Kai Sasaki. 2016-10-16 12:44:19 +09:00
Brahma Reddy Battula a0eee72053 Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit dc308e98b9.
 2016-10-15 22:13:45 +05:30
Eric Yang dc308e98b9 HADOOP-13707. Skip authorization for anonymous user to access Hadoop 
web interface in non-secure environment.  (Yuanbo Liu via eyang)
 2016-10-14 22:13:24 -07:00
Xiaoyu Yao 1eb8c0e880 HADOOP-13686. Adding additional unit test for Trash (I). Contributed by Weiwei Yang. 
(cherry picked from commit dbe663d524)
 2016-10-13 23:05:58 -07:00
Xiao Chen 562c358a83 HADOOP-13669. Addendum patch for KMS Server should log exceptions before throwing. 
(cherry picked from commit ae51b11f78)
 2016-10-13 22:32:25 -07:00
Arpit Agarwal ecd3467f11 HADOOP-13710. Supress CachingGetSpaceUsed from logging interrupted exception stacktrace. Contributed by Hanisha Koneru. 2016-10-13 11:37:12 -07:00
Xiaoyu Yao 874402a295 HADOOP-13565. KerberosAuthenticationHandler#authenticate should not rebuild SPN based on client request. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9097e2efe4)
 2016-10-13 10:54:51 -07:00
Andrew Wang f131d61ff8 HADOOP-13700. Remove unthrown IOException from TrashPolicy#initialize and #getInstance signatures. 
(cherry picked from commit 12d739a34b)
 2016-10-12 15:20:07 -07:00
Kihwal Lee 5305a392c3 HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 
(cherry picked from commit 85cd06f663)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-10-12 15:22:51 -05:00
Xiao Chen b413dd34b3 HADOOP-13698. Document caveat for KeyShell when underlying KeyProvider does not delete a key. 
(cherry picked from commit b84c4891f9)
 2016-10-11 17:05:17 -07:00
Wei-Chiu Chuang 56184de510 HADOOP-13684. Snappy may complain Hadoop is built without snappy if libhadoop is not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4b32b1420d)
 2016-10-11 13:22:25 -07:00
Mingliang Liu 21fcb5726a HADOOP-13697. LogLevel#main should not throw exception if no arguments. Contributed by Mingliang Liu 
(cherry picked from commit 2fb392a587)
 2016-10-11 11:18:46 -07:00
Mingliang Liu de30f13a0a HDFS-10985. o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions. Contributed by Mingliang Liu 2016-10-10 13:37:00 -07:00
Xiao Chen eebda43ec1 HADOOP-13669. KMS Server should log exceptions before throwing. Contributed by Suraj Acharya. 
(cherry picked from commit fc18c32540ed6a410adb123e1105729e0343b7f5)
 2016-10-10 12:51:08 -07:00
Kai Zheng ad0659557b HADOOP-13641. Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentation. Contributed by Huafeng Wang 2016-10-10 11:44:04 +06:00
Xiao Chen b9761f2fc9 HADOOP-13627. Have an explicit KerberosAuthException for UGI to throw, text from public constants. Contributed by Xiao Chen. 
(cherry picked from commit 2e853be657)
 2016-10-07 13:46:45 -07:00
Robert Kanter 7bfa595679 HADOOP-12611. TestZKSignerSecretProvider#testMultipleInit occasionally fail (ebadger via rkanter) 
(cherry picked from commit c183b9de8d)
 2016-10-07 09:37:17 -07:00
Steve Loughran a6bb21eec4 HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran. 2016-10-07 12:47:14 +01:00
Chris Nauroth a28ffd0fde HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 
(cherry picked from commit 1d330fbaf6)
 2016-10-06 13:22:35 -07:00
Chris Nauroth ecccb114ae HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 
(cherry picked from commit 2d46c3f6b7)
 2016-10-06 10:57:08 -07:00
Brahma Reddy Battula 69c1ab4ad5 HADOOP-13690. Fix typos in core-default.xml. Contributed by Yiqun Lin 
(cherry picked from commit 35b9d7de9f)
 2016-10-06 18:07:23 +05:30
Brahma Reddy Battula d9e4ad78ca HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 
(cherry picked from commit b90fc70d67)
 2016-10-06 17:48:49 +05:30
Wei-Chiu Chuang 434403a2a0 HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge. 
(cherry picked from commit c5ca216915)
 2016-10-05 17:46:46 -07:00
Akira Ajisaka bdbb75e083 HADOOP-13685. Document -safely option of rm command. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9a61b52fbe)
 2016-10-05 20:41:29 +09:00
Xiao Chen d9b686a2bd HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157)
 2016-10-03 16:02:07 -07:00
Xiao Chen 1ed23a134c HADOOP-13317. Add logs to KMS server-side to improve supportability. Contributed by Suraj Acharya. 
(cherry picked from commit 89bd6d29a6)
 2016-09-30 17:52:13 -07:00
Wei-Chiu Chuang a6863e5334 HADOOP-12974. Create a CachingGetSpaceUsed implementation that uses df. Contributed by Elliott Clark. 
(cherry picked from commit 57aec2b46b)
 2016-09-30 13:00:04 -07:00
Chris Nauroth cf954d7a48 Revert "HADOOP-13081. add the ability to create multiple UGIs/subjects from one kerberos login. Contributed by Sergey Shelukhin." 
This reverts commit ec522a1907.
 2016-09-29 13:58:55 -07:00
Kihwal Lee 92336ea37e HADOOP-13537. Support external calls in the RPC call queue. Contributed by Daryn Sharp. 
(cherry picked from commit 236ac773c9)
 2016-09-29 13:54:30 -05:00
Steve Loughran 896df3f55a HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-09-29 17:00:34 +01:00
Steve Loughran 2b6b4fd779 HADOOP-13663 Index out of range in SysInfoWindows. Contributed by Inigo Goiri 2016-09-29 11:35:37 +01:00
Kihwal Lee baf8aac05c HADOOP-11780. Prevent IPC reader thread death. Contributed by Daryn Sharp. 
(cherry picked from commit e19b37ead2)
 2016-09-28 09:46:42 -05:00
Brahma Reddy Battula e9a34ae29c HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki 2016-09-28 10:50:50 +05:30
Mingliang Liu 34ee18d73d HADOOP-13658. Replace config key literal strings with names I: hadoop common. Contributed by Chen Liang 
(cherry picked from commit 9a44a832a9)
 2016-09-27 17:39:36 -07:00
Wangda Tan 269401dc83 HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda) 
(cherry picked from commit 875062b5bc)
 2016-09-27 11:27:25 -07:00
Xiao Chen 06187e4f98 HADOOP-13638. KMS should set UGI's Configuration object properly. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit fa397e74fe)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2016-09-26 13:02:57 -07:00
Wei-Chiu Chuang 23984e1787 HADOOP-13535. Add jetty6 acceptor startup issue workaround to branch-2. Contributed by Min Shen. 2016-09-23 14:25:52 -07:00
Brahma Reddy Battula 347131fe8d HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document. 
(cherry picked from commit bbdf350ff9)
 2016-09-23 19:32:37 +05:30
Mingliang Liu f98afb3bee HADOOP-13601. Fix a log message typo in AbstractDelegationTokenSecretManager. Contributed by Mehran Hassani. 
(cherry picked from commit e80386d69d)
 2016-09-20 13:23:17 -07:00
Xiao Chen b6a60d715d HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen. 
(cherry picked from commit ea839bd48e)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2016-09-19 16:38:18 -07:00
Andrew Wang b03a0be7a3 HDFS-10823. Implement HttpFSFileSystem#listStatusIterator. 
(cherry picked from commit 8a40953058)
 2016-09-16 15:37:49 -07:00
Wei-Chiu Chuang 031d5f6c5b HADOOP-13580. If user is unauthorized, log "unauthorized" instead of "Invalid signed text:". Contributed by Wei-Chiu Chuang. 
(cherry picked from commit f6f3a447bf)
 2016-09-16 14:53:50 -07:00
Akira Ajisaka e485ace0dd HADOOP-13616. Broken code snippet area in Hadoop Benchmarking. Contributed by Kai Sasaki. 
(cherry picked from commit 7cad7b7046)
 2016-09-15 23:43:20 +09:00
Steve Loughran 3b9268cfba HADOOP-13606 swift FS to add a service load metadata file. Contributed by Steve Loughran 2016-09-14 15:44:45 +01:00
Jing Zhao 55600b38a8 HADOOP-13546. Override equals and hashCode of the default retry policy to avoid connection leakage. Contributed by Xiaobing Zhou. 
(cherry picked from commit 08d8e0ba25)
 2016-09-13 11:13:37 -07:00
Yongjun Zhang 223ce108a1 HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang) 
(cherry picked from commit d2466ec3e0)
 2016-09-12 21:46:09 -07:00
Arpit Agarwal 6948691ad1 HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy) 2016-09-10 18:22:15 -07:00
Arpit Agarwal 012b266e5e HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang. 2016-09-10 18:04:00 -07:00
Chris Douglas 806a8dea38 MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 
(cherry picked from commit 9f192cc5ac)
 2016-09-09 11:14:30 -07:00
Kihwal Lee d234990208 HADOOP-10940. RPC client does no bounds checking of responses. Contributed by Daryn Sharp. 
(cherry picked from commit d4d076876a)
 2016-09-09 10:46:37 -05:00
Xiao Chen d157733082 HADOOP-13558. UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket. Contributed by Xiao Chen. 
(cherry picked from commit 680be58aac)
 2016-09-06 20:54:38 -07:00
Kihwal Lee 614f9a62c4 HADOOP-13549. Eliminate intermediate buffer for server-side PB encoding. Contributed by Daryn Sharp. 
(cherry picked from commit 39d1b1d747)
 2016-09-06 10:15:57 -05:00
Kihwal Lee 28ea4122f0 HADOOP-13547. Optimize IPC client protobuf decoding. Contributed by Daryn Sharp. 2016-09-02 11:12:05 -05:00
Ray Chiang c142048084 Revert "YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang)" 
This reverts commit 3b19bcb8f0.
 2016-09-01 15:10:53 -07:00
Kihwal Lee caef03d742 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 
(cherry picked from commit 76cd81f4b6)
 2016-09-01 15:51:23 -05:00
Mingliang Liu 4d709be8c9 HADOOP-13375. o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky. (Contributed by Weiwei Yang) 
(cherry picked from commit dcd21d083a)
 2016-09-01 11:14:25 -07:00
Ray Chiang 3b19bcb8f0 YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang) 
(cherry picked from commit 05ede00386)
 2016-08-29 15:56:30 -07:00
Mingliang Liu d124af1a7c HADOOP-13559. Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes. (Contributed by Aaron Fabbri) 
(cherry picked from commit 6fcb04c178)
 2016-08-29 13:05:07 -07:00
Jason Lowe 039c831f5f HADOOP-13552. RetryInvocationHandler logs all remote exceptions. Contributed by Jason Lowe 
(cherry picked from commit 92d8f37155)
 2016-08-29 15:56:53 +00:00
Xiao Chen e31745e281 HADOOP-13396. Allow pluggable audit loggers in KMS. Contributed by Xiao Chen. 
(cherry picked from commit 3476156807)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMSAudit.java
 2016-08-24 11:09:54 -07:00
Wei-Chiu Chuang d19e29bd68 HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen. 2016-08-24 08:49:54 -07:00
Akira Ajisaka 2321b73037 HADOOP-13538. Deprecate getInstance and initialize methods with Path in TrashPolicy. Contributed by Yiqun Lin. 
(cherry picked from commit 092b4d5bfd)
 2016-08-24 16:50:11 +09:00
Zhe Zhang dfcbc12026 HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen. Branch-2 patch contributed by Wei-Chiu Chuang. 2016-08-23 14:46:08 -07:00
Masatake Iwasaki e1d3b8e71f HADOOP-13497. fix wrong command in CredentialProviderAPI.md. Contributed by Yuanbo Liu. 
(cherry picked from commit 8aae8d6bf0)
 2016-08-24 00:09:50 +09:00
Eric Yang 9faacbec2c HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang) 2016-08-22 18:32:06 -07:00
Xiao Chen a4b296734b HADOOP-13526. Add detailed logging in KMS for the authentication failure of proxy user. Contributed by Suraj Acharya. 
(cherry picked from commit 98e7be769a7e69bdaec4f67143949f282f6761cf)
 2016-08-22 18:08:09 -07:00
Xiao Chen c76e0b93f0 HADOOP-13487. Hadoop KMS should load old delegation tokens from Zookeeper on startup. Contributed by Xiao Chen. 
(cherry picked from commit 2f21d53e3aaf97f425f954f52b50f36dddb6b886)
 2016-08-22 14:42:25 -07:00
Akira Ajisaka 62c7d09efa HDFS-10645. Make block report size as a metric and add this metric to datanode web ui. Contributed by Yuanbo Liu. 2016-08-22 14:22:37 +09:00
Mingliang Liu 50a8b0eb6b HADOOP-13527. Add Spark to CallerContext LimitedPrivate scope. (Contributed by Weiqing Yang) 2016-08-21 09:41:19 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) a299146b6d HADOOP-13428. Fixed hadoop-common build files to generate jdiff by working around a jdiff-bug. Contributed by Wangda Tan. 
(cherry picked from commit 99603e9022)
 2016-08-19 19:10:35 -07:00
Chris Nauroth 6dba38f08b HADOOP-13252. Tune S3A provider plugin mechanism. Contributed by Steve Loughran. 
(cherry picked from commit 763f0497bb)
 2016-08-19 12:30:47 -07:00
Jing Zhao 9fdd1ea724 HADOOP-13503. Improve SaslRpcClient failure logging. Contributed by Xiaobing Zhou. 2016-08-18 14:57:38 -07:00
Mingliang Liu 3f3957ad66 HADOOP-13512. ReloadingX509TrustManager should keep reloading in case of exception. (Contributed by Mingliang Liu) 2016-08-18 14:15:01 -07:00
Junping Du d464483bf7 YARN-4676. Automatic and Asynchronous Decommissioning Nodes Status Tracking. Contributed by Diniel Zhi. 2016-08-18 07:23:29 -07:00
Steve Loughran e39d81be80 HADOOP-13405 doc for fs.s3a.acl.default indicates incorrect values. Contributed by Shen Yinjie 2016-08-18 14:36:41 +01:00
Anu Engineer ac8c3ae784 HADOOP-11786. Fix Javadoc typos in org.apache.hadoop.fs.FileSystem. Contributed by Andras Bokor. 2016-08-17 15:04:02 -07:00
Mingliang Liu 3ff4e6f3d9 HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-16 17:36:21 -07:00
Mingliang Liu 9392fe3a6b Revert "HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu)" 
This reverts commit 23161c67cf.
 2016-08-16 16:27:02 -07:00
Andrew Wang c91e6f2828 HADOOP-13494. ReconfigurableBase can log sensitive information. Contributed by Sean Mackrory. 
(cherry picked from commit 4b689e7a75)
 2016-08-16 15:02:08 -07:00
Mingliang Liu 23161c67cf HADOOP-13470. GenericTestUtils$LogCapturer is flaky. (Contributed by Mingliang Liu) 2016-08-15 20:38:40 -07:00
Mingliang Liu f9a7e59066 HDFS-10678. Documenting NNThroughputBenchmark tool. (Contributed by Mingliang Liu) 2016-08-15 20:37:55 -07:00
Mingliang Liu e36a913663 HDFS-10724. Document caller context config keys. (Contributed by Mingliang Liu) 
(cherry picked from commit 4bcbef39f7)
 2016-08-15 20:36:11 -07:00
Xiao Chen 74156ee20a HADOOP-13437. KMS should reload whitelist and default key ACLs when hot-reloading. Contributed by Xiao Chen. 
(cherry picked from commit 9daa9979a1)
 2016-08-15 18:18:31 -07:00
Varun Saxena 5c2f2ae7bc HADOOP-13333. testConf.xml ls comparators in wrong order (Vrushali C via Varun Saxena) 2016-08-16 03:05:16 +05:30