Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
23,643 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

23643 Commits

Author SHA1 Message Date
Inigo Goiri 24080666e5 HDFS-14908. LeaseManager should check parent-child relationship when filter open files. Contributed by Jinglun. 2019-12-16 18:41:45 -08:00
Mate Szalay-Beko 578bd101a6 HADOOP-16765. Fix curator dependencies for gradle projects using hadoop-minicluster. (#1763) 
Gradle behaves differently than Maven when processing exclude rules through transitive
dependencies. This causes gradle projects (e.g. in Kudu) that are using Hadoop MiniCluster
to fail because different Guava version come to the classpath through Curator. In this
fix we make sure that ZooKeeper, SLF4J and Gradle gets always excluded from Curator Client
even if they would come through other Curator dependencies transitively.
 2019-12-16 14:43:59 -08:00
Steve Loughran 382151670b HADOOP-16450. ITestS3ACommitterFactory to not use useInconsistentClient. (#1145) 
Contributed by Steve Loughran.

Change-Id: Ifb9771a73a07f744e4ed5f5e6be72473179db439
 2019-12-16 14:29:30 +01:00
Masatake Iwasaki dc6cf17b34 HDFS-15048. Fix findbug in DirectoryScanner. 2019-12-16 19:00:57 +09:00
Ayush Saxena ab32762f43 HADOOP-16755. Fix broken link in single node cluster setup documentation. Contributed by Denes Gerencser. 2019-12-16 08:44:28 +05:30
Kengo Seki fd7de2b82a HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 2019-12-16 11:04:20 +09:00
Szilard Nemeth 631dbbc6f2 YARN-9923. Introduce HealthReporter interface to support multiple health checker files. Contributed by Adam Antal 2019-12-15 17:28:04 +01:00
Ayush Saxena 7a87007545 HDFS-15038. TestFsck testFsckListCorruptSnapshotFiles is failing in trunk. Contributed by hemanthboyina. 2019-12-15 00:33:32 +05:30
Ayush Saxena 72aee114f8 HDFS-15053. RBF: Add permission check for safemode operation. Contributed by Xiaoqiao He. 2019-12-14 17:57:23 +05:30
Inigo Goiri 7fe924b1c0 HDFS-15016. RBF: getDatanodeReport() should return the latest update. Contributed by Inigo Goiri. 2019-12-13 10:51:14 -08:00
Mingliang Liu d12ad9e8ad
HADOOP-16757. Increase timeout unit test rule for MetadataStoreTestBase (#1757) 
Contributed by Mingliang Liu.

Signed-off-by: Steve Loughran <stevel@apache.org>
 2019-12-13 08:19:27 -08:00
Chen Liang 65c4660bcd HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 2019-12-12 10:22:05 -08:00
Takanobu Asanuma c210cede5c HDFS-15044. [Dynamometer] Show the line of audit log when parsing it unsuccessfully. (#1749) 2019-12-12 07:48:14 -08:00
Masatake Iwasaki bdd00f10b4
HDFS-15047. Document the new decommission monitor (HDFS-14854). (#1755) 2019-12-13 00:35:41 +09:00
He Xiaoqiao 0e28cd8f63 HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-12-12 02:23:32 -08:00
Takanobu Asanuma 93bb368094 HDFS-14983. RBF: Add dfsrouteradmin -refreshSuperUserGroupsConfiguration command option. Contributed by Xieming Li 2019-12-12 10:56:19 +09:00
Mingliang Liu b56c08b2b7
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml (#1753) 
Contributed by Mingliang Liu
 2019-12-11 11:52:53 -08:00
Erik Krogen c174d50b30 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 2019-12-11 09:39:37 -08:00
Kevin Su e573ea4908
HADOOP-16754. Fix docker failed to build yetus/hadoop. 
Contributed by Kevin Su.

Change-Id: Ie9cf27b7a7ba5923405f24d39b95b0248f3718b7
 2019-12-11 16:56:25 +00:00
Surendra Singh Lilhore c2e9783d5f HDFS-15045. DataStreamer#createBlockOutputStream() should log exception in warn. Contributed by Ravuri Sushma Sree. 2019-12-11 08:52:55 +05:30
Stephen O'Donnell c93cb6790e HDFS-14854. Create improved decommission monitor implementation. Contributed by Stephen O'Donnell. 
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
 2019-12-10 17:16:43 -08:00
Gabor Bota 875a3e97dd
HADOOP-16424. S3Guard fsck: Check internal consistency of the MetadataStore (#1691). Contributed by Gabor Bota. 2019-12-10 15:51:49 +01:00
prabhujoseph d4bde134e3 YARN-7721. Fix TestContinuousScheduling fails sporadically with NPE. 
Contributed by Wilfred Spiegelenburg.
 2019-12-10 15:00:23 +05:30
Takanobu Asanuma c4733377d0
HDFS-15040. RBF: Secured Router should not run when SecretManager is not running. (#1745) 2019-12-10 16:03:32 +09:00
Akira Ajisaka 9f09852051
HDFS-15043. RBF: The detail of the Exception is not shown in ZKDelegationTokenSecretManagerImpl (#1748) 2019-12-10 16:00:20 +09:00
Masatake Iwasaki 4dffd81bb7
HDFS-14522. Allow compact property description in xml in httpfs. (#1737) 2019-12-10 10:30:57 +09:00
Akira Ajisaka dc66de7448
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena. 2019-12-09 18:37:34 +09:00
dependabot[bot] e9c5bb8078 Bump bower from 1.7.7 to 1.8.8 (#1683) 
Bumps [bower](https://github.com/bower/bower) from 1.7.7 to 1.8.8.
- [Release notes](https://github.com/bower/bower/releases)
- [Changelog](https://github.com/bower/bower/blob/master/CHANGELOG.md)
- [Commits](https://github.com/bower/bower/compare/v1.7.7...v1.8.8)

Signed-off-by: dependabot[bot] <support@github.com>
 2019-12-09 15:46:08 +09:00
Eric Badger 8dffd8dc89 YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-12-09 01:25:10 +00:00
Eric Badger ab2cc5ac92 Revert "YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger" 
This reverts commit 76bb297db9.
 2019-12-09 01:25:10 +00:00
dependabot[bot] c1d393a156 Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2019-12-09 10:14:44 +09:00
Jonathan Hung 6d970d8100 YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 2019-12-08 16:35:02 -08:00
Jonathan Hung cd4779f317 Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources" 
This reverts commit 92bce918dc.
 2019-12-08 16:34:46 -08:00
Jonathan Hung 92bce918dc YARN-10012. Guaranteed and max capacity queue metrics for custom resources 2019-12-08 15:52:17 -08:00
Masatake Iwasaki 11cd5b6e39 HDFS-15028. Keep the capacity of volume and reduce a system call. Contributed by Yang Yun. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-12-07 08:46:45 +09:00
Wei-Chiu Chuang ecd461f940 HDFS-14751. Synchronize on diffs in DirectoryScanner. Contributed by Lisheng Sun. 2019-12-06 15:10:40 -08:00
Wei-Chiu Chuang 313b76f8e9 HDFS-14476. lock too long when fix inconsistent blocks between disk and in-memory. Contributed by Sean Chow. 2019-12-06 15:10:40 -08:00
Eric Badger 76bb297db9 YARN-9561. Add C changes for the new RuncContainerRuntime. Contributed by Eric Badger 2019-12-06 18:33:58 +00:00
Ayush Saxena 705b172b95 HDFS-14998. [SBN read] Update Observer Namenode doc for ZKFC after HDFS-14130. Contributed by Fei Hui. 2019-12-06 23:04:09 +05:30
aasha fccccc9703 HDFS-14869 Copy renamed files which are not excluded anymore by filter (#1530) 2019-12-06 17:41:25 +05:30
prabhujoseph e575df5ca6 YARN-10006. IOException used in place of YARNException in CapaitySheduler 
Contributed by Adam Antal. Reviewed by Szilard Nemeth.
 2019-12-06 12:51:28 +05:30
Masatake Iwasaki 18059acb6a
HADOOP-16744. Fix building instruction to enable zstd. (#1736) 2019-12-06 15:25:20 +09:00
Mingliang Liu 19512b21e3
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:37:17 -08:00
Szilard Nemeth 4f758dd4c6 YARN-9607. Auto-configuring rollover-size of IFile format for non-appendable filesystems. Contributed by Adam Antal 2019-12-05 21:49:00 +01:00
Szilard Nemeth cf68857631 YARN-9938. Validate Parent Queue for QueueMapping contains dynamic group as parent queue. Contributed by Manikandan R 2019-12-05 21:04:42 +01:00
Szilard Nemeth 0ab9c0398d YARN-9969. Improve yarn.scheduler.capacity.queue-mappings documentation. Contributed by Manikandan R 2019-12-05 20:53:43 +01:00
Szilard Nemeth 5cc6f945da YARN-9789. Disable Option for Write Ahead Logs of LogMutation. Contributed by Prabhu Joseph 2019-12-05 20:43:26 +01:00
Szilard Nemeth 4627dd6708 YARN-9780. SchedulerConf Mutation API does not Allow Stop and Remove Queue in a single call. Contributed by Prabhu Joseph 2019-12-05 20:38:37 +01:00
Szilard Nemeth c71befaf8f YARN-9781. SchedConfCli to get current stored scheduler configuration. Contributed by Prabhu Joseph 2019-12-05 20:31:24 +01:00
Szilard Nemeth 520fe2c99b YARN-5106. Provide a builder interface for FairScheduler allocations for use in tests. Contributed by Adam Antal 2019-12-05 17:37:40 +01:00