Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,070 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

21070 Commits

Author SHA1 Message Date
Eric Badger 355ec33416 YARN-10009. In Capacity Scheduler, DRC can treat minimum user limit percent as a max when custom resource is defined. Contributed by Eric Payne 2019-12-20 19:32:36 +00:00
Eric Badger 11c5396b4a Revert "HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui." 
This reverts commit 59cdf18d5a.
 2019-12-20 19:04:45 +00:00
Akira Ajisaka 7201384e1b
HADOOP-16771. Update checkstyle to 8.26 and maven-checkstyle-plugin to 3.1.0. Contributed by Andras Bokor. 
(cherry picked from commit f777cd398f)
 2019-12-20 13:12:02 +09:00
Inigo Goiri 59cdf18d5a HDFS-15062. Add LOG when sendIBRs failed. Contributed by Fei Hui. 
(cherry picked from commit 52d7b745c6)
 2019-12-19 09:43:40 -08:00
Jonathan Hung 0707d0a0ae YARN-9894. CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues. Contributed by Eric Payne 
(cherry picked from commit 7b93575b92)
 2019-12-18 13:18:22 -08:00
Jonathan Hung 01e2c996ff YARN-10039. Allow disabling app submission from REST endpoints 
(cherry picked from commit fddc3d55c3)
 2019-12-18 10:51:57 -08:00
Shashikant Banerjee afe00a1ca5 HDFS-15012. NN fails to parse Edit logs after applying HDFS-13101. Contributed by Shashikant Banerjee. 
(cherry picked from commit fdd96e46d1)
 2019-12-18 23:10:00 +05:30
Eric Badger 18b515322c YARN-10033. TestProportionalCapacityPreemptionPolicy not initializing vcores for effective max resources. Contributed by Eric Payne. 
(cherry picked from commit f47dcf2d4c)
 2019-12-17 17:36:52 +00:00
Chen Liang d4a6901c42 HDFS-15036. Active NameNode should not silently fail the image transfer. Contributed by Chen Liang. 
(cherry picked from commit 65c4660bcd)
 2019-12-12 15:57:15 -08:00
He Xiaoqiao 8ba4e13dcc HDFS-15050. Optimize log information when DFSInputStream meet CannotObtainBlockLengthException. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 99af49878cc47f5750b6c0edadc8a4c2fcd7a7c6)
 2019-12-11 16:34:28 -08:00
Mingliang Liu d19981fe48
HADOOP-16758. Refine testing.md to tell user better how to use auth-keys.xml (#1753) 
Contributed by Mingliang Liu
 2019-12-11 11:54:12 -08:00
Erik Krogen 3402c87353 HDFS-15032. Properly handle InvocationTargetExceptions in the proxy created by ProxyCombiner. This fixes a bug encountered by the HDFS balancer when used with Observer Nodes. Contributed by Erik Krogen. 
(cherry picked from c174d50b30)
 2019-12-11 09:40:36 -08:00
Kevin Su 0f25cbbb25 HADOOP-16754. Fix docker failed to build yetus/hadoop. 
Contributed by Kevin Su.

Change-Id: Ie9cf27b7a7ba5923405f24d39b95b0248f3718b7
 2019-12-11 16:57:30 +00:00
Akira Ajisaka a3cdba7713
YARN-9985. Unsupported transitionToObserver option displaying for rmadmin command. Contributed by Ayush Saxena. 
(cherry picked from commit dc66de7448)
 2019-12-09 18:38:47 +09:00
Sneha Vijayarajan aa9cd0a2d6
HADOOP-16660. ABFS: Make RetryCount in ExponentialRetryPolicy Configurable. 
Contributed by Sneha Vijayarajan.
 2019-12-08 21:32:13 -08:00
bilaharith c225efe237
HADOOP-16455. ABFS: Implement FileSystem.access() method. 
Contributed by Bilahari T H.
 2019-12-08 21:32:02 -08:00
Jeetesh Mangwani b1e748f45b
HADOOP-16612. Track Azure Blob File System client-perceived latency 
Contributed by Jeetesh Mangwani.

This add the ability to track the end-to-end performance of ADLS Gen 2 REST APIs by measuring latency in the Hadoop ABFS driver.
The latency information is sent back to the ADLS Gen 2 REST API endpoints in the subsequent requests.
 2019-12-08 21:31:51 -08:00
bilaharith ffeb6d8ece
HADOOP-16587. Make ABFS AAD endpoints configurable. 
Contributed by Bilahari T H.

This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate
in China.

Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c
 2019-12-08 21:31:39 -08:00
Sneha Vijayarajan 8b2c7e0c4d
HADOOP-16578 : Avoid FileSystem API calls when FileSystem already exists 2019-12-08 21:31:24 -08:00
Sneha Vijayarajan 546db6428e
HADOOP-16548 : Disable Flush() over config 2019-12-08 21:31:08 -08:00
dependabot[bot] 26367b6cc7
Bump nimbus-jose-jwt from 4.41.1 to 7.9 (#1682) 
Bumps [nimbus-jose-jwt](https://bitbucket.org/connect2id/nimbus-jose-jwt) from 4.41.1 to 7.9.
- [Changelog](https://bitbucket.org/connect2id/nimbus-jose-jwt/src/master/CHANGELOG.txt)
- [Commits](https://bitbucket.org/connect2id/nimbus-jose-jwt/branches/compare/7.9..4.41.1)

Signed-off-by: dependabot[bot] <support@github.com>
(cherry picked from commit c1d393a156)
 2019-12-09 10:19:13 +09:00
Jonathan Hung 9228e3f0ad YARN-10012. Guaranteed and max capacity queue metrics for custom resources. Contributed by Manikandan R 
(cherry picked from commit 92bce918dc)
 2019-12-08 16:36:08 -08:00
Jonathan Hung 8badcd989e Revert "YARN-10012. Guaranteed and max capacity queue metrics for custom resources" 
This reverts commit b5235f1ed0.
 2019-12-08 16:36:00 -08:00
Jonathan Hung b5235f1ed0 YARN-10012. Guaranteed and max capacity queue metrics for custom resources 
(cherry picked from commit 92bce918dc)
 2019-12-08 16:03:34 -08:00
Sunil G 69dc329acc YARN-4901. QueueMetrics needs to be cleared before MockRM is initialized. Contributed by Peter Bacsko. 
(cherry picked from commit 002dcc4ebf)
 2019-12-08 14:40:45 -08:00
Masatake Iwasaki e83684eae3 HADOOP-16744. Fix building instruction to enable zstd. (#1736) 
(cherry picked from commit 18059acb6a)
 2019-12-06 15:27:46 +09:00
Mingliang Liu 8a60429e0b
HADOOP-16735. Make it clearer in config default that EnvironmentVariableCredentialsProvider supports AWS_SESSION_TOKEN. Contributed by Mingliang Liu 
This closes #1733
 2019-12-05 17:50:28 -08:00
Tamás Pénzes dcb0454bb9 HADOOP-16729. Extract version numbers to head of pom.xml (#1732). Contributed by Tamas Penzes. 2019-12-04 10:35:21 +01:00
Aravindan Vijayan f0c1403ec3 HADOOP-16718. Allow disabling Server Name Indication (SNI) for Jetty. Contributed by Aravindan Vijayan. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Siyao Meng <smeng@cloudera.com>
(cherry picked from commit f1ab7f18c4)
 2019-12-03 15:48:15 -08:00
Erik Krogen c1ed247459 HADOOP-16700. Track response time separately in Server.Call to avoid negative RpcQueueTime values. Contributed by xuzq. 
(cherry picked from be77231452)
 2019-12-03 14:01:16 -08:00
Sahil Takiar c47d4d9c3f HADOOP-16685: FileSystem#listStatusIterator does not check if given path exists (#1695) 
(cherry picked from commit 3161813482)
 2019-12-03 13:34:41 -08:00
Surendra Singh Lilhore 298cda22a3 HDFS-15010. BlockPoolSlice#addReplicaThreadPool static pool should be initialized by static method. Contributed by Surendra Singh Lilhore. 2019-11-28 23:58:46 +05:30
Szilard Nemeth 2ad7b90505 YARN-9993. Remove incorrectly committed files from YARN-9011. Contributed by Wilfred Spiegelenburg 2019-11-28 12:39:54 +01:00
prabhujoseph e8304426b1 MAPREDUCE-7249. Fix Invalid event TA_TOO_MANY_FETCH_FAILURE at SUCCESS_CONTAINER_CLEANUP causes job failure. 
Contributed by Wilfred Spiegelenburg.
 2019-11-28 14:32:28 +05:30
Sunil G f9b872b6ec YARN-9949. Add missing queue configs for root queue in RMWebService#CapacitySchedulerInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 23:14:33 +05:30
Szilard Nemeth 8eda9fcab8 YARN-9937. Add missing queue configs in RMWebService#CapacitySchedulerQueueInfo. 
Contributed by Prabhu Joseph.
 2019-11-27 22:24:06 +05:30
prabhujoseph 66e58b5525 MAPREDUCE-7240. Fix Invalid event: TA_TOO_MANY_FETCH_FAILURE at SUCCESS_FINISHING_CONTAINER. 
Contributed by Huachao and Peter Bacsko. Reviewed by Wilfred Spiegelenburg.
 2019-11-27 16:09:24 +05:30
Szilard Nemeth 3fc8930129 YARN-9011. Race condition during decommissioning. Contributed by Peter Bacsko 2019-11-26 14:26:58 +01:00
HUAN-PING SU 59a6261e81
YARN-9966. Code duplication in UserGroupMappingPlacementRule (#1709) 
(cherry picked from commit f8e36e03b4)
 2019-11-25 15:29:37 +09:00
Ayush Saxena 32fdb3004a HDFS-14519. NameQuota is not update after concat operation, so namequota is wrong. Contributed by Ranith Sardar. 2019-11-23 02:36:38 +05:30
Szilard Nemeth dcc453b4b8 YARN-9968. Public Localizer is exiting in NodeManager due to NullPointerException. Contributed by Tarun Parimi 2019-11-22 12:59:35 +01:00
Tao Yang af495192a5 YARN-9838. Fix resource inconsistency for queues when moving app with reserved container to another queue. Contributed by jiulongzhu. 2019-11-22 16:14:16 +08:00
Surendra Singh Lilhore 39366b1370 HDFS-14940. HDFS Balancer : Do not allow to set balancer maximum network bandwidth more than 1TB. Contributed by hemanthboyina. 2019-11-22 00:33:42 +05:30
Wei-Chiu Chuang bd2f79cfd8 HADOOP-15686. Supress bogus AbstractWadlGeneratorGrammarGenerator in KMS stderr. Contributed by Wei-Chiu Chuang. 
Reviewed-by: Xiaoyu Yao <xyao@apache.org>
(cherry picked from commit 9664b9c7a6)
 2019-11-20 13:08:40 -08:00
Wanqiang Ji 4c79e2a02b
HADOOP-16719. Remove the disallowed element config within maven-checkstyle-plugin (#1724) 
(cherry picked from commit dfc61d8ea5)
 2019-11-20 16:19:45 +09:00
He Xiaoqiao 5fe9b81941 HDFS-14952. Skip safemode if blockTotal is 0 in new NN. Contributed by Xiaoqiao He. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Reviewed-by: Mukul Kumar Singh <msingh@apache.org>
(cherry picked from commit 0b50aa29fd)
 2019-11-19 12:22:30 -08:00
Szilard Nemeth 62622ab9c1 YARN-9836. General usability improvements in showSimulationTrace.html. Contributed by Adam Antal 2019-11-19 21:21:17 +01:00
Eric Yang 6951689f4c YARN-9983. Fixed typo in YARN Service overview. 
Contributed by Denes Gerencser
 2019-11-19 14:19:49 -05:00
Abhishek Modi 31591bb296 YARN-9791. Queue Mutation API does not allow to remove a config. Contributed by Prabhu Joseph. 
(cherry picked from commit 751b5a1ac8)
 2019-11-19 16:49:14 +05:30
Sunil G b04f152876 YARN-9909. Offline Format of YarnConfigurationStore. Contributed by Prabhu Joseph 2019-11-19 16:10:02 +05:30