Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,751 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

20751 Commits

Author SHA1 Message Date
Inigo Goiri 335aebb9c4 HDFS-14560. Allow block replication parameters to be refreshable. Contributed by Stephen O'Donnell. 
(cherry picked from commit 4f455290b1)
 2019-06-13 18:27:26 -07:00
Takanobu Asanuma 73829589b7 HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 
(cherry picked from commit 54f9f75a44)
 2019-06-14 10:18:46 +09:00
Inigo Goiri 65f7ec2f39 YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError. Contributed by Sushil Ks. 
(cherry picked from commit eeaf8edaa7)
 2019-06-13 14:22:16 -07:00
Sean Mackrory e0b3cbd221 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:53:40 -06:00
Stephen O'Donnell 71159f9540 HDFS-13231. Extend visualization for Decommissioning, Maintenance Mode under Datanode tab in the NameNode UI. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 50de0874d0)
 2019-06-12 10:31:03 -07:00
Sunil G 253dcde517 YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B. 
(cherry picked from commit 52128e352a)
 2019-06-12 19:25:02 +05:30
Sunil G 72203f7a12 YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-12 19:23:40 +05:30
Sunil G f1ead03672 Revert "YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl." 
This reverts commit f1d3a17d3e.
 2019-06-12 19:10:23 +05:30
Siyao Meng f4afeb6805 HADOOP-16263. Update BUILDING.txt with macOS native build instructions. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 4fecc2a95e)
 2019-06-11 15:06:06 -07:00
bibinchundatt 3303723f55 YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T. 2019-06-11 23:42:29 +05:30
bibinchundatt d9284d4a57 YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T. 
(cherry picked from commit 60c95e9b6a)
 2019-06-11 23:13:18 +05:30
bibinchundatt a37011bd5e YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3f)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java
 2019-06-11 23:01:03 +05:30
He Xiaoqiao d7560c866e HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bd46bdf9f9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2019-06-10 17:26:42 -07:00
Hanisha Koneru 210480a23c HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7217494f40)
 2019-06-10 13:46:53 -07:00
Erik Krogen f607efd195 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 
(cherry-picked from 76b94c274f)
 2019-06-07 14:22:08 -07:00
DadanielZ 9c8e40fbdb
HADOOP-16251. ABFS: add FSMainOperationsBaseTest. Re-commit to fix git metadata. 
Author: Da Zhou
(cherry picked from commit ff27e8eabd)
 2019-06-07 18:09:38 +01:00
Da Zhou bf0bb2470f
HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream. 
Contributed by Da Zhou.

(cherry picked from commit 1cef194a28)
 2019-06-07 18:09:38 +01:00
Vishwajeet Dusane 907a016142
HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file. 
Contributed by Vishwajeet Dusane.

(cherry picked from commit 1edf1914ac)
 2019-06-07 18:09:37 +01:00
Shweta Yakkali 6b115966bc
HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I6269ae66378e46eed440a76f847ae1af1fa95450
(cherry picked from commit bb8ad096e7)
 2019-06-07 18:09:37 +01:00
Shweta Yakkali 57c6060c3a
HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). 
Contributed by Shweta Yakkali.

(cherry picked from commit 13f0ee21f2)

Change-Id: Ibd010d2e6adc15f53a9c5357482e57313bf84d2e
 2019-06-07 18:09:37 +01:00
Da Zhou 3593b66693
HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI 
(Contributed by Da Zhou via Daniel Templeton)

Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843
(cherry picked from commit e374584479)
 2019-06-07 18:09:37 +01:00
Denes Gerencser ede5cbd707
HADOOP-16174. Disable wildfly logs to the console. 
Follow-on to HADOOP-15851.

Author:    Denes Gerencser <dgerencser@cloudera.com>
(cherry picked from commit ddede7ae6f)
 2019-06-07 18:09:37 +01:00
Steve Loughran 96489069b0
HADOOP-15851. Disable wildfly logs to the console. 
Contributed by Vishwajeet Dusane.

(cherry picked from commit ef9dc6c44c)
 2019-06-07 18:09:37 +01:00
Steve Loughran baa8670256
HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. 
Contributed by Da Zhou.

(cherry picked from commit bd50fa956b)
 2019-06-07 18:09:37 +01:00
Eric Yang 68aec0a98d YARN-9581. Fixed yarn logs cli to access RM2. 
Contributed by Prabhu Joseph

(cherry picked from commit cb9bc6e64c)
 2019-06-06 16:43:25 -04:00
Sunil G f1d3a17d3e YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-06 06:24:01 +05:30
Weiwei Yang 6e2b091515 YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang. 2019-06-04 15:24:37 +08:00
Sunil G 2f012044ff YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB. 
(cherry picked from commit 59719dc560)
 2019-06-03 15:54:07 +05:30
Sunil G 58042dadc3 YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB. 
(cherry picked from commit 7f46dda513)
 2019-06-03 12:25:16 +05:30
Weiwei Yang e027c87da2 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 
(cherry picked from commit 4530f4500d)
 2019-06-03 14:15:20 +08:00
Kihwal Lee 056bb773ce HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 
(cherry picked from commit 9122b9b649)
 2019-05-30 17:30:48 -05:00
Wanqiang Ji c076645e18
HADOOP-16334. Fix yetus-wrapper not working when HADOOP_YETUS_VERSION >= 0.9.0 
This closes #864

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d23e8e9232)
 2019-05-30 15:12:38 +09:00
Ayush Saxena 2cd7577a5a HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c1caab40f2)
 2019-05-29 20:58:11 -07:00
Eric Yang b2a39e8883 YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix. 
Contributed by Prabhu Joseph
 2019-05-29 18:04:13 -04:00
Eric E Payne 2e561cef47 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101)
 2019-05-29 18:43:13 +00:00
Ahmed Hussein 777f7345ef YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit abf76ac371)
 2019-05-29 12:14:01 -05:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka c917ba782e
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang. 
(cherry picked from commit 4a692bc3be)
 2019-05-28 16:54:43 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Erik Krogen c26cf22d6b HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f0)
 2019-05-24 13:18:49 -07:00
Konstantin V Shvachko 9836bac35c HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:49:06 -07:00
Shweta 7a5d9ab9ee HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93d7630062)
 2019-05-24 18:43:30 +02:00
Wanqiang Ji e4011e6886
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165)
 2019-05-22 14:25:34 +09:00
Siyao Meng a4f425d3ae HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 9dff6eff81)
 2019-05-22 13:24:58 +09:00
Srinivasu Majeti 85f3664bb8 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028a)
 2019-05-17 19:21:42 +02:00
Alexis Daboville 3f05b699ef
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac7)
 2019-05-17 15:16:59 +01:00
Andrew Olson 55603529d0
HADOOP-16294: Enable access to input options by DistCp subclasses. 
Adding a protected-scope getter for the DistCpOptions, so that a subclass does
not need to save its own copy of the inputOptions supplied to its constructor,
if it wishes to override the createInputFileListing method with logic similar
to the original implementation, i.e. calling CopyListing#buildListing with a path and input options.

Author:    Andrew Olson
(cherry picked from commit c15b3bca86)
 2019-05-16 16:13:12 +02:00
David Mollitor 6eb48542f1
HADOOP-16307. Intern User Name and Group Name in FileStatus. 
Author:    David Mollitor

(cherry picked from commit 2713dcf6e9)
 2019-05-16 16:03:06 +02:00
Bharat Viswanadham 2c72f48527
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy. 
(cherry picked from commit d4c8858586)
 2019-05-15 17:56:22 -07:00
bibinchundatt 71f5bfb822 YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T. 
(cherry picked from commit 570fa2da20)
 2019-05-15 13:31:07 +05:30