Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,789 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

20789 Commits

Author SHA1 Message Date
He Xiaoqiao d7560c866e HDFS-14553. Make queue size of BlockReportProcessingThread configurable. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit bd46bdf9f9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
 2019-06-10 17:26:42 -07:00
Hanisha Koneru 210480a23c HDFS-10659. Namenode crashes after Journalnode re-installation in an HA cluster due to missing paxos directory. Contributed by star, Hanisha Koneru. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7217494f40)
 2019-06-10 13:46:53 -07:00
Erik Krogen f607efd195 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 
(cherry-picked from 76b94c274f)
 2019-06-07 14:22:08 -07:00
DadanielZ 9c8e40fbdb
HADOOP-16251. ABFS: add FSMainOperationsBaseTest. Re-commit to fix git metadata. 
Author: Da Zhou
(cherry picked from commit ff27e8eabd)
 2019-06-07 18:09:38 +01:00
Da Zhou bf0bb2470f
HADOOP-16242. ABFS: add bufferpool to AbfsOutputStream. 
Contributed by Da Zhou.

(cherry picked from commit 1cef194a28)
 2019-06-07 18:09:38 +01:00
Vishwajeet Dusane 907a016142
HADOOP-16182. Update abfs storage back-end with "close" flag when application is done writing to a file. 
Contributed by Vishwajeet Dusane.

(cherry picked from commit 1edf1914ac)
 2019-06-07 18:09:37 +01:00
Shweta Yakkali 6b115966bc
HADOOP-16157. [Clean-up] Remove NULL check before instanceof in AzureNativeFileSystemStore 
(Contributed by Shweta Yakkali via Daniel Templeton)

Change-Id: I6269ae66378e46eed440a76f847ae1af1fa95450
(cherry picked from commit bb8ad096e7)
 2019-06-07 18:09:37 +01:00
Shweta Yakkali 57c6060c3a
HADOOP-15860. ABFS: Throw exception when directory / file name ends with a period (.). 
Contributed by Shweta Yakkali.

(cherry picked from commit 13f0ee21f2)

Change-Id: Ibd010d2e6adc15f53a9c5357482e57313bf84d2e
 2019-06-07 18:09:37 +01:00
Da Zhou 3593b66693
HADOOP-15823. ABFS: Stop requiring client ID and tenant ID for MSI 
(Contributed by Da Zhou via Daniel Templeton)

Change-Id: I546ab3a1df1efec635c08c388148e718dc4a9843
(cherry picked from commit e374584479)
 2019-06-07 18:09:37 +01:00
Denes Gerencser ede5cbd707
HADOOP-16174. Disable wildfly logs to the console. 
Follow-on to HADOOP-15851.

Author:    Denes Gerencser <dgerencser@cloudera.com>
(cherry picked from commit ddede7ae6f)
 2019-06-07 18:09:37 +01:00
Steve Loughran 96489069b0
HADOOP-15851. Disable wildfly logs to the console. 
Contributed by Vishwajeet Dusane.

(cherry picked from commit ef9dc6c44c)
 2019-06-07 18:09:37 +01:00
Steve Loughran baa8670256
HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. 
Contributed by Da Zhou.

(cherry picked from commit bd50fa956b)
 2019-06-07 18:09:37 +01:00
Eric Yang 68aec0a98d YARN-9581. Fixed yarn logs cli to access RM2. 
Contributed by Prabhu Joseph

(cherry picked from commit cb9bc6e64c)
 2019-06-06 16:43:25 -04:00
Sunil G f1d3a17d3e YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-06 06:24:01 +05:30
Weiwei Yang 6e2b091515 YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang. 2019-06-04 15:24:37 +08:00
Sunil G 2f012044ff YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB. 
(cherry picked from commit 59719dc560)
 2019-06-03 15:54:07 +05:30
Sunil G 58042dadc3 YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB. 
(cherry picked from commit 7f46dda513)
 2019-06-03 12:25:16 +05:30
Weiwei Yang e027c87da2 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 
(cherry picked from commit 4530f4500d)
 2019-06-03 14:15:20 +08:00
Kihwal Lee 056bb773ce HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 
(cherry picked from commit 9122b9b649)
 2019-05-30 17:30:48 -05:00
Wanqiang Ji c076645e18
HADOOP-16334. Fix yetus-wrapper not working when HADOOP_YETUS_VERSION >= 0.9.0 
This closes #864

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d23e8e9232)
 2019-05-30 15:12:38 +09:00
Ayush Saxena 2cd7577a5a HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c1caab40f2)
 2019-05-29 20:58:11 -07:00
Eric Yang b2a39e8883 YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix. 
Contributed by Prabhu Joseph
 2019-05-29 18:04:13 -04:00
Eric E Payne 2e561cef47 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101)
 2019-05-29 18:43:13 +00:00
Ahmed Hussein 777f7345ef YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit abf76ac371)
 2019-05-29 12:14:01 -05:00
Takanobu Asanuma a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka c917ba782e
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang. 
(cherry picked from commit 4a692bc3be)
 2019-05-28 16:54:43 +09:00
Akira Ajisaka 855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Erik Krogen c26cf22d6b HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f0)
 2019-05-24 13:18:49 -07:00
Konstantin V Shvachko 9836bac35c HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:49:06 -07:00
Shweta 7a5d9ab9ee HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93d7630062)
 2019-05-24 18:43:30 +02:00
Wanqiang Ji e4011e6886
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165)
 2019-05-22 14:25:34 +09:00
Siyao Meng a4f425d3ae HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 9dff6eff81)
 2019-05-22 13:24:58 +09:00
Srinivasu Majeti 85f3664bb8 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028a)
 2019-05-17 19:21:42 +02:00
Alexis Daboville 3f05b699ef
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac7)
 2019-05-17 15:16:59 +01:00
Andrew Olson 55603529d0
HADOOP-16294: Enable access to input options by DistCp subclasses. 
Adding a protected-scope getter for the DistCpOptions, so that a subclass does
not need to save its own copy of the inputOptions supplied to its constructor,
if it wishes to override the createInputFileListing method with logic similar
to the original implementation, i.e. calling CopyListing#buildListing with a path and input options.

Author:    Andrew Olson
(cherry picked from commit c15b3bca86)
 2019-05-16 16:13:12 +02:00
David Mollitor 6eb48542f1
HADOOP-16307. Intern User Name and Group Name in FileStatus. 
Author:    David Mollitor

(cherry picked from commit 2713dcf6e9)
 2019-05-16 16:03:06 +02:00
Bharat Viswanadham 2c72f48527
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy. 
(cherry picked from commit d4c8858586)
 2019-05-15 17:56:22 -07:00
bibinchundatt 71f5bfb822 YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T. 
(cherry picked from commit 570fa2da20)
 2019-05-15 13:31:07 +05:30
Weiwei Yang 26eb9f52fb HADOOP-16306. AliyunOSS: Remove temporary files when upload small files to OSS. Contributed by wujinhu. 
(cherry picked from commit 2d8282bb82)
 2019-05-14 14:06:42 -07:00
Sunil G f4ee38df29 YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 
(cherry picked from commit 7d831eca64)
 2019-05-14 10:49:09 -07:00
Sunil G b5b152aded YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl. 
(cherry picked from commit 64c7f36ab1)
 2019-05-10 14:28:33 +05:30
Rajat Khandelwal 12e0053932
HADOOP-16278. With S3A Filesystem, Long Running services End up Doing lot of GC and eventually die. 
Contributed by Rajat Khandelwal

(cherry picked from commit 591ca69823)
 2019-05-09 21:14:37 +01:00
Siyao Meng 095815220d HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93f2283a69)
 2019-05-06 15:47:58 -07:00
Haibo Chen c6573562cb YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad1)
 2019-05-06 11:58:31 -07:00
Eric E Payne 6fce24fb40 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 
(cherry picked from commit b094b94d43)
 2019-05-02 19:48:06 +00:00
Siyao Meng bba8971897 HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7cb46f035a)
 2019-05-01 16:49:45 -07:00
Arpit Agarwal db4c0b357d
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq. 
(cherry picked from commit 4b4200f1f8)
 2019-04-29 14:54:30 -07:00
Weiwei Yang cc0c85f04a YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph. 
(cherry picked from commit 1c8046d67e)
 2019-04-23 14:24:15 +08:00
Erik Krogen 28ff96f367 HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin. 
(cherry-picked from 1ddb48872f)
 2019-04-22 08:18:17 -07:00
Eric Yang ac85aa80d9 YARN-8587. Added retries for fetching docker exit code. 
Contributed by Charo Zhang

(cherry picked from commit c16c49b8c3)
 2019-04-19 15:40:23 -04:00