Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,578 Commits 326 Branches 371 Tags
Commit Graph

20578 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Steve Loughran
baa8670256
HADOOP-15825. ABFS: Enable some tests for namespace not enabled account using OAuth. 
Contributed by Da Zhou.

(cherry picked from commit bd50fa956b1ca25bb2136977b98a6aa6895eff8b)
 2019-06-07 18:09:37 +01:00
Eric Yang
68aec0a98d YARN-9581. Fixed yarn logs cli to access RM2. 
Contributed by Prabhu Joseph

(cherry picked from commit cb9bc6e64c590622ae04aea2c81962be59037f7a)
 2019-06-06 16:43:25 -04:00
Sunil G
f1d3a17d3e YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 2019-06-06 06:24:01 +05:30
Weiwei Yang
6e2b091515 YARN-9580. Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers. Contributed by Tao Yang. 2019-06-04 15:24:37 +08:00
Sunil G
2f012044ff YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB. 
(cherry picked from commit 59719dc560cf67f485d8e5b4a6f0f38ef97d536b)
 2019-06-03 15:54:07 +05:30
Sunil G
58042dadc3 YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB. 
(cherry picked from commit 7f46dda513fb79c349acb73bdb90b689df9cc18d)
 2019-06-03 12:25:16 +05:30
Weiwei Yang
e027c87da2 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 
(cherry picked from commit 4530f4500d308c9cefbcc5990769c04bd061ad87)
 2019-06-03 14:15:20 +08:00
Kihwal Lee
056bb773ce HDFS-14521. Suppress setReplication logging. Contributed by Kihwal Lee 
(cherry picked from commit 9122b9b6493d315c8910fbc06eaba96273ee335c)
 2019-05-30 17:30:48 -05:00
Wanqiang Ji
c076645e18
HADOOP-16334. Fix yetus-wrapper not working when HADOOP_YETUS_VERSION >= 0.9.0 
This closes #864

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d23e8e9232f030fcdeff91b9e8a625516244dc72)
 2019-05-30 15:12:38 +09:00
Ayush Saxena
2cd7577a5a HDFS-14512. ONE_SSD policy will be violated while write data with DistributedFileSystem.create(....favoredNodes). Contributed by Ayush Saxena. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit c1caab40f27e3e4f58ff1b5ef3e93efc56bbecbe)
 2019-05-29 20:58:11 -07:00
Eric Yang
b2a39e8883 YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix. 
Contributed by Prabhu Joseph
 2019-05-29 18:04:13 -04:00
Eric E Payne
2e561cef47 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101dc17038efb9f5345833d2883fce86e)
 2019-05-29 18:43:13 +00:00
Ahmed Hussein
777f7345ef YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit abf76ac371e4611c8eb371736b433e3d89c9d2ae)
 2019-05-29 12:14:01 -05:00
Takanobu Asanuma
a9a3450560 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:34:16 +09:00
Akira Ajisaka
c917ba782e
YARN-9500. Fix typos in ResourceModel.md. Contributed by leiqiang. 
(cherry picked from commit 4a692bc3be0800891ea58c13ba7d89efd358f6f6)
 2019-05-28 16:54:43 +09:00
Akira Ajisaka
855dc997d6
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:27:33 +09:00
Erik Krogen
c26cf22d6b HDFS-14500. NameNode StartupProgress should not allow new steps in an already-completed phase. Contributed by Erik Krogen. 
(cherry-picked from 55e0c134f002f74cb4a0360b6682a1b6796d1598)
 2019-05-24 13:18:49 -07:00
Konstantin V Shvachko
9836bac35c HDFS-14502. keepResults option in NNThroughputBenchmark should call saveNamespace(). Contributed by Konstantin V Shvachko. 2019-05-24 12:49:06 -07:00
Shweta
7a5d9ab9ee HDFS-14451. Incorrect header or version mismatch log message. Contributed by Shweta. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93d7630062247793a4860704fad4a31437693de1)
 2019-05-24 18:43:30 +02:00
Wanqiang Ji
e4011e6886
MAPREDUCE-7205. Treat container scheduler kill exit code as a task attempt killing event. 
This closes #821

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 67f9a7b165edecbec7c8063758202be4d8cff0f5)
 2019-05-22 14:25:34 +09:00
Siyao Meng
a4f425d3ae HDFS-14507. Document -blockingDecommission option for hdfs dfsadmin -listOpenFiles 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
(cherry picked from commit 9dff6eff819a832a5918d786675f42aa243a13e8)
 2019-05-22 13:24:58 +09:00
Srinivasu Majeti
85f3664bb8 HDFS-14323. Distcp fails in Hadoop 3.x when 2.x source webhdfs url has special characters in hdfs file path. Contributed by Srinivasu Majeti. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3e5e5b028ad7e199d08e524fe7cddeee5db51a6d)
 2019-05-17 19:21:42 +02:00
Alexis Daboville
3f05b699ef
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac703b10de59a8ca371fe957a4a338601)
 2019-05-17 15:16:59 +01:00
Andrew Olson
55603529d0
HADOOP-16294: Enable access to input options by DistCp subclasses. 
Adding a protected-scope getter for the DistCpOptions, so that a subclass does
not need to save its own copy of the inputOptions supplied to its constructor,
if it wishes to override the createInputFileListing method with logic similar
to the original implementation, i.e. calling CopyListing#buildListing with a path and input options.

Author:    Andrew Olson
(cherry picked from commit c15b3bca86a0f973ccdddd020f3ff2d5767ff1bd)
 2019-05-16 16:13:12 +02:00
David Mollitor
6eb48542f1
HADOOP-16307. Intern User Name and Group Name in FileStatus. 
Author:    David Mollitor

(cherry picked from commit 2713dcf6e9ef308ffe6102532c90b27c52d27f7c)
 2019-05-16 16:03:06 +02:00
Bharat Viswanadham
2c72f48527
HADOOP-16247. NPE in FsUrlConnection. Contributed by Karthik Palanisamy. 
(cherry picked from commit d4c8858586eeed2820f3ab21da79603b52c64594)
 2019-05-15 17:56:22 -07:00
bibinchundatt
71f5bfb822 YARN-9508. YarnConfiguration areNodeLabel enabled is costly in allocation flow. Contributed by Bilwa S T. 
(cherry picked from commit 570fa2da20706490dc7823efd0ce0cef3ddc81f9)
 2019-05-15 13:31:07 +05:30
Weiwei Yang
26eb9f52fb HADOOP-16306. AliyunOSS: Remove temporary files when upload small files to OSS. Contributed by wujinhu. 
(cherry picked from commit 2d8282bb8248e6984878626c4cdc7148aa2e7202)
 2019-05-14 14:06:42 -07:00
Sunil G
f4ee38df29 YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 
(cherry picked from commit 7d831eca645f93d064975ebae35a7cbea3bbad31)
 2019-05-14 10:49:09 -07:00
Sunil G
b5b152aded YARN-9504. [UI2] Fair scheduler queue view page does not show actual capacity. Contributed by Zoltan Siegl. 
(cherry picked from commit 64c7f36ab12faba26a2ecf1e6a0cab7eb2bcffdd)
 2019-05-10 14:28:33 +05:30
Rajat Khandelwal
12e0053932
HADOOP-16278. With S3A Filesystem, Long Running services End up Doing lot of GC and eventually die. 
Contributed by Rajat Khandelwal

(cherry picked from commit 591ca698230f25217c10c7549aff8097baa11f1e)
 2019-05-09 21:14:37 +01:00
Siyao Meng
095815220d HADOOP-16289. Allow extra jsvc startup option in hadoop_start_secure_daemon in hadoop-functions.sh. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 93f2283a69ea4e07a998f2a4065f238f9574921b)
 2019-05-06 15:47:58 -07:00
Haibo Chen
c6573562cb YARN-9529. Log correct cpu controller path on error while initializing CGroups. (Contributed by Jonathan Hung) 
(cherry picked from commit 597fa47ad125c0871f5c4deb3a883e5b3341c67b)
 2019-05-06 11:58:31 -07:00
Eric E Payne
6fce24fb40 YARN-9285: RM UI progress column is of wrong type. Contributed by Ahmed Hussein. 
(cherry picked from commit b094b94d43a46af9ddb910da24f792b95f614b08)
 2019-05-02 19:48:06 +00:00
Siyao Meng
bba8971897 HDFS-14463. Add Log Level link under NameNode and DataNode Web UI Utilities dropdown. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 7cb46f035a92056783bad23a9abc6a264d71285d)
 2019-05-01 16:49:45 -07:00
Arpit Agarwal
db4c0b357d
HDFS-13677. Dynamic refresh Disk configuration results in overwriting VolumeMap. Contributed by xuzq. 
(cherry picked from commit 4b4200f1f87ad40d9c19ba160f706ffd0470a8d4)
 2019-04-29 14:54:30 -07:00
Weiwei Yang
cc0c85f04a YARN-9325. TestQueueManagementDynamicEditPolicy fails intermittent. Contributed by Prabhu Joseph. 
(cherry picked from commit 1c8046d67ec10710e7749ed1929b09fac4b1ba94)
 2019-04-23 14:24:15 +08:00
Erik Krogen
28ff96f367 HADOOP-16265. Fix bug causing Configuration#getTimeDuration to use incorrect units when the default value is used. Contributed by starphin. 
(cherry-picked from 1ddb48872f6a4985f4d0baadbb183899226cff68)
 2019-04-22 08:18:17 -07:00
Eric Yang
ac85aa80d9 YARN-8587. Added retries for fetching docker exit code. 
Contributed by Charo Zhang

(cherry picked from commit c16c49b8c3b8e2e42c00e79a50e7ae029ebe98e2)
 2019-04-19 15:40:23 -04:00
Eric Yang
4a64dab0dd YARN-8622. Fixed container-executor compilation on MacOSX. 
Contributed by Siyao Meng

(cherry picked from commit ef97a20831677c055aa6bff6ad0649cbb3a56a86)
 2019-04-18 19:01:11 -04:00
Eric Yang
2503409977 YARN-6695. Fixed NPE in publishing appFinished events to ATSv2. 
Contributed by Prabhu Joseph

(cherry picked from commit df76cdc8959c51b71704ab5c38335f745a6f35d8)
 2019-04-18 12:30:55 -04:00
Siyao Meng
742a3ad24b YARN-9487. NodeManager native build shouldn't link against librt on macOS. Contributed by Siyao Meng. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 6e4399ea61eb4301f4b6c59633d77c0a04efb12b)
 2019-04-17 22:57:33 -07:00
Puleya7
d724f74f6b HDFS-14432. dfs.datanode.shared.file.descriptor.paths duplicated in hdfs-default.xml 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 13907d847986aa14a51cc2a3cc83151f22c43781)
 2019-04-17 21:55:45 +09:00
Akira Ajisaka
df5d8f05d9
HADOOP-16227. Upgrade checkstyle to 8.19 
(cherry picked from commit 4b4fef2f0e0ed1e185ea1058db7a65d68d4970b9)
 2019-04-15 10:47:02 +09:00
Masatake Iwasaki
03079be707 HADOOP-14544. DistCp documentation for command line options is misaligned. Contributed by Masatake Iwasaki. 
(cherry picked from commit bbdbc7a9a158f36955c2253acb0edb14219ccb04)
 2019-04-12 11:59:14 +09:00
Weiwei Yang
db185de31c YARN-9463. Add queueName info when failing with queue capacity sanity check. Contributed by Aihua Xu. 
(cherry picked from commit 8c1bba375b144fd515b389174ddb349f2d9246fa)
 2019-04-10 23:02:24 +08:00
Akira Ajisaka
b28fdb98ee
HADOOP-14635. Javadoc correction for AccessControlList#buildACL. Contributed by Yeliang Cang. 
(cherry picked from commit ab645b3caa0313f08a26bdd70fa14434d09e6cb5)
 2019-04-08 15:19:17 +09:00
Weiwei Yang
7a80b1b481 YARN-9413. Queue resource leak after app fail for CapacityScheduler. Contributed by Tao Yang. 
(cherry picked from commit ec143cbf678bd65f87fdd464c23022a2d2c54c07)
 2019-04-06 20:19:03 +08:00
Akira Ajisaka
f020ac2c06
HDFS-14407. Fix misuse of SLF4j logging API in DatasetVolumeChecker#checkAllVolumes. Contributed by Wanqiang Ji. 
(cherry picked from commit 67020f09502a4f07342dee457e47bb52b03441ae)
 2019-04-05 12:30:46 +09:00
David Mollitor
50fc9577c0
HADOOP-16208. Do Not Log InterruptedException in Client. 
Contributed by David Mollitor.

(cherry picked from commit c90736350ba158c7872a39426e7a29c5e5e0bb48)
 2019-04-04 16:16:57 +01:00