Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,011 Commits 326 Branches 371 Tags
Commit Graph

21011 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Szilard Nemeth
4bb238c480 YARN-9133. Make tests more easy to comprehend in TestGpuResourceHandler. Contributed by Peter Bacsko 2019-08-14 17:16:54 +02:00
Szilard Nemeth
4dc477b606 YARN-9140. Code cleanup in ResourcePluginManager.initialize and in TestResourcePluginManager. Contributed by Peter Bacsko 2019-08-14 17:01:41 +02:00
Szilard Nemeth
9a87e74e54 YARN-9134. No test coverage for redefining FPGA / GPU resource types in TestResourceUtils. Contributed by Peter Bacsko 2019-08-14 16:46:34 +02:00
Wei-Chiu Chuang
f5661b630a HDFS-14491. More Clarity on Namenode UI Around Blocks and Replicas. Contributed by Siyao Meng. 
(cherry picked from commit 6a43d0fbd49b3ff1ce75a2334b51a98ae476e473)
 2019-08-13 17:15:19 -07:00
Masatake Iwasaki
d7ca016d63 HDFS-14423. Percent (%) and plus (+) characters no longer work in WebHDFS. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit da0006fe0473e353ee2d489156248a01aa982dfd)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeHttpServer.java
 2019-08-14 08:45:10 +09:00
Siyao Meng
88aece4083 HDFS-14665. HttpFS: LISTSTATUS response is missing HDFS-specific fields (#1267) Contributed by Siyao Meng. 
(cherry picked from commit 6ae8bc3a4a07c6b4e7060362b749be8c7afe0560)
 2019-08-13 16:36:52 -07:00
Eric Badger
cec71691be YARN-9442. container working directory has group read permissions. Contributed by Jim Brennan. 
(cherry picked from commit 2ac029b949f041da2ee04da441c5f9f85e1f2c64)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
 2019-08-13 16:34:29 +00:00
Siyao Meng
c5f433b11f HDFS-14148. HDFS OIV ReverseXML SnapshotSection parser throws exception when there are more than one snapshottable directory (#1274) Contributed by Siyao Meng. 
(cherry picked from commit c92b49876a078ce7fb4e2a852e315de5b6410082)
 2019-08-12 17:30:18 -07:00
Szilard Nemeth
c5aea8ca56 YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph 
(cherry picked from commit e4b538bbda6dc25d7f45bffd6a4ce49f3f84acdc)
 2019-08-12 15:16:18 +02:00
Szilard Nemeth
844259203f YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph 
(cherry picked from commit b91099efd6e1fdcb31ec4ca7142439443c9ae536)
 2019-08-12 15:06:16 +02:00
Szilard Nemeth
b20fd9e212 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko 2019-08-12 14:02:17 +02:00
Da Zhou
330e450397
HADOOP-16315. ABFS: transform full UPN for named user in AclStatus 
Contributed by Da Zhou

Change-Id: Ibc78322415fcbeff89c06c8586c53f5695550290
 2019-08-12 09:41:52 +08:00
Sunil G
02b4635ff0 YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl. 
(cherry picked from commit 1c5b28659fe1310030d14e0be40dcd77b25056d6)
 2019-08-11 11:49:25 +05:30
Yiqun Lin
a7bc09c596 HDFS-13359. DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream. Contributed by Yiqun Lin. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 8a77a224c734bea0eb490f30c908748458c190c3)
 2019-08-09 18:41:03 -07:00
Szilard Nemeth
2e6beb1550 Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 
(cherry picked from commit 54ac80176e8487b7a18cd9e16a11efa289d0b7df)
 2019-08-09 13:20:10 +02:00
Sunil G
9fb6c6e2a1 YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB. 
(cherry picked from commit acffec7a92be540aa8531dbe06a3ea7bb813ab93)
 2019-08-09 16:07:04 +05:30
Szilard Nemeth
3e9071207a SUBMARINE-57. Add more elaborate message if submarine command is not recognized. Contributed by Adam Antal 
(cherry picked from commit e5f4cd0fdae7e689789dd74bfbcfa6c52895f037)
 2019-08-09 12:14:49 +02:00
Adam Antal
4c4f7d9c80 YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently (#1121) 
(cherry picked from commit cbcada804d119b837ad99de71d7f44cb4629026e)
 2019-08-09 11:43:30 +02:00
Szilard Nemeth
02d0e54596 YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak 
(cherry picked from commit e0c21c6da91776caf661661a19c368939c81fcc4)
 2019-08-09 10:23:10 +02:00
Szilard Nemeth
f0dfb8b832 YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak 
(cherry picked from commit 742e30b47381ad63e2b2fe63738cd0fe6cbce106)
 2019-08-09 10:02:35 +02:00
Szilard Nemeth
3bcf44f070 YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak 
(cherry picked from commit 72d7e570a73989aa18b737c0e642d570a55c6781)
 2019-08-09 09:50:32 +02:00
Wei-Chiu Chuang
2f2fa3dad6 HDFS-14693. NameNode should log a warning when EditLog IPC logger's pending size exceeds limit. Contributed by Xudong Cao. 
(cherry picked from commit 6ad9a11494c3aea146d7741bf0ad52ce16ad08e6)
 2019-08-08 13:51:14 -07:00
Stephen O'Donnell
a00a327594 HDFS-14459. ClosedChannelException silently ignored in FsVolumeList.addBlockPool(). Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit b0799148cf6e92be540f5665bb571418b916d789)
 2019-08-08 13:37:10 -07:00
Chao Sun
d38b617baa HDFS-14034. Support getQuotaUsage API in WebHDFS. Contributed by Chao Sun. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>

(cherry picked from 3ae775d74029b6ae82263739f598ceb25c597dcd)
 2019-08-08 10:15:48 -07:00
Eric E Payne
e47c483d9f YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang. 
(cherry picked from commit 3b38f2019e4f8d056580f3ed67ecef591011d7a6)
 2019-08-08 12:54:31 +00:00
Erik Krogen
016aa13940 HDFS-14370. Add exponential backoff to the edit log tailer to avoid spinning on empty edit tail requests. Contributed by Erik Krogen. 
(cherry picked from 827dbb11e24be294b40088a8aa46086ba8ca4ba8)
 2019-08-07 09:32:32 -07:00
Akira Ajisaka
8e59ff92b8
HADOOP-15908. hadoop-build-tools jar is downloaded from remote repository instead of using from local. Contributed by Oleksandr Shevchenko. 
(cherry picked from commit 0b0ba70b35d1c1c774d69f5682f24967d40009a8)

Conflicts:
	hadoop-project/pom.xml
 2019-08-07 09:33:39 +09:00
Lokesh Jain
f6fe18a282 HDFS-14692. Upload button should not encode complete url. Contributed by Lokesh Jain. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 954ff36360e083010f395cb9d3bd46701417e7b7)
 2019-08-06 15:31:05 -07:00
Haibo Chen
8d357343c4 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)
 2019-08-06 14:59:49 -07:00
Inigo Goiri
3e8e1570f6 HDFS-14513. FSImage which is saving should be clean while NameNode shutdown. Contributed by He Xiaoqiao. 
(cherry picked from 5740eea0818f2ba334b99eed0f085300e6b16905)
 2019-08-06 09:26:01 -07:00
Stephen O'Donnell
8414e6354b HDFS-14557. JournalNode error: Can't scan a pre-transactional edit log. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 35e0a01d7bc3149bf8b941fd85d7807d9906d27b)
 2019-08-06 07:58:12 -07:00
Jianfei Jiang
0c02217d9b
HDFS-14691. Wrong usage hint for hadoop fs command "test". 
Contributed by Jianfei Jiang.

Change-Id: I9f5e89721ff210641375fbf42a70043f0d74458e
(cherry picked from commit 71aad60e519da729249d3c06389f2efa8d9ae212)
 2019-08-05 13:09:39 +01:00
Erik Krogen
2634aaaf9a HDFS-14462 Ensure WebHDFS client throws the correct exception during writes. Contributed by Simbarashe Dzinamarira. 
(cherry picked from e7a0b8aa83c1fb933d409c514d2155e986e4e25b)
 2019-08-02 12:00:48 -07:00
Wei-Chiu Chuang
e1659fe009 HDFS-14685. DefaultAuditLogger doesn't print CallerContext. Contributed by xuzq. 
(cherry picked from commit 6757fc1a1d40ffc771bb685886b0d4a16315eca5)
 2019-08-02 10:51:55 -07:00
Wei-Chiu Chuang
490b18fd4f HADOOP-15681. AuthenticationFilter should generate valid date format for Set-Cookie header regardless of default Locale. Contributed by Cao Manh Dat. 
(cherry picked from commit e20b19543ba148cf0242884a3e553ad2f39421b3)
 2019-08-01 17:36:48 -07:00
Wei-Chiu Chuang
5c1f91fd97 HDFS-14631.The DirectoryScanner doesn't fix the wrongly placed replica. Contributed by Jinglun. 
(cherry picked from commit 32607dbd98a7ab70741a2efc98eff548c1e431c1)
 2019-08-01 15:49:05 -07:00
He Xiaoqiao
a405db8014 HDFS-13529. Fix default trash policy emptier trigger time correctly. Contributed by He Xiaoqiao. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit f86de6f76a3079c2655df9b242fc968edfb17b9d)
 2019-08-01 14:56:38 -07:00
Szilard Nemeth
e611fb878b MAPREDUCE-7225: Fix broken current folder expansion during MR job start. Contributed by Peter Bacsko. 2019-08-01 16:05:28 +02:00
Christopher Gregorian
d4492bdd9e HADOOP-16459. Backport of HADOOP-16266. Add more fine-grained processing time metrics to the RPC layer. Contributed by Christopher Gregorian. 
This commit also includes the follow-on commit 827a84778a4e3b8f165806dfd2966f0951a5e575.

(cherry-picked from f96a2df38d889f29314c57f4d94227b2e419a11f)
 2019-07-30 14:41:36 -07:00
Eric E Payne
168dc3f258 YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 42683aef1a694af883c14842bf41f30b91e039f3)
 2019-07-30 19:19:33 +00:00
Akira Ajisaka
f921c72fda
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich. 
(cherry picked from commit cbfa3f3e988b8a3f142abadfe4f18201b6438ac9)
 2019-07-29 17:39:05 -07:00
Wei-Chiu Chuang
d28e624958 HDFS-14429. Block remain in COMMITTED but not COMPLETE caused by Decommission. Contributed by Yicong Cai. 
(cherry picked from commit 8053085388fec17a40856f36821c142be32aa364)
 2019-07-29 14:32:46 -07:00
Yiqun Lin
69f1c8d77b HDFS-14632. Reduce useless #getNumLiveDataNodes call in SafeModeMonitor. Contributed by He Xiaoqiao. 
(cherry picked from commit 993dc8726b7d40ac832ae3e23b64e8541b62c4bd)
 2019-07-29 06:50:28 -07:00
Erik Krogen
5ec34bf43a HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen. 
(cherry-picked from 62efb6300619670d3e0554c3ba14c264fa0c705b)
 2019-07-26 11:39:14 -07:00
Erik Krogen
bd5033e6f9 Revert "HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena." 
This reverts commit 2478d333a6dc3123f4e84d4bbb6991f1cbda1ea6.
 2019-07-26 11:36:06 -07:00
Steve Loughran
7418764a67
HADOOP-9844. NPE when trying to create an error message response of SASL RPC 
This closes #55

Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 07530314c2130ecd1525682c59bf51f15b82c024)
 2019-07-26 17:54:32 +09:00
Masatake Iwasaki
2478d333a6 HDFS-14135. TestWebHdfsTimeouts Fails intermittently in trunk. Contributed by Ayush Saxena. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 6b8107ad97251267253fa045ba03c4749f95f530)
 2019-07-26 09:48:00 +09:00
S O'Donnell
23b470ee92
HADOOP-16443. Improve help text for setfacl --set option. 
Contributed by S O'Donnell.

Change-Id: I1da46c4c414a5d2b07ee15867508f0799440a413
 2019-07-23 10:25:59 +01:00
pingsutw
772cacdbdd
MAPREDUCE-7076. TestNNBench#testNNBenchCreateReadAndDelete failing in our internal build 
This closes #1089

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ee87e9a42e4ff1f27a6f1e5b7c7de97f8989d9b2)
 2019-07-23 10:35:36 +09:00
Jonathan Hung
15344006bc YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:46:45 -07:00