Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
24,972 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

6100 Commits

Author SHA1 Message Date
Szilard Nemeth ebe3ed5cd3 YARN-10997. Revisit allocation and reservation logging. Contributed by Andras Gyori 
(cherry picked from commit 7cb887e6c2)
 2022-07-07 21:04:15 +00:00
Benjamin Teke 838b63d836 YARN-10974. Queue filter in CS UI v1 does not work as expected. Contributed by Chengbing Liu. 2022-06-22 18:20:09 +02:00
Steve Loughran 9ca4ac0af0
HADOOP-18305. Preparing for 3.3.4 release: branch-3.3 version => 3.3.9 (#4482) 
Updating the hadoop version of branch-3.3 to 3.3.9-SNAPSHOT
pending agreement on what number its future release should take.

Using 3.3.9-SNAPSHOT puts space in for other incremental releases,
while avoiding creating JIRA release ordering and autocompletion
confusion the way adding a 3.3.10 or higher version would do.

Contributed by Steve Loughran
 2022-06-22 13:09:50 +01:00
zhengchenyu d7de378b22
YARN-11172. Fix TestClientRMTokens#testDelegationToken introduced by HDFS-16563. (#4408) 
Regression caused by HDFS-16563; the hdfs exception text was changed, but because it was
a YARN test doing the check, Yetus didn't notice.

Contributed by zhengchenyu
 2022-06-17 19:51:56 +01:00
Steve Loughran de9f994338
YARN-11173. remove redeclaration of os-maven-plugin.version from yarn-csi (#4417) 
This is a followup to HADOOP-18275 and its upgrade of os-maven-plugin.version
When that change is merged in, this MUST follow it.

Contributed by Steve Loughran

Change-Id: I61d087041561eeb8c9c42b5b7d8f0bb63f296b15
 2022-06-10 17:03:25 +01:00
Ashutosh Gupta e0732baeb8 YARN-11128. Fix comments in TestProportionalCapacityPreemptionPolicy* (#4271) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Reviewed-by: Hemanth Boyina <hemanthboyina@apache.org>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit e3e9369c1d)
 2022-05-26 15:58:47 +09:00
Owen O'Malley 1f111d6a41 YARN-11162. Set the zk acl for nodes created by ZKConfigurationStore. (#4350) 
(cherry picked from commit f390edaec4)
 2022-05-24 05:17:34 +00:00
Szilard Nemeth 90ec4418c7 YARN-11141. Capacity Scheduler does not support ambiguous queue names when moving application across queues. Contributed by Andras Gyori 2022-05-18 14:34:08 +02:00
Szilard Nemeth 4f112e3138 YARN-11126. ZKConfigurationStore Java deserialisation vulnerability. Contributed by Tamas Domok 2022-05-18 14:25:35 +02:00
Szilard Nemeth b4550b3356 YARN-10850. TimelineService v2 lists containers for all attempts when filtering for one. Contributed by Benjamin Teke 2022-05-18 14:08:41 +02:00
Zilong Zhu 2dd5852ad0 YARN-11133. YarnClient gets the wrong EffectiveMinCapacity value (#4280) 
(cherry picked from commit 79cc0caf9f)
 2022-05-18 03:03:13 +09:00
Steve Loughran fe306ce57e HADOOP-18198. Release 3.3.3: release notes and jdiff files. 
* Add the changelog and release notes
* add all jdiff XML files
* update the project pom with the new stable version

Change-Id: Iaea846c3e451bbd446b45de146845a48953d580d
 2022-05-17 19:00:09 +01:00
Ashutosh Gupta a50ce18f4d YARN-11092. Upgrade jquery ui to 1.13.1 (#4260) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 931abbd927)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/webapp/view/JQueryUI.java
 2022-05-16 23:26:46 +09:00
Ashutosh Gupta d58f9d438f
YARN-10080. Support show app id on localizer thread pool (#4283) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6985f9aabe)
 2022-05-14 01:41:31 +09:00
Ashutosh Gupta 62c6a08ffd
HADOOP-16515. Update the link to compatibility guide (#4226) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ae47846a5b)
 2022-05-08 07:39:17 +09:00
jteagles f155abc572
YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe DateTimeFormatter class (#4242) 
Co-authored-by: Jonathan Eagles <jeagles@verizonmedia.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit d4a91bd0c0)
 2022-05-03 03:15:10 +09:00
Ashutosh Gupta 4f9e607126
YARN-10187. Removing hadoop-yarn-project/hadoop-yarn/README as it is no longer maintained. (#4222) 
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a74acc755e)
 2022-05-03 03:10:43 +09:00
Steve Loughran e123de9f19
HADOOP-16202. Enhanced openFile(): mapreduce and YARN changes. (#2584/2) 
These changes ensure that sequential files are opened with the
right read policy, and split start/end is passed in.

As well as offering opportunities for filesystem clients to
choose fetch/cache/seek policies, the settings ensure that
processing text files on an s3 bucket where the default policy
is "random" will still be processed efficiently.

This commit depends on the associated hadoop-common patch,
which must be committed first.

Contributed by Steve Loughran.

Change-Id: Ic6713fd752441cf42ebe8739d05c2293a5db9f94
 2022-04-27 19:23:25 +01:00
Viraj Jasani bb13e228bc
HADOOP-17956. Replace all default Charset usage with UTF-8 (#3529) 
Change-Id: I0094a84619ce19acf340d8dd1040cfe9bd88184e
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2022-04-27 10:30:07 +01:00
Steve Loughran caecec45f5
HADOOP-17650. Bump solr to unblock build failure with Maven 3.8.1 (#2939) 
Reviewed-by: Siyao Meng <siyao@apache.org>

Contributed by Viraj Jasani
 2022-04-20 16:36:51 +01:00
Akira Ajisaka 2112ef61e0
YARN-10553. Refactor TestDistributedShell (#4159) 
(cherry picked from commit 890f2da624)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDSWithMultipleNodeManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/test/java/org/apache/hadoop/yarn/applications/distributedshell/TestDistributedShell.java

Co-authored-by: Ahmed Hussein <50450311+amahussein@users.noreply.github.com>
 2022-04-13 12:22:17 +09:00
Steve Loughran 44e662272f
HADOOP-18198. Preparing for 3.3.4 development 
Change-Id: I2bf19beb541739af22fced38c2545f09c4e1bd53
 2022-04-12 14:09:08 +01:00
Masatake Iwasaki 160b6d106d
HADOOP-18088. Replace log4j 1.x with reload4j. (#4052) 
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
 2022-04-07 08:33:13 +09:00
Peter Bacsko ab5ea272ec YARN-10720. YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging. Contributed by Qi Zhu. 
(cherry picked from commit a0deda1a77)
 2022-03-24 14:11:23 +09:00
Siyao Meng 4153c16324 YARN-10747. Bump YARN CSI protobuf version to 3.7.1 (#2946) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e
(cherry picked from commit 5b09ba75ec)
 2022-03-22 14:46:45 +09:00
Fei Hui 5a38ed2f22 HADOOP-17276. Extend CallerContext to make it include many items (#2327) 
Cherry-picked from d0d10f7e by Owen O'Malley
 2022-03-14 10:28:38 -07:00
Tamas Domok d7c375da40 YARN-11076. Upgrade jQuery version in Yarn UI2. (#4046) 
Change-Id: I3cb1677741df5a1978e83029443d4a2d5d7e3d7f
(cherry picked from commit 22fe79cee3)
 2022-03-03 23:55:26 +09:00
Szilard Nemeth 856e483592 YARN-11022. Fix the documentation for max-parallel-apps in CS. Contributed by Tamas Domok 2022-03-02 16:13:35 +01:00
Szilard Nemeth 192f53283b YARN-10894. Follow up YARN-10237: fix the new test case in TestRMWebServicesCapacitySched. Contributed by Tamas Domok 2022-03-02 16:04:38 +01:00
Szilard Nemeth 3ef3c5a05b YARN-11033. isAbsoluteResource is not correct for dynamically created queues. Contributed by Tamas Domok 2022-03-02 14:45:31 +01:00
Szilard Nemeth f06f44b1c2 YARN-11014. YARN incorrectly validates maximum capacity resources on the validation API. Contributed by Benjamin Teke 2022-03-02 14:23:00 +01:00
Szilard Nemeth 935619a28c YARN-11075. Explicitly declare serialVersionUID in LogMutation class. Contributed by Benjamin Teke 2022-03-01 18:05:04 +01:00
Akira Ajisaka 352656999f
YARN-10788. TestCsiClient fails (#3989) 
Create unix domain socket in java.io.tmpdir instead of
test.build.dir to avoid 'File name too long' error.

Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
(cherry picked from commit 7fd90cdcbe)
 2022-02-15 01:14:31 +09:00
luoyuan3471 752a7b6d49
HADOOP-18044. Hadoop - Upgrade to jQuery 3.6.0 (#3791) 
Co-authored-by: luoyuan <luoyuan@shopee.com>
(cherry picked from commit e2d620192a)
 2022-02-11 23:18:25 +08:00
Akira Ajisaka 8032b680fb YARN-10561. Upgrade node.js to 12.22.1 and yarn to 1.22.5 in YARN application catalog webapp (#2591) 
Reviewed-by: Masatake Iwasaki <iwasakims@apache.org>
(cherry picked from commit 9cb535caf2)
 2022-01-28 15:52:33 +09:00
dependabot[bot] 55192570a1 YARN-11065. Bump follow-redirects from 1.13.3 to 1.14.7 in hadoop-yarn-ui (#3890) 
Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.13.3 to 1.14.7.
- [Release notes](https://github.com/follow-redirects/follow-redirects/releases)
- [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.13.3...v1.14.7)

---
updated-dependencies:
- dependency-name: follow-redirects
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit dae33cf935)
 2022-01-20 21:45:53 +09:00
Gera Shegalov 6c58f83b78 YARN-11055. Add missing newline in cgroups-operations.c (#3851) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a94e9fcbde)
 2022-01-17 16:21:13 +09:00
Cheng Pan 7b51e6824d YARN-11053. AuxService should not use class name as default system classes (#3816) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 4db27acf06)
 2021-12-24 11:09:42 +09:00
Ashutosh Gupta 1c99810b89 YARN-8234. Improve RM system metrics publisher's performance by pushing events to timeline server in batch (#3793) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 00e2405fbd)
 2021-12-23 17:16:09 +09:00
Eric Payne ccaba2561a YARN-10178: Global Scheduler async thread crash caused by 'Comparison method violates its general contract. Contributed by Andras Gyori (gandras) and Qi Zhu (zhuqi). 
(cherry picked from commit e2d6fd075d)
 2021-12-21 19:20:21 +00:00
Szilard Nemeth 357423b57a YARN-6862. Nodemanager resource usage metrics sometimes are negative. Contributed by Benjamin Teke 2021-12-17 14:51:41 +01:00
Viraj Jasani b0c1158829
HADOOP-18033. Upgrade fasterxml Jackson to 2.13.0 (#3764) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2021-12-13 13:52:44 +09:00
Szilard Nemeth 8053b0f205 YARN-11020. [UI2] No container is found for an application attempt with a single AM container. Contributed by Andras Gyori 2021-12-09 13:02:55 +01:00
Ashutosh Gupta 53f76b9b1c YARN-9063. ATS 1.5 fails to start if RollingLevelDb files are corrupt or missing (#3728) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 5a950b8900)
 2021-12-06 20:10:38 +09:00
Prabhu Joseph edfd68e248 YARN-10820. Make GetClusterNodesRequestPBImpl thread safe. Contributed by Swathi Chandrashekar. 
(cherry picked from commit 6b4987f09c)
 2021-12-06 14:45:18 +09:00
GuoPhilipse a13a03ec10
YARN-11007. Correct words in YARN documents (#3680) 
Reviewed-by: cxorm <lianp964@gmail.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c9d64bad37)
 2021-11-25 17:51:38 +09:00
Ashutosh Gupta 17b8ad3d48
YARN-10991. Fix to ignore the grouping "[]" for resourcesStr in parseResourcesString method (#3592) 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 08f3df3ea2)
 2021-11-24 18:42:41 +09:00
Shubham Gupta c44f109860
YARN-10438. Handle null containerId in ClientRMService#getContainerReport() (#2313) 
Co-authored-by: Shubham Gupta <gshubham@microsoft.com>
(cherry picked from commit e3cd627069)
 2021-11-19 00:23:48 +09:00
Chao Sun e079fa6577 Preparing for 3.3.3 development 2021-11-16 16:02:34 -08:00
Ahmed Hussein 742d88b1c6 YARN-1115: Provide optional means for a scheduler to check real user ACLs. Contributed by Eric Payne (epayne) 2021-10-21 17:04:29 +00:00