Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,197 Commits 326 Branches 371 Tags
Commit Graph

5070 Commits

Author SHA1 Message Date
Szilard Nemeth
a762a6be29 Revert "YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko" 
This reverts commit b20fd9e21295add7e80f07b471bba5c76e433aed.
Commit is reverted since unnecessary files were added, accidentally.
 2019-08-13 15:47:57 +02:00
Szilard Nemeth
9da9b6d58e YARN-9723. ApplicationPlacementContext is not required for terminated jobs during recovery. Contributed by Prabhu Joseph 
(cherry picked from commit e4b538bbda6dc25d7f45bffd6a4ce49f3f84acdc)
 2019-08-12 15:16:49 +02:00
Szilard Nemeth
148121d889 YARN-9451. AggregatedLogsBlock shows wrong NM http port. Contributed by Prabhu Joseph 
(cherry picked from commit b91099efd6e1fdcb31ec4ca7142439443c9ae536)
 2019-08-12 15:06:48 +02:00
Szilard Nemeth
6b4ded7647 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko 2019-08-12 14:03:50 +02:00
Sunil G
58ad5ad493 YARN-9729. [UI2] Fix error message for logs when ATSv2 is offline. Contributed by Zoltan Siegl. 
(cherry picked from commit 1c5b28659fe1310030d14e0be40dcd77b25056d6)
 2019-08-11 11:49:57 +05:30
Szilard Nemeth
be9ac8adf9 Logging fileSize of log files under NM Local Dir. Contributed by Prabhu Joseph 
(cherry picked from commit 54ac80176e8487b7a18cd9e16a11efa289d0b7df)
 2019-08-09 13:23:49 +02:00
Sunil G
1c4364cb9a YARN-9715. [UI2] yarn-container-log URI need to be encoded to avoid potential misuses. Contributed by Akhil PB. 
(cherry picked from commit acffec7a92be540aa8531dbe06a3ea7bb813ab93)
 2019-08-09 16:06:07 +05:30
Adam Antal
600a61f410 YARN-9124. Resolve contradiction in ResourceUtils: addMandatoryResources / checkMandatoryResources work differently (#1121) 
(cherry picked from commit cbcada804d119b837ad99de71d7f44cb4629026e)
 2019-08-09 11:44:22 +02:00
Szilard Nemeth
410f7a3069 YARN-9092. Create an object for cgroups mount enable and cgroups mount path as they belong together. Contributed by Gergely Pollak 
(cherry picked from commit e0c21c6da91776caf661661a19c368939c81fcc4)
 2019-08-09 10:25:12 +02:00
Szilard Nemeth
b2f39f81fe YARN-9096: Some GpuResourcePlugin and ResourcePluginManager methods are synchronized unnecessarily. Contributed by Gergely Pollak 
(cherry picked from commit 742e30b47381ad63e2b2fe63738cd0fe6cbce106)
 2019-08-09 10:05:40 +02:00
Szilard Nemeth
943dfc78d1 YARN-9094: Remove unused interface method: NodeResourceUpdaterPlugin#handleUpdatedResourceFromRM. Contributed by Gergely Pollak 
(cherry picked from commit 72d7e570a73989aa18b737c0e642d570a55c6781)
 2019-08-09 09:53:14 +02:00
Eric E Payne
b131214685 YARN-9685: NPE when rendering the info table of leaf queue in non-accessible partitions. Contributed by Tao Yang. 
(cherry picked from commit 3b38f2019e4f8d056580f3ed67ecef591011d7a6)
 2019-08-08 13:08:05 +00:00
Haibo Chen
f943bff254 YARN-9559. Create AbstractContainersLauncher for pluggable ContainersLauncher logic. (Contributed by Jonathan Hung) 
(cherry picked from commit f51702d5398531835b24d812f6f95094a0e0493e)
(cherry picked from commit 8d357343c4bc9f18e25543583f8f217b8a2f621b)
 2019-08-06 15:01:06 -07:00
Eric Badger
698e74d097 YARN-8045. Reduce log output from container status calls. Contributed by Craig Condit 
(cherry picked from commit 144a55f0e3ba302327baf2e98d1e07b953dcbbfd)
 2019-08-02 20:41:26 +00:00
Eric E Payne
36af8845de YARN-9596: QueueMetrics has incorrect metrics when labelled partitions are involved. Contributed by Muhammad Samir Khan. 
(cherry picked from commit 42683aef1a694af883c14842bf41f30b91e039f3)
 2019-07-30 19:45:00 +00:00
Jonathan Hung
3ff2148482 YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonanthan Hung) 2019-07-22 10:54:08 -07:00
Weiwei Yang
48192531ad YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam. 
(cherry picked from commit 85d9111a88f94a5e6833cd142272be2c5823e922)
 2019-07-17 11:08:21 +08:00
Szilard Nemeth
30c7b43227 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 
(cherry picked from commit 18ee1092b471c5337f05809f8f01dae415e51a3a)
 2019-07-15 12:15:36 +02:00
Szilard Nemeth
bb37c6cb7f YARN-9337. Addendum to fix compilation error due to mockito spy call 2019-07-13 00:42:14 +02:00
Erik Krogen
07a6510e6a HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2019-07-12 11:03:31 -07:00
Szilard Nemeth
773591ee42 YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 
(cherry picked from commit 557056e18ea3d5b3fe3046f0ea4b4c7345ea21c5)
 2019-07-12 17:41:23 +02:00
Szilard Nemeth
531e0c0bc1 YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 
(cherry picked from commit 61b0c2bb7c0f18c4a666b96ca1603cbd4d27eb6d)
 2019-07-12 17:30:50 +02:00
Szilard Nemeth
43c89d1e2b YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 
(cherry picked from commit c416284bb7581747beef36d7899d8680fe33abbd)
 2019-07-12 17:07:25 +02:00
Szilard Nemeth
872a039bac YARN-9625. UI2 - No link to a queue on the Queues page for Fair Scheduler. Contributed by Zoltan Siegl 
(cherry picked from commit 9cec02318644c8430cbf65bcc3096ffe45992a8e)
 2019-07-11 20:02:19 +02:00
Szilard Nemeth
d590745046 YARN-9573. DistributedShell cannot specify LogAggregationContext. Contributed by Adam Antal. 2019-07-11 19:54:31 +02:00
bibinchundatt
5effeae1f3 YARN-9557. Application fails in diskchecker when ReadWriteDiskValidator is configured. Contributed by Bilwa S T. 
(cherry picked from commit 5f8395f393e0759215c56927dae1297dfdb0b955)
 2019-07-10 14:47:29 +05:30
Sunil G
9eb96b0fbf YARN-9644. First RMContext object is always leaked during switch over. Contributed by Bibin A Chundatt. 
(cherry picked from commit d18986e4e89bd4bb3600e95b4690fd32f54a41e5)
 2019-07-04 11:06:41 +05:30
Szilard Nemeth
46177ade8b YARN-9629. Support configurable MIN_LOG_ROLLING_INTERVAL. Contributed by Adam Antal. 
(cherry picked from commit a2a8be18cb5e912c8de0ea6beec1de4a99de656b)
 2019-07-03 14:24:53 +02:00
Sunil G
d2a5749482 YARN-9327. Improve synchronisation in ProtoUtils#convertToProtoFormat block. Contributed by Bibin A Chundatt. 
(cherry picked from commit 0c8813f135f8c17f88660bb92529c15bb3a157ca)
 2019-07-02 12:15:40 +05:30
Weiwei Yang
46b81a982b YARN-9655. AllocateResponse in FederationInterceptor lost applicationPriority. Contributed by hunshenshi. 
(cherry picked from commit 570eee30e5ab5cf37b1a758934987cbf61140f6a)
 2019-07-02 10:17:56 +08:00
bibinchundatt
4f622ecad8 YARN-9639. DecommissioningNodesWatcher cause memory leak. Contributed by Bilwa S T. 
(cherry picked from commit be80334cdf255616f589217726483194fb56dcc6)
 2019-06-27 10:11:30 +05:30
Zhankun Tang
829202740a YARN-9584. Should put initializeProcessTrees method call before get pid. Contributed by Wanqiang Ji. 
(cherry picked from commit 67414a1a80039e70e0afc1de171831a6e981f37a)
 2019-06-18 13:20:07 +08:00
Weiwei Yang
56a4935048 YARN-9621. Fix TestDSWithMultipleNodeManager.testDistributedShellWithPlacementConstraint on branch-3.1. Contributed by Prabhu Joseph. 2019-06-17 17:22:49 +08:00
Sean Mackrory
fee1e67453 HADOOP-16213. Update guava to 27.0-jre. Contributed by Gabor Bota. 2019-06-13 07:38:43 -06:00
Sunil G
c343554e2a YARN-9543. [UI2] Handle ATSv2 server down or failures cases gracefully in YARN UI v2. Contributed by Zoltan Siegl and Akhil P B. 
(cherry picked from commit 52128e352a30b70b83483f9290d9e94e98929705)
 2019-06-12 19:28:43 +05:30
Sunil G
bc028d3ebb YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 
(cherry picked from commit 72203f7a12c943ca231fbc40c058a1a094b009cd)
 2019-06-12 19:28:10 +05:30
Sunil G
1bb9e9a4f2 Revert "YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl." 
This reverts commit d65371c4e8bbb4ae655ccacda389cd37a18fab32.
 2019-06-12 19:27:21 +05:30
bibinchundatt
f42e246f8a YARN-9547. ContainerStatusPBImpl default execution type is not returned. Contributed by Bilwa S T. 
(cherry picked from commit 3303723f55edc6ef55a07707c4453395e82ff060)
 2019-06-11 23:43:54 +05:30
bibinchundatt
d386f595f9 YARN-9565. RMAppImpl#ranNodes not cleared on FinalTransition. Contributed by Bilwa S T. 
(cherry picked from commit 60c95e9b6a899e37ecdc8bce7bb6d9ed0dc7a6be)
 2019-06-11 23:15:02 +05:30
bibinchundatt
4a39165b41 YARN-9594. Fix missing break statement in ContainerScheduler#handle. Contributed by lujie. 
(cherry picked from commit 6d80b9bc3ff3ba8073e3faf64551b9109d2aa2ad)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/scheduler/ContainerScheduler.java
 2019-06-11 23:05:06 +05:30
Sunil G
d65371c4e8 YARN-9545. Create healthcheck REST endpoint for ATSv2. Contributed by Zoltan Siegl. 
(cherry picked from commit f1d3a17d3e67ec2acad52227a3f4eb7cca83e468)
 2019-06-06 06:25:02 +05:30
Sunil G
6be665cfc6 YARN-8906. [UI2] NM hostnames not displayed correctly in Node Heatmap Chart. Contributed by Akhil PB. 
(cherry picked from commit 59719dc560cf67f485d8e5b4a6f0f38ef97d536b)
 2019-06-03 15:54:33 +05:30
Sunil G
894ef5c07b YARN-8947. [UI2] Active User info missing from UI2. Contributed by Akhil PB. 
(cherry picked from commit 7f46dda513fb79c349acb73bdb90b689df9cc18d)
 2019-06-03 12:25:40 +05:30
Weiwei Yang
23f9508a89 YARN-9507. Fix NPE in NodeManager#serviceStop on startup failure. Contributed by Bilwa S T. 
(cherry picked from commit 4530f4500d308c9cefbcc5990769c04bd061ad87)
 2019-06-03 14:26:16 +08:00
Eric Yang
413a6b63bc YARN-9542. Fix LogsCLI guessAppOwner ignores custome file format suffix. 
Contributed by Prabhu Joseph

(cherry picked from commit b2a39e8883f8128e44543c2279dcc1835af72652)
 2019-05-29 18:05:47 -04:00
Eric E Payne
9c3ab58aa7 YARN-8625. Aggregate Resource Allocation for each job is not present in ATS. Contributed by Prabhu Joseph. 
(cherry picked from commit 3c63551101dc17038efb9f5345833d2883fce86e)
 2019-05-29 19:08:27 +00:00
Ahmed Hussein
f2202f7990 YARN-9563. Resource report REST API could return NaN or Inf (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit abf76ac371e4611c8eb371736b433e3d89c9d2ae)
 2019-05-29 12:47:27 -05:00
Takanobu Asanuma
8098ddaf40 HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:39:49 +09:00
Akira Ajisaka
f8bd5deec1
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:21 +09:00
Sunil G
f09befd2ea YARN-9519. TFile log aggregation file format is not working for yarn.log-aggregation.TFile.remote-app-log-dir config. Contributed by Adam Antal. 
(cherry picked from commit 7d831eca645f93d064975ebae35a7cbea3bbad31)
 2019-05-14 10:50:02 -07:00