Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,403 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

408 Commits

Author SHA1 Message Date
Wangda Tan cffe5c1ba0 YARN-9160. [Submarine] Document 'PYTHONPATH' environment variable setting when using -localization options. (Zhankun Tang via wangda) 
Change-Id: I115b6710eb11501e21ab3e9e7cd1a87f322e5cf8
(cherry picked from commit 2c02aa6ec2)
 2019-01-06 11:10:47 -08:00
Wangda Tan 1ad1ab5d7a YARN-9141. [Submarine] JobStatus outputs with system UTC clock, not local clock. (Zac Zhou via wangda) 
Change-Id: Ib2b17cf7860ddc9188f06498086de3326bc09519
(cherry picked from commit 8f004fe4e0)
 2019-01-06 11:10:41 -08:00
Sunil G 9bfea98746 YARN-9090. [Submarine] Update submarine installation script document. Contributed by Xun Liu. 
(cherry picked from commit 742fd978b5)
 2018-12-26 08:27:50 +05:30
Eric Yang 4e1cef3625 YARN-9084. Reset container state and defer readiness check for upgrade. 
Contributed by Chandni Singh

(cherry picked from commit ccdd982e51)
 2018-12-18 18:05:40 -05:00
Wangda Tan 29e4e5f62c YARN-9112. [Submarine] Support polling applicationId when it's not ready in cluster. (Zhankun Tang via wangda) 
Change-Id: I73d73f3d631b28fb9866faa56571839b13824a97
(cherry picked from commit 9fba6cc247)
 2018-12-12 11:52:16 -08:00
Wangda Tan ef0f835537 YARN-8714. [Submarine] Support files/tarballs to be localized for a training job. (Zhankun Tang via wangda) 
Change-Id: I845131273e52a9d81dbc813ea6d4af06b205e334
(cherry picked from commit c771fe6e10)
 2018-12-12 11:52:07 -08:00
Eric Yang 8c70728f7f YARN-9071. Improved status update for reinitialized containers. 
Contributed by Chandni Singh

(cherry picked from commit 1b790f4dd1)
 2018-12-05 19:04:55 -05:00
Wangda Tan 559cb99bf6 Revert "YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda)" 
This reverts commit 920401b296.
 2018-12-04 14:13:39 -08:00
Eric Yang a8f67ad7c9 YARN-9067. Fixed Resource Manager resource leak via YARN service. 
Contributed by Eric Yang
 2018-11-29 13:50:06 -05:00
Eric Yang db8b2a130c YARN-9056. Improved YARN service upgrade state logic for readiness check. 
Contributed by Chandni Singh

(cherry picked from commit f657a2a661)
 2018-11-27 19:45:23 -05:00
Wangda Tan 34387599c4 YARN-8960. [Submarine] Can't get submarine service status using the command of "yarn app -status" under security environment. (Zac Zhou via wangda) 
Change-Id: I21b1addc9c32817650ea744a8f2e6b5602f2f4d4
(cherry picked from commit 8b23814415)
 2018-11-19 09:01:21 -08:00
Wangda Tan bee5bf867b YARN-9001. [Submarine] Use AppAdminClient instead of ServiceClient to sumbit jobs. (Zac Zhou via wangda) 
Change-Id: I7e8d1c27ebd37e0907ca570c4f3d56fe7a859635
 2018-11-16 10:26:50 -08:00
Wangda Tan ea753046b0 YARN-8918. [Submarine] Correct method usage of str.subString in CliUtils. (Zhankun Tang via wangda) 
Change-Id: Id1f11dbab3aa838dee3a0ec8b8fd5dc32f5dd946
(cherry picked from commit 076b795b2e)
 2018-11-13 11:35:06 -08:00
Eric Yang 5399ff8177 YARN-9002. Improve keytab loading for YARN Service. 
Contributed by Gour Saha

(cherry picked from commit 2664248797)
 2018-11-10 01:54:59 -05:00
Eric Yang 91c916e209 YARN-8618. Added detection for non-upgradable service. 
Contributed by Chandni Singh

(cherry picked from commit 66f059ed1d)
 2018-10-18 20:00:50 -04:00
Sunil G bde4fd5ed9 Preparing for 3.2.0 release 2018-10-18 17:07:45 +05:30
Eric Yang fc25def7d7 YARN-8810. Fixed a YARN service bug in comparing ConfigFile object. 
Contributed by Chandni Singh
 2018-10-16 18:57:50 -04:00
Wangda Tan aa23c8d739 YARN-8798. [Submarine] Job should not be submitted if --input_path option is missing. (Zhankun Tang via wangda) 
Change-Id: I7ae0e44eb5179b04a6ac861ec1c65f3b18c38f0f
(cherry picked from commit 143d74775b)
 2018-10-16 13:53:03 -07:00
Wangda Tan 920401b296 YARN-8870. [Submarine] Add submarine installation scripts. (Xun Liu via wangda) 
Change-Id: I46e8d9fd32c7745c313030da62da41486a77b3ea
(cherry picked from commit 46d6e00166)
 2018-10-16 13:52:58 -07:00
Wangda Tan 8610c20724 YARN-8875. [Submarine] Add documentation for submarine installation script details. (Xun Liu via wangda) 
Change-Id: I1c8d39c394e5a30f967ea514919835b951f2c124
(cherry picked from commit ed08dd3b0c)
 2018-10-16 13:52:53 -07:00
Sunil G a439aa79dc YARN-8879. Kerberos principal is needed when submitting a submarine job. Contributed by Zac Zhou. 
(cherry picked from commit 753f149fd3)
 2018-10-16 22:18:30 +05:30
Sunil G 1ca5f974f1 YARN-8869. YARN Service Client might not work correctly with RM REST API for Kerberos authentication. Contributed by Eric Yang. 
(cherry picked from commit fa94d370b6)
 2018-10-15 21:22:45 +05:30
Wangda Tan 8e8b748724 YARN-8852. Add documentation for submarine installation details. (Zac Zhou via wangda) 
Change-Id: If5681d1ef37ff5dc916735eeef15a6120173d653
(cherry picked from commit a23ea68b97)
 2018-10-09 10:19:16 -07:00
Vidura Mudalige 843e7f81fe
YARN-8819. Fix findbugs warnings in YarnServiceUtils 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit b85ee0d75d)
 2018-10-07 00:55:08 +09:00
Sunil G 19ad5be651 YARN-8800. Updated documentation of Submarine with latest examples. Contributed by Wangda Tan. 2018-09-29 00:01:04 +05:30
Billie Rinaldi d1c1dde309 YARN-8734. Readiness check for remote service belonging to the same user. Contributed by Eric Yang 2018-09-28 09:40:48 -07:00
Eric Yang 913f87dada YARN-8665. Added Yarn service cancel upgrade option. 
Contributed by Chandni Singh
 2018-09-26 14:51:35 -04:00
Sunil G d0bc6a27fd YARN-8817. [Submarine] In cases when user doesn't ask HDFS path while submitting job but framework requires user to set HDFS related environments. Contributed by Wangda Tan. 2018-09-25 11:23:34 +05:30
Sunil G 0cd6346102 YARN-8769. [Submarine] Allow user to specify customized quicklink(s) when submit Submarine job. Contributed by Wangda Tan. 2018-09-21 23:39:22 +05:30
Sunil G 1824d5d1c4 YARN-8757. [Submarine] Add Tensorboard component when --tensorboard is specified. Contributed by Wangda Tan. 2018-09-19 22:18:55 +05:30
Sunil G 33d8327cff YARN-8715. Make allocation tags in the placement spec optional for node-attributes. Contributed by Weiwei Yang. 2018-09-17 10:07:45 +05:30
Andrew Purtell 568ebecdf4
YARN-8772. Annotation javax.annotation.Generated has moved 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-09-14 15:02:52 +09:00
Naganarasimha 67ae81f0e0 YARN-7863. Modify placement constraints to support node attributes. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Wangda Tan de85f841da YARN-8698. [Submarine] Failed to reset Hadoop home environment when submitting a submarine job. (Zac Zhou via wangda) 
Change-Id: If7fdc11fcc4e49bed2186ded76cd69164f34f502
 2018-09-09 17:45:31 -07:00
Giovanni Matteo Fumarola 16333782c1 HADOOP-15731. TestDistributedShell fails on Windows. Contributed by Botong Huang. 2018-09-07 14:19:10 -07:00
bibinchundatt eed8415dc1 YARN-8535. Fix DistributedShell unit tests. Contributed by Abhishek Modi. 2018-09-02 13:35:52 +05:30
Eric Yang fd089caf69 YARN-8488. Added SUCCEEDED/FAILED states to YARN service. 
Contributed by Suma Shivaprasad
 2018-08-28 13:55:28 -04:00
Eric Yang e557c6bd8d YARN-8298. Added express upgrade for YARN service. 
Contributed by Chandni Singh
 2018-08-21 19:49:26 -04:00
Eric Yang 8990eaf592 YARN-8474. Fixed ApiServiceClient kerberos negotiation. 
Contributed by Billie Rinaldi
 2018-08-16 12:46:37 -04:00
Eric Yang 4cba0741d5 YARN-8160. Support upgrade of service that use docker containers. 
Contributed by Chandni Singh
 2018-08-14 11:36:26 -04:00
Sunil G cadbc8b57f YARN-8561. [Submarine] Initial implementation: Training job submission and job history retrieval. Contributed by Wangda Tan. 2018-08-13 14:22:55 +05:30
Wangda Tan 861095f761 YARN-8407. Container launch exception in AM log should be printed in ERROR level. (Yesha Vora via wangda) 
Change-Id: I154e873df1df3503a09c41d6b3874ca195af91d9
 2018-08-07 13:01:13 -07:00
Eric Yang 23f394240e YARN-8610. Fixed initiate upgrade error message. 
Contributed by Chandni Singh
 2018-08-01 20:41:43 -04:00
Eric Yang c7ebcd76bf YARN-8579. Recover NMToken of previous attempted component data. 
Contributed by Gour Saha
 2018-07-31 18:01:02 -04:00
Billie Rinaldi b429f19d32 YARN-8571. Validate service principal format prior to launching yarn service. Contributed by Eric Yang 2018-07-27 11:30:19 -07:00
Eric Yang 8d3c068e59 YARN-8429. Improve diagnostic message when artifact is not set properly. 
Contributed by Gour Saha
 2018-07-26 20:02:13 -04:00
Eric Yang 40fad32824 YARN-8545. Return allocated resource to RM for failed container. 
Contributed by Chandni Singh
 2018-07-26 18:22:57 -04:00
Eric Yang 84d7bf1eef YARN-8360. Improve YARN service restart policy and node manager auto restart policy. 
Contributed by Suma Shivaprasad
 2018-07-23 12:57:01 -04:00
Eric Yang 121865c3f9 YARN-8299. Added CLI and REST API for query container status. 
Contributed by Chandni Singh
 2018-07-16 17:41:23 -04:00
bibinchundatt 52e1bc8539 YARN-8491. TestServiceCLI#testEnableFastLaunch fail when umask is 077. Contributed by K G Bakthavachalam. 2018-07-11 16:20:29 +05:30