Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,328 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

5069 Commits

Author SHA1 Message Date
Weiwei Yang 6468071f13 YARN-8367. Fix NPE in SingleConstraintAppPlacementAllocator when placement constraint in SchedulingRequest is null. Contributed by Weiwei Yang. 2018-05-31 20:46:39 +08:00
Billie Rinaldi 778a4a24be YARN-8350. NPE in service AM related to placement policy. Contributed by Gour Saha 2018-05-30 13:19:13 -07:00
Billie Rinaldi 96eefcc84a YARN-8368. yarn app start cli should print applicationId. Contributed by Rohith Sharma K S 2018-05-30 12:37:43 -07:00
Eric E Payne e44c0849d7 YARN-8377: Javadoc build failed in hadoop-yarn-server-nodemanager. Contributed by Takanobu Asanuma 2018-05-30 16:50:19 +00:00
Eric Yang 135941e00d YARN-8362. Bugfix logic in container retries in node manager. 
Contributed by Chandni Singh
 2018-05-29 16:56:58 -04:00
Jason Lowe 4827e9a908 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 2018-05-29 14:43:17 -05:00
Wangda Tan 17aa40f669 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
 2018-05-29 09:27:36 -07:00
Wangda Tan 3061bfcde5 YARN-8339. Service AM should localize static/archive resource types to container working directory instead of 'resources'. (Suma Shivaprasad via wangda) 
Change-Id: I9f8e8f621650347f6c2f9e3420edee9eb2f356a4
 2018-05-29 09:23:11 -07:00
Jason Lowe 31ab960f4f YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 2018-05-29 11:00:30 -05:00
Sunil G 7c343669ba YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 2018-05-28 16:32:53 +05:30
Rohith Sharma K S 88cbe57c06 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 2018-05-28 14:05:49 +05:30
Wangda Tan f24c842d52 YARN-8213. Add Capacity Scheduler performance metrics. (Weiwei Yang via wangda) 
Change-Id: Ieea6f3eeb83c90cd74233fea896f0fcd0f325d5f
 2018-05-25 21:53:20 -07:00
Eric E Payne 8d5509c681 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 2018-05-25 16:06:09 +00:00
Haibo Chen 86bc6425d4 YARN-8191. Fair scheduler: queue deletion without RM restart. (Gergo Repas via Haibo Chen) 2018-05-24 17:12:34 -07:00
Eric Yang d9852eb589 YARN-8357. Fixed NPE when YARN service is saved and not deployed. 
Contributed by Chandni Singh
 2018-05-24 16:32:13 -04:00
Eric Yang 7ff5a40218 YARN-8316. Improved diagnostic message for ATS unavailability for YARN Service. 
Contributed by Billie Rinaldi
 2018-05-24 16:26:02 -04:00
Shane Kumpf 1388de18ad YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:30:39 -06:00
Rohith Sharma K S c05b5d424b YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 2018-05-24 14:19:46 +05:30
Rohith Sharma K S 4cc0c9b0ba YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 2018-05-24 12:23:47 +05:30
Haibo Chen d996479954 YARN-4599. Set OOM control for memory cgroups. (Miklos Szegedi via Haibo Chen) 2018-05-23 16:35:37 -07:00
Inigo Goiri f09dc73001 YARN-8327. Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows. Contributed by Giovanni Matteo Fumarola. 2018-05-23 15:59:30 -07:00
Inigo Goiri d72615611c YARN-8348. Incorrect and missing AfterClass in HBase-tests to fix NPE failures. Contributed by Giovanni Matteo Fumarola. 2018-05-23 14:43:59 -07:00
Inigo Goiri e99e5bf104 YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 2018-05-23 14:15:26 -07:00
Inigo Goiri e30938af12 YARN-8336. Fix potential connection leak in SchedConfCLI and YarnWebServiceUtils. Contributed by Giovanni Matteo Fumarola. 2018-05-23 11:55:31 -07:00
Rohith Sharma K S f61e3e752e YARN-8297. Incorrect ATS Url used for Wire encrypted cluster.(addendum). Contributed by Sunil G. 2018-05-23 18:31:03 +05:30
Shane Kumpf 9837ca9cc7 YARN-8285. Remove unused environment variables from the Docker runtime. Contributed by Eric Badger 2018-05-23 06:43:44 -06:00
Miklos Szegedi 3e5f7ea986 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-22 18:10:33 -07:00
Robert Kanter b22f56c471 YARN-8273. Log aggregation does not warn if HDFS quota in target directory is exceeded (grepas via rkanter) 2018-05-22 14:24:38 -07:00
Wangda Tan 83f53e5c62 YARN-8332. Incorrect min/max allocation property name in resource types doc. (Weiwei Yang via wangda) 
Change-Id: If74f1ceed9c045a2cb2d6593741278b65ac44a9f
 2018-05-22 13:33:33 -07:00
Wangda Tan bd15d2396e YARN-8290. SystemMetricsPublisher.appACLsUpdated should be invoked after application information is published to ATS to avoid "User is not set in the application report" Exception. (Eric Yang via wangda) 
Change-Id: I0ac6ddd19740d1aa7dd07111cd11af71ddc2fcaf
 2018-05-22 13:33:33 -07:00
Eric Yang 6176d2b35c YARN-7960. Added security flag no-new-privileges for YARN Docker integration. 
Contributed by Eric Badger
 2018-05-22 13:44:58 -04:00
Jason Lowe 5f11288e41 YARN-8206. Sending a kill does not immediately kill docker containers. Contributed by Eric Badger 2018-05-22 09:27:08 -05:00
Eric E Payne 0b4c44bdee YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 2018-05-21 20:14:58 +00:00
Haibo Chen f48fec83d0 YARN-8248. Job hangs when a job requests a resource that its queue does not have. (Szilard Nemeth via Haibo Chen) 2018-05-21 08:10:41 -07:00
Eric Yang a23ff8d880 YARN-7530. Refactored YARN service API project location. 
Contributed by Chandni Singh
 2018-05-18 17:29:10 -04:00
Arun Suresh 3159bffce2 YARN-7900. [AMRMProxy] AMRMClientRelayer for stateful FederationInterceptor. (Botong Huang via asuresh) 2018-05-17 20:00:52 -07:00
Eric Yang 2f2dd22aad YARN-8296. Removed unique_component_support from YARN services. 
Contributed by Suma Shivaprasad
 2018-05-17 20:58:13 -04:00
Eric Yang 989cfdc1e0 YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh
 2018-05-17 20:56:04 -04:00
Eric Yang d45a0b7d73 YARN-8141. Removed YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS flag. 
Contributed by Chandni Singh
 2018-05-17 17:29:34 -04:00
Eric Yang 7f083ed869 YARN-8080. Add restart policy for YARN services. 
Contributed by Suma Shivaprasad
 2018-05-17 17:16:50 -04:00
Eric Yang 7802af6e9a YARN-8293. Removed "User Name for service" for deploying secure YARN service. 
Contributed by Sunil G
 2018-05-17 17:06:58 -04:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 328f0847e3 YARN-8128. Documenting the per-node per-app file limit in YARN log aggregation. Contributed by Xuan Gong. 2018-05-17 12:47:10 -07:00
Rohith Sharma K S c95795cd8f YARN-8297. Incorrect ATS Url used for Wire encrypted cluster. Contributed by Sunil G. 2018-05-17 16:24:51 +05:30
Jason Lowe be53969047 YARN-8071. Add ability to specify nodemanager environment variables individually. Contributed by Jim Brennan 2018-05-16 16:17:28 -05:00
Haibo Chen e3b7d7ac16 YARN-7933. [atsv2 read acls] Add TimelineWriter#writeDomain. (Rohith Sharma K S via Haibo Chen) 2018-05-16 13:00:01 -07:00
Eric Yang 91357c22ef YARN-8300. Fixed NPE in DefaultUpgradeComponentsFinder. 
Contributed by Suma Shivaprasad
 2018-05-16 12:38:26 -04:00
Jason Lowe d47c09dcb1 YARN-8284. get_docker_command refactoring. Contributed by Eric Badger 2018-05-16 09:23:49 -05:00
Dinesh Chitlangia 0a22860336
YARN-8123. Skip compiling old hamlet package when the Java version is 10 or upper 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-16 17:25:31 +09:00
Eric Yang 8d3b39de89 YARN-8081. Add support to upgrade a component. 
Contributed by Chandni Singh
 2018-05-15 20:40:39 -04:00
Weiwei Yang 2bb647bb91 YARN-8278. DistributedScheduling is not working in HA. Contributed by Bibin A Chundatt. 2018-05-15 17:28:19 +08:00
Sunil G 58b97c79e3 YARN-8236. Invalid kerberos principal file name cause NPE in native service. Contributed by Gour Saha. 2018-05-15 12:17:35 +05:30
Sunil G ffb9210ded YARN-8166. [UI2] Service page header links are broken. Contributed by Yesha Vora. 2018-05-15 12:13:04 +05:30
Sunil G 796b2b0ee3 YARN-8266. [UI2] Clicking on application from cluster view should redirect to application attempt page. Contributed by Yesha Vora. 2018-05-15 12:01:28 +05:30
Haibo Chen 2d00a0c71b YARN-8130 Race condition when container events are published for KILLED applications. (Rohith Sharma K S via Haibo Chen) 2018-05-14 11:08:42 -07:00
Naganarasimha 8a2b5914f3 YARN-8288. Fix wrong number of table columns in Resource Model doc. Contributed by Weiwei Yang. 2018-05-15 00:05:23 +08:00
Sunil G 89d0b87ad3 YARN-8271. [UI2] Improve labeling of certain tables. Contributed by Yesha Vora. 2018-05-14 20:59:31 +05:30
Eric Yang 0ff94563b9 YARN-8265. Improve DNS handling on docker IP changes. 
Contributed by Billie Rinaldi
 2018-05-11 22:37:43 -07:00
Jason Lowe 6c8e51ca7e YARN-7654. Support ENTRY_POINT for docker container. Contributed by Eric Yang 2018-05-11 18:56:05 -05:00
Haibo Chen 50408cfc69 YARN-3610. FairScheduler: Add steady-fair-shares to the REST API documentation. (Ray Chiang via Haibo Chen) 2018-05-11 14:07:55 -07:00
Billie Rinaldi ca612e353f YARN-8243. Flex down should remove instance with largest component instance ID first. Contributed by Gour Saha 2018-05-11 12:49:05 -07:00
Jason Lowe dc912994a1 YARN-8244. TestContainerSchedulerQueuing.testStartMultipleContainers failed. Contributed by Jim Brennan 2018-05-11 14:07:32 -05:00
Haibo Chen 1f10a36021 YARN-8268. Fair scheduler: reservable queue is configured both as parent and leaf queue. (Gergo Repas via Haibo Chen) 2018-05-11 11:34:00 -07:00
Eric Yang 8f7912e0fe YARN-8274. Fixed a bug on docker start command. 
Contributed by Jason Lowe
 2018-05-11 14:23:16 -04:00
Weiwei Yang 9db9cd95bd YARN-7003. DRAINING state of queues is not recovered after RM restart. Contributed by Tao Yang. 2018-05-11 10:47:04 +08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) d76fbbc9b8 YARN-8249. Fixed few REST APIs in RMWebServices to have static-user check. Contributed by Sunil Govindan. 2018-05-10 19:05:35 -07:00
Haibo Chen 6341c3a437 YARN-7715. Support NM promotion/demotion of running containers. (Miklos Szegedi via Haibo Chen) 2018-05-10 11:01:01 -07:00
Robert Kanter c8b53c4364 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 2018-05-10 09:31:59 -07:00
Vrushali C 3c95ca4f21 YARN-8247 Incorrect HTTP status code returned by ATSv2 for non-whitelisted users. Contributed by Rohith Sharma K S 2018-05-09 22:17:48 -07:00
bibinchundatt cc0310a526 YARN-8201. Skip stacktrace of few exception from ClientRMService. Contributed by Bilwa S T. 2018-05-10 09:15:46 +05:30
Eric Yang af4fc2e628 YARN-8261. Fixed a bug in creation of localized container directory. 
Contributed by Jason Lowe
 2018-05-09 13:32:51 -04:00
Rohith Sharma K S cd68c7cc69 YARN-8260. [UI2] Per-application tracking URL is no longer available in YARN UI2. Contributed by Sunil G. 2018-05-09 12:17:04 +05:30
Rohith Sharma K S 9acfaee1f0 YARN-8264. [UI2] GPU Info tab disappears if we click any sub link under List of Applications or List of Containers. Contributed by Sunil G. 2018-05-09 12:04:31 +05:30
Rohith Sharma K S eb7b256cc2 YARN-8239. [UI2] Clicking on Node Manager UI under AM container info / App Attempt page goes to old RM UI. Contributed by Sunil G. 2018-05-09 12:00:43 +05:30
Billie Rinaldi 1ef0a1db1d YARN-7894. Improve ATS response for DShell DS_CONTAINER when container launch fails. Contributed by Chandni Singh 2018-05-08 13:49:41 -07:00
Jason Lowe a2ea756420 YARN-8207. Docker container launch use popen have risk of shell expansion. Contributed by Eric Yang. 2018-05-08 15:30:36 -05:00
Wangda Tan 6b96a73bb0 YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
 2018-05-08 11:34:45 -07:00
Rohith Sharma K S 7450583721 YARN-8253. HTTPS Ats v2 api call fails with 'bad HTTP parsed'. Contributed by Charan Hebri. 2018-05-08 12:28:54 +05:30
Sunil G 2916c90500 YARN-8251. [UI2] Clicking on Application link at the header goes to Diagnostics Tab instead of AppAttempt Tab. Contributed by Yesha Vora. 2018-05-08 11:37:03 +05:30
Sunil G 9832265e1d YARN-5151. [UI2] Support kill application from new YARN UI. Contributed by Gergely Novák. 2018-05-08 10:35:19 +05:30
Weiwei Yang 67f239c42f YARN-8025. UsersManangers#getComputedResourceLimitForActiveUsers throws NPE due to preComputedActiveUserLimit is empty. Contributed by Tao Yang. 2018-05-07 11:33:14 +08:00
Inigo Goiri 6a69239d86 YARN-8163. Add support for Node Labels in opportunistic scheduling. Contributed by Abhishek Modi. 2018-05-04 14:59:59 -07:00
Eric Yang 8cdb032aff YARN-8223. Improved yarn auxiliary service to load jar file from HDFS. 
Contributed by Zian Chen
 2018-05-04 12:36:31 -04:00
Billie Rinaldi 6795f8072f YARN-8079. Support static and archive unmodified local resources in service AM. Contributed by Suma Shivaprasad 2018-05-04 09:27:07 -07:00
Billie Rinaldi 502914ca32 YARN-7818. Remove privileged operation warnings during container launch for the ContainerRuntimes. Contributed by Shane Kumpf 2018-05-04 08:53:55 -07:00
Eric Yang 7698737207 YARN-8226. Improved anti-affinity description in YARN Service doc. 
Contributed by Gour Saha
 2018-05-03 13:35:40 -04:00
Eric Yang 7fe3214d4b YARN-7961. Improve status message for YARN service. 
Contributed by Gour Saha
 2018-05-03 13:27:07 -04:00
Rohith Sharma K S ee2ce923a9 YARN-8217. RmAuthenticationFilterInitializer and TimelineAuthenticationFilterInitializer should use Configuration.getPropsWithPrefix instead of iterator. Contributed by Suma Shivaprasad. 2018-05-03 14:43:40 +05:30
Inigo Goiri e6a80e476d YARN-8151. Yarn RM Epoch should wrap around. Contributed by Young Chen. 2018-05-02 17:23:17 -07:00
Eric Yang f4d280f02b YARN-8194. Fixed reinitialization error for LinuxContainerExecutor. 
Contributed by Chandni Singh
 2018-05-02 20:07:19 -04:00
Konstantinos Karanasos 3b34fca4b5 YARN-8113. Update placement constraints doc with application namespaces and inter-app constraints. Contributed by Weiwei Yang. 2018-05-02 11:49:56 -07:00
Eric Yang 883f68222a YARN-8209. Fixed NPE in Yarn Service deletion. 
Contributed by Eric Badger
 2018-05-02 14:33:31 -04:00
Weiwei Yang 251f528814 YARN-8222. Fix potential NPE when gets RMApp from RM context. Contributed by Tao Yang. 2018-05-02 17:54:46 +08:00
Yiqun Lin 3265b55119
YARN-6385. Fix checkstyle warnings in TestFileSystemApplicationHistoryStore 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-02 18:14:02 +09:00
Eric Yang 24eeea8b18 YARN-7799. Improved YARN service jar file handling. 
Contributed by Billie Rinaldi
 2018-05-01 16:46:34 -04:00
Billie Rinaldi 4e1382aca4 YARN-2674. Fix distributed shell AM container relaunch during RM work preserving restart. Contributed by Shane Kumpf 2018-05-01 07:27:47 -07:00
Sunil G d6139c5106 YARN-8187. [UI2] Individual Node page does not contain breadcrumb trail. Contributed by Zian Chen. 2018-05-01 14:01:34 +05:30
Weiwei Yang 2d319e3793 YARN-8212. Pending backlog for async allocation threads should be configurable. Contributed by Tao Yang. 2018-05-01 09:47:10 +08:00
Eric Yang a966ec6e23 YARN-8228. Added hostname length check for docker container. 
Contributed by Shane Kumpf
 2018-04-30 19:12:53 -04:00
Konstantinos Karanasos 9b09555451 YARN-8195. Fix constraint cardinality check in the presence of multiple target allocation tags. Contributed by Weiwei Yang. 2018-04-30 11:54:30 -07:00
Weiwei Yang 2c95eb8111 YARN-8225. YARN precommit build failing in TestPlacementConstraintTransformations. Contributed by Shane Kumpf. 2018-04-28 17:37:37 +08:00
Wangda Tan 92c5331423 YARN-8005. Add unit tests for queue priority with dominant resource calculator. (Zian Chen via wangda) 
Change-Id: I17a645f20869a1e5d86fa7a325c93fec908b91dc
 2018-04-27 13:08:43 -07:00
Wangda Tan b1833d9ba2 YARN-8210. AMRMClient logging on every heartbeat to track updation of AM RM token causes too many log lines to be generated in AM logs. (Suma Shivaprasad via wangda) 
Change-Id: I70edd6e301fd5e78d479e1882aedc9332a0827aa
 2018-04-27 13:07:38 -07:00
Rohith Sharma K S ef3ecc308d YARN-8221. RMWebServices also need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 2018-04-27 22:58:10 +05:30
Eric Yang 24a5ccbf4b YARN-7781. Update YARN service documentation. 
Contributed by Gour Saha
 2018-04-27 12:38:30 -04:00
Eric Yang 14b47990af YARN-8204. Added a flag to disable YARN service upgrade. 
Contributed by Chandni Singh
 2018-04-27 12:24:43 -04:00
Billie Rinaldi 9ab3f97089 YARN-8211. Yarn registry dns log finds BufferUnderflowException on port ping. Contributed by Eric Yang 2018-04-27 07:58:43 -07:00
Billie Rinaldi 914d6b5cb1 YARN-8208. Add log statement for Docker client configuration file at INFO level. Contributed by Yesha Vora 2018-04-27 06:10:32 -07:00
Ewan Higgs 84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-27 19:21:37 +09:00
Sunil G 71220d218d YARN-8004. Add unit tests for inter queue preemption for dominant resource calculator. Contributed by Zian Chen. 2018-04-27 10:42:30 +05:30
Sunil G 7fc09c4f72 YARN-8215. ATS v2 returns invalid YARN_CONTAINER_ALLOCATED_HOST_HTTP_ADDRESS from NM. Contributed by Rohith Sharma K S. 2018-04-27 10:33:42 +05:30
Sunil G 1634de0fc1 YARN-8205. Application State is not updated to ATS if AM launching is delayed. Contributed by Rohith Sharma K S. 2018-04-27 10:26:57 +05:30
Eric Yang 93979db840 YARN-8122. Added component health monitoring for YARN service. 
Contributed by Gour Saha
 2018-04-26 19:17:32 -04:00
Billie Rinaldi 598483e496 YARN-8140. Improve log message when launch cmd is ran for stopped yarn service. Contributed by Eric Yang 2018-04-26 13:54:40 -07:00
Eric Yang 4a7369b095 YARN-7939. Added support to upgrade a component instance. 
Contributed by Chandni Singh
 2018-04-26 13:54:40 -07:00
Wangda Tan 2a0fa50f9d YARN-8193. YARN RM hangs abruptly (stops allocating resources) when running successive applications. (Zian Chen via wangda) 
Change-Id: Ia83dd2499ee9000b9e09ae5a932f21a13c0ddee6
 2018-04-26 13:54:40 -07:00
Sunil G 7ac0abdc13 YARN-8188. RM Nodes UI data table index for sorting column need to be corrected post Application tags display. Contributed by Weiwei Yang. 2018-04-26 13:54:40 -07:00
Wangda Tan ebedd69e25 YARN-8183. Fix ConcurrentModificationException inside RMAppAttemptMetrics#convertAtomicLongMaptoLongMap. (Suma Shivaprasad via wangda) 
Change-Id: I347871d672001653a3afe2e99adefd74e0d798cd
 2018-04-26 13:54:40 -07:00
Eric Yang 3b1dad9123 YARN-8196. Updated documentation for enabling YARN Service REST API. 
Contributed by Billie Rinaldi
 2018-04-26 13:54:40 -07:00
Owen O'Malley 2adda92de1
HDFS-7240 git commit fixup 
Signed-off-by: Owen O'Malley <omalley@apache.org>
 2018-04-26 13:52:17 -07:00
Anu Engineer 4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects. 
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
 2018-04-26 05:36:04 -07:00
Junping Du 56788d759f YARN-7598. Document how to use classpath isolation for aux-services in YARN. Contributed by Xuan Gong. 2018-04-24 18:29:14 +08:00
Robert Kanter 7ab08a9c37 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 2018-04-23 15:44:15 -07:00
Sunil G 83e5f25d5e YARN-8177. Documentation changes for auto creation of Leaf Queues with node label. Contributed by Suma Shivaprasad. 2018-04-23 13:36:27 +05:30
Sunil G b6d94fad08 YARN-7956. [UI2] Avoid duplicating Components link under Services/<ServiceName>/Components. Contributed by Yesha Vora. 2018-04-23 13:32:09 +05:30
Konstantinos Karanasos 28e2244390 YARN-8111. Simplify PlacementConstraints API by removing allocationTagToIntraApp. Contributed by Weiwei Yang. 2018-04-20 12:24:48 -07:00
Jason Lowe 766544c0b0 YARN-7786. NullPointerException while launching ApplicationMaster. Contributed by lujie 2018-04-20 13:22:20 -05:00
Eric Yang 85585f9eef YARN-8064. Bugfix for clean up container-executor cmd helper file. 
Contributed by Eric Badger
 2018-04-20 13:49:33 -04:00
Sunil G 7fd878813b Revert "YARN-7956. [UI2] Avoid duplicating Components link under Services/<ServiceName>/Components. Contributed by Yesha Vora." 
This reverts commit a11ce19f47.
 2018-04-20 22:33:54 +05:30
Sunil G a11ce19f47 YARN-7956. [UI2] Avoid duplicating Components link under Services/<ServiceName>/Components. Contributed by Yesha Vora. 2018-04-20 22:28:36 +05:30
Haibo Chen 860cc28a5a YARN-8185. Improve log in DirectoryCollection constructor. (Yufei Gu via Haibo Chen) 2018-04-20 07:41:24 -07:00
Sunil G 55e4252fc5 YARN-7830. [UI2] Post selecting grid view in Attempt page, attempt info page should also be opened with grid view. Contributed by Gergely Novák. 2018-04-20 20:05:05 +05:30
Rohith Sharma K S b0753cc2c0 YARN-8189. [UI2] Nodes page column headers are half truncated. Contributed by Sunil G. 2018-04-20 19:36:26 +05:30
Rohith Sharma K S 87e09859f1 YARN-8182. [UI2] Proxy- Clicking on nodes under Nodes HeatMap gives 401 error. Contributed by Sunil G. 2018-04-20 19:35:17 +05:30
Inigo Goiri da5bcf5f7d YARN-8186. [Router] Federation: routing getAppState REST invocations transparently to multiple RMs. Contributed by Giovanni Matteo Fumarola. 2018-04-19 16:49:29 -07:00
Sunil G 7d06806dfd YARN-6827. [ATS1/1.5] NPE exception while publishing recovering applications into ATS during RM restart. Contributed by Rohith Sharma K S. 2018-04-20 00:05:53 +05:30
Sunil G c6d7d3eb05 YARN-5888. [UI2] Improve unit tests for new YARN UI. Contributed by Akhil PB. 2018-04-19 21:37:35 +05:30
BELUGA BEHR 351e509a47
YARN-8169. Cleanup RackResolver.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-19 16:07:14 +09:00
Shashikant Banerjee e4c39f3247
HADOOP-15396. Some java source files are executable 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-19 08:35:38 +09:00
Rohith Sharma K S 034da8fc97 YARN-8171. [UI2] AM Node link from attempt page should not redirect to new tab. Contributed by Sunil G. 2018-04-18 13:39:41 +05:30
Rohith Sharma K S 241d985467 YARN-8145. yarn rmadmin -getGroups doesn't return updated groups for user. Contributed by Sunil G. 2018-04-18 13:39:41 +05:30
Yufei d426b980ac YARN-7966. Remove method AllocationConfiguration#getQueueAcl and related unit tests. Contributed by Sen Zhao. 2018-04-17 22:53:32 -07:00
Inigo Goiri db1bba857a YARN-8096. Wrong condition in AmIpFilter#getProxyAddresses() to update the proxy IP list. Contributed by Oleksandr Shevchenko. 2018-04-17 13:08:01 -07:00
Haibo Chen bb92bfb4ef YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 11:15:20 -07:00
Jason Lowe 1d6e43da51 YARN-7996. Allow user supplied Docker client configurations with YARN native services. Contributed by Shane Kumpf 2018-04-17 08:57:31 -05:00
Haibo Chen 86dfb70875 Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)" 
This reverts commit 0c14b6f0d8.
 2018-04-17 06:04:38 -07:00
Haibo Chen 0c14b6f0d8 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 05:59:03 -07:00
Sunil G 427ad7ecc4 YARN-8126. Support auto-spawning of admin configured services during bootstrap of RM. Contributed by Rohith Sharma K S. 2018-04-17 15:08:31 +05:30
Inigo Goiri f1461b2487 YARN-8164. Fix a potential NPE in AbstractSchedulerPlanFollower. Contributed by lujie. 2018-04-16 17:32:19 -07:00
Inigo Goiri dd5e18c4ae YARN-8165. Incorrect queue name logging in AbstractContainerAllocator. Contributed by Weiwei Yan. 2018-04-16 16:52:17 -07:00
Inigo Goiri 60e5c1b516 YARN-7773. YARN Federation used Mysql as state store throw exception, Unknown column 'homeSubCluster' in 'field list'. Contributed by Yiran Wu. 2018-04-16 16:26:54 -07:00
Haibo Chen 49f9acae40 YARN-8162. Remove Method DirectoryCollection#verifyDirUsingMkdir. (Yufei Gu via Haibo Chen) 2018-04-16 15:05:04 -07:00
Eric Yang 2d0662cfd5 YARN-8060. Added default readiness check for service components. 
Contributed by Billie Rinaldi
 2018-04-16 14:24:26 -04:00
Wangda Tan 896b473f1b YARN-8138. Add unit test to validate queue priority preemption works under node partition. (Zian Chen via wangda) 
Change-Id: Ibebfab98a714c12c2dc643b6d7b9754a7f813632
(cherry picked from commit 6ee62e6b1c)
 2018-04-14 11:08:36 -07:00
Rohith Sharma K S 669eb7bdea YARN-8156. Increase the default value of yarn.timeline-service.app-collector.linger-period.ms. Contributed by Charan Hebri. 2018-04-14 10:31:28 +05:30
Eric Yang 9031a76d44 YARN-8142. Improve SIGTERM handling for YARN Service Application Master. 
Contributed by Billie Rinaldi
 2018-04-13 15:34:33 -04:00
Weiwei Yang 226bedc023 YARN-8153. Guaranteed containers always stay in SCHEDULED on NM after restart. Contributed by Yang Wang. 2018-04-13 13:17:37 +08:00
Weiwei Yang 375654c36a YARN-8154. Fix missing titles in PlacementConstraints document. Contributed by Weiwei Yang. 2018-04-13 13:06:47 +08:00
Billie Rinaldi 53b3e59473 YARN-7936. Add default service AM Xmx. Contributed by Jian He 2018-04-12 12:38:00 -07:00
Eric E Payne 18844599ae YARN-8147. TestClientRMService#testGetApplications sporadically fails. Contributed by Jason Lowe 2018-04-12 17:53:57 +00:00
Jason Lowe 4571351ccc MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 2018-04-12 11:12:46 -05:00
Eric E Payne 6bb128dfb8 YARN-8120. JVM can crash with SIGSEGV when exiting due to custom leveldb logger. Contributed by Jason Lowe. 2018-04-12 16:04:23 +00:00
Haibo Chen b8597858b1 YARN-7931. [atsv2 read acls] Include domain table creation as part of schema creator. (Vrushali C via Haibo Chen) 2018-04-12 06:38:30 -07:00
Billie Rinaldi 933477e9e0 YARN-7221. Add security check for privileged docker container. Contributed by Eric Yang 2018-04-11 11:24:23 -07:00
Weiwei Yang 7eb783e263 YARN-8127. Resource leak when async scheduling is enabled. Contributed by Tao Yang. 2018-04-11 17:15:25 +08:00
Rohith Sharma K S c048711099 YARN-7941. Transitive dependencies for component are not resolved. Contributed by Billie Rinaldi. 2018-04-11 09:18:50 +05:30
Wangda Tan d919eb6efa YARN-8133. Doc link broken for yarn-service from overview page. (Rohith Sharma K S via wangda) 
Change-Id: Iacf9a004585dd59e1c0cd8f8c618a38f047cc0fe
 2018-04-10 17:33:15 -07:00
Wangda Tan 2bf9cc2c73 YARN-8116. Nodemanager fails with NumberFormatException: For input string: . (Chandni Singh via wangda) 
Change-Id: Idd30cfca59982d3fc6e47aa1b88f844a78fae94d
 2018-04-10 17:32:38 -07:00
Eric Yang c467f311d0 YARN-7973. Added ContainerRelaunch feature for Docker containers. 
Contributed by Shane Kumpf
 2018-04-10 19:25:00 -04:00
Eric Yang d553799030 YARN-7984. Improved YARN service stop/destroy and clean up. 
Contributed by Billie Rinaldi
 2018-04-10 17:40:49 -04:00
Sunil G 7c1e77dda4 YARN-7804. [UI2] Refresh action on Grid view page should not be redirected to graph view. Contributed by Gergely Novák. 2018-04-10 16:09:09 +05:30
Jason Lowe 907919d28c YARN-7667. Docker Stop grace period should be configurable. Contributed by Eric Badger 2018-04-09 17:19:21 -05:00
Sunil G 821b0de4c5 YARN-7574. Add support for Node Labels on Auto Created Leaf Queue Template. Contributed by Suma Shivaprasad. 2018-04-09 21:17:22 +05:30
bibinchundatt eb47c3de74 YARN-7905. Parent directory permission incorrect during public localization. Contributed by Bilwa S T. 2018-04-07 12:26:29 +05:30
Wangda Tan 00ebec89f1 YARN-1151. Ability to configure auxiliary services from HDFS-based JAR files. (Xuan Gong via wangda) 
Change-Id: Ied37ff11e507fc86847753ba79486652c8fadfe9
 2018-04-06 21:25:57 -07:00
Wangda Tan d4e63ccca0 YARN-8048. Support auto-spawning of admin configured services during bootstrap of RM (Rohith Sharma K S via wangda) 
Change-Id: I2d8d61ccad55e1118009294d7e17822df3cd0fd5
 2018-04-06 21:24:58 -07:00
Subru Krishnan 00905efab2 YARN-8110. AMRMProxy recover should catch for all throwable to avoid premature exit. (Botong Huang via Subru). 2018-04-06 16:31:16 -07:00
Haibo Chen 024d7c0870 YARN-8107. Give an informative message when incorrect format is used in ATSv2 filter attributes. (Rohith Sharma K S via Haibo Chen) 2018-04-06 09:39:01 -07:00
Sunil G b17dc9f5f5 YARN-8083. [UI2] All YARN related configurations are paged together in conf page. Contributed by Gergely Novák. 2018-04-06 21:53:14 +05:30
Wangda Tan 6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release 
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
 2018-04-05 15:52:39 -07:00
Rohith Sharma K S f32d6275ba YARN-8119. [UI2] Timeline Server address' url scheme should be removed while accessing via KNOX. Contributed by Sunil G. 2018-04-05 23:32:35 +05:30
Haibo Chen f8b8bd53c4 YARN-6936. [Atsv2] Retrospect storing entities into sub application table from client perspective. (Rohith Sharma K S via Haibo Chen) 2018-04-05 10:23:42 -07:00
Vrushali C 345e7624d5 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 2018-04-04 15:08:03 -07:00
Haibo Chen 3087e89135 YARN-7946. Update TimelineServerV2 doc as per YARN-7919. (Haibo Chen) 2018-04-04 11:59:31 -07:00
Konstantinos Karanasos 7853ec8d2f YARN-8013. Support application tags when defining application namespaces for placement constraints. Contributed by Weiwei Yang. 2018-04-04 10:51:58 -07:00
Sunil G 42cd367c93 YARN-8115. [UI2] URL data like nodeHTTPAddress must be encoded in UI before using to access NM. Contributed by Sreenath Somarajapuram. 2018-04-04 22:13:14 +05:30
Wangda Tan b779f4f0f6 YARN-8106. Update LogAggregationIndexedFileController to use readFull instead read to avoid IOException while loading log meta. (Prabhu Joseph via wangda) 
Change-Id: I63a65f73f8d1636e2c99ed9c8c2bbd05efcff80f
 2018-04-03 21:06:24 -07:00
Sunil G f7a17b029d YARN-7764. Findbugs warning: Resource#getResources may expose internal representation. Contributed by Weiwei Yang. 2018-04-04 09:22:35 +05:30
Miklos Szegedi 2d06d885c8 YARN-8035. Uncaught exception in ContainersMonitorImpl during relaunch due to the process ID changing. Contributed by Shane Kumpf. 2018-04-03 10:01:00 -07:00
Haibo Chen 93d47a0ed5 YARN-8051. TestRMEmbeddedElector#testCallbackSynchronization is flaky. (Robert Kanter via Haibo Chen) 2018-04-03 07:59:20 -07:00
Wangda Tan c78cb18c61 YARN-8091. Revisit checkUserAccessToQueue RM REST API. (wangda) 
Change-Id: I5fab3fe229c34e967487b7327c7b3c8ddf7cb795
 2018-04-02 15:22:05 -07:00
Jason Lowe 54a81211ea YARN-8082. Include LocalizedResource size information in the NM download log for localization. Contributed by Kuhu Shukla 2018-04-02 10:38:13 -05:00
Wangda Tan a0bde7d525 YARN-7142. Support placement policy in yarn native services. (Gour Saha via wangda) 
Change-Id: I166c67a7a34430627c17365f60bac75b6da1b434
 2018-04-02 07:26:01 -07:00
Weiwei Yang 1606dad133 YARN-7497. Add file system based scheduler configuration store. Contributed by Jiandan Yang. 2018-03-30 21:41:33 +08:00
Wangda Tan e7e20190df YARN-8086. ManagedParentQueue with no leaf queues cause JS error in new UI. (Suma Shivaprasad via wangda) 
Change-Id: I6d82af91adec02698afddde6883b1fe1924e6041
 2018-03-29 15:03:31 -07:00
Eric Yang 8fb00c3fce YARN-6257. Fix CapacityScheduler REST API JSON output. 
Contributed By Tao Yang
 2018-03-29 17:36:34 -04:00
Sunil G 7a59d60e0c YARN-8085. ResourceProfilesManager should be set in RMActiveServiceContext. Contributed by Tao Yang. 2018-03-29 21:41:16 +05:30
Sunil G 431076f637 YARN-8076. Support to specify application tags in distributed shell. Contributed by Weiwei Yang. 2018-03-29 15:55:39 +05:30
Eric Yang 3d185d62fc YARN-8069. Clean up example hostnames for RegistryDNS. 
Contributed by Billie Rinaldi
 2018-03-28 20:17:37 -04:00
Zhe Zhang 0b1c2b5fe1 YARN-7623. Fix the CapacityScheduler Queue configuration documentation. Contributed by Jonathan Hung. 2018-03-28 11:41:20 -07:00
Subru Krishnan 09999d7e01 YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 2018-03-28 11:33:19 -07:00
Subru Krishnan 725b10e3ae Revert "YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru)." 
This reverts commit 2a2ef15caf as smart-apply-patch script didn't pick the latest patch.
 2018-03-28 11:26:50 -07:00
Wangda Tan 47f711eebc YARN-6629. NPE occurred when container allocation proposal is applied but its resource requests are removed before. (Tao Yang via wangda) 
Change-Id: I805880f90b3f6798ec96ed8e8e75755f390a9ad5
 2018-03-28 11:07:45 -07:00
Miklos Szegedi cdee0a4f84 YARN-8077. The vmemLimit parameter in ContainersMonitorImpl#isProcessTreeOverLimit is confusing. Contributed by Sen Zhao. 2018-03-28 09:05:25 -07:00
Weiwei Yang 411993f6e5 YARN-7734. Fix UT failure TestContainerLogsPage#testContainerLogPageAccess. Contributed by Tao Yang. 2018-03-28 17:35:38 +08:00
Sunil G d1e378d02b YARN-8075. DShell does not fail when we ask more GPUs than available even though AM throws 'InvalidResourceRequestException'. Contributed by Wangda Tan. 2018-03-28 08:03:18 +05:30
Subru Krishnan 2a2ef15caf YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 2018-03-27 17:39:46 -07:00
Eric Yang 27d60a1634 YARN-8018. Added support for initiating yarn service upgrade. 
Contributed by Chandni Singh
 2018-03-26 18:46:31 -04:00
Sean Mackrory 82665a7887 HADOOP-15299. Bump Jackson 2 version to Jackson 2.9.x. 2018-03-26 14:05:15 -06:00
Wangda Tan 24bc5e0e4d YARN-8072. RM log is getting flooded with MemoryPlacementConstraintManager info logs. (Zian Chen via wangda) 
Change-Id: I2567f4669ad868e13c4301fe44c4bde85bb92c37
 2018-03-26 11:20:16 -07:00
Wangda Tan 9eef19b2ad YARN-8068. Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ (Sunil G via wangda) 
Change-Id: I7910bd1064a1b4dbbe2084080c060822ea6f3b48
 2018-03-26 11:19:15 -07:00
Wangda Tan 5d381570f8 YARN-8062. yarn rmadmin -getGroups returns group from which the user has been removed. (Sunil G via wangda) 
Change-Id: I80ed63846502bf7751b890b6c6c6a7c0679e2b4a
 2018-03-26 11:16:06 -07:00
Eric Yang e6b74164e9 YARN-8043. Added the exception message for failed launches running under LCE. 
Contributed by Shane Kumpf
 2018-03-26 13:21:35 -04:00
Wangda Tan 24f75e097a YARN-8070. Yarn Service API site doc broken due to unwanted character in YarnServiceAPI.md (Gour Saha via wangda) 
Change-Id: I22428b2f128d16e79ebbdeaf32869566963d0c6f
 2018-03-23 16:44:44 -07:00
Wangda Tan a90471b3e6 YARN-8016. Refine PlacementRule interface and add a app-name queue mapping rule as an example. (Zian Chen via wangda) 
Change-Id: I35caf1480e0f76f5f3a53528af09312e39414bbb
 2018-03-23 16:43:40 -07:00
Eric Yang 647058efc0 YARN-8032. Added ability to configure failure validity interval for YARN service. 
Contributed by Chandni Singh
 2018-03-23 15:54:52 -04:00
Rohith Sharma K S 75fc05f369 YARN-7986. ATSv2 REST API queries do not return results for uppercase application tags. Contributed by Charan Hebri. 2018-03-23 12:16:30 +05:30
Sunil G 22c5ddb7c4 YARN-8063. DistributedShellTimelinePlugin wrongly check for entityId instead of entityType. Contributed by Rohith Sharma K S. 2018-03-23 10:54:00 +05:30
Jason Lowe 5aa7052e31 YARN-8054. Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread. Contributed by Jonathan Eagles 2018-03-21 17:19:20 -05:00
Rohith Sharma K S 29acea5000 YARN-7581. HBase filters are not constructed correctly in ATSv2. Contributed by Habio Chen. 2018-03-21 08:11:19 +05:30
Haibo Chen 3ff6977d3e YARN-8053. Add hadoop-distcp in exclusion in hbase-server dependencies for timelineservice-hbase packages. (Rohith Sharma K S via Haibo Chen) 2018-03-20 04:36:21 -07:00
Jason Lowe f480367af6 YARN-8027. Setting hostname of docker container breaks for --net=host in docker 1.13. Contributed by Jim Brennan 2018-03-19 16:59:58 -05:00
Wangda Tan a08921ca6c YARN-8002. Support NOT_SELF and ALL namespace types for allocation tag. (Weiwei Yang via wangda) 
Change-Id: I63b4e4192a95bf7ded98c54e46a2871c72869700
 2018-03-19 11:04:27 -07:00
Sunil G 6bd130ea45 YARN-8028. Support authorizeUserAccessToQueue in RMWebServices. Contributed by Wangda Tan. 2018-03-18 11:00:30 +05:30
Sunil G 98356a3dde YARN-8040. [UI2] New YARN UI webapp does not respect current pathname for REST api. Contributed by Sunil G. 2018-03-18 10:44:33 +05:30
Yufei Gu 49c747ab18 YARN-8039. Clean up log dir configuration in TestLinuxContainerExecutorWithMocks.testStartLocalizer. Contributed by Miklos Szegedi. 2018-03-16 16:25:33 -07:00
Weiwei Yang 154cfb2b62 YARN-7636. Re-reservation count may overflow when cluster resource exhausted for a long time. contributed by Tao Yang. 2018-03-16 18:57:31 +08:00
Wangda Tan 4bf622043f YARN-7952. RM should be able to recover log aggregation status after restart/fail-over. (Xuan Gong via wangda) 
Change-Id: I725c9afe64831eda0aa6b0bebdbc79d2dd165707
 2018-03-15 13:26:45 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Eric Yang a82be7754d YARN-7999. Added file cache initializer for Linux container-executor. 
Contributed by Jason Lowe
 2018-03-14 12:21:57 -04:00
Rohith Sharma K S e6de10d0a6 YARN-8022. ResourceManager UI cluster/app/<app-id> page fails to render. Contributed by Tarun Parimi. 2018-03-14 09:42:20 +05:30
Wangda Tan b167d60763 YARN-7657. Queue Mapping could provide options to provide 'user' specific auto-created queues under a specified group parent queue. (Suma Shivaprasad via wangda) 
Change-Id: I32d566e8727840e43c0d66e39a77edef017e3a83
 2018-03-13 17:55:51 -07:00
Wangda Tan a5b27b3c67 YARN-5015. Support sliding window retry capability for container restart. (Chandni Singh via wangda) 
Change-Id: I07addd3e4ba8d98456ee2ff1d5c540a38fe61dea
 2018-03-13 17:55:17 -07:00
Miklos Szegedi a82d4a2e3a YARN-5764. NUMA awareness support for launching containers. Contributed by Devaraj K. 2018-03-13 12:36:57 -07:00
Yufei Gu ff31d8aefa YARN-8024. LOG in class MaxRunningAppsEnforcer is initialized with a faulty class. Contributed by Sen Zhao. 2018-03-12 16:35:26 -07:00
Billie Rinaldi e1f5251f3c YARN-7523. Introduce description and version field in Service record. Contributed by Chandni Singh 2018-03-10 07:49:10 -08:00
Wangda Tan ea18e70a74 Updated timeline reader to use AuthenticationFilter 
Change-Id: I961771589180c1eb377d36c37a79aa23754effbf
(cherry picked from commit 837338788eb903d0e8bbb1230694782a707891be)
 2018-03-09 22:51:08 -08:00
Owen O'Malley 3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu." 
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
 2018-03-09 22:46:30 -08:00
Billie Rinaldi 4f395063bb YARN-8000. Yarn Service component instance name shows up as component name in container record. Contributed by Chandni Singh 2018-03-09 08:50:28 -08:00
Sunil G 113f401f41 YARN-7944. [UI2] Remove master node link from headers of application pages. Contributed by Yesha Vora. 2018-03-08 23:53:36 +05:30
Weiwei Yang b451889e8e YARN-8011. TestOpportunisticContainerAllocatorAMService#testContainerPromoteAndDemoteBeforeContainerStart fails intermittently. Contributed by Tao Yang. 2018-03-08 18:13:36 +08:00
Wangda Tan 583f459431 YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: I16e081f21c5f80160564c49cc49d103bd8eb7e16
 2018-03-07 15:46:47 -08:00
Wangda Tan e718ac597f Revert "YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)" 
This reverts commit 4d53ef7eef.
 2018-03-07 15:46:47 -08:00
Wangda Tan 037d783483 YARN-7626. Allow regular expression matching in container-executor.cfg for devices and named docker volumes mount. (Zian Chen via wangda) 
Change-Id: If461277d4557922ab7e4dce9dd8dc5d0d5f22710
(cherry picked from commit 88f9138e12d2d5a1bd13f0915acef93037c1d086)
 2018-03-07 11:30:15 -08:00
Wangda Tan 4d53ef7eef YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: Ie16e34039d57df50128c73b37516ad0bc7c9590e
 2018-03-07 11:30:06 -08:00
Jason Lowe d69b31f7f7 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-03-07 09:20:34 -06:00
Rohith Sharma K S 55ba49dd07 YARN-7346. Add a profile to allow optional compilation for ATSv2 with HBase-2.0. Contributed by Haibo Chen and Rohith. 2018-03-06 11:55:21 +05:30
Takanobu Asanuma 745190ecdc
HADOOP-15287. JDK9 JavaDoc build fails due to one-character underscore identifiers in hadoop-yarn-common 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 18:15:53 -08:00
Sen Zhao 245751ffdc
YARN-7736. Fix itemization in YARN federation document 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 11:54:24 -08:00
Arun Suresh 1054b48c27 YARN-7972. Support inter-app placement constraints for allocation tags by application ID. (Weiwei Yang via asuresh) 2018-03-05 11:24:17 -08:00
Billie Rinaldi 628be58a4c YARN-7915. Trusted image log message repeated multiple times. Contributed by Shane Kumpf 2018-03-05 09:08:44 -08:00
Weiwei Yang e8c5be63f0 YARN-7995. Remove unnecessary boxings and unboxings from PlacementConstraintParser.java. Contributed by Sen Zhao. 2018-03-05 11:49:43 +08:00
Jason Lowe 83798f15f8 YARN-7511. NPE in ContainerLocalizer when localization failed for running container. Contributed by Tao Yang 2018-03-02 09:59:46 -06:00
Ray Chiang 923e1773e1 HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang) 2018-03-01 15:36:10 -08:00
Sunil G 22928c0c60 YARN-7637. GPU volume creation command fails when work preserving is disabled at NM. Contributed by Zian Chen. 2018-03-01 22:39:32 +05:30
Yufei Gu 6ca4e3017b YARN-5028. RMStateStore should trim down app state for completed applications(addendum). Contributed by Rohith Sharma K S. 2018-03-01 08:38:53 -08:00
Billie Rinaldi 5ed689e33a YARN-7958. ServiceMaster should only wait for recovery of containers with id that match the current application id. Contributed by Chandni Singh 2018-03-01 08:00:13 -08:00
Haibo Chen 10663b78c8 Revert "yarn-7346.07.patch" 
This reverts commit 5e37ca5bb4.
 2018-02-28 21:11:36 -08:00
Haibo Chen d1274c3b71 YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen) 2018-02-28 21:10:42 -08:00
Haibo Chen 5e37ca5bb4 yarn-7346.07.patch 2018-02-28 21:04:54 -08:00
Eric Yang 17f387e3c2 YARN-7985. Removed duplicate service name validation in Yarn Service. 
Contributed by Chandni Singh
 2018-02-28 20:48:55 -05:00
Billie Rinaldi 315f48e791 YARN-7955. Improve result of calling stop on an already stopped service. Contributed by Gour Saha 2018-02-28 15:01:56 -08:00
Billie Rinaldi 31009034bd YARN-7954. Set component status to STOPPED when YARN service is stopped. Contributed by Gour Saha 2018-02-28 11:57:24 -08:00
Billie Rinaldi 727c033997 YARN-7446. Remove --user flag when running privileged mode docker container. Contributed by Eric Yang 2018-02-27 14:33:57 -08:00
Wangda Tan eea0b069e7 YARN-7959. Add .vm extension to PlacementConstraints.md to ensure proper filtering. (Weiwei Yang via wangda) 
Change-Id: I34ec516994a5de7accaa5ef7044e443b340bd294
(cherry picked from commit 0d516b30769471d9114214b3a518e2146ff2f25e)
 2018-02-27 13:19:24 -08:00
Wangda Tan 5bbbbb6a1e YARN-7893. Document the FPGA isolation feature. (Zhankun Tang via wangda) 
Change-Id: I8723320377a4bd0a7db168670d183ef5543caa67
(cherry picked from commit c2af2f16718bd1d38ceb05e354f34d7bef89127d)
 2018-02-27 13:19:16 -08:00
Eric Yang b4f1ba1413 YARN-7963. Updated MockServiceAM unit test to prevent test hang. 
Contributed by Chandni Singh
 2018-02-26 18:49:01 -05:00
Konstantinos Karanasos e85188101c YARN-7921. Transform a PlacementConstraint to a string expression. Contributed by Weiwei Yang. 2018-02-26 12:15:16 -08:00
Arpit Agarwal 2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Jason Lowe 8e728f39c9 YARN-5714. ContainerExecutor does not order environment map. Contributed by Remi Catherinot and Jim Brennan 2018-02-23 15:46:35 -06:00
Sunil G d1cd573687 YARN-7949. [UI2] ArtifactsId should not be a compulsory field for new service. Contributed by Yesha Vora. 2018-02-23 16:50:02 +05:30
Sunil G cc683952d2 YARN-7675. [UI2] Support loading pre-2.8 version /scheduler REST response for queue page. Contributed by Gergely Novák. 2018-02-23 16:10:29 +05:30
Gabor Bota c36b4aa31c
HADOOP-15236. Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-23 13:55:18 +09:00
Carlo Curino 514794e1a5 YARN-7934. [GQ] Refactor preemption calculators to allow overriding for Federation Global Algos. (Contributed by curino) 2018-02-22 18:12:12 -08:00
Eric Yang 95904f6b3c YARN-7942. Add check for JAAS configuration for Yarn Service. 
Contributed by Billie Rinaldi
 2018-02-22 16:12:40 -05:00
Eric Yang 190969006d YARN-7836. Added error check for updating service components. 
(Contributed by Gour Saha)
 2018-02-22 16:08:30 -05:00
Jason Lowe 3132709b46 YARN-7916. Remove call to docker logs on failure in container-executor. Contributed by Shane Kumpf 2018-02-21 16:54:02 -06:00
Yufei Gu 92cbbfe79e YARN-5028. RMStateStore should trim down app state for completed applications. Contributed by Gergo Repas. 2018-02-21 11:42:51 -08:00
Sunil G bdd2a184d7 YARN-7947. Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps. Contributed by Eric Payne. 2018-02-21 14:35:57 +05:30
Sunil G 86b227a1fb YARN-7223. Document GPU isolation feature. Contributed by Wangda Tan. 2018-02-21 14:16:45 +05:30
Eric Yang 7280c5af82 YARN-7940. Fixed a bug in ServiceAM ZooKeeper initialization. 
(Contributed by Billie Rinaldi)
 2018-02-20 14:12:58 -05:00
Jason Lowe 9497215084 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:06:28 -06:00
Jason Lowe b9a429bb28 Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 8013475d44.
 2018-02-19 08:16:25 -06:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Rohith Sharma K S 87bdde6943 YARN-7937. Fix http method name in Cluster Application Timeout Update API example request. Contributed by Charan Hebri. 2018-02-18 14:01:23 +05:30
Rohith Sharma K S 9af30d46c6 YARN-7919. Refactor timelineservice-hbase module into submodules. Contributed by Haibo Chen. 2018-02-17 20:30:28 +05:30
Arun Suresh a1e56a6286 YARN-7918. Fix TestAMRMClientPlacementConstraints. (Gergely Novák via asuresh) 2018-02-17 03:24:55 -08:00
Wangda Tan ca1043ab90 YARN-7328. ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores. (lovekesh bansal via wangda) 
Change-Id: Ibb1faf0beefec079dae2208986976b2f64650672
 2018-02-17 12:28:08 +08:00
Sunil G aae629913c YARN-7292. Retrospect Resource Profile Behavior for overriding capability. Contributed by Wangda Tan. 2018-02-16 12:37:40 +05:30
Jason Lowe 8013475d44 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-02-15 17:09:00 -06:00
Konstantinos Karanasos 0b489e564c YARN-7920. Simplify configuration for PlacementConstraints. Contributed by Wangda Tan. 2018-02-15 14:23:38 -08:00
Eric Payne bddfe42e2c Revert "YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue" 
This reverts commit c5e6e3de1c.
 2018-02-14 14:29:20 -06:00
Wangda Tan 042ef2fa7b YARN-7789. Should fail RM if 3rd resource type is configured but RM uses DefaultResourceCalculator. (Zian Chen via wangda) 
Change-Id: I1f6a451fe16758def6f47c046a9b8a67ad7255af
 2018-02-14 23:11:10 +08:00
Eric Payne c5e6e3de1c YARN-7813: Capacity Scheduler Intra-queue Preemption should be configurable for each queue 2018-02-13 10:11:02 -06:00
Jason Lowe 5a1db60ab1 YARN-7914. Fix exit code handling for short lived Docker containers. Contributed by Shane Kumpf 2018-02-12 15:50:10 -06:00
Jason Lowe 3414fd1243 YARN-7917. Fix failing test TestDockerContainerRuntime#testLaunchContainerWithDockerTokens. Contributed by Shane Kumpf 2018-02-12 09:27:43 -06:00
Wangda Tan d02e42cee4 YARN-7739. DefaultAMSProcessor should properly check customized resource types against minimum/maximum allocation. (wangda) 
Change-Id: I10cc9341237d9a2fc0f8c855efb98a36b91389e2
 2018-02-12 10:29:37 +08:00
Wangda Tan d4c98579e3 YARN-7697. NM goes down with OOM due to leak in log-aggregation. (Xuan Gong via wangda) 
Change-Id: Ie4fc7979d834e25f37a033c314f3efceeb8f4a9e
 2018-02-12 10:28:35 +08:00
Wangda Tan 789a185c16 YARN-5848. Remove unnecessary public/crossdomain.xml from YARN UIv2 sub project. (Sunil G via wangda) 
Change-Id: Ie295f88232192e6b520c335b0332383cc6a232c0
 2018-02-12 10:27:15 +08:00
Wangda Tan e795833d8c YARN-7906. Fix mvn site fails with error: Multiple sources of package comments found for package o.a.h.y.client.api.impl. (Akira Ajisaka via wangda) 
Change-Id: I20221d97446e97f208d587eacbc60448c11ffd48
 2018-02-12 10:25:22 +08:00
Weiwei Yang a08c048832 YARN-7838. Support AND/OR constraints in Distributed Shell. Contributed by Weiwei Yang. 2018-02-11 14:20:46 +08:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development 
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
 2018-02-11 11:17:38 +08:00
Eric Yang c97d5bceb2 YARN-7909. Add charset to YARN Service REST API. (Contributed by Eric Yang) 2018-02-09 20:01:25 -05:00
Jian He ddec08d7cc YARN-7827. Stop and Delete Yarn Service from RM UI fails with HTTP ERROR 404. Contributed by Sunil G 2018-02-08 21:32:40 -08:00
Yufei Gu 1bc03ddf97 YARN-7655. Avoid AM preemption caused by RRs for specific nodes or racks. Contributed by Steven Rand. 2018-02-08 12:32:43 -08:00
Jian He eb2449d539 YARN-5428. Allow for specifying the docker client configuration directory. Contributed by Shane Kumpf 2018-02-08 11:35:30 -08:00
Jason Lowe 456705a07c YARN-7815. Make the YARN mounts added to Docker containers more restrictive. Contributed by Shane Kumpf 2018-02-07 13:09:08 -06:00
Billie Rinaldi 266da25c04 Revert "YARN-6078. Containers stuck in Localizing state. Contributed by Billie Rinaldi." 
This reverts commit e14f03dfbf.
 2018-02-07 07:17:00 -08:00
Billie Rinaldi aa461f9091 YARN-7516. Add security check for trusted docker images. Contributed by Eric Yang 2018-02-07 07:09:45 -08:00
Robert Kanter 814d701d46 YARN-7841. Cleanup AllocationFileLoaderService's reloadAllocations method (snemeth via rkanter) 2018-02-06 14:36:49 -08:00
Anu Engineer 4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Weiwei Yang bff858e910 YARN-7860. Fix UT failure TestRMWebServiceAppsNodelabel#testAppsRunning. Contributed by Sunil G. 2018-02-06 19:33:04 +08:00
Sunil G 72415c39cf YARN-7866. [UI2] Error to be displayed correctly while accessing kerberized cluster without kinit. Contributed by Sunil G. 2018-02-06 16:00:34 +05:30
Sunil G 2c326088ae YARN-7850. [UI2] Log Aggregation status to be displayed in Application Page. Contributed by Gergely Novák. 2018-02-06 15:32:09 +05:30
Jian He 87429affce YARN-7889. Missing kerberos token when check for RM REST API availability. Contributed by Eric Yang 2018-02-05 22:56:25 -08:00
Billie Rinaldi a196ee9362 YARN-7890. NPE during container relaunch. Contributed by Jason Lowe 2018-02-05 18:04:09 -08:00
Jason Lowe 60656bceca YARN-7849. TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error. Contributed by Botong Huang 2018-02-05 17:16:04 -06:00
Miklos Szegedi c7101fe21b YARN-7879. NM user is unable to access the application filecache due to permissions. Contributed by Jason Lowe. 2018-02-02 16:48:57 -08:00
Konstantinos Karanasos 50723889cc YARN-7778. Merging of placement constraints defined at different levels. Contributed by Weiwei Yang. 2018-02-02 14:46:20 -08:00
Jian He b6e50fad53 YARN-7831. YARN Service CLI should use hadoop.http.authentication.type to determine authentication method. Contributed by Eric Yang 2018-02-02 14:38:33 -08:00
Jian He f9a4d4cf23 YARN-7868. Provide improved error message when YARN service is disabled. Contributed by Eric Yang 2018-02-02 11:10:47 -08:00
Arun Suresh 6e5ba9366f YARN-7839. Modify PlacementAlgorithm to Check node capacity before placing request on node. (Panagiotis Garefalakis via asuresh) 2018-02-02 10:28:22 -08:00
Jason Lowe 682ea21f2b Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 12eaae383a.
 2018-02-01 12:14:09 -06:00
Eric Yang 0bee3849e3 YARN-7816. Allow same application name submitted by multiple users. (Contributed by Gour Saha) 2018-01-31 20:51:40 -05:00
Jason Lowe 12eaae383a YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-01-31 11:37:54 -06:00
Arun Suresh d481344783 YARN-7822. Constraint satisfaction checker support for composite OR and AND constraints. (Weiwei Yang via asuresh) 2018-01-31 01:51:08 -08:00
Arun Suresh 8d1e2c6409 Merge branch 'YARN-6592' into trunk 2018-01-31 01:42:42 -08:00
Jason Lowe 7288b08330 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-31 09:39:43 -06:00
Wangda Tan 1453a04e92 YARN-7861. [UI2] Logs page shows duplicated containers with ATS. (Sunil G via wangda) 
Change-Id: Id07d1ff8d519e0d9291ab037236641ceb901582c
 2018-01-31 21:44:42 +08:00
Wangda Tan 5ca4bf22dd YARN-7828. Clicking on yarn service should take to component tab. (Sunil G via wangda) 
Change-Id: Ib52d6727c863bed65de30b02bf60080255af11b8
 2018-01-31 21:43:08 +08:00
Sunil G 64344c345d YARN-7802. [UI2] Application regex search did not work properly with app name. Contributed by Sreenath Somarajapuram. 2018-01-31 17:40:36 +05:30
Arun Suresh 8df7666fe1 YARN-7780. Documentation for Placement Constraints. (Konstantinos Karanasos via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh add993e26a YARN-6597. Add RMContainer recovery test to verify tag population in the AllocationTagsManager. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh f8c5f5b237 YARN-7784. Fix Cluster metrics when placement processor is enabled. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh c23980c4f2 YARN-7795. Fix jenkins issues of YARN-6592 branch. (Sunil G via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 644afe5fd8 YARN-7807. Assume intra-app anti-affinity as default for scheduling request inside AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh a4c539fcdb YARN-7783. Add validation step to ensure constraints are not violated due to order in which a request is processed. (asuresh) 2018-01-31 01:30:17 -08:00
Sunil G 9b81cb0537 YARN-7779. Display allocation tags in RM web UI and expose same through REST API. Contributed by Weiwei Yang. 2018-01-31 01:30:17 -08:00
Konstantinos Karanasos adbe87abf8 YARN-7788. Factor out management of temp tags from AllocationTagsManager. (Arun Suresh via kkaranasos) 2018-01-31 01:30:17 -08:00
Arun Suresh 8bf7c44436 YARN-7763. Allow Constraints specified in the SchedulingRequest to override application level constraints. (Weiwei Yang via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 28fe7f3318 YARN-7774. Miscellaneous fixes to the PlacementProcessor. (asuresh) 2018-01-31 01:30:17 -08:00
Wangda Tan e60f51299d YARN-7745. Allow DistributedShell to take a placement specification for containers it wants to launch. (Arun Suresh via wangda) 
Change-Id: Ided146d662e944a8a4692e5d6885f23fd9bbcad5
 2018-01-31 01:30:17 -08:00
Arun Suresh 38af237969 YARN-6599. Support anti-affinity constraint via AppPlacementAllocator. (Wangda Tan via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 8779a35742 YARN-7709. Remove SELF from TargetExpression type. (Konstantinos Karanasos via asuresh) 2018-01-31 01:30:17 -08:00
Wangda Tan 29d9e4d581 YARN-6619. AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects. (Arun Suresh via wangda) 2018-01-31 01:30:17 -08:00
Arun Suresh a5c1fc881e YARN-7696. Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 4eda58c136 YARN-7681. Double-check placement constraints in scheduling phase before actual allocation is made. (Weiwei Yang via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh bdba01f73b YARN-7682. Expose canSatisfyConstraints utility function to validate a placement against a constraint. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh a52d11fb8c YARN-7613. Implement Basic algorithm for constraint based placement. (Panagiotis Garefalakis via asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh f9af15d659 YARN-7612. Add Processor Framework for Rich Placement Constraints. (asuresh) 2018-01-31 01:30:17 -08:00
Arun Suresh 1efb2b6f25 YARN-6596. Introduce Placement Constraint Manager module. (Konstantinos Karanasos via asuresh) 2018-01-31 01:30:17 -08:00