Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,399 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1228 Commits

Author SHA1 Message Date
Giovanni Matteo Fumarola 3090922805 YARN-8696. [AMRMProxy] FederationInterceptor upgrade: home sub-cluster heartbeat async. Contributed by Botong Huang. 2018-09-24 11:37:05 -07:00
Shane Kumpf 8e9afbfb66 YARN-8768. Javadoc error in node attributes. Contributed by Sunil Govindan. 2018-09-12 15:12:28 -06:00
bibinchundatt c44088ac19 YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Weiwei Yang 52194351e7 YARN-8721. Relax NE node-attribute check when attribute doesn't exist on a node. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha 67ae81f0e0 YARN-7863. Modify placement constraints to support node attributes. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha 8c947398ab YARN-8574. Allow dot in attribute values. Contributed by Bibin A Chundatt. 2018-09-12 16:01:01 +05:30
Naganarasimha eb08543c7a YARN-8103. Add CLI interface to query node attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:01 +05:30
bibinchundatt 8cf6a9a2bd YARN-7892. Revisit NodeAttribute class structure. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha 5dc7d6e0f3 YARN-8104. Add API to fetch node to attribute mapping. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha 0a01b1350d YARN-8100. Support API interface to query cluster attributes and attribute to nodes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Sunil G b9890d1f66 YARN-7875. Node Attribute store for storing and recovering attributes. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Sunil G 6f4bc49c6d YARN-7988. Refactor FSNodeLabelStore code for Node Attributes store support. Contributed by Bibin A Chundatt. 2018-09-12 16:01:00 +05:30
Naganarasimha 3b3b6efe21 YARN-7871. Node attributes reporting from NM to RM. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Naganarasimha 86d024ef2a YARN-7965. NodeAttributeManager add/get API is not working properly. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Sunil G ffcabd24c3 YARN-7856. Validate Node Attributes from NM. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
Sunil G 2f7712be09 YARN-6858. Attribute Manager to store and provide node attributes in RM. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha d312b5cf9f YARN-7757. Refactor NodeLabelsProvider to be more generic and reusable for node attributes providers. Contributed by Weiwei Yang. 2018-09-12 16:01:00 +05:30
bibinchundatt 4458b2772f YARN-7840. Update PB for prefix support of node attributes. Contributed by Naganarasimha G R. 2018-09-12 16:01:00 +05:30
Naganarasimha 1f42ce907a YARN-6855. [YARN-3409] CLI Proto Modifications to support Node Attributes. Contributed by Naganarasimha G R. 2018-09-12 16:00:59 +05:30
Giovanni Matteo Fumarola 3dc2988a37 YARN-8699. Add Yarnclient#yarnclusterMetrics API implementation in router. Contributed by Bibin A Chundatt. 2018-09-07 11:32:03 -07:00
Shane Kumpf 73625168c0 YARN-8642. Add support for tmpfs mounts with the Docker runtime. Contributed by Craig Condit 2018-08-29 07:08:37 -06:00
Eric Yang e557c6bd8d YARN-8298. Added express upgrade for YARN service. 
Contributed by Chandni Singh
 2018-08-21 19:49:26 -04:00
Giovanni Matteo Fumarola 8736fc39ac YARN-8673. [AMRMProxy] More robust responseId resync after an YarnRM master slave switch. Contributed by Botong Huang. 2018-08-20 12:22:36 -07:00
Eric Yang 74411ce0ce YARN-7417. Remove duplicated code from IndexedFileAggregatedLogsBlock 
and TFileAggregatedLogsBlock.
           Contributed by Zian Chen
 2018-08-13 16:50:00 -04:00
Sunil G 00013d6ef7 YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-09 12:18:32 +05:30
Sunil G 5b898c176f Revert "YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB." 
This reverts commit 64901abdfa.
 2018-08-08 19:48:49 +05:30
Sunil G 64901abdfa YARN-8633. Update DataTables version in yarn-common in line with JQuery 3 upgrade. Contributed by Akhil PB. 2018-08-08 19:43:29 +05:30
Wangda Tan c0599151bb YARN-7089. Mark the log-aggregation-controller APIs as public. (Zian Chen via wangda) 
Change-Id: I37851bdc5935d623a27d0973a206c997258716eb
 2018-08-07 12:37:32 -07:00
Shane Kumpf bcfc9851f4 YARN-8287. Update documentation and yarn-default related to the Docker runtime. Contributed by Craig Condit 2018-08-03 18:20:49 -06:00
Sunil G 12a095a496 YARN-7159. Normalize unit of resource objects in RM to avoid unit conversion in critical path. Contributed by Manikandan R. 2018-08-02 22:29:21 +05:30
Wangda Tan 5cc8e99147 YARN-8522. Application fails with InvalidResourceRequestException. (Zian Chen via wangda) 
Change-Id: I34dd7fa49bd4d10580c4a78051033b1068d28f1e
 2018-07-31 22:34:53 -07:00
Wangda Tan 4b540bbfcf YARN-8418. App local logs could leaked if log aggregation fails to initialize for the app. (Bibin A Chundatt via wangda) 
Change-Id: I29a23ca4b219b48c92e7975cd44cddb8b0e04104
 2018-07-31 12:08:00 -07:00
Jonathan Hung 3e06a5dcea YARN-7974. Allow updating application tracking url after registration. Contributed by Jonathan Hung 2018-07-30 17:44:18 -07:00
bibinchundatt 2b39ad2698 YARN-8584. Several typos in Log Aggregation related classes. Contributed by Szilard Nemeth. 2018-07-30 23:25:19 +05:30
bibinchundatt ff7c2eda34 YARN-8548. AllocationRespose proto setNMToken initBuilder not done. Contributed by Bilwa S T. 2018-07-24 16:17:20 +05:30
Haibo Chen 8a6bb8409c YARN-6964. Fair scheduler misuses Resources operations. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-20 10:46:28 -07:00
Haibo Chen f354f47f99 YARN-6995. Improve use of ResourceNotFoundException in resource types code. (Daniel Templeton and Szilard Nemeth via Haibo Chen) 2018-07-19 15:35:05 -07:00
Eric Yang 121865c3f9 YARN-8299. Added CLI and REST API for query container status. 
Contributed by Chandni Singh
 2018-07-16 17:41:23 -04:00
Wangda Tan 238ffff999 YARN-8524. Single parameter Resource / LightWeightResource constructor looks confusing. (Szilard Nemeth via wangda) 
Change-Id: I4ae97548b5b8d76a6bcebb2d3d70bf8e0be3c125
 2018-07-16 10:58:00 -07:00
Akira Ajisaka 88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Giovanni Matteo Fumarola ea9b608237 YARN-7899. [AMRMProxy] Stateful FederationInterceptor for pending requests. Contributed by Botong Huang. 2018-07-09 12:27:36 -07:00
Wangda Tan 83cd84b70b YARN-8506. Make GetApplicationsRequestPBImpl thread safe. (wangda) 
Change-Id: If304567abb77a01b686d82c769bdf50728484163
 2018-07-09 11:30:08 -07:00
Xiaoyu Yao 498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Eric Yang b69ba0f330 YARN-8108. Added option to disable loading existing filters to prevent 
security filter from initialize twice.
            Contributed by Sunil Govindan
 2018-06-26 17:34:57 -04:00
Bharat Viswanadham 361ffb26be YARN-8426:Upgrade jquery-ui to 1.12.1 in YARN. Contributed by Sunil Govindan 2018-06-14 13:14:25 -07:00
Jitendra Pandey f4c7c91123 HADOOP-15483. Upgrade jquery to version 3.3.1. Contributed by Lokesh Jain, Mukul Kumar Singh and Sunil Govindan. 2018-06-13 00:36:02 -07:00
Haibo Chen 24a89825f0 YARN-6931. Make the aggregation interval in AppLevelTimelineCollector configurable. (Abhishek Modi via Haibo Chen) 2018-06-12 10:03:34 -07:00
Akira Ajisaka 652bcbb3e4
YARN-8363. Upgrade commons-lang version to 3.7 in hadoop-yarn-project. Contributed by Takanobu Asanuma. 2018-06-13 00:35:37 +09:00
Wangda Tan ff583d3fa3 YARN-8349. Remove YARN registry entries when a service is killed by the RM. (Billie Rinaldi via wangda) 
Change-Id: Ia58db3637789a8921482f564aa9bdf99c45cc36c
 2018-06-01 14:49:18 -07:00
Jason Lowe 4827e9a908 YARN-8329. Docker client configuration can still be set incorrectly. Contributed by Shane Kumpf 2018-05-29 14:43:17 -05:00
Wangda Tan 17aa40f669 YARN-8369. Javadoc build failed due to 'bad use of >'. (Takanobu Asanuma via wangda) 
Change-Id: I79a42154e8f86ab1c3cc939b3745024b8eebe5f4
 2018-05-29 09:27:36 -07:00
Rohith Sharma K S 88cbe57c06 MAPREDUCE-7097. MapReduce JHS should honor yarn.webapp.filter-entity-list-by-user. Contributed by Sunil Govindan. 2018-05-28 14:05:49 +05:30
Eric E Payne 8d5509c681 YARN-8292: Fix the dominant resource preemption cannot happen when some of the resource vector becomes negative. Contributed by Wangda Tan. 2018-05-25 16:06:09 +00:00
Shane Kumpf 1388de18ad YARN-6919. Add default volume mount list. Contributed by Eric Badger 2018-05-24 09:30:39 -06:00
Rohith Sharma K S c05b5d424b YARN-8319. More YARN pages need to honor yarn.resourcemanager.display.per-user-apps. Contributed by Sunil G. 2018-05-24 14:19:46 +05:30
Rohith Sharma K S 4cc0c9b0ba YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 2018-05-24 12:23:47 +05:30
Haibo Chen d996479954 YARN-4599. Set OOM control for memory cgroups. (Miklos Szegedi via Haibo Chen) 2018-05-23 16:35:37 -07:00
Inigo Goiri f09dc73001 YARN-8327. Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows. Contributed by Giovanni Matteo Fumarola. 2018-05-23 15:59:30 -07:00
Inigo Goiri e30938af12 YARN-8336. Fix potential connection leak in SchedConfCLI and YarnWebServiceUtils. Contributed by Giovanni Matteo Fumarola. 2018-05-23 11:55:31 -07:00
Miklos Szegedi 3e5f7ea986 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-22 18:10:33 -07:00
Robert Kanter b22f56c471 YARN-8273. Log aggregation does not warn if HDFS quota in target directory is exceeded (grepas via rkanter) 2018-05-22 14:24:38 -07:00
Arun Suresh 3159bffce2 YARN-7900. [AMRMProxy] AMRMClientRelayer for stateful FederationInterceptor. (Botong Huang via asuresh) 2018-05-17 20:00:52 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 328f0847e3 YARN-8128. Documenting the per-node per-app file limit in YARN log aggregation. Contributed by Xuan Gong. 2018-05-17 12:47:10 -07:00
Jason Lowe be53969047 YARN-8071. Add ability to specify nodemanager environment variables individually. Contributed by Jim Brennan 2018-05-16 16:17:28 -05:00
Dinesh Chitlangia 0a22860336
YARN-8123. Skip compiling old hamlet package when the Java version is 10 or upper 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-05-16 17:25:31 +09:00
Robert Kanter c8b53c4364 YARN-8202. DefaultAMSProcessor should properly check units of requested custom resource types against minimum/maximum allocation (snemeth via rkanter) 2018-05-10 09:31:59 -07:00
Inigo Goiri e6a80e476d YARN-8151. Yarn RM Epoch should wrap around. Contributed by Young Chen. 2018-05-02 17:23:17 -07:00
Weiwei Yang 2c95eb8111 YARN-8225. YARN precommit build failing in TestPlacementConstraintTransformations. Contributed by Shane Kumpf. 2018-04-28 17:37:37 +08:00
Billie Rinaldi 914d6b5cb1 YARN-8208. Add log statement for Docker client configuration file at INFO level. Contributed by Yesha Vora 2018-04-27 06:10:32 -07:00
BELUGA BEHR 351e509a47
YARN-8169. Cleanup RackResolver.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-04-19 16:07:14 +09:00
Haibo Chen bb92bfb4ef YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 11:15:20 -07:00
Jason Lowe 1d6e43da51 YARN-7996. Allow user supplied Docker client configurations with YARN native services. Contributed by Shane Kumpf 2018-04-17 08:57:31 -05:00
Haibo Chen 86dfb70875 Revert "YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen)" 
This reverts commit 0c14b6f0d8.
 2018-04-17 06:04:38 -07:00
Haibo Chen 0c14b6f0d8 YARN-7088. Add application launch time to Resource Manager REST API. (Kanwaljeet Sachdev via Haibo Chen) 2018-04-17 05:59:03 -07:00
Rohith Sharma K S 669eb7bdea YARN-8156. Increase the default value of yarn.timeline-service.app-collector.linger-period.ms. Contributed by Charan Hebri. 2018-04-14 10:31:28 +05:30
Jason Lowe 4571351ccc MAPREDUCE-7069. Add ability to specify user environment variables individually. Contributed by Jim Brennan 2018-04-12 11:12:46 -05:00
Jason Lowe 907919d28c YARN-7667. Docker Stop grace period should be configurable. Contributed by Eric Badger 2018-04-09 17:19:21 -05:00
Haibo Chen f8b8bd53c4 YARN-6936. [Atsv2] Retrospect storing entities into sub application table from client perspective. (Rohith Sharma K S via Haibo Chen) 2018-04-05 10:23:42 -07:00
Vrushali C 345e7624d5 YARN-8073 TimelineClientImpl doesn't honor yarn.timeline-service.versions configuration. Contributed by Rohith Sharma K S 2018-04-04 15:08:03 -07:00
Wangda Tan b779f4f0f6 YARN-8106. Update LogAggregationIndexedFileController to use readFull instead read to avoid IOException while loading log meta. (Prabhu Joseph via wangda) 
Change-Id: I63a65f73f8d1636e2c99ed9c8c2bbd05efcff80f
 2018-04-03 21:06:24 -07:00
Wangda Tan a0bde7d525 YARN-7142. Support placement policy in yarn native services. (Gour Saha via wangda) 
Change-Id: I166c67a7a34430627c17365f60bac75b6da1b434
 2018-04-02 07:26:01 -07:00
Weiwei Yang 1606dad133 YARN-7497. Add file system based scheduler configuration store. Contributed by Jiandan Yang. 2018-03-30 21:41:33 +08:00
Subru Krishnan 725b10e3ae Revert "YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru)." 
This reverts commit 2a2ef15caf as smart-apply-patch script didn't pick the latest patch.
 2018-03-28 11:26:50 -07:00
Subru Krishnan 2a2ef15caf YARN-8010. Add config in FederationRMFailoverProxy to not bypass facade cache when failing over. (Botong Huang via Subru). 2018-03-27 17:39:46 -07:00
Sean Mackrory 82665a7887 HADOOP-15299. Bump Jackson 2 version to Jackson 2.9.x. 2018-03-26 14:05:15 -06:00
Wangda Tan 4bf622043f YARN-7952. RM should be able to recover log aggregation status after restart/fail-over. (Xuan Gong via wangda) 
Change-Id: I725c9afe64831eda0aa6b0bebdbc79d2dd165707
 2018-03-15 13:26:45 -07:00
fang zhenyi 5e013d50d1
HADOOP-15305. Replace FileUtils.writeStringToFile(File, String) with (File, String, Charset) to fix deprecation warnings. 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-15 16:12:07 +09:00
Wangda Tan a5b27b3c67 YARN-5015. Support sliding window retry capability for container restart. (Chandni Singh via wangda) 
Change-Id: I07addd3e4ba8d98456ee2ff1d5c540a38fe61dea
 2018-03-13 17:55:17 -07:00
Miklos Szegedi a82d4a2e3a YARN-5764. NUMA awareness support for launching containers. Contributed by Devaraj K. 2018-03-13 12:36:57 -07:00
Wangda Tan 583f459431 YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: I16e081f21c5f80160564c49cc49d103bd8eb7e16
 2018-03-07 15:46:47 -08:00
Wangda Tan e718ac597f Revert "YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda)" 
This reverts commit 4d53ef7eef.
 2018-03-07 15:46:47 -08:00
Wangda Tan 4d53ef7eef YARN-7891. LogAggregationIndexedFileController should support read from HAR file. (Xuan Gong via wangda) 
Change-Id: Ie16e34039d57df50128c73b37516ad0bc7c9590e
 2018-03-07 11:30:06 -08:00
Jason Lowe d69b31f7f7 YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan 2018-03-07 09:20:34 -06:00
Takanobu Asanuma 745190ecdc
HADOOP-15287. JDK9 JavaDoc build fails due to one-character underscore identifiers in hadoop-yarn-common 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-03-05 18:15:53 -08:00
Konstantinos Karanasos e85188101c YARN-7921. Transform a PlacementConstraint to a string expression. Contributed by Weiwei Yang. 2018-02-26 12:15:16 -08:00
Gabor Bota c36b4aa31c
HADOOP-15236. Fix typo in RequestHedgingProxyProvider and RequestHedgingRMFailoverProxyProvider 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-23 13:55:18 +09:00
Jason Lowe 9497215084 YARN-7813. Capacity Scheduler Intra-queue Preemption should be configurable for each queue. Contributed by Eric Payne 2018-02-19 14:06:28 -06:00
Jason Lowe b9a429bb28 Revert "YARN-7677. Docker image cannot set HADOOP_CONF_DIR. Contributed by Jim Brennan" 
This reverts commit 8013475d44.
 2018-02-19 08:16:25 -06:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-02-18 22:19:39 +09:00
Wangda Tan ca1043ab90 YARN-7328. ResourceUtils allows yarn.nodemanager.resource-types.memory-mb and .vcores to override yarn.nodemanager.resource.memory-mb and .cpu-vcores. (lovekesh bansal via wangda) 
Change-Id: Ibb1faf0beefec079dae2208986976b2f64650672
 2018-02-17 12:28:08 +08:00