Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,226 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

849 Commits

Author SHA1 Message Date
Arun Suresh 68964bbaa2 YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh) 
(cherry picked from commit 8ffabfdf4f)
(cherry picked from commit b2a654c5ee)
 2016-05-18 22:02:43 -07:00
Karthik Kambatla 2c6c701aff YARN-2882. Add an OPPORTUNISTIC ExecutionType. (Konstantinos Karanasos and Inigo Goiri via kasha) 
(cherry picked from commit fb00794368)
 2016-05-18 22:02:28 -07:00
Varun Vasudev 3c9392a31d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 
(cherry picked from commit ef1757790d)
 2016-05-18 22:45:06 +05:30
Vinod Kumar Vavilapalli e0d6a9632d YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 
(cherry picked from commit 87f5e35133)
 2016-05-09 22:47:58 -07:00
Jason Lowe ee86cef2fe YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit d0da13229c)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
 2016-05-05 14:33:01 +00:00
Vinod Kumar Vavilapalli 8262ef8318 YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 
(cherry picked from commit 9e37fe3b7a)
 2016-05-04 14:17:09 -07:00
Jason Lowe d8fcf02069 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 
(cherry picked from commit c6b4839168)
 2016-05-03 22:14:50 +00:00
Varun Vasudev 6561e3b500 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 
(cherry picked from commit 0f25a1bb52)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ProtoUtils.java
 2016-04-29 16:22:06 +05:30
Jian He 6ba39a1597 YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 
(cherry picked from commit 4a8508501b)
 2016-04-28 21:54:30 -07:00
Jian He a9707dceaf YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:28:03 -07:00
Varun Vasudev cf3e93ee73 YARN-4966. Improve yarn logs to fetch container logs without specifying nodeId. Contributed by Xuan Gong. 
(cherry picked from commit 66b07d8374)
 2016-04-27 15:57:17 +05:30
Chris Douglas 878924189d YARN-4976. Missing NullPointer check in ContainerLaunchContextPBImpl. Contributed by Giovanni Matteo Fumarola 
(cherry picked from commit 95a5046607)
 2016-04-21 10:46:35 -07:00
Vinod Kumar Vavilapalli 342bb438f2 Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 
(cherry picked from commit 477003730e)
 2016-04-18 11:49:00 -07:00
Xuan 94e4d349b0 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 
(cherry picked from commit e6c0742012)
 2016-04-16 19:40:11 -07:00
Naganarasimha 2213a40f30 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 
(cherry picked from commit fdbafbc9e5)
 2016-04-15 23:40:07 +05:30
Rohith Sharma K S d93cfefea5 YARN-4897. dataTables_wrapper change min height. Contributed by Bibin A Chundatt. 
(cherry picked from commit 4ee7f0a3e7)
 2016-04-12 10:28:18 +05:30
Jason Lowe f1a370ce8b Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla" 
This reverts commit 814ceeb489.
 2016-04-11 15:56:29 +00:00
Akira Ajisaka 8cf6630fc6 YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 
(cherry picked from commit 1ff27f9d12)
 2016-04-11 14:55:37 +09:00
Varun Vasudev 4baee3eb67 YARN-4562. YARN WebApp ignores the configuration passed to it for keystore settings. Contributed by Sergey Shelukhin. 
(cherry picked from commit 07920acc16)
 2016-04-11 09:40:28 +05:30
Jason Lowe 814ceeb489 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit 1cbcd4a491)
 2016-04-05 13:41:18 +00:00
naganarasimha 3772602848 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195)
 2016-04-04 18:08:18 +05:30
Karthik Kambatla ecfd25dc65 YARN-4657. Javadoc comment is broken for Resources.multiplyByAndAddTo(). (Daniel Templeton via kasha) 
(cherry picked from commit 81d04cae41)
 2016-04-01 16:20:18 -07:00
Arun Suresh f56266fccc Missing file for YARN-4895. 
(cherry picked from commit 5686caa9fc)
 2016-04-01 15:58:38 -07:00
Varun Vasudev 10d8f8a39c Revert "YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki." 
This reverts commit 0064cba169.

(cherry picked from commit 3488c4f2c9)
 2016-04-01 12:21:49 +05:30
Sangjin Lee b91928cd26 YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 
(cherry picked from commit 6d67420dbc)
 2016-03-31 10:49:45 -07:00
Varun Vasudev dcad99892d YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki. 
(cherry picked from commit 0064cba169)
 2016-03-31 14:06:45 +05:30
Jason Lowe 3cea00a294 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong 
(cherry picked from commit e8fc81f9c8)
 2016-03-25 20:19:15 +00:00
Junping Du 76602161c0 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong. 
(cherry picked from commit af1d125f9c)
 2016-03-23 08:58:05 -07:00
Karthik Kambatla 95b59c17a7 YARN-4732. *ProcessTree classes have too many whitespace issues 
(cherry picked from commit 7fae4c68e6)
 2016-03-20 10:47:56 -07:00
Xuan 398dbde538 Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)" 
This reverts commit 3a0b695441.

(cherry picked from commit a888b5bc53)
 2016-03-15 22:11:16 -07:00
Li Lu 94649b9cc0 YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9) 
(cherry-picked from commit 5176a69868)
 2016-03-15 11:48:37 -07:00
Li Lu 14a4110b75 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 3a0b695441)
 2016-03-15 11:42:30 -07:00
Siddharth Seth edd9abf302 YARN-4816. Fix incompatible change in SystemClock. 
(cherry picked from commit eba66a64d2)
 2016-03-14 20:15:41 -07:00
Junping Du 3d5ac829da YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 
(cherry picked from commit f291d82cd4)
 2016-03-14 08:30:07 -07:00
Li Lu 76ef097fd1 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 
(cherry-picked from commit d49cfb3504)
 2016-03-10 10:56:51 -08:00
Varun Vasudev e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Robert Kanter 872b8d90a6 YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 
(cherry picked from commit d7fdec1e6b)
 2016-02-25 16:40:05 -08:00
Robert Kanter ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 
(cherry picked from commit c4d4df8de0)
 2016-02-25 12:48:29 -08:00
Robert Kanter c2098d2470 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 
(cherry picked from commit 954dd57043)
 2016-02-24 15:00:48 -08:00
Li Lu 51162f9629 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)
 2016-02-24 13:52:11 -08:00
Sangjin Lee 432a2367ce YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba0)
 2016-02-24 09:30:37 -08:00
Varun Vasudev 2c218ca8a8 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d745)
 2016-02-23 12:32:16 +05:30
Zhe Zhang 56b7f2c2c1 HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
 2016-02-22 14:14:39 -08:00
Arun Suresh 6a238e4a6b YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 
(cherry picked from commit 23f937e3b7)
 2016-02-11 10:49:11 -08:00
Varun Vasudev 3a5b39e931 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 
(cherry picked from commit fa00d3e205)
 2016-02-11 12:09:29 +05:30
Varun Vasudev 99966b1912 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
 2016-02-04 13:33:38 +05:30
Devaraj K 43131af1e2 YARN-4100. Add Documentation for Distributed and Delegated-Centralized 
Node Labels feature. Contributed by Naganarasimha G R.

(cherry picked from commit db144eb1c5)
 2016-02-02 12:07:49 +05:30
Wangda Tan 9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:19:19 +08:00
Xuan 2f00d8d3a7 YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu

(cherry picked from commit 9fab22b366)
 2016-01-28 14:36:58 -08:00
Xuan 1d19557dda YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He

(cherry picked from commit 618bfd6ac2)
 2016-01-22 18:21:50 -08:00
Wangda Tan 23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)
 2016-01-19 21:28:51 +08:00
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Junping Du 4a30a44b11 YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 
(cherry picked from commit 02f597c5db)
 2016-01-17 17:38:42 -08:00
rohithsharmaks d3c08cde68 YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 2016-01-15 22:46:21 +05:30
Wangda Tan c0b97865e8 YARN-4571. Make app id/name available to the yarn authorizer provider for better auditing. (Jian He via wangda) 
(cherry picked from commit c0537bcd2c)
 2016-01-13 13:19:01 +08:00
Jason Lowe d164cdcda1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:07 +00:00
Akira Ajisaka df95343047 HADOOP-12584. Disable browsing the static directory in HttpServer2. Contributed by Robert Kanter. 
(cherry picked from commit 9c89bcd04212543ae279d34938ec2ad319e5ba6d)
 2016-01-12 13:34:44 +09:00
Gera Shegalov 8baf9369bb YARN-2934. Improve handling of container's stderr. (Naganarasimha G R via gera) 
(cherry picked from commit 2c17b81569)
 2015-12-24 23:52:43 -08:00
Junping Du dbc4c79b44 YARN-4400. AsyncDispatcher.waitForDrained should be final. Contributed by Daniel Templeton. 
(cherry picked from commit bb5df272b9)
 2015-12-23 08:55:55 -08:00
Junping Du 6e97a3c968 YARN-4234. New put APIs in TimelineClient for ats v1.5. Contributed by Xuan Gong. 
(cherry picked from commit 882f2f0464)
 2015-12-23 05:27:47 -08:00
cnauroth fdfcffb3d1 YARN-3458. CPU resource monitoring in Windows. Contributed by Inigo Goiri. 
(cherry picked from commit 114b590955)
 2015-12-21 16:59:17 -08:00
Wangda Tan f16daa59a4 YARN-4454. NM to nodelabel mapping going wrong after RM restart. (Bibin A Chundatt via wangda) 
(cherry picked from commit bc038b382c)
 2015-12-21 11:30:37 -08:00
Jian He 1147b086d9 YARN-4164. Changed updateApplicationPriority API to return the updated application priority. Contributed by Rohith Sharma K S 2015-12-18 14:14:31 -08:00
Wangda Tan 2f5435acbd YARN-4225. Add preemption status to yarn queue -status for capacity scheduler. (Eric Payne via wangda) 
(cherry picked from commit 7faa406f27)
 2015-12-16 13:29:02 -08:00
Wangda Tan 9f4a6ec663 YARN-4293. ResourceUtilization should be a part of yarn node CLI. (Sunil G via wangda) 
(cherry picked from commit 79c41b1d83)
 2015-12-16 13:28:56 -08:00
Jian He 34e86d8d47 YARN-4403. (AM/NM/Container)LivelinessMonitor should use monotonic time when calculating period. Contributed by Junping Du 2015-12-14 13:52:04 -08:00
Wangda Tan 9f979107c2 YARN-4309. Add container launch related debug information to container logs when a container fails. (Varun Vasudev via wangda) 
(cherry picked from commit dfcbbddb09)
 2015-12-14 11:13:51 -08:00
Xuan 455c6f39c5 YARN-3623-Addendum: Improve the description for Timeline Service Version configuration in yarn-default.xml 
(cherry picked from commit 21daa6c68a)
 2015-12-10 10:57:35 -08:00
Junping Du 0b5e76e9ff YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2015-12-10 02:57:46 -08:00
rohithsharmaks 04e056f4f5 YARN-3456. Improve handling of incomplete TimelineEntities. (Varun Saxena via rohithsharmaks) 
(cherry picked from commit 01a641bc44)
 2015-12-07 15:19:59 +05:30
Jian He 8a97ca4b00 Add missing file for YARN-4419 2015-12-03 19:11:07 -08:00
Jian He be404e1680 YARN-4405. Support node label store in non-appendable file system. Contributed by Wangda Tan 
(cherry picked from commit 755dda8dd8)
 2015-12-03 17:46:09 -08:00
Jian He 23fe1abc02 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999). Contributed by Mohammad Shahid Khan and Varun Saxena 
(cherry picked from commit 9f77ccad73)
 2015-12-03 12:49:22 -08:00
Jian He 48cbde9083 YARN-4398. Remove unnecessary synchronization in RMStateStore. Contributed by Ning Ding 2015-12-02 11:08:00 -08:00
Vinod Kumar Vavilapalli b20e77eba3 Preparing for 2.9.0 development: mvn versions:set -DnewVersion=2.9.0 2015-11-25 17:47:22 -08:00
Jason Lowe 3a2f5d6329 YARN-4365. FileSystemNodeLabelStore should check for root dir existence on startup. Contributed by Kuhu Shukla 
(cherry picked from commit f5acf94eca)
 2015-11-24 23:47:54 +00:00
Jason Lowe 9e54433c6c YARN-4132. Separate configs for nodemanager to resourcemanager connection timeout and retries. Contributed by Chang Li 
(cherry picked from commit 4ac6799d4a)
 2015-11-24 22:36:19 +00:00
Akira Ajisaka ac0ddc4fec YARN-4298. Fix findbugs warnings in hadoop-yarn-common. Contributed by Sunil G. 
(cherry picked from commit 8c0133f3d4)
 2015-11-24 18:40:56 +09:00
Jian He 2cf084ac76 Revert "YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan" 
This reverts commit a4ff03b200.

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-11-16 20:19:31 -08:00
Akira Ajisaka f6f128c867 YARN-4241. Fix typo of property name in yarn-default.xml. Contributed by Anthony Rojas. 
(cherry picked from commit 23d0db551c)
 2015-11-11 16:13:50 +09:00
Jian He a4ff03b200 YARN-3840. Resource Manager web ui issue when sorting application by id (with application having id > 9999) Contributed by Mohammad Shahid Khan 
(cherry picked from commit 8fbea531d7)
 2015-11-09 11:05:12 -08:00
Jian He 7d109ef5b5 YARN-4288. Fixed RMProxy to retry on IOException from local host. Contributed by Junping Du 
(cherry picked from commit c41699965e)
 2015-10-29 00:01:20 -07:00
Wangda Tan 9bf1e74171 YARN-4169. Fix racing condition of TestNodeStatusUpdaterForLabels. (Naganarasimha G R via wangda) 
(cherry picked from commit 6f606214e7)
 2015-10-26 16:38:39 -07:00
Wangda Tan baa2998e93 YARN-4285. Display resource usage as percentage of queue and cluster in the RM UI (Varun Vasudev via wangda) 
(cherry picked from commit 3cc73773eb)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-10-26 13:12:56 -07:00
Rohith Sharma K S 33455fa415 YARN-2729. Support script based NodeLabelsProvider Interface in Distributed Node Label Configuration Setup. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 5acdde4744)
 2015-10-26 15:45:22 +05:30
Jonathan Eagles 6db7bfbcfd YARN-4009. CORS support for ResourceManager REST API. ( Varun Vasudev via jeagles) 
(cherry picked from commit f8adeb712d)
 2015-10-23 10:43:59 -05:00
Junping Du d5145b58f6 YARN-4243. Add retry on establishing Zookeeper conenction in EmbeddedElectorService#serviceInit. Contributed by Xuan Gong. 
(cherry picked from commit 0fce5f9a49)
 2015-10-22 13:42:12 -07:00
Jonathan Eagles cc8093e5ac YARN-2513. Host framework UIs in YARN for use with the ATS (jeagles) 
(cherry picked from commit 25f8f801d1)
 2015-10-21 15:38:48 -05:00
Varun Vasudev 196d549e2b YARN-4262. Allow whitelisted users to run privileged docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit e39ae0e676)
 2015-10-19 20:21:48 +05:30
Jian He 1ba73169c7 YARN-4170. AM need to be notified with priority in AllocateResponse. Contributed by Sunil G 
(cherry picked from commit f9da5cdb2b)
 2015-10-16 15:26:46 -07:00
Varun Vasudev 342870b70f YARN-4258. Add support for controlling capabilities for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 63020c54c1)
 2015-10-15 13:41:58 +05:30
Steve Loughran 0e4fb329ed HADOOP-12478. Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath(). (stevel) 2015-10-14 20:25:33 +01:00
Steve Loughran 5eab51a24f HADOOP-10775. Shell operations to fail with meaningful errors on windows if winutils.exe not found. (stevel) 2015-10-13 21:47:54 +01:00
Devaraj K 28edc7b129 YARN-3964. Support NodeLabelsProvider at Resource Manager side. 
Contributed by Dian Fu.

(cherry picked from commit db93047881)
 2015-10-11 11:24:05 +05:30
Jason Lowe 0139517811 YARN-261. Ability to fail AM attempts. Contributed by Andrey Klochkov and Rohith Sharma K S 
(cherry picked from commit a0bca2b5ad)
 2015-10-09 14:19:41 +00:00
Jason Lowe ec74b0cd4e YARN-3943. Use separate threshold configurations for disk-full detection and disk-not-full detection. Contributed by Zhihai Xu 
(cherry picked from commit 8d226225d0)
 2015-10-08 22:26:35 +00:00
Wangda Tan d24a3b9a3c YARN-4176. Resync NM nodelabels with RM periodically for distributed nodelabels. (Bibin A Chundatt via wangda) 
(cherry picked from commit 30ac69c6bd)
 2015-10-05 15:48:40 -07:00
Harsh J 21b4ba48ce HADOOP-12458. Retries is typoed to spell Retires in parts of hadoop-yarn and hadoop-common. Contributed by Neelesh Srinivas Salian. 
(cherry picked from commit c918f7be5e)
 2015-10-03 18:39:22 +05:30
Xuan 53bddc410f YARN-1897. CLI and core support for signal container functionality. Contributed by Ming Ma 
(cherry picked from commit 8f08532bde)
 2015-10-02 18:53:04 -07:00
Jason Lowe 3e3733437f YARN-3619. ContainerMetrics unregisters during getMetrics and leads to ConcurrentModificationException. Contributed by Zhihai Xu 
(cherry picked from commit fdf02d1f26)
 2015-10-02 20:21:32 +00:00
Jian He d458805690 YARN-1651. CapacityScheduler side changes to support container resize. Contributed by Wangda Tan 
(cherry picked from commit 89cab1ba5f)

Conflicts:
	hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
 2015-09-24 10:30:04 -07:00
Jian He c19e8ffdc7 YARN-1449. AM-NM protocol changes to support container resizing. Contributed by Meng Ding & Wangda Tan) 
(cherry picked from commit 83a18add10)
 2015-09-24 10:23:25 -07:00
Jian He 89dc37450c YARN-3866. AM-RM protocol changes to support container resizing. Contributed by Meng Ding 
(cherry picked from commit dfe2cb849f)
 2015-09-24 10:23:13 -07:00
Tsuyoshi Ozawa 52cc2ca488 HADOOP-12428. Fix inconsistency between log-level guards and statements. Contributed by Jagadesh Kiran N and Jackie Chang. 
(cherry picked from commit dfd807afab)
 2015-09-22 12:54:44 +09:00
Wangda Tan fafcb4ac4e YARN-4113. RM should respect retry-interval when uses RetryPolicies.RETRY_FOREVER. (Sunil G via wangda) 
(cherry picked from commit b00392dd9c)
 2015-09-21 11:15:02 -07:00
Steve Loughran ab09573fcb YARN-3433. Jersey tests failing with Port in Use -again. (Brahma Reddy Battula) 2015-09-16 10:30:49 +01:00
Wangda Tan c31fe79979 YARN-3717. Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API. (Naganarasimha G R via wangda) 
(cherry picked from commit ae5308fe1d)
 2015-09-15 11:41:57 -07:00
Junping Du df714e25aa YARN-313. Add Admin API for supporting node resource configuration in command line. (Contributed by Inigo Goiri, Kenji Kikushima and Junping Du) 
(cherry picked from commit 73e3a49eb0)
 2015-09-15 07:58:47 -07:00
Karthik Kambatla 1f9f219062 YARN-3697. FairScheduler: ContinuousSchedulingThread can fail to shutdown. (Zhihai Xu via kasha) 
(cherry picked from commit 332b520a48)
 2015-09-13 18:08:03 -07:00
Karthik Kambatla 11e2fa151c YARN-2005. Blacklisting support for scheduling AMs. (Anubhav Dhoot via kasha) 
(cherry picked from commit 81df7b586a)
 2015-09-13 17:06:04 -07:00
Steve Loughran 988749e6ea HADOOP-12087. [JDK8] Fix javadoc errors caused by incorrect or illegal tags. (Akira AJISAKA via stevel). 2015-09-13 14:25:06 +01:00
Robert Kanter ea64a9b277 YARN-4086. Allow Aggregated Log readers to handle HAR files (rkanter) 
(cherry picked from commit 6dd6ca442a)
 2015-09-09 18:03:47 -07:00
Jian He f23be93dd1 YARN-2884. Added a proxy service in NM to proxy the the communication between AM and RM. Contributed by Kishore Chaliparambil 
(cherry picked from commit 6f72f1e600)
 2015-09-08 11:24:07 +08:00
Xuan a0b7ef15d0 YARN-4087. Followup fixes after YARN-2019 regarding RM behavior when 
state-store error occurs. Contributed by Jian He

(cherry picked from commit 9b78e6e33d)
 2015-09-07 17:46:48 -07:00
Wangda Tan 9f97b86816 YARN-4024. YARN RM should avoid unnecessary resolving IP when NMs doing heartbeat. (Hong Zhiguo via wangda) 
(cherry picked from commit bcc85e3bab)
 2015-09-04 15:14:34 -07:00
Jian He 48f5161cd5 YARN-4092. Fixed UI redirection to print useful messages when both RMs are in standby mode. Contributed by Xuan Gong 
(cherry picked from commit a3fd2ccc86)
 2015-08-31 17:34:49 -07:00
Jian He 306c535395 YARN-3250. Support admin cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit a9c8ea71aa)
 2015-08-27 13:27:34 -07:00
Jian He 387076894f YARN-4014. Support user cli interface in for Application Priority. Contributed by Rohith Sharma K S 
(cherry picked from commit 57c7ae1aff)
 2015-08-24 20:37:18 -07:00
Xuan 85363ea4bd YARN-221. Addendum patch to compilation issue which is caused by missing 
AllContainerLogAggregationPolicy. Contributed by Xuan Gong

(cherry picked from commit b71c6006f5)
 2015-08-23 16:50:02 -07:00
Xuan 205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate 
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a)
 2015-08-22 16:26:39 -07:00
Wangda Tan 6452b31760 YARN-2923. Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup. (Naganarasimha G R) 
(cherry picked from commit fc07464d1a)
 2015-08-20 16:52:57 -07:00
Karthik Kambatla d2ff763533 YARN-3534. Collect memory/cpu usage on the node. (Inigo Goiri via kasha) 
(cherry picked from commit def12933b3)
 2015-08-16 06:26:21 -07:00
rohithsharmaks c5b20e0db6 YARN-4023. Publish Application Priority to TimelineServer. (Sunil G via rohithsharmaks) 
(cherry picked from commit 1c12adb71f)
 2015-08-12 14:47:23 +05:30
Xuan 2ebdf5bfce YARN-3999. RM hangs on draing events. Contributed by Jian He 
(cherry picked from commit 3ae716fa69)
 2015-08-11 18:26:00 -07:00
Rohith Sharma K S a0da1ec010 YARN-3948. Display Application Priority in RM Web UI.(Sunil G via rohithsharmaks) 
(cherry picked from commit b6265d39c5)
 2015-08-07 10:47:08 +05:30
Arun Suresh 84ea7cd408 YARN-3961. Expose pending, running and reserved containers of a queue in REST api and yarn top (adhoot via asuresh) 
(cherry picked from commit 154c9d2e42)
 2015-08-05 23:15:33 -07:00
Xuan 8126fde884 YARN-3543. ApplicationReport should be able to tell whether the 
Application is AM managed or not. Contributed by Rohith Sharma K S

(cherry picked from commit 0306d902f5)
 2015-08-03 15:47:21 -07:00
Wangda Tan a28e23d049 YARN-3963. AddNodeLabel on duplicate label addition shows success. (Bibin A Chundatt via wangda) 
(cherry picked from commit 8acb30b016)
 2015-07-30 09:43:45 -07:00
rohithsharmaks 9c00ae0009 YARN-3919. NPEs' while stopping service after exception during CommonNodeLabelsManager#start. (varun saxena via rohithsharmaks) 2015-07-29 23:44:15 +05:30
Karthik Kambatla 8659315062 YARN-2768. Avoid cloning Resource in FSAppAttempt#updateDemand. (Hong Zhiguo via kasha) 
(cherry picked from commit 5205a330b3)
 2015-07-29 09:42:57 -07:00
Akira Ajisaka 0e7c88347e YARN-3958. TestYarnConfigurationFields should be moved to hadoop-yarn-api module. Contributed by Varun Saxena. 
(cherry picked from commit 42d4e0ae99)
 2015-07-27 11:45:41 +09:00
Rohith Sharma K S 9d15860e6d YARN-3845. Scheduler page does not render RGBA color combinations in IE11. (Contributed by Mohammad Shahid Khan) 2015-07-24 12:54:07 +05:30
Robert Kanter ba3b40ae79 YARN-3900. Protobuf layout of yarn_security_token causes errors in other protos that include it (adhoot via rkanter) 
(cherry picked from commit 1d3026e7b3)
 2015-07-23 14:47:28 -07:00
Junping Du 6772c3f4dd YARN-2019. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore. Contributed by Jian He. 
(cherry picked from commit ee98d6354b)
 2015-07-22 17:57:16 -07:00
rohithsharmaks be2334ba3a YARN-3954. Fix TestYarnConfigurationFields#testCompareConfigurationClassAgainstXml. (varun saxena via rohithsharmaks) 2015-07-23 02:10:45 +05:30
Jian He 11cb00f7a0 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. Contributed by Varun Saxena 
(cherry picked from commit 393fe71771)
 2015-07-21 15:06:16 -07:00
Wangda Tan 7b2540770e YARN-3930. FileSystemNodeLabelsStore should make sure edit log file closed when exception is thrown. (Dian Fu via wangda) 
(cherry picked from commit fa2b63ed16)
 2015-07-16 16:08:04 -07:00
Chris Douglas c6b0bbabc7 HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl 
(cherry picked from commit a431ed9075)
 2015-07-13 15:36:25 -07:00
Jian He 27366dd7c3 Revert "YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha)" 
This reverts commit aa067c6aa4.
(cherry picked from commit 2466460d4c)
 2015-07-13 14:31:55 -07:00
Akira Ajisaka 0ba2defb9e YARN-3381. Fix typo InvalidStateTransitonException. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 19295b36d9)
 2015-07-13 17:53:27 +09:00
Akira Ajisaka ad6787aa43 YARN-3069. Document missing properties in yarn-default.xml. Contributed by Ray Chiang. 
(cherry picked from commit d6675606dc)
 2015-07-13 12:57:29 +09:00
Gera Shegalov 896b8c6551 YARN-3917. getResourceCalculatorPlugin for the default should intercept all exceptions. (gera) 
(cherry picked from commit d7319dee37)
 2015-07-11 22:41:04 -07:00
Zhijie Shen 37a93c2d78 YARN-3116. RM notifies NM whether a container is an AM container or normal task container. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 1ea36299a4)
 2015-07-10 19:02:13 -07:00
Chris Douglas 66419d3672 HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl 
(cherry picked from commit 1a0752d85a)
 2015-07-09 17:48:56 -07:00
Karthik Kambatla fc989ebe16 HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha) 
(cherry picked from commit ac6048372a)
 2015-07-09 09:59:02 -07:00
Karthik Kambatla ccf18705f7 YARN-3878. AsyncDispatcher can hang while stopping if it is configured for draining events on stop. (Varun Saxena via kasha) 
(cherry picked from commit aa067c6aa4)
 2015-07-09 09:48:46 -07:00
Akira Ajisaka d260478d3a YARN-3690. [JDK8] 'mvn site' fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d6325745e2)
 2015-07-08 15:43:04 +09:00
Xuan 09639ac6b4 YARN-3882. AggregatedLogFormat should close aclScanner and ownerScanner 
after create them. Contributed by zhihai xu

(cherry picked from commit 688617d6d7)
 2015-07-04 21:54:22 -07:00
Devaraj K 3432017de0 YARN-3823. Fix mismatch in default values for 
yarn.scheduler.maximum-allocation-vcores property. Contributed by Ray
Chiang.

(cherry picked from commit 7405c59799)
 2015-07-01 11:12:43 +05:30
Gera Shegalov f9b98082b8 YARN-3768. ArrayIndexOutOfBoundsException with empty environment variables. (Zhihai Xu via gera) 
(cherry picked from commit 6f2a41e37d)
 2015-06-30 15:05:01 -07:00
Jian He 484a6c0f01 YARN-3770. SerializedException should also handle java.lang.Error on de-serialization. Contributed by Lavkesh Lahngir 
(cherry picked from commit 4672315e2d)
 2015-06-29 14:32:22 -07:00
Jian He 0100995c5e YARN-3695. ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception. Contributed by Raju Bairishetti 
(cherry picked from commit 62e583c7dc)
 2015-06-29 13:38:04 -07:00
Devaraj K bb8a2cff2a YARN-3745. SerializedException should also try to instantiate internal 
exception with the default constructor. Contributed by Lavkesh Lahngir.

(cherry picked from commit b381f88c71)
 2015-06-25 17:39:40 +05:30
Jason Lowe 14afa5d53d YARN-3809. Failed to launch new attempts because ApplicationMasterLauncher's threads all hang. Contributed by Jun Gong 
(cherry picked from commit 2a20dd9b61)
 2015-06-24 16:24:43 +00:00
Karthik Kambatla 9656ee4ee7 YARN-3842. NMProxy should retry on NMNotYetReadyException. (Robert Kanter via kasha) 
(cherry picked from commit 5ebf2817e5)
 2015-06-22 17:48:07 -07:00
Gera Shegalov 34e744ce1a MAPREDUCE-6316. Task Attempt List entries should link to the task overview. (Siqi Li via gera) 
(cherry picked from commit 63d40d5286)
 2015-06-20 11:52:11 -07:00
Devaraj K 2802c63ac3 YARN-3617. Fix WindowsResourceCalculatorPlugin.getCpuFrequency() returning 
always -1. Contributed by J.Andreina.

(cherry picked from commit 318d2cde7c)
 2015-06-17 13:55:25 +05:30
Xuan fe7f2d40fe YARN-3714. AM proxy filter can not get RM webapp address from 
yarn.resourcemanager.hostname.rm-id. Contributed by Masatake Iwasaki

(cherry picked from commit e27d5a13b0)
 2015-06-16 14:06:56 -07:00
Zhijie Shen 0dad10d845 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong. 
(cherry picked from commit 8d0ef31632)
 2015-06-08 23:21:35 -07:00
Wangda Tan 344b750915 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c)
 2015-06-04 13:20:49 -07:00
Wangda Tan 78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48f)
 2015-06-04 10:23:33 -07:00
Xuan 3f0573b059 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428)
 2015-06-03 17:21:48 -07:00
Zhijie Shen 2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d20.
 2015-06-03 14:16:51 -07:00
Wangda Tan a3734f67d3 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced95)
 2015-05-31 16:33:50 -07:00
Zhijie Shen 4e2ed34d20 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 
(cherry picked from commit a9f986bc5a41ab74171da1076d10e83c5adc284f)
 2015-05-30 21:06:32 -07:00
Wangda Tan 2fb794ecaf YARN-3716. Node-label-expression should be included by ResourceRequestPBImpl.toString. (Xianyin Xin via wangda) 
(cherry picked from commit 788bfa0359)
 2015-05-28 16:35:17 -07:00
Devaraj K 7116141d7d YARN-3722. Merge multiple TestWebAppUtils into 
o.a.h.yarn.webapp.util.TestWebAppUtils. Contributed by Masatake Iwasaki.

(cherry picked from commit 7e509f5843)
 2015-05-28 15:36:13 +05:30
Zhijie Shen 839f81a632 YARN-3700. Made generic history service load a number of latest applications according to the parameter or the configuration. Contributed by Xuan Gong. 
(cherry picked from commit 54504133f4)
 2015-05-27 16:53:47 -07:00
Wangda Tan 8857cba72e YARN-3647. RMWebServices api's should use updated api from CommonNodeLabelsManager to get NodeLabel object. (Sunil G via wangda) 
(cherry picked from commit ec0a852a37)
 2015-05-27 13:15:55 -07:00
Wangda Tan 0728a86b3a YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda) 
(cherry picked from commit cdbd66be11)
 2015-05-26 16:00:17 -07:00
Vinod Kumar Vavilapalli a67cb4826b YARN-160. Enhanced NodeManager to automatically obtain cpu/memory values from underlying OS when configured to do so. Contributed by Varun Vasudev. 
(cherry picked from commit 500a1d9c76)
 2015-05-26 11:39:40 -07:00
Xuan 84245ff3b2 YARN-2238. Filtering on UI sticks even if I move away from the page. 
Contributed by Jian He

(cherry picked from commit 39077dba2e)
 2015-05-25 22:41:02 -07:00
Vinayakumar B 19895a41ad HADOOP-11743. maven doesn't clean all the site files (Contributed by ramtin) 
(cherry picked from commit 55ed6558ba)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
 2015-05-22 16:46:50 +05:30
Devaraj K 157ecb2241 YARN-3646. Applications are getting stuck some times in case of retry 
policy forever. Contributed by Raju Bairishetti.

(cherry picked from commit 0305316d69)
 2015-05-21 20:16:09 +05:30
Jian He 521378ffa3 YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 
(cherry picked from commit 8966d42179)
 2015-05-20 16:30:44 -07:00
Wangda Tan b5ba6979b7 YARN-3583. Support of NodeLabel object instead of plain String in YarnClient side. (Sunil G via wangda) 
(cherry picked from commit 563eb1ad2a)
(cherry picked from commit b0d22b0c606fad6b4ab5443c0aed07c829b46726)
 2015-05-19 17:37:24 -07:00
Wangda Tan 7b97f049ee YARN-3565. NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String. (Naganarasimha G R via wangda) 
(cherry picked from commit b37da52a1c)
 2015-05-19 16:35:23 -07:00
Xuan 3ceb2ffe54 YARN-3541. Add version info on timeline service / generic history web UI and REST API. Contributed by Zhijie Shen 
(cherry picked from commit 76afd28862)
 2015-05-18 13:19:01 -07:00
Junping Du bc13c7d84b YARN-3505. Node's Log Aggregation Report with SUCCEED should not cached in RMApps. Contributed by Xuan Gong. 
(cherry picked from commit 15ccd967ee)
 2015-05-14 10:59:48 -07:00
Wangda Tan 8555a5146d YARN-3579. CommonNodeLabelsManager should support NodeLabel instead of string label name when getting node-to-label/label-to-label mappings. (Sunil G via wangda) 
(cherry picked from commit d4f53fc963)
 2015-05-13 13:33:38 -07:00
Wangda Tan 3df8880bbe YARN-2921. Fix MockRM/MockAM#waitForState sleep too long. (Tsuyoshi Ozawa via wangda) 
(cherry picked from commit 341a476812)
 2015-05-13 13:07:05 -07:00
Zhijie Shen 6043224458 YARN-3539. Updated timeline server documentation and marked REST APIs evolving. Contributed by Steve Loughran. 
(cherry picked from commit fcd0702c10)
 2015-05-12 21:13:47 -07:00
Karthik Kambatla e8dd0bc4fd YARN-1287. Consolidate MockClocks. (Sebastian Wong and Anubhav Dhoot via kasha) 
(cherry picked from commit 70fb37cd79)
 2015-05-09 14:35:13 -07:00
Xuan 8d07116266 YARN-2331. Distinguish shutdown during supervision vs. shutdown for 
rolling upgrade. Contributed by Jason Lowe

(cherry picked from commit 088156de43)
 2015-05-08 15:11:55 -07:00
Thomas Graves 638feaaa35 YARN-20. More information for yarn.resourcemanager.webapp.address in yarn-default.xml (Bartosz Ługowski vai tgraves) 
(cherry picked from commit f0f5e3c075)
 2015-05-08 17:22:33 +00:00
Devaraj K bf8be56292 YARN-2784. Make POM project names consistent. Contributed by Rohith. 2015-05-08 19:26:11 +05:30
Jason Lowe bcb863b156 YARN-3554. Default value for maximum nodemanager connect wait time is too high. Contributed by Naganarasimha G R 
(cherry picked from commit 9757864fd6)
 2015-05-08 13:47:48 +00:00
Tsuyoshi Ozawa b1f3590b22 YARN-3589. RM and AH web UI display DOCTYPE wrongly. Contbituted by Rohith. 
(cherry picked from commit f26700f287)
 2015-05-08 17:13:56 +09:00
Akira Ajisaka 67bebabb21 YARN-3577. Misspelling of threshold in log4j.properties for tests. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 995f1b5831fc33042de90267a689333dc110c4c3)
 2015-05-07 13:33:17 +09:00
Xuan 02cdcaec58 YARN-2123. Progress bars in Web UI always at 100% due to non-US locale. 
Contributed by Akira AJISAKA

(cherry picked from commit b7dd3a4f04)
 2015-05-05 10:45:18 -07:00
Tsuyoshi Ozawa f382602de1 YARN-1993. Cross-site scripting vulnerability in TextView.java. Contributed byKenji Kikushima. 
(cherry picked from commit e8d0ee5fc9)
 2015-05-03 10:51:39 +09:00
Junping Du 8068e79989 YARN-2454. Fix compareTo of variable UNBOUNDED in o.a.h.y.util.resource.Resources. Contributed by Xu Yang. 
(cherry picked from commit 57d9a972cb)
 2015-05-02 10:20:30 -07:00
tgraves 2e13183f60 YARN-3517. RM web ui for dumping scheduler logs should be for admins only (Varun Vasudev via tgraves) 
(cherry picked from commit 2e215484bd)
 2015-04-29 21:27:16 +00:00
Wangda Tan 8ee632caa7 YARN-2740. Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled. (Naganarasimha G R via wangda) 
(cherry picked from commit db1b674b50)
 2015-04-27 16:42:36 -07:00
Vinod Kumar Vavilapalli 0ec6e7e7f5 YARN-3413. Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime. (Wangda Tan via vinodkv) 
(cherry picked from commit f5fe35e297)
 2015-04-23 11:25:30 -07:00
Jian He 48d8d88245 YARN-3522. Fixed DistributedShell to instantiate TimeLineClient as the correct user. Contributed by Zhijie Shen 
(cherry picked from commit aa4a192feb)
 2015-04-23 11:07:54 -07:00
Junping Du 8de6045842 New parameter of CLI for decommissioning node gracefully in RMAdmin CLI. Contributed by Devaraj K 
(cherry picked from commit fad9d7e85b)
 2015-04-22 10:08:16 -07:00
Jian He 9827da95bc YARN-3451. Display attempt start time and elapsed time on the web UI. Contributed by Rohith Sharmaks 
(cherry picked from commit 6779467ab6)
 2015-04-17 15:57:36 -07:00
Junping Du b244701fad YARN-1402. Update related Web UI and CLI with exposing client API to check log aggregation status. Contributed by Xuan Gong. 
(cherry picked from commit 1db355a875)
 2015-04-17 13:20:27 -07:00
Akira Ajisaka 51c458a5c4 MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794)
 2015-04-16 10:06:03 -05:00