Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,304 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2619 Commits

Author SHA1 Message Date
Li Lu 236d1a5093 YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9) 
(cherry picked from commit f61de41736)
 2016-04-04 14:41:40 -07:00
naganarasimha 3772602848 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 5092c94195)
 2016-04-04 18:08:18 +05:30
Rohith Sharma K S c8271cd117 YARN-4607. Pagination support for AppAttempt page TotalOutstandingResource Requests table. Contributed by Bibin A Chundatt 
(cherry picked from commit 1e6f92977d)
 2016-04-04 08:13:03 +05:30
Allen Wittenauer 92a3dbe44f YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw) 
(cherry picked from commit b1394d6307)
 2016-04-01 16:57:31 -07:00
Karthik Kambatla ecfd25dc65 YARN-4657. Javadoc comment is broken for Resources.multiplyByAndAddTo(). (Daniel Templeton via kasha) 
(cherry picked from commit 81d04cae41)
 2016-04-01 16:20:18 -07:00
Arun Suresh f56266fccc Missing file for YARN-4895. 
(cherry picked from commit 5686caa9fc)
 2016-04-01 15:58:38 -07:00
Arun Suresh 24ce9ea456 YARN-4895. Add subtractFrom method to ResourceUtilization class. Contributed by Konstantinos Karanasos. 
(cherry picked from commit 82621e38a0)
 2016-04-01 15:00:02 -07:00
Varun Vasudev 10d8f8a39c Revert "YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki." 
This reverts commit 0064cba169.

(cherry picked from commit 3488c4f2c9)
 2016-04-01 12:21:49 +05:30
Robert Kanter 633f612d67 YARN-4639. Remove dead code in TestDelegationTokenRenewer added in YARN-3055 (templedf via rkanter) 
(cherry picked from commit 7a021471c3)
 2016-03-31 15:47:44 -07:00
Wangda Tan d36d9d676d YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda) 
(cherry picked from commit 12b11e2e68)
 2016-03-31 14:35:59 -07:00
Jian He 3afc2caec8 YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage 2016-03-31 14:31:38 -07:00
Sangjin Lee b91928cd26 YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 
(cherry picked from commit 6d67420dbc)
 2016-03-31 10:49:45 -07:00
Varun Vasudev dcad99892d YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki. 
(cherry picked from commit 0064cba169)
 2016-03-31 14:06:45 +05:30
Varun Vasudev cda9e1d09b YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 
(cherry picked from commit f1b8f6b2c1)
 2016-03-31 14:03:08 +05:30
Jian He f1f441b80f YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-03-30 12:46:36 -07:00
Robert Kanter b7e6d5cbfe YARN-4436. DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled (mattlamantia via rkanter) 
(cherry picked from commit 8286270466)
 2016-03-29 17:21:50 -07:00
Wangda Tan 6856a7183a YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 
(cherry picked from commit fc055a3cbe)
 2016-03-29 17:10:17 -07:00
Xuan ffe01e05cd YARN-4863. AHS Security login should be in serviceInit() instead of serviceStart(). Contributed by Junping Du 
(cherry picked from commit 80182809ae)
 2016-03-28 22:18:56 -07:00
Jason Lowe edf17fe8e5 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 
(cherry picked from commit 948b758070)
 2016-03-28 23:02:15 +00:00
Jian He c7d843af3b YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du 2016-03-28 11:13:02 -07:00
Jian He d2f9adca88 Missing file for YARN-4117. 2016-03-28 09:23:06 -07:00
Jian He bdc648ebe7 YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola 2016-03-27 20:22:49 -07:00
Karthik Kambatla 4212f2e2bf YARN-4805. Don't go through all schedulers in ParameterizedTestBase. (kasha) 
(cherry picked from commit 49ff54c860)
 2016-03-26 21:45:29 -07:00
Jason Lowe 3cea00a294 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong 
(cherry picked from commit e8fc81f9c8)
 2016-03-25 20:19:15 +00:00
Junping Du c722262c75 YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev. 
(cherry picked from commit 19b645c938)
 2016-03-23 19:35:14 -07:00
Junping Du 76602161c0 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong. 
(cherry picked from commit af1d125f9c)
 2016-03-23 08:58:05 -07:00
Karthik Kambatla 95b59c17a7 YARN-4732. *ProcessTree classes have too many whitespace issues 
(cherry picked from commit 7fae4c68e6)
 2016-03-20 10:47:56 -07:00
Eric Payne dd1e4107e5 YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger. 
(cherry picked from commit 92b7e0d413)
 2016-03-18 17:05:53 +00:00
Junping Du 66257613b4 YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API. 
(cherry picked from commit ca8106d2dd)
 2016-03-17 09:25:36 -07:00
Karthik Kambatla bbe9bb078c YARN-4812. TestFairScheduler#testContinuousScheduling fails intermittently. (kasha) 
(cherry picked from commit f84af8bd58)
 2016-03-17 05:54:40 -07:00
Wangda Tan 484976fa2b YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 
(cherry picked from commit 7e8c9beb41)
(cherry picked from commit ae14e5d07f)
 2016-03-16 17:03:35 -07:00
Karthik Kambatla ab03266831 YARN-4560. Make scheduler error checking message more user friendly. (Ray Chiang via kasha) 
(cherry picked from commit 3ef5500783)
 2016-03-15 23:47:19 -07:00
Xuan 398dbde538 Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)" 
This reverts commit 3a0b695441.

(cherry picked from commit a888b5bc53)
 2016-03-15 22:11:16 -07:00
Li Lu 94649b9cc0 YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9) 
(cherry-picked from commit 5176a69868)
 2016-03-15 11:48:37 -07:00
Li Lu 14a4110b75 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9) 
(cherry picked from commit 3a0b695441)
 2016-03-15 11:42:30 -07:00
Robert Kanter da24fde333 TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 
(cherry picked from commit 22ca176dfe)
 2016-03-15 10:05:28 -07:00
Siddharth Seth edd9abf302 YARN-4816. Fix incompatible change in SystemClock. 
(cherry picked from commit eba66a64d2)
 2016-03-14 20:15:41 -07:00
Karthik Kambatla b4c8693096 YARN-4719. Add a helper library to maintain node state and allows common queries. (kasha) 
(cherry picked from commit 20d389ce61)
 2016-03-14 14:22:21 -07:00
Junping Du 3d5ac829da YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 
(cherry picked from commit f291d82cd4)
 2016-03-14 08:30:07 -07:00
Li Lu 2b16a54fbe YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. 
This commit amends commit d49cfb3504 with a missed test file.

(cherry picked from commit 017d2c127b)
 2016-03-10 13:04:57 -08:00
Li Lu 76ef097fd1 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 
(cherry-picked from commit d49cfb3504)
 2016-03-10 10:56:51 -08:00
Allen Wittenauer 9f4b3eaac0 Revert "HADOOP-12366. expose calculated paths (aw)" 
This reverts commit 45bb467918.
 2016-03-09 09:59:19 -08:00
Chris Nauroth 38a031ce68 HADOOP-12905. Clean up CHANGES.txt RAT exclusions from pom.xml files. Contributed by Chris Nauroth. 
(cherry picked from commit 89b16d27e2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/pom.xml
	hadoop-mapreduce-project/pom.xml
 2016-03-08 21:22:57 -08:00
Wangda Tan f7b38a7fb8 YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda) 
(cherry picked from commit 0233d4e0ee)
 2016-03-08 14:28:26 -08:00
Jian He fb139b0c40 YARN-4764. Application submission fails when submitted queue is not available in scheduler xml. Contributed by Bibin A Chundatt 
(cherry picked from commit 3c33158d1c)
 2016-03-08 13:12:33 -08:00
Akira Ajisaka 0586386472 HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit f86850b544)
 2016-03-08 14:29:46 +09:00
Vinod Kumar Vavilapalli da9f39b107 YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv) 
(cherry picked from commit b2661765a5)
 2016-03-07 11:11:29 -08:00
Jason Lowe adcdcfd5c1 YARN-4760. proxy redirect to history server uses wrong URL. Contributed by Eric Badger 
(cherry picked from commit 4163e36c2b)
 2016-03-07 15:57:44 +00:00
Jason Lowe 4eace7ab43 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 
(cherry picked from commit 059caf9989)
 2016-03-07 15:45:47 +00:00
Varun Vasudev 78919f8c34 YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e8)
 2016-03-07 16:19:27 +05:30
Varun Vasudev e9a0ffc7f1 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 
(cherry picked from commit 43416187c07afb35e3267f94d0a41d8d3cfb5735)
 2016-03-07 15:23:36 +05:30
Zhihai Xu 7ac7ca48b7 YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee 
(cherry picked from commit e1ccc9622b)
 2016-03-06 19:49:47 -08:00
Rohith Sharma K S 1415e6190a YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit 77e2c3e8c7365b2aca00b6169829f87c63e4b460)
 2016-03-05 13:07:20 +05:30
Masatake Iwasaki 722182fdfb HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 
(cherry picked from commit cbd31328a6)

 Conflicts:
	hadoop-common-project/hadoop-common/src/site/markdown/CommandsManual.md
	hadoop-common-project/hadoop-common/src/site/markdown/UnixShellGuide.md
 2016-03-04 14:12:44 +09:00
Andrew Wang 670944810c HADOOP-11792. Remove all of the CHANGES.txt files. 2016-03-03 21:02:58 -08:00
Jian He 023c2d2e56 YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding 2016-03-01 13:14:51 -08:00
Jian He 589b537631 YARN-4748. ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport. Contributed by Li Lu 
(cherry picked from commit d93c22ec27)
 2016-02-29 18:19:34 -08:00
Karthik Kambatla 84172b047b YARN-4704. TestResourceManager#testResourceAllocation() fails when using FairScheduler. (Yufei Gu via kasha) 
(cherry picked from commit 9dafaaaf0d)
 2016-02-29 16:10:26 -08:00
Haohui Mai c5db4ab0b4 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 14:10:18 -08:00
Jason Lowe bd0f5085e3 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 
(cherry picked from commit c58a6d53c5)
 2016-02-29 15:26:26 +00:00
Rohith Sharma K S 2a1bb6cb67 YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit e0b14f26f5)
 2016-02-29 10:50:23 +08:00
Karthik Kambatla f3b37d8020 YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post YARN-1011. (Inigo Goiri via kasha) 
(cherry picked from commit f9692770a5)
 2016-02-28 10:01:48 -08:00
Jason Lowe 0bd7ba4ea8 YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla 
(cherry picked from commit 6b0f813e89)
 2016-02-26 20:25:56 +00:00
Ming Ma 1656bcec5f YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 
(cherry picked from commit 7f3139e54d)
 2016-02-26 08:43:14 -08:00
Robert Kanter 872b8d90a6 YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 
(cherry picked from commit d7fdec1e6b)
 2016-02-25 16:40:05 -08:00
Robert Kanter ec48848711 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 
(cherry picked from commit c4d4df8de0)
 2016-02-25 12:48:29 -08:00
Karthik Kambatla 6a75c5af09 YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE. (kasha) 
(cherry picked from commit c684f2b007)
 2016-02-24 18:34:21 -08:00
Robert Kanter c2098d2470 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 
(cherry picked from commit 954dd57043)
 2016-02-24 15:00:48 -08:00
Li Lu 51162f9629 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via 
gtcarrera9)
 2016-02-24 13:52:11 -08:00
Sangjin Lee 432a2367ce YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 
(cherry picked from commit 553b591ba0)
 2016-02-24 09:30:37 -08:00
Jason Lowe acffe82353 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 
(cherry picked from commit d284e187b8)
 2016-02-23 20:51:01 +00:00
Junping Du e3ce0ffdc3 YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou. 
(cherry picked from commit 9ed17f181d)
 2016-02-23 03:35:47 -08:00
Tsuyoshi Ozawa 4ee55d0322 YARN-4648. Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption. Contributed by Kai Sasaki. 
(cherry picked from commit 0e12114c9c)
 2016-02-23 19:50:40 +09:00
Rohith Sharma K S 544a2ff7c9 YARN-4651. Document movetoqueue option in 'YARN Commands'. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit de09f41147)
 2016-02-23 18:07:57 +08:00
Varun Vasudev 2c218ca8a8 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 
(cherry picked from commit 140cb5d745)
 2016-02-23 12:32:16 +05:30
Zhe Zhang 56b7f2c2c1 HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh. 
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
 2016-02-22 14:14:39 -08:00
Junping Du 7402448387 YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla. 
(cherry picked from commit 3fab88540f)
 2016-02-22 07:38:40 -08:00
Akira Ajisaka 34635f9e37 YARN-4707. Remove the extra char (>) from SecureContainer.md. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5e7d4d51f8)
 2016-02-22 17:18:03 +09:00
Tsuyoshi Ozawa 929e869ae0 YARN-4708. Missing default mapper type in TimelineServer performance test tool usage. Contributed by Kai Sasaki. 
(cherry picked from commit b68901d7dd)
 2016-02-22 07:55:35 +09:00
Rohith Sharma K S d4203c9aa2 YARN-4654. Yarn node label CLI should parse "=" correctly when trying to remove all labels on a node. (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit c1afac3a98)
 2016-02-18 14:17:04 +05:30
Sangjin Lee 23b5c71729 YARN-4690. Skip object allocation in FSAppAttempt#getResourceUsage when possible (Ming Ma via sjlee) 
(cherry picked from commit 7de70680fe)
 2016-02-17 21:05:10 -08:00
Karthik Kambatla 25c2597f9c YARN-4569. Remove incorrect part of maxResources in FairScheduler documentation. (Ray Chiang via kasha) 
(cherry picked from commit a0c95b5fc4)
 2016-02-17 18:43:41 -08:00
Karthik Kambatla 33c1988595 YARN-4689. FairScheduler: Cleanup preemptContainer to be more readable. (Kai Sasaki via kasha) 
(cherry picked from commit 2ab4c476ed)
 2016-02-17 18:28:08 -08:00
Jian He 68d98f70ba YARN-4653. Document YARN security model from the perspective of Application Developers. Contributed by Steve Loughran 2016-02-14 17:14:35 +08:00
Junping Du a28be9edc7 Set the release date for 2.6.4. 
(cherry picked from commit ec12ce8f48)
 2016-02-13 01:44:12 -08:00
Steve Loughran 2ea1913cad YARN-4682. AMRM client to log when AMRM token updated. (Prabhu Joseph via stevel) 2016-02-12 10:58:46 +00:00
Varun Vasudev dcead143fe YARN-4684. TestYarnCLI#testGetContainers failing in CN locale. Contributed by Bibin A Chundatt. 
(cherry picked from commit 74e9d17a813baeb4e610eb5fdd8e289660346782)
 2016-02-12 12:08:57 +05:30
Arun Suresh 6a238e4a6b YARN-2575. Create separate ACLs for Reservation create/update/delete/list ops (Sean Po via asuresh) 
(cherry picked from commit 23f937e3b7)
 2016-02-11 10:49:11 -08:00
Varun Vasudev 3a5b39e931 YARN-4655. Log uncaught exceptions/errors in various thread pools in YARN. Contributed by Sidharta Seethana. 
(cherry picked from commit fa00d3e205)
 2016-02-11 12:09:29 +05:30
Varun Vasudev 05b57c87f9 YARN-4628. Display application priority in yarn top. Contributed by Bibin A Chundatt. 
(cherry picked from commit 663a80031c)
 2016-02-11 12:00:32 +05:30
Jian He 0c9110d15e Move YARN-4519 in CHANGES.txt to 2.8 2016-02-11 10:19:42 +08:00
Jian He 358cfcf9ea YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding 2016-02-11 10:09:04 +08:00
= fc9eb4577c YARN-4420. Add REST API for List Reservations. (Sean Po via curino) 2016-02-10 14:26:38 -08:00
Arun Suresh 17f83c8803 YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh) 
(cherry picked from commit 5cf5c41a89)
 2016-02-10 09:13:04 -08:00
Steve Loughran 179b36efb5 YARN-4629. Distributed shell breaks under strong security. (Daniel Templeton via stevel) 2016-02-10 13:00:48 +00:00
Jason Lowe 57d79f908e Update CHANGES.txt to reflect recent commits to branch-2.7 and branch-2.6. 
(cherry picked from commit fba6e9f5ef)
 2016-02-09 16:55:40 +00:00
Devaraj K 329df98f49 YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing 
is configured. Contributed by Naganarasimha G R.

(cherry picked from commit 565af873d5)
 2016-02-08 15:03:11 +05:30
Varun Vasudev be04535f1a YARN-4669. Fix logging statements in resource manager's Application class. Contributed by Sidharta Seethana. 
(cherry picked from commit a4a30c789146c4e824fcc6ebcf10405711120eaa)
 2016-02-04 13:51:08 +05:30
Varun Vasudev 99966b1912 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
 2016-02-04 13:33:38 +05:30
Xuan dfad608b22 YARN-4662. Document some newly added metrics. Contributed by Jian He 
(cherry picked from commit 63c63e298c)
 2016-02-03 20:06:25 -08:00