Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,044 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2530 Commits

Author SHA1 Message Date
Varun Vasudev 05b57c87f9 YARN-4628. Display application priority in yarn top. Contributed by Bibin A Chundatt. 
(cherry picked from commit 663a80031c)
 2016-02-11 12:00:32 +05:30
Jian He 0c9110d15e Move YARN-4519 in CHANGES.txt to 2.8 2016-02-11 10:19:42 +08:00
Jian He 358cfcf9ea YARN-4138. Roll back container resource allocation after resource increase token expires. Contributed by Meng Ding 2016-02-11 10:09:04 +08:00
= fc9eb4577c YARN-4420. Add REST API for List Reservations. (Sean Po via curino) 2016-02-10 14:26:38 -08:00
Arun Suresh 17f83c8803 YARN-4360. Improve GreedyReservationAgent to support "early" allocations, and performance improvements (curino via asuresh) 
(cherry picked from commit 5cf5c41a89)
 2016-02-10 09:13:04 -08:00
Steve Loughran 179b36efb5 YARN-4629. Distributed shell breaks under strong security. (Daniel Templeton via stevel) 2016-02-10 13:00:48 +00:00
Jason Lowe 57d79f908e Update CHANGES.txt to reflect recent commits to branch-2.7 and branch-2.6. 
(cherry picked from commit fba6e9f5ef)
 2016-02-09 16:55:40 +00:00
Devaraj K 329df98f49 YARN-4667. RM Admin CLI for refreshNodesResources throws NPE when nothing 
is configured. Contributed by Naganarasimha G R.

(cherry picked from commit 565af873d5)
 2016-02-08 15:03:11 +05:30
Varun Vasudev be04535f1a YARN-4669. Fix logging statements in resource manager's Application class. Contributed by Sidharta Seethana. 
(cherry picked from commit a4a30c789146c4e824fcc6ebcf10405711120eaa)
 2016-02-04 13:51:08 +05:30
Varun Vasudev 99966b1912 YARN-4307. Display blacklisted nodes for AM container in the RM web UI. Contributed by Naganarasimha G R. 
(cherry picked from commit 308d63f382)
 2016-02-04 13:33:38 +05:30
Xuan dfad608b22 YARN-4662. Document some newly added metrics. Contributed by Jian He 
(cherry picked from commit 63c63e298c)
 2016-02-03 20:06:25 -08:00
Jason Lowe b12f16ab73 YARN-4594. container-executor fails to remove directory tree when chmod required. Contributed by Colin Patrick McCabe 
(cherry picked from commit fa328e2d39)
 2016-02-03 17:22:12 +00:00
Varun Vasudev e211ab777e YARN-4625. Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent. Contributed by Xuan Gong. 
(cherry picked from commit 1adb64e09b)
 2016-02-03 16:28:31 +05:30
Junping Du 3583c77ee8 Adding release 2.6.5 to CHANGES.txt 2016-02-02 18:09:17 -08:00
Devaraj K 43131af1e2 YARN-4100. Add Documentation for Distributed and Delegated-Centralized 
Node Labels feature. Contributed by Naganarasimha G R.

(cherry picked from commit db144eb1c5)
 2016-02-02 12:07:49 +05:30
Varun Vasudev bbda446889 YARN-4649. Add additional logging to some NM state store operations. Contributed by Sidharta Seethana 
(cherry picked from commit 1cd55e0c17)
 2016-02-02 09:26:22 +05:30
Wangda Tan 9591363ad8 YARN-4340. Add list API to reservation system. (Sean Po via wangda) 
(cherry picked from commit 9875325d5c)
 2016-02-02 10:19:19 +08:00
Jason Lowe 36aae8050e YARN-3102. Decommisioned Nodes not listed in Web UI. Contributed by Kuhu Shukla 
(cherry picked from commit ed55950164)
 2016-02-01 23:16:18 +00:00
Rohith Sharma K S 676f215595 YARN-4615. Fix random test failure in TestAbstractYarnScheduler#testResourceRequestRecoveryToTheRightAppAttempt. (Sunil G via rohithsharmaks) 
(cherry picked from commit 2673cbaf55)
 2016-02-01 10:45:05 +05:30
Jason Lowe 016c685135 YARN-4428. Redirect RM page to AHS page when AHS turned on and RM page is not available. Contributed by Chang Li 
(cherry picked from commit 772ea7b41b)
 2016-01-29 21:54:56 +00:00
Jian He 885c76d85f YARN-4617. LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy. Contributed by Rohith Sharma K S 2016-01-29 12:22:44 -08:00
Karthik Kambatla 9d1371d390 YARN-4647. Make RegisterNodeManagerRequestPBImpl thread-safe. (kasha) 
(cherry picked from commit c9a09d6926)
 2016-01-29 08:19:34 -08:00
Devaraj K 123269c07f YARN-4411. RMAppAttemptImpl#createApplicationAttemptReport throws 
IllegalArgumentException. Contributed by Bibin A Chundatt and yarntime.

(cherry picked from commit a277bdc9ed)
 2016-01-29 13:52:47 +05:30
Rohith Sharma K S b8d5df06f2 YARN-4543. Fix random test failure in TestNodeStatusUpdater.testStopReentrant. (Akihiro Suda via rohithsharmaks) 
(cherry picked from commit ac68666803)
 2016-01-29 12:30:53 +05:30
Rohith Sharma K S 0466743afb YARN-4219. addendum patch to fix javadoc errors 
(cherry picked from commit 09d831c95b)
 2016-01-29 11:53:06 +05:30
Jian He 36f0e73882 YARN-4519. Potential deadlock of CapacityScheduler between decrease container and assign containers. Contributed by Meng Ding 2016-01-28 14:52:00 -08:00
Xuan 2f00d8d3a7 YARN-4219. New levelDB cache storage for timeline v1.5. Contributed by 
Li Lu

(cherry picked from commit 9fab22b366)
 2016-01-28 14:36:58 -08:00
Jason Lowe 29195c7668 YARN-4643. Container recovery is broken with delegating container runtime. Contributed by Sidharta Seethana 
(cherry picked from commit 61382ff8fa)
 2016-01-28 19:00:51 +00:00
Rohith Sharma K S 6947a5d4be YARN-4633. Fix random test failure in TestRMRestart#testRMRestartAfterPreemption. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit ef343be82b)
 2016-01-28 21:55:12 +05:30
Karthik Kambatla c25154576b YARN-4462. FairScheduler: Disallow preemption from a queue. (Tao Jie via kasha) 
(cherry picked from commit fb238d7e5d)
 2016-01-27 12:29:25 -08:00
Rohith Sharma K S 0b8767e01a YARN-4573. Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit c01bee0108)
 2016-01-27 08:24:37 +05:30
Xuan 69f3fa71a2 YARN-4612. Fix rumen and scheduler load simulator handle killed tasks 
properly. Contributed by Ming Ma.

(cherry picked from commit 4efdf3a979)
 2016-01-26 18:18:03 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 47b20d0c74 YARN-3542. Refactored existing CPU cgroups support to use the newer and integrated ResourceHandler mechanism, and also deprecated the old LCEResourceHandler inteface hierarchy. Contributed by Varun Vasudev. 
(cherry picked from commit 2085e60a96)
 2016-01-25 16:22:47 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 97f6cfa94b Release process for 2.7.2: Set the release date for 2.7.2 
(cherry picked from commit ec4d2d9f40)
 2016-01-25 15:47:02 -08:00
Jian He 2d236c9de2 YARN-4520. Finished app info is unnecessarily persisted in NM state-store if container is acquired but not lunched on this node. Contributed by sandflee 2016-01-25 15:36:40 -08:00
Akira Ajisaka 02d4a1fce3 YARN-4592. Remove unused GetContainerStatus proto. Contributed by Chang Li. 
(cherry picked from commit 56a0c17508)
 2016-01-25 15:20:45 +09:00
rohithsharmaks 7e66b2f22c YARN-4613. Fix test failure in TestClientRMService#testGetClusterNodes. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit 10dc2c0493)
 2016-01-24 23:37:11 +05:30
rohithsharmaks 280c9b8401 YARN-4614. Fix random failure in TestApplicationPriority#testApplicationPriorityAllocationWithChangeInPriority. (Sunil G via rohithsharmaks) 
(cherry picked from commit 99829eb221)
 2016-01-23 07:58:16 +05:30
Xuan 1d19557dda YARN-4496. Improve HA ResourceManager Failover detection on the client. 
Contributed by Jian He

(cherry picked from commit 618bfd6ac2)
 2016-01-22 18:21:50 -08:00
Jason Lowe 635849fa45 YARN-4598. Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL. Contributed by tangshangwen 
(cherry picked from commit 46e5ea81e0)
 2016-01-22 21:55:59 +00:00
Jason Lowe ad84f90e88 YARN-4371. "yarn application -kill" should take multiple application ids. Contributed by Sunil G 
(cherry picked from commit e91e8b711c)
 2016-01-22 21:37:08 +00:00
rohithsharmaks 1cec12d95a YARN-4574. Fix random failure in TestAMRMClientOnRMRestart. (Takashi Ohnishi via rohithsharmaks) 
(cherry picked from commit 34a3900773)
 2016-01-22 21:18:42 +05:30
rohithsharmaks 493275b27b YARN-4497. RM might fail to restart when recovering apps whose attempts are missing. (Jun Gong via rohithsharmaks) 
(cherry picked from commit d6258b33a7)
 2016-01-22 20:29:10 +05:30
Akira Ajisaka ddc22bbcb1 YARN-4605. Spelling mistake in the help message of "yarn applicationattempt" command. Contributed by Weiwei Yang. 
(cherry picked from commit 8f58f742ae)
 2016-01-22 19:43:37 +09:00
Varun Vasudev 85561207f9 YARN-4578. Directories that are mounted in docker containers need to be more restrictive/container-specific. Contributed by Sidharta Seethana. 
(cherry picked from commit b41a7e89d1)
 2016-01-22 14:44:22 +05:30
Rohith Sharma K S 844a8e3771 YARN-4584. RM startup failure when AM attempts greater than max-attempts. (Bibin A Chundatt via rohithsharmaks) 
(cherry picked from commit e30668106d)
 2016-01-22 10:20:09 +05:30
Jason Lowe d051200a62 YARN-4610. Reservations continue looking for one app causes other apps to starve. Contributed by Jason Lowe 
(cherry picked from commit 468a53b22f)
 2016-01-21 18:32:47 +00:00
Akira Ajisaka bf4901c609 YARN-4608. Redundant code statement in WritingYarnApplications. Contributed by Kai Sasaki. 
(cherry picked from commit d3224e2d6274daf73a103c0dbe21ba69add7ca26)
 2016-01-22 02:14:53 +09:00
Karthik Kambatla 5c2f495fcc YARN-4603. FairScheduler should mention user requested queuename in error message when failed in queue ACL check. (Tao Jie via kasha) 
(cherry picked from commit 4992398aee)
 2016-01-21 17:43:49 +01:00
Jason Lowe 4e6fc3ca18 YARN-4492. Add documentation for preemption supported in Capacity scheduler. Contributed by Naganarasimha G R 
(cherry picked from commit f858ac2426)
 2016-01-21 15:39:51 +00:00
Wangda Tan 9512866740 YARN-4557. Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*. (Naganarasimha G R via wangda) 
(cherry picked from commit c9fe0371a8ab40c0d3437fadd32788b320788a7a)
 2016-01-21 11:16:58 +08:00
Xuan 66b030f608 YARN-4611. Fix scheduler load simulator to support multi-layer network 
location. Contributed by Ming Ma.

(cherry picked from commit 1708a4cd23)
 2016-01-20 15:43:02 -08:00
Xuan be3322792f YARN-4559. Make leader elector and zk store share the same curator 
client. Contributed by Jian He

(cherry picked from commit 890a2ebd1a)
 2016-01-20 14:49:04 -08:00
Wangda Tan 23deeb4230 HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo Goiri via wangda) 
(cherry picked from commit 89d1fd5dac)
 2016-01-19 21:28:51 +08:00
Jian He bb2a9a78c0 YARN-4565. Fix a bug that leads to AM resource limit not hornored when sizeBasedWeight enabled for FairOrderingPolicy. Contributed by Wangda Tan 2016-01-18 21:05:03 -08:00
Wangda Tan 704b37d0e1 YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit a44ce3f14f)
 2016-01-19 09:31:13 +08:00
Wangda Tan 1625281ccb Revert "YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda)" 
This reverts commit 3fe5728563.

Conflicts:
	hadoop-yarn-project/CHANGES.txt

(cherry picked from commit 150f5ae034)
 2016-01-19 09:31:07 +08:00
Jian He 6db022fc48 YARN-4596. SystemMetricPublisher should not swallow error messages from TimelineClient#putEntities. Contributed by Li Lu 2016-01-18 16:59:04 -08:00
rohithsharmaks da887e4252 YARN-4265 Addendum patch to fix compilation issue caused by project version number 2016-01-18 16:38:41 +05:30
Karthik Kambatla ef59521094 YARN-4526. Make SystemClock singleton so AppSchedulingInfo could use it. (kasha) 
(cherry picked from commit d40859fab1)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapred/TestTaskAttemptListenerImpl.java
 2016-01-18 10:59:34 +01:00
Wangda Tan c56d46307a Move YARN-4524 from 2.9 to 2.8 2016-01-18 17:04:41 +08:00
Wangda Tan 552f15739c YARN-4304. AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics. (Sunil G via wangda) 
(cherry picked from commit b08ecf5c75)
 2016-01-18 11:16:04 +08:00
Wangda Tan 6dab78c71a YARN-4502. Fix two AM containers get allocated when AM restart. (Vinod Kumar Vavilapalli via wangda) 
(cherry picked from commit 805a9ed85e)
 2016-01-18 11:15:59 +08:00
Wangda Tan c1193b46d4 YARN-4538. QueueMetrics pending cores and memory metrics wrong. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9523648d57)
 2016-01-18 11:15:53 +08:00
Junping Du 4a30a44b11 YARN-4265. Provide new timeline plugin storage to support fine-grained entity caching. Contributed by Li Lu and Jason Lowe 
(cherry picked from commit 02f597c5db)
 2016-01-17 17:38:42 -08:00
Colin Patrick Mccabe 48b8c99479 HADOOP-12712. Fix some cmake plugin and native build warnings (cmccabe) 
(cherry picked from commit b2c155f810)
 2016-01-15 13:22:14 -08:00
rohithsharmaks d3c08cde68 YARN-4389. Allow application to enable or disable am blacklisting. (Sunil G via rohithsharmaks) 2016-01-15 22:46:21 +05:30
Junping Du e09222306f YARN-4581. AHS writer thread leak makes RM crash while RM is recovering. Contributed by sandflee. 
(cherry picked from commit fc6d3a3b23)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2016-01-15 08:43:30 -08:00
Colin Patrick Mccabe cea1c71087 HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe) 
(cherry picked from commit b1ed28fa77)

Conflicts:
	hadoop-common-project/hadoop-common/pom.xml
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
 2016-01-14 11:13:13 -08:00
Karthik Kambatla f0923819c3 YARN-3446. FairScheduler headroom calculation should exclude nodes in the blacklist. (Zhihai Xu via kasha) 
(cherry picked from commit 9d04f26d4c)
 2016-01-14 08:33:52 -08:00
Varun Vasudev 6220a024b1 YARN-4553. Add cgroups support for docker containers. Contributed by Sidharta Seethana. 
(cherry picked from commit 3ddb92bd30)
 2016-01-14 14:30:19 +05:30
Akira Ajisaka ffca0ebfff YARN-4534. Remove the redundant symbol in yarn rmadmin help msg. Contributed by Lin Yiqun. 
(cherry picked from commit 62b0d61513)
 2016-01-14 13:59:58 +09:00
Karthik Kambatla a1ff0b4145 YARN-4551. Address the duplication between StatusUpdateWhenHealthy and StatusUpdateWhenUnhealthy transitions. (Sunil G via kasha) 
(cherry picked from commit 321072ba81)
 2016-01-13 12:09:58 -08:00
Wangda Tan c0b97865e8 YARN-4571. Make app id/name available to the yarn authorizer provider for better auditing. (Jian He via wangda) 
(cherry picked from commit c0537bcd2c)
 2016-01-13 13:19:01 +08:00
Jason Lowe d164cdcda1 YARN-4414. Nodemanager connection errors are retried at multiple levels. Contributed by Chang Li 
(cherry picked from commit 13de8359a1)
 2016-01-12 15:57:07 +00:00
Jason Lowe 8d34d899e4 Update CHANGES.txt for recent commits to 2.7 and 2.6 
(cherry picked from commit c77c35f83c)
 2016-01-12 15:45:21 +00:00
Akira Ajisaka 93b7ef8aec YARN-4567. javadoc failing on java 8. Contributed by Steve Loughran. This closes #67. 
(cherry picked from commit da1e3e3c57)
 2016-01-12 15:13:38 +09:00
Wangda Tan 15e174fe79 YARN-4582. Label-related invalid resource request exception should be able to properly handled by application. (Bibin A Chundatt via wangda) 
(cherry picked from commit 9e792da014)
 2016-01-12 12:54:09 +08:00
Akira Ajisaka df95343047 HADOOP-12584. Disable browsing the static directory in HttpServer2. Contributed by Robert Kanter. 
(cherry picked from commit 9c89bcd04212543ae279d34938ec2ad319e5ba6d)
 2016-01-12 13:34:44 +09:00
Jian He c4aed35200 Missing file for YARN-4580 2016-01-11 17:05:11 -08:00
Jian He 591e7fb221 YARN-4537. Pull out priority comparison from fifocomparator and use compound comparator for FifoOrdering policy. Contributed by Rohith Sharma K S 
(cherry picked from commit b8942be888)
 2016-01-11 16:45:14 -08:00
Rohith Sharma K S da42b01645 Add YARN-3849 to Release 2.6.4 entry in CHANGES.txt 
(cherry picked from commit 76e7270851)
 2016-01-11 12:18:12 +05:30
Jian He d4ff8fd5d8 YARN-4479. Change CS LeafQueue pendingOrderingPolicy to hornor recovered apps. Contributed by Rohith Sharma K S 
(cherry picked from commit 109e528ef5)
 2016-01-08 15:51:33 -08:00
Xuan 2cbbf76c3d YARN-4438. Implement RM leader election with curator. Contributed by Jian He 
(cherry picked from commit 89022f8d4b)
 2016-01-07 14:34:14 -08:00
Akira Ajisaka 3905435d3d YARN-4544. All the log messages about rolling monitoring interval are shown with WARN level. Contributed by Takashi Ohnishi. 
(cherry picked from commit 52b77577c4)
 2016-01-08 01:13:30 +09:00
Steve Loughran d9459ffa00 YARN-4550. Some tests in TestContainerLanch fails on non-english locale environment. (Takashi Ohnishi via stevel) 2016-01-07 14:30:07 +00:00
rohithsharmaks 4865642e7c YARN-4393. Fix intermittent test failure for TestResourceLocalizationService#testFailedDirsResourceRelease (Varun Saxana via rohithsharmaks) 
(cherry picked from commit 791c1639ae)
 2016-01-07 09:39:37 +05:30
Junping Du 818cd0e90d Add YARN-2975, YARN-3893, YARN-2902 and YARN-4354 to Release 2.6.4 entry in CHANGES.txt 
(cherry picked from commit b6c9d3fab9)
 2016-01-06 14:28:43 -08:00
Junping Du 1cc001db4c YARN-4546. ResourceManager crash due to scheduling opportunity overflow. Contributed by Jason Lowe. 
(cherry picked from commit c1462a67ff)
 2016-01-06 05:51:28 -08:00
Zhihai Xu f41db81d5f update CHANGES.txt to add YARN-3697 to 2.6.4 release 
(cherry picked from commit 96d8f1d6f3)
 2016-01-05 00:42:35 -08:00
rohithsharmaks 0492552f30 YARN-4535. Fix checkstyle error in CapacityScheduler.java (Naganarasimha G R via rohithsharmaks) 
(cherry picked from commit 6da6d87872)
 2016-01-05 12:11:48 +05:30
Akira Ajisaka 247ab92464 Set release date for 2.6.3 
(cherry picked from commit db99e30f67)
 2016-01-04 16:12:52 +09:00
Rohith Sharma K S ae25e283cf YARN-4530. LocalizedResource trigger a NPE Cause the NodeManager exit. (tangshangwen via rohithsharmaks) 
(cherry picked from commit f9e36dea96)
 2016-01-04 10:04:27 +05:30
Wangda Tan 05fa852d75 YARN-4524. Cleanup AppSchedulingInfo. (Karthik Kambatla via wangda) 2015-12-30 15:36:55 -08:00
Wangda Tan 6eefae1b33 YARN-4510. Fix SLS startup failure caused by NPE. (Bibin A Chundatt via wangda) 2015-12-30 15:34:39 -08:00
Wangda Tan 9a7d5f69ec YARN-4522. Queue acl can be checked at app submission. (Jian He via wangda) 
(cherry picked from commit 8310b2e9ff)
 2015-12-30 15:31:11 -08:00
Junping Du 401d1e6e8a YARN-1382. Remove unusableRMNodesConcurrentSet (never used) in NodeListManager to get rid of memory leak. Contributed by Rohith Sharma K S. 
(cherry picked from commit 223ce323bb)
 2015-12-30 07:53:08 -08:00
Jian He 970bb2da15 YARN-4098. Document ApplicationPriority feature. Contributed by Rohith Sharma K S 2015-12-29 16:52:14 -08:00
Jian He 3661d4907a YARN-3480. Remove attempts that are beyond max-attempt limit from state store. Contributed by Jun Gong 
(cherry picked from commit 5273413411)
 2015-12-29 15:59:42 -08:00
Wangda Tan 3efbde5378 YARN-4315. NaN in Queue percentage for cluster apps page. (Bibin A Chundatt via wangda) 
(cherry picked from commit 561abb9fee)
 2015-12-29 13:29:19 -08:00