Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,725 Commits 326 Branches 371 Tags
Commit Graph

864 Commits

Author SHA1 Message Date
Zhijie Shen
04dd05989e YARN-3544. Got back AM logs link on the RM web UI for a completed app. Contributed by Xuan Gong. 2015-04-30 13:07:15 -07:00
Karthik Kambatla
396c41b1d4 YARN-3485. FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies. (kasha) 
(cherry picked from commit 8f82970e0c247b37b2bf33aa21f6a39afa07efde)
(cherry picked from commit 4cc38df7de89a361493826c057f3363f1738a7e2)
 2015-04-28 21:07:32 -07:00
Jian He
5cf35b98e1 YARN-3472. Fixed possible leak in DelegationTokenRenewer#allTokens. Contributed by Rohith Sharmaks 
(cherry picked from commit a1afbc48b53f6bdbd30dc8eb56a7621d49c5d6db)
 2015-04-23 17:43:41 -07:00
Jian He
6cc0e9baa1 YARN-3382. Some of UserMetricsInfo metrics are incorrectly set to root queue metrics. Contributed by Rohit Agarwal 
(cherry picked from commit 944a16579fdb54c1c420162ef830ef2ac5451c69)
 2015-04-23 17:40:56 -07:00
Steve Loughran
b3f2826bfb YARN-2605. [RM HA] Rest api endpoints doing redirect incorrectly. (Xuan Gong via stevel) 2015-04-23 09:22:09 +01:00
Wangda Tan
d6105d944c YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue (Jason Lowe via wangda) 
(cherry picked from commit f47a5763acd55cb0b3f16152c7f8df06ec0e09a9)
 2015-04-17 17:23:34 -07:00
Akira Ajisaka
48d6e994bf MAPREDUCE-6300. Task list sort by task id broken. Contributed by Siqi Li. 
(cherry picked from commit 1fa8075794894b3e0053d251788277fb88e85533)
(cherry picked from commit 51c458a5c4562c54ff147824816f7b44c4b9b8f6)
 2015-04-16 10:16:48 -05:00
Jason Lowe
70ac8cda41 YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column. (Jason Lowe via wangda) 
(cherry picked from commit 73d9e4c77b2aa4734845376d20fb76e6b91b6db0)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-04-10 13:52:34 +00:00
Vinod Kumar Vavilapalli
a7fa46c2e0 Preparing for 2.7.1 development: mvn versions:set 2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli
4c6867ff01 YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token renewal of applications part of a bigger workflow. Contributed by Daryn Sharp. 
(cherry picked from commit 9c5911294e0ba71aefe4763731b0e780cde9d0ca)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-04-09 13:11:26 -07:00
Zhijie Shen
64cf079858 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad356e6318b8ce6c5c96e201149f811bd)
 2015-04-06 13:18:16 -07:00
Jian He
edc3cb736e YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 
(cherry picked from commit 658097d6da1b1aac8e01db459f0c3b456e99652f)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppAttemptBlock.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/CapacitySchedulerLeafQueueInfo.java
 2015-04-06 13:17:31 -07:00
Zhijie Shen
cf33bc1050 Revert "YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong." 
This reverts commit e4059b9cce2703c412abeacc08225c3cdfe415c1.
 2015-04-01 16:41:25 -07:00
Zhijie Shen
e4059b9cce YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad356e6318b8ce6c5c96e201149f811bd)
 2015-04-01 13:49:51 -07:00
cnauroth
57e297208d YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 
(cherry picked from commit 6ca1f12024fd7cec7b01df0f039ca59f3f365dc1)
(cherry picked from commit 342c525eaa0749175f0e3827d245642776d043a5)
 2015-03-23 10:46:22 -07:00
Wangda Tan
57a0382f4b YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 
(cherry picked from commit 6bc7710ec7f2592c4c87dd940fbe5827ef81fe72)
 2015-03-20 10:44:34 -07:00
Jian He
e67bd31ac0 YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 
(cherry picked from commit 4e886eb9cbd2dcb128bbfd17309c734083093a4c)
 2015-03-19 22:34:34 -07:00
Jian He
54e63c52c9 Backport part of YARN-3273 to rename CapacitySchedulerLeafQueueInfo#aMResourceLimit to AMResourceLimit. Contributed by Rohith Sharmaks 2015-03-17 22:08:10 -07:00
Karthik Kambatla
47e6fc2bf9 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)" 
This reverts commit c2b185def846f5577a130003a533b9c377b58fab.

(cherry picked from commit 32b43304563c2430c00bc3e142a962d2bc5f4d58)
 2015-03-17 12:32:14 -07:00
Tsuyoshi Ozawa
f15a2b23ca YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 
(cherry picked from commit 7522a643faeea2d8a8e2c7409ae60e0973e7cf38)
 2015-03-17 08:10:57 +09:00
Tsuyoshi Ozawa
2b2f7f2b90 YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 
(cherry picked from commit 3da9a97cfbcc3a1c50aaf85b1a129d4d269cd5fd)
 2015-03-16 23:20:34 +09:00
Vinod Kumar Vavilapalli
0cd9eb9987 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 863079bb874ba77918ca1c0741eae10e245995c8)
 2015-03-12 13:34:33 -07:00
Zhijie Shen
ec9816b2b3 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 
(cherry picked from commit 85f6d67fa78511f255fcfa810afc9a156a7b483b)
 2015-03-11 19:41:20 -07:00
Jian He
d579fddf24 YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 
(cherry picked from commit c3003eba6f9802f15699564a5eb7c6e34424cb14)
 2015-03-09 20:48:09 -07:00
Tsuyoshi Ozawa
28f4e6b222 HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 
(cherry picked from commit b46f9e72dbed6fd1f8cae1e12973252462d6ee15)
 2015-03-09 19:54:45 +09:00
Jason Lowe
ef3d9bdf6b YARN-3275. CapacityScheduler: Preemption happening on non-preemptable queues. Contributed by Eric Payne 
(cherry picked from commit 27e8ea820fab8dce59f4db9814e73bd60c1d4ef1)
 2015-03-06 22:45:09 +00:00
Jian He
a5f3fb4dc1 YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 
(cherry picked from commit 95bfd087dc89e57a93340604cc8b96042fa1a05a)
 2015-03-05 21:27:03 -08:00
Tsuyoshi Ozawa
6660c2f83b YARN-3249. Add a 'kill application' button to Resource Manager's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 1b672096121fef775572b517d4f5721997abbac6)
 2015-03-05 19:56:59 +09:00
Karthik Kambatla
0d62e94887 YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client. (Zhihai Xu via kasha) 
(cherry picked from commit 8d88691d162f87f95c9ed7e0a569ef08e8385d4f)
 2015-03-04 19:49:21 -08:00
Karthik Kambatla
721d7b5741 YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) 
(cherry picked from commit 22426a1c9f4bd616558089b6862fd34ab42d19a7)
 2015-03-04 18:07:33 -08:00
Jian He
888a445638 YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with a different port. Contributed by Rohith Sharmaks 
(cherry picked from commit b2f1ec312ee431aef762cfb49cb29cd6f4661e86)
 2015-03-03 16:28:55 -08:00
Wangda Tan
0a502c665b YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 
(cherry picked from commit e17e5ba9d7e2bd45ba6884f59f8045817594b284)
 2015-03-03 11:49:45 -08:00
Vinod Kumar Vavilapalli
253c78548b YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 
(cherry picked from commit 14dd647c556016d351f425ee956ccf800ccb9ce2)
 2015-03-02 17:54:01 -08:00
Jian He
76b7283fa1 YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 
(cherry picked from commit 5d0bae550f5b9a6005aa1d373cfe1ec80513dbd9)
 2015-03-02 14:43:12 -08:00
Wangda Tan
0b0be0056b YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 
(cherry picked from commit edcecedc1c39d54db0f86a1325b4db26c38d2d4d)
 2015-02-27 16:14:35 -08:00
Tsuyoshi Ozawa
79f73f4613 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 
(cherry picked from commit 01a1621930df17a745dd37892996c68fca3447d1)
 2015-02-28 00:57:01 +09:00
Konstantin V Shvachko
9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Devaraj K
f83d2e4410 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against 
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.

(cherry picked from commit 0d4296f0e0f545267f2e39a868d4ffefc9844db8)
 2015-02-26 15:47:32 +05:30
Tsuyoshi Ozawa
13e213c41a YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 
(cherry picked from commit 6cbd9f1113fca9ff86fd6ffa783ecd54b147e0db)
 2015-02-25 11:23:34 +09:00
Xuan
0ca974d0b0 YARN-2797. TestWorkPreservingRMRestart should use 
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla

(cherry picked from commit fe7a302473251b7310105a936edf220e401c613f)
 2015-02-21 19:19:08 -08:00
Wangda Tan
a1963968d2 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 
(cherry picked from commit ce5bf927c3d9f212798de1bf8706e5e9def235a1)
 2015-02-20 10:40:04 -08:00
Jason Lowe
3e8dfd1299 YARN-3194. RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node. Contributed by Rohith 
(cherry picked from commit a64dd3d24bfcb9af21eb63869924f6482b147fd3)
 2015-02-20 15:12:08 +00:00
Jian He
eaeaf80d39 YARN-933. Fixed InvalidStateTransitonException at FINAL_SAVING state in RMApp. Contributed by Rohith Sharmaks 
(cherry picked from commit c0d9b93953767608dfe429ddb9bd4c1c3bd3debf)
 2015-02-19 15:43:14 -08:00
Wangda Tan
cd5eb9c1de YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 
(cherry picked from commit d49ae725d5fa3eecf879ac42c42a368dd811f854)
 2015-02-19 11:01:47 -08:00
Tsuyoshi Ozawa
fef5efa0cb YARN-1615. Fix typos in delay scheduler's description. Contributed by Akira Ajisaka. 
(cherry picked from commit b8a14efdf535d42bcafa58d380bd2c7f4d36f8cb)
 2015-02-19 10:24:40 +09:00
Jian He
b8fe1a747e YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit 1c03376300a46722d4147f5b8f37242f68dba0a2)
 2015-02-18 16:34:42 -08:00
Jian He
8f06eaa251 YARN-3132. RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated. Contributed by Wangda Tan 
(cherry picked from commit f5da5566d9c392a5df71a2dce4c2d0d50eea51ee)
 2015-02-18 11:52:28 -08:00
Tsuyoshi Ozawa
fabac2ff30 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and Devaraj K. 
(cherry picked from commit 556386a07084b70a5d2ae0c2bd4445a348306db8)
 2015-02-17 01:54:09 +09:00
Tsuyoshi Ozawa
42368b91c4 Revert "YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj." 
This reverts commit 67e3a77720cfcb0fc540e8e0c80a5d42b8140fcc.
 2015-02-17 01:53:58 +09:00
Tsuyoshi Ozawa
67e3a77720 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj. 
(cherry picked from commit 9aae81c93421874b726c7b6ff970895c429e502d)
 2015-02-17 01:48:53 +09:00