Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,942 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

1862 Commits

Author SHA1 Message Date
Wangda Tan 5f59e621ba YARN-2901 addendum: Fixed findbugs warning caused by previously patch 
(cherry picked from commit ba9ee22ca4)
 2015-04-08 11:02:54 -07:00
Wangda Tan 97244b143c YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender. (Varun Vasudev via wangda) 
(cherry picked from commit 7af086a515)
 2015-04-08 10:58:38 -07:00
Tsuyoshi Ozawa ac32fa187c YARN-3457. NPE when NodeManager.serviceInit fails and stopRecoveryStore called. Contributed by Bibin A Chundatt. 
(cherry picked from commit dd852f5b8c)
 2015-04-08 15:56:33 +09:00
Robert Kanter b584ce6c0a YARN-3429. Fix incorrect CHANGES.txt 
(cherry picked from commit 5b8a3ae366)
 2015-04-07 16:16:08 -07:00
Xuan 0522d6970d YARN-3294. Allow dumping of Capacity Scheduler debug logs via web UI for 
a fixed time period. Contributed by Varun Vasudev

(cherry picked from commit d27e9241e8)
 2015-04-07 09:54:29 -07:00
Xuan 7941cc554b YARN-3110. Few issues in ApplicationHistory web ui. Contributed by Naganarasimha G R 
(cherry picked from commit 19a4feaf6f)
 2015-04-07 08:24:09 -07:00
Zhijie Shen 735e4f15ab Move YARN-3273 from 2.8 to 2.7. 
(cherry picked from commit 5e9f4099de4b949c838a6224442f335c67d80936)
 2015-04-06 14:15:23 -07:00
Robert Kanter 354d2c75cf YARN-2429. TestAMRMTokens.testTokenExpiry fails Intermittently with error message:Invalid AMRMToken (zxu via rkanter) 
(cherry picked from commit 99b08a748e)
 2015-04-06 14:11:50 -07:00
Tsuyoshi Ozawa d2b2d76cce TestFairScheduler.testContinuousScheduling fails Intermittently. Contributed by Zhihai Xu. 
(cherry picked from commit 53959e69f7)
 2015-04-06 20:19:47 +09:00
Xuan 1eb6d74967 YARN-3435. AM container to be allocated Appattempt AM container shown as 
null. Contributed by Bibin A Chundatt

(cherry picked from commit 96d72118f5)
 2015-04-05 00:43:22 -07:00
Wangda Tan 405aead879 YARN-2901. Add errors and warning metrics page to RM, NM web UI. (Varun Vasudev via wangda) 
(cherry picked from commit bad070fe15)
 2015-04-02 17:23:44 -07:00
Vinod Kumar Vavilapalli d8e17c58bc YARN-3365. Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control. Contributed by Sidharta Seethana. 
(cherry picked from commit b21c72777a)
 2015-04-02 16:55:00 -07:00
Sandy Ryza 6286cfd711 YARN-3415. Non-AM containers can be counted towards amResourceUsage of a fairscheduler queue (Zhihai Xu via Sandy Ryza) 2015-04-02 13:57:53 -07:00
Zhijie Shen 6d0bebc912 YARN-3430. Made headroom data available on app attempt page of RM WebUI. Contributed by Xuan Gong. 
(cherry picked from commit 8366a36ad3)
 2015-04-01 13:49:00 -07:00
Wangda Tan 865be70b02 YARN-3425. NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed. (Bibin A Chundatt via wangda) 
(cherry picked from commit 492239424a)
 2015-04-01 10:15:47 -07:00
Xuan 5e4d457674 YARN-3248. Correct fix version from branch-2.7 to branch-2.8 in the change log. 
(cherry picked from commit 2e79f1c212)
 2015-04-01 04:33:32 -07:00
Xuan e26b6e55e9 YARN-3248. Display count of nodes blacklisted by apps in the web UI. 
Contributed by Varun Vasudev

(cherry picked from commit 4728bdfa15)
 2015-04-01 04:20:37 -07:00
Tsuyoshi Ozawa 80755edb7a YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. 
(cherry picked from commit c69ba81497)
 2015-04-01 17:44:40 +09:00
Karthik Kambatla 93b29d962d YARN-3428. Debug log resources to be localized for a container. (kasha) 
(cherry picked from commit 2daa478a64)
 2015-03-31 17:35:11 -07:00
Vinod Kumar Vavilapalli c980e34bc5 YARN-3304. Addendum patch. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. (Junping Du and Karthik Kambatla via vinodkv) 
(cherry picked from commit 7610925e90)
 2015-03-31 17:28:48 -07:00
Karthik Kambatla 85c420a4d4 YARN-3412. RM tests should use MockRM where possible. (kasha) 
(cherry picked from commit 79f7f2aabf)
 2015-03-31 09:22:16 -07:00
Tsuyoshi Ozawa 99b8255693 YARN-3258. FairScheduler: Need to add more logging to investigate allocations. Contributed by Anubhav Dhoot. 
(cherry picked from commit b5a22e9838)
 2015-03-31 17:43:07 +09:00
Wangda Tan cba4ed1678 YARN-2495. Allow admin specify labels from each NM (Distributed configuration for node label). (Naganarasimha G R via wangda) 
(cherry picked from commit 2a945d24f7)
 2015-03-30 12:05:54 -07:00
Vinod Kumar Vavilapalli c5bc48946d YARN-3304. Cleaning up ResourceCalculatorProcessTree APIs for public use and removing inconsistencies in the default values. Contributed by Junping Du and Karthik Kambatla. 
(cherry picked from commit c358368f51)
 2015-03-30 10:11:12 -07:00
Ravi Prakash b1b4951452 YARN-3288. Document and fix indentation in the DockerContainerExecutor code 
(cherry picked from commit e0ccea33c9)
 2015-03-28 08:01:26 -07:00
Robert Kanter 6f0e864d61 YARN-3400. [JDK 8] Build Failure due to unreported exceptions in RPCUtil (rkanter) 
(cherry picked from commit 87130bf6b2)
 2015-03-26 11:00:37 -07:00
Haohui Mai 17f17dac4e HADOOP-10670. Allow AuthenticationFilters to load secret from signature secret files. Contributed by Kai Zheng. 2015-03-25 11:13:04 -07:00
Karthik Kambatla 2158cd2bef YARN-3397. yarn rmadmin should skip -failover. (J.Andreina via kasha) 
(cherry picked from commit c906a1de72)
 2015-03-25 07:42:45 -07:00
Xuan 0c4fa872be YARN-2213. Change proxy-user cookie log in AmIpFilter to DEBUG. 
Contributed by Varun Saxena

(cherry picked from commit e556198e71)
 2015-03-25 04:51:55 -07:00
Wangda Tan 29629020c1 YARN-3383. AdminService should use warn instead of info to log exception when operation fails. (Li Lu via wangda) 
(cherry picked from commit 97a7277a2d)
 2015-03-24 10:39:45 -07:00
Harsh J e7702ac6a1 YARN-1880. Cleanup TestApplicationClientProtocolOnHA. Contributed by ozawa. 
(cherry picked from commit 8a9bbf548dad82598354c24fcd4dc80a715da7ac)
 2015-03-24 11:58:16 +05:30
Xuan cbdcdfad6d YARN-3393. Getting application(s) goes wrong when app finishes before 
starting the attempt. Contributed by Zhijie Shen

(cherry picked from commit 9fae455e26)
 2015-03-23 20:34:29 -07:00
Karthik Kambatla 4e0c48703e YARN-2868. FairScheduler: Metric for latency to allocate first container for an application. (Ray Chiang via kasha) 
(cherry picked from commit 972f1f1ab9)
 2015-03-23 14:10:23 -07:00
Karthik Kambatla 75591e4131 YARN-3241. FairScheduler handles invalid queue names inconsistently. (Zhihai Xu via kasha) 
(cherry picked from commit 2bc097cd14)
 2015-03-23 13:24:22 -07:00
cnauroth 342c525eaa YARN-3336. FileSystem memory leak in DelegationTokenRenewer. 
(cherry picked from commit 6ca1f12024)
 2015-03-23 10:46:06 -07:00
Tsuyoshi Ozawa cbacf20755 YARN-3384. TestLogAggregationService.verifyContainerLogs fails after YARN-2777. Contributed by Naganarasimha G R. 
(cherry picked from commit 82eda771e0)
 2015-03-24 00:25:52 +09:00
Jian He 9c494cedaf YARN-3345. Add non-exclusive node label API. Contributed by Wangda Tan 
(cherry picked from commit e1feb4ea1a)
 2015-03-20 19:05:14 -07:00
Junping Du 9aedb3d688 YARN-3350. YARN RackResolver spams logs with messages at info level. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 7f1e2f9969)
 2015-03-20 18:22:23 -07:00
Jian He a3a155a34e YARN-3356. Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track used-resources-by-label. Contributed by Wangda Tan 
(cherry picked from commit 586348e4cb)
 2015-03-20 13:54:46 -07:00
Junping Du f40f17489c YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 
(cherry picked from commit d81109e588)
 2015-03-20 13:42:31 -07:00
Wangda Tan 8e142d27cb YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 
(cherry picked from commit 6bc7710ec7)
 2015-03-20 10:42:51 -07:00
Xuan 7990e1d80d YARN-2777. Mark the end of individual log in aggregated log. Contributed 
by Varun Saxena

(cherry picked from commit 1a4b528691)
 2015-03-20 10:25:26 -07:00
Jian He 3f0c9e5fe3 YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 
(cherry picked from commit 4e886eb9cb)
 2015-03-19 22:27:57 -07:00
Devaraj K a888dac344 YARN-3357. Move TestFifoScheduler to FIFO package. Contributed by Rohith 
Sharmaks.

(cherry picked from commit 93d0f4acc8)
 2015-03-19 12:18:08 +05:30
Karthik Kambatla 05627dd96d YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 
(cherry picked from commit 20b49224eb)
 2015-03-18 16:33:57 -07:00
Jian He 15ebacf03a YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 
(cherry picked from commit 658097d6da)
 2015-03-17 21:31:20 -07:00
Tsuyoshi Ozawa b00b216a91 YARN-3205. FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. Contributed by Zhihai Xu. 
(cherry picked from commit 3bc72cc16d)
 2015-03-18 11:53:40 +09:00
Jian He ab34e6975b YARN-3305. Normalize AM resource request on app submission. Contributed by Rohith Sharmaks 
(cherry picked from commit 968425e9f7)
 2015-03-17 13:50:44 -07:00
Karthik Kambatla 1e77d92d62 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)" 
This reverts commit c2b185def8.

(cherry picked from commit 32b4330456)
 2015-03-17 12:31:44 -07:00
Jian He 1c601e492f YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan. 
(cherry picked from commit 487374b7fe)
 2015-03-17 10:25:07 -07:00
Devaraj K 895588b439 YARN-3197. Confusing log generated by CapacityScheduler. Contributed by 
Varun Saxena.

(cherry picked from commit 7179f94f9d)
 2015-03-17 15:59:19 +05:30
Ravi Prakash 9f227ad696 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 
(cherry picked from commit 56085203c4)
 2015-03-16 16:18:42 -07:00
Tsuyoshi Ozawa a20427d339 YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 
(cherry picked from commit 7522a643fa)
 2015-03-17 08:10:12 +09:00
Harsh J 2684b0b440 MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration, for debug purposes. Contributed by Plamen Jeliazkov. 
(cherry picked from commit 571f75b468a4412be2178466cb5d3888b05cd076)
 2015-03-17 01:02:21 +05:30
Zhijie Shen fbe811d904 YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. 2015-03-16 10:52:32 -07:00
Tsuyoshi Ozawa d6e3ef2506 YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 
(cherry picked from commit 3da9a97cfb)
 2015-03-16 23:20:17 +09:00
Xuan b6eb36dbdc YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't 
work in ATS / RM web ui. Contributed by Naganarasimha G R

(cherry picked from commit 3ff1ba2a7b)
 2015-03-15 20:27:44 -07:00
Jonathan Eagles 44aedad5dd YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 
(cherry picked from commit 8180e676ab)
 2015-03-13 12:05:21 -05:00
Vinod Kumar Vavilapalli 53aa3a4d1f YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 863079bb87)
 2015-03-12 13:33:42 -07:00
Xuan 498d775c9b YARN-3338. Exclude jline dependency from YARN. Contributed by Zhijie 
Shen

(cherry picked from commit 06ce1d9a6c)
 2015-03-12 10:25:55 -07:00
Zhijie Shen 426535007b YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 
(cherry picked from commit 85f6d67fa7)
 2015-03-11 19:39:46 -07:00
Tsuyoshi Ozawa d29cf296cb YARN-3295. Fix documentation nits found in markdown conversion. Contributed by Masatake Iwasaki. 
(cherry picked from commit 30c428a858)
 2015-03-11 15:02:45 +09:00
Jian He 2690c72522 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty 
(cherry picked from commit a380643d20)
 2015-03-10 11:16:51 -07:00
Jian He 4021451746 YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 
(cherry picked from commit c3003eba6f)
 2015-03-09 20:47:33 -07:00
Zhijie Shen a94d23762e YARN-3287. Made TimelineClient put methods do as the correct login context. Contributed by Daryn Sharp and Jonathan Eagles. 
(cherry picked from commit d6e05c5ee2)
 2015-03-09 13:55:51 -07:00
Tsuyoshi Ozawa b46f9e72db HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa) 2015-03-09 19:53:55 +09:00
Vinod Kumar Vavilapalli d9416317a9 Preparing for 2.8.0 development. Changing SNAPSHOT version to be 2.8 on branch-2. 2015-03-08 20:32:51 -07:00
Vinod Kumar Vavilapalli 8e6f8e9a04 Adding 2.8 section in CHANGES.txt 2015-03-08 20:26:57 -07:00
Junping Du b91e1b0feb YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. Contributed by Hitesh Shah 
(cherry picked from commit 7ce3c76353)
 2015-03-08 14:48:42 -07:00
Jason Lowe ef3d9bdf6b YARN-3275. CapacityScheduler: Preemption happening on non-preemptable queues. Contributed by Eric Payne 
(cherry picked from commit 27e8ea820f)
 2015-03-06 22:45:09 +00:00
Jian He 6cef2c16de YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu 
(cherry picked from commit 21101c01f2)
 2015-03-06 14:18:56 -08:00
Xuan 56c2050ab7 YARN-3227. Timeline renew delegation token fails when RM user's TGT is 
expired. Contributed by Zhijie Shen

(cherry picked from commit d1abc5d4fc)
 2015-03-06 13:33:18 -08:00
Jian He a5f3fb4dc1 YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 
(cherry picked from commit 95bfd087dc)
 2015-03-05 21:27:03 -08:00
Vinod Kumar Vavilapalli 9c8882546a YARN-2786. Created a yarn cluster CLI and seeded with one command for listing node-labels collection. Contributed by Wangda Tan. 2015-03-05 10:56:28 -08:00
Tsuyoshi Ozawa 5680b41597 Update CHANGES.txt for YARN-2616 to fix indentation. 
(cherry picked from commit 28b85a2116)
 2015-03-06 00:53:20 +09:00
Tsuyoshi Ozawa 6660c2f83b YARN-3249. Add a 'kill application' button to Resource Manager's Web UI. Contributed by Ryu Kobayashi. 
(cherry picked from commit 1b67209612)
 2015-03-05 19:56:59 +09:00
Karthik Kambatla 0d62e94887 YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client. (Zhihai Xu via kasha) 
(cherry picked from commit 8d88691d16)
 2015-03-04 19:49:21 -08:00
Karthik Kambatla 721d7b5741 YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) 
(cherry picked from commit 22426a1c9f)
 2015-03-04 18:07:33 -08:00
Karthik Kambatla 1749094fab YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha) 
(cherry picked from commit 53947f37c7)
 2015-03-04 17:39:39 -08:00
Jason Lowe ab397194c3 YARN-3131. YarnClientImpl should check FAILED and KILLED state in submitApplication. Contributed by Chang Li 
(cherry picked from commit 03cc22945e)
 2015-03-04 18:05:20 +00:00
Jian He aafe571398 YARN-3285. (Backport YARN-3168) Convert branch-2 .apt.vm files of YARN to markdown. Contributed by Masatake Iwasaki 2015-03-03 16:44:59 -08:00
Jian He 888a445638 YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with a different port. Contributed by Rohith Sharmaks 
(cherry picked from commit b2f1ec312e)
 2015-03-03 16:28:55 -08:00
Wangda Tan 0a502c665b YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 
(cherry picked from commit e17e5ba9d7)
 2015-03-03 11:49:45 -08:00
Vinod Kumar Vavilapalli 253c78548b YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 
(cherry picked from commit 14dd647c55)
 2015-03-02 17:54:01 -08:00
Wangda Tan 19a8e48230 YARN-3270. Fix node label expression not getting set in ApplicationSubmissionContext (Rohit Agarwal via wangda) 
(cherry picked from commit abac6eb9d5)
 2015-03-02 17:24:55 -08:00
Jian He 76b7283fa1 YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 
(cherry picked from commit 5d0bae550f)
 2015-03-02 14:43:12 -08:00
Wangda Tan 0b0be0056b YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 
(cherry picked from commit edcecedc1c)
 2015-02-27 16:14:35 -08:00
Tsuyoshi Ozawa 79f73f4613 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 
(cherry picked from commit 01a1621930)
 2015-02-28 00:57:01 +09:00
Konstantin V Shvachko 9e67f2cb0d YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:26:08 -08:00
Tsuyoshi Ozawa dc348f4894 YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 773b6515ac)
 2015-02-27 00:25:12 +09:00
Devaraj K f83d2e4410 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against 
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.

(cherry picked from commit 0d4296f0e0)
 2015-02-26 15:47:32 +05:30
Jason Lowe 257087417e YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params. Contributed by Jian He 
(cherry picked from commit 1a68fc4346)
 2015-02-25 16:15:35 +00:00
Tsuyoshi Ozawa 13e213c41a YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 
(cherry picked from commit 6cbd9f1113)
 2015-02-25 11:23:34 +09:00
Devaraj K b9e82e27f5 YARN-2797. Add -help to yarn logs and nodes CLI command. Contributed by 
Jagadesh Kiran N.

(cherry picked from commit b610c68d44)
 2015-02-24 12:08:50 +05:30
Xuan 0ca974d0b0 YARN-2797. TestWorkPreservingRMRestart should use 
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla

(cherry picked from commit fe7a302473)
 2015-02-21 19:19:08 -08:00
Xuan f8435c5d3d YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY. 
Contributed by zhihai xu

(cherry picked from commit e3d290244c)
 2015-02-21 16:20:33 -08:00
Xuan fefeba4ac8 YARN-3238. Connection timeouts to nodemanagers are retried at multiple 
levels. Contributed by Jason Lowe

(cherry picked from commit 92d67ace32)
 2015-02-21 16:07:28 -08:00
Xuan 9cbe9bcbaa YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by 
Rushabh S Shah

(cherry picked from commit f56c65bb3e)
 2015-02-20 14:03:46 -08:00
Wangda Tan a1963968d2 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 
(cherry picked from commit ce5bf927c3)
 2015-02-20 10:40:04 -08:00
Junping Du 4a87a61fe9 YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu 
(cherry picked from commit c33ae271c2)
 2015-02-20 09:44:31 -08:00