Commit Graph

1711 Commits

Author SHA1 Message Date
Junping Du d81109e588 YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path. Contributed by Xuan Gong 2015-03-20 13:41:22 -07:00
Wangda Tan 6bc7710ec7 YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die. (Brahma Reddy Battula via wangda) 2015-03-20 10:42:05 -07:00
Xuan 1a4b528691 YARN-2777. Mark the end of individual log in aggregated log. Contributed
by Varun Saxena
2015-03-20 10:24:13 -07:00
Jian He 4e886eb9cb YARN-3379. Fixed missing data in localityTable and ResourceRequests table in RM WebUI. Contributed by Xuan Gong 2015-03-19 22:27:21 -07:00
Devaraj K 93d0f4acc8 YARN-3357. Move TestFifoScheduler to FIFO package. Contributed by Rohith
Sharmaks.
2015-03-19 12:16:52 +05:30
Karthik Kambatla 20b49224eb YARN-3351. AppMaster tracking URL is broken in HA. (Anubhav Dhoot via kasha) 2015-03-18 16:30:33 -07:00
Jian He 658097d6da YARN-3273. Improve scheduler UI to facilitate scheduling analysis and debugging. Contributed Rohith Sharmaks 2015-03-17 21:30:23 -07:00
Tsuyoshi Ozawa 3bc72cc16d YARN-3205. FileSystemRMStateStore should disable FileSystem Cache to avoid get a Filesystem with an old configuration. Contributed by Zhihai Xu. 2015-03-18 11:53:19 +09:00
Jian He 968425e9f7 YARN-3305. Normalize AM resource request on app submission. Contributed by Rohith Sharmaks 2015-03-17 13:49:59 -07:00
Karthik Kambatla 32b4330456 Revert "YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha)"
This reverts commit c2b185def8.
2015-03-17 12:31:15 -07:00
Jian He 487374b7fe YARN-3243. CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. Contributed by Wangda Tan. 2015-03-17 10:24:23 -07:00
Devaraj K 7179f94f9d YARN-3197. Confusing log generated by CapacityScheduler. Contributed by
Varun Saxena.
2015-03-17 15:57:57 +05:30
Ravi Prakash 56085203c4 YARN-3339. TestDockerContainerExecutor should pull a single image and not the entire centos repository. (Ravindra Kumar Naik via raviprak) 2015-03-16 16:17:58 -07:00
Tsuyoshi Ozawa 7522a643fa YARN-3349. Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry. Contributed by Zhihai Xu. 2015-03-17 08:09:55 +09:00
Harsh J 587d8be17b MAPREDUCE-4414. Add main methods to JobConf and YarnConfiguration, for debug purposes. Contributed by Plamen Jeliazkov. 2015-03-17 01:03:08 +05:30
Zhijie Shen ed4e72a20b YARN-2854. Addendum patch to fix the minor issue in the timeline service documentation. 2015-03-16 10:39:10 -07:00
Tsuyoshi Ozawa 3da9a97cfb YARN-1453. [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments. Contributed by Akira AJISAKA, Andrew Purtell, and Allen Wittenauer. 2015-03-16 23:19:05 +09:00
Xuan 3ff1ba2a7b YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't
work in ATS / RM web ui. Contributed by Naganarasimha G R
2015-03-15 20:26:10 -07:00
Zhijie Shen 6fdef76cc3 YARN-2854. Updated the documentation of the timeline service and the generic history service. Contributed by Naganarasimha G R. 2015-03-13 14:00:09 -07:00
Jonathan Eagles 8180e676ab YARN-3267. Timelineserver applies the ACL rules after applying the limit on the number of records (Chang Li via jeagles) 2015-03-13 12:04:30 -05:00
Vinod Kumar Vavilapalli 863079bb87 YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating running logs of application when rolling is enabled. Contributed by Xuan Gong. 2015-03-12 13:32:29 -07:00
Zhijie Shen 85f6d67fa7 YARN-1884. Added nodeHttpAddress into ContainerReport and fixed the link to NM web page. Contributed by Xuan Gong. 2015-03-11 19:35:19 -07:00
Tsuyoshi Ozawa 30c428a858 YARN-3295. Fix documentation nits found in markdown conversion. Contributed by Masatake Iwasaki. 2015-03-11 15:02:24 +09:00
Allen Wittenauer a5cf985bf5 YARN-2280. Resource manager web service fields are not accessible (Krisztian Horvath via aw) 2015-03-10 16:32:20 -07:00
Jian He a380643d20 YARN-3187. Documentation of Capacity Scheduler Queue mapping based on user or group. Contributed by Gururaj Shetty 2015-03-10 11:15:57 -07:00
Allen Wittenauer 7711049837 HADOOP-11668. hadoop-daemons.sh bw compat broke with --slaves change (Vinayakumar B via aw) 2015-03-09 22:31:50 -07:00
Jian He c3003eba6f YARN-3300. Outstanding_resource_requests table should not be shown in AHS. Contributed by Xuan Gong 2015-03-09 20:46:48 -07:00
Zhijie Shen d6e05c5ee2 YARN-3287. Made TimelineClient put methods do as the correct login context. Contributed by Daryn Sharp and Jonathan Eagles. 2015-03-09 13:54:36 -07:00
Junping Du 7ce3c76353 YARN-3296. Mark ResourceCalculatorProcessTree class as Public for configurable resource monitoring. Contributed by Hitesh Shah 2015-03-08 14:47:35 -07:00
Jason Lowe 27e8ea820f YARN-3275. CapacityScheduler: Preemption happening on non-preemptable queues. Contributed by Eric Payne 2015-03-06 22:37:26 +00:00
Jian He 21101c01f2 YARN-2190. Added CPU and memory limit options to the default container executor for Windows containers. Contributed by Chuan Liu 2015-03-06 14:18:11 -08:00
Allen Wittenauer 01bfe6f05b HADOOP-11653. shellprofiles should require .sh extension (Brahma Reddy Battula via aw) 2015-03-06 13:54:11 -08:00
Xuan d1abc5d4fc YARN-3227. Timeline renew delegation token fails when RM user's TGT is
expired. Contributed by Zhijie Shen
2015-03-06 13:32:05 -08:00
Jian He 95bfd087dc YARN-1809. Synchronize RM and TimeLineServer Web-UIs. Contributed by Zhijie Shen and Xuan Gong 2015-03-05 21:20:09 -08:00
Vinod Kumar Vavilapalli 138c9cadee YARN-2786. Created a yarn cluster CLI and seeded with one command for listing node-labels collection. Contributed by Wangda Tan. 2015-03-05 10:54:34 -08:00
Tsuyoshi Ozawa 1b67209612 YARN-3249. Add a 'kill application' button to Resource Manager's Web UI. Contributed by Ryu Kobayashi. 2015-03-05 19:55:56 +09:00
Karthik Kambatla 8d88691d16 YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client. (Zhihai Xu via kasha) 2015-03-04 19:49:05 -08:00
Karthik Kambatla 22426a1c9f YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending jobs. (Siqi Li via kasha) 2015-03-04 18:06:58 -08:00
Karthik Kambatla 53947f37c7 YARN-3122. Metrics for container's actual CPU usage. (Anubhav Dhoot via kasha) 2015-03-04 17:33:30 -08:00
Jason Lowe 03cc22945e YARN-3131. YarnClientImpl should check FAILED and KILLED state in submitApplication. Contributed by Chang Li 2015-03-04 18:04:22 +00:00
Jian He b2f1ec312e YARN-3222. Fixed NPE on RMNodeImpl#ReconnectNodeTransition when a node is reconnected with a different port. Contributed by Rohith Sharmaks 2015-03-03 16:28:28 -08:00
Wangda Tan e17e5ba9d7 YARN-3272. Surface container locality info in RM web UI (Jian He via wangda) 2015-03-03 11:49:01 -08:00
Tsuyoshi Ozawa d1c6accb6f HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-03-03 14:17:52 +09:00
Vinod Kumar Vavilapalli 14dd647c55 YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's available resource-limit from the parent queue. Contributed by Wangda Tan. 2015-03-02 17:52:47 -08:00
Wangda Tan abac6eb9d5 YARN-3270. Fix node label expression not getting set in ApplicationSubmissionContext (Rohit Agarwal via wangda) 2015-03-02 17:21:19 -08:00
Jian He 5d0bae550f YARN-3281. Added RMStateStore to StateMachine visualization list. Contributed by Chengbing Liu 2015-03-02 14:39:49 -08:00
Allen Wittenauer 8472d72997 YARN-3199. Fair Scheduler documentation improvements (Rohit Agarwal via aw) 2015-02-28 11:36:15 -08:00
Allen Wittenauer 2e44b75f72 YARN-3168. Convert site documentation from apt to markdown (Gururaj Shetty via aw) 2015-02-27 20:39:44 -08:00
Wangda Tan edcecedc1c YARN-3262. Surface application outstanding resource requests table in RM web UI. (Jian He via wangda) 2015-02-27 16:13:32 -08:00
Tsuyoshi Ozawa 01a1621930 YARN-2820. Retry in FileSystemRMStateStore when FS's operations fail due to IOException. Contributed by Zhihai Xu. 2015-02-28 00:56:44 +09:00
Konstantin V Shvachko 8ca0d957c4 YARN-3255. RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options. Contributed by Konstantin Shvachko. 2015-02-26 17:12:19 -08:00
Tsuyoshi Ozawa 773b6515ac YARN-3217. Remove httpclient dependency from hadoop-yarn-server-web-proxy. Contributed by Brahma Reddy Battula. 2015-02-27 00:24:29 +09:00
Devaraj K 0d4296f0e0 YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against
all Schedulers even when using ParameterizedSchedulerTestBase. Contributed
by Anubhav Dhoot.
2015-02-26 15:45:41 +05:30
Jason Lowe 1a68fc4346 YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params. Contributed by Jian He 2015-02-25 16:14:34 +00:00
Tsuyoshi Ozawa 6cbd9f1113 YARN-3247. TestQueueMappings should use CapacityScheduler explicitly. Contributed by Zhihai Xu. 2015-02-25 10:38:11 +09:00
Allen Wittenauer d4ac6822e1 YARN-2980. Move health check script related functionality to hadoop-common (Varun Saxena via aw) 2015-02-24 11:25:26 -08:00
Tsuyoshi Ozawa 9cedad11d8 Revert "HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)"
This reverts commit 946456c6d8.

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/QuotaByStorageTypeEntry.java
2015-02-25 00:32:04 +09:00
Devaraj K b610c68d44 YARN-2797. Add -help to yarn logs and nodes CLI command. Contributed by
Jagadesh Kiran N.
2015-02-24 12:04:06 +05:30
Xuan fe7a302473 YARN-2797. TestWorkPreservingRMRestart should use
ParametrizedSchedulerTestBase. Contributed by Karthik Kambatla
2015-02-21 19:17:29 -08:00
Xuan e3d290244c YARN-3236. Cleanup RMAuthenticationFilter#AUTH_HANDLER_PROPERTY.
Contributed by zhihai xu
2015-02-21 16:18:40 -08:00
Xuan 92d67ace32 YARN-3238. Connection timeouts to nodemanagers are retried at multiple
levels. Contributed by Jason Lowe
2015-02-21 16:06:12 -08:00
Xuan f56c65bb3e YARN-3237. AppLogAggregatorImpl fails to log error cause. Contributed by
Rushabh S Shah
2015-02-20 14:02:40 -08:00
Wangda Tan ce5bf927c3 YARN-3230. Clarify application states on the web UI. (Jian He via wangda) 2015-02-20 10:39:28 -08:00
Junping Du c33ae271c2 YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90. Contributed by Zhihai Xu 2015-02-20 09:43:39 -08:00
Jason Lowe a64dd3d24b YARN-3194. RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node. Contributed by Rohith 2015-02-20 15:10:10 +00:00
Jian He c0d9b93953 YARN-933. Fixed InvalidStateTransitonException at FINAL_SAVING state in RMApp. Contributed by Rohith Sharmaks 2015-02-19 15:42:39 -08:00
Wangda Tan d49ae725d5 YARN-3076. Add API/Implementation to YarnClient to retrieve label-to-node mapping (Varun Saxena via wangda) 2015-02-19 11:00:57 -08:00
Tsuyoshi Ozawa 946456c6d8 HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa) 2015-02-19 13:06:53 +09:00
Tsuyoshi Ozawa b8a14efdf5 YARN-1615. Fix typos in delay scheduler's description. Contributed by Akira Ajisaka. 2015-02-19 10:24:07 +09:00
Jian He 1c03376300 YARN-1514. Utility to benchmark ZKRMStateStore#loadState for RM HA. Contributed by Tsuyoshi OZAWA 2015-02-18 16:06:55 -08:00
Jian He f5da5566d9 YARN-3132. RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated. Contributed by Wangda Tan 2015-02-18 11:51:51 -08:00
Xuan 57db50cbe3 YARN-3207. Secondary filter matches entites which do not have the key
being filtered for. Contributed by Zhijie Shen
2015-02-17 18:17:29 -08:00
Allen Wittenauer 72389c78db HADOOP-11575. Daemon log documentation is misleading (Naganarasimha G R via aw) 2015-02-17 07:00:00 -10:00
Tsuyoshi Ozawa 556386a070 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and Devaraj K. 2015-02-17 01:53:31 +09:00
Tsuyoshi Ozawa 3f32357c36 Revert "YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj."
This reverts commit 9aae81c934.
2015-02-17 01:52:22 +09:00
Tsuyoshi Ozawa 9aae81c934 YARN-1299. Improve a log message in AppSchedulingInfo by adding application id. Contributed by Ashutosh Jindal and devaraj. 2015-02-17 01:48:22 +09:00
Tsuyoshi Ozawa 447bd7b5a6 YARN-3203. Correct a log message in AuxServices. Contributed by Brahma Reddy Battula. 2015-02-16 23:55:58 +09:00
Junping Du ab0b958a52 YARN-2749. Fix some testcases from TestLogAggregationService fails in trunk. (Contributed by Xuan Gong) 2015-02-15 06:46:32 -08:00
cnauroth 6804d68901 YARN-2899. Run TestDockerContainerExecutorWithMocks on Linux only. Contributed by Ming Ma. 2015-02-13 21:58:54 -08:00
Jason Lowe 1a0f508b63 YARN-2847. Linux native container executor segfaults if default banned user detected. Contributed by Olaf Flebbe 2015-02-13 20:20:55 +00:00
Xuan 253035491c YARN-3164. RMAdmin command usage prints incorrect command name.
Contributed by Bibin A Chundatt
2015-02-12 21:39:16 -08:00
Tsuyoshi Ozawa 110cf6b9a3 YARN-3182. Cleanup switch statement in ApplicationMasterLauncher#handle(). Contributed by Ray Chiang. 2015-02-13 14:21:21 +09:00
Tsuyoshi Ozawa b0d81e05ab YARN-2994. Document work-preserving RM restart. Contributed by Jian He. 2015-02-13 13:08:13 +09:00
Allen Wittenauer 93b941c637 HADOOP-11565. Add --slaves shell option (aw) 2015-02-12 18:01:28 -08:00
Akira Ajisaka 6a49e58cb8 YARN-3191. Log object should be initialized with its own class. Contributed by Rohith. 2015-02-12 17:58:54 -08:00
Xuan 2586915bb3 YARN-3179. Update use of Iterator to Iterable in RMAdminCLI and
CommonNodeLabelsManager. Contributed by Ray Chiang
2015-02-12 17:31:45 -08:00
Xuan 99f6bd4f7a YARN-3158. Correct log messages in ResourceTrackerService. Contributed
by Varun Saxena
2015-02-12 16:42:15 -08:00
Jian He 18297e0972 YARN-3104. Fixed RM to not generate new AMRM tokens on every heartbeat between rolling and activation. Contributed by Jason Lowe 2015-02-12 16:02:45 -08:00
Jian He 18a594257e YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track capacities-by-label. Contributed by Wangda Tan 2015-02-12 14:58:09 -08:00
Allen Wittenauer f1070230d1 YARN-2796. deprecate sbin/yarn-daemon.sh (aw) 2015-02-12 13:45:04 -08:00
Karthik Kambatla c2b185def8 YARN-3181. FairScheduler: Fix up outdated findbugs issues. (kasha) 2015-02-12 13:44:47 -08:00
Junping Du 04f5ef18f7 YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager restart. (Contributed by Jason Lowe) 2015-02-12 11:46:47 -08:00
Xuan 83be450acc YARN-3147. Clean up RM web proxy code. Contributed by Steve Loughran 2015-02-12 10:57:55 -08:00
Tsuyoshi Ozawa 7bc492adab YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt. 2015-02-12 23:48:43 +09:00
Junping Du 46c7577b98 YARN-1580. Documentation error regarding container-allocation.expiry-interval-ms (Brahma Reddy Battula via junping_du) 2015-02-12 03:38:54 -08:00
Tsuyoshi Ozawa 4cbaa74f62 Revert "YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt."
This reverts commit 95a41bf35d.
2015-02-12 18:48:17 +09:00
Tsuyoshi Ozawa 95a41bf35d YARN-3157. Refactor the exception handling in ConverterUtils#to*Id. Contributed by Bibin A Chundatt. 2015-02-12 18:41:17 +09:00
Tsuyoshi Ozawa b3bcbaf277 YARN-1237. Description for yarn.nodemanager.aux-services in yarn-default.xml is misleading. Contributed by Brahma Reddy Battula. 2015-02-12 15:17:51 +09:00
Xuan 65c69e296e YARN-3151. On Failover tracking url wrong in application cli for KILLED
application. Contributed by Rohith
2015-02-11 21:19:48 -08:00
Jason Lowe b379972ab3 YARN-3074. Nodemanager dies when localizer runner tries to write to a full disk. Contributed by Varun Saxena 2015-02-11 16:33:43 +00:00