Commit Graph

3061 Commits

Author SHA1 Message Date
Vinod Kumar Vavilapalli d18050522c YARN-5339. Fixed "yarn logs" to fail when a file is passed to -out option instead of a directory. Contributed by Xuan Gong. 2016-07-13 14:52:30 -07:00
Naganarasimha af8f480c24 YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee 2016-07-14 01:43:37 +05:30
Vinod Kumar Vavilapalli eb47163234 YARN-5200. Enhanced "yarn logs" to be able to get a list of containers whose logs are aggregated via a "show_container_log_info" option. Contributed by Xuan Gong. 2016-07-13 10:54:15 -07:00
Varun Saxena 56142171b9 YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena) 2016-07-13 21:15:41 +05:30
Rohith Sharma K S d6d41e820a YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. Contributed by sandflee. 2016-07-13 19:12:35 +05:30
Varun Saxena 06c56ff79b YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena). 2016-07-13 07:55:34 +05:30
Robert Kanter bf6f4a3b98 YARN-4366. Fix Lint Warnings in YARN Common (templedf via rkanter) 2016-07-12 15:12:25 -07:00
Jason Lowe 10b704c594 YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee 2016-07-12 20:27:41 +00:00
Junping Du 7705812c7e New addendum to YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or RUNNING AM. 2016-07-12 08:29:49 -07:00
Varun Saxena a3ac1c1943 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 2016-07-12 20:41:52 +05:30
Jian He 819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Varun Saxena 0fd3980a1f YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena). 2016-07-10 21:28:52 +05:30
Sangjin Lee 6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Sangjin Lee c5dbde0cc4 Fixed the build after the rebase with trunk. 2016-07-10 08:46:05 -07:00
Sangjin Lee b313ce06b2 YARN-5174. [documentation] several updates/corrections to timeline service documentation (Sangjin Lee, Varun Saxena, Naganarasimha G R, and Li Lu via sjlee) 2016-07-10 08:46:04 -07:00
Vrushali 6d943038f6 Cleanup changes during rebase with trunk (Vrushali C) 2016-07-10 08:46:04 -07:00
Sangjin Lee 1a227744ac YARN-5070. upgrade HBase version for first merge (Vrushali C via sjlee) 2016-07-10 08:46:04 -07:00
Varun Saxena 1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan 32b033d57c YARN-5052 Update timeline service v2 documentation to capture information about filters (Varun Saxena via Vrushali C) 2016-07-10 08:46:03 -07:00
Varun Saxena 892b193bd7 YARN-5170. Eliminate singleton converters and static method access. (Joep Rottinghuis via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan 3832795e3c YARN-5210 NPE in Distributed Shell while publishing DS_CONTAINER_START event (Varun Saxena via Vrushali C) 2016-07-10 08:46:02 -07:00
Varun Saxena c65020075e YARN-5167. Escape occurences of encodedValues. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:02 -07:00
Varun Saxena 9c926cf432 YARN-5142. fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via Varun Saxena) 2016-07-10 08:46:02 -07:00
Li Lu 0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Sangjin Lee c81a2e1d19 YARN-5015. entire time series is returned for YARN container system metrics (CPU and memory) (Varun Saxena via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee 497d1c7f51 YARN-5111. YARN container system metrics are not aggregated to application (Naganarasimha G R via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee 7b8cfa5c2f YARN-5109. timestamps are stored unencoded causing parse errors (Varun Saxena via sjlee) 2016-07-10 08:46:01 -07:00
Li Lu d729e8211b YARN-5138. fix "no findbugs output file" error for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via gtcarrera9) 2016-07-10 08:46:00 -07:00
Sangjin Lee 702236129b YARN-5095. flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled (Varun Saxena via sjlee) 2016-07-10 08:46:00 -07:00
Sangjin Lee 831a3ffd6e YARN-5097. NPE in Separator.joinEncoded() (Vrushali C via sjlee) 2016-07-10 08:46:00 -07:00
Sangjin Lee a1b6d7456f YARN-5018. Online aggregation logic should not run immediately after collectors got started (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee 8c8183e515 YARN-5093. created time shows 0 in most REST output (Varun Saxena via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee d5dfee24b1 YARN-5050. Code cleanup for TestDistributedShell (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee 4a7011a45f YARN-5096 addendum. Turned another logging statement to debug. Contributed by Sangjin Lee. 2016-07-10 08:45:59 -07:00
Varun Saxena bcdb382600 YARN-5096. timelinereader has a lot of logging that's not useful. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:58 -07:00
Varun Saxena bc6f978cd3 YARN-5045. hbase unit tests fail due to dependency issues. (Sangjin Lee via varunsaxena) 2016-07-10 08:45:58 -07:00
Sangjin Lee 089caf49fe YARN-4447. Provide a mechanism to represent complex filters and parse them at the REST layer (Varun Saxena via sjlee) 2016-07-10 08:45:57 -07:00
Li Lu c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee 69dc561b61 YARN-4986. Add a check in the coprocessor for table to operated on (Vrushali C via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee 39cce4e629 YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee c2efdc415a YARN-3863. Support complex filters in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:55 -07:00
Varun Saxena a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee 84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Varun Saxena 6f6cc647d6 YARN-4712. CPU Usage Metric is not captured properly in YARN-2928. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:54 -07:00
Sangjin Lee 9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Varun Saxena d56dde490b YARN-4700. ATS storage has one extra record each time the RM got restarted. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:53 -07:00
Sangjin Lee 0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee 960af7d471 YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee 9cb1287e9b YARN-4446. Refactor reader API for better extensibility (Varun Saxena via sjlee) 2016-07-10 08:45:52 -07:00
Naganarasimha 06f0b50a28 YARN-4644. TestRMRestart fails and findbugs issue in YARN-2928 branch (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Li Lu 9d40d9d34c YARN-4224. Support fetching entities by UID and change the REST
interface to conform to current REST APIs' in YARN. (Varun Saxena via
gtcarrera9)
2016-07-10 08:45:52 -07:00
Naganarasimha 6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Varun Saxena f5380d850e YARN-4622. TestDistributedShell fails for v2 test cases after modifications for 1.5. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:51 -07:00
Li Lu 34f02f07d5 Rebase to latest trunk 2016-07-10 08:45:51 -07:00
Li Lu 88f0294114 YARN-4200. Refactor reader classes in storage to nest under hbase
specific package name. Contributed by Li Lu.
2016-07-10 08:45:51 -07:00
Sangjin Lee cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Varun Saxena 829cceebc0 YARN-3586. RM to only get back addresses of Collectors that NM needs to know.
(Junping Du via Varun Saxena).
2016-07-10 08:45:50 -07:00
Varun Saxena eb0ac8efb1 YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:50 -07:00
Li Lu 8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase.
Contributed by Zhan Zhang.
2016-07-10 08:45:49 -07:00
Varun Saxena c4d7bbda5c YARN-4460. [Bug fix] RM fails to start when SMP is enabled. (Li Lu via Varun Saxena) 2016-07-10 08:45:49 -07:00
Xuan 2e2dbf59d1 YARN-4392. ApplicationCreatedEvent event time resets after RM
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
2016-07-10 08:45:49 -07:00
Li Lu b7d484aead YARN-4450. TestTimelineAuthenticationFilter and
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
2016-07-10 08:45:49 -07:00
Li Lu 89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no
impact when it's turned off. Contributed by Sangjin Lee.
2016-07-10 08:45:48 -07:00
Junping Du cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong.
(cherry picked from commit f910e4f639)
2016-07-10 08:45:48 -07:00
Sangjin Lee 1f710484e5 YARN-3862. Support for fetching specific configs and metrics based on prefixes (Varun Saxena via sjlee) 2016-07-10 08:45:48 -07:00
Sangjin Lee ae72f1dc77 YARN-4297. TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee 51254a6b51 YARN-4053. Change the way metric values are stored in HBase Storage (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee 09649005ca YARN-4221. Store user in app to flow table (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee 10ec5586fb YARN-4129. Refactor the SystemMetricPublisher in RM to better support newer events (Naganarasimha G R via sjlee) 2016-07-10 08:45:46 -07:00
Sangjin Lee e3e857866d YARN-4179. [reader implementation] support flow activity queries based on time (Varun Saxena via sjlee) 2016-07-10 08:45:46 -07:00
Li Lu d014f2ffd2 YARN-4237 Support additional queries for ATSv2 Web UI. Contributed by
Varun Saxena.
2016-07-10 08:45:46 -07:00
Sangjin Lee b51d0fef56 YARN-4178. [storage implementation] app id as string in row keys can cause incorrect ordering (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Sangjin Lee 0f44b5508d YARN-3864. Implement support for querying single app and all apps for a flow run (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan 708fa8b1ae YARN-4210. HBase reader throws NPE if Get returns no rows (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan da2b7bd08e YARN-4203. Add request/response logging & timing for each REST endpoint call (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan d95dc89a02 YARN-4075 [reader REST API] implement support for querying for flows and flow runs (Varun Saxena via vrushali) 2016-07-10 08:45:44 -07:00
Vrushali 10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Sangjin Lee a68e383921 YARN-3901. Populate flow run data in the flow_run & flow activity tables (Vrushali C via sjlee) 2016-07-10 08:45:44 -07:00
Sangjin Lee ee081018e4 YARN-4102. Add a "skip existing table" mode for timeline schema creator (Li Lu via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee 8d9476ec5f YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G R via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee 451c13b1de YARN-3814. REST API implementation for getting raw entities in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee c7224a285c YARN-4064. build is broken at TestHBaseTimelineWriterImpl.java (sjlee) 2016-07-10 08:45:43 -07:00
Junping Du 7a41b5501e YARN-4025. Deal with byte representations of Longs in writer code. Contributed by Sangjin Lee and Vrushali C. 2016-07-10 08:45:42 -07:00
Junping Du 477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee 102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Zhijie Shen 5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee.
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
2016-07-10 08:45:41 -07:00
Junping Du 00e85e7a2b YARN-3906. Split the application table from the entity table. Contributed by Sangjin Lee.
(cherry picked from commit bcd755eba9466ce277d3c14192c31da6462c4ab3)
2016-07-10 08:45:41 -07:00
Sangjin Lee 9e5155be36 YARN-3049. [Storage Implementation] Implement storage reader interface to fetch raw data from HBase backend (Zhijie Shen via sjlee)
(cherry picked from commit 07433c2ad52df9e844dbd90020c277d3df844dcd)
2016-07-10 08:45:41 -07:00
Zhijie Shen 9422d9b50d YARN-3984. Adjusted the event column key schema and avoided missing empty event. Contributed by Vrushali C.
(cherry picked from commit 895ccfa1ab9e701f2908586e323249f670fe5544)
2016-07-10 08:45:40 -07:00
Zhijie Shen 57e2498cd4 YARN-3993. Changed to use the AM flag in ContainerContext determine AM container in TestPerNodeTimelineCollectorsAuxService. Contributed by Sunil G.
(cherry picked from commit 9e48f9ff2ce08f3dcdd8d60bacb697664b92196f)
2016-07-10 08:45:40 -07:00
Zhijie Shen a9fab9b644 YARN-3908. Fixed bugs in HBaseTimelineWriterImpl. Contributed by Vrushali C and Sangjin Lee.
(cherry picked from commit df0ec473a84871b0effd7ca6faac776210d7df09)
2016-07-10 08:45:40 -07:00
Junping Du 8603736ef2 YARN-3949. Ensure timely flush of timeline writes. Contributed by Sangjin Lee.
(cherry picked from commit 967bef7e0396d857913caa2574afb103a5f0b81b)
2016-07-10 08:45:39 -07:00
Sangjin Lee 57d8dc2fb7 YARN-3836. add equals and hashCode to TimelineEntity and other classes in the data model (Li Lu via sjlee)
(cherry picked from commit 2d4a8f4563c06339717ca9410b2794754603fba3)
2016-07-10 08:45:39 -07:00
Sangjin Lee e27642abf4 YARN-3047. [Data Serving] Set up ATS reader with basic request serving structure and lifecycle (Varun Saxena via sjlee)
(cherry picked from commit 4c5f88fb0f04b7919738d07598b0f006a9ff91f2)
2016-07-10 08:45:39 -07:00
Zhijie Shen 2d59bc4458 YARN-3051. Created storage oriented reader interface for fetching raw entity data and made the filesystem based implementation. Contributed by Varun Saxena.
(cherry picked from commit 499ce52c7b645ec0b1cc8ac62dc9a3127b987a20)
2016-07-10 08:45:38 -07:00
Sangjin Lee 22e7ae5771 YARN-3792. Test case failures in TestDistributedShell and some issue fixes related to ATSV2 (Naganarasimha G R via sjlee)
(cherry picked from commit 84f37f1c7eefec6d139cbf091c50d6c06f734323)
2016-07-10 08:45:38 -07:00
Sangjin Lee 92d90c3a24 YARN-3706. Generalize native HBase writer for additional tables (Joep Rottinghuis via sjlee)
(cherry picked from commit 9137aeae0dec83f9eff40d12cae712dfd508c0c5)
2016-07-10 08:45:38 -07:00
Zhijie Shen f3c661e8dd YARN-3044. Made RM write app, attempt and optional container lifecycle events to timeline service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:37 -07:00
Zhijie Shen 42d7864b84 YARN-3276. Code cleanup for timeline service API records. Contributed by Junping Du.
(cherry picked from commit d88f30ba5359f59fb71b93a55e1c1d9a1c0dff8e)
2016-07-10 08:45:37 -07:00
Sangjin Lee f668212529 YARN-3721. build is broken on YARN-2928 branch due to possible dependency cycle (Li Lu via sjlee)
(cherry picked from commit a9738ceb17b50cce8844fd42bb800c7f83f15caf)
2016-07-10 08:45:37 -07:00
Sangjin Lee db891f51d0 YARN-3726. Fix TestHBaseTimelineWriterImpl unit test failure by fixing its test data (Vrushali C via sjlee)
(cherry picked from commit 4c0b6d73914f2e249795deb292f508177ea54884)
2016-07-10 08:45:36 -07:00
Sangjin Lee 5a4278ccbd YARN-3411. [Storage implementation] explore the native HBase write schema for storage (Vrushali C via sjlee)
(cherry picked from commit 7a3068854d27eadae1c57545988f5b2029bf119a)
2016-07-10 08:45:36 -07:00
Junping Du d275677e24 YARN-3634. TestMRTimelineEventHandling and TestApplication are broken. Contributed by Sangjin Lee.
(cherry picked from commit b059dd4882fd759e4762cc11c019be4b68fb74c1)
2016-07-10 08:45:36 -07:00
Zhijie Shen 51d092faef YARN-3529. Added mini HBase cluster and Phoenix support to timeline service v2 unit tests. Contributed by Li Lu. 2016-07-10 08:45:36 -07:00
Zhijie Shen 41fb5c7381 YARN-3134. Implemented Phoenix timeline writer to access HBase backend. Contributed by Li Lu.
(cherry picked from commit b3b791be466be79e4e964ad068f7a6ec701e22e1)
2016-07-10 08:45:35 -07:00
Sangjin Lee dc1f306fdc YARN-3562. unit tests failures and issues found from findbug from earlier ATS checkins (Naganarasimha G R via sjlee) 2016-07-10 08:45:35 -07:00
Sangjin Lee fc8485d893 YARN-3551. Consolidate data model change according to the backend implementation (Zhijie Shen via sale)
(cherry picked from commit 557a3950bddc837469244835f5577899080115d8)
2016-07-10 08:45:35 -07:00
Junping Du 2bdefbc4a0 YARN-3431. Sub resources of timeline entity needs to be passed to a separate endpoint. Contributed By Zhijie Shen.
(cherry picked from commit fa5cc75245a6dba549620a8b26c7b4a8aed9838e)
2016-07-10 08:45:34 -07:00
Sangjin Lee 11e8905d8d YARN-3390. Reuse TimelineCollectorManager for RM (Zhijie Shen via sjlee)
(cherry picked from commit 58221188811e0f61d842dac89e1f4ad4fd8aa182)
2016-07-10 08:45:33 -07:00
Junping Du 47f35a30bb YARN-3391. Clearly define flow ID/ flow run / flow version in API and storage. Contributed by Zhijie Shen
(cherry picked from commit 68c6232f8423e55b4d152ef3d1d66aeb2d6a555e)
2016-07-10 08:45:33 -07:00
Zhijie Shen 5712b8f9fd YARN-3334. NM uses timeline client to publish container metrics to new timeline service. Contributed by Junping Du. 2016-07-10 08:45:33 -07:00
Junping Du 42e49399ce YARN-3374. Collector's web server should randomly bind an available port. Contributed by Zhijie Shen
(cherry picked from commit 3aa898e734a1e4368ddf1d0bbd31f9b4de53ceba)
2016-07-10 08:45:32 -07:00
Junping Du d67c9bdb4d YARN-3040. Make putEntities operation be aware of the app's context. Contributed by Zhijie Shen 2016-07-10 08:45:32 -07:00
Junping Du 5e3d9a477b YARN-3034. Implement RM starting its timeline collector. Contributed by Naganarasimha G R 2016-07-10 08:45:32 -07:00
Zhijie Shen 19c1132146 YARN-3377. Fixed test failure in TestTimelineServiceClientIntegration. Contributed by Sangjin Lee.
(cherry picked from commit 04de2ceccd02edb00bd671a63e04855c132e9735)
2016-07-10 08:45:31 -07:00
Junping Du 2188a07e5b YARN-3333. Rename TimelineAggregator etc. to TimelineCollector. Contributed by Sangjin Lee 2016-07-10 08:45:31 -07:00
Zhijie Shen 9b56364080 YARN-3039. Implemented the app-level timeline aggregator discovery service. Contributed by Junping Du. 2016-07-10 08:45:31 -07:00
Zhijie Shen f0e752c14b YARN-3264. Created backing storage write interface and a POC only FS based storage implementation. Contributed by Vrushali C.
(cherry picked from commit 821b68d05d246fd57d7b7286eb2ccc075ed1eae8)
2016-07-10 08:45:30 -07:00
Zhijie Shen bf54d32750 YARN-3210. Refactored timeline aggregator according to new code organization proposed in YARN-3166. Contributed by Li Lu.
(cherry picked from commit d3ff7f06cbc66d3a23c2551e7d4c752689f46afe)
2016-07-10 08:45:30 -07:00
Zhijie Shen d45ff878c4 YARN-3125. Made the distributed shell use timeline service next gen and add an integration test for it. Contributed by Junping Du and Li Lu.
(cherry picked from commit bf08f7f0ed4900ce52f98137297dd1a47ba2a536)
2016-07-10 08:45:30 -07:00
Zhijie Shen 9d57c9c015 YARN-3087. Made the REST server of per-node aggregator work alone in NM daemon. Conntributed by Li Lu.
(cherry picked from commit 41a08ad404d4278fe598d6c222b2ae0e84bae0df)
2016-07-10 08:45:29 -07:00
Zhijie Shen 309592b9e5 YARN-3240. Added ".keep" file which was missed before.
(cherry picked from commit 6b4dc96414b3f15537670c67a0dccd048862f4f4)
2016-07-10 08:45:29 -07:00
Junping Du 4f0c7eaff3 YARN-3240. Implement client API to put generic entities. Contributed by Zhijie Shen
(cherry picked from commit 4487da249f448d5c67b712cd0aa723e764eed77d)
2016-07-10 08:45:29 -07:00
Zhijie Shen c77f86bf63 YARN-3041. Added the overall data model of timeline service next gen. Contributed by Zhijie Shen.
(cherry picked from commit fdff5d2625c129c20c44f90a0517b28cee1522bd)
2016-07-10 08:45:29 -07:00
Zhijie Shen 7c8abec0a8 YARN-3030. Set up TS aggregator with basic request serving structure and lifecycle. Contributed by Sangjin Lee.
(cherry picked from commit f26941b39028ac30c77547e2be2d657bb5bf044a)
2016-07-10 08:45:28 -07:00
Zhijie Shen 0d2da08f35 YARN-3063. Bootstrapping TimelineServer next generation module. Contributed by Zhijie Shen.
(cherry picked from commit fe7d0e7c8fc88e0721b18e1531b0899d9dbed296)
2016-07-10 08:45:28 -07:00
Junping Du 9bdb5bebea Fix OOM issue in DistributedShell. Contributed by Jian He. 2016-07-09 05:09:53 -07:00
Varun Saxena c04c5ec501 YARN-5318. Fix intermittent test failure of TestRMAdminService#testRefreshNodesResourceWithFileSystemBasedConfigurationProvider. (Jun Gong via Varun Saxena). 2016-07-09 01:13:18 +05:30
Varun Saxena 5252562edf YARN-5297. Avoid printing a stack trace when recovering an app after the RM restarts. (Junping Du via Varun Saxena). 2016-07-09 00:09:25 +05:30
Junping Du 30ee57ceb1 YARN-4939. The decommissioning Node should keep alive during NM restart. Contributed by sandflee. 2016-07-08 04:14:53 -07:00
Vinod Kumar Vavilapalli 673e5e02fe YARN-5314. Fixed a ConcurrentModificationException in ATS v1.5 EntityGroupFSTimelineStore. Contributed by Li Lu. 2016-07-07 22:37:29 -07:00
Allen Wittenauer a0035661c1 HADOOP-13349. HADOOP_CLASSPATH vs HADOOP_USER_CLASSPATH (aw) 2016-07-07 14:15:34 -07:00
Jian He 8a9d293dd6 YARN-5233. Support for specifying a path for ATS plugin jars. Contributed by Li Lu 2016-07-06 16:44:46 -07:00
Vinod Kumar Vavilapalli 4c9e1aeb94 YARN-5224. Added new web-services /containers/{containerid}/logs & /containers/{containerid}/logs/{filename} and using them in "yarn logs" CLI to get logs of finished containers of a running application. Contributed by Xuan Gong. 2016-07-06 14:10:54 -07:00
Jian He d169f5052f YARN-5227. Yarn logs command: no need to specify applicationId when
specifying containerId. Contributed by Gergely Novák
2016-07-06 10:43:27 -07:00
Wangda Tan 04f6ebb66a YARN-5294. Pass remote ip address down to YarnAuthorizationProvider. (Jian He via wangda) 2016-07-06 10:36:48 -07:00
Jian He d792a90206 YARN-5296. NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl. Contributed by Junping Du 2016-07-05 18:05:16 -07:00
Vinod Kumar Vavilapalli ce9c006430 YARN-5214. Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks. Contributed by Junping Du. 2016-07-05 16:07:28 -07:00
Varun Saxena 8e672e3c71 YARN-5286. Add RPC port info in RM web service's response when getting app status. (Jun Gong via Varun Saxena). 2016-07-05 22:56:07 +05:30
Ray Chiang 0a5def155e YARN-4568. Fix message when NodeManager runs into errors initializing the recovery directory. (rchiang) 2016-07-01 15:53:46 -07:00
Jian He c35a5a7a8d YARN-5023. TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry fails. Contributed by sandflee 2016-07-01 14:29:03 -07:00
Varun Saxena 8ade81228e YARN-5282. Fix typos in CapacityScheduler documentation. (Ray Chiang via Varun Saxena). 2016-07-01 11:38:32 +05:30
Varun Saxena abe7fc22c1 YARN-5182. MockNodes.newNodes creates one more node per rack than requested. (Karthik Kambatla via Varun Saxena). 2016-06-30 00:13:28 +05:30
Rohith Sharma K S 26b5e6116f YARN-5262. Optimize sending RMNodeFinishedContainersPulledByAMEvent for every AM heartbeat. 2016-06-29 10:08:30 +05:30
Allen Wittenauer 23c3ff85a9 HADOOP-13209. replace slaves with workers (John Smith via aw) 2016-06-28 05:53:03 -07:00
Akira Ajisaka a8a48c9125 YARN-5278. Remove unused argument in TestRMWebServicesForCSWithPartitions#setupQueueConfiguration. Contributed by Tao Jie. 2016-06-23 14:28:12 +09:00
Xuan 6ab5aa1c1f YARN-5266. Wrong exit code while trying to get app logs using regex via CLI. Contributed by Xuan Gong 2016-06-22 21:48:49 -07:00
Arun Suresh 99e5dd68d0 YARN-5171. Extend DistributedSchedulerProtocol to notify RM of containers allocated by the Node. (Inigo Goiri via asuresh) 2016-06-22 19:04:54 -07:00
Tsuyoshi Ozawa 5d58858bb6 HADOOP-9613. [JDK8] Update jersey version to latest 1.x release. 2016-06-21 08:05:32 +09:00
Sangjin Lee 5107a967fa YARN-4958. The file localization process should allow for wildcards to reduce the application footprint in the state store (Daniel Templeton via sjlee) 2016-06-20 09:56:53 -07:00
Junping Du d0162f2040 YARN-5246. NMWebAppFilter web redirects drop query parameters. Contributed by Varun Vasudev. 2016-06-19 17:44:54 -07:00
Karthik Kambatla 20f2799938 YARN-5077. Fix FSLeafQueue#getFairShare() for queues with zero fairshare. (Yufei Gu via kasha) 2016-06-17 22:24:42 -07:00
Karthik Kambatla fbbe0bb627 YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit. Addendum to fix javac warning. (Arun Suresh via kasha) 2016-06-17 22:12:50 -07:00
Junping Du c35fa4a0e5 YARN-5251. Yarn CLI to obtain App logs for last 'n' bytes fails. Contributed by Xuan Gong. 2016-06-17 08:24:24 -07:00
Junping Du e14ee0d3b5 YARN-5083. YARN CLI for AM logs does not give any error message if entered invalid am value. Contributed by Jian He. 2016-06-16 08:55:56 -07:00
Junping Du 5dfc38ff57 YARN-5237. Fix missing log files issue in rolling log aggregation. Contributed by Xuan Gong. 2016-06-15 16:17:54 -07:00
Varun Vasudev 25064fb2fb YARN-5223. Container line in yarn logs output for a live application should include the hostname for the container. Contributed by Xuan Gong. 2016-06-15 12:42:44 +05:30
Varun Vasudev d6284428e8 YARN-5122. "yarn logs" for running containers should print an explicit footer saying that the log may be incomplete. Contributed by Jian He. 2016-06-15 12:35:17 +05:30
Wangda Tan c77a1095dc YARN-1942. Deprecate toString/fromString methods from ConverterUtils and move them to records classes like ContainerId/ApplicationId, etc. (wangda) 2016-06-14 15:06:38 -07:00
Wangda Tan eec835ec17 YARN-4887. Add allocation request ID to AM-RM protocol for identifying resource-requests explicitly. (Subru Krishnan via wangda) 2016-06-13 21:57:33 -07:00
Rohith Sharma K S 28b66ae919 YARN-4989. TestWorkPreservingRMRestart#testCapacitySchedulerRecovery fails intermittently. Contributed by Ajith S. 2016-06-13 11:09:32 +05:30
Arun Suresh 7dae2b3bc4 YARN-5212. Run existing ContainerManager tests using QueuingContainerManagerImpl. (Konstantinos Karanasos via asuresh) 2016-06-12 10:05:37 -07:00
Arun Suresh 5143277958 YARN-5124. Modify AMRMClient to set the ExecutionType in the ResourceRequest. (asuresh) 2016-06-12 09:42:38 -07:00
Arun Suresh 5279af7cd4 YARN-5082. Limit ContainerId increase in fair scheduler if the num of node app reserved reached the limit (sandflee via asuresh) 2016-06-10 22:33:42 -07:00
Rohith Sharma K S e0f4620cc7 YARN-5197. RM leaks containers if running container disappears from node update. Contributed by Jason Lowe. 2016-06-11 10:22:27 +05:30
Wangda Tan 03fc6b1bb0 YARN-3426. Add jdiff support to YARN. (vinodkv via wangda) 2016-06-10 09:51:09 -07:00
Wangda Tan 244506f9c8 YARN-5208. Run TestAMRMClient TestNMClient TestYarnClient TestClientRMTokens TestAMAuthorization tests with hadoop.security.token.service.use_ip enabled. (Rohith Sharma K S via wangda) 2016-06-10 09:34:32 -07:00
Vinod Kumar Vavilapalli 9378d9428f YARN-5191. Renamed the newly added “download=true” option for getting logs via NMWebServices and AHSWebServices to be a better "format" option. (Xuan Gong via vinodkv) 2016-06-09 12:30:58 -07:00
Naganarasimha 1500a0a300 YARN-4308. ContainersAggregated CPU resource utilization reports negative usage in first few heartbeats. Contributed by Sunil G 2016-06-09 05:41:09 +05:30
Vinod Kumar Vavilapalli 5a43583c0b YARN-5080. Addendum fix to the original patch to fix YARN logs CLI. Contributed by Xuan Gong 2016-06-08 09:49:55 -07:00
Arun Suresh 3344ba70e0 YARN-5204. Properly report status of killed/stopped queued containers. (Konstantinos Karanasos via asuresh) 2016-06-08 08:31:32 -07:00
Jason Lowe 8c8a377cac YARN-5206. RegistrySecurity includes id:pass in exception text if considered invalid. Contributed by Steve Loughran 2016-06-08 14:11:25 +00:00
Arun Suresh 76f0800c21 YARN-5176. More test cases for queuing of containers at the NM. (Konstantinos Karanasos via asuresh) 2016-06-07 17:16:18 -07:00
Xuan 58be55b6e0 YARN-5199. Close LogReader in in AHSWebServices#getStreamingOutput and
FileInputStream in NMWebServices#getLogs. Contributed by Xuan Gong
2016-06-07 16:07:02 -07:00
Arun Suresh 8554aee1be Addendum patch for YARN-5180 updating findbugs-exclude.xml 2016-06-07 15:59:43 -07:00
Wangda Tan 620325e816 YARN-4837. User facing aspects of 'AM blacklisting' feature need fixing. (vinodkv via wangda) 2016-06-07 15:06:42 -07:00
Rohith Sharma K S bddea5fe5f YARN-5118. Tests fails with localizer port bind exception. Contributed by Brahma Reddy Battula. 2016-06-07 11:20:15 +05:30
Arun Suresh 3a154f75ed YARN-4525. Fix bug in RLESparseResourceAllocation.getRangeOverlapping(). (Ishai Menache and Carlo Curino via asuresh) 2016-06-06 21:18:32 -07:00
Arun Suresh 7a9b7372a1 YARN-5185. StageAllocaterGreedyRLE: Fix NPE in corner case. (Carlo Curino via asuresh) 2016-06-06 21:06:52 -07:00
Ming Ma 4a1cedc010 MAPREDUCE-5044. Have AM trigger jstack on task attempts that timeout before killing them. (Eric Payne and Gera Shegalov via mingma) 2016-06-06 14:30:51 -07:00
Arun Suresh db54670e83 YARN-5165. Fix NoOvercommitPolicy to take advantage of RLE representation of plan. (Carlo Curino via asuresh) 2016-06-03 14:49:32 -07:00
Vinod Kumar Vavilapalli f10ebc67f5 YARN-5098. Fixed ResourceManager's DelegationTokenRenewer to replace expiring system-tokens if RM stops and only restarts after a long time. Contributed by Jian He. 2016-06-03 13:00:07 -07:00
Jian He 99cc439e29 YARN-5190. Registering/unregistering container metrics in ContainerMonitorImpl and ContainerImpl causing uncaught exception in ContainerMonitorImpl. Contributed by Junping Du 2016-06-03 11:10:42 -07:00
Jian He 097baaaeba YARN-1815. Work preserving recovery of Unmanged AMs. Contributed by Subru Krishnan 2016-06-03 10:49:30 -07:00
Arun Suresh dc26601d8f YARN-5180. Allow ResourceRequest to specify an enforceExecutionType flag. (asuresh) 2016-06-02 09:01:02 -07:00
Xuan 0bc05e40fa YARN-5088. Improve "yarn log" command-line to read the last K bytes for the log files. Contributed by Xuan Gong 2016-06-01 13:44:21 -07:00
Varun Vasudev 4e1f56e111 YARN-5141. Get Container logs for the Running application from Yarn Logs CommandLine. Contributed by Xuan Gong. 2016-05-29 21:53:43 +05:30
Varun Vasudev 86fb58b7dc YARN-4007. Add support for different network setups when launching the docker container. Contributed by Sidharta Seethana. 2016-05-29 21:32:51 +05:30
Varun Vasudev 42f90ab885 YARN-4844. Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource. Contributed by Wangda Tan. 2016-05-29 21:24:16 +05:30
Karthik Kambatla f5ff05cc8a YARN-5132. Exclude generated protobuf sources from YARN Javadoc build. (Subru Krishnan via kasha) 2016-05-28 08:09:46 -07:00
Arun Suresh 4fc09a897b YARN-5117. QueuingContainerManager does not start GUARANTEED Container even if Resources are available. (Konstantinos Karanasos via asuresh) 2016-05-27 17:49:15 -07:00
Arun Suresh aa975bc781 YARN-5127. Expose ExecutionType in Container api record. (Hitesh Sharma via asuresh) 2016-05-27 14:06:32 -07:00
Akira Ajisaka e4022debf7 YARN-5166. javadoc:javadoc goal fails on hadoop-yarn-client. Contributed by Andras Bokor. 2016-05-27 23:15:32 +09:00
Junping Du 705286ccae YARN-4987. Read cache concurrency issue between read and evict in EntityGroupFS timeline store. Contributed by Li Lu. 2016-05-27 06:58:32 -07:00
Xuan bde819abbb YARN-5089. Improve "yarn log" command-line "logFiles" option to support
regex. Contributed by Xuan Gong
2016-05-26 23:49:20 -07:00
Kai Zheng 916140604f HADOOP-12911. Upgrade Hadoop MiniKDC with Kerby. Contributed by Jiajia Li 2016-05-28 14:23:39 +08:00
Rohith Sharma K S 0a544f8a3e YARN-5005. TestRMWebServices#testDumpingSchedulerLogs fails randomly. Contributed by Bibin A Chundatt. 2016-05-27 10:44:35 +05:30
Eric Payne 1b0cfc84e7 YARN-5116. Failed to execute "yarn application". Contributed by Jun Gong 2016-05-26 23:43:08 +00:00
Arun Suresh 5b41b288d0 YARN-5162. Fix Exceptions thrown during in registerAM call when Distributed Scheduling is Enabled (Hitesh Sharma via asuresh) 2016-05-26 14:56:37 -07:00
Karthik Kambatla 04ded558b0 YARN-5035. FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on. (kasha) 2016-05-26 14:41:07 -07:00
Karthik Kambatla 4f513a4a8e YARN-4866. FairScheduler: AMs can consume all vcores leading to a livelock when using FAIR policy. (Yufei Gu via kasha) 2016-05-25 22:13:27 -07:00
Carlo Curino 013532a95e YARN-4957. Add getNewReservation in ApplicationClientProtocol (Sean Po via curino) 2016-05-25 16:55:49 -07:00
Jason Lowe 1ba31fe9e9 YARN-4459. container-executor should only kill process groups. Contributed by Jun Gong 2016-05-25 21:24:55 +00:00
Karthik Kambatla 77d5ce9069 YARN-3344. Fix warning - procfs stat file is not in the expected format. (Ravindra Kumar Naik and Akira Ajisaka via kasha) 2016-05-25 10:50:54 -07:00
Robert Kanter e07519b8db YARN-4766. NM should not aggregate logs older than the retention policy (haibochen via rkanter) 2016-05-25 10:25:44 -07:00
Varun Vasudev 9a31e5dfef YARN-5115. Avoid setting CONTENT-DISPOSITION header in the container-logs web-service. Contributed by Xuan Gong. 2016-05-25 19:01:12 +05:30
Varun Vasudev c380a22031 YARN-857. Localization failures should be available in container diagnostics. Contributed by Vinod Kumar Vavilapalli. 2016-05-25 19:01:12 +05:30
Junping Du 48c931331c YARN-5131. Distributed shell AM fails when extra container arrives during finishing. Contributed by Wangda Tan. 2016-05-25 06:18:11 -07:00
Rohith Sharma K S 28bd63e92b YARN-5024. TestContainerResourceUsage#testUsageAfterAMRestartWithMultipleContainers random failure. Contributed by Bibin A Chundatt 2016-05-25 10:15:50 +05:30
Naganarasimha edd716e99c YARN-5114. Add additional tests in TestRMWebServicesApps and rectify testInvalidAppAttempts failure in 2.8. Contributed by Bibin A Chundatt 2016-05-25 06:11:38 +08:00
Karthik Kambatla f979d779e1 YARN-4878. Expose scheduling policy and max running apps over JMX for Yarn queues. (Yufei Gu via kasha) 2016-05-24 10:54:11 -07:00
Naganarasimha b4078bd17b YARN-3971. Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery. (addendum patch). Contributed by Bibin A chundatt 2016-05-24 08:06:53 +08:00
Karthik Kambatla 6d043aa4cf YARN-4979. FSAppAttempt demand calculation considers demands at multiple locality levels different. (Zhihai Xu via kasha) 2016-05-23 14:29:28 -07:00
Jason Lowe ac954486c5 YARN-5055. max apps per user can be larger than max per queue. Contributed by Eric Badger 2016-05-23 15:54:42 +00:00
Jason Lowe d1df0266cf YARN-5103. With NM recovery enabled, restarting NM multiple times results in AM restart. Contributed by Junping Du 2016-05-23 15:15:27 +00:00
Junping Du 6161d9ba52 YARN-5112. Excessive log warnings for directory permission issue on NM recovery. Contributed by Jian He.
(cherry picked from commit 867cd2f5a2)
2016-05-22 16:04:02 -07:00
Jian He d364ceac85 YARN-5020. Fix Documentation for Yarn Capacity Scheduler on Resource Calculator. Contributed by Takashi Ohnishi 2016-05-20 11:35:26 -07:00
Varun Vasudev 0287c49107 YARN-5016. Add support for a minimum retry interval for container retries. Contributed by Jun Gong. 2016-05-20 12:28:14 +05:30
Junping Du 22fcd819f0 YARN-5076. YARN web interfaces lack XFS protection. Contributed by Jonathan Maron.
(cherry picked from commit 2703ec6871)
2016-05-19 14:15:21 -07:00
Jian He feb90ffcca YARN-4002. Make ResourceTrackerService#nodeHeartbeat more concurrent. Contributed by Rohith Sharma K S & Zhiguo Hong 2016-05-19 13:01:36 -07:00
Junping Du 141873ca7d YARN-5100. The YarnApplicationState is always running in ATS even application is finished. Contributed by Xuan Gong. 2016-05-19 09:13:29 -07:00
Steve Loughran 22ff9e68d1 HADOOP-12767. Update apache httpclient version to 4.5.2; httpcore to 4.4.4. Artem Aliev via stevel. 2016-05-19 14:48:18 +01:00
Akira Ajisaka d4274c64bc YARN-5107. TestContainerMetrics fails. (aajisaka) 2016-05-19 19:53:59 +09:00
Arun Suresh 1597630681 YARN-5110. Fix OpportunisticContainerAllocator to insert complete HostAddress in issued ContainerTokenIds. (Konstantinos Karanasos via asuresh) 2016-05-18 18:46:00 -07:00
Naganarasimha f04c81c9ce YARN-4925. ContainerRequest in AMRMClient, application should be able to specify nodes/racks together with nodeLabelExpression. Contributed by Bibin A Chundatt 2016-05-19 01:30:48 +08:00
Varun Vasudev ef1757790d YARN-4913. Yarn logs should take a -out option to write to a directory. Contributed by Xuan Gong. 2016-05-18 22:44:33 +05:30
Arun Suresh 8a9ecb7584 YARN-5090. Add End-to-End test-cases for DistributedScheduling using MiniYarnCluster. (asuresh) 2016-05-17 19:01:29 -07:00
Jian He 0c6726e20d Addendum patch for YARN-4832. Contributed by Junping Du 2016-05-17 14:50:30 -07:00
Jian He fa3bc3405d YARN-4832. NM side resource value should get updated if change applied in RM side. Contributed by Junping Du 2016-05-17 12:52:19 -07:00
Arun Suresh ccc93e7812 YARN-5075. Fix findbugs warnings in hadoop-yarn-common module. (asuresh) 2016-05-16 23:22:01 -07:00
Allen Wittenauer 730bc746f9 HADOOP-12930. Dynamic subcommands for hadoop shell scripts (aw)
This commit contains the following JIRA issues:

    HADOOP-12931. bin/hadoop work for dynamic subcommands
    HADOOP-12932. bin/yarn work for dynamic subcommands
    HADOOP-12933. bin/hdfs work for dynamic subcommands
    HADOOP-12934. bin/mapred work for dynamic subcommands
    HADOOP-12935. API documentation for dynamic subcommands
    HADOOP-12936. modify hadoop-tools to take advantage of dynamic subcommands
    HADOOP-13086. enable daemonization of dynamic commands
    HADOOP-13087. env var doc update for dynamic commands
    HADOOP-13088. fix shellprofiles in hadoop-tools to allow replacement
    HADOOP-13089. hadoop distcp adds client opts twice when dynamic
    HADOOP-13094. hadoop-common unit tests for dynamic commands
    HADOOP-13095. hadoop-hdfs unit tests for dynamic commands
    HADOOP-13107. clean up how rumen is executed
    HADOOP-13108. dynamic subcommands need a way to manipulate arguments
    HADOOP-13110. add a streaming subcommand to mapred
    HADOOP-13111. convert hadoop gridmix to be dynamic
    HADOOP-13115. dynamic subcommand docs should talk about exit vs. continue program flow
    HADOOP-13117. clarify daemonization and security vars for dynamic commands
    HADOOP-13120. add a --debug message when dynamic commands have been used
    HADOOP-13121. rename sub-project shellprofiles to match the rest of Hadoop
    HADOOP-13129. fix typo in dynamic subcommand docs
    HADOOP-13151. Underscores should be escaped in dynamic subcommands document
    HADOOP-13153. fix typo in debug statement for dynamic subcommands
2016-05-16 17:54:45 -07:00
Eric Payne 1217c8f6b4 YARN-5069. TestFifoScheduler.testResourceOverCommit race condition. Contributed by Eric Badger. 2016-05-16 20:28:04 +00:00
Jason Lowe 81effb7dcd YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 2016-05-16 15:40:23 +00:00
Arun Suresh f45bc5a83e YARN-4738. Notify the RM about the status of OPPORTUNISTIC containers (Konstantinos Karanasos via asuresh) 2016-05-15 17:54:34 -07:00
Xuan 7be53b65f9 YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or
RUNNING AM. Contributed by Xuan Gong
2016-05-14 12:19:23 -07:00
Arun Suresh f0ac18d001 YARN-2888. Corrective mechanisms for rebalancing NM container queues. (asuresh) 2016-05-13 13:38:36 -07:00
Andrew Wang 3c5c57af28 HADOOP-13142. Change project version from 3.0.0 to 3.0.0-alpha1. 2016-05-12 18:27:28 -07:00
Andrew Wang ca5613af91 Revert "Update project version to 3.0.0-alpha1-SNAPSHOT."
This reverts commit 6b53802cba.
2016-05-12 15:32:45 -07:00
Jason Lowe 013000fbc2 YARN-5053. More informative diagnostics when applications killed by a user. Contributed by Eric Badger 2016-05-12 20:28:36 +00:00
Andrew Wang 6b53802cba Update project version to 3.0.0-alpha1-SNAPSHOT. 2016-05-12 11:05:05 -07:00
Sangjin Lee 0bbe01f8d5 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 2016-05-12 10:11:23 -07:00
Rohith Sharma K S b7ac85259c YARN-5068. Expose scheduler queue to application master. (Harish Jaiprakash via rohithsharmaks) 2016-05-12 15:17:49 +05:30
Arun Suresh d464f4d1c4 YARN-5049. Extend NMStateStore to save queued container information. (Konstantinos Karanasos via asuresh) 2016-05-11 19:10:17 -07:00
Karthik Kambatla 4b4e4c6ba8 YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 2016-05-11 17:36:21 -07:00
Junping Du 39f2bac38b YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 2016-05-11 09:28:35 -07:00
Arun Suresh d971bf252f YARN-5073. Refactor startContainerInternal() in ContainerManager to remove unused parameter. (Konstantinos Karanasos via asuresh) 2016-05-10 22:31:38 -07:00
Vinod Kumar Vavilapalli 87f5e35133 YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 2016-05-09 22:41:18 -07:00
Naganarasimha 2750fb900f YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 2016-05-08 22:49:25 +05:30
Yongjun Zhang 47c41e7ac7 YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 2016-05-06 21:50:09 -07:00
Jason Lowe b2ed6ae731 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 2016-05-06 22:59:39 +00:00
Wangda Tan 23248f63aa getApplicationReport call may raise NPE for removed queues. (Jian He via wangda) 2016-05-06 15:30:45 -07:00
Jian He bb62e05925 YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 2016-05-05 12:56:21 -07:00
Jason Lowe d0da13229c YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-05-05 14:07:54 +00:00
Varun Vasudev 72b047715c YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 2016-05-05 13:01:54 +05:30
Vinod Kumar Vavilapalli 9e37fe3b7a YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 2016-05-04 14:16:03 -07:00
Wangda Tan 7bd418e48c YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 2016-05-04 11:38:55 -07:00
Junping Du e61d431275 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 2016-05-04 10:35:49 -07:00
Rohith Sharma K S 75e0450593 YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 2016-05-04 09:58:26 +05:30
Jason Lowe ed54f5f1ff YARN-5003. Add container resource to RM audit log. Contributed by Nathan Roberts 2016-05-03 20:03:41 +00:00
Jason Lowe c6b4839168 YARN-4834. ProcfsBasedProcessTree doesn't track daemonized processes. Contributed by Nathan Roberts 2016-05-03 17:27:28 +00:00
Junping Du 06413da72e YARN-4851. Metric improvements for ATS v1.5 storage components. Li Lu via junping_du. 2016-05-03 04:16:21 -07:00
Robert Kanter 9d4d30243b Remove parent's env vars from child processes 2016-04-29 09:25:51 -07:00
Varun Vasudev 0f25a1bb52 YARN-3998. Add support in the NodeManager to re-launch containers. Contributed by Jun Gong. 2016-04-29 16:09:07 +05:30
Jian He 4a8508501b YARN-5009. NMLeveldbStateStoreService database can grow substantially leading to longer recovery times. Contributed by Jason Lowe 2016-04-28 21:54:11 -07:00
Jian He dd80042c42 YARN-5008. LeveldbRMStateStore database can grow substantially leading to long recovery times. Contributed by Jason Lowe 2016-04-28 21:27:25 -07:00
Li Lu f16722d2ef YARN-4956. findbug issue on LevelDBCacheTimelineStore. (Zhiyuan Yang via gtcarrera9) 2016-04-27 10:57:10 -07:00
Karthik Kambatla 185c3d4de1 YARN-4807. MockAM#waitForState sleep duration is too long. (Yufei Gu via kasha) 2016-04-27 09:43:23 -07:00
Varun Vasudev 66b07d8374 YARN-4966. Improve yarn logs to fetch container logs without specifying nodeId. Contributed by Xuan Gong. 2016-04-27 15:56:36 +05:30
Jian He 4beff01354 YARN-4983. JVM and UGI metrics disappear after RM transitioned to standby mode 2016-04-26 21:00:17 -07:00
Arun Suresh 341888a0aa YARN-4412. Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers. (asuresh) 2016-04-26 20:12:12 -07:00
Karthik Kambatla 1a3f1482e2 YARN-4795. ContainerMetrics drops records. (Daniel Templeton via kasha) 2016-04-26 06:15:36 -07:00
Karthik Kambatla 4b1dcbbe0c YARN-1297. FairScheduler: Move some logs to debug and check if debug logging is enabled 2016-04-26 05:10:09 -07:00
Arun Suresh a5fed8bbfe YARN-4991. Fix ContainerRequest Constructor to set nodelabelExpression correctly. (Bibin A Chundatt via asuresh) 2016-04-25 23:26:21 -07:00
Arun Suresh c282a08f38 YARN-2885. Create AMRMProxy request interceptor and ContainerAllocator to distribute OPPORTUNISTIC containers to appropriate Nodes (asuresh)
(cherry picked from commit 2bf025278a318b0452fdc9ece4427b4c42124e39)
2016-04-24 22:38:33 -07:00
Arun Suresh b2a654c5ee YARN-4335. Allow ResourceRequests to specify ExecutionType of a request ask (kkaranasos via asuresh)
(cherry picked from commit 8ffabfdf4f)
2016-04-22 23:20:54 -07:00
Jing Zhao 63e5412f1a HDFS-9427. HDFS should not default to ephemeral ports. Contributed by Xiaobing Zhou. 2016-04-22 15:14:40 -07:00
Wangda Tan 7cb3a3da96 YARN-4846. Fix random failures for TestCapacitySchedulerPreemption#testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers. (Bibin A Chundatt via wangda) 2016-04-22 11:40:32 -07:00
Eric Payne 3dce486d88 YARN-4556. TestFifoScheduler.testResourceOverCommit fails. Contributed by Akihiro Suda 2016-04-21 21:16:47 +00:00
Li Lu 7c6339f66a YARN-4968. A couple of AM retry unit tests need to wait SchedulerApplicationAttempt stopped. (Wangda Tan via gtcarrera9) 2016-04-21 13:25:33 -07:00
Chris Douglas 95a5046607 YARN-4976. Missing NullPointer check in ContainerLaunchContextPBImpl. Contributed by Giovanni Matteo Fumarola 2016-04-21 10:45:38 -07:00
Karthik Kambatla 170c4fd4cd YARN-4784. Fairscheduler: defaultQueueSchedulingPolicy should not accept FIFO. (Yufei Gu via kasha) 2016-04-20 23:58:12 -07:00
Karthik Kambatla 6e297836d6 YARN-4935. TestYarnClient#testSubmitIncorrectQueue fails when FairScheduler is the default. (Yufei Gu via kasha) 2016-04-20 23:40:46 -07:00
Wangda Tan 33fd95a99c YARN-4890. Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate. (Sunil G via wangda) 2016-04-20 17:37:38 -07:00
Karthik Kambatla c8172f5f14 YARN-2883. Queuing of container requests in the NM. (Konstantinos Karanasos and Arun Suresh via kasha) 2016-04-20 09:55:50 -07:00
Vinod Kumar Vavilapalli 477003730e Fixed TimelineClient to retry SocketTimeoutException too. Contributed by Xuan Gong. 2016-04-18 11:47:06 -07:00
Wangda Tan fdc46bfb37 YARN-4934. Reserved Resource for QueueMetrics needs to be handled correctly in few cases. (Sunil G via wangda) 2016-04-16 22:47:41 -07:00
Xuan e6c0742012 YARN-4965. Distributed shell AM failed due to ClientHandlerException thrown by jersey. Contributed by Junping Du 2016-04-16 19:39:18 -07:00
Arun Suresh cab9cbaa0a YARN-4468. Document the general ReservationSystem functionality, and the REST API. (subru and carlo via asuresh) 2016-04-15 16:58:49 -07:00
Jason Lowe 69f3d428d5 YARN-4940. yarn node -list -all failed if RM start with decommissioned node. Contributed by sandflee 2016-04-15 20:36:45 +00:00
Naganarasimha fdbafbc9e5 YARN-4909. Fix intermittent failures of TestRMWebServices And TestRMWithCSRFFilter. Contributed by Bibin A Chundatt 2016-04-15 23:37:05 +05:30
Jason Lowe 3150ae8108 YARN-4924. NM recovery race can lead to container not cleaned up. Contributed by sandflee 2016-04-14 19:17:14 +00:00
Robert Kanter 0d9194df00 YARN-4541. Change log message in LocalizedResource#handle() to DEBUG (rchiang via rkanter) 2016-04-13 17:45:18 -07:00
Xuan e0cb426758 YARN-4886. Add HDFS caller context for EntityGroupFSTimelineStore. Contributed by Li Lu 2016-04-13 10:38:17 -07:00
Naganarasimha 437e9d6475 YARN-4810. NM applicationpage cause internal error 500. Contributed by Bibin A Chundatt. 2016-04-12 17:59:46 +05:30
Rohith Sharma K S 4ee7f0a3e7 YARN-4897. dataTables_wrapper change min height. Contributed by Bibin A Chundatt. 2016-04-12 10:25:56 +05:30
Rohith Sharma K S ff722bbbdf YARN-4794. Deadlock in NMClientImpl. Contributed by Jian He. 2016-04-12 09:59:33 +05:30
Vinod Kumar Vavilapalli 44bbc50d91 YARN-4168. Fixed a failing test TestLogAggregationService.testLocalFileDeletionOnDiskFull. Contributed by Takashi Ohnishi. 2016-04-11 12:11:14 -07:00
Jason Lowe 2a5da97f81 Revert "YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla"
This reverts commit 1cbcd4a491.
2016-04-11 15:51:01 +00:00
Junping Du 08ddb3ac6d YARN-4928. Some yarn.server.timeline.* tests fail on Windows attempting to use a test root path containing a colon. Contributed by Gergely Novák. 2016-04-11 08:48:32 -07:00
Akira Ajisaka 1ff27f9d12 YARN-4630. Remove useless boxing/unboxing code. Contributed by Kousuke Saruta. 2016-04-11 14:55:03 +09:00
Varun Vasudev 07920acc16 YARN-4562. YARN WebApp ignores the configuration passed to it for keystore settings. Contributed by Sergey Shelukhin. 2016-04-11 09:39:57 +05:30
Akira Ajisaka 1b78b2ba17 YARN-4938. MiniYarnCluster should not request transitionToActive to RM on non-HA environment. Contributed by Eric Badger. 2016-04-11 01:31:49 +09:00
Karthik Kambatla ff95fd547b YARN-4927. TestRMHA#testTransitionedToActiveRefreshFail fails with FairScheduler. (Bibin A Chundatt via kasha) 2016-04-09 10:31:02 -07:00
Wangda Tan fcb3fcd4c6 YARN-4826. Document configuration of ReservationSystem for CapacityScheduler. (Subru Krishnan via wangda) 2016-04-08 16:21:04 -07:00
Wangda Tan ec06957941 YARN-3215. Respect labels in CapacityScheduler when computing headroom. (Naganarasimha G R via wangda) 2016-04-08 15:33:04 -07:00
Jian He 9cb0c963d2 YARN-4740. AM may not receive the container complete msg when it restarts. Contributed by Jun Gong 2016-04-08 11:20:35 -07:00
Karthik Kambatla e82f961a39 YARN-4756. Unnecessary wait in Node Status Updater during reboot. (Eric Badger via kasha) 2016-04-07 17:05:29 -07:00
Jian He 93bacda08b YARN-4769. Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI. Contributed by Varun Vasudev 2016-04-06 16:13:47 -07:00
Varun Vasudev b41e65e5bc YARN-4906. Capture container start/finish time in container metrics. Contributed by Jian He. 2016-04-06 13:41:33 +05:30
Wangda Tan 21eb428448 YARN-4699. Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node. (Sunil G via wangda) 2016-04-05 16:24:11 -07:00
Masatake Iwasaki 500e5a5952 YARN-4917. Fix typos in documentation of Capacity Scheduler. (Takashi Ohnishi via iwasakims) 2016-04-06 04:00:31 +09:00
Masatake Iwasaki 30206346cf YARN-4915. Fix typo in YARN Secure Containers documentation (Takashi Ohnishi via iwasakims) 2016-04-06 03:47:22 +09:00
Junping Du 0005816743 YARN-4916. TestNMProxy.tesNMProxyRPCRetry fails. Contributed by Tibor Kiss. 2016-04-05 09:01:08 -07:00
Junping Du 6be28bcc46 YARN-4893. Fix some intermittent test failures in TestRMAdminService. Contributed by Brahma Reddy Battula. 2016-04-05 06:57:54 -07:00
Jason Lowe 1cbcd4a491 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 2016-04-05 13:40:19 +00:00
Rohith Sharma K S 776b549e2a YARN-4609. RM Nodes list page takes too much time to load. Contributed by Bibin A Chundatt 2016-04-05 14:47:25 +05:30
Rohith Sharma K S 552237d4a3 YARN-4880. Running TestZKRMStateStorePerf with real zookeeper cluster throws NPE. Contributed by Sunil G 2016-04-05 14:26:19 +05:30
Li Lu f61de41736 YARN-4706. UI Hosting Configuration in TimelineServer doc is broken. (Akira AJISAKA via gtcarrera9) 2016-04-04 14:40:27 -07:00
naganarasimha 5092c94195 YARN-4746. yarn web services should convert parse failures of appId, appAttemptId and containerId to 400. Contributed by Bibin A Chundatt 2016-04-04 16:25:03 +05:30
Rohith Sharma K S 1e6f92977d YARN-4607. Pagination support for AppAttempt page TotalOutstandingResource Requests table. Contributed by Bibin A Chundatt 2016-04-04 08:09:29 +05:30
Karthik Kambatla 81d04cae41 YARN-4657. Javadoc comment is broken for Resources.multiplyByAndAddTo(). (Daniel Templeton via kasha) 2016-04-01 16:20:00 -07:00
Arun Suresh 5686caa9fc Missing file for YARN-4895. 2016-04-01 15:58:13 -07:00
Arun Suresh 82621e38a0 YARN-4895. Add subtractFrom method to ResourceUtilization class. Contributed by Konstantinos Karanasos. 2016-04-01 14:57:06 -07:00
Varun Vasudev 3488c4f2c9 Revert "YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki."
This reverts commit 0064cba169.
2016-04-01 12:20:40 +05:30
Wangda Tan 12b11e2e68 YARN-4634. Scheduler UI/Metrics need to consider cases like non-queue label mappings. (Sunil G via wangda) 2016-03-31 14:35:18 -07:00
Jian He 0dd9bcab97 YARN-4811. Generate histograms in ContainerMetrics for actual container resource usage 2016-03-31 14:28:13 -07:00
Robert Kanter 7a021471c3 YARN-4639. Remove dead code in TestDelegationTokenRenewer added in YARN-3055 (templedf via rkanter) 2016-03-31 13:09:09 -07:00
Sangjin Lee 6d67420dbc YARN-4183. Clarify the behavior of timeline service config properties (Naganarasimha G R via sjlee) 2016-03-31 10:49:03 -07:00
Allen Wittenauer 0a74610d1c HADOOP-11393. Revert HADOOP_PREFIX, go back to HADOOP_HOME (aw) 2016-03-31 07:51:05 -07:00
Varun Vasudev 0064cba169 YARN-4857. Add missing default configuration regarding preemption of CapacityScheduler. Contributed by Kai Sasaki. 2016-03-31 14:05:49 +05:30
Varun Vasudev f1b8f6b2c1 YARN-4884. Fix missing documentation about rmadmin command regarding node labels. Contributed by Kai Sasaki. 2016-03-31 14:01:48 +05:30
Jian He 60e4116bf1 YARN-4822. Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates. Contributed by Wangda Tan 2016-03-30 12:43:52 -07:00
Robert Kanter 8286270466 YARN-4436. DistShell ApplicationMaster.ExecBatScripStringtPath is misspelled (mattlamantia via rkanter) 2016-03-29 17:21:29 -07:00
Wangda Tan fc055a3cbe YARN-4865. Track Reserved resources in ResourceUsage and QueueCapacities. (Sunil G via wangda) 2016-03-29 17:07:55 -07:00
Xuan 80182809ae YARN-4863. AHS Security login should be in serviceInit() instead of serviceStart(). Contributed by Junping Du 2016-03-28 22:17:45 -07:00
Jason Lowe 948b758070 YARN-4773. Log aggregation performs extraneous filesystem operations when rolling log aggregation is disabled. Contributed by Jun Gong 2016-03-28 23:00:56 +00:00
Jian He 524bc3c33a YARN-998. Keep NM resource updated through dynamic resource config for RM/NM restart. Contributed by Junping Du 2016-03-28 11:12:33 -07:00
Jian He 8831b18c65 Missing file for YARN-4117. 2016-03-28 09:59:47 -07:00
Jian He 55ae143923 YARN-4117. End to end unit test with mini YARN cluster for AMRMProxy Service. Contributed by Giovanni Matteo Fumarola 2016-03-27 20:22:12 -07:00
Karthik Kambatla 49ff54c860 YARN-4805. Don't go through all schedulers in ParameterizedTestBase. (kasha) 2016-03-26 21:45:13 -07:00
Arun Suresh 00bebb7e58 YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh) 2016-03-25 15:54:38 -07:00
Jason Lowe e8fc81f9c8 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong 2016-03-25 20:15:49 +00:00
Arun Suresh 2e1d0ff4e9 YARN-4687. Document Reservation ACLs (Contributed by Sean Po) 2016-03-24 10:40:51 -07:00
Arun Suresh d820975c1a YARN-4683. Document the List Reservations REST API. (Contributed by Sean Po) 2016-03-24 10:30:29 -07:00
Arun Suresh d82e797b65 YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh) 2016-03-24 09:59:55 -07:00
Allen Wittenauer b1394d6307 YARN-4850. test-fair-scheduler.xml isn't valid xml (Yufei Gu via aw) 2016-03-24 08:15:58 -07:00
Junping Du 19b645c938 YARN-4820. ResourceManager web redirects in HA mode drops query parameters. Contributed by Varun Vasudev. 2016-03-23 19:34:30 -07:00
Allen Wittenauer 738155063e HADOOP-12857. rework hadoop-tools (aw) 2016-03-23 13:46:38 -07:00
Junping Du af1d125f9c YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. Contributed by Xuan Gong. 2016-03-23 08:57:16 -07:00
Karthik Kambatla 7fae4c68e6 YARN-4732. *ProcessTree classes have too many whitespace issues 2016-03-20 10:47:38 -07:00
Eric Payne 92b7e0d413 YARN-4686. MiniYARNCluster.start() returns before cluster is completely started. Contributed by Eric Badger. 2016-03-18 16:12:47 +00:00
Junping Du ca8106d2dd YARN-4785. inconsistent value type of the type field for LeafQueueInfo in response of RM REST API. 2016-03-17 09:04:41 -07:00
Karthik Kambatla f84af8bd58 YARN-4812. TestFairScheduler#testContinuousScheduling fails intermittently. (kasha) 2016-03-17 05:54:06 -07:00
Wangda Tan ae14e5d07f YARN-4108. CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)
(cherry picked from commit 7e8c9beb41)
2016-03-16 17:02:33 -07:00
Wangda Tan fa7a43529d Revert "CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan)"
This reverts commit 7e8c9beb41.
2016-03-16 17:02:10 -07:00
Wangda Tan 7e8c9beb41 CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request. (Wangda Tan) 2016-03-16 16:59:59 -07:00
Karthik Kambatla 3ef5500783 YARN-4560. Make scheduler error checking message more user friendly. (Ray Chiang via kasha) 2016-03-15 23:45:01 -07:00
Xuan a888b5bc53 Revert "YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9)"
This reverts commit 3a0b695441.
2016-03-15 22:10:39 -07:00
Li Lu 5176a69868 YARN-4817. Change Log Level to DEBUG for putDomain call in ATS 1.5. (Xuan Gong via gtcarrera9) 2016-03-15 11:47:39 -07:00
Li Lu 3a0b695441 YARN-4814. ATS 1.5 timelineclient impl call flush after every event write. (Xuan Gong via gtcarrera9) 2016-03-15 11:40:54 -07:00
Robert Kanter 22ca176dfe TestResourceLocalizationService.testPublicResourceInitializesLocalDir fails Intermittently due to IllegalArgumentException from cleanup (templedf via rkanter) 2016-03-15 10:05:10 -07:00
Siddharth Seth eba66a64d2 YARN-4816. Fix incompatible change in SystemClock. 2016-03-14 20:14:33 -07:00
Karthik Kambatla 20d389ce61 YARN-4719. Add a helper library to maintain node state and allows common queries. (kasha) 2016-03-14 14:19:05 -07:00
Junping Du f291d82cd4 YARN-4545. Allow YARN distributed shell to use ATS v1.5 APIs. Li Lu via junping_du 2016-03-14 08:28:38 -07:00
Li Lu 017d2c127b YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency.
This commit amends commit d49cfb3504 with a missed test file.
2016-03-10 13:02:28 -08:00
Li Lu d49cfb3504 YARN-4696. Improving EntityGroupFSTimelineStore on exception handling, test setup, and concurrency. (Steve Loughran via gtcarrera9) 2016-03-10 10:51:55 -08:00
Wangda Tan 0233d4e0ee YARN-4465. SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled. (Bibin A Chundatt via wangda) 2016-03-08 14:27:03 -08:00
Jian He 3c33158d1c YARN-4764. Application submission fails when submitted queue is not available in scheduler xml. Contributed by Bibin A Chundatt 2016-03-08 13:07:57 -08:00
Akira Ajisaka f86850b544 HADOOP-12860. Expand section "Data Encryption on HTTP" in SecureMode documentation. Contributed by Wei-Chiu Chuang. 2016-03-08 14:28:47 +09:00
Vinod Kumar Vavilapalli b2661765a5 YARN-4762. Fixed CgroupHandler's creation and usage to avoid NodeManagers crashing when LinuxContainerExecutor is enabled. (Sidharta Seethana via vinodkv) 2016-03-07 11:08:17 -08:00
Jason Lowe 4163e36c2b YARN-4760. proxy redirect to history server uses wrong URL. Contributed by Eric Badger 2016-03-07 15:56:33 +00:00
Jason Lowe 059caf9989 YARN-4744. Too many signal to container failure in case of LCE. Contributed by Sidharta Seethana 2016-03-07 15:40:01 +00:00
Varun Vasudev 8ed2e060e8 YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 2016-03-07 16:18:35 +05:30
Varun Vasudev e51a8c1056 YARN-4737. Add CSRF filter support in YARN. Contributed by Jonathan Maron. 2016-03-07 15:26:44 +05:30
Zhihai Xu e1ccc9622b YARN-4761. NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations on fair scheduler. Contributed by Sangjin Lee 2016-03-06 19:46:09 -08:00
Rohith Sharma K S 19ee185907 YARN-4763. RMApps Page crashes with NPE. (Bibin A Chundatt via rohithsharmaks) 2016-03-05 13:02:57 +05:30
Masatake Iwasaki cbd31328a6 HADOOP-12470. In-page TOC of documentation should be automatically generated by doxia macro (iwasakims) 2016-03-04 14:11:36 +09:00
Jian He 5c465df904 YARN-4671. There is no need to acquire CS lock when completing a container. Contributed by Meng Ding 2016-03-01 13:14:12 -08:00
Jian He d93c22ec27 YARN-4748. ApplicationHistoryManagerOnTimelineStore should not swallow exceptions on generateApplicationReport. Contributed by Li Lu 2016-02-29 18:19:09 -08:00
Karthik Kambatla 9dafaaaf0d YARN-4704. TestResourceManager#testResourceAllocation() fails when using FairScheduler. (Yufei Gu via kasha) 2016-02-29 16:10:12 -08:00
Haohui Mai 0fa54d45b1 HADOOP-12813. Migrate TestRPC and related codes to rebase on ProtobufRpcEngine. Contributed by Kai Zheng. 2016-02-29 11:41:00 -08:00
Jason Lowe c58a6d53c5 YARN-4731. container-executor should not follow symlinks in recursive_unlink_children. Contributed by Colin Patrick McCabe 2016-02-29 15:24:35 +00:00
Rohith Sharma K S e0b14f26f5 YARN-4566. Fix test failure in TestMiniYarnClusterNodeUtilization. (Takashi Ohnishi via rohithsharmaks) 2016-02-29 10:38:14 +08:00
Karthik Kambatla f9692770a5 YARN-4718. Rename variables in SchedulerNode to reduce ambiguity post YARN-1011. (Inigo Goiri via kasha) 2016-02-28 09:35:59 -08:00
Jason Lowe 6b0f813e89 YARN-4723. NodesListManager$UnknownNodeId ClassCastException. Contributed by Kuhu Shukla 2016-02-26 20:24:50 +00:00
Ming Ma 7f3139e54d YARN-4720. Skip unnecessary NN operations in log aggregation. (Jun Gong via mingma) 2016-02-26 08:40:05 -08:00
Robert Kanter d7fdec1e6b YARN-4579. Allow DefaultContainerExecutor container log directory permissions to be configurable (rchiang via rkanter) 2016-02-25 16:36:38 -08:00
Robert Kanter c4d4df8de0 YARN-4701. When task logs are not available, port 8041 is referenced instead of port 8042 (haibochen via rkanter) 2016-02-25 12:48:02 -08:00
Karthik Kambatla c684f2b007 YARN-4729. SchedulerApplicationAttempt#getTotalRequiredResources can throw an NPE. (kasha) 2016-02-24 18:33:57 -08:00
Robert Kanter 954dd57043 YARN-4697. NM aggregation thread pool is not bound by limits (haibochen via rkanter) 2016-02-24 15:00:24 -08:00
Li Lu 9e0f7b8b69 YARN-4680. TimerTasks leak in ATS V1.5 Writer. (Xuan Gong via
gtcarrera9)
2016-02-24 13:43:09 -08:00
Sangjin Lee 553b591ba0 YARN-4722. AsyncDispatcher logs redundant event queue sizes (Jason Lowe via sjlee) 2016-02-24 09:29:41 -08:00
Jason Lowe d284e187b8 YARN-2046. Out of band heartbeats are sent only on container kill and possibly too early. Contributed by Ming Ma 2016-02-23 20:49:09 +00:00
Junping Du 9ed17f181d YARN-3223. Resource update during NM graceful decommission. Contributed by Brook Zhou. 2016-02-23 03:30:26 -08:00
Tsuyoshi Ozawa 0e12114c9c YARN-4648. Move preemption related tests from TestFairScheduler to TestFairSchedulerPreemption. Contributed by Kai Sasaki. 2016-02-23 19:50:08 +09:00
Rohith Sharma K S de09f41147 YARN-4651. Document movetoqueue option in 'YARN Commands'. (Takashi Ohnishi via rohithsharmaks) 2016-02-23 18:03:42 +08:00
Varun Vasudev 140cb5d745 YARN-4709. NMWebServices produces incorrect JSON for containers. Contributed by Varun Saxena. 2016-02-23 12:29:25 +05:30
Zhe Zhang a2fdfff02d HADOOP-12668. Support excluding weak Ciphers in HttpServer2 through ssl-server.conf. Contributed by Vijay Singh.
Change-Id: Ie46a5427d29188935427f67e55203c19fcd83335
2016-02-22 14:13:54 -08:00
Junping Du 3fab88540f YARN-4386. refreshNodesGracefully() should send recommission event to active RMNodes only. Contributed by Kuhu Shukla. 2016-02-22 07:04:19 -08:00
Akira Ajisaka 5e7d4d51f8 YARN-4707. Remove the extra char (>) from SecureContainer.md. Contributed by Brahma Reddy Battula. 2016-02-22 17:17:40 +09:00