Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,195 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

2988 Commits

Author SHA1 Message Date
Andrew Wang da456ffd62 Preparing for 3.0.0-alpha2 development 2016-07-15 19:04:17 -07:00
Ray Chiang f5f1c81e7d YARN-5272. Handle queue names consistently in FairScheduler. (Wilfred Spiegelenburg via rchiang) 2016-07-15 14:38:50 -07:00
Arun Suresh e905a42a2c YARN=5181. ClusterNodeTracker: add method to get list of nodes matching a specific resourceName. (kasha via asuresh) 2016-07-15 14:35:12 -07:00
Wangda Tan 24db9167f1 YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda) 2016-07-15 11:40:12 -07:00
Varun Saxena a72cb3825a YARN-5383. Fix findbugs and checkstyle issues in ContainerExecutor. (Vrushali C via Varun Saxena) 2016-07-15 23:17:04 +05:30
Junping Du 7e5355c14e YARN-5339. Passing file to -out for YARN log CLI doesnt give warning or error code. Contributed by Xuan Gong. 2016-07-15 09:23:11 -07:00
Naganarasimha b5ee7dbd8d YARN-5380. NMTimelinePublisher should use getMemorySize instead of getMemory. Contributed by Vrushali C 2016-07-15 14:34:48 +05:30
Sangjin Lee e549a9af31 YARN-5379. TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently (Vrushali C via sjlee) 2016-07-14 15:19:03 -07:00
Akira Ajisaka 6cf017558a YARN-5159. Wrong Javadoc tag in MiniYarnCluster. Contributed by Andras Bokor. 2016-07-14 10:45:11 -07:00
Xuan 429347289c YARN-5363. For AM containers, or for containers of running-apps, "yarn logs" incorrectly only (tries to) shows syslog file-type by default. (Contributed by Vinod Kumar Vavilapalli) 2016-07-14 10:20:08 -07:00
Varun Vasudev 54bf14f80b YARN-5303. Clean up ContainerExecutor JavaDoc. Contributed by Daniel Templeton. 2016-07-14 19:28:11 +05:30
Varun Vasudev e5e558b0a3 YARN-4759. Fix signal handling for docker containers. Contributed by Shane Kumpf. 2016-07-14 19:27:16 +05:30
Varun Vasudev 58e1850801 YARN-5298. Mount usercache and NM filecache directories into Docker container. Contributed by Sidharta Seethana. 2016-07-14 16:11:07 +05:30
Rohith Sharma K S dbe97aa768 YARN-5299. Log Docker run command when container fails. Contributed by Varun Vasudev. 2016-07-14 11:48:10 +05:30
Vinod Kumar Vavilapalli d18050522c YARN-5339. Fixed "yarn logs" to fail when a file is passed to -out option instead of a directory. Contributed by Xuan Gong. 2016-07-13 14:52:30 -07:00
Naganarasimha af8f480c24 YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee 2016-07-14 01:43:37 +05:30
Vinod Kumar Vavilapalli eb47163234 YARN-5200. Enhanced "yarn logs" to be able to get a list of containers whose logs are aggregated via a "show_container_log_info" option. Contributed by Xuan Gong. 2016-07-13 10:54:15 -07:00
Varun Saxena 56142171b9 YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena) 2016-07-13 21:15:41 +05:30
Rohith Sharma K S d6d41e820a YARN-5362. TestRMRestart#testFinishedAppRemovalAfterRMRestart can fail. Contributed by sandflee. 2016-07-13 19:12:35 +05:30
Varun Saxena 06c56ff79b YARN-5353. ResourceManager can leak delegation tokens when they are shared across apps. (Jason Lowe via Varun Saxena). 2016-07-13 07:55:34 +05:30
Robert Kanter bf6f4a3b98 YARN-4366. Fix Lint Warnings in YARN Common (templedf via rkanter) 2016-07-12 15:12:25 -07:00
Jason Lowe 10b704c594 YARN-5317. testAMRestartNotLostContainerCompleteMsg may fail. Contributed by sandflee 2016-07-12 20:27:41 +00:00
Junping Du 7705812c7e New addendum to YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or RUNNING AM. 2016-07-12 08:29:49 -07:00
Varun Saxena a3ac1c1943 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 2016-07-12 20:41:52 +05:30
Jian He 819224dcf9 YARN-5270. Solve miscellaneous issues caused by YARN-4844. Contributed by Wangda Tan 2016-07-11 22:36:20 -07:00
Varun Saxena 0fd3980a1f YARN-5037. Fix random failure of TestRMRestart#testQueueMetricsOnRMRestart (sandflee via Varun Saxena). 2016-07-10 21:28:52 +05:30
Sangjin Lee 6cf6ab7b78 Made a number of miscellaneous fixes for javac, javadoc, and checstyle warnings. 2016-07-10 08:46:05 -07:00
Sangjin Lee c5dbde0cc4 Fixed the build after the rebase with trunk. 2016-07-10 08:46:05 -07:00
Sangjin Lee b313ce06b2 YARN-5174. [documentation] several updates/corrections to timeline service documentation (Sangjin Lee, Varun Saxena, Naganarasimha G R, and Li Lu via sjlee) 2016-07-10 08:46:04 -07:00
Vrushali 6d943038f6 Cleanup changes during rebase with trunk (Vrushali C) 2016-07-10 08:46:04 -07:00
Sangjin Lee 1a227744ac YARN-5070. upgrade HBase version for first merge (Vrushali C via sjlee) 2016-07-10 08:46:04 -07:00
Varun Saxena 1ff6833bba YARN-5243. fix several rebase and other miscellaneous issues before merge. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan 32b033d57c YARN-5052 Update timeline service v2 documentation to capture information about filters (Varun Saxena via Vrushali C) 2016-07-10 08:46:03 -07:00
Varun Saxena 892b193bd7 YARN-5170. Eliminate singleton converters and static method access. (Joep Rottinghuis via Varun Saxena) 2016-07-10 08:46:03 -07:00
Vrushali Channapattan 3832795e3c YARN-5210 NPE in Distributed Shell while publishing DS_CONTAINER_START event (Varun Saxena via Vrushali C) 2016-07-10 08:46:02 -07:00
Varun Saxena c65020075e YARN-5167. Escape occurences of encodedValues. (Sangjin Lee via Varun Saxena) 2016-07-10 08:46:02 -07:00
Varun Saxena 9c926cf432 YARN-5142. fix findbugs warnings/errors for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via Varun Saxena) 2016-07-10 08:46:02 -07:00
Li Lu 0a9b085f05 YARN-5189. Make HBaseTimeline[Reader|Writer]Impl default and move FileSystemTimeline*Impl. (Joep Rottinghuis and Sangjin Lee via gtcarrera9) 2016-07-10 08:46:01 -07:00
Sangjin Lee c81a2e1d19 YARN-5015. entire time series is returned for YARN container system metrics (CPU and memory) (Varun Saxena via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee 497d1c7f51 YARN-5111. YARN container system metrics are not aggregated to application (Naganarasimha G R via sjlee) 2016-07-10 08:46:01 -07:00
Sangjin Lee 7b8cfa5c2f YARN-5109. timestamps are stored unencoded causing parse errors (Varun Saxena via sjlee) 2016-07-10 08:46:01 -07:00
Li Lu d729e8211b YARN-5138. fix "no findbugs output file" error for hadoop-yarn-server-timelineservice-hbase-tests. (Vrushali C via gtcarrera9) 2016-07-10 08:46:00 -07:00
Sangjin Lee 702236129b YARN-5095. flow activities and flow runs are populated with wrong timestamp when RM restarts w/ recovery enabled (Varun Saxena via sjlee) 2016-07-10 08:46:00 -07:00
Sangjin Lee 831a3ffd6e YARN-5097. NPE in Separator.joinEncoded() (Vrushali C via sjlee) 2016-07-10 08:46:00 -07:00
Sangjin Lee a1b6d7456f YARN-5018. Online aggregation logic should not run immediately after collectors got started (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee 8c8183e515 YARN-5093. created time shows 0 in most REST output (Varun Saxena via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee d5dfee24b1 YARN-5050. Code cleanup for TestDistributedShell (Li Lu via sjlee) 2016-07-10 08:45:59 -07:00
Sangjin Lee 4a7011a45f YARN-5096 addendum. Turned another logging statement to debug. Contributed by Sangjin Lee. 2016-07-10 08:45:59 -07:00
Varun Saxena bcdb382600 YARN-5096. timelinereader has a lot of logging that's not useful. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:58 -07:00
Varun Saxena bc6f978cd3 YARN-5045. hbase unit tests fail due to dependency issues. (Sangjin Lee via varunsaxena) 2016-07-10 08:45:58 -07:00
Sangjin Lee 089caf49fe YARN-4447. Provide a mechanism to represent complex filters and parse them at the REST layer (Varun Saxena via sjlee) 2016-07-10 08:45:57 -07:00
Li Lu c2055a97d5 YARN-3150. Documenting the timeline service v2. (Sangjin Lee and Vrushali C via gtcarrera9) 2016-07-10 08:45:57 -07:00
Sangjin Lee 69dc561b61 YARN-4986. Add a check in the coprocessor for table to operated on (Vrushali C via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee 39cce4e629 YARN-3816. [Aggregation] App-level aggregation and accumulation for YARN system metrics (Li Lu via sjlee) 2016-07-10 08:45:56 -07:00
Sangjin Lee c2efdc415a YARN-3863. Support complex filters in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:55 -07:00
Varun Saxena a3cf40e532 YARN-3461. Consolidate flow name/version/run defaults. (Sangjin Lee via Varun Saxena) 2016-07-10 08:45:55 -07:00
Sangjin Lee 84c35ac6c4 YARN-4711. NM is going down with NPE's due to single thread processing of events by Timeline client (Naganarasimha G R via sjlee) 2016-07-10 08:45:55 -07:00
Varun Saxena 6f6cc647d6 YARN-4712. CPU Usage Metric is not captured properly in YARN-2928. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:54 -07:00
Sangjin Lee 9bdd455dce YARN-4062. Add the flush and compaction functionality via coprocessors and scanners for flow run table (Vrushali C via sjlee) 2016-07-10 08:45:54 -07:00
Varun Saxena d56dde490b YARN-4700. ATS storage has one extra record each time the RM got restarted. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:53 -07:00
Sangjin Lee 0d02ab8729 YARN-3367. Replace starting a separate thread for post entity with event loop in TimelineClient (Naganarasimha G R via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee 960af7d471 YARN-4409. Fix javadoc and checkstyle issues in timelineservice code (Varun Saxena via sjlee) 2016-07-10 08:45:53 -07:00
Sangjin Lee 9cb1287e9b YARN-4446. Refactor reader API for better extensibility (Varun Saxena via sjlee) 2016-07-10 08:45:52 -07:00
Naganarasimha 06f0b50a28 YARN-4644. TestRMRestart fails and findbugs issue in YARN-2928 branch (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Li Lu 9d40d9d34c YARN-4224. Support fetching entities by UID and change the REST 
interface to conform to current REST APIs' in YARN. (Varun Saxena via
gtcarrera9)
 2016-07-10 08:45:52 -07:00
Naganarasimha 6934b05c71 YARN-4238. createdTime and modifiedTime is not reported while publishing entities to ATSv2. (Varun Saxena via Naganarasimha G R) 2016-07-10 08:45:52 -07:00
Varun Saxena f5380d850e YARN-4622. TestDistributedShell fails for v2 test cases after modifications for 1.5. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:51 -07:00
Li Lu 34f02f07d5 Rebase to latest trunk 2016-07-10 08:45:51 -07:00
Li Lu 88f0294114 YARN-4200. Refactor reader classes in storage to nest under hbase 
specific package name. Contributed by Li Lu.
 2016-07-10 08:45:51 -07:00
Sangjin Lee cc16683cef YARN-3995. Some of the NM events are not getting published due race condition when AM container finishes in NM (Naganarasimha G R via sjlee) 2016-07-10 08:45:50 -07:00
Varun Saxena 829cceebc0 YARN-3586. RM to only get back addresses of Collectors that NM needs to know. 
(Junping Du via Varun Saxena).
 2016-07-10 08:45:50 -07:00
Varun Saxena eb0ac8efb1 YARN-4350. TestDistributedShell fails for V2 scenarios. (Naganarasimha G R via Varun Saxena) 2016-07-10 08:45:50 -07:00
Li Lu 8ef546c1ee YARN-4445. Unify the term flowId and flowName in timeline v2 codebase. 
Contributed by Zhan Zhang.
 2016-07-10 08:45:49 -07:00
Varun Saxena c4d7bbda5c YARN-4460. [Bug fix] RM fails to start when SMP is enabled. (Li Lu via Varun Saxena) 2016-07-10 08:45:49 -07:00
Xuan 2e2dbf59d1 YARN-4392. ApplicationCreatedEvent event time resets after RM 
restart/failover. Contributed by Naganarasimha G R and Xuan Gong

(cherry picked from commit 4546c7582b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/RMAppImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/TestRMAppTransitions.java
 2016-07-10 08:45:49 -07:00
Li Lu b7d484aead YARN-4450. TestTimelineAuthenticationFilter and 
TestYarnConfigurationFields fail. Contributed by Sangjin Lee.
 2016-07-10 08:45:49 -07:00
Li Lu 89e5c44f9e YARN-4356. Ensure the timeline service v.2 is disabled cleanly and has no 
impact when it's turned off. Contributed by Sangjin Lee.
 2016-07-10 08:45:48 -07:00
Junping Du cf4666bb8c YARN-3623. Add a config to indicate the Timeline Service version. Contributed by Xuan Gong. 
(cherry picked from commit f910e4f639)
 2016-07-10 08:45:48 -07:00
Sangjin Lee 1f710484e5 YARN-3862. Support for fetching specific configs and metrics based on prefixes (Varun Saxena via sjlee) 2016-07-10 08:45:48 -07:00
Sangjin Lee ae72f1dc77 YARN-4297. TestJobHistoryEventHandler and TestRMContainerAllocator failing on YARN-2928 branch (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee 51254a6b51 YARN-4053. Change the way metric values are stored in HBase Storage (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee 09649005ca YARN-4221. Store user in app to flow table (Varun Saxena via sjlee) 2016-07-10 08:45:47 -07:00
Sangjin Lee 10ec5586fb YARN-4129. Refactor the SystemMetricPublisher in RM to better support newer events (Naganarasimha G R via sjlee) 2016-07-10 08:45:46 -07:00
Sangjin Lee e3e857866d YARN-4179. [reader implementation] support flow activity queries based on time (Varun Saxena via sjlee) 2016-07-10 08:45:46 -07:00
Li Lu d014f2ffd2 YARN-4237 Support additional queries for ATSv2 Web UI. Contributed by 
Varun Saxena.
 2016-07-10 08:45:46 -07:00
Sangjin Lee b51d0fef56 YARN-4178. [storage implementation] app id as string in row keys can cause incorrect ordering (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Sangjin Lee 0f44b5508d YARN-3864. Implement support for querying single app and all apps for a flow run (Varun Saxena via sjlee) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan 708fa8b1ae YARN-4210. HBase reader throws NPE if Get returns no rows (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan da2b7bd08e YARN-4203. Add request/response logging & timing for each REST endpoint call (Varun Saxena via vrushali) 2016-07-10 08:45:45 -07:00
Vrushali Channapattan d95dc89a02 YARN-4075 [reader REST API] implement support for querying for flows and flow runs (Varun Saxena via vrushali) 2016-07-10 08:45:44 -07:00
Vrushali 10fa6da7d8 YARN-4074. [timeline reader] implement support for querying for flows and flow runs (sjlee via vrushali) 2016-07-10 08:45:44 -07:00
Sangjin Lee a68e383921 YARN-3901. Populate flow run data in the flow_run & flow activity tables (Vrushali C via sjlee) 2016-07-10 08:45:44 -07:00
Sangjin Lee ee081018e4 YARN-4102. Add a "skip existing table" mode for timeline schema creator (Li Lu via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee 8d9476ec5f YARN-4058. Miscellaneous issues in NodeManager project (Naganarasimha G R via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee 451c13b1de YARN-3814. REST API implementation for getting raw entities in TimelineReader (Varun Saxena via sjlee) 2016-07-10 08:45:43 -07:00
Sangjin Lee c7224a285c YARN-4064. build is broken at TestHBaseTimelineWriterImpl.java (sjlee) 2016-07-10 08:45:43 -07:00
Junping Du 7a41b5501e YARN-4025. Deal with byte representations of Longs in writer code. Contributed by Sangjin Lee and Vrushali C. 2016-07-10 08:45:42 -07:00
Junping Du 477a30f536 YARN-3045. Implement NM writing container lifecycle events to Timeline Service v2. Contributed by Naganarasimha G R. 2016-07-10 08:45:42 -07:00
Sangjin Lee 102b56ee96 YARN-3904. Refactor timelineservice.storage to add support to online and offline aggregation writers (Li Lu via sjlee) 2016-07-10 08:45:42 -07:00
Zhijie Shen 5088f6c76a MAPREDUCE-6337. Added a mode to replay MR job history files and put them into the timeline service v2. Contributed by Sangjin Lee. 
(cherry picked from commit 463e070a8e7c882706a96eaa20ea49bfe9982875)
 2016-07-10 08:45:41 -07:00