Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,732 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3788 Commits

Author SHA1 Message Date
Jason Lowe 6ad6882343 YARN-7286. Add support for docker to have no capabilities. Contributed by Eric Badger 
(cherry picked from commit b7dee1f060)
 2017-11-02 09:50:55 -05:00
Rohith Sharma K S e53fcd52ee addendum patch for YARN-7289. 
(cherry picked from commit 940ffe3f9c)
 2017-11-02 14:07:28 +05:30
Junping Du d1f8854a1b YARN-7400. Incorrect log preview displayed in jobhistory server ui. Contributed by Xuan Gong. 
(cherry picked from commit d57dba9942)
 2017-11-01 14:50:44 -07:00
Haibo Chen 32a712a740 YARN-7389. Make TestResourceManager Scheduler agnostic. (Robert Kanter via Haibo Chen) 
(cherry picked from commit d7f3737f3b)
 2017-11-01 14:19:59 -07:00
Daniel Templeton 956d814034 YARN-7146. Many RM unit tests failing with FairScheduler 
(Contributed by Robert Kanter via Daniel Templeton)
 2017-11-01 14:11:26 -07:00
Inigo Goiri 8180ab436a YARN-7276. Federation Router Web Service fixes. Contributed by Inigo Goiri. 2017-11-01 13:21:15 -07:00
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Subru Krishnan ac41107423 YARN-6413. FileSystem based Yarn Registry implementation. (Ellen Hui via Subru). 
(cherry picked from commit ed24da3dd7)
 2017-10-31 12:11:40 -07:00
Junping Du 76ec5eab12 HADOOP-14990. Clean up jdiff xml files added for 2.8.2 release. 
(cherry picked from commit a8083aaa6f)
 2017-10-30 15:17:30 -07:00
Vrushali C 3454c02c96 YARN-7378 Documentation changes post branch-2 merge (Vrushali C with Varun Saxena) 2017-10-30 14:09:29 -07:00
Yufei Gu 243b70eca8 YARN-6747. TestFSAppStarvation.testPreemptionEnable fails intermittently. (Contributed by Miklos Szegedi) 
(cherry picked from commit d4811c8cfa)
 2017-10-29 16:47:22 -07:00
Varun Saxena e1b3bbde60 YARN-7380. Fix findbugs warning in timeline service on branch-2 (Vrushali C via Varun Saxena). 2017-10-29 16:38:20 +05:30
Arun Suresh 2654d9973e YARN-7299. Fix TestDistributedScheduler. (asuresh) 
(cherry picked from commit 9c5c68745e)
 2017-10-27 23:11:34 -07:00
Arun Suresh 519aa0e804 YARN-7178. Add documentation for Container Update API. (asuresh) 
(cherry picked from commit 24f8c5cce3)
 2017-10-27 22:57:46 -07:00
Jason Lowe 9093ad6168 YARN-7244. ShuffleHandler is not aware of disks that are added. Contributed by Kuhu Shukla 
(cherry picked from commit 665bb147aa)
 2017-10-27 17:09:29 -05:00
Rohith Sharma K S 548116c3f5 YARN-7289. Application lifetime does not work with FairScheduler. Contributed by Miklos Szegedi. 
(cherry picked from commit 5c799ecf09)
 2017-10-27 22:51:41 +05:30
Sunil G 88c3b444bd YARN-7375. Possible NPE in RMWebapp when HA is enabled and the active RM fails. Contributed by Chandni Singh. 
(cherry picked from commit 792388e1c0)
 2017-10-27 21:11:48 +05:30
Junping Du 1f2a2ba2be Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
 2017-10-26 23:20:24 -07:00
Subru Krishnan a0dceec586 YARN-5516. Add REST API for supporting recurring reservations. (Sean Po via Subru). 
(cherry picked from commit 25932da6d1)
 2017-10-26 20:16:01 -07:00
Arun Suresh f6fde85c5c YARN-4823. Refactor the nested reservation id field in listReservation to simple string field. (subru via asuresh) 
(cherry picked from commit 00bebb7e58)
 2017-10-26 20:04:33 -07:00
Arun Suresh 098a6b8f19 YARN-4825. Remove redundant code in ClientRMService::listReservations. (subru via asuresh) 
(cherry picked from commit d82e797b65)
 2017-10-26 20:04:19 -07:00
Robert Kanter 7c26ae506a YARN-7262. Add a hierarchy into the ZKRMStateStore for delegation token znodes to prevent jute buffer overflow (rkanter) 
(cherry picked from commit b1de78619f)
 2017-10-26 17:51:05 -07:00
Robert Kanter 46783158f8 YARN-7358. TestZKConfigurationStore and TestLeveldbConfigurationStore should explicitly set capacity scheduler (haibochen via rkanter) 
(cherry picked from commit 2da654e34a)
 2017-10-26 10:49:57 -07:00
Subru Krishnan 74aee8673f YARN-4827. Document configuration of ReservationSystem for FairScheduler. (Yufei Gu via Subru). 
(cherry picked from commit 3fae675383)
 2017-10-25 15:32:03 -07:00
Arun Suresh 774a575363 YARN-4687. Document Reservation ACLs (Contributed by Sean Po) 
(cherry picked from commit 2e1d0ff4e9)
 2017-10-25 15:29:32 -07:00
Carlo Curino fa86fdc3cc YARN-7339. LocalityMulticastAMRMProxyPolicy should handle cancel request properly. (Botong Huang via curino) 
Edited cherry-pick from 1c5c2b5dde
 2017-10-25 13:56:35 -07:00
Eric Yang f6d3c101a6 YARN-7353. Improved volume mount check for directories and unit test compatibility on RHEL7. Contributed by Eric Badger. 2017-10-24 17:01:08 -04:00
Inigo Goiri 89ad6adab3 YARN-3661. Basic Federation UI. Contributed by Inigo Goiri. 2017-10-24 13:55:13 -07:00
Robert Kanter 85eadec67b YARN-7385. TestFairScheduler#testUpdateDemand and TestFSLeafQueue#testUpdateDemand are failing with NPE (yufeigu via rkanter) 
(cherry picked from commit 03af442e76)
 2017-10-24 13:43:27 -07:00
Robert Kanter 1d34a4805e YARN-7382. NoSuchElementException in FairScheduler after failover causes RM crash (rkanter) 
(cherry picked from commit 025c656572)
 2017-10-24 10:29:36 -07:00
Daniel Templeton 08cfd010a3 YARN-7357. Several methods in TestZKRMStateStore.TestZKRMStateStoreTester.TestZKRMStateStoreInternal should have @Override annotations 
(Contributed by Sen Zhao via Daniel Templeton)

(cherry picked from commit 0553f5a012e14c8ceb40a16161440384f9f3a9a8)
(cherry picked from commit 9da9023371)
 2017-10-23 13:52:15 -07:00
Eric Payne ef0d6ff2e6 YARN-4163: Audit getQueueInfo and getApplications calls 
(cherry picked from commit 921338cd86)
 2017-10-23 11:58:04 -05:00
Subru Krishnan 9bd77a6d9d YARN-6871. Add additional deSelects params in RMWebServices#getAppReport. (Tanuj Nayak via Subru). 2017-10-20 17:19:29 -07:00
Yufei Gu 93d71d97d4 YARN-7355. TestDistributedShell should be scheduler agnostic. (Contributed by Haibo) 
(cherry picked from commit 6b7c87c945)
 2017-10-20 11:49:34 -07:00
Yufei Gu 378fa3ac7d YARN-7261. Add debug message for better download latency monitoring. (Yufei Gu) 
(cherry picked from commit 0799fde35e)
 2017-10-20 11:49:34 -07:00
Yufei Gu 4c4f28c9e7 YARN-4090. Make Collections.sort() more efficient by caching resource usage. (Contributed by Yufei Gu, Shilong Zhang and Xianyin Xin) 
(cherry picked from commit 1f4cdf1068)
(cherry picked from commit 96106b8f5f)
 2017-10-20 11:49:33 -07:00
Yufei Gu f462461d4e YARN-7359. TestAppManager.testQueueSubmitWithNoPermission() should be scheduler agnostic. (Contributed by Haibo Chen) 
(cherry picked from commit 7b4b018780)
 2017-10-20 11:22:28 -07:00
Yufei Gu 1000811bcb YARN-7294. TestSignalContainer#testSignalRequestDeliveryToNM fails intermittently with Fair Scheduler. (Contributed by Miklos Szegedi) 
(cherry picked from commit cbd2b73ef8)
 2017-10-20 11:22:28 -07:00
Wangda Tan 2f476f4b2c YARN-6623. Add support to turn off launching privileged containers in the container-executor. (Varun Vasudev via wangda) 2017-10-20 11:22:28 -07:00
Wangda Tan 6a3929f2b2 YARN-6608. Backport all SLS improvements from trunk to branch-2. (Carlo Curino via wangda) 2017-10-20 11:22:26 -07:00
Wangda Tan bc2326f5f0 YARN-7170. Improve bower dependencies for YARN UI v2. (Sunil G via wangda) 2017-10-19 22:20:36 -07:00
vrushali 28c54989d6 Ensuring right apis are called as per jetty version in branch2 (trunk has different jetty) 2017-10-19 22:18:17 -07:00
Wangda Tan 017ac560f0 YARN-7338. Support same origin policy for cross site scripting prevention. (Sunil G via wangda) 2017-10-19 21:48:11 -07:00
vrushali 3a5f1a6c26 Fixing checktyle warnings during rebase 2017-10-18 02:06:49 +05:30
Sunil G ec3fb45772 YARN-7022. Improve click interaction in queue topology in new YARN UI. Contributed by Abdullah Yousufi. 2017-10-18 02:06:49 +05:30
Sunil G da104a3e78 YARN-7075. Better styling for donut charts in new YARN UI. Contributed by Da Ding. 2017-10-18 02:06:49 +05:30
Sunil G 1cf7ed8a28 YARN-7071. Add vcores and number of containers in new YARN UI node heat map. Contributed by Abdullah Yousufi. 2017-10-18 02:06:49 +05:30
Sunil G 65b90e1e1c YARN-6386. Show decommissioning nodes in new YARN UI. Contributed by Elek Marton. 2017-10-18 02:06:49 +05:30
Sunil G 1a1f39c98c YARN-5146. Support for Fair Scheduler in new YARN UI. Contributed by Abdullah Yousufi. 2017-10-18 02:06:49 +05:30
Sunil G 1635741390 YARN-6631. Refactor loader.js in new Yarn UI. Contributed by Akhil P B. 2017-10-18 02:06:49 +05:30
vrushali 6232c3aef3 Updating dependencies to work on branch-2 2017-10-18 02:06:49 +05:30
Sunil G 3aa8111594 YARN-6458. Use yarn package manager to lock down dependency versions for new web UI. Contributed by Sreenath Somarajapuram. 2017-10-18 02:06:49 +05:30
Sunil G 9e4e218ff0 YARN-6635. Refactor yarn-app pages in new YARN UI. Contributed by Akhil PB. 2017-10-18 02:06:49 +05:30
Sunil G dd40f6f0cc YARN-5705. Show timeline data from ATS v2 in new web UI. Contributed by Akhil PB. 2017-10-18 02:06:49 +05:30
Sunil G 458ba54ea5 YARN-6291. Introduce query parameters (sort, filter, etc.) for tables to keep state on refresh/navigation in new YARN UI. Contributed by Gergely Novák. 2017-10-18 02:06:48 +05:30
Sunil G c069f4f937 YARN-6402. Move 'Long Running Services' to an independent tab at top level for new Yarn UI. Contributed by Akhil PB. 2017-10-18 02:06:48 +05:30
Sunil G bef6b2f030 YARN-5153. Add a toggle button to switch between timeline view / table view for containers and application-attempts in new YARN UI. Contributed by Akhil PB. 2017-10-18 02:06:48 +05:30
Sunil G deef8d5446 YARN-6258. localBaseAddress for CORS proxy configuration is not working when suffixed with forward slash in new YARN UI. Contributed by Gergely Novák. 2017-10-18 02:06:48 +05:30
Sunil G 16cc282725 YARN-5496. Make Node Heatmap Chart categories clickable in new YARN UI. Contributed by Gergely Novák. 2017-10-18 02:06:48 +05:30
Sunil G bbd98cb019 YARN-6196. Improve Resource Donut chart with better label in Node page of new YARN UI. Contributed by Akhil PB. 2017-10-18 02:06:48 +05:30
Sunil G 03511fdaf8 YARN-5335. Use em-table in app/nodes pages for new YARN UI. Contributed by Sunil G. 2017-10-18 02:06:48 +05:30
Sunil G 148c7b8689 YARN-6184. Introduce loading icon in each page of new YARN UI. Contributed by Akhil PB. 2017-10-18 02:06:48 +05:30
Sunil G 981ccd9d90 YARN-6183. Few missing informations in Application and Application Attempt pages for new YARN UI. Contributed by Akhil PB. 2017-10-18 02:06:48 +05:30
Sunil G 9071c712d9 YARN-5912. Fix breadcrumb issues for various pages in new YARN UI. Contributed by Akhil P B. 2017-10-18 02:06:48 +05:30
Sunil G 2c93e8ee13 YARN-5917. Make navigation link active when selecting sub tabs in 'Applications' and 'Nodes' page for new UI. Contributed by Kai Sasaki. 2017-10-18 02:06:48 +05:30
Sunil G 0d1a53ad8e YARN-5866. Fix few issues reported by jshint in new YARN UI. Contributed by Akhil P B. 2017-10-18 02:06:48 +05:30
Sunil G d5271b03d3 YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB. 2017-10-18 02:06:48 +05:30
sunilg 011402e304 YARN-4849. Addendum patch to improve pom for yarn-ui. (Wangda Tan via Sunil G) 2017-10-18 02:06:48 +05:30
sunilg 4fba66c28e YARN-5785. [YARN-3368] Accessing applications and containers list from Node page is throwing few exceptions in console. (Akhil P B and Sreenath Somarajapuram via Sunil G) 2017-10-18 02:06:48 +05:30
Sreenath Somarajapuram 6c68032291 YARN-5779. [YARN-3368] Addendum patch to document limits/notes of the new YARN UI . (Sunil G via Sreenath Somarajapuram) 2017-10-18 02:06:48 +05:30
sunilg 9ebb57dfc8 YARN-5490. [YARN-3368] Fix various alignment issues and broken breadcrumb link in Node page. (Akhil P B Tan via Sunil G) 2017-10-18 02:06:48 +05:30
sunilg 36f5091d6c YARN-5497. [YARN-3368] Use different color for Undefined and Succeeded for Final State in applications page. (Akhil P B Tan via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg 94c53e7985 YARN-5500. [YARN-3368] ‘Master node' link under application tab is broken. (Akhil P B Tan via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg addc399e8b YARN-5772. [YARN-3368] Replace old Hadoop logo with new one (Akhil P B Tan via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg 43f0627a8e YARN-5779. [YARN-3368] Document limits/notes of the new YARN UI (Wangda Tan via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg 29fc43e5a4 YARN-5741. [YARN-3368] Update UI2 documentation for new UI2 path (Kai Sasaki and Wangda Tan via Sunil G) 2017-10-18 02:06:47 +05:30
Wangda Tan 7513315758 YARN-5145. [YARN-3368] Move new YARN UI configuration to HADOOP_CONF_DIR. (Sunil G and Kai Sasaki via wangda) 2017-10-18 02:06:47 +05:30
sunilg f1de967c67 YARN-4849. Addendum patch to remove unwanted files from rat exclusions. (Wangda Tan via Sunil G) 2017-10-18 02:06:47 +05:30
Wangda Tan c0c124c2bf YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda) 2017-10-18 02:06:47 +05:30
sunilg 8222307c30 YARN-5682. [YARN-3368] Fix maven build to keep all generated or downloaded files in target folder (Wangda Tan via Sunil G) 2017-10-18 02:06:47 +05:30
Wangda Tan e5c0219228 YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda) 2017-10-18 02:06:47 +05:30
sunilg 542f661cc1 YARN-4849. Addendum patch to fix ASF warnings. (Wangda Tan via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg dd8ad34305 YARN-5503. [YARN-3368] Add missing hidden files in webapp folder for deployment (Sreenath Somarajapuram via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg ade1dd96b5 YARN-5504. [YARN-3368] Fix YARN UI build pom.xml (Sreenath Somarajapuram via Sunil G) 2017-10-18 02:06:47 +05:30
vrushali 1510fea062 Updating pom to be 2.9.0-SNAPSHOT instead of 3.0 2017-10-18 02:06:47 +05:30
sunilg 48336218d4 YARN-4849. Addendum patch to fix document. (Wangda Tan via Sunil G) 2017-10-18 02:06:47 +05:30
sunilg a4021c9c36 YARN-5488. [YARN-3368] Applications table overflows beyond the page boundary(Harish Jaiprakash via Sunil G) 2017-10-18 02:06:46 +05:30
Wangda Tan a53e981adf YARN-5509. Build error due to preparing 3.0.0-alpha2 deployment. (Kai Sasaki via wangda) 2017-10-18 02:06:46 +05:30
sunilg 3e0dab4bfd YARN-3334. [YARN-3368] Introduce REFRESH button in various UI pages (Sreenath Somarajapuram via Sunil G) 2017-10-18 02:06:46 +05:30
Sunil 30d1839aa0 YARN-5321. [YARN-3368] Add resource usage for application by node managers (Wangda Tan via Sunil G) 
YARN-5320. [YARN-3368] Add resource usage by applications and queues to cluster overview page  (Wangda Tan via Sunil G)
YARN-5322. [YARN-3368] Add a node heat chart map (Wangda Tan via Sunil G)
YARN-5347. [YARN-3368] Applications page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5348. [YARN-3368] Node details page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5346. [YARN-3368] Queues page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5345. [YARN-3368] Cluster overview page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5344. [YARN-3368] Generic UI improvements (Sreenath Somarajapuram via Sunil G)
 2017-10-18 02:06:46 +05:30
Sunil b41ca1a6ea YARN-5161. [YARN-3368] Add Apache Hadoop logo in YarnUI home page. (Kai Sasaki via Sunil G) 2017-10-18 02:06:46 +05:30
Sunil 0490942c58 YARN-5183. [YARN-3368] Support for responsive navbar when window is resized. (Kai Sasaki via Sunil G) 2017-10-18 02:06:46 +05:30
Wangda Tan bb606e3d4b YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda) 
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
 2017-10-18 02:06:46 +05:30
Wangda Tan 67aa886ce7 YARN-5019. [YARN-3368] Change urls in new YARN ui from camel casing to hyphens. (Sunil G via wangda) 2017-10-18 02:06:46 +05:30
Wangda Tan b414b5f2cf YARN-4514. [YARN-3368] Cleanup hardcoded configurations, such as RM/ATS addresses. (Sunil G via wangda) 2017-10-18 02:06:46 +05:30
Wangda Tan 56654d8820 YARN-4849. [YARN-3368] cleanup code base, integrate web UI related build to mvn, and fix licenses. (wangda) 2017-10-18 02:06:46 +05:30
Wangda Tan 217a87e975 YARN-4517. Add nodes page and fix bunch of license issues. (Varun Saxena via wangda) 2017-10-18 02:06:46 +05:30
Wangda Tan 8a75207d4c YARN-4733. [YARN-3368] Initial commit of new YARN web UI. (wangda) 2017-10-18 02:06:46 +05:30
Subru Krishnan 5e7491d532 YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru). 
(cherry picked from commit 75323394fb)
 2017-10-17 12:43:36 -07:00
Haibo Chen c0ee583656 YARN-7341. TestRouterWebServiceUtil#testMergeMetrics is flakey. (Robert Kanter via Haibo Chen) 
(cherry picked from commit acabc657ff)
 2017-10-17 12:30:26 -07:00
Subru Krishnan adf5ea73d6 YARN-7095. Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru). 
(cherry picked from commit bac4e8cca8)
 2017-10-17 12:14:05 -07:00
Varun Saxena 56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
vrushali 8b98dba7ee YARN-7190 Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:41 +05:30
Varun Saxena 6ba1f34532 Fix problems in yarn.cmd due to rebase. 2017-10-17 23:36:41 +05:30
Haibo Chen 572ebd1a95 YARN-7063. TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2. (Varun Saxena via Haibo Chen) 2017-10-17 23:36:41 +05:30
Haibo Chen d77530b227 YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen) 
(cherry picked from commit 661f5eb0c6)
 2017-10-17 23:36:41 +05:30
Rohith Sharma K S 27019b5a66 YARN-7130. ATSv2 documentation changes post merge. Contributed by Varun Saxena. 
(cherry picked from commit 0bfb3a2566)
 2017-10-17 23:36:41 +05:30
Rohith Sharma K S 138d6fc083 YARN-7140. CollectorInfo should have Public visibility. Contributed by Varun Saxena. 
(cherry picked from commit 4a83170be4)
 2017-10-17 23:36:40 +05:30
Varun Saxena 95f45b89d1 YARN-7152. [ATSv2] Registering timeline client before AMRMClient service init throw exception (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ef87d34a0e)
 2017-10-17 23:36:40 +05:30
Junping Du b3201b2dab YARN-6534. ResourceManager failed due to TimelineClient try to init SSLFactory even https is not enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit 09e36ff59c05b8bd54d346d1103de0a7300875c6)
(cherry picked from commit 94607214ce)
 2017-10-17 23:36:40 +05:30
Haibo Chen 6e2de8b417 YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 461ee44d28)
 2017-10-17 23:36:40 +05:30
bibinchundatt f7637b55bb YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena 
(cherry picked from commit dcd0bedcc8)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHATimelineCollectors.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/security/TimelineReaderWhitelistAuthorizationFilter.java
 2017-10-17 23:36:40 +05:30
Varun Saxena 1289b056ac YARN-6958. Addedndum for Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice 2017-10-17 23:36:40 +05:30
Varun Saxena 87713f41b6 YARN-6905. Addendum to fix TestTimelineReaderWebServicesHBaseStorage due to missing FastNumberFormat 
(cherry picked from commit e52f26e5cf)
 2017-10-17 23:36:40 +05:30
Vrushali C a2598434ed YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen) 
(cherry picked from commit 608a06cca5)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java
 2017-10-17 23:36:40 +05:30
Varun Saxena 78a6ecccb3 Addendum for YARN-4753. Use doxia macro to generate in-page TOC of YARN site documentation 2017-10-17 23:36:40 +05:30
Varun Saxena 3d5bf1c06d YARN-6047. Documentation updates for TimelineService v2. Contributed by Rohith Sharma K S, Vrushali C and Varun Saxena 2017-10-17 23:36:40 +05:30
Varun Saxena a22c1c72c4 YARN-6316. Addendum for missing create option in documentation. 2017-10-17 23:36:40 +05:30
Varun Saxena 28bbe9ccd7 YARN-5577. [Atsv2] Document object passing in infofilters with an example (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ea29e3bc27)
 2017-10-17 23:36:40 +05:30
Haibo Chen 0f51f15097 YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen) 2017-10-17 23:36:40 +05:30
Jian He cbd5e89b24 MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 
(cherry picked from commit c6ce422a3423305ac665f80f039f61294adaa32c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2017-10-17 23:36:40 +05:30
Varun Saxena e5d63652df YARN-6861. Addendum for YARN-6861 for compilation issue (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:40 +05:30
Varun Saxena 186c99c6cc YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 73ee0d4fc05354527a4be6db9e5b83d06fc94047)
 2017-10-17 23:36:39 +05:30
Jian He 5c970f7c29 YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 5d9b4bd3e8 YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena. 
(cherry picked from commit e976bf10bb0d42e4d971358554ea736ddcdafd19)
 2017-10-17 23:36:39 +05:30
Jian He 53ddf1b15f YARN-7038. Addendum patch. Fix compilation error. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 4e1af68ae1 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 
(cherry picked from commit 387dd3da87759860dd18bd8aa95c60d962b1e6a9)
 2017-10-17 23:36:39 +05:30
Jian He 55e5742394 YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Jason Lowe db2f7dd9bd YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C 2017-10-17 23:36:39 +05:30
Rohith Sharma K S cb75e1826e YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena. 
(cherry picked from commit ffb1f572b68e73efd6410ab74a334f4e5df543f2)
 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Varun Saxena 5bda387573 YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena) 
(cherry picked from commit 7b2cb0614a04e6d0e2bc6dfd5095b7535dffa678)
 2017-10-17 23:36:39 +05:30
Varun Saxena 8d9d40aa53 Addendum for YARN-6102. 2017-10-17 23:36:39 +05:30
Varun Saxena cb5d601b2e Addendum for YARN-5648. 2017-10-17 23:36:39 +05:30
Jian He 8e4e5abbe6 YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena 
(cherry picked from commit 3829100cf75e733b2b3a4e8c6806a3696f4ae658)
 2017-10-17 23:36:39 +05:30
Jian He 50dd1372bf YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 7f06c46065936b88e9864c526bfcda3a89f3e257)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/AbstractTimelineReaderHBaseTestBase.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
 2017-10-17 23:36:39 +05:30
Jian He 9577900a91 YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 79dae624793164cd4692c86992a511310145858a)
 2017-10-17 23:36:39 +05:30
Varun Saxena c8db08dfac YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 0443928d771e3e21825b4f487e8c0865ea641970)
 2017-10-17 23:36:38 +05:30
Sunil G 4b65fe73ca YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 
(cherry picked from commit a657472b42)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 4bea8af723 YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C. 
(cherry picked from commit 20dd6d18b7787e67ef96f3b6b92ea8415a8650fd)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 4014390ef4 YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C. 
(cherry picked from commit 416312e3a7f0f83cc1c8549640c3ba75aaa3d887)
 2017-10-17 23:36:38 +05:30
Vrushali C 0f751a7fec YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 484d7e9b39 YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena. 
(cherry picked from commit 743e4731781a7d9a4a5f1f09adc510f193182158)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S d01a3f1ba2 YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C. 
(cherry picked from commit 5791ced4bd864ff6c08cb2c3ee10ab62008f0168)
 2017-10-17 23:36:38 +05:30
Varun Saxena c4f042d4c1 YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena) 
(cherry picked from commit 0b7bff706e)
 2017-10-17 23:36:38 +05:30
vrushali 48d2e4f519 YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30
Varun Saxena 473f2f00ee YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena) 
(cherry picked from commit f7cba98504a5e968bb6ae976d20b71c84a4664f2)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/apptoflow/AppToFlowColumn.java
 2017-10-17 23:36:38 +05:30
Haibo Chen d53038626e YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 47474fffac)
(cherry picked from commit 8817cb5c8424359b880c6d700e53092f0269c1bb)
 2017-10-17 23:36:38 +05:30
Haibo Chen 9b4d4228fb YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen) 
(cherry picked from commit 4c8b9d23a1b4f92bf370617a0b42ac68bdcf33ac)
 2017-10-17 23:36:38 +05:30
Vrushali Channapattan 70315de4ff YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30
Vrushali Channapattan 34570d4fc3 YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C) 2017-10-17 23:36:38 +05:30
Naganarasimha e1d2f6ebdd YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang. 
(cherry picked from commit 0f1af3178e)
 2017-10-17 23:36:37 +05:30
Varun Saxena 7338dbe5c0 YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena) 
(cherry picked from commit 793bbf216d)
 2017-10-17 23:36:37 +05:30
Varun Saxena c1f9f0529b Addendum for YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611 2017-10-17 23:36:37 +05:30
Varun Saxena ebcb575a12 Addendum for YARN-4218. Metric for resource*time that was preempted. 2017-10-17 23:36:37 +05:30
Rohith Sharma K S 1072232741 YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena. 
(cherry picked from commit 1a9439e299)
 2017-10-17 23:36:37 +05:30
Varun Saxena 16d8df68f6 YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena) 
(cherry picked from commit a4b5aa8493)
 2017-10-17 23:36:37 +05:30
Varun Saxena fd88fa8083 YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena) 
(cherry picked from commit b58777a9c9)
 2017-10-17 23:36:37 +05:30
Varun Saxena debe526f08 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7c2bc444b3)
 2017-10-17 23:36:37 +05:30
Varun Saxena 203077cf2c YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena) 
(cherry picked from commit 063b513b1c)
 2017-10-17 23:36:37 +05:30
Varun Saxena 9595c18eea YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7f09cd1d5ee70e0bdf24cdecd78cd3bc258a40f8)
 2017-10-17 23:36:37 +05:30
Varun Saxena 1369abf5b0 YARN-6318. timeline service schema creator fails if executed from a remote machine (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 68ec0d9b471356d3adef15d78111126f57ee50f8)
 2017-10-17 23:36:37 +05:30
Varun Saxena 6ec294b7fc YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ec32c4265bbf436ae61b6efc71ca33b16b70a23d)
 2017-10-17 23:36:37 +05:30
Varun Saxena e597f2de60 YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 5d9ad152a1082a7c9b8edaf57a88ae471a537599)
 2017-10-17 23:36:37 +05:30
Sangjin Lee 16c607ca2b YARN-6027 addendum. Fixed the broken build for YARN-5355-branch-2. Contributed by Sangjin Lee. 2017-10-17 23:36:37 +05:30
Varun Saxena d996afd585 YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 63c06ec44e633567c378e28898e319143593ff30)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java
 2017-10-17 23:36:36 +05:30
Sangjin Lee 6b5b8f090d YARN-6253. FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp. Contributed by Haibo Chen. 
(cherry picked from commit 34e7c30293b5a56f9f745769a29c5666bdb85d6c)
 2017-10-17 23:36:36 +05:30
Varun Saxena 59d96e247e YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ab192fd58358faff6880f3e0e867d7bdd4bb043a)
 2017-10-17 23:36:36 +05:30
Varun Saxena 4a3a4aaf7e YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R via Varun Saxena) 
(cherry picked from commit 6ba61d20d3)
 2017-10-17 23:36:36 +05:30
Varun Saxena 74e43bf287 YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 649deb72fb)
 2017-10-17 23:36:36 +05:30
Sangjin Lee bb100d35fd YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 
(cherry picked from commit 73235ab30361b41293846189f3c5fef321ae7cac)
 2017-10-17 23:36:36 +05:30
Sangjin Lee a16a684fbb YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen. 
(cherry picked from commit b92089c0e8ab1b87b8b5b55b1e3d4367ae5d847a)
 2017-10-17 23:36:36 +05:30
Sangjin Lee 5cb5b87c9a YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 
(cherry picked from commit 0327a79d79a4d56d9c7cb6889886afd2272b07d3)
 2017-10-17 23:36:36 +05:30
Varun Saxena 8c81f7fc08 Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's 
(cherry picked from commit c9246f619104bff44dd453a4ffe70104d37fd781)
 2017-10-17 23:36:36 +05:30
Varun Saxena aa09001a33 YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 4b1ba4ea314147f8a06cc4f446c1d9336de89fc1)
 2017-10-17 23:36:36 +05:30
Varun Saxena 3429517a00 YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 6baea680ba6e5df6f254ced086d6defa64fb99f0)
 2017-10-17 23:36:36 +05:30
Sangjin Lee 5b3ef55f5f YARN-5980. Update documentation for single node hbase deploy. Contributed by Vrushali C. 
(cherry picked from commit e1bdba77888723b435a235a96c8659029afd25d5)
 2017-10-17 23:36:36 +05:30
Varun Saxena d12920b10e YARN-6074. FlowRunEntity does not deserialize long values correctly (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit db490eccce)
 2017-10-17 23:36:36 +05:30
Varun Saxena 1e5e4a52f7 YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 9a2f288a6a9e30376b7fd99bf8824184a34a54c9)
 2017-10-17 23:36:35 +05:30
Naganarasimha d3ef478955 YARN-6026. A couple of spelling errors in the docs. Contributed by Grant Sohn. 2017-10-17 23:36:35 +05:30
Sangjin Lee 6aa7510681 YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C. 
(cherry picked from commit f945008d1cf5730bdebeae501ed0e42477ad219e)
 2017-10-17 23:36:35 +05:30
Sangjin Lee 3394cb25d2 YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen. 
(cherry picked from commit 8288030cb4aa3b5a9425cc0a3f6df03a3eae1dfb)
 2017-10-17 23:36:35 +05:30
Sangjin Lee 54705f549f YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen. 
(cherry picked from commit a5a55a54ab)
(cherry picked from commit 17c4ab7ebb51088caf36fafedae8c256481eeed5)
 2017-10-17 23:36:35 +05:30
Naganarasimha a5371f6b6e YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena. 
(cherry picked from commit c06114d6a3)
 2017-10-17 23:36:35 +05:30
Sangjin Lee d6d73e3c99 YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee. 
(cherry picked from commit f511cc89b6)
 2017-10-17 23:36:35 +05:30
Sangjin Lee 4f595889e7 YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu. 
(cherry picked from commit 3b2e80881eaeaa82667496d706496765ed7e29f5)
 2017-10-17 23:36:35 +05:30
Vrushali Channapattan 2fa2c3169e YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (Varun Saxena via Vrushali C) 
(cherry picked from commit 513dcf6817dd76fde8096ff04cd888d7c908461d)
(cherry picked from commit 022bf783aa)
 2017-10-17 23:36:35 +05:30
Sangjin Lee b8cfb4fcb3 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena. 
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
 2017-10-17 23:36:35 +05:30
Varun Saxena 684c2d149f YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena) 
(cherry picked from commit 25b19178dd53bc0d57ca5fd64d3464af7b59e588)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
 2017-10-17 23:36:35 +05:30
Varun Saxena 484b1d2ec1 YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:34 +05:30
Varun Saxena 6562a354a9 YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena) 
(cherry picked from commit 1c8a57550b)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 4182f892e0 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis. 
(cherry picked from commit dc7d9bac79fc303f66e97c10379d680e0c11caa4)
 2017-10-17 23:36:34 +05:30
Sangjin Lee ebf2f90528 YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S. 
(cherry picked from commit f37288c7e0127b564645e978c7aab2a186fa6be6)
 2017-10-17 23:36:34 +05:30
Varun Saxena d19726ec37 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-10-17 23:36:34 +05:30
Varun Saxena 1d7fc52578 YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena) 2017-10-17 23:36:34 +05:30
Sangjin Lee 3344f0ace5 YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S. 
(cherry picked from commit e9c4616b5e)
 2017-10-17 23:36:34 +05:30
Varun Saxena c3db98aa10 YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:34 +05:30
Sangjin Lee 853a5da041 YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S. 
(cherry picked from commit 787d5942003a5441ba455a79fa08da28f60d5777)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 4675f22e6a YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu. 
(cherry picked from commit 0c1863144649ea265da65ce25158707cc3a3fb4a)
 2017-10-17 23:36:34 +05:30
Li Lu 6a517d714e YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9) 
(cherry picked from commit 5d7ad396dbf84ffb0bacb3372d149828dfb4b4ca)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 51102019f5 YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee) 
(cherry picked from commit d0a62d8adf86f7cb2481940d212788cc5f327367)
 2017-10-17 23:36:34 +05:30
Sangjin Lee 4f6ba3187b YARN-5379. TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently (Vrushali C via sjlee) 
(cherry picked from commit e549a9af31)
 2017-10-17 23:36:34 +05:30
Naganarasimha 22f5046acb YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee 
(cherry picked from commit af8f480c24)
 2017-10-17 23:36:34 +05:30
Varun Saxena 057a4728ff YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 56142171b9)
 2017-10-17 23:36:33 +05:30
Varun Saxena 16942ea115 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena). 
(cherry picked from commit a3ac1c1943)
 2017-10-17 23:36:33 +05:30
Sangjin Lee e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Nathan Roberts b876a9314e YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe. 
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
 2017-10-16 16:44:34 -05:00
Junping Du 3d2ef54450 YARN-7124. LogAggregationTFileController deletes/renames while file is open. Contributed by Jason Lowe. 
(cherry picked from commit 1fcbe7cf5f)
 2017-10-16 14:29:02 -07:00
Arun Suresh 53cd33cc4a YARN-7275. NM Statestore cleanup for Container updates. (Kartheek Muthyala via asuresh) 
(cherry picked from commit a50be1b8f4)
 2017-10-16 13:27:08 -07:00
Wangda Tan 2c1381300f YARN-7321. Backport container-executor changes from YARN-6852 to branch-2. (Varun Vasudev via wangda) 
Change-Id: Ib6434b5a5ebe591050c3949cac12ff9e2a2bbaa5
 2017-10-14 10:48:26 -07:00
Wangda Tan 64170eb53c Addendum fix for: YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (Wangda Tan) 
Change-Id: If1fe4a62e07b25e6f1b8ba803495da87e2cb2df6
(cherry picked from commit 87ea1dff9c)
 2017-10-14 10:47:54 -07:00
Yufei Gu f2350f5120 YARN-7270. Fix unsafe casting from long to int for class Resource and its sub-classes. (Yufei) 
(cherry picked from commit 7a27c2c367)
(cherry picked from commit dbf6c48af9)
 2017-10-13 13:10:20 -07:00
Carlo Curino b9426c0bf4 YARN-7317. Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy. (contributed by Botong Huang via curino) 
(cherry picked from commit 13fcfb3d46)
 2017-10-12 10:42:36 -07:00
Wangda Tan 1ab6b4a08e YARN-6033. Add support for sections in container-executor configuration file. (Varun Vasudev via wangda) 
Change-Id: Id1ded3ee11e6242dfed5616903580f329f3bb26a
 2017-10-12 09:14:08 -07:00
Robert Kanter 759d7d58fa YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter) 
(cherry picked from commit ec8bf9e48a)
 2017-10-10 10:19:02 -07:00
Jonathan Hung 4bd3f0df42 YARN-7252. Removing queue then failing over results in exception 2017-10-09 11:12:08 -07:00
Jonathan Hung 06c8938ab0 YARN-7251. Misc changes to YARN-5734 2017-10-09 11:12:08 -07:00
Jonathan Hung 40bec38465 YARN-7238. Documentation for API based scheduler configuration management 2017-10-09 11:12:08 -07:00
Jonathan Hung c8f9fe88d5 YARN-7046. Add closing logic to configuration store 2017-10-09 11:12:08 -07:00
Wangda Tan 887b366d5f YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda) 
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
 2017-10-09 11:12:07 -07:00
Xuan 36cda2ff80 YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan 3a31c38885 YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan cd58f5da5a YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung 2017-10-09 11:12:07 -07:00
Xuan 16b27573ef YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao 2017-10-09 11:12:07 -07:00
Xuan 96f4cf7cde YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong) 2017-10-09 11:12:07 -07:00
Xuan 3c50084219 YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong) 2017-10-09 11:12:07 -07:00
Wangda Tan a47823ca98 YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda) 
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
 2017-10-09 11:12:07 -07:00
Wangda Tan 53a1268125 YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda) 2017-10-09 11:12:07 -07:00
Jonathan Hung 072527b31a YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store 2017-10-09 11:12:07 -07:00
Xuan 38cf9ccd5b YARN-5946: Create YarnConfigurationStore interface and 
InMemoryConfigurationStore class. Contributed by Jonathan Hung
 2017-10-09 11:12:07 -07:00
Jonathan Hung cd4bc54e6c YARN-5951. Changes to allow CapacityScheduler to use configuration store 2017-10-09 11:12:07 -07:00
Daniel Templeton 4d377c89bb YARN-2612 addendum: fixed javadoc error. 
(cherry picked from commit 6d6ca4c923)
(cherry picked from commit bec03f7060)
 2017-10-09 14:16:50 +02:00
Sunil G 045526aca1 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f)
 2017-10-09 14:44:33 +05:30
Yufei Gu da0a96968d YARN-7207. Cache the RM proxy server address. (Yufei Gu) 
(cherry picked from commit 72d22b753a)

# Conflicts:
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
#	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-06 09:56:24 -07:00
Yufei Gu c949e8e83a YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu) 2017-10-06 00:35:03 -07:00
Arun Suresh cb91bb3c96 YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh). 
(cherry picked from commit b733348dde)
 2017-10-05 15:22:20 -07:00
Naganarasimha 5cb3bd3b06 YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe. 
(cherry picked from commit d0046bb5c6)
 2017-10-06 00:09:46 +05:30
Chris Trezzo d12cea2098 YARN-2960. Add documentation for the YARN shared cache. 
(cherry picked from commit 7e76f85bc6)
 2017-10-05 11:23:24 -07:00
Subru Krishnan 7fd4a997f4 YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit e6e614e380)
 2017-10-04 19:30:57 -07:00
Subru Krishnan 7836a6b59a YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru) 2017-10-04 16:29:08 -07:00
Akira Ajisaka 5508b9b7c8
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails. 
(cherry picked from commit 2df1b2ac05)
 2017-10-05 06:08:46 +09:00
Sidharta S 3ecbfff937 YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-04 14:05:01 -07:00
Wangda Tan 175ff0585d YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (wangda via Jian He) 
Change-Id: I582566e09d5ef67e9cc9e9805231ccbabe57c3d0
 2017-10-03 12:19:11 -07:00
Wangda Tan 42e873d4ea YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu) 2017-10-03 12:04:56 -07:00
Wangda Tan 598329e6f8 YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda) 
Change-Id: I66c9b50ae3955dda0b5bf0674a9bfb728db27acb
 2017-10-03 09:46:35 -07:00
Wangda Tan 8beae14a07 YARN-7259. Add size-based rolling policy to LogAggregationIndexedFileController. (xgong via wangda) 
Change-Id: Ifaf82c0aee6b73b9b6ebf103aa72e131e3942f31
(cherry picked from commit 280080fad0)
 2017-10-02 15:36:11 -07:00
Yufei Gu e84d508c1b YARN-6333. Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold. (Contributed by Chetna Chaudhari via Yufei Gu) 
(cherry picked from commit 66c417167a)
 2017-09-29 18:08:55 -07:00
Wangda Tan 5fa5ba7f4d YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda) 
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
(cherry picked from commit ec2ae3060a)
(cherry picked from commit d58d75ca50)
 2017-09-29 16:53:16 -07:00
Eric Payne b0ba31c518 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918)
 2017-09-29 10:50:21 -05:00
Jason Lowe c0bfb385a8 YARN-7260. yarn.router.pipeline.cache-max-size is missing in yarn-default.xml. Contributed by Jason Lowe 2017-09-29 09:53:05 -05:00