Sunil G
9071c712d9
YARN-5912. Fix breadcrumb issues for various pages in new YARN UI. Contributed by Akhil P B.
2017-10-18 02:06:48 +05:30
Sunil G
2c93e8ee13
YARN-5917. Make navigation link active when selecting sub tabs in 'Applications' and 'Nodes' page for new UI. Contributed by Kai Sasaki.
2017-10-18 02:06:48 +05:30
Sunil G
0d1a53ad8e
YARN-5866. Fix few issues reported by jshint in new YARN UI. Contributed by Akhil P B.
2017-10-18 02:06:48 +05:30
Sunil G
d5271b03d3
YARN-5650. Render Application Timeout value in web UI. Contributed by Akhil PB.
2017-10-18 02:06:48 +05:30
sunilg
011402e304
YARN-4849. Addendum patch to improve pom for yarn-ui. (Wangda Tan via Sunil G)
2017-10-18 02:06:48 +05:30
sunilg
4fba66c28e
YARN-5785. [YARN-3368] Accessing applications and containers list from Node page is throwing few exceptions in console. (Akhil P B and Sreenath Somarajapuram via Sunil G)
2017-10-18 02:06:48 +05:30
Sreenath Somarajapuram
6c68032291
YARN-5779. [YARN-3368] Addendum patch to document limits/notes of the new YARN UI . (Sunil G via Sreenath Somarajapuram)
2017-10-18 02:06:48 +05:30
sunilg
9ebb57dfc8
YARN-5490. [YARN-3368] Fix various alignment issues and broken breadcrumb link in Node page. (Akhil P B Tan via Sunil G)
2017-10-18 02:06:48 +05:30
sunilg
36f5091d6c
YARN-5497. [YARN-3368] Use different color for Undefined and Succeeded for Final State in applications page. (Akhil P B Tan via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
94c53e7985
YARN-5500. [YARN-3368] ‘Master node' link under application tab is broken. (Akhil P B Tan via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
addc399e8b
YARN-5772. [YARN-3368] Replace old Hadoop logo with new one (Akhil P B Tan via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
43f0627a8e
YARN-5779. [YARN-3368] Document limits/notes of the new YARN UI (Wangda Tan via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
29fc43e5a4
YARN-5741. [YARN-3368] Update UI2 documentation for new UI2 path (Kai Sasaki and Wangda Tan via Sunil G)
2017-10-18 02:06:47 +05:30
Wangda Tan
7513315758
YARN-5145. [YARN-3368] Move new YARN UI configuration to HADOOP_CONF_DIR. (Sunil G and Kai Sasaki via wangda)
2017-10-18 02:06:47 +05:30
sunilg
f1de967c67
YARN-4849. Addendum patch to remove unwanted files from rat exclusions. (Wangda Tan via Sunil G)
2017-10-18 02:06:47 +05:30
Wangda Tan
c0c124c2bf
YARN-5698. [YARN-3368] Launch new YARN UI under hadoop web app port. (Sunil G via wangda)
2017-10-18 02:06:47 +05:30
sunilg
8222307c30
YARN-5682. [YARN-3368] Fix maven build to keep all generated or downloaded files in target folder (Wangda Tan via Sunil G)
2017-10-18 02:06:47 +05:30
Wangda Tan
e5c0219228
YARN-4849. Addendum patch to fix javadocs. (Sunil G via wangda)
2017-10-18 02:06:47 +05:30
sunilg
542f661cc1
YARN-4849. Addendum patch to fix ASF warnings. (Wangda Tan via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
dd8ad34305
YARN-5503. [YARN-3368] Add missing hidden files in webapp folder for deployment (Sreenath Somarajapuram via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
ade1dd96b5
YARN-5504. [YARN-3368] Fix YARN UI build pom.xml (Sreenath Somarajapuram via Sunil G)
2017-10-18 02:06:47 +05:30
vrushali
1510fea062
Updating pom to be 2.9.0-SNAPSHOT instead of 3.0
2017-10-18 02:06:47 +05:30
sunilg
48336218d4
YARN-4849. Addendum patch to fix document. (Wangda Tan via Sunil G)
2017-10-18 02:06:47 +05:30
sunilg
a4021c9c36
YARN-5488. [YARN-3368] Applications table overflows beyond the page boundary(Harish Jaiprakash via Sunil G)
2017-10-18 02:06:46 +05:30
Wangda Tan
a53e981adf
YARN-5509. Build error due to preparing 3.0.0-alpha2 deployment. (Kai Sasaki via wangda)
2017-10-18 02:06:46 +05:30
sunilg
3e0dab4bfd
YARN-3334. [YARN-3368] Introduce REFRESH button in various UI pages (Sreenath Somarajapuram via Sunil G)
2017-10-18 02:06:46 +05:30
Sunil
30d1839aa0
YARN-5321. [YARN-3368] Add resource usage for application by node managers (Wangda Tan via Sunil G)
...
YARN-5320. [YARN-3368] Add resource usage by applications and queues to cluster overview page (Wangda Tan via Sunil G)
YARN-5322. [YARN-3368] Add a node heat chart map (Wangda Tan via Sunil G)
YARN-5347. [YARN-3368] Applications page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5348. [YARN-3368] Node details page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5346. [YARN-3368] Queues page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5345. [YARN-3368] Cluster overview page improvements (Sreenath Somarajapuram via Sunil G)
YARN-5344. [YARN-3368] Generic UI improvements (Sreenath Somarajapuram via Sunil G)
2017-10-18 02:06:46 +05:30
Sunil
b41ca1a6ea
YARN-5161. [YARN-3368] Add Apache Hadoop logo in YarnUI home page. (Kai Sasaki via Sunil G)
2017-10-18 02:06:46 +05:30
Sunil
0490942c58
YARN-5183. [YARN-3368] Support for responsive navbar when window is resized. (Kai Sasaki via Sunil G)
2017-10-18 02:06:46 +05:30
Wangda Tan
bb606e3d4b
YARN-4515. [YARN-3368] Support hosting web UI framework inside YARN RM. (Sunil G via wangda)
...
YARN-5000. [YARN-3368] App attempt page is not loading when timeline server is not started (Sunil G via wangda)
YARN-5038. [YARN-3368] Application and Container pages shows wrong values when RM is stopped. (Sunil G via wangda)
2017-10-18 02:06:46 +05:30
Wangda Tan
67aa886ce7
YARN-5019. [YARN-3368] Change urls in new YARN ui from camel casing to hyphens. (Sunil G via wangda)
2017-10-18 02:06:46 +05:30
Wangda Tan
b414b5f2cf
YARN-4514. [YARN-3368] Cleanup hardcoded configurations, such as RM/ATS addresses. (Sunil G via wangda)
2017-10-18 02:06:46 +05:30
Wangda Tan
56654d8820
YARN-4849. [YARN-3368] cleanup code base, integrate web UI related build to mvn, and fix licenses. (wangda)
2017-10-18 02:06:46 +05:30
Wangda Tan
217a87e975
YARN-4517. Add nodes page and fix bunch of license issues. (Varun Saxena via wangda)
2017-10-18 02:06:46 +05:30
Wangda Tan
8a75207d4c
YARN-4733. [YARN-3368] Initial commit of new YARN web UI. (wangda)
2017-10-18 02:06:46 +05:30
Subru Krishnan
5e7491d532
YARN-7311. Fix TestRMWebServicesReservation parametrization for fair scheduler. (Yufei Gu via Subru).
...
(cherry picked from commit 75323394fb
)
2017-10-17 12:43:36 -07:00
Haibo Chen
c0ee583656
YARN-7341. TestRouterWebServiceUtil#testMergeMetrics is flakey. (Robert Kanter via Haibo Chen)
...
(cherry picked from commit acabc657ff
)
2017-10-17 12:30:26 -07:00
Subru Krishnan
adf5ea73d6
YARN-7095. Federation: routing getNode/getNodes/getMetrics REST invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit bac4e8cca8
)
2017-10-17 12:14:05 -07:00
Varun Saxena
56e0873e22
Fix javac, javadoc and checkstyle issues before branch-2 merge.
2017-10-17 23:36:41 +05:30
vrushali
8b98dba7ee
YARN-7190 Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath (Contributed by Varun Saxena via Vrushali C)
2017-10-17 23:36:41 +05:30
Varun Saxena
6ba1f34532
Fix problems in yarn.cmd due to rebase.
2017-10-17 23:36:41 +05:30
Haibo Chen
572ebd1a95
YARN-7063. TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2. (Varun Saxena via Haibo Chen)
2017-10-17 23:36:41 +05:30
Haibo Chen
d77530b227
YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen)
...
(cherry picked from commit 661f5eb0c6
)
2017-10-17 23:36:41 +05:30
Rohith Sharma K S
27019b5a66
YARN-7130. ATSv2 documentation changes post merge. Contributed by Varun Saxena.
...
(cherry picked from commit 0bfb3a2566
)
2017-10-17 23:36:41 +05:30
Rohith Sharma K S
138d6fc083
YARN-7140. CollectorInfo should have Public visibility. Contributed by Varun Saxena.
...
(cherry picked from commit 4a83170be4
)
2017-10-17 23:36:40 +05:30
Varun Saxena
95f45b89d1
YARN-7152. [ATSv2] Registering timeline client before AMRMClient service init throw exception (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit ef87d34a0e
)
2017-10-17 23:36:40 +05:30
Junping Du
b3201b2dab
YARN-6534. ResourceManager failed due to TimelineClient try to init SSLFactory even https is not enabled. Contributed by Rohith Sharma K S.
...
(cherry picked from commit 09e36ff59c05b8bd54d346d1103de0a7300875c6)
(cherry picked from commit 94607214ce
)
2017-10-17 23:36:40 +05:30
Haibo Chen
6e2de8b417
YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen)
...
(cherry picked from commit 461ee44d28
)
2017-10-17 23:36:40 +05:30
bibinchundatt
f7637b55bb
YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena
...
(cherry picked from commit dcd0bedcc8
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHATimelineCollectors.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/security/TimelineReaderWhitelistAuthorizationFilter.java
2017-10-17 23:36:40 +05:30
Varun Saxena
1289b056ac
YARN-6958. Addedndum for Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice
2017-10-17 23:36:40 +05:30
Varun Saxena
87713f41b6
YARN-6905. Addendum to fix TestTimelineReaderWebServicesHBaseStorage due to missing FastNumberFormat
...
(cherry picked from commit e52f26e5cf
)
2017-10-17 23:36:40 +05:30
Vrushali C
a2598434ed
YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen)
...
(cherry picked from commit 608a06cca5
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java
2017-10-17 23:36:40 +05:30
Varun Saxena
78a6ecccb3
Addendum for YARN-4753. Use doxia macro to generate in-page TOC of YARN site documentation
2017-10-17 23:36:40 +05:30
Varun Saxena
3d5bf1c06d
YARN-6047. Documentation updates for TimelineService v2. Contributed by Rohith Sharma K S, Vrushali C and Varun Saxena
2017-10-17 23:36:40 +05:30
Varun Saxena
a22c1c72c4
YARN-6316. Addendum for missing create option in documentation.
2017-10-17 23:36:40 +05:30
Varun Saxena
28bbe9ccd7
YARN-5577. [Atsv2] Document object passing in infofilters with an example (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit ea29e3bc27
)
2017-10-17 23:36:40 +05:30
Haibo Chen
0f51f15097
YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen)
2017-10-17 23:36:40 +05:30
Jian He
cbd5e89b24
MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena
...
(cherry picked from commit c6ce422a3423305ac665f80f039f61294adaa32c)
Conflicts:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
2017-10-17 23:36:40 +05:30
Varun Saxena
e5d63652df
YARN-6861. Addendum for YARN-6861 for compilation issue (Rohith Sharma K S via Varun Saxena)
2017-10-17 23:36:40 +05:30
Varun Saxena
186c99c6cc
YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit 73ee0d4fc05354527a4be6db9e5b83d06fc94047)
2017-10-17 23:36:39 +05:30
Jian He
5c970f7c29
YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena
2017-10-17 23:36:39 +05:30
Rohith Sharma K S
5d9b4bd3e8
YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena.
...
(cherry picked from commit e976bf10bb0d42e4d971358554ea736ddcdafd19)
2017-10-17 23:36:39 +05:30
Jian He
53ddf1b15f
YARN-7038. Addendum patch. Fix compilation error. Contributed by Varun Saxena
2017-10-17 23:36:39 +05:30
Rohith Sharma K S
4e1af68ae1
YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena.
...
(cherry picked from commit 387dd3da87759860dd18bd8aa95c60d962b1e6a9)
2017-10-17 23:36:39 +05:30
Jian He
55e5742394
YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena
2017-10-17 23:36:39 +05:30
Jason Lowe
db2f7dd9bd
YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C
2017-10-17 23:36:39 +05:30
Rohith Sharma K S
cb75e1826e
YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena.
...
(cherry picked from commit ffb1f572b68e73efd6410ab74a334f4e5df543f2)
2017-10-17 23:36:39 +05:30
Rohith Sharma K S
5a7c272e0a
YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena.
2017-10-17 23:36:39 +05:30
Varun Saxena
5bda387573
YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena)
...
(cherry picked from commit 7b2cb0614a04e6d0e2bc6dfd5095b7535dffa678)
2017-10-17 23:36:39 +05:30
Varun Saxena
8d9d40aa53
Addendum for YARN-6102.
2017-10-17 23:36:39 +05:30
Varun Saxena
cb5d601b2e
Addendum for YARN-5648.
2017-10-17 23:36:39 +05:30
Jian He
8e4e5abbe6
YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena
...
(cherry picked from commit 3829100cf75e733b2b3a4e8c6806a3696f4ae658)
2017-10-17 23:36:39 +05:30
Jian He
50dd1372bf
YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena
...
(cherry picked from commit 7f06c46065936b88e9864c526bfcda3a89f3e257)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/AbstractTimelineReaderHBaseTestBase.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
2017-10-17 23:36:39 +05:30
Jian He
9577900a91
YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena
...
(cherry picked from commit 79dae624793164cd4692c86992a511310145858a)
2017-10-17 23:36:39 +05:30
Varun Saxena
c8db08dfac
YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit 0443928d771e3e21825b4f487e8c0865ea641970)
2017-10-17 23:36:38 +05:30
Sunil G
4b65fe73ca
YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S.
...
(cherry picked from commit a657472b42
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
2017-10-17 23:36:38 +05:30
Rohith Sharma K S
4bea8af723
YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C.
...
(cherry picked from commit 20dd6d18b7787e67ef96f3b6b92ea8415a8650fd)
2017-10-17 23:36:38 +05:30
Rohith Sharma K S
4014390ef4
YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C.
...
(cherry picked from commit 416312e3a7f0f83cc1c8549640c3ba75aaa3d887)
2017-10-17 23:36:38 +05:30
Vrushali C
0f751a7fec
YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C)
2017-10-17 23:36:38 +05:30
Rohith Sharma K S
484d7e9b39
YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena.
...
(cherry picked from commit 743e4731781a7d9a4a5f1f09adc510f193182158)
2017-10-17 23:36:38 +05:30
Rohith Sharma K S
d01a3f1ba2
YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C.
...
(cherry picked from commit 5791ced4bd864ff6c08cb2c3ee10ab62008f0168)
2017-10-17 23:36:38 +05:30
Varun Saxena
c4f042d4c1
YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena)
...
(cherry picked from commit 0b7bff706e
)
2017-10-17 23:36:38 +05:30
vrushali
48d2e4f519
YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C)
2017-10-17 23:36:38 +05:30
Varun Saxena
473f2f00ee
YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena)
...
(cherry picked from commit f7cba98504a5e968bb6ae976d20b71c84a4664f2)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/apptoflow/AppToFlowColumn.java
2017-10-17 23:36:38 +05:30
Haibo Chen
d53038626e
YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen)
...
(cherry picked from commit 47474fffac
)
(cherry picked from commit 8817cb5c8424359b880c6d700e53092f0269c1bb)
2017-10-17 23:36:38 +05:30
Haibo Chen
9b4d4228fb
YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen)
...
(cherry picked from commit 4c8b9d23a1b4f92bf370617a0b42ac68bdcf33ac)
2017-10-17 23:36:38 +05:30
Vrushali Channapattan
70315de4ff
YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C)
2017-10-17 23:36:38 +05:30
Vrushali Channapattan
34570d4fc3
YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C)
2017-10-17 23:36:38 +05:30
Naganarasimha
e1d2f6ebdd
YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang.
...
(cherry picked from commit 0f1af3178e
)
2017-10-17 23:36:37 +05:30
Varun Saxena
7338dbe5c0
YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena)
...
(cherry picked from commit 793bbf216d
)
2017-10-17 23:36:37 +05:30
Varun Saxena
c1f9f0529b
Addendum for YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611
2017-10-17 23:36:37 +05:30
Varun Saxena
ebcb575a12
Addendum for YARN-4218. Metric for resource*time that was preempted.
2017-10-17 23:36:37 +05:30
Rohith Sharma K S
1072232741
YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena.
...
(cherry picked from commit 1a9439e299
)
2017-10-17 23:36:37 +05:30
Varun Saxena
16d8df68f6
YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena)
...
(cherry picked from commit a4b5aa8493
)
2017-10-17 23:36:37 +05:30
Varun Saxena
fd88fa8083
YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena)
...
(cherry picked from commit b58777a9c9
)
2017-10-17 23:36:37 +05:30
Varun Saxena
debe526f08
YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena)
...
(cherry picked from commit 7c2bc444b3
)
2017-10-17 23:36:37 +05:30
Varun Saxena
203077cf2c
YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena)
...
(cherry picked from commit 063b513b1c
)
2017-10-17 23:36:37 +05:30
Varun Saxena
9595c18eea
YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena)
...
(cherry picked from commit 7f09cd1d5ee70e0bdf24cdecd78cd3bc258a40f8)
2017-10-17 23:36:37 +05:30
Varun Saxena
1369abf5b0
YARN-6318. timeline service schema creator fails if executed from a remote machine (Sangjin Lee via Varun Saxena)
...
(cherry picked from commit 68ec0d9b471356d3adef15d78111126f57ee50f8)
2017-10-17 23:36:37 +05:30
Varun Saxena
6ec294b7fc
YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit ec32c4265bbf436ae61b6efc71ca33b16b70a23d)
2017-10-17 23:36:37 +05:30
Varun Saxena
e597f2de60
YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit 5d9ad152a1082a7c9b8edaf57a88ae471a537599)
2017-10-17 23:36:37 +05:30
Sangjin Lee
16c607ca2b
YARN-6027 addendum. Fixed the broken build for YARN-5355-branch-2. Contributed by Sangjin Lee.
2017-10-17 23:36:37 +05:30
Varun Saxena
d996afd585
YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit 63c06ec44e633567c378e28898e319143593ff30)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java
2017-10-17 23:36:36 +05:30
Sangjin Lee
6b5b8f090d
YARN-6253. FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp. Contributed by Haibo Chen.
...
(cherry picked from commit 34e7c30293b5a56f9f745769a29c5666bdb85d6c)
2017-10-17 23:36:36 +05:30
Varun Saxena
59d96e247e
YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit ab192fd58358faff6880f3e0e867d7bdd4bb043a)
2017-10-17 23:36:36 +05:30
Varun Saxena
4a3a4aaf7e
YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R via Varun Saxena)
...
(cherry picked from commit 6ba61d20d3
)
2017-10-17 23:36:36 +05:30
Varun Saxena
74e43bf287
YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena)
...
(cherry picked from commit 649deb72fb
)
2017-10-17 23:36:36 +05:30
Sangjin Lee
bb100d35fd
YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R.
...
(cherry picked from commit 73235ab30361b41293846189f3c5fef321ae7cac)
2017-10-17 23:36:36 +05:30
Sangjin Lee
a16a684fbb
YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen.
...
(cherry picked from commit b92089c0e8ab1b87b8b5b55b1e3d4367ae5d847a)
2017-10-17 23:36:36 +05:30
Sangjin Lee
5cb5b87c9a
YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C.
...
(cherry picked from commit 0327a79d79a4d56d9c7cb6889886afd2272b07d3)
2017-10-17 23:36:36 +05:30
Varun Saxena
8c81f7fc08
Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's
...
(cherry picked from commit c9246f619104bff44dd453a4ffe70104d37fd781)
2017-10-17 23:36:36 +05:30
Varun Saxena
aa09001a33
YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit 4b1ba4ea314147f8a06cc4f446c1d9336de89fc1)
2017-10-17 23:36:36 +05:30
Varun Saxena
3429517a00
YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena)
...
(cherry picked from commit 6baea680ba6e5df6f254ced086d6defa64fb99f0)
2017-10-17 23:36:36 +05:30
Sangjin Lee
5b3ef55f5f
YARN-5980. Update documentation for single node hbase deploy. Contributed by Vrushali C.
...
(cherry picked from commit e1bdba77888723b435a235a96c8659029afd25d5)
2017-10-17 23:36:36 +05:30
Varun Saxena
d12920b10e
YARN-6074. FlowRunEntity does not deserialize long values correctly (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit db490eccce
)
2017-10-17 23:36:36 +05:30
Varun Saxena
1e5e4a52f7
YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena)
...
(cherry picked from commit 9a2f288a6a9e30376b7fd99bf8824184a34a54c9)
2017-10-17 23:36:35 +05:30
Naganarasimha
d3ef478955
YARN-6026. A couple of spelling errors in the docs. Contributed by Grant Sohn.
2017-10-17 23:36:35 +05:30
Sangjin Lee
6aa7510681
YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C.
...
(cherry picked from commit f945008d1cf5730bdebeae501ed0e42477ad219e)
2017-10-17 23:36:35 +05:30
Sangjin Lee
3394cb25d2
YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
...
(cherry picked from commit 8288030cb4aa3b5a9425cc0a3f6df03a3eae1dfb)
2017-10-17 23:36:35 +05:30
Sangjin Lee
54705f549f
YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen.
...
(cherry picked from commit a5a55a54ab
)
(cherry picked from commit 17c4ab7ebb51088caf36fafedae8c256481eeed5)
2017-10-17 23:36:35 +05:30
Naganarasimha
a5371f6b6e
YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena.
...
(cherry picked from commit c06114d6a3
)
2017-10-17 23:36:35 +05:30
Sangjin Lee
d6d73e3c99
YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee.
...
(cherry picked from commit f511cc89b6
)
2017-10-17 23:36:35 +05:30
Sangjin Lee
4f595889e7
YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu.
...
(cherry picked from commit 3b2e80881eaeaa82667496d706496765ed7e29f5)
2017-10-17 23:36:35 +05:30
Vrushali Channapattan
2fa2c3169e
YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (Varun Saxena via Vrushali C)
...
(cherry picked from commit 513dcf6817dd76fde8096ff04cd888d7c908461d)
(cherry picked from commit 022bf783aa
)
2017-10-17 23:36:35 +05:30
Sangjin Lee
b8cfb4fcb3
YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena.
...
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
2017-10-17 23:36:35 +05:30
Varun Saxena
684c2d149f
YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena)
...
(cherry picked from commit 25b19178dd53bc0d57ca5fd64d3464af7b59e588)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
2017-10-17 23:36:35 +05:30
Varun Saxena
484b1d2ec1
YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena)
2017-10-17 23:36:34 +05:30
Varun Saxena
6562a354a9
YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena)
...
(cherry picked from commit 1c8a57550b
)
2017-10-17 23:36:34 +05:30
Sangjin Lee
4182f892e0
YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis.
...
(cherry picked from commit dc7d9bac79fc303f66e97c10379d680e0c11caa4)
2017-10-17 23:36:34 +05:30
Sangjin Lee
ebf2f90528
YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S.
...
(cherry picked from commit f37288c7e0127b564645e978c7aab2a186fa6be6)
2017-10-17 23:36:34 +05:30
Varun Saxena
d19726ec37
YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena)
2017-10-17 23:36:34 +05:30
Varun Saxena
1d7fc52578
YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena)
2017-10-17 23:36:34 +05:30
Sangjin Lee
3344f0ace5
YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S.
...
(cherry picked from commit e9c4616b5e
)
2017-10-17 23:36:34 +05:30
Varun Saxena
c3db98aa10
YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena)
2017-10-17 23:36:34 +05:30
Sangjin Lee
853a5da041
YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S.
...
(cherry picked from commit 787d5942003a5441ba455a79fa08da28f60d5777)
2017-10-17 23:36:34 +05:30
Sangjin Lee
4675f22e6a
YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu.
...
(cherry picked from commit 0c1863144649ea265da65ce25158707cc3a3fb4a)
2017-10-17 23:36:34 +05:30
Li Lu
6a517d714e
YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9)
...
(cherry picked from commit 5d7ad396dbf84ffb0bacb3372d149828dfb4b4ca)
2017-10-17 23:36:34 +05:30
Sangjin Lee
51102019f5
YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee)
...
(cherry picked from commit d0a62d8adf86f7cb2481940d212788cc5f327367)
2017-10-17 23:36:34 +05:30
Sangjin Lee
4f6ba3187b
YARN-5379. TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently (Vrushali C via sjlee)
...
(cherry picked from commit e549a9af31
)
2017-10-17 23:36:34 +05:30
Naganarasimha
22f5046acb
YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee
...
(cherry picked from commit af8f480c24
)
2017-10-17 23:36:34 +05:30
Varun Saxena
057a4728ff
YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena)
...
(cherry picked from commit 56142171b9
)
2017-10-17 23:36:33 +05:30
Varun Saxena
16942ea115
YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena).
...
(cherry picked from commit a3ac1c1943
)
2017-10-17 23:36:33 +05:30
Sangjin Lee
e212e562ec
YARN-5355. Backported YARN-2928 into our branch-2 feature branch.
2017-10-17 23:36:33 +05:30
Nathan Roberts
b876a9314e
YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe.
...
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
2017-10-16 16:44:34 -05:00
Junping Du
3d2ef54450
YARN-7124. LogAggregationTFileController deletes/renames while file is open. Contributed by Jason Lowe.
...
(cherry picked from commit 1fcbe7cf5f
)
2017-10-16 14:29:02 -07:00
Arun Suresh
53cd33cc4a
YARN-7275. NM Statestore cleanup for Container updates. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit a50be1b8f4
)
2017-10-16 13:27:08 -07:00
Wangda Tan
2c1381300f
YARN-7321. Backport container-executor changes from YARN-6852 to branch-2. (Varun Vasudev via wangda)
...
Change-Id: Ib6434b5a5ebe591050c3949cac12ff9e2a2bbaa5
2017-10-14 10:48:26 -07:00
Wangda Tan
64170eb53c
Addendum fix for: YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (Wangda Tan)
...
Change-Id: If1fe4a62e07b25e6f1b8ba803495da87e2cb2df6
(cherry picked from commit 87ea1dff9c
)
2017-10-14 10:47:54 -07:00
Yufei Gu
f2350f5120
YARN-7270. Fix unsafe casting from long to int for class Resource and its sub-classes. (Yufei)
...
(cherry picked from commit 7a27c2c367
)
(cherry picked from commit dbf6c48af9
)
2017-10-13 13:10:20 -07:00
Carlo Curino
b9426c0bf4
YARN-7317. Fix overallocation resulted from ceiling in LocalityMulticastAMRMProxyPolicy. (contributed by Botong Huang via curino)
...
(cherry picked from commit 13fcfb3d46
)
2017-10-12 10:42:36 -07:00
Wangda Tan
1ab6b4a08e
YARN-6033. Add support for sections in container-executor configuration file. (Varun Vasudev via wangda)
...
Change-Id: Id1ded3ee11e6242dfed5616903580f329f3bb26a
2017-10-12 09:14:08 -07:00
Robert Kanter
759d7d58fa
YARN-7309. TestClientRMService#testUpdateApplicationPriorityRequest and TestClientRMService#testUpdatePriorityAndKillAppWithZeroClusterResource test functionality not supported by FairScheduler (rkanter)
...
(cherry picked from commit ec8bf9e48a
)
2017-10-10 10:19:02 -07:00
Jonathan Hung
4bd3f0df42
YARN-7252. Removing queue then failing over results in exception
2017-10-09 11:12:08 -07:00
Jonathan Hung
06c8938ab0
YARN-7251. Misc changes to YARN-5734
2017-10-09 11:12:08 -07:00
Jonathan Hung
40bec38465
YARN-7238. Documentation for API based scheduler configuration management
2017-10-09 11:12:08 -07:00
Jonathan Hung
c8f9fe88d5
YARN-7046. Add closing logic to configuration store
2017-10-09 11:12:08 -07:00
Wangda Tan
887b366d5f
YARN-6840. Implement zookeeper based store for scheduler configuration updates. (Jonathan Hung via wangda)
...
Change-Id: I9debea674fe8c7e4109d4ca136965a1ea4c48bcc
2017-10-09 11:12:07 -07:00
Xuan
36cda2ff80
YARN-7024: Fix issues on recovery in LevelDB store. Contributed by Jonathan Hung
2017-10-09 11:12:07 -07:00
Xuan
3a31c38885
YARN-6322: Disable queue refresh when configuration mutation is enabled. Contributed by Jonathan Hung
2017-10-09 11:12:07 -07:00
Xuan
cd58f5da5a
YARN-5947: Create LeveldbConfigurationStore class using Leveldb as backing store. Contributed by Jonathan Hung
2017-10-09 11:12:07 -07:00
Xuan
16b27573ef
YARN-5953 addendum: Move QueueConfigInfo and SchedConfUpdateInfo to package org.apache.hadoop.yarn.webapp.dao
2017-10-09 11:12:07 -07:00
Xuan
96f4cf7cde
YARN-5953:Create CLI for changing YARN configurations. (Jonathan Hung via xgong)
2017-10-09 11:12:07 -07:00
Xuan
3c50084219
YARN-6575. Support global configuration mutation in MutableConfProvider. (Jonathan Hung via Xuan Gong)
2017-10-09 11:12:07 -07:00
Wangda Tan
a47823ca98
YARN-5949. Add pluggable configuration ACL policy interface and implementation. (Jonathan Hung via wangda)
...
Change-Id: Ib98e82ff753bede21fcab2e6ca9ec1e7a5a2008f
2017-10-09 11:12:07 -07:00
Wangda Tan
53a1268125
YARN-5952. Create REST API for changing YARN scheduler configurations. (Jonathan Hung via wangda)
2017-10-09 11:12:07 -07:00
Jonathan Hung
072527b31a
YARN-5948. Implement MutableConfigurationManager for handling storage into configuration store
2017-10-09 11:12:07 -07:00
Xuan
38cf9ccd5b
YARN-5946: Create YarnConfigurationStore interface and
...
InMemoryConfigurationStore class. Contributed by Jonathan Hung
2017-10-09 11:12:07 -07:00
Jonathan Hung
cd4bc54e6c
YARN-5951. Changes to allow CapacityScheduler to use configuration store
2017-10-09 11:12:07 -07:00
Daniel Templeton
4d377c89bb
YARN-2612 addendum: fixed javadoc error.
...
(cherry picked from commit 6d6ca4c923
)
(cherry picked from commit bec03f7060
)
2017-10-09 14:16:50 +02:00
Sunil G
045526aca1
YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne.
...
(cherry picked from commit 8137254a1f
)
2017-10-09 14:44:33 +05:30
Yufei Gu
da0a96968d
YARN-7207. Cache the RM proxy server address. (Yufei Gu)
...
(cherry picked from commit 72d22b753a
)
# Conflicts:
# hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContext.java
# hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
2017-10-06 09:56:24 -07:00
Yufei Gu
c949e8e83a
YARN-2162. Add ability in Fair Scheduler to optionally configure maxResources in terms of percentage. (Yufei Gu)
2017-10-06 00:35:03 -07:00
Arun Suresh
cb91bb3c96
YARN-7258. Add Node and Rack Hints to Opportunistic Scheduler. (Kartheek Muthyala via asuresh).
...
(cherry picked from commit b733348dde
)
2017-10-05 15:22:20 -07:00
Naganarasimha
5cb3bd3b06
YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe.
...
(cherry picked from commit d0046bb5c6
)
2017-10-06 00:09:46 +05:30
Chris Trezzo
d12cea2098
YARN-2960. Add documentation for the YARN shared cache.
...
(cherry picked from commit 7e76f85bc6
)
2017-10-05 11:23:24 -07:00
Subru Krishnan
7fd4a997f4
YARN-5329. Placement Agent enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru).
...
(cherry picked from commit e6e614e380
)
2017-10-04 19:30:57 -07:00
Subru Krishnan
7836a6b59a
YARN-2037. Add work preserving restart support for Unmanaged AMs. (Botong Huang via Subru)
2017-10-04 16:29:08 -07:00
Akira Ajisaka
5508b9b7c8
YARN-7044. TestContainerAllocation#testAMContainerAllocationWhenDNSUnavailable fails.
...
(cherry picked from commit 2df1b2ac05
)
2017-10-05 06:08:46 +09:00
Sidharta S
3ecbfff937
YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe
2017-10-04 14:05:01 -07:00
Wangda Tan
175ff0585d
YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (wangda via Jian He)
...
Change-Id: I582566e09d5ef67e9cc9e9805231ccbabe57c3d0
2017-10-03 12:19:11 -07:00
Wangda Tan
42e873d4ea
YARN-6625. yarn application -list returns a tracking URL for AM that doesn't work in secured and HA environment. (Yufei Gu)
2017-10-03 12:04:56 -07:00
Wangda Tan
598329e6f8
YARN-6550. Capture launch_container.sh logs to a separate log file. (Suma Shivaprasad via wangda)
...
Change-Id: I66c9b50ae3955dda0b5bf0674a9bfb728db27acb
2017-10-03 09:46:35 -07:00
Wangda Tan
8beae14a07
YARN-7259. Add size-based rolling policy to LogAggregationIndexedFileController. (xgong via wangda)
...
Change-Id: Ifaf82c0aee6b73b9b6ebf103aa72e131e3942f31
(cherry picked from commit 280080fad0
)
2017-10-02 15:36:11 -07:00
Yufei Gu
e84d508c1b
YARN-6333. Improve doc for minSharePreemptionTimeout, fairSharePreemptionTimeout and fairSharePreemptionThreshold. (Contributed by Chetna Chaudhari via Yufei Gu)
...
(cherry picked from commit 66c417167a
)
2017-09-29 18:08:55 -07:00
Wangda Tan
5fa5ba7f4d
YARN-6509. Add a size threshold beyond which yarn logs will require a force option. (Xuan Gong via wangda)
...
Change-Id: I755fe903337d4ff9ec35dae5b9cce638794e1d0f
(cherry picked from commit ec2ae3060a
)
(cherry picked from commit d58d75ca50
)
2017-09-29 16:53:16 -07:00
Eric Payne
b0ba31c518
YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe.
...
(cherry picked from commit 7f6118f918
)
2017-09-29 10:50:21 -05:00
Jason Lowe
c0bfb385a8
YARN-7260. yarn.router.pipeline.cache-max-size is missing in yarn-default.xml. Contributed by Jason Lowe
2017-09-29 09:53:05 -05:00
Chris Trezzo
cba1891b61
YARN-7250. Update Shared cache client api to use URLs.
...
(cherry picked from commit c114da5e64
)
2017-09-28 15:32:18 -07:00
Subru Krishnan
ffcf5ba1ce
YARN-6962. Add support for updateContainers when allocating using FederationInterceptor. (Botong Huang via Subru).
...
(cherry picked from commit ca669f9f8b
)
2017-09-28 13:11:32 -07:00
Subru Krishnan
d6da014f67
YARN-6955. Handle concurrent register AM requests in FederationInterceptor. (Botong Huang via Subru).
...
(cherry picked from commit c61f2c4198
)
2017-09-28 13:11:19 -07:00
Jason Lowe
913a64e4c9
YARN-7248. NM returns new SCHEDULED container status to older clients. Contributed by Arun Suresh
...
(cherry picked from commit 85d81ae58e
)
Conflicts:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
2017-09-28 14:25:01 -05:00
Arun Suresh
53c1115908
YARN-6691. Update YARN daemon startup/shutdown scripts to include Router service. (Giovanni Matteo Fumarola via asuresh)
2017-09-28 11:41:43 -07:00
Xuan
c143708acf
YARN-7257. AggregatedLogsBlock reports a bad 'end' value as a bad
...
'start' value. Contributed by Jason Lowe
(cherry picked from commit 28c4957fcc
)
(cherry picked from commit 81e7872793
)
2017-09-27 16:04:06 -07:00
Rohith Sharma K S
c570ddaa66
YARN-7157. Add admin configuration to filter per-user's apps in secure cluster. Contributed by Sunil G.
2017-09-27 11:49:59 +05:30
Jason Lowe
c4de765fcb
Revert "YARN-6570. No logs were found for running application, running"
...
This reverts commit 6aaca3d08e
.
2017-09-26 10:44:16 -05:00
Rohith Sharma K S
c27f682032
YARN-65. Reduce RM app memory footprint once app has completed. Contributed by Manikandan R.
...
(cherry picked from commit 06e5a7b5cf
)
2017-09-26 14:50:34 +05:30
Wangda Tan
179d8dc83c
YARN-7118. AHS REST API can return NullPointerException. (Billie Rinaldi via wangda)
...
Change-Id: I16ca93385ab80e3680d4d173b8d1b79929ea7174
2017-09-25 21:41:00 -07:00
Arun Suresh
e7199cb494
YARN-7240. Add more states and transitions to stabilize the NM Container state machine. (Kartheek Muthyala via asuresh)
...
(cherry picked from commit df800f6cf3
)
2017-09-25 16:36:58 -07:00
Akira Ajisaka
c0dacda546
YARN-7153. Remove duplicated code in AMRMClientAsyncImpl.java. Contributed by Sen Zhao.
...
(cherry picked from commit e928ee583c
)
2017-09-26 02:05:35 +09:00
Anu Engineer
d0c0bd573b
Revert "YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi."
...
This reverts commit de1d747644
.
2017-09-23 11:12:16 -07:00
Junping Du
de1d747644
YARN-7118. AHS REST API can return NullPointerException. Contributed by Billie Rinaldi.
...
(cherry picked from commit 4002bf0a9e
)
2017-09-22 13:59:03 -07:00
Carlo Curino
7cd9018b13
[YARN FEDERATION BACKPORT] Fixing more Java 1.7 compilation issues
2017-09-21 18:19:36 -07:00
Carlo Curino
88b32edb8f
YARN-7010. Federation: routing REST invocations transparently to multiple RMs (part 2 - getApps). (Contributed by Giovanni Matteo Fumarola via curino)
...
(cherry picked from commit cc8893edc0
)
2017-09-21 18:09:44 -07:00
Arun Suresh
2aacb9d3fb
YARN-6923. Metrics for Federation Router. (Giovanni Matteo Fumarola via asuresh)
...
(cherry picked from commit ae8fb13b31
)
2017-09-21 18:09:37 -07:00
Arun Suresh
ac090b38ad
YARN-5603. Metrics for Federation StateStore. (Ellen Hui via asuresh)
...
(cherry picked from commit 75abc9a8e2
)
2017-09-21 18:09:30 -07:00
Subru Krishnan
261f769d79
YARN-6900. ZooKeeper based implementation of the FederationStateStore. (Íñigo Goiri via Subru).
...
(cherry picked from commit de462da04e
)
2017-09-21 18:09:23 -07:00
Subru Krishnan
9ad067efe9
YARN-6996. Change javax.cache library implementation from JSR107 to Apache Geronimo. (Ray Chiang via Subru).
...
(cherry picked from commit 18f3603bce
)
2017-09-21 18:09:16 -07:00
Carlo Curino
8220b19af7
YARN-6896. Federation: routing REST invocations transparently to multiple RMs (part 1 - basic execution). (Contributed by Giovanni Matteo Fumarola via curino)
...
(cherry picked from commit cc59b5fb26
)
2017-09-21 18:09:07 -07:00
Subru Krishnan
a1ee4ad77f
YARN-6970. Add PoolInitializationException as retriable exception in FederationFacade. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit ad2a350662
)
2017-09-21 18:08:30 -07:00
Carlo Curino
d11be2dcac
[YARN FEDERATION BACKPORT] Fix compilation issues due to: hadoop-router/pom.xml versions and Java 1.7
2017-09-21 17:56:46 -07:00
Allen Wittenauer
baa2bec843
YARN-2280. Resource manager web service fields are not accessible (Krisztian Horvath via aw)
...
(cherry picked from commit a5cf985bf5
)
2017-09-21 17:52:25 -07:00
Carlo Curino
d3afff7603
YARN-6853. Add MySql Scripts for FederationStateStore. (Contributed by Giovanni Matteo Fumarola via curino)
...
(cherry picked from commit 874ddbf0b5
)
(cherry picked from commit 9625a030de
)
2017-09-21 17:15:36 -07:00
Carlo Curino
bfd967d338
YARN-5412. Create a proxy chain for ResourceManager REST API in the Router. (Contributed by Giovanni Matteo Fumarola via curino)
...
(cherry picked from commit b6240b92ab
)
(cherry picked from commit acda6b96a4
)
2017-09-21 17:13:28 -07:00
Subru Krishnan
049f7c84a1
YARN-6866. Minor clean-up and fixes in anticipation of YARN-2915 merge with trunk. (Botong Huang via Subru).
...
(cherry picked from commit 40453879ec
)
(cherry picked from commit 86f05c3312
)
2017-09-21 16:55:43 -07:00
Subru Krishnan
66980c00f6
Bumping up hadoop-yarn-server-router module to 3.0.0-beta1 post rebase.
...
(cherry picked from commit 69e3ed2680
)
(cherry picked from commit 8737c2a8e9
)
2017-09-21 16:55:42 -07:00
Subru Krishnan
58e2458c2e
Bumping up yarn-server-router (new) module pom to beta1 and fixing imports post rebase.
...
(cherry picked from commit f1eff212c5
)
(cherry picked from commit 712e97d4cf
)
2017-09-21 16:55:41 -07:00
Subru Krishnan
fdbd214d4a
YARN-6821. Move FederationStateStore SQL DDL files from test resource to sbin.
...
(cherry picked from commit cd9db822f0
)
(cherry picked from commit 3e1dc7ece0
)
2017-09-21 16:55:40 -07:00
Subru Krishnan
055138c5c4
YARN-6815. FederationStateStoreFacade return behavior should be consistent irrespective of whether caching is enabled or not.
...
(cherry picked from commit 8820693cd7
)
(cherry picked from commit 825108d08a
)
2017-09-21 16:55:40 -07:00
Subru Krishnan
d3fb1711a4
YARN-6807. Adding required missing configs to Federation configuration guide based on e2e testing. (Tanuj Nayak via Subru).
...
(cherry picked from commit b4ac9d1b63
)
(cherry picked from commit f427e4201f
)
2017-09-21 16:55:39 -07:00
Subru Krishnan
169037cbb4
YARN-3659. Federation: routing client invocations transparently to multiple RMs. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 52daa6d971
)
(cherry picked from commit 43a97174fe
)
2017-09-21 16:55:33 -07:00
Subru Krishnan
8ed4daddc4
YARN-6724. Add ability to blacklist sub-clusters when invoking Routing policies. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit f8e5de5969
)
(cherry picked from commit 4cfec943b1
)
2017-09-21 16:55:01 -07:00
Subru Krishnan
35a38330ef
YARN-6511. Federation: transparently spanning application across multiple sub-clusters. (Botong Huang via Subru).
...
(cherry picked from commit 8c988d235e
)
(cherry picked from commit 70b1a757f1
)
2017-09-21 16:49:55 -07:00
Subru Krishnan
7ede8c1a53
YARN-3666. Federation Intercepting and propagating AM- home RM communications. (Botong Huang via Subru).
...
(cherry picked from commit 2399eb8200
)
(cherry picked from commit bed1832c93
)
2017-09-21 16:49:54 -07:00
Subru Krishnan
7dd6caff9e
YARN-6484. Documenting the YARN Federation feature. (Carlo Curino via Subru).
...
(cherry picked from commit 749ca890e6
)
(cherry picked from commit d6c29b2f5b
)
2017-09-21 16:49:54 -07:00
Subru Krishnan
b9fdc24e43
YARN-6666. Fix unit test failure in TestRouterClientRMService. (Botong Huang via Subru).
...
(cherry picked from commit e750907d0a
)
(cherry picked from commit 42949a6e01
)
2017-09-21 16:49:48 -07:00
Subru Krishnan
e4832bef6d
Bumping up hadoop-yarn-server-router module to 3.0.0-alpha4 post rebase.
...
(cherry picked from commit bd9c7b1bec
)
(cherry picked from commit f317e0cb48
)
2017-09-21 16:47:50 -07:00
Subru Krishnan
9476d86ce8
YARN-5531. UnmanagedAM pool manager for federating application across clusters. (Botong Huang via Subru).
...
(cherry picked from commit 73bb2102ce
)
(cherry picked from commit 859aa1f9d6
)
2017-09-21 16:47:43 -07:00
Subru Krishnan
7f00f938f2
YARN-5413. Create a proxy chain for ResourceManager Admin API in the Router. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 67846a5519
)
(cherry picked from commit 7444406d6d
)
2017-09-21 16:47:14 -07:00
Subru Krishnan
dc0a2e6c59
YARN-5411. Create a proxy chain for ApplicationClientProtocol in the Router. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 4846069061
)
(cherry picked from commit 80e1904000
)
2017-09-21 16:43:08 -07:00
Carlo Curino
3c5dfa0fe5
YARN-3663. Federation State and Policy Store (DBMS implementation). (Giovanni Matteo Fumarola via curino).
...
(cherry picked from commit be99c1fe2e
)
(cherry picked from commit 5272af8c7e
)
2017-09-21 16:33:31 -07:00
Carlo Curino
c38ac05bb4
YARN-6203: Occasional test failure in TestWeightedRandomRouterPolicy (curino)
...
(cherry picked from commit 121d55bd29
)
(cherry picked from commit c4bbdfeb66
)
2017-09-21 16:25:11 -07:00
Subru Krishnan
b9dcf9283e
YARN-5602. Utils for Federation State and Policy Store. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 326a2e6bde
)
(cherry picked from commit e1da8f0667
)
2017-09-21 16:25:10 -07:00
Carlo Curino
b40bdafe31
YARN-6370. Properly handle rack requests for non-active subclusters in LocalityMulticastAMRMProxyPolicy. (Contributed by Botong Huang via curino).
...
(cherry picked from commit ce419881c3
)
(cherry picked from commit 1c64e1709b
)
2017-09-21 16:25:10 -07:00
Subru Krishnan
61c07e4f3f
YARN-6247. Share a single instance of SubClusterResolver instead of instantiating one per AM. (Botong Huang via Subru)
...
(cherry picked from commit 51aeb2ce0c
)
(cherry picked from commit 86b2bec56e
)
2017-09-21 16:25:09 -07:00
Carlo Curino
6191fac914
YARN-6190. Validation and synchronization fixes in LocalityMulticastAMRMProxyPolicy. (Botong Huang via curino)
...
(cherry picked from commit 5c486961cd
)
(cherry picked from commit 8623644f45
)
2017-09-21 16:25:08 -07:00
Subru Krishnan
2dca88b924
YARN-5410. Bootstrap Router server module. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit 04f111394b
)
(cherry picked from commit 193ec456d6
)
2017-09-21 16:25:02 -07:00
Subru Krishnan
ce9110ab5c
YARN-6093. Minor bugs with AMRMtoken renewal and state store availability when using FederationRMFailoverProxyProvider during RM failover. (Botong Huang via Subru).
...
(cherry picked from commit 66500f4fa6
)
(cherry picked from commit 98b45b0ed3
)
2017-09-21 16:23:47 -07:00
Subru Krishnan
002a77dcce
YARN-5872. Add AlwayReject policies for router and amrmproxy. (Carlo Curino via Subru).
...
(cherry picked from commit db26bade05
)
(cherry picked from commit 91803305e5
)
2017-09-21 16:23:46 -07:00
Subru Krishnan
e933a1790c
YARN-5905. Update the RM webapp host that is reported as part of Federation membership to current primary RM's IP.
...
(cherry picked from commit e0f0e7afed
)
(cherry picked from commit 78032e51d9
)
2017-09-21 16:23:45 -07:00
Subru Krishnan
d32ffa9e5e
YARN-5676. Add a HashBasedRouterPolicy, and small policies and test refactoring. (Carlo Curino via Subru).
...
(cherry picked from commit 575137f41c
)
(cherry picked from commit 4128c9522d
)
2017-09-21 16:23:45 -07:00
Subru Krishnan
d87d2b546e
YARN-5634. Simplify initialization/use of RouterPolicy via a RouterPolicyFacade. (Carlo Curino via Subru).
...
(cherry picked from commit d7672ce2bd
)
(cherry picked from commit 083a05bc14
)
2017-09-21 16:23:44 -07:00
Subru Krishnan
b3a1ab711c
YARN-5391. PolicyManager to tie together Router/AMRM Federation policies. (Carlo Curino via Subru).
...
(cherry picked from commit 20893682ec
)
(cherry picked from commit 93d9fdeca6
)
2017-09-21 16:23:43 -07:00
Subru Krishnan
5c5ab53f3d
YARN-5325. Stateless ARMRMProxy policies implementation. (Carlo Curino via Subru).
...
(cherry picked from commit 11c5336522
)
(cherry picked from commit 1dadd0b45a
)
2017-09-21 16:23:42 -07:00
Subru Krishnan
f792f7b068
YARN-5324. Stateless Federation router policies implementation. (Carlo Curino via Subru).
...
(cherry picked from commit 1298127bda
)
(cherry picked from commit 0662996b6a
)
2017-09-21 16:23:42 -07:00
Subru Krishnan
23c42408ba
YARN-5323. Policies APIs for Federation Router and AMRMProxy policies. (Carlo Curino via Subru).
...
(cherry picked from commit f8208fe0b5
)
(cherry picked from commit 81472778d7
)
2017-09-21 16:23:41 -07:00
Jian He
aac8755125
YARN-5601. Make the RM epoch base value configurable. Contributed by Subru Krishnan
...
(cherry picked from commit 9ca2aba9cc
)
(cherry picked from commit 2797507d51
)
2017-09-21 16:23:40 -07:00
Subru Krishnan
8409fef0d9
YARN-5612. Return SubClusterId in FederationStateStoreFacade#addApplicationHomeSubCluster for Router Failover. (Giovanni Matteo Fumarola via Subru).
...
(cherry picked from commit ac1ba2a304
)
(cherry picked from commit 1882bc10fb
)
2017-09-21 16:23:39 -07:00
Jian He
ea2ecdb796
YARN-3671. Integrate Federation services with ResourceManager. Contributed by Subru Krishnan
...
(cherry picked from commit 8573c286e2
)
(cherry picked from commit 94a24567d6
)
2017-09-21 16:23:33 -07:00
Jian He
615c912b61
YARN-3673. Create a FailoverProxy for Federation services. Contributed by Subru Krishnan
...
(cherry picked from commit 3307564a5f
)
(cherry picked from commit bdfad4523f
)
2017-09-21 16:13:06 -07:00
Subru Krishnan
32a8618f39
YARN-5467. InputValidator for the FederationStateStore internal APIs. (Giovanni Matteo Fumarola via Subru)
...
(cherry picked from commit bd44182e70
)
(cherry picked from commit cfafd173bd
)
2017-09-21 16:13:06 -07:00