Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,390 Commits 326 Branches 371 Tags
Commit Graph

55 Commits

Author SHA1 Message Date
Haibo Chen
b19267259b YARN-8107. Give an informative message when incorrect format is used in ATSv2 filter attributes. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 024d7c08704e6a5fcc1f53a8f56a44c84c8d5fa0)
(cherry picked from commit b232dcab33c4f3e131b9699119167af8916879c4)
(cherry picked from commit ca105d1c8eb7563fcf7465ed48b8e7ec89eeee59)
 2018-04-06 09:47:25 -07:00
Haibo Chen
a7d8b4d775 YARN-6936. [Atsv2] Retrospect storing entities into sub application table from client perspective. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit f8b8bd53c4797d406bea5b1b0cdb179e209169cc)
(cherry picked from commit 6658018410defe3d7e4ea2a465e68d1cfdde6ada)
(cherry picked from commit 9cb378f5c762f3b26598d3ade448219e5801fa2f)
 2018-04-05 11:45:01 -07:00
Haibo Chen
45d012f208 YARN-7835. Race condition in NM while publishing events if second attempt is launched on the same node. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit d1274c3b71549cb000868500c293cafd880b3713)
 2018-02-28 21:24:17 -08:00
Varun Saxena
d72d2440f4 YARN-7662. [ATSv2] Define new set of configurations for reader and collectors to bind (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit c0aeb666a4d43aac196569d9ec6768d62139d2b9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2017-12-19 22:51:56 +05:30
Arun Suresh
71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Varun Saxena
56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
vrushali
8b98dba7ee YARN-7190 Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:41 +05:30
bibinchundatt
f7637b55bb YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena 
(cherry picked from commit dcd0bedcc8113dd5e1d852c242ae9511d1b3d58e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHATimelineCollectors.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/security/TimelineReaderWhitelistAuthorizationFilter.java
 2017-10-17 23:36:40 +05:30
Varun Saxena
1289b056ac YARN-6958. Addedndum for Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice 2017-10-17 23:36:40 +05:30
Varun Saxena
e5d63652df YARN-6861. Addendum for YARN-6861 for compilation issue (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:40 +05:30
Varun Saxena
186c99c6cc YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 73ee0d4fc05354527a4be6db9e5b83d06fc94047)
 2017-10-17 23:36:39 +05:30
Jian He
5c970f7c29 YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Jian He
55e5742394 YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Jason Lowe
db2f7dd9bd YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C 2017-10-17 23:36:39 +05:30
Rohith Sharma K S
cb75e1826e YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena. 
(cherry picked from commit ffb1f572b68e73efd6410ab74a334f4e5df543f2)
 2017-10-17 23:36:39 +05:30
Rohith Sharma K S
5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Jian He
50dd1372bf YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 7f06c46065936b88e9864c526bfcda3a89f3e257)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/AbstractTimelineReaderHBaseTestBase.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
 2017-10-17 23:36:39 +05:30
Jian He
9577900a91 YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 79dae624793164cd4692c86992a511310145858a)
 2017-10-17 23:36:39 +05:30
Varun Saxena
c8db08dfac YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 0443928d771e3e21825b4f487e8c0865ea641970)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S
4bea8af723 YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C. 
(cherry picked from commit 20dd6d18b7787e67ef96f3b6b92ea8415a8650fd)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S
484d7e9b39 YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena. 
(cherry picked from commit 743e4731781a7d9a4a5f1f09adc510f193182158)
 2017-10-17 23:36:38 +05:30
Haibo Chen
9b4d4228fb YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen) 
(cherry picked from commit 4c8b9d23a1b4f92bf370617a0b42ac68bdcf33ac)
 2017-10-17 23:36:38 +05:30
Vrushali Channapattan
70315de4ff YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30
Vrushali Channapattan
34570d4fc3 YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C) 2017-10-17 23:36:38 +05:30
Naganarasimha
e1d2f6ebdd YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang. 
(cherry picked from commit 0f1af3178eb2a38aaaa1e1a27edd409cda19a198)
 2017-10-17 23:36:37 +05:30
Varun Saxena
7338dbe5c0 YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena) 
(cherry picked from commit 793bbf216dc6d44ea57c778037bd461513fa6078)
 2017-10-17 23:36:37 +05:30
Varun Saxena
fd88fa8083 YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena) 
(cherry picked from commit b58777a9c9a5b6f2e4bcfd2b3bede33f25f80dec)
 2017-10-17 23:36:37 +05:30
Varun Saxena
203077cf2c YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena) 
(cherry picked from commit 063b513b1c10987461caab3d26c8543c6e657bf7)
 2017-10-17 23:36:37 +05:30
Varun Saxena
9595c18eea YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7f09cd1d5ee70e0bdf24cdecd78cd3bc258a40f8)
 2017-10-17 23:36:37 +05:30
Varun Saxena
6ec294b7fc YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ec32c4265bbf436ae61b6efc71ca33b16b70a23d)
 2017-10-17 23:36:37 +05:30
Varun Saxena
e597f2de60 YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 5d9ad152a1082a7c9b8edaf57a88ae471a537599)
 2017-10-17 23:36:37 +05:30
Varun Saxena
d996afd585 YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 63c06ec44e633567c378e28898e319143593ff30)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java
 2017-10-17 23:36:36 +05:30
Varun Saxena
59d96e247e YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ab192fd58358faff6880f3e0e867d7bdd4bb043a)
 2017-10-17 23:36:36 +05:30
Varun Saxena
74e43bf287 YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 649deb72fbb62568b4ea0d67444df6faaaed169d)
 2017-10-17 23:36:36 +05:30
Sangjin Lee
a16a684fbb YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen. 
(cherry picked from commit b92089c0e8ab1b87b8b5b55b1e3d4367ae5d847a)
 2017-10-17 23:36:36 +05:30
Sangjin Lee
5cb5b87c9a YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 
(cherry picked from commit 0327a79d79a4d56d9c7cb6889886afd2272b07d3)
 2017-10-17 23:36:36 +05:30
Varun Saxena
8c81f7fc08 Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's 
(cherry picked from commit c9246f619104bff44dd453a4ffe70104d37fd781)
 2017-10-17 23:36:36 +05:30
Varun Saxena
aa09001a33 YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 4b1ba4ea314147f8a06cc4f446c1d9336de89fc1)
 2017-10-17 23:36:36 +05:30
Varun Saxena
3429517a00 YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 6baea680ba6e5df6f254ced086d6defa64fb99f0)
 2017-10-17 23:36:36 +05:30
Varun Saxena
1e5e4a52f7 YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 9a2f288a6a9e30376b7fd99bf8824184a34a54c9)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
6aa7510681 YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C. 
(cherry picked from commit f945008d1cf5730bdebeae501ed0e42477ad219e)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
3394cb25d2 YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen. 
(cherry picked from commit 8288030cb4aa3b5a9425cc0a3f6df03a3eae1dfb)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
54705f549f YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen. 
(cherry picked from commit a5a55a54ab1568e941062ea3dabdd237f71f15c4)
(cherry picked from commit 17c4ab7ebb51088caf36fafedae8c256481eeed5)
 2017-10-17 23:36:35 +05:30
Naganarasimha
a5371f6b6e YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena. 
(cherry picked from commit c06114d6a360dddeb66c2dd9ad4fa5dae0cfbfb1)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
d6d73e3c99 YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee. 
(cherry picked from commit f511cc89b66997e496f630bdd299d3068d43fd31)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
4f595889e7 YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu. 
(cherry picked from commit 3b2e80881eaeaa82667496d706496765ed7e29f5)
 2017-10-17 23:36:35 +05:30
Sangjin Lee
4182f892e0 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis. 
(cherry picked from commit dc7d9bac79fc303f66e97c10379d680e0c11caa4)
 2017-10-17 23:36:34 +05:30
Sangjin Lee
ebf2f90528 YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S. 
(cherry picked from commit f37288c7e0127b564645e978c7aab2a186fa6be6)
 2017-10-17 23:36:34 +05:30
Varun Saxena
d19726ec37 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-10-17 23:36:34 +05:30
Varun Saxena
1d7fc52578 YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena) 2017-10-17 23:36:34 +05:30