Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,076 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

14076 Commits

Author SHA1 Message Date
Jason Lowe db2f7dd9bd YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C 2017-10-17 23:36:39 +05:30
Rohith Sharma K S cb75e1826e YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena. 
(cherry picked from commit ffb1f572b68e73efd6410ab74a334f4e5df543f2)
 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Varun Saxena 5bda387573 YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena) 
(cherry picked from commit 7b2cb0614a04e6d0e2bc6dfd5095b7535dffa678)
 2017-10-17 23:36:39 +05:30
Varun Saxena 8d9d40aa53 Addendum for YARN-6102. 2017-10-17 23:36:39 +05:30
Varun Saxena cb5d601b2e Addendum for YARN-5648. 2017-10-17 23:36:39 +05:30
Jian He 8e4e5abbe6 YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena 
(cherry picked from commit 3829100cf75e733b2b3a4e8c6806a3696f4ae658)
 2017-10-17 23:36:39 +05:30
Jian He 50dd1372bf YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 7f06c46065936b88e9864c526bfcda3a89f3e257)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/AbstractTimelineReaderHBaseTestBase.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
 2017-10-17 23:36:39 +05:30
Jian He 9577900a91 YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 79dae624793164cd4692c86992a511310145858a)
 2017-10-17 23:36:39 +05:30
Varun Saxena c8db08dfac YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 0443928d771e3e21825b4f487e8c0865ea641970)
 2017-10-17 23:36:38 +05:30
Sunil G 4b65fe73ca YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 
(cherry picked from commit a657472b42)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 4bea8af723 YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C. 
(cherry picked from commit 20dd6d18b7787e67ef96f3b6b92ea8415a8650fd)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 4014390ef4 YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C. 
(cherry picked from commit 416312e3a7f0f83cc1c8549640c3ba75aaa3d887)
 2017-10-17 23:36:38 +05:30
Vrushali C 0f751a7fec YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 484d7e9b39 YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena. 
(cherry picked from commit 743e4731781a7d9a4a5f1f09adc510f193182158)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S d01a3f1ba2 YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C. 
(cherry picked from commit 5791ced4bd864ff6c08cb2c3ee10ab62008f0168)
 2017-10-17 23:36:38 +05:30
Varun Saxena 82de1fd84a YARN-6707. [ATSv2] Update HBase version to 1.2.6 (Vrushali C via Varun Saxena) 
(cherry picked from commit e86eef9812)

Conflicts:
	hadoop-project/pom.xml
 2017-10-17 23:36:38 +05:30
Varun Saxena c4f042d4c1 YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena) 
(cherry picked from commit 0b7bff706e)
 2017-10-17 23:36:38 +05:30
vrushali 48d2e4f519 YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30
Varun Saxena 473f2f00ee YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena) 
(cherry picked from commit f7cba98504a5e968bb6ae976d20b71c84a4664f2)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/apptoflow/AppToFlowColumn.java
 2017-10-17 23:36:38 +05:30
Haibo Chen d53038626e YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 47474fffac)
(cherry picked from commit 8817cb5c8424359b880c6d700e53092f0269c1bb)
 2017-10-17 23:36:38 +05:30
Haibo Chen 9b4d4228fb YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen) 
(cherry picked from commit 4c8b9d23a1b4f92bf370617a0b42ac68bdcf33ac)
 2017-10-17 23:36:38 +05:30
Vrushali Channapattan 70315de4ff YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30
Vrushali Channapattan 34570d4fc3 YARN-6375 App level aggregation should not consider metric values reported in the previous aggregation cycle (Varun Saxena via Vrushali C) 2017-10-17 23:36:38 +05:30
Naganarasimha e1d2f6ebdd YARN-6518. Fix warnings from Spotbugs in hadoop-yarn-server-timelineservice. Contributed by Weiwei Yang. 
(cherry picked from commit 0f1af3178e)
 2017-10-17 23:36:37 +05:30
Varun Saxena 7338dbe5c0 YARN-6455. Enhance the timelinewriter.flush() race condition fix (Haibo Chen via Varun Saxena) 
(cherry picked from commit 793bbf216d)
 2017-10-17 23:36:37 +05:30
Varun Saxena c1f9f0529b Addendum for YARN-5865. Retrospect updateApplicationPriority api to handle state store exception in align with YARN-5611 2017-10-17 23:36:37 +05:30
Varun Saxena ebcb575a12 Addendum for YARN-4218. Metric for resource*time that was preempted. 2017-10-17 23:36:37 +05:30
Rohith Sharma K S 1072232741 YARN-6424. TimelineCollector is not stopped when an app finishes in RM. Contributed by Varun Saxena. 
(cherry picked from commit 1a9439e299)
 2017-10-17 23:36:37 +05:30
Varun Saxena 16d8df68f6 YARN-6377. NMTimelinePublisher#serviceStop does not stop timeline clients (Haibo Chen via Varun Saxena) 
(cherry picked from commit a4b5aa8493)
 2017-10-17 23:36:37 +05:30
Varun Saxena fd88fa8083 YARN-6376. Exceptions caused by synchronous putEntities requests can be swallowed (Haibo Chen via Varun Saxena) 
(cherry picked from commit b58777a9c9)
 2017-10-17 23:36:37 +05:30
Varun Saxena debe526f08 YARN-6342. Make TimelineV2Client's drain timeout after stop configurable (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7c2bc444b3)
 2017-10-17 23:36:37 +05:30
Varun Saxena 203077cf2c YARN-6357. Implement putEntitiesAsync API in TimelineCollector (Haibo Chen via Varun Saxena) 
(cherry picked from commit 063b513b1c)
 2017-10-17 23:36:37 +05:30
Varun Saxena 9595c18eea YARN-6146. Add Builder methods for TimelineEntityFilters (Haibo Chen via Varun Saxena) 
(cherry picked from commit 7f09cd1d5ee70e0bdf24cdecd78cd3bc258a40f8)
 2017-10-17 23:36:37 +05:30
Varun Saxena 1369abf5b0 YARN-6318. timeline service schema creator fails if executed from a remote machine (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 68ec0d9b471356d3adef15d78111126f57ee50f8)
 2017-10-17 23:36:37 +05:30
Varun Saxena 6ec294b7fc YARN-6237. Move UID constant to TimelineReaderUtils (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ec32c4265bbf436ae61b6efc71ca33b16b70a23d)
 2017-10-17 23:36:37 +05:30
Varun Saxena e597f2de60 YARN-6256. Add FROM_ID info key for timeline entities in reader response (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 5d9ad152a1082a7c9b8edaf57a88ae471a537599)
 2017-10-17 23:36:37 +05:30
Sangjin Lee 16c607ca2b YARN-6027 addendum. Fixed the broken build for YARN-5355-branch-2. Contributed by Sangjin Lee. 2017-10-17 23:36:37 +05:30
Varun Saxena d996afd585 YARN-6027. Support fromid(offset) filter for /flows API (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 63c06ec44e633567c378e28898e319143593ff30)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/TestTimelineReaderWebServicesHBaseStorage.java
 2017-10-17 23:36:36 +05:30
Sangjin Lee 6b5b8f090d YARN-6253. FlowAcitivityColumnPrefix.store(byte[] rowKey, ...) drops timestamp. Contributed by Haibo Chen. 
(cherry picked from commit 34e7c30293b5a56f9f745769a29c5666bdb85d6c)
 2017-10-17 23:36:36 +05:30
Varun Saxena 59d96e247e YARN-6069. CORS support in timeline v2 (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ab192fd58358faff6880f3e0e867d7bdd4bb043a)
 2017-10-17 23:36:36 +05:30
Varun Saxena 4a3a4aaf7e YARN-6159. Documentation changes for TimelineV2Client (Naganarasimha G R via Varun Saxena) 
(cherry picked from commit 6ba61d20d3)
 2017-10-17 23:36:36 +05:30
Varun Saxena 74e43bf287 YARN-6170. TimelineReaderServer should wait to join with HttpServer2 (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 649deb72fb)
 2017-10-17 23:36:36 +05:30
Sangjin Lee bb100d35fd YARN-4675. Reorganize TimelineClient and TimelineClientImpl into separate classes for ATSv1.x and ATSv2. Contributed by Naganarasimha G R. 
(cherry picked from commit 73235ab30361b41293846189f3c5fef321ae7cac)
 2017-10-17 23:36:36 +05:30
Sangjin Lee a16a684fbb YARN-5928. Move ATSv2 HBase backend code into a new module that is only dependent at runtime by yarn servers. Contributed by Haibo Chen. 
(cherry picked from commit b92089c0e8ab1b87b8b5b55b1e3d4367ae5d847a)
 2017-10-17 23:36:36 +05:30
Sangjin Lee 5cb5b87c9a YARN-6094. Update the coprocessor to be a dynamically loaded one. Contributed by Vrushali C. 
(cherry picked from commit 0327a79d79a4d56d9c7cb6889886afd2272b07d3)
 2017-10-17 23:36:36 +05:30
Varun Saxena 8c81f7fc08 Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's 
(cherry picked from commit c9246f619104bff44dd453a4ffe70104d37fd781)
 2017-10-17 23:36:36 +05:30
Varun Saxena aa09001a33 YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 4b1ba4ea314147f8a06cc4f446c1d9336de89fc1)
 2017-10-17 23:36:36 +05:30
Varun Saxena 3429517a00 YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena) 
(cherry picked from commit 6baea680ba6e5df6f254ced086d6defa64fb99f0)
 2017-10-17 23:36:36 +05:30
Sangjin Lee 5b3ef55f5f YARN-5980. Update documentation for single node hbase deploy. Contributed by Vrushali C. 
(cherry picked from commit e1bdba77888723b435a235a96c8659029afd25d5)
 2017-10-17 23:36:36 +05:30