Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14,503 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

14503 Commits

Author SHA1 Message Date
Varun Saxena 56e0873e22 Fix javac, javadoc and checkstyle issues before branch-2 merge. 2017-10-17 23:36:41 +05:30
vrushali 8b98dba7ee YARN-7190 Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:41 +05:30
Varun Saxena 6ba1f34532 Fix problems in yarn.cmd due to rebase. 2017-10-17 23:36:41 +05:30
Haibo Chen 572ebd1a95 YARN-7063. TestTimelineReaderWebServicesHBaseStorage fails with NoClassDefFoundError on TSv2 branch2. (Varun Saxena via Haibo Chen) 2017-10-17 23:36:41 +05:30
Haibo Chen d77530b227 YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen) 
(cherry picked from commit 661f5eb0c6)
 2017-10-17 23:36:41 +05:30
Rohith Sharma K S 27019b5a66 YARN-7130. ATSv2 documentation changes post merge. Contributed by Varun Saxena. 
(cherry picked from commit 0bfb3a2566)
 2017-10-17 23:36:41 +05:30
Rohith Sharma K S 138d6fc083 YARN-7140. CollectorInfo should have Public visibility. Contributed by Varun Saxena. 
(cherry picked from commit 4a83170be4)
 2017-10-17 23:36:40 +05:30
Varun Saxena 95f45b89d1 YARN-7152. [ATSv2] Registering timeline client before AMRMClient service init throw exception (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ef87d34a0e)
 2017-10-17 23:36:40 +05:30
Junping Du b3201b2dab YARN-6534. ResourceManager failed due to TimelineClient try to init SSLFactory even https is not enabled. Contributed by Rohith Sharma K S. 
(cherry picked from commit 09e36ff59c05b8bd54d346d1103de0a7300875c6)
(cherry picked from commit 94607214ce)
 2017-10-17 23:36:40 +05:30
Haibo Chen 6e2de8b417 YARN-6435. [ATSv2] Can't retrieve more than 1000 versions of metrics in time series. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 461ee44d28)
 2017-10-17 23:36:40 +05:30
bibinchundatt f7637b55bb YARN-7141. Move logging APIs to slf4j in timelineservice after ATSv2 merge. Contributed by Varun Saxena 
(cherry picked from commit dcd0bedcc8)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMHATimelineCollectors.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/security/TimelineReaderWhitelistAuthorizationFilter.java
 2017-10-17 23:36:40 +05:30
Varun Saxena 1289b056ac YARN-6958. Addedndum for Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice 2017-10-17 23:36:40 +05:30
Varun Saxena 87713f41b6 YARN-6905. Addendum to fix TestTimelineReaderWebServicesHBaseStorage due to missing FastNumberFormat 
(cherry picked from commit e52f26e5cf)
 2017-10-17 23:36:40 +05:30
Vrushali C a2598434ed YARN-6905 Multiple HBaseTimelineStorage test failures due to missing FastNumberFormat (Contributed by Haibo Chen) 
(cherry picked from commit 608a06cca5)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/storage/TestHBaseTimelineStorageEntities.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/common/HBaseTimelineStorageUtils.java
 2017-10-17 23:36:40 +05:30
Varun Saxena 78a6ecccb3 Addendum for YARN-4753. Use doxia macro to generate in-page TOC of YARN site documentation 2017-10-17 23:36:40 +05:30
Varun Saxena 3d5bf1c06d YARN-6047. Documentation updates for TimelineService v2. Contributed by Rohith Sharma K S, Vrushali C and Varun Saxena 2017-10-17 23:36:40 +05:30
Varun Saxena a22c1c72c4 YARN-6316. Addendum for missing create option in documentation. 2017-10-17 23:36:40 +05:30
Varun Saxena 28bbe9ccd7 YARN-5577. [Atsv2] Document object passing in infofilters with an example (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit ea29e3bc27)
 2017-10-17 23:36:40 +05:30
Haibo Chen 0f51f15097 YARN-6323. Rolling upgrade/config change is broken on timeline v2. (Vrushali C via Haibo Chen) 2017-10-17 23:36:40 +05:30
Jian He cbd5e89b24 MAPREDUCE-6838. [ATSv2 Security] Add timeline delegation token received in allocate response to UGI. Contributed by Varun Saxena 
(cherry picked from commit c6ce422a3423305ac665f80f039f61294adaa32c)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerAllocator.java
 2017-10-17 23:36:40 +05:30
Varun Saxena e5d63652df YARN-6861. Addendum for YARN-6861 for compilation issue (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:40 +05:30
Varun Saxena 186c99c6cc YARN-6861. Reader API for sub application entities (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 73ee0d4fc05354527a4be6db9e5b83d06fc94047)
 2017-10-17 23:36:39 +05:30
Jian He 5c970f7c29 YARN-6134. [ATSv2 Security] Regenerate delegation token for app just before token expires if app collector is active. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 5d9b4bd3e8 YARN-7041. Nodemanager NPE running jobs with security off. Contributed by Varun Saxena. 
(cherry picked from commit e976bf10bb0d42e4d971358554ea736ddcdafd19)
 2017-10-17 23:36:39 +05:30
Jian He 53ddf1b15f YARN-7038. Addendum patch. Fix compilation error. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 4e1af68ae1 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 
(cherry picked from commit 387dd3da87759860dd18bd8aa95c60d962b1e6a9)
 2017-10-17 23:36:39 +05:30
Jian He 55e5742394 YARN-7006. [ATSv2 Security] Changes for authentication for CollectorNodemanagerProtocol. Contributed by Varun Saxena 2017-10-17 23:36:39 +05:30
Jason Lowe db2f7dd9bd YARN-6820. Restrict read access to timelineservice v2 data. Contributed by Vrushali C 2017-10-17 23:36:39 +05:30
Rohith Sharma K S cb75e1826e YARN-6133. [ATSv2 Security] Renew delegation token for app automatically if an app collector is active. Contributed by Varun Saxena. 
(cherry picked from commit ffb1f572b68e73efd6410ab74a334f4e5df543f2)
 2017-10-17 23:36:39 +05:30
Rohith Sharma K S 5a7c272e0a YARN-6130. [ATSv2 Security] Generate a delegation token for AM when app collector is created and pass it to AM via NM and RM. Contributed by Varun Saxena. 2017-10-17 23:36:39 +05:30
Varun Saxena 5bda387573 YARN-6874. Supplement timestamp for min start/max end time columns in flow run table to avoid overwrite (Vrushali C via Varun Saxena) 
(cherry picked from commit 7b2cb0614a04e6d0e2bc6dfd5095b7535dffa678)
 2017-10-17 23:36:39 +05:30
Varun Saxena 8d9d40aa53 Addendum for YARN-6102. 2017-10-17 23:36:39 +05:30
Varun Saxena cb5d601b2e Addendum for YARN-5648. 2017-10-17 23:36:39 +05:30
Jian He 8e4e5abbe6 YARN-5648. [ATSv2 Security] Client side changes for authentication. Contributed by Varun Saxena 
(cherry picked from commit 3829100cf75e733b2b3a4e8c6806a3696f4ae658)
 2017-10-17 23:36:39 +05:30
Jian He 50dd1372bf YARN-6638. [ATSv2 Security] Timeline reader side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 7f06c46065936b88e9864c526bfcda3a89f3e257)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/reader/AbstractTimelineReaderHBaseTestBase.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/reader/TimelineReaderServer.java
 2017-10-17 23:36:39 +05:30
Jian He 9577900a91 YARN-5647. [ATSv2 Security] Collector side changes for loading auth filters and principals. Contributed by Varun Saxena 
(cherry picked from commit 79dae624793164cd4692c86992a511310145858a)
 2017-10-17 23:36:39 +05:30
Varun Saxena c8db08dfac YARN-6734. Ensure sub-application user is extracted & sent to timeline service (Rohith Sharma K S via Varun Saxena) 
(cherry picked from commit 0443928d771e3e21825b4f487e8c0865ea641970)
 2017-10-17 23:36:38 +05:30
Sunil G 4b65fe73ca YARN-6102. RMActiveService context to be updated with new RMContext on failover. Contributed by Rohith Sharma K S. 
(cherry picked from commit a657472b42)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMActiveServiceContext.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMContextImpl.java
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 4bea8af723 YARN-6888. Refactor AppLevelTimelineCollector such that RM does not have aggregator threads created. Contributed by Vrushali C. 
(cherry picked from commit 20dd6d18b7787e67ef96f3b6b92ea8415a8650fd)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 4014390ef4 YARN-6733. Add table for storing sub-application entities. Contributed by Vrushali C. 
(cherry picked from commit 416312e3a7f0f83cc1c8549640c3ba75aaa3d887)
 2017-10-17 23:36:38 +05:30
Vrushali C 0f751a7fec YARN-6850 Ensure that supplemented timestamp is stored only for flow run metrics (Contributed by Varun Saxena via Vrushali C) 2017-10-17 23:36:38 +05:30
Rohith Sharma K S 484d7e9b39 YARN-4455. Support fetching metrics by time range. Contributed by Varun Saxena. 
(cherry picked from commit 743e4731781a7d9a4a5f1f09adc510f193182158)
 2017-10-17 23:36:38 +05:30
Rohith Sharma K S d01a3f1ba2 YARN-6801. NPE in RM while setting collectors map in NodeHeartbeatResponse. Contributed by Vrushali C. 
(cherry picked from commit 5791ced4bd864ff6c08cb2c3ee10ab62008f0168)
 2017-10-17 23:36:38 +05:30
Varun Saxena 82de1fd84a YARN-6707. [ATSv2] Update HBase version to 1.2.6 (Vrushali C via Varun Saxena) 
(cherry picked from commit e86eef9812)

Conflicts:
	hadoop-project/pom.xml
 2017-10-17 23:36:38 +05:30
Varun Saxena c4f042d4c1 YARN-6604. Allow metric TTL for Application table to be specified through cmd (Haibo Chen via Varun Saxena) 
(cherry picked from commit 0b7bff706e)
 2017-10-17 23:36:38 +05:30
vrushali 48d2e4f519 YARN-6316 Provide help information and documentation for TimelineSchemaCreator (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30
Varun Saxena 473f2f00ee YARN-6658. Remove columnFor() methods of Columns in HBaseTimeline backend (Haibo Chen via Varun Saxena) 
(cherry picked from commit f7cba98504a5e968bb6ae976d20b71c84a4664f2)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/src/main/java/org/apache/hadoop/yarn/server/timelineservice/storage/apptoflow/AppToFlowColumn.java
 2017-10-17 23:36:38 +05:30
Haibo Chen d53038626e YARN-6555. Store application flow context in NM state store for work-preserving restart. (Rohith Sharma K S via Haibo Chen) 
(cherry picked from commit 47474fffac)
(cherry picked from commit 8817cb5c8424359b880c6d700e53092f0269c1bb)
 2017-10-17 23:36:38 +05:30
Haibo Chen 9b4d4228fb YARN-6561. Update exception message during timeline collector aux service initialization. (Vrushali C via Haibo Chen) 
(cherry picked from commit 4c8b9d23a1b4f92bf370617a0b42ac68bdcf33ac)
 2017-10-17 23:36:38 +05:30
Vrushali Channapattan 70315de4ff YARN-6563 ConcurrentModificationException in TimelineCollectorManager while stopping RM (Contributed by Haibo Chen via Vrushali C) 2017-10-17 23:36:38 +05:30