13830 Commits

Author SHA1 Message Date
Varun Saxena
8c81f7fc08 Addendum for YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's
(cherry picked from commit c9246f619104bff44dd453a4ffe70104d37fd781)
2017-10-17 23:36:36 +05:30
Varun Saxena
aa09001a33 YARN-6064. Support fromId for flowRuns and flow/flowRun apps REST API's (Rohith Sharma K S via Varun Saxena)
(cherry picked from commit 4b1ba4ea314147f8a06cc4f446c1d9336de89fc1)
2017-10-17 23:36:36 +05:30
Varun Saxena
3429517a00 YARN-5378. Accommodate app-id->cluster mapping (Sangjin Lee via Varun Saxena)
(cherry picked from commit 6baea680ba6e5df6f254ced086d6defa64fb99f0)
2017-10-17 23:36:36 +05:30
Sangjin Lee
5b3ef55f5f YARN-5980. Update documentation for single node hbase deploy. Contributed by Vrushali C.
(cherry picked from commit e1bdba77888723b435a235a96c8659029afd25d5)
2017-10-17 23:36:36 +05:30
Varun Saxena
d12920b10e YARN-6074. FlowRunEntity does not deserialize long values correctly (Rohith Sharma K S via Varun Saxena)
(cherry picked from commit db490eccced3c42ac27253ca6cbaf10a77e0e116)
2017-10-17 23:36:36 +05:30
Varun Saxena
1e5e4a52f7 YARN-5585. [Atsv2] Reader side changes for entity prefix and support for pagination via additional filters (Rohith Sharma K S via Varun Saxena)
(cherry picked from commit 9a2f288a6a9e30376b7fd99bf8824184a34a54c9)
2017-10-17 23:36:35 +05:30
Naganarasimha
d3ef478955 YARN-6026. A couple of spelling errors in the docs. Contributed by Grant Sohn. 2017-10-17 23:36:35 +05:30
Sangjin Lee
6aa7510681 YARN-5976. Update hbase version to 1.2. Contributed by Vrushali C.
(cherry picked from commit f945008d1cf5730bdebeae501ed0e42477ad219e)
2017-10-17 23:36:35 +05:30
Sangjin Lee
3394cb25d2 YARN-5925. Extract hbase-backend-exclusive utility methods from TimelineStorageUtil. Contributed by Haibo Chen.
(cherry picked from commit 8288030cb4aa3b5a9425cc0a3f6df03a3eae1dfb)
2017-10-17 23:36:35 +05:30
Sangjin Lee
c8a8e680bd MAPREDUCE-6818. Remove direct reference to TimelineClientImpl. Contributed by Li Lu.
(cherry picked from commit 6217b87f4a056cf704cef2e073b386b7803415de)
2017-10-17 23:36:35 +05:30
Sangjin Lee
54705f549f YARN-5922. Remove direct references of HBaseTimelineWriter/Reader in core ATS classes. Contributed by Haibo Chen.
(cherry picked from commit a5a55a54ab1568e941062ea3dabdd237f71f15c4)
(cherry picked from commit 17c4ab7ebb51088caf36fafedae8c256481eeed5)
2017-10-17 23:36:35 +05:30
Naganarasimha
a5371f6b6e YARN-5572. HBaseTimelineWriterImpl appears to reference a bad property name. Contributed by Varun Saxena.
(cherry picked from commit c06114d6a360dddeb66c2dd9ad4fa5dae0cfbfb1)
2017-10-17 23:36:35 +05:30
Sangjin Lee
d6d73e3c99 YARN-5433. Audit dependencies for Category-X. Contributed by Sangjin Lee.
(cherry picked from commit f511cc89b66997e496f630bdd299d3068d43fd31)
2017-10-17 23:36:35 +05:30
Sangjin Lee
4f595889e7 YARN-5739. Provide timeline reader API to list available timeline entity types for one application. Contributed by Li Lu.
(cherry picked from commit 3b2e80881eaeaa82667496d706496765ed7e29f5)
2017-10-17 23:36:35 +05:30
Vrushali Channapattan
2fa2c3169e YARN-4765 Split TestHBaseTimelineStorage into multiple test classes (Varun Saxena via Vrushali C)
(cherry picked from commit 513dcf6817dd76fde8096ff04cd888d7c908461d)
(cherry picked from commit 022bf783aa89c1c81374ebef5dba2df95b7563b5)
2017-10-17 23:36:35 +05:30
Sangjin Lee
b8cfb4fcb3 YARN-5792. Adopt the id prefix for YARN, MR, and DS entities. Contributed by Varun Saxena.
(cherry picked from commit f734977b27a514ce0561638c0a6a17b1ef093026)
2017-10-17 23:36:35 +05:30
Varun Saxena
684c2d149f YARN-3359. Addendum for Recover collector list when RM fails over (Li Lu via Varun Saxena)
(cherry picked from commit 25b19178dd53bc0d57ca5fd64d3464af7b59e588)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
2017-10-17 23:36:35 +05:30
Varun Saxena
484b1d2ec1 YARN-5599. Publish AM launch command to ATS (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:34 +05:30
Varun Saxena
6562a354a9 YARN-3359. Recover collector list when RM fails over (Li Lu via Varun Saxena)
(cherry picked from commit 1c8a57550bea2280d41a208baf6fea59f66c0d75)
2017-10-17 23:36:34 +05:30
Sangjin Lee
4182f892e0 YARN-5265. Make HBase configuration for the timeline service configurable. Contributed by Joep Rottinghuis.
(cherry picked from commit dc7d9bac79fc303f66e97c10379d680e0c11caa4)
2017-10-17 23:36:34 +05:30
Sangjin Lee
ebf2f90528 YARN-5715. Introduce entity prefix for return and sort order. Contributed by Rohith Sharma K S.
(cherry picked from commit f37288c7e0127b564645e978c7aab2a186fa6be6)
2017-10-17 23:36:34 +05:30
Varun Saxena
d19726ec37 YARN-3649. Allow configurable prefix for hbase table names like prod, exp, test etc (Vrushali C via Varun Saxena) 2017-10-17 23:36:34 +05:30
Varun Saxena
1d7fc52578 YARN-5747. Application timeline metric aggregation in timeline v2 will lose last round aggregation when an application finishes (Li Lu via Varun Saxena) 2017-10-17 23:36:34 +05:30
Sangjin Lee
3344f0ace5 YARN-5561. [Atsv2] : Support for ability to retrieve apps/app-attempt/containers and entities via REST. Contributed by Rohith Sharma K S.
(cherry picked from commit e9c4616b5e47e9c616799abc532269572ab24e6e)
2017-10-17 23:36:34 +05:30
Varun Saxena
c3db98aa10 YARN-5743. [Atsv2] Publish queue name and RMAppMetrics to ATS (Rohith Sharma K S via Varun Saxena) 2017-10-17 23:36:34 +05:30
Sangjin Lee
853a5da041 YARN-5699. Retrospect yarn entity fields which are publishing in events info fields. Contributed by Rohith Sharma K S.
(cherry picked from commit 787d5942003a5441ba455a79fa08da28f60d5777)
2017-10-17 23:36:34 +05:30
Sangjin Lee
4675f22e6a YARN-5638. Introduce a collector timestamp to uniquely identify collectors creation order in collector discovery. Contributed by Li Lu.
(cherry picked from commit 0c1863144649ea265da65ce25158707cc3a3fb4a)
2017-10-17 23:36:34 +05:30
Li Lu
6a517d714e YARN-5229. Refactor #isApplicationEntity and #getApplicationEvent from HBaseTimelineWriterImpl. (Vrushali C via gtcarrera9)
(cherry picked from commit 5d7ad396dbf84ffb0bacb3372d149828dfb4b4ca)
2017-10-17 23:36:34 +05:30
Sangjin Lee
51102019f5 YARN-5156. YARN_CONTAINER_FINISHED of YARN_CONTAINERs will always have running state (Vrushali C via sjlee)
(cherry picked from commit d0a62d8adf86f7cb2481940d212788cc5f327367)
2017-10-17 23:36:34 +05:30
Sangjin Lee
4f6ba3187b YARN-5379. TestHBaseTimelineStorage. testWriteApplicationToHBase() fails intermittently (Vrushali C via sjlee)
(cherry picked from commit e549a9af3177f6ee83477cde8bd7d0ed72d6ecec)
2017-10-17 23:36:34 +05:30
Naganarasimha
22f5046acb YARN-5364. timelineservice modules have indirect dependencies on mapreduce artifacts. Contributed by Sangjin Lee
(cherry picked from commit af8f480c2482b40e9f5a2d29fb5bc7069979fa2e)
2017-10-17 23:36:34 +05:30
Varun Saxena
057a4728ff YARN-5359. FileSystemTimelineReader/Writer uses unix-specific default storage path (Sangjin Lee via Varun Saxena)
(cherry picked from commit 56142171b9528646f26072e022902549a16c8f27)
2017-10-17 23:36:33 +05:30
Varun Saxena
0820df611d MAPREDUCE-6731. TestMRTimelineEventHandling#testMRNewTimelineServiceEventHandling fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena).
(cherry picked from commit d14e729a55c0ca4053c537290392b74f732ce939)
2017-10-17 23:36:33 +05:30
Varun Saxena
16942ea115 YARN-5354. TestDistributedShell.checkTimelineV2 fails randomly for concurrent tests. (Sangjin Lee via Varun Saxena).
(cherry picked from commit a3ac1c1943518c2a242dd21dac0f17f21bc3cd7f)
2017-10-17 23:36:33 +05:30
Sangjin Lee
e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Xiao Chen
a103b323e6 Revert "HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar."
This reverts commit 1452ba139ea845ad9e82a5902d239b59a789bff3.
2017-10-17 10:11:38 -07:00
Robert Kanter
8a1ed84c77 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-16 16:51:28 -07:00
Wei-Chiu Chuang
1452ba139e HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar.
(cherry picked from commit e906108fc98a011630d12a43e557b81d7ef7ea5d)
2017-10-16 16:44:35 -07:00
Nathan Roberts
b876a9314e YARN-7333. container-executor fails to remove entries from a directory that is not writable or executable. Contributed by Jason Lowe.
(cherry picked from commit 8620140a6a3ec0117675ede06d92d830da3da551)
2017-10-16 16:44:34 -05:00
Junping Du
3d2ef54450 YARN-7124. LogAggregationTFileController deletes/renames while file is open. Contributed by Jason Lowe.
(cherry picked from commit 1fcbe7cf5f2f7c609cd4158912f151848268ca9e)
2017-10-16 14:29:02 -07:00
Xiao Chen
0bddcf10ef HADOOP-14949. TestKMS#testACLs fails intermittently.
(cherry picked from commit 152317d40ece8abfa76fb5bd29f04878c1b0d3d8)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
2017-10-16 14:20:05 -07:00
Arun Suresh
53cd33cc4a YARN-7275. NM Statestore cleanup for Container updates. (Kartheek Muthyala via asuresh)
(cherry picked from commit a50be1b8f432f50c940b66d12c7de87b95ea47c0)
2017-10-16 13:27:08 -07:00
Wei-Chiu Chuang
a596c6f5eb HDFS-12485. expunge may fail to remove trash from encryption zone. Contributed by Wei-Chiu Chuang.
(cherry picked from commit 8dbc8909c92d502d10a7f94d1de3171878a43b04)
2017-10-16 12:59:41 -07:00
Xiao Chen
b2f27641dd HDFS-12642. Log block and datanode details in BlockRecoveryWorker.
(cherry picked from commit 971b3ca1acc15d9a1f8e774291813dafd16ef704)
2017-10-16 10:34:24 -07:00
Xiao Chen
d82edb49c7 HDFS-12603. Enable async edit logging by default. Contributed by Andrew Wang.
(cherry picked from commit d0311dfc5fe92a0e0f3fb8ae92e412edce609740)
2017-10-16 09:51:27 -07:00
Steve Loughran
8902178d3e HADOOP-14935. Azure: POSIX permissions are taking effect in access() method even when authorization is enabled.
Contributed by Santhosh G Nayak

Cherry-picked from commit 9fcc3a1fc8cab873034f5c308ceb2d5671a954e8
2017-10-16 16:06:41 +01:00
Xiao Chen
4b734fe386 HDFS-12659. Update TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to increase heartbeat recheck interval. Contributed by Ajay Kumar.
(cherry picked from commit 11a7729c1474aa379a62c0440c8c1a11be261a75)
2017-10-15 19:28:38 -07:00
Wangda Tan
2c1381300f YARN-7321. Backport container-executor changes from YARN-6852 to branch-2. (Varun Vasudev via wangda)
Change-Id: Ib6434b5a5ebe591050c3949cac12ff9e2a2bbaa5
2017-10-14 10:48:26 -07:00
Wangda Tan
64170eb53c Addendum fix for: YARN-7269. Tracking URL in the app state does not get redirected to ApplicationMaster for Running applications. (Wangda Tan)
Change-Id: If1fe4a62e07b25e6f1b8ba803495da87e2cb2df6
(cherry picked from commit 87ea1dff9c431fb88e064e497b35fc17c65f1d53)
2017-10-14 10:47:54 -07:00
Yufei Gu
f2350f5120 YARN-7270. Fix unsafe casting from long to int for class Resource and its sub-classes. (Yufei)
(cherry picked from commit 7a27c2c367518e1bf6ee393391a2f9b412113819)
(cherry picked from commit dbf6c48af9aa2f3b41b1146643db6eb0140b38c5)
2017-10-13 13:10:20 -07:00