Commit Graph

16687 Commits

Author SHA1 Message Date
Robert Kanter aa21b01fcb YARN-7146. Many RM unit tests failing with FairScheduler (rkanter)
(cherry picked from commit bb34ae9554)
2017-09-13 15:45:28 -07:00
Jason Lowe 12e9558ed1 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim
(cherry picked from commit 4d98936eec)
2017-09-13 17:22:29 -05:00
Arpit Agarwal 78e9a98c4a HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:11 -07:00
Eric Payne 4f6e8e08dc YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe.
(cherry picked from commit 3860be7961)
2017-09-13 14:46:04 -05:00
Andrew Wang 7b6b2dc93a HADOOP-14857. Fix downstream shaded client integration test. Contributed by Sean Busbey.
(cherry picked from commit 8277fab2be)
2017-09-13 10:59:08 -07:00
Chen Liang d1fcab3ea9 HADOOP-14804. correct wrong parameters format order in core-default.xml. Contributed by Chen Hongfei. 2017-09-13 10:55:13 -07:00
Andrew Wang 4687d3d6f6 HDFS-12222. Document and test BlockLocation for erasure-coded files. Contributed by Huafeng Wang.
(cherry picked from commit f4b6267465)
2017-09-12 17:35:33 -07:00
Arun Suresh 783db4792c YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix.
(cherry picked from commit a1c398e0dc)
2017-09-12 17:04:25 -07:00
Arun Suresh 1230b60fa6 YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh)
(cherry picked from commit 2ae72692fc)
2017-09-12 16:38:58 -07:00
Arpit Agarwal e5a65ebc28 HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:22:19 -07:00
Jason Lowe 4e147f7e96 HADOOP-14843. Improve FsPermission symbolic parsing unit test coverage. Contributed by Bharat Viswanadham
(cherry picked from commit 86f4d1c66c)
2017-09-12 16:52:39 -05:00
Ray Chiang 99e7b26443 HADOOP-14856. Fix AWS, Jetty, HBase, Ehcache entries for NOTICE.txt. (rchiang)
(cherry picked from commit 2ffe93ab60)
2017-09-12 13:38:12 -07:00
Ray Chiang 5f6e387ba9 HADOOP-14797. Update re2j version to 1.1. (rchiang)
(cherry picked from commit 288c3b12d46fccdb9638d823b1b6b0cbc076ec16)
2017-09-12 11:35:15 -07:00
Andrew Wang 4e458e93f3 HDFS-12417. Disable flaky TestDFSStripedOutputStreamWithFailure.
(cherry picked from commit 80ee89b287)
2017-09-12 11:20:59 -07:00
Ray Chiang 619d61080c HADOOP-14653. Update joda-time version to 2.9.9. (rchiang)
(cherry picked from commit a6432ba5a1)
2017-09-12 11:13:21 -07:00
Ray Chiang 85db3b2aeb HADOOP-14648. Bump commons-configuration2 to 2.1.1. (rchiang)
(cherry picked from commit 39818259c3)
2017-09-12 10:54:10 -07:00
Ray Chiang fb5d12524c HADOOP-14796. Update json-simple version to 1.1.1. (rchiang)
(cherry picked from commit af45cd1be6)
2017-09-12 10:36:28 -07:00
Ray Chiang 656c6d1acc HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)
(cherry picked from commit 556812c179)
2017-09-12 10:19:58 -07:00
Ray Chiang f811306c86 HADOOP-14798. Update sshd-core and related mina-core library versions. (rchiang)
(cherry picked from commit 3876fe4e500efcb88fff93a86acf9626d961c8c7)
2017-09-12 10:00:25 -07:00
Wangda Tan d2e3818823 YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda)
Change-Id: I1c39ed5c59dee739ba5044b61b3ef5ed203b79c1
(cherry picked from commit e74d1be04b)
2017-09-12 08:09:47 -07:00
Haibo Chen 13310f91a5 YARN-7132. FairScheduler.initScheduler() contains a surprising unary plus. (Yeliang Cang via Haibo Chen)
(cherry picked from commit fa531788fd)
2017-09-11 20:49:33 -07:00
Haibo Chen 4f58533389 YARN-7128. The error message in TimelineSchemaCreator is not enough to find out the error. (Jinjiang Ling via Haibo Chen)
(cherry picked from commit 661f5eb0c6)
2017-09-11 16:22:06 -07:00
Ray Chiang 0b69a22f86 HADOOP-14655. Update httpcore version to 4.4.6. (rchiang)
(cherry picked from commit 088a18b44d)
2017-09-11 13:55:49 -07:00
Ray Chiang 14bec7043a HADOOP-14654. Update httpclient version to 4.5.3. (rchiang) 2017-09-11 13:43:48 -07:00
Andrew Wang cb95c442da Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah."
This reverts commit 0a6d5c0cf1.
2017-09-11 13:35:48 -07:00
Haibo Chen 1a2e8196d1 YARN-7181. CPUTimeTracker.updateElapsedJiffies can report negative usage. (Miklos Szegedi via Haibo Chen)
(cherry picked from commit d2d8f665e6)
2017-09-11 12:48:38 -07:00
Sunil G 07cbb81e8b YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S.
(cherry picked from commit 722ee84194)
2017-09-11 10:36:29 +05:30
Arpit Agarwal b4dc2fa247 HDFS-10391. Always enable NameNode service RPC port. Contributed by Gergely Novak. 2017-09-09 09:16:51 -07:00
Sunil G ad09c8f299 YARN-7104. Improve Nodes Heatmap in new YARN UI with better color coding. Contributed by Da Ding.
(cherry picked from commit ab8368d2e0)
2017-09-09 06:54:26 +05:30
Sunil G 6b54eec371 YARN-7022. Improve click interaction in queue topology in new YARN UI. Contributed by Abdullah Yousufi.
(cherry picked from commit ed162b7aa0)
2017-09-09 06:42:55 +05:30
Sunil G fd63f8112b YARN-7075. Better styling for donut charts in new YARN UI. Contributed by Da Ding.
(cherry picked from commit 275980bb1e)
2017-09-09 06:33:14 +05:30
Sunil G f3c9b2e40e YARN-6992. Kill application button is visible even if the application is FINISHED in RM UI. Contributed by Suma Shivaprasad.
(cherry picked from commit 6f101e7df1)
2017-09-09 06:25:53 +05:30
Haibo Chen 8ff2523687 HADOOP-14847. Remove Guava Supplier and change to java Supplier in AMRMClient and AMRMClientAysnc. (Bharat Viswanadham via Haibo Chen)
(cherry picked from commit 8edc60531f)
2017-09-08 14:35:03 -07:00
Robert Kanter f5f343a665 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter)
(cherry picked from commit a323f73bae)
2017-09-08 13:14:52 -07:00
John Zhuge bb107f5d28 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria.
(cherry picked from commit a4661850c1)
2017-09-08 12:57:58 -07:00
Rohith Sharma K S 720718eecc YARN-7130. ATSv2 documentation changes post merge. Contributed by Varun Saxena.
(cherry picked from commit 0bfb3a2566)
2017-09-08 22:42:02 +05:30
Rohith Sharma K S e8e1a73cf5 YARN-7140. CollectorInfo should have Public visibility. Contributed by Varun Saxena.
(cherry picked from commit 4a83170be4)
2017-09-08 22:41:52 +05:30
Arun Suresh abe3085f23 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh)
(cherry picked from commit 1f53ae7972)
2017-09-08 09:25:14 -07:00
Sunil G dfc174b747 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S.
(cherry picked from commit 56d93d2e39)
2017-09-08 20:06:06 +05:30
Steve Loughran 70c95d15e3
HADOOP-13421. Switch to v2 of the S3 List Objects API in S3A.
Contributed by Aaron Fabbri

(cherry picked from commit 5bbca80428)
2017-09-08 12:08:36 +01:00
Xiao Chen 3a3762d9e7 HDFS-12400. Provide a way for NN to drain the local key cache before re-encryption.
(cherry picked from commit 420baed31fdd811ac94c74dc53502ee79f37ba91)
2017-09-07 20:51:09 -07:00
Andrew Wang 986e8fc553 HDFS-12218. Addendum. Rename split EC / replicated block metrics in BlockManager.
(cherry picked from commit 4e50dc976a)
2017-09-07 16:57:37 -07:00
Andrew Wang 4c1f8f3b26 HDFS-12218. Rename split EC / replicated block metrics in BlockManager.
(cherry picked from commit 40c2f31f8d)
2017-09-07 16:57:35 -07:00
Xiao Chen 9a3334ec0f HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots.
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
2017-09-07 16:30:26 -07:00
Jason Lowe 7287932397 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf
(cherry picked from commit b0b535d9d5)
2017-09-07 16:25:41 -05:00
Steve Loughran 5757600b8d
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter.
Contributed by Yonger.

(cherry picked from commit 83449abb40)
2017-09-07 19:58:03 +01:00
Arun Suresh 05b4abf537 YARN-6978. Add updateContainer API to NMClient. (Kartheek Muthyala via asuresh)
(cherry picked from commit c41118a7f8)
2017-09-07 10:28:18 -07:00
Yongjun Zhang 5c9ffa1cf9 HDFS-12357. Let NameNode to bypass external attribute provider for configured users. Contributed by Yongjun Zhang, Arun Suresh.
(cherry picked from commit d77ed238a9)
2017-09-07 10:48:18 -07:00
Steve Loughran e5bb78d3d9
HADOOP-14520. WASB: Block compaction for Azure Block Blobs.
Contributed by Georgi Chalakov

(cherry picked from commit 13eda50003)
2017-09-07 18:38:24 +01:00
Kai Zheng 7415a21ee2 HDFS-12402. Refactor ErasureCodingPolicyManager and related codes. Contributed by Sammi Chen 2017-09-07 20:37:11 +08:00