Commit Graph

13736 Commits

Author SHA1 Message Date
Arpit Agarwal d8d2a221f8 HDFS-11450. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang. 2017-08-18 11:43:17 -07:00
Arpit Agarwal 850593133b Revert "HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru."
This reverts commit bdc425057e.
2017-08-18 10:16:13 -07:00
bibinchundatt ecf1a029bb YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su.
(cherry picked from commit e05fa3451d)
2017-08-18 20:31:26 +05:30
Steve Loughran c42e694ee5
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted.
Contributed by Thomas Marquardt

(cherry picked from commit c6b4e656b7)
2017-08-18 14:38:04 +01:00
Manoj Govindassamy a5bc30170c HDFS-12316. Verify HDFS snapshot deletion doesn't crash the ongoing file writes.
(cherry picked from commit 4230872dd6)
2017-08-17 16:25:15 -07:00
Sunil G 5511cea86b YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad.
(cherry picked from commit f9a0e23381)
2017-08-17 15:10:07 +05:30
Naganarasimha fa3094e436 YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung.
(cherry picked from commit 0a3aa40fe7)
2017-08-17 10:42:00 +05:30
Robert Kanter 9de332d502 MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter)
(cherry picked from commit ab051bd42e)
2017-08-16 16:15:48 -07:00
Mingliang Liu 03d15fa58e HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze 2017-08-16 16:06:13 -07:00
Jason Lowe 107e1722d4 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:35:41 -05:00
Jason Lowe cf30380d46 YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter
(cherry picked from commit 14553061be)
2017-08-16 13:06:13 -05:00
Steve Loughran d5680c08ed
HADOOP-14660. wasb: improve throughput by 34% when account limit exceeded.
Contributed by Thomas Marquardt.
2017-08-16 13:49:47 +01:00
bibinchundatt 67f9968407 YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R. 2017-08-16 14:52:09 +05:30
Akira Ajisaka a28a3dc8f2
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka.
(cherry picked from commit 588c190afd)
2017-08-16 14:07:08 +09:00
Subru Krishnan ad46a90872 HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru).
(cherry picked from commit 75dd866bfb)
2017-08-15 16:54:48 -07:00
Ravi Prakash 895a35eac0 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed
(cherry picked from commit f34646d652)
2017-08-15 15:45:37 -07:00
Steve Loughran b30522c59e
HADOOP-14662. Update azure-storage sdk to version 5.4.0.
Contributed by Thomas Marquard
2017-08-15 16:27:31 +01:00
Arun Suresh 7b22df3da6 YARN-5978. ContainerScheduler and ContainerManager changes to support ExecType update. (Kartheek Muthyala via asuresh)
(cherry picked from commit 4d7be1d857)
2017-08-14 19:57:34 -07:00
Arpit Agarwal bdc425057e HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru. 2017-08-14 15:53:43 -07:00
Jason Lowe 56d0e757c7 YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne
(cherry picked from commit 5558792894)
2017-08-14 15:34:50 -05:00
Jason Lowe e5ba40cc98 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles
(cherry picked from commit 3325ef653d)
2017-08-14 14:41:38 -05:00
Subru Krishnan a70efb6138 HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru). 2017-08-14 11:03:50 -07:00
Carlo Curino 394573780b YARN-6687. Validate that the duration of the periodic reservation is less than the periodicity. (subru via curino)
(cherry picked from commit 28d97b79b6)
2017-08-14 11:01:56 -07:00
Jian He 630ada1b4e YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-14 10:56:39 -07:00
John Zhuge 0efc590e6d HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria.
(cherry picked from commit 7769e96149)
2017-08-13 00:27:33 -07:00
John Zhuge ed4d6aa2c1 HDFS-11303. Hedged read might hang infinitely if read data from all DN failed . Contributed by Chen Zhang, Wei-chiu Chuang, and John Zhuge.
(cherry picked from commit 8b242f09a6)
2017-08-11 21:55:23 -07:00
Daniel Templeton 7d3cea232d Revert "YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator"
This reverts commit ce0cdc50d9.
2017-08-11 20:52:07 -07:00
Daniel Templeton ce0cdc50d9 YARN-6882. AllocationFileLoaderService.reloadAllocations() should use the diamond operator
(Contributed by Larry Lo via Daniel Templeton)

(cherry picked from commit 0996acde6c)
2017-08-11 14:51:42 -07:00
Daniel Templeton 11e3673748 YARN-6967. Limit application attempt's diagnostic message size thoroughly
(Contributed by Chengbing Liu via Daniel Templeton)

(cherry picked from commit 65364defb4)
2017-08-11 14:31:11 -07:00
Daniel Templeton 2566e5fcf7 YARN-6884. AllocationFileLoaderService.loadQueue() has an if without braces
(Contributed by weiyuan via Daniel Templeton)

(cherry picked from commit c7680d4cc4)
2017-08-11 14:23:17 -07:00
Karthik Kambatla f6c74ac5ea YARN-5927. BaseContainerManagerTest::waitForNMContainerState timeout accounting is not accurate. (Kai Sasaki via kasha)
(cherry picked from commit 8c4b6d16a5)
2017-08-11 12:18:29 -07:00
John Zhuge e208a4e7ee HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge.
(cherry picked from commit 582648befa)
2017-08-11 10:54:41 -07:00
John Zhuge 76c4aee2e3 HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang
(cherry picked from commit 00160f71b6)
2017-08-11 10:54:30 -07:00
John Zhuge 9ba3268fa3 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang
(cherry picked from commit 59d59667a8)
2017-08-11 10:54:25 -07:00
Steve Loughran 9c6e8ece98
HADOOP-14754. TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes.
Contributed by John Zhuge.

(cherry picked from commit d964062f66)
2017-08-11 10:58:26 +01:00
Wei-Chiu Chuang 304cc1a856 HADOOP-14743. CompositeGroupsMapping should not swallow exceptions. Contributed by Wei-Chiu Chuang.
(cherry picked from commit a8b75466b2)
2017-08-10 09:36:21 -07:00
Steve Loughran 96befde480
HADOOP-14183. Remove service loader config file for wasb fs.
Contributed by Esfandiar Manii.

(cherry picked from commit 54356b1e83)
2017-08-10 16:49:51 +01:00
Steve Loughran bbf8c4bacb
HADOOP-14715. TestWasbRemoteCallHelper failing.
Contributed by Esfandiar Manii.
2017-08-10 16:49:19 +01:00
Akira Ajisaka 1f80e05eb4
YARN-6958. Moving logging APIs over to slf4j in hadoop-yarn-server-timelineservice. Contributed by Yeliang Cang. 2017-08-10 22:14:44 +09:00
Ravi Prakash c556cdaaf4 MAPREDUCE-6923. Optimize MapReduce Shuffle I/O for small partitions. Contributed by Robert Schmidtke.
(cherry picked from commit ac7d0604bc)
2017-08-09 15:40:57 -07:00
Kihwal Lee 306abf4980 HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah.
(cherry picked from commit b5c02f95b5)
2017-08-09 16:48:55 -05:00
Jian He 8b7590057e YARN-6804. Allow custom hostname for docker containers in native services. Contributed by Billie Rinaldi 2017-08-09 11:12:38 -07:00
Kihwal Lee 74bcc8d1f2 HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 2017-08-09 09:31:53 -05:00
Haohui Mai 3e2753daa1 HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran. 2017-08-08 16:33:54 -07:00
Haohui Mai 21d76f710e HDFS-10326. Disable setting tcp socket send/receive buffers for write pipelines. Contributed by Daryn Sharp. 2017-08-08 14:58:44 -07:00
Subru Krishnan e9ec1e8f87 YARN-6897. Refactoring RMWebServices by moving some util methods to RMWebAppUtil. (Giovanni Matteo Fumarola via Subru). 2017-08-08 14:01:22 -07:00
Daniel Templeton c5b3d80a4b YARN-6879. TestLeafQueue.testDRFUserLimits() has commented out code
(Contributed by Angela Wang via Daniel Templeton)

(cherry picked from commit e0c24145d2)
2017-08-08 13:36:49 -07:00
Jason Lowe 50dfef7275 MAPREDUCE-6927. MR job should only set tracking url if history was successfully written. Contributed by Eric Badger
(cherry picked from commit 735fce5bec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
2017-08-08 14:55:55 -05:00
Junping Du e9c21d14b4 YARN-6811. [ATS1.5] All history logs should be kept under its own User Directory. Contributed by Rohith Sharma K S. 2017-08-08 12:45:55 -07:00
Daniel Templeton 675d368950 YARN-6757. Refactor the usage of yarn.nodemanager.linux-container-executor.cgroups.mount-path
(Contributed by Miklos Szegedi via Daniel Templeton)

(cherry picked from commit 47b145b9b4)
2017-08-08 11:12:34 -07:00