Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,839 Commits 326 Branches 371 Tags
Commit Graph

17839 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
6895550043 HDFS-14095. EC: Track Erasure Coding commands in DFS statistics. Contributed by Ayush Saxena. 
(cherry picked from commit f534736867eed962899615ca1b7eb68bcf591d17)
 2018-11-30 00:21:07 +05:30
Steve Loughran
065a1e72f2
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple" 
This reverts commit 829a2e4d271f05afb209ddc834cd4a0e85492eda.

(cherry picked from commit d0edd37269bb40290b409d583bcf3b70897c13e0)
 2018-11-29 17:58:15 +00:00
bibinchundatt
b42c2679ad YARN-9069. Fix SchedulerInfo#getSchedulerType for custom schedulers. Contributed by Bilwa S T. 
(cherry picked from commit 07142f54a8c7f70857e99c041f3a2a5189c809b5)
 2018-11-29 22:08:48 +05:30
Jason Lowe
4c238b50df MAPREDUCE-7164. FileOutputCommitter does not report progress while merging paths. Contributed by Kuhu Shukla 
(cherry picked from commit 4d8de7ab690ef919b392b12d856482a6a1f2bb3d)
 2018-11-28 16:10:02 -06:00
Jason Lowe
e4dcc3e606 YARN-8812. Containers fail during creating a symlink which started with hyphen for a resource file. Contributed by Oleksandr Shevchenko 
(cherry picked from commit 3ce99e32f7d7887412cae8337cd4ebeb3b2ee308)
 2018-11-28 08:56:04 -06:00
Steve Loughran
7dac294110
HADOOP-15932. Oozie unable to create sharelib in s3a filesystem. 
Contributed by Steve Loughran.

(cherry picked from commit 4c106fca0ca91536e288f11052568406a0b84300)
 2018-11-27 20:57:16 +00:00
Vinayakumar B
8569555fd1 HDFS-13816. dfs.getQuotaUsage() throws NPE on non-existent dir instead of FileNotFoundException. Contributed by Vinayakumar B. 
(cherry picked from commit b09828145432c8d986ac8f05ec33608d8e611328)
 2018-11-26 18:23:20 +05:30
Vinayakumar B
c2e65cac5e HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena. 
(cherry picked from commit 23b441c2253bcb3be5229d3c5eb1e165369070c0)
 2018-11-26 18:14:30 +05:30
Weiwei Yang
8d930ae344 HADOOP-15943. AliyunOSS: add missing owner & group attributes for oss FileStatus. Contributed by wujinhu. 
(cherry picked from commit 5ff0cf86a940fd83f1425794921cc075b19f1108)
 2018-11-23 14:39:23 +08:00
Weiwei Yang
8a67f63664 HADOOP-15919. AliyunOSS: Enable Yarn to use OSS. Contributed by wujinhu. 
(cherry picked from commit be0708c6eb3c759e43639e0f8df7c9910308fb97)
 2018-11-19 14:50:06 +08:00
Weiwei Yang
067e39f781 YARN-8833. Avoid potential integer overflow when computing fair shares. Contributed by liyakun. 
(cherry picked from commit d027a24f0349b60efa5125c330058f123771748f)
 2018-11-18 23:25:31 +08:00
Vinayakumar B
a19c72d7e8 HDFS-14056. Fix error messages in HDFS-12716. Contributed by Ayush Saxena. 
(cherry picked from commit 886b935630bf2d46f4c6773de013c093d2e44c38)
 2018-11-16 18:02:01 +05:30
Vinayakumar B
1b6fa47334 HDFS-13963. NN UI is broken with IE11. Contributed by Ayush Saxena. 
(cherry picked from commit 8571507efa3af3e4a5213a7e24c65e26162eac4d)
 2018-11-16 17:36:50 +05:30
Akira Ajisaka
6dc6978eeb
HADOOP-15925. The config and log of gpg-agent are removed in create-release script. Contributed by Dinesh Chitlangia. 
(cherry picked from commit af5958f7b6a209d6a6622acee47a66a4a921828d)
 2018-11-16 13:18:32 +09:00
Akira Ajisaka
2db612fff9
HADOOP-15930. Exclude MD5 checksum files from release artifact. 
(cherry picked from commit df5e863fee544c9283e28a21c2788c008d7e3e04)
 2018-11-15 10:47:53 +09:00
Sammi Chen
64cb97fb44 HADOOP-15917. AliyunOSS: fix incorrect ReadOps and WriteOps in statistics. Contributed by Jinhu Wu. 
(cherry picked from commit 3fade865ce84dcf68bcd7de5a5ed1c7d904796e9)
 2018-11-14 13:09:11 +08:00
Brahma Reddy Battula
04bba91585 HDFS-14070. Refactor NameNodeWebHdfsMethods to allow better extensibility. Contributed by CR Hota 
(cherry picked from commit e7b63baca1e10b28d8b4462fd80537b871951aa3)
 2018-11-13 13:12:56 +05:30
Akira Ajisaka
ff1cbb48c8
HADOOP-15923. create-release script should set max-cache-ttl as well as default-cache-ttl for gpg-agent. 
(cherry picked from commit 703b2860a49577629e7b3ef461d8a61292e79c88)
 2018-11-13 13:42:19 +09:00
Arpit Agarwal
f514c08900 HDFS-14065. Failed Storage Locations shows nothing in the Datanode Volume Failures. Contributed by Ayush Saxena. 
(cherry picked from commit b6d4e19f34f474ea8068ebb374f55e0db2f714da)
 2018-11-12 15:31:56 -08:00
Akira Ajisaka
50ce9fd4e2
YARN-8233. NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null. Contributed by Tao Yang. 2018-11-10 18:02:57 +09:00
Akira Ajisaka
ca331c8057
HADOOP-15916. Upgrade Maven Surefire plugin to 3.0.0-M1. 
(cherry picked from commit a736b5da15084e8eb93d2f68f8eccc506ff7bea7)
(cherry picked from commit 7b62c9684d517394d1d360eb8d6dee36ff659062)
 2018-11-10 00:28:46 +09:00
Kitti Nanasi
bbc94de477 HDFS-14039. ec -listPolicies doesn't show correct state for the default policy when the default is not RS(6,3). Contributed by Kitti Nanasi. 
Signed-off-by: Xiao Chen <xiao@apache.org>
(cherry picked from commit 8d99648c203004045a9339ad27258092969145d6)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ErasureCodingPolicyManager.java

(cherry picked from commit 01b8197893c84fa24bb94945f85b3e2ebe8259a5)
 2018-11-08 10:22:32 -08:00
Vinayakumar B
76be3515bf HDFS-13772. Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled. Contributed by Ayush Saxena 
(cherry picked from commit 8df2eb8119188b8e5515295523afc23046e1db81)
 2018-11-08 10:22:32 -08:00
Weiwei Yang
be2a0a5e23 YARN-8977. Remove unnecessary type casting when calling AbstractYarnScheduler#getSchedulerNode. Contributed by Wanqiang Ji. 
(cherry picked from commit c96cbe8659587cfc114a96aab1be5cc85029fe44)
 2018-11-07 22:54:34 +08:00
Takanobu Asanuma
eb365039da YARN-8866. Fix a parsing error for crossdomain.xml. 
(cherry picked from commit 8dc1f6dbf712a65390a9a6859f62fec0481af31b)
 2018-11-07 18:28:15 +09:00
Jason Lowe
e830d73adb MAPREDUCE-7156. NullPointerException when reaching max shuffle connections. Contributed by Peter Bacsko 
(cherry picked from commit ba1f9d66d94ed0b85084d7c40c09a87478b3a05a)
 2018-11-06 18:11:39 -06:00
Inigo Goiri
c4d390a775 HDFS-14048. DFSOutputStream close() throws exception on subsequent call after DataNode restart. Contributed by Erik Krogen. 
(cherry picked from commit 887244de4adebe27693ed4ad3296a6f700cfa8c1)
 2018-11-06 11:21:19 -08:00
Jason Lowe
f3e6682ba7 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit ab6aa4c7265db5bcbb446c2f779289023d454b81)
 2018-11-06 08:59:47 -06:00
Weiwei Yang
ddb349ceb3 YARN-8970. Improve the debug message in CS#allocateContainerOnSingleNode. Contributed by Zhankun Tang. 
(cherry picked from commit 5d6554c722f08f79bce904e021243605ee75bae3)
 2018-11-06 14:54:33 +08:00
Weiwei Yang
c3a0f07db0 YARN-8969. AbstractYarnScheduler#getNodeTracker should return generic type to avoid type casting. Contributed by Wanqiang Ji. 
(cherry picked from commit c7fcca0d7ec9e31d43ef3040ecd576ec808f1f8b)
 2018-11-06 13:43:24 +08:00
Inigo Goiri
4b47daa640 HDFS-14043. Tolerate corrupted seen_txid file. Contributed by Lukas Majercak. 
(cherry picked from commit f3296501e09fa7f1e81548dfcefa56f20fe337ca)
(cherry picked from commit 9bf4f3d61403b06f3d6a092dacab382c7c131e19)
 2018-11-05 16:53:28 -08:00
Akira Ajisaka
aee665f2db
HADOOP-15900. Update JSch versions in LICENSE.txt. 
(cherry picked from commit d43cc5db0ff80958ca873df1dc2fa00054e37175)
 2018-11-05 17:52:36 +09:00
Akira Ajisaka
869b1cee67
HADOOP-15899. Update AWS Java SDK versions in NOTICE.txt. 
(cherry picked from commit 73d8f8761e969ed6054cec0f58ae86d13232e516)
 2018-11-05 15:32:23 +09:00
Jonathan Hung
68ec0fa0a8 YARN-7225. Add queue and partition info to RM audit log. Contributed by Eric Payne 
(cherry picked from commit 2ab611d48b7669b31bd2c9b918f47251da77d0f6)
 2018-11-01 14:31:57 -07:00
Akira Ajisaka
2ae21d5387
Addendum: Fix potential FSImage corruption. Contributed by Xiao Chen. 
(cherry picked from commit dd70b1f53836961029767660398819a3b04eec58)
 2018-10-31 17:03:47 +09:00
Xiao Chen
20c9a12bc7 HDFS-14027. DFSStripedOutputStream should implement both hsync methods. 
(cherry picked from commit db7e636824a36b90ba1c8e9b2fba1162771700fe)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSStripedOutputStream.java

(cherry picked from commit 399645ebc162371a63eb81840e01ed52261f43bc)
 2018-10-29 19:14:12 -07:00
Wei-Chiu Chuang
0f34ff7722 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao." 
This reverts commit fa01a8fa0e345c533e843504743baa00020be706.
 2018-10-28 17:07:29 -07:00
Anu Engineer
c5b704a34c HDFS-14028. HDFS OIV temporary dir deletes folder. 
Contributed by Adam Antal.

(cherry picked from commit 4f10d7e23fc0b757a9e94bc448187a6211b90f10)
 2018-10-26 16:55:55 -07:00
Sunil G
d91cb0a5e7 HADOOP-15815. Upgrade Eclipse Jetty version to 9.3.24. Cotributed by Boris Vulikh. 
(cherry picked from commit e28c00c290a06813d156a845b93e806f71413dbf)
 2018-10-26 13:48:23 +05:30
Sammi Chen
c5a227062f HADOOP-15868. AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy. Contributed by Jinhu Wu. 
(cherry picked from commit 7574d18538e838f40581519080d7c8621c65e53b)
(cherry picked from commit 366541d834f70fd6f8d4c5296a9e844236c6fd74)
 2018-10-26 15:30:06 +08:00
Arpit Agarwal
f6469adbb1 HDFS-14029. Sleep in TestLazyPersistFiles should be put into a loop. Contributed by Adam Antal. 
(cherry picked from commit 9b899f1ebd5126a756fceb43459164165488b203)
 2018-10-25 15:14:39 -07:00
Bharat Viswanadham
96bdaf3b4a HADOOP-15882. Upgrade maven-shade-plugin from 2.4.3 to 3.2.0. Contributed by Takanobu Asanuma. 
(cherry picked from commit 34b2521f5e7388e6a4d76af537ef147206fd9b72)
 2018-10-25 13:42:50 -07:00
Wei-Chiu Chuang
fa01a8fa0e HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 
(cherry picked from commit fb2b72e6fce019130e10964a644b94cddbab1c06)
(cherry picked from commit fd02c501c0cea3ec55956e11b390111519cedc00)
(cherry picked from commit ef9f8ca13d8dca8031c045bad801fceb3e00aa15)
 2018-10-25 09:38:26 -07:00
Ajay Kumar
2caf69debd HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Wei-Chiu Chuang. 2018-10-24 22:35:06 -07:00
Inigo Goiri
1d5390679e HDFS-14003. Fix findbugs warning in trunk for FSImageFormatPBINode. Contributed by Yiqun Lin. 
(cherry picked from commit 0e56c883cd2310f3ff9d62afb306b1ab27419c36)
 2018-10-24 16:14:22 -07:00
Xiao Chen
e8832418bf Fix potential FSImage corruption. Contributed by Daryn Sharp and Vinayakumar B. 
(cherry picked from commit f1996ccbaee734d423caa9d47a571cfff98ef42c)
(cherry picked from commit 96cedb87b94c07c11152580bf36978186d622b50)
 2018-10-24 16:10:51 -07:00
Inigo Goiri
6aef514795 HDFS-14021. TestReconstructStripedBlocksWithRackAwareness#testReconstructForNotEnoughRacks fails intermittently. Contributed by Xiao Chen. 
(cherry picked from commit c1874046e2f9275ba330090fcf12c5611c6d6fc4)
 2018-10-24 09:38:48 -07:00
Akira Ajisaka
e402791a51
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cdfccefba5457ae3d561e9ce0109da3f)

Conflicts:
	dev-support/docker/Dockerfile
 2018-10-24 13:56:19 +09:00
Jason Lowe
a9bc3e3708 YARN-8904. TestRMDelegationTokens can fail in testRMDTMasterKeyStateOnRollingMasterKey. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 93fb3b4b9cbc63e65d590676a5a318d555a25904)
 2018-10-23 12:57:49 -05:00
Wei-Chiu Chuang
e4464f9598 Revert "HDFS-13941. make storageId in BlockPoolTokenSecretManager.checkAccess optional. Contributed by Ajay Kumar." 
This reverts commit d8cd07d5f5cc321d2f6c625a902f78e0c43433a5.
 2018-10-23 05:58:10 -07:00