Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,670 Commits 326 Branches 371 Tags
Commit Graph

13670 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
2a5a313539
MAPREDUCE-6697. Concurrent task limits should only be applied when necessary. Contributed by Nathan Roberts. 
(cherry picked from commit a5c0476a990ec1e7eb34ce2462a45aa52cc1350d)
 2017-06-28 10:51:23 +09:00
Mingliang Liu
ce57458ec5 HADOOP-14546. Azure: Concurrent I/O does not work when secure.mode is enabled. Contributed by Thomas 
(cherry picked from commit 7e031c2c18b8812ec9f843ed3b4abe9e6d12bb28)
 2017-06-27 17:32:56 -07:00
Mingliang Liu
1b1065a163 HADOOP-14573. regression: Azure tests which capture logs failing with move to SLF4J. Contributed by Steve Loughran 
(cherry picked from commit 686a634f01e454537d1a9d531330e60a60233ca4)
 2017-06-27 16:49:48 -07:00
Jason Lowe
44b3c6ffaa YARN-6738. LevelDBCacheTimelineStore should reuse ObjectMapper instances. Contributed by Zoltan Haindrich 
(cherry picked from commit 63ce1593c5b78eb172773e7498d9c321debe81e8)
 2017-06-27 17:13:49 -05:00
Mingliang Liu
ea402c364f HADOOP-14594. ITestS3AFileOperationCost::testFakeDirectoryDeletion to uncomment metric assertions. Contributed by Mingliang Liu 
(cherry picked from commit bc4dfe9c9cb31f39e8f1532d5c5837faf2e92bde)
 2017-06-27 13:20:03 -07:00
Steve Loughran
15d88b77d1
HADOOP-14536. Update azure-storage sdk to version 5.3.0 
Contributed by Georgi Chalakov
 2017-06-27 15:23:06 +01:00
Akira Ajisaka
a896c0f11e
HDFS-12040. TestFsDatasetImpl.testCleanShutdownOfVolume fails. Contributed by hu xiaodong. 
(cherry picked from commit 07defa4c09db6e8c552167019ca0d2444cfc8fe7)
 2017-06-27 20:49:56 +09:00
Steve Loughran
3a87591df8 HADOOP-14461 Azure: handle failure gracefully in case of missing account access key. Contributed by Mingliang Liu. 
(cherry picked from commit 48f4a229a93b65df9e99df2505719dd3538f1188)
 2017-06-26 15:22:44 -07:00
Ravi Prakash
ef491f4637 HDFS-11993. Add log info when connect to datanode socket address failed. Contributed by chencan 
(cherry picked from commit a9d3412b4ce40f5ab5a18756ede7e0606b653171)
 2017-06-26 13:25:32 -07:00
Akira Ajisaka
b59b96deee
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong. 2017-06-27 03:28:09 +09:00
Akira Ajisaka
239b382153
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he. 
(cherry picked from commit a594f96fb1155d94fe08ed8f85dca789a9a4c790)
 2017-06-26 16:41:44 +09:00
Naganarasimha
92e266c2b6 YARN-5006. ResourceManager quit due to ApplicationStateData exceed the limit size of znode in zk. Contributed by Bibin A Chundatt. 2017-06-24 23:14:51 +05:30
Arpit Agarwal
4b420e0678 HADOOP-14543. ZKFC should use getAversion() while setting the zkacl. Contributed by Brahma Reddy Battula. 2017-06-23 16:50:56 -07:00
Steve Loughran
093eb0d361
HADOOP-14547. [WASB] the configured retry policy is not used for all storage operations. 
Contributed by Thomas.

(cherry picked from commit c6bd73c6c5760c3a52203e6a51628587ceec6896)
 2017-06-23 21:40:05 +01:00
Yufei Gu
4c59b446e2 YARN-5876. TestResourceTrackerService#testGracefulDecommissionWithApp fails intermittently on trunk. (Robert Kanter via Yufei Gu) 
(cherry picked from commit 0b77262890d76b0a3a35fa64befc8a406bc70b27)
 2017-06-23 13:30:20 -07:00
Kihwal Lee
0719345909 Addendum for HADOOP-14146. Use commons Base64 instead of jdk8 Base64. 2017-06-23 13:26:29 -05:00
Kihwal Lee
4c6184b41c Addendum for HADOOP-14146 jdk7 support. 2017-06-23 09:57:51 -05:00
Steve Loughran
5858bc70a2
HADOOP-14568. GenericTestUtils#waitFor missing parameter verification. 
Contributed by Yiqun Lin

(cherry picked from commit e5db9af8a00ded73ee1c75bb4c0d5d50e15a305c)
 2017-06-23 14:41:08 +01:00
Arpit Agarwal
f9f6ef83eb HDFS-12010. TestCopyPreserveFlag fails consistently because of mismatch in access time. Contributed by Mukul Kumar Singh. 2017-06-22 15:43:00 -07:00
Arun Suresh
66f780e6ab YARN-6127. Add support for work preserving NM restart when AMRMProxy is enabled. (Botong Huang via asuresh). 2017-06-22 15:07:38 -07:00
Akira Ajisaka
897d5a4d24
HADOOP-14542. Add IOUtils.cleanupWithLogger that accepts slf4j logger API. Contributed by Chen Liang. 
(cherry picked from commit b64951905e64f6fed581c28634be6ed15c190633)
 2017-06-22 17:43:33 +09:00
Daryn Sharp
d4115d71b5 HADOOP-14146. KerberosAuthenticationHandler should authenticate with SPN in AP-REQ. Contributed by Daryn Sharp 2017-06-21 11:05:12 -05:00
Yongjun Zhang
ba487ead42 HADOOP-14429. FTPFileSystem#getFsAction always returns FsAction.NONE. (Hongyuan Li via Yongjun Zhang) 
(cherry picked from commit 5157f6c46ec342fb650b3c5853061ed1e4a182b6)
 2017-06-20 17:07:01 -07:00
Kihwal Lee
8f533ca76b HDFS-11960. Successfully closed files can stay under-replicated. Contributed by Kihwal Lee. 2017-06-20 09:16:37 -05:00
Brahma Reddy Battula
933fad2e17 HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han. 
(cherry picked from commit 2e9daa2e27536484846d9c5f020c6afaf27932ce)
 2017-06-20 20:45:33 +08:00
Akira Ajisaka
35f6c7c9c7
YARN-6713. Fix dead link in the Javadoc of FairSchedulerEventLog.java. Contributed by Weiwei Yang. 
(cherry picked from commit 099dfe92a9743bcb7e27506f8877b8b23b0ff95a)
 2017-06-20 17:23:06 +09:00
Akira Ajisaka
6b3c13d543
HDFS-11345. Document the configuration key for FSNamesystem lock fairness. Contributed by Erik Krogen. 
(cherry picked from commit 2c769167dbdb66c52d2ba7b7193a686444085570)
 2017-06-20 16:36:47 +09:00
Xiao Chen
b9bd72c676 HADOOP-14515. Specifically configure zookeeper-related log levels in KMS log4j. 
(cherry picked from commit 4cdae5d88ef7851ae284a115868dc4bbe24ae1bd)
 2017-06-19 23:12:28 -07:00
Akira Ajisaka
71626fd0b9
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 2017-06-20 13:55:42 +09:00
Akira Ajisaka
8a77cf1ee4
HADOOP-14296. Move logging APIs over to slf4j in hadoop-tools. 2017-06-20 13:21:12 +09:00
Masatake Iwasaki
b19ca498ae HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 
(cherry picked from commit d954a64730c00346476322743462cde857164177)
 2017-06-20 08:09:09 +09:00
Naganarasimha
578318473c YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp 
(cherry picked from commit ee89ac84e68d3e181b75c63f74a0444f9d28146f)
 2017-06-19 22:45:21 +05:30
Andrew Wang
1ec1aebd35 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-19 09:52:07 -07:00
Akira Ajisaka
6d82db40d5
HADOOP-14540. Replace MRv1 specific terms in HostsFileReader. Contributed by hu xiaodong. 
(cherry picked from commit 300804595e5b8fd447ecb530fcd0ae5b0e59bb63)
 2017-06-19 19:40:19 +09:00
Akira Ajisaka
2d0b6ab65d
HADOOP-14538. Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 7ade5124b8b6c52a084ec187c531017eee0f1884)
 2017-06-19 19:10:13 +09:00
Naganarasimha
ac058c75a4 YARN-6719. Fix findbugs warnings in SLSCapacityScheduler.java. Contributed by Akira Ajisaka. 2017-06-19 12:41:13 +05:30
Masatake Iwasaki
023dcdc73d HADOOP-14424. Add CRC32C performance test. Contributed by LiXin Ge. 
(cherry picked from commit 75043d342935828cfe8bdabfab8784c83f797887)
 2017-06-19 09:18:02 +09:00
Naganarasimha
3106f256e2 YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang. 
(cherry picked from commit 7582dedad1c73eabdc3eeece0a3a860e7bb33c1a)
 2017-06-18 19:56:11 +05:30
Lei Xu
33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a09a7fcc29eceb8dc3859d6298da6882)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal
9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Ravi Prakash
182e98c703 MAPREDUCE-6897. Add Unit Test to ensure Job end notification gets sent even when appMaster stop gets YarnRuntimeException. Contributed by Gergely Novák 
(cherry picked from commit 82bbcbf37f8137112a6270932b2ad7572785c387)
 2017-06-16 14:27:06 -07:00
Jason Lowe
43fb793174 Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit 576cc258f9ac1bccb13c01f147b360c526188467.
 2017-06-16 14:49:26 -05:00
Jason Lowe
576cc258f9 MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4bea90e8545e9e06af9c07e89d2be43d9)
 2017-06-16 14:26:15 -05:00
Xiao Chen
2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Steve Loughran
4e6348f346
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0. 
Contributed by Hongyuan Li.

(cherry picked from commit 942ecbbc9876427051965ddd167df5cc1cc253e3)
 2017-06-16 09:46:34 +01:00
Akira Ajisaka
12cb3456d7
HADOOP-14289. Move log4j APIs over to slf4j in hadoop-common. 
(cherry picked from commit e855cc4ed467d03f4f03e8b3a1ff27654b349a52)
 2017-06-16 13:32:16 +09:00
Lei Xu
abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Xiao Chen
7d81b0beab HADOOP-14523. OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory. Contributed by Misha Dmitriev. 
(cherry picked from commit ef8edab930338646551cbe3c7e7cf954e21c0f9a)
 2017-06-15 10:41:34 -07:00
Akira Ajisaka
0fd0612896
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin." 
This reverts commit 88fa17c2383baafbfb6b43b69e08cf40493c774a.
 2017-06-16 00:21:54 +09:00
Steve Loughran
31446f8e10
HADOOP-14506. Add create() contract test that verifies ancestor dir creation. 
Contributed by Sean Mackrory.

(cherry picked from commit d780a67864d2c687d23b69798c19ee865e38cc5a)
 2017-06-15 14:51:50 +01:00