Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,240 Commits 326 Branches 371 Tags
Commit Graph

13240 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Masatake Iwasaki
b19ca498ae HDFS-11995. HDFS Architecture documentation incorrectly describes writing to a local temporary file. Contributed by Nandakumar. 
(cherry picked from commit d954a64730c00346476322743462cde857164177)
 2017-06-20 08:09:09 +09:00
Naganarasimha
578318473c YARN-6680. Avoid locking overhead for NO_LABEL lookups. Contributed by Daryn Sharp 
(cherry picked from commit ee89ac84e68d3e181b75c63f74a0444f9d28146f)
 2017-06-19 22:45:21 +05:30
Andrew Wang
1ec1aebd35 HDFS-10480. Add an admin command to list currently open files. Contributed by Manoj Govindassamy. 2017-06-19 09:52:07 -07:00
Akira Ajisaka
6d82db40d5
HADOOP-14540. Replace MRv1 specific terms in HostsFileReader. Contributed by hu xiaodong. 
(cherry picked from commit 300804595e5b8fd447ecb530fcd0ae5b0e59bb63)
 2017-06-19 19:40:19 +09:00
Akira Ajisaka
2d0b6ab65d
HADOOP-14538. Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 7ade5124b8b6c52a084ec187c531017eee0f1884)
 2017-06-19 19:10:13 +09:00
Naganarasimha
ac058c75a4 YARN-6719. Fix findbugs warnings in SLSCapacityScheduler.java. Contributed by Akira Ajisaka. 2017-06-19 12:41:13 +05:30
Masatake Iwasaki
023dcdc73d HADOOP-14424. Add CRC32C performance test. Contributed by LiXin Ge. 
(cherry picked from commit 75043d342935828cfe8bdabfab8784c83f797887)
 2017-06-19 09:18:02 +09:00
Naganarasimha
3106f256e2 YARN-6517. Fix warnings from Spotbugs in hadoop-yarn-common(addendum). Contributed by Weiwei Yang. 
(cherry picked from commit 7582dedad1c73eabdc3eeece0a3a860e7bb33c1a)
 2017-06-18 19:56:11 +05:30
Lei Xu
33afa1fdca HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 6460df21a09a7fcc29eceb8dc3859d6298da6882)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
 2017-06-16 17:32:29 -07:00
Arpit Agarwal
9529513f18 HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru. 2017-06-16 16:41:08 -07:00
Ravi Prakash
182e98c703 MAPREDUCE-6897. Add Unit Test to ensure Job end notification gets sent even when appMaster stop gets YarnRuntimeException. Contributed by Gergely Novák 
(cherry picked from commit 82bbcbf37f8137112a6270932b2ad7572785c387)
 2017-06-16 14:27:06 -07:00
Jason Lowe
43fb793174 Revert "MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko" 
This reverts commit 576cc258f9ac1bccb13c01f147b360c526188467.
 2017-06-16 14:49:26 -05:00
Jason Lowe
576cc258f9 MAPREDUCE-6898. TestKill.testKillTask is flaky. Contributed by Peter Bacsko 
(cherry picked from commit e78542f4bea90e8545e9e06af9c07e89d2be43d9)
 2017-06-16 14:26:15 -05:00
Xiao Chen
2afe9722af HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively. 2017-06-16 09:49:41 -07:00
Steve Loughran
4e6348f346
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0. 
Contributed by Hongyuan Li.

(cherry picked from commit 942ecbbc9876427051965ddd167df5cc1cc253e3)
 2017-06-16 09:46:34 +01:00
Akira Ajisaka
12cb3456d7
HADOOP-14289. Move log4j APIs over to slf4j in hadoop-common. 
(cherry picked from commit e855cc4ed467d03f4f03e8b3a1ff27654b349a52)
 2017-06-16 13:32:16 +09:00
Lei Xu
abac844c90 HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei) 
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)

    Conflicts:
    hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
    hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
 2017-06-15 10:59:11 -07:00
Xiao Chen
7d81b0beab HADOOP-14523. OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory. Contributed by Misha Dmitriev. 
(cherry picked from commit ef8edab930338646551cbe3c7e7cf954e21c0f9a)
 2017-06-15 10:41:34 -07:00
Akira Ajisaka
0fd0612896
Revert "HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin." 
This reverts commit 88fa17c2383baafbfb6b43b69e08cf40493c774a.
 2017-06-16 00:21:54 +09:00
Steve Loughran
31446f8e10
HADOOP-14506. Add create() contract test that verifies ancestor dir creation. 
Contributed by Sean Mackrory.

(cherry picked from commit d780a67864d2c687d23b69798c19ee865e38cc5a)
 2017-06-15 14:51:50 +01:00
Akira Ajisaka
88fa17c238
HDFS-11736. OIV tests should not write outside 'target' directory. Contributed by Yiqun Lin. 
(cherry picked from commit 315f07700dcfe32513b87f63070db854ecb3cf48)
 2017-06-15 13:47:22 +09:00
Junping Du
03bd121ba1 MAPREDUCE-6895. Job end notification not send due to YarnRuntimeException. Contributed by yunjiong zhao. 
(cherry picked from commit 6ed54f3439ea9c7af6bf129ebe1938380febb5e2)
 2017-06-13 21:45:21 -07:00
Tsz-Wo Nicholas Sze
670224ae59 HDFS-11947. When constructing a thread name, BPOfferService may print a bogus warning message. Contributed by Weiwei Yang 2017-06-13 09:47:14 +08:00
Arpit Agarwal
eda4bf5196 HDFS-11967. TestJMXGet fails occasionally. Contributed by Arpit Agarwal. 2017-06-12 16:04:08 -07:00
Arpit Agarwal
097b1f74e5 HDFS-11907. Add metric for time taken by NameNode resource check. Contributed by Chen Liang. 2017-06-12 15:22:36 -07:00
Jonathan Eagles
f901132a8a HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles) 2017-06-12 17:19:46 -05:00
Daniel Templeton
6d7e847cfc HADOOP-14310. RolloverSignerSecretProvider.LOG should be @VisibleForTesting 
(Contributed by Arun Shanmugam Kumar via Daniel Templeton)

(cherry picked from commit 86368cc7667adfe224df77b039c2ffe8de7f889a)
 2017-06-12 09:56:35 -07:00
bibinchundatt
3f9acf65fb YARN-6703. RM startup failure with old state store due to version mismatch. Contributed by Varun Saxena 
(cherry picked from commit d64c842743da0d9d91c46985a9fd7350ea14b204)
 2017-06-12 18:32:23 +05:30
Mingliang Liu
ae2457e1dd HADOOP-14465. LdapGroupsMapping - support user and group search base. Contributed by Shwetha G S and Mingliang Liu 
(cherry picked from commit a2121cb0d907be439d19cd1165a0371b37a5fe68)
 2017-06-09 16:26:10 -07:00
Eric Payne
33605e0896 YARN-6585. RM fails to start when upgrading from 2.7 for clusters with node labels. Contributed by Sunil G. 
(cherry picked from commit 5578af860335ae44c9780082508c3dcf726f60fc)
 2017-06-09 16:18:16 -05:00
Mingliang Liu
c953e0ce8b HADOOP-14512. WASB atomic rename should not throw exception if the file is neither in src nor in dst when doing the rename. Contributed by Duo Xu 
(cherry picked from commit 325163f23f727e82379d4a385b73aa3a04a510f6)
 2017-06-09 13:07:35 -07:00
Naganarasimha
85d561a5b3 YARN-6681. Improve performance of AssignmentInformation datastructures. Contributed by Daryn sharp. 2017-06-09 11:46:51 +05:30
Xiao Chen
aaf533e8a0 HADOOP-13854. KMS should log error details in KMSExceptionsProvider. 
(cherry picked from commit c4b5c32669423b9a792f33f9f8333d95528f2515)
 2017-06-08 22:38:31 -07:00
Xiao Chen
ffaf24e308 HADOOP-13174. Add more debug logs for delegation tokens and authentication. 
(cherry picked from commit 4a56bde6ba1f72588a25cd96acc76089706cb786)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java
 2017-06-08 21:34:09 -07:00
John Zhuge
87f4428de0 HADOOP-14511. WritableRpcEngine.Invocation#toString NPE on null parameters. Contributed by John Zhuge. 2017-06-08 21:26:02 -07:00
Mingliang Liu
8188b9dd05 HADOOP-14283. Upgrade AWS SDK to 1.11.134. Contributed by Aaron Fabbri 
(cherry picked from commit 7c9694c56712e20589d83b31e71c7bbddc131b8d)
 2017-06-08 15:07:08 -07:00
Mingliang Liu
facb6812d9 HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 2017-06-08 14:46:31 -07:00
Daniel Templeton
c5b28c03a1 YARN-6679. Reduce Resource instance overhead via non-PBImpl 
(Contributed by Daryn Sharp via Daniel Templeton)
 2017-06-08 10:41:32 -07:00
John Zhuge
2448d84572 HDFS-11861. ipc.Client.Connection#sendRpcRequest should log request name. Contributed by John Zhuge. 
(cherry picked from commit 5672ae7b37ce75086a1cb5bb9a388288fc913eb7)
 2017-06-07 15:54:08 -07:00
Mingliang Liu
de4e4ec774 HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan 
(cherry picked from commit df35e4cc3b0c67bcaa3d809c6cb5a0a2d82986af)
 2017-06-07 13:55:11 -07:00
Brahma Reddy Battula
a5a5bcdb60 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f008fc7945702c0fad7d1028f75bfc6)
 2017-06-07 16:37:11 +08:00
Brahma Reddy Battula
fca08f8362 HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 2017-06-07 16:54:29 +08:00
Yiqun Lin
72f0fdfb59 HDFS-11929. Document missing processor of hdfs oiv_legacy command. Contributed by LiXin Ge. 
(cherry picked from commit 5ec71639cf41858c687781445b8360d50cad5b55)
 2017-06-07 13:44:33 +08:00
Vinayakumar B
9a26120809 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:57:47 +05:30
Andrew Wang
5c819585f2 HDFS-11840. Log HDFS Mover exception message of exit to its own log. Contributed by LiXin Ge. 
(cherry picked from commit c31cb879a3595f76d01408e6d066a0bde20be043)
 2017-06-06 14:57:51 -07:00
Arpit Agarwal
226688a12c HDFS-11932. BPServiceActor thread name is not correctly set. Contributed by Chen Liang. 2017-06-06 13:51:25 -07:00
Yongjun Zhang
1e52314117 HDFS-11914. Add more diagnosis info for fsimage transfer failure. Contributed by Yongjun Zhang. 
(cherry picked from commit 6a28a2b914664fbbb5267f13e82d42b3bb49610c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
 2017-06-06 12:31:11 -07:00
Mingliang Liu
e3a72c2b5c HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 536f057158c445a57049f6c392869ae2f0be4f24)
 2017-06-06 11:13:42 -07:00
Mingliang Liu
bacfd1baae HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu 
(cherry picked from commit 6b5285bbcb439944ba6c4701571ffbb00258d5a1)
 2017-06-06 11:07:42 -07:00
Kihwal Lee
f7e597b56d HADOOP-14035. Reduce fair call queue backoff's impact on clients. Contributed by Daryn Sharp. 2017-06-06 12:19:44 -05:00