Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,657 Commits 326 Branches 371 Tags
Commit Graph

13657 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiao Chen
ffaf24e308 HADOOP-13174. Add more debug logs for delegation tokens and authentication. 
(cherry picked from commit 4a56bde6ba1f72588a25cd96acc76089706cb786)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java
 2017-06-08 21:34:09 -07:00
John Zhuge
87f4428de0 HADOOP-14511. WritableRpcEngine.Invocation#toString NPE on null parameters. Contributed by John Zhuge. 2017-06-08 21:26:02 -07:00
Mingliang Liu
8188b9dd05 HADOOP-14283. Upgrade AWS SDK to 1.11.134. Contributed by Aaron Fabbri 
(cherry picked from commit 7c9694c56712e20589d83b31e71c7bbddc131b8d)
 2017-06-08 15:07:08 -07:00
Mingliang Liu
facb6812d9 HDFS-11945. Internal lease recovery may not be retried for a long time. Contributed by Kihwal Lee 2017-06-08 14:46:31 -07:00
Daniel Templeton
c5b28c03a1 YARN-6679. Reduce Resource instance overhead via non-PBImpl 
(Contributed by Daryn Sharp via Daniel Templeton)
 2017-06-08 10:41:32 -07:00
John Zhuge
2448d84572 HDFS-11861. ipc.Client.Connection#sendRpcRequest should log request name. Contributed by John Zhuge. 
(cherry picked from commit 5672ae7b37ce75086a1cb5bb9a388288fc913eb7)
 2017-06-07 15:54:08 -07:00
Mingliang Liu
de4e4ec774 HADOOP-14500. Azure: TestFileSystemOperationExceptionHandling{,MultiThreaded} fails. Contributed by Rajesh Balamohan 
(cherry picked from commit df35e4cc3b0c67bcaa3d809c6cb5a0a2d82986af)
 2017-06-07 13:55:11 -07:00
Brahma Reddy Battula
a5a5bcdb60 MAPREDUCE-6676. NNBench should Throw IOException when rename and delete fails. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 24181f559f008fc7945702c0fad7d1028f75bfc6)
 2017-06-07 16:37:11 +08:00
Brahma Reddy Battula
fca08f8362 HDFS-11711. DN should not delete the block On "Too many open files" Exception. Contributed by Brahma Reddy Battula. 2017-06-07 16:54:29 +08:00
Yiqun Lin
72f0fdfb59 HDFS-11929. Document missing processor of hdfs oiv_legacy command. Contributed by LiXin Ge. 
(cherry picked from commit 5ec71639cf41858c687781445b8360d50cad5b55)
 2017-06-07 13:44:33 +08:00
Vinayakumar B
9a26120809 HDFS-11708. Positional read will fail if replicas moved to different DNs after stream is opened. Contributed by Vinayakumar B. 2017-06-07 10:57:47 +05:30
Andrew Wang
5c819585f2 HDFS-11840. Log HDFS Mover exception message of exit to its own log. Contributed by LiXin Ge. 
(cherry picked from commit c31cb879a3595f76d01408e6d066a0bde20be043)
 2017-06-06 14:57:51 -07:00
Arpit Agarwal
226688a12c HDFS-11932. BPServiceActor thread name is not correctly set. Contributed by Chen Liang. 2017-06-06 13:51:25 -07:00
Yongjun Zhang
1e52314117 HDFS-11914. Add more diagnosis info for fsimage transfer failure. Contributed by Yongjun Zhang. 
(cherry picked from commit 6a28a2b914664fbbb5267f13e82d42b3bb49610c)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Util.java
 2017-06-06 12:31:11 -07:00
Mingliang Liu
e3a72c2b5c HADOOP-14491. Azure has messed doc structure. Contributed by Mingliang Liu 
(cherry picked from commit 536f057158c445a57049f6c392869ae2f0be4f24)
 2017-06-06 11:13:42 -07:00
Mingliang Liu
bacfd1baae HADOOP-14472. Azure: TestReadAndSeekPageBlobAfterWrite fails intermittently. Contributed by Mingliang Liu 
(cherry picked from commit 6b5285bbcb439944ba6c4701571ffbb00258d5a1)
 2017-06-06 11:07:42 -07:00
Kihwal Lee
f7e597b56d HADOOP-14035. Reduce fair call queue backoff's impact on clients. Contributed by Daryn Sharp. 2017-06-06 12:19:44 -05:00
Brahma Reddy Battula
e889c826d7 HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li. 
(cherry picked from commit 66c6fd831497944f4f49c5ce42c69a302b7d7bc0)
 2017-06-06 12:32:37 +08:00
Christopher Douglas
95c2978640 YARN-1471. Preserve scheduler typeinfo in simulator to work with resource monitors. 
Contributed by Carlo Curino. Backport by Ye Zhou.

(cherry picked from commit e07f3c87cd3542760efcf713f909375b69eec00f)
 2017-06-05 16:13:51 -07:00
Mingliang Liu
53e90a8064 HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan 
(cherry picked from commit 5fd9742c83fbeae96bf0913bdcdf77fafbf15b2f)
 2017-06-05 16:01:27 -07:00
Kihwal Lee
e2ea601231 HDFS-10816. TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor. Contributed by Eric Badger. 2017-06-05 15:24:54 -05:00
Arpit Agarwal
baff06f019 HDFS-11928. Segment overflow in FileDistributionCalculator. Contributed by LiXin Ge. 2017-06-05 13:21:35 -07:00
Mingliang Liu
867026d94e HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 
(cherry picked from commit 6aeda55bb8f741d9dafd41f6dfbf1a88acdd4003)
 2017-06-05 11:31:13 -07:00
Brahma Reddy Battula
ca76378aad HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger. 
(cherry picked from commit abdd609e51a80388493417126c3bc9b1badc0ac1)
 2017-06-06 00:21:59 +08:00
Yiqun Lin
2e8557de12 HDFS-11899. ASF License warnings generated intermittently in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 367da9b7b8837d2ffa30a7d2716e463adf2d7007)
 2017-06-03 22:10:01 +08:00
Xiao Chen
5e40efe0ec HADOOP-14474. Use OpenJDK 7 instead of Oracle JDK 7 to avoid oracle-java7-installer failures. Contributed by Akira Ajisaka. 2017-06-02 09:29:05 -07:00
Daniel Templeton
0bcbf2a09b YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application 
(Contributed by Akira Ajisaka via Daniel Templeton)

(cherry picked from commit 73ecb19312879d54e1cbe80199fe950d81c81104)
 2017-06-02 08:52:27 -07:00
Brahma Reddy Battula
8e119f1e90 HADOOP-14436. Remove the redundant colon in ViewFs.md. Contributed by maobaolong. 
(cherry picked from commit 056cc72885471d6952ff182670e4b4a38421603d)
 2017-06-02 22:42:23 +08:00
Brahma Reddy Battula
ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Yiqun Lin
870b5e39d2 HDFS-11359. DFSAdmin report command supports displaying maintenance state datanodes. Contributed by Yiqun Lin. 2017-06-02 12:53:31 +08:00
Andrew Wang
d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e4726a70ab0eab57c2d4480a04446a8dc)
 2017-06-01 15:20:22 -07:00
Xiao Chen
eb98773ac8 HDFS-11904. Reuse iip in unprotectedRemoveXAttrs calls. 
(cherry picked from commit 219f4c199e45f8ce7f41192493bf0dc8f1e5dc30)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
 2017-06-01 14:16:58 -07:00
Xiao Chen
91b72a8aa5 HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2017-06-01 14:05:43 -07:00
Mingliang Liu
0b78db35fd HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu 
(cherry picked from commit ece33208b89a6a06c1b362d10fab31343fe5df13)
 2017-06-01 11:52:37 -07:00
Brahma Reddy Battula
5cbdefbcc0 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c008f2b9cece8c0a0cef78140398ac464)
 2017-06-01 22:39:22 +08:00
Daniel Templeton
406d8afdd4 HADOOP-9849. License information is missing for native CRC32 code 
(Contributed by Andrew Wang via Daniel Templeton)

(cherry picked from commit 92243484f9b868aa18618759145e9dd554e245c5)
 2017-05-31 15:59:29 -07:00
Daniel Templeton
10b7a5fa98 YARN-6246. Identifying starved apps does not need the scheduler writelock 
(Contributed by Karthik Kambatla via Daniel Templeton)

(cherry picked from commit d5b71e4175c13679d451710be150fc461a661263)
 2017-05-31 15:50:11 -07:00
Kihwal Lee
212a566088 HDFS-11856. Ability to re-add Upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 12:45:33 -05:00
Nathan Roberts
c112bf6838 YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles. 
(cherry picked from commit 4369690ce63566131aee28696bf2683a3cb20205)
 2017-05-31 12:35:50 -05:00
Kihwal Lee
5b0baeab5e HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 11:02:34 -05:00
Jonathan Eagles
f5f12b576e YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles) 
(cherry picked from commit cbfed0e82f57e96b8d5309e0613057963840554f)
 2017-05-31 10:19:51 -05:00
Varun Vasudev
1eecde3355 YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf. 
(cherry picked from commit 547f18cb96aeda55cc19b38be2be4d631b3a5f4f)
 2017-05-31 16:16:33 +05:30
Lei Xu
a3ad1a39c5 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 91d6fe151f2e3de21b0a9423ade921e771957d90)
 2017-05-30 11:11:00 -07:00
John Zhuge
f72f194e24 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 
(cherry picked from commit 6c6a7a59622ba7c1e4faa5534f4479de0cd84993)
 2017-05-28 22:29:03 -07:00
Yiqun Lin
d36e64b8b2 HDFS-11446. TestMaintenanceState#testWithNNAndDNRestart fails intermittently. Contributed by Yiqun Lin. 
(cherry picked from commit 31058b243e9575d90f038bb2fdf5a556710f6f7f)
 2017-05-28 11:27:09 +08:00
Mingliang Liu
9d7e67ccf9 HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari 
(cherry picked from commit 89bb8bfe582ba85566cede321b233bb642f1c675)
 2017-05-26 17:59:14 -07:00
Arpit Agarwal
f200399496 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:02:53 -07:00
Inigo Goiri
e336d92b99 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6c853c0939ea9ba14388897cc258071)
 2017-05-26 13:40:16 -07:00
Jason Lowe
3c0eb9c415 YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla 
(cherry picked from commit aea42930bbb9566ea6988f684dbd72a72a2bdadf)
 2017-05-26 09:39:22 -05:00
Xiao Chen
54971c4195 HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J. 2017-05-25 21:01:39 -07:00