Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,240 Commits 326 Branches 371 Tags
Commit Graph

13240 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Brahma Reddy Battula
e889c826d7 HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li. 
(cherry picked from commit 66c6fd831497944f4f49c5ce42c69a302b7d7bc0)
 2017-06-06 12:32:37 +08:00
Christopher Douglas
95c2978640 YARN-1471. Preserve scheduler typeinfo in simulator to work with resource monitors. 
Contributed by Carlo Curino. Backport by Ye Zhou.

(cherry picked from commit e07f3c87cd3542760efcf713f909375b69eec00f)
 2017-06-05 16:13:51 -07:00
Mingliang Liu
53e90a8064 HADOOP-14478. Optimize NativeAzureFsInputStream for positional reads. Contributed by Rajesh Balamohan 
(cherry picked from commit 5fd9742c83fbeae96bf0913bdcdf77fafbf15b2f)
 2017-06-05 16:01:27 -07:00
Kihwal Lee
e2ea601231 HDFS-10816. TestComputeInvalidateWork#testDatanodeReRegistration fails due to race between test and replication monitor. Contributed by Eric Badger. 2017-06-05 15:24:54 -05:00
Arpit Agarwal
baff06f019 HDFS-11928. Segment overflow in FileDistributionCalculator. Contributed by LiXin Ge. 2017-06-05 13:21:35 -07:00
Mingliang Liu
867026d94e HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu 
(cherry picked from commit 6aeda55bb8f741d9dafd41f6dfbf1a88acdd4003)
 2017-06-05 11:31:13 -07:00
Brahma Reddy Battula
ca76378aad HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger. 
(cherry picked from commit abdd609e51a80388493417126c3bc9b1badc0ac1)
 2017-06-06 00:21:59 +08:00
Yiqun Lin
2e8557de12 HDFS-11899. ASF License warnings generated intermittently in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 367da9b7b8837d2ffa30a7d2716e463adf2d7007)
 2017-06-03 22:10:01 +08:00
Xiao Chen
5e40efe0ec HADOOP-14474. Use OpenJDK 7 instead of Oracle JDK 7 to avoid oracle-java7-installer failures. Contributed by Akira Ajisaka. 2017-06-02 09:29:05 -07:00
Daniel Templeton
0bcbf2a09b YARN-6208. Improve the log when FinishAppEvent sent to the NodeManager which didn't run the application 
(Contributed by Akira Ajisaka via Daniel Templeton)

(cherry picked from commit 73ecb19312879d54e1cbe80199fe950d81c81104)
 2017-06-02 08:52:27 -07:00
Brahma Reddy Battula
8e119f1e90 HADOOP-14436. Remove the redundant colon in ViewFs.md. Contributed by maobaolong. 
(cherry picked from commit 056cc72885471d6952ff182670e4b4a38421603d)
 2017-06-02 22:42:23 +08:00
Brahma Reddy Battula
ac2af66e77 HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang. 2017-06-02 22:22:43 +08:00
Yiqun Lin
870b5e39d2 HDFS-11359. DFSAdmin report command supports displaying maintenance state datanodes. Contributed by Yiqun Lin. 2017-06-02 12:53:31 +08:00
Andrew Wang
d0a4e7e95d HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev. 
(cherry picked from commit 7101477e4726a70ab0eab57c2d4480a04446a8dc)
 2017-06-01 15:20:22 -07:00
Xiao Chen
eb98773ac8 HDFS-11904. Reuse iip in unprotectedRemoveXAttrs calls. 
(cherry picked from commit 219f4c199e45f8ce7f41192493bf0dc8f1e5dc30)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirErasureCodingOp.java
 2017-06-01 14:16:58 -07:00
Xiao Chen
91b72a8aa5 HDFS-11741. Long running balancer may fail due to expired DataEncryptionKey. Contributed by Wei-Chiu Chuang and Xiao Chen. 
(cherry picked from commit 068e23b896c63b0f817e6b91d73c994be1551eb2)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/security/token/block/BlockTokenSecretManager.java
 2017-06-01 14:05:43 -07:00
Mingliang Liu
0b78db35fd HADOOP-14460. Azure: update doc for live and contract tests. Contributed by Mingliang Liu 
(cherry picked from commit ece33208b89a6a06c1b362d10fab31343fe5df13)
 2017-06-01 11:52:37 -07:00
Brahma Reddy Battula
5cbdefbcc0 HDFS-11893. Fix TestDFSShell.testMoveWithTargetPortEmpty failure. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 0dcf843c008f2b9cece8c0a0cef78140398ac464)
 2017-06-01 22:39:22 +08:00
Daniel Templeton
406d8afdd4 HADOOP-9849. License information is missing for native CRC32 code 
(Contributed by Andrew Wang via Daniel Templeton)

(cherry picked from commit 92243484f9b868aa18618759145e9dd554e245c5)
 2017-05-31 15:59:29 -07:00
Daniel Templeton
10b7a5fa98 YARN-6246. Identifying starved apps does not need the scheduler writelock 
(Contributed by Karthik Kambatla via Daniel Templeton)

(cherry picked from commit d5b71e4175c13679d451710be150fc461a661263)
 2017-05-31 15:50:11 -07:00
Kihwal Lee
212a566088 HDFS-11856. Ability to re-add Upgrading nodes to pipeline for future pipeline updates. Contributed by Vinayakumar B. 2017-05-31 12:45:33 -05:00
Nathan Roberts
c112bf6838 YARN-6649. RollingLevelDBTimelineServer throws RuntimeException if object decoding ever fails runtime exception. Contributed by Jon Eagles. 
(cherry picked from commit 4369690ce63566131aee28696bf2683a3cb20205)
 2017-05-31 12:35:50 -05:00
Kihwal Lee
5b0baeab5e HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B. 2017-05-31 11:02:34 -05:00
Jonathan Eagles
f5f12b576e YARN-6497. Method length of ResourceManager#serviceInit() is too long (Gergely Novák via jeagles) 
(cherry picked from commit cbfed0e82f57e96b8d5309e0613057963840554f)
 2017-05-31 10:19:51 -05:00
Varun Vasudev
1eecde3355 YARN-6366. Refactor the NodeManager DeletionService to support additional DeletionTask types. Contributed by Shane Kumpf. 
(cherry picked from commit 547f18cb96aeda55cc19b38be2be4d631b3a5f4f)
 2017-05-31 16:16:33 +05:30
Lei Xu
a3ad1a39c5 HDFS-11659. TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten fail due to no DataNode available for pipeline recovery. Contributed by Lei (Eddy) Xu. 
(cherry picked from commit 91d6fe151f2e3de21b0a9423ade921e771957d90)
 2017-05-30 11:11:00 -07:00
John Zhuge
f72f194e24 HADOOP-14464. hadoop-aws doc header warning #5 line wrapped. Contributed by John Zhuge. 
(cherry picked from commit 6c6a7a59622ba7c1e4faa5534f4479de0cd84993)
 2017-05-28 22:29:03 -07:00
Yiqun Lin
d36e64b8b2 HDFS-11446. TestMaintenanceState#testWithNNAndDNRestart fails intermittently. Contributed by Yiqun Lin. 
(cherry picked from commit 31058b243e9575d90f038bb2fdf5a556710f6f7f)
 2017-05-28 11:27:09 +08:00
Mingliang Liu
9d7e67ccf9 HADOOP-14442. Owner support for ranger-wasb integration. Contributed by Varada Hemeswari 
(cherry picked from commit 89bb8bfe582ba85566cede321b233bb642f1c675)
 2017-05-26 17:59:14 -07:00
Arpit Agarwal
f200399496 HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang. 2017-05-26 16:02:53 -07:00
Inigo Goiri
e336d92b99 Fix NPE in LazyPersistFileScrubber. Contributed by Inigo Goiri. 
(cherry picked from commit 303c8dc9b6c853c0939ea9ba14388897cc258071)
 2017-05-26 13:40:16 -07:00
Jason Lowe
3c0eb9c415 YARN-6641. Non-public resource localization on a bad disk causes subsequent containers failure. Contributed by Kuhu Shukla 
(cherry picked from commit aea42930bbb9566ea6988f684dbd72a72a2bdadf)
 2017-05-26 09:39:22 -05:00
Xiao Chen
54971c4195 HDFS-11421. Make WebHDFS' ACLs RegEx configurable. Contributed by Harsh J. 2017-05-25 21:01:39 -07:00
Kihwal Lee
2cb63433ab HDFS-11817. A faulty node can cause a lease leak and NPE on accessing data. Contributed by Kihwal Lee. 2017-05-25 17:21:56 -05:00
Yufei Gu
fc6cb4b2dd YARN-6582. FSAppAttempt demand can be updated atomically in updateDemand(). (Karthik Kambatla via Yufei Gu) 
(cherry picked from commit 87590090c887829e874a7132be9cf8de061437d6)
 2017-05-25 14:25:49 -07:00
Jason Lowe
4c44ff69df YARN-6643. TestRMFailover fails rarely due to port conflict. Contributed by Robert Kanter 
(cherry picked from commit 3fd6a2da4e537423d1462238e10cc9e1f698d1c2)
 2017-05-25 16:09:08 -05:00
Brahma Reddy Battula
7bad748091 HDFS-11445. FSCK shows overall health stauts as corrupt even one replica is corrupt. Contributed by Brahma Reddy Battula. 2017-05-25 22:38:12 +08:00
Steve Loughran
1efba62fd7
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem's 
getFileStatus method is always 0.
Contributed by Hongyuan Li.
 2017-05-25 15:17:43 +01:00
Steve Loughran
07c14b35ec
HADOOP-14399. Configuration does not correctly XInclude absolute file URIs. 
Contributed by Jonathan Eagles

(cherry picked from commit 1ba9704eec22c75f8aec653ee15eb6767b5a7f4b)
 2017-05-25 15:03:01 +01:00
Varun Vasudev
1a6c532301 Addendum patch to fix Docker sanitization. 
(cherry picked from commit 2ff2a1f50e8c7c0f33676b010b256d6c8daf912d)
 2017-05-25 14:54:08 +05:30
Akira Ajisaka
ca1c0cbc62
YARN-6141. ppc64le on Linux doesn't trigger __linux get_executable codepath. Contributed by Sonia Garudi and Ayappan. 
(cherry picked from commit bc28da65fb1c67904aa3cefd7273cb7423521014)
 2017-05-25 17:07:50 +09:00
Yongjun Zhang
dd552a97b7 HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang) 2017-05-24 19:05:40 -07:00
Yongjun Zhang
49dc547252 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-05-24 18:54:51 -07:00
Yongjun Zhang
97317b1aaa Revert "HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li." 
This reverts commit c411adac32cda42433b15979253248336487d977.
 2017-05-24 18:54:51 -07:00
Yongjun Zhang
39474ed50d Revert "HADOOP-14407. DistCp - Introduce a configurable copy buffer size. (Omkar Aradhya K S via Yongjun Zhang)" 
This reverts commit fe185e2c3a07d4e6a82ed4fd255c98ec8b561ad6.
 2017-05-24 18:54:50 -07:00
Wei-Chiu Chuang
bd0138ea0a Revert "HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory." 
This reverts commit 5c96ef365de1a7ffc6a13f2d64cda9a26d59e4aa.
 2017-05-24 18:08:30 -07:00
Wei-Chiu Chuang
fe2c28df70 Revert "HDFS-11515. -du throws ConcurrentModificationException. Contributed by Istvan Fajth, Wei-Chiu Chuang." 
This reverts commit 4f1f16f1a67dcb6db705a73bf7fcf3f0f9d7094c.
 2017-05-24 18:08:10 -07:00
Amar Kamat
3a8f3bcccb HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk) 
(cherry picked from commit 82d57ee7fee82919b42066af38063a27d0998806)
 2017-05-24 17:01:49 -07:00
Subru Krishnan
d1724ef919 YARN-6587. Refactor of ResourceManager#startWebApp in a Util class. (Giovanni Matteo Fumarola via Subru). 2017-05-24 13:40:49 -07:00
Brahma Reddy Battula
6f55be197c HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dcf4559ebdb5bb11d03b264a9875bff316b89eef)
 2017-05-25 01:27:52 +08:00