Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,769 Commits 326 Branches 371 Tags
Commit Graph

17769 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Inigo Goiri
11d3db2dba HDFS-13681. Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 8762e9cf10fa100dd5f7fd695f5e52b75a94c5d4)
 2018-06-15 16:50:20 -07:00
Inigo Goiri
69abd1fc5c HDFS-13686. Add overall metrics for FSNamesystemLock. Contributed by Lukas Majercak. 
(cherry picked from commit d31a3ce767d3bb68bdbb4f36d45600eab9f4f8b7)
 2018-06-15 16:00:16 -07:00
Wei-Chiu Chuang
73badce597 HDFS-13174. hdfs mover -p /path times out after 20 min. Contributed by Istvan Fajth. 
(cherry picked from commit c966a3837af1c1a1c4a441f491b0d76d5c9e5d78)
(cherry picked from commit 975d4b3d603632a5edacb138cf4a1ce92ebed02e)
 2018-06-15 13:38:11 -07:00
Inigo Goiri
fe1926a50f HDFS-13676. TestEditLogRace fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit eebeb6033fd09791fcbff626f128a98e393f0a88)
 2018-06-15 13:08:47 -07:00
Inigo Goiri
94e0b9839f HDFS-13673. TestNameNodeMetrics fails on Windows. Contributed by Zuoming Zhang. 
(cherry picked from commit 43d994e4a6dfd1c24eafb909d6f8a0663b20769a)
 2018-06-15 10:34:33 -07:00
Inigo Goiri
a3c0c03899 HDFS-13563. TestDFSAdminWithHA times out on Windows. Contributed by Lukas Majercak. 
(cherry picked from commit 8d4926f38bf53b32453cd2bc7322c8818f752f85)
 2018-06-14 09:59:50 -07:00
Rohith Sharma K S
6542e31b78 YARN-8155. Improve ATSv2 client logging in RM and NM publisher. Contributed by Abhishek Modi. 
(cherry picked from commit 9119b3cf8f883aa2d5df534afc0c50249fed03c6)
 2018-06-14 13:51:05 +05:30
Yiqun Lin
b94b071b10 HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 
(cherry picked from commit 8e7548d33be9c4874daab18b2e774bdc2ed216d3)
 2018-06-13 20:24:12 +08:00
Sunil G
d9b00cdd6b YARN-8404. Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down. Contributed by Rohith Sharma K S 
(cherry picked from commit 6307962b932e0ee69ba61f5796388c175d79195a)
 2018-06-13 16:10:24 +05:30
Weiwei Yang
ef105abb70 YARN-8394. Improve data locality documentation for Capacity Scheduler. Contributed by Weiwei Yang. 
(Cherry picked from commit 29024a62038c297f11e8992601f2522ffffc7da7)
 2018-06-13 13:57:35 +08:00
Inigo Goiri
189b6bac8c HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit aeaf9fec62f10699d1c809d66444520fe4533c2c)
 2018-06-12 14:18:19 -07:00
Inigo Goiri
fa9d4a7493 YARN-8422. TestAMSimulator failing with NPE. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit c35481594ffc372e3f846b0c8ebc2ff9e36ffdb0)
 2018-06-12 11:01:22 -07:00
Inigo Goiri
6a31dc9927 HADOOP-15529. ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 6e756e8a620e4d6dc3192986679060c52063489b)
 2018-06-12 10:25:36 -07:00
Rohith Sharma K S
96e7b7e8ae YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace06cfd2b3193a14cd455297f8f989617)
 2018-06-12 17:25:22 +05:30
Inigo Goiri
26ed145763 YARN-8370. Some Node Manager tests fail on Windows due to improper path/file separator. Contributed by Anbang Hu. 
(cherry picked from commit 2b2f672022547e8c19658213ac5a4090bf5b6c72)
 2018-06-11 19:26:58 -07:00
Steve Loughran
df338f2e1a HADOOP-15506. Upgrade Azure Storage Sdk version to 7.0.0 and update corresponding code blocks. 
Contributed by Esfandiar Manii.

(cherry picked from commit d901be679554eb6b323f3bc6e8de267d85dd2e06)
(cherry picked from commit baac7c2b285454d71d0371505fb7a3403a548176)
 2018-06-11 15:33:36 -07:00
Yongjun Zhang
8202c334fe Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release. 
(cherry picked from commit 5a82f10e3254bc0745c2dda6bcef888d3ff0d1c3)
 2018-06-11 14:28:44 -07:00
Inigo Goiri
6eea6d28de HDFS-13653. Make dfs.client.failover.random.order a per nameservice configuration. Contributed by Ekanth Sethuramalingam. 
(cherry picked from commit 784dbbd2d1fc0c361fee851d3f9b3aa50be5481d)
 2018-06-11 11:18:04 -07:00
Sammi Chen
b874110275 HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen. 
(cherry picked from commit 18201b882a38ad875358c5d23c09b0ef903c2f91)
 2018-06-11 13:58:57 +08:00
Bharat Viswanadham
7c144e4ae4 HDFS-13667:Typo: Marking all datandoes as stale. Contributed by Nanda Kumar 
(cherry picked from commit ccfb816d39878abf4172933327d788c59b9eb082)
 2018-06-09 16:44:47 -07:00
Arpit Agarwal
ea05171ceb HDFS-12670. can't renew HDFS tokens with only the hdfs client jar. Contributed by Arpit Agarwal. 2018-06-08 17:58:02 -07:00
Chao Sun
d62b0f84f1 HDFS-13664. Refactor ConfiguredFailoverProxyProvider to make inheritance easier. Contributed by Chao Sun. 
(cherry picked from commit fba1c42adc1c8ae57951e1865ec2ab05c8707bdf)
 2018-06-08 16:42:18 -07:00
Inigo Goiri
500057c40c YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:38 -07:00
Brahma Reddy Battula
a45e9812d6 HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9654dd1f472052c4bb4a48a7412149c2e4859a10)
(cherry picked from commit 630f43faf7a1d4aaaefd860e3dcf25c288ca1d2f)
 2018-06-07 09:18:20 -07:00
Wei-Chiu Chuang
8e423a7779 HDFS-13659. Add more test coverage for contentSummary for snapshottable path. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e39b113db0f2e4bcf93f873801be770e472601da)
(cherry picked from commit b2e919dc1a8688bb6499f7334ad67c49dcc1bc93)
 2018-06-07 08:32:53 -07:00
Xiao Chen
3843abe482 HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 
(cherry picked from commit ba4011d64fadef3bee5920ccedbcdac01794cc23)
 2018-06-05 21:19:19 -07:00
Inigo Goiri
49cfb11d8c MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 0afc036deb35df7e86ede3dcebc430c8f05ed368)
 2018-06-05 12:25:59 -07:00
Akira Ajisaka
cc735146dc
HDFS-13545. "guarded" is misspelled as "gaurded" in FSPermissionChecker.java. Contributed by Jianchao Jia. 
(cherry picked from commit 0e3c31579d1c733decfb9b00721bdc7b86be60f2)
 2018-06-05 17:55:33 +09:00
Inigo Goiri
e794ff2dcb HDFS-13650. Randomize baseDir for MiniDFSCluster in TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 8d31ddcfeb5c2e97692dd386e5bd62f433b44f8e)
 2018-06-04 18:29:12 -07:00
Inigo Goiri
076dc95c76 HDFS-13649. Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 16316b60112842cf8a328b79c89297f6a8b30e7b)
 2018-06-04 18:22:35 -07:00
Inigo Goiri
2e5f849ea5 HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 
(cherry picked from commit a28848104f2a067669f9b1706cd0da05ba71bcb7)
 2018-06-04 18:13:50 -07:00
Bharat Viswanadham
f1e1c068c0 HADOOP-15137. ClassNotFoundException: org.apache.hadoop.yarn.server.api.DistributedSchedulingAMProtocol when using hadoop-client-minicluster. Contributed by Bharat Viswanadham 
(cherry picked from commit f30f2dc4085c51b6b6850430213f47b97a763b7f)
 2018-06-04 16:51:22 -07:00
Robert Kanter
d5e6d0d5f4 YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a44390bc1a01113dce4be4e629637c3e8a)
 2018-06-04 15:42:46 -07:00
Inigo Goiri
9fe265f44e MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit dad1bb868f7a28a486fd500f102c8c174e6dd272)
 2018-06-04 12:57:11 -07:00
Miklos Szegedi
a3afd69051 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 11:07:08 -07:00
Rushabh Shah
4ecb7aaf77 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 
(cherry picked from commit e2289c8d1496a5eff88e6bcb8776a11d45371ffc)
 2018-06-04 09:58:38 -05:00
Wei-Chiu Chuang
63fc8f82cb HDFS-13155. BlockPlacementPolicyDefault.chooseTargetInOrder Not Checking Return Value for NULL. Contributed by Zsolt Venczel. 
(cherry picked from commit bccdfeee0aaef9cb98d09ee39909b63fdcbeeafc)
(cherry picked from commit 302c36ffdce4cba247f084737a539022d56d8f6d)
 2018-06-04 07:04:32 -07:00
Xiao Chen
3ee5fd16f4 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by liaoyuxiangqin and Zsolt Venczel. 
(cherry picked from commit 9efb4b7db00d79aded52997ec89a1be94ecdd268)
 2018-06-03 22:37:36 -07:00
Inigo Goiri
77d98ae1e6 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit a804b7c9d2986556c91c1741c1ae7ac2b9c579f3)
 2018-06-02 20:26:38 -07:00
Inigo Goiri
a274d21ea3 HDFS-13651. TestReencryptionHandler fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit e308ac88d399e9bed3581ff4b1450aa2070dcedf)
 2018-06-02 20:13:55 -07:00
Inigo Goiri
d40c49257a HDFS-13637. RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE. Contributed by CR Hota. 
(cherry picked from commit e11d67404945d80db2eb8a99453606419dbdc938)
 2018-06-01 17:00:12 -07:00
Daniel Templeton
f13e01fdfe HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2 
(Contributed by Haibo Yan via Daniel Templeton)

Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7
(cherry picked from commit cba319499822a2475c60c43ea71f8e78237e139f)
(cherry picked from commit 09fd1348e855302f6f238917a98997d935c373c8)
 2018-06-01 14:45:01 -07:00
Chao Sun
c6becec392 HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 
(cherry picked from commit ff013d2c952272f3176dcf624251b05d610503b5)
 2018-05-31 17:42:41 -07:00
Konstantin V Shvachko
3d7e345823 HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 15:00:58 -07:00
Inigo Goiri
33c4f60b06 HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 
(cherry picked from commit e1102865e6e16babaff9346212c824cb1f47c31c)
 2018-05-30 10:07:16 -07:00
Yongjun Zhang
f0de11ba98 Preparing for 3.0.4 development 2018-05-29 23:40:26 -07:00
Kihwal Lee
65e55097da Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 2018-05-29 14:30:29 -05:00
Jason Lowe
d5708bbcdc YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:15:07 -05:00
Inigo Goiri
1f594f31dd HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 3c75f8e4933221fa60a87e86a3db5e4727530b6f)
 2018-05-29 09:12:08 -07:00
Inigo Goiri
595b44e2dc HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu. 2018-05-28 16:47:43 -07:00