Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,597 Commits 326 Branches 371 Tags
Commit Graph

17597 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Inigo Goiri
500057c40c YARN-8359. Exclude containermanager.linux test classes on Windows. Contributed by Jason Lowe. 
(cherry picked from commit 3492a1db2c0654ce5375360caa74a34f928f23be)
 2018-06-07 17:10:38 -07:00
Brahma Reddy Battula
a45e9812d6 HDFS-12950. [oiv] ls will fail in secure cluster. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 9654dd1f472052c4bb4a48a7412149c2e4859a10)
(cherry picked from commit 630f43faf7a1d4aaaefd860e3dcf25c288ca1d2f)
 2018-06-07 09:18:20 -07:00
Wei-Chiu Chuang
8e423a7779 HDFS-13659. Add more test coverage for contentSummary for snapshottable path. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit e39b113db0f2e4bcf93f873801be770e472601da)
(cherry picked from commit b2e919dc1a8688bb6499f7334ad67c49dcc1bc93)
 2018-06-07 08:32:53 -07:00
Xiao Chen
3843abe482 HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 
(cherry picked from commit ba4011d64fadef3bee5920ccedbcdac01794cc23)
 2018-06-05 21:19:19 -07:00
Inigo Goiri
49cfb11d8c MAPREDUCE-7103. Fix TestHistoryViewerPrinter on windows due to a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit 0afc036deb35df7e86ede3dcebc430c8f05ed368)
 2018-06-05 12:25:59 -07:00
Akira Ajisaka
cc735146dc
HDFS-13545. "guarded" is misspelled as "gaurded" in FSPermissionChecker.java. Contributed by Jianchao Jia. 
(cherry picked from commit 0e3c31579d1c733decfb9b00721bdc7b86be60f2)
 2018-06-05 17:55:33 +09:00
Inigo Goiri
e794ff2dcb HDFS-13650. Randomize baseDir for MiniDFSCluster in TestDFSStripedInputStream and TestDFSStripedInputStreamWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 8d31ddcfeb5c2e97692dd386e5bd62f433b44f8e)
 2018-06-04 18:29:12 -07:00
Inigo Goiri
076dc95c76 HDFS-13649. Randomize baseDir for MiniDFSCluster in TestReconstructStripedFile and TestReconstructStripedFileWithRandomECPolicy. Contributed by Anbang Hu. 
(cherry picked from commit 16316b60112842cf8a328b79c89297f6a8b30e7b)
 2018-06-04 18:22:35 -07:00
Inigo Goiri
2e5f849ea5 HDFS-13652. Randomize baseDir for MiniDFSCluster in TestBlockScanner. Contributed by Anbang Hu. 
(cherry picked from commit a28848104f2a067669f9b1706cd0da05ba71bcb7)
 2018-06-04 18:13:50 -07:00
Bharat Viswanadham
f1e1c068c0 HADOOP-15137. ClassNotFoundException: org.apache.hadoop.yarn.server.api.DistributedSchedulingAMProtocol when using hadoop-client-minicluster. Contributed by Bharat Viswanadham 
(cherry picked from commit f30f2dc4085c51b6b6850430213f47b97a763b7f)
 2018-06-04 16:51:22 -07:00
Robert Kanter
d5e6d0d5f4 YARN-4677. RMNodeResourceUpdateEvent update from scheduler can lead to race condition (wilfreds and gphillips via rkanter) 
(cherry picked from commit 0cd145a44390bc1a01113dce4be4e629637c3e8a)
 2018-06-04 15:42:46 -07:00
Inigo Goiri
9fe265f44e MAPREDUCE-7105. Fix TestNativeCollectorOnlyHandler.testOnCall on Windows because of the path format. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit dad1bb868f7a28a486fd500f102c8c174e6dd272)
 2018-06-04 12:57:11 -07:00
Miklos Szegedi
a3afd69051 YARN-8382. cgroup file leak in NM. Contributed by Hu Ziqian. 
(cherry picked from commit 925fdf761a513130e23c10575c7328c8681cff1d)
 2018-06-04 11:07:08 -07:00
Rushabh Shah
4ecb7aaf77 HDFS-13281 Namenode#createFile should be /.reserved/raw/ aware.. Contributed by Rushabh S Shah 
(cherry picked from commit e2289c8d1496a5eff88e6bcb8776a11d45371ffc)
 2018-06-04 09:58:38 -05:00
Wei-Chiu Chuang
63fc8f82cb HDFS-13155. BlockPlacementPolicyDefault.chooseTargetInOrder Not Checking Return Value for NULL. Contributed by Zsolt Venczel. 
(cherry picked from commit bccdfeee0aaef9cb98d09ee39909b63fdcbeeafc)
(cherry picked from commit 302c36ffdce4cba247f084737a539022d56d8f6d)
 2018-06-04 07:04:32 -07:00
Xiao Chen
3ee5fd16f4 HDFS-13339. Volume reference can't be released and may lead to deadlock when DataXceiver does a check volume. Contributed by liaoyuxiangqin and Zsolt Venczel. 
(cherry picked from commit 9efb4b7db00d79aded52997ec89a1be94ecdd268)
 2018-06-03 22:37:36 -07:00
Inigo Goiri
77d98ae1e6 MAPREDUCE-7102. Fix TestJavaSerialization for Windows due a mismatch line separator. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit a804b7c9d2986556c91c1741c1ae7ac2b9c579f3)
 2018-06-02 20:26:38 -07:00
Inigo Goiri
a274d21ea3 HDFS-13651. TestReencryptionHandler fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit e308ac88d399e9bed3581ff4b1450aa2070dcedf)
 2018-06-02 20:13:55 -07:00
Inigo Goiri
d40c49257a HDFS-13637. RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE. Contributed by CR Hota. 
(cherry picked from commit e11d67404945d80db2eb8a99453606419dbdc938)
 2018-06-01 17:00:12 -07:00
Daniel Templeton
f13e01fdfe HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2 
(Contributed by Haibo Yan via Daniel Templeton)

Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7
(cherry picked from commit cba319499822a2475c60c43ea71f8e78237e139f)
(cherry picked from commit 09fd1348e855302f6f238917a98997d935c373c8)
 2018-06-01 14:45:01 -07:00
Chao Sun
c6becec392 HDFS-13602. Add checkOperation(WRITE) checks in FSNamesystem. Contributed by Chao Sun. 
(cherry picked from commit ff013d2c952272f3176dcf624251b05d610503b5)
 2018-05-31 17:42:41 -07:00
Konstantin V Shvachko
3d7e345823 HDFS-12978. Fine-grained locking while consuming journal stream. Contributed by Konstantin Shvachko. 2018-05-31 15:00:58 -07:00
Inigo Goiri
33c4f60b06 HDFS-13632. Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA. Contributed by Anbang Hu. 
(cherry picked from commit e1102865e6e16babaff9346212c824cb1f47c31c)
 2018-05-30 10:07:16 -07:00
Yongjun Zhang
f0de11ba98 Preparing for 3.0.4 development 2018-05-29 23:40:26 -07:00
Kihwal Lee
65e55097da Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 2018-05-29 14:30:29 -05:00
Jason Lowe
d5708bbcdc YARN-8338. TimelineService V1.5 doesn't come up after HADOOP-15406. Contributed by Vinod Kumar Vavilapalli 
(cherry picked from commit 31ab960f4f931df273481927b897388895d803ba)
 2018-05-29 11:15:07 -05:00
Inigo Goiri
1f594f31dd HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 3c75f8e4933221fa60a87e86a3db5e4727530b6f)
 2018-05-29 09:12:08 -07:00
Inigo Goiri
595b44e2dc HADOOP-15498. TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows. Contributed by Anbang Hu. 2018-05-28 16:47:43 -07:00
Inigo Goiri
128cfa101a HDFS-13591. TestDFSShell#testSetrepLow fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 9dbf4f01665d5480a70395a24519cbab5d4db0c5)
 2018-05-28 16:35:32 -07:00
Sunil G
521ada1a11 YARN-4781. Support intra-queue preemption for fairness ordering policy. Contributed by Eric Payne. 
(cherry picked from commit 7c343669baf660df3b70d58987d6e68aec54d6fa)
 2018-05-28 16:34:29 +05:30
Karthik Palanisamy
a2cc099ddf
HADOOP-15449. Increase default timeout of ZK session to avoid frequent NameNode failover 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 61df174e8b3d582183306cabfa2347c8b96322ff)
 2018-05-28 19:42:26 +09:00
Inigo Goiri
1c1553670d HDFS-13620. Randomize the test directory path for TestHDFSFileSystemContract. Contributed by Anbang Hu. 
(cherry picked from commit 8605a38514b4f7a2a549c7ecf8e1421e61bb4d67)
 2018-05-25 19:46:16 -07:00
Inigo Goiri
ee31d9d0dc HDFS-13619. TestAuditLoggerWithCommands fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 13d25289076b39daf481fb1ee15939dbfe4a6b23)
 2018-05-25 13:33:30 -07:00
Xiao Chen
c2c2c57e21 HADOOP-15473. Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997. Contributed by Gabor Bota. 
(cherry picked from commit 02322de3f95ba78a22c057037ef61aa3ab1d3824)
 2018-05-25 09:42:01 -07:00
Inigo Goiri
3b0da097a4 HDFS-13618. Fix TestDataNodeFaultInjector test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit a8e9ee3fdf3aa1b889dfdd3bade91ba754ed9ee1)
 2018-05-25 09:11:34 -07:00
Miklos Szegedi
b0f7b0dfd5 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-24 15:36:35 -07:00
Andrew Wang
731abae557 HDFS-13611. Unsafe use of Text as a ConcurrentHashMap key in PBHelperClient. 
(cherry picked from commit c9b63deb533274ca8ef4939f6cd13f728a067f7b)
 2018-05-24 09:56:40 -07:00
Rohith Sharma K S
4711980d77 YARN-8346. Upgrading to 3.1 kills running containers with error 'Opportunistic container queue is full'. Contributed by Jason Lowe. 
(cherry picked from commit 4cc0c9b0baa93f5a1c0623eee353874e858a7caa)
 2018-05-24 12:26:25 +05:30
Inigo Goiri
f7274ca54a YARN-8327. Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit f09dc73001fd5f3319765fa997f4b0ca9e8f2aff)
 2018-05-23 16:00:47 -07:00
Arpit Agarwal
0e7ea77354 HADOOP-15486. Make NetworkTopology#netLock fair. Contributed by Nanda kumar. 2018-05-23 14:25:08 -07:00
Inigo Goiri
7c5a5f31dc YARN-8344. Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit e99e5bf104e9664bc1b43a2639d87355d47a77e2)
 2018-05-23 14:16:37 -07:00
Inigo Goiri
506f61e186 HDFS-13493. Reduce the HttpServer2 thread count on DataNodes. Contributed by Erik Krogen. 
(cherry picked from commit cddbbe5f690e4617413f6e986adc6fa900629f03)
 2018-05-23 12:13:06 -07:00
Inigo Goiri
c5f29d62eb HDFS-13587. TestQuorumJournalManager fails on Windows. Contributed by Anbang Hu. 
(cherry picked from commit c13dea87d9de7a9872fc8b0c939b41b1666a61e5)
 2018-05-23 11:37:27 -07:00
Inigo Goiri
8d8ef081a2 HDFS-13588. Fix TestFsDatasetImpl test failures on Windows. Contributed by Xiao Liang. 
(cherry picked from commit c0c9b7a8ef2618b7641a0452d9277abd26815de2)
 2018-05-23 09:47:32 -07:00
Sammi Chen
6db710b9d8 HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 
(cherry picked from commit 34e8b9f9a86fb03156861482643fba11bdee1dd4)
(cherry picked from commit 7d71b3a1cc0f2572d8643bc2faeb878dfe028b8b)
 2018-05-23 19:20:32 +08:00
Akira Ajisaka
bd98d4e77c
Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka. 2018-05-23 17:19:18 +09:00
Andrew Wang
5b3a6e3678 HDFS-13601. Optimize ByteString conversions in PBHelper. 
(cherry picked from commit 1d2640b6132e8308c07476badd2d1482be68a298)
(cherry picked from commit d47c0fc39a48dace6c7a2332ee041885cc4bb89c)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/dev-support/findbugsExcludeFile.xml
 2018-05-22 23:57:05 -07:00
Kihwal Lee
96313e03c1 HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal. 
(cherry picked from commit bcc8e76badc1341a6cf995c8e44fa5e422158de8)
 2018-05-22 11:23:55 -05:00
Wangda Tan
a5a9c8cf0f YARN-8232. RMContainer lost queue name when RM HA happens. (Hu Ziqian via wangda) 
Change-Id: Ia21e1da6871570c993bbedde76ce32929e95970f
(cherry picked from commit 6b96a73bb0f0ad1c877a062b19091e3e15a33ec4)
 2018-05-22 10:34:15 -05:00
Eric E Payne
1b6c662546 YARN-8179: Preemption does not happen due to natural_termination_factor when DRF is used. Contributed by Kyungwan Nam. 
(cherry picked from commit 0b4c44bdeef62945b592d5761666ad026b629c0b)
 2018-05-21 20:31:39 +00:00