Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,376 Commits 326 Branches 371 Tags
Commit Graph

13376 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Jason Lowe
f5a9287bb1 YARN-6501. FSSchedulerNode.java fails to compile with JDK7. Contributed by John Zhuge 2017-04-20 09:52:44 -05:00
John Zhuge
6865746ea4 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 2017-04-19 22:22:36 -07:00
John Zhuge
5046a45507 HDFS-11685. TestDistributedFileSystem.java fails to compile. Contributed by John Zhuge. 2017-04-19 22:10:18 -07:00
Andrew Wang
efb790b0d6 HDFS-11660. TestFsDatasetCache#testPageRounder fails intermittently with AssertionError. 
(cherry picked from commit 74a723852d9ef265d0fd46334f2f885bfb247a3a)
 2017-04-19 18:10:16 -07:00
John Zhuge
ef74a5661b Revert "HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge." 
This reverts commit 3a39dd5391527623a70296410f4754d8d235310a.
 2017-04-19 13:33:38 -07:00
John Zhuge
3a39dd5391 HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge. 
(cherry picked from commit 0344bea3fd4031622edd828a610c9fdc23c53d26)
 2017-04-19 12:04:03 -07:00
Yiqun Lin
e278eb3ab5 HDFS-11316. TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 724bb8635144dd16fc18cf95a78db6c3d5d1c468)
 2017-04-19 12:15:52 -05:00
Daniel Templeton
5225ae50b7 HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via Daniel Templeton) 
(cherry picked from commit d4483a2fe47ccc589e71c6fe5501878e6142c26f)
 2017-04-19 09:42:30 -07:00
Steve Loughran
1ae6be46b4
HADOOP-14321. explicitly exclude s3a root dir ITests from parallel runs. 
Contributed by Steve Loughran
 2017-04-19 10:19:43 +01:00
Arpit Agarwal
5c092097b6 HDFS-11603. Improve slow mirror/disk warnings in BlockReceiver. 2017-04-18 16:20:43 -07:00
Arpit Agarwal
6534bc0a01 HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. Contributed by Hanisha Koneru. 
This is an empty commit with the correct commit message for HDFS-11511.
The actual fix was made via commit ID f16bb799573bea6da2535ee863ed15ca271ef504.
 2017-04-18 15:35:17 -07:00
Arpit Agarwal
fc358b89b8 HDFS-11511. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 
This is an empty commit with the correct commit message for HDFS-11511.
The fix was previously made via commit ID fbe86471cf6d6397a89e473eb328d86a66c513e2.
 2017-04-18 15:34:46 -07:00
Kengo Seki
6cfceee865
HADOOP-14315. Python example in the rack awareness document doesn't work due to bad indentation 
This closes #214

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 654372db859656a2201ae9f9f7c374c6564ea34d)
 2017-04-18 17:15:35 +09:00
Akira Ajisaka
a17bd70892
HADOOP-14318. Remove non-existent setfattr command option from FileSystemShell.md. Contributed by Doris Gu. 
(cherry picked from commit 84a8848aaecd56b4cc85185924c2e33674165e01)
 2017-04-18 16:45:14 +09:00
Konstantin V Shvachko
c3840bde54 HDFS-11634. Optimize BlockIterator when interating starts in the middle. Contributed by Konstantin V Shvachko. 
(cherry picked from commit 8dfcd95d580bb090af7f40af0a57061518c18c8c)
 2017-04-17 17:06:18 -07:00
Zhe Zhang
051ab77806 HDFS-11615. FSNamesystemLock metrics can be inaccurate due to millisecond precision. Contributed by Erik Krogen. 
(cherry picked from commit ad49098eb324e238d97db68d7239ed2c4d84afa0)
 2017-04-17 16:28:58 -07:00
Junping Du
9289f4527d YARN-6304. Skip rm.transitionToActive call to RM if RM is already active. Contributed by Rohith Sharma K S. 
(cherry picked from commit 6ed9d362242d36bfae4083bf3b1c2eb32fbb2f72)
 2017-04-17 15:52:02 -07:00
Karthik Kambatla
6f5912c106 YARN-6432. FairScheduler: Reserve preempted resources for corresponding applications. (Miklos Szegedi via kasha) 2017-04-17 14:34:28 -07:00
Wangda Tan
6ade9e6ce6 YARN-6406. Remove SchedulerRequestKeys when no more pending ResourceRequest. (Arun Suresh via wangda) 
(cherry picked from commit 87e2ef8c985bb72a916477e8783359f2859f7890)
 2017-04-15 21:34:29 -07:00
Haibo Chen
d4f553d42f MAPREDUCE-6673. Add a test example job that grows in memory usage over time (Karthik Kambatla via Haibo Chen) 
Change-Id: Iccfc8c67c38c526cc61726d87bfcbcf69ac36fea
(cherry picked from commit 25ac44709b4bbed78b607ea48021237b64e01b9f)
 2017-04-14 17:37:37 -07:00
Arun Suresh
8515d35bd5 HADOOP-14311. Add python2.7-dev to Dockerfile (Allen Wittenauer via asuresh) 
(cherry picked from commit 0ac17dc644c0429ff8a6f743bf9d3ecdd7458e58)
 2017-04-14 16:58:00 -07:00
Jason Lowe
1699b20526 YARN-6480. Timeout is too aggressive for TestAMRestart.testPreemptedAMRestartOnRMRestart. Contributed by Eric Badger 
(cherry picked from commit 416880550214e58f2284a045ad0c96ba4aa78ea8)
 2017-04-14 17:17:46 -05:00
Karthik Kambatla
53da65ef8b YARN-6433. Only accessible cgroup mount directories should be selected for a controller. (Miklos Szegedi via kasha) 
(cherry picked from commit 8a1d7480f73906d8e0342690ec6c6b008d6de21b)
 2017-04-14 15:07:54 -07:00
Yongjun Zhang
c411adac32 HADOOP-11794. Enable distcp to copy blocks in parallel. Contributed by Yongjun Zhang, Wei-Chiu Chuang, Xiao Chen, Rosie Li. 2017-04-14 10:14:02 -07:00
Eric Payne
157a9f3104 YARN-5994. TestCapacityScheduler.testAMLimitUsage fails intermittently. Contributed by Eric Badger 
(cherry-picked from commit a41f8dd58e27d8835fbb64eeaba5d7416df0499d)
 2017-04-14 11:04:18 -05:00
Wangda Tan
5756256280 YARN-6216. Unify Container Resizing code paths with Container Updates making it scheduler agnostic. (Arun Suresh via wangda) 
(cherry picked from commit eac6b4c35c50e555c2f1b5f913bb2c4d839f1ff4)
 2017-04-13 13:16:31 -07:00
Wangda Tan
408d23477f YARN-6040. Introduce api independent PendingAsk to replace usage of ResourceRequest within Scheduler classes. (wangda) 2017-04-13 11:11:57 -07:00
Hanisha Koneru
107f685f64 HDFS-11630. TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds. Contributed by Hanisha Koneru. 2017-04-12 17:11:36 -07:00
Hanisha Koneru
ed8bb51f59 HDFS-11560. Expose slow disks via NameNode JMX. Contributed by Hanisha Koneru. 2017-04-12 16:59:37 -07:00
Mingliang Liu
03aa142e35 HADOOP-14274. Azure: Simplify Ranger-WASB policy model. Contributed by Sivaguru Sankaridurg 
(cherry picked from commit 0cab57223e3f54be17a5f27cefdb6d1da1b073e5)
 2017-04-12 16:09:20 -07:00
Mingliang Liu
508389ee54 HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu 
(cherry picked from commit 507845074970b90330db86bc1ff0124def550bb5)
 2017-04-12 14:30:54 -07:00
Junping Du
d77cada182 YARN-3760. FSDataOutputStream leak in AggregatedLogFormat.LogWriter.close(). Contributed by Haibo Chen. 
(cherry picked from commit c26ccf1adb3a72df3f68e1150b86b813c691203a)
(cherry picked from commit 9d9087a67ad599e678948d5c1036e51cb8cf5e3a)
 2017-04-12 13:54:58 -07:00
Kihwal Lee
8ed230c805 HDFS-11648. Lazy construct the IIP pathname. Contributed by Daryn Sharp. 2017-04-12 13:28:26 -05:00
Daniel Templeton
f20aa38a1d HADOOP-14246. Authentication Tokens should use SecureRandom instead of Random and 256 bit secrets 
(Conttributed by Robert Konter via Daniel Templeton)

(cherry picked from commit 4dd6206547de8f694532579e37ba8103bafaeb12)
 2017-04-12 11:19:53 -07:00
Chris Nauroth
1bfb382296 HADOOP-14301. Deprecate SharedInstanceProfileCredentialsProvider in branch-2. Contributed by Mingliang Liu. 2017-04-12 10:07:00 -07:00
Eric Payne
607295d21f YARN-6450. TestContainerManagerWithLCE requires override for each new test added to ContainerManagerTest? Contributed by Jason Lowe. 
(cherry picked from commit a16ab2be91e29f0aa4ad3488d35977cb5d1e1f5c)
 2017-04-12 10:26:31 -05:00
Chris Nauroth
d5e2bd4096 HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
 2017-04-11 21:18:47 -07:00
Mingliang Liu
f1a85b2be3 HDFS-11558. BPServiceActor thread name is too long. Contributed by Xiaobing Zhou 2017-04-11 18:09:43 -07:00
Hanisha Koneru
f16bb79957 HDFS-11551. Support Timeout when checking single disk. Contributed by Hanisha Koneru. 2017-04-11 16:54:28 -07:00
Steve Loughran
5efb3ade1f HADOOP-14066 VersionInfo should be marked as public API 
(cherry picked from commit 2ed65540a3fe73db5d2dd0d25e658ccf13c9188a)
 2017-04-11 15:06:40 -07:00
Wangda Tan
2f08c86b7a YARN-6439. Fix ReservationSystem creation of default ReservationQueue. (Carlo Curino via wangda) 
(cherry picked from commit 4d4ad0ebb76198c90c5317f19ce85a6696132c8f)
 2017-04-11 14:59:34 -07:00
Naganarasimha
b04c09163b YARN-6463. Correct spelling mistake in FileSystemRMStateStore. Contributed by Yeliang Cang. 
(cherry picked from commit 7d873c465b815816e9955cf40f4e141dd2e7d82b)
 2017-04-11 22:46:09 +05:30
Jason Lowe
b29fb55798 YARN-6461. TestRMAdminCLI has very low test timeouts. Contributed by Eric Badger 
(cherry picked from commit 90d97372ed11aaa95d338db8583f578da159834a)
 2017-04-11 09:17:42 -05:00
Jason Lowe
0cb22a0f15 YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu 
(cherry picked from commit 0e065f2edead41f6f894c9d70def389af1c0f239)
 2017-04-11 08:45:16 -05:00
Akira Ajisaka
121fcb74f3
HADOOP-13545. Update HSQLDB to 2.3.4. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit aabf08dd0707995e471d41ec4158c6af597c55dd)
 2017-04-11 14:39:31 +09:00
Konstantinos Karanasos
9b3ba25980 YARN-6344. Add parameter for rack locality delay in CapacityScheduler. (kkaranasos) 
(cherry picked from commit 3587b46164514e6751d7cebb09c8ab5e988853b4)
 2017-04-10 17:22:29 -07:00
Daniel Templeton
c1ae753de1 YARN-6372. Add default value for NM disk validator (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit e486569197e10bf48731d9e7ef5c031987c26acc)
 2017-04-10 14:57:19 -07:00
Andrew Wang
dab1deb9a0 HADOOP-14293. Initialize FakeTimer with a less trivial value. 
(cherry picked from commit be144117a885cb39bc192279c96cbe3790dc77b1)
 2017-04-10 11:37:31 -07:00
Daniel Templeton
2dae9299ef YARN-6343. Docker docs MR example is broken (Contributed by Prashant Jha via Daniel Templeton) 
(cherry picked from commit 14a39907c636420800210b984f2f78b639c8382c)
 2017-04-10 10:03:06 -07:00
Akira Ajisaka
21af5be51a
HADOOP-14290. Update SLF4J from 1.7.10 to 1.7.25. 
(cherry picked from commit 5d38504071d72745c42588c7a6e76bcffb13832c)
 2017-04-09 01:48:03 +09:00