Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,111 Commits 326 Branches 371 Tags
Commit Graph

12111 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arpit Agarwal
012b266e5e HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang. 2016-09-10 18:04:00 -07:00
Arpit Agarwal
ad0ac6cced HDFS-10682. Replace FsDatasetImpl object lock with a separate lock object. (Contributed by Chen Liang) 2016-09-10 17:48:27 -07:00
Xiao Chen
bb6d866207 HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy. 2016-09-09 18:25:35 -07:00
Andrew Wang
f448ce2a89 HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions. 
(cherry picked from commit cba973f036a2c8fcc21a5826b8306247ec154c7b)
 2016-09-09 12:16:08 -07:00
Chris Douglas
806a8dea38 MAPREDUCE-6628. Potential memory leak in CryptoOutputStream. Contributed by Mariappan Asokan 
(cherry picked from commit 9f192cc5ac4a6145e2eeaecba0a754d31e601898)
 2016-09-09 11:14:30 -07:00
Steve Loughran
6a016faf59 HADOOP-13540 improve section on troubleshooting s3a auth problems. Contributed by Steve Loughran 2016-09-09 18:54:34 +01:00
Kihwal Lee
d234990208 HADOOP-10940. RPC client does no bounds checking of responses. Contributed by Daryn Sharp. 
(cherry picked from commit d4d076876a8d0002bd3a73491d8459d11cb4896c)
 2016-09-09 10:46:37 -05:00
Andrew Wang
72ea641468 HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao. 
(cherry picked from commit b07c266dca7f303c793b432a0738d593728cf2b3)
 2016-09-08 18:30:26 -07:00
Akira Ajisaka
c0756dfef6 HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin. 
(cherry picked from commit 20a20c2f6e1b3b8aa6a58a824ad3aadc349dc761)
 2016-09-09 01:35:36 +09:00
Akira Ajisaka
fe6dd66174 HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer. 
(cherry picked from commit 63f594892ecd4687e37a99790288e36eb278849f)
 2016-09-08 15:18:25 +09:00
Steve Loughran
5f8b6f0313 HADOOP-13541 explicitly declare the Joda time version S3A depends on. Contributed by Stevel Loughran 2016-09-07 12:24:15 +01:00
Karthik Kambatla
a064375e68 YARN-5616. Clean up WeightAdjuster. (Yufei Gu via kasha) 
(cherry picked from commit c0e492e50fa98d423c8a61c94d2d1f9553558b6d)
 2016-09-06 22:40:49 -07:00
Xiao Chen
d157733082 HADOOP-13558. UserGroupInformation created from a Subject incorrectly tries to renew the Kerberos ticket. Contributed by Xiao Chen. 
(cherry picked from commit 680be58aac03a9ffab6b07c8fde9602ddb9dc858)
 2016-09-06 20:54:38 -07:00
Arpit Agarwal
f9415b7a7e HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula) 2016-09-06 13:44:22 -07:00
Xiao Chen
3025992bd3 HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee. 
(cherry picked from commit f6c0b7543f612de756ff0c03e9a2c6e33b496a36)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirAppendOp.java
 2016-09-06 11:52:30 -07:00
Chris Nauroth
e28930a38b HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth. 
(cherry picked from commit d152557cf7f4d2288524c222fcbaf152bdc038b0)
 2016-09-06 09:40:41 -07:00
Kihwal Lee
614f9a62c4 HADOOP-13549. Eliminate intermediate buffer for server-side PB encoding. Contributed by Daryn Sharp. 
(cherry picked from commit 39d1b1d747b1e325792b897b3264272f32b756a9)
 2016-09-06 10:15:57 -05:00
Varun Vasudev
ec3a651b17 YARN-5576. Allow resource localization while container is running. Contributed by Jian He. 2016-09-06 20:07:54 +05:30
Rohith Sharma K S
de9f046598 YARN-5608. TestAMRMClient.setup() fails with ArrayOutOfBoundsException. Contributed by Daniel Templeton. 
(cherry picked from commit f9557127b3660e138cab8d9535a589677809384a)
 2016-09-06 14:13:41 +05:30
Karthik Kambatla
a5748eec7b YARN-5264. Store all queue-specific information in FSQueue. (Yufei Gu via kasha) 
(cherry picked from commit f6ea9be5473ab66798b0536317d2f32c5348eb57)
 2016-09-02 15:02:20 -07:00
Ray Chiang
4ee1729cb0 YARN-5549. AMLauncher#createAMContainerLaunchContext() should not log the command to be launched indiscriminately. (Daniel Templeton via rchiang) 2016-09-02 14:57:05 -07:00
Andrew Wang
40eab77562 HDFS-10833. Fix JSON errors in WebHDFS.md examples. 
(cherry picked from commit cbd909ce2a5ac1da258f756fa1f93e84dd20b926)
 2016-09-02 14:50:41 -07:00
Lei Xu
d6bfb2176c HDFS-10822. Log DataNodes in the write pipeline. John Zhuge via Lei Xu 
(cherry picked from commit 5a8c5064d1a1d596b1f5c385299a86ec6ab9ad1e)
 2016-09-02 11:02:38 -07:00
Kihwal Lee
28ea4122f0 HADOOP-13547. Optimize IPC client protobuf decoding. Contributed by Daryn Sharp. 2016-09-02 11:12:05 -05:00
Varun Vasudev
1222433729 YARN-5555. Scheduler UI: "% of Queue" is inaccurate if leaf queue is hierarchically nested. Contributed by Eric Payne. 
(cherry picked from commit 05f5c0f631680cffc36a79550c351620615445db)

 Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/common/fica/FiCaSchedulerApp.java
 2016-09-02 16:16:31 +05:30
Ray Chiang
c142048084 Revert "YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang)" 
This reverts commit 3b19bcb8f0f70606a8080bc46fa64cd34f6280d7.
 2016-09-01 15:10:53 -07:00
Karthik Kambatla
736dd1646d YARN-5566. Client-side NM graceful decom is not triggered when jobs finish. (Robert Kanter via kasha) 
(cherry picked from commit 74f4bae45597f4794e99e33309130ddff647b21f)
 2016-09-01 14:44:41 -07:00
Allen Wittenauer
b1466b8124 HDFS-9877. HDFS Namenode UI: Fix browsing directories that need to be encoded (Ravi Prakash via aw) 
(cherry picked from commit 15f018434c5b715729488fd0b03a11f1bc943470)
 2016-09-01 14:24:13 -07:00
Allen Wittenauer
b9eedc24a6 HDFS-7987. Allow files / directories to be moved (Ravi Prakash via aw) 
(cherry picked from commit d44f4745b4a186dd06dd6837a85ad90a237d7d97)
 2016-09-01 14:23:42 -07:00
Allen Wittenauer
743c0ebd67 HDFS-7767. Use the noredirect flag in WebHDFS to allow web browsers to upload files via the NN UI (Ravi Prakash via aw) 
(cherry picked from commit 99a771cd7a3f792a76ac89c406b82a983c059d28)
 2016-09-01 14:22:23 -07:00
Kihwal Lee
caef03d742 HADOOP-13465. Design Server.Call to be extensible for unified call queue. Contributed by Daryn Sharp. 
(cherry picked from commit 76cd81f4b656f0d40a4b2122e15f04ea53d8020b)
 2016-09-01 15:51:23 -05:00
Mingliang Liu
4d709be8c9 HADOOP-13375. o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky. (Contributed by Weiwei Yang) 
(cherry picked from commit dcd21d083ab2a66fc3ca3bfda03887461698b7b1)
 2016-09-01 11:14:25 -07:00
Varun Vasudev
34f9330651 YARN-5596. Fix failing unit test in TestDockerContainerRuntime. Contributed by Sidharta Seethana. 
(cherry picked from commit 08f55ccbb075b90a6e534f4ac0f9840f2e6890d5)
 2016-09-01 14:10:18 +05:30
Zhe Zhang
776eccd873 Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2. 
(cherry picked from commit 6c600360ca469d5fe0f017d681585db06c80c9cc)
 2016-08-31 23:44:54 -07:00
Zhe Zhang
f87e2f5d05 HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen. 
(cherry picked from commit 6f4b0d33ca339e3724623a1d23c101f8cfd3cdd5)
 2016-08-31 15:54:03 -07:00
Andrew Wang
72d90cbe32 HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator. 
(cherry picked from commit 85bab5fb572194fda38854f1f21c670925058009)
 2016-08-31 14:33:14 -07:00
Allen Wittenauer
72334a4be3 HDFS-7766. Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect (Ravi Prakash via aw) 
(cherry picked from commit 4b0f55b6ea1665e2118fd573f72a6fcd1fce20d6)
 2016-08-31 14:33:10 -07:00
Kihwal Lee
34042ccbb9 HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 01721dd88ee532d20eda841254437da4dfd69db5)
 2016-08-31 14:09:23 -05:00
Arun Suresh
b279f42d79 YARN-5221. Expose UpdateResourceRequest API to allow AM to request for change in container properties. (asuresh) 
(cherry picked from commit d6d9cff21b7b6141ed88359652cf22e8973c0661)
 2016-08-30 16:33:31 -07:00
Ming Ma
56c9a96a76 HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma. 
(cherry picked from commit 9dcbdbdb5a34d85910707f81ebc1bb1f81c99978)
 2016-08-30 14:09:18 -07:00
Wei-Chiu Chuang
abbd95b790 HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan. 
(cherry picked from commit c4ee6915a14e00342755d7cdcbf2d61518f306aa)
 2016-08-30 10:44:27 -07:00
Jason Lowe
f70f522e80 MAPREDUCE-4784. TestRecovery occasionally fails. Contributed by Haibo Chen 
(cherry picked from commit af508605a9edc126c170160291dbc2fe58b66dea)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRecovery.java
 2016-08-30 14:09:14 +00:00
Zhe Zhang
320be1b3c6 HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi. 
(cherry picked from commit 4bd45f54eedd449a98a90540698c6ceb47454fec)
 2016-08-29 23:42:46 -07:00
Xiao Chen
72863f0049 HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge. 
(cherry picked from commit 046f6bf0f944311fc70de3a96fc6ab8211cbbbd8)
 2016-08-29 17:48:49 -07:00
Ray Chiang
3b19bcb8f0 YARN-5567. Fix script exit code checking in NodeHealthScriptRunner#reportHealthStatus. (Yufei Gu via rchiang) 
(cherry picked from commit 05ede003868871addc30162e9707c3dc14ed6b7a)
 2016-08-29 15:56:30 -07:00
Zhe Zhang
3ad4f0823c YARN-5550. TestYarnCLI#testGetContainers should format according to CONTAINER_PATTERN. Contributed by Jonathan Hung. 
(cherry picked from commit ed6ff5cd789cab621fbfb88c8e8de4f215cd2961)
 2016-08-29 14:49:03 -07:00
Yongjun Zhang
4bed2a4b9c HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin. 
(cherry picked from commit 5d1609ddf275e4907bd224bf618e2aad4b262888)
 2016-08-29 14:08:36 -07:00
Mingliang Liu
d124af1a7c HADOOP-13559. Remove close() within try-with-resources in ChecksumFileSystem/ChecksumFs classes. (Contributed by Aaron Fabbri) 
(cherry picked from commit 6fcb04c1780ac3dca5b986f1bcd558fffccb3eb9)
 2016-08-29 13:05:07 -07:00
Jason Lowe
44d0af2611 MAPREDUCE-6768. TestRecovery.testSpeculative failed with NPE. Contributed by Haibo Chen 2016-08-29 19:57:38 +00:00
Mingliang Liu
875c14004d HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu) 
(cherry picked from commit 6742fb6e68d349055f985eb640d845e689d75384)
 2016-08-29 10:19:33 -07:00