11451 Commits

Author SHA1 Message Date
Xiaoyu Yao
ac510e5cd0 HDFS-10676. Add namenode metric to measure time spent in generating EDEKs. Contributed by Hanisha Koneru. 2016-07-28 16:16:44 -07:00
Jason Lowe
95fb2fe394 YARN-5438. TimelineClientImpl leaking FileSystem Instances causing Long running services like HiverServer2 daemon going OOM. Contributed by Rohith Sharma K S
(cherry picked from commit a1890c32c52fed69ec09efad0fccf49ed8c2e21e)
2016-07-28 21:49:21 +00:00
Junping Du
08358c8d4e YARN-5432. Lock already held by another process while LevelDB cache store creation for dag. Contributed by Li Lu.
(cherry picked from commit 7f3c306e2ed9e865171324898decb9b587d90e73)
(cherry picked from commit 932bf79244e92c93ba673ca77691f3997da6719b)
2016-07-28 06:56:00 -07:00
Akira Ajisaka
960caf624d HDFS-10696. TestHDFSCLI fails. Contributed by Kai Sasaki.
(cherry picked from commit 414fbfab41470923eb82a21628709e51fd3a3f6e)
(cherry picked from commit ab49c2b39943eb5a3510d22c09ae5630232bdddc)
2016-07-28 16:59:49 +09:00
Yongjun Zhang
71c49b1786 HDFS-10667. Report more accurate info about data corruption location. (Yuanbo Liu via Yongjun Zhang)
(cherry picked from commit eb7ff0c9927131f4a797148b970a95a1abf7d847)
2016-07-27 15:49:56 -07:00
Brahma Reddy Battula
5dc29ab0f2 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
(cherry picked from commit 096d3fa25cbbdda4fa2ad30ea5a76f33e2631c7d)
2016-07-27 10:31:29 +05:30
Chris Nauroth
244285488d HADOOP-13422. ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process. Contributed by Sergey Shelukhin.
(cherry picked from commit 255ea45e50e102505ee61eb0ba45ea93035abe3c)
(cherry picked from commit df2ed6b2c4f5b42a57e211e58d4a7350c9ac6fb8)
2016-07-26 15:33:42 -07:00
Arpit Agarwal
45a6ee97b2 HDFS-10642. TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently. (Contributed by Mingliang Liu) 2016-07-26 12:45:06 -07:00
Wei-Chiu Chuang
53bf632702 HDFS-9937. Update dfsadmin command line help and HdfsQuotaAdminGuide. Contributed by Kai Sasaki.
(cherry picked from commit 25f0a9b3ab136d44f70eb2805598e2df8325612a)
2016-07-26 08:31:00 -07:00
Steve Loughran
7921ad949f Revert "HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu."
This reverts commit 32468b1ba5fe5e00e92656a8bf1591114383d7b3.
2016-07-26 14:31:02 +01:00
Masatake Iwasaki
6b5d546014 HDFS-10671. Fix typo in HdfsRollingUpgrade.md. Contributed by Yiqun Lin.
(cherry picked from commit 59466b8c180716dda7aa670728580a88e54eb4d2)
2016-07-26 19:52:23 +09:00
Brahma Reddy Battula
32468b1ba5 HDFS-10668. Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount. Contributed by Mingliang Liu.
(cherry picked from commit 7cac7655fd84ac394250705b31e3927fe548e34a)
2016-07-26 11:00:11 +05:30
Akira Ajisaka
e377e94f64 Revert "HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg."
This reverts commit 91f47ab5f97989872339a934ea351bf68a077553.
2016-07-26 11:06:10 +09:00
Vinitha Reddy Gankidi
9ab153d7ae HDFS-10301. Interleaving processing of storages from repeated block reports causes false zombie storage detection, removes valid blocks. Contributed by Vinitha Gankidi. 2016-07-25 18:59:42 -07:00
Jing Zhao
62800dc8b6 HDFS-10688. BPServiceActor may run into a tight loop for sending block report when hitting IOException. Contributed by Chen Liang.
(cherry picked from commit 0cde9e12a7175e4d8bc4ccd5c36055b280d1fbd6)
(cherry picked from commit 30c772fc595b71d3b26a13bd0473570a84b5e807)
2016-07-25 18:44:06 -07:00
Jason Lowe
e098a732ef MAPREDUCE-6744. Increase timeout on TestDFSIO tests. Contributed by Eric Badger
(cherry picked from commit 703fdf86c6e42d6d555090f670ca5cbd1ef48bf9)
2016-07-25 21:56:33 +00:00
Arpit Agarwal
00cbd3136e HDFS-10660. Expose storage policy apis via HDFSAdmin interface. (Contributed by Rakesh R) 2016-07-25 09:51:59 -07:00
Jing Zhao
54be36a6d6 HDFS-9894. Add unsetStoragePolicy API to FileContext/AbstractFileSystem and derivatives. Contributed by Xiaobing Zhou.
(cherry picked from commit 7149cdb3c2d9dd390cd8668883cbe5db94090e0a)
2016-07-25 09:51:59 -07:00
Arpit Agarwal
bbf711c030 HDFS-9534. Add CLI command to clear storage policy from a path. (Contributed by Xiaobing Zhou) 2016-07-25 09:51:59 -07:00
Steve Loughran
71c8bdd50c HADOOP-13188 S3A file-create should throw error rather than overwrite directories. Contributed by Steve Loughran 2016-07-25 15:48:49 +01:00
Steve Loughran
5f9bffbb66 HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and mkdirs(). Contributed by Rajesh Balamohan 2016-07-25 14:43:45 +01:00
Steve Loughran
7c46303de3 HADOOP-13389 TestS3ATemporaryCredentials.testSTS error when using IAM credentials. Contributed by Steven K. Wong 2016-07-23 17:58:46 +01:00
Chris Nauroth
0456069aca HADOOP-13207. Specify FileSystem listStatus, listFiles and RemoteIterator. Contributed by Steve Loughran.
(cherry picked from commit 5fda66e4a38c7597bc0bc9c94748c8491b7f3750)
(cherry picked from commit e6ba69b4db0463b99cfa594394eca0480478e965)
2016-07-22 15:39:12 -07:00
Xiao Chen
8559442ed1 HDFS-10225. DataNode hot swap drives should disallow storage type changes. Contributed by Lei (Eddy) Xu.
(cherry picked from commit 132deb4cacc413a85a6af2f390ec79a76c91961c)
2016-07-22 15:04:21 -07:00
mattf-horton
c96cb3fd48 HADOOP-13382. Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, hadoop-openstack. Contributed by Matt Foley. 2016-07-21 16:01:05 -07:00
Chris Nauroth
af3c860bb1 HADOOP-13240. TestAclCommands.testSetfaclValidations fail. Contributed by John Zhuge.
(cherry picked from commit 43cf6b101dacd96bacfd199826b717f6946109af)
(cherry picked from commit 382bcf21e7f7f752ba901f856abf080e8700e890)
2016-07-21 14:12:47 -07:00
Jakob Homan
9442e1d48e HADOOP-12009. Clarify FileSystem.listStatus() sorting order & fix FileSystemContractBaseTest:testListStatus. backport to 2.8 2016-07-21 21:50:25 +01:00
Akira Ajisaka
3dcbd2df56 MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently in slow VM testbed. Contributed by Junping Du.
(cherry picked from commit ecff7d03595b45d7a5d6aed73b9931fef85a56ee)
(cherry picked from commit f4d94f2246da3a015a105cb7f55ec1ebc451e0e7)
2016-07-21 11:38:26 -07:00
Jing Zhao
6a7fe835f1 HDFS-10653. Optimize conversion from path string to components. Contributed by Daryn Sharp.
(cherry picked from commit bd3dcf46e263b6e6aa3fca6a5d9936cc49e3280f)
2016-07-21 11:29:27 -07:00
Akira Ajisaka
2f14d585c4 HDFS-10287. MiniDFSCluster should implement AutoCloseable. Contributed by Andras Bokor.
(cherry picked from commit fcde6940e0cbdedb1105007e4857137ecdfa1284)
(cherry picked from commit 1222889f1a3b45df30c48a80d8c2bec7ad45ee43)
2016-07-21 11:28:46 -07:00
Rohith Sharma K S
7234fce25b YARN-5092. TestRMDelegationTokens fails intermittently. Contributed by Jason Lowe.
(cherry picked from commit 557a245d83c358211f51b3ab1950d707a70679d9)
2016-07-21 12:49:52 +05:30
Rohith Sharma K S
cdcd131d5f YARN-1126. Add validation of users input nodes-states options to nodes CLI. Contributed by Wei Yan.
(cherry picked from commit be34b2a8fd30a1a0e803ac8b2198c695600a9e46)
2016-07-21 12:25:14 +05:30
Akira Ajisaka
66f3056090 YARN-4883. Make consistent operation name in AdminService. Contributed by Kai Sasaki.
(cherry picked from commit c63afdbe14df7359841aa4b213252a8a275e8c4b)
(cherry picked from commit 2bf5c4eb1a181ad9d4b337388d9fff19773fa2c1)
2016-07-20 16:52:22 -07:00
Allen Wittenauer
35885e9540 HDFS-9852. hdfs dfs -setfacl error message is misleading (Wei-Chiu Chuang via aw)
(cherry picked from commit b3649adf6a1b2dc47566b4b0d652bd4e0a6a8056)
(cherry picked from commit 84c785036f5014d4e6066b344c2c235d1ea5028f)
2016-07-20 11:30:52 -07:00
Vinod Kumar Vavilapalli
8c5101ea6d YARN-5340. Fixed a race condition in RollingLevelDBTimelineStore that caused loss of Timeline events. Contributed by Li Lu.
(cherry picked from commit 1c9d2ab503ea5a3f16757351af9603041059b390)
2016-07-20 08:39:19 -07:00
Steve Loughran
c177823ebe HADOOP-13387 users always get told off for using S3 —even when not using it. Contributed by stevel. 2016-07-20 14:01:17 +01:00
Steve Loughran
0f1e02a298 HADOOP-13212 Provide an option to set the socket buffers in S3AFileSystem (Rajesh Balamohan) 2016-07-20 13:42:19 +01:00
Varun Vasudev
478a2bb185 YARN-5309. Fix SSLFactory truststore reloader thread leak in TimelineClientImpl. Contributed by Weiwei Yang. 2016-07-20 13:02:34 +05:30
Akira Ajisaka
f2508d78b2 HADOOP-12991. Conflicting default ports in DelegateToFileSystem. Contributed by Kai Sasaki.
(cherry picked from commit dc065dd64ca3e101b0c0a7bcc7d7a067b77d6c82)
(cherry picked from commit 8fe2c450b38ab51087eea867fd1dd960915a6fed)
2016-07-19 16:06:35 -07:00
Akira Ajisaka
91f47ab5f9 HDFS-10620. StringBuilder created and appended even if logging is disabled. Contributed by Staffan Friberg.
(cherry picked from commit f7dabe3addf3f6eb32ea9b8ec1354fb442ce4222)
(cherry picked from commit 2685e16f381532d53aea413ce1c0a14181ee7c89)
2016-07-19 15:19:02 -07:00
Akira Ajisaka
d8181a8cb8 HDFS-10603. Fix flaky tests in org.apache.hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot. Contributed by Yiqun Lin.
(cherry picked from commit fc570b55b9d1d1d57329eca223f443e2c86f34b7)
(cherry picked from commit c92601e9814a041800e466fabdcc3c44309364c4)
2016-07-19 13:52:17 -07:00
Akira Ajisaka
cf79ac5fa9 HADOOP-13202. Avoid possible overflow in org.apache.hadoop.util.bloom.BloomFilter#getNBytes. Contributed by Kai Sasaki.
(cherry picked from commit c2bcffb34ebe4399b523a44f69d23761648bd0c4)
(cherry picked from commit 09375baad1ba732b73701cdbd0ee77618ed6f687)
2016-07-18 11:50:53 -07:00
Tsuyoshi Ozawa
e3c4989d74 HADOOP-13298. Fix the leftover L&N files in hadoop-build-tools/src/main/resources/META-INF/. (ozawa)
(cherry picked from commit ea9f43781e5a9a4a6121dd3be106ced6b811754d)
2016-07-16 08:11:30 +09:00
Jitendra Pandey
a363277be5 HADOOP-13368. DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation. Contributed by Mingliang Liu. 2016-07-15 14:32:20 -07:00
Wangda Tan
343633a6e8 YARN-4484. Available Resource calculation for a queue is not correct when used with labels. (Sunil G via wangda)
(cherry picked from commit 24db9167f16ba643a186624b33a6b9b80020f476)
(cherry picked from commit e34e1aa4fe0a0826439227175fc3321f840dddd4)
2016-07-15 11:46:17 -07:00
Akira Ajisaka
83126ac90d HDFS-10628. Log HDFS Balancer exit message to its own log. Contributed by Jiayi Zhou.
(cherry picked from commit c48e9d608c1daf7826f2e738c0a67d8eab79d8de)
(cherry picked from commit 617149837531062e4e1ee9eeb50a324766b14c60)
2016-07-15 11:43:54 -07:00
Akira Ajisaka
bc21763da4 HADOOP-13289. Remove unused variables in TestFairCallQueue. Contributed by Ye Zhou.
(cherry picked from commit 13814832331ad691b898cd9f602b0a0a771d7928)
(cherry picked from commit 2231ef22f576d9a7d65276cf90efd0f45cb362f5)
2016-07-14 23:15:53 -07:00
Akira Ajisaka
6a04d9ab63 HADOOP-13351. TestDFSClientSocketSize buffer size tests are flaky. Contributed by Aaron Fabbri and Mingliang Liu.
(cherry picked from commit 5537c6b23430285ebee33c6d9b69d3ec1e9b17b1)
(cherry picked from commit c9f5348bbfca664c6ab4c13f40cae0a2c5754da1)
2016-07-14 14:42:55 -07:00
Akira Ajisaka
2f7857baea YARN-5159. Wrong Javadoc tag in MiniYarnCluster. Contributed by Andras Bokor.
(cherry picked from commit 6cf017558a3d06240b95d1b56c953591ece97c92)
(cherry picked from commit 66640777275bccfdff417670d84706e3d722d113)
2016-07-14 10:46:11 -07:00
Zhe Zhang
cbd885b6fa HADOOP-13290. Appropriate use of generics in FairCallQueue. Contributed by Jonathan Hung.
(cherry picked from commit 728bf7f6988e13f17f03d2e3a4037b7238c4fd45)
(cherry picked from commit b95f1af8a9aafb3e482c7b8825f2097f2ceb0e8e)
2016-07-13 16:47:01 -07:00