Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,892 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

11892 Commits

Author SHA1 Message Date
Mingliang Liu 9b2a719031 HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang 
(cherry picked from commit b36aaa913c)
 2016-10-12 17:41:07 -07:00
Andrew Wang 9bde45d2fe HADOOP-13700. Remove unthrown IOException from TrashPolicy#initialize and #getInstance signatures. 
(cherry picked from commit 12d739a34b)
(cherry picked from commit f131d61ff8)
 2016-10-12 15:20:10 -07:00
Kihwal Lee 136c6f6f7d HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 
(cherry picked from commit 5305a392c3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
 2016-10-12 15:38:34 -05:00
Mingliang Liu 0f481e2022 HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou 2016-10-11 17:18:48 -07:00
Xiao Chen bc5a9931d7 HADOOP-13698. Document caveat for KeyShell when underlying KeyProvider does not delete a key. 
(cherry picked from commit b84c4891f9)
 2016-10-11 17:05:20 -07:00
Xiaoyu Yao 6f2de54eb1 HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle. 
(cherry picked from commit 61f0490a73)
 2016-10-11 16:00:17 -07:00
Andrew Wang 67f0319db6 HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dacd3ec66b)
 2016-10-11 15:07:28 -07:00
Wei-Chiu Chuang 773e2e6b6c HADOOP-13684. Snappy may complain Hadoop is built without snappy if libhadoop is not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4b32b1420d)
(cherry picked from commit 56184de510)
 2016-10-11 13:22:41 -07:00
Wei-Chiu Chuang aeb75213d4 HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma. 
(cherry picked from commit a6b3dc69d7a0835fa9159a0536c24ff6525e6c77)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2016-10-11 13:18:32 -07:00
Kihwal Lee 7f514e0c05 HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp. 
(cherry picked from commit e57fa81d95)
 2016-10-11 09:41:47 -05:00
Mingliang Liu 5a39a7560a HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou 2016-10-10 17:46:31 -07:00
Chris Nauroth 67d8301e55 HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth. 
(cherry picked from commit d152557cf7)
(cherry picked from commit e28930a38b)
 2016-10-10 14:01:19 -07:00
Chris Nauroth 027d76fa8c HADOOP-13446. Support running isolated unit tests separate from AWS integration tests. Contributed by Chris Nauroth. 
(cherry picked from commit cda68de9b9)
 2016-10-10 14:01:13 -07:00
Chris Nauroth d29dc39791 HADOOP-13208. S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories. Contributed by Steve Loughran. 
(cherry picked from commit 822d661b8f)
(cherry picked from commit 986162f977)
 2016-10-10 14:01:07 -07:00
Mingliang Liu def48f5228 HDFS-10985. o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions. Contributed by Mingliang Liu 
(cherry picked from commit de30f13a0a)
 2016-10-10 13:40:39 -07:00
Mingliang Liu 1c7baa505b HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula 
(cherry picked from commit b963818621)
 2016-10-10 13:26:48 -07:00
Xiao Chen 97105d8733 HADOOP-13669. KMS Server should log exceptions before throwing. Contributed by Suraj Acharya. 
(cherry picked from commit fc18c32540ed6a410adb123e1105729e0343b7f5)
 2016-10-10 12:50:59 -07:00
Kai Zheng 583283db25 HADOOP-13641. Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentation. Contributed by Huafeng Wang 2016-10-10 11:45:07 +06:00
Varun Vasudev d1841b4750 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-09 13:22:01 +05:30
Varun Vasudev 50409404bf YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e8)
(cherry picked from commit 78919f8c34)
 2016-10-09 11:53:03 +05:30
Xiao Chen 998e44a972 HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory. 
(cherry picked from commit 3ea027e970ec7c95a10e581435cdb6c3a6b64abb)
 2016-10-07 17:31:42 -07:00
Kihwal Lee 0cc64759ab HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp. 
(cherry picked from commit 3565c9af17)
 2016-10-07 14:38:11 -05:00
Chris Nauroth c6e69f6fb5 HADOOP-13692. hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. Contributed by Chris Nauroth. 
(cherry picked from commit 69620f9559)
(cherry picked from commit 2de2d0df1d)
 2016-10-07 11:41:40 -07:00
Kihwal Lee d55b7bf1c7 HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn Sharp. 
(cherry picked from commit 76c03acc6d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-10-07 13:25:56 -05:00
Robert Kanter dfc0e3f543 HADOOP-12611. TestZKSignerSecretProvider#testMultipleInit occasionally fail (ebadger via rkanter) 
(cherry picked from commit c183b9de8d)
 2016-10-07 10:51:28 -07:00
Junping Du 2ff6ffe0ef YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin. 
(cherry picked from commit 459a4833a9)
(cherry picked from commit 8bee319130)
 2016-10-07 07:51:01 -07:00
Steve Loughran 96cc056cb7 HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran. 2016-10-07 13:22:20 +01:00
Steve Loughran d24933ad3a HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-10-07 13:12:04 +01:00
Andrew Wang 7296999c89 HADOOP-13689. Do not attach javadoc and sources jars during non-dist build. 
(cherry picked from commit bf372173d0)
 2016-10-06 15:08:30 -07:00
Kihwal Lee 29afba8a15 HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp. 
(cherry picked from commit e3a9666d28)
 2016-10-06 16:37:53 -05:00
Chris Nauroth e36b665f90 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 
(cherry picked from commit 1d330fbaf6)
(cherry picked from commit a28ffd0fde)
 2016-10-06 13:24:54 -07:00
Chris Nauroth b9ce40e12c HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 
(cherry picked from commit 2d46c3f6b7)
(cherry picked from commit ecccb114ae)
 2016-10-06 10:57:15 -07:00
Rohith Sharma K S 319b101b7e YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G. 
(cherry picked from commit 4d2f380d78)
 2016-10-06 20:44:39 +05:30
Brahma Reddy Battula a4ee4f9e51 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 
(cherry picked from commit b90fc70d67)
 2016-10-06 17:49:48 +05:30
Karthik Kambatla c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Andrew Wang a2cdd2215e HDFS-10962. TestRequestHedgingProxyProvider is flaky. 
(cherry picked from commit e68c7b96c7)
(cherry picked from commit 6b2a853dd7)
 2016-10-05 15:09:13 -07:00
Jason Lowe f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Steve Loughran f66863f848 HADOOP-12667 s3a to support createNonRecursive API. Contributed by Sean Mackrory 2016-10-05 15:01:52 +01:00
Akira Ajisaka c0699a1f55 HADOOP-13685. Document -safely option of rm command. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9a61b52fbe)
(cherry picked from commit bdbb75e083)
 2016-10-05 20:41:51 +09:00
Varun Saxena 3656b84085 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit d677b68c25)
 2016-10-04 20:57:01 +00:00
Kihwal Lee d03240e0bf HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp. 
(cherry picked from commit 44f48ee96e)
 2016-10-04 15:29:39 -05:00
Chris Nauroth 4ea1e73e5b HADOOP-13674. S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. Contributed by Chris Nauroth. 
(cherry picked from commit 88b9444a81)
(cherry picked from commit 213f0ad7f0)
 2016-10-04 10:37:10 -07:00
Kihwal Lee 33cafc9fcd HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah. 
(cherry picked from commit 382307cbdd)
 2016-10-04 11:13:39 -05:00
Brahma Reddy Battula 253e2442e4 HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore 
(cherry picked from commit ef7f06f7d1)
 2016-10-04 18:53:12 +05:30
Brahma Reddy Battula 1ae35b6fff HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N 
(cherry picked from commit 5ea0210c7f)
 2016-10-04 18:39:31 +05:30
Mingliang Liu 438402a710 HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger 
(cherry picked from commit f61e3d13e9)
 2016-10-03 22:44:22 -04:00
Mingliang Liu 11ea5a59c2 HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula 
(cherry picked from commit d963ecb918)
 2016-10-03 22:04:49 -04:00
Xiao Chen bf043e63f9 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157)
 2016-10-03 16:02:11 -07:00
Xiaoyu Yao b0b29de8ab HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu. 
(cherry picked from commit 607705c488)
 2016-10-03 11:05:58 -07:00
Naganarasimha 90443140a5 YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 13:29:18 -04:00