Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,692 Commits 326 Branches 371 Tags
Commit Graph

11692 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mingliang Liu
9b2a719031 HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang 
(cherry picked from commit b36aaa913c54829adbf5fa852af7f45be4db0f07)
 2016-10-12 17:41:07 -07:00
Andrew Wang
9bde45d2fe HADOOP-13700. Remove unthrown IOException from TrashPolicy#initialize and #getInstance signatures. 
(cherry picked from commit 12d739a34ba868b3f7f5adf7f37a60d4aca9061b)
(cherry picked from commit f131d61ff8bc87e74c67fec3ae2b50ee50dd0d2c)
 2016-10-12 15:20:10 -07:00
Kihwal Lee
136c6f6f7d HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah. 
(cherry picked from commit 5305a392c39d298ecf38ca2dfd2526adeee9cd38)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
 2016-10-12 15:38:34 -05:00
Mingliang Liu
0f481e2022 HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou 2016-10-11 17:18:48 -07:00
Xiao Chen
bc5a9931d7 HADOOP-13698. Document caveat for KeyShell when underlying KeyProvider does not delete a key. 
(cherry picked from commit b84c4891f9eca8d56593e48e9df88be42e24220d)
 2016-10-11 17:05:20 -07:00
Xiaoyu Yao
6f2de54eb1 HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle. 
(cherry picked from commit 61f0490a73085bbaf6639d9234277e59dc1145db)
 2016-10-11 16:00:17 -07:00
Andrew Wang
67f0319db6 HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit dacd3ec66b111be24131957c986f0c748cf9ea26)
 2016-10-11 15:07:28 -07:00
Wei-Chiu Chuang
773e2e6b6c HADOOP-13684. Snappy may complain Hadoop is built without snappy if libhadoop is not found. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 4b32b1420d98ea23460d05ae94f2698109b3d6f7)
(cherry picked from commit 56184de510e27990c36726e2a648e8ce4b06504c)
 2016-10-11 13:22:41 -07:00
Wei-Chiu Chuang
aeb75213d4 HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma. 
(cherry picked from commit a6b3dc69d7a0835fa9159a0536c24ff6525e6c77)

Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2016-10-11 13:18:32 -07:00
Kihwal Lee
7f514e0c05 HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp. 
(cherry picked from commit e57fa81d9559a93d77fd724f7792326c31a490be)
 2016-10-11 09:41:47 -05:00
Mingliang Liu
5a39a7560a HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou 2016-10-10 17:46:31 -07:00
Chris Nauroth
67d8301e55 HADOOP-13447. Refactor S3AFileSystem to support introduction of separate metadata repository and tests. Contributed by Chris Nauroth. 
(cherry picked from commit d152557cf7f4d2288524c222fcbaf152bdc038b0)
(cherry picked from commit e28930a38b3868bc701aeef229831292320b4a3a)
 2016-10-10 14:01:19 -07:00
Chris Nauroth
027d76fa8c HADOOP-13446. Support running isolated unit tests separate from AWS integration tests. Contributed by Chris Nauroth. 
(cherry picked from commit cda68de9b913878210118a6637a1d0eb950c26ad)
 2016-10-10 14:01:13 -07:00
Chris Nauroth
d29dc39791 HADOOP-13208. S3A listFiles(recursive=true) to do a bulk listObjects instead of walking the pseudo-tree of directories. Contributed by Steve Loughran. 
(cherry picked from commit 822d661b8fcc42bec6eea958d9fd02ef1aaa4b6c)
(cherry picked from commit 986162f9776837b1343073d721f89be8f3362085)
 2016-10-10 14:01:07 -07:00
Mingliang Liu
def48f5228 HDFS-10985. o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions. Contributed by Mingliang Liu 
(cherry picked from commit de30f13a0ae22f0ffc647e4c1daf19ca9c01bb1a)
 2016-10-10 13:40:39 -07:00
Mingliang Liu
1c7baa505b HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula 
(cherry picked from commit b963818621c200160bb37624f177bdcb059de4eb)
 2016-10-10 13:26:48 -07:00
Xiao Chen
97105d8733 HADOOP-13669. KMS Server should log exceptions before throwing. Contributed by Suraj Acharya. 
(cherry picked from commit fc18c32540ed6a410adb123e1105729e0343b7f5)
 2016-10-10 12:50:59 -07:00
Kai Zheng
583283db25 HADOOP-13641. Update UGI#spawnAutoRenewalThreadForUserCreds to reduce indentation. Contributed by Huafeng Wang 2016-10-10 11:45:07 +06:00
Varun Vasudev
d1841b4750 YARN-5704. Provide config knobs to control enabling/disabling new/work in progress features in container-executor. Contributed by Sidharta Seethana. 2016-10-09 13:22:01 +05:30
Varun Vasudev
50409404bf YARN-4245. Generalize config file handling in container-executor. Contributed by Sidharta Seethana. 
(cherry picked from commit 8ed2e060e80c0def3fcb7604e0bd27c1c24d291e)
(cherry picked from commit 78919f8c341ec645cf9134991e3ae89a929b9184)
 2016-10-09 11:53:03 +05:30
Xiao Chen
998e44a972 HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory. 
(cherry picked from commit 3ea027e970ec7c95a10e581435cdb6c3a6b64abb)
 2016-10-07 17:31:42 -07:00
Kihwal Lee
0cc64759ab HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp. 
(cherry picked from commit 3565c9af17ab05bf9e7f68b71b6c6850df772bb9)
 2016-10-07 14:38:11 -05:00
Chris Nauroth
c6e69f6fb5 HADOOP-13692. hadoop-aws should declare explicit dependency on Jackson 2 jars to prevent classpath conflicts. Contributed by Chris Nauroth. 
(cherry picked from commit 69620f955997250d1b543d86d4907ee50218152a)
(cherry picked from commit 2de2d0df1d4e7f5f965d787d53a44db5acefcde9)
 2016-10-07 11:41:40 -07:00
Kihwal Lee
d55b7bf1c7 HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn Sharp. 
(cherry picked from commit 76c03acc6df8f379e9e1b09b82d733337b9db655)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2016-10-07 13:25:56 -05:00
Robert Kanter
dfc0e3f543 HADOOP-12611. TestZKSignerSecretProvider#testMultipleInit occasionally fail (ebadger via rkanter) 
(cherry picked from commit c183b9de8d072a35dcde96a20b1550981f886e86)
 2016-10-07 10:51:28 -07:00
Junping Du
2ff6ffe0ef YARN-5659. getPathFromYarnURL should use standard methods. Contributed by Sergey Shelukhin. 
(cherry picked from commit 459a4833a90437a52787a41c2759a4b18cfe411c)
(cherry picked from commit 8bee319130f0637ba88ce2e9da18571f9788d027)
 2016-10-07 07:51:01 -07:00
Steve Loughran
96cc056cb7 HADOOP-12977 s3a to handle delete("/", true) robustly. Contributed by Steve Loughran. 2016-10-07 13:22:20 +01:00
Steve Loughran
d24933ad3a HADOOP-13164 Optimize S3AFileSystem::deleteUnnecessaryFakeDirectories. Contributed by Rajesh Balamohan. 2016-10-07 13:12:04 +01:00
Andrew Wang
7296999c89 HADOOP-13689. Do not attach javadoc and sources jars during non-dist build. 
(cherry picked from commit bf372173d0f7cb97b62556cbd199a075254b96e6)
 2016-10-06 15:08:30 -07:00
Kihwal Lee
29afba8a15 HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp. 
(cherry picked from commit e3a9666d285c62afb0d50abea74d9e2ffe2767a8)
 2016-10-06 16:37:53 -05:00
Chris Nauroth
e36b665f90 HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth. 
(cherry picked from commit 1d330fbaf6b50802750aa461640773fb788ef884)
(cherry picked from commit a28ffd0fdeff64a12612e60f4ebc5e6311b4b112)
 2016-10-06 13:24:54 -07:00
Chris Nauroth
b9ce40e12c HADOOP-13323. Downgrade stack trace on FS load from Warn to debug. Contributed by Steve Loughran. 
(cherry picked from commit 2d46c3f6b7d55b6a2f124d07fe26d37359615df4)
(cherry picked from commit ecccb114ae1b09526d11385df7085b6dd3376e2d)
 2016-10-06 10:57:15 -07:00
Rohith Sharma K S
319b101b7e YARN-5101. YARN_APPLICATION_UPDATED event is parsed in ApplicationHistoryManagerOnTimelineStore#convertToApplicationReport with reversed order. Contributed by Sunil G. 
(cherry picked from commit 4d2f380d787a6145f45c87ba663079fedbf645b8)
 2016-10-06 20:44:39 +05:30
Brahma Reddy Battula
a4ee4f9e51 HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen 
(cherry picked from commit b90fc70d671481564e468550c770c925f25d7db0)
 2016-10-06 17:49:48 +05:30
Karthik Kambatla
c955a59d63 MAPREDUCE-6740. Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval. (Haibo Chen via kasha) 2016-10-05 23:12:45 -07:00
Andrew Wang
a2cdd2215e HDFS-10962. TestRequestHedgingProxyProvider is flaky. 
(cherry picked from commit e68c7b96c76b243da53ae539e7c760bf2c08f26e)
(cherry picked from commit 6b2a853dd797c27cab988452f0f5d1f42e9b4bdd)
 2016-10-05 15:09:13 -07:00
Jason Lowe
f1b74a3d9f MAPREDUCE-6741. Add MR support to redact job conf properties. Contributed by Haibo Chen 2016-10-05 15:24:10 +00:00
Steve Loughran
f66863f848 HADOOP-12667 s3a to support createNonRecursive API. Contributed by Sean Mackrory 2016-10-05 15:01:52 +01:00
Akira Ajisaka
c0699a1f55 HADOOP-13685. Document -safely option of rm command. Contributed by Xiaoyu Yao. 
(cherry picked from commit 9a61b52fbefd2970af78bb283ae174d462d57a13)
(cherry picked from commit bdbb75e083218e11d8c8e34d38e4dc859b444c97)
 2016-10-05 20:41:51 +09:00
Varun Saxena
3656b84085 YARN-5491. Fix random failure of TestCapacityScheduler#testCSQueueBlocked (Bibin A Chundatt via Varun Saxena) 
(cherry picked from commit d677b68c2599445fff56db4df26448a8bad0f5dd)
 2016-10-04 20:57:01 +00:00
Kihwal Lee
d03240e0bf HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp. 
(cherry picked from commit 44f48ee96ee6b2a3909911c37bfddb0c963d5ffc)
 2016-10-04 15:29:39 -05:00
Chris Nauroth
4ea1e73e5b HADOOP-13674. S3A can provide a more detailed error message when accessing a bucket through an incorrect S3 endpoint. Contributed by Chris Nauroth. 
(cherry picked from commit 88b9444a81081da9b168d2e290f9552b58a4d8c6)
(cherry picked from commit 213f0ad7f0e846cf8023b12cf8e5f804950b1aa1)
 2016-10-04 10:37:10 -07:00
Kihwal Lee
33cafc9fcd HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah. 
(cherry picked from commit 382307cbdd94107350fe6fad1acf87d63c9be9d6)
 2016-10-04 11:13:39 -05:00
Brahma Reddy Battula
253e2442e4 HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore 
(cherry picked from commit ef7f06f7d1561db13bd3b07a5f62815ec29e1cdf)
 2016-10-04 18:53:12 +05:30
Brahma Reddy Battula
1ae35b6fff HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N 
(cherry picked from commit 5ea0210c7fea1324973e7f69256bc53f31ee4cf7)
 2016-10-04 18:39:31 +05:30
Mingliang Liu
438402a710 HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger 
(cherry picked from commit f61e3d13e9610cbd09886359553f27d6480f6735)
 2016-10-03 22:44:22 -04:00
Mingliang Liu
11ea5a59c2 HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula 
(cherry picked from commit d963ecb9185b8fcfba86bec8420a8daf7140a071)
 2016-10-03 22:04:49 -04:00
Xiao Chen
bf043e63f9 HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen. 
(cherry picked from commit 853d65a157362661ccab10379c2d82e780382f83)
 2016-10-03 16:02:11 -07:00
Xiaoyu Yao
b0b29de8ab HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu. 
(cherry picked from commit 607705c488fa5263d851cee578a2d319e6e52ecd)
 2016-10-03 11:05:58 -07:00
Naganarasimha
90443140a5 YARN-4855. Should check if node exists when replace nodelabels. Contributeed by Tao Jie 2016-10-03 13:29:18 -04:00