Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,961 Commits 326 Branches 371 Tags
Commit Graph

11961 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arpit Agarwal
3f2bfdba3e HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin. 2016-11-03 10:33:28 -07:00
Naganarasimha
605e9bf3bf YARN-5697. Use CliParser to parse options in RMAdminCLI. Contributed by Tao Jie. 2016-11-03 10:31:30 +05:30
Andrew Wang
e393dd788e HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory. 2016-11-02 19:11:25 -07:00
Varun Saxena
8602e2cb0f YARN-5815. Random failure of TestApplicationPriority.testOrderOfActivatingThePriorityApplicationOnRMRestart (Bibin A Chundatt via Varun Saxena) 2016-11-03 00:43:28 +05:30
Brahma Reddy Battula
77142827e8 HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu. 
(cherry picked from commit 35e6081020d34be966724f44df678f1f5ee6f6d8)
 2016-11-02 10:40:22 +05:30
Andrew Wang
9b64e30ccf HADOOP-13583. Incorporate checkcompatibility script which runs Java API Compliance Checker. 
(cherry picked from commit 0cd22d66691fdbc3fb3e0a35ad1625236c3ae3f7)
(cherry picked from commit 92fff7d065d11df4115063e4bbfb98fa0a763718)
 2016-11-01 16:36:40 -07:00
Mingliang Liu
10eb21e52e HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou 
(cherry picked from commit fa1512a740b2ed2661743d6b5483ef3eb49e5634)
 2016-11-01 14:50:16 -07:00
Jason Lowe
4a70bd86da YARN-5001. Aggregated Logs root directory is created with wrong group if nonexistent. Contributed by Haibo Chen 
(cherry picked from commit 76893a41003d57d94eb1a5f486010815266046af)
 2016-11-01 20:24:44 +00:00
Ravi Prakash
8504e20b2e HADOOP-13773. Wrong HADOOP_CLIENT_OPTS in hadoop-env on branch-2. Contributed by Fei Hui 
(cherry picked from commit 045501a538800a48b77e688e46c55dce54473041)
 2016-11-01 10:49:45 -07:00
Kihwal Lee
db6dfeca1a HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp. 
(cherry picked from commit 7c1a1834e49cf88c64837d92b78bbc07ea9e9efc)
 2016-11-01 08:26:25 -05:00
Akira Ajisaka
689bbade9a HADOOP-13603. Ignore package line length checkstyle rule. Contributed by Shane Kumpf. 
(cherry picked from commit 5b577f43a269caeee776a59695427985d0cd1697)
(cherry picked from commit f0d4d7a1782ed743feb6625e207fb89a19fd6957)
 2016-11-01 21:00:04 +09:00
Akira Ajisaka
61128c50de MAPREDUCE-6795. Update the document for JobConf#setNumReduceTasks. Contributed by Yiqun Lin. 
(cherry picked from commit 235f39b7c44c5907bd4de063892905577c35cd80)
(cherry picked from commit a3897f77aefaf0a9823b31a1d3e8f4d1eded8642)
 2016-11-01 20:52:52 +09:00
Brahma Reddy Battula
c895f08936 HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R. 
(cherry picked from commit 1c5708f6ded6ba9fc007eed93c1f65ba44451241)
 2016-11-01 10:53:37 +05:30
Kihwal Lee
91640a0170 Addendum of HDFS-10722 to include HDFS-7964 logic. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
(cherry picked from commit 3b2554f3b0f686a66e3df20b7c381babb9bc74d3)
(cherry picked from commit c2bb9bf00c4a32644f96c673078a812087ef3c46)
 2016-10-31 16:37:00 -07:00
Zhe Zhang
7900b1e803 Revert "HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp." 
This reverts commit c2bb9bf00c4a32644f96c673078a812087ef3c46.
 2016-10-31 16:35:05 -07:00
Kihwal Lee
c2bb9bf00c HDFS-10722. Fix race condition in TestEditLog#testBatchedSyncWithClosedLogs. Contributed by Daryn Sharp. 
(cherry picked from commit 4a8e6dc02fb1d0dc89c3981b444575494511444a)
(cherry picked from commit 3b2554f3b0f686a66e3df20b7c381babb9bc74d3)
 2016-10-31 16:30:39 -07:00
Akira Ajisaka
a17fc1dca3 HDFS-10485. Fix findbugs warning in FSEditLog.java. (aajisaka) 
(cherry picked from commit e620530301fd3e62537d4b7bc3d8ed296bda1ffc)
(cherry picked from commit b8c1db6f63f9d01f44f8f43921aadfe999eefd76)
 2016-10-31 16:24:35 -07:00
Jing Zhao
e327325a37 HDFS-7964. Add support for async edit logging. Contributed by Daryn Sharp. 
(cherry picked from commit 2151716832ad14932dd65b1a4e47e64d8d6cd767)
(cherry picked from commit 3cb7ae11a839c01b8be629774874c1873f51b747)
 2016-10-31 16:24:35 -07:00
Mingliang Liu
fece3c881b HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu 
(cherry picked from commit 6bb12ecc3f63afb2070296c2db009c6f9a2d884b)
 2016-10-31 16:06:41 -07:00
Steve Loughran
a83f10c90d HADOOP-13680. fs.s3a.readahead.range to use getLongBytes. Contributed by Abhishek Modi. 2016-10-31 20:53:22 +00:00
Arpit Agarwal
f0a6d17e3b HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru. 2016-10-31 12:40:03 -07:00
Naganarasimha
b8ae7a8ee4 YARN-4498. Application level node labels stats to be available in REST. Contributed by Bibin A Chundatt. 
(cherry picked from commit e4023f0d8177ac37a75685b55c11991ca4ea2c23)
 2016-10-31 19:09:53 +05:30
Akira Ajisaka
757b38ea43 MAPREDUCE-6799. Document mapreduce.jobhistory.webapp.https.address in mapred-default.xml. Contributed by Yiqun Lin. 
(cherry picked from commit f646fe34be704a22cbea322cce2826f8f201bd34)
(cherry picked from commit ef13a5ef9d7859cbb720ee46ebed8172358ad0d5)
 2016-10-31 15:19:37 +09:00
Varun Saxena
367cb029d4 YARN-5773. RM recovery too slow due to LeafQueue#activateApplications (Bibin A Chundatt via Varun Saxena) 2016-10-30 11:50:54 +05:30
Wei-Chiu Chuang
9b6d27793d HDFS-11061. Update dfs -count -t command line help and documentation. Contributed by Yiqun Lin. 
(cherry picked from commit 112f04eb1b771e410ffd232a46e0ee2bc7b99d06)
(cherry picked from commit 9ba283c27d73962a894b33a89f17d3ec697bb388)
 2016-10-28 15:37:53 -07:00
Mingliang Liu
616c1194fe HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou 
(cherry picked from commit 2848a27700e76fc2c2d3ab87adba54ca8e93ba39)
 2016-10-28 11:18:26 -07:00
Mingliang Liu
a8a99f3423 HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger 
(cherry picked from commit 8a9388e5f6d622152798aaaa256064919e4fec37)
 2016-10-28 11:01:09 -07:00
Jason Lowe
93165c1ade YARN-4963. capacity scheduler: Make number of OFF_SWITCH assignments per heartbeat configurable. Contributed by Nathan Roberts 2016-10-28 17:44:56 +00:00
Jason Lowe
9cc7ab4e96 YARN-5767. Fix the order that resources are cleaned up from the local Public/Private caches. Contributed by Chris Trezzo 
(cherry picked from commit 1b79c417dca17bcd2e031864bc6ca07254c61b47)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java
 2016-10-28 16:11:59 +00:00
Jason Lowe
83bb428e6c YARN-5027. NM should clean up app log dirs after NM restart. Contributed by sandflee 
(cherry picked from commit 7146359bfd436a76585fb1f3ea93716795308cec)
 2016-10-28 15:50:22 +00:00
Jason Lowe
558e53b10b HADOOP-13770. Shell.checkIsBashSupported swallowed an interrupted exception. Contributed by Wei-Chiu Chuang 
(cherry picked from commit c017171da00a6cd71a2901c84a0298ce14a49e23)
 2016-10-28 15:06:57 +00:00
Jason Lowe
0a405c4f71 MAPREDUCE-6724. Single shuffle to memory must not exceed Integer#MAX_VALUE. (Haibo Chen via gera) 
(cherry picked from commit 6890d5b472320fa7592ed1b08b623c55a27089c6)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestMergeManager.java
 2016-10-28 14:40:28 +00:00
Naganarasimha
8a664eba7d MAPREDUCE-2631. Potential resource leaks in BinaryProtocol$TeeOutputStream.java. Contributed by Sunil G. 
(cherry picked from commit 28660f51af161a9fa301523d96a6f8ae4ebd6edd)
 2016-10-28 06:04:03 +05:30
Erik Krogen
2ec90a1a4b HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen. 
(cherry picked from commit f3ac1f41b8fa82a0ac87a207d7afa2061d90a9bd)
 2016-10-27 16:04:58 -07:00
Carlo Curino
b32f14d806 YARN-3432. Cluster metrics have wrong Total Memory when there is reserved memory on CS. (Brahma Reddy Battula via curino) 
(cherry picked from commit 892a8348fceb42069ea9877251c413fe33415e16)
(cherry picked from commit 58ac40b55296834a8e3f3375caddc03bee901e9a)
 2016-10-27 15:23:04 -07:00
Subru Krishnan
55ba220721 YARN-3568. TestAMRMTokens should use some random port. (Takashi Ohnishi via Subru). 
(cherry picked from commit 79ae78dcbec183ab53b26de408b4517e5a151878)
(cherry picked from commit 4274600b9527706259ec5df62a13a5a66adc3ff2)
 2016-10-27 15:19:01 -07:00
Wei-Chiu Chuang
bc2f429c0e HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 31ff42b51037632ec871f29efc0fa894e1b738d0)
(cherry picked from commit 2cf3138d72a1392d13f6225b4a9fca4fe7ced132)
 2016-10-27 13:47:26 -07:00
Jason Lowe
d21dc8aa6c YARN-4831. Recovered containers will be killed after NM stateful restart. Contributed by Siqi Li 
(cherry picked from commit 7e3c327d316b33d6a09bfd4e65e7e5384943bb1d)
 2016-10-27 20:45:18 +00:00
Kihwal Lee
40dbf2a18d HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee 
(cherry picked from commit ae48c496dce8d0eae4571fc64e6850d602bae688)
 2016-10-27 14:18:48 -05:00
Kihwal Lee
5b0d32b4aa Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu" 
This reverts commit d6dfc76dfee4489b2a55cfb8562970c3212b15b5.
 2016-10-27 14:10:52 -05:00
Brahma Reddy Battula
d6dfc76dfe HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu 
(cherry picked from commit ac35ee9393e0afce9fede1d2052e7bf4032312fd)
 2016-10-27 20:23:07 +05:30
Brahma Reddy Battula
26cf988c3a HADOOP-13201. Print the directory paths when ViewFs denies the rename operation on internal dirs. Contributed by Tianyin Xiu 
(cherry picked from commit 0c837db8a874079dd5db83a7eef9c4d2b9d0e9ff)
 2016-10-27 19:51:05 +05:30
Steve Loughran
5275d37495 Revert "HADOOP-13514. Upgrade maven surefire plugin to 2.19.1. Contributed by Ewan Higgs." 
This reverts commit 444312b54b890ca7b0bfe6f997007ef0ad82b00e.
 2016-10-27 15:21:29 +02:00
Steve Loughran
ca6f0a1d77 Revert "Addendum patch for HADOOP-13514 Upgrade maven surefire plugin to 2.19.1. Contributed by Akira Ajisaka." 
This reverts commit 9a98feb769dbff87d337c4297ee8d7a945fed055.
 2016-10-27 15:21:17 +02:00
Naganarasimha
27029393b7 MAPREDUCE-6541. Exclude scheduled reducer memory when calculating available mapper slots from headroom to avoid deadlock. Contributed by Varun Saxena 
(cherry picked from commit 060558c6f221ded0b014189d5b82eee4cc7b576b)
 2016-10-27 18:06:34 +05:30
Naganarasimha
3b9df97158 YARN-3848. TestNodeLabelContainerAllocation is timing out. Contributed by Varun Saxena 
(cherry picked from commit 6c8830992c734313fc95ac9988539c4d813c3581)
 2016-10-27 17:16:34 +05:30
Vinayakumar B
7279ce831d HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su) 
(cherry picked from commit 1cf6ec4ad4e1f4ea71f912923b5e8627b61ef482)
(cherry picked from commit 144f7a9e0eb3084eaaff7ab0bee1f5943e16d136)
 2016-10-27 16:56:25 +05:30
Masatake Iwasaki
6d9b6fac86 HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran. 
(cherry picked from commit 0bdd263d82a4510f16df49238d57c9f78ac28ae7)
 2016-10-27 15:55:35 +09:00
Mingliang Liu
a7684f6517 HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger 
(cherry picked from commit 55e1fb8e3221941321e6f5e04b334246c5f23027)
 2016-10-26 14:18:00 -07:00
Wei-Chiu Chuang
9a98feb769 Addendum patch for HADOOP-13514 Upgrade maven surefire plugin to 2.19.1. Contributed by Akira Ajisaka. 
(cherry picked from commit e48b592f8ba1d8a89587f2c4403d861f2d015a9a)
(cherry picked from commit 0c96ceaca9d66b07d4ed5c04697b4a826812918d)
 2016-10-26 14:10:26 -07:00