Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,171 Commits 326 Branches 371 Tags
Commit Graph

8171 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Karthik Kambatla
e6c73a1c1a HADOOP-11213. Typos in html pages: SecureMode and EncryptedShuffle. (Wei Yan via kasha) 
(cherry picked from commit 6f1e3667cff3e95f33ba14a3cb7d1e99518d502a)
 2014-12-19 15:07:39 -08:00
Colin Patrick Mccabe
bcaf15e2fa HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume (cmccabe) 
(cherry picked from commit 8fa265a290792ff42635ff9b42416c634f88bdf3)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
 2014-12-19 13:19:52 -08:00
Colin Patrick Mccabe
c60e3bdc4b HADOOP-11422. Check CryptoCodec is AES-CTR for Crypto input/output stream (Yi Liu via Colin P. McCabe) 
(cherry picked from commit e369d98ba24906abcb35c0906cd57750c0d21fdb)
 2014-12-19 13:12:20 -08:00
Robert Kanter
22b38cf29a MAPREDUCE-6045. need close the DataInputStream after open it in TestMapReduce.java (zxu via rkanter) 
(cherry picked from commit d9e4d67d18811e16d5b0a76ea8228d333ded195f)
 2014-12-19 12:00:00 -08:00
Colin Patrick Mccabe
22f72c7946 HDFS-7552. Change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration (Liang Xie via Colin P. McCabe) 
(cherry picked from commit a4876c130f1627e59ef055e586640d1933fc49af)
 2014-12-19 11:20:43 -08:00
Colin Patrick Mccabe
fe38ed2ee7 HADOOP-11430. Add GenericTestUtils#disableLog, GenericTestUtils#setLogLevel (cmccabe) 
(cherry picked from commit 6635ccd21766faf8f6a15d937dc8919a9075c988)
 2014-12-18 19:41:59 -08:00
Haohui Mai
99c176ea6a HADOOP-11431. Clean up redundant maven-site-plugin configuration. Contributed by Herve Boutemy. 2014-12-18 16:24:17 -08:00
Jason Lowe
173664d70f YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie). Contributed by Jian He 
(cherry picked from commit 0402bada1989258ecbfdc437cb339322a1f55a97)
 2014-12-18 23:31:11 +00:00
cnauroth
947475c127 HDFS-7431. log message for InvalidMagicNumberException may be incorrect. Contributed by Yi Liu. 
(cherry picked from commit 5df7ecb33ab24de903f0fd98e2a055164874def5)
 2014-12-18 15:06:00 -08:00
Andrew Wang
8d98d87745 HDFS-7530. Allow renaming of encryption zone roots. Contributed by Charles Lamb. 
(cherry picked from commit b0b9084433d5e80131429e6e76858b099deb2dda)
 2014-12-18 14:18:14 -08:00
Haohui Mai
27ef6e4a6e HADOOP-11428. Remove obsolete reference to Cygwin in BUILDING.txt. Contributed by Arpit Agarwal. 2014-12-18 13:38:15 -08:00
Jason Lowe
84ea92879c HADOOP-11409. FileContext.getFileContext can stack overflow if default fs misconfigured. Contributed by Gera Shegalov 
(cherry picked from commit b9d49761f72078a0a83137ba8197d08b71f385e0)
 2014-12-18 21:28:57 +00:00
Haohui Mai
a508001ddd HADOOP-11385. Prevent cross site scripting attack on JMXJSONServlet. Contributed by Haohui Mai. 2014-12-18 11:39:17 -08:00
Haohui Mai
b1703c5c8f HDFS-7543. Avoid path resolution when getting FileStatus for audit logs. Contributed by Haohui Mai. 2014-12-18 11:25:03 -08:00
Colin Patrick Mccabe
2743b94c27 HADOOP-11427. ChunkedArrayList: fix removal via iterator and implement get (cmccabe) 
(cherry picked from commit 07619aa516deb9094a18e0c64ce66ff9c8b05e92)
 2014-12-18 11:05:33 -08:00
Kihwal Lee
8bffaa46fc HDFS-7373. Clean up temporary files after fsimage transfer failures. Contributed by Kihwal Lee 
(cherry picked from commit c0d666c74e9ea76564a2458c6c0a78ae7afa9fea)
 2014-12-18 13:00:18 -06:00
Haohui Mai
8854c78015 HADOOP-11125. Remove redundant tests in TestOsSecureRandom. Contributed by Masanori Oyama. 2014-12-18 10:59:48 -08:00
Junping Du
cec9300201 YARN-2949. Add documentation for CGroups. (Contributed by Varun Vasudev) 
(cherry picked from commit 389f881d423c1f7c2bb90ff521e59eb8c7d26214)
 2014-12-18 06:26:53 -08:00
yliu
2d1ddbe58e HADOOP-11358. Tests for encryption/decryption with IV calculation overflow. (yliu) 2014-12-18 00:43:31 +08:00
Junping Du
11e4ec52a1 YARN-2972. DelegationTokenRenewer thread pool never expands. Contributed by Jason Lowe 
(cherry picked from commit 2b4b0e8847048850206f091c6870a02e08cfe836)
 2014-12-17 17:41:21 -08:00
Colin Patrick Mccabe
dda1fc169d HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 3b173d95171d01ab55042b1162569d1cf14a8d43)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-12-17 16:43:33 -08:00
Karthik Kambatla
d0d42c4e32 YARN-2203. [YARN-1492] Web UI for cache manager. (Chris Trezzo via kasha) 
(cherry picked from commit b7f64823e11f745783607ae5f3f97b5e8e64c389)
 2014-12-17 16:34:33 -08:00
Karthik Kambatla
375a1e5971 YARN-2944. InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance. (Chris Trezzo via kasha) 
(cherry picked from commit a1bd1409649da96c9fde4a9f9398d7711bc6c281)
 2014-12-17 16:25:02 -08:00
Karthik Kambatla
2d832ad2eb YARN-2964. FSLeafQueue#assignContainer - document the reason for using both write and read locks. (Tsuyoshi Ozawa via kasha) 
(cherry picked from commit f2d150ea1205b77a75c347ace667b4cd060aaf40)
 2014-12-17 16:24:17 -08:00
Colin Patrick Mccabe
883ad3bf5a HADOOP-11421. Add IOUtils#listDirectory (cmccabe) 
(cherry picked from commit 9937eef7f7f04a7dd3d504ae7ec5852d488a1f6a)
 2014-12-17 15:18:35 -08:00
Haohui Mai
c28449d4b5 HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack. Contributed by Li Lu. 
(cherry picked from commit 2e98ad34ce64a9e5184c53447004de20a637f927)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:14 -08:00
Steve Loughran
e71484be18 HADOOP-11412 POMs mention "The Apache Software License" rather than "Apache License". (Herve Boutemy via stevel) 
(cherry picked from commit 07bb0b0bbb4732687c4992e8ce61a43ce8ba3db6)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:13 -08:00
cnauroth
5c32c4b2e4 HADOOP-10809. SyncableDataOutputStream must implement sync method on branch-2. 2014-12-17 14:57:13 -08:00
Aaron T. Myers
66808e6aac HADOOP-10400. Incorporate new S3A FileSystem implementation. Contributed by Jordan Mendelson and Dave Wang. 
(cherry picked from commit 24d920b80eb3626073925a1d0b6dcf148add8cc0)

Conflicts:
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/BasicAWSCredentialsProvider.java
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java
	hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AOutputStream.java
	hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/TestS3AContractRename.java
	hadoop-tools/hadoop-aws/src/test/resources/contract/s3a.xml
 2014-12-17 14:57:13 -08:00
cnauroth
535d98bbb3 HADOOP-11248. Add hadoop configuration to disable Azure Filesystem metrics collection. Contributed by Shanyu Zhao. 
(cherry picked from commit 734eeb4f357ad3210355a0d3fdbc80706a770d61)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:13 -08:00
cnauroth
1646cc9f29 HADOOP-11188. hadoop-azure: automatically expand page blobs when they become full. Contributed by Eric Hanson. 
(cherry picked from commit f4b7e99f4ebac5b0295b7f7f42eb5705af41f079)
 2014-12-17 14:57:13 -08:00
cnauroth
5a737026cc HADOOP-10809. hadoop-azure: page blob support. Contributed by Dexter Bradshaw, Mostafa Elhemali, Eric Hanson, and Mike Liddell. 
(cherry picked from commit 2217e2f8ff418b88eac6ad36cafe3a9795a11f40)
 2014-12-17 14:57:13 -08:00
Allen Wittenauer
9a2e4f4f98 HADOOP-10946. Fix a bunch of typos in log messages (Ray Chiang via aw) 
(cherry picked from commit 9f03a7c018bb2c497cd0ef758f1a3e08e8163d06)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:13 -08:00
cnauroth
156e78c1da HADOOP-10690. Correct CHANGES.txt. 2014-12-17 14:57:13 -08:00
Chris Nauroth
cb6eb5931f HADOOP-10690. Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close(). Contributed by Chen He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604236 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 61bf9f779919206296d2ce84a8a6b2d912709a59)
 2014-12-17 14:57:12 -08:00
Chris Nauroth
02a3bf8bda HADOOP-10840. Fix OutOfMemoryError caused by metrics system in Azure File System. Contributed by Shanyu Zhao. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1611247 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 0a02b5a19bcca8d0c49d2adfc3909aebdf9d606e)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:12 -08:00
Chris Nauroth
b928b8c775 HADOOP-10728. Metrics system for Windows Azure Storage Filesystem. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1605187 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 0d91576ec31f63402f2db6107a04155368e2632d)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:12 -08:00
Chris Nauroth
80d005c66f HADOOP-10689. InputStream is not closed in AzureNativeFileSystemStore#retrieve(). Contributed by Chen He. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1604233 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 8300b9fb385b47672d98ea62ab291991424f3cce)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
 2014-12-17 14:57:12 -08:00
Chris Nauroth
82268d87bc HADOOP-9629. Support Windows Azure Storage - Blob as a file system in Hadoop. Contributed by Dexter Bradshaw, Mostafa Elhemali, Xi Fang, Johannes Klein, David Lao, Mike Liddell, Chuan Liu, Lengning Liu, Ivan Mitic, Michael Rys, Alexander Stojanovic, Brian Swan, and Min Wei. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1601781 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 81bc395deb3ba00567dc067d6ca71bacf9e3bc82)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-project/pom.xml
	hadoop-tools/hadoop-tools-dist/pom.xml
	hadoop-tools/pom.xml
 2014-12-17 14:56:05 -08:00
Haohui Mai
a7c5456c72 HDFS-7528. Consolidate symlink-related implementation into a single class. Contributed by Haohui Mai. 2014-12-17 14:54:24 -08:00
Haohui Mai
e869276005 HADOOP-11420. Use latest maven-site-plugin and replace link to svn with link to git. Contributed by Hervé Boutemy. 2014-12-17 14:05:59 -08:00
Karthik Kambatla
300a665307 YARN-2189. Admin service for cache manager. Addendum to fix merge conflicts 2014-12-17 11:47:24 -08:00
Colin Patrick Mccabe
89d6ac498a HADOOP-11416. Move ChunkedArrayList into hadoop-common (cmccabe) 
(cherry picked from commit 4281c96e24739387bc2084f819c0176d0051a5e9)
 2014-12-17 10:36:59 -08:00
cnauroth
36068768d8 HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions. Contributed by Chris Nauroth. 
(cherry picked from commit e996a1bfd4f3ada6cbd9633fe68fda1e0c910bdf)
 2014-12-16 15:32:23 -08:00
Haohui Mai
e870851b4a HDFS-7536. Remove unused CryptoCodec in org.apache.hadoop.fs.Hdfs. Contributed by Haohui Mai. 2014-12-16 13:26:52 -08:00
Haohui Mai
ba0969fb0e Revert HADOOP-11356 in branch-2. 2014-12-16 13:21:58 -08:00
Colin Patrick Mccabe
1b91a6d011 HDFS-7494. Checking of closed in DFSInputStream#pread() should be protected by synchronization (Ted Yu via Colin P. McCabe) 
(cherry picked from commit a97a1e73177974cff8afafad6ca43a96563f3c61)
 2014-12-16 11:08:02 -08:00
Jian He
878350a0e8 YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments before sending to RM. Contributed by Rohith Sharmaks 
(cherry picked from commit c65f1b382ec5ec93dccf459dbf8b2c93c3e150ab)
 2014-12-16 11:01:24 -08:00
Kihwal Lee
b70c0c295c HDFS-6425. Large postponedMisreplicatedBlocks has impact on blockReport latency. Contributed by Ming Ma. 
(cherry picked from commit b7923a356e9f111619375b94d12749d634069347)
 2014-12-16 10:31:57 -06:00
Steve Loughran
a1b8b9ca59 HADOOP-11412 POMs mention "The Apache Software License" rather than "Apache License". (Herve Boutemy via stevel) 2014-12-16 11:03:02 +00:00