Commit Graph

2546 Commits

Author SHA1 Message Date
Akira Ajisaka 236c4ab511 HADOOP-12359. hadoop fs -getmerge doc is wrong. Contributed by Jagadesh Kiran N.
(cherry picked from commit 2e251a7674)
2015-09-01 20:56:46 +09:00
yliu 2345627ad3 HADOOP-12367. Move TestFileUtil's test resources to resources folder. (wang via yliu) 2015-09-01 16:25:11 +08:00
Andrew Wang 8dbe2a12bb HADOOP-12368. Mark ViewFileSystemBaseTest and ViewFsBaseTest as abstract.
(cherry picked from commit 7ad3556ed3)
2015-08-31 18:19:12 -07:00
Lei Xu de42fce390 HADOOP-12346. Increase some default timeouts / retries for S3a connector. (Sean Mackrory via Lei (Eddy) Xu) 2015-08-29 09:58:57 -07:00
Arpit Agarwal f1d5f1417e HDFS-8983. NameNode support for protected directories. (Contributed by Arpit Agarwal) 2015-08-29 09:57:53 -07:00
Andrew Wang 7a668981f5 HADOOP-12362. Set hadoop.tmp.dir and hadoop.log.dir in pom. Contributed by Charlie Helin.
(cherry picked from commit 90fe7bcc3b)
2015-08-27 10:17:04 -07:00
Jing Zhao 10c4d6df61 HDFS-8896. DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager. Contributed by Walter Su.
(cherry picked from commit f44b599003)
2015-08-26 16:28:38 -07:00
Harsh J e993498302 HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals before deleting. Contributed by Casey Brotherton.
(cherry picked from commit af78767870)
2015-08-25 11:22:08 +05:30
Xiaoyu Yao 7a0a31586a HADOOP-12325. RPC Metrics : Add the ability track and log slow RPCs. Contributed by Anu Engineer
(cherry picked from commit 48774d0a45)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
2015-08-24 14:33:37 -07:00
Xuan 205b98c059 YARN-221. NM should provide a way for AM to tell it not to aggregate
logs. Contributed by Ming Ma

(cherry picked from commit 37e1c3d82a)
2015-08-22 16:26:39 -07:00
Xiaoyu Yao a7e5a88b59 HADOOP-12347. Fix mismatch parameter name in javadocs of AuthToken#setMaxInactives. Contributed by Xiaoyu Yao
(cherry picked from commit caa636bf10)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
2015-08-21 16:41:34 -07:00
Vinod Kumar Vavilapalli 45be1240d9 Creating 2.6.2 entries in CHANGES.txt files.
(cherry picked from commit 33474939a0)
2015-08-20 17:23:54 -07:00
Akira Ajisaka 5fb8ac18d1 HADOOP-12061. Incorrect command in single cluster setup document. Contributed by Kengo Seki. 2015-08-20 11:11:12 +09:00
Robert Kanter acf241242d HADOOP-12317. Applications fail on NM restart on some linux distro because NM container recovery declares AM container as LOST (adhoot via rkanter)
(cherry picked from commit 1e06299df8)
2015-08-19 19:01:03 -07:00
Arpit Agarwal 10a0ddbac3 HDFS-8911. NameNode Metric : Add Editlog counters as a JMX metric. (Contributed by Anu Engineer) 2015-08-19 09:42:48 -07:00
Tsz-Wo Nicholas Sze 7245f7fad2 HDFS-8826. In Balancer, add an option to specify the source node list so that balancer only selects blocks to move from those nodes. 2015-08-18 19:26:58 -07:00
Chris Douglas 79af15fd37 HDFS-8435. Support CreateFlag in WebHDFS. Contributed by Jakob Homan 2015-08-18 17:33:21 -07:00
Benoy Antony d84e4a90d0 hadoop-12050. Enable MaxInactiveInterval for hadoop http auth token. Contributed by Huizhi Lu.
(cherry picked from commit 71aedfabf3)
2015-08-18 13:53:47 -07:00
Allen Wittenauer 40d8faf868 HADOOP-9891. CLIMiniCluster instructions fail with MiniYarnCluster ClassNotFoundException (Darrell Taylor via aw)
(cherry picked from commit 4d8fb8c19c)
2015-08-18 12:32:09 -07:00
Arpit Agarwal 85cb596e2e HDFS-8880. NameNode metrics logging. (Arpit Agarwal) 2015-08-17 16:55:24 -07:00
Xiaoyu Yao 27ccbd51f6 HDFS-8883. NameNode Metrics : Add FSNameSystem lock Queue Length. Contributed by Anu Engineer.
(cherry picked from commit a7862d5fe4)
2015-08-17 10:23:56 -07:00
Vinayakumar B eef0526127 HADOOP-7139. Allow appending to existing SequenceFiles (Contributed by kanaka kumar avvaru)
Moved to 2.6.1

(cherry picked from commit fc508b41a9)
2015-08-14 15:55:45 +05:30
Vinayakumar B 796b94df1e HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification pipe is full (zhaoyunjiong via cmccabe)
Moved to 2.6.1

(cherry picked from commit 05ed69058f)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
2015-08-14 12:55:25 +05:30
Vinayakumar B a6ec5c3dec HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu.
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit e7aa81394d)
2015-08-14 12:24:29 +05:30
Vinayakumar B e0b1744de9 HADOOP-10786. Fix UGI#reloginFromKeytab on Java 8. Contributed by Stephen Chu.
Moved CHANGES.txt entry to 2.6.1

(cherry picked from commit d6050f06a3b7e049541b1cb4597c388abf00a5be)
2015-08-14 12:10:22 +05:30
Arpit Agarwal 25c4dab891 HADOOP-12322. Typos in rpcmetrics.java. (Contributed by Anu Engineer) 2015-08-13 21:17:39 -07:00
yliu 073cb16d81 HADOOP-12295. Improve NetworkTopology#InnerNode#remove logic. (yliu) 2015-08-13 16:42:40 +08:00
cnauroth bf8b09d4b8 HADOOP-12258. Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression. Contributed by Zhihai Xu.
(cherry picked from commit 6cc8e38db5)
2015-08-12 16:45:05 -07:00
Aaron T. Myers 43f386e748 HADOOP-12318. Expose underlying LDAP exceptions in SaslPlainServer. Contributed by Mike Yoder.
(cherry picked from commit 28ea4c068ea0152af682058b02311bba81780770)
2015-08-12 15:16:36 -07:00
Andrew Wang d53296281a HDFS-8887. Expose storage type and storage ID in BlockLocation.
(cherry picked from commit 1ea1a8334e)
2015-08-11 23:33:06 -07:00
Xuan 2ebdf5bfce YARN-3999. RM hangs on draing events. Contributed by Jian He
(cherry picked from commit 3ae716fa69)
2015-08-11 18:26:00 -07:00
Jian He 5950c1f6f8 HADOOP-11932. MetricsSinkAdapter may hang when being stopped. Contributed by Brahma Reddy Battula
(cherry picked from commit f59612edd7)
2015-08-05 16:14:09 -07:00
cnauroth b83d475252 HADOOP-12304. Applications using FileContext fail with the default file system configured to be wasb/s3/etc. Contributed by Chris Nauroth.
(cherry picked from commit 3da0bedaec)
2015-08-05 14:56:54 -07:00
Colin Patrick Mccabe e2d46ac5a4 HADOOP-12302. Fix native compilation on Windows after HADOOP-7824 (Vinayakumar B via Colin P. McCabe)
(cherry picked from commit 4ab49a4365)
2015-08-05 10:01:32 -07:00
Colin Patrick Mccabe e92107b18f HADOOP-12280. Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe)
(cherry picked from commit 6f83274afc)
2015-08-05 09:43:03 -07:00
Akira Ajisaka e55eb6f20c HADOOP-12274. Remove direct download link from BULIDING.txt. Contributed by Caleb Severn.
(cherry picked from commit 52f3525586)
2015-08-05 17:47:43 +09:00
Lei Xu f7b0f292e7 HADOOP-12269. Update aws-sdk dependency to 1.10.6 (Thomas Demoor via Lei (Eddy) Xu)
(cherry picked from commit d5403747b5)
2015-08-04 18:56:07 -07:00
Zhihai Xu b9ec7f15d0 HADOOP-12268. AbstractContractAppendTest#testRenameFileBeingAppended misses rename operation. Contributed by Zhihai Xu
(cherry picked from commit a628a6f126)
2015-07-31 21:47:34 -07:00
Colin Patrick Mccabe 7d68f17731 HADOOP-12183. Annotate the HTrace span created by FsShell with the command-line arguments passed by the user (Masatake Iwasaki via Colin P. McCabe)
(cherry picked from commit 454da959c7)
2015-07-31 15:14:35 -07:00
Colin Patrick Mccabe 07a042aa3c HADOOP-7824. NativeIO.java flags and identifiers must be set correctly for each platform, not hardcoded to their Linux values (Martin Walsh via Colin P. McCabe)
(cherry picked from commit 21d10ccc6e)
2015-07-31 15:02:01 -07:00
Harsh J 68034bd6c2 HADOOP-12271. Hadoop Jar Error Should Be More Explanatory. Contributed by Josh Elser.
(cherry picked from commit 2087eaf684)
2015-07-31 10:13:07 +05:30
Jason Lowe e6b4714d0b HADOOP-10945. 4-digit octal umask permissions throws a parse error. Contributed by Chang Li
(cherry picked from commit 6f0a35724f)
2015-07-29 15:25:59 +00:00
Akira Ajisaka dff49aee08 Revert "HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)"
This reverts commit 17319c4a65.
2015-07-29 16:52:54 +09:00
Jakob Homan 71a5aa267d HDFS-8180. AbstractFileSystem Implementation for WebHdfs. Contributed by Sathosh G Nayak.
(cherry picked from commit 0712a8103f)
2015-07-28 21:11:02 -07:00
Colin Patrick Mccabe 5875935669 HADOOP-12175. FsShell must load SpanReceierHost to support tracing (Masatake Iwasaki via Colin P. McCabe)
(cherry picked from commit 69b095730b)
2015-07-28 12:56:36 -07:00
Arun Suresh ffe6a34d5b HDFS-7858. Improve HA Namenode Failover detection on the client. (asuresh)
(cherry picked from commit 030fcfa99c)
2015-07-27 23:03:34 -07:00
Akira Ajisaka c650ab0037 HADOOP-12245. References to misspelled REMAINING_QUATA in FileSystemShell.md. Contributed by Gabor Liptak.
(cherry picked from commit e21dde501a)
2015-07-28 11:34:16 +09:00
Allen Wittenauer cf348f2eb8 HADOOP-11807. add a lint mode to releasedocmaker (ramtin via aw)
(cherry picked from commit 8e657fba2f)
(cherry picked from commit 098ba450cc)
2015-07-24 18:32:05 -07:00
Colin Patrick Mccabe 2419711369 HADOOP-12170. hadoop-common's JNIFlags.cmake is redundant and can be removed (Alan Burlison via Colin P. McCabe)
(cherry picked from commit e4b0c74434)
2015-07-24 13:23:51 -07:00
Allen Wittenauer fed9f767b1 HADOOP-11813. releasedocmaker.py should use today's date instead of unreleased (Darrell Taylor via aw)
(cherry picked from commit f325522c14)
2015-07-24 11:55:57 -07:00
Allen Wittenauer 2a0688fdb8 HADOOP-11797. releasedocmaker.py needs to put ASF headers on output (aw)
(cherry picked from commit 8d3c0f601d)
2015-07-24 11:55:56 -07:00
Robert Kanter 731ed9cad8 HADOOP-12259. Utility to Dynamic port allocation (brahmareddy via rkanter)
(cherry picked from commit ee233ec95c)
2015-07-24 09:42:04 -07:00
Akira Ajisaka 17319c4a65 HDFS-6682. Add a metric to expose the timestamp of the oldest under-replicated block. (aajisaka)
(cherry picked from commit 02c01815ec)
2015-07-24 11:39:16 +09:00
Andrew Wang cf258b6d98 HADOOP-12189. Improve CallQueueManager#swapQueue to make queue elements drop nearly impossible. Contributed by Zhihai Xu.
(cherry picked from commit 6736a1ab70)
2015-07-23 14:42:44 -07:00
Allen Wittenauer a37285620d HADOOP-11731. Rework the changelog and releasenotes (aw)
(cherry picked from commit f383fd9b6c)

 Conflicts:
	hadoop-project/src/site/site.xml
2015-07-23 14:08:10 -07:00
Arpit Agarwal 601b503895 HADOOP-12161. Add getStoragePolicy API to the FileSystem interface. (Contributed by Brahma Reddy Battula) 2015-07-23 10:14:01 -07:00
Colin Patrick Mccabe d9876e69ee HADOOP-12184. Remove unused Linux-specific constants in NativeIO (Martin Walsh via Colin P. McCabe)
(cherry picked from commit 1b3bceb58c)
2015-07-22 11:35:14 -07:00
cnauroth bb080b32d4 HADOOP-12239. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB. Contributed by Duo Xu.
(cherry picked from commit efa97243ec)
2015-07-22 11:17:00 -07:00
Vinayakumar B 03d68b5575 HADOOP-12017. Hadoop archives command should use configurable replication factor when closing (Contributed by Bibin A Chundatt)
(cherry picked from commit 94c6a4aa85)
2015-07-22 10:27:03 +05:30
cnauroth aab1c14df6 HDFS-8773. Few FSNamesystem metrics are not documented in the Metrics page. Contributed by Rakesh R.
(cherry picked from commit a26cc66f38)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-07-21 14:13:15 -07:00
cnauroth 57d55d40dd HDFS-8721. Add a metric for number of encryption zones. Contributed by Rakesh R.
(cherry picked from commit cb03768b1b)
2015-07-21 13:56:07 -07:00
Vinayakumar B 8ef3821356 HADOOP-12214. Parse 'HadoopArchive' commandline using cli Options. (Contributed by Vinayakumar B)
(cherry picked from commit 87f29c6b8a)
2015-07-21 13:13:31 +05:30
Akira Ajisaka d591f2af5d HADOOP-12081. Fix UserGroupInformation.java to support 64-bit zLinux. (aajisaka)
(cherry picked from commit 773c670943)
2015-07-21 11:22:36 +09:00
Steve Loughran d3c23c7b18 HADOOP-11893. Mark org.apache.hadoop.security.token.Token as @InterfaceAudience.Public. (Brahma Reddy Battula via stevel) 2015-07-20 13:22:03 +01:00
Steve Loughran 577acf89f8 HADOOP-12051. ProtobufRpcEngine.invoke() should use Exception.toString() over getMessage() in logging/span events. (Varun Saxena via stevel) 2015-07-20 13:13:09 +01:00
Steve Loughran d68b044f0f HADOOP-12088. KMSClientProvider uses equalsIgnoreCase("application/json"). (Brahma Reddy Battula via stevel) 2015-07-20 13:02:51 +01:00
Steve Loughran 208b9eed9e HADOOP-12209 Comparable type should be in FileStatus. (Yong Zhang via stevel) 2015-07-20 12:32:32 +01:00
Steve Loughran f3296a1984 HADOOP-12235 hadoop-openstack junit & mockito dependencies should be "provided". (Ted Yu via stevel) 2015-07-20 11:22:22 +01:00
Haohui Mai 391d2d88f2 HDFS-8767. RawLocalFileSystem.listStatus() returns null for UNIX pipefile. Contributed by kanaka kumar avvaru. 2015-07-16 15:22:11 -07:00
Tsuyoshi Ozawa 90c5bf0605 HADOOP-12240. Fix tests requiring native library to be skipped in non-native profile. Contributed by Masatake Iwasaki.
(cherry picked from commit 90bda9c611)
2015-07-16 16:05:11 +09:00
Tsuyoshi Ozawa f3f7872fbf HADOOP-10615. FileInputStream in JenkinsHash#main() is never closed. Contributed by Chen He.
(cherry picked from commit 111e6a3fdf)
2015-07-16 14:09:22 +09:00
Tsuyoshi Ozawa 59ba845a2f HADOOP-12153. ByteBufferReadable doesn't declare @InterfaceAudience and @InterfaceStability. Contributed by Brahma Reddy Battula.
(cherry picked from commit cec1d43db0)
2015-07-15 14:19:29 +09:00
cnauroth 5e8c01e7a2 HADOOP-12232. Upgrade Tomcat dependency to 6.0.44. Contributed by Chris Nauroth.
(cherry picked from commit 6b56f05b9ea3ee45abd5d00b14fab93a353b1332)
2015-07-14 15:22:39 -07:00
Tsuyoshi Ozawa 64059d8884 HADOOP-12191. Bzip2Factory is not thread safe. Contributed by Brahma Reddy Battula.
(cherry picked from commit 4084eaf943)
2015-07-14 19:15:36 +09:00
Chris Douglas c6b0bbabc7 HADOOP-12211. Collect disk usage on the node. Contributed by Robert Grandl
(cherry picked from commit a431ed9075)
2015-07-13 15:36:25 -07:00
Akira Ajisaka ad6787aa43 YARN-3069. Document missing properties in yarn-default.xml. Contributed by Ray Chiang.
(cherry picked from commit d6675606dc)
2015-07-13 12:57:29 +09:00
Ming Ma 5e6bbe6031 HADOOP-12107. long running apps may have a huge number of StatisticsData instances under FileSystem (Sangjin Lee via Ming Ma)
(cherry picked from commit 8e1bdc17d9)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
2015-07-10 08:32:27 -07:00
Chris Douglas 66419d3672 HADOOP-12210. Collect network usage on the node. Contributed by Robert Grandl
(cherry picked from commit 1a0752d85a)
2015-07-09 17:48:56 -07:00
Karthik Kambatla fc989ebe16 HADOOP-12180. Move ResourceCalculatorPlugin from YARN to Common. (Chris Douglas via kasha)
(cherry picked from commit ac6048372a)
2015-07-09 09:59:02 -07:00
Akira Ajisaka ee07df688a HADOOP-12200. TestCryptoStreamsWithOpensslAesCtrCryptoCodec should be skipped in non-native profile. Contributed by Masatake Iwasaki.
(cherry picked from commit 6b7692c468)
2015-07-09 13:15:37 +09:00
Colin Patrick Mccabe 016d661caf HADOOP-12201. Add tracing to FileSystem#createFileSystem and Globber#glob (cmccabe)
(cherry picked from commit b8832fcf1e)
2015-07-08 20:12:11 -07:00
Karthik Kambatla 4227fff3c6 HADOOP-11878. FileContext#fixRelativePart should check for not null for a more informative exception. (Brahma Reddy Battula via kasha)
(cherry picked from commit 2e3d83f97b)
2015-07-08 11:35:09 -07:00
Andrew Wang cc73f23713 HADOOP-12194. Support for incremental generation in the protoc plugin.
(cherry picked from commit 625d7ed9eb)
2015-07-08 11:11:07 -07:00
Andrew Wang 9bbc3e4ad4 HADOOP-12195. Add annotation to package-info.java file to workaround MCOMPILER-205.
(cherry picked from commit 3dc92e84c2)
2015-07-07 11:14:30 -07:00
Andrew Wang 54e9b904c4 HADOOP-12193. Rename Touchz.java to Touch.java.
(cherry picked from commit aa96a8c0c5)
2015-07-07 11:14:28 -07:00
Chris Douglas 62424b5d99 HADOOP-5732. Add SFTPFileSystem. Contributed by Ramtin Boustani and Inigo Goiri
(cherry picked from commit 559425dcb9)
2015-07-07 10:09:29 -07:00
Colin Patrick Mccabe 4da63e5f5f HADOOP-11974. Fix FIONREAD #include on Solaris (Alan Burlison via Colin P. McCabe)
(cherry picked from commit 81f3644376)
2015-07-07 08:36:52 -07:00
Vinayakumar B 2b99310c32 HADOOP-12186. ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times (Contributed by zhihai xu)
Moved CHANGES.txt entry to 2.7.2

(cherry picked from commit e0febce0e7)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
2015-07-07 18:16:14 +05:30
Varun Vasudev 16ca0fcf99 YARN-2194. Fix bug causing CGroups functionality to fail on RHEL7. Contributed by Wei Yan.
(cherry-picked from commit 7e3ebc0fc59dbd1d7768b5a6c05af18b8c8fea4d)
2015-07-07 17:05:06 +05:30
Vinayakumar B c5c0577248 HADOOP-12117. Potential NPE from Configuration#loadProperty with allowNullValueProperties set. (Contributed by zhihai xu)
(cherry picked from commit 99c8c5839b)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
2015-07-07 16:18:54 +05:30
Vinod Kumar Vavilapalli (I am also known as @tshooter.) a8bf34cad8 Release process for 2.7.1: Set the release date for 2.7.1.
(cherry picked from commit bf89ddb9b8)
2015-07-06 16:41:00 -07:00
Chris Douglas 360164e41e HADOOP-12185. NetworkTopology is not efficient adding/getting/removing nodes. Contributed by Inigo Goiri
(cherry picked from commit 47a69ec7a5)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/NetworkTopology.java
2015-07-06 15:15:45 -07:00
cnauroth d01eaef40f HADOOP-12045. Enable LocalFileSystem#setTimes to change atime. Contributed by Kazuho Fujii.
(cherry picked from commit ed1e3ce482)
2015-07-06 13:40:26 -07:00
Vinayakumar B 91d2c59968 HADOOP-12186. ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times (Contributed by zhihai xu)
(cherry picked from commit 233cab89ad)
2015-07-06 15:40:16 +05:30
Chris Douglas f2c0539d38 HADOOP-12173. NetworkTopology::add calls toString always. Contributed by Inigo Goiri
(cherry picked from commit e59f6fad6a)
2015-07-02 21:49:43 -07:00
cnauroth 0d94a293a8 HADOOP-12172. FsShell mkdir -p makes an unnecessary check for the existence of the parent. Contributed by Chris Nauroth.
(cherry picked from commit f3796224bf)
2015-07-01 19:48:17 -07:00
Colin Patrick Mccabe 490bef0856 HADOOP-12171. Shorten overly-long htrace span names for server (cmccabe)
(cherry picked from commit a78d5074fb)
2015-07-01 17:57:33 -07:00
Lei Xu fad478e7ce HADOOP-12164. Fix TestMove and TestFsShellReturnCode failed to get command name using reflection. (Lei Xu)
(cherry picked from commit 47fab573bcb7b1129648499571b518f7b9773f89)

Conflicts:
	hadoop-common-project/hadoop-common/CHANGES.txt
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFsShellReturnCode.java
2015-06-30 17:37:52 -07:00
cnauroth f7ba3c0045 HADOOP-12116. Fix unrecommended syntax usages in hadoop/hdfs/yarn script for cygwin in branch-2. Contributed by Li Lu. 2015-06-30 16:55:10 -07:00
Robert Kanter 3f79ee7494 HADOOP-12159. Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces (rchiang via rkanter)
(cherry picked from commit aaafa0b2ee)
2015-06-30 16:50:42 -07:00