Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,414 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

11414 Commits

Author SHA1 Message Date
Steve Loughran 1370dfc775 HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel 
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
 2016-05-12 19:23:31 +01:00
Steve Loughran eab6e633ee MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 19:00:08 +01:00
Sangjin Lee d6d13ec670 YARN-4577. Enable aux services to have their own custom classpath/jar file (Xuan Gong via sale) 
(cherry picked from commit 0bbe01f8d5)
 2016-05-12 10:14:24 -07:00
Steve Loughran 389dd91dd4 HADOOP-13116 Jets3tNativeS3FileSystemContractTest does not run. 2016-05-12 16:36:23 +01:00
Steve Loughran 257b2f5beb HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel. 2016-05-12 13:57:05 +01:00
Tsz-Wo Nicholas Sze 69768bf9e0 HDFS-10346. Implement asynchronous setPermission/setOwner for DistributedFileSystem. Contributed by Xiaobing Zhou 2016-05-11 17:57:20 -07:00
Karthik Kambatla f96b2b6fac YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba8)
 2016-05-11 17:36:42 -07:00
Colin Patrick Mccabe 15bfcde238 HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe) 
(cherry picked from commit 687233f20d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
 2016-05-11 14:02:03 -07:00
Larry McCay 788f80b617 HADOOP-12942. hadoop credential commands non-obviously use password of "none" (Mike Yoder via lmccay) 2016-05-11 14:38:21 -04:00
Chris Nauroth 31279ae45e HADOOP-13008. Add XFS Filter for UIs to Hadoop Common. Contributed by Larry McCay. 
(cherry picked from commit dee279b532)
 2016-05-11 10:58:42 -07:00
Junping Du 9dc3683d87 YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b)
 2016-05-11 09:33:16 -07:00
Steve Loughran 0ff3ec1f67 HADOOP-13125 FS Contract tests don't report FS initialization errors well. (stevel) 2016-05-11 14:19:41 +01:00
Tsuyoshi Ozawa eb4182f8f5 HADOOP-10694. Remove synchronized input streams from Writable deserialization. Contributed by Gopal V and Rajesh Balamohan. 
(cherry picked from commit 6e56578031)
 2016-05-10 17:27:43 -07:00
Chris Nauroth ba48834431 HADOOP-12801. Suppress obsolete S3FileSystem tests. Contributed by Chris Nauroth. 
(cherry picked from commit d3dbfa7feee7922ddcd0b2e97898adf4e0c37de3)
 2016-05-10 15:14:36 -07:00
Eric Payne de4daf9d22 YARN-4768. getAvailablePhysicalMemorySize can be inaccurate on linux. Contributed by Nathan Roberts. 
(cherry picked from commit 6b1c1cb01c)
 2016-05-10 21:37:24 +00:00
Steve Loughran f47ea0cc3f HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel) 2016-05-10 21:40:13 +01:00
Steve Loughran 092b199741 HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple. (Bolke de Bruin via stevel). 2016-05-10 21:33:11 +01:00
Colin Patrick Mccabe 43f32d3f4a HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 
(cherry picked from commit f0252ad252)
 2016-05-10 11:20:46 -07:00
Akira Ajisaka 7cbf306e4c HADOOP-13118. Fix IOUtils#cleanup and IOUtils#closeStream javadoc. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 0f0c6415af)
 2016-05-10 10:03:41 -07:00
Harsh J e2216a96d7 HADOOP-7256. Resource leak during failure scenario of closing of resources. Contributed by Ramkrishna S. Vasudevan. (harsh) 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1388893 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 28023b7759)
 2016-05-10 09:49:00 -07:00
Akira Ajisaka 901588542b HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated. Contributed by Gergely Novák. 2016-05-10 09:32:33 -07:00
Akira Ajisaka a5bfb2dd25 Revert "HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated." 
This reverts commit c297df5995.
 2016-05-10 09:32:20 -07:00
Akira Ajisaka c297df5995 HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated. 2016-05-10 09:29:30 -07:00
Jason Lowe d3dfed3fd0 MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b)
 2016-05-10 16:06:26 +00:00
Eric Payne 77d147e0ec MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 
(cherry picked from commit cd35b692de)
 2016-05-10 16:04:09 +00:00
Vinod Kumar Vavilapalli e0d6a9632d YARN-4842. Fixed "yarn logs" command to guess (and thus not require) the appOwner argument when viewing another user's logs. Contributed by Ram Venkatesh and Xuan Gong. 
(cherry picked from commit 87f5e35133)
 2016-05-09 22:47:58 -07:00
Jian He 007d6d1d57 YARN-4778. Support specifying resources for task containers in SLS. Contributed by Wangda Tan 
(cherry picked from commit 996a210ab0)
 2016-05-09 14:50:00 -07:00
Kihwal Lee 2c782fdb17 HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah. 
(cherry picked from commit b9e5a32fa1)
 2016-05-09 15:51:42 -05:00
Naganarasimha 3732a1e985 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900f)
 2016-05-08 23:02:07 +05:30
Yongjun Zhang 5172d0e7b1 YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7)
 2016-05-06 22:38:56 -07:00
Andrew Wang 1d563f4d7e HDFS-9939. Increase DecompressorStream skip buffer size. Contributed by John Zhuge. 
(cherry picked from commit 8ef75a702e)
 2016-05-06 17:23:31 -07:00
Andrew Wang c328a1ef5e HADOOP-12701. Run checkstyle on test source files. Contributed by John Zhuge. 
(cherry picked from commit e097c9a124)
 2016-05-06 17:21:04 -07:00
Jason Lowe 3895058a67 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae731)
 2016-05-06 23:00:25 +00:00
Wangda Tan b68e6b1d6d getApplicationReport call may raise NPE for removed queues. (Jian He via wangda) 
(cherry picked from commit 23248f63aa)
 2016-05-06 15:32:15 -07:00
Jason Lowe d1aa3e0f89 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81)
 2016-05-06 22:27:24 +00:00
Masatake Iwasaki 4ecbe2e9bf HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 
(cherry picked from commit 2835f146b5)
 2016-05-06 15:47:51 +09:00
Vinod Kumar Vavilapalli e9c7a17bc8 MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720)
 2016-05-05 19:05:44 -07:00
Xiaoyu Yao 35859872b7 HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit dacd1f50fe)
 2016-05-05 18:15:32 -07:00
Xiaoyu Yao d18a991817 HADOOP-13098. Dynamic LogLevel setting page should accept case-insensitive log level string. (Junping Du via xyao) 
(cherry picked from commit 4e5e87ddd4)
 2016-05-05 17:58:41 -07:00
Tsz-Wo Nicholas Sze b66b85094a HADOOP-13103 Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException 2016-05-05 15:50:50 -07:00
Jian He a6b24c62ab YARN-4390. Do surgical preemption based on reserved container in CapacityScheduler. Contributed by Wangda Tan 
(cherry picked from commit bb62e05925)
 2016-05-05 12:56:55 -07:00
Jason Lowe ee86cef2fe YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit d0da13229c)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNode.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/rmnode/RMNodeImpl.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/MockNodes.java
 2016-05-05 14:33:01 +00:00
Varun Vasudev 38a3b86141 YARN-4595. Add support for configurable read-only mounts when launching Docker containers. Contributed by Billie Rinaldi. 
(cherry picked from commit 72b047715c)
 2016-05-05 13:02:38 +05:30
Robert Kanter f9d6092108 MAPREDUCE-6675. TestJobImpl.testUnusableNode failed (haibochen via rkanter) 
(cherry picked from commit 9d3fcdfbb3)
 2016-05-04 22:42:34 -07:00
Robert Kanter cca8e9b90c MAPREDUCE-6677. LocalContainerAllocator doesn't specify resource of the containers allocated (haibochen via rkanter) 
(cherry picked from commit ecce3b7d53)
 2016-05-04 22:19:42 -07:00
Ming Ma 5b53833171 HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma) 
(cherry picked from commit 1268cf5fbe)
 2016-05-04 17:04:44 -07:00
Vinod Kumar Vavilapalli 8262ef8318 YARN-4905. Improved "yarn logs" command-line to optionally show log metadata also. Contributed by Xuan Gong. 
(cherry picked from commit 9e37fe3b7a)
 2016-05-04 14:17:09 -07:00
Wangda Tan 585299146a YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c)
 2016-05-04 11:39:25 -07:00
Junping Du 1ffb0c43d6 YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
 2016-05-04 10:36:31 -07:00
Jing Zhao d43d8a1bcd HADOOP-12469. distcp should not ignore the ignoreFailures option. Contributed by Mingliang Liu. 
(cherry picked from commit af942585a1)
 2016-05-04 10:24:06 -07:00