Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,134 Commits 326 Branches 371 Tags
Commit Graph

11134 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Chris Nauroth
28a2f28e42 HADOOP-13148. TestDistCpViewFs to include IOExceptions in test error reports. Contributed by Steve Loughran. 
(cherry picked from commit b9685e85d59e69e5dd64802fa9025dad1b315be5)
(cherry picked from commit bb41547fb42b9dd2b80ec2a7286db6d35fd251b5)
 2016-05-16 11:53:33 -07:00
Jason Lowe
70faa87ccf YARN-4325. Nodemanager log handlers fail to send finished/failed events in some cases. Contributed by Junping Du 
(cherry picked from commit 81effb7dcde2b31423438d6f1b8b8204d4ca05b3)
 2016-05-16 15:43:42 +00:00
Andrew Wang
29ed65ff9b HDFS-10333. Intermittent org.apache.hadoop.hdfs.TestFileAppend failure in trunk. Contributed by Yiqun Lin. 
(cherry picked from commit 45788204ae2ac82ccb3b4fe2fd22aead1dd79f0d)
(cherry picked from commit 413f3dabbd398b7d837893963556d9e6dd9e10e2)
 2016-05-15 22:15:18 -07:00
Xuan
4eeec4e35b YARN-5080. Cannot obtain logs using YARN CLI -am for either KILLED or 
RUNNING AM. Contributed by Xuan Gong

(cherry picked from commit 7be53b65f95dae89c760cde354b42f15f5ee3b5f)
(cherry picked from commit 3fe4107909393dea91adac9fda16c45814e01648)
 2016-05-14 12:22:33 -07:00
Li Lu
6b1af6ce68 HADOOP-13083. The number of javadocs warnings is limited to 100. (Gergely Novák via gtcarrera9) 
(cherry picked from commit 3fa1380c221b9d659fb82c42284505ef19da38d0)
 2016-05-13 13:46:28 -07:00
Sangjin Lee
29699d8e7e HADOOP-12971. FileSystemShell doc should explain relative path (John Zhuge via sale) 
(cherry picked from commit 1f2794b4faf119a1ab66c8184ac84cec710d52a0)
 2016-05-13 10:03:06 -07:00
Jason Lowe
ff8caccc87 MAPREDUCE-6558. multibyte delimiters with compressed input files generate duplicate records. Contributed by Wilfred Spiegelenburg 
(cherry picked from commit 9227dfc25f373a99cb66ad7d6bacef8dcf336f77)
 2016-05-13 14:36:13 +00:00
Steve Loughran
a84850b4ca HADOOP-13113 Enable parallel test execution for hadoop-aws. Chris Nauroth via stevel 2016-05-13 10:46:15 +01:00
Andrew Wang
b5fd6dd4a8 HADOOP-12868. Fix hadoop-openstack undeclared and unused dependencies. Contributed by Masatake Iwasaki. 
(cherry picked from commit fa440a39b7883183b9cf43c056908ca7bf68e6a1)
 2016-05-12 23:58:26 -07:00
Andrew Wang
5da54eb3d4 HADOOP-11180. Change log message "token.Token: Cannot find class for token kind kms-dt" to debug. Contributed by Yi Liu. 
(cherry picked from commit 3bfd29949fef6d8fb5df4c6398e77cde320d8c2d)
(cherry picked from commit d6f6b99c83b0945c7d125a57650041d2e707ba0e)
 2016-05-12 23:49:00 -07:00
Steve Loughran
8d74a58042 HADOOP-13028 add low level counter metrics for S3A; use in read performance tests. contributed by: stevel 
patch includes
HADOOP-12844 Recover when S3A fails on IOException in read()
HADOOP-13058 S3A FS fails during init against a read-only FS if multipart purge
HADOOP-13047 S3a Forward seek in stream length to be configurable
 2016-05-12 19:23:18 +01:00
Steve Loughran
9b8f7a8fb2 MAPREDUCE-6639 Process hangs in LocatedFileStatusFetcher if FileSystem.get throws. Ryan Blue via stevel 2016-05-12 18:59:56 +01:00
Steve Loughran
9cbb942465 HADOOP-13116 Jets3tNativeS3FileSystemContractTest does not run. 2016-05-12 16:36:10 +01:00
Steve Loughran
af5e5d6d47 HADOOP-13122 Customize User-Agent header sent in HTTP requests by S3A. Chris Nauroth via stevel. 2016-05-12 13:56:48 +01:00
Karthik Kambatla
a9b0041abc YARN-4995. FairScheduler: Display per-queue demand on the scheduler page. (xupeng via kasha) 
(cherry picked from commit 4b4e4c6ba83bc5c41d7bb69bb2483bcfe894a260)
(cherry picked from commit f96b2b6fac84f565d4d0cf78c927f0b116364cfc)
 2016-05-11 17:40:57 -07:00
Colin Patrick Mccabe
5f04795aed HADOOP-13065. Add a new interface for retrieving FS and FC Statistics (Mingliang Liu via cmccabe) 
(cherry picked from commit 687233f20d24c29041929dd0a99d963cec54b6df)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java

(cherry picked from commit 15bfcde23832189accce8293a053c8f0ef1aeaae)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
 2016-05-11 14:24:51 -07:00
Varun Vasudev
4d275d0a25 HADOOP-12749. Create a threadpoolexecutor that overrides afterExecute to log uncaught exceptions/errors. Contributed by Sidharta Seethana. 
(cherry picked from commit d37eb828ffa09d55936964f555ea351b946d286e)
(cherry picked from commit 2451a0a8e08f122061b2defd0ba5b832f8689e53)
 2016-05-11 14:12:18 -07:00
Larry McCay
ee2c2afdc3 HADOOP-12942. hadoop credential commands non-obviously use password of "none" (Mike Yoder via lmccay) 2016-05-11 14:39:13 -04:00
Chris Nauroth
e9ee258d04 HADOOP-13008. Add XFS Filter for UIs to Hadoop Common. Contributed by Larry McCay. 
(cherry picked from commit dee279b532e7286362518b531c9daea9ae8606f4)
(cherry picked from commit 31279ae45eeb74d0955014ceffacb5c40d2f3ee5)
 2016-05-11 10:58:56 -07:00
Junping Du
6d1300880f YARN-5029. RM needs to send update event with YarnApplicationState as Running to ATS/AHS. Contributed by Xuan Gong. 
(cherry picked from commit 39f2bac38b111f90d3402229201cdb4315f5d4f5)
(cherry picked from commit 9dc3683d87912d9b56a69557360c19a5896d0e51)
 2016-05-11 09:40:51 -07:00
Steve Loughran
ed2230ec7d HADOOP-13125 FS Contract tests don't report FS initialization errors well. (stevel) 2016-05-11 14:19:12 +01:00
Chris Nauroth
bbd463b3b2 HADOOP-12801. Suppress obsolete S3FileSystem tests. Contributed by Chris Nauroth. 
(cherry picked from commit d3dbfa7feee7922ddcd0b2e97898adf4e0c37de3)
(cherry picked from commit ba4883443141354fa457887fb2501226f6a0fa1f)
 2016-05-10 15:14:42 -07:00
Eric Payne
235caef236 YARN-4768. getAvailablePhysicalMemorySize can be inaccurate on linux. Contributed by Nathan Roberts. 
(cherry picked from commit 6b1c1cb01cbf979f46cd3ea9308b7745c5595b4f)
 2016-05-10 21:45:37 +00:00
Steve Loughran
11b170adc8 HADOOP-12982 Document missing S3A and S3 properties. (Wei-Chiu Chuang via stevel) 2016-05-10 21:40:23 +01:00
Steve Loughran
d13cd394e5 HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple. (Bolke de Bruin via stevel). 2016-05-10 21:33:26 +01:00
Colin Patrick Mccabe
154332bb54 HDFS-10377. CacheReplicationMonitor shutdown log message should use INFO level (Yiqun Lin via cmccabe) 
(cherry picked from commit f0252ad2525c90b2b89ae2ff1de12482da2c8ddd)
(cherry picked from commit 43f32d3f4a86d8f2a302530831003496c78fd561)
 2016-05-10 11:21:16 -07:00
Akira Ajisaka
a84c4d7d4b HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated. Contributed by Gergely Novák. 
(cherry picked from commit 901588542b573cbefa91f7e0e6032f7163160a85)
 2016-05-10 09:33:55 -07:00
Akira Ajisaka
1622fc1373 Revert "HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated." 
This reverts commit f25dcb935e9dbc0ad2dfe4466302d8b65868e172.
 2016-05-10 09:33:18 -07:00
Akira Ajisaka
f25dcb935e HDFS-6187. Update the document of hftp/hsftp in branch-2 to mention that they are deprecated. 
(cherry picked from commit c297df59951dcde44474fed341fa495bb6308936)
 2016-05-10 09:29:47 -07:00
Jason Lowe
da48923d3a MAPREDUCE-6684. High contention on scanning of user directory under immediate_done in Job History Server. Contributed by Haibo Chen 
(cherry picked from commit 5ffb54694b52657f3b7de4560474ab740734e1b2)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/HistoryFileManager.java
 2016-05-10 16:16:32 +00:00
Eric Payne
730ce76160 MAPREDUCE-6678. Allow ShuffleHandler readahead without drop-behind. Contributed by Nathan Roberts. 
(cherry picked from commit cd35b692de88e3afe7f41405da635c3fbd9b4650)
 2016-05-10 16:07:16 +00:00
Kihwal Lee
d7cda6d031 HDFS-10372. Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume. Contributed by Rushabh Shah. 
(cherry picked from commit b9e5a32fa14b727b44118ec7f43fb95de05a7c2c)
 2016-05-09 15:52:12 -05:00
Naganarasimha
b47259caf0 YARN-4926. Change nodelabel rest API invalid reponse status to 400. Contributed by Bibin A Chundatt 
(cherry picked from commit 2750fb900fc9c468570d051cb19e3a379a898c38)
 2016-05-08 23:06:15 +05:30
Yongjun Zhang
5ae5b869aa YARN-5048. DelegationTokenRenewer#skipTokenRenewal may throw NPE (Jian He via Yongjun Zhang) 
(cherry picked from commit 47c41e7ac7e6b905a58550f8899f629c1cf8b138)
 2016-05-06 22:43:44 -07:00
Andrew Wang
8653bcbe67 HDFS-9939. Increase DecompressorStream skip buffer size. Contributed by John Zhuge. 
(cherry picked from commit 8ef75a702e3b4b8ca7794421c44da7f8d44daa4c)
 2016-05-06 17:26:51 -07:00
Jason Lowe
da24fe95f8 YARN-4747. AHS error 500 due to NPE when container start event is missing. Contributed by Varun Saxena 
(cherry picked from commit b2ed6ae73197990a950ce71ece80c0f23221c384)
 2016-05-06 23:00:53 +00:00
Jason Lowe
7fc44311d0 MAPREDUCE-6689. MapReduce job can infinitely increase number of reducer resource requests. Contributed by Wangda Tan 
(cherry picked from commit c9bb96fa81fc925e33ccc0b02c98cc2d929df120)
 2016-05-06 22:28:03 +00:00
Karthik Kambatla
367e757d8d YARN-4947. Test timeout is happening for TestRMWebServicesNodes. Contributed by Bibin A Chundatt 2016-05-06 14:28:24 +05:30
Masatake Iwasaki
c80b47e73f HDFS-2043. TestHFlush failing intermittently. Contributed by Lin Yiqun. 
(cherry picked from commit 2835f146b551f911ea13307e7762759813898757)
 2016-05-06 15:50:48 +09:00
Vinod Kumar Vavilapalli
921562720e MAPREDUCE-6514. Fixed MapReduce ApplicationMaster to properly updated resources ask after ramping down of all reducers avoiding job hangs. Contributed by Varun Saxena and Wangda Tan. 
(cherry picked from commit 8d48266720dcf0e71cfd87fef18b60a53aa1bef9)
 2016-05-05 20:43:55 -07:00
Xiaoyu Yao
59eeee7501 HDFS-10324. Trash directory in an encryption zone should be pre-created with correct permissions. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit dacd1f50feb24ccdf6155b2b7a6126fe21a47ad0)
(cherry picked from commit 35859872b7e339ac0b57f8001c349afe027ee4d2)
 2016-05-05 18:23:00 -07:00
Xiaoyu Yao
4d0b19eec2 HADOOP-13098. Dynamic LogLevel setting page should accept case-insensitive log level string. (Junping Du via xyao) 
(cherry picked from commit 4e5e87ddd4a47dbea2b23387782e7cd47dec560e)
 2016-05-05 18:01:38 -07:00
Tsz-Wo Nicholas Sze
9daa36b8ea HDFS-10346: add org.apache.hadoop.util.StringUtils import. 2016-05-05 15:58:06 -07:00
Tsz-Wo Nicholas Sze
528240f593 HADOOP-13103 Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException 2016-05-05 15:53:51 -07:00
Wangda Tan
3620d0e623 YARN-4984. LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak. (Junping Du via wangda) 
(cherry picked from commit 7bd418e48c71590fc8026d69f9b8f8ad42f2aade)
 2016-05-05 10:08:37 -07:00
Jason Lowe
b452ecf4a0 YARN-4311. Removing nodes from include and exclude lists will not remove them from decommissioned nodes list. Contributed by Kuhu Shukla 
(cherry picked from commit ee86cef2fee992fb773e87c895000ffc2ee788b0)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2016-05-05 14:40:35 +00:00
Ming Ma
d6e95ae47b HDFS-10320. Rack failures may result in NN terminate. (Xiao Chen via mingma) 
(cherry picked from commit 1268cf5fbe4458fa75ad0662512d352f9e8d3470)
 2016-05-04 17:07:36 -07:00
Junping Du
22ac37615a YARN-4920. ATS/NM should support a link to dowload/get the logs in text format. Contributed by Xuan Gong. 
(cherry picked from commit 3cf223166d452a0f58f92676837a9edb8ddc1139)
(cherry picked from commit c79dc07dc193904f2586a5d64ea2f4e56d2396b8)
 2016-05-04 09:49:08 -07:00
Masatake Iwasaki
d53121da36 HADOOP-12101. Add automatic search of default Configuration variables to TestConfigurationFieldsBase. Contributed by Ray Chiang. 
(cherry picked from commit 355325bcc7111fa4aac801fd23a26422ffabaf7c)
 2016-05-04 15:22:55 +09:00
Arpit Agarwal
ce94203566 HDFS-9902. Support different values of dfs.datanode.du.reserved per storage type. (Contributed by Brahma Reddy Battula) 2016-05-03 16:53:07 -07:00