Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,371 Commits 326 Branches 371 Tags
Commit Graph

9371 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Vinayakumar B
d21bc811d8 HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 48ca23def1d1c28448a65238814070e79c8f4c4e)
 2015-06-10 11:07:04 +05:30
Arpit Agarwal
2d67dff15f HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:46 -07:00
Xiaoyu Yao
d8d9041187 HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R. 
(cherry picked from commit 3c2397cb041068172c8ad209996414d7371d2eca)
 2015-06-09 12:21:53 -07:00
Zhijie Shen
0dad10d845 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong. 
(cherry picked from commit 8d0ef3163287d811381e94cc0101dbc9847ee2db)
 2015-06-08 23:21:35 -07:00
cnauroth
f5b0cce7fa HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic. 
(cherry picked from commit c45784bc9031353b938f4756473937cca759b3dc)
 2015-06-08 22:42:24 -07:00
Xiaoyu Yao
116a72096d HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula 
(cherry picked from commit 927577c87ca19e8b5b75722f78e2def6d9386576)
 2015-06-08 22:03:07 -07:00
cnauroth
5a002aef12 MAPREDUCE-6392. Document mapred class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 025a3a8be07ba2b0c27ee39034b506d91aa486fc)
 2015-06-08 16:22:12 -07:00
cnauroth
d9016b0286 YARN-3786. Document yarn class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit a531b058aef48c9bf2e5366ed110e1f817316c1a)
 2015-06-08 16:18:43 -07:00
cnauroth
cc960734e4 HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d2832b3d4243c6c470c774bc33fd13f70b3e7b72)
 2015-06-08 16:15:53 -07:00
Arpit Agarwal
82e772bdbb HADOOP-12054. RPC client should not retry for InvalidToken exceptions. (Contributed by Varun Saxena) 2015-06-08 15:45:25 -07:00
Xuan
869304dc83 YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2b2465dfac1f147b6bb20d878b69a8cc3e85c8ad)
 2015-06-08 15:44:25 -07:00
Tsuyoshi Ozawa
c6cdecc6b3 MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang. 
(cherry picked from commit 6afe20a7a4bbfa30fce3e3c9873ad43201987998)
 2015-06-08 15:06:23 -07:00
Jian He
a24ead8c6d YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla 
(cherry picked from commit 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f)
 2015-06-08 14:51:25 -07:00
cnauroth
8ee50d8ca7 HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit 0e80d51983942dca3348c8a8401bad3ecbaab010)
 2015-06-08 08:39:14 -07:00
Steve Loughran
c3c2b4d318 HADOOP-12052 IPC client downgrades all exception types to IOE, breaks callers trying to use them. (Brahma Reddy Battula via stevel) 2015-06-08 13:02:26 +01:00
Devaraj K
8c643e3bf9 YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.

(cherry picked from commit 126321eded7dc38c1eef2cfde9365404c924a5cb)
 2015-06-08 15:33:38 +05:30
Devaraj K
2a01c01af7 YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.

(cherry picked from commit c7ee6c151c5771043a6de3b8a951cea13f59dd7b)
 2015-06-08 11:56:58 +05:30
cnauroth
953c4a049a HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer. 
(cherry picked from commit a6cb4894b45b0b909aca716058590dd6f0e3a6f9)
 2015-06-07 13:02:01 -07:00
Karthik Kambatla
322e7d0d5e YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha) 
(cherry picked from commit bd69ea408f8fdd8293836ce1089fe9b01616f2f7)
 2015-06-07 11:38:14 -07:00
Xiaoyu Yao
c8dc113943 HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b61b48999530851da177db2e8998858ffc01f77e)
 2015-06-06 18:48:52 -07:00
cnauroth
f99744b7a0 HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. 
(cherry picked from commit 71de367c5e80ea76d1e8d21f0216cd6b879dcee5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
 2015-06-06 09:45:46 -07:00
cnauroth
1d1841d40f HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang. 
(cherry picked from commit e46cb800028c95f9bce575d05268cd10d0913222)
 2015-06-06 09:17:11 -07:00
Xiaoyu Yao
1a6cdf97bb HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:33:06 -07:00
Jason Lowe
588c22e5a0 MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li 
(cherry picked from commit b3ffa870034d06608a1946e2d9ce7dbd535a2c53)
 2015-06-05 22:39:52 +00:00
Andrew Wang
1137fd4c5b HADOOP-12056. Use DirectoryStream in DiskChecker#checkDirs to detect errors when listing a directory. Contributed by Zhihai Xu. 
(cherry picked from commit bc11e158b1726174fae2c7d2e8aa1f5005bf42eb)
 2015-06-05 13:52:43 -07:00
Andrew Wang
83a97907cc HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey. 
(cherry picked from commit 2dbc40e6086026ef02747223982aa68f2d328ade)
 2015-06-05 13:11:12 -07:00
Xuan
4d385b48f0 YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 
(cherry picked from commit 3e000a919fede85230fcfb06309a1f1d5e0c479c)
 2015-06-05 12:49:51 -07:00
Karthik Kambatla
429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 
(cherry picked from commit 75885852cc19dd6de12e62498b112d5d70ce87f4)
 2015-06-05 09:42:12 -07:00
Karthik Kambatla
1a2e6e8091 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
 2015-06-05 09:17:34 -07:00
Devaraj K
37ad3b3fef MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.

(cherry picked from commit 09b8241e6e369ee00241efcc2e048e710dbcccd1)
 2015-06-05 21:23:48 +05:30
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Andrew Wang
320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486ed99e1433d4e5118608da8dd365a934)
 2015-06-04 15:35:12 -07:00
Zhijie Shen
8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 
(cherry picked from commit 18dd01d6bf67f4d522b947454c1f4347d1cbbc19)
 2015-06-04 14:48:05 -07:00
Wangda Tan
344b750915 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c32d8fd8a83e46e686eacd14e5a533c)
 2015-06-04 13:20:49 -07:00
Gera Shegalov
2cc2488164 HADOOP-11924. Tolerate JDK-8047340-related exceptions in Shell#isSetSidAvailable preventing class init. (Tsuyoshi Ozawa via gera) 
(cherry picked from commit 9f6d67844da3c6ca1de66d426790e741e77f8038)
 2015-06-04 11:47:22 -07:00
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Jian He
2e99210e4e YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran 
(cherry picked from commit 1970ca7cbcdb7efa160d0cedc2e3e22c1401fad6)
 2015-06-04 11:14:36 -07:00
Andrew Wang
ccc834395b HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R. 
(cherry picked from commit 5149dc7b975f0e90a14e3da02685594028534805)
 2015-06-04 11:09:28 -07:00
Arpit Agarwal
75e962f367 HADOOP-11994. smart-apply-patch wrongly assumes that git is infallible. (Contributed by Kengo Seki) 2015-06-04 10:54:12 -07:00
Jian He
6325e4b7dd YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 
(cherry picked from commit 6ad4e59cfc111a92747fdb1fb99cc6378044832a)
 2015-06-04 10:53:24 -07:00
Kihwal Lee
f0f277ddd0 HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. 
(cherry picked from commit ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)
 2015-06-04 12:52:49 -05:00
Wangda Tan
78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48fe236b404cf3a125ac9d1f7714e291e)
 2015-06-04 10:23:33 -07:00
Akira Ajisaka
1cccd1eeb3 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cbaf3ddff0c9ae2d905d42828aadedf2a)
 2015-06-05 01:46:33 +09:00
Junping Du
d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 
(cherry picked from commit d7e7f6aa03c67b6a6ccf664adcb06d90bc963e58)
 2015-06-04 05:00:32 -07:00
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Vinayakumar B
2ce7659bf0 HADOOP-12019. update BUILDING.txt to include python for 'mvn site' in windows (Contributed by Vinayakumar B) 
(cherry picked from commit 1bb79c9d019ef12585eb1761f477cafcf859b1f0)
 2015-06-04 10:44:04 +05:30
Akira Ajisaka
e02df98f7d HADOOP-12018. smart-apply-patch.sh fails if the patch edits CR+LF files and is created by 'git diff --no-prefix'. Contributed by Kengo Seki. 
(cherry picked from commit b5f0d294f828240ea9a2395db94492e87af0260b)
 2015-06-04 11:16:24 +09:00
Robert Kanter
ee3002933a MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 
(cherry picked from commit cc70df98e74142331043a611a3bd8a53ff6a2242)
 2015-06-03 18:42:16 -07:00
Xuan
3f0573b059 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428f65bb008b5c451f6f09e61e1000300)
 2015-06-03 17:21:48 -07:00
Gera Shegalov
25c1e54d3f MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 
(cherry picked from commit d90c13e2da8867661bf19a802add70145ab9a462)
 2015-06-03 16:57:54 -07:00