Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,536 Commits 326 Branches 371 Tags
Commit Graph

9536 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yongjun Zhang
cd578fa289 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961410b6ac2b275cdcff4b95d75727e8d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
 2015-06-14 12:01:28 -07:00
cnauroth
92144ca75c HDFS-8593. Calculation of effective layout version mishandles comparison to current layout version in storage. Contributed by Chris Nauroth. 
(cherry picked from commit b8341f1cd89791c51b396ad531ec7fcc631be149)
 2015-06-13 10:38:03 -07:00
Andrew Wang
2776255ed6 HDFS-8499. Refactor BlockInfo class hierarchy with static helper class. Contributed by Zhe Zhang. 
(cherry picked from commit 355e98ab7c7dd997378a8624f642199713467fb8)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
 2015-06-12 11:39:02 -07:00
Colin Patrick Mccabe
378bb484bb HDFS-7923. The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages (cmccabe) 
(cherry picked from commit 12b5b06c063d93e6c683c9b6fac9a96912f59e59)
 2015-06-12 11:29:05 -07:00
Akira Ajisaka
e397cca456 HADOOP-11971. Move test utilities for tracing from hadoop-hdfs to hadoop-common. Contributed by Masatake Iwasaki. 
(cherry picked from commit e4489d97e5f0cec601b93e1b883fa194a7353229)
 2015-06-12 07:26:05 -07:00
Devaraj K
3863342ca9 YARN-3794. TestRMEmbeddedElector fails because of ambiguous LOG reference. 
Contributed by Chengbing Liu.

(cherry picked from commit d8dcfa98e3ca6a6fea414fd503589bb83b7a9c51)
 2015-06-12 13:45:05 +05:30
Haohui Mai
f1ee2eaf86 HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 18:59:53 -07:00
Arpit Agarwal
a560e654d9 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:50 -07:00
Colin Patrick Mccabe
754dd3496c HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbbadd4db41be83347e8c454049f4a7561)
 2015-06-11 14:28:41 -07:00
Andrew Wang
086eb49424 HDFS-8573. Move creation of restartMeta file logic from BlockReceiver to ReplicaInPipeline. Contributed by Eddy Xu. 
(cherry picked from commit b258b344bb76af6492828201959e36b45f0f75b8)
 2015-06-11 10:12:40 -07:00
Vinayakumar B
7454ed2b59 HADOOP-12074. in Shell.java#runCommand() rethrow InterruptedException as InterruptedIOException (Contributed by Lavkesh Lahngir) 
(cherry picked from commit 95c73d49b1bb459b626a9ac52acadb8f5fa724de)
 2015-06-11 12:00:17 +05:30
Xuan
8a6c7d9973 YARN-3785. Support for Resource as an argument during submitApp call in 
MockRM test class. Contributed by Sunil G

(cherry picked from commit 5583f88bf7f1852dc0907ce55d0755e4fb22107a)
 2015-06-10 21:41:58 -07:00
Andrew Wang
9523e2c098 HDFS-8549. Abort the balancer if an upgrade is in progress. 
(cherry picked from commit a7a7768341f1b7d3a8f2686e2f4d00c57f2e1d4f)
 2015-06-10 13:43:00 -07:00
Devaraj K
b117527bf3 MAPREDUCE-6389. Fix BaileyBorweinPlouffe CLI usage message. Contributed by 
Brahma Reddy Battula.

(cherry picked from commit c7729efee8727b59f2c78cd5a3ad23fa84139068)
 2015-06-10 21:06:35 +05:30
Devaraj K
5426e3e929 MAPREDUCE-6350. JobHistory doesn't support fully-functional search. 
Contributed by Siqi Li.
 2015-06-10 12:43:31 +05:30
Vinayakumar B
d21bc811d8 HADOOP-7817. RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos() (Contributed by kanaka kumar avvaru) 
(cherry picked from commit 48ca23def1d1c28448a65238814070e79c8f4c4e)
 2015-06-10 11:07:04 +05:30
Arpit Agarwal
2d67dff15f HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:46 -07:00
Xiaoyu Yao
d8d9041187 HDFS-8568. TestClusterId#testFormatWithEmptyClusterIdOption is failing. Contributed by Rakesh R. 
(cherry picked from commit 3c2397cb041068172c8ad209996414d7371d2eca)
 2015-06-09 12:21:53 -07:00
Zhijie Shen
0dad10d845 YARN-3787. Allowed generic history service to load a number of applications whose started time is within the given range. Contributed by Xuan Gong. 
(cherry picked from commit 8d0ef3163287d811381e94cc0101dbc9847ee2db)
 2015-06-08 23:21:35 -07:00
cnauroth
f5b0cce7fa HADOOP-12073. Azure FileSystem PageBlobInputStream does not return -1 on EOF. Contributed by Ivan Mitic. 
(cherry picked from commit c45784bc9031353b938f4756473937cca759b3dc)
 2015-06-08 22:42:24 -07:00
Xiaoyu Yao
116a72096d HDFS-8552. Fix hdfs CLI usage message for namenode and zkfc. Contributed by Brahma Reddy Battula 
(cherry picked from commit 927577c87ca19e8b5b75722f78e2def6d9386576)
 2015-06-08 22:03:07 -07:00
cnauroth
5a002aef12 MAPREDUCE-6392. Document mapred class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 025a3a8be07ba2b0c27ee39034b506d91aa486fc)
 2015-06-08 16:22:12 -07:00
cnauroth
d9016b0286 YARN-3786. Document yarn class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit a531b058aef48c9bf2e5366ed110e1f817316c1a)
 2015-06-08 16:18:43 -07:00
cnauroth
cc960734e4 HDFS-8553. Document hdfs class path options. Contributed by Brahma Reddy Battula. 
(cherry picked from commit d2832b3d4243c6c470c774bc33fd13f70b3e7b72)
 2015-06-08 16:15:53 -07:00
Arpit Agarwal
82e772bdbb HADOOP-12054. RPC client should not retry for InvalidToken exceptions. (Contributed by Varun Saxena) 2015-06-08 15:45:25 -07:00
Xuan
869304dc83 YARN-3778. Fix Yarn resourcemanger CLI usage. Contributed by Brahma Reddy Battula 
(cherry picked from commit 2b2465dfac1f147b6bb20d878b69a8cc3e85c8ad)
 2015-06-08 15:44:25 -07:00
Tsuyoshi Ozawa
c6cdecc6b3 MAPREDUCE-6388. Remove deprecation warnings from JobHistoryServer classes. Contributed by Ray Chiang. 
(cherry picked from commit 6afe20a7a4bbfa30fce3e3c9873ad43201987998)
 2015-06-08 15:06:23 -07:00
Jian He
a24ead8c6d YARN-2716. Refactor ZKRMStateStore retry code with Apache Curator. Contributed by Karthik Kambatla 
(cherry picked from commit 960b8f19ca98dbcfdd30f2f1f275b8718d2e872f)
 2015-06-08 14:51:25 -07:00
cnauroth
8ee50d8ca7 HDFS-8554. TestDatanodeLayoutUpgrade fails on Windows. Contributed by Chris Nauroth. 
(cherry picked from commit 0e80d51983942dca3348c8a8401bad3ecbaab010)
 2015-06-08 08:39:14 -07:00
Steve Loughran
c3c2b4d318 HADOOP-12052 IPC client downgrades all exception types to IOE, breaks callers trying to use them. (Brahma Reddy Battula via stevel) 2015-06-08 13:02:26 +01:00
Devaraj K
8c643e3bf9 YARN-3747. TestLocalDirsHandlerService should delete the created test 
directory logDir2. Contributed by David Moore.

(cherry picked from commit 126321eded7dc38c1eef2cfde9365404c924a5cb)
 2015-06-08 15:33:38 +05:30
Devaraj K
2a01c01af7 YARN-3780. Should use equals when compare Resource in 
RMNodeImpl#ReconnectNodeTransition. Contributed by zhihai xu.

(cherry picked from commit c7ee6c151c5771043a6de3b8a951cea13f59dd7b)
 2015-06-08 11:56:58 +05:30
cnauroth
953c4a049a HDFS-8539. Hdfs doesnt have class debug in windows. Contributed by Anu Engineer. 
(cherry picked from commit a6cb4894b45b0b909aca716058590dd6f0e3a6f9)
 2015-06-07 13:02:01 -07:00
Karthik Kambatla
322e7d0d5e YARN-3655. FairScheduler: potential livelock due to maxAMShare limitation and container reservation. (Zhihai Xu via kasha) 
(cherry picked from commit bd69ea408f8fdd8293836ce1089fe9b01616f2f7)
 2015-06-07 11:38:14 -07:00
Xiaoyu Yao
c8dc113943 HDFS-8116. Cleanup uncessary if LOG.isDebugEnabled() from RollingWindowManager. Contributed by Brahma Reddy Battula. 
(cherry picked from commit b61b48999530851da177db2e8998858ffc01f77e)
 2015-06-06 18:48:52 -07:00
cnauroth
f99744b7a0 HDFS-8432. Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases. Contributed by Chris Nauroth. 
(cherry picked from commit 71de367c5e80ea76d1e8d21f0216cd6b879dcee5)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
 2015-06-06 09:45:46 -07:00
cnauroth
1d1841d40f HADOOP-12055. Deprecate usage of NativeIO#link. Contributed by Andrew Wang. 
(cherry picked from commit e46cb800028c95f9bce575d05268cd10d0913222)
 2015-06-06 09:17:11 -07:00
Xiaoyu Yao
1a6cdf97bb HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:33:06 -07:00
Jason Lowe
588c22e5a0 MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li 
(cherry picked from commit b3ffa870034d06608a1946e2d9ce7dbd535a2c53)
 2015-06-05 22:39:52 +00:00
Andrew Wang
1137fd4c5b HADOOP-12056. Use DirectoryStream in DiskChecker#checkDirs to detect errors when listing a directory. Contributed by Zhihai Xu. 
(cherry picked from commit bc11e158b1726174fae2c7d2e8aa1f5005bf42eb)
 2015-06-05 13:52:43 -07:00
Andrew Wang
83a97907cc HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey. 
(cherry picked from commit 2dbc40e6086026ef02747223982aa68f2d328ade)
 2015-06-05 13:11:12 -07:00
Xuan
4d385b48f0 YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 
(cherry picked from commit 3e000a919fede85230fcfb06309a1f1d5e0c479c)
 2015-06-05 12:49:51 -07:00
Karthik Kambatla
429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 
(cherry picked from commit 75885852cc19dd6de12e62498b112d5d70ce87f4)
 2015-06-05 09:42:12 -07:00
Karthik Kambatla
1a2e6e8091 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
 2015-06-05 09:17:34 -07:00
Devaraj K
37ad3b3fef MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.

(cherry picked from commit 09b8241e6e369ee00241efcc2e048e710dbcccd1)
 2015-06-05 21:23:48 +05:30
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Andrew Wang
320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486ed99e1433d4e5118608da8dd365a934)
 2015-06-04 15:35:12 -07:00
Zhijie Shen
8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 
(cherry picked from commit 18dd01d6bf67f4d522b947454c1f4347d1cbbc19)
 2015-06-04 14:48:05 -07:00
Wangda Tan
344b750915 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c32d8fd8a83e46e686eacd14e5a533c)
 2015-06-04 13:20:49 -07:00
Gera Shegalov
2cc2488164 HADOOP-11924. Tolerate JDK-8047340-related exceptions in Shell#isSetSidAvailable preventing class init. (Tsuyoshi Ozawa via gera) 
(cherry picked from commit 9f6d67844da3c6ca1de66d426790e741e77f8038)
 2015-06-04 11:47:22 -07:00