Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,399 Commits 326 Branches 371 Tags
Commit Graph

9399 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaoyu Yao
1a6cdf97bb HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:33:06 -07:00
Jason Lowe
588c22e5a0 MAPREDUCE-6354. ShuffleHandler should be able to log shuffle connections. Contributed by Chang Li 
(cherry picked from commit b3ffa870034d06608a1946e2d9ce7dbd535a2c53)
 2015-06-05 22:39:52 +00:00
Andrew Wang
1137fd4c5b HADOOP-12056. Use DirectoryStream in DiskChecker#checkDirs to detect errors when listing a directory. Contributed by Zhihai Xu. 
(cherry picked from commit bc11e158b1726174fae2c7d2e8aa1f5005bf42eb)
 2015-06-05 13:52:43 -07:00
Andrew Wang
83a97907cc HADOOP-12059. S3Credentials should support use of CredentialProvider. Contributed by Sean Busbey. 
(cherry picked from commit 2dbc40e6086026ef02747223982aa68f2d328ade)
 2015-06-05 13:11:12 -07:00
Xuan
4d385b48f0 YARN-1462. AHS API and other AHS changes to handle tags for completed MR jobs. Contributed by Xuan Gong 
(cherry picked from commit 3e000a919fede85230fcfb06309a1f1d5e0c479c)
 2015-06-05 12:49:51 -07:00
Karthik Kambatla
429754de09 YARN-3259. FairScheduler: Trigger fairShare updates on node events. (Anubhav Dhoot via kasha) 
(cherry picked from commit 75885852cc19dd6de12e62498b112d5d70ce87f4)
 2015-06-05 09:42:12 -07:00
Karthik Kambatla
1a2e6e8091 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
 2015-06-05 09:17:34 -07:00
Devaraj K
37ad3b3fef MAPREDUCE-6383. Pi job (QuasiMonteCarlo) should not try to read the 
results file if its job fails. Contributed by Harsh J.

(cherry picked from commit 09b8241e6e369ee00241efcc2e048e710dbcccd1)
 2015-06-05 21:23:48 +05:30
Devaraj K
7f075f2b75 MAPREDUCE-6377. JHS sorting on state column not working in webUi. 
Contributed by zhihai xu.

(cherry picked from commit 790a861766dcd60212d83f444f2f96d3acf20a94)

Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
 2015-06-05 16:03:02 +05:30
Andrew Wang
320e4cc0e7 HDFS-8535. Clarify that dfs usage in dfsadmin -report output includes all block replicas. Contributed by Eddy Xu. 
(cherry picked from commit b2540f486ed99e1433d4e5118608da8dd365a934)
 2015-06-04 15:35:12 -07:00
Zhijie Shen
8f04ac9c34 YARN-3766. Fixed the apps table column error of generic history web UI. Contributed by Xuan Gong. 
(cherry picked from commit 18dd01d6bf67f4d522b947454c1f4347d1cbbc19)
 2015-06-04 14:48:05 -07:00
Wangda Tan
344b750915 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c32d8fd8a83e46e686eacd14e5a533c)
 2015-06-04 13:20:49 -07:00
Gera Shegalov
2cc2488164 HADOOP-11924. Tolerate JDK-8047340-related exceptions in Shell#isSetSidAvailable preventing class init. (Tsuyoshi Ozawa via gera) 
(cherry picked from commit 9f6d67844da3c6ca1de66d426790e741e77f8038)
 2015-06-04 11:47:22 -07:00
Vinod Kumar Vavilapalli
6044b5e5f6 MAPREDUCE-5232. Add a configuration to be able to log classpath and other system properties on mapreduce JVMs startup. Contributed by Sangjin Lee. 
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1482643 13f79535-47bb-0310-9956-ffa450edef68
(cherry picked from commit 4d8e350750748b919ee2158690a44cd9fd80dcae)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java
 2015-06-04 11:15:30 -07:00
Jian He
2e99210e4e YARN-2392. Add more diags about app retry limits on AM failures. Contributed by Steve Loughran 
(cherry picked from commit 1970ca7cbcdb7efa160d0cedc2e3e22c1401fad6)
 2015-06-04 11:14:36 -07:00
Andrew Wang
ccc834395b HDFS-8532. Make the visibility of DFSOutputStream#streamer member variable to private. Contributed by Rakesh R. 
(cherry picked from commit 5149dc7b975f0e90a14e3da02685594028534805)
 2015-06-04 11:09:28 -07:00
Arpit Agarwal
75e962f367 HADOOP-11994. smart-apply-patch wrongly assumes that git is infallible. (Contributed by Kengo Seki) 2015-06-04 10:54:12 -07:00
Jian He
6325e4b7dd YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 
(cherry picked from commit 6ad4e59cfc111a92747fdb1fb99cc6378044832a)
 2015-06-04 10:53:24 -07:00
Kihwal Lee
f0f277ddd0 HDFS-8463. Calling DFSInputStream.seekToNewSource just after stream creation causes NullPointerException. Contributed by Masatake Iwasaki. 
(cherry picked from commit ade6d9a61eb2e57a975f0efcdf8828d51ffec5fd)
 2015-06-04 12:52:49 -05:00
Wangda Tan
78d626fa89 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48fe236b404cf3a125ac9d1f7714e291e)
 2015-06-04 10:23:33 -07:00
Akira Ajisaka
1cccd1eeb3 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cbaf3ddff0c9ae2d905d42828aadedf2a)
 2015-06-05 01:46:33 +09:00
Junping Du
d8c7ee1b53 YARN-41. The RM should handle the graceful shutdown of the NM. Contributed by Devaraj K. 
(cherry picked from commit d7e7f6aa03c67b6a6ccf664adcb06d90bc963e58)
 2015-06-04 05:00:32 -07:00
Gera Shegalov
bae1ec5f29 MAPREDUCE-6382. Don't escape HTML links in Diagnostics in JHS job overview. (Siqi Li via gera) 
(cherry picked from commit e8302071ff16247a3d0cfc2dcca02da1bbf4db18)
 2015-06-03 23:52:16 -07:00
Vinayakumar B
2ce7659bf0 HADOOP-12019. update BUILDING.txt to include python for 'mvn site' in windows (Contributed by Vinayakumar B) 
(cherry picked from commit 1bb79c9d019ef12585eb1761f477cafcf859b1f0)
 2015-06-04 10:44:04 +05:30
Akira Ajisaka
e02df98f7d HADOOP-12018. smart-apply-patch.sh fails if the patch edits CR+LF files and is created by 'git diff --no-prefix'. Contributed by Kengo Seki. 
(cherry picked from commit b5f0d294f828240ea9a2395db94492e87af0260b)
 2015-06-04 11:16:24 +09:00
Robert Kanter
ee3002933a MAPREDUCE-5965. Hadoop streaming throws error if list of input files is high. Error is: "error=7, Argument list too long at if number of input file is high" (wilfreds via rkanter) 
(cherry picked from commit cc70df98e74142331043a611a3bd8a53ff6a2242)
 2015-06-03 18:42:16 -07:00
Xuan
3f0573b059 YARN-3749. We should make a copy of configuration when init 
MiniYARNCluster with multiple RMs. Contributed by Chun Chen

(cherry picked from commit 5766a04428f65bb008b5c451f6f09e61e1000300)
 2015-06-03 17:21:48 -07:00
Gera Shegalov
25c1e54d3f MAPREDUCE-6174. Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. (Eric Payne via gera) 
(cherry picked from commit d90c13e2da8867661bf19a802add70145ab9a462)
 2015-06-03 16:57:54 -07:00
Zhijie Shen
2fcd3fe6ba Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 4e2ed34d2023a7cd3842e41aba1995f5d0f53cf2.
 2015-06-03 14:16:51 -07:00
Zhijie Shen
5e976fffe3 Revert "YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in" 
This reverts commit 26ad3293ec103d0f3eb66177360324e38610234d.
 2015-06-03 14:16:51 -07:00
Karthik Kambatla
62d51b889a YARN-3762. FairScheduler: CME on FSParentQueue#getQueueUserAclInfo. (kasha) 
(cherry picked from commit edb9cd0f7aa1ecaf34afaa120e3d79583e0ec689)
 2015-06-03 13:50:02 -07:00
Jason Lowe
5ecc647ae0 MAPREDUCE-6374. Distributed Cache File visibility should check permission of full path. Contributed by Chang Li 
(cherry picked from commit 107da29ff9b6efd02b21c96df60c58ac51590ebd)
 2015-06-03 20:20:47 +00:00
Jason Lowe
752caa95a4 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa510f553f4a6a232b4694b6a4cce88ae)
 2015-06-03 19:45:18 +00:00
Zhijie Shen
aaa567514a YARN-3751. Fixed AppInfo to check if used resources are null. Contributed by Sunil G. 
(cherry picked from commit dbc4f64937ea2b4c941a3ac49afc4eeba3f5b763)
 2015-06-03 11:52:39 -07:00
Vinayakumar B
539ff92c2a HDFS-3716. Purger should remove stale fsimage ckpt files (Contributed by J.Andreina) 
(cherry picked from commit c59e7456304def867afe35d6ee009ab572f8fdeb)
 2015-06-03 15:31:01 +05:30
Vinayakumar B
0ed9c2d8fe HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 2015-06-03 15:15:44 +05:30
Vinayakumar B
4bb725b4dd HDFS-8470. fsimage loading progress should update inode, delegation token and cache pool count. (Contributed by surendra singh lilhore) 
(cherry picked from commit e965dcec378cb807856372425307598792977604)
 2015-06-03 14:25:12 +05:30
Vinayakumar B
066e45bcb6 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd917e8641e902c5f0695e65ded472f9a)
 2015-06-03 12:12:22 +05:30
Akira Ajisaka
84cae6c101 HADOOP-11991. test-patch.sh isn't re-executed even if smart-apply-patch.sh is modified. Contributed by Kengo Seki. 
(cherry picked from commit a6a8ee3c24e2457ce3ccc9a4a4e2535b223f547e)
 2015-06-03 15:01:52 +09:00
Colin Patrick Mccabe
12497b0543 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8896c4e669dba1fcf86fa5d6fee7c6d)
 2015-06-02 20:06:53 -07:00
Andrew Wang
a80a68c298 HDFS-8513. Rename BlockPlacementPolicyRackFaultTolarent to BlockPlacementPolicyRackFaultTolerant. (wang) 
(cherry picked from commit c1d50a91f7c05e4aaf4655380c8dcd11703ff158)
 2015-06-02 15:48:31 -07:00
Andrew Wang
379ece15fe HDFS-8386. Improve synchronization of 'streamer' reference in DFSOutputStream. Contributed by Rakesh R. 
(cherry picked from commit efc510a570cf880e7df1b69932aa41932658ee51)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2015-06-02 15:42:27 -07:00
Colin Patrick Mccabe
17e369511d HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c642589dec4e663479771d0ae1782038b63)
 2015-06-02 11:42:08 -07:00
Vinayakumar B
d1da842f86 HDFS-8256. '-storagepolicies , -blockId ,-replicaDetails ' options are missed out in usage and from documentation (Contributed by J.Andreina) 
(cherry picked from commit a2bd6217ebd68ae8cdd7814722659eebcf53004b)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
 2015-06-02 16:25:53 +05:30
Andrew Wang
52ff1610cd HADOOP-12037. Fix wrong classname in example configuration of hadoop-auth documentation. Contributed by Masatake Iwasaki. 
(cherry picked from commit 990078b927fb1189a99055f3758a4f4f639f2a9d)
 2015-06-01 18:05:06 -07:00
Jing Zhao
640ea994ed HDFS-8489. Subclass BlockInfo to represent contiguous blocks. Contributed by Zhe Zhang. 
(cherry picked from commit cdc13efb1af54d931585d25c5ba696a012412828)
 2015-06-01 11:45:04 -07:00
Xuan
26ad3293ec YARN-1462. Correct fix version from branch-2.7.1 to branch-2.8 in 
CHANGES.txt

(cherry picked from commit 0b5cfacde638bc25cc010cd9236369237b4e51a8)
 2015-06-01 11:40:24 -07:00
Wangda Tan
a3734f67d3 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced957a8471733e0e9490878bd68429fe24)
 2015-05-31 16:33:50 -07:00
Xuan
4563411e04 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai

(cherry picked from commit 9686261ecb872ad159fac3ca44f1792143c6d7db)
 2015-05-31 15:38:39 -07:00
Tsuyoshi Ozawa
b91cadcafd HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS. Contributed by Archana T. 
(cherry picked from commit c2356cb0bc3a0ad7d15c03fc63591608bfd5cbe9)
 2015-06-01 00:42:11 +09:00