Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,915 Commits 326 Branches 371 Tags
Commit Graph

8915 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Colin Patrick Mccabe
e5f9703419 HADOOP-12100. ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission (Bibin A. Chundatt via Colin P. McCabe) 
(cherry picked from commit 6e0a9f92fe0052d39b95a605174b3f6423c6aae7)
(cherry picked from commit 2946e92f79a73cccf9533f31ad40471046496cb2)
 2015-06-18 11:39:54 -07:00
Xuan
c0a419b134 YARN-3804. Both RM are on standBy state when kerberos user not in yarn.admin.acl. Contributed by Varun Saxena 2015-06-17 16:27:01 -07:00
Kihwal Lee
6de4d52c31 HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee. 
(cherry picked from commit c74517c46bf00af408ed866b6577623cdec02de1)
 2015-06-16 15:44:38 -05:00
Xiaoyu Yao
fe35ba1df9 HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao. 
(cherry picked from commit 4e88ff5b27cc33d311ab7a7248c3cf6303997ddd)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-16 12:51:34 -07:00
Yongjun Zhang
f4ae56d605 HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula) 
(cherry picked from commit b1d3ed02c16113f2f6bd74f56e3d293c0daf64e2)
 2015-06-15 21:51:21 -07:00
Tsuyoshi Ozawa
6c527f5514 YARN-3711. Documentation of ResourceManager HA should explain configurations about listen addresses. Contributed by Masatake Iwasaki. 
(cherry picked from commit e8c514373f2d258663497a33ffb3b231d0743b57)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2015-06-16 10:16:15 +09:00
Tsz-Wo Nicholas Sze
632cb542f6 HDFS-8361. Choose SSD over DISK in block placement. 2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze
7a248f2e9e HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina 2015-06-15 16:13:56 -07:00
Arpit Agarwal
a1810d75f0 HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal) 2015-06-14 20:18:31 -07:00
Arpit Agarwal
1145d09e0a HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal) 2015-06-14 16:24:42 -07:00
Yongjun Zhang
6dda57a817 HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang. 
(cherry picked from commit b0dc291961410b6ac2b275cdcff4b95d75727e8d)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java

(cherry picked from commit cd578fa289252b0ae1e27f7e7b097cceeb2c30ff)
 2015-06-14 12:04:16 -07:00
Haohui Mai
3651feb49d HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai. 2015-06-11 19:14:48 -07:00
Arpit Agarwal
3ec9b63586 HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal) 2015-06-11 15:08:15 -07:00
Colin Patrick Mccabe
0bcc68337b HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe) 
(cherry picked from commit 587309bbbadd4db41be83347e8c454049f4a7561)
(cherry picked from commit 754dd3496ce166f3743a8512f555ade6a7c2135d)
 2015-06-11 14:30:21 -07:00
Arpit Agarwal
105cadb421 HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal) 2015-06-09 20:58:55 -07:00
Xiaoyu Yao
8c1a46ca4f HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao. 2015-06-05 17:13:40 -07:00
Karthik Kambatla
d404caaa43 MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) 
(cherry picked from commit 6786daab33792c9c426adf0fc6b361f193d356a8)
(cherry picked from commit 1a2e6e8091d8578950a8623a023422ba5ee87d53)
 2015-06-05 09:19:52 -07:00
Wangda Tan
6b05575c72 Add missing test file of YARN-3733 
(cherry picked from commit 405bbcf68c32d8fd8a83e46e686eacd14e5a533c)
 2015-06-04 13:19:41 -07:00
Jian He
a78ca0fadc YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent to another. Contributed by Wangda Tan 
(cherry picked from commit 6ad4e59cfc111a92747fdb1fb99cc6378044832a)
 2015-06-04 10:53:47 -07:00
Wangda Tan
e74e4d7bb9 YARN-3733. Fix DominantRC#compare() does not work as expected if cluster resource is empty. (Rohith Sharmaks via wangda) 
(cherry picked from commit ebd797c48fe236b404cf3a125ac9d1f7714e291e)
 2015-06-04 10:26:06 -07:00
Akira Ajisaka
718dca4859 HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages. Contributed by Kazuho Fujii. 
(cherry picked from commit dbed757cbaf3ddff0c9ae2d905d42828aadedf2a)
(cherry picked from commit 1cccd1eeb3c1c22f570f657ba1eb80a9a37b1496)
 2015-06-05 01:47:01 +09:00
Jason Lowe
3d2c3f8648 YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled. Contributed by Rohith Sharmaks 
(cherry picked from commit e13b671aa510f553f4a6a232b4694b6a4cce88ae)

Conflicts:

	hadoop-yarn-project/CHANGES.txt
 2015-06-03 19:46:51 +00:00
Vinayakumar B
dd98cfd328 HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina) 
(cherry picked from commit 0ed9c2d8fec93b5dac9c305eda272ad8dfd869a9)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
 2015-06-03 15:18:42 +05:30
Vinayakumar B
8090a6ee63 HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina) 
(cherry picked from commit 54f83d9bd917e8641e902c5f0695e65ded472f9a)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
 2015-06-03 12:20:54 +05:30
Colin Patrick Mccabe
a7058caa72 HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe) 
(cherry picked from commit 806e407ac8896c4e669dba1fcf86fa5d6fee7c6d)
(cherry picked from commit 12497b0543176306edbd5cd8660a59b1e2e3a1c8)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
 2015-06-02 20:07:35 -07:00
Colin Patrick Mccabe
323ffccfe9 HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe) 
(cherry picked from commit 03fb5c642589dec4e663479771d0ae1782038b63)
(cherry picked from commit 17e369511dd18d6e25e10ddb0cd9511f7ec8f469)
 2015-06-02 11:46:12 -07:00
Xuan
b34825b0cb YARN-3753. RM failed to come up with "java.io.IOException: Wait for ZKClient creation timed out". Contributed by Jian He 2015-06-02 10:28:14 -07:00
Kihwal Lee
1ff3f16ed0 HDFS-5215. dfs.datanode.du.reserved is not considered while computing 
available space ( Brahma Reddy Battula via Yongjun Zhang)
(cherry picked from commit 66763bb06f107f0e072c773a5feb25903c688ddc)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2015-06-02 08:14:57 -05:00
Xuan
1ad0d43fb4 Revert "YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong." 
This reverts commit 2032e8d1a0c6abe154a64df69b0124cdae9f78e0.
 2015-06-01 11:29:36 -07:00
Wangda Tan
3f926f4b20 YARN-3725. App submission via REST API is broken in secure mode due to Timeline DT service address is empty. (Zhijie Shen via wangda) 
(cherry picked from commit 5cc3fced957a8471733e0e9490878bd68429fe24)
 2015-05-31 16:35:25 -07:00
Xuan
0943e0f5a8 YARN-2900. Application (Attempt and Container) Not Found in AHS results 
in Internal Server Error (500). Contributed by Zhijie Shen and Mit Desai
 2015-05-31 15:41:13 -07:00
Zhijie Shen
2032e8d1a0 YARN-1462. Made RM write application tags to timeline server and exposed them to users via generic history web UI and REST API. Contributed by Xuan Gong. 2015-05-30 09:39:49 -07:00
Vinod Kumar Vavilapalli
9449b72429 Fixed more FilesSystemRMStateStore issues. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 9acd24fec408c182683968ae2c75ca2c38866a60)
 2015-05-29 13:29:44 -07:00
Wangda Tan
bb8350388b YARN-3489. RMServerUtils.validateResourceRequests should only obtain queue info once. (Varun Saxena via wangda) 2015-05-28 17:04:20 -07:00
cnauroth
7489b128db HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop. Contributed by Larry McCay. 
(cherry picked from commit 860b8373c3a851386b8cd2d4265dd35e5aabc941)
(cherry picked from commit 33648268ce0f79bf51facafa3d151612e3d00ddb)
 2015-05-28 15:02:10 -07:00
Xuan
3f188272af YARN-3723. Need to clearly document primaryFilter and otherInfo value 
type. Contributed by Zhijie Shen

(cherry picked from commit 3077c299da4c5142503c9f92aad4b82349b2fde2)
(cherry picked from commit 550b55146d012fed7331bcd9b46fa6d130019105)
 2015-05-28 10:21:06 -07:00
cnauroth
02ab202a2b YARN-3626. On Windows localized resources are not moved to the front of the classpath when they should be. Contributed by Craig Welch. 
(cherry picked from commit 7bc33b63c78c6ee9a04c0c1511ef16e63f7f3074)
(cherry picked from commit 6ed8989a6f897437ab53c85c42fa8b58fe556359)
 2015-05-27 14:21:26 -07:00
Wangda Tan
fb57a1aac8 YARN-3686. CapacityScheduler should trim default_node_label_expression. (Sunil G via wangda) 
(cherry picked from commit cdbd66be111c93c85a409d47284e588c453ecae9)
 2015-05-26 16:29:27 -07:00
Xuan
23994e824a YARN-2238. Filtering on UI sticks even if I move away from the page. 
Contributed by Jian He

(cherry picked from commit 39077dba2e877420e7470df253f6154f6ecc64ec)
(cherry picked from commit 84245ff3b25af1ada941e8f93fe0f591a300de6f)
 2015-05-25 22:41:41 -07:00
Xuan
eb4d1ed612 YARN-3701. Isolating the error of generating a single app report when 
getting all apps from generic history service. Contributed by Zhijie
Shen

(cherry picked from commit 455b3acf0e82b214e06bd7b538968252945cd3c4)
(cherry picked from commit 33be070a5e944bdc04693e7be6f39fd3d145d8d1)
 2015-05-22 14:36:30 -07:00
Karthik Kambatla
c60054743f YARN-3675. FairScheduler: RM quits when node removal races with continuous-scheduling on the same node. (Anubhav Dhoot via kasha) 
(cherry picked from commit a8b50e46737c11936ba72c427da69b2365a07aac)
(cherry picked from commit e8ac88d4fe8aaef9d2e5fb76e6bc50223ff0e495)
 2015-05-21 13:43:22 -07:00
Xiaoyu Yao
c550825673 HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran. 
(cherry picked from commit 05e04f34f27149537fdb89f46af26bee14531ca4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
 2015-05-21 12:49:49 -07:00
Devaraj K
b68c338b17 YARN-3646. Applications are getting stuck some times in case of retry 
policy forever. Contributed by Raju Bairishetti.

(cherry picked from commit 0305316d6932e6f1a05021354d77b6934e57e171)
 2015-05-21 20:16:53 +05:30
Akira Ajisaka
7a8e076ffb YARN-3694. Fix dead link for TimelineServer REST API. Contributed by Jagadesh Kiran N. 
(cherry picked from commit a5def580879428bc7af3c030ef33554e0519f072)
(cherry picked from commit 6d7e7ef1c4df613004f20911136db5be8b474265)
 2015-05-21 23:15:59 +09:00
Jian He
ce45e4e82e YARN-3609. Load node labels from storage inside RM serviceStart. Contributed by Wangda Tan 2015-05-20 17:06:59 -07:00
Xuan
b275818925 YARN-3681. yarn cmd says "could not find main class 'queue'" in windows. Contributed by Craig Welch and Varun Saxena 
(cherry picked from commit a665b22cfa645ac5749a480fda2ef88a498b35ea)
 2015-05-20 14:44:31 -07:00
Wangda Tan
114d41aeb2 Backport "YARN-2918. RM should not fail on startup if queue's configured labels do not exist in cluster-node-labels. Contributed by Wangda Tan" to 2.7.1 2015-05-20 13:27:01 -07:00
Tsuyoshi Ozawa
6c7840f5b5 YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager. Contributed by Vinod Kumar Vavilapalli. 
(cherry picked from commit 7401e5b5e8060b6b027d714b5ceb641fcfe5b598)
 2015-05-20 09:02:30 +09:00
Tsuyoshi Ozawa
6b03ec5137 MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host. Contributed by Junping Du. 2015-05-19 12:01:14 -07:00
Arun Suresh
273d2f9753 HADOOP-11973. Ensure ZkDelegationTokenSecretManager namespace znodes get created with ACLs. (Gregory Chanan via asuresh) 
(cherry picked from commit fd3cb533d2495ea220ab2e468835a43a784d7532)
 2015-05-19 11:37:57 -07:00