Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,500 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

7500 Commits

Author SHA1 Message Date
Jian He a057552468 YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu 
(cherry picked from commit 0fd0ebae64)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java
 2014-10-22 16:19:40 -07:00
Vinod Kumar Vavilapalli e282805f1c YARN-1063. Augmented Hadoop common winutils to have the ability to create containers as domain users. Contributed by Remus Rusanu. 
Committed as a YARN patch even though all the code changes are in common.
(cherry picked from commit 5ca97f1e60)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-22 16:10:49 -07:00
Karthik Kambatla 6064b9d02f HADOOP-11122. Fix findbugs in ZK DelegationTokenSecretManagers. (Arun Suresh via kasha) 
(cherry picked from commit 70719e5c62)
 2014-10-22 14:36:06 -07:00
Steve Loughran ada23f3dd3 YARN-2700 TestSecureRMRegistryOperations failing on windows: auth problems 2014-10-22 21:12:52 +01:00
Jian He cca473a321 MAPREDUCE-6126. Fixed Rumen JobBuilder to ignore NormalizedResourceEvent. Contributed by Junping Du 
Conflicts:
	hadoop-mapreduce-project/CHANGES.txt
(cherry picked from commit 9b973b7c83)
 2014-10-22 10:58:21 -07:00
Steve Loughran 8fdff5b791 YARN-2692 ktutil test hanging on some machines/ktutil versions (stevel) 2014-10-22 14:34:57 +01:00
Zhijie Shen b1ca12bc0b YARN-2721. Suppress NodeExist exception thrown by ZKRMStateStore when it retries creating znode. Contributed by Jian He. 
(cherry picked from commit 7e3b5e6f5c)
 2014-10-21 21:53:27 -07:00
Vinod Kumar Vavilapalli a1c5d4c51f YARN-2715. Fixed ResourceManager to respect common configurations for proxy users/groups beyond just the YARN level config. Contributed by Zhijie Shen. 
(cherry picked from commit c0e034336c)
 2014-10-21 20:11:14 -07:00
Zhijie Shen 7df8cbdd0f YARN-2709. Made timeline client getDelegationToken API retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit b2942762d7)
 2014-10-21 16:09:23 -07:00
Colin Patrick Mccabe 1ec2dbf093 HDFS-7154. Fix returning value of starting reconfiguration task (Lei Xu via Colin P. McCabe) 
(cherry picked from commit 7aab5fa1bd)
(cherry picked from commit 8c3b007236)
 2014-10-21 15:42:39 -07:00
cnauroth 8727f9e1b0 YARN-2720. Windows: Wildcard classpath variables not expanded against resources contained in archives. Contributed by Craig Welch. 
(cherry picked from commit 6637e3cf95)
(cherry picked from commit a066134277)
 2014-10-21 12:36:43 -07:00
Brandon Li 20108263fb HDFS-7215.Add JvmPauseMonitor to NFS gateway. Contributed by Brandon Li 
(cherry picked from commit 4e134a02a4)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

(cherry picked from commit c800e058ea)
 2014-10-21 11:56:26 -07:00
Brandon Li 3fa1ee2923 HDFS-7259. Unresponseive NFS mount point due to deferred COMMIT response. Contributed by Brandon Li 
(cherry picked from commit 33e020cb19)
 2014-10-21 11:05:50 -07:00
Jason Lowe 413fe4cb02 YARN-90. NodeManager should identify failed disks becoming good again. Contributed by Varun Vasudev 
(cherry picked from commit 6f2028bd15)

Conflicts:

	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDirectoryCollection.java
 2014-10-21 17:43:22 +00:00
Zhijie Shen fcdeba204c YARN-2717. Avoided duplicate logging when container logs are not found. Contributed by Xuan Gong. 
(cherry picked from commit 171f2376d2)
 2014-10-20 22:24:19 -07:00
Jian He 852ffea84e Missing file for YARN-1879 
(cherry picked from commit 4a78a75228)
 2014-10-20 21:15:30 -07:00
Jian He 23b4af488e Missing file for YARN-2701 
(cherry picked from commit 4fa1fb3193)
 2014-10-20 20:53:28 -07:00
Jian He c7db30522c YARN-2701. Potential race condition in startLocalizer when using LinuxContainerExecutor. Contributed by Xuan Gong 
(cherry picked from commit 2839365f23)
 2014-10-20 20:53:28 -07:00
Vinod Kumar Vavilapalli debb3686d3 YARN-1972. Added a secure container-executor for Windows. Contributed by Remus Rusanu. 
commit ba7f31c2ee is the corresponding trunk commit, this is a slightly different patch for branch-2.
(cherry picked from commit 3326fba382)

Conflicts:
	hadoop-yarn-project/CHANGES.txt
 2014-10-20 20:47:49 -07:00
Zhijie Shen 2af7f48625 YARN-2582. Fixed Log CLI and Web UI for showing aggregated logs of LRS. Contributed Xuan Gong. 
(cherry picked from commit e90718fa5a)
 2014-10-20 13:43:03 -07:00
Zhijie Shen 6d78dc3ee4 YARN-2673. Made timeline client put APIs retry if ConnectException happens. Contributed by Li Lu. 
(cherry picked from commit 89427419a3)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/yarn-default.xml
 2014-10-20 12:31:33 -07:00
Karthik Kambatla cdf265de4f HADOOP-11194. Ignore .keep files (kasha) 
(cherry picked from commit d5084b9fa3)
 2014-10-20 11:30:13 -04:00
Ivan Mitic a78dd99d67 Revert "MAPREDUCE-5911. Terasort TeraOutputFormat does not check for output directory existance. Contributed by Bruno P. Kinoshita." 
This reverts commit 71e05b1900.
 2014-10-19 23:43:38 -04:00
Ivan Mitic 71e05b1900 MAPREDUCE-5911. Terasort TeraOutputFormat does not check for output directory existance. Contributed by Bruno P. Kinoshita. 
(cherry picked from commit 7bbda6ef92)
 2014-10-19 19:15:55 -04:00
Vinod Kumar Vavilapalli 88b7e5ca65 YARN-2504. Enhanced RM Admin CLI to support management of node-labels. Contribyted by Wangda Tan. 
(cherry picked from commit 8256766498)
 2014-10-18 12:09:02 -07:00
Vinod Kumar Vavilapalli 24b0529a9f YARN-2676. Enhanced Timeline auth-filter to support proxy users. Contributed by Zhijie Shen. 
(cherry picked from commit 39063cd36f)
 2014-10-17 22:04:14 -07:00
Tsz-Wo Nicholas Sze 0ee6ddb782 HDFS-7260. Change DFSOutputStream.MAX_PACKETS to be configurable. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
 2014-10-17 18:34:23 -07:00
Vinod Kumar Vavilapalli ce42210651 YARN-2705. Fixed bugs in ResourceManager node-label manager that were causing test-failures: added a dummy in-memory labels-manager. Contributed by Wangda Tan. 
(cherry picked from commit e9c66e8fd2)
 2014-10-17 18:27:34 -07:00
Jian He 4d03915be9 YARN-1879. Marked Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over. Contributed by Tsuyoshi OZAWA 
(cherry picked from commit c3de2412eb)
 2014-10-17 16:36:45 -07:00
arp 9183dadb73 HDFS-6921. Add LazyPersist flag to FileStatus. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsLocatedFileStatus.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-10-17 16:00:53 -07:00
cnauroth 2e98379bcb HDFS-7090. Use unbuffered writes when persisting in-memory replicas. Contributed by Xiaoyu Yao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-17 16:00:53 -07:00
cnauroth 1b6d115ccf HDFS-7112. LazyWriter should use either async IO or one thread per physical disk. Contributed by Xiaoyu Yao. 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
 2014-10-17 16:00:53 -07:00
arp 137f0b8cbf HDFS-7171. Fix Jenkins failures in HDFS-6581 branch. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
 2014-10-17 16:00:53 -07:00
arp e6457a0e08 HDFS-7129. Metrics to track usage of memory for writes. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:53 -07:00
arp d61daff9bf HDFS-7159. Use block storage policy to set lazy persist preference. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
 2014-10-17 16:00:52 -07:00
arp ba31a9503c HDFS-7144. Fix findbugs warnings in RamDiskReplicaTracker. (Contributed by Tsz Wo Nicholas Sze) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:52 -07:00
arp 3481f5d4f3 HDFS-7155. Bugfix in createLocatedFileStatus caused by bad merge. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:52 -07:00
arp 05cfd68863 HDFS-7143. Fix findbugs warnings in HDFS-6581 branch. (Contributed by Tsz Wo Nicholas Sze) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:52 -07:00
arp 9de4d1897a HDFS-6932. Balancer and Mover tools should ignore replicas on RAM_DISK. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
 2014-10-17 16:00:52 -07:00
arp a59ce50627 HDFS-6990. Add unit test for evict/delete RAM_DISK block with open handle. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:52 -07:00
arp e9eebaa012 HDFS-7100. Make eviction scheme pluggable. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
 2014-10-17 16:00:52 -07:00
arp 144198892f HDFS-7108. Fix unit test failures in SimulatedFsDataset. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:51 -07:00
arp ebdbb5ec24 HDFS-7091. Add forwarding constructor for INodeFile for existing callers. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSPermissionChecker.java
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
 2014-10-17 16:00:51 -07:00
arp d32ce5794a HDFS-7084. FsDatasetImpl#copyBlockFiles debug log can be improved. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:51 -07:00
arp b7062e8d3e HDFS-7080. Fix finalize and upgrade unit test failures. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:51 -07:00
arp 7d168d9ff9 HDFS-7079. Few more unit test fixes for HDFS-6581. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:51 -07:00
arp d3ccbd8e65 HDFS-7064. Fix unit test failures in HDFS-6581 branch. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:51 -07:00
arp 6cc206b8ad HDFS-6978. Directory scanner should correctly reconcile blocks on RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:51 -07:00
arp 2fdd9f51ce HDFS-7066. LazyWriter#evictBlocks misses a null check for replicaState. (Contributed by Xiaoyu Yao) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
 2014-10-17 16:00:50 -07:00
arp bcec6582cf HDFS-6991. Notify NN of evicted block before deleting it from RAM disk. (Arpit Agarwal) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
	hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
 2014-10-17 16:00:50 -07:00