0aaa2b9ba7
HDFS-6929. NN periodically unlinks lazy persist files with missing replicas from namespace. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
2014-10-17 13:42:00 -07:00
49a72e1360
HDFS-6927. Initial unit tests for Lazy Persist files. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
2014-10-17 13:42:00 -07:00
3b262caf11
HDFS-6926. DN support for saving replicas to persistent storage and evicting in-memory replicas. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
2014-10-17 13:42:00 -07:00
193528c84d
HDFS-6925. DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDirectoryScanner.java
2014-10-17 13:42:00 -07:00
3b7d4715a1
HDFS-6923. Propagate LazyPersist flag to DNs via DataTransferProtocol. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
2014-10-17 13:42:00 -07:00
183f4a4dfb
HDFS-6922. Add LazyPersist flag to INodeFile, save it in FsImage and edit logs. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormat.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileAttributes.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeLayoutVersion.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FSImageFormatPBSnapshot.java
hadoop-hdfs-project/hadoop-hdfs/src/main/proto/fsimage.proto
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSPermissionChecker.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestINodeFile.java
2014-10-17 13:42:00 -07:00
b1d085e7e3
HDFS-6924. Add new RAM_DISK storage type. (Arpit Agarwal)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-6581.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/StorageType.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
2014-10-17 13:42:00 -07:00
8885b75d78
MAPREDUCE-5542. Killing a job just as it finishes can generate an NPE in client. Contributed by Rohith
...
(cherry picked from commit 209b1699fc
2014-10-17 19:53:17 +00:00
79428e021f
YARN-2588. Standby RM fails to transitionToActive if previous transitionToActive failed with ZK exception. Contributed by Rohith Sharmaks
...
(cherry picked from commit a6aa6e42ca
2014-10-17 10:55:04 -07:00
99ce4277a8
YARN-2699. Fixed a bug in CommonNodeLabelsManager that caused tests to fail when using ephemeral ports on NodeIDs. Contributed by Wangda Tan.
...
(cherry picked from commit abae63caf9
2014-10-17 08:59:22 -07:00
6f19ba7688
HDFS-7252. small refinement to the use of isInAnEZ in FSNamesystem. (Yi Liu via vinayakumarb)
...
(cherry picked from commit 368743140d
2014-10-17 18:05:16 +05:30
7c87c2b42e
HDFS-7242. Code improvement for FSN#checkUnreadableBySuperuser. (Contributed by Yi Liu)
...
(cherry picked from commit 1c3ff0b7c8
2014-10-17 17:49:37 +05:30
fea721daf9
HDFS-6995. Block should be placed in the client's 'rack-local' node if 'client-local' node is not available (vinayakumarb)
...
Adding missed test file
(cherry picked from commit cba1f9e389
2014-10-17 17:34:54 +05:30
10110b6eae
Moved YARN-2682 from 2.6 to 2.7 in CHANGES.txt
...
(cherry picked from commit 72093fd8cb
2014-10-16 19:03:29 -07:00
a78f1f8da8
YARN-2682. Updated WindowsSecureContainerExecutor to not use DefaultContainerExecutor#getFirstApplicationDir and use getWorkingDir() instead. Contributed by Zhihai Xu
...
(cherry picked from commit 0fd0ebae64
2014-10-16 18:15:43 -07:00
a85767f376
HADOOP-11182. GraphiteSink emits wrong timestamps (Sascha Coenen via raviprak) Fix CHANGES.txt
2014-10-16 16:53:38 -07:00
caba212f12
YARN-2621. Simplify the output when the user doesn't have the access for getDomain(s). Contributed by Zhijie Shen
...
(cherry picked from commit 233d446be1
2014-10-16 16:38:00 -07:00
fddbf52caa
YARN-2689 TestSecureRMRegistryOperations failing on windows: secure ZK won't start
2014-10-16 14:21:49 -07:00
aef8dbde91
HDFS-5089. When a LayoutVersion support SNAPSHOT, it must support FSIMAGE_NAME_OPTIMIZATION.
2014-10-15 20:58:01 -07:00
603293da92
HDFS-7208. NN doesn't schedule replication when a DN storage fails. Contributed by Ming Ma
2014-10-15 20:49:54 -07:00
76cf2250de
YARN-2685. Fixed a bug in CommonNodeLabelsManager that caused wrong resource tracking per label when a host runs multiple node-managers. Contributed by Wangda Tan.
...
(cherry picked from commit b3056c266a
2014-10-15 18:48:36 -07:00
e8e3a36213
YARN-2496. Enhanced Capacity Scheduler to have basic support for allocating resources based on node-labels. Contributed by Wangda Tan.
...
YARN-2500. Ehnaced ResourceManager to support schedulers allocating resources based on node-labels. Contributed by Wangda Tan.
(cherry picked from commit f2ea555ac6
2014-10-15 18:34:19 -07:00
08eeb3e5b6
HADOOP-11181. GraphiteSink emits wrong timestamps (Sascha Coenen via raviprak)
...
(cherry picked from commit 466f08792f
2014-10-15 15:52:46 -07:00
f93d2ea27e
YARN-2312. Deprecated old ContainerId#getId API and updated MapReduce to use ContainerId#getContainerId instead. Contributed by Tsuyoshi OZAWA
2014-10-15 15:28:26 -07:00
025b6c13e9
MAPREDUCE-5970. Provide a boolean switch to enable MR-AM profiling. Contributed by Gera Shegalov
...
(cherry picked from commit f19771a24c
2014-10-15 17:51:53 +00:00
e68fef4d81
HDFS-7185. The active NameNode will not accept an fsimage sent from the standby during rolling upgrade. Contributed by Jing Zhao.
2014-10-15 10:28:02 -07:00
1179a0cb07
MAPREDUCE-5873. Shuffle bandwidth computation includes time spent waiting for maps. Contributed by Siqi Li
...
(cherry picked from commit b9edad6403
2014-10-15 15:54:57 +00:00
3ee0a5892b
HDFS-7190. Bad use of Preconditions in startFileInternal(). Contributed by Dawson Choong.
2014-10-14 23:42:36 -07:00
bec165e163
YARN-2656. Made RM web services authentication filter support proxy user. Contributed by Varun Vasudev and Zhijie Shen.
...
(cherry picked from commit 1220bb72d4
2014-10-14 21:56:44 -07:00
174138ec26
HDFS-7201. Fix typos in hdfs-default.xml. Contributed by Dawson Choong.
2014-10-14 17:36:23 -07:00
8aeda20b25
HADOOP-11181. Generalized o.a.h.s.t.d.DelegationTokenManager to handle all sub-classes of AbstractDelegationTokenIdentifier. Contributed by Zhijie Shen.
...
(cherry picked from commit cdce88376a
2014-10-14 11:37:28 -07:00
6c3ac6a485
HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao.
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
2014-10-14 10:33:51 -07:00
262fef119e
HADOOP-11198. Fix typo in javadoc for FileSystem#listStatus(). Contributed by Li Lu.
2014-10-13 20:43:17 -07:00
dd0857fd70
HDFS-7237. The command "hdfs namenode -rollingUpgrade" throws ArrayIndexOutOfBoundsException.
2014-10-13 18:30:21 -07:00
f3132eee10
HADOOP-11176. KMSClientProvider authentication fails when both currentUgi and loginUgi are a proxied user. Contributed by Arun Suresh.
...
(cherry picked from commit 0e57aa3bf6
2014-10-13 18:10:23 -07:00
88455173e8
YARN-2566. DefaultContainerExecutor should pick a working directory randomly. (Zhihai Xu via kasha)
...
(cherry picked from commit cc93e7e683
2014-10-13 16:32:42 -07:00
a1116b56a4
YARN-2641. Decommission nodes on -refreshNodes instead of next NM-RM heartbeat. (Zhihai Xu via kasha)
...
(cherry picked from commit da709a2eac
2014-10-13 16:24:00 -07:00
31a4bf7321
Missing Changes.txt for YARN-2308
...
(cherry picked from commit 178bc505da
2014-10-13 14:20:18 -07:00
536254f5e7
YARN-2308. Changed CapacityScheduler to explicitly throw exception if the queue
...
to which the apps were submitted is changed across RM restart. Contributed by Craig Welch & Chang Li
(cherry picked from commit f9680d9a16
2014-10-13 14:16:12 -07:00
6b5741d423
YARN-2377. Localization exception stack traces are not passed as diagnostic info. Contributed by Gera Shegalov
...
(cherry picked from commit a56ea01002
2014-10-13 18:32:38 +00:00
e51ae64761
YARN-2651. Spun off LogRollingInterval from LogAggregationContext. Contributed by Xuan Gong.
...
(cherry picked from commit 4aed2d8e91
2014-10-13 10:55:09 -07:00
c3dfc71193
HDFS-6544. Broken Link for GFS in package.html. Contributed by Suraj Nayak M.
2014-10-13 10:33:31 -07:00
715f222a80
HDFS-7236. Fix TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots. Contributed by Yongjun Zhang.
2014-10-13 10:28:12 -07:00
942fb67a04
MAPREDUCE-6115. TestPipeApplication#testSubmitter fails in trunk. Contributed by Binglin Chang
...
(cherry picked from commit 793dbf2465
2014-10-13 16:05:29 +00:00
9ed9ea8c52
YARN-2667. Fix the release audit warning caused by hadoop-yarn-registry. Contributed by Yi Liu
...
(cherry picked from commit 344a10ad5e
2014-10-13 15:40:16 +00:00
e17e3bdbc4
MAPREDUCE-6125. TestContainerLauncherImpl sometimes fails. Contributed by Mit Desai
...
(cherry picked from commit bbe80cdc7b
2014-10-13 15:09:26 +00:00
7bfd9e068d
MAPREDUCE-5875. Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild. (Gera Shegalov via kasha)
...
(cherry picked from commit ed0e0ef9748ce5b231de677ab41c3035137bbde4)
2014-10-11 22:49:46 -07:00
38dedc2f07
YARN-2668 yarn-registry JAR won't link against ZK 3.4.5. (stevel)
2014-10-11 18:33:29 -07:00
16710758f1
HADOOP-11193. Fix uninitialized variables in NativeIO.c. Contributed by Xiaoyu Yao.
2014-10-10 23:02:41 -07:00
fff6a1f77e
YARN-2501. Enhanced AMRMClient library to support requests against node labels. Contributed by Wangda Tan.
...
(cherry picked from commit a5ec3d0809
2014-10-10 19:58:42 -07:00
arp
Jason Lowe
Jian He
Vinod Kumar Vavilapalli
Vinayakumar B
Ravi Prakash
Steve Loughran
Tsz-Wo Nicholas Sze
Ravi Prakash
Jing Zhao
Haohui Mai
Zhijie Shen
Aaron T. Myers
Karthik Kambatla