diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/CHANGES.0.1.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/CHANGES.0.1.0.md new file mode 100644 index 00000000000..434d7c354f8 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/CHANGES.0.1.0.md @@ -0,0 +1,117 @@ + + +# Apache Hadoop Changelog + +## Release 0.1.0 - 2006-04-02 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-80](https://issues.apache.org/jira/browse/HADOOP-80) | binary key | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-46](https://issues.apache.org/jira/browse/HADOOP-46) | user-specified job names | Major | . | Doug Cutting | Owen O'Malley | +| [HADOOP-44](https://issues.apache.org/jira/browse/HADOOP-44) | RPC exceptions should include remote stack trace | Major | ipc | Doug Cutting | Doug Cutting | +| [HADOOP-37](https://issues.apache.org/jira/browse/HADOOP-37) | A way to determine the size and overall activity of the cluster | Major | . | Owen O'Malley | | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-103](https://issues.apache.org/jira/browse/HADOOP-103) | introduce a common parent class for Mapper and Reducer | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-87](https://issues.apache.org/jira/browse/HADOOP-87) | SequenceFile performance degrades substantially compression is on and large values are encountered | Major | io | Sameer Paranjpye | Doug Cutting | +| [HADOOP-79](https://issues.apache.org/jira/browse/HADOOP-79) | listFiles optimization | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-67](https://issues.apache.org/jira/browse/HADOOP-67) | Added statistic/reporting info to DFS | Trivial | . | Barry Kaplan | Doug Cutting | +| [HADOOP-60](https://issues.apache.org/jira/browse/HADOOP-60) | Specification of alternate conf. directory | Minor | . | stack | | +| [HADOOP-49](https://issues.apache.org/jira/browse/HADOOP-49) | JobClient cannot use a non-default server (unlike DFSShell) | Major | . | Michel Tourn | Michel Tourn | +| [HADOOP-45](https://issues.apache.org/jira/browse/HADOOP-45) | JobTracker should log task errors | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-41](https://issues.apache.org/jira/browse/HADOOP-41) | JAVA\_OPTS for the TaskRunner Child | Minor | conf | stack | | +| [HADOOP-38](https://issues.apache.org/jira/browse/HADOOP-38) | default splitter should incorporate fs block size | Major | . | Doug Cutting | | +| [HADOOP-36](https://issues.apache.org/jira/browse/HADOOP-36) | Adding some uniformity/convenience to environment management | Major | conf | Bryan Pendleton | | +| [HADOOP-33](https://issues.apache.org/jira/browse/HADOOP-33) | DF enhancement: performance and win XP support | Minor | fs | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-30](https://issues.apache.org/jira/browse/HADOOP-30) | DFS shell: support for ls -r and cat | Major | . | Michel Tourn | | +| [HADOOP-25](https://issues.apache.org/jira/browse/HADOOP-25) | a new map/reduce example and moving the examples from src/java to src/examples | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-20](https://issues.apache.org/jira/browse/HADOOP-20) | Mapper, Reducer need an occasion to cleanup after the last record is processed. | Major | . | Michel Tourn | | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-112](https://issues.apache.org/jira/browse/HADOOP-112) | copyFromLocal should exclude .crc files | Minor | . | Monu Ogbe | Doug Cutting | +| [HADOOP-110](https://issues.apache.org/jira/browse/HADOOP-110) | new key and value instances are allocated before each map | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-107](https://issues.apache.org/jira/browse/HADOOP-107) | Namenode errors "Failed to complete filename.crc because dir.getFile()==null and null" | Major | . | Igor Bolotin | Doug Cutting | +| [HADOOP-102](https://issues.apache.org/jira/browse/HADOOP-102) | Two identical consecutive loops in FSNamesystem.chooseTarget() | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-100](https://issues.apache.org/jira/browse/HADOOP-100) | Inconsistent locking of the JobTracker.taskTrackers field | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-98](https://issues.apache.org/jira/browse/HADOOP-98) | The JobTracker's count of the number of running maps and reduces is wrong | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-97](https://issues.apache.org/jira/browse/HADOOP-97) | DFSShell.cat returns NullPointerException if file does not exist | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-93](https://issues.apache.org/jira/browse/HADOOP-93) | allow minimum split size configurable | Major | . | Hairong Kuang | Doug Cutting | +| [HADOOP-86](https://issues.apache.org/jira/browse/HADOOP-86) | If corrupted map outputs, reducers get stuck fetching forever | Major | . | stack | Doug Cutting | +| [HADOOP-84](https://issues.apache.org/jira/browse/HADOOP-84) | client should report file name in which IO exception occurs | Minor | . | Yoram Arnon | Konstantin Shvachko | +| [HADOOP-83](https://issues.apache.org/jira/browse/HADOOP-83) | infinite retries accessing a missing block | Major | . | Yoram Arnon | Konstantin Shvachko | +| [HADOOP-82](https://issues.apache.org/jira/browse/HADOOP-82) | JobTracker loses it: NoSuchElementException | Minor | . | stack | | +| [HADOOP-81](https://issues.apache.org/jira/browse/HADOOP-81) | speculative execution is only controllable from the default config | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-78](https://issues.apache.org/jira/browse/HADOOP-78) | rpc commands not buffered | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-77](https://issues.apache.org/jira/browse/HADOOP-77) | hang / crash when input folder does not exists. | Critical | . | Stefan Groschupf | | +| [HADOOP-70](https://issues.apache.org/jira/browse/HADOOP-70) | the two file system tests TestDFS and TestFileSystem take too long | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-66](https://issues.apache.org/jira/browse/HADOOP-66) | dfs client writes all data for a chunk to /tmp | Major | . | Sameer Paranjpye | Doug Cutting | +| [HADOOP-57](https://issues.apache.org/jira/browse/HADOOP-57) | hadoop dfs -ls / does not show root of file system | Minor | . | Yoram Arnon | | +| [HADOOP-52](https://issues.apache.org/jira/browse/HADOOP-52) | mapred input and output dirs must be absolute | Major | . | Doug Cutting | Owen O'Malley | +| [HADOOP-42](https://issues.apache.org/jira/browse/HADOOP-42) | PositionCache decrements its position for reads at the end of file | Major | fs | Konstantin Shvachko | | +| [HADOOP-40](https://issues.apache.org/jira/browse/HADOOP-40) | bufferSize argument is ignored in FileSystem.create(File, boolean, int) | Minor | fs | Konstantin Shvachko | | +| [HADOOP-34](https://issues.apache.org/jira/browse/HADOOP-34) | Build Paths Relative to PWD in build.xml | Trivial | . | Jeremy Bensley | | +| [HADOOP-28](https://issues.apache.org/jira/browse/HADOOP-28) | webapps broken | Major | . | Owen O'Malley | | +| [HADOOP-22](https://issues.apache.org/jira/browse/HADOOP-22) | remove unused imports | Trivial | . | Sami Siren | | +| [HADOOP-21](https://issues.apache.org/jira/browse/HADOOP-21) | the webapps need to be updated for the move from nutch | Minor | . | Owen O'Malley | | +| [HADOOP-19](https://issues.apache.org/jira/browse/HADOOP-19) | Datanode corruption | Critical | . | Rod Taylor | Doug Cutting | +| [HADOOP-16](https://issues.apache.org/jira/browse/HADOOP-16) | RPC call times out while indexing map task is computing splits | Major | . | Chris Schneider | Mike Cafarella | +| [HADOOP-12](https://issues.apache.org/jira/browse/HADOOP-12) | InputFormat used in job must be in JobTracker classpath (not loaded from job JAR) | Minor | . | Bryan Pendleton | | +| [HADOOP-10](https://issues.apache.org/jira/browse/HADOOP-10) | ndfs.replication is not documented within the nutch-default.xml configuration file. | Trivial | . | Rod Taylor | | +| [HADOOP-7](https://issues.apache.org/jira/browse/HADOOP-7) | MapReduce has a series of problems concerning task-allocation to worker nodes | Major | . | Mike Cafarella | | +| [HADOOP-6](https://issues.apache.org/jira/browse/HADOOP-6) | missing build directory in classpath | Minor | . | Owen O'Malley | | +| [HADOOP-5](https://issues.apache.org/jira/browse/HADOOP-5) | need commons-logging-api jar file | Minor | . | Owen O'Malley | | +| [HADOOP-3](https://issues.apache.org/jira/browse/HADOOP-3) | Output directories are not cleaned up before the reduces run | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2](https://issues.apache.org/jira/browse/HADOOP-2) | Reused Keys and Values fail with a Combiner | Major | . | Owen O'Malley | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1](https://issues.apache.org/jira/browse/HADOOP-1) | initial import of code from Nutch | Major | . | Doug Cutting | Doug Cutting | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/RELEASENOTES.0.1.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/RELEASENOTES.0.1.0.md new file mode 100644 index 00000000000..d2b896d9bf2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.0/RELEASENOTES.0.1.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.1.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/CHANGES.0.1.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/CHANGES.0.1.1.md new file mode 100644 index 00000000000..21dfefc047b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/CHANGES.0.1.1.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.1.1 - 2006-04-07 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-116](https://issues.apache.org/jira/browse/HADOOP-116) | cleaning up /tmp/hadoop/mapred/system | Major | . | raghavendra prabhu | Doug Cutting | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-125](https://issues.apache.org/jira/browse/HADOOP-125) | LocalFileSystem.makeAbsolute bug on Windows | Minor | fs | p sutter | Doug Cutting | +| [HADOOP-117](https://issues.apache.org/jira/browse/HADOOP-117) | mapred temporary files not deleted | Blocker | . | raghavendra prabhu | Doug Cutting | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/RELEASENOTES.0.1.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/RELEASENOTES.0.1.1.md new file mode 100644 index 00000000000..213673c05a0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.1.1/RELEASENOTES.0.1.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.1.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/CHANGES.0.10.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/CHANGES.0.10.0.md new file mode 100644 index 00000000000..b09a09e5af7 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/CHANGES.0.10.0.md @@ -0,0 +1,123 @@ + + +# Apache Hadoop Changelog + +## Release 0.10.0 - 2007-01-05 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-811](https://issues.apache.org/jira/browse/HADOOP-811) | Patch to support multi-threaded MapRunnable | Major | . | Alejandro Abdelnur | Doug Cutting | +| [HADOOP-681](https://issues.apache.org/jira/browse/HADOOP-681) | Adminstrative hook to pull live nodes out of a HDFS cluster | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-574](https://issues.apache.org/jira/browse/HADOOP-574) | want FileSystem implementation for Amazon S3 | Major | fs | Doug Cutting | | +| [HADOOP-454](https://issues.apache.org/jira/browse/HADOOP-454) | hadoop du optionally behave like unix's du -s | Trivial | . | Marco Nicosia | Hairong Kuang | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-853](https://issues.apache.org/jira/browse/HADOOP-853) | Move site directories to docs directories | Minor | documentation | Nigel Daley | Doug Cutting | +| [HADOOP-850](https://issues.apache.org/jira/browse/HADOOP-850) | Add Writable implementations for variable-length integer types. | Minor | io | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-837](https://issues.apache.org/jira/browse/HADOOP-837) | RunJar should unpack jar files into hadoop.tmp.dir | Major | util | Hairong Kuang | Hairong Kuang | +| [HADOOP-806](https://issues.apache.org/jira/browse/HADOOP-806) | NameNode WebUI : Include link to each of datanodes | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-804](https://issues.apache.org/jira/browse/HADOOP-804) | Cut down on the "mumbling" in the Task process' stdout/stderr | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-796](https://issues.apache.org/jira/browse/HADOOP-796) | Node failing tasks and failed tasks should be more easily accessible through jobtracker history. | Minor | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-783](https://issues.apache.org/jira/browse/HADOOP-783) | Hadoop dfs -put and -get accept '-' to indicate stdin/stdout | Minor | . | Marco Nicosia | Wendy Chien | +| [HADOOP-763](https://issues.apache.org/jira/browse/HADOOP-763) | NameNode benchmark using mapred is insufficient | Minor | test | Nigel Daley | Nigel Daley | +| [HADOOP-756](https://issues.apache.org/jira/browse/HADOOP-756) | new dfsadmin command to wait until safe mode is exited | Minor | . | Owen O'Malley | dhruba borthakur | +| [HADOOP-720](https://issues.apache.org/jira/browse/HADOOP-720) | Write a white paper on Hadoop File System Architecture, Design and Features | Major | documentation | dhruba borthakur | dhruba borthakur | +| [HADOOP-717](https://issues.apache.org/jira/browse/HADOOP-717) | When there are few reducers, sorting should be done by mappers | Major | . | arkady borkovsky | Owen O'Malley | +| [HADOOP-676](https://issues.apache.org/jira/browse/HADOOP-676) | JobClient should print user friendly messages for standard errors | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-621](https://issues.apache.org/jira/browse/HADOOP-621) | When a dfs -cat command is killed by the user, the correspondig hadoop process does not get aborted | Minor | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-619](https://issues.apache.org/jira/browse/HADOOP-619) | Unify Map-Reduce and Streaming to take the same globbed input specification | Major | . | eric baldeschwieler | Sanjay Dahiya | +| [HADOOP-618](https://issues.apache.org/jira/browse/HADOOP-618) | JobProfile and JobSubmissionProtocol should be public | Major | . | Runping Qi | Arun C Murthy | +| [HADOOP-571](https://issues.apache.org/jira/browse/HADOOP-571) | Path should use URI syntax | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-525](https://issues.apache.org/jira/browse/HADOOP-525) | Need raw comparators for hadoop record types | Major | record | Sameer Paranjpye | Milind Bhandarkar | +| [HADOOP-470](https://issues.apache.org/jira/browse/HADOOP-470) | Some improvements in the DFS content browsing UI | Minor | . | Devaraj Das | Hairong Kuang | +| [HADOOP-451](https://issues.apache.org/jira/browse/HADOOP-451) | Add a Split interface | Major | . | Doug Cutting | Owen O'Malley | +| [HADOOP-371](https://issues.apache.org/jira/browse/HADOOP-371) | ant tar should package contrib jars | Major | build | Michel Tourn | Nigel Daley | +| [HADOOP-331](https://issues.apache.org/jira/browse/HADOOP-331) | map outputs should be written to a single output file with an index | Major | . | eric baldeschwieler | Devaraj Das | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-849](https://issues.apache.org/jira/browse/HADOOP-849) | randomwriter fails with 'java.lang.OutOfMemoryError: Java heap space' in the 'reduce' task | Major | . | Arun C Murthy | Devaraj Das | +| [HADOOP-846](https://issues.apache.org/jira/browse/HADOOP-846) | Progress report is not sent during the intermediate sorts in the map phase | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-844](https://issues.apache.org/jira/browse/HADOOP-844) | Metrics messages are sent on a fixed-delay schedule instead of a fixed-rate schedule | Minor | metrics | David Bowen | | +| [HADOOP-841](https://issues.apache.org/jira/browse/HADOOP-841) | native hadoop libraries don't build properly with 64-bit OS and a 32-bit jvm | Major | build | Arun C Murthy | Arun C Murthy | +| [HADOOP-840](https://issues.apache.org/jira/browse/HADOOP-840) | the task tracker is getting blocked by long deletes of local files | Major | . | Owen O'Malley | Mahadev konar | +| [HADOOP-838](https://issues.apache.org/jira/browse/HADOOP-838) | TaskRunner.run() doesn't pass along the 'java.library.path' to the child (task) jvm | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-836](https://issues.apache.org/jira/browse/HADOOP-836) | unit tests fail on windows (/C:/cygwin/... is invalid) | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-835](https://issues.apache.org/jira/browse/HADOOP-835) | conf not set for the default Codec when initializing a Reader for a record-compressed sequence file | Major | io | Hairong Kuang | Hairong Kuang | +| [HADOOP-829](https://issues.apache.org/jira/browse/HADOOP-829) | Separate the datanode contents that is written to the fsimage vs the contents used in over-the-wire communication | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-825](https://issues.apache.org/jira/browse/HADOOP-825) | If the default file system is set using the new uri syntax, the namenode will not start | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-824](https://issues.apache.org/jira/browse/HADOOP-824) | DFSShell should become FSShell | Major | . | Doug Cutting | | +| [HADOOP-823](https://issues.apache.org/jira/browse/HADOOP-823) | DataNode will not start up if any directories from dfs.data.dir are missing | Major | . | Bryan Pendleton | Sameer Paranjpye | +| [HADOOP-818](https://issues.apache.org/jira/browse/HADOOP-818) | ant clean test-contrib doesn't work | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-814](https://issues.apache.org/jira/browse/HADOOP-814) | Increase dfs scalability by optimizing locking on namenode. | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-813](https://issues.apache.org/jira/browse/HADOOP-813) | map tasks lost during sort | Major | . | Owen O'Malley | Devaraj Das | +| [HADOOP-802](https://issues.apache.org/jira/browse/HADOOP-802) | mapred.speculative.execution description in hadoop-defauls.xml is not complete | Trivial | conf | Nigel Daley | Nigel Daley | +| [HADOOP-794](https://issues.apache.org/jira/browse/HADOOP-794) | JobTracker crashes with ArithmeticException | Major | . | Nigel Daley | Owen O'Malley | +| [HADOOP-792](https://issues.apache.org/jira/browse/HADOOP-792) | Invalid dfs -mv can trash your entire dfs | Major | . | Chris Schneider | | +| [HADOOP-786](https://issues.apache.org/jira/browse/HADOOP-786) | PhasedFileSystem should use debug level log for ignored exception. | Trivial | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-782](https://issues.apache.org/jira/browse/HADOOP-782) | TaskTracker.java:killOverflowingTasks & TaskTracker.java:markUnresponsiveTasks only put the tip in tasksToCleanup queue, they don't update the runningJobs | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-777](https://issues.apache.org/jira/browse/HADOOP-777) | the tasktracker hostname is not fully qualified | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-774](https://issues.apache.org/jira/browse/HADOOP-774) | Datanodes fails to heartbeat when a directory with a large number of blocks is deleted | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-770](https://issues.apache.org/jira/browse/HADOOP-770) | When JobTracker gets restarted, Job Tracker History doesn't show the jobs that were running. (incomplete jobs) | Minor | . | Koji Noguchi | Sanjay Dahiya | +| [HADOOP-764](https://issues.apache.org/jira/browse/HADOOP-764) | The memory consumption of processReport() in the namenode can be reduced | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-752](https://issues.apache.org/jira/browse/HADOOP-752) | Possible locking issues in HDFS Namenode | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-745](https://issues.apache.org/jira/browse/HADOOP-745) | NameNode throws FileNotFoundException: Parent path does not exist on startup | Major | . | Nigel Daley | dhruba borthakur | +| [HADOOP-738](https://issues.apache.org/jira/browse/HADOOP-738) | dfs get or copyToLocal should not copy crc file | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-737](https://issues.apache.org/jira/browse/HADOOP-737) | TaskTracker's job cleanup loop should check for finished job before deleting local directories | Critical | . | Sanjay Dahiya | Arun C Murthy | +| [HADOOP-700](https://issues.apache.org/jira/browse/HADOOP-700) | bin/hadoop includes in classpath all jar files in HADOOP\_HOME | Major | scripts | Nigel Daley | Doug Cutting | +| [HADOOP-673](https://issues.apache.org/jira/browse/HADOOP-673) | the task execution environment should have a current working directory that is task specific | Major | . | Owen O'Malley | Mahadev konar | +| [HADOOP-629](https://issues.apache.org/jira/browse/HADOOP-629) | none of the rpc servers check the protcol name for validity | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-628](https://issues.apache.org/jira/browse/HADOOP-628) | hadoop hdfs -cat replaces some characters with question marks. | Major | . | arkady borkovsky | Wendy Chien | +| [HADOOP-596](https://issues.apache.org/jira/browse/HADOOP-596) | TaskTracker taskstatus's phase doesnt get updated on phase transition causing wrong values displayed in WI | Major | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-546](https://issues.apache.org/jira/browse/HADOOP-546) | Task tracker doesnt generate job.xml in jobcache for some tasks ( possibly for only rescheduled tasks) | Critical | . | Sanjay Dahiya | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/RELEASENOTES.0.10.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/RELEASENOTES.0.10.0.md new file mode 100644 index 00000000000..63ab3480bc7 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.0/RELEASENOTES.0.10.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.10.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/CHANGES.0.10.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/CHANGES.0.10.1.md new file mode 100644 index 00000000000..dc0018d8e0a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/CHANGES.0.10.1.md @@ -0,0 +1,77 @@ + + +# Apache Hadoop Changelog + +## Release 0.10.1 - 2007-01-10 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-851](https://issues.apache.org/jira/browse/HADOOP-851) | Implement the LzoCodec with support for the lzo compression algorithms | Major | io | Arun C Murthy | Arun C Murthy | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-880](https://issues.apache.org/jira/browse/HADOOP-880) | Recursive delete for an S3 directory does not actually delete files or subdirectories | Major | fs | Tom White | Tom White | +| [HADOOP-879](https://issues.apache.org/jira/browse/HADOOP-879) | SequenceFileInputFormat can no longer read from data produced by MapFileOutputFormat | Major | . | Bryan Pendleton | Doug Cutting | +| [HADOOP-873](https://issues.apache.org/jira/browse/HADOOP-873) | native libraries aren't loaded unless the user specifies the java.library.path in the child jvm options | Major | util | Owen O'Malley | Owen O'Malley | +| [HADOOP-871](https://issues.apache.org/jira/browse/HADOOP-871) | java.library.path is wrongly initialized by bin/hadoop when only pre-built libs are present, but custom-built ones aren't | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-868](https://issues.apache.org/jira/browse/HADOOP-868) | Fix the merge method on Maps to limit the number of open files | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-866](https://issues.apache.org/jira/browse/HADOOP-866) | dfs -get should remove existing crc file if -crc is not specified | Major | fs | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-865](https://issues.apache.org/jira/browse/HADOOP-865) | Files written to S3 but never closed can't be deleted | Major | fs | Bryan Pendleton | Tom White | +| [HADOOP-864](https://issues.apache.org/jira/browse/HADOOP-864) | bin/hadoop jar throws file creation exception for temp files | Minor | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-863](https://issues.apache.org/jira/browse/HADOOP-863) | MapTask prints info log message when the progress-reporting thread starts | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-857](https://issues.apache.org/jira/browse/HADOOP-857) | IOException when running map reduce on S3 filesystem | Major | fs | Tom White | | +| [HADOOP-815](https://issues.apache.org/jira/browse/HADOOP-815) | Investigate and fix the extremely large memory-footprint of JobTracker | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-600](https://issues.apache.org/jira/browse/HADOOP-600) | Race condition in JobTracker updating the task tracker's status while declaring it lost | Major | . | Owen O'Malley | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/RELEASENOTES.0.10.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/RELEASENOTES.0.10.1.md new file mode 100644 index 00000000000..15a0fd5d421 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.10.1/RELEASENOTES.0.10.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.10.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/CHANGES.0.11.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/CHANGES.0.11.0.md new file mode 100644 index 00000000000..ccd45655db1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/CHANGES.0.11.0.md @@ -0,0 +1,118 @@ + + +# Apache Hadoop Changelog + +## Release 0.11.0 - 2007-02-02 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-967](https://issues.apache.org/jira/browse/HADOOP-967) | flip boolean to have rpc clients send a header | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-961](https://issues.apache.org/jira/browse/HADOOP-961) | a cli tool to get the event logs from a job | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-918](https://issues.apache.org/jira/browse/HADOOP-918) | Examples of Abacus using Python plugins | Major | . | Runping Qi | Runping Qi | +| [HADOOP-908](https://issues.apache.org/jira/browse/HADOOP-908) | Hadoop Abacus, a package for performing simple counting/aggregation | Major | . | Runping Qi | Runping Qi | +| [HADOOP-884](https://issues.apache.org/jira/browse/HADOOP-884) | Create scripts to run Hadoop on Amazon EC2 | Major | scripts | Tom White | Tom White | +| [HADOOP-852](https://issues.apache.org/jira/browse/HADOOP-852) | want ant task for record definitions | Major | record | Doug Cutting | Milind Bhandarkar | +| [HADOOP-732](https://issues.apache.org/jira/browse/HADOOP-732) | SequenceFile's header should allow to store metadata in the form of key/value pairs | Major | io | Runping Qi | Runping Qi | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-963](https://issues.apache.org/jira/browse/HADOOP-963) | improve the stack trace returned by RPC client | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-936](https://issues.apache.org/jira/browse/HADOOP-936) | More updates to metric names to conform to HADOOP-887 | Minor | . | Nigel Daley | | +| [HADOOP-897](https://issues.apache.org/jira/browse/HADOOP-897) | Need a simpler way to specify arbitrary options to java compiler while building Hadoop | Minor | build | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-890](https://issues.apache.org/jira/browse/HADOOP-890) | Update tag and metric names to conform to HADOOP-887 | Minor | metrics | Nigel Daley | Nigel Daley | +| [HADOOP-862](https://issues.apache.org/jira/browse/HADOOP-862) | Add handling of s3 to CopyFile tool | Minor | util | stack | | +| [HADOOP-842](https://issues.apache.org/jira/browse/HADOOP-842) | change the open method in ClientProtocol to take an additional argument: clientMachine | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-833](https://issues.apache.org/jira/browse/HADOOP-833) | need documentation of native build requirements | Major | documentation | Doug Cutting | Arun C Murthy | +| [HADOOP-830](https://issues.apache.org/jira/browse/HADOOP-830) | Improve the performance of the Merge phase | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-805](https://issues.apache.org/jira/browse/HADOOP-805) | JobClient should print the Task's stdout and stderr to the clients console | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-801](https://issues.apache.org/jira/browse/HADOOP-801) | job tracker should keep a log of task completion and failure | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-788](https://issues.apache.org/jira/browse/HADOOP-788) | Streaming should use a subclass of TextInputFormat for reading text inputs. | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-726](https://issues.apache.org/jira/browse/HADOOP-726) | HDFS locking mechanisms should be simplified or removed | Minor | . | Sameer Paranjpye | Raghu Angadi | +| [HADOOP-692](https://issues.apache.org/jira/browse/HADOOP-692) | Rack-aware Replica Placement | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-659](https://issues.apache.org/jira/browse/HADOOP-659) | Boost the priority of re-replicating blocks that are far from their replication target | Major | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-602](https://issues.apache.org/jira/browse/HADOOP-602) | Remove Lucene dependency | Major | . | Andrzej Bialecki | Milind Bhandarkar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-969](https://issues.apache.org/jira/browse/HADOOP-969) | deadlock in job tracker RetireJobs | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-965](https://issues.apache.org/jira/browse/HADOOP-965) | Isolation Runner looking for job.jar in wrong directory | Major | . | Dennis Kubes | | +| [HADOOP-964](https://issues.apache.org/jira/browse/HADOOP-964) | ClassNotFoundException in ReduceTaskRunner | Blocker | . | Dennis Kubes | | +| [HADOOP-962](https://issues.apache.org/jira/browse/HADOOP-962) | Hadoop EC2 scripts are not executable | Major | scripts | Tom White | Tom White | +| [HADOOP-959](https://issues.apache.org/jira/browse/HADOOP-959) | TestCheckpoint fails on Windows | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-937](https://issues.apache.org/jira/browse/HADOOP-937) | data node re-registration | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-935](https://issues.apache.org/jira/browse/HADOOP-935) | Abacus should not delete the output dir | Major | . | Runping Qi | Runping Qi | +| [HADOOP-934](https://issues.apache.org/jira/browse/HADOOP-934) | TaskTracker sends duplicate status when updating task metrics throws exception | Major | . | Nigel Daley | Arun C Murthy | +| [HADOOP-929](https://issues.apache.org/jira/browse/HADOOP-929) | PhasedFileSystem should implement get/set configuration | Major | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-922](https://issues.apache.org/jira/browse/HADOOP-922) | Optimize small reads and seeks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-920](https://issues.apache.org/jira/browse/HADOOP-920) | MapFileOutputFormat and SequenceFileOutputFormat use incorrect key/value classes in map/reduce tasks | Major | . | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-916](https://issues.apache.org/jira/browse/HADOOP-916) | HADOOP-908 patch causes javadoc warnings | Trivial | . | Nigel Daley | Nigel Daley | +| [HADOOP-912](https://issues.apache.org/jira/browse/HADOOP-912) | TestMiniMRWithDFS fails sporadically | Major | . | Nigel Daley | Arun C Murthy | +| [HADOOP-909](https://issues.apache.org/jira/browse/HADOOP-909) | dfs "du" shows that the size of a subdirectory is 0 | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-905](https://issues.apache.org/jira/browse/HADOOP-905) | Code to qualify inputDirs doesn't affect path validation | Major | fs | Kenji Matsuoka | | +| [HADOOP-902](https://issues.apache.org/jira/browse/HADOOP-902) | NPE in DFSOutputStream.closeBackupStream() | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-901](https://issues.apache.org/jira/browse/HADOOP-901) | Make S3FileSystem do recursive renames | Major | fs | Tom White | | +| [HADOOP-899](https://issues.apache.org/jira/browse/HADOOP-899) | Removal of deprecated code (in v0.10.0) from trunk breaks libhdfs | Major | . | Sameer Paranjpye | Sameer Paranjpye | +| [HADOOP-898](https://issues.apache.org/jira/browse/HADOOP-898) | namenode generates infinite stream of null pointers | Major | . | Owen O'Malley | Raghu Angadi | +| [HADOOP-886](https://issues.apache.org/jira/browse/HADOOP-886) | thousands of TimerThreads created by metrics API | Major | metrics | Nigel Daley | Nigel Daley | +| [HADOOP-881](https://issues.apache.org/jira/browse/HADOOP-881) | job history web/ui does not count task failures correctly | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-856](https://issues.apache.org/jira/browse/HADOOP-856) | fsck reports a non-existant DFS path as healthy | Minor | . | Nigel Daley | Milind Bhandarkar | +| [HADOOP-855](https://issues.apache.org/jira/browse/HADOOP-855) | HDFS should repair corrupted files | Major | . | Wendy Chien | Wendy Chien | +| [HADOOP-781](https://issues.apache.org/jira/browse/HADOOP-781) | Remove from trunk things deprecated in 0.10 branch. | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-758](https://issues.apache.org/jira/browse/HADOOP-758) | FileNotFound on DFS block file | Major | . | Owen O'Malley | Raghu Angadi | +| [HADOOP-757](https://issues.apache.org/jira/browse/HADOOP-757) | "Bad File Descriptor" in closing DFS file | Major | . | Owen O'Malley | Raghu Angadi | +| [HADOOP-735](https://issues.apache.org/jira/browse/HADOOP-735) | The underlying data structure, ByteArrayOutputStream, for buffer type of Hadoop record is inappropriate | Major | record | Runping Qi | Milind Bhandarkar | +| [HADOOP-731](https://issues.apache.org/jira/browse/HADOOP-731) | Sometimes when a dfs file is accessed and one copy has a checksum error the I/O command fails, even if another copy is alright. | Major | . | Dick King | Wendy Chien | +| [HADOOP-549](https://issues.apache.org/jira/browse/HADOOP-549) | NullPointerException in TaskReport's serialization code | Major | . | Michel Tourn | Owen O'Malley | +| [HADOOP-405](https://issues.apache.org/jira/browse/HADOOP-405) | Duplicate browseDirectory.jsp | Minor | . | Konstantin Shvachko | navychen | +| [HADOOP-309](https://issues.apache.org/jira/browse/HADOOP-309) | NullPointerException in StatusHttpServer | Minor | . | Konstantin Shvachko | navychen | +| [HADOOP-227](https://issues.apache.org/jira/browse/HADOOP-227) | Namespace check pointing is not performed until the namenode restarts. | Major | . | Konstantin Shvachko | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/RELEASENOTES.0.11.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/RELEASENOTES.0.11.0.md new file mode 100644 index 00000000000..85d59dc7272 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.0/RELEASENOTES.0.11.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.11.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/CHANGES.0.11.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/CHANGES.0.11.1.md new file mode 100644 index 00000000000..2a01ac441a1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/CHANGES.0.11.1.md @@ -0,0 +1,68 @@ + + +# Apache Hadoop Changelog + +## Release 0.11.1 - 2007-02-09 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-987](https://issues.apache.org/jira/browse/HADOOP-987) | pendingTransfer does not remove a block from neededReplications correctly after targets are chosen | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-984](https://issues.apache.org/jira/browse/HADOOP-984) | Exception while retrieving a map output index file is not reported to JobTracker | Critical | . | Nigel Daley | Arun C Murthy | +| [HADOOP-976](https://issues.apache.org/jira/browse/HADOOP-976) | SequenceFile.Metadata class should be public | Major | . | Runping Qi | Runping Qi | +| [HADOOP-917](https://issues.apache.org/jira/browse/HADOOP-917) | NPE in org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue | Major | io | Sami Siren | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/RELEASENOTES.0.11.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/RELEASENOTES.0.11.1.md new file mode 100644 index 00000000000..119b8d20b25 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.1/RELEASENOTES.0.11.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.11.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/CHANGES.0.11.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/CHANGES.0.11.2.md new file mode 100644 index 00000000000..669831d8926 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/CHANGES.0.11.2.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.11.2 - 2007-02-16 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1014](https://issues.apache.org/jira/browse/HADOOP-1014) | map/reduce is corrupting data between map and reduce | Blocker | . | Owen O'Malley | Devaraj Das | +| [HADOOP-1013](https://issues.apache.org/jira/browse/HADOOP-1013) | ArithmeticException when number of racks is zero | Major | . | James P. White | Hairong Kuang | +| [HADOOP-1009](https://issues.apache.org/jira/browse/HADOOP-1009) | Infinite loop in chooseTarget | Major | . | dhruba borthakur | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/RELEASENOTES.0.11.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/RELEASENOTES.0.11.2.md new file mode 100644 index 00000000000..24da1479020 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.11.2/RELEASENOTES.0.11.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.11.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/CHANGES.0.12.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/CHANGES.0.12.0.md new file mode 100644 index 00000000000..bcac77275e2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/CHANGES.0.12.0.md @@ -0,0 +1,129 @@ + + +# Apache Hadoop Changelog + +## Release 0.12.0 - 2007-03-02 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1032](https://issues.apache.org/jira/browse/HADOOP-1032) | Support for caching Job JARs | Minor | . | Gautam Kowshik | Gautam Kowshik | +| [HADOOP-492](https://issues.apache.org/jira/browse/HADOOP-492) | Global counters | Major | . | arkady borkovsky | David Bowen | +| [HADOOP-491](https://issues.apache.org/jira/browse/HADOOP-491) | streaming jobs should allow programs that don't do any IO for a long time | Major | . | arkady borkovsky | Arun C Murthy | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1043](https://issues.apache.org/jira/browse/HADOOP-1043) | Optimize the shuffle phase (increase the parallelism) | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1042](https://issues.apache.org/jira/browse/HADOOP-1042) | Improve the handling of failed map output fetches | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1041](https://issues.apache.org/jira/browse/HADOOP-1041) | Counter names are ugly | Major | . | Owen O'Malley | David Bowen | +| [HADOOP-1040](https://issues.apache.org/jira/browse/HADOOP-1040) | Improvement of RandomWriter example to use custom InputFormat, OutputFormat, and Counters | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1033](https://issues.apache.org/jira/browse/HADOOP-1033) | Rewrite AmazonEC2 wiki page | Minor | scripts | Tom White | Tom White | +| [HADOOP-1030](https://issues.apache.org/jira/browse/HADOOP-1030) | in unit tests, set ipc timeout in one place | Minor | test | Doug Cutting | Doug Cutting | +| [HADOOP-1025](https://issues.apache.org/jira/browse/HADOOP-1025) | remove dead code in Server.java | Minor | ipc | Doug Cutting | Doug Cutting | +| [HADOOP-1017](https://issues.apache.org/jira/browse/HADOOP-1017) | Optimization: Reduce Overhead from ReflectionUtils.newInstance | Major | util | Ron Bodkin | | +| [HADOOP-1007](https://issues.apache.org/jira/browse/HADOOP-1007) | Names used for map, reduce, and shuffle metrics should be unique | Trivial | metrics | Nigel Daley | Nigel Daley | +| [HADOOP-997](https://issues.apache.org/jira/browse/HADOOP-997) | Implement S3 retry mechanism for failed block transfers | Major | fs | Tom White | Tom White | +| [HADOOP-985](https://issues.apache.org/jira/browse/HADOOP-985) | Namenode should identify DataNodes as ip:port instead of hostname:port | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-982](https://issues.apache.org/jira/browse/HADOOP-982) | A couple setter functions and toString method for BytesWritable. | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-977](https://issues.apache.org/jira/browse/HADOOP-977) | The output from the user's task should be tagged and sent to the resepective console streams. | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-975](https://issues.apache.org/jira/browse/HADOOP-975) | Separation of user tasks' stdout and stderr streams | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-972](https://issues.apache.org/jira/browse/HADOOP-972) | Improve the rack-aware replica placement performance | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-954](https://issues.apache.org/jira/browse/HADOOP-954) | Metrics should offer complete set of static report methods or none at all | Minor | metrics | Nigel Daley | David Bowen | +| [HADOOP-952](https://issues.apache.org/jira/browse/HADOOP-952) | Create a public (shared) Hadoop EC2 AMI | Major | scripts | Tom White | Tom White | +| [HADOOP-943](https://issues.apache.org/jira/browse/HADOOP-943) | fsck to show the filename of the corrupted file | Trivial | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-941](https://issues.apache.org/jira/browse/HADOOP-941) | Enhancements to Hadoop record I/O - Part 1 | Major | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-940](https://issues.apache.org/jira/browse/HADOOP-940) | pendingReplications of FSNamesystem is not informative | Major | . | Hairong Kuang | dhruba borthakur | +| [HADOOP-928](https://issues.apache.org/jira/browse/HADOOP-928) | make checksums optional per FileSystem | Major | fs | Doug Cutting | Hairong Kuang | +| [HADOOP-889](https://issues.apache.org/jira/browse/HADOOP-889) | DFS unit tests have duplicate code | Minor | test | Doug Cutting | Milind Bhandarkar | +| [HADOOP-882](https://issues.apache.org/jira/browse/HADOOP-882) | S3FileSystem should retry if there is a communication problem with S3 | Major | fs | Tom White | Tom White | +| [HADOOP-867](https://issues.apache.org/jira/browse/HADOOP-867) | job client should generate input fragments before the job is submitted | Major | . | Owen O'Malley | | +| [HADOOP-858](https://issues.apache.org/jira/browse/HADOOP-858) | clean up smallJobsBenchmark and move to src/test/org/apache/hadoop/mapred | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-432](https://issues.apache.org/jira/browse/HADOOP-432) | support undelete, snapshots, or other mechanism to recover lost files | Major | . | Yoram Arnon | Doug Cutting | +| [HADOOP-333](https://issues.apache.org/jira/browse/HADOOP-333) | we should have some checks that the sort benchmark generates correct outputs | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-248](https://issues.apache.org/jira/browse/HADOOP-248) | locating map outputs via random probing is inefficient | Major | . | Owen O'Malley | Devaraj Das | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1056](https://issues.apache.org/jira/browse/HADOOP-1056) | Decommission only recognizes IP addesses in hosts and exclude files on refresh. | Major | . | Wendy Chien | Wendy Chien | +| [HADOOP-1049](https://issues.apache.org/jira/browse/HADOOP-1049) | race condition in setting up ipc connections | Major | ipc | Owen O'Malley | Devaraj Das | +| [HADOOP-1046](https://issues.apache.org/jira/browse/HADOOP-1046) | Datanode should periodically clean up /tmp from partially received (and not completed) block files | Major | . | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-1044](https://issues.apache.org/jira/browse/HADOOP-1044) | TestDecommission fails because it attempts to transfer block to a dead datanode | Major | test | Wendy Chien | Wendy Chien | +| [HADOOP-1039](https://issues.apache.org/jira/browse/HADOOP-1039) | Reduce the time taken by TestCheckpoint | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1037](https://issues.apache.org/jira/browse/HADOOP-1037) | bin/slaves.sh not compatible with /bin/dash | Major | . | Doug Cutting | | +| [HADOOP-1036](https://issues.apache.org/jira/browse/HADOOP-1036) | task gets lost during assignment | Critical | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-1034](https://issues.apache.org/jira/browse/HADOOP-1034) | RuntimeException and Error not catched in DataNode.DataXceiver.run() | Major | . | Philippe Gassmann | | +| [HADOOP-1029](https://issues.apache.org/jira/browse/HADOOP-1029) | streaming doesn't work with multiple maps | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1027](https://issues.apache.org/jira/browse/HADOOP-1027) | Fix the RAM FileSystem/Merge problems (reported in HADOOP-1014) | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1021](https://issues.apache.org/jira/browse/HADOOP-1021) | TestMiniMRLocalFS and TestMiniMRCaching broken on Windows | Major | test | Nigel Daley | | +| [HADOOP-1020](https://issues.apache.org/jira/browse/HADOOP-1020) | Path class on Windows seems broken | Major | . | Nigel Daley | Doug Cutting | +| [HADOOP-1010](https://issues.apache.org/jira/browse/HADOOP-1010) | getReordReader methof of InputFormat class should handle null reporter argument | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1006](https://issues.apache.org/jira/browse/HADOOP-1006) | The "-local" option does work properly with test programs | Minor | test | Gautam Kowshik | Doug Cutting | +| [HADOOP-1000](https://issues.apache.org/jira/browse/HADOOP-1000) | Loggers in the Task framework should not write the the Tasks stderr | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-994](https://issues.apache.org/jira/browse/HADOOP-994) | DFS Scalability : a BlockReport that returns large number of blocks-to-be-deleted cause datanode to lost connectivity to namenode | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-992](https://issues.apache.org/jira/browse/HADOOP-992) | The mini/mr cluster for testing always uses the local file system rather than the namenode that was passed in | Major | test | Owen O'Malley | Owen O'Malley | +| [HADOOP-990](https://issues.apache.org/jira/browse/HADOOP-990) | Datanode doesn't retry when write to one (full)drive fail | Major | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-973](https://issues.apache.org/jira/browse/HADOOP-973) | NPE in FSDataset during heavy Namenode load | Major | . | Nigel Daley | dhruba borthakur | +| [HADOOP-947](https://issues.apache.org/jira/browse/HADOOP-947) | isReplicationInProgress() is very heavyweight | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-933](https://issues.apache.org/jira/browse/HADOOP-933) | Application defined InputSplits do not work | Major | . | Benjamin Reed | Owen O'Malley | +| [HADOOP-923](https://issues.apache.org/jira/browse/HADOOP-923) | DFS Scalability: datanode heartbeat timeouts cause cascading timeouts of other datanodes | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-893](https://issues.apache.org/jira/browse/HADOOP-893) | dead datanode set should be maintained in the file handle or file system for hdfs | Major | . | Owen O'Malley | Raghu Angadi | +| [HADOOP-878](https://issues.apache.org/jira/browse/HADOOP-878) | reducer NONE does not work with multiple maps | Minor | . | Mahadev konar | Arun C Murthy | +| [HADOOP-803](https://issues.apache.org/jira/browse/HADOOP-803) | Reducing memory consumption on Namenode : Part 1 | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-761](https://issues.apache.org/jira/browse/HADOOP-761) | Unit tests should cleanup created files in /tmp. It causes tests to fail if more than one users run tests on same machine. | Minor | test | Sanjay Dahiya | Nigel Daley | +| [HADOOP-654](https://issues.apache.org/jira/browse/HADOOP-654) | jobs fail with some hardware/system failures on a small number of nodes | Minor | . | Yoram Arnon | Arun C Murthy | +| [HADOOP-649](https://issues.apache.org/jira/browse/HADOOP-649) | Jobs without any map and reduce operations seems to be lost after their execution | Major | . | Thomas Friol | Owen O'Malley | +| [HADOOP-564](https://issues.apache.org/jira/browse/HADOOP-564) | we should use hdfs:// in all API URIs | Major | . | eric baldeschwieler | Wendy Chien | +| [HADOOP-476](https://issues.apache.org/jira/browse/HADOOP-476) | Streaming should check for correctness of the task | Major | . | arkady borkovsky | Arun C Murthy | +| [HADOOP-442](https://issues.apache.org/jira/browse/HADOOP-442) | slaves file should include an 'exclude' section, to prevent "bad" datanodes and tasktrackers from disrupting a cluster | Major | conf | Yoram Arnon | Wendy Chien | +| [HADOOP-338](https://issues.apache.org/jira/browse/HADOOP-338) | the number of maps in the JobConf does not match reality | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-109](https://issues.apache.org/jira/browse/HADOOP-109) | Blocks are not replicated when... | Major | . | Konstantin Shvachko | Konstantin Shvachko | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1051](https://issues.apache.org/jira/browse/HADOOP-1051) | Add checkstyle target to ant build file | Major | build, test | Tom White | Tom White | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/RELEASENOTES.0.12.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/RELEASENOTES.0.12.0.md new file mode 100644 index 00000000000..e8d86befc15 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.0/RELEASENOTES.0.12.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.12.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/CHANGES.0.12.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/CHANGES.0.12.1.md new file mode 100644 index 00000000000..0c69aad003f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/CHANGES.0.12.1.md @@ -0,0 +1,87 @@ + + +# Apache Hadoop Changelog + +## Release 0.12.1 - 2007-03-17 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1096](https://issues.apache.org/jira/browse/HADOOP-1096) | Rename InputArchive and OutputArchive and make them public | Major | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-1053](https://issues.apache.org/jira/browse/HADOOP-1053) | Make Record I/O functionally modular from the rest of Hadoop | Major | record | Milind Bhandarkar | Milind Bhandarkar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1129](https://issues.apache.org/jira/browse/HADOOP-1129) | The DFSClient hides IOExceptions in flush | Major | . | Owen O'Malley | Hairong Kuang | +| [HADOOP-1128](https://issues.apache.org/jira/browse/HADOOP-1128) | Missing progress information in map tasks | Major | . | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-1126](https://issues.apache.org/jira/browse/HADOOP-1126) | Optimize CPU usage when cluster restarts | Major | . | dhruba borthakur | Hairong Kuang | +| [HADOOP-1117](https://issues.apache.org/jira/browse/HADOOP-1117) | DFS Scalability: When the namenode is restarted it consumes 80% CPU | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1115](https://issues.apache.org/jira/browse/HADOOP-1115) | copyToLocal doesn't copy directories | Blocker | . | Nigel Daley | | +| [HADOOP-1112](https://issues.apache.org/jira/browse/HADOOP-1112) | Race condition in Hadoop metrics | Major | . | David Bowen | | +| [HADOOP-1109](https://issues.apache.org/jira/browse/HADOOP-1109) | Streaming, NPE when reading sequencefile | Major | . | Koji Noguchi | | +| [HADOOP-1108](https://issues.apache.org/jira/browse/HADOOP-1108) | Checksumed file system should retry reading if a different replica is found when handle ChecksumException | Blocker | . | dhruba borthakur | Hairong Kuang | +| [HADOOP-1099](https://issues.apache.org/jira/browse/HADOOP-1099) | NullPointerException in JobInProgress.getTaskInProgress | Major | . | Nigel Daley | Gautam Kowshik | +| [HADOOP-1092](https://issues.apache.org/jira/browse/HADOOP-1092) | NullPointerException in HeartbeatMonitor thread | Blocker | . | Nigel Daley | Hairong Kuang | +| [HADOOP-1091](https://issues.apache.org/jira/browse/HADOOP-1091) | NPE from Simon in JT stdout | Major | . | David Bowen | David Bowen | +| [HADOOP-1089](https://issues.apache.org/jira/browse/HADOOP-1089) | The c++ version of write and read v-int don't agree with the java versions | Major | record | Owen O'Malley | Milind Bhandarkar | +| [HADOOP-1088](https://issues.apache.org/jira/browse/HADOOP-1088) | Csv and Xml serialization for buffers do not work for byte value of -1 | Blocker | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-1083](https://issues.apache.org/jira/browse/HADOOP-1083) | Replication not occuring after cluster restart when datanodes missing | Blocker | . | Nigel Daley | Hairong Kuang | +| [HADOOP-1082](https://issues.apache.org/jira/browse/HADOOP-1082) | NullpointerException in ChecksumFileSystem$FSInputChecker.seek | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1080](https://issues.apache.org/jira/browse/HADOOP-1080) | Cygwin path translation should occur earlier in bin/hadoop | Major | scripts | Andrzej Bialecki | | +| [HADOOP-1077](https://issues.apache.org/jira/browse/HADOOP-1077) | Race condition in fetching map outputs (might lead to hung reduces) | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-1070](https://issues.apache.org/jira/browse/HADOOP-1070) | Number of racks and datanode double temporarily when upgrading from 0.10.1 to 0.11.2 | Blocker | . | Nigel Daley | Konstantin Shvachko | +| [HADOOP-1067](https://issues.apache.org/jira/browse/HADOOP-1067) | Compile fails if Checkstyle jar is present in lib directory | Major | build | Tom White | Tom White | +| [HADOOP-1060](https://issues.apache.org/jira/browse/HADOOP-1060) | IndexOutOfBoundsException in JobInProgress.updateTaskStatus leads to hung jobs | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1035](https://issues.apache.org/jira/browse/HADOOP-1035) | StackOverflowError in FSDataSet | Blocker | . | Philippe Gassmann | Raghu Angadi | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/RELEASENOTES.0.12.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/RELEASENOTES.0.12.1.md new file mode 100644 index 00000000000..5051099f1af --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.1/RELEASENOTES.0.12.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.12.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/CHANGES.0.12.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/CHANGES.0.12.2.md new file mode 100644 index 00000000000..f473cbf5bc6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/CHANGES.0.12.2.md @@ -0,0 +1,68 @@ + + +# Apache Hadoop Changelog + +## Release 0.12.2 - 2007-03-23 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1150](https://issues.apache.org/jira/browse/HADOOP-1150) | Streaming -reducer and -mapper no longer have defaults | Major | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1145](https://issues.apache.org/jira/browse/HADOOP-1145) | XmlRecordInput class should be public | Major | record | Runping Qi | Milind Bhandarkar | +| [HADOOP-1140](https://issues.apache.org/jira/browse/HADOOP-1140) | Deadlock bug involving the o.a.h.metrics package | Major | metrics | David Bowen | David Bowen | +| [HADOOP-1135](https://issues.apache.org/jira/browse/HADOOP-1135) | A block report processing may incorrectly cause the namenode to delete blocks | Major | . | dhruba borthakur | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/RELEASENOTES.0.12.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/RELEASENOTES.0.12.2.md new file mode 100644 index 00000000000..dffca2b2458 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.2/RELEASENOTES.0.12.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.12.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/CHANGES.0.12.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/CHANGES.0.12.3.md new file mode 100644 index 00000000000..62a8420b8c6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/CHANGES.0.12.3.md @@ -0,0 +1,72 @@ + + +# Apache Hadoop Changelog + +## Release 0.12.3 - 2007-04-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1192](https://issues.apache.org/jira/browse/HADOOP-1192) | Du command takes a noticable longer time to execute on a large dfs than the 0.11 release | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1191](https://issues.apache.org/jira/browse/HADOOP-1191) | MapTask should wait for the status reporting thread to die before invoking the TaskUmbilicalProtocol.done(taskid) | Major | . | Devaraj Das | Doug Cutting | +| [HADOOP-1179](https://issues.apache.org/jira/browse/HADOOP-1179) | task Tracker should be restarted if its jetty http server cannot serve get-map-output files | Major | . | Runping Qi | Devaraj Das | +| [HADOOP-1177](https://issues.apache.org/jira/browse/HADOOP-1177) | Lack of logging of exceptions in MapOutputLocation.getFile | Major | . | Owen O'Malley | Devaraj Das | +| [HADOOP-1175](https://issues.apache.org/jira/browse/HADOOP-1175) | the user output/logs are broken from the web ui | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-1162](https://issues.apache.org/jira/browse/HADOOP-1162) | Record IO: seariliizing a byte buffer to CSV fails if buffer contains bytes less than 16. | Major | . | David Bowen | Milind Bhandarkar | +| [HADOOP-1123](https://issues.apache.org/jira/browse/HADOOP-1123) | LocalFileSystem gets a NullPointerException when tries to recover from ChecksumError | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-1105](https://issues.apache.org/jira/browse/HADOOP-1105) | Reducers don't make "progress" while iterating through values | Major | . | Owen O'Malley | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/RELEASENOTES.0.12.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/RELEASENOTES.0.12.3.md new file mode 100644 index 00000000000..a9cb6510cd8 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.12.3/RELEASENOTES.0.12.3.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.12.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/CHANGES.0.13.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/CHANGES.0.13.0.md new file mode 100644 index 00000000000..5dcd5c656b7 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/CHANGES.0.13.0.md @@ -0,0 +1,195 @@ + + +# Apache Hadoop Changelog + +## Release 0.13.0 - 2007-06-08 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1251](https://issues.apache.org/jira/browse/HADOOP-1251) | A method to get the InputSplit from a Mapper | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1247](https://issues.apache.org/jira/browse/HADOOP-1247) | Make Hadoop Abacus work with Hadoop Streaming | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1217](https://issues.apache.org/jira/browse/HADOOP-1217) | Specify a junit test timeout in build.xml files | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-1216](https://issues.apache.org/jira/browse/HADOOP-1216) | Hadoop should support reduce none option | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1120](https://issues.apache.org/jira/browse/HADOOP-1120) | Contribute some code helping implement map/reduce apps for joining data from multiple sources | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1111](https://issues.apache.org/jira/browse/HADOOP-1111) | Job completion notification to a job configured URL | Major | . | Alejandro Abdelnur | | +| [HADOOP-702](https://issues.apache.org/jira/browse/HADOOP-702) | DFS Upgrade Proposal | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-485](https://issues.apache.org/jira/browse/HADOOP-485) | allow a different comparator for grouping keys in calls to reduce | Major | . | Owen O'Malley | Tahir Hashmi | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1326](https://issues.apache.org/jira/browse/HADOOP-1326) | Return the RunningJob from JobClient.runJob | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1324](https://issues.apache.org/jira/browse/HADOOP-1324) | FSError encountered by one running task should not be fatal to other tasks on that node | Major | . | Devaraj Das | Arun C Murthy | +| [HADOOP-1315](https://issues.apache.org/jira/browse/HADOOP-1315) | Hadoop Streaming code clean up | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1308](https://issues.apache.org/jira/browse/HADOOP-1308) | Tighten generic Class restrictions in JobConf.java | Minor | . | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-1304](https://issues.apache.org/jira/browse/HADOOP-1304) | MAX\_TASK\_FAILURES should be configurable | Major | . | Christian Kunz | Devaraj Das | +| [HADOOP-1290](https://issues.apache.org/jira/browse/HADOOP-1290) | Move Hadoop Abacus to hadoop.mapred.lib | Major | . | Runping Qi | | +| [HADOOP-1284](https://issues.apache.org/jira/browse/HADOOP-1284) | clean up the protocol between stream mapper/reducer and the framework | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1276](https://issues.apache.org/jira/browse/HADOOP-1276) | TaskTracker expiry interval is not configurable | Major | . | Alejandro Abdelnur | Arun C Murthy | +| [HADOOP-1270](https://issues.apache.org/jira/browse/HADOOP-1270) | Randomize the fetch of map outputs | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1263](https://issues.apache.org/jira/browse/HADOOP-1263) | retry logic when dfs exist or open fails temporarily, e.g because of timeout | Major | . | Christian Kunz | Hairong Kuang | +| [HADOOP-1260](https://issues.apache.org/jira/browse/HADOOP-1260) | need code review guidelines | Major | build | Nigel Daley | Nigel Daley | +| [HADOOP-1250](https://issues.apache.org/jira/browse/HADOOP-1250) | Remove the MustangFile class from streaming and promote the chmod into FileUtils | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1214](https://issues.apache.org/jira/browse/HADOOP-1214) | the first step for streaming clean up | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1213](https://issues.apache.org/jira/browse/HADOOP-1213) | When RPC call fails then log call message detail | Minor | ipc | Nigel Daley | Doug Cutting | +| [HADOOP-1194](https://issues.apache.org/jira/browse/HADOOP-1194) | map output should not do block level compression | Major | . | Runping Qi | Arun C Murthy | +| [HADOOP-1190](https://issues.apache.org/jira/browse/HADOOP-1190) | Fix unchecked warnings | Major | . | Tom White | Tom White | +| [HADOOP-1167](https://issues.apache.org/jira/browse/HADOOP-1167) | InMemoryFileSystem uses synchronizedtMaps with maps that are locked anyways | Minor | fs | Owen O'Malley | Owen O'Malley | +| [HADOOP-1166](https://issues.apache.org/jira/browse/HADOOP-1166) | Pull the NullOutputFormat into the lib package | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1165](https://issues.apache.org/jira/browse/HADOOP-1165) | Code for toString in code generated by Record I/O Compiler can be generic | Minor | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-1161](https://issues.apache.org/jira/browse/HADOOP-1161) | need improved release process | Major | build | Doug Cutting | Doug Cutting | +| [HADOOP-1148](https://issues.apache.org/jira/browse/HADOOP-1148) | re-indent all code | Minor | . | Doug Cutting | Doug Cutting | +| [HADOOP-1144](https://issues.apache.org/jira/browse/HADOOP-1144) | Hadoop should allow a configurable percentage of failed map tasks before declaring a job failed. | Major | . | Christian Kunz | Arun C Murthy | +| [HADOOP-1133](https://issues.apache.org/jira/browse/HADOOP-1133) | Tools to analyze and debug namenode on a production cluster | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1131](https://issues.apache.org/jira/browse/HADOOP-1131) | Add a closeAll() static method to FileSystem | Minor | . | Philippe Gassmann | | +| [HADOOP-1127](https://issues.apache.org/jira/browse/HADOOP-1127) | Speculative Execution and output of Reduce tasks | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1116](https://issues.apache.org/jira/browse/HADOOP-1116) | Add maxmemory="256m" in the junit call of build-contrib.xml | Major | build | Philippe Gassmann | | +| [HADOOP-1101](https://issues.apache.org/jira/browse/HADOOP-1101) | Add more statistics in the web-ui to do with tasks | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1094](https://issues.apache.org/jira/browse/HADOOP-1094) | Optimize readFields and write methods in record I/O | Major | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-1068](https://issues.apache.org/jira/browse/HADOOP-1068) | Improve error message for 0 datanode case | Major | . | Owen O'Malley | dhruba borthakur | +| [HADOOP-988](https://issues.apache.org/jira/browse/HADOOP-988) | Namenode should use single map for block to its meta data. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-978](https://issues.apache.org/jira/browse/HADOOP-978) | AlreadyBeingCreatedException detail message could contain more useful info | Minor | . | Nigel Daley | Konstantin Shvachko | +| [HADOOP-971](https://issues.apache.org/jira/browse/HADOOP-971) | DFS Scalabilty: Improve name node performance by adding a hostname to datanodes map | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-968](https://issues.apache.org/jira/browse/HADOOP-968) | Reduce shuffle and merge should be done a child JVM | Major | . | Owen O'Malley | Devaraj Das | +| [HADOOP-819](https://issues.apache.org/jira/browse/HADOOP-819) | LineRecordWriter should not always insert tab char between key and value | Major | . | Runping Qi | Runping Qi | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1452](https://issues.apache.org/jira/browse/HADOOP-1452) | map output transfers of more than 2^31 bytes output are failing | Blocker | . | Christian Kunz | Owen O'Malley | +| [HADOOP-1435](https://issues.apache.org/jira/browse/HADOOP-1435) | FileSystem.globPaths should not create a Path from an empty string | Blocker | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-1431](https://issues.apache.org/jira/browse/HADOOP-1431) | Map tasks can't timeout for failing to call progress | Blocker | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-1427](https://issues.apache.org/jira/browse/HADOOP-1427) | Typo in GzipCodec.createInputStream - bufferSize | Blocker | io | Espen Amble Kolstad | Espen Amble Kolstad | +| [HADOOP-1411](https://issues.apache.org/jira/browse/HADOOP-1411) | AlreadyBeingCreatedException from task retries | Blocker | . | Nigel Daley | Hairong Kuang | +| [HADOOP-1407](https://issues.apache.org/jira/browse/HADOOP-1407) | Failed tasks not killing job | Blocker | . | Nigel Daley | Arun C Murthy | +| [HADOOP-1388](https://issues.apache.org/jira/browse/HADOOP-1388) | Possible Null Pointer Dereference in taskdetails.jsp | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1386](https://issues.apache.org/jira/browse/HADOOP-1386) | The constructor of Path should not take an empty string as a parameter | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1385](https://issues.apache.org/jira/browse/HADOOP-1385) | MD5Hash has a bad hash function | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-1369](https://issues.apache.org/jira/browse/HADOOP-1369) | Inconsistent synchronization of TaskTracker fields | Blocker | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1368](https://issues.apache.org/jira/browse/HADOOP-1368) | Inconsistent synchronization of 3 fields in JobInProgress.java | Blocker | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1363](https://issues.apache.org/jira/browse/HADOOP-1363) | waitForCompletion() calls Thread.sleep() with a lock held | Blocker | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1361](https://issues.apache.org/jira/browse/HADOOP-1361) | seek calls in 3 io classes ignore result of skipBytes(int) | Blocker | io | Nigel Daley | Hairong Kuang | +| [HADOOP-1358](https://issues.apache.org/jira/browse/HADOOP-1358) | seek call ignores result of skipBytes(int) | Blocker | . | Nigel Daley | Hairong Kuang | +| [HADOOP-1356](https://issues.apache.org/jira/browse/HADOOP-1356) | ValueHistogram.addNextValue(Object) ignores return value of String.substring(int, int) | Blocker | . | Nigel Daley | Runping Qi | +| [HADOOP-1354](https://issues.apache.org/jira/browse/HADOOP-1354) | Null pointer dereference of paths in FsShell.dus(String) | Blocker | fs | Nigel Daley | Hairong Kuang | +| [HADOOP-1353](https://issues.apache.org/jira/browse/HADOOP-1353) | Null pointer dereference of nodeInfo in FSNamesystem.removeDatanode(DatanodeID) | Blocker | . | Nigel Daley | dhruba borthakur | +| [HADOOP-1350](https://issues.apache.org/jira/browse/HADOOP-1350) | Shuffle started taking a very long time after the HADOOP-1176 fix | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-1345](https://issues.apache.org/jira/browse/HADOOP-1345) | Checksum object does not get restored to the old state in retries when handle ChecksumException | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1332](https://issues.apache.org/jira/browse/HADOOP-1332) | Sporadic unit test failures (TestMiniMRClasspath, TestMiniMRLocalFS, TestMiniMRDFSCaching) | Blocker | . | Nigel Daley | Arun C Murthy | +| [HADOOP-1322](https://issues.apache.org/jira/browse/HADOOP-1322) | Tasktracker blacklist leads to hung jobs in single-node cluster | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1312](https://issues.apache.org/jira/browse/HADOOP-1312) | heartbeat monitor thread goes away | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1310](https://issues.apache.org/jira/browse/HADOOP-1310) | Fix unchecked warnings in aggregate code | Major | . | Tom White | Tom White | +| [HADOOP-1299](https://issues.apache.org/jira/browse/HADOOP-1299) | Once RPC.stopClient has been called, RPC can not be used again | Minor | ipc | stack | stack | +| [HADOOP-1297](https://issues.apache.org/jira/browse/HADOOP-1297) | datanode sending block reports to namenode once every second | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1294](https://issues.apache.org/jira/browse/HADOOP-1294) | Fix unchecked warnings in main Hadoop code under Java 6. | Major | test | Tom White | Tom White | +| [HADOOP-1293](https://issues.apache.org/jira/browse/HADOOP-1293) | stderr from streaming skipped after first 20 lines. | Minor | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-1279](https://issues.apache.org/jira/browse/HADOOP-1279) | list of completed jobs purges jobs based on submission not on completion age | Major | . | Alejandro Abdelnur | Arun C Murthy | +| [HADOOP-1278](https://issues.apache.org/jira/browse/HADOOP-1278) | Fix the per-job tasktracker 'blacklist' | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1275](https://issues.apache.org/jira/browse/HADOOP-1275) | job notification property in hadoop-default.xml is misspelled | Trivial | . | Alejandro Abdelnur | | +| [HADOOP-1272](https://issues.apache.org/jira/browse/HADOOP-1272) | Extract InnerClasses from FSNamesystem into separate classes | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1271](https://issues.apache.org/jira/browse/HADOOP-1271) | The StreamBaseRecordReader is unable to log record data that's not UTF-8 | Minor | . | Gautam Kowshik | Arun C Murthy | +| [HADOOP-1262](https://issues.apache.org/jira/browse/HADOOP-1262) | file corruption detected because dfs client does not use replica blocks for checksum file | Major | . | dhruba borthakur | Hairong Kuang | +| [HADOOP-1258](https://issues.apache.org/jira/browse/HADOOP-1258) | TestCheckpoint test case doesn't wait for MiniDFSCluster to be active | Trivial | test | Nigel Daley | Nigel Daley | +| [HADOOP-1256](https://issues.apache.org/jira/browse/HADOOP-1256) | Dfs image loading and edits loading creates multiple instances of DatanodeDescriptor for the same datanode | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1255](https://issues.apache.org/jira/browse/HADOOP-1255) | Name-node falls into infinite loop trying to remove a dead node. | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-1253](https://issues.apache.org/jira/browse/HADOOP-1253) | ConcurrentModificationException and NPE in JobControl | Minor | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-1252](https://issues.apache.org/jira/browse/HADOOP-1252) | Disk problems should be handled better by the MR framework | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1244](https://issues.apache.org/jira/browse/HADOOP-1244) | stop-dfs.sh incorrectly specifies slaves file for stopping datanode | Minor | . | Michael Bieniosek | dhruba borthakur | +| [HADOOP-1243](https://issues.apache.org/jira/browse/HADOOP-1243) | ClientProtocol.versionID should be 11 | Major | . | Konstantin Shvachko | dhruba borthakur | +| [HADOOP-1242](https://issues.apache.org/jira/browse/HADOOP-1242) | dfs upgrade/downgrade problems | Blocker | . | Owen O'Malley | Konstantin Shvachko | +| [HADOOP-1241](https://issues.apache.org/jira/browse/HADOOP-1241) | Null PointerException in processReport when namenode is restarted | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1239](https://issues.apache.org/jira/browse/HADOOP-1239) | Classes in src/test/testjar need package name | Trivial | test | Jim Kellerman | Jim Kellerman | +| [HADOOP-1238](https://issues.apache.org/jira/browse/HADOOP-1238) | maps\_running metric is only updated at the end of the task | Minor | metrics | Michael Bieniosek | David Bowen | +| [HADOOP-1224](https://issues.apache.org/jira/browse/HADOOP-1224) | "Browse the filesystem" link pointing to a dead data-node | Major | . | Konstantin Shvachko | Enis Soztutar | +| [HADOOP-1219](https://issues.apache.org/jira/browse/HADOOP-1219) | Spurious progress messages should be discarded after a task is done | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1218](https://issues.apache.org/jira/browse/HADOOP-1218) | In TaskTracker the access to RunningJob object is not synchronized in one place | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1211](https://issues.apache.org/jira/browse/HADOOP-1211) | Remove deprecated constructor and unused static members in DataNode class | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1205](https://issues.apache.org/jira/browse/HADOOP-1205) | The open method of FSNamesystem should be synchronized | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1204](https://issues.apache.org/jira/browse/HADOOP-1204) | Re-factor InputFormat/RecordReader related classes | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1203](https://issues.apache.org/jira/browse/HADOOP-1203) | UpgradeUtilities should use MiniDFSCluster to start and stop NameNode/DataNodes | Major | test | Nigel Daley | Nigel Daley | +| [HADOOP-1200](https://issues.apache.org/jira/browse/HADOOP-1200) | Datanode should periodically do a disk check | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1198](https://issues.apache.org/jira/browse/HADOOP-1198) | ipc.client.timeout of 2000ms for test cases seems too small; causes too many timeouts and leads to hung test cases | Major | test | Arun C Murthy | Arun C Murthy | +| [HADOOP-1189](https://issues.apache.org/jira/browse/HADOOP-1189) | Still seeing some unexpected 'No space left on device' exceptions | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1187](https://issues.apache.org/jira/browse/HADOOP-1187) | DFS Scalability: avoid scanning entire list of datanodes in getAdditionalBlocks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1184](https://issues.apache.org/jira/browse/HADOOP-1184) | Decommission fails if a block that needs replication has only one replica | Major | . | dhruba borthakur | | +| [HADOOP-1178](https://issues.apache.org/jira/browse/HADOOP-1178) | NullPointer Exception in org.apache.hadoop.dfs.NameNode.isDir on namenode restart | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1176](https://issues.apache.org/jira/browse/HADOOP-1176) | Reduce hang on huge map output | Major | . | Hairong Kuang | Arun C Murthy | +| [HADOOP-1170](https://issues.apache.org/jira/browse/HADOOP-1170) | Very high CPU usage on data nodes because of FSDataset.checkDataDir() on every connect | Major | . | Igor Bolotin | | +| [HADOOP-1169](https://issues.apache.org/jira/browse/HADOOP-1169) | CopyFiles skips src files of s3 urls | Minor | util | stack | | +| [HADOOP-1164](https://issues.apache.org/jira/browse/HADOOP-1164) | TestReplicationPolicy doesn't use port 0 for the NameNode | Major | test | Owen O'Malley | Owen O'Malley | +| [HADOOP-1163](https://issues.apache.org/jira/browse/HADOOP-1163) | Ganglia metrics reporting is misconfigured | Minor | metrics | Michael Bieniosek | | +| [HADOOP-1160](https://issues.apache.org/jira/browse/HADOOP-1160) | DistributedFileSystem doesn't close the RawDistributedFileSystem on close. | Blocker | . | Owen O'Malley | Hairong Kuang | +| [HADOOP-1156](https://issues.apache.org/jira/browse/HADOOP-1156) | NullPointerException in MiniDFSCluster | Major | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-1154](https://issues.apache.org/jira/browse/HADOOP-1154) | streaming hang. (PipeMapRed$MROutputThread gone) | Major | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-1153](https://issues.apache.org/jira/browse/HADOOP-1153) | DataNode and FSNamesystem don't shutdown cleanly | Major | . | Nigel Daley | Konstantin Shvachko | +| [HADOOP-1152](https://issues.apache.org/jira/browse/HADOOP-1152) | Reduce task hang failing in MapOutputCopier.copyOutput | Major | . | Koji Noguchi | Tahir Hashmi | +| [HADOOP-1151](https://issues.apache.org/jira/browse/HADOOP-1151) | streaming PipeMapRed prints system info to stderr | Trivial | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-1149](https://issues.apache.org/jira/browse/HADOOP-1149) | DFS Scalability: high cpu usage in addStoredBlock | Major | . | dhruba borthakur | Raghu Angadi | +| [HADOOP-1146](https://issues.apache.org/jira/browse/HADOOP-1146) | "Reduce input records" counter name is misleading | Major | . | David Bowen | David Bowen | +| [HADOOP-1137](https://issues.apache.org/jira/browse/HADOOP-1137) | StatusHttpServer assumes that resources for /static are in files | Major | . | Benjamin Reed | | +| [HADOOP-1136](https://issues.apache.org/jira/browse/HADOOP-1136) | exception in UnderReplicatedBlocks:add when ther are more replicas of a block than required | Major | . | dhruba borthakur | Hairong Kuang | +| [HADOOP-1122](https://issues.apache.org/jira/browse/HADOOP-1122) | Divide-by-zero exception in chooseTarget | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1114](https://issues.apache.org/jira/browse/HADOOP-1114) | bin/hadoop script clobbers CLASSPATH | Minor | scripts | Michael Bieniosek | Doug Cutting | +| [HADOOP-1110](https://issues.apache.org/jira/browse/HADOOP-1110) | JobTracker WebUI "Map input records" a little off. | Trivial | . | Koji Noguchi | David Bowen | +| [HADOOP-1093](https://issues.apache.org/jira/browse/HADOOP-1093) | NNBench generates millions of NotReplicatedYetException in Namenode log | Major | . | Nigel Daley | dhruba borthakur | +| [HADOOP-1090](https://issues.apache.org/jira/browse/HADOOP-1090) | In SortValidator, the check for whether a file belongs to sort-input or sort-output dir is weak | Major | . | Devaraj Das | Arun C Murthy | +| [HADOOP-1085](https://issues.apache.org/jira/browse/HADOOP-1085) | Remove 'port rolling' from Mini{DFS\|MR}Cluster | Major | test | Arun C Murthy | Arun C Murthy | +| [HADOOP-1081](https://issues.apache.org/jira/browse/HADOOP-1081) | JAVA\_PLATFORM with spaces (i.e. Mac OS X-ppc-32) breaks bin/hadoop script | Major | scripts | Andrzej Bialecki | | +| [HADOOP-1073](https://issues.apache.org/jira/browse/HADOOP-1073) | DFS Scalability: high CPU usage in choosing replication targets and file open | Major | . | dhruba borthakur | Hairong Kuang | +| [HADOOP-1071](https://issues.apache.org/jira/browse/HADOOP-1071) | RPC$VersionMismatch exception is not fatal to JobTracker | Major | . | Nigel Daley | Tahir Hashmi | +| [HADOOP-1064](https://issues.apache.org/jira/browse/HADOOP-1064) | dfsclient logging messages should have appropriate log levels | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1063](https://issues.apache.org/jira/browse/HADOOP-1063) | MiniDFSCluster exists a race condition that lead to data node resources are not properly released | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-1061](https://issues.apache.org/jira/browse/HADOOP-1061) | S3 listSubPaths bug | Critical | fs | Mike Smith | | +| [HADOOP-1050](https://issues.apache.org/jira/browse/HADOOP-1050) | Do not count lost tasktracker against the job | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1047](https://issues.apache.org/jira/browse/HADOOP-1047) | TestReplication fails because DFS does not guarantee all the replicas are placed when a file is closed | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1011](https://issues.apache.org/jira/browse/HADOOP-1011) | ConcurrentModificationException in JobHistory | Major | . | Nigel Daley | Tahir Hashmi | +| [HADOOP-1001](https://issues.apache.org/jira/browse/HADOOP-1001) | the output of the map is not type checked against the specified types | Major | . | Owen O'Malley | Tahir Hashmi | +| [HADOOP-672](https://issues.apache.org/jira/browse/HADOOP-672) | dfs shell enhancements | Minor | . | Yoram Arnon | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/RELEASENOTES.0.13.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/RELEASENOTES.0.13.0.md new file mode 100644 index 00000000000..41c18bd6a2b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.0/RELEASENOTES.0.13.0.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.13.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-1063](https://issues.apache.org/jira/browse/HADOOP-1063) | *Major* | **MiniDFSCluster exists a race condition that lead to data node resources are not properly released** + +Resolved race condition in shutting down MiniDFSCluster data node that prevented resources from being deallocated properly. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.1/CHANGES.0.13.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.1/CHANGES.0.13.1.md new file mode 100644 index 00000000000..18929409133 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.1/CHANGES.0.13.1.md @@ -0,0 +1,64 @@ + + +# Apache Hadoop Changelog + +## Release 0.13.1 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.1/RELEASENOTES.0.13.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.1/RELEASENOTES.0.13.1.md new file mode 100644 index 00000000000..5b95ea132c1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.13.1/RELEASENOTES.0.13.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.13.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/CHANGES.0.14.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/CHANGES.0.14.0.md new file mode 100644 index 00000000000..541f4c90c82 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/CHANGES.0.14.0.md @@ -0,0 +1,224 @@ + + +# Apache Hadoop Changelog + +## Release 0.14.0 - 2007-08-20 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1597](https://issues.apache.org/jira/browse/HADOOP-1597) | Distributed upgrade status reporting and post upgrade features. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1570](https://issues.apache.org/jira/browse/HADOOP-1570) | Add a per-job configuration knob to control loading of native hadoop libraries | Major | io | Arun C Murthy | Arun C Murthy | +| [HADOOP-1568](https://issues.apache.org/jira/browse/HADOOP-1568) | NameNode Schema for HttpFileSystem | Major | fs | Chris Douglas | Chris Douglas | +| [HADOOP-1562](https://issues.apache.org/jira/browse/HADOOP-1562) | Report Java VM metrics | Major | metrics | David Bowen | David Bowen | +| [HADOOP-1515](https://issues.apache.org/jira/browse/HADOOP-1515) | MultiFileSplit, MultiFileInputFormat | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1508](https://issues.apache.org/jira/browse/HADOOP-1508) | ant Task for FsShell operations | Minor | build, fs | Chris Douglas | Chris Douglas | +| [HADOOP-1469](https://issues.apache.org/jira/browse/HADOOP-1469) | Asynchronous table creation | Minor | . | James Kennedy | stack | +| [HADOOP-1447](https://issues.apache.org/jira/browse/HADOOP-1447) | Support for textInputFormat in contrib/data\_join | Minor | . | Senthil Subramanian | Senthil Subramanian | +| [HADOOP-1437](https://issues.apache.org/jira/browse/HADOOP-1437) | Eclipse plugin for developing and executing MapReduce programs on Hadoop | Major | . | Eugene Hung | Christophe Taton | +| [HADOOP-1433](https://issues.apache.org/jira/browse/HADOOP-1433) | Add job priority | Minor | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-1379](https://issues.apache.org/jira/browse/HADOOP-1379) | Integrate Findbugs into nightly build process | Major | test | Nigel Daley | Nigel Daley | +| [HADOOP-1377](https://issues.apache.org/jira/browse/HADOOP-1377) | Creation time and modification time for hadoop files and directories | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1134](https://issues.apache.org/jira/browse/HADOOP-1134) | Block level CRCs in HDFS | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-234](https://issues.apache.org/jira/browse/HADOOP-234) | Hadoop Pipes for writing map/reduce jobs in C++ and python | Major | . | Sanjay Dahiya | Owen O'Malley | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1653](https://issues.apache.org/jira/browse/HADOOP-1653) | FSDirectory class code cleanup | Trivial | . | Christophe Taton | Christophe Taton | +| [HADOOP-1620](https://issues.apache.org/jira/browse/HADOOP-1620) | FileSystem should have fewer abstract methods | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-1585](https://issues.apache.org/jira/browse/HADOOP-1585) | GenericWritable should use generics | Minor | io | Espen Amble Kolstad | Espen Amble Kolstad | +| [HADOOP-1582](https://issues.apache.org/jira/browse/HADOOP-1582) | hdfsRead and hdfsPread should return 0 instead of -1 at end-of-file. | Minor | . | Christian Kunz | Christian Kunz | +| [HADOOP-1580](https://issues.apache.org/jira/browse/HADOOP-1580) | provide better error message when subprocesses fail in hadoop streaming | Minor | . | John Heidemann | John Heidemann | +| [HADOOP-1547](https://issues.apache.org/jira/browse/HADOOP-1547) | Provide examples for aggregate library | Major | . | Tom White | Runping Qi | +| [HADOOP-1518](https://issues.apache.org/jira/browse/HADOOP-1518) | Add session id to metric data | Major | . | David Bowen | David Bowen | +| [HADOOP-1485](https://issues.apache.org/jira/browse/HADOOP-1485) | Metrics should be there for reporting shuffle failures/successes | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1484](https://issues.apache.org/jira/browse/HADOOP-1484) | Kill jobs from web interface | Minor | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1478](https://issues.apache.org/jira/browse/HADOOP-1478) | The blockStream of DFSClient.FSInputStream should not be buffered | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1473](https://issues.apache.org/jira/browse/HADOOP-1473) | Make jobids unique across jobtracker restarts | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1470](https://issues.apache.org/jira/browse/HADOOP-1470) | Rework FSInputChecker and FSOutputSummer to support checksum code sharing between ChecksumFileSystem and block level crc dfs | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-1467](https://issues.apache.org/jira/browse/HADOOP-1467) | Remove redundant counters from WordCount example | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1462](https://issues.apache.org/jira/browse/HADOOP-1462) | Better progress reporting from a Task | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-1457](https://issues.apache.org/jira/browse/HADOOP-1457) | Counters for monitoring task assignments | Minor | . | Devaraj Das | Arun C Murthy | +| [HADOOP-1455](https://issues.apache.org/jira/browse/HADOOP-1455) | Allow any key-value pair on the command line of 'hadoop pipes' to be added to the JobConf | Major | . | Christian Kunz | Devaraj Das | +| [HADOOP-1450](https://issues.apache.org/jira/browse/HADOOP-1450) | checksums should be closer to data generation and consumption | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-1440](https://issues.apache.org/jira/browse/HADOOP-1440) | JobClient should not sort input-splits | Major | . | Milind Bhandarkar | Senthil Subramanian | +| [HADOOP-1438](https://issues.apache.org/jira/browse/HADOOP-1438) | Grammatical / wording / copy edits for Hadoop Distributed File System: Architecture and Design white paper | Trivial | documentation | Luke Nezda | | +| [HADOOP-1429](https://issues.apache.org/jira/browse/HADOOP-1429) | RPC Server won't go quietly | Minor | ipc | stack | stack | +| [HADOOP-1417](https://issues.apache.org/jira/browse/HADOOP-1417) | Exclude some Findbugs detectors | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-1413](https://issues.apache.org/jira/browse/HADOOP-1413) | A new example to do tile placements using distributed dancing links | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1408](https://issues.apache.org/jira/browse/HADOOP-1408) | fix warning about cast of Map\\> in jobhistory.jsp | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1376](https://issues.apache.org/jira/browse/HADOOP-1376) | RandomWriter should be tweaked to generate input data for terasort | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1343](https://issues.apache.org/jira/browse/HADOOP-1343) | Deprecate the Configuration.set(String,Object) method and make Configuration Iterable | Major | conf | Owen O'Malley | Owen O'Malley | +| [HADOOP-1342](https://issues.apache.org/jira/browse/HADOOP-1342) | A configurable limit on the number of unique values should be set on the UniqueValueCount and ValueHistogram aggregators | Major | . | Runping Qi | Runping Qi | +| [HADOOP-1340](https://issues.apache.org/jira/browse/HADOOP-1340) | md5 file in filecache should inherit replication factor from the file it belongs to. | Major | . | Christian Kunz | dhruba borthakur | +| [HADOOP-1320](https://issues.apache.org/jira/browse/HADOOP-1320) | Rewrite 'random-writer' to use '-reducer NONE' | Minor | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1292](https://issues.apache.org/jira/browse/HADOOP-1292) | dfs -copyToLocal should guarantee file is complete | Major | . | eric baldeschwieler | Tsz Wo Nicholas Sze | +| [HADOOP-1286](https://issues.apache.org/jira/browse/HADOOP-1286) | Distributed cluster upgrade | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1283](https://issues.apache.org/jira/browse/HADOOP-1283) | Eliminate internal UTF8 to String and vice versa conversions in the name-node. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1147](https://issues.apache.org/jira/browse/HADOOP-1147) | remove all @author tags from source | Minor | . | Doug Cutting | Doug Cutting | +| [HADOOP-1066](https://issues.apache.org/jira/browse/HADOOP-1066) | http://lucene.apache.org/hadoop/ front page is not user-friendly | Minor | documentation | Marco Nicosia | Doug Cutting | +| [HADOOP-1028](https://issues.apache.org/jira/browse/HADOOP-1028) | Servers should log startup and shutdown messages | Major | . | Nigel Daley | Tsz Wo Nicholas Sze | +| [HADOOP-1023](https://issues.apache.org/jira/browse/HADOOP-1023) | better links to mailing list archives | Major | documentation | Daniel Naber | Tom White | +| [HADOOP-1003](https://issues.apache.org/jira/browse/HADOOP-1003) | Proposal to batch commits to edits log. | Major | . | Raghu Angadi | dhruba borthakur | +| [HADOOP-894](https://issues.apache.org/jira/browse/HADOOP-894) | dfs client protocol should allow asking for parts of the block map | Major | . | Owen O'Malley | Konstantin Shvachko | +| [HADOOP-800](https://issues.apache.org/jira/browse/HADOOP-800) | More improvements to DFS browsing WI | Major | . | arkady borkovsky | Enis Soztutar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1717](https://issues.apache.org/jira/browse/HADOOP-1717) | TestDFSUpgradeFromImage fails on Solaris | Blocker | test | Nigel Daley | Raghu Angadi | +| [HADOOP-1716](https://issues.apache.org/jira/browse/HADOOP-1716) | TestPipes.testPipes fails | Blocker | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1714](https://issues.apache.org/jira/browse/HADOOP-1714) | TestDFSUpgradeFromImage fails on Windows | Blocker | test | Nigel Daley | Raghu Angadi | +| [HADOOP-1712](https://issues.apache.org/jira/browse/HADOOP-1712) | Unhandled exception in Block CRC upgrade on datanode. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1698](https://issues.apache.org/jira/browse/HADOOP-1698) | 7500+ reducers/partitions causes job to hang | Blocker | . | Srikanth Kakani | Devaraj Das | +| [HADOOP-1681](https://issues.apache.org/jira/browse/HADOOP-1681) | Re organize StreamJob::submitAndMonitorJob() Exception handling | Minor | . | Lohit Vijayarenu | | +| [HADOOP-1680](https://issues.apache.org/jira/browse/HADOOP-1680) | Improvements to Block CRC upgrade messages | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1668](https://issues.apache.org/jira/browse/HADOOP-1668) | add INCOMPATIBLE CHANGES section to CHANGES.txt for Hadoop 0.14 | Blocker | documentation | Nigel Daley | Nigel Daley | +| [HADOOP-1666](https://issues.apache.org/jira/browse/HADOOP-1666) | The FsShell Object cannot be used for multiple fs commands. | Minor | fs | dhruba borthakur | dhruba borthakur | +| [HADOOP-1665](https://issues.apache.org/jira/browse/HADOOP-1665) | DFS Trash feature bugs | Blocker | . | Nigel Daley | dhruba borthakur | +| [HADOOP-1663](https://issues.apache.org/jira/browse/HADOOP-1663) | streaming returning 0 when submitJob fails with Exception | Major | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-1659](https://issues.apache.org/jira/browse/HADOOP-1659) | job id / job name mix-up | Blocker | . | Christian Kunz | Arun C Murthy | +| [HADOOP-1657](https://issues.apache.org/jira/browse/HADOOP-1657) | NNBench benchmark hangs with trunk | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1649](https://issues.apache.org/jira/browse/HADOOP-1649) | Performance regression with Block CRCs | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1647](https://issues.apache.org/jira/browse/HADOOP-1647) | DistributedFileSystem.getFileStatus() fails for path "/" | Blocker | . | Enis Soztutar | dhruba borthakur | +| [HADOOP-1640](https://issues.apache.org/jira/browse/HADOOP-1640) | TestDecommission fails on Windows | Blocker | . | Nigel Daley | dhruba borthakur | +| [HADOOP-1639](https://issues.apache.org/jira/browse/HADOOP-1639) | TestSymLink is failing fairly often and is blocking the regression | Major | . | Owen O'Malley | Mahadev konar | +| [HADOOP-1638](https://issues.apache.org/jira/browse/HADOOP-1638) | Master node unable to bind to DNS hostname | Minor | contrib/cloud | Stu Hood | | +| [HADOOP-1635](https://issues.apache.org/jira/browse/HADOOP-1635) | Keypair Name Hardcoded | Minor | contrib/cloud | Stu Hood | | +| [HADOOP-1634](https://issues.apache.org/jira/browse/HADOOP-1634) | EC2 launch-hadoop-cluster awk Problem | Minor | contrib/cloud | Stu Hood | | +| [HADOOP-1632](https://issues.apache.org/jira/browse/HADOOP-1632) | IllegalArgumentException in fsck | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-1625](https://issues.apache.org/jira/browse/HADOOP-1625) | "could not move files" exception in DataXceiver | Blocker | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-1624](https://issues.apache.org/jira/browse/HADOOP-1624) | Unknown op code exception in DataXceiver. | Blocker | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-1623](https://issues.apache.org/jira/browse/HADOOP-1623) | dfs -cp infinite loop creating sub-directories | Blocker | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-1619](https://issues.apache.org/jira/browse/HADOOP-1619) | FSInputChecker attempts to seek past EOF | Blocker | fs | Nigel Daley | Hairong Kuang | +| [HADOOP-1613](https://issues.apache.org/jira/browse/HADOOP-1613) | The dfs webui (dfshealth) shows "Last Contact" as a negative number | Minor | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1603](https://issues.apache.org/jira/browse/HADOOP-1603) | Replication gets set to 1 sometimes when Namenode restarted. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1599](https://issues.apache.org/jira/browse/HADOOP-1599) | TestCopyFiles with IllegalArgumentException on Windows | Blocker | fs | Nigel Daley | Senthil Subramanian | +| [HADOOP-1596](https://issues.apache.org/jira/browse/HADOOP-1596) | TestSymLink is failing | Blocker | . | Doug Cutting | Owen O'Malley | +| [HADOOP-1590](https://issues.apache.org/jira/browse/HADOOP-1590) | Jobtracker web interface contains several absolute href links instead of relative ones | Major | . | Thomas Friol | | +| [HADOOP-1587](https://issues.apache.org/jira/browse/HADOOP-1587) | Tasks run by MiniMRCluster don't get sysprops from TestCases | Blocker | test | Alejandro Abdelnur | Devaraj Das | +| [HADOOP-1584](https://issues.apache.org/jira/browse/HADOOP-1584) | Bug in readFields of GenericWritable | Minor | io | Espen Amble Kolstad | Espen Amble Kolstad | +| [HADOOP-1578](https://issues.apache.org/jira/browse/HADOOP-1578) | Data-nodes should send storage ID to the name-node during registration | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1576](https://issues.apache.org/jira/browse/HADOOP-1576) | web interface inconsistencies when using speculative execution | Blocker | . | Christian Kunz | Arun C Murthy | +| [HADOOP-1564](https://issues.apache.org/jira/browse/HADOOP-1564) | Write unit tests to detect CRC corruption | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1556](https://issues.apache.org/jira/browse/HADOOP-1556) | 9 unit test failures: file.out.index already exists | Major | . | Nigel Daley | Devaraj Das | +| [HADOOP-1554](https://issues.apache.org/jira/browse/HADOOP-1554) | Fix the JobHistory to display things like the number of nodes the job ran on, the number of killed/failed tasks | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1553](https://issues.apache.org/jira/browse/HADOOP-1553) | Extensive logging of C++ application can slow down task by an order of magnitude | Blocker | . | Christian Kunz | Owen O'Malley | +| [HADOOP-1551](https://issues.apache.org/jira/browse/HADOOP-1551) | libhdfs API is out of sync with Filesystem API | Blocker | . | Christian Kunz | Sameer Paranjpye | +| [HADOOP-1546](https://issues.apache.org/jira/browse/HADOOP-1546) | The DFS WebUI shows an incorrect column for file Creatin Time | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1542](https://issues.apache.org/jira/browse/HADOOP-1542) | Incorrect task/tip being scheduled (looks like speculative execution) | Blocker | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1536](https://issues.apache.org/jira/browse/HADOOP-1536) | libhdfs tests failing | Blocker | . | Nigel Daley | dhruba borthakur | +| [HADOOP-1535](https://issues.apache.org/jira/browse/HADOOP-1535) | Wrong comparator used to merge files in Reduce phase | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-1524](https://issues.apache.org/jira/browse/HADOOP-1524) | Task Logs userlogs don't show up for a while | Major | . | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-1520](https://issues.apache.org/jira/browse/HADOOP-1520) | IndexOutOfBoundsException in FSEditLog.processIOError | Blocker | . | Nigel Daley | dhruba borthakur | +| [HADOOP-1517](https://issues.apache.org/jira/browse/HADOOP-1517) | Three methods in FSNamesystem should not be synchronized. | Critical | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1514](https://issues.apache.org/jira/browse/HADOOP-1514) | Progress reporting not handled for the case where a reducer currently doesn't have anything to fetch | Blocker | . | Devaraj Das | Vivek Ratan | +| [HADOOP-1513](https://issues.apache.org/jira/browse/HADOOP-1513) | A likely race condition between the creation of a directory and checking for its existence in the DiskChecker class | Critical | fs | Devaraj Das | Devaraj Das | +| [HADOOP-1512](https://issues.apache.org/jira/browse/HADOOP-1512) | TestTextInputFormat fails on Windows | Major | . | Konstantin Shvachko | | +| [HADOOP-1504](https://issues.apache.org/jira/browse/HADOOP-1504) | terminate-hadoop-cluster may be overzealous | Blocker | fs/s3 | Doug Cutting | Tom White | +| [HADOOP-1503](https://issues.apache.org/jira/browse/HADOOP-1503) | Fix for broken build by HADOOP-1498 | Major | . | stack | stack | +| [HADOOP-1501](https://issues.apache.org/jira/browse/HADOOP-1501) | Block reports from all datanodes arrive at the namenode within a small band of time | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1493](https://issues.apache.org/jira/browse/HADOOP-1493) | possible double setting of java.library.path introduced by HADOOP-838 | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1492](https://issues.apache.org/jira/browse/HADOOP-1492) | DataNode version mismatch during handshake() causes NullPointerException. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1489](https://issues.apache.org/jira/browse/HADOOP-1489) | Input file get truncated for text files with \r\n | Major | io | Bwolen Yang | | +| [HADOOP-1486](https://issues.apache.org/jira/browse/HADOOP-1486) | ReplicationMonitor thread goes away | Blocker | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-1482](https://issues.apache.org/jira/browse/HADOOP-1482) | SecondaryNameNode does not roll ports | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1479](https://issues.apache.org/jira/browse/HADOOP-1479) | NPE in HStore#get if StoreFile only has keys \< than passed key | Minor | . | stack | stack | +| [HADOOP-1475](https://issues.apache.org/jira/browse/HADOOP-1475) | local filecache disappears | Blocker | . | Christian Kunz | Owen O'Malley | +| [HADOOP-1472](https://issues.apache.org/jira/browse/HADOOP-1472) | Timed-out tasks are marked as 'KILLED' rather than as 'FAILED' which means the framework doesn't fail a TIP with 4 or more timed-out attempts | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1461](https://issues.apache.org/jira/browse/HADOOP-1461) | Corner-case deadlock in TaskTracker | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1459](https://issues.apache.org/jira/browse/HADOOP-1459) | FileSystem.getFileCacheHints returns IP addresses rather than hostnames, which breaks 'data-locality' in map-reduce | Blocker | . | Arun C Murthy | dhruba borthakur | +| [HADOOP-1456](https://issues.apache.org/jira/browse/HADOOP-1456) | TestDecommission fails with assertion Number of replicas for block1 expected:\<3\> but was:\<2\> | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1453](https://issues.apache.org/jira/browse/HADOOP-1453) | exists() not necessary before DFS.open | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1448](https://issues.apache.org/jira/browse/HADOOP-1448) | Setting the replication factor of a file too high causes namenode cpu overload | Major | . | dhruba borthakur | Hairong Kuang | +| [HADOOP-1446](https://issues.apache.org/jira/browse/HADOOP-1446) | Metrics from the TaskTracker are updated only when map/reduce tasks start/end/fail | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1444](https://issues.apache.org/jira/browse/HADOOP-1444) | Block allocation method does not check pendingCreates for duplicate block ids | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1443](https://issues.apache.org/jira/browse/HADOOP-1443) | TestFileCorruption fails with ArrayIndexOutOfBoundsException | Critical | . | Nigel Daley | Konstantin Shvachko | +| [HADOOP-1442](https://issues.apache.org/jira/browse/HADOOP-1442) | Zero-byte input files are not included in InputSplit | Major | . | Milind Bhandarkar | Senthil Subramanian | +| [HADOOP-1428](https://issues.apache.org/jira/browse/HADOOP-1428) | ChecksumFileSystem : some operations implicitly not supported. | Major | fs | Raghu Angadi | | +| [HADOOP-1414](https://issues.apache.org/jira/browse/HADOOP-1414) | Findbugs - Bad Practice | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1412](https://issues.apache.org/jira/browse/HADOOP-1412) | FindBugs: Dodgy bugs in fs, filecache, io, and util packages | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1406](https://issues.apache.org/jira/browse/HADOOP-1406) | Metrics based on Map-Reduce Counters are not cleaned up | Major | . | David Bowen | David Bowen | +| [HADOOP-1400](https://issues.apache.org/jira/browse/HADOOP-1400) | JobClient rpc times out getting job status | Blocker | . | Nigel Daley | Owen O'Malley | +| [HADOOP-1396](https://issues.apache.org/jira/browse/HADOOP-1396) | FileNotFound exception on DFS block | Blocker | . | Devaraj Das | dhruba borthakur | +| [HADOOP-1394](https://issues.apache.org/jira/browse/HADOOP-1394) | FindBugs : Performance : in dfs | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1393](https://issues.apache.org/jira/browse/HADOOP-1393) | using Math.abs(Random.getInt()) does not guarantee a positive number | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1392](https://issues.apache.org/jira/browse/HADOOP-1392) | FindBugs : Fix some correctness bugs reported in DFS, FS, etc. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1390](https://issues.apache.org/jira/browse/HADOOP-1390) | Inconsistent Synchronization cleanup for {Configuration, TaskLog, MapTask, Server}.java | Minor | conf, ipc | Devaraj Das | Devaraj Das | +| [HADOOP-1387](https://issues.apache.org/jira/browse/HADOOP-1387) | FindBugs -\> Performance | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1372](https://issues.apache.org/jira/browse/HADOOP-1372) | DFS Clients should start using the org.apache.hadoop.fs.LocalDirAllocator | Major | . | Devaraj Das | dhruba borthakur | +| [HADOOP-1367](https://issues.apache.org/jira/browse/HADOOP-1367) | Inconsistent synchronization of NetworkTopology.distFrom; locked 50% of time | Major | io | Nigel Daley | Hairong Kuang | +| [HADOOP-1364](https://issues.apache.org/jira/browse/HADOOP-1364) | Inconsistent synchronization of SequenceFile$Reader.noBufferedValues; locked 66% of time | Minor | io | Nigel Daley | Owen O'Malley | +| [HADOOP-1359](https://issues.apache.org/jira/browse/HADOOP-1359) | Variable dereferenced then later checked for null | Minor | . | Nigel Daley | Hairong Kuang | +| [HADOOP-1357](https://issues.apache.org/jira/browse/HADOOP-1357) | Call to equals() comparing different types in CopyFiles.cleanup(Configuration, JobConf, String, String) | Minor | util | Nigel Daley | Arun C Murthy | +| [HADOOP-1355](https://issues.apache.org/jira/browse/HADOOP-1355) | Possible null pointer dereference in TaskLogAppender.append(LoggingEvent) | Minor | . | Nigel Daley | Arun C Murthy | +| [HADOOP-1344](https://issues.apache.org/jira/browse/HADOOP-1344) | getJobName not accessible from JobClient | Minor | . | Michael Bieniosek | | +| [HADOOP-1335](https://issues.apache.org/jira/browse/HADOOP-1335) | C++ reducers under hadoop-pipes are not started when there are no key-value pairs to be reduced | Major | . | Christian Kunz | Owen O'Malley | +| [HADOOP-1311](https://issues.apache.org/jira/browse/HADOOP-1311) | Bug in BytesWritable.set(byte[] newData, int offset, int length) | Major | io | Srikanth Kakani | dhruba borthakur | +| [HADOOP-1300](https://issues.apache.org/jira/browse/HADOOP-1300) | deletion of excess replicas does not take into account 'rack-locality' | Major | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-1285](https://issues.apache.org/jira/browse/HADOOP-1285) | ChecksumFileSystem : Can't read when io.file.buffer.size \< bytePerChecksum | Major | fs | Raghu Angadi | | +| [HADOOP-1269](https://issues.apache.org/jira/browse/HADOOP-1269) | DFS Scalability: namenode throughput impacted becuase of global FSNamesystem lock | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1261](https://issues.apache.org/jira/browse/HADOOP-1261) | Restart of the same data-node should not generate edits log records. | Minor | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-1234](https://issues.apache.org/jira/browse/HADOOP-1234) | map tasks fail because they do not find application in file cache | Major | . | Christian Kunz | Arun C Murthy | +| [HADOOP-1226](https://issues.apache.org/jira/browse/HADOOP-1226) | makeQualified should return an instance of a DfsPath when passed a DfsPath | Major | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-1207](https://issues.apache.org/jira/browse/HADOOP-1207) | hdfs -rm should NOT fail if one of the files to remove is missing | Minor | . | arkady borkovsky | Tsz Wo Nicholas Sze | +| [HADOOP-1197](https://issues.apache.org/jira/browse/HADOOP-1197) | The misleading Configuration.set(String, Object) should be removed | Minor | conf | Owen O'Malley | Owen O'Malley | +| [HADOOP-1193](https://issues.apache.org/jira/browse/HADOOP-1193) | Map/reduce job gets OutOfMemoryException when set map out to be compressed | Blocker | . | Hairong Kuang | Arun C Murthy | +| [HADOOP-1139](https://issues.apache.org/jira/browse/HADOOP-1139) | All block trasitions should be logged at log level INFO | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1124](https://issues.apache.org/jira/browse/HADOOP-1124) | ChecksumFileSystem does not handle ChecksumError correctly | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-1084](https://issues.apache.org/jira/browse/HADOOP-1084) | updating a hdfs file, doesn't cause the distributed file cache to update itself | Blocker | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-911](https://issues.apache.org/jira/browse/HADOOP-911) | Multithreading issue with libhdfs library | Blocker | . | Christian Kunz | Christian Kunz | +| [HADOOP-690](https://issues.apache.org/jira/browse/HADOOP-690) | NPE in jobcontrol | Major | . | Johan Oskarsson | Owen O'Malley | +| [HADOOP-71](https://issues.apache.org/jira/browse/HADOOP-71) | The SequenceFileRecordReader uses the default FileSystem rather than the supplied one | Major | . | Owen O'Malley | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1629](https://issues.apache.org/jira/browse/HADOOP-1629) | Block CRC Unit Tests: upgrade test | Blocker | . | Nigel Daley | Raghu Angadi | +| [HADOOP-1628](https://issues.apache.org/jira/browse/HADOOP-1628) | Block CRC Unit Tests: protocol tests | Blocker | . | Nigel Daley | Raghu Angadi | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1449](https://issues.apache.org/jira/browse/HADOOP-1449) | Example for contrib/data\_join | Minor | . | Senthil Subramanian | Senthil Subramanian | +| [HADOOP-1336](https://issues.apache.org/jira/browse/HADOOP-1336) | turn on speculative execution by defaul | Major | . | Owen O'Malley | Owen O'Malley | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/RELEASENOTES.0.14.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/RELEASENOTES.0.14.0.md new file mode 100644 index 00000000000..9d34985161a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.0/RELEASENOTES.0.14.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.14.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/CHANGES.0.14.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/CHANGES.0.14.1.md new file mode 100644 index 00000000000..21b297ac45f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/CHANGES.0.14.1.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.14.1 - 2007-09-03 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1798](https://issues.apache.org/jira/browse/HADOOP-1798) | The counts of currently running maps and reduces isn't maintained correctly when task trackers fail | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1790](https://issues.apache.org/jira/browse/HADOOP-1790) | hadoop seems not to support multi-homed installations | Blocker | . | Torsten Curdt | Doug Cutting | +| [HADOOP-1740](https://issues.apache.org/jira/browse/HADOOP-1740) | Certain Pipes tasks fail, after exiting the C++ application | Blocker | . | Srikanth Kakani | Devaraj Das | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/RELEASENOTES.0.14.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/RELEASENOTES.0.14.1.md new file mode 100644 index 00000000000..40f7a45ee90 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.1/RELEASENOTES.0.14.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.14.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/CHANGES.0.14.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/CHANGES.0.14.2.md new file mode 100644 index 00000000000..dc5382830f8 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/CHANGES.0.14.2.md @@ -0,0 +1,74 @@ + + +# Apache Hadoop Changelog + +## Release 0.14.2 - 2007-10-09 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1997](https://issues.apache.org/jira/browse/HADOOP-1997) | TestCheckpoint fails on Windows | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1978](https://issues.apache.org/jira/browse/HADOOP-1978) | Name-node should remove edits.new during startup rather than renaming it to edits. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1977](https://issues.apache.org/jira/browse/HADOOP-1977) | hadoop job -kill , -status causes NullPointerException | Blocker | . | Lohit Vijayarenu | Enis Soztutar | +| [HADOOP-1972](https://issues.apache.org/jira/browse/HADOOP-1972) | LzoCompressor.finished doesn't check whether the user actually wants to finish compressing | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1970](https://issues.apache.org/jira/browse/HADOOP-1970) | tasktracker hang in reduce. Deadlock between main and comm thread | Blocker | . | Koji Noguchi | Vivek Ratan | +| [HADOOP-1961](https://issues.apache.org/jira/browse/HADOOP-1961) | -get, -copyToLocal fail when single filename is passed | Blocker | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-1955](https://issues.apache.org/jira/browse/HADOOP-1955) | Corrupted block replication retries for ever | Blocker | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-1948](https://issues.apache.org/jira/browse/HADOOP-1948) | Spurious error message during block crc upgrade. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1862](https://issues.apache.org/jira/browse/HADOOP-1862) | reduces are getting stuck trying to find map outputs | Blocker | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-1783](https://issues.apache.org/jira/browse/HADOOP-1783) | keyToPath in Jets3tFileSystemStore needs to return absolute path | Major | fs/s3 | Ahad Rana | Tom White | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/RELEASENOTES.0.14.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/RELEASENOTES.0.14.2.md new file mode 100644 index 00000000000..33e5578f68e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.2/RELEASENOTES.0.14.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.14.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/CHANGES.0.14.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/CHANGES.0.14.3.md new file mode 100644 index 00000000000..afc35b2fe5f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/CHANGES.0.14.3.md @@ -0,0 +1,68 @@ + + +# Apache Hadoop Changelog + +## Release 0.14.3 - 2007-10-19 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2072](https://issues.apache.org/jira/browse/HADOOP-2072) | RawLocalFileStatus is causing Path problems | Major | fs | Dennis Kubes | | +| [HADOOP-2053](https://issues.apache.org/jira/browse/HADOOP-2053) | OutOfMemoryError : Java heap space errors in hadoop 0.14 | Blocker | . | Lohit Vijayarenu | Arun C Murthy | +| [HADOOP-2043](https://issues.apache.org/jira/browse/HADOOP-2043) | 0.14.2 release compiled with Java 1.6 instead of Java 1.5 | Blocker | build | Doug Cutting | Doug Cutting | +| [HADOOP-2036](https://issues.apache.org/jira/browse/HADOOP-2036) | NPE in JvmMetrics.doThreadUpdates | Blocker | metrics | Koji Noguchi | Nigel Daley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/RELEASENOTES.0.14.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/RELEASENOTES.0.14.3.md new file mode 100644 index 00000000000..222e0a0ea2a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.3/RELEASENOTES.0.14.3.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.14.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/CHANGES.0.14.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/CHANGES.0.14.4.md new file mode 100644 index 00000000000..c2f17c5efc1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/CHANGES.0.14.4.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.14.4 - 2007-11-26 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2169](https://issues.apache.org/jira/browse/HADOOP-2169) | libhdfs makefile wrongly sets up DT\_SONAME field of libhdfs.so | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-2001](https://issues.apache.org/jira/browse/HADOOP-2001) | Deadlock in jobtracker | Blocker | . | Michael Bieniosek | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2140](https://issues.apache.org/jira/browse/HADOOP-2140) | C and C++ files are missing Apache license header | Blocker | . | Owen O'Malley | Owen O'Malley | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/RELEASENOTES.0.14.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/RELEASENOTES.0.14.4.md new file mode 100644 index 00000000000..274308241e2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.14.4/RELEASENOTES.0.14.4.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.14.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/CHANGES.0.15.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/CHANGES.0.15.0.md new file mode 100644 index 00000000000..e2d136bbb28 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/CHANGES.0.15.0.md @@ -0,0 +1,206 @@ + + +# Apache Hadoop Changelog + +## Release 0.15.0 - 2007-10-19 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1963](https://issues.apache.org/jira/browse/HADOOP-1963) | Code contribution of Kosmos Filesystem implementation of Hadoop Filesystem interface | Major | fs | Sriram Rao | Sriram Rao | +| [HADOOP-1914](https://issues.apache.org/jira/browse/HADOOP-1914) | HDFS should have a NamenodeProtocol to allow secondary namenodes and rebalancing processes to communicate with a primary namenode | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1894](https://issues.apache.org/jira/browse/HADOOP-1894) | Add fancy graphs for mapred task statuses | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1880](https://issues.apache.org/jira/browse/HADOOP-1880) | SleepJob | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1851](https://issues.apache.org/jira/browse/HADOOP-1851) | Map output compression codec cannot be set independently of job output compression codec | Major | . | Riccardo Boscolo | Arun C Murthy | +| [HADOOP-1822](https://issues.apache.org/jira/browse/HADOOP-1822) | Allow SOCKS proxy configuration to remotely access the DFS and submit Jobs | Minor | ipc | Christophe Taton | Christophe Taton | +| [HADOOP-1809](https://issues.apache.org/jira/browse/HADOOP-1809) | Add link to irc channel #hadoop | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1727](https://issues.apache.org/jira/browse/HADOOP-1727) | Make ...hbase.io.MapWritable more generic so that it can be included in ...hadoop.io | Minor | io | Jim Kellerman | Jim Kellerman | +| [HADOOP-1351](https://issues.apache.org/jira/browse/HADOOP-1351) | Want to kill a particular task or attempt | Major | . | Owen O'Malley | Enis Soztutar | +| [HADOOP-789](https://issues.apache.org/jira/browse/HADOOP-789) | DFS shell should return a list of nodes for a file saying that where the blocks for these files are located. | Minor | . | Mahadev konar | Mahadev konar | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2046](https://issues.apache.org/jira/browse/HADOOP-2046) | Documentation: improve mapred javadocs | Blocker | documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-1971](https://issues.apache.org/jira/browse/HADOOP-1971) | Constructing a JobConf without a class leads to a very misleading error message. | Minor | . | Ted Dunning | Enis Soztutar | +| [HADOOP-1968](https://issues.apache.org/jira/browse/HADOOP-1968) | Wildcard input syntax (glob) should support {} | Major | fs | eric baldeschwieler | Hairong Kuang | +| [HADOOP-1942](https://issues.apache.org/jira/browse/HADOOP-1942) | Increase the concurrency of transaction logging to edits log | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1933](https://issues.apache.org/jira/browse/HADOOP-1933) | Consider include/exclude files while listing datanodes. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1926](https://issues.apache.org/jira/browse/HADOOP-1926) | Design/implement a set of compression benchmarks for the map-reduce framework | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1921](https://issues.apache.org/jira/browse/HADOOP-1921) | Save the configuration of completed/failed jobs and make them available via the web-ui. | Major | . | Arun C Murthy | Amar Kamat | +| [HADOOP-1908](https://issues.apache.org/jira/browse/HADOOP-1908) | Restructure data node code so that block sending/receiving is seperated from data transfer header handling | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1906](https://issues.apache.org/jira/browse/HADOOP-1906) | JobConf should warn about the existance of obsolete mapred-default.xml. | Major | conf | Owen O'Malley | Arun C Murthy | +| [HADOOP-1881](https://issues.apache.org/jira/browse/HADOOP-1881) | Update documentation for hadoop's configuration post HADOOP-785 | Major | documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-1878](https://issues.apache.org/jira/browse/HADOOP-1878) | Change priority feature in the job details JSP page misses spaces between each priority link | Trivial | . | Thomas Friol | Thomas Friol | +| [HADOOP-1803](https://issues.apache.org/jira/browse/HADOOP-1803) | Generalize making contrib bin content executable in ant package target | Minor | build | stack | stack | +| [HADOOP-1779](https://issues.apache.org/jira/browse/HADOOP-1779) | Small INodeDirectory enhancement to get all existing INodes components on a path | Trivial | . | Christophe Taton | Christophe Taton | +| [HADOOP-1777](https://issues.apache.org/jira/browse/HADOOP-1777) | Typo issue in the job details JSP page | Trivial | . | Thomas Friol | Thomas Friol | +| [HADOOP-1774](https://issues.apache.org/jira/browse/HADOOP-1774) | Remove use of INode.parent in Block CRC upgrade | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1767](https://issues.apache.org/jira/browse/HADOOP-1767) | JobClient CLI cleanup and improvement | Minor | . | Christophe Taton | Christophe Taton | +| [HADOOP-1766](https://issues.apache.org/jira/browse/HADOOP-1766) | Merging Block and BlockInfo classes on name-node. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1762](https://issues.apache.org/jira/browse/HADOOP-1762) | Namenode does not need to store storageID and datanodeID persistently | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1759](https://issues.apache.org/jira/browse/HADOOP-1759) | File name should be represented by a byte array instead of a String | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1756](https://issues.apache.org/jira/browse/HADOOP-1756) | Add toString() methods to some Writable types | Major | io | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-1750](https://issues.apache.org/jira/browse/HADOOP-1750) | We should log better if something goes wrong with the process fork | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1744](https://issues.apache.org/jira/browse/HADOOP-1744) | Small cleanup of DistributedFileSystem and DFSClient (next) | Trivial | . | Christophe Taton | Christophe Taton | +| [HADOOP-1743](https://issues.apache.org/jira/browse/HADOOP-1743) | INode refactoring | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1731](https://issues.apache.org/jira/browse/HADOOP-1731) | contrib jar file names should include hadoop version number | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-1718](https://issues.apache.org/jira/browse/HADOOP-1718) | Test coverage target in build files using clover | Major | build | Nigel Daley | Nigel Daley | +| [HADOOP-1703](https://issues.apache.org/jira/browse/HADOOP-1703) | Small cleanup of DistributedFileSystem and DFSClient | Trivial | . | Christophe Taton | Christophe Taton | +| [HADOOP-1693](https://issues.apache.org/jira/browse/HADOOP-1693) | Remove LOG members from PendingReplicationBlocks and ReplicationTargetChooser. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1687](https://issues.apache.org/jira/browse/HADOOP-1687) | Name-node memory size estimates and optimization proposal. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1667](https://issues.apache.org/jira/browse/HADOOP-1667) | organize CHANGES.txt messages into sections for future releases | Major | documentation | Doug Cutting | Doug Cutting | +| [HADOOP-1654](https://issues.apache.org/jira/browse/HADOOP-1654) | IOUtils class | Major | io | Enis Soztutar | Enis Soztutar | +| [HADOOP-1626](https://issues.apache.org/jira/browse/HADOOP-1626) | DFSAdmin. Help messages are missing for -finalizeUpgrade and -metasave. | Blocker | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-1621](https://issues.apache.org/jira/browse/HADOOP-1621) | Make FileStatus a concrete class | Major | fs | Chris Douglas | Chris Douglas | +| [HADOOP-1610](https://issues.apache.org/jira/browse/HADOOP-1610) | Add metrics for failed tasks | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1595](https://issues.apache.org/jira/browse/HADOOP-1595) | Add an option to setReplication method to wait for completion of replication | Major | . | Christian Kunz | Tsz Wo Nicholas Sze | +| [HADOOP-1592](https://issues.apache.org/jira/browse/HADOOP-1592) | Print the diagnostic error messages for FAILED task-attempts to the user console via TaskCompletionEvents | Major | . | Arun C Murthy | Amar Kamat | +| [HADOOP-1500](https://issues.apache.org/jira/browse/HADOOP-1500) | typo's in dfs webui | Trivial | . | Nigel Daley | Nigel Daley | +| [HADOOP-1436](https://issues.apache.org/jira/browse/HADOOP-1436) | Redesign Tool and ToolBase API and releted functionality | Major | util | Enis Soztutar | Enis Soztutar | +| [HADOOP-1425](https://issues.apache.org/jira/browse/HADOOP-1425) | Rework the various programs in 'examples' to extend ToolBase | Minor | . | Arun C Murthy | Enis Soztutar | +| [HADOOP-1266](https://issues.apache.org/jira/browse/HADOOP-1266) | Remove DatanodeDescriptor dependency from NetworkTopology | Major | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-1231](https://issues.apache.org/jira/browse/HADOOP-1231) | Add generics to Mapper and Reducer interfaces | Major | . | Owen O'Malley | Tom White | +| [HADOOP-1158](https://issues.apache.org/jira/browse/HADOOP-1158) | JobTracker should collect statistics of failed map output fetches, and take decisions to reexecute map tasks and/or restart the (possibly faulty) Jetty server on the TaskTracker | Major | . | Devaraj Das | Arun C Murthy | +| [HADOOP-785](https://issues.apache.org/jira/browse/HADOOP-785) | Divide the server and client configurations | Major | conf | Owen O'Malley | Arun C Murthy | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2103](https://issues.apache.org/jira/browse/HADOOP-2103) | HADOOP-2046 caused some javadoc anomalies | Major | documentation | Nigel Daley | Nigel Daley | +| [HADOOP-2102](https://issues.apache.org/jira/browse/HADOOP-2102) | ToolBase doesn't keep configuration | Blocker | util | Dennis Kubes | Dennis Kubes | +| [HADOOP-2080](https://issues.apache.org/jira/browse/HADOOP-2080) | ChecksumFileSystem checksum file size incorrect. | Blocker | fs | Richard Lee | Owen O'Malley | +| [HADOOP-2073](https://issues.apache.org/jira/browse/HADOOP-2073) | Datanode corruption if machine dies while writing VERSION file | Blocker | . | Michael Bieniosek | Konstantin Shvachko | +| [HADOOP-2072](https://issues.apache.org/jira/browse/HADOOP-2072) | RawLocalFileStatus is causing Path problems | Major | fs | Dennis Kubes | | +| [HADOOP-2070](https://issues.apache.org/jira/browse/HADOOP-2070) | Test org.apache.hadoop.mapred.pipes.TestPipes.unknown failed | Blocker | . | Mukund Madhugiri | Owen O'Malley | +| [HADOOP-2051](https://issues.apache.org/jira/browse/HADOOP-2051) | JobTracker's TaskCommitQueue is vulnerable to non-IOExceptions | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-2048](https://issues.apache.org/jira/browse/HADOOP-2048) | DISTCP mapper should report progress more often | Blocker | . | Runping Qi | Chris Douglas | +| [HADOOP-2044](https://issues.apache.org/jira/browse/HADOOP-2044) | Namenode encounters ClassCastException exceptions for INodeFileUnderConstruction | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2033](https://issues.apache.org/jira/browse/HADOOP-2033) | In SequenceFile sync doesn't work unless the file is compressed (block or record) | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2031](https://issues.apache.org/jira/browse/HADOOP-2031) | Lost tasktracker not handled properly leading to tips wrongly being kept as completed, and hence not rescheduled | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-2028](https://issues.apache.org/jira/browse/HADOOP-2028) | distcp fails if log dir not specified and destination not present | Blocker | util | Chris Douglas | Chris Douglas | +| [HADOOP-2026](https://issues.apache.org/jira/browse/HADOOP-2026) | Namenode prints out too many log lines for "Number of transactions" | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2023](https://issues.apache.org/jira/browse/HADOOP-2023) | TestLocalDirAllocator fails on Windows | Blocker | fs | Mukund Madhugiri | Hairong Kuang | +| [HADOOP-2022](https://issues.apache.org/jira/browse/HADOOP-2022) | Task times are not saved correctly (bug in hadoop-1874) | Blocker | . | Devaraj Das | Amar Kamat | +| [HADOOP-2018](https://issues.apache.org/jira/browse/HADOOP-2018) | Broken pipe SocketException in DataNode$DataXceiver | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-2016](https://issues.apache.org/jira/browse/HADOOP-2016) | Race condition in removing a KILLED task from tasktracker | Blocker | . | Devaraj Das | Arun C Murthy | +| [HADOOP-1997](https://issues.apache.org/jira/browse/HADOOP-1997) | TestCheckpoint fails on Windows | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1992](https://issues.apache.org/jira/browse/HADOOP-1992) | Sort validation is taking considerably longer than before | Blocker | . | Mukund Madhugiri | Arun C Murthy | +| [HADOOP-1983](https://issues.apache.org/jira/browse/HADOOP-1983) | jobs using pipes interface with tasks not using java output format have a good chance of not updating progress and timing out | Major | . | Christian Kunz | Owen O'Malley | +| [HADOOP-1978](https://issues.apache.org/jira/browse/HADOOP-1978) | Name-node should remove edits.new during startup rather than renaming it to edits. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1973](https://issues.apache.org/jira/browse/HADOOP-1973) | NPE at JobTracker startup.. | Blocker | . | Gautam Kowshik | Amareshwari Sriramadasu | +| [HADOOP-1961](https://issues.apache.org/jira/browse/HADOOP-1961) | -get, -copyToLocal fail when single filename is passed | Blocker | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-1959](https://issues.apache.org/jira/browse/HADOOP-1959) | Use of File.separator in StatusHttpServer prevents running Junit tests inside eclipse on Windows | Minor | . | Jim Kellerman | Jim Kellerman | +| [HADOOP-1955](https://issues.apache.org/jira/browse/HADOOP-1955) | Corrupted block replication retries for ever | Blocker | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-1953](https://issues.apache.org/jira/browse/HADOOP-1953) | the job tracker should wait beteween calls to try and delete the system directory | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1948](https://issues.apache.org/jira/browse/HADOOP-1948) | Spurious error message during block crc upgrade. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1946](https://issues.apache.org/jira/browse/HADOOP-1946) | du should be not called on every heartbeat | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-1944](https://issues.apache.org/jira/browse/HADOOP-1944) | Maps which ran on trackers declared 'lost' are being marked as FAILED rather than KILLED | Blocker | . | Arun C Murthy | Devaraj Das | +| [HADOOP-1940](https://issues.apache.org/jira/browse/HADOOP-1940) | TestDFSUpgradeFromImage doesn't shut down its MiniDFSCluster | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-1935](https://issues.apache.org/jira/browse/HADOOP-1935) | NullPointerException in internalReleaseCreate | Blocker | . | Konstantin Shvachko | dhruba borthakur | +| [HADOOP-1934](https://issues.apache.org/jira/browse/HADOOP-1934) | the os.name string on Mac OS contains spaces, which causes the c++ compilation to fail | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1932](https://issues.apache.org/jira/browse/HADOOP-1932) | Test dfs.TestFileCreation.testFileCreation failed on Windows | Blocker | test | Mukund Madhugiri | dhruba borthakur | +| [HADOOP-1930](https://issues.apache.org/jira/browse/HADOOP-1930) | Too many fetch-failures issue | Blocker | . | Christian Kunz | Arun C Murthy | +| [HADOOP-1925](https://issues.apache.org/jira/browse/HADOOP-1925) | Hadoop Pipes doesn't compile on solaris | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1910](https://issues.apache.org/jira/browse/HADOOP-1910) | Extra checks in DFS.create() are not necessary. | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1907](https://issues.apache.org/jira/browse/HADOOP-1907) | JobClient.runJob kills the job for failed tasks with no diagnostics | Major | . | Christian Kunz | Christian Kunz | +| [HADOOP-1904](https://issues.apache.org/jira/browse/HADOOP-1904) | ArrayIndexOutOfBoundException in BlocksMap | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1897](https://issues.apache.org/jira/browse/HADOOP-1897) | about.html page is there but not linked. | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-1892](https://issues.apache.org/jira/browse/HADOOP-1892) | In the Job UI, some links don't work | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-1890](https://issues.apache.org/jira/browse/HADOOP-1890) | Revert a debug patch. | Trivial | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1889](https://issues.apache.org/jira/browse/HADOOP-1889) | Fix path in EC2 scripts for building your own AMI | Major | contrib/cloud | Tom White | Tom White | +| [HADOOP-1887](https://issues.apache.org/jira/browse/HADOOP-1887) | ArrayIndexOutOfBoundsException with trunk | Major | . | Raghu Angadi | dhruba borthakur | +| [HADOOP-1885](https://issues.apache.org/jira/browse/HADOOP-1885) | Race condition in MiniDFSCluster shutdown | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-1882](https://issues.apache.org/jira/browse/HADOOP-1882) | Remove extra '\*'s from FsShell.limitDecimal() | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1875](https://issues.apache.org/jira/browse/HADOOP-1875) | multiple dfs.client.buffer.dir directories are not treated as alternatives | Blocker | fs | Christian Kunz | Hairong Kuang | +| [HADOOP-1874](https://issues.apache.org/jira/browse/HADOOP-1874) | lost task trackers -- jobs hang | Blocker | . | Christian Kunz | Devaraj Das | +| [HADOOP-1846](https://issues.apache.org/jira/browse/HADOOP-1846) | DatanodeReport should distinguish live datanodes from dead datanodes | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1840](https://issues.apache.org/jira/browse/HADOOP-1840) | Task's diagnostic messages are lost sometimes | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1838](https://issues.apache.org/jira/browse/HADOOP-1838) | Files created with an pre-0.15 gets blocksize as zero, causing performance degradation | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1832](https://issues.apache.org/jira/browse/HADOOP-1832) | listTables() returns duplicate tables | Major | . | Andrew Hitchcock | Jim Kellerman | +| [HADOOP-1825](https://issues.apache.org/jira/browse/HADOOP-1825) | hadoop-daemon.sh script fails if HADOOP\_PID\_DIR doesn't exist | Minor | scripts | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-1819](https://issues.apache.org/jira/browse/HADOOP-1819) | The JobTracker should ensure that it is running on the right host. | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1818](https://issues.apache.org/jira/browse/HADOOP-1818) | MutliFileInputFormat returns "empty" MultiFileSplit when number of paths \< number of splits | Major | . | Thomas Friol | Thomas Friol | +| [HADOOP-1817](https://issues.apache.org/jira/browse/HADOOP-1817) | MultiFileSplit does not write and read the total length | Major | . | Thomas Friol | Thomas Friol | +| [HADOOP-1812](https://issues.apache.org/jira/browse/HADOOP-1812) | TestIPC and TestRPC should use dynamically allocated ports | Major | ipc | Doug Cutting | Doug Cutting | +| [HADOOP-1810](https://issues.apache.org/jira/browse/HADOOP-1810) | Incorrect Value type in MRBench (SmallJobs) | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-1806](https://issues.apache.org/jira/browse/HADOOP-1806) | DfsTask no longer compiles | Major | build | Chris Douglas | Chris Douglas | +| [HADOOP-1795](https://issues.apache.org/jira/browse/HADOOP-1795) | Task.moveTaskOutputs is escaping special characters in output filenames | Critical | . | FrĂ©dĂ©ric Bertin | FrĂ©dĂ©ric Bertin | +| [HADOOP-1792](https://issues.apache.org/jira/browse/HADOOP-1792) | df command doesn't exist under windows | Major | fs | Benjamin Francisoud | Mahadev konar | +| [HADOOP-1788](https://issues.apache.org/jira/browse/HADOOP-1788) | Increase the buffer size of pipes from 1k to 128k | Blocker | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-1783](https://issues.apache.org/jira/browse/HADOOP-1783) | keyToPath in Jets3tFileSystemStore needs to return absolute path | Major | fs/s3 | Ahad Rana | Tom White | +| [HADOOP-1775](https://issues.apache.org/jira/browse/HADOOP-1775) | MapWritable and SortedMapWritable - Writable problems | Major | io | Jim Kellerman | Jim Kellerman | +| [HADOOP-1772](https://issues.apache.org/jira/browse/HADOOP-1772) | Hadoop does not run in Cygwin in Windows | Critical | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-1771](https://issues.apache.org/jira/browse/HADOOP-1771) | streaming hang when IOException in MROutputThread. (NPE) | Blocker | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-1758](https://issues.apache.org/jira/browse/HADOOP-1758) | processing escapes in a jute record is quadratic | Blocker | record | Dick King | Vivek Ratan | +| [HADOOP-1749](https://issues.apache.org/jira/browse/HADOOP-1749) | TestDFSUpgrade some times fails with an assert | Major | . | Raghu Angadi | Enis Soztutar | +| [HADOOP-1748](https://issues.apache.org/jira/browse/HADOOP-1748) | Task Trackers fail to launch tasks when they have relative log directories configured | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-1739](https://issues.apache.org/jira/browse/HADOOP-1739) | ConnectException in TaskTracker Child | Major | . | Srikanth Kakani | Doug Cutting | +| [HADOOP-1708](https://issues.apache.org/jira/browse/HADOOP-1708) | make files visible in the namespace as soon as they are created | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1695](https://issues.apache.org/jira/browse/HADOOP-1695) | Secondary Namenode halt when SocketTimeoutException at startup | Blocker | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-1692](https://issues.apache.org/jira/browse/HADOOP-1692) | DfsTask cache interferes with operation | Minor | util | Chris Douglas | Chris Douglas | +| [HADOOP-1689](https://issues.apache.org/jira/browse/HADOOP-1689) | .sh scripts do not work on Solaris | Minor | scripts | David Biesack | Doug Cutting | +| [HADOOP-1656](https://issues.apache.org/jira/browse/HADOOP-1656) | HDFS does not record the blocksize for a file | Major | . | Sameer Paranjpye | dhruba borthakur | +| [HADOOP-1651](https://issues.apache.org/jira/browse/HADOOP-1651) | Some improvements in progress reporting | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-1636](https://issues.apache.org/jira/browse/HADOOP-1636) | constant should be user-configurable: MAX\_COMPLETE\_USER\_JOBS\_IN\_MEMORY | Major | . | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-1601](https://issues.apache.org/jira/browse/HADOOP-1601) | GenericWritable should use ReflectionUtils.newInstance to avoid problems with classloaders | Major | io | Owen O'Malley | Enis Soztutar | +| [HADOOP-1573](https://issues.apache.org/jira/browse/HADOOP-1573) | Support for 0 reducers in PIPES | Major | . | Christian Kunz | Owen O'Malley | +| [HADOOP-1569](https://issues.apache.org/jira/browse/HADOOP-1569) | distcp should use the Path -\> FileSystem interface like the rest of Hadoop | Major | util | Owen O'Malley | Chris Douglas | +| [HADOOP-1565](https://issues.apache.org/jira/browse/HADOOP-1565) | DFSScalability: reduce memory usage of namenode | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1463](https://issues.apache.org/jira/browse/HADOOP-1463) | dfs.datanode.du.reserved semantics being violated | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1316](https://issues.apache.org/jira/browse/HADOOP-1316) | "Go to parent directory" does not work on windows. | Minor | . | Konstantin Shvachko | Mahadev konar | +| [HADOOP-1076](https://issues.apache.org/jira/browse/HADOOP-1076) | Periodic checkpointing cannot resume if the secondary name-node fails. | Major | . | Konstantin Shvachko | dhruba borthakur | +| [HADOOP-1018](https://issues.apache.org/jira/browse/HADOOP-1018) | Single lost heartbeat leads to a "Lost task tracker" | Major | . | Andrzej Bialecki | Arun C Murthy | +| [HADOOP-999](https://issues.apache.org/jira/browse/HADOOP-999) | DFS Client should create file when the user creates the file | Major | . | Owen O'Malley | Tsz Wo Nicholas Sze | +| [HADOOP-932](https://issues.apache.org/jira/browse/HADOOP-932) | File locking interface and implementation should be remvoed. | Minor | fs | Raghu Angadi | Raghu Angadi | +| [HADOOP-795](https://issues.apache.org/jira/browse/HADOOP-795) | hdfs -cp /a/b/c /x/y acts like hdfs -cp /a/b/c/\* /x/y | Minor | . | arkady borkovsky | Mahadev konar | +| [HADOOP-120](https://issues.apache.org/jira/browse/HADOOP-120) | Reading an ArrayWriter does not work because valueClass does not get initialized | Major | io | Dick King | Cameron Pope | +| [HADOOP-89](https://issues.apache.org/jira/browse/HADOOP-89) | files are not visible until they are closed | Critical | . | Yoram Arnon | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1879](https://issues.apache.org/jira/browse/HADOOP-1879) | Warnings With JDK1.6.0\_02 | Minor | . | Nilay Vaish | Nilay Vaish | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/RELEASENOTES.0.15.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/RELEASENOTES.0.15.0.md new file mode 100644 index 00000000000..ea88f02867c --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.0/RELEASENOTES.0.15.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.15.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/CHANGES.0.15.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/CHANGES.0.15.1.md new file mode 100644 index 00000000000..670e22a0521 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/CHANGES.0.15.1.md @@ -0,0 +1,71 @@ + + +# Apache Hadoop Changelog + +## Release 0.15.1 - 2007-11-27 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1917](https://issues.apache.org/jira/browse/HADOOP-1917) | Need configuration guides for Hadoop | Critical | documentation | Sameer Paranjpye | Arun C Murthy | +| [HADOOP-713](https://issues.apache.org/jira/browse/HADOOP-713) | dfs list operation is too expensive | Blocker | . | Hairong Kuang | dhruba borthakur | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2205](https://issues.apache.org/jira/browse/HADOOP-2205) | Regenerate entire hadoop website since site.xml was changed by HADOOP-1917 | Blocker | documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-2174](https://issues.apache.org/jira/browse/HADOOP-2174) | distcp throws a NullPointerException in the close() method of mapper class due to the Reporter becoming invalid | Critical | util | Riccardo Boscolo | Chris Douglas | +| [HADOOP-2172](https://issues.apache.org/jira/browse/HADOOP-2172) | PositionCache was removed from FSDataInputStream, causes extremely bad MapFile performance | Blocker | fs | Johan Oskarsson | Doug Cutting | +| [HADOOP-2169](https://issues.apache.org/jira/browse/HADOOP-2169) | libhdfs makefile wrongly sets up DT\_SONAME field of libhdfs.so | Critical | . | Arun C Murthy | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2140](https://issues.apache.org/jira/browse/HADOOP-2140) | C and C++ files are missing Apache license header | Blocker | . | Owen O'Malley | Owen O'Malley | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/RELEASENOTES.0.15.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/RELEASENOTES.0.15.1.md new file mode 100644 index 00000000000..05d175f0f7a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.1/RELEASENOTES.0.15.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.15.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/CHANGES.0.15.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/CHANGES.0.15.2.md new file mode 100644 index 00000000000..10fc49ee806 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/CHANGES.0.15.2.md @@ -0,0 +1,79 @@ + + +# Apache Hadoop Changelog + +## Release 0.15.2 - 2008-01-08 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2382](https://issues.apache.org/jira/browse/HADOOP-2382) | include hadoop-default.html in subversion | Minor | documentation | Doug Cutting | | +| [HADOOP-2160](https://issues.apache.org/jira/browse/HADOOP-2160) | separate website from user documentation | Major | documentation | Doug Cutting | Doug Cutting | +| [HADOOP-1327](https://issues.apache.org/jira/browse/HADOOP-1327) | Doc on Streaming | Major | documentation | Runping Qi | Rob Weltman | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2486](https://issues.apache.org/jira/browse/HADOOP-2486) | Dropping records at reducer. InMemoryFileSystem NPE. | Blocker | . | Koji Noguchi | Devaraj Das | +| [HADOOP-2460](https://issues.apache.org/jira/browse/HADOOP-2460) | NameNode could delete wrong edits file when there is an error | Major | . | Raghu Angadi | dhruba borthakur | +| [HADOOP-2456](https://issues.apache.org/jira/browse/HADOOP-2456) | German locale makes NameNode web interface crash | Minor | . | Matthias Friedrich | Matthias Friedrich | +| [HADOOP-2437](https://issues.apache.org/jira/browse/HADOOP-2437) | final map output not evenly distributed across multiple disks | Blocker | . | Christian Kunz | Arun C Murthy | +| [HADOOP-2422](https://issues.apache.org/jira/browse/HADOOP-2422) | dfs -cat multiple files fail with 'Unable to write to output stream.' | Blocker | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-2378](https://issues.apache.org/jira/browse/HADOOP-2378) | last TaskCompletionEvent gets added to the job after the job is marked as completed | Blocker | . | Alejandro Abdelnur | Devaraj Das | +| [HADOOP-2246](https://issues.apache.org/jira/browse/HADOOP-2246) | In CHANGES.txt, move HADOOP-1851 & HADOOP-1231 to INCOMPATIBLE CHANGES section | Blocker | documentation | Devaraj Das | Arun C Murthy | +| [HADOOP-2238](https://issues.apache.org/jira/browse/HADOOP-2238) | TaskGraphServlet does not set Content-Type | Major | . | Paul Saab | | +| [HADOOP-2228](https://issues.apache.org/jira/browse/HADOOP-2228) | Jobs fail because job.xml exists | Major | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-2227](https://issues.apache.org/jira/browse/HADOOP-2227) | wrong usage of mapred.local.dir.minspacestart | Critical | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-2158](https://issues.apache.org/jira/browse/HADOOP-2158) | hdfsListDirectory in libhdfs does not scale | Blocker | . | Christian Kunz | Christian Kunz | +| [HADOOP-2129](https://issues.apache.org/jira/browse/HADOOP-2129) | distcp between two clusters does not work if it is run on the target cluster | Critical | util | Murtaza A. Basrai | Doug Cutting | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/RELEASENOTES.0.15.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/RELEASENOTES.0.15.2.md new file mode 100644 index 00000000000..801d958ea52 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.2/RELEASENOTES.0.15.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.15.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/CHANGES.0.15.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/CHANGES.0.15.3.md new file mode 100644 index 00000000000..9c15db6c64f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/CHANGES.0.15.3.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 0.15.3 - 2008-01-21 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2574](https://issues.apache.org/jira/browse/HADOOP-2574) | bugs in mapred tutorial | Major | documentation | Doug Cutting | Arun C Murthy | +| [HADOOP-2570](https://issues.apache.org/jira/browse/HADOOP-2570) | streaming jobs fail after HADOOP-2227 | Blocker | . | Lohit Vijayarenu | Amareshwari Sriramadasu | +| [HADOOP-2562](https://issues.apache.org/jira/browse/HADOOP-2562) | globPaths does not support {ab,cd} as it claims to | Blocker | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-2540](https://issues.apache.org/jira/browse/HADOOP-2540) | Empty blocks make fsck report corrupt, even when it isn't | Blocker | . | Allen Wittenauer | dhruba borthakur | +| [HADOOP-650](https://issues.apache.org/jira/browse/HADOOP-650) | bin/hadoop dfs -lsr / is broken | Minor | . | Mahadev konar | Mahadev konar | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/RELEASENOTES.0.15.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/RELEASENOTES.0.15.3.md new file mode 100644 index 00000000000..c5e628d8d9d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.3/RELEASENOTES.0.15.3.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.15.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/CHANGES.0.15.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/CHANGES.0.15.4.md new file mode 100644 index 00000000000..41aa5f003cb --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/CHANGES.0.15.4.md @@ -0,0 +1,65 @@ + + +# Apache Hadoop Changelog + +## Release 0.15.4 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2647](https://issues.apache.org/jira/browse/HADOOP-2647) | dfs -put hangs | Major | . | Lohit Vijayarenu | Raghu Angadi | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/RELEASENOTES.0.15.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/RELEASENOTES.0.15.4.md new file mode 100644 index 00000000000..679f0d39d99 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.15.4/RELEASENOTES.0.15.4.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.15.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/CHANGES.0.16.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/CHANGES.0.16.0.md new file mode 100644 index 00000000000..1422bdc5598 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/CHANGES.0.16.0.md @@ -0,0 +1,241 @@ + + +# Apache Hadoop Changelog + +## Release 0.16.0 - 2008-02-07 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2603](https://issues.apache.org/jira/browse/HADOOP-2603) | SequenceFileAsBinaryInputFormat | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-2567](https://issues.apache.org/jira/browse/HADOOP-2567) | add FileSystem#getHomeDirectory() method | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-2543](https://issues.apache.org/jira/browse/HADOOP-2543) | No-permission-checking mode for smooth transition to 0.16's permissions features. | Major | . | Sanjay Radia | Hairong Kuang | +| [HADOOP-2529](https://issues.apache.org/jira/browse/HADOOP-2529) | DFS User Guide | Major | documentation | Raghu Angadi | Raghu Angadi | +| [HADOOP-2514](https://issues.apache.org/jira/browse/HADOOP-2514) | Trash and permissions don't mix | Major | . | Robert Chansler | Doug Cutting | +| [HADOOP-2487](https://issues.apache.org/jira/browse/HADOOP-2487) | Provide an option to get job status for all jobs run by or submitted to a job tracker | Major | . | Hemanth Yamijala | Amareshwari Sriramadasu | +| [HADOOP-2447](https://issues.apache.org/jira/browse/HADOOP-2447) | HDFS should be capable of limiting the total number of inodes in the system | Major | . | Sameer Paranjpye | dhruba borthakur | +| [HADOOP-2398](https://issues.apache.org/jira/browse/HADOOP-2398) | Additional Instrumentation for NameNode, RPC Layer and JMX support | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-2381](https://issues.apache.org/jira/browse/HADOOP-2381) | Support permission information in FileStatus | Major | fs | Tsz Wo Nicholas Sze | Raghu Angadi | +| [HADOOP-2367](https://issues.apache.org/jira/browse/HADOOP-2367) | Get representative hprof information from tasks | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2336](https://issues.apache.org/jira/browse/HADOOP-2336) | Shell commands to access and modify file permissions | Major | fs | Raghu Angadi | Raghu Angadi | +| [HADOOP-2288](https://issues.apache.org/jira/browse/HADOOP-2288) | Change FileSystem API to support access control. | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2229](https://issues.apache.org/jira/browse/HADOOP-2229) | Provide a simple login implementation | Major | fs | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HADOOP-2184](https://issues.apache.org/jira/browse/HADOOP-2184) | RPC Support for user permissions and authentication. | Major | ipc | Tsz Wo Nicholas Sze | Raghu Angadi | +| [HADOOP-2145](https://issues.apache.org/jira/browse/HADOOP-2145) | need 'doc' target that runs forrest | Major | build | Doug Cutting | Doug Cutting | +| [HADOOP-2085](https://issues.apache.org/jira/browse/HADOOP-2085) | Map-side joins on sorted, equally-partitioned datasets | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-2045](https://issues.apache.org/jira/browse/HADOOP-2045) | credits page should have more information | Major | documentation | Doug Cutting | Doug Cutting | +| [HADOOP-2012](https://issues.apache.org/jira/browse/HADOOP-2012) | Periodic verification at the Datanode | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-1912](https://issues.apache.org/jira/browse/HADOOP-1912) | Datanode should support block replacement | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1652](https://issues.apache.org/jira/browse/HADOOP-1652) | Rebalance data blocks when new data nodes added or data nodes become full | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-1604](https://issues.apache.org/jira/browse/HADOOP-1604) | admins should be able to finalize namenode upgrades without running the cluster | Critical | . | Owen O'Malley | Konstantin Shvachko | +| [HADOOP-1301](https://issues.apache.org/jira/browse/HADOOP-1301) | resource management proviosioning for Hadoop | Major | . | Pete Wyckoff | Hemanth Yamijala | +| [HADOOP-1298](https://issues.apache.org/jira/browse/HADOOP-1298) | adding user info to file | Major | fs | Kurtis Heimerl | Tsz Wo Nicholas Sze | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2645](https://issues.apache.org/jira/browse/HADOOP-2645) | Additional metrics & jmx beans and cleanup to use the recent metrics libraries | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-2596](https://issues.apache.org/jira/browse/HADOOP-2596) | add SequenceFile.createWriter() method that takes block size as parameter | Minor | io | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-2566](https://issues.apache.org/jira/browse/HADOOP-2566) | need FileSystem#globStatus method | Major | fs | Doug Cutting | Hairong Kuang | +| [HADOOP-2552](https://issues.apache.org/jira/browse/HADOOP-2552) | enable hdfs permission checking by default | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-2547](https://issues.apache.org/jira/browse/HADOOP-2547) | remove use of 'magic number' in build.xml | Trivial | build | Hrishikesh | Hrishikesh | +| [HADOOP-2469](https://issues.apache.org/jira/browse/HADOOP-2469) | WritableUtils.clone should take Configuration rather than JobConf | Minor | io | stack | stack | +| [HADOOP-2466](https://issues.apache.org/jira/browse/HADOOP-2466) | FileInputFormat computeSplitSize() method, change visibility to protected and make it a member method | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-2464](https://issues.apache.org/jira/browse/HADOOP-2464) | Test permissions related shell commands with DFS | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2457](https://issues.apache.org/jira/browse/HADOOP-2457) | Add a 'forrest.home' property for the 'docs' target in build.xml | Minor | documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-2432](https://issues.apache.org/jira/browse/HADOOP-2432) | If HDFS is going to throw an exception "File does not exist" it should include the name of the file | Minor | . | Jim Kellerman | Jim Kellerman | +| [HADOOP-2425](https://issues.apache.org/jira/browse/HADOOP-2425) | TextOutputFormat should special case Text | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2401](https://issues.apache.org/jira/browse/HADOOP-2401) | Lease holder information should be passed in ClientProtocol.abandonBlock(...) | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2390](https://issues.apache.org/jira/browse/HADOOP-2390) | Document the user-controls for intermediate/output compression via forrest | Major | documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-2349](https://issues.apache.org/jira/browse/HADOOP-2349) | FSEditLog.logEdit(byte op, Writable w1, Writable w2) should accept variable numbers of Writable, instead of two. | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2298](https://issues.apache.org/jira/browse/HADOOP-2298) | ant target without source and docs | Major | build | Gautam Kowshik | Hrishikesh | +| [HADOOP-2268](https://issues.apache.org/jira/browse/HADOOP-2268) | JobControl classes should use interfaces rather than implemenations | Minor | . | Adrian Woodhead | Adrian Woodhead | +| [HADOOP-2233](https://issues.apache.org/jira/browse/HADOOP-2233) | General example for modeling m/r load in Java | Minor | test | Chris Douglas | Chris Douglas | +| [HADOOP-2232](https://issues.apache.org/jira/browse/HADOOP-2232) | Add option to disable nagles algorithm in the IPC Server | Major | ipc | Clint Morgan | Clint Morgan | +| [HADOOP-2208](https://issues.apache.org/jira/browse/HADOOP-2208) | Reduce frequency of Counter updates in the task tracker status | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2185](https://issues.apache.org/jira/browse/HADOOP-2185) | Server ports: to roll or not to roll. | Major | conf | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2149](https://issues.apache.org/jira/browse/HADOOP-2149) | Pure name-node benchmarks. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2134](https://issues.apache.org/jira/browse/HADOOP-2134) | Remove developer-centric requirements from overview.html | Major | documentation | Arun C Murthy | Jim Kellerman | +| [HADOOP-2131](https://issues.apache.org/jira/browse/HADOOP-2131) | Speculative execution should be allowed for reducers only | Critical | . | Srikanth Kakani | Amareshwari Sriramadasu | +| [HADOOP-2127](https://issues.apache.org/jira/browse/HADOOP-2127) | Add pipes sort example | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2113](https://issues.apache.org/jira/browse/HADOOP-2113) | Add "-text" command to FsShell to decode SequenceFile to stdout | Minor | fs | Chris Douglas | Chris Douglas | +| [HADOOP-2105](https://issues.apache.org/jira/browse/HADOOP-2105) | Clarify requirements for Hadoop in overview.html | Minor | . | Jim Kellerman | Jim Kellerman | +| [HADOOP-2086](https://issues.apache.org/jira/browse/HADOOP-2086) | ability to add dependencies to a job after construction | Major | . | Adrian Woodhead | Adrian Woodhead | +| [HADOOP-2077](https://issues.apache.org/jira/browse/HADOOP-2077) | Logging version number (and compiled date) at STARTUP\_MSG | Trivial | . | Koji Noguchi | Arun C Murthy | +| [HADOOP-2058](https://issues.apache.org/jira/browse/HADOOP-2058) | Allow adding additional datanodes to MiniDFSCluster | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-1989](https://issues.apache.org/jira/browse/HADOOP-1989) | Add support for simulated Data Nodes - helpful for testing and performance benchmarking of the Name Node without having a large cluster | Minor | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-1965](https://issues.apache.org/jira/browse/HADOOP-1965) | Handle map output buffers better | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-1900](https://issues.apache.org/jira/browse/HADOOP-1900) | the heartbeat and task event queries interval should be set dynamically by the JobTracker | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-1876](https://issues.apache.org/jira/browse/HADOOP-1876) | Persisting completed jobs status | Critical | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-1873](https://issues.apache.org/jira/browse/HADOOP-1873) | User permissions for Map/Reduce | Major | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-1857](https://issues.apache.org/jira/browse/HADOOP-1857) | Ability to run a script when a task fails to capture stack traces | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-1855](https://issues.apache.org/jira/browse/HADOOP-1855) | fsck should verify block placement | Major | . | dhruba borthakur | Konstantin Shvachko | +| [HADOOP-1848](https://issues.apache.org/jira/browse/HADOOP-1848) | Redesign of Eclipse plug-in interface with Hadoop | Major | . | Christophe Taton | Christophe Taton | +| [HADOOP-1843](https://issues.apache.org/jira/browse/HADOOP-1843) | Remove deprecated code in Configuration/JobConf | Major | conf | Arun C Murthy | Arun C Murthy | +| [HADOOP-1841](https://issues.apache.org/jira/browse/HADOOP-1841) | IPC server should write repsonses asynchronously | Major | ipc | Doug Cutting | dhruba borthakur | +| [HADOOP-1839](https://issues.apache.org/jira/browse/HADOOP-1839) | Link-ify the Pending/Running/Complete/Killed tasks/task-attempts on jobdetails.jsp | Major | . | Arun C Murthy | Amar Kamat | +| [HADOOP-1742](https://issues.apache.org/jira/browse/HADOOP-1742) | FSNamesystem.startFile() javadoc is inconsistent | Minor | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-1719](https://issues.apache.org/jira/browse/HADOOP-1719) | Improve the utilization of shuffle copier threads | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-1707](https://issues.apache.org/jira/browse/HADOOP-1707) | Remove the DFS Client disk-based cache | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1660](https://issues.apache.org/jira/browse/HADOOP-1660) | add support for native library toDistributedCache | Major | . | Alejandro Abdelnur | Arun C Murthy | +| [HADOOP-1302](https://issues.apache.org/jira/browse/HADOOP-1302) | Remove deprecated contrib/abacus code | Major | . | Doug Cutting | Enis Soztutar | +| [HADOOP-1274](https://issues.apache.org/jira/browse/HADOOP-1274) | Configuring different number of mappers and reducers per TaskTracker | Major | . | Koji Noguchi | Amareshwari Sriramadasu | +| [HADOOP-1210](https://issues.apache.org/jira/browse/HADOOP-1210) | Log counters in job history | Minor | . | Albert Chern | Owen O'Malley | +| [HADOOP-1185](https://issues.apache.org/jira/browse/HADOOP-1185) | dynamically change log levels | Major | util | dhruba borthakur | Tsz Wo Nicholas Sze | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2768](https://issues.apache.org/jira/browse/HADOOP-2768) | DFSIO write performance benchmark shows a regression | Blocker | . | Mukund Madhugiri | dhruba borthakur | +| [HADOOP-2755](https://issues.apache.org/jira/browse/HADOOP-2755) | dfs fsck extremely slow, dfs ls times out | Blocker | . | Christian Kunz | Tsz Wo Nicholas Sze | +| [HADOOP-2740](https://issues.apache.org/jira/browse/HADOOP-2740) | Modify HOD to work with changes mentioned in HADOOP-2404 | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-2734](https://issues.apache.org/jira/browse/HADOOP-2734) | docs link to lucene.apache.org | Major | documentation | Doug Cutting | Doug Cutting | +| [HADOOP-2732](https://issues.apache.org/jira/browse/HADOOP-2732) | ab{5[6-9],[6-9][6-9]}.gz should not be treated as an illegal glob | Blocker | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-2723](https://issues.apache.org/jira/browse/HADOOP-2723) | Hadoop 2367- Does not respect JobConf.getProfileEnabled() | Blocker | . | Clint Morgan | Amareshwari Sriramadasu | +| [HADOOP-2720](https://issues.apache.org/jira/browse/HADOOP-2720) | Update HOD in Hadoop 0.16 | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-2714](https://issues.apache.org/jira/browse/HADOOP-2714) | Unit test fails on Windows: rg.apache.hadoop.dfs.TestDecommission | Blocker | . | Mukund Madhugiri | dhruba borthakur | +| [HADOOP-2713](https://issues.apache.org/jira/browse/HADOOP-2713) | Unit test fails on Windows: org.apache.hadoop.dfs.TestDatanodeDeath | Blocker | . | Mukund Madhugiri | dhruba borthakur | +| [HADOOP-2691](https://issues.apache.org/jira/browse/HADOOP-2691) | Some junit tests fail with the exception: All datanodes are bad. Aborting... | Major | . | Hairong Kuang | dhruba borthakur | +| [HADOOP-2687](https://issues.apache.org/jira/browse/HADOOP-2687) | 1707 added errant INFO-level logging to DFSClient | Blocker | . | stack | stack | +| [HADOOP-2659](https://issues.apache.org/jira/browse/HADOOP-2659) | The commands in DFSAdmin should require admin privilege | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2652](https://issues.apache.org/jira/browse/HADOOP-2652) | Fix permission issues for HftpFileSystem | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2649](https://issues.apache.org/jira/browse/HADOOP-2649) | The ReplicationMonitor sleep period should be configurable | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2646](https://issues.apache.org/jira/browse/HADOOP-2646) | SortValidator broken with fully-qualified working directories | Blocker | test | Doug Cutting | Arun C Murthy | +| [HADOOP-2640](https://issues.apache.org/jira/browse/HADOOP-2640) | MultiFileSplitInputFormat always returns 1 split when avgLengthPerSplit \> Integer.MAX\_VALUE | Blocker | . | FrĂ©dĂ©ric Bertin | Enis Soztutar | +| [HADOOP-2639](https://issues.apache.org/jira/browse/HADOOP-2639) | Reducers stuck in shuffle | Blocker | . | Amareshwari Sriramadasu | Arun C Murthy | +| [HADOOP-2633](https://issues.apache.org/jira/browse/HADOOP-2633) | Revert change to fsck made as part of permissions implementation | Blocker | . | Robert Chansler | Tsz Wo Nicholas Sze | +| [HADOOP-2626](https://issues.apache.org/jira/browse/HADOOP-2626) | RawLocalFileStatus is badly handling URIs | Major | fs | FrĂ©dĂ©ric Bertin | Doug Cutting | +| [HADOOP-2620](https://issues.apache.org/jira/browse/HADOOP-2620) | 'bin/hadoop fs -help' does not list file permissions commands. | Trivial | fs | Raghu Angadi | Raghu Angadi | +| [HADOOP-2614](https://issues.apache.org/jira/browse/HADOOP-2614) | dfs web interfaces should run as a configurable user account | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2605](https://issues.apache.org/jira/browse/HADOOP-2605) | leading slash in mapred.task.tracker.report.bindAddress | Major | conf | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2601](https://issues.apache.org/jira/browse/HADOOP-2601) | TestNNThroughput should not use a fixed namenode port | Major | . | Hairong Kuang | Konstantin Shvachko | +| [HADOOP-2583](https://issues.apache.org/jira/browse/HADOOP-2583) | Potential Eclipse plug-in UI loop when editing location parameters | Minor | contrib/eclipse-plugin | Christophe Taton | Christophe Taton | +| [HADOOP-2582](https://issues.apache.org/jira/browse/HADOOP-2582) | hadoop dfs -copyToLocal creates zero byte files, when source file does not exists | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-2576](https://issues.apache.org/jira/browse/HADOOP-2576) | Namenode performance degradation over time | Blocker | . | Christian Kunz | Raghu Angadi | +| [HADOOP-2571](https://issues.apache.org/jira/browse/HADOOP-2571) | javac generates a warning in test/o.a.h.io.FileBench | Trivial | test | Chris Douglas | Chris Douglas | +| [HADOOP-2549](https://issues.apache.org/jira/browse/HADOOP-2549) | hdfs does not honor dfs.du.reserved setting | Critical | . | Joydeep Sen Sarma | Hairong Kuang | +| [HADOOP-2538](https://issues.apache.org/jira/browse/HADOOP-2538) | NPE in TaskLog.java | Trivial | . | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-2537](https://issues.apache.org/jira/browse/HADOOP-2537) | make build process compatible with Ant 1.7.0 | Major | build | Nigel Daley | Hrishikesh | +| [HADOOP-2535](https://issues.apache.org/jira/browse/HADOOP-2535) | Remove support for deprecated mapred.child.heap.size and indentation fix in TaskRunner.java | Minor | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-2523](https://issues.apache.org/jira/browse/HADOOP-2523) | Unit test fails on Windows: TestDFSShell.testFilePermissions | Blocker | . | Mukund Madhugiri | Raghu Angadi | +| [HADOOP-2512](https://issues.apache.org/jira/browse/HADOOP-2512) | error stream handling in Shell executor | Blocker | util | Raghu Angadi | Raghu Angadi | +| [HADOOP-2511](https://issues.apache.org/jira/browse/HADOOP-2511) | HADOOP-2344 introduced a javadoc warning | Major | documentation | Arun C Murthy | Arun C Murthy | +| [HADOOP-2509](https://issues.apache.org/jira/browse/HADOOP-2509) | Add rat target to build | Major | build | Nigel Daley | Hrishikesh | +| [HADOOP-2503](https://issues.apache.org/jira/browse/HADOOP-2503) | REST Insert / Select | Critical | . | Billy Pearson | Bryan Duxbury | +| [HADOOP-2494](https://issues.apache.org/jira/browse/HADOOP-2494) | Set +x on contrib/\*/bin/\* in packaged tar bundle | Major | scripts | stack | stack | +| [HADOOP-2492](https://issues.apache.org/jira/browse/HADOOP-2492) | ConcurrentModificationException in org.apache.hadoop.ipc.Server.Responder | Major | ipc | Devaraj Das | dhruba borthakur | +| [HADOOP-2481](https://issues.apache.org/jira/browse/HADOOP-2481) | NNBench should periodically report its progress | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-2476](https://issues.apache.org/jira/browse/HADOOP-2476) | Unit test fails on Windows: TestCopyFiles.testCopyFromLocalToLocal | Blocker | fs | Mukund Madhugiri | Raghu Angadi | +| [HADOOP-2459](https://issues.apache.org/jira/browse/HADOOP-2459) | Running 'ant docs tar' includes src/docs/build in the resulting tar file | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-2453](https://issues.apache.org/jira/browse/HADOOP-2453) | wordcount-simple example gives ParseException with examples configuration file | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2452](https://issues.apache.org/jira/browse/HADOOP-2452) | Eclipse plug-in build.xml issue | Trivial | build | Christophe Taton | Christophe Taton | +| [HADOOP-2446](https://issues.apache.org/jira/browse/HADOOP-2446) | TestHDFSServerPorts fails. | Major | test | Raghu Angadi | Nigel Daley | +| [HADOOP-2442](https://issues.apache.org/jira/browse/HADOOP-2442) | Unit test failed: org.apache.hadoop.fs.TestLocalFileSystemPermission.testLocalFSsetOwner | Critical | fs | Mukund Madhugiri | Raghu Angadi | +| [HADOOP-2434](https://issues.apache.org/jira/browse/HADOOP-2434) | MapFile.get on HDFS in TRUNK is WAY!!! slower than 0.15.x | Blocker | io | stack | stack | +| [HADOOP-2420](https://issues.apache.org/jira/browse/HADOOP-2420) | Use exit code to detect normal errors while excuting 'ls' in Local FS | Blocker | fs | Raghu Angadi | Raghu Angadi | +| [HADOOP-2404](https://issues.apache.org/jira/browse/HADOOP-2404) | HADOOP-2185 breaks compatibility with hadoop-0.15.0 | Blocker | conf | Arun C Murthy | Owen O'Malley | +| [HADOOP-2402](https://issues.apache.org/jira/browse/HADOOP-2402) | Lzo compression compresses each write from TextOutputFormat | Major | io, native | Chris Douglas | Chris Douglas | +| [HADOOP-2376](https://issues.apache.org/jira/browse/HADOOP-2376) | The sort example shouldn't override the number of maps | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2368](https://issues.apache.org/jira/browse/HADOOP-2368) | Lots of unit tests fail on Windows with exception: Login failed: CreateProcess | Blocker | . | Mukund Madhugiri | Tsz Wo Nicholas Sze | +| [HADOOP-2365](https://issues.apache.org/jira/browse/HADOOP-2365) | Result of HashFunction.hash() contains all identical values | Minor | . | Andrzej Bialecki | Jim Kellerman | +| [HADOOP-2363](https://issues.apache.org/jira/browse/HADOOP-2363) | Unit tests fail if there is another instance of Hadoop | Major | test | Raghu Angadi | Konstantin Shvachko | +| [HADOOP-2359](https://issues.apache.org/jira/browse/HADOOP-2359) | PendingReplicationMonitor thread received exception. java.lang.InterruptedException | Major | . | Owen O'Malley | dhruba borthakur | +| [HADOOP-2352](https://issues.apache.org/jira/browse/HADOOP-2352) | Remove AC\_LIB\_CHECK from src/native/configure.ac to ensure libhadoop.so doesn't have a dependency on libz.so/liblzo.so | Major | native | Arun C Murthy | Arun C Murthy | +| [HADOOP-2344](https://issues.apache.org/jira/browse/HADOOP-2344) | Free up the buffers (input and error) while executing a shell command before waiting for it to finish. | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-2323](https://issues.apache.org/jira/browse/HADOOP-2323) | JobTracker.close() prints stack traces for exceptions that are not errors | Minor | . | Jim Kellerman | Jim Kellerman | +| [HADOOP-2314](https://issues.apache.org/jira/browse/HADOOP-2314) | TestBlockReplacement occasionally get into an infinite loop | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-2313](https://issues.apache.org/jira/browse/HADOOP-2313) | build does not fail when libhdfs build fails | Minor | . | Nigel Daley | Nigel Daley | +| [HADOOP-2300](https://issues.apache.org/jira/browse/HADOOP-2300) | mapred.tasktracker.tasks.maximum is completely ignored | Blocker | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-2285](https://issues.apache.org/jira/browse/HADOOP-2285) | TextInputFormat is slow compared to reading files. | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2284](https://issues.apache.org/jira/browse/HADOOP-2284) | BasicTypeSorterBase.compare calls progress on each compare | Major | . | Owen O'Malley | Amar Kamat | +| [HADOOP-2275](https://issues.apache.org/jira/browse/HADOOP-2275) | Erroneous detection of corrupted file when namenode fails to allocate any datanodes for newly allocated block | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2272](https://issues.apache.org/jira/browse/HADOOP-2272) | findbugs currently fails due to hadoop-streaming having moved | Major | build | Adrian Woodhead | stack | +| [HADOOP-2271](https://issues.apache.org/jira/browse/HADOOP-2271) | chmod in ant package target fails | Major | build | Adrian Woodhead | Adrian Woodhead | +| [HADOOP-2256](https://issues.apache.org/jira/browse/HADOOP-2256) | TestBlockReplacement unit test failed. | Major | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-2248](https://issues.apache.org/jira/browse/HADOOP-2248) | Word count example is spending 24% of the time in incrCounter | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2247](https://issues.apache.org/jira/browse/HADOOP-2247) | Mappers fail easily due to repeated failures | Blocker | . | Srikanth Kakani | Amar Kamat | +| [HADOOP-2245](https://issues.apache.org/jira/browse/HADOOP-2245) | TestRecordMR and TestAggregates fail once in a while | Major | . | Devaraj Das | Adrian Woodhead | +| [HADOOP-2244](https://issues.apache.org/jira/browse/HADOOP-2244) | MapWritable.readFields needs to clear internal hash else instance accumulates entries forever | Major | io | stack | stack | +| [HADOOP-2228](https://issues.apache.org/jira/browse/HADOOP-2228) | Jobs fail because job.xml exists | Major | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-2220](https://issues.apache.org/jira/browse/HADOOP-2220) | Reduce tasks fail too easily because of repeated fetch failures | Blocker | . | Christian Kunz | Amar Kamat | +| [HADOOP-2216](https://issues.apache.org/jira/browse/HADOOP-2216) | Job UI doesnot show running tasks and complete tasks correctly. | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-2215](https://issues.apache.org/jira/browse/HADOOP-2215) | Change documentation in cluster\_setup.html and mapred\_tutorial.html post HADOOP-1274 | Major | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2212](https://issues.apache.org/jira/browse/HADOOP-2212) | java.lang.ArithmeticException: / by zero in ChecksumFileSystem.open | Critical | fs | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-2209](https://issues.apache.org/jira/browse/HADOOP-2209) | SecondaryNamenode process should exit if it encounters Runtime exceptions | Major | . | dhruba borthakur | | +| [HADOOP-2204](https://issues.apache.org/jira/browse/HADOOP-2204) | DFSTestUtil.waitReplication does not wait. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2189](https://issues.apache.org/jira/browse/HADOOP-2189) | Incrementing user counters should count as progress | Blocker | . | Owen O'Malley | Devaraj Das | +| [HADOOP-2151](https://issues.apache.org/jira/browse/HADOOP-2151) | FileSyste.globPaths does not validate the return list of Paths | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-2121](https://issues.apache.org/jira/browse/HADOOP-2121) | Unexpected IOException in DFSOutputStream.close() | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2112](https://issues.apache.org/jira/browse/HADOOP-2112) | TestMiniMRMapRedDebugScript fails due to a missing file | Blocker | . | Devaraj Das | Arun C Murthy | +| [HADOOP-2104](https://issues.apache.org/jira/browse/HADOOP-2104) | clover description attribute suppresses all other targets in -projecthelp | Trivial | build | Chris Douglas | Chris Douglas | +| [HADOOP-2100](https://issues.apache.org/jira/browse/HADOOP-2100) | hadoop-daemon.sh script fails if HADOOP\_PID\_DIR doesn't exist | Major | scripts | Arun C Murthy | Michael Bieniosek | +| [HADOOP-2098](https://issues.apache.org/jira/browse/HADOOP-2098) | File handles for log files are still open in case of jobs with 0 maps | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-2096](https://issues.apache.org/jira/browse/HADOOP-2096) | The file used to localize job.xml should be closed. | Minor | . | Amar Kamat | Amar Kamat | +| [HADOOP-2092](https://issues.apache.org/jira/browse/HADOOP-2092) | Pipes C++ task does not die even if the Java tasks die | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2089](https://issues.apache.org/jira/browse/HADOOP-2089) | Multiple caheArchive does not work in Hadoop streaming | Critical | . | Milind Bhandarkar | Lohit Vijayarenu | +| [HADOOP-2078](https://issues.apache.org/jira/browse/HADOOP-2078) | Name-node should be able to close empty files. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2072](https://issues.apache.org/jira/browse/HADOOP-2072) | RawLocalFileStatus is causing Path problems | Major | fs | Dennis Kubes | | +| [HADOOP-2071](https://issues.apache.org/jira/browse/HADOOP-2071) | StreamXmlRecordReader throws java.io.IOException: Mark/reset exception in hadoop 0.14 | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-1984](https://issues.apache.org/jira/browse/HADOOP-1984) | some reducer stuck at copy phase and progress extremely slowly | Critical | . | Runping Qi | Amar Kamat | +| [HADOOP-1952](https://issues.apache.org/jira/browse/HADOOP-1952) | Streaming does not handle invalid -inputformat (typo by users for example) | Minor | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-1898](https://issues.apache.org/jira/browse/HADOOP-1898) | locking for the ReflectionUtils.logThreadInfo is too conservative | Major | util | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-1642](https://issues.apache.org/jira/browse/HADOOP-1642) | Jobs using LocalJobRunner + JobControl fails | Critical | . | Johan Oskarsson | Doug Cutting | +| [HADOOP-1348](https://issues.apache.org/jira/browse/HADOOP-1348) | Configuration XML bug: comments inside values | Critical | conf | Eelco Lempsink | Rajagopal Natarajan | +| [HADOOP-1281](https://issues.apache.org/jira/browse/HADOOP-1281) | Speculative map tasks aren't getting killed although the TIP completed | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-1245](https://issues.apache.org/jira/browse/HADOOP-1245) | value for mapred.tasktracker.tasks.maximum taken from jobtracker, not tasktracker | Major | . | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-1195](https://issues.apache.org/jira/browse/HADOOP-1195) | NullPointerException in FSNamesystem due to getDatanode() return value is not checked | Major | . | Konstantin Shvachko | dhruba borthakur | +| [HADOOP-1130](https://issues.apache.org/jira/browse/HADOOP-1130) | Remove unused ClientFinalizer in DFSClient | Major | . | Philippe Gassmann | Chris Douglas | +| [HADOOP-290](https://issues.apache.org/jira/browse/HADOOP-290) | Fix Datanode transfer thread logging | Minor | . | Dennis Kubes | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2449](https://issues.apache.org/jira/browse/HADOOP-2449) | Restore the old NN Bench that was replaced by a MR NN Bench | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-2431](https://issues.apache.org/jira/browse/HADOOP-2431) | Test HDFS File Permissions | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-2406](https://issues.apache.org/jira/browse/HADOOP-2406) | Micro-benchmark to measure read/write times through InputFormats | Major | fs, test | Chris Douglas | Chris Douglas | +| [HADOOP-2369](https://issues.apache.org/jira/browse/HADOOP-2369) | Representative mix of jobs for large cluster throughput benchmarking | Major | test | Chris Douglas | Runping Qi | +| [HADOOP-2342](https://issues.apache.org/jira/browse/HADOOP-2342) | create a micro-benchmark for measure local-file versus hdfs read | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2000](https://issues.apache.org/jira/browse/HADOOP-2000) | Re-write NNBench to use MapReduce | Major | test | Mukund Madhugiri | Mukund Madhugiri | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/RELEASENOTES.0.16.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/RELEASENOTES.0.16.0.md new file mode 100644 index 00000000000..e0ee47922ea --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.0/RELEASENOTES.0.16.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.16.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/CHANGES.0.16.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/CHANGES.0.16.1.md new file mode 100644 index 00000000000..4c403be3d75 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/CHANGES.0.16.1.md @@ -0,0 +1,104 @@ + + +# Apache Hadoop Changelog + +## Release 0.16.1 - 2008-03-14 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2923](https://issues.apache.org/jira/browse/HADOOP-2923) | Check in missing files from HADOOP-2603 | Major | . | Owen O'Malley | Chris Douglas | +| [HADOOP-2735](https://issues.apache.org/jira/browse/HADOOP-2735) | Setting default tmp directory for java createTempFile (java.io.tmpdir) | Critical | . | Koji Noguchi | Amareshwari Sriramadasu | +| [HADOOP-2371](https://issues.apache.org/jira/browse/HADOOP-2371) | Candidate user guide for permissions feature of Hadoop DFS | Major | . | Robert Chansler | Robert Chansler | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2911](https://issues.apache.org/jira/browse/HADOOP-2911) | [HOD] Make the information printed by allocate and info commands less verbose and clearer | Major | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2861](https://issues.apache.org/jira/browse/HADOOP-2861) | [HOD] Improve the user interface for the HOD commands | Major | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2730](https://issues.apache.org/jira/browse/HADOOP-2730) | Update HOD documentation | Major | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2958](https://issues.apache.org/jira/browse/HADOOP-2958) | Test utility no longer works in trunk | Minor | test | Chris Douglas | Chris Douglas | +| [HADOOP-2931](https://issues.apache.org/jira/browse/HADOOP-2931) | exception in DFSClient.create: Stream closed | Major | . | Michael Bieniosek | Michael Bieniosek | +| [HADOOP-2925](https://issues.apache.org/jira/browse/HADOOP-2925) | [HOD] Create mapred system directory using a naming convention that will avoid clashes in multi-user shared cluster scenario. | Major | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-2915](https://issues.apache.org/jira/browse/HADOOP-2915) | mapred output files and directories should be created as the job submitter, not tasktracker or jobtracker | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2904](https://issues.apache.org/jira/browse/HADOOP-2904) | 3 minor fixes in the rpc metrics area. | Major | . | girish vaitheeswaran | dhruba borthakur | +| [HADOOP-2903](https://issues.apache.org/jira/browse/HADOOP-2903) | Data type mismatch exception raised from pushMetric | Major | metrics | girish vaitheeswaran | girish vaitheeswaran | +| [HADOOP-2894](https://issues.apache.org/jira/browse/HADOOP-2894) | task trackers can't survive a job tracker bounce | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2883](https://issues.apache.org/jira/browse/HADOOP-2883) | Extensive write failures | Blocker | . | Christian Kunz | dhruba borthakur | +| [HADOOP-2869](https://issues.apache.org/jira/browse/HADOOP-2869) | Deprecate and remove SequenceFile.setCompressionType | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-2847](https://issues.apache.org/jira/browse/HADOOP-2847) | [HOD] Idle cluster cleanup does not work if the JobTracker becomes unresponsive to RPC calls | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-2843](https://issues.apache.org/jira/browse/HADOOP-2843) | mapred.join access control is overly restrictive | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-2840](https://issues.apache.org/jira/browse/HADOOP-2840) | Gridmix test script fails to run java sort tests | Major | test | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-2814](https://issues.apache.org/jira/browse/HADOOP-2814) | NPE in datanode during TestDataTransferProtocol. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2813](https://issues.apache.org/jira/browse/HADOOP-2813) | Unit test fails on Linux: org.apache.hadoop.fs.TestDU.testDU | Blocker | fs | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2811](https://issues.apache.org/jira/browse/HADOOP-2811) | method Counters.makeCompactString() does not insert separator char ',' between the counters of different groups. | Critical | . | Runping Qi | Runping Qi | +| [HADOOP-2809](https://issues.apache.org/jira/browse/HADOOP-2809) | [HOD] Syslog configuration, syslog-address, does not work in HOD 0.4 | Critical | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2808](https://issues.apache.org/jira/browse/HADOOP-2808) | FileUtil::copy ignores "overwrite" formal | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-2789](https://issues.apache.org/jira/browse/HADOOP-2789) | Race condition in ipc.Server prevents responce being written back to client. | Critical | ipc | Clint Morgan | Raghu Angadi | +| [HADOOP-2788](https://issues.apache.org/jira/browse/HADOOP-2788) | chgrp missing from hadoop dfs options | Critical | . | Mukund Madhugiri | Raghu Angadi | +| [HADOOP-2787](https://issues.apache.org/jira/browse/HADOOP-2787) | The constant org.apache.hadoop.fs.permission.FsPermission.UMASK\_LABEL should be "dfs.umask", instead of "hadoop.dfs.umask" | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2785](https://issues.apache.org/jira/browse/HADOOP-2785) | Typo in peridioc block verification patch | Trivial | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2780](https://issues.apache.org/jira/browse/HADOOP-2780) | Socket receive buffer size on datanode too large | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2769](https://issues.apache.org/jira/browse/HADOOP-2769) | TestNNThroughputBenchmark should not used a fixed http port | Major | test | Owen O'Malley | Owen O'Malley | +| [HADOOP-2766](https://issues.apache.org/jira/browse/HADOOP-2766) | [HOD] No way to set HADOOP\_OPTS environment variable to the Hadoop daemons through HOD | Critical | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2756](https://issues.apache.org/jira/browse/HADOOP-2756) | NPE in DFSClient in hbase under load | Minor | . | stack | Raghu Angadi | +| [HADOOP-2754](https://issues.apache.org/jira/browse/HADOOP-2754) | Path filter for Local file system list .crc files | Major | . | Amareshwari Sriramadasu | Hairong Kuang | +| [HADOOP-2733](https://issues.apache.org/jira/browse/HADOOP-2733) | Compiler warnings in TestClusterMapReduceTestCase and TestJobStatusPersistency | Major | test | Konstantin Shvachko | Tsz Wo Nicholas Sze | +| [HADOOP-2725](https://issues.apache.org/jira/browse/HADOOP-2725) | Distcp truncates some files when copying | Critical | util | Murtaza A. Basrai | Tsz Wo Nicholas Sze | +| [HADOOP-2716](https://issues.apache.org/jira/browse/HADOOP-2716) | Balancer should require superuser privilege | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2683](https://issues.apache.org/jira/browse/HADOOP-2683) | Provide a way to specifiy login out side an RPC | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-2391](https://issues.apache.org/jira/browse/HADOOP-2391) | Speculative Execution race condition with output paths | Major | . | Dennis Kubes | Amareshwari Sriramadasu | +| [HADOOP-1188](https://issues.apache.org/jira/browse/HADOOP-1188) | processIOError() should update fstime file | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2852](https://issues.apache.org/jira/browse/HADOOP-2852) | Update gridmix to avoid artificially long tail | Major | test | Chris Douglas | Chris Douglas | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2918](https://issues.apache.org/jira/browse/HADOOP-2918) | Enhance log messages to better debug "No lease on file" message | Major | . | dhruba borthakur | dhruba borthakur | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/RELEASENOTES.0.16.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/RELEASENOTES.0.16.1.md new file mode 100644 index 00000000000..39257cc0b99 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.1/RELEASENOTES.0.16.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.16.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/CHANGES.0.16.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/CHANGES.0.16.2.md new file mode 100644 index 00000000000..1c528f4056a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/CHANGES.0.16.2.md @@ -0,0 +1,87 @@ + + +# Apache Hadoop Changelog + +## Release 0.16.2 - 2008-04-02 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3128](https://issues.apache.org/jira/browse/HADOOP-3128) | TestDFSPermission due to not throwing exception. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3108](https://issues.apache.org/jira/browse/HADOOP-3108) | NPE in FSDirectory.unprotectedSetPermission | Blocker | . | Koji Noguchi | Konstantin Shvachko | +| [HADOOP-3107](https://issues.apache.org/jira/browse/HADOOP-3107) | fsck failing with NPE | Blocker | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-3105](https://issues.apache.org/jira/browse/HADOOP-3105) | compile-core-test fails for branch 0.16 | Blocker | . | Amareshwari Sriramadasu | Alejandro Abdelnur | +| [HADOOP-3104](https://issues.apache.org/jira/browse/HADOOP-3104) | MultithreadMapRunner keeps consuming records even if trheads are not available | Critical | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-3103](https://issues.apache.org/jira/browse/HADOOP-3103) | [HOD] Hadoop.tmp.dir should not be set to cluster directory | Blocker | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3098](https://issues.apache.org/jira/browse/HADOOP-3098) | dfs -chown does not like "\_" underscore in user name | Blocker | fs | Koji Noguchi | Raghu Angadi | +| [HADOOP-3084](https://issues.apache.org/jira/browse/HADOOP-3084) | distcp fails for files with zero length | Blocker | util | Mukund Madhugiri | Chris Douglas | +| [HADOOP-3070](https://issues.apache.org/jira/browse/HADOOP-3070) | Trash not being expunged, Trash Emptier thread gone by NPE | Blocker | fs | Koji Noguchi | Koji Noguchi | +| [HADOOP-3056](https://issues.apache.org/jira/browse/HADOOP-3056) | distcp seems to be broken in 0.16.1 | Blocker | util | Christian Kunz | Chris Douglas | +| [HADOOP-3049](https://issues.apache.org/jira/browse/HADOOP-3049) | MultithreadedMapRunner eats RuntimeExceptions | Blocker | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-3042](https://issues.apache.org/jira/browse/HADOOP-3042) | Update the Javadoc in JobConf.getOutputPath to reflect the actual temporary path | Major | documentation | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-3039](https://issues.apache.org/jira/browse/HADOOP-3039) | Runtime exceptions not killing job | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3033](https://issues.apache.org/jira/browse/HADOOP-3033) | Datanode fails write to DFS file with exception message "Trying to change block file offset" | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3027](https://issues.apache.org/jira/browse/HADOOP-3027) | JobTracker shuts down during initialization if the NameNode is down | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3011](https://issues.apache.org/jira/browse/HADOOP-3011) | Distcp deleting target directory | Blocker | util | Koji Noguchi | Chris Douglas | +| [HADOOP-3007](https://issues.apache.org/jira/browse/HADOOP-3007) | DataNode pipelining : failure on mirror results in failure on upstream datanode | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3003](https://issues.apache.org/jira/browse/HADOOP-3003) | FileSystem cache key should be updated after a FileSystem object is created | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2985](https://issues.apache.org/jira/browse/HADOOP-2985) | LocalJobRunner gets NullPointerException if there is no output directory | Critical | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-2978](https://issues.apache.org/jira/browse/HADOOP-2978) | JobHistory log format for COUNTER is ambigurous | Major | . | Runping Qi | Runping Qi | +| [HADOOP-2944](https://issues.apache.org/jira/browse/HADOOP-2944) | redesigned plugin has missing functionality | Major | contrib/eclipse-plugin | Chris Dyer | Christophe Taton | +| [HADOOP-2833](https://issues.apache.org/jira/browse/HADOOP-2833) | JobClient.submitJob(...) should not use "Dr Who" as a default username | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3111](https://issues.apache.org/jira/browse/HADOOP-3111) | Remove HBase from Hadoop contrib | Major | . | Jim Kellerman | Jim Kellerman | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/RELEASENOTES.0.16.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/RELEASENOTES.0.16.2.md new file mode 100644 index 00000000000..2ba4ee46c9d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.2/RELEASENOTES.0.16.2.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.16.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3098](https://issues.apache.org/jira/browse/HADOOP-3098) | *Blocker* | **dfs -chown does not like "\_" underscore in user name** + +chown and chgrp commands allow more flexible names compared to 0.16. See 'fs -help chown'. Inform Users : YES. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/CHANGES.0.16.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/CHANGES.0.16.3.md new file mode 100644 index 00000000000..955623bccdf --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/CHANGES.0.16.3.md @@ -0,0 +1,71 @@ + + +# Apache Hadoop Changelog + +## Release 0.16.3 - 2008-04-16 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3195](https://issues.apache.org/jira/browse/HADOOP-3195) | TestFileSystem fails randomly | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3182](https://issues.apache.org/jira/browse/HADOOP-3182) | JobClient creates submitJobDir with SYSTEM\_DIR\_PERMISSION ( rwx-wx-wx) | Blocker | . | Lohit Vijayarenu | Tsz Wo Nicholas Sze | +| [HADOOP-3159](https://issues.apache.org/jira/browse/HADOOP-3159) | FileSystem cache keep overwriting cached value | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3154](https://issues.apache.org/jira/browse/HADOOP-3154) | Job successful but dropping records (when disk full) | Blocker | . | Koji Noguchi | Devaraj Das | +| [HADOOP-3139](https://issues.apache.org/jira/browse/HADOOP-3139) | DistributedFileSystem.close() deadlock and FileSystem.closeAll() warning | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3069](https://issues.apache.org/jira/browse/HADOOP-3069) | A failure on SecondaryNameNode truncates the primary NameNode image. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3010](https://issues.apache.org/jira/browse/HADOOP-3010) | ConcurrentModificationException from org.apache.hadoop.ipc.Server$Responder in JobTracker | Major | ipc | Amar Kamat | Raghu Angadi | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/RELEASENOTES.0.16.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/RELEASENOTES.0.16.3.md new file mode 100644 index 00000000000..ae7811035d5 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.3/RELEASENOTES.0.16.3.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.16.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3182](https://issues.apache.org/jira/browse/HADOOP-3182) | *Blocker* | **JobClient creates submitJobDir with SYSTEM\_DIR\_PERMISSION ( rwx-wx-wx)** + +Changed \ from 733 to 777, so that a shared JobTracker can be started by a non-superuser account. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/CHANGES.0.16.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/CHANGES.0.16.4.md new file mode 100644 index 00000000000..ae027ce8b0b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/CHANGES.0.16.4.md @@ -0,0 +1,68 @@ + + +# Apache Hadoop Changelog + +## Release 0.16.4 - 2008-05-05 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3304](https://issues.apache.org/jira/browse/HADOOP-3304) | [HOD] logcondense fails if DFS has files that are not log files, but match a certain pattern | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3294](https://issues.apache.org/jira/browse/HADOOP-3294) | distcp leaves empty blocks afte successful execution | Blocker | util | Christian Kunz | Tsz Wo Nicholas Sze | +| [HADOOP-3186](https://issues.apache.org/jira/browse/HADOOP-3186) | Incorrect permission checking on mv | Blocker | . | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-3138](https://issues.apache.org/jira/browse/HADOOP-3138) | distcp fail copying to /user/\/\ (with permission on) | Blocker | . | Koji Noguchi | Raghu Angadi | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/RELEASENOTES.0.16.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/RELEASENOTES.0.16.4.md new file mode 100644 index 00000000000..fb5fc834b9d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.16.4/RELEASENOTES.0.16.4.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.16.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3304](https://issues.apache.org/jira/browse/HADOOP-3304) | *Blocker* | **[HOD] logcondense fails if DFS has files that are not log files, but match a certain pattern** + +Fixed the regular expression used by the logcondense utility to generate the list of log files to purge from HDFS. Also, improved error handling, so a single invalid log file does not fail the purge process. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/CHANGES.0.17.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/CHANGES.0.17.0.md new file mode 100644 index 00000000000..bf0fd321951 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/CHANGES.0.17.0.md @@ -0,0 +1,265 @@ + + +# Apache Hadoop Changelog + +## Release 0.17.0 - 2008-05-20 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3280](https://issues.apache.org/jira/browse/HADOOP-3280) | virtual address space limits break streaming apps | Blocker | . | Rick Cox | Arun C Murthy | +| [HADOOP-3266](https://issues.apache.org/jira/browse/HADOOP-3266) | Remove HOD changes from CHANGES.txt, as they are now inside src/contrib/hod | Major | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3239](https://issues.apache.org/jira/browse/HADOOP-3239) | exists() calls logs FileNotFoundException in namenode log | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3137](https://issues.apache.org/jira/browse/HADOOP-3137) | [HOD] Update hod version number | Major | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3091](https://issues.apache.org/jira/browse/HADOOP-3091) | hadoop dfs -put should support multiple src | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3060](https://issues.apache.org/jira/browse/HADOOP-3060) | MiniMRCluster is ignoring parameter taskTrackerFirst | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2873](https://issues.apache.org/jira/browse/HADOOP-2873) | Namenode fails to re-start after cluster shutdown - DFSClient: Could not obtain blocks even all datanodes were up & live | Major | . | AndrĂ© Martin | dhruba borthakur | +| [HADOOP-2854](https://issues.apache.org/jira/browse/HADOOP-2854) | Remove the deprecated ipc.Server.getUserInfo() | Blocker | . | Tsz Wo Nicholas Sze | Lohit Vijayarenu | +| [HADOOP-2839](https://issues.apache.org/jira/browse/HADOOP-2839) | Remove deprecated methods in FileSystem | Blocker | fs | Hairong Kuang | Lohit Vijayarenu | +| [HADOOP-2831](https://issues.apache.org/jira/browse/HADOOP-2831) | Remove the deprecated INode.getAbsoluteName() | Blocker | . | Tsz Wo Nicholas Sze | Lohit Vijayarenu | +| [HADOOP-2828](https://issues.apache.org/jira/browse/HADOOP-2828) | Remove deprecated methods in Configuration.java | Major | conf | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2826](https://issues.apache.org/jira/browse/HADOOP-2826) | FileSplit.getFile(), LineRecordReader. readLine() need to be removed | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2825](https://issues.apache.org/jira/browse/HADOOP-2825) | MapOutputLocation.getFile() needs to be removed | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2824](https://issues.apache.org/jira/browse/HADOOP-2824) | One of MiniMRCluster constructors needs tobe removed | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2823](https://issues.apache.org/jira/browse/HADOOP-2823) | SimpleCharStream.getColumn(), getLine() methods to be removed. | Major | record | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2822](https://issues.apache.org/jira/browse/HADOOP-2822) | Remove deprecated classes in mapred | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2821](https://issues.apache.org/jira/browse/HADOOP-2821) | Remove deprecated classes in util | Major | util | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2820](https://issues.apache.org/jira/browse/HADOOP-2820) | Remove deprecated classes in streaming | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2819](https://issues.apache.org/jira/browse/HADOOP-2819) | Remove deprecated methods in JobConf() | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2818](https://issues.apache.org/jira/browse/HADOOP-2818) | Remove deprecated Counters.getDisplayName(), getCounterNames(), getCounter(String counterName) | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2765](https://issues.apache.org/jira/browse/HADOOP-2765) | setting memory limits for tasks | Major | . | Joydeep Sen Sarma | Amareshwari Sriramadasu | +| [HADOOP-2634](https://issues.apache.org/jira/browse/HADOOP-2634) | Deprecate exists() and isDir() to simplify ClientProtocol. | Blocker | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-2563](https://issues.apache.org/jira/browse/HADOOP-2563) | Remove deprecated FileSystem#listPaths() | Blocker | fs | Doug Cutting | Lohit Vijayarenu | +| [HADOOP-2470](https://issues.apache.org/jira/browse/HADOOP-2470) | Open and isDir should be removed from ClientProtocol | Major | . | Hairong Kuang | Tsz Wo Nicholas Sze | +| [HADOOP-2410](https://issues.apache.org/jira/browse/HADOOP-2410) | Make EC2 cluster nodes more independent of each other | Major | contrib/cloud | Tom White | Chris K Wensel | +| [HADOOP-2399](https://issues.apache.org/jira/browse/HADOOP-2399) | Input key and value to combiner and reducer should be reused | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2345](https://issues.apache.org/jira/browse/HADOOP-2345) | new transactions to support HDFS Appends | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2219](https://issues.apache.org/jira/browse/HADOOP-2219) | du like command to count number of files under a given directory | Major | . | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-2192](https://issues.apache.org/jira/browse/HADOOP-2192) | dfs mv command differs from POSIX standards | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2178](https://issues.apache.org/jira/browse/HADOOP-2178) | Job history on HDFS | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2116](https://issues.apache.org/jira/browse/HADOOP-2116) | Job.local.dir to be exposed to tasks | Major | . | Milind Bhandarkar | Amareshwari Sriramadasu | +| [HADOOP-2027](https://issues.apache.org/jira/browse/HADOOP-2027) | FileSystem should provide byte ranges for file locations | Major | fs | Owen O'Malley | Lohit Vijayarenu | +| [HADOOP-1986](https://issues.apache.org/jira/browse/HADOOP-1986) | Add support for a general serialization mechanism for Map Reduce | Major | . | Tom White | Tom White | +| [HADOOP-1985](https://issues.apache.org/jira/browse/HADOOP-1985) | Abstract node to switch mapping into a topology service class used by namenode and jobtracker | Major | . | eric baldeschwieler | Devaraj Das | +| [HADOOP-771](https://issues.apache.org/jira/browse/HADOOP-771) | Namenode should return error when trying to delete non-empty directory | Major | . | Milind Bhandarkar | Mahadev konar | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3152](https://issues.apache.org/jira/browse/HADOOP-3152) | Make index interval configuable when using MapFileOutputFormat for map-reduce job | Minor | io | Rong-En Fan | Doug Cutting | +| [HADOOP-3048](https://issues.apache.org/jira/browse/HADOOP-3048) | Stringifier | Blocker | io | Enis Soztutar | Enis Soztutar | +| [HADOOP-3001](https://issues.apache.org/jira/browse/HADOOP-3001) | FileSystems should track how many bytes are read and written | Blocker | fs | Owen O'Malley | Owen O'Malley | +| [HADOOP-2951](https://issues.apache.org/jira/browse/HADOOP-2951) | contrib package provides a utility to build or update an index +A contrib package to update an index using Map/Reduce | Major | . | Ning Li | Doug Cutting | +| [HADOOP-2906](https://issues.apache.org/jira/browse/HADOOP-2906) | output format classes that can write to different files depending on keys and/or config variable | Major | . | Runping Qi | Runping Qi | +| [HADOOP-2657](https://issues.apache.org/jira/browse/HADOOP-2657) | Enhancements to DFSClient to support flushing data at any point in time | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2063](https://issues.apache.org/jira/browse/HADOOP-2063) | Command to pull corrupted files | Blocker | fs | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-2055](https://issues.apache.org/jira/browse/HADOOP-2055) | JobConf should have a setInputPathFilter method | Minor | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-1593](https://issues.apache.org/jira/browse/HADOOP-1593) | FsShell should work with paths in non-default FileSystem | Major | fs | Doug Cutting | Mahadev konar | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3174](https://issues.apache.org/jira/browse/HADOOP-3174) | Improve documentation and supply an example for MultiFileInputFormat | Major | documentation | Enis Soztutar | Enis Soztutar | +| [HADOOP-3143](https://issues.apache.org/jira/browse/HADOOP-3143) | Decrease the number of slaves in TestMiniMRDFSSort to 3. | Major | test | Owen O'Malley | Nigel Daley | +| [HADOOP-3123](https://issues.apache.org/jira/browse/HADOOP-3123) | Build native libraries on Solaris | Major | build | Tom White | Tom White | +| [HADOOP-3099](https://issues.apache.org/jira/browse/HADOOP-3099) | Need new options in distcp for preserving ower, group and permission | Blocker | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3092](https://issues.apache.org/jira/browse/HADOOP-3092) | Show counter values from "job -status" command | Major | scripts | Tom White | Tom White | +| [HADOOP-3046](https://issues.apache.org/jira/browse/HADOOP-3046) | Text and BytesWritable's raw comparators should use the lengths provided instead of rebuilding them from scratch using readInt | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2996](https://issues.apache.org/jira/browse/HADOOP-2996) | StreamUtils abuses StringBuffers | Trivial | . | Dave Brosius | Dave Brosius | +| [HADOOP-2994](https://issues.apache.org/jira/browse/HADOOP-2994) | DFSClient calls toString on strings. | Trivial | . | Dave Brosius | Dave Brosius | +| [HADOOP-2993](https://issues.apache.org/jira/browse/HADOOP-2993) | Specify which JAVA\_HOME should be set | Major | documentation | Jason Rennie | Arun C Murthy | +| [HADOOP-2947](https://issues.apache.org/jira/browse/HADOOP-2947) | [HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2939](https://issues.apache.org/jira/browse/HADOOP-2939) | Make the Hudson patch process an executable ant target | Minor | test | Nigel Daley | Nigel Daley | +| [HADOOP-2919](https://issues.apache.org/jira/browse/HADOOP-2919) | Create fewer copies of buffer data during sort/spill | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-2902](https://issues.apache.org/jira/browse/HADOOP-2902) | replace accesss of "fs.default.name" with FileSystem accessor methods | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-2895](https://issues.apache.org/jira/browse/HADOOP-2895) | String for configuring profiling should be customizable | Major | . | Martin Traverso | Martin Traverso | +| [HADOOP-2888](https://issues.apache.org/jira/browse/HADOOP-2888) | Enhancements to gridmix scripts | Major | test | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-2886](https://issues.apache.org/jira/browse/HADOOP-2886) | Track individual RPC metrics. | Major | metrics | girish vaitheeswaran | dhruba borthakur | +| [HADOOP-2841](https://issues.apache.org/jira/browse/HADOOP-2841) | Dfs methods should not throw RemoteException | Major | . | Hairong Kuang | Konstantin Shvachko | +| [HADOOP-2810](https://issues.apache.org/jira/browse/HADOOP-2810) | Need new Hadoop Core logo | Minor | documentation | Nigel Daley | Nigel Daley | +| [HADOOP-2804](https://issues.apache.org/jira/browse/HADOOP-2804) | Formatable changes log as html | Minor | documentation | Nigel Daley | Nigel Daley | +| [HADOOP-2796](https://issues.apache.org/jira/browse/HADOOP-2796) | For script option hod should exit with distinguishable exit codes for script code and hod exit code. | Major | contrib/hod | Karam Singh | Hemanth Yamijala | +| [HADOOP-2758](https://issues.apache.org/jira/browse/HADOOP-2758) | Reduce memory copies when data is read from DFS | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2690](https://issues.apache.org/jira/browse/HADOOP-2690) | Adding support into build.xml to build a special hadoop jar file that has the MiniDFSCluster and MiniMRCluster classes among others necessary for building and running the unit tests of Pig on the local mini cluster | Major | build | Xu Zhang | Enis Soztutar | +| [HADOOP-2559](https://issues.apache.org/jira/browse/HADOOP-2559) | DFS should place one replica per rack | Major | . | Runping Qi | Lohit Vijayarenu | +| [HADOOP-2555](https://issues.apache.org/jira/browse/HADOOP-2555) | Refactor the HTable#get and HTable#getRow methods to avoid repetition of retry-on-failure logic | Minor | . | Peter Dolan | Bryan Duxbury | +| [HADOOP-2551](https://issues.apache.org/jira/browse/HADOOP-2551) | hadoop-env.sh needs finer granularity | Blocker | scripts | Allen Wittenauer | Raghu Angadi | +| [HADOOP-2473](https://issues.apache.org/jira/browse/HADOOP-2473) | EC2 termination script should support termination by group | Major | contrib/cloud | Tom White | Chris K Wensel | +| [HADOOP-2423](https://issues.apache.org/jira/browse/HADOOP-2423) | The codes in FSDirectory.mkdirs(...) is inefficient. | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2239](https://issues.apache.org/jira/browse/HADOOP-2239) | Security: Need to be able to encrypt Hadoop socket connections | Major | . | Allen Wittenauer | Chris Douglas | +| [HADOOP-2148](https://issues.apache.org/jira/browse/HADOOP-2148) | Inefficient FSDataset.getBlockFile() | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2057](https://issues.apache.org/jira/browse/HADOOP-2057) | streaming should optionally treat a non-zero exit status of a child process as a failed task | Major | . | Rick Cox | Rick Cox | +| [HADOOP-1677](https://issues.apache.org/jira/browse/HADOOP-1677) | improve semantics of the hadoop dfs command | Minor | . | Nigel Daley | Mahadev konar | +| [HADOOP-1622](https://issues.apache.org/jira/browse/HADOOP-1622) | Hadoop should provide a way to allow the user to specify jar file(s) the user job depends on | Major | . | Runping Qi | Mahadev konar | +| [HADOOP-1228](https://issues.apache.org/jira/browse/HADOOP-1228) | Eclipse project files | Minor | build | Albert Strasheim | Tom White | +| [HADOOP-910](https://issues.apache.org/jira/browse/HADOOP-910) | Reduces can do merges for the on-disk map output files in parallel with their copying | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-730](https://issues.apache.org/jira/browse/HADOOP-730) | Local file system uses copy to implement rename | Major | fs | Owen O'Malley | Chris Douglas | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3701](https://issues.apache.org/jira/browse/HADOOP-3701) | Too many trash sockets and trash pipes opened | Major | . | He Yongqiang | | +| [HADOOP-3382](https://issues.apache.org/jira/browse/HADOOP-3382) | Memory leak when files are not cleanly closed | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3372](https://issues.apache.org/jira/browse/HADOOP-3372) | TestUlimit fails on LINUX | Blocker | . | Lohit Vijayarenu | Arun C Murthy | +| [HADOOP-3322](https://issues.apache.org/jira/browse/HADOOP-3322) | Hadoop rpc metrics do not get pushed to the MetricsRecord | Blocker | metrics | girish vaitheeswaran | girish vaitheeswaran | +| [HADOOP-3286](https://issues.apache.org/jira/browse/HADOOP-3286) | Gridmix jobs' output dir names may collide | Major | test | Runping Qi | Runping Qi | +| [HADOOP-3285](https://issues.apache.org/jira/browse/HADOOP-3285) | map tasks with node local splits do not always read from local nodes | Blocker | . | Runping Qi | Owen O'Malley | +| [HADOOP-3279](https://issues.apache.org/jira/browse/HADOOP-3279) | TaskTracker should check for SUCCEEDED task status in addition to COMMIT\_PENDING status when it fails maps due to lost map outputs | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-3263](https://issues.apache.org/jira/browse/HADOOP-3263) | job history browser throws exception if job name or user name is null. | Blocker | . | Amareshwari Sriramadasu | Arun C Murthy | +| [HADOOP-3256](https://issues.apache.org/jira/browse/HADOOP-3256) | JobHistory file on HDFS should not use the 'job name' | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3251](https://issues.apache.org/jira/browse/HADOOP-3251) | WARN message on command line when a hadoop jar command is executed | Blocker | . | Mukund Madhugiri | Arun C Murthy | +| [HADOOP-3247](https://issues.apache.org/jira/browse/HADOOP-3247) | gridmix scripts have a few bugs | Major | test | Runping Qi | Runping Qi | +| [HADOOP-3242](https://issues.apache.org/jira/browse/HADOOP-3242) | SequenceFileAsBinaryRecordReader seems always to read from the start of a file, not the start of the split. | Major | . | Runping Qi | Chris Douglas | +| [HADOOP-3237](https://issues.apache.org/jira/browse/HADOOP-3237) | Unit test failed on windows: TestDFSShell.testErrOutPut | Blocker | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-3229](https://issues.apache.org/jira/browse/HADOOP-3229) | Map OutputCollector does not report progress on writes | Major | . | Alejandro Abdelnur | Doug Cutting | +| [HADOOP-3225](https://issues.apache.org/jira/browse/HADOOP-3225) | FsShell showing null instead of a error message | Blocker | . | Tsz Wo Nicholas Sze | Mahadev konar | +| [HADOOP-3224](https://issues.apache.org/jira/browse/HADOOP-3224) | hadoop dfs -du /dirPath does not work with hadoop-0.17 branch | Blocker | . | Runping Qi | Lohit Vijayarenu | +| [HADOOP-3223](https://issues.apache.org/jira/browse/HADOOP-3223) | Hadoop dfs -help for permissions contains a typo | Blocker | . | Milind Bhandarkar | Raghu Angadi | +| [HADOOP-3220](https://issues.apache.org/jira/browse/HADOOP-3220) | Safemode log message need to be corrected. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3208](https://issues.apache.org/jira/browse/HADOOP-3208) | WritableDeserializer does not pass the Configuration to deserialized Writables | Blocker | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-3204](https://issues.apache.org/jira/browse/HADOOP-3204) | LocalFSMerger needs to catch throwable | Blocker | . | Koji Noguchi | Amar Kamat | +| [HADOOP-3183](https://issues.apache.org/jira/browse/HADOOP-3183) | Unit test fails on Windows: TestJobShell.testJobShell | Blocker | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-3178](https://issues.apache.org/jira/browse/HADOOP-3178) | gridmix scripts for small and medium jobs need to be changed to handle input paths differently | Blocker | test | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-3175](https://issues.apache.org/jira/browse/HADOOP-3175) | "-get file -" does not work | Blocker | fs | Raghu Angadi | Edward J. Yoon | +| [HADOOP-3168](https://issues.apache.org/jira/browse/HADOOP-3168) | reduce amount of logging in hadoop streaming | Major | . | Joydeep Sen Sarma | Zheng Shao | +| [HADOOP-3166](https://issues.apache.org/jira/browse/HADOOP-3166) | SpillThread throws ArrayIndexOutOfBoundsException, which is ignored by MapTask | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3165](https://issues.apache.org/jira/browse/HADOOP-3165) | FsShell no longer accepts stdin as a source for -put/-copyFromLocal | Blocker | . | Chris Douglas | Lohit Vijayarenu | +| [HADOOP-3162](https://issues.apache.org/jira/browse/HADOOP-3162) | Map/reduce stops working with comma separated input paths | Blocker | . | Runping Qi | Amareshwari Sriramadasu | +| [HADOOP-3161](https://issues.apache.org/jira/browse/HADOOP-3161) | TestFileAppend fails on Mac since HADOOP-2655 was committed | Minor | test | Nigel Daley | Nigel Daley | +| [HADOOP-3157](https://issues.apache.org/jira/browse/HADOOP-3157) | TestMiniMRLocalFS fails in trunk on Windows | Blocker | test | Lohit Vijayarenu | Doug Cutting | +| [HADOOP-3153](https://issues.apache.org/jira/browse/HADOOP-3153) | [HOD] Hod should deallocate cluster if there's a problem in writing information to the state file | Major | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3146](https://issues.apache.org/jira/browse/HADOOP-3146) | DFSOutputStream.flush should be renamed as DFSOutputStream.fsync | Blocker | . | Runping Qi | dhruba borthakur | +| [HADOOP-3140](https://issues.apache.org/jira/browse/HADOOP-3140) | JobTracker should not try to promote a (map) task if it does not write to DFS at all | Major | . | Runping Qi | Amar Kamat | +| [HADOOP-3124](https://issues.apache.org/jira/browse/HADOOP-3124) | DFS data node should not use hard coded 10 minutes as write timeout. | Major | . | Runping Qi | Raghu Angadi | +| [HADOOP-3118](https://issues.apache.org/jira/browse/HADOOP-3118) | Namenode NPE while loading fsimage after a cluster upgrade from older disk format | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3114](https://issues.apache.org/jira/browse/HADOOP-3114) | TestDFSShell fails on Windows. | Major | fs | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-3106](https://issues.apache.org/jira/browse/HADOOP-3106) | Update documentation in mapred\_tutorial to add Debugging | Major | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3094](https://issues.apache.org/jira/browse/HADOOP-3094) | BytesWritable.toString prints bytes above 0x80 as FFFFFF80 | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-3093](https://issues.apache.org/jira/browse/HADOOP-3093) | ma/reduce throws the following exception if "io.serializations" is not set: | Major | . | Runping Qi | Amareshwari Sriramadasu | +| [HADOOP-3089](https://issues.apache.org/jira/browse/HADOOP-3089) | streaming should accept stderr from task before first key arrives | Major | . | Rick Cox | Rick Cox | +| [HADOOP-3087](https://issues.apache.org/jira/browse/HADOOP-3087) | JobInfo session object is not refreshed in loadHistory.jsp if same job is accessed again. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3086](https://issues.apache.org/jira/browse/HADOOP-3086) | Test case was missed in commit of HADOOP-3040 | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3083](https://issues.apache.org/jira/browse/HADOOP-3083) | Remove lease when file is renamed | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3080](https://issues.apache.org/jira/browse/HADOOP-3080) | Remove flush calls from JobHistory | Blocker | . | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-3073](https://issues.apache.org/jira/browse/HADOOP-3073) | SocketOutputStream.close() should close the channel. | Blocker | ipc | Raghu Angadi | Raghu Angadi | +| [HADOOP-3067](https://issues.apache.org/jira/browse/HADOOP-3067) | DFSInputStream 'pread' does not close its sockets | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3066](https://issues.apache.org/jira/browse/HADOOP-3066) | Should not require superuser privilege to query if hdfs is in safe mode | Major | . | Jim Kellerman | Jim Kellerman | +| [HADOOP-3065](https://issues.apache.org/jira/browse/HADOOP-3065) | Namenode does not process block report if the rack-location script is not provided on namenode | Blocker | . | dhruba borthakur | Devaraj Das | +| [HADOOP-3064](https://issues.apache.org/jira/browse/HADOOP-3064) | Exception with file globbing closures | Major | . | Tom White | Hairong Kuang | +| [HADOOP-3050](https://issues.apache.org/jira/browse/HADOOP-3050) | Cluster fall into infinite loop trying to replicate a block to a target that aready has this replica. | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-3044](https://issues.apache.org/jira/browse/HADOOP-3044) | NNBench does not use the right configuration for the mapper | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-3041](https://issues.apache.org/jira/browse/HADOOP-3041) | Within a task, the value ofJobConf.getOutputPath() method is modified | Blocker | . | Alejandro Abdelnur | Amareshwari Sriramadasu | +| [HADOOP-3040](https://issues.apache.org/jira/browse/HADOOP-3040) | Streaming should assume an empty key if the first character on a line is the seperator (stream.map.output.field.separator, by default, tab) | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3036](https://issues.apache.org/jira/browse/HADOOP-3036) | Fix findBugs warnings in UpgradeUtilities. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3031](https://issues.apache.org/jira/browse/HADOOP-3031) | Remove compiler warnings for ant test | Minor | . | Amareshwari Sriramadasu | Chris Douglas | +| [HADOOP-3030](https://issues.apache.org/jira/browse/HADOOP-3030) | InMemoryFileSystem.reserveSpaceWithChecksum does not look at failures while reserving space for the file in question | Major | fs | Devaraj Das | Devaraj Das | +| [HADOOP-3029](https://issues.apache.org/jira/browse/HADOOP-3029) | Misleading log message "firstbadlink" printed by datanodes | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3025](https://issues.apache.org/jira/browse/HADOOP-3025) | ChecksumFileSystem needs to support the new delete method | Blocker | fs | Devaraj Das | Mahadev konar | +| [HADOOP-3018](https://issues.apache.org/jira/browse/HADOOP-3018) | Eclipse plugin fails to compile due to missing RPC.stopClient() method | Blocker | contrib/eclipse-plugin | Tom White | Christophe Taton | +| [HADOOP-3012](https://issues.apache.org/jira/browse/HADOOP-3012) | dfs -mv file to user home directory fails silently if the user home directory does not exist | Blocker | fs | Mukund Madhugiri | Mahadev konar | +| [HADOOP-3009](https://issues.apache.org/jira/browse/HADOOP-3009) | TestFileCreation fails while restarting cluster | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3008](https://issues.apache.org/jira/browse/HADOOP-3008) | SocketIOWithTimeout does not handle thread interruption | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3006](https://issues.apache.org/jira/browse/HADOOP-3006) | DataNode sends wrong length in header while pipelining. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2995](https://issues.apache.org/jira/browse/HADOOP-2995) | StreamBaseRecordReader's getProgress returns just 0 or 1 | Minor | . | Dave Brosius | Dave Brosius | +| [HADOOP-2992](https://issues.apache.org/jira/browse/HADOOP-2992) | Sequential distributed upgrades. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2983](https://issues.apache.org/jira/browse/HADOOP-2983) | [HOD] local\_fqdn() returns None when gethostbyname\_ex doesnt return any FQDNs. | Blocker | contrib/hod | Craig Macdonald | Hemanth Yamijala | +| [HADOOP-2982](https://issues.apache.org/jira/browse/HADOOP-2982) | [HOD] checknodes should look for free nodes without the jobs attribute | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-2976](https://issues.apache.org/jira/browse/HADOOP-2976) | Blocks staying underreplicated (for unclosed file) | Minor | . | Koji Noguchi | dhruba borthakur | +| [HADOOP-2974](https://issues.apache.org/jira/browse/HADOOP-2974) | ipc unit tests fail due to connection errors | Blocker | ipc | Mukund Madhugiri | Raghu Angadi | +| [HADOOP-2973](https://issues.apache.org/jira/browse/HADOOP-2973) | Unit test fails on Windows: org.apache.hadoop.dfs.TestLocalDFS.testWorkingDirectory | Blocker | . | Mukund Madhugiri | Tsz Wo Nicholas Sze | +| [HADOOP-2972](https://issues.apache.org/jira/browse/HADOOP-2972) | org.apache.hadoop.dfs.TestDFSShell.testErrOutPut fails on Windows with NullPointerException | Blocker | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2971](https://issues.apache.org/jira/browse/HADOOP-2971) | SocketTimeoutException in unit tests | Major | io | Raghu Angadi | Raghu Angadi | +| [HADOOP-2970](https://issues.apache.org/jira/browse/HADOOP-2970) | Wrong class definition for hodlib/Hod/hod.py for Python \< 2.5.1 | Major | contrib/hod | Luca Telloli | Vinod Kumar Vavilapalli | +| [HADOOP-2955](https://issues.apache.org/jira/browse/HADOOP-2955) | ant test fail for TestCrcCorruption with OutofMemory. | Blocker | . | Mahadev konar | Raghu Angadi | +| [HADOOP-2943](https://issues.apache.org/jira/browse/HADOOP-2943) | Compression for intermediate map output is broken | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-2938](https://issues.apache.org/jira/browse/HADOOP-2938) | some of the fs commands don't globPaths. | Major | fs | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-2936](https://issues.apache.org/jira/browse/HADOOP-2936) | HOD should generate hdfs://host:port on the client side configs. | Major | contrib/hod | Mahadev konar | Vinod Kumar Vavilapalli | +| [HADOOP-2934](https://issues.apache.org/jira/browse/HADOOP-2934) | NPE while loading FSImage | Major | . | Raghu Angadi | dhruba borthakur | +| [HADOOP-2932](https://issues.apache.org/jira/browse/HADOOP-2932) | Trash initialization generates "deprecated filesystem name" warning even if the name is correct. | Blocker | conf, fs | Konstantin Shvachko | Mahadev konar | +| [HADOOP-2927](https://issues.apache.org/jira/browse/HADOOP-2927) | Unit test fails on Windows: org.apache.hadoop.fs.TestDU.testDU | Blocker | fs | Mukund Madhugiri | Konstantin Shvachko | +| [HADOOP-2924](https://issues.apache.org/jira/browse/HADOOP-2924) | HOD is trying to bring up task tracker on port which is already in close\_wait state | Critical | contrib/hod | Aroop Maliakkal | Vinod Kumar Vavilapalli | +| [HADOOP-2912](https://issues.apache.org/jira/browse/HADOOP-2912) | Unit test fails: org.apache.hadoop.dfs.TestFsck.testFsck. This is a regression | Blocker | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2908](https://issues.apache.org/jira/browse/HADOOP-2908) | forrest docs for dfs shell commands and semantics. | Major | documentation | Mahadev konar | Mahadev konar | +| [HADOOP-2901](https://issues.apache.org/jira/browse/HADOOP-2901) | the job tracker should not start 2 info servers | Blocker | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-2899](https://issues.apache.org/jira/browse/HADOOP-2899) | [HOD] hdfs:///mapredsystem directory not cleaned up after deallocation | Major | contrib/hod | Luca Telloli | Hemanth Yamijala | +| [HADOOP-2891](https://issues.apache.org/jira/browse/HADOOP-2891) | The dfsclient on exit deletes files that are open and not closed. | Major | . | Mahadev konar | dhruba borthakur | +| [HADOOP-2890](https://issues.apache.org/jira/browse/HADOOP-2890) | HDFS should recover when replicas of block have different sizes (due to corrupted block) | Major | . | Lohit Vijayarenu | dhruba borthakur | +| [HADOOP-2871](https://issues.apache.org/jira/browse/HADOOP-2871) | Unit tests (16) fail on Windows due to java.lang.IllegalArgumentException causing MiniMRCluster to not start up | Blocker | . | Mukund Madhugiri | Amareshwari Sriramadasu | +| [HADOOP-2870](https://issues.apache.org/jira/browse/HADOOP-2870) | Datanode.shutdown() and Namenode.stop() should close all rpc connections | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-2863](https://issues.apache.org/jira/browse/HADOOP-2863) | FSDataOutputStream should not flush() inside close(). | Major | fs | Raghu Angadi | Raghu Angadi | +| [HADOOP-2855](https://issues.apache.org/jira/browse/HADOOP-2855) | [HOD] HOD fails to allocate a cluster if the tarball specified is a relative path | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2848](https://issues.apache.org/jira/browse/HADOOP-2848) | [HOD] If a cluster directory is deleted, hod -o list must show it, and deallocate should work. | Major | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-2845](https://issues.apache.org/jira/browse/HADOOP-2845) | dfsadmin disk utilization report on Solaris is wrong | Major | fs | Martin Traverso | Martin Traverso | +| [HADOOP-2844](https://issues.apache.org/jira/browse/HADOOP-2844) | A SequenceFile.Reader object is not closed properly in CopyFiles | Major | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2832](https://issues.apache.org/jira/browse/HADOOP-2832) | bad code indentation in DFSClient | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2817](https://issues.apache.org/jira/browse/HADOOP-2817) | Remove deprecated mapred.tasktracker.tasks.maximum and clusterStatus.getMaxTasks() | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2806](https://issues.apache.org/jira/browse/HADOOP-2806) | Streaming has no way to force entire record (or null) as key | Minor | . | Marco Nicosia | Amareshwari Sriramadasu | +| [HADOOP-2800](https://issues.apache.org/jira/browse/HADOOP-2800) | SetFile.Writer deprecated by mistake? | Trivial | io | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-2790](https://issues.apache.org/jira/browse/HADOOP-2790) | TaskInProgress.hasSpeculativeTask is very inefficient | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-2783](https://issues.apache.org/jira/browse/HADOOP-2783) | hod/hodlib/Common/xmlrpc.py uses HodInterruptException without importing it | Minor | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-2779](https://issues.apache.org/jira/browse/HADOOP-2779) | build scripts broken by moving hbase to subproject | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-2767](https://issues.apache.org/jira/browse/HADOOP-2767) | org.apache.hadoop.net.NetworkTopology.InnerNode#getLeaf does not return the last node on a rack when used with an excluded node | Minor | . | Mark Butler | Hairong Kuang | +| [HADOOP-2738](https://issues.apache.org/jira/browse/HADOOP-2738) | Text is not subclassable because set(Text) and compareTo(Object) access the other instance's private members directly | Minor | io | Jim Kellerman | Jim Kellerman | +| [HADOOP-2727](https://issues.apache.org/jira/browse/HADOOP-2727) | Web UI links to Hadoop homepage has to change to new hadoop homepage | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-2679](https://issues.apache.org/jira/browse/HADOOP-2679) | There is a small typeo in hdfs\_test.c when testing the success of the local hadoop initialization | Trivial | . | Jason | dhruba borthakur | +| [HADOOP-2655](https://issues.apache.org/jira/browse/HADOOP-2655) | Copy on write for data and metadata files in the presence of snapshots | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2606](https://issues.apache.org/jira/browse/HADOOP-2606) | Namenode unstable when replicating 500k blocks at once | Major | . | Koji Noguchi | Konstantin Shvachko | +| [HADOOP-2373](https://issues.apache.org/jira/browse/HADOOP-2373) | Name node silently changes state | Major | . | Robert Chansler | Konstantin Shvachko | +| [HADOOP-2346](https://issues.apache.org/jira/browse/HADOOP-2346) | DataNode should have timeout on socket writes. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2195](https://issues.apache.org/jira/browse/HADOOP-2195) | dfs mkdir command differs from POSIX standards | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2194](https://issues.apache.org/jira/browse/HADOOP-2194) | dfs cat on a file that does not exist throws a java IOException | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2193](https://issues.apache.org/jira/browse/HADOOP-2193) | dfs rm and rmr commands differ from POSIX standards | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2191](https://issues.apache.org/jira/browse/HADOOP-2191) | dfs du and dus commands differ from POSIX standards | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2190](https://issues.apache.org/jira/browse/HADOOP-2190) | dfs ls and lsr commands differ from POSIX standards | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-2119](https://issues.apache.org/jira/browse/HADOOP-2119) | JobTracker becomes non-responsive if the task trackers finish task too fast | Critical | . | Runping Qi | Amar Kamat | +| [HADOOP-1967](https://issues.apache.org/jira/browse/HADOOP-1967) | hadoop dfs -ls, -get, -mv command's source/destination URI are inconsistent | Major | . | Lohit Vijayarenu | Doug Cutting | +| [HADOOP-1911](https://issues.apache.org/jira/browse/HADOOP-1911) | infinite loop in dfs -cat command. | Blocker | . | Koji Noguchi | Chris Douglas | +| [HADOOP-1902](https://issues.apache.org/jira/browse/HADOOP-1902) | du command throws an exception when the directory is not specified | Major | . | Mukund Madhugiri | Mahadev konar | +| [HADOOP-1373](https://issues.apache.org/jira/browse/HADOOP-1373) | checkPath() throws IllegalArgumentException | Blocker | fs | Konstantin Shvachko | Edward J. Yoon | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2997](https://issues.apache.org/jira/browse/HADOOP-2997) | Add test for non-writable serializer | Blocker | . | Tom White | Tom White | +| [HADOOP-2775](https://issues.apache.org/jira/browse/HADOOP-2775) | [HOD] Put in place unit test framework for HOD | Major | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2981](https://issues.apache.org/jira/browse/HADOOP-2981) | Follow Apache process for getting ready to put crypto code in to project | Major | . | Owen O'Malley | Owen O'Malley | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/RELEASENOTES.0.17.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/RELEASENOTES.0.17.0.md new file mode 100644 index 00000000000..467f2ac62a6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.0/RELEASENOTES.0.17.0.md @@ -0,0 +1,604 @@ + + +# Apache Hadoop 0.17.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3382](https://issues.apache.org/jira/browse/HADOOP-3382) | *Blocker* | **Memory leak when files are not cleanly closed** + +Fixed a memory leak associated with 'abandoned' files (i.e. not cleanly closed). This held up significant amounts of memory depending on activity and how long NameNode has been running. + + +--- + +* [HADOOP-3280](https://issues.apache.org/jira/browse/HADOOP-3280) | *Blocker* | **virtual address space limits break streaming apps** + +This patch adds the mapred.child.ulimit to limit the virtual memory for children processes to the given value. + + +--- + +* [HADOOP-3266](https://issues.apache.org/jira/browse/HADOOP-3266) | *Major* | **Remove HOD changes from CHANGES.txt, as they are now inside src/contrib/hod** + +Moved HOD change items from CHANGES.txt to a new file src/contrib/hod/CHANGES.txt. + + +--- + +* [HADOOP-3239](https://issues.apache.org/jira/browse/HADOOP-3239) | *Major* | **exists() calls logs FileNotFoundException in namenode log** + +getFileInfo returns null for File not found instead of throwing FileNotFoundException + + +--- + +* [HADOOP-3223](https://issues.apache.org/jira/browse/HADOOP-3223) | *Blocker* | **Hadoop dfs -help for permissions contains a typo** + +Minor typo fix in help message for chmod. impact : none. + + +--- + +* [HADOOP-3204](https://issues.apache.org/jira/browse/HADOOP-3204) | *Blocker* | **LocalFSMerger needs to catch throwable** + +Fixes LocalFSMerger in ReduceTask.java to handle errors/exceptions better. Prior to this all exceptions except IOException would be silently ignored. + + +--- + +* [HADOOP-3168](https://issues.apache.org/jira/browse/HADOOP-3168) | *Major* | **reduce amount of logging in hadoop streaming** + +Decreases the frequency of logging from streaming from every 100 records to every 10,000 records. + + +--- + +* [HADOOP-3162](https://issues.apache.org/jira/browse/HADOOP-3162) | *Blocker* | **Map/reduce stops working with comma separated input paths** + +The public methods org.apache.hadoop.mapred.JobConf.setInputPath(Path) and org.apache.hadoop.mapred.JobConf.addInputPath(Path) are deprecated. And the methods have the semantics of branch 0.16. +The following public APIs are added in org.apache.hadoop.mapred.FileInputFormat : +public static void setInputPaths(JobConf job, Path... paths); +public static void setInputPaths(JobConf job, String commaSeparatedPaths); +public static void addInputPath(JobConf job, Path path); +public static void addInputPaths(JobConf job, String commaSeparatedPaths); +Earlier code calling JobConf.setInputPath(Path), JobConf.addInputPath(Path) should now call FileInputFormat.setInputPaths(JobConf, Path...) and FileInputFormat.addInputPath(Path) respectively + + +--- + +* [HADOOP-3152](https://issues.apache.org/jira/browse/HADOOP-3152) | *Minor* | **Make index interval configuable when using MapFileOutputFormat for map-reduce job** + +Add a static method MapFile#setIndexInterval(Configuration, int interval) so that MapReduce jobs that use MapFileOutputFormat can set the index interval. + + +--- + +* [HADOOP-3140](https://issues.apache.org/jira/browse/HADOOP-3140) | *Major* | **JobTracker should not try to promote a (map) task if it does not write to DFS at all** + +Tasks that don't generate any output are not inserted in the commit queue of the JobTracker. They are marked as SUCCESSFUL by the TaskTracker and the JobTracker updates their state short-circuiting the commit queue. + + +--- + +* [HADOOP-3137](https://issues.apache.org/jira/browse/HADOOP-3137) | *Major* | **[HOD] Update hod version number** + +Build script was changed to make HOD versions follow Hadoop version numbers. As a result of this change, the next version of HOD would not be 0.5, but would be synchronized to the Hadoop version number. Users who rely on the version number of HOD should note the unexpected jump in version numbers. + + +--- + +* [HADOOP-3124](https://issues.apache.org/jira/browse/HADOOP-3124) | *Major* | **DFS data node should not use hard coded 10 minutes as write timeout.** + +Makes DataNode socket write timeout configurable. User impact : none. + + +--- + +* [HADOOP-3099](https://issues.apache.org/jira/browse/HADOOP-3099) | *Blocker* | **Need new options in distcp for preserving ower, group and permission** + +Added a new option -p to distcp for preserving file/directory status. +-p[rbugp] Preserve status + r: replication number + b: block size + u: user + g: group + p: permission + -p alone is equivalent to -prbugp + + +--- + +* [HADOOP-3093](https://issues.apache.org/jira/browse/HADOOP-3093) | *Major* | **ma/reduce throws the following exception if "io.serializations" is not set:** + +The following public APIs are added in org.apache.hadoop.conf.Configuration + String[] Configuration.getStrings(String name, String... defaultValue) and + void Configuration.setStrings(String name, String... values) + + +--- + +* [HADOOP-3091](https://issues.apache.org/jira/browse/HADOOP-3091) | *Major* | **hadoop dfs -put should support multiple src** + +hadoop dfs -put accepts multiple sources when destination is a directory. + + +--- + +* [HADOOP-3073](https://issues.apache.org/jira/browse/HADOOP-3073) | *Blocker* | **SocketOutputStream.close() should close the channel.** + +SocketOutputStream.close() closes the underlying channel. Increase compatibility with java.net.Socket.getOutputStream. User Impact : none. + + +--- + +* [HADOOP-3060](https://issues.apache.org/jira/browse/HADOOP-3060) | *Major* | **MiniMRCluster is ignoring parameter taskTrackerFirst** + +The parameter boolean taskTrackerFirst is removed from org.apache.hadoop.mapred.MiniMRCluster constructors. +Thus signature of following APIs + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, boolean taskTrackerFirst, int numDir) + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, boolean taskTrackerFirst, int numDir, String[] racks) + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, boolean taskTrackerFirst, int numDir, String[] racks, String[] hosts) + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, boolean taskTrackerFirst, int numDir, String[] racks, String[] hosts, UnixUserGroupInformation ugi ) +is changed to + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, int numDir) + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, int numDir, String[] racks) + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, int numDir, String[] racks, String[] hosts) + public MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, int numDir, String[] racks, String[] hosts, UnixUserGroupInformation ugi ) +respectively. +Since the old signatures were not deprecated, any code using the old constructors must be changed to use the new constructors. + + +--- + +* [HADOOP-3048](https://issues.apache.org/jira/browse/HADOOP-3048) | *Blocker* | **Stringifier** + + A new Interface and a default implementation to convert and restore serializations of objects to strings. + + +--- + +* [HADOOP-3041](https://issues.apache.org/jira/browse/HADOOP-3041) | *Blocker* | **Within a task, the value ofJobConf.getOutputPath() method is modified** + +1. Deprecates JobConf.setOutputPath and JobConf.getOutputPath +JobConf.getOutputPath() still returns the same value that it used to return. +2. Deprecates OutputFormatBase. Adds FileOutputFormat. Existing output formats extending OutputFormatBase, now extend FileOutputFormat. +3. Adds the following APIs in FileOutputFormat : +public static void setOutputPath(JobConf conf, Path outputDir); // sets mapred.output.dir +public static Path getOutputPath(JobConf conf) ; // gets mapred.output.dir +public static Path getWorkOutputPath(JobConf conf); // gets mapred.work.output.dir +4. static void setWorkOutputPath(JobConf conf, Path outputDir) is also added to FileOutputFormat. This is used by the framework to set mapred.work.output.dir as task's temporary output dir . + + +--- + +* [HADOOP-3040](https://issues.apache.org/jira/browse/HADOOP-3040) | *Major* | **Streaming should assume an empty key if the first character on a line is the seperator (stream.map.output.field.separator, by default, tab)** + +If the first character on a line is the separator, empty key is assumed, and the whole line is the value (due to a bug this was not the case). + + +--- + +* [HADOOP-3001](https://issues.apache.org/jira/browse/HADOOP-3001) | *Blocker* | **FileSystems should track how many bytes are read and written** + +Adds new framework map/reduce counters that track the number of bytes read and written to HDFS, local, KFS, and S3 file systems. + + +--- + +* [HADOOP-2982](https://issues.apache.org/jira/browse/HADOOP-2982) | *Blocker* | **[HOD] checknodes should look for free nodes without the jobs attribute** + +The number of free nodes in the cluster is computed using a better algorithm that filters out inconsistencies in node status as reported by Torque. + + +--- + +* [HADOOP-2947](https://issues.apache.org/jira/browse/HADOOP-2947) | *Blocker* | **[HOD] Hod should redirect stderr and stdout of Hadoop daemons to assist debugging** + +The stdout and stderr streams of daemons are redirected to files that are created under the hadoop log directory. Users can now send kill 3 signals to the daemons to get stack traces and thread dumps for debugging. + + +--- + +* [HADOOP-2899](https://issues.apache.org/jira/browse/HADOOP-2899) | *Major* | **[HOD] hdfs:///mapredsystem directory not cleaned up after deallocation** + +The mapred system directory generated by HOD is cleaned up at cluster deallocation time. + + +--- + +* [HADOOP-2873](https://issues.apache.org/jira/browse/HADOOP-2873) | *Major* | **Namenode fails to re-start after cluster shutdown - DFSClient: Could not obtain blocks even all datanodes were up & live** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-2855](https://issues.apache.org/jira/browse/HADOOP-2855) | *Blocker* | **[HOD] HOD fails to allocate a cluster if the tarball specified is a relative path** + +Changes were made to handle relative paths correctly for important HOD options such as the cluster directory, tarball option, and script file. + + +--- + +* [HADOOP-2854](https://issues.apache.org/jira/browse/HADOOP-2854) | *Blocker* | **Remove the deprecated ipc.Server.getUserInfo()** + +Removes deprecated method Server.getUserInfo() + + +--- + +* [HADOOP-2839](https://issues.apache.org/jira/browse/HADOOP-2839) | *Blocker* | **Remove deprecated methods in FileSystem** + +Removes deprecated API FileSystem#globPaths() + + +--- + +* [HADOOP-2831](https://issues.apache.org/jira/browse/HADOOP-2831) | *Blocker* | **Remove the deprecated INode.getAbsoluteName()** + +Removes deprecated method INode#getAbsoluteName() + + +--- + +* [HADOOP-2828](https://issues.apache.org/jira/browse/HADOOP-2828) | *Major* | **Remove deprecated methods in Configuration.java** + +The following deprecated methods in org.apache.hadoop.conf.Configuration are removed. +public Object getObject(String name) +public void setObject(String name, Object value) +public Object get(String name, Object defaultValue) +public void set(String name, Object value) +and public Iterator entries() + + +--- + +* [HADOOP-2826](https://issues.apache.org/jira/browse/HADOOP-2826) | *Major* | **FileSplit.getFile(), LineRecordReader. readLine() need to be removed** + +The deprecated methods, public File org.apache.hadoop.mapred.FileSplit.getFile() and + public static long org.apache.hadoop.mapred.LineRecordReader.readLine(InputStream in, OutputStream out) +are removed. +The constructor org.apache.hadoop.mapred.LineRecordReader.LineReader(InputStream in, Configuration conf) 's visibility is made public. +The signature of the public org.apache.hadoop.streaming.UTF8ByteArrayUtils.readLIne(InputStream) method is changed to UTF8ByteArrayUtils.readLIne(LineReader, Text). Since the old signature is not deprecated, any code using the old method must be changed to use the new method. + + +--- + +* [HADOOP-2825](https://issues.apache.org/jira/browse/HADOOP-2825) | *Major* | **MapOutputLocation.getFile() needs to be removed** + +The deprecated method, public long org.apache.hadoop.mapred.MapOutputLocation.getFile(FileSystem fileSys, Path localFilename, int reduce, Progressable pingee, int timeout) is removed. + + +--- + +* [HADOOP-2824](https://issues.apache.org/jira/browse/HADOOP-2824) | *Major* | **One of MiniMRCluster constructors needs tobe removed** + +The deprecated constructor org.apache.hadoop.mapred.MiniMRCluster.MiniMRCluster(int jobTrackerPort, int taskTrackerPort, int numTaskTrackers, String namenode, boolean taskTrackerFirst) is removed. + + +--- + +* [HADOOP-2823](https://issues.apache.org/jira/browse/HADOOP-2823) | *Major* | **SimpleCharStream.getColumn(), getLine() methods to be removed.** + +The deprecated methods in org.apache.hadoop.record.compiler.generated.SimpleCharStream : +public int getColumn() +and public int getLine() are removed + + +--- + +* [HADOOP-2822](https://issues.apache.org/jira/browse/HADOOP-2822) | *Major* | **Remove deprecated classes in mapred** + +The deprecated classes org.apache.hadoop.mapred.InputFormatBase and org.apache.hadoop.mapred.PhasedFileSystem are removed. + + +--- + +* [HADOOP-2821](https://issues.apache.org/jira/browse/HADOOP-2821) | *Major* | **Remove deprecated classes in util** + +The deprecated classes org.apache.hadoop.util.ShellUtil and org.apache.hadoop.util.ToolBase are removed. + + +--- + +* [HADOOP-2820](https://issues.apache.org/jira/browse/HADOOP-2820) | *Major* | **Remove deprecated classes in streaming** + +The deprecated classes org.apache.hadoop.streaming.StreamLineRecordReader, org.apache.hadoop.streaming.StreamOutputFormat and org.apache.hadoop.streaming.StreamSequenceRecordReader are removed + + +--- + +* [HADOOP-2819](https://issues.apache.org/jira/browse/HADOOP-2819) | *Major* | **Remove deprecated methods in JobConf()** + +The following deprecated methods are removed from org.apache.hadoop.JobConf : +public Class getInputKeyClass() +public void setInputKeyClass(Class theClass) +public Class getInputValueClass() +public void setInputValueClass(Class theClass) + +The methods, public boolean org.apache.hadoop.JobConf.getSpeculativeExecution() and +public void org.apache.hadoop.JobConf.setSpeculativeExecution(boolean speculativeExecution) are undeprecated. + + +--- + +* [HADOOP-2818](https://issues.apache.org/jira/browse/HADOOP-2818) | *Major* | **Remove deprecated Counters.getDisplayName(), getCounterNames(), getCounter(String counterName)** + +The deprecated methods public String org.apache.hadoop.mapred.Counters.getDisplayName(String counter) and +public synchronized Collection\ org.apache.hadoop.mapred.Counters.getCounterNames() are removed. +The deprecated method public synchronized long org.apache.hadoop.mapred.Counters.getCounter(String counterName) is undeprecated. + + +--- + +* [HADOOP-2817](https://issues.apache.org/jira/browse/HADOOP-2817) | *Major* | **Remove deprecated mapred.tasktracker.tasks.maximum and clusterStatus.getMaxTasks()** + +The deprecated method public int org.apache.hadoop.mapred.ClusterStatus.getMaxTasks() is removed. +The deprecated configuration property "mapred.tasktracker.tasks.maximum" is removed. + + +--- + +* [HADOOP-2796](https://issues.apache.org/jira/browse/HADOOP-2796) | *Major* | **For script option hod should exit with distinguishable exit codes for script code and hod exit code.** + +A provision to reliably detect a failing script's exit code was added. In case the hod script option returned a non-zero exit code, users can now look for a 'script.exitcode' file written to the HOD cluster directory. If this file is present, it means the script failed with the returned exit code. + + +--- + +* [HADOOP-2775](https://issues.apache.org/jira/browse/HADOOP-2775) | *Major* | **[HOD] Put in place unit test framework for HOD** + +A unit testing framework based on pyunit is added to HOD. Developers contributing patches to HOD should now contribute unit tests along with the patches where possible. + + +--- + +* [HADOOP-2765](https://issues.apache.org/jira/browse/HADOOP-2765) | *Major* | **setting memory limits for tasks** + +This feature enables specifying ulimits for streaming/pipes tasks. Now pipes and streaming tasks have same virtual memory available as the java process which invokes them. Ulimit value will be the same as -Xmx value for java processes provided using mapred.child.java.opts. + + +--- + +* [HADOOP-2758](https://issues.apache.org/jira/browse/HADOOP-2758) | *Major* | **Reduce memory copies when data is read from DFS** + +DataNode takes 50% less CPU while serving data to clients. + + +--- + +* [HADOOP-2657](https://issues.apache.org/jira/browse/HADOOP-2657) | *Major* | **Enhancements to DFSClient to support flushing data at any point in time** + +A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes. + + +--- + +* [HADOOP-2634](https://issues.apache.org/jira/browse/HADOOP-2634) | *Blocker* | **Deprecate exists() and isDir() to simplify ClientProtocol.** + +Deprecates exists() from ClientProtocol + + +--- + +* [HADOOP-2563](https://issues.apache.org/jira/browse/HADOOP-2563) | *Blocker* | **Remove deprecated FileSystem#listPaths()** + +Removes deprecated method FileSystem#listPaths() + + +--- + +* [HADOOP-2559](https://issues.apache.org/jira/browse/HADOOP-2559) | *Major* | **DFS should place one replica per rack** + +Change DFS block placement to allocate the first replica locally, the second off-rack, and the third intra-rack from the second. + + +--- + +* [HADOOP-2551](https://issues.apache.org/jira/browse/HADOOP-2551) | *Blocker* | **hadoop-env.sh needs finer granularity** + +New environment variables were introduced to allow finer grained control of Java options passed to server and client JVMs. See the new *\_OPTS variables in conf/hadoop-env.sh. + + +--- + +* [HADOOP-2470](https://issues.apache.org/jira/browse/HADOOP-2470) | *Major* | **Open and isDir should be removed from ClientProtocol** + +Open and isDir were removed from ClientProtocol. + + +--- + +* [HADOOP-2423](https://issues.apache.org/jira/browse/HADOOP-2423) | *Major* | **The codes in FSDirectory.mkdirs(...) is inefficient.** + +Improved FSDirectory.mkdirs(...) performance. In NNThroughputBenchmark-create, the ops per sec in was improved ~54%. + + +--- + +* [HADOOP-2410](https://issues.apache.org/jira/browse/HADOOP-2410) | *Major* | **Make EC2 cluster nodes more independent of each other** + +The command "hadoop-ec2 run" has been replaced by "hadoop-ec2 launch-cluster \ \", and "hadoop-ec2 start-hadoop" has been removed since Hadoop is started on instance start up. See http://wiki.apache.org/hadoop/AmazonEC2 for details. + + +--- + +* [HADOOP-2399](https://issues.apache.org/jira/browse/HADOOP-2399) | *Major* | **Input key and value to combiner and reducer should be reused** + +The key and value objects that are given to the Combiner and Reducer are now reused between calls. This is much more efficient, but the user can not assume the objects are constant. + + +--- + +* [HADOOP-2345](https://issues.apache.org/jira/browse/HADOOP-2345) | *Major* | **new transactions to support HDFS Appends** + +Introduce new namenode transactions to support appending to HDFS files. + + +--- + +* [HADOOP-2239](https://issues.apache.org/jira/browse/HADOOP-2239) | *Major* | **Security: Need to be able to encrypt Hadoop socket connections** + +This patch adds a new FileSystem, HftpsFileSystem, that allows access to HDFS data over HTTPS. + + +--- + +* [HADOOP-2219](https://issues.apache.org/jira/browse/HADOOP-2219) | *Major* | **du like command to count number of files under a given directory** + +Added a new fs command fs -count for counting the number of bytes, files and directories under a given path. + +Added a new RPC getContentSummary(String path) to ClientProtocol. + + +--- + +* [HADOOP-2192](https://issues.apache.org/jira/browse/HADOOP-2192) | *Major* | **dfs mv command differs from POSIX standards** + +this patch makes dfs -mv more like linux mv command getting rid of unnecessary output in dfs -mv and returns an error message when moving non existent files/directories --- mv: cannot stat "filename": No such file or directory. + + +--- + +* [HADOOP-2178](https://issues.apache.org/jira/browse/HADOOP-2178) | *Major* | **Job history on HDFS** + +This feature provides facility to store job history on DFS. Now cluster admin can provide either localFS location or DFS location using configuration property "mapred.job.history.location" to store job histroy. History will be logged in user specified location also. User can specify history location using configuration property "mapred.job.history.user.location" . +The classes org.apache.hadoop.mapred.DefaultJobHistoryParser.MasterIndex and org.apache.hadoop.mapred.DefaultJobHistoryParser.MasterIndexParseListener, and public method org.apache.hadoop.mapred.DefaultJobHistoryParser.parseMasterIndex are not available. +The signature of public method org.apache.hadoop.mapred.DefaultJobHistoryParser.parseJobTasks(File jobHistoryFile, JobHistory.JobInfo job) is changed to DefaultJobHistoryParser.parseJobTasks(String jobHistoryFile, JobHistory.JobInfo job, FileSystem fs). +The signature of public method org.apache.hadoop.mapred.JobHistory.parseHistory(File path, Listener l) is changed to JobHistory.parseHistoryFromFS(String path, Listener l, FileSystem fs) + + +--- + +* [HADOOP-2119](https://issues.apache.org/jira/browse/HADOOP-2119) | *Critical* | **JobTracker becomes non-responsive if the task trackers finish task too fast** + +This removes many inefficiencies in task placement and scheduling logic. The JobTracker would perform linear scans of the list of submitted tasks in cases where it did not find an obvious candidate task for a node. With better data structures for managing job state, all task placement operations now run in constant time (in most cases). Also, the task output promotions are batched. + + +--- + +* [HADOOP-2116](https://issues.apache.org/jira/browse/HADOOP-2116) | *Major* | **Job.local.dir to be exposed to tasks** + +This issue restructures local job directory on the tasktracker. +Users are provided with a job-specific shared directory (mapred-local/taskTracker/jobcache/$jobid/ work) for using it as scratch space, through configuration property and system property "job.local.dir". Now, the directory "../work" is not available from the task's cwd. + + +--- + +* [HADOOP-2063](https://issues.apache.org/jira/browse/HADOOP-2063) | *Blocker* | **Command to pull corrupted files** + +Added a new option -ignoreCrc to fs -get, or equivalently, fs -copyToLocal, such that crc checksum will be ignored for the command. The use of this option is to download the corrupted files. + + +--- + +* [HADOOP-2055](https://issues.apache.org/jira/browse/HADOOP-2055) | *Minor* | **JobConf should have a setInputPathFilter method** + +This issue provides users the ability to specify what paths to ignore for processing in the job input directory (apart from the filenames that start with "\_" and "."). Defines two new APIs - FileInputFormat.setInputPathFilter(JobConf, PathFilter), and, FileInputFormat.getInputPathFilter(JobConf). + + +--- + +* [HADOOP-2027](https://issues.apache.org/jira/browse/HADOOP-2027) | *Major* | **FileSystem should provide byte ranges for file locations** + +New FileSystem API getFileBlockLocations to return the number of bytes in each block in a file via a single rpc to the namenode to speed up job planning. Deprecates getFileCacheHints. + + +--- + +* [HADOOP-1986](https://issues.apache.org/jira/browse/HADOOP-1986) | *Major* | **Add support for a general serialization mechanism for Map Reduce** + +Programs that implement the raw Mapper or Reducer interfaces will need modification to compile with this release. For example, + +class MyMapper implements Mapper { + public void map(WritableComparable key, Writable val, + OutputCollector out, Reporter reporter) throws IOException { + // ... + } + // ... +} + +will need to be changed to refer to the parameterized type. For example: + +class MyMapper implements Mapper\ { + public void map(WritableComparable key, Writable val, + OutputCollector\ out, Reporter reporter) throws IOException { + // ... + } + // ... +} + +Similarly implementations of the following raw interfaces will need modification: InputFormat, OutputCollector, OutputFormat, Partitioner, RecordReader, RecordWriter + + +--- + +* [HADOOP-1985](https://issues.apache.org/jira/browse/HADOOP-1985) | *Major* | **Abstract node to switch mapping into a topology service class used by namenode and jobtracker** + +This issue introduces rack awareness for map tasks. It also moves the rack resolution logic to the central servers - NameNode & JobTracker. The administrator can specify a loadable class given by topology.node.switch.mapping.impl to specify the class implementing the logic for rack resolution. The class must implement a method - resolve(List\ names), where names is the list of DNS-names/IP-addresses that we want resolved. The return value is a list of resolved network paths of the form /foo/rack, where rack is the rackID where the node belongs to and foo is the switch where multiple racks are connected, and so on. The default implementation of this class is packaged along with hadoop and points to org.apache.hadoop.net.ScriptBasedMapping and this class loads a script that can be used for rack resolution. The script location is configurable. It is specified by topology.script.file.name and defaults to an empty script. In the case where the script name is empty, /default-rack is returned for all dns-names/IP-addresses. The loadable topology.node.switch.mapping.impl provides administrators fleixibilty to define how their site's node resolution should happen. +For mapred, one can also specify the level of the cache w.r.t the number of levels in the resolved network path - defaults to two. This means that the JobTracker will cache tasks at the host level and at the rack level. +Known issue: the task caching will not work with levels greater than 2 (beyond racks). This bug is tracked in HADOOP-3296. + + +--- + +* [HADOOP-1622](https://issues.apache.org/jira/browse/HADOOP-1622) | *Major* | **Hadoop should provide a way to allow the user to specify jar file(s) the user job depends on** + +This patch allows new command line options for + +hadoop jar +which are + +hadoop jar -files \ -libjars \ -archives \ + +-files options allows you to speficy comma seperated list of path which would be present in your current working directory of your task +-libjars option allows you to add jars to the classpaths of the maps and reduces. +-archives allows you to pass archives as arguments that are unzipped/unjarred and a link with name of the jar/zip are created in the current working directory if tasks. + + +--- + +* [HADOOP-1593](https://issues.apache.org/jira/browse/HADOOP-1593) | *Major* | **FsShell should work with paths in non-default FileSystem** + +This bug allows non default path to specifeid in fsshell commands. + +So, you can now specify hadoop dfs -ls hdfs://remotehost1:port/path + and hadoop dfs -ls hdfs://remotehost2:port/path without changing the config. + + +--- + +* [HADOOP-910](https://issues.apache.org/jira/browse/HADOOP-910) | *Major* | **Reduces can do merges for the on-disk map output files in parallel with their copying** + +Reducers now perform merges of shuffle data (both in-memory and on disk) while fetching map outputs. Earlier, during shuffle they used to merge only the in-memory outputs. + + +--- + +* [HADOOP-771](https://issues.apache.org/jira/browse/HADOOP-771) | *Major* | **Namenode should return error when trying to delete non-empty directory** + +This patch adds a new api to file system i.e delete(path, boolean), deprecating the previous delete(path). +the new api recursively deletes files only if boolean is set to true. +If path is a file, the boolean value does not matter, if path is a directory and the directory is non empty delete(path, false) will throw an exception and delete(path, true) will delete all files recursively. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/CHANGES.0.17.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/CHANGES.0.17.1.md new file mode 100644 index 00000000000..991cbd7ef56 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/CHANGES.0.17.1.md @@ -0,0 +1,74 @@ + + +# Apache Hadoop Changelog + +## Release 0.17.1 - 2008-06-23 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3565](https://issues.apache.org/jira/browse/HADOOP-3565) | JavaSerialization can throw java.io.StreamCorruptedException | Major | . | Tom White | Tom White | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3550](https://issues.apache.org/jira/browse/HADOOP-3550) | Reduce tasks failing with OOM | Blocker | . | Arun C Murthy | Chris Douglas | +| [HADOOP-3526](https://issues.apache.org/jira/browse/HADOOP-3526) | contrib/data\_join doesn't work | Blocker | . | Spyros Blanas | Spyros Blanas | +| [HADOOP-3522](https://issues.apache.org/jira/browse/HADOOP-3522) | ValuesIterator.next() doesn't return a new object, thus failing many equals() tests. | Major | . | Spyros Blanas | Owen O'Malley | +| [HADOOP-3477](https://issues.apache.org/jira/browse/HADOOP-3477) | release tar.gz contains duplicate files | Major | build | Adam Heath | Adam Heath | +| [HADOOP-3475](https://issues.apache.org/jira/browse/HADOOP-3475) | MapOutputBuffer allocates 4x as much space to record capacity as intended | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3472](https://issues.apache.org/jira/browse/HADOOP-3472) | MapFile.Reader getClosest() function returns incorrect results when before is true | Major | io | Todd Lipcon | stack | +| [HADOOP-3442](https://issues.apache.org/jira/browse/HADOOP-3442) | QuickSort may get into unbounded recursion | Blocker | . | Runping Qi | Chris Douglas | +| [HADOOP-2159](https://issues.apache.org/jira/browse/HADOOP-2159) | Namenode stuck in safemode | Major | . | Christian Kunz | Hairong Kuang | +| [HADOOP-1979](https://issues.apache.org/jira/browse/HADOOP-1979) | fsck on namenode without datanodes takes too much time | Minor | . | Koji Noguchi | Lohit Vijayarenu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/RELEASENOTES.0.17.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/RELEASENOTES.0.17.1.md new file mode 100644 index 00000000000..7cc43a3a183 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.1/RELEASENOTES.0.17.1.md @@ -0,0 +1,38 @@ + + +# Apache Hadoop 0.17.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3565](https://issues.apache.org/jira/browse/HADOOP-3565) | *Major* | **JavaSerialization can throw java.io.StreamCorruptedException** + +Change the Java serialization framework, which is not enabled by default, to correctly make the objects independent of the previous objects. + + +--- + +* [HADOOP-1979](https://issues.apache.org/jira/browse/HADOOP-1979) | *Minor* | **fsck on namenode without datanodes takes too much time** + +Improved performance of {{fsck}} by better management of the data stream on the client side. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/CHANGES.0.17.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/CHANGES.0.17.2.md new file mode 100644 index 00000000000..2ee5df69c58 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/CHANGES.0.17.2.md @@ -0,0 +1,77 @@ + + +# Apache Hadoop Changelog + +## Release 0.17.2 - 2008-08-11 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4773](https://issues.apache.org/jira/browse/HADOOP-4773) | namenode startup error, hadoop-user-namenode.pid permission denied. | Critical | . | Focus | | +| [HADOOP-3931](https://issues.apache.org/jira/browse/HADOOP-3931) | Bug in MapTask.MapOutputBuffer.collect leads to an unnecessary and harmful 'reset' | Blocker | . | Arun C Murthy | Chris Douglas | +| [HADOOP-3859](https://issues.apache.org/jira/browse/HADOOP-3859) | 1000 concurrent read on a single file failing the task/client | Blocker | . | Koji Noguchi | Johan Oskarsson | +| [HADOOP-3813](https://issues.apache.org/jira/browse/HADOOP-3813) | RPC queue overload of JobTracker | Major | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-3760](https://issues.apache.org/jira/browse/HADOOP-3760) | DFS operations fail because of Stream closed error | Blocker | . | Amar Kamat | Lohit Vijayarenu | +| [HADOOP-3758](https://issues.apache.org/jira/browse/HADOOP-3758) | Excessive exceptions in HDFS namenode log file | Blocker | . | Jim Huang | Lohit Vijayarenu | +| [HADOOP-3707](https://issues.apache.org/jira/browse/HADOOP-3707) | Frequent DiskOutOfSpaceException on almost-full datanodes | Blocker | . | Koji Noguchi | Raghu Angadi | +| [HADOOP-3685](https://issues.apache.org/jira/browse/HADOOP-3685) | Unbalanced replication target | Blocker | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-3681](https://issues.apache.org/jira/browse/HADOOP-3681) | Infinite loop in dfs close | Blocker | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-3678](https://issues.apache.org/jira/browse/HADOOP-3678) | Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3633](https://issues.apache.org/jira/browse/HADOOP-3633) | Uncaught exception in DataXceiveServer | Blocker | . | Koji Noguchi | Konstantin Shvachko | +| [HADOOP-3370](https://issues.apache.org/jira/browse/HADOOP-3370) | failed tasks may stay forever in TaskTracker.runningJobs | Critical | . | Zheng Shao | Zheng Shao | +| [HADOOP-3002](https://issues.apache.org/jira/browse/HADOOP-3002) | HDFS should not remove blocks while in safemode. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/RELEASENOTES.0.17.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/RELEASENOTES.0.17.2.md new file mode 100644 index 00000000000..22c90b81d2a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.2/RELEASENOTES.0.17.2.md @@ -0,0 +1,52 @@ + + +# Apache Hadoop 0.17.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3859](https://issues.apache.org/jira/browse/HADOOP-3859) | *Blocker* | **1000 concurrent read on a single file failing the task/client** + +Allows the user to change the maximum number of xceivers in the datanode. + + +--- + +* [HADOOP-3760](https://issues.apache.org/jira/browse/HADOOP-3760) | *Blocker* | **DFS operations fail because of Stream closed error** + +Fix a bug with HDFS file close() mistakenly introduced by HADOOP-3681. + + +--- + +* [HADOOP-3707](https://issues.apache.org/jira/browse/HADOOP-3707) | *Blocker* | **Frequent DiskOutOfSpaceException on almost-full datanodes** + +NameNode keeps a count of number of blocks scheduled to be written to a datanode and uses it to avoid allocating more blocks than a datanode can hold. + + +--- + +* [HADOOP-3678](https://issues.apache.org/jira/browse/HADOOP-3678) | *Blocker* | **Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log** + +Avoid spurious exceptions logged at DataNode when clients read from DFS. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/CHANGES.0.17.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/CHANGES.0.17.3.md new file mode 100644 index 00000000000..1b8a3ed2864 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/CHANGES.0.17.3.md @@ -0,0 +1,70 @@ + + +# Apache Hadoop Changelog + +## Release 0.17.3 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4326](https://issues.apache.org/jira/browse/HADOOP-4326) | ChecksumFileSystem does not override all create(...) methods | Blocker | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4318](https://issues.apache.org/jira/browse/HADOOP-4318) | distcp fails | Blocker | . | Christian Kunz | Tsz Wo Nicholas Sze | +| [HADOOP-4277](https://issues.apache.org/jira/browse/HADOOP-4277) | Checksum verification is disabled for LocalFS | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4271](https://issues.apache.org/jira/browse/HADOOP-4271) | Bug in FSInputChecker makes it possible to read from an invalid buffer | Blocker | fs | Ning Li | Ning Li | +| [HADOOP-3217](https://issues.apache.org/jira/browse/HADOOP-3217) | [HOD] Be less agressive when querying job status from resource manager. | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4164](https://issues.apache.org/jira/browse/HADOOP-4164) | Chinese translation of core docs | Major | documentation | Xuebing Yan | Xuebing Yan | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/RELEASENOTES.0.17.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/RELEASENOTES.0.17.3.md new file mode 100644 index 00000000000..dd019268ecb --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.17.3/RELEASENOTES.0.17.3.md @@ -0,0 +1,45 @@ + + +# Apache Hadoop 0.17.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-4277](https://issues.apache.org/jira/browse/HADOOP-4277) | *Blocker* | **Checksum verification is disabled for LocalFS** + +Checksum verification was mistakenly disabled for LocalFileSystem. + + +--- + +* [HADOOP-4271](https://issues.apache.org/jira/browse/HADOOP-4271) | *Blocker* | **Bug in FSInputChecker makes it possible to read from an invalid buffer** + +Checksum input stream can sometimes return invalid data to the user. + + +--- + +* [HADOOP-4164](https://issues.apache.org/jira/browse/HADOOP-4164) | *Major* | **Chinese translation of core docs** + +Chinese translation for hadoop 0.17.x core docs. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md new file mode 100644 index 00000000000..7df47c0b83e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/CHANGES.0.18.0.md @@ -0,0 +1,337 @@ + + +# Apache Hadoop Changelog + +## Release 0.18.0 - 2008-08-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3837](https://issues.apache.org/jira/browse/HADOOP-3837) | hadop streaming does not use progress reporting to detect hung tasks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3808](https://issues.apache.org/jira/browse/HADOOP-3808) | [HOD] Include job tracker RPC in notes attribute after job submission | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3703](https://issues.apache.org/jira/browse/HADOOP-3703) | [HOD] logcondense needs to use the new pattern of output in hadoop dfs -lsr | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3683](https://issues.apache.org/jira/browse/HADOOP-3683) | Hadoop dfs metric FilesListed shows number of files listed instead of operations | Major | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3665](https://issues.apache.org/jira/browse/HADOOP-3665) | WritableComparator newKey() fails for NullWritable | Minor | io | Lukas Vlcek | Chris Douglas | +| [HADOOP-3610](https://issues.apache.org/jira/browse/HADOOP-3610) | [HOD] HOD does not automatically create a cluster directory for the script option | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3598](https://issues.apache.org/jira/browse/HADOOP-3598) | Map-Reduce framework needlessly creates temporary \_${taskid} directories for Maps | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3569](https://issues.apache.org/jira/browse/HADOOP-3569) | KFS input stream read() returns 4 bytes instead of 1 | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3512](https://issues.apache.org/jira/browse/HADOOP-3512) | Split map/reduce tools into separate jars | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3486](https://issues.apache.org/jira/browse/HADOOP-3486) | Change default for initial block report to 0 sec and document it in hadoop-defaults.xml | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-3483](https://issues.apache.org/jira/browse/HADOOP-3483) | [HOD] Improvements with cluster directory handling | Major | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3459](https://issues.apache.org/jira/browse/HADOOP-3459) | Change dfs -ls listing to closely match format on Linux | Major | . | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-3452](https://issues.apache.org/jira/browse/HADOOP-3452) | fsck exit code would be better if non-zero when FS corrupt | Minor | . | Pete Wyckoff | Lohit Vijayarenu | +| [HADOOP-3417](https://issues.apache.org/jira/browse/HADOOP-3417) | JobClient should not have a static configuration for cli parsing | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-3405](https://issues.apache.org/jira/browse/HADOOP-3405) | Make mapred internal classes package-local | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-3390](https://issues.apache.org/jira/browse/HADOOP-3390) | Remove deprecated ClientProtocol.abandonFileInProgress() | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3379](https://issues.apache.org/jira/browse/HADOOP-3379) | Document the "stream.non.zero.exit.status.is.failure" knob for streaming | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3329](https://issues.apache.org/jira/browse/HADOOP-3329) | DatanodeDescriptor objects stored in FSImage may be out dated. | Major | . | Tsz Wo Nicholas Sze | dhruba borthakur | +| [HADOOP-3317](https://issues.apache.org/jira/browse/HADOOP-3317) | add default port for hdfs namenode | Minor | . | Doug Cutting | Doug Cutting | +| [HADOOP-3310](https://issues.apache.org/jira/browse/HADOOP-3310) | Lease recovery for append | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3283](https://issues.apache.org/jira/browse/HADOOP-3283) | Need a mechanism for data nodes to update generation stamps. | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3265](https://issues.apache.org/jira/browse/HADOOP-3265) | Remove deprecated API getFileCacheHints | Major | fs | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3226](https://issues.apache.org/jira/browse/HADOOP-3226) | Run combiner when merging spills from map output | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3193](https://issues.apache.org/jira/browse/HADOOP-3193) | Discovery of corrupt block reported in name node log | Minor | . | Robert Chansler | Chris Douglas | +| [HADOOP-3184](https://issues.apache.org/jira/browse/HADOOP-3184) | HOD gracefully exclude "bad" nodes during ring formation | Major | contrib/hod | Marco Nicosia | Hemanth Yamijala | +| [HADOOP-3113](https://issues.apache.org/jira/browse/HADOOP-3113) | DFSOututStream.flush() should flush data to real block file on DataNode. | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3095](https://issues.apache.org/jira/browse/HADOOP-3095) | Validating input paths and creating splits is slow on S3 | Major | fs, fs/s3 | Tom White | Tom White | +| [HADOOP-3035](https://issues.apache.org/jira/browse/HADOOP-3035) | Data nodes should inform the name-node about block crc errors. | Major | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-2909](https://issues.apache.org/jira/browse/HADOOP-2909) | Improve IPC idle connection management | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-2865](https://issues.apache.org/jira/browse/HADOOP-2865) | FsShell.ls() should print file attributes first then the path name. | Major | . | Konstantin Shvachko | Edward J. Yoon | +| [HADOOP-2797](https://issues.apache.org/jira/browse/HADOOP-2797) | Withdraw CRC upgrade from HDFS | Critical | . | Robert Chansler | Raghu Angadi | +| [HADOOP-2703](https://issues.apache.org/jira/browse/HADOOP-2703) | New files under lease (before close) still shows up as MISSING files/blocks in fsck | Minor | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-2656](https://issues.apache.org/jira/browse/HADOOP-2656) | Support for upgrading existing cluster to facilitate appends to HDFS files | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2585](https://issues.apache.org/jira/browse/HADOOP-2585) | Automatic namespace recovery from the secondary image. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2188](https://issues.apache.org/jira/browse/HADOOP-2188) | RPC should send a ping rather than use client timeouts | Major | ipc | Owen O'Malley | Hairong Kuang | +| [HADOOP-2181](https://issues.apache.org/jira/browse/HADOOP-2181) | Input Split details for maps should be logged | Minor | . | Lohit Vijayarenu | Amareshwari Sriramadasu | +| [HADOOP-2065](https://issues.apache.org/jira/browse/HADOOP-2065) | Replication policy for corrupted block | Major | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-1702](https://issues.apache.org/jira/browse/HADOOP-1702) | Reduce buffer copies when data is written to DFS | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-544](https://issues.apache.org/jira/browse/HADOOP-544) | Replace the job, tip and task ids with objects. | Major | . | Owen O'Malley | Enis Soztutar | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3502](https://issues.apache.org/jira/browse/HADOOP-3502) | Quota API needs documentation in Forrest | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3460](https://issues.apache.org/jira/browse/HADOOP-3460) | SequenceFileAsBinaryOutputFormat | Minor | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-3336](https://issues.apache.org/jira/browse/HADOOP-3336) | Direct a subset of namenode RPC events for audit logging | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3307](https://issues.apache.org/jira/browse/HADOOP-3307) | Archives in Hadoop. | Major | fs | Mahadev konar | Mahadev konar | +| [HADOOP-3250](https://issues.apache.org/jira/browse/HADOOP-3250) | Extend FileSystem API to allow appending to files | Major | fs | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3246](https://issues.apache.org/jira/browse/HADOOP-3246) | FTP client over HDFS | Major | util | Ankur | Ankur | +| [HADOOP-3230](https://issues.apache.org/jira/browse/HADOOP-3230) | Add command line access to named counters | Major | scripts | Tom White | Tom White | +| [HADOOP-3221](https://issues.apache.org/jira/browse/HADOOP-3221) | Need a "LineBasedTextInputFormat" | Major | . | Milind Bhandarkar | Amareshwari Sriramadasu | +| [HADOOP-3188](https://issues.apache.org/jira/browse/HADOOP-3188) | compaction utility for directories | Major | . | Robert Chansler | Robert Chansler | +| [HADOOP-3187](https://issues.apache.org/jira/browse/HADOOP-3187) | Quotas for name space management | Major | . | Robert Chansler | Hairong Kuang | +| [HADOOP-3074](https://issues.apache.org/jira/browse/HADOOP-3074) | URLStreamHandler for the DFS | Major | util | Christophe Taton | Christophe Taton | +| [HADOOP-3061](https://issues.apache.org/jira/browse/HADOOP-3061) | Writable for single byte and double | Major | io | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-3022](https://issues.apache.org/jira/browse/HADOOP-3022) | Fast Cluster Restart | Major | . | Robert Chansler | Konstantin Shvachko | +| [HADOOP-1915](https://issues.apache.org/jira/browse/HADOOP-1915) | adding counters methods using String (as opposed to Enum) | Minor | . | Alejandro Abdelnur | Tom White | +| [HADOOP-1328](https://issues.apache.org/jira/browse/HADOOP-1328) | Hadoop Streaming needs to provide a way for the stream plugin to update global counters | Major | . | Runping Qi | Tom White | +| [HADOOP-930](https://issues.apache.org/jira/browse/HADOOP-930) | Add support for reading regular (non-block-based) files from S3 in S3FileSystem | Major | fs | Tom White | Tom White | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3599](https://issues.apache.org/jira/browse/HADOOP-3599) | The new setCombineOnceOnly shouldn't take a JobConf, since it is a method on JobConf | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3572](https://issues.apache.org/jira/browse/HADOOP-3572) | setQuotas usage interface has some minor bugs. | Minor | . | Mahadev konar | Hairong Kuang | +| [HADOOP-3547](https://issues.apache.org/jira/browse/HADOOP-3547) | Improve documentation about distributing native libraries via DistributedCache | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3535](https://issues.apache.org/jira/browse/HADOOP-3535) | IOUtils.close needs better documentation | Blocker | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-3532](https://issues.apache.org/jira/browse/HADOOP-3532) | Create build targets to create api change reports using jdiff | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-3501](https://issues.apache.org/jira/browse/HADOOP-3501) | deprecate InMemoryFileSystem | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-3492](https://issues.apache.org/jira/browse/HADOOP-3492) | add forrest documentation for user archives | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3467](https://issues.apache.org/jira/browse/HADOOP-3467) | The javadoc for FileSystem.deleteOnExit should have more description | Blocker | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3464](https://issues.apache.org/jira/browse/HADOOP-3464) | [HOD] HOD can improve error messages by reporting failures on compute nodes back to hod client | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3455](https://issues.apache.org/jira/browse/HADOOP-3455) | IPC.Client synchronisation looks weak | Major | ipc | Steve Loughran | Hairong Kuang | +| [HADOOP-3448](https://issues.apache.org/jira/browse/HADOOP-3448) | Add some more hints of the problem when datanode and namenode don't match | Minor | . | Steve Loughran | Steve Loughran | +| [HADOOP-3434](https://issues.apache.org/jira/browse/HADOOP-3434) | Retain cause of bind failure in Server.bind | Major | . | Steve Loughran | Steve Loughran | +| [HADOOP-3429](https://issues.apache.org/jira/browse/HADOOP-3429) | Increase the buffersize for the streaming parent java process's streams | Major | . | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-3406](https://issues.apache.org/jira/browse/HADOOP-3406) | Document controls for profiling maps & reduces | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3400](https://issues.apache.org/jira/browse/HADOOP-3400) | Facilitate creation of temporary files in HDFS | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3398](https://issues.apache.org/jira/browse/HADOOP-3398) | ReduceTask::closestPowerOf2 is inefficient | Trivial | . | Chris Douglas | Chris Douglas | +| [HADOOP-3377](https://issues.apache.org/jira/browse/HADOOP-3377) | Use StringUtils#replaceAll instead of | Trivial | . | Brice Arnould | Brice Arnould | +| [HADOOP-3369](https://issues.apache.org/jira/browse/HADOOP-3369) | Fast block processing during name-node startup. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3366](https://issues.apache.org/jira/browse/HADOOP-3366) | Shuffle/Merge improvements | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3364](https://issues.apache.org/jira/browse/HADOOP-3364) | Faster image and log edits loading. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3355](https://issues.apache.org/jira/browse/HADOOP-3355) | Configuration should accept decimal and hexadecimal values | Major | conf | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3350](https://issues.apache.org/jira/browse/HADOOP-3350) | distcp should permit users to limit the number of maps | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3345](https://issues.apache.org/jira/browse/HADOOP-3345) | Enhance the hudson-test-patch target | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-3334](https://issues.apache.org/jira/browse/HADOOP-3334) | Move lease handling codes out from FSNamesystem | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3332](https://issues.apache.org/jira/browse/HADOOP-3332) | improving the logging during shuffling | Blocker | . | Runping Qi | Devaraj Das | +| [HADOOP-3308](https://issues.apache.org/jira/browse/HADOOP-3308) | Improve QuickSort by excluding values eq the pivot from the partition | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3297](https://issues.apache.org/jira/browse/HADOOP-3297) | The way in which ReduceTask/TaskTracker gets completion events during shuffle can be improved | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-3295](https://issues.apache.org/jira/browse/HADOOP-3295) | Allow TextOutputFormat to use configurable separators | Minor | io | Zheng Shao | Zheng Shao | +| [HADOOP-3277](https://issues.apache.org/jira/browse/HADOOP-3277) | hod should better errors message when deallocate is fired on non allocated directory. | Minor | contrib/hod | Karam Singh | | +| [HADOOP-3274](https://issues.apache.org/jira/browse/HADOOP-3274) | The default constructor of BytesWritable should not create a 100-byte array. | Minor | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3270](https://issues.apache.org/jira/browse/HADOOP-3270) | Constant DatanodeCommand should be stored in static fianl immutable variables. | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3254](https://issues.apache.org/jira/browse/HADOOP-3254) | FSNamesystem.gotHeartbeat(..., Object[] xferResults, Object[] deleteList) should not use Object[] as pass-by-reference parameters | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3177](https://issues.apache.org/jira/browse/HADOOP-3177) | Expose DFSOutputStream.fsync API though the FileSystem interface | Major | . | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3164](https://issues.apache.org/jira/browse/HADOOP-3164) | Use FileChannel.transferTo() when data is read from DataNode. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3144](https://issues.apache.org/jira/browse/HADOOP-3144) | better fault tolerance for corrupted text files | Major | . | Joydeep Sen Sarma | Zheng Shao | +| [HADOOP-3096](https://issues.apache.org/jira/browse/HADOOP-3096) | Improve documentation about the Task Execution Environment in the Map-Reduce tutorial | Blocker | documentation | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-3058](https://issues.apache.org/jira/browse/HADOOP-3058) | Hadoop DFS to report more replication metrics | Minor | metrics | Marco Nicosia | Lohit Vijayarenu | +| [HADOOP-3013](https://issues.apache.org/jira/browse/HADOOP-3013) | fsck to show (checksum) corrupted files | Major | . | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-2987](https://issues.apache.org/jira/browse/HADOOP-2987) | Keep two generations of fsimage | Major | . | Robert Chansler | Konstantin Shvachko | +| [HADOOP-2910](https://issues.apache.org/jira/browse/HADOOP-2910) | Throttle IPC Client/Server during bursts of requests or server slowdown | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-2867](https://issues.apache.org/jira/browse/HADOOP-2867) | Add a task's cwd to it's LD\_LIBRARY\_PATH | Major | . | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-2857](https://issues.apache.org/jira/browse/HADOOP-2857) | libhdfs: no way to set JVM args other than classpath | Minor | . | Craig Macdonald | Craig Macdonald | +| [HADOOP-2799](https://issues.apache.org/jira/browse/HADOOP-2799) | Replace org.apache.hadoop.io.Closeable with java.io.Closeable | Minor | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2762](https://issues.apache.org/jira/browse/HADOOP-2762) | Better documentation of controls for memory limits on hadoop daemons and Map-Reduce tasks | Blocker | documentation, scripts | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-2661](https://issues.apache.org/jira/browse/HADOOP-2661) | Replicator log should include block id | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-2461](https://issues.apache.org/jira/browse/HADOOP-2461) | Configuration should trim property names and accept decimal, hexadecimal, and octal numbers | Minor | conf | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2154](https://issues.apache.org/jira/browse/HADOOP-2154) | Non-interleaved checksums would optimize block transfers. | Major | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-2019](https://issues.apache.org/jira/browse/HADOOP-2019) | DistributedFileCache should support .tgz files in addition to jars and zip files | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-4](https://issues.apache.org/jira/browse/HADOOP-4) | tool to mount dfs on linux | Major | . | John Xing | Pete Wyckoff | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3947](https://issues.apache.org/jira/browse/HADOOP-3947) | TaskTrackers fail to connect back upon a re-init action | Blocker | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-3901](https://issues.apache.org/jira/browse/HADOOP-3901) | CLASSPATH in bin/hadoop script is set incorrectly for cygwin | Blocker | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3897](https://issues.apache.org/jira/browse/HADOOP-3897) | SecondaryNameNode fails with NullPointerException | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3884](https://issues.apache.org/jira/browse/HADOOP-3884) | eclipse plugin build is broken with current eclipse versions | Blocker | contrib/eclipse-plugin | Doug Cutting | Doug Cutting | +| [HADOOP-3865](https://issues.apache.org/jira/browse/HADOOP-3865) | SecondaryNameNode runs out of memory | Blocker | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3855](https://issues.apache.org/jira/browse/HADOOP-3855) | Fix import of MiniDFSCluster in TestCompressedEmptyMapOutputs.java | Blocker | test | Arun C Murthy | Arun C Murthy | +| [HADOOP-3827](https://issues.apache.org/jira/browse/HADOOP-3827) | Jobs with empty map-outputs and intermediate compression fail | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3806](https://issues.apache.org/jira/browse/HADOOP-3806) | Remove debug message from Quicksort | Trivial | . | Chris Douglas | | +| [HADOOP-3798](https://issues.apache.org/jira/browse/HADOOP-3798) | compile-core-test fails to compile | Blocker | build | Mukund Madhugiri | Mukund Madhugiri | +| [HADOOP-3794](https://issues.apache.org/jira/browse/HADOOP-3794) | KFS implementation needs to return directory modification time | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3776](https://issues.apache.org/jira/browse/HADOOP-3776) | NPE in NameNode with unknown blocks | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3774](https://issues.apache.org/jira/browse/HADOOP-3774) | Typos in shell output | Blocker | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3762](https://issues.apache.org/jira/browse/HADOOP-3762) | Task tracker died due to OOM | Blocker | . | Runping Qi | Doug Cutting | +| [HADOOP-3755](https://issues.apache.org/jira/browse/HADOOP-3755) | the gridmix scripts do not work with hod 0.4 | Major | . | Runping Qi | Runping Qi | +| [HADOOP-3752](https://issues.apache.org/jira/browse/HADOOP-3752) | Audit logging fails to record rename | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3743](https://issues.apache.org/jira/browse/HADOOP-3743) | -libjars, -files and -archives options do not work with 0.18 | Blocker | . | Mahadev konar | Amareshwari Sriramadasu | +| [HADOOP-3737](https://issues.apache.org/jira/browse/HADOOP-3737) | CompressedWritable throws OutOfMemoryError | Major | io | Grant Glouser | Grant Glouser | +| [HADOOP-3724](https://issues.apache.org/jira/browse/HADOOP-3724) | Namenode does not start due to exception throw while saving Image | Blocker | . | Lohit Vijayarenu | dhruba borthakur | +| [HADOOP-3718](https://issues.apache.org/jira/browse/HADOOP-3718) | KFS: write(int v) API writes out an integer rather than a byte | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3716](https://issues.apache.org/jira/browse/HADOOP-3716) | KFS listStatus() returns NULL on empty directories | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3706](https://issues.apache.org/jira/browse/HADOOP-3706) | CompositeInputFormat: Unable to wrap custom InputFormats | Major | . | Jingkei Ly | Jingkei Ly | +| [HADOOP-3693](https://issues.apache.org/jira/browse/HADOOP-3693) | Fix documentation for Archives, distcp and native libraries | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3692](https://issues.apache.org/jira/browse/HADOOP-3692) | Fix documentation for Cluster setup and Quick start guides | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3691](https://issues.apache.org/jira/browse/HADOOP-3691) | Fix mapred docs | Blocker | documentation | Amareshwari Sriramadasu | Jothi Padmanabhan | +| [HADOOP-3688](https://issues.apache.org/jira/browse/HADOOP-3688) | Fix up HDFS docs | Blocker | . | Robert Chansler | Robert Chansler | +| [HADOOP-3677](https://issues.apache.org/jira/browse/HADOOP-3677) | Problems with generation stamp upgrade | Blocker | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-3673](https://issues.apache.org/jira/browse/HADOOP-3673) | Deadlock in Datanode RPC servers | Blocker | . | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3670](https://issues.apache.org/jira/browse/HADOOP-3670) | JobTracker running out of heap space | Blocker | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-3668](https://issues.apache.org/jira/browse/HADOOP-3668) | Clean up HOD documentation | Blocker | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3653](https://issues.apache.org/jira/browse/HADOOP-3653) | test-patch target not working on hudson.zones.apache.org due to HADOOP-3480 | Blocker | . | Nigel Daley | Brice Arnould | +| [HADOOP-3649](https://issues.apache.org/jira/browse/HADOOP-3649) | ArrayIndexOutOfBounds in FSNamesystem.getBlockLocationsInternal | Blocker | . | Arun C Murthy | Lohit Vijayarenu | +| [HADOOP-3647](https://issues.apache.org/jira/browse/HADOOP-3647) | Corner-case in IFile leads to failed tasks | Blocker | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3645](https://issues.apache.org/jira/browse/HADOOP-3645) | MetricsTimeVaryingRate returns wrong value for metric\_avg\_time | Blocker | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3639](https://issues.apache.org/jira/browse/HADOOP-3639) | Exception when closing DFSClient while multiple files are open | Blocker | . | Benjamin Gufler | Benjamin Gufler | +| [HADOOP-3635](https://issues.apache.org/jira/browse/HADOOP-3635) | Uncaught exception in DataBlockScanner | Blocker | . | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-3630](https://issues.apache.org/jira/browse/HADOOP-3630) | CompositeRecordReader: key and values can be in uninitialized state if files being joined have no records | Major | . | Jingkei Ly | Chris Douglas | +| [HADOOP-3615](https://issues.apache.org/jira/browse/HADOOP-3615) | DatanodeProtocol.versionID should be 16L | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3606](https://issues.apache.org/jira/browse/HADOOP-3606) | Update streaming documentation | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3604](https://issues.apache.org/jira/browse/HADOOP-3604) | Reduce stuck at shuffling phase | Blocker | . | Runping Qi | Arun C Murthy | +| [HADOOP-3603](https://issues.apache.org/jira/browse/HADOOP-3603) | Setting spill threshold to 100% fails to detect spill for records | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3597](https://issues.apache.org/jira/browse/HADOOP-3597) | SortValidator always uses the default file system irrespective of the actual input | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-3593](https://issues.apache.org/jira/browse/HADOOP-3593) | Update MapRed tutorial | Blocker | documentation | Devaraj Das | Devaraj Das | +| [HADOOP-3590](https://issues.apache.org/jira/browse/HADOOP-3590) | Null pointer exception in JobTracker when the task tracker is not yet resolved | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-3588](https://issues.apache.org/jira/browse/HADOOP-3588) | Bug report for archives | Blocker | . | Hairong Kuang | Mahadev konar | +| [HADOOP-3586](https://issues.apache.org/jira/browse/HADOOP-3586) | keep combiner backward compatible with earlier versions of hadoop | Blocker | . | Olga Natkovich | Chris Douglas | +| [HADOOP-3580](https://issues.apache.org/jira/browse/HADOOP-3580) | Using a har file as input for the Sort example fails | Blocker | . | Jothi Padmanabhan | Mahadev konar | +| [HADOOP-3576](https://issues.apache.org/jira/browse/HADOOP-3576) | hadoop dfs -mv throws NullPointerException | Blocker | . | Lohit Vijayarenu | Tsz Wo Nicholas Sze | +| [HADOOP-3575](https://issues.apache.org/jira/browse/HADOOP-3575) | clover target broken after src restructuring | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-3571](https://issues.apache.org/jira/browse/HADOOP-3571) | ArrayIndexOutOfBoundsException in BlocksMap$BlockInfo.setPrevious | Blocker | . | Tsz Wo Nicholas Sze | Konstantin Shvachko | +| [HADOOP-3564](https://issues.apache.org/jira/browse/HADOOP-3564) | Sometime after successful hod allocation datanode fails to come up with java.net.BindException for dfs.datanode.ipc.address | Blocker | contrib/hod | Karam Singh | Vinod Kumar Vavilapalli | +| [HADOOP-3561](https://issues.apache.org/jira/browse/HADOOP-3561) | With trash enabled, 'hadoop fs -rmr .' still fully deletes the working dir | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-3560](https://issues.apache.org/jira/browse/HADOOP-3560) | Archvies sometimes create empty part files. | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3559](https://issues.apache.org/jira/browse/HADOOP-3559) | test-libhdfs fails on linux | Blocker | . | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3552](https://issues.apache.org/jira/browse/HADOOP-3552) | forrest doc for hadoop commands | Blocker | documentation | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-3548](https://issues.apache.org/jira/browse/HADOOP-3548) | The tools.jar is not included in the distribution | Blocker | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-3546](https://issues.apache.org/jira/browse/HADOOP-3546) | TaskTracker re-initialization gets stuck in cleaning up | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3545](https://issues.apache.org/jira/browse/HADOOP-3545) | archive is failing with "Illegal Capacity" error | Blocker | . | Jothi Padmanabhan | Mahadev konar | +| [HADOOP-3544](https://issues.apache.org/jira/browse/HADOOP-3544) | The command "archive" is missing in the example in docs/hadoop\_archives.html (and pdf) | Blocker | documentation | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-3542](https://issues.apache.org/jira/browse/HADOOP-3542) | Hadoop archives should not create \_logs file in the final archive directory. | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3539](https://issues.apache.org/jira/browse/HADOOP-3539) | Cygwin: cygpath displays an error message in running bin/hadoop script | Blocker | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3537](https://issues.apache.org/jira/browse/HADOOP-3537) | Datanode not starting up with java.lang.StringIndexOutOfBoundsException in NetworkTopology.remove | Blocker | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-3534](https://issues.apache.org/jira/browse/HADOOP-3534) | The namenode ignores ioexceptions in close | Blocker | . | Owen O'Malley | Tsz Wo Nicholas Sze | +| [HADOOP-3533](https://issues.apache.org/jira/browse/HADOOP-3533) | The api to JobTracker and TaskTracker have changed incompatibly | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3531](https://issues.apache.org/jira/browse/HADOOP-3531) | Hod does not report job tracker failure on hod client side when job tracker fails to come up | Blocker | contrib/hod | Karam Singh | Hemanth Yamijala | +| [HADOOP-3528](https://issues.apache.org/jira/browse/HADOOP-3528) | Metrics FilesCreated and files\_deleted metrics do not match. | Blocker | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3523](https://issues.apache.org/jira/browse/HADOOP-3523) | [HOD] If a job does not exist in Torque's list of jobs, HOD allocate on previously allocated directory fails. | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3521](https://issues.apache.org/jira/browse/HADOOP-3521) | Hadoop mapreduce task metrics, unable to send metrics data. | Blocker | . | Eric Yang | Arun C Murthy | +| [HADOOP-3520](https://issues.apache.org/jira/browse/HADOOP-3520) | Generation stamp upgrade fails TestDFSUpgradeFromImage | Blocker | . | Konstantin Shvachko | dhruba borthakur | +| [HADOOP-3519](https://issues.apache.org/jira/browse/HADOOP-3519) | NPE in DFS FileSystem rename | Blocker | . | Tom White | Hairong Kuang | +| [HADOOP-3517](https://issues.apache.org/jira/browse/HADOOP-3517) | The last InMemory merge may be missed | Blocker | . | Devaraj Das | Arun C Murthy | +| [HADOOP-3516](https://issues.apache.org/jira/browse/HADOOP-3516) | TestHarFileSystem.testArchives fails with NullPointerException | Blocker | test | Nigel Daley | Subru Krishnan | +| [HADOOP-3513](https://issues.apache.org/jira/browse/HADOOP-3513) | Improve NNThroughputBenchmark log messages. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3511](https://issues.apache.org/jira/browse/HADOOP-3511) | Namenode should not restore the root's quota if the quota was not in the image | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3509](https://issues.apache.org/jira/browse/HADOOP-3509) | FSNamesystem.close() throws NullPointerException | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3505](https://issues.apache.org/jira/browse/HADOOP-3505) | omissions in HOD documentation | Blocker | contrib/hod, documentation | Ari Rabkin | Vinod Kumar Vavilapalli | +| [HADOOP-3503](https://issues.apache.org/jira/browse/HADOOP-3503) | Race condition when client and namenode start block recovery simultaneously | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3496](https://issues.apache.org/jira/browse/HADOOP-3496) | TestHarFileSystem.testArchives fails | Blocker | fs | Amareshwari Sriramadasu | Tom White | +| [HADOOP-3493](https://issues.apache.org/jira/browse/HADOOP-3493) | TestStreamingFailure fails. | Major | . | Amareshwari Sriramadasu | Lohit Vijayarenu | +| [HADOOP-3491](https://issues.apache.org/jira/browse/HADOOP-3491) | Name-node shutdown causes InterruptedException in ResolutionMonitor | Major | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-3489](https://issues.apache.org/jira/browse/HADOOP-3489) | NPE in SafeModeMonitor | Major | . | Konstantin Shvachko | Lohit Vijayarenu | +| [HADOOP-3487](https://issues.apache.org/jira/browse/HADOOP-3487) | Balancer should not allocate a thread per block move | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3480](https://issues.apache.org/jira/browse/HADOOP-3480) | Need to update Eclipse template to reflect current trunk | Blocker | build | Tsz Wo Nicholas Sze | Brice Arnould | +| [HADOOP-3476](https://issues.apache.org/jira/browse/HADOOP-3476) | Code cleanup needed in fuse-dfs | Major | . | Tsz Wo Nicholas Sze | Pete Wyckoff | +| [HADOOP-3471](https://issues.apache.org/jira/browse/HADOOP-3471) | TestIndexedSort sometimes fails | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-3468](https://issues.apache.org/jira/browse/HADOOP-3468) | Compile error: FTPFileSystem.java:26: cannot access org.apache.commons.net.ftp.FTP | Blocker | fs | Tsz Wo Nicholas Sze | Ankur | +| [HADOOP-3463](https://issues.apache.org/jira/browse/HADOOP-3463) | hadoop scripts don't change directory to hadoop\_home | Critical | scripts | Owen O'Malley | Owen O'Malley | +| [HADOOP-3454](https://issues.apache.org/jira/browse/HADOOP-3454) | Text.find incorrectly searches beyond the end of the buffer | Major | . | Chad Whipkey | Chad Whipkey | +| [HADOOP-3453](https://issues.apache.org/jira/browse/HADOOP-3453) | ipc.Client.close() throws NullPointerException | Major | ipc | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HADOOP-3451](https://issues.apache.org/jira/browse/HADOOP-3451) | test-libhdfs fails on Linux | Blocker | test | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3443](https://issues.apache.org/jira/browse/HADOOP-3443) | map outputs should not be renamed between partitions | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3440](https://issues.apache.org/jira/browse/HADOOP-3440) | TaskRunner creates a symlink with name 'null' if a file is added to DistributedCache without fragment | Minor | . | Abhijit Bagri | Devaraj Das | +| [HADOOP-3435](https://issues.apache.org/jira/browse/HADOOP-3435) | test-patch fail if sh != bash | Major | . | Brice Arnould | Brice Arnould | +| [HADOOP-3427](https://issues.apache.org/jira/browse/HADOOP-3427) | In ReduceTask::fetchOutputs, wait for result can be improved slightly | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-3424](https://issues.apache.org/jira/browse/HADOOP-3424) | the value returned from getPartition should be checked to make sure it is in the range 0..#reduces-1 | Major | . | Owen O'Malley | Chris Douglas | +| [HADOOP-3419](https://issues.apache.org/jira/browse/HADOOP-3419) | TestFsck fails once in a while on WINDOWS/LINUX | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3418](https://issues.apache.org/jira/browse/HADOOP-3418) | NameNode does not restart if parent directory of a "FileUnderConstruction" is deleted. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-3413](https://issues.apache.org/jira/browse/HADOOP-3413) | SequenceFile.Reader doesn't use the Serialization framework | Critical | io | Arun C Murthy | Tom White | +| [HADOOP-3410](https://issues.apache.org/jira/browse/HADOOP-3410) | KFS implementation needs to return file modification time | Minor | . | Sriram Rao | Sriram Rao | +| [HADOOP-3409](https://issues.apache.org/jira/browse/HADOOP-3409) | NameNode should save the root inode into fsimage | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3408](https://issues.apache.org/jira/browse/HADOOP-3408) | Change FSNamesytem status metrics to IntValue | Major | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3403](https://issues.apache.org/jira/browse/HADOOP-3403) | Job tracker's ExpireTackers thread gets NullPointerException if a tasktracker is lost. | Blocker | . | Amareshwari Sriramadasu | Arun C Murthy | +| [HADOOP-3401](https://issues.apache.org/jira/browse/HADOOP-3401) | Update FileBench to use the "work" directory for SequenceFileOutputFormat | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-3399](https://issues.apache.org/jira/browse/HADOOP-3399) | Debug log not removed in ipc client | Trivial | ipc | Raghu Angadi | Raghu Angadi | +| [HADOOP-3396](https://issues.apache.org/jira/browse/HADOOP-3396) | Unit test TestDatanodeBlockScanner fails on Windows | Critical | . | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3393](https://issues.apache.org/jira/browse/HADOOP-3393) | TestHDFSServerPorts fails on LINUX (NFS mounted directory) and on WINDOWS | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3391](https://issues.apache.org/jira/browse/HADOOP-3391) | HADOOP-3248 introduced a findbugs warning. | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3388](https://issues.apache.org/jira/browse/HADOOP-3388) | TestDatanodeBlockScanner failed while trying to corrupt replicas | Major | test | dhruba borthakur | dhruba borthakur | +| [HADOOP-3381](https://issues.apache.org/jira/browse/HADOOP-3381) | INode interlinks can multiply effect of memory leaks | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3376](https://issues.apache.org/jira/browse/HADOOP-3376) | [HOD] HOD should have a way to detect and deal with clusters that violate/exceed resource manager limits | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3375](https://issues.apache.org/jira/browse/HADOOP-3375) | Lease paths are sometimes not removed from LeaseManager.sortedLeasesByPath | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3371](https://issues.apache.org/jira/browse/HADOOP-3371) | MBeanUtil dumps stacktrace from registerMBean | Minor | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3365](https://issues.apache.org/jira/browse/HADOOP-3365) | SequenceFile.Sorter.MergeQueue.next does an unnecessary copy of the key | Major | io | Arun C Murthy | Devaraj Das | +| [HADOOP-3363](https://issues.apache.org/jira/browse/HADOOP-3363) | HDFS throws a InconsistentFSStateException when the name node starts up on a directory that isnt formatted | Blocker | . | Steve Loughran | Konstantin Shvachko | +| [HADOOP-3351](https://issues.apache.org/jira/browse/HADOOP-3351) | Fix history viewer | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3349](https://issues.apache.org/jira/browse/HADOOP-3349) | FSNamesystem.changeLease(src, dst) incorrectly updates the paths inside a lease | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3348](https://issues.apache.org/jira/browse/HADOOP-3348) | TestUrlStreamHandler hangs on LINUX | Major | fs | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3340](https://issues.apache.org/jira/browse/HADOOP-3340) | hadoop dfs metrics shows 0 | Major | metrics | Eric Yang | Lohit Vijayarenu | +| [HADOOP-3339](https://issues.apache.org/jira/browse/HADOOP-3339) | DFS Write pipeline does not detect defective datanode correctly if it times out. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3338](https://issues.apache.org/jira/browse/HADOOP-3338) | trunk doesn't compile after HADOOP-544 was committed | Blocker | . | Nigel Daley | Christophe Taton | +| [HADOOP-3337](https://issues.apache.org/jira/browse/HADOOP-3337) | Name-node fails to start because DatanodeInfo format changed. | Blocker | . | Konstantin Shvachko | Tsz Wo Nicholas Sze | +| [HADOOP-3335](https://issues.apache.org/jira/browse/HADOOP-3335) | 'make clean' in src/c++/libhdfs does 'rm -rf /\*' | Critical | build | Doug Cutting | Doug Cutting | +| [HADOOP-3333](https://issues.apache.org/jira/browse/HADOOP-3333) | job failing because of reassigning same tasktracker to failing tasks | Blocker | . | Christian Kunz | Jothi Padmanabhan | +| [HADOOP-3326](https://issues.apache.org/jira/browse/HADOOP-3326) | ReduceTask should not sleep for 200 ms while waiting for merge to finish | Major | . | Owen O'Malley | Sharad Agarwal | +| [HADOOP-3320](https://issues.apache.org/jira/browse/HADOOP-3320) | NPE in NetworkTopology.getDistance() | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-3318](https://issues.apache.org/jira/browse/HADOOP-3318) | Hadoop streaming doesn't recognize "Darwin" as an OS but Soylatte (OpenJDK port to Mac) reports that rather than "Mac OS X" | Major | . | Sam Pullara | Sam Pullara | +| [HADOOP-3313](https://issues.apache.org/jira/browse/HADOOP-3313) | RPC::Invoker makes unnecessary calls to System.currentTimeMillis | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-3309](https://issues.apache.org/jira/browse/HADOOP-3309) | Unit test fails on Windows: org.apache.hadoop.mapred.TestMiniMRDFSSort.unknown | Major | . | Mukund Madhugiri | Lohit Vijayarenu | +| [HADOOP-3301](https://issues.apache.org/jira/browse/HADOOP-3301) | Misleading error message when S3 URI contains hostname containing an underscore | Major | fs/s3 | Tom White | Tom White | +| [HADOOP-3300](https://issues.apache.org/jira/browse/HADOOP-3300) | FindBugs warnings in NetworkTopology | Major | . | Tom White | Tom White | +| [HADOOP-3299](https://issues.apache.org/jira/browse/HADOOP-3299) | org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException | Major | io | Jason | Chris Douglas | +| [HADOOP-3296](https://issues.apache.org/jira/browse/HADOOP-3296) | Some levels are skipped while creating the task cache in JobInProgress | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-3282](https://issues.apache.org/jira/browse/HADOOP-3282) | TestCheckpoint occasionally fails because of the port issues. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3272](https://issues.apache.org/jira/browse/HADOOP-3272) | Reduce redundant copy of Block object in BlocksMap.map hash map | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3269](https://issues.apache.org/jira/browse/HADOOP-3269) | NameNode doesn't startup when restarted after running an MR job | Blocker | . | Devaraj Das | Tsz Wo Nicholas Sze | +| [HADOOP-3268](https://issues.apache.org/jira/browse/HADOOP-3268) | TestUrlStreamHandler.testFileUrls fails on Windows | Major | test | Nigel Daley | Christophe Taton | +| [HADOOP-3259](https://issues.apache.org/jira/browse/HADOOP-3259) | Configuration.substituteVars() needs to handle security exceptions | Trivial | conf | Steve Loughran | Edward J. Yoon | +| [HADOOP-3248](https://issues.apache.org/jira/browse/HADOOP-3248) | Improve Namenode startup performance | Major | . | girish vaitheeswaran | dhruba borthakur | +| [HADOOP-3240](https://issues.apache.org/jira/browse/HADOOP-3240) | TestJobShell should not create files in the current directory | Blocker | test | Tsz Wo Nicholas Sze | Mahadev konar | +| [HADOOP-3232](https://issues.apache.org/jira/browse/HADOOP-3232) | Datanodes time out | Critical | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-3203](https://issues.apache.org/jira/browse/HADOOP-3203) | TaskTracker::localizeJob doesn't provide the correct size to LocalDirAllocator | Major | . | Chris Douglas | Amareshwari Sriramadasu | +| [HADOOP-3176](https://issues.apache.org/jira/browse/HADOOP-3176) | Change lease record when a open-for-write-file gets renamed | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3160](https://issues.apache.org/jira/browse/HADOOP-3160) | remove exists() from ClientProtocol and NameNode | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3151](https://issues.apache.org/jira/browse/HADOOP-3151) | Hod should have better error messages. | Major | contrib/hod | Arkady Borkovsky | Vinod Kumar Vavilapalli | +| [HADOOP-3135](https://issues.apache.org/jira/browse/HADOOP-3135) | if the 'mapred.system.dir' in the client jobconf is different from the JobTracker's value job submission fails | Critical | . | Alejandro Abdelnur | Subru Krishnan | +| [HADOOP-3130](https://issues.apache.org/jira/browse/HADOOP-3130) | Shuffling takes too long to get the last map output. | Major | . | Runping Qi | Amar Kamat | +| [HADOOP-3127](https://issues.apache.org/jira/browse/HADOOP-3127) | rm /user/\/.Trash/\_\_\_\_ only moves it back to .Trash | Minor | fs | Koji Noguchi | Brice Arnould | +| [HADOOP-3119](https://issues.apache.org/jira/browse/HADOOP-3119) | Text.getBytes() | Trivial | . | Andrew Gudkov | Tim Nelson | +| [HADOOP-3109](https://issues.apache.org/jira/browse/HADOOP-3109) | RPC should accepted connections even when rpc queue is full (ie undo part of HADOOP-2910) | Blocker | . | Sanjay Radia | Hairong Kuang | +| [HADOOP-3101](https://issues.apache.org/jira/browse/HADOOP-3101) | 'bin/hadoop job' should display the help and silently exit | Minor | . | Amar Kamat | Edward J. Yoon | +| [HADOOP-3085](https://issues.apache.org/jira/browse/HADOOP-3085) | pushMetric() method of various metric util classes should catch exceptions | Major | metrics | Runping Qi | Chris Douglas | +| [HADOOP-3076](https://issues.apache.org/jira/browse/HADOOP-3076) | [HOD] If a cluster directory is specified as a relative path, an existing script.exitcode file will not be deleted. | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2961](https://issues.apache.org/jira/browse/HADOOP-2961) | [HOD] Hod expects port info though external host is not mentioned. | Minor | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2930](https://issues.apache.org/jira/browse/HADOOP-2930) | make {start,stop}-balancer.sh work even if hadoop-daemon.sh isn't in the PATH | Trivial | scripts | Spiros Papadimitriou | Spiros Papadimitriou | +| [HADOOP-2928](https://issues.apache.org/jira/browse/HADOOP-2928) | Remove deprecated methods getContentLength() in ClientProtocol, NameNode, FileSystem, DistributedFileSystem and DFSClient | Blocker | . | Tsz Wo Nicholas Sze | Lohit Vijayarenu | +| [HADOOP-2905](https://issues.apache.org/jira/browse/HADOOP-2905) | fsck -move triggers NPE in namenode | Major | . | Michael Bieniosek | Lohit Vijayarenu | +| [HADOOP-2793](https://issues.apache.org/jira/browse/HADOOP-2793) | Links for worst performing shuffle tasks are broken in Analyze Job. | Minor | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-2669](https://issues.apache.org/jira/browse/HADOOP-2669) | DFS client lost lease during writing into DFS files | Major | . | Runping Qi | dhruba borthakur | +| [HADOOP-2565](https://issues.apache.org/jira/browse/HADOOP-2565) | DFSPath cache of FileStatus can become stale | Major | . | Doug Cutting | Tsz Wo Nicholas Sze | +| [HADOOP-2427](https://issues.apache.org/jira/browse/HADOOP-2427) | Cleanup of mapred.local.dir after maptask is complete | Major | . | Lohit Vijayarenu | Amareshwari Sriramadasu | +| [HADOOP-2393](https://issues.apache.org/jira/browse/HADOOP-2393) | TaskTracker locks up removing job files within a synchronized method | Critical | . | Joydeep Sen Sarma | Amareshwari Sriramadasu | +| [HADOOP-2294](https://issues.apache.org/jira/browse/HADOOP-2294) | In hdfs.h , the comment says you release the result of a hdfsListDirectory with a freehdfsFileInfo, but should say hdfsFreeFileInfo | Trivial | . | Dick King | Craig Macdonald | +| [HADOOP-2132](https://issues.apache.org/jira/browse/HADOOP-2132) | Killing successfully completed jobs moves them to failed | Critical | . | Srikanth Kakani | Jothi Padmanabhan | +| [HADOOP-2095](https://issues.apache.org/jira/browse/HADOOP-2095) | Reducer failed due to Out ofMemory | Major | . | Runping Qi | Arun C Murthy | +| [HADOOP-1318](https://issues.apache.org/jira/browse/HADOOP-1318) | Do not fail completed maps on lost tasktrackers if '-reducer NONE' is specified | Minor | . | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-236](https://issues.apache.org/jira/browse/HADOOP-236) | job tracker should refuse connection from a task tracker with a different version number | Major | . | Hairong Kuang | Sharad Agarwal | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3100](https://issues.apache.org/jira/browse/HADOOP-3100) | Develop tests to test the DFS command line interface | Major | test | Mukund Madhugiri | Mukund Madhugiri | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3541](https://issues.apache.org/jira/browse/HADOOP-3541) | Namespace recovery from the secondary image should be documented. | Blocker | documentation | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-2984](https://issues.apache.org/jira/browse/HADOOP-2984) | Distcp should have forrest documentation | Blocker | util | Owen O'Malley | Chris Douglas | +| [HADOOP-2632](https://issues.apache.org/jira/browse/HADOOP-2632) | Discussion of fsck operation in the permissions regime | Major | . | Robert Chansler | Robert Chansler | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md new file mode 100644 index 00000000000..81278d16780 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.0/RELEASENOTES.0.18.0.md @@ -0,0 +1,545 @@ + + +# Apache Hadoop 0.18.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-3837](https://issues.apache.org/jira/browse/HADOOP-3837) | *Major* | **hadop streaming does not use progress reporting to detect hung tasks** + +Changed streaming tasks to adhere to task timeout value specified in the job configuration. + + +--- + +* [HADOOP-3808](https://issues.apache.org/jira/browse/HADOOP-3808) | *Blocker* | **[HOD] Include job tracker RPC in notes attribute after job submission** + +Modified HOD to include the RPC port of the JobTracker in the 'notes' attribute of the resource manager. The RPC port is included as the string 'Mapred RPC Port:\'. Tools that depend on the value of the notes attribute must change to parse this new value. + + +--- + +* [HADOOP-3703](https://issues.apache.org/jira/browse/HADOOP-3703) | *Blocker* | **[HOD] logcondense needs to use the new pattern of output in hadoop dfs -lsr** + +Modified logcondense.py to use the new format of hadoop dfs -lsr output. This version of logcondense would not work with previous versions of Hadoop and hence is incompatible. + + +--- + +* [HADOOP-3683](https://issues.apache.org/jira/browse/HADOOP-3683) | *Major* | **Hadoop dfs metric FilesListed shows number of files listed instead of operations** + +Change FileListed to getNumGetListingOps and add CreateFileOps, DeleteFileOps and AddBlockOps metrics. + + +--- + +* [HADOOP-3677](https://issues.apache.org/jira/browse/HADOOP-3677) | *Blocker* | **Problems with generation stamp upgrade** + +Simplify generation stamp upgrade by making is a local upgrade on datandodes. Deleted distributed upgrade. + + +--- + +* [HADOOP-3665](https://issues.apache.org/jira/browse/HADOOP-3665) | *Minor* | **WritableComparator newKey() fails for NullWritable** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-3610](https://issues.apache.org/jira/browse/HADOOP-3610) | *Blocker* | **[HOD] HOD does not automatically create a cluster directory for the script option** + +Modified HOD to automatically create a cluster directory if the one specified with the script command does not exist. + + +--- + +* [HADOOP-3598](https://issues.apache.org/jira/browse/HADOOP-3598) | *Blocker* | **Map-Reduce framework needlessly creates temporary \_${taskid} directories for Maps** + +Changed Map-Reduce framework to no longer create temporary task output directories for staging outputs if staging outputs isn't necessary. ${mapred.out.dir}/\_temporary/\_${taskid} + + +--- + +* [HADOOP-3569](https://issues.apache.org/jira/browse/HADOOP-3569) | *Minor* | **KFS input stream read() returns 4 bytes instead of 1** + +Fixed KFS to have read() read and return 1 byte instead of 4. + + +--- + +* [HADOOP-3564](https://issues.apache.org/jira/browse/HADOOP-3564) | *Blocker* | **Sometime after successful hod allocation datanode fails to come up with java.net.BindException for dfs.datanode.ipc.address** + +Modifed HOD to generate the dfs.datanode.ipc.address parameter in the hadoop-site.xml of datanodes that it launches. + + +--- + +* [HADOOP-3512](https://issues.apache.org/jira/browse/HADOOP-3512) | *Major* | **Split map/reduce tools into separate jars** + +Separated Distcp, Logalyzer and Archiver into a tools jar. + + +--- + +* [HADOOP-3486](https://issues.apache.org/jira/browse/HADOOP-3486) | *Major* | **Change default for initial block report to 0 sec and document it in hadoop-defaults.xml** + +Changed the default value of dfs.blockreport.initialDelay to be 0 seconds. + + +--- + +* [HADOOP-3483](https://issues.apache.org/jira/browse/HADOOP-3483) | *Major* | **[HOD] Improvements with cluster directory handling** + +Modified HOD to create a cluster directory if one does not exist and to auto-deallocate a cluster while reallocating it, if it is already dead. + + +--- + +* [HADOOP-3464](https://issues.apache.org/jira/browse/HADOOP-3464) | *Major* | **[HOD] HOD can improve error messages by reporting failures on compute nodes back to hod client** + +Implemented a mechanism to transfer HOD errors that occur on compute nodes to the submit node running the HOD client, so users have good feedback on why an allocation failed. + + +--- + +* [HADOOP-3460](https://issues.apache.org/jira/browse/HADOOP-3460) | *Minor* | **SequenceFileAsBinaryOutputFormat** + +Created SequenceFileAsBinaryOutputFormat to write raw bytes as keys and values to a SequenceFile. + + +--- + +* [HADOOP-3459](https://issues.apache.org/jira/browse/HADOOP-3459) | *Major* | **Change dfs -ls listing to closely match format on Linux** + +Changed the output of the "fs -ls" command to more closely match familiar Linux format. Applications that parse the command output should be reviewed. + + +--- + +* [HADOOP-3452](https://issues.apache.org/jira/browse/HADOOP-3452) | *Minor* | **fsck exit code would be better if non-zero when FS corrupt** + +Changed exit status of fsck to report whether the files system is healthy or corrupt. + + +--- + +* [HADOOP-3429](https://issues.apache.org/jira/browse/HADOOP-3429) | *Major* | **Increase the buffersize for the streaming parent java process's streams** + +Increased the size of the buffer used in the communication between the Java task and the Streaming process to 128KB. + + +--- + +* [HADOOP-3427](https://issues.apache.org/jira/browse/HADOOP-3427) | *Major* | **In ReduceTask::fetchOutputs, wait for result can be improved slightly** + +Changed shuffle scheduler policy to wait for notifications from shuffle threads before scheduling more. + + +--- + +* [HADOOP-3417](https://issues.apache.org/jira/browse/HADOOP-3417) | *Major* | **JobClient should not have a static configuration for cli parsing** + +Removed the public class org.apache.hadoop.mapred.JobShell. +Command line options -libjars, -files and -archives are moved to GenericCommands. Thus applications have to implement org.apache.hadoop.util.Tool to use the options. + + +--- + +* [HADOOP-3405](https://issues.apache.org/jira/browse/HADOOP-3405) | *Major* | **Make mapred internal classes package-local** + +Refactored previously public classes MapTaskStatus, ReduceTaskStatus, JobSubmissionProtocol, CompletedJobStatusStore to be package local. + + +--- + +* [HADOOP-3390](https://issues.apache.org/jira/browse/HADOOP-3390) | *Major* | **Remove deprecated ClientProtocol.abandonFileInProgress()** + +Removed deprecated ClientProtocol.abandonFileInProgress(). + + +--- + +* [HADOOP-3379](https://issues.apache.org/jira/browse/HADOOP-3379) | *Blocker* | **Document the "stream.non.zero.exit.status.is.failure" knob for streaming** + +Set default value for configuration property "stream.non.zero.exit.status.is.failure" to be "true". + + +--- + +* [HADOOP-3376](https://issues.apache.org/jira/browse/HADOOP-3376) | *Major* | **[HOD] HOD should have a way to detect and deal with clusters that violate/exceed resource manager limits** + +Modified HOD client to look for specific messages related to resource limit overruns and take appropriate actions - such as either failing to allocate the cluster, or issuing a warning to the user. A tool is provided, specific to Maui and Torque, that will set these specific messages. + + +--- + +* [HADOOP-3366](https://issues.apache.org/jira/browse/HADOOP-3366) | *Major* | **Shuffle/Merge improvements** + +Improved shuffle so that all fetched map-outputs are kept in-memory before being merged by stalling the shuffle so that the in-memory merge executes and frees up memory for the shuffle. + + +--- + +* [HADOOP-3355](https://issues.apache.org/jira/browse/HADOOP-3355) | *Major* | **Configuration should accept decimal and hexadecimal values** + +Added support for hexadecimal values in Configuration + + +--- + +* [HADOOP-3339](https://issues.apache.org/jira/browse/HADOOP-3339) | *Major* | **DFS Write pipeline does not detect defective datanode correctly if it times out.** + +Improved failure handling of last Data Node in write pipeline. + + +--- + +* [HADOOP-3336](https://issues.apache.org/jira/browse/HADOOP-3336) | *Major* | **Direct a subset of namenode RPC events for audit logging** + +Added a log4j appender that emits events from FSNamesystem for audit logging + + +--- + +* [HADOOP-3329](https://issues.apache.org/jira/browse/HADOOP-3329) | *Major* | **DatanodeDescriptor objects stored in FSImage may be out dated.** + +Changed format of file system image to not store locations of last block. + + +--- + +* [HADOOP-3326](https://issues.apache.org/jira/browse/HADOOP-3326) | *Major* | **ReduceTask should not sleep for 200 ms while waiting for merge to finish** + +Changed fetchOutputs() so that LocalFSMerger and InMemFSMergeThread threads are spawned only once. The thread gets notified when something is ready for merge. The merge happens when thresholds are met. + + +--- + +* [HADOOP-3317](https://issues.apache.org/jira/browse/HADOOP-3317) | *Minor* | **add default port for hdfs namenode** + +Changed the default port for "hdfs:" URIs to be 8020, so that one may simply use URIs of the form "hdfs\://example.com/dir/file". + + +--- + +* [HADOOP-3310](https://issues.apache.org/jira/browse/HADOOP-3310) | *Major* | **Lease recovery for append** + +Implemented Lease Recovery to sync the last bock of a file. Added ClientDatanodeProtocol for client trigging block recovery. Changed DatanodeProtocol to support block synchronization. Changed InterDatanodeProtocol to support block update. + + +--- + +* [HADOOP-3307](https://issues.apache.org/jira/browse/HADOOP-3307) | *Major* | **Archives in Hadoop.** + +Introduced archive feature to Hadoop. A Map/Reduce job can be run to create an archive with indexes. A FileSystem abstraction is provided over the archive. + + +--- + +* [HADOOP-3299](https://issues.apache.org/jira/browse/HADOOP-3299) | *Major* | **org.apache.hadoop.mapred.join.CompositeInputFormat does not initialize TextInput format files with the configuration resulting in an NullPointerException** + +Changed the TextInputFormat and KeyValueTextInput classes to initialize the compressionCodecs member variable before dereferencing it. + + +--- + +* [HADOOP-3283](https://issues.apache.org/jira/browse/HADOOP-3283) | *Major* | **Need a mechanism for data nodes to update generation stamps.** + +Added an IPC server in DataNode and a new IPC protocol InterDatanodeProtocol. Added conf properties dfs.datanode.ipc.address and dfs.datanode.handler.count with defaults "0.0.0.0:50020" and 3, respectively. +Changed the serialization in DatanodeRegistration and DatanodeInfo, and therefore, updated the versionID in ClientProtocol, DatanodeProtocol, NamenodeProtocol. + + +--- + +* [HADOOP-3265](https://issues.apache.org/jira/browse/HADOOP-3265) | *Major* | **Remove deprecated API getFileCacheHints** + +Removed deprecated API getFileCacheHints + + +--- + +* [HADOOP-3246](https://issues.apache.org/jira/browse/HADOOP-3246) | *Major* | **FTP client over HDFS** + +Introduced an FTPFileSystem backed by Apache Commons FTPClient to directly store data into HDFS. + + +--- + +* [HADOOP-3232](https://issues.apache.org/jira/browse/HADOOP-3232) | *Critical* | **Datanodes time out** + +Changed 'du' command to run in a seperate thread so that it does not block user. + + +--- + +* [HADOOP-3230](https://issues.apache.org/jira/browse/HADOOP-3230) | *Major* | **Add command line access to named counters** + +Added command line tool "job -counter \ \ \" to access counters. + + +--- + +* [HADOOP-3226](https://issues.apache.org/jira/browse/HADOOP-3226) | *Major* | **Run combiner when merging spills from map output** + +Changed policy for running combiner. The combiner may be run multiple times as the map's output is sorted and merged. Additionally, it may be run on the reduce side as data is merged. The old semantics are available in Hadoop 0.18 if the user calls: +job.setCombineOnlyOnce(true); + + +--- + +* [HADOOP-3221](https://issues.apache.org/jira/browse/HADOOP-3221) | *Major* | **Need a "LineBasedTextInputFormat"** + +Added org.apache.hadoop.mapred.lib.NLineInputFormat ,which splits N lines of input as one split. N can be specified by configuration property "mapred.line.input.format.linespermap", which defaults to 1. + + +--- + +* [HADOOP-3193](https://issues.apache.org/jira/browse/HADOOP-3193) | *Minor* | **Discovery of corrupt block reported in name node log** + +Added reporter to FSNamesystem stateChangeLog, and a new metric to track the number of corrupted replicas. + + +--- + +* [HADOOP-3187](https://issues.apache.org/jira/browse/HADOOP-3187) | *Major* | **Quotas for name space management** + +Introduced directory quota as hard limits on the number of names in the tree rooted at that directory. An administrator may set quotas on individual directories explicitly. Newly created directories have no associated quota. File/directory creations fault if the quota would be exceeded. The attempt to set a quota faults if the directory would be in violation of the new quota. + + +--- + +* [HADOOP-3184](https://issues.apache.org/jira/browse/HADOOP-3184) | *Major* | **HOD gracefully exclude "bad" nodes during ring formation** + +Modified HOD to handle master (NameNode or JobTracker) failures on bad nodes by trying to bring them up on another node in the ring. Introduced new property ringmaster.max-master-failures to specify the maximum number of times a master is allowed to fail. + + +--- + +* [HADOOP-3177](https://issues.apache.org/jira/browse/HADOOP-3177) | *Major* | **Expose DFSOutputStream.fsync API though the FileSystem interface** + +Added a new public interface Syncable which declares the sync() operation. FSDataOutputStream implements Syncable. If the wrappedStream in FSDataOutputStream is Syncalbe, calling FSDataOutputStream.sync() is equivalent to call wrappedStream.sync(). Otherwise, FSDataOutputStream.sync() is a no-op. Both DistributedFileSystem and LocalFileSystem support the sync() operation. + + +--- + +* [HADOOP-3164](https://issues.apache.org/jira/browse/HADOOP-3164) | *Major* | **Use FileChannel.transferTo() when data is read from DataNode.** + +Changed data node to use FileChannel.tranferTo() to transfer block data. + + +--- + +* [HADOOP-3135](https://issues.apache.org/jira/browse/HADOOP-3135) | *Critical* | **if the 'mapred.system.dir' in the client jobconf is different from the JobTracker's value job submission fails** + +Changed job submission protocol to not allow submission if the client's value of mapred.system.dir does not match the job tracker's. Deprecated JobConf.getSystemDir(); use JobClient.getSystemDir(). + + +--- + +* [HADOOP-3113](https://issues.apache.org/jira/browse/HADOOP-3113) | *Major* | **DFSOututStream.flush() should flush data to real block file on DataNode.** + +Added sync() method to FSDataOutputStream to really, really persist data in HDFS. InterDatanodeProtocol to implement this feature. + + +--- + +* [HADOOP-3095](https://issues.apache.org/jira/browse/HADOOP-3095) | *Major* | **Validating input paths and creating splits is slow on S3** + +Added overloaded method getFileBlockLocations(FileStatus, long, long). This is an incompatible change for FileSystem implementations which override getFileBlockLocations(Path, long, long). They should have the signature of this method changed to getFileBlockLocations(FileStatus, long, long) to work correctly. + + +--- + +* [HADOOP-3061](https://issues.apache.org/jira/browse/HADOOP-3061) | *Major* | **Writable for single byte and double** + +Introduced ByteWritable and DoubleWritable (implementing WritableComparable) implementations for Byte and Double. + + +--- + +* [HADOOP-3058](https://issues.apache.org/jira/browse/HADOOP-3058) | *Minor* | **Hadoop DFS to report more replication metrics** + +Added FSNamesystem status metrics. + + +--- + +* [HADOOP-3035](https://issues.apache.org/jira/browse/HADOOP-3035) | *Major* | **Data nodes should inform the name-node about block crc errors.** + +Changed protocol for transferring blocks between data nodes to report corrupt blocks to data node for re-replication from a good replica. + + +--- + +* [HADOOP-3013](https://issues.apache.org/jira/browse/HADOOP-3013) | *Major* | **fsck to show (checksum) corrupted files** + +fsck reports corrupt blocks in the system. + + +--- + +* [HADOOP-2909](https://issues.apache.org/jira/browse/HADOOP-2909) | *Major* | **Improve IPC idle connection management** + +Removed property ipc.client.maxidletime from the default configuration. The allowed idle time is twice ipc.client.connection.maxidletime. + + +--- + +* [HADOOP-2867](https://issues.apache.org/jira/browse/HADOOP-2867) | *Major* | **Add a task's cwd to it's LD\_LIBRARY\_PATH** + +Added task's cwd to its LD\_LIBRARY\_PATH. + + +--- + +* [HADOOP-2865](https://issues.apache.org/jira/browse/HADOOP-2865) | *Major* | **FsShell.ls() should print file attributes first then the path name.** + +Changed the output of the "fs -ls" command to more closely match familiar Linux format. Additional changes were made by HADOOP-3459. Applications that parse the command output should be reviewed. + + +--- + +* [HADOOP-2797](https://issues.apache.org/jira/browse/HADOOP-2797) | *Critical* | **Withdraw CRC upgrade from HDFS** + +Withdrew the upgrade-to-CRC facility. HDFS will no longer support upgrades from versions without CRCs for block data. Users upgrading from version 0.13 or earlier must first upgrade to an intermediate (0.14, 0.15, 0.16, 0.17) version before doing upgrade to version 0.18 or later. + + +--- + +* [HADOOP-2703](https://issues.apache.org/jira/browse/HADOOP-2703) | *Minor* | **New files under lease (before close) still shows up as MISSING files/blocks in fsck** + +Changed fsck to ignore files opened for writing. Introduced new option "-openforwrite" to explicitly show open files. + + +--- + +* [HADOOP-2656](https://issues.apache.org/jira/browse/HADOOP-2656) | *Major* | **Support for upgrading existing cluster to facilitate appends to HDFS files** + +Associated a generation stamp with each block. On data nodes, the generation stamp is stored as part of the file name of the block's meta-data file. + + +--- + +* [HADOOP-2585](https://issues.apache.org/jira/browse/HADOOP-2585) | *Major* | **Automatic namespace recovery from the secondary image.** + +Improved management of replicas of the name space image. If all replicas on the Name Node are lost, the latest check point can be loaded from the secondary Name Node. Use parameter "-importCheckpoint" and specify the location with "fs.checkpoint.dir." The directory structure on the secondary Name Node has changed to match the primary Name Node. + + +--- + +* [HADOOP-2427](https://issues.apache.org/jira/browse/HADOOP-2427) | *Major* | **Cleanup of mapred.local.dir after maptask is complete** + +The current working directory of a task, i.e. ${mapred.local.dir}/taskTracker/jobcache/\/\/work is cleanedup, as soon as the task is finished. + + +--- + +* [HADOOP-2188](https://issues.apache.org/jira/browse/HADOOP-2188) | *Major* | **RPC should send a ping rather than use client timeouts** + +Replaced timeouts with pings to check that client connection is alive. Removed the property ipc.client.timeout from the default Hadoop configuration. Removed the metric RpcOpsDiscardedOPsNum. + + +--- + +* [HADOOP-2181](https://issues.apache.org/jira/browse/HADOOP-2181) | *Minor* | **Input Split details for maps should be logged** + +Added logging for input splits in job tracker log and job history log. Added web UI for viewing input splits in the job UI and history UI. + + +--- + +* [HADOOP-2132](https://issues.apache.org/jira/browse/HADOOP-2132) | *Critical* | **Killing successfully completed jobs moves them to failed** + +Change "job -kill" to only allow a job that is in the RUNNING or PREP state to be killed. + + +--- + +* [HADOOP-2095](https://issues.apache.org/jira/browse/HADOOP-2095) | *Major* | **Reducer failed due to Out ofMemory** + +Reduced in-memory copies of keys and values as they flow through the Map-Reduce framework. Changed the storage of intermediate map outputs to use new IFile instead of SequenceFile for better compression. + + +--- + +* [HADOOP-2065](https://issues.apache.org/jira/browse/HADOOP-2065) | *Major* | **Replication policy for corrupted block** + +Added "corrupt" flag to LocatedBlock to indicate that all replicas of the block thought to be corrupt. + + +--- + +* [HADOOP-2019](https://issues.apache.org/jira/browse/HADOOP-2019) | *Major* | **DistributedFileCache should support .tgz files in addition to jars and zip files** + +Added support for .tar, .tgz and .tar.gz files in DistributedCache. File sizes are limited to 2GB. + + +--- + +* [HADOOP-1915](https://issues.apache.org/jira/browse/HADOOP-1915) | *Minor* | **adding counters methods using String (as opposed to Enum)** + +Provided a new method to update counters. "incrCounter(String group, String counter, long amount)" + + +--- + +* [HADOOP-1702](https://issues.apache.org/jira/browse/HADOOP-1702) | *Major* | **Reduce buffer copies when data is written to DFS** + +Reduced buffer copies as data is written to HDFS. The order of sending data bytes and control information has changed, but this will not be observed by client applications. + + +--- + +* [HADOOP-1328](https://issues.apache.org/jira/browse/HADOOP-1328) | *Major* | **Hadoop Streaming needs to provide a way for the stream plugin to update global counters** + +Introduced a way for a streaming process to update global counters and status using stderr stream to emit information. Use "reporter:counter:\,\,\ " to update a counter. Use "reporter:status:\" to update status. + + +--- + +* [HADOOP-930](https://issues.apache.org/jira/browse/HADOOP-930) | *Major* | **Add support for reading regular (non-block-based) files from S3 in S3FileSystem** + +Added support for reading and writing native S3 files. Native S3 files are referenced using s3n URIs. See http://wiki.apache.org/hadoop/AmazonS3 for more details. + + +--- + +* [HADOOP-544](https://issues.apache.org/jira/browse/HADOOP-544) | *Major* | **Replace the job, tip and task ids with objects.** + +Introduced new classes JobID, TaskID and TaskAttemptID, which should be used instead of their string counterparts. Deprecated functions in JobClient, TaskReport, RunningJob, jobcontrol.Job and TaskCompletionEvent that use string arguments. Applications can use xxxID.toString() and xxxID.forName() methods to convert/restore objects to/from strings. + + +--- + +* [HADOOP-236](https://issues.apache.org/jira/browse/HADOOP-236) | *Major* | **job tracker should refuse connection from a task tracker with a different version number** + +Changed connection protocol job tracker and task tracker so that task tracker will not connect to a job tracker with a different build version. + + +--- + +* [HADOOP-4](https://issues.apache.org/jira/browse/HADOOP-4) | *Major* | **tool to mount dfs on linux** + +Introduced FUSE module for HDFS. Module allows mount of HDFS as a Unix filesystem, and optionally the export of that mount point to other machines. Writes are disabled. rmdir, mv, mkdir, rm are supported, but not cp, touch, and the like. Usage information is attached to the Jira record. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md new file mode 100644 index 00000000000..8fa57cbef58 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/CHANGES.0.18.1.md @@ -0,0 +1,74 @@ + + +# Apache Hadoop Changelog + +## Release 0.18.1 - 2008-09-17 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4145](https://issues.apache.org/jira/browse/HADOOP-4145) | [HOD] Support an accounting plugin script for HOD | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3934](https://issues.apache.org/jira/browse/HADOOP-3934) | Update log4j from 1.2.13 to 1.2.15 | Blocker | . | Don Kim | Owen O'Malley | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4161](https://issues.apache.org/jira/browse/HADOOP-4161) | [HOD] Uncaught exceptions can potentially hang hod-client. | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-4060](https://issues.apache.org/jira/browse/HADOOP-4060) | [HOD] Make HOD to roll log files on the client | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-4046](https://issues.apache.org/jira/browse/HADOOP-4046) | WritableComparator's constructor should be protected instead of private | Major | io | Owen O'Malley | Chris Douglas | +| [HADOOP-4040](https://issues.apache.org/jira/browse/HADOOP-4040) | Remove the hardcoded ipc.client.connection.maxidletime setting from the TaskTracker.Child.main() | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4037](https://issues.apache.org/jira/browse/HADOOP-4037) | HadoopQA contrib -1 comments due to inconsistency in eclipse plugin declared jars | Blocker | . | Nigel Daley | Nigel Daley | +| [HADOOP-3995](https://issues.apache.org/jira/browse/HADOOP-3995) | renameTo(src, dst) does not restore src name in case of quota failure. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3940](https://issues.apache.org/jira/browse/HADOOP-3940) | Reduce often attempts in memory merge with no work | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-3821](https://issues.apache.org/jira/browse/HADOOP-3821) | SequenceFile's Reader.decompressorPool or Writer.decompressorPool gets into an inconsistent state when calling close() more than once | Major | io | Peter Voss | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md new file mode 100644 index 00000000000..136744359d6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.1/RELEASENOTES.0.18.1.md @@ -0,0 +1,38 @@ + + +# Apache Hadoop 0.18.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-4060](https://issues.apache.org/jira/browse/HADOOP-4060) | *Blocker* | **[HOD] Make HOD to roll log files on the client** + +HOD client was modified to roll over client logs being written to the cluster directory. A new configuration parameter, hod.log-rollover-count, was introduced to specify how many rollover log files to retain. + + +--- + +* [HADOOP-3995](https://issues.apache.org/jira/browse/HADOOP-3995) | *Blocker* | **renameTo(src, dst) does not restore src name in case of quota failure.** + +In case of quota failure on HDFS, rename does not restore source filename. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/CHANGES.0.18.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/CHANGES.0.18.2.md new file mode 100644 index 00000000000..774cfca1e09 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/CHANGES.0.18.2.md @@ -0,0 +1,88 @@ + + +# Apache Hadoop Changelog + +## Release 0.18.2 - 2008-11-03 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4228](https://issues.apache.org/jira/browse/HADOOP-4228) | dfs datanode metrics, bytes\_read, bytes\_written overflows due to incorrect type used. | Blocker | metrics | Eric Yang | Eric Yang | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-2421](https://issues.apache.org/jira/browse/HADOOP-2421) | Release JDiff report of changes between different versions of Hadoop | Minor | documentation | Nigel Daley | Doug Cutting | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4533](https://issues.apache.org/jira/browse/HADOOP-4533) | HDFS client of hadoop 0.18.1 and HDFS server 0.18.2 (0.18 branch) not compatible | Blocker | . | Runping Qi | Hairong Kuang | +| [HADOOP-4526](https://issues.apache.org/jira/browse/HADOOP-4526) | fsck failing with NullPointerException (return value 0) | Major | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-4517](https://issues.apache.org/jira/browse/HADOOP-4517) | unstable dfs when running jobs on 0.18.1 | Blocker | . | Christian Kunz | Tsz Wo Nicholas Sze | +| [HADOOP-4483](https://issues.apache.org/jira/browse/HADOOP-4483) | getBlockArray in DatanodeDescriptor does not honor passed in maxblocks value | Critical | . | Ahad Rana | Ahad Rana | +| [HADOOP-4469](https://issues.apache.org/jira/browse/HADOOP-4469) | ant jar file not being included in tar distribution | Blocker | build | Nigel Daley | Nigel Daley | +| [HADOOP-4467](https://issues.apache.org/jira/browse/HADOOP-4467) | SerializationFactory should use current context ClassLoader | Blocker | . | Chris K Wensel | Chris K Wensel | +| [HADOOP-4407](https://issues.apache.org/jira/browse/HADOOP-4407) | HADOOP-4395 should use a Java 1.5 API for 0.18 | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4399](https://issues.apache.org/jira/browse/HADOOP-4399) | fuse-dfs per FD context is not thread safe and can cause segfaults and corruptions | Blocker | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4395](https://issues.apache.org/jira/browse/HADOOP-4395) | Reloading FSImage and FSEditLog may erase user and group information | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4369](https://issues.apache.org/jira/browse/HADOOP-4369) | Metric Averages are not averages | Blocker | metrics | Brian Bockelman | Brian Bockelman | +| [HADOOP-4351](https://issues.apache.org/jira/browse/HADOOP-4351) | ArrayIndexOutOfBoundsException during fsck | Blocker | . | Brian Bockelman | Hairong Kuang | +| [HADOOP-4340](https://issues.apache.org/jira/browse/HADOOP-4340) | "hadoop jar" always returns exit code 0 (success) to the shell when jar throws a fatal exception | Major | . | David Litster | Arun C Murthy | +| [HADOOP-4326](https://issues.apache.org/jira/browse/HADOOP-4326) | ChecksumFileSystem does not override all create(...) methods | Blocker | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4314](https://issues.apache.org/jira/browse/HADOOP-4314) | TestReplication fails quite often | Blocker | test | Raghu Angadi | Raghu Angadi | +| [HADOOP-4292](https://issues.apache.org/jira/browse/HADOOP-4292) | append() does not work for LocalFileSystem | Blocker | fs | Raghu Angadi | Hairong Kuang | +| [HADOOP-4277](https://issues.apache.org/jira/browse/HADOOP-4277) | Checksum verification is disabled for LocalFS | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4271](https://issues.apache.org/jira/browse/HADOOP-4271) | Bug in FSInputChecker makes it possible to read from an invalid buffer | Blocker | fs | Ning Li | Ning Li | +| [HADOOP-3914](https://issues.apache.org/jira/browse/HADOOP-3914) | checksumOk implementation in DFSClient can break applications | Blocker | . | Christian Kunz | Christian Kunz | +| [HADOOP-3786](https://issues.apache.org/jira/browse/HADOOP-3786) | Changes in HOD documentation | Blocker | documentation | Suman Sehgal | Vinod Kumar Vavilapalli | +| [HADOOP-3614](https://issues.apache.org/jira/browse/HADOOP-3614) | TestLeaseRecovery fails when run with assertions enabled. | Blocker | . | Konstantin Shvachko | Tsz Wo Nicholas Sze | +| [HADOOP-3217](https://issues.apache.org/jira/browse/HADOOP-3217) | [HOD] Be less agressive when querying job status from resource manager. | Blocker | contrib/hod | Hemanth Yamijala | Hemanth Yamijala | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4403](https://issues.apache.org/jira/browse/HADOOP-4403) | TestLeaseRecovery.testBlockSynchronization failed on trunk | Blocker | test | Hemanth Yamijala | Tsz Wo Nicholas Sze | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/RELEASENOTES.0.18.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/RELEASENOTES.0.18.2.md new file mode 100644 index 00000000000..a9ec7c35c6f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.2/RELEASENOTES.0.18.2.md @@ -0,0 +1,59 @@ + + +# Apache Hadoop 0.18.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-4467](https://issues.apache.org/jira/browse/HADOOP-4467) | *Blocker* | **SerializationFactory should use current context ClassLoader** + +SerializationFactory now uses the current context ClassLoader allowing for user supplied Serialization instances. + + +--- + +* [HADOOP-4314](https://issues.apache.org/jira/browse/HADOOP-4314) | *Blocker* | **TestReplication fails quite often** + +Simulated Datanodes should not include blocks that are still being written in their block report. + + +--- + +* [HADOOP-4277](https://issues.apache.org/jira/browse/HADOOP-4277) | *Blocker* | **Checksum verification is disabled for LocalFS** + +Checksum verification was mistakenly disabled for LocalFileSystem. + + +--- + +* [HADOOP-4271](https://issues.apache.org/jira/browse/HADOOP-4271) | *Blocker* | **Bug in FSInputChecker makes it possible to read from an invalid buffer** + +Checksum input stream can sometimes return invalid data to the user. + + +--- + +* [HADOOP-4228](https://issues.apache.org/jira/browse/HADOOP-4228) | *Blocker* | **dfs datanode metrics, bytes\_read, bytes\_written overflows due to incorrect type used.** + +Changed bytes\_read, bytes\_written to type long to prevent metrics overflow. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/CHANGES.0.18.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/CHANGES.0.18.3.md new file mode 100644 index 00000000000..97d64768a29 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/CHANGES.0.18.3.md @@ -0,0 +1,115 @@ + + +# Apache Hadoop Changelog + +## Release 0.18.3 - 2009-01-29 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4997](https://issues.apache.org/jira/browse/HADOOP-4997) | workaround for tmp file handling on DataNodes in 0.18 (HADOOP-4663) | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4659](https://issues.apache.org/jira/browse/HADOOP-4659) | Root cause of connection failure is being lost to code that uses it for delaying startup | Blocker | ipc | Steve Loughran | Steve Loughran | +| [HADOOP-4061](https://issues.apache.org/jira/browse/HADOOP-4061) | Large number of decommission freezes the Namenode | Major | . | Koji Noguchi | Tsz Wo Nicholas Sze | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3780](https://issues.apache.org/jira/browse/HADOOP-3780) | JobTracker should synchronously resolve the tasktracker's network location when the tracker registers | Major | . | Amar Kamat | Amar Kamat | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5077](https://issues.apache.org/jira/browse/HADOOP-5077) | JavaDoc errors in 0.18.3 | Blocker | util | Raghu Angadi | Raghu Angadi | +| [HADOOP-4983](https://issues.apache.org/jira/browse/HADOOP-4983) | Job counters sometimes go down as tasks run without task failures | Critical | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-4971](https://issues.apache.org/jira/browse/HADOOP-4971) | Block report times from datanodes could converge to same time. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4961](https://issues.apache.org/jira/browse/HADOOP-4961) | ConcurrentModificationException in lease recovery of empty files. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4951](https://issues.apache.org/jira/browse/HADOOP-4951) | Lease monitor does not own the LeaseManager lock in changing leases. | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4935](https://issues.apache.org/jira/browse/HADOOP-4935) | Manual leaving of safe mode may lead to data lost | Major | . | Hairong Kuang | Konstantin Shvachko | +| [HADOOP-4924](https://issues.apache.org/jira/browse/HADOOP-4924) | Race condition in re-init of TaskTracker | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4919](https://issues.apache.org/jira/browse/HADOOP-4919) | [HOD] Provide execute access to JT history directory path for group | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4910](https://issues.apache.org/jira/browse/HADOOP-4910) | NameNode should exclude corrupt replicas when choosing excessive replicas to delete | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4904](https://issues.apache.org/jira/browse/HADOOP-4904) | Deadlock while leaving safe mode. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4840](https://issues.apache.org/jira/browse/HADOOP-4840) | TestNodeCount sometimes fails with NullPointerException | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4822](https://issues.apache.org/jira/browse/HADOOP-4822) | 0.18 cannot be compiled in Java 5. | Blocker | util | Tsz Wo Nicholas Sze | | +| [HADOOP-4810](https://issues.apache.org/jira/browse/HADOOP-4810) | Data lost at cluster startup time | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4806](https://issues.apache.org/jira/browse/HADOOP-4806) | HDFS rename does not work correctly if src contains Java regular expression special characters | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4797](https://issues.apache.org/jira/browse/HADOOP-4797) | RPC Server can leave a lot of direct buffers | Blocker | ipc | Raghu Angadi | Raghu Angadi | +| [HADOOP-4795](https://issues.apache.org/jira/browse/HADOOP-4795) | Lease monitor may get into an infinite loop | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4778](https://issues.apache.org/jira/browse/HADOOP-4778) | Check for zero size block meta file when updating a block. | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4746](https://issues.apache.org/jira/browse/HADOOP-4746) | Job output directory should be normalized | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4742](https://issues.apache.org/jira/browse/HADOOP-4742) | Mistake delete replica in hadoop 0.18.1 | Blocker | . | Wang Xu | Wang Xu | +| [HADOOP-4734](https://issues.apache.org/jira/browse/HADOOP-4734) | Some lease recovery codes in 0.19 or trunk should also be committed in 0.18. | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4726](https://issues.apache.org/jira/browse/HADOOP-4726) | documentation typos: "the the" | Minor | documentation | Tsz Wo Nicholas Sze | Edward J. Yoon | +| [HADOOP-4717](https://issues.apache.org/jira/browse/HADOOP-4717) | Removal of default port# in NameNode.getUri() cause a map/reduce job failed to prompt temporay output | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4714](https://issues.apache.org/jira/browse/HADOOP-4714) | map tasks timing out during merge phase | Major | . | Christian Kunz | Jothi Padmanabhan | +| [HADOOP-4713](https://issues.apache.org/jira/browse/HADOOP-4713) | librecordio does not scale to large records | Blocker | record | Christian Kunz | Christian Kunz | +| [HADOOP-4703](https://issues.apache.org/jira/browse/HADOOP-4703) | DataNode.createInterDataNodeProtocolProxy should not wait for proxy forever while recovering lease | Major | . | Hairong Kuang | Tsz Wo Nicholas Sze | +| [HADOOP-4702](https://issues.apache.org/jira/browse/HADOOP-4702) | Failed block replication leaves an incomplete block in receiver's tmp data directory | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4679](https://issues.apache.org/jira/browse/HADOOP-4679) | Datanode prints tons of log messages: Waiting for threadgroup to exit, active theads is XX | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4654](https://issues.apache.org/jira/browse/HADOOP-4654) | remove temporary output directory of failed tasks | Major | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-4647](https://issues.apache.org/jira/browse/HADOOP-4647) | NamenodeFsck creates a new DFSClient but never closes it | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4643](https://issues.apache.org/jira/browse/HADOOP-4643) | NameNode should exclude excessive replicas when counting live replicas for a block | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4635](https://issues.apache.org/jira/browse/HADOOP-4635) | Memory leak ? | Blocker | . | Marc-Olivier Fleury | Pete Wyckoff | +| [HADOOP-4620](https://issues.apache.org/jira/browse/HADOOP-4620) | Streaming mapper never completes if the mapper does not write to stdout | Major | . | Runping Qi | Ravi Gummadi | +| [HADOOP-4616](https://issues.apache.org/jira/browse/HADOOP-4616) | assertion makes fuse-dfs exit when reading incomplete data | Blocker | . | Marc-Olivier Fleury | Pete Wyckoff | +| [HADOOP-4614](https://issues.apache.org/jira/browse/HADOOP-4614) | "Too many open files" error while processing a large gzip file | Blocker | . | Abdul Qadeer | Yuri Pradkin | +| [HADOOP-4613](https://issues.apache.org/jira/browse/HADOOP-4613) | browseBlock.jsp does not generate "genstamp" property. | Major | . | Konstantin Shvachko | Johan Oskarsson | +| [HADOOP-4610](https://issues.apache.org/jira/browse/HADOOP-4610) | Always calculate mis-replicated blocks when safe-mode is turned off. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4597](https://issues.apache.org/jira/browse/HADOOP-4597) | Under-replicated blocks are not calculated if the name-node is forced out of safe-mode. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4556](https://issues.apache.org/jira/browse/HADOOP-4556) | Block went missing | Major | . | Robert Chansler | Hairong Kuang | +| [HADOOP-4542](https://issues.apache.org/jira/browse/HADOOP-4542) | Fault in TestDistributedUpgrade | Minor | test | Robert Chansler | Raghu Angadi | +| [HADOOP-4499](https://issues.apache.org/jira/browse/HADOOP-4499) | DFSClient should invoke checksumOk only once. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4257](https://issues.apache.org/jira/browse/HADOOP-4257) | TestLeaseRecovery2.testBlockSynchronization failing. | Blocker | test | Vinod Kumar Vavilapalli | Tsz Wo Nicholas Sze | +| [HADOOP-3883](https://issues.apache.org/jira/browse/HADOOP-3883) | TestFileCreation fails once in a while | Blocker | test | Lohit Vijayarenu | Tsz Wo Nicholas Sze | +| [HADOOP-3121](https://issues.apache.org/jira/browse/HADOOP-3121) | dfs -lsr fail with "Could not get listing " | Minor | fs | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-1980](https://issues.apache.org/jira/browse/HADOOP-1980) | 'dfsadmin -safemode enter' should prevent the namenode from leaving safemode automatically after startup | Minor | . | Koji Noguchi | Konstantin Shvachko | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4824](https://issues.apache.org/jira/browse/HADOOP-4824) | Should not use File.setWritable(..) in 0.18 | Major | . | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HADOOP-4823](https://issues.apache.org/jira/browse/HADOOP-4823) | Should not use java.util.NavigableMap in 0.18 | Major | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4150](https://issues.apache.org/jira/browse/HADOOP-4150) | Include librecordio as part of the release | Blocker | build | Koji Noguchi | Giridharan Kesavan | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/RELEASENOTES.0.18.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/RELEASENOTES.0.18.3.md new file mode 100644 index 00000000000..7c767da261b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.3/RELEASENOTES.0.18.3.md @@ -0,0 +1,104 @@ + + +# Apache Hadoop 0.18.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5077](https://issues.apache.org/jira/browse/HADOOP-5077) | *Blocker* | **JavaDoc errors in 0.18.3** + +Fix couple of JavaDoc warnings. + + +--- + +* [HADOOP-4997](https://issues.apache.org/jira/browse/HADOOP-4997) | *Blocker* | **workaround for tmp file handling on DataNodes in 0.18 (HADOOP-4663)** + +Revert tmp files handling on DataNodes back to 0.17. sync() introduced in 0.18 has less gaurantees. + + +--- + +* [HADOOP-4971](https://issues.apache.org/jira/browse/HADOOP-4971) | *Blocker* | **Block report times from datanodes could converge to same time.** + +A long (unexpected) delay at datanodes could make subsequent block reports from many datanode at the same time. + + +--- + +* [HADOOP-4797](https://issues.apache.org/jira/browse/HADOOP-4797) | *Blocker* | **RPC Server can leave a lot of direct buffers** + +Improve how RPC server reads and writes large buffers. Avoids soft-leak of direct buffers and excess copies in NIO layer. + + +--- + +* [HADOOP-4679](https://issues.apache.org/jira/browse/HADOOP-4679) | *Major* | **Datanode prints tons of log messages: Waiting for threadgroup to exit, active theads is XX** + +1. Only datanode's offerService thread shutdown the datanode to avoid deadlock; +2. Datanode checks disk in case of failure on creating a block file. + + +--- + +* [HADOOP-4659](https://issues.apache.org/jira/browse/HADOOP-4659) | *Blocker* | **Root cause of connection failure is being lost to code that uses it for delaying startup** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-4635](https://issues.apache.org/jira/browse/HADOOP-4635) | *Blocker* | **Memory leak ?** + +fix memory leak of user/group information in fuse-dfs + + +--- + +* [HADOOP-4620](https://issues.apache.org/jira/browse/HADOOP-4620) | *Major* | **Streaming mapper never completes if the mapper does not write to stdout** + +This patch HADOOP-4620.patch +(1) solves the hanging problem on map side with empty input and nonempty output — this map task generates output properly to intermediate files similar to other map tasks. +(2) solves the problem of hanging reducer with empty input to reduce task and nonempty output — this reduce task doesn't generate output if input to reduce task is empty. + + +--- + +* [HADOOP-4542](https://issues.apache.org/jira/browse/HADOOP-4542) | *Minor* | **Fault in TestDistributedUpgrade** + +TestDistributedUpgrade used succeed for wrong reasons. + + +--- + +* [HADOOP-4150](https://issues.apache.org/jira/browse/HADOOP-4150) | *Blocker* | **Include librecordio as part of the release** + +Included librecordio in release for use by xerces-c (ant -Dlibrecordio=true -Dxercescroot=\) + + +--- + +* [HADOOP-4061](https://issues.apache.org/jira/browse/HADOOP-4061) | *Major* | **Large number of decommission freezes the Namenode** + +Added a new conf property dfs.namenode.decommission.nodes.per.interval so that NameNode checks decommission status of x nodes for every y seconds, where x is the value of dfs.namenode.decommission.nodes.per.interval and y is the value of dfs.namenode.decommission.interval. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/CHANGES.0.18.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/CHANGES.0.18.4.md new file mode 100644 index 00000000000..b5cb699d0c1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/CHANGES.0.18.4.md @@ -0,0 +1,73 @@ + + +# Apache Hadoop Changelog + +## Release 0.18.4 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6017](https://issues.apache.org/jira/browse/HADOOP-6017) | NameNode and SecondaryNameNode fail to restart because of abnormal filenames. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-5644](https://issues.apache.org/jira/browse/HADOOP-5644) | Namnode is stuck in safe mode | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5557](https://issues.apache.org/jira/browse/HADOOP-5557) | Two minor problems in TestOverReplicatedBlocks | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5465](https://issues.apache.org/jira/browse/HADOOP-5465) | Blocks remain under-replicated | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5412](https://issues.apache.org/jira/browse/HADOOP-5412) | TestInjectionForSimulatedStorage occasionally fails on timeout | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5311](https://issues.apache.org/jira/browse/HADOOP-5311) | Write pipeline recovery fails | Blocker | . | Hairong Kuang | dhruba borthakur | +| [HADOOP-5192](https://issues.apache.org/jira/browse/HADOOP-5192) | Block reciever should not remove a finalized block when block replication fails | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5134](https://issues.apache.org/jira/browse/HADOOP-5134) | FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap | Blocker | . | Hairong Kuang | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5114](https://issues.apache.org/jira/browse/HADOOP-5114) | A bunch of mapred unit tests are failing on Windows | Minor | test | Ramya Sunil | Raghu Angadi | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/RELEASENOTES.0.18.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/RELEASENOTES.0.18.4.md new file mode 100644 index 00000000000..725c5c45ca2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.18.4/RELEASENOTES.0.18.4.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.18.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5114](https://issues.apache.org/jira/browse/HADOOP-5114) | *Minor* | **A bunch of mapred unit tests are failing on Windows** + +Remove timeout for accept() in DataNode. This made accept() fail in JDK on Windows and caused many tests to fail. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/CHANGES.0.19.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/CHANGES.0.19.0.md new file mode 100644 index 00000000000..7b62b094d36 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/CHANGES.0.19.0.md @@ -0,0 +1,422 @@ + + +# Apache Hadoop Changelog + +## Release 0.19.0 - 2008-11-20 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4430](https://issues.apache.org/jira/browse/HADOOP-4430) | Namenode Web UI capacity report is inconsistent with Balancer | Blocker | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4293](https://issues.apache.org/jira/browse/HADOOP-4293) | Remove WritableJobConf | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4281](https://issues.apache.org/jira/browse/HADOOP-4281) | Capacity reported in some of the commands is not consistent with the Web UI reported data | Blocker | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4227](https://issues.apache.org/jira/browse/HADOOP-4227) | Remove the deprecated, unused class ShellCommand. | Minor | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4190](https://issues.apache.org/jira/browse/HADOOP-4190) | Changes to JobHistory makes it backward incompatible | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-4116](https://issues.apache.org/jira/browse/HADOOP-4116) | Balancer should provide better resource management | Blocker | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-3981](https://issues.apache.org/jira/browse/HADOOP-3981) | Need a distributed file checksum algorithm for HDFS | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3963](https://issues.apache.org/jira/browse/HADOOP-3963) | libhdfs should never exit on its own but rather return errors to the calling application | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3938](https://issues.apache.org/jira/browse/HADOOP-3938) | Quotas for disk space management | Major | . | Robert Chansler | Raghu Angadi | +| [HADOOP-3911](https://issues.apache.org/jira/browse/HADOOP-3911) | ' -blocks ' option not being recognized | Minor | fs, util | Koji Noguchi | Lohit Vijayarenu | +| [HADOOP-3889](https://issues.apache.org/jira/browse/HADOOP-3889) | distcp: Better Error Message should be thrown when accessing source files/directory with no read permission | Minor | . | Peeyush Bishnoi | Tsz Wo Nicholas Sze | +| [HADOOP-3837](https://issues.apache.org/jira/browse/HADOOP-3837) | hadop streaming does not use progress reporting to detect hung tasks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3796](https://issues.apache.org/jira/browse/HADOOP-3796) | fuse-dfs should take rw,ro,trashon,trashoff,protected=blah mount arguments rather than them being compiled in | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3792](https://issues.apache.org/jira/browse/HADOOP-3792) | exit code from "hadoop dfs -test ..." is wrong for Unix shell | Minor | fs | Ben Slusky | Ben Slusky | +| [HADOOP-3722](https://issues.apache.org/jira/browse/HADOOP-3722) | Provide a unified way to pass jobconf options from bin/hadoop | Minor | conf | Matei Zaharia | Enis Soztutar | +| [HADOOP-3667](https://issues.apache.org/jira/browse/HADOOP-3667) | Remove deprecated methods in JobConf | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3652](https://issues.apache.org/jira/browse/HADOOP-3652) | Remove deprecated class OutputFormatBase | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3595](https://issues.apache.org/jira/browse/HADOOP-3595) | Remove deprecated mapred.combine.once functionality | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3245](https://issues.apache.org/jira/browse/HADOOP-3245) | Provide ability to persist running jobs (extend HADOOP-1876) | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-3150](https://issues.apache.org/jira/browse/HADOOP-3150) | Move task file promotion into the task | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-3062](https://issues.apache.org/jira/browse/HADOOP-3062) | Need to capture the metrics for the network ios generate by dfs reads/writes and map/reduce shuffling and break them down by racks | Major | metrics | Runping Qi | Chris Douglas | +| [HADOOP-2816](https://issues.apache.org/jira/browse/HADOOP-2816) | Cluster summary at name node web has confusing report for space utilization | Major | . | Robert Chansler | Suresh Srinivas | +| [HADOOP-2325](https://issues.apache.org/jira/browse/HADOOP-2325) | Require Java 6 | Major | build | Doug Cutting | Doug Cutting | +| [HADOOP-1869](https://issues.apache.org/jira/browse/HADOOP-1869) | access times of HDFS files | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-1700](https://issues.apache.org/jira/browse/HADOOP-1700) | Append to files in HDFS | Major | . | stack | dhruba borthakur | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4406](https://issues.apache.org/jira/browse/HADOOP-4406) | Make TCTLSeparatedProtocol configurable and have DynamicSerDe initialize, initialize the SerDe | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4301](https://issues.apache.org/jira/browse/HADOOP-4301) | Forrest doc for skip bad records feature | Blocker | documentation | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4260](https://issues.apache.org/jira/browse/HADOOP-4260) | support show partitions in hive | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4176](https://issues.apache.org/jira/browse/HADOOP-4176) | Implement getFileChecksum(Path) in HftpFileSystem | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4120](https://issues.apache.org/jira/browse/HADOOP-4120) | [Hive] print time taken by query in interactive shell | Minor | . | Raghotham Murthy | Raghotham Murthy | +| [HADOOP-4106](https://issues.apache.org/jira/browse/HADOOP-4106) | add time, permission and user attribute support to fuse-dfs | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4104](https://issues.apache.org/jira/browse/HADOOP-4104) | add time, permission and user attribute support to libhdfs | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4095](https://issues.apache.org/jira/browse/HADOOP-4095) | [Hive] enhance describe table & partition | Major | . | Prasad Chakka | Namit Jain | +| [HADOOP-4086](https://issues.apache.org/jira/browse/HADOOP-4086) | Add limit to Hive QL | Major | . | Ashish Thusoo | Namit Jain | +| [HADOOP-4084](https://issues.apache.org/jira/browse/HADOOP-4084) | Add explain plan capabilities to Hive QL | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4070](https://issues.apache.org/jira/browse/HADOOP-4070) | [Hive] Provide a mechanism for registering UDFs from the query language | Major | . | Tom White | Tom White | +| [HADOOP-3992](https://issues.apache.org/jira/browse/HADOOP-3992) | Synthetic Load Generator for NameNode testing | Major | . | Robert Chansler | Hairong Kuang | +| [HADOOP-3941](https://issues.apache.org/jira/browse/HADOOP-3941) | Extend FileSystem API to return file-checksums/file-digests | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3939](https://issues.apache.org/jira/browse/HADOOP-3939) | DistCp should support an option for deleting non-existing files. | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3924](https://issues.apache.org/jira/browse/HADOOP-3924) | Add a 'Killed' job status | Critical | . | Alejandro Abdelnur | Subru Krishnan | +| [HADOOP-3873](https://issues.apache.org/jira/browse/HADOOP-3873) | DistCp should have an option for limiting the number of files/bytes being copied | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3854](https://issues.apache.org/jira/browse/HADOOP-3854) | org.apache.hadoop.http.HttpServer should support user configurable filter | Major | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3829](https://issues.apache.org/jira/browse/HADOOP-3829) | Narrown down skipped records based on user acceptable value | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-3828](https://issues.apache.org/jira/browse/HADOOP-3828) | Write skipped records' bytes to DFS | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-3754](https://issues.apache.org/jira/browse/HADOOP-3754) | Support a Thrift Interface to access files/directories in HDFS | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3746](https://issues.apache.org/jira/browse/HADOOP-3746) | A fair sharing job scheduler | Minor | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-3730](https://issues.apache.org/jira/browse/HADOOP-3730) | add new JobConf constructor that disables loading default configurations | Major | conf | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-3714](https://issues.apache.org/jira/browse/HADOOP-3714) | Bash tab completion support | Trivial | scripts | Chris Smith | Chris Smith | +| [HADOOP-3702](https://issues.apache.org/jira/browse/HADOOP-3702) | add support for chaining Maps in a single Map and after a Reduce [M\*/RM\*] | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-3698](https://issues.apache.org/jira/browse/HADOOP-3698) | Implement access control for submitting jobs to queues in the JobTracker | Major | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3695](https://issues.apache.org/jira/browse/HADOOP-3695) | [HOD] Have an ability to run multiple slaves per node | Major | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3585](https://issues.apache.org/jira/browse/HADOOP-3585) | Hardware Failure Monitoring in large clusters running Hadoop/HDFS | Minor | metrics | Ioannis Koltsidas | Ioannis Koltsidas | +| [HADOOP-3485](https://issues.apache.org/jira/browse/HADOOP-3485) | fix writes | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3479](https://issues.apache.org/jira/browse/HADOOP-3479) | Implement configuration items useful for Hadoop resource manager (v1) | Major | conf | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3445](https://issues.apache.org/jira/browse/HADOOP-3445) | Implementing core scheduler functionality in Resource Manager (V1) for Hadoop | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-3402](https://issues.apache.org/jira/browse/HADOOP-3402) | Add example code to support run terasort on hadoop | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-3361](https://issues.apache.org/jira/browse/HADOOP-3361) | Implement renames for NativeS3FileSystem | Major | fs/s3 | Tom White | Tom White | +| [HADOOP-3149](https://issues.apache.org/jira/browse/HADOOP-3149) | supporting multiple outputs for M/R jobs | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-3019](https://issues.apache.org/jira/browse/HADOOP-3019) | want input sampler & sorted partitioner | Major | . | Doug Cutting | Chris Douglas | +| [HADOOP-2664](https://issues.apache.org/jira/browse/HADOOP-2664) | lzop-compatible CompresionCodec | Major | io | Chris Douglas | Chris Douglas | +| [HADOOP-2658](https://issues.apache.org/jira/browse/HADOOP-2658) | Design and Implement a Test Plan to support appends to HDFS files | Blocker | test | dhruba borthakur | dhruba borthakur | +| [HADOOP-2536](https://issues.apache.org/jira/browse/HADOOP-2536) | MapReduce for MySQL | Minor | . | Fredrik Hedberg | Fredrik Hedberg | +| [HADOOP-1823](https://issues.apache.org/jira/browse/HADOOP-1823) | want InputFormat for bzip2 files | Major | . | Doug Cutting | | +| [HADOOP-1480](https://issues.apache.org/jira/browse/HADOOP-1480) | pipes should be able to set user counters | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-372](https://issues.apache.org/jira/browse/HADOOP-372) | should allow to specify different inputformat classes for different input dirs for Map/Reduce jobs | Major | . | Runping Qi | Chris Smith | +| [HADOOP-153](https://issues.apache.org/jira/browse/HADOOP-153) | skip records that fail Task | Major | . | Doug Cutting | Sharad Agarwal | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4433](https://issues.apache.org/jira/browse/HADOOP-4433) | Improve data loader for collecting metrics and log files from hadoop and system | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4431](https://issues.apache.org/jira/browse/HADOOP-4431) | Add versionning/tags to Chukwa Chunk | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-4424](https://issues.apache.org/jira/browse/HADOOP-4424) | menu layout change for Hadoop documentation | Blocker | documentation | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4353](https://issues.apache.org/jira/browse/HADOOP-4353) | enable multi-line query from Hive CLI | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4345](https://issues.apache.org/jira/browse/HADOOP-4345) | Hive: Check that partitioning predicate is present when hive.partition.pruning = strict | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4307](https://issues.apache.org/jira/browse/HADOOP-4307) | add an option to describe table to show extended properties of the table such as serialization/deserialization properties | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4279](https://issues.apache.org/jira/browse/HADOOP-4279) | write the random number generator seed to log in the append-related tests | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4252](https://issues.apache.org/jira/browse/HADOOP-4252) | Catch Ctrl-C in Hive CLI so that corresponding hadoop jobs can be killed | Minor | . | Prasad Chakka | Pete Wyckoff | +| [HADOOP-4231](https://issues.apache.org/jira/browse/HADOOP-4231) | Hive: converting complex objects to JSON failed. | Minor | . | Zheng Shao | Zheng Shao | +| [HADOOP-4230](https://issues.apache.org/jira/browse/HADOOP-4230) | Hive: GroupBy should not pass the whole row from mapper to reducer | Blocker | . | Zheng Shao | Ashish Thusoo | +| [HADOOP-4205](https://issues.apache.org/jira/browse/HADOOP-4205) | [Hive] metastore and ql to use the refactored SerDe library | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-4194](https://issues.apache.org/jira/browse/HADOOP-4194) | Add JobConf and JobID to job related methods in JobTrackerInstrumentation | Major | . | Mac Yang | Mac Yang | +| [HADOOP-4181](https://issues.apache.org/jira/browse/HADOOP-4181) | some minor things to make Hadoop friendlier to git | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-4174](https://issues.apache.org/jira/browse/HADOOP-4174) | Move non-client methods ou of ClientProtocol | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4151](https://issues.apache.org/jira/browse/HADOOP-4151) | Add a memcmp-compatible interface for key types | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-4138](https://issues.apache.org/jira/browse/HADOOP-4138) | [Hive] refactor the SerDe library | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-4124](https://issues.apache.org/jira/browse/HADOOP-4124) | Changing priority of a job should be available in CLI and available on the web UI only along with the Kill Job actions | Major | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4117](https://issues.apache.org/jira/browse/HADOOP-4117) | Improve configurability of Hadoop EC2 instances | Major | contrib/cloud | Tom White | Tom White | +| [HADOOP-4113](https://issues.apache.org/jira/browse/HADOOP-4113) | libhdfs should never exit on its own but rather return errors to the calling application - missing diff files | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4094](https://issues.apache.org/jira/browse/HADOOP-4094) | [Hive]implement hive-site.xml similar to hadoop-site.xml | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4090](https://issues.apache.org/jira/browse/HADOOP-4090) | The configuration file lists two paths to hadoop directories (bin and conf). Startup should check that these are valid directories and give appropriate messages. | Minor | . | Ashish Thusoo | Raghotham Murthy | +| [HADOOP-4083](https://issues.apache.org/jira/browse/HADOOP-4083) | change new config attribute queue.name to mapred.job.queue.name | Major | . | Owen O'Malley | Hemanth Yamijala | +| [HADOOP-4075](https://issues.apache.org/jira/browse/HADOOP-4075) | test-patch.sh should output the ant commands that it runs | Major | build | Nigel Daley | Ramya Sunil | +| [HADOOP-4062](https://issues.apache.org/jira/browse/HADOOP-4062) | IPC client does not need to be synchronized on the output stream when a connection is closed | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-4053](https://issues.apache.org/jira/browse/HADOOP-4053) | Schedulers need to know when a job has completed | Blocker | . | Vivek Ratan | Amar Kamat | +| [HADOOP-3975](https://issues.apache.org/jira/browse/HADOOP-3975) | test-patch can report the modifications found in the workspace along with the error message | Minor | test | Hemanth Yamijala | Ramya Sunil | +| [HADOOP-3965](https://issues.apache.org/jira/browse/HADOOP-3965) | Make DataBlockScanner package private | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3948](https://issues.apache.org/jira/browse/HADOOP-3948) | Separate Namenodes edits and fsimage | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3944](https://issues.apache.org/jira/browse/HADOOP-3944) | TupleWritable listed as public class but cannot be used without methods private to the package | Trivial | documentation | Michael Andrews | Chris Douglas | +| [HADOOP-3943](https://issues.apache.org/jira/browse/HADOOP-3943) | NetworkTopology.pseudoSortByDistance does not need to be a synchronized method | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3935](https://issues.apache.org/jira/browse/HADOOP-3935) | Extract classes from DataNode.java | Trivial | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-3930](https://issues.apache.org/jira/browse/HADOOP-3930) | Decide how to integrate scheduler info into CLI and job tracker web page | Major | . | Matei Zaharia | Sreekanth Ramakrishnan | +| [HADOOP-3908](https://issues.apache.org/jira/browse/HADOOP-3908) | Better error message if llibhdfs.so doesn't exist | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3905](https://issues.apache.org/jira/browse/HADOOP-3905) | Create a generic interface for edits log. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3892](https://issues.apache.org/jira/browse/HADOOP-3892) | Include Unix group name in JobConf | Trivial | conf | Matei Zaharia | Matei Zaharia | +| [HADOOP-3866](https://issues.apache.org/jira/browse/HADOOP-3866) | Improve Hadoop Jobtracker Admin | Major | scripts | craig weisenfluh | craig weisenfluh | +| [HADOOP-3861](https://issues.apache.org/jira/browse/HADOOP-3861) | Make MapFile.Reader and Writer implement java.io.Closeable | Major | io | Tom White | Tom White | +| [HADOOP-3860](https://issues.apache.org/jira/browse/HADOOP-3860) | Compare name-node performance when journaling is performed into local hard-drives or nfs. | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-3853](https://issues.apache.org/jira/browse/HADOOP-3853) | Move multiple input format extension to library package | Major | . | Tom White | Tom White | +| [HADOOP-3852](https://issues.apache.org/jira/browse/HADOOP-3852) | If ShellCommandExecutor had a toString() operator that listed the command run, its error messages may be more meaningful | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-3844](https://issues.apache.org/jira/browse/HADOOP-3844) | include message of local exception in Client call failures | Minor | ipc | Steve Loughran | Steve Loughran | +| [HADOOP-3805](https://issues.apache.org/jira/browse/HADOOP-3805) | improve fuse-dfs write performance which is 33% slower than hadoop dfs -copyFromLocal | Minor | . | Pete Wyckoff | | +| [HADOOP-3780](https://issues.apache.org/jira/browse/HADOOP-3780) | JobTracker should synchronously resolve the tasktracker's network location when the tracker registers | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-3769](https://issues.apache.org/jira/browse/HADOOP-3769) | expose static SampleMapper and SampleReducer classes of GenericMRLoadGenerator class for gridmix reuse | Major | test | Lingyun Yang | Lingyun Yang | +| [HADOOP-3759](https://issues.apache.org/jira/browse/HADOOP-3759) | Provide ability to run memory intensive jobs without affecting other running tasks on the nodes | Major | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-3747](https://issues.apache.org/jira/browse/HADOOP-3747) | Add counter support to MultipleOutputs | Minor | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-3721](https://issues.apache.org/jira/browse/HADOOP-3721) | CompositeRecordReader::next is unnecessarily complex | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3719](https://issues.apache.org/jira/browse/HADOOP-3719) | Chukwa | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-3694](https://issues.apache.org/jira/browse/HADOOP-3694) | if MiniDFS startup time could be improved, testing time would be reduced | Major | test | Steve Loughran | Doug Cutting | +| [HADOOP-3684](https://issues.apache.org/jira/browse/HADOOP-3684) | The data\_join should allow the user to implement a customer cloning function | Major | . | Runping Qi | Runping Qi | +| [HADOOP-3664](https://issues.apache.org/jira/browse/HADOOP-3664) | Remove deprecated methods introduced in changes to validating input paths (HADOOP-3095) | Major | . | Tom White | Tom White | +| [HADOOP-3661](https://issues.apache.org/jira/browse/HADOOP-3661) | Normalize fuse-dfs handling of moving things to trash wrt the way hadoop dfs does it (only when non posix trash flag is enabled in compile) | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3660](https://issues.apache.org/jira/browse/HADOOP-3660) | Add replication factor for injecting blocks in the data node cluster | Major | benchmarks | Sanjay Radia | Sanjay Radia | +| [HADOOP-3655](https://issues.apache.org/jira/browse/HADOOP-3655) | provide more control options for the junit run | Minor | build | Steve Loughran | Steve Loughran | +| [HADOOP-3646](https://issues.apache.org/jira/browse/HADOOP-3646) | Providing bzip2 as codec | Major | conf, io | Abdul Qadeer | Abdul Qadeer | +| [HADOOP-3638](https://issues.apache.org/jira/browse/HADOOP-3638) | Cache the iFile index files in memory to reduce seeks during map output serving | Major | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-3624](https://issues.apache.org/jira/browse/HADOOP-3624) | CreateEditsLog could be improved to create tree directory structure | Minor | test | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3620](https://issues.apache.org/jira/browse/HADOOP-3620) | Namenode should synchronously resolve a datanode's network location when the datanode registers | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-3617](https://issues.apache.org/jira/browse/HADOOP-3617) | Writes from map serialization include redundant checks for accounting space | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3605](https://issues.apache.org/jira/browse/HADOOP-3605) | Added an abort on unset AWS\_ACCOUNT\_ID to luanch-hadoop-master | Minor | contrib/cloud | Al Hoang | Al Hoang | +| [HADOOP-3581](https://issues.apache.org/jira/browse/HADOOP-3581) | Prevent memory intensive user tasks from taking down nodes | Major | . | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3577](https://issues.apache.org/jira/browse/HADOOP-3577) | Tools to inject blocks into name node and simulated data nodes for testing | Minor | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-3563](https://issues.apache.org/jira/browse/HADOOP-3563) | Seperate out datanode and namenode functionality of generation stamp upgrade process | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-3556](https://issues.apache.org/jira/browse/HADOOP-3556) | Substitute the synchronized code in MD5Hash to avoid lock contention. Use ThreadLocal instead. | Major | io | IvĂĄn de Prado | IvĂĄn de Prado | +| [HADOOP-3549](https://issues.apache.org/jira/browse/HADOOP-3549) | meaningful errno values in libhdfs | Major | . | Ben Slusky | Ben Slusky | +| [HADOOP-3514](https://issues.apache.org/jira/browse/HADOOP-3514) | Reduce seeks during shuffle, by inline crcs | Major | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-3498](https://issues.apache.org/jira/browse/HADOOP-3498) | File globbing alternation should be able to span path components | Major | fs | Tom White | Tom White | +| [HADOOP-3478](https://issues.apache.org/jira/browse/HADOOP-3478) | The algorithm to decide map re-execution on fetch failures can be improved | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-3446](https://issues.apache.org/jira/browse/HADOOP-3446) | The reduce task should not flush the in memory file system before starting the reducer | Critical | . | Owen O'Malley | Chris Douglas | +| [HADOOP-3412](https://issues.apache.org/jira/browse/HADOOP-3412) | Refactor the scheduler out of the JobTracker | Minor | . | Brice Arnould | Brice Arnould | +| [HADOOP-3368](https://issues.apache.org/jira/browse/HADOOP-3368) | Can commons-logging.properties be pulled from hadoop-core? | Major | build | Steve Loughran | Steve Loughran | +| [HADOOP-3342](https://issues.apache.org/jira/browse/HADOOP-3342) | Better safety of killing jobs via web interface | Minor | . | Daniel Naber | Enis Soztutar | +| [HADOOP-3341](https://issues.apache.org/jira/browse/HADOOP-3341) | make key-value separators in hadoop streaming fully configurable | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-3328](https://issues.apache.org/jira/browse/HADOOP-3328) | DFS write pipeline : only the last datanode needs to verify checksum | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3202](https://issues.apache.org/jira/browse/HADOOP-3202) | Deprecate org.apache.hadoop.fs.FileUtil.fullyDelete(FileSystem fs, Path dir) | Major | fs | Tsz Wo Nicholas Sze | Amareshwari Sriramadasu | +| [HADOOP-3169](https://issues.apache.org/jira/browse/HADOOP-3169) | LeaseChecker daemon should not be started in DFSClient constructor | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-2411](https://issues.apache.org/jira/browse/HADOOP-2411) | Add support for larger EC2 instance types | Major | contrib/cloud | Tom White | Chris K Wensel | +| [HADOOP-2330](https://issues.apache.org/jira/browse/HADOOP-2330) | Preallocate transaction log to improve namenode transaction logging performance | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-2302](https://issues.apache.org/jira/browse/HADOOP-2302) | Streaming should provide an option for numerical sort of keys | Major | . | Lohit Vijayarenu | Devaraj Das | +| [HADOOP-2165](https://issues.apache.org/jira/browse/HADOOP-2165) | Augment JobHistory to store tasks' userlogs | Major | . | Arun C Murthy | Vinod Kumar Vavilapalli | +| [HADOOP-2130](https://issues.apache.org/jira/browse/HADOOP-2130) | Pipes submit job should be Non-blocking | Critical | . | Srikanth Kakani | Arun C Murthy | +| [HADOOP-1627](https://issues.apache.org/jira/browse/HADOOP-1627) | DFSAdmin incorrectly reports cluster data. | Minor | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-657](https://issues.apache.org/jira/browse/HADOOP-657) | Free temporary space should be modelled better | Major | . | Owen O'Malley | Ari Rabkin | +| [HADOOP-249](https://issues.apache.org/jira/browse/HADOOP-249) | Improving Map -\> Reduce performance and Task JVM reuse | Major | . | Benjamin Reed | Devaraj Das | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4595](https://issues.apache.org/jira/browse/HADOOP-4595) | JVM Reuse triggers RuntimeException("Invalid state") | Major | . | Aaron Kimball | Devaraj Das | +| [HADOOP-4552](https://issues.apache.org/jira/browse/HADOOP-4552) | Deadlock in RPC Server | Major | ipc | Raghu Angadi | Raghu Angadi | +| [HADOOP-4525](https://issues.apache.org/jira/browse/HADOOP-4525) | config ipc.server.tcpnodelay is no loger being respected | Major | ipc | Clint Morgan | Clint Morgan | +| [HADOOP-4510](https://issues.apache.org/jira/browse/HADOOP-4510) | FileOutputFormat protects getTaskOutputPath | Blocker | . | Chris K Wensel | Chris K Wensel | +| [HADOOP-4500](https://issues.apache.org/jira/browse/HADOOP-4500) | multifilesplit is using job default filesystem incorrectly | Major | . | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-4498](https://issues.apache.org/jira/browse/HADOOP-4498) | JobHistory does not escape literal jobName when used in a regex pattern | Blocker | . | Chris K Wensel | Chris K Wensel | +| [HADOOP-4471](https://issues.apache.org/jira/browse/HADOOP-4471) | Capacity Scheduler should maintain the right ordering of jobs in its running queue | Blocker | . | Vivek Ratan | Amar Kamat | +| [HADOOP-4466](https://issues.apache.org/jira/browse/HADOOP-4466) | SequenceFileOutputFormat is coupled to WritableComparable and Writable | Blocker | io | Chris K Wensel | Chris K Wensel | +| [HADOOP-4457](https://issues.apache.org/jira/browse/HADOOP-4457) | Input split logging in history is broken in 0.19 | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4455](https://issues.apache.org/jira/browse/HADOOP-4455) | Upload the derby.jar and TestSeDe.jar needed for fixes to 0.19 bugs | Blocker | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4449](https://issues.apache.org/jira/browse/HADOOP-4449) | Minor formatting changes to quota related commands | Trivial | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4446](https://issues.apache.org/jira/browse/HADOOP-4446) | Update Scheduling Information display in Web UI | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [HADOOP-4439](https://issues.apache.org/jira/browse/HADOOP-4439) | Cleanup memory related resource management | Blocker | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4438](https://issues.apache.org/jira/browse/HADOOP-4438) | Add new/missing dfs commands in forrest | Blocker | documentation | Hemanth Yamijala | Suresh Srinivas | +| [HADOOP-4427](https://issues.apache.org/jira/browse/HADOOP-4427) | Add new/missing commands in forrest | Blocker | documentation | Sharad Agarwal | Sreekanth Ramakrishnan | +| [HADOOP-4425](https://issues.apache.org/jira/browse/HADOOP-4425) | Edits log takes much longer to load | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-4423](https://issues.apache.org/jira/browse/HADOOP-4423) | FSDataset.getStoredBlock(id) should not return corrupted information | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4418](https://issues.apache.org/jira/browse/HADOOP-4418) | Update documentation in forrest for Mapred, streaming and pipes | Blocker | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4410](https://issues.apache.org/jira/browse/HADOOP-4410) | TestMiniMRDebugScript fails on trunk | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4405](https://issues.apache.org/jira/browse/HADOOP-4405) | all creation of hadoop dfs queries from with in hive shell | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4404](https://issues.apache.org/jira/browse/HADOOP-4404) | saveFSImage() should remove files from a storage directory that do not correspond to its type. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4400](https://issues.apache.org/jira/browse/HADOOP-4400) | Add "hdfs://" to fs.default.name on quickstart.html | Trivial | documentation | Jeff Hammerbacher | Jeff Hammerbacher | +| [HADOOP-4393](https://issues.apache.org/jira/browse/HADOOP-4393) | Merge AccessControlException and AccessControlIOException into one exception class | Blocker | fs | Owen O'Malley | Owen O'Malley | +| [HADOOP-4387](https://issues.apache.org/jira/browse/HADOOP-4387) | TestHDFSFileSystemContract fails on windows | Blocker | test | Raghu Angadi | Raghu Angadi | +| [HADOOP-4380](https://issues.apache.org/jira/browse/HADOOP-4380) | Make new classes in mapred package private instead of public | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4378](https://issues.apache.org/jira/browse/HADOOP-4378) | TestJobQueueInformation fails regularly | Blocker | test | Tsz Wo Nicholas Sze | Sreekanth Ramakrishnan | +| [HADOOP-4376](https://issues.apache.org/jira/browse/HADOOP-4376) | Fix line formatting in hadoop-default.xml for hadoop.http.filter.initializers | Blocker | conf | Enis Soztutar | Enis Soztutar | +| [HADOOP-4373](https://issues.apache.org/jira/browse/HADOOP-4373) | Guaranteed Capacity calculation is not calculated correctly | Blocker | . | Karam Singh | Hemanth Yamijala | +| [HADOOP-4367](https://issues.apache.org/jira/browse/HADOOP-4367) | Hive: UDAF functions cannot handle NULL values | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-4366](https://issues.apache.org/jira/browse/HADOOP-4366) | Provide way to replace existing column names for columnSet tables | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4361](https://issues.apache.org/jira/browse/HADOOP-4361) | Corner cases in killJob from command line | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4358](https://issues.apache.org/jira/browse/HADOOP-4358) | NPE from CreateEditsLog | Blocker | test | Chris Douglas | Raghu Angadi | +| [HADOOP-4356](https://issues.apache.org/jira/browse/HADOOP-4356) | [Hive] for a 2-stage map-reduce job, number of reducers not set correctly | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4355](https://issues.apache.org/jira/browse/HADOOP-4355) | hive 2 case sensitivity issues | Major | . | Zheng Shao | | +| [HADOOP-4344](https://issues.apache.org/jira/browse/HADOOP-4344) | Hive: Partition pruning causes semantic exception with joins | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4342](https://issues.apache.org/jira/browse/HADOOP-4342) | [hive] bug in partition pruning | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4336](https://issues.apache.org/jira/browse/HADOOP-4336) | fix sampling bug in fractional bucket case | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4335](https://issues.apache.org/jira/browse/HADOOP-4335) | FsShell -ls fails for file systems without owners or groups | Major | scripts | David Phillips | David Phillips | +| [HADOOP-4333](https://issues.apache.org/jira/browse/HADOOP-4333) | add ability to drop partitions through DDL | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4330](https://issues.apache.org/jira/browse/HADOOP-4330) | Hive: AS clause with subqueries having group bys is not propogated to the outer query block | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4329](https://issues.apache.org/jira/browse/HADOOP-4329) | Hive: [] operator with maps does not work | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4327](https://issues.apache.org/jira/browse/HADOOP-4327) | Create table hive does not set delimeters | Major | . | Edward Capriolo | Namit Jain | +| [HADOOP-4321](https://issues.apache.org/jira/browse/HADOOP-4321) | Document the capacity scheduler in Forrest | Blocker | documentation | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4320](https://issues.apache.org/jira/browse/HADOOP-4320) | [Hive] TCTLSeparatedProtocol implement maps/lists/sets read/writes | Major | . | Pete Wyckoff | | +| [HADOOP-4319](https://issues.apache.org/jira/browse/HADOOP-4319) | fuse-dfs dfs\_read function may return less than the requested #of bytes even if EOF not reached | Blocker | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4316](https://issues.apache.org/jira/browse/HADOOP-4316) | [Hive] extra new lines at output | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4315](https://issues.apache.org/jira/browse/HADOOP-4315) | Hive: Cleanup temporary files once the job is done | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4309](https://issues.apache.org/jira/browse/HADOOP-4309) | eclipse-plugin no longer compiles on trunk | Blocker | contrib/eclipse-plugin | Chris Douglas | Chris Douglas | +| [HADOOP-4303](https://issues.apache.org/jira/browse/HADOOP-4303) | Hive: trim and rtrim UDFs behaviors are reversed | Major | . | Ashish Thusoo | Ashish Thusoo | +| [HADOOP-4302](https://issues.apache.org/jira/browse/HADOOP-4302) | TestReduceFetch fails intermittently | Blocker | . | Devaraj Das | Chris Douglas | +| [HADOOP-4299](https://issues.apache.org/jira/browse/HADOOP-4299) | Unable to access a file by a different user in the same group when permissions is set to 770 or when permissions is turned OFF | Blocker | . | Ramya Sunil | Hairong Kuang | +| [HADOOP-4296](https://issues.apache.org/jira/browse/HADOOP-4296) | Spasm of JobClient failures on successful jobs every once in a while | Blocker | . | Joydeep Sen Sarma | dhruba borthakur | +| [HADOOP-4294](https://issues.apache.org/jira/browse/HADOOP-4294) | Hive: Parser should pass field schema to SerDe | Major | . | Zheng Shao | | +| [HADOOP-4288](https://issues.apache.org/jira/browse/HADOOP-4288) | java.lang.NullPointerException is observed in Jobtracker log while call heartbeat | Blocker | . | Karam Singh | Amar Kamat | +| [HADOOP-4287](https://issues.apache.org/jira/browse/HADOOP-4287) | [mapred] jobqueue\_details.jsp shows negative count of running and waiting reduces with CapacityTaskScheduler. | Blocker | . | Vinod Kumar Vavilapalli | Sreekanth Ramakrishnan | +| [HADOOP-4282](https://issues.apache.org/jira/browse/HADOOP-4282) | User configurable filter fails to filter accesses to certain directories | Blocker | . | Kan Zhang | Tsz Wo Nicholas Sze | +| [HADOOP-4280](https://issues.apache.org/jira/browse/HADOOP-4280) | test-libhdfs consistently fails on trunk | Blocker | . | Raghu Angadi | Pete Wyckoff | +| [HADOOP-4278](https://issues.apache.org/jira/browse/HADOOP-4278) | TestDatanodeDeath failed occasionally | Blocker | . | Tsz Wo Nicholas Sze | dhruba borthakur | +| [HADOOP-4275](https://issues.apache.org/jira/browse/HADOOP-4275) | New public methods added to the \*ID classes | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4274](https://issues.apache.org/jira/browse/HADOOP-4274) | Capacity scheduler's implementation of getJobs modifies the list of running jobs inadvertently | Blocker | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4273](https://issues.apache.org/jira/browse/HADOOP-4273) | [Hive] job submission exception if input is null | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4272](https://issues.apache.org/jira/browse/HADOOP-4272) | Hive: metadataTypedColumnsetSerDe should check if SERIALIZATION.LIB is old columnsetSerDe | Major | . | Zheng Shao | Prasad Chakka | +| [HADOOP-4269](https://issues.apache.org/jira/browse/HADOOP-4269) | LineRecordReader.LineReader should use util.LineReader | Major | util | Chris Douglas | Chris Douglas | +| [HADOOP-4267](https://issues.apache.org/jira/browse/HADOOP-4267) | TestDBJob failed on Linux | Blocker | . | Raghu Angadi | Enis Soztutar | +| [HADOOP-4266](https://issues.apache.org/jira/browse/HADOOP-4266) | Hive: Support "IS NULL", "IS NOT NULL", and size(x) for map and list | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-4265](https://issues.apache.org/jira/browse/HADOOP-4265) | [Hive] error when user specifies the delimiter | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4261](https://issues.apache.org/jira/browse/HADOOP-4261) | Jobs failing in the init stage will never cleanup | Blocker | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-4256](https://issues.apache.org/jira/browse/HADOOP-4256) | Remove Completed and Failed Job tables from jobqueue\_details.jsp | Blocker | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-4254](https://issues.apache.org/jira/browse/HADOOP-4254) | Cannot setSpaceQuota to 1TB | Blocker | . | Tsz Wo Nicholas Sze | Raghu Angadi | +| [HADOOP-4250](https://issues.apache.org/jira/browse/HADOOP-4250) | Remove short names of serdes from Deserializer, Serializer & SerDe interface and relevant code. | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4249](https://issues.apache.org/jira/browse/HADOOP-4249) | Declare hsqldb.jar in eclipse plugin | Blocker | contrib/eclipse-plugin | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4248](https://issues.apache.org/jira/browse/HADOOP-4248) | Remove HADOOP-1230 API from 0.19 | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4247](https://issues.apache.org/jira/browse/HADOOP-4247) | hadoop jar throwing exception when running examples | Blocker | . | Hemanth Yamijala | Owen O'Malley | +| [HADOOP-4246](https://issues.apache.org/jira/browse/HADOOP-4246) | Reduce task copy errors may not kill it eventually | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4242](https://issues.apache.org/jira/browse/HADOOP-4242) | Remove an extra ";" in FSDirectory | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4241](https://issues.apache.org/jira/browse/HADOOP-4241) | -hiveconf config parameters in hive cli should override all config variables | Major | . | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-4236](https://issues.apache.org/jira/browse/HADOOP-4236) | JobTracker.killJob() fails to kill a job if the job is not yet initialized | Blocker | . | Amar Kamat | Sharad Agarwal | +| [HADOOP-4232](https://issues.apache.org/jira/browse/HADOOP-4232) | Race condition in JVM reuse when more than one slot becomes free | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4225](https://issues.apache.org/jira/browse/HADOOP-4225) | FSEditLog logs modification time instead of access time. | Blocker | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4213](https://issues.apache.org/jira/browse/HADOOP-4213) | NPE in TestLimitTasksPerJobTaskScheduler | Major | test | Tsz Wo Nicholas Sze | Sreekanth Ramakrishnan | +| [HADOOP-4209](https://issues.apache.org/jira/browse/HADOOP-4209) | The TaskAttemptID should not have the JobTracker start time | Blocker | . | Owen O'Malley | Amar Kamat | +| [HADOOP-4200](https://issues.apache.org/jira/browse/HADOOP-4200) | Hadoop-Patch build is failing | Major | build | Ramya Sunil | Ramya Sunil | +| [HADOOP-4197](https://issues.apache.org/jira/browse/HADOOP-4197) | Need to update DATA\_TRANSFER\_VERSION | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4195](https://issues.apache.org/jira/browse/HADOOP-4195) | SequenceFile.Writer close() uses compressor after returning it to CodecPool. | Major | io | Hong Tang | Arun C Murthy | +| [HADOOP-4189](https://issues.apache.org/jira/browse/HADOOP-4189) | HADOOP-3245 is incomplete | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-4183](https://issues.apache.org/jira/browse/HADOOP-4183) | select \* to console issues in Hive | Major | . | Joydeep Sen Sarma | | +| [HADOOP-4175](https://issues.apache.org/jira/browse/HADOOP-4175) | Incorporate metastore server review comments | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4173](https://issues.apache.org/jira/browse/HADOOP-4173) | TestProcfsBasedProcessTree failing on Windows machine | Major | test, util | Ramya Sunil | Vinod Kumar Vavilapalli | +| [HADOOP-4169](https://issues.apache.org/jira/browse/HADOOP-4169) | 'compressed' keyword in DDL syntax misleading and does not compress | Major | . | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-4163](https://issues.apache.org/jira/browse/HADOOP-4163) | If a reducer failed at shuffling stage, the task should fail, not just logging an exception | Blocker | . | Runping Qi | Sharad Agarwal | +| [HADOOP-4155](https://issues.apache.org/jira/browse/HADOOP-4155) | JobHisotry::JOBTRACKER\_START\_TIME is not initialized properly | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-4154](https://issues.apache.org/jira/browse/HADOOP-4154) | Fix javac warning in WritableUtils | Minor | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4149](https://issues.apache.org/jira/browse/HADOOP-4149) | JobQueueJobInProgressListener.jobUpdated() might not work as expected | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-4147](https://issues.apache.org/jira/browse/HADOOP-4147) | Remove JobWithTaskContext from JobInProgress | Trivial | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-4146](https://issues.apache.org/jira/browse/HADOOP-4146) | [Hive] null pointer exception on a join | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4139](https://issues.apache.org/jira/browse/HADOOP-4139) | [Hive] multi group by statement is not optimized | Major | . | Namit Jain | Namit Jain | +| [HADOOP-4135](https://issues.apache.org/jira/browse/HADOOP-4135) | change max length of database columns for metastore to 767 | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4133](https://issues.apache.org/jira/browse/HADOOP-4133) | remove derby.log files form repository and also change the location where these files get created | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4129](https://issues.apache.org/jira/browse/HADOOP-4129) | Memory limits of TaskTracker and Tasks should be in kiloBytes. | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-4125](https://issues.apache.org/jira/browse/HADOOP-4125) | Reduce cleanup tip web ui is does not show attempts | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4121](https://issues.apache.org/jira/browse/HADOOP-4121) | HistoryViewer initialization failure should log exception trace | Trivial | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4112](https://issues.apache.org/jira/browse/HADOOP-4112) | Got ArrayOutOfBound exception while analyzing the job history | Major | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-4100](https://issues.apache.org/jira/browse/HADOOP-4100) | Scheduler.assignTasks should not be dealing with cleanupTask | Major | . | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-4099](https://issues.apache.org/jira/browse/HADOOP-4099) | HFTP interface compatibility with older releases broken | Blocker | fs | Kan Zhang | dhruba borthakur | +| [HADOOP-4097](https://issues.apache.org/jira/browse/HADOOP-4097) | Hive interaction with speculative execution is broken | Critical | . | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-4093](https://issues.apache.org/jira/browse/HADOOP-4093) | [Hive]unify Table.getCols() & get\_fields() | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4089](https://issues.apache.org/jira/browse/HADOOP-4089) | Check if the tmp file used in the CLI exists before using it. | Major | . | Ashish Thusoo | | +| [HADOOP-4087](https://issues.apache.org/jira/browse/HADOOP-4087) | Make Hive metastore server to work for PHP & Python clients | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4078](https://issues.apache.org/jira/browse/HADOOP-4078) | TestKosmosFileSystem fails on trunk | Blocker | fs | Amareshwari Sriramadasu | Lohit Vijayarenu | +| [HADOOP-4077](https://issues.apache.org/jira/browse/HADOOP-4077) | Access permissions for setting access times and modification times for files | Blocker | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4071](https://issues.apache.org/jira/browse/HADOOP-4071) | FSNameSystem.isReplicationInProgress should add an underReplicated block to the neededReplication queue using method "add" not "update" | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4054](https://issues.apache.org/jira/browse/HADOOP-4054) | During edit log loading, an underconstruction file's lease gets removed twice | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4050](https://issues.apache.org/jira/browse/HADOOP-4050) | TestFairScheduler failed on Linux | Major | . | Tsz Wo Nicholas Sze | Matei Zaharia | +| [HADOOP-4036](https://issues.apache.org/jira/browse/HADOOP-4036) | Increment InterTrackerProtocol version number due to changes in HADOOP-3759 | Major | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4030](https://issues.apache.org/jira/browse/HADOOP-4030) | LzopCodec shouldn't be in the default list of codecs i.e. io.compression.codecs | Major | io | Arun C Murthy | Arun C Murthy | +| [HADOOP-4027](https://issues.apache.org/jira/browse/HADOOP-4027) | When streaming utility is run without specifying mapper/reducer/input/output options, it returns 0. | Major | . | Ramya Sunil | | +| [HADOOP-4023](https://issues.apache.org/jira/browse/HADOOP-4023) | javadoc warnings: incorrect references | Major | documentation | Tsz Wo Nicholas Sze | Owen O'Malley | +| [HADOOP-4018](https://issues.apache.org/jira/browse/HADOOP-4018) | limit memory usage in jobtracker | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4014](https://issues.apache.org/jira/browse/HADOOP-4014) | DFS upgrade fails on Windows | Blocker | fs | NOMURA Yoshihide | Konstantin Shvachko | +| [HADOOP-3991](https://issues.apache.org/jira/browse/HADOOP-3991) | updates to hadoop-ec2-env.sh for 0.18.0 | Minor | contrib/cloud | Karl Anderson | Tom White | +| [HADOOP-3985](https://issues.apache.org/jira/browse/HADOOP-3985) | TestHDFSServerPorts fails on trunk | Major | . | Amar Kamat | Hairong Kuang | +| [HADOOP-3970](https://issues.apache.org/jira/browse/HADOOP-3970) | Counters written to the job history cannot be recovered back | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-3968](https://issues.apache.org/jira/browse/HADOOP-3968) | test-libhdfs fails on trunk | Major | . | Lohit Vijayarenu | Pete Wyckoff | +| [HADOOP-3964](https://issues.apache.org/jira/browse/HADOOP-3964) | javadoc warnings by failmon | Major | build | Tsz Wo Nicholas Sze | dhruba borthakur | +| [HADOOP-3962](https://issues.apache.org/jira/browse/HADOOP-3962) | Shell command "fs -count" should support paths with different file systsms | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3961](https://issues.apache.org/jira/browse/HADOOP-3961) | resource estimation works badly in some cases | Blocker | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-3959](https://issues.apache.org/jira/browse/HADOOP-3959) | [HOD] --resource\_manager.options is not passed to qsub | Major | contrib/hod | Craig Macdonald | Vinod Kumar Vavilapalli | +| [HADOOP-3958](https://issues.apache.org/jira/browse/HADOOP-3958) | TestMapRed ignores failures of the test case | Major | test | Owen O'Malley | Owen O'Malley | +| [HADOOP-3957](https://issues.apache.org/jira/browse/HADOOP-3957) | Fix javac warnings in DistCp and the corresponding tests | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3954](https://issues.apache.org/jira/browse/HADOOP-3954) | Skip records enabled as default. | Critical | . | Koji Noguchi | Sharad Agarwal | +| [HADOOP-3952](https://issues.apache.org/jira/browse/HADOOP-3952) | TestDataJoin references dfs.MiniDFSCluster instead of hdfs.MiniDFSCluster | Major | test | Owen O'Malley | Owen O'Malley | +| [HADOOP-3951](https://issues.apache.org/jira/browse/HADOOP-3951) | The package name used in FSNamesystem is incorrect | Trivial | . | Tsz Wo Nicholas Sze | Chris Douglas | +| [HADOOP-3950](https://issues.apache.org/jira/browse/HADOOP-3950) | TestMapRed and TestMiniMRDFSSort failed on trunk | Major | test | Tsz Wo Nicholas Sze | Enis Soztutar | +| [HADOOP-3949](https://issues.apache.org/jira/browse/HADOOP-3949) | javadoc warnings: Multiple sources of package comments found for package | Major | build, documentation | Tsz Wo Nicholas Sze | Jerome Boulon | +| [HADOOP-3946](https://issues.apache.org/jira/browse/HADOOP-3946) | TestMapRed fails on trunk | Blocker | test | Amareshwari Sriramadasu | Tom White | +| [HADOOP-3937](https://issues.apache.org/jira/browse/HADOOP-3937) | Job history may get disabled due to overly long job names | Major | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-3933](https://issues.apache.org/jira/browse/HADOOP-3933) | DataNode's BlockSender sends more data than necessary | Minor | . | Ning Li | Ning Li | +| [HADOOP-3919](https://issues.apache.org/jira/browse/HADOOP-3919) | hadoop conf got slightly mangled by 3772 | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-3910](https://issues.apache.org/jira/browse/HADOOP-3910) | Are ClusterTestDFSNamespaceLogging and ClusterTestDFS still valid tests? | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3907](https://issues.apache.org/jira/browse/HADOOP-3907) | INodeDirectoryWithQuota should be in its own .java file | Minor | . | Steve Loughran | Tsz Wo Nicholas Sze | +| [HADOOP-3904](https://issues.apache.org/jira/browse/HADOOP-3904) | A few tests still using old hdfs package name | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3875](https://issues.apache.org/jira/browse/HADOOP-3875) | Fix TaskTracker's heartbeat timer to note the time the hearbeat RPC returned to decide next heartbeat time | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3864](https://issues.apache.org/jira/browse/HADOOP-3864) | JobTracker lockup due to JobInProgress.initTasks taking significant time for large jobs on large clusters | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3863](https://issues.apache.org/jira/browse/HADOOP-3863) | Use a thread-local rather than static ENCODER/DECODER variables in Text for synchronization | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3851](https://issues.apache.org/jira/browse/HADOOP-3851) | spelling error in FSNamesystemMetrics log message | Trivial | . | Steve Loughran | Steve Loughran | +| [HADOOP-3848](https://issues.apache.org/jira/browse/HADOOP-3848) | TaskTracker.localizeJob calls getSystemDir for each task rather than caching it | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-3846](https://issues.apache.org/jira/browse/HADOOP-3846) | CreateEditsLog used for benchmark misses creating parent directories | Minor | benchmarks | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3836](https://issues.apache.org/jira/browse/HADOOP-3836) | TestMultipleOutputs will fail if it is ran more than one times | Major | test | Tsz Wo Nicholas Sze | Alejandro Abdelnur | +| [HADOOP-3831](https://issues.apache.org/jira/browse/HADOOP-3831) | slow-reading dfs clients do not recover from datanode-write-timeouts | Major | . | Christian Kunz | Raghu Angadi | +| [HADOOP-3820](https://issues.apache.org/jira/browse/HADOOP-3820) | gridmix-env has a syntax error, and wrongly defines USE\_REAL\_DATASET by default | Major | benchmarks | Arun C Murthy | Arun C Murthy | +| [HADOOP-3819](https://issues.apache.org/jira/browse/HADOOP-3819) | can not get svn revision # at build time if locale is not english | Minor | build | Rong-En Fan | Rong-En Fan | +| [HADOOP-3816](https://issues.apache.org/jira/browse/HADOOP-3816) | KFS changes for faster directory listing | Minor | fs | Sriram Rao | Sriram Rao | +| [HADOOP-3814](https://issues.apache.org/jira/browse/HADOOP-3814) | [HOD] Remove dfs.client.buffer.dir generation, as this is removed in Hadoop 0.19. | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-3795](https://issues.apache.org/jira/browse/HADOOP-3795) | NameNode does not save image if different dfs.name.dir have different checkpoint stamps | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3785](https://issues.apache.org/jira/browse/HADOOP-3785) | FileSystem cache should be case-insensitive | Major | fs | Doug Cutting | Bill de hOra | +| [HADOOP-3783](https://issues.apache.org/jira/browse/HADOOP-3783) | "deprecated filesystem name" warning on EC2 | Minor | contrib/cloud | Stuart Sierra | Tom White | +| [HADOOP-3778](https://issues.apache.org/jira/browse/HADOOP-3778) | seek(long) in DFSInputStream should catch socket exception for retry later | Minor | . | Luo Ning | Luo Ning | +| [HADOOP-3777](https://issues.apache.org/jira/browse/HADOOP-3777) | Failure to load native lzo libraries causes job failure | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-3776](https://issues.apache.org/jira/browse/HADOOP-3776) | NPE in NameNode with unknown blocks | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-3773](https://issues.apache.org/jira/browse/HADOOP-3773) | Setting the conf twice in Pipes Submitter | Trivial | . | Koji Noguchi | Koji Noguchi | +| [HADOOP-3756](https://issues.apache.org/jira/browse/HADOOP-3756) | dfs.client.buffer.dir isn't used in hdfs, but it's still in conf/hadoop-default.xml | Trivial | . | Michael Bieniosek | Raghu Angadi | +| [HADOOP-3732](https://issues.apache.org/jira/browse/HADOOP-3732) | Block scanner should read block information during initialization. | Blocker | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-3728](https://issues.apache.org/jira/browse/HADOOP-3728) | Cannot run more than one instance of examples.SleepJob at the same time. | Minor | . | Brice Arnould | Brice Arnould | +| [HADOOP-3726](https://issues.apache.org/jira/browse/HADOOP-3726) | TestCLI loses exception details on setup/teardown | Minor | test | Steve Loughran | Steve Loughran | +| [HADOOP-3725](https://issues.apache.org/jira/browse/HADOOP-3725) | TestMiniMRMapRedDebugScript loses exception details | Minor | test | Steve Loughran | Steve Loughran | +| [HADOOP-3723](https://issues.apache.org/jira/browse/HADOOP-3723) | libhdfs only accepts O\_WRONLY and O\_RDONLY so does not accept things like O\_WRONLY \| O\_CREAT | Minor | . | Pete Wyckoff | Pi Song | +| [HADOOP-3720](https://issues.apache.org/jira/browse/HADOOP-3720) | dfsadmin -refreshNodes should re-read the config file. | Major | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3711](https://issues.apache.org/jira/browse/HADOOP-3711) | Streaming input is not parsed properly to find the separator | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3705](https://issues.apache.org/jira/browse/HADOOP-3705) | CompositeInputFormat is unable to parse InputFormat classes with names containing '\_' or '$' | Major | . | Jingkei Ly | Chris Douglas | +| [HADOOP-3658](https://issues.apache.org/jira/browse/HADOOP-3658) | Incorrect destination IP logged for receiving blocks | Minor | . | Koji Noguchi | Chris Douglas | +| [HADOOP-3643](https://issues.apache.org/jira/browse/HADOOP-3643) | jobtasks.jsp when called for running tasks should ignore completed TIPs | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-3640](https://issues.apache.org/jira/browse/HADOOP-3640) | NativeS3FsInputStream read() method for reading a single byte is incorrect | Major | fs/s3 | Tom White | Tom White | +| [HADOOP-3623](https://issues.apache.org/jira/browse/HADOOP-3623) | LeaseManager needs refactoring. | Major | . | Konstantin Shvachko | Tsz Wo Nicholas Sze | +| [HADOOP-3592](https://issues.apache.org/jira/browse/HADOOP-3592) | org.apache.hadoop.fs.FileUtil.copy() will leak input streams if the destination can't be opened | Minor | fs | Steve Loughran | Bill de hOra | +| [HADOOP-3570](https://issues.apache.org/jira/browse/HADOOP-3570) | Including user specified jar files in the client side classpath path in Hadoop 0.17 streaming | Major | . | Suhas Gogate | Sharad Agarwal | +| [HADOOP-3560](https://issues.apache.org/jira/browse/HADOOP-3560) | Archvies sometimes create empty part files. | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3543](https://issues.apache.org/jira/browse/HADOOP-3543) | Need to increment the year field for the copyright notice | Trivial | documentation | Chris Douglas | Chris Douglas | +| [HADOOP-3542](https://issues.apache.org/jira/browse/HADOOP-3542) | Hadoop archives should not create \_logs file in the final archive directory. | Blocker | . | Mahadev konar | Mahadev konar | +| [HADOOP-3528](https://issues.apache.org/jira/browse/HADOOP-3528) | Metrics FilesCreated and files\_deleted metrics do not match. | Blocker | metrics | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-3506](https://issues.apache.org/jira/browse/HADOOP-3506) | Occasional NPE in Jets3tFileSystemStore | Major | fs/s3 | Robert | Tom White | +| [HADOOP-3488](https://issues.apache.org/jira/browse/HADOOP-3488) | the rsync command in hadoop-daemon.sh also rsync the logs folder from the master, what deletes the datanode / tasktracker log files. | Critical | scripts | Stefan Groschupf | Craig Macdonald | +| [HADOOP-3319](https://issues.apache.org/jira/browse/HADOOP-3319) | [HOD]checknodes prints errors messages on stdout | Major | contrib/hod | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-3155](https://issues.apache.org/jira/browse/HADOOP-3155) | reducers stuck at shuffling | Blocker | . | Runping Qi | dhruba borthakur | +| [HADOOP-3131](https://issues.apache.org/jira/browse/HADOOP-3131) | enabling BLOCK compression for map outputs breaks the reduce progress counters | Major | . | Colin Evans | Matei Zaharia | +| [HADOOP-3076](https://issues.apache.org/jira/browse/HADOOP-3076) | [HOD] If a cluster directory is specified as a relative path, an existing script.exitcode file will not be deleted. | Blocker | contrib/hod | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-2403](https://issues.apache.org/jira/browse/HADOOP-2403) | JobHistory log files contain data that cannot be parsed by org.apache.hadoop.mapred.JobHistory | Critical | . | Runping Qi | Amareshwari Sriramadasu | +| [HADOOP-2168](https://issues.apache.org/jira/browse/HADOOP-2168) | Pipes with a C++ record reader does not update progress in the map until it is 100% | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-1945](https://issues.apache.org/jira/browse/HADOOP-1945) | pipes examples aren't in the release | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-9](https://issues.apache.org/jira/browse/HADOOP-9) | mapred.local.dir temp dir. space allocation limited by smallest area | Minor | . | Paul Baclace | Ari Rabkin | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4464](https://issues.apache.org/jira/browse/HADOOP-4464) | Separate testClientTriggeredLeaseRecovery() out from TestFileCreation | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4426](https://issues.apache.org/jira/browse/HADOOP-4426) | TestCapacityScheduler is broken | Blocker | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4390](https://issues.apache.org/jira/browse/HADOOP-4390) | Hive: test for case sensitivity in serde2 thrift serde | Minor | . | Zheng Shao | | +| [HADOOP-4259](https://issues.apache.org/jira/browse/HADOOP-4259) | findbugs should run over the tools.jar also | Minor | test | Owen O'Malley | Chris Douglas | +| [HADOOP-4237](https://issues.apache.org/jira/browse/HADOOP-4237) | TestStreamingBadRecords.testNarrowDown fails intermittently | Minor | test | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4069](https://issues.apache.org/jira/browse/HADOOP-4069) | TestKosmosFileSystem can fail when run through ant test on systems shared by users | Minor | fs | Hemanth Yamijala | Lohit Vijayarenu | +| [HADOOP-4056](https://issues.apache.org/jira/browse/HADOOP-4056) | Unit test for DynamicSerDe | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3966](https://issues.apache.org/jira/browse/HADOOP-3966) | Place the new findbugs warnings introduced by the patch in the /tmp directory when "ant test-patch" is run. | Minor | test | Ramya Sunil | Ramya Sunil | +| [HADOOP-3790](https://issues.apache.org/jira/browse/HADOOP-3790) | Add more unit tests to test appending to files in HDFS | Blocker | test | dhruba borthakur | Tsz Wo Nicholas Sze | +| [HADOOP-3587](https://issues.apache.org/jira/browse/HADOOP-3587) | contrib/data\_join needs unit tests | Major | test | Chris Douglas | Chris Douglas | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4354](https://issues.apache.org/jira/browse/HADOOP-4354) | Separate TestDatanodeDeath.testDatanodeDeath() into 4 tests | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4186](https://issues.apache.org/jira/browse/HADOOP-4186) | Move LineRecordReader.LineReader class to util package | Major | . | Tom White | Tom White | +| [HADOOP-4184](https://issues.apache.org/jira/browse/HADOOP-4184) | Fix simple module dependencies between core, hdfs and mapred | Major | . | Tom White | Tom White | +| [HADOOP-3824](https://issues.apache.org/jira/browse/HADOOP-3824) | Refactor org.apache.hadoop.mapred.StatusHttpServer | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-3601](https://issues.apache.org/jira/browse/HADOOP-3601) | Hive as a contrib project | Minor | . | Joydeep Sen Sarma | Ashish Thusoo | +| [HADOOP-4105](https://issues.apache.org/jira/browse/HADOOP-4105) | libhdfs wiki is very out-of-date and contains mostly broken links | Minor | documentation | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4076](https://issues.apache.org/jira/browse/HADOOP-4076) | fuse-dfs REAME lists wrong ant flags and is not specific in some place | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3942](https://issues.apache.org/jira/browse/HADOOP-3942) | Update DistCp documentation | Blocker | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3791](https://issues.apache.org/jira/browse/HADOOP-3791) | Use generics in ReflectionUtils | Trivial | . | Chris Smith | Chris Smith | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/RELEASENOTES.0.19.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/RELEASENOTES.0.19.0.md new file mode 100644 index 00000000000..2348e76166c --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.0/RELEASENOTES.0.19.0.md @@ -0,0 +1,529 @@ + + +# Apache Hadoop 0.19.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-4466](https://issues.apache.org/jira/browse/HADOOP-4466) | *Blocker* | **SequenceFileOutputFormat is coupled to WritableComparable and Writable** + +Ensure that SequenceFileOutputFormat isn't tied to Writables and can be used with other Serialization frameworks. + + +--- + +* [HADOOP-4433](https://issues.apache.org/jira/browse/HADOOP-4433) | *Major* | **Improve data loader for collecting metrics and log files from hadoop and system** + +- Added startup and shutdown script +- Added torque metrics data loader +- Improve handling of Exec Plugin +- Added Test cases for File Tailing Adaptors +- Added Test cases for Start streaming at specific offset + + +--- + +* [HADOOP-4430](https://issues.apache.org/jira/browse/HADOOP-4430) | *Blocker* | **Namenode Web UI capacity report is inconsistent with Balancer** + +Changed reporting in the NameNode Web UI to more closely reflect the behavior of the re-balancer. Removed no longer used config parameter dfs.datanode.du.pct from hadoop-default.xml. + + +--- + +* [HADOOP-4293](https://issues.apache.org/jira/browse/HADOOP-4293) | *Major* | **Remove WritableJobConf** + +Made Configuration Writable and rename the old write method to writeXml. + + +--- + +* [HADOOP-4281](https://issues.apache.org/jira/browse/HADOOP-4281) | *Blocker* | **Capacity reported in some of the commands is not consistent with the Web UI reported data** + +Changed command "hadoop dfsadmin -report" to be consistent with Web UI for both Namenode and Datanode reports. "Total raw bytes" is changed to "Configured Capacity". "Present Capacity" is newly added to indicate the present capacity of the DFS. "Remaining raw bytes" is changed to "DFS Remaining". "Used raw bytes" is changed to "DFS Used". "% used" is changed to "DFS Used%". Applications that parse command output should be reviewed. + + +--- + +* [HADOOP-4227](https://issues.apache.org/jira/browse/HADOOP-4227) | *Minor* | **Remove the deprecated, unused class ShellCommand.** + +Removed the deprecated class org.apache.hadoop.fs.ShellCommand. + + +--- + +* [HADOOP-4205](https://issues.apache.org/jira/browse/HADOOP-4205) | *Major* | **[Hive] metastore and ql to use the refactored SerDe library** + +Improved Hive metastore and ql to use the refactored SerDe library. + + +--- + +* [HADOOP-4190](https://issues.apache.org/jira/browse/HADOOP-4190) | *Blocker* | **Changes to JobHistory makes it backward incompatible** + +Changed job history format to add a dot at end of each line. + + +--- + +* [HADOOP-4176](https://issues.apache.org/jira/browse/HADOOP-4176) | *Major* | **Implement getFileChecksum(Path) in HftpFileSystem** + +Implemented getFileChecksum(Path) in HftpFileSystemfor distcp support. + + +--- + +* [HADOOP-4138](https://issues.apache.org/jira/browse/HADOOP-4138) | *Major* | **[Hive] refactor the SerDe library** + +Introduced new SerDe library for src/contrib/hive. + + +--- + +* [HADOOP-4117](https://issues.apache.org/jira/browse/HADOOP-4117) | *Major* | **Improve configurability of Hadoop EC2 instances** + +Changed scripts to pass initialization script for EC2 instances at boot time (as EC2 user data) rather than embedding initialization information in the EC2 image. This change makes it easy to customize the hadoop-site.xml file for your cluster before launch, by editing the hadoop-ec2-init-remote.sh script, or by setting the environment variable USER\_DATA\_FILE in hadoop-ec2-env.sh to run a script of your choice. + + +--- + +* [HADOOP-4116](https://issues.apache.org/jira/browse/HADOOP-4116) | *Blocker* | **Balancer should provide better resource management** + +Changed DataNode protocol version without impact to clients other than to compel use of current version of client application. + + +--- + +* [HADOOP-4106](https://issues.apache.org/jira/browse/HADOOP-4106) | *Major* | **add time, permission and user attribute support to fuse-dfs** + +Added time, permission and user attribute support to libhdfs. + + +--- + +* [HADOOP-4086](https://issues.apache.org/jira/browse/HADOOP-4086) | *Major* | **Add limit to Hive QL** + +Added LIMIT to Hive query language. + + +--- + +* [HADOOP-4084](https://issues.apache.org/jira/browse/HADOOP-4084) | *Major* | **Add explain plan capabilities to Hive QL** + +Introduced "EXPLAIN" plan for Hive. + + +--- + +* [HADOOP-4018](https://issues.apache.org/jira/browse/HADOOP-4018) | *Major* | **limit memory usage in jobtracker** + +Introduced new configuration parameter mapred.max.tasks.per.job to specifie the maximum number of tasks per job. + + +--- + +* [HADOOP-3992](https://issues.apache.org/jira/browse/HADOOP-3992) | *Major* | **Synthetic Load Generator for NameNode testing** + +Added a synthetic load generation facility to the test directory. + + +--- + +* [HADOOP-3981](https://issues.apache.org/jira/browse/HADOOP-3981) | *Major* | **Need a distributed file checksum algorithm for HDFS** + +Implemented MD5-of-xxxMD5-of-yyyCRC32 which is a distributed file checksum algorithm for HDFS, where xxx is the number of CRCs per block and yyy is the number of bytes per CRC. + +Changed DistCp to use file checksum for comparing files if both source and destination FileSystem(s) support getFileChecksum(...). + + +--- + +* [HADOOP-3970](https://issues.apache.org/jira/browse/HADOOP-3970) | *Major* | **Counters written to the job history cannot be recovered back** + +Added getEscapedCompactString() and fromEscapedCompactString() to Counters.java to represent counters as Strings and to reconstruct the counters from the Strings. + + +--- + +* [HADOOP-3963](https://issues.apache.org/jira/browse/HADOOP-3963) | *Minor* | **libhdfs should never exit on its own but rather return errors to the calling application** + +Modified libhdfs to return NULL or error code when unrecoverable error occurs rather than exiting itself. + + +--- + +* [HADOOP-3941](https://issues.apache.org/jira/browse/HADOOP-3941) | *Major* | **Extend FileSystem API to return file-checksums/file-digests** + +Added new FileSystem APIs: FileChecksum and FileSystem.getFileChecksum(Path). + + +--- + +* [HADOOP-3939](https://issues.apache.org/jira/browse/HADOOP-3939) | *Major* | **DistCp should support an option for deleting non-existing files.** + +Added a new option -delete to DistCp so that if the files/directories exist in dst but not in src will be deleted. It uses FsShell to do delete, so that it will use trash if the trash is enable. + + +--- + +* [HADOOP-3938](https://issues.apache.org/jira/browse/HADOOP-3938) | *Major* | **Quotas for disk space management** + +Introducted byte space quotas for directories. The count shell command modified to report both name and byte quotas. + + +--- + +* [HADOOP-3930](https://issues.apache.org/jira/browse/HADOOP-3930) | *Major* | **Decide how to integrate scheduler info into CLI and job tracker web page** + +Changed TaskScheduler to expose API for Web UI and Command Line Tool. + + +--- + +* [HADOOP-3911](https://issues.apache.org/jira/browse/HADOOP-3911) | *Minor* | **' -blocks ' option not being recognized** + +Added a check to fsck options to make sure -files is not the first option so as to resolve conflicts with GenericOptionsParser. + + +--- + +* [HADOOP-3908](https://issues.apache.org/jira/browse/HADOOP-3908) | *Minor* | **Better error message if llibhdfs.so doesn't exist** + +Improved Fuse-dfs better error message if llibhdfs.so doesn't exist. + + +--- + +* [HADOOP-3889](https://issues.apache.org/jira/browse/HADOOP-3889) | *Minor* | **distcp: Better Error Message should be thrown when accessing source files/directory with no read permission** + +Changed DistCp error messages when there is a RemoteException. Changed the corresponding return value from -999 to -3. + + +--- + +* [HADOOP-3873](https://issues.apache.org/jira/browse/HADOOP-3873) | *Major* | **DistCp should have an option for limiting the number of files/bytes being copied** + +Added two new options -filelimit \ and -sizelimit \ to DistCp for limiting the total number of files and the total size in bytes, respectively. + + +--- + +* [HADOOP-3854](https://issues.apache.org/jira/browse/HADOOP-3854) | *Major* | **org.apache.hadoop.http.HttpServer should support user configurable filter** + +Added a configuration property hadoop.http.filter.initializers and a class org.apache.hadoop.http.FilterInitializer for supporting servlet filter. Cluster administrator could possibly configure customized filters for their web site. + + +--- + +* [HADOOP-3837](https://issues.apache.org/jira/browse/HADOOP-3837) | *Major* | **hadop streaming does not use progress reporting to detect hung tasks** + +Changed streaming tasks to adhere to task timeout value specified in the job configuration. + + +--- + +* [HADOOP-3829](https://issues.apache.org/jira/browse/HADOOP-3829) | *Major* | **Narrown down skipped records based on user acceptable value** + +Introduced new config parameter org.apache.hadoop.mapred.SkipBadRecords.setMapperMaxSkipRecords to set range of records to be skipped in the neighborhood of a failed record. + + +--- + +* [HADOOP-3828](https://issues.apache.org/jira/browse/HADOOP-3828) | *Major* | **Write skipped records' bytes to DFS** + +Skipped records can optionally be written to the HDFS. Refer org.apache.hadoop.mapred.SkipBadRecords.setSkipOutputPath for setting the output path. + + +--- + +* [HADOOP-3796](https://issues.apache.org/jira/browse/HADOOP-3796) | *Major* | **fuse-dfs should take rw,ro,trashon,trashoff,protected=blah mount arguments rather than them being compiled in** + +Changed Fuse configuration to use mount options. + + +--- + +* [HADOOP-3792](https://issues.apache.org/jira/browse/HADOOP-3792) | *Minor* | **exit code from "hadoop dfs -test ..." is wrong for Unix shell** + +Changed exit code from hadoop.fs.FsShell -test to match the usual Unix convention. + + +--- + +* [HADOOP-3746](https://issues.apache.org/jira/browse/HADOOP-3746) | *Minor* | **A fair sharing job scheduler** + +Introduced Fair Scheduler. + + +--- + +* [HADOOP-3730](https://issues.apache.org/jira/browse/HADOOP-3730) | *Major* | **add new JobConf constructor that disables loading default configurations** + + Added a JobConf constructor that disables loading default configurations so as to take all default values from the JobTracker's configuration. + + +--- + +* [HADOOP-3722](https://issues.apache.org/jira/browse/HADOOP-3722) | *Minor* | **Provide a unified way to pass jobconf options from bin/hadoop** + +Changed streaming StreamJob and Submitter to implement Tool and Configurable, and to use GenericOptionsParser arguments -fs, -jt, -conf, -D, -libjars, -files, and -archives. Deprecated -jobconf, -cacheArchive, -dfs, -cacheArchive, -additionalconfspec, from streaming and pipes in favor of the generic options. Removed from streaming -config, -mapred.job.tracker, and -cluster. + + +--- + +* [HADOOP-3719](https://issues.apache.org/jira/browse/HADOOP-3719) | *Major* | **Chukwa** + +Introduced Chukwa data collection and analysis framework. + + +--- + +* [HADOOP-3714](https://issues.apache.org/jira/browse/HADOOP-3714) | *Trivial* | **Bash tab completion support** + +Adds a new contrib, bash-tab-completion, which enables bash tab completion for the bin/hadoop script. See the README file in the contrib directory for the installation. + + +--- + +* [HADOOP-3702](https://issues.apache.org/jira/browse/HADOOP-3702) | *Major* | **add support for chaining Maps in a single Map and after a Reduce [M\*/RM\*]** + +Introduced ChainMapper and the ChainReducer classes to allow composing chains of Maps and Reduces in a single Map/Reduce job, something like MAP+ REDUCE MAP*. + + +--- + +* [HADOOP-3695](https://issues.apache.org/jira/browse/HADOOP-3695) | *Major* | **[HOD] Have an ability to run multiple slaves per node** + +Added an ability in HOD to start multiple workers (TaskTrackers and/or DataNodes) per node to assist testing and simulation of scale. A configuration variable ringmaster.workers\_per\_ring was added to specify the number of workers to start. + + +--- + +* [HADOOP-3684](https://issues.apache.org/jira/browse/HADOOP-3684) | *Major* | **The data\_join should allow the user to implement a customer cloning function** + +Allowed user to overwrite clone function in a subclass of TaggedMapOutput class. + + +--- + +* [HADOOP-3667](https://issues.apache.org/jira/browse/HADOOP-3667) | *Major* | **Remove deprecated methods in JobConf** + +Removed the following deprecated methods from JobConf: + addInputPath(Path) + getInputPaths() + getMapOutputCompressionType() + getOutputPath() + getSystemDir() + setInputPath(Path) + setMapOutputCompressionType(CompressionType style) + setOutputPath(Path) + + +--- + +* [HADOOP-3652](https://issues.apache.org/jira/browse/HADOOP-3652) | *Major* | **Remove deprecated class OutputFormatBase** + +Removed deprecated org.apache.hadoop.mapred.OutputFormatBase. + + +--- + +* [HADOOP-3646](https://issues.apache.org/jira/browse/HADOOP-3646) | *Major* | **Providing bzip2 as codec** + +Introduced support for bzip2 compressed files. + + +--- + +* [HADOOP-3601](https://issues.apache.org/jira/browse/HADOOP-3601) | *Minor* | **Hive as a contrib project** + +Introduced Hive Data Warehouse built on top of Hadoop that enables structuring Hadoop files as tables and partitions and allows users to query this data through a SQL like language using a command line interface. + + +--- + +* [HADOOP-3595](https://issues.apache.org/jira/browse/HADOOP-3595) | *Major* | **Remove deprecated mapred.combine.once functionality** + + Removed deprecated methods for mapred.combine.once functionality. + + +--- + +* [HADOOP-3585](https://issues.apache.org/jira/browse/HADOOP-3585) | *Minor* | **Hardware Failure Monitoring in large clusters running Hadoop/HDFS** + +Added FailMon as a contrib project for hardware failure monitoring and analysis, under /src/contrib/failmon. Created User Manual and Quick Start Guide. + + +--- + +* [HADOOP-3581](https://issues.apache.org/jira/browse/HADOOP-3581) | *Major* | **Prevent memory intensive user tasks from taking down nodes** + +Added the ability to kill process trees transgressing memory limits. TaskTracker uses the configuration parameters introduced in HADOOP-3759. In addition, mapred.tasktracker.taskmemorymanager.monitoring-interval specifies the interval for which TT waits between cycles of monitoring tasks' memory usage, and mapred.tasktracker.procfsbasedprocesstree.sleeptime-before-sigkill specifies the time TT waits for sending a SIGKILL to a process-tree that has overrun memory limits, after it has been sent a SIGTERM. + + +--- + +* [HADOOP-3549](https://issues.apache.org/jira/browse/HADOOP-3549) | *Major* | **meaningful errno values in libhdfs** + +Improved error reporting for libhdfs so permission problems now return EACCES. + + +--- + +* [HADOOP-3498](https://issues.apache.org/jira/browse/HADOOP-3498) | *Major* | **File globbing alternation should be able to span path components** + +Extended file globbing alternation to cross path components. For example, {/a/b,/c/d} expands to a path that matches the files /a/b and /c/d. + + +--- + +* [HADOOP-3485](https://issues.apache.org/jira/browse/HADOOP-3485) | *Minor* | **fix writes** + +Introduce write support for Fuse; requires Linux kernel 2.6.15 or better. + + +--- + +* [HADOOP-3478](https://issues.apache.org/jira/browse/HADOOP-3478) | *Major* | **The algorithm to decide map re-execution on fetch failures can be improved** + +Changed reducers to fetch maps in the same order for a given host to speed up identification of the faulty maps; reducers still randomize the host selection to distribute load. + + +--- + +* [HADOOP-3445](https://issues.apache.org/jira/browse/HADOOP-3445) | *Major* | **Implementing core scheduler functionality in Resource Manager (V1) for Hadoop** + +Introduced Capacity Task Scheduler. + + +--- + +* [HADOOP-3412](https://issues.apache.org/jira/browse/HADOOP-3412) | *Minor* | **Refactor the scheduler out of the JobTracker** + +Added the ability to chose between many schedulers, and to limit the number of running tasks per job. + + +--- + +* [HADOOP-3245](https://issues.apache.org/jira/browse/HADOOP-3245) | *Major* | **Provide ability to persist running jobs (extend HADOOP-1876)** + +Introduced recovery of jobs when JobTracker restarts. This facility is off by default. Introduced config parameters mapred.jobtracker.restart.recover, mapred.jobtracker.job.history.block.size, and mapred.jobtracker.job.history.buffer.size. + + +--- + +* [HADOOP-3150](https://issues.apache.org/jira/browse/HADOOP-3150) | *Major* | **Move task file promotion into the task** + +Moved task file promotion to the Task. When the task has finished, it will do a commit and is declared SUCCEDED. Job cleanup is done by a separate task. Job is declared SUCCEDED/FAILED after the cleanup task has finished. Added public classes org.apache.hadoop.mapred.JobContext, TaskAttemptContext, OutputCommitter and FileOutputCommiitter. Added public APIs: public OutputCommitter getOutputCommitter() and +public void setOutputCommitter(Class\ theClass) in org.apache.hadoop.mapred.JobConf + + +--- + +* [HADOOP-3149](https://issues.apache.org/jira/browse/HADOOP-3149) | *Major* | **supporting multiple outputs for M/R jobs** + +Introduced MultipleOutputs class so Map/Reduce jobs can write data to different output files. Each output can use a different OutputFormat. Outpufiles are created within the job output directory. FileOutputFormat.getPathForCustomFile() creates a filename under the outputdir that is named with the task ID and task type (i.e. myfile-r-00001). + + +--- + +* [HADOOP-3062](https://issues.apache.org/jira/browse/HADOOP-3062) | *Major* | **Need to capture the metrics for the network ios generate by dfs reads/writes and map/reduce shuffling and break them down by racks** + +Introduced additional log records for data transfers. + + +--- + +* [HADOOP-3019](https://issues.apache.org/jira/browse/HADOOP-3019) | *Major* | **want input sampler & sorted partitioner** + +Added a partitioner that effects a total order of output data, and an input sampler for generating the partition keyset for TotalOrderPartitioner for when the map's input keytype and distribution approximates its output. + + +--- + +* [HADOOP-2816](https://issues.apache.org/jira/browse/HADOOP-2816) | *Major* | **Cluster summary at name node web has confusing report for space utilization** + +Improved space reporting for NameNode Web UI. Applications that parse the Web UI output should be reviewed. + + +--- + +* [HADOOP-2664](https://issues.apache.org/jira/browse/HADOOP-2664) | *Major* | **lzop-compatible CompresionCodec** + +Introduced LZOP codec. + + +--- + +* [HADOOP-2411](https://issues.apache.org/jira/browse/HADOOP-2411) | *Major* | **Add support for larger EC2 instance types** + +Added support for c1.* instance types and associated kernels for EC2. + + +--- + +* [HADOOP-2325](https://issues.apache.org/jira/browse/HADOOP-2325) | *Major* | **Require Java 6** + +Hadoop now requires Java 6. + + +--- + +* [HADOOP-2302](https://issues.apache.org/jira/browse/HADOOP-2302) | *Major* | ** Streaming should provide an option for numerical sort of keys** + +Introduced numerical key comparison for streaming. + + +--- + +* [HADOOP-1869](https://issues.apache.org/jira/browse/HADOOP-1869) | *Major* | **access times of HDFS files** + +Added HDFS file access times. By default, access times will be precise to the most recent hour boundary. A configuration parameter dfs.access.time.precision (milliseconds) is used to control this precision. Setting a value of 0 will disable persisting access times for HDFS files. + + +--- + +* [HADOOP-1823](https://issues.apache.org/jira/browse/HADOOP-1823) | *Major* | **want InputFormat for bzip2 files** + +bzip2 provided as codec in 0.19.0 https://issues.apache.org/jira/browse/HADOOP-3646 + + +--- + +* [HADOOP-1700](https://issues.apache.org/jira/browse/HADOOP-1700) | *Major* | **Append to files in HDFS** + +Introduced append operation for HDFS files. + + +--- + +* [HADOOP-249](https://issues.apache.org/jira/browse/HADOOP-249) | *Major* | **Improving Map -\> Reduce performance and Task JVM reuse** + +Enabled task JVMs to be reused via the job config mapred.job.reuse.jvm.num.tasks. + + +--- + +* [HADOOP-153](https://issues.apache.org/jira/browse/HADOOP-153) | *Major* | **skip records that fail Task** + +Introduced record skipping where tasks fail on certain records. (org.apache.hadoop.mapred.SkipBadRecords) + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/CHANGES.0.19.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/CHANGES.0.19.1.md new file mode 100644 index 00000000000..275ea5b2c09 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/CHANGES.0.19.1.md @@ -0,0 +1,112 @@ + + +# Apache Hadoop Changelog + +## Release 0.19.1 - 2009-02-24 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5225](https://issues.apache.org/jira/browse/HADOOP-5225) | workaround for tmp file handling on DataNodes in 0.19.1 (HADOOP-4663) | Blocker | . | Nigel Daley | Raghu Angadi | +| [HADOOP-5224](https://issues.apache.org/jira/browse/HADOOP-5224) | Disable append | Blocker | . | Nigel Daley | | +| [HADOOP-4061](https://issues.apache.org/jira/browse/HADOOP-4061) | Large number of decommission freezes the Namenode | Major | . | Koji Noguchi | Tsz Wo Nicholas Sze | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5034](https://issues.apache.org/jira/browse/HADOOP-5034) | NameNode should send both replication and deletion requests to DataNode in one reply to a heartbeat | Major | . | Hairong Kuang | Hairong Kuang | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5127](https://issues.apache.org/jira/browse/HADOOP-5127) | FSDirectory should not have public methods. | Major | . | Konstantin Shvachko | Jakob Homan | +| [HADOOP-5086](https://issues.apache.org/jira/browse/HADOOP-5086) | Trash URI semantics can be relaxed | Minor | fs | Chris Douglas | Chris Douglas | +| [HADOOP-4739](https://issues.apache.org/jira/browse/HADOOP-4739) | Minor enhancements to some sections of the Map/Reduce tutorial | Trivial | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-3894](https://issues.apache.org/jira/browse/HADOOP-3894) | DFSClient chould log errors better, and provide better diagnostics | Trivial | . | Steve Loughran | Steve Loughran | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5665](https://issues.apache.org/jira/browse/HADOOP-5665) | Namenode could not be formatted because the "whoami" program could not be run. | Major | . | Evelyn Sylvia | | +| [HADOOP-5268](https://issues.apache.org/jira/browse/HADOOP-5268) | Using MultipleOutputFormat and setting reducers to 0 causes org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException and job to fail | Major | . | Thibaut | | +| [HADOOP-5193](https://issues.apache.org/jira/browse/HADOOP-5193) | SecondaryNameNode does not rollImage because of incorrect calculation of edits modification time. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-5192](https://issues.apache.org/jira/browse/HADOOP-5192) | Block reciever should not remove a finalized block when block replication fails | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5166](https://issues.apache.org/jira/browse/HADOOP-5166) | JobTracker fails to restart if recovery and ACLs are enabled | Blocker | . | Karam Singh | Amar Kamat | +| [HADOOP-5161](https://issues.apache.org/jira/browse/HADOOP-5161) | Accepted sockets do not get placed in DataXceiverServer#childSockets | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5156](https://issues.apache.org/jira/browse/HADOOP-5156) | TestHeartbeatHandling uses MiniDFSCluster.getNamesystem() which does not exist in branch 0.20 | Major | test | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-5134](https://issues.apache.org/jira/browse/HADOOP-5134) | FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap | Blocker | . | Hairong Kuang | dhruba borthakur | +| [HADOOP-5067](https://issues.apache.org/jira/browse/HADOOP-5067) | Failed/Killed attempts column in jobdetails.jsp does not show the number of failed/killed attempts correctly | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5009](https://issues.apache.org/jira/browse/HADOOP-5009) | DataNode#shutdown sometimes leaves data block scanner verification log unclosed | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5008](https://issues.apache.org/jira/browse/HADOOP-5008) | TestReplication#testPendingReplicationRetry leaves an opened fd unclosed | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-5002](https://issues.apache.org/jira/browse/HADOOP-5002) | 2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19 | Blocker | . | Ravi Gummadi | Amareshwari Sriramadasu | +| [HADOOP-4992](https://issues.apache.org/jira/browse/HADOOP-4992) | TestCustomOutputCommitter fails on hadoop-0.19 | Blocker | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-4983](https://issues.apache.org/jira/browse/HADOOP-4983) | Job counters sometimes go down as tasks run without task failures | Critical | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-4982](https://issues.apache.org/jira/browse/HADOOP-4982) | TestFsck does not run in Eclipse. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4967](https://issues.apache.org/jira/browse/HADOOP-4967) | Inconsistent state in JVM manager | Major | . | Amareshwari Sriramadasu | Devaraj Das | +| [HADOOP-4966](https://issues.apache.org/jira/browse/HADOOP-4966) | Setup tasks are not removed from JobTracker's taskIdToTIPMap even after the job completes | Major | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-4965](https://issues.apache.org/jira/browse/HADOOP-4965) | DFSClient should log instead of printing into std err. | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4955](https://issues.apache.org/jira/browse/HADOOP-4955) | Make DBOutputFormat us column names from setOutput(...) | Major | . | Kevin Peterson | Kevin Peterson | +| [HADOOP-4943](https://issues.apache.org/jira/browse/HADOOP-4943) | fair share scheduler does not utilize all slots if the task trackers are configured heterogeneously | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-4924](https://issues.apache.org/jira/browse/HADOOP-4924) | Race condition in re-init of TaskTracker | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4918](https://issues.apache.org/jira/browse/HADOOP-4918) | Fix bzip2 work with SequenceFile | Major | io | Zheng Shao | Zheng Shao | +| [HADOOP-4906](https://issues.apache.org/jira/browse/HADOOP-4906) | TaskTracker running out of memory after running several tasks | Blocker | . | Arun C Murthy | Sharad Agarwal | +| [HADOOP-4862](https://issues.apache.org/jira/browse/HADOOP-4862) | A spurious IOException log on DataNode is not completely removed | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4847](https://issues.apache.org/jira/browse/HADOOP-4847) | OutputCommitter is loaded in the TaskTracker in localizeConfiguration | Blocker | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-4836](https://issues.apache.org/jira/browse/HADOOP-4836) | Minor typos in documentation and comments | Trivial | documentation | JordĂ  Polo | JordĂ  Polo | +| [HADOOP-4821](https://issues.apache.org/jira/browse/HADOOP-4821) | Usage description in the Quotas guide documentations are incorrect | Minor | documentation | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4797](https://issues.apache.org/jira/browse/HADOOP-4797) | RPC Server can leave a lot of direct buffers | Blocker | ipc | Raghu Angadi | Raghu Angadi | +| [HADOOP-4760](https://issues.apache.org/jira/browse/HADOOP-4760) | HDFS streams should not throw exceptions when closed twice | Major | fs, fs/s3 | Alejandro Abdelnur | Enis Soztutar | +| [HADOOP-4759](https://issues.apache.org/jira/browse/HADOOP-4759) | HADOOP-4654 to be fixed for branches \>= 0.19 | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4731](https://issues.apache.org/jira/browse/HADOOP-4731) | Job is not removed from the waiting jobs queue upon completion. | Major | . | Hemanth Yamijala | Amar Kamat | +| [HADOOP-4727](https://issues.apache.org/jira/browse/HADOOP-4727) | Groups do not work for fuse-dfs out of the box on 0.19.0 | Blocker | . | Brian Bockelman | Brian Bockelman | +| [HADOOP-4720](https://issues.apache.org/jira/browse/HADOOP-4720) | docs/api does not contain the hdfs directory after building | Major | build | Ramya Sunil | | +| [HADOOP-4697](https://issues.apache.org/jira/browse/HADOOP-4697) | KFS::getBlockLocations() fails with files having multiple blocks | Major | fs | Lohit Vijayarenu | Sriram Rao | +| [HADOOP-4635](https://issues.apache.org/jira/browse/HADOOP-4635) | Memory leak ? | Blocker | . | Marc-Olivier Fleury | Pete Wyckoff | +| [HADOOP-4632](https://issues.apache.org/jira/browse/HADOOP-4632) | TestJobHistoryVersion should not create directory in current dir. | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-4616](https://issues.apache.org/jira/browse/HADOOP-4616) | assertion makes fuse-dfs exit when reading incomplete data | Blocker | . | Marc-Olivier Fleury | Pete Wyckoff | +| [HADOOP-4508](https://issues.apache.org/jira/browse/HADOOP-4508) | FSDataOutputStream.getPos() == 0when appending to existing file and should be file length | Major | fs | Pete Wyckoff | dhruba borthakur | +| [HADOOP-4494](https://issues.apache.org/jira/browse/HADOOP-4494) | libhdfs does not call FileSystem.append when O\_APPEND passed to hdfsOpenFile | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4420](https://issues.apache.org/jira/browse/HADOOP-4420) | JobTracker.killJob() doesn't check for the JobID being valid | Minor | . | Steve Loughran | Aaron Kimball | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/RELEASENOTES.0.19.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/RELEASENOTES.0.19.1.md new file mode 100644 index 00000000000..25984a4bb73 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.1/RELEASENOTES.0.19.1.md @@ -0,0 +1,101 @@ + + +# Apache Hadoop 0.19.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5225](https://issues.apache.org/jira/browse/HADOOP-5225) | *Blocker* | **workaround for tmp file handling on DataNodes in 0.19.1 (HADOOP-4663)** + +Work around for tmp file handling. sync() does not work as a result. + + +--- + +* [HADOOP-5224](https://issues.apache.org/jira/browse/HADOOP-5224) | *Blocker* | **Disable append** + +HDFS append() is disabled. It throws UnsupportedOperationException. + + +--- + +* [HADOOP-5034](https://issues.apache.org/jira/browse/HADOOP-5034) | *Major* | **NameNode should send both replication and deletion requests to DataNode in one reply to a heartbeat** + +This patch changes the DatanodeProtocoal version number from 18 to 19. The patch allows NameNode to send both block replication and deletion request to a DataNode in response to a heartbeat. + + +--- + +* [HADOOP-5002](https://issues.apache.org/jira/browse/HADOOP-5002) | *Blocker* | **2 core tests TestFileOutputFormat and TestHarFileSystem are failing in branch 19** + +This patch solves the null pointer exception issue in the 2 core tests TestFileOutputFormat and TestHarFileSystem in branch 19. + + +--- + +* [HADOOP-4943](https://issues.apache.org/jira/browse/HADOOP-4943) | *Major* | **fair share scheduler does not utilize all slots if the task trackers are configured heterogeneously** + +HADOOP-4943: Fixed fair share scheduler to utilize all slots when the task trackers are configured heterogeneously. + + +--- + +* [HADOOP-4906](https://issues.apache.org/jira/browse/HADOOP-4906) | *Blocker* | **TaskTracker running out of memory after running several tasks** + +Fix the tasktracker for OOM exception by sharing the jobconf properties across tasks of the same job. Earlier a new instance was held for each task. With this fix, the job level configuration properties are shared across tasks of the same job. + + +--- + +* [HADOOP-4862](https://issues.apache.org/jira/browse/HADOOP-4862) | *Blocker* | **A spurious IOException log on DataNode is not completely removed** + +Minor : HADOOP-3678 did not remove all the cases of spurious IOExceptions logged by DataNode. + + +--- + +* [HADOOP-4797](https://issues.apache.org/jira/browse/HADOOP-4797) | *Blocker* | **RPC Server can leave a lot of direct buffers** + +Improve how RPC server reads and writes large buffers. Avoids soft-leak of direct buffers and excess copies in NIO layer. + + +--- + +* [HADOOP-4635](https://issues.apache.org/jira/browse/HADOOP-4635) | *Blocker* | **Memory leak ?** + +fix memory leak of user/group information in fuse-dfs + + +--- + +* [HADOOP-4494](https://issues.apache.org/jira/browse/HADOOP-4494) | *Major* | **libhdfs does not call FileSystem.append when O\_APPEND passed to hdfsOpenFile** + +libhdfs supports O\_APPEND flag + + +--- + +* [HADOOP-4061](https://issues.apache.org/jira/browse/HADOOP-4061) | *Major* | **Large number of decommission freezes the Namenode** + +Added a new conf property dfs.namenode.decommission.nodes.per.interval so that NameNode checks decommission status of x nodes for every y seconds, where x is the value of dfs.namenode.decommission.nodes.per.interval and y is the value of dfs.namenode.decommission.interval. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/CHANGES.0.19.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/CHANGES.0.19.2.md new file mode 100644 index 00000000000..0a823a22566 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/CHANGES.0.19.2.md @@ -0,0 +1,104 @@ + + +# Apache Hadoop Changelog + +## Release 0.19.2 - 2009-07-23 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5332](https://issues.apache.org/jira/browse/HADOOP-5332) | Make support for file append API configurable | Blocker | . | Nigel Daley | dhruba borthakur | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5379](https://issues.apache.org/jira/browse/HADOOP-5379) | Throw exception instead of writing to System.err when there is a CRC error on CBZip2InputStream | Minor | io | Rodrigo Schmidt | Rodrigo Schmidt | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6017](https://issues.apache.org/jira/browse/HADOOP-6017) | NameNode and SecondaryNameNode fail to restart because of abnormal filenames. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-5951](https://issues.apache.org/jira/browse/HADOOP-5951) | StorageInfo needs Apache license header. | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5816](https://issues.apache.org/jira/browse/HADOOP-5816) | ArrayIndexOutOfBoundsException when using KeyFieldBasedComparator | Minor | . | Min Zhou | He Yongqiang | +| [HADOOP-5728](https://issues.apache.org/jira/browse/HADOOP-5728) | FSEditLog.printStatistics may cause IndexOutOfBoundsException | Major | . | Wang Xu | Wang Xu | +| [HADOOP-5671](https://issues.apache.org/jira/browse/HADOOP-5671) | DistCp.sameFile(..) should return true if src fs does not support checksum | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5644](https://issues.apache.org/jira/browse/HADOOP-5644) | Namnode is stuck in safe mode | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5579](https://issues.apache.org/jira/browse/HADOOP-5579) | libhdfs does not set errno correctly | Major | . | Brian Bockelman | Brian Bockelman | +| [HADOOP-5557](https://issues.apache.org/jira/browse/HADOOP-5557) | Two minor problems in TestOverReplicatedBlocks | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5554](https://issues.apache.org/jira/browse/HADOOP-5554) | DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-5551](https://issues.apache.org/jira/browse/HADOOP-5551) | Namenode permits directory destruction on overwrite | Critical | . | Brian Bockelman | Brian Bockelman | +| [HADOOP-5549](https://issues.apache.org/jira/browse/HADOOP-5549) | ReplicationMonitor should schedule both replication and deletion work in one iteration | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5522](https://issues.apache.org/jira/browse/HADOOP-5522) | Document job setup/cleaup tasks and task cleanup tasks in mapred tutorial | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5479](https://issues.apache.org/jira/browse/HADOOP-5479) | NameNode should not send empty block replication request to DataNode | Critical | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5465](https://issues.apache.org/jira/browse/HADOOP-5465) | Blocks remain under-replicated | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5449](https://issues.apache.org/jira/browse/HADOOP-5449) | Verify if JobHistory.HistoryCleaner works as expected | Blocker | . | Amar Kamat | Amareshwari Sriramadasu | +| [HADOOP-5446](https://issues.apache.org/jira/browse/HADOOP-5446) | TaskTracker metrics are disabled | Major | metrics | Chris Douglas | Chris Douglas | +| [HADOOP-5440](https://issues.apache.org/jira/browse/HADOOP-5440) | Successful taskid are not removed from TaskMemoryManager | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5421](https://issues.apache.org/jira/browse/HADOOP-5421) | HADOOP-4638 has broken 0.19 compilation | Blocker | . | Amar Kamat | Devaraj Das | +| [HADOOP-5392](https://issues.apache.org/jira/browse/HADOOP-5392) | JobTracker crashes during recovery if job files are garbled | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-5384](https://issues.apache.org/jira/browse/HADOOP-5384) | DataNodeCluster should not create blocks with generationStamp == 1 | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5376](https://issues.apache.org/jira/browse/HADOOP-5376) | JobInProgress.obtainTaskCleanupTask() throws an ArrayIndexOutOfBoundsException | Blocker | . | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [HADOOP-5374](https://issues.apache.org/jira/browse/HADOOP-5374) | NPE in JobTracker.getTasksToSave() method | Major | . | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [HADOOP-5333](https://issues.apache.org/jira/browse/HADOOP-5333) | The libhdfs append API is not coded correctly | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-5326](https://issues.apache.org/jira/browse/HADOOP-5326) | bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs | Major | io | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-5285](https://issues.apache.org/jira/browse/HADOOP-5285) | JobTracker hangs for long periods of time | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das | +| [HADOOP-5280](https://issues.apache.org/jira/browse/HADOOP-5280) | When expiring a lost launched task, JT doesn't remove the attempt from the taskidToTIPMap. | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das | +| [HADOOP-5269](https://issues.apache.org/jira/browse/HADOOP-5269) | TaskTracker.runningTasks holding FAILED\_UNCLEAN and KILLED\_UNCLEAN taskStatuses forever in some cases. | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5247](https://issues.apache.org/jira/browse/HADOOP-5247) | NPEs in JobTracker and JobClient when mapred.jobtracker.completeuserjobs.maximum is set to zero. | Blocker | . | Vinod Kumar Vavilapalli | Amar Kamat | +| [HADOOP-5241](https://issues.apache.org/jira/browse/HADOOP-5241) | Reduce tasks get stuck because of over-estimated task size (regression from 0.18) | Blocker | . | Andy Pavlo | Sharad Agarwal | +| [HADOOP-5233](https://issues.apache.org/jira/browse/HADOOP-5233) | Reducer not Succeded after 100% | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5231](https://issues.apache.org/jira/browse/HADOOP-5231) | Negative number of maps in cluster summary | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5213](https://issues.apache.org/jira/browse/HADOOP-5213) | BZip2CompressionOutputStream NullPointerException | Blocker | io | Zheng Shao | Zheng Shao | +| [HADOOP-5154](https://issues.apache.org/jira/browse/HADOOP-5154) | 4-way deadlock in FairShare scheduler | Blocker | . | Vinod Kumar Vavilapalli | Matei Zaharia | +| [HADOOP-5146](https://issues.apache.org/jira/browse/HADOOP-5146) | LocalDirAllocator misses files on the local filesystem | Blocker | . | Arun C Murthy | Devaraj Das | +| [HADOOP-4780](https://issues.apache.org/jira/browse/HADOOP-4780) | Task Tracker burns a lot of cpu in calling getLocalCache | Major | . | Runping Qi | He Yongqiang | +| [HADOOP-4719](https://issues.apache.org/jira/browse/HADOOP-4719) | The ls shell command documentation is out-dated | Major | documentation | Tsz Wo Nicholas Sze | Ravi Phulari | +| [HADOOP-4638](https://issues.apache.org/jira/browse/HADOOP-4638) | Exception thrown in/from RecoveryManager.recover() should be caught and handled | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-3998](https://issues.apache.org/jira/browse/HADOOP-3998) | Got an exception from ClientFinalizer when the JT is terminated | Blocker | . | Amar Kamat | dhruba borthakur | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/RELEASENOTES.0.19.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/RELEASENOTES.0.19.2.md new file mode 100644 index 00000000000..5cf2ebf6819 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.19.2/RELEASENOTES.0.19.2.md @@ -0,0 +1,38 @@ + + +# Apache Hadoop 0.19.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5332](https://issues.apache.org/jira/browse/HADOOP-5332) | *Blocker* | **Make support for file append API configurable** + +Appending to HDFS files is supported only if dfs.support.append is set to true. + + +--- + +* [HADOOP-4780](https://issues.apache.org/jira/browse/HADOOP-4780) | *Major* | **Task Tracker burns a lot of cpu in calling getLocalCache** + +make DistributedCache remember the size of each cache directory + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/CHANGES.0.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/CHANGES.0.2.0.md new file mode 100644 index 00000000000..9769965386b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/CHANGES.0.2.0.md @@ -0,0 +1,119 @@ + + +# Apache Hadoop Changelog + +## Release 0.2.0 - 2006-05-05 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-191](https://issues.apache.org/jira/browse/HADOOP-191) | add hadoopStreaming to src/contrib | Major | . | Michel Tourn | Doug Cutting | +| [HADOOP-189](https://issues.apache.org/jira/browse/HADOOP-189) | Add job jar lib, classes, etc. to CLASSPATH when in standalone mode | Major | . | stack | Doug Cutting | +| [HADOOP-148](https://issues.apache.org/jira/browse/HADOOP-148) | add a failure count to task trackers | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-132](https://issues.apache.org/jira/browse/HADOOP-132) | An API for reporting performance metrics | Major | . | David Bowen | | +| [HADOOP-65](https://issues.apache.org/jira/browse/HADOOP-65) | add a record I/O framework to hadoop | Minor | io, ipc | Sameer Paranjpye | | +| [HADOOP-51](https://issues.apache.org/jira/browse/HADOOP-51) | per-file replication counts | Major | . | Doug Cutting | Konstantin Shvachko | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-198](https://issues.apache.org/jira/browse/HADOOP-198) | adding owen's examples to exampledriver | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-178](https://issues.apache.org/jira/browse/HADOOP-178) | piggyback block work requests to heartbeats and move block replication/deletion startup delay from datanodes to namenode | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-177](https://issues.apache.org/jira/browse/HADOOP-177) | improvement to browse through the map/reduce tasks | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-173](https://issues.apache.org/jira/browse/HADOOP-173) | optimize allocation of tasks w/ local data | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-170](https://issues.apache.org/jira/browse/HADOOP-170) | setReplication and related bug fixes | Major | fs | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-167](https://issues.apache.org/jira/browse/HADOOP-167) | reducing the number of Configuration & JobConf objects created | Major | conf | Owen O'Malley | Owen O'Malley | +| [HADOOP-166](https://issues.apache.org/jira/browse/HADOOP-166) | IPC is unable to invoke methods that use interfaces as parameter | Minor | ipc | Stefan Groschupf | Doug Cutting | +| [HADOOP-150](https://issues.apache.org/jira/browse/HADOOP-150) | tip and task names should reflect the job name | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-144](https://issues.apache.org/jira/browse/HADOOP-144) | the dfs client id isn't relatable to the map/reduce task ids | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-142](https://issues.apache.org/jira/browse/HADOOP-142) | failed tasks should be rescheduled on different hosts after other jobs | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-138](https://issues.apache.org/jira/browse/HADOOP-138) | stop all tasks | Trivial | . | Stefan Groschupf | Doug Cutting | +| [HADOOP-131](https://issues.apache.org/jira/browse/HADOOP-131) | Separate start/stop-dfs.sh and start/stop-mapred.sh scripts | Minor | . | Chris A. Mattmann | Doug Cutting | +| [HADOOP-129](https://issues.apache.org/jira/browse/HADOOP-129) | FileSystem should not name files with java.io.File | Major | fs | Doug Cutting | Doug Cutting | +| [HADOOP-116](https://issues.apache.org/jira/browse/HADOOP-116) | cleaning up /tmp/hadoop/mapred/system | Major | . | raghavendra prabhu | Doug Cutting | +| [HADOOP-114](https://issues.apache.org/jira/browse/HADOOP-114) | Non-informative error message | Trivial | . | Rod Taylor | Doug Cutting | +| [HADOOP-96](https://issues.apache.org/jira/browse/HADOOP-96) | name server should log decisions that affect data: block creation, removal, replication | Critical | . | Yoram Arnon | Hairong Kuang | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-192](https://issues.apache.org/jira/browse/HADOOP-192) | Trivial JRE 1.5 versus 1.4 bug | Blocker | . | David Bowen | | +| [HADOOP-190](https://issues.apache.org/jira/browse/HADOOP-190) | Job fails though task succeeded if we fail to exit | Major | . | stack | | +| [HADOOP-188](https://issues.apache.org/jira/browse/HADOOP-188) | more unprotected RPC calls in JobClient.runJob allow loss of job due to timeout | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-186](https://issues.apache.org/jira/browse/HADOOP-186) | communication problems in the task tracker cause long latency | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-185](https://issues.apache.org/jira/browse/HADOOP-185) | tasks are lost during pollForNewTask | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-183](https://issues.apache.org/jira/browse/HADOOP-183) | adjust file replication factor when loading image and edits according to replication.min and replication.max | Minor | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-182](https://issues.apache.org/jira/browse/HADOOP-182) | lost task trackers should not update status of completed jobs | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-174](https://issues.apache.org/jira/browse/HADOOP-174) | jobclient kills job for one timeout | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-172](https://issues.apache.org/jira/browse/HADOOP-172) | rpc doesn't handle returning null for a String[] | Blocker | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-169](https://issues.apache.org/jira/browse/HADOOP-169) | a single failure from locateMapOutputs kills the entire job | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-168](https://issues.apache.org/jira/browse/HADOOP-168) | JobSubmissionProtocol and InterTrackerProtocol don't include "throws IOException" on all methods | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-162](https://issues.apache.org/jira/browse/HADOOP-162) | concurrent modification exception in FSNamesystem.Lease.releaseLocks | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-160](https://issues.apache.org/jira/browse/HADOOP-160) | sleeping with locks held | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-157](https://issues.apache.org/jira/browse/HADOOP-157) | job fails because pendingCreates is not cleaned up after a task fails | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-151](https://issues.apache.org/jira/browse/HADOOP-151) | RPC code has socket leak? | Major | ipc | p sutter | Doug Cutting | +| [HADOOP-143](https://issues.apache.org/jira/browse/HADOOP-143) | exception call stacks are word wrapped in webapp | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-139](https://issues.apache.org/jira/browse/HADOOP-139) | Deadlock in LocalFileSystem lock/release | Major | fs | Igor Bolotin | Doug Cutting | +| [HADOOP-137](https://issues.apache.org/jira/browse/HADOOP-137) | Different TaskTrackers may get the same task tracker id, thus cause many problems. | Critical | . | Runping Qi | Owen O'Malley | +| [HADOOP-134](https://issues.apache.org/jira/browse/HADOOP-134) | JobTracker trapped in a loop if it fails to localize a task | Major | . | Runping Qi | Owen O'Malley | +| [HADOOP-133](https://issues.apache.org/jira/browse/HADOOP-133) | the TaskTracker.Child.ping thread calls exit | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-128](https://issues.apache.org/jira/browse/HADOOP-128) | Failure to replicate dfs block kills client | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-126](https://issues.apache.org/jira/browse/HADOOP-126) | "hadoop dfs -cp" does not copy crc files | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-125](https://issues.apache.org/jira/browse/HADOOP-125) | LocalFileSystem.makeAbsolute bug on Windows | Minor | fs | p sutter | Doug Cutting | +| [HADOOP-118](https://issues.apache.org/jira/browse/HADOOP-118) | Namenode does not always clean up pendingCreates | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-117](https://issues.apache.org/jira/browse/HADOOP-117) | mapred temporary files not deleted | Blocker | . | raghavendra prabhu | Doug Cutting | +| [HADOOP-92](https://issues.apache.org/jira/browse/HADOOP-92) | Error Reporting/logging in MapReduce | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-69](https://issues.apache.org/jira/browse/HADOOP-69) | Unchecked lookup value causes NPE in FSNamesystemgetDatanodeHints | Major | . | Bryan Pendleton | | +| [HADOOP-68](https://issues.apache.org/jira/browse/HADOOP-68) | "Cannot abandon block during write to \" and "Cannot obtain additional block for file \" errors during dfs write test | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-63](https://issues.apache.org/jira/browse/HADOOP-63) | problem with webapp when start a jobtracker | Minor | . | Hairong Kuang | Hairong Kuang | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-194](https://issues.apache.org/jira/browse/HADOOP-194) | Distributed checkup of the file system consistency. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-193](https://issues.apache.org/jira/browse/HADOOP-193) | DFS i/o benchmark. | Major | fs | Konstantin Shvachko | | +| [HADOOP-187](https://issues.apache.org/jira/browse/HADOOP-187) | simple distributed dfs random data writer & sort example applications | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-184](https://issues.apache.org/jira/browse/HADOOP-184) | hadoop nightly build and regression test on a cluster | Minor | . | Mahadev konar | Mahadev konar | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/RELEASENOTES.0.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/RELEASENOTES.0.2.0.md new file mode 100644 index 00000000000..75182d19b40 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.0/RELEASENOTES.0.2.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.2.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/CHANGES.0.2.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/CHANGES.0.2.1.md new file mode 100644 index 00000000000..19e53c2c1f7 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/CHANGES.0.2.1.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.2.1 - 2006-05-12 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-207](https://issues.apache.org/jira/browse/HADOOP-207) | Patch to HADOOP-96 uses long deprecated call | Critical | . | Bryan Pendleton | Hairong Kuang | +| [HADOOP-201](https://issues.apache.org/jira/browse/HADOOP-201) | hadoop dfs -report throws exception | Minor | . | Johan Oskarsson | Doug Cutting | +| [HADOOP-199](https://issues.apache.org/jira/browse/HADOOP-199) | reduce copy progress not updating | Minor | . | Owen O'Malley | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/RELEASENOTES.0.2.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/RELEASENOTES.0.2.1.md new file mode 100644 index 00000000000..0e5f0337d3d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.2.1/RELEASENOTES.0.2.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.2.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md new file mode 100644 index 00000000000..2e03f9e95a5 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/CHANGES.0.20.0.md @@ -0,0 +1,347 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.0 - 2009-04-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5531](https://issues.apache.org/jira/browse/HADOOP-5531) | Remove Chukwa on branch-0.20 | Blocker | . | Nigel Daley | Nigel Daley | +| [HADOOP-4970](https://issues.apache.org/jira/browse/HADOOP-4970) | Use the full path when move files to .Trash/Current | Major | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4826](https://issues.apache.org/jira/browse/HADOOP-4826) | Admin command saveNamespace. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4789](https://issues.apache.org/jira/browse/HADOOP-4789) | Change fair scheduler to share between pools by default, not between invidual jobs | Minor | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-4783](https://issues.apache.org/jira/browse/HADOOP-4783) | History files are given world readable permissions. | Blocker | . | Hemanth Yamijala | Amareshwari Sriramadasu | +| [HADOOP-4631](https://issues.apache.org/jira/browse/HADOOP-4631) | Split the default configurations into 3 parts | Major | conf | Owen O'Malley | Sharad Agarwal | +| [HADOOP-4618](https://issues.apache.org/jira/browse/HADOOP-4618) | Move http server from FSNamesystem into NameNode. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4576](https://issues.apache.org/jira/browse/HADOOP-4576) | Modify pending tasks count in the UI to pending jobs count in the UI | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [HADOOP-4572](https://issues.apache.org/jira/browse/HADOOP-4572) | INode and its sub-classes should be package private | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4567](https://issues.apache.org/jira/browse/HADOOP-4567) | GetFileBlockLocations should return the NetworkTopology information of the machines that hosts those blocks | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4445](https://issues.apache.org/jira/browse/HADOOP-4445) | Wrong number of running map/reduce tasks are displayed in queue information. | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [HADOOP-4435](https://issues.apache.org/jira/browse/HADOOP-4435) | The JobTracker should display the amount of heap memory used | Minor | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4422](https://issues.apache.org/jira/browse/HADOOP-4422) | S3 file systems should not create bucket | Major | fs/s3 | David Phillips | David Phillips | +| [HADOOP-4253](https://issues.apache.org/jira/browse/HADOOP-4253) | Fix warnings generated by FindBugs | Major | conf, fs, record | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4210](https://issues.apache.org/jira/browse/HADOOP-4210) | Findbugs warnings are printed related to equals implementation of several classes | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4188](https://issues.apache.org/jira/browse/HADOOP-4188) | Remove Task's dependency on concrete file systems | Major | . | Tom White | Sharad Agarwal | +| [HADOOP-4103](https://issues.apache.org/jira/browse/HADOOP-4103) | Alert for missing blocks | Major | . | Christian Kunz | Raghu Angadi | +| [HADOOP-4035](https://issues.apache.org/jira/browse/HADOOP-4035) | Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory requirements and task trackers free memory | Blocker | . | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-4029](https://issues.apache.org/jira/browse/HADOOP-4029) | NameNode should report status and performance for each replica of image and log | Major | . | Robert Chansler | Boris Shkolnik | +| [HADOOP-3986](https://issues.apache.org/jira/browse/HADOOP-3986) | JobClient should not have a static configuration | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3923](https://issues.apache.org/jira/browse/HADOOP-3923) | Deprecate org.apache.hadoop.mapred.StatusHttpServer | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3750](https://issues.apache.org/jira/browse/HADOOP-3750) | Fix and enforce module dependencies | Major | . | Tom White | Sharad Agarwal | +| [HADOOP-3497](https://issues.apache.org/jira/browse/HADOOP-3497) | File globbing with a PathFilter is too restrictive | Major | fs | Tom White | Tom White | +| [HADOOP-3422](https://issues.apache.org/jira/browse/HADOOP-3422) | Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen | Major | metrics | Jason | stack | +| [HADOOP-3344](https://issues.apache.org/jira/browse/HADOOP-3344) | libhdfs: always builds 32bit, even when x86\_64 Java used | Major | build | Craig Macdonald | Giridharan Kesavan | +| [HADOOP-1650](https://issues.apache.org/jira/browse/HADOOP-1650) | Upgrade Jetty to 6.x | Major | . | Devaraj Das | Chris Douglas | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4937](https://issues.apache.org/jira/browse/HADOOP-4937) | [HOD] Include ringmaster RPC port information in the notes attribute | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4804](https://issues.apache.org/jira/browse/HADOOP-4804) | Create Forrest documentation for the fair scheduler | Minor | . | Matei Zaharia | Sreekanth Ramakrishnan | +| [HADOOP-4661](https://issues.apache.org/jira/browse/HADOOP-4661) | distch: a tool for distributed ch{mod,own} | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4622](https://issues.apache.org/jira/browse/HADOOP-4622) | Explicitly specify interpretor for non-native binaries | Minor | . | Fredrik Hedberg | Fredrik Hedberg | +| [HADOOP-4575](https://issues.apache.org/jira/browse/HADOOP-4575) | An independent HTTPS proxy for HDFS | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-4348](https://issues.apache.org/jira/browse/HADOOP-4348) | Adding service-level authorization to Hadoop | Major | security | Kan Zhang | Arun C Murthy | +| [HADOOP-4284](https://issues.apache.org/jira/browse/HADOOP-4284) | Support for user configurable global filters on HttpServer | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-4179](https://issues.apache.org/jira/browse/HADOOP-4179) | Hadoop-Vaidya : Rule based performance diagnostic tool for Map/Reduce jobs | Major | . | Suhas Gogate | Suhas Gogate | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6150](https://issues.apache.org/jira/browse/HADOOP-6150) | Need to be able to instantiate a comparator instance from a comparator string without creating a TFile.Reader object | Minor | io | Hong Tang | Hong Tang | +| [HADOOP-5521](https://issues.apache.org/jira/browse/HADOOP-5521) | Remove dependency of testcases on RESTART\_COUNT | Major | test | Amar Kamat | Ravi Gummadi | +| [HADOOP-5520](https://issues.apache.org/jira/browse/HADOOP-5520) | Typo in diskQuota help documentation | Trivial | documentation | Ravi Phulari | Ravi Phulari | +| [HADOOP-5519](https://issues.apache.org/jira/browse/HADOOP-5519) | Remove claims from mapred-default.xml that prime numbers of tasks are helpful. | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5514](https://issues.apache.org/jira/browse/HADOOP-5514) | Add waiting/failed tasks to JobTracker metrics | Major | metrics | Chris Douglas | Chris Douglas | +| [HADOOP-5468](https://issues.apache.org/jira/browse/HADOOP-5468) | Change Hadoop doc menu to sub-menus | Major | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-5466](https://issues.apache.org/jira/browse/HADOOP-5466) | CSS Style Changes for Hadoop Doc Headers and Code | Major | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-5395](https://issues.apache.org/jira/browse/HADOOP-5395) | When queue ACLs are enabled, the error message shown when a job is submitted to a non-existent queue is misleading | Major | . | Hemanth Yamijala | rahul k singh | +| [HADOOP-5379](https://issues.apache.org/jira/browse/HADOOP-5379) | Throw exception instead of writing to System.err when there is a CRC error on CBZip2InputStream | Minor | io | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-5275](https://issues.apache.org/jira/browse/HADOOP-5275) | ivy directory should be there in hadoop tar ball | Major | build | Suman Sehgal | Giridharan Kesavan | +| [HADOOP-5122](https://issues.apache.org/jira/browse/HADOOP-5122) | libhdfs test conf uses deprecated fs.default.name value | Trivial | test | Craig Macdonald | Craig Macdonald | +| [HADOOP-5030](https://issues.apache.org/jira/browse/HADOOP-5030) | Chukwa RPM build improvements | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4980](https://issues.apache.org/jira/browse/HADOOP-4980) | Cleanup the Capacity Scheduler code | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-4950](https://issues.apache.org/jira/browse/HADOOP-4950) | CompressorStream and BlockCompressorStream should be public | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-4944](https://issues.apache.org/jira/browse/HADOOP-4944) | Allow Xinclude in hadoop config file | Minor | conf | Rama, Ramasamy | Rama, Ramasamy | +| [HADOOP-4920](https://issues.apache.org/jira/browse/HADOOP-4920) | do not keep forrest output in subversion | Major | documentation | Doug Cutting | Doug Cutting | +| [HADOOP-4916](https://issues.apache.org/jira/browse/HADOOP-4916) | adjust build.xml to reference external properties when building the "init.d" for the rpm | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4909](https://issues.apache.org/jira/browse/HADOOP-4909) | Clean up map/reduce api to take JobContext in the static methods | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4880](https://issues.apache.org/jira/browse/HADOOP-4880) | Improvements to TestJobTrackerRestart | Major | test | Arun C Murthy | Amar Kamat | +| [HADOOP-4843](https://issues.apache.org/jira/browse/HADOOP-4843) | Collect Job History log file and Job Conf file into Chukwa | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4838](https://issues.apache.org/jira/browse/HADOOP-4838) | Cleanup Metrics and the MBeans | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-4837](https://issues.apache.org/jira/browse/HADOOP-4837) | Move HADOOP\_CONF\_DIR to chukwa-env.sh | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-4828](https://issues.apache.org/jira/browse/HADOOP-4828) | Update documentation for default configuration | Major | documentation | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4827](https://issues.apache.org/jira/browse/HADOOP-4827) | Improve data aggregation in database | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4758](https://issues.apache.org/jira/browse/HADOOP-4758) | Add a splitter for metrics contexts | Minor | metrics | Chris Douglas | Chris Douglas | +| [HADOOP-4753](https://issues.apache.org/jira/browse/HADOOP-4753) | gridmix2 code can be condensed | Minor | benchmarks | Chris Douglas | Chris Douglas | +| [HADOOP-4749](https://issues.apache.org/jira/browse/HADOOP-4749) | reducer should output input data size when shuffling is done | Major | . | Zheng Shao | He Yongqiang | +| [HADOOP-4747](https://issues.apache.org/jira/browse/HADOOP-4747) | Reuse FileStatus in FsShell where possible | Minor | fs | David Phillips | David Phillips | +| [HADOOP-4708](https://issues.apache.org/jira/browse/HADOOP-4708) | Add support for dfsadmin commands for test TestCLI unit test | Major | test | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4705](https://issues.apache.org/jira/browse/HADOOP-4705) | [HOD] Grant read permissions for files/directories created by hod as part of provisioning hadoop clusters | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4699](https://issues.apache.org/jira/browse/HADOOP-4699) | Change TaskTracker.MapOutputServlet to send only the IFile segment, validate checksum in Reduce | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-4690](https://issues.apache.org/jira/browse/HADOOP-4690) | fuse-dfs - create source file/function + utils + config + main source files | Minor | . | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-4668](https://issues.apache.org/jira/browse/HADOOP-4668) | Improve JavaDoc on JobConf.setCombinerClass to better document restrictions on combiners | Major | documentation | Owen O'Malley | Owen O'Malley | +| [HADOOP-4666](https://issues.apache.org/jira/browse/HADOOP-4666) | Launch reduces only after a few maps have run in the Fair Scheduler | Major | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-4664](https://issues.apache.org/jira/browse/HADOOP-4664) | Parallelize job initialization | Blocker | . | Matei Zaharia | Jothi Padmanabhan | +| [HADOOP-4649](https://issues.apache.org/jira/browse/HADOOP-4649) | Improve abstraction for spill indices | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-4640](https://issues.apache.org/jira/browse/HADOOP-4640) | Add ability to split text files compressed with lzo | Trivial | io | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-4628](https://issues.apache.org/jira/browse/HADOOP-4628) | Move Hive out of Core | Major | . | Owen O'Malley | | +| [HADOOP-4599](https://issues.apache.org/jira/browse/HADOOP-4599) | BlocksMap and BlockInfo should be package private. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4565](https://issues.apache.org/jira/browse/HADOOP-4565) | MultiFileInputSplit can use data locality information to create splits | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4545](https://issues.apache.org/jira/browse/HADOOP-4545) | Add an example of a secondary sort | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4523](https://issues.apache.org/jira/browse/HADOOP-4523) | Enhance how memory-intensive user tasks are handled | Major | . | Vivek Ratan | Vinod Kumar Vavilapalli | +| [HADOOP-4485](https://issues.apache.org/jira/browse/HADOOP-4485) | ant compile-native shorthand | Trivial | build | Enis Soztutar | Enis Soztutar | +| [HADOOP-4454](https://issues.apache.org/jira/browse/HADOOP-4454) | Support comments in 'slaves' file | Minor | conf, scripts | Rama, Ramasamy | Rama, Ramasamy | +| [HADOOP-4453](https://issues.apache.org/jira/browse/HADOOP-4453) | Improve ssl handling for distcp | Major | . | Chris Douglas | Kan Zhang | +| [HADOOP-4440](https://issues.apache.org/jira/browse/HADOOP-4440) | TestJobInProgressListener should also test for jobs killed in queued state | Minor | test | Amar Kamat | Amar Kamat | +| [HADOOP-4437](https://issues.apache.org/jira/browse/HADOOP-4437) | Use qMC sequence to improve the accuracy of PiEstimator | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4339](https://issues.apache.org/jira/browse/HADOOP-4339) | Improve FsShell -du/-dus and FileSystem.getContentSummary efficiency | Major | fs | David Phillips | David Phillips | +| [HADOOP-4305](https://issues.apache.org/jira/browse/HADOOP-4305) | repeatedly blacklisted tasktrackers should get declared dead | Major | . | Christian Kunz | Amareshwari Sriramadasu | +| [HADOOP-4276](https://issues.apache.org/jira/browse/HADOOP-4276) | The mapred.\*ID classes are inefficient for hashCode and serialization | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-4244](https://issues.apache.org/jira/browse/HADOOP-4244) | In "ant test-patch", runContribTestOnEclipseFiles should not be run on Hudson only | Major | build, test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4234](https://issues.apache.org/jira/browse/HADOOP-4234) | KFS: Allow KFS layer to interface with multiple KFS namenodes | Minor | fs | Sriram Rao | Sriram Rao | +| [HADOOP-4226](https://issues.apache.org/jira/browse/HADOOP-4226) | LineReader::readLine cleanup | Minor | . | Yuri Pradkin | Yuri Pradkin | +| [HADOOP-4178](https://issues.apache.org/jira/browse/HADOOP-4178) | The capacity scheduler's defaults for queues should be configurable. | Blocker | . | Owen O'Malley | Sreekanth Ramakrishnan | +| [HADOOP-4126](https://issues.apache.org/jira/browse/HADOOP-4126) | Allow access to HDFS web UI on EC2 | Major | contrib/cloud | Tom White | Tom White | +| [HADOOP-3770](https://issues.apache.org/jira/browse/HADOOP-3770) | improve composition, submission and result collection of gridmix | Major | benchmarks | Lingyun Yang | Runping Qi | +| [HADOOP-3461](https://issues.apache.org/jira/browse/HADOOP-3461) | Remove dfs.StringBytesWritbale | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-3180](https://issues.apache.org/jira/browse/HADOOP-3180) | add name of faulty class to WritableName.getClass IOException upon ClassNotFoundException | Trivial | io | Pete Wyckoff | Pete Wyckoff | +| [HADOOP-3136](https://issues.apache.org/jira/browse/HADOOP-3136) | Assign multiple tasks per TaskTracker heartbeat | Major | . | Devaraj Das | Arun C Murthy | +| [HADOOP-3063](https://issues.apache.org/jira/browse/HADOOP-3063) | BloomMapFile - fail-fast version of MapFile for sparsely populated key space | Major | io | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-2774](https://issues.apache.org/jira/browse/HADOOP-2774) | Add counters to show number of key/values that have been sorted and merged in the maps and reduces | Major | . | Owen O'Malley | Ravi Gummadi | +| [HADOOP-1230](https://issues.apache.org/jira/browse/HADOOP-1230) | Replace parameters with context objects in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-466](https://issues.apache.org/jira/browse/HADOOP-466) | Startup scripts will not start instances of Hadoop daemons w/different configs w/o setting separate PID directories | Major | conf | Vetle Roeim | | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5645](https://issues.apache.org/jira/browse/HADOOP-5645) | After HADOOP-4920 we need a place to checkin releasenotes.html | Major | build | Nigel Daley | Nigel Daley | +| [HADOOP-5644](https://issues.apache.org/jira/browse/HADOOP-5644) | Namnode is stuck in safe mode | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5607](https://issues.apache.org/jira/browse/HADOOP-5607) | TestCapacityScheduler fails with NPE | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-5605](https://issues.apache.org/jira/browse/HADOOP-5605) | All the replicas incorrectly got marked as corrupt. | Blocker | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-5588](https://issues.apache.org/jira/browse/HADOOP-5588) | hadoop commands seem extremely slow in 0.20 branch | Blocker | fs | Koji Noguchi | Hairong Kuang | +| [HADOOP-5585](https://issues.apache.org/jira/browse/HADOOP-5585) | FileSystem statistic counters are too high when JVM reuse is enabled. | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5577](https://issues.apache.org/jira/browse/HADOOP-5577) | The Job.waitForCompletion doesn't print status as the job runs | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5576](https://issues.apache.org/jira/browse/HADOOP-5576) | LocalJobRunner does not run jobs using new MapReduce API | Blocker | . | Tom White | Tom White | +| [HADOOP-5571](https://issues.apache.org/jira/browse/HADOOP-5571) | TupleWritable can return incorrect results if it contains more than 32 values | Major | . | Jingkei Ly | Jingkei Ly | +| [HADOOP-5565](https://issues.apache.org/jira/browse/HADOOP-5565) | The job instrumentation API needs to have a method for finalizeJob, | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5564](https://issues.apache.org/jira/browse/HADOOP-5564) | hadoop command uses large JVM heap size | Blocker | build | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5557](https://issues.apache.org/jira/browse/HADOOP-5557) | Two minor problems in TestOverReplicatedBlocks | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5554](https://issues.apache.org/jira/browse/HADOOP-5554) | DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-5551](https://issues.apache.org/jira/browse/HADOOP-5551) | Namenode permits directory destruction on overwrite | Critical | . | Brian Bockelman | Brian Bockelman | +| [HADOOP-5549](https://issues.apache.org/jira/browse/HADOOP-5549) | ReplicationMonitor should schedule both replication and deletion work in one iteration | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5548](https://issues.apache.org/jira/browse/HADOOP-5548) | Observed negative running maps on the job tracker | Blocker | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-5534](https://issues.apache.org/jira/browse/HADOOP-5534) | Deadlock triggered by FairScheduler scheduler's servlet due to changes from HADOOP-5214. | Blocker | . | Vinod Kumar Vavilapalli | rahul k singh | +| [HADOOP-5522](https://issues.apache.org/jira/browse/HADOOP-5522) | Document job setup/cleaup tasks and task cleanup tasks in mapred tutorial | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5516](https://issues.apache.org/jira/browse/HADOOP-5516) | TaskMemoryManagerThread crashes in a corner case | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-5505](https://issues.apache.org/jira/browse/HADOOP-5505) | TestMissingBlocksAlert fails on 0.20. | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5499](https://issues.apache.org/jira/browse/HADOOP-5499) | JobTracker metrics do not match job counters | Major | metrics | Chris Douglas | | +| [HADOOP-5493](https://issues.apache.org/jira/browse/HADOOP-5493) | Shuffle copiers do not return Codecs back to the pool at the end of shuffling | Blocker | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-5490](https://issues.apache.org/jira/browse/HADOOP-5490) | TestParallelInitialization failed on NoSuchElementException | Blocker | . | Hairong Kuang | Jothi Padmanabhan | +| [HADOOP-5484](https://issues.apache.org/jira/browse/HADOOP-5484) | TestRecoveryManager fails wtih FileAlreadyExistsException | Major | test | Hairong Kuang | Amar Kamat | +| [HADOOP-5483](https://issues.apache.org/jira/browse/HADOOP-5483) | Directory/file cleanup thread throws IllegalStateException | Major | . | Hairong Kuang | | +| [HADOOP-5479](https://issues.apache.org/jira/browse/HADOOP-5479) | NameNode should not send empty block replication request to DataNode | Critical | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5473](https://issues.apache.org/jira/browse/HADOOP-5473) | Race condition in command-line kill for a task | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5471](https://issues.apache.org/jira/browse/HADOOP-5471) | SyncLogs thread in Child.java would update wrong file for a cleanup attempt, in some cases. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5465](https://issues.apache.org/jira/browse/HADOOP-5465) | Blocks remain under-replicated | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5463](https://issues.apache.org/jira/browse/HADOOP-5463) | Balancer throws "Not a host:port pair" unless port is specified in fs.default.name | Minor | . | Stuart White | Stuart White | +| [HADOOP-5459](https://issues.apache.org/jira/browse/HADOOP-5459) | CRC errors not detected reading intermediate output into memory with problematic length | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5440](https://issues.apache.org/jira/browse/HADOOP-5440) | Successful taskid are not removed from TaskMemoryManager | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5437](https://issues.apache.org/jira/browse/HADOOP-5437) | Unit test for jvm -reuse | Blocker | . | Amareshwari Sriramadasu | Owen O'Malley | +| [HADOOP-5432](https://issues.apache.org/jira/browse/HADOOP-5432) | TestHdfsProxy fails on 0.20 | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5417](https://issues.apache.org/jira/browse/HADOOP-5417) | IPC client drops interrupted exceptions | Blocker | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-5416](https://issues.apache.org/jira/browse/HADOOP-5416) | Wrong description of " hadoop fs -test " in FS Shell guide . | Trivial | documentation | Ravi Phulari | Ravi Phulari | +| [HADOOP-5414](https://issues.apache.org/jira/browse/HADOOP-5414) | IO exception while executing hadoop fs -touchz fileName | Major | fs | Ravi Phulari | Hairong Kuang | +| [HADOOP-5394](https://issues.apache.org/jira/browse/HADOOP-5394) | JobTracker might schedule 2 attempts of the same task with the same attempt id across restarts | Critical | . | Amar Kamat | Amar Kamat | +| [HADOOP-5390](https://issues.apache.org/jira/browse/HADOOP-5390) | hdfsproxy includes duplicate jars in tarball, source in binary tarball | Major | build | Chris Douglas | zhiyong zhang | +| [HADOOP-5384](https://issues.apache.org/jira/browse/HADOOP-5384) | DataNodeCluster should not create blocks with generationStamp == 1 | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5382](https://issues.apache.org/jira/browse/HADOOP-5382) | The new map/reduce api doesn't support combiners | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-5376](https://issues.apache.org/jira/browse/HADOOP-5376) | JobInProgress.obtainTaskCleanupTask() throws an ArrayIndexOutOfBoundsException | Blocker | . | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [HADOOP-5338](https://issues.apache.org/jira/browse/HADOOP-5338) | Reduce tasks are stuck waiting for map outputs when none are in progress | Blocker | . | Ramya Sunil | Amar Kamat | +| [HADOOP-5337](https://issues.apache.org/jira/browse/HADOOP-5337) | JobTracker greedily schedules tasks without running tasks to join | Major | . | Karam Singh | Amar Kamat | +| [HADOOP-5328](https://issues.apache.org/jira/browse/HADOOP-5328) | Renaming of Job histroy file is incorrect if Jobtracker is restarted multimple times | Major | . | Karam Singh | Amar Kamat | +| [HADOOP-5327](https://issues.apache.org/jira/browse/HADOOP-5327) | Job files for a job failing because of ACLs are not clean from the system directory | Blocker | . | Karam Singh | Amar Kamat | +| [HADOOP-5326](https://issues.apache.org/jira/browse/HADOOP-5326) | bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs | Major | io | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-5320](https://issues.apache.org/jira/browse/HADOOP-5320) | TestMapReduceLocal is missing a close() that is causing it to fail while running the test on NFS | Minor | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5306](https://issues.apache.org/jira/browse/HADOOP-5306) | Job History file can have empty string as http port after JobTracker Restart in case of lost TT, which can result in NumberFormatException when JT is restarted 2nd time | Blocker | . | Ravi Gummadi | Amar Kamat | +| [HADOOP-5305](https://issues.apache.org/jira/browse/HADOOP-5305) | Unit test org.apache.hadoop.fs.TestCopyFiles.testMapCount fails on trunk | Minor | test | Ramya Sunil | Tsz Wo Nicholas Sze | +| [HADOOP-5298](https://issues.apache.org/jira/browse/HADOOP-5298) | Unit test fails out on trunk org.apache.hadoop.http.TestServletFilter.testServletFilter | Major | test | Lee Tucker | Tsz Wo Nicholas Sze | +| [HADOOP-5292](https://issues.apache.org/jira/browse/HADOOP-5292) | KFS: calling getFileBlockLocations() on 0-length file causes a NPE | Major | . | Sriram Rao | Sriram Rao | +| [HADOOP-5285](https://issues.apache.org/jira/browse/HADOOP-5285) | JobTracker hangs for long periods of time | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das | +| [HADOOP-5282](https://issues.apache.org/jira/browse/HADOOP-5282) | Running tasks on lost trackers are incorrectly logging the attempt and task failures | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-5281](https://issues.apache.org/jira/browse/HADOOP-5281) | GzipCodec fails second time it is used in a process | Blocker | . | Ben Maurer | Chris Douglas | +| [HADOOP-5280](https://issues.apache.org/jira/browse/HADOOP-5280) | When expiring a lost launched task, JT doesn't remove the attempt from the taskidToTIPMap. | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das | +| [HADOOP-5274](https://issues.apache.org/jira/browse/HADOOP-5274) | gridmix2 is not getting compiled to generate gridmix.jar | Critical | benchmarks | Suman Sehgal | Chris Douglas | +| [HADOOP-5272](https://issues.apache.org/jira/browse/HADOOP-5272) | JobTracker does not log TIP start information after restart | Critical | . | Amar Kamat | Amar Kamat | +| [HADOOP-5269](https://issues.apache.org/jira/browse/HADOOP-5269) | TaskTracker.runningTasks holding FAILED\_UNCLEAN and KILLED\_UNCLEAN taskStatuses forever in some cases. | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5259](https://issues.apache.org/jira/browse/HADOOP-5259) | Job with output hdfs:/user/\/outputpath (no authority) fails with Wrong FS | Major | fs | Koji Noguchi | Doug Cutting | +| [HADOOP-5255](https://issues.apache.org/jira/browse/HADOOP-5255) | Fix for HADOOP-5079 HashFunction inadvertently destroys some randomness | Minor | io | stack | Jonathan Ellis | +| [HADOOP-5248](https://issues.apache.org/jira/browse/HADOOP-5248) | Job directories could remain undeleted in some scenarios after job completes. | Blocker | . | Hemanth Yamijala | Devaraj Das | +| [HADOOP-5247](https://issues.apache.org/jira/browse/HADOOP-5247) | NPEs in JobTracker and JobClient when mapred.jobtracker.completeuserjobs.maximum is set to zero. | Blocker | . | Vinod Kumar Vavilapalli | Amar Kamat | +| [HADOOP-5241](https://issues.apache.org/jira/browse/HADOOP-5241) | Reduce tasks get stuck because of over-estimated task size (regression from 0.18) | Blocker | . | Andy Pavlo | Sharad Agarwal | +| [HADOOP-5233](https://issues.apache.org/jira/browse/HADOOP-5233) | Reducer not Succeded after 100% | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5227](https://issues.apache.org/jira/browse/HADOOP-5227) | distcp -delete option deletes all files from the destination directory | Major | . | Suhas Gogate | Tsz Wo Nicholas Sze | +| [HADOOP-5214](https://issues.apache.org/jira/browse/HADOOP-5214) | ConcurrentModificationException in FairScheduler.getTotalSlots | Blocker | . | Vinod Kumar Vavilapalli | rahul k singh | +| [HADOOP-5211](https://issues.apache.org/jira/browse/HADOOP-5211) | TestSetupAndCleanupFailure fails with timeout | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-5154](https://issues.apache.org/jira/browse/HADOOP-5154) | 4-way deadlock in FairShare scheduler | Blocker | . | Vinod Kumar Vavilapalli | Matei Zaharia | +| [HADOOP-5146](https://issues.apache.org/jira/browse/HADOOP-5146) | LocalDirAllocator misses files on the local filesystem | Blocker | . | Arun C Murthy | Devaraj Das | +| [HADOOP-5145](https://issues.apache.org/jira/browse/HADOOP-5145) | Balancer sometimes runs out of memory after days or weeks running | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5142](https://issues.apache.org/jira/browse/HADOOP-5142) | MapWritable#putAll does not store classes | Minor | io | Doğacan GĂŒney | | +| [HADOOP-5139](https://issues.apache.org/jira/browse/HADOOP-5139) | RPC call throws IllegalArgumentException complaining duplicate metrics registration | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-5111](https://issues.apache.org/jira/browse/HADOOP-5111) | Generic mapreduce classes cannot be used with Job::set\* methods | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5085](https://issues.apache.org/jira/browse/HADOOP-5085) | Copying a file to local with Crc throws an exception | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-5079](https://issues.apache.org/jira/browse/HADOOP-5079) | HashFunction inadvertently destroys some randomness | Major | util | Jonathan Ellis | Jonathan Ellis | +| [HADOOP-5075](https://issues.apache.org/jira/browse/HADOOP-5075) | Potential infinite loop in updateMinSlots | Blocker | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-5068](https://issues.apache.org/jira/browse/HADOOP-5068) | testClusterBlockingForLackOfMemory in TestCapacityScheduler fails randomly | Major | . | Sreekanth Ramakrishnan | Vinod Kumar Vavilapalli | +| [HADOOP-5066](https://issues.apache.org/jira/browse/HADOOP-5066) | ant binary should not compile docs | Major | build | Amareshwari Sriramadasu | Giridharan Kesavan | +| [HADOOP-5065](https://issues.apache.org/jira/browse/HADOOP-5065) | setOutputFormatClass in mapreduce.Job fails for SequenceFileOutputFormat | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-5048](https://issues.apache.org/jira/browse/HADOOP-5048) | Sometimes job is still displayed in jobqueue\_details page for long time after job was killed. | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [HADOOP-5026](https://issues.apache.org/jira/browse/HADOOP-5026) | Startup scripts should be svn-executable | Minor | . | Andy Konwinski | Andy Konwinski | +| [HADOOP-4999](https://issues.apache.org/jira/browse/HADOOP-4999) | IndexOutOfBoundsException in FSEditLog | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4993](https://issues.apache.org/jira/browse/HADOOP-4993) | chukwa agent startup should be more modular | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4988](https://issues.apache.org/jira/browse/HADOOP-4988) | An earlier fix, for HADOOP-4373, results in a problem with reclaiming capacity when one or more queues have a capacity equal to zero | Blocker | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-4979](https://issues.apache.org/jira/browse/HADOOP-4979) | Capacity Scheduler does not always return no task to a TT if a job's memry requirements are not met | Major | . | Vivek Ratan | Vivek Ratan | +| [HADOOP-4977](https://issues.apache.org/jira/browse/HADOOP-4977) | Deadlock between reclaimCapacity and assignTasks | Blocker | . | Matei Zaharia | Vivek Ratan | +| [HADOOP-4963](https://issues.apache.org/jira/browse/HADOOP-4963) | Logs saying org.apache.hadoop.util.DiskChecker$DiskErrorException in TaskTracker are not relevant | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4956](https://issues.apache.org/jira/browse/HADOOP-4956) | NPE when jobdetails.jsp | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-4949](https://issues.apache.org/jira/browse/HADOOP-4949) | Native compilation is broken | Blocker | . | Chris Douglas | Chris Douglas | +| [HADOOP-4947](https://issues.apache.org/jira/browse/HADOOP-4947) | use regex to parse chukwa commands | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4925](https://issues.apache.org/jira/browse/HADOOP-4925) | Chukwa HTTP connection policies are too hard to configure | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4914](https://issues.apache.org/jira/browse/HADOOP-4914) | Support chkconfig for chukwa start up scripts | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4896](https://issues.apache.org/jira/browse/HADOOP-4896) | hdfs fsck does not load hdfs configuration. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-4894](https://issues.apache.org/jira/browse/HADOOP-4894) | fix Chukwa's jettyCollector.sh | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4892](https://issues.apache.org/jira/browse/HADOOP-4892) | File descriptor leak in Chukwa's ExecPlugin | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4889](https://issues.apache.org/jira/browse/HADOOP-4889) | Chukwa RPM needs to chown user/group in build stage instead of post install stage. | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4884](https://issues.apache.org/jira/browse/HADOOP-4884) | Change Date format pattern for Time Series graph | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4879](https://issues.apache.org/jira/browse/HADOOP-4879) | TestJobTrackerRestart fails on trunk | Blocker | test | Arun C Murthy | Owen O'Malley | +| [HADOOP-4878](https://issues.apache.org/jira/browse/HADOOP-4878) | After introduction of ivy ant test-patch always returns -1 score | Blocker | build | Sreekanth Ramakrishnan | Giridharan Kesavan | +| [HADOOP-4876](https://issues.apache.org/jira/browse/HADOOP-4876) | Capacity reclaimation for queue would not work as intended | Blocker | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-4874](https://issues.apache.org/jira/browse/HADOOP-4874) | Remove bindings to lzo | Blocker | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-4873](https://issues.apache.org/jira/browse/HADOOP-4873) | display minMaps/Reduces on advanced scheduler page | Major | . | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-4869](https://issues.apache.org/jira/browse/HADOOP-4869) | Lost Trackers may not be able to join back | Blocker | . | Devaraj Das | Amareshwari Sriramadasu | +| [HADOOP-4860](https://issues.apache.org/jira/browse/HADOOP-4860) | File Tailing Adaptor test cases causes random failure | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4855](https://issues.apache.org/jira/browse/HADOOP-4855) | Fix help message in MRAdmin | Minor | fs, security | Arun C Murthy | Arun C Murthy | +| [HADOOP-4854](https://issues.apache.org/jira/browse/HADOOP-4854) | Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file | Minor | . | Vivek Ratan | Sreekanth Ramakrishnan | +| [HADOOP-4849](https://issues.apache.org/jira/browse/HADOOP-4849) | Document service level authorization - HADOOP-4348 | Major | documentation, scripts | Arun C Murthy | Arun C Murthy | +| [HADOOP-4845](https://issues.apache.org/jira/browse/HADOOP-4845) | Shuffle counter issues | Major | . | Chris Douglas | He Yongqiang | +| [HADOOP-4844](https://issues.apache.org/jira/browse/HADOOP-4844) | Document deprecation of o.a.h.fs.permission.AccessControlException better | Minor | documentation, fs | Arun C Murthy | Arun C Murthy | +| [HADOOP-4830](https://issues.apache.org/jira/browse/HADOOP-4830) | Have end to end tests based on MiniMRCluster to verify that queue capacities are honoured. | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-4825](https://issues.apache.org/jira/browse/HADOOP-4825) | Chukwa - change jps to ps for process status detection | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4820](https://issues.apache.org/jira/browse/HADOOP-4820) | Remove unused method FSNamesystem.deleteInSafeMode | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4818](https://issues.apache.org/jira/browse/HADOOP-4818) | Enable JobTracker Instrumentation API with user job configuration | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4811](https://issues.apache.org/jira/browse/HADOOP-4811) | Import style sheet images for Chukwa | Trivial | . | Eric Yang | Eric Yang | +| [HADOOP-4805](https://issues.apache.org/jira/browse/HADOOP-4805) | Remove black list feature from Chukwa Agent to Chukwa Collector communication | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4796](https://issues.apache.org/jira/browse/HADOOP-4796) | Test target for chukwa build.xml needs to comply to hadoop build.xml test suites | Major | build | Eric Yang | Eric Yang | +| [HADOOP-4792](https://issues.apache.org/jira/browse/HADOOP-4792) | Chukwa build process generates files not tracked by svn | Major | build | Chris Douglas | Chris Douglas | +| [HADOOP-4791](https://issues.apache.org/jira/browse/HADOOP-4791) | Improve rpm build process for Chukwa | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4787](https://issues.apache.org/jira/browse/HADOOP-4787) | TestTrackerBlacklistAcrossJobs fails randomly | Blocker | test | Hemanth Yamijala | Amareshwari Sriramadasu | +| [HADOOP-4786](https://issues.apache.org/jira/browse/HADOOP-4786) | TestTrackerBlacklistAcrossJobs compilation is broken | Blocker | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-4785](https://issues.apache.org/jira/browse/HADOOP-4785) | avoid two calls to System.currentTimeMillis() in heartbeat | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4782](https://issues.apache.org/jira/browse/HADOOP-4782) | [HOD] HOD opens up directory permissions more than required | Blocker | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4774](https://issues.apache.org/jira/browse/HADOOP-4774) | CapacityScheduler should work even if capacity-scheduler.xml is not in the classpath i.e with the code defaults | Major | . | Amar Kamat | Sreekanth Ramakrishnan | +| [HADOOP-4771](https://issues.apache.org/jira/browse/HADOOP-4771) | FSImage saveFSImage() will have problem. | Major | . | Ruyue Ma | Ruyue Ma | +| [HADOOP-4770](https://issues.apache.org/jira/browse/HADOOP-4770) | gridmix2 run script doesn't work on trunk | Trivial | benchmarks | Chris Douglas | Chris Douglas | +| [HADOOP-4744](https://issues.apache.org/jira/browse/HADOOP-4744) | Wrong resolution of hostname and port | Blocker | . | Amareshwari Sriramadasu | Jothi Padmanabhan | +| [HADOOP-4738](https://issues.apache.org/jira/browse/HADOOP-4738) | saveVersion.sh could write a package-info.java that cannot be compiled when used with git | Trivial | build | Hemanth Yamijala | Chris Douglas | +| [HADOOP-4737](https://issues.apache.org/jira/browse/HADOOP-4737) | Job end notification should give status as "KILLED" when job gets killed. | Major | . | Ruchir Shah | Amareshwari Sriramadasu | +| [HADOOP-4732](https://issues.apache.org/jira/browse/HADOOP-4732) | connectionTimeout and readTimeout are passed in wrong order in ReduceTask.java | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4716](https://issues.apache.org/jira/browse/HADOOP-4716) | testRestartWithLostTracker frequently times out | Blocker | . | Johan Oskarsson | Amar Kamat | +| [HADOOP-4706](https://issues.apache.org/jira/browse/HADOOP-4706) | IFileOutputStream.close should call close of the underlying stream | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-4704](https://issues.apache.org/jira/browse/HADOOP-4704) | javadoc: "the the" =\> "the" | Trivial | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4698](https://issues.apache.org/jira/browse/HADOOP-4698) | TestMapRed fails with 64bit JDK | Major | test | Nigel Daley | Nigel Daley | +| [HADOOP-4695](https://issues.apache.org/jira/browse/HADOOP-4695) | TestGlobalFilter.testServletFilter fails | Blocker | test | Tsz Wo Nicholas Sze | Kan Zhang | +| [HADOOP-4692](https://issues.apache.org/jira/browse/HADOOP-4692) | Namenode in infinite loop for replicating/deleting corrupted block | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-4691](https://issues.apache.org/jira/browse/HADOOP-4691) | In the javadoc of IndexedSortable.compare(...), the link is wrong. | Minor | documentation, util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4683](https://issues.apache.org/jira/browse/HADOOP-4683) | Move the call to getMapCompletionEvents in ReduceTask.ReduceCopier.fetchOutputs to a separate thread | Major | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-4677](https://issues.apache.org/jira/browse/HADOOP-4677) | FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect. | Major | fs | Hong Tang | Hong Tang | +| [HADOOP-4676](https://issues.apache.org/jira/browse/HADOOP-4676) | Go back to Job has a wrong URL on blacklisted trackers page | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4671](https://issues.apache.org/jira/browse/HADOOP-4671) | Make some "flag" booleans volatile because they are set/read in different threads | Minor | . | Owen O'Malley | Chris Douglas | +| [HADOOP-4669](https://issues.apache.org/jira/browse/HADOOP-4669) | A file-open sometimes may not set the access time of a file | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-4650](https://issues.apache.org/jira/browse/HADOOP-4650) | local.cache.size is set to 10 GB, while DEFAULT\_CACHE\_SIZE is set to 1 MB | Trivial | filecache | Jeff Hammerbacher | Jeff Hammerbacher | +| [HADOOP-4645](https://issues.apache.org/jira/browse/HADOOP-4645) | hdfs proxy doesn't package itself right. | Major | . | Owen O'Malley | Kan Zhang | +| [HADOOP-4638](https://issues.apache.org/jira/browse/HADOOP-4638) | Exception thrown in/from RecoveryManager.recover() should be caught and handled | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-4623](https://issues.apache.org/jira/browse/HADOOP-4623) | Running tasks are not maintained by JobInProgress if speculation is off | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-4621](https://issues.apache.org/jira/browse/HADOOP-4621) | javadoc: warning - Multiple sources of package comments found for some packages | Major | documentation | Tsz Wo Nicholas Sze | Kan Zhang | +| [HADOOP-4608](https://issues.apache.org/jira/browse/HADOOP-4608) | Examples -Driver does not check first argument. | Major | . | Edward J. Yoon | Edward J. Yoon | +| [HADOOP-4606](https://issues.apache.org/jira/browse/HADOOP-4606) | There is a cygpath error if log directory does not exist | Minor | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4598](https://issues.apache.org/jira/browse/HADOOP-4598) | '-setrep' command skips under-replicated blocks | Minor | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-4589](https://issues.apache.org/jira/browse/HADOOP-4589) | Fix the PiEstimator output messages and code comments | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4587](https://issues.apache.org/jira/browse/HADOOP-4587) | typo in javadoc for map.input.file | Trivial | documentation | Koji Noguchi | Koji Noguchi | +| [HADOOP-4583](https://issues.apache.org/jira/browse/HADOOP-4583) | Code optimization/cleanup in HDFS | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4579](https://issues.apache.org/jira/browse/HADOOP-4579) | StorageDirectory is not removed from the storage directories list when writing to an edits fails | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4571](https://issues.apache.org/jira/browse/HADOOP-4571) | chukwa conf files are not cleaned up | Minor | . | Tsz Wo Nicholas Sze | Eric Yang | +| [HADOOP-4558](https://issues.apache.org/jira/browse/HADOOP-4558) | Scheduler fails to reclaim capacity if Jobs are submitted to queue one after the other | Major | . | Karam Singh | Amar Kamat | +| [HADOOP-4530](https://issues.apache.org/jira/browse/HADOOP-4530) | In fsck, HttpServletResponse sendError fails with IllegalStateException | Major | . | Koji Noguchi | Hairong Kuang | +| [HADOOP-4513](https://issues.apache.org/jira/browse/HADOOP-4513) | Capacity scheduler should initialize tasks asynchronously | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [HADOOP-4429](https://issues.apache.org/jira/browse/HADOOP-4429) | Misconfigured UNIX Groups Break Hadoop | Minor | . | Alex Loddengaard | Alex Loddengaard | +| [HADOOP-4408](https://issues.apache.org/jira/browse/HADOOP-4408) | FsAction operations shouldn't create new Objects | Minor | fs | Chris Douglas | Chris Douglas | +| [HADOOP-4388](https://issues.apache.org/jira/browse/HADOOP-4388) | Bug in Datanode transferBlocks code | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4377](https://issues.apache.org/jira/browse/HADOOP-4377) | Race condition creating S3 buffer directory for NativeS3FileSystem | Major | fs/s3 | David Phillips | David Phillips | +| [HADOOP-4374](https://issues.apache.org/jira/browse/HADOOP-4374) | JVM should not be killed but given an opportunity to exit gracefully | Major | . | Devaraj Das | Ravi Gummadi | +| [HADOOP-4346](https://issues.apache.org/jira/browse/HADOOP-4346) | Hadoop triggers a "soft" fd leak. | Major | io | Raghu Angadi | Raghu Angadi | +| [HADOOP-4325](https://issues.apache.org/jira/browse/HADOOP-4325) | Hadoop SocketInputStream.read() should return -1 in case of EOF. | Minor | io | Raghu Angadi | Raghu Angadi | +| [HADOOP-4262](https://issues.apache.org/jira/browse/HADOOP-4262) | message generated when the client exception has a null message is not useful | Minor | ipc | Steve Loughran | | +| [HADOOP-4245](https://issues.apache.org/jira/browse/HADOOP-4245) | KFS: Update the kfs jar file | Major | fs | Sriram Rao | Sriram Rao | +| [HADOOP-4238](https://issues.apache.org/jira/browse/HADOOP-4238) | [mapred] Unavailable schedulingInfo in "hadoop list [all]" output should be marked "N/A" | Major | . | Vinod Kumar Vavilapalli | Sreekanth Ramakrishnan | +| [HADOOP-4207](https://issues.apache.org/jira/browse/HADOOP-4207) | update derby.jar to eliminate the one time wait of 2-3 minutes while running junit tests | Minor | . | Prasad Chakka | Prasad Chakka | +| [HADOOP-4204](https://issues.apache.org/jira/browse/HADOOP-4204) | Fix warnings generated by FindBugs | Blocker | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-4141](https://issues.apache.org/jira/browse/HADOOP-4141) | ScriptBasedMapping has infinite loop with invalid jobconf parameter | Minor | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-3921](https://issues.apache.org/jira/browse/HADOOP-3921) | Clover breaks nightly builds | Major | build | Tom White | Tom White | +| [HADOOP-3842](https://issues.apache.org/jira/browse/HADOOP-3842) | There is a window where the JobTracker is in the RUNNING state (i.e ready to accept jobs) and never executes them. | Major | . | Amar Kamat | Hemanth Yamijala | +| [HADOOP-3810](https://issues.apache.org/jira/browse/HADOOP-3810) | NameNode seems unstable on a cluster with little space left | Major | . | Raghu Angadi | Hairong Kuang | +| [HADOOP-3293](https://issues.apache.org/jira/browse/HADOOP-3293) | When an input split spans cross block boundary, the split location should be the host having most of bytes on it. | Major | . | Runping Qi | Jothi Padmanabhan | +| [HADOOP-2771](https://issues.apache.org/jira/browse/HADOOP-2771) | changing the number of reduces dramatically changes the time of the map time | Major | . | Owen O'Malley | | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4857](https://issues.apache.org/jira/browse/HADOOP-4857) | TestUlimit is failing after Hadoop-4620 | Blocker | test | Sharad Agarwal | Ravi Gummadi | +| [HADOOP-4728](https://issues.apache.org/jira/browse/HADOOP-4728) | Tests for NameNode -checkpoint option with different configurations | Minor | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4722](https://issues.apache.org/jira/browse/HADOOP-4722) | adding tests for quotas command line error messages | Minor | test | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4688](https://issues.apache.org/jira/browse/HADOOP-4688) | multiple spills/reducers test case for mergeParts() needed | Major | . | Yuri Pradkin | Chris Douglas | +| [HADOOP-4505](https://issues.apache.org/jira/browse/HADOOP-4505) | Add a unit test to test faulty setup task and cleanup task killing the job | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-4458](https://issues.apache.org/jira/browse/HADOOP-4458) | Add a unit test for applications creating symlinks in wokring directory | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-3192](https://issues.apache.org/jira/browse/HADOOP-3192) | Test behavior of very large (32GB) name nodes | Major | . | Robert Chansler | | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-4612](https://issues.apache.org/jira/browse/HADOOP-4612) | Remove RunJar's dependency on JobClient | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4187](https://issues.apache.org/jira/browse/HADOOP-4187) | Create a MapReduce-specific ReflectionUtils that handles JobConf and JobConfigurable | Major | . | Tom White | Sharad Agarwal | +| [HADOOP-4185](https://issues.apache.org/jira/browse/HADOOP-4185) | Add setVerifyChecksum() method to FileSystem | Major | fs | Tom White | Sharad Agarwal | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md new file mode 100644 index 00000000000..339d7abbd25 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.0/RELEASENOTES.0.20.0.md @@ -0,0 +1,347 @@ + + +# Apache Hadoop 0.20.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5565](https://issues.apache.org/jira/browse/HADOOP-5565) | *Major* | **The job instrumentation API needs to have a method for finalizeJob,** + +Add finalizeJob & terminateJob methods to JobTrackerInstrumentation class + + +--- + +* [HADOOP-5548](https://issues.apache.org/jira/browse/HADOOP-5548) | *Blocker* | **Observed negative running maps on the job tracker** + +Adds synchronization for JobTracker methods in RecoveryManager. + + +--- + +* [HADOOP-5531](https://issues.apache.org/jira/browse/HADOOP-5531) | *Blocker* | **Remove Chukwa on branch-0.20** + +Disabled Chukwa unit tests for 0.20 branch only. + + +--- + +* [HADOOP-5521](https://issues.apache.org/jira/browse/HADOOP-5521) | *Major* | **Remove dependency of testcases on RESTART\_COUNT** + +This patch makes TestJobHistory and its dependent testcases independent of RESTART\_COUNT. + + +--- + +* [HADOOP-5468](https://issues.apache.org/jira/browse/HADOOP-5468) | *Major* | **Change Hadoop doc menu to sub-menus** + +Reformatted HTML documentation for Hadoop to use submenus at the left column. + + +--- + +* [HADOOP-5030](https://issues.apache.org/jira/browse/HADOOP-5030) | *Major* | **Chukwa RPM build improvements** + +Changed RPM install location to the value specified by build.properties file. + + +--- + +* [HADOOP-4970](https://issues.apache.org/jira/browse/HADOOP-4970) | *Major* | **Use the full path when move files to .Trash/Current** + +Changed trash facility to use absolute path of the deleted file. + + +--- + +* [HADOOP-4873](https://issues.apache.org/jira/browse/HADOOP-4873) | *Major* | **display minMaps/Reduces on advanced scheduler page** + +Changed fair scheduler UI to display minMaps and minReduces variables. + + +--- + +* [HADOOP-4843](https://issues.apache.org/jira/browse/HADOOP-4843) | *Major* | **Collect Job History log file and Job Conf file into Chukwa** + +Introduced Chuckwa collection of job history. + + +--- + +* [HADOOP-4827](https://issues.apache.org/jira/browse/HADOOP-4827) | *Major* | **Improve data aggregation in database** + +Improved framework for data aggregation in Chuckwa. + + +--- + +* [HADOOP-4826](https://issues.apache.org/jira/browse/HADOOP-4826) | *Major* | **Admin command saveNamespace.** + +Introduced new dfsadmin command saveNamespace to command the name service to do an immediate save of the file system image. + + +--- + +* [HADOOP-4789](https://issues.apache.org/jira/browse/HADOOP-4789) | *Minor* | **Change fair scheduler to share between pools by default, not between invidual jobs** + +Changed fair scheduler to divide resources equally between pools, not jobs. + + +--- + +* [HADOOP-4783](https://issues.apache.org/jira/browse/HADOOP-4783) | *Blocker* | **History files are given world readable permissions.** + +Changed history directory permissions to 750 and history file permissions to 740. + + +--- + +* [HADOOP-4749](https://issues.apache.org/jira/browse/HADOOP-4749) | *Major* | **reducer should output input data size when shuffling is done** + +Added a new counter REDUCE\_INPUT\_BYTES. + + +--- + +* [HADOOP-4661](https://issues.apache.org/jira/browse/HADOOP-4661) | *Major* | **distch: a tool for distributed ch{mod,own}** + +Introduced distch tool for parallel ch{mod, own, grp}. + + +--- + +* [HADOOP-4631](https://issues.apache.org/jira/browse/HADOOP-4631) | *Major* | **Split the default configurations into 3 parts** + +Split hadoop-default.xml into core-default.xml, hdfs-default.xml and mapreduce-default.xml. + + +--- + +* [HADOOP-4618](https://issues.apache.org/jira/browse/HADOOP-4618) | *Major* | **Move http server from FSNamesystem into NameNode.** + +Moved HTTP server from FSNameSystem to NameNode. Removed FSNamesystem.getNameNodeInfoPort(). Replaced FSNamesystem.getDFSNameNodeMachine() and FSNamesystem.getDFSNameNodePort() with new method FSNamesystem.getDFSNameNodeAddress(). Removed constructor NameNode(bindAddress, conf). + + +--- + +* [HADOOP-4576](https://issues.apache.org/jira/browse/HADOOP-4576) | *Major* | **Modify pending tasks count in the UI to pending jobs count in the UI** + +Changed capacity scheduler UI to better present number of running and pending tasks. + + +--- + +* [HADOOP-4575](https://issues.apache.org/jira/browse/HADOOP-4575) | *Major* | **An independent HTTPS proxy for HDFS** + +Introduced independent HSFTP proxy server for authenticated access to clusters. + + +--- + +* [HADOOP-4572](https://issues.apache.org/jira/browse/HADOOP-4572) | *Major* | **INode and its sub-classes should be package private** + +Moved org.apache.hadoop.hdfs.{CreateEditsLog, NNThroughputBenchmark} to org.apache.hadoop.hdfs.server.namenode. + + +--- + +* [HADOOP-4567](https://issues.apache.org/jira/browse/HADOOP-4567) | *Major* | **GetFileBlockLocations should return the NetworkTopology information of the machines that hosts those blocks** + +Changed GetFileBlockLocations to return topology information for nodes that host the block replicas. + + +--- + +* [HADOOP-4565](https://issues.apache.org/jira/browse/HADOOP-4565) | *Major* | **MultiFileInputSplit can use data locality information to create splits** + +Improved MultiFileInputFormat so that multiple blocks from the same node or same rack can be combined into a single split. + + +--- + +* [HADOOP-4454](https://issues.apache.org/jira/browse/HADOOP-4454) | *Minor* | **Support comments in 'slaves' file** + +Changed processing of conf/slaves file to allow # to begin a comment. + + +--- + +* [HADOOP-4445](https://issues.apache.org/jira/browse/HADOOP-4445) | *Major* | **Wrong number of running map/reduce tasks are displayed in queue information.** + +Changed JobTracker UI to better present the number of active tasks. + + +--- + +* [HADOOP-4435](https://issues.apache.org/jira/browse/HADOOP-4435) | *Minor* | **The JobTracker should display the amount of heap memory used** + +Changed JobTracker web status page to display the amount of heap memory in use. This changes the JobSubmissionProtocol. + + +--- + +* [HADOOP-4422](https://issues.apache.org/jira/browse/HADOOP-4422) | *Major* | **S3 file systems should not create bucket** + +Modified Hadoop file system to no longer create S3 buckets. Applications can create buckets for their S3 file systems by other means, for example, using the JetS3t API. + + +--- + +* [HADOOP-4374](https://issues.apache.org/jira/browse/HADOOP-4374) | *Major* | **JVM should not be killed but given an opportunity to exit gracefully** + +This patch (1) Adds a shutdownHook that does syncLogs sothat logs of the current task are flushed and log.index is up to date in cases like System.exit(), or killed using signals(other than SIGKILL). +(2) Changes writeToIndexFile() to write to a temporary index file first and then rename to log.index sothat updates to log.index file are atomic. + + +--- + +* [HADOOP-4305](https://issues.apache.org/jira/browse/HADOOP-4305) | *Major* | **repeatedly blacklisted tasktrackers should get declared dead** + +Improved TaskTracker blacklisting strategy to better exclude faulty tracker from executing tasks. + + +--- + +* [HADOOP-4284](https://issues.apache.org/jira/browse/HADOOP-4284) | *Major* | **Support for user configurable global filters on HttpServer** + +Introduced HttpServer method to support global filters. + + +--- + +* [HADOOP-4253](https://issues.apache.org/jira/browse/HADOOP-4253) | *Major* | **Fix warnings generated by FindBugs** + +Removed from class org.apache.hadoop.fs.RawLocalFileSystem deprecated methods public String getName(), public void lock(Path p, boolean shared) and public void release(Path p). + + +--- + +* [HADOOP-4234](https://issues.apache.org/jira/browse/HADOOP-4234) | *Minor* | **KFS: Allow KFS layer to interface with multiple KFS namenodes** + +Changed KFS glue layer to allow applications to interface with multiple KFS metaservers. + + +--- + +* [HADOOP-4210](https://issues.apache.org/jira/browse/HADOOP-4210) | *Major* | **Findbugs warnings are printed related to equals implementation of several classes** + +Changed public class org.apache.hadoop.mapreduce.ID to be an abstract class. Removed from class org.apache.hadoop.mapreduce.ID the methods public static ID read(DataInput in) and public static ID forName(String str). + + +--- + +* [HADOOP-4188](https://issues.apache.org/jira/browse/HADOOP-4188) | *Major* | **Remove Task's dependency on concrete file systems** + +Removed Task's dependency on concrete file systems by taking list from FileSystem class. Added statistics table to FileSystem class. Deprecated FileSystem method getStatistics(Class\ cls). + + +--- + +* [HADOOP-4179](https://issues.apache.org/jira/browse/HADOOP-4179) | *Major* | **Hadoop-Vaidya : Rule based performance diagnostic tool for Map/Reduce jobs** + +Introduced Vaidya rule based performance diagnostic tool for Map/Reduce jobs. + + +--- + +* [HADOOP-4103](https://issues.apache.org/jira/browse/HADOOP-4103) | *Major* | **Alert for missing blocks** + +Modified dfsadmin -report to report under replicated blocks. blocks with corrupt replicas, and missing blocks". + + +--- + +* [HADOOP-4035](https://issues.apache.org/jira/browse/HADOOP-4035) | *Blocker* | **Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory requirements and task trackers free memory** + +Changed capacity scheduler policy to take note of task memory requirements and task tracker memory availability. + + +--- + +* [HADOOP-4029](https://issues.apache.org/jira/browse/HADOOP-4029) | *Major* | **NameNode should report status and performance for each replica of image and log** + +Added name node storage information to the dfshealth page, and moved data node information to a separated page. + + +--- + +* [HADOOP-3986](https://issues.apache.org/jira/browse/HADOOP-3986) | *Major* | **JobClient should not have a static configuration** + +Removed classes org.apache.hadoop.mapred.JobShell and org.apache.hadoop.mapred.TestJobShell. Removed from JobClient methods static void setCommandLineConfig(Configuration conf) and public static Configuration getCommandLineConfig(). + + +--- + +* [HADOOP-3923](https://issues.apache.org/jira/browse/HADOOP-3923) | *Minor* | **Deprecate org.apache.hadoop.mapred.StatusHttpServer** + +Moved class org.apache.hadoop.mapred.StatusHttpServer to org.apache.hadoop.http.HttpServer. + + +--- + +* [HADOOP-3750](https://issues.apache.org/jira/browse/HADOOP-3750) | *Major* | **Fix and enforce module dependencies** + +Removed deprecated method parseArgs from org.apache.hadoop.fs.FileSystem. + + +--- + +* [HADOOP-3497](https://issues.apache.org/jira/browse/HADOOP-3497) | *Major* | **File globbing with a PathFilter is too restrictive** + +Changed the semantics of file globbing with a PathFilter (using the globStatus method of FileSystem). Previously, the filtering was too restrictive, so that a glob of /*/* and a filter that only accepts /a/b would not have matched /a/b. With this change /a/b does match. + + +--- + +* [HADOOP-3422](https://issues.apache.org/jira/browse/HADOOP-3422) | *Major* | **Ganglia counter metrics are all reported with the metric name "value", so the counter values can not be seen** + +Changed names of ganglia metrics to avoid conflicts and to better identify source function. + + +--- + +* [HADOOP-3344](https://issues.apache.org/jira/browse/HADOOP-3344) | *Major* | **libhdfs: always builds 32bit, even when x86\_64 Java used** + +Changed build procedure for libhdfs to build correctly for different platforms. Build instructions are in the Jira item. + + +--- + +* [HADOOP-3063](https://issues.apache.org/jira/browse/HADOOP-3063) | *Major* | **BloomMapFile - fail-fast version of MapFile for sparsely populated key space** + +Introduced BloomMapFile subclass of MapFile that creates a Bloom filter from all keys. + + +--- + +* [HADOOP-1650](https://issues.apache.org/jira/browse/HADOOP-1650) | *Major* | **Upgrade Jetty to 6.x** + +Upgraded all core servers to use Jetty 6 + + +--- + +* [HADOOP-1230](https://issues.apache.org/jira/browse/HADOOP-1230) | *Major* | **Replace parameters with context objects in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes** + +Replaced parameters with context obejcts in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md new file mode 100644 index 00000000000..59af3f2ce60 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/CHANGES.0.20.1.md @@ -0,0 +1,139 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.1 - 2009-09-01 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5881](https://issues.apache.org/jira/browse/HADOOP-5881) | Simplify configuration related to task-memory-monitoring and memory-based scheduling | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-5726](https://issues.apache.org/jira/browse/HADOOP-5726) | Remove pre-emption from the capacity scheduler code base | Major | . | Hemanth Yamijala | rahul k singh | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6080](https://issues.apache.org/jira/browse/HADOOP-6080) | Handling of Trash with quota | Major | fs | Koji Noguchi | Jakob Homan | +| [HADOOP-5714](https://issues.apache.org/jira/browse/HADOOP-5714) | Metric to show number of fs.exists (or number of getFileInfo) calls | Minor | metrics | Koji Noguchi | Jakob Homan | +| [HADOOP-3315](https://issues.apache.org/jira/browse/HADOOP-3315) | New binary file format | Major | io | Owen O'Malley | Hong Tang | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-635](https://issues.apache.org/jira/browse/HDFS-635) | HDFS Project page does not show 0.20.1 documentation/release information. | Major | documentation | Andy Sautins | | +| [HDFS-527](https://issues.apache.org/jira/browse/HDFS-527) | Refactor DFSClient constructors | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-767](https://issues.apache.org/jira/browse/MAPREDUCE-767) | to remove mapreduce dependency on commons-cli2 | Major | contrib/streaming | Giridharan Kesavan | Amar Kamat | +| [MAPREDUCE-465](https://issues.apache.org/jira/browse/MAPREDUCE-465) | Deprecate org.apache.hadoop.mapred.lib.MultithreadedMapRunner | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6215](https://issues.apache.org/jira/browse/HADOOP-6215) | fix GenericOptionParser to deal with -D with '=' in the value | Major | . | Owen O'Malley | Amar Kamat | +| [HADOOP-6145](https://issues.apache.org/jira/browse/HADOOP-6145) | No error message for deleting non-existant file or directory. | Major | fs | Suman Sehgal | Jakob Homan | +| [HADOOP-6141](https://issues.apache.org/jira/browse/HADOOP-6141) | hadoop 0.20 branch "test-patch" is broken | Major | build | Hong Tang | Hong Tang | +| [HADOOP-6139](https://issues.apache.org/jira/browse/HADOOP-6139) | Incomplete help message is displayed for rm and rmr options. | Minor | . | Suman Sehgal | Jakob Homan | +| [HADOOP-6017](https://issues.apache.org/jira/browse/HADOOP-6017) | NameNode and SecondaryNameNode fail to restart because of abnormal filenames. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-5951](https://issues.apache.org/jira/browse/HADOOP-5951) | StorageInfo needs Apache license header. | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5937](https://issues.apache.org/jira/browse/HADOOP-5937) | Correct info message "Use hadoop dfs -safemode option" to " Use hdfs dfsadmin -safemode option" . | Minor | . | Ravi Phulari | Ravi Phulari | +| [HADOOP-5932](https://issues.apache.org/jira/browse/HADOOP-5932) | MemoryMatcher logs 0 as freeMemOnTT even though there are free slots available on TaskTraker | Major | . | Karam Singh | Vinod Kumar Vavilapalli | +| [HADOOP-5924](https://issues.apache.org/jira/browse/HADOOP-5924) | JT fails to recover the jobs after restart after HADOOP:4372 | Major | . | Ramya Sunil | Amar Kamat | +| [HADOOP-5921](https://issues.apache.org/jira/browse/HADOOP-5921) | JobTracker does not come up because of NotReplicatedYetException | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-5920](https://issues.apache.org/jira/browse/HADOOP-5920) | TestJobHistory fails some times. | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-5908](https://issues.apache.org/jira/browse/HADOOP-5908) | ArithmeticException in heartbeats with zero map jobs | Major | . | Vinod Kumar Vavilapalli | Amar Kamat | +| [HADOOP-5884](https://issues.apache.org/jira/browse/HADOOP-5884) | Capacity scheduler should account high memory jobs as using more capacity of the queue | Major | . | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-5883](https://issues.apache.org/jira/browse/HADOOP-5883) | TaskMemoryMonitorThread might shoot down tasks even if their processes momentarily exceed the requested memory | Major | . | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-5882](https://issues.apache.org/jira/browse/HADOOP-5882) | Progress is not updated when the New Reducer is running reduce phase | Blocker | . | Jothi Padmanabhan | Amareshwari Sriramadasu | +| [HADOOP-5850](https://issues.apache.org/jira/browse/HADOOP-5850) | map/reduce doesn't run jobs with 0 maps | Critical | . | Owen O'Malley | Vinod Kumar Vavilapalli | +| [HADOOP-5828](https://issues.apache.org/jira/browse/HADOOP-5828) | Use absolute path for JobTracker's mapred.local.dir in MiniMRCluster | Major | test | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-5746](https://issues.apache.org/jira/browse/HADOOP-5746) | Errors encountered in MROutputThread after the last map/reduce call can go undetected | Major | . | Devaraj Das | Amar Kamat | +| [HADOOP-5736](https://issues.apache.org/jira/browse/HADOOP-5736) | Update CapacityScheduler documentation to reflect latest changes | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5719](https://issues.apache.org/jira/browse/HADOOP-5719) | Jobs failed during job initalization are never removed from Capacity Schedulers waiting list | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5718](https://issues.apache.org/jira/browse/HADOOP-5718) | Capacity Scheduler should not check for presence of default queue while starting up. | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5711](https://issues.apache.org/jira/browse/HADOOP-5711) | Change Namenode file close log to info | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5691](https://issues.apache.org/jira/browse/HADOOP-5691) | org.apache.hadoop.mapreduce.Reducer should not be abstract. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5688](https://issues.apache.org/jira/browse/HADOOP-5688) | HftpFileSystem.getChecksum(..) does not work for the paths with scheme and authority | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5655](https://issues.apache.org/jira/browse/HADOOP-5655) | TestMRServerPorts fails on java.net.BindException | Major | . | Hairong Kuang | Devaraj Das | +| [HADOOP-5654](https://issues.apache.org/jira/browse/HADOOP-5654) | TestReplicationPolicy.\ fails on java.net.BindException | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-5648](https://issues.apache.org/jira/browse/HADOOP-5648) | Not able to generate gridmix.jar on already compiled version of hadoop | Major | benchmarks | Suman Sehgal | Giridharan Kesavan | +| [HADOOP-5646](https://issues.apache.org/jira/browse/HADOOP-5646) | TestQueueCapacities is failing Hudson tests for the last few builds | Major | . | Jothi Padmanabhan | Vinod Kumar Vavilapalli | +| [HADOOP-5641](https://issues.apache.org/jira/browse/HADOOP-5641) | Possible NPE in CapacityScheduler's MemoryMatcher | Major | . | Vinod Kumar Vavilapalli | Hemanth Yamijala | +| [HADOOP-5636](https://issues.apache.org/jira/browse/HADOOP-5636) | Job is left in Running state after a killJob | Critical | . | Amareshwari Sriramadasu | Amar Kamat | +| [HADOOP-5539](https://issues.apache.org/jira/browse/HADOOP-5539) | o.a.h.mapred.Merger not maintaining map out compression on intermediate files | Blocker | . | Billy Pearson | Jothi Padmanabhan | +| [HADOOP-5533](https://issues.apache.org/jira/browse/HADOOP-5533) | Recovery duration shown on the jobtracker webpage is inaccurate | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-5349](https://issues.apache.org/jira/browse/HADOOP-5349) | When the size required for a path is -1, LocalDirAllocator.getLocalPathForWrite fails with a DiskCheckerException when the disk it selects is bad. | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-5213](https://issues.apache.org/jira/browse/HADOOP-5213) | BZip2CompressionOutputStream NullPointerException | Blocker | io | Zheng Shao | Zheng Shao | +| [HADOOP-5210](https://issues.apache.org/jira/browse/HADOOP-5210) | Reduce Task Progress shows \> 100% when the total size of map outputs (for a single reducer) is high | Minor | . | Jothi Padmanabhan | Ravi Gummadi | +| [HADOOP-4674](https://issues.apache.org/jira/browse/HADOOP-4674) | hadoop fs -help should list detailed help info for the following commands: test, text, tail, stat & touchz | Trivial | fs | David NeSmith | Ravi Phulari | +| [HADOOP-4626](https://issues.apache.org/jira/browse/HADOOP-4626) | API link in forrest doc should point to the same version of hadoop. | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1022](https://issues.apache.org/jira/browse/HDFS-1022) | Merge under-10-min tests specs into one file | Major | test | Erik Steffl | Erik Steffl | +| [HDFS-525](https://issues.apache.org/jira/browse/HDFS-525) | ListPathsServlet.java uses static SimpleDateFormat that has threading issues | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-438](https://issues.apache.org/jira/browse/HDFS-438) | Improve help message for quotas | Minor | . | Raghu Angadi | Raghu Angadi | +| [HDFS-167](https://issues.apache.org/jira/browse/HDFS-167) | DFSClient continues to retry indefinitely | Minor | hdfs-client | Derek Wollenstein | Bill Zeller | +| [HDFS-26](https://issues.apache.org/jira/browse/HDFS-26) | HADOOP-5862 for version .20 (Namespace quota exceeded message unclear) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-924](https://issues.apache.org/jira/browse/MAPREDUCE-924) | TestPipes must not directly invoke 'main' of pipes as an exit from main could cause the testcase to crash. | Major | pipes | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-911](https://issues.apache.org/jira/browse/MAPREDUCE-911) | TestTaskFail fail sometimes | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-838](https://issues.apache.org/jira/browse/MAPREDUCE-838) | Task succeeds even when committer.commitTask fails with IOException | Blocker | task | Koji Noguchi | Amareshwari Sriramadasu | +| [MAPREDUCE-834](https://issues.apache.org/jira/browse/MAPREDUCE-834) | When TaskTracker config use old memory management values its memory monitoring is diabled. | Major | . | Karam Singh | Sreekanth Ramakrishnan | +| [MAPREDUCE-832](https://issues.apache.org/jira/browse/MAPREDUCE-832) | Too many WARN messages about deprecated memorty config variables in JobTacker log | Major | . | Karam Singh | rahul k singh | +| [MAPREDUCE-818](https://issues.apache.org/jira/browse/MAPREDUCE-818) | org.apache.hadoop.mapreduce.Counters.getGroup returns null if the group name doesnt exist. | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-807](https://issues.apache.org/jira/browse/MAPREDUCE-807) | Stray user files in mapred.system.dir with permissions other than 777 can prevent the jobtracker from starting up. | Blocker | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-805](https://issues.apache.org/jira/browse/MAPREDUCE-805) | Deadlock in Jobtracker | Major | . | Michael Tamm | Amar Kamat | +| [MAPREDUCE-796](https://issues.apache.org/jira/browse/MAPREDUCE-796) | Encountered "ClassCastException" on tasktracker while running wordcount with MultithreadedMapRunner | Major | examples | Suman Sehgal | Amar Kamat | +| [MAPREDUCE-745](https://issues.apache.org/jira/browse/MAPREDUCE-745) | TestRecoveryManager fails sometimes | Major | jobtracker | Amareshwari Sriramadasu | Amar Kamat | +| [MAPREDUCE-735](https://issues.apache.org/jira/browse/MAPREDUCE-735) | ArrayIndexOutOfBoundsException is thrown by KeyFieldBasedPartitioner | Major | . | Suman Sehgal | Amar Kamat | +| [MAPREDUCE-687](https://issues.apache.org/jira/browse/MAPREDUCE-687) | TestMiniMRMapRedDebugScript fails sometimes | Major | test | Amar Kamat | Amareshwari Sriramadasu | +| [MAPREDUCE-657](https://issues.apache.org/jira/browse/MAPREDUCE-657) | CompletedJobStatusStore hardcodes filesystem to hdfs | Major | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-565](https://issues.apache.org/jira/browse/MAPREDUCE-565) | Partitioner does not work with new API | Blocker | task | Jothi Padmanabhan | Owen O'Malley | +| [MAPREDUCE-430](https://issues.apache.org/jira/browse/MAPREDUCE-430) | Task stuck in cleanup with OutOfMemoryErrors | Major | . | Amareshwari Sriramadasu | Amar Kamat | +| [MAPREDUCE-421](https://issues.apache.org/jira/browse/MAPREDUCE-421) | mapred pipes might return exit code 0 even when failing | Major | pipes | Christian Kunz | Christian Kunz | +| [MAPREDUCE-383](https://issues.apache.org/jira/browse/MAPREDUCE-383) | pipes combiner does not reset properly after a spill | Major | . | Christian Kunz | Christian Kunz | +| [MAPREDUCE-179](https://issues.apache.org/jira/browse/MAPREDUCE-179) | setProgress not called for new RecordReaders | Blocker | . | Chris Douglas | Chris Douglas | +| [MAPREDUCE-130](https://issues.apache.org/jira/browse/MAPREDUCE-130) | Delete the jobconf copy from the log directory of the JobTracker when the job is retired | Major | . | Devaraj Das | Amar Kamat | +| [MAPREDUCE-124](https://issues.apache.org/jira/browse/MAPREDUCE-124) | When abortTask of OutputCommitter fails with an Exception for a map-only job, the task is marked as success | Major | . | Jothi Padmanabhan | Amareshwari Sriramadasu | +| [MAPREDUCE-40](https://issues.apache.org/jira/browse/MAPREDUCE-40) | Memory management variables need a backwards compatibility option after HADOOP-5881 | Blocker | . | Hemanth Yamijala | rahul k singh | +| [MAPREDUCE-18](https://issues.apache.org/jira/browse/MAPREDUCE-18) | Under load the shuffle sometimes gets incorrect data | Blocker | . | Owen O'Malley | Ravi Gummadi | +| [MAPREDUCE-2](https://issues.apache.org/jira/browse/MAPREDUCE-2) | ArrayOutOfIndex error in KeyFieldBasedPartitioner on empty key | Major | . | Amar Kamat | Amar Kamat | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6213](https://issues.apache.org/jira/browse/HADOOP-6213) | Remove commons dependency on commons-cli2 | Blocker | util | Amar Kamat | Amar Kamat | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/RELEASENOTES.0.20.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/RELEASENOTES.0.20.1.md new file mode 100644 index 00000000000..afe74f186a8 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.1/RELEASENOTES.0.20.1.md @@ -0,0 +1,218 @@ + + +# Apache Hadoop 0.20.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-6213](https://issues.apache.org/jira/browse/HADOOP-6213) | *Blocker* | **Remove commons dependency on commons-cli2** + +GenericOptionsParser in branch 0.20 depends on commons-cli2. This jira removes the dependency of branch 0.20 on commons-cli2 completely. The problem is seen after 'ant binary' where all the library files are copied to '$hadoop-home/lib' which already has commons-cli2. + + +--- + +* [HADOOP-6080](https://issues.apache.org/jira/browse/HADOOP-6080) | *Major* | **Handling of Trash with quota** + +Provide a new option to rm and rmr, -skipTrash, which will immediately delete the files specified, rather than moving them to the trash. + + +--- + +* [HADOOP-5924](https://issues.apache.org/jira/browse/HADOOP-5924) | *Major* | **JT fails to recover the jobs after restart after HADOOP:4372** + +Post HADOOP-4372, empty job history files caused NPE. This issues fixes that by creating new files if no old file is found. + + +--- + +* [HADOOP-5921](https://issues.apache.org/jira/browse/HADOOP-5921) | *Major* | **JobTracker does not come up because of NotReplicatedYetException** + +Jobtracker crashes if it fails to create jobtracker.info file (i.e if sufficient datanodes are not up). With this patch it keeps on retrying on IOExceptions assuming IOExceptions in jobtracker.info creation implies that the hdfs is not in *ready *state. + + +--- + +* [HADOOP-5920](https://issues.apache.org/jira/browse/HADOOP-5920) | *Major* | **TestJobHistory fails some times.** + +TestJobHistory fails as jobtracker is restarted very fast (within a minute) and history files from earlier testcases were not cleaned up. This patch cleans up the history-dir and mapred-system-dir after every test. + + +--- + +* [HADOOP-5884](https://issues.apache.org/jira/browse/HADOOP-5884) | *Major* | **Capacity scheduler should account high memory jobs as using more capacity of the queue** + +Fixes Capacity scheduler to account more capacity of a queue for a high memory job. Done by considering these jobs to +take more slots proportionally with respect to a slot's default memory size. + + +--- + +* [HADOOP-5881](https://issues.apache.org/jira/browse/HADOOP-5881) | *Major* | **Simplify configuration related to task-memory-monitoring and memory-based scheduling** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-5746](https://issues.apache.org/jira/browse/HADOOP-5746) | *Major* | **Errors encountered in MROutputThread after the last map/reduce call can go undetected** + +If the child (streaming) process returns successfully and the MROutputThread throws an error, there was no way to detect that as all the IOExceptions was ignored. Such issues can occur when DFS clients were closed etc. Now a check for errors (in threads) is made before finishing off the task and an exception is thrown that fails he task. + + +--- + +* [HADOOP-5726](https://issues.apache.org/jira/browse/HADOOP-5726) | *Major* | **Remove pre-emption from the capacity scheduler code base** + +Removed pre-emption from capacity scheduler. The impact of this change is that capacities for queues can no longer be guaranteed within a given span of time. Also changed configuration variables to remove pre-emption related variables and better reflect the absence of guarantees. + + +--- + +* [HADOOP-5210](https://issues.apache.org/jira/browse/HADOOP-5210) | *Minor* | **Reduce Task Progress shows \> 100% when the total size of map outputs (for a single reducer) is high** + +This patch resets the variable totalBytesProcessed before the final merge sothat it will be used for calculating the progress of reducePhase(the 3rd phase of reduce task) correctly. + + +--- + +* [HADOOP-3315](https://issues.apache.org/jira/browse/HADOOP-3315) | *Major* | **New binary file format** + +Add a new, binary file format TFile. + + +--- + +* [MAPREDUCE-838](https://issues.apache.org/jira/browse/MAPREDUCE-838) | *Blocker* | **Task succeeds even when committer.commitTask fails with IOException** + +Fixed a bug in the way commit of task outputs happens. The bug was that if commit fails with IOException, the task would be declared as successful. + + +--- + +* [MAPREDUCE-834](https://issues.apache.org/jira/browse/MAPREDUCE-834) | *Major* | **When TaskTracker config use old memory management values its memory monitoring is diabled.** + +The tasktracker's startup code was modified to use deprecated memory management configuration variables, when specified, and enable memory monitoring of tasks. + + +--- + +* [MAPREDUCE-832](https://issues.apache.org/jira/browse/MAPREDUCE-832) | *Major* | **Too many WARN messages about deprecated memorty config variables in JobTacker log** + +Reduced the frequency of log messages printed when a deprecated memory management variable is found in configuration of a job. + + +--- + +* [MAPREDUCE-818](https://issues.apache.org/jira/browse/MAPREDUCE-818) | *Minor* | **org.apache.hadoop.mapreduce.Counters.getGroup returns null if the group name doesnt exist.** + +Fixed a bug in the new org.apache.hadoop.mapreduce.Counters.getGroup() method to return an empty group if group name doesn't exist, instead of null, thus making sure that it is in sync with the Javadoc. + + +--- + +* [MAPREDUCE-807](https://issues.apache.org/jira/browse/MAPREDUCE-807) | *Blocker* | **Stray user files in mapred.system.dir with permissions other than 777 can prevent the jobtracker from starting up.** + +The JobTracker tries to delete the mapred.system.dir when it is starting up (with the job recovery disabled). The fix provided by this jira is that JobTracker will fail (bail out) with AccessControlException if it fails to delete files/directories in mapred.system.dir due to access control issues. + + +--- + +* [MAPREDUCE-805](https://issues.apache.org/jira/browse/MAPREDUCE-805) | *Major* | **Deadlock in Jobtracker** + +Job initialization process was changed to not change (run) states during initialization. The reason is two fold +- this can lead to deadlock as state changes require circular locking (i.e JobInProgress requires JobTracker lock) +- events were not raised as these state changes were not informed/propogated back to the JobTracker + +Now the JobTracker takes care of initializing/failing/killing the job and raising appropriate events. The simple rule that was enforced was that "The JobTracker lock is *must* before changing the run-state of a job". + + +--- + +* [MAPREDUCE-796](https://issues.apache.org/jira/browse/MAPREDUCE-796) | *Major* | **Encountered "ClassCastException" on tasktracker while running wordcount with MultithreadedMapRunner** + +Multithreaded mapper was modified to create a new Runtime exception (object) from a throwable instead of casting a throwable into a RuntimeException, once the Multithreaded map encounters a fault. + + +--- + +* [MAPREDUCE-767](https://issues.apache.org/jira/browse/MAPREDUCE-767) | *Major* | **to remove mapreduce dependency on commons-cli2** + +Removes the dependency of hadoop-mapred from commons-cli2 and uses commons-cli1.2 for command-line parsing. + + +--- + +* [MAPREDUCE-745](https://issues.apache.org/jira/browse/MAPREDUCE-745) | *Major* | **TestRecoveryManager fails sometimes** + +JobTracker was changed to take an identifier as an argument. This helps in testcases where the jobtracker/mapred-cluster is (re)started in a short span of time and the chances of jobtracker identifier clashing are high. Also the RecoveryManager was modified to throw an exception if a job fails in init during the recovery process. The reason being that this event will trigger a job failure in the recovery process and will remove the failed job from further initialization and processing. + + +--- + +* [MAPREDUCE-657](https://issues.apache.org/jira/browse/MAPREDUCE-657) | *Major* | **CompletedJobStatusStore hardcodes filesystem to hdfs** + +CompletedJobStatusStore was hardcored to persist to hdfs. This patch allows to persist to local fs. Just qualify mapred.job.tracker.persist.jobstatus.dir with file:// + + +--- + +* [MAPREDUCE-430](https://issues.apache.org/jira/browse/MAPREDUCE-430) | *Major* | **Task stuck in cleanup with OutOfMemoryErrors** + +Various code paths in the framework caught Throwable and tried to do inline cleanup. In case of OOM errors, such inline-cleanups can result into hung jvms. With this fix, the TaskTracker provides a api to report fatal errors (any throwable other than FSErrror and Exceptions). On catching a Throwable, Mapper/Reducer tries to inform the TT. + + +--- + +* [MAPREDUCE-383](https://issues.apache.org/jira/browse/MAPREDUCE-383) | *Major* | **pipes combiner does not reset properly after a spill** + +Fixed a bug in Pipes combiner to reset the spilled bytes count after the spill. + + +--- + +* [MAPREDUCE-130](https://issues.apache.org/jira/browse/MAPREDUCE-130) | *Major* | **Delete the jobconf copy from the log directory of the JobTracker when the job is retired** + +When a job is initialized, it localizes the job conf to the logs dir. Without this patch I never gets deleted. Now when the job retires, the conf is deleted. This local copy is required to display on the webui. + + +--- + +* [MAPREDUCE-40](https://issues.apache.org/jira/browse/MAPREDUCE-40) | *Blocker* | **Memory management variables need a backwards compatibility option after HADOOP-5881** + +Fixed backwards compatibility by re-introducing and deprecating removed memory monitoring related configuration options. + + +--- + +* [MAPREDUCE-18](https://issues.apache.org/jira/browse/MAPREDUCE-18) | *Blocker* | **Under load the shuffle sometimes gets incorrect data** + +This patch adds the mapid and reduceid in the http header of mapoutput when being sent to reduce node. Also validates compressed length, decompressed length, mapid and reduceid from http header at reduce node. + + +--- + +* [MAPREDUCE-2](https://issues.apache.org/jira/browse/MAPREDUCE-2) | *Major* | **ArrayOutOfIndex error in KeyFieldBasedPartitioner on empty key** + +KeyFieldBasedPartitioner throws ArrayOutOfIndex when passed an empty key. This patch hashes empty key to 0 hashcode. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/CHANGES.0.20.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/CHANGES.0.20.2.md new file mode 100644 index 00000000000..fb21db9fbde --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/CHANGES.0.20.2.md @@ -0,0 +1,109 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.2 - 2010-02-16 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-793](https://issues.apache.org/jira/browse/HDFS-793) | DataNode should first receive the whole packet ack message before it constructs and sends its own ack message for the packet | Blocker | datanode | Hairong Kuang | Hairong Kuang | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-1849](https://issues.apache.org/jira/browse/HADOOP-1849) | IPC server max queue size should be configurable | Major | ipc | Raghu Angadi | Konstantin Shvachko | +| [MAPREDUCE-623](https://issues.apache.org/jira/browse/MAPREDUCE-623) | Resolve javac warnings in mapred | Major | build | Jothi Padmanabhan | Jothi Padmanabhan | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6576](https://issues.apache.org/jira/browse/HADOOP-6576) | TestStreamingStatus is failing on 0.20 branch | Major | . | Chris Douglas | Todd Lipcon | +| [HADOOP-6575](https://issues.apache.org/jira/browse/HADOOP-6575) | Tests do not run on 0.20 branch | Major | . | Chris Douglas | Chris Douglas | +| [HADOOP-6524](https://issues.apache.org/jira/browse/HADOOP-6524) | Contrib tests are failing Clover'ed build | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6506](https://issues.apache.org/jira/browse/HADOOP-6506) | Failing tests prevent the rest of test targets from execution. | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6498](https://issues.apache.org/jira/browse/HADOOP-6498) | IPC client bug may cause rpc call hang | Blocker | ipc | Ruyue Ma | Ruyue Ma | +| [HADOOP-6460](https://issues.apache.org/jira/browse/HADOOP-6460) | Namenode runs of out of memory due to memory leak in ipc Server | Blocker | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6428](https://issues.apache.org/jira/browse/HADOOP-6428) | HttpServer sleeps with negative values | Major | . | Tsz Wo Nicholas Sze | Konstantin Boudnik | +| [HADOOP-6315](https://issues.apache.org/jira/browse/HADOOP-6315) | GzipCodec should not represent BuiltInZlibInflater as decompressorType | Major | io | Aaron Kimball | Aaron Kimball | +| [HADOOP-6269](https://issues.apache.org/jira/browse/HADOOP-6269) | Missing synchronization for defaultResources in Configuration.addResource | Major | conf | Todd Lipcon | Sreekanth Ramakrishnan | +| [HADOOP-6231](https://issues.apache.org/jira/browse/HADOOP-6231) | Allow caching of filesystem instances to be disabled on a per-instance basis | Major | fs | Tom White | Ben Slusky | +| [HADOOP-6097](https://issues.apache.org/jira/browse/HADOOP-6097) | Multiple bugs w/ Hadoop archives | Major | fs | Ben Slusky | Ben Slusky | +| [HADOOP-5759](https://issues.apache.org/jira/browse/HADOOP-5759) | IllegalArgumentException when CombineFileInputFormat is used as job InputFormat | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5623](https://issues.apache.org/jira/browse/HADOOP-5623) | Streaming: process provided status messages are overwritten every 10 seoncds | Major | . | Rick Cox | Rick Cox | +| [HADOOP-5612](https://issues.apache.org/jira/browse/HADOOP-5612) | Some c++ scripts are not chmodded before ant execution | Major | build | Todd Lipcon | Todd Lipcon | +| [HADOOP-5611](https://issues.apache.org/jira/browse/HADOOP-5611) | C++ libraries do not build on Debian Lenny | Critical | . | Todd Lipcon | Todd Lipcon | +| [HDFS-927](https://issues.apache.org/jira/browse/HDFS-927) | DFSInputStream retries too many times for new block locations | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-872](https://issues.apache.org/jira/browse/HDFS-872) | DFSClient 0.20.1 is incompatible with HDFS 0.20.2 | Major | datanode, hdfs-client | Bassam Tabbara | Todd Lipcon | +| [HDFS-781](https://issues.apache.org/jira/browse/HDFS-781) | Metrics PendingDeletionBlocks is not decremented | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-761](https://issues.apache.org/jira/browse/HDFS-761) | Failure to process rename operation from edits log due to quota verification | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-745](https://issues.apache.org/jira/browse/HDFS-745) | TestFsck timeout on 0.20. | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-732](https://issues.apache.org/jira/browse/HDFS-732) | HDFS files are ending up truncated | Blocker | hdfs-client | Christian Kunz | Tsz Wo Nicholas Sze | +| [HDFS-723](https://issues.apache.org/jira/browse/HDFS-723) | Deadlock in DFSClient#DFSOutputStream | Blocker | . | Hairong Kuang | Hairong Kuang | +| [HDFS-677](https://issues.apache.org/jira/browse/HDFS-677) | Rename failure due to quota results in deletion of src directory | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-596](https://issues.apache.org/jira/browse/HDFS-596) | Memory leak in libhdfs: hdfsFreeFileInfo() in libhdfs does not free memory for mOwner and mGroup | Blocker | fuse-dfs | Zhang Bingjun | Zhang Bingjun | +| [HDFS-579](https://issues.apache.org/jira/browse/HDFS-579) | HADOOP-3792 update of DfsTask incomplete | Major | hdfs-client | Christian Kunz | Christian Kunz | +| [HDFS-187](https://issues.apache.org/jira/browse/HDFS-187) | TestStartup fails if hdfs is running in the same machine | Major | test | Tsz Wo Nicholas Sze | Todd Lipcon | +| [HDFS-185](https://issues.apache.org/jira/browse/HDFS-185) | Chown , chgrp , chmod operations allowed when namenode is in safemode . | Major | . | Ravi Phulari | Ravi Phulari | +| [HDFS-101](https://issues.apache.org/jira/browse/HDFS-101) | DFS write pipeline : DFSClient sometimes does not detect second datanode failure | Blocker | datanode | Raghu Angadi | Hairong Kuang | +| [MAPREDUCE-1251](https://issues.apache.org/jira/browse/MAPREDUCE-1251) | c++ utils doesn't compile | Major | . | Eli Collins | Eli Collins | +| [MAPREDUCE-1182](https://issues.apache.org/jira/browse/MAPREDUCE-1182) | Reducers fail with OutOfMemoryError while copying Map outputs | Blocker | . | Chandra Prakash Bhagtani | Chandra Prakash Bhagtani | +| [MAPREDUCE-1163](https://issues.apache.org/jira/browse/MAPREDUCE-1163) | hdfsJniHelper.h: Yahoo! specific paths are encoded | Trivial | . | Allen Wittenauer | Allen Wittenauer | +| [MAPREDUCE-1147](https://issues.apache.org/jira/browse/MAPREDUCE-1147) | Map output records counter missing for map-only jobs in new API | Blocker | . | Chris Douglas | Amar Kamat | +| [MAPREDUCE-1070](https://issues.apache.org/jira/browse/MAPREDUCE-1070) | Deadlock in FairSchedulerServlet | Major | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1068](https://issues.apache.org/jira/browse/MAPREDUCE-1068) | In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present | Major | contrib/streaming | Peeyush Bishnoi | Amareshwari Sriramadasu | +| [MAPREDUCE-1010](https://issues.apache.org/jira/browse/MAPREDUCE-1010) | Adding tests for changes in archives. | Minor | harchive | Mahadev konar | Mahadev konar | +| [MAPREDUCE-979](https://issues.apache.org/jira/browse/MAPREDUCE-979) | JobConf.getMemoryFor{Map\|Reduce}Task doesn't fallback to newer config knobs when mapred.taskmaxvmem is set to DISABLED\_MEMORY\_LIMIT of -1 | Blocker | jobtracker, tasktracker | Arun C Murthy | Sreekanth Ramakrishnan | +| [MAPREDUCE-826](https://issues.apache.org/jira/browse/MAPREDUCE-826) | harchive doesn't use ToolRunner / harchive returns 0 even if the job fails with exception | Trivial | harchive | Koji Noguchi | Koji Noguchi | +| [MAPREDUCE-617](https://issues.apache.org/jira/browse/MAPREDUCE-617) | Streaming should not throw java.lang.RuntimeException and ERROR while displaying help | Minor | contrib/streaming | Karam Singh | | +| [MAPREDUCE-433](https://issues.apache.org/jira/browse/MAPREDUCE-433) | TestReduceFetch failed. | Major | . | Tsz Wo Nicholas Sze | Chris Douglas | +| [MAPREDUCE-112](https://issues.apache.org/jira/browse/MAPREDUCE-112) | Reduce Input Records and Reduce Output Records counters are not being set when using the new Mapreduce reducer API | Blocker | . | Jothi Padmanabhan | Jothi Padmanabhan | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-919](https://issues.apache.org/jira/browse/HDFS-919) | Create test to validate the BlocksVerified metric | Major | test | gary murry | | +| [HDFS-907](https://issues.apache.org/jira/browse/HDFS-907) | Add tests for getBlockLocations and totalLoad metrics. | Minor | namenode | Ravi Phulari | Ravi Phulari | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md new file mode 100644 index 00000000000..f363e0a7fc5 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.2/RELEASENOTES.0.20.2.md @@ -0,0 +1,150 @@ + + +# Apache Hadoop 0.20.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-6498](https://issues.apache.org/jira/browse/HADOOP-6498) | *Blocker* | **IPC client bug may cause rpc call hang** + +Correct synchronization error in IPC where handler thread could hang if request reader got an error. + + +--- + +* [HADOOP-6460](https://issues.apache.org/jira/browse/HADOOP-6460) | *Blocker* | **Namenode runs of out of memory due to memory leak in ipc Server** + +If an IPC server response buffer has grown to than 1MB, it is replaced by a smaller buffer to free up the Java heap that was used. This will improve the longevity of the name service. + + +--- + +* [HADOOP-6428](https://issues.apache.org/jira/browse/HADOOP-6428) | *Major* | **HttpServer sleeps with negative values** + +Corrected arithmetic error that made sleep times less than zero. + + +--- + +* [HADOOP-6231](https://issues.apache.org/jira/browse/HADOOP-6231) | *Major* | **Allow caching of filesystem instances to be disabled on a per-instance basis** + +Allow a general mechanism to disable the cache on a per filesystem basis by using property fs.\.impl.disable.cache. eg. fs.har.impl.disable.cache in core-default.xml + + +--- + +* [HADOOP-6097](https://issues.apache.org/jira/browse/HADOOP-6097) | *Major* | **Multiple bugs w/ Hadoop archives** + +Bugs fixed for Hadoop archives: character escaping in paths, LineReader and file system caching. + + +--- + +* [HDFS-793](https://issues.apache.org/jira/browse/HDFS-793) | *Blocker* | **DataNode should first receive the whole packet ack message before it constructs and sends its own ack message for the packet** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-781](https://issues.apache.org/jira/browse/HDFS-781) | *Blocker* | **Metrics PendingDeletionBlocks is not decremented** + +Correct PendingDeletionBlocks metric to properly decrement counts. + + +--- + +* [HDFS-761](https://issues.apache.org/jira/browse/HDFS-761) | *Major* | **Failure to process rename operation from edits log due to quota verification** + +Corrected an error when checking quota policy that resulted in a failure to read the edits log, stopping the primary/secondary name node. + + +--- + +* [HDFS-677](https://issues.apache.org/jira/browse/HDFS-677) | *Blocker* | **Rename failure due to quota results in deletion of src directory** + +Rename properly considers the case where both source and destination are over quota; operation will fail with error indication. + + +--- + +* [HDFS-596](https://issues.apache.org/jira/browse/HDFS-596) | *Blocker* | **Memory leak in libhdfs: hdfsFreeFileInfo() in libhdfs does not free memory for mOwner and mGroup** + +Memory leak in function hdfsFreeFileInfo in libhdfs. This bug affects fuse-dfs severely. + + +--- + +* [MAPREDUCE-1182](https://issues.apache.org/jira/browse/MAPREDUCE-1182) | *Blocker* | **Reducers fail with OutOfMemoryError while copying Map outputs** + +Modifies shuffle related memory parameters to use 'long' from 'int' so that sizes greater than maximum integer size are handled correctly + + +--- + +* [MAPREDUCE-1147](https://issues.apache.org/jira/browse/MAPREDUCE-1147) | *Blocker* | **Map output records counter missing for map-only jobs in new API** + +Adds a counter to track the number of records emitted by map writing directly to HDFS i.e map tasks of job with 0 reducers. + + +--- + +* [MAPREDUCE-1068](https://issues.apache.org/jira/browse/MAPREDUCE-1068) | *Major* | **In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present** + +Fix streaming job to show proper message if file is is not present, for -file option. + + +--- + +* [MAPREDUCE-979](https://issues.apache.org/jira/browse/MAPREDUCE-979) | *Blocker* | **JobConf.getMemoryFor{Map\|Reduce}Task doesn't fallback to newer config knobs when mapred.taskmaxvmem is set to DISABLED\_MEMORY\_LIMIT of -1** + +Added support to fallback to new task memory configuration when deprecated memory configuration values are set to disabled. + + +--- + +* [MAPREDUCE-826](https://issues.apache.org/jira/browse/MAPREDUCE-826) | *Trivial* | **harchive doesn't use ToolRunner / harchive returns 0 even if the job fails with exception** + +Use ToolRunner for archives job and return non zero error code on failure. + + +--- + +* [MAPREDUCE-623](https://issues.apache.org/jira/browse/MAPREDUCE-623) | *Major* | **Resolve javac warnings in mapred** + +Removes javac warnings by either resolving them or suppressing them (wherever resolution is not possible) + + +--- + +* [MAPREDUCE-433](https://issues.apache.org/jira/browse/MAPREDUCE-433) | *Major* | **TestReduceFetch failed.** + +Resolves the test failure by modifying the test to base it on spill counters rather than on bytes read/written. It also introduces a new configuration parameter "mapred.job.shuffle.input.buffer.percent" to provide finer grained control on the memory limit to be used during shuffle. + + +--- + +* [MAPREDUCE-112](https://issues.apache.org/jira/browse/MAPREDUCE-112) | *Blocker* | **Reduce Input Records and Reduce Output Records counters are not being set when using the new Mapreduce reducer API** + +Updates of counters for reduce input and output records were added in the new API so they are available for jobs using the new API. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md new file mode 100644 index 00000000000..d951eefc1fb --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/CHANGES.0.20.203.0.md @@ -0,0 +1,95 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.203.0 - 2011-05-11 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7108](https://issues.apache.org/jira/browse/HADOOP-7108) | hadoop-0.20.100 | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-4343](https://issues.apache.org/jira/browse/HADOOP-4343) | Adding user and service-to-service authentication to Hadoop | Blocker | . | Kan Zhang | Kan Zhang | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7247](https://issues.apache.org/jira/browse/HADOOP-7247) | Fix documentation to reflect new jar names | Major | . | Owen O'Malley | Owen O'Malley | +| [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | Make BLOCK\_INVALIDATE\_LIMIT configurable | Minor | namenode | Arun C Murthy | Tsz Wo Nicholas Sze | +| [HDFS-457](https://issues.apache.org/jira/browse/HDFS-457) | better handling of volume failure in Data Node storage | Major | datanode | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-2355](https://issues.apache.org/jira/browse/MAPREDUCE-2355) | Add an out of band heartbeat damper | Major | jobtracker | Owen O'Malley | Arun C Murthy | +| [MAPREDUCE-2316](https://issues.apache.org/jira/browse/MAPREDUCE-2316) | Update docs for CapacityScheduler | Major | capacity-sched, documentation | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-1943](https://issues.apache.org/jira/browse/MAPREDUCE-1943) | Implement limits on per-job JobConf, Counters, StatusReport, Split-Sizes | Major | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | separate jvm param for mapper and reducer | Minor | . | Koji Noguchi | Arun C Murthy | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7259](https://issues.apache.org/jira/browse/HADOOP-7259) | contrib modules should include build.properties from parent. | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-7258](https://issues.apache.org/jira/browse/HADOOP-7258) | Gzip codec should not return null decompressors | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7253](https://issues.apache.org/jira/browse/HADOOP-7253) | Fix default config | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7246](https://issues.apache.org/jira/browse/HADOOP-7246) | The default log4j configuration causes warnings about EventCounter | Major | . | Owen O'Malley | Luke Lu | +| [HADOOP-7243](https://issues.apache.org/jira/browse/HADOOP-7243) | Fix contrib unit tests (fairshare, hdfsproxy, datajoin, streaming) | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7232](https://issues.apache.org/jira/browse/HADOOP-7232) | Fix javadoc warnings | Blocker | documentation | Owen O'Malley | Owen O'Malley | +| [HADOOP-7215](https://issues.apache.org/jira/browse/HADOOP-7215) | RPC clients must connect over a network interface corresponding to the host name in the client's kerberos principal key | Blocker | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7190](https://issues.apache.org/jira/browse/HADOOP-7190) | Put metrics v1 back into the hadoop-20-security branch | Major | metrics | Owen O'Malley | Owen O'Malley | +| [HADOOP-7163](https://issues.apache.org/jira/browse/HADOOP-7163) | "java.net.SocketTimeoutException: 60000 millis timeout" happens a lot | Major | ipc | Owen O'Malley | Devaraj Das | +| [HADOOP-7143](https://issues.apache.org/jira/browse/HADOOP-7143) | Hive Hadoop20SShims depends on removed HadoopArchives | Major | fs | Joep Rottinghuis | Joep Rottinghuis | +| [HADOOP-7040](https://issues.apache.org/jira/browse/HADOOP-7040) | DiskChecker:mkdirsWithExistsCheck swallows FileNotFoundException. | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6907](https://issues.apache.org/jira/browse/HADOOP-6907) | Rpc client doesn't use the per-connection conf to figure out server's Kerberos principal | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp | Major | test | Ravi Gummadi | Ravi Gummadi | +| [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1022](https://issues.apache.org/jira/browse/HDFS-1022) | Merge under-10-min tests specs into one file | Major | test | Erik Steffl | Erik Steffl | +| [MAPREDUCE-2365](https://issues.apache.org/jira/browse/MAPREDUCE-2365) | Add counters for FileInputFormat (BYTES\_READ) and FileOutputFormat (BYTES\_WRITTEN) | Major | . | Owen O'Malley | Siddharth Seth | +| [MAPREDUCE-2278](https://issues.apache.org/jira/browse/MAPREDUCE-2278) | DistributedCache shouldn't hold a ref to JobConf | Major | distributed-cache, tasktracker | Arun C Murthy | Chris Douglas | +| [MAPREDUCE-1699](https://issues.apache.org/jira/browse/MAPREDUCE-1699) | JobHistory shouldn't be disabled for any reason | Major | jobtracker | Arun C Murthy | Krishna Ramachandran | +| [MAPREDUCE-1280](https://issues.apache.org/jira/browse/MAPREDUCE-1280) | Eclipse Plugin does not work with Eclipse Ganymede (3.4) | Major | . | Aaron Kimball | Alex Kozlov | +| [MAPREDUCE-1233](https://issues.apache.org/jira/browse/MAPREDUCE-1233) | Incorrect Waiting maps/reduces in Jobtracker metrics | Major | jobtracker | V.Karthikeyan | Luke Lu | +| [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | Capacity Scheduler scheduling information is hard to read / should be tabular format | Major | capacity-sched | Allen Wittenauer | Krishna Ramachandran | +| [MAPREDUCE-323](https://issues.apache.org/jira/browse/MAPREDUCE-323) | Improve the way job history files are managed | Critical | jobtracker | Amar Kamat | Dick King | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md new file mode 100644 index 00000000000..1c1dd4c392f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.0/RELEASENOTES.0.20.203.0.md @@ -0,0 +1,117 @@ + + +# Apache Hadoop 0.20.203.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | *Major* | **TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp** + +Removed dependency of testcase on /tmp and made it to use test.build.data directory instead. + + +--- + +* [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | *Minor* | **Make BLOCK\_INVALIDATE\_LIMIT configurable** + +Added a new configuration property dfs.block.invalidate.limit for FSNamesystem.blockInvalidateLimit. + + +--- + +* [HDFS-457](https://issues.apache.org/jira/browse/HDFS-457) | *Major* | **better handling of volume failure in Data Node storage** + +Datanode can continue if a volume for replica storage fails. Previously a datanode resigned if any volume failed. + + +--- + +* [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | *Major* | **Capacity Scheduler scheduling information is hard to read / should be tabular format** + +Add CapacityScheduler servlet to enhance web UI for queue information. + + +--- + +* [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | *Minor* | **separate jvm param for mapper and reducer** + +Allow map and reduce jvm parameters, environment variables and ulimit to be set separately. + +Configuration changes: + add mapred.map.child.java.opts + add mapred.reduce.child.java.opts + add mapred.map.child.env + add mapred.reduce.child.ulimit + add mapred.map.child.env + add mapred.reduce.child.ulimit + deprecated mapred.child.java.opts + deprecated mapred.child.env + deprecated mapred.child.ulimit + + +--- + +* [MAPREDUCE-323](https://issues.apache.org/jira/browse/MAPREDUCE-323) | *Critical* | **Improve the way job history files are managed** + +This patch does four things: + + * it changes the directory structure of the done directory that holds history logs for jobs that are completed, + * it builds toy databases for completed jobs, so we no longer have to scan 2N files on DFS to find out facts about the N jobs that have completed since the job tracker started [which can be hundreds of thousands of files in practical cases], + * it changes the job history browser to display more information and allow more filtering criteria, and + * it creates a new programmatic interface for finding files matching user-chosen criteria. This allows users to no longer be concerned with our methods of storing them, in turn allowing us to change those at will. + +The new API described above, which can be used to programmatically obtain history file PATHs given search criteria, is described below: + + package org.apache.hadoop.mapreduce.jobhistory; + ... + + // this interface is within O.A.H.mapreduce.jobhistory.JobHistory: + + // holds information about one job hostory log in the done + // job history logs + public static class JobHistoryJobRecord { + public Path getPath() { ... } + public String getJobIDString() { ... } + public long getSubmitTime() { ... } + public String getUserName() { ... } + public String getJobName() { ... } + } + + public class JobHistoryRecordRetriever implements Iterator\ { + // usual Interface methods -- remove() throws UnsupportedOperationException + // returns the number of calls to next() that will succeed + public int numMatches() { ... } + } + + // returns a JobHistoryRecordRetriever that delivers all Path's of job matching job history files, + // in no particular order. Any criterion that is null or the empty string does not constrain. + // All criteria that are specified are applied conjunctively, except that if there's more than + // one date you retrieve all Path's matching ANY date. + // soughtUser and soughtJobid must match exactly. + // soughtJobName can match the entire job name or any substring. + // dates must be in the format exactly MM/DD/YYYY . + // Dates' leading digits must be 2's . We're incubating a Y3K problem. + public JobHistoryRecordRetriever getMatchingJob + (String soughtUser, String soughtJobName, String[] dateStrings, String soughtJobid) + throws IOException + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md new file mode 100644 index 00000000000..def258c98c2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/CHANGES.0.20.203.1.md @@ -0,0 +1,66 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.203.1 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7330](https://issues.apache.org/jira/browse/HADOOP-7330) | The metrics source mbean implementation should return the attribute value instead of the object | Major | metrics | Luke Lu | Luke Lu | +| [HADOOP-7297](https://issues.apache.org/jira/browse/HADOOP-7297) | Error in the documentation regarding Checkpoint/Backup Node | Trivial | documentation | arnaud p | Harsh J | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/RELEASENOTES.0.20.203.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/RELEASENOTES.0.20.203.1.md new file mode 100644 index 00000000000..5827567e144 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.203.1/RELEASENOTES.0.20.203.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.20.203.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md new file mode 100644 index 00000000000..b7eb2b997a1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/CHANGES.0.20.204.0.md @@ -0,0 +1,127 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.204.0 - 2011-09-02 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | Create an rpm integration project | Major | . | Owen O'Malley | Eric Yang | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7398](https://issues.apache.org/jira/browse/HADOOP-7398) | create a mechanism to suppress the HADOOP\_HOME deprecated warning | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7144](https://issues.apache.org/jira/browse/HADOOP-7144) | Expose JMX with something like JMXProxyServlet | Major | . | Luke Lu | Robert Joseph Evans | +| [MAPREDUCE-2558](https://issues.apache.org/jira/browse/MAPREDUCE-2558) | Add queue-level metrics 0.20-security branch | Major | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-1938](https://issues.apache.org/jira/browse/MAPREDUCE-1938) | Ability for having user's classes take precedence over the system classes for tasks' classpath | Blocker | job submission, task, tasktracker | Devaraj Das | Krishna Ramachandran | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7277](https://issues.apache.org/jira/browse/HADOOP-7277) | Add Eclipse launch tasks for the 0.20-security branch | Minor | build | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [HADOOP-7248](https://issues.apache.org/jira/browse/HADOOP-7248) | Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy for 0.20-\* based sources | Minor | build | Konstantin Boudnik | Thomas Graves | +| [HDFS-1773](https://issues.apache.org/jira/browse/HDFS-1773) | Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists | Minor | namenode | Tanping Wang | Tanping Wang | +| [MAPREDUCE-2524](https://issues.apache.org/jira/browse/MAPREDUCE-2524) | Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle | Minor | tasktracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2495](https://issues.apache.org/jira/browse/MAPREDUCE-2495) | The distributed cache cleanup thread has no monitoring to check to see if it has died for some reason | Minor | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2490](https://issues.apache.org/jira/browse/MAPREDUCE-2490) | Log blacklist debug count | Trivial | jobtracker | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2479](https://issues.apache.org/jira/browse/MAPREDUCE-2479) | Backport MAPREDUCE-1568 to hadoop security branch | Major | tasktracker | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2456](https://issues.apache.org/jira/browse/MAPREDUCE-2456) | Show the reducer taskid and map/reduce tasktrackers for "Failed fetch notification #\_ for task attempt..." log messages | Trivial | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7475](https://issues.apache.org/jira/browse/HADOOP-7475) | hadoop-setup-single-node.sh is broken | Blocker | . | Eric Yang | Eric Yang | +| [HADOOP-7373](https://issues.apache.org/jira/browse/HADOOP-7373) | Tarball deployment doesn't work with {start,stop}-{dfs,mapred} | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7364](https://issues.apache.org/jira/browse/HADOOP-7364) | TestMiniMRDFSCaching fails if test.build.dir is set to something other than build/test | Major | test | Thomas Graves | Thomas Graves | +| [HADOOP-7356](https://issues.apache.org/jira/browse/HADOOP-7356) | RPM packages broke bin/hadoop script for hadoop 0.20.205 | Blocker | . | Eric Yang | Eric Yang | +| [HADOOP-7330](https://issues.apache.org/jira/browse/HADOOP-7330) | The metrics source mbean implementation should return the attribute value instead of the object | Major | metrics | Luke Lu | Luke Lu | +| [HADOOP-7324](https://issues.apache.org/jira/browse/HADOOP-7324) | Ganglia plugins for metrics v2 | Blocker | metrics | Luke Lu | Priyo Mustafi | +| [HADOOP-7274](https://issues.apache.org/jira/browse/HADOOP-7274) | CLONE - IOUtils.readFully and IOUtils.skipFully have typo in exception creation's message | Minor | util | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7232](https://issues.apache.org/jira/browse/HADOOP-7232) | Fix javadoc warnings | Blocker | documentation | Owen O'Malley | Owen O'Malley | +| [HDFS-2057](https://issues.apache.org/jira/browse/HDFS-2057) | Wait time to terminate the threads causing unit tests to take longer time | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-2023](https://issues.apache.org/jira/browse/HDFS-2023) | Backport of NPE for File.list and File.listFiles | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1878](https://issues.apache.org/jira/browse/HDFS-1878) | TestHDFSServerPorts unit test failure - race condition in FSNamesystem.close() causes NullPointerException without serious consequence | Minor | namenode | Matt Foley | Matt Foley | +| [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1758](https://issues.apache.org/jira/browse/HDFS-1758) | Web UI JSP pages thread safety issue | Minor | tools | Tanping Wang | Tanping Wang | +| [HDFS-1750](https://issues.apache.org/jira/browse/HDFS-1750) | fs -ls hftp://file not working | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1692](https://issues.apache.org/jira/browse/HDFS-1692) | In secure mode, Datanode process doesn't exit when disks fail. | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1592](https://issues.apache.org/jira/browse/HDFS-1592) | Datanode startup doesn't honor volumes.tolerated | Major | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-1258](https://issues.apache.org/jira/browse/HDFS-1258) | Clearing namespace quota on "/" corrupts FS image | Blocker | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1189](https://issues.apache.org/jira/browse/HDFS-1189) | Quota counts missed between clear quota and set quota | Major | namenode | Kang Xiao | John George | +| [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | a small % of all tasks fail with DefaultTaskController | Blocker | task, task-controller, tasktracker | Allen Wittenauer | Owen O'Malley | +| [MAPREDUCE-2804](https://issues.apache.org/jira/browse/MAPREDUCE-2804) | "Creation of symlink to attempt log dir failed." message is not useful | Blocker | . | Allen Wittenauer | Owen O'Malley | +| [MAPREDUCE-2651](https://issues.apache.org/jira/browse/MAPREDUCE-2651) | Race condition in Linux Task Controller for job log directory creation | Major | task-controller | Bharath Mundlapudi | Bharath Mundlapudi | +| [MAPREDUCE-2621](https://issues.apache.org/jira/browse/MAPREDUCE-2621) | TestCapacityScheduler fails with "Queue "q1" does not exist" | Minor | . | Sherry Chen | Sherry Chen | +| [MAPREDUCE-2555](https://issues.apache.org/jira/browse/MAPREDUCE-2555) | JvmInvalidate errors in the gridmix TT logs | Minor | tasktracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2529](https://issues.apache.org/jira/browse/MAPREDUCE-2529) | Recognize Jetty bug 1342 and handle it | Major | tasktracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2514](https://issues.apache.org/jira/browse/MAPREDUCE-2514) | ReinitTrackerAction class name misspelled RenitTrackerAction in task tracker log | Trivial | tasktracker | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2451](https://issues.apache.org/jira/browse/MAPREDUCE-2451) | Log the reason string of healthcheck script | Trivial | jobtracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2447](https://issues.apache.org/jira/browse/MAPREDUCE-2447) | Set JvmContext sooner for a task - MR2429 | Minor | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2443](https://issues.apache.org/jira/browse/MAPREDUCE-2443) | Fix FI build - broken after MR-2429 | Minor | test | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2429](https://issues.apache.org/jira/browse/MAPREDUCE-2429) | Check jvmid during task status report | Major | tasktracker | Arun C Murthy | Siddharth Seth | +| [MAPREDUCE-2418](https://issues.apache.org/jira/browse/MAPREDUCE-2418) | Errors not shown in the JobHistory servlet (specifically Counter Limit Exceeded) | Minor | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2411](https://issues.apache.org/jira/browse/MAPREDUCE-2411) | When you submit a job to a queue with no ACLs you get an inscrutible NPE | Minor | . | Dick King | Dick King | +| [MAPREDUCE-2409](https://issues.apache.org/jira/browse/MAPREDUCE-2409) | Distributed Cache does not differentiate between file /archive for files with the same path | Major | distributed-cache | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2366](https://issues.apache.org/jira/browse/MAPREDUCE-2366) | TaskTracker can't retrieve stdout and stderr from web UI | Major | tasktracker | Owen O'Malley | Dick King | +| [MAPREDUCE-2364](https://issues.apache.org/jira/browse/MAPREDUCE-2364) | Shouldn't hold lock on rjob while localizing resources. | Major | tasktracker | Owen O'Malley | Devaraj Das | +| [MAPREDUCE-2362](https://issues.apache.org/jira/browse/MAPREDUCE-2362) | Unit test failures: TestBadRecords and TestTaskTrackerMemoryManager | Major | test | Owen O'Malley | Greg Roelofs | +| [MAPREDUCE-2360](https://issues.apache.org/jira/browse/MAPREDUCE-2360) | Pig fails when using non-default FileSystem | Major | client | Owen O'Malley | | +| [MAPREDUCE-2359](https://issues.apache.org/jira/browse/MAPREDUCE-2359) | Distributed cache doesn't use non-default FileSystems correctly | Major | . | Owen O'Malley | Krishna Ramachandran | +| [MAPREDUCE-2358](https://issues.apache.org/jira/browse/MAPREDUCE-2358) | MapReduce assumes HDFS as the default filesystem | Major | . | Owen O'Malley | Krishna Ramachandran | +| [MAPREDUCE-2357](https://issues.apache.org/jira/browse/MAPREDUCE-2357) | When extending inputsplit (non-FileSplit), all exceptions are ignored | Major | task | Owen O'Malley | Luke Lu | +| [MAPREDUCE-2356](https://issues.apache.org/jira/browse/MAPREDUCE-2356) | A task succeeded even though there were errors on all attempts. | Major | . | Owen O'Malley | Luke Lu | +| [MAPREDUCE-517](https://issues.apache.org/jira/browse/MAPREDUCE-517) | The capacity-scheduler should assign multiple tasks per heartbeat | Critical | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-118](https://issues.apache.org/jira/browse/MAPREDUCE-118) | Job.getJobID() will always return null | Blocker | client | Amar Kamat | Amareshwari Sriramadasu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2218](https://issues.apache.org/jira/browse/HDFS-2218) | Disable TestHdfsProxy.testHdfsProxyInterface in 0.20-security and branch-1 until HDFS-2217 is fixed | Blocker | contrib/hdfsproxy, test | Matt Foley | Matt Foley | +| [HDFS-2044](https://issues.apache.org/jira/browse/HDFS-2044) | TestQueueProcessingStatistics failing automatic test due to timing issues | Major | test | Matt Foley | Matt Foley | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-1767](https://issues.apache.org/jira/browse/HDFS-1767) | Namenode should ignore non-initial block reports from datanodes when in safemode during startup | Major | datanode | Matt Foley | Matt Foley | +| [HDFS-1541](https://issues.apache.org/jira/browse/HDFS-1541) | Not marking datanodes dead When namenode in safemode | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file | Major | datanode | Matt Foley | Matt Foley | +| [MAPREDUCE-2415](https://issues.apache.org/jira/browse/MAPREDUCE-2415) | Distribute TaskTracker userlogs onto multiple disks | Major | task-controller, tasktracker | Bharath Mundlapudi | Bharath Mundlapudi | +| [MAPREDUCE-2413](https://issues.apache.org/jira/browse/MAPREDUCE-2413) | TaskTracker should handle disk failures at both startup and runtime | Major | task-controller, tasktracker | Bharath Mundlapudi | Ravi Gummadi | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md new file mode 100644 index 00000000000..7a5f56093ba --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.0/RELEASENOTES.0.20.204.0.md @@ -0,0 +1,83 @@ + + +# Apache Hadoop 0.20.204.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | *Major* | **Create an rpm integration project** + +Added RPM/DEB packages to build system. + + +--- + +* [HDFS-2218](https://issues.apache.org/jira/browse/HDFS-2218) | *Blocker* | **Disable TestHdfsProxy.testHdfsProxyInterface in 0.20-security and branch-1 until HDFS-2217 is fixed** + +Test case TestHdfsProxy.testHdfsProxyInterface has been temporarily disabled for this release, due to failure in the Hudson automated test environment. + + +--- + +* [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | *Major* | **Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file** + +Batch hardlinking during "upgrade" snapshots, cutting time from aprx 8 minutes per volume to aprx 8 seconds. Validated in both Linux and Windows. Depends on prior integration with patch for HADOOP-7133. + + +--- + +* [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | *Blocker* | **a small % of all tasks fail with DefaultTaskController** + +Fixed a race condition in writing the log index file that caused tasks to 'fail'. + + +--- + +* [MAPREDUCE-2804](https://issues.apache.org/jira/browse/MAPREDUCE-2804) | *Blocker* | **"Creation of symlink to attempt log dir failed." message is not useful** + +Removed duplicate chmods of job log dir that were vulnerable to race conditions between tasks. Also improved the messages when the symlinks failed to be created. + + +--- + +* [MAPREDUCE-2529](https://issues.apache.org/jira/browse/MAPREDUCE-2529) | *Major* | **Recognize Jetty bug 1342 and handle it** + +Added 2 new config parameters: + +mapreduce.reduce.shuffle.catch.exception.stack.regex +mapreduce.reduce.shuffle.catch.exception.message.regex + + +--- + +* [MAPREDUCE-2524](https://issues.apache.org/jira/browse/MAPREDUCE-2524) | *Minor* | **Backport trunk heuristics for failing maps when we get fetch failures retrieving map output during shuffle** + +Added a new configuration option: mapreduce.reduce.shuffle.maxfetchfailures, and removed a no longer used option: mapred.reduce.copy.backoff. + + +--- + +* [MAPREDUCE-2479](https://issues.apache.org/jira/browse/MAPREDUCE-2479) | *Major* | **Backport MAPREDUCE-1568 to hadoop security branch** + +Added mapreduce.tasktracker.distributedcache.checkperiod to the task tracker that defined the period to wait while cleaning up the distributed cache. The default is 1 min. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md new file mode 100644 index 00000000000..56c3f856ae5 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/CHANGES.0.20.204.1.md @@ -0,0 +1,64 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.204.1 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md new file mode 100644 index 00000000000..db70b821405 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.204.1/RELEASENOTES.0.20.204.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.20.204.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/CHANGES.0.20.205.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/CHANGES.0.20.205.0.md new file mode 100644 index 00000000000..f032539a6ee --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/CHANGES.0.20.205.0.md @@ -0,0 +1,191 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.205.0 - 2011-10-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2202](https://issues.apache.org/jira/browse/HDFS-2202) | Changes to balancer bandwidth should not require datanode restart. | Major | balancer & mover, datanode | Eric Payne | Eric Payne | +| [HDFS-1554](https://issues.apache.org/jira/browse/HDFS-1554) | Append 0.20: New semantics for recoverLease | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-630](https://issues.apache.org/jira/browse/HDFS-630) | In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block. | Major | hdfs-client, namenode | Ruyue Ma | Cosmin Lehene | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7594](https://issues.apache.org/jira/browse/HADOOP-7594) | Support HTTP REST in HttpServer | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7119](https://issues.apache.org/jira/browse/HADOOP-7119) | add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-6889](https://issues.apache.org/jira/browse/HADOOP-6889) | Make RPC to have an option to timeout | Major | ipc | Hairong Kuang | John George | +| [HDFS-1520](https://issues.apache.org/jira/browse/HDFS-1520) | HDFS 20 append: Lightweight NameNode operation to trigger lease recovery | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-200](https://issues.apache.org/jira/browse/HDFS-200) | In HDFS, sync() not yet guarantees data available to the new readers | Blocker | . | Tsz Wo Nicholas Sze | dhruba borthakur | +| [MAPREDUCE-2777](https://issues.apache.org/jira/browse/MAPREDUCE-2777) | Backport MAPREDUCE-220 to Hadoop 20 security branch | Major | . | Jonathan Eagles | Amar Kamat | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7720](https://issues.apache.org/jira/browse/HADOOP-7720) | improve the hadoop-setup-conf.sh to read in the hbase user and setup the configs | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7707](https://issues.apache.org/jira/browse/HADOOP-7707) | improve config generator to allow users to specify proxy user, turn append on or off, turn webhdfs on or off | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7655](https://issues.apache.org/jira/browse/HADOOP-7655) | provide a small validation script that smoke tests the installed cluster | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-7472](https://issues.apache.org/jira/browse/HADOOP-7472) | RPC client should deal with the IP address changes | Minor | ipc | Kihwal Lee | Kihwal Lee | +| [HADOOP-7432](https://issues.apache.org/jira/browse/HADOOP-7432) | Back-port HADOOP-7110 to 0.20-security | Major | . | Sherry Chen | Sherry Chen | +| [HADOOP-7343](https://issues.apache.org/jira/browse/HADOOP-7343) | backport HADOOP-7008 and HADOOP-7042 to branch-0.20-security | Minor | test | Thomas Graves | Thomas Graves | +| [HADOOP-7314](https://issues.apache.org/jira/browse/HADOOP-7314) | Add support for throwing UnknownHostException when a host doesn't resolve | Major | . | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [HDFS-1555](https://issues.apache.org/jira/browse/HDFS-1555) | HDFS 20 append: Disallow pipeline recovery if a file is already being lease recovered | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-1211](https://issues.apache.org/jira/browse/HDFS-1211) | 0.20 append: Block receiver should not log "rewind" packets at INFO level | Minor | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1210](https://issues.apache.org/jira/browse/HDFS-1210) | DFSClient should log exception when block recovery fails | Trivial | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-1054](https://issues.apache.org/jira/browse/HDFS-1054) | Remove unnecessary sleep after failure in nextBlockOutputStream | Major | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-895](https://issues.apache.org/jira/browse/HDFS-895) | Allow hflush/sync to occur in parallel with new writes to the file | Major | hdfs-client | dhruba borthakur | Todd Lipcon | +| [HDFS-826](https://issues.apache.org/jira/browse/HDFS-826) | Allow a mechanism for an application to detect that datanode(s) have died in the write pipeline | Major | hdfs-client | dhruba borthakur | dhruba borthakur | +| [MAPREDUCE-2981](https://issues.apache.org/jira/browse/MAPREDUCE-2981) | Backport trunk fairscheduler to 0.20-security branch | Major | contrib/fair-share | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-2729](https://issues.apache.org/jira/browse/MAPREDUCE-2729) | Reducers are always counted having "pending tasks" even if they can't be scheduled yet because not enough of their mappers have completed | Major | . | Sherry Chen | Sherry Chen | +| [MAPREDUCE-2494](https://issues.apache.org/jira/browse/MAPREDUCE-2494) | Make the distributed cache delete entires using LRU priority | Major | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7724](https://issues.apache.org/jira/browse/HADOOP-7724) | hadoop-setup-conf.sh should put proxy user info into the core-site.xml | Major | . | Giridharan Kesavan | Arpit Gupta | +| [HADOOP-7721](https://issues.apache.org/jira/browse/HADOOP-7721) | dfs.web.authentication.kerberos.principal expects the full hostname and does not replace \_HOST with the hostname | Major | . | Arpit Gupta | Jitendra Nath Pandey | +| [HADOOP-7715](https://issues.apache.org/jira/browse/HADOOP-7715) | see log4j Error when running mr jobs and certain dfs calls | Major | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7711](https://issues.apache.org/jira/browse/HADOOP-7711) | hadoop-env.sh generated from templates has duplicate info | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7708](https://issues.apache.org/jira/browse/HADOOP-7708) | config generator does not update the properties file if on exists already | Critical | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7691](https://issues.apache.org/jira/browse/HADOOP-7691) | hadoop deb pkg should take a diff group id | Major | . | Giridharan Kesavan | Eric Yang | +| [HADOOP-7684](https://issues.apache.org/jira/browse/HADOOP-7684) | jobhistory server and secondarynamenode should have init.d script | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7683](https://issues.apache.org/jira/browse/HADOOP-7683) | hdfs-site.xml template has properties that are not used in 20 | Minor | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-7681](https://issues.apache.org/jira/browse/HADOOP-7681) | log4j.properties is missing properties for security audit and hdfs audit should be changed to info | Minor | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7679](https://issues.apache.org/jira/browse/HADOOP-7679) | log4j.properties templates does not define mapred.jobsummary.logger | Major | conf | Ramya Sunil | Ramya Sunil | +| [HADOOP-7674](https://issues.apache.org/jira/browse/HADOOP-7674) | TestKerberosName fails in 20 branch. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7658](https://issues.apache.org/jira/browse/HADOOP-7658) | to fix hadoop config template | Major | . | Giridharan Kesavan | Eric Yang | +| [HADOOP-7649](https://issues.apache.org/jira/browse/HADOOP-7649) | TestMapredGroupMappingServiceRefresh and TestRefreshUserMappings fail after HADOOP-7625 | Blocker | security, test | Kihwal Lee | Jitendra Nath Pandey | +| [HADOOP-7645](https://issues.apache.org/jira/browse/HADOOP-7645) | HTTP auth tests requiring Kerberos infrastructure are not disabled on branch-0.20-security | Blocker | security | Aaron T. Myers | Jitendra Nath Pandey | +| [HADOOP-7644](https://issues.apache.org/jira/browse/HADOOP-7644) | Fix the delegation token tests to use the new style renewers | Blocker | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-7637](https://issues.apache.org/jira/browse/HADOOP-7637) | Fair scheduler configuration file is not bundled in RPM | Major | build | Eric Yang | Eric Yang | +| [HADOOP-7633](https://issues.apache.org/jira/browse/HADOOP-7633) | log4j.properties should be added to the hadoop conf on deploy | Major | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7631](https://issues.apache.org/jira/browse/HADOOP-7631) | In mapred-site.xml, stream.tmpdir is mapped to ${mapred.temp.dir} which is undeclared. | Major | conf | Ramya Sunil | Eric Yang | +| [HADOOP-7630](https://issues.apache.org/jira/browse/HADOOP-7630) | hadoop-metrics2.properties should have a property \*.period set to a default value foe metrics | Major | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7626](https://issues.apache.org/jira/browse/HADOOP-7626) | Allow overwrite of HADOOP\_CLASSPATH and HADOOP\_OPTS | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7625](https://issues.apache.org/jira/browse/HADOOP-7625) | TestDelegationToken is failing in 205 | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7615](https://issues.apache.org/jira/browse/HADOOP-7615) | Binary layout does not put share/hadoop/contrib/\*.jar into the class path | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7610](https://issues.apache.org/jira/browse/HADOOP-7610) | /etc/profile.d does not exist on Debian | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7603](https://issues.apache.org/jira/browse/HADOOP-7603) | Set default hdfs, mapred uid, and hadoop group gid for RPM packages | Major | . | Eric Yang | Eric Yang | +| [HADOOP-7602](https://issues.apache.org/jira/browse/HADOOP-7602) | wordcount, sort etc on har files fails with NPE | Major | . | John George | John George | +| [HADOOP-7599](https://issues.apache.org/jira/browse/HADOOP-7599) | Improve hadoop setup conf script to setup secure Hadoop cluster | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7596](https://issues.apache.org/jira/browse/HADOOP-7596) | Enable jsvc to work with Hadoop RPM package | Major | build | Eric Yang | Eric Yang | +| [HADOOP-7539](https://issues.apache.org/jira/browse/HADOOP-7539) | merge hadoop archive goodness from trunk to .20 | Major | . | John George | John George | +| [HADOOP-7400](https://issues.apache.org/jira/browse/HADOOP-7400) | HdfsProxyTests fails when the -Dtest.build.dir and -Dbuild.test is set | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-6833](https://issues.apache.org/jira/browse/HADOOP-6833) | IPC leaks call parameters when exceptions thrown | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-6722](https://issues.apache.org/jira/browse/HADOOP-6722) | NetUtils.connect should check that it hasn't connected a socket to itself | Major | util | Todd Lipcon | Todd Lipcon | +| [HDFS-2411](https://issues.apache.org/jira/browse/HDFS-2411) | with webhdfs enabled in secure mode the auth to local mappings are not being respected. | Major | webhdfs | Arpit Gupta | Jitendra Nath Pandey | +| [HDFS-2408](https://issues.apache.org/jira/browse/HDFS-2408) | DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append | Blocker | hdfs-client | stack | stack | +| [HDFS-2405](https://issues.apache.org/jira/browse/HDFS-2405) | hadoop dfs command with webhdfs fails on secure hadoop | Critical | webhdfs | Arpit Gupta | Jitendra Nath Pandey | +| [HDFS-2392](https://issues.apache.org/jira/browse/HDFS-2392) | Dist with hftp is failing again | Critical | namenode | Rajit Saha | Daryn Sharp | +| [HDFS-2375](https://issues.apache.org/jira/browse/HDFS-2375) | TestFileAppend4 fails in 0.20.205 branch | Blocker | hdfs-client | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2373](https://issues.apache.org/jira/browse/HDFS-2373) | Commands using webhdfs and hftp print unnecessary debug information on the console with security enabled | Major | webhdfs | Arpit Gupta | Arpit Gupta | +| [HDFS-2368](https://issues.apache.org/jira/browse/HDFS-2368) | defaults created for web keytab and principal, these properties should not have defaults | Major | . | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2361](https://issues.apache.org/jira/browse/HDFS-2361) | hftp is broken | Critical | namenode | Rajit Saha | Jitendra Nath Pandey | +| [HDFS-2359](https://issues.apache.org/jira/browse/HDFS-2359) | NPE found in Datanode log while Disk failed during different HDFS operation | Major | datanode | Rajit Saha | Jonathan Eagles | +| [HDFS-2358](https://issues.apache.org/jira/browse/HDFS-2358) | NPE when the default filesystem's uri has no authority | Major | namenode | Rajit Saha | Daryn Sharp | +| [HDFS-2342](https://issues.apache.org/jira/browse/HDFS-2342) | TestSleepJob and TestHdfsProxy broken after HDFS-2284 | Blocker | build | Kihwal Lee | Tsz Wo Nicholas Sze | +| [HDFS-2333](https://issues.apache.org/jira/browse/HDFS-2333) | HDFS-2284 introduced 2 findbugs warnings on trunk | Major | . | Ivan Kelly | Tsz Wo Nicholas Sze | +| [HDFS-2331](https://issues.apache.org/jira/browse/HDFS-2331) | Hdfs compilation fails | Major | hdfs-client | Abhijit Suresh Shingate | Abhijit Suresh Shingate | +| [HDFS-2328](https://issues.apache.org/jira/browse/HDFS-2328) | hftp throws NPE if security is not enabled on remote cluster | Critical | . | Daryn Sharp | Owen O'Malley | +| [HDFS-2325](https://issues.apache.org/jira/browse/HDFS-2325) | Fuse-DFS fails to build on Hadoop 20.203.0 | Blocker | fuse-dfs, libhdfs | Charles Earl | Kihwal Lee | +| [HDFS-2320](https://issues.apache.org/jira/browse/HDFS-2320) | Make merged protocol changes from 0.20-append to 0.20-security compatible with previous releases. | Major | datanode, hdfs-client, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2309](https://issues.apache.org/jira/browse/HDFS-2309) | TestRenameWhileOpen fails in branch-0.20-security | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2300](https://issues.apache.org/jira/browse/HDFS-2300) | TestFileAppend4 and TestMultiThreadedSync fail on 20.append and 20-security. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2259](https://issues.apache.org/jira/browse/HDFS-2259) | DN web-UI doesn't work with paths that contain html | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-2190](https://issues.apache.org/jira/browse/HDFS-2190) | NN fails to start if it encounters an empty or malformed fstime file | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2117](https://issues.apache.org/jira/browse/HDFS-2117) | DiskChecker#mkdirsWithExistsAndPermissionCheck may return true even when the dir is not created | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-2053](https://issues.apache.org/jira/browse/HDFS-2053) | Bug in INodeDirectory#computeContentSummary warning | Minor | namenode | Michael Noll | Michael Noll | +| [HDFS-1836](https://issues.apache.org/jira/browse/HDFS-1836) | Thousand of CLOSE\_WAIT socket | Major | hdfs-client | Dennis Cheung | Bharath Mundlapudi | +| [HDFS-1779](https://issues.apache.org/jira/browse/HDFS-1779) | After NameNode restart , Clients can not read partial files even after client invokes Sync. | Major | datanode, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-1346](https://issues.apache.org/jira/browse/HDFS-1346) | DFSClient receives out of order packet ack | Major | datanode, hdfs-client | Hairong Kuang | Hairong Kuang | +| [HDFS-1260](https://issues.apache.org/jira/browse/HDFS-1260) | 0.20: Block lost when multiple DNs trying to recover it to different genstamps | Critical | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1218](https://issues.apache.org/jira/browse/HDFS-1218) | 20 append: Blocks recovered on startup should be treated with lower priority during block synchronization | Critical | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1207](https://issues.apache.org/jira/browse/HDFS-1207) | 0.20-append: stallReplicationWork should be volatile | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1204](https://issues.apache.org/jira/browse/HDFS-1204) | 0.20: Lease expiration should recover single files, not entire lease holder | Major | . | Todd Lipcon | sam rash | +| [HDFS-1202](https://issues.apache.org/jira/browse/HDFS-1202) | DataBlockScanner throws NPE when updated before initialized | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1197](https://issues.apache.org/jira/browse/HDFS-1197) | Blocks are considered "complete" prematurely after commitBlockSynchronization or DN restart | Major | datanode, hdfs-client, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1186](https://issues.apache.org/jira/browse/HDFS-1186) | 0.20: DNs should interrupt writers at start of recovery | Blocker | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1164](https://issues.apache.org/jira/browse/HDFS-1164) | TestHdfsProxy is failing | Major | contrib/hdfsproxy | Eli Collins | Todd Lipcon | +| [HDFS-1141](https://issues.apache.org/jira/browse/HDFS-1141) | completeFile does not check lease ownership | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1118](https://issues.apache.org/jira/browse/HDFS-1118) | DFSOutputStream socket leak when cannot connect to DataNode | Major | . | Zheng Shao | Zheng Shao | +| [HDFS-988](https://issues.apache.org/jira/browse/HDFS-988) | saveNamespace race can corrupt the edits log | Blocker | namenode | dhruba borthakur | Eli Collins | +| [HDFS-724](https://issues.apache.org/jira/browse/HDFS-724) | Pipeline close hangs if one of the datanode is not responsive. | Blocker | datanode, hdfs-client | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HDFS-606](https://issues.apache.org/jira/browse/HDFS-606) | ConcurrentModificationException in invalidateCorruptReplicas() | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-142](https://issues.apache.org/jira/browse/HDFS-142) | In 0.20, move blocks being written into a blocksBeingWritten directory | Blocker | . | Raghu Angadi | dhruba borthakur | +| [MAPREDUCE-3112](https://issues.apache.org/jira/browse/MAPREDUCE-3112) | Calling hadoop cli inside mapreduce job leads to errors | Major | contrib/streaming | Eric Yang | Eric Yang | +| [MAPREDUCE-3081](https://issues.apache.org/jira/browse/MAPREDUCE-3081) | Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh | Major | contrib/vaidya | vitthal (Suhas) Gogate | | +| [MAPREDUCE-3076](https://issues.apache.org/jira/browse/MAPREDUCE-3076) | TestSleepJob fails | Blocker | test | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2915](https://issues.apache.org/jira/browse/MAPREDUCE-2915) | LinuxTaskController does not work when JniBasedUnixGroupsNetgroupMapping or JniBasedUnixGroupsMapping is enabled | Major | task-controller | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-2852](https://issues.apache.org/jira/browse/MAPREDUCE-2852) | Jira for YDH bug 2854624 | Major | tasktracker | Eli Collins | Kihwal Lee | +| [MAPREDUCE-2764](https://issues.apache.org/jira/browse/MAPREDUCE-2764) | Fix renewal of dfs delegation tokens | Major | . | Daryn Sharp | Owen O'Malley | +| [MAPREDUCE-2705](https://issues.apache.org/jira/browse/MAPREDUCE-2705) | tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed | Major | tasktracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2650](https://issues.apache.org/jira/browse/MAPREDUCE-2650) | back-port MAPREDUCE-2238 to 0.20-security | Major | . | Sherry Chen | Sherry Chen | +| [MAPREDUCE-2610](https://issues.apache.org/jira/browse/MAPREDUCE-2610) | Inconsistent API JobClient.getQueueAclsForCurrentUser | Major | client | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-2549](https://issues.apache.org/jira/browse/MAPREDUCE-2549) | Potential resource leaks in HadoopServer.java, RunOnHadoopWizard.java and Environment.java | Major | contrib/eclipse-plugin, contrib/streaming | Devaraj K | Devaraj K | +| [MAPREDUCE-2489](https://issues.apache.org/jira/browse/MAPREDUCE-2489) | Jobsplits with random hostnames can make the queue unusable | Major | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2324](https://issues.apache.org/jira/browse/MAPREDUCE-2324) | Job should fail if a reduce task can't be scheduled anywhere | Major | . | Todd Lipcon | Robert Joseph Evans | +| [MAPREDUCE-2187](https://issues.apache.org/jira/browse/MAPREDUCE-2187) | map tasks timeout during sorting | Major | . | Gianmarco De Francisci Morales | Anupam Seth | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-1252](https://issues.apache.org/jira/browse/HDFS-1252) | TestDFSConcurrentFileOperations broken in 0.20-appendj | Major | test | Todd Lipcon | Todd Lipcon | +| [HDFS-1242](https://issues.apache.org/jira/browse/HDFS-1242) | 0.20 append: Add test for appendFile() race solved in HDFS-142 | Major | . | Todd Lipcon | Todd Lipcon | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2404](https://issues.apache.org/jira/browse/HDFS-2404) | webhdfs liststatus json response is not correct | Major | webhdfs | Arpit Gupta | Suresh Srinivas | +| [HDFS-2403](https://issues.apache.org/jira/browse/HDFS-2403) | The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2395](https://issues.apache.org/jira/browse/HDFS-2395) | webhdfs api's should return a root element in the json response | Critical | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2385](https://issues.apache.org/jira/browse/HDFS-2385) | Support delegation token renewal in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2366](https://issues.apache.org/jira/browse/HDFS-2366) | webhdfs throws a npe when ugi is null from getDelegationToken | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2356](https://issues.apache.org/jira/browse/HDFS-2356) | webhdfs: support case insensitive query parameter names | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2348](https://issues.apache.org/jira/browse/HDFS-2348) | Support getContentSummary and getFileChecksum in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2340](https://issues.apache.org/jira/browse/HDFS-2340) | Support getFileBlockLocations and getDelegationToken in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2338](https://issues.apache.org/jira/browse/HDFS-2338) | Configuration option to enable/disable webhdfs. | Major | webhdfs | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2318](https://issues.apache.org/jira/browse/HDFS-2318) | Provide authentication to webhdfs using SPNEGO | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2317](https://issues.apache.org/jira/browse/HDFS-2317) | Read access to HDFS using HTTP REST | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2284](https://issues.apache.org/jira/browse/HDFS-2284) | Write Http access to HDFS | Major | . | Sanjay Radia | Tsz Wo Nicholas Sze | +| [HDFS-1057](https://issues.apache.org/jira/browse/HDFS-1057) | Concurrent readers hit ChecksumExceptions if following a writer to very end of file | Blocker | datanode | Todd Lipcon | sam rash | +| [HDFS-561](https://issues.apache.org/jira/browse/HDFS-561) | Fix write pipeline READ\_TIMEOUT | Major | datanode, hdfs-client | Kan Zhang | Kan Zhang | +| [MAPREDUCE-2928](https://issues.apache.org/jira/browse/MAPREDUCE-2928) | MR-2413 improvements | Major | tasktracker | Eli Collins | Eli Collins | +| [MAPREDUCE-2780](https://issues.apache.org/jira/browse/MAPREDUCE-2780) | Standardize the value of token service | Major | . | Daryn Sharp | Daryn Sharp | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-1795](https://issues.apache.org/jira/browse/HDFS-1795) | Port 0.20-append changes onto 0.20-security-203 | Major | . | Andrew Purtell | | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/RELEASENOTES.0.20.205.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/RELEASENOTES.0.20.205.0.md new file mode 100644 index 00000000000..269401a2cfe --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.205.0/RELEASENOTES.0.20.205.0.md @@ -0,0 +1,195 @@ + + +# Apache Hadoop 0.20.205.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7724](https://issues.apache.org/jira/browse/HADOOP-7724) | *Major* | **hadoop-setup-conf.sh should put proxy user info into the core-site.xml** + +Fixed hadoop-setup-conf.sh to put proxy user in core-site.xml. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7720](https://issues.apache.org/jira/browse/HADOOP-7720) | *Major* | **improve the hadoop-setup-conf.sh to read in the hbase user and setup the configs** + +Added parameter for HBase user to setup config script. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7715](https://issues.apache.org/jira/browse/HADOOP-7715) | *Major* | **see log4j Error when running mr jobs and certain dfs calls** + +Removed unnecessary security logger configuration. (Eric Yang) + + +--- + +* [HADOOP-7711](https://issues.apache.org/jira/browse/HADOOP-7711) | *Major* | **hadoop-env.sh generated from templates has duplicate info** + +Fixed recursive sourcing of HADOOP\_OPTS environment variables (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7708](https://issues.apache.org/jira/browse/HADOOP-7708) | *Critical* | **config generator does not update the properties file if on exists already** + +Fixed hadoop-setup-conf.sh to handle config file consistently. (Eric Yang) + + +--- + +* [HADOOP-7707](https://issues.apache.org/jira/browse/HADOOP-7707) | *Major* | **improve config generator to allow users to specify proxy user, turn append on or off, turn webhdfs on or off** + +Added toggle for dfs.support.append, webhdfs and hadoop proxy user to setup config script. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7691](https://issues.apache.org/jira/browse/HADOOP-7691) | *Major* | **hadoop deb pkg should take a diff group id** + +Fixed conflict uid for install packages. (Eric Yang) + + +--- + +* [HADOOP-7684](https://issues.apache.org/jira/browse/HADOOP-7684) | *Major* | **jobhistory server and secondarynamenode should have init.d script** + +Added init.d script for jobhistory server and secondary namenode. (Eric Yang) + + +--- + +* [HADOOP-7681](https://issues.apache.org/jira/browse/HADOOP-7681) | *Minor* | **log4j.properties is missing properties for security audit and hdfs audit should be changed to info** + +HADOOP-7681. Fixed security and hdfs audit log4j properties +(Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7655](https://issues.apache.org/jira/browse/HADOOP-7655) | *Major* | **provide a small validation script that smoke tests the installed cluster** + +Committed to trunk and v23, since code reviewed by Eric. + + +--- + +* [HADOOP-7603](https://issues.apache.org/jira/browse/HADOOP-7603) | *Major* | **Set default hdfs, mapred uid, and hadoop group gid for RPM packages** + +Set hdfs uid, mapred uid, and hadoop gid to fixed numbers (201, 202, and 123, respectively). + + +--- + +* [HADOOP-7119](https://issues.apache.org/jira/browse/HADOOP-7119) | *Major* | **add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles** + +Adding support for Kerberos HTTP SPNEGO authentication to the Hadoop web-consoles + + +--- + +* [HDFS-2358](https://issues.apache.org/jira/browse/HDFS-2358) | *Major* | **NPE when the default filesystem's uri has no authority** + +Give meaningful error message instead of NPE. + + +--- + +* [HDFS-2338](https://issues.apache.org/jira/browse/HDFS-2338) | *Major* | **Configuration option to enable/disable webhdfs.** + +Added a conf property dfs.webhdfs.enabled for enabling/disabling webhdfs. + + +--- + +* [HDFS-2318](https://issues.apache.org/jira/browse/HDFS-2318) | *Major* | **Provide authentication to webhdfs using SPNEGO** + +Added two new conf properties dfs.web.authentication.kerberos.principal and dfs.web.authentication.kerberos.keytab for the SPNEGO servlet filter. + + +--- + +* [HDFS-2202](https://issues.apache.org/jira/browse/HDFS-2202) | *Major* | **Changes to balancer bandwidth should not require datanode restart.** + +New dfsadmin command added: [-setBalancerBandwidth \] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing. + +This is an incompatible change in 0.23. The versions of ClientProtocol and DatanodeProtocol are changed. + + +--- + +* [HDFS-1554](https://issues.apache.org/jira/browse/HDFS-1554) | *Major* | **Append 0.20: New semantics for recoverLease** + +Change recoverLease API to return if the file is closed or not. It also change the semantics of recoverLease to start lease recovery immediately. + + +--- + +* [HDFS-630](https://issues.apache.org/jira/browse/HDFS-630) | *Major* | **In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-3112](https://issues.apache.org/jira/browse/MAPREDUCE-3112) | *Major* | **Calling hadoop cli inside mapreduce job leads to errors** + +Removed inheritance of certain server environment variables (HADOOP\_OPTS and HADOOP\_ROOT\_LOGGER) in task attempt process. + + +--- + +* [MAPREDUCE-3081](https://issues.apache.org/jira/browse/MAPREDUCE-3081) | *Major* | **Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh** + +contrib/vaidya/bin/vaidya.sh script fixed to use appropriate jars and classpath + + +--- + +* [MAPREDUCE-2777](https://issues.apache.org/jira/browse/MAPREDUCE-2777) | *Major* | **Backport MAPREDUCE-220 to Hadoop 20 security branch** + +Adds cumulative cpu usage and total heap usage to task counters. This is a backport of MAPREDUCE-220 and MAPREDUCE-2469. + + +--- + +* [MAPREDUCE-2764](https://issues.apache.org/jira/browse/MAPREDUCE-2764) | *Major* | **Fix renewal of dfs delegation tokens** + +Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http. + + +--- + +* [MAPREDUCE-2494](https://issues.apache.org/jira/browse/MAPREDUCE-2494) | *Major* | **Make the distributed cache delete entires using LRU priority** + +Added config option mapreduce.tasktracker.cache.local.keep.pct to the TaskTracker. It is the target percentage of the local distributed cache that should be kept in between garbage collection runs. In practice it will delete unused distributed cache entries in LRU order until the size of the cache is less than mapreduce.tasktracker.cache.local.keep.pct of the maximum cache size. This is a floating point value between 0.0 and 1.0. The default is 0.95. + + +--- + +* [MAPREDUCE-2187](https://issues.apache.org/jira/browse/MAPREDUCE-2187) | *Major* | **map tasks timeout during sorting** + +I just committed this. Thanks Anupam! + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/CHANGES.0.20.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/CHANGES.0.20.3.md new file mode 100644 index 00000000000..1bac3d5522b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/CHANGES.0.20.3.md @@ -0,0 +1,103 @@ + + +# Apache Hadoop Changelog + +## Release 0.20.3 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6701](https://issues.apache.org/jira/browse/HADOOP-6701) | Incorrect exit codes for "dfs -chown", "dfs -chgrp" | Minor | fs | Ravi Phulari | Ravi Phulari | +| [HADOOP-6382](https://issues.apache.org/jira/browse/HADOOP-6382) | publish hadoop jars to apache mvn repo. | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HDFS-132](https://issues.apache.org/jira/browse/HDFS-132) | Namenode in Safemode reports to Simon non-zero number of deleted files during startup | Minor | namenode | Hairong Kuang | Suresh Srinivas | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7240](https://issues.apache.org/jira/browse/HADOOP-7240) | Update eclipse .classpath template | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-6882](https://issues.apache.org/jira/browse/HADOOP-6882) | Update the patch level of Jetty | Major | . | Owen O'Malley | Owen O'Malley | +| [HDFS-1013](https://issues.apache.org/jira/browse/HDFS-1013) | Miscellaneous improvements to HTML markup for web UIs | Minor | . | Todd Lipcon | Eugene Koontz | +| [MAPREDUCE-1832](https://issues.apache.org/jira/browse/MAPREDUCE-1832) | Support for file sizes less than 1MB in DFSIO benchmark. | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-1734](https://issues.apache.org/jira/browse/MAPREDUCE-1734) | Un-deprecate the old MapReduce API in the 0.20 branch | Blocker | documentation | Tom White | Todd Lipcon | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7116](https://issues.apache.org/jira/browse/HADOOP-7116) | raise contrib junit test jvm memory size to 512mb | Major | test | Owen O'Malley | Owen O'Malley | +| [HADOOP-6928](https://issues.apache.org/jira/browse/HADOOP-6928) | Fix BooleanWritable comparator in 0.20 | Major | io | Owen O'Malley | Johannes Zillmann | +| [HADOOP-6923](https://issues.apache.org/jira/browse/HADOOP-6923) | Native Libraries do not load if a different platform signature is returned from org.apache.hadoop.util.PlatformName | Major | native | Stephen Watt | Stephen Watt | +| [HADOOP-6881](https://issues.apache.org/jira/browse/HADOOP-6881) | The efficient comparators aren't always used except for BytesWritable and Text | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6833](https://issues.apache.org/jira/browse/HADOOP-6833) | IPC leaks call parameters when exceptions thrown | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-6760](https://issues.apache.org/jira/browse/HADOOP-6760) | WebServer shouldn't increase port number in case of negative port setting caused by Jetty's race | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6724](https://issues.apache.org/jira/browse/HADOOP-6724) | IPC doesn't properly handle IOEs thrown by socket factory | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-6702](https://issues.apache.org/jira/browse/HADOOP-6702) | Incorrect exit codes for "dfs -chown", "dfs -chgrp" when input is given in wildcard format. | Minor | fs | Ravi Phulari | Ravi Phulari | +| [HDFS-1836](https://issues.apache.org/jira/browse/HDFS-1836) | Thousand of CLOSE\_WAIT socket | Major | hdfs-client | Dennis Cheung | Bharath Mundlapudi | +| [HDFS-1543](https://issues.apache.org/jira/browse/HDFS-1543) | Reduce dev. cycle time by moving system testing artifacts from default build and push to maven for HDFS | Major | . | Arun C Murthy | Luke Lu | +| [HDFS-1406](https://issues.apache.org/jira/browse/HDFS-1406) | TestCLI fails on Ubuntu with default /etc/hosts | Minor | . | Todd Lipcon | Konstantin Boudnik | +| [HDFS-1404](https://issues.apache.org/jira/browse/HDFS-1404) | TestNodeCount logic incorrect in branch-0.20 | Minor | namenode, test | Todd Lipcon | Todd Lipcon | +| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-1258](https://issues.apache.org/jira/browse/HDFS-1258) | Clearing namespace quota on "/" corrupts FS image | Blocker | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1240](https://issues.apache.org/jira/browse/HDFS-1240) | TestDFSShell failing in branch-20 | Critical | test | Todd Lipcon | Todd Lipcon | +| [HDFS-1041](https://issues.apache.org/jira/browse/HDFS-1041) | DFSClient does not retry in getFileChecksum(..) | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-955](https://issues.apache.org/jira/browse/HDFS-955) | FSImage.saveFSImage can lose edits | Blocker | namenode | Todd Lipcon | Konstantin Shvachko | +| [HDFS-909](https://issues.apache.org/jira/browse/HDFS-909) | Race condition between rollEditLog or rollFSImage ant FSEditsLog.write operations corrupts edits log | Blocker | namenode | Cosmin Lehene | Todd Lipcon | +| [HDFS-908](https://issues.apache.org/jira/browse/HDFS-908) | TestDistributedFileSystem fails with Wrong FS on weird hosts | Minor | test | Todd Lipcon | Todd Lipcon | +| [HDFS-727](https://issues.apache.org/jira/browse/HDFS-727) | bug setting block size hdfsOpenFile | Blocker | libhdfs | Eli Collins | Eli Collins | +| [HDFS-15](https://issues.apache.org/jira/browse/HDFS-15) | Rack replication policy can be violated for over replicated blocks | Critical | . | Hairong Kuang | Jitendra Nath Pandey | +| [MAPREDUCE-2262](https://issues.apache.org/jira/browse/MAPREDUCE-2262) | Capacity Scheduler unit tests fail with class not found | Major | capacity-sched | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1880](https://issues.apache.org/jira/browse/MAPREDUCE-1880) | "java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result." while running "hadoop jar hadoop-0.20.1+169.89-examples.jar pi 4 30" | Minor | examples | Victor Pakhomov | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1522](https://issues.apache.org/jira/browse/MAPREDUCE-1522) | FileInputFormat may change the file system of an input path | Blocker | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1407](https://issues.apache.org/jira/browse/MAPREDUCE-1407) | Invalid example in the documentation of org.apache.hadoop.mapreduce.{Mapper,Reducer} | Trivial | documentation | Benoit Sigoure | Benoit Sigoure | +| [MAPREDUCE-1372](https://issues.apache.org/jira/browse/MAPREDUCE-1372) | ConcurrentModificationException in JobInProgress | Blocker | jobtracker | Amareshwari Sriramadasu | Dick King | +| [MAPREDUCE-1280](https://issues.apache.org/jira/browse/MAPREDUCE-1280) | Eclipse Plugin does not work with Eclipse Ganymede (3.4) | Major | . | Aaron Kimball | Alex Kozlov | +| [MAPREDUCE-118](https://issues.apache.org/jira/browse/MAPREDUCE-118) | Job.getJobID() will always return null | Blocker | client | Amar Kamat | Amareshwari Sriramadasu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6637](https://issues.apache.org/jira/browse/HADOOP-6637) | Benchmark overhead of RPC session establishment | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7372](https://issues.apache.org/jira/browse/HADOOP-7372) | Remove ref of 20.3 release from branch-0.20 CHANGES.txt | Major | documentation | Eli Collins | Eli Collins | +| [HDFS-1286](https://issues.apache.org/jira/browse/HDFS-1286) | Dry entropy pool on Hudson boxes causing test timeouts | Major | test | Todd Lipcon | Konstantin Boudnik | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/RELEASENOTES.0.20.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/RELEASENOTES.0.20.3.md new file mode 100644 index 00000000000..5e71d303751 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.20.3/RELEASENOTES.0.20.3.md @@ -0,0 +1,45 @@ + + +# Apache Hadoop 0.20.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-6701](https://issues.apache.org/jira/browse/HADOOP-6701) | *Minor* | ** Incorrect exit codes for "dfs -chown", "dfs -chgrp"** + +Commands chmod, chown and chgrp now returns non zero exit code and an error message on failure instead of returning zero. + + +--- + +* [HADOOP-6382](https://issues.apache.org/jira/browse/HADOOP-6382) | *Major* | **publish hadoop jars to apache mvn repo.** + +The hadoop jars are renamed from previous hadoop-\-\.jar to hadoop-\-\.jar. Applications and documentation need to be updated to use the new file naming scheme. + + +--- + +* [HDFS-132](https://issues.apache.org/jira/browse/HDFS-132) | *Minor* | **Namenode in Safemode reports to Simon non-zero number of deleted files during startup** + +With this incompatible change, under metrics context "dfs", the record name "FSDirectory" is no longer available. The metrics "files\_deleted" from the deleted record "FSDirectory" is now available in metrics context "dfs", record name "namenode" with the metrics name "FilesDeleted". + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/CHANGES.0.21.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/CHANGES.0.21.0.md new file mode 100644 index 00000000000..da203dcb4ad --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/CHANGES.0.21.0.md @@ -0,0 +1,1360 @@ + + +# Apache Hadoop Changelog + +## Release 0.21.0 - 2010-08-23 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6701](https://issues.apache.org/jira/browse/HADOOP-6701) | Incorrect exit codes for "dfs -chown", "dfs -chgrp" | Minor | fs | Ravi Phulari | Ravi Phulari | +| [HADOOP-6686](https://issues.apache.org/jira/browse/HADOOP-6686) | Remove redundant exception class name in unwrapped exceptions thrown at the RPC client | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6577](https://issues.apache.org/jira/browse/HADOOP-6577) | IPC server response buffer reset threshold should be configurable | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6569](https://issues.apache.org/jira/browse/HADOOP-6569) | FsShell#cat should avoid calling unecessary getFileStatus before opening a file to read | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-6367](https://issues.apache.org/jira/browse/HADOOP-6367) | Move Access Token implementation from Common to HDFS | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-6299](https://issues.apache.org/jira/browse/HADOOP-6299) | Use JAAS LoginContext for our login | Major | security | Arun C Murthy | Owen O'Malley | +| [HADOOP-6230](https://issues.apache.org/jira/browse/HADOOP-6230) | Move process tree, and memory calculator classes out of Common into Map/Reduce. | Major | util | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-6203](https://issues.apache.org/jira/browse/HADOOP-6203) | Improve error message when moving to trash fails due to quota issue | Major | fs | Jakob Homan | Boris Shkolnik | +| [HADOOP-6201](https://issues.apache.org/jira/browse/HADOOP-6201) | FileSystem::ListStatus should throw FileNotFoundException | Major | fs | Jakob Homan | Jakob Homan | +| [HADOOP-5913](https://issues.apache.org/jira/browse/HADOOP-5913) | Allow administrators to be able to start and stop queues | Major | . | rahul k singh | rahul k singh | +| [HADOOP-5879](https://issues.apache.org/jira/browse/HADOOP-5879) | GzipCodec should read compression level etc from configuration | Major | io | Zheng Shao | He Yongqiang | +| [HADOOP-5861](https://issues.apache.org/jira/browse/HADOOP-5861) | s3n files are not getting split by default | Major | fs/s3 | Joydeep Sen Sarma | Tom White | +| [HADOOP-5738](https://issues.apache.org/jira/browse/HADOOP-5738) | Split waiting tasks field in JobTracker metrics to individual tasks | Major | metrics | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5679](https://issues.apache.org/jira/browse/HADOOP-5679) | Resolve findbugs warnings in core/streaming/pipes/examples | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5620](https://issues.apache.org/jira/browse/HADOOP-5620) | discp can preserve modification times of files | Major | . | dhruba borthakur | Rodrigo Schmidt | +| [HADOOP-5485](https://issues.apache.org/jira/browse/HADOOP-5485) | Authorisation machanism required for acceesing jobtracker url :- jobtracker.com:port/scheduler | Major | . | Aroop Maliakkal | Vinod Kumar Vavilapalli | +| [HADOOP-5464](https://issues.apache.org/jira/browse/HADOOP-5464) | DFSClient does not treat write timeout of 0 properly | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5438](https://issues.apache.org/jira/browse/HADOOP-5438) | Merge FileSystem.create and FileSystem.append | Major | fs | He Yongqiang | He Yongqiang | +| [HADOOP-5258](https://issues.apache.org/jira/browse/HADOOP-5258) | Provide dfsadmin functionality to report on namenode's view of network topology | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-5219](https://issues.apache.org/jira/browse/HADOOP-5219) | SequenceFile is using mapred property | Major | io | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5176](https://issues.apache.org/jira/browse/HADOOP-5176) | TestDFSIO reports itself as TestFDSIO | Trivial | benchmarks | Bryan Duxbury | Ravi Phulari | +| [HADOOP-5094](https://issues.apache.org/jira/browse/HADOOP-5094) | Show dead nodes information in dfsadmin -report | Minor | . | Jim Huang | Jakob Homan | +| [HADOOP-5022](https://issues.apache.org/jira/browse/HADOOP-5022) | [HOD] logcondense should delete all hod logs for a user, including jobtracker logs | Blocker | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi | +| [HADOOP-4942](https://issues.apache.org/jira/browse/HADOOP-4942) | Remove getName() and getNamed(String name, Configuration conf) | Major | fs | Tsz Wo Nicholas Sze | Jakob Homan | +| [HADOOP-4941](https://issues.apache.org/jira/browse/HADOOP-4941) | Remove getBlockSize(Path f), getLength(Path f) and getReplication(Path src) | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4940](https://issues.apache.org/jira/browse/HADOOP-4940) | Remove delete(Path f) | Major | fs | Tsz Wo Nicholas Sze | Enis Soztutar | +| [HADOOP-4895](https://issues.apache.org/jira/browse/HADOOP-4895) | Remove deprecated methods in DFSClient | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4861](https://issues.apache.org/jira/browse/HADOOP-4861) | Add disk usage with human-readable size (-duh) | Trivial | . | Bryan Duxbury | Todd Lipcon | +| [HADOOP-4779](https://issues.apache.org/jira/browse/HADOOP-4779) | Remove deprecated FileSystem methods | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4756](https://issues.apache.org/jira/browse/HADOOP-4756) | Create a command line tool to access JMX exported properties from a NameNode server | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4655](https://issues.apache.org/jira/browse/HADOOP-4655) | FileSystem.CACHE should be ref-counted | Major | fs | Hong Tang | dhruba borthakur | +| [HADOOP-4648](https://issues.apache.org/jira/browse/HADOOP-4648) | Remove ChecksumDistriubtedFileSystem and InMemoryFileSystem | Major | fs | Owen O'Malley | Chris Douglas | +| [HADOOP-4539](https://issues.apache.org/jira/browse/HADOOP-4539) | Streaming Edits to a Backup Node. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4268](https://issues.apache.org/jira/browse/HADOOP-4268) | Permission checking in fsck | Major | . | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HADOOP-3953](https://issues.apache.org/jira/browse/HADOOP-3953) | Sticky bit for directories | Major | . | Koji Noguchi | Jakob Homan | +| [HADOOP-2827](https://issues.apache.org/jira/browse/HADOOP-2827) | Remove deprecated NetUtils.getServerAddress | Major | conf, util | dhruba borthakur | Chris Douglas | +| [HDFS-1024](https://issues.apache.org/jira/browse/HDFS-1024) | SecondaryNamenode fails to checkpoint because namenode fails with CancelledKeyException | Blocker | . | dhruba borthakur | Dmytro Molkov | +| [HDFS-1016](https://issues.apache.org/jira/browse/HDFS-1016) | HDFS side change for HADOOP-6569 | Major | hdfs-client | Hairong Kuang | Hairong Kuang | +| [HDFS-985](https://issues.apache.org/jira/browse/HDFS-985) | HDFS should issue multiple RPCs for listing a large directory | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-984](https://issues.apache.org/jira/browse/HDFS-984) | Delegation Tokens should be persisted in Namenode | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-946](https://issues.apache.org/jira/browse/HDFS-946) | NameNode should not return full path name when lisitng a diretory or getting the status of a file | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-892](https://issues.apache.org/jira/browse/HDFS-892) | optionally use Avro for namenode RPC | Major | namenode | Doug Cutting | Doug Cutting | +| [HDFS-873](https://issues.apache.org/jira/browse/HDFS-873) | DataNode directories as URIs | Major | datanode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-793](https://issues.apache.org/jira/browse/HDFS-793) | DataNode should first receive the whole packet ack message before it constructs and sends its own ack message for the packet | Blocker | datanode | Hairong Kuang | Hairong Kuang | +| [HDFS-764](https://issues.apache.org/jira/browse/HDFS-764) | Moving Access Token implementation from Common to HDFS | Major | . | Kan Zhang | Kan Zhang | +| [HDFS-737](https://issues.apache.org/jira/browse/HDFS-737) | Improvement in metasave output | Major | namenode | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-660](https://issues.apache.org/jira/browse/HDFS-660) | Remove deprecated methods from InterDatanodeProtocol. | Major | datanode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-630](https://issues.apache.org/jira/browse/HDFS-630) | In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block. | Major | hdfs-client, namenode | Ruyue Ma | Cosmin Lehene | +| [HDFS-618](https://issues.apache.org/jira/browse/HDFS-618) | Support for non-recursive mkdir in HDFS | Major | hdfs-client, namenode | Kan Zhang | Kan Zhang | +| [HDFS-617](https://issues.apache.org/jira/browse/HDFS-617) | Support for non-recursive create() in HDFS | Major | hdfs-client, namenode | Kan Zhang | Kan Zhang | +| [HDFS-602](https://issues.apache.org/jira/browse/HDFS-602) | Atempt to make a directory under an existing file on DistributedFileSystem should throw an FileAlreadyExistsException instead of FileNotFoundException | Major | hdfs-client, namenode | Boris Shkolnik | Boris Shkolnik | +| [HDFS-538](https://issues.apache.org/jira/browse/HDFS-538) | DistributedFileSystem::listStatus incorrectly returns null for empty result sets | Major | . | Jakob Homan | Jakob Homan | +| [HDFS-514](https://issues.apache.org/jira/browse/HDFS-514) | DFSClient.namenode is a public field. Should be private. | Major | hdfs-client | Bill Zeller | Bill Zeller | +| [HDFS-512](https://issues.apache.org/jira/browse/HDFS-512) | Set block id as the key to Block | Major | . | Hairong Kuang | Konstantin Shvachko | +| [HDFS-288](https://issues.apache.org/jira/browse/HDFS-288) | Redundant computation in hashCode() implemenation | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1607](https://issues.apache.org/jira/browse/MAPREDUCE-1607) | Task controller may not set permissions for a task cleanup attempt's log directory | Major | task-controller | Hemanth Yamijala | Amareshwari Sriramadasu | +| [MAPREDUCE-1493](https://issues.apache.org/jira/browse/MAPREDUCE-1493) | Authorization for job-history pages | Major | jobtracker, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1385](https://issues.apache.org/jira/browse/MAPREDUCE-1385) | Make changes to MapReduce for the new UserGroupInformation APIs (HADOOP-6299) | Major | . | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1307](https://issues.apache.org/jira/browse/MAPREDUCE-1307) | Introduce the concept of Job Permissions | Major | security | Devaraj Das | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1287](https://issues.apache.org/jira/browse/MAPREDUCE-1287) | Avoid calling Partitioner with only 1 reducer | Minor | . | Ed Mazur | Chris Douglas | +| [MAPREDUCE-1218](https://issues.apache.org/jira/browse/MAPREDUCE-1218) | Collecting cpu and memory usage for TaskTrackers | Major | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1213](https://issues.apache.org/jira/browse/MAPREDUCE-1213) | TaskTrackers restart is very slow because it deletes distributed cache directory synchronously | Major | . | dhruba borthakur | Zheng Shao | +| [MAPREDUCE-1097](https://issues.apache.org/jira/browse/MAPREDUCE-1097) | Changes/fixes to support Vertica 3.5 | Minor | contrib/vertica | Omer Trajman | Omer Trajman | +| [MAPREDUCE-967](https://issues.apache.org/jira/browse/MAPREDUCE-967) | TaskTracker does not need to fully unjar job jars | Major | tasktracker | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-963](https://issues.apache.org/jira/browse/MAPREDUCE-963) | mapred's FileAlreadyExistsException should be deprecated in favor of hadoop-common's one. | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-899](https://issues.apache.org/jira/browse/MAPREDUCE-899) | When using LinuxTaskController, localized files may become accessible to unintended users if permissions are misconfigured. | Major | tasktracker | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [MAPREDUCE-895](https://issues.apache.org/jira/browse/MAPREDUCE-895) | FileSystem::ListStatus will now throw FileNotFoundException, MapRed needs updated | Major | . | Jakob Homan | Jakob Homan | +| [MAPREDUCE-873](https://issues.apache.org/jira/browse/MAPREDUCE-873) | Simplify Job Recovery | Major | jobtracker | Devaraj Das | Sharad Agarwal | +| [MAPREDUCE-862](https://issues.apache.org/jira/browse/MAPREDUCE-862) | Modify UI to support a hierarchy of queues | Major | . | Hemanth Yamijala | V.V.Chaitanya Krishna | +| [MAPREDUCE-849](https://issues.apache.org/jira/browse/MAPREDUCE-849) | Renaming of configuration property names in mapreduce | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-817](https://issues.apache.org/jira/browse/MAPREDUCE-817) | Add a cache for retired jobs with minimal job info and provide a way to access history file url | Major | client, jobtracker | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-777](https://issues.apache.org/jira/browse/MAPREDUCE-777) | A method for finding and tracking jobs from the new API | Major | client | Owen O'Malley | Amareshwari Sriramadasu | +| [MAPREDUCE-772](https://issues.apache.org/jira/browse/MAPREDUCE-772) | Chaging LineRecordReader algo so that it does not need to skip backwards in the stream | Major | . | Abdul Qadeer | Abdul Qadeer | +| [MAPREDUCE-766](https://issues.apache.org/jira/browse/MAPREDUCE-766) | Enhance -list-blacklisted-trackers to display host name, blacklisted reason and blacklist report. | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [MAPREDUCE-711](https://issues.apache.org/jira/browse/MAPREDUCE-711) | Move Distributed Cache from Common to Map/Reduce | Major | . | Owen O'Malley | Vinod Kumar Vavilapalli | +| [MAPREDUCE-516](https://issues.apache.org/jira/browse/MAPREDUCE-516) | Fix the 'cluster drain' problem in the Capacity Scheduler wrt High RAM Jobs | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-479](https://issues.apache.org/jira/browse/MAPREDUCE-479) | Add reduce ID to shuffle clienttrace | Minor | . | Jiaqi Tan | Jiaqi Tan | +| [MAPREDUCE-358](https://issues.apache.org/jira/browse/MAPREDUCE-358) | Change org.apache.hadoop.examples. AggregateWordCount and org.apache.hadoop.examples.AggregateWordHistogram to use new mapreduce api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-355](https://issues.apache.org/jira/browse/MAPREDUCE-355) | Change org.apache.hadoop.mapred.join to use new api | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-245](https://issues.apache.org/jira/browse/MAPREDUCE-245) | Job and JobControl classes should return interfaces rather than implementations | Major | . | Tom White | Tom White | +| [MAPREDUCE-157](https://issues.apache.org/jira/browse/MAPREDUCE-157) | Job History log file format is not friendly for external tools. | Major | . | Owen O'Malley | Jothi Padmanabhan | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6869](https://issues.apache.org/jira/browse/HADOOP-6869) | Functionality to create file or folder on a remote daemon side | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6594](https://issues.apache.org/jira/browse/HADOOP-6594) | Update hdfs script to provide fetchdt tool | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-6580](https://issues.apache.org/jira/browse/HADOOP-6580) | UGI should contain authentication method. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6573](https://issues.apache.org/jira/browse/HADOOP-6573) | Delegation Tokens should be persisted. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6566](https://issues.apache.org/jira/browse/HADOOP-6566) | Hadoop daemons should not start up if the ownership/permissions on the directories used at runtime are misconfigured | Major | security | Devaraj Das | Arun C Murthy | +| [HADOOP-6547](https://issues.apache.org/jira/browse/HADOOP-6547) | Move the Delegation Token feature to common since both HDFS and MapReduce needs it | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6517](https://issues.apache.org/jira/browse/HADOOP-6517) | Ability to add/get tokens from UserGroupInformation | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6510](https://issues.apache.org/jira/browse/HADOOP-6510) | doAs for proxy user | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6497](https://issues.apache.org/jira/browse/HADOOP-6497) | Introduce wrapper around FSDataInputStream providing Avro SeekableInput interface | Major | fs | Aaron Kimball | Aaron Kimball | +| [HADOOP-6433](https://issues.apache.org/jira/browse/HADOOP-6433) | Add AsyncDiskService that is used in both hdfs and mapreduce | Major | . | Zheng Shao | Zheng Shao | +| [HADOOP-6419](https://issues.apache.org/jira/browse/HADOOP-6419) | Change RPC layer to support SASL based mutual authentication | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-6415](https://issues.apache.org/jira/browse/HADOOP-6415) | Adding a common token interface for both job token and delegation token | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-6408](https://issues.apache.org/jira/browse/HADOOP-6408) | Add a /conf servlet to dump running configuration | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-6337](https://issues.apache.org/jira/browse/HADOOP-6337) | Update FilterInitializer class to be more visible and take a conf for further development | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-6332](https://issues.apache.org/jira/browse/HADOOP-6332) | Large-scale Automated Test Framework | Major | test | Arun C Murthy | Konstantin Boudnik | +| [HADOOP-6323](https://issues.apache.org/jira/browse/HADOOP-6323) | Serialization should provide comparators | Major | io | Doug Cutting | Aaron Kimball | +| [HADOOP-6313](https://issues.apache.org/jira/browse/HADOOP-6313) | Expose flush APIs to application users | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-6270](https://issues.apache.org/jira/browse/HADOOP-6270) | FileContext needs to provide deleteOnExit functionality | Major | fs | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6235](https://issues.apache.org/jira/browse/HADOOP-6235) | Adding a new method for getting server default values from a FileSystem | Major | fs | Kan Zhang | Kan Zhang | +| [HADOOP-6226](https://issues.apache.org/jira/browse/HADOOP-6226) | Create a LimitedByteArrayOutputStream that does not expand its buffer on write | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-6218](https://issues.apache.org/jira/browse/HADOOP-6218) | Split TFile by Record Sequence Number | Major | . | Hong Tang | Hong Tang | +| [HADOOP-6173](https://issues.apache.org/jira/browse/HADOOP-6173) | src/native/packageNativeHadoop.sh only packages files with "hadoop" in the name | Minor | build, scripts | Hong Tang | Hong Tang | +| [HADOOP-6165](https://issues.apache.org/jira/browse/HADOOP-6165) | Add metadata to Serializations | Blocker | contrib/serialization | Tom White | Tom White | +| [HADOOP-6120](https://issues.apache.org/jira/browse/HADOOP-6120) | Add support for Avro types in hadoop | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5887](https://issues.apache.org/jira/browse/HADOOP-5887) | Sqoop should create tables in Hive metastore after importing to HDFS | Major | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-5844](https://issues.apache.org/jira/browse/HADOOP-5844) | Use mysqldump when connecting to local mysql instance in Sqoop | Major | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-5815](https://issues.apache.org/jira/browse/HADOOP-5815) | Sqoop: A database import tool for Hadoop | Major | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-5752](https://issues.apache.org/jira/browse/HADOOP-5752) | Provide examples of using offline image viewer (oiv) to analyze hadoop file systems | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-5745](https://issues.apache.org/jira/browse/HADOOP-5745) | Allow setting the default value of maxRunningJobs for all pools | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-5643](https://issues.apache.org/jira/browse/HADOOP-5643) | Ability to blacklist tasktracker | Major | . | Rajiv Chittajallu | Amar Kamat | +| [HADOOP-5528](https://issues.apache.org/jira/browse/HADOOP-5528) | Binary partitioner | Major | . | Klaas Bosteels | Klaas Bosteels | +| [HADOOP-5518](https://issues.apache.org/jira/browse/HADOOP-5518) | MRUnit unit test library | Major | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-5469](https://issues.apache.org/jira/browse/HADOOP-5469) | Exposing Hadoop metrics via HTTP | Major | metrics | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-5467](https://issues.apache.org/jira/browse/HADOOP-5467) | Create an offline fsimage image viewer | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-5366](https://issues.apache.org/jira/browse/HADOOP-5366) | Support for retrieving files using standard HTTP clients like curl | Major | . | Kan Zhang | zhiyong zhang | +| [HADOOP-5363](https://issues.apache.org/jira/browse/HADOOP-5363) | Proxying for multiple HDFS clusters of different versions | Major | . | Kan Zhang | zhiyong zhang | +| [HADOOP-5257](https://issues.apache.org/jira/browse/HADOOP-5257) | Export namenode/datanode functionality through a pluggable RPC layer | Minor | . | Carlos Valiente | Carlos Valiente | +| [HADOOP-5232](https://issues.apache.org/jira/browse/HADOOP-5232) | preparing HadoopPatchQueueAdmin.sh,test-patch.sh scripts to run builds on hudson slaves. | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5175](https://issues.apache.org/jira/browse/HADOOP-5175) | Option to prohibit jars unpacking | Major | . | Andrew Gudkov | Todd Lipcon | +| [HADOOP-5052](https://issues.apache.org/jira/browse/HADOOP-5052) | Add an example for computing exact digits of Pi | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5042](https://issues.apache.org/jira/browse/HADOOP-5042) | Add expiration handling to the chukwa log4j appender | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5018](https://issues.apache.org/jira/browse/HADOOP-5018) | Chukwa should support pipelined writers | Major | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-4952](https://issues.apache.org/jira/browse/HADOOP-4952) | Improved files system interface for the application writer. | Major | fs | Sanjay Radia | Sanjay Radia | +| [HADOOP-4927](https://issues.apache.org/jira/browse/HADOOP-4927) | Part files on the output filesystem are created irrespective of whether the corresponding task has anything to write there | Major | . | Devaraj Das | Jothi Padmanabhan | +| [HADOOP-4829](https://issues.apache.org/jira/browse/HADOOP-4829) | Allow FileSystem shutdown hook to be disabled | Minor | fs | Bryan Duxbury | Todd Lipcon | +| [HADOOP-4768](https://issues.apache.org/jira/browse/HADOOP-4768) | Dynamic Priority Scheduler that allows queue shares to be controlled dynamically by a currency | Major | . | Thomas Sandholm | Thomas Sandholm | +| [HADOOP-4368](https://issues.apache.org/jira/browse/HADOOP-4368) | Superuser privileges required to do "df" | Minor | . | Brian Bockelman | Craig Macdonald | +| [HADOOP-4359](https://issues.apache.org/jira/browse/HADOOP-4359) | Access Token: Support for data access authorization checking on DataNodes | Major | . | Kan Zhang | Kan Zhang | +| [HADOOP-4012](https://issues.apache.org/jira/browse/HADOOP-4012) | Providing splitting support for bzip2 compressed files | Major | io | Abdul Qadeer | Abdul Qadeer | +| [HADOOP-3741](https://issues.apache.org/jira/browse/HADOOP-3741) | SecondaryNameNode has http server on dfs.secondary.http.address but without any contents | Major | . | Lohit Vijayarenu | Tsz Wo Nicholas Sze | +| [HDFS-2137](https://issues.apache.org/jira/browse/HDFS-2137) | Datanode Disk Fail Inplace | Major | datanode | Bharath Mundlapudi | | +| [HDFS-1091](https://issues.apache.org/jira/browse/HDFS-1091) | Implement listStatus that returns an Iterator of FileStatus | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-999](https://issues.apache.org/jira/browse/HDFS-999) | Secondary namenode should login using kerberos if security is configured | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-993](https://issues.apache.org/jira/browse/HDFS-993) | Namenode should issue a delegation token only for kerberos authenticated clients. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-991](https://issues.apache.org/jira/browse/HDFS-991) | Allow browsing the filesystem over http using delegation tokens | Major | . | Owen O'Malley | Owen O'Malley | +| [HDFS-935](https://issues.apache.org/jira/browse/HDFS-935) | Real user in delegation token. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-905](https://issues.apache.org/jira/browse/HDFS-905) | Make changes to HDFS for the new UserGroupInformation APIs (HADOOP-6299) | Major | . | Devaraj Das | Jakob Homan | +| [HDFS-814](https://issues.apache.org/jira/browse/HDFS-814) | Add an api to get the visible length of a DFSDataInputStream. | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-731](https://issues.apache.org/jira/browse/HDFS-731) | Support new Syncable interface in HDFS | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-702](https://issues.apache.org/jira/browse/HDFS-702) | Add Hdfs Impl for the new file system interface | Major | namenode | Sanjay Radia | Sanjay Radia | +| [HDFS-654](https://issues.apache.org/jira/browse/HDFS-654) | HDFS needs to support new rename introduced for FileContext | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-610](https://issues.apache.org/jira/browse/HDFS-610) | Add support for FileContext | Major | hdfs-client, namenode | Sanjay Radia | Sanjay Radia | +| [HDFS-595](https://issues.apache.org/jira/browse/HDFS-595) | FsPermission tests need to be updated for new octal configuration parameter from HADOOP-6234 | Major | hdfs-client | Jakob Homan | Jakob Homan | +| [HDFS-567](https://issues.apache.org/jira/browse/HDFS-567) | Two contrib tools to facilitate searching for block history information | Major | tools | Bill Zeller | Jitendra Nath Pandey | +| [HDFS-503](https://issues.apache.org/jira/browse/HDFS-503) | Implement erasure coding as a layer on HDFS | Major | contrib/raid | dhruba borthakur | dhruba borthakur | +| [HDFS-492](https://issues.apache.org/jira/browse/HDFS-492) | Expose corrupt replica/block information | Major | namenode | Bill Zeller | Bill Zeller | +| [HDFS-461](https://issues.apache.org/jira/browse/HDFS-461) | Analyzing file size distribution. | Major | test, tools | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-459](https://issues.apache.org/jira/browse/HDFS-459) | Job History Log Analyzer | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-458](https://issues.apache.org/jira/browse/HDFS-458) | Create target for 10 minute patch test build for hdfs | Major | build, test | Jakob Homan | Jakob Homan | +| [HDFS-447](https://issues.apache.org/jira/browse/HDFS-447) | proxy to call LDAP for IP lookup and get user ID and directories, validate requested URL | Critical | contrib/hdfsproxy | zhiyong zhang | zhiyong zhang | +| [HDFS-245](https://issues.apache.org/jira/browse/HDFS-245) | Create symbolic links in HDFS | Major | . | dhruba borthakur | Eli Collins | +| [HDFS-235](https://issues.apache.org/jira/browse/HDFS-235) | Add support for byte-ranges to hftp | Major | . | Venkatesh Seetharam | Bill Zeller | +| [HDFS-222](https://issues.apache.org/jira/browse/HDFS-222) | Support for concatenating of files into a single file | Major | . | Venkatesh Seetharam | Boris Shkolnik | +| [HDFS-204](https://issues.apache.org/jira/browse/HDFS-204) | Revive number of files listed metrics | Major | namenode | Koji Noguchi | Jitendra Nath Pandey | +| [MAPREDUCE-1774](https://issues.apache.org/jira/browse/MAPREDUCE-1774) | Large-scale Automated Framework | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1673](https://issues.apache.org/jira/browse/MAPREDUCE-1673) | Start and Stop scripts for the RaidNode | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1464](https://issues.apache.org/jira/browse/MAPREDUCE-1464) | In JobTokenIdentifier change method getUsername to getUser which returns UGI | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-1383](https://issues.apache.org/jira/browse/MAPREDUCE-1383) | Allow storage and caching of delegation token. | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1338](https://issues.apache.org/jira/browse/MAPREDUCE-1338) | need security keys storage solution | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1335](https://issues.apache.org/jira/browse/MAPREDUCE-1335) | Add SASL DIGEST-MD5 authentication to TaskUmbilicalProtocol | Major | . | Kan Zhang | Kan Zhang | +| [MAPREDUCE-1304](https://issues.apache.org/jira/browse/MAPREDUCE-1304) | Add counters for task time spent in GC | Major | task | Todd Lipcon | Aaron Kimball | +| [MAPREDUCE-1295](https://issues.apache.org/jira/browse/MAPREDUCE-1295) | We need a job trace manipulator to build gridmix runs. | Major | tools/rumen | Dick King | Dick King | +| [MAPREDUCE-1167](https://issues.apache.org/jira/browse/MAPREDUCE-1167) | Make ProcfsBasedProcessTree collect rss memory information | Major | tasktracker | Scott Chen | Scott Chen | +| [MAPREDUCE-1074](https://issues.apache.org/jira/browse/MAPREDUCE-1074) | Provide documentation for Mark/Reset functionality | Major | documentation | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-980](https://issues.apache.org/jira/browse/MAPREDUCE-980) | Modify JobHistory to use Avro for serialization instead of raw JSON | Major | . | Jothi Padmanabhan | Doug Cutting | +| [MAPREDUCE-853](https://issues.apache.org/jira/browse/MAPREDUCE-853) | Support a hierarchy of queues in the Map/Reduce framework | Major | jobtracker | Hemanth Yamijala | | +| [MAPREDUCE-824](https://issues.apache.org/jira/browse/MAPREDUCE-824) | Support a hierarchy of queues in the capacity scheduler | Major | capacity-sched | Hemanth Yamijala | rahul k singh | +| [MAPREDUCE-800](https://issues.apache.org/jira/browse/MAPREDUCE-800) | MRUnit should support the new API | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-798](https://issues.apache.org/jira/browse/MAPREDUCE-798) | MRUnit should be able to test a succession of MapReduce passes | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-776](https://issues.apache.org/jira/browse/MAPREDUCE-776) | Gridmix: Trace-based benchmark for Map/Reduce | Major | benchmarks | Chris Douglas | Chris Douglas | +| [MAPREDUCE-775](https://issues.apache.org/jira/browse/MAPREDUCE-775) | Add input/output formatters for Vertica clustered ADBMS. | Major | contrib/vertica | Omer Trajman | Omer Trajman | +| [MAPREDUCE-768](https://issues.apache.org/jira/browse/MAPREDUCE-768) | Configuration information should generate dump in a standard format. | Major | . | rahul k singh | V.V.Chaitanya Krishna | +| [MAPREDUCE-751](https://issues.apache.org/jira/browse/MAPREDUCE-751) | Rumen: a tool to extract job characterization data from job tracker logs | Major | tools/rumen | Dick King | Dick King | +| [MAPREDUCE-740](https://issues.apache.org/jira/browse/MAPREDUCE-740) | Provide summary information per job once a job is finished. | Major | jobtracker | Hong Tang | Arun C Murthy | +| [MAPREDUCE-728](https://issues.apache.org/jira/browse/MAPREDUCE-728) | Mumak: Map-Reduce Simulator | Major | . | Arun C Murthy | Hong Tang | +| [MAPREDUCE-707](https://issues.apache.org/jira/browse/MAPREDUCE-707) | Provide a jobconf property for explicitly assigning a job to a pool | Trivial | contrib/fair-share | Matei Zaharia | Alan Heirich | +| [MAPREDUCE-706](https://issues.apache.org/jira/browse/MAPREDUCE-706) | Support for FIFO pools in the fair scheduler | Major | contrib/fair-share | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-698](https://issues.apache.org/jira/browse/MAPREDUCE-698) | Per-pool task limits for the fair scheduler | Major | contrib/fair-share | Matei Zaharia | Kevin Peterson | +| [MAPREDUCE-679](https://issues.apache.org/jira/browse/MAPREDUCE-679) | XML-based metrics as JSP servlet for JobTracker | Major | jobtracker | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-567](https://issues.apache.org/jira/browse/MAPREDUCE-567) | Add a new example MR that always fails | Major | examples | Philip Zeyliger | Philip Zeyliger | +| [MAPREDUCE-551](https://issues.apache.org/jira/browse/MAPREDUCE-551) | Add preemption to the fair scheduler | Major | contrib/fair-share | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-546](https://issues.apache.org/jira/browse/MAPREDUCE-546) | Provide sample fair scheduler config file in conf/ and use it by default if no other config file is specified | Minor | . | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-532](https://issues.apache.org/jira/browse/MAPREDUCE-532) | Allow admins of the Capacity Scheduler to set a hard-limit on the capacity of a queue | Major | capacity-sched | Rajiv Chittajallu | rahul k singh | +| [MAPREDUCE-467](https://issues.apache.org/jira/browse/MAPREDUCE-467) | Collect information about number of tasks succeeded / total per time unit for a tasktracker. | Major | . | Hemanth Yamijala | Sharad Agarwal | +| [MAPREDUCE-211](https://issues.apache.org/jira/browse/MAPREDUCE-211) | Provide a node health check script and run it periodically to check the node health status | Major | . | Aroop Maliakkal | Sreekanth Ramakrishnan | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6813](https://issues.apache.org/jira/browse/HADOOP-6813) | Add a new newInstance method in FileSystem that takes a "user" as argument | Blocker | fs | Devaraj Das | Devaraj Das | +| [HADOOP-6798](https://issues.apache.org/jira/browse/HADOOP-6798) | Align Ivy version for all Hadoop subprojects. | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6794](https://issues.apache.org/jira/browse/HADOOP-6794) | Move configuration and script files post split | Blocker | conf, scripts | Tom White | Tom White | +| [HADOOP-6777](https://issues.apache.org/jira/browse/HADOOP-6777) | Implement a functionality for suspend and resume a process. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6769](https://issues.apache.org/jira/browse/HADOOP-6769) | Add an API in FileSystem to get FileSystem instances based on users | Blocker | . | Devaraj Das | Devaraj Das | +| [HADOOP-6717](https://issues.apache.org/jira/browse/HADOOP-6717) | Log levels in o.a.h.security.Groups too high | Trivial | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6713](https://issues.apache.org/jira/browse/HADOOP-6713) | The RPC server Listener thread is a scalability bottleneck | Major | ipc | dhruba borthakur | Dmytro Molkov | +| [HADOOP-6709](https://issues.apache.org/jira/browse/HADOOP-6709) | Re-instate deprecated FileSystem methods that were removed after 0.20 | Blocker | fs | Tom White | Tom White | +| [HADOOP-6678](https://issues.apache.org/jira/browse/HADOOP-6678) | Remove FileContext#isFile, isDirectory and exists | Major | fs | Hairong Kuang | Eli Collins | +| [HADOOP-6657](https://issues.apache.org/jira/browse/HADOOP-6657) | Common portion of MAPREDUCE-1545 | Major | . | Luke Lu | Luke Lu | +| [HADOOP-6635](https://issues.apache.org/jira/browse/HADOOP-6635) | Install or deploy source jars to maven repo | Major | build | Patrick Angeles | Patrick Angeles | +| [HADOOP-6589](https://issues.apache.org/jira/browse/HADOOP-6589) | Better error messages for RPC clients when authentication fails | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-6585](https://issues.apache.org/jira/browse/HADOOP-6585) | Add FileStatus#isDirectory and isFile | Blocker | fs | Eli Collins | Eli Collins | +| [HADOOP-6583](https://issues.apache.org/jira/browse/HADOOP-6583) | Capture metrics for authentication/authorization at the RPC layer | Major | ipc, security | Devaraj Das | Devaraj Das | +| [HADOOP-6582](https://issues.apache.org/jira/browse/HADOOP-6582) | Token class should have a toString, equals and hashcode method | Major | security | Devaraj Das | Boris Shkolnik | +| [HADOOP-6579](https://issues.apache.org/jira/browse/HADOOP-6579) | A utility for reading and writing tokens into a URL safe string. | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-6559](https://issues.apache.org/jira/browse/HADOOP-6559) | The RPC client should try to re-login when it detects that the TGT expired | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6543](https://issues.apache.org/jira/browse/HADOOP-6543) | Allow authentication-enabled RPC clients to connect to authentication-disabled RPC servers | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-6537](https://issues.apache.org/jira/browse/HADOOP-6537) | Proposal for exceptions thrown by FileContext and Abstract File System | Major | . | Jitendra Nath Pandey | Suresh Srinivas | +| [HADOOP-6534](https://issues.apache.org/jira/browse/HADOOP-6534) | LocalDirAllocator should use whitespace trimming configuration getters | Major | conf, fs | Todd Lipcon | Todd Lipcon | +| [HADOOP-6531](https://issues.apache.org/jira/browse/HADOOP-6531) | add FileUtil.fullyDeleteContents(dir) api to delete contents of a directory | Minor | fs | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-6518](https://issues.apache.org/jira/browse/HADOOP-6518) | Kerberos login in UGI should honor KRB5CCNAME | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-6515](https://issues.apache.org/jira/browse/HADOOP-6515) | Make maximum number of http threads configurable | Major | . | Scott Chen | Scott Chen | +| [HADOOP-6492](https://issues.apache.org/jira/browse/HADOOP-6492) | Make avro serialization APIs public | Major | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-6479](https://issues.apache.org/jira/browse/HADOOP-6479) | TestUTF8 assertions could fail with better text | Minor | test | Steve Loughran | Steve Loughran | +| [HADOOP-6471](https://issues.apache.org/jira/browse/HADOOP-6471) | StringBuffer -\> StringBuilder - conversion of references as necessary | Major | . | Karthik K | Karthik K | +| [HADOOP-6467](https://issues.apache.org/jira/browse/HADOOP-6467) | Performance improvement for liststatus on directories in hadoop archives. | Major | fs | Mahadev konar | Mahadev konar | +| [HADOOP-6443](https://issues.apache.org/jira/browse/HADOOP-6443) | Serialization classes accept invalid metadata | Major | io | Aaron Kimball | Aaron Kimball | +| [HADOOP-6435](https://issues.apache.org/jira/browse/HADOOP-6435) | Make RPC.waitForProxy with timeout public | Major | ipc | Steve Loughran | Steve Loughran | +| [HADOOP-6434](https://issues.apache.org/jira/browse/HADOOP-6434) | Make HttpServer slightly easier to manage/diagnose faults with | Minor | . | Steve Loughran | Steve Loughran | +| [HADOOP-6420](https://issues.apache.org/jira/browse/HADOOP-6420) | String-to-String Maps should be embeddable in Configuration | Major | conf | Aaron Kimball | Aaron Kimball | +| [HADOOP-6413](https://issues.apache.org/jira/browse/HADOOP-6413) | Move TestReflectionUtils to Common | Major | test | Todd Lipcon | Todd Lipcon | +| [HADOOP-6407](https://issues.apache.org/jira/browse/HADOOP-6407) | Have a way to automatically update Eclipse .classpath file when new libs are added to the classpath through Ivy | Minor | build | Konstantin Boudnik | Tom White | +| [HADOOP-6403](https://issues.apache.org/jira/browse/HADOOP-6403) | Deprecate EC2 bash scripts | Major | contrib/cloud | Tom White | Tom White | +| [HADOOP-6400](https://issues.apache.org/jira/browse/HADOOP-6400) | Log errors getting Unix UGI | Minor | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6394](https://issues.apache.org/jira/browse/HADOOP-6394) | Helper class for FileContext tests | Major | test | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6366](https://issues.apache.org/jira/browse/HADOOP-6366) | Reduce ivy console output to ovservable level | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6343](https://issues.apache.org/jira/browse/HADOOP-6343) | Stack trace of any runtime exceptions should be recorded in the server logs. | Major | ipc | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6326](https://issues.apache.org/jira/browse/HADOOP-6326) | Hundson runs should check for AspectJ warnings and report failure if any is present | Critical | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6318](https://issues.apache.org/jira/browse/HADOOP-6318) | Upgrade to Avro 1.2.0 | Major | io, ipc | Doug Cutting | Doug Cutting | +| [HADOOP-6307](https://issues.apache.org/jira/browse/HADOOP-6307) | Support reading on un-closed SequenceFile | Major | io | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6305](https://issues.apache.org/jira/browse/HADOOP-6305) | Unify build property names to facilitate cross-projects modifications | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6301](https://issues.apache.org/jira/browse/HADOOP-6301) | Need to post Injection HowTo to Apache Hadoop's Wiki | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6279](https://issues.apache.org/jira/browse/HADOOP-6279) | Add JVM memory usage to JvmMetrics | Minor | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-6271](https://issues.apache.org/jira/browse/HADOOP-6271) | Fix FileContext to allow both recursive and non recursive create and mkdir | Major | fs | Sanjay Radia | Sanjay Radia | +| [HADOOP-6268](https://issues.apache.org/jira/browse/HADOOP-6268) | Add ivy jar to .gitignore | Minor | build | Todd Lipcon | Todd Lipcon | +| [HADOOP-6267](https://issues.apache.org/jira/browse/HADOOP-6267) | build-contrib.xml unnecessarily enforces that contrib projects be located in contrib/ dir | Minor | build | Todd Lipcon | Todd Lipcon | +| [HADOOP-6252](https://issues.apache.org/jira/browse/HADOOP-6252) | Provide method to determine if a deprecated key was set in the config file | Major | conf | Jakob Homan | Jakob Homan | +| [HADOOP-6246](https://issues.apache.org/jira/browse/HADOOP-6246) | Update umask code to use key deprecation facilities from HADOOP-6105 | Major | fs | Jakob Homan | Jakob Homan | +| [HADOOP-6233](https://issues.apache.org/jira/browse/HADOOP-6233) | Changes in common to rename the config keys as detailed in HDFS-531. | Major | fs | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6224](https://issues.apache.org/jira/browse/HADOOP-6224) | Add a method to WritableUtils performing a bounded read of a String | Major | io | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-6216](https://issues.apache.org/jira/browse/HADOOP-6216) | HDFS Web UI displays comments from dfs.exclude file and counts them as dead nodes | Major | util | Jim Huang | Ravi Phulari | +| [HADOOP-6204](https://issues.apache.org/jira/browse/HADOOP-6204) | Implementing aspects development and fault injeciton framework for Hadoop | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6182](https://issues.apache.org/jira/browse/HADOOP-6182) | Adding Apache License Headers and reduce releaseaudit warnings to zero | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-6166](https://issues.apache.org/jira/browse/HADOOP-6166) | Improve PureJavaCrc32 | Major | performance, util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6161](https://issues.apache.org/jira/browse/HADOOP-6161) | Add get/setEnum to Configuration | Minor | conf | Chris Douglas | Chris Douglas | +| [HADOOP-6160](https://issues.apache.org/jira/browse/HADOOP-6160) | releaseaudit (rats) should not be run againt the entire release binary | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-6150](https://issues.apache.org/jira/browse/HADOOP-6150) | Need to be able to instantiate a comparator instance from a comparator string without creating a TFile.Reader object | Minor | io | Hong Tang | Hong Tang | +| [HADOOP-6148](https://issues.apache.org/jira/browse/HADOOP-6148) | Implement a pure Java CRC32 calculator | Major | performance, util | Owen O'Malley | Scott Carey | +| [HADOOP-6146](https://issues.apache.org/jira/browse/HADOOP-6146) | Upgrade to JetS3t version 0.7.1 | Major | fs/s3 | Tom White | Tom White | +| [HADOOP-6133](https://issues.apache.org/jira/browse/HADOOP-6133) | ReflectionUtils performance regression | Major | conf | Todd Lipcon | Todd Lipcon | +| [HADOOP-6109](https://issues.apache.org/jira/browse/HADOOP-6109) | Handle large (several MB) text input lines in a reasonable amount of time | Major | io | thushara wijeratna | thushara wijeratna | +| [HADOOP-6106](https://issues.apache.org/jira/browse/HADOOP-6106) | Provide an option in ShellCommandExecutor to timeout commands that do not complete within a certain amount of time. | Major | util | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [HADOOP-6105](https://issues.apache.org/jira/browse/HADOOP-6105) | Provide a way to automatically handle backward compatibility of deprecated keys | Major | conf | Hemanth Yamijala | V.V.Chaitanya Krishna | +| [HADOOP-6099](https://issues.apache.org/jira/browse/HADOOP-6099) | Allow configuring the IPC module to send pings | Major | ipc | dhruba borthakur | dhruba borthakur | +| [HADOOP-5976](https://issues.apache.org/jira/browse/HADOOP-5976) | create script to provide classpath for external tools | Major | scripts | Owen O'Malley | Owen O'Malley | +| [HADOOP-5968](https://issues.apache.org/jira/browse/HADOOP-5968) | Sqoop should only print a warning about mysql import speed once | Minor | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-5967](https://issues.apache.org/jira/browse/HADOOP-5967) | Sqoop should only use a single map task | Minor | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-5961](https://issues.apache.org/jira/browse/HADOOP-5961) | DataNode should understand generic hadoop options | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5925](https://issues.apache.org/jira/browse/HADOOP-5925) | EC2 scripts should exit on error | Major | contrib/cloud | Tom White | Tom White | +| [HADOOP-5897](https://issues.apache.org/jira/browse/HADOOP-5897) | Add more Metrics to Namenode to capture heap usage | Major | metrics | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5896](https://issues.apache.org/jira/browse/HADOOP-5896) | Remove the dependency of GenericOptionsParser on Option.withArgPattern | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5890](https://issues.apache.org/jira/browse/HADOOP-5890) | Use exponential backoff on Thread.sleep during DN shutdown | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-5873](https://issues.apache.org/jira/browse/HADOOP-5873) | Remove deprecated methods randomDataNode() and getDatanodeByIndex(..) in FSNamesystem | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5867](https://issues.apache.org/jira/browse/HADOOP-5867) | Cleaning NNBench\* off javac warnings | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-5858](https://issues.apache.org/jira/browse/HADOOP-5858) | Eliminate UTF8 and fix warnings in test/hdfs-with-mr package | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-5857](https://issues.apache.org/jira/browse/HADOOP-5857) | Refactor hdfs jsp codes | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5854](https://issues.apache.org/jira/browse/HADOOP-5854) | findbugs : fix "Inconsistent Synchronization" warnings in hdfs | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5839](https://issues.apache.org/jira/browse/HADOOP-5839) | fixes to ec2 scripts to allow remote job submission | Major | contrib/cloud | Joydeep Sen Sarma | Joydeep Sen Sarma | +| [HADOOP-5838](https://issues.apache.org/jira/browse/HADOOP-5838) | Remove a few javac warnings under hdfs | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5822](https://issues.apache.org/jira/browse/HADOOP-5822) | Fix javac warnings in several dfs tests related to unncessary casts | Major | test | Jakob Homan | Jakob Homan | +| [HADOOP-5784](https://issues.apache.org/jira/browse/HADOOP-5784) | The length of the heartbeat cycle should be configurable. | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-5771](https://issues.apache.org/jira/browse/HADOOP-5771) | Create unit test for LinuxTaskController | Major | security, test | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5733](https://issues.apache.org/jira/browse/HADOOP-5733) | Add map/reduce slot capacity and lost map/reduce slot capacity to JobTracker metrics | Major | metrics | Hong Tang | Sreekanth Ramakrishnan | +| [HADOOP-5727](https://issues.apache.org/jira/browse/HADOOP-5727) | Faster, simpler id.hashCode() which does not allocate memory | Major | . | Shevek | Shevek | +| [HADOOP-5721](https://issues.apache.org/jira/browse/HADOOP-5721) | Provide EditLogFileInputStream and EditLogFileOutputStream as independent classes | Minor | . | Luca Telloli | | +| [HADOOP-5717](https://issues.apache.org/jira/browse/HADOOP-5717) | Create public enum class for the Framework counters in org.apache.hadoop.mapreduce | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [HADOOP-5705](https://issues.apache.org/jira/browse/HADOOP-5705) | Improved tries in TotalOrderPartitioner to eliminate large leaf nodes. | Major | . | Dick King | Dick King | +| [HADOOP-5687](https://issues.apache.org/jira/browse/HADOOP-5687) | Hadoop NameNode throws NPE if fs.default.name is the default value | Minor | . | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-5675](https://issues.apache.org/jira/browse/HADOOP-5675) | DistCp should not launch a job if it is not necessary | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5664](https://issues.apache.org/jira/browse/HADOOP-5664) | Use of ReentrantLock.lock() in MapOutputBuffer takes up too much cpu time | Minor | . | Bryan Duxbury | Chris Douglas | +| [HADOOP-5657](https://issues.apache.org/jira/browse/HADOOP-5657) | Validate data passed through TestReduceFetch | Minor | test | Chris Douglas | Chris Douglas | +| [HADOOP-5638](https://issues.apache.org/jira/browse/HADOOP-5638) | More improvement on block placement performance | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5625](https://issues.apache.org/jira/browse/HADOOP-5625) | Add I/O duration time in client trace | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HADOOP-5613](https://issues.apache.org/jira/browse/HADOOP-5613) | change S3Exception to checked exception | Minor | fs/s3 | Andrew Hitchcock | Andrew Hitchcock | +| [HADOOP-5603](https://issues.apache.org/jira/browse/HADOOP-5603) | Improve block placement performance | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5596](https://issues.apache.org/jira/browse/HADOOP-5596) | Make ObjectWritable support EnumSet | Major | io | He Yongqiang | He Yongqiang | +| [HADOOP-5595](https://issues.apache.org/jira/browse/HADOOP-5595) | NameNode does not need to run a replicator to choose a random DataNode | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5589](https://issues.apache.org/jira/browse/HADOOP-5589) | TupleWritable: Lift implicit limit on the number of values that can be stored | Major | . | Jingkei Ly | Jingkei Ly | +| [HADOOP-5572](https://issues.apache.org/jira/browse/HADOOP-5572) | The map progress value should have a separate phase for doing the final sort. | Major | . | Owen O'Malley | Ravi Gummadi | +| [HADOOP-5509](https://issues.apache.org/jira/browse/HADOOP-5509) | PendingReplicationBlocks should not start monitor in constructor. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-5502](https://issues.apache.org/jira/browse/HADOOP-5502) | Backup and checkpoint nodes should be documented | Major | documentation | Konstantin Shvachko | Jakob Homan | +| [HADOOP-5500](https://issues.apache.org/jira/browse/HADOOP-5500) | Allow number of fields to be supplied when field names are not known in DBOutputFormat#setOutput() | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-5494](https://issues.apache.org/jira/browse/HADOOP-5494) | IFile.Reader should have a nextRawKey/nextRawValue | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-5455](https://issues.apache.org/jira/browse/HADOOP-5455) | default "hadoop-metrics.properties" doesn't mention "rpc" context | Minor | documentation, metrics | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-5450](https://issues.apache.org/jira/browse/HADOOP-5450) | Add support for application-specific typecodes to typed bytes | Blocker | . | Klaas Bosteels | Klaas Bosteels | +| [HADOOP-5423](https://issues.apache.org/jira/browse/HADOOP-5423) | It should be posible to specify metadata for the output file produced by SequenceFile.Sorter.sort | Major | io | Michael Tamm | Michael Tamm | +| [HADOOP-5419](https://issues.apache.org/jira/browse/HADOOP-5419) | Provide a way for users to find out what operations they can do on which M/R queues | Major | . | Hemanth Yamijala | rahul k singh | +| [HADOOP-5396](https://issues.apache.org/jira/browse/HADOOP-5396) | Queue ACLs should be refreshed without requiring a restart of the job tracker | Major | . | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [HADOOP-5369](https://issues.apache.org/jira/browse/HADOOP-5369) | Small tweaks to reduce MapFile index size | Major | . | Ben Maurer | Ben Maurer | +| [HADOOP-5365](https://issues.apache.org/jira/browse/HADOOP-5365) | hdfsprxoy should log every access | Major | . | Kan Zhang | zhiyong zhang | +| [HADOOP-5364](https://issues.apache.org/jira/browse/HADOOP-5364) | Adding SSL certificate expiration warning to hdfsproxy | Major | . | Kan Zhang | zhiyong zhang | +| [HADOOP-5331](https://issues.apache.org/jira/browse/HADOOP-5331) | KFS: Add support for append | Major | . | Sriram Rao | Sriram Rao | +| [HADOOP-5317](https://issues.apache.org/jira/browse/HADOOP-5317) | Provide documentation for LazyOutput Feature | Major | documentation | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5279](https://issues.apache.org/jira/browse/HADOOP-5279) | test-patch.sh scirpt should just call the test-core target as part of runtestcore function. | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5266](https://issues.apache.org/jira/browse/HADOOP-5266) | Values Iterator should support "mark" and "reset" | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5264](https://issues.apache.org/jira/browse/HADOOP-5264) | TaskTracker should have single conf reference | Major | . | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5240](https://issues.apache.org/jira/browse/HADOOP-5240) | 'ant javadoc' does not check whether outputs are up to date and always rebuilds | Major | build | Aaron Kimball | Aaron Kimball | +| [HADOOP-5222](https://issues.apache.org/jira/browse/HADOOP-5222) | Add offset in client trace | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HADOOP-5205](https://issues.apache.org/jira/browse/HADOOP-5205) | Change CHUKWA\_IDENT\_STRING from "demo" to "TODO-AGENTS-INSTANCE-NAME" | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5196](https://issues.apache.org/jira/browse/HADOOP-5196) | avoiding unnecessary byte[] allocation in SequenceFile.CompressedBytes and SequenceFile.UncompressedBytes | Minor | io | Hong Tang | Hong Tang | +| [HADOOP-5147](https://issues.apache.org/jira/browse/HADOOP-5147) | remove refs to slaves file | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-5144](https://issues.apache.org/jira/browse/HADOOP-5144) | manual way of turning on restore of failed storage replicas for namenode | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-5135](https://issues.apache.org/jira/browse/HADOOP-5135) | Separate the core, hdfs and mapred junit tests | Major | build | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5126](https://issues.apache.org/jira/browse/HADOOP-5126) | Empty file BlocksWithLocations.java should be removed | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-5124](https://issues.apache.org/jira/browse/HADOOP-5124) | A few optimizations to FsNamesystem#RecentInvalidateSets | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5107](https://issues.apache.org/jira/browse/HADOOP-5107) | split the core, hdfs, and mapred jars from each other and publish them independently to the Maven repository | Major | build | Owen O'Malley | Giridharan Kesavan | +| [HADOOP-5101](https://issues.apache.org/jira/browse/HADOOP-5101) | optimizing build.xml target dependencies | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5088](https://issues.apache.org/jira/browse/HADOOP-5088) | include releaseaudit as part of test-patch.sh script | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5038](https://issues.apache.org/jira/browse/HADOOP-5038) | remove System.out.println statement | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5033](https://issues.apache.org/jira/browse/HADOOP-5033) | chukwa writer API is confusing | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-5023](https://issues.apache.org/jira/browse/HADOOP-5023) | Add Tomcat support to hdfsproxy | Major | . | Kan Zhang | zhiyong zhang | +| [HADOOP-5015](https://issues.apache.org/jira/browse/HADOOP-5015) | Separate block/replica management code from FSNamesystem | Major | . | Hairong Kuang | Suresh Srinivas | +| [HADOOP-4936](https://issues.apache.org/jira/browse/HADOOP-4936) | Improvements to TestSafeMode | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-4885](https://issues.apache.org/jira/browse/HADOOP-4885) | Try to restore failed replicas of Name Node storage (at checkpoint time) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4868](https://issues.apache.org/jira/browse/HADOOP-4868) | Split the hadoop script into 3 parts | Major | scripts | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-4859](https://issues.apache.org/jira/browse/HADOOP-4859) | Make the M/R Job output dir unique for Daily rolling | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-4842](https://issues.apache.org/jira/browse/HADOOP-4842) | Streaming combiner should allow command, not just JavaClass | Major | . | Marco Nicosia | Amareshwari Sriramadasu | +| [HADOOP-4794](https://issues.apache.org/jira/browse/HADOOP-4794) | separate branch for HadoopVersionAnnotation | Major | build | Owen O'Malley | Chris Douglas | +| [HADOOP-4788](https://issues.apache.org/jira/browse/HADOOP-4788) | Set mapred.fairscheduler.assignmultiple to true by default | Trivial | . | Matei Zaharia | Matei Zaharia | +| [HADOOP-4656](https://issues.apache.org/jira/browse/HADOOP-4656) | Add a user to groups mapping service | Major | security | Arun C Murthy | Boris Shkolnik | +| [HADOOP-4546](https://issues.apache.org/jira/browse/HADOOP-4546) | Minor fix in dfs to make hadoop work in AIX | Major | . | Arun Venugopal | Bill Habermaas | +| [HADOOP-4372](https://issues.apache.org/jira/browse/HADOOP-4372) | Improve the way the job history files are managed during job recovery | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-4365](https://issues.apache.org/jira/browse/HADOOP-4365) | Configuration.getProps() should be made protected for ease of overriding | Major | conf | Steve Loughran | Steve Loughran | +| [HADOOP-4191](https://issues.apache.org/jira/browse/HADOOP-4191) | Add a testcase for jobhistory | Major | test | Amar Kamat | Ravi Gummadi | +| [HADOOP-3659](https://issues.apache.org/jira/browse/HADOOP-3659) | Patch to allow hadoop native to compile on Mac OS X | Minor | native | Colin Evans | Colin Evans | +| [HADOOP-2898](https://issues.apache.org/jira/browse/HADOOP-2898) | HOD should allow setting MapReduce UI ports within a port range | Blocker | contrib/hod | Luca Telloli | Peeyush Bishnoi | +| [HADOOP-2838](https://issues.apache.org/jira/browse/HADOOP-2838) | Add HADOOP\_LIBRARY\_PATH config setting so Hadoop will include external directories for jni | Major | . | Owen O'Malley | Amar Kamat | +| [HADOOP-2721](https://issues.apache.org/jira/browse/HADOOP-2721) | Use job control for tasks (and therefore for pipes and streaming) | Major | . | Owen O'Malley | Ravi Gummadi | +| [HADOOP-2141](https://issues.apache.org/jira/browse/HADOOP-2141) | speculative execution start up condition based on completion time | Major | . | Koji Noguchi | Andy Konwinski | +| [HADOOP-1722](https://issues.apache.org/jira/browse/HADOOP-1722) | Make streaming to handle non-utf8 byte array | Major | . | Runping Qi | Klaas Bosteels | +| [HDFS-1199](https://issues.apache.org/jira/browse/HDFS-1199) | Extract a subset of tests for smoke (DOA) validation. | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1170](https://issues.apache.org/jira/browse/HDFS-1170) | Add more assertions to TestLargeDirectoryDelete | Minor | test | Steve Loughran | Steve Loughran | +| [HDFS-1161](https://issues.apache.org/jira/browse/HDFS-1161) | Make DN minimum valid volumes configurable | Blocker | datanode | Eli Collins | Eli Collins | +| [HDFS-1134](https://issues.apache.org/jira/browse/HDFS-1134) | Large-scale Automated Framework | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1126](https://issues.apache.org/jira/browse/HDFS-1126) | Change HDFS to depend on Hadoop 'common' artifacts instead of 'core' | Blocker | . | Tom White | Tom White | +| [HDFS-1107](https://issues.apache.org/jira/browse/HDFS-1107) | Turn on append by default. | Blocker | hdfs-client | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1092](https://issues.apache.org/jira/browse/HDFS-1092) | Use logging rather than System.err in MiniDFSCluster | Minor | test | Karthik K | Karthik K | +| [HDFS-1089](https://issues.apache.org/jira/browse/HDFS-1089) | Remove uses of FileContext#isFile, isDirectory and exists | Major | test | Eli Collins | Eli Collins | +| [HDFS-1087](https://issues.apache.org/jira/browse/HDFS-1087) | Use StringBuilder instead of Formatter for audit logs | Minor | namenode | Chris Douglas | Chris Douglas | +| [HDFS-1083](https://issues.apache.org/jira/browse/HDFS-1083) | Update TestHDFSCLI to not to expect exception class name in the error messages | Minor | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1078](https://issues.apache.org/jira/browse/HDFS-1078) | update libhdfs build process to produce static libraries | Minor | libhdfs | sam rash | sam rash | +| [HDFS-1054](https://issues.apache.org/jira/browse/HDFS-1054) | Remove unnecessary sleep after failure in nextBlockOutputStream | Major | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-1047](https://issues.apache.org/jira/browse/HDFS-1047) | Install/deploy source jars to Maven repo | Major | build | Patrick Angeles | Patrick Angeles | +| [HDFS-1031](https://issues.apache.org/jira/browse/HDFS-1031) | Enhance the webUi to list a few of the corrupted files in HDFS | Major | . | dhruba borthakur | AndrĂ© Oriani | +| [HDFS-1012](https://issues.apache.org/jira/browse/HDFS-1012) | documentLocation attribute in LdapEntry for HDFSProxy isn't specific to a cluster | Major | contrib/hdfsproxy | Srikanth Sundarrajan | Srikanth Sundarrajan | +| [HDFS-1011](https://issues.apache.org/jira/browse/HDFS-1011) | Improve Logging in HDFSProxy to include cluster name associated with the request | Minor | contrib/hdfsproxy | Srikanth Sundarrajan | Ramesh Sekaran | +| [HDFS-1009](https://issues.apache.org/jira/browse/HDFS-1009) | Support Kerberos authorization in HDFSProxy | Major | contrib/hdfsproxy | Srikanth Sundarrajan | Srikanth Sundarrajan | +| [HDFS-998](https://issues.apache.org/jira/browse/HDFS-998) | The servlets should quote server generated strings sent in the response | Major | . | Devaraj Das | Chris Douglas | +| [HDFS-997](https://issues.apache.org/jira/browse/HDFS-997) | DataNode local directories should have narrow permissions | Major | datanode | Arun C Murthy | Luke Lu | +| [HDFS-994](https://issues.apache.org/jira/browse/HDFS-994) | Provide methods for obtaining delegation token from Namenode for hftp and other uses | Major | . | Jakob Homan | Jakob Homan | +| [HDFS-986](https://issues.apache.org/jira/browse/HDFS-986) | Push HADOOP-6551 into HDFS | Major | . | Owen O'Malley | Owen O'Malley | +| [HDFS-968](https://issues.apache.org/jira/browse/HDFS-968) | s/StringBuffer/StringBuilder - as necessary | Major | . | Karthik K | Karthik K | +| [HDFS-949](https://issues.apache.org/jira/browse/HDFS-949) | Move Delegation token into Common so that we can use it for MapReduce also | Major | security | Owen O'Malley | Owen O'Malley | +| [HDFS-933](https://issues.apache.org/jira/browse/HDFS-933) | Add createIdentifier() implementation to DelegationTokenSecretManager | Major | . | Kan Zhang | Kan Zhang | +| [HDFS-930](https://issues.apache.org/jira/browse/HDFS-930) | o.a.h.hdfs.server.datanode.DataXceiver - run() - Version mismatch exception - more context to help debugging | Minor | documentation | Karthik K | Karthik K | +| [HDFS-921](https://issues.apache.org/jira/browse/HDFS-921) | Convert TestDFSClientRetries::testNotYetReplicatedErrors to Mockito | Major | test | Jakob Homan | Jakob Homan | +| [HDFS-883](https://issues.apache.org/jira/browse/HDFS-883) | Datanode shutdown should log problems with Storage.unlockAll() | Minor | datanode | Steve Loughran | Steve Loughran | +| [HDFS-854](https://issues.apache.org/jira/browse/HDFS-854) | Datanode should scan devices in parallel to generate block report | Major | datanode | dhruba borthakur | Dmytro Molkov | +| [HDFS-850](https://issues.apache.org/jira/browse/HDFS-850) | Display more memory details on the web ui | Minor | . | Dmytro Molkov | Dmytro Molkov | +| [HDFS-844](https://issues.apache.org/jira/browse/HDFS-844) | Log the filename when file locking fails | Major | . | Tom White | Tom White | +| [HDFS-832](https://issues.apache.org/jira/browse/HDFS-832) | HDFS side of HADOOP-6222. | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-826](https://issues.apache.org/jira/browse/HDFS-826) | Allow a mechanism for an application to detect that datanode(s) have died in the write pipeline | Major | hdfs-client | dhruba borthakur | dhruba borthakur | +| [HDFS-822](https://issues.apache.org/jira/browse/HDFS-822) | Appends to already-finalized blocks can rename across volumes | Blocker | datanode | Todd Lipcon | Hairong Kuang | +| [HDFS-806](https://issues.apache.org/jira/browse/HDFS-806) | Add new unit tests to the 10-mins 'run-commit-test' target | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-800](https://issues.apache.org/jira/browse/HDFS-800) | The last block of a file under construction may change to the COMPLETE state in response to getAdditionalBlock or completeFileInternal | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-786](https://issues.apache.org/jira/browse/HDFS-786) | Implement getContentSummary(..) in HftpFileSystem | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-767](https://issues.apache.org/jira/browse/HDFS-767) | Job failure due to BlockMissingException | Major | . | Ning Zhang | Ning Zhang | +| [HDFS-758](https://issues.apache.org/jira/browse/HDFS-758) | Improve reporting of progress of decommissioning | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-755](https://issues.apache.org/jira/browse/HDFS-755) | Read multiple checksum chunks at once in DFSInputStream | Major | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-754](https://issues.apache.org/jira/browse/HDFS-754) | Reduce ivy console output to observable level | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-736](https://issues.apache.org/jira/browse/HDFS-736) | commitBlockSynchronization() should directly update block GS and length. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-729](https://issues.apache.org/jira/browse/HDFS-729) | fsck option to list only corrupted files | Major | namenode | dhruba borthakur | Rodrigo Schmidt | +| [HDFS-728](https://issues.apache.org/jira/browse/HDFS-728) | Create a comprehensive functional test for append | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-704](https://issues.apache.org/jira/browse/HDFS-704) | Unify build property names to facilitate cross-projects modifications | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-703](https://issues.apache.org/jira/browse/HDFS-703) | Replace current fault injection implementation with one from Common | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-685](https://issues.apache.org/jira/browse/HDFS-685) | Use the user-to-groups mapping service in the NameNode | Major | namenode | Arun C Murthy | Boris Shkolnik | +| [HDFS-680](https://issues.apache.org/jira/browse/HDFS-680) | Add new access method to a copy of a block's replica | Major | . | Konstantin Boudnik | Konstantin Shvachko | +| [HDFS-631](https://issues.apache.org/jira/browse/HDFS-631) | Changes in HDFS to rename the config keys as detailed in HDFS-531. | Major | datanode, hdfs-client, namenode, test | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-605](https://issues.apache.org/jira/browse/HDFS-605) | There's not need to run fault-inject tests by 'run-test-hdfs-with-mr' target | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-598](https://issues.apache.org/jira/browse/HDFS-598) | Eclipse launch task for HDFS | Trivial | build | Eli Collins | Eli Collins | +| [HDFS-584](https://issues.apache.org/jira/browse/HDFS-584) | Fail the fault-inject build if any advices are mis-bound | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-581](https://issues.apache.org/jira/browse/HDFS-581) | Introduce an iterator over blocks in the block report array. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-578](https://issues.apache.org/jira/browse/HDFS-578) | Support for using server default values for blockSize and replication when creating a file | Major | hdfs-client, namenode | Kan Zhang | Kan Zhang | +| [HDFS-563](https://issues.apache.org/jira/browse/HDFS-563) | Simplify the codes in FSNamesystem.getBlockLocations(..) | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-549](https://issues.apache.org/jira/browse/HDFS-549) | Allow non fault-inject specific tests execution with an explicit -Dtestcase=... setting | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-548](https://issues.apache.org/jira/browse/HDFS-548) | TestFsck takes nearly 10 minutes to run - a quarter of the entire hdfs-test time | Major | test | Jakob Homan | Hairong Kuang | +| [HDFS-546](https://issues.apache.org/jira/browse/HDFS-546) | DatanodeDescriptor block iterator should be BlockInfo based rather than Block. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-539](https://issues.apache.org/jira/browse/HDFS-539) | Fault injeciton utlis for pipeline testing needs to be refactored for future reuse by other tests | Minor | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-532](https://issues.apache.org/jira/browse/HDFS-532) | Allow applications to know that a read request failed because block is missing | Major | hdfs-client | dhruba borthakur | dhruba borthakur | +| [HDFS-531](https://issues.apache.org/jira/browse/HDFS-531) | Renaming of configuration keys | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-530](https://issues.apache.org/jira/browse/HDFS-530) | Refactor TestFileAppend\* to remove code duplications | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-529](https://issues.apache.org/jira/browse/HDFS-529) | More redundant block searches in BlockManager. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-527](https://issues.apache.org/jira/browse/HDFS-527) | Refactor DFSClient constructors | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-511](https://issues.apache.org/jira/browse/HDFS-511) | Redundant block searches in BlockManager. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-510](https://issues.apache.org/jira/browse/HDFS-510) | Rename DatanodeBlockInfo to be ReplicaInfo | Major | datanode | Hairong Kuang | Jakob Homan | +| [HDFS-504](https://issues.apache.org/jira/browse/HDFS-504) | HDFS updates the modification time of a file when the file is closed. | Minor | namenode | Chun Zhang | Chun Zhang | +| [HDFS-496](https://issues.apache.org/jira/browse/HDFS-496) | Use PureJavaCrc32 in HDFS | Minor | datanode, hdfs-client, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-493](https://issues.apache.org/jira/browse/HDFS-493) | Only fault-injected tests have to be executed by run-test-\*-faul-inject targets; none of fault-injected tests need to be ran normal testing process | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-490](https://issues.apache.org/jira/browse/HDFS-490) | eliminate the usage of FileSystem.create( ) depracated by Hadoop-5438 | Minor | test | He Yongqiang | He Yongqiang | +| [HDFS-457](https://issues.apache.org/jira/browse/HDFS-457) | better handling of volume failure in Data Node storage | Major | datanode | Boris Shkolnik | Boris Shkolnik | +| [HDFS-443](https://issues.apache.org/jira/browse/HDFS-443) | New metrics in namenode to capture lost heartbeats. | Major | namenode | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-412](https://issues.apache.org/jira/browse/HDFS-412) | Hadoop JMX usage makes Nagios monitoring impossible | Major | . | Brian Bockelman | Brian Bockelman | +| [HDFS-385](https://issues.apache.org/jira/browse/HDFS-385) | Design a pluggable interface to place replicas of blocks in HDFS | Major | . | dhruba borthakur | dhruba borthakur | +| [HDFS-381](https://issues.apache.org/jira/browse/HDFS-381) | Datanode should report deletion of blocks to Namenode explicitly | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-352](https://issues.apache.org/jira/browse/HDFS-352) | saveNamespace command should be documented. | Major | documentation | Konstantin Shvachko | Ravi Phulari | +| [HDFS-278](https://issues.apache.org/jira/browse/HDFS-278) | Should DFS outputstream's close wait forever? | Major | . | Raghu Angadi | dhruba borthakur | +| [HDFS-265](https://issues.apache.org/jira/browse/HDFS-265) | Revisit append | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-173](https://issues.apache.org/jira/browse/HDFS-173) | Recursively deleting a directory with millions of files makes NameNode unresponsive for other commands until the deletion completes | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-1856](https://issues.apache.org/jira/browse/MAPREDUCE-1856) | Extract a subset of tests for smoke (DOA) validation | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1832](https://issues.apache.org/jira/browse/MAPREDUCE-1832) | Support for file sizes less than 1MB in DFSIO benchmark. | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-1751](https://issues.apache.org/jira/browse/MAPREDUCE-1751) | Change MapReduce to depend on Hadoop 'common' artifacts instead of 'core' | Blocker | build | Tom White | Tom White | +| [MAPREDUCE-1749](https://issues.apache.org/jira/browse/MAPREDUCE-1749) | Pull configuration strings out of JobContext | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1735](https://issues.apache.org/jira/browse/MAPREDUCE-1735) | Un-deprecate the old MapReduce API in the 0.21 branch | Blocker | . | Tom White | Tom White | +| [MAPREDUCE-1656](https://issues.apache.org/jira/browse/MAPREDUCE-1656) | JobStory should provide queue info. | Minor | . | Hong Tang | Hong Tang | +| [MAPREDUCE-1627](https://issues.apache.org/jira/browse/MAPREDUCE-1627) | HadoopArchives should not uses DistCp method | Minor | harchive | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1613](https://issues.apache.org/jira/browse/MAPREDUCE-1613) | Install/deploy source jars to Maven repo | Minor | build | Patrick Angeles | | +| [MAPREDUCE-1593](https://issues.apache.org/jira/browse/MAPREDUCE-1593) | [Rumen] Improvements to random seed generation | Trivial | tools/rumen | Tamas Sarlos | Tamas Sarlos | +| [MAPREDUCE-1590](https://issues.apache.org/jira/browse/MAPREDUCE-1590) | Move HarFileSystem from Hadoop Common to Mapreduce tools. | Major | harchive | Mahadev konar | Mahadev konar | +| [MAPREDUCE-1579](https://issues.apache.org/jira/browse/MAPREDUCE-1579) | archive: check and possibly replace the space charater in paths | Blocker | harchive | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1570](https://issues.apache.org/jira/browse/MAPREDUCE-1570) | Shuffle stage - Key and Group Comparators | Minor | contrib/mrunit | Chris White | Chris White | +| [MAPREDUCE-1569](https://issues.apache.org/jira/browse/MAPREDUCE-1569) | Mock Contexts & Configurations | Minor | contrib/mrunit | Chris White | Chris White | +| [MAPREDUCE-1568](https://issues.apache.org/jira/browse/MAPREDUCE-1568) | TrackerDistributedCacheManager should clean up cache in a background thread | Major | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1556](https://issues.apache.org/jira/browse/MAPREDUCE-1556) | upgrade to Avro 1.3.0 | Major | jobtracker | Doug Cutting | Doug Cutting | +| [MAPREDUCE-1535](https://issues.apache.org/jira/browse/MAPREDUCE-1535) | Replace usage of FileStatus#isDir() | Blocker | . | Eli Collins | Eli Collins | +| [MAPREDUCE-1527](https://issues.apache.org/jira/browse/MAPREDUCE-1527) | QueueManager should issue warning if mapred-queues.xml is skipped. | Major | . | Hong Tang | Hong Tang | +| [MAPREDUCE-1518](https://issues.apache.org/jira/browse/MAPREDUCE-1518) | On contrib/raid, the RaidNode currently runs the deletion check for parity files on directories too. It would be better if it didn't. | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1514](https://issues.apache.org/jira/browse/MAPREDUCE-1514) | Add documentation on permissions, limitations, error handling for archives. | Major | documentation | Mahadev konar | Mahadev konar | +| [MAPREDUCE-1512](https://issues.apache.org/jira/browse/MAPREDUCE-1512) | RAID could use HarFileSystem directly instead of FileSystem.get | Minor | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1503](https://issues.apache.org/jira/browse/MAPREDUCE-1503) | Push HADOOP-6551 into MapReduce | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1491](https://issues.apache.org/jira/browse/MAPREDUCE-1491) | Use HAR filesystem to merge parity files | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1489](https://issues.apache.org/jira/browse/MAPREDUCE-1489) | DataDrivenDBInputFormat should not query the database when generating only one split | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1470](https://issues.apache.org/jira/browse/MAPREDUCE-1470) | Move Delegation token into Common so that we can use it for MapReduce also | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1466](https://issues.apache.org/jira/browse/MAPREDUCE-1466) | FileInputFormat should save #input-files in JobConf | Minor | client | Arun C Murthy | Luke Lu | +| [MAPREDUCE-1460](https://issues.apache.org/jira/browse/MAPREDUCE-1460) | Oracle support in DataDrivenDBInputFormat | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1440](https://issues.apache.org/jira/browse/MAPREDUCE-1440) | MapReduce should use the short form of the user names | Major | security | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1428](https://issues.apache.org/jira/browse/MAPREDUCE-1428) | Make block size and the size of archive created files configurable. | Major | harchive | Mahadev konar | Mahadev konar | +| [MAPREDUCE-1425](https://issues.apache.org/jira/browse/MAPREDUCE-1425) | archive throws OutOfMemoryError | Major | harchive | Tsz Wo Nicholas Sze | Mahadev konar | +| [MAPREDUCE-1423](https://issues.apache.org/jira/browse/MAPREDUCE-1423) | Improve performance of CombineFileInputFormat when multiple pools are configured | Major | client | dhruba borthakur | dhruba borthakur | +| [MAPREDUCE-1403](https://issues.apache.org/jira/browse/MAPREDUCE-1403) | Save file-sizes of each of the artifacts in DistributedCache in the JobConf | Major | client | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-1367](https://issues.apache.org/jira/browse/MAPREDUCE-1367) | LocalJobRunner should support parallel mapper execution | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1337](https://issues.apache.org/jira/browse/MAPREDUCE-1337) | Generify StreamJob for better readability | Major | . | Karthik K | Karthik K | +| [MAPREDUCE-1317](https://issues.apache.org/jira/browse/MAPREDUCE-1317) | Reducing memory consumption of rumen objects | Major | tools/rumen | Hong Tang | Hong Tang | +| [MAPREDUCE-1309](https://issues.apache.org/jira/browse/MAPREDUCE-1309) | I want to change the rumen job trace generator to use a more modular internal structure, to allow for more input log formats | Major | tools/rumen | Dick King | Dick King | +| [MAPREDUCE-1306](https://issues.apache.org/jira/browse/MAPREDUCE-1306) | [MUMAK] Randomize the arrival of heartbeat responses | Major | contrib/mumak | Tamas Sarlos | Tamas Sarlos | +| [MAPREDUCE-1305](https://issues.apache.org/jira/browse/MAPREDUCE-1305) | Running distcp with -delete incurs avoidable penalties | Major | distcp | Peter Romianowski | Peter Romianowski | +| [MAPREDUCE-1302](https://issues.apache.org/jira/browse/MAPREDUCE-1302) | TrackerDistributedCacheManager can delete file asynchronously | Major | tasktracker | Zheng Shao | Zheng Shao | +| [MAPREDUCE-1265](https://issues.apache.org/jira/browse/MAPREDUCE-1265) | Include tasktracker name in the task attempt error log | Trivial | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1250](https://issues.apache.org/jira/browse/MAPREDUCE-1250) | Refactor job token to use a common token interface | Major | security | Kan Zhang | Kan Zhang | +| [MAPREDUCE-1231](https://issues.apache.org/jira/browse/MAPREDUCE-1231) | Distcp is very slow | Major | distcp | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-1229](https://issues.apache.org/jira/browse/MAPREDUCE-1229) | [Mumak] Allow customization of job submission policy | Major | contrib/mumak | Hong Tang | Hong Tang | +| [MAPREDUCE-1221](https://issues.apache.org/jira/browse/MAPREDUCE-1221) | Kill tasks on a node if the free physical memory on that machine falls below a configured threshold | Major | tasktracker | dhruba borthakur | Scott Chen | +| [MAPREDUCE-1198](https://issues.apache.org/jira/browse/MAPREDUCE-1198) | Alternatively schedule different types of tasks in fair share scheduler | Major | contrib/fair-share | Scott Chen | Scott Chen | +| [MAPREDUCE-1189](https://issues.apache.org/jira/browse/MAPREDUCE-1189) | Reduce ivy console output to ovservable level | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1185](https://issues.apache.org/jira/browse/MAPREDUCE-1185) | URL to JT webconsole for running job and job history should be the same | Major | jobtracker | Sharad Agarwal | Amareshwari Sriramadasu | +| [MAPREDUCE-1103](https://issues.apache.org/jira/browse/MAPREDUCE-1103) | Additional JobTracker metrics | Major | jobtracker | Arun C Murthy | Sharad Agarwal | +| [MAPREDUCE-1084](https://issues.apache.org/jira/browse/MAPREDUCE-1084) | Implementing aspects development and fault injeciton framework for MapReduce | Major | build, test | Konstantin Boudnik | Sreekanth Ramakrishnan | +| [MAPREDUCE-1083](https://issues.apache.org/jira/browse/MAPREDUCE-1083) | Use the user-to-groups mapping service in the JobTracker | Major | jobtracker | Arun C Murthy | Boris Shkolnik | +| [MAPREDUCE-1048](https://issues.apache.org/jira/browse/MAPREDUCE-1048) | Show total slot usage in cluster summary on jobtracker webui | Major | jobtracker | Amar Kamat | Amareshwari Sriramadasu | +| [MAPREDUCE-1012](https://issues.apache.org/jira/browse/MAPREDUCE-1012) | Context interfaces should be Public Evolving | Blocker | client | Tom White | Tom White | +| [MAPREDUCE-1011](https://issues.apache.org/jira/browse/MAPREDUCE-1011) | Git and Subversion ignore of build.properties | Major | build | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-972](https://issues.apache.org/jira/browse/MAPREDUCE-972) | distcp can timeout during rename operation to s3 | Major | distcp, documentation | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-966](https://issues.apache.org/jira/browse/MAPREDUCE-966) | Rumen interface improvement | Major | tools/rumen | Hong Tang | Hong Tang | +| [MAPREDUCE-960](https://issues.apache.org/jira/browse/MAPREDUCE-960) | Unnecessary copy in mapreduce.lib.input.KeyValueLineRecordReader | Major | . | Chris Douglas | Chris Douglas | +| [MAPREDUCE-954](https://issues.apache.org/jira/browse/MAPREDUCE-954) | The new interface's Context objects should be interfaces | Major | client | Owen O'Malley | Arun C Murthy | +| [MAPREDUCE-953](https://issues.apache.org/jira/browse/MAPREDUCE-953) | Generate configuration dump for hierarchial queue configuration | Blocker | jobtracker | rahul k singh | V.V.Chaitanya Krishna | +| [MAPREDUCE-947](https://issues.apache.org/jira/browse/MAPREDUCE-947) | OutputCommitter should have an abortJob method | Major | . | Owen O'Malley | Amar Kamat | +| [MAPREDUCE-944](https://issues.apache.org/jira/browse/MAPREDUCE-944) | Extend FairShare scheduler to fair-share memory usage in the cluster | Major | contrib/fair-share | dhruba borthakur | dhruba borthakur | +| [MAPREDUCE-936](https://issues.apache.org/jira/browse/MAPREDUCE-936) | Allow a load difference in fairshare scheduler | Major | contrib/fair-share | Zheng Shao | Zheng Shao | +| [MAPREDUCE-931](https://issues.apache.org/jira/browse/MAPREDUCE-931) | rumen should use its own interpolation classes to create runtimes for simulated tasks | Minor | tools/rumen | Dick King | Dick King | +| [MAPREDUCE-930](https://issues.apache.org/jira/browse/MAPREDUCE-930) | rumen should interpret job history log input paths with respect to default FS, not local FS | Minor | tools/rumen | Dick King | Chris Douglas | +| [MAPREDUCE-910](https://issues.apache.org/jira/browse/MAPREDUCE-910) | MRUnit should support counters | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-905](https://issues.apache.org/jira/browse/MAPREDUCE-905) | Add Eclipse launch tasks for MapReduce | Minor | . | Philip Zeyliger | Philip Zeyliger | +| [MAPREDUCE-903](https://issues.apache.org/jira/browse/MAPREDUCE-903) | Adding AVRO jar to eclipse classpath | Major | . | Philip Zeyliger | Philip Zeyliger | +| [MAPREDUCE-893](https://issues.apache.org/jira/browse/MAPREDUCE-893) | Provide an ability to refresh queue configuration without restart. | Major | jobtracker | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [MAPREDUCE-885](https://issues.apache.org/jira/browse/MAPREDUCE-885) | More efficient SQL queries for DBInputFormat | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-875](https://issues.apache.org/jira/browse/MAPREDUCE-875) | Make DBRecordReader execute queries lazily | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-874](https://issues.apache.org/jira/browse/MAPREDUCE-874) | The name "PiEstimator" is misleading | Minor | examples | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-847](https://issues.apache.org/jira/browse/MAPREDUCE-847) | Adding Apache License Headers and reduce releaseaudit warnings to zero | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-830](https://issues.apache.org/jira/browse/MAPREDUCE-830) | Providing BZip2 splitting support for Text data | Major | . | Abdul Qadeer | Abdul Qadeer | +| [MAPREDUCE-797](https://issues.apache.org/jira/browse/MAPREDUCE-797) | MRUnit MapReduceDriver should support combiners | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-788](https://issues.apache.org/jira/browse/MAPREDUCE-788) | Modify gridmix2 to use new api. | Major | benchmarks | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-784](https://issues.apache.org/jira/browse/MAPREDUCE-784) | Modify TestUserDefinedCounters to use LocalJobRunner instead of MiniMR | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-782](https://issues.apache.org/jira/browse/MAPREDUCE-782) | Use PureJavaCrc32 in mapreduce spills | Minor | performance | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-781](https://issues.apache.org/jira/browse/MAPREDUCE-781) | distcp overrides user-selected job name | Major | distcp | Rob Weltman | Venkatesh Seetharam | +| [MAPREDUCE-779](https://issues.apache.org/jira/browse/MAPREDUCE-779) | Add node health failures into JobTrackerStatistics | Major | jobtracker | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [MAPREDUCE-765](https://issues.apache.org/jira/browse/MAPREDUCE-765) | eliminate the usage of FileSystem.create( ) depracated by Hadoop-5438 | Minor | distcp, jobtracker | He Yongqiang | He Yongqiang | +| [MAPREDUCE-742](https://issues.apache.org/jira/browse/MAPREDUCE-742) | Improve the java comments for the π examples | Minor | documentation, examples | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-739](https://issues.apache.org/jira/browse/MAPREDUCE-739) | Allow relative paths to be created inside archives. | Major | harchive | Mahadev konar | Mahadev konar | +| [MAPREDUCE-712](https://issues.apache.org/jira/browse/MAPREDUCE-712) | RandomTextWriter example is CPU bound | Major | examples | Khaled Elmeleegy | Chris Douglas | +| [MAPREDUCE-701](https://issues.apache.org/jira/browse/MAPREDUCE-701) | Make TestRackAwareTaskPlacement a unit test | Minor | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-664](https://issues.apache.org/jira/browse/MAPREDUCE-664) | distcp with -delete option does not display number of files deleted from the target that were not present on source | Major | distcp | Suhas Gogate | Ravi Gummadi | +| [MAPREDUCE-654](https://issues.apache.org/jira/browse/MAPREDUCE-654) | Add an option -count to distcp for displaying some info about the src files | Major | distcp | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-649](https://issues.apache.org/jira/browse/MAPREDUCE-649) | distcp should validate the data copied | Major | distcp | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-632](https://issues.apache.org/jira/browse/MAPREDUCE-632) | Merge TestCustomOutputCommitter with TestCommandLineJobSubmission | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-625](https://issues.apache.org/jira/browse/MAPREDUCE-625) | Modify TestTaskLimits to improve execution time | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-502](https://issues.apache.org/jira/browse/MAPREDUCE-502) | Allow jobtracker to be configured with zero completed jobs in memory | Major | . | Amar Kamat | Amar Kamat | +| [MAPREDUCE-476](https://issues.apache.org/jira/browse/MAPREDUCE-476) | extend DistributedCache to work locally (LocalJobRunner) | Minor | . | sam rash | Philip Zeyliger | +| [MAPREDUCE-463](https://issues.apache.org/jira/browse/MAPREDUCE-463) | The job setup and cleanup tasks should be optional | Major | . | Owen O'Malley | Amareshwari Sriramadasu | +| [MAPREDUCE-353](https://issues.apache.org/jira/browse/MAPREDUCE-353) | Allow shuffle read and connection timeouts to be configurable | Major | . | Arun C Murthy | Ravi Gummadi | +| [MAPREDUCE-336](https://issues.apache.org/jira/browse/MAPREDUCE-336) | The logging level of the tasks should be configurable by the job | Major | . | Owen O'Malley | Arun C Murthy | +| [MAPREDUCE-318](https://issues.apache.org/jira/browse/MAPREDUCE-318) | Refactor reduce shuffle code | Major | performance, task | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-284](https://issues.apache.org/jira/browse/MAPREDUCE-284) | Improvements to RPC between Child and TaskTracker | Major | . | Arun C Murthy | Ravi Gummadi | +| [MAPREDUCE-277](https://issues.apache.org/jira/browse/MAPREDUCE-277) | Job history counters should be avaible on the UI. | Blocker | jobtracker | Amareshwari Sriramadasu | Jothi Padmanabhan | +| [MAPREDUCE-271](https://issues.apache.org/jira/browse/MAPREDUCE-271) | Change examples code to use new mapreduce api. | Major | examples | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-270](https://issues.apache.org/jira/browse/MAPREDUCE-270) | TaskTracker could send an out-of-band heartbeat when the last running map/reduce completes | Major | . | Arun C Murthy | Arun C Murthy | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6881](https://issues.apache.org/jira/browse/HADOOP-6881) | The efficient comparators aren't always used except for BytesWritable and Text | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6875](https://issues.apache.org/jira/browse/HADOOP-6875) | [Herriot] Cleanup of temp. configurations is needed upon restart of a cluster | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6860](https://issues.apache.org/jira/browse/HADOOP-6860) | 'compile-fault-inject' should never be called directly. | Minor | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6847](https://issues.apache.org/jira/browse/HADOOP-6847) | Problem staging 0.21.0 artifacts to Apache Nexus Maven Repository | Blocker | build | Tom White | Giridharan Kesavan | +| [HADOOP-6828](https://issues.apache.org/jira/browse/HADOOP-6828) | Herrior uses old way of accessing logs directories | Major | test | Konstantin Boudnik | Sreekanth Ramakrishnan | +| [HADOOP-6826](https://issues.apache.org/jira/browse/HADOOP-6826) | Revert FileSystem create method that takes CreateFlags | Blocker | fs | Tom White | Tom White | +| [HADOOP-6821](https://issues.apache.org/jira/browse/HADOOP-6821) | Document changes to memory monitoring | Blocker | documentation | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-6819](https://issues.apache.org/jira/browse/HADOOP-6819) | [Herriot] Shell command for getting the new exceptions in the logs returning exitcode 1 after executing successfully. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6800](https://issues.apache.org/jira/browse/HADOOP-6800) | Harmonize JAR library versions | Blocker | . | Tom White | Tom White | +| [HADOOP-6790](https://issues.apache.org/jira/browse/HADOOP-6790) | Instrumented (Herriot) build uses too wide mask to include aspect files. | Minor | test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6788](https://issues.apache.org/jira/browse/HADOOP-6788) | [Herriot] Exception exclusion functionality is not working correctly. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6782](https://issues.apache.org/jira/browse/HADOOP-6782) | TestAvroRpc fails with avro-1.3.1 and avro-1.3.2 | Major | . | Jitendra Nath Pandey | Doug Cutting | +| [HADOOP-6748](https://issues.apache.org/jira/browse/HADOOP-6748) | Remove hadoop.cluster.administrators | Major | security | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-6742](https://issues.apache.org/jira/browse/HADOOP-6742) | Add methods HADOOP-6709 from to TestFilterFileSystem | Major | . | Eli Collins | Eli Collins | +| [HADOOP-6727](https://issues.apache.org/jira/browse/HADOOP-6727) | Remove UnresolvedLinkException from public FileContext APIs | Blocker | fs | Eli Collins | Eli Collins | +| [HADOOP-6724](https://issues.apache.org/jira/browse/HADOOP-6724) | IPC doesn't properly handle IOEs thrown by socket factory | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-6723](https://issues.apache.org/jira/browse/HADOOP-6723) | unchecked exceptions thrown in IPC Connection orphan clients | Critical | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-6722](https://issues.apache.org/jira/browse/HADOOP-6722) | NetUtils.connect should check that it hasn't connected a socket to itself | Major | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-6719](https://issues.apache.org/jira/browse/HADOOP-6719) | Missing methods on FilterFs | Major | . | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-6703](https://issues.apache.org/jira/browse/HADOOP-6703) | Prevent renaming a file, symlink or directory to itself | Minor | . | Eli Collins | Eli Collins | +| [HADOOP-6702](https://issues.apache.org/jira/browse/HADOOP-6702) | Incorrect exit codes for "dfs -chown", "dfs -chgrp" when input is given in wildcard format. | Minor | fs | Ravi Phulari | Ravi Phulari | +| [HADOOP-6698](https://issues.apache.org/jira/browse/HADOOP-6698) | Revert the io.serialization package to 0.20.2's api | Blocker | io | Owen O'Malley | Tom White | +| [HADOOP-6691](https://issues.apache.org/jira/browse/HADOOP-6691) | TestFileSystemCaching sometimes hang | Major | test | Hairong Kuang | Hairong Kuang | +| [HADOOP-6690](https://issues.apache.org/jira/browse/HADOOP-6690) | FilterFileSystem doesn't overwrite setTimes | Major | . | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-6677](https://issues.apache.org/jira/browse/HADOOP-6677) | InterfaceAudience.LimitedPrivate should take a string not an enum | Minor | . | Alan Gates | Tom White | +| [HADOOP-6654](https://issues.apache.org/jira/browse/HADOOP-6654) | Example in WritableComparable javadoc doesn't compile | Trivial | io | Tom White | Tom White | +| [HADOOP-6646](https://issues.apache.org/jira/browse/HADOOP-6646) | Move HarfileSystem out of Hadoop Common. | Major | fs | Mahadev konar | Mahadev konar | +| [HADOOP-6645](https://issues.apache.org/jira/browse/HADOOP-6645) | Bugs on listStatus for HarFileSystem | Major | fs | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-6640](https://issues.apache.org/jira/browse/HADOOP-6640) | FileSystem.get() does RPC retries within a static synchronized block | Critical | fs | Alejandro Abdelnur | Hairong Kuang | +| [HADOOP-6634](https://issues.apache.org/jira/browse/HADOOP-6634) | AccessControlList uses full-principal names to verify acls causing queue-acls to fail | Major | security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-6631](https://issues.apache.org/jira/browse/HADOOP-6631) | FileUtil.fullyDelete() should continue to delete other files despite failure at any level. | Major | fs, util | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [HADOOP-6630](https://issues.apache.org/jira/browse/HADOOP-6630) | hadoop-config.sh fails to get executed if hadoop wrapper scripts are in path | Major | . | Allen Wittenauer | Allen Wittenauer | +| [HADOOP-6609](https://issues.apache.org/jira/browse/HADOOP-6609) | Deadlock in DFSClient#getBlockLocations even with the security disabled | Major | io | Hairong Kuang | Owen O'Malley | +| [HADOOP-6593](https://issues.apache.org/jira/browse/HADOOP-6593) | TextRecordInputStream doesn't close SequenceFile.Reader | Minor | fs | Chase Bradford | Chase Bradford | +| [HADOOP-6591](https://issues.apache.org/jira/browse/HADOOP-6591) | HarFileSystem cannot handle paths with the space character | Major | fs | Tsz Wo Nicholas Sze | Rodrigo Schmidt | +| [HADOOP-6572](https://issues.apache.org/jira/browse/HADOOP-6572) | RPC responses may be out-of-order with respect to SASL | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-6570](https://issues.apache.org/jira/browse/HADOOP-6570) | RPC#stopProxy throws NullPointerExcption if getProxyEngine(proxy) returns null | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-6560](https://issues.apache.org/jira/browse/HADOOP-6560) | HarFileSystem throws NPE for har://hdfs-/foo | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6558](https://issues.apache.org/jira/browse/HADOOP-6558) | archive does not work with distcp -update | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6552](https://issues.apache.org/jira/browse/HADOOP-6552) | KEYTAB\_KERBEROS\_OPTIONS in UserGroupInformation should have options for automatic renewal of keytab based tickets | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6551](https://issues.apache.org/jira/browse/HADOOP-6551) | Delegation tokens when renewed or cancelled should throw an exception that explains what went wrong | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6549](https://issues.apache.org/jira/browse/HADOOP-6549) | TestDoAsEffectiveUser should use ip address of the host for superuser ip check | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6548](https://issues.apache.org/jira/browse/HADOOP-6548) | Replace org.mortbay.log.Log imports with commons logging | Trivial | fs, io | Chris Douglas | Chris Douglas | +| [HADOOP-6546](https://issues.apache.org/jira/browse/HADOOP-6546) | BloomMapFile can return false negatives | Major | io | Clark Jefcoat | Clark Jefcoat | +| [HADOOP-6545](https://issues.apache.org/jira/browse/HADOOP-6545) | Cached FileSystem objects can lead to wrong token being used in setting up connections | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6540](https://issues.apache.org/jira/browse/HADOOP-6540) | Contrib unit tests have invalid XML for core-site, etc. | Blocker | . | Aaron Kimball | Aaron Kimball | +| [HADOOP-6522](https://issues.apache.org/jira/browse/HADOOP-6522) | TestUTF8 fails | Critical | io | Todd Lipcon | Doug Cutting | +| [HADOOP-6521](https://issues.apache.org/jira/browse/HADOOP-6521) | FsPermission:SetUMask not updated to use new-style umask setting. | Major | fs | Jakob Homan | Suresh Srinivas | +| [HADOOP-6520](https://issues.apache.org/jira/browse/HADOOP-6520) | UGI should load tokens from the environment | Major | . | Owen O'Malley | Devaraj Das | +| [HADOOP-6505](https://issues.apache.org/jira/browse/HADOOP-6505) | sed in build.xml fails | Minor | build | Allen Wittenauer | Allen Wittenauer | +| [HADOOP-6504](https://issues.apache.org/jira/browse/HADOOP-6504) | Invalid example in the documentation of org.apache.hadoop.util.Tool | Trivial | documentation | Benoit Sigoure | Benoit Sigoure | +| [HADOOP-6489](https://issues.apache.org/jira/browse/HADOOP-6489) | Findbug report: LI\_LAZY\_INIT\_STATIC, OBL\_UNSATISFIED\_OBLIGATION | Major | fs, io, util | Erik Steffl | Erik Steffl | +| [HADOOP-6462](https://issues.apache.org/jira/browse/HADOOP-6462) | contrib/cloud failing, target "compile" does not exist | Major | build | Steve Loughran | Tom White | +| [HADOOP-6461](https://issues.apache.org/jira/browse/HADOOP-6461) | webapps aren't located correctly post-split | Blocker | util | Todd Lipcon | Steve Loughran | +| [HADOOP-6452](https://issues.apache.org/jira/browse/HADOOP-6452) | Hadoop JSP pages don't work under a security manager | Minor | . | Steve Loughran | Steve Loughran | +| [HADOOP-6451](https://issues.apache.org/jira/browse/HADOOP-6451) | Contrib tests are not being run | Blocker | build | Tom White | Tom White | +| [HADOOP-6441](https://issues.apache.org/jira/browse/HADOOP-6441) | Prevent remote CSS attacks in Hostname and UTF-7. | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-6439](https://issues.apache.org/jira/browse/HADOOP-6439) | Shuffle deadlocks on wrong number of maps | Blocker | conf | Owen O'Malley | V.V.Chaitanya Krishna | +| [HADOOP-6414](https://issues.apache.org/jira/browse/HADOOP-6414) | Add command line help for -expunge command. | Trivial | . | Ravi Phulari | Ravi Phulari | +| [HADOOP-6411](https://issues.apache.org/jira/browse/HADOOP-6411) | Remove deprecated file src/test/hadoop-site.xml | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6405](https://issues.apache.org/jira/browse/HADOOP-6405) | Update Eclipse configuration to match changes to Ivy configuration | Major | build | Edwin Chan | | +| [HADOOP-6404](https://issues.apache.org/jira/browse/HADOOP-6404) | Rename the generated artifacts to common instead of core | Blocker | build | Owen O'Malley | Tom White | +| [HADOOP-6402](https://issues.apache.org/jira/browse/HADOOP-6402) | testConf.xsl is not well-formed XML | Trivial | test | Steve Loughran | Steve Loughran | +| [HADOOP-6398](https://issues.apache.org/jira/browse/HADOOP-6398) | Build is broken after HADOOP-6395 patch has been applied | Blocker | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6396](https://issues.apache.org/jira/browse/HADOOP-6396) | Provide a description in the exception when an error is encountered parsing umask | Major | fs | Jakob Homan | Jakob Homan | +| [HADOOP-6395](https://issues.apache.org/jira/browse/HADOOP-6395) | Inconsistent versions of libraries are being included | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6391](https://issues.apache.org/jira/browse/HADOOP-6391) | Classpath should not be part of command line arguments | Major | scripts | Cristian Ivascu | Cristian Ivascu | +| [HADOOP-6390](https://issues.apache.org/jira/browse/HADOOP-6390) | Block slf4j-simple from avro's pom | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6386](https://issues.apache.org/jira/browse/HADOOP-6386) | NameNode's HttpServer can't instantiate InetSocketAddress: IllegalArgumentException is thrown | Blocker | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6375](https://issues.apache.org/jira/browse/HADOOP-6375) | Update documentation for FsShell du command | Major | documentation | Todd Lipcon | Todd Lipcon | +| [HADOOP-6374](https://issues.apache.org/jira/browse/HADOOP-6374) | JUnit tests should never depend on anything in conf | Blocker | test | Owen O'Malley | Anatoli Fomenko | +| [HADOOP-6347](https://issues.apache.org/jira/browse/HADOOP-6347) | run-test-core-fault-inject runs a test case twice if -Dtestcase is set | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6344](https://issues.apache.org/jira/browse/HADOOP-6344) | rm and rmr fail to correctly move the user's files to the trash prior to deleting when they are over quota. | Major | fs | gary murry | Jakob Homan | +| [HADOOP-6341](https://issues.apache.org/jira/browse/HADOOP-6341) | Hudson giving a +1 though no tests are included. | Major | build | Hemanth Yamijala | Giridharan Kesavan | +| [HADOOP-6334](https://issues.apache.org/jira/browse/HADOOP-6334) | GenericOptionsParser does not understand uri for -files -libjars and -archives option | Major | util | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-6327](https://issues.apache.org/jira/browse/HADOOP-6327) | Fix build error for one of the FileContext Tests | Major | test | Sanjay Radia | Sanjay Radia | +| [HADOOP-6314](https://issues.apache.org/jira/browse/HADOOP-6314) | "bin/hadoop fs -help count" fails to show help about only "count" command. | Major | fs | Ravi Phulari | Ravi Phulari | +| [HADOOP-6303](https://issues.apache.org/jira/browse/HADOOP-6303) | Eclipse .classpath template has outdated jar files and is missing some new ones. | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6293](https://issues.apache.org/jira/browse/HADOOP-6293) | FsShell -text should work on filesystems other than the default | Minor | fs | Chris Douglas | Chris Douglas | +| [HADOOP-6286](https://issues.apache.org/jira/browse/HADOOP-6286) | The Glob methods in FileContext doe not deal with URIs correctly | Major | fs | Sanjay Radia | Boris Shkolnik | +| [HADOOP-6285](https://issues.apache.org/jira/browse/HADOOP-6285) | HttpServer.QuotingInputFilter has the wrong signature for getParameterMap | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-6283](https://issues.apache.org/jira/browse/HADOOP-6283) | The exception meessage in FileUtil$HardLink.getLinkCount(..) is not clear | Minor | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6281](https://issues.apache.org/jira/browse/HADOOP-6281) | HtmlQuoting throws NullPointerException | Major | . | Tsz Wo Nicholas Sze | Owen O'Malley | +| [HADOOP-6274](https://issues.apache.org/jira/browse/HADOOP-6274) | TestLocalFSFileContextMainOperations tests wrongly expect a certain order to be returned. | Major | test | gary murry | gary murry | +| [HADOOP-6257](https://issues.apache.org/jira/browse/HADOOP-6257) | Two TestFileSystem classes are confusing hadoop-hdfs-hdfwithmr | Minor | build, fs, test | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-6254](https://issues.apache.org/jira/browse/HADOOP-6254) | s3n fails with SocketTimeoutException | Major | fs/s3 | Andrew Hitchcock | Andrew Hitchcock | +| [HADOOP-6250](https://issues.apache.org/jira/browse/HADOOP-6250) | test-patch.sh doesn't clean up conf/\*.xml files after the trunk run. | Major | build | rahul k singh | rahul k singh | +| [HADOOP-6243](https://issues.apache.org/jira/browse/HADOOP-6243) | NPE in handling deprecated configuration keys. | Blocker | conf | Konstantin Shvachko | Sreekanth Ramakrishnan | +| [HADOOP-6240](https://issues.apache.org/jira/browse/HADOOP-6240) | Rename operation is not consistent between different implementations of FileSystem | Major | fs | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6234](https://issues.apache.org/jira/browse/HADOOP-6234) | Permission configuration files should use octal and symbolic | Major | . | Allen Wittenauer | Jakob Homan | +| [HADOOP-6229](https://issues.apache.org/jira/browse/HADOOP-6229) | Atempt to make a directory under an existing file on LocalFileSystem should throw an Exception. | Major | fs | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6227](https://issues.apache.org/jira/browse/HADOOP-6227) | Configuration does not lock parameters marked final if they have no value. | Major | conf | Hemanth Yamijala | Amareshwari Sriramadasu | +| [HADOOP-6199](https://issues.apache.org/jira/browse/HADOOP-6199) | Add the documentation for io.map.index.skip in core-default | Major | io | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-6196](https://issues.apache.org/jira/browse/HADOOP-6196) | sync(0); next() breaks SequenceFile | Major | . | Jay Booth | Jay Booth | +| [HADOOP-6192](https://issues.apache.org/jira/browse/HADOOP-6192) | Shell.getUlimitMemoryCommand is tied to Map-Reduce | Major | util | Arun C Murthy | Arun C Murthy | +| [HADOOP-6188](https://issues.apache.org/jira/browse/HADOOP-6188) | TestHDFSTrash fails because of TestTrash in common | Major | test | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6184](https://issues.apache.org/jira/browse/HADOOP-6184) | Provide a configuration dump in json format. | Major | . | rahul k singh | V.V.Chaitanya Krishna | +| [HADOOP-6181](https://issues.apache.org/jira/browse/HADOOP-6181) | Fixes for Eclipse template | Minor | build | Carlos Valiente | Carlos Valiente | +| [HADOOP-6180](https://issues.apache.org/jira/browse/HADOOP-6180) | Namenode slowed down when many files with same filename were moved to Trash | Minor | . | Koji Noguchi | Boris Shkolnik | +| [HADOOP-6177](https://issues.apache.org/jira/browse/HADOOP-6177) | FSInputChecker.getPos() would return position greater than the file size | Major | . | Hong Tang | Hong Tang | +| [HADOOP-6175](https://issues.apache.org/jira/browse/HADOOP-6175) | Incorret version compilation with es\_ES.ISO8859-15 locale on Solaris 10 | Major | build | Urko Benito | Urko Benito | +| [HADOOP-6172](https://issues.apache.org/jira/browse/HADOOP-6172) | bin/hadoop version not working | Minor | build | Hong Tang | Hong Tang | +| [HADOOP-6169](https://issues.apache.org/jira/browse/HADOOP-6169) | Removing deprecated method calls in TFile | Minor | . | Hong Tang | Hong Tang | +| [HADOOP-6152](https://issues.apache.org/jira/browse/HADOOP-6152) | Hadoop scripts do not correctly put jars on the classpath | Blocker | scripts | Aaron Kimball | Aaron Kimball | +| [HADOOP-6151](https://issues.apache.org/jira/browse/HADOOP-6151) | The servlets should quote html characters | Critical | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-6142](https://issues.apache.org/jira/browse/HADOOP-6142) | archives relative path changes in common. | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-6138](https://issues.apache.org/jira/browse/HADOOP-6138) | eliminate the depracate warnings introduced by H-5438 | Minor | fs | He Yongqiang | He Yongqiang | +| [HADOOP-6137](https://issues.apache.org/jira/browse/HADOOP-6137) | to fix project specific test-patch requirements | Critical | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-6132](https://issues.apache.org/jira/browse/HADOOP-6132) | RPC client opens an extra connection for VersionedProtocol | Major | ipc | Kan Zhang | Kan Zhang | +| [HADOOP-6131](https://issues.apache.org/jira/browse/HADOOP-6131) | A sysproperty should not be set unless the property is set on the ant command line in build.xml. | Trivial | build | Hong Tang | Hong Tang | +| [HADOOP-6124](https://issues.apache.org/jira/browse/HADOOP-6124) | patchJavacWarnings and trunkJavacWarnings are not consistent. | Critical | build | Tsz Wo Nicholas Sze | Giridharan Kesavan | +| [HADOOP-6123](https://issues.apache.org/jira/browse/HADOOP-6123) | hdfs script does not work after project split. | Major | scripts | Konstantin Shvachko | Sharad Agarwal | +| [HADOOP-6122](https://issues.apache.org/jira/browse/HADOOP-6122) | 64 javac compiler warnings | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6114](https://issues.apache.org/jira/browse/HADOOP-6114) | bug in documentation: org.apache.hadoop.fs.FileStatus.getLen() | Major | documentation | Dmitry Rzhevskiy | Dmitry Rzhevskiy | +| [HADOOP-6112](https://issues.apache.org/jira/browse/HADOOP-6112) | to fix hudsonPatchQueueAdmin for different projects | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-6103](https://issues.apache.org/jira/browse/HADOOP-6103) | Configuration clone constructor does not clone all the members. | Major | conf | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-6096](https://issues.apache.org/jira/browse/HADOOP-6096) | Fix Eclipse project and classpath files following project split | Major | build | Tom White | Tom White | +| [HADOOP-6090](https://issues.apache.org/jira/browse/HADOOP-6090) | GridMix is broke after upgrading random(text)writer to newer mapreduce apis | Major | benchmarks | Arun C Murthy | Amareshwari Sriramadasu | +| [HADOOP-6079](https://issues.apache.org/jira/browse/HADOOP-6079) | In DataTransferProtocol, the serialization of proxySource is not consistent | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6076](https://issues.apache.org/jira/browse/HADOOP-6076) | Forrest documentation compilation is broken because of HADOOP-5913 | Blocker | documentation | Hemanth Yamijala | Hemanth Yamijala | +| [HADOOP-6031](https://issues.apache.org/jira/browse/HADOOP-6031) | Remove @author tags from Java source files | Minor | documentation | Ravi Phulari | Ravi Phulari | +| [HADOOP-6017](https://issues.apache.org/jira/browse/HADOOP-6017) | NameNode and SecondaryNameNode fail to restart because of abnormal filenames. | Blocker | . | Raghu Angadi | Tsz Wo Nicholas Sze | +| [HADOOP-6009](https://issues.apache.org/jira/browse/HADOOP-6009) | S3N listStatus incorrectly returns null instead of empty array when called on empty root | Major | fs/s3 | Ian Nowland | Ian Nowland | +| [HADOOP-6004](https://issues.apache.org/jira/browse/HADOOP-6004) | BlockLocation deserialization is incorrect | Major | fs | Jakob Homan | Jakob Homan | +| [HADOOP-5989](https://issues.apache.org/jira/browse/HADOOP-5989) | streaming tests fails trunk builds | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5981](https://issues.apache.org/jira/browse/HADOOP-5981) | HADOOP-2838 doesnt work as expected | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-5980](https://issues.apache.org/jira/browse/HADOOP-5980) | LD\_LIBRARY\_PATH not passed to tasks spawned off by LinuxTaskController | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5963](https://issues.apache.org/jira/browse/HADOOP-5963) | unnecessary exception catch in NNBench | Minor | test | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-5958](https://issues.apache.org/jira/browse/HADOOP-5958) | Use JDK 1.6 File APIs in DF.java wherever possible | Major | fs | Devaraj Das | Aaron Kimball | +| [HADOOP-5956](https://issues.apache.org/jira/browse/HADOOP-5956) | org.apache.hadoop.hdfsproxy.TestHdfsProxy.testHdfsProxyInterface test fails on trunk | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5954](https://issues.apache.org/jira/browse/HADOOP-5954) | Fix javac warnings in HDFS tests | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5953](https://issues.apache.org/jira/browse/HADOOP-5953) | KosmosFileSystem.isDirectory() should not be deprecated. | Major | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5951](https://issues.apache.org/jira/browse/HADOOP-5951) | StorageInfo needs Apache license header. | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5947](https://issues.apache.org/jira/browse/HADOOP-5947) | org.apache.hadoop.mapred.lib.TestCombineFileInputFormat fails trunk builds | Critical | . | Giridharan Kesavan | Sharad Agarwal | +| [HADOOP-5944](https://issues.apache.org/jira/browse/HADOOP-5944) | BlockManager needs Apache license header. | Major | . | Konstantin Shvachko | Suresh Srinivas | +| [HADOOP-5940](https://issues.apache.org/jira/browse/HADOOP-5940) | trunk eclipse-plugin build fails while trying to copy commons-cli jar from the lib dir | Major | contrib/eclipse-plugin | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5935](https://issues.apache.org/jira/browse/HADOOP-5935) | Hudson's release audit warnings link is broken | Major | build | Tom White | Giridharan Kesavan | +| [HADOOP-5902](https://issues.apache.org/jira/browse/HADOOP-5902) | 4 contrib test cases are failing for the svn committed code | Blocker | . | Abdul Qadeer | | +| [HADOOP-5900](https://issues.apache.org/jira/browse/HADOOP-5900) | Minor correction in HDFS Documentation | Minor | documentation | Ravi Phulari | Ravi Phulari | +| [HADOOP-5899](https://issues.apache.org/jira/browse/HADOOP-5899) | Minor - move info log to the right place to avoid printing unnecessary log | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5895](https://issues.apache.org/jira/browse/HADOOP-5895) | Log message shows -ve number of bytes to be merged in the final merge pass when there are no intermediate merges and merge factor is \> number of segments | Major | . | Ravi Gummadi | Ravi Gummadi | +| [HADOOP-5891](https://issues.apache.org/jira/browse/HADOOP-5891) | If dfs.http.address is default, SecondaryNameNode can't find NameNode | Major | fs | Todd Lipcon | Todd Lipcon | +| [HADOOP-5878](https://issues.apache.org/jira/browse/HADOOP-5878) | Fix hdfs jsp import and Serializable javac warnings | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5877](https://issues.apache.org/jira/browse/HADOOP-5877) | Fix javac warnings in TestHDFSServerPorts, TestCheckpoint, TestNameEditsConfig, TestStartup and TestStorageRestore | Major | test | Jakob Homan | Jakob Homan | +| [HADOOP-5866](https://issues.apache.org/jira/browse/HADOOP-5866) | Move DeprecatedUTF8 to o.a.h.hdfs | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5864](https://issues.apache.org/jira/browse/HADOOP-5864) | Fix DMI and OBL findbugs in packages hdfs and metrics | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5859](https://issues.apache.org/jira/browse/HADOOP-5859) | FindBugs : fix "wait() or sleep() with locks held" warnings in hdfs | Major | . | Kan Zhang | Kan Zhang | +| [HADOOP-5856](https://issues.apache.org/jira/browse/HADOOP-5856) | FindBugs : fix "unsafe multithreaded use of DateFormat" warning in hdfs | Major | . | Kan Zhang | Kan Zhang | +| [HADOOP-5855](https://issues.apache.org/jira/browse/HADOOP-5855) | Fix javac warnings for DisallowedDatanodeException and UnsupportedActionException | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5853](https://issues.apache.org/jira/browse/HADOOP-5853) | Undeprecate HttpServer.addInternalServlet method to fix javac warnings | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5847](https://issues.apache.org/jira/browse/HADOOP-5847) | Streaming unit tests failing for a while on trunk | Major | . | Hemanth Yamijala | Giridharan Kesavan | +| [HADOOP-5845](https://issues.apache.org/jira/browse/HADOOP-5845) | Build successful despite test failure on test-core target | Major | test | Chris Douglas | Sharad Agarwal | +| [HADOOP-5842](https://issues.apache.org/jira/browse/HADOOP-5842) | Fix a few javac warnings under packages fs and util | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-5841](https://issues.apache.org/jira/browse/HADOOP-5841) | Resolve findbugs warnings in DistributedFileSystem.java, DatanodeInfo.java, BlocksMap.java, DataNodeDescriptor.java | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-5836](https://issues.apache.org/jira/browse/HADOOP-5836) | Bug in S3N handling of directory markers using an object with a trailing "/" causes jobs to fail | Major | fs/s3 | Ian Nowland | Ian Nowland | +| [HADOOP-5835](https://issues.apache.org/jira/browse/HADOOP-5835) | Fix findbugs warnings | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5829](https://issues.apache.org/jira/browse/HADOOP-5829) | Fix javac warnings | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5827](https://issues.apache.org/jira/browse/HADOOP-5827) | Remove unwanted file that got checked in by accident | Major | . | Kan Zhang | Kan Zhang | +| [HADOOP-5824](https://issues.apache.org/jira/browse/HADOOP-5824) | remove OP\_READ\_METADATA functionality from Datanode | Major | . | Kan Zhang | Kan Zhang | +| [HADOOP-5823](https://issues.apache.org/jira/browse/HADOOP-5823) | Handling javac "deprecated" warning for using UTF8 | Major | build | Raghu Angadi | Raghu Angadi | +| [HADOOP-5820](https://issues.apache.org/jira/browse/HADOOP-5820) | Fix findbugs warnings for http related codes in hdfs | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5818](https://issues.apache.org/jira/browse/HADOOP-5818) | Revert the renaming from checkSuperuserPrivilege to checkAccess by HADOOP-5643 | Major | . | Tsz Wo Nicholas Sze | Amar Kamat | +| [HADOOP-5809](https://issues.apache.org/jira/browse/HADOOP-5809) | Job submission fails if hadoop.tmp.dir exists | Critical | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5808](https://issues.apache.org/jira/browse/HADOOP-5808) | Fix hdfs un-used import warnings | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5805](https://issues.apache.org/jira/browse/HADOOP-5805) | problem using top level s3 buckets as input/output directories | Major | fs/s3 | Arun Jacob | Ian Nowland | +| [HADOOP-5804](https://issues.apache.org/jira/browse/HADOOP-5804) | neither s3.block.size not fs.s3.block.size are honoured | Major | fs/s3 | Mathieu Poumeyrol | Tom White | +| [HADOOP-5801](https://issues.apache.org/jira/browse/HADOOP-5801) | JobTracker should refresh the hosts list upon recovery | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-5782](https://issues.apache.org/jira/browse/HADOOP-5782) | Make formatting of BlockManager.java similar to FSNamesystem.java to simplify porting patch | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5780](https://issues.apache.org/jira/browse/HADOOP-5780) | Fix slightly confusing log from "-metaSave" on NameNode | Minor | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-5775](https://issues.apache.org/jira/browse/HADOOP-5775) | HdfsProxy Unit Test should not depend on HDFSPROXY\_CONF\_DIR environment | Major | . | zhiyong zhang | zhiyong zhang | +| [HADOOP-5764](https://issues.apache.org/jira/browse/HADOOP-5764) | Hadoop Vaidya test rule (ReadingHDFSFilesAsSideEffect) fails w/ exception if number of map input bytes for a job is zero. | Major | . | Suhas Gogate | | +| [HADOOP-5762](https://issues.apache.org/jira/browse/HADOOP-5762) | distcp does not copy empty directories | Major | . | dhruba borthakur | Rodrigo Schmidt | +| [HADOOP-5737](https://issues.apache.org/jira/browse/HADOOP-5737) | UGI checks in testcases are broken | Major | security, test | Amar Kamat | Amar Kamat | +| [HADOOP-5734](https://issues.apache.org/jira/browse/HADOOP-5734) | HDFS architecture documentation describes outdated placement policy | Minor | documentation | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-5715](https://issues.apache.org/jira/browse/HADOOP-5715) | Should conf/mapred-queue-acls.xml be added to the ignore list? | Major | conf | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5710](https://issues.apache.org/jira/browse/HADOOP-5710) | Counter MAP\_INPUT\_BYTES missing from new mapreduce api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5709](https://issues.apache.org/jira/browse/HADOOP-5709) | Remove the additional synchronization in MapTask.MapOutputBuffer.Buffer.write | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5704](https://issues.apache.org/jira/browse/HADOOP-5704) | Scheduler test code does not compile | Major | test | Chris Douglas | Chris Douglas | +| [HADOOP-5661](https://issues.apache.org/jira/browse/HADOOP-5661) | Resolve findbugs warnings in mapred | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5658](https://issues.apache.org/jira/browse/HADOOP-5658) | Eclipse templates fail out of the box; need updating | Major | build | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-5656](https://issues.apache.org/jira/browse/HADOOP-5656) | Counter for S3N Read Bytes does not work | Minor | fs/s3 | Ian Nowland | Ian Nowland | +| [HADOOP-5652](https://issues.apache.org/jira/browse/HADOOP-5652) | Reduce does not respect in-memory segment memory limit when number of on disk segments == io.sort.factor | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-5650](https://issues.apache.org/jira/browse/HADOOP-5650) | Namenode log that indicates why it is not leaving safemode may be confusing | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-5635](https://issues.apache.org/jira/browse/HADOOP-5635) | distributed cache doesn't work with other distributed file systems | Minor | filecache | Andrew Hitchcock | Andrew Hitchcock | +| [HADOOP-5604](https://issues.apache.org/jira/browse/HADOOP-5604) | TestBinaryPartitioner javac warnings. | Major | test | Konstantin Shvachko | | +| [HADOOP-5592](https://issues.apache.org/jira/browse/HADOOP-5592) | Hadoop Streaming - GzipCodec | Minor | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-5582](https://issues.apache.org/jira/browse/HADOOP-5582) | Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation). | Major | . | Suhas Gogate | Suhas Gogate | +| [HADOOP-5581](https://issues.apache.org/jira/browse/HADOOP-5581) | libhdfs does not get FileNotFoundException | Major | . | Brian Bockelman | Brian Bockelman | +| [HADOOP-5561](https://issues.apache.org/jira/browse/HADOOP-5561) | Javadoc-dev ant target runs out of heap space | Major | build | Jakob Homan | Jakob Homan | +| [HADOOP-5511](https://issues.apache.org/jira/browse/HADOOP-5511) | Add Apache License to EditLogBackupOutputStream | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-5507](https://issues.apache.org/jira/browse/HADOOP-5507) | javadoc warning in JMXGet | Minor | . | Tsz Wo Nicholas Sze | Boris Shkolnik | +| [HADOOP-5491](https://issues.apache.org/jira/browse/HADOOP-5491) | Better control memory usage in contrib/index | Minor | . | Ning Li | Ning Li | +| [HADOOP-5489](https://issues.apache.org/jira/browse/HADOOP-5489) | hadoop-env.sh still refers to java1.5 | Trivial | conf | Steve Loughran | Steve Loughran | +| [HADOOP-5488](https://issues.apache.org/jira/browse/HADOOP-5488) | HADOOP-2721 doesn't clean up descendant processes of a jvm that exits cleanly after running a task successfully | Major | . | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [HADOOP-5486](https://issues.apache.org/jira/browse/HADOOP-5486) | ReliabilityTest does not test lostTrackers, some times. | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5477](https://issues.apache.org/jira/browse/HADOOP-5477) | TestCLI fails | Major | test | Amar Kamat | Jakob Homan | +| [HADOOP-5476](https://issues.apache.org/jira/browse/HADOOP-5476) | calling new SequenceFile.Reader(...) leaves an InputStream open, if the given sequence file is broken | Major | io | Michael Tamm | Michael Tamm | +| [HADOOP-5472](https://issues.apache.org/jira/browse/HADOOP-5472) | Distcp does not support globbing of input paths | Major | . | dhruba borthakur | Rodrigo Schmidt | +| [HADOOP-5462](https://issues.apache.org/jira/browse/HADOOP-5462) | Glibc double free exception thrown when chown syscall fails. | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5458](https://issues.apache.org/jira/browse/HADOOP-5458) | Remove Chukwa from .gitignore | Trivial | . | Chris Douglas | Chris Douglas | +| [HADOOP-5456](https://issues.apache.org/jira/browse/HADOOP-5456) | javadoc warning: can't find restoreFailedStorage() in ClientProtocol | Minor | documentation | Tsz Wo Nicholas Sze | Boris Shkolnik | +| [HADOOP-5442](https://issues.apache.org/jira/browse/HADOOP-5442) | The job history display needs to be paged | Major | . | Owen O'Malley | Amar Kamat | +| [HADOOP-5420](https://issues.apache.org/jira/browse/HADOOP-5420) | Support killing of process groups in LinuxTaskController binary | Major | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [HADOOP-5406](https://issues.apache.org/jira/browse/HADOOP-5406) | Misnamed function in ZlibCompressor.c | Minor | native | Lars Francke | Lars Francke | +| [HADOOP-5386](https://issues.apache.org/jira/browse/HADOOP-5386) | To Probe free ports dynamically for Unit test to replace fixed ports | Major | . | zhiyong zhang | zhiyong zhang | +| [HADOOP-5347](https://issues.apache.org/jira/browse/HADOOP-5347) | bbp example cannot be run. | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5341](https://issues.apache.org/jira/browse/HADOOP-5341) | hadoop-daemon isn't compatible after HADOOP-4868 | Major | scripts | Owen O'Malley | Sharad Agarwal | +| [HADOOP-5322](https://issues.apache.org/jira/browse/HADOOP-5322) | comments in JobInProgress related to TaskCommitThread are not valid | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5314](https://issues.apache.org/jira/browse/HADOOP-5314) | needToSave incorrectly calculated in loadFSImage() | Major | . | Konstantin Shvachko | Jakob Homan | +| [HADOOP-5300](https://issues.apache.org/jira/browse/HADOOP-5300) | "ant javadoc-dev" does not work | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5278](https://issues.apache.org/jira/browse/HADOOP-5278) | Finish time of a TIP is incorrectly logged to the jobhistory upon jobtracker restart | Major | . | Amar Kamat | Amar Kamat | +| [HADOOP-5276](https://issues.apache.org/jira/browse/HADOOP-5276) | Upon a lost tracker, the task's start time is reset to 0 | Critical | . | Amar Kamat | Amar Kamat | +| [HADOOP-5273](https://issues.apache.org/jira/browse/HADOOP-5273) | License header missing in TestJobInProgress.java | Minor | documentation | Amar Kamat | Jakob Homan | +| [HADOOP-5253](https://issues.apache.org/jira/browse/HADOOP-5253) | to remove duplicate calls to the cn-docs target. | Minor | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5252](https://issues.apache.org/jira/browse/HADOOP-5252) | Streaming overrides -inputformat option | Major | . | Klaas Bosteels | Klaas Bosteels | +| [HADOOP-5251](https://issues.apache.org/jira/browse/HADOOP-5251) | TestHdfsProxy and TestProxyUgiManager frequently fail | Critical | . | Johan Oskarsson | Nigel Daley | +| [HADOOP-5229](https://issues.apache.org/jira/browse/HADOOP-5229) | duplicate variables in build.xml hadoop.version vs version let build fails at assert-hadoop-jar-exists | Trivial | build | Stefan Groschupf | Stefan Groschupf | +| [HADOOP-5226](https://issues.apache.org/jira/browse/HADOOP-5226) | Add license headers to html and jsp files | Major | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5218](https://issues.apache.org/jira/browse/HADOOP-5218) | libhdfs unit test failed because it was unable to start namenode/datanode | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-5213](https://issues.apache.org/jira/browse/HADOOP-5213) | BZip2CompressionOutputStream NullPointerException | Blocker | io | Zheng Shao | Zheng Shao | +| [HADOOP-5212](https://issues.apache.org/jira/browse/HADOOP-5212) | cygwin path translation not happening correctly after Hadoop-4868 | Major | scripts | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5209](https://issues.apache.org/jira/browse/HADOOP-5209) | Update year to 2009 for javadoc | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5206](https://issues.apache.org/jira/browse/HADOOP-5206) | All "unprotected\*" methods of FSDirectory should synchronize on the root. | Major | . | Konstantin Shvachko | Jakob Homan | +| [HADOOP-5204](https://issues.apache.org/jira/browse/HADOOP-5204) | hudson trunk build failure due to autoheader failure in create-c++-configure-libhdfs task | Blocker | build | Lee Tucker | Sreekanth Ramakrishnan | +| [HADOOP-5203](https://issues.apache.org/jira/browse/HADOOP-5203) | TT's version build is too restrictive | Major | . | Runping Qi | Rick Cox | +| [HADOOP-5198](https://issues.apache.org/jira/browse/HADOOP-5198) | NPE in Shell.runCommand() | Blocker | util | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-5194](https://issues.apache.org/jira/browse/HADOOP-5194) | DiskErrorException in TaskTracker when running a job | Blocker | . | Tsz Wo Nicholas Sze | Ravi Gummadi | +| [HADOOP-5191](https://issues.apache.org/jira/browse/HADOOP-5191) | After creation and startup of the hadoop namenode on AIX or Solaris, you will only be allowed to connect to the namenode via hostname but not IP. | Minor | . | Bill Habermaas | Raghu Angadi | +| [HADOOP-5172](https://issues.apache.org/jira/browse/HADOOP-5172) | Chukwa : TestAgentConfig.testInitAdaptors\_vs\_Checkpoint regularly fails | Major | test | Raghu Angadi | Jerome Boulon | +| [HADOOP-5149](https://issues.apache.org/jira/browse/HADOOP-5149) | HistoryViewer throws IndexOutOfBoundsException when there are files or directories not confrming to log file name convention | Minor | . | Hong Tang | Hong Tang | +| [HADOOP-5148](https://issues.apache.org/jira/browse/HADOOP-5148) | make watchdog disable-able | Minor | . | Ari Rabkin | Ari Rabkin | +| [HADOOP-5138](https://issues.apache.org/jira/browse/HADOOP-5138) | Current Chukwa Trunk failed contrib unit tests. | Critical | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5113](https://issues.apache.org/jira/browse/HADOOP-5113) | logcondense should delete hod logs for a user , whose username has any of the characters in the value passed to "-l" options | Major | contrib/hod | Peeyush Bishnoi | Peeyush Bishnoi | +| [HADOOP-5103](https://issues.apache.org/jira/browse/HADOOP-5103) | Too many logs saying "Adding new node" on JobClient console | Major | . | Amareshwari Sriramadasu | Jothi Padmanabhan | +| [HADOOP-5100](https://issues.apache.org/jira/browse/HADOOP-5100) | Chukwa Log4JMetricsContext class should append new log to current log file | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5095](https://issues.apache.org/jira/browse/HADOOP-5095) | chukwa watchdog does not monitor the system correctly | Major | . | Eric Yang | Jerome Boulon | +| [HADOOP-5078](https://issues.apache.org/jira/browse/HADOOP-5078) | Broken AMI/AKI for ec2 on hadoop | Major | contrib/cloud | Mathieu Poumeyrol | Tom White | +| [HADOOP-5072](https://issues.apache.org/jira/browse/HADOOP-5072) | testSequenceFileGzipCodec won't pass without native gzip codec | Major | test | Zheng Shao | Zheng Shao | +| [HADOOP-5070](https://issues.apache.org/jira/browse/HADOOP-5070) | Update the year for the copyright to 2009 | Blocker | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5050](https://issues.apache.org/jira/browse/HADOOP-5050) | TestDFSShell fails intermittently | Major | . | Amareshwari Sriramadasu | Jakob Homan | +| [HADOOP-5039](https://issues.apache.org/jira/browse/HADOOP-5039) | Hourly&daily rolling are not using the right path | Major | . | Jerome Boulon | Jerome Boulon | +| [HADOOP-5032](https://issues.apache.org/jira/browse/HADOOP-5032) | CHUKWA\_CONF\_DIR environment variable needs to be exported to shell script | Major | . | Eric Yang | Eric Yang | +| [HADOOP-5031](https://issues.apache.org/jira/browse/HADOOP-5031) | metrics aggregation is incorrect in database | Major | . | Eric Yang | Eric Yang | +| [HADOOP-5017](https://issues.apache.org/jira/browse/HADOOP-5017) | NameNode.namesystem should be private | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4985](https://issues.apache.org/jira/browse/HADOOP-4985) | IOException is abused in FSDirectory | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4975](https://issues.apache.org/jira/browse/HADOOP-4975) | CompositeRecordReader: ClassLoader set in JobConf is not passed onto WrappedRecordReaders | Major | . | Jingkei Ly | Jingkei Ly | +| [HADOOP-4960](https://issues.apache.org/jira/browse/HADOOP-4960) | Hadoop metrics are showing in irregular intervals | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4959](https://issues.apache.org/jira/browse/HADOOP-4959) | System metrics does not output correctly for Redhat 5.1. | Major | . | Eric Yang | Eric Yang | +| [HADOOP-4948](https://issues.apache.org/jira/browse/HADOOP-4948) | ant test-patch does not work | Major | scripts | Tsz Wo Nicholas Sze | Giridharan Kesavan | +| [HADOOP-4933](https://issues.apache.org/jira/browse/HADOOP-4933) | ConcurrentModificationException in JobHistory.java | Blocker | . | Amar Kamat | Amar Kamat | +| [HADOOP-4864](https://issues.apache.org/jira/browse/HADOOP-4864) | -libjars with multiple jars broken when client and cluster reside on different OSs | Minor | filecache | Stuart White | Amareshwari Sriramadasu | +| [HADOOP-4584](https://issues.apache.org/jira/browse/HADOOP-4584) | Slow generation of blockReport at DataNode causes delay of sending heartbeat to NameNode | Major | . | Hairong Kuang | Suresh Srinivas | +| [HADOOP-4220](https://issues.apache.org/jira/browse/HADOOP-4220) | Job Restart tests take 10 minutes, can time out very easily | Blocker | test | Steve Loughran | Amar Kamat | +| [HADOOP-4045](https://issues.apache.org/jira/browse/HADOOP-4045) | Increment checkpoint if we see failures in rollEdits | Critical | . | Lohit Vijayarenu | Boris Shkolnik | +| [HADOOP-4041](https://issues.apache.org/jira/browse/HADOOP-4041) | IsolationRunner does not work as documented | Major | documentation | Yuri Pradkin | Philip Zeyliger | +| [HADOOP-3426](https://issues.apache.org/jira/browse/HADOOP-3426) | Datanode does not start up if the local machines DNS isnt working right and dfs.datanode.dns.interface==default | Minor | . | Steve Loughran | Steve Loughran | +| [HADOOP-3327](https://issues.apache.org/jira/browse/HADOOP-3327) | Shuffling fetchers waited too long between map output fetch re-tries | Major | . | Runping Qi | Amareshwari Sriramadasu | +| [HADOOP-3205](https://issues.apache.org/jira/browse/HADOOP-3205) | Read multiple chunks directly from FSInputChecker subclass into user buffers | Major | fs | Raghu Angadi | Todd Lipcon | +| [HADOOP-2413](https://issues.apache.org/jira/browse/HADOOP-2413) | Is FSNamesystem.fsNamesystemObject unique? | Minor | . | Tsz Wo Nicholas Sze | Konstantin Shvachko | +| [HADOOP-2366](https://issues.apache.org/jira/browse/HADOOP-2366) | Space in the value for dfs.data.dir can cause great problems | Major | conf | Ted Dunning | Michele Catasta | +| [HADOOP-2337](https://issues.apache.org/jira/browse/HADOOP-2337) | Trash never closes FileSystem | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1313](https://issues.apache.org/jira/browse/HDFS-1313) | HdfsProxy changes from HDFS-481 missed in y20.1xx | Major | contrib/hdfsproxy | Rohini Palaniswamy | Rohini Palaniswamy | +| [HDFS-1311](https://issues.apache.org/jira/browse/HDFS-1311) | Running tests with 'testcase' cause triple execution of the same test case | Minor | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1299](https://issues.apache.org/jira/browse/HDFS-1299) | 'compile-fault-inject' never should be called directly. | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1292](https://issues.apache.org/jira/browse/HDFS-1292) | Allow artifacts to be published to the staging Apache Nexus Maven Repository | Blocker | build | Tom White | Giridharan Kesavan | +| [HDFS-1267](https://issues.apache.org/jira/browse/HDFS-1267) | fuse-dfs does not compile | Critical | fuse-dfs | Tom White | Devaraj Das | +| [HDFS-1258](https://issues.apache.org/jira/browse/HDFS-1258) | Clearing namespace quota on "/" corrupts FS image | Blocker | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1256](https://issues.apache.org/jira/browse/HDFS-1256) | libhdfs is missing from the tarball | Blocker | . | Tom White | Tom White | +| [HDFS-1255](https://issues.apache.org/jira/browse/HDFS-1255) | test-libhdfs.sh fails | Blocker | test | Tom White | Tom White | +| [HDFS-1212](https://issues.apache.org/jira/browse/HDFS-1212) | Harmonize HDFS JAR library versions with Common | Blocker | build | Tom White | Tom White | +| [HDFS-1193](https://issues.apache.org/jira/browse/HDFS-1193) | -mvn-system-deploy target is broken which inturn fails the mvn-deploy task leading to unstable mapreduce build. | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HDFS-1181](https://issues.apache.org/jira/browse/HDFS-1181) | Move configuration and script files post project split | Blocker | scripts | Tom White | Tom White | +| [HDFS-1173](https://issues.apache.org/jira/browse/HDFS-1173) | Fix references to 0.22 in 0.21 branch | Major | . | Tom White | Tom White | +| [HDFS-1159](https://issues.apache.org/jira/browse/HDFS-1159) | clean-cache target removes wrong ivy cache | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1104](https://issues.apache.org/jira/browse/HDFS-1104) | Fsck triggers full GC on NameNode | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1101](https://issues.apache.org/jira/browse/HDFS-1101) | TestDiskError.testLocalDirs() fails | Major | . | Konstantin Shvachko | Chris Douglas | +| [HDFS-1088](https://issues.apache.org/jira/browse/HDFS-1088) | Prevent renaming a symlink to its target | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-1074](https://issues.apache.org/jira/browse/HDFS-1074) | TestProxyUtil fails | Major | contrib/hdfsproxy | Tsz Wo Nicholas Sze | Srikanth Sundarrajan | +| [HDFS-1072](https://issues.apache.org/jira/browse/HDFS-1072) | AlreadyBeingCreatedException with HDFS\_NameNode as the lease holder | Major | hdfs-client, namenode | Tsz Wo Nicholas Sze | Erik Steffl | +| [HDFS-1046](https://issues.apache.org/jira/browse/HDFS-1046) | Build fails trying to download an old version of tomcat | Blocker | build, contrib/hdfsproxy | gary murry | Srikanth Sundarrajan | +| [HDFS-1041](https://issues.apache.org/jira/browse/HDFS-1041) | DFSClient does not retry in getFileChecksum(..) | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1015](https://issues.apache.org/jira/browse/HDFS-1015) | Intermittent failure in TestSecurityTokenEditLog | Major | namenode, test | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1014](https://issues.apache.org/jira/browse/HDFS-1014) | Error in reading delegation tokens from edit logs. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1010](https://issues.apache.org/jira/browse/HDFS-1010) | HDFSProxy: Retrieve group information from UnixUserGroupInformation instead of LdapEntry | Major | contrib/hdfsproxy | Srikanth Sundarrajan | Srikanth Sundarrajan | +| [HDFS-1000](https://issues.apache.org/jira/browse/HDFS-1000) | libhdfs needs to be updated to use the new UGI | Blocker | . | Devaraj Das | Devaraj Das | +| [HDFS-995](https://issues.apache.org/jira/browse/HDFS-995) | Replace usage of FileStatus#isDir() | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-966](https://issues.apache.org/jira/browse/HDFS-966) | NameNode recovers lease even in safemode | Major | namenode | dhruba borthakur | dhruba borthakur | +| [HDFS-965](https://issues.apache.org/jira/browse/HDFS-965) | TestDelegationToken fails in trunk | Major | test | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-961](https://issues.apache.org/jira/browse/HDFS-961) | dfs\_readdir incorrectly parses paths | Major | fuse-dfs | Eli Collins | Eli Collins | +| [HDFS-939](https://issues.apache.org/jira/browse/HDFS-939) | libhdfs test is broken | Blocker | libhdfs | Eli Collins | Eli Collins | +| [HDFS-938](https://issues.apache.org/jira/browse/HDFS-938) | Replace calls to UGI.getUserName() with UGI.getShortUserName() | Major | hdfs-client, namenode | Jakob Homan | Jakob Homan | +| [HDFS-927](https://issues.apache.org/jira/browse/HDFS-927) | DFSInputStream retries too many times for new block locations | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-913](https://issues.apache.org/jira/browse/HDFS-913) | TestRename won't run automatically from 'run-test-hdfs-faul-inject' target | Major | test | Konstantin Boudnik | Suresh Srinivas | +| [HDFS-909](https://issues.apache.org/jira/browse/HDFS-909) | Race condition between rollEditLog or rollFSImage ant FSEditsLog.write operations corrupts edits log | Blocker | namenode | Cosmin Lehene | Todd Lipcon | +| [HDFS-897](https://issues.apache.org/jira/browse/HDFS-897) | ReplicasMap remove has a bug in generation stamp comparison | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-894](https://issues.apache.org/jira/browse/HDFS-894) | DatanodeID.ipcPort is not updated when existing node re-registers | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-885](https://issues.apache.org/jira/browse/HDFS-885) | Datanode toString() NPEs on null dnRegistration | Minor | datanode | Steve Loughran | Steve Loughran | +| [HDFS-880](https://issues.apache.org/jira/browse/HDFS-880) | TestNNLeaseRecovery fails on windows | Major | test | Konstantin Shvachko | Konstantin Boudnik | +| [HDFS-877](https://issues.apache.org/jira/browse/HDFS-877) | Client-driven block verification not functioning | Major | hdfs-client, test | Todd Lipcon | Todd Lipcon | +| [HDFS-868](https://issues.apache.org/jira/browse/HDFS-868) | Link to Hadoop Upgrade Wiki is broken | Trivial | documentation | Chris A. Mattmann | | +| [HDFS-861](https://issues.apache.org/jira/browse/HDFS-861) | fuse-dfs does not support O\_RDWR | Minor | fuse-dfs | Brian Bockelman | Brian Bockelman | +| [HDFS-859](https://issues.apache.org/jira/browse/HDFS-859) | fuse-dfs utime behavior causes issues with tar | Minor | fuse-dfs | Brian Bockelman | Brian Bockelman | +| [HDFS-858](https://issues.apache.org/jira/browse/HDFS-858) | Incorrect return codes for fuse-dfs | Minor | fuse-dfs | Brian Bockelman | Brian Bockelman | +| [HDFS-857](https://issues.apache.org/jira/browse/HDFS-857) | Incorrect type for fuse-dfs capacity can cause "df" to return negative values on 32-bit machines | Minor | fuse-dfs | Brian Bockelman | Brian Bockelman | +| [HDFS-856](https://issues.apache.org/jira/browse/HDFS-856) | Hardcoded replication level for new files in fuse-dfs | Minor | fuse-dfs | Brian Bockelman | Brian Bockelman | +| [HDFS-849](https://issues.apache.org/jira/browse/HDFS-849) | TestFiDataTransferProtocol2#pipeline\_Fi\_18 sometimes fails | Major | test | Hairong Kuang | Hairong Kuang | +| [HDFS-840](https://issues.apache.org/jira/browse/HDFS-840) | Update File Context tests to use FileContextTestHelper | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-825](https://issues.apache.org/jira/browse/HDFS-825) | Build fails to pull latest hadoop-core-\* artifacts | Critical | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-824](https://issues.apache.org/jira/browse/HDFS-824) | Stop lease checker in TestReadWhileWriting | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-823](https://issues.apache.org/jira/browse/HDFS-823) | In Checkpointer the getImage servlet is added to public rather than internal servlet list | Major | namenode | Jakob Homan | Jakob Homan | +| [HDFS-812](https://issues.apache.org/jira/browse/HDFS-812) | FSNamesystem#internalReleaseLease throws NullPointerException on a single-block file's lease recovery | Blocker | namenode | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-802](https://issues.apache.org/jira/browse/HDFS-802) | Update Eclipse configuration to match changes to Ivy configuration | Major | build | Edwin Chan | Edward J. Yoon | +| [HDFS-797](https://issues.apache.org/jira/browse/HDFS-797) | TestHDFSCLI much slower after HDFS-265 merge | Blocker | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-792](https://issues.apache.org/jira/browse/HDFS-792) | TestHDFSCLI is failing | Blocker | test | Konstantin Boudnik | Todd Lipcon | +| [HDFS-791](https://issues.apache.org/jira/browse/HDFS-791) | Build is broken after HDFS-787 patch has been applied | Blocker | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-787](https://issues.apache.org/jira/browse/HDFS-787) | Make the versions of libraries consistent | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HDFS-785](https://issues.apache.org/jira/browse/HDFS-785) | Missing license header in java source files. | Minor | documentation | Ravi Phulari | Ravi Phulari | +| [HDFS-783](https://issues.apache.org/jira/browse/HDFS-783) | libhdfs tests brakes code coverage runs with Clover | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-781](https://issues.apache.org/jira/browse/HDFS-781) | Metrics PendingDeletionBlocks is not decremented | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-775](https://issues.apache.org/jira/browse/HDFS-775) | FSDataset calls getCapacity() twice -bug? | Minor | datanode | Steve Loughran | Steve Loughran | +| [HDFS-774](https://issues.apache.org/jira/browse/HDFS-774) | Intermittent race condition in TestFiPipelines | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-763](https://issues.apache.org/jira/browse/HDFS-763) | DataBlockScanner reporting of bad blocks is slightly misleading | Major | datanode | dhruba borthakur | dhruba borthakur | +| [HDFS-762](https://issues.apache.org/jira/browse/HDFS-762) | Trying to start the balancer throws a NPE | Major | . | Cristian Ivascu | Cristian Ivascu | +| [HDFS-761](https://issues.apache.org/jira/browse/HDFS-761) | Failure to process rename operation from edits log due to quota verification | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-760](https://issues.apache.org/jira/browse/HDFS-760) | "fs -put" fails if dfs.umask is set to 63 | Major | . | Tsz Wo Nicholas Sze | | +| [HDFS-757](https://issues.apache.org/jira/browse/HDFS-757) | Unit tests failure for RAID | Major | contrib/raid | dhruba borthakur | dhruba borthakur | +| [HDFS-756](https://issues.apache.org/jira/browse/HDFS-756) | libhdfs unit tests do not run | Critical | libhdfs | dhruba borthakur | Eli Collins | +| [HDFS-751](https://issues.apache.org/jira/browse/HDFS-751) | TestCrcCorruption succeeds but is not testing anything of value | Major | . | dhruba borthakur | dhruba borthakur | +| [HDFS-750](https://issues.apache.org/jira/browse/HDFS-750) | TestRename build failure | Blocker | build | Suresh Srinivas | Suresh Srinivas | +| [HDFS-741](https://issues.apache.org/jira/browse/HDFS-741) | TestHFlush test doesn't seek() past previously written part of the file | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-735](https://issues.apache.org/jira/browse/HDFS-735) | TestReadWhileWriting has wrong line termination symbols | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-726](https://issues.apache.org/jira/browse/HDFS-726) | Eclipse .classpath template has outdated jar files and is missing some new ones. | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-725](https://issues.apache.org/jira/browse/HDFS-725) | Support the build error fix for HADOOP-6327 | Major | test | Sanjay Radia | Sanjay Radia | +| [HDFS-724](https://issues.apache.org/jira/browse/HDFS-724) | Pipeline close hangs if one of the datanode is not responsive. | Blocker | datanode, hdfs-client | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HDFS-722](https://issues.apache.org/jira/browse/HDFS-722) | The pointcut callCreateBlockWriteStream in FSDatasetAspects is broken | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-720](https://issues.apache.org/jira/browse/HDFS-720) | NPE in BlockReceiver$PacketResponder.run(BlockReceiver.java:923) | Major | datanode | stack | | +| [HDFS-709](https://issues.apache.org/jira/browse/HDFS-709) | TestDFSShell failure | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-706](https://issues.apache.org/jira/browse/HDFS-706) | Intermittent failures in TestFiHFlush | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-699](https://issues.apache.org/jira/browse/HDFS-699) | Primary datanode should compare replicas' on disk lengths | Major | datanode | Tsz Wo Nicholas Sze | Hairong Kuang | +| [HDFS-695](https://issues.apache.org/jira/browse/HDFS-695) | RaidNode should read in configuration from hdfs-site.xml | Major | contrib/raid | dhruba borthakur | dhruba borthakur | +| [HDFS-691](https://issues.apache.org/jira/browse/HDFS-691) | Limitation on java.io.InputStream.available() | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-690](https://issues.apache.org/jira/browse/HDFS-690) | TestAppend2#testComplexAppend failed on "Too many open files" | Blocker | test | Hairong Kuang | Hairong Kuang | +| [HDFS-688](https://issues.apache.org/jira/browse/HDFS-688) | Add configuration resources to DFSAdmin | Major | hdfs-client | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-682](https://issues.apache.org/jira/browse/HDFS-682) | TestBlockUnderConstruction fails | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-679](https://issues.apache.org/jira/browse/HDFS-679) | Appending to a partial chunk incorrectly assumes the first packet fills up the partial chunk | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-677](https://issues.apache.org/jira/browse/HDFS-677) | Rename failure due to quota results in deletion of src directory | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-673](https://issues.apache.org/jira/browse/HDFS-673) | BlockReceiver#PacketResponder should not remove a packet from the ack queue before its ack is sent | Blocker | datanode | Hairong Kuang | Hairong Kuang | +| [HDFS-647](https://issues.apache.org/jira/browse/HDFS-647) | Internal server errors | Major | . | gary murry | Owen O'Malley | +| [HDFS-646](https://issues.apache.org/jira/browse/HDFS-646) | missing test-contrib ant target would break hudson patch test process | Blocker | build | Giridharan Kesavan | Giridharan Kesavan | +| [HDFS-641](https://issues.apache.org/jira/browse/HDFS-641) | Move all of the benchmarks and tests that depend on mapreduce to mapreduce | Blocker | test | Owen O'Malley | Owen O'Malley | +| [HDFS-640](https://issues.apache.org/jira/browse/HDFS-640) | TestHDFSFileContextMainOperations uses old FileContext.mkdirs(..) | Major | test | Tsz Wo Nicholas Sze | Suresh Srinivas | +| [HDFS-638](https://issues.apache.org/jira/browse/HDFS-638) | The build.xml refences jars that don't exist | Major | build | Owen O'Malley | Owen O'Malley | +| [HDFS-637](https://issues.apache.org/jira/browse/HDFS-637) | DataNode sends a Success ack when block write fails | Blocker | datanode | Hairong Kuang | Hairong Kuang | +| [HDFS-629](https://issues.apache.org/jira/browse/HDFS-629) | Remove ReplicationTargetChooser.java along with fixing import warnings. | Major | namenode | Konstantin Shvachko | dhruba borthakur | +| [HDFS-625](https://issues.apache.org/jira/browse/HDFS-625) | ListPathsServlet throws NullPointerException | Major | namenode | Tsz Wo Nicholas Sze | Suresh Srinivas | +| [HDFS-622](https://issues.apache.org/jira/browse/HDFS-622) | checkMinReplication should count only live node. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-615](https://issues.apache.org/jira/browse/HDFS-615) | TestLargeDirectoryDelete fails with NullPointerException | Blocker | namenode | Eli Collins | | +| [HDFS-614](https://issues.apache.org/jira/browse/HDFS-614) | TestDatanodeBlockScanner obtain should data-node directories directly from MiniDFSCluster | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-612](https://issues.apache.org/jira/browse/HDFS-612) | FSDataset should not use org.mortbay.log.Log | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-611](https://issues.apache.org/jira/browse/HDFS-611) | Heartbeats times from Datanodes increase when there are plenty of blocks to delete | Major | datanode | dhruba borthakur | Zheng Shao | +| [HDFS-609](https://issues.apache.org/jira/browse/HDFS-609) | Create a file with the append flag does not work in HDFS | Blocker | . | Hairong Kuang | Tom White | +| [HDFS-606](https://issues.apache.org/jira/browse/HDFS-606) | ConcurrentModificationException in invalidateCorruptReplicas() | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-601](https://issues.apache.org/jira/browse/HDFS-601) | TestBlockReport should obtain data directories from MiniHDFSCluster | Major | test | Konstantin Shvachko | Konstantin Boudnik | +| [HDFS-596](https://issues.apache.org/jira/browse/HDFS-596) | Memory leak in libhdfs: hdfsFreeFileInfo() in libhdfs does not free memory for mOwner and mGroup | Blocker | fuse-dfs | Zhang Bingjun | Zhang Bingjun | +| [HDFS-587](https://issues.apache.org/jira/browse/HDFS-587) | Test programs support only default queue. | Major | test | Sreekanth Ramakrishnan | Erik Steffl | +| [HDFS-586](https://issues.apache.org/jira/browse/HDFS-586) | TestBlocksWithNotEnoughRacks fails | Major | test | Hairong Kuang | Jitendra Nath Pandey | +| [HDFS-568](https://issues.apache.org/jira/browse/HDFS-568) | TestServiceLevelAuthorization fails on latest build in Hudson | Minor | test | gary murry | Amareshwari Sriramadasu | +| [HDFS-553](https://issues.apache.org/jira/browse/HDFS-553) | BlockSender reports wrong failed position in ChecksumException | Major | datanode | Hairong Kuang | Hairong Kuang | +| [HDFS-534](https://issues.apache.org/jira/browse/HDFS-534) | Required avro classes are missing | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-525](https://issues.apache.org/jira/browse/HDFS-525) | ListPathsServlet.java uses static SimpleDateFormat that has threading issues | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-500](https://issues.apache.org/jira/browse/HDFS-500) | Fix lingering and new javac warnings | Minor | namenode | Jakob Homan | Jakob Homan | +| [HDFS-489](https://issues.apache.org/jira/browse/HDFS-489) | Updated TestHDFSCLI for changes from HADOOP-6139 | Major | test | Jakob Homan | Jakob Homan | +| [HDFS-484](https://issues.apache.org/jira/browse/HDFS-484) | bin-package and package doesnt seem to package any jar file | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HDFS-483](https://issues.apache.org/jira/browse/HDFS-483) | Data transfer (aka pipeline) implementation cannot tolerate exceptions | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | | +| [HDFS-482](https://issues.apache.org/jira/browse/HDFS-482) | change HsftpFileSystem's ssl.client.do.not.authenticate.server configuration setting to ssl-client.xml | Major | contrib/hdfsproxy | zhiyong zhang | Srikanth Sundarrajan | +| [HDFS-481](https://issues.apache.org/jira/browse/HDFS-481) | Bug Fixes + HdfsProxy to use proxy user to impresonate the real user | Major | contrib/hdfsproxy | zhiyong zhang | Srikanth Sundarrajan | +| [HDFS-480](https://issues.apache.org/jira/browse/HDFS-480) | Typo in jar name in build.xml | Major | build, scripts | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-472](https://issues.apache.org/jira/browse/HDFS-472) | Document hdfsproxy design and set-up guide | Major | contrib/hdfsproxy | zhiyong zhang | zhiyong zhang | +| [HDFS-466](https://issues.apache.org/jira/browse/HDFS-466) | hdfs\_write infinite loop when dfs fails and cannot write files \> 2 GB | Major | . | Pete Wyckoff | Pete Wyckoff | +| [HDFS-464](https://issues.apache.org/jira/browse/HDFS-464) | Memory leaks in libhdfs | Blocker | libhdfs | Christian Kunz | Christian Kunz | +| [HDFS-463](https://issues.apache.org/jira/browse/HDFS-463) | CreateEditsLog utility broken due to FSImage URL scheme check | Major | tools | Suresh Srinivas | Suresh Srinivas | +| [HDFS-462](https://issues.apache.org/jira/browse/HDFS-462) | Unit tests not working under Windows | Major | namenode, test | Luca Telloli | Jakob Homan | +| [HDFS-456](https://issues.apache.org/jira/browse/HDFS-456) | Problems with dfs.name.edits.dirs as URI | Blocker | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-454](https://issues.apache.org/jira/browse/HDFS-454) | HDFS workflow in JIRA does not match MAPREDUCE, HADOOP | Major | . | Aaron Kimball | Owen O'Malley | +| [HDFS-446](https://issues.apache.org/jira/browse/HDFS-446) | Offline Image Viewer Ls visitor incorrectly says 'output file' instead of 'input file' | Minor | test, tools | Jakob Homan | Jakob Homan | +| [HDFS-445](https://issues.apache.org/jira/browse/HDFS-445) | pread() fails when cached block locations are no longer valid | Major | . | Kan Zhang | Kan Zhang | +| [HDFS-441](https://issues.apache.org/jira/browse/HDFS-441) | TestFTPFileSystem fails | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-440](https://issues.apache.org/jira/browse/HDFS-440) | javadoc warnings: broken links | Major | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-439](https://issues.apache.org/jira/browse/HDFS-439) | HADOOP-5961 is incorrectly committed. | Blocker | . | Raghu Angadi | Raghu Angadi | +| [HDFS-438](https://issues.apache.org/jira/browse/HDFS-438) | Improve help message for quotas | Minor | . | Raghu Angadi | Raghu Angadi | +| [HDFS-423](https://issues.apache.org/jira/browse/HDFS-423) | Unbreak FUSE build and fuse\_dfs\_wrapper.sh | Major | fuse-dfs | Giridharan Kesavan | Eli Collins | +| [HDFS-415](https://issues.apache.org/jira/browse/HDFS-415) | Unchecked exception thrown inside of BlockReceiver cause some threads hang | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-195](https://issues.apache.org/jira/browse/HDFS-195) | Need to handle access token expiration when re-establishing the pipeline for dfs write | Major | . | Kan Zhang | Kan Zhang | +| [HDFS-192](https://issues.apache.org/jira/browse/HDFS-192) | TestBackupNode sometimes fails | Major | namenode | Tsz Wo Nicholas Sze | Konstantin Shvachko | +| [HDFS-187](https://issues.apache.org/jira/browse/HDFS-187) | TestStartup fails if hdfs is running in the same machine | Major | test | Tsz Wo Nicholas Sze | Todd Lipcon | +| [HDFS-181](https://issues.apache.org/jira/browse/HDFS-181) | INode.getPathComponents throws NPE when given a non-absolute path | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-167](https://issues.apache.org/jira/browse/HDFS-167) | DFSClient continues to retry indefinitely | Minor | hdfs-client | Derek Wollenstein | Bill Zeller | +| [HDFS-145](https://issues.apache.org/jira/browse/HDFS-145) | FSNameSystem#addStoredBlock does not handle inconsistent block length correctly | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-127](https://issues.apache.org/jira/browse/HDFS-127) | DFSClient block read failures cause open DFSInputStream to become unusable | Major | hdfs-client | Igor Bolotin | Igor Bolotin | +| [HDFS-119](https://issues.apache.org/jira/browse/HDFS-119) | logSync() may block NameNode forever. | Major | namenode | Konstantin Shvachko | Suresh Srinivas | +| [HDFS-101](https://issues.apache.org/jira/browse/HDFS-101) | DFS write pipeline : DFSClient sometimes does not detect second datanode failure | Blocker | datanode | Raghu Angadi | Hairong Kuang | +| [HDFS-95](https://issues.apache.org/jira/browse/HDFS-95) | UnknownHostException if the system can't determine its own name and you go DNS.getIPs("name-of-an-unknown-interface"); | Major | . | Steve Loughran | Steve Loughran | +| [HDFS-94](https://issues.apache.org/jira/browse/HDFS-94) | The "Heap Size" in HDFS web ui may not be accurate | Major | . | Tsz Wo Nicholas Sze | Dmytro Molkov | +| [HDFS-76](https://issues.apache.org/jira/browse/HDFS-76) | Namespace quota exceeded message unclear | Major | . | eric baldeschwieler | Boris Shkolnik | +| [HDFS-29](https://issues.apache.org/jira/browse/HDFS-29) | In Datanode, update block may fail due to length inconsistency | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-15](https://issues.apache.org/jira/browse/HDFS-15) | Rack replication policy can be violated for over replicated blocks | Critical | . | Hairong Kuang | Jitendra Nath Pandey | +| [MAPREDUCE-2014](https://issues.apache.org/jira/browse/MAPREDUCE-2014) | Remove task-controller from 0.21 branch | Major | security | Tom White | Tom White | +| [MAPREDUCE-2012](https://issues.apache.org/jira/browse/MAPREDUCE-2012) | Some contrib tests fail in branch 0.21 and trunk | Blocker | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1980](https://issues.apache.org/jira/browse/MAPREDUCE-1980) | TaskAttemptUnsuccessfulCompletionEvent.java incorrectly logs MAP\_ATTEMPT\_KILLED as event type for reduce tasks | Major | . | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1942](https://issues.apache.org/jira/browse/MAPREDUCE-1942) | 'compile-fault-inject' should never be called directly. | Minor | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1926](https://issues.apache.org/jira/browse/MAPREDUCE-1926) | MapReduce distribution is missing build-utils.xml | Blocker | build | Tom White | Tom White | +| [MAPREDUCE-1920](https://issues.apache.org/jira/browse/MAPREDUCE-1920) | Job.getCounters() returns null when using a cluster | Critical | . | Aaron Kimball | Tom White | +| [MAPREDUCE-1885](https://issues.apache.org/jira/browse/MAPREDUCE-1885) | Trunk compilation is broken because of FileSystem api change in HADOOP-6826 | Major | . | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1880](https://issues.apache.org/jira/browse/MAPREDUCE-1880) | "java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result." while running "hadoop jar hadoop-0.20.1+169.89-examples.jar pi 4 30" | Minor | examples | Victor Pakhomov | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1876](https://issues.apache.org/jira/browse/MAPREDUCE-1876) | TaskAttemptStartedEvent.java incorrectly logs MAP\_ATTEMPT\_STARTED as event type for reduce tasks | Major | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1870](https://issues.apache.org/jira/browse/MAPREDUCE-1870) | Harmonize MapReduce JAR library versions with Common and HDFS | Blocker | build | Tom White | Tom White | +| [MAPREDUCE-1853](https://issues.apache.org/jira/browse/MAPREDUCE-1853) | MultipleOutputs does not cache TaskAttemptContext | Critical | task | Torsten Curdt | Torsten Curdt | +| [MAPREDUCE-1845](https://issues.apache.org/jira/browse/MAPREDUCE-1845) | FairScheduler.tasksToPeempt() can return negative number | Major | contrib/fair-share | Scott Chen | Scott Chen | +| [MAPREDUCE-1810](https://issues.apache.org/jira/browse/MAPREDUCE-1810) | 0.21 build is broken | Major | build | Sharad Agarwal | Tom White | +| [MAPREDUCE-1765](https://issues.apache.org/jira/browse/MAPREDUCE-1765) | Streaming doc - change StreamXmlRecord to StreamXmlRecordReader | Minor | contrib/streaming, documentation | Corinne Chandel | Corinne Chandel | +| [MAPREDUCE-1747](https://issues.apache.org/jira/browse/MAPREDUCE-1747) | Remove documentation for the 'unstable' job-acls feature | Blocker | documentation | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1728](https://issues.apache.org/jira/browse/MAPREDUCE-1728) | Oracle timezone strings do not match Java | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1727](https://issues.apache.org/jira/browse/MAPREDUCE-1727) | TestJobACLs fails after HADOOP-6686 | Major | test | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [MAPREDUCE-1725](https://issues.apache.org/jira/browse/MAPREDUCE-1725) | Fix MapReduce API incompatibilities between 0.20 and 0.21 | Blocker | client | Tom White | Tom White | +| [MAPREDUCE-1705](https://issues.apache.org/jira/browse/MAPREDUCE-1705) | Archiving and Purging of parity files should handle globbed policies | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1697](https://issues.apache.org/jira/browse/MAPREDUCE-1697) | Document the behavior of -file option in streaming and deprecate it in favour of generic -files option. | Major | contrib/streaming, documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1695](https://issues.apache.org/jira/browse/MAPREDUCE-1695) | capacity scheduler is not included in findbugs/javadoc targets | Major | capacity-sched | Hong Tang | Hong Tang | +| [MAPREDUCE-1694](https://issues.apache.org/jira/browse/MAPREDUCE-1694) | streaming documentation appears to be wrong on overriding settings w/-D | Major | contrib/streaming, documentation | Allen Wittenauer | | +| [MAPREDUCE-1692](https://issues.apache.org/jira/browse/MAPREDUCE-1692) | Remove TestStreamedMerge from the streaming tests | Minor | contrib/streaming | Sreekanth Ramakrishnan | Amareshwari Sriramadasu | +| [MAPREDUCE-1659](https://issues.apache.org/jira/browse/MAPREDUCE-1659) | RaidNode should write temp files on /tmp and add random numbers to their names to avoid conflicts | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1657](https://issues.apache.org/jira/browse/MAPREDUCE-1657) | After task logs directory is deleted, tasklog servlet displays wrong error message about job ACLs | Major | tasktracker | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1635](https://issues.apache.org/jira/browse/MAPREDUCE-1635) | ResourceEstimator does not work after MAPREDUCE-842 | Major | tasktracker | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1629](https://issues.apache.org/jira/browse/MAPREDUCE-1629) | Get rid of fakeBlockLocations() on HarFileSystem, since it's not used | Trivial | . | Rodrigo Schmidt | Mahadev konar | +| [MAPREDUCE-1628](https://issues.apache.org/jira/browse/MAPREDUCE-1628) | HarFileSystem shows incorrect replication numbers and permissions | Major | harchive | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1622](https://issues.apache.org/jira/browse/MAPREDUCE-1622) | Include slf4j dependencies in binary tarball | Minor | build | Chris Douglas | Chris Douglas | +| [MAPREDUCE-1618](https://issues.apache.org/jira/browse/MAPREDUCE-1618) | JobStatus.getJobAcls() and setJobAcls should have javadoc | Trivial | documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1615](https://issues.apache.org/jira/browse/MAPREDUCE-1615) | ant test on trunk does not compile. | Blocker | . | Mahadev konar | Chris Douglas | +| [MAPREDUCE-1612](https://issues.apache.org/jira/browse/MAPREDUCE-1612) | job conf file is not accessible from job history web page | Major | jobtracker | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1611](https://issues.apache.org/jira/browse/MAPREDUCE-1611) | Refresh nodes and refresh queues doesnt work with service authorization enabled | Blocker | security | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1610](https://issues.apache.org/jira/browse/MAPREDUCE-1610) | Forrest documentation should be updated to reflect the changes in MAPREDUCE-856 | Major | documentation | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1609](https://issues.apache.org/jira/browse/MAPREDUCE-1609) | TaskTracker.localizeJob should not set permissions on job log directory recursively | Major | tasktracker | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1606](https://issues.apache.org/jira/browse/MAPREDUCE-1606) | TestJobACLs may timeout as there are no slots for launching JOB\_CLEANUP task | Major | test | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1604](https://issues.apache.org/jira/browse/MAPREDUCE-1604) | Job acls should be documented in forrest. | Major | documentation, security | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1602](https://issues.apache.org/jira/browse/MAPREDUCE-1602) | When the src does not exist, archive shows IndexOutOfBoundsException | Major | harchive | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1596](https://issues.apache.org/jira/browse/MAPREDUCE-1596) | MapReduce trunk snapshot is not being published to maven | Critical | build | Aaron Kimball | Giridharan Kesavan | +| [MAPREDUCE-1585](https://issues.apache.org/jira/browse/MAPREDUCE-1585) | Create Hadoop Archives version 2 with filenames URL-encoded | Major | harchive | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1578](https://issues.apache.org/jira/browse/MAPREDUCE-1578) | HadoopArchives.java should not use HarFileSystem.VERSION | Major | harchive | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1573](https://issues.apache.org/jira/browse/MAPREDUCE-1573) | TestStreamingAsDifferentUser fails if run as tt\_user | Major | task-controller, test | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1547](https://issues.apache.org/jira/browse/MAPREDUCE-1547) | Build Hadoop-Mapreduce-trunk and Mapreduce-trunk-Commit fails | Major | build | Iyappan Srinivasan | Giridharan Kesavan | +| [MAPREDUCE-1538](https://issues.apache.org/jira/browse/MAPREDUCE-1538) | TrackerDistributedCacheManager can fail because the number of subdirectories reaches system limit | Major | tasktracker | Scott Chen | Scott Chen | +| [MAPREDUCE-1537](https://issues.apache.org/jira/browse/MAPREDUCE-1537) | TestDelegationTokenRenewal fails | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-1536](https://issues.apache.org/jira/browse/MAPREDUCE-1536) | DataDrivenDBInputFormat does not split date columns correctly. | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1523](https://issues.apache.org/jira/browse/MAPREDUCE-1523) | Sometimes rumen trace generator fails to extract the job finish time. | Major | tools/rumen | Hong Tang | Dick King | +| [MAPREDUCE-1520](https://issues.apache.org/jira/browse/MAPREDUCE-1520) | TestMiniMRLocalFS fails on trunk | Major | . | Devaraj Das | Amareshwari Sriramadasu | +| [MAPREDUCE-1519](https://issues.apache.org/jira/browse/MAPREDUCE-1519) | RaidNode fails to create new parity file if an older version already exists | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1515](https://issues.apache.org/jira/browse/MAPREDUCE-1515) | need to pass down java5 and forrest home variables | Major | build | Owen O'Malley | Al Thompson | +| [MAPREDUCE-1508](https://issues.apache.org/jira/browse/MAPREDUCE-1508) | NPE in TestMultipleLevelCaching on error cleanup path | Major | test | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1497](https://issues.apache.org/jira/browse/MAPREDUCE-1497) | Suppress warning on inconsistent TaskTracker.indexCache synchronization | Major | tasktracker | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1494](https://issues.apache.org/jira/browse/MAPREDUCE-1494) | TestJobDirCleanup verifies wrong jobcache directory | Minor | tasktracker, test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1490](https://issues.apache.org/jira/browse/MAPREDUCE-1490) | Raid client throws NullPointerException during initialization | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1482](https://issues.apache.org/jira/browse/MAPREDUCE-1482) | Better handling of task diagnostic information stored in the TaskInProgress | Major | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1476](https://issues.apache.org/jira/browse/MAPREDUCE-1476) | committer.needsTaskCommit should not be called for a task cleanup attempt | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1474](https://issues.apache.org/jira/browse/MAPREDUCE-1474) | forrest docs for archives is out of date. | Major | documentation | Mahadev konar | Mahadev konar | +| [MAPREDUCE-1448](https://issues.apache.org/jira/browse/MAPREDUCE-1448) | [Mumak] mumak.sh does not honor --config option. | Major | . | Hong Tang | Hong Tang | +| [MAPREDUCE-1435](https://issues.apache.org/jira/browse/MAPREDUCE-1435) | symlinks in cwd of the task are not handled properly after MAPREDUCE-896 | Major | tasktracker | Amareshwari Sriramadasu | Ravi Gummadi | +| [MAPREDUCE-1422](https://issues.apache.org/jira/browse/MAPREDUCE-1422) | Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in all mapred-local-directories succeeds always | Major | task-controller, tasktracker | Ravi Gummadi | Amar Kamat | +| [MAPREDUCE-1421](https://issues.apache.org/jira/browse/MAPREDUCE-1421) | LinuxTaskController tests failing on trunk after the commit of MAPREDUCE-1385 | Major | task-controller, tasktracker, test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1420](https://issues.apache.org/jira/browse/MAPREDUCE-1420) | TestTTResourceReporting failing in trunk | Major | test | Iyappan Srinivasan | Scott Chen | +| [MAPREDUCE-1417](https://issues.apache.org/jira/browse/MAPREDUCE-1417) | Forrest documentation should be updated to reflect the changes in MAPREDUCE-744 | Major | documentation | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [MAPREDUCE-1412](https://issues.apache.org/jira/browse/MAPREDUCE-1412) | TestTaskTrackerBlacklisting fails sometimes | Minor | test | Chris Douglas | Chris Douglas | +| [MAPREDUCE-1409](https://issues.apache.org/jira/browse/MAPREDUCE-1409) | FileOutputCommitter.abortTask should not catch IOException | Major | tasktracker | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1408](https://issues.apache.org/jira/browse/MAPREDUCE-1408) | Allow customization of job submission policies | Major | contrib/gridmix | rahul k singh | rahul k singh | +| [MAPREDUCE-1406](https://issues.apache.org/jira/browse/MAPREDUCE-1406) | JobContext.MAP\_COMBINE\_MIN\_SPILLS is misspelled | Trivial | . | Chris Douglas | Chris Douglas | +| [MAPREDUCE-1400](https://issues.apache.org/jira/browse/MAPREDUCE-1400) | sed in build.xml fails | Minor | . | Allen Wittenauer | Allen Wittenauer | +| [MAPREDUCE-1399](https://issues.apache.org/jira/browse/MAPREDUCE-1399) | The archive command shows a null error message | Major | harchive | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1398](https://issues.apache.org/jira/browse/MAPREDUCE-1398) | TaskLauncher remains stuck on tasks waiting for free nodes even if task is killed. | Major | tasktracker | Hemanth Yamijala | Amareshwari Sriramadasu | +| [MAPREDUCE-1397](https://issues.apache.org/jira/browse/MAPREDUCE-1397) | NullPointerException observed during task failures | Minor | tasktracker | Ramya Sunil | Amareshwari Sriramadasu | +| [MAPREDUCE-1378](https://issues.apache.org/jira/browse/MAPREDUCE-1378) | Args in job details links on jobhistory.jsp are not URL encoded | Trivial | jobtracker | E. Sammer | E. Sammer | +| [MAPREDUCE-1372](https://issues.apache.org/jira/browse/MAPREDUCE-1372) | ConcurrentModificationException in JobInProgress | Blocker | jobtracker | Amareshwari Sriramadasu | Dick King | +| [MAPREDUCE-1369](https://issues.apache.org/jira/browse/MAPREDUCE-1369) | JUnit tests should never depend on anything in conf | Blocker | test | Anatoli Fomenko | Anatoli Fomenko | +| [MAPREDUCE-1365](https://issues.apache.org/jira/browse/MAPREDUCE-1365) | TestTaskTrackerBlacklisting.AtestTrackerBlacklistingForJobFailures is mistyped. | Trivial | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1358](https://issues.apache.org/jira/browse/MAPREDUCE-1358) | Utils.OutputLogFilter incorrectly filters for \_logs | Major | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1348](https://issues.apache.org/jira/browse/MAPREDUCE-1348) | Package org.apache.hadoop.blockforensics does not match directory name | Major | build | Tom White | Tom White | +| [MAPREDUCE-1342](https://issues.apache.org/jira/browse/MAPREDUCE-1342) | Potential JT deadlock in faulty TT tracking | Major | jobtracker | Todd Lipcon | Amareshwari Sriramadasu | +| [MAPREDUCE-1322](https://issues.apache.org/jira/browse/MAPREDUCE-1322) | TestStreamingAsDifferentUser fails on trunk | Major | contrib/streaming, test | Amareshwari Sriramadasu | Devaraj Das | +| [MAPREDUCE-1316](https://issues.apache.org/jira/browse/MAPREDUCE-1316) | JobTracker holds stale references to retired jobs via unreported tasks | Blocker | jobtracker | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1314](https://issues.apache.org/jira/browse/MAPREDUCE-1314) | Some logs have wrong configuration names. | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1301](https://issues.apache.org/jira/browse/MAPREDUCE-1301) | TestDebugScriptWithLinuxTaskController fails | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1294](https://issues.apache.org/jira/browse/MAPREDUCE-1294) | Build fails to pull latest hadoop-core-\* artifacts | Critical | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1293](https://issues.apache.org/jira/browse/MAPREDUCE-1293) | AutoInputFormat doesn't work with non-default FileSystems | Major | contrib/streaming | Andrew Hitchcock | Andrew Hitchcock | +| [MAPREDUCE-1285](https://issues.apache.org/jira/browse/MAPREDUCE-1285) | DistCp cannot handle -delete if destination is local filesystem | Major | distcp | Peter Romianowski | Peter Romianowski | +| [MAPREDUCE-1284](https://issues.apache.org/jira/browse/MAPREDUCE-1284) | TestLocalizationWithLinuxTaskController fails | Major | tasktracker, test | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1276](https://issues.apache.org/jira/browse/MAPREDUCE-1276) | Shuffle connection logic needs correction | Blocker | task | Jothi Padmanabhan | Amareshwari Sriramadasu | +| [MAPREDUCE-1267](https://issues.apache.org/jira/browse/MAPREDUCE-1267) | Fix typo in mapred-default.xml | Minor | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1260](https://issues.apache.org/jira/browse/MAPREDUCE-1260) | Update Eclipse configuration to match changes to Ivy configuration | Major | build | Edwin Chan | | +| [MAPREDUCE-1258](https://issues.apache.org/jira/browse/MAPREDUCE-1258) | Fair scheduler event log not logging job info | Minor | contrib/fair-share | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-1249](https://issues.apache.org/jira/browse/MAPREDUCE-1249) | mapreduce.reduce.shuffle.read.timeout's default value should be 3 minutes, in mapred-default.xml | Blocker | task | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1245](https://issues.apache.org/jira/browse/MAPREDUCE-1245) | TestFairScheduler fails with "too many open files" error | Major | test | Vinod Kumar Vavilapalli | Sharad Agarwal | +| [MAPREDUCE-1244](https://issues.apache.org/jira/browse/MAPREDUCE-1244) | eclipse-plugin fails with missing dependencies | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-1241](https://issues.apache.org/jira/browse/MAPREDUCE-1241) | JobTracker should not crash when mapred-queues.xml does not exist | Blocker | . | Owen O'Malley | Todd Lipcon | +| [MAPREDUCE-1239](https://issues.apache.org/jira/browse/MAPREDUCE-1239) | Mapreduce test build is broken after HADOOP-5107 | Blocker | build | Vinod Kumar Vavilapalli | Giridharan Kesavan | +| [MAPREDUCE-1230](https://issues.apache.org/jira/browse/MAPREDUCE-1230) | Vertica streaming adapter doesn't handle nulls in all cases | Major | contrib/vertica | Omer Trajman | Omer Trajman | +| [MAPREDUCE-1222](https://issues.apache.org/jira/browse/MAPREDUCE-1222) | [Mumak] We should not include nodes with numeric ips in cluster topology. | Major | contrib/mumak | Hong Tang | Hong Tang | +| [MAPREDUCE-1219](https://issues.apache.org/jira/browse/MAPREDUCE-1219) | JobTracker Metrics causes undue load on JobTracker | Major | . | Jothi Padmanabhan | Sreekanth Ramakrishnan | +| [MAPREDUCE-1212](https://issues.apache.org/jira/browse/MAPREDUCE-1212) | Mapreduce contrib project ivy dependencies are not included in binary target | Critical | build | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1196](https://issues.apache.org/jira/browse/MAPREDUCE-1196) | MAPREDUCE-947 incompatibly changed FileOutputCommitter | Blocker | client | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-1186](https://issues.apache.org/jira/browse/MAPREDUCE-1186) | While localizing a DistributedCache file, TT sets permissions recursively on the whole base-dir | Major | tasktracker | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [MAPREDUCE-1178](https://issues.apache.org/jira/browse/MAPREDUCE-1178) | MultipleInputs fails with ClassCastException | Blocker | . | Jay Booth | Amareshwari Sriramadasu | +| [MAPREDUCE-1177](https://issues.apache.org/jira/browse/MAPREDUCE-1177) | TestTaskTrackerMemoryManager retries a task for more than 100 times. | Blocker | tasktracker, test | Amareshwari Sriramadasu | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1171](https://issues.apache.org/jira/browse/MAPREDUCE-1171) | Lots of fetch failures | Blocker | task | Christian Kunz | Amareshwari Sriramadasu | +| [MAPREDUCE-1165](https://issues.apache.org/jira/browse/MAPREDUCE-1165) | SerialUtils.hh: \_\_PRETTY\_FUNCTION\_\_ is a GNU extension and not portable | Major | . | Allen Wittenauer | Allen Wittenauer | +| [MAPREDUCE-1161](https://issues.apache.org/jira/browse/MAPREDUCE-1161) | NotificationTestCase should not lock current thread | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1160](https://issues.apache.org/jira/browse/MAPREDUCE-1160) | Two log statements at INFO level fill up jobtracker logs | Major | jobtracker | Hemanth Yamijala | Ravi Gummadi | +| [MAPREDUCE-1158](https://issues.apache.org/jira/browse/MAPREDUCE-1158) | running\_maps is not decremented when the tasks of a job is killed/failed | Major | jobtracker | Ramya Sunil | Sharad Agarwal | +| [MAPREDUCE-1155](https://issues.apache.org/jira/browse/MAPREDUCE-1155) | Streaming tests swallow exceptions | Minor | contrib/streaming | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1153](https://issues.apache.org/jira/browse/MAPREDUCE-1153) | Metrics counting tasktrackers and blacklisted tasktrackers are not updated when trackers are decommissioned. | Major | jobtracker | Hemanth Yamijala | Sharad Agarwal | +| [MAPREDUCE-1152](https://issues.apache.org/jira/browse/MAPREDUCE-1152) | JobTrackerInstrumentation.killed{Map/Reduce} is never called | Major | . | Sharad Agarwal | | +| [MAPREDUCE-1143](https://issues.apache.org/jira/browse/MAPREDUCE-1143) | runningMapTasks counter is not properly decremented in case of failed Tasks. | Blocker | . | rahul k singh | rahul k singh | +| [MAPREDUCE-1140](https://issues.apache.org/jira/browse/MAPREDUCE-1140) | Per cache-file refcount can become negative when tasks release distributed-cache files | Major | tasktracker | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [MAPREDUCE-1133](https://issues.apache.org/jira/browse/MAPREDUCE-1133) | Eclipse .classpath template has outdated jar files and is missing some new ones. | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1131](https://issues.apache.org/jira/browse/MAPREDUCE-1131) | Using profilers other than hprof can cause JobClient to report job failure | Major | client | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1128](https://issues.apache.org/jira/browse/MAPREDUCE-1128) | MRUnit Allows Iteration Twice | Minor | contrib/mrunit | Ed Kohlwey | Aaron Kimball | +| [MAPREDUCE-1124](https://issues.apache.org/jira/browse/MAPREDUCE-1124) | TestGridmixSubmission fails sometimes | Major | contrib/gridmix | Amareshwari Sriramadasu | Chris Douglas | +| [MAPREDUCE-1119](https://issues.apache.org/jira/browse/MAPREDUCE-1119) | When tasks fail to report status, show tasks's stack dump before killing | Major | tasktracker | Todd Lipcon | Aaron Kimball | +| [MAPREDUCE-1117](https://issues.apache.org/jira/browse/MAPREDUCE-1117) | ClusterMetrics return metrics for tasks instead of slots' | Major | . | Sharad Agarwal | Amareshwari Sriramadasu | +| [MAPREDUCE-1111](https://issues.apache.org/jira/browse/MAPREDUCE-1111) | JT Jetty UI not working if we run mumak.sh off packaged distribution directory. | Major | contrib/mumak | Hong Tang | Hong Tang | +| [MAPREDUCE-1105](https://issues.apache.org/jira/browse/MAPREDUCE-1105) | CapacityScheduler: It should be possible to set queue hard-limit beyond it's actual capacity | Blocker | capacity-sched | Arun C Murthy | rahul k singh | +| [MAPREDUCE-1104](https://issues.apache.org/jira/browse/MAPREDUCE-1104) | RecoveryManager not initialized in SimulatorJobTracker led to NPE in JT Jetty server | Major | contrib/mumak | Hong Tang | Hong Tang | +| [MAPREDUCE-1098](https://issues.apache.org/jira/browse/MAPREDUCE-1098) | Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks. | Major | tasktracker | Sreekanth Ramakrishnan | Amareshwari Sriramadasu | +| [MAPREDUCE-1090](https://issues.apache.org/jira/browse/MAPREDUCE-1090) | Modify log statement in Tasktracker log related to memory monitoring to include attempt id. | Major | tasktracker | Hemanth Yamijala | Hemanth Yamijala | +| [MAPREDUCE-1089](https://issues.apache.org/jira/browse/MAPREDUCE-1089) | Fair Scheduler preemption triggers NPE when tasks are scheduled but not running | Major | contrib/fair-share | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1086](https://issues.apache.org/jira/browse/MAPREDUCE-1086) | hadoop commands in streaming tasks are trying to write to tasktracker's log | Major | tasktracker | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1077](https://issues.apache.org/jira/browse/MAPREDUCE-1077) | When rumen reads a truncated job tracker log, it produces a job whose outcome is SUCCESS. Should be null. | Major | tools/rumen | Dick King | Dick King | +| [MAPREDUCE-1076](https://issues.apache.org/jira/browse/MAPREDUCE-1076) | ClusterStatus class should be deprecated | Blocker | client | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1075](https://issues.apache.org/jira/browse/MAPREDUCE-1075) | getQueue(String queue) in JobTracker would return NPE for invalid queue name | Major | . | V.V.Chaitanya Krishna | V.V.Chaitanya Krishna | +| [MAPREDUCE-1071](https://issues.apache.org/jira/browse/MAPREDUCE-1071) | o.a.h.mapreduce.jobhistory.EventReader constructor should expect DataInputStream | Major | . | Hong Tang | Hong Tang | +| [MAPREDUCE-1065](https://issues.apache.org/jira/browse/MAPREDUCE-1065) | Modify the mapred tutorial documentation to use new mapreduce api. | Blocker | documentation | Amareshwari Sriramadasu | Aaron Kimball | +| [MAPREDUCE-1062](https://issues.apache.org/jira/browse/MAPREDUCE-1062) | MRReliability test does not work with retired jobs | Major | test | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan | +| [MAPREDUCE-1041](https://issues.apache.org/jira/browse/MAPREDUCE-1041) | TaskStatuses map in TaskInProgress should be made package private instead of protected | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-1038](https://issues.apache.org/jira/browse/MAPREDUCE-1038) | Mumak's compile-aspects target weaves aspects even though there are no changes to the Mumak's sources | Major | build | Vinod Kumar Vavilapalli | Aaron Kimball | +| [MAPREDUCE-1031](https://issues.apache.org/jira/browse/MAPREDUCE-1031) | ant tar target doens't seem to compile tests in contrib projects | Blocker | build | Arun C Murthy | Aaron Kimball | +| [MAPREDUCE-1030](https://issues.apache.org/jira/browse/MAPREDUCE-1030) | Reduce tasks are getting starved in capacity scheduler | Blocker | capacity-sched | rahul k singh | rahul k singh | +| [MAPREDUCE-1029](https://issues.apache.org/jira/browse/MAPREDUCE-1029) | TestCopyFiles fails on testHftpAccessControl() | Blocker | build | Amar Kamat | Jothi Padmanabhan | +| [MAPREDUCE-1028](https://issues.apache.org/jira/browse/MAPREDUCE-1028) | Cleanup tasks are scheduled using high memory configuration, leaving tasks in unassigned state. | Blocker | jobtracker | Hemanth Yamijala | Ravi Gummadi | +| [MAPREDUCE-1022](https://issues.apache.org/jira/browse/MAPREDUCE-1022) | Trunk tests fail because of test-failure in Vertica | Blocker | test | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1018](https://issues.apache.org/jira/browse/MAPREDUCE-1018) | Document changes to the memory management and scheduling model | Blocker | documentation | Hemanth Yamijala | Hemanth Yamijala | +| [MAPREDUCE-1016](https://issues.apache.org/jira/browse/MAPREDUCE-1016) | Make the format of the Job History be JSON instead of Avro binary | Major | . | Owen O'Malley | Doug Cutting | +| [MAPREDUCE-1014](https://issues.apache.org/jira/browse/MAPREDUCE-1014) | After the 0.21 branch, MapReduce trunk doesn't compile | Blocker | . | Devaraj Das | Ravi Gummadi | +| [MAPREDUCE-1009](https://issues.apache.org/jira/browse/MAPREDUCE-1009) | Forrest documentation needs to be updated to describes features provided for supporting hierarchical queues | Blocker | documentation | Hemanth Yamijala | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1007](https://issues.apache.org/jira/browse/MAPREDUCE-1007) | MAPREDUCE-777 breaks the UI for hierarchial Queues. | Blocker | jobtracker | rahul k singh | V.V.Chaitanya Krishna | +| [MAPREDUCE-1003](https://issues.apache.org/jira/browse/MAPREDUCE-1003) | trunk build fails when -Declipse.home is set | Major | . | Giridharan Kesavan | Ravi Gummadi | +| [MAPREDUCE-1002](https://issues.apache.org/jira/browse/MAPREDUCE-1002) | After MAPREDUCE-862, command line queue-list doesn't print any queues | Major | client | Vinod Kumar Vavilapalli | V.V.Chaitanya Krishna | +| [MAPREDUCE-1000](https://issues.apache.org/jira/browse/MAPREDUCE-1000) | JobHistory.initDone() should retain the try ... catch in the body | Major | jobtracker | Hong Tang | Jothi Padmanabhan | +| [MAPREDUCE-988](https://issues.apache.org/jira/browse/MAPREDUCE-988) | ant package does not copy the capacity-scheduler.jar under HADOOP\_HOME/build/hadoop-mapred-0.21.0-dev/contrib/capacity-scheduler | Major | build | Iyappan Srinivasan | Hong Tang | +| [MAPREDUCE-986](https://issues.apache.org/jira/browse/MAPREDUCE-986) | rumen makes a task with a null type when one of the task lines is truncated | Major | tools/rumen | Dick King | Dick King | +| [MAPREDUCE-977](https://issues.apache.org/jira/browse/MAPREDUCE-977) | Missing jackson jars from Eclipse template | Major | build | Tom White | Tom White | +| [MAPREDUCE-973](https://issues.apache.org/jira/browse/MAPREDUCE-973) | Move test utilities from examples to test | Trivial | examples, test | Chris Douglas | Chris Douglas | +| [MAPREDUCE-971](https://issues.apache.org/jira/browse/MAPREDUCE-971) | distcp does not always remove distcp.tmp.dir | Major | distcp | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-968](https://issues.apache.org/jira/browse/MAPREDUCE-968) | NPE in distcp encountered when placing \_logs directory on S3FileSystem | Major | distcp | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-964](https://issues.apache.org/jira/browse/MAPREDUCE-964) | Inaccurate values in jobSummary logs | Critical | . | Rajiv Chittajallu | Sreekanth Ramakrishnan | +| [MAPREDUCE-962](https://issues.apache.org/jira/browse/MAPREDUCE-962) | NPE in ProcfsBasedProcessTree.destroy() | Major | tasktracker | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [MAPREDUCE-952](https://issues.apache.org/jira/browse/MAPREDUCE-952) | Previously removed Task.Counter reintroduced by MAPREDUCE-318 | Blocker | task | Arun C Murthy | Jothi Padmanabhan | +| [MAPREDUCE-946](https://issues.apache.org/jira/browse/MAPREDUCE-946) | Fix regression in LineRecordReader to comply with line length parameters | Blocker | . | Chris Douglas | Chris Douglas | +| [MAPREDUCE-945](https://issues.apache.org/jira/browse/MAPREDUCE-945) | Test programs support only default queue. | Major | test | Suman Sehgal | Sreekanth Ramakrishnan | +| [MAPREDUCE-941](https://issues.apache.org/jira/browse/MAPREDUCE-941) | vaidya script calls awk instead of nawk | Trivial | . | Allen Wittenauer | Chad Metcalf | +| [MAPREDUCE-915](https://issues.apache.org/jira/browse/MAPREDUCE-915) | For secure environments, the Map/Reduce debug script must be run as the user. | Blocker | security, tasktracker | Hemanth Yamijala | Devaraj Das | +| [MAPREDUCE-913](https://issues.apache.org/jira/browse/MAPREDUCE-913) | TaskRunner crashes with NPE resulting in held up slots, UNINITIALIZED tasks and hung TaskTracker | Blocker | tasktracker | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | +| [MAPREDUCE-912](https://issues.apache.org/jira/browse/MAPREDUCE-912) | apache license header missing for some java files | Major | . | Amareshwari Sriramadasu | Chad Metcalf | +| [MAPREDUCE-896](https://issues.apache.org/jira/browse/MAPREDUCE-896) | Users can set non-writable permissions on temporary files for TT and can abuse disk usage. | Major | tasktracker | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [MAPREDUCE-890](https://issues.apache.org/jira/browse/MAPREDUCE-890) | After HADOOP-4491, the user who started mapred system is not able to run job. | Blocker | tasktracker | Karam Singh | Ravi Gummadi | +| [MAPREDUCE-889](https://issues.apache.org/jira/browse/MAPREDUCE-889) | binary communication formats added to Streaming by HADOOP-1722 should be documented | Blocker | documentation | Amareshwari Sriramadasu | Klaas Bosteels | +| [MAPREDUCE-884](https://issues.apache.org/jira/browse/MAPREDUCE-884) | TestReduceFetchFromPartialMem fails sometimes | Major | test | Amar Kamat | Jothi Padmanabhan | +| [MAPREDUCE-879](https://issues.apache.org/jira/browse/MAPREDUCE-879) | TestTaskTrackerLocalization fails on MAC OS | Blocker | test | Devaraj Das | Sreekanth Ramakrishnan | +| [MAPREDUCE-877](https://issues.apache.org/jira/browse/MAPREDUCE-877) | Required avro class are missing in contrib projects | Blocker | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-868](https://issues.apache.org/jira/browse/MAPREDUCE-868) | Trunk can't be compiled since Avro dependencies cannot be resolved | Blocker | build | Arun C Murthy | | +| [MAPREDUCE-867](https://issues.apache.org/jira/browse/MAPREDUCE-867) | trunk builds fails as ivy is lookin for avro jar from the local resolver | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-859](https://issues.apache.org/jira/browse/MAPREDUCE-859) | Unable to run examples with current trunk | Major | build | Jothi Padmanabhan | Ravi Gummadi | +| [MAPREDUCE-852](https://issues.apache.org/jira/browse/MAPREDUCE-852) | ExampleDriver is incorrectly set as a Main-Class in tools in build.xml | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-848](https://issues.apache.org/jira/browse/MAPREDUCE-848) | TestCapacityScheduler is failing | Major | capacity-sched | Devaraj Das | Amar Kamat | +| [MAPREDUCE-845](https://issues.apache.org/jira/browse/MAPREDUCE-845) | build.xml hard codes findbugs heap size, in some configurations 512M is insufficient to successfully build | Minor | build | Lee Tucker | Lee Tucker | +| [MAPREDUCE-840](https://issues.apache.org/jira/browse/MAPREDUCE-840) | DBInputFormat leaves open transaction | Minor | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-839](https://issues.apache.org/jira/browse/MAPREDUCE-839) | unit test TestMiniMRChildTask fails on mac os-x | Minor | . | Hong Tang | Hong Tang | +| [MAPREDUCE-825](https://issues.apache.org/jira/browse/MAPREDUCE-825) | JobClient completion poll interval of 5s causes slow tests in local mode | Minor | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-809](https://issues.apache.org/jira/browse/MAPREDUCE-809) | Job summary logs show status of completed jobs as RUNNING | Major | jobtracker | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-808](https://issues.apache.org/jira/browse/MAPREDUCE-808) | Buffer objects incorrectly serialized to typed bytes | Major | contrib/streaming | Klaas Bosteels | Klaas Bosteels | +| [MAPREDUCE-799](https://issues.apache.org/jira/browse/MAPREDUCE-799) | Some of MRUnit's self-tests were not being run | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-792](https://issues.apache.org/jira/browse/MAPREDUCE-792) | javac warnings in DBInputFormat | Minor | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-787](https://issues.apache.org/jira/browse/MAPREDUCE-787) | -files, -archives should honor user given symlink path | Major | client | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-773](https://issues.apache.org/jira/browse/MAPREDUCE-773) | LineRecordReader can report non-zero progress while it is processing a compressed stream | Major | task | Devaraj Das | Devaraj Das | +| [MAPREDUCE-771](https://issues.apache.org/jira/browse/MAPREDUCE-771) | Setup and cleanup tasks remain in UNASSIGNED state for a long time on tasktrackers with long running high RAM tasks | Blocker | jobtracker | Hemanth Yamijala | Hemanth Yamijala | +| [MAPREDUCE-769](https://issues.apache.org/jira/browse/MAPREDUCE-769) | findbugs and javac warnings on trunk is non-zero | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-764](https://issues.apache.org/jira/browse/MAPREDUCE-764) | TypedBytesInput's readRaw() does not preserve custom type codes | Blocker | contrib/streaming | Klaas Bosteels | Klaas Bosteels | +| [MAPREDUCE-760](https://issues.apache.org/jira/browse/MAPREDUCE-760) | TestNodeRefresh might not work as expected | Major | test | Amar Kamat | Amar Kamat | +| [MAPREDUCE-754](https://issues.apache.org/jira/browse/MAPREDUCE-754) | NPE in expiry thread when a TT is lost | Minor | jobtracker | Ramya Sunil | Amar Kamat | +| [MAPREDUCE-743](https://issues.apache.org/jira/browse/MAPREDUCE-743) | Progress of map phase in map task is not updated properly | Major | task | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-734](https://issues.apache.org/jira/browse/MAPREDUCE-734) | java.util.ConcurrentModificationException observed in unreserving slots for HiRam Jobs | Major | capacity-sched | Karam Singh | Arun C Murthy | +| [MAPREDUCE-733](https://issues.apache.org/jira/browse/MAPREDUCE-733) | When running ant test TestTrackerBlacklistAcrossJobs, losing task tracker heartbeat exception occurs. | Major | tasktracker | Iyappan Srinivasan | Arun C Murthy | +| [MAPREDUCE-732](https://issues.apache.org/jira/browse/MAPREDUCE-732) | node health check script should not log "UNHEALTHY" status for every heartbeat in INFO mode | Minor | . | Ramya Sunil | Sreekanth Ramakrishnan | +| [MAPREDUCE-722](https://issues.apache.org/jira/browse/MAPREDUCE-722) | More slots are getting reserved for HiRAM job tasks then required | Major | capacity-sched | Karam Singh | Vinod Kumar Vavilapalli | +| [MAPREDUCE-717](https://issues.apache.org/jira/browse/MAPREDUCE-717) | Fix some corner case issues in speculative execution (post hadoop-2141) | Major | jobtracker | Devaraj Das | Devaraj Das | +| [MAPREDUCE-716](https://issues.apache.org/jira/browse/MAPREDUCE-716) | org.apache.hadoop.mapred.lib.db.DBInputformat not working with oracle | Major | . | evanand | Aaron Kimball | +| [MAPREDUCE-709](https://issues.apache.org/jira/browse/MAPREDUCE-709) | node health check script does not display the correct message on timeout | Minor | . | Ramya Sunil | Sreekanth Ramakrishnan | +| [MAPREDUCE-708](https://issues.apache.org/jira/browse/MAPREDUCE-708) | node health check script does not refresh the "reason for blacklisting" | Minor | tasktracker | Ramya Sunil | Sreekanth Ramakrishnan | +| [MAPREDUCE-702](https://issues.apache.org/jira/browse/MAPREDUCE-702) | eclipse-plugin jar target fails during packaging | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-694](https://issues.apache.org/jira/browse/MAPREDUCE-694) | JSP jars should be added to dependcy list for Capacity scheduler | Major | build, capacity-sched | Sreekanth Ramakrishnan | Giridharan Kesavan | +| [MAPREDUCE-683](https://issues.apache.org/jira/browse/MAPREDUCE-683) | TestJobTrackerRestart fails with Map task completion events ordering mismatch | Major | jobtracker | Sreekanth Ramakrishnan | Amar Kamat | +| [MAPREDUCE-682](https://issues.apache.org/jira/browse/MAPREDUCE-682) | Reserved tasktrackers should be removed when a node is globally blacklisted | Major | jobtracker | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [MAPREDUCE-680](https://issues.apache.org/jira/browse/MAPREDUCE-680) | Reuse of Writable objects is improperly handled by MRUnit | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-677](https://issues.apache.org/jira/browse/MAPREDUCE-677) | TestNodeRefresh timesout | Major | test | Amar Kamat | Amar Kamat | +| [MAPREDUCE-676](https://issues.apache.org/jira/browse/MAPREDUCE-676) | Existing diagnostic rules fail for MAP ONLY jobs | Major | . | Suhas Gogate | Suhas Gogate | +| [MAPREDUCE-671](https://issues.apache.org/jira/browse/MAPREDUCE-671) | Update ignore list | Trivial | build | Chris Douglas | Chris Douglas | +| [MAPREDUCE-659](https://issues.apache.org/jira/browse/MAPREDUCE-659) | gridmix2 not compiling under mapred module trunk/src/benchmarks/gridmix2 | Critical | build | Iyappan Srinivasan | Giridharan Kesavan | +| [MAPREDUCE-658](https://issues.apache.org/jira/browse/MAPREDUCE-658) | NPE in distcp if source path does not exist | Major | distcp | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-648](https://issues.apache.org/jira/browse/MAPREDUCE-648) | Two distcp bugs | Minor | distcp | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-646](https://issues.apache.org/jira/browse/MAPREDUCE-646) | distcp should place the file distcp\_src\_files in distributed cache | Major | distcp | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-645](https://issues.apache.org/jira/browse/MAPREDUCE-645) | When disctp is used to overwrite a file, it should return immediately with an error message | Minor | distcp | Ramya Sunil | Ravi Gummadi | +| [MAPREDUCE-642](https://issues.apache.org/jira/browse/MAPREDUCE-642) | distcp could have an option to preserve the full source path | Major | distcp | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-639](https://issues.apache.org/jira/browse/MAPREDUCE-639) | Update the TeraSort to reflect the new benchmark rules for '09 | Major | examples | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-637](https://issues.apache.org/jira/browse/MAPREDUCE-637) | Check in the codes that compute the 10^15+1st bit of π | Major | examples | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-630](https://issues.apache.org/jira/browse/MAPREDUCE-630) | TestKillCompletedJob can be modified to improve execution times | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-628](https://issues.apache.org/jira/browse/MAPREDUCE-628) | TestJobInProgress brings up MinMR/DFS clusters for every test | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-627](https://issues.apache.org/jira/browse/MAPREDUCE-627) | Modify TestTrackerBlacklistAcrossJobs to improve execution time | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-626](https://issues.apache.org/jira/browse/MAPREDUCE-626) | Modify TestLostTracker to improve execution time | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-522](https://issues.apache.org/jira/browse/MAPREDUCE-522) | Rewrite TestQueueCapacities to make it simpler and avoid timeout errors | Major | capacity-sched | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [MAPREDUCE-419](https://issues.apache.org/jira/browse/MAPREDUCE-419) | mapred.userlog.limit.kb has inconsistent defaults | Minor | . | Philip Zeyliger | Philip Zeyliger | +| [MAPREDUCE-416](https://issues.apache.org/jira/browse/MAPREDUCE-416) | Move the completed jobs' history files to a DONE subdirectory inside the configured history directory | Major | . | Devaraj Das | Amar Kamat | +| [MAPREDUCE-408](https://issues.apache.org/jira/browse/MAPREDUCE-408) | TestKillSubProcesses fails with assertion failure sometimes | Major | test | Amareshwari Sriramadasu | Ravi Gummadi | +| [MAPREDUCE-153](https://issues.apache.org/jira/browse/MAPREDUCE-153) | TestJobInProgressListener sometimes timesout | Major | . | Amar Kamat | Amar Kamat | +| [MAPREDUCE-144](https://issues.apache.org/jira/browse/MAPREDUCE-144) | TaskMemoryManager should log process-tree's status while killing tasks. | Major | tasktracker | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-64](https://issues.apache.org/jira/browse/MAPREDUCE-64) | Map-side sort is hampered by io.sort.record.percent | Major | performance, task | Arun C Murthy | Chris Douglas | +| [MAPREDUCE-28](https://issues.apache.org/jira/browse/MAPREDUCE-28) | TestQueueManager takes too long and times out some times | Major | jobtracker, test | Amareshwari Sriramadasu | V.V.Chaitanya Krishna | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6836](https://issues.apache.org/jira/browse/HADOOP-6836) | [Herriot]: Generic method for adding/modifying the attributes for new configuration. | Major | test | Iyappan Srinivasan | Vinay Kumar Thota | +| [HADOOP-6738](https://issues.apache.org/jira/browse/HADOOP-6738) | Move cluster\_setup.xml from MapReduce to Common | Blocker | . | Tom White | Tom White | +| [HADOOP-6705](https://issues.apache.org/jira/browse/HADOOP-6705) | jiracli fails to upload test-patch comments to jira | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-6689](https://issues.apache.org/jira/browse/HADOOP-6689) | Add directory renaming test to FileContextMainOperationsBaseTest | Minor | fs, test | Eli Collins | Eli Collins | +| [HADOOP-6563](https://issues.apache.org/jira/browse/HADOOP-6563) | Add more tests to FileContextSymlinkBaseTest that cover intermediate symlinks in paths | Major | fs, test | Eli Collins | Eli Collins | +| [HADOOP-6309](https://issues.apache.org/jira/browse/HADOOP-6309) | Enable asserts for tests by default | Major | build | Eli Collins | Eli Collins | +| [HADOOP-6261](https://issues.apache.org/jira/browse/HADOOP-6261) | Junit tests for FileContextURI | Blocker | test | Ravi Phulari | Ravi Phulari | +| [HADOOP-6260](https://issues.apache.org/jira/browse/HADOOP-6260) | Unit tests for FileSystemContextUtil. | Major | fs | gary murry | gary murry | +| [HADOOP-6222](https://issues.apache.org/jira/browse/HADOOP-6222) | Core doesn't have TestCommonCLI facility | Major | test | Boris Shkolnik | Konstantin Boudnik | +| [HADOOP-6176](https://issues.apache.org/jira/browse/HADOOP-6176) | Adding a couple private methods to AccessTokenHandler for testing purposes | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-5955](https://issues.apache.org/jira/browse/HADOOP-5955) | TestFileOuputFormat can use LOCAL\_MR instead of CLUSTER\_MR | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5952](https://issues.apache.org/jira/browse/HADOOP-5952) | Hudson -1 wording change | Minor | build | gary murry | gary murry | +| [HADOOP-5948](https://issues.apache.org/jira/browse/HADOOP-5948) | Modify TestJavaSerialization to use LocalJobRunner instead of MiniMR/DFS cluster | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [HADOOP-5457](https://issues.apache.org/jira/browse/HADOOP-5457) | Failing contrib tests should not stop the build | Major | test | Chris Douglas | Giridharan Kesavan | +| [HADOOP-5081](https://issues.apache.org/jira/browse/HADOOP-5081) | Split TestCLI into HDFS, Mapred and Core tests | Minor | test | Ramya Sunil | Sharad Agarwal | +| [HADOOP-5080](https://issues.apache.org/jira/browse/HADOOP-5080) | Update TestCLI with additional test cases. | Minor | test | Ramya Sunil | | +| [HDFS-1099](https://issues.apache.org/jira/browse/HDFS-1099) | Add test for umask backward compatibility | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1043](https://issues.apache.org/jira/browse/HDFS-1043) | Benchmark overhead of server-side group resolution of users | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-919](https://issues.apache.org/jira/browse/HDFS-919) | Create test to validate the BlocksVerified metric | Major | test | gary murry | | +| [HDFS-907](https://issues.apache.org/jira/browse/HDFS-907) | Add tests for getBlockLocations and totalLoad metrics. | Minor | namenode | Ravi Phulari | Ravi Phulari | +| [HDFS-902](https://issues.apache.org/jira/browse/HDFS-902) | Move RAID from HDFS to MR | Major | contrib/raid | Eli Collins | Eli Collins | +| [HDFS-813](https://issues.apache.org/jira/browse/HDFS-813) | Enable the append test in TestReadWhileWriting | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-804](https://issues.apache.org/jira/browse/HDFS-804) | New unit tests for concurrent lease recovery | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-714](https://issues.apache.org/jira/browse/HDFS-714) | Create fault injection test for the new pipeline close | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-713](https://issues.apache.org/jira/browse/HDFS-713) | Need to properly check the type of the test class from an aspect | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-710](https://issues.apache.org/jira/browse/HDFS-710) | Add actions with constraints to the pipeline fault injection tests | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-705](https://issues.apache.org/jira/browse/HDFS-705) | Create an adapter to access some of package-private methods of DataNode from tests | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-669](https://issues.apache.org/jira/browse/HDFS-669) | Add unit tests framework (Mockito) | Major | build | Eli Collins | Konstantin Boudnik | +| [HDFS-451](https://issues.apache.org/jira/browse/HDFS-451) | Test DataTransferProtocol with fault injection | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-409](https://issues.apache.org/jira/browse/HDFS-409) | Add more access token tests | Major | datanode, hdfs-client | Kan Zhang | Kan Zhang | +| [MAPREDUCE-1359](https://issues.apache.org/jira/browse/MAPREDUCE-1359) | TypedBytes TestIO doesn't mkdir its test dir first | Major | contrib/streaming | Todd Lipcon | Anatoli Fomenko | +| [MAPREDUCE-1061](https://issues.apache.org/jira/browse/MAPREDUCE-1061) | Gridmix unit test should validate input/output bytes | Major | . | Chris Douglas | Chris Douglas | +| [MAPREDUCE-1050](https://issues.apache.org/jira/browse/MAPREDUCE-1050) | Introduce a mock object testing framework | Major | test | Tom White | Tom White | +| [MAPREDUCE-793](https://issues.apache.org/jira/browse/MAPREDUCE-793) | Create a new test that consolidates a few tests to be included in the commit-test list | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-785](https://issues.apache.org/jira/browse/MAPREDUCE-785) | Refactor TestReduceFetchFromPartialMem into a separate test | Major | . | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-686](https://issues.apache.org/jira/browse/MAPREDUCE-686) | Move TestSpeculativeExecution.Fake\* into a separate class so that it can be used by other tests also | Major | test | Jothi Padmanabhan | Jothi Padmanabhan | +| [MAPREDUCE-670](https://issues.apache.org/jira/browse/MAPREDUCE-670) | Create target for 10 minute patch test build for mapreduce | Major | build | Jothi Padmanabhan | Jothi Padmanabhan | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6771](https://issues.apache.org/jira/browse/HADOOP-6771) | Herriot's artifact id for Maven deployment should be set to hadoop-core-instrumented | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6752](https://issues.apache.org/jira/browse/HADOOP-6752) | Remote cluster control functionality needs JavaDocs improvement | Major | test | Konstantin Boudnik | Balaji Rajagopalan | +| [HADOOP-6692](https://issues.apache.org/jira/browse/HADOOP-6692) | Add FileContext#listStatus that returns an iterator | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-6668](https://issues.apache.org/jira/browse/HADOOP-6668) | Apply audience and stability annotations to classes in common | Blocker | documentation | Tom White | Tom White | +| [HADOOP-6658](https://issues.apache.org/jira/browse/HADOOP-6658) | Exclude Public elements in generated Javadoc | Blocker | documentation | Tom White | Tom White | +| [HADOOP-6568](https://issues.apache.org/jira/browse/HADOOP-6568) | Authorization for default servlets | Major | security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-6538](https://issues.apache.org/jira/browse/HADOOP-6538) | Set hadoop.security.authentication to "simple" by default | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6486](https://issues.apache.org/jira/browse/HADOOP-6486) | fix common classes to work with Avro 1.3 reflection | Major | ipc | Doug Cutting | Doug Cutting | +| [HADOOP-6422](https://issues.apache.org/jira/browse/HADOOP-6422) | permit RPC protocols to be implemented by Avro | Major | ipc | Doug Cutting | Doug Cutting | +| [HADOOP-6410](https://issues.apache.org/jira/browse/HADOOP-6410) | Rename TestCLI class to prevent JUnit from trying to run this class as a test | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6409](https://issues.apache.org/jira/browse/HADOOP-6409) | TestHDFSCLI has to check if it's running any testcases at all | Blocker | . | Konstantin Boudnik | Todd Lipcon | +| [HADOOP-6223](https://issues.apache.org/jira/browse/HADOOP-6223) | New improved FileSystem interface for those implementing new files systems. | Major | fs | Sanjay Radia | Sanjay Radia | +| [HADOOP-6170](https://issues.apache.org/jira/browse/HADOOP-6170) | add Avro-based RPC serialization | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-5217](https://issues.apache.org/jira/browse/HADOOP-5217) | Split the AllTestDriver for core, hdfs and mapred | Major | test | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-5120](https://issues.apache.org/jira/browse/HADOOP-5120) | UpgradeManagerNamenode and UpgradeObjectNamenode should not use FSNamesystem.getFSNamesystem() | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5097](https://issues.apache.org/jira/browse/HADOOP-5097) | Remove static variable JspHelper.fsn | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-5073](https://issues.apache.org/jira/browse/HADOOP-5073) | Hadoop 1.0 Interface Classification - scope (visibility - public/private) and stability | Major | . | Sanjay Radia | Jakob Homan | +| [HADOOP-5045](https://issues.apache.org/jira/browse/HADOOP-5045) | FileSystem.isDirectory() should not be deprecated. | Major | fs | Tsz Wo Nicholas Sze | Suresh Srinivas | +| [HADOOP-5037](https://issues.apache.org/jira/browse/HADOOP-5037) | Deprecate FSNamesystem.getFSNamesystem() and change fsNamesystemObject to private | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-4930](https://issues.apache.org/jira/browse/HADOOP-4930) | Implement setuid executable for Linux to assist in launching tasks as job owners | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan | +| [HADOOP-4490](https://issues.apache.org/jira/browse/HADOOP-4490) | Map and Reduce tasks should run as the user who submitted the job | Major | security | Arun C Murthy | Hemanth Yamijala | +| [HDFS-1100](https://issues.apache.org/jira/browse/HDFS-1100) | Override TestFcHdfsSymlink#unwrapException | Major | test | Eli Collins | Eli Collins | +| [HDFS-1067](https://issues.apache.org/jira/browse/HDFS-1067) | Create block recovery tests that handle errors | Major | test | Hairong Kuang | Hairong Kuang | +| [HDFS-1057](https://issues.apache.org/jira/browse/HDFS-1057) | Concurrent readers hit ChecksumExceptions if following a writer to very end of file | Blocker | datanode | Todd Lipcon | sam rash | +| [HDFS-730](https://issues.apache.org/jira/browse/HDFS-730) | Add fault injection tests for pipleline close ack | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-719](https://issues.apache.org/jira/browse/HDFS-719) | Add more fault injection tests for pipeline close | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-716](https://issues.apache.org/jira/browse/HDFS-716) | Define a pointcut for pipeline close | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-676](https://issues.apache.org/jira/browse/HDFS-676) | NPE in FSDataset.updateReplicaUnderRecovery(..) | Major | datanode | Tsz Wo Nicholas Sze | Konstantin Shvachko | +| [HDFS-668](https://issues.apache.org/jira/browse/HDFS-668) | TestFileAppend3#TC7 sometimes hangs | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-663](https://issues.apache.org/jira/browse/HDFS-663) | DFSIO for append | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-616](https://issues.apache.org/jira/browse/HDFS-616) | Create functional tests for new design of the block report | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-564](https://issues.apache.org/jira/browse/HDFS-564) | Adding pipeline test 17-35 | Blocker | test | Kan Zhang | Hairong Kuang | +| [HDFS-561](https://issues.apache.org/jira/browse/HDFS-561) | Fix write pipeline READ\_TIMEOUT | Major | datanode, hdfs-client | Kan Zhang | Kan Zhang | +| [HDFS-552](https://issues.apache.org/jira/browse/HDFS-552) | Change TestFiDataTransferProtocol to junit 4 and add a few new tests | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-551](https://issues.apache.org/jira/browse/HDFS-551) | Create new functional test for a block report. | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-524](https://issues.apache.org/jira/browse/HDFS-524) | Further DataTransferProtocol code refactoring. | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-521](https://issues.apache.org/jira/browse/HDFS-521) | Create new tests for pipeline | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-520](https://issues.apache.org/jira/browse/HDFS-520) | Create new tests for block recovery | Major | test | Konstantin Boudnik | Hairong Kuang | +| [HDFS-519](https://issues.apache.org/jira/browse/HDFS-519) | Create new tests for lease recovery | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-508](https://issues.apache.org/jira/browse/HDFS-508) | Factor out BlockInfo from BlocksMap | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-501](https://issues.apache.org/jira/browse/HDFS-501) | Use enum to define the constants in DataTransferProtocol | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-498](https://issues.apache.org/jira/browse/HDFS-498) | Add development guide and framework documentation | Major | documentation | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-475](https://issues.apache.org/jira/browse/HDFS-475) | Create a separate targets for fault injection related test and jar files creation files | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-444](https://issues.apache.org/jira/browse/HDFS-444) | Current fault injection framework implementation doesn't allow to change probability levels dynamically | Minor | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-436](https://issues.apache.org/jira/browse/HDFS-436) | AspectJ framework for HDFS code and tests | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-396](https://issues.apache.org/jira/browse/HDFS-396) | Process dfs.name.edits.dirs as URI | Major | . | Luca Telloli | Luca Telloli | +| [HDFS-377](https://issues.apache.org/jira/browse/HDFS-377) | Code Refactoring: separate codes which implement DataTransferProtocol | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-254](https://issues.apache.org/jira/browse/HDFS-254) | Add more unit test for HDFS symlinks | Major | . | dhruba borthakur | Eli Collins | +| [MAPREDUCE-1791](https://issues.apache.org/jira/browse/MAPREDUCE-1791) | Remote cluster control functionality needs JavaDocs improvement | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1650](https://issues.apache.org/jira/browse/MAPREDUCE-1650) | Exclude Private elements from generated MapReduce Javadoc | Major | documentation | Tom White | Tom White | +| [MAPREDUCE-1625](https://issues.apache.org/jira/browse/MAPREDUCE-1625) | Improve grouping of packages in Javadoc | Blocker | documentation | Tom White | Tom White | +| [MAPREDUCE-1623](https://issues.apache.org/jira/browse/MAPREDUCE-1623) | Apply audience and stability annotations to classes in mapred package | Blocker | documentation | Tom White | Tom White | +| [MAPREDUCE-1457](https://issues.apache.org/jira/browse/MAPREDUCE-1457) | For secure job execution, couple of more UserGroupInformation.doAs needs to be added | Major | . | Devaraj Das | Jakob Homan | +| [MAPREDUCE-1455](https://issues.apache.org/jira/browse/MAPREDUCE-1455) | Authorization for servlets | Major | jobtracker, security, tasktracker | Devaraj Das | Ravi Gummadi | +| [MAPREDUCE-1454](https://issues.apache.org/jira/browse/MAPREDUCE-1454) | The servlets should quote server generated strings sent in the response | Major | . | Devaraj Das | Chris Douglas | +| [MAPREDUCE-1433](https://issues.apache.org/jira/browse/MAPREDUCE-1433) | Create a Delegation token for MapReduce | Major | security | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1432](https://issues.apache.org/jira/browse/MAPREDUCE-1432) | Add the hooks in JobTracker and TaskTracker to load tokens from the token cache into the user's UGI | Major | . | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1430](https://issues.apache.org/jira/browse/MAPREDUCE-1430) | JobTracker should be able to renew delegation tokens for the jobs | Major | jobtracker | Devaraj Das | Boris Shkolnik | +| [MAPREDUCE-1326](https://issues.apache.org/jira/browse/MAPREDUCE-1326) | fi tests don't use fi-site.xml | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-1209](https://issues.apache.org/jira/browse/MAPREDUCE-1209) | Move common specific part of the test TestReflectionUtils out of mapred into common | Blocker | test | Vinod Kumar Vavilapalli | Todd Lipcon | +| [MAPREDUCE-1201](https://issues.apache.org/jira/browse/MAPREDUCE-1201) | Make ProcfsBasedProcessTree collect CPU usage information | Major | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1190](https://issues.apache.org/jira/browse/MAPREDUCE-1190) | Add package.html to pi and pi.math packages. | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1033](https://issues.apache.org/jira/browse/MAPREDUCE-1033) | Resolve location of scripts and configuration files after project split | Blocker | . | Vinod Kumar Vavilapalli | Tom White | +| [MAPREDUCE-1026](https://issues.apache.org/jira/browse/MAPREDUCE-1026) | Shuffle should be secure | Major | security | Owen O'Malley | Boris Shkolnik | +| [MAPREDUCE-975](https://issues.apache.org/jira/browse/MAPREDUCE-975) | Add an API in job client to get the history file url for a given job id | Major | client, jobtracker | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-943](https://issues.apache.org/jira/browse/MAPREDUCE-943) | TestNodeRefresh timesout occasionally | Major | jobtracker | Amareshwari Sriramadasu | Amar Kamat | +| [MAPREDUCE-898](https://issues.apache.org/jira/browse/MAPREDUCE-898) | Change DistributedCache to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-871](https://issues.apache.org/jira/browse/MAPREDUCE-871) | Job/Task local files have incorrect group ownership set by LinuxTaskController binary | Major | tasktracker | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-870](https://issues.apache.org/jira/browse/MAPREDUCE-870) | Clean up the job Retire code | Major | . | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-861](https://issues.apache.org/jira/browse/MAPREDUCE-861) | Modify queue configuration format and parsing to support a hierarchy of queues. | Major | jobtracker | Hemanth Yamijala | rahul k singh | +| [MAPREDUCE-856](https://issues.apache.org/jira/browse/MAPREDUCE-856) | Localized files from DistributedCache should have right access-control | Major | tasktracker | Arun C Murthy | Vinod Kumar Vavilapalli | +| [MAPREDUCE-842](https://issues.apache.org/jira/browse/MAPREDUCE-842) | Per-job local data on the TaskTracker node should have right access-control | Major | security, task-controller, tasktracker | Arun C Murthy | Vinod Kumar Vavilapalli | +| [MAPREDUCE-814](https://issues.apache.org/jira/browse/MAPREDUCE-814) | Move completed Job history files to HDFS | Major | jobtracker | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-744](https://issues.apache.org/jira/browse/MAPREDUCE-744) | Support in DistributedCache to share cache files with other users after HADOOP-4493 | Major | distributed-cache, security, tasktracker | Vinod Kumar Vavilapalli | Devaraj Das | +| [MAPREDUCE-656](https://issues.apache.org/jira/browse/MAPREDUCE-656) | Change org.apache.hadoop.mapred.SequenceFile\* classes to use new api | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-655](https://issues.apache.org/jira/browse/MAPREDUCE-655) | Change KeyValueLineRecordReader and KeyValueTextInputFormat to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-375](https://issues.apache.org/jira/browse/MAPREDUCE-375) | Change org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapred.MapFileOutputFormat to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-373](https://issues.apache.org/jira/browse/MAPREDUCE-373) | Change org.apache.hadoop.mapred.lib. FieldSelectionMapReduce to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-372](https://issues.apache.org/jira/browse/MAPREDUCE-372) | Change org.apache.hadoop.mapred.lib.ChainMapper/Reducer to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-371](https://issues.apache.org/jira/browse/MAPREDUCE-371) | Change org.apache.hadoop.mapred.lib.KeyFieldBasedComparator and org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner to use new api | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-370](https://issues.apache.org/jira/browse/MAPREDUCE-370) | Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-369](https://issues.apache.org/jira/browse/MAPREDUCE-369) | Change org.apache.hadoop.mapred.lib.MultipleInputs to use new api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-364](https://issues.apache.org/jira/browse/MAPREDUCE-364) | Change org.apache.hadoop.examples.MultiFileWordCount to use new mapreduce api. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-361](https://issues.apache.org/jira/browse/MAPREDUCE-361) | Change org.apache.hadoop.examples.terasort to use new mapreduce api | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-181](https://issues.apache.org/jira/browse/MAPREDUCE-181) | Secure job submission | Major | . | Amar Kamat | Devaraj Das | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6839](https://issues.apache.org/jira/browse/HADOOP-6839) | [Herriot] Implement a functionality for getting the user list for creating proxy users. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6772](https://issues.apache.org/jira/browse/HADOOP-6772) | Utilities for system tests specific. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HADOOP-6507](https://issues.apache.org/jira/browse/HADOOP-6507) | Hadoop Common Docs - delete 3 doc files that do not belong under Common | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-6477](https://issues.apache.org/jira/browse/HADOOP-6477) | 0.21.0 - upload of the latest snapshot to apache snapshot repository | Major | . | Karthik K | | +| [HADOOP-6353](https://issues.apache.org/jira/browse/HADOOP-6353) | Create Apache Wiki page for JSure and FlashLight tools | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6346](https://issues.apache.org/jira/browse/HADOOP-6346) | Add support for specifying unpack pattern regex to RunJar.unJar | Major | conf, util | Todd Lipcon | Todd Lipcon | +| [HADOOP-6329](https://issues.apache.org/jira/browse/HADOOP-6329) | Add build-fi directory to the ignore list | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-6321](https://issues.apache.org/jira/browse/HADOOP-6321) | Hadoop Common - Site logo | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-6292](https://issues.apache.org/jira/browse/HADOOP-6292) | Native Libraries Guide - Update | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-6217](https://issues.apache.org/jira/browse/HADOOP-6217) | Hadoop Doc Split: Common Docs | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [HADOOP-6155](https://issues.apache.org/jira/browse/HADOOP-6155) | deprecate Record IO | Major | record | Owen O'Malley | Tom White | +| [HDFS-1277](https://issues.apache.org/jira/browse/HDFS-1277) | [Herriot] New property for multi user list. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HDFS-1174](https://issues.apache.org/jira/browse/HDFS-1174) | New properties for suspend and resume process. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HDFS-869](https://issues.apache.org/jira/browse/HDFS-869) | 0.21.0 - snapshot incorrect dependency published in .pom files | Critical | build | Karthik K | Giridharan Kesavan | +| [HDFS-715](https://issues.apache.org/jira/browse/HDFS-715) | Hadoop HDFS - Site Logo | Blocker | . | Corinne Chandel | Corinne Chandel | +| [HDFS-651](https://issues.apache.org/jira/browse/HDFS-651) | HDFS Docs - fix listing of docs in the doc menu | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [HDFS-574](https://issues.apache.org/jira/browse/HDFS-574) | Hadoop Doc Split: HDFS Docs | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [HDFS-256](https://issues.apache.org/jira/browse/HDFS-256) | Split HDFS into sub project | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-1896](https://issues.apache.org/jira/browse/MAPREDUCE-1896) | [Herriot] New property for multi user list. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-1812](https://issues.apache.org/jira/browse/MAPREDUCE-1812) | New properties for suspend and resume process. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-1404](https://issues.apache.org/jira/browse/MAPREDUCE-1404) | Cluster-Setup and Single-Node-Setup Docs | Blocker | documentation | Corinne Chandel | Tom White | +| [MAPREDUCE-1388](https://issues.apache.org/jira/browse/MAPREDUCE-1388) | Move RAID from HDFS to MR | Major | . | Eli Collins | Eli Collins | +| [MAPREDUCE-1352](https://issues.apache.org/jira/browse/MAPREDUCE-1352) | 0.21.0 - snapshot incorrect dependency published in .pom files | Critical | build | Karthik K | Giridharan Kesavan | +| [MAPREDUCE-1121](https://issues.apache.org/jira/browse/MAPREDUCE-1121) | Hadoop MapReduce - Site Logo | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [MAPREDUCE-1063](https://issues.apache.org/jira/browse/MAPREDUCE-1063) | Document Gridmix benchmark | Minor | benchmarks | Chris Douglas | Chris Douglas | +| [MAPREDUCE-916](https://issues.apache.org/jira/browse/MAPREDUCE-916) | Hadoop Doc Split: MapReduce Docs | Blocker | documentation | Corinne Chandel | Corinne Chandel | +| [MAPREDUCE-878](https://issues.apache.org/jira/browse/MAPREDUCE-878) | Rename fair scheduler design doc to fair-scheduler-design-doc.tex and add Apache license header | Trivial | contrib/fair-share, documentation | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-831](https://issues.apache.org/jira/browse/MAPREDUCE-831) | Put fair scheduler design doc in SVN | Trivial | contrib/fair-share | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-819](https://issues.apache.org/jira/browse/MAPREDUCE-819) | DistCP Guide - updates | Major | documentation | Corinne Chandel | Corinne Chandel | +| [MAPREDUCE-813](https://issues.apache.org/jira/browse/MAPREDUCE-813) | Streaming Doc and M/R-Tutorial Doc - updates | Minor | documentation | Corinne Chandel | | +| [MAPREDUCE-665](https://issues.apache.org/jira/browse/MAPREDUCE-665) | Move libhdfs to HDFS project | Blocker | build | Tsz Wo Nicholas Sze | Eli Collins | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/RELEASENOTES.0.21.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/RELEASENOTES.0.21.0.md new file mode 100644 index 00000000000..97016b2b4a9 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.0/RELEASENOTES.0.21.0.md @@ -0,0 +1,2232 @@ + + +# Apache Hadoop 0.21.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-6813](https://issues.apache.org/jira/browse/HADOOP-6813) | *Blocker* | **Add a new newInstance method in FileSystem that takes a "user" as argument** + +I've just committed this to 0.21. + + +--- + +* [HADOOP-6748](https://issues.apache.org/jira/browse/HADOOP-6748) | *Major* | **Remove hadoop.cluster.administrators** + +Removed configuration property "hadoop.cluster.administrators". Added constructor public HttpServer(String name, String bindAddress, int port, boolean findPort, Configuration conf, AccessControlList adminsAcl) in HttpServer, which takes cluster administrators acl as a parameter. + + +--- + +* [HADOOP-6701](https://issues.apache.org/jira/browse/HADOOP-6701) | *Minor* | ** Incorrect exit codes for "dfs -chown", "dfs -chgrp"** + +Commands chmod, chown and chgrp now returns non zero exit code and an error message on failure instead of returning zero. + + +--- + +* [HADOOP-6692](https://issues.apache.org/jira/browse/HADOOP-6692) | *Major* | **Add FileContext#listStatus that returns an iterator** + +This issue adds Iterator\ listStatus(Path) to FileContext, moves FileStatus[] listStatus(Path) to FileContext#Util, and adds Iterator\ listStatusItor(Path) to AbstractFileSystem which provides a default implementation by using FileStatus[] listStatus(Path). + + +--- + +* [HADOOP-6686](https://issues.apache.org/jira/browse/HADOOP-6686) | *Major* | **Remove redundant exception class name in unwrapped exceptions thrown at the RPC client** + +The exceptions thrown by the RPC client no longer carries a redundant exception class name in exception message. + + +--- + +* [HADOOP-6577](https://issues.apache.org/jira/browse/HADOOP-6577) | *Major* | **IPC server response buffer reset threshold should be configurable** + +Add hidden configuration option "ipc.server.max.response.size" to change the default 1 MB, the maximum size when large IPC handler response buffer is reset. + + +--- + +* [HADOOP-6569](https://issues.apache.org/jira/browse/HADOOP-6569) | *Major* | **FsShell#cat should avoid calling unecessary getFileStatus before opening a file to read** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6568](https://issues.apache.org/jira/browse/HADOOP-6568) | *Major* | **Authorization for default servlets** + +Added web-authorization for the default servlets - /logs, /stacks, /logLevel, /metrics, /conf, so that only cluster administrators can access these servlets. hadoop.cluster.administrators is the new configuration in core-default.xml that can be used to specify the ACL against which an authenticated user should be verified if he/she is an administrator. + + +--- + +* [HADOOP-6537](https://issues.apache.org/jira/browse/HADOOP-6537) | *Major* | **Proposal for exceptions thrown by FileContext and Abstract File System** + +Detailed exceptions declared in FileContext and AbstractFileSystem + + +--- + +* [HADOOP-6531](https://issues.apache.org/jira/browse/HADOOP-6531) | *Minor* | **add FileUtil.fullyDeleteContents(dir) api to delete contents of a directory** + +Added an api FileUtil.fullyDeleteContents(String dir) to delete contents of the directory. + + +--- + +* [HADOOP-6515](https://issues.apache.org/jira/browse/HADOOP-6515) | *Major* | **Make maximum number of http threads configurable** + +HADOOP-6515. Make maximum number of http threads configurable (Scott Chen via zshao) + + +--- + +* [HADOOP-6489](https://issues.apache.org/jira/browse/HADOOP-6489) | *Major* | **Findbug report: LI\_LAZY\_INIT\_STATIC, OBL\_UNSATISFIED\_OBLIGATION** + +Fix 3 findsbugs warnings. + + +--- + +* [HADOOP-6441](https://issues.apache.org/jira/browse/HADOOP-6441) | *Major* | **Prevent remote CSS attacks in Hostname and UTF-7.** + +Quotes the characters coming out of getRequestUrl and getServerName in HttpServer.java as per the specification in HADOOP-6151. + + +--- + +* [HADOOP-6433](https://issues.apache.org/jira/browse/HADOOP-6433) | *Major* | **Add AsyncDiskService that is used in both hdfs and mapreduce** + +HADOOP-6433. Add AsyncDiskService for asynchronous disk services. + + +--- + +* [HADOOP-6386](https://issues.apache.org/jira/browse/HADOOP-6386) | *Blocker* | **NameNode's HttpServer can't instantiate InetSocketAddress: IllegalArgumentException is thrown** + +Improved initialization sequence so that Port Out of Range error when starting web server will less likely interrupt testing. + + +--- + +* [HADOOP-6367](https://issues.apache.org/jira/browse/HADOOP-6367) | *Major* | **Move Access Token implementation from Common to HDFS** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6344](https://issues.apache.org/jira/browse/HADOOP-6344) | *Major* | **rm and rmr fail to correctly move the user's files to the trash prior to deleting when they are over quota.** + +Trash feature notifies user of over-quota condition rather than silently deleting files/directories; deletion can be compelled with "rm -skiptrash". + + +--- + +* [HADOOP-6343](https://issues.apache.org/jira/browse/HADOOP-6343) | *Major* | **Stack trace of any runtime exceptions should be recorded in the server logs.** + +Record runtime exceptions in server log to facilitate fault analysis. + + +--- + +* [HADOOP-6313](https://issues.apache.org/jira/browse/HADOOP-6313) | *Major* | **Expose flush APIs to application users** + +FSOutputDataStream implement Syncable interface to provide hflush and hsync APIs to the application users. + + +--- + +* [HADOOP-6299](https://issues.apache.org/jira/browse/HADOOP-6299) | *Major* | **Use JAAS LoginContext for our login** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6281](https://issues.apache.org/jira/browse/HADOOP-6281) | *Major* | **HtmlQuoting throws NullPointerException** + +Fixed null pointer error when quoting HTML in the case JSP has no parameters. + + +--- + +* [HADOOP-6235](https://issues.apache.org/jira/browse/HADOOP-6235) | *Major* | **Adding a new method for getting server default values from a FileSystem** + +New FileSystem method reports default parameters that would be used by server. See also HDFS-578. + + +--- + +* [HADOOP-6234](https://issues.apache.org/jira/browse/HADOOP-6234) | *Major* | **Permission configuration files should use octal and symbolic** + +New configuration option dfs.umaskmode sets umask with octal or symbolic value. + + +--- + +* [HADOOP-6230](https://issues.apache.org/jira/browse/HADOOP-6230) | *Major* | **Move process tree, and memory calculator classes out of Common into Map/Reduce.** + +Moved process tree, and memory calculator classes out of Common project into the Map/Reduce project. + + +--- + +* [HADOOP-6226](https://issues.apache.org/jira/browse/HADOOP-6226) | *Major* | **Create a LimitedByteArrayOutputStream that does not expand its buffer on write** + +New LimitedByteArrayOutputStream does not expand buffer on writes. + + +--- + +* [HADOOP-6223](https://issues.apache.org/jira/browse/HADOOP-6223) | *Major* | **New improved FileSystem interface for those implementing new files systems.** + +Add new file system interface AbstractFileSystem with implementation of some file systems that delegate to old FileSystem. + + +--- + +* [HADOOP-6203](https://issues.apache.org/jira/browse/HADOOP-6203) | *Major* | **Improve error message when moving to trash fails due to quota issue** + +Improved error message suggests using -skpTrash option when hdfs -rm fails to move to trash because of quota. + + +--- + +* [HADOOP-6201](https://issues.apache.org/jira/browse/HADOOP-6201) | *Major* | **FileSystem::ListStatus should throw FileNotFoundException** + +FileSystem listStatus method throws FileNotFoundException for all implementations. Application code should catch or propagate FileNotFoundException. + + +--- + +* [HADOOP-6184](https://issues.apache.org/jira/browse/HADOOP-6184) | *Major* | **Provide a configuration dump in json format.** + +New Configuration.dumpConfiguration(Configuration, Writer) writes configuration parameters in the JSON format. + + +--- + +* [HADOOP-6170](https://issues.apache.org/jira/browse/HADOOP-6170) | *Major* | **add Avro-based RPC serialization** + +RPC can use Avro serialization. + + +--- + +* [HADOOP-6161](https://issues.apache.org/jira/browse/HADOOP-6161) | *Minor* | **Add get/setEnum to Configuration** + +Added following APIs to Configuration: +- public \\> T getEnum(String name, T defaultValue) +- public \\> void setEnum(String name, T value) + + +--- + +* [HADOOP-6151](https://issues.apache.org/jira/browse/HADOOP-6151) | *Critical* | **The servlets should quote html characters** + +The input parameters for all of the servlets will have the 5 html meta characters quoted. The characters are '&', '\<', '\>', '"' and the apostrophe. The goal is to ensure that our web ui servlets can't be used for cross site scripting (XSS) attacks. In particular, it blocks the frequent (especially for errors) case where the servlet echos back the parameters to the user. + + +--- + +* [HADOOP-6120](https://issues.apache.org/jira/browse/HADOOP-6120) | *Major* | **Add support for Avro types in hadoop** + +New Avro serialization in .../io/serializer/avro. + + +--- + +* [HADOOP-5976](https://issues.apache.org/jira/browse/HADOOP-5976) | *Major* | **create script to provide classpath for external tools** + +New Hadoop script command classpath prints the path to the Hadoop jar and libraries. + + +--- + +* [HADOOP-5913](https://issues.apache.org/jira/browse/HADOOP-5913) | *Major* | **Allow administrators to be able to start and stop queues** + +New mradmin command -refreshQueues reads new configuration of ACLs and queue states from mapred-queues.xml. If the new queue state is not "running," jobs in progress will continue, but no other jobs from that queue will be started. + + +--- + +* [HADOOP-5887](https://issues.apache.org/jira/browse/HADOOP-5887) | *Major* | **Sqoop should create tables in Hive metastore after importing to HDFS** + +New Sqoop argument --hive-import facilitates loading data into Hive. + + +--- + +* [HADOOP-5879](https://issues.apache.org/jira/browse/HADOOP-5879) | *Major* | **GzipCodec should read compression level etc from configuration** + +Provide an ability to configure the compression level and strategy for codecs. Compressors need to be 'reinited' with new characteristics such as compression level etc. and hence an incompatible addition to the api. + + +--- + +* [HADOOP-5861](https://issues.apache.org/jira/browse/HADOOP-5861) | *Major* | **s3n files are not getting split by default** + +Files stored on the native S3 filesystem (s3n:// URIs) now report a block size determined by the fs.s3n.block.size property (default 64MB). + + +--- + +* [HADOOP-5815](https://issues.apache.org/jira/browse/HADOOP-5815) | *Major* | **Sqoop: A database import tool for Hadoop** + +New contribution Sqoop is a JDBC-based database import tool for Hadoop. + + +--- + +* [HADOOP-5784](https://issues.apache.org/jira/browse/HADOOP-5784) | *Major* | **The length of the heartbeat cycle should be configurable.** + +Introduced a configuration parameter, mapred.heartbeats.in.second, as an expert option, that defines how many heartbeats a jobtracker can process in a second. Administrators can set this to an appropriate value based on cluster size and expected processing time on the jobtracker to achieve a balance between jobtracker scalability and latency of jobs. + + +--- + +* [HADOOP-5771](https://issues.apache.org/jira/browse/HADOOP-5771) | *Major* | **Create unit test for LinuxTaskController** + +Added unit tests for verifying LinuxTaskController functionality. + + +--- + +* [HADOOP-5752](https://issues.apache.org/jira/browse/HADOOP-5752) | *Major* | **Provide examples of using offline image viewer (oiv) to analyze hadoop file systems** + +Additional examples and documentation for HDFS Offline Image Viewer Tool show how to generate Pig-friendly data and to do analysis with Pig. + + +--- + +* [HADOOP-5745](https://issues.apache.org/jira/browse/HADOOP-5745) | *Major* | **Allow setting the default value of maxRunningJobs for all pools** + +New Fair Scheduler configuration parameter sets a default limit on number of running jobs for all pools. + + +--- + +* [HADOOP-5738](https://issues.apache.org/jira/browse/HADOOP-5738) | *Major* | **Split waiting tasks field in JobTracker metrics to individual tasks** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-5737](https://issues.apache.org/jira/browse/HADOOP-5737) | *Major* | **UGI checks in testcases are broken** + +Fixed JobTracker to use it's own credentials instead of the job's credentials for accessing mapred.system.dir. Also added APIs in the JobTracker to get the FileSystem objects as per the JobTracker's configuration. + + +--- + +* [HADOOP-5679](https://issues.apache.org/jira/browse/HADOOP-5679) | *Major* | **Resolve findbugs warnings in core/streaming/pipes/examples** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-5675](https://issues.apache.org/jira/browse/HADOOP-5675) | *Minor* | **DistCp should not launch a job if it is not necessary** + +Distcp will no longer start jobs that move no data. + + +--- + +* [HADOOP-5643](https://issues.apache.org/jira/browse/HADOOP-5643) | *Major* | **Ability to blacklist tasktracker** + +New mradmin command -refreshNodes updates the job tracker's node lists. + + +--- + +* [HADOOP-5620](https://issues.apache.org/jira/browse/HADOOP-5620) | *Major* | **discp can preserve modification times of files** + +New DistCp option -pt preserves last modification and last access times of copied files. + + +--- + +* [HADOOP-5592](https://issues.apache.org/jira/browse/HADOOP-5592) | *Minor* | **Hadoop Streaming - GzipCodec** + +Updates streaming documentation to correct the name used for the GZipCodec. + + +--- + +* [HADOOP-5582](https://issues.apache.org/jira/browse/HADOOP-5582) | *Major* | **Hadoop Vaidya throws number format exception due to changes in the job history counters string format (escaped compact representation).** + +Fixed error parsing job history counters after change of counter format. + + +--- + +* [HADOOP-5528](https://issues.apache.org/jira/browse/HADOOP-5528) | *Major* | **Binary partitioner** + +New BinaryPartitioner that partitions BinaryComparable keys by hashing a configurable part of the bytes array corresponding to the key. + + +--- + +* [HADOOP-5518](https://issues.apache.org/jira/browse/HADOOP-5518) | *Major* | **MRUnit unit test library** + +New contribution MRUnit helps authors of map-reduce programs write unit tests with JUnit. + + +--- + +* [HADOOP-5485](https://issues.apache.org/jira/browse/HADOOP-5485) | *Major* | **Authorisation machanism required for acceesing jobtracker url :- jobtracker.com:port/scheduler** + +New Fair Scheduler configuration parameter webinterface.private.actions controls whether changes to pools and priorities are permitted from the web interface. Changes are not permitted by default. + + +--- + +* [HADOOP-5469](https://issues.apache.org/jira/browse/HADOOP-5469) | *Major* | **Exposing Hadoop metrics via HTTP** + +New server web page .../metrics allows convenient access to metrics data via JSON and text. + + +--- + +* [HADOOP-5467](https://issues.apache.org/jira/browse/HADOOP-5467) | *Major* | **Create an offline fsimage image viewer** + +New Offline Image Viewer (oiv) tool reads an fsimage file and writes the data in a variety of user-friendly formats, including XML. + + +--- + +* [HADOOP-5464](https://issues.apache.org/jira/browse/HADOOP-5464) | *Major* | **DFSClient does not treat write timeout of 0 properly** + +Zero values for dfs.socket.timeout and dfs.datanode.socket.write.timeout are now respected. Previously zero values for these parameters resulted in a 5 second timeout. + + +--- + +* [HADOOP-5457](https://issues.apache.org/jira/browse/HADOOP-5457) | *Major* | **Failing contrib tests should not stop the build** + +Fixed the build to make sure that all the unit tests in contrib are run, regardless of the success/failure status of the previous projects' tests. + + +--- + +* [HADOOP-5438](https://issues.apache.org/jira/browse/HADOOP-5438) | *Major* | **Merge FileSystem.create and FileSystem.append** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-5396](https://issues.apache.org/jira/browse/HADOOP-5396) | *Major* | **Queue ACLs should be refreshed without requiring a restart of the job tracker** + +Job Tracker queue ACLs can be changed without restarting Job Tracker. + + +--- + +* [HADOOP-5363](https://issues.apache.org/jira/browse/HADOOP-5363) | *Major* | **Proxying for multiple HDFS clusters of different versions** + +New HDFS proxy server (Tomcat based) allows clients controlled access to clusters with different versions. See Hadoop-5366 for information on using curl and wget. + + +--- + +* [HADOOP-5258](https://issues.apache.org/jira/browse/HADOOP-5258) | *Major* | **Provide dfsadmin functionality to report on namenode's view of network topology** + +New dfsAdmin command -printTopology shows topology as understood by the namenode. + + +--- + +* [HADOOP-5257](https://issues.apache.org/jira/browse/HADOOP-5257) | *Minor* | **Export namenode/datanode functionality through a pluggable RPC layer** + +New plugin facility for namenode and datanode instantiates classes named in new configuration properties dfs.datanode.plugins and dfs.namenode.plugins. + + +--- + +* [HADOOP-5222](https://issues.apache.org/jira/browse/HADOOP-5222) | *Minor* | **Add offset in client trace** + +Include IO offset to client trace logging output. + + +--- + +* [HADOOP-5219](https://issues.apache.org/jira/browse/HADOOP-5219) | *Major* | **SequenceFile is using mapred property** + +New configuration parameter io.seqfile.local.dir for use by SequenceFile replaces mapred.local.dir. + + +--- + +* [HADOOP-5191](https://issues.apache.org/jira/browse/HADOOP-5191) | *Minor* | **After creation and startup of the hadoop namenode on AIX or Solaris, you will only be allowed to connect to the namenode via hostname but not IP.** + +Accessing HDFS with any ip, hostname, or proxy should work as long as it points to the interface NameNode is listening on. + + +--- + +* [HADOOP-5176](https://issues.apache.org/jira/browse/HADOOP-5176) | *Trivial* | **TestDFSIO reports itself as TestFDSIO** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-5175](https://issues.apache.org/jira/browse/HADOOP-5175) | *Major* | **Option to prohibit jars unpacking** + +Jars passed to the -libjars option of hadoop jars are no longer unpacked inside mapred.local.dir. + + +--- + +* [HADOOP-5144](https://issues.apache.org/jira/browse/HADOOP-5144) | *Major* | **manual way of turning on restore of failed storage replicas for namenode** + +New DFSAdmin command -restoreFailedStorage true|false|check sets policy for restoring failed fsimage/editslog volumes. + + +--- + +* [HADOOP-5094](https://issues.apache.org/jira/browse/HADOOP-5094) | *Minor* | **Show dead nodes information in dfsadmin -report** + +Changed df dfsadmin -report to list live and dead nodes, and attempt to resolve the hostname of datanode ip addresses. + + +--- + +* [HADOOP-5073](https://issues.apache.org/jira/browse/HADOOP-5073) | *Major* | **Hadoop 1.0 Interface Classification - scope (visibility - public/private) and stability** + +Annotation mechanism enables interface classification. + + +--- + +* [HADOOP-5052](https://issues.apache.org/jira/browse/HADOOP-5052) | *Major* | **Add an example for computing exact digits of Pi** + +New example BaileyBorweinPlouffe computes digits of pi. (World record!) + + +--- + +* [HADOOP-5042](https://issues.apache.org/jira/browse/HADOOP-5042) | *Major* | ** Add expiration handling to the chukwa log4j appender** + +Chukwwa Log4J appender options allow a retention policy to limit number of files. + + +--- + +* [HADOOP-5022](https://issues.apache.org/jira/browse/HADOOP-5022) | *Blocker* | **[HOD] logcondense should delete all hod logs for a user, including jobtracker logs** + +New logcondense option retain-master-logs indicates whether the script should delete master logs as part of its cleanup process. By default this option is false; master logs are deleted. Earlier versions of logcondense did not delete master logs. + + +--- + +* [HADOOP-5018](https://issues.apache.org/jira/browse/HADOOP-5018) | *Major* | **Chukwa should support pipelined writers** + +Chukwa supports pipelined writers for improved extensibility. + + +--- + +* [HADOOP-4952](https://issues.apache.org/jira/browse/HADOOP-4952) | *Major* | **Improved files system interface for the application writer.** + +New FileContext API introduced to replace FileSystem API. FileContext will be the version-compatible API for future releases. FileSystem API will be deprecated in the next release. + + +--- + +* [HADOOP-4942](https://issues.apache.org/jira/browse/HADOOP-4942) | *Major* | **Remove getName() and getNamed(String name, Configuration conf)** + +Removed deprecated methods getName() and getNamed(String, Configuration) from FileSystem and descendant classes. + + +--- + +* [HADOOP-4941](https://issues.apache.org/jira/browse/HADOOP-4941) | *Major* | **Remove getBlockSize(Path f), getLength(Path f) and getReplication(Path src)** + +Removed deprecated FileSystem methods getBlockSize(Path f), getLength(Path f), and getReplication(Path src). + + +--- + +* [HADOOP-4940](https://issues.apache.org/jira/browse/HADOOP-4940) | *Major* | **Remove delete(Path f)** + +Removed deprecated method FileSystem.delete(Path). + + +--- + +* [HADOOP-4933](https://issues.apache.org/jira/browse/HADOOP-4933) | *Blocker* | **ConcurrentModificationException in JobHistory.java** + +Fixed a synchronization bug in job history content parsing that could result in garbled history data or a ConcurrentModificationException. + + +--- + +* [HADOOP-4927](https://issues.apache.org/jira/browse/HADOOP-4927) | *Major* | **Part files on the output filesystem are created irrespective of whether the corresponding task has anything to write there** + +All output part files are created regardless of whether the corresponding task has output. + + +--- + +* [HADOOP-4895](https://issues.apache.org/jira/browse/HADOOP-4895) | *Major* | **Remove deprecated methods in DFSClient** + +Removed deprecated methods DFSClient.getHints() and DFSClient.isDirectory(). + + +--- + +* [HADOOP-4885](https://issues.apache.org/jira/browse/HADOOP-4885) | *Major* | **Try to restore failed replicas of Name Node storage (at checkpoint time)** + +Patch introduces new configuration switch dfs.name.dir.restore (boolean) enabling this functionality. Documentation needs to be updated. + +UPDATE: Config key is now "dfs.namenode.name.dir.restore" for 1.x and 2.x+ versions of HDFS + + +--- + +* [HADOOP-4861](https://issues.apache.org/jira/browse/HADOOP-4861) | *Trivial* | **Add disk usage with human-readable size (-duh)** + +Output of hadoop fs -dus changed to be consistent with hadoop fs -du and with Linux du. Users who previously parsed this output should update their scripts. New feature hadoop fs -du -h may be used for human readable output. + + +--- + +* [HADOOP-4842](https://issues.apache.org/jira/browse/HADOOP-4842) | *Major* | **Streaming combiner should allow command, not just JavaClass** + +Streaming option -combiner allows any streaming command (not just Java class) to be a combiner. + + +--- + +* [HADOOP-4829](https://issues.apache.org/jira/browse/HADOOP-4829) | *Minor* | **Allow FileSystem shutdown hook to be disabled** + +New configuration parameter fs.automatic.close can be set false to disable the JVM shutdown hook that automatically closes FileSystems. + + +--- + +* [HADOOP-4779](https://issues.apache.org/jira/browse/HADOOP-4779) | *Major* | **Remove deprecated FileSystem methods** + +Removed deprecated FileSystem methods . + + +--- + +* [HADOOP-4768](https://issues.apache.org/jira/browse/HADOOP-4768) | *Major* | **Dynamic Priority Scheduler that allows queue shares to be controlled dynamically by a currency** + +New contribution Dynamic Scheduler implements dynamic priorities with a currency model. Usage instructions are in the Jira item. + + +--- + +* [HADOOP-4756](https://issues.apache.org/jira/browse/HADOOP-4756) | *Major* | **Create a command line tool to access JMX exported properties from a NameNode server** + +New HDFS tool JMXGet facilitates command line access to statistics via JMX. + + +--- + +* [HADOOP-4655](https://issues.apache.org/jira/browse/HADOOP-4655) | *Major* | **FileSystem.CACHE should be ref-counted** + +Every invocation of FileSystem.newInstance() returns a newly allocated FileSystem object. This may be an incompatible change for applications that relied on FileSystem object identity. + + +--- + +* [HADOOP-4648](https://issues.apache.org/jira/browse/HADOOP-4648) | *Major* | **Remove ChecksumDistriubtedFileSystem and InMemoryFileSystem** + +Removed obsolete, deprecated subclasses of ChecksumFileSystem (InMemoryFileSystem, ChecksumDistributedFileSystem). + + +--- + +* [HADOOP-4539](https://issues.apache.org/jira/browse/HADOOP-4539) | *Major* | **Streaming Edits to a Backup Node.** + +Introduced backup node which maintains the up-to-date state of the namespace by receiving edits from the namenode, and checkpoint node, which creates checkpoints of the name space. These facilities replace the secondary namenode. + + +--- + +* [HADOOP-4368](https://issues.apache.org/jira/browse/HADOOP-4368) | *Minor* | **Superuser privileges required to do "df"** + +New filesystem shell command -df reports capacity, space used and space free. Any user may execute this command without special privileges. + + +--- + +* [HADOOP-4359](https://issues.apache.org/jira/browse/HADOOP-4359) | *Major* | **Access Token: Support for data access authorization checking on DataNodes** + +Introduced access tokens as capabilities for accessing datanodes. This change to internal protocols does not affect client applications. + + +--- + +* [HADOOP-4268](https://issues.apache.org/jira/browse/HADOOP-4268) | *Major* | **Permission checking in fsck** + +Fsck now checks permissions as directories are traversed. Any user can now use fsck, but information is provided only for directories the user has permission to read. + + +--- + +* [HADOOP-4041](https://issues.apache.org/jira/browse/HADOOP-4041) | *Major* | **IsolationRunner does not work as documented** + +Fixed a bug in IsolationRunner to make it work for map tasks. + + +--- + +* [HADOOP-4012](https://issues.apache.org/jira/browse/HADOOP-4012) | *Major* | **Providing splitting support for bzip2 compressed files** + +BZip2 files can now be split. + + +--- + +* [HADOOP-3953](https://issues.apache.org/jira/browse/HADOOP-3953) | *Major* | **Sticky bit for directories** + +UNIX-style sticky bit implemented for HDFS directories. When the sticky bit is set on a directory, files in that directory may be deleted or renamed only by a superuser or the file's owner. + + +--- + +* [HADOOP-3741](https://issues.apache.org/jira/browse/HADOOP-3741) | *Major* | **SecondaryNameNode has http server on dfs.secondary.http.address but without any contents** + +Backup namenode's web UI default page now has some useful content. + + +--- + +* [HADOOP-2827](https://issues.apache.org/jira/browse/HADOOP-2827) | *Major* | **Remove deprecated NetUtils.getServerAddress** + +Removed deprecated NetUtils.getServerAddress. + + +--- + +* [HADOOP-1722](https://issues.apache.org/jira/browse/HADOOP-1722) | *Major* | **Make streaming to handle non-utf8 byte array** + +Streaming allows binary (or other non-UTF8) streams. + + +--- + +* [HDFS-1024](https://issues.apache.org/jira/browse/HDFS-1024) | *Blocker* | **SecondaryNamenode fails to checkpoint because namenode fails with CancelledKeyException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-1016](https://issues.apache.org/jira/browse/HDFS-1016) | *Major* | **HDFS side change for HADOOP-6569** + +When cat a directory or a non-existent file from the command line, the error message gets printed becomes +cat: io.java.FileNotFoundException: File does not exist: \ + + +--- + +* [HDFS-1012](https://issues.apache.org/jira/browse/HDFS-1012) | *Major* | **documentLocation attribute in LdapEntry for HDFSProxy isn't specific to a cluster** + +Support for fully qualified HDFS path in addition to simple unqualified path. +The qualified path indicates that the path is accessible on the specific HDFS. Non qualified path is qualified in all clusters. + + +--- + +* [HDFS-998](https://issues.apache.org/jira/browse/HDFS-998) | *Major* | **The servlets should quote server generated strings sent in the response** + +The servlets should quote server generated strings sent in the response. + + +--- + +* [HDFS-985](https://issues.apache.org/jira/browse/HDFS-985) | *Major* | **HDFS should issue multiple RPCs for listing a large directory** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-984](https://issues.apache.org/jira/browse/HDFS-984) | *Major* | **Delegation Tokens should be persisted in Namenode** + +Layout version is set to -24 reflecting changes in edits log and fsimage format related to persisting delegation tokens. + + +--- + +* [HDFS-946](https://issues.apache.org/jira/browse/HDFS-946) | *Major* | **NameNode should not return full path name when lisitng a diretory or getting the status of a file** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-913](https://issues.apache.org/jira/browse/HDFS-913) | *Major* | **TestRename won't run automatically from 'run-test-hdfs-faul-inject' target** + +HDFS-913. Rename fault injection test TestRename.java to TestFiRename.java to include it in tests run by ant target run-test-hdfs-fault-inject. + + +--- + +* [HDFS-897](https://issues.apache.org/jira/browse/HDFS-897) | *Major* | **ReplicasMap remove has a bug in generation stamp comparison** + +Fixed a bug in ReplicasMap.remove method, which compares the generation stamp of the replica removed to itself instead of the the block passed to the method to identify the replica to be removed. + + +--- + +* [HDFS-892](https://issues.apache.org/jira/browse/HDFS-892) | *Major* | **optionally use Avro for namenode RPC** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-873](https://issues.apache.org/jira/browse/HDFS-873) | *Major* | **DataNode directories as URIs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-850](https://issues.apache.org/jira/browse/HDFS-850) | *Minor* | **Display more memory details on the web ui** + +Changes the format of the message with Heap usage on the NameNode web page. + + +--- + +* [HDFS-814](https://issues.apache.org/jira/browse/HDFS-814) | *Major* | **Add an api to get the visible length of a DFSDataInputStream.** + +Add an api to get the visible length of a DFSDataInputStream. + + +--- + +* [HDFS-793](https://issues.apache.org/jira/browse/HDFS-793) | *Blocker* | **DataNode should first receive the whole packet ack message before it constructs and sends its own ack message for the packet** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-785](https://issues.apache.org/jira/browse/HDFS-785) | *Minor* | **Missing license header in java source files.** + +Add the Apache license header to several files that are missing it. + + +--- + +* [HDFS-781](https://issues.apache.org/jira/browse/HDFS-781) | *Blocker* | **Metrics PendingDeletionBlocks is not decremented** + +Correct PendingDeletionBlocks metric to properly decrement counts. + + +--- + +* [HDFS-764](https://issues.apache.org/jira/browse/HDFS-764) | *Major* | **Moving Access Token implementation from Common to HDFS** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-761](https://issues.apache.org/jira/browse/HDFS-761) | *Major* | **Failure to process rename operation from edits log due to quota verification** + +Corrected an error when checking quota policy that resulted in a failure to read the edits log, stopping the primary/secondary name node. + + +--- + +* [HDFS-758](https://issues.apache.org/jira/browse/HDFS-758) | *Major* | **Improve reporting of progress of decommissioning** + +New name node web UI page displays details of decommissioning progress. (dfsnodelist.jsp?whatNodes=DECOMMISSIONING) + + +--- + +* [HDFS-737](https://issues.apache.org/jira/browse/HDFS-737) | *Major* | **Improvement in metasave output** + +Add full path name of the file to the under replicated block information and summary of total number of files, blocks, live and dead datanodes to metasave output. + + +--- + +* [HDFS-702](https://issues.apache.org/jira/browse/HDFS-702) | *Major* | **Add Hdfs Impl for the new file system interface** + +Add HDFS implementation of AbstractFileSystem. + + +--- + +* [HDFS-677](https://issues.apache.org/jira/browse/HDFS-677) | *Blocker* | **Rename failure due to quota results in deletion of src directory** + +Rename properly considers the case where both source and destination are over quota; operation will fail with error indication. + + +--- + +* [HDFS-660](https://issues.apache.org/jira/browse/HDFS-660) | *Major* | **Remove deprecated methods from InterDatanodeProtocol.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-631](https://issues.apache.org/jira/browse/HDFS-631) | *Major* | **Changes in HDFS to rename the config keys as detailed in HDFS-531.** + +File system configuration keys renamed as a step toward API standardization and backward compatibility. + + +--- + +* [HDFS-630](https://issues.apache.org/jira/browse/HDFS-630) | *Major* | **In DFSOutputStream.nextBlockOutputStream(), the client can exclude specific datanodes when locating the next block.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-625](https://issues.apache.org/jira/browse/HDFS-625) | *Major* | **ListPathsServlet throws NullPointerException** + +Corrected error where listing path no longer in name space could stop ListPathsServlet until system restarted. + + +--- + +* [HDFS-618](https://issues.apache.org/jira/browse/HDFS-618) | *Major* | **Support for non-recursive mkdir in HDFS** + +New DFSClient.mkdir(...) allows option of not creating missing parent(s). + + +--- + +* [HDFS-617](https://issues.apache.org/jira/browse/HDFS-617) | *Major* | **Support for non-recursive create() in HDFS** + +New DFSClient.create(...) allows option of not creating missing parent(s). + + +--- + +* [HDFS-602](https://issues.apache.org/jira/browse/HDFS-602) | *Major* | **Atempt to make a directory under an existing file on DistributedFileSystem should throw an FileAlreadyExistsException instead of FileNotFoundException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-596](https://issues.apache.org/jira/browse/HDFS-596) | *Blocker* | **Memory leak in libhdfs: hdfsFreeFileInfo() in libhdfs does not free memory for mOwner and mGroup** + +Memory leak in function hdfsFreeFileInfo in libhdfs. This bug affects fuse-dfs severely. + + +--- + +* [HDFS-595](https://issues.apache.org/jira/browse/HDFS-595) | *Major* | **FsPermission tests need to be updated for new octal configuration parameter from HADOOP-6234** + +Unit tests updated to match syntax of new configuration parameters. + + +--- + +* [HDFS-578](https://issues.apache.org/jira/browse/HDFS-578) | *Major* | **Support for using server default values for blockSize and replication when creating a file** + +New FileSystem.getServerDefaults() reports the server's default file creation parameters. + + +--- + +* [HDFS-567](https://issues.apache.org/jira/browse/HDFS-567) | *Major* | **Two contrib tools to facilitate searching for block history information** + +New contribution Block Forensics aids investigation of missing blocks. + + +--- + +* [HDFS-538](https://issues.apache.org/jira/browse/HDFS-538) | *Major* | **DistributedFileSystem::listStatus incorrectly returns null for empty result sets** + +FileSystem.listStatus() previously returned null for empty or nonexistent directories; will now return empty array for empty directories and throw FileNotFoundException for non-existent directory. Client code should be updated for new semantics. + + +--- + +* [HDFS-514](https://issues.apache.org/jira/browse/HDFS-514) | *Major* | **DFSClient.namenode is a public field. Should be private.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-512](https://issues.apache.org/jira/browse/HDFS-512) | *Major* | **Set block id as the key to Block** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-503](https://issues.apache.org/jira/browse/HDFS-503) | *Major* | **Implement erasure coding as a layer on HDFS** + +This patch implements an optional layer over HDFS that implements offline erasure-coding. It can be used to reduce the total storage requirements of DFS. + + +--- + +* [HDFS-492](https://issues.apache.org/jira/browse/HDFS-492) | *Major* | **Expose corrupt replica/block information** + +New server web pages provide block information: corrupt\_replicas\_xml and block\_info\_xml. + + +--- + +* [HDFS-457](https://issues.apache.org/jira/browse/HDFS-457) | *Major* | **better handling of volume failure in Data Node storage** + +Datanode can continue if a volume for replica storage fails. Previously a datanode resigned if any volume failed. + + +--- + +* [HDFS-385](https://issues.apache.org/jira/browse/HDFS-385) | *Major* | **Design a pluggable interface to place replicas of blocks in HDFS** + +New experimental API BlockPlacementPolicy allows investigating alternate rules for locating block replicas. + + +--- + +* [HDFS-288](https://issues.apache.org/jira/browse/HDFS-288) | *Major* | **Redundant computation in hashCode() implemenation** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-245](https://issues.apache.org/jira/browse/HDFS-245) | *Major* | **Create symbolic links in HDFS** + +HDFS-245. Adds a symlink implementation to HDFS. This complements the new symlink feature added in HADOOP-6421 + + +--- + +* [HDFS-235](https://issues.apache.org/jira/browse/HDFS-235) | *Major* | **Add support for byte-ranges to hftp** + +HFTP can now serve a specific byte range from a file + + +--- + +* [MAPREDUCE-1747](https://issues.apache.org/jira/browse/MAPREDUCE-1747) | *Blocker* | **Remove documentation for the 'unstable' job-acls feature** + +Removed the documentation for the 'unstable' job-acls feature from branch 0.21. + + +--- + +* [MAPREDUCE-1727](https://issues.apache.org/jira/browse/MAPREDUCE-1727) | *Major* | **TestJobACLs fails after HADOOP-6686** + +Fixed a testcase problem in TestJobACLs. + + +--- + +* [MAPREDUCE-1697](https://issues.apache.org/jira/browse/MAPREDUCE-1697) | *Major* | **Document the behavior of -file option in streaming and deprecate it in favour of generic -files option.** + +Documented the behavior of -file option in streaming and deprecated it in favor of generic -files option. + + +--- + +* [MAPREDUCE-1692](https://issues.apache.org/jira/browse/MAPREDUCE-1692) | *Minor* | **Remove TestStreamedMerge from the streaming tests** + +Removed streaming testcase which tested non-existent functionality in Streaming. + + +--- + +* [MAPREDUCE-1657](https://issues.apache.org/jira/browse/MAPREDUCE-1657) | *Major* | **After task logs directory is deleted, tasklog servlet displays wrong error message about job ACLs** + +Fixed a bug in tasklog servlet which displayed wrong error message about job ACLs - an access control error instead of the expected log files gone error - after task logs directory is deleted. + + +--- + +* [MAPREDUCE-1635](https://issues.apache.org/jira/browse/MAPREDUCE-1635) | *Major* | **ResourceEstimator does not work after MAPREDUCE-842** + +Fixed a bug related to resource estimation for disk-based scheduling by modifying TaskTracker to return correct map output size for the completed maps and -1 for other tasks or failures. + + +--- + +* [MAPREDUCE-1612](https://issues.apache.org/jira/browse/MAPREDUCE-1612) | *Major* | **job conf file is not accessible from job history web page** + +Fixed a bug related to access of job\_conf.xml from the history web page of a job. + + +--- + +* [MAPREDUCE-1611](https://issues.apache.org/jira/browse/MAPREDUCE-1611) | *Blocker* | **Refresh nodes and refresh queues doesnt work with service authorization enabled** + +Fixed a bug that caused all the AdminOperationsProtocol operations to fail when service-level authorization is enabled. The problem is solved by registering AdminOperationsProtocol also with MapReducePolicyProvider. + + +--- + +* [MAPREDUCE-1610](https://issues.apache.org/jira/browse/MAPREDUCE-1610) | *Major* | **Forrest documentation should be updated to reflect the changes in MAPREDUCE-856** + +Updated forrest documentation to reflect the changes to make localized files from DistributedCache have right access-control on TaskTrackers(MAPREDUCE-856). + + +--- + +* [MAPREDUCE-1609](https://issues.apache.org/jira/browse/MAPREDUCE-1609) | *Major* | **TaskTracker.localizeJob should not set permissions on job log directory recursively** + +Fixed TaskTracker so that it does not set permissions on job-log directory recursively. This fix both improves the performance of job localization as well as avoids a bug related to launching of task-cleanup attempts after TaskTracker's restart. + + +--- + +* [MAPREDUCE-1607](https://issues.apache.org/jira/browse/MAPREDUCE-1607) | *Major* | **Task controller may not set permissions for a task cleanup attempt's log directory** + +Fixed initialization of a task-cleanup attempt's log directory by setting correct permissions via task-controller. Added new log4j properties hadoop.tasklog.iscleanup and log4j.appender.TLA.isCleanup to conf/log4j.properties. Changed the userlogs for a task-cleanup attempt to go into its own directory instead of the original attempt directory. This is an incompatible change as old userlogs of cleanup attempt-dirs before this release will no longer be visible. + + +--- + +* [MAPREDUCE-1606](https://issues.apache.org/jira/browse/MAPREDUCE-1606) | *Major* | **TestJobACLs may timeout as there are no slots for launching JOB\_CLEANUP task** + +Fixed TestJobACLs test timeout failure because of no slots for launching JOB\_CLEANUP task. + + +--- + +* [MAPREDUCE-1568](https://issues.apache.org/jira/browse/MAPREDUCE-1568) | *Major* | **TrackerDistributedCacheManager should clean up cache in a background thread** + +MAPREDUCE-1568. TrackerDistributedCacheManager should clean up cache in a background thread. (Scott Chen via zshao) + + +--- + +* [MAPREDUCE-1493](https://issues.apache.org/jira/browse/MAPREDUCE-1493) | *Major* | **Authorization for job-history pages** + +Added web-authorization for job-history pages. This is an incompatible change - it changes the JobHistory format by adding job-acls to job-history files and JobHistory currently does not have the support to read older versions of history files. + + +--- + +* [MAPREDUCE-1482](https://issues.apache.org/jira/browse/MAPREDUCE-1482) | *Major* | **Better handling of task diagnostic information stored in the TaskInProgress** + +Limit the size of diagnostics-string and state-string shipped as part of task status. This will help keep the JobTracker's memory usage under control. Diagnostic string and state string are capped to 1024 chars. + + +--- + +* [MAPREDUCE-1476](https://issues.apache.org/jira/browse/MAPREDUCE-1476) | *Major* | **committer.needsTaskCommit should not be called for a task cleanup attempt** + +Fixed Map/Reduce framework to not call commit task for special tasks like job setup/cleanup and task cleanup. + + +--- + +* [MAPREDUCE-1466](https://issues.apache.org/jira/browse/MAPREDUCE-1466) | *Minor* | **FileInputFormat should save #input-files in JobConf** + +Added a private configuration variable mapreduce.input.num.files, to store number of input files being processed by M/R job. + + +--- + +* [MAPREDUCE-1455](https://issues.apache.org/jira/browse/MAPREDUCE-1455) | *Major* | **Authorization for servlets** + +Adds job-level authorization to servlets(other than history related servlets) for accessing job related info. Deprecates mapreduce.jobtracker.permissions.supergroup and adds the config mapreduce.cluster.permissions.supergroup at cluster level sothat it will be used by TaskTracker also. Authorization checks are done if authentication is succeeded and mapreduce.cluster.job-authorization-enabled is set to true. + + +--- + +* [MAPREDUCE-1454](https://issues.apache.org/jira/browse/MAPREDUCE-1454) | *Major* | **The servlets should quote server generated strings sent in the response** + +Servlets should quote server generated strings sent in the response + + +--- + +* [MAPREDUCE-1435](https://issues.apache.org/jira/browse/MAPREDUCE-1435) | *Major* | **symlinks in cwd of the task are not handled properly after MAPREDUCE-896** + +Fixes bugs in linux task controller and TaskRunner.setupWorkDir() related to handling of symlinks. + + +--- + +* [MAPREDUCE-1430](https://issues.apache.org/jira/browse/MAPREDUCE-1430) | *Major* | **JobTracker should be able to renew delegation tokens for the jobs** + +mapreduce.job.complete.cancel.delegation.tokens - if false - don't cancel delegation token renewal when the job is complete, because it may be used by some other job. + + +--- + +* [MAPREDUCE-1423](https://issues.apache.org/jira/browse/MAPREDUCE-1423) | *Major* | **Improve performance of CombineFileInputFormat when multiple pools are configured** + +MAPREDUCE-1423. Improve performance of CombineFileInputFormat when multiple pools are configured. (Dhruba Borthakur via zshao) + + +--- + +* [MAPREDUCE-1422](https://issues.apache.org/jira/browse/MAPREDUCE-1422) | *Major* | **Changing permissions of files/dirs under job-work-dir may be needed sothat cleaning up of job-dir in all mapred-local-directories succeeds always** + +Introduced enableJobForCleanup() api in TaskController. This api enables deletion of stray files (with no write permissions for task-tracker) from job's work dir. Note that the behavior is similar to TaskController#enableTaskForCleanup() except the path on which the 'chmod' is done is the job's work dir. + + +--- + +* [MAPREDUCE-1420](https://issues.apache.org/jira/browse/MAPREDUCE-1420) | *Major* | **TestTTResourceReporting failing in trunk** + +Fixed a bug in the testcase TestTTResourceReporting. + + +--- + +* [MAPREDUCE-1417](https://issues.apache.org/jira/browse/MAPREDUCE-1417) | *Major* | **Forrest documentation should be updated to reflect the changes in MAPREDUCE-744** + +Updated forrest documentation to reflect the changes w.r.t public and private distributed cache files. + + +--- + +* [MAPREDUCE-1403](https://issues.apache.org/jira/browse/MAPREDUCE-1403) | *Major* | **Save file-sizes of each of the artifacts in DistributedCache in the JobConf** + +Added private configuration variables: mapred.cache.files.filesizes and mapred.cache.archives.filesizes to store sizes of distributed cache artifacts per job. This can be used by tools like Gridmix in simulation runs. + + +--- + +* [MAPREDUCE-1398](https://issues.apache.org/jira/browse/MAPREDUCE-1398) | *Major* | **TaskLauncher remains stuck on tasks waiting for free nodes even if task is killed.** + +Fixed TaskLauncher to stop waiting for blocking slots, for a TIP that is killed / failed while it is in queue. + + +--- + +* [MAPREDUCE-1397](https://issues.apache.org/jira/browse/MAPREDUCE-1397) | *Minor* | **NullPointerException observed during task failures** + +Fixed a race condition involving JvmRunner.kill() and KillTaskAction, which was leading to an NullPointerException causing a transient inconsistent state in JvmManager and failure of tasks. + + +--- + +* [MAPREDUCE-1385](https://issues.apache.org/jira/browse/MAPREDUCE-1385) | *Major* | **Make changes to MapReduce for the new UserGroupInformation APIs (HADOOP-6299)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-1383](https://issues.apache.org/jira/browse/MAPREDUCE-1383) | *Major* | **Allow storage and caching of delegation token.** + +mapreduce.job.hdfs-servers - declares hdfs servers to be used by the job, so client can pre-fetch delegation tokens for thsese servers (comma separated list of NameNodes). + + +--- + +* [MAPREDUCE-1342](https://issues.apache.org/jira/browse/MAPREDUCE-1342) | *Major* | **Potential JT deadlock in faulty TT tracking** + +Fix for a potential deadlock in the global blacklist of tasktrackers feature. + + +--- + +* [MAPREDUCE-1338](https://issues.apache.org/jira/browse/MAPREDUCE-1338) | *Major* | **need security keys storage solution** + +new command line argument: + tokensFile - path to the file with clients secret keys in JSON format + + +--- + +* [MAPREDUCE-1316](https://issues.apache.org/jira/browse/MAPREDUCE-1316) | *Blocker* | **JobTracker holds stale references to retired jobs via unreported tasks** + +JobTracker holds stale references to TaskInProgress objects and hence indirectly holds reference to retired jobs resulting into memory leak. Only task-attempts which are yet to report their status are left behind in the memory. All the task-attempts are now removed from the JobTracker by iterating over the scheduled task-attempt ids instead of iterating over available task statuses. + + +--- + +* [MAPREDUCE-1307](https://issues.apache.org/jira/browse/MAPREDUCE-1307) | *Major* | **Introduce the concept of Job Permissions** + +Added job-level authorization to MapReduce. JobTracker will now use the cluster configuration "mapreduce.cluster.job-authorization-enabled" to enable the checks to verify the authority of access of jobs where ever needed. Introduced two job-configuration properties to specify ACLs: "mapreduce.job.acl-view-job" and "mapreduce.job.acl-modify-job". For now, RPCs related to job-level counters, task-level counters and tasks' diagnostic information are protected by "mapreduce.job.acl-view-job" ACL. "mapreduce.job.acl-modify-job" protects killing of a job, killing a task of a job, failing a task of a job and setting the priority of a job. Irrespective of the above two ACLs, job-owner, superuser and members of supergroup configured on JobTracker via mapred.permissions.supergroup, can do all the view and modification operations. + + +--- + +* [MAPREDUCE-1306](https://issues.apache.org/jira/browse/MAPREDUCE-1306) | *Major* | **[MUMAK] Randomize the arrival of heartbeat responses** + +[MUMAK] Randomize the arrival of heartbeat responses + + +--- + +* [MAPREDUCE-1287](https://issues.apache.org/jira/browse/MAPREDUCE-1287) | *Minor* | **Avoid calling Partitioner with only 1 reducer** + +For jobs with only one reducer, the Partitioner will no longer be called. Applications depending on Partitioners modifying records for single reducer jobs will need to move this functionality elsewhere. + + +--- + +* [MAPREDUCE-1284](https://issues.apache.org/jira/browse/MAPREDUCE-1284) | *Major* | **TestLocalizationWithLinuxTaskController fails** + +Fixes a bug in linux task controller by making the paths array passed to fts\_open() as null-terminated as per the man page. + + +--- + +* [MAPREDUCE-1230](https://issues.apache.org/jira/browse/MAPREDUCE-1230) | *Major* | **Vertica streaming adapter doesn't handle nulls in all cases** + +Fixes null handling in records returned from VerticaInputFormat + + +--- + +* [MAPREDUCE-1218](https://issues.apache.org/jira/browse/MAPREDUCE-1218) | *Major* | **Collecting cpu and memory usage for TaskTrackers** + +This patch allows TaskTracker reports it's current available memory and CPU usage to JobTracker through heartbeat. The information can be used for scheduling and monitoring in the JobTracker. This patch changes the version of InterTrackerProtocal. + + +--- + +* [MAPREDUCE-1213](https://issues.apache.org/jira/browse/MAPREDUCE-1213) | *Major* | **TaskTrackers restart is very slow because it deletes distributed cache directory synchronously** + +Directories specified in mapred.local.dir that can not be created now cause the TaskTracker to fail to start. + + +--- + +* [MAPREDUCE-1185](https://issues.apache.org/jira/browse/MAPREDUCE-1185) | *Major* | **URL to JT webconsole for running job and job history should be the same** + +If the running job is retired, then Job url is redirected to the history page. To construct the history url, JobTracker maintains the mapping of job id to history file names. The entries from mapping is purged for jobs older than mapreduce.jobtracker.jobhistory.maxage configured value. + + +--- + +* [MAPREDUCE-1171](https://issues.apache.org/jira/browse/MAPREDUCE-1171) | *Blocker* | **Lots of fetch failures** + +Added two expert level configuration properties. +1. "mapreduce.reduce.shuffle.notify.readerror" to know whether to send notification to JobTracker after every read error or not. If the configuration is false, read errors are treated similar to connection errors. +2. "mapreduce.reduce.shuffle.maxfetchfailures" to specify the maximum number of the fetch failures after which the failure will be notified to JobTracker. + + +--- + +* [MAPREDUCE-1160](https://issues.apache.org/jira/browse/MAPREDUCE-1160) | *Major* | **Two log statements at INFO level fill up jobtracker logs** + +Changed some log statements that were filling up jobtracker logs to debug level. + + +--- + +* [MAPREDUCE-1158](https://issues.apache.org/jira/browse/MAPREDUCE-1158) | *Major* | **running\_maps is not decremented when the tasks of a job is killed/failed** + +Fix Jobtracker running maps/reduces metrics. + + +--- + +* [MAPREDUCE-1153](https://issues.apache.org/jira/browse/MAPREDUCE-1153) | *Major* | **Metrics counting tasktrackers and blacklisted tasktrackers are not updated when trackers are decommissioned.** + +Update the number of trackers and blacklisted trackers metrics when trackers are decommissioned. + + +--- + +* [MAPREDUCE-1143](https://issues.apache.org/jira/browse/MAPREDUCE-1143) | *Blocker* | **runningMapTasks counter is not properly decremented in case of failed Tasks.** + +Corrects the behaviour of tasks counters in case of failed tasks.Incorrect counter values can lead to bad scheduling decisions .This jira rectifies the problem by making sure decrement properly happens incase of failed tasks. + + +--- + +* [MAPREDUCE-1140](https://issues.apache.org/jira/browse/MAPREDUCE-1140) | *Major* | **Per cache-file refcount can become negative when tasks release distributed-cache files** + +Fixed a bug in DistributedCache, to not decrement reference counts for unreferenced files in error conditions. + + +--- + +* [MAPREDUCE-1124](https://issues.apache.org/jira/browse/MAPREDUCE-1124) | *Major* | **TestGridmixSubmission fails sometimes** + +Fixed errors that caused occasional failure of TestGridmixSubmission, and additionally refactored some Gridmix code. + + +--- + +* [MAPREDUCE-1105](https://issues.apache.org/jira/browse/MAPREDUCE-1105) | *Blocker* | **CapacityScheduler: It should be possible to set queue hard-limit beyond it's actual capacity** + +Replaced the existing max task limits variables "mapred.capacity-scheduler.queue.\.max.map.slots" and "mapred.capacity-scheduler.queue.\.max.reduce.slots" with "mapred.capacity-scheduler.queue.\.maximum-capacity" . + +max task limit variables were used to throttle the queue, i.e, these were the hard limit and not allowing queue to grow further. +maximum-capacity variable defines a limit beyond which a queue cannot use the capacity of the cluster. This provides a means to limit how much excess capacity a queue can use. + +maximum-capacity variable behavior is different from max task limit variables, as maximum-capacity is a percentage and it grows and shrinks in absolute terms based on total cluster capacity.Also same maximum-capacity percentage is applied to both map and reduce. + + +--- + +* [MAPREDUCE-1103](https://issues.apache.org/jira/browse/MAPREDUCE-1103) | *Major* | **Additional JobTracker metrics** + +Add following additional job tracker metrics: +Reserved{Map, Reduce}Slots +Occupied{Map, Reduce}Slots +Running{Map, Reduce}Tasks +Killed{Map, Reduce}Tasks + +FailedJobs +KilledJobs +PrepJobs +RunningJobs + +TotalTrackers +BlacklistedTrackers +DecommissionedTrackers + + +--- + +* [MAPREDUCE-1098](https://issues.apache.org/jira/browse/MAPREDUCE-1098) | *Major* | **Incorrect synchronization in DistributedCache causes TaskTrackers to freeze up during localization of Cache for tasks.** + +Fixed the distributed cache's localizeCache to lock only the uri it is localizing. + + +--- + +* [MAPREDUCE-1097](https://issues.apache.org/jira/browse/MAPREDUCE-1097) | *Minor* | **Changes/fixes to support Vertica 3.5** + +Adds support for Vertica 3.5 truncate table, deploy\_design and numeric types. + + +--- + +* [MAPREDUCE-1090](https://issues.apache.org/jira/browse/MAPREDUCE-1090) | *Major* | **Modify log statement in Tasktracker log related to memory monitoring to include attempt id.** + +Modified log statement in task memory monitoring thread to include task attempt id. + + +--- + +* [MAPREDUCE-1086](https://issues.apache.org/jira/browse/MAPREDUCE-1086) | *Major* | **hadoop commands in streaming tasks are trying to write to tasktracker's log** + +This patch makes TT to set HADOOP\_ROOT\_LOGGER to INFO,TLA by default in the environment of taskjvm and its children. + + +--- + +* [MAPREDUCE-1063](https://issues.apache.org/jira/browse/MAPREDUCE-1063) | *Minor* | **Document Gridmix benchmark** + +Created new Forrest documentation for Gridmix. + + +--- + +* [MAPREDUCE-1062](https://issues.apache.org/jira/browse/MAPREDUCE-1062) | *Major* | **MRReliability test does not work with retired jobs** + +Ensure that MRReliability works with retired-jobs feature turned on. + + +--- + +* [MAPREDUCE-1048](https://issues.apache.org/jira/browse/MAPREDUCE-1048) | *Major* | **Show total slot usage in cluster summary on jobtracker webui** + +Added occupied map/reduce slots and reserved map/reduce slots to the "Cluster Summary" table on jobtracker web ui. + + +--- + +* [MAPREDUCE-1030](https://issues.apache.org/jira/browse/MAPREDUCE-1030) | *Blocker* | **Reduce tasks are getting starved in capacity scheduler** + +Modified the scheduling logic in capacity scheduler to return a map and a reduce task per heartbeat. + + +--- + +* [MAPREDUCE-1028](https://issues.apache.org/jira/browse/MAPREDUCE-1028) | *Blocker* | **Cleanup tasks are scheduled using high memory configuration, leaving tasks in unassigned state.** + +Makes taskCleanup tasks to use 1 slot even for high memory jobs. + + +--- + +* [MAPREDUCE-1018](https://issues.apache.org/jira/browse/MAPREDUCE-1018) | *Blocker* | **Document changes to the memory management and scheduling model** + +Updated documentation related to (1) the changes in the configuration for memory management of tasks and (2) the modifications in scheduling model of CapacityTaskScheduler to include memory requirement by tasks. + + +--- + +* [MAPREDUCE-1003](https://issues.apache.org/jira/browse/MAPREDUCE-1003) | *Major* | **trunk build fails when -Declipse.home is set** + +Minor changes to HadoopJob.java in eclipse-plugin contrib project to accommodate changes in JobStatus (MAPREDUCE-777) + + +--- + +* [MAPREDUCE-975](https://issues.apache.org/jira/browse/MAPREDUCE-975) | *Major* | **Add an API in job client to get the history file url for a given job id** + +Adds an API Cluster#getJobHistoryUrl(JobID jobId) to get the history url for a given job id. The API does not check for the validity of job id or existence of the history file. It just constructs the url based on history folder, job id and the current user. + + +--- + +* [MAPREDUCE-967](https://issues.apache.org/jira/browse/MAPREDUCE-967) | *Major* | **TaskTracker does not need to fully unjar job jars** + +For efficiency, TaskTrackers no longer unjar the job jar into the job cache directory. Users who previously depended on this functionality for shipping non-code dependencies can use the undocumented configuration parameter "mapreduce.job.jar.unpack.pattern" to cause specific jar contents to be unpacked. + + +--- + +* [MAPREDUCE-964](https://issues.apache.org/jira/browse/MAPREDUCE-964) | *Critical* | **Inaccurate values in jobSummary logs** + +Updates to task timing information were fixed in some code paths that led to inconsistent metering of task run times. Also, checks were added to guard against inconsistencies and record information about state if they should happen to occur. + + +--- + +* [MAPREDUCE-963](https://issues.apache.org/jira/browse/MAPREDUCE-963) | *Major* | **mapred's FileAlreadyExistsException should be deprecated in favor of hadoop-common's one.** + +Deprecate o.a.h.mapred.FileAlreadyExistsException and replace it with o.a.h.fs.FileAlreadyExistsException. + + +--- + +* [MAPREDUCE-962](https://issues.apache.org/jira/browse/MAPREDUCE-962) | *Major* | **NPE in ProcfsBasedProcessTree.destroy()** + +Fixes an issue of NPE in ProcfsBasedProcessTree in a corner case. + + +--- + +* [MAPREDUCE-954](https://issues.apache.org/jira/browse/MAPREDUCE-954) | *Major* | **The new interface's Context objects should be interfaces** + +Changed Map-Reduce context objects to be interfaces. + + +--- + +* [MAPREDUCE-947](https://issues.apache.org/jira/browse/MAPREDUCE-947) | *Major* | **OutputCommitter should have an abortJob method** + +Introduced abortJob() method in OutputCommitter which will be invoked when the job fails or is killed. By default it invokes OutputCommitter.cleanupJob(). Deprecated OutputCommitter.cleanupJob() and introduced OutputCommitter.commitJob() method which will be invoked for successful jobs. Also a \_SUCCESS file is created in the output folder for successful jobs. A configuration parameter mapreduce.fileoutputcommitter.marksuccessfuljobs can be set to false to disable creation of \_SUCCESS file, or to true to enable creation of the \_SUCCESS file. + + +--- + +* [MAPREDUCE-943](https://issues.apache.org/jira/browse/MAPREDUCE-943) | *Major* | **TestNodeRefresh timesout occasionally** + +TestNodeRefresh timed out as the code to do with node refresh got removed. This patch removes the testcase. + + +--- + +* [MAPREDUCE-913](https://issues.apache.org/jira/browse/MAPREDUCE-913) | *Blocker* | **TaskRunner crashes with NPE resulting in held up slots, UNINITIALIZED tasks and hung TaskTracker** + +Fixed TaskTracker to avoid hung and unusable slots when TaskRunner crashes with NPE and leaves tasks in UNINITIALIZED state for ever. + + +--- + +* [MAPREDUCE-899](https://issues.apache.org/jira/browse/MAPREDUCE-899) | *Major* | **When using LinuxTaskController, localized files may become accessible to unintended users if permissions are misconfigured.** + +Added configuration "mapreduce.tasktracker.group", a group name to which TaskTracker belongs. When LinuxTaskController is used, task-controller binary's group owner should be this group. The same should be specified in task-controller.cfg also. + + +--- + +* [MAPREDUCE-895](https://issues.apache.org/jira/browse/MAPREDUCE-895) | *Major* | **FileSystem::ListStatus will now throw FileNotFoundException, MapRed needs updated** + +The semantics for dealing with non-existent paths passed to FileSystem::listStatus() were updated and solidified in HADOOP-6201 and HDFS-538. Existing code within MapReduce that relied on the previous behavior of some FileSystem implementations of returning null has been updated to catch or propagate a FileNotFoundException, per the method's contract. + + +--- + +* [MAPREDUCE-893](https://issues.apache.org/jira/browse/MAPREDUCE-893) | *Major* | **Provide an ability to refresh queue configuration without restart.** + +Extended the framework's refresh-queue mechanism to support refresh of scheduler specific queue properties and implemented this refresh operation for some of the capacity scheduler properties. With this feature, one can refresh some of the capacity-scheduler's queue related properties - queue capacities, user-limits per queue, max map/reduce capacity and max-jobs per user to initialize while the system is running and without restarting JT. Even after this, some features like changing enable/disable priorities, adding/removing queues are not supported in capacity-scheduler. + + +--- + +* [MAPREDUCE-890](https://issues.apache.org/jira/browse/MAPREDUCE-890) | *Blocker* | **After HADOOP-4491, the user who started mapred system is not able to run job.** + +Fixed a bug that failed jobs that are run by the same user who started the mapreduce system(cluster). + + +--- + +* [MAPREDUCE-873](https://issues.apache.org/jira/browse/MAPREDUCE-873) | *Major* | **Simplify Job Recovery** + +Simplifies job recovery. On jobtracker restart, incomplete jobs are resubmitted and all tasks reexecute. +This JIRA removes a public constructor in JobInProgress. + + +--- + +* [MAPREDUCE-871](https://issues.apache.org/jira/browse/MAPREDUCE-871) | *Major* | **Job/Task local files have incorrect group ownership set by LinuxTaskController binary** + +Fixed LinuxTaskController binary so that permissions of local files on TT are set correctly: user owned by the job-owner and group-owned by the group owner of the binary and \_not\_ the primary group of the TaskTracker. + + +--- + +* [MAPREDUCE-870](https://issues.apache.org/jira/browse/MAPREDUCE-870) | *Major* | **Clean up the job Retire code** + +Removed the Job Retire thread and the associated configuration parameters. Job is purged from memory as soon as the history file is copied to HDFS. Only JobStatus object is retained in the retired jobs cache. + + +--- + +* [MAPREDUCE-862](https://issues.apache.org/jira/browse/MAPREDUCE-862) | *Major* | **Modify UI to support a hierarchy of queues** + +- The command line of hadoop queue -list and -info was changed to support hierarchical queues. So, they would now print information about child queues, wherever relevant. +- The Web UI of the JobTracker was changed to list queues and queue information in a separate page. + + +--- + +* [MAPREDUCE-861](https://issues.apache.org/jira/browse/MAPREDUCE-861) | *Major* | **Modify queue configuration format and parsing to support a hierarchy of queues.** + +Added support for hierarchical queues in the Map/Reduce framework with the following changes: +- mapred-queues.xml is modified to a new XML template as mentioned in the JIRA. +- Modified JobQueueInfo to contain a handle to child queues. +- Added new APIs in the client to get 'root' queues, so that the entire hierarchy of queues can be iterated. +-Added new APIs to get the child queues for a given queue . + + +--- + +* [MAPREDUCE-856](https://issues.apache.org/jira/browse/MAPREDUCE-856) | *Major* | **Localized files from DistributedCache should have right access-control** + +Fixed TaskTracker and related classes so as to set correct and most restrictive access control for DistributedCache files/archives. + - To do this, it changed the directory structure of per-job local files on a TaskTracker to the following: +$mapred.local.dir + `-- taskTracker + `-- $user + |- distcache + `-- jobcache + - Distributed cache files/archives are now user-owned by the job-owner and the group-owned by the special group-owner of the task-controller binary. The files/archives are set most private permissions possible, and as soon as possible, immediately after the files/dirs are first localized on the TT. + - As depicted by the new directory structure, a directory corresponding to each user is created on each TT when that particular user's first task are assigned to the corresponding TT. These user directories remain on the TT forever are not cleaned when unused, which is targeted to be fixed via MAPREDUCE-1019. + - The distributed cache files are now accessible \_only\_ by the user who first localized them. Sharing of these files across users is no longer possible, but is targeted for future versions via MAPREDUCE-744. + + +--- + +* [MAPREDUCE-852](https://issues.apache.org/jira/browse/MAPREDUCE-852) | *Major* | **ExampleDriver is incorrectly set as a Main-Class in tools in build.xml** + +Changed the target name from "tools-jar" to "tools" in build.xml. + + +--- + +* [MAPREDUCE-849](https://issues.apache.org/jira/browse/MAPREDUCE-849) | *Major* | **Renaming of configuration property names in mapreduce** + +Rename and categorize configuration keys into - cluster, jobtracker, tasktracker, job, client. Constants are defined for all keys in java and code is changed to use constants instead of direct strings. All old keys are deprecated except of examples and tests. The change is incompatible because support for old keys is not provided for config keys in examples. + + +--- + +* [MAPREDUCE-848](https://issues.apache.org/jira/browse/MAPREDUCE-848) | *Major* | **TestCapacityScheduler is failing** + +MAPREDUCE-805 changed the way the job was initialized. Capacity schedulers testcases were not modified as part of MAPREDUCE-805. This patch fixes this bug. + + +--- + +* [MAPREDUCE-845](https://issues.apache.org/jira/browse/MAPREDUCE-845) | *Minor* | **build.xml hard codes findbugs heap size, in some configurations 512M is insufficient to successfully build** + +Changes the heapsize for findbugs to a parameter which can be changed on the build command line. + + +--- + +* [MAPREDUCE-842](https://issues.apache.org/jira/browse/MAPREDUCE-842) | *Major* | **Per-job local data on the TaskTracker node should have right access-control** + +Modified TaskTracker and related classes so that per-job local data on the TaskTracker node has right access-control. Important changes: + - All files/directories of the job on the TaskTracker are now user-owned by the job-owner and group-owner by a special TaskTracker's group. + - The permissions of the file/directories are set to the most restrictive permissions possible. + - Files/dirs shareable by all tasks of the job on this TT are set proper access control as soon as possible, i.e immediately after job-localization and those that are private to a single task are set access control after the corresponding task's localization. + - Also fixes MAPREDUCE-131 which is related to a bug because of which tasks hang when the taskcontroller.cfg has multiple entries for mapred.local.dir + - A new configuration entry hadoop.log.dir corresponding to the hadoop.log.dir in TT's configuration is now needed in task-controller.cfg so as to support restricted access control for userlogs of the tasks on the TaskTracker. + + +--- + +* [MAPREDUCE-830](https://issues.apache.org/jira/browse/MAPREDUCE-830) | *Major* | **Providing BZip2 splitting support for Text data** + +Splitting support for BZip2 Text data + + +--- + +* [MAPREDUCE-824](https://issues.apache.org/jira/browse/MAPREDUCE-824) | *Major* | **Support a hierarchy of queues in the capacity scheduler** + +Support hierarchical queues in the CapacityScheduler to allow for more predictable sharing of cluster resources. + + +--- + +* [MAPREDUCE-817](https://issues.apache.org/jira/browse/MAPREDUCE-817) | *Major* | **Add a cache for retired jobs with minimal job info and provide a way to access history file url** + +Provides a way to configure the cache of JobStatus objects for the retired jobs. +Adds an API in RunningJob to access history file url. +Adds a LRU based cache for job history files loaded in memory when accessed via JobTracker web UI. +Adds Retired Jobs table on the Jobtracker UI. The job move from Running to Completed/Failed table. Then job move to Retired table when it is purged from memory. The Retired table shows last 100 retired jobs. The Completed/Failed jobs table are only shown if there are non-zero jobs in the table. + + +--- + +* [MAPREDUCE-814](https://issues.apache.org/jira/browse/MAPREDUCE-814) | *Major* | **Move completed Job history files to HDFS** + +Provides an ability to move completed job history files to a HDFS location via configuring "mapred.job.tracker.history.completed.location". If the directory location does not already exist, it would be created by jobtracker. + + +--- + +* [MAPREDUCE-809](https://issues.apache.org/jira/browse/MAPREDUCE-809) | *Major* | **Job summary logs show status of completed jobs as RUNNING** + +Fix job-summary logs to correctly record final status of FAILED and KILLED jobs. + + +--- + +* [MAPREDUCE-800](https://issues.apache.org/jira/browse/MAPREDUCE-800) | *Major* | **MRUnit should support the new API** + +Support new API in unit tests developed with MRUnit. + + +--- + +* [MAPREDUCE-798](https://issues.apache.org/jira/browse/MAPREDUCE-798) | *Major* | **MRUnit should be able to test a succession of MapReduce passes** + +Add PipelineMapReduceDriver to MRUnit to support testing a pipeline of MapReduce passes + + +--- + +* [MAPREDUCE-797](https://issues.apache.org/jira/browse/MAPREDUCE-797) | *Major* | **MRUnit MapReduceDriver should support combiners** + +Add Combiner support to MapReduceDriver in MRUnit + + +--- + +* [MAPREDUCE-793](https://issues.apache.org/jira/browse/MAPREDUCE-793) | *Major* | **Create a new test that consolidates a few tests to be included in the commit-test list** + +Creates a new test to test several miscellaneous functionality at one shot instead of running a job for each, to be used as a fast test for the ant commit-tests target. + + +--- + +* [MAPREDUCE-788](https://issues.apache.org/jira/browse/MAPREDUCE-788) | *Major* | **Modify gridmix2 to use new api.** + +Modifies Gridmix2 to use the new Map/Reduce API + + +--- + +* [MAPREDUCE-787](https://issues.apache.org/jira/browse/MAPREDUCE-787) | *Major* | **-files, -archives should honor user given symlink path** + +Fix JobSubmitter to honor user given symlink in the path. + + +--- + +* [MAPREDUCE-785](https://issues.apache.org/jira/browse/MAPREDUCE-785) | *Major* | **Refactor TestReduceFetchFromPartialMem into a separate test** + +Moves TestReduceFetchFromPartialMem out of TestReduceFetch into a separate test to enable it to be included in the commit-tests target. + + +--- + +* [MAPREDUCE-784](https://issues.apache.org/jira/browse/MAPREDUCE-784) | *Major* | **Modify TestUserDefinedCounters to use LocalJobRunner instead of MiniMR** + +Modifies TestUserDefinedCounters to use LocalJobRunner instead of using MiniMR cluster + + +--- + +* [MAPREDUCE-777](https://issues.apache.org/jira/browse/MAPREDUCE-777) | *Major* | **A method for finding and tracking jobs from the new API** + +Enhance the Context Objects api to add features to find and track jobs. + + +--- + +* [MAPREDUCE-775](https://issues.apache.org/jira/browse/MAPREDUCE-775) | *Major* | **Add input/output formatters for Vertica clustered ADBMS.** + +Add native and streaming support for Vertica as an input or output format taking advantage of parallel read and write properties of the DBMS. + + +--- + +* [MAPREDUCE-773](https://issues.apache.org/jira/browse/MAPREDUCE-773) | *Major* | **LineRecordReader can report non-zero progress while it is processing a compressed stream** + +Modifies LineRecordReader to report an approximate progress, instead of just returning 0, when using compressed streams. + + +--- + +* [MAPREDUCE-772](https://issues.apache.org/jira/browse/MAPREDUCE-772) | *Major* | **Chaging LineRecordReader algo so that it does not need to skip backwards in the stream** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-768](https://issues.apache.org/jira/browse/MAPREDUCE-768) | *Major* | **Configuration information should generate dump in a standard format.** + +Provides an ability to dump jobtracker configuration in JSON format to standard output and exits. +To dump, use hadoop jobtracker -dumpConfiguration +The format of the dump is {"properties":[{"key":\,"value":\,"isFinal":\,"resource" : \}] } + + +--- + +* [MAPREDUCE-766](https://issues.apache.org/jira/browse/MAPREDUCE-766) | *Major* | **Enhance -list-blacklisted-trackers to display host name, blacklisted reason and blacklist report.** + +Enhanced -list-blacklisted-trackers to include the reason for blacklisting a node. Modified JobSubmissionProtocol's version as the ClusterStatus is changed to have a new class. The format of the -list-blacklisted-trackers command line interface has also changed to show the reason. + + +--- + +* [MAPREDUCE-760](https://issues.apache.org/jira/browse/MAPREDUCE-760) | *Major* | **TestNodeRefresh might not work as expected** + +TestNodeRefresh waits for the newly added tracker to join before starting the testing. + + +--- + +* [MAPREDUCE-754](https://issues.apache.org/jira/browse/MAPREDUCE-754) | *Minor* | **NPE in expiry thread when a TT is lost** + +Fixes the NPE in 'refreshNodes', ExpiryTracker thread and heartbeat. NPE occurred in the following cases +- a blacklisted tracker is either decommissioned or expires. +- a lost tracker gets blacklisted + + +--- + +* [MAPREDUCE-744](https://issues.apache.org/jira/browse/MAPREDUCE-744) | *Major* | **Support in DistributedCache to share cache files with other users after HADOOP-4493** + +Fixed DistributedCache to support sharing of the local cache files with other users on the same TaskTracker. The cache files are checked at the client side for public/private access on the file system, and that information is passed in the configuration. The TaskTrackers look at the configuration for each file during task localization, and, if the file was public on the filesystem, they are localized to a common space for sharing by all users' tasks on the TaskTracker. Else the file is localized to the user's private directory on the local filesystem. + + +--- + +* [MAPREDUCE-740](https://issues.apache.org/jira/browse/MAPREDUCE-740) | *Major* | **Provide summary information per job once a job is finished.** + +Log a job-summary at the end of a job, while allowing it to be configured to use a custom appender if desired. + + +--- + +* [MAPREDUCE-739](https://issues.apache.org/jira/browse/MAPREDUCE-739) | *Major* | **Allow relative paths to be created inside archives.** + +Allow creating archives with relative paths with a -p option on the command line. + + +--- + +* [MAPREDUCE-732](https://issues.apache.org/jira/browse/MAPREDUCE-732) | *Minor* | **node health check script should not log "UNHEALTHY" status for every heartbeat in INFO mode** + +Changed log level of addition of blacklisted reason in the JobTracker log to debug instead of INFO + + +--- + +* [MAPREDUCE-717](https://issues.apache.org/jira/browse/MAPREDUCE-717) | *Major* | **Fix some corner case issues in speculative execution (post hadoop-2141)** + +Fixes some edge cases while using speculative execution + + +--- + +* [MAPREDUCE-711](https://issues.apache.org/jira/browse/MAPREDUCE-711) | *Major* | **Move Distributed Cache from Common to Map/Reduce** + +- Removed distributed cache classes and package from the Common project. +- Added the same to the mapreduce project. +- This will mean that users using Distributed Cache will now necessarily need the mapreduce jar in Hadoop 0.21. +- Modified the package name to o.a.h.mapreduce.filecache from o.a.h.filecache and deprecated the old package name. + + +--- + +* [MAPREDUCE-707](https://issues.apache.org/jira/browse/MAPREDUCE-707) | *Trivial* | **Provide a jobconf property for explicitly assigning a job to a pool** + +add mapred.fairscheduler.pool property to define which pool a job belongs to. + + +--- + +* [MAPREDUCE-706](https://issues.apache.org/jira/browse/MAPREDUCE-706) | *Major* | **Support for FIFO pools in the fair scheduler** + +Support for FIFO pools added to the Fair Scheduler. + + +--- + +* [MAPREDUCE-701](https://issues.apache.org/jira/browse/MAPREDUCE-701) | *Minor* | **Make TestRackAwareTaskPlacement a unit test** + +Modifies TestRackAwareTaskPlacement to not use MiniMR/DFS Cluster for testing, thereby making it a unit test + + +--- + +* [MAPREDUCE-698](https://issues.apache.org/jira/browse/MAPREDUCE-698) | *Major* | **Per-pool task limits for the fair scheduler** + +Per-pool map and reduce caps for Fair Scheduler. + + +--- + +* [MAPREDUCE-686](https://issues.apache.org/jira/browse/MAPREDUCE-686) | *Major* | **Move TestSpeculativeExecution.Fake\* into a separate class so that it can be used by other tests also** + +Consolidate the Mock Objects used for testing in a separate class(FakeObjectUtiltities) to ease re-usability + + +--- + +* [MAPREDUCE-683](https://issues.apache.org/jira/browse/MAPREDUCE-683) | *Major* | **TestJobTrackerRestart fails with Map task completion events ordering mismatch** + +TestJobTrackerRestart failed because of stale filemanager cache (which was created once per jvm). This patch makes sure that the filemanager is inited upon every JobHistory.init() and hence upon every restart. Note that this wont happen in production as upon a restart the new jobtracker will start in a new jvm and hence a new cache will be created. + + +--- + +* [MAPREDUCE-682](https://issues.apache.org/jira/browse/MAPREDUCE-682) | *Major* | **Reserved tasktrackers should be removed when a node is globally blacklisted** + +Jobtracker was modified to cleanup reservations created on tasktracker nodes to support high RAM jobs, when the nodes are blacklisted. + + +--- + +* [MAPREDUCE-679](https://issues.apache.org/jira/browse/MAPREDUCE-679) | *Major* | **XML-based metrics as JSP servlet for JobTracker** + +Added XML-based JobTracker status JSP page for metrics reporting + + +--- + +* [MAPREDUCE-677](https://issues.apache.org/jira/browse/MAPREDUCE-677) | *Major* | **TestNodeRefresh timesout** + +TestNodeRefresh sometimes timed out. This happened because the test started a MR cluster with 2 trackers and ran a half-waiting-mapper job. Tasks that have id \> total-maps/2 wait for a signal. Because of 2 trackers, the tasks got scheduled out of order (locality) and hence the job got stuck. The fix is to start only one tracker and then add a new tracker later. + + +--- + +* [MAPREDUCE-676](https://issues.apache.org/jira/browse/MAPREDUCE-676) | *Major* | **Existing diagnostic rules fail for MAP ONLY jobs** + +hadoop vaidya counter names LOCAL\_BYTES\_READ and LOCAL\_BYTES\_WRITTEN are changed to respectively FILE\_BYTES\_READ, FILE\_BYTES\_WRITTEN as per current hadoop counter names. + + +--- + +* [MAPREDUCE-670](https://issues.apache.org/jira/browse/MAPREDUCE-670) | *Major* | ** Create target for 10 minute patch test build for mapreduce** + +Added a new target 'test-commit' to the build.xml file which runs tests specified in the file src/test/commit-tests. The tests specified in src/test/commit-tests should provide maximum coverage and all the tests should run within 10mins. + + +--- + +* [MAPREDUCE-656](https://issues.apache.org/jira/browse/MAPREDUCE-656) | *Major* | **Change org.apache.hadoop.mapred.SequenceFile\* classes to use new api** + +Ports the SequenceFile* classes to the new Map/Reduce API + + +--- + +* [MAPREDUCE-655](https://issues.apache.org/jira/browse/MAPREDUCE-655) | *Major* | **Change KeyValueLineRecordReader and KeyValueTextInputFormat to use new api.** + +Ports KeyValueLineRecordReader and KeyValueTextInputFormat the new Map/Reduce API + + +--- + +* [MAPREDUCE-646](https://issues.apache.org/jira/browse/MAPREDUCE-646) | *Major* | **distcp should place the file distcp\_src\_files in distributed cache** + +Patch increases the replication factor of \_distcp\_src\_files to sqrt(min(maxMapsOnCluster, totalMapsInThisJob)) sothat many maps won't access the same replica of the file \_distcp\_src\_files at the same time. + + +--- + +* [MAPREDUCE-642](https://issues.apache.org/jira/browse/MAPREDUCE-642) | *Major* | **distcp could have an option to preserve the full source path** + +DistCp now has a "-basedir" option that allows you to set the sufix of the source path that will be copied to the destination. + + +--- + +* [MAPREDUCE-632](https://issues.apache.org/jira/browse/MAPREDUCE-632) | *Major* | **Merge TestCustomOutputCommitter with TestCommandLineJobSubmission** + +Modifies TestCommandLineJobSubmission to add a test for testing custom output committer and removes TestCustomOutputCommitter + + +--- + +* [MAPREDUCE-630](https://issues.apache.org/jira/browse/MAPREDUCE-630) | *Minor* | **TestKillCompletedJob can be modified to improve execution times** + +Modifies TestKillCompletedJob to rid of its dependence on MiniMR clusters and makes it a unit test + + +--- + +* [MAPREDUCE-627](https://issues.apache.org/jira/browse/MAPREDUCE-627) | *Minor* | **Modify TestTrackerBlacklistAcrossJobs to improve execution time** + +Modifies TestTrackerBlacklistAcrossJobs to use mock objects for testing instead of running a full-fledged job using MiniMR clusters. + + +--- + +* [MAPREDUCE-626](https://issues.apache.org/jira/browse/MAPREDUCE-626) | *Minor* | **Modify TestLostTracker to improve execution time** + +Modifies TestLostTracker to use Mock objects instead of running full-fledged jobs using the MiniMR clusters. + + +--- + +* [MAPREDUCE-625](https://issues.apache.org/jira/browse/MAPREDUCE-625) | *Minor* | **Modify TestTaskLimits to improve execution time** + +Modifies TestTaskLimits to do unit testing instead of running jobs using MR clusters + + +--- + +* [MAPREDUCE-551](https://issues.apache.org/jira/browse/MAPREDUCE-551) | *Major* | **Add preemption to the fair scheduler** + +Added support for preemption in the fair scheduler. The new configuration options for enabling this are described in the fair scheduler documentation. + + +--- + +* [MAPREDUCE-532](https://issues.apache.org/jira/browse/MAPREDUCE-532) | *Major* | **Allow admins of the Capacity Scheduler to set a hard-limit on the capacity of a queue** + +Provided ability in the capacity scheduler to limit the number of slots that can be concurrently used per queue at any given time. + + +--- + +* [MAPREDUCE-516](https://issues.apache.org/jira/browse/MAPREDUCE-516) | *Major* | **Fix the 'cluster drain' problem in the Capacity Scheduler wrt High RAM Jobs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-502](https://issues.apache.org/jira/browse/MAPREDUCE-502) | *Major* | **Allow jobtracker to be configured with zero completed jobs in memory** + +If the number of jobs per user exceeded mapred.jobtracker.completeuserjobs.maximum then the job was flushed out of the jobtracker's memory after the job finishes min-time (hardcoded to 1 min). This caused jobclient's fail with NPE. In this patch the min-time to retain a job is made configurable (mapred.jobtracker.retirejob.interval.min). + + +--- + +* [MAPREDUCE-479](https://issues.apache.org/jira/browse/MAPREDUCE-479) | *Minor* | **Add reduce ID to shuffle clienttrace** + +Adds Reduce Attempt ID to ClientTrace log messages, and adds Reduce Attempt ID to HTTP query string sent to mapOutputServlet. Extracts partition number from attempt ID. + + +--- + +* [MAPREDUCE-476](https://issues.apache.org/jira/browse/MAPREDUCE-476) | *Minor* | **extend DistributedCache to work locally (LocalJobRunner)** + +Extended DistributedCache to work with LocalJobRunner. + + +--- + +* [MAPREDUCE-467](https://issues.apache.org/jira/browse/MAPREDUCE-467) | *Major* | **Collect information about number of tasks succeeded / total per time unit for a tasktracker.** + +Provide ability to collect statistics about tasks completed and succeeded for each tracker in time windows. The statistics is available on the jobtrackers' nodes UI page. + + +--- + +* [MAPREDUCE-463](https://issues.apache.org/jira/browse/MAPREDUCE-463) | *Major* | **The job setup and cleanup tasks should be optional** + +Added Configuration property "mapred.committer.job.setup.cleanup.needed" to specify whether job-setup and job-cleanup is needed for the job output committer. The default value is true. +Added Job.setJobSetupCleanupNeeded and JobContext.getJobSetupCleanupNeeded api for setting/getting the configuration. +If the configuration is set to false, no setup or cleanup will be done. + + +--- + +* [MAPREDUCE-416](https://issues.apache.org/jira/browse/MAPREDUCE-416) | *Major* | **Move the completed jobs' history files to a DONE subdirectory inside the configured history directory** + +Once the job is done, the history file and associated conf file is moved to history.folder/done folder. This is done to avoid garbling the running jobs' folder and the framework no longer gets affected with the files in the done folder. This helps in 2 was +1) ls on running folder (recovery) is faster with less files +2) changes in running folder results into FileNotFoundException. + + +So with existing code, the best way to keep the running folder clean is to note the id's of running job and then move files that are not in this list to the done folder. Note that on an avg there will be 2 files in the history folder namely +1) job history file +2) conf file. + +With restart, there might be more than 2 files, mostly the extra conf files. In such a case keep the oldest conf file (based on timestamp) and delete the rest. Note that this its better to do this when the jobtracker is down. + + +--- + +* [MAPREDUCE-408](https://issues.apache.org/jira/browse/MAPREDUCE-408) | *Major* | **TestKillSubProcesses fails with assertion failure sometimes** + +Fixed a bug in the testcase TestKillSubProcesses. + + +--- + +* [MAPREDUCE-375](https://issues.apache.org/jira/browse/MAPREDUCE-375) | *Major* | ** Change org.apache.hadoop.mapred.lib.NLineInputFormat and org.apache.hadoop.mapred.MapFileOutputFormat to use new api.** + +Ports NLineInputFormat and MapFileOutputFormat to the new Map/Reduce API + + +--- + +* [MAPREDUCE-373](https://issues.apache.org/jira/browse/MAPREDUCE-373) | *Major* | **Change org.apache.hadoop.mapred.lib. FieldSelectionMapReduce to use new api.** + +Ports FieldSelectionMapReduce to the new Map/Reduce API + + +--- + +* [MAPREDUCE-371](https://issues.apache.org/jira/browse/MAPREDUCE-371) | *Major* | **Change org.apache.hadoop.mapred.lib.KeyFieldBasedComparator and org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner to use new api** + +Ports KeyFieldBasedComparator and KeyFieldBasedPartitioner to the new Map/Reduce API + + +--- + +* [MAPREDUCE-370](https://issues.apache.org/jira/browse/MAPREDUCE-370) | *Major* | **Change org.apache.hadoop.mapred.lib.MultipleOutputs to use new api.** + +Ports MultipleOutputs to the new Map/Reduce API + + +--- + +* [MAPREDUCE-369](https://issues.apache.org/jira/browse/MAPREDUCE-369) | *Major* | **Change org.apache.hadoop.mapred.lib.MultipleInputs to use new api.** + +Patch that ports MultipleInputs, DelegatingInputFormat, DelegatingMapper and TaggedInputSplit to the new Map/Reduce API + + +--- + +* [MAPREDUCE-358](https://issues.apache.org/jira/browse/MAPREDUCE-358) | *Major* | **Change org.apache.hadoop.examples. AggregateWordCount and org.apache.hadoop.examples.AggregateWordHistogram to use new mapreduce api.** + +Modifies AggregateWordCount and AggregateWordHistogram examples to use the new Map/Reduce API + + +--- + +* [MAPREDUCE-355](https://issues.apache.org/jira/browse/MAPREDUCE-355) | *Major* | **Change org.apache.hadoop.mapred.join to use new api** + +Ports the mapred.join library to the new Map/Reduce API + + +--- + +* [MAPREDUCE-353](https://issues.apache.org/jira/browse/MAPREDUCE-353) | *Major* | **Allow shuffle read and connection timeouts to be configurable** + +Expert level config properties mapred.shuffle.connect.timeout and mapred.shuffle.read.timeout that are to be used at cluster level are added by this patch. + + +--- + +* [MAPREDUCE-336](https://issues.apache.org/jira/browse/MAPREDUCE-336) | *Major* | **The logging level of the tasks should be configurable by the job** + +Allow logging level of map/reduce tasks to be configurable. +Configuration changes: + add mapred.map.child.log.level + add mapred.reduce.child.log.level + + +--- + +* [MAPREDUCE-318](https://issues.apache.org/jira/browse/MAPREDUCE-318) | *Major* | **Refactor reduce shuffle code** + +Refactors shuffle code out of ReduceTask into separate classes in a new package(org.apache.hadoop.mapreduce.task.reduce) +Incorporates MAPREDUCE-240, batches up several map output files from a TT to a reducer in a single transfer +Introduces new Shuffle counters to keep track of shuffle errors + + +--- + +* [MAPREDUCE-284](https://issues.apache.org/jira/browse/MAPREDUCE-284) | *Major* | **Improvements to RPC between Child and TaskTracker** + +Enables tcp.nodelay for RPC between Child and TaskTracker. + + +--- + +* [MAPREDUCE-277](https://issues.apache.org/jira/browse/MAPREDUCE-277) | *Blocker* | **Job history counters should be avaible on the UI.** + +Modifies job history parser and Web UI to display counters + + +--- + +* [MAPREDUCE-270](https://issues.apache.org/jira/browse/MAPREDUCE-270) | *Major* | **TaskTracker could send an out-of-band heartbeat when the last running map/reduce completes** + +Introduced an option to allow tasktrackers to send an out of band heartbeat on task-completion to improve job latency. A new configuration option mapreduce.tasktracker.outofband.heartbeat is defined, which can be enabled to send this heartbeat. + + +--- + +* [MAPREDUCE-245](https://issues.apache.org/jira/browse/MAPREDUCE-245) | *Major* | **Job and JobControl classes should return interfaces rather than implementations** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-211](https://issues.apache.org/jira/browse/MAPREDUCE-211) | *Major* | **Provide a node health check script and run it periodically to check the node health status** + +Provides ability to run a health check script on the tasktracker nodes and blacklist nodes if they are unhealthy. + + +--- + +* [MAPREDUCE-157](https://issues.apache.org/jira/browse/MAPREDUCE-157) | *Major* | **Job History log file format is not friendly for external tools.** + +Changes the Job History file format to use JSON. +Simplifies the Job History Parsing logic +Removes duplication of code between HistoryViewer and the JSP files +History Files are now named as JobID\_user +Introduces a new cluster level configuration "mapreduce.cluster.jobhistory.maxage" for configuring the amount of time history files are kept before getting cleaned up +The configuration "hadoop.job.history.user.location" is no longer supported. + + +--- + +* [MAPREDUCE-153](https://issues.apache.org/jira/browse/MAPREDUCE-153) | *Major* | **TestJobInProgressListener sometimes timesout** + +Only one MR cluster is brought up and hence there is no scope of jobid clashing. + + +--- + +* [MAPREDUCE-144](https://issues.apache.org/jira/browse/MAPREDUCE-144) | *Major* | **TaskMemoryManager should log process-tree's status while killing tasks.** + +Modified TaskMemoryManager so that it logs a map/reduce task's process-tree's status just before it is killed when it grows out of its configured memory limits. The log dump is in the format " |- PID PPID PGRPID SESSID CMD\_NAME VMEM\_USAGE(BYTES) FULL\_CMD\_LINE". + +This is useful for debugging the cause for a map/reduce task and it's corresponding process-tree to be killed by the TaskMemoryManager. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/CHANGES.0.21.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/CHANGES.0.21.1.md new file mode 100644 index 00000000000..7acc4fc4b1f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/CHANGES.0.21.1.md @@ -0,0 +1,123 @@ + + +# Apache Hadoop Changelog + +## Release 0.21.1 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-1905](https://issues.apache.org/jira/browse/MAPREDUCE-1905) | Context.setStatus() and progress() api are ignored | Blocker | task | Amareshwari Sriramadasu | Amareshwari Sriramadasu | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-2040](https://issues.apache.org/jira/browse/MAPREDUCE-2040) | Forrest Documentation for Dynamic Priority Scheduler | Minor | contrib/dynamic-scheduler | Thomas Sandholm | Thomas Sandholm | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7193](https://issues.apache.org/jira/browse/HADOOP-7193) | Help message is wrong for touchz command. | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7177](https://issues.apache.org/jira/browse/HADOOP-7177) | CodecPool should report which compressor it is using | Trivial | native | Allen Wittenauer | Allen Wittenauer | +| [HADOOP-7117](https://issues.apache.org/jira/browse/HADOOP-7117) | Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml | Major | conf | Patrick Angeles | Harsh J | +| [HADOOP-6786](https://issues.apache.org/jira/browse/HADOOP-6786) | test-patch needs to verify Herriot integrity | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1596](https://issues.apache.org/jira/browse/HDFS-1596) | Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml | Major | documentation, namenode | Patrick Angeles | Harsh J | +| [HDFS-1343](https://issues.apache.org/jira/browse/HDFS-1343) | Instrumented build should be concentrated in one build area | Minor | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2140](https://issues.apache.org/jira/browse/MAPREDUCE-2140) | Re-generate fair scheduler design doc PDF | Trivial | . | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-1501](https://issues.apache.org/jira/browse/MAPREDUCE-1501) | FileInputFormat to support multi-level/recursive directory listing | Major | . | Zheng Shao | Zheng Shao | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7215](https://issues.apache.org/jira/browse/HADOOP-7215) | RPC clients must connect over a network interface corresponding to the host name in the client's kerberos principal key | Blocker | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7194](https://issues.apache.org/jira/browse/HADOOP-7194) | Potential Resource leak in IOUtils.java | Major | io | Devaraj K | Devaraj K | +| [HADOOP-7183](https://issues.apache.org/jira/browse/HADOOP-7183) | WritableComparator.get should not cache comparator objects | Blocker | . | Todd Lipcon | Tom White | +| [HADOOP-7174](https://issues.apache.org/jira/browse/HADOOP-7174) | null is displayed in the console,if the src path is invalid while doing copyToLocal operation from commandLine | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7162](https://issues.apache.org/jira/browse/HADOOP-7162) | FsShell: call srcFs.listStatus(src) twice | Minor | fs | Alexey Diomin | Alexey Diomin | +| [HADOOP-7120](https://issues.apache.org/jira/browse/HADOOP-7120) | 200 new Findbugs warnings | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7053](https://issues.apache.org/jira/browse/HADOOP-7053) | wrong FSNamesystem Audit logging setting in conf/log4j.properties | Minor | conf | Jingguo Yao | Jingguo Yao | +| [HADOOP-7052](https://issues.apache.org/jira/browse/HADOOP-7052) | misspelling of threshold in conf/log4j.properties | Major | conf | Jingguo Yao | Jingguo Yao | +| [HADOOP-7019](https://issues.apache.org/jira/browse/HADOOP-7019) | Refactor build targets to enable faster cross project dev cycles. | Major | build | Owen O'Malley | Luke Lu | +| [HADOOP-6993](https://issues.apache.org/jira/browse/HADOOP-6993) | Broken link on cluster setup page of docs | Major | documentation | Aaron T. Myers | Eli Collins | +| [HADOOP-6971](https://issues.apache.org/jira/browse/HADOOP-6971) | Clover build doesn't generate per-test coverage | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6969](https://issues.apache.org/jira/browse/HADOOP-6969) | CHANGES.txt does not reflect the release of version 0.21.0. | Major | . | Konstantin Shvachko | Tom White | +| [HADOOP-6954](https://issues.apache.org/jira/browse/HADOOP-6954) | Sources JARs are not correctly published to the Maven repository | Major | build | Tom White | Tom White | +| [HADOOP-6925](https://issues.apache.org/jira/browse/HADOOP-6925) | BZip2Codec incorrectly implements read() | Critical | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-6833](https://issues.apache.org/jira/browse/HADOOP-6833) | IPC leaks call parameters when exceptions thrown | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1750](https://issues.apache.org/jira/browse/HDFS-1750) | fs -ls hftp://file not working | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1728](https://issues.apache.org/jira/browse/HDFS-1728) | SecondaryNameNode.checkpointSize is in byte but not MB. | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1665](https://issues.apache.org/jira/browse/HDFS-1665) | Balancer sleeps inadequately | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1612](https://issues.apache.org/jira/browse/HDFS-1612) | HDFS Design Documentation is outdated | Minor | documentation | Joe Crobak | Joe Crobak | +| [HDFS-1598](https://issues.apache.org/jira/browse/HDFS-1598) | ListPathsServlet excludes .\*.crc files | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1552](https://issues.apache.org/jira/browse/HDFS-1552) | Remove java5 dependencies from build | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1548](https://issues.apache.org/jira/browse/HDFS-1548) | Fault-injection tests are executed multiple times if invoked with run-test-hdfs-fault-inject target | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1474](https://issues.apache.org/jira/browse/HDFS-1474) | ant binary-system is broken | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1452](https://issues.apache.org/jira/browse/HDFS-1452) | ant compile-contrib is broken | Major | contrib/hdfsproxy | Jakob Homan | Konstantin Boudnik | +| [HDFS-1444](https://issues.apache.org/jira/browse/HDFS-1444) | Test related code of build.xml is error-prone and needs to be re-aligned. | Minor | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1420](https://issues.apache.org/jira/browse/HDFS-1420) | Clover build doesn't generate per-test coverage | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1416](https://issues.apache.org/jira/browse/HDFS-1416) | CHANGES.txt does not reflect the release version 0.21.0. | Major | . | Konstantin Shvachko | Tom White | +| [HDFS-1413](https://issues.apache.org/jira/browse/HDFS-1413) | Broken links to HDFS Wiki in hdfs site and documentation. | Major | documentation | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1411](https://issues.apache.org/jira/browse/HDFS-1411) | The startup command of the Backup Node is "bin/hdfs namenode -backup" | Minor | documentation | Ching-Shen Chen | Ching-Shen Chen | +| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-1363](https://issues.apache.org/jira/browse/HDFS-1363) | startFileInternal should return the last block of the file opened for append as an under-construction block | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1206](https://issues.apache.org/jira/browse/HDFS-1206) | TestFiHFlush fails intermittently | Major | test | Tsz Wo Nicholas Sze | Konstantin Boudnik | +| [HDFS-1189](https://issues.apache.org/jira/browse/HDFS-1189) | Quota counts missed between clear quota and set quota | Major | namenode | Kang Xiao | John George | +| [HDFS-996](https://issues.apache.org/jira/browse/HDFS-996) | JUnit tests should never depend on anything in conf | Blocker | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2317](https://issues.apache.org/jira/browse/MAPREDUCE-2317) | HadoopArchives throwing NullPointerException while creating hadoop archives (.har files) | Minor | harchive | Devaraj K | Devaraj K | +| [MAPREDUCE-2228](https://issues.apache.org/jira/browse/MAPREDUCE-2228) | Remove java5 dependencies from build | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2223](https://issues.apache.org/jira/browse/MAPREDUCE-2223) | TestMRCLI might fail on Ubuntu with default /etc/hosts | Minor | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2134](https://issues.apache.org/jira/browse/MAPREDUCE-2134) | ant binary-system is broken in mapreduce project. | Major | build | Vinay Kumar Thota | Konstantin Boudnik | +| [MAPREDUCE-2090](https://issues.apache.org/jira/browse/MAPREDUCE-2090) | Clover build doesn't generate per-test coverage | Major | build, test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2086](https://issues.apache.org/jira/browse/MAPREDUCE-2086) | CHANGES.txt does not reflect the release of version 0.21.0. | Major | . | Konstantin Shvachko | Tom White | +| [MAPREDUCE-2032](https://issues.apache.org/jira/browse/MAPREDUCE-2032) | TestJobOutputCommitter fails in ant test run | Major | task | Amareshwari Sriramadasu | Dick King | +| [MAPREDUCE-1984](https://issues.apache.org/jira/browse/MAPREDUCE-1984) | herriot TestCluster fails because exclusion is not there | Major | . | Balaji Rajagopalan | Balaji Rajagopalan | +| [MAPREDUCE-1929](https://issues.apache.org/jira/browse/MAPREDUCE-1929) | Allow artifacts to be published to the staging Apache Nexus Maven Repository | Blocker | build | Tom White | Tom White | +| [MAPREDUCE-1897](https://issues.apache.org/jira/browse/MAPREDUCE-1897) | trunk build broken on compile-mapred-test | Major | test | Greg Roelofs | Konstantin Boudnik | +| [MAPREDUCE-1280](https://issues.apache.org/jira/browse/MAPREDUCE-1280) | Eclipse Plugin does not work with Eclipse Ganymede (3.4) | Major | . | Aaron Kimball | Alex Kozlov | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6934](https://issues.apache.org/jira/browse/HADOOP-6934) | test for ByteWritable comparator | Major | record | Johannes Zillmann | Johannes Zillmann | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7072](https://issues.apache.org/jira/browse/HADOOP-7072) | Remove java5 dependencies from build | Major | build | Konstantin Boudnik | Konstantin Boudnik | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6944](https://issues.apache.org/jira/browse/HADOOP-6944) | [Herriot] Implement a functionality for getting proxy users definitions like groups and hosts. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-1809](https://issues.apache.org/jira/browse/MAPREDUCE-1809) | Ant build changes for Streaming system tests in contrib projects. | Major | build | Vinay Kumar Thota | Vinay Kumar Thota | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/RELEASENOTES.0.21.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/RELEASENOTES.0.21.1.md new file mode 100644 index 00000000000..1d88d20ea93 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.21.1/RELEASENOTES.0.21.1.md @@ -0,0 +1,94 @@ + + +# Apache Hadoop 0.21.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7193](https://issues.apache.org/jira/browse/HADOOP-7193) | *Minor* | **Help message is wrong for touchz command.** + +Updated the help for the touchz command. + + +--- + +* [HADOOP-7117](https://issues.apache.org/jira/browse/HADOOP-7117) | *Major* | **Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml** + +Removed references to the older fs.checkpoint.* properties that resided in core-site.xml + + +--- + +* [HADOOP-6971](https://issues.apache.org/jira/browse/HADOOP-6971) | *Major* | **Clover build doesn't generate per-test coverage** + +This fix requires that test coverage is running under Clover 3.0+ + + +--- + +* [HADOOP-6944](https://issues.apache.org/jira/browse/HADOOP-6944) | *Major* | **[Herriot] Implement a functionality for getting proxy users definitions like groups and hosts.** + +I have just committed this to 0.21 and trunk. Thanks Vinay. + + +--- + +* [HDFS-1596](https://issues.apache.org/jira/browse/HDFS-1596) | *Major* | **Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml** + +Removed references to the older fs.checkpoint.* properties that resided in core-site.xml + + +--- + +* [HDFS-1420](https://issues.apache.org/jira/browse/HDFS-1420) | *Major* | **Clover build doesn't generate per-test coverage** + +This fix requires that test coverage is running under Clover 3.0+ + + +--- + +* [MAPREDUCE-2090](https://issues.apache.org/jira/browse/MAPREDUCE-2090) | *Major* | **Clover build doesn't generate per-test coverage** + +This fix requires that test coverage is running under Clover 3.0+ + + +--- + +* [MAPREDUCE-2040](https://issues.apache.org/jira/browse/MAPREDUCE-2040) | *Minor* | **Forrest Documentation for Dynamic Priority Scheduler** + +Forrest Documentation for Dynamic Priority Scheduler + + +--- + +* [MAPREDUCE-2032](https://issues.apache.org/jira/browse/MAPREDUCE-2032) | *Major* | **TestJobOutputCommitter fails in ant test run** + +Clears a problem that {{TestJobCleanup}} leaves behind files that cause {{TestJobOutputCommitter}} to error out. + + +--- + +* [MAPREDUCE-1905](https://issues.apache.org/jira/browse/MAPREDUCE-1905) | *Blocker* | **Context.setStatus() and progress() api are ignored** + +Moved the api public Counter getCounter(Enum\ counterName), public Counter getCounter(String groupName, String counterName) from org.apache.hadoop.mapreduce.TaskInputOutputContext to org.apache.hadoop.mapreduce.TaskAttemptContext + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/CHANGES.0.22.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/CHANGES.0.22.0.md new file mode 100644 index 00000000000..2869b09fe26 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/CHANGES.0.22.0.md @@ -0,0 +1,762 @@ + + +# Apache Hadoop Changelog + +## Release 0.22.0 - 2011-12-10 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7229](https://issues.apache.org/jira/browse/HADOOP-7229) | Absolute path to kinit in auto-renewal thread | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7137](https://issues.apache.org/jira/browse/HADOOP-7137) | Remove hod contrib | Major | . | Nigel Daley | Nigel Daley | +| [HADOOP-7013](https://issues.apache.org/jira/browse/HADOOP-7013) | Add boolean field isCorrupt to BlockLocation | Major | . | Patrick Kling | Patrick Kling | +| [HADOOP-6949](https://issues.apache.org/jira/browse/HADOOP-6949) | Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting | Major | io | Navis | Matt Foley | +| [HADOOP-6905](https://issues.apache.org/jira/browse/HADOOP-6905) | Better logging messages when a delegation token is invalid | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-6835](https://issues.apache.org/jira/browse/HADOOP-6835) | Support concatenated gzip files | Major | io | Tom White | Greg Roelofs | +| [HADOOP-6787](https://issues.apache.org/jira/browse/HADOOP-6787) | Factor out glob pattern code from FileContext and Filesystem | Major | fs | Luke Lu | Luke Lu | +| [HADOOP-6730](https://issues.apache.org/jira/browse/HADOOP-6730) | Bug in FileContext#copy and provide base class for FileContext tests | Major | fs, test | Eli Collins | Ravi Phulari | +| [HDFS-1825](https://issues.apache.org/jira/browse/HDFS-1825) | Remove thriftfs contrib | Major | . | Nigel Daley | Nigel Daley | +| [HDFS-1560](https://issues.apache.org/jira/browse/HDFS-1560) | dfs.data.dir permissions should default to 700 | Minor | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1435](https://issues.apache.org/jira/browse/HDFS-1435) | Provide an option to store fsimage compressed | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1315](https://issues.apache.org/jira/browse/HDFS-1315) | Add fsck event to audit log and remove other audit log events corresponding to FSCK listStatus and open calls | Major | namenode, tools | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1109](https://issues.apache.org/jira/browse/HDFS-1109) | HFTP and URL Encoding | Major | contrib/hdfsproxy, datanode | Dmytro Molkov | Dmytro Molkov | +| [HDFS-1080](https://issues.apache.org/jira/browse/HDFS-1080) | SecondaryNameNode image transfer should use the defined http address rather than local ip address | Major | namenode | Jakob Homan | Jakob Homan | +| [HDFS-1061](https://issues.apache.org/jira/browse/HDFS-1061) | Memory footprint optimization for INodeFile object. | Minor | namenode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-903](https://issues.apache.org/jira/browse/HDFS-903) | NN should verify images and edit logs on startup | Critical | namenode | Eli Collins | Hairong Kuang | +| [HDFS-330](https://issues.apache.org/jira/browse/HDFS-330) | Datanode Web UIs should provide robots.txt | Trivial | datanode | Allen Wittenauer | Allen Wittenauer | +| [HDFS-202](https://issues.apache.org/jira/browse/HDFS-202) | Add a bulk FIleSystem.getFileBlockLocations | Major | hdfs-client, namenode | Arun C Murthy | Hairong Kuang | +| [MAPREDUCE-1905](https://issues.apache.org/jira/browse/MAPREDUCE-1905) | Context.setStatus() and progress() api are ignored | Blocker | task | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1866](https://issues.apache.org/jira/browse/MAPREDUCE-1866) | Remove deprecated class org.apache.hadoop.streaming.UTF8ByteArrayUtils | Minor | contrib/streaming | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1733](https://issues.apache.org/jira/browse/MAPREDUCE-1733) | Authentication between pipes processes and java counterparts. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-1683](https://issues.apache.org/jira/browse/MAPREDUCE-1683) | Remove JNI calls from ClusterStatus cstr | Major | jobtracker | Chris Douglas | Luke Lu | +| [MAPREDUCE-1664](https://issues.apache.org/jira/browse/MAPREDUCE-1664) | Job Acls affect Queue Acls | Major | security | Ravi Gummadi | Ravi Gummadi | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6996](https://issues.apache.org/jira/browse/HADOOP-6996) | Allow CodecFactory to return a codec object given a codec' class name | Major | io | Hairong Kuang | Hairong Kuang | +| [HADOOP-6978](https://issues.apache.org/jira/browse/HADOOP-6978) | Add JNI support for secure IO operations | Blocker | io, native, security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6892](https://issues.apache.org/jira/browse/HADOOP-6892) | Common component of HDFS-1150 (Verify datanodes' identities to clients in secure clusters) | Major | security | Jakob Homan | Jakob Homan | +| [HADOOP-6889](https://issues.apache.org/jira/browse/HADOOP-6889) | Make RPC to have an option to timeout | Major | ipc | Hairong Kuang | John George | +| [HADOOP-6870](https://issues.apache.org/jira/browse/HADOOP-6870) | Add FileSystem#listLocatedStatus to list a directory's content together with each file's block locations | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-6832](https://issues.apache.org/jira/browse/HADOOP-6832) | Provide a web server plugin that uses a static user for the web UI | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-6600](https://issues.apache.org/jira/browse/HADOOP-6600) | mechanism for authorization check for inter-server protocols | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6586](https://issues.apache.org/jira/browse/HADOOP-6586) | Log authentication and authorization failures and successes | Major | security | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6472](https://issues.apache.org/jira/browse/HADOOP-6472) | add tokenCache option to GenericOptionsParser for passing file with secret keys to a map reduce job | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-4487](https://issues.apache.org/jira/browse/HADOOP-4487) | Security features for Hadoop | Major | security | Kan Zhang | Kan Zhang | +| [HDFS-1330](https://issues.apache.org/jira/browse/HDFS-1330) | Make RPCs to DataNodes timeout | Major | datanode | Hairong Kuang | John George | +| [HDFS-1318](https://issues.apache.org/jira/browse/HDFS-1318) | HDFS Namenode and Datanode WebUI information needs to be accessible programmatically for scripts | Major | . | Suresh Srinivas | Tanping Wang | +| [HDFS-1150](https://issues.apache.org/jira/browse/HDFS-1150) | Verify datanodes' identities to clients in secure clusters | Major | datanode | Jakob Homan | Jakob Homan | +| [HDFS-1111](https://issues.apache.org/jira/browse/HDFS-1111) | getCorruptFiles() should give some hint that the list is not complete | Major | . | Rodrigo Schmidt | Sriram Rao | +| [HDFS-1096](https://issues.apache.org/jira/browse/HDFS-1096) | allow dfsadmin/mradmin refresh of superuser proxy group mappings | Major | security | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1079](https://issues.apache.org/jira/browse/HDFS-1079) | HDFS implementation should throw exceptions defined in AbstractFileSystem | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1005](https://issues.apache.org/jira/browse/HDFS-1005) | Fsck security | Major | . | Jitendra Nath Pandey | Boris Shkolnik | +| [HDFS-1004](https://issues.apache.org/jira/browse/HDFS-1004) | Update NN to support Kerberized SSL from HADOOP-6584 | Major | namenode | Jakob Homan | Jakob Homan | +| [HDFS-1003](https://issues.apache.org/jira/browse/HDFS-1003) | authorization checks for inter-server protocol (based on HADOOP-6600) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-992](https://issues.apache.org/jira/browse/HDFS-992) | Re-factor block access token implementation to conform to the generic Token interface in Common | Major | security | Kan Zhang | Kan Zhang | +| [HDFS-811](https://issues.apache.org/jira/browse/HDFS-811) | Add metrics, failure reporting and additional tests for HDFS-457 | Minor | test | Ravi Phulari | Eli Collins | +| [HDFS-752](https://issues.apache.org/jira/browse/HDFS-752) | Add interface classification stable & scope to HDFS | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-708](https://issues.apache.org/jira/browse/HDFS-708) | A stress-test tool for HDFS. | Major | test, tools | Konstantin Shvachko | Joshua Harlow | +| [HDFS-528](https://issues.apache.org/jira/browse/HDFS-528) | Add ability for safemode to wait for a minimum number of live datanodes | Major | scripts | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1804](https://issues.apache.org/jira/browse/MAPREDUCE-1804) | Stress-test tool for HDFS introduced in HDFS-708 | Major | benchmarks, test | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-1680](https://issues.apache.org/jira/browse/MAPREDUCE-1680) | Add a metrics to track the number of heartbeats processed | Major | jobtracker | Hong Tang | Dick King | +| [MAPREDUCE-1594](https://issues.apache.org/jira/browse/MAPREDUCE-1594) | Support for Sleep Jobs in gridmix | Major | contrib/gridmix | rahul k singh | rahul k singh | +| [MAPREDUCE-1517](https://issues.apache.org/jira/browse/MAPREDUCE-1517) | streaming should support running on background | Major | contrib/streaming | Bochun Bai | Bochun Bai | +| [MAPREDUCE-1516](https://issues.apache.org/jira/browse/MAPREDUCE-1516) | JobTracker should issue a delegation token only for kerberos authenticated client | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-220](https://issues.apache.org/jira/browse/MAPREDUCE-220) | Collecting cpu and memory usage for MapReduce tasks | Major | task, tasktracker | Hong Tang | Scott Chen | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7861](https://issues.apache.org/jira/browse/HADOOP-7861) | changes2html.pl should generate links to HADOOP, HDFS, and MAPREDUCE jiras | Major | documentation | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-7786](https://issues.apache.org/jira/browse/HADOOP-7786) | Remove HDFS-specific configuration keys defined in FsConfig | Major | . | Eli Collins | Eli Collins | +| [HADOOP-7457](https://issues.apache.org/jira/browse/HADOOP-7457) | Remove out-of-date Chinese language documentation | Blocker | documentation | Jakob Homan | Jakob Homan | +| [HADOOP-7358](https://issues.apache.org/jira/browse/HADOOP-7358) | Improve log levels when exceptions caught in RPC handler | Minor | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7355](https://issues.apache.org/jira/browse/HADOOP-7355) | Add audience and stability annotations to HttpServer class | Major | . | stack | stack | +| [HADOOP-7346](https://issues.apache.org/jira/browse/HADOOP-7346) | Send back nicer error to clients using outdated IPC version | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7335](https://issues.apache.org/jira/browse/HADOOP-7335) | Force entropy to come from non-true random for tests | Minor | build, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-7325](https://issues.apache.org/jira/browse/HADOOP-7325) | hadoop command - do not accept class names starting with a hyphen | Minor | scripts | Brock Noland | Brock Noland | +| [HADOOP-7244](https://issues.apache.org/jira/browse/HADOOP-7244) | Documentation change for updated configuration keys | Blocker | documentation | Tom White | Tom White | +| [HADOOP-7241](https://issues.apache.org/jira/browse/HADOOP-7241) | fix typo of command 'hadoop fs -help tail' | Minor | fs, test | Wei Yongjun | Wei Yongjun | +| [HADOOP-7193](https://issues.apache.org/jira/browse/HADOOP-7193) | Help message is wrong for touchz command. | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7192](https://issues.apache.org/jira/browse/HADOOP-7192) | fs -stat docs aren't updated to reflect the format features | Trivial | documentation | Harsh J | Harsh J | +| [HADOOP-7189](https://issues.apache.org/jira/browse/HADOOP-7189) | Add ability to enable 'debug' property in JAAS configuration | Minor | security | Todd Lipcon | Ted Yu | +| [HADOOP-7177](https://issues.apache.org/jira/browse/HADOOP-7177) | CodecPool should report which compressor it is using | Trivial | native | Allen Wittenauer | Allen Wittenauer | +| [HADOOP-7154](https://issues.apache.org/jira/browse/HADOOP-7154) | Should set MALLOC\_ARENA\_MAX in hadoop-config.sh | Minor | scripts | Todd Lipcon | Todd Lipcon | +| [HADOOP-7134](https://issues.apache.org/jira/browse/HADOOP-7134) | configure files that are generated as part of the released tarball need to have executable bit set | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-7117](https://issues.apache.org/jira/browse/HADOOP-7117) | Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml | Major | conf | Patrick Angeles | Harsh J | +| [HADOOP-7110](https://issues.apache.org/jira/browse/HADOOP-7110) | Implement chmod with JNI | Major | io, native | Todd Lipcon | Todd Lipcon | +| [HADOOP-7106](https://issues.apache.org/jira/browse/HADOOP-7106) | Re-organize hadoop subversion layout | Blocker | build | Nigel Daley | Todd Lipcon | +| [HADOOP-7054](https://issues.apache.org/jira/browse/HADOOP-7054) | Change NN LoadGenerator to use the new FileContext api | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-7032](https://issues.apache.org/jira/browse/HADOOP-7032) | Assert type constraints in the FileStatus constructor | Major | fs | Eli Collins | Eli Collins | +| [HADOOP-7010](https://issues.apache.org/jira/browse/HADOOP-7010) | Typo in FileSystem.java | Minor | fs | Jingguo Yao | Jingguo Yao | +| [HADOOP-7009](https://issues.apache.org/jira/browse/HADOOP-7009) | MD5Hash provides a public factory method that creates an instance of MessageDigest | Major | io | Hairong Kuang | Hairong Kuang | +| [HADOOP-7008](https://issues.apache.org/jira/browse/HADOOP-7008) | Enable test-patch.sh to have a configured number of acceptable findbugs and javadoc warnings | Major | test | Nigel Daley | Giridharan Kesavan | +| [HADOOP-7007](https://issues.apache.org/jira/browse/HADOOP-7007) | update the hudson-test-patch target to work with the latest test-patch script. | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-7005](https://issues.apache.org/jira/browse/HADOOP-7005) | Update test-patch.sh to remove callback to Hudson master | Major | test | Nigel Daley | Nigel Daley | +| [HADOOP-6987](https://issues.apache.org/jira/browse/HADOOP-6987) | Use JUnit Rule to optionally fail test cases that run more than 10 seconds | Major | test | Jakob Homan | Jakob Homan | +| [HADOOP-6985](https://issues.apache.org/jira/browse/HADOOP-6985) | Suggest that HADOOP\_OPTS be preserved in hadoop-env.sh.template | Minor | . | Ramkumar Vadali | Ramkumar Vadali | +| [HADOOP-6977](https://issues.apache.org/jira/browse/HADOOP-6977) | Herriot daemon clients should vend statistics | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6950](https://issues.apache.org/jira/browse/HADOOP-6950) | Suggest that HADOOP\_CLASSPATH should be preserved in hadoop-env.sh.template | Trivial | scripts | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-6943](https://issues.apache.org/jira/browse/HADOOP-6943) | The GroupMappingServiceProvider interface should be public | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-6911](https://issues.apache.org/jira/browse/HADOOP-6911) | doc update for DelegationTokenFetcher (part of HDFS-1036) | Minor | . | Boris Shkolnik | | +| [HADOOP-6903](https://issues.apache.org/jira/browse/HADOOP-6903) | Make AbstractFileSystem's methods public to allow filter-Fs like implementions in a differnt package than fs | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-6890](https://issues.apache.org/jira/browse/HADOOP-6890) | Improve listFiles API introduced by HADOOP-6870 | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-6884](https://issues.apache.org/jira/browse/HADOOP-6884) | Add LOG.isDebugEnabled() guard for each LOG.debug("...") | Major | . | Erik Steffl | Erik Steffl | +| [HADOOP-6879](https://issues.apache.org/jira/browse/HADOOP-6879) | Provide SSH based (Jsch) remote execution API for system tests | Major | build, test | Iyappan Srinivasan | Konstantin Boudnik | +| [HADOOP-6877](https://issues.apache.org/jira/browse/HADOOP-6877) | Common part of HDFS-1178 | Major | ipc | Kan Zhang | Kan Zhang | +| [HADOOP-6862](https://issues.apache.org/jira/browse/HADOOP-6862) | Add api to add user/group to AccessControlList | Major | security | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-6861](https://issues.apache.org/jira/browse/HADOOP-6861) | Method in Credentials to read and write a token storage file. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6859](https://issues.apache.org/jira/browse/HADOOP-6859) | Introduce additional statistics to FileSystem | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6856](https://issues.apache.org/jira/browse/HADOOP-6856) | SequenceFile and MapFile need cleanup to remove redundant constructors | Major | io | Owen O'Malley | Owen O'Malley | +| [HADOOP-6845](https://issues.apache.org/jira/browse/HADOOP-6845) | TokenStorage renamed to Credentials. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6825](https://issues.apache.org/jira/browse/HADOOP-6825) | FileStatus needs unit tests | Major | . | Rodrigo Schmidt | Rodrigo Schmidt | +| [HADOOP-6818](https://issues.apache.org/jira/browse/HADOOP-6818) | Provide a JNI-based implementation of GroupMappingServiceProvider | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6814](https://issues.apache.org/jira/browse/HADOOP-6814) | Method in UGI to get the authentication method of the real user. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6811](https://issues.apache.org/jira/browse/HADOOP-6811) | Remove EC2 bash scripts | Blocker | . | Tom White | Tom White | +| [HADOOP-6805](https://issues.apache.org/jira/browse/HADOOP-6805) | add buildDTServiceName method to SecurityUtil (as part of MAPREDUCE-1718) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6791](https://issues.apache.org/jira/browse/HADOOP-6791) | Refresh for proxy superuser config (common part for HDFS-1096) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6761](https://issues.apache.org/jira/browse/HADOOP-6761) | Improve Trash Emptier | Major | . | Dmytro Molkov | Dmytro Molkov | +| [HADOOP-6745](https://issues.apache.org/jira/browse/HADOOP-6745) | adding some java doc to Server.RpcMetrics, UGI | Minor | ipc | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6714](https://issues.apache.org/jira/browse/HADOOP-6714) | FsShell 'hadoop fs -text' does not support compression codecs | Major | . | Patrick Angeles | Patrick Angeles | +| [HADOOP-6693](https://issues.apache.org/jira/browse/HADOOP-6693) | Add metrics to track kerberos login activity | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6674](https://issues.apache.org/jira/browse/HADOOP-6674) | Performance Improvement in Secure RPC | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6632](https://issues.apache.org/jira/browse/HADOOP-6632) | Support for using different Kerberos keys for different instances of Hadoop services | Major | . | Kan Zhang | Kan Zhang | +| [HADOOP-6623](https://issues.apache.org/jira/browse/HADOOP-6623) | Add StringUtils.split for non-escaped single-character separator | Minor | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-6605](https://issues.apache.org/jira/browse/HADOOP-6605) | Add JAVA\_HOME detection to hadoop-config | Minor | . | Chad Metcalf | Eli Collins | +| [HADOOP-6599](https://issues.apache.org/jira/browse/HADOOP-6599) | Split RPC metrics into summary and detailed metrics | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6584](https://issues.apache.org/jira/browse/HADOOP-6584) | Provide Kerberized SSL encryption for webservices | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-6578](https://issues.apache.org/jira/browse/HADOOP-6578) | Configuration should trim whitespace around a lot of value types | Minor | conf | Todd Lipcon | Michele Catasta | +| [HADOOP-6562](https://issues.apache.org/jira/browse/HADOOP-6562) | FileContextSymlinkBaseTest should use FileContextTestHelper | Minor | test | Eli Collins | Eli Collins | +| [HADOOP-6436](https://issues.apache.org/jira/browse/HADOOP-6436) | Remove auto-generated native build files | Major | . | Eli Collins | Roman Shaposhnik | +| [HADOOP-6298](https://issues.apache.org/jira/browse/HADOOP-6298) | BytesWritable#getBytes is a bad name that leads to programming mistakes | Major | . | Nathan Marz | Owen O'Malley | +| [HADOOP-6056](https://issues.apache.org/jira/browse/HADOOP-6056) | Use java.net.preferIPv4Stack to force IPv4 | Major | scripts | Steve Loughran | Michele Catasta | +| [HADOOP-4675](https://issues.apache.org/jira/browse/HADOOP-4675) | Current Ganglia metrics implementation is incompatible with Ganglia 3.1 | Major | metrics | Brian Bockelman | Brian Bockelman | +| [HDFS-2286](https://issues.apache.org/jira/browse/HDFS-2286) | DataXceiverServer logs AsynchronousCloseException at shutdown | Trivial | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2054](https://issues.apache.org/jira/browse/HDFS-2054) | BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() | Minor | datanode | Kihwal Lee | Kihwal Lee | +| [HDFS-2039](https://issues.apache.org/jira/browse/HDFS-2039) | TestNameNodeMetrics uses a bad test root path | Major | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1980](https://issues.apache.org/jira/browse/HDFS-1980) | Move build/webapps deeper in directory heirarchy to aid eclipse users | Major | build | Todd Lipcon | Todd Lipcon | +| [HDFS-1957](https://issues.apache.org/jira/browse/HDFS-1957) | Documentation for HFTP | Minor | documentation | Ari Rabkin | Ari Rabkin | +| [HDFS-1954](https://issues.apache.org/jira/browse/HDFS-1954) | Improve corrupt files warning message on NameNode web UI | Major | namenode | philo vivero | Patrick Hunt | +| [HDFS-1947](https://issues.apache.org/jira/browse/HDFS-1947) | DFSClient should use mapreduce.task.attempt.id | Trivial | hdfs-client | Eli Collins | Eli Collins | +| [HDFS-1935](https://issues.apache.org/jira/browse/HDFS-1935) | Build should not redownload ivy on every invocation | Minor | build | Todd Lipcon | Joep Rottinghuis | +| [HDFS-1866](https://issues.apache.org/jira/browse/HDFS-1866) | Document dfs.datanode.max.transfer.threads in hdfs-default.xml | Minor | datanode, documentation | Eli Collins | Harsh J | +| [HDFS-1861](https://issues.apache.org/jira/browse/HDFS-1861) | Rename dfs.datanode.max.xcievers and bump its default value | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-1736](https://issues.apache.org/jira/browse/HDFS-1736) | Break dependency between DatanodeJspHelper and FsShell | Minor | datanode | Daryn Sharp | Daryn Sharp | +| [HDFS-1619](https://issues.apache.org/jira/browse/HDFS-1619) | Remove AC\_TYPE\* from the libhdfs | Major | libhdfs | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-1618](https://issues.apache.org/jira/browse/HDFS-1618) | configure files that are generated as part of the released tarball need to have executable bit set | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-1596](https://issues.apache.org/jira/browse/HDFS-1596) | Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml | Major | documentation, namenode | Patrick Angeles | Harsh J | +| [HDFS-1582](https://issues.apache.org/jira/browse/HDFS-1582) | Remove auto-generated native build files | Major | libhdfs | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-1513](https://issues.apache.org/jira/browse/HDFS-1513) | Fix a number of warnings | Minor | . | Eli Collins | Eli Collins | +| [HDFS-1491](https://issues.apache.org/jira/browse/HDFS-1491) | Update Hdfs to match the change of methods from protected to public in AbstractFileSystem (Hadoop-6903) | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-1485](https://issues.apache.org/jira/browse/HDFS-1485) | Typo in BlockPlacementPolicy.java | Minor | . | Jingguo Yao | Jingguo Yao | +| [HDFS-1472](https://issues.apache.org/jira/browse/HDFS-1472) | Refactor DFSck to allow programmatic access to output | Major | tools | Ramkumar Vadali | Ramkumar Vadali | +| [HDFS-1457](https://issues.apache.org/jira/browse/HDFS-1457) | Limit transmission rate when transfering image between primary and secondary NNs | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1456](https://issues.apache.org/jira/browse/HDFS-1456) | Provide builder for constructing instances of MiniDFSCluster | Major | test | Jakob Homan | Jakob Homan | +| [HDFS-1454](https://issues.apache.org/jira/browse/HDFS-1454) | Update the documentation to reflect true client caching strategy | Major | documentation, hdfs-client | Jeff Hammerbacher | Harsh J | +| [HDFS-1434](https://issues.apache.org/jira/browse/HDFS-1434) | Refactor Datanode#startDataNode method | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1426](https://issues.apache.org/jira/browse/HDFS-1426) | Remove unused method BlockInfo#listCount | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1417](https://issues.apache.org/jira/browse/HDFS-1417) | Add @Override annotation to SimulatedFSDataset methods that implement FSDatasetInterface | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1408](https://issues.apache.org/jira/browse/HDFS-1408) | Herriot NN and DN clients should vend statistics | Major | test | Al Thompson | Konstantin Boudnik | +| [HDFS-1407](https://issues.apache.org/jira/browse/HDFS-1407) | Use Block in DataTransferProtocol | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1395](https://issues.apache.org/jira/browse/HDFS-1395) | Add @Override annotation to FSDataset methods that implement FSDatasetInterface | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1387](https://issues.apache.org/jira/browse/HDFS-1387) | Update HDFS permissions guide for security | Major | documentation, security | Todd Lipcon | Todd Lipcon | +| [HDFS-1383](https://issues.apache.org/jira/browse/HDFS-1383) | Better error messages on hftp | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1369](https://issues.apache.org/jira/browse/HDFS-1369) | Invalid javadoc reference in FSDatasetMBean.java | Trivial | datanode | Eli Collins | Eli Collins | +| [HDFS-1368](https://issues.apache.org/jira/browse/HDFS-1368) | Add a block counter to DatanodeDescriptor | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1356](https://issues.apache.org/jira/browse/HDFS-1356) | Provide information as to whether or not security is enabled on web interface for NameNode (part of HADOOP-6822) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1353](https://issues.apache.org/jira/browse/HDFS-1353) | Remove most of getBlockLocation optimization | Major | namenode | Jakob Homan | Jakob Homan | +| [HDFS-1320](https://issues.apache.org/jira/browse/HDFS-1320) | Add LOG.isDebugEnabled() guard for each LOG.debug("...") | Major | . | Erik Steffl | Erik Steffl | +| [HDFS-1307](https://issues.apache.org/jira/browse/HDFS-1307) | Add start time, end time and total time taken for FSCK to FSCK report | Major | namenode, tools | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1304](https://issues.apache.org/jira/browse/HDFS-1304) | There is no unit test for HftpFileSystem.open(..) | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1302](https://issues.apache.org/jira/browse/HDFS-1302) | Use writeTokenStorageToStream method in Credentials to store credentials to a file. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1298](https://issues.apache.org/jira/browse/HDFS-1298) | Add support in HDFS to update statistics that tracks number of file system operations in FileSystem | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1297](https://issues.apache.org/jira/browse/HDFS-1297) | Fix some comments | Trivial | documentation | Jeff Ames | Jeff Ames | +| [HDFS-1272](https://issues.apache.org/jira/browse/HDFS-1272) | HDFS changes corresponding to rename of TokenStorage to Credentials | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1205](https://issues.apache.org/jira/browse/HDFS-1205) | FSDatasetAsyncDiskService should name its threads | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1203](https://issues.apache.org/jira/browse/HDFS-1203) | DataNode should sleep before reentering service loop after an exception | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1201](https://issues.apache.org/jira/browse/HDFS-1201) | Support for using different Kerberos keys for Namenode and datanode. | Major | . | Jitendra Nath Pandey | Kan Zhang | +| [HDFS-1190](https://issues.apache.org/jira/browse/HDFS-1190) | Remove unused getNamenode() method from DataNode. | Minor | datanode | Jeff Ames | Jeff Ames | +| [HDFS-1187](https://issues.apache.org/jira/browse/HDFS-1187) | Modify fetchdt to allow renewing and canceling token | Major | security | Owen O'Malley | Owen O'Malley | +| [HDFS-1185](https://issues.apache.org/jira/browse/HDFS-1185) | Remove duplicate now() functions in DataNode, FSNamesystem | Minor | datanode, namenode | Jeff Ames | Jeff Ames | +| [HDFS-1184](https://issues.apache.org/jira/browse/HDFS-1184) | Replace tabs in code with spaces | Minor | . | Jeff Ames | Jeff Ames | +| [HDFS-1183](https://issues.apache.org/jira/browse/HDFS-1183) | Remove some duplicate code in NamenodeJspHelper.java | Minor | namenode | Jeff Ames | Jeff Ames | +| [HDFS-1178](https://issues.apache.org/jira/browse/HDFS-1178) | The NameNode servlets should not use RPC to connect to the NameNode | Major | namenode | Owen O'Malley | Owen O'Malley | +| [HDFS-1160](https://issues.apache.org/jira/browse/HDFS-1160) | Improve some FSDataset warnings and comments | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-1140](https://issues.apache.org/jira/browse/HDFS-1140) | Speedup INode.getPathComponents | Minor | namenode | Dmytro Molkov | Dmytro Molkov | +| [HDFS-1114](https://issues.apache.org/jira/browse/HDFS-1114) | Reducing NameNode memory usage by an alternate hash table | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1110](https://issues.apache.org/jira/browse/HDFS-1110) | Namenode heap optimization - reuse objects for commonly used file names | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1093](https://issues.apache.org/jira/browse/HDFS-1093) | Improve namenode scalability by splitting the FSNamesystem synchronized section in a read/write lock | Major | namenode | dhruba borthakur | dhruba borthakur | +| [HDFS-1081](https://issues.apache.org/jira/browse/HDFS-1081) | Performance regression in DistributedFileSystem::getFileBlockLocations in secure systems | Major | security | Jakob Homan | Jakob Homan | +| [HDFS-1055](https://issues.apache.org/jira/browse/HDFS-1055) | Improve thread naming for DataXceivers | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1035](https://issues.apache.org/jira/browse/HDFS-1035) | Generate Eclipse's .classpath file from Ivy config | Major | build | Tom White | Nigel Daley | +| [HDFS-1033](https://issues.apache.org/jira/browse/HDFS-1033) | In secure clusters, NN and SNN should verify that the remote principal during image and edits transfer | Major | namenode, security | Jakob Homan | Jakob Homan | +| [HDFS-1028](https://issues.apache.org/jira/browse/HDFS-1028) | INode.getPathNames could split more efficiently | Minor | namenode | Todd Lipcon | Dmytro Molkov | +| [HDFS-1023](https://issues.apache.org/jira/browse/HDFS-1023) | Allow http server to start as regular principal if https principal not defined. | Major | namenode | Jakob Homan | Jakob Homan | +| [HDFS-947](https://issues.apache.org/jira/browse/HDFS-947) | The namenode should redirect a hftp request to read a file to the datanode that has the maximum number of local replicas | Major | . | dhruba borthakur | Dmytro Molkov | +| [HDFS-941](https://issues.apache.org/jira/browse/HDFS-941) | Datanode xceiver protocol should allow reuse of a connection | Major | datanode, hdfs-client, performance | Todd Lipcon | bc Wong | +| [HDFS-895](https://issues.apache.org/jira/browse/HDFS-895) | Allow hflush/sync to occur in parallel with new writes to the file | Major | hdfs-client | dhruba borthakur | Todd Lipcon | +| [HDFS-884](https://issues.apache.org/jira/browse/HDFS-884) | DataNode makeInstance should report the directory list when failing to start up | Minor | datanode | Steve Loughran | Steve Loughran | +| [HDFS-882](https://issues.apache.org/jira/browse/HDFS-882) | Datanode could log what hostname and ports its listening on on startup | Minor | datanode | Steve Loughran | Steve Loughran | +| [HDFS-881](https://issues.apache.org/jira/browse/HDFS-881) | Refactor DataNode Packet header into DataTransferProtocol | Major | . | Todd Lipcon | Todd Lipcon | +| [HDFS-853](https://issues.apache.org/jira/browse/HDFS-853) | The HDFS webUI should show a metric that summarizes whether the cluster is balanced regarding disk space usage | Major | namenode | dhruba borthakur | Dmytro Molkov | +| [HDFS-718](https://issues.apache.org/jira/browse/HDFS-718) | configuration parameter to prevent accidental formatting of HDFS filesystem | Minor | namenode | Andrew Ryan | Andrew Ryan | +| [HDFS-599](https://issues.apache.org/jira/browse/HDFS-599) | Improve Namenode robustness by prioritizing datanode heartbeats over client requests | Major | namenode | dhruba borthakur | Dmytro Molkov | +| [HDFS-556](https://issues.apache.org/jira/browse/HDFS-556) | Provide info on failed volumes in the web ui | Major | namenode | Jakob Homan | Eli Collins | +| [HDFS-455](https://issues.apache.org/jira/browse/HDFS-455) | Make NN and DN handle in a intuitive way comma-separated configuration strings | Minor | datanode, namenode | Michele Catasta | Michele Catasta | +| [MAPREDUCE-2505](https://issues.apache.org/jira/browse/MAPREDUCE-2505) | Explain how to use ACLs in the fair scheduler | Minor | contrib/fair-share, documentation | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-2502](https://issues.apache.org/jira/browse/MAPREDUCE-2502) | JobSubmitter should use mapreduce.job.maps | Trivial | job submission | Eli Collins | Eli Collins | +| [MAPREDUCE-2410](https://issues.apache.org/jira/browse/MAPREDUCE-2410) | document multiple keys per reducer oddity in hadoop streaming FAQ | Minor | contrib/streaming, documentation | Dieter Plaetinck | Harsh J | +| [MAPREDUCE-2372](https://issues.apache.org/jira/browse/MAPREDUCE-2372) | TaskLogAppender mechanism shouldn't be set in log4j.properties | Major | task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2337](https://issues.apache.org/jira/browse/MAPREDUCE-2337) | Remove dependence of public MapReduce API on classes in server package | Major | . | Tom White | Tom White | +| [MAPREDUCE-2314](https://issues.apache.org/jira/browse/MAPREDUCE-2314) | configure files that are generated as part of the released tarball need to have executable bit set | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [MAPREDUCE-2260](https://issues.apache.org/jira/browse/MAPREDUCE-2260) | Remove auto-generated native build files | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [MAPREDUCE-2184](https://issues.apache.org/jira/browse/MAPREDUCE-2184) | Port DistRaid.java to new mapreduce API | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2167](https://issues.apache.org/jira/browse/MAPREDUCE-2167) | Faster directory traversal for raid node | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2150](https://issues.apache.org/jira/browse/MAPREDUCE-2150) | RaidNode should periodically fix corrupt blocks | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2147](https://issues.apache.org/jira/browse/MAPREDUCE-2147) | JobInProgress has some redundant lines in its ctor | Trivial | jobtracker | Harsh J | Harsh J | +| [MAPREDUCE-2145](https://issues.apache.org/jira/browse/MAPREDUCE-2145) | upgrade clover on the builds servers to v 3.0.2 | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-2141](https://issues.apache.org/jira/browse/MAPREDUCE-2141) | Add an "extra data" field to Task for use by Mesos | Minor | . | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-2140](https://issues.apache.org/jira/browse/MAPREDUCE-2140) | Re-generate fair scheduler design doc PDF | Trivial | . | Matei Zaharia | Matei Zaharia | +| [MAPREDUCE-2132](https://issues.apache.org/jira/browse/MAPREDUCE-2132) | Need a command line option in RaidShell to fix blocks using raid | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2126](https://issues.apache.org/jira/browse/MAPREDUCE-2126) | JobQueueJobInProgressListener's javadoc is inconsistent with source code | Minor | jobtracker | Jingguo Yao | Jingguo Yao | +| [MAPREDUCE-2103](https://issues.apache.org/jira/browse/MAPREDUCE-2103) | task-controller shouldn't require o-r permissions | Trivial | task-controller | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2093](https://issues.apache.org/jira/browse/MAPREDUCE-2093) | Herriot JT and TT clients should vend statistics | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2046](https://issues.apache.org/jira/browse/MAPREDUCE-2046) | A CombineFileInputSplit cannot be less than a dfs block | Major | . | Namit Jain | dhruba borthakur | +| [MAPREDUCE-2026](https://issues.apache.org/jira/browse/MAPREDUCE-2026) | JobTracker.getJobCounters() should not hold JobTracker lock while calling JobInProgress.getCounters() | Major | . | Scott Chen | Joydeep Sen Sarma | +| [MAPREDUCE-1945](https://issues.apache.org/jira/browse/MAPREDUCE-1945) | Support for using different Kerberos keys for Jobtracker and TaskTrackers | Major | . | Kan Zhang | Kan Zhang | +| [MAPREDUCE-1936](https://issues.apache.org/jira/browse/MAPREDUCE-1936) | [gridmix3] Make Gridmix3 more customizable. | Major | contrib/gridmix | Hong Tang | Hong Tang | +| [MAPREDUCE-1935](https://issues.apache.org/jira/browse/MAPREDUCE-1935) | HFTP needs to be updated to use delegation tokens (from HDFS-1007) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1931](https://issues.apache.org/jira/browse/MAPREDUCE-1931) | Gridmix forrest documentation | Major | contrib/gridmix | rahul k singh | Ranjit Mathew | +| [MAPREDUCE-1918](https://issues.apache.org/jira/browse/MAPREDUCE-1918) | Add documentation to Rumen | Major | tools/rumen | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1893](https://issues.apache.org/jira/browse/MAPREDUCE-1893) | Multiple reducers for Slive | Major | benchmarks, test | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-1892](https://issues.apache.org/jira/browse/MAPREDUCE-1892) | RaidNode can allow layered policies more efficiently | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1878](https://issues.apache.org/jira/browse/MAPREDUCE-1878) | Add MRUnit documentation | Major | contrib/mrunit | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1868](https://issues.apache.org/jira/browse/MAPREDUCE-1868) | Add read timeout on userlog pull | Major | client | Krishna Ramachandran | Krishna Ramachandran | +| [MAPREDUCE-1851](https://issues.apache.org/jira/browse/MAPREDUCE-1851) | Document configuration parameters in streaming | Major | contrib/streaming, documentation | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1850](https://issues.apache.org/jira/browse/MAPREDUCE-1850) | Include job submit host information (name and ip) in jobconf and jobdetails display | Major | . | Krishna Ramachandran | Krishna Ramachandran | +| [MAPREDUCE-1848](https://issues.apache.org/jira/browse/MAPREDUCE-1848) | Put number of speculative, data local, rack local tasks in JobTracker metrics | Major | jobtracker | Scott Chen | Scott Chen | +| [MAPREDUCE-1840](https://issues.apache.org/jira/browse/MAPREDUCE-1840) | [Gridmix] Exploit/Add security features in GridMix | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1838](https://issues.apache.org/jira/browse/MAPREDUCE-1838) | DistRaid map tasks have large variance in running times | Minor | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1832](https://issues.apache.org/jira/browse/MAPREDUCE-1832) | Support for file sizes less than 1MB in DFSIO benchmark. | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-1829](https://issues.apache.org/jira/browse/MAPREDUCE-1829) | JobInProgress.findSpeculativeTask should use min() to find the candidate instead of sort() | Major | jobtracker | Scott Chen | Scott Chen | +| [MAPREDUCE-1819](https://issues.apache.org/jira/browse/MAPREDUCE-1819) | RaidNode should be smarter in submitting Raid jobs | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1818](https://issues.apache.org/jira/browse/MAPREDUCE-1818) | RaidNode should specify a pool name incase the cluster is using FairScheduler | Minor | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1816](https://issues.apache.org/jira/browse/MAPREDUCE-1816) | HAR files used for RAID parity need to have configurable partfile size | Minor | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1798](https://issues.apache.org/jira/browse/MAPREDUCE-1798) | normalize property names for JT kerberos principal names in configuration (from HADOOP 6633) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1785](https://issues.apache.org/jira/browse/MAPREDUCE-1785) | Add streaming config option for not emitting the key | Minor | contrib/streaming | Eli Collins | Eli Collins | +| [MAPREDUCE-1783](https://issues.apache.org/jira/browse/MAPREDUCE-1783) | Task Initialization should be delayed till when a job can be run | Major | contrib/fair-share | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1778](https://issues.apache.org/jira/browse/MAPREDUCE-1778) | CompletedJobStatusStore initialization should fail if {mapred.job.tracker.persist.jobstatus.dir} is unwritable | Major | jobtracker | Amar Kamat | Krishna Ramachandran | +| [MAPREDUCE-1762](https://issues.apache.org/jira/browse/MAPREDUCE-1762) | Add a setValue() method in Counter | Major | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1761](https://issues.apache.org/jira/browse/MAPREDUCE-1761) | FairScheduler should allow separate configuration of node and rack locality wait time | Major | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1711](https://issues.apache.org/jira/browse/MAPREDUCE-1711) | Gridmix should provide an option to submit jobs to the same queues as specified in the trace. | Major | contrib/gridmix | Hong Tang | rahul k singh | +| [MAPREDUCE-1626](https://issues.apache.org/jira/browse/MAPREDUCE-1626) | Publish Javadoc for all contrib packages with user-facing APIs | Major | documentation | Tom White | Jolly Chen | +| [MAPREDUCE-1592](https://issues.apache.org/jira/browse/MAPREDUCE-1592) | Generate Eclipse's .classpath file from Ivy config | Major | build | Tom White | Tom White | +| [MAPREDUCE-1548](https://issues.apache.org/jira/browse/MAPREDUCE-1548) | Hadoop archives should be able to preserve times and other properties from original files | Major | harchive | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1546](https://issues.apache.org/jira/browse/MAPREDUCE-1546) | Jobtracker JSP pages should automatically redirect to the corresponding history page if not in memory | Minor | . | Scott Chen | Scott Chen | +| [MAPREDUCE-1545](https://issues.apache.org/jira/browse/MAPREDUCE-1545) | Add 'first-task-launched' to job-summary | Major | jobtracker | Arun C Murthy | Luke Lu | +| [MAPREDUCE-1526](https://issues.apache.org/jira/browse/MAPREDUCE-1526) | Cache the job related information while submitting the job , this would avoid many RPC calls to JobTracker. | Major | contrib/gridmix | rahul k singh | rahul k singh | +| [MAPREDUCE-1492](https://issues.apache.org/jira/browse/MAPREDUCE-1492) | Delete or recreate obsolete har files used on hdfs raid | Major | contrib/raid | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-1382](https://issues.apache.org/jira/browse/MAPREDUCE-1382) | MRAsyncDiscService should tolerate missing local.dir | Major | . | Scott Chen | Zheng Shao | +| [MAPREDUCE-1376](https://issues.apache.org/jira/browse/MAPREDUCE-1376) | Support for varied user submission in Gridmix | Major | contrib/gridmix | Chris Douglas | Chris Douglas | +| [MAPREDUCE-1354](https://issues.apache.org/jira/browse/MAPREDUCE-1354) | Incremental enhancements to the JobTracker for better scalability | Critical | jobtracker | Devaraj Das | Dick King | +| [MAPREDUCE-1253](https://issues.apache.org/jira/browse/MAPREDUCE-1253) | Making Mumak work with Capacity-Scheduler | Major | contrib/mumak | Anirban Dasgupta | Anirban Dasgupta | +| [MAPREDUCE-1248](https://issues.apache.org/jira/browse/MAPREDUCE-1248) | Redundant memory copying in StreamKeyValUtil | Minor | contrib/streaming | Ruibang He | Ruibang He | +| [MAPREDUCE-1159](https://issues.apache.org/jira/browse/MAPREDUCE-1159) | Limit Job name on jobtracker.jsp to be 80 char long | Trivial | . | Zheng Shao | Harsh J | +| [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | separate jvm param for mapper and reducer | Minor | . | Koji Noguchi | Arun C Murthy | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7697](https://issues.apache.org/jira/browse/HADOOP-7697) | Remove dependency on different version of slf4j in avro | Major | build | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-7663](https://issues.apache.org/jira/browse/HADOOP-7663) | TestHDFSTrash failing on 22 | Major | test | Mayank Bansal | Mayank Bansal | +| [HADOOP-7646](https://issues.apache.org/jira/browse/HADOOP-7646) | Make hadoop-common use same version of avro as HBase | Major | io, ipc | Joep Rottinghuis | Joep Rottinghuis | +| [HADOOP-7577](https://issues.apache.org/jira/browse/HADOOP-7577) | TT does not start due to backwards compatibility wrt. EventCounter | Minor | metrics | Joep Rottinghuis | Joep Rottinghuis | +| [HADOOP-7568](https://issues.apache.org/jira/browse/HADOOP-7568) | SequenceFile should not print into stdout | Major | io | Konstantin Shvachko | Plamen Jeliazkov | +| [HADOOP-7514](https://issues.apache.org/jira/browse/HADOOP-7514) | Build fails with ClassCastException when running both mvn-install and mvn-deploy targets | Minor | build | Joep Rottinghuis | Joep Rottinghuis | +| [HADOOP-7513](https://issues.apache.org/jira/browse/HADOOP-7513) | mvn-deploy target fails | Major | build | Joep Rottinghuis | Joep Rottinghuis | +| [HADOOP-7450](https://issues.apache.org/jira/browse/HADOOP-7450) | Bump jetty to 6.1.26 | Blocker | build | Jitendra Nath Pandey | Konstantin Boudnik | +| [HADOOP-7390](https://issues.apache.org/jira/browse/HADOOP-7390) | VersionInfo not generated properly in git after unsplit | Minor | build | Thomas Graves | Todd Lipcon | +| [HADOOP-7351](https://issues.apache.org/jira/browse/HADOOP-7351) | Regression: HttpServer#getWebAppsPath used to be protected so subclasses could supply alternate webapps path but it was made private by HADOOP-6461 | Major | . | stack | stack | +| [HADOOP-7349](https://issues.apache.org/jira/browse/HADOOP-7349) | HADOOP-7121 accidentally disabled some tests | Major | ipc, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-7318](https://issues.apache.org/jira/browse/HADOOP-7318) | MD5Hash factory should reset the digester it returns | Critical | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-7312](https://issues.apache.org/jira/browse/HADOOP-7312) | core-default.xml lists configuration version as 0.21 | Minor | conf | Todd Lipcon | Harsh J | +| [HADOOP-7302](https://issues.apache.org/jira/browse/HADOOP-7302) | webinterface.private.actions should not be in common | Major | documentation | Ari Rabkin | Ari Rabkin | +| [HADOOP-7300](https://issues.apache.org/jira/browse/HADOOP-7300) | Configuration methods that return collections are inconsistent about mutability | Major | conf | Todd Lipcon | Todd Lipcon | +| [HADOOP-7296](https://issues.apache.org/jira/browse/HADOOP-7296) | The FsPermission(FsPermission) constructor does not use the sticky bit | Minor | fs | Siddharth Seth | Siddharth Seth | +| [HADOOP-7287](https://issues.apache.org/jira/browse/HADOOP-7287) | Configuration deprecation mechanism doesn't work properly for GenericOptionsParser/Tools | Blocker | conf | Todd Lipcon | Aaron T. Myers | +| [HADOOP-7252](https://issues.apache.org/jira/browse/HADOOP-7252) | JUnit shows up as a compile time dependency | Minor | build, conf, test | Pony | Harsh J | +| [HADOOP-7245](https://issues.apache.org/jira/browse/HADOOP-7245) | FsConfig should use constants in CommonConfigurationKeys | Major | . | Tom White | Tom White | +| [HADOOP-7194](https://issues.apache.org/jira/browse/HADOOP-7194) | Potential Resource leak in IOUtils.java | Major | io | Devaraj K | Devaraj K | +| [HADOOP-7187](https://issues.apache.org/jira/browse/HADOOP-7187) | Socket Leak in org.apache.hadoop.metrics.ganglia.GangliaContext | Major | metrics | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7184](https://issues.apache.org/jira/browse/HADOOP-7184) | Remove deprecated local.cache.size from core-default.xml | Major | documentation, filecache | Todd Lipcon | Todd Lipcon | +| [HADOOP-7183](https://issues.apache.org/jira/browse/HADOOP-7183) | WritableComparator.get should not cache comparator objects | Blocker | . | Todd Lipcon | Tom White | +| [HADOOP-7174](https://issues.apache.org/jira/browse/HADOOP-7174) | null is displayed in the console,if the src path is invalid while doing copyToLocal operation from commandLine | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7172](https://issues.apache.org/jira/browse/HADOOP-7172) | SecureIO should not check owner on non-secure clusters that have no native support | Critical | io, security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7162](https://issues.apache.org/jira/browse/HADOOP-7162) | FsShell: call srcFs.listStatus(src) twice | Minor | fs | Alexey Diomin | Alexey Diomin | +| [HADOOP-7156](https://issues.apache.org/jira/browse/HADOOP-7156) | getpwuid\_r is not thread-safe on RHEL6 | Critical | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7146](https://issues.apache.org/jira/browse/HADOOP-7146) | RPC server leaks file descriptors | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7145](https://issues.apache.org/jira/browse/HADOOP-7145) | Configuration.getLocalPath should trim whitespace from the provided directories | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7140](https://issues.apache.org/jira/browse/HADOOP-7140) | IPC Reader threads do not stop when server stops | Critical | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7126](https://issues.apache.org/jira/browse/HADOOP-7126) | TestDFSShell fails in trunk | Major | . | Po Cheung | Po Cheung | +| [HADOOP-7122](https://issues.apache.org/jira/browse/HADOOP-7122) | Timed out shell commands leak Timer threads | Critical | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-7121](https://issues.apache.org/jira/browse/HADOOP-7121) | Exceptions while serializing IPC call response are not handled well | Critical | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7120](https://issues.apache.org/jira/browse/HADOOP-7120) | 200 new Findbugs warnings | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7118](https://issues.apache.org/jira/browse/HADOOP-7118) | NPE in Configuration.writeXml | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7104](https://issues.apache.org/jira/browse/HADOOP-7104) | Remove unnecessary DNS reverse lookups from RPC layer | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-7102](https://issues.apache.org/jira/browse/HADOOP-7102) | Remove "fs.ramfs.impl" field from core-deafult.xml | Major | conf | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-7100](https://issues.apache.org/jira/browse/HADOOP-7100) | Build broken by HADOOP-6811 | Blocker | build, contrib/cloud | Todd Lipcon | Todd Lipcon | +| [HADOOP-7097](https://issues.apache.org/jira/browse/HADOOP-7097) | java.library.path missing basedir | Blocker | build, native | Noah Watkins | Noah Watkins | +| [HADOOP-7094](https://issues.apache.org/jira/browse/HADOOP-7094) | hadoop.css got lost during project split | Blocker | build | Todd Lipcon | Konstantin Boudnik | +| [HADOOP-7093](https://issues.apache.org/jira/browse/HADOOP-7093) | Servlets should default to text/plain | Major | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-7091](https://issues.apache.org/jira/browse/HADOOP-7091) | reloginFromKeytab() should happen even if TGT can't be found | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-7089](https://issues.apache.org/jira/browse/HADOOP-7089) | Fix link resolution logic in hadoop-config.sh | Minor | scripts | Eli Collins | Eli Collins | +| [HADOOP-7087](https://issues.apache.org/jira/browse/HADOOP-7087) | SequenceFile.createWriter ignores FileSystem parameter | Blocker | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-7082](https://issues.apache.org/jira/browse/HADOOP-7082) | Configuration.writeXML should not hold lock while outputting | Critical | conf | Todd Lipcon | Todd Lipcon | +| [HADOOP-7070](https://issues.apache.org/jira/browse/HADOOP-7070) | JAAS configuration should delegate unknown application names to pre-existing configuration | Critical | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7068](https://issues.apache.org/jira/browse/HADOOP-7068) | Ivy resolve force mode should be turned off by default | Major | . | Luke Lu | Luke Lu | +| [HADOOP-7057](https://issues.apache.org/jira/browse/HADOOP-7057) | IOUtils.readFully and IOUtils.skipFully have typo in exception creation's message | Minor | util | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-7053](https://issues.apache.org/jira/browse/HADOOP-7053) | wrong FSNamesystem Audit logging setting in conf/log4j.properties | Minor | conf | Jingguo Yao | Jingguo Yao | +| [HADOOP-7052](https://issues.apache.org/jira/browse/HADOOP-7052) | misspelling of threshold in conf/log4j.properties | Major | conf | Jingguo Yao | Jingguo Yao | +| [HADOOP-7046](https://issues.apache.org/jira/browse/HADOOP-7046) | 1 Findbugs warning on trunk and branch-0.22 | Blocker | security | Nigel Daley | Po Cheung | +| [HADOOP-7038](https://issues.apache.org/jira/browse/HADOOP-7038) | saveVersion script includes an additional \r while running whoami under windows | Minor | build | Wang Xu | Wang Xu | +| [HADOOP-7028](https://issues.apache.org/jira/browse/HADOOP-7028) | ant eclipse does not include requisite ant.jar in the classpath | Minor | build | Patrick Angeles | Patrick Angeles | +| [HADOOP-7011](https://issues.apache.org/jira/browse/HADOOP-7011) | KerberosName.main(...) throws NPE | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7006](https://issues.apache.org/jira/browse/HADOOP-7006) | hadoop fs -getmerge does not work using codebase from trunk. | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-6993](https://issues.apache.org/jira/browse/HADOOP-6993) | Broken link on cluster setup page of docs | Major | documentation | Aaron T. Myers | Eli Collins | +| [HADOOP-6991](https://issues.apache.org/jira/browse/HADOOP-6991) | SequenceFile::Reader discards length for files, does not call openFile | Minor | . | Chris Douglas | Chris Douglas | +| [HADOOP-6989](https://issues.apache.org/jira/browse/HADOOP-6989) | TestSetFile is failing on trunk | Major | . | Jakob Homan | Chris Douglas | +| [HADOOP-6984](https://issues.apache.org/jira/browse/HADOOP-6984) | NPE from SequenceFile::Writer.CompressionCodecOption | Minor | io | Chris Douglas | Chris Douglas | +| [HADOOP-6975](https://issues.apache.org/jira/browse/HADOOP-6975) | integer overflow in S3InputStream for blocks \> 2GB | Major | . | Patrick Kling | Patrick Kling | +| [HADOOP-6970](https://issues.apache.org/jira/browse/HADOOP-6970) | SecurityAuth.audit should be generated under /build | Major | build | Konstantin Shvachko | Boris Shkolnik | +| [HADOOP-6965](https://issues.apache.org/jira/browse/HADOOP-6965) | Method in UGI to get Kerberos ticket. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6951](https://issues.apache.org/jira/browse/HADOOP-6951) | Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-6947](https://issues.apache.org/jira/browse/HADOOP-6947) | Kerberos relogin should set refreshKrb5Config to true | Major | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6940](https://issues.apache.org/jira/browse/HADOOP-6940) | RawLocalFileSystem's markSupported method misnamed markSupport | Minor | fs | Tom White | Tom White | +| [HADOOP-6938](https://issues.apache.org/jira/browse/HADOOP-6938) | ConnectionId.getRemotePrincipal() should check if security is enabled | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-6933](https://issues.apache.org/jira/browse/HADOOP-6933) | TestListFiles is flaky | Minor | test | Todd Lipcon | Todd Lipcon | +| [HADOOP-6932](https://issues.apache.org/jira/browse/HADOOP-6932) | Namenode start (init) fails because of invalid kerberos key, even when security set to "simple" | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6930](https://issues.apache.org/jira/browse/HADOOP-6930) | AvroRpcEngine doesn't work with generated Avro code | Major | ipc | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-6926](https://issues.apache.org/jira/browse/HADOOP-6926) | SocketInputStream incorrectly implements read() | Minor | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-6925](https://issues.apache.org/jira/browse/HADOOP-6925) | BZip2Codec incorrectly implements read() | Critical | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-6922](https://issues.apache.org/jira/browse/HADOOP-6922) | COMMON part of MAPREDUCE-1664 | Major | documentation, security | Ravi Gummadi | Ravi Gummadi | +| [HADOOP-6913](https://issues.apache.org/jira/browse/HADOOP-6913) | Circular initialization between UserGroupInformation and KerberosName | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-6907](https://issues.apache.org/jira/browse/HADOOP-6907) | Rpc client doesn't use the per-connection conf to figure out server's Kerberos principal | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-6906](https://issues.apache.org/jira/browse/HADOOP-6906) | FileContext copy() utility doesn't work with recursive copying of directories. | Major | fs | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-6900](https://issues.apache.org/jira/browse/HADOOP-6900) | FileSystem#listLocatedStatus should not throw generic RuntimeException to indicate error conditions | Major | . | Suresh Srinivas | Hairong Kuang | +| [HADOOP-6899](https://issues.apache.org/jira/browse/HADOOP-6899) | RawLocalFileSystem#setWorkingDir() does not work for relative names | Major | fs | Sanjay Radia | Sanjay Radia | +| [HADOOP-6898](https://issues.apache.org/jira/browse/HADOOP-6898) | FileSystem.copyToLocal creates files with 777 permissions | Blocker | fs, security | Todd Lipcon | Aaron T. Myers | +| [HADOOP-6888](https://issues.apache.org/jira/browse/HADOOP-6888) | Being able to close all cached FileSystem objects for a given UGI | Major | fs, security | Kan Zhang | Kan Zhang | +| [HADOOP-6885](https://issues.apache.org/jira/browse/HADOOP-6885) | Fix java doc warnings in Groups and RefreshUserMappingsProtocol | Major | security | Eli Collins | Eli Collins | +| [HADOOP-6873](https://issues.apache.org/jira/browse/HADOOP-6873) | using delegation token over hftp for long running clients (part of hdfs 1296) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6853](https://issues.apache.org/jira/browse/HADOOP-6853) | Common component of HDFS-1045 | Major | . | Jakob Homan | Jakob Homan | +| [HADOOP-6834](https://issues.apache.org/jira/browse/HADOOP-6834) | TFile.append compares initial key against null lastKey | Major | io | Ahad Rana | Hong Tang | +| [HADOOP-6833](https://issues.apache.org/jira/browse/HADOOP-6833) | IPC leaks call parameters when exceptions thrown | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-6815](https://issues.apache.org/jira/browse/HADOOP-6815) | refreshSuperUserGroupsConfiguration should use server side configuration for the refresh | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6812](https://issues.apache.org/jira/browse/HADOOP-6812) | fs.inmemory.size.mb not listed in conf. Cluster setup page gives wrong advice. | Major | documentation | Edward Capriolo | Chris Douglas | +| [HADOOP-6781](https://issues.apache.org/jira/browse/HADOOP-6781) | security audit log shouldn't have exception in it. | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6778](https://issues.apache.org/jira/browse/HADOOP-6778) | add isRunning() method to AbstractDelegationTokenSecretManager (for HDFS-1044) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6763](https://issues.apache.org/jira/browse/HADOOP-6763) | Remove verbose logging from the Groups class | Major | . | Owen O'Malley | Boris Shkolnik | +| [HADOOP-6758](https://issues.apache.org/jira/browse/HADOOP-6758) | MapFile.fix does not allow index interval definition | Major | . | Gianmarco De Francisci Morales | Gianmarco De Francisci Morales | +| [HADOOP-6756](https://issues.apache.org/jira/browse/HADOOP-6756) | Clean up and add documentation for configuration keys in CommonConfigurationKeys.java | Major | fs | Erik Steffl | Erik Steffl | +| [HADOOP-6747](https://issues.apache.org/jira/browse/HADOOP-6747) | TestNetUtils fails on Mac OS X | Major | . | Luke Lu | Todd Lipcon | +| [HADOOP-6724](https://issues.apache.org/jira/browse/HADOOP-6724) | IPC doesn't properly handle IOEs thrown by socket factory | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-6715](https://issues.apache.org/jira/browse/HADOOP-6715) | AccessControlList.toString() returns empty string when we set acl to "\*" | Major | security, util | Ravi Gummadi | Ravi Gummadi | +| [HADOOP-6706](https://issues.apache.org/jira/browse/HADOOP-6706) | Relogin behavior for RPC clients could be improved | Major | security | Devaraj Das | Devaraj Das | +| [HADOOP-6702](https://issues.apache.org/jira/browse/HADOOP-6702) | Incorrect exit codes for "dfs -chown", "dfs -chgrp" when input is given in wildcard format. | Minor | fs | Ravi Phulari | Ravi Phulari | +| [HADOOP-6682](https://issues.apache.org/jira/browse/HADOOP-6682) | NetUtils:normalizeHostName does not process hostnames starting with [a-f] correctly | Major | io | Jakob Homan | Jakob Homan | +| [HADOOP-6670](https://issues.apache.org/jira/browse/HADOOP-6670) | UserGroupInformation doesn't support use in hash tables | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-6669](https://issues.apache.org/jira/browse/HADOOP-6669) | zlib.compress.level ignored for DefaultCodec initialization | Minor | io | Koji Noguchi | Koji Noguchi | +| [HADOOP-6663](https://issues.apache.org/jira/browse/HADOOP-6663) | BlockDecompressorStream get EOF exception when decompressing the file compressed from empty file | Major | io | Kang Xiao | Kang Xiao | +| [HADOOP-6656](https://issues.apache.org/jira/browse/HADOOP-6656) | Security framework needs to renew Kerberos tickets while the process is running | Major | . | Owen O'Malley | Devaraj Das | +| [HADOOP-6652](https://issues.apache.org/jira/browse/HADOOP-6652) | ShellBasedUnixGroupsMapping shouldn't have a cache | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-6648](https://issues.apache.org/jira/browse/HADOOP-6648) | Credentials should ignore null tokens | Major | security | Owen O'Malley | Devaraj Das | +| [HADOOP-6642](https://issues.apache.org/jira/browse/HADOOP-6642) | Fix javac, javadoc, findbugs warnings | Major | . | Arun C Murthy | Po Cheung | +| [HADOOP-6620](https://issues.apache.org/jira/browse/HADOOP-6620) | NPE if renewer is passed as null in getDelegationToken | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6613](https://issues.apache.org/jira/browse/HADOOP-6613) | RPC server should check for version mismatch first | Major | ipc, security | Kan Zhang | Kan Zhang | +| [HADOOP-6612](https://issues.apache.org/jira/browse/HADOOP-6612) | Protocols RefreshUserToGroupMappingsProtocol and RefreshAuthorizationPolicyProtocol will fail with security enabled | Major | security | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-6536](https://issues.apache.org/jira/browse/HADOOP-6536) | FileUtil.fullyDelete(dir) behavior is not defined when we pass a symlink as the argument | Major | fs | Amareshwari Sriramadasu | Ravi Gummadi | +| [HADOOP-6496](https://issues.apache.org/jira/browse/HADOOP-6496) | HttpServer sends wrong content-type for CSS files (and others) | Minor | . | Lars Francke | Ivan Mitic | +| [HADOOP-6482](https://issues.apache.org/jira/browse/HADOOP-6482) | GenericOptionsParser constructor that takes Options and String[] ignores options | Minor | util | Chris Wilkes | Eli Collins | +| [HADOOP-6404](https://issues.apache.org/jira/browse/HADOOP-6404) | Rename the generated artifacts to common instead of core | Blocker | build | Owen O'Malley | Tom White | +| [HADOOP-6344](https://issues.apache.org/jira/browse/HADOOP-6344) | rm and rmr fail to correctly move the user's files to the trash prior to deleting when they are over quota. | Major | fs | gary murry | Jakob Homan | +| [HDFS-2573](https://issues.apache.org/jira/browse/HDFS-2573) | TestFiDataXceiverServer is failing, not testing OOME | Major | datanode, test | Konstantin Shvachko | Konstantin Boudnik | +| [HDFS-2514](https://issues.apache.org/jira/browse/HDFS-2514) | Link resolution bug for intermediate symlinks with relative targets | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-2491](https://issues.apache.org/jira/browse/HDFS-2491) | TestBalancer can fail when datanode utilization and avgUtilization is exactly same. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2452](https://issues.apache.org/jira/browse/HDFS-2452) | OutOfMemoryError in DataXceiverServer takes down the DataNode | Major | datanode | Konstantin Shvachko | Uma Maheswara Rao G | +| [HDFS-2388](https://issues.apache.org/jira/browse/HDFS-2388) | Remove dependency on different version of slf4j in avro | Major | build | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2383](https://issues.apache.org/jira/browse/HDFS-2383) | TestDfsOverAvroRpc is failing on 0.22 | Blocker | test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2346](https://issues.apache.org/jira/browse/HDFS-2346) | TestHost2NodesMap & TestReplicasMap will fail depending upon execution order of test methods | Blocker | test | Uma Maheswara Rao G | Laxman | +| [HDFS-2343](https://issues.apache.org/jira/browse/HDFS-2343) | Make hdfs use same version of avro as HBase | Blocker | hdfs-client | Joep Rottinghuis | Joep Rottinghuis | +| [HDFS-2341](https://issues.apache.org/jira/browse/HDFS-2341) | Contribs not building | Blocker | build | Joep Rottinghuis | Joep Rottinghuis | +| [HDFS-2315](https://issues.apache.org/jira/browse/HDFS-2315) | Build fails with ant 1.7.0 but works with 1.8.0 | Blocker | build | Joep Rottinghuis | Joep Rottinghuis | +| [HDFS-2297](https://issues.apache.org/jira/browse/HDFS-2297) | FindBugs OutOfMemoryError | Blocker | build | Joep Rottinghuis | Joep Rottinghuis | +| [HDFS-2290](https://issues.apache.org/jira/browse/HDFS-2290) | Block with corrupt replica is not getting replicated | Major | namenode | Konstantin Shvachko | Benoy Antony | +| [HDFS-2287](https://issues.apache.org/jira/browse/HDFS-2287) | TestParallelRead has a small off-by-one bug | Trivial | test | Todd Lipcon | Todd Lipcon | +| [HDFS-2281](https://issues.apache.org/jira/browse/HDFS-2281) | NPE in checkpoint during processIOError() | Major | namenode | Konstantin Shvachko | Uma Maheswara Rao G | +| [HDFS-2280](https://issues.apache.org/jira/browse/HDFS-2280) | BackupNode fails with MD5 checksum Exception during checkpoint if BN's image is outdated. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2271](https://issues.apache.org/jira/browse/HDFS-2271) | startJournalSpool should invoke ProcessIOError with failed storage directories if createEditLogFile throws any exception. | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2258](https://issues.apache.org/jira/browse/HDFS-2258) | TestLeaseRecovery2 fails as lease hard limit is not reset to default | Major | namenode, test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2232](https://issues.apache.org/jira/browse/HDFS-2232) | TestHDFSCLI fails on 0.22 branch | Blocker | test | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-2214](https://issues.apache.org/jira/browse/HDFS-2214) | Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT | Major | build | Joep Rottinghuis | Joep Rottinghuis | +| [HDFS-2211](https://issues.apache.org/jira/browse/HDFS-2211) | Build does not pass along properties to contrib builds | Blocker | build | Joep Rottinghuis | Joep Rottinghuis | +| [HDFS-2189](https://issues.apache.org/jira/browse/HDFS-2189) | guava-r09 dependency missing from "ivy/hadoop-hdfs-template.xml" in HDFS. | Blocker | . | Plamen Jeliazkov | Joep Rottinghuis | +| [HDFS-2071](https://issues.apache.org/jira/browse/HDFS-2071) | Use of isConnected() in DataXceiver is invalid | Minor | datanode | Kihwal Lee | Kihwal Lee | +| [HDFS-2012](https://issues.apache.org/jira/browse/HDFS-2012) | Recurring failure of TestBalancer due to incorrect treatment of nodes whose utilization equals avgUtilization. | Blocker | balancer & mover, test | Aaron T. Myers | Uma Maheswara Rao G | +| [HDFS-2002](https://issues.apache.org/jira/browse/HDFS-2002) | Incorrect computation of needed blocks in getTurnOffTip() | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-2000](https://issues.apache.org/jira/browse/HDFS-2000) | Missing deprecation for io.bytes.per.checksum | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1981](https://issues.apache.org/jira/browse/HDFS-1981) | When namenode goes down while checkpointing and if is started again subsequent Checkpointing is always failing | Blocker | namenode | ramkrishna.s.vasudevan | Uma Maheswara Rao G | +| [HDFS-1978](https://issues.apache.org/jira/browse/HDFS-1978) | All but first option in LIBHDFS\_OPTS is ignored | Major | libhdfs | Brock Noland | Eli Collins | +| [HDFS-1969](https://issues.apache.org/jira/browse/HDFS-1969) | Running rollback on new-version namenode destroys namespace | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1965](https://issues.apache.org/jira/browse/HDFS-1965) | IPCs done using block token-based tickets can't reuse connections | Critical | security | Todd Lipcon | Todd Lipcon | +| [HDFS-1964](https://issues.apache.org/jira/browse/HDFS-1964) | Incorrect HTML unescaping in DatanodeJspHelper.java | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1952](https://issues.apache.org/jira/browse/HDFS-1952) | FSEditLog.open() appears to succeed even if all EDITS directories fail | Major | . | Matt Foley | Andrew | +| [HDFS-1943](https://issues.apache.org/jira/browse/HDFS-1943) | fail to start datanode while start-dfs.sh is executed by root user | Blocker | scripts | Wei Yongjun | Matt Foley | +| [HDFS-1936](https://issues.apache.org/jira/browse/HDFS-1936) | Updating the layout version from HDFS-1822 causes upgrade problems. | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1932](https://issues.apache.org/jira/browse/HDFS-1932) | Ensure that HDFS configuration deprecations are set up in every spot that HDFS configurations are loaded. | Critical | . | Todd Lipcon | Jolly Chen | +| [HDFS-1925](https://issues.apache.org/jira/browse/HDFS-1925) | SafeModeInfo should use DFS\_NAMENODE\_SAFEMODE\_THRESHOLD\_PCT\_DEFAULT instead of 0.95 | Major | . | Konstantin Shvachko | Joey Echeverria | +| [HDFS-1921](https://issues.apache.org/jira/browse/HDFS-1921) | Save namespace can cause NN to be unable to come up on restart | Blocker | . | Aaron T. Myers | Matt Foley | +| [HDFS-1909](https://issues.apache.org/jira/browse/HDFS-1909) | TestHDFSCLI fails due to typo in expected output | Major | . | Tom White | Tom White | +| [HDFS-1897](https://issues.apache.org/jira/browse/HDFS-1897) | Documention refers to removed option dfs.network.script | Minor | documentation | Ari Rabkin | Andrew Whang | +| [HDFS-1891](https://issues.apache.org/jira/browse/HDFS-1891) | TestBackupNode fails intermittently | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1850](https://issues.apache.org/jira/browse/HDFS-1850) | DN should transmit absolute failed volume count rather than increments to the NN | Major | datanode, namenode | Eli Collins | Eli Collins | +| [HDFS-1845](https://issues.apache.org/jira/browse/HDFS-1845) | symlink comes up as directory after namenode restart | Major | . | John George | John George | +| [HDFS-1823](https://issues.apache.org/jira/browse/HDFS-1823) | start-dfs.sh script fails if HADOOP\_HOME is not set | Blocker | scripts | Tom White | Tom White | +| [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1821](https://issues.apache.org/jira/browse/HDFS-1821) | FileContext.createSymlink with kerberos enabled sets wrong owner | Major | . | John George | John George | +| [HDFS-1806](https://issues.apache.org/jira/browse/HDFS-1806) | TestBlockReport.blockReport\_08() and \_09() are timing-dependent and likely to fail on fast servers | Major | datanode, namenode | Matt Foley | Matt Foley | +| [HDFS-1786](https://issues.apache.org/jira/browse/HDFS-1786) | Some cli test cases expect a "null" message | Minor | test | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-1782](https://issues.apache.org/jira/browse/HDFS-1782) | FSNamesystem.startFileInternal(..) throws NullPointerException | Major | namenode | John George | John George | +| [HDFS-1781](https://issues.apache.org/jira/browse/HDFS-1781) | jsvc executable delivered into wrong package... | Major | scripts | John George | John George | +| [HDFS-1750](https://issues.apache.org/jira/browse/HDFS-1750) | fs -ls hftp://file not working | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1728](https://issues.apache.org/jira/browse/HDFS-1728) | SecondaryNameNode.checkpointSize is in byte but not MB. | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1665](https://issues.apache.org/jira/browse/HDFS-1665) | Balancer sleeps inadequately | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1625](https://issues.apache.org/jira/browse/HDFS-1625) | TestDataNodeMXBean fails if disk space usage changes during test run | Minor | test | Todd Lipcon | Tsz Wo Nicholas Sze | +| [HDFS-1621](https://issues.apache.org/jira/browse/HDFS-1621) | Fix references to hadoop-common-${version} in build.xml | Major | . | Todd Lipcon | Jolly Chen | +| [HDFS-1615](https://issues.apache.org/jira/browse/HDFS-1615) | seek() on closed DFS input stream throws NPE | Major | . | Todd Lipcon | Scott Carey | +| [HDFS-1612](https://issues.apache.org/jira/browse/HDFS-1612) | HDFS Design Documentation is outdated | Minor | documentation | Joe Crobak | Joe Crobak | +| [HDFS-1602](https://issues.apache.org/jira/browse/HDFS-1602) | NameNode storage failed replica restoration is broken | Major | namenode | Konstantin Boudnik | Boris Shkolnik | +| [HDFS-1598](https://issues.apache.org/jira/browse/HDFS-1598) | ListPathsServlet excludes .\*.crc files | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1597](https://issues.apache.org/jira/browse/HDFS-1597) | Batched edit log syncs can reset synctxid throw assertions | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1591](https://issues.apache.org/jira/browse/HDFS-1591) | Fix javac, javadoc, findbugs warnings | Major | . | Po Cheung | Po Cheung | +| [HDFS-1575](https://issues.apache.org/jira/browse/HDFS-1575) | viewing block from web UI broken | Blocker | . | Todd Lipcon | Aaron T. Myers | +| [HDFS-1572](https://issues.apache.org/jira/browse/HDFS-1572) | Checkpointer should trigger checkpoint with specified period. | Blocker | . | Liyin Liang | Jakob Homan | +| [HDFS-1561](https://issues.apache.org/jira/browse/HDFS-1561) | BackupNode listens on default host | Blocker | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1550](https://issues.apache.org/jira/browse/HDFS-1550) | NPE when listing a file with no location | Blocker | hdfs-client | Hairong Kuang | Hairong Kuang | +| [HDFS-1544](https://issues.apache.org/jira/browse/HDFS-1544) | Ivy resolve force mode should be turned off by default | Major | . | Luke Lu | Luke Lu | +| [HDFS-1542](https://issues.apache.org/jira/browse/HDFS-1542) | Deadlock in Configuration.writeXml when serialized form is larger than one DFS block | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-1532](https://issues.apache.org/jira/browse/HDFS-1532) | Exclude Findbugs warning in FSImageFormat$Saver | Major | test | Todd Lipcon | Todd Lipcon | +| [HDFS-1531](https://issues.apache.org/jira/browse/HDFS-1531) | Clean up stack traces due to duplicate MXBean registration | Trivial | datanode, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1529](https://issues.apache.org/jira/browse/HDFS-1529) | Incorrect handling of interrupts in waitForAckedSeqno can cause deadlock | Blocker | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-1527](https://issues.apache.org/jira/browse/HDFS-1527) | SocketOutputStream.transferToFully fails for blocks \>= 2GB on 32 bit JVM | Major | datanode | Patrick Kling | Patrick Kling | +| [HDFS-1524](https://issues.apache.org/jira/browse/HDFS-1524) | Image loader should make sure to read every byte in image file | Blocker | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1523](https://issues.apache.org/jira/browse/HDFS-1523) | TestLargeBlock is failing on trunk | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1511](https://issues.apache.org/jira/browse/HDFS-1511) | 98 Release Audit warnings on trunk and branch-0.22 | Blocker | . | Nigel Daley | Jakob Homan | +| [HDFS-1507](https://issues.apache.org/jira/browse/HDFS-1507) | TestAbandonBlock should abandon a block | Minor | test | Eli Collins | Eli Collins | +| [HDFS-1505](https://issues.apache.org/jira/browse/HDFS-1505) | saveNamespace appears to succeed even if all directories fail to save | Blocker | . | Todd Lipcon | Aaron T. Myers | +| [HDFS-1504](https://issues.apache.org/jira/browse/HDFS-1504) | FSImageSaver should catch all exceptions, not just IOE | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1503](https://issues.apache.org/jira/browse/HDFS-1503) | TestSaveNamespace fails | Minor | test | Eli Collins | Todd Lipcon | +| [HDFS-1502](https://issues.apache.org/jira/browse/HDFS-1502) | TestBlockRecovery triggers NPE in assert | Minor | . | Eli Collins | Hairong Kuang | +| [HDFS-1500](https://issues.apache.org/jira/browse/HDFS-1500) | TestOfflineImageViewer failing on trunk | Major | test, tools | Todd Lipcon | Todd Lipcon | +| [HDFS-1498](https://issues.apache.org/jira/browse/HDFS-1498) | FSDirectory#unprotectedConcat calls setModificationTime on a file | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-1487](https://issues.apache.org/jira/browse/HDFS-1487) | FSDirectory.removeBlock() should update diskspace count of the block owner node | Major | namenode | Zhong Wang | Zhong Wang | +| [HDFS-1483](https://issues.apache.org/jira/browse/HDFS-1483) | DFSClient.getBlockLocations returns BlockLocations with no indication that the corresponding blocks are corrupt | Major | hdfs-client | Patrick Kling | Patrick Kling | +| [HDFS-1467](https://issues.apache.org/jira/browse/HDFS-1467) | Append pipeline never succeeds with more than one replica | Blocker | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1466](https://issues.apache.org/jira/browse/HDFS-1466) | TestFcHdfsSymlink relies on /tmp/test not existing | Minor | test | Todd Lipcon | Eli Collins | +| [HDFS-1440](https://issues.apache.org/jira/browse/HDFS-1440) | TestComputeInvalidateWork fails intermittently | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1433](https://issues.apache.org/jira/browse/HDFS-1433) | Fix test failures - TestPread and TestFileLimit | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1409](https://issues.apache.org/jira/browse/HDFS-1409) | The "register" method of the BackupNode class should be "UnsupportedActionException("register")" | Trivial | namenode | Ching-Shen Chen | Ching-Shen Chen | +| [HDFS-1399](https://issues.apache.org/jira/browse/HDFS-1399) | Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-1364](https://issues.apache.org/jira/browse/HDFS-1364) | HFTP client should support relogin from keytab | Major | security | Kan Zhang | Jitendra Nath Pandey | +| [HDFS-1361](https://issues.apache.org/jira/browse/HDFS-1361) | Add -fileStatus operation to NNThroughputBenchmark | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1357](https://issues.apache.org/jira/browse/HDFS-1357) | HFTP traffic served by DataNode shouldn't use service port on NameNode | Major | datanode, security | Kan Zhang | Kan Zhang | +| [HDFS-1355](https://issues.apache.org/jira/browse/HDFS-1355) | ant veryclean (clean-cache) doesn't clean enough | Major | build | Luke Lu | Luke Lu | +| [HDFS-1352](https://issues.apache.org/jira/browse/HDFS-1352) | Fix jsvc.location | Major | build | Eli Collins | Eli Collins | +| [HDFS-1349](https://issues.apache.org/jira/browse/HDFS-1349) | Remove empty java files | Major | . | Tom White | Eli Collins | +| [HDFS-1347](https://issues.apache.org/jira/browse/HDFS-1347) | TestDelegationToken uses mortbay.log for logging | Major | test | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1340](https://issues.apache.org/jira/browse/HDFS-1340) | A null delegation token is appended to the url if security is disabled when browsing filesystem. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1334](https://issues.apache.org/jira/browse/HDFS-1334) | open in HftpFileSystem does not add delegation tokens to the url. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1317](https://issues.apache.org/jira/browse/HDFS-1317) | HDFSProxy needs additional changes to work after changes to streamFile servlet in HDFS-1109 | Major | contrib/hdfsproxy | Rohini Palaniswamy | Rohini Palaniswamy | +| [HDFS-1308](https://issues.apache.org/jira/browse/HDFS-1308) | job conf key for the services name of DelegationToken for HFTP url is constructed incorrectly in HFTPFileSystem (part of MR-1718) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1301](https://issues.apache.org/jira/browse/HDFS-1301) | TestHDFSProxy need to use server side conf for ProxyUser stuff. | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1296](https://issues.apache.org/jira/browse/HDFS-1296) | using delegation token over hftp for long running clients (spawn from hdfs-1007). | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1289](https://issues.apache.org/jira/browse/HDFS-1289) | Datanode secure mode is broken | Major | datanode | Kan Zhang | Kan Zhang | +| [HDFS-1284](https://issues.apache.org/jira/browse/HDFS-1284) | TestBlockToken fails | Major | test | Konstantin Shvachko | Kan Zhang | +| [HDFS-1283](https://issues.apache.org/jira/browse/HDFS-1283) | ant eclipse-files has drifted again | Major | build | Jakob Homan | Jakob Homan | +| [HDFS-1258](https://issues.apache.org/jira/browse/HDFS-1258) | Clearing namespace quota on "/" corrupts FS image | Blocker | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1250](https://issues.apache.org/jira/browse/HDFS-1250) | Namenode accepts block report from dead datanodes | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1206](https://issues.apache.org/jira/browse/HDFS-1206) | TestFiHFlush fails intermittently | Major | test | Tsz Wo Nicholas Sze | Konstantin Boudnik | +| [HDFS-1202](https://issues.apache.org/jira/browse/HDFS-1202) | DataBlockScanner throws NPE when updated before initialized | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1198](https://issues.apache.org/jira/browse/HDFS-1198) | Resolving cross-realm principals | Major | namenode | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1192](https://issues.apache.org/jira/browse/HDFS-1192) | refreshSuperUserGroupsConfiguration should use server side configuration for the refresh (for HADOOP-6815) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1189](https://issues.apache.org/jira/browse/HDFS-1189) | Quota counts missed between clear quota and set quota | Major | namenode | Kang Xiao | John George | +| [HDFS-1164](https://issues.apache.org/jira/browse/HDFS-1164) | TestHdfsProxy is failing | Major | contrib/hdfsproxy | Eli Collins | Todd Lipcon | +| [HDFS-1163](https://issues.apache.org/jira/browse/HDFS-1163) | normalize property names for JT/NN kerberos principal names in configuration (from HADOOP 6633) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1157](https://issues.apache.org/jira/browse/HDFS-1157) | Modifications introduced by HDFS-1150 are breaking aspect's bindings | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1146](https://issues.apache.org/jira/browse/HDFS-1146) | Javadoc for getDelegationTokenSecretManager in FSNamesystem | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1145](https://issues.apache.org/jira/browse/HDFS-1145) | When NameNode is shutdown it tries to exit safemode | Major | namenode | dhruba borthakur | dhruba borthakur | +| [HDFS-1141](https://issues.apache.org/jira/browse/HDFS-1141) | completeFile does not check lease ownership | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1138](https://issues.apache.org/jira/browse/HDFS-1138) | Modification times are being overwritten when FSImage loads | Major | . | Dmytro Molkov | Dmytro Molkov | +| [HDFS-1130](https://issues.apache.org/jira/browse/HDFS-1130) | Pass Administrator acl to HTTPServer for common servlet access. | Major | security | Amareshwari Sriramadasu | Devaraj Das | +| [HDFS-1118](https://issues.apache.org/jira/browse/HDFS-1118) | DFSOutputStream socket leak when cannot connect to DataNode | Major | . | Zheng Shao | Zheng Shao | +| [HDFS-1112](https://issues.apache.org/jira/browse/HDFS-1112) | Edit log buffer should not grow unboundedly | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1085](https://issues.apache.org/jira/browse/HDFS-1085) | hftp read failing silently | Major | datanode | Koji Noguchi | Tsz Wo Nicholas Sze | +| [HDFS-1045](https://issues.apache.org/jira/browse/HDFS-1045) | In secure clusters, re-login is necessary for https clients before opening connections | Major | security | Jakob Homan | Jakob Homan | +| [HDFS-1044](https://issues.apache.org/jira/browse/HDFS-1044) | Cannot submit mapreduce job from secure client to unsecure sever | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1039](https://issues.apache.org/jira/browse/HDFS-1039) | Service should be set in the token in JspHelper.getUGI | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1038](https://issues.apache.org/jira/browse/HDFS-1038) | In nn\_browsedfscontent.jsp fetch delegation token only if security is enabled. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1036](https://issues.apache.org/jira/browse/HDFS-1036) | in DelegationTokenFetch dfs.getURI returns no port | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1027](https://issues.apache.org/jira/browse/HDFS-1027) | Update year to 2010. | Trivial | . | Ravi Phulari | Ravi Phulari | +| [HDFS-1021](https://issues.apache.org/jira/browse/HDFS-1021) | specify correct server principal for RefreshAuthorizationPolicyProtocol and RefreshUserToGroupMappingsProtocol protocols in DFSAdmin (for HADOOP-6612) | Major | security | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1019](https://issues.apache.org/jira/browse/HDFS-1019) | Incorrect default values for delegation tokens in hdfs-default.xml | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1017](https://issues.apache.org/jira/browse/HDFS-1017) | browsedfs jsp should call JspHelper.getUGI rather than using createRemoteUser() | Major | security | Jakob Homan | Jakob Homan | +| [HDFS-1007](https://issues.apache.org/jira/browse/HDFS-1007) | HFTP needs to be updated to use delegation tokens | Major | security | Devaraj Das | Devaraj Das | +| [HDFS-1006](https://issues.apache.org/jira/browse/HDFS-1006) | getImage/putImage http requests should be https for the case of security enabled. | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1001](https://issues.apache.org/jira/browse/HDFS-1001) | DataXceiver and BlockReader disagree on when to send/recv CHECKSUM\_OK | Minor | datanode | bc Wong | bc Wong | +| [HDFS-988](https://issues.apache.org/jira/browse/HDFS-988) | saveNamespace race can corrupt the edits log | Blocker | namenode | dhruba borthakur | Eli Collins | +| [HDFS-977](https://issues.apache.org/jira/browse/HDFS-977) | DataNode.createInterDataNodeProtocolProxy() guards a log at the wrong level | Trivial | datanode | Steve Loughran | Harsh J | +| [HDFS-970](https://issues.apache.org/jira/browse/HDFS-970) | FSImage writing should always fsync before close | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-939](https://issues.apache.org/jira/browse/HDFS-939) | libhdfs test is broken | Blocker | libhdfs | Eli Collins | Eli Collins | +| [HDFS-912](https://issues.apache.org/jira/browse/HDFS-912) | sed in build.xml fails | Minor | . | Allen Wittenauer | Allen Wittenauer | +| [HDFS-900](https://issues.apache.org/jira/browse/HDFS-900) | Corrupt replicas are not tracked correctly through block report from DN | Blocker | . | Todd Lipcon | Konstantin Shvachko | +| [HDFS-874](https://issues.apache.org/jira/browse/HDFS-874) | TestHDFSFileContextMainOperations fails on weirdly configured DNS hosts | Major | hdfs-client, test | Todd Lipcon | Todd Lipcon | +| [HDFS-829](https://issues.apache.org/jira/browse/HDFS-829) | hdfsJniHelper.c: #include \ is not portable | Major | . | Allen Wittenauer | Allen Wittenauer | +| [HDFS-727](https://issues.apache.org/jira/browse/HDFS-727) | bug setting block size hdfsOpenFile | Blocker | libhdfs | Eli Collins | Eli Collins | +| [HDFS-671](https://issues.apache.org/jira/browse/HDFS-671) | Documentation change for updated configuration keys. | Blocker | . | Jitendra Nath Pandey | Tom White | +| [HDFS-613](https://issues.apache.org/jira/browse/HDFS-613) | TestBalancer and TestBlockTokenWithDFS fail Balancer assert | Major | test | Konstantin Shvachko | Todd Lipcon | +| [HDFS-96](https://issues.apache.org/jira/browse/HDFS-96) | HDFS does not support blocks greater than 2GB | Major | . | dhruba borthakur | Patrick Kling | +| [MAPREDUCE-3438](https://issues.apache.org/jira/browse/MAPREDUCE-3438) | TestRaidNode fails because of "Too many open files" | Major | contrib/raid | Konstantin Shvachko | Ramkumar Vadali | +| [MAPREDUCE-3429](https://issues.apache.org/jira/browse/MAPREDUCE-3429) | Few contrib tests are failing because of the missing commons-lang dependency | Major | capacity-sched, contrib/gridmix | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-3151](https://issues.apache.org/jira/browse/MAPREDUCE-3151) | Contrib tests failing | Major | contrib/vertica | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-3139](https://issues.apache.org/jira/browse/MAPREDUCE-3139) | SlivePartitioner generates negative partitions | Major | test | Konstantin Shvachko | Jakob Homan | +| [MAPREDUCE-3138](https://issues.apache.org/jira/browse/MAPREDUCE-3138) | Allow for applications to deal with MAPREDUCE-954 | Blocker | client, mrv2 | Arun C Murthy | Owen O'Malley | +| [MAPREDUCE-3088](https://issues.apache.org/jira/browse/MAPREDUCE-3088) | Clover 2.4.3 breaks build for 0.22 branch | Major | build | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-3039](https://issues.apache.org/jira/browse/MAPREDUCE-3039) | Make mapreduce use same version of avro as HBase | Major | capacity-sched, contrib/fair-share, contrib/gridmix, contrib/mrunit, contrib/mumak, contrib/raid, contrib/streaming, jobhistoryserver | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-3026](https://issues.apache.org/jira/browse/MAPREDUCE-3026) | When user adds hierarchical queues to the cluster, mapred queue -list returns NULL Pointer Exception | Major | jobtracker | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-3025](https://issues.apache.org/jira/browse/MAPREDUCE-3025) | Contribs not building | Blocker | build | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-2991](https://issues.apache.org/jira/browse/MAPREDUCE-2991) | queueinfo.jsp fails to show queue status if any Capacity scheduler queue name has dash/hiphen in it. | Major | scheduler | Priyo Mustafi | Priyo Mustafi | +| [MAPREDUCE-2948](https://issues.apache.org/jira/browse/MAPREDUCE-2948) | Hadoop streaming test failure, post MR-2767 | Major | contrib/streaming | Milind Bhandarkar | Mahadev konar | +| [MAPREDUCE-2940](https://issues.apache.org/jira/browse/MAPREDUCE-2940) | Build fails with ant 1.7.0 but works with 1.8.0 | Major | build | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-2936](https://issues.apache.org/jira/browse/MAPREDUCE-2936) | Contrib Raid compilation broken after HDFS-1620 | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2779](https://issues.apache.org/jira/browse/MAPREDUCE-2779) | JobSplitWriter.java can't handle large job.split file | Major | job submission | Ming Ma | Ming Ma | +| [MAPREDUCE-2767](https://issues.apache.org/jira/browse/MAPREDUCE-2767) | Remove Linux task-controller from 0.22 branch | Blocker | security | Milind Bhandarkar | Milind Bhandarkar | +| [MAPREDUCE-2753](https://issues.apache.org/jira/browse/MAPREDUCE-2753) | Generated POMs hardcode dependency on hadoop-common version 0.22.0-SNAPSHOT | Major | build | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-2752](https://issues.apache.org/jira/browse/MAPREDUCE-2752) | Build does not pass along properties to contrib builds | Minor | build | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-2571](https://issues.apache.org/jira/browse/MAPREDUCE-2571) | CombineFileInputFormat.getSplits throws a java.lang.ArrayStoreException | Blocker | . | Bochun Bai | Bochun Bai | +| [MAPREDUCE-2531](https://issues.apache.org/jira/browse/MAPREDUCE-2531) | org.apache.hadoop.mapred.jobcontrol.getAssignedJobID throw class cast exception | Blocker | client | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2516](https://issues.apache.org/jira/browse/MAPREDUCE-2516) | option to control sensitive web actions | Minor | . | Ari Rabkin | Ari Rabkin | +| [MAPREDUCE-2515](https://issues.apache.org/jira/browse/MAPREDUCE-2515) | MapReduce references obsolete options | Major | jobtracker | Ari Rabkin | Ari Rabkin | +| [MAPREDUCE-2487](https://issues.apache.org/jira/browse/MAPREDUCE-2487) | ChainReducer uses MAPPER\_BY\_VALUE instead of REDUCER\_BY\_VALUE | Minor | . | Forrest Vines | Devaraj K | +| [MAPREDUCE-2486](https://issues.apache.org/jira/browse/MAPREDUCE-2486) | 0.22 - snapshot incorrect dependency published in .pom files | Blocker | . | Dmitriy V. Ryaboy | Todd Lipcon | +| [MAPREDUCE-2472](https://issues.apache.org/jira/browse/MAPREDUCE-2472) | Extra whitespace in mapred.child.java.opts breaks JVM initialization | Major | task-controller | Todd Lipcon | Aaron T. Myers | +| [MAPREDUCE-2457](https://issues.apache.org/jira/browse/MAPREDUCE-2457) | job submission should inject group.name (on the JT side) | Critical | jobtracker | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-2448](https://issues.apache.org/jira/browse/MAPREDUCE-2448) | NoSuchMethodError: org.apache.hadoop.hdfs.TestDatanodeBlockScanner.corruptReplica(..) | Minor | contrib/raid, test | Tsz Wo Nicholas Sze | Eli Collins | +| [MAPREDUCE-2445](https://issues.apache.org/jira/browse/MAPREDUCE-2445) | TestMiniMRWithDFSWithDistinctUsers is very broken | Major | security, test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2437](https://issues.apache.org/jira/browse/MAPREDUCE-2437) | SLive should process only part\* files while generating the report. | Blocker | test | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-2428](https://issues.apache.org/jira/browse/MAPREDUCE-2428) | start-mapred.sh script fails if HADOOP\_HOME is not set | Blocker | . | Tom White | Tom White | +| [MAPREDUCE-2394](https://issues.apache.org/jira/browse/MAPREDUCE-2394) | JUnit output format doesn't propagate into some contrib builds | Blocker | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2392](https://issues.apache.org/jira/browse/MAPREDUCE-2392) | TaskTracker shutdown in the tests sometimes take 60s | Major | . | Tom White | Tom White | +| [MAPREDUCE-2336](https://issues.apache.org/jira/browse/MAPREDUCE-2336) | Tool-related packages should be in the Tool javadoc group | Major | documentation | Tom White | Tom White | +| [MAPREDUCE-2327](https://issues.apache.org/jira/browse/MAPREDUCE-2327) | MapTask doesn't need to put username information in SpillRecord | Blocker | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2317](https://issues.apache.org/jira/browse/MAPREDUCE-2317) | HadoopArchives throwing NullPointerException while creating hadoop archives (.har files) | Minor | harchive | Devaraj K | Devaraj K | +| [MAPREDUCE-2315](https://issues.apache.org/jira/browse/MAPREDUCE-2315) | javadoc is failing in nightly | Blocker | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2311](https://issues.apache.org/jira/browse/MAPREDUCE-2311) | TestFairScheduler failing on trunk | Blocker | contrib/fair-share | Todd Lipcon | Scott Chen | +| [MAPREDUCE-2304](https://issues.apache.org/jira/browse/MAPREDUCE-2304) | TestMRCLI fails when hostname has a hyphen (-) | Minor | test | Priyo Mustafi | Priyo Mustafi | +| [MAPREDUCE-2285](https://issues.apache.org/jira/browse/MAPREDUCE-2285) | MiniMRCluster does not start after ant test-patch | Blocker | test | Ramkumar Vadali | Todd Lipcon | +| [MAPREDUCE-2284](https://issues.apache.org/jira/browse/MAPREDUCE-2284) | TestLocalRunner.testMultiMaps times out | Critical | test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2283](https://issues.apache.org/jira/browse/MAPREDUCE-2283) | TestBlockFixer hangs initializing MiniMRCluster | Blocker | contrib/raid | Nigel Daley | Ramkumar Vadali | +| [MAPREDUCE-2282](https://issues.apache.org/jira/browse/MAPREDUCE-2282) | MapReduce tests don't compile following HDFS-1561 | Blocker | test | Tom White | Konstantin Shvachko | +| [MAPREDUCE-2281](https://issues.apache.org/jira/browse/MAPREDUCE-2281) | Fix javac, javadoc, findbugs warnings | Major | . | Po Cheung | Po Cheung | +| [MAPREDUCE-2277](https://issues.apache.org/jira/browse/MAPREDUCE-2277) | TestCapacitySchedulerWithJobTracker fails sometimes | Minor | capacity-sched | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2272](https://issues.apache.org/jira/browse/MAPREDUCE-2272) | Job ACL file should not be executable | Trivial | tasktracker | Todd Lipcon | Harsh J | +| [MAPREDUCE-2256](https://issues.apache.org/jira/browse/MAPREDUCE-2256) | FairScheduler fairshare preemption from multiple pools may preempt all tasks from one pool causing that pool to go below fairshare. | Major | contrib/fair-share | Priyo Mustafi | Priyo Mustafi | +| [MAPREDUCE-2253](https://issues.apache.org/jira/browse/MAPREDUCE-2253) | Servlets should specify content type | Critical | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2251](https://issues.apache.org/jira/browse/MAPREDUCE-2251) | Remove mapreduce.job.userhistorylocation config | Major | . | Todd Lipcon | Harsh J | +| [MAPREDUCE-2238](https://issues.apache.org/jira/browse/MAPREDUCE-2238) | Undeletable build directories | Critical | build, test | Eli Collins | Todd Lipcon | +| [MAPREDUCE-2224](https://issues.apache.org/jira/browse/MAPREDUCE-2224) | Synchronization bugs in JvmManager | Critical | tasktracker | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2222](https://issues.apache.org/jira/browse/MAPREDUCE-2222) | Ivy resolve force mode should be turned off by default | Major | . | Luke Lu | Luke Lu | +| [MAPREDUCE-2219](https://issues.apache.org/jira/browse/MAPREDUCE-2219) | JT should not try to remove mapred.system.dir during startup | Major | jobtracker | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2200](https://issues.apache.org/jira/browse/MAPREDUCE-2200) | TestUmbilicalProtocolWithJobToken is failing without Krb evironment: needs to be conditional | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2195](https://issues.apache.org/jira/browse/MAPREDUCE-2195) | New property for local conf directory in system-test-mapreduce.xml file. | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2188](https://issues.apache.org/jira/browse/MAPREDUCE-2188) | The new API MultithreadedMapper doesn't call the initialize method of the RecordReader | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-2179](https://issues.apache.org/jira/browse/MAPREDUCE-2179) | RaidBlockSender.java compilation fails | Blocker | contrib/raid | Giridharan Kesavan | Ramkumar Vadali | +| [MAPREDUCE-2173](https://issues.apache.org/jira/browse/MAPREDUCE-2173) | Race condition in TestBlockFixer causes intermittent failure | Major | . | Patrick Kling | Patrick Kling | +| [MAPREDUCE-2146](https://issues.apache.org/jira/browse/MAPREDUCE-2146) | Raid should not affect access time of a source file | Minor | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2143](https://issues.apache.org/jira/browse/MAPREDUCE-2143) | HarFileSystem is not able to handle spaces in its path | Major | harchive | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2127](https://issues.apache.org/jira/browse/MAPREDUCE-2127) | mapreduce trunk builds are failing on hudson | Major | build, pipes | Giridharan Kesavan | Bruno MahĂ© | +| [MAPREDUCE-2099](https://issues.apache.org/jira/browse/MAPREDUCE-2099) | RaidNode should recreate outdated parity HARs | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2096](https://issues.apache.org/jira/browse/MAPREDUCE-2096) | Secure local filesystem IO from symlink vulnerabilities | Blocker | jobtracker, security, tasktracker | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2095](https://issues.apache.org/jira/browse/MAPREDUCE-2095) | Gridmix unable to run for compressed traces(.gz format). | Major | contrib/gridmix | Vinay Kumar Thota | Ranjit Mathew | +| [MAPREDUCE-2084](https://issues.apache.org/jira/browse/MAPREDUCE-2084) | Deprecated org.apache.hadoop.util package in MapReduce produces deprecations in Common classes in Eclipse | Blocker | documentation | Tom White | Tom White | +| [MAPREDUCE-2082](https://issues.apache.org/jira/browse/MAPREDUCE-2082) | Race condition in writing the jobtoken password file when launching pipes jobs | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-2078](https://issues.apache.org/jira/browse/MAPREDUCE-2078) | TraceBuilder unable to generate the traces while giving the job history path by globing. | Major | tools/rumen | Vinay Kumar Thota | Amar Kamat | +| [MAPREDUCE-2077](https://issues.apache.org/jira/browse/MAPREDUCE-2077) | Name clash in the deprecated o.a.h.util.MemoryCalculatorPlugin | Major | . | Luke Lu | Luke Lu | +| [MAPREDUCE-2067](https://issues.apache.org/jira/browse/MAPREDUCE-2067) | Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies | Major | security | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-2059](https://issues.apache.org/jira/browse/MAPREDUCE-2059) | RecoveryManager attempts to add jobtracker.info | Major | jobtracker | Dan Adkins | Subroto Sanyal | +| [MAPREDUCE-2054](https://issues.apache.org/jira/browse/MAPREDUCE-2054) | Hierarchical queue implementation broke dynamic queue addition in Dynamic Scheduler | Major | contrib/dynamic-scheduler | Thomas Sandholm | Thomas Sandholm | +| [MAPREDUCE-2032](https://issues.apache.org/jira/browse/MAPREDUCE-2032) | TestJobOutputCommitter fails in ant test run | Major | task | Amareshwari Sriramadasu | Dick King | +| [MAPREDUCE-2031](https://issues.apache.org/jira/browse/MAPREDUCE-2031) | TestTaskLauncher and TestTaskTrackerLocalization fail with NPE in trunk. | Major | tasktracker, test | Amareshwari Sriramadasu | Ravi Gummadi | +| [MAPREDUCE-2029](https://issues.apache.org/jira/browse/MAPREDUCE-2029) | DistributedRaidFileSystem not removed from cache on close() | Major | contrib/raid | Paul Yang | Ramkumar Vadali | +| [MAPREDUCE-2023](https://issues.apache.org/jira/browse/MAPREDUCE-2023) | TestDFSIO read test may not read specified bytes. | Major | benchmarks | Hong Tang | Hong Tang | +| [MAPREDUCE-2022](https://issues.apache.org/jira/browse/MAPREDUCE-2022) | trunk build broken | Blocker | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-2021](https://issues.apache.org/jira/browse/MAPREDUCE-2021) | CombineFileInputFormat returns duplicate hostnames in split locations | Major | client | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-2000](https://issues.apache.org/jira/browse/MAPREDUCE-2000) | Rumen is not able to extract counters for Job history logs from Hadoop 0.20 | Major | tools/rumen | Hong Tang | Hong Tang | +| [MAPREDUCE-1999](https://issues.apache.org/jira/browse/MAPREDUCE-1999) | ClientProtocol incorrectly uses hdfs DelegationTokenSelector | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-1996](https://issues.apache.org/jira/browse/MAPREDUCE-1996) | API: Reducer.reduce() method detail misstatement | Trivial | documentation | Glynn Durham | Harsh J | +| [MAPREDUCE-1993](https://issues.apache.org/jira/browse/MAPREDUCE-1993) | TestTrackerDistributedCacheManagerWithLinuxTaskController fails on trunk | Major | . | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1992](https://issues.apache.org/jira/browse/MAPREDUCE-1992) | NPE in JobTracker's constructor | Major | jobtracker | Ravi Gummadi | Kan Zhang | +| [MAPREDUCE-1989](https://issues.apache.org/jira/browse/MAPREDUCE-1989) | Gridmix3 doesn't emit out proper mesage when user resolver is set and no user list is given | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1982](https://issues.apache.org/jira/browse/MAPREDUCE-1982) | [Rumen] TraceBuilder's output shows jobname as NULL for jobhistory files with valid jobnames | Major | tools/rumen | Amar Kamat | Ravi Gummadi | +| [MAPREDUCE-1979](https://issues.apache.org/jira/browse/MAPREDUCE-1979) | "Output directory already exists" error in gridmix when gridmix.output.directory is not defined | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1975](https://issues.apache.org/jira/browse/MAPREDUCE-1975) | gridmix shows unnecessary InterruptedException | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1974](https://issues.apache.org/jira/browse/MAPREDUCE-1974) | FairScheduler can preempt the same task many times | Major | contrib/fair-share | Scott Chen | Scott Chen | +| [MAPREDUCE-1961](https://issues.apache.org/jira/browse/MAPREDUCE-1961) | [gridmix3] ConcurrentModificationException when shutting down Gridmix | Major | contrib/gridmix | Hong Tang | Hong Tang | +| [MAPREDUCE-1958](https://issues.apache.org/jira/browse/MAPREDUCE-1958) | using delegation token over hftp for long running clients (part of hdfs 1296) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1925](https://issues.apache.org/jira/browse/MAPREDUCE-1925) | TestRumenJobTraces fails in trunk | Major | tools/rumen | Amareshwari Sriramadasu | Ravi Gummadi | +| [MAPREDUCE-1915](https://issues.apache.org/jira/browse/MAPREDUCE-1915) | IndexCache - getIndexInformation - check reduce index Out Of Bounds | Trivial | tasktracker | Rares Vernica | Priyo Mustafi | +| [MAPREDUCE-1911](https://issues.apache.org/jira/browse/MAPREDUCE-1911) | Fix errors in -info option in streaming | Major | contrib/streaming | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1908](https://issues.apache.org/jira/browse/MAPREDUCE-1908) | DistributedRaidFileSystem does not handle ChecksumException correctly | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1900](https://issues.apache.org/jira/browse/MAPREDUCE-1900) | MapReduce daemons should close FileSystems that are not needed anymore | Major | jobtracker, tasktracker | Devaraj Das | Kan Zhang | +| [MAPREDUCE-1894](https://issues.apache.org/jira/browse/MAPREDUCE-1894) | DistributedRaidFileSystem.readFully() does not return | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1888](https://issues.apache.org/jira/browse/MAPREDUCE-1888) | Streaming overrides user given output key and value types. | Major | contrib/streaming | Amareshwari Sriramadasu | Ravi Gummadi | +| [MAPREDUCE-1887](https://issues.apache.org/jira/browse/MAPREDUCE-1887) | MRAsyncDiskService does not properly absolutize volume root paths | Major | . | Aaron Kimball | Aaron Kimball | +| [MAPREDUCE-1880](https://issues.apache.org/jira/browse/MAPREDUCE-1880) | "java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result." while running "hadoop jar hadoop-0.20.1+169.89-examples.jar pi 4 30" | Minor | examples | Victor Pakhomov | Tsz Wo Nicholas Sze | +| [MAPREDUCE-1867](https://issues.apache.org/jira/browse/MAPREDUCE-1867) | Remove unused methods in org.apache.hadoop.streaming.StreamUtil | Minor | contrib/streaming | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1865](https://issues.apache.org/jira/browse/MAPREDUCE-1865) | [Rumen] Rumen should also support jobhistory files generated using trunk | Major | tools/rumen | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1864](https://issues.apache.org/jira/browse/MAPREDUCE-1864) | PipeMapRed.java has uninitialized members log\_ and LOGNAME | Major | contrib/streaming | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1863](https://issues.apache.org/jira/browse/MAPREDUCE-1863) | [Rumen] Null failedMapAttemptCDFs in job traces generated by Rumen | Major | tools/rumen | Amar Kamat | Amar Kamat | +| [MAPREDUCE-1857](https://issues.apache.org/jira/browse/MAPREDUCE-1857) | Remove unused streaming configuration from src | Trivial | contrib/streaming | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1853](https://issues.apache.org/jira/browse/MAPREDUCE-1853) | MultipleOutputs does not cache TaskAttemptContext | Critical | task | Torsten Curdt | Torsten Curdt | +| [MAPREDUCE-1845](https://issues.apache.org/jira/browse/MAPREDUCE-1845) | FairScheduler.tasksToPeempt() can return negative number | Major | contrib/fair-share | Scott Chen | Scott Chen | +| [MAPREDUCE-1836](https://issues.apache.org/jira/browse/MAPREDUCE-1836) | Refresh for proxy superuser config (mr part for HDFS-1096) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1834](https://issues.apache.org/jira/browse/MAPREDUCE-1834) | TestSimulatorDeterministicReplay timesout on trunk | Major | contrib/mumak | Amareshwari Sriramadasu | Hong Tang | +| [MAPREDUCE-1825](https://issues.apache.org/jira/browse/MAPREDUCE-1825) | jobqueue\_details.jsp and FairSchedulerServelet should not call finishedMaps and finishedReduces when job is not initialized | Major | jobtracker | Amareshwari Sriramadasu | Scott Chen | +| [MAPREDUCE-1820](https://issues.apache.org/jira/browse/MAPREDUCE-1820) | InputSampler does not create a deep copy of the key object when creating a sample, which causes problems with some formats like SequenceFile\ | Major | . | Alex Kozlov | Alex Kozlov | +| [MAPREDUCE-1813](https://issues.apache.org/jira/browse/MAPREDUCE-1813) | NPE in PipeMapred.MRErrorThread | Major | contrib/streaming | Amareshwari Sriramadasu | Ravi Gummadi | +| [MAPREDUCE-1784](https://issues.apache.org/jira/browse/MAPREDUCE-1784) | IFile should check for null compressor | Minor | . | Eli Collins | Eli Collins | +| [MAPREDUCE-1780](https://issues.apache.org/jira/browse/MAPREDUCE-1780) | AccessControlList.toString() is used for serialization of ACL in JobStatus.java | Major | jobtracker | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-1773](https://issues.apache.org/jira/browse/MAPREDUCE-1773) | streaming doesn't support jobclient.output.filter | Major | contrib/streaming | Alok Singh | Amareshwari Sriramadasu | +| [MAPREDUCE-1772](https://issues.apache.org/jira/browse/MAPREDUCE-1772) | Hadoop streaming doc should not use IdentityMapper as an example | Minor | contrib/streaming, documentation | Marco Nicosia | Amareshwari Sriramadasu | +| [MAPREDUCE-1754](https://issues.apache.org/jira/browse/MAPREDUCE-1754) | Replace mapred.persmissions.supergroup with an acl : mapreduce.cluster.administrators | Major | jobtracker | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1718](https://issues.apache.org/jira/browse/MAPREDUCE-1718) | job conf key for the services name of DelegationToken for HFTP url is constructed incorrectly in HFTPFileSystem | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1707](https://issues.apache.org/jira/browse/MAPREDUCE-1707) | TaskRunner can get NPE in getting ugi from TaskTracker | Major | tasktracker | Amareshwari Sriramadasu | Vinod Kumar Vavilapalli | +| [MAPREDUCE-1701](https://issues.apache.org/jira/browse/MAPREDUCE-1701) | AccessControlException while renewing a delegation token in not correctly handled in the JobTracker | Major | jobtracker | Devaraj Das | Boris Shkolnik | +| [MAPREDUCE-1686](https://issues.apache.org/jira/browse/MAPREDUCE-1686) | ClassNotFoundException for custom format classes provided in libjars | Minor | contrib/streaming, test | Paul Burkhardt | Paul Burkhardt | +| [MAPREDUCE-1670](https://issues.apache.org/jira/browse/MAPREDUCE-1670) | RAID should avoid policies that scan their own destination path | Major | contrib/raid | Rodrigo Schmidt | Ramkumar Vadali | +| [MAPREDUCE-1668](https://issues.apache.org/jira/browse/MAPREDUCE-1668) | RaidNode should only Har a directory if all its parity files have been created | Major | contrib/raid | Rodrigo Schmidt | Ramkumar Vadali | +| [MAPREDUCE-1662](https://issues.apache.org/jira/browse/MAPREDUCE-1662) | TaskRunner.prepare() and close() can be removed | Major | tasktracker | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [MAPREDUCE-1621](https://issues.apache.org/jira/browse/MAPREDUCE-1621) | Streaming's TextOutputReader.getLastOutput throws NPE if it has never read any output | Major | contrib/streaming | Todd Lipcon | Amareshwari Sriramadasu | +| [MAPREDUCE-1617](https://issues.apache.org/jira/browse/MAPREDUCE-1617) | TestBadRecords failed once in our test runs | Major | test | Amareshwari Sriramadasu | Luke Lu | +| [MAPREDUCE-1599](https://issues.apache.org/jira/browse/MAPREDUCE-1599) | MRBench reuses jobConf and credentials there in. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-1597](https://issues.apache.org/jira/browse/MAPREDUCE-1597) | combinefileinputformat does not work with non-splittable files | Major | . | Namit Jain | Amareshwari Sriramadasu | +| [MAPREDUCE-1566](https://issues.apache.org/jira/browse/MAPREDUCE-1566) | Need to add a mechanism to import tokens and secrets into a submitted job. | Major | security | Owen O'Malley | Jitendra Nath Pandey | +| [MAPREDUCE-1559](https://issues.apache.org/jira/browse/MAPREDUCE-1559) | The DelegationTokenRenewal timer task should use the jobtracker's credentials to create the filesystem | Major | jobtracker | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1558](https://issues.apache.org/jira/browse/MAPREDUCE-1558) | specify correct server principal for RefreshAuthorizationPolicyProtocol and RefreshUserToGroupMappingsProtocol protocols in MRAdmin (for HADOOP-6612) | Major | security | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-1543](https://issues.apache.org/jira/browse/MAPREDUCE-1543) | Log messages of JobACLsManager should use security logging of HADOOP-6586 | Major | security | Vinod Kumar Vavilapalli | Luke Lu | +| [MAPREDUCE-1533](https://issues.apache.org/jira/browse/MAPREDUCE-1533) | Reduce or remove usage of String.format() usage in CapacityTaskScheduler.updateQSIObjects and Counters.makeEscapedString() | Major | jobtracker | Rajesh Balamohan | Dick King | +| [MAPREDUCE-1532](https://issues.apache.org/jira/browse/MAPREDUCE-1532) | Delegation token is obtained as the superuser | Major | job submission, security | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1528](https://issues.apache.org/jira/browse/MAPREDUCE-1528) | TokenStorage should not be static | Major | . | Owen O'Malley | Jitendra Nath Pandey | +| [MAPREDUCE-1505](https://issues.apache.org/jira/browse/MAPREDUCE-1505) | Cluster class should create the rpc client only when needed | Major | client | Devaraj Das | Dick King | +| [MAPREDUCE-1375](https://issues.apache.org/jira/browse/MAPREDUCE-1375) | TestFileArgs fails intermittently | Major | contrib/streaming, test | Amar Kamat | Todd Lipcon | +| [MAPREDUCE-1288](https://issues.apache.org/jira/browse/MAPREDUCE-1288) | DistributedCache localizes only once per cache URI | Critical | distributed-cache, security, tasktracker | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1280](https://issues.apache.org/jira/browse/MAPREDUCE-1280) | Eclipse Plugin does not work with Eclipse Ganymede (3.4) | Major | . | Aaron Kimball | Alex Kozlov | +| [MAPREDUCE-1225](https://issues.apache.org/jira/browse/MAPREDUCE-1225) | TT successfully localizes a task even though the corresponding cache-file has already changed on DFS. | Major | tasktracker | Vinod Kumar Vavilapalli | Zhong Wang | +| [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | Capacity Scheduler scheduling information is hard to read / should be tabular format | Major | capacity-sched | Allen Wittenauer | Krishna Ramachandran | +| [MAPREDUCE-1085](https://issues.apache.org/jira/browse/MAPREDUCE-1085) | For tasks, "ulimit -v -1" is being run when user doesn't specify mapred.child.ulimit | Minor | tasktracker | Ravi Gummadi | Todd Lipcon | +| [MAPREDUCE-869](https://issues.apache.org/jira/browse/MAPREDUCE-869) | Documentation for config to set map/reduce task environment | Major | documentation, task | Arun C Murthy | Alejandro Abdelnur | +| [MAPREDUCE-714](https://issues.apache.org/jira/browse/MAPREDUCE-714) | JobConf.findContainingJar unescapes unnecessarily on Linux | Major | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-647](https://issues.apache.org/jira/browse/MAPREDUCE-647) | Update the DistCp forrest doc to make it consistent with the latest changes (5472, 5620, 5762, 5826) | Major | documentation | Rodrigo Schmidt | Rodrigo Schmidt | +| [MAPREDUCE-577](https://issues.apache.org/jira/browse/MAPREDUCE-577) | Duplicate Mapper input when using StreamXmlRecordReader | Major | contrib/streaming | David Campbell | Ravi Gummadi | +| [MAPREDUCE-572](https://issues.apache.org/jira/browse/MAPREDUCE-572) | If #link is missing from uri format of -cacheArchive then streaming does not throw error. | Minor | contrib/streaming | Karam Singh | Amareshwari Sriramadasu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7783](https://issues.apache.org/jira/browse/HADOOP-7783) | Add more symlink tests that cover intermediate links | Major | fs | Eli Collins | Eli Collins | +| [HADOOP-7298](https://issues.apache.org/jira/browse/HADOOP-7298) | Add test utility for writing multi-threaded tests | Major | test | Todd Lipcon | Todd Lipcon | +| [HADOOP-7034](https://issues.apache.org/jira/browse/HADOOP-7034) | Add TestPath tests to cover dot, dot dot, and slash normalization | Major | fs | Eli Collins | Eli Collins | +| [HADOOP-7024](https://issues.apache.org/jira/browse/HADOOP-7024) | Create a test method for adding file systems during tests. | Major | test | Kan Zhang | Kan Zhang | +| [HADOOP-6934](https://issues.apache.org/jira/browse/HADOOP-6934) | test for ByteWritable comparator | Major | record | Johannes Zillmann | Johannes Zillmann | +| [HADOOP-6803](https://issues.apache.org/jira/browse/HADOOP-6803) | Add native gzip read/write coverage to TestCodec | Major | io | Eli Collins | Eli Collins | +| [HDFS-1855](https://issues.apache.org/jira/browse/HDFS-1855) | TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy() part 2 fails in two different ways | Major | test | Matt Foley | Matt Foley | +| [HDFS-1762](https://issues.apache.org/jira/browse/HDFS-1762) | Allow TestHDFSCLI to be run against a cluster | Major | build, test | Tom White | Konstantin Boudnik | +| [HDFS-1562](https://issues.apache.org/jira/browse/HDFS-1562) | Add rack policy tests | Major | namenode, test | Eli Collins | Eli Collins | +| [HDFS-1310](https://issues.apache.org/jira/browse/HDFS-1310) | TestFileConcurrentReader fails | Major | hdfs-client | Suresh Srinivas | sam rash | +| [HDFS-1132](https://issues.apache.org/jira/browse/HDFS-1132) | Refactor TestFileStatus | Minor | test | Eli Collins | Eli Collins | +| [HDFS-982](https://issues.apache.org/jira/browse/HDFS-982) | TestDelegationToken#testDelegationTokenWithRealUser is failing | Blocker | contrib/hdfsproxy, security | Eli Collins | Po Cheung | +| [HDFS-981](https://issues.apache.org/jira/browse/HDFS-981) | test-contrib fails due to test-cactus failure | Blocker | contrib/hdfsproxy | Eli Collins | Konstantin Boudnik | +| [HDFS-697](https://issues.apache.org/jira/browse/HDFS-697) | Enable asserts for tests by default | Major | . | Eli Collins | Eli Collins | +| [HDFS-696](https://issues.apache.org/jira/browse/HDFS-696) | Java assertion failures triggered by tests | Major | test | Eli Collins | Eli Collins | +| [MAPREDUCE-3156](https://issues.apache.org/jira/browse/MAPREDUCE-3156) | Allow TestMRCLI to be run against a cluster | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2241](https://issues.apache.org/jira/browse/MAPREDUCE-2241) | ClusterWithLinuxTaskController should accept relative path on the command line | Trivial | task-controller, test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2180](https://issues.apache.org/jira/browse/MAPREDUCE-2180) | Add coverage of fair scheduler servlet to system test | Minor | contrib/fair-share | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2073](https://issues.apache.org/jira/browse/MAPREDUCE-2073) | TestTrackerDistributedCacheManager should be up-front about requirements on build environment | Trivial | distributed-cache, test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2051](https://issues.apache.org/jira/browse/MAPREDUCE-2051) | Contribute a fair scheduler preemption system test | Major | contrib/fair-share | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2034](https://issues.apache.org/jira/browse/MAPREDUCE-2034) | TestSubmitJob triggers NPE instead of permissions error | Trivial | test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1092](https://issues.apache.org/jira/browse/MAPREDUCE-1092) | Enable asserts for tests by default | Major | test | Eli Collins | Eli Collins | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6683](https://issues.apache.org/jira/browse/HADOOP-6683) | the first optimization: ZlibCompressor does not fully utilize the buffer | Minor | io | Kang Xiao | Kang Xiao | +| [HDFS-1997](https://issues.apache.org/jira/browse/HDFS-1997) | Image transfer process misreports client side exceptions | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1473](https://issues.apache.org/jira/browse/HDFS-1473) | Refactor storage management into separate classes than fsimage file reading/writing | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1462](https://issues.apache.org/jira/browse/HDFS-1462) | Refactor edit log loading to a separate class from edit log writing | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1319](https://issues.apache.org/jira/browse/HDFS-1319) | Fix location of re-login for secondary namenode from HDFS-999 | Major | namenode | Jakob Homan | Jakob Homan | +| [HDFS-1119](https://issues.apache.org/jira/browse/HDFS-1119) | Refactor BlocksMap with GettableSet | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1071](https://issues.apache.org/jira/browse/HDFS-1071) | savenamespace should write the fsimage to all configured fs.name.dir in parallel | Major | namenode | dhruba borthakur | Dmytro Molkov | +| [HDFS-1057](https://issues.apache.org/jira/browse/HDFS-1057) | Concurrent readers hit ChecksumExceptions if following a writer to very end of file | Blocker | datanode | Todd Lipcon | sam rash | +| [HDFS-259](https://issues.apache.org/jira/browse/HDFS-259) | Remove intentionally corrupt 0.13 directory layout creation | Major | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2234](https://issues.apache.org/jira/browse/MAPREDUCE-2234) | If Localizer can't create task log directory, it should fail on the spot | Major | tasktracker | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1970](https://issues.apache.org/jira/browse/MAPREDUCE-1970) | Reed-Solomon code implementation to be used in raid | Major | contrib/raid | Scott Chen | Scott Chen | +| [MAPREDUCE-927](https://issues.apache.org/jira/browse/MAPREDUCE-927) | Cleanup of task-logs should happen in TaskTracker instead of the Child | Major | security, tasktracker | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7291](https://issues.apache.org/jira/browse/HADOOP-7291) | Update Hudson job not to run test-contrib | Major | . | Eli Collins | Eli Collins | +| [HADOOP-7283](https://issues.apache.org/jira/browse/HADOOP-7283) | Include 32-bit and 64-bit native libraries in Jenkins tarball builds | Blocker | build | Tom White | Tom White | +| [HADOOP-6846](https://issues.apache.org/jira/browse/HADOOP-6846) | Scripts for building Hadoop 0.22.0 release | Major | build | Tom White | Tom White | +| [HDFS-2513](https://issues.apache.org/jira/browse/HDFS-2513) | Bump jetty to 6.1.26 | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1948](https://issues.apache.org/jira/browse/HDFS-1948) | Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery' | Major | . | stack | stack | +| [HDFS-1946](https://issues.apache.org/jira/browse/HDFS-1946) | HDFS part of HADOOP-7291 | Major | . | Eli Collins | Eli Collins | +| [HDFS-1167](https://issues.apache.org/jira/browse/HDFS-1167) | New property for local conf directory in system-test-hdfs.xml file. | Major | test | Vinay Kumar Thota | Vinay Kumar Thota | +| [HDFS-712](https://issues.apache.org/jira/browse/HDFS-712) | Move libhdfs from mr to hdfs | Major | libhdfs | Eli Collins | Eli Collins | +| [MAPREDUCE-3311](https://issues.apache.org/jira/browse/MAPREDUCE-3311) | Bump jetty to 6.1.26 | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2939](https://issues.apache.org/jira/browse/MAPREDUCE-2939) | Ant setup on hadoop7 jenkins host | Major | build | Joep Rottinghuis | Joep Rottinghuis | +| [MAPREDUCE-2383](https://issues.apache.org/jira/browse/MAPREDUCE-2383) | Improve documentation of DistributedCache methods | Major | distributed-cache, documentation | Todd Lipcon | Harsh J | +| [MAPREDUCE-2169](https://issues.apache.org/jira/browse/MAPREDUCE-2169) | Integrated Reed-Solomon code with RaidNode | Major | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2142](https://issues.apache.org/jira/browse/MAPREDUCE-2142) | Refactor RaidNode to remove dependence on map reduce | Major | . | Patrick Kling | Patrick Kling | +| [MAPREDUCE-231](https://issues.apache.org/jira/browse/MAPREDUCE-231) | Split map/reduce into sub-project | Major | . | Owen O'Malley | Owen O'Malley | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/RELEASENOTES.0.22.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/RELEASENOTES.0.22.0.md new file mode 100644 index 00000000000..30368cc60a0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.0/RELEASENOTES.0.22.0.md @@ -0,0 +1,630 @@ + + +# Apache Hadoop 0.22.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7302](https://issues.apache.org/jira/browse/HADOOP-7302) | *Major* | **webinterface.private.actions should not be in common** + +Option webinterface.private.actions has been renamed to mapreduce.jobtracker.webinterface.trusted and should be specified in mapred-site.xml instead of core-site.xml + + +--- + +* [HADOOP-7229](https://issues.apache.org/jira/browse/HADOOP-7229) | *Major* | **Absolute path to kinit in auto-renewal thread** + +When Hadoop's Kerberos integration is enabled, it is now required that either {{kinit}} be on the path for user accounts running the Hadoop client, or that the {{hadoop.kerberos.kinit.command}} configuration option be manually set to the absolute path to {{kinit}}. + + +--- + +* [HADOOP-7193](https://issues.apache.org/jira/browse/HADOOP-7193) | *Minor* | **Help message is wrong for touchz command.** + +Updated the help for the touchz command. + + +--- + +* [HADOOP-7192](https://issues.apache.org/jira/browse/HADOOP-7192) | *Trivial* | **fs -stat docs aren't updated to reflect the format features** + +Updated the web documentation to reflect the formatting abilities of 'fs -stat'. + + +--- + +* [HADOOP-7156](https://issues.apache.org/jira/browse/HADOOP-7156) | *Critical* | **getpwuid\_r is not thread-safe on RHEL6** + +Adds a new configuration hadoop.work.around.non.threadsafe.getpwuid which can be used to enable a mutex around this call to workaround thread-unsafe implementations of getpwuid\_r. Users should consult http://wiki.apache.org/hadoop/KnownBrokenPwuidImplementations for a list of such systems. + + +--- + +* [HADOOP-7137](https://issues.apache.org/jira/browse/HADOOP-7137) | *Major* | **Remove hod contrib** + +Removed contrib related build targets. + + +--- + +* [HADOOP-7134](https://issues.apache.org/jira/browse/HADOOP-7134) | *Major* | **configure files that are generated as part of the released tarball need to have executable bit set** + +I have just committed this to trunk and branch-0.22. Thanks Roman! + + +--- + +* [HADOOP-7117](https://issues.apache.org/jira/browse/HADOOP-7117) | *Major* | **Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml** + +Removed references to the older fs.checkpoint.* properties that resided in core-site.xml + + +--- + +* [HADOOP-7089](https://issues.apache.org/jira/browse/HADOOP-7089) | *Minor* | **Fix link resolution logic in hadoop-config.sh** + +Updates hadoop-config.sh to always resolve symlinks when determining HADOOP\_HOME. Bash built-ins or POSIX:2001 compliant cmds are now required. + + +--- + +* [HADOOP-7013](https://issues.apache.org/jira/browse/HADOOP-7013) | *Major* | **Add boolean field isCorrupt to BlockLocation** + +This patch has changed the serialization format of BlockLocation. + + +--- + +* [HADOOP-7005](https://issues.apache.org/jira/browse/HADOOP-7005) | *Major* | **Update test-patch.sh to remove callback to Hudson master** + +N/A + + +--- + +* [HADOOP-6949](https://issues.apache.org/jira/browse/HADOOP-6949) | *Major* | **Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting** + +Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5. +Introduces ArrayPrimitiveWritable for a much more efficient wire format to transmit arrays of primitives over RPC. ObjectWritable uses the new writable for array of primitives for RPC and continues to use existing format for on-disk data. + + +--- + +* [HADOOP-6922](https://issues.apache.org/jira/browse/HADOOP-6922) | *Major* | **COMMON part of MAPREDUCE-1664** + +Makes AccessControlList a writable and updates documentation for Job ACLs. + + +--- + +* [HADOOP-6905](https://issues.apache.org/jira/browse/HADOOP-6905) | *Major* | **Better logging messages when a delegation token is invalid** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6835](https://issues.apache.org/jira/browse/HADOOP-6835) | *Major* | **Support concatenated gzip files** + +Processing of concatenated gzip files formerly stopped (quietly) at the end of the first substream/"member"; now processing will continue to the end of the concatenated stream, like gzip(1) does. (bzip2 support is unaffected by this patch.) + + +--- + +* [HADOOP-6787](https://issues.apache.org/jira/browse/HADOOP-6787) | *Major* | **Factor out glob pattern code from FileContext and Filesystem** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6730](https://issues.apache.org/jira/browse/HADOOP-6730) | *Major* | **Bug in FileContext#copy and provide base class for FileContext tests** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6693](https://issues.apache.org/jira/browse/HADOOP-6693) | *Major* | **Add metrics to track kerberos login activity** + +New metrics "login" of type MetricTimeVaryingRate is added under new metrics context name "ugi" and metrics record name "ugi". + + +--- + +* [HADOOP-6683](https://issues.apache.org/jira/browse/HADOOP-6683) | *Minor* | **the first optimization: ZlibCompressor does not fully utilize the buffer** + +Improve the buffer utilization of ZlibCompressor to avoid invoking a JNI per write request. + + +--- + +* [HADOOP-6663](https://issues.apache.org/jira/browse/HADOOP-6663) | *Major* | **BlockDecompressorStream get EOF exception when decompressing the file compressed from empty file** + +Fix EOF exception in BlockDecompressorStream when decompressing previous compressed empty file + + +--- + +* [HADOOP-6599](https://issues.apache.org/jira/browse/HADOOP-6599) | *Major* | **Split RPC metrics into summary and detailed metrics** + +Split existing RpcMetrics into RpcMetrics and RpcDetailedMetrics. The new RpcDetailedMetrics has per method usage details and is available under context name "rpc" and record name "detailed-metrics" + + +--- + +* [HADOOP-6436](https://issues.apache.org/jira/browse/HADOOP-6436) | *Major* | **Remove auto-generated native build files** + +The native build run when from trunk now requires autotools, libtool and openssl dev libraries. + + +--- + +* [HADOOP-6344](https://issues.apache.org/jira/browse/HADOOP-6344) | *Major* | **rm and rmr fail to correctly move the user's files to the trash prior to deleting when they are over quota.** + +Trash feature notifies user of over-quota condition rather than silently deleting files/directories; deletion can be compelled with "rm -skiptrash". + + +--- + +* [HADOOP-4675](https://issues.apache.org/jira/browse/HADOOP-4675) | *Major* | **Current Ganglia metrics implementation is incompatible with Ganglia 3.1** + +Support for reporting metrics to Ganglia 3.1 servers + + +--- + +* [HDFS-1948](https://issues.apache.org/jira/browse/HDFS-1948) | *Major* | **Forward port 'hdfs-1520 lightweight namenode operation to trigger lease reccovery'** + +Adds method to NameNode/ClientProtocol that allows for rude revoke of lease on current lease holder + + +--- + +* [HDFS-1825](https://issues.apache.org/jira/browse/HDFS-1825) | *Major* | **Remove thriftfs contrib** + +Removed thriftfs contrib component. + + +--- + +* [HDFS-1596](https://issues.apache.org/jira/browse/HDFS-1596) | *Major* | **Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml** + +Removed references to the older fs.checkpoint.* properties that resided in core-site.xml + + +--- + +* [HDFS-1582](https://issues.apache.org/jira/browse/HDFS-1582) | *Major* | **Remove auto-generated native build files** + +The native build run when from trunk now requires autotools, libtool and openssl dev libraries. + + +--- + +* [HDFS-1560](https://issues.apache.org/jira/browse/HDFS-1560) | *Minor* | **dfs.data.dir permissions should default to 700** + +The permissions on datanode data directories (configured by dfs.datanode.data.dir.perm) now default to 0700. Upon startup, the datanode will automatically change the permissions to match the configured value. + + +--- + +* [HDFS-1457](https://issues.apache.org/jira/browse/HDFS-1457) | *Major* | **Limit transmission rate when transfering image between primary and secondary NNs** + +Add a configuration variable dfs.image.transfer.bandwidthPerSec to allow the user to specify the amount of bandwidth for transferring image and edits. Its default value is 0 indicating no throttling. + + +--- + +* [HDFS-1435](https://issues.apache.org/jira/browse/HDFS-1435) | *Major* | **Provide an option to store fsimage compressed** + +This provides an option to store fsimage compressed. The layout version is bumped to -25. The user could configure if s/he wants the fsimage to be compressed or not and which codec to use. By default the fsimage is not compressed. + + +--- + +* [HDFS-1318](https://issues.apache.org/jira/browse/HDFS-1318) | *Major* | **HDFS Namenode and Datanode WebUI information needs to be accessible programmatically for scripts** + +resubmit the patch for HDFS1318 as Hudson was down last week. + + +--- + +* [HDFS-1315](https://issues.apache.org/jira/browse/HDFS-1315) | *Major* | **Add fsck event to audit log and remove other audit log events corresponding to FSCK listStatus and open calls** + +When running fsck, audit log events are not logged for listStatus and open are not logged. A new event with cmd=fsck is logged with ugi field set to the user requesting fsck and src field set to the fsck path. + + +--- + +* [HDFS-1109](https://issues.apache.org/jira/browse/HDFS-1109) | *Major* | **HFTP and URL Encoding** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-1096](https://issues.apache.org/jira/browse/HDFS-1096) | *Major* | **allow dfsadmin/mradmin refresh of superuser proxy group mappings** + +changed protocol name (may be used in hadoop-policy.xml) +from security.refresh.usertogroups.mappings.protocol.acl to security.refresh.user.mappings.protocol.acl + + +--- + +* [HDFS-1080](https://issues.apache.org/jira/browse/HDFS-1080) | *Major* | **SecondaryNameNode image transfer should use the defined http address rather than local ip address** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-1079](https://issues.apache.org/jira/browse/HDFS-1079) | *Major* | **HDFS implementation should throw exceptions defined in AbstractFileSystem** + +Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException. + + +--- + +* [HDFS-1061](https://issues.apache.org/jira/browse/HDFS-1061) | *Minor* | **Memory footprint optimization for INodeFile object.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-1035](https://issues.apache.org/jira/browse/HDFS-1035) | *Major* | **Generate Eclipse's .classpath file from Ivy config** + +Added support to auto-generate the Eclipse .classpath file from ivy. + + +--- + +* [HDFS-903](https://issues.apache.org/jira/browse/HDFS-903) | *Critical* | **NN should verify images and edit logs on startup** + +Store fsimage MD5 checksum in VERSION file. Validate checksum when loading a fsimage. Layout version bumped. + + +--- + +* [HDFS-712](https://issues.apache.org/jira/browse/HDFS-712) | *Major* | **Move libhdfs from mr to hdfs** + +Moved the libhdfs package to the HDFS subproject. + + +--- + +* [HDFS-708](https://issues.apache.org/jira/browse/HDFS-708) | *Major* | **A stress-test tool for HDFS.** + +Does not currently provide anything but uniform distribution. +Uses some older depreciated class interfaces (for mapper and reducer) +This was tested on 0.20 and 0.22 (locally) so it should be fairly backwards compatible. + + +--- + +* [HDFS-330](https://issues.apache.org/jira/browse/HDFS-330) | *Trivial* | **Datanode Web UIs should provide robots.txt** + +A robots.txt is now in place which will prevent well behaved crawlers from perusing Hadoop web interfaces. + + +--- + +* [HDFS-202](https://issues.apache.org/jira/browse/HDFS-202) | *Major* | **Add a bulk FIleSystem.getFileBlockLocations** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-3151](https://issues.apache.org/jira/browse/MAPREDUCE-3151) | *Major* | **Contrib tests failing** + +Confirmed that problem of finding ivy file occurs w/o patch with ant 1.7, and not with patch (with either ant 1.7 or 1.8). +Other unit tests are still failing the test steps themselves on my laptop, but that is not due not finding the ivy file. + + +--- + +* [MAPREDUCE-2516](https://issues.apache.org/jira/browse/MAPREDUCE-2516) | *Minor* | **option to control sensitive web actions** + +Configuration option webinterface.private.actions has been renamed to mapreduce.jobtracker.webinterface.trusted + + +--- + +* [MAPREDUCE-2410](https://issues.apache.org/jira/browse/MAPREDUCE-2410) | *Minor* | **document multiple keys per reducer oddity in hadoop streaming FAQ** + +Add an FAQ entry regarding the differences between Java API and Streaming development of MR programs. + + +--- + +* [MAPREDUCE-2272](https://issues.apache.org/jira/browse/MAPREDUCE-2272) | *Trivial* | **Job ACL file should not be executable** + +Job ACL files now have permissions set to 600 (previously 700). + + +--- + +* [MAPREDUCE-2260](https://issues.apache.org/jira/browse/MAPREDUCE-2260) | *Major* | **Remove auto-generated native build files** + +The native build run when from trunk now requires autotools, libtool and openssl dev libraries. + + +--- + +* [MAPREDUCE-2251](https://issues.apache.org/jira/browse/MAPREDUCE-2251) | *Major* | **Remove mapreduce.job.userhistorylocation config** + +Remove the now defunct property `mapreduce.job.userhistorylocation`. + + +--- + +* [MAPREDUCE-2147](https://issues.apache.org/jira/browse/MAPREDUCE-2147) | *Trivial* | **JobInProgress has some redundant lines in its ctor** + +Remove some redundant lines from JobInProgress's constructor which was re-initializing things unnecessarily. + + +--- + +* [MAPREDUCE-2096](https://issues.apache.org/jira/browse/MAPREDUCE-2096) | *Blocker* | **Secure local filesystem IO from symlink vulnerabilities** + +The TaskTracker now uses the libhadoop JNI library to operate securely on local files when security is enabled. Secure clusters must ensure that libhadoop.so is available to the TaskTracker. + + +--- + +* [MAPREDUCE-2054](https://issues.apache.org/jira/browse/MAPREDUCE-2054) | *Major* | **Hierarchical queue implementation broke dynamic queue addition in Dynamic Scheduler** + +Fix Dynamic Priority Scheduler to work with hierarchical queue names + + +--- + +* [MAPREDUCE-2032](https://issues.apache.org/jira/browse/MAPREDUCE-2032) | *Major* | **TestJobOutputCommitter fails in ant test run** + +Clears a problem that {{TestJobCleanup}} leaves behind files that cause {{TestJobOutputCommitter}} to error out. + + +--- + +* [MAPREDUCE-1996](https://issues.apache.org/jira/browse/MAPREDUCE-1996) | *Trivial* | **API: Reducer.reduce() method detail misstatement** + +Fix a misleading documentation note about the usage of Reporter objects in Reducers. + + +--- + +* [MAPREDUCE-1905](https://issues.apache.org/jira/browse/MAPREDUCE-1905) | *Blocker* | **Context.setStatus() and progress() api are ignored** + +Moved the api public Counter getCounter(Enum\ counterName), public Counter getCounter(String groupName, String counterName) from org.apache.hadoop.mapreduce.TaskInputOutputContext to org.apache.hadoop.mapreduce.TaskAttemptContext + + +--- + +* [MAPREDUCE-1887](https://issues.apache.org/jira/browse/MAPREDUCE-1887) | *Major* | **MRAsyncDiskService does not properly absolutize volume root paths** + +MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root paths. (Aaron Kimball via zshao) + + +--- + +* [MAPREDUCE-1866](https://issues.apache.org/jira/browse/MAPREDUCE-1866) | *Minor* | **Remove deprecated class org.apache.hadoop.streaming.UTF8ByteArrayUtils** + +Removed public deprecated class org.apache.hadoop.streaming.UTF8ByteArrayUtils. + + +--- + +* [MAPREDUCE-1836](https://issues.apache.org/jira/browse/MAPREDUCE-1836) | *Major* | **Refresh for proxy superuser config (mr part for HDFS-1096)** + +changing name of the protocol (may be used in hadoop-policy.xml) +from +security.refresh.usertogroups.mappings.protocol.acl +to +security.refresh.user.mappings.protocol.acl + + +--- + +* [MAPREDUCE-1829](https://issues.apache.org/jira/browse/MAPREDUCE-1829) | *Major* | **JobInProgress.findSpeculativeTask should use min() to find the candidate instead of sort()** + +Improved performance of the method JobInProgress.findSpeculativeTask() which is in the critical heartbeat code path. + + +--- + +* [MAPREDUCE-1813](https://issues.apache.org/jira/browse/MAPREDUCE-1813) | *Major* | **NPE in PipeMapred.MRErrorThread** + +Fixed an NPE in streaming that occurs when there is no input to reduce and the streaming reducer sends status updates by writing "reporter:status: xxx" statements to stderr. + + +--- + +* [MAPREDUCE-1785](https://issues.apache.org/jira/browse/MAPREDUCE-1785) | *Minor* | **Add streaming config option for not emitting the key** + +Added a configuration property "stream.map.input.ignoreKey" to specify whether to ignore key or not while writing input for the mapper. This configuration parameter is valid only if stream.map.input.writer.class is org.apache.hadoop.streaming.io.TextInputWriter.class. For all other InputWriter's, key is always written. + + +--- + +* [MAPREDUCE-1780](https://issues.apache.org/jira/browse/MAPREDUCE-1780) | *Major* | **AccessControlList.toString() is used for serialization of ACL in JobStatus.java** + +Fixes serialization of job-acls in JobStatus to use AccessControlList.write() instead of AccessControlList.toString(). + + +--- + +* [MAPREDUCE-1773](https://issues.apache.org/jira/browse/MAPREDUCE-1773) | *Major* | **streaming doesn't support jobclient.output.filter** + +Improved console messaging for streaming jobs by using the generic JobClient API itself instead of the existing streaming-specific code. + + +--- + +* [MAPREDUCE-1733](https://issues.apache.org/jira/browse/MAPREDUCE-1733) | *Major* | **Authentication between pipes processes and java counterparts.** + +This jira introduces backward incompatibility. Existing pipes applications MUST be recompiled with new hadoop pipes library once the changes in this jira are deployed. + + +--- + +* [MAPREDUCE-1707](https://issues.apache.org/jira/browse/MAPREDUCE-1707) | *Major* | **TaskRunner can get NPE in getting ugi from TaskTracker** + +Fixed a bug that causes TaskRunner to get NPE in getting ugi from TaskTracker and subsequently crashes it resulting in a failing task after task-timeout period. + + +--- + +* [MAPREDUCE-1683](https://issues.apache.org/jira/browse/MAPREDUCE-1683) | *Major* | **Remove JNI calls from ClusterStatus cstr** + +Removes JNI calls to get jvm current/max heap usage in ClusterStatus. Any instances of ClusterStatus serialized in a prior version will not be correctly deserialized using the updated class. + + +--- + +* [MAPREDUCE-1680](https://issues.apache.org/jira/browse/MAPREDUCE-1680) | *Major* | **Add a metrics to track the number of heartbeats processed** + +Added a metric to track number of heartbeats processed by the JobTracker. + + +--- + +* [MAPREDUCE-1664](https://issues.apache.org/jira/browse/MAPREDUCE-1664) | *Major* | **Job Acls affect Queue Acls** + +* Removed aclsEnabled flag from queues configuration files. +* Removed the configuration property mapreduce.cluster.job-authorization-enabled. +* Added mapreduce.cluster.acls.enabled as the single configuration property in mapred-default.xml that enables the authorization checks for all job level and queue level operations. +* To enable authorization of users to do job level and queue level operations, mapreduce.cluster.acls.enabled is to be set to true in JobTracker's configuration and in all TaskTrackers' configurations. +* To get access to a job, it is enough for a user to be part of one of the access lists i.e. either job-acl or queue-admins-acl(unlike before, when, one has to be part of both the lists). +* Queue administrators(configured via acl-administer-jobs) of a queue can do all view-job and modify-job operations on all jobs submitted to that queue. +* ClusterOwner(who started the mapreduce cluster) and cluster administrators(configured via mapreduce.cluster.permissions.supergroup) can do all job level operations and queue level operations on all jobs on all queues in that cluster irrespective of job-acls and queue-acls configured. +* JobOwner(who submitted job to a queue) can do all view-job and modify-job operations on his/her job irrespective of job-acls and queue-acls. +* Since aclsEnabled flag is removed from queues configuration files, "refresh of queues configuration" will not change mapreduce.cluster.acls.enabled on the fly. mapreduce.cluster.acls.enabled can be modified only when restarting the mapreduce cluster. + + +--- + +* [MAPREDUCE-1592](https://issues.apache.org/jira/browse/MAPREDUCE-1592) | *Major* | **Generate Eclipse's .classpath file from Ivy config** + +Added support to auto-generate the Eclipse .classpath file from ivy. + + +--- + +* [MAPREDUCE-1558](https://issues.apache.org/jira/browse/MAPREDUCE-1558) | *Major* | **specify correct server principal for RefreshAuthorizationPolicyProtocol and RefreshUserToGroupMappingsProtocol protocols in MRAdmin (for HADOOP-6612)** + +new config: +hadoop.security.service.user.name.key +this setting points to the server principal for RefreshUserToGroupMappingsProtocol. +The value should be either NN or JT principal depending if it is used in DFAdmin or MRAdmin. The value is set by the application. No need for default value. + + +--- + +* [MAPREDUCE-1543](https://issues.apache.org/jira/browse/MAPREDUCE-1543) | *Major* | **Log messages of JobACLsManager should use security logging of HADOOP-6586** + +Adds the audit logging facility to MapReduce. All authorization/authentication events are logged to audit log. Audit log entries are stored as key=value. + + +--- + +* [MAPREDUCE-1533](https://issues.apache.org/jira/browse/MAPREDUCE-1533) | *Major* | **Reduce or remove usage of String.format() usage in CapacityTaskScheduler.updateQSIObjects and Counters.makeEscapedString()** + +Incremental enhancements to the JobTracker to optimize heartbeat handling. + + +--- + +* [MAPREDUCE-1517](https://issues.apache.org/jira/browse/MAPREDUCE-1517) | *Major* | **streaming should support running on background** + +Adds -background option to run a streaming job in background. + + +--- + +* [MAPREDUCE-1505](https://issues.apache.org/jira/browse/MAPREDUCE-1505) | *Major* | **Cluster class should create the rpc client only when needed** + +Lazily construct a connection to the JobTracker from the job-submission client. + + +--- + +* [MAPREDUCE-1354](https://issues.apache.org/jira/browse/MAPREDUCE-1354) | *Critical* | **Incremental enhancements to the JobTracker for better scalability** + +Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling. + + +--- + +* [MAPREDUCE-1159](https://issues.apache.org/jira/browse/MAPREDUCE-1159) | *Trivial* | **Limit Job name on jobtracker.jsp to be 80 char long** + +Job names on jobtracker.jsp should be 80 characters long at most. + + +--- + +* [MAPREDUCE-1118](https://issues.apache.org/jira/browse/MAPREDUCE-1118) | *Major* | **Capacity Scheduler scheduling information is hard to read / should be tabular format** + +Add CapacityScheduler servlet to enhance web UI for queue information. + + +--- + +* [MAPREDUCE-927](https://issues.apache.org/jira/browse/MAPREDUCE-927) | *Major* | **Cleanup of task-logs should happen in TaskTracker instead of the Child** + +Moved Task log cleanup into a separate thread in TaskTracker. +Added configuration "mapreduce.job.userlog.retain.hours" to specify the time(in hours) for which the user-logs are to be retained after the job completion. + + +--- + +* [MAPREDUCE-572](https://issues.apache.org/jira/browse/MAPREDUCE-572) | *Minor* | **If #link is missing from uri format of -cacheArchive then streaming does not throw error.** + +Improved streaming job failure when #link is missing from uri format of -cacheArchive. Earlier it used to fail when launching individual tasks, now it fails during job submission itself. + + +--- + +* [MAPREDUCE-478](https://issues.apache.org/jira/browse/MAPREDUCE-478) | *Minor* | **separate jvm param for mapper and reducer** + +Allow map and reduce jvm parameters, environment variables and ulimit to be set separately. + +Configuration changes: + add mapred.map.child.java.opts + add mapred.reduce.child.java.opts + add mapred.map.child.env + add mapred.reduce.child.ulimit + add mapred.map.child.env + add mapred.reduce.child.ulimit + deprecated mapred.child.java.opts + deprecated mapred.child.env + deprecated mapred.child.ulimit + + +--- + +* [MAPREDUCE-220](https://issues.apache.org/jira/browse/MAPREDUCE-220) | *Major* | **Collecting cpu and memory usage for MapReduce tasks** + +Collect cpu and memory statistics per task. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/CHANGES.0.22.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/CHANGES.0.22.1.md new file mode 100644 index 00000000000..f77dab8c450 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/CHANGES.0.22.1.md @@ -0,0 +1,113 @@ + + +# Apache Hadoop Changelog + +## Release 0.22.1 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-6453](https://issues.apache.org/jira/browse/HADOOP-6453) | Hadoop wrapper script shouldn't ignore an existing JAVA\_LIBRARY\_PATH | Minor | scripts | Chad Metcalf | | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7937](https://issues.apache.org/jira/browse/HADOOP-7937) | Forward port SequenceFile#syncFs and friends from Hadoop 1.x | Major | io | Eli Collins | Tom White | +| [HADOOP-7119](https://issues.apache.org/jira/browse/HADOOP-7119) | add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3837](https://issues.apache.org/jira/browse/MAPREDUCE-3837) | Job tracker is not able to recover job in case of crash and after that no user can submit job. | Major | . | Mayank Bansal | Mayank Bansal | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7338](https://issues.apache.org/jira/browse/HADOOP-7338) | LocalDirAllocator improvements for MR-2178 | Major | . | Todd Lipcon | Benoy Antony | +| [HADOOP-7272](https://issues.apache.org/jira/browse/HADOOP-7272) | Remove unnecessary security related info logs | Major | ipc, security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-6995](https://issues.apache.org/jira/browse/HADOOP-6995) | Allow wildcards to be used in ProxyUsers configurations | Minor | security | Todd Lipcon | Todd Lipcon | +| [HDFS-2246](https://issues.apache.org/jira/browse/HDFS-2246) | Shortcut a local client reads to a Datanodes files directly | Major | . | Sanjay Radia | Jitendra Nath Pandey | +| [HDFS-1601](https://issues.apache.org/jira/browse/HDFS-1601) | Pipeline ACKs are sent as lots of tiny TCP packets | Major | datanode | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-4405](https://issues.apache.org/jira/browse/MAPREDUCE-4405) | Adding test case for HierarchicalQueue in TestJobQueueClient | Minor | client | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-4403](https://issues.apache.org/jira/browse/MAPREDUCE-4403) | Adding test case for resubmission of jobs in TestRecoveryManager | Minor | jobtracker | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-4349](https://issues.apache.org/jira/browse/MAPREDUCE-4349) | Distributed Cache gives inconsistent result if cache Archive files get deleted from task tracker | Minor | . | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-2353](https://issues.apache.org/jira/browse/MAPREDUCE-2353) | Make the MR changes to reflect the API changes in SecureIO library | Major | security, task, tasktracker | Devaraj Das | Benoy Antony | +| [MAPREDUCE-1521](https://issues.apache.org/jira/browse/MAPREDUCE-1521) | Protection against incorrectly configured reduces | Major | jobtracker | Arun C Murthy | Mahadev konar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7680](https://issues.apache.org/jira/browse/HADOOP-7680) | TestHardLink fails on Mac OS X, when gnu stat is in path | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-7621](https://issues.apache.org/jira/browse/HADOOP-7621) | alfredo config should be in a file not readable by users | Critical | security | Alejandro Abdelnur | Aaron T. Myers | +| [HADOOP-7115](https://issues.apache.org/jira/browse/HADOOP-7115) | Add a cache for getpwuid\_r and getpwgid\_r calls | Major | . | Arun C Murthy | Alejandro Abdelnur | +| [HDFS-3402](https://issues.apache.org/jira/browse/HDFS-3402) | Fix hdfs scripts for secure datanodes | Minor | scripts, security | Benoy Antony | Benoy Antony | +| [HDFS-3368](https://issues.apache.org/jira/browse/HDFS-3368) | Missing blocks due to bad DataNodes coming up and down. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2991](https://issues.apache.org/jira/browse/HDFS-2991) | failure to load edits: ClassCastException | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2877](https://issues.apache.org/jira/browse/HDFS-2877) | If locking of a storage dir fails, it will remove the other NN's lock file on exit | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2718](https://issues.apache.org/jira/browse/HDFS-2718) | Optimize OP\_ADD in edits loading | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2698](https://issues.apache.org/jira/browse/HDFS-2698) | BackupNode is downloading image from NameNode for every checkpoint | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-1910](https://issues.apache.org/jira/browse/HDFS-1910) | when dfs.name.dir and dfs.name.edits.dir are same fsimage will be saved twice every time | Minor | namenode | Gokul | | +| [HDFS-1584](https://issues.apache.org/jira/browse/HDFS-1584) | Need to check TGT and renew if needed when fetching delegation tokens using HFTP | Major | security | Kan Zhang | Benoy Antony | +| [MAPREDUCE-5706](https://issues.apache.org/jira/browse/MAPREDUCE-5706) | toBeDeleted parent directories aren't being cleaned up | Major | security | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4404](https://issues.apache.org/jira/browse/MAPREDUCE-4404) | Adding Test case for TestMRJobClient to verify the user name | Minor | client | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-4360](https://issues.apache.org/jira/browse/MAPREDUCE-4360) | Capacity Scheduler Hierarchical leaf queue does not honor the max capacity of container queue | Major | . | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-4318](https://issues.apache.org/jira/browse/MAPREDUCE-4318) | TestRecoveryManager should not use raw and deprecated configuration parameters. | Major | test | Konstantin Shvachko | Benoy Antony | +| [MAPREDUCE-4314](https://issues.apache.org/jira/browse/MAPREDUCE-4314) | Synchronization in JvmManager for 0.22 branch | Major | tasktracker | Konstantin Shvachko | Benoy Antony | +| [MAPREDUCE-4164](https://issues.apache.org/jira/browse/MAPREDUCE-4164) | Hadoop 22 Exception thrown after task completion causes its reexecution | Major | tasktracker | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-3863](https://issues.apache.org/jira/browse/MAPREDUCE-3863) | 0.22 branch mvn deploy is not publishing hadoop-streaming JAR | Critical | build | Alejandro Abdelnur | Benoy Antony | +| [MAPREDUCE-3725](https://issues.apache.org/jira/browse/MAPREDUCE-3725) | Hadoop 22 hadoop job -list returns user name as NULL | Major | client | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-3593](https://issues.apache.org/jira/browse/MAPREDUCE-3593) | MAPREDUCE Impersonation is not working in 22 | Major | job submission | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-2452](https://issues.apache.org/jira/browse/MAPREDUCE-2452) | Delegation token cancellation shouldn't hold global JobTracker lock | Major | jobtracker | Devaraj Das | Devaraj Das | +| [MAPREDUCE-2420](https://issues.apache.org/jira/browse/MAPREDUCE-2420) | JobTracker should be able to renew delegation token over HTTP | Major | . | Boris Shkolnik | Boris Shkolnik | +| [MAPREDUCE-2377](https://issues.apache.org/jira/browse/MAPREDUCE-2377) | task-controller fails to parse configuration if it doesn't end in \n | Major | task-controller | Todd Lipcon | Benoy Antony | +| [MAPREDUCE-2178](https://issues.apache.org/jira/browse/MAPREDUCE-2178) | Race condition in LinuxTaskController permissions handling | Major | security, task-controller | Todd Lipcon | Benoy Antony | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8383](https://issues.apache.org/jira/browse/HADOOP-8383) | TestKerberosAuthenticator fails | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-8381](https://issues.apache.org/jira/browse/HADOOP-8381) | Substitute \_HOST with hostname for HTTP principals | Minor | security | Benoy Antony | Benoy Antony | +| [HDFS-2886](https://issues.apache.org/jira/browse/HDFS-2886) | CreateEditLogs should generate a realistic edit log. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8357](https://issues.apache.org/jira/browse/HADOOP-8357) | Restore security in Hadoop 0.22 branch | Major | security | Konstantin Shvachko | Benoy Antony | +| [MAPREDUCE-4249](https://issues.apache.org/jira/browse/MAPREDUCE-4249) | Fix failures in streaming test TestFileArgs | Minor | security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4248](https://issues.apache.org/jira/browse/MAPREDUCE-4248) | TestRecoveryManager fails | Minor | security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4247](https://issues.apache.org/jira/browse/MAPREDUCE-4247) | TestTaskTrackerLocalization fails | Minor | security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4246](https://issues.apache.org/jira/browse/MAPREDUCE-4246) | Failure in deleting user directories in Secure hadoop | Major | security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4244](https://issues.apache.org/jira/browse/MAPREDUCE-4244) | Fix an issue related to do with setting of correct groups for tasks | Minor | security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4243](https://issues.apache.org/jira/browse/MAPREDUCE-4243) | Modify mapreduce build to include task-controller | Minor | build | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4240](https://issues.apache.org/jira/browse/MAPREDUCE-4240) | Revert MAPREDUCE-2767 | Minor | security | Benoy Antony | Benoy Antony | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/RELEASENOTES.0.22.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/RELEASENOTES.0.22.1.md new file mode 100644 index 00000000000..32b973d6efa --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.22.1/RELEASENOTES.0.22.1.md @@ -0,0 +1,64 @@ + + +# Apache Hadoop 0.22.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7119](https://issues.apache.org/jira/browse/HADOOP-7119) | *Major* | **add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles** + +Adding support for Kerberos HTTP SPNEGO authentication to the Hadoop web-consoles + + +--- + +* [HADOOP-6995](https://issues.apache.org/jira/browse/HADOOP-6995) | *Minor* | **Allow wildcards to be used in ProxyUsers configurations** + +When configuring proxy users and hosts, the special wildcard value "*" may be specified to match any host or any user. + + +--- + +* [HADOOP-6453](https://issues.apache.org/jira/browse/HADOOP-6453) | *Minor* | **Hadoop wrapper script shouldn't ignore an existing JAVA\_LIBRARY\_PATH** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-2246](https://issues.apache.org/jira/browse/HDFS-2246) | *Major* | **Shortcut a local client reads to a Datanodes files directly** + +1. New configurations +a. dfs.block.local-path-access.user is the key in datanode configuration to specify the user allowed to do short circuit read. +b. dfs.client.read.shortcircuit is the key to enable short circuit read at the client side configuration. +c. dfs.client.read.shortcircuit.skip.checksum is the key to bypass checksum check at the client side. +2. By default none of the above are enabled and short circuit read will not kick in. +3. If security is on, the feature can be used only for user that has kerberos credentials at the client, therefore map reduce tasks cannot benefit from it in general. + + +--- + +* [MAPREDUCE-3725](https://issues.apache.org/jira/browse/MAPREDUCE-3725) | *Major* | **Hadoop 22 hadoop job -list returns user name as NULL** + +Submitting the patch after setting the user name at the client side + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/CHANGES.0.23.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/CHANGES.0.23.0.md new file mode 100644 index 00000000000..e7a65068ec0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/CHANGES.0.23.0.md @@ -0,0 +1,1182 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.0 - 2011-11-11 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7547](https://issues.apache.org/jira/browse/HADOOP-7547) | Fix the warning in writable classes.[ WritableComparable is a raw type. References to generic type WritableComparable\ should be parameterized ] | Minor | io | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7507](https://issues.apache.org/jira/browse/HADOOP-7507) | jvm metrics all use the same namespace | Major | metrics | Jeff Bean | Alejandro Abdelnur | +| [HADOOP-7374](https://issues.apache.org/jira/browse/HADOOP-7374) | Don't add tools.jar to the classpath when running Hadoop | Major | scripts | Eli Collins | Eli Collins | +| [HADOOP-7331](https://issues.apache.org/jira/browse/HADOOP-7331) | Make hadoop-daemon.sh to return 1 if daemon processes did not get started | Trivial | scripts | Tanping Wang | Tanping Wang | +| [HADOOP-7286](https://issues.apache.org/jira/browse/HADOOP-7286) | Refactor FsShell's du/dus/df | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7264](https://issues.apache.org/jira/browse/HADOOP-7264) | Bump avro version to at least 1.4.1 | Major | io | Luke Lu | Luke Lu | +| [HADOOP-7227](https://issues.apache.org/jira/browse/HADOOP-7227) | Remove protocol version check at proxy creation in Hadoop RPC. | Major | ipc | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7153](https://issues.apache.org/jira/browse/HADOOP-7153) | MapWritable violates contract of Map interface for equals() and hashCode() | Minor | io | Nicholas Telford | Nicholas Telford | +| [HADOOP-7136](https://issues.apache.org/jira/browse/HADOOP-7136) | Remove failmon contrib | Major | . | Nigel Daley | Nigel Daley | +| [HADOOP-6949](https://issues.apache.org/jira/browse/HADOOP-6949) | Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting | Major | io | Navis | Matt Foley | +| [HADOOP-6921](https://issues.apache.org/jira/browse/HADOOP-6921) | metrics2: metrics plugins | Major | . | Luke Lu | Luke Lu | +| [HADOOP-6920](https://issues.apache.org/jira/browse/HADOOP-6920) | Metrics2: metrics instrumentation | Major | . | Luke Lu | Luke Lu | +| [HADOOP-6904](https://issues.apache.org/jira/browse/HADOOP-6904) | A baby step towards inter-version RPC communications | Major | ipc | Hairong Kuang | Hairong Kuang | +| [HADOOP-6864](https://issues.apache.org/jira/browse/HADOOP-6864) | Provide a JNI-based implementation of ShellBasedUnixGroupsNetgroupMapping (implementation of GroupMappingServiceProvider) | Major | security | Erik Steffl | Boris Shkolnik | +| [HADOOP-6432](https://issues.apache.org/jira/browse/HADOOP-6432) | Statistics support in FileContext | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | Create an rpm integration project | Major | . | Owen O'Malley | Eric Yang | +| [HADOOP-2081](https://issues.apache.org/jira/browse/HADOOP-2081) | Configuration getInt, getLong, and getFloat replace invalid numbers with the default value | Major | conf | Owen O'Malley | Harsh J | +| [HDFS-2210](https://issues.apache.org/jira/browse/HDFS-2210) | Remove hdfsproxy | Major | contrib/hdfsproxy | Eli Collins | Eli Collins | +| [HDFS-2202](https://issues.apache.org/jira/browse/HDFS-2202) | Changes to balancer bandwidth should not require datanode restart. | Major | balancer & mover, datanode | Eric Payne | Eric Payne | +| [HDFS-2107](https://issues.apache.org/jira/browse/HDFS-2107) | Move block management code to a package | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2087](https://issues.apache.org/jira/browse/HDFS-2087) | Add methods to DataTransferProtocol interface | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2066](https://issues.apache.org/jira/browse/HDFS-2066) | Create a package and individual class files for DataTransferProtocol | Major | datanode, hdfs-client, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2058](https://issues.apache.org/jira/browse/HDFS-2058) | DataTransfer Protocol using protobufs | Major | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1966](https://issues.apache.org/jira/browse/HDFS-1966) | Encapsulate individual DataTransferProtocol op header | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1963](https://issues.apache.org/jira/browse/HDFS-1963) | HDFS rpm integration project | Major | build | Eric Yang | Eric Yang | +| [HDFS-1945](https://issues.apache.org/jira/browse/HDFS-1945) | Removed deprecated fields in DataTransferProtocol | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1761](https://issues.apache.org/jira/browse/HDFS-1761) | Add a new DataTransferProtocol operation, Op.TRANSFER\_BLOCK, instead of using RPC | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1723](https://issues.apache.org/jira/browse/HDFS-1723) | quota errors messages should use the same scale | Minor | . | Allen Wittenauer | Jim Plush | +| [HDFS-1703](https://issues.apache.org/jira/browse/HDFS-1703) | HDFS federation: Improve start/stop scripts and add script to decommission datanodes | Minor | scripts | Tanping Wang | Tanping Wang | +| [HDFS-1675](https://issues.apache.org/jira/browse/HDFS-1675) | Transfer RBW between datanodes | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1606](https://issues.apache.org/jira/browse/HDFS-1606) | Provide a stronger data guarantee in the write pipeline | Major | datanode, hdfs-client, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1560](https://issues.apache.org/jira/browse/HDFS-1560) | dfs.data.dir permissions should default to 700 | Minor | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1536](https://issues.apache.org/jira/browse/HDFS-1536) | Improve HDFS WebUI | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-1526](https://issues.apache.org/jira/browse/HDFS-1526) | Dfs client name for a map/reduce task should have some randomness | Major | hdfs-client | Hairong Kuang | Hairong Kuang | +| [HDFS-1117](https://issues.apache.org/jira/browse/HDFS-1117) | HDFS portion of HADOOP-6728 (ovehaul metrics framework) | Major | . | Luke Lu | Luke Lu | +| [HDFS-1073](https://issues.apache.org/jira/browse/HDFS-1073) | Simpler model for Namenode's fs Image and edit Logs | Major | . | Sanjay Radia | Todd Lipcon | +| [MAPREDUCE-3205](https://issues.apache.org/jira/browse/MAPREDUCE-3205) | MR2 memory limits should be pmem, not vmem | Blocker | mrv2, nodemanager | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3041](https://issues.apache.org/jira/browse/MAPREDUCE-3041) | Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol | Blocker | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-2736](https://issues.apache.org/jira/browse/MAPREDUCE-2736) | Remove unused contrib components dependent on MR1 | Major | jobtracker, tasktracker | Eli Collins | Eli Collins | +| [MAPREDUCE-2606](https://issues.apache.org/jira/browse/MAPREDUCE-2606) | Remove IsolationRunner | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-2455](https://issues.apache.org/jira/browse/MAPREDUCE-2455) | Remove deprecated JobTracker.State in favour of JobTrackerStatus | Major | build, client | Tom White | Tom White | +| [MAPREDUCE-2430](https://issues.apache.org/jira/browse/MAPREDUCE-2430) | Remove mrunit contrib | Major | . | Nigel Daley | Nigel Daley | +| [MAPREDUCE-1738](https://issues.apache.org/jira/browse/MAPREDUCE-1738) | MapReduce portion of HADOOP-6728 (ovehaul metrics framework) | Major | . | Luke Lu | Luke Lu | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7705](https://issues.apache.org/jira/browse/HADOOP-7705) | Add a log4j back end that can push out JSON data, one per line | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-7594](https://issues.apache.org/jira/browse/HADOOP-7594) | Support HTTP REST in HttpServer | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7493](https://issues.apache.org/jira/browse/HADOOP-7493) | [HDFS-362] Provide ShortWritable class in hadoop. | Major | io | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7443](https://issues.apache.org/jira/browse/HADOOP-7443) | Add CRC32C as another DataChecksum implementation | Major | io, util | Todd Lipcon | Todd Lipcon | +| [HADOOP-7257](https://issues.apache.org/jira/browse/HADOOP-7257) | A client side mount table to give per-application/per-job file system view | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-7214](https://issues.apache.org/jira/browse/HADOOP-7214) | Hadoop /usr/bin/groups equivalent | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7206](https://issues.apache.org/jira/browse/HADOOP-7206) | Integrate Snappy compression | Major | . | Eli Collins | Alejandro Abdelnur | +| [HADOOP-7144](https://issues.apache.org/jira/browse/HADOOP-7144) | Expose JMX with something like JMXProxyServlet | Major | . | Luke Lu | Robert Joseph Evans | +| [HADOOP-7119](https://issues.apache.org/jira/browse/HADOOP-7119) | add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-6889](https://issues.apache.org/jira/browse/HADOOP-6889) | Make RPC to have an option to timeout | Major | ipc | Hairong Kuang | John George | +| [HADOOP-6385](https://issues.apache.org/jira/browse/HADOOP-6385) | dfs does not support -rmdir (was HDFS-639) | Minor | fs | Scott Phillips | Daryn Sharp | +| [HDFS-2471](https://issues.apache.org/jira/browse/HDFS-2471) | Add Federation feature, configuration and tools documentation | Major | documentation | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2083](https://issues.apache.org/jira/browse/HDFS-2083) | Adopt JMXJsonServlet into HDFS in order to query statistics | Major | . | Tanping Wang | Tanping Wang | +| [HDFS-2055](https://issues.apache.org/jira/browse/HDFS-2055) | Add hflush support to libhdfs | Major | libhdfs | Travis Crawford | Travis Crawford | +| [HDFS-1873](https://issues.apache.org/jira/browse/HDFS-1873) | Federation Cluster Management Web Console | Major | . | Tanping Wang | Tanping Wang | +| [HDFS-1814](https://issues.apache.org/jira/browse/HDFS-1814) | HDFS portion of HADOOP-7214 - Hadoop /usr/bin/groups equivalent | Major | hdfs-client, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1751](https://issues.apache.org/jira/browse/HDFS-1751) | Intrinsic limits for HDFS files, directories | Major | datanode | Daryn Sharp | Daryn Sharp | +| [HDFS-1448](https://issues.apache.org/jira/browse/HDFS-1448) | Create multi-format parser for edits logs file, support binary and XML formats initially | Major | tools | Erik Steffl | Erik Steffl | +| [HDFS-1330](https://issues.apache.org/jira/browse/HDFS-1330) | Make RPCs to DataNodes timeout | Major | datanode | Hairong Kuang | John George | +| [HDFS-1052](https://issues.apache.org/jira/browse/HDFS-1052) | HDFS scalability with multiple namenodes | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-2719](https://issues.apache.org/jira/browse/MAPREDUCE-2719) | MR-279: Write a shell command application | Major | mrv2 | Sharad Agarwal | Hitesh Shah | +| [MAPREDUCE-2692](https://issues.apache.org/jira/browse/MAPREDUCE-2692) | Ensure AM Restart and Recovery-on-restart is complete | Major | mrv2 | Amol Kekre | Sharad Agarwal | +| [MAPREDUCE-2543](https://issues.apache.org/jira/browse/MAPREDUCE-2543) | [Gridmix] Add support for HighRam jobs | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-2533](https://issues.apache.org/jira/browse/MAPREDUCE-2533) | MR-279: Metrics for reserved resource in ResourceManager | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2532](https://issues.apache.org/jira/browse/MAPREDUCE-2532) | MR-279: Metrics for NodeManager | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2527](https://issues.apache.org/jira/browse/MAPREDUCE-2527) | MR-279: Metrics for MRAppMaster | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2521](https://issues.apache.org/jira/browse/MAPREDUCE-2521) | Mapreduce RPM integration project | Major | build | Eric Yang | Eric Yang | +| [MAPREDUCE-2473](https://issues.apache.org/jira/browse/MAPREDUCE-2473) | MR portion of HADOOP-7214 - Hadoop /usr/bin/groups equivalent | Major | jobtracker | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-2438](https://issues.apache.org/jira/browse/MAPREDUCE-2438) | MR-279: WebApp for Job History | Major | mrv2 | Mahadev konar | Krishna Ramachandran | +| [MAPREDUCE-2434](https://issues.apache.org/jira/browse/MAPREDUCE-2434) | MR-279: ResourceManager metrics | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2408](https://issues.apache.org/jira/browse/MAPREDUCE-2408) | Make Gridmix emulate usage of data compression | Major | contrib/gridmix | Ravi Gummadi | Amar Kamat | +| [MAPREDUCE-2407](https://issues.apache.org/jira/browse/MAPREDUCE-2407) | Make Gridmix emulate usage of Distributed Cache files | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-2323](https://issues.apache.org/jira/browse/MAPREDUCE-2323) | Add metrics to the fair scheduler | Major | contrib/fair-share | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2037](https://issues.apache.org/jira/browse/MAPREDUCE-2037) | Capturing interim progress times, CPU usage, and memory usage, when tasks reach certain progress thresholds | Major | . | Dick King | Dick King | +| [MAPREDUCE-1938](https://issues.apache.org/jira/browse/MAPREDUCE-1938) | Ability for having user's classes take precedence over the system classes for tasks' classpath | Blocker | job submission, task, tasktracker | Devaraj Das | Krishna Ramachandran | +| [MAPREDUCE-461](https://issues.apache.org/jira/browse/MAPREDUCE-461) | Enable ServicePlugins for the JobTracker | Minor | . | Fredrik Hedberg | Fredrik Hedberg | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8619](https://issues.apache.org/jira/browse/HADOOP-8619) | WritableComparator must implement no-arg constructor | Major | io | Radim Kolar | Chris Douglas | +| [HADOOP-7789](https://issues.apache.org/jira/browse/HADOOP-7789) | Minor edits to top-level site | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-7785](https://issues.apache.org/jira/browse/HADOOP-7785) | Add equals, hashcode, toString to DataChecksum | Major | io, util | Todd Lipcon | Todd Lipcon | +| [HADOOP-7772](https://issues.apache.org/jira/browse/HADOOP-7772) | javadoc the topology classes | Trivial | . | Steve Loughran | Steve Loughran | +| [HADOOP-7763](https://issues.apache.org/jira/browse/HADOOP-7763) | Add top-level navigation to APT docs | Major | documentation | Tom White | Tom White | +| [HADOOP-7749](https://issues.apache.org/jira/browse/HADOOP-7749) | Add NetUtils call which provides more help in exception messages | Minor | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-7743](https://issues.apache.org/jira/browse/HADOOP-7743) | Add Maven profile to create a full source tarball | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7737](https://issues.apache.org/jira/browse/HADOOP-7737) | normalize hadoop-mapreduce & hadoop-dist dist/tar build with common/hdfs | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7720](https://issues.apache.org/jira/browse/HADOOP-7720) | improve the hadoop-setup-conf.sh to read in the hbase user and setup the configs | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7710](https://issues.apache.org/jira/browse/HADOOP-7710) | create a script to setup application in order to create root directories for application such hbase, hcat, hive etc | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-7709](https://issues.apache.org/jira/browse/HADOOP-7709) | Running a set of methods in a Single Test Class | Major | . | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7707](https://issues.apache.org/jira/browse/HADOOP-7707) | improve config generator to allow users to specify proxy user, turn append on or off, turn webhdfs on or off | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7668](https://issues.apache.org/jira/browse/HADOOP-7668) | Add a NetUtils method that can tell if an InetAddress belongs to local host | Minor | util | Suresh Srinivas | Steve Loughran | +| [HADOOP-7664](https://issues.apache.org/jira/browse/HADOOP-7664) | o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same. | Minor | conf | Ravi Prakash | Ravi Prakash | +| [HADOOP-7655](https://issues.apache.org/jira/browse/HADOOP-7655) | provide a small validation script that smoke tests the installed cluster | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-7642](https://issues.apache.org/jira/browse/HADOOP-7642) | create hadoop-dist module where TAR stitching would happen | Major | build | Alejandro Abdelnur | Tom White | +| [HADOOP-7627](https://issues.apache.org/jira/browse/HADOOP-7627) | Improve MetricsAsserts to give more understandable output on failure | Minor | metrics, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-7624](https://issues.apache.org/jira/browse/HADOOP-7624) | Set things up for a top level hadoop-tools module | Major | build | Vinod Kumar Vavilapalli | Alejandro Abdelnur | +| [HADOOP-7612](https://issues.apache.org/jira/browse/HADOOP-7612) | Change test-patch to run tests for all nested modules | Major | build | Tom White | Tom White | +| [HADOOP-7595](https://issues.apache.org/jira/browse/HADOOP-7595) | Upgrade dependency to Avro 1.5.3 | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7555](https://issues.apache.org/jira/browse/HADOOP-7555) | Add a eclipse-generated files to .gitignore | Trivial | build | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7552](https://issues.apache.org/jira/browse/HADOOP-7552) | FileUtil#fullyDelete doesn't throw IOE but lists it in the throws clause | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-7531](https://issues.apache.org/jira/browse/HADOOP-7531) | Add servlet util methods for handling paths in requests | Major | util | Eli Collins | Eli Collins | +| [HADOOP-7509](https://issues.apache.org/jira/browse/HADOOP-7509) | Improve message when Authentication is required | Trivial | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7491](https://issues.apache.org/jira/browse/HADOOP-7491) | hadoop command should respect HADOOP\_OPTS when given a class name | Major | scripts | Eli Collins | Eli Collins | +| [HADOOP-7474](https://issues.apache.org/jira/browse/HADOOP-7474) | Refactor ClientCache out of WritableRpcEngine. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7472](https://issues.apache.org/jira/browse/HADOOP-7472) | RPC client should deal with the IP address changes | Minor | ipc | Kihwal Lee | Kihwal Lee | +| [HADOOP-7463](https://issues.apache.org/jira/browse/HADOOP-7463) | Adding a configuration parameter to SecurityInfo interface. | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-7460](https://issues.apache.org/jira/browse/HADOOP-7460) | Support for pluggable Trash policies | Major | fs | dhruba borthakur | Usman Masood | +| [HADOOP-7457](https://issues.apache.org/jira/browse/HADOOP-7457) | Remove out-of-date Chinese language documentation | Blocker | documentation | Jakob Homan | Jakob Homan | +| [HADOOP-7451](https://issues.apache.org/jira/browse/HADOOP-7451) | merge for MR-279: Generalize StringUtils#join | Major | . | Matt Foley | Matt Foley | +| [HADOOP-7449](https://issues.apache.org/jira/browse/HADOOP-7449) | merge for MR-279: add Data(In,Out)putByteBuffer to work with ByteBuffer similar to Data(In,Out)putBuffer for byte[] | Major | . | Matt Foley | Matt Foley | +| [HADOOP-7448](https://issues.apache.org/jira/browse/HADOOP-7448) | merge for MR-279: HttpServer /stacks servlet should use plain text content type | Major | . | Matt Foley | Matt Foley | +| [HADOOP-7446](https://issues.apache.org/jira/browse/HADOOP-7446) | Implement CRC32C native code using SSE4.2 instructions | Major | native, performance | Todd Lipcon | Todd Lipcon | +| [HADOOP-7445](https://issues.apache.org/jira/browse/HADOOP-7445) | Implement bulk checksum verification using efficient native code | Major | native, util | Todd Lipcon | Todd Lipcon | +| [HADOOP-7444](https://issues.apache.org/jira/browse/HADOOP-7444) | Add Checksum API to verify and calculate checksums "in bulk" | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7438](https://issues.apache.org/jira/browse/HADOOP-7438) | Using the hadoop-deamon.sh script to start nodes leads to a depricated warning | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7434](https://issues.apache.org/jira/browse/HADOOP-7434) | Display error when using "daemonlog -setlevel" with illegal level | Minor | . | äž„é‡‘ćŒ | äž„é‡‘ćŒ | +| [HADOOP-7430](https://issues.apache.org/jira/browse/HADOOP-7430) | Improve error message when moving to trash fails due to quota issue | Minor | fs | Ravi Prakash | Ravi Prakash | +| [HADOOP-7392](https://issues.apache.org/jira/browse/HADOOP-7392) | Implement capability of querying individual property of a mbean using JMXProxyServlet | Major | . | Tanping Wang | Tanping Wang | +| [HADOOP-7384](https://issues.apache.org/jira/browse/HADOOP-7384) | Allow test-patch to be more flexible about patch format | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7379](https://issues.apache.org/jira/browse/HADOOP-7379) | Add ability to include Protobufs in ObjectWritable | Major | io, ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7375](https://issues.apache.org/jira/browse/HADOOP-7375) | Add resolvePath method to FileContext | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-7361](https://issues.apache.org/jira/browse/HADOOP-7361) | Provide overwrite option (-overwrite/-f) in put and copyFromLocal command line options | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7360](https://issues.apache.org/jira/browse/HADOOP-7360) | FsShell does not preserve relative paths with globs | Major | fs | Daryn Sharp | Kihwal Lee | +| [HADOOP-7337](https://issues.apache.org/jira/browse/HADOOP-7337) | Annotate PureJavaCrc32 as a public API | Minor | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7333](https://issues.apache.org/jira/browse/HADOOP-7333) | Performance improvement in PureJavaCrc32 | Minor | performance, util | Eric Caspole | Eric Caspole | +| [HADOOP-7329](https://issues.apache.org/jira/browse/HADOOP-7329) | incomplete help message is displayed for df -h option | Minor | fs | XieXianshan | XieXianshan | +| [HADOOP-7328](https://issues.apache.org/jira/browse/HADOOP-7328) | When a serializer class is missing, return null, not throw an NPE. | Major | io | Harsh J | Harsh J | +| [HADOOP-7320](https://issues.apache.org/jira/browse/HADOOP-7320) | Refactor FsShell's copy & move commands | Major | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-7316](https://issues.apache.org/jira/browse/HADOOP-7316) | Add public javadocs to FSDataInputStream and FSDataOutputStream | Major | documentation | Jonathan Hsieh | Eli Collins | +| [HADOOP-7314](https://issues.apache.org/jira/browse/HADOOP-7314) | Add support for throwing UnknownHostException when a host doesn't resolve | Major | . | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [HADOOP-7306](https://issues.apache.org/jira/browse/HADOOP-7306) | Start metrics system even if config files are missing | Major | metrics | Luke Lu | Luke Lu | +| [HADOOP-7305](https://issues.apache.org/jira/browse/HADOOP-7305) | Eclipse project files are incomplete | Minor | build | Niels Basjes | Niels Basjes | +| [HADOOP-7301](https://issues.apache.org/jira/browse/HADOOP-7301) | FSDataInputStream should expose a getWrappedStream method | Major | . | Jonathan Hsieh | Jonathan Hsieh | +| [HADOOP-7289](https://issues.apache.org/jira/browse/HADOOP-7289) | ivy: test conf should not extend common conf | Major | build | Tsz Wo Nicholas Sze | Eric Yang | +| [HADOOP-7285](https://issues.apache.org/jira/browse/HADOOP-7285) | Refactor FsShell's test | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7275](https://issues.apache.org/jira/browse/HADOOP-7275) | Refactor FsShell's stat | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7271](https://issues.apache.org/jira/browse/HADOOP-7271) | Standardize error messages | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7267](https://issues.apache.org/jira/browse/HADOOP-7267) | Refactor FsShell's rm/rmr/expunge | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7265](https://issues.apache.org/jira/browse/HADOOP-7265) | Keep track of relative paths | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7251](https://issues.apache.org/jira/browse/HADOOP-7251) | Refactor FsShell's getmerge | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7250](https://issues.apache.org/jira/browse/HADOOP-7250) | Refactor FsShell's setrep | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7249](https://issues.apache.org/jira/browse/HADOOP-7249) | Refactor FsShell's chmod/chown/chgrp | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7241](https://issues.apache.org/jira/browse/HADOOP-7241) | fix typo of command 'hadoop fs -help tail' | Minor | fs, test | Wei Yongjun | Wei Yongjun | +| [HADOOP-7238](https://issues.apache.org/jira/browse/HADOOP-7238) | Refactor FsShell's cat & text | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7237](https://issues.apache.org/jira/browse/HADOOP-7237) | Refactor FsShell's touchz | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7236](https://issues.apache.org/jira/browse/HADOOP-7236) | Refactor FsShell's mkdir | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7235](https://issues.apache.org/jira/browse/HADOOP-7235) | Refactor FsShell's tail | Major | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-7233](https://issues.apache.org/jira/browse/HADOOP-7233) | Refactor FsShell's ls | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7209](https://issues.apache.org/jira/browse/HADOOP-7209) | Extensions to FsShell | Major | . | Olga Natkovich | Daryn Sharp | +| [HADOOP-7205](https://issues.apache.org/jira/browse/HADOOP-7205) | automatically determine JAVA\_HOME on OS X | Trivial | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-7202](https://issues.apache.org/jira/browse/HADOOP-7202) | Improve Command base class | Major | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-7193](https://issues.apache.org/jira/browse/HADOOP-7193) | Help message is wrong for touchz command. | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7180](https://issues.apache.org/jira/browse/HADOOP-7180) | Improve CommandFormat | Minor | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7177](https://issues.apache.org/jira/browse/HADOOP-7177) | CodecPool should report which compressor it is using | Trivial | native | Allen Wittenauer | Allen Wittenauer | +| [HADOOP-7167](https://issues.apache.org/jira/browse/HADOOP-7167) | Allow using a file to exclude certain tests from build | Minor | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7159](https://issues.apache.org/jira/browse/HADOOP-7159) | RPC server should log the client hostname when read exception happened | Trivial | ipc | Scott Chen | Scott Chen | +| [HADOOP-7133](https://issues.apache.org/jira/browse/HADOOP-7133) | CLONE to COMMON - HDFS-1445 Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file | Major | util | Matt Foley | Matt Foley | +| [HADOOP-7131](https://issues.apache.org/jira/browse/HADOOP-7131) | set() and toString Methods of the org.apache.hadoop.io.Text class does not include the root exception, in the wrapping RuntimeException. | Minor | io | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7117](https://issues.apache.org/jira/browse/HADOOP-7117) | Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml | Major | conf | Patrick Angeles | Harsh J | +| [HADOOP-7114](https://issues.apache.org/jira/browse/HADOOP-7114) | FsShell should dump all exceptions at DEBUG level | Minor | fs | Todd Lipcon | Todd Lipcon | +| [HADOOP-7112](https://issues.apache.org/jira/browse/HADOOP-7112) | Issue a warning when GenericOptionsParser libjars are not on local filesystem | Major | conf, filecache | Tom White | Tom White | +| [HADOOP-7096](https://issues.apache.org/jira/browse/HADOOP-7096) | Allow setting of end-of-record delimiter for TextInputFormat | Major | . | Ahmed Radwan | Ahmed Radwan | +| [HADOOP-7078](https://issues.apache.org/jira/browse/HADOOP-7078) | Add better javadocs for RawComparator interface | Trivial | . | Todd Lipcon | Harsh J | +| [HADOOP-7061](https://issues.apache.org/jira/browse/HADOOP-7061) | unprecise javadoc for CompressionCodec | Minor | io | Jingguo Yao | Jingguo Yao | +| [HADOOP-7060](https://issues.apache.org/jira/browse/HADOOP-7060) | A more elegant FileSystem#listCorruptFileBlocks API | Major | fs | Hairong Kuang | Patrick Kling | +| [HADOOP-7059](https://issues.apache.org/jira/browse/HADOOP-7059) | Remove "unused" warning in native code | Major | native | Noah Watkins | Noah Watkins | +| [HADOOP-7058](https://issues.apache.org/jira/browse/HADOOP-7058) | Expose number of bytes in FSOutputSummer buffer to implementatins | Trivial | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7049](https://issues.apache.org/jira/browse/HADOOP-7049) | TestReconfiguration should be junit v4 | Trivial | conf | Patrick Kling | Patrick Kling | +| [HADOOP-7048](https://issues.apache.org/jira/browse/HADOOP-7048) | Wrong description of Block-Compressed SequenceFile Format in SequenceFile's javadoc | Minor | io | Jingguo Yao | Jingguo Yao | +| [HADOOP-7042](https://issues.apache.org/jira/browse/HADOOP-7042) | Update test-patch.sh to include failed test names and move test-patch.properties | Minor | test | Nigel Daley | Nigel Daley | +| [HADOOP-7023](https://issues.apache.org/jira/browse/HADOOP-7023) | Add listCorruptFileBlocks to FileSystem | Major | . | Patrick Kling | Patrick Kling | +| [HADOOP-7014](https://issues.apache.org/jira/browse/HADOOP-7014) | Generalize CLITest structure and interfaces to facilitate upstream adoption (e.g. for web testing) | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-6994](https://issues.apache.org/jira/browse/HADOOP-6994) | Api to get delegation token in AbstractFileSystem | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-6929](https://issues.apache.org/jira/browse/HADOOP-6929) | RPC should have a way to pass Security information other than protocol annotations | Major | ipc, security | Sharad Agarwal | Sharad Agarwal | +| [HADOOP-6887](https://issues.apache.org/jira/browse/HADOOP-6887) | Need a separate metrics per garbage collector | Major | metrics | Bharath Mundlapudi | Luke Lu | +| [HADOOP-6764](https://issues.apache.org/jira/browse/HADOOP-6764) | Add number of reader threads and queue length as configuration parameters in RPC.getServer | Major | ipc | Dmytro Molkov | Dmytro Molkov | +| [HADOOP-6578](https://issues.apache.org/jira/browse/HADOOP-6578) | Configuration should trim whitespace around a lot of value types | Minor | conf | Todd Lipcon | Michele Catasta | +| [HADOOP-6436](https://issues.apache.org/jira/browse/HADOOP-6436) | Remove auto-generated native build files | Major | . | Eli Collins | Roman Shaposhnik | +| [HADOOP-6376](https://issues.apache.org/jira/browse/HADOOP-6376) | slaves file to have a header specifying the format of conf/slaves file | Minor | conf | Karthik K | Karthik K | +| [HADOOP-1886](https://issues.apache.org/jira/browse/HADOOP-1886) | Undocumented parameters in FilesSystem | Trivial | fs | Konstantin Shvachko | Frank Conrad | +| [HDFS-2521](https://issues.apache.org/jira/browse/HDFS-2521) | Remove custom checksum headers from data transfer protocol | Major | datanode, hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-2512](https://issues.apache.org/jira/browse/HDFS-2512) | Add textual error message to data transfer protocol responses | Major | datanode, hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-2507](https://issues.apache.org/jira/browse/HDFS-2507) | HA: Allow saveNamespace operations to be canceled | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2500](https://issues.apache.org/jira/browse/HDFS-2500) | Avoid file system operations in BPOfferService thread while processing deletes | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2485](https://issues.apache.org/jira/browse/HDFS-2485) | Improve code layout and constants in UnderReplicatedBlocks | Trivial | datanode | Steve Loughran | Steve Loughran | +| [HDFS-2465](https://issues.apache.org/jira/browse/HDFS-2465) | Add HDFS support for fadvise readahead and drop-behind | Major | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-2401](https://issues.apache.org/jira/browse/HDFS-2401) | Running a set of methods in a Single Test Class | Major | build | Jonathan Eagles | Jonathan Eagles | +| [HDFS-2371](https://issues.apache.org/jira/browse/HDFS-2371) | Refactor BlockSender.java for better readability | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2355](https://issues.apache.org/jira/browse/HDFS-2355) | Federation: enable using the same configuration file across all the nodes in the cluster. | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2294](https://issues.apache.org/jira/browse/HDFS-2294) | Download of commons-daemon TAR should not be under target | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2286](https://issues.apache.org/jira/browse/HDFS-2286) | DataXceiverServer logs AsynchronousCloseException at shutdown | Trivial | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2273](https://issues.apache.org/jira/browse/HDFS-2273) | Refactor BlockManager.recentInvalidateSets to a new class | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2260](https://issues.apache.org/jira/browse/HDFS-2260) | Refactor BlockReader into an interface and implementation | Major | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-2241](https://issues.apache.org/jira/browse/HDFS-2241) | Remove implementing FSConstants interface just to access the constants defined in the interface | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2238](https://issues.apache.org/jira/browse/HDFS-2238) | NamenodeFsck.toString() uses StringBuilder with + operator | Minor | namenode | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-2230](https://issues.apache.org/jira/browse/HDFS-2230) | hdfs it not resolving the latest common test jars published post common mavenization | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HDFS-2227](https://issues.apache.org/jira/browse/HDFS-2227) | HDFS-2018 Part 2 : getRemoteEditLogManifest should pull it's information from FileJournalManager | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-2226](https://issues.apache.org/jira/browse/HDFS-2226) | Clean up counting of operations in FSEditLogLoader | Trivial | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2225](https://issues.apache.org/jira/browse/HDFS-2225) | HDFS-2018 Part 1 : Refactor file management so its not in classes which should be generic | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-2212](https://issues.apache.org/jira/browse/HDFS-2212) | Refactor double-buffering code out of EditLogOutputStreams | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2209](https://issues.apache.org/jira/browse/HDFS-2209) | Make MiniDFS easier to embed in other apps | Minor | test | Steve Loughran | Steve Loughran | +| [HDFS-2205](https://issues.apache.org/jira/browse/HDFS-2205) | Log message for failed connection to datanode is not followed by a success message. | Major | hdfs-client | Ravi Prakash | Ravi Prakash | +| [HDFS-2198](https://issues.apache.org/jira/browse/HDFS-2198) | Remove hardcoded configuration keys | Minor | datanode, hdfs-client, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2197](https://issues.apache.org/jira/browse/HDFS-2197) | Refactor RPC call implementations out of NameNode class | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2187](https://issues.apache.org/jira/browse/HDFS-2187) | HDFS-1580: Make EditLogInputStream act like an iterator over FSEditLogOps | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-2180](https://issues.apache.org/jira/browse/HDFS-2180) | Refactor NameNode HTTP server into new class | Major | . | Todd Lipcon | Todd Lipcon | +| [HDFS-2161](https://issues.apache.org/jira/browse/HDFS-2161) | Move utilities to DFSUtil | Minor | balancer & mover, datanode, hdfs-client, namenode, security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2157](https://issues.apache.org/jira/browse/HDFS-2157) | Improve header comment in o.a.h.hdfs.server.namenode.NameNode | Major | documentation, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2144](https://issues.apache.org/jira/browse/HDFS-2144) | If SNN shuts down during initialization it does not log the cause | Major | namenode | Ravi Prakash | Ravi Prakash | +| [HDFS-2143](https://issues.apache.org/jira/browse/HDFS-2143) | Federation: we should link to the live nodes and dead nodes to cluster web console | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-2118](https://issues.apache.org/jira/browse/HDFS-2118) | Couple dfs data dir improvements | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-2116](https://issues.apache.org/jira/browse/HDFS-2116) | Cleanup TestStreamFile and TestByteRangeInputStream | Minor | test | Eli Collins | Plamen Jeliazkov | +| [HDFS-2110](https://issues.apache.org/jira/browse/HDFS-2110) | Some StreamFile and ByteRangeInputStream cleanup | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-2073](https://issues.apache.org/jira/browse/HDFS-2073) | Namenode is missing @Override annotations | Minor | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2056](https://issues.apache.org/jira/browse/HDFS-2056) | Update fetchdt usage | Minor | documentation, tools | Tanping Wang | Tanping Wang | +| [HDFS-2054](https://issues.apache.org/jira/browse/HDFS-2054) | BlockSender.sendChunk() prints ERROR for connection closures encountered during transferToFully() | Minor | datanode | Kihwal Lee | Kihwal Lee | +| [HDFS-2046](https://issues.apache.org/jira/browse/HDFS-2046) | Force entropy to come from non-true random for tests | Major | build, test | Todd Lipcon | Todd Lipcon | +| [HDFS-2040](https://issues.apache.org/jira/browse/HDFS-2040) | Only build libhdfs if a flag is passed | Minor | . | Eli Collins | Eli Collins | +| [HDFS-2029](https://issues.apache.org/jira/browse/HDFS-2029) | Improve TestWriteRead | Trivial | test | Tsz Wo Nicholas Sze | John George | +| [HDFS-2024](https://issues.apache.org/jira/browse/HDFS-2024) | Eclipse format HDFS Junit test hdfs/TestWriteRead.java | Trivial | test | CW Chung | CW Chung | +| [HDFS-2003](https://issues.apache.org/jira/browse/HDFS-2003) | Separate FSEditLog reading logic from editLog memory state building logic | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-1996](https://issues.apache.org/jira/browse/HDFS-1996) | ivy: hdfs test jar should be independent to common test jar | Major | build | Tsz Wo Nicholas Sze | Eric Yang | +| [HDFS-1995](https://issues.apache.org/jira/browse/HDFS-1995) | Minor modification to both dfsclusterhealth and dfshealth pages for Web UI | Minor | . | Tanping Wang | Tanping Wang | +| [HDFS-1959](https://issues.apache.org/jira/browse/HDFS-1959) | Better error message for missing namenode directory | Minor | . | Eli Collins | Eli Collins | +| [HDFS-1958](https://issues.apache.org/jira/browse/HDFS-1958) | Format confirmation prompt should be more lenient of its input | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1939](https://issues.apache.org/jira/browse/HDFS-1939) | ivy: test conf should not extend common conf | Major | build | Tsz Wo Nicholas Sze | Eric Yang | +| [HDFS-1937](https://issues.apache.org/jira/browse/HDFS-1937) | Umbrella JIRA for improving DataTransferProtocol | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1906](https://issues.apache.org/jira/browse/HDFS-1906) | Remove logging exception stack trace when one of the datanode targets to read from is not reachable | Minor | hdfs-client | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1899](https://issues.apache.org/jira/browse/HDFS-1899) | GenericTestUtils.formatNamenode is misplaced | Major | . | Todd Lipcon | Ted Yu | +| [HDFS-1890](https://issues.apache.org/jira/browse/HDFS-1890) | A few improvements on the LeaseRenewer.pendingCreates map | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1870](https://issues.apache.org/jira/browse/HDFS-1870) | Refactor DFSClient.LeaseChecker | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1865](https://issues.apache.org/jira/browse/HDFS-1865) | Share LeaseChecker thread among DFSClients | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1861](https://issues.apache.org/jira/browse/HDFS-1861) | Rename dfs.datanode.max.xcievers and bump its default value | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-1846](https://issues.apache.org/jira/browse/HDFS-1846) | Don't fill preallocated portion of edits log with 0x00 | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1843](https://issues.apache.org/jira/browse/HDFS-1843) | Discover file not found early for file append | Minor | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1840](https://issues.apache.org/jira/browse/HDFS-1840) | Terminate LeaseChecker when all writing files are closed. | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1833](https://issues.apache.org/jira/browse/HDFS-1833) | Refactor BlockReceiver | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1831](https://issues.apache.org/jira/browse/HDFS-1831) | HDFS equivalent of HADOOP-7223 changes to handle FileContext createFlag combinations | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1817](https://issues.apache.org/jira/browse/HDFS-1817) | Split TestFiDataTransferProtocol.java into two files | Trivial | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1789](https://issues.apache.org/jira/browse/HDFS-1789) | Refactor frequently used codes from DFSOutputStream, BlockReceiver and DataXceiver | Minor | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1785](https://issues.apache.org/jira/browse/HDFS-1785) | Cleanup BlockReceiver and DataXceiver | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1774](https://issues.apache.org/jira/browse/HDFS-1774) | Small optimization to FSDataset | Minor | datanode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-1773](https://issues.apache.org/jira/browse/HDFS-1773) | Remove a datanode from cluster if include list is not empty and this datanode is removed from both include and exclude lists | Minor | namenode | Tanping Wang | Tanping Wang | +| [HDFS-1763](https://issues.apache.org/jira/browse/HDFS-1763) | Replace hard-coded option strings with variables from DFSConfigKeys | Minor | . | Eli Collins | Eli Collins | +| [HDFS-1757](https://issues.apache.org/jira/browse/HDFS-1757) | Don't compile fuse-dfs by default | Major | fuse-dfs | Eli Collins | Eli Collins | +| [HDFS-1741](https://issues.apache.org/jira/browse/HDFS-1741) | Provide a minimal pom file to allow integration of HDFS into Sonar analysis | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1739](https://issues.apache.org/jira/browse/HDFS-1739) | When DataNode throws DiskOutOfSpaceException, it will be helpfull to the user if we log the available volume size and configured block size. | Minor | datanode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-1731](https://issues.apache.org/jira/browse/HDFS-1731) | Allow using a file to exclude certain tests from build | Minor | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1636](https://issues.apache.org/jira/browse/HDFS-1636) | If dfs.name.dir points to an empty dir, namenode format shouldn't require confirmation | Minor | namenode | Todd Lipcon | Harsh J | +| [HDFS-1630](https://issues.apache.org/jira/browse/HDFS-1630) | Checksum fsedits | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1628](https://issues.apache.org/jira/browse/HDFS-1628) | AccessControlException should display the full path | Minor | namenode | Ramya Sunil | John George | +| [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | Make BLOCK\_INVALIDATE\_LIMIT configurable | Minor | namenode | Arun C Murthy | Tsz Wo Nicholas Sze | +| [HDFS-1620](https://issues.apache.org/jira/browse/HDFS-1620) | Rename HdfsConstants -\> HdfsServerConstants, FSConstants -\> HdfsConstants | Minor | . | Tsz Wo Nicholas Sze | Harsh J | +| [HDFS-1601](https://issues.apache.org/jira/browse/HDFS-1601) | Pipeline ACKs are sent as lots of tiny TCP packets | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1596](https://issues.apache.org/jira/browse/HDFS-1596) | Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml | Major | documentation, namenode | Patrick Angeles | Harsh J | +| [HDFS-1588](https://issues.apache.org/jira/browse/HDFS-1588) | Add dfs.hosts.exclude to DFSConfigKeys and use constant in stead of hardcoded string | Major | . | Erik Steffl | Erik Steffl | +| [HDFS-1583](https://issues.apache.org/jira/browse/HDFS-1583) | Improve backup-node sync performance by wrapping RPC parameters | Major | namenode | Liyin Liang | Liyin Liang | +| [HDFS-1582](https://issues.apache.org/jira/browse/HDFS-1582) | Remove auto-generated native build files | Major | libhdfs | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-1573](https://issues.apache.org/jira/browse/HDFS-1573) | LeaseChecker thread name trace not that useful | Trivial | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-1568](https://issues.apache.org/jira/browse/HDFS-1568) | Improve DataXceiver error logging | Minor | datanode | Todd Lipcon | Joey Echeverria | +| [HDFS-1547](https://issues.apache.org/jira/browse/HDFS-1547) | Improve decommission mechanism | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1539](https://issues.apache.org/jira/browse/HDFS-1539) | prevent data loss when a cluster suffers a power loss | Major | datanode, hdfs-client, namenode | dhruba borthakur | dhruba borthakur | +| [HDFS-1534](https://issues.apache.org/jira/browse/HDFS-1534) | Fix some incorrect logs in FSDirectory | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-1518](https://issues.apache.org/jira/browse/HDFS-1518) | Wrong description in FSNamesystem's javadoc | Minor | namenode | Jingguo Yao | Jingguo Yao | +| [HDFS-1513](https://issues.apache.org/jira/browse/HDFS-1513) | Fix a number of warnings | Minor | . | Eli Collins | Eli Collins | +| [HDFS-1510](https://issues.apache.org/jira/browse/HDFS-1510) | Add test-patch.properties required by test-patch.sh | Minor | . | Nigel Daley | Nigel Daley | +| [HDFS-1509](https://issues.apache.org/jira/browse/HDFS-1509) | Resync discarded directories in fs.name.dir during saveNamespace command | Major | namenode | dhruba borthakur | dhruba borthakur | +| [HDFS-1506](https://issues.apache.org/jira/browse/HDFS-1506) | Refactor fsimage loading code | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1486](https://issues.apache.org/jira/browse/HDFS-1486) | Generalize CLITest structure and interfaces to facilitate upstream adoption (e.g. for web testing) | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1481](https://issues.apache.org/jira/browse/HDFS-1481) | NameNode should validate fsimage before rolling | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1476](https://issues.apache.org/jira/browse/HDFS-1476) | listCorruptFileBlocks should be functional while the name node is still in safe mode | Major | namenode | Patrick Kling | Patrick Kling | +| [HDFS-1458](https://issues.apache.org/jira/browse/HDFS-1458) | Improve checkpoint performance by avoiding unnecessary image downloads | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1442](https://issues.apache.org/jira/browse/HDFS-1442) | Api to get delegation token in Hdfs | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1378](https://issues.apache.org/jira/browse/HDFS-1378) | Edit log replay should track and report file offsets in case of errors | Major | namenode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-1335](https://issues.apache.org/jira/browse/HDFS-1335) | HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode | Major | hdfs-client, namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1332](https://issues.apache.org/jira/browse/HDFS-1332) | When unable to place replicas, BlockPlacementPolicy should log reasons nodes were excluded | Minor | namenode | Todd Lipcon | Ted Yu | +| [HDFS-1120](https://issues.apache.org/jira/browse/HDFS-1120) | Make DataNode's block-to-device placement policy pluggable | Major | datanode | Jeff Hammerbacher | Harsh J | +| [HDFS-560](https://issues.apache.org/jira/browse/HDFS-560) | Proposed enhancements/tuning to hadoop-hdfs/build.xml | Minor | build | Steve Loughran | Steve Loughran | +| [HDFS-420](https://issues.apache.org/jira/browse/HDFS-420) | Fuse-dfs should cache fs handles | Major | fuse-dfs | Dima Brodsky | Brian Bockelman | +| [MAPREDUCE-3322](https://issues.apache.org/jira/browse/MAPREDUCE-3322) | Create a better index.html for maven docs | Major | documentation, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3275](https://issues.apache.org/jira/browse/MAPREDUCE-3275) | Add docs for WebAppProxy | Critical | documentation, mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3239](https://issues.apache.org/jira/browse/MAPREDUCE-3239) | Use new createSocketAddr API in MRv2 to give better error messages on misconfig | Minor | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3237](https://issues.apache.org/jira/browse/MAPREDUCE-3237) | Move LocalJobRunner to hadoop-mapreduce-client-core module | Major | client | Tom White | Tom White | +| [MAPREDUCE-3190](https://issues.apache.org/jira/browse/MAPREDUCE-3190) | bin/yarn should barf early if HADOOP\_COMMON\_HOME or HADOOP\_HDFS\_HOME are not set | Major | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3189](https://issues.apache.org/jira/browse/MAPREDUCE-3189) | Add link decoration back to MR2's CSS | Major | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3187](https://issues.apache.org/jira/browse/MAPREDUCE-3187) | Add names for various unnamed threads in MR2 | Minor | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3171](https://issues.apache.org/jira/browse/MAPREDUCE-3171) | normalize nodemanager native code compilation with common/hdfs native | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3162](https://issues.apache.org/jira/browse/MAPREDUCE-3162) | Separate application-init and container-init event types in NM's ApplicationImpl FSM | Minor | mrv2, nodemanager | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3161](https://issues.apache.org/jira/browse/MAPREDUCE-3161) | Improve javadoc and fix some typos in MR2 code | Minor | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3154](https://issues.apache.org/jira/browse/MAPREDUCE-3154) | Validate the Jobs Output Specification as the first statement in JobSubmitter.submitJobInternal(Job, Cluster) method | Major | client, mrv2 | Abhijit Suresh Shingate | Abhijit Suresh Shingate | +| [MAPREDUCE-3133](https://issues.apache.org/jira/browse/MAPREDUCE-3133) | Running a set of methods in a Single Test Class | Major | build | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3113](https://issues.apache.org/jira/browse/MAPREDUCE-3113) | the scripts yarn-daemon.sh and yarn are not working properly | Minor | mrv2 | XieXianshan | XieXianshan | +| [MAPREDUCE-3090](https://issues.apache.org/jira/browse/MAPREDUCE-3090) | Change MR AM to use ApplicationAttemptId rather than \ everywhere | Major | applicationmaster, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3014](https://issues.apache.org/jira/browse/MAPREDUCE-3014) | Rename and invert logic of '-cbuild' profile to 'native' and off by default | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3001](https://issues.apache.org/jira/browse/MAPREDUCE-3001) | Map Reduce JobHistory and AppMaster UI should have ability to display task specific counters. | Blocker | jobhistoryserver, mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2990](https://issues.apache.org/jira/browse/MAPREDUCE-2990) | Health Report on Resource Manager UI is null if the NM's are all healthy. | Blocker | mrv2 | Mahadev konar | Subroto Sanyal | +| [MAPREDUCE-2966](https://issues.apache.org/jira/browse/MAPREDUCE-2966) | Add ShutDown hooks for MRV2 processes | Major | applicationmaster, jobhistoryserver, nodemanager, resourcemanager | Abhijit Suresh Shingate | Abhijit Suresh Shingate | +| [MAPREDUCE-2961](https://issues.apache.org/jira/browse/MAPREDUCE-2961) | Increase the default threadpool size for container launching in the application master. | Blocker | mrv2 | Mahadev konar | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2930](https://issues.apache.org/jira/browse/MAPREDUCE-2930) | Generate state graph from the State Machine Definition | Major | mrv2 | Sharad Agarwal | Binglin Chang | +| [MAPREDUCE-2894](https://issues.apache.org/jira/browse/MAPREDUCE-2894) | Improvements to YARN apis | Blocker | mrv2 | Arun C Murthy | | +| [MAPREDUCE-2893](https://issues.apache.org/jira/browse/MAPREDUCE-2893) | Removing duplicate service provider in hadoop-mapreduce-client-jobclient | Trivial | client | Liang-Chi Hsieh | Liang-Chi Hsieh | +| [MAPREDUCE-2890](https://issues.apache.org/jira/browse/MAPREDUCE-2890) | Documentation for MRv2 | Blocker | documentation, mrv2 | Arun C Murthy | | +| [MAPREDUCE-2880](https://issues.apache.org/jira/browse/MAPREDUCE-2880) | Fix classpath construction for MRv2 | Blocker | mrv2 | Luke Lu | Arun C Murthy | +| [MAPREDUCE-2864](https://issues.apache.org/jira/browse/MAPREDUCE-2864) | Renaming of configuration property names in yarn | Major | jobhistoryserver, mrv2, nodemanager, resourcemanager | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2848](https://issues.apache.org/jira/browse/MAPREDUCE-2848) | Upgrade avro to 1.5.2 | Major | . | Luke Lu | Luke Lu | +| [MAPREDUCE-2805](https://issues.apache.org/jira/browse/MAPREDUCE-2805) | Update RAID for HDFS-2241 | Minor | contrib/raid | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2802](https://issues.apache.org/jira/browse/MAPREDUCE-2802) | [MR-279] Jobhistory filenames should have jobID to help in better parsing | Critical | mrv2 | Ramya Sunil | Jonathan Eagles | +| [MAPREDUCE-2726](https://issues.apache.org/jira/browse/MAPREDUCE-2726) | MR-279: Add the jobFile to the web UI | Blocker | mrv2 | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2707](https://issues.apache.org/jira/browse/MAPREDUCE-2707) | ProtoOverHadoopRpcEngine without using TunnelProtocol over WritableRpc | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-2701](https://issues.apache.org/jira/browse/MAPREDUCE-2701) | MR-279: app/Job.java needs UGI for the user that launched it | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2691](https://issues.apache.org/jira/browse/MAPREDUCE-2691) | Finish up the cleanup of distributed cache file resources and related tests. | Major | mrv2 | Amol Kekre | Siddharth Seth | +| [MAPREDUCE-2682](https://issues.apache.org/jira/browse/MAPREDUCE-2682) | Add a -classpath option to bin/mapred | Trivial | . | Arun C Murthy | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2680](https://issues.apache.org/jira/browse/MAPREDUCE-2680) | Enhance job-client cli to show queue information for running jobs | Minor | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2679](https://issues.apache.org/jira/browse/MAPREDUCE-2679) | MR-279: Merge MR-279 related minor patches into trunk | Trivial | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2676](https://issues.apache.org/jira/browse/MAPREDUCE-2676) | MR-279: JobHistory Job page needs reformatted | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2675](https://issues.apache.org/jira/browse/MAPREDUCE-2675) | MR-279: JobHistory Server main page needs to be reformatted | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2672](https://issues.apache.org/jira/browse/MAPREDUCE-2672) | MR-279: JobHistory Server needs Analysis this job | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2664](https://issues.apache.org/jira/browse/MAPREDUCE-2664) | MR 279: Implement JobCounters for MRv2 + Fix for Map Data Locality | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2629](https://issues.apache.org/jira/browse/MAPREDUCE-2629) | Class loading quirk prevents inner class method compilation | Minor | task | Eric Caspole | Eric Caspole | +| [MAPREDUCE-2624](https://issues.apache.org/jira/browse/MAPREDUCE-2624) | Update RAID for HDFS-2107 | Major | contrib/raid | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2623](https://issues.apache.org/jira/browse/MAPREDUCE-2623) | Update ClusterMapReduceTestCase to use MiniDFSCluster.Builder | Minor | test | Jim Plush | Harsh J | +| [MAPREDUCE-2611](https://issues.apache.org/jira/browse/MAPREDUCE-2611) | MR 279: Metrics, finishTimes, etc in JobHistory | Major | mrv2 | Siddharth Seth | | +| [MAPREDUCE-2602](https://issues.apache.org/jira/browse/MAPREDUCE-2602) | Allow setting of end-of-record delimiter for TextInputFormat (for the old API) | Major | . | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-2596](https://issues.apache.org/jira/browse/MAPREDUCE-2596) | Gridmix should notify job failures | Major | benchmarks, contrib/gridmix | Arun C Murthy | Amar Kamat | +| [MAPREDUCE-2580](https://issues.apache.org/jira/browse/MAPREDUCE-2580) | MR 279: RM UI should redirect finished jobs to History UI | Minor | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2551](https://issues.apache.org/jira/browse/MAPREDUCE-2551) | MR 279: Implement JobSummaryLog | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2501](https://issues.apache.org/jira/browse/MAPREDUCE-2501) | MR-279: Attach sources in builds | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2495](https://issues.apache.org/jira/browse/MAPREDUCE-2495) | The distributed cache cleanup thread has no monitoring to check to see if it has died for some reason | Minor | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2494](https://issues.apache.org/jira/browse/MAPREDUCE-2494) | Make the distributed cache delete entires using LRU priority | Major | distributed-cache | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2492](https://issues.apache.org/jira/browse/MAPREDUCE-2492) | [MAPREDUCE] The new MapReduce API should make available task's progress to the task | Major | task | Amar Kamat | Amar Kamat | +| [MAPREDUCE-2490](https://issues.apache.org/jira/browse/MAPREDUCE-2490) | Log blacklist debug count | Trivial | jobtracker | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2478](https://issues.apache.org/jira/browse/MAPREDUCE-2478) | MR 279: Improve history server | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2474](https://issues.apache.org/jira/browse/MAPREDUCE-2474) | Add docs to the new API Partitioner on how to access Job Configuration data | Minor | documentation | Harsh J | Harsh J | +| [MAPREDUCE-2469](https://issues.apache.org/jira/browse/MAPREDUCE-2469) | Task counters should also report the total heap usage of the task | Major | task | Amar Kamat | Amar Kamat | +| [MAPREDUCE-2462](https://issues.apache.org/jira/browse/MAPREDUCE-2462) | MR 279: Write job conf along with JobHistory, other minor improvements | Minor | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2459](https://issues.apache.org/jira/browse/MAPREDUCE-2459) | Cache HAR filesystem metadata | Major | harchive | Mac Yang | Mac Yang | +| [MAPREDUCE-2456](https://issues.apache.org/jira/browse/MAPREDUCE-2456) | Show the reducer taskid and map/reduce tasktrackers for "Failed fetch notification #\_ for task attempt..." log messages | Trivial | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2449](https://issues.apache.org/jira/browse/MAPREDUCE-2449) | Allow for command line arguments when performing "Run on Hadoop" action. | Minor | contrib/eclipse-plugin | Jeff Zemerick | Jeff Zemerick | +| [MAPREDUCE-2432](https://issues.apache.org/jira/browse/MAPREDUCE-2432) | MR-279: Install sanitized poms for downstream sanity | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2424](https://issues.apache.org/jira/browse/MAPREDUCE-2424) | MR-279: counters/UI/etc. for uber-AppMaster (in-cluster LocalJobRunner for MRv2) | Major | mrv2 | Greg Roelofs | Greg Roelofs | +| [MAPREDUCE-2414](https://issues.apache.org/jira/browse/MAPREDUCE-2414) | MR-279: Use generic interfaces for protocols | Major | mrv2 | Arun C Murthy | Siddharth Seth | +| [MAPREDUCE-2405](https://issues.apache.org/jira/browse/MAPREDUCE-2405) | MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2) | Major | mrv2 | Mahadev konar | Greg Roelofs | +| [MAPREDUCE-2403](https://issues.apache.org/jira/browse/MAPREDUCE-2403) | MR-279: Improve job history event handling in AM to log to HDFS | Major | mrv2 | Mahadev konar | Krishna Ramachandran | +| [MAPREDUCE-2399](https://issues.apache.org/jira/browse/MAPREDUCE-2399) | The embedded web framework for MAPREDUCE-279 | Major | . | Arun C Murthy | Luke Lu | +| [MAPREDUCE-2381](https://issues.apache.org/jira/browse/MAPREDUCE-2381) | JobTracker instrumentation not consistent about error handling | Major | . | Philip Zeyliger | Philip Zeyliger | +| [MAPREDUCE-2367](https://issues.apache.org/jira/browse/MAPREDUCE-2367) | Allow using a file to exclude certain tests from build | Minor | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2351](https://issues.apache.org/jira/browse/MAPREDUCE-2351) | mapred.job.tracker.history.completed.location should support an arbitrary filesystem URI | Major | . | Tom White | Tom White | +| [MAPREDUCE-2326](https://issues.apache.org/jira/browse/MAPREDUCE-2326) | Port gridmix changes from hadoop-0.20.100 to trunk | Major | . | Arun C Murthy | | +| [MAPREDUCE-2302](https://issues.apache.org/jira/browse/MAPREDUCE-2302) | Add static factory methods in GaloisField | Major | contrib/raid | Scott Chen | Scott Chen | +| [MAPREDUCE-2263](https://issues.apache.org/jira/browse/MAPREDUCE-2263) | MapReduce side of HADOOP-6904 | Major | . | Hairong Kuang | Hairong Kuang | +| [MAPREDUCE-2260](https://issues.apache.org/jira/browse/MAPREDUCE-2260) | Remove auto-generated native build files | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [MAPREDUCE-2254](https://issues.apache.org/jira/browse/MAPREDUCE-2254) | Allow setting of end-of-record delimiter for TextInputFormat | Major | . | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-2250](https://issues.apache.org/jira/browse/MAPREDUCE-2250) | Fix logging in raid code. | Trivial | contrib/raid | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2249](https://issues.apache.org/jira/browse/MAPREDUCE-2249) | Better to check the reflexive property of the object while overriding equals method of it | Major | . | Bhallamudi Venkata Siva Kamesh | Devaraj K | +| [MAPREDUCE-2248](https://issues.apache.org/jira/browse/MAPREDUCE-2248) | DistributedRaidFileSystem should unraid only the corrupt block | Major | . | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-2243](https://issues.apache.org/jira/browse/MAPREDUCE-2243) | Close all the file streams propely in a finally block to avoid their leakage. | Minor | jobtracker, tasktracker | Bhallamudi Venkata Siva Kamesh | Devaraj K | +| [MAPREDUCE-2239](https://issues.apache.org/jira/browse/MAPREDUCE-2239) | BlockPlacementPolicyRaid should call getBlockLocations only when necessary | Major | contrib/raid | Scott Chen | Scott Chen | +| [MAPREDUCE-2225](https://issues.apache.org/jira/browse/MAPREDUCE-2225) | MultipleOutputs should not require the use of 'Writable' | Blocker | job submission | Harsh J | Harsh J | +| [MAPREDUCE-2207](https://issues.apache.org/jira/browse/MAPREDUCE-2207) | Task-cleanup task should not be scheduled on the node that the task just failed | Major | jobtracker | Scott Chen | Liyin Liang | +| [MAPREDUCE-2206](https://issues.apache.org/jira/browse/MAPREDUCE-2206) | The task-cleanup tasks should be optional | Major | jobtracker | Scott Chen | Scott Chen | +| [MAPREDUCE-2203](https://issues.apache.org/jira/browse/MAPREDUCE-2203) | Wong javadoc for TaskRunner's appendJobJarClasspaths method | Trivial | . | Jingguo Yao | Jingguo Yao | +| [MAPREDUCE-2202](https://issues.apache.org/jira/browse/MAPREDUCE-2202) | Generalize CLITest structure and interfaces to facilitate upstream adoption (e.g. for web or system testing) | Major | . | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2156](https://issues.apache.org/jira/browse/MAPREDUCE-2156) | Raid-aware FSCK | Major | contrib/raid | Patrick Kling | Patrick Kling | +| [MAPREDUCE-2155](https://issues.apache.org/jira/browse/MAPREDUCE-2155) | RaidNode should optionally dispatch map reduce jobs to fix corrupt blocks (instead of fixing locally) | Major | contrib/raid | Patrick Kling | Patrick Kling | +| [MAPREDUCE-2153](https://issues.apache.org/jira/browse/MAPREDUCE-2153) | Bring in more job configuration properties in to the trace file | Major | tools/rumen | Ravi Gummadi | Rajesh Balamohan | +| [MAPREDUCE-2107](https://issues.apache.org/jira/browse/MAPREDUCE-2107) | Emulate Memory Usage of Tasks in GridMix3 | Major | contrib/gridmix | Ranjit Mathew | Amar Kamat | +| [MAPREDUCE-2106](https://issues.apache.org/jira/browse/MAPREDUCE-2106) | Emulate CPU Usage of Tasks in GridMix3 | Major | contrib/gridmix | Ranjit Mathew | Amar Kamat | +| [MAPREDUCE-2105](https://issues.apache.org/jira/browse/MAPREDUCE-2105) | Simulate Load Incrementally and Adaptively in GridMix3 | Major | contrib/gridmix | Ranjit Mathew | Amar Kamat | +| [MAPREDUCE-2026](https://issues.apache.org/jira/browse/MAPREDUCE-2026) | JobTracker.getJobCounters() should not hold JobTracker lock while calling JobInProgress.getCounters() | Major | . | Scott Chen | Joydeep Sen Sarma | +| [MAPREDUCE-1978](https://issues.apache.org/jira/browse/MAPREDUCE-1978) | [Rumen] TraceBuilder should provide recursive input folder scanning | Major | tools/rumen | Amar Kamat | Ravi Gummadi | +| [MAPREDUCE-1906](https://issues.apache.org/jira/browse/MAPREDUCE-1906) | Lower default minimum heartbeat interval for tasktracker \> Jobtracker | Major | jobtracker, performance, tasktracker | Scott Carey | Todd Lipcon | +| [MAPREDUCE-1831](https://issues.apache.org/jira/browse/MAPREDUCE-1831) | BlockPlacement policy for RAID | Major | contrib/raid | Scott Chen | Scott Chen | +| [MAPREDUCE-1783](https://issues.apache.org/jira/browse/MAPREDUCE-1783) | Task Initialization should be delayed till when a job can be run | Major | contrib/fair-share | Ramkumar Vadali | Ramkumar Vadali | +| [MAPREDUCE-1752](https://issues.apache.org/jira/browse/MAPREDUCE-1752) | Implement getFileBlockLocations in HarFilesystem | Major | harchive | Dmytro Molkov | Dmytro Molkov | +| [MAPREDUCE-1706](https://issues.apache.org/jira/browse/MAPREDUCE-1706) | Log RAID recoveries on HDFS | Major | contrib/raid | Rodrigo Schmidt | Scott Chen | +| [MAPREDUCE-1702](https://issues.apache.org/jira/browse/MAPREDUCE-1702) | CPU/Memory emulation for GridMix3 | Minor | contrib/gridmix | Jaideep | | +| [MAPREDUCE-1624](https://issues.apache.org/jira/browse/MAPREDUCE-1624) | Document the job credentials and associated details to do with delegation tokens (on the client side) | Major | documentation | Devaraj Das | Devaraj Das | +| [MAPREDUCE-1461](https://issues.apache.org/jira/browse/MAPREDUCE-1461) | Feature to instruct rumen-folder utility to skip jobs worth of specific duration | Major | tools/rumen | Rajesh Balamohan | Rajesh Balamohan | +| [MAPREDUCE-1207](https://issues.apache.org/jira/browse/MAPREDUCE-1207) | Allow admins to set java options for map/reduce tasks | Blocker | client, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-1159](https://issues.apache.org/jira/browse/MAPREDUCE-1159) | Limit Job name on jobtracker.jsp to be 80 char long | Trivial | . | Zheng Shao | Harsh J | +| [MAPREDUCE-901](https://issues.apache.org/jira/browse/MAPREDUCE-901) | Move Framework Counters into a TaskMetric structure | Major | task | Owen O'Malley | Luke Lu | +| [MAPREDUCE-279](https://issues.apache.org/jira/browse/MAPREDUCE-279) | Map-Reduce 2.0 | Major | mrv2 | Arun C Murthy | | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9128](https://issues.apache.org/jira/browse/HADOOP-9128) | MetricsDynamicMBeanBase can cause high cpu load | Major | metrics | Nate Putnam | | +| [HADOOP-8389](https://issues.apache.org/jira/browse/HADOOP-8389) | MetricsDynamicMBeanBase throws IllegalArgumentException for empty attribute list | Major | metrics | Elias Ross | | +| [HADOOP-7798](https://issues.apache.org/jira/browse/HADOOP-7798) | Release artifacts need to be signed for Nexus | Blocker | build | Arun C Murthy | Doug Cutting | +| [HADOOP-7797](https://issues.apache.org/jira/browse/HADOOP-7797) | Fix the repository name to support pushing to the staging area of Nexus | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-7782](https://issues.apache.org/jira/browse/HADOOP-7782) | Aggregate project javadocs | Critical | build | Arun C Murthy | Tom White | +| [HADOOP-7778](https://issues.apache.org/jira/browse/HADOOP-7778) | FindBugs warning in Token.getKind() | Major | . | Tom White | Tom White | +| [HADOOP-7771](https://issues.apache.org/jira/browse/HADOOP-7771) | NPE when running hdfs dfs -copyToLocal, -get etc | Blocker | . | John George | John George | +| [HADOOP-7770](https://issues.apache.org/jira/browse/HADOOP-7770) | ViewFS getFileChecksum throws FileNotFoundException for files in /tmp and /user | Blocker | viewfs | Ravi Prakash | Ravi Prakash | +| [HADOOP-7768](https://issues.apache.org/jira/browse/HADOOP-7768) | PreCommit-HADOOP-Build is failing on hadoop-auth-examples | Blocker | build | Jonathan Eagles | Tom White | +| [HADOOP-7766](https://issues.apache.org/jira/browse/HADOOP-7766) | The auth to local mappings are not being respected, with webhdfs and security enabled. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7764](https://issues.apache.org/jira/browse/HADOOP-7764) | Allow both ACL list and global path spec filters to HttpServer | Blocker | . | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7755](https://issues.apache.org/jira/browse/HADOOP-7755) | Detect MapReduce PreCommit Trunk builds silently failing when running test-patch.sh | Blocker | build | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7745](https://issues.apache.org/jira/browse/HADOOP-7745) | I switched variable names in HADOOP-7509 | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7744](https://issues.apache.org/jira/browse/HADOOP-7744) | Incorrect exit code for hadoop-core-test tests when exception thrown | Major | test | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7740](https://issues.apache.org/jira/browse/HADOOP-7740) | security audit logger is not on by default, fix the log4j properties to enable the logger | Minor | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7728](https://issues.apache.org/jira/browse/HADOOP-7728) | hadoop-setup-conf.sh should be modified to enable task memory manager | Major | conf | Ramya Sunil | Ramya Sunil | +| [HADOOP-7724](https://issues.apache.org/jira/browse/HADOOP-7724) | hadoop-setup-conf.sh should put proxy user info into the core-site.xml | Major | . | Giridharan Kesavan | Arpit Gupta | +| [HADOOP-7721](https://issues.apache.org/jira/browse/HADOOP-7721) | dfs.web.authentication.kerberos.principal expects the full hostname and does not replace \_HOST with the hostname | Major | . | Arpit Gupta | Jitendra Nath Pandey | +| [HADOOP-7715](https://issues.apache.org/jira/browse/HADOOP-7715) | see log4j Error when running mr jobs and certain dfs calls | Major | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7711](https://issues.apache.org/jira/browse/HADOOP-7711) | hadoop-env.sh generated from templates has duplicate info | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7708](https://issues.apache.org/jira/browse/HADOOP-7708) | config generator does not update the properties file if on exists already | Critical | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7691](https://issues.apache.org/jira/browse/HADOOP-7691) | hadoop deb pkg should take a diff group id | Major | . | Giridharan Kesavan | Eric Yang | +| [HADOOP-7684](https://issues.apache.org/jira/browse/HADOOP-7684) | jobhistory server and secondarynamenode should have init.d script | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7681](https://issues.apache.org/jira/browse/HADOOP-7681) | log4j.properties is missing properties for security audit and hdfs audit should be changed to info | Minor | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7671](https://issues.apache.org/jira/browse/HADOOP-7671) | Add license headers to hadoop-common/src/main/packages/templates/conf/ | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7663](https://issues.apache.org/jira/browse/HADOOP-7663) | TestHDFSTrash failing on 22 | Major | test | Mayank Bansal | Mayank Bansal | +| [HADOOP-7662](https://issues.apache.org/jira/browse/HADOOP-7662) | logs servlet should use pathspec of /\* | Major | . | Thomas Graves | Thomas Graves | +| [HADOOP-7658](https://issues.apache.org/jira/browse/HADOOP-7658) | to fix hadoop config template | Major | . | Giridharan Kesavan | Eric Yang | +| [HADOOP-7639](https://issues.apache.org/jira/browse/HADOOP-7639) | yarn ui not properly filtered in HttpServer | Major | . | Thomas Graves | Thomas Graves | +| [HADOOP-7637](https://issues.apache.org/jira/browse/HADOOP-7637) | Fair scheduler configuration file is not bundled in RPM | Major | build | Eric Yang | Eric Yang | +| [HADOOP-7633](https://issues.apache.org/jira/browse/HADOOP-7633) | log4j.properties should be added to the hadoop conf on deploy | Major | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7631](https://issues.apache.org/jira/browse/HADOOP-7631) | In mapred-site.xml, stream.tmpdir is mapped to ${mapred.temp.dir} which is undeclared. | Major | conf | Ramya Sunil | Eric Yang | +| [HADOOP-7630](https://issues.apache.org/jira/browse/HADOOP-7630) | hadoop-metrics2.properties should have a property \*.period set to a default value foe metrics | Major | conf | Arpit Gupta | Eric Yang | +| [HADOOP-7629](https://issues.apache.org/jira/browse/HADOOP-7629) | regression with MAPREDUCE-2289 - setPermission passed immutable FsPermission (rpc failure) | Major | . | Patrick Hunt | Todd Lipcon | +| [HADOOP-7626](https://issues.apache.org/jira/browse/HADOOP-7626) | Allow overwrite of HADOOP\_CLASSPATH and HADOOP\_OPTS | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7610](https://issues.apache.org/jira/browse/HADOOP-7610) | /etc/profile.d does not exist on Debian | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7608](https://issues.apache.org/jira/browse/HADOOP-7608) | SnappyCodec check for Hadoop native lib is wrong | Major | io | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7606](https://issues.apache.org/jira/browse/HADOOP-7606) | Upgrade Jackson to version 1.7.1 to match the version required by Jersey | Major | test | Aaron T. Myers | Alejandro Abdelnur | +| [HADOOP-7604](https://issues.apache.org/jira/browse/HADOOP-7604) | Hadoop Auth examples pom in 0.23 point to 0.24 versions. | Critical | . | Mahadev konar | Mahadev konar | +| [HADOOP-7603](https://issues.apache.org/jira/browse/HADOOP-7603) | Set default hdfs, mapred uid, and hadoop group gid for RPM packages | Major | . | Eric Yang | Eric Yang | +| [HADOOP-7599](https://issues.apache.org/jira/browse/HADOOP-7599) | Improve hadoop setup conf script to setup secure Hadoop cluster | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7598](https://issues.apache.org/jira/browse/HADOOP-7598) | smart-apply-patch.sh does not handle patching from a sub directory correctly. | Major | build | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-7593](https://issues.apache.org/jira/browse/HADOOP-7593) | AssertionError in TestHttpServer.testMaxThreads() | Major | test | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HADOOP-7589](https://issues.apache.org/jira/browse/HADOOP-7589) | Prefer mvn test -DskipTests over mvn compile in test-patch.sh | Major | build | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-7580](https://issues.apache.org/jira/browse/HADOOP-7580) | Add a version of getLocalPathForWrite to LocalDirAllocator which doesn't create dirs | Major | . | Siddharth Seth | Siddharth Seth | +| [HADOOP-7578](https://issues.apache.org/jira/browse/HADOOP-7578) | Fix test-patch to be able to run on MR patches. | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-7576](https://issues.apache.org/jira/browse/HADOOP-7576) | Fix findbugs warnings in Hadoop Auth (Alfredo) | Major | security | Tom White | Tsz Wo Nicholas Sze | +| [HADOOP-7575](https://issues.apache.org/jira/browse/HADOOP-7575) | Support fully qualified paths as part of LocalDirAllocator | Minor | fs | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7568](https://issues.apache.org/jira/browse/HADOOP-7568) | SequenceFile should not print into stdout | Major | io | Konstantin Shvachko | Plamen Jeliazkov | +| [HADOOP-7566](https://issues.apache.org/jira/browse/HADOOP-7566) | MR tests are failing webapps/hdfs not found in CLASSPATH | Major | . | Mahadev konar | Alejandro Abdelnur | +| [HADOOP-7563](https://issues.apache.org/jira/browse/HADOOP-7563) | hadoop-config.sh setup CLASSPATH, HADOOP\_HDFS\_HOME and HADOOP\_MAPRED\_HOME incorrectly | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7545](https://issues.apache.org/jira/browse/HADOOP-7545) | common -tests jar should not include properties and configs | Critical | build, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-7536](https://issues.apache.org/jira/browse/HADOOP-7536) | Correct the dependency version regressions introduced in HADOOP-6671 | Major | build | Kihwal Lee | Alejandro Abdelnur | +| [HADOOP-7529](https://issues.apache.org/jira/browse/HADOOP-7529) | Possible deadlock in metrics2 | Critical | metrics | Todd Lipcon | Luke Lu | +| [HADOOP-7523](https://issues.apache.org/jira/browse/HADOOP-7523) | Test org.apache.hadoop.fs.TestFilterFileSystem fails due to java.lang.NoSuchMethodException | Blocker | test | John Lee | John Lee | +| [HADOOP-7520](https://issues.apache.org/jira/browse/HADOOP-7520) | hadoop-main fails to deploy | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7499](https://issues.apache.org/jira/browse/HADOOP-7499) | Add method for doing a sanity check on hostnames in NetUtils | Major | util | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [HADOOP-7471](https://issues.apache.org/jira/browse/HADOOP-7471) | the saveVersion.sh script sometimes fails to extract SVN URL | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7442](https://issues.apache.org/jira/browse/HADOOP-7442) | Docs in core-default.xml still reference deprecated config "topology.script.file.name" | Major | conf, documentation | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7440](https://issues.apache.org/jira/browse/HADOOP-7440) | HttpServer.getParameterValues throws NPE for missing parameters | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7437](https://issues.apache.org/jira/browse/HADOOP-7437) | IOUtils.copybytes will suppress the stream closure exceptions. | Major | io | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7428](https://issues.apache.org/jira/browse/HADOOP-7428) | IPC connection is orphaned with null 'out' member | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7419](https://issues.apache.org/jira/browse/HADOOP-7419) | new hadoop-config.sh doesn't manage classpath for HADOOP\_CONF\_DIR correctly | Major | . | Todd Lipcon | Bing Zheng | +| [HADOOP-7402](https://issues.apache.org/jira/browse/HADOOP-7402) | TestConfiguration doesn't clean up after itself | Trivial | test | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7389](https://issues.apache.org/jira/browse/HADOOP-7389) | Use of TestingGroups by tests causes subsequent tests to fail | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7385](https://issues.apache.org/jira/browse/HADOOP-7385) | Remove StringUtils.stringifyException(ie) in logger functions | Minor | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HADOOP-7383](https://issues.apache.org/jira/browse/HADOOP-7383) | HDFS needs to export protobuf library dependency in pom | Blocker | build | Todd Lipcon | Todd Lipcon | +| [HADOOP-7377](https://issues.apache.org/jira/browse/HADOOP-7377) | Fix command name handling affecting DFSAdmin | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7357](https://issues.apache.org/jira/browse/HADOOP-7357) | hadoop.io.compress.TestCodec#main() should exit with non-zero exit code if test failed | Trivial | test | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-7356](https://issues.apache.org/jira/browse/HADOOP-7356) | RPM packages broke bin/hadoop script for hadoop 0.20.205 | Blocker | . | Eric Yang | Eric Yang | +| [HADOOP-7353](https://issues.apache.org/jira/browse/HADOOP-7353) | Cleanup FsShell and prevent masking of RTE stacktraces | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7342](https://issues.apache.org/jira/browse/HADOOP-7342) | Add an utility API in FileUtil for JDK File.list | Minor | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HADOOP-7341](https://issues.apache.org/jira/browse/HADOOP-7341) | Fix option parsing in CommandFormat | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7336](https://issues.apache.org/jira/browse/HADOOP-7336) | TestFileContextResolveAfs will fail with default test.build.data property. | Minor | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7327](https://issues.apache.org/jira/browse/HADOOP-7327) | FileSystem.listStatus() throws NullPointerException instead of IOException upon access permission failure | Minor | fs | Matt Foley | Matt Foley | +| [HADOOP-7324](https://issues.apache.org/jira/browse/HADOOP-7324) | Ganglia plugins for metrics v2 | Blocker | metrics | Luke Lu | Priyo Mustafi | +| [HADOOP-7322](https://issues.apache.org/jira/browse/HADOOP-7322) | Adding a util method in FileUtil for JDK File.listFiles | Minor | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HADOOP-7292](https://issues.apache.org/jira/browse/HADOOP-7292) | Metrics 2 TestSinkQueue is racy | Minor | metrics | Luke Lu | Luke Lu | +| [HADOOP-7287](https://issues.apache.org/jira/browse/HADOOP-7287) | Configuration deprecation mechanism doesn't work properly for GenericOptionsParser/Tools | Blocker | conf | Todd Lipcon | Aaron T. Myers | +| [HADOOP-7284](https://issues.apache.org/jira/browse/HADOOP-7284) | Trash and shell's rm does not work for viewfs | Major | viewfs | Sanjay Radia | Sanjay Radia | +| [HADOOP-7282](https://issues.apache.org/jira/browse/HADOOP-7282) | getRemoteIp could return null in cases where the call is ongoing but the ip went away. | Major | ipc | John George | John George | +| [HADOOP-7276](https://issues.apache.org/jira/browse/HADOOP-7276) | Hadoop native builds fail on ARM due to -m32 | Major | native | Trevor Robinson | Trevor Robinson | +| [HADOOP-7268](https://issues.apache.org/jira/browse/HADOOP-7268) | FileContext.getLocalFSFileContext() behavior needs to be fixed w.r.t tokens | Major | fs, security | Devaraj Das | Jitendra Nath Pandey | +| [HADOOP-7261](https://issues.apache.org/jira/browse/HADOOP-7261) | Disable IPV6 for junit tests | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7259](https://issues.apache.org/jira/browse/HADOOP-7259) | contrib modules should include build.properties from parent. | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-7258](https://issues.apache.org/jira/browse/HADOOP-7258) | Gzip codec should not return null decompressors | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-7231](https://issues.apache.org/jira/browse/HADOOP-7231) | Fix synopsis for -count | Major | util | Daryn Sharp | Daryn Sharp | +| [HADOOP-7223](https://issues.apache.org/jira/browse/HADOOP-7223) | FileContext createFlag combinations during create are not clearly defined | Major | fs | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7216](https://issues.apache.org/jira/browse/HADOOP-7216) | HADOOP-7202 broke TestDFSShell in HDFS | Major | test | Aaron T. Myers | Daryn Sharp | +| [HADOOP-7215](https://issues.apache.org/jira/browse/HADOOP-7215) | RPC clients must connect over a network interface corresponding to the host name in the client's kerberos principal key | Blocker | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7210](https://issues.apache.org/jira/browse/HADOOP-7210) | Chown command is not working from FSShell. | Major | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7208](https://issues.apache.org/jira/browse/HADOOP-7208) | equals() and hashCode() implementation need to change in StandardSocketFactory | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7207](https://issues.apache.org/jira/browse/HADOOP-7207) | fs member of FSShell is not really needed | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HADOOP-7194](https://issues.apache.org/jira/browse/HADOOP-7194) | Potential Resource leak in IOUtils.java | Major | io | Devaraj K | Devaraj K | +| [HADOOP-7187](https://issues.apache.org/jira/browse/HADOOP-7187) | Socket Leak in org.apache.hadoop.metrics.ganglia.GangliaContext | Major | metrics | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7178](https://issues.apache.org/jira/browse/HADOOP-7178) | FileSystem should have an option to control the .crc file creations at Local. | Major | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7176](https://issues.apache.org/jira/browse/HADOOP-7176) | Redesign FsShell | Major | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-7175](https://issues.apache.org/jira/browse/HADOOP-7175) | Add isEnabled() to Trash | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7174](https://issues.apache.org/jira/browse/HADOOP-7174) | null is displayed in the console,if the src path is invalid while doing copyToLocal operation from commandLine | Minor | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7172](https://issues.apache.org/jira/browse/HADOOP-7172) | SecureIO should not check owner on non-secure clusters that have no native support | Critical | io, security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7171](https://issues.apache.org/jira/browse/HADOOP-7171) | Support UGI in FileContext API | Major | security | Owen O'Malley | Jitendra Nath Pandey | +| [HADOOP-7162](https://issues.apache.org/jira/browse/HADOOP-7162) | FsShell: call srcFs.listStatus(src) twice | Minor | fs | Alexey Diomin | Alexey Diomin | +| [HADOOP-7151](https://issues.apache.org/jira/browse/HADOOP-7151) | Document need for stable hashCode() in WritableComparable | Minor | . | Dmitriy V. Ryaboy | Dmitriy V. Ryaboy | +| [HADOOP-7120](https://issues.apache.org/jira/browse/HADOOP-7120) | 200 new Findbugs warnings | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7111](https://issues.apache.org/jira/browse/HADOOP-7111) | Several TFile tests failing when native libraries are present | Critical | io | Todd Lipcon | Aaron T. Myers | +| [HADOOP-7101](https://issues.apache.org/jira/browse/HADOOP-7101) | UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context | Blocker | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7098](https://issues.apache.org/jira/browse/HADOOP-7098) | tasktracker property not set in conf/hadoop-env.sh | Major | conf | Bernd Fondermann | Bernd Fondermann | +| [HADOOP-7090](https://issues.apache.org/jira/browse/HADOOP-7090) | Possible resource leaks in hadoop core code | Major | fs/s3, io | Gokul | Uma Maheswara Rao G | +| [HADOOP-7089](https://issues.apache.org/jira/browse/HADOOP-7089) | Fix link resolution logic in hadoop-config.sh | Minor | scripts | Eli Collins | Eli Collins | +| [HADOOP-7071](https://issues.apache.org/jira/browse/HADOOP-7071) | test-patch.sh has bad ps arg | Minor | build | Nigel Daley | Nigel Daley | +| [HADOOP-7057](https://issues.apache.org/jira/browse/HADOOP-7057) | IOUtils.readFully and IOUtils.skipFully have typo in exception creation's message | Minor | util | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-7055](https://issues.apache.org/jira/browse/HADOOP-7055) | Update of commons logging libraries causes EventCounter to count logging events incorrectly | Major | metrics | Jingguo Yao | Jingguo Yao | +| [HADOOP-7053](https://issues.apache.org/jira/browse/HADOOP-7053) | wrong FSNamesystem Audit logging setting in conf/log4j.properties | Minor | conf | Jingguo Yao | Jingguo Yao | +| [HADOOP-7052](https://issues.apache.org/jira/browse/HADOOP-7052) | misspelling of threshold in conf/log4j.properties | Major | conf | Jingguo Yao | Jingguo Yao | +| [HADOOP-7046](https://issues.apache.org/jira/browse/HADOOP-7046) | 1 Findbugs warning on trunk and branch-0.22 | Blocker | security | Nigel Daley | Po Cheung | +| [HADOOP-7045](https://issues.apache.org/jira/browse/HADOOP-7045) | TestDU fails on systems with local file systems with extended attributes | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-7015](https://issues.apache.org/jira/browse/HADOOP-7015) | RawLocalFileSystem#listStatus does not deal with a directory whose entries are changing ( e.g. in a multi-thread or multi-process environment) | Minor | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-6939](https://issues.apache.org/jira/browse/HADOOP-6939) | Inconsistent lock ordering in AbstractDelegationTokenSecretManager | Minor | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-6912](https://issues.apache.org/jira/browse/HADOOP-6912) | Guard against NPE when calling UGI.isLoginKeytabBased() | Major | security | Kan Zhang | Kan Zhang | +| [HADOOP-6754](https://issues.apache.org/jira/browse/HADOOP-6754) | DefaultCodec.createOutputStream() leaks memory | Major | io | Aaron Kimball | Aaron Kimball | +| [HADOOP-6622](https://issues.apache.org/jira/browse/HADOOP-6622) | Token should not print the password in toString. | Major | security | Jitendra Nath Pandey | Eli Collins | +| [HADOOP-6508](https://issues.apache.org/jira/browse/HADOOP-6508) | Incorrect values for metrics with CompositeContext | Major | metrics | Amareshwari Sriramadasu | Luke Lu | +| [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp | Major | test | Ravi Gummadi | Ravi Gummadi | +| [HDFS-2491](https://issues.apache.org/jira/browse/HDFS-2491) | TestBalancer can fail when datanode utilization and avgUtilization is exactly same. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2467](https://issues.apache.org/jira/browse/HDFS-2467) | HftpFileSystem uses incorrect compare for finding delegation tokens | Major | . | Owen O'Malley | Owen O'Malley | +| [HDFS-2452](https://issues.apache.org/jira/browse/HDFS-2452) | OutOfMemoryError in DataXceiverServer takes down the DataNode | Major | datanode | Konstantin Shvachko | Uma Maheswara Rao G | +| [HDFS-2445](https://issues.apache.org/jira/browse/HDFS-2445) | Incorrect exit code for hadoop-hdfs-test tests when exception thrown | Major | test | Jonathan Eagles | Jonathan Eagles | +| [HDFS-2436](https://issues.apache.org/jira/browse/HDFS-2436) | FSNamesystem.setTimes(..) expects the path is a file. | Major | . | Arpit Gupta | Uma Maheswara Rao G | +| [HDFS-2422](https://issues.apache.org/jira/browse/HDFS-2422) | The NN should tolerate the same number of low-resource volumes as failed volumes | Major | namenode | Jeff Bean | Aaron T. Myers | +| [HDFS-2414](https://issues.apache.org/jira/browse/HDFS-2414) | TestDFSRollback fails intermittently | Critical | namenode, test | Robert Joseph Evans | Todd Lipcon | +| [HDFS-2412](https://issues.apache.org/jira/browse/HDFS-2412) | Add backwards-compatibility layer for FSConstants | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HDFS-2411](https://issues.apache.org/jira/browse/HDFS-2411) | with webhdfs enabled in secure mode the auth to local mappings are not being respected. | Major | webhdfs | Arpit Gupta | Jitendra Nath Pandey | +| [HDFS-2409](https://issues.apache.org/jira/browse/HDFS-2409) | \_HOST in dfs.web.authentication.kerberos.principal. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2368](https://issues.apache.org/jira/browse/HDFS-2368) | defaults created for web keytab and principal, these properties should not have defaults | Major | . | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2361](https://issues.apache.org/jira/browse/HDFS-2361) | hftp is broken | Critical | namenode | Rajit Saha | Jitendra Nath Pandey | +| [HDFS-2347](https://issues.apache.org/jira/browse/HDFS-2347) | checkpointTxnCount's comment still saying about editlog size | Trivial | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2346](https://issues.apache.org/jira/browse/HDFS-2346) | TestHost2NodesMap & TestReplicasMap will fail depending upon execution order of test methods | Blocker | test | Uma Maheswara Rao G | Laxman | +| [HDFS-2344](https://issues.apache.org/jira/browse/HDFS-2344) | Fix the TestOfflineEditsViewer test failure in 0.23 branch | Major | test | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2333](https://issues.apache.org/jira/browse/HDFS-2333) | HDFS-2284 introduced 2 findbugs warnings on trunk | Major | . | Ivan Kelly | Tsz Wo Nicholas Sze | +| [HDFS-2331](https://issues.apache.org/jira/browse/HDFS-2331) | Hdfs compilation fails | Major | hdfs-client | Abhijit Suresh Shingate | Abhijit Suresh Shingate | +| [HDFS-2323](https://issues.apache.org/jira/browse/HDFS-2323) | start-dfs.sh script fails for tarball install | Major | . | Tom White | Tom White | +| [HDFS-2322](https://issues.apache.org/jira/browse/HDFS-2322) | the build fails in Windows because commons-daemon TAR cannot be fetched | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2314](https://issues.apache.org/jira/browse/HDFS-2314) | MRV1 test compilation broken after HDFS-2197 | Major | test | Vinod Kumar Vavilapalli | Todd Lipcon | +| [HDFS-2290](https://issues.apache.org/jira/browse/HDFS-2290) | Block with corrupt replica is not getting replicated | Major | namenode | Konstantin Shvachko | Benoy Antony | +| [HDFS-2289](https://issues.apache.org/jira/browse/HDFS-2289) | jsvc isn't part of the artifact | Blocker | . | Arun C Murthy | Alejandro Abdelnur | +| [HDFS-2267](https://issues.apache.org/jira/browse/HDFS-2267) | DataXceiver thread name incorrect while waiting on op during keepalive | Trivial | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2258](https://issues.apache.org/jira/browse/HDFS-2258) | TestLeaseRecovery2 fails as lease hard limit is not reset to default | Major | namenode, test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2245](https://issues.apache.org/jira/browse/HDFS-2245) | BlockManager.chooseTarget(..) throws NPE | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2240](https://issues.apache.org/jira/browse/HDFS-2240) | Possible deadlock between LeaseRenewer and its factory | Critical | hdfs-client | Todd Lipcon | Tsz Wo Nicholas Sze | +| [HDFS-2235](https://issues.apache.org/jira/browse/HDFS-2235) | Encode servlet paths | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-2232](https://issues.apache.org/jira/browse/HDFS-2232) | TestHDFSCLI fails on 0.22 branch | Blocker | test | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-2229](https://issues.apache.org/jira/browse/HDFS-2229) | Deadlock in NameNode | Blocker | namenode | Vinod Kumar Vavilapalli | Tsz Wo Nicholas Sze | +| [HDFS-2186](https://issues.apache.org/jira/browse/HDFS-2186) | DN volume failures on startup are not counted | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-2156](https://issues.apache.org/jira/browse/HDFS-2156) | rpm should only require the same major version as common | Major | . | Owen O'Malley | Eric Yang | +| [HDFS-2153](https://issues.apache.org/jira/browse/HDFS-2153) | DFSClientAdapter should be put under test | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2132](https://issues.apache.org/jira/browse/HDFS-2132) | Potential resource leak in EditLogFileOutputStream.close | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2114](https://issues.apache.org/jira/browse/HDFS-2114) | re-commission of a decommissioned node does not delete excess replica | Major | . | John George | John George | +| [HDFS-2109](https://issues.apache.org/jira/browse/HDFS-2109) | Store uMask as member variable to DFSClient.Conf | Major | hdfs-client | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-2092](https://issues.apache.org/jira/browse/HDFS-2092) | Create a light inner conf class in DFSClient | Major | hdfs-client | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-2086](https://issues.apache.org/jira/browse/HDFS-2086) | If the include hosts list contains host name, after restarting namenode, datanodes registrant is denied | Major | namenode | Tanping Wang | Tanping Wang | +| [HDFS-2082](https://issues.apache.org/jira/browse/HDFS-2082) | SecondaryNameNode web interface doesn't show the right info | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2067](https://issues.apache.org/jira/browse/HDFS-2067) | Bump DATA\_TRANSFER\_VERSION in trunk for protobufs | Major | datanode, hdfs-client | Todd Lipcon | Tsz Wo Nicholas Sze | +| [HDFS-2065](https://issues.apache.org/jira/browse/HDFS-2065) | Fix NPE in DFSClient.getFileChecksum | Major | . | Bharath Mundlapudi | Uma Maheswara Rao G | +| [HDFS-2061](https://issues.apache.org/jira/browse/HDFS-2061) | two minor bugs in BlockManager block report processing | Minor | namenode | Matt Foley | Matt Foley | +| [HDFS-2053](https://issues.apache.org/jira/browse/HDFS-2053) | Bug in INodeDirectory#computeContentSummary warning | Minor | namenode | Michael Noll | Michael Noll | +| [HDFS-2041](https://issues.apache.org/jira/browse/HDFS-2041) | Some mtimes and atimes are lost when edit logs are replayed | Major | . | Todd Lipcon | Todd Lipcon | +| [HDFS-2034](https://issues.apache.org/jira/browse/HDFS-2034) | length in getBlockRange becomes -ve when reading only from currently being written blk | Minor | hdfs-client | John George | John George | +| [HDFS-2030](https://issues.apache.org/jira/browse/HDFS-2030) | Fix the usability of namenode upgrade command | Minor | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-2022](https://issues.apache.org/jira/browse/HDFS-2022) | ant binary should build libhdfs | Major | build | Eli Collins | Eric Yang | +| [HDFS-2021](https://issues.apache.org/jira/browse/HDFS-2021) | TestWriteRead failed with inconsistent visible length of a file | Major | datanode | CW Chung | John George | +| [HDFS-2020](https://issues.apache.org/jira/browse/HDFS-2020) | TestDFSUpgradeFromImage fails | Major | datanode, test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2019](https://issues.apache.org/jira/browse/HDFS-2019) | Fix all the places where Java method File.list is used with FileUtil.list API | Minor | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-2014](https://issues.apache.org/jira/browse/HDFS-2014) | bin/hdfs no longer works from a source checkout | Critical | scripts | Todd Lipcon | Eric Yang | +| [HDFS-2011](https://issues.apache.org/jira/browse/HDFS-2011) | Removal and restoration of storage directories on checkpointing failure doesn't work properly | Major | namenode | Ravi Prakash | Ravi Prakash | +| [HDFS-2002](https://issues.apache.org/jira/browse/HDFS-2002) | Incorrect computation of needed blocks in getTurnOffTip() | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-1999](https://issues.apache.org/jira/browse/HDFS-1999) | Tests use deprecated configs | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1998](https://issues.apache.org/jira/browse/HDFS-1998) | make refresh-namodenodes.sh refreshing all namenodes | Minor | scripts | Tanping Wang | Tanping Wang | +| [HDFS-1990](https://issues.apache.org/jira/browse/HDFS-1990) | Resource leaks in HDFS | Minor | datanode | ramkrishna.s.vasudevan | Uma Maheswara Rao G | +| [HDFS-1986](https://issues.apache.org/jira/browse/HDFS-1986) | Add an option for user to return http or https ports regardless of security is on/off in DFSUtil.getInfoServer() | Minor | tools | Tanping Wang | Tanping Wang | +| [HDFS-1964](https://issues.apache.org/jira/browse/HDFS-1964) | Incorrect HTML unescaping in DatanodeJspHelper.java | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1955](https://issues.apache.org/jira/browse/HDFS-1955) | FSImage.doUpgrade() was made too fault-tolerant by HDFS-1826 | Major | namenode | Matt Foley | Matt Foley | +| [HDFS-1953](https://issues.apache.org/jira/browse/HDFS-1953) | Change name node mxbean name in cluster web console | Minor | . | Tanping Wang | Tanping Wang | +| [HDFS-1952](https://issues.apache.org/jira/browse/HDFS-1952) | FSEditLog.open() appears to succeed even if all EDITS directories fail | Major | . | Matt Foley | Andrew | +| [HDFS-1943](https://issues.apache.org/jira/browse/HDFS-1943) | fail to start datanode while start-dfs.sh is executed by root user | Blocker | scripts | Wei Yongjun | Matt Foley | +| [HDFS-1938](https://issues.apache.org/jira/browse/HDFS-1938) | Reference ivy-hdfs.classpath not found. | Minor | build | Tsz Wo Nicholas Sze | Eric Yang | +| [HDFS-1936](https://issues.apache.org/jira/browse/HDFS-1936) | Updating the layout version from HDFS-1822 causes upgrade problems. | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1934](https://issues.apache.org/jira/browse/HDFS-1934) | Fix NullPointerException when File.listFiles() API returns null | Major | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1927](https://issues.apache.org/jira/browse/HDFS-1927) | audit logs could ignore certain xsactions and also could contain "ip=null" | Major | namenode | John George | John George | +| [HDFS-1921](https://issues.apache.org/jira/browse/HDFS-1921) | Save namespace can cause NN to be unable to come up on restart | Blocker | . | Aaron T. Myers | Matt Foley | +| [HDFS-1920](https://issues.apache.org/jira/browse/HDFS-1920) | libhdfs does not build for ARM processors | Major | libhdfs | Trevor Robinson | Trevor Robinson | +| [HDFS-1917](https://issues.apache.org/jira/browse/HDFS-1917) | Clean up duplication of dependent jar files | Major | build | Eric Yang | Eric Yang | +| [HDFS-1914](https://issues.apache.org/jira/browse/HDFS-1914) | Federation: namenode storage directory must be configurable specific to a namenode | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1908](https://issues.apache.org/jira/browse/HDFS-1908) | DataTransferTestUtil$CountdownDoosAction.run(..) throws NullPointerException | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1907](https://issues.apache.org/jira/browse/HDFS-1907) | BlockMissingException upon concurrent read and write: reader was doing file position read while writer is doing write without hflush | Major | hdfs-client | CW Chung | John George | +| [HDFS-1905](https://issues.apache.org/jira/browse/HDFS-1905) | Improve the usability of namenode -format | Minor | namenode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1898](https://issues.apache.org/jira/browse/HDFS-1898) | Tests failing on trunk due to use of NameNode.format | Critical | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1889](https://issues.apache.org/jira/browse/HDFS-1889) | incorrect path in start/stop dfs script | Major | . | John George | John George | +| [HDFS-1888](https://issues.apache.org/jira/browse/HDFS-1888) | MiniDFSCluster#corruptBlockOnDatanodes() access must be public for MapReduce contrib raid | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1881](https://issues.apache.org/jira/browse/HDFS-1881) | Federation: after taking snapshot the current directory of datanode is empty | Major | datanode | Tanping Wang | Tanping Wang | +| [HDFS-1876](https://issues.apache.org/jira/browse/HDFS-1876) | One MiniDFSCluster ignores numDataNodes parameter | Blocker | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1875](https://issues.apache.org/jira/browse/HDFS-1875) | MiniDFSCluster hard-codes dfs.datanode.address to localhost | Major | test | Eric Payne | Eric Payne | +| [HDFS-1871](https://issues.apache.org/jira/browse/HDFS-1871) | Tests using MiniDFSCluster fail to compile due to HDFS-1052 changes | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1869](https://issues.apache.org/jira/browse/HDFS-1869) | mkdirs should use the supplied permission for all of the created directories | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-1845](https://issues.apache.org/jira/browse/HDFS-1845) | symlink comes up as directory after namenode restart | Major | . | John George | John George | +| [HDFS-1835](https://issues.apache.org/jira/browse/HDFS-1835) | DataNode.setNewStorageID pulls entropy from /dev/random | Major | datanode | John Carrino | John Carrino | +| [HDFS-1829](https://issues.apache.org/jira/browse/HDFS-1829) | TestNodeCount waits forever, errs without giving information | Major | namenode | Matt Foley | Matt Foley | +| [HDFS-1827](https://issues.apache.org/jira/browse/HDFS-1827) | TestBlockReplacement waits forever, errs without giving information | Major | namenode | Matt Foley | Matt Foley | +| [HDFS-1824](https://issues.apache.org/jira/browse/HDFS-1824) | delay instantiation of file system object until it is needed (linked to HADOOP-7207) | Major | . | Boris Shkolnik | Boris Shkolnik | +| [HDFS-1823](https://issues.apache.org/jira/browse/HDFS-1823) | start-dfs.sh script fails if HADOOP\_HOME is not set | Blocker | scripts | Tom White | Tom White | +| [HDFS-1822](https://issues.apache.org/jira/browse/HDFS-1822) | Editlog opcodes overlap between 20 security and later releases | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1821](https://issues.apache.org/jira/browse/HDFS-1821) | FileContext.createSymlink with kerberos enabled sets wrong owner | Major | . | John George | John George | +| [HDFS-1818](https://issues.apache.org/jira/browse/HDFS-1818) | TestHDFSCLI is failing on trunk | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1812](https://issues.apache.org/jira/browse/HDFS-1812) | Address the cleanup issues in TestHDFSCLI.java | Minor | test | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-1808](https://issues.apache.org/jira/browse/HDFS-1808) | TestBalancer waits forever, errs without giving information | Major | datanode, namenode | Matt Foley | Matt Foley | +| [HDFS-1806](https://issues.apache.org/jira/browse/HDFS-1806) | TestBlockReport.blockReport\_08() and \_09() are timing-dependent and likely to fail on fast servers | Major | datanode, namenode | Matt Foley | Matt Foley | +| [HDFS-1797](https://issues.apache.org/jira/browse/HDFS-1797) | New findbugs warning introduced by HDFS-1120 | Major | . | Todd Lipcon | Todd Lipcon | +| [HDFS-1786](https://issues.apache.org/jira/browse/HDFS-1786) | Some cli test cases expect a "null" message | Minor | test | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-1782](https://issues.apache.org/jira/browse/HDFS-1782) | FSNamesystem.startFileInternal(..) throws NullPointerException | Major | namenode | John George | John George | +| [HDFS-1781](https://issues.apache.org/jira/browse/HDFS-1781) | jsvc executable delivered into wrong package... | Major | scripts | John George | John George | +| [HDFS-1776](https://issues.apache.org/jira/browse/HDFS-1776) | Bug in Concat code | Major | . | Dmytro Molkov | Bharath Mundlapudi | +| [HDFS-1760](https://issues.apache.org/jira/browse/HDFS-1760) | problems with getFullPathName | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-1750](https://issues.apache.org/jira/browse/HDFS-1750) | fs -ls hftp://file not working | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1748](https://issues.apache.org/jira/browse/HDFS-1748) | Balancer utilization classification is incomplete | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1734](https://issues.apache.org/jira/browse/HDFS-1734) | 'Chunk size to view' option is not working in Name Node UI. | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-1728](https://issues.apache.org/jira/browse/HDFS-1728) | SecondaryNameNode.checkpointSize is in byte but not MB. | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1727](https://issues.apache.org/jira/browse/HDFS-1727) | fsck command can display command usage if user passes any illegal argument | Minor | . | Uma Maheswara Rao G | sravankorumilli | +| [HDFS-1692](https://issues.apache.org/jira/browse/HDFS-1692) | In secure mode, Datanode process doesn't exit when disks fail. | Major | datanode | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1691](https://issues.apache.org/jira/browse/HDFS-1691) | double static declaration in Configuration.addDefaultResource("hdfs-default.xml"); | Minor | tools | Alexey Diomin | Alexey Diomin | +| [HDFS-1665](https://issues.apache.org/jira/browse/HDFS-1665) | Balancer sleeps inadequately | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1656](https://issues.apache.org/jira/browse/HDFS-1656) | getDelegationToken in HftpFileSystem should renew TGT if needed. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-1627](https://issues.apache.org/jira/browse/HDFS-1627) | Fix NullPointerException in Secondary NameNode | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1625](https://issues.apache.org/jira/browse/HDFS-1625) | TestDataNodeMXBean fails if disk space usage changes during test run | Minor | test | Todd Lipcon | Tsz Wo Nicholas Sze | +| [HDFS-1612](https://issues.apache.org/jira/browse/HDFS-1612) | HDFS Design Documentation is outdated | Minor | documentation | Joe Crobak | Joe Crobak | +| [HDFS-1611](https://issues.apache.org/jira/browse/HDFS-1611) | Some logical issues need to address. | Minor | hdfs-client, namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-1602](https://issues.apache.org/jira/browse/HDFS-1602) | NameNode storage failed replica restoration is broken | Major | namenode | Konstantin Boudnik | Boris Shkolnik | +| [HDFS-1600](https://issues.apache.org/jira/browse/HDFS-1600) | editsStored.xml cause release audit warning | Major | build, test | Tsz Wo Nicholas Sze | Todd Lipcon | +| [HDFS-1598](https://issues.apache.org/jira/browse/HDFS-1598) | ListPathsServlet excludes .\*.crc files | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1594](https://issues.apache.org/jira/browse/HDFS-1594) | When the disk becomes full Namenode is getting shutdown and not able to recover | Major | namenode | Devaraj K | Aaron T. Myers | +| [HDFS-1592](https://issues.apache.org/jira/browse/HDFS-1592) | Datanode startup doesn't honor volumes.tolerated | Major | . | Bharath Mundlapudi | Bharath Mundlapudi | +| [HDFS-1585](https://issues.apache.org/jira/browse/HDFS-1585) | HDFS-1547 broke MR build | Blocker | test | Todd Lipcon | Todd Lipcon | +| [HDFS-1551](https://issues.apache.org/jira/browse/HDFS-1551) | fix the pom template's version | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HDFS-1540](https://issues.apache.org/jira/browse/HDFS-1540) | Make Datanode handle errors to namenode.register call more elegantly | Major | datanode | dhruba borthakur | dhruba borthakur | +| [HDFS-1533](https://issues.apache.org/jira/browse/HDFS-1533) | A more elegant FileSystem#listCorruptFileBlocks API (HDFS portion) | Major | hdfs-client | Patrick Kling | Patrick Kling | +| [HDFS-1524](https://issues.apache.org/jira/browse/HDFS-1524) | Image loader should make sure to read every byte in image file | Blocker | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1523](https://issues.apache.org/jira/browse/HDFS-1523) | TestLargeBlock is failing on trunk | Major | test | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1516](https://issues.apache.org/jira/browse/HDFS-1516) | mvn-install is broken after 0.22 branch creation | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HDFS-1511](https://issues.apache.org/jira/browse/HDFS-1511) | 98 Release Audit warnings on trunk and branch-0.22 | Blocker | . | Nigel Daley | Jakob Homan | +| [HDFS-1505](https://issues.apache.org/jira/browse/HDFS-1505) | saveNamespace appears to succeed even if all directories fail to save | Blocker | . | Todd Lipcon | Aaron T. Myers | +| [HDFS-1503](https://issues.apache.org/jira/browse/HDFS-1503) | TestSaveNamespace fails | Minor | test | Eli Collins | Todd Lipcon | +| [HDFS-1502](https://issues.apache.org/jira/browse/HDFS-1502) | TestBlockRecovery triggers NPE in assert | Minor | . | Eli Collins | Hairong Kuang | +| [HDFS-1480](https://issues.apache.org/jira/browse/HDFS-1480) | All replicas of a block can end up on the same rack when some datanodes are decommissioning. | Major | namenode | T Meyarivan | Todd Lipcon | +| [HDFS-1467](https://issues.apache.org/jira/browse/HDFS-1467) | Append pipeline never succeeds with more than one replica | Blocker | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-1463](https://issues.apache.org/jira/browse/HDFS-1463) | accessTime updates should not occur in safeMode | Major | namenode | dhruba borthakur | dhruba borthakur | +| [HDFS-1381](https://issues.apache.org/jira/browse/HDFS-1381) | HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters | Major | test | Jakob Homan | Jim Plush | +| [HDFS-1377](https://issues.apache.org/jira/browse/HDFS-1377) | Quota bug for partial blocks allows quotas to be violated | Blocker | namenode | Eli Collins | Eli Collins | +| [HDFS-1371](https://issues.apache.org/jira/browse/HDFS-1371) | One bad node can incorrectly flag many files as corrupt | Major | hdfs-client, namenode | Koji Noguchi | Tanping Wang | +| [HDFS-1360](https://issues.apache.org/jira/browse/HDFS-1360) | TestBlockRecovery should bind ephemeral ports | Minor | test | Todd Lipcon | Todd Lipcon | +| [HDFS-1321](https://issues.apache.org/jira/browse/HDFS-1321) | If service port and main port are the same, there is no clear log message explaining the issue. | Minor | namenode | gary murry | Jim Plush | +| [HDFS-1257](https://issues.apache.org/jira/browse/HDFS-1257) | Race condition on FSNamesystem#recentInvalidateSets introduced by HADOOP-5124 | Major | namenode | Ramkumar Vadali | Eric Payne | +| [HDFS-1206](https://issues.apache.org/jira/browse/HDFS-1206) | TestFiHFlush fails intermittently | Major | test | Tsz Wo Nicholas Sze | Konstantin Boudnik | +| [HDFS-1189](https://issues.apache.org/jira/browse/HDFS-1189) | Quota counts missed between clear quota and set quota | Major | namenode | Kang Xiao | John George | +| [HDFS-1149](https://issues.apache.org/jira/browse/HDFS-1149) | Lease reassignment is not persisted to edit log | Major | namenode | Todd Lipcon | Aaron T. Myers | +| [HDFS-1001](https://issues.apache.org/jira/browse/HDFS-1001) | DataXceiver and BlockReader disagree on when to send/recv CHECKSUM\_OK | Minor | datanode | bc Wong | bc Wong | +| [HDFS-863](https://issues.apache.org/jira/browse/HDFS-863) | Potential deadlock in TestOverReplicatedBlocks | Major | test | Todd Lipcon | Ken Goodhope | +| [HDFS-73](https://issues.apache.org/jira/browse/HDFS-73) | DFSOutputStream does not close all the sockets | Blocker | hdfs-client | Raghu Angadi | Uma Maheswara Rao G | +| [MAPREDUCE-3337](https://issues.apache.org/jira/browse/MAPREDUCE-3337) | Missing license headers for some files | Blocker | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3332](https://issues.apache.org/jira/browse/MAPREDUCE-3332) | contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling | Trivial | contrib/raid | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3321](https://issues.apache.org/jira/browse/MAPREDUCE-3321) | Disable some failing legacy tests for MRv2 builds to go through | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3317](https://issues.apache.org/jira/browse/MAPREDUCE-3317) | Rumen TraceBuilder is emiting null as hostname | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3316](https://issues.apache.org/jira/browse/MAPREDUCE-3316) | Rebooted link is not working properly | Major | resourcemanager | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3313](https://issues.apache.org/jira/browse/MAPREDUCE-3313) | TestResourceTrackerService failing in trunk some times | Blocker | mrv2, test | Ravi Gummadi | Hitesh Shah | +| [MAPREDUCE-3306](https://issues.apache.org/jira/browse/MAPREDUCE-3306) | Cannot run apps after MAPREDUCE-2989 | Blocker | mrv2, nodemanager | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3304](https://issues.apache.org/jira/browse/MAPREDUCE-3304) | TestRMContainerAllocator#testBlackListedNodes fails intermittently | Major | mrv2, test | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3296](https://issues.apache.org/jira/browse/MAPREDUCE-3296) | Pending(9) findBugs warnings | Major | build | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3295](https://issues.apache.org/jira/browse/MAPREDUCE-3295) | TestAMAuthorization failing on branch 0.23. | Critical | . | Mahadev konar | | +| [MAPREDUCE-3292](https://issues.apache.org/jira/browse/MAPREDUCE-3292) | In secure mode job submission fails with Provider org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer not found. | Critical | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3290](https://issues.apache.org/jira/browse/MAPREDUCE-3290) | list-active-trackers throws NPE | Major | mrv2 | Ramya Sunil | Arun C Murthy | +| [MAPREDUCE-3288](https://issues.apache.org/jira/browse/MAPREDUCE-3288) | Mapreduce 23 builds failing | Blocker | mrv2 | Ramya Sunil | Mahadev konar | +| [MAPREDUCE-3285](https://issues.apache.org/jira/browse/MAPREDUCE-3285) | Tests on branch-0.23 failing | Blocker | mrv2 | Arun C Murthy | Siddharth Seth | +| [MAPREDUCE-3284](https://issues.apache.org/jira/browse/MAPREDUCE-3284) | bin/mapred queue fails with JobQueueClient ClassNotFoundException | Major | mrv2 | Ramya Sunil | Arun C Murthy | +| [MAPREDUCE-3282](https://issues.apache.org/jira/browse/MAPREDUCE-3282) | bin/mapred job -list throws exception | Critical | mrv2 | Ramya Sunil | Arun C Murthy | +| [MAPREDUCE-3281](https://issues.apache.org/jira/browse/MAPREDUCE-3281) | TestLinuxContainerExecutorWithMocks failing on trunk. | Blocker | test | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3279](https://issues.apache.org/jira/browse/MAPREDUCE-3279) | TestJobHistoryParsing broken | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3274](https://issues.apache.org/jira/browse/MAPREDUCE-3274) | Race condition in MR App Master Preemtion can cause a dead lock | Blocker | applicationmaster, mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3269](https://issues.apache.org/jira/browse/MAPREDUCE-3269) | Jobsummary logs not being moved to a separate file | Blocker | mrv2 | Ramya Sunil | Mahadev konar | +| [MAPREDUCE-3264](https://issues.apache.org/jira/browse/MAPREDUCE-3264) | mapreduce.job.user.name needs to be set automatically | Blocker | mrv2 | Todd Lipcon | Arun C Murthy | +| [MAPREDUCE-3263](https://issues.apache.org/jira/browse/MAPREDUCE-3263) | compile-mapred-test target fails | Blocker | build, mrv2 | Ramya Sunil | Hitesh Shah | +| [MAPREDUCE-3262](https://issues.apache.org/jira/browse/MAPREDUCE-3262) | A few events are not handled by the NodeManager in failure scenarios | Critical | mrv2, nodemanager | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3261](https://issues.apache.org/jira/browse/MAPREDUCE-3261) | AM unable to release containers | Major | applicationmaster | Chris Riccomini | | +| [MAPREDUCE-3259](https://issues.apache.org/jira/browse/MAPREDUCE-3259) | ContainerLocalizer should get the proper java.library.path from LinuxContainerExecutor | Blocker | mrv2, nodemanager | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-3258](https://issues.apache.org/jira/browse/MAPREDUCE-3258) | Job counters missing from AM and history UI | Blocker | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3254](https://issues.apache.org/jira/browse/MAPREDUCE-3254) | Streaming jobs failing with PipeMapRunner ClassNotFoundException | Blocker | contrib/streaming, mrv2 | Ramya Sunil | Arun C Murthy | +| [MAPREDUCE-3253](https://issues.apache.org/jira/browse/MAPREDUCE-3253) | ContextFactory throw NoSuchFieldException | Blocker | mrv2 | Daniel Dai | Arun C Murthy | +| [MAPREDUCE-3252](https://issues.apache.org/jira/browse/MAPREDUCE-3252) | MR2: Map tasks rewrite data once even if output fits in sort buffer | Critical | mrv2, task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3248](https://issues.apache.org/jira/browse/MAPREDUCE-3248) | Log4j logs from unit tests are lost | Blocker | test | Arun C Murthy | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3242](https://issues.apache.org/jira/browse/MAPREDUCE-3242) | Trunk compilation broken with bad interaction from MAPREDUCE-3070 and MAPREDUCE-3239. | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3241](https://issues.apache.org/jira/browse/MAPREDUCE-3241) | (Rumen)TraceBuilder throws IllegalArgumentException | Major | . | Devaraj K | Amar Kamat | +| [MAPREDUCE-3240](https://issues.apache.org/jira/browse/MAPREDUCE-3240) | NM should send a SIGKILL for completed containers also | Blocker | mrv2, nodemanager | Vinod Kumar Vavilapalli | Hitesh Shah | +| [MAPREDUCE-3228](https://issues.apache.org/jira/browse/MAPREDUCE-3228) | MR AM hangs when one node goes bad | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3226](https://issues.apache.org/jira/browse/MAPREDUCE-3226) | Few reduce tasks hanging in a gridmix-run | Blocker | mrv2, task | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3212](https://issues.apache.org/jira/browse/MAPREDUCE-3212) | Message displays while executing yarn command should be proper | Minor | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3209](https://issues.apache.org/jira/browse/MAPREDUCE-3209) | Jenkins reports 160 FindBugs warnings | Major | build, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3208](https://issues.apache.org/jira/browse/MAPREDUCE-3208) | NPE while flushing TaskLogAppender | Minor | mrv2 | liangzhaowang | liangzhaowang | +| [MAPREDUCE-3204](https://issues.apache.org/jira/browse/MAPREDUCE-3204) | mvn site:site fails on MapReduce | Major | build | Suresh Srinivas | Alejandro Abdelnur | +| [MAPREDUCE-3203](https://issues.apache.org/jira/browse/MAPREDUCE-3203) | Fix some javac warnings in MRAppMaster. | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3199](https://issues.apache.org/jira/browse/MAPREDUCE-3199) | TestJobMonitorAndPrint is broken on trunk | Major | mrv2, test | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3198](https://issues.apache.org/jira/browse/MAPREDUCE-3198) | Change mode for hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/mock-container-executor to 755 | Trivial | mrv2 | Hitesh Shah | Arun C Murthy | +| [MAPREDUCE-3197](https://issues.apache.org/jira/browse/MAPREDUCE-3197) | TestMRClientService failing on building clean checkout of branch 0.23 | Major | mrv2 | Anupam Seth | Mahadev konar | +| [MAPREDUCE-3196](https://issues.apache.org/jira/browse/MAPREDUCE-3196) | TestLinuxContainerExecutorWithMocks fails on Mac OSX | Major | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3192](https://issues.apache.org/jira/browse/MAPREDUCE-3192) | Fix Javadoc warning in JobClient.java and Cluster.java | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-3188](https://issues.apache.org/jira/browse/MAPREDUCE-3188) | Lots of errors in logs when daemon startup fails | Major | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3186](https://issues.apache.org/jira/browse/MAPREDUCE-3186) | User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed. | Blocker | mrv2 | Ramgopal N | Eric Payne | +| [MAPREDUCE-3185](https://issues.apache.org/jira/browse/MAPREDUCE-3185) | RM Web UI does not sort the columns in some cases. | Critical | mrv2 | Mahadev konar | Jonathan Eagles | +| [MAPREDUCE-3183](https://issues.apache.org/jira/browse/MAPREDUCE-3183) | hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header | Trivial | build | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3181](https://issues.apache.org/jira/browse/MAPREDUCE-3181) | Terasort fails with Kerberos exception on secure cluster | Blocker | mrv2 | Anupam Seth | Arun C Murthy | +| [MAPREDUCE-3179](https://issues.apache.org/jira/browse/MAPREDUCE-3179) | Incorrect exit code for hadoop-mapreduce-test tests when exception thrown | Major | mrv2, test | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3176](https://issues.apache.org/jira/browse/MAPREDUCE-3176) | ant mapreduce tests are timing out | Blocker | mrv2, test | Ravi Prakash | Hitesh Shah | +| [MAPREDUCE-3170](https://issues.apache.org/jira/browse/MAPREDUCE-3170) | Trunk nightly commit builds are failing. | Critical | build, mrv1, mrv2 | Mahadev konar | Hitesh Shah | +| [MAPREDUCE-3167](https://issues.apache.org/jira/browse/MAPREDUCE-3167) | container-executor is not being packaged with the assembly target. | Minor | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3166](https://issues.apache.org/jira/browse/MAPREDUCE-3166) | Make Rumen use job history api instead of relying on current history file name format | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3165](https://issues.apache.org/jira/browse/MAPREDUCE-3165) | Ensure logging option is set on child command line | Blocker | applicationmaster, mrv2 | Arun C Murthy | Todd Lipcon | +| [MAPREDUCE-3163](https://issues.apache.org/jira/browse/MAPREDUCE-3163) | JobClient spews errors when killing MR2 job | Blocker | job submission, mrv2 | Todd Lipcon | Mahadev konar | +| [MAPREDUCE-3159](https://issues.apache.org/jira/browse/MAPREDUCE-3159) | DefaultContainerExecutor removes appcache dir on every localization | Blocker | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3158](https://issues.apache.org/jira/browse/MAPREDUCE-3158) | Fix trunk build failures | Major | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3157](https://issues.apache.org/jira/browse/MAPREDUCE-3157) | Rumen TraceBuilder is skipping analyzing 0.20 history files | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3153](https://issues.apache.org/jira/browse/MAPREDUCE-3153) | TestFileOutputCommitter.testFailAbort() is failing on trunk on Jenkins | Major | mrv2, test | Vinod Kumar Vavilapalli | Mahadev konar | +| [MAPREDUCE-3143](https://issues.apache.org/jira/browse/MAPREDUCE-3143) | Complete aggregation of user-logs spit out by containers onto DFS | Major | mrv2, nodemanager | Vinod Kumar Vavilapalli | | +| [MAPREDUCE-3140](https://issues.apache.org/jira/browse/MAPREDUCE-3140) | Invalid JobHistory URL for failed applications | Major | mrv2 | Bhallamudi Venkata Siva Kamesh | Subroto Sanyal | +| [MAPREDUCE-3138](https://issues.apache.org/jira/browse/MAPREDUCE-3138) | Allow for applications to deal with MAPREDUCE-954 | Blocker | client, mrv2 | Arun C Murthy | Owen O'Malley | +| [MAPREDUCE-3126](https://issues.apache.org/jira/browse/MAPREDUCE-3126) | mr job stuck because reducers using all slots and mapper isn't scheduled | Blocker | mrv2 | Thomas Graves | Arun C Murthy | +| [MAPREDUCE-3125](https://issues.apache.org/jira/browse/MAPREDUCE-3125) | app master web UI shows reduce task progress 100% even though reducers not complete and state running/scheduled | Critical | mrv2 | Thomas Graves | Hitesh Shah | +| [MAPREDUCE-3124](https://issues.apache.org/jira/browse/MAPREDUCE-3124) | mapper failed with failed to load native libs | Blocker | mrv2 | Thomas Graves | John George | +| [MAPREDUCE-3123](https://issues.apache.org/jira/browse/MAPREDUCE-3123) | Symbolic links with special chars causing container/task.sh to fail | Blocker | mrv2 | Thomas Graves | Hitesh Shah | +| [MAPREDUCE-3114](https://issues.apache.org/jira/browse/MAPREDUCE-3114) | Invalid ApplicationMaster URL in Applications Page | Major | mrv2 | Subroto Sanyal | Subroto Sanyal | +| [MAPREDUCE-3112](https://issues.apache.org/jira/browse/MAPREDUCE-3112) | Calling hadoop cli inside mapreduce job leads to errors | Major | contrib/streaming | Eric Yang | Eric Yang | +| [MAPREDUCE-3110](https://issues.apache.org/jira/browse/MAPREDUCE-3110) | TestRPC.testUnknownCall() is failing | Major | mrv2, test | Devaraj K | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3095](https://issues.apache.org/jira/browse/MAPREDUCE-3095) | fairscheduler ivy including wrong version for hdfs | Major | mrv2 | John George | John George | +| [MAPREDUCE-3092](https://issues.apache.org/jira/browse/MAPREDUCE-3092) | Remove JOB\_ID\_COMPARATOR usage in JobHistory.java | Minor | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3087](https://issues.apache.org/jira/browse/MAPREDUCE-3087) | CLASSPATH not the same after MAPREDUCE-2880 | Critical | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3081](https://issues.apache.org/jira/browse/MAPREDUCE-3081) | Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh | Major | contrib/vaidya | vitthal (Suhas) Gogate | | +| [MAPREDUCE-3078](https://issues.apache.org/jira/browse/MAPREDUCE-3078) | Application's progress isn't updated from AM to RM. | Blocker | applicationmaster, mrv2, resourcemanager | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3073](https://issues.apache.org/jira/browse/MAPREDUCE-3073) | Build failure for MRv1 caused due to changes to MRConstants. | Blocker | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3071](https://issues.apache.org/jira/browse/MAPREDUCE-3071) | app master configuration web UI link under the Job menu opens up application menu | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3070](https://issues.apache.org/jira/browse/MAPREDUCE-3070) | NM not able to register with RM after NM restart | Blocker | mrv2, nodemanager | Ravi Teja Ch N V | Devaraj K | +| [MAPREDUCE-3068](https://issues.apache.org/jira/browse/MAPREDUCE-3068) | Should set MALLOC\_ARENA\_MAX for all YARN daemons and AMs/Containers | Blocker | mrv2 | Vinod Kumar Vavilapalli | Chris Riccomini | +| [MAPREDUCE-3067](https://issues.apache.org/jira/browse/MAPREDUCE-3067) | Container exit status not set properly to launched process's exit code on successful completion of process | Blocker | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3066](https://issues.apache.org/jira/browse/MAPREDUCE-3066) | YARN NM fails to start | Major | mrv2, nodemanager | Chris Riccomini | Chris Riccomini | +| [MAPREDUCE-3064](https://issues.apache.org/jira/browse/MAPREDUCE-3064) | 27 unit test failures with Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local" | Blocker | . | Thomas Graves | Venu Gopala Rao | +| [MAPREDUCE-3062](https://issues.apache.org/jira/browse/MAPREDUCE-3062) | YARN NM/RM fail to start | Major | mrv2, nodemanager, resourcemanager | Chris Riccomini | Chris Riccomini | +| [MAPREDUCE-3059](https://issues.apache.org/jira/browse/MAPREDUCE-3059) | QueueMetrics do not have metrics for aggregate containers-allocated and aggregate containers-released | Blocker | mrv2 | Karam Singh | Devaraj K | +| [MAPREDUCE-3058](https://issues.apache.org/jira/browse/MAPREDUCE-3058) | Sometimes task keeps on running while its Syslog says that it is shutdown | Critical | contrib/gridmix, mrv2 | Karam Singh | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3057](https://issues.apache.org/jira/browse/MAPREDUCE-3057) | Job History Server goes of OutOfMemory with 1200 Jobs and Heap Size set to 10 GB | Blocker | jobhistoryserver, mrv2 | Karam Singh | Eric Payne | +| [MAPREDUCE-3056](https://issues.apache.org/jira/browse/MAPREDUCE-3056) | Jobs are failing when those are submitted by other users | Blocker | applicationmaster, mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3055](https://issues.apache.org/jira/browse/MAPREDUCE-3055) | Simplify parameter passing to Application Master from Client. SImplify approach to pass info such appId, ClusterTimestamp and failcount required by App Master. | Minor | mrv2 | Hitesh Shah | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3054](https://issues.apache.org/jira/browse/MAPREDUCE-3054) | Unable to kill submitted jobs | Blocker | mrv2 | Siddharth Seth | Mahadev konar | +| [MAPREDUCE-3053](https://issues.apache.org/jira/browse/MAPREDUCE-3053) | YARN Protobuf RPC Failures in RM | Major | mrv2, resourcemanager | Chris Riccomini | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3050](https://issues.apache.org/jira/browse/MAPREDUCE-3050) | YarnScheduler needs to expose Resource Usage Information | Blocker | mrv2, resourcemanager | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3048](https://issues.apache.org/jira/browse/MAPREDUCE-3048) | Fix test-patch to run tests via "mvn clean install test" | Major | build | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3044](https://issues.apache.org/jira/browse/MAPREDUCE-3044) | Pipes jobs stuck without making progress | Blocker | mrv2 | Ramya Sunil | Mahadev konar | +| [MAPREDUCE-3042](https://issues.apache.org/jira/browse/MAPREDUCE-3042) | YARN RM fails to start | Major | mrv2, resourcemanager | Chris Riccomini | Chris Riccomini | +| [MAPREDUCE-3040](https://issues.apache.org/jira/browse/MAPREDUCE-3040) | TestMRJobs, TestMRJobsWithHistoryService, TestMROldApiJobs fail | Major | mrv2 | Thomas Graves | Arun C Murthy | +| [MAPREDUCE-3038](https://issues.apache.org/jira/browse/MAPREDUCE-3038) | job history server not starting because conf() missing HsController | Blocker | mrv2 | Thomas Graves | Jeffrey Naisbitt | +| [MAPREDUCE-3036](https://issues.apache.org/jira/browse/MAPREDUCE-3036) | Some of the Resource Manager memory metrics go negative. | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3035](https://issues.apache.org/jira/browse/MAPREDUCE-3035) | MR V2 jobhistory does not contain rack information | Critical | mrv2 | Karam Singh | chackaravarthy | +| [MAPREDUCE-3033](https://issues.apache.org/jira/browse/MAPREDUCE-3033) | JobClient requires mapreduce.jobtracker.address config even when mapreduce.framework.name is set to yarn | Blocker | job submission, mrv2 | Karam Singh | Hitesh Shah | +| [MAPREDUCE-3032](https://issues.apache.org/jira/browse/MAPREDUCE-3032) | JobHistory doesn't have error information from failed tasks | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Devaraj K | +| [MAPREDUCE-3031](https://issues.apache.org/jira/browse/MAPREDUCE-3031) | Job Client goes into infinite loop when we kill AM | Blocker | mrv2 | Karam Singh | Siddharth Seth | +| [MAPREDUCE-3030](https://issues.apache.org/jira/browse/MAPREDUCE-3030) | RM is not processing heartbeat and continuously giving the message 'Node not found rebooting' | Blocker | mrv2, resourcemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-3028](https://issues.apache.org/jira/browse/MAPREDUCE-3028) | Support job end notification in .next /0.23 | Blocker | mrv2 | Mohammad Kamrul Islam | Ravi Prakash | +| [MAPREDUCE-3023](https://issues.apache.org/jira/browse/MAPREDUCE-3023) | Queue state is not being translated properly (is always assumed to be running) | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3021](https://issues.apache.org/jira/browse/MAPREDUCE-3021) | all yarn webapps use same base name of "yarn/" | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3020](https://issues.apache.org/jira/browse/MAPREDUCE-3020) | Node link in reduce task attempt page is not working [Job History Page] | Major | jobhistoryserver | chackaravarthy | chackaravarthy | +| [MAPREDUCE-3018](https://issues.apache.org/jira/browse/MAPREDUCE-3018) | Streaming jobs with -file option fail to run. | Blocker | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3017](https://issues.apache.org/jira/browse/MAPREDUCE-3017) | The Web UI shows FINISHED for killed/successful/failed jobs. | Blocker | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3006](https://issues.apache.org/jira/browse/MAPREDUCE-3006) | MapReduce AM exits prematurely before completely writing and closing the JobHistory file | Major | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3005](https://issues.apache.org/jira/browse/MAPREDUCE-3005) | MR app hangs because of a NPE in ResourceManager | Major | mrv2 | Vinod Kumar Vavilapalli | Arun C Murthy | +| [MAPREDUCE-3004](https://issues.apache.org/jira/browse/MAPREDUCE-3004) | sort example fails in shuffle/reduce stage as it assumes a local job by default | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3003](https://issues.apache.org/jira/browse/MAPREDUCE-3003) | Publish MR JARs to Maven snapshot repository | Major | build | Tom White | Alejandro Abdelnur | +| [MAPREDUCE-2999](https://issues.apache.org/jira/browse/MAPREDUCE-2999) | hadoop.http.filter.initializers not working properly on yarn UI | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2998](https://issues.apache.org/jira/browse/MAPREDUCE-2998) | Failing to contact Am/History for jobs: java.io.EOFException in DataInputStream | Critical | mrv2 | Jeffrey Naisbitt | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2997](https://issues.apache.org/jira/browse/MAPREDUCE-2997) | MR task fails before launch itself with an NPE in ContainerLauncher | Major | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2996](https://issues.apache.org/jira/browse/MAPREDUCE-2996) | Log uberized information into JobHistory and use the same via CompletedJob | Blocker | jobhistoryserver, mrv2 | Vinod Kumar Vavilapalli | Jonathan Eagles | +| [MAPREDUCE-2995](https://issues.apache.org/jira/browse/MAPREDUCE-2995) | MR AM crashes when a container-launch hangs on a faulty NM | Major | mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2994](https://issues.apache.org/jira/browse/MAPREDUCE-2994) | Parse Error is coming for App ID when we click application link on the RM UI | Major | mrv2, resourcemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-2991](https://issues.apache.org/jira/browse/MAPREDUCE-2991) | queueinfo.jsp fails to show queue status if any Capacity scheduler queue name has dash/hiphen in it. | Major | scheduler | Priyo Mustafi | Priyo Mustafi | +| [MAPREDUCE-2987](https://issues.apache.org/jira/browse/MAPREDUCE-2987) | RM UI display logged in user as null | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2985](https://issues.apache.org/jira/browse/MAPREDUCE-2985) | findbugs error in ResourceLocalizationService.handle(LocalizationEvent) | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2984](https://issues.apache.org/jira/browse/MAPREDUCE-2984) | Throwing NullPointerException when we open the container page | Minor | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-2979](https://issues.apache.org/jira/browse/MAPREDUCE-2979) | Remove ClientProtocolProvider configuration under mapreduce-client-core | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2975](https://issues.apache.org/jira/browse/MAPREDUCE-2975) | ResourceManager Delegate is not getting initialized with yarn-site.xml as default configuration. | Blocker | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2971](https://issues.apache.org/jira/browse/MAPREDUCE-2971) | ant build mapreduce fails protected access jc.displayJobList(jobs); | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2970](https://issues.apache.org/jira/browse/MAPREDUCE-2970) | Null Pointer Exception while submitting a Job, If mapreduce.framework.name property is not set. | Major | job submission, mrv2 | Venu Gopala Rao | Venu Gopala Rao | +| [MAPREDUCE-2965](https://issues.apache.org/jira/browse/MAPREDUCE-2965) | Streamline hashCode(), equals(), compareTo() and toString() for all IDs | Blocker | mrv2 | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-2963](https://issues.apache.org/jira/browse/MAPREDUCE-2963) | TestMRJobs hangs waiting to connect to history server. | Critical | . | Mahadev konar | Siddharth Seth | +| [MAPREDUCE-2958](https://issues.apache.org/jira/browse/MAPREDUCE-2958) | mapred-default.xml not merged from mr279 | Critical | mrv2 | Thomas Graves | Arun C Murthy | +| [MAPREDUCE-2954](https://issues.apache.org/jira/browse/MAPREDUCE-2954) | Deadlock in NM with threads racing for ApplicationAttemptId | Critical | mrv2 | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-2953](https://issues.apache.org/jira/browse/MAPREDUCE-2953) | JobClient fails due to a race in RM, removes staged files and in turn crashes MR AM | Major | mrv2, resourcemanager | Vinod Kumar Vavilapalli | Thomas Graves | +| [MAPREDUCE-2952](https://issues.apache.org/jira/browse/MAPREDUCE-2952) | Application failure diagnostics are not consumed in a couple of cases | Blocker | mrv2, resourcemanager | Vinod Kumar Vavilapalli | Arun C Murthy | +| [MAPREDUCE-2949](https://issues.apache.org/jira/browse/MAPREDUCE-2949) | NodeManager in a inconsistent state if a service startup fails. | Major | mrv2, nodemanager | Ravi Teja Ch N V | Ravi Teja Ch N V | +| [MAPREDUCE-2948](https://issues.apache.org/jira/browse/MAPREDUCE-2948) | Hadoop streaming test failure, post MR-2767 | Major | contrib/streaming | Milind Bhandarkar | Mahadev konar | +| [MAPREDUCE-2947](https://issues.apache.org/jira/browse/MAPREDUCE-2947) | Sort fails on YARN+MR with lots of task failures | Major | mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2938](https://issues.apache.org/jira/browse/MAPREDUCE-2938) | Missing log stmt for app submission fail CS | Trivial | mrv2, scheduler | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2937](https://issues.apache.org/jira/browse/MAPREDUCE-2937) | Errors in Application failures are not shown in the client trace. | Critical | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2936](https://issues.apache.org/jira/browse/MAPREDUCE-2936) | Contrib Raid compilation broken after HDFS-1620 | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2925](https://issues.apache.org/jira/browse/MAPREDUCE-2925) | job -status \ is giving continuously info message for completed jobs on the console | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-2917](https://issues.apache.org/jira/browse/MAPREDUCE-2917) | Corner case in container reservations | Major | mrv2, resourcemanager | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2916](https://issues.apache.org/jira/browse/MAPREDUCE-2916) | Ivy build for MRv1 fails with bad organization for common daemon. | Major | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2913](https://issues.apache.org/jira/browse/MAPREDUCE-2913) | TestMRJobs.testFailingMapper does not assert the correct thing. | Critical | mrv2, test | Robert Joseph Evans | Jonathan Eagles | +| [MAPREDUCE-2908](https://issues.apache.org/jira/browse/MAPREDUCE-2908) | Fix findbugs warnings in Map Reduce. | Critical | mrv2 | Mahadev konar | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2907](https://issues.apache.org/jira/browse/MAPREDUCE-2907) | ResourceManager logs filled with [INFO] debug messages from org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue | Major | mrv2, resourcemanager | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-2904](https://issues.apache.org/jira/browse/MAPREDUCE-2904) | HDFS jars added incorrectly to yarn classpath | Major | . | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-2886](https://issues.apache.org/jira/browse/MAPREDUCE-2886) | Fix Javadoc warnings in MapReduce. | Critical | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2885](https://issues.apache.org/jira/browse/MAPREDUCE-2885) | mapred-config.sh doesn't look for $HADOOP\_COMMON\_HOME/libexec/hadoop-config.sh | Blocker | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2882](https://issues.apache.org/jira/browse/MAPREDUCE-2882) | TestLineRecordReader depends on ant jars | Minor | test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2881](https://issues.apache.org/jira/browse/MAPREDUCE-2881) | mapreduce ant compilation fails "java.lang.IllegalStateException: impossible to get artifacts" | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-2879](https://issues.apache.org/jira/browse/MAPREDUCE-2879) | Change mrv2 version to be 0.23.0-SNAPSHOT | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2877](https://issues.apache.org/jira/browse/MAPREDUCE-2877) | Add missing Apache license header in some files in MR and also add the rat plugin to the poms. | Major | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2876](https://issues.apache.org/jira/browse/MAPREDUCE-2876) | ContainerAllocationExpirer appears to use the incorrect configs | Critical | mrv2 | Robert Joseph Evans | Anupam Seth | +| [MAPREDUCE-2874](https://issues.apache.org/jira/browse/MAPREDUCE-2874) | ApplicationId printed in 2 different formats and has 2 different toString routines that are used | Major | mrv2 | Thomas Graves | Eric Payne | +| [MAPREDUCE-2868](https://issues.apache.org/jira/browse/MAPREDUCE-2868) | ant build broken in hadoop-mapreduce dir | Major | build | Thomas Graves | Mahadev konar | +| [MAPREDUCE-2867](https://issues.apache.org/jira/browse/MAPREDUCE-2867) | Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager. | Major | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2860](https://issues.apache.org/jira/browse/MAPREDUCE-2860) | Fix log4j logging in the maven test cases. | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-2859](https://issues.apache.org/jira/browse/MAPREDUCE-2859) | mapreduce trunk is broken with eclipse plugin contrib | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-2854](https://issues.apache.org/jira/browse/MAPREDUCE-2854) | update INSTALL with config necessary run mapred on yarn | Major | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | a small % of all tasks fail with DefaultTaskController | Blocker | task, task-controller, tasktracker | Allen Wittenauer | Owen O'Malley | +| [MAPREDUCE-2844](https://issues.apache.org/jira/browse/MAPREDUCE-2844) | [MR-279] Incorrect node ID info | Trivial | mrv2 | Ramya Sunil | Ravi Teja Ch N V | +| [MAPREDUCE-2843](https://issues.apache.org/jira/browse/MAPREDUCE-2843) | [MR-279] Node entries on the RM UI are not sortable | Major | mrv2 | Ramya Sunil | Abhijit Suresh Shingate | +| [MAPREDUCE-2840](https://issues.apache.org/jira/browse/MAPREDUCE-2840) | mr279 TestUberAM.testSleepJob test fails | Minor | mrv2 | Thomas Graves | Jonathan Eagles | +| [MAPREDUCE-2839](https://issues.apache.org/jira/browse/MAPREDUCE-2839) | MR Jobs fail on a secure cluster with viewfs | Major | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2821](https://issues.apache.org/jira/browse/MAPREDUCE-2821) | [MR-279] Missing fields in job summary logs | Blocker | mrv2 | Ramya Sunil | Mahadev konar | +| [MAPREDUCE-2808](https://issues.apache.org/jira/browse/MAPREDUCE-2808) | pull MAPREDUCE-2797 into mr279 branch | Minor | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2800](https://issues.apache.org/jira/browse/MAPREDUCE-2800) | clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files | Major | mrv2 | Ramya Sunil | Siddharth Seth | +| [MAPREDUCE-2797](https://issues.apache.org/jira/browse/MAPREDUCE-2797) | Some java files cannot be compiled | Major | contrib/raid, test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2796](https://issues.apache.org/jira/browse/MAPREDUCE-2796) | [MR-279] Start time for all the apps is set to 0 | Major | mrv2 | Ramya Sunil | Devaraj K | +| [MAPREDUCE-2794](https://issues.apache.org/jira/browse/MAPREDUCE-2794) | [MR-279] Incorrect metrics value for AvailableGB per queue per user | Blocker | mrv2 | Ramya Sunil | John George | +| [MAPREDUCE-2791](https://issues.apache.org/jira/browse/MAPREDUCE-2791) | [MR-279] Missing/incorrect info on job -status CLI | Blocker | mrv2 | Ramya Sunil | Devaraj K | +| [MAPREDUCE-2789](https://issues.apache.org/jira/browse/MAPREDUCE-2789) | [MR:279] Update the scheduling info on CLI | Major | mrv2 | Ramya Sunil | Eric Payne | +| [MAPREDUCE-2788](https://issues.apache.org/jira/browse/MAPREDUCE-2788) | Normalize requests in FifoScheduler.allocate to prevent NPEs later | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-2783](https://issues.apache.org/jira/browse/MAPREDUCE-2783) | mr279 job history handling after killing application | Critical | mrv2 | Thomas Graves | Eric Payne | +| [MAPREDUCE-2781](https://issues.apache.org/jira/browse/MAPREDUCE-2781) | mr279 RM application finishtime not set | Minor | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2779](https://issues.apache.org/jira/browse/MAPREDUCE-2779) | JobSplitWriter.java can't handle large job.split file | Major | job submission | Ming Ma | Ming Ma | +| [MAPREDUCE-2776](https://issues.apache.org/jira/browse/MAPREDUCE-2776) | MR 279: Fix some of the yarn findbug warnings | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2775](https://issues.apache.org/jira/browse/MAPREDUCE-2775) | [MR-279] Decommissioned node does not shutdown | Blocker | mrv2 | Ramya Sunil | Devaraj K | +| [MAPREDUCE-2774](https://issues.apache.org/jira/browse/MAPREDUCE-2774) | [MR-279] Add a startup msg while starting RM/NM | Minor | mrv2 | Ramya Sunil | Venu Gopala Rao | +| [MAPREDUCE-2773](https://issues.apache.org/jira/browse/MAPREDUCE-2773) | [MR-279] server.api.records.NodeHealthStatus renamed but not updated in client NodeHealthStatus.java | Minor | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2772](https://issues.apache.org/jira/browse/MAPREDUCE-2772) | MR-279: mrv2 no longer compiles against trunk after common mavenization. | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2767](https://issues.apache.org/jira/browse/MAPREDUCE-2767) | Remove Linux task-controller from 0.22 branch | Blocker | security | Milind Bhandarkar | Milind Bhandarkar | +| [MAPREDUCE-2764](https://issues.apache.org/jira/browse/MAPREDUCE-2764) | Fix renewal of dfs delegation tokens | Major | . | Daryn Sharp | Owen O'Malley | +| [MAPREDUCE-2763](https://issues.apache.org/jira/browse/MAPREDUCE-2763) | IllegalArgumentException while using the dist cache | Major | mrv2 | Ramya Sunil | | +| [MAPREDUCE-2762](https://issues.apache.org/jira/browse/MAPREDUCE-2762) | [MR-279] - Cleanup staging dir after job completion | Blocker | mrv2 | Ramya Sunil | Mahadev konar | +| [MAPREDUCE-2760](https://issues.apache.org/jira/browse/MAPREDUCE-2760) | mapreduce.jobtracker.split.metainfo.maxsize typoed in mapred-default.xml | Minor | documentation | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2756](https://issues.apache.org/jira/browse/MAPREDUCE-2756) | JobControl can drop jobs if an error occurs | Minor | client, mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2754](https://issues.apache.org/jira/browse/MAPREDUCE-2754) | MR-279: AM logs are incorrectly going to stderr and error messages going incorrectly to stdout | Blocker | mrv2 | Ramya Sunil | Ravi Teja Ch N V | +| [MAPREDUCE-2751](https://issues.apache.org/jira/browse/MAPREDUCE-2751) | [MR-279] Lot of local files left on NM after the app finish. | Blocker | mrv2 | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-2749](https://issues.apache.org/jira/browse/MAPREDUCE-2749) | [MR-279] NM registers with RM even before it starts various servers | Major | mrv2 | Vinod Kumar Vavilapalli | Thomas Graves | +| [MAPREDUCE-2740](https://issues.apache.org/jira/browse/MAPREDUCE-2740) | MultipleOutputs in new API creates needless TaskAttemptContexts | Major | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2738](https://issues.apache.org/jira/browse/MAPREDUCE-2738) | Missing cluster level stats on the RM UI | Blocker | mrv2 | Ramya Sunil | Robert Joseph Evans | +| [MAPREDUCE-2737](https://issues.apache.org/jira/browse/MAPREDUCE-2737) | Update the progress of jobs on client side | Major | mrv2 | Ramya Sunil | Siddharth Seth | +| [MAPREDUCE-2735](https://issues.apache.org/jira/browse/MAPREDUCE-2735) | MR279: finished applications should be added to an application summary log | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2732](https://issues.apache.org/jira/browse/MAPREDUCE-2732) | Some tests using FSNamesystem.LOG cannot be compiled | Major | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2727](https://issues.apache.org/jira/browse/MAPREDUCE-2727) | MR-279: SleepJob throws divide by zero exception when count = 0 | Major | mrv2 | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2716](https://issues.apache.org/jira/browse/MAPREDUCE-2716) | MR279: MRReliabilityTest job fails because of missing job-file. | Major | mrv2 | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2711](https://issues.apache.org/jira/browse/MAPREDUCE-2711) | TestBlockPlacementPolicyRaid cannot be compiled | Major | contrib/raid | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2710](https://issues.apache.org/jira/browse/MAPREDUCE-2710) | Update DFSClient.stringifyToken(..) in JobSubmitter.printTokens(..) for HDFS-2161 | Major | client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2706](https://issues.apache.org/jira/browse/MAPREDUCE-2706) | MR-279: Submit jobs beyond the max jobs per queue limit no longer gets logged | Major | mrv2 | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2705](https://issues.apache.org/jira/browse/MAPREDUCE-2705) | tasks localized and launched serially by TaskLauncher - causing other tasks to be delayed | Major | tasktracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2697](https://issues.apache.org/jira/browse/MAPREDUCE-2697) | Enhance CS to cap concurrently running jobs | Major | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2693](https://issues.apache.org/jira/browse/MAPREDUCE-2693) | NPE in AM causes it to lose containers which are never returned back to RM | Critical | mrv2 | Amol Kekre | Hitesh Shah | +| [MAPREDUCE-2690](https://issues.apache.org/jira/browse/MAPREDUCE-2690) | Construct the web page for default scheduler | Major | mrv2 | Ramya Sunil | Eric Payne | +| [MAPREDUCE-2689](https://issues.apache.org/jira/browse/MAPREDUCE-2689) | InvalidStateTransisiton when AM is not assigned to a job | Major | mrv2 | Ramya Sunil | | +| [MAPREDUCE-2687](https://issues.apache.org/jira/browse/MAPREDUCE-2687) | Non superusers unable to launch apps in both secure and non-secure cluster | Blocker | mrv2 | Ramya Sunil | Mahadev konar | +| [MAPREDUCE-2678](https://issues.apache.org/jira/browse/MAPREDUCE-2678) | MR-279: minimum-user-limit-percent no longer honored | Major | capacity-sched | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2677](https://issues.apache.org/jira/browse/MAPREDUCE-2677) | MR-279: 404 error while accessing pages from history server | Major | mrv2 | Ramya Sunil | Robert Joseph Evans | +| [MAPREDUCE-2670](https://issues.apache.org/jira/browse/MAPREDUCE-2670) | Fixing spelling mistake in FairSchedulerServlet.java | Trivial | . | Eli Collins | Eli Collins | +| [MAPREDUCE-2668](https://issues.apache.org/jira/browse/MAPREDUCE-2668) | MR-279: APPLICATION\_STOP is never sent to AuxServices | Blocker | mrv2 | Robert Joseph Evans | Thomas Graves | +| [MAPREDUCE-2667](https://issues.apache.org/jira/browse/MAPREDUCE-2667) | MR279: mapred job -kill leaves application in RUNNING state | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2663](https://issues.apache.org/jira/browse/MAPREDUCE-2663) | MR-279: Refactoring StateMachineFactory inner classes | Minor | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-2661](https://issues.apache.org/jira/browse/MAPREDUCE-2661) | MR-279: Accessing MapTaskImpl from TaskImpl | Minor | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-2655](https://issues.apache.org/jira/browse/MAPREDUCE-2655) | MR279: Audit logs for YARN | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2652](https://issues.apache.org/jira/browse/MAPREDUCE-2652) | MR-279: Cannot run multiple NMs on a single node | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2649](https://issues.apache.org/jira/browse/MAPREDUCE-2649) | MR279: Fate of finished Applications on RM | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2646](https://issues.apache.org/jira/browse/MAPREDUCE-2646) | MR-279: AM with same sized maps and reduces hangs in presence of failing maps | Critical | applicationmaster, mrv2 | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-2644](https://issues.apache.org/jira/browse/MAPREDUCE-2644) | NodeManager fails to create containers when NM\_LOG\_DIR is not explicitly set in the Configuration | Major | mrv2 | Josh Wills | Josh Wills | +| [MAPREDUCE-2630](https://issues.apache.org/jira/browse/MAPREDUCE-2630) | MR-279: refreshQueues leads to NPEs when used w/FifoScheduler | Minor | mrv2 | Josh Wills | Josh Wills | +| [MAPREDUCE-2628](https://issues.apache.org/jira/browse/MAPREDUCE-2628) | MR-279: Add compiled on date to NM and RM info/about page | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2625](https://issues.apache.org/jira/browse/MAPREDUCE-2625) | MR-279: Add Node Manager Version to NM info page | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2620](https://issues.apache.org/jira/browse/MAPREDUCE-2620) | Update RAID for HDFS-2087 | Major | contrib/raid | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2618](https://issues.apache.org/jira/browse/MAPREDUCE-2618) | MR-279: 0 map, 0 reduce job fails with Null Pointer Exception | Major | mrv2 | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2615](https://issues.apache.org/jira/browse/MAPREDUCE-2615) | MR 279: KillJob should go through AM whenever possible | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2603](https://issues.apache.org/jira/browse/MAPREDUCE-2603) | Gridmix system tests are failing due to high ram emulation enable by default for normal mr jobs in the trace which exceeds the solt capacity. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-2598](https://issues.apache.org/jira/browse/MAPREDUCE-2598) | MR 279: miscellaneous UI, NPE fixes for JobHistory, UI | Minor | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2595](https://issues.apache.org/jira/browse/MAPREDUCE-2595) | MR279: update yarn INSTALL doc | Minor | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2588](https://issues.apache.org/jira/browse/MAPREDUCE-2588) | Raid is not compile after DataTransferProtocol refactoring | Major | contrib/raid | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2587](https://issues.apache.org/jira/browse/MAPREDUCE-2587) | MR279: Fix RM version in the cluster-\>about page | Minor | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2582](https://issues.apache.org/jira/browse/MAPREDUCE-2582) | MR 279: Cleanup JobHistory event generation | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2581](https://issues.apache.org/jira/browse/MAPREDUCE-2581) | Spelling errors in log messages (MapTask) | Trivial | . | Dave Syer | Tim Sell | +| [MAPREDUCE-2576](https://issues.apache.org/jira/browse/MAPREDUCE-2576) | Typo in comment in SimulatorLaunchTaskAction.java | Trivial | . | Sherry Chen | Tim Sell | +| [MAPREDUCE-2575](https://issues.apache.org/jira/browse/MAPREDUCE-2575) | TestMiniMRDFSCaching fails if test.build.dir is set to something other than build/test | Major | test | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2573](https://issues.apache.org/jira/browse/MAPREDUCE-2573) | New findbugs warning after MAPREDUCE-2494 | Major | . | Todd Lipcon | Robert Joseph Evans | +| [MAPREDUCE-2569](https://issues.apache.org/jira/browse/MAPREDUCE-2569) | MR-279: Restarting resource manager with root capacity not equal to 100 percent should result in error | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2566](https://issues.apache.org/jira/browse/MAPREDUCE-2566) | MR 279: YarnConfiguration should reloadConfiguration if instantiated with a non YarnConfiguration object | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2559](https://issues.apache.org/jira/browse/MAPREDUCE-2559) | ant binary fails due to missing c++ lib dir | Major | build | Eric Yang | Eric Yang | +| [MAPREDUCE-2556](https://issues.apache.org/jira/browse/MAPREDUCE-2556) | MR 279: NodeStatus.getNodeHealthStatus().setBlah broken | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2552](https://issues.apache.org/jira/browse/MAPREDUCE-2552) | MR 279: NPE when requesting attemptids for completed jobs | Minor | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2550](https://issues.apache.org/jira/browse/MAPREDUCE-2550) | bin/mapred no longer works from a source checkout | Blocker | build | Eric Yang | Eric Yang | +| [MAPREDUCE-2541](https://issues.apache.org/jira/browse/MAPREDUCE-2541) | Race Condition in IndexCache(readIndexFileToCache,removeMap) causes value of totalMemoryUsed corrupt, which may cause TaskTracker continue throw Exception | Critical | tasktracker | Binglin Chang | Binglin Chang | +| [MAPREDUCE-2537](https://issues.apache.org/jira/browse/MAPREDUCE-2537) | MR-279: The RM writes its log to yarn-mapred-resourcemanager-\.out | Minor | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2534](https://issues.apache.org/jira/browse/MAPREDUCE-2534) | MR-279: Fix CI breaking hard coded version in jobclient pom | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2531](https://issues.apache.org/jira/browse/MAPREDUCE-2531) | org.apache.hadoop.mapred.jobcontrol.getAssignedJobID throw class cast exception | Blocker | client | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-2529](https://issues.apache.org/jira/browse/MAPREDUCE-2529) | Recognize Jetty bug 1342 and handle it | Major | tasktracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2518](https://issues.apache.org/jira/browse/MAPREDUCE-2518) | missing t flag in distcp help message '-p[rbugp]' | Major | distcp | Wei Yongjun | Wei Yongjun | +| [MAPREDUCE-2514](https://issues.apache.org/jira/browse/MAPREDUCE-2514) | ReinitTrackerAction class name misspelled RenitTrackerAction in task tracker log | Trivial | tasktracker | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-2509](https://issues.apache.org/jira/browse/MAPREDUCE-2509) | MR-279: Fix NPE in UI for pending attempts | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2504](https://issues.apache.org/jira/browse/MAPREDUCE-2504) | MR 279: race in JobHistoryEventHandler stop | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2500](https://issues.apache.org/jira/browse/MAPREDUCE-2500) | MR 279: PB factories are not thread safe | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2497](https://issues.apache.org/jira/browse/MAPREDUCE-2497) | missing spaces in error messages | Trivial | . | Robert Henry | Eli Collins | +| [MAPREDUCE-2489](https://issues.apache.org/jira/browse/MAPREDUCE-2489) | Jobsplits with random hostnames can make the queue unusable | Major | jobtracker | Jeffrey Naisbitt | Jeffrey Naisbitt | +| [MAPREDUCE-2483](https://issues.apache.org/jira/browse/MAPREDUCE-2483) | Clean up duplication of dependent jar files | Major | build | Eric Yang | Eric Yang | +| [MAPREDUCE-2480](https://issues.apache.org/jira/browse/MAPREDUCE-2480) | MR-279: mr app should not depend on hard-coded version of shuffle | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2475](https://issues.apache.org/jira/browse/MAPREDUCE-2475) | Disable IPV6 for junit tests | Major | test | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-2470](https://issues.apache.org/jira/browse/MAPREDUCE-2470) | Receiving NPE occasionally on RunningJob.getCounters() call | Major | client | Aaron Baff | Robert Joseph Evans | +| [MAPREDUCE-2467](https://issues.apache.org/jira/browse/MAPREDUCE-2467) | HDFS-1052 changes break the raid contrib module in MapReduce | Major | contrib/raid | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-2466](https://issues.apache.org/jira/browse/MAPREDUCE-2466) | TestFileInputFormat.testLocality failing after federation merge | Blocker | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2463](https://issues.apache.org/jira/browse/MAPREDUCE-2463) | Job History files are not moving to done folder when job history location is hdfs location | Major | jobtracker | Devaraj K | Devaraj K | +| [MAPREDUCE-2460](https://issues.apache.org/jira/browse/MAPREDUCE-2460) | TestFairSchedulerSystem failing on Hudson | Blocker | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2458](https://issues.apache.org/jira/browse/MAPREDUCE-2458) | MR-279: Rename sanitized pom.xml in build directory to work around IDE bug | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2451](https://issues.apache.org/jira/browse/MAPREDUCE-2451) | Log the reason string of healthcheck script | Trivial | jobtracker | Thomas Graves | Thomas Graves | +| [MAPREDUCE-2440](https://issues.apache.org/jira/browse/MAPREDUCE-2440) | MR-279: Name clashes in TypeConverter | Major | mrv2 | Luke Lu | Luke Lu | +| [MAPREDUCE-2439](https://issues.apache.org/jira/browse/MAPREDUCE-2439) | MR-279: Fix YarnRemoteException to give more details. | Major | mrv2 | Mahadev konar | Siddharth Seth | +| [MAPREDUCE-2433](https://issues.apache.org/jira/browse/MAPREDUCE-2433) | MR-279: YARNApplicationConstants hard code app master jar version | Blocker | mrv2 | Luke Lu | Mahadev konar | +| [MAPREDUCE-2429](https://issues.apache.org/jira/browse/MAPREDUCE-2429) | Check jvmid during task status report | Major | tasktracker | Arun C Murthy | Siddharth Seth | +| [MAPREDUCE-2428](https://issues.apache.org/jira/browse/MAPREDUCE-2428) | start-mapred.sh script fails if HADOOP\_HOME is not set | Blocker | . | Tom White | Tom White | +| [MAPREDUCE-2417](https://issues.apache.org/jira/browse/MAPREDUCE-2417) | In Gridmix, in RoundRobinUserResolver mode, the testing/proxy users are not associated with unique users in a trace | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-2416](https://issues.apache.org/jira/browse/MAPREDUCE-2416) | In Gridmix, in RoundRobinUserResolver, the list of groups for a user obtained from users-list-file is incorrect | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-2409](https://issues.apache.org/jira/browse/MAPREDUCE-2409) | Distributed Cache does not differentiate between file /archive for files with the same path | Major | distributed-cache | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2395](https://issues.apache.org/jira/browse/MAPREDUCE-2395) | TestBlockFixer timing out on trunk | Critical | contrib/raid | Todd Lipcon | Ramkumar Vadali | +| [MAPREDUCE-2379](https://issues.apache.org/jira/browse/MAPREDUCE-2379) | Distributed cache sizing configurations are missing from mapred-default.xml | Major | distributed-cache, documentation | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2365](https://issues.apache.org/jira/browse/MAPREDUCE-2365) | Add counters for FileInputFormat (BYTES\_READ) and FileOutputFormat (BYTES\_WRITTEN) | Major | . | Owen O'Malley | Siddharth Seth | +| [MAPREDUCE-2317](https://issues.apache.org/jira/browse/MAPREDUCE-2317) | HadoopArchives throwing NullPointerException while creating hadoop archives (.har files) | Minor | harchive | Devaraj K | Devaraj K | +| [MAPREDUCE-2311](https://issues.apache.org/jira/browse/MAPREDUCE-2311) | TestFairScheduler failing on trunk | Blocker | contrib/fair-share | Todd Lipcon | Scott Chen | +| [MAPREDUCE-2307](https://issues.apache.org/jira/browse/MAPREDUCE-2307) | Exception thrown in Jobtracker logs, when the Scheduler configured is FairScheduler. | Minor | contrib/fair-share | Devaraj K | Devaraj K | +| [MAPREDUCE-2290](https://issues.apache.org/jira/browse/MAPREDUCE-2290) | TestTaskCommit missing getProtocolSignature override | Major | test | Eli Collins | Eli Collins | +| [MAPREDUCE-2271](https://issues.apache.org/jira/browse/MAPREDUCE-2271) | TestSetupTaskScheduling failing in trunk | Blocker | jobtracker | Todd Lipcon | Liyin Liang | +| [MAPREDUCE-2258](https://issues.apache.org/jira/browse/MAPREDUCE-2258) | IFile reader closes stream and compressor in wrong order | Major | task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2215](https://issues.apache.org/jira/browse/MAPREDUCE-2215) | A more elegant FileSystem#listCorruptFileBlocks API (RAID changes) | Major | contrib/raid | Patrick Kling | Patrick Kling | +| [MAPREDUCE-2199](https://issues.apache.org/jira/browse/MAPREDUCE-2199) | build is broken 0.22 branch creation | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [MAPREDUCE-2185](https://issues.apache.org/jira/browse/MAPREDUCE-2185) | Infinite loop at creating splits using CombineFileInputFormat | Major | job submission | Hairong Kuang | Ramkumar Vadali | +| [MAPREDUCE-2172](https://issues.apache.org/jira/browse/MAPREDUCE-2172) | test-patch.properties contains incorrect/version-dependent values of OK\_FINDBUGS\_WARNINGS and OK\_RELEASEAUDIT\_WARNINGS | Major | . | Patrick Kling | Nigel Daley | +| [MAPREDUCE-2137](https://issues.apache.org/jira/browse/MAPREDUCE-2137) | Mapping between Gridmix jobs and the corresponding original MR jobs is needed | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-2127](https://issues.apache.org/jira/browse/MAPREDUCE-2127) | mapreduce trunk builds are failing on hudson | Major | build, pipes | Giridharan Kesavan | Bruno MahĂ© | +| [MAPREDUCE-2104](https://issues.apache.org/jira/browse/MAPREDUCE-2104) | Rumen TraceBuilder Does Not Emit CPU/Memory Usage Details in Traces | Major | tools/rumen | Ranjit Mathew | Amar Kamat | +| [MAPREDUCE-2074](https://issues.apache.org/jira/browse/MAPREDUCE-2074) | Task should fail when symlink creation fail | Minor | distributed-cache | Koji Noguchi | Priyo Mustafi | +| [MAPREDUCE-1996](https://issues.apache.org/jira/browse/MAPREDUCE-1996) | API: Reducer.reduce() method detail misstatement | Trivial | documentation | Glynn Durham | Harsh J | +| [MAPREDUCE-1811](https://issues.apache.org/jira/browse/MAPREDUCE-1811) | Job.monitorAndPrintJob() should print status of the job at completion | Minor | client | Amareshwari Sriramadasu | Harsh J | +| [MAPREDUCE-1788](https://issues.apache.org/jira/browse/MAPREDUCE-1788) | o.a.h.mapreduce.Job shouldn't make a copy of the JobConf | Major | client | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-1506](https://issues.apache.org/jira/browse/MAPREDUCE-1506) | Assertion failure in TestTaskTrackerMemoryManager | Major | tasktracker | Aaron Kimball | | +| [MAPREDUCE-1334](https://issues.apache.org/jira/browse/MAPREDUCE-1334) | contrib/index - test - TestIndexUpdater fails due to an additional presence of file \_SUCCESS in hdfs | Major | contrib/index | Karthik K | Karthik K | +| [MAPREDUCE-1242](https://issues.apache.org/jira/browse/MAPREDUCE-1242) | Chain APIs error misleading | Trivial | . | Amogh Vasekar | Harsh J | +| [MAPREDUCE-993](https://issues.apache.org/jira/browse/MAPREDUCE-993) | bin/hadoop job -events \ \ \<#-of-events\> help message is confusing | Minor | jobtracker | Iyappan Srinivasan | Harsh J | +| [MAPREDUCE-587](https://issues.apache.org/jira/browse/MAPREDUCE-587) | Stream test TestStreamingExitStatus fails with Out of Memory | Minor | contrib/streaming | Steve Loughran | Amar Kamat | +| [MAPREDUCE-517](https://issues.apache.org/jira/browse/MAPREDUCE-517) | The capacity-scheduler should assign multiple tasks per heartbeat | Critical | . | Arun C Murthy | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7526](https://issues.apache.org/jira/browse/HADOOP-7526) | Add TestPath tests for URI conversion and reserved characters | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-7298](https://issues.apache.org/jira/browse/HADOOP-7298) | Add test utility for writing multi-threaded tests | Major | test | Todd Lipcon | Todd Lipcon | +| [HADOOP-7230](https://issues.apache.org/jira/browse/HADOOP-7230) | Move -fs usage tests from hdfs into common | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-2522](https://issues.apache.org/jira/browse/HDFS-2522) | Disable TestDfsOverAvroRpc in 0.23 | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2332](https://issues.apache.org/jira/browse/HDFS-2332) | Add test for HADOOP-7629: using an immutable FsPermission as an IPC parameter | Major | test | Todd Lipcon | Todd Lipcon | +| [HDFS-2233](https://issues.apache.org/jira/browse/HDFS-2233) | Add WebUI tests with URI reserved chars in the path and filename | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-2154](https://issues.apache.org/jira/browse/HDFS-2154) | TestDFSShell should use test dir | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2131](https://issues.apache.org/jira/browse/HDFS-2131) | Tests for HADOOP-7361 | Major | test | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2111](https://issues.apache.org/jira/browse/HDFS-2111) | Add tests for ensuring that the DN will start with a few bad data directories (Part 1 of testing DiskChecker) | Major | datanode, test | Harsh J | Harsh J | +| [HDFS-2100](https://issues.apache.org/jira/browse/HDFS-2100) | Improve TestStorageRestore | Minor | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1983](https://issues.apache.org/jira/browse/HDFS-1983) | Fix path display for copy & rm | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1968](https://issues.apache.org/jira/browse/HDFS-1968) | Enhance TestWriteRead to support File Append and Position Read | Minor | test | CW Chung | CW Chung | +| [HDFS-1933](https://issues.apache.org/jira/browse/HDFS-1933) | Update tests for FsShell's "test" | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1931](https://issues.apache.org/jira/browse/HDFS-1931) | Update tests for du/dus/df | Major | . | Daryn Sharp | Daryn Sharp | +| [HDFS-1928](https://issues.apache.org/jira/browse/HDFS-1928) | Fix path display for touchz | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1912](https://issues.apache.org/jira/browse/HDFS-1912) | Update tests for FsShell standardized error messages | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1911](https://issues.apache.org/jira/browse/HDFS-1911) | HDFS tests for viewfs | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-1903](https://issues.apache.org/jira/browse/HDFS-1903) | Fix path display for rm/rmr | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1902](https://issues.apache.org/jira/browse/HDFS-1902) | Fix path display for setrep | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1877](https://issues.apache.org/jira/browse/HDFS-1877) | Create a functional test for file read/write | Minor | test | CW Chung | CW Chung | +| [HDFS-1862](https://issues.apache.org/jira/browse/HDFS-1862) | Improve test reliability of HDFS-1594 | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1855](https://issues.apache.org/jira/browse/HDFS-1855) | TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy() part 2 fails in two different ways | Major | test | Matt Foley | Matt Foley | +| [HDFS-1844](https://issues.apache.org/jira/browse/HDFS-1844) | Move -fs usage tests from hdfs into common | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-1770](https://issues.apache.org/jira/browse/HDFS-1770) | TestFiRename fails due to invalid block size | Minor | . | Eli Collins | Eli Collins | +| [HDFS-780](https://issues.apache.org/jira/browse/HDFS-780) | Revive TestFuseDFS | Major | fuse-dfs | Eli Collins | Eli Collins | +| [MAPREDUCE-2782](https://issues.apache.org/jira/browse/MAPREDUCE-2782) | MR-279: Unit (mockito) tests for CS | Major | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2536](https://issues.apache.org/jira/browse/MAPREDUCE-2536) | TestMRCLI broke due to change in usage output | Minor | test | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-2426](https://issues.apache.org/jira/browse/MAPREDUCE-2426) | Make TestFairSchedulerSystem fail with more verbose output | Trivial | contrib/fair-share | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2331](https://issues.apache.org/jira/browse/MAPREDUCE-2331) | Add coverage of task graph servlet to fair scheduler system test | Major | . | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2081](https://issues.apache.org/jira/browse/MAPREDUCE-2081) | [GridMix3] Implement functionality for get the list of job traces which has different intervals. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-1927](https://issues.apache.org/jira/browse/MAPREDUCE-1927) | unit test for HADOOP-6835 (concatenated gzip support) | Minor | test | Greg Roelofs | Greg Roelofs | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7753](https://issues.apache.org/jira/browse/HADOOP-7753) | Support fadvise and sync\_data\_range in NativeIO, add ReadaheadPool class | Major | io, native, performance | Todd Lipcon | Todd Lipcon | +| [HADOOP-7564](https://issues.apache.org/jira/browse/HADOOP-7564) | Remove test-patch SVN externals | Major | . | Tom White | Tom White | +| [HADOOP-7561](https://issues.apache.org/jira/browse/HADOOP-7561) | Make test-patch only run tests for changed modules | Major | . | Tom White | Tom White | +| [HADOOP-7560](https://issues.apache.org/jira/browse/HADOOP-7560) | Make hadoop-common a POM module with sub-modules (common & alfredo) | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7533](https://issues.apache.org/jira/browse/HADOOP-7533) | Allow test-patch to be run from any subproject directory | Major | . | Tom White | Tom White | +| [HADOOP-7528](https://issues.apache.org/jira/browse/HADOOP-7528) | Maven build fails in Windows | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7525](https://issues.apache.org/jira/browse/HADOOP-7525) | Make arguments to test-patch optional | Major | scripts | Tom White | Tom White | +| [HADOOP-7515](https://issues.apache.org/jira/browse/HADOOP-7515) | test-patch reports the wrong number of javadoc warnings | Major | build | Tom White | Tom White | +| [HADOOP-7508](https://issues.apache.org/jira/browse/HADOOP-7508) | compiled nativelib is in wrong directory and it is not picked up by surefire setup | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7502](https://issues.apache.org/jira/browse/HADOOP-7502) | Use canonical (IDE friendly) generated-sources directory for generated sources | Major | . | Luke Lu | Luke Lu | +| [HADOOP-7501](https://issues.apache.org/jira/browse/HADOOP-7501) | publish Hadoop Common artifacts (post HADOOP-6671) to Apache SNAPSHOTs repo | Major | build | Alejandro Abdelnur | Tom White | +| [HADOOP-7498](https://issues.apache.org/jira/browse/HADOOP-7498) | Remove legacy TAR layout creation | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7496](https://issues.apache.org/jira/browse/HADOOP-7496) | break Maven TAR & bintar profiles into just LAYOUT & TAR proper | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7469](https://issues.apache.org/jira/browse/HADOOP-7469) | add a standard handler for socket connection problems which improves diagnostics | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-7465](https://issues.apache.org/jira/browse/HADOOP-7465) | A several tiny improvements for the LOG format | Trivial | fs, ipc | XieXianshan | XieXianshan | +| [HADOOP-7380](https://issues.apache.org/jira/browse/HADOOP-7380) | Add client failover functionality to o.a.h.io.(ipc\|retry) | Major | ha, ipc | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-6919](https://issues.apache.org/jira/browse/HADOOP-6919) | Metrics2: metrics framework | Major | metrics | Luke Lu | Luke Lu | +| [HADOOP-6683](https://issues.apache.org/jira/browse/HADOOP-6683) | the first optimization: ZlibCompressor does not fully utilize the buffer | Minor | io | Kang Xiao | Kang Xiao | +| [HADOOP-6671](https://issues.apache.org/jira/browse/HADOOP-6671) | To use maven for hadoop common builds | Major | build | Giridharan Kesavan | Alejandro Abdelnur | +| [HDFS-2540](https://issues.apache.org/jira/browse/HDFS-2540) | Change WebHdfsFileSystem to two-step create/append | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2539](https://issues.apache.org/jira/browse/HDFS-2539) | Support doAs and GETHOMEDIRECTORY in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2528](https://issues.apache.org/jira/browse/HDFS-2528) | webhdfs rest call to a secure dn fails when a token is sent | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2527](https://issues.apache.org/jira/browse/HDFS-2527) | Remove the use of Range header from webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2501](https://issues.apache.org/jira/browse/HDFS-2501) | add version prefix and root methods to webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2494](https://issues.apache.org/jira/browse/HDFS-2494) | [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. | Major | webhdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2493](https://issues.apache.org/jira/browse/HDFS-2493) | Remove reference to FSNamesystem in blockmanagement classes | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2453](https://issues.apache.org/jira/browse/HDFS-2453) | tail using a webhdfs uri throws an error | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2441](https://issues.apache.org/jira/browse/HDFS-2441) | webhdfs returns two content-type headers | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2439](https://issues.apache.org/jira/browse/HDFS-2439) | webhdfs open an invalid path leads to a 500 which states a npe, we should return a 404 with appropriate error message | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2432](https://issues.apache.org/jira/browse/HDFS-2432) | webhdfs setreplication api should return a 403 when called on a directory | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2428](https://issues.apache.org/jira/browse/HDFS-2428) | webhdfs api parameter validation should be better | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2427](https://issues.apache.org/jira/browse/HDFS-2427) | webhdfs mkdirs api call creates path with 777 permission, we should default it to 755 | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2424](https://issues.apache.org/jira/browse/HDFS-2424) | webhdfs liststatus json does not convert to a valid xml document | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2416](https://issues.apache.org/jira/browse/HDFS-2416) | distcp with a webhdfs uri on a secure cluster fails | Major | webhdfs | Arpit Gupta | Jitendra Nath Pandey | +| [HDFS-2404](https://issues.apache.org/jira/browse/HDFS-2404) | webhdfs liststatus json response is not correct | Major | webhdfs | Arpit Gupta | Suresh Srinivas | +| [HDFS-2403](https://issues.apache.org/jira/browse/HDFS-2403) | The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2395](https://issues.apache.org/jira/browse/HDFS-2395) | webhdfs api's should return a root element in the json response | Critical | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2385](https://issues.apache.org/jira/browse/HDFS-2385) | Support delegation token renewal in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2366](https://issues.apache.org/jira/browse/HDFS-2366) | webhdfs throws a npe when ugi is null from getDelegationToken | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2363](https://issues.apache.org/jira/browse/HDFS-2363) | Move datanodes size printing to BlockManager from FSNameSystem's metasave API | Minor | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2356](https://issues.apache.org/jira/browse/HDFS-2356) | webhdfs: support case insensitive query parameter names | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2348](https://issues.apache.org/jira/browse/HDFS-2348) | Support getContentSummary and getFileChecksum in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2340](https://issues.apache.org/jira/browse/HDFS-2340) | Support getFileBlockLocations and getDelegationToken in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2338](https://issues.apache.org/jira/browse/HDFS-2338) | Configuration option to enable/disable webhdfs. | Major | webhdfs | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2318](https://issues.apache.org/jira/browse/HDFS-2318) | Provide authentication to webhdfs using SPNEGO | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2317](https://issues.apache.org/jira/browse/HDFS-2317) | Read access to HDFS using HTTP REST | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2284](https://issues.apache.org/jira/browse/HDFS-2284) | Write Http access to HDFS | Major | . | Sanjay Radia | Tsz Wo Nicholas Sze | +| [HDFS-2266](https://issues.apache.org/jira/browse/HDFS-2266) | Add a Namesystem interface to avoid directly referring to FSNamesystem | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2265](https://issues.apache.org/jira/browse/HDFS-2265) | Remove unnecessary BlockTokenSecretManager fields/methods from BlockManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2239](https://issues.apache.org/jira/browse/HDFS-2239) | Reduce access levels of the fields and methods in FSNamesystem | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2237](https://issues.apache.org/jira/browse/HDFS-2237) | Change UnderReplicatedBlocks from public to package private | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2228](https://issues.apache.org/jira/browse/HDFS-2228) | Move block and datanode code from FSNamesystem to BlockManager and DatanodeManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2200](https://issues.apache.org/jira/browse/HDFS-2200) | Set FSNamesystem.LOG to package private | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2199](https://issues.apache.org/jira/browse/HDFS-2199) | Move blockTokenSecretManager from FSNamesystem to BlockManager | Major | namenode | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-2191](https://issues.apache.org/jira/browse/HDFS-2191) | Move datanodeMap from FSNamesystem to DatanodeManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2167](https://issues.apache.org/jira/browse/HDFS-2167) | Move dnsToSwitchMapping and hostsReader from FSNamesystem to DatanodeManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2159](https://issues.apache.org/jira/browse/HDFS-2159) | Deprecate DistributedFileSystem.getClient() | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2149](https://issues.apache.org/jira/browse/HDFS-2149) | Move EditLogOp serialization formats into FsEditLogOp implementations | Major | namenode | Ivan Kelly | Ivan Kelly | +| [HDFS-2147](https://issues.apache.org/jira/browse/HDFS-2147) | Move cluster network topology to block management | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2141](https://issues.apache.org/jira/browse/HDFS-2141) | Remove NameNode roles Active and Standby (they become states) | Major | ha, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2140](https://issues.apache.org/jira/browse/HDFS-2140) | Move Host2NodesMap to block management | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2134](https://issues.apache.org/jira/browse/HDFS-2134) | Move DecommissionManager to block management | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2112](https://issues.apache.org/jira/browse/HDFS-2112) | Move ReplicationMonitor to block management | Major | namenode | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-2108](https://issues.apache.org/jira/browse/HDFS-2108) | Move datanode heartbeat handling to BlockManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2069](https://issues.apache.org/jira/browse/HDFS-2069) | Incorrect default trash interval value in the docs | Trivial | documentation | Ravi Phulari | Harsh J | +| [HDFS-1923](https://issues.apache.org/jira/browse/HDFS-1923) | Intermittent recurring failure in TestFiDataTransferProtocol2.pipeline\_Fi\_29 | Major | test | Matt Foley | Tsz Wo Nicholas Sze | +| [HDFS-1922](https://issues.apache.org/jira/browse/HDFS-1922) | Recurring failure in TestJMXGet.testNameNode since build 477 on May 11 | Major | test | Matt Foley | Luke Lu | +| [HDFS-1884](https://issues.apache.org/jira/browse/HDFS-1884) | Improve TestDFSStorageStateRecovery | Major | test | Matt Foley | Aaron T. Myers | +| [HDFS-1883](https://issues.apache.org/jira/browse/HDFS-1883) | Recurring failures in TestBackupNode since HDFS-1052 | Major | test | Matt Foley | | +| [HDFS-1856](https://issues.apache.org/jira/browse/HDFS-1856) | TestDatanodeBlockScanner waits forever, errs without giving information | Major | test | Matt Foley | Matt Foley | +| [HDFS-1854](https://issues.apache.org/jira/browse/HDFS-1854) | make failure message more useful in DFSTestUtil.waitReplication() | Major | test | Matt Foley | Matt Foley | +| [HDFS-1828](https://issues.apache.org/jira/browse/HDFS-1828) | TestBlocksWithNotEnoughRacks intermittently fails assert | Major | namenode | Matt Foley | Matt Foley | +| [HDFS-1826](https://issues.apache.org/jira/browse/HDFS-1826) | NameNode should save image to name directories in parallel during upgrade | Major | namenode | Hairong Kuang | Matt Foley | +| [HDFS-1767](https://issues.apache.org/jira/browse/HDFS-1767) | Namenode should ignore non-initial block reports from datanodes when in safemode during startup | Major | datanode | Matt Foley | Matt Foley | +| [HDFS-1629](https://issues.apache.org/jira/browse/HDFS-1629) | Add a method to BlockPlacementPolicy for not removing the chosen nodes | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-1557](https://issues.apache.org/jira/browse/HDFS-1557) | Separate Storage from FSImage | Major | namenode | Ivan Kelly | Ivan Kelly | +| [HDFS-1541](https://issues.apache.org/jira/browse/HDFS-1541) | Not marking datanodes dead When namenode in safemode | Major | namenode | Hairong Kuang | Hairong Kuang | +| [HDFS-1473](https://issues.apache.org/jira/browse/HDFS-1473) | Refactor storage management into separate classes than fsimage file reading/writing | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file | Major | datanode | Matt Foley | Matt Foley | +| [HDFS-1398](https://issues.apache.org/jira/browse/HDFS-1398) | HDFS federation: Upgrade and rolling back of Federation | Major | . | Tanping Wang | | +| [HDFS-1295](https://issues.apache.org/jira/browse/HDFS-1295) | Improve namenode restart times by short-circuiting the first block reports from datanodes | Major | namenode | dhruba borthakur | Matt Foley | +| [HDFS-1070](https://issues.apache.org/jira/browse/HDFS-1070) | Speedup NameNode image loading and saving by storing local file names | Major | namenode | Hairong Kuang | Hairong Kuang | +| [MAPREDUCE-3257](https://issues.apache.org/jira/browse/MAPREDUCE-3257) | Authorization checks needed for AM-\>RM protocol | Blocker | applicationmaster, mrv2, resourcemanager, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3256](https://issues.apache.org/jira/browse/MAPREDUCE-3256) | Authorization checks needed for AM-\>NM protocol | Blocker | applicationmaster, mrv2, nodemanager, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3250](https://issues.apache.org/jira/browse/MAPREDUCE-3250) | When AM restarts, client keeps reconnecting to the new AM and prints a lots of logs. | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3249](https://issues.apache.org/jira/browse/MAPREDUCE-3249) | Recovery of MR AMs with reduces fails the subsequent generation of the job | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3233](https://issues.apache.org/jira/browse/MAPREDUCE-3233) | AM fails to restart when first AM is killed | Blocker | mrv2 | Karam Singh | Mahadev konar | +| [MAPREDUCE-3220](https://issues.apache.org/jira/browse/MAPREDUCE-3220) | ant test TestCombineOutputCollector failing on trunk | Minor | mrv2, test | Hitesh Shah | Devaraj K | +| [MAPREDUCE-3175](https://issues.apache.org/jira/browse/MAPREDUCE-3175) | Yarn httpservers not created with access Control lists | Blocker | mrv2 | Thomas Graves | Jonathan Eagles | +| [MAPREDUCE-3148](https://issues.apache.org/jira/browse/MAPREDUCE-3148) | Port MAPREDUCE-2702 to old mapred api | Blocker | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3146](https://issues.apache.org/jira/browse/MAPREDUCE-3146) | Add a MR specific command line to dump logs for a given TaskAttemptID | Critical | mrv2, nodemanager | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-3144](https://issues.apache.org/jira/browse/MAPREDUCE-3144) | Augment JobHistory to include information needed for serving aggregated logs. | Critical | mrv2 | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-3141](https://issues.apache.org/jira/browse/MAPREDUCE-3141) | Yarn+MR secure mode is broken, uncovered after MAPREDUCE-3056 | Blocker | applicationmaster, mrv2, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3137](https://issues.apache.org/jira/browse/MAPREDUCE-3137) | Fix broken merge of MR-2719 to 0.23 branch for the distributed shell test case | Trivial | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3136](https://issues.apache.org/jira/browse/MAPREDUCE-3136) | Add docs for setting up real-world MRv2 clusters | Blocker | documentation, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3134](https://issues.apache.org/jira/browse/MAPREDUCE-3134) | Add documentation for CapacityScheduler | Blocker | documentation, mrv2, scheduler | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3127](https://issues.apache.org/jira/browse/MAPREDUCE-3127) | Unable to restrict users based on resourcemanager.admin.acls value set | Blocker | mrv2, resourcemanager | Amol Kekre | Arun C Murthy | +| [MAPREDUCE-3104](https://issues.apache.org/jira/browse/MAPREDUCE-3104) | Implement Application ACLs, Queue ACLs and their interaction | Blocker | mrv2, resourcemanager, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3103](https://issues.apache.org/jira/browse/MAPREDUCE-3103) | Implement Job ACLs for MRAppMaster | Blocker | mrv2, security | Vinod Kumar Vavilapalli | Mahadev konar | +| [MAPREDUCE-3099](https://issues.apache.org/jira/browse/MAPREDUCE-3099) | Add docs for setting up a single node MRv2 cluster. | Major | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3098](https://issues.apache.org/jira/browse/MAPREDUCE-3098) | Report Application status as well as ApplicationMaster status in GetApplicationReportResponse | Blocker | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3013](https://issues.apache.org/jira/browse/MAPREDUCE-3013) | Remove YarnConfiguration.YARN\_SECURITY\_INFO | Major | mrv2, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3007](https://issues.apache.org/jira/browse/MAPREDUCE-3007) | JobClient cannot talk to JobHistory server in secure mode | Major | jobhistoryserver, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2989](https://issues.apache.org/jira/browse/MAPREDUCE-2989) | JobHistory should link to task logs | Critical | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2988](https://issues.apache.org/jira/browse/MAPREDUCE-2988) | Reenable TestLinuxContainerExecutor reflecting the current NM code. | Critical | mrv2, security, test | Eric Payne | Robert Joseph Evans | +| [MAPREDUCE-2977](https://issues.apache.org/jira/browse/MAPREDUCE-2977) | ResourceManager needs to renew and cancel tokens associated with a job | Blocker | mrv2, resourcemanager, security | Owen O'Malley | Arun C Murthy | +| [MAPREDUCE-2933](https://issues.apache.org/jira/browse/MAPREDUCE-2933) | Change allocate call to return ContainerStatus for completed containers rather than Container | Blocker | applicationmaster, mrv2, nodemanager, resourcemanager | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2909](https://issues.apache.org/jira/browse/MAPREDUCE-2909) | Docs for remaining records in yarn-api | Major | documentation, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2899](https://issues.apache.org/jira/browse/MAPREDUCE-2899) | Replace major parts of ApplicationSubmissionContext with a ContainerLaunchContext | Major | mrv2, resourcemanager | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2898](https://issues.apache.org/jira/browse/MAPREDUCE-2898) | Docs for core protocols in yarn-api - ContainerManager | Major | documentation, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2897](https://issues.apache.org/jira/browse/MAPREDUCE-2897) | Docs for core protocols in yarn-api - ClientRMProtocol | Major | documentation, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2896](https://issues.apache.org/jira/browse/MAPREDUCE-2896) | Remove all apis other than getters and setters in all org/apache/hadoop/yarn/api/records/\* | Major | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2891](https://issues.apache.org/jira/browse/MAPREDUCE-2891) | Docs for core protocols in yarn-api - AMRMProtocol | Major | documentation, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-2889](https://issues.apache.org/jira/browse/MAPREDUCE-2889) | Add docs for writing new application frameworks | Critical | documentation, mrv2 | Arun C Murthy | Hitesh Shah | +| [MAPREDUCE-2858](https://issues.apache.org/jira/browse/MAPREDUCE-2858) | MRv2 WebApp Security | Blocker | applicationmaster, mrv2, security | Luke Lu | Robert Joseph Evans | +| [MAPREDUCE-2807](https://issues.apache.org/jira/browse/MAPREDUCE-2807) | MR-279: AM restart does not work after RM refactor | Major | applicationmaster, mrv2, resourcemanager | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-2792](https://issues.apache.org/jira/browse/MAPREDUCE-2792) | [MR-279] Replace IP addresses with hostnames | Blocker | mrv2, security | Ramya Sunil | Vinod Kumar Vavilapalli | +| [MAPREDUCE-2766](https://issues.apache.org/jira/browse/MAPREDUCE-2766) | [MR-279] Set correct permissions for files in dist cache | Blocker | mrv2 | Ramya Sunil | Hitesh Shah | +| [MAPREDUCE-2747](https://issues.apache.org/jira/browse/MAPREDUCE-2747) | [MR-279] [Security] Cleanup LinuxContainerExecutor binary sources | Blocker | mrv2, nodemanager, security | Vinod Kumar Vavilapalli | Robert Joseph Evans | +| [MAPREDUCE-2746](https://issues.apache.org/jira/browse/MAPREDUCE-2746) | [MR-279] [Security] Yarn servers can't communicate with each other with hadoop.security.authorization set to true | Blocker | mrv2, security | Vinod Kumar Vavilapalli | Arun C Murthy | +| [MAPREDUCE-2708](https://issues.apache.org/jira/browse/MAPREDUCE-2708) | [MR-279] Design and implement MR Application Master recovery | Blocker | applicationmaster, mrv2 | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-2702](https://issues.apache.org/jira/browse/MAPREDUCE-2702) | [MR-279] OutputCommitter changes for MR Application Master recovery | Blocker | applicationmaster, mrv2 | Sharad Agarwal | Sharad Agarwal | +| [MAPREDUCE-2696](https://issues.apache.org/jira/browse/MAPREDUCE-2696) | Container logs aren't getting cleaned up when LogAggregation is disabled | Major | mrv2, nodemanager | Arun C Murthy | Siddharth Seth | +| [MAPREDUCE-2666](https://issues.apache.org/jira/browse/MAPREDUCE-2666) | MR-279: Need to retrieve shuffle port number on ApplicationMaster restart | Blocker | mrv2 | Robert Joseph Evans | Jonathan Eagles | +| [MAPREDUCE-2641](https://issues.apache.org/jira/browse/MAPREDUCE-2641) | Fix the ExponentiallySmoothedTaskRuntimeEstimator and its unit test | Minor | mrv2 | Josh Wills | Josh Wills | +| [MAPREDUCE-2522](https://issues.apache.org/jira/browse/MAPREDUCE-2522) | MR 279: Security for JobHistory service | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-2422](https://issues.apache.org/jira/browse/MAPREDUCE-2422) | Removed unused internal methods from DistributedCache | Major | client | Tom White | Tom White | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7762](https://issues.apache.org/jira/browse/HADOOP-7762) | Common side of MR-2736 (MR1 removal) | Major | scripts | Eli Collins | Eli Collins | +| [HADOOP-7579](https://issues.apache.org/jira/browse/HADOOP-7579) | Rename package names from alfredo to auth | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7512](https://issues.apache.org/jira/browse/HADOOP-7512) | Fix example mistake in WritableComparable javadocs | Trivial | documentation | Harsh J | Harsh J | +| [HADOOP-7001](https://issues.apache.org/jira/browse/HADOOP-7001) | Allow configuration changes without restarting configured nodes | Major | conf | Patrick Kling | Patrick Kling | +| [HADOOP-6158](https://issues.apache.org/jira/browse/HADOOP-6158) | Move CyclicIteration to HDFS | Minor | util | Owen O'Malley | Eli Collins | +| [HDFS-2196](https://issues.apache.org/jira/browse/HDFS-2196) | Make ant build system work with hadoop-common JAR generated by Maven | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2096](https://issues.apache.org/jira/browse/HDFS-2096) | Mavenization of hadoop-hdfs | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-2986](https://issues.apache.org/jira/browse/MAPREDUCE-2986) | Multiple node managers support for the MiniYARNCluster | Critical | mrv2, test | Anupam Seth | Anupam Seth | +| [MAPREDUCE-2741](https://issues.apache.org/jira/browse/MAPREDUCE-2741) | Make ant build system work with hadoop-common JAR generated by Maven | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-2622](https://issues.apache.org/jira/browse/MAPREDUCE-2622) | Remove the last remaining reference to "io.sort.mb" | Minor | test | Harsh J | Harsh J | +| [MAPREDUCE-2563](https://issues.apache.org/jira/browse/MAPREDUCE-2563) | Gridmix high ram jobs emulation system tests. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-2554](https://issues.apache.org/jira/browse/MAPREDUCE-2554) | Gridmix distributed cache emulation system tests. | Major | contrib/gridmix | Vinay Kumar Thota | Yiting Wu | +| [MAPREDUCE-2544](https://issues.apache.org/jira/browse/MAPREDUCE-2544) | Gridmix compression emulation system tests. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-2517](https://issues.apache.org/jira/browse/MAPREDUCE-2517) | Porting Gridmix v3 system tests into trunk branch. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-2053](https://issues.apache.org/jira/browse/MAPREDUCE-2053) | [Herriot] Test Gridmix file pool for different input file sizes based on pool minimum size. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-2033](https://issues.apache.org/jira/browse/MAPREDUCE-2033) | [Herriot] Gridmix generate data tests with various submission policies and different user resolvers. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/RELEASENOTES.0.23.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/RELEASENOTES.0.23.0.md new file mode 100644 index 00000000000..ffe52ab025d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.0/RELEASENOTES.0.23.0.md @@ -0,0 +1,1175 @@ + + +# Apache Hadoop 0.23.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7740](https://issues.apache.org/jira/browse/HADOOP-7740) | *Minor* | **security audit logger is not on by default, fix the log4j properties to enable the logger** + +Fixed security audit logger configuration. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7728](https://issues.apache.org/jira/browse/HADOOP-7728) | *Major* | **hadoop-setup-conf.sh should be modified to enable task memory manager** + +Enable task memory management to be configurable via hadoop config setup script. + + +--- + +* [HADOOP-7724](https://issues.apache.org/jira/browse/HADOOP-7724) | *Major* | **hadoop-setup-conf.sh should put proxy user info into the core-site.xml** + +Fixed hadoop-setup-conf.sh to put proxy user in core-site.xml. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7720](https://issues.apache.org/jira/browse/HADOOP-7720) | *Major* | **improve the hadoop-setup-conf.sh to read in the hbase user and setup the configs** + +Added parameter for HBase user to setup config script. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7715](https://issues.apache.org/jira/browse/HADOOP-7715) | *Major* | **see log4j Error when running mr jobs and certain dfs calls** + +Removed unnecessary security logger configuration. (Eric Yang) + + +--- + +* [HADOOP-7711](https://issues.apache.org/jira/browse/HADOOP-7711) | *Major* | **hadoop-env.sh generated from templates has duplicate info** + +Fixed recursive sourcing of HADOOP\_OPTS environment variables (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7708](https://issues.apache.org/jira/browse/HADOOP-7708) | *Critical* | **config generator does not update the properties file if on exists already** + +Fixed hadoop-setup-conf.sh to handle config file consistently. (Eric Yang) + + +--- + +* [HADOOP-7707](https://issues.apache.org/jira/browse/HADOOP-7707) | *Major* | **improve config generator to allow users to specify proxy user, turn append on or off, turn webhdfs on or off** + +Added toggle for dfs.support.append, webhdfs and hadoop proxy user to setup config script. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7691](https://issues.apache.org/jira/browse/HADOOP-7691) | *Major* | **hadoop deb pkg should take a diff group id** + +Fixed conflict uid for install packages. (Eric Yang) + + +--- + +* [HADOOP-7684](https://issues.apache.org/jira/browse/HADOOP-7684) | *Major* | **jobhistory server and secondarynamenode should have init.d script** + +Added init.d script for jobhistory server and secondary namenode. (Eric Yang) + + +--- + +* [HADOOP-7681](https://issues.apache.org/jira/browse/HADOOP-7681) | *Minor* | **log4j.properties is missing properties for security audit and hdfs audit should be changed to info** + +HADOOP-7681. Fixed security and hdfs audit log4j properties +(Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7668](https://issues.apache.org/jira/browse/HADOOP-7668) | *Minor* | **Add a NetUtils method that can tell if an InetAddress belongs to local host** + +closing again + + +--- + +* [HADOOP-7655](https://issues.apache.org/jira/browse/HADOOP-7655) | *Major* | **provide a small validation script that smoke tests the installed cluster** + +Committed to trunk and v23, since code reviewed by Eric. + + +--- + +* [HADOOP-7603](https://issues.apache.org/jira/browse/HADOOP-7603) | *Major* | **Set default hdfs, mapred uid, and hadoop group gid for RPM packages** + +Set hdfs uid, mapred uid, and hadoop gid to fixed numbers (201, 202, and 123, respectively). + + +--- + +* [HADOOP-7547](https://issues.apache.org/jira/browse/HADOOP-7547) | *Minor* | **Fix the warning in writable classes.[ WritableComparable is a raw type. References to generic type WritableComparable\ should be parameterized ]** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-7507](https://issues.apache.org/jira/browse/HADOOP-7507) | *Major* | **jvm metrics all use the same namespace** + +JVM metrics published to Ganglia now include the process name as part of the gmetric name. + + +--- + +* [HADOOP-7379](https://issues.apache.org/jira/browse/HADOOP-7379) | *Major* | **Add ability to include Protobufs in ObjectWritable** + +Protocol buffer-generated types may now be used as arguments or return values for Hadoop RPC. + + +--- + +* [HADOOP-7374](https://issues.apache.org/jira/browse/HADOOP-7374) | *Major* | **Don't add tools.jar to the classpath when running Hadoop** + +The scripts that run Hadoop no longer automatically add tools.jar from the JDK to the classpath (if it is present). If your job depends on tools.jar in the JDK you will need to add this dependency in your job. + + +--- + +* [HADOOP-7331](https://issues.apache.org/jira/browse/HADOOP-7331) | *Trivial* | **Make hadoop-daemon.sh to return 1 if daemon processes did not get started** + +hadoop-daemon.sh now returns a non-zero exit code if it detects that the daemon was not still running after 3 seconds. + + +--- + +* [HADOOP-7322](https://issues.apache.org/jira/browse/HADOOP-7322) | *Minor* | **Adding a util method in FileUtil for JDK File.listFiles** + +Use of this new utility method avoids null result from File.listFiles(), and consequent NPEs. + + +--- + +* [HADOOP-7305](https://issues.apache.org/jira/browse/HADOOP-7305) | *Minor* | **Eclipse project files are incomplete** + +Added missing library during creation of the eclipse project files. + + +--- + +* [HADOOP-7286](https://issues.apache.org/jira/browse/HADOOP-7286) | *Major* | **Refactor FsShell's du/dus/df** + +The "Found X items" header on the output of the "du" command has been removed to more closely match unix. The displayed paths now correspond to the command line arguments instead of always being a fully qualified URI. For example, the output will have relative paths if the command line arguments are relative paths. + + +--- + +* [HADOOP-7264](https://issues.apache.org/jira/browse/HADOOP-7264) | *Major* | **Bump avro version to at least 1.4.1** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-7257](https://issues.apache.org/jira/browse/HADOOP-7257) | *Major* | **A client side mount table to give per-application/per-job file system view** + +viewfs - client-side mount table. + + +--- + +* [HADOOP-7227](https://issues.apache.org/jira/browse/HADOOP-7227) | *Major* | **Remove protocol version check at proxy creation in Hadoop RPC.** + +1. Protocol version check is removed from proxy creation, instead version check is performed at server in every rpc call. +2. This change is backward incompatible because format of the rpc messages is changed to include client version, client method hash and rpc version. +3. rpc version is introduced which should change when the format of rpc messages is changed. + + +--- + +* [HADOOP-7193](https://issues.apache.org/jira/browse/HADOOP-7193) | *Minor* | **Help message is wrong for touchz command.** + +Updated the help for the touchz command. + + +--- + +* [HADOOP-7153](https://issues.apache.org/jira/browse/HADOOP-7153) | *Minor* | **MapWritable violates contract of Map interface for equals() and hashCode()** + +MapWritable now implements equals() and hashCode() based on the map contents rather than object identity in order to correctly implement the Map interface. + + +--- + +* [HADOOP-7136](https://issues.apache.org/jira/browse/HADOOP-7136) | *Major* | **Remove failmon contrib** + +Failmon removed from contrib codebase. + + +--- + +* [HADOOP-7133](https://issues.apache.org/jira/browse/HADOOP-7133) | *Major* | **CLONE to COMMON - HDFS-1445 Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file** + +This is the COMMON portion of a fix requiring coordinated change of COMMON and HDFS. Please see HDFS-1445 for HDFS portion and release note. + + +--- + +* [HADOOP-7119](https://issues.apache.org/jira/browse/HADOOP-7119) | *Major* | **add Kerberos HTTP SPNEGO authentication support to Hadoop JT/NN/DN/TT web-consoles** + +Adding support for Kerberos HTTP SPNEGO authentication to the Hadoop web-consoles + + +--- + +* [HADOOP-7117](https://issues.apache.org/jira/browse/HADOOP-7117) | *Major* | **Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml** + +Removed references to the older fs.checkpoint.* properties that resided in core-site.xml + + +--- + +* [HADOOP-7089](https://issues.apache.org/jira/browse/HADOOP-7089) | *Minor* | **Fix link resolution logic in hadoop-config.sh** + +Updates hadoop-config.sh to always resolve symlinks when determining HADOOP\_HOME. Bash built-ins or POSIX:2001 compliant cmds are now required. + + +--- + +* [HADOOP-7059](https://issues.apache.org/jira/browse/HADOOP-7059) | *Major* | **Remove "unused" warning in native code** + +Adds \_\_attribute\_\_ ((unused)) + + +--- + +* [HADOOP-7023](https://issues.apache.org/jira/browse/HADOOP-7023) | *Major* | **Add listCorruptFileBlocks to FileSystem** + +Add a new API listCorruptFileBlocks to FIleContext that returns a list of files that have corrupt blocks. + + +--- + +* [HADOOP-6949](https://issues.apache.org/jira/browse/HADOOP-6949) | *Major* | **Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting** + +Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5. +Introduces ArrayPrimitiveWritable for a much more efficient wire format to transmit arrays of primitives over RPC. ObjectWritable uses the new writable for array of primitives for RPC and continues to use existing format for on-disk data. + + +--- + +* [HADOOP-6921](https://issues.apache.org/jira/browse/HADOOP-6921) | *Major* | **metrics2: metrics plugins** + +Metrics names are standardized to CapitalizedCamelCase. See release note of HADOOP-6918 and HADOOP-6920. + + +--- + +* [HADOOP-6920](https://issues.apache.org/jira/browse/HADOOP-6920) | *Major* | **Metrics2: metrics instrumentation** + +Metrics names are standardized to use CapitalizedCamelCase. Some examples of this is: +# Metrics names using "\_" is changed to new naming scheme. Eg: bytes\_written changes to BytesWritten. +# All metrics names start with capitals. Example: threadsBlocked changes to ThreadsBlocked. + + +--- + +* [HADOOP-6919](https://issues.apache.org/jira/browse/HADOOP-6919) | *Major* | **Metrics2: metrics framework** + +New metrics2 framework for Hadoop. + + +--- + +* [HADOOP-6904](https://issues.apache.org/jira/browse/HADOOP-6904) | *Major* | **A baby step towards inter-version RPC communications** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6864](https://issues.apache.org/jira/browse/HADOOP-6864) | *Major* | **Provide a JNI-based implementation of ShellBasedUnixGroupsNetgroupMapping (implementation of GroupMappingServiceProvider)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6683](https://issues.apache.org/jira/browse/HADOOP-6683) | *Minor* | **the first optimization: ZlibCompressor does not fully utilize the buffer** + +Improve the buffer utilization of ZlibCompressor to avoid invoking a JNI per write request. + + +--- + +* [HADOOP-6436](https://issues.apache.org/jira/browse/HADOOP-6436) | *Major* | **Remove auto-generated native build files** + +The native build run when from trunk now requires autotools, libtool and openssl dev libraries. + + +--- + +* [HADOOP-6432](https://issues.apache.org/jira/browse/HADOOP-6432) | *Major* | **Statistics support in FileContext** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6385](https://issues.apache.org/jira/browse/HADOOP-6385) | *Minor* | **dfs does not support -rmdir (was HDFS-639)** + +The "rm" family of FsShell commands now supports -rmdir and -f options. + + +--- + +* [HADOOP-6255](https://issues.apache.org/jira/browse/HADOOP-6255) | *Major* | **Create an rpm integration project** + +Added RPM/DEB packages to build system. + + +--- + +* [HADOOP-5647](https://issues.apache.org/jira/browse/HADOOP-5647) | *Major* | **TestJobHistory fails if /tmp/\_logs is not writable to. Testcase should not depend on /tmp** + +Removed dependency of testcase on /tmp and made it to use test.build.data directory instead. + + +--- + +* [HADOOP-2081](https://issues.apache.org/jira/browse/HADOOP-2081) | *Major* | **Configuration getInt, getLong, and getFloat replace invalid numbers with the default value** + +Invalid configuration values now result in a number format exception rather than the default value being used. + + +--- + +* [HDFS-2465](https://issues.apache.org/jira/browse/HDFS-2465) | *Major* | **Add HDFS support for fadvise readahead and drop-behind** + +HDFS now has the ability to use posix\_fadvise and sync\_data\_range syscalls to manage the OS buffer cache. This support is currently considered experimental, and may be enabled by configuring the following keys: +dfs.datanode.drop.cache.behind.writes - set to true to drop data out of the buffer cache after writing +dfs.datanode.drop.cache.behind.reads - set to true to drop data out of the buffer cache when performing sequential reads +dfs.datanode.sync.behind.writes - set to true to trigger dirty page writeback immediately after writing data +dfs.datanode.readahead.bytes - set to a non-zero value to trigger readahead for sequential reads + + +--- + +* [HDFS-2355](https://issues.apache.org/jira/browse/HDFS-2355) | *Major* | **Federation: enable using the same configuration file across all the nodes in the cluster.** + +This change allows when running multiple namenodes on different hosts, sharing the same configuration file across all the nodes in the cluster (Datanodes, NamNode, BackupNode, SecondaryNameNode), without the need to define dfs.federation.nameservice.id parameter. + + +--- + +* [HDFS-2338](https://issues.apache.org/jira/browse/HDFS-2338) | *Major* | **Configuration option to enable/disable webhdfs.** + +Added a conf property dfs.webhdfs.enabled for enabling/disabling webhdfs. + + +--- + +* [HDFS-2318](https://issues.apache.org/jira/browse/HDFS-2318) | *Major* | **Provide authentication to webhdfs using SPNEGO** + +Added two new conf properties dfs.web.authentication.kerberos.principal and dfs.web.authentication.kerberos.keytab for the SPNEGO servlet filter. + + +--- + +* [HDFS-2210](https://issues.apache.org/jira/browse/HDFS-2210) | *Major* | **Remove hdfsproxy** + +The hdfsproxy contrib component is no longer supported. + + +--- + +* [HDFS-2202](https://issues.apache.org/jira/browse/HDFS-2202) | *Major* | **Changes to balancer bandwidth should not require datanode restart.** + +New dfsadmin command added: [-setBalancerBandwidth \] where bandwidth is max network bandwidth in bytes per second that the balancer is allowed to use on each datanode during balacing. + +This is an incompatible change in 0.23. The versions of ClientProtocol and DatanodeProtocol are changed. + + +--- + +* [HDFS-2107](https://issues.apache.org/jira/browse/HDFS-2107) | *Major* | **Move block management code to a package** + +Moved block management codes to a new package org.apache.hadoop.hdfs.server.blockmanagement. + + +--- + +* [HDFS-2087](https://issues.apache.org/jira/browse/HDFS-2087) | *Major* | **Add methods to DataTransferProtocol interface** + +Declare methods in DataTransferProtocol interface, and change Sender and Receiver to implement the interface. + + +--- + +* [HDFS-2066](https://issues.apache.org/jira/browse/HDFS-2066) | *Major* | **Create a package and individual class files for DataTransferProtocol** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-2058](https://issues.apache.org/jira/browse/HDFS-2058) | *Major* | **DataTransfer Protocol using protobufs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-2055](https://issues.apache.org/jira/browse/HDFS-2055) | *Major* | **Add hflush support to libhdfs** + +Add hdfsHFlush to libhdfs. + + +--- + +* [HDFS-1966](https://issues.apache.org/jira/browse/HDFS-1966) | *Major* | **Encapsulate individual DataTransferProtocol op header** + +Added header classes for individual DataTransferProtocol op headers. + + +--- + +* [HDFS-1963](https://issues.apache.org/jira/browse/HDFS-1963) | *Major* | **HDFS rpm integration project** + +Create HDFS RPM package + + +--- + +* [HDFS-1945](https://issues.apache.org/jira/browse/HDFS-1945) | *Major* | **Removed deprecated fields in DataTransferProtocol** + +Removed the deprecated fields in DataTransferProtocol. + + +--- + +* [HDFS-1939](https://issues.apache.org/jira/browse/HDFS-1939) | *Major* | **ivy: test conf should not extend common conf** + +* Removed duplicated jars in test class path. + + +--- + +* [HDFS-1917](https://issues.apache.org/jira/browse/HDFS-1917) | *Major* | **Clean up duplication of dependent jar files** + +Remove packaging of duplicated third party jar files + + +--- + +* [HDFS-1869](https://issues.apache.org/jira/browse/HDFS-1869) | *Major* | **mkdirs should use the supplied permission for all of the created directories** + +A multi-level mkdir is now POSIX compliant. Instead of creating intermediate directories with the permissions of the parent directory, intermediate directories are created with permission bits of rwxrwxrwx (0777) as modified by the current umask, plus write and search permission for the owner. + + +--- + +* [HDFS-1843](https://issues.apache.org/jira/browse/HDFS-1843) | *Minor* | **Discover file not found early for file append** + +I have committed this. Thanks to Bharath! + + +--- + +* [HDFS-1826](https://issues.apache.org/jira/browse/HDFS-1826) | *Major* | **NameNode should save image to name directories in parallel during upgrade** + +I've committed this. Thanks, Matt! + + +--- + +* [HDFS-1814](https://issues.apache.org/jira/browse/HDFS-1814) | *Major* | **HDFS portion of HADOOP-7214 - Hadoop /usr/bin/groups equivalent** + +Introduces a new command, "hdfs groups", which displays what groups are associated with a user as seen by the NameNode. + + +--- + +* [HDFS-1761](https://issues.apache.org/jira/browse/HDFS-1761) | *Major* | **Add a new DataTransferProtocol operation, Op.TRANSFER\_BLOCK, instead of using RPC** + +Add a new DataTransferProtocol operation, Op.TRANSFER\_BLOCK, for transferring RBW/Finalized with acknowledgement and without using RPC. + + +--- + +* [HDFS-1723](https://issues.apache.org/jira/browse/HDFS-1723) | *Minor* | **quota errors messages should use the same scale** + +Updated the Quota exceptions to now use human readable output. + + +--- + +* [HDFS-1703](https://issues.apache.org/jira/browse/HDFS-1703) | *Minor* | **HDFS federation: Improve start/stop scripts and add script to decommission datanodes** + +The masters file is no longer used to indicate which hosts to start the 2NN on. The 2NN is now started on hosts when dfs.namenode.secondary.http-address is configured with a non-wildcard IP. + + +--- + +* [HDFS-1675](https://issues.apache.org/jira/browse/HDFS-1675) | *Major* | **Transfer RBW between datanodes** + +Added a new stage TRANSFER\_RBW to DataTransferProtocol + + +--- + +* [HDFS-1636](https://issues.apache.org/jira/browse/HDFS-1636) | *Minor* | **If dfs.name.dir points to an empty dir, namenode format shouldn't require confirmation** + +If dfs.name.dir points to an empty dir, namenode -format no longer requires confirmation. + + +--- + +* [HDFS-1626](https://issues.apache.org/jira/browse/HDFS-1626) | *Minor* | **Make BLOCK\_INVALIDATE\_LIMIT configurable** + +Added a new configuration property dfs.block.invalidate.limit for FSNamesystem.blockInvalidateLimit. + + +--- + +* [HDFS-1620](https://issues.apache.org/jira/browse/HDFS-1620) | *Minor* | **Rename HdfsConstants -\> HdfsServerConstants, FSConstants -\> HdfsConstants** + +Rename HdfsConstants interface to HdfsServerConstants, FSConstants interface to HdfsConstants + + +--- + +* [HDFS-1606](https://issues.apache.org/jira/browse/HDFS-1606) | *Major* | **Provide a stronger data guarantee in the write pipeline** + +Added two configuration properties, dfs.client.block.write.replace-datanode-on-failure.enable and dfs.client.block.write.replace-datanode-on-failure.policy. Added a new feature to replace datanode on failure in DataTransferProtocol. Added getAdditionalDatanode(..) in ClientProtocol. + + +--- + +* [HDFS-1596](https://issues.apache.org/jira/browse/HDFS-1596) | *Major* | **Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml** + +Removed references to the older fs.checkpoint.* properties that resided in core-site.xml + + +--- + +* [HDFS-1594](https://issues.apache.org/jira/browse/HDFS-1594) | *Major* | **When the disk becomes full Namenode is getting shutdown and not able to recover** + +Implemented a daemon thread to monitor the disk usage for periodically and if the disk usage reaches the threshold value, put the name node into Safe mode so that no modification to file system will occur. Once the disk usage reaches below the threshold, name node will be put out of the safe mode. Here threshold value and interval to check the disk usage are configurable. + + +--- + +* [HDFS-1582](https://issues.apache.org/jira/browse/HDFS-1582) | *Major* | **Remove auto-generated native build files** + +The native build run when from trunk now requires autotools, libtool and openssl dev libraries. + + +--- + +* [HDFS-1560](https://issues.apache.org/jira/browse/HDFS-1560) | *Minor* | **dfs.data.dir permissions should default to 700** + +The permissions on datanode data directories (configured by dfs.datanode.data.dir.perm) now default to 0700. Upon startup, the datanode will automatically change the permissions to match the configured value. + + +--- + +* [HDFS-1547](https://issues.apache.org/jira/browse/HDFS-1547) | *Major* | **Improve decommission mechanism** + +Summary of changes to the decommissioning process: +# After nodes are decommissioned, they are not shutdown. The decommissioned nodes are not used for writes. For reads, the decommissioned nodes are given as the last location to read from. +# Number of live and dead decommissioned nodes are displayed in the namenode webUI. +# Decommissioned nodes free capacity is not count towards the the cluster free capacity. + + +--- + +* [HDFS-1536](https://issues.apache.org/jira/browse/HDFS-1536) | *Major* | **Improve HDFS WebUI** + +On web UI, missing block number now becomes accurate and under-replicated blocks do not include missing blocks. + + +--- + +* [HDFS-1526](https://issues.apache.org/jira/browse/HDFS-1526) | *Major* | **Dfs client name for a map/reduce task should have some randomness** + +Make a client name has this format: DFSClient\_applicationid\_randomint\_threadid, where applicationid = mapred.task.id or else = "NONMAPREDUCE". + + +--- + +* [HDFS-1448](https://issues.apache.org/jira/browse/HDFS-1448) | *Major* | **Create multi-format parser for edits logs file, support binary and XML formats initially** + +Offline edits viewer feature adds oev tool to hdfs script. Oev makes it possible to convert edits logs to/from native binary and XML formats. It uses the same framework as Offline image viewer. + +Example usage: + +$HADOOP\_HOME/bin/hdfs oev -i edits -o output.xml + + +--- + +* [HDFS-1445](https://issues.apache.org/jira/browse/HDFS-1445) | *Major* | **Batch the calls in DataStorage to FileUtil.createHardLink(), so we call it once per directory instead of once per file** + +Batch hardlinking during "upgrade" snapshots, cutting time from aprx 8 minutes per volume to aprx 8 seconds. Validated in both Linux and Windows. Depends on prior integration with patch for HADOOP-7133. + + +--- + +* [HDFS-1381](https://issues.apache.org/jira/browse/HDFS-1381) | *Major* | **HDFS javadocs hard-code references to dfs.namenode.name.dir and dfs.datanode.data.dir parameters** + +Updated the JavaDocs to appropriately represent the new Configuration Keys that are used in the code. The docs did not match the code. + + +--- + +* [HDFS-1321](https://issues.apache.org/jira/browse/HDFS-1321) | *Minor* | **If service port and main port are the same, there is no clear log message explaining the issue.** + +Added a check to match the sure RPC and HTTP Port's on the NameNode were not set to the same value, otherwise an IOException is throw with the appropriate message. + + +--- + +* [HDFS-1120](https://issues.apache.org/jira/browse/HDFS-1120) | *Major* | **Make DataNode's block-to-device placement policy pluggable** + +Make the DataNode's block-volume choosing policy pluggable. + + +--- + +* [HDFS-1117](https://issues.apache.org/jira/browse/HDFS-1117) | *Major* | **HDFS portion of HADOOP-6728 (ovehaul metrics framework)** + +Metrics names are standardized to use CapitalizedCamelCase. Some examples: +# Metrics names using "\_" is changed to new naming scheme. Eg: bytes\_written changes to BytesWritten. +# All metrics names start with capitals. Example: threadsBlocked changes to ThreadsBlocked. + + +--- + +* [HDFS-1073](https://issues.apache.org/jira/browse/HDFS-1073) | *Major* | **Simpler model for Namenode's fs Image and edit Logs** + +The NameNode's storage layout for its name directories has been reorganized to be more robust. Each edit now has a unique transaction ID, and each file is associated with a transaction ID (for checkpoints) or a range of transaction IDs (for edit logs). + + +--- + +* [HDFS-1070](https://issues.apache.org/jira/browse/HDFS-1070) | *Major* | **Speedup NameNode image loading and saving by storing local file names** + +This changes the fsimage format to be +root directory-1 directory-2 ... directoy-n. +Each directory stores all its children in the following format: +Directory\_full\_path\_name num\_of\_children child-1 ... child-n. +Each inode stores only the last component of its path name into fsimage. +This change requires an upgrade at deployment. + + +--- + +* [MAPREDUCE-3317](https://issues.apache.org/jira/browse/MAPREDUCE-3317) | *Major* | **Rumen TraceBuilder is emiting null as hostname** + +Fixes Rumen to get correct hostName that includes rackName in attempt info. + + +--- + +* [MAPREDUCE-3241](https://issues.apache.org/jira/browse/MAPREDUCE-3241) | *Major* | **(Rumen)TraceBuilder throws IllegalArgumentException** + +Rumen is fixed to ignore the AMRestartedEvent. + + +--- + +* [MAPREDUCE-3205](https://issues.apache.org/jira/browse/MAPREDUCE-3205) | *Blocker* | **MR2 memory limits should be pmem, not vmem** + +Resource limits are now expressed and enforced in terms of physical memory, rather than virtual memory. The virtual memory limit is set as a configurable multiple of the physical limit. The NodeManager's memory usage is now configured in units of MB rather than GB. + + +--- + +* [MAPREDUCE-3186](https://issues.apache.org/jira/browse/MAPREDUCE-3186) | *Blocker* | **User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed.** + +New Yarn configuration property: + +Name: yarn.app.mapreduce.am.scheduler.connection.retries +Description: Number of times AM should retry to contact RM if connection is lost. + + +--- + +* [MAPREDUCE-3166](https://issues.apache.org/jira/browse/MAPREDUCE-3166) | *Major* | **Make Rumen use job history api instead of relying on current history file name format** + +Makes Rumen use job history api instead of relying on current history file name format. + + +--- + +* [MAPREDUCE-3157](https://issues.apache.org/jira/browse/MAPREDUCE-3157) | *Major* | **Rumen TraceBuilder is skipping analyzing 0.20 history files** + +Fixes TraceBuilder to handle 0.20 history file names also. + + +--- + +* [MAPREDUCE-3112](https://issues.apache.org/jira/browse/MAPREDUCE-3112) | *Major* | **Calling hadoop cli inside mapreduce job leads to errors** + +Removed inheritance of certain server environment variables (HADOOP\_OPTS and HADOOP\_ROOT\_LOGGER) in task attempt process. + + +--- + +* [MAPREDUCE-3081](https://issues.apache.org/jira/browse/MAPREDUCE-3081) | *Major* | **Change the name format for hadoop core and vaidya jar to be hadoop-{core/vaidya}-{version}.jar in vaidya.sh** + +contrib/vaidya/bin/vaidya.sh script fixed to use appropriate jars and classpath + + +--- + +* [MAPREDUCE-3042](https://issues.apache.org/jira/browse/MAPREDUCE-3042) | *Major* | **YARN RM fails to start** + +Simple typo fix to allow ResourceManager to start instead of fail + + +--- + +* [MAPREDUCE-3041](https://issues.apache.org/jira/browse/MAPREDUCE-3041) | *Blocker* | **Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-2930](https://issues.apache.org/jira/browse/MAPREDUCE-2930) | *Major* | **Generate state graph from the State Machine Definition** + +Generate state graph from State Machine Definition + + +--- + +* [MAPREDUCE-2858](https://issues.apache.org/jira/browse/MAPREDUCE-2858) | *Blocker* | **MRv2 WebApp Security** + +A new server has been added to yarn. It is a web proxy that sits in front of the AM web UI. The server is controlled by the yarn.web-proxy.address config. If that config is set, and it points to an address that is different then the RM web interface then a separate proxy server needs to be launched. + +This can be done by running + +yarn-daemon.sh start proxyserver + +If a separate proxy server is needed other configs also may need to be set, if security is enabled. +yarn.web-proxy.principal +yarn.web-proxy.keytab + +The proxy server is stateless and should be able to support a VIP or other load balancing sitting in front of multiple instances of this server. + + +--- + +* [MAPREDUCE-2846](https://issues.apache.org/jira/browse/MAPREDUCE-2846) | *Blocker* | **a small % of all tasks fail with DefaultTaskController** + +Fixed a race condition in writing the log index file that caused tasks to 'fail'. + + +--- + +* [MAPREDUCE-2789](https://issues.apache.org/jira/browse/MAPREDUCE-2789) | *Major* | **[MR:279] Update the scheduling info on CLI** + +"mapred/job -list" now contains map/reduce, container, and resource information. + + +--- + +* [MAPREDUCE-2764](https://issues.apache.org/jira/browse/MAPREDUCE-2764) | *Major* | **Fix renewal of dfs delegation tokens** + +Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http. + + +--- + +* [MAPREDUCE-2736](https://issues.apache.org/jira/browse/MAPREDUCE-2736) | *Major* | **Remove unused contrib components dependent on MR1** + +The pre-MR2 MapReduce implementation (JobTracker, TaskTracer, etc) and contrib components are no longer supported. This implementation is currently supported in the 0.20.20x releases. + + +--- + +* [MAPREDUCE-2719](https://issues.apache.org/jira/browse/MAPREDUCE-2719) | *Major* | **MR-279: Write a shell command application** + +Adding a simple, DistributedShell application as an alternate framework to MapReduce and to act as an illustrative example for porting applications to YARN. + + +--- + +* [MAPREDUCE-2702](https://issues.apache.org/jira/browse/MAPREDUCE-2702) | *Blocker* | **[MR-279] OutputCommitter changes for MR Application Master recovery** + +Enhance OutputCommitter and FileOutputCommitter to allow for recover of tasks across job restart. + + +--- + +* [MAPREDUCE-2649](https://issues.apache.org/jira/browse/MAPREDUCE-2649) | *Major* | **MR279: Fate of finished Applications on RM** + +New config added: + + // the maximum number of completed applications the RM keeps \yarn.server.resourcemanager.expire.applications.completed.max\ + + +--- + +* [MAPREDUCE-2606](https://issues.apache.org/jira/browse/MAPREDUCE-2606) | *Major* | **Remove IsolationRunner** + +IsolationRunner is no longer maintained. See MAPREDUCE-2637 for its replacement. + + +--- + +* [MAPREDUCE-2596](https://issues.apache.org/jira/browse/MAPREDUCE-2596) | *Major* | **Gridmix should notify job failures** + +Gridmix now prints a summary information after every run. It summarizes the runs w.r.t input trace details, input data statistics, cli arguments, data-gen runtime, simulation runtimes etc and also the cluster w.r.t map slots, reduce slots, jobtracker-address, hdfs-address etc. + + +--- + +* [MAPREDUCE-2563](https://issues.apache.org/jira/browse/MAPREDUCE-2563) | *Major* | **Gridmix high ram jobs emulation system tests.** + +Adds system tests to test the High-Ram feature in Gridmix. + + +--- + +* [MAPREDUCE-2554](https://issues.apache.org/jira/browse/MAPREDUCE-2554) | *Major* | **Gridmix distributed cache emulation system tests.** + +Adds distributed cache related system tests to Gridmix. + + +--- + +* [MAPREDUCE-2544](https://issues.apache.org/jira/browse/MAPREDUCE-2544) | *Major* | **Gridmix compression emulation system tests.** + +Adds system tests for testing the compression emulation feature of Gridmix. + + +--- + +* [MAPREDUCE-2543](https://issues.apache.org/jira/browse/MAPREDUCE-2543) | *Major* | **[Gridmix] Add support for HighRam jobs** + +Adds High-Ram feature emulation in Gridmix. + + +--- + +* [MAPREDUCE-2529](https://issues.apache.org/jira/browse/MAPREDUCE-2529) | *Major* | **Recognize Jetty bug 1342 and handle it** + +Added 2 new config parameters: + +mapreduce.reduce.shuffle.catch.exception.stack.regex +mapreduce.reduce.shuffle.catch.exception.message.regex + + +--- + +* [MAPREDUCE-2521](https://issues.apache.org/jira/browse/MAPREDUCE-2521) | *Major* | **Mapreduce RPM integration project** + +Created rpm and debian packages for MapReduce. + + +--- + +* [MAPREDUCE-2517](https://issues.apache.org/jira/browse/MAPREDUCE-2517) | *Major* | **Porting Gridmix v3 system tests into trunk branch.** + +Adds system tests to Gridmix. These system tests cover various features like job types (load and sleep), user resolvers (round-robin, submitter-user, echo) and submission modes (stress, replay and serial). + + +--- + +* [MAPREDUCE-2494](https://issues.apache.org/jira/browse/MAPREDUCE-2494) | *Major* | **Make the distributed cache delete entires using LRU priority** + +Added config option mapreduce.tasktracker.cache.local.keep.pct to the TaskTracker. It is the target percentage of the local distributed cache that should be kept in between garbage collection runs. In practice it will delete unused distributed cache entries in LRU order until the size of the cache is less than mapreduce.tasktracker.cache.local.keep.pct of the maximum cache size. This is a floating point value between 0.0 and 1.0. The default is 0.95. + + +--- + +* [MAPREDUCE-2492](https://issues.apache.org/jira/browse/MAPREDUCE-2492) | *Major* | **[MAPREDUCE] The new MapReduce API should make available task's progress to the task** + +Map and Reduce task can access the attempt's overall progress via TaskAttemptContext. + + +--- + +* [MAPREDUCE-2483](https://issues.apache.org/jira/browse/MAPREDUCE-2483) | *Major* | **Clean up duplication of dependent jar files** + +Removed duplicated hadoop-common library dependencies. + + +--- + +* [MAPREDUCE-2478](https://issues.apache.org/jira/browse/MAPREDUCE-2478) | *Major* | **MR 279: Improve history server** + +Looks great. I just committed this. Thanks Siddharth! + + +--- + +* [MAPREDUCE-2474](https://issues.apache.org/jira/browse/MAPREDUCE-2474) | *Minor* | **Add docs to the new API Partitioner on how to access Job Configuration data** + +Improve the Partitioner interface's docs to help fetch Job Configuration objects. + + +--- + +* [MAPREDUCE-2473](https://issues.apache.org/jira/browse/MAPREDUCE-2473) | *Major* | **MR portion of HADOOP-7214 - Hadoop /usr/bin/groups equivalent** + +Introduces a new command, "mapred groups", which displays what groups are associated with a user as seen by the JobTracker. + + +--- + +* [MAPREDUCE-2469](https://issues.apache.org/jira/browse/MAPREDUCE-2469) | *Major* | **Task counters should also report the total heap usage of the task** + +Task attempt's total heap usage gets recorded and published via counters as COMMITTED\_HEAP\_BYTES. + + +--- + +* [MAPREDUCE-2455](https://issues.apache.org/jira/browse/MAPREDUCE-2455) | *Major* | **Remove deprecated JobTracker.State in favour of JobTrackerStatus** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-2434](https://issues.apache.org/jira/browse/MAPREDUCE-2434) | *Major* | **MR-279: ResourceManager metrics** + +I just committed this. Thanks Luke! + + +--- + +* [MAPREDUCE-2430](https://issues.apache.org/jira/browse/MAPREDUCE-2430) | *Major* | **Remove mrunit contrib** + +MRUnit is now available as a separate Apache project. + + +--- + +* [MAPREDUCE-2417](https://issues.apache.org/jira/browse/MAPREDUCE-2417) | *Major* | **In Gridmix, in RoundRobinUserResolver mode, the testing/proxy users are not associated with unique users in a trace** + +Fixes Gridmix in RoundRobinUserResolver mode to map testing/proxy users to unique users in a trace. + + +--- + +* [MAPREDUCE-2416](https://issues.apache.org/jira/browse/MAPREDUCE-2416) | *Major* | **In Gridmix, in RoundRobinUserResolver, the list of groups for a user obtained from users-list-file is incorrect** + +Removes the restriction of specifying group names in users-list file for Gridmix in RoundRobinUserResolver mode. + + +--- + +* [MAPREDUCE-2408](https://issues.apache.org/jira/browse/MAPREDUCE-2408) | *Major* | **Make Gridmix emulate usage of data compression** + +Emulates the MapReduce compression feature in Gridmix. By default, compression emulation is turned on. Compression emulation can be disabled by setting 'gridmix.compression-emulation.enable' to 'false'. Use 'gridmix.compression-emulation.map-input.decompression-ratio', 'gridmix.compression-emulation.map-output.compression-ratio' and 'gridmix.compression-emulation.reduce-output.compression-ratio' to configure the compression ratios at map input, map output and reduce output side respectively. Currently, compression ratios in the range [0.07, 0.68] are supported. Gridmix auto detects whether map-input, map output and reduce output should emulate compression based on original job's compression related configuration parameters. + + +--- + +* [MAPREDUCE-2407](https://issues.apache.org/jira/browse/MAPREDUCE-2407) | *Major* | **Make Gridmix emulate usage of Distributed Cache files** + +Makes Gridmix emulate HDFS based distributed cache files and local file system based distributed cache files. + + +--- + +* [MAPREDUCE-2405](https://issues.apache.org/jira/browse/MAPREDUCE-2405) | *Major* | **MR-279: Implement uber-AppMaster (in-cluster LocalJobRunner for MRv2)** + +An efficient implementation of small jobs by running all tasks in the MR ApplicationMaster JVM, there-by affecting lower latency. + + +--- + +* [MAPREDUCE-2260](https://issues.apache.org/jira/browse/MAPREDUCE-2260) | *Major* | **Remove auto-generated native build files** + +The native build run when from trunk now requires autotools, libtool and openssl dev libraries. + + +--- + +* [MAPREDUCE-2254](https://issues.apache.org/jira/browse/MAPREDUCE-2254) | *Major* | **Allow setting of end-of-record delimiter for TextInputFormat** + +TextInputFormat may now split lines with delimiters other than newline, by specifying a configuration parameter "textinputformat.record.delimiter" + + +--- + +* [MAPREDUCE-2225](https://issues.apache.org/jira/browse/MAPREDUCE-2225) | *Blocker* | **MultipleOutputs should not require the use of 'Writable'** + +MultipleOutputs should not require the use/check of 'Writable' interfaces in key and value classes. + + +--- + +* [MAPREDUCE-2207](https://issues.apache.org/jira/browse/MAPREDUCE-2207) | *Major* | **Task-cleanup task should not be scheduled on the node that the task just failed** + +Task-cleanup task should not be scheduled on the node that the task just failed + + +--- + +* [MAPREDUCE-2153](https://issues.apache.org/jira/browse/MAPREDUCE-2153) | *Major* | **Bring in more job configuration properties in to the trace file** + +Adds job configuration parameters to the job trace. The configuration parameters are stored under the 'jobProperties' field as key-value pairs. + + +--- + +* [MAPREDUCE-2137](https://issues.apache.org/jira/browse/MAPREDUCE-2137) | *Major* | **Mapping between Gridmix jobs and the corresponding original MR jobs is needed** + +New configuration properties gridmix.job.original-job-id and gridmix.job.original-job-name in the configuration of simulated job are exposed/documented to gridmix user for mapping between original cluster's jobs and simulated jobs. + + +--- + +* [MAPREDUCE-2107](https://issues.apache.org/jira/browse/MAPREDUCE-2107) | *Major* | **Emulate Memory Usage of Tasks in GridMix3** + +Adds total heap usage emulation to Gridmix. Also, Gridmix can configure the simulated task's JVM heap options with max heap options obtained from the original task (via Rumen). Use 'gridmix.task.jvm-options.enable' to disable the task max heap options configuration. + + +--- + +* [MAPREDUCE-2106](https://issues.apache.org/jira/browse/MAPREDUCE-2106) | *Major* | **Emulate CPU Usage of Tasks in GridMix3** + +Adds cumulative cpu usage emulation to Gridmix + + +--- + +* [MAPREDUCE-2104](https://issues.apache.org/jira/browse/MAPREDUCE-2104) | *Major* | **Rumen TraceBuilder Does Not Emit CPU/Memory Usage Details in Traces** + +Adds cpu, physical memory, virtual memory and heap usages to TraceBuilder's output. + + +--- + +* [MAPREDUCE-2037](https://issues.apache.org/jira/browse/MAPREDUCE-2037) | *Major* | **Capturing interim progress times, CPU usage, and memory usage, when tasks reach certain progress thresholds** + +Capture intermediate task resource consumption information: +* Time taken so far +* CPU load [either at the time the data are taken, or exponentially smoothed] +* Memory load [also either at the time the data are taken, or exponentially smoothed] + +This would be taken at intervals that depend on the task progress plateaus. For example, reducers have three progress ranges - [0-1/3], (1/3-2/3], and (2/3-3/3] - where fundamentally different activities happen. Mappers have different boundaries that are not symmetrically placed [0-9/10], (9/10-1]. Data capture boundaries should coincide with activity boundaries. For the state information capture [CPU and memory] we should average over the covered interval. + + +--- + +* [MAPREDUCE-1996](https://issues.apache.org/jira/browse/MAPREDUCE-1996) | *Trivial* | **API: Reducer.reduce() method detail misstatement** + +Fix a misleading documentation note about the usage of Reporter objects in Reducers. + + +--- + +* [MAPREDUCE-1978](https://issues.apache.org/jira/browse/MAPREDUCE-1978) | *Major* | **[Rumen] TraceBuilder should provide recursive input folder scanning** + +Adds -recursive option to TraceBuilder for scanning the input directories recursively. + + +--- + +* [MAPREDUCE-1906](https://issues.apache.org/jira/browse/MAPREDUCE-1906) | *Major* | **Lower default minimum heartbeat interval for tasktracker \> Jobtracker** + +The default minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on small clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value. + + +--- + +* [MAPREDUCE-1811](https://issues.apache.org/jira/browse/MAPREDUCE-1811) | *Minor* | **Job.monitorAndPrintJob() should print status of the job at completion** + +Print the resultant status of a Job on completion instead of simply saying 'Complete'. + + +--- + +* [MAPREDUCE-1738](https://issues.apache.org/jira/browse/MAPREDUCE-1738) | *Major* | **MapReduce portion of HADOOP-6728 (ovehaul metrics framework)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-1461](https://issues.apache.org/jira/browse/MAPREDUCE-1461) | *Major* | **Feature to instruct rumen-folder utility to skip jobs worth of specific duration** + +Added a ''-starts-after' option to Rumen's Folder utility. The time duration specified after the '-starts-after' option is an offset with respect to the submit time of the first job in the input trace. Jobs in the input trace having a submit time (relative to the first job's submit time) lesser than the specified offset will be ignored. + + +--- + +* [MAPREDUCE-1242](https://issues.apache.org/jira/browse/MAPREDUCE-1242) | *Trivial* | **Chain APIs error misleading** + +Fix a misleading exception message in case the Chained Mappers have mismatch in input/output Key/Value pairs between them. + + +--- + +* [MAPREDUCE-1159](https://issues.apache.org/jira/browse/MAPREDUCE-1159) | *Trivial* | **Limit Job name on jobtracker.jsp to be 80 char long** + +Job names on jobtracker.jsp should be 80 characters long at most. + + +--- + +* [MAPREDUCE-993](https://issues.apache.org/jira/browse/MAPREDUCE-993) | *Minor* | **bin/hadoop job -events \ \ \<#-of-events\> help message is confusing** + +Added a helpful description message to the `mapred job -events` command. + + +--- + +* [MAPREDUCE-901](https://issues.apache.org/jira/browse/MAPREDUCE-901) | *Major* | **Move Framework Counters into a TaskMetric structure** + +Efficient implementation of MapReduce framework counters. + + +--- + +* [MAPREDUCE-587](https://issues.apache.org/jira/browse/MAPREDUCE-587) | *Minor* | **Stream test TestStreamingExitStatus fails with Out of Memory** + +Fixed the streaming test TestStreamingExitStatus's failure due to an OutOfMemory error by reducing the testcase's io.sort.mb. + + +--- + +* [MAPREDUCE-279](https://issues.apache.org/jira/browse/MAPREDUCE-279) | *Major* | **Map-Reduce 2.0** + +MapReduce has undergone a complete re-haul in hadoop-0.23 and we now have, what we call, MapReduce 2.0 (MRv2). + +The fundamental idea of MRv2 is to split up the two major functionalities of the JobTracker, resource management and job scheduling/monitoring, into separate daemons. The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM). An application is either a single job in the classical sense of Map-Reduce jobs or a DAG of jobs. The ResourceManager and per-node slave, the NodeManager (NM), form the data-computation framework. The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system. The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks. + +The ResourceManager has two main components: +* Scheduler (S) +* ApplicationsManager (ASM) + +The Scheduler is responsible for allocating resources to the various running applications subject to familiar constraints of capacities, queues etc. The Scheduler is pure scheduler in the sense that it performs no monitoring or tracking of status for the application. Also, it offers no guarantees on restarting failed tasks either due to application failure or hardware failures. The Scheduler performs its scheduling function based the resource requirements of the applications; it does so based on the abstract notion of a Resource Container which incorporates elements such as memory, cpu, disk, network etc. + +The Scheduler has a pluggable policy plug-in, which is responsible for partitioning the cluster resources among the various queues, applications etc. The current Map-Reduce schedulers such as the CapacityScheduler and the FairScheduler would be some examples of the plug-in. + +The CapacityScheduler supports hierarchical queues to allow for more predictable sharing of cluster resources. +The ApplicationsManager is responsible for accepting job-submissions, negotiating the first container for executing the application specific ApplicationMaster and provides the service for restarting the ApplicationMaster container on failure. + +The NodeManager is the per-machine framework agent who is responsible for launching the applications' containers, monitoring their resource usage (cpu, memory, disk, network) and reporting the same to the Scheduler. + +The per-application ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler, tracking their status and monitoring for progress. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/CHANGES.0.23.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/CHANGES.0.23.1.md new file mode 100644 index 00000000000..1d69f4e59a2 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/CHANGES.0.23.1.md @@ -0,0 +1,484 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.1 - 2012-02-17 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8013](https://issues.apache.org/jira/browse/HADOOP-8013) | ViewFileSystem does not honor setVerifyChecksum | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7470](https://issues.apache.org/jira/browse/HADOOP-7470) | move up to Jackson 1.8.8 | Minor | util | Steve Loughran | Enis Soztutar | +| [HADOOP-7348](https://issues.apache.org/jira/browse/HADOOP-7348) | Modify the option of FsShell getmerge from [addnl] to [-nl] for consistency | Major | fs | XieXianshan | XieXianshan | +| [MAPREDUCE-3720](https://issues.apache.org/jira/browse/MAPREDUCE-3720) | Command line listJobs should not visit each AM | Major | client, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7808](https://issues.apache.org/jira/browse/HADOOP-7808) | Port token service changes from 205 | Major | fs, security | Daryn Sharp | Daryn Sharp | +| [HDFS-2316](https://issues.apache.org/jira/browse/HDFS-2316) | [umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-2765](https://issues.apache.org/jira/browse/MAPREDUCE-2765) | DistCp Rewrite | Major | distcp, mrv2 | Mithun Radhakrishnan | Mithun Radhakrishnan | +| [MAPREDUCE-778](https://issues.apache.org/jira/browse/MAPREDUCE-778) | [Rumen] Need a standalone JobHistory log anonymizer | Major | tools/rumen | Hong Tang | Amar Kamat | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8027](https://issues.apache.org/jira/browse/HADOOP-8027) | Visiting /jmx on the daemon web interfaces may print unnecessary error in logs | Minor | metrics | Harsh J | Aaron T. Myers | +| [HADOOP-8015](https://issues.apache.org/jira/browse/HADOOP-8015) | ChRootFileSystem should extend FilterFileSystem | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8009](https://issues.apache.org/jira/browse/HADOOP-8009) | Create hadoop-client and hadoop-minicluster artifacts for downstream projects | Critical | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7987](https://issues.apache.org/jira/browse/HADOOP-7987) | Support setting the run-as user in unsecure mode | Major | security | Devaraj Das | Jitendra Nath Pandey | +| [HADOOP-7939](https://issues.apache.org/jira/browse/HADOOP-7939) | Improve Hadoop subcomponent integration in Hadoop 0.23 | Major | build, conf, documentation, scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-7934](https://issues.apache.org/jira/browse/HADOOP-7934) | Normalize dependencies versions across all modules | Critical | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7919](https://issues.apache.org/jira/browse/HADOOP-7919) | [Doc] Remove hadoop.logfile.\* properties. | Trivial | documentation | Harsh J | Harsh J | +| [HADOOP-7910](https://issues.apache.org/jira/browse/HADOOP-7910) | add configuration methods to handle human readable size values | Minor | conf | Sho Shimauchi | Sho Shimauchi | +| [HADOOP-7890](https://issues.apache.org/jira/browse/HADOOP-7890) | Redirect hadoop script's deprecation message to stderr | Trivial | scripts | Koji Noguchi | Koji Noguchi | +| [HADOOP-7858](https://issues.apache.org/jira/browse/HADOOP-7858) | Drop some info logging to DEBUG level in IPC, metrics, and HTTP | Trivial | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-7841](https://issues.apache.org/jira/browse/HADOOP-7841) | Run tests with non-secure random | Trivial | build | Todd Lipcon | Todd Lipcon | +| [HADOOP-7804](https://issues.apache.org/jira/browse/HADOOP-7804) | enable hadoop config generator to set dfs.block.local-path-access.user to enable short circuit read | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7792](https://issues.apache.org/jira/browse/HADOOP-7792) | Common component for HDFS-2416: Add verifyToken method to AbstractDelegationTokenSecretManager | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7777](https://issues.apache.org/jira/browse/HADOOP-7777) | Implement a base class for DNSToSwitchMapping implementations that can offer extra topology information | Major | util | Steve Loughran | Steve Loughran | +| [HADOOP-7761](https://issues.apache.org/jira/browse/HADOOP-7761) | Improve performance of raw comparisons | Major | io, performance, util | Todd Lipcon | Todd Lipcon | +| [HADOOP-7758](https://issues.apache.org/jira/browse/HADOOP-7758) | Make GlobFilter class public | Major | fs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7736](https://issues.apache.org/jira/browse/HADOOP-7736) | Remove duplicate call of Path#normalizePath during initialization. | Trivial | fs | Harsh J | Harsh J | +| [HADOOP-7657](https://issues.apache.org/jira/browse/HADOOP-7657) | Add support for LZ4 compression | Major | . | Bert Sanders | Binglin Chang | +| [HADOOP-7574](https://issues.apache.org/jira/browse/HADOOP-7574) | Improvement for FSshell -stat | Trivial | fs | XieXianshan | XieXianshan | +| [HADOOP-7504](https://issues.apache.org/jira/browse/HADOOP-7504) | hadoop-metrics.properties missing some Ganglia31 options | Trivial | metrics | Eli Collins | Harsh J | +| [HADOOP-7424](https://issues.apache.org/jira/browse/HADOOP-7424) | Log an error if the topology script doesn't handle multiple args | Major | . | Eli Collins | Uma Maheswara Rao G | +| [HADOOP-6886](https://issues.apache.org/jira/browse/HADOOP-6886) | LocalFileSystem Needs createNonRecursive API | Minor | fs | Nicolas Spiegelberg | Nicolas Spiegelberg | +| [HADOOP-6840](https://issues.apache.org/jira/browse/HADOOP-6840) | Support non-recursive create() in FileSystem & SequenceFile.Writer | Minor | fs, io | Nicolas Spiegelberg | Nicolas Spiegelberg | +| [HADOOP-6614](https://issues.apache.org/jira/browse/HADOOP-6614) | RunJar should provide more diags when it can't create a temp file | Minor | util | Steve Loughran | Jonathan Hsieh | +| [HADOOP-4515](https://issues.apache.org/jira/browse/HADOOP-4515) | conf.getBoolean must be case insensitive | Minor | . | Abhijit Bagri | Sho Shimauchi | +| [HDFS-3139](https://issues.apache.org/jira/browse/HDFS-3139) | Minor Datanode logging improvement | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-2868](https://issues.apache.org/jira/browse/HDFS-2868) | Add number of active transfer threads to the DataNode status | Minor | datanode | Harsh J | Harsh J | +| [HDFS-2826](https://issues.apache.org/jira/browse/HDFS-2826) | Test case for HDFS-1476 (safemode can initialize repl queues before exiting) | Minor | namenode, test | Todd Lipcon | Todd Lipcon | +| [HDFS-2825](https://issues.apache.org/jira/browse/HDFS-2825) | Add test hook to turn off the writer preferring its local DN | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2817](https://issues.apache.org/jira/browse/HDFS-2817) | Combine the two TestSafeMode test suites | Minor | test | Todd Lipcon | Todd Lipcon | +| [HDFS-2814](https://issues.apache.org/jira/browse/HDFS-2814) | NamenodeMXBean does not account for svn revision in the version information | Minor | . | Hitesh Shah | Hitesh Shah | +| [HDFS-2803](https://issues.apache.org/jira/browse/HDFS-2803) | Adding logging to LeaseRenewer for better lease expiration triage. | Minor | namenode | Jimmy Xiang | Jimmy Xiang | +| [HDFS-2788](https://issues.apache.org/jira/browse/HDFS-2788) | HdfsServerConstants#DN\_KEEPALIVE\_TIMEOUT is dead code | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-2761](https://issues.apache.org/jira/browse/HDFS-2761) | Improve Hadoop subcomponent integration in Hadoop 0.23 | Major | build, hdfs-client, scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-2729](https://issues.apache.org/jira/browse/HDFS-2729) | Update BlockManager's comments regarding the invalid block set | Minor | namenode | Harsh J | Harsh J | +| [HDFS-2726](https://issues.apache.org/jira/browse/HDFS-2726) | "Exception in createBlockOutputStream" shouldn't delete exception stack trace | Major | . | Michael Bieniosek | Harsh J | +| [HDFS-2675](https://issues.apache.org/jira/browse/HDFS-2675) | Reduce verbosity when double-closing edit logs | Trivial | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2654](https://issues.apache.org/jira/browse/HDFS-2654) | Make BlockReaderLocal not extend RemoteBlockReader2 | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-2653](https://issues.apache.org/jira/browse/HDFS-2653) | DFSClient should cache whether addrs are non-local when short-circuiting is enabled | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-2604](https://issues.apache.org/jira/browse/HDFS-2604) | Add a log message to show if WebHDFS is enabled | Minor | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2570](https://issues.apache.org/jira/browse/HDFS-2570) | Add descriptions for dfs.\*.https.address in hdfs-default.xml | Trivial | documentation | Eli Collins | Eli Collins | +| [HDFS-2568](https://issues.apache.org/jira/browse/HDFS-2568) | Use a set to manage child sockets in XceiverServer | Trivial | datanode | Harsh J | Harsh J | +| [HDFS-2566](https://issues.apache.org/jira/browse/HDFS-2566) | Move BPOfferService to be a non-inner class | Minor | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2563](https://issues.apache.org/jira/browse/HDFS-2563) | Some cleanup in BPOfferService | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2562](https://issues.apache.org/jira/browse/HDFS-2562) | Refactor DN configuration variables out of DataNode class | Minor | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2560](https://issues.apache.org/jira/browse/HDFS-2560) | Refactor BPOfferService to be a static inner class | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2536](https://issues.apache.org/jira/browse/HDFS-2536) | Remove unused imports | Trivial | namenode | Aaron T. Myers | Harsh J | +| [HDFS-2533](https://issues.apache.org/jira/browse/HDFS-2533) | Remove needless synchronization on FSDataSet.getBlockFile | Minor | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-2511](https://issues.apache.org/jira/browse/HDFS-2511) | Add dev script to generate HDFS protobufs | Minor | build | Todd Lipcon | Alejandro Abdelnur | +| [HDFS-2502](https://issues.apache.org/jira/browse/HDFS-2502) | hdfs-default.xml should include dfs.name.dir.restore | Minor | documentation | Eli Collins | Harsh J | +| [HDFS-2454](https://issues.apache.org/jira/browse/HDFS-2454) | Move maxXceiverCount check to before starting the thread in dataXceiver | Minor | datanode | Uma Maheswara Rao G | Harsh J | +| [HDFS-2397](https://issues.apache.org/jira/browse/HDFS-2397) | Undeprecate SecondaryNameNode | Major | namenode | Todd Lipcon | Eli Collins | +| [HDFS-2349](https://issues.apache.org/jira/browse/HDFS-2349) | DN should log a WARN, not an INFO when it detects a corruption during block transfer | Trivial | datanode | Harsh J | Harsh J | +| [HDFS-2335](https://issues.apache.org/jira/browse/HDFS-2335) | DataNodeCluster and NNStorage always pull fresh entropy | Major | datanode, namenode | Eli Collins | Uma Maheswara Rao G | +| [HDFS-2246](https://issues.apache.org/jira/browse/HDFS-2246) | Shortcut a local client reads to a Datanodes files directly | Major | . | Sanjay Radia | Jitendra Nath Pandey | +| [HDFS-2178](https://issues.apache.org/jira/browse/HDFS-2178) | HttpFS - a read/write Hadoop file system proxy | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2080](https://issues.apache.org/jira/browse/HDFS-2080) | Speed up DFS read path by lessening checksum overhead | Major | hdfs-client, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-554](https://issues.apache.org/jira/browse/HDFS-554) | BlockInfo.ensureCapacity may get a speedup from System.arraycopy() | Minor | namenode | Steve Loughran | Harsh J | +| [HDFS-362](https://issues.apache.org/jira/browse/HDFS-362) | FSEditLog should not writes long and short as UTF8 and should not use ArrayWritable for writing non-array items | Major | namenode | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [MAPREDUCE-3771](https://issues.apache.org/jira/browse/MAPREDUCE-3771) | Port MAPREDUCE-1735 to trunk/0.23 | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3756](https://issues.apache.org/jira/browse/MAPREDUCE-3756) | Make single shuffle limit configurable | Major | mrv2 | Arun C Murthy | Hitesh Shah | +| [MAPREDUCE-3693](https://issues.apache.org/jira/browse/MAPREDUCE-3693) | Add admin env to mapred-default.xml | Minor | mrv2 | Roman Shaposhnik | Roman Shaposhnik | +| [MAPREDUCE-3692](https://issues.apache.org/jira/browse/MAPREDUCE-3692) | yarn-resourcemanager out and log files can get big | Blocker | mrv2 | Eli Collins | Eli Collins | +| [MAPREDUCE-3679](https://issues.apache.org/jira/browse/MAPREDUCE-3679) | AM logs and others should not automatically refresh after every 1 second. | Major | mrv2 | Mahadev konar | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3610](https://issues.apache.org/jira/browse/MAPREDUCE-3610) | Some parts in MR use old property dfs.block.size | Minor | . | Sho Shimauchi | Sho Shimauchi | +| [MAPREDUCE-3597](https://issues.apache.org/jira/browse/MAPREDUCE-3597) | Provide a way to access other info of history file from Rumentool | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3481](https://issues.apache.org/jira/browse/MAPREDUCE-3481) | [Gridmix] Improve STRESS mode locking | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-3415](https://issues.apache.org/jira/browse/MAPREDUCE-3415) | improve MiniMRYarnCluster & DistributedShell JAR resolution | Major | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3411](https://issues.apache.org/jira/browse/MAPREDUCE-3411) | Performance Upgrade for jQuery | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3371](https://issues.apache.org/jira/browse/MAPREDUCE-3371) | Review and improve the yarn-api javadocs. | Minor | documentation, mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3369](https://issues.apache.org/jira/browse/MAPREDUCE-3369) | Migrate MR1 tests to run on MR2 using the new interfaces introduced in MAPREDUCE-3169 | Major | mrv1, mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-3360](https://issues.apache.org/jira/browse/MAPREDUCE-3360) | Provide information about lost nodes in the UI. | Critical | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3341](https://issues.apache.org/jira/browse/MAPREDUCE-3341) | Enhance logging of initalized queue limit values | Major | mrv2 | Anupam Seth | Anupam Seth | +| [MAPREDUCE-3331](https://issues.apache.org/jira/browse/MAPREDUCE-3331) | Improvement to single node cluster setup documentation for 0.23 | Minor | mrv2 | Anupam Seth | Anupam Seth | +| [MAPREDUCE-3325](https://issues.apache.org/jira/browse/MAPREDUCE-3325) | Improvements to CapacityScheduler doc | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3299](https://issues.apache.org/jira/browse/MAPREDUCE-3299) | Add AMInfo table to the AM job page | Minor | mrv2 | Siddharth Seth | Jonathan Eagles | +| [MAPREDUCE-3265](https://issues.apache.org/jira/browse/MAPREDUCE-3265) | Reduce log level on MR2 IPC construction, etc | Blocker | mrv2 | Todd Lipcon | Arun C Murthy | +| [MAPREDUCE-3238](https://issues.apache.org/jira/browse/MAPREDUCE-3238) | Small cleanup in SchedulerApp | Trivial | mrv2 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3169](https://issues.apache.org/jira/browse/MAPREDUCE-3169) | Create a new MiniMRCluster equivalent which only provides client APIs cross MR1 and MR2 | Major | mrv1, mrv2, test | Todd Lipcon | Ahmed Radwan | +| [MAPREDUCE-3147](https://issues.apache.org/jira/browse/MAPREDUCE-3147) | Handle leaf queues with the same name properly | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-2863](https://issues.apache.org/jira/browse/MAPREDUCE-2863) | Support web-services for RM & NM | Blocker | mrv2, nodemanager, resourcemanager | Arun C Murthy | Thomas Graves | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8055](https://issues.apache.org/jira/browse/HADOOP-8055) | Distribution tar.gz does not contain etc/hadoop/core-site.xml | Major | build | Eric Charles | Harsh J | +| [HADOOP-8054](https://issues.apache.org/jira/browse/HADOOP-8054) | NPE with FilterFileSystem | Critical | fs | Amareshwari Sriramadasu | Daryn Sharp | +| [HADOOP-8052](https://issues.apache.org/jira/browse/HADOOP-8052) | Hadoop Metrics2 should emit Float.MAX\_VALUE (instead of Double.MAX\_VALUE) to avoid making Ganglia's gmetad core | Major | metrics | Varun Kapoor | Varun Kapoor | +| [HADOOP-8018](https://issues.apache.org/jira/browse/HADOOP-8018) | Hudson auto test for HDFS has started throwing javadoc: warning - Error fetching URL: http://java.sun.com/javase/6/docs/api/package-list | Major | build, test | Matt Foley | Jonathan Eagles | +| [HADOOP-8012](https://issues.apache.org/jira/browse/HADOOP-8012) | hadoop-daemon.sh and yarn-daemon.sh are trying to mkdir and chow log/pid dirs which can fail | Minor | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-8006](https://issues.apache.org/jira/browse/HADOOP-8006) | TestFSInputChecker is failing in trunk. | Major | fs | Uma Maheswara Rao G | Daryn Sharp | +| [HADOOP-8002](https://issues.apache.org/jira/browse/HADOOP-8002) | SecurityUtil acquired token message should be a debug rather than info | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8001](https://issues.apache.org/jira/browse/HADOOP-8001) | ChecksumFileSystem's rename doesn't correctly handle checksum files | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8000](https://issues.apache.org/jira/browse/HADOOP-8000) | fetchdt command not available in bin/hadoop | Critical | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-7999](https://issues.apache.org/jira/browse/HADOOP-7999) | "hadoop archive" fails with ClassNotFoundException | Critical | scripts | Jason Lowe | Jason Lowe | +| [HADOOP-7998](https://issues.apache.org/jira/browse/HADOOP-7998) | CheckFileSystem does not correctly honor setVerifyChecksum | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-7993](https://issues.apache.org/jira/browse/HADOOP-7993) | Hadoop ignores old-style config options for enabling compressed output | Major | conf | Anupam Seth | Anupam Seth | +| [HADOOP-7988](https://issues.apache.org/jira/browse/HADOOP-7988) | Upper case in hostname part of the principals doesn't work with kerberos. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7986](https://issues.apache.org/jira/browse/HADOOP-7986) | Add config for History Server protocol in hadoop-policy for service level authorization. | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-7982](https://issues.apache.org/jira/browse/HADOOP-7982) | UserGroupInformation fails to login if thread's context classloader can't load HadoopLoginModule | Major | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7981](https://issues.apache.org/jira/browse/HADOOP-7981) | Improve documentation for org.apache.hadoop.io.compress.Decompressor.getRemaining | Major | io | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7975](https://issues.apache.org/jira/browse/HADOOP-7975) | Add entry to XML defaults for new LZ4 codec | Minor | . | Harsh J | Harsh J | +| [HADOOP-7974](https://issues.apache.org/jira/browse/HADOOP-7974) | TestViewFsTrash incorrectly determines the user's home directory | Major | fs, test | Eli Collins | Harsh J | +| [HADOOP-7971](https://issues.apache.org/jira/browse/HADOOP-7971) | hadoop \ removed - should be added back, but deprecated | Blocker | . | Thomas Graves | Prashant Sharma | +| [HADOOP-7964](https://issues.apache.org/jira/browse/HADOOP-7964) | Deadlock in class init. | Blocker | security, util | Kihwal Lee | Daryn Sharp | +| [HADOOP-7963](https://issues.apache.org/jira/browse/HADOOP-7963) | test failures: TestViewFileSystemWithAuthorityLocalFileSystem and TestViewFileSystemLocalFileSystem | Blocker | . | Thomas Graves | Siddharth Seth | +| [HADOOP-7949](https://issues.apache.org/jira/browse/HADOOP-7949) | Updated maxIdleTime default in the code to match core-default.xml | Trivial | ipc | Eli Collins | Eli Collins | +| [HADOOP-7948](https://issues.apache.org/jira/browse/HADOOP-7948) | Shell scripts created by hadoop-dist/pom.xml to build tar do not properly propagate failure | Minor | build | Michajlo Matijkiw | Michajlo Matijkiw | +| [HADOOP-7936](https://issues.apache.org/jira/browse/HADOOP-7936) | There's a Hoop README in the root dir of the tarball | Major | build | Eli Collins | Alejandro Abdelnur | +| [HADOOP-7933](https://issues.apache.org/jira/browse/HADOOP-7933) | Viewfs changes for MAPREDUCE-3529 | Critical | viewfs | Siddharth Seth | Siddharth Seth | +| [HADOOP-7917](https://issues.apache.org/jira/browse/HADOOP-7917) | compilation of protobuf files fails in windows/cygwin | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7914](https://issues.apache.org/jira/browse/HADOOP-7914) | duplicate declaration of hadoop-hdfs test-jar | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7912](https://issues.apache.org/jira/browse/HADOOP-7912) | test-patch should run eclipse:eclipse to verify that it does not break again | Major | build | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-7907](https://issues.apache.org/jira/browse/HADOOP-7907) | hadoop-tools JARs are not part of the distro | Blocker | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7902](https://issues.apache.org/jira/browse/HADOOP-7902) | skipping name rules setting (if already set) should be done on UGI initialization only | Major | . | Tsz Wo Nicholas Sze | Alejandro Abdelnur | +| [HADOOP-7898](https://issues.apache.org/jira/browse/HADOOP-7898) | Fix javadoc warnings in AuthenticationToken.java | Minor | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7887](https://issues.apache.org/jira/browse/HADOOP-7887) | KerberosAuthenticatorHandler is not setting KerberosName name rules from configuration | Critical | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7878](https://issues.apache.org/jira/browse/HADOOP-7878) | Regression HADOOP-7777 switch changes break HDFS tests when the isSingleSwitch() predicate is used | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-7870](https://issues.apache.org/jira/browse/HADOOP-7870) | fix SequenceFile#createWriter with boolean createParent arg to respect createParent. | Major | . | Jonathan Hsieh | Jonathan Hsieh | +| [HADOOP-7864](https://issues.apache.org/jira/browse/HADOOP-7864) | Building mvn site with Maven \< 3.0.2 causes OOM errors | Major | build | Andrew Bayer | Andrew Bayer | +| [HADOOP-7859](https://issues.apache.org/jira/browse/HADOOP-7859) | TestViewFsHdfs.testgetFileLinkStatus is failing an assert | Major | fs | Eli Collins | Eli Collins | +| [HADOOP-7854](https://issues.apache.org/jira/browse/HADOOP-7854) | UGI getCurrentUser is not synchronized | Critical | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-7853](https://issues.apache.org/jira/browse/HADOOP-7853) | multiple javax security configurations cause conflicts | Blocker | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-7851](https://issues.apache.org/jira/browse/HADOOP-7851) | Configuration.getClasses() never returns the default value. | Major | conf | Amar Kamat | Uma Maheswara Rao G | +| [HADOOP-7843](https://issues.apache.org/jira/browse/HADOOP-7843) | compilation failing because workDir not initialized in RunJar.java | Major | . | John George | John George | +| [HADOOP-7837](https://issues.apache.org/jira/browse/HADOOP-7837) | no NullAppender in the log4j config | Major | conf | Steve Loughran | Eli Collins | +| [HADOOP-7813](https://issues.apache.org/jira/browse/HADOOP-7813) | test-patch +1 patches that introduce javadoc and findbugs warnings in some cases | Major | build, test | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7811](https://issues.apache.org/jira/browse/HADOOP-7811) | TestUserGroupInformation#testGetServerSideGroups test fails in chroot | Major | security, test | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-7810](https://issues.apache.org/jira/browse/HADOOP-7810) | move hadoop archive to core from tools | Blocker | . | John George | John George | +| [HADOOP-7802](https://issues.apache.org/jira/browse/HADOOP-7802) | Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh. | Major | . | Bruno MahĂ© | Bruno MahĂ© | +| [HADOOP-7801](https://issues.apache.org/jira/browse/HADOOP-7801) | HADOOP\_PREFIX cannot be overriden | Major | build | Bruno MahĂ© | Bruno MahĂ© | +| [HADOOP-7787](https://issues.apache.org/jira/browse/HADOOP-7787) | Make source tarball use conventional name. | Major | build | Bruno MahĂ© | Bruno MahĂ© | +| [HADOOP-6490](https://issues.apache.org/jira/browse/HADOOP-6490) | Path.normalize should use StringUtils.replace in favor of String.replace | Minor | fs | Zheng Shao | Uma Maheswara Rao G | +| [HDFS-2923](https://issues.apache.org/jira/browse/HDFS-2923) | Namenode IPC handler count uses the wrong configuration key | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2893](https://issues.apache.org/jira/browse/HDFS-2893) | The start/stop scripts don't start/stop the 2NN when using the default configuration | Minor | . | Eli Collins | Eli Collins | +| [HDFS-2889](https://issues.apache.org/jira/browse/HDFS-2889) | getNumCurrentReplicas is package private but should be public on 0.23 (see HDFS-2408) | Major | hdfs-client | Gregory Chanan | Gregory Chanan | +| [HDFS-2869](https://issues.apache.org/jira/browse/HDFS-2869) | Error in Webhdfs documentation for mkdir | Minor | webhdfs | Harsh J | Harsh J | +| [HDFS-2840](https://issues.apache.org/jira/browse/HDFS-2840) | TestHostnameFilter should work with localhost or localhost.localdomain | Major | test | Eli Collins | Alejandro Abdelnur | +| [HDFS-2837](https://issues.apache.org/jira/browse/HDFS-2837) | mvn javadoc:javadoc not seeing LimitedPrivate class | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-2836](https://issues.apache.org/jira/browse/HDFS-2836) | HttpFSServer still has 2 javadoc warnings in trunk | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-2835](https://issues.apache.org/jira/browse/HDFS-2835) | Fix org.apache.hadoop.hdfs.tools.GetConf$Command Findbug issue | Major | tools | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-2827](https://issues.apache.org/jira/browse/HDFS-2827) | Cannot save namespace after renaming a directory above a file with an open lease | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2822](https://issues.apache.org/jira/browse/HDFS-2822) | processMisReplicatedBlock incorrectly identifies under-construction blocks as under-replicated | Major | ha, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2818](https://issues.apache.org/jira/browse/HDFS-2818) | dfshealth.jsp missing space between role and node name | Trivial | namenode | Todd Lipcon | Devaraj K | +| [HDFS-2816](https://issues.apache.org/jira/browse/HDFS-2816) | Fix missing license header in hadoop-hdfs-project/hadoop-hdfs-httpfs/dev-support/findbugsExcludeFile.xml | Trivial | . | Hitesh Shah | Hitesh Shah | +| [HDFS-2810](https://issues.apache.org/jira/browse/HDFS-2810) | Leases not properly getting renewed by clients | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-2791](https://issues.apache.org/jira/browse/HDFS-2791) | If block report races with closing of file, replica is incorrectly marked corrupt | Major | datanode, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2790](https://issues.apache.org/jira/browse/HDFS-2790) | FSNamesystem.setTimes throws exception with wrong configuration name in the message | Minor | . | Arpit Gupta | Arpit Gupta | +| [HDFS-2751](https://issues.apache.org/jira/browse/HDFS-2751) | Datanode drops OS cache behind reads even for short reads | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2722](https://issues.apache.org/jira/browse/HDFS-2722) | HttpFs shouldn't be using an int for block size | Major | hdfs-client | Harsh J | Harsh J | +| [HDFS-2710](https://issues.apache.org/jira/browse/HDFS-2710) | HDFS part of MAPREDUCE-3529, HADOOP-7933 | Critical | . | Siddharth Seth | | +| [HDFS-2707](https://issues.apache.org/jira/browse/HDFS-2707) | HttpFS should read the hadoop-auth secret from a file instead inline from the configuration | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2706](https://issues.apache.org/jira/browse/HDFS-2706) | Use configuration for blockInvalidateLimit if it is set | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2705](https://issues.apache.org/jira/browse/HDFS-2705) | HttpFS server should check that upload requests have correct content-type | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2658](https://issues.apache.org/jira/browse/HDFS-2658) | HttpFS introduced 70 javadoc warnings | Major | . | Eli Collins | Alejandro Abdelnur | +| [HDFS-2657](https://issues.apache.org/jira/browse/HDFS-2657) | TestHttpFSServer and TestServerWebApp are failing on trunk | Major | . | Eli Collins | Alejandro Abdelnur | +| [HDFS-2649](https://issues.apache.org/jira/browse/HDFS-2649) | eclipse:eclipse build fails for hadoop-hdfs-httpfs | Major | build | Jason Lowe | Jason Lowe | +| [HDFS-2646](https://issues.apache.org/jira/browse/HDFS-2646) | Hadoop HttpFS introduced 4 findbug warnings. | Major | . | Uma Maheswara Rao G | Alejandro Abdelnur | +| [HDFS-2640](https://issues.apache.org/jira/browse/HDFS-2640) | Javadoc generation hangs | Major | . | Tom White | Tom White | +| [HDFS-2614](https://issues.apache.org/jira/browse/HDFS-2614) | hadoop dist tarball is missing hdfs headers | Major | build | Bruno MahĂ© | Alejandro Abdelnur | +| [HDFS-2606](https://issues.apache.org/jira/browse/HDFS-2606) | webhdfs client filesystem impl must set the content-type header for create/append | Critical | webhdfs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2596](https://issues.apache.org/jira/browse/HDFS-2596) | TestDirectoryScanner doesn't test parallel scans | Major | datanode, test | Eli Collins | Eli Collins | +| [HDFS-2594](https://issues.apache.org/jira/browse/HDFS-2594) | webhdfs HTTP API should implement getDelegationTokens() instead getDelegationToken() | Critical | webhdfs | Alejandro Abdelnur | Tsz Wo Nicholas Sze | +| [HDFS-2590](https://issues.apache.org/jira/browse/HDFS-2590) | Some links in WebHDFS forrest doc do not work | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2588](https://issues.apache.org/jira/browse/HDFS-2588) | hdfs jsp pages missing DOCTYPE [post-split branches] | Trivial | scripts | Dave Vronay | Dave Vronay | +| [HDFS-2575](https://issues.apache.org/jira/browse/HDFS-2575) | DFSTestUtil may create empty files | Minor | test | Todd Lipcon | Todd Lipcon | +| [HDFS-2567](https://issues.apache.org/jira/browse/HDFS-2567) | When 0 DNs are available, show a proper error when trying to browse DFS via web UI | Major | namenode | Harsh J | Harsh J | +| [HDFS-2553](https://issues.apache.org/jira/browse/HDFS-2553) | BlockPoolSliceScanner spinning in loop | Critical | datanode | Todd Lipcon | Uma Maheswara Rao G | +| [HDFS-2545](https://issues.apache.org/jira/browse/HDFS-2545) | Webhdfs: Support multiple namenodes in federation | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2544](https://issues.apache.org/jira/browse/HDFS-2544) | Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh. | Major | scripts | Bruno MahĂ© | Bruno MahĂ© | +| [HDFS-2543](https://issues.apache.org/jira/browse/HDFS-2543) | HADOOP\_PREFIX cannot be overriden | Major | scripts | Bruno MahĂ© | Bruno MahĂ© | +| [HDFS-2541](https://issues.apache.org/jira/browse/HDFS-2541) | For a sufficiently large value of blocks, the DN Scanner may request a random number with a negative seed value. | Major | datanode | Harsh J | Harsh J | +| [HDFS-1314](https://issues.apache.org/jira/browse/HDFS-1314) | dfs.blocksize accepts only absolute value | Minor | . | Karim Saadah | Sho Shimauchi | +| [HDFS-442](https://issues.apache.org/jira/browse/HDFS-442) | dfsthroughput in test.jar throws NPE | Minor | test | Ramya Sunil | Harsh J | +| [HDFS-69](https://issues.apache.org/jira/browse/HDFS-69) | Improve dfsadmin command line help | Minor | . | Ravi Phulari | Harsh J | +| [MAPREDUCE-3880](https://issues.apache.org/jira/browse/MAPREDUCE-3880) | Allow for 32-bit container-executor | Blocker | build, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3858](https://issues.apache.org/jira/browse/MAPREDUCE-3858) | Task attempt failure during commit results in task never completing | Critical | mrv2 | Tom White | Tom White | +| [MAPREDUCE-3856](https://issues.apache.org/jira/browse/MAPREDUCE-3856) | Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm. | Critical | mrv2 | Eric Payne | Eric Payne | +| [MAPREDUCE-3843](https://issues.apache.org/jira/browse/MAPREDUCE-3843) | Job summary log file found missing on the RM host | Critical | jobhistoryserver, mrv2 | Anupam Seth | Anupam Seth | +| [MAPREDUCE-3840](https://issues.apache.org/jira/browse/MAPREDUCE-3840) | JobEndNotifier doesn't use the proxyToUse during connecting | Blocker | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3834](https://issues.apache.org/jira/browse/MAPREDUCE-3834) | If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table | Critical | mr-am, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3833](https://issues.apache.org/jira/browse/MAPREDUCE-3833) | Capacity scheduler queue refresh doesn't recompute queue capacities properly | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3828](https://issues.apache.org/jira/browse/MAPREDUCE-3828) | Broken urls: AM tracking url and jobhistory url in a single node setup. | Major | mrv2 | Ahmed Radwan | Siddharth Seth | +| [MAPREDUCE-3826](https://issues.apache.org/jira/browse/MAPREDUCE-3826) | RM UI when loaded throws a message stating Data Tables warning and then the column sorting stops working | Major | mrv2 | Arpit Gupta | Jonathan Eagles | +| [MAPREDUCE-3822](https://issues.apache.org/jira/browse/MAPREDUCE-3822) | TestJobCounters is failing intermittently on trunk and 0.23. | Critical | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3817](https://issues.apache.org/jira/browse/MAPREDUCE-3817) | bin/mapred command cannot run distcp and archive jobs | Major | mrv2 | Arpit Gupta | Arpit Gupta | +| [MAPREDUCE-3814](https://issues.apache.org/jira/browse/MAPREDUCE-3814) | MR1 compile fails | Major | mrv1, mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3808](https://issues.apache.org/jira/browse/MAPREDUCE-3808) | NPE in FileOutputCommitter when running a 0 reduce job | Blocker | mrv2 | Siddharth Seth | Robert Joseph Evans | +| [MAPREDUCE-3804](https://issues.apache.org/jira/browse/MAPREDUCE-3804) | yarn webapp interface vulnerable to cross scripting attacks | Major | jobhistoryserver, mrv2, resourcemanager | Dave Thompson | Dave Thompson | +| [MAPREDUCE-3795](https://issues.apache.org/jira/browse/MAPREDUCE-3795) | "job -status" command line output is malformed | Major | mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3794](https://issues.apache.org/jira/browse/MAPREDUCE-3794) | Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-3791](https://issues.apache.org/jira/browse/MAPREDUCE-3791) | can't build site in hadoop-yarn-server-common | Major | documentation, mrv2 | Roman Shaposhnik | Mahadev konar | +| [MAPREDUCE-3784](https://issues.apache.org/jira/browse/MAPREDUCE-3784) | maxActiveApplications(\|PerUser) per queue is too low for small clusters | Major | mrv2 | Ramya Sunil | Arun C Murthy | +| [MAPREDUCE-3780](https://issues.apache.org/jira/browse/MAPREDUCE-3780) | RM assigns containers to killed applications | Blocker | mrv2 | Ramya Sunil | Hitesh Shah | +| [MAPREDUCE-3775](https://issues.apache.org/jira/browse/MAPREDUCE-3775) | Change MiniYarnCluster to escape special chars in testname | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3774](https://issues.apache.org/jira/browse/MAPREDUCE-3774) | yarn-default.xml should be moved to hadoop-yarn-common. | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3770](https://issues.apache.org/jira/browse/MAPREDUCE-3770) | [Rumen] Zombie.getJobConf() results into NPE | Critical | tools/rumen | Amar Kamat | Amar Kamat | +| [MAPREDUCE-3765](https://issues.apache.org/jira/browse/MAPREDUCE-3765) | FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3764](https://issues.apache.org/jira/browse/MAPREDUCE-3764) | AllocatedGB etc metrics incorrect if min-allocation-mb isn't a multiple of 1GB | Critical | mrv2 | Siddharth Seth | Arun C Murthy | +| [MAPREDUCE-3762](https://issues.apache.org/jira/browse/MAPREDUCE-3762) | Resource Manager fails to come up with default capacity scheduler configs. | Critical | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3760](https://issues.apache.org/jira/browse/MAPREDUCE-3760) | Blacklisted NMs should not appear in Active nodes list | Major | mrv2 | Ramya Sunil | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3759](https://issues.apache.org/jira/browse/MAPREDUCE-3759) | ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes | Major | mrv2 | Ramya Sunil | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3754](https://issues.apache.org/jira/browse/MAPREDUCE-3754) | RM webapp should have pages filtered based on App-state | Major | mrv2, webapps | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3752](https://issues.apache.org/jira/browse/MAPREDUCE-3752) | Headroom should be capped by queue max-cap | Blocker | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3749](https://issues.apache.org/jira/browse/MAPREDUCE-3749) | ConcurrentModificationException in counter groups | Blocker | mrv2 | Tom White | Tom White | +| [MAPREDUCE-3748](https://issues.apache.org/jira/browse/MAPREDUCE-3748) | Move CS related nodeUpdate log messages to DEBUG | Minor | mrv2 | Ramya Sunil | Ramya Sunil | +| [MAPREDUCE-3747](https://issues.apache.org/jira/browse/MAPREDUCE-3747) | Memory Total is not refreshed until an app is launched | Major | mrv2 | Ramya Sunil | Arun C Murthy | +| [MAPREDUCE-3744](https://issues.apache.org/jira/browse/MAPREDUCE-3744) | Unable to retrieve application logs via "yarn logs" or "mapred job -logs" | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3742](https://issues.apache.org/jira/browse/MAPREDUCE-3742) | "yarn logs" command fails with ClassNotFoundException | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3737](https://issues.apache.org/jira/browse/MAPREDUCE-3737) | The Web Application Proxy's is not documented very well | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3735](https://issues.apache.org/jira/browse/MAPREDUCE-3735) | Add distcp jar to the distribution (tar) | Blocker | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3733](https://issues.apache.org/jira/browse/MAPREDUCE-3733) | Add Apache License Header to hadoop-distcp/pom.xml | Major | . | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3732](https://issues.apache.org/jira/browse/MAPREDUCE-3732) | CS should only use 'activeUsers with pending requests' for computing user-limits | Blocker | mrv2, resourcemanager, scheduler | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3727](https://issues.apache.org/jira/browse/MAPREDUCE-3727) | jobtoken location property in jobconf refers to wrong jobtoken file | Critical | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3723](https://issues.apache.org/jira/browse/MAPREDUCE-3723) | TestAMWebServicesJobs & TestHSWebServicesJobs incorrectly asserting tests | Major | mrv2, test, webapps | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3721](https://issues.apache.org/jira/browse/MAPREDUCE-3721) | Race in shuffle can cause it to hang | Blocker | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3717](https://issues.apache.org/jira/browse/MAPREDUCE-3717) | JobClient test jar has missing files to run all the test programs. | Blocker | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3716](https://issues.apache.org/jira/browse/MAPREDUCE-3716) | java.io.File.createTempFile fails in map/reduce tasks | Blocker | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3714](https://issues.apache.org/jira/browse/MAPREDUCE-3714) | Reduce hangs in a corner case | Blocker | mrv2, task | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3713](https://issues.apache.org/jira/browse/MAPREDUCE-3713) | Incorrect headroom reported to jobs | Blocker | mrv2, resourcemanager | Siddharth Seth | Arun C Murthy | +| [MAPREDUCE-3712](https://issues.apache.org/jira/browse/MAPREDUCE-3712) | The mapreduce tar does not contain the hadoop-mapreduce-client-jobclient-tests.jar. | Blocker | mrv2 | Ravi Prakash | Mahadev konar | +| [MAPREDUCE-3710](https://issues.apache.org/jira/browse/MAPREDUCE-3710) | last split generated by FileInputFormat.getSplits may not have the best locality | Major | mrv1, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3709](https://issues.apache.org/jira/browse/MAPREDUCE-3709) | TestDistributedShell is failing | Major | mrv2, test | Eli Collins | Hitesh Shah | +| [MAPREDUCE-3708](https://issues.apache.org/jira/browse/MAPREDUCE-3708) | Metrics: Incorrect Apps Submitted Count | Major | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3705](https://issues.apache.org/jira/browse/MAPREDUCE-3705) | ant build fails on 0.23 branch | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3703](https://issues.apache.org/jira/browse/MAPREDUCE-3703) | ResourceManager should provide node lists in JMX output | Critical | mrv2, resourcemanager | Eric Payne | Eric Payne | +| [MAPREDUCE-3702](https://issues.apache.org/jira/browse/MAPREDUCE-3702) | internal server error trying access application master via proxy with filter enabled | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3701](https://issues.apache.org/jira/browse/MAPREDUCE-3701) | Delete HadoopYarnRPC from 0.23 branch. | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3699](https://issues.apache.org/jira/browse/MAPREDUCE-3699) | Default RPC handlers are very low for YARN servers | Major | mrv2 | Vinod Kumar Vavilapalli | Hitesh Shah | +| [MAPREDUCE-3697](https://issues.apache.org/jira/browse/MAPREDUCE-3697) | Hadoop Counters API limits Oozie's working across different hadoop versions | Blocker | mrv2 | John George | Mahadev konar | +| [MAPREDUCE-3696](https://issues.apache.org/jira/browse/MAPREDUCE-3696) | MR job via oozie does not work on hadoop 23 | Blocker | mrv2 | John George | John George | +| [MAPREDUCE-3691](https://issues.apache.org/jira/browse/MAPREDUCE-3691) | webservices add support to compress response | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3689](https://issues.apache.org/jira/browse/MAPREDUCE-3689) | RM web UI doesn't handle newline in job name | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3684](https://issues.apache.org/jira/browse/MAPREDUCE-3684) | LocalDistributedCacheManager does not shut down its thread pool | Major | client | Tom White | Tom White | +| [MAPREDUCE-3683](https://issues.apache.org/jira/browse/MAPREDUCE-3683) | Capacity scheduler LeafQueues maximum capacity calculation issues | Blocker | mrv2 | Thomas Graves | Arun C Murthy | +| [MAPREDUCE-3681](https://issues.apache.org/jira/browse/MAPREDUCE-3681) | capacity scheduler LeafQueues calculate used capacity wrong | Critical | mrv2 | Thomas Graves | Arun C Murthy | +| [MAPREDUCE-3669](https://issues.apache.org/jira/browse/MAPREDUCE-3669) | Getting a lot of PriviledgedActionException / SaslException when running a job | Blocker | mrv2 | Thomas Graves | Mahadev konar | +| [MAPREDUCE-3664](https://issues.apache.org/jira/browse/MAPREDUCE-3664) | HDFS Federation Documentation has incorrect configuration example | Minor | documentation | praveen sripati | Brandon Li | +| [MAPREDUCE-3657](https://issues.apache.org/jira/browse/MAPREDUCE-3657) | State machine visualize build fails | Minor | build, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3656](https://issues.apache.org/jira/browse/MAPREDUCE-3656) | Sort job on 350 scale is consistently failing with latest MRV2 code | Blocker | applicationmaster, mrv2, resourcemanager | Karam Singh | Siddharth Seth | +| [MAPREDUCE-3652](https://issues.apache.org/jira/browse/MAPREDUCE-3652) | org.apache.hadoop.mapred.TestWebUIAuthorization.testWebUIAuthorization fails | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3651](https://issues.apache.org/jira/browse/MAPREDUCE-3651) | TestQueueManagerRefresh fails | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3649](https://issues.apache.org/jira/browse/MAPREDUCE-3649) | Job End notification gives an error on calling back. | Blocker | mrv2 | Mahadev konar | Ravi Prakash | +| [MAPREDUCE-3648](https://issues.apache.org/jira/browse/MAPREDUCE-3648) | TestJobConf failing | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3646](https://issues.apache.org/jira/browse/MAPREDUCE-3646) | Remove redundant URL info from "mapred job" output | Major | client, mrv2 | Ramya Sunil | Jonathan Eagles | +| [MAPREDUCE-3645](https://issues.apache.org/jira/browse/MAPREDUCE-3645) | TestJobHistory fails | Blocker | mrv1 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3639](https://issues.apache.org/jira/browse/MAPREDUCE-3639) | TokenCache likely broken for FileSystems which don't issue delegation tokens | Blocker | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3625](https://issues.apache.org/jira/browse/MAPREDUCE-3625) | CapacityScheduler web-ui display of queue's used capacity is broken | Critical | mrv2 | Arun C Murthy | Jason Lowe | +| [MAPREDUCE-3624](https://issues.apache.org/jira/browse/MAPREDUCE-3624) | bin/yarn script adds jdk tools.jar to the classpath. | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3617](https://issues.apache.org/jira/browse/MAPREDUCE-3617) | Remove yarn default values for resource manager and nodemanager principal | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3615](https://issues.apache.org/jira/browse/MAPREDUCE-3615) | mapred ant test failures | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3608](https://issues.apache.org/jira/browse/MAPREDUCE-3608) | MAPREDUCE-3522 commit causes compilation to fail | Major | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3604](https://issues.apache.org/jira/browse/MAPREDUCE-3604) | Streaming's check for local mode is broken | Blocker | contrib/streaming | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3596](https://issues.apache.org/jira/browse/MAPREDUCE-3596) | Sort benchmark got hang after completion of 99% map phase | Blocker | applicationmaster, mrv2 | Ravi Prakash | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3588](https://issues.apache.org/jira/browse/MAPREDUCE-3588) | bin/yarn broken after MAPREDUCE-3366 | Blocker | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3586](https://issues.apache.org/jira/browse/MAPREDUCE-3586) | Lots of AMs hanging around in PIG testing | Blocker | mr-am, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3582](https://issues.apache.org/jira/browse/MAPREDUCE-3582) | Move successfully passing MR1 tests to MR2 maven tree. | Major | mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-3579](https://issues.apache.org/jira/browse/MAPREDUCE-3579) | ConverterUtils should not include a port in a path for a URL with no port | Major | mrv2 | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-3564](https://issues.apache.org/jira/browse/MAPREDUCE-3564) | TestStagingCleanup and TestJobEndNotifier are failing on trunk. | Blocker | mrv2 | Mahadev konar | Siddharth Seth | +| [MAPREDUCE-3563](https://issues.apache.org/jira/browse/MAPREDUCE-3563) | LocalJobRunner doesn't handle Jobs using o.a.h.mapreduce.OutputCommitter | Major | mrv2 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3560](https://issues.apache.org/jira/browse/MAPREDUCE-3560) | TestRMNodeTransitions is failing on trunk | Blocker | mrv2, resourcemanager, test | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-3557](https://issues.apache.org/jira/browse/MAPREDUCE-3557) | MR1 test fail to compile because of missing hadoop-archives dependency | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3549](https://issues.apache.org/jira/browse/MAPREDUCE-3549) | write api documentation for web service apis for RM, NM, mapreduce app master, and job history server | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3544](https://issues.apache.org/jira/browse/MAPREDUCE-3544) | gridmix build is broken, requires hadoop-archives to be added as ivy dependency | Major | build, tools/rumen | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3542](https://issues.apache.org/jira/browse/MAPREDUCE-3542) | Support "FileSystemCounter" legacy counter group name for compatibility | Major | . | Tom White | Tom White | +| [MAPREDUCE-3541](https://issues.apache.org/jira/browse/MAPREDUCE-3541) | Fix broken TestJobQueueClient test | Blocker | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3537](https://issues.apache.org/jira/browse/MAPREDUCE-3537) | DefaultContainerExecutor has a race condn. with multiple concurrent containers | Blocker | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3532](https://issues.apache.org/jira/browse/MAPREDUCE-3532) | When 0 is provided as port number in yarn.nodemanager.webapp.address, NMs webserver component picks up random port, NM keeps on Reporting 0 port to RM | Critical | mrv2, nodemanager | Karam Singh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3531](https://issues.apache.org/jira/browse/MAPREDUCE-3531) | Sometimes java.lang.IllegalArgumentException: Invalid key to HMAC computation in NODE\_UPDATE also causing RM to stop scheduling | Blocker | mrv2, resourcemanager, scheduler | Karam Singh | Robert Joseph Evans | +| [MAPREDUCE-3530](https://issues.apache.org/jira/browse/MAPREDUCE-3530) | Sometimes NODE\_UPDATE to the scheduler throws an NPE causing the scheduling to stop | Blocker | mrv2, resourcemanager, scheduler | Karam Singh | Arun C Murthy | +| [MAPREDUCE-3529](https://issues.apache.org/jira/browse/MAPREDUCE-3529) | TokenCache does not cache viewfs credentials correctly | Critical | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3528](https://issues.apache.org/jira/browse/MAPREDUCE-3528) | The task timeout check interval should be configurable independent of mapreduce.task.timeout | Major | mr-am, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3527](https://issues.apache.org/jira/browse/MAPREDUCE-3527) | Fix minor API incompatibilities between 1.0 and 0.23 | Major | . | Tom White | Tom White | +| [MAPREDUCE-3522](https://issues.apache.org/jira/browse/MAPREDUCE-3522) | Capacity Scheduler ACLs not inherited by default | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3521](https://issues.apache.org/jira/browse/MAPREDUCE-3521) | Hadoop Streaming ignores unknown parameters | Minor | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3518](https://issues.apache.org/jira/browse/MAPREDUCE-3518) | mapred queue -info \ -showJobs throws NPE | Critical | client, mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3513](https://issues.apache.org/jira/browse/MAPREDUCE-3513) | Capacity Scheduler web UI has a spelling mistake for Memory. | Trivial | mrv2 | Mahadev konar | chackaravarthy | +| [MAPREDUCE-3510](https://issues.apache.org/jira/browse/MAPREDUCE-3510) | Capacity Scheduler inherited ACLs not displayed by mapred queue -showacls | Major | capacity-sched, mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3505](https://issues.apache.org/jira/browse/MAPREDUCE-3505) | yarn APPLICATION\_CLASSPATH needs to be overridable | Major | mrv2 | Bruno MahĂ© | Ahmed Radwan | +| [MAPREDUCE-3500](https://issues.apache.org/jira/browse/MAPREDUCE-3500) | MRJobConfig creates an LD\_LIBRARY\_PATH using the platform ARCH | Major | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3499](https://issues.apache.org/jira/browse/MAPREDUCE-3499) | New MiniMR does not setup proxyuser configuration correctly, thus tests using doAs do not work | Blocker | mrv2, test | Alejandro Abdelnur | John George | +| [MAPREDUCE-3496](https://issues.apache.org/jira/browse/MAPREDUCE-3496) | Yarn initializes ACL operations from capacity scheduler config in a non-deterministic order | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3490](https://issues.apache.org/jira/browse/MAPREDUCE-3490) | RMContainerAllocator counts failed maps towards Reduce ramp up | Blocker | mr-am, mrv2 | Siddharth Seth | Sharad Agarwal | +| [MAPREDUCE-3488](https://issues.apache.org/jira/browse/MAPREDUCE-3488) | Streaming jobs are failing because the main class isnt set in the pom files. | Blocker | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3487](https://issues.apache.org/jira/browse/MAPREDUCE-3487) | jobhistory web ui task counters no longer links to singletakecounter page | Critical | mrv2 | Thomas Graves | Jason Lowe | +| [MAPREDUCE-3484](https://issues.apache.org/jira/browse/MAPREDUCE-3484) | JobEndNotifier is getting interrupted before completing all its retries. | Major | mr-am, mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3479](https://issues.apache.org/jira/browse/MAPREDUCE-3479) | JobClient#getJob cannot find local jobs | Major | client | Tom White | Tom White | +| [MAPREDUCE-3478](https://issues.apache.org/jira/browse/MAPREDUCE-3478) | Cannot build against ZooKeeper 3.4.0 | Minor | mrv2 | Andrew Bayer | Tom White | +| [MAPREDUCE-3477](https://issues.apache.org/jira/browse/MAPREDUCE-3477) | Hadoop site documentation cannot be built anymore on trunk and branch-0.23 | Major | documentation, mrv2 | Bruno MahĂ© | Jonathan Eagles | +| [MAPREDUCE-3465](https://issues.apache.org/jira/browse/MAPREDUCE-3465) | org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23 | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3464](https://issues.apache.org/jira/browse/MAPREDUCE-3464) | mapreduce jsp pages missing DOCTYPE [post-split branches] | Trivial | . | Dave Vronay | Dave Vronay | +| [MAPREDUCE-3463](https://issues.apache.org/jira/browse/MAPREDUCE-3463) | Second AM fails to recover properly when first AM is killed with java.lang.IllegalArgumentException causing lost job | Blocker | applicationmaster, mrv2 | Karam Singh | Siddharth Seth | +| [MAPREDUCE-3462](https://issues.apache.org/jira/browse/MAPREDUCE-3462) | Job submission failing in JUnit tests | Blocker | mrv2, test | Amar Kamat | Ravi Prakash | +| [MAPREDUCE-3460](https://issues.apache.org/jira/browse/MAPREDUCE-3460) | MR AM can hang if containers are allocated on a node blacklisted by the AM | Blocker | mr-am, mrv2 | Siddharth Seth | Robert Joseph Evans | +| [MAPREDUCE-3458](https://issues.apache.org/jira/browse/MAPREDUCE-3458) | Fix findbugs warnings in hadoop-examples | Major | mrv2 | Arun C Murthy | Devaraj K | +| [MAPREDUCE-3456](https://issues.apache.org/jira/browse/MAPREDUCE-3456) | $HADOOP\_PREFIX/bin/yarn should set defaults for $HADOOP\_\*\_HOME | Blocker | mrv2 | Eric Payne | Eric Payne | +| [MAPREDUCE-3454](https://issues.apache.org/jira/browse/MAPREDUCE-3454) | [Gridmix] TestDistCacheEmulation is broken | Major | contrib/gridmix | Amar Kamat | Hitesh Shah | +| [MAPREDUCE-3453](https://issues.apache.org/jira/browse/MAPREDUCE-3453) | RM web ui application details page shows RM cluster about information | Major | mrv2 | Thomas Graves | Jonathan Eagles | +| [MAPREDUCE-3452](https://issues.apache.org/jira/browse/MAPREDUCE-3452) | fifoscheduler web ui page always shows 0% used for the queue | Major | mrv2 | Thomas Graves | Jonathan Eagles | +| [MAPREDUCE-3450](https://issues.apache.org/jira/browse/MAPREDUCE-3450) | NM port info no longer available in JobHistory | Major | mr-am, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3448](https://issues.apache.org/jira/browse/MAPREDUCE-3448) | TestCombineOutputCollector javac unchecked warning on mocked generics | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3447](https://issues.apache.org/jira/browse/MAPREDUCE-3447) | mapreduce examples not working | Blocker | mrv2 | Thomas Graves | Mahadev konar | +| [MAPREDUCE-3444](https://issues.apache.org/jira/browse/MAPREDUCE-3444) | trunk/0.23 builds broken | Blocker | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3443](https://issues.apache.org/jira/browse/MAPREDUCE-3443) | Oozie jobs are running as oozie user even though they create the jobclient as doAs. | Blocker | mrv2 | Mahadev konar | Mahadev konar | +| [MAPREDUCE-3437](https://issues.apache.org/jira/browse/MAPREDUCE-3437) | Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT | Blocker | build, mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3436](https://issues.apache.org/jira/browse/MAPREDUCE-3436) | JobHistory webapp address should use the host from the jobhistory address | Major | mrv2, webapps | Bruno MahĂ© | Ahmed Radwan | +| [MAPREDUCE-3434](https://issues.apache.org/jira/browse/MAPREDUCE-3434) | Nightly build broken | Blocker | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3427](https://issues.apache.org/jira/browse/MAPREDUCE-3427) | streaming tests fail with MR2 | Blocker | contrib/streaming, mrv2 | Alejandro Abdelnur | Hitesh Shah | +| [MAPREDUCE-3422](https://issues.apache.org/jira/browse/MAPREDUCE-3422) | Counter display names are not being picked up | Major | mrv2 | Tom White | Jonathan Eagles | +| [MAPREDUCE-3420](https://issues.apache.org/jira/browse/MAPREDUCE-3420) | [Umbrella ticket] Make uber jobs functional | Major | mrv2 | Hitesh Shah | | +| [MAPREDUCE-3417](https://issues.apache.org/jira/browse/MAPREDUCE-3417) | job access controls not working app master and job history UI's | Blocker | mrv2 | Thomas Graves | Jonathan Eagles | +| [MAPREDUCE-3413](https://issues.apache.org/jira/browse/MAPREDUCE-3413) | RM web ui applications not sorted in any order by default | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3412](https://issues.apache.org/jira/browse/MAPREDUCE-3412) | 'ant docs' is broken | Major | . | Amar Kamat | Amar Kamat | +| [MAPREDUCE-3408](https://issues.apache.org/jira/browse/MAPREDUCE-3408) | yarn-daemon.sh unconditionnaly sets yarn.root.logger | Major | mrv2, nodemanager, resourcemanager | Bruno MahĂ© | Bruno MahĂ© | +| [MAPREDUCE-3407](https://issues.apache.org/jira/browse/MAPREDUCE-3407) | Wrong jar getting used in TestMR\*Jobs\* for MiniMRYarnCluster | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3404](https://issues.apache.org/jira/browse/MAPREDUCE-3404) | Speculative Execution: speculative map tasks launched even if -Dmapreduce.map.speculative=false | Critical | job submission, mrv2 | patrick white | Eric Payne | +| [MAPREDUCE-3398](https://issues.apache.org/jira/browse/MAPREDUCE-3398) | Log Aggregation broken in Secure Mode | Blocker | mrv2, nodemanager | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3391](https://issues.apache.org/jira/browse/MAPREDUCE-3391) | Connecting to CM is logged as Connecting to RM | Minor | applicationmaster | Subroto Sanyal | Subroto Sanyal | +| [MAPREDUCE-3389](https://issues.apache.org/jira/browse/MAPREDUCE-3389) | MRApps loads the 'mrapp-generated-classpath' file with classpath from the build machine | Critical | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3387](https://issues.apache.org/jira/browse/MAPREDUCE-3387) | A tracking URL of N/A before the app master is launched breaks oozie | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3382](https://issues.apache.org/jira/browse/MAPREDUCE-3382) | Network ACLs can prevent AMs to ping the Job-end notification URL | Critical | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Ravi Prakash | +| [MAPREDUCE-3379](https://issues.apache.org/jira/browse/MAPREDUCE-3379) | LocalResourceTracker should not tracking deleted cache entries | Major | mrv2, nodemanager | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3376](https://issues.apache.org/jira/browse/MAPREDUCE-3376) | Old mapred API combiner uses NULL reporter | Major | mrv1, mrv2 | Robert Joseph Evans | Subroto Sanyal | +| [MAPREDUCE-3373](https://issues.apache.org/jira/browse/MAPREDUCE-3373) | Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh. | Major | . | Bruno MahĂ© | Bruno MahĂ© | +| [MAPREDUCE-3372](https://issues.apache.org/jira/browse/MAPREDUCE-3372) | HADOOP\_PREFIX cannot be overriden | Major | . | Bruno MahĂ© | Bruno MahĂ© | +| [MAPREDUCE-3370](https://issues.apache.org/jira/browse/MAPREDUCE-3370) | MiniMRYarnCluster uses a hard coded path location for the MapReduce application jar | Major | mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-3368](https://issues.apache.org/jira/browse/MAPREDUCE-3368) | compile-mapred-test fails | Critical | build, mrv2 | Ramya Sunil | Hitesh Shah | +| [MAPREDUCE-3366](https://issues.apache.org/jira/browse/MAPREDUCE-3366) | Mapreduce component should use consistent directory structure layout as HDFS/common | Major | mrv2 | Eric Yang | Eric Yang | +| [MAPREDUCE-3355](https://issues.apache.org/jira/browse/MAPREDUCE-3355) | AM scheduling hangs frequently with sort job on 350 nodes | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3354](https://issues.apache.org/jira/browse/MAPREDUCE-3354) | JobHistoryServer should be started by bin/mapred and not by bin/yarn | Blocker | jobhistoryserver, mrv2 | Vinod Kumar Vavilapalli | Jonathan Eagles | +| [MAPREDUCE-3349](https://issues.apache.org/jira/browse/MAPREDUCE-3349) | No rack-name logged in JobHistory for unsuccessful tasks | Blocker | mrv2 | Vinod Kumar Vavilapalli | Amar Kamat | +| [MAPREDUCE-3346](https://issues.apache.org/jira/browse/MAPREDUCE-3346) | Rumen LoggedTaskAttempt getHostName call returns hostname as null | Blocker | tools/rumen | Karam Singh | Amar Kamat | +| [MAPREDUCE-3345](https://issues.apache.org/jira/browse/MAPREDUCE-3345) | Race condition in ResourceManager causing TestContainerManagerSecurity to fail sometimes | Major | mrv2, resourcemanager | Vinod Kumar Vavilapalli | Hitesh Shah | +| [MAPREDUCE-3344](https://issues.apache.org/jira/browse/MAPREDUCE-3344) | o.a.h.mapreduce.Reducer since 0.21 blindly casts to ReduceContext.ValueIterator | Major | . | Brock Noland | Brock Noland | +| [MAPREDUCE-3342](https://issues.apache.org/jira/browse/MAPREDUCE-3342) | JobHistoryServer doesn't show job queue | Critical | jobhistoryserver, mrv2 | Thomas Graves | Jonathan Eagles | +| [MAPREDUCE-3339](https://issues.apache.org/jira/browse/MAPREDUCE-3339) | Job is getting hanged indefinitely,if the child processes are killed on the NM. KILL\_CONTAINER eventtype is continuosly sent to the containers that are not existing | Blocker | mrv2 | Ramgopal N | Siddharth Seth | +| [MAPREDUCE-3336](https://issues.apache.org/jira/browse/MAPREDUCE-3336) | com.google.inject.internal.Preconditions not public api - shouldn't be using it | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3333](https://issues.apache.org/jira/browse/MAPREDUCE-3333) | MR AM for sort-job going out of memory | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3329](https://issues.apache.org/jira/browse/MAPREDUCE-3329) | capacity schedule maximum-capacity allowed to be less then capacity | Blocker | mrv2 | Thomas Graves | Arun C Murthy | +| [MAPREDUCE-3328](https://issues.apache.org/jira/browse/MAPREDUCE-3328) | mapred queue -list output inconsistent and missing child queues | Critical | mrv2 | Thomas Graves | Ravi Prakash | +| [MAPREDUCE-3327](https://issues.apache.org/jira/browse/MAPREDUCE-3327) | RM web ui scheduler link doesn't show correct max value for queues | Critical | mrv2 | Thomas Graves | Anupam Seth | +| [MAPREDUCE-3326](https://issues.apache.org/jira/browse/MAPREDUCE-3326) | RM web UI scheduler link not as useful as should be | Critical | mrv2 | Thomas Graves | Jason Lowe | +| [MAPREDUCE-3324](https://issues.apache.org/jira/browse/MAPREDUCE-3324) | Not All HttpServer tools links (stacks,logs,config,metrics) are accessible through all UI servers | Critical | jobhistoryserver, mrv2, nodemanager | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3312](https://issues.apache.org/jira/browse/MAPREDUCE-3312) | Make MR AM not send a stopContainer w/o corresponding start container | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3291](https://issues.apache.org/jira/browse/MAPREDUCE-3291) | App fail to launch due to delegation token not found in cache | Blocker | mrv2 | Ramya Sunil | Robert Joseph Evans | +| [MAPREDUCE-3280](https://issues.apache.org/jira/browse/MAPREDUCE-3280) | MR AM should not read the username from configuration | Major | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3243](https://issues.apache.org/jira/browse/MAPREDUCE-3243) | Invalid tracking URL for streaming jobs | Major | contrib/streaming, mrv2 | Ramya Sunil | Jonathan Eagles | +| [MAPREDUCE-3194](https://issues.apache.org/jira/browse/MAPREDUCE-3194) | "mapred mradmin" command is broken in mrv2 | Major | mrv2 | Siddharth Seth | Jason Lowe | +| [MAPREDUCE-3121](https://issues.apache.org/jira/browse/MAPREDUCE-3121) | DFIP aka 'NodeManager should handle Disk-Failures In Place' | Blocker | mrv2, nodemanager | Vinod Kumar Vavilapalli | Ravi Gummadi | +| [MAPREDUCE-3045](https://issues.apache.org/jira/browse/MAPREDUCE-3045) | Elapsed time filter on jobhistory server displays incorrect table entries | Minor | jobhistoryserver, mrv2 | Ramya Sunil | Jonathan Eagles | +| [MAPREDUCE-2950](https://issues.apache.org/jira/browse/MAPREDUCE-2950) | [Gridmix] TestUserResolve fails in trunk | Major | contrib/gridmix | Amar Kamat | Ravi Gummadi | +| [MAPREDUCE-2784](https://issues.apache.org/jira/browse/MAPREDUCE-2784) | [Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode. | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-2450](https://issues.apache.org/jira/browse/MAPREDUCE-2450) | Calls from running tasks to TaskTracker methods sometimes fail and incur a 60s timeout | Major | . | Matei Zaharia | Rajesh Balamohan | +| [MAPREDUCE-1744](https://issues.apache.org/jira/browse/MAPREDUCE-1744) | DistributedCache creates its own FileSytem instance when adding a file/archive to the path | Major | . | Dick King | Dick King | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-3854](https://issues.apache.org/jira/browse/MAPREDUCE-3854) | Reinstate environment variable tests in TestMiniMRChildTask | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-3803](https://issues.apache.org/jira/browse/MAPREDUCE-3803) | HDFS-2864 broke ant compilation | Major | build | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3595](https://issues.apache.org/jira/browse/MAPREDUCE-3595) | Add missing TestCounters#testCounterValue test from branch 1 to 0.23 | Major | test | Tom White | Tom White | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7590](https://issues.apache.org/jira/browse/HADOOP-7590) | Mavenize streaming and MR examples | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-2879](https://issues.apache.org/jira/browse/HDFS-2879) | Change FSDataset to package private | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2864](https://issues.apache.org/jira/browse/HDFS-2864) | Remove redundant methods and a constant from FSDataset | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2786](https://issues.apache.org/jira/browse/HDFS-2786) | Fix host-based token incompatibilities in DFSUtil | Major | namenode, security | Daryn Sharp | Kihwal Lee | +| [HDFS-2785](https://issues.apache.org/jira/browse/HDFS-2785) | Update webhdfs and httpfs for host-based token support | Major | webhdfs | Daryn Sharp | Robert Joseph Evans | +| [HDFS-2784](https://issues.apache.org/jira/browse/HDFS-2784) | Update hftp and hdfs for host-based token support | Major | hdfs-client, namenode, security | Daryn Sharp | Kihwal Lee | +| [HDFS-2130](https://issues.apache.org/jira/browse/HDFS-2130) | Switch default checksum to CRC32C | Major | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-2129](https://issues.apache.org/jira/browse/HDFS-2129) | Simplify BlockReader to not inherit from FSInputChecker | Major | hdfs-client, performance | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3846](https://issues.apache.org/jira/browse/MAPREDUCE-3846) | Restarted+Recovered AM hangs in some corner cases | Critical | mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3827](https://issues.apache.org/jira/browse/MAPREDUCE-3827) | Counters aggregation slowed down significantly after MAPREDUCE-3749 | Blocker | mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3823](https://issues.apache.org/jira/browse/MAPREDUCE-3823) | Counters are getting calculated twice at job-finish and delaying clients. | Major | mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3815](https://issues.apache.org/jira/browse/MAPREDUCE-3815) | Data Locality suffers if the AM asks for containers using IPs instead of hostnames | Critical | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3813](https://issues.apache.org/jira/browse/MAPREDUCE-3813) | RackResolver should maintain a cache to avoid repetitive lookups. | Major | mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3810](https://issues.apache.org/jira/browse/MAPREDUCE-3810) | MR AM's ContainerAllocator is assigning the allocated containers very slowly | Blocker | mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3809](https://issues.apache.org/jira/browse/MAPREDUCE-3809) | Tasks may take upto 3 seconds to exit after completion | Blocker | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3802](https://issues.apache.org/jira/browse/MAPREDUCE-3802) | If an MR AM dies twice it looks like the process freezes | Critical | applicationmaster, mrv2 | Robert Joseph Evans | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3718](https://issues.apache.org/jira/browse/MAPREDUCE-3718) | Default AM heartbeat interval should be one second | Major | mrv2, performance | Vinod Kumar Vavilapalli | Hitesh Shah | +| [MAPREDUCE-3711](https://issues.apache.org/jira/browse/MAPREDUCE-3711) | AppMaster recovery for Medium to large jobs take long time | Blocker | mrv2 | Siddharth Seth | Robert Joseph Evans | +| [MAPREDUCE-3698](https://issues.apache.org/jira/browse/MAPREDUCE-3698) | Client cannot talk to the history server in secure mode | Blocker | mrv2 | Siddharth Seth | Mahadev konar | +| [MAPREDUCE-3641](https://issues.apache.org/jira/browse/MAPREDUCE-3641) | CapacityScheduler should be more conservative assigning off-switch requests | Blocker | mrv2, scheduler | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3640](https://issues.apache.org/jira/browse/MAPREDUCE-3640) | AMRecovery should pick completed task form partial JobHistory files | Blocker | mrv2 | Siddharth Seth | Arun C Murthy | +| [MAPREDUCE-3618](https://issues.apache.org/jira/browse/MAPREDUCE-3618) | TaskHeartbeatHandler holds a global lock for all task-updates | Major | mrv2, performance | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3616](https://issues.apache.org/jira/browse/MAPREDUCE-3616) | Thread pool for launching containers in MR AM not expanding as expected | Major | mr-am, mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3572](https://issues.apache.org/jira/browse/MAPREDUCE-3572) | MR AM's dispatcher is blocked by heartbeats to ResourceManager | Critical | mr-am, mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3569](https://issues.apache.org/jira/browse/MAPREDUCE-3569) | TaskAttemptListener holds a global lock for all task-updates | Critical | mr-am, mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3568](https://issues.apache.org/jira/browse/MAPREDUCE-3568) | Optimize Job's progress calculations in MR AM | Critical | mr-am, mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3567](https://issues.apache.org/jira/browse/MAPREDUCE-3567) | Extraneous JobConf objects in AM heap | Major | mr-am, mrv2, performance | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3566](https://issues.apache.org/jira/browse/MAPREDUCE-3566) | MR AM slows down due to repeatedly constructing ContainerLaunchContext | Critical | mr-am, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3553](https://issues.apache.org/jira/browse/MAPREDUCE-3553) | Add support for data returned when exceptions thrown from web service apis to be in either xml or in JSON | Minor | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3548](https://issues.apache.org/jira/browse/MAPREDUCE-3548) | write unit tests for web services for mapreduce app master and job history server | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3547](https://issues.apache.org/jira/browse/MAPREDUCE-3547) | finish unit tests for web services for RM and NM | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3534](https://issues.apache.org/jira/browse/MAPREDUCE-3534) | Compression benchmark run-time increased by 13% in 0.23 | Blocker | mrv2 | Vinay Kumar Thota | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3525](https://issues.apache.org/jira/browse/MAPREDUCE-3525) | Shuffle benchmark is nearly 1.5x slower in 0.23 | Blocker | mrv2 | Karam Singh | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3519](https://issues.apache.org/jira/browse/MAPREDUCE-3519) | Deadlock in LocalDirsHandlerService and ShuffleHandler | Blocker | mrv2, nodemanager | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3512](https://issues.apache.org/jira/browse/MAPREDUCE-3512) | Batch jobHistory disk flushes | Blocker | mr-am, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3511](https://issues.apache.org/jira/browse/MAPREDUCE-3511) | Counters occupy a good part of AM heap | Blocker | mr-am, mrv2 | Siddharth Seth | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3485](https://issues.apache.org/jira/browse/MAPREDUCE-3485) | DISKS\_FAILED -101 error code should be defined in same location as ABORTED\_CONTAINER\_EXIT\_STATUS | Major | mrv2 | Hitesh Shah | Ravi Gummadi | +| [MAPREDUCE-3433](https://issues.apache.org/jira/browse/MAPREDUCE-3433) | Finding counters by legacy group name returns empty counters | Major | client, mrv2 | Tom White | Tom White | +| [MAPREDUCE-3426](https://issues.apache.org/jira/browse/MAPREDUCE-3426) | uber-jobs tried to write outputs into wrong dir | Blocker | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3402](https://issues.apache.org/jira/browse/MAPREDUCE-3402) | AMScalability test of Sleep job with 100K 1-sec maps regressed into running very slowly | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3399](https://issues.apache.org/jira/browse/MAPREDUCE-3399) | ContainerLocalizer should request new resources after completing the current one | Blocker | mrv2, nodemanager | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3392](https://issues.apache.org/jira/browse/MAPREDUCE-3392) | Cluster.getDelegationToken() throws NPE if client.getDelegationToken() returns null. | Blocker | . | John George | John George | +| [MAPREDUCE-3380](https://issues.apache.org/jira/browse/MAPREDUCE-3380) | Token infrastructure for running clients which are not kerberos authenticated | Blocker | mr-am, mrv2 | Alejandro Abdelnur | Mahadev konar | +| [MAPREDUCE-3221](https://issues.apache.org/jira/browse/MAPREDUCE-3221) | ant test TestSubmitJob failing on trunk | Minor | mrv2, test | Hitesh Shah | Devaraj K | +| [MAPREDUCE-3219](https://issues.apache.org/jira/browse/MAPREDUCE-3219) | ant test TestDelegationToken failing on trunk | Minor | mrv2, test | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3217](https://issues.apache.org/jira/browse/MAPREDUCE-3217) | ant test TestAuditLogger fails on trunk | Minor | mrv2, test | Hitesh Shah | Devaraj K | +| [MAPREDUCE-3215](https://issues.apache.org/jira/browse/MAPREDUCE-3215) | org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk | Minor | mrv2 | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3102](https://issues.apache.org/jira/browse/MAPREDUCE-3102) | NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor | Major | mrv2, security | Vinod Kumar Vavilapalli | Hitesh Shah | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7877](https://issues.apache.org/jira/browse/HADOOP-7877) | Federation: update Balancer documentation | Major | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2587](https://issues.apache.org/jira/browse/HDFS-2587) | Add WebHDFS apt doc | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2574](https://issues.apache.org/jira/browse/HDFS-2574) | remove references to deprecated properties in hdfs-site.xml template and hdfs-default.xml | Trivial | documentation | Joe Crobak | Joe Crobak | +| [HDFS-2552](https://issues.apache.org/jira/browse/HDFS-2552) | Add WebHdfs Forrest doc | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-3811](https://issues.apache.org/jira/browse/MAPREDUCE-3811) | Make the Client-AM IPC retry count configurable | Critical | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3630](https://issues.apache.org/jira/browse/MAPREDUCE-3630) | NullPointerException running teragen | Critical | mrv2 | Amol Kekre | Mahadev konar | +| [MAPREDUCE-3468](https://issues.apache.org/jira/browse/MAPREDUCE-3468) | Change version to 0.23.1 for ant builds on the 23 branch | Major | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3375](https://issues.apache.org/jira/browse/MAPREDUCE-3375) | Memory Emulation system tests. | Major | . | Vinay Kumar Thota | Vinay Kumar Thota | +| [MAPREDUCE-3297](https://issues.apache.org/jira/browse/MAPREDUCE-3297) | Move Log Related components from yarn-server-nodemanager to yarn-common | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3251](https://issues.apache.org/jira/browse/MAPREDUCE-3251) | Network ACLs can prevent some clients to talk to MR ApplicationMaster | Critical | mrv2 | Anupam Seth | Anupam Seth | +| [MAPREDUCE-2733](https://issues.apache.org/jira/browse/MAPREDUCE-2733) | Gridmix v3 cpu emulation system tests. | Major | . | Vinay Kumar Thota | Vinay Kumar Thota | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/RELEASENOTES.0.23.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/RELEASENOTES.0.23.1.md new file mode 100644 index 00000000000..08c9df02def --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.1/RELEASENOTES.0.23.1.md @@ -0,0 +1,677 @@ + + +# Apache Hadoop 0.23.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8013](https://issues.apache.org/jira/browse/HADOOP-8013) | *Major* | **ViewFileSystem does not honor setVerifyChecksum** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8009](https://issues.apache.org/jira/browse/HADOOP-8009) | *Critical* | **Create hadoop-client and hadoop-minicluster artifacts for downstream projects** + +Generate integration artifacts "org.apache.hadoop:hadoop-client" and "org.apache.hadoop:hadoop-minicluster" containing all the jars needed to use Hadoop client APIs, and to run Hadoop MiniClusters, respectively. Push these artifacts to the maven repository when mvn-deploy, along with existing artifacts. + + +--- + +* [HADOOP-7986](https://issues.apache.org/jira/browse/HADOOP-7986) | *Major* | **Add config for History Server protocol in hadoop-policy for service level authorization.** + +Adding config for MapReduce History Server protocol in hadoop-policy.xml for service level authorization. + + +--- + +* [HADOOP-7963](https://issues.apache.org/jira/browse/HADOOP-7963) | *Blocker* | **test failures: TestViewFileSystemWithAuthorityLocalFileSystem and TestViewFileSystemLocalFileSystem** + +Fix ViewFS to catch a null canonical service-name and pass tests TestViewFileSystem* + + +--- + +* [HADOOP-7851](https://issues.apache.org/jira/browse/HADOOP-7851) | *Major* | **Configuration.getClasses() never returns the default value.** + +Fixed Configuration.getClasses() API to return the default value if the key is not set. + + +--- + +* [HADOOP-7802](https://issues.apache.org/jira/browse/HADOOP-7802) | *Major* | **Hadoop scripts unconditionally source "$bin"/../libexec/hadoop-config.sh.** + +Here is a patch to enable this behavior + + +--- + +* [HADOOP-7470](https://issues.apache.org/jira/browse/HADOOP-7470) | *Minor* | **move up to Jackson 1.8.8** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-7348](https://issues.apache.org/jira/browse/HADOOP-7348) | *Major* | **Modify the option of FsShell getmerge from [addnl] to [-nl] for consistency** + +The 'fs -getmerge' tool now uses a -nl flag to determine if adding a newline at end of each file is required, in favor of the 'addnl' boolean flag that was used earlier. + + +--- + +* [HDFS-2316](https://issues.apache.org/jira/browse/HDFS-2316) | *Major* | **[umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP** + +Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP. +Previous hftp feature was a read-only FileSystem and does not provide "write" accesses. + + +--- + +* [HDFS-2246](https://issues.apache.org/jira/browse/HDFS-2246) | *Major* | **Shortcut a local client reads to a Datanodes files directly** + +1. New configurations +a. dfs.block.local-path-access.user is the key in datanode configuration to specify the user allowed to do short circuit read. +b. dfs.client.read.shortcircuit is the key to enable short circuit read at the client side configuration. +c. dfs.client.read.shortcircuit.skip.checksum is the key to bypass checksum check at the client side. +2. By default none of the above are enabled and short circuit read will not kick in. +3. If security is on, the feature can be used only for user that has kerberos credentials at the client, therefore map reduce tasks cannot benefit from it in general. + + +--- + +* [HDFS-2130](https://issues.apache.org/jira/browse/HDFS-2130) | *Major* | **Switch default checksum to CRC32C** + +The default checksum algorithm used on HDFS is now CRC32C. Data from previous versions of Hadoop can still be read backwards-compatibly. + + +--- + +* [HDFS-2129](https://issues.apache.org/jira/browse/HDFS-2129) | *Major* | **Simplify BlockReader to not inherit from FSInputChecker** + +BlockReader has been reimplemented to use direct byte buffers. If you use a custom socket factory, it must generate sockets that have associated Channels. + + +--- + +* [HDFS-1314](https://issues.apache.org/jira/browse/HDFS-1314) | *Minor* | **dfs.blocksize accepts only absolute value** + +The default blocksize property 'dfs.blocksize' now accepts unit symbols to be used instead of byte length. Values such as "10k", "128m", "1g" are now OK to provide instead of just no. of bytes as was before. + + +--- + +* [MAPREDUCE-3854](https://issues.apache.org/jira/browse/MAPREDUCE-3854) | *Major* | **Reinstate environment variable tests in TestMiniMRChildTask** + +Fixed and reenabled tests related to MR child JVM's environmental variables in TestMiniMRChildTask. + + +--- + +* [MAPREDUCE-3846](https://issues.apache.org/jira/browse/MAPREDUCE-3846) | *Critical* | **Restarted+Recovered AM hangs in some corner cases** + +Addressed MR AM hanging issues during AM restart and then the recovery. + + +--- + +* [MAPREDUCE-3834](https://issues.apache.org/jira/browse/MAPREDUCE-3834) | *Critical* | **If multiple hosts for a split belong to the same rack, the rack is added multiple times in the AM request table** + +Changed MR AM to not add the same rack entry multiple times into the container request table when multiple hosts for a split happen to be on the same rack + + +--- + +* [MAPREDUCE-3815](https://issues.apache.org/jira/browse/MAPREDUCE-3815) | *Critical* | **Data Locality suffers if the AM asks for containers using IPs instead of hostnames** + +Fixed MR AM to always use hostnames and never IPs when requesting containers so that scheduler can give off data local containers correctly. + + +--- + +* [MAPREDUCE-3808](https://issues.apache.org/jira/browse/MAPREDUCE-3808) | *Blocker* | **NPE in FileOutputCommitter when running a 0 reduce job** + +Fixed an NPE in FileOutputCommitter for jobs with maps but no reduces. + + +--- + +* [MAPREDUCE-3804](https://issues.apache.org/jira/browse/MAPREDUCE-3804) | *Major* | **yarn webapp interface vulnerable to cross scripting attacks** + +fix cross scripting attacks vulnerability through webapp interface. + + +--- + +* [MAPREDUCE-3802](https://issues.apache.org/jira/browse/MAPREDUCE-3802) | *Critical* | **If an MR AM dies twice it looks like the process freezes** + +Added test to validate that AM can crash multiple times and still can recover successfully after MAPREDUCE-3846. + + +--- + +* [MAPREDUCE-3784](https://issues.apache.org/jira/browse/MAPREDUCE-3784) | *Major* | **maxActiveApplications(\|PerUser) per queue is too low for small clusters** + +Fixed CapacityScheduler so that maxActiveApplication and maxActiveApplicationsPerUser per queue are not too low for small clusters. + + +--- + +* [MAPREDUCE-3774](https://issues.apache.org/jira/browse/MAPREDUCE-3774) | *Major* | **yarn-default.xml should be moved to hadoop-yarn-common.** + +MAPREDUCE-3774. Moved yarn-default.xml to hadoop-yarn-common from hadoop-server-common. + + +--- + +* [MAPREDUCE-3760](https://issues.apache.org/jira/browse/MAPREDUCE-3760) | *Major* | **Blacklisted NMs should not appear in Active nodes list** + +Changed active nodes list to not contain unhealthy nodes on the webUI and metrics. + + +--- + +* [MAPREDUCE-3754](https://issues.apache.org/jira/browse/MAPREDUCE-3754) | *Major* | **RM webapp should have pages filtered based on App-state** + +Modified RM UI to filter applications based on state of the applications. + + +--- + +* [MAPREDUCE-3752](https://issues.apache.org/jira/browse/MAPREDUCE-3752) | *Blocker* | **Headroom should be capped by queue max-cap** + +Modified application limits to include queue max-capacities besides the usual user limits. + + +--- + +* [MAPREDUCE-3732](https://issues.apache.org/jira/browse/MAPREDUCE-3732) | *Blocker* | **CS should only use 'activeUsers with pending requests' for computing user-limits** + +Modified CapacityScheduler to use only users with pending requests for computing user-limits. + + +--- + +* [MAPREDUCE-3720](https://issues.apache.org/jira/browse/MAPREDUCE-3720) | *Major* | **Command line listJobs should not visit each AM** + +Changed bin/mapred job -list to not print job-specific information not available at RM. + +Very minor incompatibility in cmd-line output, inevitable due to MRv2 architecture. + + +--- + +* [MAPREDUCE-3716](https://issues.apache.org/jira/browse/MAPREDUCE-3716) | *Blocker* | **java.io.File.createTempFile fails in map/reduce tasks** + +Fixing YARN+MR to allow MR jobs to be able to use java.io.File.createTempFile to create temporary files as part of their tasks. + + +--- + +* [MAPREDUCE-3714](https://issues.apache.org/jira/browse/MAPREDUCE-3714) | *Blocker* | **Reduce hangs in a corner case** + +Fixed EventFetcher and Fetcher threads to shut-down properly so that reducers don't hang in corner cases. + + +--- + +* [MAPREDUCE-3713](https://issues.apache.org/jira/browse/MAPREDUCE-3713) | *Blocker* | **Incorrect headroom reported to jobs** + +Fixed the way head-room is allocated to applications by CapacityScheduler so that it deducts current-usage per user and not per-application. + + +--- + +* [MAPREDUCE-3711](https://issues.apache.org/jira/browse/MAPREDUCE-3711) | *Blocker* | **AppMaster recovery for Medium to large jobs take long time** + +Fixed MR AM recovery so that only single selected task output is recovered and thus reduce the unnecessarily bloated recovery time. + + +--- + +* [MAPREDUCE-3710](https://issues.apache.org/jira/browse/MAPREDUCE-3710) | *Major* | **last split generated by FileInputFormat.getSplits may not have the best locality** + +Improved FileInputFormat to return better locality for the last split. + + +--- + +* [MAPREDUCE-3703](https://issues.apache.org/jira/browse/MAPREDUCE-3703) | *Critical* | **ResourceManager should provide node lists in JMX output** + +New JMX Bean in ResourceManager to provide list of live node managers: + +Hadoop:service=ResourceManager,name=RMNMInfo LiveNodeManagers + + +--- + +* [MAPREDUCE-3699](https://issues.apache.org/jira/browse/MAPREDUCE-3699) | *Major* | **Default RPC handlers are very low for YARN servers** + +Increased RPC handlers for all YARN servers to reasonable values for working at scale. + + +--- + +* [MAPREDUCE-3656](https://issues.apache.org/jira/browse/MAPREDUCE-3656) | *Blocker* | **Sort job on 350 scale is consistently failing with latest MRV2 code** + +Fixed a race condition in MR AM which is failing the sort benchmark consistently. + + +--- + +* [MAPREDUCE-3641](https://issues.apache.org/jira/browse/MAPREDUCE-3641) | *Blocker* | **CapacityScheduler should be more conservative assigning off-switch requests** + +Making CapacityScheduler more conservative so as to assign only one off-switch container in a single scheduling iteration. + + +--- + +* [MAPREDUCE-3639](https://issues.apache.org/jira/browse/MAPREDUCE-3639) | *Blocker* | **TokenCache likely broken for FileSystems which don't issue delegation tokens** + +Fixed TokenCache to work with absent FileSystem canonical service-names. + + +--- + +* [MAPREDUCE-3630](https://issues.apache.org/jira/browse/MAPREDUCE-3630) | *Critical* | **NullPointerException running teragen** + +Committed to trunk and branch-0.23. Thanks Mahadev. + + +--- + +* [MAPREDUCE-3618](https://issues.apache.org/jira/browse/MAPREDUCE-3618) | *Major* | **TaskHeartbeatHandler holds a global lock for all task-updates** + +Fixed TaskHeartbeatHandler to not hold a global lock for all task-updates. + + +--- + +* [MAPREDUCE-3597](https://issues.apache.org/jira/browse/MAPREDUCE-3597) | *Major* | **Provide a way to access other info of history file from Rumentool** + +Rumen now provides {{Parsed*}} objects. These objects provide extra information that are not provided by {{Logged*}} objects. + + +--- + +* [MAPREDUCE-3586](https://issues.apache.org/jira/browse/MAPREDUCE-3586) | *Blocker* | **Lots of AMs hanging around in PIG testing** + +Modified CompositeService to avoid duplicate stop operations thereby solving race conditions in MR AM shutdown. + + +--- + +* [MAPREDUCE-3568](https://issues.apache.org/jira/browse/MAPREDUCE-3568) | *Critical* | **Optimize Job's progress calculations in MR AM** + +Optimized Job's progress calculations in MR AM. + + +--- + +* [MAPREDUCE-3564](https://issues.apache.org/jira/browse/MAPREDUCE-3564) | *Blocker* | **TestStagingCleanup and TestJobEndNotifier are failing on trunk.** + +Fixed failures in TestStagingCleanup and TestJobEndNotifier tests. + + +--- + +* [MAPREDUCE-3549](https://issues.apache.org/jira/browse/MAPREDUCE-3549) | *Blocker* | **write api documentation for web service apis for RM, NM, mapreduce app master, and job history server** + +new files added: A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/WebServicesIntro.apt.vm +A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/NodeManagerRest.apt.vm +A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/ResourceManagerRest.apt.vm +A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/MapredAppMasterRest.apt.vm +A hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/HistoryServerRest.apt.vm + +The hadoop-project/src/site/site.xml is split into separate patch. + + +--- + +* [MAPREDUCE-3532](https://issues.apache.org/jira/browse/MAPREDUCE-3532) | *Critical* | **When 0 is provided as port number in yarn.nodemanager.webapp.address, NMs webserver component picks up random port, NM keeps on Reporting 0 port to RM** + +Modified NM to report correct http address when an ephemeral web port is configured. + + +--- + +* [MAPREDUCE-3530](https://issues.apache.org/jira/browse/MAPREDUCE-3530) | *Blocker* | **Sometimes NODE\_UPDATE to the scheduler throws an NPE causing the scheduling to stop** + +Fixed an NPE occuring during scheduling in the ResourceManager. + + +--- + +* [MAPREDUCE-3528](https://issues.apache.org/jira/browse/MAPREDUCE-3528) | *Major* | **The task timeout check interval should be configurable independent of mapreduce.task.timeout** + +Fixed TaskHeartBeatHandler to use a new configuration for the thread loop interval separate from task-timeout configuration property. + + +--- + +* [MAPREDUCE-3519](https://issues.apache.org/jira/browse/MAPREDUCE-3519) | *Blocker* | **Deadlock in LocalDirsHandlerService and ShuffleHandler** + +Fixed a deadlock in NodeManager LocalDirectories's handling service. + + +--- + +* [MAPREDUCE-3512](https://issues.apache.org/jira/browse/MAPREDUCE-3512) | *Blocker* | **Batch jobHistory disk flushes** + +Batching JobHistory flushing to DFS so that we don't flush for every event slowing down AM. + + +--- + +* [MAPREDUCE-3511](https://issues.apache.org/jira/browse/MAPREDUCE-3511) | *Blocker* | **Counters occupy a good part of AM heap** + +Removed a multitude of cloned/duplicate counters in the AM thereby reducing the AM heap size and preventing full GCs. + + +--- + +* [MAPREDUCE-3490](https://issues.apache.org/jira/browse/MAPREDUCE-3490) | *Blocker* | **RMContainerAllocator counts failed maps towards Reduce ramp up** + +Fixed MapReduce AM to count failed maps also towards Reduce ramp up. + + +--- + +* [MAPREDUCE-3487](https://issues.apache.org/jira/browse/MAPREDUCE-3487) | *Critical* | **jobhistory web ui task counters no longer links to singletakecounter page** + +Fixed JobHistory web-UI to display links to single task's counters' page. + + +--- + +* [MAPREDUCE-3484](https://issues.apache.org/jira/browse/MAPREDUCE-3484) | *Major* | **JobEndNotifier is getting interrupted before completing all its retries.** + +Fixed JobEndNotifier to not get interrupted before completing all its retries. + + +--- + +* [MAPREDUCE-3481](https://issues.apache.org/jira/browse/MAPREDUCE-3481) | *Major* | **[Gridmix] Improve STRESS mode locking** + +Modified Gridmix STRESS mode locking structure. The submitted thread and the polling thread now run simultaneously without blocking each other. + + +--- + +* [MAPREDUCE-3462](https://issues.apache.org/jira/browse/MAPREDUCE-3462) | *Blocker* | **Job submission failing in JUnit tests** + +Fixed failing JUnit tests in Gridmix. + + +--- + +* [MAPREDUCE-3426](https://issues.apache.org/jira/browse/MAPREDUCE-3426) | *Blocker* | **uber-jobs tried to write outputs into wrong dir** + +Fixed MR AM in uber mode to write map intermediate outputs in the correct directory to work properly in secure mode. + + +--- + +* [MAPREDUCE-3417](https://issues.apache.org/jira/browse/MAPREDUCE-3417) | *Blocker* | **job access controls not working app master and job history UI's** + +Fixed job-access-controls to work with MR AM and JobHistoryServer web-apps. + + +--- + +* [MAPREDUCE-3412](https://issues.apache.org/jira/browse/MAPREDUCE-3412) | *Major* | **'ant docs' is broken** + +Fixes 'ant docs' by removing stale references to capacity-scheduler docs. + + +--- + +* [MAPREDUCE-3407](https://issues.apache.org/jira/browse/MAPREDUCE-3407) | *Minor* | **Wrong jar getting used in TestMR\*Jobs\* for MiniMRYarnCluster** + +Fixed pom files to refer to the correct MR app-jar needed by the integration tests. + + +--- + +* [MAPREDUCE-3404](https://issues.apache.org/jira/browse/MAPREDUCE-3404) | *Critical* | **Speculative Execution: speculative map tasks launched even if -Dmapreduce.map.speculative=false** + +Corrected MR AM to honor speculative configuration and enable speculating either maps or reduces. + + +--- + +* [MAPREDUCE-3399](https://issues.apache.org/jira/browse/MAPREDUCE-3399) | *Blocker* | **ContainerLocalizer should request new resources after completing the current one** + +Modified ContainerLocalizer to send a heartbeat to NM immediately after downloading a resource instead of always waiting for a second. + + +--- + +* [MAPREDUCE-3398](https://issues.apache.org/jira/browse/MAPREDUCE-3398) | *Blocker* | **Log Aggregation broken in Secure Mode** + +Fixed log aggregation to work correctly in secure mode. Contributed by Siddharth Seth. + + +--- + +* [MAPREDUCE-3392](https://issues.apache.org/jira/browse/MAPREDUCE-3392) | *Blocker* | **Cluster.getDelegationToken() throws NPE if client.getDelegationToken() returns null.** + +Fixed Cluster's getDelegationToken's API to return null when there isn't a supported token. + + +--- + +* [MAPREDUCE-3387](https://issues.apache.org/jira/browse/MAPREDUCE-3387) | *Critical* | **A tracking URL of N/A before the app master is launched breaks oozie** + +Fixed AM's tracking URL to always go through the proxy, even before the job started, so that it works properly with oozie throughout the job execution. + + +--- + +* [MAPREDUCE-3382](https://issues.apache.org/jira/browse/MAPREDUCE-3382) | *Critical* | **Network ACLs can prevent AMs to ping the Job-end notification URL** + +Enhanced MR AM to use a proxy to ping the job-end notification URL. + + +--- + +* [MAPREDUCE-3379](https://issues.apache.org/jira/browse/MAPREDUCE-3379) | *Major* | **LocalResourceTracker should not tracking deleted cache entries** + +Fixed LocalResourceTracker in NodeManager to remove deleted cache entries correctly. + + +--- + +* [MAPREDUCE-3375](https://issues.apache.org/jira/browse/MAPREDUCE-3375) | *Major* | **Memory Emulation system tests.** + +Added system tests to test the memory emulation feature in Gridmix. + + +--- + +* [MAPREDUCE-3368](https://issues.apache.org/jira/browse/MAPREDUCE-3368) | *Critical* | **compile-mapred-test fails** + +Fixed ant test compilation. + + +--- + +* [MAPREDUCE-3360](https://issues.apache.org/jira/browse/MAPREDUCE-3360) | *Critical* | **Provide information about lost nodes in the UI.** + +Added information about lost/rebooted/decommissioned nodes on the webapps. + + +--- + +* [MAPREDUCE-3355](https://issues.apache.org/jira/browse/MAPREDUCE-3355) | *Blocker* | **AM scheduling hangs frequently with sort job on 350 nodes** + +Fixed MR AM's ContainerLauncher to handle node-command timeouts correctly. + + +--- + +* [MAPREDUCE-3349](https://issues.apache.org/jira/browse/MAPREDUCE-3349) | *Blocker* | **No rack-name logged in JobHistory for unsuccessful tasks** + +Unsuccessful tasks now log hostname and rackname to job history. + + +--- + +* [MAPREDUCE-3345](https://issues.apache.org/jira/browse/MAPREDUCE-3345) | *Major* | **Race condition in ResourceManager causing TestContainerManagerSecurity to fail sometimes** + +Fixed a race condition in ResourceManager that was causing TestContainerManagerSecurity to fail sometimes. + + +--- + +* [MAPREDUCE-3342](https://issues.apache.org/jira/browse/MAPREDUCE-3342) | *Critical* | **JobHistoryServer doesn't show job queue** + +Fixed JobHistoryServer to also show the job's queue name. + + +--- + +* [MAPREDUCE-3339](https://issues.apache.org/jira/browse/MAPREDUCE-3339) | *Blocker* | **Job is getting hanged indefinitely,if the child processes are killed on the NM. KILL\_CONTAINER eventtype is continuosly sent to the containers that are not existing** + +Fixed MR AM to stop considering node blacklisting after the number of nodes blacklisted crosses a threshold. + + +--- + +* [MAPREDUCE-3333](https://issues.apache.org/jira/browse/MAPREDUCE-3333) | *Blocker* | **MR AM for sort-job going out of memory** + +Fixed bugs in ContainerLauncher of MR AppMaster due to which per-container connections to NodeManager were lingering long enough to hit the ulimits on number of processes. + + +--- + +* [MAPREDUCE-3325](https://issues.apache.org/jira/browse/MAPREDUCE-3325) | *Major* | **Improvements to CapacityScheduler doc** + +document changes only. + + +--- + +* [MAPREDUCE-3312](https://issues.apache.org/jira/browse/MAPREDUCE-3312) | *Major* | **Make MR AM not send a stopContainer w/o corresponding start container** + +Modified MR AM to not send a stop-container request for a container that isn't launched at all. + + +--- + +* [MAPREDUCE-3299](https://issues.apache.org/jira/browse/MAPREDUCE-3299) | *Minor* | **Add AMInfo table to the AM job page** + +Added AMInfo table to the MR AM job pages to list all the job-attempts when AM restarts and recovers. + + +--- + +* [MAPREDUCE-3297](https://issues.apache.org/jira/browse/MAPREDUCE-3297) | *Major* | **Move Log Related components from yarn-server-nodemanager to yarn-common** + +Moved log related components into yarn-common so that HistoryServer and clients can use them without depending on the yarn-server-nodemanager module. + + +--- + +* [MAPREDUCE-3280](https://issues.apache.org/jira/browse/MAPREDUCE-3280) | *Major* | **MR AM should not read the username from configuration** + +Removed the unnecessary job user-name configuration in mapred-site.xml. + + +--- + +* [MAPREDUCE-3221](https://issues.apache.org/jira/browse/MAPREDUCE-3221) | *Minor* | **ant test TestSubmitJob failing on trunk** + +Fixed a bug in TestSubmitJob. + + +--- + +* [MAPREDUCE-3219](https://issues.apache.org/jira/browse/MAPREDUCE-3219) | *Minor* | **ant test TestDelegationToken failing on trunk** + +Reenabled and fixed bugs in the failing test TestDelegationToken. + + +--- + +* [MAPREDUCE-3217](https://issues.apache.org/jira/browse/MAPREDUCE-3217) | *Minor* | **ant test TestAuditLogger fails on trunk** + +Reenabled and fixed bugs in the failing ant test TestAuditLogger. + + +--- + +* [MAPREDUCE-3215](https://issues.apache.org/jira/browse/MAPREDUCE-3215) | *Minor* | **org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk** + +Reneabled and fixed bugs in the failing test TestNoJobSetupCleanup. + + +--- + +* [MAPREDUCE-3102](https://issues.apache.org/jira/browse/MAPREDUCE-3102) | *Major* | **NodeManager should fail fast with wrong configuration or permissions for LinuxContainerExecutor** + +Changed NodeManager to fail fast when LinuxContainerExecutor has wrong configuration or permissions. + + +--- + +* [MAPREDUCE-2950](https://issues.apache.org/jira/browse/MAPREDUCE-2950) | *Major* | **[Gridmix] TestUserResolve fails in trunk** + +Fixes bug in TestUserResolve. + + +--- + +* [MAPREDUCE-2863](https://issues.apache.org/jira/browse/MAPREDUCE-2863) | *Blocker* | **Support web-services for RM & NM** + +Support for web-services in YARN and MR components. + + +--- + +* [MAPREDUCE-2784](https://issues.apache.org/jira/browse/MAPREDUCE-2784) | *Major* | **[Gridmix] TestGridmixSummary fails with NPE when run in DEBUG mode.** + +Fixed bugs in ExecutionSummarizer and ResourceUsageMatcher. + + +--- + +* [MAPREDUCE-2765](https://issues.apache.org/jira/browse/MAPREDUCE-2765) | *Major* | **DistCp Rewrite** + +DistCpV2 added to hadoop-tools. + + +--- + +* [MAPREDUCE-2733](https://issues.apache.org/jira/browse/MAPREDUCE-2733) | *Major* | **Gridmix v3 cpu emulation system tests.** + +Adds system tests for the CPU emulation feature in Gridmix3. + + +--- + +* [MAPREDUCE-778](https://issues.apache.org/jira/browse/MAPREDUCE-778) | *Major* | **[Rumen] Need a standalone JobHistory log anonymizer** + +Added an anonymizer tool to Rumen. Anonymizer takes a Rumen trace file and/or topology as input. It supports persistence and plugins to override the default behavior. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/CHANGES.0.23.10.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/CHANGES.0.23.10.md new file mode 100644 index 00000000000..82bef03ce6e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/CHANGES.0.23.10.md @@ -0,0 +1,132 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.10 - 2013-12-09 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-707](https://issues.apache.org/jira/browse/YARN-707) | Add user info in the YARN ClientToken | Blocker | . | Bikas Saha | Jason Lowe | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9686](https://issues.apache.org/jira/browse/HADOOP-9686) | Easy access to final parameters in Configuration | Major | conf | Jason Lowe | Jason Lowe | +| [HADOOP-8704](https://issues.apache.org/jira/browse/HADOOP-8704) | add request logging to jetty/httpserver | Major | . | Thomas Graves | Jonathan Eagles | +| [HDFS-5010](https://issues.apache.org/jira/browse/HDFS-5010) | Reduce the frequency of getCurrentUser() calls from namenode | Major | namenode, performance | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-5640](https://issues.apache.org/jira/browse/MAPREDUCE-5640) | Rename TestLineRecordReader in jobclient module | Trivial | test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-1981](https://issues.apache.org/jira/browse/MAPREDUCE-1981) | Improve getSplits performance by using listLocatedStatus | Major | job submission | Hairong Kuang | Hairong Kuang | +| [YARN-985](https://issues.apache.org/jira/browse/YARN-985) | Nodemanager should log where a resource was localized | Major | nodemanager | Ravi Prakash | Ravi Prakash | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10091](https://issues.apache.org/jira/browse/HADOOP-10091) | Job with a har archive as input fails on 0.23 | Blocker | fs | Jason Lowe | Jason Lowe | +| [HADOOP-9757](https://issues.apache.org/jira/browse/HADOOP-9757) | Har metadata cache can grow without limit | Major | fs | Jason Lowe | Cristina L. Abad | +| [HADOOP-9476](https://issues.apache.org/jira/browse/HADOOP-9476) | Some test cases in TestUserGroupInformation fail if ran after testSetLoginUser. | Major | security, test | Kihwal Lee | Robert Parker | +| [HDFS-5558](https://issues.apache.org/jira/browse/HDFS-5558) | LeaseManager monitor thread can crash if the last block is complete but another block is not. | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5557](https://issues.apache.org/jira/browse/HDFS-5557) | Write pipeline recovery for the last packet in the block may cause rejection of valid replicas | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5526](https://issues.apache.org/jira/browse/HDFS-5526) | Datanode cannot roll back to previous layout version | Blocker | datanode | Tsz Wo Nicholas Sze | Kihwal Lee | +| [HDFS-5438](https://issues.apache.org/jira/browse/HDFS-5438) | Flaws in block report processing can cause data loss | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-5346](https://issues.apache.org/jira/browse/HDFS-5346) | Avoid unnecessary call to getNumLiveDataNodes() for each block during IBR processing | Major | namenode, performance | Kihwal Lee | Ravi Prakash | +| [HDFS-5047](https://issues.apache.org/jira/browse/HDFS-5047) | Supress logging of full stack trace of quota and lease exceptions | Major | namenode | Kihwal Lee | Robert Parker | +| [HDFS-5028](https://issues.apache.org/jira/browse/HDFS-5028) | LeaseRenewer throw java.util.ConcurrentModificationException when timeout | Major | . | zhaoyunjiong | zhaoyunjiong | +| [HDFS-4998](https://issues.apache.org/jira/browse/HDFS-4998) | TestUnderReplicatedBlocks fails intermittently | Major | test | Kihwal Lee | Kihwal Lee | +| [HDFS-4995](https://issues.apache.org/jira/browse/HDFS-4995) | Make getContentSummary() less expensive | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4994](https://issues.apache.org/jira/browse/HDFS-4994) | Audit log getContentSummary() calls | Minor | namenode | Kihwal Lee | Robert Parker | +| [HDFS-4993](https://issues.apache.org/jira/browse/HDFS-4993) | fsck can fail if a file is renamed or deleted | Major | . | Kihwal Lee | Robert Parker | +| [HDFS-4984](https://issues.apache.org/jira/browse/HDFS-4984) | Incorrect Quota counting in INodeFile | Major | namenode | Kihwal Lee | Jing Zhao | +| [HDFS-4972](https://issues.apache.org/jira/browse/HDFS-4972) | [branch-0.23] permission check and operation are done in a separate lock for getBlockLocations() | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4943](https://issues.apache.org/jira/browse/HDFS-4943) | WebHdfsFileSystem does not work when original file path has encoded chars | Minor | webhdfs | Jerry He | Jerry He | +| [HDFS-4482](https://issues.apache.org/jira/browse/HDFS-4482) | ReplicationMonitor thread can exit with NPE due to the race between delete and replication of same file. | Blocker | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-4329](https://issues.apache.org/jira/browse/HDFS-4329) | DFSShell issues with directories with spaces in name | Major | hdfs-client | Andy Isaacson | Cristina L. Abad | +| [HDFS-3970](https://issues.apache.org/jira/browse/HDFS-3970) | BlockPoolSliceStorage#doRollback(..) should use BlockPoolSliceStorage instead of DataStorage to read prev version file. | Major | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-3020](https://issues.apache.org/jira/browse/HDFS-3020) | Auto-logSync based on edit log buffer size broken | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-5587](https://issues.apache.org/jira/browse/MAPREDUCE-5587) | TestTextOutputFormat fails on JDK7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5586](https://issues.apache.org/jira/browse/MAPREDUCE-5586) | TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5585](https://issues.apache.org/jira/browse/MAPREDUCE-5585) | TestCopyCommitter#testNoCommitAction Fails on JDK7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5569](https://issues.apache.org/jira/browse/MAPREDUCE-5569) | FloatSplitter is not generating correct splits | Major | . | Nathan Roberts | Nathan Roberts | +| [MAPREDUCE-5561](https://issues.apache.org/jira/browse/MAPREDUCE-5561) | org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk | Critical | . | Cindy Li | Karthik Kambatla | +| [MAPREDUCE-5543](https://issues.apache.org/jira/browse/MAPREDUCE-5543) | In-memory map outputs can be leaked after shuffle completes in 0.23 | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5513](https://issues.apache.org/jira/browse/MAPREDUCE-5513) | ConcurrentModificationException in JobControl | Major | . | Jason Lowe | Robert Parker | +| [MAPREDUCE-5504](https://issues.apache.org/jira/browse/MAPREDUCE-5504) | mapred queue -info inconsistent with types | Major | client | Thomas Graves | Kousuke Saruta | +| [MAPREDUCE-5475](https://issues.apache.org/jira/browse/MAPREDUCE-5475) | MRClientService does not verify ACLs properly | Blocker | mr-am, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5468](https://issues.apache.org/jira/browse/MAPREDUCE-5468) | AM recovery does not work for map only jobs | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5440](https://issues.apache.org/jira/browse/MAPREDUCE-5440) | TestCopyCommitter Fails on JDK7 | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5425](https://issues.apache.org/jira/browse/MAPREDUCE-5425) | Junit in TestJobHistoryServer failing in jdk 7 | Major | jobhistoryserver | Ashwin Shankar | Robert Parker | +| [MAPREDUCE-5419](https://issues.apache.org/jira/browse/MAPREDUCE-5419) | TestSlive is getting FileNotFound Exception | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5414](https://issues.apache.org/jira/browse/MAPREDUCE-5414) | TestTaskAttempt fails jdk7 with NullPointerException | Major | test | Nemon Lou | Nemon Lou | +| [MAPREDUCE-5380](https://issues.apache.org/jira/browse/MAPREDUCE-5380) | Invalid mapred command should return non-zero exit code | Major | . | Stephen Chu | Stephen Chu | +| [MAPREDUCE-5373](https://issues.apache.org/jira/browse/MAPREDUCE-5373) | TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently | Major | . | Chuan Liu | Jonathan Eagles | +| [MAPREDUCE-5317](https://issues.apache.org/jira/browse/MAPREDUCE-5317) | Stale files left behind for failed jobs | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-5251](https://issues.apache.org/jira/browse/MAPREDUCE-5251) | Reducer should not implicate map attempt if it has insufficient space to fetch map output | Major | mrv2 | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-5001](https://issues.apache.org/jira/browse/MAPREDUCE-5001) | LocalJobRunner has race condition resulting in job failures | Major | . | Brock Noland | Sandy Ryza | +| [MAPREDUCE-4716](https://issues.apache.org/jira/browse/MAPREDUCE-4716) | TestHsWebServicesJobsQuery.testJobsQueryStateInvalid fails with jdk7 | Major | jobhistoryserver | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4579](https://issues.apache.org/jira/browse/MAPREDUCE-4579) | TestTaskAttempt fails jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4571](https://issues.apache.org/jira/browse/MAPREDUCE-4571) | TestHsWebServicesJobs fails on jdk7 | Major | webapps | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3193](https://issues.apache.org/jira/browse/MAPREDUCE-3193) | FileInputFormat doesn't read files recursively in the input path dir | Major | mrv1, mrv2 | Ramgopal N | Devaraj K | +| [YARN-1419](https://issues.apache.org/jira/browse/YARN-1419) | TestFifoScheduler.testAppAttemptMetrics fails intermittently under jdk7 | Minor | scheduler | Jonathan Eagles | Jonathan Eagles | +| [YARN-1386](https://issues.apache.org/jira/browse/YARN-1386) | NodeManager mistakenly loses resources and relocalizes them | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1243](https://issues.apache.org/jira/browse/YARN-1243) | ResourceManager: Error in handling event type NODE\_UPDATE to the scheduler - NPE at SchedulerApp.java:411 | Major | capacityscheduler | Sanjay Upadhyay | Jason Lowe | +| [YARN-1176](https://issues.apache.org/jira/browse/YARN-1176) | RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes | Critical | resourcemanager | Thomas Graves | Jonathan Eagles | +| [YARN-1101](https://issues.apache.org/jira/browse/YARN-1101) | Active nodes can be decremented below 0 | Major | resourcemanager | Robert Parker | Robert Parker | +| [YARN-1036](https://issues.apache.org/jira/browse/YARN-1036) | Distributed Cache gives inconsistent result if cache files get deleted from task tracker | Major | nodemanager | Ravi Prakash | Ravi Prakash | +| [YARN-1031](https://issues.apache.org/jira/browse/YARN-1031) | JQuery UI components reference external css in branch-23 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [YARN-949](https://issues.apache.org/jira/browse/YARN-949) | Failed log aggregation can leave a file open. | Major | nodemanager | Kihwal Lee | Kihwal Lee | +| [YARN-337](https://issues.apache.org/jira/browse/YARN-337) | RM handles killed application tracking URL poorly | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-155](https://issues.apache.org/jira/browse/YARN-155) | TestAppManager intermittently fails with jdk7 | Major | . | Thomas Graves | Thomas Graves | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5610](https://issues.apache.org/jira/browse/MAPREDUCE-5610) | TestSleepJob fails in jdk7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [YARN-1119](https://issues.apache.org/jira/browse/YARN-1119) | Add ClusterMetrics checks to tho TestRMNodeTransitions tests | Major | resourcemanager | Robert Parker | Mit Desai | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9956](https://issues.apache.org/jira/browse/HADOOP-9956) | RPC listener inefficiently assigns connections to readers | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9955](https://issues.apache.org/jira/browse/HADOOP-9955) | RPC idle connection closing is extremely inefficient | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9748](https://issues.apache.org/jira/browse/HADOOP-9748) | Reduce blocking on UGI.ensureInitialized | Critical | security | Daryn Sharp | Daryn Sharp | +| [HDFS-5239](https://issues.apache.org/jira/browse/HDFS-5239) | Allow FSNamesystem lock fairness to be configurable | Major | namenode | Daryn Sharp | Daryn Sharp | +| [YARN-573](https://issues.apache.org/jira/browse/YARN-573) | Shared data structures in Public Localizer and Private Localizer are not Thread safe. | Critical | . | Omkar Vinit Joshi | Omkar Vinit Joshi | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/RELEASENOTES.0.23.10.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/RELEASENOTES.0.23.10.md new file mode 100644 index 00000000000..f1388d0a7a3 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.10/RELEASENOTES.0.23.10.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.23.10 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [YARN-707](https://issues.apache.org/jira/browse/YARN-707) | *Blocker* | **Add user info in the YARN ClientToken** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/CHANGES.0.23.11.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/CHANGES.0.23.11.md new file mode 100644 index 00000000000..3259d86bc4b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/CHANGES.0.23.11.md @@ -0,0 +1,105 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.11 - 2014-06-25 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10454](https://issues.apache.org/jira/browse/HADOOP-10454) | Provide FileContext version of har file system | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10164](https://issues.apache.org/jira/browse/HADOOP-10164) | Allow UGI to login with a known Subject | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-7688](https://issues.apache.org/jira/browse/HADOOP-7688) | When a servlet filter throws an exception in init(..), the Jetty server failed silently. | Major | . | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-6191](https://issues.apache.org/jira/browse/HDFS-6191) | Disable quota checks when replaying edit log. | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-5637](https://issues.apache.org/jira/browse/HDFS-5637) | try to refeatchToken while local read InvalidToken occurred | Major | hdfs-client, security | Liang Xie | Liang Xie | +| [HDFS-4461](https://issues.apache.org/jira/browse/HDFS-4461) | DirectoryScanner: volume path prefix takes up memory for every block that is scanned | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10588](https://issues.apache.org/jira/browse/HADOOP-10588) | Workaround for jetty6 acceptor startup issue | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10332](https://issues.apache.org/jira/browse/HADOOP-10332) | HttpServer's jetty audit log always logs 200 OK | Major | . | Daryn Sharp | Jonathan Eagles | +| [HADOOP-10146](https://issues.apache.org/jira/browse/HADOOP-10146) | Workaround JDK7 Process fd close bug | Critical | util | Daryn Sharp | Daryn Sharp | +| [HADOOP-10129](https://issues.apache.org/jira/browse/HADOOP-10129) | Distcp may succeed when it fails | Critical | tools/distcp | Daryn Sharp | Daryn Sharp | +| [HADOOP-10112](https://issues.apache.org/jira/browse/HADOOP-10112) | har file listing doesn't work with wild card | Major | tools | Brandon Li | Brandon Li | +| [HADOOP-10110](https://issues.apache.org/jira/browse/HADOOP-10110) | hadoop-auth has a build break due to missing dependency | Blocker | build | Chuan Liu | Chuan Liu | +| [HADOOP-10081](https://issues.apache.org/jira/browse/HADOOP-10081) | Client.setupIOStreams can leak socket resources on exception or error | Critical | ipc | Jason Lowe | Tsuyoshi Ozawa | +| [HADOOP-9230](https://issues.apache.org/jira/browse/HADOOP-9230) | TestUniformSizeInputFormat fails intermittently | Major | test | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-8826](https://issues.apache.org/jira/browse/HADOOP-8826) | Docs still refer to 0.20.205 as stable line | Minor | . | Robert Joseph Evans | Mit Desai | +| [HDFS-6449](https://issues.apache.org/jira/browse/HDFS-6449) | Incorrect counting in ContentSummaryComputationContext in 0.23. | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6166](https://issues.apache.org/jira/browse/HDFS-6166) | revisit balancer so\_timeout | Blocker | balancer & mover | Nathan Roberts | Nathan Roberts | +| [HDFS-5881](https://issues.apache.org/jira/browse/HDFS-5881) | Fix skip() of the short-circuit local reader (legacy). | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5806](https://issues.apache.org/jira/browse/HDFS-5806) | balancer should set SoTimeout to avoid indefinite hangs | Major | balancer & mover | Nathan Roberts | Nathan Roberts | +| [HDFS-5728](https://issues.apache.org/jira/browse/HDFS-5728) | [Diskfull] Block recovery will fail if the metafile does not have crc for all chunks of the block | Critical | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-4576](https://issues.apache.org/jira/browse/HDFS-4576) | Webhdfs authentication issues | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-5789](https://issues.apache.org/jira/browse/MAPREDUCE-5789) | Average Reduce time is incorrect on Job Overview page | Major | jobhistoryserver, webapps | Rushabh S Shah | Rushabh S Shah | +| [MAPREDUCE-5778](https://issues.apache.org/jira/browse/MAPREDUCE-5778) | JobSummary does not escape newlines in the job name | Major | jobhistoryserver | Jason Lowe | Akira AJISAKA | +| [MAPREDUCE-5757](https://issues.apache.org/jira/browse/MAPREDUCE-5757) | ConcurrentModificationException in JobControl.toList | Major | client | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5746](https://issues.apache.org/jira/browse/MAPREDUCE-5746) | Job diagnostics can implicate wrong task for a failed job | Major | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5744](https://issues.apache.org/jira/browse/MAPREDUCE-5744) | Job hangs because RMContainerAllocator$AssignedRequests.preemptReduce() violates the comparator contract | Blocker | . | Sangjin Lee | Gera Shegalov | +| [MAPREDUCE-5689](https://issues.apache.org/jira/browse/MAPREDUCE-5689) | MRAppMaster does not preempt reducers when scheduled maps cannot be fulfilled | Critical | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [MAPREDUCE-5623](https://issues.apache.org/jira/browse/MAPREDUCE-5623) | TestJobCleanup fails because of RejectedExecutionException and NPE. | Major | . | Tsuyoshi Ozawa | Jason Lowe | +| [MAPREDUCE-5454](https://issues.apache.org/jira/browse/MAPREDUCE-5454) | TestDFSIO fails intermittently on JDK7 | Major | test | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-3191](https://issues.apache.org/jira/browse/MAPREDUCE-3191) | docs for map output compression incorrectly reference SequenceFile | Trivial | . | Todd Lipcon | Chen He | +| [YARN-3829](https://issues.apache.org/jira/browse/YARN-3829) | The History Tracking UI is broken for Tez application on ResourceManager WebUI | Critical | applications | Irina Easterling | | +| [YARN-1932](https://issues.apache.org/jira/browse/YARN-1932) | Javascript injection on the job status page | Blocker | . | Mit Desai | Mit Desai | +| [YARN-1670](https://issues.apache.org/jira/browse/YARN-1670) | aggregated log writer can write more log data then it says is the log length | Critical | . | Thomas Graves | Mit Desai | +| [YARN-1592](https://issues.apache.org/jira/browse/YARN-1592) | CapacityScheduler tries to reserve more than a node's total memory on branch-0.23 | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-1180](https://issues.apache.org/jira/browse/YARN-1180) | Update capacity scheduler docs to include types on the configs | Trivial | capacityscheduler | Thomas Graves | Chen He | +| [YARN-1145](https://issues.apache.org/jira/browse/YARN-1145) | Potential file handle leak in aggregated logs web ui | Major | . | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-1053](https://issues.apache.org/jira/browse/YARN-1053) | Diagnostic message from ContainerExitEvent is ignored in ContainerImpl | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-853](https://issues.apache.org/jira/browse/YARN-853) | maximum-am-resource-percent doesn't work after refreshQueues command | Major | capacityscheduler | Devaraj K | Devaraj K | +| [YARN-500](https://issues.apache.org/jira/browse/YARN-500) | ResourceManager webapp is using next port if configured port is already in use | Major | resourcemanager | Nishan Shetty | Kenji Kikushima | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10148](https://issues.apache.org/jira/browse/HADOOP-10148) | backport hadoop-10107 to branch-0.23 | Minor | ipc | Chen He | Chen He | +| [YARN-1575](https://issues.apache.org/jira/browse/YARN-1575) | Public localizer crashes with "Localized unkown resource" | Critical | nodemanager | Jason Lowe | Jason Lowe | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/RELEASENOTES.0.23.11.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/RELEASENOTES.0.23.11.md new file mode 100644 index 00000000000..7fb4718e9cd --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.11/RELEASENOTES.0.23.11.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.23.11 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/CHANGES.0.23.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/CHANGES.0.23.2.md new file mode 100644 index 00000000000..cd1c26aebf6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/CHANGES.0.23.2.md @@ -0,0 +1,177 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.2 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8164](https://issues.apache.org/jira/browse/HADOOP-8164) | Handle paths using back slash as path separator for windows only | Major | fs | Suresh Srinivas | Daryn Sharp | +| [HADOOP-8131](https://issues.apache.org/jira/browse/HADOOP-8131) | FsShell put doesn't correctly handle a non-existent dir | Critical | . | Daryn Sharp | Daryn Sharp | +| [HDFS-2887](https://issues.apache.org/jira/browse/HDFS-2887) | Define a FSVolume interface | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2943](https://issues.apache.org/jira/browse/HDFS-2943) | Expose last checkpoint time and transaction stats as JMX metrics | Major | namenode | Aaron T. Myers | Aaron T. Myers | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8071](https://issues.apache.org/jira/browse/HADOOP-8071) | Avoid an extra packet in client code when nagling is disabled | Minor | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-8048](https://issues.apache.org/jira/browse/HADOOP-8048) | Allow merging of Credentials | Major | util | Daryn Sharp | Daryn Sharp | +| [HDFS-3066](https://issues.apache.org/jira/browse/HDFS-3066) | cap space usage of default log4j rolling policy (hdfs specific changes) | Major | scripts | Patrick Hunt | Patrick Hunt | +| [HDFS-3024](https://issues.apache.org/jira/browse/HDFS-3024) | Improve performance of stringification in addStoredBlock | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2985](https://issues.apache.org/jira/browse/HDFS-2985) | Improve logging when replicas are marked as corrupt | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2981](https://issues.apache.org/jira/browse/HDFS-2981) | The default value of dfs.client.block.write.replace-datanode-on-failure.enable should be true | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2907](https://issues.apache.org/jira/browse/HDFS-2907) | Make FSDataset in Datanode Pluggable | Minor | . | Sanjay Radia | Tsz Wo Nicholas Sze | +| [HDFS-2655](https://issues.apache.org/jira/browse/HDFS-2655) | BlockReaderLocal#skip performs unnecessary IO | Major | datanode | Eli Collins | Brandon Li | +| [HDFS-2506](https://issues.apache.org/jira/browse/HDFS-2506) | Umbrella jira for tracking separation of wire protocol datatypes from the implementation types | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-1217](https://issues.apache.org/jira/browse/HDFS-1217) | Some methods in the NameNdoe should not be public | Major | namenode | Tsz Wo Nicholas Sze | Laxman | +| [MAPREDUCE-3989](https://issues.apache.org/jira/browse/MAPREDUCE-3989) | cap space usage of default log4j rolling policy (mr specific changes) | Major | . | Patrick Hunt | Patrick Hunt | +| [MAPREDUCE-3922](https://issues.apache.org/jira/browse/MAPREDUCE-3922) | Fix the potential problem compiling 32 bit binaries on a x86\_64 host. | Minor | build, mrv2 | Eugene Koontz | Hitesh Shah | +| [MAPREDUCE-3901](https://issues.apache.org/jira/browse/MAPREDUCE-3901) | lazy load JobHistory Task and TaskAttempt details | Major | jobhistoryserver, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3864](https://issues.apache.org/jira/browse/MAPREDUCE-3864) | Fix cluster setup docs for correct SNN HTTPS parameters | Minor | documentation, security | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3849](https://issues.apache.org/jira/browse/MAPREDUCE-3849) | Change TokenCache's reading of the binary token file | Major | security | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3730](https://issues.apache.org/jira/browse/MAPREDUCE-3730) | Allow restarted NM to rejoin cluster before RM expires it | Minor | mrv2, resourcemanager | Jason Lowe | Jason Lowe | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8176](https://issues.apache.org/jira/browse/HADOOP-8176) | Disambiguate the destination of FsShell copies | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8146](https://issues.apache.org/jira/browse/HADOOP-8146) | FsShell commands cannot be interrupted | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8140](https://issues.apache.org/jira/browse/HADOOP-8140) | dfs -getmerge should process its argments better | Major | . | arkady borkovsky | Daryn Sharp | +| [HADOOP-8137](https://issues.apache.org/jira/browse/HADOOP-8137) | Site side links for commands manual (MAPREDUCE-3497) | Major | documentation | Vinod Kumar Vavilapalli | Thomas Graves | +| [HADOOP-8123](https://issues.apache.org/jira/browse/HADOOP-8123) | hadoop-project invalid pom warnings prevent transitive dependency resolution | Critical | build | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-8083](https://issues.apache.org/jira/browse/HADOOP-8083) | javadoc generation for some modules is not done under target/ | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8082](https://issues.apache.org/jira/browse/HADOOP-8082) | add hadoop-client and hadoop-minicluster to the dependency-management section | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8074](https://issues.apache.org/jira/browse/HADOOP-8074) | Small bug in hadoop error message for unknown commands | Trivial | scripts | Eli Collins | Colin Patrick McCabe | +| [HADOOP-8066](https://issues.apache.org/jira/browse/HADOOP-8066) | The full docs build intermittently fails | Major | build | Aaron T. Myers | Andrew Bayer | +| [HADOOP-8064](https://issues.apache.org/jira/browse/HADOOP-8064) | Remove unnecessary dependency on w3c.org in document processing | Major | build | Kihwal Lee | Kihwal Lee | +| [HADOOP-8057](https://issues.apache.org/jira/browse/HADOOP-8057) | hadoop-setup-conf.sh not working because of some extra spaces. | Major | scripts | Vinayakumar B | Vinayakumar B | +| [HADOOP-8051](https://issues.apache.org/jira/browse/HADOOP-8051) | HttpFS documentation it is not wired to the generated site | Major | documentation | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8050](https://issues.apache.org/jira/browse/HADOOP-8050) | Deadlock in metrics | Major | metrics | Kihwal Lee | Kihwal Lee | +| [HADOOP-8046](https://issues.apache.org/jira/browse/HADOOP-8046) | Revert StaticMapping semantics to the existing ones, add DNS mapping diagnostics in progress | Minor | . | Steve Loughran | Steve Loughran | +| [HADOOP-8042](https://issues.apache.org/jira/browse/HADOOP-8042) | When copying a file out of HDFS, modifying it, and uploading it back into HDFS, the put fails due to a CRC mismatch | Critical | fs | Kevin J. Price | Daryn Sharp | +| [HADOOP-8036](https://issues.apache.org/jira/browse/HADOOP-8036) | TestViewFsTrash assumes the user's home directory is 2 levels deep | Major | fs, test | Eli Collins | Colin Patrick McCabe | +| [HADOOP-8035](https://issues.apache.org/jira/browse/HADOOP-8035) | Hadoop Maven site is inefficient and runs phases redundantly | Minor | build | Andrew Bayer | Andrew Bayer | +| [HADOOP-7874](https://issues.apache.org/jira/browse/HADOOP-7874) | native libs should be under lib/native/ dir | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7660](https://issues.apache.org/jira/browse/HADOOP-7660) | Maven generated .classpath doesnot includes "target/generated-test-source/java" as source directory. | Minor | build | Laxman | Laxman | +| [HADOOP-6502](https://issues.apache.org/jira/browse/HADOOP-6502) | DistributedFileSystem#listStatus is very slow when listing a directory with a size of 1300 | Critical | util | Hairong Kuang | Sharad Agarwal | +| [HDFS-3853](https://issues.apache.org/jira/browse/HDFS-3853) | Port MiniDFSCluster enableManagedDfsDirsRedundancy option to branch-2 | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3160](https://issues.apache.org/jira/browse/HDFS-3160) | httpfs should exec catalina instead of forking it | Major | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-3101](https://issues.apache.org/jira/browse/HDFS-3101) | cannot read empty file using webhdfs | Major | webhdfs | Zhanwei Wang | Tsz Wo Nicholas Sze | +| [HDFS-3032](https://issues.apache.org/jira/browse/HDFS-3032) | Lease renewer tries forever even if renewal is not possible | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3012](https://issues.apache.org/jira/browse/HDFS-3012) | Exception while renewing delegation token | Critical | . | Ramya Sunil | Robert Joseph Evans | +| [HDFS-3008](https://issues.apache.org/jira/browse/HDFS-3008) | Negative caching of local addrs doesn't work | Major | hdfs-client | Eli Collins | Eli Collins | +| [HDFS-3006](https://issues.apache.org/jira/browse/HDFS-3006) | Webhdfs "SETOWNER" call returns incorrect content-type | Major | webhdfs | bc Wong | Tsz Wo Nicholas Sze | +| [HDFS-2969](https://issues.apache.org/jira/browse/HDFS-2969) | ExtendedBlock.equals is incorrectly implemented | Critical | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2950](https://issues.apache.org/jira/browse/HDFS-2950) | Secondary NN HTTPS address should be listed as a NAMESERVICE\_SPECIFIC\_KEY | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2944](https://issues.apache.org/jira/browse/HDFS-2944) | Typo in hdfs-default.xml causes dfs.client.block.write.replace-datanode-on-failure.enable to be mistakenly disabled | Major | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2938](https://issues.apache.org/jira/browse/HDFS-2938) | Recursive delete of a large directory makes namenode unresponsive | Major | namenode | Suresh Srinivas | Hari Mankude | +| [HDFS-2815](https://issues.apache.org/jira/browse/HDFS-2815) | Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed. | Critical | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2764](https://issues.apache.org/jira/browse/HDFS-2764) | TestBackupNode is racy | Major | namenode, test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2725](https://issues.apache.org/jira/browse/HDFS-2725) | hdfs script usage information is missing the information about "dfs" command | Major | hdfs-client | Prashant Sharma | | +| [HDFS-2525](https://issues.apache.org/jira/browse/HDFS-2525) | Race between BlockPoolSliceScanner and append | Critical | datanode | Todd Lipcon | Brandon Li | +| [HDFS-776](https://issues.apache.org/jira/browse/HDFS-776) | Fix exception handling in Balancer | Critical | balancer & mover | Owen O'Malley | Uma Maheswara Rao G | +| [MAPREDUCE-4061](https://issues.apache.org/jira/browse/MAPREDUCE-4061) | RM only has 1 AM launcher thread | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4043](https://issues.apache.org/jira/browse/MAPREDUCE-4043) | Secret keys set in Credentials are not seen by tasks | Blocker | mrv2, security | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4034](https://issues.apache.org/jira/browse/MAPREDUCE-4034) | Unable to view task logs on history server with mapreduce.job.acl-view-job=\* | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4025](https://issues.apache.org/jira/browse/MAPREDUCE-4025) | AM can crash if task attempt reports bogus progress value | Blocker | mr-am, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4006](https://issues.apache.org/jira/browse/MAPREDUCE-4006) | history server container log web UI sometimes combines stderr/stdout/syslog contents together | Major | jobhistoryserver, mrv2 | Jason Lowe | Siddharth Seth | +| [MAPREDUCE-4005](https://issues.apache.org/jira/browse/MAPREDUCE-4005) | AM container logs URL is broken for completed apps when log aggregation is enabled | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3982](https://issues.apache.org/jira/browse/MAPREDUCE-3982) | TestEmptyJob fails with FileNotFound | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3977](https://issues.apache.org/jira/browse/MAPREDUCE-3977) | LogAggregationService leaks log aggregator objects | Critical | mrv2, nodemanager | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3976](https://issues.apache.org/jira/browse/MAPREDUCE-3976) | TestRMContainerAllocator failing | Major | mrv2 | Bikas Saha | Jason Lowe | +| [MAPREDUCE-3975](https://issues.apache.org/jira/browse/MAPREDUCE-3975) | Default value not set for Configuration parameter mapreduce.job.local.dir | Blocker | mrv2 | Eric Payne | Eric Payne | +| [MAPREDUCE-3964](https://issues.apache.org/jira/browse/MAPREDUCE-3964) | ResourceManager does not have JVM metrics | Critical | mrv2, resourcemanager | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3961](https://issues.apache.org/jira/browse/MAPREDUCE-3961) | Map/ReduceSlotMillis computation incorrect | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-3960](https://issues.apache.org/jira/browse/MAPREDUCE-3960) | web proxy doesn't forward request to AM with configured hostname/IP | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3954](https://issues.apache.org/jira/browse/MAPREDUCE-3954) | Clean up passing HEAPSIZE to yarn and mapred commands. | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3931](https://issues.apache.org/jira/browse/MAPREDUCE-3931) | MR tasks failing due to changing timestamps on Resources to download | Major | mrv2 | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-3930](https://issues.apache.org/jira/browse/MAPREDUCE-3930) | The AM page for a Reducer that has not been launched causes an NPE | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3929](https://issues.apache.org/jira/browse/MAPREDUCE-3929) | output of mapred -showacl is not clear | Major | mrv2 | John George | John George | +| [MAPREDUCE-3920](https://issues.apache.org/jira/browse/MAPREDUCE-3920) | Revise yarn default port number selection | Major | nodemanager, resourcemanager | Dave Thompson | Dave Thompson | +| [MAPREDUCE-3918](https://issues.apache.org/jira/browse/MAPREDUCE-3918) | proc\_historyserver no longer in command line arguments for HistoryServer | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3913](https://issues.apache.org/jira/browse/MAPREDUCE-3913) | RM application webpage is unresponsive after 2000 jobs | Critical | mrv2, webapps | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3910](https://issues.apache.org/jira/browse/MAPREDUCE-3910) | user not allowed to submit jobs even though queue -showacls shows it allows | Blocker | mrv2 | John George | John George | +| [MAPREDUCE-3904](https://issues.apache.org/jira/browse/MAPREDUCE-3904) | [NPE] Job history produced with mapreduce.cluster.acls.enabled false can not be viewed with mapreduce.cluster.acls.enabled true | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3903](https://issues.apache.org/jira/browse/MAPREDUCE-3903) | no admin override to view jobs on mr app master and job history server | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3897](https://issues.apache.org/jira/browse/MAPREDUCE-3897) | capacity scheduler - maxActiveApplicationsPerUser calculation can be wrong | Critical | mrv2 | Thomas Graves | Eric Payne | +| [MAPREDUCE-3896](https://issues.apache.org/jira/browse/MAPREDUCE-3896) | pig job through oozie hangs | Blocker | jobhistoryserver, mrv2 | John George | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3884](https://issues.apache.org/jira/browse/MAPREDUCE-3884) | PWD should be first in the classpath of MR tasks | Critical | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3878](https://issues.apache.org/jira/browse/MAPREDUCE-3878) | Null user on filtered jobhistory job page | Critical | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3866](https://issues.apache.org/jira/browse/MAPREDUCE-3866) | bin/yarn prints the command line unnecessarily | Minor | mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3862](https://issues.apache.org/jira/browse/MAPREDUCE-3862) | Nodemanager can appear to hang on shutdown due to lingering DeletionService threads | Major | mrv2, nodemanager | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3852](https://issues.apache.org/jira/browse/MAPREDUCE-3852) | test TestLinuxResourceCalculatorPlugin failing | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3816](https://issues.apache.org/jira/browse/MAPREDUCE-3816) | capacity scheduler web ui bar graphs for used capacity wrong | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3792](https://issues.apache.org/jira/browse/MAPREDUCE-3792) | job -list displays only the jobs submitted by a particular user | Critical | mrv2 | Ramya Sunil | Jason Lowe | +| [MAPREDUCE-3790](https://issues.apache.org/jira/browse/MAPREDUCE-3790) | Broken pipe on streaming job can lead to truncated output for a successful job | Major | contrib/streaming, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3738](https://issues.apache.org/jira/browse/MAPREDUCE-3738) | NM can hang during shutdown if AppLogAggregatorImpl thread dies unexpectedly | Critical | mrv2, nodemanager | Jason Lowe | Jason Lowe | +| [MAPREDUCE-3706](https://issues.apache.org/jira/browse/MAPREDUCE-3706) | HTTP Circular redirect error on the job attempts page | Critical | mrv2 | Thomas Graves | Robert Joseph Evans | +| [MAPREDUCE-3687](https://issues.apache.org/jira/browse/MAPREDUCE-3687) | If AM dies before it returns new tracking URL, proxy redirects to http://N/A/ and doesn't return error code | Major | mrv2 | David Capwell | Ravi Prakash | +| [MAPREDUCE-3686](https://issues.apache.org/jira/browse/MAPREDUCE-3686) | history server web ui - job counter values for map/reduce not shown properly | Critical | mrv2 | Thomas Graves | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3680](https://issues.apache.org/jira/browse/MAPREDUCE-3680) | FifoScheduler web service rest API can print out invalid JSON | Major | mrv2 | Thomas Graves | | +| [MAPREDUCE-3634](https://issues.apache.org/jira/browse/MAPREDUCE-3634) | All daemons should crash instead of hanging around when their EventHandlers get exceptions | Major | mrv2 | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3614](https://issues.apache.org/jira/browse/MAPREDUCE-3614) | finalState UNDEFINED if AM is killed by hand | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3583](https://issues.apache.org/jira/browse/MAPREDUCE-3583) | ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException | Critical | . | Ted Yu | Ted Yu | +| [MAPREDUCE-3497](https://issues.apache.org/jira/browse/MAPREDUCE-3497) | missing documentation for yarn cli and subcommands - similar to commands\_manual.html | Major | documentation, mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3034](https://issues.apache.org/jira/browse/MAPREDUCE-3034) | NM should act on a REBOOT command from RM | Critical | mrv2, nodemanager | Vinod Kumar Vavilapalli | Devaraj K | +| [MAPREDUCE-3009](https://issues.apache.org/jira/browse/MAPREDUCE-3009) | RM UI -\> Applications -\> Application(Job History) -\> Map Tasks -\> Task ID -\> Node link is not working | Major | jobhistoryserver, mrv2 | chackaravarthy | chackaravarthy | +| [MAPREDUCE-2855](https://issues.apache.org/jira/browse/MAPREDUCE-2855) | ResourceBundle lookup during counter name resolution takes a lot of time | Major | . | Todd Lipcon | Siddharth Seth | +| [MAPREDUCE-2793](https://issues.apache.org/jira/browse/MAPREDUCE-2793) | [MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs | Critical | mrv2 | Ramya Sunil | Bikas Saha | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-3104](https://issues.apache.org/jira/browse/HDFS-3104) | Add tests for mkdir -p | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-3098](https://issues.apache.org/jira/browse/HDFS-3098) | Update FsShell tests for quoted metachars | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-3060](https://issues.apache.org/jira/browse/HDFS-3060) | Bump TestDistributedUpgrade#testDistributedUpgrade timeout | Minor | test | Eli Collins | Eli Collins | +| [HDFS-2038](https://issues.apache.org/jira/browse/HDFS-2038) | Update test to handle relative paths with globs | Critical | test | Daryn Sharp | Kihwal Lee | +| [MAPREDUCE-3877](https://issues.apache.org/jira/browse/MAPREDUCE-3877) | Add a test to formalise the current state transitions of the yarn lifecycle | Minor | mrv2 | Steve Loughran | Steve Loughran | +| [MAPREDUCE-3798](https://issues.apache.org/jira/browse/MAPREDUCE-3798) | TestJobCleanup testCustomCleanup is failing | Major | test | Ravi Prakash | Ravi Prakash | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8175](https://issues.apache.org/jira/browse/HADOOP-8175) | Add mkdir -p flag | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8173](https://issues.apache.org/jira/browse/HADOOP-8173) | FsShell needs to handle quoted metachars | Major | fs | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3944](https://issues.apache.org/jira/browse/MAPREDUCE-3944) | JobHistory web services are slower then the UI and can easly overload the JH | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8032](https://issues.apache.org/jira/browse/HADOOP-8032) | mvn site:stage-deploy should be able to use the scp protocol to stage documents | Major | build, documentation | Ravi Prakash | Ravi Prakash | +| [HDFS-2931](https://issues.apache.org/jira/browse/HDFS-2931) | Switch the DataNode's BlockVolumeChoosingPolicy to be a private-audience interface | Minor | datanode | Harsh J | Harsh J | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/RELEASENOTES.0.23.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/RELEASENOTES.0.23.2.md new file mode 100644 index 00000000000..65915656942 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.2/RELEASENOTES.0.23.2.md @@ -0,0 +1,199 @@ + + +# Apache Hadoop 0.23.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8175](https://issues.apache.org/jira/browse/HADOOP-8175) | *Major* | **Add mkdir -p flag** + +FsShell mkdir now accepts a -p flag. Like unix, mkdir -p will not fail if the directory already exists. Unlike unix, intermediate directories are always created, regardless of the flag, to avoid incompatibilities at this time. + + +--- + +* [HADOOP-8164](https://issues.apache.org/jira/browse/HADOOP-8164) | *Major* | **Handle paths using back slash as path separator for windows only** + +This jira only allows providing paths using back slash as separator on Windows. The back slash on *nix system will be used as escape character. The support for paths using back slash as path separator will be removed in HADOOP-8139 in release 23.3. + + +--- + +* [HADOOP-8131](https://issues.apache.org/jira/browse/HADOOP-8131) | *Critical* | **FsShell put doesn't correctly handle a non-existent dir** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-2950](https://issues.apache.org/jira/browse/HDFS-2950) | *Minor* | **Secondary NN HTTPS address should be listed as a NAMESERVICE\_SPECIFIC\_KEY** + +The configuration dfs.secondary.https.port has been renamed to dfs.namenode.secondary.https-port for consistency. The old configuration is still supported via a deprecation path. + + +--- + +* [HDFS-2907](https://issues.apache.org/jira/browse/HDFS-2907) | *Minor* | **Make FSDataset in Datanode Pluggable** + +Add a private conf property dfs.datanode.fsdataset.factory to make FSDataset in Datanode pluggable. + + +--- + +* [HDFS-2887](https://issues.apache.org/jira/browse/HDFS-2887) | *Major* | **Define a FSVolume interface** + +FSVolume, is a part of FSDatasetInterface implementation, should not be referred outside FSDataset. A new FSVolumeInterface is defined. The BlockVolumeChoosingPolicy.chooseVolume(..) method signature is also updated. + + +--- + +* [MAPREDUCE-3982](https://issues.apache.org/jira/browse/MAPREDUCE-3982) | *Critical* | **TestEmptyJob fails with FileNotFound** + +Fixed FileOutputCommitter to not err out for an 'empty-job' whose tasks don't write any outputs. + + +--- + +* [MAPREDUCE-3975](https://issues.apache.org/jira/browse/MAPREDUCE-3975) | *Blocker* | **Default value not set for Configuration parameter mapreduce.job.local.dir** + +Exporting mapreduce.job.local.dir for mapreduce tasks to use as job-level shared scratch space. + + +--- + +* [MAPREDUCE-3954](https://issues.apache.org/jira/browse/MAPREDUCE-3954) | *Blocker* | **Clean up passing HEAPSIZE to yarn and mapred commands.** + +Added new envs to separate heap size for different daemons started via bin scripts. + + +--- + +* [MAPREDUCE-3931](https://issues.apache.org/jira/browse/MAPREDUCE-3931) | *Major* | **MR tasks failing due to changing timestamps on Resources to download** + +Changed PB implementation of LocalResource to take locks so that race conditions don't fail tasks by inadvertantly changing the timestamps. + + +--- + +* [MAPREDUCE-3922](https://issues.apache.org/jira/browse/MAPREDUCE-3922) | *Minor* | **Fix the potential problem compiling 32 bit binaries on a x86\_64 host.** + +Fixed build to not compile 32bit container-executor binary by default on all platforms. + + +--- + +* [MAPREDUCE-3920](https://issues.apache.org/jira/browse/MAPREDUCE-3920) | *Major* | **Revise yarn default port number selection** + +port number changes for resourcemanager and nodemanager + + +--- + +* [MAPREDUCE-3910](https://issues.apache.org/jira/browse/MAPREDUCE-3910) | *Blocker* | **user not allowed to submit jobs even though queue -showacls shows it allows** + +Fixed a bug in CapacityScheduler LeafQueue which was causing app-submission to fail. + + +--- + +* [MAPREDUCE-3901](https://issues.apache.org/jira/browse/MAPREDUCE-3901) | *Major* | **lazy load JobHistory Task and TaskAttempt details** + +Modified JobHistory records in YARN to lazily load job and task reports so as to improve UI response times. + + +--- + +* [MAPREDUCE-3866](https://issues.apache.org/jira/browse/MAPREDUCE-3866) | *Minor* | **bin/yarn prints the command line unnecessarily** + +Fixed the bin/yarn script to not print the command line unnecessarily. + + +--- + +* [MAPREDUCE-3798](https://issues.apache.org/jira/browse/MAPREDUCE-3798) | *Major* | **TestJobCleanup testCustomCleanup is failing** + +Fixed failing TestJobCleanup.testCusomCleanup() and moved it to the maven build. + + +--- + +* [MAPREDUCE-3792](https://issues.apache.org/jira/browse/MAPREDUCE-3792) | *Critical* | **job -list displays only the jobs submitted by a particular user** + +Fix "bin/mapred job -list" to display all jobs instead of only the jobs owned by the user. + + +--- + +* [MAPREDUCE-3738](https://issues.apache.org/jira/browse/MAPREDUCE-3738) | *Critical* | **NM can hang during shutdown if AppLogAggregatorImpl thread dies unexpectedly** + +Committed to trunk and branch-0.23. Thanks Jason. + + +--- + +* [MAPREDUCE-3730](https://issues.apache.org/jira/browse/MAPREDUCE-3730) | *Minor* | **Allow restarted NM to rejoin cluster before RM expires it** + +Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry. + + +--- + +* [MAPREDUCE-3686](https://issues.apache.org/jira/browse/MAPREDUCE-3686) | *Critical* | **history server web ui - job counter values for map/reduce not shown properly** + +Fixed two bugs in Counters because of which web app displays zero counter values for framework counters. + + +--- + +* [MAPREDUCE-3634](https://issues.apache.org/jira/browse/MAPREDUCE-3634) | *Major* | **All daemons should crash instead of hanging around when their EventHandlers get exceptions** + +Fixed all daemons to crash instead of hanging around when their EventHandlers get exceptions. + + +--- + +* [MAPREDUCE-3614](https://issues.apache.org/jira/browse/MAPREDUCE-3614) | *Major* | ** finalState UNDEFINED if AM is killed by hand** + +Fixed MR AM to close history file quickly and send a correct final state to the RM when it is killed. + + +--- + +* [MAPREDUCE-3009](https://issues.apache.org/jira/browse/MAPREDUCE-3009) | *Major* | **RM UI -\> Applications -\> Application(Job History) -\> Map Tasks -\> Task ID -\> Node link is not working** + +Fixed node link on JobHistory webapp. + + +--- + +* [MAPREDUCE-2855](https://issues.apache.org/jira/browse/MAPREDUCE-2855) | *Major* | **ResourceBundle lookup during counter name resolution takes a lot of time** + +Passing a cached class-loader to ResourceBundle creator to minimize counter names lookup time. + + +--- + +* [MAPREDUCE-2793](https://issues.apache.org/jira/browse/MAPREDUCE-2793) | *Critical* | **[MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs** + +Corrected AppIDs, JobIDs, TaskAttemptIDs to be of correct format on the web pages. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/CHANGES.0.23.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/CHANGES.0.23.3.md new file mode 100644 index 00000000000..e997154e617 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/CHANGES.0.23.3.md @@ -0,0 +1,334 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.3 - 2012-09-20 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8551](https://issues.apache.org/jira/browse/HADOOP-8551) | fs -mkdir creates parent directories without the -p option | Major | fs | Robert Joseph Evans | John George | +| [HDFS-3318](https://issues.apache.org/jira/browse/HDFS-3318) | Hftp hangs on transfers \>2GB | Blocker | hdfs-client | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4311](https://issues.apache.org/jira/browse/MAPREDUCE-4311) | Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings | Major | capacity-sched, mrv2 | Thomas Graves | Karthik Kambatla | +| [MAPREDUCE-4072](https://issues.apache.org/jira/browse/MAPREDUCE-4072) | User set java.library.path seems to overwrite default creating problems native lib loading | Major | mrv2 | Anupam Seth | Anupam Seth | +| [MAPREDUCE-3812](https://issues.apache.org/jira/browse/MAPREDUCE-3812) | Lower default allocation sizes, fix allocation configurations and document them | Major | mrv2, performance | Vinod Kumar Vavilapalli | Harsh J | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2978](https://issues.apache.org/jira/browse/HDFS-2978) | The NameNode should expose name dir statuses via JMX | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-3773](https://issues.apache.org/jira/browse/MAPREDUCE-3773) | Add queue metrics with buckets for job run times | Major | jobtracker | Owen O'Malley | Owen O'Malley | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8700](https://issues.apache.org/jira/browse/HADOOP-8700) | Move the checksum type constants to an enum | Minor | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-8635](https://issues.apache.org/jira/browse/HADOOP-8635) | Cannot cancel paths registered deleteOnExit | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8535](https://issues.apache.org/jira/browse/HADOOP-8535) | Cut hadoop build times in half (upgrade maven-compiler-plugin to 2.5.1) | Major | build | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-8525](https://issues.apache.org/jira/browse/HADOOP-8525) | Provide Improved Traceability for Configuration | Trivial | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8373](https://issues.apache.org/jira/browse/HADOOP-8373) | Port RPC.getServerAddress to 0.23 | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-8335](https://issues.apache.org/jira/browse/HADOOP-8335) | Improve Configuration's address handling | Major | util | Daryn Sharp | Daryn Sharp | +| [HADOOP-8286](https://issues.apache.org/jira/browse/HADOOP-8286) | Simplify getting a socket address from conf | Major | conf | Daryn Sharp | Daryn Sharp | +| [HADOOP-8242](https://issues.apache.org/jira/browse/HADOOP-8242) | AbstractDelegationTokenIdentifier: add getter methods for owner and realuser | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8240](https://issues.apache.org/jira/browse/HADOOP-8240) | Allow users to specify a checksum type on create() | Major | fs | Kihwal Lee | Kihwal Lee | +| [HADOOP-8239](https://issues.apache.org/jira/browse/HADOOP-8239) | Extend MD5MD5CRC32FileChecksum to show the actual checksum type being used | Major | fs | Kihwal Lee | Kihwal Lee | +| [HADOOP-8227](https://issues.apache.org/jira/browse/HADOOP-8227) | Allow RPC to limit ephemeral port range. | Blocker | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8141](https://issues.apache.org/jira/browse/HADOOP-8141) | Add method to init krb5 cipher suites | Trivial | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7510](https://issues.apache.org/jira/browse/HADOOP-7510) | Tokens should use original hostname provided instead of ip | Major | security | Daryn Sharp | Daryn Sharp | +| [HDFS-2652](https://issues.apache.org/jira/browse/HDFS-2652) | Port token service changes from 205 | Major | . | Daryn Sharp | Daryn Sharp | +| [HDFS-2421](https://issues.apache.org/jira/browse/HDFS-2421) | Improve the concurrency of SerialNumberMap in NameNode | Major | namenode | Hairong Kuang | Jing Zhao | +| [HDFS-208](https://issues.apache.org/jira/browse/HDFS-208) | name node should warn if only one dir is listed in dfs.name.dir | Minor | namenode | Allen Wittenauer | Uma Maheswara Rao G | +| [MAPREDUCE-4614](https://issues.apache.org/jira/browse/MAPREDUCE-4614) | Simplify debugging a job's tokens | Major | client, task | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4375](https://issues.apache.org/jira/browse/MAPREDUCE-4375) | Show Configuration Tracability in MR UI | Major | applicationmaster | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4301](https://issues.apache.org/jira/browse/MAPREDUCE-4301) | Dedupe some strings in MRAM for memory savings | Major | applicationmaster | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4283](https://issues.apache.org/jira/browse/MAPREDUCE-4283) | Display tail of aggregated logs by default | Major | jobhistoryserver, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4210](https://issues.apache.org/jira/browse/MAPREDUCE-4210) | Expose listener address for WebApp | Major | webapps | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4205](https://issues.apache.org/jira/browse/MAPREDUCE-4205) | retrofit all JVM shutdown hooks to use ShutdownHookManager | Major | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4190](https://issues.apache.org/jira/browse/MAPREDUCE-4190) | Improve web UI for task attempts userlog link | Major | mrv2, webapps | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4151](https://issues.apache.org/jira/browse/MAPREDUCE-4151) | RM scheduler web page should filter apps to those that are relevant to scheduling | Major | mrv2, webapps | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4079](https://issues.apache.org/jira/browse/MAPREDUCE-4079) | Allow MR AppMaster to limit ephemeral port range. | Blocker | mr-am, mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4059](https://issues.apache.org/jira/browse/MAPREDUCE-4059) | The history server should have a separate pluggable storage/query interface | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4017](https://issues.apache.org/jira/browse/MAPREDUCE-4017) | Add jobname to jobsummary log | Trivial | jobhistoryserver, jobtracker | Koji Noguchi | Thomas Graves | +| [MAPREDUCE-3935](https://issues.apache.org/jira/browse/MAPREDUCE-3935) | Annotate Counters.Counter and Counters.Group as @Public | Major | client | Tom White | Tom White | +| [MAPREDUCE-3907](https://issues.apache.org/jira/browse/MAPREDUCE-3907) | Document entries mapred-default.xml for the jobhistory server. | Minor | documentation | Eugene Koontz | Eugene Koontz | +| [MAPREDUCE-3906](https://issues.apache.org/jira/browse/MAPREDUCE-3906) | Fix inconsistency in documentation regarding mapreduce.jobhistory.principal | Trivial | documentation | Eugene Koontz | Eugene Koontz | +| [MAPREDUCE-3871](https://issues.apache.org/jira/browse/MAPREDUCE-3871) | Allow symlinking in LocalJobRunner DistributedCache | Major | distributed-cache | Tom White | Tom White | +| [MAPREDUCE-3850](https://issues.apache.org/jira/browse/MAPREDUCE-3850) | Avoid redundant calls for tokens in TokenCache | Major | security | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3842](https://issues.apache.org/jira/browse/MAPREDUCE-3842) | stop webpages from automatic refreshing | Critical | mrv2, webapps | Alejandro Abdelnur | Thomas Graves | +| [MAPREDUCE-3659](https://issues.apache.org/jira/browse/MAPREDUCE-3659) | Host-based token support | Major | security | Daryn Sharp | Daryn Sharp | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8727](https://issues.apache.org/jira/browse/HADOOP-8727) | Gracefully deprecate dfs.umaskmode in 2.x onwards | Major | conf | Harsh J | Harsh J | +| [HADOOP-8726](https://issues.apache.org/jira/browse/HADOOP-8726) | The Secrets in Credentials are not available to MR tasks | Major | security | Benoy Antony | Daryn Sharp | +| [HADOOP-8725](https://issues.apache.org/jira/browse/HADOOP-8725) | MR is broken when security is off | Blocker | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-8709](https://issues.apache.org/jira/browse/HADOOP-8709) | globStatus changed behavior from 0.20/1.x | Critical | fs | Jason Lowe | Jason Lowe | +| [HADOOP-8703](https://issues.apache.org/jira/browse/HADOOP-8703) | distcpV2: turn CRC checking off for 0 byte size | Major | . | Dave Thompson | Dave Thompson | +| [HADOOP-8697](https://issues.apache.org/jira/browse/HADOOP-8697) | TestWritableName fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8695](https://issues.apache.org/jira/browse/HADOOP-8695) | TestPathData fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8693](https://issues.apache.org/jira/browse/HADOOP-8693) | TestSecurityUtil fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8692](https://issues.apache.org/jira/browse/HADOOP-8692) | TestLocalDirAllocator fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8637](https://issues.apache.org/jira/browse/HADOOP-8637) | FilterFileSystem#setWriteChecksum is broken | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8634](https://issues.apache.org/jira/browse/HADOOP-8634) | Ensure FileSystem#close doesn't squawk for deleteOnExit paths | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8633](https://issues.apache.org/jira/browse/HADOOP-8633) | Interrupted FsShell copies may leave tmp files | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8627](https://issues.apache.org/jira/browse/HADOOP-8627) | FS deleteOnExit may delete the wrong path | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8614](https://issues.apache.org/jira/browse/HADOOP-8614) | IOUtils#skipFully hangs forever on EOF | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8613](https://issues.apache.org/jira/browse/HADOOP-8613) | AbstractDelegationTokenIdentifier#getUser() should set token auth type | Critical | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-8611](https://issues.apache.org/jira/browse/HADOOP-8611) | Allow fall-back to the shell-based implementation when JNI-based users-group mapping fails | Major | security | Kihwal Lee | Robert Parker | +| [HADOOP-8606](https://issues.apache.org/jira/browse/HADOOP-8606) | FileSystem.get may return the wrong filesystem | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8599](https://issues.apache.org/jira/browse/HADOOP-8599) | Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file | Major | fs | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-8587](https://issues.apache.org/jira/browse/HADOOP-8587) | HarFileSystem access of harMetaCache isn't threadsafe | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-8573](https://issues.apache.org/jira/browse/HADOOP-8573) | Configuration tries to read from an inputstream resource multiple times. | Major | conf | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8550](https://issues.apache.org/jira/browse/HADOOP-8550) | hadoop fs -touchz automatically created parent directories | Major | fs | Robert Joseph Evans | John George | +| [HADOOP-8543](https://issues.apache.org/jira/browse/HADOOP-8543) | Invalid pom.xml files on 0.23 branch | Major | build | Radim Kolar | Radim Kolar | +| [HADOOP-8501](https://issues.apache.org/jira/browse/HADOOP-8501) | Gridmix fails to compile on OpenJDK7u4 | Major | benchmarks | Radim Kolar | Radim Kolar | +| [HADOOP-8495](https://issues.apache.org/jira/browse/HADOOP-8495) | Update Netty to avoid leaking file descriptors during shuffle | Critical | build | Jason Lowe | Jason Lowe | +| [HADOOP-8450](https://issues.apache.org/jira/browse/HADOOP-8450) | Remove src/test/system | Trivial | test | Colin Patrick McCabe | Eli Collins | +| [HADOOP-8393](https://issues.apache.org/jira/browse/HADOOP-8393) | hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMaster | Major | scripts | Patrick Hunt | Patrick Hunt | +| [HADOOP-8390](https://issues.apache.org/jira/browse/HADOOP-8390) | TestFileSystemCanonicalization fails with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8370](https://issues.apache.org/jira/browse/HADOOP-8370) | Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found | Major | native | Trevor Robinson | Trevor Robinson | +| [HADOOP-8342](https://issues.apache.org/jira/browse/HADOOP-8342) | HDFS command fails with exception following merge of HADOOP-8325 | Major | fs | Randy Clayton | Alejandro Abdelnur | +| [HADOOP-8341](https://issues.apache.org/jira/browse/HADOOP-8341) | Fix or filter findbugs issues in hadoop-tools | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8334](https://issues.apache.org/jira/browse/HADOOP-8334) | HttpServer sometimes returns incorrect port | Major | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-8330](https://issues.apache.org/jira/browse/HADOOP-8330) | TestSequenceFile.testCreateUsesFsArg() is broken | Minor | test | John George | John George | +| [HADOOP-8328](https://issues.apache.org/jira/browse/HADOOP-8328) | Duplicate FileSystem Statistics object for 'file' scheme | Major | fs | Tom White | Tom White | +| [HADOOP-8327](https://issues.apache.org/jira/browse/HADOOP-8327) | distcpv2 and distcpv1 jars should not coexist | Major | . | Dave Thompson | Dave Thompson | +| [HADOOP-8325](https://issues.apache.org/jira/browse/HADOOP-8325) | Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhook | Critical | fs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8317](https://issues.apache.org/jira/browse/HADOOP-8317) | Update maven-assembly-plugin to 2.3 - fix build on FreeBSD | Major | build | Radim Kolar | | +| [HADOOP-8305](https://issues.apache.org/jira/browse/HADOOP-8305) | distcp over viewfs is broken | Major | viewfs | John George | John George | +| [HADOOP-8288](https://issues.apache.org/jira/browse/HADOOP-8288) | Remove references of mapred.child.ulimit etc. since they are not being used any more | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-8225](https://issues.apache.org/jira/browse/HADOOP-8225) | DistCp fails when invoked by Oozie | Blocker | security | Mithun Radhakrishnan | Daryn Sharp | +| [HADOOP-8197](https://issues.apache.org/jira/browse/HADOOP-8197) | Configuration logs WARNs on every use of a deprecated key | Critical | conf | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8180](https://issues.apache.org/jira/browse/HADOOP-8180) | Remove hsqldb since its not needed from pom.xml | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-8179](https://issues.apache.org/jira/browse/HADOOP-8179) | risk of NPE in CopyCommands processArguments() | Minor | fs | Steve Loughran | Daryn Sharp | +| [HADOOP-8172](https://issues.apache.org/jira/browse/HADOOP-8172) | Configuration no longer sets all keys in a deprecated key list. | Critical | conf | Robert Joseph Evans | Anupam Seth | +| [HADOOP-8167](https://issues.apache.org/jira/browse/HADOOP-8167) | Configuration deprecation logic breaks backwards compatibility | Blocker | conf | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8144](https://issues.apache.org/jira/browse/HADOOP-8144) | pseudoSortByDistance in NetworkTopology doesn't work properly if no local node and first node is local rack node | Minor | io | Junping Du | Junping Du | +| [HADOOP-8129](https://issues.apache.org/jira/browse/HADOOP-8129) | ViewFileSystemTestSetup setupForViewFileSystem is erring when the user's home directory is somewhere other than /home (eg. /User) etc. | Major | fs, test | Ravi Prakash | Ahmed Radwan | +| [HADOOP-8110](https://issues.apache.org/jira/browse/HADOOP-8110) | TestViewFsTrash occasionally fails | Major | fs | Tsz Wo Nicholas Sze | Jason Lowe | +| [HADOOP-8104](https://issues.apache.org/jira/browse/HADOOP-8104) | Inconsistent Jackson versions | Major | . | Colin Patrick McCabe | Alejandro Abdelnur | +| [HADOOP-8088](https://issues.apache.org/jira/browse/HADOOP-8088) | User-group mapping cache incorrectly does negative caching on transient failures | Major | security | Kihwal Lee | Kihwal Lee | +| [HADOOP-8060](https://issues.apache.org/jira/browse/HADOOP-8060) | Add a capability to discover and set checksum types per file. | Major | fs, util | Kihwal Lee | Kihwal Lee | +| [HADOOP-8014](https://issues.apache.org/jira/browse/HADOOP-8014) | ViewFileSystem does not correctly implement getDefaultBlockSize, getDefaultReplication, getContentSummary | Major | fs | Daryn Sharp | John George | +| [HADOOP-8005](https://issues.apache.org/jira/browse/HADOOP-8005) | Multiple SLF4J binding message in .out file for all daemons | Major | scripts | Joe Crobak | Jason Lowe | +| [HADOOP-7967](https://issues.apache.org/jira/browse/HADOOP-7967) | Need generalized multi-token filesystem support | Critical | fs, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-7621](https://issues.apache.org/jira/browse/HADOOP-7621) | alfredo config should be in a file not readable by users | Critical | security | Alejandro Abdelnur | Aaron T. Myers | +| [HADOOP-6963](https://issues.apache.org/jira/browse/HADOOP-6963) | Fix FileUtil.getDU. It should not include the size of the directory or follow symbolic links | Critical | fs | Owen O'Malley | Ravi Prakash | +| [HDFS-3890](https://issues.apache.org/jira/browse/HDFS-3890) | filecontext mkdirs doesn't apply umask as expected | Critical | . | Thomas Graves | Thomas Graves | +| [HDFS-3873](https://issues.apache.org/jira/browse/HDFS-3873) | Hftp assumes security is disabled if token fetch fails | Major | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3861](https://issues.apache.org/jira/browse/HDFS-3861) | Deadlock in DFSClient | Blocker | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3852](https://issues.apache.org/jira/browse/HDFS-3852) | TestHftpDelegationToken is broken after HADOOP-8225 | Major | hdfs-client, security | Aaron T. Myers | Daryn Sharp | +| [HDFS-3841](https://issues.apache.org/jira/browse/HDFS-3841) | Port HDFS-3835 to branch-0.23 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-3794](https://issues.apache.org/jira/browse/HDFS-3794) | WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header. | Major | webhdfs | Ravi Prakash | Ravi Prakash | +| [HDFS-3788](https://issues.apache.org/jira/browse/HDFS-3788) | distcp can't copy large files using webhdfs due to missing Content-Length header | Critical | webhdfs | Eli Collins | Tsz Wo Nicholas Sze | +| [HDFS-3718](https://issues.apache.org/jira/browse/HDFS-3718) | Datanode won't shutdown because of runaway DataBlockScanner thread | Critical | datanode | Kihwal Lee | Kihwal Lee | +| [HDFS-3696](https://issues.apache.org/jira/browse/HDFS-3696) | Create files with WebHdfsFileSystem goes OOM when file size is big | Critical | webhdfs | Kihwal Lee | Tsz Wo Nicholas Sze | +| [HDFS-3688](https://issues.apache.org/jira/browse/HDFS-3688) | Namenode loses datanode hostname if datanode re-registers | Major | datanode | Jason Lowe | Jason Lowe | +| [HDFS-3646](https://issues.apache.org/jira/browse/HDFS-3646) | LeaseRenewer can hold reference to inactive DFSClient instances forever | Critical | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3622](https://issues.apache.org/jira/browse/HDFS-3622) | Backport HDFS-3541 to branch-0.23 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-3603](https://issues.apache.org/jira/browse/HDFS-3603) | Decouple TestHDFSTrash from TestTrash | Major | test | Jason Lowe | Jason Lowe | +| [HDFS-3597](https://issues.apache.org/jira/browse/HDFS-3597) | SNN can fail to start on upgrade | Minor | namenode | Andy Isaacson | Andy Isaacson | +| [HDFS-3591](https://issues.apache.org/jira/browse/HDFS-3591) | Backport HDFS-3357 to branch-0.23 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-3581](https://issues.apache.org/jira/browse/HDFS-3581) | FSPermissionChecker#checkPermission sticky bit check missing range check | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-3577](https://issues.apache.org/jira/browse/HDFS-3577) | WebHdfsFileSystem can not read files larger than 24KB | Blocker | webhdfs | Alejandro Abdelnur | Tsz Wo Nicholas Sze | +| [HDFS-3486](https://issues.apache.org/jira/browse/HDFS-3486) | offlineimageviewer can't read fsimage files that contain persistent delegation tokens | Minor | security, tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3442](https://issues.apache.org/jira/browse/HDFS-3442) | Incorrect count for Missing Replicas in FSCK report | Minor | . | suja s | Andrew Wang | +| [HDFS-3359](https://issues.apache.org/jira/browse/HDFS-3359) | DFSClient.close should close cached sockets | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-3334](https://issues.apache.org/jira/browse/HDFS-3334) | ByteRangeInputStream leaks streams | Major | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3331](https://issues.apache.org/jira/browse/HDFS-3331) | setBalancerBandwidth do not checkSuperuserPrivilege | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3321](https://issues.apache.org/jira/browse/HDFS-3321) | Error message for insufficient data nodes to come out of safemode is wrong. | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-3312](https://issues.apache.org/jira/browse/HDFS-3312) | Hftp selects wrong token service | Blocker | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3308](https://issues.apache.org/jira/browse/HDFS-3308) | hftp/webhdfs can't get tokens if authority has no port | Critical | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-3177](https://issues.apache.org/jira/browse/HDFS-3177) | Allow DFSClient to find out and use the CRC type being used for a file. | Major | datanode, hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3176](https://issues.apache.org/jira/browse/HDFS-3176) | JsonUtil should not parse the MD5MD5CRC32FileChecksum bytes on its own. | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3166](https://issues.apache.org/jira/browse/HDFS-3166) | Hftp connections do not have a timeout | Critical | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3136](https://issues.apache.org/jira/browse/HDFS-3136) | Multiple SLF4J binding warning | Major | build | Jason Lowe | Jason Lowe | +| [HDFS-3067](https://issues.apache.org/jira/browse/HDFS-3067) | NPE in DFSInputStream.readBuffer if read is repeated on corrupted block | Major | hdfs-client | Henry Robinson | Henry Robinson | +| [HDFS-3037](https://issues.apache.org/jira/browse/HDFS-3037) | TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy | Minor | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2285](https://issues.apache.org/jira/browse/HDFS-2285) | BackupNode should reject requests trying to modify namespace | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [MAPREDUCE-4641](https://issues.apache.org/jira/browse/MAPREDUCE-4641) | Exception in commitJob marks job as successful in job history | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4633](https://issues.apache.org/jira/browse/MAPREDUCE-4633) | history server doesn't set permissions on all subdirs | Critical | jobhistoryserver | Thomas Graves | oss.wakayama | +| [MAPREDUCE-4612](https://issues.apache.org/jira/browse/MAPREDUCE-4612) | job summary file permissions not set when its created | Critical | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4611](https://issues.apache.org/jira/browse/MAPREDUCE-4611) | MR AM dies badly when Node is decomissioned | Critical | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4604](https://issues.apache.org/jira/browse/MAPREDUCE-4604) | In mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker. | Critical | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4600](https://issues.apache.org/jira/browse/MAPREDUCE-4600) | TestTokenCache.java from MRV1 no longer compiles | Critical | . | Robert Joseph Evans | Daryn Sharp | +| [MAPREDUCE-4570](https://issues.apache.org/jira/browse/MAPREDUCE-4570) | ProcfsBasedProcessTree#constructProcessInfo() prints a warning if procfsDir/\/stat is not found. | Minor | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4569](https://issues.apache.org/jira/browse/MAPREDUCE-4569) | TestHsWebServicesJobsQuery fails on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4562](https://issues.apache.org/jira/browse/MAPREDUCE-4562) | Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name | Major | . | Jarek Jarcec Cecho | Jarek Jarcec Cecho | +| [MAPREDUCE-4504](https://issues.apache.org/jira/browse/MAPREDUCE-4504) | SortValidator writes to wrong directory | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4496](https://issues.apache.org/jira/browse/MAPREDUCE-4496) | AM logs link is missing user name | Major | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4493](https://issues.apache.org/jira/browse/MAPREDUCE-4493) | Distibuted Cache Compatability Issues | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4492](https://issues.apache.org/jira/browse/MAPREDUCE-4492) | Configuring total queue capacity between 100.5 and 99.5 at perticular level is sucessfull | Minor | mrv2 | Nishan Shetty | Mayank Bansal | +| [MAPREDUCE-4467](https://issues.apache.org/jira/browse/MAPREDUCE-4467) | IndexCache failures due to missing synchronization | Critical | nodemanager | Andrey Klochkov | Kihwal Lee | +| [MAPREDUCE-4457](https://issues.apache.org/jira/browse/MAPREDUCE-4457) | mr job invalid transition TA\_TOO\_MANY\_FETCH\_FAILURE at FAILED | Critical | mrv2 | Thomas Graves | Robert Joseph Evans | +| [MAPREDUCE-4456](https://issues.apache.org/jira/browse/MAPREDUCE-4456) | LocalDistributedCacheManager can get an ArrayIndexOutOfBounds when creating symlinks | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4449](https://issues.apache.org/jira/browse/MAPREDUCE-4449) | Incorrect MR\_HISTORY\_STORAGE property name in JHAdminConfig | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4448](https://issues.apache.org/jira/browse/MAPREDUCE-4448) | Nodemanager crashes upon application cleanup if aggregation failed to start | Critical | mrv2, nodemanager | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4444](https://issues.apache.org/jira/browse/MAPREDUCE-4444) | nodemanager fails to start when one of the local-dirs is bad | Blocker | nodemanager | Nathan Roberts | Jason Lowe | +| [MAPREDUCE-4437](https://issues.apache.org/jira/browse/MAPREDUCE-4437) | Race in MR ApplicationMaster can cause reducers to never be scheduled | Critical | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4423](https://issues.apache.org/jira/browse/MAPREDUCE-4423) | Potential infinite fetching of map output | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4419](https://issues.apache.org/jira/browse/MAPREDUCE-4419) | ./mapred queue -info \ -showJobs displays all the jobs irrespective of \ | Major | mrv2 | Nishan Shetty | Devaraj K | +| [MAPREDUCE-4402](https://issues.apache.org/jira/browse/MAPREDUCE-4402) | TestFileInputFormat fails intermittently | Major | test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4395](https://issues.apache.org/jira/browse/MAPREDUCE-4395) | Possible NPE at ClientDistributedCacheManager#determineTimestamps | Critical | distributed-cache, job submission, mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4392](https://issues.apache.org/jira/browse/MAPREDUCE-4392) | Counters.makeCompactString() changed behavior from 0.20 | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4387](https://issues.apache.org/jira/browse/MAPREDUCE-4387) | RM gets fatal error and exits during TestRM | Major | resourcemanager | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-4384](https://issues.apache.org/jira/browse/MAPREDUCE-4384) | Race conditions in IndexCache | Major | nodemanager | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-4379](https://issues.apache.org/jira/browse/MAPREDUCE-4379) | Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts | Blocker | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4376](https://issues.apache.org/jira/browse/MAPREDUCE-4376) | TestClusterMRNotification times out | Major | mrv2, test | Jason Lowe | Kihwal Lee | +| [MAPREDUCE-4372](https://issues.apache.org/jira/browse/MAPREDUCE-4372) | Deadlock in Resource Manager between SchedulerEventDispatcher.EventProcessor and Shutdown hook manager | Major | mrv2, resourcemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4361](https://issues.apache.org/jira/browse/MAPREDUCE-4361) | Fix detailed metrics for protobuf-based RPC on 0.23 | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4341](https://issues.apache.org/jira/browse/MAPREDUCE-4341) | add types to capacity scheduler properties documentation | Major | capacity-sched, mrv2 | Thomas Graves | Karthik Kambatla | +| [MAPREDUCE-4320](https://issues.apache.org/jira/browse/MAPREDUCE-4320) | gridmix mainClass wrong in pom.xml | Major | contrib/gridmix | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4307](https://issues.apache.org/jira/browse/MAPREDUCE-4307) | TeraInputFormat calls FileSystem.getDefaultBlockSize() without a Path - Failure when using ViewFileSystem | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4302](https://issues.apache.org/jira/browse/MAPREDUCE-4302) | NM goes down if error encountered during log aggregation | Critical | nodemanager | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4300](https://issues.apache.org/jira/browse/MAPREDUCE-4300) | OOM in AM can turn it into a zombie. | Major | applicationmaster | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4299](https://issues.apache.org/jira/browse/MAPREDUCE-4299) | Terasort hangs with MR2 FifoScheduler | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4297](https://issues.apache.org/jira/browse/MAPREDUCE-4297) | Usersmap file in gridmix should not fail on empty lines | Major | contrib/gridmix | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4295](https://issues.apache.org/jira/browse/MAPREDUCE-4295) | RM crashes due to DNS issue | Critical | mrv2, resourcemanager | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4290](https://issues.apache.org/jira/browse/MAPREDUCE-4290) | JobStatus.getState() API is giving ambiguous values | Major | mrv2 | Nishan Shetty | Devaraj K | +| [MAPREDUCE-4270](https://issues.apache.org/jira/browse/MAPREDUCE-4270) | data\_join test classes are in the wrong packge | Major | mrv2 | Brock Noland | Thomas Graves | +| [MAPREDUCE-4269](https://issues.apache.org/jira/browse/MAPREDUCE-4269) | documentation: Gridmix has javadoc warnings in StressJobFactory | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4267](https://issues.apache.org/jira/browse/MAPREDUCE-4267) | mavenize pipes | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4264](https://issues.apache.org/jira/browse/MAPREDUCE-4264) | Got ClassCastException when using mapreduce.history.server.delegationtoken.required=true | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4252](https://issues.apache.org/jira/browse/MAPREDUCE-4252) | MR2 job never completes with 1 pending task | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4250](https://issues.apache.org/jira/browse/MAPREDUCE-4250) | hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMaster | Major | nodemanager | Patrick Hunt | Patrick Hunt | +| [MAPREDUCE-4238](https://issues.apache.org/jira/browse/MAPREDUCE-4238) | mavenize data\_join | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4237](https://issues.apache.org/jira/browse/MAPREDUCE-4237) | TestNodeStatusUpdater can fail if localhost has a domain associated with it | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4233](https://issues.apache.org/jira/browse/MAPREDUCE-4233) | NPE can happen in RMNMNodeInfo. | Critical | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4228](https://issues.apache.org/jira/browse/MAPREDUCE-4228) | mapreduce.job.reduce.slowstart.completedmaps is not working properly to delay the scheduling of the reduce tasks | Major | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4226](https://issues.apache.org/jira/browse/MAPREDUCE-4226) | ConcurrentModificationException in FileSystemCounterGroup | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4220](https://issues.apache.org/jira/browse/MAPREDUCE-4220) | RM apps page starttime/endtime sorts are incorrect | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4215](https://issues.apache.org/jira/browse/MAPREDUCE-4215) | RM app page shows 500 error on appid parse error | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4211](https://issues.apache.org/jira/browse/MAPREDUCE-4211) | Error conditions (missing appid, appid not found) are masked in the RM app page | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4209](https://issues.apache.org/jira/browse/MAPREDUCE-4209) | junit dependency in hadoop-mapreduce-client is missing scope test | Major | build | Radim Kolar | | +| [MAPREDUCE-4206](https://issues.apache.org/jira/browse/MAPREDUCE-4206) | Sorting by Last Health-Update on the RM nodes page sorts does not work correctly | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4197](https://issues.apache.org/jira/browse/MAPREDUCE-4197) | Include the hsqldb jar in the hadoop-mapreduce tar file | Major | . | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4194](https://issues.apache.org/jira/browse/MAPREDUCE-4194) | ConcurrentModificationError in DirectoryCollection | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4189](https://issues.apache.org/jira/browse/MAPREDUCE-4189) | TestContainerManagerSecurity is failing | Critical | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4169](https://issues.apache.org/jira/browse/MAPREDUCE-4169) | Container Logs appear in unsorted order | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4165](https://issues.apache.org/jira/browse/MAPREDUCE-4165) | Committing is misspelled as commiting in task logs | Trivial | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4160](https://issues.apache.org/jira/browse/MAPREDUCE-4160) | some mrv1 ant tests fail with timeout - due to 4156 | Major | test | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4159](https://issues.apache.org/jira/browse/MAPREDUCE-4159) | Job is running in Uber mode after setting "mapreduce.job.ubertask.maxreduces" to zero | Major | mrv2 | Nishan Shetty | Devaraj K | +| [MAPREDUCE-4156](https://issues.apache.org/jira/browse/MAPREDUCE-4156) | ant build fails compiling JobInProgress | Major | build | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4152](https://issues.apache.org/jira/browse/MAPREDUCE-4152) | map task left hanging after AM dies trying to connect to RM | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4144](https://issues.apache.org/jira/browse/MAPREDUCE-4144) | ResourceManager NPE while handling NODE\_UPDATE | Critical | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4140](https://issues.apache.org/jira/browse/MAPREDUCE-4140) | mapreduce classes incorrectly importing "clover.org.apache.\*" classes | Major | client, mrv2 | Patrick Hunt | Patrick Hunt | +| [MAPREDUCE-4139](https://issues.apache.org/jira/browse/MAPREDUCE-4139) | Potential ResourceManager deadlock when SchedulerEventDispatcher is stopped | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4133](https://issues.apache.org/jira/browse/MAPREDUCE-4133) | MR over viewfs is broken | Major | . | John George | John George | +| [MAPREDUCE-4129](https://issues.apache.org/jira/browse/MAPREDUCE-4129) | Lots of unneeded counters log messages | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4128](https://issues.apache.org/jira/browse/MAPREDUCE-4128) | AM Recovery expects all attempts of a completed task to also be completed. | Major | mrv2 | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4117](https://issues.apache.org/jira/browse/MAPREDUCE-4117) | mapred job -status throws NullPointerException | Critical | client, mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4102](https://issues.apache.org/jira/browse/MAPREDUCE-4102) | job counters not available in Jobhistory webui for killed jobs | Major | webapps | Thomas Graves | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4099](https://issues.apache.org/jira/browse/MAPREDUCE-4099) | ApplicationMaster may fail to remove staging directory | Critical | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4097](https://issues.apache.org/jira/browse/MAPREDUCE-4097) | tools testcases fail because missing mrapp-generated-classpath file in classpath | Major | build | Alejandro Abdelnur | Roman Shaposhnik | +| [MAPREDUCE-4092](https://issues.apache.org/jira/browse/MAPREDUCE-4092) | commitJob Exception does not fail job (regression in 0.23 vs 0.20) | Blocker | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4091](https://issues.apache.org/jira/browse/MAPREDUCE-4091) | tools testcases failing because of MAPREDUCE-4082 | Critical | build, test | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4089](https://issues.apache.org/jira/browse/MAPREDUCE-4089) | Hung Tasks never time out. | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4082](https://issues.apache.org/jira/browse/MAPREDUCE-4082) | hadoop-mapreduce-client-app's mrapp-generated-classpath file should not be in the module JAR | Critical | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4074](https://issues.apache.org/jira/browse/MAPREDUCE-4074) | Client continuously retries to RM When RM goes down before launching Application Master | Major | . | Devaraj K | xieguiming | +| [MAPREDUCE-4073](https://issues.apache.org/jira/browse/MAPREDUCE-4073) | CS assigns multiple off-switch containers when using multi-level-queues | Critical | mrv2, scheduler | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4068](https://issues.apache.org/jira/browse/MAPREDUCE-4068) | Jars in lib subdirectory of the submittable JAR are not added to the classpath | Blocker | mrv2 | Ahmed Radwan | Robert Kanter | +| [MAPREDUCE-4062](https://issues.apache.org/jira/browse/MAPREDUCE-4062) | AM Launcher thread can hang forever | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4060](https://issues.apache.org/jira/browse/MAPREDUCE-4060) | Multiple SLF4J binding warning | Major | build | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4053](https://issues.apache.org/jira/browse/MAPREDUCE-4053) | Counters group names deprecation is wrong, iterating over group names deprecated names don't show up | Major | mrv2 | Alejandro Abdelnur | Robert Joseph Evans | +| [MAPREDUCE-4050](https://issues.apache.org/jira/browse/MAPREDUCE-4050) | Invalid node link | Major | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4048](https://issues.apache.org/jira/browse/MAPREDUCE-4048) | NullPointerException exception while accessing the Application Master UI | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4040](https://issues.apache.org/jira/browse/MAPREDUCE-4040) | History links should use hostname rather than IP address. | Minor | jobhistoryserver, mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4031](https://issues.apache.org/jira/browse/MAPREDUCE-4031) | Node Manager hangs on shut down | Critical | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4024](https://issues.apache.org/jira/browse/MAPREDUCE-4024) | RM webservices can't query on finalStatus | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4020](https://issues.apache.org/jira/browse/MAPREDUCE-4020) | Web services returns incorrect JSON for deep queue tree | Major | mrv2, webapps | Jason Lowe | Anupam Seth | +| [MAPREDUCE-4012](https://issues.apache.org/jira/browse/MAPREDUCE-4012) | Hadoop Job setup error leaves no useful info to users (when LinuxTaskController is used) | Minor | . | Koji Noguchi | Thomas Graves | +| [MAPREDUCE-4010](https://issues.apache.org/jira/browse/MAPREDUCE-4010) | TestWritableJobConf fails on trunk | Critical | mrv2 | Jason Lowe | Alejandro Abdelnur | +| [MAPREDUCE-3999](https://issues.apache.org/jira/browse/MAPREDUCE-3999) | Tracking link gives an error if the AppMaster hasn't started yet | Major | mrv2, webapps | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3992](https://issues.apache.org/jira/browse/MAPREDUCE-3992) | Reduce fetcher doesn't verify HTTP status code of response | Major | mrv1 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3988](https://issues.apache.org/jira/browse/MAPREDUCE-3988) | mapreduce.job.local.dir doesn't point to a single directory on a node. | Major | mrv2 | Vinod Kumar Vavilapalli | Eric Payne | +| [MAPREDUCE-3947](https://issues.apache.org/jira/browse/MAPREDUCE-3947) | yarn.app.mapreduce.am.resource.mb not documented | Minor | . | Todd Lipcon | Devaraj K | +| [MAPREDUCE-3932](https://issues.apache.org/jira/browse/MAPREDUCE-3932) | MR tasks failing and crashing the AM when available-resources/headRoom becomes zero | Critical | mr-am, mrv2 | Vinod Kumar Vavilapalli | Robert Joseph Evans | +| [MAPREDUCE-3927](https://issues.apache.org/jira/browse/MAPREDUCE-3927) | Shuffle hang when set map.failures.percent | Critical | mrv2 | MengWang | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3893](https://issues.apache.org/jira/browse/MAPREDUCE-3893) | allow capacity scheduler configs maximum-applications and maximum-am-resource-percent configurable on a per queue basis | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3889](https://issues.apache.org/jira/browse/MAPREDUCE-3889) | job client tries to use /tasklog interface, but that doesn't exist anymore | Critical | mrv2 | Thomas Graves | Devaraj K | +| [MAPREDUCE-3870](https://issues.apache.org/jira/browse/MAPREDUCE-3870) | Invalid App Metrics | Major | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3782](https://issues.apache.org/jira/browse/MAPREDUCE-3782) | teragen terasort jobs fail when using webhdfs:// | Critical | mrv2 | Arpit Gupta | Jason Lowe | +| [MAPREDUCE-3728](https://issues.apache.org/jira/browse/MAPREDUCE-3728) | ShuffleHandler can't access results when configured in a secure mode | Critical | mrv2, nodemanager | Roman Shaposhnik | Ding Yuan | +| [MAPREDUCE-3682](https://issues.apache.org/jira/browse/MAPREDUCE-3682) | Tracker URL says AM tasks run on localhost | Major | mrv2 | David Capwell | Ravi Prakash | +| [MAPREDUCE-3672](https://issues.apache.org/jira/browse/MAPREDUCE-3672) | Killed maps shouldn't be counted towards JobCounter.NUM\_FAILED\_MAPS | Major | mr-am, mrv2 | Vinod Kumar Vavilapalli | Anupam Seth | +| [MAPREDUCE-3650](https://issues.apache.org/jira/browse/MAPREDUCE-3650) | testGetTokensForHftpFS() fails | Blocker | mrv2 | Thomas Graves | Ravi Prakash | +| [MAPREDUCE-3621](https://issues.apache.org/jira/browse/MAPREDUCE-3621) | TestDBJob and TestDataDrivenDBInputFormat ant tests fail | Major | mrv2 | Thomas Graves | Ravi Prakash | +| [MAPREDUCE-3543](https://issues.apache.org/jira/browse/MAPREDUCE-3543) | Mavenize Gridmix. | Critical | mrv2 | Mahadev konar | Thomas Graves | +| [MAPREDUCE-3506](https://issues.apache.org/jira/browse/MAPREDUCE-3506) | Calling getPriority on JobInfo after parsing a history log with JobHistoryParser throws a NullPointerException | Minor | client, mrv2 | Ratandeep Ratti | Jason Lowe | +| [MAPREDUCE-3350](https://issues.apache.org/jira/browse/MAPREDUCE-3350) | Per-app RM page should have the list of application-attempts like on the app JHS page | Critical | mrv2, webapps | Vinod Kumar Vavilapalli | Jonathan Eagles | +| [MAPREDUCE-3348](https://issues.apache.org/jira/browse/MAPREDUCE-3348) | mapred job -status fails to give info even if the job is present in History | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3082](https://issues.apache.org/jira/browse/MAPREDUCE-3082) | archive command take wrong path for input file with current directory | Major | harchive | Rajit Saha | John George | +| [MAPREDUCE-2374](https://issues.apache.org/jira/browse/MAPREDUCE-2374) | "Text File Busy" errors launching MR tasks | Major | . | Todd Lipcon | Andy Isaacson | +| [MAPREDUCE-2289](https://issues.apache.org/jira/browse/MAPREDUCE-2289) | Permissions race can make getStagingDir fail on local filesystem | Major | job submission | Todd Lipcon | Ahmed Radwan | +| [YARN-87](https://issues.apache.org/jira/browse/YARN-87) | NM ResourceLocalizationService does not set permissions of local cache directories | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-68](https://issues.apache.org/jira/browse/YARN-68) | NodeManager will refuse to shutdown indefinitely due to container log aggregation | Major | nodemanager | patrick white | Daryn Sharp | +| [YARN-66](https://issues.apache.org/jira/browse/YARN-66) | aggregated logs permissions not set properly | Critical | nodemanager | Thomas Graves | Thomas Graves | +| [YARN-63](https://issues.apache.org/jira/browse/YARN-63) | RMNodeImpl is missing valid transitions from the UNHEALTHY state | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-58](https://issues.apache.org/jira/browse/YARN-58) | NM leaks filesystems | Critical | nodemanager | Daryn Sharp | Jason Lowe | +| [YARN-31](https://issues.apache.org/jira/browse/YARN-31) | TestDelegationTokenRenewer fails on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [YARN-27](https://issues.apache.org/jira/browse/YARN-27) | Failed refreshQueues due to misconfiguration prevents further refreshing of queues | Major | . | Ramya Sunil | Arun C Murthy | +| [YARN-25](https://issues.apache.org/jira/browse/YARN-25) | remove old aggregated logs | Major | . | Thomas Graves | Robert Joseph Evans | +| [YARN-14](https://issues.apache.org/jira/browse/YARN-14) | Symlinks to peer distributed cache files no longer work | Major | nodemanager | Jason Lowe | Jason Lowe | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8283](https://issues.apache.org/jira/browse/HADOOP-8283) | Allow tests to control token service value | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-3258](https://issues.apache.org/jira/browse/HDFS-3258) | Test for HADOOP-8144 (pseudoSortByDistance in NetworkTopology for first rack local node) | Major | test | Eli Collins | Junping Du | +| [HDFS-2505](https://issues.apache.org/jira/browse/HDFS-2505) | Add a test to verify getFileChecksum works with ViewFS | Minor | test | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4212](https://issues.apache.org/jira/browse/MAPREDUCE-4212) | TestJobClientGetJob sometimes fails | Major | test | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3983](https://issues.apache.org/jira/browse/MAPREDUCE-3983) | TestTTResourceReporting can fail, and should just be deleted | Major | mrv1 | Robert Joseph Evans | Ravi Prakash | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-3168](https://issues.apache.org/jira/browse/HDFS-3168) | Clean up FSNamesystem and BlockManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-4163](https://issues.apache.org/jira/browse/MAPREDUCE-4163) | consistently set the bind address | Major | mrv2 | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4162](https://issues.apache.org/jira/browse/MAPREDUCE-4162) | Correctly set token service | Major | client, mrv2 | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4161](https://issues.apache.org/jira/browse/MAPREDUCE-4161) | create sockets consistently | Major | client, mrv2 | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3972](https://issues.apache.org/jira/browse/MAPREDUCE-3972) | Locking and exception issues in JobHistory Server. | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3942](https://issues.apache.org/jira/browse/MAPREDUCE-3942) | Randomize master key generation for ApplicationTokenSecretManager and roll it every so often | Major | mrv2, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3940](https://issues.apache.org/jira/browse/MAPREDUCE-3940) | ContainerTokens should have an expiry interval | Major | mrv2, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3613](https://issues.apache.org/jira/browse/MAPREDUCE-3613) | web service calls header contains 2 content types | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [YARN-60](https://issues.apache.org/jira/browse/YARN-60) | NMs rejects all container tokens after secret key rolls | Blocker | nodemanager | Daryn Sharp | Vinod Kumar Vavilapalli | +| [YARN-39](https://issues.apache.org/jira/browse/YARN-39) | RM-NM secret-keys should be randomly generated and rolled every so often | Critical | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-4134](https://issues.apache.org/jira/browse/MAPREDUCE-4134) | Remove references of mapred.child.ulimit etc. since they are not being used any more | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4051](https://issues.apache.org/jira/browse/MAPREDUCE-4051) | Remove the empty hadoop-mapreduce-project/assembly/all.xml file | Major | . | Ravi Prakash | Ravi Prakash | +| [YARN-1](https://issues.apache.org/jira/browse/YARN-1) | Move YARN out of hadoop-mapreduce | Major | . | Arun C Murthy | Arun C Murthy | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/RELEASENOTES.0.23.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/RELEASENOTES.0.23.3.md new file mode 100644 index 00000000000..b003778a7bc --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.3/RELEASENOTES.0.23.3.md @@ -0,0 +1,115 @@ + + +# Apache Hadoop 0.23.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8703](https://issues.apache.org/jira/browse/HADOOP-8703) | *Major* | **distcpV2: turn CRC checking off for 0 byte size** + +distcp skips CRC on 0 byte files. + + +--- + +* [HADOOP-8551](https://issues.apache.org/jira/browse/HADOOP-8551) | *Major* | **fs -mkdir creates parent directories without the -p option** + +FsShell's "mkdir" no longer implicitly creates all non-existent parent directories. The command adopts the posix compliant behavior of requiring the "-p" flag to auto-create parent directories. + + +--- + +* [HADOOP-8327](https://issues.apache.org/jira/browse/HADOOP-8327) | *Major* | **distcpv2 and distcpv1 jars should not coexist** + +Resolve sporadic distcp issue due to having two DistCp classes (v1 & v2) in the classpath. + + +--- + +* [HDFS-3318](https://issues.apache.org/jira/browse/HDFS-3318) | *Blocker* | **Hftp hangs on transfers \>2GB** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4311](https://issues.apache.org/jira/browse/MAPREDUCE-4311) | *Major* | **Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4072](https://issues.apache.org/jira/browse/MAPREDUCE-4072) | *Major* | **User set java.library.path seems to overwrite default creating problems native lib loading** + +-Djava.library.path in mapred.child.java.opts can cause issues with native libraries. LD\_LIBRARY\_PATH through mapred.child.env should be used instead. + + +--- + +* [MAPREDUCE-4017](https://issues.apache.org/jira/browse/MAPREDUCE-4017) | *Trivial* | **Add jobname to jobsummary log** + +The Job Summary log may contain commas in values that are escaped by a '\' character. This was true before, but is more likely to be exposed now. + + +--- + +* [MAPREDUCE-3940](https://issues.apache.org/jira/browse/MAPREDUCE-3940) | *Major* | **ContainerTokens should have an expiry interval** + +ContainerTokens now have an expiry interval so that stale tokens cannot be used for launching containers. + + +--- + +* [MAPREDUCE-3812](https://issues.apache.org/jira/browse/MAPREDUCE-3812) | *Major* | **Lower default allocation sizes, fix allocation configurations and document them** + +Removes two sets of previously available config properties: + +1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and, +2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb ) + +In favor of two new, generically named properties: + +1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers. +2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers. + +Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides. + +Also changes the default minimum and maximums to 128 MB and 10 GB respectively. + + +--- + +* [MAPREDUCE-3543](https://issues.apache.org/jira/browse/MAPREDUCE-3543) | *Critical* | **Mavenize Gridmix.** + +Note that to apply this you should first run the script - ./MAPREDUCE-3543v3.sh svn, then apply the patch. + +If this is merged to more then trunk, the version inside of hadoop-tools/hadoop-gridmix/pom.xml will need to be udpated accordingly. + + +--- + +* [MAPREDUCE-3348](https://issues.apache.org/jira/browse/MAPREDUCE-3348) | *Major* | **mapred job -status fails to give info even if the job is present in History** + +Fixed a bug in MR client to redirect to JobHistoryServer correctly when RM forgets the app. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/CHANGES.0.23.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/CHANGES.0.23.4.md new file mode 100644 index 00000000000..eeba2cbbf4a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/CHANGES.0.23.4.md @@ -0,0 +1,96 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.4 - 2012-10-15 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7788](https://issues.apache.org/jira/browse/HADOOP-7788) | HA: Simple HealthMonitor class to watch an HAService | Major | ha | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-4651](https://issues.apache.org/jira/browse/MAPREDUCE-4651) | Benchmarking random reads with DFSIO | Major | benchmarks, test | Konstantin Shvachko | Konstantin Shvachko | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8623](https://issues.apache.org/jira/browse/HADOOP-8623) | hadoop jar command should respect HADOOP\_OPTS | Minor | scripts | Steven Willis | Steven Willis | +| [HADOOP-8183](https://issues.apache.org/jira/browse/HADOOP-8183) | Stop using "mapred.used.genericoptionsparser" to avoid unnecessary warnings | Minor | util | Harsh J | Harsh J | +| [MAPREDUCE-4645](https://issues.apache.org/jira/browse/MAPREDUCE-4645) | Providing a random seed to Slive should make the sequence of filenames completely deterministic | Major | performance, test | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4408](https://issues.apache.org/jira/browse/MAPREDUCE-4408) | allow jobs to set a JAR that is in the distributed cached | Major | mrv1, mrv2 | Alejandro Abdelnur | Robert Kanter | +| [MAPREDUCE-2786](https://issues.apache.org/jira/browse/MAPREDUCE-2786) | TestDFSIO should also test compression reading/writing from command-line. | Minor | benchmarks | Plamen Jeliazkov | Plamen Jeliazkov | +| [YARN-137](https://issues.apache.org/jira/browse/YARN-137) | Change the default scheduler to the CapacityScheduler | Major | scheduler | Siddharth Seth | Siddharth Seth | +| [YARN-57](https://issues.apache.org/jira/browse/YARN-57) | Plugable process tree | Major | nodemanager | Radim Kolar | Radim Kolar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8843](https://issues.apache.org/jira/browse/HADOOP-8843) | Old trash directories are never deleted on upgrade from 1.x | Critical | . | Robert Joseph Evans | Jason Lowe | +| [HADOOP-8822](https://issues.apache.org/jira/browse/HADOOP-8822) | relnotes.py was deleted post mavenization | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8684](https://issues.apache.org/jira/browse/HADOOP-8684) | Deadlock between WritableComparator and WritableComparable | Minor | io | Hiroshi Ikeda | Jing Zhao | +| [HADOOP-8310](https://issues.apache.org/jira/browse/HADOOP-8310) | FileContext#checkPath should handle URIs with no port | Major | fs | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3922](https://issues.apache.org/jira/browse/HDFS-3922) | 0.22 and 0.23 namenode throws away blocks under construction on restart | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-3860](https://issues.apache.org/jira/browse/HDFS-3860) | HeartbeatManager#Monitor may wrongly hold the writelock of namesystem | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-3831](https://issues.apache.org/jira/browse/HDFS-3831) | Failure to renew tokens due to test-sources left in classpath | Critical | security | Jason Lowe | Jason Lowe | +| [HDFS-3731](https://issues.apache.org/jira/browse/HDFS-3731) | 2.0 release upgrade must handle blocks being written from 1.0 | Blocker | datanode | Suresh Srinivas | Kihwal Lee | +| [HDFS-3626](https://issues.apache.org/jira/browse/HDFS-3626) | Creating file with invalid path can corrupt edit log | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3553](https://issues.apache.org/jira/browse/HDFS-3553) | Hftp proxy tokens are broken | Blocker | . | Daryn Sharp | Daryn Sharp | +| [HDFS-3373](https://issues.apache.org/jira/browse/HDFS-3373) | FileContext HDFS implementation can leak socket caches | Major | hdfs-client | Todd Lipcon | John George | +| [MAPREDUCE-4691](https://issues.apache.org/jira/browse/MAPREDUCE-4691) | Historyserver can report "Unknown job" after RM says job has completed | Critical | jobhistoryserver, mrv2 | Jason Lowe | Robert Joseph Evans | +| [MAPREDUCE-4689](https://issues.apache.org/jira/browse/MAPREDUCE-4689) | JobClient.getMapTaskReports on failed job results in NPE | Major | client | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4647](https://issues.apache.org/jira/browse/MAPREDUCE-4647) | We should only unjar jobjar if there is a lib directory in it. | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4646](https://issues.apache.org/jira/browse/MAPREDUCE-4646) | client does not receive job diagnostics for failed jobs | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4193](https://issues.apache.org/jira/browse/MAPREDUCE-4193) | broken doc link for yarn-default.xml in site.xml | Major | documentation | Patrick Hunt | Patrick Hunt | +| [YARN-138](https://issues.apache.org/jira/browse/YARN-138) | Improve default config values for YARN | Major | resourcemanager, scheduler | Arun C Murthy | Harsh J | +| [YARN-108](https://issues.apache.org/jira/browse/YARN-108) | FSDownload can create cache directories with the wrong permissions | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-106](https://issues.apache.org/jira/browse/YARN-106) | Nodemanager needs to set permissions of local directories | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-93](https://issues.apache.org/jira/browse/YARN-93) | Diagnostics missing from applications that have finished but failed | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-88](https://issues.apache.org/jira/browse/YARN-88) | DefaultContainerExecutor can fail to set proper permissions | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-75](https://issues.apache.org/jira/browse/YARN-75) | RMContainer should handle a RELEASE event while RUNNING | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-42](https://issues.apache.org/jira/browse/YARN-42) | Node Manager throws NPE on startup | Major | nodemanager | Devaraj K | Devaraj K | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/RELEASENOTES.0.23.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/RELEASENOTES.0.23.4.md new file mode 100644 index 00000000000..ec7073dd14e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.4/RELEASENOTES.0.23.4.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.23.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/CHANGES.0.23.5.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/CHANGES.0.23.5.md new file mode 100644 index 00000000000..56568ab8fb4 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/CHANGES.0.23.5.md @@ -0,0 +1,164 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.5 - 2012-11-29 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-4080](https://issues.apache.org/jira/browse/HDFS-4080) | Add a separate logger for block state change logs to enable turning off those logs | Major | namenode | Kihwal Lee | Kihwal Lee | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8932](https://issues.apache.org/jira/browse/HADOOP-8932) | JNI-based user-group mapping modules can be too chatty on lookup failures | Major | security | Kihwal Lee | Kihwal Lee | +| [HADOOP-8930](https://issues.apache.org/jira/browse/HADOOP-8930) | Cumulative code coverage calculation | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-8926](https://issues.apache.org/jira/browse/HADOOP-8926) | hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data | Major | util | Gopal V | Gopal V | +| [HADOOP-8889](https://issues.apache.org/jira/browse/HADOOP-8889) | Upgrade to Surefire 2.12.3 | Major | build, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-8851](https://issues.apache.org/jira/browse/HADOOP-8851) | Use -XX:+HeapDumpOnOutOfMemoryError JVM option in the forked tests | Minor | test | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8789](https://issues.apache.org/jira/browse/HADOOP-8789) | Tests setLevel(Level.OFF) should be Level.ERROR | Minor | test | Andy Isaacson | Andy Isaacson | +| [HADOOP-8755](https://issues.apache.org/jira/browse/HADOOP-8755) | Print thread dump when tests fail due to timeout | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HDFS-3483](https://issues.apache.org/jira/browse/HDFS-3483) | Better error message when hdfs fsck is run against a ViewFS config | Major | . | Stephen Chu | Stephen Fritz | +| [MAPREDUCE-4802](https://issues.apache.org/jira/browse/MAPREDUCE-4802) | Takes a long time to load the task list on the AM for large jobs | Major | mr-am, mrv2, webapps | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4763](https://issues.apache.org/jira/browse/MAPREDUCE-4763) | repair test org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken | Minor | . | Ivan A. Veselovsky | | +| [MAPREDUCE-4752](https://issues.apache.org/jira/browse/MAPREDUCE-4752) | Reduce MR AM memory usage through String Interning | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4666](https://issues.apache.org/jira/browse/MAPREDUCE-4666) | JVM metrics for history server | Minor | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4517](https://issues.apache.org/jira/browse/MAPREDUCE-4517) | Too many INFO messages written out during AM to RM heartbeat | Minor | applicationmaster | James Kinley | Jason Lowe | +| [MAPREDUCE-4229](https://issues.apache.org/jira/browse/MAPREDUCE-4229) | Counter names' memory usage can be decreased by interning | Major | jobtracker | Todd Lipcon | Miomir Boljanovic | +| [YARN-216](https://issues.apache.org/jira/browse/YARN-216) | Remove jquery theming support | Major | . | Todd Lipcon | Robert Joseph Evans | +| [YARN-165](https://issues.apache.org/jira/browse/YARN-165) | RM should point tracking URL to RM web page for app when AM fails | Blocker | resourcemanager | Jason Lowe | Jason Lowe | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9025](https://issues.apache.org/jira/browse/HADOOP-9025) | org.apache.hadoop.tools.TestCopyListing failing | Major | . | Robert Joseph Evans | Jonathan Eagles | +| [HADOOP-9022](https://issues.apache.org/jira/browse/HADOOP-9022) | Hadoop distcp tool fails to copy file if -m 0 specified | Major | . | Haiyang Jiang | Jonathan Eagles | +| [HADOOP-8986](https://issues.apache.org/jira/browse/HADOOP-8986) | Server$Call object is never released after it is sent | Critical | ipc | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8962](https://issues.apache.org/jira/browse/HADOOP-8962) | RawLocalFileSystem.listStatus fails when a child filename contains a colon | Critical | fs | Jason Lowe | Jason Lowe | +| [HADOOP-8906](https://issues.apache.org/jira/browse/HADOOP-8906) | paths with multiple globs are unreliable | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8819](https://issues.apache.org/jira/browse/HADOOP-8819) | Should use && instead of & in a few places in FTPFileSystem,FTPInputStream,S3InputStream,ViewFileSystem,ViewFs | Major | fs | Brandon Li | Brandon Li | +| [HADOOP-8791](https://issues.apache.org/jira/browse/HADOOP-8791) | rm "Only deletes non empty directory and files." | Major | documentation | Bertrand Dechoux | Jing Zhao | +| [HADOOP-8775](https://issues.apache.org/jira/browse/HADOOP-8775) | MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete | Major | . | Sandy Ryza | Sandy Ryza | +| [HADOOP-8386](https://issues.apache.org/jira/browse/HADOOP-8386) | hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu) | Major | scripts | Christopher Berner | Christopher Berner | +| [HDFS-4186](https://issues.apache.org/jira/browse/HDFS-4186) | logSync() is called with the write lock held while releasing lease | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4182](https://issues.apache.org/jira/browse/HDFS-4182) | SecondaryNameNode leaks NameCache entries | Critical | namenode | Todd Lipcon | Robert Joseph Evans | +| [HDFS-4181](https://issues.apache.org/jira/browse/HDFS-4181) | LeaseManager tries to double remove and prints extra messages | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4172](https://issues.apache.org/jira/browse/HDFS-4172) | namenode does not URI-encode parameters when building URI for datanode request | Minor | namenode | Derek Dagit | Derek Dagit | +| [HDFS-4162](https://issues.apache.org/jira/browse/HDFS-4162) | Some malformed and unquoted HTML strings are returned from datanode web ui | Minor | datanode | Derek Dagit | Derek Dagit | +| [HDFS-4090](https://issues.apache.org/jira/browse/HDFS-4090) | getFileChecksum() result incompatible when called against zero-byte files. | Critical | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-4075](https://issues.apache.org/jira/browse/HDFS-4075) | Reduce recommissioning overhead | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4016](https://issues.apache.org/jira/browse/HDFS-4016) | back-port HDFS-3582 to branch-0.23 | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HDFS-3996](https://issues.apache.org/jira/browse/HDFS-3996) | Add debug log removed in HDFS-3873 back | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3990](https://issues.apache.org/jira/browse/HDFS-3990) | NN's health report has severe performance problems | Critical | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-3919](https://issues.apache.org/jira/browse/HDFS-3919) | MiniDFSCluster:waitClusterUp can hang forever | Minor | test | Andy Isaacson | Andy Isaacson | +| [HDFS-3905](https://issues.apache.org/jira/browse/HDFS-3905) | Secure cluster cannot use hftp to an insecure cluster | Critical | hdfs-client, security | Daryn Sharp | Daryn Sharp | +| [HDFS-3829](https://issues.apache.org/jira/browse/HDFS-3829) | TestHftpURLTimeouts fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HDFS-3824](https://issues.apache.org/jira/browse/HDFS-3824) | TestHftpDelegationToken fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HDFS-3224](https://issues.apache.org/jira/browse/HDFS-3224) | Bug in check for DN re-registration with different storage ID | Minor | . | Eli Collins | Jason Lowe | +| [MAPREDUCE-4801](https://issues.apache.org/jira/browse/MAPREDUCE-4801) | ShuffleHandler can generate large logs due to prematurely closed channels | Critical | . | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4797](https://issues.apache.org/jira/browse/MAPREDUCE-4797) | LocalContainerAllocator can loop forever trying to contact the RM | Major | applicationmaster | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4787](https://issues.apache.org/jira/browse/MAPREDUCE-4787) | TestJobMonitorAndPrint is broken | Major | test | Ravi Prakash | Robert Parker | +| [MAPREDUCE-4786](https://issues.apache.org/jira/browse/MAPREDUCE-4786) | Job End Notification retry interval is 5 milliseconds by default | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4782](https://issues.apache.org/jira/browse/MAPREDUCE-4782) | NLineInputFormat skips first line of last InputSplit | Blocker | client | Mark Fuhs | Mark Fuhs | +| [MAPREDUCE-4774](https://issues.apache.org/jira/browse/MAPREDUCE-4774) | JobImpl does not handle asynchronous task events in FAILED state | Major | applicationmaster, mrv2 | Ivan A. Veselovsky | Jason Lowe | +| [MAPREDUCE-4772](https://issues.apache.org/jira/browse/MAPREDUCE-4772) | Fetch failures can take way too long for a map to be restarted | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4771](https://issues.apache.org/jira/browse/MAPREDUCE-4771) | KeyFieldBasedPartitioner not partitioning properly when configured | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4751](https://issues.apache.org/jira/browse/MAPREDUCE-4751) | AM stuck in KILL\_WAIT for days | Major | . | Ravi Prakash | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4748](https://issues.apache.org/jira/browse/MAPREDUCE-4748) | Invalid event: T\_ATTEMPT\_SUCCEEDED at SUCCEEDED | Blocker | mrv2 | Robert Joseph Evans | Jason Lowe | +| [MAPREDUCE-4746](https://issues.apache.org/jira/browse/MAPREDUCE-4746) | The MR Application Master does not have a config to set environment variables | Major | applicationmaster | Robert Parker | Robert Parker | +| [MAPREDUCE-4741](https://issues.apache.org/jira/browse/MAPREDUCE-4741) | WARN and ERROR messages logged during normal AM shutdown | Minor | applicationmaster, mrv2 | Jason Lowe | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4740](https://issues.apache.org/jira/browse/MAPREDUCE-4740) | only .jars can be added to the Distributed Cache classpath | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4733](https://issues.apache.org/jira/browse/MAPREDUCE-4733) | Reducer can fail to make progress during shuffle if too many reducers complete consecutively | Major | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4730](https://issues.apache.org/jira/browse/MAPREDUCE-4730) | AM crashes due to OOM while serving up map task completion events | Blocker | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4729](https://issues.apache.org/jira/browse/MAPREDUCE-4729) | job history UI not showing all job attempts | Major | jobhistoryserver | Thomas Graves | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4724](https://issues.apache.org/jira/browse/MAPREDUCE-4724) | job history web ui applications page should be sorted to display last app first | Major | jobhistoryserver | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4721](https://issues.apache.org/jira/browse/MAPREDUCE-4721) | Task startup time in JHS is same as job startup time. | Major | jobhistoryserver | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4720](https://issues.apache.org/jira/browse/MAPREDUCE-4720) | Browser thinks History Server main page JS is taking too long | Major | . | Robert Joseph Evans | Ravi Prakash | +| [MAPREDUCE-4705](https://issues.apache.org/jira/browse/MAPREDUCE-4705) | Historyserver links expire before the history data does | Critical | jobhistoryserver, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4674](https://issues.apache.org/jira/browse/MAPREDUCE-4674) | Hadoop examples secondarysort has a typo "secondarysrot" in the usage | Minor | . | Robert Justice | Robert Justice | +| [MAPREDUCE-4554](https://issues.apache.org/jira/browse/MAPREDUCE-4554) | Job Credentials are not transmitted if security is turned off | Major | job submission, security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4549](https://issues.apache.org/jira/browse/MAPREDUCE-4549) | Distributed cache conflicts breaks backwards compatability | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4521](https://issues.apache.org/jira/browse/MAPREDUCE-4521) | mapreduce.user.classpath.first incompatibility with 0.20/1.x | Major | mrv2 | Jason Lowe | Ravi Prakash | +| [MAPREDUCE-4479](https://issues.apache.org/jira/browse/MAPREDUCE-4479) | Fix parameter order in assertEquals() in TestCombineInputFileFormat.java | Major | test | Mariappan Asokan | Mariappan Asokan | +| [MAPREDUCE-4425](https://issues.apache.org/jira/browse/MAPREDUCE-4425) | Speculation + Fetch failures can lead to a hung job | Critical | mrv2 | Siddharth Seth | Jason Lowe | +| [MAPREDUCE-1806](https://issues.apache.org/jira/browse/MAPREDUCE-1806) | CombineFileInputFormat does not work with paths not on default FS | Major | harchive | Paul Yang | Gera Shegalov | +| [YARN-214](https://issues.apache.org/jira/browse/YARN-214) | RMContainerImpl does not handle event EXPIRE at state RUNNING | Major | resourcemanager | Jason Lowe | Jonathan Eagles | +| [YARN-212](https://issues.apache.org/jira/browse/YARN-212) | NM state machine ignores an APPLICATION\_CONTAINER\_FINISHED event when it shouldn't | Blocker | nodemanager | Nathan Roberts | Nathan Roberts | +| [YARN-206](https://issues.apache.org/jira/browse/YARN-206) | TestApplicationCleanup.testContainerCleanup occasionally fails | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-201](https://issues.apache.org/jira/browse/YARN-201) | CapacityScheduler can take a very long time to schedule containers if requests are off cluster | Critical | capacityscheduler | Jason Lowe | Jason Lowe | +| [YARN-189](https://issues.apache.org/jira/browse/YARN-189) | deadlock in RM - AMResponse object | Blocker | resourcemanager | Thomas Graves | Thomas Graves | +| [YARN-180](https://issues.apache.org/jira/browse/YARN-180) | Capacity scheduler - containers that get reserved create container token to early | Critical | capacityscheduler | Thomas Graves | Arun C Murthy | +| [YARN-178](https://issues.apache.org/jira/browse/YARN-178) | Fix custom ProcessTree instance creation | Critical | . | Radim Kolar | Radim Kolar | +| [YARN-177](https://issues.apache.org/jira/browse/YARN-177) | CapacityScheduler - adding a queue while the RM is running has wacky results | Critical | capacityscheduler | Thomas Graves | Arun C Murthy | +| [YARN-174](https://issues.apache.org/jira/browse/YARN-174) | TestNodeStatusUpdater is failing in trunk | Major | nodemanager | Robert Joseph Evans | Vinod Kumar Vavilapalli | +| [YARN-166](https://issues.apache.org/jira/browse/YARN-166) | capacity scheduler doesn't allow capacity \< 1.0 | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-163](https://issues.apache.org/jira/browse/YARN-163) | Retrieving container log via NM webapp can hang with multibyte characters in log | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-161](https://issues.apache.org/jira/browse/YARN-161) | Yarn Common has multiple compiler warnings for unchecked operations | Major | api | Chris Nauroth | Chris Nauroth | +| [YARN-159](https://issues.apache.org/jira/browse/YARN-159) | RM web ui applications page should be sorted to display last app first | Major | resourcemanager | Thomas Graves | Thomas Graves | +| [YARN-151](https://issues.apache.org/jira/browse/YARN-151) | Browser thinks RM main page JS is taking too long | Major | . | Robert Joseph Evans | Ravi Prakash | +| [YARN-144](https://issues.apache.org/jira/browse/YARN-144) | MiniMRYarnCluster launches RM and JHS on default ports | Major | . | Robert Parker | Robert Parker | +| [YARN-139](https://issues.apache.org/jira/browse/YARN-139) | Interrupted Exception within AsyncDispatcher leads to user confusion | Major | api | Nathan Roberts | Vinod Kumar Vavilapalli | +| [YARN-131](https://issues.apache.org/jira/browse/YARN-131) | Incorrect ACL properties in capacity scheduler documentation | Major | capacityscheduler | Ahmed Radwan | Ahmed Radwan | +| [YARN-116](https://issues.apache.org/jira/browse/YARN-116) | RM is missing ability to add include/exclude files without a restart | Major | resourcemanager | xieguiming | xieguiming | +| [YARN-102](https://issues.apache.org/jira/browse/YARN-102) | Move the apache licence header to the top of the file in MemStore.java | Trivial | resourcemanager | Devaraj K | Devaraj K | +| [YARN-43](https://issues.apache.org/jira/browse/YARN-43) | TestResourceTrackerService fail intermittently on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [YARN-32](https://issues.apache.org/jira/browse/YARN-32) | TestApplicationTokens fails intermintently on jdk7 | Major | . | Thomas Graves | Vinod Kumar Vavilapalli | +| [YARN-30](https://issues.apache.org/jira/browse/YARN-30) | TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [YARN-28](https://issues.apache.org/jira/browse/YARN-28) | TestCompositeService fails on jdk7 | Major | . | Thomas Graves | Thomas Graves | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-4107](https://issues.apache.org/jira/browse/MAPREDUCE-4107) | Fix tests in org.apache.hadoop.ipc.TestSocketFactory | Major | mrv2 | Devaraj K | Devaraj K | +| [YARN-219](https://issues.apache.org/jira/browse/YARN-219) | NM should aggregate logs when application finishes. | Critical | nodemanager | Robert Joseph Evans | Robert Joseph Evans | +| [YARN-202](https://issues.apache.org/jira/browse/YARN-202) | Log Aggregation generates a storm of fsync() for namenode | Critical | . | Kihwal Lee | Kihwal Lee | +| [YARN-188](https://issues.apache.org/jira/browse/YARN-188) | Coverage fixing for CapacityScheduler | Major | capacityscheduler | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-186](https://issues.apache.org/jira/browse/YARN-186) | Coverage fixing LinuxContainerExecutor | Major | resourcemanager, scheduler | Aleksey Gorshkov | Aleksey Gorshkov | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8860](https://issues.apache.org/jira/browse/HADOOP-8860) | Split MapReduce and YARN sections in documentation navigation | Major | documentation | Tom White | Tom White | +| [MAPREDUCE-4596](https://issues.apache.org/jira/browse/MAPREDUCE-4596) | Split StateMachine state from states seen by MRClientProtocol (for Job, Task, TaskAttempt) | Major | applicationmaster, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4266](https://issues.apache.org/jira/browse/MAPREDUCE-4266) | remove Ant remnants from MR | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-154](https://issues.apache.org/jira/browse/YARN-154) | Create Yarn trunk and commit jobs | Major | . | Eli Collins | Robert Joseph Evans | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/RELEASENOTES.0.23.5.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/RELEASENOTES.0.23.5.md new file mode 100644 index 00000000000..d8efa4021fd --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.5/RELEASENOTES.0.23.5.md @@ -0,0 +1,38 @@ + + +# Apache Hadoop 0.23.5 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8926](https://issues.apache.org/jira/browse/HADOOP-8926) | *Major* | **hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data** + +Speed up Crc32 by improving the cache hit-ratio of hadoop.util.PureJavaCrc32 + + +--- + +* [HDFS-4080](https://issues.apache.org/jira/browse/HDFS-4080) | *Major* | **Add a separate logger for block state change logs to enable turning off those logs** + +Add a separate logger "BlockStateChange" for block state change logs. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/CHANGES.0.23.6.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/CHANGES.0.23.6.md new file mode 100644 index 00000000000..2a34ad72750 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/CHANGES.0.23.6.md @@ -0,0 +1,137 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.6 - 2013-02-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8992](https://issues.apache.org/jira/browse/HADOOP-8992) | Enhance unit-test coverage of class HarFileSystem | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8931](https://issues.apache.org/jira/browse/HADOOP-8931) | Add Java version to startup message | Trivial | . | Eli Collins | Eli Collins | +| [HADOOP-8561](https://issues.apache.org/jira/browse/HADOOP-8561) | Introduce HADOOP\_PROXY\_USER for secure impersonation in child hadoop client processes | Major | security | Luke Lu | Yu Gao | +| [MAPREDUCE-4899](https://issues.apache.org/jira/browse/MAPREDUCE-4899) | Provide a plugin to the Yarn Web App Proxy to generate tracking links for M/R appllications given the ID | Major | . | Derek Dagit | Derek Dagit | +| [MAPREDUCE-4845](https://issues.apache.org/jira/browse/MAPREDUCE-4845) | ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2 | Major | client | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4811](https://issues.apache.org/jira/browse/MAPREDUCE-4811) | JobHistoryServer should show when it was started in WebUI About page | Minor | jobhistoryserver, mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4810](https://issues.apache.org/jira/browse/MAPREDUCE-4810) | Add admin command options for ApplicationMaster | Minor | applicationmaster | Jason Lowe | Jerry Chen | +| [MAPREDUCE-4764](https://issues.apache.org/jira/browse/MAPREDUCE-4764) | repair test org.apache.hadoop.mapreduce.security.TestBinaryTokenFile | Major | . | Ivan A. Veselovsky | | +| [YARN-285](https://issues.apache.org/jira/browse/YARN-285) | RM should be able to provide a tracking link for apps that have already been purged | Major | . | Derek Dagit | Derek Dagit | +| [YARN-80](https://issues.apache.org/jira/browse/YARN-80) | Support delay scheduling for node locality in MR2's capacity scheduler | Major | capacityscheduler | Todd Lipcon | Arun C Murthy | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9255](https://issues.apache.org/jira/browse/HADOOP-9255) | relnotes.py missing last jira | Critical | scripts | Thomas Graves | Thomas Graves | +| [HADOOP-9242](https://issues.apache.org/jira/browse/HADOOP-9242) | Duplicate surefire plugin config in hadoop-common | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-9181](https://issues.apache.org/jira/browse/HADOOP-9181) | Set daemon flag for HttpServer's QueuedThreadPool | Major | . | Liang Xie | Liang Xie | +| [HADOOP-9169](https://issues.apache.org/jira/browse/HADOOP-9169) | Bring branch-0.23 ExitUtil up to same level as branch-2 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-9152](https://issues.apache.org/jira/browse/HADOOP-9152) | HDFS can report negative DFS Used on clusters with very small amounts of data | Minor | fs | Brock Noland | Brock Noland | +| [HADOOP-9135](https://issues.apache.org/jira/browse/HADOOP-9135) | JniBasedUnixGroupsMappingWithFallback should log at debug rather than info during fallback | Trivial | security | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9108](https://issues.apache.org/jira/browse/HADOOP-9108) | Add a method to clear terminateCalled to ExitUtil for test cases | Major | util | Kihwal Lee | Kihwal Lee | +| [HADOOP-9105](https://issues.apache.org/jira/browse/HADOOP-9105) | FsShell -moveFromLocal erroneously fails | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-9097](https://issues.apache.org/jira/browse/HADOOP-9097) | Maven RAT plugin is not checking all source files | Critical | build | Tom White | Thomas Graves | +| [HADOOP-9072](https://issues.apache.org/jira/browse/HADOOP-9072) | Hadoop-Common-0.23-Build Fails to build in Jenkins | Major | . | Robert Parker | Robert Parker | +| [HADOOP-7868](https://issues.apache.org/jira/browse/HADOOP-7868) | Hadoop native fails to compile when default linker option is -Wl,--as-needed | Major | native | James Page | Trevor Robinson | +| [HDFS-4426](https://issues.apache.org/jira/browse/HDFS-4426) | Secondary namenode shuts down immediately after startup | Blocker | namenode | Jason Lowe | Arpit Agarwal | +| [HDFS-4385](https://issues.apache.org/jira/browse/HDFS-4385) | Maven RAT plugin is not checking all source files | Critical | build | Thomas Graves | Thomas Graves | +| [HDFS-4315](https://issues.apache.org/jira/browse/HDFS-4315) | DNs with multiple BPs can have BPOfferServices fail to start due to unsynchronized map access | Major | datanode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4270](https://issues.apache.org/jira/browse/HDFS-4270) | Replications of the highest priority should be allowed to choose a source datanode that has reached its max replication limit | Minor | namenode | Derek Dagit | Derek Dagit | +| [HDFS-4254](https://issues.apache.org/jira/browse/HDFS-4254) | testAllEditsDirsFailOnFlush makes subsequent test cases fail (0.23.6 only) | Major | test | Kihwal Lee | Kihwal Lee | +| [HDFS-4242](https://issues.apache.org/jira/browse/HDFS-4242) | Map.Entry is incorrectly used in LeaseManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4233](https://issues.apache.org/jira/browse/HDFS-4233) | NN keeps serving even after no journals started while rolling edit | Blocker | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4232](https://issues.apache.org/jira/browse/HDFS-4232) | NN fails to write a fsimage with stale leases | Blocker | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4186](https://issues.apache.org/jira/browse/HDFS-4186) | logSync() is called with the write lock held while releasing lease | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-2757](https://issues.apache.org/jira/browse/HDFS-2757) | Cannot read a local block that's being written to when using the local read short circuit | Major | . | Jean-Daniel Cryans | Jean-Daniel Cryans | +| [MAPREDUCE-4934](https://issues.apache.org/jira/browse/MAPREDUCE-4934) | Maven RAT plugin is not checking all source files | Critical | build | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4925](https://issues.apache.org/jira/browse/MAPREDUCE-4925) | The pentomino option parser may be buggy | Major | examples | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4921](https://issues.apache.org/jira/browse/MAPREDUCE-4921) | JobClient should acquire HS token with RM principal | Blocker | client | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4913](https://issues.apache.org/jira/browse/MAPREDUCE-4913) | TestMRAppMaster#testMRAppMasterMissingStaging occasionally exits | Major | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4902](https://issues.apache.org/jira/browse/MAPREDUCE-4902) | Fix typo "receievd" should be "received" in log output | Trivial | . | Albert Chu | Albert Chu | +| [MAPREDUCE-4895](https://issues.apache.org/jira/browse/MAPREDUCE-4895) | Fix compilation failure of org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulators | Major | . | Dennis Y | Dennis Y | +| [MAPREDUCE-4894](https://issues.apache.org/jira/browse/MAPREDUCE-4894) | Renewal / cancellation of JobHistory tokens | Blocker | jobhistoryserver, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4890](https://issues.apache.org/jira/browse/MAPREDUCE-4890) | Invalid TaskImpl state transitions when task fails while speculating | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4861](https://issues.apache.org/jira/browse/MAPREDUCE-4861) | Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4856](https://issues.apache.org/jira/browse/MAPREDUCE-4856) | TestJobOutputCommitter uses same directory as TestJobCleanup | Major | test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4848](https://issues.apache.org/jira/browse/MAPREDUCE-4848) | TaskAttemptContext cast error during AM recovery | Major | mr-am | Jason Lowe | Jerry Chen | +| [MAPREDUCE-4842](https://issues.apache.org/jira/browse/MAPREDUCE-4842) | Shuffle race can hang reducer | Blocker | mrv2 | Jason Lowe | Mariappan Asokan | +| [MAPREDUCE-4836](https://issues.apache.org/jira/browse/MAPREDUCE-4836) | Elapsed time for running tasks on AM web UI tasks page is 0 | Major | . | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4833](https://issues.apache.org/jira/browse/MAPREDUCE-4833) | Task can get stuck in FAIL\_CONTAINER\_CLEANUP | Critical | applicationmaster, mrv2 | Robert Joseph Evans | Robert Parker | +| [MAPREDUCE-4832](https://issues.apache.org/jira/browse/MAPREDUCE-4832) | MR AM can get in a split brain situation | Critical | applicationmaster | Robert Joseph Evans | Jason Lowe | +| [MAPREDUCE-4825](https://issues.apache.org/jira/browse/MAPREDUCE-4825) | JobImpl.finished doesn't expect ERROR as a final job state | Major | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4819](https://issues.apache.org/jira/browse/MAPREDUCE-4819) | AM can rerun job after reporting final job status to the client | Blocker | mr-am | Jason Lowe | Bikas Saha | +| [MAPREDUCE-4817](https://issues.apache.org/jira/browse/MAPREDUCE-4817) | Hardcoded task ping timeout kills tasks localizing large amounts of data | Critical | applicationmaster, mr-am | Jason Lowe | Thomas Graves | +| [MAPREDUCE-4813](https://issues.apache.org/jira/browse/MAPREDUCE-4813) | AM timing out during job commit | Critical | applicationmaster | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4678](https://issues.apache.org/jira/browse/MAPREDUCE-4678) | Running the Pentomino example with defaults throws java.lang.NegativeArraySizeException | Minor | examples | Chris McConnell | Chris McConnell | +| [MAPREDUCE-4279](https://issues.apache.org/jira/browse/MAPREDUCE-4279) | getClusterStatus() fails with null pointer exception when running jobs in local mode | Major | jobtracker | Rahul Jain | Devaraj K | +| [YARN-354](https://issues.apache.org/jira/browse/YARN-354) | WebAppProxyServer exits immediately after startup | Blocker | . | Liang Xie | Liang Xie | +| [YARN-334](https://issues.apache.org/jira/browse/YARN-334) | Maven RAT plugin is not checking all source files | Critical | . | Thomas Graves | Thomas Graves | +| [YARN-325](https://issues.apache.org/jira/browse/YARN-325) | RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completing | Blocker | capacityscheduler | Jason Lowe | Arun C Murthy | +| [YARN-320](https://issues.apache.org/jira/browse/YARN-320) | RM should always be able to renew its own tokens | Blocker | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-293](https://issues.apache.org/jira/browse/YARN-293) | Node Manager leaks LocalizerRunner object for every Container | Critical | nodemanager | Devaraj K | Robert Joseph Evans | +| [YARN-266](https://issues.apache.org/jira/browse/YARN-266) | RM and JHS Web UIs are blank because AppsBlock is not escaping string properly | Critical | resourcemanager | Ravi Prakash | Ravi Prakash | +| [YARN-258](https://issues.apache.org/jira/browse/YARN-258) | RM web page UI shows Invalid Date for start and finish times | Major | resourcemanager | Ravi Prakash | Ravi Prakash | +| [YARN-251](https://issues.apache.org/jira/browse/YARN-251) | Proxy URI generation fails for blank tracking URIs | Major | resourcemanager | Tom White | Tom White | +| [YARN-225](https://issues.apache.org/jira/browse/YARN-225) | Proxy Link in RM UI thows NPE in Secure mode | Critical | resourcemanager | Devaraj K | Devaraj K | +| [YARN-223](https://issues.apache.org/jira/browse/YARN-223) | Change processTree interface to work better with native code | Critical | . | Radim Kolar | Radim Kolar | +| [YARN-170](https://issues.apache.org/jira/browse/YARN-170) | NodeManager stop() gets called twice on shutdown | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-72](https://issues.apache.org/jira/browse/YARN-72) | NM should handle cleaning up containers when it shuts down | Major | nodemanager | Hitesh Shah | Sandy Ryza | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9217](https://issues.apache.org/jira/browse/HADOOP-9217) | Print thread dumps when hadoop-common tests fail | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-9038](https://issues.apache.org/jira/browse/HADOOP-9038) | provide unit-test coverage of class org.apache.hadoop.fs.LocalDirAllocator.AllocatorPerContext.PathIterator | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9020](https://issues.apache.org/jira/browse/HADOOP-9020) | Add a SASL PLAIN server | Major | ipc, security | Daryn Sharp | Daryn Sharp | +| [HDFS-4248](https://issues.apache.org/jira/browse/HDFS-4248) | Renames may remove file leases | Blocker | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4247](https://issues.apache.org/jira/browse/HDFS-4247) | saveNamespace should be tolerant of dangling lease | Blocker | namenode | Daryn Sharp | Daryn Sharp | +| [YARN-280](https://issues.apache.org/jira/browse/YARN-280) | RM does not reject app submission with invalid tokens | Major | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-204](https://issues.apache.org/jira/browse/YARN-204) | test coverage for org.apache.hadoop.tools | Major | applications | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-50](https://issues.apache.org/jira/browse/YARN-50) | Implement renewal / cancellation of Delegation Tokens | Blocker | . | Siddharth Seth | Siddharth Seth | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8427](https://issues.apache.org/jira/browse/HADOOP-8427) | Convert Forrest docs to APT, incremental | Major | documentation | Eli Collins | Andy Isaacson | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/RELEASENOTES.0.23.6.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/RELEASENOTES.0.23.6.md new file mode 100644 index 00000000000..5038bad8a67 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.6/RELEASENOTES.0.23.6.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.23.6 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/CHANGES.0.23.7.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/CHANGES.0.23.7.md new file mode 100644 index 00000000000..ff0352f72ca --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/CHANGES.0.23.7.md @@ -0,0 +1,195 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.7 - 2013-04-18 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-395](https://issues.apache.org/jira/browse/HDFS-395) | DFS Scalability: Incremental block reports | Major | datanode, namenode | dhruba borthakur | Tomasz Nykiel | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9209](https://issues.apache.org/jira/browse/HADOOP-9209) | Add shell command to dump file checksums | Major | fs, tools | Todd Lipcon | Todd Lipcon | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9379](https://issues.apache.org/jira/browse/HADOOP-9379) | capture the ulimit info after printing the log to the console | Trivial | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9374](https://issues.apache.org/jira/browse/HADOOP-9374) | Add tokens from -tokenCacheFile into UGI | Major | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9352](https://issues.apache.org/jira/browse/HADOOP-9352) | Expose UGI.setLoginUser for tests | Major | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9336](https://issues.apache.org/jira/browse/HADOOP-9336) | Allow UGI of current connection to be queried | Critical | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9253](https://issues.apache.org/jira/browse/HADOOP-9253) | Capture ulimit info in the logs at service start time | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9247](https://issues.apache.org/jira/browse/HADOOP-9247) | parametrize Clover "generateXxx" properties to make them re-definable via -D in mvn calls | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9216](https://issues.apache.org/jira/browse/HADOOP-9216) | CompressionCodecFactory#getCodecClasses should trim the result of parsing by Configuration. | Major | io | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-9147](https://issues.apache.org/jira/browse/HADOOP-9147) | Add missing fields to FIleStatus.toString | Trivial | . | Jonathan Allen | Jonathan Allen | +| [HADOOP-8849](https://issues.apache.org/jira/browse/HADOOP-8849) | FileUtil#fullyDelete should grant the target directories +rwx permissions before trying to delete them | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8711](https://issues.apache.org/jira/browse/HADOOP-8711) | provide an option for IPC server users to avoid printing stack information for certain exceptions | Major | ipc | Brandon Li | Brandon Li | +| [HADOOP-8462](https://issues.apache.org/jira/browse/HADOOP-8462) | Native-code implementation of bzip2 codec | Major | io | Govind Kamat | Govind Kamat | +| [HADOOP-8214](https://issues.apache.org/jira/browse/HADOOP-8214) | make hadoop script recognize a full set of deprecated commands | Major | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-8075](https://issues.apache.org/jira/browse/HADOOP-8075) | Lower native-hadoop library log from info to debug | Major | native | Eli Collins | Hızır Sefa Ä°rken | +| [HADOOP-7886](https://issues.apache.org/jira/browse/HADOOP-7886) | Add toString to FileStatus | Minor | . | Jakob Homan | SreeHari | +| [HADOOP-7358](https://issues.apache.org/jira/browse/HADOOP-7358) | Improve log levels when exceptions caught in RPC handler | Minor | ipc | Todd Lipcon | Todd Lipcon | +| [HDFS-3817](https://issues.apache.org/jira/browse/HDFS-3817) | avoid printing stack information for SafeModeException | Major | namenode | Brandon Li | Brandon Li | +| [MAPREDUCE-5079](https://issues.apache.org/jira/browse/MAPREDUCE-5079) | Recovery should restore task state from job history info directly | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4990](https://issues.apache.org/jira/browse/MAPREDUCE-4990) | Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput() | Trivial | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4989](https://issues.apache.org/jira/browse/MAPREDUCE-4989) | JSONify DataTables input data for Attempts page | Major | jobhistoryserver, mr-am | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4949](https://issues.apache.org/jira/browse/MAPREDUCE-4949) | Enable multiple pi jobs to run in parallel | Minor | examples | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4907](https://issues.apache.org/jira/browse/MAPREDUCE-4907) | TrackerDistributedCacheManager issues too many getFileStatus calls | Major | mrv1, tasktracker | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4822](https://issues.apache.org/jira/browse/MAPREDUCE-4822) | Unnecessary conversions in History Events | Trivial | jobhistoryserver | Robert Joseph Evans | Chu Tong | +| [MAPREDUCE-4458](https://issues.apache.org/jira/browse/MAPREDUCE-4458) | Warn if java.library.path is used for AM or Task | Major | mrv2 | Robert Joseph Evans | Robert Parker | +| [YARN-525](https://issues.apache.org/jira/browse/YARN-525) | make CS node-locality-delay refreshable | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-443](https://issues.apache.org/jira/browse/YARN-443) | allow OS scheduling priority of NM to be different than the containers it launches | Major | nodemanager | Thomas Graves | Thomas Graves | +| [YARN-249](https://issues.apache.org/jira/browse/YARN-249) | Capacity Scheduler web page should show list of active users per queue like it used to (in 1.x) | Major | capacityscheduler | Ravi Prakash | Ravi Prakash | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9406](https://issues.apache.org/jira/browse/HADOOP-9406) | hadoop-client leaks dependency on JDK tools jar | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-9339](https://issues.apache.org/jira/browse/HADOOP-9339) | IPC.Server incorrectly sets UGI auth type | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9303](https://issues.apache.org/jira/browse/HADOOP-9303) | command manual dfsadmin missing entry for restoreFailedStorage option | Major | . | Thomas Graves | Andy Isaacson | +| [HADOOP-9302](https://issues.apache.org/jira/browse/HADOOP-9302) | HDFS docs not linked from top level | Major | documentation | Thomas Graves | Andy Isaacson | +| [HADOOP-9289](https://issues.apache.org/jira/browse/HADOOP-9289) | FsShell rm -f fails for non-matching globs | Blocker | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-9278](https://issues.apache.org/jira/browse/HADOOP-9278) | HarFileSystem may leak file handle | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9231](https://issues.apache.org/jira/browse/HADOOP-9231) | Parametrize staging URL for the uniformity of distributionManagement | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-9221](https://issues.apache.org/jira/browse/HADOOP-9221) | Convert remaining xdocs to APT | Major | . | Andy Isaacson | Andy Isaacson | +| [HADOOP-9212](https://issues.apache.org/jira/browse/HADOOP-9212) | Potential deadlock in FileSystem.Cache/IPC/UGI | Major | fs | Tom White | Tom White | +| [HADOOP-9193](https://issues.apache.org/jira/browse/HADOOP-9193) | hadoop script can inadvertently expand wildcard arguments when delegating to hdfs script | Minor | scripts | Jason Lowe | Andy Isaacson | +| [HADOOP-9190](https://issues.apache.org/jira/browse/HADOOP-9190) | packaging docs is broken | Major | documentation | Thomas Graves | Andy Isaacson | +| [HADOOP-9155](https://issues.apache.org/jira/browse/HADOOP-9155) | FsPermission should have different default value, 777 for directory and 666 for file | Minor | . | Binglin Chang | Binglin Chang | +| [HADOOP-9154](https://issues.apache.org/jira/browse/HADOOP-9154) | SortedMapWritable#putAll() doesn't add key/value classes to the map | Major | io | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9124](https://issues.apache.org/jira/browse/HADOOP-9124) | SortedMapWritable violates contract of Map interface for equals() and hashCode() | Minor | io | Patrick Hunt | Surenkumar Nihalani | +| [HADOOP-8878](https://issues.apache.org/jira/browse/HADOOP-8878) | uppercase namenode hostname causes hadoop dfs calls with webhdfs filesystem and fsck to fail when security is on | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8857](https://issues.apache.org/jira/browse/HADOOP-8857) | hadoop.http.authentication.signature.secret.file docs should not state that secret is randomly generated | Minor | security | Eli Collins | Alejandro Abdelnur | +| [HADOOP-8816](https://issues.apache.org/jira/browse/HADOOP-8816) | HTTP Error 413 full HEAD if using kerberos authentication | Major | net | Moritz Moeller | Moritz Moeller | +| [HADOOP-8346](https://issues.apache.org/jira/browse/HADOOP-8346) | Changes to support Kerberos with non Sun JVM (HADOOP-6941) broke SPNEGO | Blocker | security | Alejandro Abdelnur | Devaraj Das | +| [HADOOP-8251](https://issues.apache.org/jira/browse/HADOOP-8251) | SecurityUtil.fetchServiceTicket broken after HADOOP-6941 | Blocker | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6941](https://issues.apache.org/jira/browse/HADOOP-6941) | Support non-SUN JREs in UserGroupInformation | Major | . | Stephen Watt | Devaraj Das | +| [HDFS-4649](https://issues.apache.org/jira/browse/HDFS-4649) | Webhdfs cannot list large directories | Blocker | namenode, security, webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4581](https://issues.apache.org/jira/browse/HDFS-4581) | DataNode#checkDiskError should not be called on network errors | Major | datanode | Rohit Kochar | Rohit Kochar | +| [HDFS-4553](https://issues.apache.org/jira/browse/HDFS-4553) | Webhdfs will NPE on some unexpected response codes | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4544](https://issues.apache.org/jira/browse/HDFS-4544) | Error in deleting blocks should not do check disk, for all types of errors | Major | . | Amareshwari Sriramadasu | Arpit Agarwal | +| [HDFS-4532](https://issues.apache.org/jira/browse/HDFS-4532) | RPC call queue may fill due to current user lookup | Critical | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4495](https://issues.apache.org/jira/browse/HDFS-4495) | Allow client-side lease renewal to be retried beyond soft-limit | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-4462](https://issues.apache.org/jira/browse/HDFS-4462) | 2NN will fail to checkpoint after an HDFS upgrade from a pre-federation version of HDFS | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4444](https://issues.apache.org/jira/browse/HDFS-4444) | Add space between total transaction time and number of transactions in FSEditLog#printStatistics | Trivial | . | Stephen Chu | Stephen Chu | +| [HDFS-4426](https://issues.apache.org/jira/browse/HDFS-4426) | Secondary namenode shuts down immediately after startup | Blocker | namenode | Jason Lowe | Arpit Agarwal | +| [HDFS-4288](https://issues.apache.org/jira/browse/HDFS-4288) | NN accepts incremental BR as IBR in safemode | Critical | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4222](https://issues.apache.org/jira/browse/HDFS-4222) | NN is unresponsive and loses heartbeats of DNs when Hadoop is configured to use LDAP and LDAP has issues | Minor | namenode | Xiaobo Peng | Xiaobo Peng | +| [HDFS-4128](https://issues.apache.org/jira/browse/HDFS-4128) | 2NN gets stuck in inconsistent state if edit log replay fails in the middle | Major | namenode | Todd Lipcon | Kihwal Lee | +| [HDFS-4072](https://issues.apache.org/jira/browse/HDFS-4072) | On file deletion remove corresponding blocks pending replication | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-3344](https://issues.apache.org/jira/browse/HDFS-3344) | Unreliable corrupt blocks counting in TestProcessCorruptBlocks | Major | namenode | Tsz Wo Nicholas Sze | Kihwal Lee | +| [HDFS-3256](https://issues.apache.org/jira/browse/HDFS-3256) | HDFS considers blocks under-replicated if topology script is configured with only 1 rack | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3119](https://issues.apache.org/jira/browse/HDFS-3119) | Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file | Minor | namenode | J.Andreina | Ashish Singhi | +| [HDFS-2434](https://issues.apache.org/jira/browse/HDFS-2434) | TestNameNodeMetrics.testCorruptBlock fails intermittently | Major | test | Uma Maheswara Rao G | Jing Zhao | +| [HDFS-1765](https://issues.apache.org/jira/browse/HDFS-1765) | Block Replication should respect under-replication block priority | Major | namenode | Hairong Kuang | Uma Maheswara Rao G | +| [MAPREDUCE-5137](https://issues.apache.org/jira/browse/MAPREDUCE-5137) | AM web UI: clicking on Map Task results in 500 error | Major | applicationmaster | Thomas Graves | Thomas Graves | +| [MAPREDUCE-5075](https://issues.apache.org/jira/browse/MAPREDUCE-5075) | DistCp leaks input file handles | Major | distcp | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5060](https://issues.apache.org/jira/browse/MAPREDUCE-5060) | Fetch failures that time out only count against the first map task | Critical | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-5053](https://issues.apache.org/jira/browse/MAPREDUCE-5053) | java.lang.InternalError from decompression codec cause reducer to fail | Major | . | Robert Parker | Robert Parker | +| [MAPREDUCE-5043](https://issues.apache.org/jira/browse/MAPREDUCE-5043) | Fetch failure processing can cause AM event queue to backup and eventually OOM | Blocker | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5042](https://issues.apache.org/jira/browse/MAPREDUCE-5042) | Reducer unable to fetch for a map task that was recovered | Blocker | mr-am, security | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5027](https://issues.apache.org/jira/browse/MAPREDUCE-5027) | Shuffle does not limit number of outstanding connections | Major | . | Jason Lowe | Robert Parker | +| [MAPREDUCE-5023](https://issues.apache.org/jira/browse/MAPREDUCE-5023) | History Server Web Services missing Job Counters | Critical | jobhistoryserver, webapps | Kendall Thrapp | Ravi Prakash | +| [MAPREDUCE-5009](https://issues.apache.org/jira/browse/MAPREDUCE-5009) | Killing the Task Attempt slated for commit does not clear the value from the Task commitAttempt member | Critical | mrv1 | Robert Parker | Robert Parker | +| [MAPREDUCE-5000](https://issues.apache.org/jira/browse/MAPREDUCE-5000) | TaskImpl.getCounters() can return the counters for the wrong task attempt when task is speculating | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4992](https://issues.apache.org/jira/browse/MAPREDUCE-4992) | AM hangs in RecoveryService when recovering tasks with speculative attempts | Critical | mr-am | Robert Parker | Robert Parker | +| [MAPREDUCE-4969](https://issues.apache.org/jira/browse/MAPREDUCE-4969) | TestKeyValueTextInputFormat test fails with Open JDK 7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [MAPREDUCE-4953](https://issues.apache.org/jira/browse/MAPREDUCE-4953) | HadoopPipes misuses fprintf | Major | pipes | Andy Isaacson | Andy Isaacson | +| [MAPREDUCE-4946](https://issues.apache.org/jira/browse/MAPREDUCE-4946) | Type conversion of map completion events leads to performance problems with large jobs | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4893](https://issues.apache.org/jira/browse/MAPREDUCE-4893) | MR AppMaster can do sub-optimal assignment of containers to map tasks leading to poor node locality | Major | applicationmaster | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4871](https://issues.apache.org/jira/browse/MAPREDUCE-4871) | AM uses mapreduce.jobtracker.split.metainfo.maxsize but mapred-default has mapreduce.job.split.metainfo.maxsize | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4794](https://issues.apache.org/jira/browse/MAPREDUCE-4794) | DefaultSpeculator generates error messages on normal shutdown | Major | applicationmaster | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4671](https://issues.apache.org/jira/browse/MAPREDUCE-4671) | AM does not tell the RM about container requests that are no longer needed | Major | . | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4637](https://issues.apache.org/jira/browse/MAPREDUCE-4637) | Killing an unassigned task attempt causes the job to fail | Major | mrv2 | Tom White | Mayank Bansal | +| [MAPREDUCE-4470](https://issues.apache.org/jira/browse/MAPREDUCE-4470) | Fix TestCombineFileInputFormat.testForEmptyFile | Major | test | Kihwal Lee | Ilya Katsov | +| [MAPREDUCE-4278](https://issues.apache.org/jira/browse/MAPREDUCE-4278) | cannot run two local jobs in parallel from the same gateway. | Major | . | Araceli Henley | Sandy Ryza | +| [MAPREDUCE-4007](https://issues.apache.org/jira/browse/MAPREDUCE-4007) | JobClient getJob(JobID) should return NULL if the job does not exist (for backwards compatibility) | Major | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3952](https://issues.apache.org/jira/browse/MAPREDUCE-3952) | In MR2, when Total input paths to process == 1, CombinefileInputFormat.getSplits() returns 0 split. | Major | mrv2 | Zhenxiao Luo | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3685](https://issues.apache.org/jira/browse/MAPREDUCE-3685) | There are some bugs in implementation of MergeManager | Critical | mrv2 | anty.rao | anty | +| [YARN-460](https://issues.apache.org/jira/browse/YARN-460) | CS user left in list of active users for the queue even when application finished | Blocker | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-448](https://issues.apache.org/jira/browse/YARN-448) | Remove unnecessary hflush from log aggregation | Major | nodemanager | Kihwal Lee | Kihwal Lee | +| [YARN-426](https://issues.apache.org/jira/browse/YARN-426) | Failure to download a public resource on a node prevents further downloads of the resource from that node | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-410](https://issues.apache.org/jira/browse/YARN-410) | New lines in diagnostics for a failed app on the per-application page make it hard to read | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-400](https://issues.apache.org/jira/browse/YARN-400) | RM can return null application resource usage report leading to NPE in client | Critical | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-376](https://issues.apache.org/jira/browse/YARN-376) | Apps that have completed can appear as RUNNING on the NM UI | Blocker | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-364](https://issues.apache.org/jira/browse/YARN-364) | AggregatedLogDeletionService can take too long to delete logs | Major | . | Jason Lowe | Jason Lowe | +| [YARN-362](https://issues.apache.org/jira/browse/YARN-362) | Unexpected extra results when using webUI table search | Minor | . | Jason Lowe | Ravi Prakash | +| [YARN-360](https://issues.apache.org/jira/browse/YARN-360) | Allow apps to concurrently register tokens for renewal | Critical | . | Daryn Sharp | Daryn Sharp | +| [YARN-357](https://issues.apache.org/jira/browse/YARN-357) | App submission should not be synchronized | Major | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-355](https://issues.apache.org/jira/browse/YARN-355) | RM app submission jams under load | Blocker | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-354](https://issues.apache.org/jira/browse/YARN-354) | WebAppProxyServer exits immediately after startup | Blocker | . | Liang Xie | Liang Xie | +| [YARN-345](https://issues.apache.org/jira/browse/YARN-345) | Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager | Critical | nodemanager | Devaraj K | Robert Parker | +| [YARN-343](https://issues.apache.org/jira/browse/YARN-343) | Capacity Scheduler maximum-capacity value -1 is invalid | Major | capacityscheduler | Thomas Graves | Xuan Gong | +| [YARN-269](https://issues.apache.org/jira/browse/YARN-269) | Resource Manager not logging the health\_check\_script result when taking it out | Major | resourcemanager | Thomas Graves | Jason Lowe | +| [YARN-236](https://issues.apache.org/jira/browse/YARN-236) | RM should point tracking URL to RM web page when app fails to start | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-227](https://issues.apache.org/jira/browse/YARN-227) | Application expiration difficult to debug for end-users | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-150](https://issues.apache.org/jira/browse/YARN-150) | AppRejectedTransition does not unregister app from master service and scheduler | Major | . | Bikas Saha | Bikas Saha | +| [YARN-133](https://issues.apache.org/jira/browse/YARN-133) | update web services docs for RM clusterMetrics | Major | resourcemanager | Thomas Graves | Ravi Prakash | +| [YARN-109](https://issues.apache.org/jira/browse/YARN-109) | .tmp file is not deleted for localized archives | Major | nodemanager | Jason Lowe | Mayank Bansal | +| [YARN-83](https://issues.apache.org/jira/browse/YARN-83) | Change package of YarnClient to include apache | Major | client | Bikas Saha | Bikas Saha | +| [YARN-40](https://issues.apache.org/jira/browse/YARN-40) | Provide support for missing yarn commands | Major | client | Devaraj K | Devaraj K | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9067](https://issues.apache.org/jira/browse/HADOOP-9067) | provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long) | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8157](https://issues.apache.org/jira/browse/HADOOP-8157) | TestRPCCallBenchmark#testBenchmarkWithWritable fails with RTE | Major | . | Eli Collins | Todd Lipcon | +| [MAPREDUCE-5007](https://issues.apache.org/jira/browse/MAPREDUCE-5007) | fix coverage org.apache.hadoop.mapreduce.v2.hs | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4991](https://issues.apache.org/jira/browse/MAPREDUCE-4991) | coverage for gridmix | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4972](https://issues.apache.org/jira/browse/MAPREDUCE-4972) | Coverage fixing for org.apache.hadoop.mapreduce.jobhistory | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4905](https://issues.apache.org/jira/browse/MAPREDUCE-4905) | test org.apache.hadoop.mapred.pipes | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4875](https://issues.apache.org/jira/browse/MAPREDUCE-4875) | coverage fixing for org.apache.hadoop.mapred | Major | test | Aleksey Gorshkov | Aleksey Gorshkov | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-4577](https://issues.apache.org/jira/browse/HDFS-4577) | Webhdfs operations should declare if authentication is required | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4567](https://issues.apache.org/jira/browse/HDFS-4567) | Webhdfs does not need a token for token operations | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4566](https://issues.apache.org/jira/browse/HDFS-4566) | Webdhfs token cancelation should use authentication | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4560](https://issues.apache.org/jira/browse/HDFS-4560) | Webhdfs cannot use tokens obtained by another user | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4548](https://issues.apache.org/jira/browse/HDFS-4548) | Webhdfs doesn't renegotiate SPNEGO token | Blocker | . | Daryn Sharp | Daryn Sharp | +| [HDFS-4542](https://issues.apache.org/jira/browse/HDFS-4542) | Webhdfs doesn't support secure proxy users | Blocker | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-2495](https://issues.apache.org/jira/browse/HDFS-2495) | Increase granularity of write operations in ReplicationMonitor thus reducing contention for write lock | Major | namenode | Tomasz Nykiel | Tomasz Nykiel | +| [HDFS-2477](https://issues.apache.org/jira/browse/HDFS-2477) | Optimize computing the diff between a block report and the namenode state. | Major | namenode | Tomasz Nykiel | Tomasz Nykiel | +| [HDFS-2476](https://issues.apache.org/jira/browse/HDFS-2476) | More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks | Major | namenode | Tomasz Nykiel | Tomasz Nykiel | +| [YARN-468](https://issues.apache.org/jira/browse/YARN-468) | coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-200](https://issues.apache.org/jira/browse/YARN-200) | yarn log does not output all needed information, and is in a binary format | Major | . | Robert Joseph Evans | Ravi Prakash | +| [YARN-29](https://issues.apache.org/jira/browse/YARN-29) | Add a yarn-client module | Major | client | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/RELEASENOTES.0.23.7.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/RELEASENOTES.0.23.7.md new file mode 100644 index 00000000000..17ff954b881 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.7/RELEASENOTES.0.23.7.md @@ -0,0 +1,45 @@ + + +# Apache Hadoop 0.23.7 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9147](https://issues.apache.org/jira/browse/HADOOP-9147) | *Trivial* | **Add missing fields to FIleStatus.toString** + +Update FileStatus.toString to include missing fields + + +--- + +* [HDFS-395](https://issues.apache.org/jira/browse/HDFS-395) | *Major* | **DFS Scalability: Incremental block reports** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4637](https://issues.apache.org/jira/browse/MAPREDUCE-4637) | *Major* | **Killing an unassigned task attempt causes the job to fail** + +Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/CHANGES.0.23.8.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/CHANGES.0.23.8.md new file mode 100644 index 00000000000..707f411a45d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/CHANGES.0.23.8.md @@ -0,0 +1,89 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.8 - 2013-06-05 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9614](https://issues.apache.org/jira/browse/HADOOP-9614) | smart-test-patch.sh hangs for new version of patch (2.7.1) | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-9504](https://issues.apache.org/jira/browse/HADOOP-9504) | MetricsDynamicMBeanBase has concurrency issues in createMBeanInfo | Critical | metrics | Liang Xie | Liang Xie | +| [HADOOP-9469](https://issues.apache.org/jira/browse/HADOOP-9469) | mapreduce/yarn source jars not included in dist tarball | Major | . | Thomas Graves | Robert Parker | +| [HDFS-4835](https://issues.apache.org/jira/browse/HDFS-4835) | Port trunk WebHDFS changes to branch-0.23 | Critical | webhdfs | Robert Parker | Robert Parker | +| [HDFS-4807](https://issues.apache.org/jira/browse/HDFS-4807) | DFSOutputStream.createSocketForPipeline() should not include timeout extension on connect | Major | . | Kihwal Lee | Cristina L. Abad | +| [HDFS-4714](https://issues.apache.org/jira/browse/HDFS-4714) | Log short messages in Namenode RPC server for exceptions meant for clients | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4699](https://issues.apache.org/jira/browse/HDFS-4699) | TestPipelinesFailover#testPipelineRecoveryStress fails sporadically | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4690](https://issues.apache.org/jira/browse/HDFS-4690) | Namenode exits if entering safemode while secret manager is edit logging | Critical | namenode, security | Daryn Sharp | Daryn Sharp | +| [HDFS-4477](https://issues.apache.org/jira/browse/HDFS-4477) | Secondary namenode may retain old tokens | Critical | security | Kihwal Lee | Daryn Sharp | +| [HDFS-3875](https://issues.apache.org/jira/browse/HDFS-3875) | Issue handling checksum errors in write pipeline | Critical | datanode, hdfs-client | Todd Lipcon | Kihwal Lee | +| [MAPREDUCE-5211](https://issues.apache.org/jira/browse/MAPREDUCE-5211) | Reducer intermediate files can collide during merge | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5168](https://issues.apache.org/jira/browse/MAPREDUCE-5168) | Reducer can OOM during shuffle because on-disk output stream not released | Critical | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5147](https://issues.apache.org/jira/browse/MAPREDUCE-5147) | Maven build should create hadoop-mapreduce-client-app-VERSION.jar directly | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5065](https://issues.apache.org/jira/browse/MAPREDUCE-5065) | DistCp should skip checksum comparisons if block-sizes are different on source/target. | Major | distcp | Mithun Radhakrishnan | Mithun Radhakrishnan | +| [MAPREDUCE-5059](https://issues.apache.org/jira/browse/MAPREDUCE-5059) | Job overview shows average merge time larger than for any reduce attempt | Major | jobhistoryserver, webapps | Jason Lowe | Omkar Vinit Joshi | +| [MAPREDUCE-4927](https://issues.apache.org/jira/browse/MAPREDUCE-4927) | Historyserver 500 error due to NPE when accessing specific counters page for failed job | Major | jobhistoryserver | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-4383](https://issues.apache.org/jira/browse/MAPREDUCE-4383) | HadoopPipes.cc needs to include unistd.h | Minor | pipes | Andy Isaacson | Andy Isaacson | +| [YARN-690](https://issues.apache.org/jira/browse/YARN-690) | RM exits on token cancel/renew problems | Blocker | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-476](https://issues.apache.org/jira/browse/YARN-476) | ProcfsBasedProcessTree info message confuses users | Minor | . | Jason Lowe | Sandy Ryza | +| [YARN-363](https://issues.apache.org/jira/browse/YARN-363) | yarn proxyserver fails to find webapps/proxy directory on startup | Major | . | Jason Lowe | Kenji Kikushima | +| [YARN-71](https://issues.apache.org/jira/browse/YARN-71) | Ensure/confirm that the NodeManager cleans up local-dirs on restart | Critical | nodemanager | Vinod Kumar Vavilapalli | Xuan Gong | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9233](https://issues.apache.org/jira/browse/HADOOP-9233) | Cover package org.apache.hadoop.io.compress.zlib with unit tests | Major | . | Vadim Bondarev | Vadim Bondarev | +| [HADOOP-9222](https://issues.apache.org/jira/browse/HADOOP-9222) | Cover package with org.apache.hadoop.io.lz4 unit tests | Major | . | Vadim Bondarev | Vadim Bondarev | +| [MAPREDUCE-5015](https://issues.apache.org/jira/browse/MAPREDUCE-5015) | Coverage fix for org.apache.hadoop.mapreduce.tools.CLI | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-548](https://issues.apache.org/jira/browse/YARN-548) | Add tests for YarnUncaughtExceptionHandler | Major | . | Vadim Bondarev | Vadim Bondarev | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/RELEASENOTES.0.23.8.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/RELEASENOTES.0.23.8.md new file mode 100644 index 00000000000..b3f8e71779c --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.8/RELEASENOTES.0.23.8.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.23.8 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/CHANGES.0.23.9.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/CHANGES.0.23.9.md new file mode 100644 index 00000000000..6d02d85848a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/CHANGES.0.23.9.md @@ -0,0 +1,82 @@ + + +# Apache Hadoop Changelog + +## Release 0.23.9 - 2013-07-08 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5268](https://issues.apache.org/jira/browse/MAPREDUCE-5268) | Improve history server startup performance | Major | jobhistoryserver | Jason Lowe | Karthik Kambatla | +| [YARN-169](https://issues.apache.org/jira/browse/YARN-169) | Update log4j.appender.EventCounter to use org.apache.hadoop.log.metrics.EventCounter | Minor | nodemanager | Anthony Rojas | Anthony Rojas | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9582](https://issues.apache.org/jira/browse/HADOOP-9582) | Non-existent file to "hadoop fs -conf" doesn't throw error | Major | conf | Ashwin Shankar | Ashwin Shankar | +| [HADOOP-9581](https://issues.apache.org/jira/browse/HADOOP-9581) | hadoop --config non-existent directory should result in error | Major | scripts | Ashwin Shankar | Ashwin Shankar | +| [HDFS-4878](https://issues.apache.org/jira/browse/HDFS-4878) | On Remove Block, Block is not Removed from neededReplications queue | Major | namenode | Tao Luo | Tao Luo | +| [HDFS-4867](https://issues.apache.org/jira/browse/HDFS-4867) | metaSave NPEs when there are invalid blocks in repl queue. | Major | namenode | Kihwal Lee | Plamen Jeliazkov | +| [HDFS-4862](https://issues.apache.org/jira/browse/HDFS-4862) | SafeModeInfo.isManual() returns true when resources are low even if it wasn't entered into manually | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-4832](https://issues.apache.org/jira/browse/HDFS-4832) | Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave | Critical | . | Ravi Prakash | Ravi Prakash | +| [HDFS-4765](https://issues.apache.org/jira/browse/HDFS-4765) | Permission check of symlink deletion incorrectly throws UnresolvedLinkException | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4205](https://issues.apache.org/jira/browse/HDFS-4205) | fsck fails with symlinks | Major | hdfs-client | Andy Isaacson | Jason Lowe | +| [MAPREDUCE-5316](https://issues.apache.org/jira/browse/MAPREDUCE-5316) | job -list-attempt-ids command does not handle illegal task-state | Major | client | Ashwin Shankar | Ashwin Shankar | +| [MAPREDUCE-5315](https://issues.apache.org/jira/browse/MAPREDUCE-5315) | DistCp reports success even on failure. | Critical | distcp | Mithun Radhakrishnan | Mithun Radhakrishnan | +| [MAPREDUCE-5308](https://issues.apache.org/jira/browse/MAPREDUCE-5308) | Shuffling to memory can get out-of-sync when fetching multiple compressed map outputs | Major | . | Nathan Roberts | Nathan Roberts | +| [MAPREDUCE-4019](https://issues.apache.org/jira/browse/MAPREDUCE-4019) | -list-attempt-ids is not working | Minor | client | B Anil Kumar | Ashwin Shankar | +| [YARN-742](https://issues.apache.org/jira/browse/YARN-742) | Log aggregation causes a lot of redundant setPermission calls | Major | nodemanager | Kihwal Lee | Jason Lowe | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5084](https://issues.apache.org/jira/browse/MAPREDUCE-5084) | fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-478](https://issues.apache.org/jira/browse/YARN-478) | fix coverage org.apache.hadoop.yarn.webapp.log | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-427](https://issues.apache.org/jira/browse/YARN-427) | Coverage fix for org.apache.hadoop.yarn.server.api.\* | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/RELEASENOTES.0.23.9.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/RELEASENOTES.0.23.9.md new file mode 100644 index 00000000000..81a267fdd34 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.23.9/RELEASENOTES.0.23.9.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 0.23.9 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HDFS-4832](https://issues.apache.org/jira/browse/HDFS-4832) | *Critical* | **Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave** + +This change makes name node keep its internal replication queues and data node state updated in manual safe mode. This allows metrics and UI to present up-to-date information while in safe mode. The behavior during start-up safe mode is unchanged. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/CHANGES.0.24.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/CHANGES.0.24.0.md new file mode 100644 index 00000000000..13f09582581 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/CHANGES.0.24.0.md @@ -0,0 +1,95 @@ + + +# Apache Hadoop Changelog + +## Release 0.24.0 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7547](https://issues.apache.org/jira/browse/HADOOP-7547) | Fix the warning in writable classes.[ WritableComparable is a raw type. References to generic type WritableComparable\ should be parameterized ] | Minor | io | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-7507](https://issues.apache.org/jira/browse/HADOOP-7507) | jvm metrics all use the same namespace | Major | metrics | Jeff Bean | Alejandro Abdelnur | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7705](https://issues.apache.org/jira/browse/HADOOP-7705) | Add a log4j back end that can push out JSON data, one per line | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-7594](https://issues.apache.org/jira/browse/HADOOP-7594) | Support HTTP REST in HttpServer | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7772](https://issues.apache.org/jira/browse/HADOOP-7772) | javadoc the topology classes | Trivial | . | Steve Loughran | Steve Loughran | +| [HADOOP-7763](https://issues.apache.org/jira/browse/HADOOP-7763) | Add top-level navigation to APT docs | Major | documentation | Tom White | Tom White | +| [HADOOP-7743](https://issues.apache.org/jira/browse/HADOOP-7743) | Add Maven profile to create a full source tarball | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7737](https://issues.apache.org/jira/browse/HADOOP-7737) | normalize hadoop-mapreduce & hadoop-dist dist/tar build with common/hdfs | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7668](https://issues.apache.org/jira/browse/HADOOP-7668) | Add a NetUtils method that can tell if an InetAddress belongs to local host | Minor | util | Suresh Srinivas | Steve Loughran | +| [HADOOP-7642](https://issues.apache.org/jira/browse/HADOOP-7642) | create hadoop-dist module where TAR stitching would happen | Major | build | Alejandro Abdelnur | Tom White | +| [HADOOP-7595](https://issues.apache.org/jira/browse/HADOOP-7595) | Upgrade dependency to Avro 1.5.3 | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7509](https://issues.apache.org/jira/browse/HADOOP-7509) | Improve message when Authentication is required | Trivial | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7360](https://issues.apache.org/jira/browse/HADOOP-7360) | FsShell does not preserve relative paths with globs | Major | fs | Daryn Sharp | Kihwal Lee | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7782](https://issues.apache.org/jira/browse/HADOOP-7782) | Aggregate project javadocs | Critical | build | Arun C Murthy | Tom White | +| [HADOOP-7778](https://issues.apache.org/jira/browse/HADOOP-7778) | FindBugs warning in Token.getKind() | Major | . | Tom White | Tom White | +| [HADOOP-7771](https://issues.apache.org/jira/browse/HADOOP-7771) | NPE when running hdfs dfs -copyToLocal, -get etc | Blocker | . | John George | John George | +| [HADOOP-7766](https://issues.apache.org/jira/browse/HADOOP-7766) | The auth to local mappings are not being respected, with webhdfs and security enabled. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7745](https://issues.apache.org/jira/browse/HADOOP-7745) | I switched variable names in HADOOP-7509 | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7728](https://issues.apache.org/jira/browse/HADOOP-7728) | hadoop-setup-conf.sh should be modified to enable task memory manager | Major | conf | Ramya Sunil | Ramya Sunil | +| [HADOOP-7721](https://issues.apache.org/jira/browse/HADOOP-7721) | dfs.web.authentication.kerberos.principal expects the full hostname and does not replace \_HOST with the hostname | Major | . | Arpit Gupta | Jitendra Nath Pandey | +| [HADOOP-7663](https://issues.apache.org/jira/browse/HADOOP-7663) | TestHDFSTrash failing on 22 | Major | test | Mayank Bansal | Mayank Bansal | +| [HADOOP-7608](https://issues.apache.org/jira/browse/HADOOP-7608) | SnappyCodec check for Hadoop native lib is wrong | Major | io | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7606](https://issues.apache.org/jira/browse/HADOOP-7606) | Upgrade Jackson to version 1.7.1 to match the version required by Jersey | Major | test | Aaron T. Myers | Alejandro Abdelnur | +| [HADOOP-7599](https://issues.apache.org/jira/browse/HADOOP-7599) | Improve hadoop setup conf script to setup secure Hadoop cluster | Major | scripts | Eric Yang | Eric Yang | +| [HADOOP-7593](https://issues.apache.org/jira/browse/HADOOP-7593) | AssertionError in TestHttpServer.testMaxThreads() | Major | test | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HADOOP-7589](https://issues.apache.org/jira/browse/HADOOP-7589) | Prefer mvn test -DskipTests over mvn compile in test-patch.sh | Major | build | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-7576](https://issues.apache.org/jira/browse/HADOOP-7576) | Fix findbugs warnings in Hadoop Auth (Alfredo) | Major | security | Tom White | Tsz Wo Nicholas Sze | +| [HADOOP-7568](https://issues.apache.org/jira/browse/HADOOP-7568) | SequenceFile should not print into stdout | Major | io | Konstantin Shvachko | Plamen Jeliazkov | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7564](https://issues.apache.org/jira/browse/HADOOP-7564) | Remove test-patch SVN externals | Major | . | Tom White | Tom White | +| [HADOOP-7469](https://issues.apache.org/jira/browse/HADOOP-7469) | add a standard handler for socket connection problems which improves diagnostics | Minor | util | Steve Loughran | Steve Loughran | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7579](https://issues.apache.org/jira/browse/HADOOP-7579) | Rename package names from alfredo to auth | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/RELEASENOTES.0.24.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/RELEASENOTES.0.24.0.md new file mode 100644 index 00000000000..7aa2984c867 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.24.0/RELEASENOTES.0.24.0.md @@ -0,0 +1,52 @@ + + +# Apache Hadoop 0.24.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7728](https://issues.apache.org/jira/browse/HADOOP-7728) | *Major* | **hadoop-setup-conf.sh should be modified to enable task memory manager** + +Enable task memory management to be configurable via hadoop config setup script. + + +--- + +* [HADOOP-7668](https://issues.apache.org/jira/browse/HADOOP-7668) | *Minor* | **Add a NetUtils method that can tell if an InetAddress belongs to local host** + +closing again + + +--- + +* [HADOOP-7547](https://issues.apache.org/jira/browse/HADOOP-7547) | *Minor* | **Fix the warning in writable classes.[ WritableComparable is a raw type. References to generic type WritableComparable\ should be parameterized ]** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-7507](https://issues.apache.org/jira/browse/HADOOP-7507) | *Major* | **jvm metrics all use the same namespace** + +JVM metrics published to Ganglia now include the process name as part of the gmetric name. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/CHANGES.0.3.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/CHANGES.0.3.0.md new file mode 100644 index 00000000000..db6218fc240 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/CHANGES.0.3.0.md @@ -0,0 +1,102 @@ + + +# Apache Hadoop Changelog + +## Release 0.3.0 - 2006-06-02 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-256](https://issues.apache.org/jira/browse/HADOOP-256) | Implement a C api for hadoop dfs | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-233](https://issues.apache.org/jira/browse/HADOOP-233) | add a http status server for the task trackers | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-222](https://issues.apache.org/jira/browse/HADOOP-222) | Set replication from dfsshell | Trivial | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-209](https://issues.apache.org/jira/browse/HADOOP-209) | Add a program to recursively copy directories across file systems | Major | fs | Milind Bhandarkar | | +| [HADOOP-115](https://issues.apache.org/jira/browse/HADOOP-115) | permit reduce input types to differ from reduce output types | Major | . | Runping Qi | Runping Qi | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-254](https://issues.apache.org/jira/browse/HADOOP-254) | use http to shuffle data between the maps and the reduces | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-218](https://issues.apache.org/jira/browse/HADOOP-218) | Inefficient calls to get configuration values in TaskInprogress | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-216](https://issues.apache.org/jira/browse/HADOOP-216) | Task Detail web page missing progress | Trivial | . | Bryan Pendleton | Doug Cutting | +| [HADOOP-212](https://issues.apache.org/jira/browse/HADOOP-212) | allow changes to dfs block size | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-211](https://issues.apache.org/jira/browse/HADOOP-211) | logging improvements for Hadoop | Minor | . | Sameer Paranjpye | Sameer Paranjpye | +| [HADOOP-208](https://issues.apache.org/jira/browse/HADOOP-208) | add failure page to webapp | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-204](https://issues.apache.org/jira/browse/HADOOP-204) | Need to tweak a few things in the metrics package to support the Simon plugin | Major | metrics | David Bowen | David Bowen | +| [HADOOP-202](https://issues.apache.org/jira/browse/HADOOP-202) | sort should use a smaller number of reduces | Trivial | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-195](https://issues.apache.org/jira/browse/HADOOP-195) | improve performance of map output transfers | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-75](https://issues.apache.org/jira/browse/HADOOP-75) | dfs should check full file availability only at close | Minor | . | Doug Cutting | Milind Bhandarkar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-270](https://issues.apache.org/jira/browse/HADOOP-270) | possible deadlock when shut down a datanode thread | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-265](https://issues.apache.org/jira/browse/HADOOP-265) | Abort tasktracker if it can not write to its local directories | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-264](https://issues.apache.org/jira/browse/HADOOP-264) | WritableFactory has no permissions to create DatanodeRegistration | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-259](https://issues.apache.org/jira/browse/HADOOP-259) | map output http client does not timeout | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-251](https://issues.apache.org/jira/browse/HADOOP-251) | progress report failures kill task | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-247](https://issues.apache.org/jira/browse/HADOOP-247) | The Reduce Task thread for reporting progress during the sort exits in case of any IOException | Critical | . | Mahadev konar | Mahadev konar | +| [HADOOP-241](https://issues.apache.org/jira/browse/HADOOP-241) | TestCopyFiles fails under cygwin due to incorrect path | Minor | fs | Konstantin Shvachko | Milind Bhandarkar | +| [HADOOP-238](https://issues.apache.org/jira/browse/HADOOP-238) | map outputs transfers fail with EOFException | Blocker | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-235](https://issues.apache.org/jira/browse/HADOOP-235) | LocalFileSystem.openRaw() throws the wrong string for FileNotFoundException | Major | . | Benjamin Reed | | +| [HADOOP-229](https://issues.apache.org/jira/browse/HADOOP-229) | hadoop cp should generate a better number of map tasks | Minor | fs | Yoram Arnon | Milind Bhandarkar | +| [HADOOP-228](https://issues.apache.org/jira/browse/HADOOP-228) | hadoop cp should have a -config option | Minor | fs | Yoram Arnon | Milind Bhandarkar | +| [HADOOP-219](https://issues.apache.org/jira/browse/HADOOP-219) | SequenceFile#handleChecksumException NPE | Trivial | io | stack | Doug Cutting | +| [HADOOP-217](https://issues.apache.org/jira/browse/HADOOP-217) | IllegalAcessException when creating a Block object via WritableFactories | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-205](https://issues.apache.org/jira/browse/HADOOP-205) | the job tracker does not schedule enough map on the cluster | Major | . | Owen O'Malley | Mahadev konar | +| [HADOOP-200](https://issues.apache.org/jira/browse/HADOOP-200) | The map task names are sent to the reduces | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-180](https://issues.apache.org/jira/browse/HADOOP-180) | task tracker times out cleaning big job | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-179](https://issues.apache.org/jira/browse/HADOOP-179) | task tracker ghosts remain after 10 minutes | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-163](https://issues.apache.org/jira/browse/HADOOP-163) | If a DFS datanode cannot write onto its file system. it should tell the name node not to assign new blocks to it. | Major | . | Runping Qi | Hairong Kuang | +| [HADOOP-161](https://issues.apache.org/jira/browse/HADOOP-161) | dfs blocks define equal, but not hashcode | Major | . | Owen O'Malley | Milind Bhandarkar | +| [HADOOP-146](https://issues.apache.org/jira/browse/HADOOP-146) | potential conflict in block id's, leading to data corruption | Major | . | Yoram Arnon | Konstantin Shvachko | +| [HADOOP-141](https://issues.apache.org/jira/browse/HADOOP-141) | Disk thrashing / task timeouts during map output copy phase | Major | . | p sutter | Owen O'Malley | +| [HADOOP-124](https://issues.apache.org/jira/browse/HADOOP-124) | don't permit two datanodes to run from same dfs.data.dir | Critical | . | Bryan Pendleton | Konstantin Shvachko | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-220](https://issues.apache.org/jira/browse/HADOOP-220) | Add -dfs and -jt command-line parameters to specify namenode and jobtracker. | Major | fs | Milind Bhandarkar | Milind Bhandarkar | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/RELEASENOTES.0.3.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/RELEASENOTES.0.3.0.md new file mode 100644 index 00000000000..e648a8ec3a1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.0/RELEASENOTES.0.3.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.3.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/CHANGES.0.3.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/CHANGES.0.3.1.md new file mode 100644 index 00000000000..ba7aebfd170 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/CHANGES.0.3.1.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 0.3.1 - 2006-06-05 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-276](https://issues.apache.org/jira/browse/HADOOP-276) | No appenders could be found for logger | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-274](https://issues.apache.org/jira/browse/HADOOP-274) | The new logging framework puts application logs into server directory in hadoop.log | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-272](https://issues.apache.org/jira/browse/HADOOP-272) | bin/hadoop dfs -rm \ crashes in log4j code | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-262](https://issues.apache.org/jira/browse/HADOOP-262) | the reduce tasks do not report progress if they the map output locations is empty. | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-245](https://issues.apache.org/jira/browse/HADOOP-245) | record io translator doesn't strip path names | Major | record | Owen O'Malley | Milind Bhandarkar | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/RELEASENOTES.0.3.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/RELEASENOTES.0.3.1.md new file mode 100644 index 00000000000..de4e1cd3466 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.1/RELEASENOTES.0.3.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.3.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/CHANGES.0.3.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/CHANGES.0.3.2.md new file mode 100644 index 00000000000..6b67b0ad05e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/CHANGES.0.3.2.md @@ -0,0 +1,77 @@ + + +# Apache Hadoop Changelog + +## Release 0.3.2 - 2006-06-09 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-275](https://issues.apache.org/jira/browse/HADOOP-275) | log4j changes for hadoopStreaming | Major | . | Michel Tourn | Doug Cutting | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-279](https://issues.apache.org/jira/browse/HADOOP-279) | running without the hadoop script causes warnings about log4j not being configured correctly | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-269](https://issues.apache.org/jira/browse/HADOOP-269) | add FAQ to Wiki | Major | documentation | Doug Cutting | Doug Cutting | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-294](https://issues.apache.org/jira/browse/HADOOP-294) | dfs client error retries aren't happening (already being created and not replicated yet) | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-292](https://issues.apache.org/jira/browse/HADOOP-292) | hadoop dfs commands should not output superfluous data to stdout | Minor | . | Yoram Arnon | Owen O'Malley | +| [HADOOP-289](https://issues.apache.org/jira/browse/HADOOP-289) | Datanodes need to catch SocketTimeoutException and UnregisteredDatanodeException | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-285](https://issues.apache.org/jira/browse/HADOOP-285) | Data nodes cannot re-join the cluster once connection is lost | Blocker | . | Konstantin Shvachko | Hairong Kuang | +| [HADOOP-284](https://issues.apache.org/jira/browse/HADOOP-284) | dfs timeout on open | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-282](https://issues.apache.org/jira/browse/HADOOP-282) | the datanode crashes if it starts before the namenode | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-280](https://issues.apache.org/jira/browse/HADOOP-280) | AllTestDriver has incorrect class name for DistributedFSCheck test | Major | . | Konstantin Shvachko | | +| [HADOOP-277](https://issues.apache.org/jira/browse/HADOOP-277) | Race condition in Configuration.getLocalPath() | Major | . | p sutter | Sameer Paranjpye | +| [HADOOP-242](https://issues.apache.org/jira/browse/HADOOP-242) | job fails because of "No valid local directories in property: " exception | Major | . | Yoram Arnon | Owen O'Malley | +| [HADOOP-240](https://issues.apache.org/jira/browse/HADOOP-240) | namenode should not log failed mkdirs at warning level | Minor | . | Hairong Kuang | Hairong Kuang | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/RELEASENOTES.0.3.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/RELEASENOTES.0.3.2.md new file mode 100644 index 00000000000..de9320f3ed7 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.3.2/RELEASENOTES.0.3.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.3.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/CHANGES.0.4.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/CHANGES.0.4.0.md new file mode 100644 index 00000000000..c5a6a5d19b6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/CHANGES.0.4.0.md @@ -0,0 +1,88 @@ + + +# Apache Hadoop Changelog + +## Release 0.4.0 - 2006-06-28 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-296](https://issues.apache.org/jira/browse/HADOOP-296) | Do not assign blocks to a datanode with \< x mb free | Major | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-250](https://issues.apache.org/jira/browse/HADOOP-250) | HTTP Browsing interface for DFS Health/Status | Major | . | Devaraj Das | | +| [HADOOP-123](https://issues.apache.org/jira/browse/HADOOP-123) | mini map/reduce cluster for junit tests | Major | . | Owen O'Malley | Milind Bhandarkar | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-328](https://issues.apache.org/jira/browse/HADOOP-328) | add a -i option to distcp to ignore read errors of the input files | Major | util | Owen O'Malley | Owen O'Malley | +| [HADOOP-326](https://issues.apache.org/jira/browse/HADOOP-326) | cleanup of dead field (map ouput port) | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-314](https://issues.apache.org/jira/browse/HADOOP-314) | remove the append phase in sorting the reduce inputs | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-305](https://issues.apache.org/jira/browse/HADOOP-305) | tasktracker waits for 10 seconds for asking for a task. | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-301](https://issues.apache.org/jira/browse/HADOOP-301) | the randomwriter example will clobber the output file | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-298](https://issues.apache.org/jira/browse/HADOOP-298) | nicer reports of progress for distcp | Minor | util | Owen O'Malley | Owen O'Malley | +| [HADOOP-271](https://issues.apache.org/jira/browse/HADOOP-271) | add links to task tracker http server from task details and failure pages | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-59](https://issues.apache.org/jira/browse/HADOOP-59) | support generic command-line options | Minor | conf | Doug Cutting | Hairong Kuang | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-325](https://issues.apache.org/jira/browse/HADOOP-325) | ClassNotFoundException under jvm 1.6 | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-319](https://issues.apache.org/jira/browse/HADOOP-319) | FileSystem "close" does not remove the closed fs from the fs map | Major | fs | Hairong Kuang | Hairong Kuang | +| [HADOOP-318](https://issues.apache.org/jira/browse/HADOOP-318) | Progress in writing a DFS file does not count towards Job progress and can make the task timeout | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-317](https://issues.apache.org/jira/browse/HADOOP-317) | "connection was forcibly closed" Exception in RPC on Windows | Major | ipc | Konstantin Shvachko | Doug Cutting | +| [HADOOP-316](https://issues.apache.org/jira/browse/HADOOP-316) | job tracker has a deadlock | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-311](https://issues.apache.org/jira/browse/HADOOP-311) | dfs client timeout on read kills task | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-304](https://issues.apache.org/jira/browse/HADOOP-304) | UnregisteredDatanodeException message correction | Trivial | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-299](https://issues.apache.org/jira/browse/HADOOP-299) | maps from second jobs will not run until the first job finishes completely | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-278](https://issues.apache.org/jira/browse/HADOOP-278) | a missing map/reduce input directory does not produce a user-visible error message | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-210](https://issues.apache.org/jira/browse/HADOOP-210) | Namenode not able to accept connections | Major | . | Mahadev konar | Devaraj Das | +| [HADOOP-135](https://issues.apache.org/jira/browse/HADOOP-135) | Potential deadlock in JobTracker. | Major | . | Konstantin Shvachko | Owen O'Malley | +| [HADOOP-99](https://issues.apache.org/jira/browse/HADOOP-99) | task trackers can only be assigned one task every heartbeat | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-27](https://issues.apache.org/jira/browse/HADOOP-27) | MapRed tries to allocate tasks to nodes that have no available disk space | Major | . | Mike Cafarella | | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/RELEASENOTES.0.4.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/RELEASENOTES.0.4.0.md new file mode 100644 index 00000000000..36576ddef4b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.4.0/RELEASENOTES.0.4.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.4.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/CHANGES.0.5.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/CHANGES.0.5.0.md new file mode 100644 index 00000000000..1aeccf97ee9 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/CHANGES.0.5.0.md @@ -0,0 +1,122 @@ + + +# Apache Hadoop Changelog + +## Release 0.5.0 - 2006-08-04 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-425](https://issues.apache.org/jira/browse/HADOOP-425) | a python word count example that runs under jython | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-412](https://issues.apache.org/jira/browse/HADOOP-412) | provide an input format that fetches a subset of sequence file records | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-386](https://issues.apache.org/jira/browse/HADOOP-386) | Periodically move blocks from full nodes to those with space | Major | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-381](https://issues.apache.org/jira/browse/HADOOP-381) | keeping files for tasks that match regex on task id | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-369](https://issues.apache.org/jira/browse/HADOOP-369) | Added ability to copy all part-files into one output file | Trivial | . | Johan Oskarsson | Johan Oskarsson | +| [HADOOP-359](https://issues.apache.org/jira/browse/HADOOP-359) | add optional compression of map outputs | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-347](https://issues.apache.org/jira/browse/HADOOP-347) | Implement HDFS content browsing interface | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-342](https://issues.apache.org/jira/browse/HADOOP-342) | Design/Implement a tool to support archival and analysis of logfiles. | Major | . | Arun C Murthy | | +| [HADOOP-339](https://issues.apache.org/jira/browse/HADOOP-339) | making improvements to the jobclients to get information on currenlyl running jobs and the jobqueue | Minor | . | Mahadev konar | Mahadev konar | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-410](https://issues.apache.org/jira/browse/HADOOP-410) | Using HashMap instead of TreeMap for some maps in Namenode yields 17% performance improvement | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-409](https://issues.apache.org/jira/browse/HADOOP-409) | expose JobConf properties as environment variables | Major | . | Michel Tourn | | +| [HADOOP-396](https://issues.apache.org/jira/browse/HADOOP-396) | Writable DatanodeID | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-395](https://issues.apache.org/jira/browse/HADOOP-395) | infoPort field should be a DatanodeID member | Major | . | Konstantin Shvachko | Devaraj Das | +| [HADOOP-394](https://issues.apache.org/jira/browse/HADOOP-394) | MiniDFSCluster shudown order | Minor | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-392](https://issues.apache.org/jira/browse/HADOOP-392) | Improve the UI for DFS content browsing | Major | . | Devaraj Das | | +| [HADOOP-361](https://issues.apache.org/jira/browse/HADOOP-361) | junit with pure-Java hadoopStreaming combiner; remove CRLF in some files | Major | . | Michel Tourn | | +| [HADOOP-356](https://issues.apache.org/jira/browse/HADOOP-356) | Build and test hadoopStreaming nightly | Major | . | Michel Tourn | | +| [HADOOP-355](https://issues.apache.org/jira/browse/HADOOP-355) | hadoopStreaming: fix APIs, -reduce NONE, StreamSequenceRecordReader | Major | . | Michel Tourn | | +| [HADOOP-345](https://issues.apache.org/jira/browse/HADOOP-345) | JobConf access to name-values | Major | . | Michel Tourn | Michel Tourn | +| [HADOOP-341](https://issues.apache.org/jira/browse/HADOOP-341) | Enhance distcp to handle \*http\* as a 'source protocol'. | Major | util | Arun C Murthy | Arun C Murthy | +| [HADOOP-340](https://issues.apache.org/jira/browse/HADOOP-340) | Using wildcards in config pathnames | Minor | conf | Johan Oskarsson | Doug Cutting | +| [HADOOP-335](https://issues.apache.org/jira/browse/HADOOP-335) | factor out the namespace image/transaction log writing | Major | . | Owen O'Malley | Konstantin Shvachko | +| [HADOOP-321](https://issues.apache.org/jira/browse/HADOOP-321) | DatanodeInfo refactoring | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-302](https://issues.apache.org/jira/browse/HADOOP-302) | class Text (replacement for class UTF8) was: HADOOP-136 | Major | io | Michel Tourn | Hairong Kuang | +| [HADOOP-260](https://issues.apache.org/jira/browse/HADOOP-260) | the start up scripts should take a command line parameter --config making it easy to run multiple hadoop installation on same machines | Minor | . | Mahadev konar | Milind Bhandarkar | +| [HADOOP-252](https://issues.apache.org/jira/browse/HADOOP-252) | add versioning to RPC | Major | ipc | Yoram Arnon | | +| [HADOOP-237](https://issues.apache.org/jira/browse/HADOOP-237) | Standard set of Performance Metrics for Hadoop | Major | metrics | Milind Bhandarkar | Milind Bhandarkar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-415](https://issues.apache.org/jira/browse/HADOOP-415) | DFSNodesStatus() should sort data nodes. | Major | . | Konstantin Shvachko | | +| [HADOOP-404](https://issues.apache.org/jira/browse/HADOOP-404) | Regression tests are not working. | Major | . | Mahadev konar | | +| [HADOOP-393](https://issues.apache.org/jira/browse/HADOOP-393) | The validateUTF function of class Text throws MalformedInputException for valid UTF8 code containing ascii chars | Major | io | Hairong Kuang | Hairong Kuang | +| [HADOOP-391](https://issues.apache.org/jira/browse/HADOOP-391) | test-contrib with spaces in classpath (Windows) | Major | . | Michel Tourn | | +| [HADOOP-389](https://issues.apache.org/jira/browse/HADOOP-389) | MiniMapReduce tests get stuck because of some timing issues with initialization of tasktrackers. | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-388](https://issues.apache.org/jira/browse/HADOOP-388) | the hadoop-daemons.sh fails with "no such file or directory" when used from a relative path | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-387](https://issues.apache.org/jira/browse/HADOOP-387) | LocalJobRunner assigns duplicate mapid's | Major | . | Sami Siren | | +| [HADOOP-385](https://issues.apache.org/jira/browse/HADOOP-385) | rcc does not generate correct Java code for the field of a record type | Major | . | Hairong Kuang | Milind Bhandarkar | +| [HADOOP-384](https://issues.apache.org/jira/browse/HADOOP-384) | improved error messages for file checksum errors | Minor | fs | Owen O'Malley | Owen O'Malley | +| [HADOOP-383](https://issues.apache.org/jira/browse/HADOOP-383) | unit tests fail on windows | Major | . | Owen O'Malley | Michel Tourn | +| [HADOOP-380](https://issues.apache.org/jira/browse/HADOOP-380) | The reduce tasks poll for mapoutputs in a loop | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-377](https://issues.apache.org/jira/browse/HADOOP-377) | Configuration does not handle URL | Major | . | Jean-Baptiste Quenot | | +| [HADOOP-376](https://issues.apache.org/jira/browse/HADOOP-376) | Datanode does not scan for an open http port | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-375](https://issues.apache.org/jira/browse/HADOOP-375) | Introduce a way for datanodes to register their HTTP info ports with the NameNode | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-368](https://issues.apache.org/jira/browse/HADOOP-368) | DistributedFSCheck should cleanup, seek, and report missing files. | Minor | fs | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-365](https://issues.apache.org/jira/browse/HADOOP-365) | datanode crashes on startup with ClassCastException | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-364](https://issues.apache.org/jira/browse/HADOOP-364) | rpc versioning broke out-of-order server launches | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-362](https://issues.apache.org/jira/browse/HADOOP-362) | tasks can get lost when reporting task completion to the JobTracker has an error | Major | . | Devaraj Das | Owen O'Malley | +| [HADOOP-360](https://issues.apache.org/jira/browse/HADOOP-360) | hadoop-daemon starts but does not stop servers under cygWin | Major | . | Konstantin Shvachko | | +| [HADOOP-358](https://issues.apache.org/jira/browse/HADOOP-358) | NPE in Path.equals | Major | fs | FrĂ©dĂ©ric Bertin | Doug Cutting | +| [HADOOP-354](https://issues.apache.org/jira/browse/HADOOP-354) | All daemons should have public methods to start and stop them | Major | . | Barry Kaplan | | +| [HADOOP-352](https://issues.apache.org/jira/browse/HADOOP-352) | Portability of hadoop shell scripts for deployment | Major | . | Jean-Baptiste Quenot | | +| [HADOOP-350](https://issues.apache.org/jira/browse/HADOOP-350) | In standalone mode, 'org.apache.commons.cli cannot be resolved' | Minor | . | stack | | +| [HADOOP-344](https://issues.apache.org/jira/browse/HADOOP-344) | TaskTracker passes incorrect file path to DF under cygwin | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-327](https://issues.apache.org/jira/browse/HADOOP-327) | ToolBase calls System.exit | Major | util | Owen O'Malley | Hairong Kuang | +| [HADOOP-313](https://issues.apache.org/jira/browse/HADOOP-313) | A stand alone driver for individual tasks | Major | . | Michel Tourn | Michel Tourn | +| [HADOOP-226](https://issues.apache.org/jira/browse/HADOOP-226) | DFSShell problems. Incorrect block replication detection in fsck. | Major | . | Konstantin Shvachko | | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-418](https://issues.apache.org/jira/browse/HADOOP-418) | hadoopStreaming test jobconf -\> env.var. mapping | Major | . | Michel Tourn | | +| [HADOOP-411](https://issues.apache.org/jira/browse/HADOOP-411) | junit test for HADOOP-59: support generic command-line options | Major | . | Hairong Kuang | Hairong Kuang | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-351](https://issues.apache.org/jira/browse/HADOOP-351) | Remove Jetty dependency | Major | ipc | Barry Kaplan | Devaraj Das | +| [HADOOP-307](https://issues.apache.org/jira/browse/HADOOP-307) | Many small jobs benchmark for MapReduce | Minor | . | Sanjay Dahiya | Sanjay Dahiya | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/RELEASENOTES.0.5.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/RELEASENOTES.0.5.0.md new file mode 100644 index 00000000000..3fe30f53c35 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.5.0/RELEASENOTES.0.5.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.5.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/CHANGES.0.6.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/CHANGES.0.6.0.md new file mode 100644 index 00000000000..5cd65677c90 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/CHANGES.0.6.0.md @@ -0,0 +1,105 @@ + + +# Apache Hadoop Changelog + +## Release 0.6.0 - 2006-09-08 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-497](https://issues.apache.org/jira/browse/HADOOP-497) | DataNodes and TaskTrackers should be able to report hostnames and ips relative to customizable network interfaces and nameservers | Minor | util | Lorenzo Thione | Lorenzo Thione | +| [HADOOP-456](https://issues.apache.org/jira/browse/HADOOP-456) | Checkpointing and logging of data node descriptors | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-441](https://issues.apache.org/jira/browse/HADOOP-441) | SequenceFile should support 'custom compressors' | Major | io | Arun C Murthy | Arun C Murthy | +| [HADOOP-322](https://issues.apache.org/jira/browse/HADOOP-322) | Need a job control utility to submit and monitor a group of jobs which have DAG dependency | Major | . | Runping Qi | Runping Qi | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-499](https://issues.apache.org/jira/browse/HADOOP-499) | Avoid the use of Strings to improve the performance of hadoop streaming | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-488](https://issues.apache.org/jira/browse/HADOOP-488) | Change ToolBase.doMain() to return a status code | Major | . | Andrzej Bialecki | | +| [HADOOP-486](https://issues.apache.org/jira/browse/HADOOP-486) | adding username to jobstatus | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-483](https://issues.apache.org/jira/browse/HADOOP-483) | Document libhdfs and fix some OS specific stuff in Makefile | Major | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-474](https://issues.apache.org/jira/browse/HADOOP-474) | support compressed text files as input and output | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-468](https://issues.apache.org/jira/browse/HADOOP-468) | Setting scheduling priority in hadoop-env.sh | Minor | . | Vetle Roeim | | +| [HADOOP-464](https://issues.apache.org/jira/browse/HADOOP-464) | Troubleshooting message for RunJar (bin/hadoop jar) | Major | util | Michel Tourn | | +| [HADOOP-463](https://issues.apache.org/jira/browse/HADOOP-463) | variable expansion in Configuration | Major | conf | Michel Tourn | | +| [HADOOP-450](https://issues.apache.org/jira/browse/HADOOP-450) | Remove the need for users to specify the types of the inputs | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-437](https://issues.apache.org/jira/browse/HADOOP-437) | support gzip input files | Major | . | Michel Tourn | | +| [HADOOP-312](https://issues.apache.org/jira/browse/HADOOP-312) | Connections should not be cached | Major | ipc | Devaraj Das | Devaraj Das | +| [HADOOP-64](https://issues.apache.org/jira/browse/HADOOP-64) | DataNode should be capable of managing multiple volumes | Minor | . | Sameer Paranjpye | Milind Bhandarkar | +| [HADOOP-54](https://issues.apache.org/jira/browse/HADOOP-54) | SequenceFile should compress blocks, not individual entries | Major | io | Doug Cutting | Arun C Murthy | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-517](https://issues.apache.org/jira/browse/HADOOP-517) | readLine function of UTF8ByteArrayUtils does not handle end of line correctly | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-507](https://issues.apache.org/jira/browse/HADOOP-507) | Runtime exception in org.apache.hadoop.io.WritableFactories.newInstance when trying to startup namenode/datanode | Major | util | Arun C Murthy | Owen O'Malley | +| [HADOOP-501](https://issues.apache.org/jira/browse/HADOOP-501) | toString(resources, sb) in Configuration.java throws a ClassCastException since resources can be loaded from an URL | Minor | . | Thomas Friol | | +| [HADOOP-473](https://issues.apache.org/jira/browse/HADOOP-473) | TextInputFormat does not correctly handle all line endings | Major | . | Dennis Kubes | | +| [HADOOP-469](https://issues.apache.org/jira/browse/HADOOP-469) | Portability of hadoop shell scripts for deployment | Major | . | Jean-Baptiste Quenot | | +| [HADOOP-460](https://issues.apache.org/jira/browse/HADOOP-460) | Small jobs benchmark fails with current Hadoop due to UTF8 -\> Text ClassCastException | Major | . | Sanjay Dahiya | | +| [HADOOP-458](https://issues.apache.org/jira/browse/HADOOP-458) | libhdfs corrupts memory | Major | . | Christian Kunz | Konstantin Shvachko | +| [HADOOP-455](https://issues.apache.org/jira/browse/HADOOP-455) | Text class should support the DEL character | Major | io | Hairong Kuang | Hairong Kuang | +| [HADOOP-453](https://issues.apache.org/jira/browse/HADOOP-453) | bug in Text.setCapacity( int len ) | Minor | . | Sami Siren | | +| [HADOOP-434](https://issues.apache.org/jira/browse/HADOOP-434) | Use Hadoop scripts to run smallJobsBenchmark to avoid classpath issues. | Major | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-427](https://issues.apache.org/jira/browse/HADOOP-427) | DatanodeInfo class should be used instead of DatanodeDescriptor in the jsp & related files | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-426](https://issues.apache.org/jira/browse/HADOOP-426) | streaming unit tests failing on SunOS | Major | . | Doug Cutting | Michel Tourn | +| [HADOOP-424](https://issues.apache.org/jira/browse/HADOOP-424) | mapreduce jobs fail when no split is returned via inputFormat.getSplits | Major | . | FrĂ©dĂ©ric Bertin | | +| [HADOOP-421](https://issues.apache.org/jira/browse/HADOOP-421) | replace String in hadoop record io with the new Text class | Major | record | Owen O'Malley | Milind Bhandarkar | +| [HADOOP-419](https://issues.apache.org/jira/browse/HADOOP-419) | libdfs doesn't work with application threads | Major | . | Christian Kunz | Owen O'Malley | +| [HADOOP-400](https://issues.apache.org/jira/browse/HADOOP-400) | the job tracker re-runs failed tasks on the same node | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-324](https://issues.apache.org/jira/browse/HADOOP-324) | "IOException: No space left on device" is handled incorrectly | Major | . | Konstantin Shvachko | Wendy Chien | +| [HADOOP-320](https://issues.apache.org/jira/browse/HADOOP-320) | bin/hadoop dfs -mv does not mv source's checksum file if source is a file | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-286](https://issues.apache.org/jira/browse/HADOOP-286) | copyFromLocal throws LeaseExpiredException | Major | . | Runping Qi | Konstantin Shvachko | +| [HADOOP-281](https://issues.apache.org/jira/browse/HADOOP-281) | dfs.FSDirectory.mkdirs can create sub-directories of a file! | Major | . | Sameer Paranjpye | Wendy Chien | +| [HADOOP-196](https://issues.apache.org/jira/browse/HADOOP-196) | Fix buggy uselessness of Configuration( Configuration other) constructor | Major | conf | alan wootton | | +| [HADOOP-176](https://issues.apache.org/jira/browse/HADOOP-176) | comparators of integral writable types are not transitive for inequalities | Major | io | Dick King | | +| [HADOOP-50](https://issues.apache.org/jira/browse/HADOOP-50) | dfs datanode should store blocks in multiple directories | Major | . | Doug Cutting | Milind Bhandarkar | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-408](https://issues.apache.org/jira/browse/HADOOP-408) | Unit tests take too long to run. | Minor | . | Doug Cutting | | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/RELEASENOTES.0.6.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/RELEASENOTES.0.6.0.md new file mode 100644 index 00000000000..ca86d839ded --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.0/RELEASENOTES.0.6.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.6.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/CHANGES.0.6.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/CHANGES.0.6.1.md new file mode 100644 index 00000000000..031aa7fcb65 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/CHANGES.0.6.1.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 0.6.1 - 2006-09-13 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-529](https://issues.apache.org/jira/browse/HADOOP-529) | SequenceFile fails task with NullPointerException in codec initialization | Blocker | . | Sanjay Dahiya | Owen O'Malley | +| [HADOOP-526](https://issues.apache.org/jira/browse/HADOOP-526) | datanode lock message causes NullPointerException | Major | . | Owen O'Malley | Milind Bhandarkar | +| [HADOOP-523](https://issues.apache.org/jira/browse/HADOOP-523) | TextInputformat .isSplittable() fails with NullPointerException with hadoop 0.6.1 | Major | . | Sanjay Dahiya | Owen O'Malley | +| [HADOOP-521](https://issues.apache.org/jira/browse/HADOOP-521) | classloader problem for clients | Major | io | Christian Kunz | Owen O'Malley | +| [HADOOP-520](https://issues.apache.org/jira/browse/HADOOP-520) | libhdfs returns success even when writing to a file fails | Major | . | Christian Kunz | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/RELEASENOTES.0.6.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/RELEASENOTES.0.6.1.md new file mode 100644 index 00000000000..39da5b8a7e1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.1/RELEASENOTES.0.6.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.6.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/CHANGES.0.6.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/CHANGES.0.6.2.md new file mode 100644 index 00000000000..858c777ca7f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/CHANGES.0.6.2.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.6.2 - 2006-09-18 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-374](https://issues.apache.org/jira/browse/HADOOP-374) | native support for gzipped text files | Major | . | Yoram Arnon | | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-534](https://issues.apache.org/jira/browse/HADOOP-534) | Jobconf should set the default output value class to be Text | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-532](https://issues.apache.org/jira/browse/HADOOP-532) | Writable underrun in sort example | Major | io | Owen O'Malley | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/RELEASENOTES.0.6.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/RELEASENOTES.0.6.2.md new file mode 100644 index 00000000000..cd13ea2675b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.6.2/RELEASENOTES.0.6.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.6.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/CHANGES.0.7.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/CHANGES.0.7.0.md new file mode 100644 index 00000000000..95657c8d4ce --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/CHANGES.0.7.0.md @@ -0,0 +1,103 @@ + + +# Apache Hadoop Changelog + +## Release 0.7.0 - 2006-10-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-567](https://issues.apache.org/jira/browse/HADOOP-567) | The build script should record the Hadoop version into the build | Major | util | Owen O'Malley | Owen O'Malley | +| [HADOOP-559](https://issues.apache.org/jira/browse/HADOOP-559) | Support file patterns in dfs commands | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-542](https://issues.apache.org/jira/browse/HADOOP-542) | on-the-fly merge sort, HADOOP-540, reformat | Major | . | Michel Tourn | | +| [HADOOP-522](https://issues.apache.org/jira/browse/HADOOP-522) | MapFile should support block compression | Major | io | Doug Cutting | Doug Cutting | +| [HADOOP-519](https://issues.apache.org/jira/browse/HADOOP-519) | HDFS File API should be extended to include positional read | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-306](https://issues.apache.org/jira/browse/HADOOP-306) | Safe mode and name node startup procedures | Major | . | Konstantin Shvachko | Konstantin Shvachko | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-560](https://issues.apache.org/jira/browse/HADOOP-560) | tasks should have a "killed" state | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-556](https://issues.apache.org/jira/browse/HADOOP-556) | Streaming should send keep-alive signals to Reporter every 10 seconds, not every 100 records | Major | . | Michel Tourn | | +| [HADOOP-551](https://issues.apache.org/jira/browse/HADOOP-551) | reduce the number of lines printed to the console during execution | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-527](https://issues.apache.org/jira/browse/HADOOP-527) | Allow to specify the bind address for all hadoop servers | Major | . | Philippe Gassmann | | +| [HADOOP-487](https://issues.apache.org/jira/browse/HADOOP-487) | misspelt DFS host name gives null pointer exception in getProtocolVersion | Major | . | Dick King | Sameer Paranjpye | +| [HADOOP-431](https://issues.apache.org/jira/browse/HADOOP-431) | default behaviour of dfsShell -rm should resemble 'rm -i', not 'rm -rf' | Major | . | Yoram Arnon | Sameer Paranjpye | +| [HADOOP-263](https://issues.apache.org/jira/browse/HADOOP-263) | task status should include timestamps for when a job transitions | Major | . | Owen O'Malley | Sanjay Dahiya | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-581](https://issues.apache.org/jira/browse/HADOOP-581) | Datanode's offerService does not handle rpc timeouts | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-568](https://issues.apache.org/jira/browse/HADOOP-568) | small jobs benchmark fails: task is UNASSIGNED | Major | . | Yoram Arnon | Owen O'Malley | +| [HADOOP-566](https://issues.apache.org/jira/browse/HADOOP-566) | hadoop-daemons.sh fails with "no such file or directory" when used from a relative symlinked path | Major | scripts | Lee Faris | Doug Cutting | +| [HADOOP-552](https://issues.apache.org/jira/browse/HADOOP-552) | getMapOutput doesn't reliably detect errors and throw to the caller | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-550](https://issues.apache.org/jira/browse/HADOOP-550) | Text constructure can throw exception | Major | . | Bryan Pendleton | Hairong Kuang | +| [HADOOP-547](https://issues.apache.org/jira/browse/HADOOP-547) | ReduceTaskRunner can miss sending hearbeats if no map output copy finishes within "mapred.task.timeout" | Major | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-545](https://issues.apache.org/jira/browse/HADOOP-545) | Unused parameter in hadoop-default.xml | Trivial | conf | Philippe Gassmann | | +| [HADOOP-540](https://issues.apache.org/jira/browse/HADOOP-540) | Streaming should send keep-alive signals to Reporter | Major | . | Michel Tourn | | +| [HADOOP-537](https://issues.apache.org/jira/browse/HADOOP-537) | clean-libhdfs target of build.xml does not work on windows | Major | . | Konstantin Shvachko | Arun C Murthy | +| [HADOOP-536](https://issues.apache.org/jira/browse/HADOOP-536) | Broke ant test on windows | Major | . | Mahadev konar | | +| [HADOOP-533](https://issues.apache.org/jira/browse/HADOOP-533) | TestDFSShellGenericOptions creates a sub-directory in conf/ | Minor | . | Doug Cutting | Hairong Kuang | +| [HADOOP-530](https://issues.apache.org/jira/browse/HADOOP-530) | Error message does not expose mismached key or value class name correctly in Sequence file | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-513](https://issues.apache.org/jira/browse/HADOOP-513) | IllegalStateException is thrown by TaskTracker | Major | . | Konstantin Shvachko | Owen O'Malley | +| [HADOOP-508](https://issues.apache.org/jira/browse/HADOOP-508) | random seeks using FSDataInputStream can become invalid such that reads return invalid data | Major | . | Christian Kunz | Milind Bhandarkar | +| [HADOOP-506](https://issues.apache.org/jira/browse/HADOOP-506) | job tracker hangs on to dead task trackers "forever" | Minor | . | Yoram Arnon | Sanjay Dahiya | +| [HADOOP-444](https://issues.apache.org/jira/browse/HADOOP-444) | In streaming with a NONE reducer, you get duplicate files if a mapper fails, is restarted, and succeeds next time. | Major | . | Dick King | Michel Tourn | +| [HADOOP-438](https://issues.apache.org/jira/browse/HADOOP-438) | DFS pathname limitation. | Major | . | Konstantin Shvachko | Wendy Chien | +| [HADOOP-423](https://issues.apache.org/jira/browse/HADOOP-423) | file paths are not normalized | Major | . | Christian Kunz | Wendy Chien | +| [HADOOP-343](https://issues.apache.org/jira/browse/HADOOP-343) | In case of dead task tracker, the copy mapouts try copying all mapoutputs from this tasktracker | Major | . | Mahadev konar | Sameer Paranjpye | +| [HADOOP-293](https://issues.apache.org/jira/browse/HADOOP-293) | map reduce job fail without reporting a reason | Major | . | Yoram Arnon | Owen O'Malley | +| [HADOOP-288](https://issues.apache.org/jira/browse/HADOOP-288) | RFC: Efficient file caching | Major | . | Michel Tourn | Mahadev konar | +| [HADOOP-261](https://issues.apache.org/jira/browse/HADOOP-261) | when map outputs are lost, nothing is shown in the webapp about why the map task failed | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-255](https://issues.apache.org/jira/browse/HADOOP-255) | Client Calls are not cancelled after a call timeout | Major | ipc | Naveen Nalam | Owen O'Malley | +| [HADOOP-243](https://issues.apache.org/jira/browse/HADOOP-243) | WI shows progress as 100.00% before actual completion (rounding error) | Trivial | . | Yoram Arnon | Owen O'Malley | +| [HADOOP-239](https://issues.apache.org/jira/browse/HADOOP-239) | job tracker WI drops jobs after 24 hours | Minor | . | Yoram Arnon | Sanjay Dahiya | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-548](https://issues.apache.org/jira/browse/HADOOP-548) | add a switch to allow unit tests to show output | Minor | . | Owen O'Malley | Owen O'Malley | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/RELEASENOTES.0.7.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/RELEASENOTES.0.7.0.md new file mode 100644 index 00000000000..55530fea6f0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.0/RELEASENOTES.0.7.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.7.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/CHANGES.0.7.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/CHANGES.0.7.1.md new file mode 100644 index 00000000000..9fdf922ea7a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/CHANGES.0.7.1.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 0.7.1 - 2006-10-11 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-594](https://issues.apache.org/jira/browse/HADOOP-594) | Safemode default threshold should be 0.999 | Major | . | Konstantin Shvachko | Konstantin Shvachko | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-598](https://issues.apache.org/jira/browse/HADOOP-598) | rpc timeout in Task.done kills task | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-597](https://issues.apache.org/jira/browse/HADOOP-597) | transmission errors to the reduce will cause map output to be considered lost | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-593](https://issues.apache.org/jira/browse/HADOOP-593) | NullPointerException in JobTracker's ExireTaskTracker thread | Critical | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-592](https://issues.apache.org/jira/browse/HADOOP-592) | NullPointerException in toString for RPC connection objects | Minor | . | Owen O'Malley | Owen O'Malley | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/RELEASENOTES.0.7.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/RELEASENOTES.0.7.1.md new file mode 100644 index 00000000000..b4660b8e94d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.1/RELEASENOTES.0.7.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.7.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/CHANGES.0.7.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/CHANGES.0.7.2.md new file mode 100644 index 00000000000..49b17dcbeca --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/CHANGES.0.7.2.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.7.2 - 2006-10-18 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-666](https://issues.apache.org/jira/browse/HADOOP-666) | hadoop dfs does not complain about unknown option | Minor | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-607](https://issues.apache.org/jira/browse/HADOOP-607) | Classpath of tasks is not set correctly (bug happens for multiple mapred local dirs only) | Critical | . | Mahadev konar | Mahadev konar | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-609](https://issues.apache.org/jira/browse/HADOOP-609) | Mini map/reduce cluster should use multiple temp directories | Major | . | Owen O'Malley | Mahadev konar | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/RELEASENOTES.0.7.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/RELEASENOTES.0.7.2.md new file mode 100644 index 00000000000..747f6dad93c --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.7.2/RELEASENOTES.0.7.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.7.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/CHANGES.0.8.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/CHANGES.0.8.0.md new file mode 100644 index 00000000000..4e2e9ea66eb --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/CHANGES.0.8.0.md @@ -0,0 +1,104 @@ + + +# Apache Hadoop Changelog + +## Release 0.8.0 - 2006-11-03 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-636](https://issues.apache.org/jira/browse/HADOOP-636) | MapFile constructor should accept Progressible | Major | io | Doug Cutting | Doug Cutting | +| [HADOOP-635](https://issues.apache.org/jira/browse/HADOOP-635) | hadoop dfs copy, move commands should accept multiple source files as arguments | Major | . | dhruba borthakur | | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-670](https://issues.apache.org/jira/browse/HADOOP-670) | Generic types for FSNamesystem | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-660](https://issues.apache.org/jira/browse/HADOOP-660) | Format of junit output should be configurable | Minor | . | Nigel Daley | Nigel Daley | +| [HADOOP-647](https://issues.apache.org/jira/browse/HADOOP-647) | Map outputs can't have a different type of compression from the reduce outputs | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-626](https://issues.apache.org/jira/browse/HADOOP-626) | NNBench example comments are incorrect and code contains cut-and-paste error | Trivial | . | Nigel Daley | Nigel Daley | +| [HADOOP-625](https://issues.apache.org/jira/browse/HADOOP-625) | add a little servlet to display the server's thread call stacks | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-586](https://issues.apache.org/jira/browse/HADOOP-586) | Job Name should not ben empty, if its is not given bu user, Hadoop should use original Jar name as the job name. | Trivial | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-583](https://issues.apache.org/jira/browse/HADOOP-583) | DFSClient should use debug level to log missing data-node. | Minor | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-578](https://issues.apache.org/jira/browse/HADOOP-578) | Failed tasks should not be put at the end of the job tracker's queue | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-498](https://issues.apache.org/jira/browse/HADOOP-498) | fsck should execute faster | Major | . | Yoram Arnon | Milind Bhandarkar | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-671](https://issues.apache.org/jira/browse/HADOOP-671) | Distributed cache creates unnecessary symlinks if asked for creating symlinks | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-669](https://issues.apache.org/jira/browse/HADOOP-669) | Upgrade to trunk causes data loss. | Blocker | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-665](https://issues.apache.org/jira/browse/HADOOP-665) | hadoop -rmr does NOT process multiple arguments and does not complain | Minor | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-664](https://issues.apache.org/jira/browse/HADOOP-664) | build doesn't fail if libhdfs test(s) fail | Minor | . | Nigel Daley | Nigel Daley | +| [HADOOP-663](https://issues.apache.org/jira/browse/HADOOP-663) | ant test is failing | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-658](https://issues.apache.org/jira/browse/HADOOP-658) | source headers must conform to new Apache guidelines | Major | . | Doug Cutting | Doug Cutting | +| [HADOOP-651](https://issues.apache.org/jira/browse/HADOOP-651) | fsck does not handle arguments -blocks and -locations correctly | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-642](https://issues.apache.org/jira/browse/HADOOP-642) | Explicit timeout for ipc.Client | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-641](https://issues.apache.org/jira/browse/HADOOP-641) | Name-node should demand a block report from resurrected data-nodes. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-638](https://issues.apache.org/jira/browse/HADOOP-638) | TaskTracker missing synchronization around tasks variable access | Major | . | Nigel Daley | Nigel Daley | +| [HADOOP-634](https://issues.apache.org/jira/browse/HADOOP-634) | Test files missing copyright headers | Trivial | . | Nigel Daley | Nigel Daley | +| [HADOOP-633](https://issues.apache.org/jira/browse/HADOOP-633) | if the jobinit thread gets killed the jobtracker keeps running without doing anything. | Minor | . | Mahadev konar | Owen O'Malley | +| [HADOOP-627](https://issues.apache.org/jira/browse/HADOOP-627) | MiniMRCluster missing synchronization | Major | . | Nigel Daley | Nigel Daley | +| [HADOOP-624](https://issues.apache.org/jira/browse/HADOOP-624) | fix warning about pathSpec should start with '/' or '\*' : mapOutput | Minor | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-610](https://issues.apache.org/jira/browse/HADOOP-610) | Task Tracker offerService does not adequately protect from exceptions | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-599](https://issues.apache.org/jira/browse/HADOOP-599) | DFS Always Reports 0 Bytes Used | Minor | . | Albert Chern | Raghu Angadi | +| [HADOOP-588](https://issues.apache.org/jira/browse/HADOOP-588) | JobTracker History bug - kill() ed tasks are logged wrongly as finished. | Major | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-572](https://issues.apache.org/jira/browse/HADOOP-572) | Chain reaction in a big cluster caused by simultaneous failure of only a few data-nodes. | Major | . | Konstantin Shvachko | Sameer Paranjpye | +| [HADOOP-563](https://issues.apache.org/jira/browse/HADOOP-563) | DFS client should try to re-new lease if it gets a lease expiration exception when it adds a block to a file | Major | . | Runping Qi | dhruba borthakur | +| [HADOOP-561](https://issues.apache.org/jira/browse/HADOOP-561) | one replica of a file should be written locally if possible | Major | . | Yoram Arnon | dhruba borthakur | +| [HADOOP-554](https://issues.apache.org/jira/browse/HADOOP-554) | hadoop dfs command line doesn't exit with status code on error | Major | . | Marco Nicosia | dhruba borthakur | +| [HADOOP-553](https://issues.apache.org/jira/browse/HADOOP-553) | DataNode and NameNode main() should catch and report exceptions. | Major | . | Konstantin Shvachko | Raghu Angadi | +| [HADOOP-514](https://issues.apache.org/jira/browse/HADOOP-514) | namenode heartbeat interval should be configurable | Major | . | Wendy Chien | Milind Bhandarkar | +| [HADOOP-482](https://issues.apache.org/jira/browse/HADOOP-482) | unit test hangs when a cluster is running on the same machine | Minor | . | Wendy Chien | Wendy Chien | +| [HADOOP-477](https://issues.apache.org/jira/browse/HADOOP-477) | Streaming should execute Unix commands and scripts in well known languages without user specifying the path | Major | . | arkady borkovsky | dhruba borthakur | +| [HADOOP-462](https://issues.apache.org/jira/browse/HADOOP-462) | DFSShell throws out arrayoutofbounds exceptions if the number of arguments is not right | Minor | . | Mahadev konar | dhruba borthakur | +| [HADOOP-399](https://issues.apache.org/jira/browse/HADOOP-399) | the javadoc currently generates lot of warnings about bad fields | Trivial | . | Owen O'Malley | Nigel Daley | +| [HADOOP-373](https://issues.apache.org/jira/browse/HADOOP-373) | Some calls to mkdirs do not check return value | Major | . | Wendy Chien | Wendy Chien | +| [HADOOP-90](https://issues.apache.org/jira/browse/HADOOP-90) | DFS is succeptible to data loss in case of name node failure | Major | . | Yoram Arnon | Konstantin Shvachko | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/RELEASENOTES.0.8.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/RELEASENOTES.0.8.0.md new file mode 100644 index 00000000000..120f6a5f386 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.8.0/RELEASENOTES.0.8.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.8.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/CHANGES.0.9.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/CHANGES.0.9.0.md new file mode 100644 index 00000000000..d0247a55751 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/CHANGES.0.9.0.md @@ -0,0 +1,115 @@ + + +# Apache Hadoop Changelog + +## Release 0.9.0 - 2006-12-01 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-611](https://issues.apache.org/jira/browse/HADOOP-611) | SequenceFile.Sorter should have a merge method that returns an iterator | Major | io | Owen O'Malley | Devaraj Das | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-725](https://issues.apache.org/jira/browse/HADOOP-725) | chooseTargets method in FSNamesystem is very inefficient | Major | . | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-721](https://issues.apache.org/jira/browse/HADOOP-721) | jobconf.jsp shouldn't find the jobconf.xsl via http | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-689](https://issues.apache.org/jira/browse/HADOOP-689) | hadoop should provide a common way to wrap instances with different types into one type | Major | io | Feng Jiang | | +| [HADOOP-688](https://issues.apache.org/jira/browse/HADOOP-688) | move dfs administrative interfaces to a separate command | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-677](https://issues.apache.org/jira/browse/HADOOP-677) | RPC should send a fixed header and version at the start of connection | Major | ipc | Owen O'Malley | Owen O'Malley | +| [HADOOP-668](https://issues.apache.org/jira/browse/HADOOP-668) | improvement to DFS browsing WI | Minor | . | Yoram Arnon | Hairong Kuang | +| [HADOOP-661](https://issues.apache.org/jira/browse/HADOOP-661) | JobConf for a job should be viewable from the web/ui | Major | . | Owen O'Malley | Arun C Murthy | +| [HADOOP-655](https://issues.apache.org/jira/browse/HADOOP-655) | remove deprecations | Minor | . | Doug Cutting | Doug Cutting | +| [HADOOP-613](https://issues.apache.org/jira/browse/HADOOP-613) | The final merge on the reduces should feed the reduce directly | Major | . | Owen O'Malley | Devaraj Das | +| [HADOOP-565](https://issues.apache.org/jira/browse/HADOOP-565) | Upgrade Jetty to 6.x | Major | . | Owen O'Malley | Sanjay Dahiya | +| [HADOOP-538](https://issues.apache.org/jira/browse/HADOOP-538) | Implement a nio's 'direct buffer' based wrapper over zlib to improve performance of java.util.zip.{De\|In}flater as a 'custom codec' | Major | io | Arun C Murthy | Arun C Murthy | +| [HADOOP-489](https://issues.apache.org/jira/browse/HADOOP-489) | Seperating user logs from system logs in map reduce | Minor | . | Mahadev konar | Arun C Murthy | +| [HADOOP-76](https://issues.apache.org/jira/browse/HADOOP-76) | Implement speculative re-execution of reduces | Minor | . | Doug Cutting | Sanjay Dahiya | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-750](https://issues.apache.org/jira/browse/HADOOP-750) | race condition on stalled map output fetches | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-749](https://issues.apache.org/jira/browse/HADOOP-749) | The jobfailures.jsp gets a NullPointerException after a task tracker has been lost | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-747](https://issues.apache.org/jira/browse/HADOOP-747) | RecordIO compiler does not produce correct Java code when buffer is used as key or value in map | Major | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-741](https://issues.apache.org/jira/browse/HADOOP-741) | Fix average progress calculation in speculative execution of maps + fix pending issues in speculative execution of reduces | Major | . | Sanjay Dahiya | Sanjay Dahiya | +| [HADOOP-739](https://issues.apache.org/jira/browse/HADOOP-739) | TestIPC occassionally fails with BindException | Minor | test | Nigel Daley | Nigel Daley | +| [HADOOP-736](https://issues.apache.org/jira/browse/HADOOP-736) | Roll back Jetty6.0.1 to Jetty5.1.4 | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-734](https://issues.apache.org/jira/browse/HADOOP-734) | Link to the FAQ in the "Documentation" section of the hadoop website | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HADOOP-733](https://issues.apache.org/jira/browse/HADOOP-733) | dfs shell has inconsistent exit codes | Major | . | Christian Kunz | dhruba borthakur | +| [HADOOP-729](https://issues.apache.org/jira/browse/HADOOP-729) | packageNativeHadoop.sh has non-standard sh code | Critical | scripts | Arun C Murthy | Arun C Murthy | +| [HADOOP-728](https://issues.apache.org/jira/browse/HADOOP-728) | Map-reduce task does not produce correct results when -reducer NONE is specified through streaming | Major | . | dhruba borthakur | Sanjay Dahiya | +| [HADOOP-724](https://issues.apache.org/jira/browse/HADOOP-724) | bin/hadoop:111 uses java directly, it should use JAVA\_HOME | Minor | scripts | Sanjay Dahiya | Arun C Murthy | +| [HADOOP-723](https://issues.apache.org/jira/browse/HADOOP-723) | Race condition exists in the method MapOutputLocation.getFile | Major | . | Devaraj Das | Owen O'Malley | +| [HADOOP-722](https://issues.apache.org/jira/browse/HADOOP-722) | native-hadoop deficiencies | Critical | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-716](https://issues.apache.org/jira/browse/HADOOP-716) | Javadoc warning in SequenceFile.java | Minor | io | Devaraj Das | Devaraj Das | +| [HADOOP-715](https://issues.apache.org/jira/browse/HADOOP-715) | build.xml sets up wrong 'hadoop.log.dir' property for 'ant test' | Minor | test | Arun C Murthy | Arun C Murthy | +| [HADOOP-712](https://issues.apache.org/jira/browse/HADOOP-712) | Record-IO XML serialization is broken for control characters | Major | record | Milind Bhandarkar | Milind Bhandarkar | +| [HADOOP-710](https://issues.apache.org/jira/browse/HADOOP-710) | block size isn't honored | Major | . | Nigel Daley | Milind Bhandarkar | +| [HADOOP-709](https://issues.apache.org/jira/browse/HADOOP-709) | streaming job with Control characters in the command causes runtime exception in the job tracker | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-708](https://issues.apache.org/jira/browse/HADOOP-708) | test-libhdfs.sh does not properly capture and return error status | Major | test | Nigel Daley | Nigel Daley | +| [HADOOP-705](https://issues.apache.org/jira/browse/HADOOP-705) | IOException: job.xml already exists | Major | . | Nigel Daley | Mahadev konar | +| [HADOOP-699](https://issues.apache.org/jira/browse/HADOOP-699) | "Browse the filesystem" link on Name Node redirects to wrong port on DataNode | Major | . | Raghu Angadi | Raghu Angadi | +| [HADOOP-698](https://issues.apache.org/jira/browse/HADOOP-698) | When DFS client fails to read from a datanode, the failed datanode is not excluded from target reselection | Major | . | Hairong Kuang | Milind Bhandarkar | +| [HADOOP-696](https://issues.apache.org/jira/browse/HADOOP-696) | TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits() | Minor | test | Sameer Paranjpye | Sameer Paranjpye | +| [HADOOP-695](https://issues.apache.org/jira/browse/HADOOP-695) | Unexpected NPE from the next method of StreamLineRecordReader fails map/reduce jobs | Major | . | Hairong Kuang | Hairong Kuang | +| [HADOOP-694](https://issues.apache.org/jira/browse/HADOOP-694) | jobtracker expireluanching tasks throws out Nullpointer exceptions | Minor | . | Mahadev konar | Mahadev konar | +| [HADOOP-687](https://issues.apache.org/jira/browse/HADOOP-687) | Upgrade to Jetty 6 does not patch bin/hadoop | Critical | scripts | Sameer Paranjpye | Sameer Paranjpye | +| [HADOOP-683](https://issues.apache.org/jira/browse/HADOOP-683) | bin/hadoop.sh doesn't work for /bin/dash (eg ubuntu 6.10b) | Trivial | scripts | James Todd | | +| [HADOOP-682](https://issues.apache.org/jira/browse/HADOOP-682) | hadoop namenode -format doesnt work anymore if target directory doesnt exist | Major | . | Sanjay Dahiya | | +| [HADOOP-652](https://issues.apache.org/jira/browse/HADOOP-652) | Not all Datastructures are updated when a block is deleted | Major | . | Raghu Angadi | | +| [HADOOP-646](https://issues.apache.org/jira/browse/HADOOP-646) | name node server does not load large (\> 2^31 bytes) edits file | Critical | . | Christian Kunz | Milind Bhandarkar | +| [HADOOP-645](https://issues.apache.org/jira/browse/HADOOP-645) | Map-reduce task does not finish correctly when -reducer NONE is specified | Major | . | dhruba borthakur | dhruba borthakur | +| [HADOOP-637](https://issues.apache.org/jira/browse/HADOOP-637) | ipc.Server has memory leak -- serious issue for namenode server | Major | ipc | Christian Kunz | Raghu Angadi | +| [HADOOP-604](https://issues.apache.org/jira/browse/HADOOP-604) | DataNodes get NullPointerException and become unresponsive on 50010 | Major | . | Christian Kunz | Raghu Angadi | +| [HADOOP-459](https://issues.apache.org/jira/browse/HADOOP-459) | libhdfs leaks memory when writing to files | Major | . | Christian Kunz | Sameer Paranjpye | +| [HADOOP-447](https://issues.apache.org/jira/browse/HADOOP-447) | DistributedFileSystem.getBlockSize(Path) does not resolve absolute path | Major | . | Benjamin Reed | Raghu Angadi | +| [HADOOP-430](https://issues.apache.org/jira/browse/HADOOP-430) | http server should die if the Datanode fails. | Major | . | Konstantin Shvachko | Wendy Chien | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-382](https://issues.apache.org/jira/browse/HADOOP-382) | add a unit test for multiple datanodes in a machine | Major | . | Yoram Arnon | Milind Bhandarkar | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/RELEASENOTES.0.9.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/RELEASENOTES.0.9.0.md new file mode 100644 index 00000000000..e031d5b18fd --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.0/RELEASENOTES.0.9.0.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.9.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/CHANGES.0.9.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/CHANGES.0.9.1.md new file mode 100644 index 00000000000..925918f9eb6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/CHANGES.0.9.1.md @@ -0,0 +1,66 @@ + + +# Apache Hadoop Changelog + +## Release 0.9.1 - 2006-12-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-780](https://issues.apache.org/jira/browse/HADOOP-780) | ReduceTask.ValueIterator should apply job configuration to Configurable values | Major | . | Andrzej Bialecki | Andrzej Bialecki | +| [HADOOP-779](https://issues.apache.org/jira/browse/HADOOP-779) | Hadoop streaming does not work with gzipped input | Major | . | Hairong Kuang | Hairong Kuang | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/RELEASENOTES.0.9.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/RELEASENOTES.0.9.1.md new file mode 100644 index 00000000000..fad93aeae02 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.1/RELEASENOTES.0.9.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.9.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/CHANGES.0.9.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/CHANGES.0.9.2.md new file mode 100644 index 00000000000..e24821060c6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/CHANGES.0.9.2.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 0.9.2 - 2006-12-15 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-827](https://issues.apache.org/jira/browse/HADOOP-827) | turn off speculative execution by default | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-791](https://issues.apache.org/jira/browse/HADOOP-791) | deadlock issue in taskstracker. | Major | . | Mahadev konar | Mahadev konar | +| [HADOOP-639](https://issues.apache.org/jira/browse/HADOOP-639) | task cleanup messages can get lost, causing task trackers to keep tasks forever | Major | . | Owen O'Malley | Arun C Murthy | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/RELEASENOTES.0.9.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/RELEASENOTES.0.9.2.md new file mode 100644 index 00000000000..5e91e58d774 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/0.9.2/RELEASENOTES.0.9.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 0.9.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/CHANGES.1.0.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/CHANGES.1.0.0.md new file mode 100644 index 00000000000..3b3542015a0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/CHANGES.1.0.0.md @@ -0,0 +1,123 @@ + + +# Apache Hadoop Changelog + +## Release 1.0.0 - 2011-12-27 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-617](https://issues.apache.org/jira/browse/HDFS-617) | Support for non-recursive create() in HDFS | Major | hdfs-client, namenode | Kan Zhang | Kan Zhang | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2316](https://issues.apache.org/jira/browse/HDFS-2316) | [umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2137](https://issues.apache.org/jira/browse/HDFS-2137) | Datanode Disk Fail Inplace | Major | datanode | Bharath Mundlapudi | | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7804](https://issues.apache.org/jira/browse/HADOOP-7804) | enable hadoop config generator to set dfs.block.local-path-access.user to enable short circuit read | Major | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7710](https://issues.apache.org/jira/browse/HADOOP-7710) | create a script to setup application in order to create root directories for application such hbase, hcat, hive etc | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-7664](https://issues.apache.org/jira/browse/HADOOP-7664) | o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same. | Minor | conf | Ravi Prakash | Ravi Prakash | +| [HADOOP-6886](https://issues.apache.org/jira/browse/HADOOP-6886) | LocalFileSystem Needs createNonRecursive API | Minor | fs | Nicolas Spiegelberg | Nicolas Spiegelberg | +| [HADOOP-6840](https://issues.apache.org/jira/browse/HADOOP-6840) | Support non-recursive create() in FileSystem & SequenceFile.Writer | Minor | fs, io | Nicolas Spiegelberg | Nicolas Spiegelberg | +| [HADOOP-5124](https://issues.apache.org/jira/browse/HADOOP-5124) | A few optimizations to FsNamesystem#RecentInvalidateSets | Major | . | Hairong Kuang | Hairong Kuang | +| [HDFS-2604](https://issues.apache.org/jira/browse/HDFS-2604) | Add a log message to show if WebHDFS is enabled | Minor | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2246](https://issues.apache.org/jira/browse/HDFS-2246) | Shortcut a local client reads to a Datanodes files directly | Major | . | Sanjay Radia | Jitendra Nath Pandey | +| [MAPREDUCE-3169](https://issues.apache.org/jira/browse/MAPREDUCE-3169) | Create a new MiniMRCluster equivalent which only provides client APIs cross MR1 and MR2 | Major | mrv1, mrv2, test | Todd Lipcon | Ahmed Radwan | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9128](https://issues.apache.org/jira/browse/HADOOP-9128) | MetricsDynamicMBeanBase can cause high cpu load | Major | metrics | Nate Putnam | | +| [HADOOP-8389](https://issues.apache.org/jira/browse/HADOOP-8389) | MetricsDynamicMBeanBase throws IllegalArgumentException for empty attribute list | Major | metrics | Elias Ross | | +| [HADOOP-8067](https://issues.apache.org/jira/browse/HADOOP-8067) | Errors building hadoop-gpl-compression with Hadoop 1.0.0 and HBase 0.92.0 | Major | build | John Lewis | | +| [HADOOP-7903](https://issues.apache.org/jira/browse/HADOOP-7903) | hadoop artifacts do not contain 64 bit libhdfs native lib | Major | build | Arpit Gupta | Matt Foley | +| [HADOOP-7869](https://issues.apache.org/jira/browse/HADOOP-7869) | HADOOP\_HOME warning happens all of the time | Critical | scripts | Owen O'Malley | Owen O'Malley | +| [HADOOP-7865](https://issues.apache.org/jira/browse/HADOOP-7865) | Test Failures in 1.0.0 hdfs/common | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7854](https://issues.apache.org/jira/browse/HADOOP-7854) | UGI getCurrentUser is not synchronized | Critical | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-7816](https://issues.apache.org/jira/browse/HADOOP-7816) | Allow HADOOP\_HOME deprecated warning suppression based on config specified in hadoop-env.sh | Major | . | Dave Thompson | Dave Thompson | +| [HADOOP-7815](https://issues.apache.org/jira/browse/HADOOP-7815) | Map memory mb is being incorrectly set by hadoop-setup-conf.sh | Minor | conf | Ramya Sunil | Ramya Sunil | +| [HADOOP-7784](https://issues.apache.org/jira/browse/HADOOP-7784) | secure datanodes fail to come up stating jsvc not found | Major | . | Arpit Gupta | Eric Yang | +| [HADOOP-7765](https://issues.apache.org/jira/browse/HADOOP-7765) | Debian package contain both system and tar ball layout | Major | build | Eric Yang | Eric Yang | +| [HADOOP-7740](https://issues.apache.org/jira/browse/HADOOP-7740) | security audit logger is not on by default, fix the log4j properties to enable the logger | Minor | conf | Arpit Gupta | Arpit Gupta | +| [HADOOP-7728](https://issues.apache.org/jira/browse/HADOOP-7728) | hadoop-setup-conf.sh should be modified to enable task memory manager | Major | conf | Ramya Sunil | Ramya Sunil | +| [HADOOP-7461](https://issues.apache.org/jira/browse/HADOOP-7461) | Jackson Dependency Not Declared in Hadoop POM | Major | build | Ron Bodkin | Giridharan Kesavan | +| [HADOOP-7116](https://issues.apache.org/jira/browse/HADOOP-7116) | raise contrib junit test jvm memory size to 512mb | Major | test | Owen O'Malley | Owen O'Malley | +| [HDFS-2673](https://issues.apache.org/jira/browse/HDFS-2673) | While Namenode processing the blocksBeingWrittenReport, it will log incorrect number blocks count | Trivial | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2590](https://issues.apache.org/jira/browse/HDFS-2590) | Some links in WebHDFS forrest doc do not work | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2589](https://issues.apache.org/jira/browse/HDFS-2589) | unnecessary hftp token fetch and renewal thread | Major | security | Daryn Sharp | Daryn Sharp | +| [HDFS-2450](https://issues.apache.org/jira/browse/HDFS-2450) | Only complete hostname is supported to access data via hdfs:// | Major | . | Rajit Saha | Daryn Sharp | +| [HDFS-2346](https://issues.apache.org/jira/browse/HDFS-2346) | TestHost2NodesMap & TestReplicasMap will fail depending upon execution order of test methods | Blocker | test | Uma Maheswara Rao G | Laxman | +| [HDFS-2065](https://issues.apache.org/jira/browse/HDFS-2065) | Fix NPE in DFSClient.getFileChecksum | Major | . | Bharath Mundlapudi | Uma Maheswara Rao G | +| [HDFS-1943](https://issues.apache.org/jira/browse/HDFS-1943) | fail to start datanode while start-dfs.sh is executed by root user | Blocker | scripts | Wei Yongjun | Matt Foley | +| [HDFS-1257](https://issues.apache.org/jira/browse/HDFS-1257) | Race condition on FSNamesystem#recentInvalidateSets introduced by HADOOP-5124 | Major | namenode | Ramkumar Vadali | Eric Payne | +| [HDFS-838](https://issues.apache.org/jira/browse/HDFS-838) | libhdfs causes a segfault due to race condition | Major | libhdfs | Brian Bockelman | | +| [HDFS-611](https://issues.apache.org/jira/browse/HDFS-611) | Heartbeats times from Datanodes increase when there are plenty of blocks to delete | Major | datanode | dhruba borthakur | Zheng Shao | +| [MAPREDUCE-4935](https://issues.apache.org/jira/browse/MAPREDUCE-4935) | Support timeout limitation to MRv1 job end notifications | Major | jobtracker, mrv1 | Olga Shen | Brad Liu | +| [MAPREDUCE-3480](https://issues.apache.org/jira/browse/MAPREDUCE-3480) | TestJvmReuse fails in 1.0 | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-3475](https://issues.apache.org/jira/browse/MAPREDUCE-3475) | JT can't renew its own tokens | Major | jobtracker | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3374](https://issues.apache.org/jira/browse/MAPREDUCE-3374) | src/c++/task-controller/configure is not set executable in the tarball and that prevents task-controller from rebuilding | Major | task-controller | Roman Shaposhnik | | +| [MAPREDUCE-3319](https://issues.apache.org/jira/browse/MAPREDUCE-3319) | multifilewc from hadoop examples seems to be broken in 0.20.205.0 | Blocker | examples | Roman Shaposhnik | Subroto Sanyal | +| [MAPREDUCE-1744](https://issues.apache.org/jira/browse/MAPREDUCE-1744) | DistributedCache creates its own FileSytem instance when adding a file/archive to the path | Major | . | Dick King | Dick King | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-2540](https://issues.apache.org/jira/browse/HDFS-2540) | Change WebHdfsFileSystem to two-step create/append | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2539](https://issues.apache.org/jira/browse/HDFS-2539) | Support doAs and GETHOMEDIRECTORY in webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2528](https://issues.apache.org/jira/browse/HDFS-2528) | webhdfs rest call to a secure dn fails when a token is sent | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2527](https://issues.apache.org/jira/browse/HDFS-2527) | Remove the use of Range header from webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2501](https://issues.apache.org/jira/browse/HDFS-2501) | add version prefix and root methods to webhdfs | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2494](https://issues.apache.org/jira/browse/HDFS-2494) | [webhdfs] When Getting the file using OP=OPEN with DN http address, ESTABLISHED sockets are growing. | Major | webhdfs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2453](https://issues.apache.org/jira/browse/HDFS-2453) | tail using a webhdfs uri throws an error | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2441](https://issues.apache.org/jira/browse/HDFS-2441) | webhdfs returns two content-type headers | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2439](https://issues.apache.org/jira/browse/HDFS-2439) | webhdfs open an invalid path leads to a 500 which states a npe, we should return a 404 with appropriate error message | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2432](https://issues.apache.org/jira/browse/HDFS-2432) | webhdfs setreplication api should return a 403 when called on a directory | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2428](https://issues.apache.org/jira/browse/HDFS-2428) | webhdfs api parameter validation should be better | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2427](https://issues.apache.org/jira/browse/HDFS-2427) | webhdfs mkdirs api call creates path with 777 permission, we should default it to 755 | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2424](https://issues.apache.org/jira/browse/HDFS-2424) | webhdfs liststatus json does not convert to a valid xml document | Major | webhdfs | Arpit Gupta | Tsz Wo Nicholas Sze | +| [HDFS-2416](https://issues.apache.org/jira/browse/HDFS-2416) | distcp with a webhdfs uri on a secure cluster fails | Major | webhdfs | Arpit Gupta | Jitendra Nath Pandey | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7923](https://issues.apache.org/jira/browse/HADOOP-7923) | Automatically update doc versions | Major | build, documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2552](https://issues.apache.org/jira/browse/HDFS-2552) | Add WebHdfs Forrest doc | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/RELEASENOTES.1.0.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/RELEASENOTES.1.0.0.md new file mode 100644 index 00000000000..0ec8c49b5f9 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.0/RELEASENOTES.1.0.0.md @@ -0,0 +1,72 @@ + + +# Apache Hadoop 1.0.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-7923](https://issues.apache.org/jira/browse/HADOOP-7923) | *Major* | **Automatically update doc versions** + +Docs version number is now automatically updated by reference to the build number. + + +--- + +* [HADOOP-7740](https://issues.apache.org/jira/browse/HADOOP-7740) | *Minor* | **security audit logger is not on by default, fix the log4j properties to enable the logger** + +Fixed security audit logger configuration. (Arpit Gupta via Eric Yang) + + +--- + +* [HADOOP-7728](https://issues.apache.org/jira/browse/HADOOP-7728) | *Major* | **hadoop-setup-conf.sh should be modified to enable task memory manager** + +Enable task memory management to be configurable via hadoop config setup script. + + +--- + +* [HDFS-2316](https://issues.apache.org/jira/browse/HDFS-2316) | *Major* | **[umbrella] WebHDFS: a complete FileSystem implementation for accessing HDFS over HTTP** + +Provide WebHDFS as a complete FileSystem implementation for accessing HDFS over HTTP. +Previous hftp feature was a read-only FileSystem and does not provide "write" accesses. + + +--- + +* [HDFS-2246](https://issues.apache.org/jira/browse/HDFS-2246) | *Major* | **Shortcut a local client reads to a Datanodes files directly** + +1. New configurations +a. dfs.block.local-path-access.user is the key in datanode configuration to specify the user allowed to do short circuit read. +b. dfs.client.read.shortcircuit is the key to enable short circuit read at the client side configuration. +c. dfs.client.read.shortcircuit.skip.checksum is the key to bypass checksum check at the client side. +2. By default none of the above are enabled and short circuit read will not kick in. +3. If security is on, the feature can be used only for user that has kerberos credentials at the client, therefore map reduce tasks cannot benefit from it in general. + + +--- + +* [HDFS-617](https://issues.apache.org/jira/browse/HDFS-617) | *Major* | **Support for non-recursive create() in HDFS** + +New DFSClient.create(...) allows option of not creating missing parent(s). + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/CHANGES.1.0.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/CHANGES.1.0.1.md new file mode 100644 index 00000000000..161abc0936f --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/CHANGES.1.0.1.md @@ -0,0 +1,78 @@ + + +# Apache Hadoop Changelog + +## Release 1.0.1 - 2012-02-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8037](https://issues.apache.org/jira/browse/HADOOP-8037) | Binary tarball does not preserve platform info for native builds, and RPMs fail to provide needed symlinks for libhadoop.so | Blocker | build | Matt Foley | Matt Foley | +| [HADOOP-7470](https://issues.apache.org/jira/browse/HADOOP-7470) | move up to Jackson 1.8.8 | Minor | util | Steve Loughran | Enis Soztutar | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8009](https://issues.apache.org/jira/browse/HADOOP-8009) | Create hadoop-client and hadoop-minicluster artifacts for downstream projects | Critical | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7987](https://issues.apache.org/jira/browse/HADOOP-7987) | Support setting the run-as user in unsecure mode | Major | security | Devaraj Das | Jitendra Nath Pandey | +| [HDFS-2814](https://issues.apache.org/jira/browse/HDFS-2814) | NamenodeMXBean does not account for svn revision in the version information | Minor | . | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-3607](https://issues.apache.org/jira/browse/MAPREDUCE-3607) | Port missing new API mapreduce lib classes to 1.x | Major | client | Tom White | Tom White | +| [MAPREDUCE-3184](https://issues.apache.org/jira/browse/MAPREDUCE-3184) | Improve handling of fetch failures when a tasktracker is not responding on HTTP | Major | jobtracker | Todd Lipcon | Todd Lipcon | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8052](https://issues.apache.org/jira/browse/HADOOP-8052) | Hadoop Metrics2 should emit Float.MAX\_VALUE (instead of Double.MAX\_VALUE) to avoid making Ganglia's gmetad core | Major | metrics | Varun Kapoor | Varun Kapoor | +| [HADOOP-8010](https://issues.apache.org/jira/browse/HADOOP-8010) | hadoop-config.sh spews error message when HADOOP\_HOME\_WARN\_SUPPRESS is set to true and HADOOP\_HOME is present | Minor | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-7988](https://issues.apache.org/jira/browse/HADOOP-7988) | Upper case in hostname part of the principals doesn't work with kerberos. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7964](https://issues.apache.org/jira/browse/HADOOP-7964) | Deadlock in class init. | Blocker | security, util | Kihwal Lee | Daryn Sharp | +| [HADOOP-7960](https://issues.apache.org/jira/browse/HADOOP-7960) | Port HADOOP-5203 to branch-1, build version comparison is too restrictive | Major | . | Giridharan Kesavan | Matt Foley | +| [HDFS-2379](https://issues.apache.org/jira/browse/HDFS-2379) | 0.20: Allow block reports to proceed without holding FSDataset lock | Critical | datanode | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3343](https://issues.apache.org/jira/browse/MAPREDUCE-3343) | TaskTracker Out of Memory because of distributed cache | Major | mrv1 | Ahmed Radwan | zhaoyunjiong | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/RELEASENOTES.1.0.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/RELEASENOTES.1.0.1.md new file mode 100644 index 00000000000..1745f2bfb8a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.1/RELEASENOTES.1.0.1.md @@ -0,0 +1,52 @@ + + +# Apache Hadoop 1.0.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8037](https://issues.apache.org/jira/browse/HADOOP-8037) | *Blocker* | **Binary tarball does not preserve platform info for native builds, and RPMs fail to provide needed symlinks for libhadoop.so** + +This fix is marked "incompatible" only because it changes the bin-tarball directory structure to be consistent with the source tarball directory structure. The source tarball is unchanged. RPMs and DEBs now use an intermediate bin-tarball with an "${os.arch}" tag (like the packages themselves). The un-tagged bin-tarball is now multi-platform and retains the structure of the source tarball; it is in fact generated by target "tar", not by target "binary". Finally, in the 64-bit RPMs and DEBs, the native libs go in the "lib64" directory instead of "lib". + + +--- + +* [HADOOP-8009](https://issues.apache.org/jira/browse/HADOOP-8009) | *Critical* | **Create hadoop-client and hadoop-minicluster artifacts for downstream projects** + +Generate integration artifacts "org.apache.hadoop:hadoop-client" and "org.apache.hadoop:hadoop-minicluster" containing all the jars needed to use Hadoop client APIs, and to run Hadoop MiniClusters, respectively. Push these artifacts to the maven repository when mvn-deploy, along with existing artifacts. + + +--- + +* [HADOOP-7470](https://issues.apache.org/jira/browse/HADOOP-7470) | *Minor* | **move up to Jackson 1.8.8** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-3184](https://issues.apache.org/jira/browse/MAPREDUCE-3184) | *Major* | **Improve handling of fetch failures when a tasktracker is not responding on HTTP** + +The TaskTracker now has a thread which monitors for a known Jetty bug in which the selector thread starts spinning and map output can no longer be served. If the bug is detected, the TaskTracker will shut itself down. This feature can be disabled by setting mapred.tasktracker.jetty.cpu.check.enabled to false. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/CHANGES.1.0.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/CHANGES.1.0.2.md new file mode 100644 index 00000000000..7c2cfab475e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/CHANGES.1.0.2.md @@ -0,0 +1,81 @@ + + +# Apache Hadoop Changelog + +## Release 1.0.2 - 2012-04-03 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7206](https://issues.apache.org/jira/browse/HADOOP-7206) | Integrate Snappy compression | Major | . | Eli Collins | Alejandro Abdelnur | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8090](https://issues.apache.org/jira/browse/HADOOP-8090) | rename hadoop 64 bit rpm/deb package name | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-5450](https://issues.apache.org/jira/browse/HADOOP-5450) | Add support for application-specific typecodes to typed bytes | Blocker | . | Klaas Bosteels | Klaas Bosteels | +| [HADOOP-1722](https://issues.apache.org/jira/browse/HADOOP-1722) | Make streaming to handle non-utf8 byte array | Major | . | Runping Qi | Klaas Bosteels | +| [HDFS-3075](https://issues.apache.org/jira/browse/HDFS-3075) | Backport HADOOP-4885 to branch-1 | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-2701](https://issues.apache.org/jira/browse/HDFS-2701) | Cleanup FS\* processIOError methods | Major | namenode | Eli Collins | Eli Collins | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8201](https://issues.apache.org/jira/browse/HADOOP-8201) | create the configure script for native compilation as part of the build | Blocker | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-8132](https://issues.apache.org/jira/browse/HADOOP-8132) | 64bit secure datanodes do not start as the jsvc path is wrong | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8050](https://issues.apache.org/jira/browse/HADOOP-8050) | Deadlock in metrics | Major | metrics | Kihwal Lee | Kihwal Lee | +| [HDFS-3101](https://issues.apache.org/jira/browse/HDFS-3101) | cannot read empty file using webhdfs | Major | webhdfs | Zhanwei Wang | Tsz Wo Nicholas Sze | +| [HDFS-3006](https://issues.apache.org/jira/browse/HDFS-3006) | Webhdfs "SETOWNER" call returns incorrect content-type | Major | webhdfs | bc Wong | Tsz Wo Nicholas Sze | +| [HDFS-2703](https://issues.apache.org/jira/browse/HDFS-2703) | removedStorageDirs is not updated everywhere we remove a storage dir | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-2702](https://issues.apache.org/jira/browse/HDFS-2702) | A single failed name dir can cause the NN to exit | Critical | namenode | Eli Collins | Eli Collins | +| [MAPREDUCE-3851](https://issues.apache.org/jira/browse/MAPREDUCE-3851) | Allow more aggressive action on detection of the jetty issue | Major | tasktracker | Kihwal Lee | Thomas Graves | +| [MAPREDUCE-3824](https://issues.apache.org/jira/browse/MAPREDUCE-3824) | Distributed caches are not removed properly | Critical | distributed-cache | Allen Wittenauer | Thomas Graves | +| [MAPREDUCE-3583](https://issues.apache.org/jira/browse/MAPREDUCE-3583) | ProcfsBasedProcessTree#constructProcessInfo() may throw NumberFormatException | Critical | . | Ted Yu | Ted Yu | +| [MAPREDUCE-764](https://issues.apache.org/jira/browse/MAPREDUCE-764) | TypedBytesInput's readRaw() does not preserve custom type codes | Blocker | contrib/streaming | Klaas Bosteels | Klaas Bosteels | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/RELEASENOTES.1.0.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/RELEASENOTES.1.0.2.md new file mode 100644 index 00000000000..5352f62e663 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.2/RELEASENOTES.1.0.2.md @@ -0,0 +1,49 @@ + + +# Apache Hadoop 1.0.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-1722](https://issues.apache.org/jira/browse/HADOOP-1722) | *Major* | **Make streaming to handle non-utf8 byte array** + +Streaming allows binary (or other non-UTF8) streams. + + +--- + +* [MAPREDUCE-3851](https://issues.apache.org/jira/browse/MAPREDUCE-3851) | *Major* | **Allow more aggressive action on detection of the jetty issue** + +added new configuration variables to control when TT aborts if it sees a certain number of exceptions: + + // Percent of shuffle exceptions (out of sample size) seen before it's + // fatal - acceptable values are from 0 to 1.0, 0 disables the check. + // ie. 0.3 = 30% of the last X number of requests matched the exception, + // so abort. + conf.getFloat( + "mapreduce.reduce.shuffle.catch.exception.percent.limit.fatal", 0); + + // The number of trailing requests we track, used for the fatal + // limit calculation + conf.getInt("mapreduce.reduce.shuffle.catch.exception.sample.size", 1000); + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/CHANGES.1.0.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/CHANGES.1.0.3.md new file mode 100644 index 00000000000..f12cd21ec5b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/CHANGES.1.0.3.md @@ -0,0 +1,90 @@ + + +# Apache Hadoop Changelog + +## Release 1.0.3 - 2012-05-07 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-5528](https://issues.apache.org/jira/browse/HADOOP-5528) | Binary partitioner | Major | . | Klaas Bosteels | Klaas Bosteels | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8352](https://issues.apache.org/jira/browse/HADOOP-8352) | We should always generate a new configure script for the c++ code | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-8188](https://issues.apache.org/jira/browse/HADOOP-8188) | Fix the build process to do with jsvc, with IBM's JDK as the underlying jdk | Major | . | Devaraj Das | Devaraj Das | +| [HADOOP-8027](https://issues.apache.org/jira/browse/HADOOP-8027) | Visiting /jmx on the daemon web interfaces may print unnecessary error in logs | Minor | metrics | Harsh J | Aaron T. Myers | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8346](https://issues.apache.org/jira/browse/HADOOP-8346) | Changes to support Kerberos with non Sun JVM (HADOOP-6941) broke SPNEGO | Blocker | security | Alejandro Abdelnur | Devaraj Das | +| [HADOOP-8338](https://issues.apache.org/jira/browse/HADOOP-8338) | Can't renew or cancel HDFS delegation tokens over secure RPC | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-8294](https://issues.apache.org/jira/browse/HADOOP-8294) | IPC Connection becomes unusable even if server address was temporarilly unresolvable | Critical | ipc | Kihwal Lee | Kihwal Lee | +| [HADOOP-8293](https://issues.apache.org/jira/browse/HADOOP-8293) | The native library's Makefile.am doesn't include JNI path | Major | build | Owen O'Malley | Owen O'Malley | +| [HADOOP-8251](https://issues.apache.org/jira/browse/HADOOP-8251) | SecurityUtil.fetchServiceTicket broken after HADOOP-6941 | Blocker | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-8151](https://issues.apache.org/jira/browse/HADOOP-8151) | Error handling in snappy decompressor throws invalid exceptions | Major | io, native | Todd Lipcon | Matt Foley | +| [HADOOP-7381](https://issues.apache.org/jira/browse/HADOOP-7381) | FindBugs OutOfMemoryError | Major | build | Joep Rottinghuis | Joep Rottinghuis | +| [HADOOP-6941](https://issues.apache.org/jira/browse/HADOOP-6941) | Support non-SUN JREs in UserGroupInformation | Major | . | Stephen Watt | Devaraj Das | +| [HADOOP-6924](https://issues.apache.org/jira/browse/HADOOP-6924) | Build fails with non-Sun JREs due to different pathing to the operating system architecture shared libraries | Major | . | Stephen Watt | Devaraj Das | +| [HDFS-3374](https://issues.apache.org/jira/browse/HDFS-3374) | hdfs' TestDelegationToken fails intermittently with a race condition | Major | namenode | Owen O'Malley | Owen O'Malley | +| [HDFS-3310](https://issues.apache.org/jira/browse/HDFS-3310) | Make sure that we abort when no edit log directories are left | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3265](https://issues.apache.org/jira/browse/HDFS-3265) | PowerPc Build error. | Major | build | Kumar Ravi | Kumar Ravi | +| [HDFS-3127](https://issues.apache.org/jira/browse/HDFS-3127) | failure in recovering removed storage directories should not stop checkpoint process | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-3061](https://issues.apache.org/jira/browse/HDFS-3061) | Backport HDFS-1487 to branch-1 | Blocker | namenode | Alex Holmes | Kihwal Lee | +| [HDFS-1041](https://issues.apache.org/jira/browse/HDFS-1041) | DFSClient does not retry in getFileChecksum(..) | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-119](https://issues.apache.org/jira/browse/HDFS-119) | logSync() may block NameNode forever. | Major | namenode | Konstantin Shvachko | Suresh Srinivas | +| [MAPREDUCE-4207](https://issues.apache.org/jira/browse/MAPREDUCE-4207) | Remove System.out.println() in FileInputFormat | Major | mrv1 | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-4154](https://issues.apache.org/jira/browse/MAPREDUCE-4154) | streaming MR job succeeds even if the streaming command fails | Major | . | Thejas M Nair | Devaraj Das | +| [MAPREDUCE-4003](https://issues.apache.org/jira/browse/MAPREDUCE-4003) | log.index (No such file or directory) AND Task process exit with nonzero status of 126 | Major | task-controller, tasktracker | toughman | Koji Noguchi | +| [MAPREDUCE-3857](https://issues.apache.org/jira/browse/MAPREDUCE-3857) | Grep example ignores mapred.job.queue.name | Major | examples | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-3377](https://issues.apache.org/jira/browse/MAPREDUCE-3377) | Compatibility issue with 0.20.203. | Major | . | Jane Chen | Jane Chen | +| [MAPREDUCE-1238](https://issues.apache.org/jira/browse/MAPREDUCE-1238) | mapred metrics shows negative count of waiting maps and reduces | Major | jobtracker | Ramya Sunil | Thomas Graves | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/RELEASENOTES.1.0.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/RELEASENOTES.1.0.3.md new file mode 100644 index 00000000000..7652dbf157e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.3/RELEASENOTES.1.0.3.md @@ -0,0 +1,39 @@ + + +# Apache Hadoop 1.0.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8352](https://issues.apache.org/jira/browse/HADOOP-8352) | *Major* | **We should always generate a new configure script for the c++ code** + +If you are compiling c++, the configure script will now be automatically regenerated as it should be. +This requires autoconf version 2.61 or greater. + + +--- + +* [HADOOP-5528](https://issues.apache.org/jira/browse/HADOOP-5528) | *Major* | **Binary partitioner** + +New BinaryPartitioner that partitions BinaryComparable keys by hashing a configurable part of the bytes array corresponding to the key. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/CHANGES.1.0.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/CHANGES.1.0.4.md new file mode 100644 index 00000000000..6325e46a6bd --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/CHANGES.1.0.4.md @@ -0,0 +1,67 @@ + + +# Apache Hadoop Changelog + +## Release 1.0.4 - 2012-10-12 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7154](https://issues.apache.org/jira/browse/HADOOP-7154) | Should set MALLOC\_ARENA\_MAX in hadoop-config.sh | Minor | scripts | Todd Lipcon | Todd Lipcon | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-3652](https://issues.apache.org/jira/browse/HDFS-3652) | 1.x: FSEditLog failure removes the wrong edit stream when storage dirs have same name | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-4399](https://issues.apache.org/jira/browse/MAPREDUCE-4399) | Fix performance regression in shuffle | Major | performance, tasktracker | Luke Lu | Luke Lu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/RELEASENOTES.1.0.4.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/RELEASENOTES.1.0.4.md new file mode 100644 index 00000000000..fa14781da61 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.0.4/RELEASENOTES.1.0.4.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 1.0.4 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/CHANGES.1.1.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/CHANGES.1.1.0.md new file mode 100644 index 00000000000..c4caca7de22 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/CHANGES.1.1.0.md @@ -0,0 +1,199 @@ + + +# Apache Hadoop Changelog + +## Release 1.1.0 - 2012-10-13 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8552](https://issues.apache.org/jira/browse/HADOOP-8552) | Conflict: Same security.log.file for multiple users. | Major | conf, security | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-8365](https://issues.apache.org/jira/browse/HADOOP-8365) | Add flag to disable durable sync | Blocker | . | Eli Collins | Eli Collins | +| [HADOOP-8314](https://issues.apache.org/jira/browse/HADOOP-8314) | HttpServer#hasAdminAccess should return false if authorization is enabled but user is not authenticated | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8230](https://issues.apache.org/jira/browse/HADOOP-8230) | Enable sync by default and disable append | Major | . | Eli Collins | Eli Collins | +| [HADOOP-8154](https://issues.apache.org/jira/browse/HADOOP-8154) | DNS#getIPs shouldn't silently return the local host IP for bogus interface names | Major | conf | Eli Collins | Eli Collins | +| [HADOOP-5464](https://issues.apache.org/jira/browse/HADOOP-5464) | DFSClient does not treat write timeout of 0 properly | Major | . | Raghu Angadi | Raghu Angadi | +| [HDFS-3522](https://issues.apache.org/jira/browse/HDFS-3522) | If NN is in safemode, it should throw SafeModeException when getBlockLocations has zero locations | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-3044](https://issues.apache.org/jira/browse/HDFS-3044) | fsck move should be non-destructive by default | Major | namenode | Eli Collins | Colin Patrick McCabe | +| [HDFS-2617](https://issues.apache.org/jira/browse/HDFS-2617) | Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution | Major | security | Jakob Homan | Jakob Homan | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7823](https://issues.apache.org/jira/browse/HADOOP-7823) | port HADOOP-4012 to branch-1 (splitting support for bzip2) | Major | . | Tim Broberg | Andrew Purtell | +| [HADOOP-7806](https://issues.apache.org/jira/browse/HADOOP-7806) | Support binding to sub-interfaces | Major | util | Harsh J | Harsh J | +| [HDFS-3150](https://issues.apache.org/jira/browse/HDFS-3150) | Add option for clients to contact DNs via hostname | Major | datanode, hdfs-client | Eli Collins | Eli Collins | +| [HDFS-3148](https://issues.apache.org/jira/browse/HDFS-3148) | The client should be able to use multiple local interfaces for data transfer | Major | hdfs-client, performance | Eli Collins | Eli Collins | +| [HDFS-3055](https://issues.apache.org/jira/browse/HDFS-3055) | Implement recovery mode for branch-1 | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [MAPREDUCE-3837](https://issues.apache.org/jira/browse/MAPREDUCE-3837) | Job tracker is not able to recover job in case of crash and after that no user can submit job. | Major | . | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-3118](https://issues.apache.org/jira/browse/MAPREDUCE-3118) | Backport Gridmix and Rumen features from trunk to Hadoop 0.20 security branch | Major | contrib/gridmix, tools/rumen | Ravi Gummadi | Ravi Gummadi | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8748](https://issues.apache.org/jira/browse/HADOOP-8748) | Move dfsclient retry to a util class | Minor | io | Arun C Murthy | Arun C Murthy | +| [HADOOP-8656](https://issues.apache.org/jira/browse/HADOOP-8656) | backport forced daemon shutdown of HADOOP-8353 into branch-1 | Minor | bin | Steve Loughran | Roman Shaposhnik | +| [HADOOP-8430](https://issues.apache.org/jira/browse/HADOOP-8430) | Backport new FileSystem methods introduced by HADOOP-8014 to branch-1 | Major | . | Eli Collins | Eli Collins | +| [HADOOP-8209](https://issues.apache.org/jira/browse/HADOOP-8209) | Add option to relax build-version check for branch-1 | Major | . | Eli Collins | Eli Collins | +| [HADOOP-7509](https://issues.apache.org/jira/browse/HADOOP-7509) | Improve message when Authentication is required | Trivial | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-6995](https://issues.apache.org/jira/browse/HADOOP-6995) | Allow wildcards to be used in ProxyUsers configurations | Minor | security | Todd Lipcon | Todd Lipcon | +| [HDFS-3871](https://issues.apache.org/jira/browse/HDFS-3871) | Change NameNodeProxies to use HADOOP-8748 | Minor | hdfs-client | Arun C Murthy | Arun C Murthy | +| [HDFS-3814](https://issues.apache.org/jira/browse/HDFS-3814) | Make the replication monitor multipliers configurable in 1.x | Major | namenode | Suresh Srinivas | Jing Zhao | +| [HDFS-3703](https://issues.apache.org/jira/browse/HDFS-3703) | Decrease the datanode failure detection time | Major | datanode, namenode | Nicolas Liochon | Jing Zhao | +| [HDFS-3667](https://issues.apache.org/jira/browse/HDFS-3667) | Add retry support to WebHdfsFileSystem | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3617](https://issues.apache.org/jira/browse/HDFS-3617) | Port HDFS-96 to branch-1 (support blocks greater than 2GB) | Major | . | Matt Foley | Harsh J | +| [HDFS-3596](https://issues.apache.org/jira/browse/HDFS-3596) | Improve FSEditLog pre-allocation in branch-1 | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3516](https://issues.apache.org/jira/browse/HDFS-3516) | Check content-type in WebHdfsFileSystem | Major | hdfs-client, webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3504](https://issues.apache.org/jira/browse/HDFS-3504) | Configurable retry in DFSClient | Major | hdfs-client | Siddharth Seth | Tsz Wo Nicholas Sze | +| [HDFS-3131](https://issues.apache.org/jira/browse/HDFS-3131) | Improve TestStorageRestore | Minor | . | Tsz Wo Nicholas Sze | Brandon Li | +| [HDFS-3094](https://issues.apache.org/jira/browse/HDFS-3094) | add -nonInteractive and -force option to namenode -format command | Major | . | Arpit Gupta | Arpit Gupta | +| [HDFS-2872](https://issues.apache.org/jira/browse/HDFS-2872) | Add sanity checks during edits loading that generation stamps are non-decreasing | Major | namenode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-2654](https://issues.apache.org/jira/browse/HDFS-2654) | Make BlockReaderLocal not extend RemoteBlockReader2 | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-2653](https://issues.apache.org/jira/browse/HDFS-2653) | DFSClient should cache whether addrs are non-local when short-circuiting is enabled | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-2638](https://issues.apache.org/jira/browse/HDFS-2638) | Improve a block recovery log | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-2465](https://issues.apache.org/jira/browse/HDFS-2465) | Add HDFS support for fadvise readahead and drop-behind | Major | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-1378](https://issues.apache.org/jira/browse/HDFS-1378) | Edit log replay should track and report file offsets in case of errors | Major | namenode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-496](https://issues.apache.org/jira/browse/HDFS-496) | Use PureJavaCrc32 in HDFS | Minor | datanode, hdfs-client, performance | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-4603](https://issues.apache.org/jira/browse/MAPREDUCE-4603) | Allow JobClient to retry job-submission when JT is in safemode | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4511](https://issues.apache.org/jira/browse/MAPREDUCE-4511) | Add IFile readahead | Major | mrv1, mrv2, performance | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4328](https://issues.apache.org/jira/browse/MAPREDUCE-4328) | Add the option to quiesce the JobTracker | Major | mrv1 | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4001](https://issues.apache.org/jira/browse/MAPREDUCE-4001) | Improve MAPREDUCE-3789's fix logic by looking at job's slot demands instead | Minor | capacity-sched | Harsh J | Harsh J | +| [MAPREDUCE-3597](https://issues.apache.org/jira/browse/MAPREDUCE-3597) | Provide a way to access other info of history file from Rumentool | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3395](https://issues.apache.org/jira/browse/MAPREDUCE-3395) | Add mapred.disk.healthChecker.interval to mapred-default.xml | Trivial | documentation | Eli Collins | Eli Collins | +| [MAPREDUCE-3394](https://issues.apache.org/jira/browse/MAPREDUCE-3394) | Add log guard for a debug message in ReduceTask | Trivial | task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3365](https://issues.apache.org/jira/browse/MAPREDUCE-3365) | Uncomment eventlog settings from the documentation | Trivial | contrib/fair-share | Sho Shimauchi | Sho Shimauchi | +| [MAPREDUCE-3289](https://issues.apache.org/jira/browse/MAPREDUCE-3289) | Make use of fadvise in the NM's shuffle handler | Major | mrv2, nodemanager, performance | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3278](https://issues.apache.org/jira/browse/MAPREDUCE-3278) | 0.20: avoid a busy-loop in ReduceTask scheduling | Major | mrv1, performance, task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2919](https://issues.apache.org/jira/browse/MAPREDUCE-2919) | The JT web UI should show job start times | Minor | jobtracker | Eli Collins | Harsh J | +| [MAPREDUCE-2836](https://issues.apache.org/jira/browse/MAPREDUCE-2836) | Provide option to fail jobs when submitted to non-existent pools. | Minor | contrib/fair-share | Jeff Bean | Ahmed Radwan | +| [MAPREDUCE-2835](https://issues.apache.org/jira/browse/MAPREDUCE-2835) | Make per-job counter limits configurable | Major | . | Tom White | Tom White | +| [MAPREDUCE-2103](https://issues.apache.org/jira/browse/MAPREDUCE-2103) | task-controller shouldn't require o-r permissions | Trivial | task-controller | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-1906](https://issues.apache.org/jira/browse/MAPREDUCE-1906) | Lower default minimum heartbeat interval for tasktracker \> Jobtracker | Major | jobtracker, performance, tasktracker | Scott Carey | Todd Lipcon | +| [MAPREDUCE-782](https://issues.apache.org/jira/browse/MAPREDUCE-782) | Use PureJavaCrc32 in mapreduce spills | Minor | performance | Todd Lipcon | Todd Lipcon | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8617](https://issues.apache.org/jira/browse/HADOOP-8617) | backport pure Java CRC32 calculator changes to branch-1 | Major | performance | Brandon Li | Brandon Li | +| [HADOOP-8445](https://issues.apache.org/jira/browse/HADOOP-8445) | Token should not print the password in toString | Major | security | Ravi Prakash | Ravi Prakash | +| [HADOOP-8417](https://issues.apache.org/jira/browse/HADOOP-8417) | HADOOP-6963 didn't update hadoop-core-pom-template.xml | Major | . | Ted Yu | Ted Yu | +| [HADOOP-8399](https://issues.apache.org/jira/browse/HADOOP-8399) | Remove JDK5 dependency from Hadoop 1.0+ line | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-8329](https://issues.apache.org/jira/browse/HADOOP-8329) | Build fails with Java 7 | Major | build | Kumar Ravi | Eli Collins | +| [HADOOP-8269](https://issues.apache.org/jira/browse/HADOOP-8269) | Fix some javadoc warnings on branch-1 | Trivial | documentation | Eli Collins | Eli Collins | +| [HADOOP-8159](https://issues.apache.org/jira/browse/HADOOP-8159) | NetworkTopology: getLeaf should check for invalid topologies | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-7988](https://issues.apache.org/jira/browse/HADOOP-7988) | Upper case in hostname part of the principals doesn't work with kerberos. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7982](https://issues.apache.org/jira/browse/HADOOP-7982) | UserGroupInformation fails to login if thread's context classloader can't load HadoopLoginModule | Major | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-7908](https://issues.apache.org/jira/browse/HADOOP-7908) | Fix three javadoc warnings on branch-1 | Trivial | documentation | Eli Collins | Eli Collins | +| [HADOOP-7898](https://issues.apache.org/jira/browse/HADOOP-7898) | Fix javadoc warnings in AuthenticationToken.java | Minor | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7879](https://issues.apache.org/jira/browse/HADOOP-7879) | DistributedFileSystem#createNonRecursive should also incrementWriteOps statistics. | Trivial | . | Jonathan Hsieh | Jonathan Hsieh | +| [HADOOP-7870](https://issues.apache.org/jira/browse/HADOOP-7870) | fix SequenceFile#createWriter with boolean createParent arg to respect createParent. | Major | . | Jonathan Hsieh | Jonathan Hsieh | +| [HADOOP-7745](https://issues.apache.org/jira/browse/HADOOP-7745) | I switched variable names in HADOOP-7509 | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-7666](https://issues.apache.org/jira/browse/HADOOP-7666) | branch-0.20-security doesn't include o.a.h.security.TestAuthenticationFilter | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7665](https://issues.apache.org/jira/browse/HADOOP-7665) | branch-0.20-security doesn't include SPNEGO settings in core-default.xml | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7653](https://issues.apache.org/jira/browse/HADOOP-7653) | tarball doesn't include .eclipse.templates | Minor | build | Jonathan Natkins | Jonathan Natkins | +| [HADOOP-7634](https://issues.apache.org/jira/browse/HADOOP-7634) | Cluster setup docs specify wrong owner for task-controller.cfg | Minor | documentation, security | Eli Collins | Eli Collins | +| [HADOOP-7629](https://issues.apache.org/jira/browse/HADOOP-7629) | regression with MAPREDUCE-2289 - setPermission passed immutable FsPermission (rpc failure) | Major | . | Patrick Hunt | Todd Lipcon | +| [HADOOP-7621](https://issues.apache.org/jira/browse/HADOOP-7621) | alfredo config should be in a file not readable by users | Critical | security | Alejandro Abdelnur | Aaron T. Myers | +| [HADOOP-7297](https://issues.apache.org/jira/browse/HADOOP-7297) | Error in the documentation regarding Checkpoint/Backup Node | Trivial | documentation | arnaud p | Harsh J | +| [HADOOP-6947](https://issues.apache.org/jira/browse/HADOOP-6947) | Kerberos relogin should set refreshKrb5Config to true | Major | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6546](https://issues.apache.org/jira/browse/HADOOP-6546) | BloomMapFile can return false negatives | Major | io | Clark Jefcoat | Clark Jefcoat | +| [HADOOP-6527](https://issues.apache.org/jira/browse/HADOOP-6527) | UserGroupInformation::createUserForTesting clobbers already defined group mappings | Major | security | Jakob Homan | Ivan Mitic | +| [HADOOP-5836](https://issues.apache.org/jira/browse/HADOOP-5836) | Bug in S3N handling of directory markers using an object with a trailing "/" causes jobs to fail | Major | fs/s3 | Ian Nowland | Ian Nowland | +| [HDFS-3966](https://issues.apache.org/jira/browse/HDFS-3966) | For branch-1, TestFileCreation should use JUnit4 to make assumeTrue work | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-3701](https://issues.apache.org/jira/browse/HDFS-3701) | HDFS may miss the final block when reading a file opened for writing if one of the datanode is dead | Critical | hdfs-client | Nicolas Liochon | Nicolas Liochon | +| [HDFS-3698](https://issues.apache.org/jira/browse/HDFS-3698) | TestHftpFileSystem is failing in branch-1 due to changed default secure port | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3696](https://issues.apache.org/jira/browse/HDFS-3696) | Create files with WebHdfsFileSystem goes OOM when file size is big | Critical | webhdfs | Kihwal Lee | Tsz Wo Nicholas Sze | +| [HDFS-3551](https://issues.apache.org/jira/browse/HDFS-3551) | WebHDFS CREATE does not use client location for redirection | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3518](https://issues.apache.org/jira/browse/HDFS-3518) | Provide API to check HDFS operational state | Major | hdfs-client | Bikas Saha | Tsz Wo Nicholas Sze | +| [HDFS-3466](https://issues.apache.org/jira/browse/HDFS-3466) | The SPNEGO filter for the NameNode should come out of the web keytab file | Major | namenode, security | Owen O'Malley | Owen O'Malley | +| [HDFS-3461](https://issues.apache.org/jira/browse/HDFS-3461) | HFTP should use the same port & protocol for getting the delegation token | Major | . | Owen O'Malley | Owen O'Malley | +| [HDFS-3453](https://issues.apache.org/jira/browse/HDFS-3453) | HDFS does not use ClientProtocol in a backward-compatible way | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3330](https://issues.apache.org/jira/browse/HDFS-3330) | If GetImageServlet throws an Error or RTE, response has HTTP "OK" status | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3176](https://issues.apache.org/jira/browse/HDFS-3176) | JsonUtil should not parse the MD5MD5CRC32FileChecksum bytes on its own. | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3078](https://issues.apache.org/jira/browse/HDFS-3078) | 2NN https port setting is broken | Major | . | Eli Collins | Eli Collins | +| [HDFS-3008](https://issues.apache.org/jira/browse/HDFS-3008) | Negative caching of local addrs doesn't work | Major | hdfs-client | Eli Collins | Eli Collins | +| [HDFS-2877](https://issues.apache.org/jira/browse/HDFS-2877) | If locking of a storage dir fails, it will remove the other NN's lock file on exit | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2869](https://issues.apache.org/jira/browse/HDFS-2869) | Error in Webhdfs documentation for mkdir | Minor | webhdfs | Harsh J | Harsh J | +| [HDFS-2790](https://issues.apache.org/jira/browse/HDFS-2790) | FSNamesystem.setTimes throws exception with wrong configuration name in the message | Minor | . | Arpit Gupta | Arpit Gupta | +| [HDFS-2751](https://issues.apache.org/jira/browse/HDFS-2751) | Datanode drops OS cache behind reads even for short reads | Major | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-2741](https://issues.apache.org/jira/browse/HDFS-2741) | dfs.datanode.max.xcievers missing in 0.20.205.0 | Minor | . | Markus Jelsma | | +| [HDFS-2728](https://issues.apache.org/jira/browse/HDFS-2728) | Remove dfsadmin -printTopology from branch-1 docs since it does not exist | Minor | namenode | Harsh J | Harsh J | +| [HDFS-2637](https://issues.apache.org/jira/browse/HDFS-2637) | The rpc timeout for block recovery is too low | Major | hdfs-client | Eli Collins | Eli Collins | +| [HDFS-2547](https://issues.apache.org/jira/browse/HDFS-2547) | ReplicationTargetChooser has incorrect block placement comments | Trivial | namenode | Harsh J | Harsh J | +| [HDFS-2541](https://issues.apache.org/jira/browse/HDFS-2541) | For a sufficiently large value of blocks, the DN Scanner may request a random number with a negative seed value. | Major | datanode | Harsh J | Harsh J | +| [HDFS-2305](https://issues.apache.org/jira/browse/HDFS-2305) | Running multiple 2NNs can result in corrupt file system | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1910](https://issues.apache.org/jira/browse/HDFS-1910) | when dfs.name.dir and dfs.name.edits.dir are same fsimage will be saved twice every time | Minor | namenode | Gokul | | +| [MAPREDUCE-4698](https://issues.apache.org/jira/browse/MAPREDUCE-4698) | TestJobHistoryConfig throws Exception in testJobHistoryLogging | Minor | . | Gopal V | Gopal V | +| [MAPREDUCE-4675](https://issues.apache.org/jira/browse/MAPREDUCE-4675) | TestKillSubProcesses fails as the process is still alive after the job is done | Major | test | Arpit Gupta | Bikas Saha | +| [MAPREDUCE-4673](https://issues.apache.org/jira/browse/MAPREDUCE-4673) | make TestRawHistoryFile and TestJobHistoryServer more robust | Major | test | Arpit Gupta | Arpit Gupta | +| [MAPREDUCE-4558](https://issues.apache.org/jira/browse/MAPREDUCE-4558) | TestJobTrackerSafeMode is failing | Major | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4400](https://issues.apache.org/jira/browse/MAPREDUCE-4400) | Fix performance regression for small jobs/workflows | Major | performance, task | Luke Lu | Luke Lu | +| [MAPREDUCE-4241](https://issues.apache.org/jira/browse/MAPREDUCE-4241) | Pipes examples do not compile on Ubuntu 12.04 | Major | build, examples | Andrew Bayer | Andrew Bayer | +| [MAPREDUCE-4095](https://issues.apache.org/jira/browse/MAPREDUCE-4095) | TestJobInProgress#testLocality uses a bogus topology | Major | . | Eli Collins | Colin Patrick McCabe | +| [MAPREDUCE-4088](https://issues.apache.org/jira/browse/MAPREDUCE-4088) | Task stuck in JobLocalizer prevented other tasks on the same node from committing | Critical | mrv1 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4087](https://issues.apache.org/jira/browse/MAPREDUCE-4087) | [Gridmix] GenerateDistCacheData job of Gridmix can become slow in some cases | Major | . | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3789](https://issues.apache.org/jira/browse/MAPREDUCE-3789) | CapacityTaskScheduler may perform unnecessary reservations in heterogenous tracker environments | Critical | capacity-sched, scheduler | Harsh J | Harsh J | +| [MAPREDUCE-3674](https://issues.apache.org/jira/browse/MAPREDUCE-3674) | If invoked with no queueName request param, jobqueue\_details.jsp injects a null queue name into schedulers. | Critical | jobtracker | Harsh J | Harsh J | +| [MAPREDUCE-3419](https://issues.apache.org/jira/browse/MAPREDUCE-3419) | Don't mark exited TT threads as dead in MiniMRCluster | Major | tasktracker, test | Eli Collins | Eli Collins | +| [MAPREDUCE-3405](https://issues.apache.org/jira/browse/MAPREDUCE-3405) | MAPREDUCE-3015 broke compilation of contrib scheduler tests | Critical | capacity-sched, contrib/fair-share | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2932](https://issues.apache.org/jira/browse/MAPREDUCE-2932) | Missing instrumentation plugin class shouldn't crash the TT startup per design | Trivial | tasktracker | Harsh J | Harsh J | +| [MAPREDUCE-2905](https://issues.apache.org/jira/browse/MAPREDUCE-2905) | CapBasedLoadManager incorrectly allows assignment when assignMultiple is true (was: assignmultiple per job) | Major | contrib/fair-share | Jeff Bean | Jeff Bean | +| [MAPREDUCE-2903](https://issues.apache.org/jira/browse/MAPREDUCE-2903) | Map Tasks graph is throwing XML Parse error when Job is executed with 0 maps | Major | jobtracker | Devaraj K | Devaraj K | +| [MAPREDUCE-2806](https://issues.apache.org/jira/browse/MAPREDUCE-2806) | [Gridmix] Load job fails with timeout errors when resource emulation is turned on | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-2377](https://issues.apache.org/jira/browse/MAPREDUCE-2377) | task-controller fails to parse configuration if it doesn't end in \n | Major | task-controller | Todd Lipcon | Benoy Antony | +| [MAPREDUCE-2129](https://issues.apache.org/jira/browse/MAPREDUCE-2129) | Job may hang if mapreduce.job.committer.setup.cleanup.needed=false and mapreduce.map/reduce.failures.maxpercent\>0 | Major | jobtracker | Kang Xiao | Subroto Sanyal | +| [MAPREDUCE-1740](https://issues.apache.org/jira/browse/MAPREDUCE-1740) | NPE in getMatchingLevelForNodes when node locations are variable depth | Major | jobtracker | Todd Lipcon | Ahmed Radwan | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7942](https://issues.apache.org/jira/browse/HADOOP-7942) | enabling clover coverage reports fails hadoop unit test compilation | Major | . | Giridharan Kesavan | Jitendra Nath Pandey | +| [HDFS-3129](https://issues.apache.org/jira/browse/HDFS-3129) | NetworkTopology: add test that getLeaf should check for invalid topologies | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-2332](https://issues.apache.org/jira/browse/HDFS-2332) | Add test for HADOOP-7629: using an immutable FsPermission as an IPC parameter | Major | test | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2073](https://issues.apache.org/jira/browse/MAPREDUCE-2073) | TestTrackerDistributedCacheManager should be up-front about requirements on build environment | Trivial | distributed-cache, test | Todd Lipcon | Todd Lipcon | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7753](https://issues.apache.org/jira/browse/HADOOP-7753) | Support fadvise and sync\_data\_range in NativeIO, add ReadaheadPool class | Major | io, native, performance | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3424](https://issues.apache.org/jira/browse/MAPREDUCE-3424) | Some LinuxTaskController cleanup | Minor | tasktracker | Eli Collins | Eli Collins | +| [MAPREDUCE-3015](https://issues.apache.org/jira/browse/MAPREDUCE-3015) | Add local dir failure info to metrics and the web UI | Major | tasktracker | Eli Collins | Eli Collins | +| [MAPREDUCE-3008](https://issues.apache.org/jira/browse/MAPREDUCE-3008) | [Gridmix] Improve cumulative CPU usage emulation for short running tasks | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-2957](https://issues.apache.org/jira/browse/MAPREDUCE-2957) | The TT should not re-init if it has no good local dirs | Major | tasktracker | Eli Collins | Eli Collins | +| [MAPREDUCE-2850](https://issues.apache.org/jira/browse/MAPREDUCE-2850) | Add test for TaskTracker disk failure handling (MR-2413) | Major | tasktracker | Eli Collins | Ravi Gummadi | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-2517](https://issues.apache.org/jira/browse/MAPREDUCE-2517) | Porting Gridmix v3 system tests into trunk branch. | Major | contrib/gridmix | Vinay Kumar Thota | Vinay Kumar Thota | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/RELEASENOTES.1.1.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/RELEASENOTES.1.1.0.md new file mode 100644 index 00000000000..6caadbff84d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.0/RELEASENOTES.1.1.0.md @@ -0,0 +1,216 @@ + + +# Apache Hadoop 1.1.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8552](https://issues.apache.org/jira/browse/HADOOP-8552) | *Major* | **Conflict: Same security.log.file for multiple users.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8365](https://issues.apache.org/jira/browse/HADOOP-8365) | *Blocker* | **Add flag to disable durable sync** + +This patch enables durable sync by default. Installation where HBase was not used, that used to run without setting "dfs.support.append" or setting it to false explicitly in the configuration, must add a new flag "dfs.durable.sync" and set it to false to preserve the previous semantics. + + +--- + +* [HADOOP-8314](https://issues.apache.org/jira/browse/HADOOP-8314) | *Major* | **HttpServer#hasAdminAccess should return false if authorization is enabled but user is not authenticated** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8230](https://issues.apache.org/jira/browse/HADOOP-8230) | *Major* | **Enable sync by default and disable append** + +Append is not supported in Hadoop 1.x. Please upgrade to 2.x if you need append. If you enabled dfs.support.append for HBase, you're OK, as durable sync (why HBase required dfs.support.append) is now enabled by default. If you really need the previous functionality, to turn on the append functionality set the flag "dfs.support.broken.append" to true. + + +--- + +* [HADOOP-8154](https://issues.apache.org/jira/browse/HADOOP-8154) | *Major* | **DNS#getIPs shouldn't silently return the local host IP for bogus interface names** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-6995](https://issues.apache.org/jira/browse/HADOOP-6995) | *Minor* | **Allow wildcards to be used in ProxyUsers configurations** + +When configuring proxy users and hosts, the special wildcard value "*" may be specified to match any host or any user. + + +--- + +* [HADOOP-5464](https://issues.apache.org/jira/browse/HADOOP-5464) | *Major* | **DFSClient does not treat write timeout of 0 properly** + +Zero values for dfs.socket.timeout and dfs.datanode.socket.write.timeout are now respected. Previously zero values for these parameters resulted in a 5 second timeout. + + +--- + +* [HDFS-3814](https://issues.apache.org/jira/browse/HDFS-3814) | *Major* | **Make the replication monitor multipliers configurable in 1.x** + +This change adds two new configuration parameters. +# {{dfs.namenode.invalidate.work.pct.per.iteration}} for controlling deletion rate of blocks. +# {{dfs.namenode.replication.work.multiplier.per.iteration}} for controlling replication rate. This in turn allows controlling the time it takes for decommissioning. + +Please see hdfs-default.xml for detailed description. + + +--- + +* [HDFS-3703](https://issues.apache.org/jira/browse/HDFS-3703) | *Major* | **Decrease the datanode failure detection time** + +This jira adds a new DataNode state called "stale" at the NameNode. DataNodes are marked as stale if it does not send heartbeat message to NameNode within the timeout configured using the configuration parameter "dfs.namenode.stale.datanode.interval" in seconds (default value is 30 seconds). NameNode picks a stale datanode as the last target to read from when returning block locations for reads. + +This feature is by default turned * off *. To turn on the feature, set the HDFS configuration "dfs.namenode.check.stale.datanode" to true. + + +--- + +* [HDFS-3522](https://issues.apache.org/jira/browse/HDFS-3522) | *Major* | **If NN is in safemode, it should throw SafeModeException when getBlockLocations has zero locations** + +getBlockLocations(), and hence open() for read, will now throw SafeModeException if the NameNode is still in safe mode and there are no replicas reported yet for one of the blocks in the file. + + +--- + +* [HDFS-3518](https://issues.apache.org/jira/browse/HDFS-3518) | *Major* | **Provide API to check HDFS operational state** + +Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. + + +--- + +* [HDFS-3094](https://issues.apache.org/jira/browse/HDFS-3094) | *Major* | **add -nonInteractive and -force option to namenode -format command** + +The 'namenode -format' command now supports the flags '-nonInteractive' and '-force' to improve usefulness without user input. + + +--- + +* [HDFS-3055](https://issues.apache.org/jira/browse/HDFS-3055) | *Minor* | **Implement recovery mode for branch-1** + +This is a new feature. It is documented in hdfs\_user\_guide.xml. + + +--- + +* [HDFS-3044](https://issues.apache.org/jira/browse/HDFS-3044) | *Major* | **fsck move should be non-destructive by default** + +The fsck "move" option is no longer destructive. It copies the accessible blocks of corrupt files to lost and found as before, but no longer deletes the corrupt files after copying the blocks. The original, destructive behavior can be enabled by specifying both the "move" and "delete" options. + + +--- + +* [HDFS-2741](https://issues.apache.org/jira/browse/HDFS-2741) | *Minor* | **dfs.datanode.max.xcievers missing in 0.20.205.0** + +Document and raise the maximum allowed transfer threads on a DataNode to 4096. This helps Apache HBase in particular. + + +--- + +* [HDFS-2617](https://issues.apache.org/jira/browse/HDFS-2617) | *Major* | **Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution** + +Due to the requirement that KSSL use weak encryption types for Kerberos tickets, HTTP authentication to the NameNode will now use SPNEGO by default. This will require users of previous branch-1 releases with security enabled to modify their configurations and create new Kerberos principals in order to use SPNEGO. The old behavior of using KSSL can optionally be enabled by setting the configuration option "hadoop.security.use-weak-http-crypto" to "true". + + +--- + +* [HDFS-2465](https://issues.apache.org/jira/browse/HDFS-2465) | *Major* | **Add HDFS support for fadvise readahead and drop-behind** + +HDFS now has the ability to use posix\_fadvise and sync\_data\_range syscalls to manage the OS buffer cache. This support is currently considered experimental, and may be enabled by configuring the following keys: +dfs.datanode.drop.cache.behind.writes - set to true to drop data out of the buffer cache after writing +dfs.datanode.drop.cache.behind.reads - set to true to drop data out of the buffer cache when performing sequential reads +dfs.datanode.sync.behind.writes - set to true to trigger dirty page writeback immediately after writing data +dfs.datanode.readahead.bytes - set to a non-zero value to trigger readahead for sequential reads + + +--- + +* [MAPREDUCE-4698](https://issues.apache.org/jira/browse/MAPREDUCE-4698) | *Minor* | **TestJobHistoryConfig throws Exception in testJobHistoryLogging** + +Optionally call initialize/initializeFileSystem in JobTracker::startTracker() to allow for proper initialization when offerService is not being called. + + +--- + +* [MAPREDUCE-4675](https://issues.apache.org/jira/browse/MAPREDUCE-4675) | *Major* | **TestKillSubProcesses fails as the process is still alive after the job is done** + +Fixed a race condition caused in TestKillSubProcesses caused due to a recent commit. + + +--- + +* [MAPREDUCE-4673](https://issues.apache.org/jira/browse/MAPREDUCE-4673) | *Major* | **make TestRawHistoryFile and TestJobHistoryServer more robust** + +Fixed TestRawHistoryFile and TestJobHistoryServer to not write to /tmp. + + +--- + +* [MAPREDUCE-4087](https://issues.apache.org/jira/browse/MAPREDUCE-4087) | *Major* | **[Gridmix] GenerateDistCacheData job of Gridmix can become slow in some cases** + +Fixes the issue of GenerateDistCacheData job slowness. + + +--- + +* [MAPREDUCE-3597](https://issues.apache.org/jira/browse/MAPREDUCE-3597) | *Major* | **Provide a way to access other info of history file from Rumentool** + +Rumen now provides {{Parsed*}} objects. These objects provide extra information that are not provided by {{Logged*}} objects. + + +--- + +* [MAPREDUCE-3118](https://issues.apache.org/jira/browse/MAPREDUCE-3118) | *Major* | **Backport Gridmix and Rumen features from trunk to Hadoop 0.20 security branch** + +Backports latest features from trunk to 0.20.206 branch. + + +--- + +* [MAPREDUCE-3008](https://issues.apache.org/jira/browse/MAPREDUCE-3008) | *Major* | **[Gridmix] Improve cumulative CPU usage emulation for short running tasks** + +Improves cumulative CPU emulation for short running tasks. + + +--- + +* [MAPREDUCE-2517](https://issues.apache.org/jira/browse/MAPREDUCE-2517) | *Major* | **Porting Gridmix v3 system tests into trunk branch.** + +Adds system tests to Gridmix. These system tests cover various features like job types (load and sleep), user resolvers (round-robin, submitter-user, echo) and submission modes (stress, replay and serial). + + +--- + +* [MAPREDUCE-1906](https://issues.apache.org/jira/browse/MAPREDUCE-1906) | *Major* | **Lower default minimum heartbeat interval for tasktracker \> Jobtracker** + +The default minimum heartbeat interval has been dropped from 3 seconds to 300ms to increase scheduling throughput on small clusters. Users may tune mapreduce.jobtracker.heartbeats.in.second to adjust this value. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/CHANGES.1.1.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/CHANGES.1.1.1.md new file mode 100644 index 00000000000..f2685d4a56b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/CHANGES.1.1.1.md @@ -0,0 +1,84 @@ + + +# Apache Hadoop Changelog + +## Release 1.1.1 - 2012-11-27 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-528](https://issues.apache.org/jira/browse/HDFS-528) | Add ability for safemode to wait for a minimum number of live datanodes | Major | scripts | Todd Lipcon | Todd Lipcon | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8823](https://issues.apache.org/jira/browse/HADOOP-8823) | ant package target should not depend on cn-docs | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4174](https://issues.apache.org/jira/browse/HDFS-4174) | Backport HDFS-1031 to branch-1: to list a few of the corrupted files in WebUI | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-1539](https://issues.apache.org/jira/browse/HDFS-1539) | prevent data loss when a cluster suffers a power loss | Major | datanode, hdfs-client, namenode | dhruba borthakur | dhruba borthakur | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9017](https://issues.apache.org/jira/browse/HADOOP-9017) | fix hadoop-client-pom-template.xml and hadoop-client-pom-template.xml for version | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-8995](https://issues.apache.org/jira/browse/HADOOP-8995) | Remove unnecessary bogus exception log from Configuration | Minor | . | Jing Zhao | Jing Zhao | +| [HADOOP-8882](https://issues.apache.org/jira/browse/HADOOP-8882) | uppercase namenode host name causes fsck to fail when useKsslAuth is on | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8878](https://issues.apache.org/jira/browse/HADOOP-8878) | uppercase namenode hostname causes hadoop dfs calls with webhdfs filesystem and fsck to fail when security is on | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8745](https://issues.apache.org/jira/browse/HADOOP-8745) | Incorrect version numbers in hadoop-core POM | Minor | . | Matthias Friedrich | Matthias Friedrich | +| [HDFS-4161](https://issues.apache.org/jira/browse/HDFS-4161) | HDFS keeps a thread open for every file writer | Major | hdfs-client | Suresh Srinivas | Tsz Wo Nicholas Sze | +| [HDFS-4134](https://issues.apache.org/jira/browse/HDFS-4134) | hadoop namenode & datanode entry points should return negative exit code on bad arguments | Minor | namenode | Steve Loughran | | +| [HDFS-4105](https://issues.apache.org/jira/browse/HDFS-4105) | the SPNEGO user for secondary namenode should use the web keytab | Major | . | Arpit Gupta | Arpit Gupta | +| [HDFS-3846](https://issues.apache.org/jira/browse/HDFS-3846) | Namenode deadlock in branch-1 | Major | namenode | Tsz Wo Nicholas Sze | Brandon Li | +| [HDFS-3791](https://issues.apache.org/jira/browse/HDFS-3791) | Backport HDFS-173 to Branch-1 : Recursively deleting a directory with millions of files makes NameNode unresponsive for other commands until the deletion completes | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-3658](https://issues.apache.org/jira/browse/HDFS-3658) | TestDFSClientRetries#testNamenodeRestart failed | Major | . | Eli Collins | Tsz Wo Nicholas Sze | +| [HDFS-2815](https://issues.apache.org/jira/browse/HDFS-2815) | Namenode is not coming out of safemode when we perform ( NN crash + restart ) . Also FSCK report shows blocks missed. | Critical | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [MAPREDUCE-4792](https://issues.apache.org/jira/browse/MAPREDUCE-4792) | Unit Test TestJobTrackerRestartWithLostTracker fails with ant-1.8.4 | Major | test | Amir Sanjar | Amir Sanjar | +| [MAPREDUCE-4782](https://issues.apache.org/jira/browse/MAPREDUCE-4782) | NLineInputFormat skips first line of last InputSplit | Blocker | client | Mark Fuhs | Mark Fuhs | +| [MAPREDUCE-4749](https://issues.apache.org/jira/browse/MAPREDUCE-4749) | Killing multiple attempts of a task taker longer as more attempts are killed | Major | . | Arpit Gupta | Arpit Gupta | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-1108](https://issues.apache.org/jira/browse/HDFS-1108) | Log newly allocated blocks | Major | ha, namenode | dhruba borthakur | Todd Lipcon | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/RELEASENOTES.1.1.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/RELEASENOTES.1.1.1.md new file mode 100644 index 00000000000..c1523c33812 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.1/RELEASENOTES.1.1.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 1.1.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/CHANGES.1.1.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/CHANGES.1.1.2.md new file mode 100644 index 00000000000..ab774fb6489 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/CHANGES.1.1.2.md @@ -0,0 +1,89 @@ + + +# Apache Hadoop Changelog + +## Release 1.1.2 - 2013-02-15 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8567](https://issues.apache.org/jira/browse/HADOOP-8567) | Port conf servlet to dump running configuration to branch 1.x | Major | conf | Junping Du | Jing Zhao | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9111](https://issues.apache.org/jira/browse/HADOOP-9111) | Fix failed testcases with @ignore annotation In branch-1 | Minor | test | Jing Zhao | Jing Zhao | +| [HADOOP-8561](https://issues.apache.org/jira/browse/HADOOP-8561) | Introduce HADOOP\_PROXY\_USER for secure impersonation in child hadoop client processes | Major | security | Luke Lu | Yu Gao | +| [HDFS-4252](https://issues.apache.org/jira/browse/HDFS-4252) | Improve confusing log message that prints exception when editlog read is completed | Major | namenode | Suresh Srinivas | Jing Zhao | +| [MAPREDUCE-4397](https://issues.apache.org/jira/browse/MAPREDUCE-4397) | Introduce HADOOP\_SECURITY\_CONF\_DIR for task-controller | Major | task-controller | Luke Lu | Yu Gao | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9115](https://issues.apache.org/jira/browse/HADOOP-9115) | Deadlock in configuration when writing configuration to hdfs | Blocker | . | Arpit Gupta | Jing Zhao | +| [HADOOP-8880](https://issues.apache.org/jira/browse/HADOOP-8880) | Missing jersey jars as dependency in the pom causes hive tests to fail | Major | . | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-8419](https://issues.apache.org/jira/browse/HADOOP-8419) | GzipCodec NPE upon reset with IBM JDK | Major | io | Luke Lu | Yu Li | +| [HADOOP-8418](https://issues.apache.org/jira/browse/HADOOP-8418) | Fix UGI for IBM JDK running on Windows | Major | security | Luke Lu | Yu Gao | +| [HDFS-5996](https://issues.apache.org/jira/browse/HDFS-5996) | hadoop 1.1.2. hdfs write bug | Major | fuse-dfs | WangMeng | | +| [HDFS-4423](https://issues.apache.org/jira/browse/HDFS-4423) | Checkpoint exception causes fatal damage to fsimage. | Blocker | namenode | ChenFolin | Chris Nauroth | +| [HDFS-4208](https://issues.apache.org/jira/browse/HDFS-4208) | NameNode could be stuck in SafeMode due to never-created blocks | Critical | namenode | Brandon Li | Brandon Li | +| [HDFS-3727](https://issues.apache.org/jira/browse/HDFS-3727) | When using SPNEGO, NN should not try to log in using KSSL principal | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-4888](https://issues.apache.org/jira/browse/MAPREDUCE-4888) | NLineInputFormat drops data in 1.1 and beyond | Blocker | mrv1 | Robert Joseph Evans | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4859](https://issues.apache.org/jira/browse/MAPREDUCE-4859) | TestRecoveryManager fails on branch-1 | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4858](https://issues.apache.org/jira/browse/MAPREDUCE-4858) | TestWebUIAuthorization fails on branch-1 | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4798](https://issues.apache.org/jira/browse/MAPREDUCE-4798) | TestJobHistoryServer fails some times with 'java.lang.AssertionError: Address already in use' | Minor | jobhistoryserver, test | sam liu | sam liu | +| [MAPREDUCE-4699](https://issues.apache.org/jira/browse/MAPREDUCE-4699) | TestFairScheduler & TestCapacityScheduler fails due to JobHistory exception | Minor | . | Gopal V | Gopal V | +| [MAPREDUCE-4697](https://issues.apache.org/jira/browse/MAPREDUCE-4697) | TestMapredHeartbeat fails assertion on HeartbeatInterval | Minor | . | Gopal V | Gopal V | +| [MAPREDUCE-4696](https://issues.apache.org/jira/browse/MAPREDUCE-4696) | TestMRServerPorts throws NullReferenceException | Minor | . | Gopal V | Gopal V | +| [MAPREDUCE-4478](https://issues.apache.org/jira/browse/MAPREDUCE-4478) | TaskTracker's heartbeat is out of control | Major | . | Liyin Liang | Liyin Liang | +| [MAPREDUCE-4396](https://issues.apache.org/jira/browse/MAPREDUCE-4396) | Make LocalJobRunner work with private distributed cache | Minor | client | Luke Lu | Yu Gao | +| [MAPREDUCE-4272](https://issues.apache.org/jira/browse/MAPREDUCE-4272) | SortedRanges.Range#compareTo is not spec compliant | Major | task | Luke Lu | Yu Gao | +| [MAPREDUCE-2374](https://issues.apache.org/jira/browse/MAPREDUCE-2374) | "Text File Busy" errors launching MR tasks | Major | . | Todd Lipcon | Andy Isaacson | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9051](https://issues.apache.org/jira/browse/HADOOP-9051) | “ant test” will build failed for trying to delete a file | Minor | test | meng gong | Luke Lu | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/RELEASENOTES.1.1.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/RELEASENOTES.1.1.2.md new file mode 100644 index 00000000000..0570d73e6a8 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.2/RELEASENOTES.1.1.2.md @@ -0,0 +1,55 @@ + + +# Apache Hadoop 1.1.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9115](https://issues.apache.org/jira/browse/HADOOP-9115) | *Blocker* | **Deadlock in configuration when writing configuration to hdfs** + +This fixes a bug where Hive could trigger a deadlock condition in the Hadoop configuration management code. + + +--- + +* [HADOOP-8567](https://issues.apache.org/jira/browse/HADOOP-8567) | *Major* | **Port conf servlet to dump running configuration to branch 1.x** + +Users can use the conf servlet to get the server-side configuration. Users can + +1) connect to http\_server\_url/conf or http\_server\_url/conf?format=xml and get XML-based configuration description; +2) connect to http\_server\_url/conf?format=json and get JSON-based configuration description. + + +--- + +* [HDFS-5996](https://issues.apache.org/jira/browse/HDFS-5996) | *Major* | **hadoop 1.1.2. hdfs write bug** + +someone has discovered this bug and it has been resolved + + +--- + +* [MAPREDUCE-4478](https://issues.apache.org/jira/browse/MAPREDUCE-4478) | *Major* | **TaskTracker's heartbeat is out of control** + +Fixed a bug in TaskTracker's heartbeat to keep it under control. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/CHANGES.1.1.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/CHANGES.1.1.3.md new file mode 100644 index 00000000000..8765b233866 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/CHANGES.1.1.3.md @@ -0,0 +1,65 @@ + + +# Apache Hadoop Changelog + +## Release 1.1.3 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-5028](https://issues.apache.org/jira/browse/HDFS-5028) | LeaseRenewer throw java.util.ConcurrentModificationException when timeout | Major | . | zhaoyunjiong | zhaoyunjiong | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/RELEASENOTES.1.1.3.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/RELEASENOTES.1.1.3.md new file mode 100644 index 00000000000..dfce0e86910 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.1.3/RELEASENOTES.1.1.3.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 1.1.3 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md new file mode 100644 index 00000000000..6c82ffbbf69 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/CHANGES.1.2.0.md @@ -0,0 +1,280 @@ + + +# Apache Hadoop Changelog + +## Release 1.2.0 - 2013-05-13 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8164](https://issues.apache.org/jira/browse/HADOOP-8164) | Handle paths using back slash as path separator for windows only | Major | fs | Suresh Srinivas | Daryn Sharp | +| [HDFS-4350](https://issues.apache.org/jira/browse/HDFS-4350) | Make enabling of stale marking on read and write paths independent | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-4122](https://issues.apache.org/jira/browse/HDFS-4122) | Cleanup HDFS logs and reduce the size of logged messages | Major | datanode, hdfs-client, namenode | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-4737](https://issues.apache.org/jira/browse/MAPREDUCE-4737) | Hadoop does not close output file / does not call Mapper.cleanup if exception in map | Major | . | Daniel Dai | Arun C Murthy | +| [MAPREDUCE-4629](https://issues.apache.org/jira/browse/MAPREDUCE-4629) | Remove JobHistory.DEBUG\_MODE | Major | . | Karthik Kambatla | Karthik Kambatla | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9090](https://issues.apache.org/jira/browse/HADOOP-9090) | Support on-demand publish of metrics | Minor | metrics | Mostafa Elhemali | Mostafa Elhemali | +| [HADOOP-8988](https://issues.apache.org/jira/browse/HADOOP-8988) | Backport HADOOP-8343 to branch-1 | Major | conf | Jing Zhao | Jing Zhao | +| [HADOOP-8820](https://issues.apache.org/jira/browse/HADOOP-8820) | Backport HADOOP-8469 and HADOOP-8470: add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup) | Major | net | Junping Du | Junping Du | +| [HADOOP-8023](https://issues.apache.org/jira/browse/HADOOP-8023) | Add unset() method to Configuration | Critical | conf | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-4776](https://issues.apache.org/jira/browse/HDFS-4776) | Backport SecondaryNameNode web ui to branch-1 | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4774](https://issues.apache.org/jira/browse/HDFS-4774) | Backport HDFS-4525 'Provide an API for knowing whether file is closed or not' to branch-1 | Major | hdfs-client, namenode | Ted Yu | Ted Yu | +| [HDFS-4597](https://issues.apache.org/jira/browse/HDFS-4597) | Backport WebHDFS concat to branch-1 | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4219](https://issues.apache.org/jira/browse/HDFS-4219) | Port slive to branch-1 | Major | . | Arpit Gupta | Arpit Gupta | +| [HDFS-3942](https://issues.apache.org/jira/browse/HDFS-3942) | Backport HDFS-3495: Update balancer policy for Network Topology with additional 'NodeGroup' layer | Major | balancer & mover | Junping Du | Junping Du | +| [HDFS-3941](https://issues.apache.org/jira/browse/HDFS-3941) | Backport HDFS-3498 and HDFS3601: update replica placement policy for new added "NodeGroup" layer topology | Major | namenode | Junping Du | Junping Du | +| [HDFS-3601](https://issues.apache.org/jira/browse/HDFS-3601) | Implementation of ReplicaPlacementPolicyNodeGroup to support 4-layer network topology | Major | namenode | Junping Du | Junping Du | +| [HDFS-3515](https://issues.apache.org/jira/browse/HDFS-3515) | Port HDFS-1457 to branch-1 | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-3495](https://issues.apache.org/jira/browse/HDFS-3495) | Update Balancer to support new NetworkTopology with NodeGroup | Major | balancer & mover | Junping Du | Junping Du | +| [MAPREDUCE-5129](https://issues.apache.org/jira/browse/MAPREDUCE-5129) | Add tag info to JH files | Minor | . | Billie Rinaldi | Billie Rinaldi | +| [MAPREDUCE-5081](https://issues.apache.org/jira/browse/MAPREDUCE-5081) | Backport DistCpV2 and the related JIRAs to branch-1 | Major | distcp | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-4824](https://issues.apache.org/jira/browse/MAPREDUCE-4824) | Provide a mechanism for jobs to indicate they should not be recovered on restart | Major | mrv1 | Tom White | Tom White | +| [MAPREDUCE-4660](https://issues.apache.org/jira/browse/MAPREDUCE-4660) | Update task placement policy for NetworkTopology with 'NodeGroup' layer | Major | jobtracker, mrv1, scheduler | Junping Du | Junping Du | +| [MAPREDUCE-4355](https://issues.apache.org/jira/browse/MAPREDUCE-4355) | Add RunningJob.getJobStatus() | Major | mrv1, mrv2 | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-3678](https://issues.apache.org/jira/browse/MAPREDUCE-3678) | The Map tasks logs should have the value of input split it processed | Major | mrv1, mrv2 | Bejoy KS | Harsh J | +| [MAPREDUCE-987](https://issues.apache.org/jira/browse/MAPREDUCE-987) | Exposing MiniDFS and MiniMR clusters as a single process command-line | Minor | build, test | Philip Zeyliger | Ahmed Radwan | +| [MAPREDUCE-461](https://issues.apache.org/jira/browse/MAPREDUCE-461) | Enable ServicePlugins for the JobTracker | Minor | . | Fredrik Hedberg | Fredrik Hedberg | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9434](https://issues.apache.org/jira/browse/HADOOP-9434) | Backport HADOOP-9267 to branch-1 | Minor | bin | Yu Li | Yu Li | +| [HADOOP-9379](https://issues.apache.org/jira/browse/HADOOP-9379) | capture the ulimit info after printing the log to the console | Trivial | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9253](https://issues.apache.org/jira/browse/HADOOP-9253) | Capture ulimit info in the logs at service start time | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9071](https://issues.apache.org/jira/browse/HADOOP-9071) | configure ivy log levels for resolve/retrieve | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-8971](https://issues.apache.org/jira/browse/HADOOP-8971) | Backport: hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data (HADOOP-8926) | Major | util | Gopal V | Gopal V | +| [HADOOP-8968](https://issues.apache.org/jira/browse/HADOOP-8968) | Add a flag to completely disable the worker version check | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8951](https://issues.apache.org/jira/browse/HADOOP-8951) | RunJar to fail with user-comprehensible error message if jar missing | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-8931](https://issues.apache.org/jira/browse/HADOOP-8931) | Add Java version to startup message | Trivial | . | Eli Collins | Eli Collins | +| [HADOOP-8711](https://issues.apache.org/jira/browse/HADOOP-8711) | provide an option for IPC server users to avoid printing stack information for certain exceptions | Major | ipc | Brandon Li | Brandon Li | +| [HADOOP-7688](https://issues.apache.org/jira/browse/HADOOP-7688) | When a servlet filter throws an exception in init(..), the Jetty server failed silently. | Major | . | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HADOOP-7096](https://issues.apache.org/jira/browse/HADOOP-7096) | Allow setting of end-of-record delimiter for TextInputFormat | Major | . | Ahmed Radwan | Ahmed Radwan | +| [HDFS-4651](https://issues.apache.org/jira/browse/HDFS-4651) | Offline Image Viewer backport to branch-1 | Major | tools | Chris Nauroth | Chris Nauroth | +| [HDFS-4635](https://issues.apache.org/jira/browse/HDFS-4635) | Move BlockManager#computeCapacity to LightWeightGSet | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4551](https://issues.apache.org/jira/browse/HDFS-4551) | Change WebHDFS buffersize behavior to improve default performance | Major | webhdfs | Mark Wagner | Mark Wagner | +| [HDFS-4320](https://issues.apache.org/jira/browse/HDFS-4320) | Add a separate configuration for namenode rpc address instead of only using fs.default.name | Major | datanode, namenode | Mostafa Elhemali | Mostafa Elhemali | +| [HDFS-4062](https://issues.apache.org/jira/browse/HDFS-4062) | In branch-1, FSNameSystem#invalidateWorkForOneNode and FSNameSystem#computeReplicationWorkForBlock should print logs outside of the namesystem lock | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-4057](https://issues.apache.org/jira/browse/HDFS-4057) | NameNode.namesystem should be private. Use getNamesystem() instead. | Minor | namenode | Brandon Li | Brandon Li | +| [HDFS-3940](https://issues.apache.org/jira/browse/HDFS-3940) | Add Gset#clear method and clear the block map when namenode is shutdown | Minor | . | Eli Collins | Suresh Srinivas | +| [HDFS-3838](https://issues.apache.org/jira/browse/HDFS-3838) | fix the typo in FSEditLog.java: isToterationEnabled should be isTolerationEnabled | Trivial | namenode | Brandon Li | Brandon Li | +| [HDFS-3819](https://issues.apache.org/jira/browse/HDFS-3819) | Should check whether invalidate work percentage default value is not greater than 1.0f | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-3817](https://issues.apache.org/jira/browse/HDFS-3817) | avoid printing stack information for SafeModeException | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-3697](https://issues.apache.org/jira/browse/HDFS-3697) | Enable fadvise readahead by default | Minor | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-3647](https://issues.apache.org/jira/browse/HDFS-3647) | Backport HDFS-2868 (Add number of active transfer threads to the DataNode status) to branch-1 | Major | datanode | Steve Hoffman | Harsh J | +| [HDFS-3604](https://issues.apache.org/jira/browse/HDFS-3604) | Add dfs.webhdfs.enabled to hdfs-default.xml | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3596](https://issues.apache.org/jira/browse/HDFS-3596) | Improve FSEditLog pre-allocation in branch-1 | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3521](https://issues.apache.org/jira/browse/HDFS-3521) | Allow namenode to tolerate edit log corruption | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3498](https://issues.apache.org/jira/browse/HDFS-3498) | Make Replica Removal Policy pluggable and ReplicaPlacementPolicyDefault extensible for reusing code in subclass | Major | namenode | Junping Du | Junping Du | +| [HDFS-3479](https://issues.apache.org/jira/browse/HDFS-3479) | backport HDFS-3335 (check for edit log corruption at the end of the log) to branch-1 | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3163](https://issues.apache.org/jira/browse/HDFS-3163) | TestHDFSCLI.testAll fails if the user name is not all lowercase | Trivial | test | Brandon Li | Brandon Li | +| [HDFS-2533](https://issues.apache.org/jira/browse/HDFS-2533) | Remove needless synchronization on FSDataSet.getBlockFile | Minor | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-1957](https://issues.apache.org/jira/browse/HDFS-1957) | Documentation for HFTP | Minor | documentation | Ari Rabkin | Ari Rabkin | +| [HDFS-385](https://issues.apache.org/jira/browse/HDFS-385) | Design a pluggable interface to place replicas of blocks in HDFS | Major | . | dhruba borthakur | dhruba borthakur | +| [MAPREDUCE-4907](https://issues.apache.org/jira/browse/MAPREDUCE-4907) | TrackerDistributedCacheManager issues too many getFileStatus calls | Major | mrv1, tasktracker | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4845](https://issues.apache.org/jira/browse/MAPREDUCE-4845) | ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2 | Major | client | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4837](https://issues.apache.org/jira/browse/MAPREDUCE-4837) | Add webservices for jobtracker | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4703](https://issues.apache.org/jira/browse/MAPREDUCE-4703) | Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. | Major | mrv1, mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4565](https://issues.apache.org/jira/browse/MAPREDUCE-4565) | Backport MR-2855 to branch-1: ResourceBundle lookup during counter name resolution takes a lot of time | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4556](https://issues.apache.org/jira/browse/MAPREDUCE-4556) | FairScheduler: PoolSchedulable#updateDemand() has potential redundant computation | Minor | contrib/fair-share | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4499](https://issues.apache.org/jira/browse/MAPREDUCE-4499) | Looking for speculative tasks is very expensive in 1.x | Major | mrv1, performance | Nathan Roberts | Koji Noguchi | +| [MAPREDUCE-4464](https://issues.apache.org/jira/browse/MAPREDUCE-4464) | Reduce tasks failing with NullPointerException in ConcurrentHashMap.get() | Minor | task | Clint Heath | Clint Heath | +| [MAPREDUCE-4415](https://issues.apache.org/jira/browse/MAPREDUCE-4415) | Backport the Job.getInstance methods from MAPREDUCE-1505 to branch-1 | Major | mrv1 | Harsh J | Harsh J | +| [MAPREDUCE-4408](https://issues.apache.org/jira/browse/MAPREDUCE-4408) | allow jobs to set a JAR that is in the distributed cached | Major | mrv1, mrv2 | Alejandro Abdelnur | Robert Kanter | +| [MAPREDUCE-2931](https://issues.apache.org/jira/browse/MAPREDUCE-2931) | CLONE - LocalJobRunner should support parallel mapper execution | Major | . | Forest Tan | Sandy Ryza | +| [MAPREDUCE-2770](https://issues.apache.org/jira/browse/MAPREDUCE-2770) | Improve hadoop.job.history.location doc in mapred-default.xml | Trivial | documentation | Eli Collins | Sandy Ryza | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9544](https://issues.apache.org/jira/browse/HADOOP-9544) | backport UTF8 encoding fixes to branch-1 | Major | io | Chris Nauroth | Chris Nauroth | +| [HADOOP-9543](https://issues.apache.org/jira/browse/HADOOP-9543) | TestFsShellReturnCode may fail in branch-1 | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9537](https://issues.apache.org/jira/browse/HADOOP-9537) | Backport AIX patches to branch-1 | Major | security | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9502](https://issues.apache.org/jira/browse/HADOOP-9502) | chmod does not return error exit codes for some exceptions | Minor | fs | Ramya Sunil | Tsz Wo Nicholas Sze | +| [HADOOP-9492](https://issues.apache.org/jira/browse/HADOOP-9492) | Fix the typo in testConf.xml to make it consistent with FileUtil#copy() | Trivial | test | Jing Zhao | Jing Zhao | +| [HADOOP-9473](https://issues.apache.org/jira/browse/HADOOP-9473) | typo in FileUtil copy() method | Trivial | fs | Glen Mazza | | +| [HADOOP-9467](https://issues.apache.org/jira/browse/HADOOP-9467) | Metrics2 record filtering (.record.filter.include/exclude) does not filter by name | Major | metrics | Chris Nauroth | Chris Nauroth | +| [HADOOP-9458](https://issues.apache.org/jira/browse/HADOOP-9458) | In branch-1, RPC.getProxy(..) may call proxy.getProtocolVersion(..) without retry | Critical | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9451](https://issues.apache.org/jira/browse/HADOOP-9451) | Node with one topology layer should be handled as fault topology when NodeGroup layer is enabled | Major | net | Junping Du | Junping Du | +| [HADOOP-9375](https://issues.apache.org/jira/browse/HADOOP-9375) | Port HADOOP-7290 to branch-1 to fix TestUserGroupInformation failure | Trivial | test | Xiaobo Peng | Suresh Srinivas | +| [HADOOP-9369](https://issues.apache.org/jira/browse/HADOOP-9369) | DNS#reverseDns() can return hostname with . appended at the end | Major | net | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9349](https://issues.apache.org/jira/browse/HADOOP-9349) | Confusing output when running hadoop version from one hadoop installation when HADOOP\_HOME points to another | Major | tools | Sandy Ryza | Sandy Ryza | +| [HADOOP-9191](https://issues.apache.org/jira/browse/HADOOP-9191) | TestAccessControlList and TestJobHistoryConfig fail with JDK7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9179](https://issues.apache.org/jira/browse/HADOOP-9179) | TestFileSystem fails with open JDK7 | Major | . | Brandon Li | Brandon Li | +| [HADOOP-9154](https://issues.apache.org/jira/browse/HADOOP-9154) | SortedMapWritable#putAll() doesn't add key/value classes to the map | Major | io | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9124](https://issues.apache.org/jira/browse/HADOOP-9124) | SortedMapWritable violates contract of Map interface for equals() and hashCode() | Minor | io | Patrick Hunt | Surenkumar Nihalani | +| [HADOOP-9099](https://issues.apache.org/jira/browse/HADOOP-9099) | NetUtils.normalizeHostName fails on domains where UnknownHost resolves to an IP address | Minor | test | Ivan Mitic | Ivan Mitic | +| [HADOOP-9098](https://issues.apache.org/jira/browse/HADOOP-9098) | Add missing license headers | Blocker | build | Tom White | Arpit Agarwal | +| [HADOOP-9095](https://issues.apache.org/jira/browse/HADOOP-9095) | TestNNThroughputBenchmark fails in branch-1 | Minor | net | Tsz Wo Nicholas Sze | Jing Zhao | +| [HADOOP-9036](https://issues.apache.org/jira/browse/HADOOP-9036) | TestSinkQueue.testConcurrentConsumers fails intermittently (Backports HADOOP-7292) | Major | . | Ivan Mitic | Suresh Srinivas | +| [HADOOP-8963](https://issues.apache.org/jira/browse/HADOOP-8963) | CopyFromLocal doesn't always create user directory | Trivial | . | Billie Rinaldi | Arpit Gupta | +| [HADOOP-8917](https://issues.apache.org/jira/browse/HADOOP-8917) | add LOCALE.US to toLowerCase in SecurityUtil.replacePattern | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8900](https://issues.apache.org/jira/browse/HADOOP-8900) | BuiltInGzipDecompressor throws IOException - stored gzip size doesn't match decompressed size | Major | . | Slavik Krassovsky | Andy Isaacson | +| [HADOOP-8861](https://issues.apache.org/jira/browse/HADOOP-8861) | FSDataOutputStream.sync should call flush() if the underlying wrapped stream is not Syncable | Major | fs | Amareshwari Sriramadasu | Amareshwari Sriramadasu | +| [HADOOP-8832](https://issues.apache.org/jira/browse/HADOOP-8832) | backport serviceplugin to branch-1 | Major | . | Brandon Li | Brandon Li | +| [HADOOP-8819](https://issues.apache.org/jira/browse/HADOOP-8819) | Should use && instead of & in a few places in FTPFileSystem,FTPInputStream,S3InputStream,ViewFileSystem,ViewFs | Major | fs | Brandon Li | Brandon Li | +| [HADOOP-8791](https://issues.apache.org/jira/browse/HADOOP-8791) | rm "Only deletes non empty directory and files." | Major | documentation | Bertrand Dechoux | Jing Zhao | +| [HADOOP-8786](https://issues.apache.org/jira/browse/HADOOP-8786) | HttpServer continues to start even if AuthenticationFilter fails to init | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-8781](https://issues.apache.org/jira/browse/HADOOP-8781) | hadoop-config.sh should add JAVA\_LIBRARY\_PATH to LD\_LIBRARY\_PATH | Major | scripts | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8767](https://issues.apache.org/jira/browse/HADOOP-8767) | secondary namenode on slave machines | Minor | bin | giovanni delussu | giovanni delussu | +| [HADOOP-8613](https://issues.apache.org/jira/browse/HADOOP-8613) | AbstractDelegationTokenIdentifier#getUser() should set token auth type | Critical | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-8612](https://issues.apache.org/jira/browse/HADOOP-8612) | Backport HADOOP-8599 to branch-1 (Non empty response when read beyond eof) | Major | fs | Matt Foley | Eli Collins | +| [HADOOP-8611](https://issues.apache.org/jira/browse/HADOOP-8611) | Allow fall-back to the shell-based implementation when JNI-based users-group mapping fails | Major | security | Kihwal Lee | Robert Parker | +| [HADOOP-8606](https://issues.apache.org/jira/browse/HADOOP-8606) | FileSystem.get may return the wrong filesystem | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8587](https://issues.apache.org/jira/browse/HADOOP-8587) | HarFileSystem access of harMetaCache isn't threadsafe | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-8586](https://issues.apache.org/jira/browse/HADOOP-8586) | Fixup a bunch of SPNEGO misspellings | Major | . | Eli Collins | Eli Collins | +| [HADOOP-8580](https://issues.apache.org/jira/browse/HADOOP-8580) | ant compile-native fails with automake version 1.11.3 | Major | . | Eugene Koontz | | +| [HADOOP-8512](https://issues.apache.org/jira/browse/HADOOP-8512) | AuthenticatedURL should reset the Token when the server returns other than OK on authentication | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8460](https://issues.apache.org/jira/browse/HADOOP-8460) | Document proper setting of HADOOP\_PID\_DIR and HADOOP\_SECURE\_DN\_PID\_DIR | Major | documentation | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8423](https://issues.apache.org/jira/browse/HADOOP-8423) | MapFile.Reader.get() crashes jvm or throws EOFException on Snappy or LZO block-compressed data | Major | io | Jason B | Todd Lipcon | +| [HADOOP-8386](https://issues.apache.org/jira/browse/HADOOP-8386) | hadoop script doesn't work if 'cd' prints to stdout (default behavior in Ubuntu) | Major | scripts | Christopher Berner | Christopher Berner | +| [HADOOP-8355](https://issues.apache.org/jira/browse/HADOOP-8355) | SPNEGO filter throws/logs exception when authentication fails | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8249](https://issues.apache.org/jira/browse/HADOOP-8249) | invalid hadoop-auth cookies should trigger authentication if info is avail before returning HTTP 401 | Major | security | bc Wong | Alejandro Abdelnur | +| [HADOOP-7868](https://issues.apache.org/jira/browse/HADOOP-7868) | Hadoop native fails to compile when default linker option is -Wl,--as-needed | Major | native | James Page | Trevor Robinson | +| [HADOOP-7836](https://issues.apache.org/jira/browse/HADOOP-7836) | TestSaslRPC#testDigestAuthMethodHostBasedToken fails with hostname localhost.localdomain | Minor | ipc, test | Eli Collins | Daryn Sharp | +| [HADOOP-7827](https://issues.apache.org/jira/browse/HADOOP-7827) | jsp pages missing DOCTYPE | Trivial | . | Dave Vronay | Dave Vronay | +| [HADOOP-7698](https://issues.apache.org/jira/browse/HADOOP-7698) | jsvc target fails on x86\_64 | Critical | build | Daryn Sharp | Daryn Sharp | +| [HADOOP-7101](https://issues.apache.org/jira/browse/HADOOP-7101) | UserGroupInformation.getCurrentUser() fails when called from non-Hadoop JAAS context | Blocker | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-6496](https://issues.apache.org/jira/browse/HADOOP-6496) | HttpServer sends wrong content-type for CSS files (and others) | Minor | . | Lars Francke | Ivan Mitic | +| [HDFS-4715](https://issues.apache.org/jira/browse/HDFS-4715) | Backport HDFS-3577 and other related WebHDFS issue to branch-1 | Major | webhdfs | Tsz Wo Nicholas Sze | Mark Wagner | +| [HDFS-4558](https://issues.apache.org/jira/browse/HDFS-4558) | start balancer failed with NPE | Critical | balancer & mover | Wenwu Peng | Junping Du | +| [HDFS-4544](https://issues.apache.org/jira/browse/HDFS-4544) | Error in deleting blocks should not do check disk, for all types of errors | Major | . | Amareshwari Sriramadasu | Arpit Agarwal | +| [HDFS-4519](https://issues.apache.org/jira/browse/HDFS-4519) | Support override of jsvc binary and log file locations when launching secure datanode. | Major | datanode, scripts | Chris Nauroth | Chris Nauroth | +| [HDFS-4518](https://issues.apache.org/jira/browse/HDFS-4518) | Finer grained metrics for HDFS capacity | Major | . | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4479](https://issues.apache.org/jira/browse/HDFS-4479) | logSync() with the FSNamesystem lock held in commitBlockSynchronization | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-4466](https://issues.apache.org/jira/browse/HDFS-4466) | Remove the deadlock from AbstractDelegationTokenSecretManager | Major | namenode, security | Brandon Li | Brandon Li | +| [HDFS-4444](https://issues.apache.org/jira/browse/HDFS-4444) | Add space between total transaction time and number of transactions in FSEditLog#printStatistics | Trivial | . | Stephen Chu | Stephen Chu | +| [HDFS-4413](https://issues.apache.org/jira/browse/HDFS-4413) | Secondary namenode won't start if HDFS isn't the default file system | Major | namenode | Mostafa Elhemali | Mostafa Elhemali | +| [HDFS-4358](https://issues.apache.org/jira/browse/HDFS-4358) | TestCheckpoint failure with JDK7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4355](https://issues.apache.org/jira/browse/HDFS-4355) | TestNameNodeMetrics.testCorruptBlock fails with open JDK7 | Major | test | Brandon Li | Brandon Li | +| [HDFS-4351](https://issues.apache.org/jira/browse/HDFS-4351) | Fix BlockPlacementPolicyDefault#chooseTarget when avoiding stale nodes | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4337](https://issues.apache.org/jira/browse/HDFS-4337) | Backport HDFS-4240 to branch-1: Make sure nodes are avoided to place replica if some replica are already under the same nodegroup. | Major | namenode | Junping Du | meng gong | +| [HDFS-4240](https://issues.apache.org/jira/browse/HDFS-4240) | In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup | Major | namenode | Junping Du | Junping Du | +| [HDFS-4222](https://issues.apache.org/jira/browse/HDFS-4222) | NN is unresponsive and loses heartbeats of DNs when Hadoop is configured to use LDAP and LDAP has issues | Minor | namenode | Xiaobo Peng | Xiaobo Peng | +| [HDFS-4207](https://issues.apache.org/jira/browse/HDFS-4207) | All hadoop fs operations fail if the default fs is down even if a different file system is specified in the command | Minor | hdfs-client | Steve Loughran | Jing Zhao | +| [HDFS-4180](https://issues.apache.org/jira/browse/HDFS-4180) | TestFileCreation fails in branch-1 but not branch-1.1 | Minor | test | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4168](https://issues.apache.org/jira/browse/HDFS-4168) | TestDFSUpgradeFromImage fails in branch-1 | Major | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4072](https://issues.apache.org/jira/browse/HDFS-4072) | On file deletion remove corresponding blocks pending replication | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-3963](https://issues.apache.org/jira/browse/HDFS-3963) | backport namenode/datanode serviceplugin to branch-1 | Major | . | Brandon Li | Brandon Li | +| [HDFS-3961](https://issues.apache.org/jira/browse/HDFS-3961) | FSEditLog preallocate() needs to reset the position of PREALLOCATE\_BUFFER when more than 1MB size is needed | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-3754](https://issues.apache.org/jira/browse/HDFS-3754) | BlockSender doesn't shutdown ReadaheadPool threads | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3698](https://issues.apache.org/jira/browse/HDFS-3698) | TestHftpFileSystem is failing in branch-1 due to changed default secure port | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3679](https://issues.apache.org/jira/browse/HDFS-3679) | fuse\_dfs notrash option sets usetrash | Minor | fuse-dfs | Conrad Meyer | Conrad Meyer | +| [HDFS-3628](https://issues.apache.org/jira/browse/HDFS-3628) | The dfsadmin -setBalancerBandwidth command on branch-1 does not check for superuser privileges | Blocker | datanode, namenode | Harsh J | Harsh J | +| [HDFS-3595](https://issues.apache.org/jira/browse/HDFS-3595) | TestEditLogLoading fails in branch-1 | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3540](https://issues.apache.org/jira/browse/HDFS-3540) | Further improvement on recovery mode and edit log toleration in branch-1 | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3499](https://issues.apache.org/jira/browse/HDFS-3499) | Make NetworkTopology support user specified topology class | Major | datanode | Junping Du | Junping Du | +| [HDFS-3402](https://issues.apache.org/jira/browse/HDFS-3402) | Fix hdfs scripts for secure datanodes | Minor | scripts, security | Benoy Antony | Benoy Antony | +| [HDFS-2827](https://issues.apache.org/jira/browse/HDFS-2827) | Cannot save namespace after renaming a directory above a file with an open lease | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2757](https://issues.apache.org/jira/browse/HDFS-2757) | Cannot read a local block that's being written to when using the local read short circuit | Major | . | Jean-Daniel Cryans | Jean-Daniel Cryans | +| [MAPREDUCE-5202](https://issues.apache.org/jira/browse/MAPREDUCE-5202) | Revert MAPREDUCE-4397 to avoid using incorrect config files | Major | . | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-5198](https://issues.apache.org/jira/browse/MAPREDUCE-5198) | Race condition in cleanup during task tracker renint with LinuxTaskController | Major | tasktracker | Arpit Gupta | Arpit Gupta | +| [MAPREDUCE-5169](https://issues.apache.org/jira/browse/MAPREDUCE-5169) | Job recovery fails if job tracker is restarted after the job is submitted but before its initialized | Major | . | Arpit Gupta | Arun C Murthy | +| [MAPREDUCE-5166](https://issues.apache.org/jira/browse/MAPREDUCE-5166) | ConcurrentModificationException in LocalJobRunner | Blocker | . | Gunther Hagleitner | Sandy Ryza | +| [MAPREDUCE-5158](https://issues.apache.org/jira/browse/MAPREDUCE-5158) | Cleanup required when mapreduce.job.restart.recover is set to false | Major | jobtracker | Yesha Vora | Mayank Bansal | +| [MAPREDUCE-5154](https://issues.apache.org/jira/browse/MAPREDUCE-5154) | staging directory deletion fails because delegation tokens have been cancelled | Major | jobtracker | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5131](https://issues.apache.org/jira/browse/MAPREDUCE-5131) | Provide better handling of job status related apis during JT restart | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-5066](https://issues.apache.org/jira/browse/MAPREDUCE-5066) | JobTracker should set a timeout when calling into job.end.notification.url | Major | . | Ivan Mitic | Ivan Mitic | +| [MAPREDUCE-5049](https://issues.apache.org/jira/browse/MAPREDUCE-5049) | CombineFileInputFormat counts all compressed files non-splitable | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5035](https://issues.apache.org/jira/browse/MAPREDUCE-5035) | Update MR1 memory configuration docs | Major | mrv1 | Tom White | Tom White | +| [MAPREDUCE-5028](https://issues.apache.org/jira/browse/MAPREDUCE-5028) | Maps fail when io.sort.mb is set to high value | Critical | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5008](https://issues.apache.org/jira/browse/MAPREDUCE-5008) | Merger progress miscounts with respect to EOF\_MARKER | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4970](https://issues.apache.org/jira/browse/MAPREDUCE-4970) | Child tasks (try to) create security audit log files | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4969](https://issues.apache.org/jira/browse/MAPREDUCE-4969) | TestKeyValueTextInputFormat test fails with Open JDK 7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [MAPREDUCE-4967](https://issues.apache.org/jira/browse/MAPREDUCE-4967) | TestJvmReuse fails on assertion | Major | tasktracker, test | Chris Nauroth | Karthik Kambatla | +| [MAPREDUCE-4963](https://issues.apache.org/jira/browse/MAPREDUCE-4963) | StatisticsCollector improperly keeps track of "Last Day" and "Last Hour" statistics for new TaskTrackers | Major | mrv1 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4962](https://issues.apache.org/jira/browse/MAPREDUCE-4962) | jobdetails.jsp uses display name instead of real name to get counters | Major | jobtracker, mrv1 | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4933](https://issues.apache.org/jira/browse/MAPREDUCE-4933) | MR1 final merge asks for length of file it just wrote before flushing it | Major | mrv1, task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4930](https://issues.apache.org/jira/browse/MAPREDUCE-4930) | Backport MAPREDUCE-4678 and MAPREDUCE-4925 to branch-1 | Major | examples | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4929](https://issues.apache.org/jira/browse/MAPREDUCE-4929) | mapreduce.task.timeout is ignored | Major | mrv1 | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4924](https://issues.apache.org/jira/browse/MAPREDUCE-4924) | flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR | Trivial | mrv1 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4923](https://issues.apache.org/jira/browse/MAPREDUCE-4923) | Add toString method to TaggedInputSplit | Minor | mrv1, mrv2, task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4916](https://issues.apache.org/jira/browse/MAPREDUCE-4916) | TestTrackerDistributedCacheManager is flaky due to other badly written tests in branch-1 | Major | . | Arun C Murthy | Xuan Gong | +| [MAPREDUCE-4915](https://issues.apache.org/jira/browse/MAPREDUCE-4915) | TestShuffleExceptionCount fails with open JDK7 | Major | test | Brandon Li | Brandon Li | +| [MAPREDUCE-4914](https://issues.apache.org/jira/browse/MAPREDUCE-4914) | TestMiniMRDFSSort fails with openJDK7 | Major | test | Brandon Li | Brandon Li | +| [MAPREDUCE-4909](https://issues.apache.org/jira/browse/MAPREDUCE-4909) | TestKeyValueTextInputFormat fails with Open JDK 7 on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [MAPREDUCE-4904](https://issues.apache.org/jira/browse/MAPREDUCE-4904) | TestMultipleLevelCaching failed in branch-1 | Major | test | meng gong | Junping Du | +| [MAPREDUCE-4860](https://issues.apache.org/jira/browse/MAPREDUCE-4860) | DelegationTokenRenewal attempts to renew token even after a job is removed | Major | security | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4850](https://issues.apache.org/jira/browse/MAPREDUCE-4850) | Job recovery may fail if staging directory has been deleted | Major | mrv1 | Tom White | Tom White | +| [MAPREDUCE-4843](https://issues.apache.org/jira/browse/MAPREDUCE-4843) | When using DefaultTaskController, JobLocalizer not thread safe | Critical | tasktracker | zhaoyunjiong | Karthik Kambatla | +| [MAPREDUCE-4806](https://issues.apache.org/jira/browse/MAPREDUCE-4806) | Cleanup: Some (5) private methods in JobTracker.RecoveryManager are not used anymore after MAPREDUCE-3837 | Major | mrv1 | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4778](https://issues.apache.org/jira/browse/MAPREDUCE-4778) | Fair scheduler event log is only written if directory exists on HDFS | Major | jobtracker, scheduler | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4765](https://issues.apache.org/jira/browse/MAPREDUCE-4765) | Restarting the JobTracker programmatically can cause DelegationTokenRenewal to throw an exception | Minor | jobtracker, mrv1 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4706](https://issues.apache.org/jira/browse/MAPREDUCE-4706) | FairScheduler#dump(): Computing of # running maps and reduces is commented out | Critical | contrib/fair-share | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4662](https://issues.apache.org/jira/browse/MAPREDUCE-4662) | JobHistoryFilesManager thread pool never expands | Major | jobhistoryserver | Thomas Graves | Kihwal Lee | +| [MAPREDUCE-4652](https://issues.apache.org/jira/browse/MAPREDUCE-4652) | ValueAggregatorJob sets the wrong job jar | Major | examples, mrv1 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4643](https://issues.apache.org/jira/browse/MAPREDUCE-4643) | Make job-history cleanup-period configurable | Major | jobhistoryserver | Karthik Kambatla | Sandy Ryza | +| [MAPREDUCE-4595](https://issues.apache.org/jira/browse/MAPREDUCE-4595) | TestLostTracker failing - possibly due to a race in JobHistory.JobHistoryFilesManager#run() | Critical | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4576](https://issues.apache.org/jira/browse/MAPREDUCE-4576) | Large dist cache can block tasktracker heartbeat | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4572](https://issues.apache.org/jira/browse/MAPREDUCE-4572) | Can not access user logs - Jetty is not configured by default to serve aliases/symlinks | Major | tasktracker, webapps | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4463](https://issues.apache.org/jira/browse/MAPREDUCE-4463) | JobTracker recovery fails with HDFS permission issue | Blocker | mrv1 | Tom White | Tom White | +| [MAPREDUCE-4451](https://issues.apache.org/jira/browse/MAPREDUCE-4451) | fairscheduler fail to init job with kerberos authentication configured | Major | contrib/fair-share | Erik.fang | Erik.fang | +| [MAPREDUCE-4434](https://issues.apache.org/jira/browse/MAPREDUCE-4434) | Backport MR-2779 (JobSplitWriter.java can't handle large job.split file) to branch-1 | Major | mrv1 | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4385](https://issues.apache.org/jira/browse/MAPREDUCE-4385) | FairScheduler.maxTasksToAssign() should check for fairscheduler.assignmultiple.maps \< TaskTracker.availableSlots | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4359](https://issues.apache.org/jira/browse/MAPREDUCE-4359) | Potential deadlock in Counters | Major | . | Todd Lipcon | Tom White | +| [MAPREDUCE-4317](https://issues.apache.org/jira/browse/MAPREDUCE-4317) | Job view ACL checks are too permissive | Major | mrv1 | Harsh J | Karthik Kambatla | +| [MAPREDUCE-4315](https://issues.apache.org/jira/browse/MAPREDUCE-4315) | jobhistory.jsp throws 500 when a .txt file is found in /done | Major | jobhistoryserver | Alexander Alten-Lorenz | Sandy Ryza | +| [MAPREDUCE-4278](https://issues.apache.org/jira/browse/MAPREDUCE-4278) | cannot run two local jobs in parallel from the same gateway. | Major | . | Araceli Henley | Sandy Ryza | +| [MAPREDUCE-4195](https://issues.apache.org/jira/browse/MAPREDUCE-4195) | With invalid queueName request param, jobqueue\_details.jsp shows NPE | Critical | jobtracker | Gera Shegalov | | +| [MAPREDUCE-4036](https://issues.apache.org/jira/browse/MAPREDUCE-4036) | Streaming TestUlimit fails on CentOS 6 | Major | test | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3993](https://issues.apache.org/jira/browse/MAPREDUCE-3993) | Graceful handling of codec errors during decompression | Major | mrv1, mrv2 | Todd Lipcon | Karthik Kambatla | +| [MAPREDUCE-3727](https://issues.apache.org/jira/browse/MAPREDUCE-3727) | jobtoken location property in jobconf refers to wrong jobtoken file | Critical | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-2289](https://issues.apache.org/jira/browse/MAPREDUCE-2289) | Permissions race can make getStagingDir fail on local filesystem | Major | job submission | Todd Lipcon | Ahmed Radwan | +| [MAPREDUCE-2264](https://issues.apache.org/jira/browse/MAPREDUCE-2264) | Job status exceeds 100% in some cases | Major | jobtracker | Adam Kramer | Devaraj K | +| [MAPREDUCE-2217](https://issues.apache.org/jira/browse/MAPREDUCE-2217) | The expire launching task should cover the UNASSIGNED task | Major | jobtracker | Scott Chen | Karthik Kambatla | +| [MAPREDUCE-1806](https://issues.apache.org/jira/browse/MAPREDUCE-1806) | CombineFileInputFormat does not work with paths not on default FS | Major | harchive | Paul Yang | Gera Shegalov | +| [MAPREDUCE-1684](https://issues.apache.org/jira/browse/MAPREDUCE-1684) | ClusterStatus can be cached in CapacityTaskScheduler.assignTasks() | Major | capacity-sched | Amareshwari Sriramadasu | Koji Noguchi | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9175](https://issues.apache.org/jira/browse/HADOOP-9175) | TestWritableName fails with Open JDK 7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9174](https://issues.apache.org/jira/browse/HADOOP-9174) | TestSecurityUtil fails on Open JDK 7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4256](https://issues.apache.org/jira/browse/HDFS-4256) | Backport concatenation of files into a single file to branch-1 | Major | namenode | Suresh Srinivas | Sanjay Radia | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8817](https://issues.apache.org/jira/browse/HADOOP-8817) | Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1 | Major | . | Junping Du | Junping Du | +| [HADOOP-8470](https://issues.apache.org/jira/browse/HADOOP-8470) | Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup) | Major | . | Junping Du | Junping Du | +| [HADOOP-8469](https://issues.apache.org/jira/browse/HADOOP-8469) | Make NetworkTopology class pluggable | Major | . | Junping Du | Junping Du | +| [HADOOP-7754](https://issues.apache.org/jira/browse/HADOOP-7754) | Expose file descriptors from Hadoop-wrapped local FileSystems | Major | native, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-4071](https://issues.apache.org/jira/browse/HDFS-4071) | Add number of stale DataNodes to metrics for Branch-1 | Minor | datanode, namenode | Jing Zhao | Jing Zhao | +| [HDFS-3912](https://issues.apache.org/jira/browse/HDFS-3912) | Detecting and avoiding stale datanodes for writing | Major | . | Jing Zhao | Jing Zhao | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/RELEASENOTES.1.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/RELEASENOTES.1.2.0.md new file mode 100644 index 00000000000..9f1e70599e3 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.0/RELEASENOTES.1.2.0.md @@ -0,0 +1,176 @@ + + +# Apache Hadoop 1.2.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8971](https://issues.apache.org/jira/browse/HADOOP-8971) | *Major* | **Backport: hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data (HADOOP-8926)** + +Backport cache-aware improvements for PureJavaCrc32 from trunk (HADOOP-8926) + + +--- + +* [HADOOP-8817](https://issues.apache.org/jira/browse/HADOOP-8817) | *Major* | **Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1** + +A new 4-layer network topology NetworkToplogyWithNodeGroup is available to make Hadoop more robust and efficient in virtualized environment. + + +--- + +* [HADOOP-8470](https://issues.apache.org/jira/browse/HADOOP-8470) | *Major* | **Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup)** + +This patch should be checked in together (or after) with JIRA Hadoop-8469: https://issues.apache.org/jira/browse/HADOOP-8469 + + +--- + +* [HADOOP-8164](https://issues.apache.org/jira/browse/HADOOP-8164) | *Major* | **Handle paths using back slash as path separator for windows only** + +This jira only allows providing paths using back slash as separator on Windows. The back slash on *nix system will be used as escape character. The support for paths using back slash as path separator will be removed in HADOOP-8139 in release 23.3. + + +--- + +* [HADOOP-7698](https://issues.apache.org/jira/browse/HADOOP-7698) | *Critical* | **jsvc target fails on x86\_64** + +The jsvc build target is now supported for Mac OSX and other platforms as well. + + +--- + +* [HDFS-4519](https://issues.apache.org/jira/browse/HDFS-4519) | *Major* | **Support override of jsvc binary and log file locations when launching secure datanode.** + +With this improvement the following options are available in release 1.2.0 and later on 1.x release stream: +1. jsvc location can be overridden by setting environment variable JSVC\_HOME. Defaults to jsvc binary packaged within the Hadoop distro. +2. jsvc log output is directed to the file defined by JSVC\_OUTFILE. Defaults to $HADOOP\_LOG\_DIR/jsvc.out. +3. jsvc error output is directed to the file defined by JSVC\_ERRFILE file. Defaults to $HADOOP\_LOG\_DIR/jsvc.err. + +With this improvement the following options are available in release 2.0.4 and later on 2.x release stream: +1. jsvc log output is directed to the file defined by JSVC\_OUTFILE. Defaults to $HADOOP\_LOG\_DIR/jsvc.out. +2. jsvc error output is directed to the file defined by JSVC\_ERRFILE file. Defaults to $HADOOP\_LOG\_DIR/jsvc.err. + +For overriding jsvc location on 2.x releases, here is the release notes from HDFS-2303: +To run secure Datanodes users must install jsvc for their platform and set JSVC\_HOME to point to the location of jsvc in their environment. + + +--- + +* [HDFS-4350](https://issues.apache.org/jira/browse/HDFS-4350) | *Major* | **Make enabling of stale marking on read and write paths independent** + +This patch makes an incompatible configuration change, as described below: +In releases 1.1.0 and other point releases 1.1.x, the configuration parameter "dfs.namenode.check.stale.datanode" could be used to turn on checking for the stale nodes. This configuration is no longer supported in release 1.2.0 onwards and is renamed as "dfs.namenode.avoid.read.stale.datanode". + +How feature works and configuring this feature: +As described in HDFS-3703 release notes, datanode stale period can be configured using parameter "dfs.namenode.stale.datanode.interval" in seconds (default value is 30 seconds). NameNode can be configured to use this staleness information for reads using configuration "dfs.namenode.avoid.read.stale.datanode". When this parameter is set to true, namenode picks a stale datanode as the last target to read from when returning block locations for reads. Using staleness information for writes is as described in the releases notes of HDFS-3912. + + +--- + +* [HDFS-4337](https://issues.apache.org/jira/browse/HDFS-4337) | *Major* | **Backport HDFS-4240 to branch-1: Make sure nodes are avoided to place replica if some replica are already under the same nodegroup.** + +Backport HDFS-4240 to branch-1 + + +--- + +* [HDFS-4320](https://issues.apache.org/jira/browse/HDFS-4320) | *Major* | **Add a separate configuration for namenode rpc address instead of only using fs.default.name** + +The namenode RPC address is currently identified from configuration "fs.default.name". In some setups where default FS is other than HDFS, the "fs.default.name" cannot be used to get the namenode address. When such a setup co-exists with HDFS, with this change namenode can be identified using a separate configuration parameter "dfs.namenode.rpc-address". + +"dfs.namenode.rpc-address", when configured, overrides fs.default.name for identifying namenode RPC address. + + +--- + +* [HDFS-4122](https://issues.apache.org/jira/browse/HDFS-4122) | *Major* | **Cleanup HDFS logs and reduce the size of logged messages** + +The change from this jira changes the content of some of the log messages. No log message are removed. Only the content of the log messages is changed to reduce the size. If you have a tool that depends on the exact content of the log, please look at the patch and make appropriate updates to the tool. + + +--- + +* [HDFS-4071](https://issues.apache.org/jira/browse/HDFS-4071) | *Minor* | **Add number of stale DataNodes to metrics for Branch-1** + +This jira adds a new metric with name "StaleDataNodes" under metrics context "dfs" of type Gauge. This tracks the number of DataNodes marked as stale. A DataNode is marked stale when the heartbeat message from the DataNode is not received within the configured time ""dfs.namenode.stale.datanode.interval". + + +Please see hdfs-default.xml documentation corresponding to "dfs.namenode.stale.datanode.interval" for more details on how to configure this feature. When this feature is not configured, this metrics would return zero. + + +--- + +* [HDFS-3697](https://issues.apache.org/jira/browse/HDFS-3697) | *Minor* | **Enable fadvise readahead by default** + +The datanode now performs 4MB readahead by default when reading data from its disks, if the native libraries are present. This has been shown to improve performance in many workloads. The feature may be disabled by setting dfs.datanode.readahead.bytes to "0". + + +--- + +* [HDFS-385](https://issues.apache.org/jira/browse/HDFS-385) | *Major* | **Design a pluggable interface to place replicas of blocks in HDFS** + +New experimental API BlockPlacementPolicy allows investigating alternate rules for locating block replicas. + + +--- + +* [MAPREDUCE-4737](https://issues.apache.org/jira/browse/MAPREDUCE-4737) | *Major* | ** Hadoop does not close output file / does not call Mapper.cleanup if exception in map** + +Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. + + +--- + +* [MAPREDUCE-4629](https://issues.apache.org/jira/browse/MAPREDUCE-4629) | *Major* | **Remove JobHistory.DEBUG\_MODE** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4565](https://issues.apache.org/jira/browse/MAPREDUCE-4565) | *Major* | **Backport MR-2855 to branch-1: ResourceBundle lookup during counter name resolution takes a lot of time** + +Passing a cached class-loader to ResourceBundle creator to minimize counter names lookup time. + + +--- + +* [MAPREDUCE-4451](https://issues.apache.org/jira/browse/MAPREDUCE-4451) | *Major* | **fairscheduler fail to init job with kerberos authentication configured** + +Using FairScheduler with security configured, job initialization fails. The problem is that threads in JobInitializer runs as RPC user instead of jobtracker, pre-start all the threads fix this bug + + +--- + +* [MAPREDUCE-4415](https://issues.apache.org/jira/browse/MAPREDUCE-4415) | *Major* | **Backport the Job.getInstance methods from MAPREDUCE-1505 to branch-1** + +Backported new APIs to get a Job object to 1.2.0 from 2.0.0. Job API static methods Job.getInstance(), Job.getInstance(Configuration) and Job.getInstance(Configuration, jobName) are now available across both releases to avoid porting pain. + + +--- + +* [MAPREDUCE-3678](https://issues.apache.org/jira/browse/MAPREDUCE-3678) | *Major* | **The Map tasks logs should have the value of input split it processed** + +A map-task's syslogs now carries basic info on the InputSplit it processed. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/CHANGES.1.2.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/CHANGES.1.2.1.md new file mode 100644 index 00000000000..84a10aa446d --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/CHANGES.1.2.1.md @@ -0,0 +1,84 @@ + + +# Apache Hadoop Changelog + +## Release 1.2.1 - 2013-08-01 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5368](https://issues.apache.org/jira/browse/MAPREDUCE-5368) | Save memory by set capacity, load factor and concurrency level for ConcurrentHashMap in TaskInProgress | Major | mrv1 | zhaoyunjiong | zhaoyunjiong | +| [MAPREDUCE-4838](https://issues.apache.org/jira/browse/MAPREDUCE-4838) | Add extra info to JH files | Major | . | Arun C Murthy | Zhijie Shen | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9730](https://issues.apache.org/jira/browse/HADOOP-9730) | fix hadoop.spec to add task-log4j.properties | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [HADOOP-9665](https://issues.apache.org/jira/browse/HADOOP-9665) | BlockDecompressorStream#decompress will throw EOFException instead of return -1 when EOF | Critical | . | Zhijie Shen | Zhijie Shen | +| [HADOOP-9504](https://issues.apache.org/jira/browse/HADOOP-9504) | MetricsDynamicMBeanBase has concurrency issues in createMBeanInfo | Critical | metrics | Liang Xie | Liang Xie | +| [HDFS-4880](https://issues.apache.org/jira/browse/HDFS-4880) | Diagnostic logging while loading name/edits files | Major | namenode | Arpit Agarwal | Suresh Srinivas | +| [HDFS-4699](https://issues.apache.org/jira/browse/HDFS-4699) | TestPipelinesFailover#testPipelineRecoveryStress fails sporadically | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4581](https://issues.apache.org/jira/browse/HDFS-4581) | DataNode#checkDiskError should not be called on network errors | Major | datanode | Rohit Kochar | Rohit Kochar | +| [HDFS-4261](https://issues.apache.org/jira/browse/HDFS-4261) | TestBalancerWithNodeGroup times out | Major | balancer & mover | Tsz Wo Nicholas Sze | Junping Du | +| [MAPREDUCE-5375](https://issues.apache.org/jira/browse/MAPREDUCE-5375) | Delegation Token renewal exception in jobtracker logs | Critical | . | Venkat Ranganathan | Venkat Ranganathan | +| [MAPREDUCE-5364](https://issues.apache.org/jira/browse/MAPREDUCE-5364) | Deadlock between RenewalTimerTask methods cancel() and run() | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5351](https://issues.apache.org/jira/browse/MAPREDUCE-5351) | JobTracker memory leak caused by CleanupQueue reopening FileSystem | Critical | jobtracker | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5318](https://issues.apache.org/jira/browse/MAPREDUCE-5318) | Ampersand in JSPUtil.java is not escaped | Minor | jobtracker | Bohou Li | Bohou Li | +| [MAPREDUCE-5260](https://issues.apache.org/jira/browse/MAPREDUCE-5260) | Job failed because of JvmManager running into inconsistent state | Major | tasktracker | zhaoyunjiong | zhaoyunjiong | +| [MAPREDUCE-5256](https://issues.apache.org/jira/browse/MAPREDUCE-5256) | CombineInputFormat isn't thread safe affecting HiveServer | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5206](https://issues.apache.org/jira/browse/MAPREDUCE-5206) | JT can show the same job multiple times in Retired Jobs section | Minor | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-5148](https://issues.apache.org/jira/browse/MAPREDUCE-5148) | Syslog missing from Map/Reduce tasks | Major | tasktracker | Yesha Vora | Arun C Murthy | +| [MAPREDUCE-4490](https://issues.apache.org/jira/browse/MAPREDUCE-4490) | JVM reuse is incompatible with LinuxTaskController (and therefore incompatible with Security) | Critical | task-controller, tasktracker | George Datskos | sam liu | +| [MAPREDUCE-3859](https://issues.apache.org/jira/browse/MAPREDUCE-3859) | CapacityScheduler incorrectly utilizes extra-resources of queue for high-memory jobs | Major | capacity-sched | Sergey Tryuber | Sergey Tryuber | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10371](https://issues.apache.org/jira/browse/HADOOP-10371) | The eclipse-plugin cannot work in my environment | Minor | contrib/eclipse-plugin | huangxing | | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/RELEASENOTES.1.2.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/RELEASENOTES.1.2.1.md new file mode 100644 index 00000000000..8835bae1615 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.1/RELEASENOTES.1.2.1.md @@ -0,0 +1,38 @@ + + +# Apache Hadoop 1.2.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10371](https://issues.apache.org/jira/browse/HADOOP-10371) | *Minor* | **The eclipse-plugin cannot work in my environment** + +fixed + + +--- + +* [MAPREDUCE-3859](https://issues.apache.org/jira/browse/MAPREDUCE-3859) | *Major* | **CapacityScheduler incorrectly utilizes extra-resources of queue for high-memory jobs** + +Fixed wrong CapacityScheduler resource allocation for high memory consumption jobs + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/CHANGES.1.2.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/CHANGES.1.2.2.md new file mode 100644 index 00000000000..936cd811f54 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/CHANGES.1.2.2.md @@ -0,0 +1,66 @@ + + +# Apache Hadoop Changelog + +## Release 1.2.2 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-6009](https://issues.apache.org/jira/browse/MAPREDUCE-6009) | Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled in a reduce slot | Blocker | client, job submission | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5450](https://issues.apache.org/jira/browse/MAPREDUCE-5450) | Unnecessary Configuration instantiation in IFileInputStream slows down merge - Port to branch-1 | Blocker | mrv1 | Stanislav Barton | Stanislav Barton | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/RELEASENOTES.1.2.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/RELEASENOTES.1.2.2.md new file mode 100644 index 00000000000..9ecfb2d54d4 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.2.2/RELEASENOTES.1.2.2.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 1.2.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [MAPREDUCE-5450](https://issues.apache.org/jira/browse/MAPREDUCE-5450) | *Blocker* | **Unnecessary Configuration instantiation in IFileInputStream slows down merge - Port to branch-1** + +Fixes blank Configuration object creation overhead by reusing the Job configuration in InMemoryReader. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/CHANGES.1.3.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/CHANGES.1.3.0.md new file mode 100644 index 00000000000..04f639fa70e --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/CHANGES.1.3.0.md @@ -0,0 +1,151 @@ + + +# Apache Hadoop Changelog + +## Release 1.3.0 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5777](https://issues.apache.org/jira/browse/MAPREDUCE-5777) | Support utf-8 text with BOM (byte order marker) | Major | . | bc Wong | zhihai xu | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10614](https://issues.apache.org/jira/browse/HADOOP-10614) | CBZip2InputStream is not threadsafe | Major | . | Xiangrui Meng | Xiangrui Meng | +| [HADOOP-9855](https://issues.apache.org/jira/browse/HADOOP-9855) | Backport HADOOP-6578 to branch-1 | Major | . | James Kinley | James Kinley | +| [HADOOP-9450](https://issues.apache.org/jira/browse/HADOOP-9450) | HADOOP\_USER\_CLASSPATH\_FIRST is not honored; CLASSPATH is PREpended instead of APpended | Major | scripts | Mitch Wyle | Harsh J | +| [HADOOP-8873](https://issues.apache.org/jira/browse/HADOOP-8873) | Port HADOOP-8175 (Add mkdir -p flag) to branch-1 | Major | . | Eli Collins | Akira AJISAKA | +| [HDFS-7312](https://issues.apache.org/jira/browse/HDFS-7312) | Update DistCp v1 to optionally not use tmp location (branch-1 only) | Minor | tools | Joseph Prosser | Joseph Prosser | +| [HDFS-5367](https://issues.apache.org/jira/browse/HDFS-5367) | Restoring namenode storage locks namenode due to unnecessary fsimage write | Major | . | zhaoyunjiong | zhaoyunjiong | +| [HDFS-5038](https://issues.apache.org/jira/browse/HDFS-5038) | Backport several branch-2 APIs to branch-1 | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-4963](https://issues.apache.org/jira/browse/HDFS-4963) | Improve multihoming support in namenode | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4903](https://issues.apache.org/jira/browse/HDFS-4903) | Print trash configuration and trash emptier state in namenode log | Minor | namenode | Suresh Srinivas | Arpit Agarwal | +| [HDFS-4521](https://issues.apache.org/jira/browse/HDFS-4521) | invalid network topologies should not be cached | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [MAPREDUCE-6088](https://issues.apache.org/jira/browse/MAPREDUCE-6088) | TestTokenCache tests should use their own JobConf instances | Major | mrv1, test | zhihai xu | zhihai xu | +| [MAPREDUCE-5712](https://issues.apache.org/jira/browse/MAPREDUCE-5712) | Backport Fair Scheduler pool placement by secondary group | Major | scheduler | Ted Malaska | Ted Malaska | +| [MAPREDUCE-5651](https://issues.apache.org/jira/browse/MAPREDUCE-5651) | Backport Fair Scheduler queue placement policies to branch-1 | Major | scheduler | Sandy Ryza | Ted Malaska | +| [MAPREDUCE-5609](https://issues.apache.org/jira/browse/MAPREDUCE-5609) | Add debug log message when sending job end notification | Major | . | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5457](https://issues.apache.org/jira/browse/MAPREDUCE-5457) | Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5408](https://issues.apache.org/jira/browse/MAPREDUCE-5408) | CLONE - The logging level of the tasks should be configurable by the job | Major | . | Owen O'Malley | Arun C Murthy | +| [MAPREDUCE-5406](https://issues.apache.org/jira/browse/MAPREDUCE-5406) | Improve logging around Task Tracker exiting with JVM manager inconsistent state | Major | tasktracker | Chelsey Chang | Chelsey Chang | +| [MAPREDUCE-5367](https://issues.apache.org/jira/browse/MAPREDUCE-5367) | Local jobs all use same local working directory | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-3310](https://issues.apache.org/jira/browse/MAPREDUCE-3310) | Custom grouping comparator cannot be set for Combiners | Major | client | Mathias Herberts | Alejandro Abdelnur | +| [MAPREDUCE-2351](https://issues.apache.org/jira/browse/MAPREDUCE-2351) | mapred.job.tracker.history.completed.location should support an arbitrary filesystem URI | Major | . | Tom White | Tom White | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11035](https://issues.apache.org/jira/browse/HADOOP-11035) | distcp on mr1(branch-1) fails with NPE using a short relative source path. | Major | tools | zhihai xu | zhihai xu | +| [HADOOP-10562](https://issues.apache.org/jira/browse/HADOOP-10562) | Namenode exits on exception without printing stack trace in AbstractDelegationTokenSecretManager | Critical | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-10142](https://issues.apache.org/jira/browse/HADOOP-10142) | Avoid groups lookup for unprivileged users such as "dr.who" | Major | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-10090](https://issues.apache.org/jira/browse/HADOOP-10090) | Jobtracker metrics not updated properly after execution of a mapreduce job | Major | metrics | Ivan Mitic | Ivan Mitic | +| [HADOOP-10009](https://issues.apache.org/jira/browse/HADOOP-10009) | Backport HADOOP-7808 to branch-1 | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-9801](https://issues.apache.org/jira/browse/HADOOP-9801) | Configuration#writeXml uses platform defaulting encoding, which may mishandle multi-byte characters. | Major | conf | Chris Nauroth | Chris Nauroth | +| [HADOOP-9768](https://issues.apache.org/jira/browse/HADOOP-9768) | chown and chgrp reject users and groups with spaces on platforms where spaces are otherwise acceptable | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9678](https://issues.apache.org/jira/browse/HADOOP-9678) | TestRPC#testStopsAllThreads intermittently fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9507](https://issues.apache.org/jira/browse/HADOOP-9507) | LocalFileSystem rename() is broken in some cases when destination exists | Minor | fs | Mostafa Elhemali | Chris Nauroth | +| [HADOOP-9307](https://issues.apache.org/jira/browse/HADOOP-9307) | BufferedFSInputStream.read returns wrong results after certain seeks | Major | fs | Todd Lipcon | Todd Lipcon | +| [HADOOP-7140](https://issues.apache.org/jira/browse/HADOOP-7140) | IPC Reader threads do not stop when server stops | Critical | . | Todd Lipcon | Todd Lipcon | +| [HDFS-7503](https://issues.apache.org/jira/browse/HDFS-7503) | Namenode restart after large deletions can cause slow processReport (due to logging) | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6822](https://issues.apache.org/jira/browse/HDFS-6822) | In branch-1, Namenode and datanode fails to replace "\_HOST" to hostname for hadoop.http.authentication.kerberos.principal | Major | security | Jing Zhao | Jing Zhao | +| [HDFS-6649](https://issues.apache.org/jira/browse/HDFS-6649) | Documentation for setrep is wrong | Trivial | documentation | Alexander Fahlke | Akira AJISAKA | +| [HDFS-6141](https://issues.apache.org/jira/browse/HDFS-6141) | WebHdfsFileSystem#toUrl does not perform character escaping. | Major | webhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-5944](https://issues.apache.org/jira/browse/HDFS-5944) | LeaseManager:findLeaseWithPrefixPath can't handle path like /a/b/ right and cause SecondaryNameNode failed do checkpoint | Major | namenode | zhaoyunjiong | zhaoyunjiong | +| [HDFS-5685](https://issues.apache.org/jira/browse/HDFS-5685) | DistCp will fail to copy with -delete switch | Major | hdfs-client | Yongjun Zhang | Yongjun Zhang | +| [HDFS-5516](https://issues.apache.org/jira/browse/HDFS-5516) | WebHDFS does not require user name when anonymous http requests are disallowed. | Major | webhdfs | Chris Nauroth | Miodrag Radulovic | +| [HDFS-5245](https://issues.apache.org/jira/browse/HDFS-5245) | shouldRetry() in WebHDFSFileSystem generates excessive warnings | Minor | webhdfs | Haohui Mai | Haohui Mai | +| [HDFS-5211](https://issues.apache.org/jira/browse/HDFS-5211) | Race condition between DistributedFileSystem#close and FileSystem#close can cause return of a closed DistributedFileSystem instance from the FileSystem cache. | Major | hdfs-client | Chris Nauroth | Chris Nauroth | +| [HDFS-5003](https://issues.apache.org/jira/browse/HDFS-5003) | TestNNThroughputBenchmark failed caused by existing directories | Minor | test | Xi Fang | Xi Fang | +| [HDFS-4944](https://issues.apache.org/jira/browse/HDFS-4944) | WebHDFS cannot create a file path containing characters that must be URI-encoded, such as space. | Major | webhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-4898](https://issues.apache.org/jira/browse/HDFS-4898) | BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rack | Minor | namenode | Eric Sirianni | Tsz Wo Nicholas Sze | +| [HDFS-4794](https://issues.apache.org/jira/browse/HDFS-4794) | Browsing filesystem via webui throws kerberos exception when NN service RPC is enabled in a secure cluster | Major | security | Benoy Antony | Benoy Antony | +| [HDFS-4622](https://issues.apache.org/jira/browse/HDFS-4622) | Remove redundant synchronized from FSNamesystem#rollEditLog in branch-1 | Trivial | . | Jing Zhao | Jing Zhao | +| [HDFS-2264](https://issues.apache.org/jira/browse/HDFS-2264) | NamenodeProtocol has the wrong value for clientPrincipal in KerberosInfo annotation | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-6198](https://issues.apache.org/jira/browse/MAPREDUCE-6198) | NPE from JobTracker#resolveAndAddToTopology in MR1 cause initJob and heartbeat failure. | Major | mrv1 | zhihai xu | zhihai xu | +| [MAPREDUCE-6196](https://issues.apache.org/jira/browse/MAPREDUCE-6196) | Fix BigDecimal ArithmeticException in PiEstimator | Minor | . | Ray Chiang | Ray Chiang | +| [MAPREDUCE-6170](https://issues.apache.org/jira/browse/MAPREDUCE-6170) | TestUlimit failure on JDK8 | Major | contrib/streaming | bc Wong | bc Wong | +| [MAPREDUCE-6147](https://issues.apache.org/jira/browse/MAPREDUCE-6147) | Support mapreduce.input.fileinputformat.split.maxsize | Minor | mrv1 | zhihai xu | zhihai xu | +| [MAPREDUCE-6076](https://issues.apache.org/jira/browse/MAPREDUCE-6076) | Zero map split input length combine with none zero map split input length may cause MR1 job hung sometimes. | Major | mrv1 | zhihai xu | zhihai xu | +| [MAPREDUCE-6012](https://issues.apache.org/jira/browse/MAPREDUCE-6012) | DBInputSplit creates invalid ranges on Oracle | Major | . | Julien Serdaru | Wei Yan | +| [MAPREDUCE-6009](https://issues.apache.org/jira/browse/MAPREDUCE-6009) | Map-only job with new-api runs wrong OutputCommitter when cleanup scheduled in a reduce slot | Blocker | client, job submission | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5979](https://issues.apache.org/jira/browse/MAPREDUCE-5979) | FairScheduler: zero weight can cause sort failures | Major | scheduler | Anubhav Dhoot | Anubhav Dhoot | +| [MAPREDUCE-5968](https://issues.apache.org/jira/browse/MAPREDUCE-5968) | Work directory is not deleted when downloadCacheObject throws IOException | Major | mrv1 | zhihai xu | zhihai xu | +| [MAPREDUCE-5966](https://issues.apache.org/jira/browse/MAPREDUCE-5966) | MR1 FairScheduler use of custom weight adjuster is not thread safe for comparisons | Major | scheduler | Anubhav Dhoot | Anubhav Dhoot | +| [MAPREDUCE-5877](https://issues.apache.org/jira/browse/MAPREDUCE-5877) | Inconsistency between JT/TT for tasks taking a long time to launch | Critical | jobtracker, tasktracker | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5822](https://issues.apache.org/jira/browse/MAPREDUCE-5822) | FairScheduler does not preempt due to fairshare-starvation when fairshare is 1 | Major | scheduler | Anubhav Dhoot | Anubhav Dhoot | +| [MAPREDUCE-5808](https://issues.apache.org/jira/browse/MAPREDUCE-5808) | Port output replication factor configurable for terasort to Hadoop 1.x | Minor | examples | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5710](https://issues.apache.org/jira/browse/MAPREDUCE-5710) | Backport MAPREDUCE-1305 to branch-1 | Major | . | Yongjun Zhang | Yongjun Zhang | +| [MAPREDUCE-5702](https://issues.apache.org/jira/browse/MAPREDUCE-5702) | TaskLogServlet#printTaskLog has spurious HTML closing tags | Trivial | task | Karthik Kambatla | Robert Kanter | +| [MAPREDUCE-5698](https://issues.apache.org/jira/browse/MAPREDUCE-5698) | Backport MAPREDUCE-1285 to branch-1 | Major | . | Yongjun Zhang | Yongjun Zhang | +| [MAPREDUCE-5660](https://issues.apache.org/jira/browse/MAPREDUCE-5660) | Log info about possible thrashing (when using memory-based scheduling in Capacity Scheduler) is not printed | Trivial | capacity-sched, mrv1, tasktracker | Adam Kawa | Adam Kawa | +| [MAPREDUCE-5569](https://issues.apache.org/jira/browse/MAPREDUCE-5569) | FloatSplitter is not generating correct splits | Major | . | Nathan Roberts | Nathan Roberts | +| [MAPREDUCE-5556](https://issues.apache.org/jira/browse/MAPREDUCE-5556) | mapred docs have incorrect classpath | Trivial | . | Allen Wittenauer | Harsh J | +| [MAPREDUCE-5512](https://issues.apache.org/jira/browse/MAPREDUCE-5512) | TaskTracker hung after failed reconnect to the JobTracker | Major | tasktracker | Ivan Mitic | Ivan Mitic | +| [MAPREDUCE-5508](https://issues.apache.org/jira/browse/MAPREDUCE-5508) | JobTracker memory leak caused by unreleased FileSystem objects in JobInProgress#cleanupJob | Critical | jobtracker | Xi Fang | Xi Fang | +| [MAPREDUCE-5405](https://issues.apache.org/jira/browse/MAPREDUCE-5405) | Job recovery can fail if task log directory symlink from prior run still exists | Major | mrv1 | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5288](https://issues.apache.org/jira/browse/MAPREDUCE-5288) | ResourceEstimator#getEstimatedTotalMapOutputSize suffers from divide by zero issues | Major | mrv1 | Harsh J | Karthik Kambatla | +| [MAPREDUCE-5272](https://issues.apache.org/jira/browse/MAPREDUCE-5272) | A Minor Error in Javadoc of TestMRWithDistributedCache in Branch-1 | Trivial | test | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5250](https://issues.apache.org/jira/browse/MAPREDUCE-5250) | Searching for ';' in JobTracker History throws ArrayOutOfBoundException | Minor | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5218](https://issues.apache.org/jira/browse/MAPREDUCE-5218) | Annotate (comment) internal classes as Private | Minor | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5217](https://issues.apache.org/jira/browse/MAPREDUCE-5217) | DistCp fails when launched by Oozie in a secure cluster | Major | distcp, security | Venkat Ranganathan | Venkat Ranganathan | +| [MAPREDUCE-5183](https://issues.apache.org/jira/browse/MAPREDUCE-5183) | In, TaskTracker#reportProgress logging of 0.0-1.0 progress is followed by percent sign | Minor | mrv1, tasktracker | Sandy Ryza | Niranjan Singh | +| [MAPREDUCE-5133](https://issues.apache.org/jira/browse/MAPREDUCE-5133) | TestSubmitJob.testSecureJobExecution is flaky due to job dir deletion race | Major | test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5095](https://issues.apache.org/jira/browse/MAPREDUCE-5095) | TestShuffleExceptionCount#testCheckException fails occasionally with JDK7 | Major | . | Arpit Agarwal | Arpit Agarwal | +| [MAPREDUCE-5073](https://issues.apache.org/jira/browse/MAPREDUCE-5073) | TestJobStatusPersistency.testPersistency fails on JDK7 | Major | test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5072](https://issues.apache.org/jira/browse/MAPREDUCE-5072) | TestDelegationTokenRenewal.testDTRenewal fails in MR1 on jdk7 | Major | test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5070](https://issues.apache.org/jira/browse/MAPREDUCE-5070) | TestClusterStatus.testClusterMetrics fails on JDK7 | Major | test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5047](https://issues.apache.org/jira/browse/MAPREDUCE-5047) | keep.failed.task.files=true causes job failure on secure clusters | Major | task, tasktracker | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5046](https://issues.apache.org/jira/browse/MAPREDUCE-5046) | backport MAPREDUCE-1423 to mapred.lib.CombineFileInputFormat | Major | client | Sangjin Lee | | +| [MAPREDUCE-4366](https://issues.apache.org/jira/browse/MAPREDUCE-4366) | mapred metrics shows negative count of waiting maps and reduces | Major | jobtracker | Thomas Graves | Sandy Ryza | +| [MAPREDUCE-2817](https://issues.apache.org/jira/browse/MAPREDUCE-2817) | MiniRMCluster hardcodes 'mapred.local.dir' configuration to 'build/test/mapred/local' | Minor | test | Alejandro Abdelnur | Robert Kanter | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9624](https://issues.apache.org/jira/browse/HADOOP-9624) | TestFSMainOperationsLocalFileSystem failed when the Hadoop test root path has "X" in its name | Minor | test | Xi Fang | Xi Fang | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9573](https://issues.apache.org/jira/browse/HADOOP-9573) | Fix test-patch script to work with the enhanced PreCommit-Admin script. | Major | build | Giridharan Kesavan | Giridharan Kesavan | +| [MAPREDUCE-4049](https://issues.apache.org/jira/browse/MAPREDUCE-4049) | plugin for generic shuffle service | Major | performance, task, tasktracker | Avner BenHanoch | Avner BenHanoch | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/RELEASENOTES.1.3.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/RELEASENOTES.1.3.0.md new file mode 100644 index 00000000000..76800689d35 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/1.3.0/RELEASENOTES.1.3.0.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop 1.3.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8873](https://issues.apache.org/jira/browse/HADOOP-8873) | *Major* | **Port HADOOP-8175 (Add mkdir -p flag) to branch-1** + +FsShell mkdir now accepts a -p flag. Like unix, mkdir -p will not fail if the directory already exists. Unlike unix, intermediate directories are always created, regardless of the flag, to avoid incompatibilities. + + +--- + +* [HDFS-7312](https://issues.apache.org/jira/browse/HDFS-7312) | *Minor* | **Update DistCp v1 to optionally not use tmp location (branch-1 only)** + +DistCp v1 currently copies files to a temporary location and then renames that to the specified destination. This can cause performance issues on file systems such as S3. A -skiptmp flag is added to bypass this step and copy directly to the destination. + + +--- + +* [HDFS-5685](https://issues.apache.org/jira/browse/HDFS-5685) | *Major* | **DistCp will fail to copy with -delete switch** + +Has dependency on MAPREDUCE-1285/MAPREDUCE-5698 + + +--- + +* [MAPREDUCE-5777](https://issues.apache.org/jira/browse/MAPREDUCE-5777) | *Major* | **Support utf-8 text with BOM (byte order marker)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5408](https://issues.apache.org/jira/browse/MAPREDUCE-5408) | *Major* | **CLONE - The logging level of the tasks should be configurable by the job** + +Allow logging level of map/reduce tasks to be configurable. +Configuration changes: + add mapred.map.child.log.level + add mapred.reduce.child.log.level + + +--- + +* [MAPREDUCE-4049](https://issues.apache.org/jira/browse/MAPREDUCE-4049) | *Major* | **plugin for generic shuffle service** + +Allow ReduceTask loading a third party plugin for shuffle (and merge) instead of the default shuffle. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/CHANGES.2.0.0-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/CHANGES.2.0.0-alpha.md new file mode 100644 index 00000000000..52aa432ded5 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/CHANGES.2.0.0-alpha.md @@ -0,0 +1,398 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.0-alpha - 2012-05-23 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8314](https://issues.apache.org/jira/browse/HADOOP-8314) | HttpServer#hasAdminAccess should return false if authorization is enabled but user is not authenticated | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8270](https://issues.apache.org/jira/browse/HADOOP-8270) | hadoop-daemon.sh stop action should return 0 for an already stopped service | Minor | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-8184](https://issues.apache.org/jira/browse/HADOOP-8184) | ProtoBuf RPC engine does not need it own reply packet - it can use the IPC layer reply packet. | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-8154](https://issues.apache.org/jira/browse/HADOOP-8154) | DNS#getIPs shouldn't silently return the local host IP for bogus interface names | Major | conf | Eli Collins | Eli Collins | +| [HADOOP-8149](https://issues.apache.org/jira/browse/HADOOP-8149) | cap space usage of default log4j rolling policy | Major | conf | Patrick Hunt | Patrick Hunt | +| [HADOOP-7524](https://issues.apache.org/jira/browse/HADOOP-7524) | Change RPC to allow multiple protocols including multiple versions of the same protocol | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HDFS-3286](https://issues.apache.org/jira/browse/HDFS-3286) | When the threshold value for balancer is 0(zero) ,unexpected output is displayed | Major | balancer & mover | J.Andreina | Ashish Singhi | +| [HDFS-3164](https://issues.apache.org/jira/browse/HDFS-3164) | Move DatanodeInfo#hostName to DatanodeID | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3144](https://issues.apache.org/jira/browse/HDFS-3144) | Refactor DatanodeID#getName by use | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3138](https://issues.apache.org/jira/browse/HDFS-3138) | Move DatanodeInfo#ipcPort to DatanodeID | Major | . | Eli Collins | Eli Collins | +| [HDFS-3137](https://issues.apache.org/jira/browse/HDFS-3137) | Bump LAST\_UPGRADABLE\_LAYOUT\_VERSION to -16 | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-3044](https://issues.apache.org/jira/browse/HDFS-3044) | fsck move should be non-destructive by default | Major | namenode | Eli Collins | Colin Patrick McCabe | +| [HDFS-2303](https://issues.apache.org/jira/browse/HDFS-2303) | Unbundle jsvc | Major | build, scripts | Roman Shaposhnik | Mingjie Lai | +| [HDFS-395](https://issues.apache.org/jira/browse/HDFS-395) | DFS Scalability: Incremental block reports | Major | datanode, namenode | dhruba borthakur | Tomasz Nykiel | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8343](https://issues.apache.org/jira/browse/HADOOP-8343) | Allow configuration of authorization for JmxJsonServlet and MetricsServlet | Major | util | Philip Zeyliger | Alejandro Abdelnur | +| [HADOOP-8206](https://issues.apache.org/jira/browse/HADOOP-8206) | Common portion of ZK-based failover controller | Major | ha | Todd Lipcon | Todd Lipcon | +| [HADOOP-8121](https://issues.apache.org/jira/browse/HADOOP-8121) | Active Directory Group Mapping Service | Major | security | Jonathan Natkins | Jonathan Natkins | +| [HADOOP-7876](https://issues.apache.org/jira/browse/HADOOP-7876) | Allow access to BlockKey/DelegationKey encoded key for RPC over protobuf | Major | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7806](https://issues.apache.org/jira/browse/HADOOP-7806) | Support binding to sub-interfaces | Major | util | Harsh J | Harsh J | +| [HADOOP-7454](https://issues.apache.org/jira/browse/HADOOP-7454) | Common side of High Availability Framework (HDFS-1623) | Major | . | Aaron T. Myers | | +| [HADOOP-7030](https://issues.apache.org/jira/browse/HADOOP-7030) | Add TableMapping topology implementation to read host to rack mapping from a file | Major | . | Patrick Angeles | Tom White | +| [HDFS-3167](https://issues.apache.org/jira/browse/HDFS-3167) | CLI-based driver for MiniDFSCluster | Minor | test | Henry Robinson | Henry Robinson | +| [HDFS-3148](https://issues.apache.org/jira/browse/HDFS-3148) | The client should be able to use multiple local interfaces for data transfer | Major | hdfs-client, performance | Eli Collins | Eli Collins | +| [HDFS-3102](https://issues.apache.org/jira/browse/HDFS-3102) | Add CLI tool to initialize the shared-edits dir | Major | ha, namenode | Todd Lipcon | Aaron T. Myers | +| [HDFS-3004](https://issues.apache.org/jira/browse/HDFS-3004) | Implement Recovery Mode | Major | tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3000](https://issues.apache.org/jira/browse/HDFS-3000) | Add a public API for setting quotas | Major | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2941](https://issues.apache.org/jira/browse/HDFS-2941) | Add an administrative command to download a copy of the fsimage from the NN | Major | hdfs-client, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2731](https://issues.apache.org/jira/browse/HDFS-2731) | HA: Autopopulate standby name dirs if they're empty | Major | ha | Aaron T. Myers | Todd Lipcon | +| [HDFS-2430](https://issues.apache.org/jira/browse/HDFS-2430) | The number of failed or low-resource volumes the NN can tolerate should be configurable | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1623](https://issues.apache.org/jira/browse/HDFS-1623) | High Availability Framework for HDFS NN | Major | . | Sanjay Radia | | +| [HDFS-234](https://issues.apache.org/jira/browse/HDFS-234) | Integration with BookKeeper logging system | Major | . | Luca Telloli | Ivan Kelly | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8366](https://issues.apache.org/jira/browse/HADOOP-8366) | Use ProtoBuf for RpcResponseHeader | Blocker | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-8356](https://issues.apache.org/jira/browse/HADOOP-8356) | FileSystem service loading mechanism should print the FileSystem impl it is failing to load | Major | fs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8353](https://issues.apache.org/jira/browse/HADOOP-8353) | hadoop-daemon.sh and yarn-daemon.sh can be misleading on stop | Major | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-8350](https://issues.apache.org/jira/browse/HADOOP-8350) | Improve NetUtils.getInputStream to return a stream which has a tunable timeout | Major | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-8285](https://issues.apache.org/jira/browse/HADOOP-8285) | Use ProtoBuf for RpcPayLoadHeader | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-8280](https://issues.apache.org/jira/browse/HADOOP-8280) | Move VersionUtil/TestVersionUtil and GenericTestUtils from HDFS into Common. | Major | test, util | Ahmed Radwan | Ahmed Radwan | +| [HADOOP-8236](https://issues.apache.org/jira/browse/HADOOP-8236) | haadmin should have configurable timeouts for failover commands | Major | ha | Philip Zeyliger | Todd Lipcon | +| [HADOOP-8214](https://issues.apache.org/jira/browse/HADOOP-8214) | make hadoop script recognize a full set of deprecated commands | Major | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-8200](https://issues.apache.org/jira/browse/HADOOP-8200) | Remove HADOOP\_[JOBTRACKER\|TASKTRACKER]\_OPTS | Minor | conf | Eli Collins | Eli Collins | +| [HADOOP-8193](https://issues.apache.org/jira/browse/HADOOP-8193) | Refactor FailoverController/HAAdmin code to add an abstract class for "target" services | Major | ha | Todd Lipcon | Todd Lipcon | +| [HADOOP-8185](https://issues.apache.org/jira/browse/HADOOP-8185) | Update namenode -format documentation and add -nonInteractive and -force | Major | documentation | Arpit Gupta | Arpit Gupta | +| [HADOOP-8183](https://issues.apache.org/jira/browse/HADOOP-8183) | Stop using "mapred.used.genericoptionsparser" to avoid unnecessary warnings | Minor | util | Harsh J | Harsh J | +| [HADOOP-8163](https://issues.apache.org/jira/browse/HADOOP-8163) | Improve ActiveStandbyElector to provide hooks for fencing old active | Major | ha | Todd Lipcon | Todd Lipcon | +| [HADOOP-8152](https://issues.apache.org/jira/browse/HADOOP-8152) | Expand public APIs for security library classes | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-8118](https://issues.apache.org/jira/browse/HADOOP-8118) | Print the stack trace of InstanceAlreadyExistsException in trace level | Minor | metrics | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-8117](https://issues.apache.org/jira/browse/HADOOP-8117) | Upgrade test build to Surefire 2.12 | Trivial | build, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-8113](https://issues.apache.org/jira/browse/HADOOP-8113) | Correction to BUILDING.txt: HDFS needs ProtocolBuffer, too (not just MapReduce) | Trivial | documentation | Eugene Koontz | Eugene Koontz | +| [HADOOP-8098](https://issues.apache.org/jira/browse/HADOOP-8098) | KerberosAuthenticatorHandler should use \_HOST replacement to resolve principal name | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8086](https://issues.apache.org/jira/browse/HADOOP-8086) | KerberosName silently sets defaultRealm to "" if the Kerberos config is not found, it should log a WARN | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8085](https://issues.apache.org/jira/browse/HADOOP-8085) | Add RPC metrics to ProtobufRpcEngine | Major | ipc, metrics | Suresh Srinivas | Hari Mankude | +| [HADOOP-8084](https://issues.apache.org/jira/browse/HADOOP-8084) | Protobuf RPC engine can be optimized to not do copying for the RPC request/response | Major | ipc | Devaraj Das | Devaraj Das | +| [HADOOP-8077](https://issues.apache.org/jira/browse/HADOOP-8077) | HA: fencing method should be able to be configured on a per-NN or per-NS basis | Major | ha | Todd Lipcon | Todd Lipcon | +| [HADOOP-8070](https://issues.apache.org/jira/browse/HADOOP-8070) | Add standalone benchmark of protobuf IPC | Major | benchmarks, ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-8007](https://issues.apache.org/jira/browse/HADOOP-8007) | HA: use substitution token for fencing argument | Major | ha | Aaron T. Myers | Todd Lipcon | +| [HADOOP-7987](https://issues.apache.org/jira/browse/HADOOP-7987) | Support setting the run-as user in unsecure mode | Major | security | Devaraj Das | Jitendra Nath Pandey | +| [HADOOP-7957](https://issues.apache.org/jira/browse/HADOOP-7957) | Classes deriving GetGroupsBase should be able to override proxy creation. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7899](https://issues.apache.org/jira/browse/HADOOP-7899) | Generate proto java files as part of the build | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7875](https://issues.apache.org/jira/browse/HADOOP-7875) | Add helper class to unwrap RemoteException from ServiceException thrown on protobuf based RPC | Major | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7729](https://issues.apache.org/jira/browse/HADOOP-7729) | Send back valid HTTP response if user hits IPC port with HTTP GET | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7727](https://issues.apache.org/jira/browse/HADOOP-7727) | fix some typos and tabs in CHANGES.TXT | Trivial | build | Steve Loughran | Steve Loughran | +| [HADOOP-7717](https://issues.apache.org/jira/browse/HADOOP-7717) | Move handling of concurrent client fail-overs to RetryInvocationHandler | Major | ipc | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7716](https://issues.apache.org/jira/browse/HADOOP-7716) | RPC protocol registration on SS does not log the protocol name (only the class which may be different) | Minor | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-7693](https://issues.apache.org/jira/browse/HADOOP-7693) | fix RPC.Server#addProtocol to work in AvroRpcEngine | Major | ipc | Doug Cutting | Doug Cutting | +| [HADOOP-7687](https://issues.apache.org/jira/browse/HADOOP-7687) | Make getProtocolSignature public | Minor | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-7635](https://issues.apache.org/jira/browse/HADOOP-7635) | RetryInvocationHandler should release underlying resources on close | Major | ipc | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7607](https://issues.apache.org/jira/browse/HADOOP-7607) | Simplify the RPC proxy cleanup process | Major | ipc | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7549](https://issues.apache.org/jira/browse/HADOOP-7549) | Use JDK ServiceLoader mechanism to find FileSystem implementations | Major | fs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-7350](https://issues.apache.org/jira/browse/HADOOP-7350) | Use ServiceLoader to discover compression codec classes | Major | conf, io | Tom White | Tom White | +| [HDFS-3418](https://issues.apache.org/jira/browse/HDFS-3418) | Rename BlockWithLocationsProto datanodeIDs field to storageIDs | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3378](https://issues.apache.org/jira/browse/HDFS-3378) | Remove DFS\_NAMENODE\_SECONDARY\_HTTPS\_PORT\_KEY and DEFAULT | Trivial | . | Eli Collins | Eli Collins | +| [HDFS-3375](https://issues.apache.org/jira/browse/HDFS-3375) | Put client name in DataXceiver thread name for readBlock and keepalive | Trivial | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-3365](https://issues.apache.org/jira/browse/HDFS-3365) | Enable users to disable socket caching in DFS client configuration | Minor | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-3319](https://issues.apache.org/jira/browse/HDFS-3319) | DFSOutputStream should not start a thread in constructors | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3294](https://issues.apache.org/jira/browse/HDFS-3294) | Fix indentation in NamenodeWebHdfsMethods and DatanodeWebHdfsMethods | Trivial | datanode, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3279](https://issues.apache.org/jira/browse/HDFS-3279) | One of the FSEditLog constructors should be moved to TestEditLog | Minor | namenode | Tsz Wo Nicholas Sze | Arpit Gupta | +| [HDFS-3263](https://issues.apache.org/jira/browse/HDFS-3263) | HttpFS should read HDFS config from Hadoop site.xml files | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3259](https://issues.apache.org/jira/browse/HDFS-3259) | NameNode#initializeSharedEdits should populate shared edits dir with edit log segments | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3249](https://issues.apache.org/jira/browse/HDFS-3249) | Use ToolRunner.confirmPrompt in NameNode | Trivial | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3247](https://issues.apache.org/jira/browse/HDFS-3247) | Improve bootstrapStandby behavior when original NN is not active | Minor | ha | Todd Lipcon | Todd Lipcon | +| [HDFS-3244](https://issues.apache.org/jira/browse/HDFS-3244) | Remove dead writable code from hdfs/protocol | Major | . | Eli Collins | Eli Collins | +| [HDFS-3240](https://issues.apache.org/jira/browse/HDFS-3240) | Drop log level of "heartbeat: ..." in BPServiceActor to DEBUG | Trivial | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-3238](https://issues.apache.org/jira/browse/HDFS-3238) | ServerCommand and friends don't need to be writables | Major | . | Eli Collins | Eli Collins | +| [HDFS-3226](https://issues.apache.org/jira/browse/HDFS-3226) | Allow GetConf tool to print arbitrary keys | Major | tools | Todd Lipcon | Todd Lipcon | +| [HDFS-3206](https://issues.apache.org/jira/browse/HDFS-3206) | Miscellaneous xml cleanups for OEV | Minor | tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3204](https://issues.apache.org/jira/browse/HDFS-3204) | Minor modification to JournalProtocol.proto to make it generic | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-3179](https://issues.apache.org/jira/browse/HDFS-3179) | Improve the error message: DataStreamer throw an exception, "nodes.length != original.length + 1" on single datanode cluster | Major | hdfs-client | Zhanwei Wang | Tsz Wo Nicholas Sze | +| [HDFS-3172](https://issues.apache.org/jira/browse/HDFS-3172) | dfs.upgrade.permission is dead code | Trivial | namenode | Eli Collins | Eli Collins | +| [HDFS-3171](https://issues.apache.org/jira/browse/HDFS-3171) | The DatanodeID "name" field is overloaded | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3169](https://issues.apache.org/jira/browse/HDFS-3169) | TestFsck should test multiple -move operations in a row | Minor | test | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3158](https://issues.apache.org/jira/browse/HDFS-3158) | LiveNodes member of NameNodeMXBean should list non-DFS used space and capacity per DN | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3120](https://issues.apache.org/jira/browse/HDFS-3120) | Enable hsync and hflush by default | Major | . | Eli Collins | Eli Collins | +| [HDFS-3094](https://issues.apache.org/jira/browse/HDFS-3094) | add -nonInteractive and -force option to namenode -format command | Major | . | Arpit Gupta | Arpit Gupta | +| [HDFS-3091](https://issues.apache.org/jira/browse/HDFS-3091) | Update the usage limitations of ReplaceDatanodeOnFailure policy in the config description for the smaller clusters. | Major | datanode, hdfs-client, namenode | Uma Maheswara Rao G | Tsz Wo Nicholas Sze | +| [HDFS-3084](https://issues.apache.org/jira/browse/HDFS-3084) | FenceMethod.tryFence() and ShellCommandFencer should pass namenodeId as well as host:port | Major | ha | Philip Zeyliger | Todd Lipcon | +| [HDFS-3071](https://issues.apache.org/jira/browse/HDFS-3071) | haadmin failover command does not provide enough detail for when target NN is not ready to be active | Major | ha | Philip Zeyliger | Todd Lipcon | +| [HDFS-3056](https://issues.apache.org/jira/browse/HDFS-3056) | Add an interface for DataBlockScanner logging | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3050](https://issues.apache.org/jira/browse/HDFS-3050) | rework OEV to share more code with the NameNode | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3036](https://issues.apache.org/jira/browse/HDFS-3036) | Remove unused method DFSUtil#isDefaultNamenodeAddress | Trivial | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3030](https://issues.apache.org/jira/browse/HDFS-3030) | Remove getProtocolVersion and getProtocolSignature from translators | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-3021](https://issues.apache.org/jira/browse/HDFS-3021) | Use generic type to declare FSDatasetInterface | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3014](https://issues.apache.org/jira/browse/HDFS-3014) | FSEditLogOp and its subclasses should have toString() method | Major | namenode | Sho Shimauchi | Sho Shimauchi | +| [HDFS-3003](https://issues.apache.org/jira/browse/HDFS-3003) | Remove getHostPortString() from NameNode, replace it with NetUtils.getHostPortString() | Trivial | namenode | Brandon Li | Brandon Li | +| [HDFS-2983](https://issues.apache.org/jira/browse/HDFS-2983) | Relax the build version check to permit rolling upgrades within a release | Major | . | Eli Collins | Aaron T. Myers | +| [HDFS-2895](https://issues.apache.org/jira/browse/HDFS-2895) | Remove Writable wire protocol related code that is no longer necessary | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2708](https://issues.apache.org/jira/browse/HDFS-2708) | Stats for the # of blocks per DN | Minor | datanode, namenode | Eli Collins | Aaron T. Myers | +| [HDFS-2650](https://issues.apache.org/jira/browse/HDFS-2650) | Replace @inheritDoc with @Override | Minor | . | Hari Mankude | Hari Mankude | +| [HDFS-2564](https://issues.apache.org/jira/browse/HDFS-2564) | Cleanup unnecessary exceptions thrown and unnecessary casts | Minor | datanode, hdfs-client, namenode | Hari Mankude | Hari Mankude | +| [HDFS-2496](https://issues.apache.org/jira/browse/HDFS-2496) | Separate datatypes for DatanodeProtocol | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2413](https://issues.apache.org/jira/browse/HDFS-2413) | Add public APIs for safemode | Major | hdfs-client | Todd Lipcon | Harsh J | +| [HDFS-2410](https://issues.apache.org/jira/browse/HDFS-2410) | Further clean up hard-coded configuration keys | Minor | datanode, namenode, test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2351](https://issues.apache.org/jira/browse/HDFS-2351) | Change Namenode and Datanode to register each of their protocols seperately | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-2337](https://issues.apache.org/jira/browse/HDFS-2337) | DFSClient shouldn't keep multiple RPC proxy references | Major | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2223](https://issues.apache.org/jira/browse/HDFS-2223) | Untangle depencencies between NN components | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2188](https://issues.apache.org/jira/browse/HDFS-2188) | HDFS-1580: Make FSEditLog create its journals from a list of URIs rather than NNStorage | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-1580](https://issues.apache.org/jira/browse/HDFS-1580) | Add interface for generic Write Ahead Logging mechanisms | Major | namenode | Ivan Kelly | Jitendra Nath Pandey | +| [HDFS-309](https://issues.apache.org/jira/browse/HDFS-309) | FSEditLog should log progress during replay | Major | . | Todd Lipcon | Sho Shimauchi | +| [MAPREDUCE-4219](https://issues.apache.org/jira/browse/MAPREDUCE-4219) | make default container-executor.conf.dir be a path relative to the container-executor binary | Major | security | Roman Shaposhnik | Roman Shaposhnik | +| [MAPREDUCE-4138](https://issues.apache.org/jira/browse/MAPREDUCE-4138) | Reduce memory usage of counters due to non-static nested classes | Major | . | Tom White | Tom White | +| [MAPREDUCE-4103](https://issues.apache.org/jira/browse/MAPREDUCE-4103) | Fix HA docs for changes to shell command fencer args | Major | documentation | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-4093](https://issues.apache.org/jira/browse/MAPREDUCE-4093) | Improve RM WebApp start up when proxy address is not set | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3991](https://issues.apache.org/jira/browse/MAPREDUCE-3991) | Streaming FAQ has some wrong instructions about input files splitting | Trivial | documentation | Harsh J | Harsh J | +| [MAPREDUCE-3955](https://issues.apache.org/jira/browse/MAPREDUCE-3955) | Replace ProtoOverHadoopRpcEngine with ProtobufRpcEngine. | Blocker | mrv2 | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [MAPREDUCE-3909](https://issues.apache.org/jira/browse/MAPREDUCE-3909) | javadoc the Service interfaces | Trivial | mrv2 | Steve Loughran | Steve Loughran | +| [MAPREDUCE-3885](https://issues.apache.org/jira/browse/MAPREDUCE-3885) | Apply the fix similar to HADOOP-8084 | Major | mrv2 | Devaraj Das | Devaraj Das | +| [MAPREDUCE-3883](https://issues.apache.org/jira/browse/MAPREDUCE-3883) | Document yarn.nodemanager.delete.debug-delay-sec configuration property | Minor | documentation, mrv2 | Eugene Koontz | Eugene Koontz | +| [MAPREDUCE-2934](https://issues.apache.org/jira/browse/MAPREDUCE-2934) | MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process | Major | mrv2 | Aaron T. Myers | Aaron T. Myers | +| [MAPREDUCE-2887](https://issues.apache.org/jira/browse/MAPREDUCE-2887) | MR changes to match HADOOP-7524 (multiple RPC protocols) | Major | . | Sanjay Radia | Sanjay Radia | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8355](https://issues.apache.org/jira/browse/HADOOP-8355) | SPNEGO filter throws/logs exception when authentication fails | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8349](https://issues.apache.org/jira/browse/HADOOP-8349) | ViewFS doesn't work when the root of a file system is mounted | Major | viewfs | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-8347](https://issues.apache.org/jira/browse/HADOOP-8347) | Hadoop Common logs misspell 'successful' | Major | security | Philip Zeyliger | Philip Zeyliger | +| [HADOOP-8310](https://issues.apache.org/jira/browse/HADOOP-8310) | FileContext#checkPath should handle URIs with no port | Major | fs | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-8309](https://issues.apache.org/jira/browse/HADOOP-8309) | Pseudo & Kerberos AuthenticationHandler should use getType() to create token | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8296](https://issues.apache.org/jira/browse/HADOOP-8296) | hadoop/yarn daemonlog usage wrong | Minor | . | Thomas Graves | Devaraj K | +| [HADOOP-8282](https://issues.apache.org/jira/browse/HADOOP-8282) | start-all.sh refers incorrectly start-dfs.sh existence for starting start-yarn.sh | Minor | scripts | Devaraj K | Devaraj K | +| [HADOOP-8275](https://issues.apache.org/jira/browse/HADOOP-8275) | Range check DelegationKey length | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8264](https://issues.apache.org/jira/browse/HADOOP-8264) | Remove irritating double double quotes in front of hostname | Trivial | . | Bernd Fondermann | Bernd Fondermann | +| [HADOOP-8263](https://issues.apache.org/jira/browse/HADOOP-8263) | Stringification of IPC calls not useful | Minor | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-8261](https://issues.apache.org/jira/browse/HADOOP-8261) | Har file system doesn't deal with FS URIs with a host but no port | Major | fs | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-8251](https://issues.apache.org/jira/browse/HADOOP-8251) | SecurityUtil.fetchServiceTicket broken after HADOOP-6941 | Blocker | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-8243](https://issues.apache.org/jira/browse/HADOOP-8243) | Security support broken in CLI (manual) failover controller | Critical | ha, security | Todd Lipcon | Todd Lipcon | +| [HADOOP-8238](https://issues.apache.org/jira/browse/HADOOP-8238) | NetUtils#getHostNameOfIP blows up if given ip:port string w/o port | Major | . | Eli Collins | Eli Collins | +| [HADOOP-8218](https://issues.apache.org/jira/browse/HADOOP-8218) | RPC.closeProxy shouldn't throw error when closing a mock | Critical | ipc, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-8204](https://issues.apache.org/jira/browse/HADOOP-8204) | TestHealthMonitor fails occasionally | Major | . | Tom White | Todd Lipcon | +| [HADOOP-8202](https://issues.apache.org/jira/browse/HADOOP-8202) | stopproxy() is not closing the proxies correctly | Minor | ipc | Hari Mankude | Hari Mankude | +| [HADOOP-8199](https://issues.apache.org/jira/browse/HADOOP-8199) | Fix issues in start-all.sh and stop-all.sh | Major | . | Nishan Shetty | Devaraj K | +| [HADOOP-8191](https://issues.apache.org/jira/browse/HADOOP-8191) | SshFenceByTcpPort uses netcat incorrectly | Major | ha | Philip Zeyliger | Todd Lipcon | +| [HADOOP-8189](https://issues.apache.org/jira/browse/HADOOP-8189) | LdapGroupsMapping shouldn't throw away IOException | Major | security | Jonathan Natkins | Jonathan Natkins | +| [HADOOP-8169](https://issues.apache.org/jira/browse/HADOOP-8169) | javadoc generation fails with java.lang.OutOfMemoryError: Java heap space | Critical | build | Thomas Graves | Thomas Graves | +| [HADOOP-8159](https://issues.apache.org/jira/browse/HADOOP-8159) | NetworkTopology: getLeaf should check for invalid topologies | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8119](https://issues.apache.org/jira/browse/HADOOP-8119) | Fix javac warnings in TestAuthenticationFilter | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-7988](https://issues.apache.org/jira/browse/HADOOP-7988) | Upper case in hostname part of the principals doesn't work with kerberos. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7968](https://issues.apache.org/jira/browse/HADOOP-7968) | Errant println left in RPC.getHighestSupportedProtocol | Minor | ipc | Todd Lipcon | Sho Shimauchi | +| [HADOOP-7940](https://issues.apache.org/jira/browse/HADOOP-7940) | method clear() in org.apache.hadoop.io.Text does not work | Major | io | Aaron, | Csaba Miklos | +| [HADOOP-7931](https://issues.apache.org/jira/browse/HADOOP-7931) | o.a.h.ipc.WritableRpcEngine should have a way to force initialization | Major | ipc | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7920](https://issues.apache.org/jira/browse/HADOOP-7920) | Remove Avro RPC | Major | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7900](https://issues.apache.org/jira/browse/HADOOP-7900) | LocalDirAllocator confChanged() accesses conf.get() twice | Major | fs | Ravi Gummadi | Ravi Gummadi | +| [HADOOP-7897](https://issues.apache.org/jira/browse/HADOOP-7897) | ProtobufRPCEngine client side exception mechanism is not consistent with WritableRpcEngine | Major | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7892](https://issues.apache.org/jira/browse/HADOOP-7892) | IPC logs too verbose after "RpcKind" introduction | Trivial | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7888](https://issues.apache.org/jira/browse/HADOOP-7888) | TestFailoverProxy fails intermittently on trunk | Major | test | Jason Lowe | Jason Lowe | +| [HADOOP-7833](https://issues.apache.org/jira/browse/HADOOP-7833) | Inner classes of org.apache.hadoop.ipc.protobuf.HadoopRpcProtos generates findbugs warnings which results in -1 for findbugs | Major | ipc | John Lee | John Lee | +| [HADOOP-7827](https://issues.apache.org/jira/browse/HADOOP-7827) | jsp pages missing DOCTYPE | Trivial | . | Dave Vronay | Dave Vronay | +| [HADOOP-7704](https://issues.apache.org/jira/browse/HADOOP-7704) | JsonFactory can be created only once and used for every next request to create JsonGenerator inside JMXJsonServlet | Minor | . | Devaraj K | Devaraj K | +| [HADOOP-7695](https://issues.apache.org/jira/browse/HADOOP-7695) | RPC.stopProxy can throw unintended exception while logging error | Major | ipc | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-7669](https://issues.apache.org/jira/browse/HADOOP-7669) | Fix newly introduced release audit warning. | Minor | . | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-6941](https://issues.apache.org/jira/browse/HADOOP-6941) | Support non-SUN JREs in UserGroupInformation | Major | . | Stephen Watt | Devaraj Das | +| [HADOOP-6924](https://issues.apache.org/jira/browse/HADOOP-6924) | Build fails with non-Sun JREs due to different pathing to the operating system architecture shared libraries | Major | . | Stephen Watt | Devaraj Das | +| [HDFS-3396](https://issues.apache.org/jira/browse/HDFS-3396) | FUSE build fails on Ubuntu 12.04 | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3395](https://issues.apache.org/jira/browse/HDFS-3395) | NN doesn't start with HA+security enabled and HTTP address set to 0.0.0.0 | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3376](https://issues.apache.org/jira/browse/HDFS-3376) | DFSClient fails to make connection to DN if there are many unusable cached sockets | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-3357](https://issues.apache.org/jira/browse/HDFS-3357) | DataXceiver reads from client socket with incorrect/no timeout | Critical | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-3351](https://issues.apache.org/jira/browse/HDFS-3351) | NameNode#initializeGenericKeys should always set fs.defaultFS regardless of whether HA or Federation is enabled | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3350](https://issues.apache.org/jira/browse/HDFS-3350) | findbugs warning: INodeFileUnderConstruction doesn't override INodeFile.equals(Object) | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3336](https://issues.apache.org/jira/browse/HDFS-3336) | hdfs launcher script will be better off not special casing namenode command with regards to hadoop.security.logger | Minor | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-3332](https://issues.apache.org/jira/browse/HDFS-3332) | NullPointerException in DN when directoryscanner is trying to report bad blocks | Major | datanode | amith | amith | +| [HDFS-3330](https://issues.apache.org/jira/browse/HDFS-3330) | If GetImageServlet throws an Error or RTE, response has HTTP "OK" status | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3328](https://issues.apache.org/jira/browse/HDFS-3328) | NPE in DataNode.getIpcPort | Minor | datanode | Uma Maheswara Rao G | Eli Collins | +| [HDFS-3326](https://issues.apache.org/jira/browse/HDFS-3326) | Append enabled log message uses the wrong variable | Trivial | namenode | J.Andreina | Matthew Jacobs | +| [HDFS-3314](https://issues.apache.org/jira/browse/HDFS-3314) | HttpFS operation for getHomeDirectory is incorrect | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3309](https://issues.apache.org/jira/browse/HDFS-3309) | HttpFS (Hoop) chmod not supporting octal and sticky bit permissions | Major | . | Romain Rigaux | Alejandro Abdelnur | +| [HDFS-3305](https://issues.apache.org/jira/browse/HDFS-3305) | GetImageServlet should consider SBN a valid requestor in a secure HA setup | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3303](https://issues.apache.org/jira/browse/HDFS-3303) | RemoteEditLogManifest doesn't need to implements Writable | Minor | namenode | Brandon Li | Brandon Li | +| [HDFS-3284](https://issues.apache.org/jira/browse/HDFS-3284) | bootstrapStandby fails in secure cluster | Minor | ha, security | Todd Lipcon | Todd Lipcon | +| [HDFS-3280](https://issues.apache.org/jira/browse/HDFS-3280) | DFSOutputStream.sync should not be synchronized | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-3268](https://issues.apache.org/jira/browse/HDFS-3268) | Hdfs mishandles token service & incompatible with HA | Critical | ha, hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3260](https://issues.apache.org/jira/browse/HDFS-3260) | TestDatanodeRegistration should set minimum DN version in addition to minimum NN version | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3256](https://issues.apache.org/jira/browse/HDFS-3256) | HDFS considers blocks under-replicated if topology script is configured with only 1 rack | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3255](https://issues.apache.org/jira/browse/HDFS-3255) | HA DFS returns wrong token service | Critical | ha, hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3254](https://issues.apache.org/jira/browse/HDFS-3254) | Branch-2 build broken due to wrong version number in fuse-dfs' pom.xml | Major | fuse-dfs | Anupam Seth | Anupam Seth | +| [HDFS-3248](https://issues.apache.org/jira/browse/HDFS-3248) | bootstrapstanby repeated twice in hdfs namenode usage message | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3236](https://issues.apache.org/jira/browse/HDFS-3236) | NameNode does not initialize generic conf keys when started with -initializeSharedEditsDir | Minor | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3234](https://issues.apache.org/jira/browse/HDFS-3234) | Accidentally left log message in GetConf after HDFS-3226 | Trivial | tools | Todd Lipcon | Todd Lipcon | +| [HDFS-3222](https://issues.apache.org/jira/browse/HDFS-3222) | DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block. | Major | hdfs-client | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-3214](https://issues.apache.org/jira/browse/HDFS-3214) | InterDatanodeProtocolServerSideTranslatorPB doesn't handle null response from initReplicaRecovery | Blocker | datanode | Todd Lipcon | Todd Lipcon | +| [HDFS-3210](https://issues.apache.org/jira/browse/HDFS-3210) | JsonUtil#toJsonMap for for a DatanodeInfo should use "ipAddr" instead of "name" | Major | . | Eli Collins | Eli Collins | +| [HDFS-3208](https://issues.apache.org/jira/browse/HDFS-3208) | Bogus entries in hosts files are incorrectly displayed in the report | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-3202](https://issues.apache.org/jira/browse/HDFS-3202) | NamespaceInfo PB translation drops build version | Major | datanode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3199](https://issues.apache.org/jira/browse/HDFS-3199) | TestValidateConfigurationSettings is failing | Major | . | Eli Collins | Todd Lipcon | +| [HDFS-3181](https://issues.apache.org/jira/browse/HDFS-3181) | testHardLeaseRecoveryAfterNameNodeRestart fails when length before restart is 1 byte less than CRC chunk size | Minor | test | Colin Patrick McCabe | Tsz Wo Nicholas Sze | +| [HDFS-3156](https://issues.apache.org/jira/browse/HDFS-3156) | TestDFSHAAdmin is failing post HADOOP-8202 | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3143](https://issues.apache.org/jira/browse/HDFS-3143) | TestGetBlocks.testGetBlocks is failing | Major | test | Eli Collins | Arpit Gupta | +| [HDFS-3142](https://issues.apache.org/jira/browse/HDFS-3142) | TestHDFSCLI.testAll is failing | Blocker | test | Eli Collins | Brandon Li | +| [HDFS-3132](https://issues.apache.org/jira/browse/HDFS-3132) | Findbugs warning on HDFS trunk | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3119](https://issues.apache.org/jira/browse/HDFS-3119) | Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file | Minor | namenode | J.Andreina | Ashish Singhi | +| [HDFS-3109](https://issues.apache.org/jira/browse/HDFS-3109) | Remove hsqldb exclusions from pom.xml | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-3100](https://issues.apache.org/jira/browse/HDFS-3100) | failed to append data | Major | datanode | Zhanwei Wang | Brandon Li | +| [HDFS-3099](https://issues.apache.org/jira/browse/HDFS-3099) | SecondaryNameNode does not properly initialize metrics system | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3093](https://issues.apache.org/jira/browse/HDFS-3093) | TestAllowFormat is trying to be interactive | Critical | . | Todd Lipcon | Todd Lipcon | +| [HDFS-3083](https://issues.apache.org/jira/browse/HDFS-3083) | Cannot run an MR job with HA and security enabled when second-listed NN active | Critical | ha, security | Mingjie Lai | Aaron T. Myers | +| [HDFS-3070](https://issues.apache.org/jira/browse/HDFS-3070) | HDFS balancer doesn't ensure that hdfs-site.xml is loaded | Major | balancer & mover | Stephen Chu | Aaron T. Myers | +| [HDFS-3062](https://issues.apache.org/jira/browse/HDFS-3062) | Fail to submit mapred job on a secured-HA-HDFS: logic URI cannot be picked up by job submission. | Critical | ha, security | Mingjie Lai | Mingjie Lai | +| [HDFS-3057](https://issues.apache.org/jira/browse/HDFS-3057) | httpfs and hdfs launcher scripts should honor CATALINA\_HOME and HADOOP\_LIBEXEC\_DIR | Major | scripts | Roman Shaposhnik | Roman Shaposhnik | +| [HDFS-3038](https://issues.apache.org/jira/browse/HDFS-3038) | Add FSEditLog.metrics to findbugs exclude list | Trivial | . | Todd Lipcon | Todd Lipcon | +| [HDFS-3026](https://issues.apache.org/jira/browse/HDFS-3026) | HA: Handle failure during HA state transition | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3020](https://issues.apache.org/jira/browse/HDFS-3020) | Auto-logSync based on edit log buffer size broken | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3005](https://issues.apache.org/jira/browse/HDFS-3005) | ConcurrentModificationException in FSDataset$FSVolume.getDfsUsed(..) | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2995](https://issues.apache.org/jira/browse/HDFS-2995) | start-dfs.sh should only start the 2NN for namenodes with dfs.namenode.secondary.http-address configured | Major | scripts | Todd Lipcon | Eli Collins | +| [HDFS-2968](https://issues.apache.org/jira/browse/HDFS-2968) | Protocol translator for BlockRecoveryCommand broken when multiple blocks need recovery | Blocker | datanode, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2878](https://issues.apache.org/jira/browse/HDFS-2878) | TestBlockRecovery does not compile | Blocker | test | Eli Collins | Todd Lipcon | +| [HDFS-2799](https://issues.apache.org/jira/browse/HDFS-2799) | Trim fs.checkpoint.dir values | Major | namenode | Eli Collins | amith | +| [HDFS-2768](https://issues.apache.org/jira/browse/HDFS-2768) | BackupNode stop can not close proxy connections because it is not a proxy instance. | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2765](https://issues.apache.org/jira/browse/HDFS-2765) | TestNameEditsConfigs is incorrectly swallowing IOE | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2739](https://issues.apache.org/jira/browse/HDFS-2739) | SecondaryNameNode doesn't start up | Critical | . | Sho Shimauchi | Jitendra Nath Pandey | +| [HDFS-2700](https://issues.apache.org/jira/browse/HDFS-2700) | TestDataNodeMultipleRegistrations is failing in trunk | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2696](https://issues.apache.org/jira/browse/HDFS-2696) | Fix the fuse-fds build | Major | build, fuse-dfs | Petru Dimulescu | Bruno MahĂ© | +| [HDFS-2694](https://issues.apache.org/jira/browse/HDFS-2694) | Removal of Avro broke non-PB NN services | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2676](https://issues.apache.org/jira/browse/HDFS-2676) | Remove Avro RPC | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2532](https://issues.apache.org/jira/browse/HDFS-2532) | TestDfsOverAvroRpc timing out in trunk | Critical | test | Todd Lipcon | Uma Maheswara Rao G | +| [HDFS-2526](https://issues.apache.org/jira/browse/HDFS-2526) | (Client)NamenodeProtocolTranslatorR23 do not need to keep a reference to rpcProxyWithoutRetry | Major | hdfs-client, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2497](https://issues.apache.org/jira/browse/HDFS-2497) | Fix TestBackupNode failure | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2481](https://issues.apache.org/jira/browse/HDFS-2481) | Unknown protocol: org.apache.hadoop.hdfs.protocol.ClientProtocol | Major | . | Tsz Wo Nicholas Sze | Sanjay Radia | +| [HDFS-2405](https://issues.apache.org/jira/browse/HDFS-2405) | hadoop dfs command with webhdfs fails on secure hadoop | Critical | webhdfs | Arpit Gupta | Jitendra Nath Pandey | +| [HDFS-1765](https://issues.apache.org/jira/browse/HDFS-1765) | Block Replication should respect under-replication block priority | Major | namenode | Hairong Kuang | Uma Maheswara Rao G | +| [HDFS-891](https://issues.apache.org/jira/browse/HDFS-891) | DataNode no longer needs to check for dfs.network.script | Minor | datanode | Steve Loughran | Harsh J | +| [MAPREDUCE-4231](https://issues.apache.org/jira/browse/MAPREDUCE-4231) | Update RAID to not to use FSInodeInfo | Major | contrib/raid | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-4202](https://issues.apache.org/jira/browse/MAPREDUCE-4202) | TestYarnClientProtocolProvider is broken | Major | test | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4193](https://issues.apache.org/jira/browse/MAPREDUCE-4193) | broken doc link for yarn-default.xml in site.xml | Major | documentation | Patrick Hunt | Patrick Hunt | +| [MAPREDUCE-4147](https://issues.apache.org/jira/browse/MAPREDUCE-4147) | YARN should not have a compile-time dependency on HDFS | Major | . | Tom White | Tom White | +| [MAPREDUCE-4105](https://issues.apache.org/jira/browse/MAPREDUCE-4105) | Yarn RackResolver ignores rack configurations | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4098](https://issues.apache.org/jira/browse/MAPREDUCE-4098) | TestMRApps testSetClasspath fails | Major | test | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4095](https://issues.apache.org/jira/browse/MAPREDUCE-4095) | TestJobInProgress#testLocality uses a bogus topology | Major | . | Eli Collins | Colin Patrick McCabe | +| [MAPREDUCE-4081](https://issues.apache.org/jira/browse/MAPREDUCE-4081) | TestMROutputFormat.java does not compile | Blocker | build, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4076](https://issues.apache.org/jira/browse/MAPREDUCE-4076) | Stream job fails with ZipException when use yarn jar command | Blocker | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4066](https://issues.apache.org/jira/browse/MAPREDUCE-4066) | To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value | Minor | job submission, mrv2 | xieguiming | xieguiming | +| [MAPREDUCE-4057](https://issues.apache.org/jira/browse/MAPREDUCE-4057) | Compilation error in RAID | Major | contrib/raid | Tsz Wo Nicholas Sze | Devaraj K | +| [MAPREDUCE-4008](https://issues.apache.org/jira/browse/MAPREDUCE-4008) | ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists | Major | mrv2, scheduler | Devaraj K | Devaraj K | +| [MAPREDUCE-4007](https://issues.apache.org/jira/browse/MAPREDUCE-4007) | JobClient getJob(JobID) should return NULL if the job does not exist (for backwards compatibility) | Major | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3974](https://issues.apache.org/jira/browse/MAPREDUCE-3974) | TestSubmitJob in MR1 tests doesn't compile after HDFS-1623 merge | Blocker | . | Arun C Murthy | Aaron T. Myers | +| [MAPREDUCE-3958](https://issues.apache.org/jira/browse/MAPREDUCE-3958) | RM: Remove RMNodeState and replace it with NodeState | Major | mrv2 | Bikas Saha | Bikas Saha | +| [MAPREDUCE-3952](https://issues.apache.org/jira/browse/MAPREDUCE-3952) | In MR2, when Total input paths to process == 1, CombinefileInputFormat.getSplits() returns 0 split. | Major | mrv2 | Zhenxiao Luo | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3933](https://issues.apache.org/jira/browse/MAPREDUCE-3933) | Failures because MALLOC\_ARENA\_MAX is not set | Major | mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-3916](https://issues.apache.org/jira/browse/MAPREDUCE-3916) | various issues with running yarn proxyserver | Critical | mrv2, resourcemanager, webapps | Roman Shaposhnik | Devaraj K | +| [MAPREDUCE-3869](https://issues.apache.org/jira/browse/MAPREDUCE-3869) | Distributed shell application fails with NoClassDefFoundError | Blocker | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3867](https://issues.apache.org/jira/browse/MAPREDUCE-3867) | MiniMRYarn/MiniYarn uses fixed ports | Major | test | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-3818](https://issues.apache.org/jira/browse/MAPREDUCE-3818) | Trunk MRV1 compilation is broken. | Blocker | build, test | Vinod Kumar Vavilapalli | Suresh Srinivas | +| [MAPREDUCE-3740](https://issues.apache.org/jira/browse/MAPREDUCE-3740) | Mapreduce Trunk compilation fails | Blocker | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3578](https://issues.apache.org/jira/browse/MAPREDUCE-3578) | starting nodemanager as 'root' gives "Unknown -jvm option" | Major | nodemanager | Gilad Wolff | Tom White | +| [MAPREDUCE-3545](https://issues.apache.org/jira/browse/MAPREDUCE-3545) | Remove Avro RPC | Major | . | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-3431](https://issues.apache.org/jira/browse/MAPREDUCE-3431) | NPE in Resource Manager shutdown | Minor | resourcemanager | Steve Loughran | Steve Loughran | +| [MAPREDUCE-3377](https://issues.apache.org/jira/browse/MAPREDUCE-3377) | Compatibility issue with 0.20.203. | Major | . | Jane Chen | Jane Chen | +| [MAPREDUCE-3353](https://issues.apache.org/jira/browse/MAPREDUCE-3353) | Need a RM-\>AM channel to inform AMs about faulty/unhealthy/lost nodes | Major | applicationmaster, mrv2, resourcemanager | Vinod Kumar Vavilapalli | Bikas Saha | +| [MAPREDUCE-3173](https://issues.apache.org/jira/browse/MAPREDUCE-3173) | MRV2 UI doesn't work properly without internet | Critical | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-2942](https://issues.apache.org/jira/browse/MAPREDUCE-2942) | TestNMAuditLogger.testNMAuditLoggerWithIP failing | Critical | . | Vinod Kumar Vavilapalli | Thomas Graves | +| [MAPREDUCE-1740](https://issues.apache.org/jira/browse/MAPREDUCE-1740) | NPE in getMatchingLevelForNodes when node locations are variable depth | Major | jobtracker | Todd Lipcon | Ahmed Radwan | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8157](https://issues.apache.org/jira/browse/HADOOP-8157) | TestRPCCallBenchmark#testBenchmarkWithWritable fails with RTE | Major | . | Eli Collins | Todd Lipcon | +| [HDFS-3129](https://issues.apache.org/jira/browse/HDFS-3129) | NetworkTopology: add test that getLeaf should check for invalid topologies | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8211](https://issues.apache.org/jira/browse/HADOOP-8211) | Update commons-net version to 3.1 | Major | io, performance | Eli Collins | Eli Collins | +| [HADOOP-8210](https://issues.apache.org/jira/browse/HADOOP-8210) | Common side of HDFS-3148 | Major | io, performance | Eli Collins | Eli Collins | +| [HADOOP-7994](https://issues.apache.org/jira/browse/HADOOP-7994) | Remove getProtocolVersion and getProtocolSignature from the client side translator and server side implementation | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7965](https://issues.apache.org/jira/browse/HADOOP-7965) | Support for protocol version and signature in PB | Major | ipc | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HADOOP-7913](https://issues.apache.org/jira/browse/HADOOP-7913) | Fix bug in ProtoBufRpcEngine - | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-7862](https://issues.apache.org/jira/browse/HADOOP-7862) | Move the support for multiple protocols to lower layer so that Writable, PB and Avro can all use it | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-7776](https://issues.apache.org/jira/browse/HADOOP-7776) | Make the Ipc-Header in a RPC-Payload an explicit header | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-7773](https://issues.apache.org/jira/browse/HADOOP-7773) | Add support for protocol buffer based RPC engine | Major | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-7557](https://issues.apache.org/jira/browse/HADOOP-7557) | Make IPC header be extensible | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-3363](https://issues.apache.org/jira/browse/HDFS-3363) | blockmanagement should stop using INodeFile & INodeFileUC | Minor | namenode | John George | John George | +| [HDFS-3339](https://issues.apache.org/jira/browse/HDFS-3339) | change INode to package private | Minor | namenode | John George | John George | +| [HDFS-3322](https://issues.apache.org/jira/browse/HDFS-3322) | Update file context to use HdfsDataInputStream and HdfsDataOutputStream | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3298](https://issues.apache.org/jira/browse/HDFS-3298) | Add HdfsDataOutputStream as a public API | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3282](https://issues.apache.org/jira/browse/HDFS-3282) | Add HdfsDataInputStream as a public API | Major | hdfs-client | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-3211](https://issues.apache.org/jira/browse/HDFS-3211) | JournalProtocol changes required for introducing epoch and fencing | Major | ha, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-3187](https://issues.apache.org/jira/browse/HDFS-3187) | Upgrade guava to 11.0.2 | Minor | build | Todd Lipcon | Todd Lipcon | +| [HDFS-3155](https://issues.apache.org/jira/browse/HDFS-3155) | Clean up FSDataset implemenation related code. | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3130](https://issues.apache.org/jira/browse/HDFS-3130) | Move FSDataset implemenation to a package | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3126](https://issues.apache.org/jira/browse/HDFS-3126) | Journal stream from the namenode to backup needs to have a timeout | Major | ha, namenode | Hari Mankude | Hari Mankude | +| [HDFS-3105](https://issues.apache.org/jira/browse/HDFS-3105) | Add DatanodeStorage information to block recovery | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3089](https://issues.apache.org/jira/browse/HDFS-3089) | Move FSDatasetInterface and other related classes/interfaces to a package | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3088](https://issues.apache.org/jira/browse/HDFS-3088) | Move FSDatasetInterface inner classes to a package | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3086](https://issues.apache.org/jira/browse/HDFS-3086) | Change Datanode not to send storage list in registration - it will be sent in block report | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3082](https://issues.apache.org/jira/browse/HDFS-3082) | Clean up FSDatasetInterface | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-2899](https://issues.apache.org/jira/browse/HDFS-2899) | Service protocol change to support multiple storages added in HDFS-2880 | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2880](https://issues.apache.org/jira/browse/HDFS-2880) | Protocol buffer changes to add support multiple storages | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2801](https://issues.apache.org/jira/browse/HDFS-2801) | Provide a method in client side translators to check for a methods supported in underlying protocol. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2697](https://issues.apache.org/jira/browse/HDFS-2697) | Move RefreshAuthPolicy, RefreshUserMappings, GetUserMappings protocol to protocol buffers | Major | . | Suresh Srinivas | Jitendra Nath Pandey | +| [HDFS-2687](https://issues.apache.org/jira/browse/HDFS-2687) | Tests are failing with ClassCastException, due to new protocol changes | Major | test | Uma Maheswara Rao G | Suresh Srinivas | +| [HDFS-2669](https://issues.apache.org/jira/browse/HDFS-2669) | Enable protobuf rpc for ClientNamenodeProtocol | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-2666](https://issues.apache.org/jira/browse/HDFS-2666) | TestBackupNode fails | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2663](https://issues.apache.org/jira/browse/HDFS-2663) | Optional parameters are not handled correctly | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2661](https://issues.apache.org/jira/browse/HDFS-2661) | Enable protobuf RPC for DatanodeProtocol | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2651](https://issues.apache.org/jira/browse/HDFS-2651) | ClientNameNodeProtocol Translators for Protocol Buffers | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-2647](https://issues.apache.org/jira/browse/HDFS-2647) | Enable protobuf RPC for InterDatanodeProtocol, ClientDatanodeProtocol, JournalProtocol and NamenodeProtocol | Major | balancer & mover, datanode, hdfs-client, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2642](https://issues.apache.org/jira/browse/HDFS-2642) | Protobuf translators for DatanodeProtocol | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2636](https://issues.apache.org/jira/browse/HDFS-2636) | Implement protobuf service for ClientDatanodeProtocol | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2629](https://issues.apache.org/jira/browse/HDFS-2629) | Implement protobuf service for InterDatanodeProtocol | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2618](https://issues.apache.org/jira/browse/HDFS-2618) | Implement protobuf service for NamenodeProtocol | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2597](https://issues.apache.org/jira/browse/HDFS-2597) | ClientNameNodeProtocol in Protocol Buffers | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-2581](https://issues.apache.org/jira/browse/HDFS-2581) | Implement protobuf service for JournalProtocol | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2520](https://issues.apache.org/jira/browse/HDFS-2520) | Protobuf - Add protobuf service for InterDatanodeProtocol | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2519](https://issues.apache.org/jira/browse/HDFS-2519) | Protobuf - Add protobuf service for DatanodeProtocol | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2518](https://issues.apache.org/jira/browse/HDFS-2518) | Protobuf - Add protobuf service for NamenodeProtocol | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2517](https://issues.apache.org/jira/browse/HDFS-2517) | Protobuf - Add protocol service for JournalProtocol | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2499](https://issues.apache.org/jira/browse/HDFS-2499) | Fix RPC client creation bug from HDFS-2459 | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2495](https://issues.apache.org/jira/browse/HDFS-2495) | Increase granularity of write operations in ReplicationMonitor thus reducing contention for write lock | Major | namenode | Tomasz Nykiel | Tomasz Nykiel | +| [HDFS-2489](https://issues.apache.org/jira/browse/HDFS-2489) | Move commands Finalize and Register out of DatanodeCommand class. | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2488](https://issues.apache.org/jira/browse/HDFS-2488) | Separate datatypes for InterDatanodeProtocol | Major | datanode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2480](https://issues.apache.org/jira/browse/HDFS-2480) | Separate datatypes for NamenodeProtocol | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2479](https://issues.apache.org/jira/browse/HDFS-2479) | HDFS Client Data Types in Protocol Buffers | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-2477](https://issues.apache.org/jira/browse/HDFS-2477) | Optimize computing the diff between a block report and the namenode state. | Major | namenode | Tomasz Nykiel | Tomasz Nykiel | +| [HDFS-2476](https://issues.apache.org/jira/browse/HDFS-2476) | More CPU efficient data structure for under-replicated/over-replicated/invalidate blocks | Major | namenode | Tomasz Nykiel | Tomasz Nykiel | +| [HDFS-2459](https://issues.apache.org/jira/browse/HDFS-2459) | Separate datatypes for Journal protocol | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2334](https://issues.apache.org/jira/browse/HDFS-2334) | Add Closeable to JournalManager | Major | namenode | Ivan Kelly | Ivan Kelly | +| [HDFS-2181](https://issues.apache.org/jira/browse/HDFS-2181) | Separate HDFS Client wire protocol data types | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-2158](https://issues.apache.org/jira/browse/HDFS-2158) | Add JournalSet to manage the set of journals. | Major | . | Jitendra Nath Pandey | Jitendra Nath Pandey | +| [HDFS-2018](https://issues.apache.org/jira/browse/HDFS-2018) | 1073: Move all journal stream management code into one place | Major | . | Ivan Kelly | Ivan Kelly | +| [MAPREDUCE-4113](https://issues.apache.org/jira/browse/MAPREDUCE-4113) | Fix tests org.apache.hadoop.mapred.TestClusterMRNotification | Major | mrv2, test | Devaraj K | Devaraj K | +| [MAPREDUCE-4112](https://issues.apache.org/jira/browse/MAPREDUCE-4112) | Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase | Major | mrv2, test | Devaraj K | Devaraj K | +| [MAPREDUCE-4111](https://issues.apache.org/jira/browse/MAPREDUCE-4111) | Fix tests in org.apache.hadoop.mapred.TestJobName | Major | mrv2, test | Devaraj K | Devaraj K | +| [MAPREDUCE-4110](https://issues.apache.org/jira/browse/MAPREDUCE-4110) | Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath & org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers | Major | mrv2, test | Devaraj K | Devaraj K | +| [MAPREDUCE-4108](https://issues.apache.org/jira/browse/MAPREDUCE-4108) | Fix tests in org.apache.hadoop.util.TestRunJar | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4107](https://issues.apache.org/jira/browse/MAPREDUCE-4107) | Fix tests in org.apache.hadoop.ipc.TestSocketFactory | Major | mrv2 | Devaraj K | Devaraj K | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-860](https://issues.apache.org/jira/browse/HDFS-860) | fuse-dfs truncate behavior causes issues with scp | Minor | fuse-dfs | Brian Bockelman | Brian Bockelman | +| [HADOOP-8359](https://issues.apache.org/jira/browse/HADOOP-8359) | Clear up javadoc warnings in hadoop-common-project | Trivial | conf | Harsh J | Anupam Seth | +| [HADOOP-8142](https://issues.apache.org/jira/browse/HADOOP-8142) | Update versions from 0.23.2 to 0.23.3 | Major | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3111](https://issues.apache.org/jira/browse/HDFS-3111) | Missing license headers in trunk | Trivial | . | Todd Lipcon | Uma Maheswara Rao G | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/RELEASENOTES.2.0.0-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/RELEASENOTES.2.0.0-alpha.md new file mode 100644 index 00000000000..e013b64b1c4 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.0-alpha/RELEASENOTES.2.0.0-alpha.md @@ -0,0 +1,157 @@ + + +# Apache Hadoop 2.0.0-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8314](https://issues.apache.org/jira/browse/HADOOP-8314) | *Major* | **HttpServer#hasAdminAccess should return false if authorization is enabled but user is not authenticated** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8270](https://issues.apache.org/jira/browse/HADOOP-8270) | *Minor* | **hadoop-daemon.sh stop action should return 0 for an already stopped service** + +The daemon stop action no longer returns failure when stopping an already stopped service. + + +--- + +* [HADOOP-8184](https://issues.apache.org/jira/browse/HADOOP-8184) | *Major* | **ProtoBuf RPC engine does not need it own reply packet - it can use the IPC layer reply packet.** + +This change will affect the output of errors for some Hadoop CLI commands. Specifically, the name of the exception class will no longer appear, and instead only the text of the exception message will appear. + + +--- + +* [HADOOP-8154](https://issues.apache.org/jira/browse/HADOOP-8154) | *Major* | **DNS#getIPs shouldn't silently return the local host IP for bogus interface names** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8149](https://issues.apache.org/jira/browse/HADOOP-8149) | *Major* | **cap space usage of default log4j rolling policy** + +Hadoop log files are now rolled by size instead of date (daily) by default. Tools that depend on the log file name format will need to be updated. Users who would like to maintain the previous settings of hadoop.root.logger and hadoop.security.logger can use their current log4j.properties files and update the HADOOP\_ROOT\_LOGGER and HADOOP\_SECURITY\_LOGGER environment variables to use DRFA and DRFAS respectively. + + +--- + +* [HADOOP-7704](https://issues.apache.org/jira/browse/HADOOP-7704) | *Minor* | **JsonFactory can be created only once and used for every next request to create JsonGenerator inside JMXJsonServlet** + +Reduce number of object created by JMXJsonServlet. (Devaraj K via Eric Yang) + + +--- + +* [HADOOP-7524](https://issues.apache.org/jira/browse/HADOOP-7524) | *Major* | **Change RPC to allow multiple protocols including multiple versions of the same protocol** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-3286](https://issues.apache.org/jira/browse/HDFS-3286) | *Major* | **When the threshold value for balancer is 0(zero) ,unexpected output is displayed** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-3164](https://issues.apache.org/jira/browse/HDFS-3164) | *Major* | **Move DatanodeInfo#hostName to DatanodeID** + +This change modifies DatanodeID, which is part of the client to server protocol, therefore clients must be upgraded with servers. + + +--- + +* [HDFS-3144](https://issues.apache.org/jira/browse/HDFS-3144) | *Major* | **Refactor DatanodeID#getName by use** + +This change modifies DatanodeID, which is part of the client to server protocol, therefore clients must be upgraded with servers. + + +--- + +* [HDFS-3138](https://issues.apache.org/jira/browse/HDFS-3138) | *Major* | **Move DatanodeInfo#ipcPort to DatanodeID** + +This change modifies DatanodeID, which is part of the client to server protocol, therefore clients must be upgraded with servers. + + +--- + +* [HDFS-3137](https://issues.apache.org/jira/browse/HDFS-3137) | *Major* | **Bump LAST\_UPGRADABLE\_LAYOUT\_VERSION to -16** + +Upgrade from Hadoop versions earlier than 0.18 is not supported as of 2.0. To upgrade from an earlier release, first upgrade to 0.18, and then upgrade again from there. + + +--- + +* [HDFS-3094](https://issues.apache.org/jira/browse/HDFS-3094) | *Major* | **add -nonInteractive and -force option to namenode -format command** + +The 'namenode -format' command now supports the flags '-nonInteractive' and '-force' to improve usefulness without user input. + + +--- + +* [HDFS-3044](https://issues.apache.org/jira/browse/HDFS-3044) | *Major* | **fsck move should be non-destructive by default** + +The fsck "move" option is no longer destructive. It copies the accessible blocks of corrupt files to lost and found as before, but no longer deletes the corrupt files after copying the blocks. The original, destructive behavior can be enabled by specifying both the "move" and "delete" options. + + +--- + +* [HDFS-3004](https://issues.apache.org/jira/browse/HDFS-3004) | *Major* | **Implement Recovery Mode** + +This is a new feature. It is documented in hdfs\_user\_guide.xml. + + +--- + +* [HDFS-2731](https://issues.apache.org/jira/browse/HDFS-2731) | *Major* | **HA: Autopopulate standby name dirs if they're empty** + +The HA NameNode may now be started with the "-bootstrapStandby" flag. This causes it to copy the namespace information and most recent checkpoint from its HA pair, and save it to local storage, allowing an HA setup to be bootstrapped without use of rsync or external tools. + + +--- + +* [HDFS-2303](https://issues.apache.org/jira/browse/HDFS-2303) | *Major* | **Unbundle jsvc** + +To run secure Datanodes users must install jsvc for their platform and set JSVC\_HOME to point to the location of jsvc in their environment. + + +--- + +* [HDFS-395](https://issues.apache.org/jira/browse/HDFS-395) | *Major* | **DFS Scalability: Incremental block reports** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-3818](https://issues.apache.org/jira/browse/MAPREDUCE-3818) | *Blocker* | **Trunk MRV1 compilation is broken.** + +Fixed broken compilation in TestSubmitJob after the patch for HDFS-2895. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/CHANGES.2.0.1-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/CHANGES.2.0.1-alpha.md new file mode 100644 index 00000000000..acd6e8f9fae --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/CHANGES.2.0.1-alpha.md @@ -0,0 +1,66 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.1-alpha - 2012-07-25 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8552](https://issues.apache.org/jira/browse/HADOOP-8552) | Conflict: Same security.log.file for multiple users. | Major | conf, security | Karthik Kambatla | Karthik Kambatla | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-3667](https://issues.apache.org/jira/browse/HDFS-3667) | Add retry support to WebHdfsFileSystem | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/RELEASENOTES.2.0.1-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/RELEASENOTES.2.0.1-alpha.md new file mode 100644 index 00000000000..ca573635493 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.1-alpha/RELEASENOTES.2.0.1-alpha.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 2.0.1-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8552](https://issues.apache.org/jira/browse/HADOOP-8552) | *Major* | **Conflict: Same security.log.file for multiple users.** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/CHANGES.2.0.2-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/CHANGES.2.0.2-alpha.md new file mode 100644 index 00000000000..a9bbab806e6 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/CHANGES.2.0.2-alpha.md @@ -0,0 +1,714 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.2-alpha - 2012-10-09 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8794](https://issues.apache.org/jira/browse/HADOOP-8794) | Modifiy bin/hadoop to point to HADOOP\_YARN\_HOME | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [HADOOP-8710](https://issues.apache.org/jira/browse/HADOOP-8710) | Remove ability for users to easily run the trash emptier | Major | fs | Eli Collins | Eli Collins | +| [HADOOP-8689](https://issues.apache.org/jira/browse/HADOOP-8689) | Make trash a server side configuration option | Major | fs | Eli Collins | Eli Collins | +| [HADOOP-8551](https://issues.apache.org/jira/browse/HADOOP-8551) | fs -mkdir creates parent directories without the -p option | Major | fs | Robert Joseph Evans | John George | +| [HADOOP-8458](https://issues.apache.org/jira/browse/HADOOP-8458) | Add management hook to AuthenticationHandler to enable delegation token operations support | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8388](https://issues.apache.org/jira/browse/HADOOP-8388) | Remove unused BlockLocation serialization | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8368](https://issues.apache.org/jira/browse/HADOOP-8368) | Use CMake rather than autotools to build native code | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3675](https://issues.apache.org/jira/browse/HDFS-3675) | libhdfs: follow documented return codes | Minor | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3522](https://issues.apache.org/jira/browse/HDFS-3522) | If NN is in safemode, it should throw SafeModeException when getBlockLocations has zero locations | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-3446](https://issues.apache.org/jira/browse/HDFS-3446) | HostsFileReader silently ignores bad includes/excludes | Major | namenode | Matthew Jacobs | Matthew Jacobs | +| [HDFS-3318](https://issues.apache.org/jira/browse/HDFS-3318) | Hftp hangs on transfers \>2GB | Blocker | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-2727](https://issues.apache.org/jira/browse/HDFS-2727) | libhdfs should get the default block size from the server | Minor | libhdfs | Sho Shimauchi | Colin Patrick McCabe | +| [HDFS-2686](https://issues.apache.org/jira/browse/HDFS-2686) | Remove DistributedUpgrade related code | Major | datanode, namenode | Todd Lipcon | Suresh Srinivas | +| [HDFS-2617](https://issues.apache.org/jira/browse/HDFS-2617) | Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution | Major | security | Jakob Homan | Jakob Homan | +| [MAPREDUCE-4629](https://issues.apache.org/jira/browse/MAPREDUCE-4629) | Remove JobHistory.DEBUG\_MODE | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4311](https://issues.apache.org/jira/browse/MAPREDUCE-4311) | Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings | Major | capacity-sched, mrv2 | Thomas Graves | Karthik Kambatla | +| [MAPREDUCE-4072](https://issues.apache.org/jira/browse/MAPREDUCE-4072) | User set java.library.path seems to overwrite default creating problems native lib loading | Major | mrv2 | Anupam Seth | Anupam Seth | +| [MAPREDUCE-3812](https://issues.apache.org/jira/browse/MAPREDUCE-3812) | Lower default allocation sizes, fix allocation configurations and document them | Major | mrv2, performance | Vinod Kumar Vavilapalli | Harsh J | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8644](https://issues.apache.org/jira/browse/HADOOP-8644) | AuthenticatedURL should be able to use SSLFactory | Critical | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8581](https://issues.apache.org/jira/browse/HADOOP-8581) | add support for HTTPS to the web UIs | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8465](https://issues.apache.org/jira/browse/HADOOP-8465) | hadoop-auth should support ephemeral authentication | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8135](https://issues.apache.org/jira/browse/HADOOP-8135) | Add ByteBufferReadable interface to FSDataInputStream | Major | fs | Henry Robinson | Henry Robinson | +| [HDFS-3637](https://issues.apache.org/jira/browse/HDFS-3637) | Add support for encrypting the DataTransferProtocol | Major | datanode, hdfs-client, security | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3535](https://issues.apache.org/jira/browse/HDFS-3535) | Audit logging should log denied accesses | Major | namenode | Andy Isaacson | Andy Isaacson | +| [HDFS-3150](https://issues.apache.org/jira/browse/HDFS-3150) | Add option for clients to contact DNs via hostname | Major | datanode, hdfs-client | Eli Collins | Eli Collins | +| [HDFS-3113](https://issues.apache.org/jira/browse/HDFS-3113) | httpfs does not support delegation tokens | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3042](https://issues.apache.org/jira/browse/HDFS-3042) | Automatic failover support for NN HA | Major | auto-failover, ha | Todd Lipcon | Todd Lipcon | +| [HDFS-2978](https://issues.apache.org/jira/browse/HDFS-2978) | The NameNode should expose name dir statuses via JMX | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2793](https://issues.apache.org/jira/browse/HDFS-2793) | Add an admin command to trigger an edit log roll | Major | namenode | Aaron T. Myers | Todd Lipcon | +| [HDFS-744](https://issues.apache.org/jira/browse/HDFS-744) | Support hsync in HDFS | Major | datanode, hdfs-client | Hairong Kuang | Lars Hofhansl | +| [MAPREDUCE-4417](https://issues.apache.org/jira/browse/MAPREDUCE-4417) | add support for encrypted shuffle | Major | mrv2, security | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4355](https://issues.apache.org/jira/browse/MAPREDUCE-4355) | Add RunningJob.getJobStatus() | Major | mrv1, mrv2 | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-3773](https://issues.apache.org/jira/browse/MAPREDUCE-3773) | Add queue metrics with buckets for job run times | Major | jobtracker | Owen O'Malley | Owen O'Malley | +| [MAPREDUCE-3451](https://issues.apache.org/jira/browse/MAPREDUCE-3451) | Port Fair Scheduler to MR2 | Major | mrv2, scheduler | NO NAME | NO NAME | +| [MAPREDUCE-987](https://issues.apache.org/jira/browse/MAPREDUCE-987) | Exposing MiniDFS and MiniMR clusters as a single process command-line | Minor | build, test | Philip Zeyliger | Ahmed Radwan | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8754](https://issues.apache.org/jira/browse/HADOOP-8754) | Deprecate all the RPC.getServer() variants | Minor | ipc | Brandon Li | Brandon Li | +| [HADOOP-8748](https://issues.apache.org/jira/browse/HADOOP-8748) | Move dfsclient retry to a util class | Minor | io | Arun C Murthy | Arun C Murthy | +| [HADOOP-8700](https://issues.apache.org/jira/browse/HADOOP-8700) | Move the checksum type constants to an enum | Minor | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-8687](https://issues.apache.org/jira/browse/HADOOP-8687) | Upgrade log4j to 1.2.17 | Minor | . | Eli Collins | Eli Collins | +| [HADOOP-8635](https://issues.apache.org/jira/browse/HADOOP-8635) | Cannot cancel paths registered deleteOnExit | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8624](https://issues.apache.org/jira/browse/HADOOP-8624) | ProtobufRpcEngine should log all RPCs if TRACE logging is enabled | Minor | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-8623](https://issues.apache.org/jira/browse/HADOOP-8623) | hadoop jar command should respect HADOOP\_OPTS | Minor | scripts | Steven Willis | Steven Willis | +| [HADOOP-8620](https://issues.apache.org/jira/browse/HADOOP-8620) | Add -Drequire.fuse and -Drequire.snappy | Minor | build | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8609](https://issues.apache.org/jira/browse/HADOOP-8609) | IPC server logs a useless message when shutting down socket | Major | . | Todd Lipcon | Jon Zuanich | +| [HADOOP-8541](https://issues.apache.org/jira/browse/HADOOP-8541) | Better high-percentile latency metrics | Major | metrics | Andrew Wang | Andrew Wang | +| [HADOOP-8535](https://issues.apache.org/jira/browse/HADOOP-8535) | Cut hadoop build times in half (upgrade maven-compiler-plugin to 2.5.1) | Major | build | Jonathan Eagles | Jonathan Eagles | +| [HADOOP-8533](https://issues.apache.org/jira/browse/HADOOP-8533) | Remove Parallel Call in IPC | Major | ipc | Suresh Srinivas | Brandon Li | +| [HADOOP-8531](https://issues.apache.org/jira/browse/HADOOP-8531) | SequenceFile Writer can throw out a better error if a serializer or deserializer isn't available | Trivial | io | Harsh J | madhukara phatak | +| [HADOOP-8525](https://issues.apache.org/jira/browse/HADOOP-8525) | Provide Improved Traceability for Configuration | Trivial | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8524](https://issues.apache.org/jira/browse/HADOOP-8524) | Allow users to get source of a Configuration parameter | Trivial | conf | Harsh J | Harsh J | +| [HADOOP-8463](https://issues.apache.org/jira/browse/HADOOP-8463) | hadoop.security.auth\_to\_local needs a key definition and doc | Major | security | Eli Collins | madhukara phatak | +| [HADOOP-8398](https://issues.apache.org/jira/browse/HADOOP-8398) | Cleanup BlockLocation | Minor | . | Eli Collins | Eli Collins | +| [HADOOP-8373](https://issues.apache.org/jira/browse/HADOOP-8373) | Port RPC.getServerAddress to 0.23 | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-8367](https://issues.apache.org/jira/browse/HADOOP-8367) | Improve documentation of declaringClassProtocolName in rpc headers | Major | . | Sanjay Radia | Sanjay Radia | +| [HADOOP-8362](https://issues.apache.org/jira/browse/HADOOP-8362) | Improve exception message when Configuration.set() is called with a null key or value | Trivial | conf | Todd Lipcon | madhukara phatak | +| [HADOOP-8361](https://issues.apache.org/jira/browse/HADOOP-8361) | Avoid out-of-memory problems when deserializing strings | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8358](https://issues.apache.org/jira/browse/HADOOP-8358) | Config-related WARN for dfs.web.ugi can be avoided. | Trivial | conf | Harsh J | Harsh J | +| [HADOOP-8340](https://issues.apache.org/jira/browse/HADOOP-8340) | SNAPSHOT build versions should compare as less than their eventual final release | Minor | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-8335](https://issues.apache.org/jira/browse/HADOOP-8335) | Improve Configuration's address handling | Major | util | Daryn Sharp | Daryn Sharp | +| [HADOOP-8323](https://issues.apache.org/jira/browse/HADOOP-8323) | Revert HADOOP-7940 and improve javadocs and test for Text.clear() | Critical | io | Harsh J | Harsh J | +| [HADOOP-8286](https://issues.apache.org/jira/browse/HADOOP-8286) | Simplify getting a socket address from conf | Major | conf | Daryn Sharp | Daryn Sharp | +| [HADOOP-8278](https://issues.apache.org/jira/browse/HADOOP-8278) | Make sure components declare correct set of dependencies | Major | build | Tom White | Tom White | +| [HADOOP-8244](https://issues.apache.org/jira/browse/HADOOP-8244) | Improve comments on ByteBufferReadable.read | Major | . | Henry Robinson | Henry Robinson | +| [HADOOP-8242](https://issues.apache.org/jira/browse/HADOOP-8242) | AbstractDelegationTokenIdentifier: add getter methods for owner and realuser | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8240](https://issues.apache.org/jira/browse/HADOOP-8240) | Allow users to specify a checksum type on create() | Major | fs | Kihwal Lee | Kihwal Lee | +| [HADOOP-8239](https://issues.apache.org/jira/browse/HADOOP-8239) | Extend MD5MD5CRC32FileChecksum to show the actual checksum type being used | Major | fs | Kihwal Lee | Kihwal Lee | +| [HADOOP-8227](https://issues.apache.org/jira/browse/HADOOP-8227) | Allow RPC to limit ephemeral port range. | Blocker | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8224](https://issues.apache.org/jira/browse/HADOOP-8224) | Don't hardcode hdfs.audit.logger in the scripts | Major | conf | Eli Collins | Tomohiko Kinebuchi | +| [HADOOP-8075](https://issues.apache.org/jira/browse/HADOOP-8075) | Lower native-hadoop library log from info to debug | Major | native | Eli Collins | Hızır Sefa Ä°rken | +| [HADOOP-7510](https://issues.apache.org/jira/browse/HADOOP-7510) | Tokens should use original hostname provided instead of ip | Major | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-6802](https://issues.apache.org/jira/browse/HADOOP-6802) | Remove FS\_CLIENT\_BUFFER\_DIR\_KEY = "fs.client.buffer.dir" from CommonConfigurationKeys.java (not used, deprecated) | Major | conf, fs | Erik Steffl | Sho Shimauchi | +| [HADOOP-3450](https://issues.apache.org/jira/browse/HADOOP-3450) | Add tests to Local Directory Allocator for asserting their URI-returning capability | Minor | fs | Ari Rabkin | Sho Shimauchi | +| [HDFS-3907](https://issues.apache.org/jira/browse/HDFS-3907) | Allow multiple users for local block readers | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3887](https://issues.apache.org/jira/browse/HDFS-3887) | Remove redundant chooseTarget methods in BlockPlacementPolicy.java | Trivial | namenode | Jing Zhao | Jing Zhao | +| [HDFS-3871](https://issues.apache.org/jira/browse/HDFS-3871) | Change NameNodeProxies to use HADOOP-8748 | Minor | hdfs-client | Arun C Murthy | Arun C Murthy | +| [HDFS-3866](https://issues.apache.org/jira/browse/HDFS-3866) | HttpFS POM should have property where to download tomcat from | Minor | build | Ryan Hennig | Plamen Jeliazkov | +| [HDFS-3844](https://issues.apache.org/jira/browse/HDFS-3844) | Add @Override where necessary and remove unnecessary {@inheritdoc} and imports | Trivial | . | Jing Zhao | Jing Zhao | +| [HDFS-3819](https://issues.apache.org/jira/browse/HDFS-3819) | Should check whether invalidate work percentage default value is not greater than 1.0f | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-3802](https://issues.apache.org/jira/browse/HDFS-3802) | StartupOption.name in HdfsServerConstants should be final | Trivial | . | Jing Zhao | Jing Zhao | +| [HDFS-3796](https://issues.apache.org/jira/browse/HDFS-3796) | Speed up edit log tests by avoiding fsync() | Minor | test | Todd Lipcon | Todd Lipcon | +| [HDFS-3765](https://issues.apache.org/jira/browse/HDFS-3765) | Namenode INITIALIZESHAREDEDITS should be able to initialize all shared storages | Major | ha | Vinayakumar B | Vinayakumar B | +| [HDFS-3723](https://issues.apache.org/jira/browse/HDFS-3723) | All commands should support meaningful --help | Major | scripts, tools | E. Sammer | Jing Zhao | +| [HDFS-3711](https://issues.apache.org/jira/browse/HDFS-3711) | Manually convert remaining tests to JUnit4 | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-3697](https://issues.apache.org/jira/browse/HDFS-3697) | Enable fadvise readahead by default | Minor | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-3672](https://issues.apache.org/jira/browse/HDFS-3672) | Expose disk-location information for blocks to enable better scheduling | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-3666](https://issues.apache.org/jira/browse/HDFS-3666) | Plumb more exception messages to terminate | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3663](https://issues.apache.org/jira/browse/HDFS-3663) | MiniDFSCluster should capture the code path that led to the first ExitException | Major | test | Eli Collins | Eli Collins | +| [HDFS-3659](https://issues.apache.org/jira/browse/HDFS-3659) | Add missing @Override to methods across the hadoop-hdfs project | Minor | documentation | Brandon Li | Brandon Li | +| [HDFS-3650](https://issues.apache.org/jira/browse/HDFS-3650) | Use MutableQuantiles to provide latency histograms for various operations | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-3641](https://issues.apache.org/jira/browse/HDFS-3641) | Move server Util time methods to common and use now instead of System#currentTimeMillis | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3613](https://issues.apache.org/jira/browse/HDFS-3613) | GSet prints some INFO level values, which aren't really very useful to all | Trivial | namenode | Harsh J | Andrew Wang | +| [HDFS-3612](https://issues.apache.org/jira/browse/HDFS-3612) | Single namenode image directory config warning can be improved | Trivial | namenode | Harsh J | Andy Isaacson | +| [HDFS-3610](https://issues.apache.org/jira/browse/HDFS-3610) | fuse\_dfs: Provide a way to use the default (configured) NN URI | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3607](https://issues.apache.org/jira/browse/HDFS-3607) | log a message when fuse\_dfs is not built | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3604](https://issues.apache.org/jira/browse/HDFS-3604) | Add dfs.webhdfs.enabled to hdfs-default.xml | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3583](https://issues.apache.org/jira/browse/HDFS-3583) | Convert remaining tests to Junit4 | Major | test | Eli Collins | Andrew Wang | +| [HDFS-3582](https://issues.apache.org/jira/browse/HDFS-3582) | Hook daemon process exit for testing | Minor | test | Eli Collins | Eli Collins | +| [HDFS-3568](https://issues.apache.org/jira/browse/HDFS-3568) | fuse\_dfs: add support for security | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3537](https://issues.apache.org/jira/browse/HDFS-3537) | Move libhdfs and fuse-dfs source to native subdirectories | Minor | fuse-dfs, libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3520](https://issues.apache.org/jira/browse/HDFS-3520) | Add transfer rate logging to TransferFsImage | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-3516](https://issues.apache.org/jira/browse/HDFS-3516) | Check content-type in WebHdfsFileSystem | Major | hdfs-client, webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3514](https://issues.apache.org/jira/browse/HDFS-3514) | Add missing TestParallelLocalRead | Major | test | Henry Robinson | Henry Robinson | +| [HDFS-3513](https://issues.apache.org/jira/browse/HDFS-3513) | HttpFS should cache filesystems | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3504](https://issues.apache.org/jira/browse/HDFS-3504) | Configurable retry in DFSClient | Major | hdfs-client | Siddharth Seth | Tsz Wo Nicholas Sze | +| [HDFS-3481](https://issues.apache.org/jira/browse/HDFS-3481) | Refactor HttpFS handling of JAX-RS query string parameters | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3475](https://issues.apache.org/jira/browse/HDFS-3475) | Make the replication and invalidation rates configurable | Trivial | . | Harsh J | Harsh J | +| [HDFS-3454](https://issues.apache.org/jira/browse/HDFS-3454) | Balancer unconditionally logs InterruptedException at INFO level on shutdown if security is enabled | Minor | balancer & mover | Eli Collins | Eli Collins | +| [HDFS-3438](https://issues.apache.org/jira/browse/HDFS-3438) | BootstrapStandby should not require a rollEdits on active node | Major | ha | Todd Lipcon | Todd Lipcon | +| [HDFS-3419](https://issues.apache.org/jira/browse/HDFS-3419) | Cleanup LocatedBlock | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3417](https://issues.apache.org/jira/browse/HDFS-3417) | Rename BalancerDatanode#getName to getDisplayName to be consistent with Datanode | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-3416](https://issues.apache.org/jira/browse/HDFS-3416) | Cleanup DatanodeID and DatanodeRegistration constructors used by testing | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-3404](https://issues.apache.org/jira/browse/HDFS-3404) | Make putImage in GetImageServlet infer remote address to fetch from request | Major | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3401](https://issues.apache.org/jira/browse/HDFS-3401) | Cleanup DatanodeDescriptor creation in the tests | Major | datanode, test | Eli Collins | Eli Collins | +| [HDFS-3400](https://issues.apache.org/jira/browse/HDFS-3400) | DNs should be able start with jsvc even if security is disabled | Major | datanode, scripts | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3394](https://issues.apache.org/jira/browse/HDFS-3394) | Do not use generic in INodeFile.getLastBlock() | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3390](https://issues.apache.org/jira/browse/HDFS-3390) | DFSAdmin should print full stack traces of errors when DEBUG logging is enabled | Minor | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3372](https://issues.apache.org/jira/browse/HDFS-3372) | offlineEditsViewer should be able to read a binary edits file with recovery mode | Minor | tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3343](https://issues.apache.org/jira/browse/HDFS-3343) | Improve metrics for DN read latency | Major | datanode | Todd Lipcon | Andrew Wang | +| [HDFS-3341](https://issues.apache.org/jira/browse/HDFS-3341) | Change minimum RPC versions to 2.0.0-SNAPSHOT instead of 2.0.0 | Minor | . | Todd Lipcon | Todd Lipcon | +| [HDFS-3335](https://issues.apache.org/jira/browse/HDFS-3335) | check for edit log corruption at the end of the log | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3276](https://issues.apache.org/jira/browse/HDFS-3276) | initializeSharedEdits should have a -nonInteractive flag | Minor | ha, namenode | Vinithra Varadharajan | Todd Lipcon | +| [HDFS-3230](https://issues.apache.org/jira/browse/HDFS-3230) | Cleanup DatanodeID creation in the tests | Minor | test | Eli Collins | Eli Collins | +| [HDFS-3227](https://issues.apache.org/jira/browse/HDFS-3227) | Mavenise libhdfs tests | Major | libhdfs | Henry Robinson | | +| [HDFS-3170](https://issues.apache.org/jira/browse/HDFS-3170) | Add more useful metrics for write latency | Major | datanode | Todd Lipcon | Matthew Jacobs | +| [HDFS-3134](https://issues.apache.org/jira/browse/HDFS-3134) | Harden edit log loader against malformed or malicious input | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3110](https://issues.apache.org/jira/browse/HDFS-3110) | libhdfs implementation of direct read API | Major | libhdfs, performance | Henry Robinson | Henry Robinson | +| [HDFS-3040](https://issues.apache.org/jira/browse/HDFS-3040) | TestMulitipleNNDataBlockScanner is misspelled | Trivial | test | Aaron T. Myers | madhukara phatak | +| [HDFS-3002](https://issues.apache.org/jira/browse/HDFS-3002) | TestNameNodeMetrics need not wait for metrics update with new metrics framework | Trivial | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2988](https://issues.apache.org/jira/browse/HDFS-2988) | Improve error message when storage directory lock fails | Minor | namenode | Todd Lipcon | Miomir Boljanovic | +| [HDFS-2885](https://issues.apache.org/jira/browse/HDFS-2885) | Remove "federation" from the nameservice config options | Major | namenode | Eli Collins | Tsz Wo Nicholas Sze | +| [HDFS-2834](https://issues.apache.org/jira/browse/HDFS-2834) | ByteBuffer-based read API for DFSInputStream | Major | hdfs-client, performance | Henry Robinson | Henry Robinson | +| [HDFS-2652](https://issues.apache.org/jira/browse/HDFS-2652) | Port token service changes from 205 | Major | . | Daryn Sharp | Daryn Sharp | +| [HDFS-2421](https://issues.apache.org/jira/browse/HDFS-2421) | Improve the concurrency of SerialNumberMap in NameNode | Major | namenode | Hairong Kuang | Jing Zhao | +| [HDFS-2391](https://issues.apache.org/jira/browse/HDFS-2391) | Newly set BalancerBandwidth value is not displayed anywhere | Major | balancer & mover | Rajit Saha | Harsh J | +| [HDFS-1013](https://issues.apache.org/jira/browse/HDFS-1013) | Miscellaneous improvements to HTML markup for web UIs | Minor | . | Todd Lipcon | Eugene Koontz | +| [HDFS-799](https://issues.apache.org/jira/browse/HDFS-799) | libhdfs must call DetachCurrentThread when a thread is destroyed | Major | . | Christian Kunz | Colin Patrick McCabe | +| [MAPREDUCE-4638](https://issues.apache.org/jira/browse/MAPREDUCE-4638) | MR AppMaster shouldn't rely on YARN\_APPLICATION\_CLASSPATH providing MR jars | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4614](https://issues.apache.org/jira/browse/MAPREDUCE-4614) | Simplify debugging a job's tokens | Major | client, task | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4511](https://issues.apache.org/jira/browse/MAPREDUCE-4511) | Add IFile readahead | Major | mrv1, mrv2, performance | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4422](https://issues.apache.org/jira/browse/MAPREDUCE-4422) | YARN\_APPLICATION\_CLASSPATH needs a documented default value in YarnConfiguration | Major | nodemanager | Arun C Murthy | Ahmed Radwan | +| [MAPREDUCE-4408](https://issues.apache.org/jira/browse/MAPREDUCE-4408) | allow jobs to set a JAR that is in the distributed cached | Major | mrv1, mrv2 | Alejandro Abdelnur | Robert Kanter | +| [MAPREDUCE-4375](https://issues.apache.org/jira/browse/MAPREDUCE-4375) | Show Configuration Tracability in MR UI | Major | applicationmaster | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4301](https://issues.apache.org/jira/browse/MAPREDUCE-4301) | Dedupe some strings in MRAM for memory savings | Major | applicationmaster | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4283](https://issues.apache.org/jira/browse/MAPREDUCE-4283) | Display tail of aggregated logs by default | Major | jobhistoryserver, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4274](https://issues.apache.org/jira/browse/MAPREDUCE-4274) | MapOutputBuffer should use native byte order for kvmeta | Minor | performance, task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-4210](https://issues.apache.org/jira/browse/MAPREDUCE-4210) | Expose listener address for WebApp | Major | webapps | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4205](https://issues.apache.org/jira/browse/MAPREDUCE-4205) | retrofit all JVM shutdown hooks to use ShutdownHookManager | Major | mrv2 | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4190](https://issues.apache.org/jira/browse/MAPREDUCE-4190) | Improve web UI for task attempts userlog link | Major | mrv2, webapps | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4157](https://issues.apache.org/jira/browse/MAPREDUCE-4157) | ResourceManager should not kill apps that are well behaved | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4151](https://issues.apache.org/jira/browse/MAPREDUCE-4151) | RM scheduler web page should filter apps to those that are relevant to scheduling | Major | mrv2, webapps | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4146](https://issues.apache.org/jira/browse/MAPREDUCE-4146) | Support limits on task status string length and number of block locations in branch-2 | Major | . | Tom White | Ahmed Radwan | +| [MAPREDUCE-4079](https://issues.apache.org/jira/browse/MAPREDUCE-4079) | Allow MR AppMaster to limit ephemeral port range. | Blocker | mr-am, mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4059](https://issues.apache.org/jira/browse/MAPREDUCE-4059) | The history server should have a separate pluggable storage/query interface | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4017](https://issues.apache.org/jira/browse/MAPREDUCE-4017) | Add jobname to jobsummary log | Trivial | jobhistoryserver, jobtracker | Koji Noguchi | Thomas Graves | +| [MAPREDUCE-3921](https://issues.apache.org/jira/browse/MAPREDUCE-3921) | MR AM should act on the nodes liveliness information when nodes go up/down/unhealthy | Major | mr-am, mrv2 | Vinod Kumar Vavilapalli | Bikas Saha | +| [MAPREDUCE-3907](https://issues.apache.org/jira/browse/MAPREDUCE-3907) | Document entries mapred-default.xml for the jobhistory server. | Minor | documentation | Eugene Koontz | Eugene Koontz | +| [MAPREDUCE-3906](https://issues.apache.org/jira/browse/MAPREDUCE-3906) | Fix inconsistency in documentation regarding mapreduce.jobhistory.principal | Trivial | documentation | Eugene Koontz | Eugene Koontz | +| [MAPREDUCE-3871](https://issues.apache.org/jira/browse/MAPREDUCE-3871) | Allow symlinking in LocalJobRunner DistributedCache | Major | distributed-cache | Tom White | Tom White | +| [MAPREDUCE-3850](https://issues.apache.org/jira/browse/MAPREDUCE-3850) | Avoid redundant calls for tokens in TokenCache | Major | security | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3842](https://issues.apache.org/jira/browse/MAPREDUCE-3842) | stop webpages from automatic refreshing | Critical | mrv2, webapps | Alejandro Abdelnur | Thomas Graves | +| [MAPREDUCE-3659](https://issues.apache.org/jira/browse/MAPREDUCE-3659) | Host-based token support | Major | security | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3289](https://issues.apache.org/jira/browse/MAPREDUCE-3289) | Make use of fadvise in the NM's shuffle handler | Major | mrv2, nodemanager, performance | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-2786](https://issues.apache.org/jira/browse/MAPREDUCE-2786) | TestDFSIO should also test compression reading/writing from command-line. | Minor | benchmarks | Plamen Jeliazkov | Plamen Jeliazkov | +| [YARN-420](https://issues.apache.org/jira/browse/YARN-420) | Enable the RM to work with AM's that are not managed by it | Major | applications/unmanaged-AM-launcher | Bikas Saha | Bikas Saha | +| [YARN-419](https://issues.apache.org/jira/browse/YARN-419) | Add client side for Unmanaged-AMs | Major | applications/unmanaged-AM-launcher | Bikas Saha | Bikas Saha | +| [YARN-137](https://issues.apache.org/jira/browse/YARN-137) | Change the default scheduler to the CapacityScheduler | Major | scheduler | Siddharth Seth | Siddharth Seth | +| [YARN-80](https://issues.apache.org/jira/browse/YARN-80) | Support delay scheduling for node locality in MR2's capacity scheduler | Major | capacityscheduler | Todd Lipcon | Arun C Murthy | +| [YARN-10](https://issues.apache.org/jira/browse/YARN-10) | dist-shell shouldn't have a (test) dependency on hadoop-mapreduce-client-core | Major | . | Arun C Murthy | Hitesh Shah | +| [YARN-9](https://issues.apache.org/jira/browse/YARN-9) | Rename YARN\_HOME to HADOOP\_YARN\_HOME | Major | . | Arun C Murthy | Vinod Kumar Vavilapalli | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8801](https://issues.apache.org/jira/browse/HADOOP-8801) | ExitUtil#terminate should capture the exception stack trace | Major | . | Eli Collins | Eli Collins | +| [HADOOP-8781](https://issues.apache.org/jira/browse/HADOOP-8781) | hadoop-config.sh should add JAVA\_LIBRARY\_PATH to LD\_LIBRARY\_PATH | Major | scripts | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8775](https://issues.apache.org/jira/browse/HADOOP-8775) | MR2 distcp permits non-positive value to -bandwidth option which causes job never to complete | Major | . | Sandy Ryza | Sandy Ryza | +| [HADOOP-8770](https://issues.apache.org/jira/browse/HADOOP-8770) | NN should not RPC to self to find trash defaults (causes deadlock) | Blocker | trash | Todd Lipcon | Eli Collins | +| [HADOOP-8766](https://issues.apache.org/jira/browse/HADOOP-8766) | FileContextMainOperationsBaseTest should randomize the root dir | Major | test | Eli Collins | Colin Patrick McCabe | +| [HADOOP-8764](https://issues.apache.org/jira/browse/HADOOP-8764) | CMake: HADOOP-8737 broke ARM build | Major | build | Trevor Robinson | Trevor Robinson | +| [HADOOP-8749](https://issues.apache.org/jira/browse/HADOOP-8749) | HADOOP-8031 changed the way in which relative xincludes are handled in Configuration. | Major | conf | Ahmed Radwan | Ahmed Radwan | +| [HADOOP-8747](https://issues.apache.org/jira/browse/HADOOP-8747) | Syntax error on cmake version 2.6 patch 2 in JNIFlags.cmake | Major | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8737](https://issues.apache.org/jira/browse/HADOOP-8737) | cmake: always use JAVA\_HOME to find libjvm.so, jni.h, jni\_md.h | Minor | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8727](https://issues.apache.org/jira/browse/HADOOP-8727) | Gracefully deprecate dfs.umaskmode in 2.x onwards | Major | conf | Harsh J | Harsh J | +| [HADOOP-8726](https://issues.apache.org/jira/browse/HADOOP-8726) | The Secrets in Credentials are not available to MR tasks | Major | security | Benoy Antony | Daryn Sharp | +| [HADOOP-8725](https://issues.apache.org/jira/browse/HADOOP-8725) | MR is broken when security is off | Blocker | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-8722](https://issues.apache.org/jira/browse/HADOOP-8722) | Update BUILDING.txt with latest snappy info | Minor | documentation | Eli Collins | Colin Patrick McCabe | +| [HADOOP-8721](https://issues.apache.org/jira/browse/HADOOP-8721) | ZKFC should not retry 45 times when attempting a graceful fence during a failover | Critical | auto-failover, ha | suja s | Vinayakumar B | +| [HADOOP-8720](https://issues.apache.org/jira/browse/HADOOP-8720) | TestLocalFileSystem should use test root subdirectory | Trivial | test | Vlad Rozov | Vlad Rozov | +| [HADOOP-8709](https://issues.apache.org/jira/browse/HADOOP-8709) | globStatus changed behavior from 0.20/1.x | Critical | fs | Jason Lowe | Jason Lowe | +| [HADOOP-8703](https://issues.apache.org/jira/browse/HADOOP-8703) | distcpV2: turn CRC checking off for 0 byte size | Major | . | Dave Thompson | Dave Thompson | +| [HADOOP-8699](https://issues.apache.org/jira/browse/HADOOP-8699) | some common testcases create core-site.xml in test-classes making other testcases to fail | Critical | test | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8697](https://issues.apache.org/jira/browse/HADOOP-8697) | TestWritableName fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8695](https://issues.apache.org/jira/browse/HADOOP-8695) | TestPathData fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8693](https://issues.apache.org/jira/browse/HADOOP-8693) | TestSecurityUtil fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8692](https://issues.apache.org/jira/browse/HADOOP-8692) | TestLocalDirAllocator fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8686](https://issues.apache.org/jira/browse/HADOOP-8686) | Fix warnings in native code | Minor | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8660](https://issues.apache.org/jira/browse/HADOOP-8660) | TestPseudoAuthenticator failing with NPE | Major | . | Eli Collins | Alejandro Abdelnur | +| [HADOOP-8659](https://issues.apache.org/jira/browse/HADOOP-8659) | Native libraries must build with soft-float ABI for Oracle JVM on ARM | Major | native | Trevor Robinson | Colin Patrick McCabe | +| [HADOOP-8655](https://issues.apache.org/jira/browse/HADOOP-8655) | In TextInputFormat, while specifying textinputformat.record.delimiter the character/character sequences in data file similar to starting character/starting character sequence in delimiter were found missing in certain cases in the Map Output | Major | util | Arun A K | | +| [HADOOP-8654](https://issues.apache.org/jira/browse/HADOOP-8654) | TextInputFormat delimiter bug:- Input Text portion ends with & Delimiter starts with same char/char sequence | Major | util | Gelesh | | +| [HADOOP-8648](https://issues.apache.org/jira/browse/HADOOP-8648) | libhadoop: native CRC32 validation crashes when io.bytes.per.checksum=1 | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8637](https://issues.apache.org/jira/browse/HADOOP-8637) | FilterFileSystem#setWriteChecksum is broken | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8634](https://issues.apache.org/jira/browse/HADOOP-8634) | Ensure FileSystem#close doesn't squawk for deleteOnExit paths | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8633](https://issues.apache.org/jira/browse/HADOOP-8633) | Interrupted FsShell copies may leave tmp files | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8632](https://issues.apache.org/jira/browse/HADOOP-8632) | Configuration leaking class-loaders | Major | conf | Costin Leau | Costin Leau | +| [HADOOP-8627](https://issues.apache.org/jira/browse/HADOOP-8627) | FS deleteOnExit may delete the wrong path | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8626](https://issues.apache.org/jira/browse/HADOOP-8626) | Typo in default setting for hadoop.security.group.mapping.ldap.search.filter.user | Major | security | Jonathan Natkins | Jonathan Natkins | +| [HADOOP-8614](https://issues.apache.org/jira/browse/HADOOP-8614) | IOUtils#skipFully hangs forever on EOF | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8613](https://issues.apache.org/jira/browse/HADOOP-8613) | AbstractDelegationTokenIdentifier#getUser() should set token auth type | Critical | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-8611](https://issues.apache.org/jira/browse/HADOOP-8611) | Allow fall-back to the shell-based implementation when JNI-based users-group mapping fails | Major | security | Kihwal Lee | Robert Parker | +| [HADOOP-8606](https://issues.apache.org/jira/browse/HADOOP-8606) | FileSystem.get may return the wrong filesystem | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8599](https://issues.apache.org/jira/browse/HADOOP-8599) | Non empty response from FileSystem.getFileBlockLocations when asking for data beyond the end of file | Major | fs | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-8587](https://issues.apache.org/jira/browse/HADOOP-8587) | HarFileSystem access of harMetaCache isn't threadsafe | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-8586](https://issues.apache.org/jira/browse/HADOOP-8586) | Fixup a bunch of SPNEGO misspellings | Major | . | Eli Collins | Eli Collins | +| [HADOOP-8585](https://issues.apache.org/jira/browse/HADOOP-8585) | Fix initialization circularity between UserGroupInformation and HadoopConfiguration | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8573](https://issues.apache.org/jira/browse/HADOOP-8573) | Configuration tries to read from an inputstream resource multiple times. | Major | conf | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8566](https://issues.apache.org/jira/browse/HADOOP-8566) | AvroReflectSerializer.accept(Class) throws a NPE if the class has no package (primitive types and arrays) | Major | io | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8563](https://issues.apache.org/jira/browse/HADOOP-8563) | don't package hadoop-pipes examples/bin | Minor | build | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8550](https://issues.apache.org/jira/browse/HADOOP-8550) | hadoop fs -touchz automatically created parent directories | Major | fs | Robert Joseph Evans | John George | +| [HADOOP-8547](https://issues.apache.org/jira/browse/HADOOP-8547) | Package hadoop-pipes examples/bin directory (again) | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8543](https://issues.apache.org/jira/browse/HADOOP-8543) | Invalid pom.xml files on 0.23 branch | Major | build | Radim Kolar | Radim Kolar | +| [HADOOP-8538](https://issues.apache.org/jira/browse/HADOOP-8538) | CMake builds fail on ARM | Major | native | Trevor Robinson | Trevor Robinson | +| [HADOOP-8537](https://issues.apache.org/jira/browse/HADOOP-8537) | Two TFile tests failing recently | Major | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-8512](https://issues.apache.org/jira/browse/HADOOP-8512) | AuthenticatedURL should reset the Token when the server returns other than OK on authentication | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8509](https://issues.apache.org/jira/browse/HADOOP-8509) | JarFinder duplicate entry: META-INF/MANIFEST.MF exception | Minor | util | Matteo Bertozzi | Alejandro Abdelnur | +| [HADOOP-8507](https://issues.apache.org/jira/browse/HADOOP-8507) | Avoid OOM while deserializing DelegationTokenIdentifer | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8501](https://issues.apache.org/jira/browse/HADOOP-8501) | Gridmix fails to compile on OpenJDK7u4 | Major | benchmarks | Radim Kolar | Radim Kolar | +| [HADOOP-8499](https://issues.apache.org/jira/browse/HADOOP-8499) | Lower min.user.id to 500 for the tests | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8495](https://issues.apache.org/jira/browse/HADOOP-8495) | Update Netty to avoid leaking file descriptors during shuffle | Critical | build | Jason Lowe | Jason Lowe | +| [HADOOP-8488](https://issues.apache.org/jira/browse/HADOOP-8488) | test-patch.sh gives +1 even if the native build fails. | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8485](https://issues.apache.org/jira/browse/HADOOP-8485) | Don't hardcode "Apache Hadoop 0.23" in the docs | Minor | documentation | Eli Collins | Eli Collins | +| [HADOOP-8481](https://issues.apache.org/jira/browse/HADOOP-8481) | update BUILDING.txt to talk about cmake rather than autotools | Trivial | documentation | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8480](https://issues.apache.org/jira/browse/HADOOP-8480) | The native build should honor -DskipTests | Trivial | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8466](https://issues.apache.org/jira/browse/HADOOP-8466) | hadoop-client POM incorrectly excludes avro | Major | build | Bruno MahĂ© | Bruno MahĂ© | +| [HADOOP-8460](https://issues.apache.org/jira/browse/HADOOP-8460) | Document proper setting of HADOOP\_PID\_DIR and HADOOP\_SECURE\_DN\_PID\_DIR | Major | documentation | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8452](https://issues.apache.org/jira/browse/HADOOP-8452) | DN logs backtrace when running under jsvc and /jmx is loaded | Minor | . | Andy Isaacson | Andy Isaacson | +| [HADOOP-8450](https://issues.apache.org/jira/browse/HADOOP-8450) | Remove src/test/system | Trivial | test | Colin Patrick McCabe | Eli Collins | +| [HADOOP-8449](https://issues.apache.org/jira/browse/HADOOP-8449) | hadoop fs -text fails with compressed sequence files with the codec file extension | Minor | . | Joey Echeverria | Harsh J | +| [HADOOP-8444](https://issues.apache.org/jira/browse/HADOOP-8444) | Fix the tests FSMainOperationsBaseTest.java and F ileContextMainOperationsBaseTest.java to avoid potential test failure | Major | fs, test | Mariappan Asokan | madhukara phatak | +| [HADOOP-8438](https://issues.apache.org/jira/browse/HADOOP-8438) | hadoop-validate-setup.sh refers to examples jar file which doesn't exist | Major | . | Devaraj K | Devaraj K | +| [HADOOP-8433](https://issues.apache.org/jira/browse/HADOOP-8433) | Don't set HADOOP\_LOG\_DIR in hadoop-env.sh | Major | scripts | Brahma Reddy Battula | Brahma Reddy Battula | +| [HADOOP-8431](https://issues.apache.org/jira/browse/HADOOP-8431) | Running distcp wo args throws IllegalArgumentException | Major | . | Eli Collins | Sandy Ryza | +| [HADOOP-8423](https://issues.apache.org/jira/browse/HADOOP-8423) | MapFile.Reader.get() crashes jvm or throws EOFException on Snappy or LZO block-compressed data | Major | io | Jason B | Todd Lipcon | +| [HADOOP-8422](https://issues.apache.org/jira/browse/HADOOP-8422) | Deprecate FileSystem#getDefault\* and getServerDefault methods that don't take a Path argument | Minor | fs | Eli Collins | Eli Collins | +| [HADOOP-8408](https://issues.apache.org/jira/browse/HADOOP-8408) | MR doesn't work with a non-default ViewFS mount table and security enabled | Major | viewfs | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-8406](https://issues.apache.org/jira/browse/HADOOP-8406) | CompressionCodecFactory.CODEC\_PROVIDERS iteration is thread-unsafe | Major | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-8400](https://issues.apache.org/jira/browse/HADOOP-8400) | All commands warn "Kerberos krb5 configuration not found" when security is not enabled | Major | security | Eli Collins | Alejandro Abdelnur | +| [HADOOP-8393](https://issues.apache.org/jira/browse/HADOOP-8393) | hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMaster | Major | scripts | Patrick Hunt | Patrick Hunt | +| [HADOOP-8390](https://issues.apache.org/jira/browse/HADOOP-8390) | TestFileSystemCanonicalization fails with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8372](https://issues.apache.org/jira/browse/HADOOP-8372) | normalizeHostName() in NetUtils is not working properly in resolving a hostname start with numeric character | Major | io, util | Junping Du | Junping Du | +| [HADOOP-8370](https://issues.apache.org/jira/browse/HADOOP-8370) | Native build failure: javah: class file for org.apache.hadoop.classification.InterfaceAudience not found | Major | native | Trevor Robinson | Trevor Robinson | +| [HADOOP-8342](https://issues.apache.org/jira/browse/HADOOP-8342) | HDFS command fails with exception following merge of HADOOP-8325 | Major | fs | Randy Clayton | Alejandro Abdelnur | +| [HADOOP-8341](https://issues.apache.org/jira/browse/HADOOP-8341) | Fix or filter findbugs issues in hadoop-tools | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8334](https://issues.apache.org/jira/browse/HADOOP-8334) | HttpServer sometimes returns incorrect port | Major | . | Daryn Sharp | Daryn Sharp | +| [HADOOP-8330](https://issues.apache.org/jira/browse/HADOOP-8330) | TestSequenceFile.testCreateUsesFsArg() is broken | Minor | test | John George | John George | +| [HADOOP-8329](https://issues.apache.org/jira/browse/HADOOP-8329) | Build fails with Java 7 | Major | build | Kumar Ravi | Eli Collins | +| [HADOOP-8328](https://issues.apache.org/jira/browse/HADOOP-8328) | Duplicate FileSystem Statistics object for 'file' scheme | Major | fs | Tom White | Tom White | +| [HADOOP-8327](https://issues.apache.org/jira/browse/HADOOP-8327) | distcpv2 and distcpv1 jars should not coexist | Major | . | Dave Thompson | Dave Thompson | +| [HADOOP-8325](https://issues.apache.org/jira/browse/HADOOP-8325) | Add a ShutdownHookManager to be used by different components instead of the JVM shutdownhook | Critical | fs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8317](https://issues.apache.org/jira/browse/HADOOP-8317) | Update maven-assembly-plugin to 2.3 - fix build on FreeBSD | Major | build | Radim Kolar | | +| [HADOOP-8316](https://issues.apache.org/jira/browse/HADOOP-8316) | Audit logging should be disabled by default | Major | conf | Eli Collins | Eli Collins | +| [HADOOP-8305](https://issues.apache.org/jira/browse/HADOOP-8305) | distcp over viewfs is broken | Major | viewfs | John George | John George | +| [HADOOP-8288](https://issues.apache.org/jira/browse/HADOOP-8288) | Remove references of mapred.child.ulimit etc. since they are not being used any more | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-8287](https://issues.apache.org/jira/browse/HADOOP-8287) | etc/hadoop is missing hadoop-env.sh | Major | conf | Eli Collins | Eli Collins | +| [HADOOP-8268](https://issues.apache.org/jira/browse/HADOOP-8268) | A few pom.xml across Hadoop project may fail XML validation | Major | build | Radim Kolar | Radim Kolar | +| [HADOOP-8249](https://issues.apache.org/jira/browse/HADOOP-8249) | invalid hadoop-auth cookies should trigger authentication if info is avail before returning HTTP 401 | Major | security | bc Wong | Alejandro Abdelnur | +| [HADOOP-8225](https://issues.apache.org/jira/browse/HADOOP-8225) | DistCp fails when invoked by Oozie | Blocker | security | Mithun Radhakrishnan | Daryn Sharp | +| [HADOOP-8197](https://issues.apache.org/jira/browse/HADOOP-8197) | Configuration logs WARNs on every use of a deprecated key | Critical | conf | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8180](https://issues.apache.org/jira/browse/HADOOP-8180) | Remove hsqldb since its not needed from pom.xml | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-8179](https://issues.apache.org/jira/browse/HADOOP-8179) | risk of NPE in CopyCommands processArguments() | Minor | fs | Steve Loughran | Daryn Sharp | +| [HADOOP-8172](https://issues.apache.org/jira/browse/HADOOP-8172) | Configuration no longer sets all keys in a deprecated key list. | Critical | conf | Robert Joseph Evans | Anupam Seth | +| [HADOOP-8168](https://issues.apache.org/jira/browse/HADOOP-8168) | empty-string owners or groups causes {{MissingFormatWidthException}} in o.a.h.fs.shell.Ls.ProcessPath() | Major | fs | Eugene Koontz | Eugene Koontz | +| [HADOOP-8167](https://issues.apache.org/jira/browse/HADOOP-8167) | Configuration deprecation logic breaks backwards compatibility | Blocker | conf | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8144](https://issues.apache.org/jira/browse/HADOOP-8144) | pseudoSortByDistance in NetworkTopology doesn't work properly if no local node and first node is local rack node | Minor | io | Junping Du | Junping Du | +| [HADOOP-8129](https://issues.apache.org/jira/browse/HADOOP-8129) | ViewFileSystemTestSetup setupForViewFileSystem is erring when the user's home directory is somewhere other than /home (eg. /User) etc. | Major | fs, test | Ravi Prakash | Ahmed Radwan | +| [HADOOP-8110](https://issues.apache.org/jira/browse/HADOOP-8110) | TestViewFsTrash occasionally fails | Major | fs | Tsz Wo Nicholas Sze | Jason Lowe | +| [HADOOP-8104](https://issues.apache.org/jira/browse/HADOOP-8104) | Inconsistent Jackson versions | Major | . | Colin Patrick McCabe | Alejandro Abdelnur | +| [HADOOP-8088](https://issues.apache.org/jira/browse/HADOOP-8088) | User-group mapping cache incorrectly does negative caching on transient failures | Major | security | Kihwal Lee | Kihwal Lee | +| [HADOOP-8060](https://issues.apache.org/jira/browse/HADOOP-8060) | Add a capability to discover and set checksum types per file. | Major | fs, util | Kihwal Lee | Kihwal Lee | +| [HADOOP-8031](https://issues.apache.org/jira/browse/HADOOP-8031) | Configuration class fails to find embedded .jar resources; should use URL.openStream() | Major | conf | Elias Ross | Elias Ross | +| [HADOOP-8014](https://issues.apache.org/jira/browse/HADOOP-8014) | ViewFileSystem does not correctly implement getDefaultBlockSize, getDefaultReplication, getContentSummary | Major | fs | Daryn Sharp | John George | +| [HADOOP-8005](https://issues.apache.org/jira/browse/HADOOP-8005) | Multiple SLF4J binding message in .out file for all daemons | Major | scripts | Joe Crobak | Jason Lowe | +| [HADOOP-7967](https://issues.apache.org/jira/browse/HADOOP-7967) | Need generalized multi-token filesystem support | Critical | fs, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-7868](https://issues.apache.org/jira/browse/HADOOP-7868) | Hadoop native fails to compile when default linker option is -Wl,--as-needed | Major | native | James Page | Trevor Robinson | +| [HADOOP-7818](https://issues.apache.org/jira/browse/HADOOP-7818) | DiskChecker#checkDir should fail if the directory is not executable | Minor | util | Eli Collins | madhukara phatak | +| [HADOOP-7703](https://issues.apache.org/jira/browse/HADOOP-7703) | WebAppContext should also be stopped and cleared | Major | . | Devaraj K | Devaraj K | +| [HADOOP-6963](https://issues.apache.org/jira/browse/HADOOP-6963) | Fix FileUtil.getDU. It should not include the size of the directory or follow symbolic links | Critical | fs | Owen O'Malley | Ravi Prakash | +| [HADOOP-3886](https://issues.apache.org/jira/browse/HADOOP-3886) | Error in javadoc of Reporter, Mapper and Progressable | Minor | documentation | brien colwell | Jingguo Yao | +| [HDFS-3972](https://issues.apache.org/jira/browse/HDFS-3972) | Trash emptier fails in secure HA cluster | Critical | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3928](https://issues.apache.org/jira/browse/HDFS-3928) | MiniDFSCluster should reset the first ExitException on shutdown | Major | test | Eli Collins | Eli Collins | +| [HDFS-3902](https://issues.apache.org/jira/browse/HDFS-3902) | TestDatanodeBlockScanner#testBlockCorruptionPolicy is broken | Minor | . | Andy Isaacson | Andy Isaacson | +| [HDFS-3895](https://issues.apache.org/jira/browse/HDFS-3895) | hadoop-client must include commons-cli | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3890](https://issues.apache.org/jira/browse/HDFS-3890) | filecontext mkdirs doesn't apply umask as expected | Critical | . | Thomas Graves | Thomas Graves | +| [HDFS-3888](https://issues.apache.org/jira/browse/HDFS-3888) | BlockPlacementPolicyDefault code cleanup | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-3879](https://issues.apache.org/jira/browse/HDFS-3879) | Fix findbugs warning in TransferFsImage on branch-2 | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-3873](https://issues.apache.org/jira/browse/HDFS-3873) | Hftp assumes security is disabled if token fetch fails | Major | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3864](https://issues.apache.org/jira/browse/HDFS-3864) | NN does not update internal file mtime for OP\_CLOSE when reading from the edit log | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3861](https://issues.apache.org/jira/browse/HDFS-3861) | Deadlock in DFSClient | Blocker | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3860](https://issues.apache.org/jira/browse/HDFS-3860) | HeartbeatManager#Monitor may wrongly hold the writelock of namesystem | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-3856](https://issues.apache.org/jira/browse/HDFS-3856) | TestHDFSServerPorts failure is causing surefire fork failure | Blocker | test | Thomas Graves | Eli Collins | +| [HDFS-3853](https://issues.apache.org/jira/browse/HDFS-3853) | Port MiniDFSCluster enableManagedDfsDirsRedundancy option to branch-2 | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3852](https://issues.apache.org/jira/browse/HDFS-3852) | TestHftpDelegationToken is broken after HADOOP-8225 | Major | hdfs-client, security | Aaron T. Myers | Daryn Sharp | +| [HDFS-3849](https://issues.apache.org/jira/browse/HDFS-3849) | When re-loading the FSImage, we should clear the existing genStamp and leases. | Critical | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3841](https://issues.apache.org/jira/browse/HDFS-3841) | Port HDFS-3835 to branch-0.23 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-3837](https://issues.apache.org/jira/browse/HDFS-3837) | Fix DataNode.recoverBlock findbugs warning | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3835](https://issues.apache.org/jira/browse/HDFS-3835) | Long-lived 2NN cannot perform a checkpoint if security is enabled and the NN restarts with outstanding delegation tokens | Major | namenode, security | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3833](https://issues.apache.org/jira/browse/HDFS-3833) | TestDFSShell fails on Windows due to file concurrent read write | Major | test | Brandon Li | Brandon Li | +| [HDFS-3832](https://issues.apache.org/jira/browse/HDFS-3832) | Remove protocol methods related to DistributedUpgrade | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-3830](https://issues.apache.org/jira/browse/HDFS-3830) | test\_libhdfs\_threaded: use forceNewInstance | Minor | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3816](https://issues.apache.org/jira/browse/HDFS-3816) | Invalidate work percentage default value should be 0.32f instead of 32 | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-3808](https://issues.apache.org/jira/browse/HDFS-3808) | fuse\_dfs: postpone libhdfs intialization until after fork | Critical | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3803](https://issues.apache.org/jira/browse/HDFS-3803) | BlockPoolSliceScanner new work period notice is very chatty at INFO level | Minor | datanode | Andrew Purtell | | +| [HDFS-3794](https://issues.apache.org/jira/browse/HDFS-3794) | WebHDFS Open used with Offset returns the original (and incorrect) Content Length in the HTTP Header. | Major | webhdfs | Ravi Prakash | Ravi Prakash | +| [HDFS-3790](https://issues.apache.org/jira/browse/HDFS-3790) | test\_fuse\_dfs.c doesn't compile on centos 5 | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3788](https://issues.apache.org/jira/browse/HDFS-3788) | distcp can't copy large files using webhdfs due to missing Content-Length header | Critical | webhdfs | Eli Collins | Tsz Wo Nicholas Sze | +| [HDFS-3760](https://issues.apache.org/jira/browse/HDFS-3760) | primitiveCreate is a write, not a read | Minor | hdfs-client | Andy Isaacson | Andy Isaacson | +| [HDFS-3758](https://issues.apache.org/jira/browse/HDFS-3758) | TestFuseDFS test failing | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3756](https://issues.apache.org/jira/browse/HDFS-3756) | DelegationTokenFetcher creates 2 HTTP connections, the second one not properly configured | Critical | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3755](https://issues.apache.org/jira/browse/HDFS-3755) | Creating an already-open-for-write file with overwrite=true fails | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3754](https://issues.apache.org/jira/browse/HDFS-3754) | BlockSender doesn't shutdown ReadaheadPool threads | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-3738](https://issues.apache.org/jira/browse/HDFS-3738) | TestDFSClientRetries#testFailuresArePerOperation sets incorrect timeout config | Minor | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3733](https://issues.apache.org/jira/browse/HDFS-3733) | Audit logs should include WebHDFS access | Major | webhdfs | Andy Isaacson | Andy Isaacson | +| [HDFS-3732](https://issues.apache.org/jira/browse/HDFS-3732) | fuse\_dfs: incorrect configuration value checked for connection expiry timer period | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3731](https://issues.apache.org/jira/browse/HDFS-3731) | 2.0 release upgrade must handle blocks being written from 1.0 | Blocker | datanode | Suresh Srinivas | Kihwal Lee | +| [HDFS-3724](https://issues.apache.org/jira/browse/HDFS-3724) | add InterfaceAudience annotations to HttpFS classes and making inner enum static | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3721](https://issues.apache.org/jira/browse/HDFS-3721) | hsync support broke wire compatibility | Critical | datanode, hdfs-client | Todd Lipcon | Aaron T. Myers | +| [HDFS-3720](https://issues.apache.org/jira/browse/HDFS-3720) | hdfs.h must get packaged | Major | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3718](https://issues.apache.org/jira/browse/HDFS-3718) | Datanode won't shutdown because of runaway DataBlockScanner thread | Critical | datanode | Kihwal Lee | Kihwal Lee | +| [HDFS-3715](https://issues.apache.org/jira/browse/HDFS-3715) | Fix TestFileCreation#testFileCreationNamenodeRestart | Major | test | Eli Collins | Andrew Wang | +| [HDFS-3710](https://issues.apache.org/jira/browse/HDFS-3710) | libhdfs misuses O\_RDONLY/WRONLY/RDWR | Minor | libhdfs | Andy Isaacson | Andy Isaacson | +| [HDFS-3707](https://issues.apache.org/jira/browse/HDFS-3707) | TestFSInputChecker: improper use of skip | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3696](https://issues.apache.org/jira/browse/HDFS-3696) | Create files with WebHdfsFileSystem goes OOM when file size is big | Critical | webhdfs | Kihwal Lee | Tsz Wo Nicholas Sze | +| [HDFS-3690](https://issues.apache.org/jira/browse/HDFS-3690) | BlockPlacementPolicyDefault incorrectly casts LOG | Major | . | Eli Collins | Eli Collins | +| [HDFS-3688](https://issues.apache.org/jira/browse/HDFS-3688) | Namenode loses datanode hostname if datanode re-registers | Major | datanode | Jason Lowe | Jason Lowe | +| [HDFS-3683](https://issues.apache.org/jira/browse/HDFS-3683) | Edit log replay progress indicator shows \>100% complete | Minor | namenode | Todd Lipcon | Plamen Jeliazkov | +| [HDFS-3679](https://issues.apache.org/jira/browse/HDFS-3679) | fuse\_dfs notrash option sets usetrash | Minor | fuse-dfs | Conrad Meyer | Conrad Meyer | +| [HDFS-3673](https://issues.apache.org/jira/browse/HDFS-3673) | libhdfs: fix some compiler warnings | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3664](https://issues.apache.org/jira/browse/HDFS-3664) | BlockManager race when stopping active services | Major | test | Eli Collins | Colin Patrick McCabe | +| [HDFS-3658](https://issues.apache.org/jira/browse/HDFS-3658) | TestDFSClientRetries#testNamenodeRestart failed | Major | . | Eli Collins | Tsz Wo Nicholas Sze | +| [HDFS-3646](https://issues.apache.org/jira/browse/HDFS-3646) | LeaseRenewer can hold reference to inactive DFSClient instances forever | Critical | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3633](https://issues.apache.org/jira/browse/HDFS-3633) | libhdfs: hdfsDelete should pass JNI\_FALSE or JNI\_TRUE | Minor | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3629](https://issues.apache.org/jira/browse/HDFS-3629) | fix the typo in the error message about inconsistent storage layout version | Trivial | namenode | Brandon Li | Brandon Li | +| [HDFS-3622](https://issues.apache.org/jira/browse/HDFS-3622) | Backport HDFS-3541 to branch-0.23 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-3615](https://issues.apache.org/jira/browse/HDFS-3615) | Two BlockTokenSecretManager findbugs warnings | Major | security | Eli Collins | Aaron T. Myers | +| [HDFS-3611](https://issues.apache.org/jira/browse/HDFS-3611) | NameNode prints unnecessary WARNs about edit log normally skipping a few bytes | Trivial | namenode | Harsh J | Colin Patrick McCabe | +| [HDFS-3609](https://issues.apache.org/jira/browse/HDFS-3609) | libhdfs: don't force the URI to look like hdfs://hostname:port | Major | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3608](https://issues.apache.org/jira/browse/HDFS-3608) | fuse\_dfs: detect changes in UID ticket cache | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3605](https://issues.apache.org/jira/browse/HDFS-3605) | Block mistakenly marked corrupt during edit log catchup phase of failover | Major | ha, namenode | Brahma Reddy Battula | Todd Lipcon | +| [HDFS-3603](https://issues.apache.org/jira/browse/HDFS-3603) | Decouple TestHDFSTrash from TestTrash | Major | test | Jason Lowe | Jason Lowe | +| [HDFS-3597](https://issues.apache.org/jira/browse/HDFS-3597) | SNN can fail to start on upgrade | Minor | namenode | Andy Isaacson | Andy Isaacson | +| [HDFS-3591](https://issues.apache.org/jira/browse/HDFS-3591) | Backport HDFS-3357 to branch-0.23 | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HDFS-3581](https://issues.apache.org/jira/browse/HDFS-3581) | FSPermissionChecker#checkPermission sticky bit check missing range check | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-3580](https://issues.apache.org/jira/browse/HDFS-3580) | incompatible types; no instance(s) of type variable(s) V exist so that V conforms to boolean compiling HttpFSServer.java with OpenJDK | Minor | . | Andy Isaacson | Andy Isaacson | +| [HDFS-3579](https://issues.apache.org/jira/browse/HDFS-3579) | libhdfs: fix exception handling | Major | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3577](https://issues.apache.org/jira/browse/HDFS-3577) | WebHdfsFileSystem can not read files larger than 24KB | Blocker | webhdfs | Alejandro Abdelnur | Tsz Wo Nicholas Sze | +| [HDFS-3575](https://issues.apache.org/jira/browse/HDFS-3575) | HttpFS does not log Exception Stacktraces | Minor | . | Brock Noland | Brock Noland | +| [HDFS-3574](https://issues.apache.org/jira/browse/HDFS-3574) | Fix small race and do some cleanup in GetImageServlet | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3572](https://issues.apache.org/jira/browse/HDFS-3572) | Cleanup code which inits SPNEGO in HttpServer | Minor | namenode, security | Todd Lipcon | Todd Lipcon | +| [HDFS-3559](https://issues.apache.org/jira/browse/HDFS-3559) | DFSTestUtil: use Builder class to construct DFSTestUtil instances | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3555](https://issues.apache.org/jira/browse/HDFS-3555) | idle client socket triggers DN ERROR log (should be INFO or DEBUG) | Major | datanode, hdfs-client | Jeff Lord | Andy Isaacson | +| [HDFS-3551](https://issues.apache.org/jira/browse/HDFS-3551) | WebHDFS CREATE does not use client location for redirection | Major | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3548](https://issues.apache.org/jira/browse/HDFS-3548) | NamenodeFsck.copyBlock fails to create a Block Reader | Critical | namenode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-3541](https://issues.apache.org/jira/browse/HDFS-3541) | Deadlock between recovery, xceiver and packet responder | Major | datanode | suja s | Vinayakumar B | +| [HDFS-3539](https://issues.apache.org/jira/browse/HDFS-3539) | libhdfs code cleanups | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3531](https://issues.apache.org/jira/browse/HDFS-3531) | EditLogFileOutputStream#preallocate should check for incomplete writes | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3524](https://issues.apache.org/jira/browse/HDFS-3524) | TestFileLengthOnClusterRestart failed due to error message change | Major | test | Eli Collins | Brandon Li | +| [HDFS-3518](https://issues.apache.org/jira/browse/HDFS-3518) | Provide API to check HDFS operational state | Major | hdfs-client | Bikas Saha | Tsz Wo Nicholas Sze | +| [HDFS-3517](https://issues.apache.org/jira/browse/HDFS-3517) | TestStartup should bind ephemeral ports | Minor | test | Eli Collins | Eli Collins | +| [HDFS-3505](https://issues.apache.org/jira/browse/HDFS-3505) | DirectoryScanner does not join all threads in shutdown | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3501](https://issues.apache.org/jira/browse/HDFS-3501) | Checkpointing with security enabled will stop working after ticket lifetime expires | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3491](https://issues.apache.org/jira/browse/HDFS-3491) | HttpFs does not set permissions correctly | Major | . | Romain Rigaux | Alejandro Abdelnur | +| [HDFS-3490](https://issues.apache.org/jira/browse/HDFS-3490) | DN WebHDFS methods throw NPE if Namenode RPC address param not specified | Minor | webhdfs | Todd Lipcon | Tsz Wo Nicholas Sze | +| [HDFS-3487](https://issues.apache.org/jira/browse/HDFS-3487) | offlineimageviewer should give byte offset information when it encounters an exception | Minor | tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3486](https://issues.apache.org/jira/browse/HDFS-3486) | offlineimageviewer can't read fsimage files that contain persistent delegation tokens | Minor | security, tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3485](https://issues.apache.org/jira/browse/HDFS-3485) | DataTransferThrottler will over-throttle when currentTimeMillis jumps | Minor | . | Andy Isaacson | Andy Isaacson | +| [HDFS-3484](https://issues.apache.org/jira/browse/HDFS-3484) | hdfs fsck doesn't work if NN HTTP address is set to 0.0.0.0 even if NN RPC address is configured | Minor | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3480](https://issues.apache.org/jira/browse/HDFS-3480) | Multiple SLF4J binding warning | Major | build | Eli Collins | Vinayakumar B | +| [HDFS-3469](https://issues.apache.org/jira/browse/HDFS-3469) | start-dfs.sh will start zkfc, but stop-dfs.sh will not stop zkfc similarly. | Minor | auto-failover | Vinayakumar B | Vinayakumar B | +| [HDFS-3466](https://issues.apache.org/jira/browse/HDFS-3466) | The SPNEGO filter for the NameNode should come out of the web keytab file | Major | namenode, security | Owen O'Malley | Owen O'Malley | +| [HDFS-3460](https://issues.apache.org/jira/browse/HDFS-3460) | HttpFS proxyuser validation with Kerberos ON uses full principal name | Critical | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3444](https://issues.apache.org/jira/browse/HDFS-3444) | hdfs groups command doesn't work with security enabled | Major | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3442](https://issues.apache.org/jira/browse/HDFS-3442) | Incorrect count for Missing Replicas in FSCK report | Minor | . | suja s | Andrew Wang | +| [HDFS-3440](https://issues.apache.org/jira/browse/HDFS-3440) | should more effectively limit stream memory consumption when reading corrupt edit logs | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3436](https://issues.apache.org/jira/browse/HDFS-3436) | adding new datanode to existing pipeline fails in case of Append/Recovery | Major | datanode | Brahma Reddy Battula | Vinayakumar B | +| [HDFS-3433](https://issues.apache.org/jira/browse/HDFS-3433) | GetImageServlet should allow administrative requestors when security is enabled | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3428](https://issues.apache.org/jira/browse/HDFS-3428) | move DelegationTokenRenewer to common | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3422](https://issues.apache.org/jira/browse/HDFS-3422) | TestStandbyIsHot timeouts too aggressive | Minor | test | Todd Lipcon | Todd Lipcon | +| [HDFS-3415](https://issues.apache.org/jira/browse/HDFS-3415) | During NameNode starting up, it may pick wrong storage directory inspector when the layout versions of the storage directories are different | Major | namenode | Brahma Reddy Battula | Brandon Li | +| [HDFS-3414](https://issues.apache.org/jira/browse/HDFS-3414) | Balancer does not find NameNode if rpc-address or servicerpc-address are not set in client configs | Minor | balancer & mover | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3413](https://issues.apache.org/jira/browse/HDFS-3413) | TestFailureToReadEdits timing out | Critical | ha, test | Todd Lipcon | Aaron T. Myers | +| [HDFS-3398](https://issues.apache.org/jira/browse/HDFS-3398) | Client will not retry when primaryDN is down once it's just got pipeline | Minor | hdfs-client | Brahma Reddy Battula | amith | +| [HDFS-3391](https://issues.apache.org/jira/browse/HDFS-3391) | TestPipelinesFailover#testLeaseRecoveryAfterFailover is failing | Critical | . | Arun C Murthy | Todd Lipcon | +| [HDFS-3385](https://issues.apache.org/jira/browse/HDFS-3385) | ClassCastException when trying to append a file | Major | namenode | Brahma Reddy Battula | Tsz Wo Nicholas Sze | +| [HDFS-3368](https://issues.apache.org/jira/browse/HDFS-3368) | Missing blocks due to bad DataNodes coming up and down. | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-3359](https://issues.apache.org/jira/browse/HDFS-3359) | DFSClient.close should close cached sockets | Critical | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-3334](https://issues.apache.org/jira/browse/HDFS-3334) | ByteRangeInputStream leaks streams | Major | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3331](https://issues.apache.org/jira/browse/HDFS-3331) | setBalancerBandwidth do not checkSuperuserPrivilege | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3321](https://issues.apache.org/jira/browse/HDFS-3321) | Error message for insufficient data nodes to come out of safemode is wrong. | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-3312](https://issues.apache.org/jira/browse/HDFS-3312) | Hftp selects wrong token service | Blocker | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3308](https://issues.apache.org/jira/browse/HDFS-3308) | hftp/webhdfs can't get tokens if authority has no port | Critical | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-3306](https://issues.apache.org/jira/browse/HDFS-3306) | fuse\_dfs: don't lock release operations | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3275](https://issues.apache.org/jira/browse/HDFS-3275) | Format command overwrites contents of non-empty shared edits dir if name dirs are empty without any prompting | Major | ha, namenode | Vinithra Varadharajan | amith | +| [HDFS-3266](https://issues.apache.org/jira/browse/HDFS-3266) | DFSTestUtil#waitCorruptReplicas doesn't sleep between checks | Minor | . | Aaron T. Myers | madhukara phatak | +| [HDFS-3243](https://issues.apache.org/jira/browse/HDFS-3243) | TestParallelRead timing out on jenkins | Major | hdfs-client, test | Todd Lipcon | Henry Robinson | +| [HDFS-3235](https://issues.apache.org/jira/browse/HDFS-3235) | MiniDFSClusterManager doesn't correctly support -format option | Minor | . | Henry Robinson | Henry Robinson | +| [HDFS-3194](https://issues.apache.org/jira/browse/HDFS-3194) | DataNode block scanner is running too frequently | Major | datanode | suja s | Andy Isaacson | +| [HDFS-3177](https://issues.apache.org/jira/browse/HDFS-3177) | Allow DFSClient to find out and use the CRC type being used for a file. | Major | datanode, hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3176](https://issues.apache.org/jira/browse/HDFS-3176) | JsonUtil should not parse the MD5MD5CRC32FileChecksum bytes on its own. | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-3166](https://issues.apache.org/jira/browse/HDFS-3166) | Hftp connections do not have a timeout | Critical | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-3157](https://issues.apache.org/jira/browse/HDFS-3157) | Error in deleting block is keep on coming from DN even after the block report and directory scanning has happened | Major | namenode | J.Andreina | Ashish Singhi | +| [HDFS-3136](https://issues.apache.org/jira/browse/HDFS-3136) | Multiple SLF4J binding warning | Major | build | Jason Lowe | Jason Lowe | +| [HDFS-3067](https://issues.apache.org/jira/browse/HDFS-3067) | NPE in DFSInputStream.readBuffer if read is repeated on corrupted block | Major | hdfs-client | Henry Robinson | Henry Robinson | +| [HDFS-3054](https://issues.apache.org/jira/browse/HDFS-3054) | distcp -skipcrccheck has no effect | Major | tools | patrick white | Colin Patrick McCabe | +| [HDFS-3048](https://issues.apache.org/jira/browse/HDFS-3048) | Small race in BlockManager#close | Major | namenode | Eli Collins | Andy Isaacson | +| [HDFS-3037](https://issues.apache.org/jira/browse/HDFS-3037) | TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy | Minor | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-3031](https://issues.apache.org/jira/browse/HDFS-3031) | HA: Fix complete() and getAdditionalBlock() RPCs to be idempotent. | Major | ha | Stephen Chu | Todd Lipcon | +| [HDFS-2982](https://issues.apache.org/jira/browse/HDFS-2982) | Startup performance suffers when there are many edit log segments | Critical | namenode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-2966](https://issues.apache.org/jira/browse/HDFS-2966) | TestNameNodeMetrics tests can fail under load | Minor | test | Steve Loughran | Steve Loughran | +| [HDFS-2963](https://issues.apache.org/jira/browse/HDFS-2963) | Console Output is confusing while executing metasave (dfsadmin command) | Minor | . | J.Andreina | Andrew Wang | +| [HDFS-2914](https://issues.apache.org/jira/browse/HDFS-2914) | HA: Standby should not enter safemode when resources are low | Major | ha, namenode | Hari Mankude | Vinayakumar B | +| [HDFS-2800](https://issues.apache.org/jira/browse/HDFS-2800) | HA: TestStandbyCheckpoints.testCheckpointCancellation is racy | Major | ha, test | Aaron T. Myers | Todd Lipcon | +| [HDFS-2797](https://issues.apache.org/jira/browse/HDFS-2797) | Fix misuses of InputStream#skip in the edit log code | Major | ha, namenode | Aaron T. Myers | Colin Patrick McCabe | +| [HDFS-2759](https://issues.apache.org/jira/browse/HDFS-2759) | Pre-allocate HDFS edit log files after writing version number | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2757](https://issues.apache.org/jira/browse/HDFS-2757) | Cannot read a local block that's being written to when using the local read short circuit | Major | . | Jean-Daniel Cryans | Jean-Daniel Cryans | +| [HDFS-2619](https://issues.apache.org/jira/browse/HDFS-2619) | Remove my personal email address from the libhdfs build file. | Major | build | Owen O'Malley | Owen O'Malley | +| [HDFS-2285](https://issues.apache.org/jira/browse/HDFS-2285) | BackupNode should reject requests trying to modify namespace | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-2025](https://issues.apache.org/jira/browse/HDFS-2025) | Go Back to File View link is not working in tail.jsp | Minor | datanode | sravankorumilli | Ashish Singhi | +| [HDFS-1490](https://issues.apache.org/jira/browse/HDFS-1490) | TransferFSImage should timeout | Minor | namenode | Dmytro Molkov | Vinayakumar B | +| [HDFS-1249](https://issues.apache.org/jira/browse/HDFS-1249) | with fuse-dfs, chown which only has owner (or only group) argument fails with Input/output error. | Minor | fuse-dfs | matsusaka kentaro | Colin Patrick McCabe | +| [HDFS-1153](https://issues.apache.org/jira/browse/HDFS-1153) | dfsnodelist.jsp should handle invalid input parameters | Minor | datanode | Ravi Phulari | Ravi Phulari | +| [HDFS-766](https://issues.apache.org/jira/browse/HDFS-766) | Error message not clear for set space quota out of boundary values. | Minor | . | Ravi Phulari | Jon Zuanich | +| [HDFS-711](https://issues.apache.org/jira/browse/HDFS-711) | hdfsUtime does not handle atime = 0 or mtime = 0 correctly | Major | documentation | freestyler | Colin Patrick McCabe | +| [HDFS-470](https://issues.apache.org/jira/browse/HDFS-470) | libhdfs should handle 0-length reads from FSInputStream correctly | Minor | . | Pete Wyckoff | Colin Patrick McCabe | +| [MAPREDUCE-4691](https://issues.apache.org/jira/browse/MAPREDUCE-4691) | Historyserver can report "Unknown job" after RM says job has completed | Critical | jobhistoryserver, mrv2 | Jason Lowe | Robert Joseph Evans | +| [MAPREDUCE-4689](https://issues.apache.org/jira/browse/MAPREDUCE-4689) | JobClient.getMapTaskReports on failed job results in NPE | Major | client | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4649](https://issues.apache.org/jira/browse/MAPREDUCE-4649) | mr-jobhistory-daemon.sh needs to be updated post YARN-1 | Major | jobhistoryserver | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4647](https://issues.apache.org/jira/browse/MAPREDUCE-4647) | We should only unjar jobjar if there is a lib directory in it. | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4646](https://issues.apache.org/jira/browse/MAPREDUCE-4646) | client does not receive job diagnostics for failed jobs | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4642](https://issues.apache.org/jira/browse/MAPREDUCE-4642) | MiniMRClientClusterFactory should not use job.setJar() | Major | test | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4641](https://issues.apache.org/jira/browse/MAPREDUCE-4641) | Exception in commitJob marks job as successful in job history | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4635](https://issues.apache.org/jira/browse/MAPREDUCE-4635) | MR side of YARN-83. Changing package of YarnClient | Major | . | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4633](https://issues.apache.org/jira/browse/MAPREDUCE-4633) | history server doesn't set permissions on all subdirs | Critical | jobhistoryserver | Thomas Graves | oss.wakayama | +| [MAPREDUCE-4612](https://issues.apache.org/jira/browse/MAPREDUCE-4612) | job summary file permissions not set when its created | Critical | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4611](https://issues.apache.org/jira/browse/MAPREDUCE-4611) | MR AM dies badly when Node is decomissioned | Critical | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4610](https://issues.apache.org/jira/browse/MAPREDUCE-4610) | Support deprecated mapreduce.job.counters.limit property in MR2 | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4608](https://issues.apache.org/jira/browse/MAPREDUCE-4608) | hadoop-mapreduce-client is missing some dependencies | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4604](https://issues.apache.org/jira/browse/MAPREDUCE-4604) | In mapred-default, mapreduce.map.maxattempts & mapreduce.reduce.maxattempts defaults are set to 4 as well as mapreduce.job.maxtaskfailures.per.tracker. | Critical | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4600](https://issues.apache.org/jira/browse/MAPREDUCE-4600) | TestTokenCache.java from MRV1 no longer compiles | Critical | . | Robert Joseph Evans | Daryn Sharp | +| [MAPREDUCE-4580](https://issues.apache.org/jira/browse/MAPREDUCE-4580) | Change MapReduce to use the yarn-client module | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4579](https://issues.apache.org/jira/browse/MAPREDUCE-4579) | TestTaskAttempt fails jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4577](https://issues.apache.org/jira/browse/MAPREDUCE-4577) | HDFS-3672 broke TestCombineFileInputFormat.testMissingBlocks() test | Minor | test | Alejandro Abdelnur | Aaron T. Myers | +| [MAPREDUCE-4572](https://issues.apache.org/jira/browse/MAPREDUCE-4572) | Can not access user logs - Jetty is not configured by default to serve aliases/symlinks | Major | tasktracker, webapps | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4570](https://issues.apache.org/jira/browse/MAPREDUCE-4570) | ProcfsBasedProcessTree#constructProcessInfo() prints a warning if procfsDir/\/stat is not found. | Minor | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4569](https://issues.apache.org/jira/browse/MAPREDUCE-4569) | TestHsWebServicesJobsQuery fails on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4562](https://issues.apache.org/jira/browse/MAPREDUCE-4562) | Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name | Major | . | Jarek Jarcec Cecho | Jarek Jarcec Cecho | +| [MAPREDUCE-4504](https://issues.apache.org/jira/browse/MAPREDUCE-4504) | SortValidator writes to wrong directory | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4503](https://issues.apache.org/jira/browse/MAPREDUCE-4503) | Should throw InvalidJobConfException if duplicates found in cacheArchives or cacheFiles | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4498](https://issues.apache.org/jira/browse/MAPREDUCE-4498) | Remove hsqldb jar from Hadoop runtime classpath | Critical | build, examples | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4496](https://issues.apache.org/jira/browse/MAPREDUCE-4496) | AM logs link is missing user name | Major | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4494](https://issues.apache.org/jira/browse/MAPREDUCE-4494) | TestFifoScheduler failing with Metrics source QueueMetrics,q0=default already exists! | Major | mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4493](https://issues.apache.org/jira/browse/MAPREDUCE-4493) | Distibuted Cache Compatability Issues | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4492](https://issues.apache.org/jira/browse/MAPREDUCE-4492) | Configuring total queue capacity between 100.5 and 99.5 at perticular level is sucessfull | Minor | mrv2 | Nishan Shetty | Mayank Bansal | +| [MAPREDUCE-4484](https://issues.apache.org/jira/browse/MAPREDUCE-4484) | Incorrect IS\_MINI\_YARN\_CLUSTER property name in YarnConfiguration | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4483](https://issues.apache.org/jira/browse/MAPREDUCE-4483) | 2.0 build does not work | Major | . | John George | John George | +| [MAPREDUCE-4470](https://issues.apache.org/jira/browse/MAPREDUCE-4470) | Fix TestCombineFileInputFormat.testForEmptyFile | Major | test | Kihwal Lee | Ilya Katsov | +| [MAPREDUCE-4467](https://issues.apache.org/jira/browse/MAPREDUCE-4467) | IndexCache failures due to missing synchronization | Critical | nodemanager | Andrey Klochkov | Kihwal Lee | +| [MAPREDUCE-4465](https://issues.apache.org/jira/browse/MAPREDUCE-4465) | Update description of yarn.nodemanager.address property | Trivial | . | Bo Wang | Bo Wang | +| [MAPREDUCE-4457](https://issues.apache.org/jira/browse/MAPREDUCE-4457) | mr job invalid transition TA\_TOO\_MANY\_FETCH\_FAILURE at FAILED | Critical | mrv2 | Thomas Graves | Robert Joseph Evans | +| [MAPREDUCE-4456](https://issues.apache.org/jira/browse/MAPREDUCE-4456) | LocalDistributedCacheManager can get an ArrayIndexOutOfBounds when creating symlinks | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4449](https://issues.apache.org/jira/browse/MAPREDUCE-4449) | Incorrect MR\_HISTORY\_STORAGE property name in JHAdminConfig | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4448](https://issues.apache.org/jira/browse/MAPREDUCE-4448) | Nodemanager crashes upon application cleanup if aggregation failed to start | Critical | mrv2, nodemanager | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4447](https://issues.apache.org/jira/browse/MAPREDUCE-4447) | Remove aop from cruft from the ant build | Major | build | Eli Collins | Eli Collins | +| [MAPREDUCE-4444](https://issues.apache.org/jira/browse/MAPREDUCE-4444) | nodemanager fails to start when one of the local-dirs is bad | Blocker | nodemanager | Nathan Roberts | Jason Lowe | +| [MAPREDUCE-4441](https://issues.apache.org/jira/browse/MAPREDUCE-4441) | Fix build issue caused by MR-3451 | Blocker | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4440](https://issues.apache.org/jira/browse/MAPREDUCE-4440) | Change SchedulerApp & SchedulerNode to be a minimal interface | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4437](https://issues.apache.org/jira/browse/MAPREDUCE-4437) | Race in MR ApplicationMaster can cause reducers to never be scheduled | Critical | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4432](https://issues.apache.org/jira/browse/MAPREDUCE-4432) | Confusing warning message when GenericOptionsParser is not used | Trivial | . | Gabriel Reid | | +| [MAPREDUCE-4423](https://issues.apache.org/jira/browse/MAPREDUCE-4423) | Potential infinite fetching of map output | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4419](https://issues.apache.org/jira/browse/MAPREDUCE-4419) | ./mapred queue -info \ -showJobs displays all the jobs irrespective of \ | Major | mrv2 | Nishan Shetty | Devaraj K | +| [MAPREDUCE-4416](https://issues.apache.org/jira/browse/MAPREDUCE-4416) | Some tests fail if Clover is enabled | Critical | client, mrv2 | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-4407](https://issues.apache.org/jira/browse/MAPREDUCE-4407) | Add hadoop-yarn-server-tests-\-tests.jar to hadoop dist package | Major | build, mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4406](https://issues.apache.org/jira/browse/MAPREDUCE-4406) | Users should be able to specify the MiniCluster ResourceManager and JobHistoryServer ports | Major | mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4402](https://issues.apache.org/jira/browse/MAPREDUCE-4402) | TestFileInputFormat fails intermittently | Major | test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4395](https://issues.apache.org/jira/browse/MAPREDUCE-4395) | Possible NPE at ClientDistributedCacheManager#determineTimestamps | Critical | distributed-cache, job submission, mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4392](https://issues.apache.org/jira/browse/MAPREDUCE-4392) | Counters.makeCompactString() changed behavior from 0.20 | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4387](https://issues.apache.org/jira/browse/MAPREDUCE-4387) | RM gets fatal error and exits during TestRM | Major | resourcemanager | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-4384](https://issues.apache.org/jira/browse/MAPREDUCE-4384) | Race conditions in IndexCache | Major | nodemanager | Kihwal Lee | Kihwal Lee | +| [MAPREDUCE-4383](https://issues.apache.org/jira/browse/MAPREDUCE-4383) | HadoopPipes.cc needs to include unistd.h | Minor | pipes | Andy Isaacson | Andy Isaacson | +| [MAPREDUCE-4380](https://issues.apache.org/jira/browse/MAPREDUCE-4380) | Empty Userlogs directory is getting created under logs directory | Minor | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4379](https://issues.apache.org/jira/browse/MAPREDUCE-4379) | Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts | Blocker | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4376](https://issues.apache.org/jira/browse/MAPREDUCE-4376) | TestClusterMRNotification times out | Major | mrv2, test | Jason Lowe | Kihwal Lee | +| [MAPREDUCE-4372](https://issues.apache.org/jira/browse/MAPREDUCE-4372) | Deadlock in Resource Manager between SchedulerEventDispatcher.EventProcessor and Shutdown hook manager | Major | mrv2, resourcemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4361](https://issues.apache.org/jira/browse/MAPREDUCE-4361) | Fix detailed metrics for protobuf-based RPC on 0.23 | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4342](https://issues.apache.org/jira/browse/MAPREDUCE-4342) | Distributed Cache gives inconsistent result if cache files get deleted from task tracker | Major | . | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-4341](https://issues.apache.org/jira/browse/MAPREDUCE-4341) | add types to capacity scheduler properties documentation | Major | capacity-sched, mrv2 | Thomas Graves | Karthik Kambatla | +| [MAPREDUCE-4336](https://issues.apache.org/jira/browse/MAPREDUCE-4336) | Distributed Shell fails when used with the CapacityScheduler | Major | mrv2 | Siddharth Seth | Ahmed Radwan | +| [MAPREDUCE-4320](https://issues.apache.org/jira/browse/MAPREDUCE-4320) | gridmix mainClass wrong in pom.xml | Major | contrib/gridmix | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4313](https://issues.apache.org/jira/browse/MAPREDUCE-4313) | TestTokenCache doesn't compile due TokenCache.getDelegationToken compilation error | Blocker | build, test | Eli Collins | Robert Joseph Evans | +| [MAPREDUCE-4307](https://issues.apache.org/jira/browse/MAPREDUCE-4307) | TeraInputFormat calls FileSystem.getDefaultBlockSize() without a Path - Failure when using ViewFileSystem | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4306](https://issues.apache.org/jira/browse/MAPREDUCE-4306) | Problem running Distributed Shell applications as a user other than the one started the daemons | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4302](https://issues.apache.org/jira/browse/MAPREDUCE-4302) | NM goes down if error encountered during log aggregation | Critical | nodemanager | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4300](https://issues.apache.org/jira/browse/MAPREDUCE-4300) | OOM in AM can turn it into a zombie. | Major | applicationmaster | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4299](https://issues.apache.org/jira/browse/MAPREDUCE-4299) | Terasort hangs with MR2 FifoScheduler | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4297](https://issues.apache.org/jira/browse/MAPREDUCE-4297) | Usersmap file in gridmix should not fail on empty lines | Major | contrib/gridmix | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4295](https://issues.apache.org/jira/browse/MAPREDUCE-4295) | RM crashes due to DNS issue | Critical | mrv2, resourcemanager | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4290](https://issues.apache.org/jira/browse/MAPREDUCE-4290) | JobStatus.getState() API is giving ambiguous values | Major | mrv2 | Nishan Shetty | Devaraj K | +| [MAPREDUCE-4276](https://issues.apache.org/jira/browse/MAPREDUCE-4276) | Allow setting yarn.nodemanager.delete.debug-delay-sec property to "-1" for easier container debugging. | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4270](https://issues.apache.org/jira/browse/MAPREDUCE-4270) | data\_join test classes are in the wrong packge | Major | mrv2 | Brock Noland | Thomas Graves | +| [MAPREDUCE-4269](https://issues.apache.org/jira/browse/MAPREDUCE-4269) | documentation: Gridmix has javadoc warnings in StressJobFactory | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4267](https://issues.apache.org/jira/browse/MAPREDUCE-4267) | mavenize pipes | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4264](https://issues.apache.org/jira/browse/MAPREDUCE-4264) | Got ClassCastException when using mapreduce.history.server.delegationtoken.required=true | Blocker | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4262](https://issues.apache.org/jira/browse/MAPREDUCE-4262) | NM gives wrong log message saying "Connected to ResourceManager" before trying to connect | Minor | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4252](https://issues.apache.org/jira/browse/MAPREDUCE-4252) | MR2 job never completes with 1 pending task | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4250](https://issues.apache.org/jira/browse/MAPREDUCE-4250) | hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMaster | Major | nodemanager | Patrick Hunt | Patrick Hunt | +| [MAPREDUCE-4238](https://issues.apache.org/jira/browse/MAPREDUCE-4238) | mavenize data\_join | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4237](https://issues.apache.org/jira/browse/MAPREDUCE-4237) | TestNodeStatusUpdater can fail if localhost has a domain associated with it | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4233](https://issues.apache.org/jira/browse/MAPREDUCE-4233) | NPE can happen in RMNMNodeInfo. | Critical | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4228](https://issues.apache.org/jira/browse/MAPREDUCE-4228) | mapreduce.job.reduce.slowstart.completedmaps is not working properly to delay the scheduling of the reduce tasks | Major | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4226](https://issues.apache.org/jira/browse/MAPREDUCE-4226) | ConcurrentModificationException in FileSystemCounterGroup | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4224](https://issues.apache.org/jira/browse/MAPREDUCE-4224) | TestFifoScheduler throws org.apache.hadoop.metrics2.MetricsException | Major | mrv2, scheduler, test | Devaraj K | Devaraj K | +| [MAPREDUCE-4220](https://issues.apache.org/jira/browse/MAPREDUCE-4220) | RM apps page starttime/endtime sorts are incorrect | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4215](https://issues.apache.org/jira/browse/MAPREDUCE-4215) | RM app page shows 500 error on appid parse error | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4211](https://issues.apache.org/jira/browse/MAPREDUCE-4211) | Error conditions (missing appid, appid not found) are masked in the RM app page | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4209](https://issues.apache.org/jira/browse/MAPREDUCE-4209) | junit dependency in hadoop-mapreduce-client is missing scope test | Major | build | Radim Kolar | | +| [MAPREDUCE-4206](https://issues.apache.org/jira/browse/MAPREDUCE-4206) | Sorting by Last Health-Update on the RM nodes page sorts does not work correctly | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4197](https://issues.apache.org/jira/browse/MAPREDUCE-4197) | Include the hsqldb jar in the hadoop-mapreduce tar file | Major | . | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4194](https://issues.apache.org/jira/browse/MAPREDUCE-4194) | ConcurrentModificationError in DirectoryCollection | Major | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4189](https://issues.apache.org/jira/browse/MAPREDUCE-4189) | TestContainerManagerSecurity is failing | Critical | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4169](https://issues.apache.org/jira/browse/MAPREDUCE-4169) | Container Logs appear in unsorted order | Minor | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4165](https://issues.apache.org/jira/browse/MAPREDUCE-4165) | Committing is misspelled as commiting in task logs | Trivial | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4160](https://issues.apache.org/jira/browse/MAPREDUCE-4160) | some mrv1 ant tests fail with timeout - due to 4156 | Major | test | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4159](https://issues.apache.org/jira/browse/MAPREDUCE-4159) | Job is running in Uber mode after setting "mapreduce.job.ubertask.maxreduces" to zero | Major | mrv2 | Nishan Shetty | Devaraj K | +| [MAPREDUCE-4156](https://issues.apache.org/jira/browse/MAPREDUCE-4156) | ant build fails compiling JobInProgress | Major | build | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4152](https://issues.apache.org/jira/browse/MAPREDUCE-4152) | map task left hanging after AM dies trying to connect to RM | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4148](https://issues.apache.org/jira/browse/MAPREDUCE-4148) | MapReduce should not have a compile-time dependency on HDFS | Major | mrv2 | Tom White | Tom White | +| [MAPREDUCE-4144](https://issues.apache.org/jira/browse/MAPREDUCE-4144) | ResourceManager NPE while handling NODE\_UPDATE | Critical | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4140](https://issues.apache.org/jira/browse/MAPREDUCE-4140) | mapreduce classes incorrectly importing "clover.org.apache.\*" classes | Major | client, mrv2 | Patrick Hunt | Patrick Hunt | +| [MAPREDUCE-4139](https://issues.apache.org/jira/browse/MAPREDUCE-4139) | Potential ResourceManager deadlock when SchedulerEventDispatcher is stopped | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4133](https://issues.apache.org/jira/browse/MAPREDUCE-4133) | MR over viewfs is broken | Major | . | John George | John George | +| [MAPREDUCE-4129](https://issues.apache.org/jira/browse/MAPREDUCE-4129) | Lots of unneeded counters log messages | Major | mrv2 | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4128](https://issues.apache.org/jira/browse/MAPREDUCE-4128) | AM Recovery expects all attempts of a completed task to also be completed. | Major | mrv2 | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4117](https://issues.apache.org/jira/browse/MAPREDUCE-4117) | mapred job -status throws NullPointerException | Critical | client, mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4102](https://issues.apache.org/jira/browse/MAPREDUCE-4102) | job counters not available in Jobhistory webui for killed jobs | Major | webapps | Thomas Graves | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4099](https://issues.apache.org/jira/browse/MAPREDUCE-4099) | ApplicationMaster may fail to remove staging directory | Critical | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4097](https://issues.apache.org/jira/browse/MAPREDUCE-4097) | tools testcases fail because missing mrapp-generated-classpath file in classpath | Major | build | Alejandro Abdelnur | Roman Shaposhnik | +| [MAPREDUCE-4092](https://issues.apache.org/jira/browse/MAPREDUCE-4092) | commitJob Exception does not fail job (regression in 0.23 vs 0.20) | Blocker | mrv2 | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-4091](https://issues.apache.org/jira/browse/MAPREDUCE-4091) | tools testcases failing because of MAPREDUCE-4082 | Critical | build, test | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4089](https://issues.apache.org/jira/browse/MAPREDUCE-4089) | Hung Tasks never time out. | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4082](https://issues.apache.org/jira/browse/MAPREDUCE-4082) | hadoop-mapreduce-client-app's mrapp-generated-classpath file should not be in the module JAR | Critical | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4074](https://issues.apache.org/jira/browse/MAPREDUCE-4074) | Client continuously retries to RM When RM goes down before launching Application Master | Major | . | Devaraj K | xieguiming | +| [MAPREDUCE-4073](https://issues.apache.org/jira/browse/MAPREDUCE-4073) | CS assigns multiple off-switch containers when using multi-level-queues | Critical | mrv2, scheduler | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4068](https://issues.apache.org/jira/browse/MAPREDUCE-4068) | Jars in lib subdirectory of the submittable JAR are not added to the classpath | Blocker | mrv2 | Ahmed Radwan | Robert Kanter | +| [MAPREDUCE-4062](https://issues.apache.org/jira/browse/MAPREDUCE-4062) | AM Launcher thread can hang forever | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4060](https://issues.apache.org/jira/browse/MAPREDUCE-4060) | Multiple SLF4J binding warning | Major | build | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4053](https://issues.apache.org/jira/browse/MAPREDUCE-4053) | Counters group names deprecation is wrong, iterating over group names deprecated names don't show up | Major | mrv2 | Alejandro Abdelnur | Robert Joseph Evans | +| [MAPREDUCE-4050](https://issues.apache.org/jira/browse/MAPREDUCE-4050) | Invalid node link | Major | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4048](https://issues.apache.org/jira/browse/MAPREDUCE-4048) | NullPointerException exception while accessing the Application Master UI | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-4040](https://issues.apache.org/jira/browse/MAPREDUCE-4040) | History links should use hostname rather than IP address. | Minor | jobhistoryserver, mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-4031](https://issues.apache.org/jira/browse/MAPREDUCE-4031) | Node Manager hangs on shut down | Critical | mrv2, nodemanager | Devaraj K | Devaraj K | +| [MAPREDUCE-4024](https://issues.apache.org/jira/browse/MAPREDUCE-4024) | RM webservices can't query on finalStatus | Major | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4020](https://issues.apache.org/jira/browse/MAPREDUCE-4020) | Web services returns incorrect JSON for deep queue tree | Major | mrv2, webapps | Jason Lowe | Anupam Seth | +| [MAPREDUCE-4012](https://issues.apache.org/jira/browse/MAPREDUCE-4012) | Hadoop Job setup error leaves no useful info to users (when LinuxTaskController is used) | Minor | . | Koji Noguchi | Thomas Graves | +| [MAPREDUCE-4010](https://issues.apache.org/jira/browse/MAPREDUCE-4010) | TestWritableJobConf fails on trunk | Critical | mrv2 | Jason Lowe | Alejandro Abdelnur | +| [MAPREDUCE-4002](https://issues.apache.org/jira/browse/MAPREDUCE-4002) | MultiFileWordCount job fails if the input path is not from default file system | Major | examples | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3999](https://issues.apache.org/jira/browse/MAPREDUCE-3999) | Tracking link gives an error if the AppMaster hasn't started yet | Major | mrv2, webapps | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-3993](https://issues.apache.org/jira/browse/MAPREDUCE-3993) | Graceful handling of codec errors during decompression | Major | mrv1, mrv2 | Todd Lipcon | Karthik Kambatla | +| [MAPREDUCE-3992](https://issues.apache.org/jira/browse/MAPREDUCE-3992) | Reduce fetcher doesn't verify HTTP status code of response | Major | mrv1 | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-3988](https://issues.apache.org/jira/browse/MAPREDUCE-3988) | mapreduce.job.local.dir doesn't point to a single directory on a node. | Major | mrv2 | Vinod Kumar Vavilapalli | Eric Payne | +| [MAPREDUCE-3947](https://issues.apache.org/jira/browse/MAPREDUCE-3947) | yarn.app.mapreduce.am.resource.mb not documented | Minor | . | Todd Lipcon | Devaraj K | +| [MAPREDUCE-3932](https://issues.apache.org/jira/browse/MAPREDUCE-3932) | MR tasks failing and crashing the AM when available-resources/headRoom becomes zero | Critical | mr-am, mrv2 | Vinod Kumar Vavilapalli | Robert Joseph Evans | +| [MAPREDUCE-3927](https://issues.apache.org/jira/browse/MAPREDUCE-3927) | Shuffle hang when set map.failures.percent | Critical | mrv2 | MengWang | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3893](https://issues.apache.org/jira/browse/MAPREDUCE-3893) | allow capacity scheduler configs maximum-applications and maximum-am-resource-percent configurable on a per queue basis | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [MAPREDUCE-3889](https://issues.apache.org/jira/browse/MAPREDUCE-3889) | job client tries to use /tasklog interface, but that doesn't exist anymore | Critical | mrv2 | Thomas Graves | Devaraj K | +| [MAPREDUCE-3873](https://issues.apache.org/jira/browse/MAPREDUCE-3873) | Nodemanager is not getting decommisioned if the absolute ip is given in exclude file. | Minor | mrv2, nodemanager | Nishan Shetty | xieguiming | +| [MAPREDUCE-3870](https://issues.apache.org/jira/browse/MAPREDUCE-3870) | Invalid App Metrics | Major | mrv2 | Bhallamudi Venkata Siva Kamesh | Bhallamudi Venkata Siva Kamesh | +| [MAPREDUCE-3782](https://issues.apache.org/jira/browse/MAPREDUCE-3782) | teragen terasort jobs fail when using webhdfs:// | Critical | mrv2 | Arpit Gupta | Jason Lowe | +| [MAPREDUCE-3728](https://issues.apache.org/jira/browse/MAPREDUCE-3728) | ShuffleHandler can't access results when configured in a secure mode | Critical | mrv2, nodemanager | Roman Shaposhnik | Ding Yuan | +| [MAPREDUCE-3682](https://issues.apache.org/jira/browse/MAPREDUCE-3682) | Tracker URL says AM tasks run on localhost | Major | mrv2 | David Capwell | Ravi Prakash | +| [MAPREDUCE-3672](https://issues.apache.org/jira/browse/MAPREDUCE-3672) | Killed maps shouldn't be counted towards JobCounter.NUM\_FAILED\_MAPS | Major | mr-am, mrv2 | Vinod Kumar Vavilapalli | Anupam Seth | +| [MAPREDUCE-3650](https://issues.apache.org/jira/browse/MAPREDUCE-3650) | testGetTokensForHftpFS() fails | Blocker | mrv2 | Thomas Graves | Ravi Prakash | +| [MAPREDUCE-3621](https://issues.apache.org/jira/browse/MAPREDUCE-3621) | TestDBJob and TestDataDrivenDBInputFormat ant tests fail | Major | mrv2 | Thomas Graves | Ravi Prakash | +| [MAPREDUCE-3543](https://issues.apache.org/jira/browse/MAPREDUCE-3543) | Mavenize Gridmix. | Critical | mrv2 | Mahadev konar | Thomas Graves | +| [MAPREDUCE-3506](https://issues.apache.org/jira/browse/MAPREDUCE-3506) | Calling getPriority on JobInfo after parsing a history log with JobHistoryParser throws a NullPointerException | Minor | client, mrv2 | Ratandeep Ratti | Jason Lowe | +| [MAPREDUCE-3493](https://issues.apache.org/jira/browse/MAPREDUCE-3493) | Add the default mapreduce.shuffle.port property to mapred-default.xml | Minor | mrv2 | Ahmed Radwan | | +| [MAPREDUCE-3350](https://issues.apache.org/jira/browse/MAPREDUCE-3350) | Per-app RM page should have the list of application-attempts like on the app JHS page | Critical | mrv2, webapps | Vinod Kumar Vavilapalli | Jonathan Eagles | +| [MAPREDUCE-3348](https://issues.apache.org/jira/browse/MAPREDUCE-3348) | mapred job -status fails to give info even if the job is present in History | Major | mrv2 | Devaraj K | Devaraj K | +| [MAPREDUCE-3082](https://issues.apache.org/jira/browse/MAPREDUCE-3082) | archive command take wrong path for input file with current directory | Major | harchive | Rajit Saha | John George | +| [MAPREDUCE-2739](https://issues.apache.org/jira/browse/MAPREDUCE-2739) | MR-279: Update installation docs (remove YarnClientFactory) | Minor | mrv2 | Ahmed Radwan | Bo Wang | +| [MAPREDUCE-2374](https://issues.apache.org/jira/browse/MAPREDUCE-2374) | "Text File Busy" errors launching MR tasks | Major | . | Todd Lipcon | Andy Isaacson | +| [MAPREDUCE-2289](https://issues.apache.org/jira/browse/MAPREDUCE-2289) | Permissions race can make getStagingDir fail on local filesystem | Major | job submission | Todd Lipcon | Ahmed Radwan | +| [MAPREDUCE-2220](https://issues.apache.org/jira/browse/MAPREDUCE-2220) | Fix new API FileOutputFormat-related typos in mapred-default.xml | Minor | documentation | Rui KUBO | Rui KUBO | +| [YARN-174](https://issues.apache.org/jira/browse/YARN-174) | TestNodeStatusUpdater is failing in trunk | Major | nodemanager | Robert Joseph Evans | Vinod Kumar Vavilapalli | +| [YARN-138](https://issues.apache.org/jira/browse/YARN-138) | Improve default config values for YARN | Major | resourcemanager, scheduler | Arun C Murthy | Harsh J | +| [YARN-108](https://issues.apache.org/jira/browse/YARN-108) | FSDownload can create cache directories with the wrong permissions | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-106](https://issues.apache.org/jira/browse/YARN-106) | Nodemanager needs to set permissions of local directories | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-88](https://issues.apache.org/jira/browse/YARN-88) | DefaultContainerExecutor can fail to set proper permissions | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-87](https://issues.apache.org/jira/browse/YARN-87) | NM ResourceLocalizationService does not set permissions of local cache directories | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-83](https://issues.apache.org/jira/browse/YARN-83) | Change package of YarnClient to include apache | Major | client | Bikas Saha | Bikas Saha | +| [YARN-79](https://issues.apache.org/jira/browse/YARN-79) | Calling YarnClientImpl.close throws Exception | Major | client | Bikas Saha | Vinod Kumar Vavilapalli | +| [YARN-75](https://issues.apache.org/jira/browse/YARN-75) | RMContainer should handle a RELEASE event while RUNNING | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-68](https://issues.apache.org/jira/browse/YARN-68) | NodeManager will refuse to shutdown indefinitely due to container log aggregation | Major | nodemanager | patrick white | Daryn Sharp | +| [YARN-66](https://issues.apache.org/jira/browse/YARN-66) | aggregated logs permissions not set properly | Critical | nodemanager | Thomas Graves | Thomas Graves | +| [YARN-63](https://issues.apache.org/jira/browse/YARN-63) | RMNodeImpl is missing valid transitions from the UNHEALTHY state | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-58](https://issues.apache.org/jira/browse/YARN-58) | NM leaks filesystems | Critical | nodemanager | Daryn Sharp | Jason Lowe | +| [YARN-42](https://issues.apache.org/jira/browse/YARN-42) | Node Manager throws NPE on startup | Major | nodemanager | Devaraj K | Devaraj K | +| [YARN-37](https://issues.apache.org/jira/browse/YARN-37) | TestRMAppTransitions.testAppSubmittedKilled passes for the wrong reason | Minor | resourcemanager | Jason Lowe | Mayank Bansal | +| [YARN-36](https://issues.apache.org/jira/browse/YARN-36) | branch-2.1.0-alpha doesn't build | Blocker | . | Eli Collins | Radim Kolar | +| [YARN-31](https://issues.apache.org/jira/browse/YARN-31) | TestDelegationTokenRenewer fails on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [YARN-27](https://issues.apache.org/jira/browse/YARN-27) | Failed refreshQueues due to misconfiguration prevents further refreshing of queues | Major | . | Ramya Sunil | Arun C Murthy | +| [YARN-25](https://issues.apache.org/jira/browse/YARN-25) | remove old aggregated logs | Major | . | Thomas Graves | Robert Joseph Evans | +| [YARN-22](https://issues.apache.org/jira/browse/YARN-22) | Using URI for yarn.nodemanager log dirs fails | Minor | . | Eli Collins | Mayank Bansal | +| [YARN-15](https://issues.apache.org/jira/browse/YARN-15) | YarnConfiguration DEFAULT\_YARN\_APPLICATION\_CLASSPATH should be updated | Critical | nodemanager | Alejandro Abdelnur | Arun C Murthy | +| [YARN-14](https://issues.apache.org/jira/browse/YARN-14) | Symlinks to peer distributed cache files no longer work | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-13](https://issues.apache.org/jira/browse/YARN-13) | Merge of yarn reorg into branch-2 copied trunk tree | Critical | . | Todd Lipcon | | +| [YARN-12](https://issues.apache.org/jira/browse/YARN-12) | Several Findbugs issues with new FairScheduler in YARN | Major | scheduler | Junping Du | Junping Du | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8283](https://issues.apache.org/jira/browse/HADOOP-8283) | Allow tests to control token service value | Major | test | Daryn Sharp | Daryn Sharp | +| [HDFS-3709](https://issues.apache.org/jira/browse/HDFS-3709) | TestStartup tests still binding to the ephemeral port | Major | test | Eli Collins | Eli Collins | +| [HDFS-3665](https://issues.apache.org/jira/browse/HDFS-3665) | Add a test for renaming across file systems via a symlink | Major | test | Eli Collins | Eli Collins | +| [HDFS-3634](https://issues.apache.org/jira/browse/HDFS-3634) | Add self-contained, mavenized fuse\_dfs test | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3606](https://issues.apache.org/jira/browse/HDFS-3606) | libhdfs: create self-contained unit test | Minor | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3291](https://issues.apache.org/jira/browse/HDFS-3291) | add test that covers HttpFS working w/ a non-HDFS Hadoop filesystem | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3258](https://issues.apache.org/jira/browse/HDFS-3258) | Test for HADOOP-8144 (pseudoSortByDistance in NetworkTopology for first rack local node) | Major | test | Eli Collins | Junping Du | +| [MAPREDUCE-4212](https://issues.apache.org/jira/browse/MAPREDUCE-4212) | TestJobClientGetJob sometimes fails | Major | test | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3983](https://issues.apache.org/jira/browse/MAPREDUCE-3983) | TestTTResourceReporting can fail, and should just be deleted | Major | mrv1 | Robert Joseph Evans | Ravi Prakash | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-7754](https://issues.apache.org/jira/browse/HADOOP-7754) | Expose file descriptors from Hadoop-wrapped local FileSystems | Major | native, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-3502](https://issues.apache.org/jira/browse/HDFS-3502) | Change INodeFile and INodeFileUnderConstruction to package private | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3474](https://issues.apache.org/jira/browse/HDFS-3474) | Cleanup Exception handling in BookKeeper journal manager | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-3468](https://issues.apache.org/jira/browse/HDFS-3468) | Make BKJM-ZK session timeout configurable. | Major | . | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-3452](https://issues.apache.org/jira/browse/HDFS-3452) | BKJM:Switch from standby to active fails and NN gets shut down due to delay in clearing of lock | Blocker | . | suja s | Uma Maheswara Rao G | +| [HDFS-3441](https://issues.apache.org/jira/browse/HDFS-3441) | Race condition between rolling logs at active NN and purging at standby | Major | . | suja s | Rakesh R | +| [HDFS-3423](https://issues.apache.org/jira/browse/HDFS-3423) | BKJM: NN startup is failing, when tries to recoverUnfinalizedSegments() a bad inProgress\_ ZNodes | Major | . | Rakesh R | Ivan Kelly | +| [HDFS-3408](https://issues.apache.org/jira/browse/HDFS-3408) | BKJM : Namenode format fails, if there is no BK root | Minor | namenode | Rakesh R | Rakesh R | +| [HDFS-3389](https://issues.apache.org/jira/browse/HDFS-3389) | Document the BKJM usage in Namenode HA. | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-3369](https://issues.apache.org/jira/browse/HDFS-3369) | change variable names referring to inode in blockmanagement to more appropriate | Minor | namenode | John George | John George | +| [HDFS-3190](https://issues.apache.org/jira/browse/HDFS-3190) | Simple refactors in existing NN code to assist QuorumJournalManager extension | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3168](https://issues.apache.org/jira/browse/HDFS-3168) | Clean up FSNamesystem and BlockManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3058](https://issues.apache.org/jira/browse/HDFS-3058) | HA: Bring BookKeeperJournalManager up to date with HA changes | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-2717](https://issues.apache.org/jira/browse/HDFS-2717) | BookKeeper Journal output stream doesn't check addComplete rc | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-2330](https://issues.apache.org/jira/browse/HDFS-2330) | In NNStorage.java, IOExceptions of stream closures can mask root exceptions. | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [MAPREDUCE-4163](https://issues.apache.org/jira/browse/MAPREDUCE-4163) | consistently set the bind address | Major | mrv2 | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4162](https://issues.apache.org/jira/browse/MAPREDUCE-4162) | Correctly set token service | Major | client, mrv2 | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4161](https://issues.apache.org/jira/browse/MAPREDUCE-4161) | create sockets consistently | Major | client, mrv2 | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-3972](https://issues.apache.org/jira/browse/MAPREDUCE-3972) | Locking and exception issues in JobHistory Server. | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3942](https://issues.apache.org/jira/browse/MAPREDUCE-3942) | Randomize master key generation for ApplicationTokenSecretManager and roll it every so often | Major | mrv2, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3940](https://issues.apache.org/jira/browse/MAPREDUCE-3940) | ContainerTokens should have an expiry interval | Major | mrv2, security | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-3613](https://issues.apache.org/jira/browse/MAPREDUCE-3613) | web service calls header contains 2 content types | Critical | mrv2 | Thomas Graves | Thomas Graves | +| [YARN-60](https://issues.apache.org/jira/browse/YARN-60) | NMs rejects all container tokens after secret key rolls | Blocker | nodemanager | Daryn Sharp | Vinod Kumar Vavilapalli | +| [YARN-39](https://issues.apache.org/jira/browse/YARN-39) | RM-NM secret-keys should be randomly generated and rolled every so often | Critical | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-29](https://issues.apache.org/jira/browse/YARN-29) | Add a yarn-client module | Major | client | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8403](https://issues.apache.org/jira/browse/HADOOP-8403) | bump up POMs version to 2.0.1-SNAPSHOT | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4134](https://issues.apache.org/jira/browse/MAPREDUCE-4134) | Remove references of mapred.child.ulimit etc. since they are not being used any more | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4051](https://issues.apache.org/jira/browse/MAPREDUCE-4051) | Remove the empty hadoop-mapreduce-project/assembly/all.xml file | Major | . | Ravi Prakash | Ravi Prakash | +| [YARN-154](https://issues.apache.org/jira/browse/YARN-154) | Create Yarn trunk and commit jobs | Major | . | Eli Collins | Robert Joseph Evans | +| [YARN-1](https://issues.apache.org/jira/browse/YARN-1) | Move YARN out of hadoop-mapreduce | Major | . | Arun C Murthy | Arun C Murthy | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md new file mode 100644 index 00000000000..91d0ddc9fc1 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.2-alpha/RELEASENOTES.2.0.2-alpha.md @@ -0,0 +1,273 @@ + + +# Apache Hadoop 2.0.2-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-8794](https://issues.apache.org/jira/browse/HADOOP-8794) | *Major* | **Modifiy bin/hadoop to point to HADOOP\_YARN\_HOME** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8710](https://issues.apache.org/jira/browse/HADOOP-8710) | *Major* | **Remove ability for users to easily run the trash emptier** + +The trash emptier may no longer be run using "hadoop org.apache.hadoop.fs.Trash". The trash emptier runs on the NameNode (if configured). Old trash checkpoints may be deleted using "hadoop fs -expunge". + + +--- + +* [HADOOP-8703](https://issues.apache.org/jira/browse/HADOOP-8703) | *Major* | **distcpV2: turn CRC checking off for 0 byte size** + +distcp skips CRC on 0 byte files. + + +--- + +* [HADOOP-8689](https://issues.apache.org/jira/browse/HADOOP-8689) | *Major* | **Make trash a server side configuration option** + +If fs.trash.interval is configured on the server then the client's value for this configuration is ignored. + + +--- + +* [HADOOP-8551](https://issues.apache.org/jira/browse/HADOOP-8551) | *Major* | **fs -mkdir creates parent directories without the -p option** + +FsShell's "mkdir" no longer implicitly creates all non-existent parent directories. The command adopts the posix compliant behavior of requiring the "-p" flag to auto-create parent directories. + + +--- + +* [HADOOP-8533](https://issues.apache.org/jira/browse/HADOOP-8533) | *Major* | **Remove Parallel Call in IPC** + +Merged the change to branch-2 + + +--- + +* [HADOOP-8458](https://issues.apache.org/jira/browse/HADOOP-8458) | *Major* | **Add management hook to AuthenticationHandler to enable delegation token operations support** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8388](https://issues.apache.org/jira/browse/HADOOP-8388) | *Minor* | **Remove unused BlockLocation serialization** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8368](https://issues.apache.org/jira/browse/HADOOP-8368) | *Minor* | **Use CMake rather than autotools to build native code** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8327](https://issues.apache.org/jira/browse/HADOOP-8327) | *Major* | **distcpv2 and distcpv1 jars should not coexist** + +Resolve sporadic distcp issue due to having two DistCp classes (v1 & v2) in the classpath. + + +--- + +* [HADOOP-7703](https://issues.apache.org/jira/browse/HADOOP-7703) | *Major* | **WebAppContext should also be stopped and cleared** + +Improved excpetion handling of shutting down web server. (Devaraj K via Eric Yang) + + +--- + +* [HDFS-3755](https://issues.apache.org/jira/browse/HDFS-3755) | *Major* | **Creating an already-open-for-write file with overwrite=true fails** + +This is an incompatible change: Before this change, if a file is already open for write by one client, and another client calls fs.create() with overwrite=true, an AlreadyBeingCreatedException is thrown. After this change, the file will be deleted and the new file will be created successfully. + + +--- + +* [HDFS-3697](https://issues.apache.org/jira/browse/HDFS-3697) | *Minor* | **Enable fadvise readahead by default** + +The datanode now performs 4MB readahead by default when reading data from its disks, if the native libraries are present. This has been shown to improve performance in many workloads. The feature may be disabled by setting dfs.datanode.readahead.bytes to "0". + + +--- + +* [HDFS-3675](https://issues.apache.org/jira/browse/HDFS-3675) | *Minor* | **libhdfs: follow documented return codes** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-3522](https://issues.apache.org/jira/browse/HDFS-3522) | *Major* | **If NN is in safemode, it should throw SafeModeException when getBlockLocations has zero locations** + +getBlockLocations(), and hence open() for read, will now throw SafeModeException if the NameNode is still in safe mode and there are no replicas reported yet for one of the blocks in the file. + + +--- + +* [HDFS-3518](https://issues.apache.org/jira/browse/HDFS-3518) | *Major* | **Provide API to check HDFS operational state** + +Add a utility method HdfsUtils.isHealthy(uri) for checking if the given HDFS is healthy. + + +--- + +* [HDFS-3475](https://issues.apache.org/jira/browse/HDFS-3475) | *Trivial* | **Make the replication and invalidation rates configurable** + +This change adds two new configuration parameters. +# {{dfs.namenode.invalidate.work.pct.per.iteration}} for controlling deletion rate of blocks. +# {{dfs.namenode.replication.work.multiplier.per.iteration}} for controlling replication rate. This in turn allows controlling the time it takes for decommissioning. + +Please see hdfs-default.xml for detailed description. + + +--- + +* [HDFS-3446](https://issues.apache.org/jira/browse/HDFS-3446) | *Major* | **HostsFileReader silently ignores bad includes/excludes** + +HDFS no longer silently ignores missing or unreadable host files specified by dfs.hosts or dfs.hosts.exclude. In order to specify that no hosts should be included or excluded, administrators should either refrain from setting the relevant config properties, or create an empty file in order to represent an empty list. + + +--- + +* [HDFS-3318](https://issues.apache.org/jira/browse/HDFS-3318) | *Blocker* | **Hftp hangs on transfers \>2GB** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-3110](https://issues.apache.org/jira/browse/HDFS-3110) | *Major* | **libhdfs implementation of direct read API** + +libhdfs is enhanced to read directly into user-supplied buffers when possible, reducing the number of memory copies. + + +--- + +* [HDFS-2793](https://issues.apache.org/jira/browse/HDFS-2793) | *Major* | **Add an admin command to trigger an edit log roll** + +Introduced a new command, "hdfs dfsadmin -rollEdits" which requests that the active NameNode roll its edit log. This can be useful for administrators manually backing up log segments. + + +--- + +* [HDFS-2727](https://issues.apache.org/jira/browse/HDFS-2727) | *Minor* | **libhdfs should get the default block size from the server** + +libhdfs now uses the server block size configuration rather than the deprecated dfs.block.size client configuration. + + +--- + +* [HDFS-2686](https://issues.apache.org/jira/browse/HDFS-2686) | *Major* | **Remove DistributedUpgrade related code** + +This jira removes functionality that has not been used/applicable since release 0.17. The incompatibility introduced by this change will not affect any HDFS users. + + +--- + +* [HDFS-2617](https://issues.apache.org/jira/browse/HDFS-2617) | *Major* | **Replaced Kerberized SSL for image transfer and fsck with SPNEGO-based solution** + +Due to the requirement that KSSL use weak encryption types for Kerberos tickets, HTTP authentication to the NameNode will now use SPNEGO by default. This will require users of previous branch-1 releases with security enabled to modify their configurations and create new Kerberos principals in order to use SPNEGO. The old behavior of using KSSL can optionally be enabled by setting the configuration option "hadoop.security.use-weak-http-crypto" to "true". + + +--- + +* [MAPREDUCE-4629](https://issues.apache.org/jira/browse/MAPREDUCE-4629) | *Major* | **Remove JobHistory.DEBUG\_MODE** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4311](https://issues.apache.org/jira/browse/MAPREDUCE-4311) | *Major* | **Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4072](https://issues.apache.org/jira/browse/MAPREDUCE-4072) | *Major* | **User set java.library.path seems to overwrite default creating problems native lib loading** + +-Djava.library.path in mapred.child.java.opts can cause issues with native libraries. LD\_LIBRARY\_PATH through mapred.child.env should be used instead. + + +--- + +* [MAPREDUCE-4017](https://issues.apache.org/jira/browse/MAPREDUCE-4017) | *Trivial* | **Add jobname to jobsummary log** + +The Job Summary log may contain commas in values that are escaped by a '\' character. This was true before, but is more likely to be exposed now. + + +--- + +* [MAPREDUCE-3940](https://issues.apache.org/jira/browse/MAPREDUCE-3940) | *Major* | **ContainerTokens should have an expiry interval** + +ContainerTokens now have an expiry interval so that stale tokens cannot be used for launching containers. + + +--- + +* [MAPREDUCE-3873](https://issues.apache.org/jira/browse/MAPREDUCE-3873) | *Minor* | **Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.** + +Fixed NodeManagers' decommissioning at RM to accept IP addresses also. + + +--- + +* [MAPREDUCE-3812](https://issues.apache.org/jira/browse/MAPREDUCE-3812) | *Major* | **Lower default allocation sizes, fix allocation configurations and document them** + +Removes two sets of previously available config properties: + +1. ( yarn.scheduler.fifo.minimum-allocation-mb and yarn.scheduler.fifo.maximum-allocation-mb ) and, +2. ( yarn.scheduler.capacity.minimum-allocation-mb and yarn.scheduler.capacity.maximum-allocation-mb ) + +In favor of two new, generically named properties: + +1. yarn.scheduler.minimum-allocation-mb - This acts as the floor value of memory resource requests for containers. +2. yarn.scheduler.maximum-allocation-mb - This acts as the ceiling value of memory resource requests for containers. + +Both these properties need to be set at the ResourceManager (RM) to take effect, as the RM is where the scheduler resides. + +Also changes the default minimum and maximums to 128 MB and 10 GB respectively. + + +--- + +* [MAPREDUCE-3543](https://issues.apache.org/jira/browse/MAPREDUCE-3543) | *Critical* | **Mavenize Gridmix.** + +Note that to apply this you should first run the script - ./MAPREDUCE-3543v3.sh svn, then apply the patch. + +If this is merged to more then trunk, the version inside of hadoop-tools/hadoop-gridmix/pom.xml will need to be udpated accordingly. + + +--- + +* [MAPREDUCE-3348](https://issues.apache.org/jira/browse/MAPREDUCE-3348) | *Major* | **mapred job -status fails to give info even if the job is present in History** + +Fixed a bug in MR client to redirect to JobHistoryServer correctly when RM forgets the app. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/CHANGES.2.0.3-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/CHANGES.2.0.3-alpha.md new file mode 100644 index 00000000000..88437955c97 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/CHANGES.2.0.3-alpha.md @@ -0,0 +1,588 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.3-alpha - 2013-02-14 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9070](https://issues.apache.org/jira/browse/HADOOP-9070) | Kerberos SASL server cannot find kerberos key | Blocker | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-8999](https://issues.apache.org/jira/browse/HADOOP-8999) | SASL negotiation is flawed | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HDFS-4451](https://issues.apache.org/jira/browse/HDFS-4451) | hdfs balancer command returns exit code 1 on success instead of 0 | Major | balancer & mover | Joshua Blatt | | +| [HDFS-4369](https://issues.apache.org/jira/browse/HDFS-4369) | GetBlockKeysResponseProto does not handle null response | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4367](https://issues.apache.org/jira/browse/HDFS-4367) | GetDataEncryptionKeyResponseProto does not handle null response | Blocker | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4364](https://issues.apache.org/jira/browse/HDFS-4364) | GetLinkTargetResponseProto does not handle null path | Blocker | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4362](https://issues.apache.org/jira/browse/HDFS-4362) | GetDelegationTokenResponseProto does not handle null token | Critical | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4350](https://issues.apache.org/jira/browse/HDFS-4350) | Make enabling of stale marking on read and write paths independent | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-4122](https://issues.apache.org/jira/browse/HDFS-4122) | Cleanup HDFS logs and reduce the size of logged messages | Major | datanode, hdfs-client, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4080](https://issues.apache.org/jira/browse/HDFS-4080) | Add a separate logger for block state change logs to enable turning off those logs | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-1331](https://issues.apache.org/jira/browse/HDFS-1331) | dfs -test should work like /bin/test | Minor | tools | Allen Wittenauer | Andy Isaacson | +| [MAPREDUCE-4928](https://issues.apache.org/jira/browse/MAPREDUCE-4928) | Use token request messages defined in hadoop common | Major | applicationmaster, security | Suresh Srinivas | Suresh Srinivas | +| [MAPREDUCE-4123](https://issues.apache.org/jira/browse/MAPREDUCE-4123) | ./mapred groups gives NoClassDefFoundError | Critical | mrv2 | Nishan Shetty | Devaraj K | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9090](https://issues.apache.org/jira/browse/HADOOP-9090) | Support on-demand publish of metrics | Minor | metrics | Mostafa Elhemali | Mostafa Elhemali | +| [HADOOP-9054](https://issues.apache.org/jira/browse/HADOOP-9054) | Add AuthenticationHandler that uses Kerberos but allows for an alternate form of authentication for browsers | Major | security | Robert Kanter | Robert Kanter | +| [HADOOP-8597](https://issues.apache.org/jira/browse/HADOOP-8597) | FsShell's Text command should be able to read avro data files | Major | fs | Harsh J | Ivan Vladimirov Ivanov | +| [HDFS-4456](https://issues.apache.org/jira/browse/HDFS-4456) | Add concat to HttpFS and WebHDFS REST API docs | Major | webhdfs | Tsz Wo Nicholas Sze | Plamen Jeliazkov | +| [HDFS-4213](https://issues.apache.org/jira/browse/HDFS-4213) | When the client calls hsync, allows the client to update the file length in the NameNode | Major | hdfs-client, namenode | Jing Zhao | Jing Zhao | +| [HDFS-3598](https://issues.apache.org/jira/browse/HDFS-3598) | WebHDFS: support file concat | Major | webhdfs | Tsz Wo Nicholas Sze | Plamen Jeliazkov | +| [HDFS-3077](https://issues.apache.org/jira/browse/HDFS-3077) | Quorum-based protocol for reading and writing edit logs | Major | ha, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3049](https://issues.apache.org/jira/browse/HDFS-3049) | During the normal loading NN startup process, fall back on a different EditLog if we see one that is corrupt | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-1245](https://issues.apache.org/jira/browse/HDFS-1245) | Plugable block id generation | Major | namenode | Dmytro Molkov | Konstantin Shvachko | +| [MAPREDUCE-4808](https://issues.apache.org/jira/browse/MAPREDUCE-4808) | Refactor MapOutput and MergeManager to facilitate reuse by Shuffle implementations | Major | . | Arun C Murthy | Mariappan Asokan | +| [MAPREDUCE-4520](https://issues.apache.org/jira/browse/MAPREDUCE-4520) | Add experimental support for MR AM to schedule CPUs along-with memory | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-3678](https://issues.apache.org/jira/browse/MAPREDUCE-3678) | The Map tasks logs should have the value of input split it processed | Major | mrv1, mrv2 | Bejoy KS | Harsh J | +| [MAPREDUCE-2454](https://issues.apache.org/jira/browse/MAPREDUCE-2454) | Allow external sorter plugin for MR | Minor | . | Mariappan Asokan | Mariappan Asokan | +| [YARN-286](https://issues.apache.org/jira/browse/YARN-286) | Add a YARN ApplicationClassLoader | Major | applications | Tom White | Tom White | +| [YARN-187](https://issues.apache.org/jira/browse/YARN-187) | Add hierarchical queues to the fair scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-146](https://issues.apache.org/jira/browse/YARN-146) | Add unit tests for computing fair share in the fair scheduler | Major | resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-145](https://issues.apache.org/jira/browse/YARN-145) | Add a Web UI to the fair share scheduler | Major | resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-2](https://issues.apache.org/jira/browse/YARN-2) | Enhance CS to schedule accounting for both memory and cpu cores | Major | capacityscheduler, scheduler | Arun C Murthy | Arun C Murthy | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9276](https://issues.apache.org/jira/browse/HADOOP-9276) | Allow BoundedByteArrayOutputStream to be resettable | Minor | . | Arun C Murthy | Arun C Murthy | +| [HADOOP-9247](https://issues.apache.org/jira/browse/HADOOP-9247) | parametrize Clover "generateXxx" properties to make them re-definable via -D in mvn calls | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9216](https://issues.apache.org/jira/browse/HADOOP-9216) | CompressionCodecFactory#getCodecClasses should trim the result of parsing by Configuration. | Major | io | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-9192](https://issues.apache.org/jira/browse/HADOOP-9192) | Move token related request/response messages to common | Major | security | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9162](https://issues.apache.org/jira/browse/HADOOP-9162) | Add utility to check native library availability | Minor | native | Binglin Chang | Binglin Chang | +| [HADOOP-9153](https://issues.apache.org/jira/browse/HADOOP-9153) | Support createNonRecursive in ViewFileSystem | Major | viewfs | Sandy Ryza | Sandy Ryza | +| [HADOOP-9147](https://issues.apache.org/jira/browse/HADOOP-9147) | Add missing fields to FIleStatus.toString | Trivial | . | Jonathan Allen | Jonathan Allen | +| [HADOOP-9127](https://issues.apache.org/jira/browse/HADOOP-9127) | Update documentation for ZooKeeper Failover Controller | Major | documentation | Daisuke Kobayashi | Daisuke Kobayashi | +| [HADOOP-9118](https://issues.apache.org/jira/browse/HADOOP-9118) | FileSystemContractBaseTest test data for read/write isn't rigorous enough | Trivial | test | Steve Loughran | | +| [HADOOP-9106](https://issues.apache.org/jira/browse/HADOOP-9106) | Allow configuration of IPC connect timeout | Major | ipc | Todd Lipcon | Robert Parker | +| [HADOOP-9093](https://issues.apache.org/jira/browse/HADOOP-9093) | Move all the Exception in PathExceptions to o.a.h.fs package | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9004](https://issues.apache.org/jira/browse/HADOOP-9004) | Allow security unit tests to use external KDC | Major | security, test | Stephen Chu | Stephen Chu | +| [HADOOP-8998](https://issues.apache.org/jira/browse/HADOOP-8998) | set Cache-Control no-cache header on all dynamic content | Minor | . | Andy Isaacson | Alejandro Abdelnur | +| [HADOOP-8992](https://issues.apache.org/jira/browse/HADOOP-8992) | Enhance unit-test coverage of class HarFileSystem | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8985](https://issues.apache.org/jira/browse/HADOOP-8985) | Add namespace declarations in .proto files for languages other than java | Minor | ha, ipc | Binglin Chang | Binglin Chang | +| [HADOOP-8951](https://issues.apache.org/jira/browse/HADOOP-8951) | RunJar to fail with user-comprehensible error message if jar missing | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-8932](https://issues.apache.org/jira/browse/HADOOP-8932) | JNI-based user-group mapping modules can be too chatty on lookup failures | Major | security | Kihwal Lee | Kihwal Lee | +| [HADOOP-8931](https://issues.apache.org/jira/browse/HADOOP-8931) | Add Java version to startup message | Trivial | . | Eli Collins | Eli Collins | +| [HADOOP-8930](https://issues.apache.org/jira/browse/HADOOP-8930) | Cumulative code coverage calculation | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-8929](https://issues.apache.org/jira/browse/HADOOP-8929) | Add toString, other improvements for SampleQuantiles | Major | metrics | Todd Lipcon | Todd Lipcon | +| [HADOOP-8926](https://issues.apache.org/jira/browse/HADOOP-8926) | hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data | Major | util | Gopal V | Gopal V | +| [HADOOP-8925](https://issues.apache.org/jira/browse/HADOOP-8925) | Remove the packaging | Minor | build | Eli Collins | Eli Collins | +| [HADOOP-8922](https://issues.apache.org/jira/browse/HADOOP-8922) | Provide alternate JSONP output for JMXJsonServlet to allow javascript in browser dashboard | Trivial | metrics | Damien Hardy | Damien Hardy | +| [HADOOP-8909](https://issues.apache.org/jira/browse/HADOOP-8909) | Hadoop Common Maven protoc calls must not depend on external sh script | Major | build | Chris Nauroth | Chris Nauroth | +| [HADOOP-8894](https://issues.apache.org/jira/browse/HADOOP-8894) | GenericTestUtils.waitFor should dump thread stacks on timeout | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-8889](https://issues.apache.org/jira/browse/HADOOP-8889) | Upgrade to Surefire 2.12.3 | Major | build, test | Todd Lipcon | Todd Lipcon | +| [HADOOP-8866](https://issues.apache.org/jira/browse/HADOOP-8866) | SampleQuantiles#query is O(N^2) instead of O(N) | Minor | . | Andrew Wang | Andrew Wang | +| [HADOOP-8851](https://issues.apache.org/jira/browse/HADOOP-8851) | Use -XX:+HeapDumpOnOutOfMemoryError JVM option in the forked tests | Minor | test | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8849](https://issues.apache.org/jira/browse/HADOOP-8849) | FileUtil#fullyDelete should grant the target directories +rwx permissions before trying to delete them | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8812](https://issues.apache.org/jira/browse/HADOOP-8812) | ExitUtil#terminate should print Exception#toString | Minor | . | Eli Collins | Eli Collins | +| [HADOOP-8806](https://issues.apache.org/jira/browse/HADOOP-8806) | libhadoop.so: dlopen should be better at locating libsnappy.so, etc. | Minor | build | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8804](https://issues.apache.org/jira/browse/HADOOP-8804) | Improve Web UIs when the wildcard address is used | Minor | . | Eli Collins | Senthil V Kumar | +| [HADOOP-8789](https://issues.apache.org/jira/browse/HADOOP-8789) | Tests setLevel(Level.OFF) should be Level.ERROR | Minor | test | Andy Isaacson | Andy Isaacson | +| [HADOOP-8755](https://issues.apache.org/jira/browse/HADOOP-8755) | Print thread dump when tests fail due to timeout | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-8736](https://issues.apache.org/jira/browse/HADOOP-8736) | Add Builder for building an RPC server | Major | ipc | Brandon Li | Brandon Li | +| [HADOOP-8712](https://issues.apache.org/jira/browse/HADOOP-8712) | Change default hadoop.security.group.mapping | Minor | security | Robert Parker | Robert Parker | +| [HADOOP-8561](https://issues.apache.org/jira/browse/HADOOP-8561) | Introduce HADOOP\_PROXY\_USER for secure impersonation in child hadoop client processes | Major | security | Luke Lu | Yu Gao | +| [HADOOP-7886](https://issues.apache.org/jira/browse/HADOOP-7886) | Add toString to FileStatus | Minor | . | Jakob Homan | SreeHari | +| [HADOOP-7688](https://issues.apache.org/jira/browse/HADOOP-7688) | When a servlet filter throws an exception in init(..), the Jetty server failed silently. | Major | . | Tsz Wo Nicholas Sze | Uma Maheswara Rao G | +| [HDFS-4393](https://issues.apache.org/jira/browse/HDFS-4393) | Empty request and responses in protocol translators can be static final members | Minor | . | Brandon Li | Brandon Li | +| [HDFS-4392](https://issues.apache.org/jira/browse/HDFS-4392) | Use NetUtils#getFreeSocketPort in MiniDFSCluster | Trivial | test | Andrew Purtell | Andrew Purtell | +| [HDFS-4381](https://issues.apache.org/jira/browse/HDFS-4381) | Document fsimage format details in FSImageFormat class javadoc | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-4375](https://issues.apache.org/jira/browse/HDFS-4375) | Use token request messages defined in hadoop common | Major | namenode, security | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4259](https://issues.apache.org/jira/browse/HDFS-4259) | Improve pipeline DN replacement failure message | Minor | hdfs-client | Harsh J | Harsh J | +| [HDFS-4231](https://issues.apache.org/jira/browse/HDFS-4231) | Introduce HAState for BackupNode | Major | ha, namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-4214](https://issues.apache.org/jira/browse/HDFS-4214) | OfflineEditsViewer should print out the offset at which it encountered an error | Trivial | tools | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4155](https://issues.apache.org/jira/browse/HDFS-4155) | libhdfs implementation of hsync API | Major | libhdfs | Liang Xie | Liang Xie | +| [HDFS-4153](https://issues.apache.org/jira/browse/HDFS-4153) | Add START\_MSG/SHUTDOWN\_MSG for JournalNode | Major | journal-node | Liang Xie | Liang Xie | +| [HDFS-4143](https://issues.apache.org/jira/browse/HDFS-4143) | Change INodeFile.blocks to private | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4121](https://issues.apache.org/jira/browse/HDFS-4121) | Add namespace declarations in hdfs .proto files for languages other than java | Minor | . | Binglin Chang | Binglin Chang | +| [HDFS-4110](https://issues.apache.org/jira/browse/HDFS-4110) | Refine JNStorage log | Trivial | journal-node | Liang Xie | Liang Xie | +| [HDFS-4088](https://issues.apache.org/jira/browse/HDFS-4088) | Remove "throws QuotaExceededException" from an INodeDirectoryWithQuota constructor | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4074](https://issues.apache.org/jira/browse/HDFS-4074) | Remove empty constructors for INode | Trivial | namenode | Brandon Li | Brandon Li | +| [HDFS-4073](https://issues.apache.org/jira/browse/HDFS-4073) | Two minor improvements to FSDirectory | Minor | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4068](https://issues.apache.org/jira/browse/HDFS-4068) | DatanodeID and DatanodeInfo member should be private | Minor | datanode | Eli Collins | Eli Collins | +| [HDFS-4058](https://issues.apache.org/jira/browse/HDFS-4058) | DirectoryScanner may fail with IOOB if the directory scanning threads return out of volume order | Major | datanode | Eli Collins | Eli Collins | +| [HDFS-4048](https://issues.apache.org/jira/browse/HDFS-4048) | Use ERROR instead of INFO for volume failure logs | Major | . | Stephen Chu | Stephen Chu | +| [HDFS-4041](https://issues.apache.org/jira/browse/HDFS-4041) | Hadoop HDFS Maven protoc calls must not depend on external sh script | Major | build | Chris Nauroth | Chris Nauroth | +| [HDFS-4037](https://issues.apache.org/jira/browse/HDFS-4037) | Rename the getReplication() method in BlockCollection to getBlockReplication() | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4014](https://issues.apache.org/jira/browse/HDFS-4014) | Fix warnings found by findbugs2 | Major | . | Eli Collins | Eli Collins | +| [HDFS-4008](https://issues.apache.org/jira/browse/HDFS-4008) | TestBalancerWithEncryptedTransfer needs a timeout | Minor | test | Eli Collins | Eli Collins | +| [HDFS-4000](https://issues.apache.org/jira/browse/HDFS-4000) | TestParallelLocalRead fails with "input ByteBuffers must be direct buffers" | Major | . | Eli Collins | Colin Patrick McCabe | +| [HDFS-3957](https://issues.apache.org/jira/browse/HDFS-3957) | Change MutableQuantiles to use a shared thread for rolling over metrics | Minor | . | Andrew Wang | Andrew Wang | +| [HDFS-3939](https://issues.apache.org/jira/browse/HDFS-3939) | NN RPC address cleanup | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-3925](https://issues.apache.org/jira/browse/HDFS-3925) | Prettify PipelineAck#toString() for printing to a log | Minor | . | Andrew Wang | Andrew Wang | +| [HDFS-3910](https://issues.apache.org/jira/browse/HDFS-3910) | DFSTestUtil#waitReplication should timeout | Minor | test | Eli Collins | Eli Collins | +| [HDFS-3896](https://issues.apache.org/jira/browse/HDFS-3896) | Add descriptions for dfs.namenode.rpc-address and dfs.namenode.servicerpc-address to hdfs-default.xml | Minor | . | Jeff Lord | Jeff Lord | +| [HDFS-3813](https://issues.apache.org/jira/browse/HDFS-3813) | Log error message if security and WebHDFS are enabled but principal/keytab are not configured | Major | security, webhdfs | Stephen Chu | Stephen Chu | +| [HDFS-3703](https://issues.apache.org/jira/browse/HDFS-3703) | Decrease the datanode failure detection time | Major | datanode, namenode | Nicolas Liochon | Jing Zhao | +| [HDFS-3682](https://issues.apache.org/jira/browse/HDFS-3682) | MiniDFSCluster#init should provide more info when it fails | Minor | test | Eli Collins | Todd Lipcon | +| [HDFS-3680](https://issues.apache.org/jira/browse/HDFS-3680) | Allow customized audit logging in HDFS FSNamesystem | Minor | namenode | Marcelo Vanzin | Marcelo Vanzin | +| [HDFS-3483](https://issues.apache.org/jira/browse/HDFS-3483) | Better error message when hdfs fsck is run against a ViewFS config | Major | . | Stephen Chu | Stephen Fritz | +| [HDFS-2946](https://issues.apache.org/jira/browse/HDFS-2946) | HA: Put a cap on the number of completed edits files retained by the NN | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-2656](https://issues.apache.org/jira/browse/HDFS-2656) | Implement a pure c client based on webhdfs | Major | webhdfs | Zhanwei Wang | Jing Zhao | +| [MAPREDUCE-4977](https://issues.apache.org/jira/browse/MAPREDUCE-4977) | Documentation for pluggable shuffle and pluggable sort | Major | documentation | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4971](https://issues.apache.org/jira/browse/MAPREDUCE-4971) | Minor extensibility enhancements | Minor | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4949](https://issues.apache.org/jira/browse/MAPREDUCE-4949) | Enable multiple pi jobs to run in parallel | Minor | examples | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4907](https://issues.apache.org/jira/browse/MAPREDUCE-4907) | TrackerDistributedCacheManager issues too many getFileStatus calls | Major | mrv1, tasktracker | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4899](https://issues.apache.org/jira/browse/MAPREDUCE-4899) | Provide a plugin to the Yarn Web App Proxy to generate tracking links for M/R appllications given the ID | Major | . | Derek Dagit | Derek Dagit | +| [MAPREDUCE-4845](https://issues.apache.org/jira/browse/MAPREDUCE-4845) | ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2 | Major | client | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4838](https://issues.apache.org/jira/browse/MAPREDUCE-4838) | Add extra info to JH files | Major | . | Arun C Murthy | Zhijie Shen | +| [MAPREDUCE-4822](https://issues.apache.org/jira/browse/MAPREDUCE-4822) | Unnecessary conversions in History Events | Trivial | jobhistoryserver | Robert Joseph Evans | Chu Tong | +| [MAPREDUCE-4811](https://issues.apache.org/jira/browse/MAPREDUCE-4811) | JobHistoryServer should show when it was started in WebUI About page | Minor | jobhistoryserver, mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4810](https://issues.apache.org/jira/browse/MAPREDUCE-4810) | Add admin command options for ApplicationMaster | Minor | applicationmaster | Jason Lowe | Jerry Chen | +| [MAPREDUCE-4802](https://issues.apache.org/jira/browse/MAPREDUCE-4802) | Takes a long time to load the task list on the AM for large jobs | Major | mr-am, mrv2, webapps | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4777](https://issues.apache.org/jira/browse/MAPREDUCE-4777) | In TestIFile, testIFileReaderWithCodec relies on testIFileWriterWithCodec | Minor | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4764](https://issues.apache.org/jira/browse/MAPREDUCE-4764) | repair test org.apache.hadoop.mapreduce.security.TestBinaryTokenFile | Major | . | Ivan A. Veselovsky | | +| [MAPREDUCE-4763](https://issues.apache.org/jira/browse/MAPREDUCE-4763) | repair test org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken | Minor | . | Ivan A. Veselovsky | | +| [MAPREDUCE-4752](https://issues.apache.org/jira/browse/MAPREDUCE-4752) | Reduce MR AM memory usage through String Interning | Major | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4736](https://issues.apache.org/jira/browse/MAPREDUCE-4736) | Remove obsolete option [-rootDir] from TestDFSIO | Trivial | test | Brandon Li | Brandon Li | +| [MAPREDUCE-4723](https://issues.apache.org/jira/browse/MAPREDUCE-4723) | Fix warnings found by findbugs 2 | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4703](https://issues.apache.org/jira/browse/MAPREDUCE-4703) | Add the ability to start the MiniMRClientCluster using the configurations used before it is being stopped. | Major | mrv1, mrv2, test | Ahmed Radwan | Ahmed Radwan | +| [MAPREDUCE-4666](https://issues.apache.org/jira/browse/MAPREDUCE-4666) | JVM metrics for history server | Minor | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4616](https://issues.apache.org/jira/browse/MAPREDUCE-4616) | Improvement to MultipleOutputs javadocs | Minor | documentation | Tony Burton | Tony Burton | +| [MAPREDUCE-4517](https://issues.apache.org/jira/browse/MAPREDUCE-4517) | Too many INFO messages written out during AM to RM heartbeat | Minor | applicationmaster | James Kinley | Jason Lowe | +| [MAPREDUCE-4458](https://issues.apache.org/jira/browse/MAPREDUCE-4458) | Warn if java.library.path is used for AM or Task | Major | mrv2 | Robert Joseph Evans | Robert Parker | +| [MAPREDUCE-4229](https://issues.apache.org/jira/browse/MAPREDUCE-4229) | Counter names' memory usage can be decreased by interning | Major | jobtracker | Todd Lipcon | Miomir Boljanovic | +| [YARN-331](https://issues.apache.org/jira/browse/YARN-331) | Fill in missing fair scheduler documentation | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-328](https://issues.apache.org/jira/browse/YARN-328) | Use token request messages defined in hadoop common | Major | resourcemanager | Suresh Srinivas | Suresh Srinivas | +| [YARN-315](https://issues.apache.org/jira/browse/YARN-315) | Use security token protobuf definition from hadoop common | Major | . | Suresh Srinivas | Suresh Srinivas | +| [YARN-285](https://issues.apache.org/jira/browse/YARN-285) | RM should be able to provide a tracking link for apps that have already been purged | Major | . | Derek Dagit | Derek Dagit | +| [YARN-277](https://issues.apache.org/jira/browse/YARN-277) | Use AMRMClient in DistributedShell to exemplify the approach | Major | . | Bikas Saha | Bikas Saha | +| [YARN-254](https://issues.apache.org/jira/browse/YARN-254) | Update fair scheduler web UI for hierarchical queues | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-222](https://issues.apache.org/jira/browse/YARN-222) | Fair scheduler should create queue for each user by default | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-216](https://issues.apache.org/jira/browse/YARN-216) | Remove jquery theming support | Major | . | Todd Lipcon | Robert Joseph Evans | +| [YARN-184](https://issues.apache.org/jira/browse/YARN-184) | Remove unnecessary locking in fair scheduler, and address findbugs excludes. | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-183](https://issues.apache.org/jira/browse/YARN-183) | Clean up fair scheduler code | Minor | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-169](https://issues.apache.org/jira/browse/YARN-169) | Update log4j.appender.EventCounter to use org.apache.hadoop.log.metrics.EventCounter | Minor | nodemanager | Anthony Rojas | Anthony Rojas | +| [YARN-165](https://issues.apache.org/jira/browse/YARN-165) | RM should point tracking URL to RM web page for app when AM fails | Blocker | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-129](https://issues.apache.org/jira/browse/YARN-129) | Simplify classpath construction for mini YARN tests | Major | client | Tom White | Tom White | +| [YARN-57](https://issues.apache.org/jira/browse/YARN-57) | Plugable process tree | Major | nodemanager | Radim Kolar | Radim Kolar | +| [YARN-23](https://issues.apache.org/jira/browse/YARN-23) | FairScheduler: FSQueueSchedulable#updateDemand() - potential redundant aggregation | Major | scheduler | Karthik Kambatla | Karthik Kambatla | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9289](https://issues.apache.org/jira/browse/HADOOP-9289) | FsShell rm -f fails for non-matching globs | Blocker | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-9278](https://issues.apache.org/jira/browse/HADOOP-9278) | HarFileSystem may leak file handle | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9260](https://issues.apache.org/jira/browse/HADOOP-9260) | Hadoop version may be not correct when starting name node or data node | Critical | . | Jerry Chen | Chris Nauroth | +| [HADOOP-9255](https://issues.apache.org/jira/browse/HADOOP-9255) | relnotes.py missing last jira | Critical | scripts | Thomas Graves | Thomas Graves | +| [HADOOP-9252](https://issues.apache.org/jira/browse/HADOOP-9252) | StringUtils.humanReadableInt(..) has a race condition | Minor | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9231](https://issues.apache.org/jira/browse/HADOOP-9231) | Parametrize staging URL for the uniformity of distributionManagement | Major | build | Konstantin Boudnik | Konstantin Boudnik | +| [HADOOP-9221](https://issues.apache.org/jira/browse/HADOOP-9221) | Convert remaining xdocs to APT | Major | . | Andy Isaacson | Andy Isaacson | +| [HADOOP-9215](https://issues.apache.org/jira/browse/HADOOP-9215) | when using cmake-2.6, libhadoop.so doesn't get created (only libhadoop.so.1.0.0) | Blocker | . | Thomas Graves | Colin Patrick McCabe | +| [HADOOP-9212](https://issues.apache.org/jira/browse/HADOOP-9212) | Potential deadlock in FileSystem.Cache/IPC/UGI | Major | fs | Tom White | Tom White | +| [HADOOP-9203](https://issues.apache.org/jira/browse/HADOOP-9203) | RPCCallBenchmark should find a random available port | Trivial | ipc, test | Andrew Purtell | Andrew Purtell | +| [HADOOP-9193](https://issues.apache.org/jira/browse/HADOOP-9193) | hadoop script can inadvertently expand wildcard arguments when delegating to hdfs script | Minor | scripts | Jason Lowe | Andy Isaacson | +| [HADOOP-9190](https://issues.apache.org/jira/browse/HADOOP-9190) | packaging docs is broken | Major | documentation | Thomas Graves | Andy Isaacson | +| [HADOOP-9183](https://issues.apache.org/jira/browse/HADOOP-9183) | Potential deadlock in ActiveStandbyElector | Major | ha | Tom White | Tom White | +| [HADOOP-9181](https://issues.apache.org/jira/browse/HADOOP-9181) | Set daemon flag for HttpServer's QueuedThreadPool | Major | . | Liang Xie | Liang Xie | +| [HADOOP-9178](https://issues.apache.org/jira/browse/HADOOP-9178) | src/main/conf is missing hadoop-policy.xml | Minor | . | Sandy Ryza | Sandy Ryza | +| [HADOOP-9173](https://issues.apache.org/jira/browse/HADOOP-9173) | Add security token protobuf definition to common and use it in hdfs | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9155](https://issues.apache.org/jira/browse/HADOOP-9155) | FsPermission should have different default value, 777 for directory and 666 for file | Minor | . | Binglin Chang | Binglin Chang | +| [HADOOP-9152](https://issues.apache.org/jira/browse/HADOOP-9152) | HDFS can report negative DFS Used on clusters with very small amounts of data | Minor | fs | Brock Noland | Brock Noland | +| [HADOOP-9135](https://issues.apache.org/jira/browse/HADOOP-9135) | JniBasedUnixGroupsMappingWithFallback should log at debug rather than info during fallback | Trivial | security | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9124](https://issues.apache.org/jira/browse/HADOOP-9124) | SortedMapWritable violates contract of Map interface for equals() and hashCode() | Minor | io | Patrick Hunt | Surenkumar Nihalani | +| [HADOOP-9113](https://issues.apache.org/jira/browse/HADOOP-9113) | o.a.h.fs.TestDelegationTokenRenewer is failing intermittently | Major | security, test | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9105](https://issues.apache.org/jira/browse/HADOOP-9105) | FsShell -moveFromLocal erroneously fails | Major | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-9103](https://issues.apache.org/jira/browse/HADOOP-9103) | UTF8 class does not properly decode Unicode characters outside the basic multilingual plane | Major | io | yixiaohua | Todd Lipcon | +| [HADOOP-9097](https://issues.apache.org/jira/browse/HADOOP-9097) | Maven RAT plugin is not checking all source files | Critical | build | Tom White | Thomas Graves | +| [HADOOP-9072](https://issues.apache.org/jira/browse/HADOOP-9072) | Hadoop-Common-0.23-Build Fails to build in Jenkins | Major | . | Robert Parker | Robert Parker | +| [HADOOP-9064](https://issues.apache.org/jira/browse/HADOOP-9064) | Augment DelegationTokenRenewer API to cancel the tokens on calls to removeRenewAction | Major | security | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9049](https://issues.apache.org/jira/browse/HADOOP-9049) | DelegationTokenRenewer needs to be Singleton and FileSystems should register/deregister to/from. | Major | security | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9041](https://issues.apache.org/jira/browse/HADOOP-9041) | FileSystem initialization can go into infinite loop | Critical | fs | Radim Kolar | Radim Kolar | +| [HADOOP-9025](https://issues.apache.org/jira/browse/HADOOP-9025) | org.apache.hadoop.tools.TestCopyListing failing | Major | . | Robert Joseph Evans | Jonathan Eagles | +| [HADOOP-9022](https://issues.apache.org/jira/browse/HADOOP-9022) | Hadoop distcp tool fails to copy file if -m 0 specified | Major | . | Haiyang Jiang | Jonathan Eagles | +| [HADOOP-8994](https://issues.apache.org/jira/browse/HADOOP-8994) | TestDFSShell creates file named "noFileHere", making further tests hard to understand | Minor | test | Andy Isaacson | Andy Isaacson | +| [HADOOP-8986](https://issues.apache.org/jira/browse/HADOOP-8986) | Server$Call object is never released after it is sent | Critical | ipc | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8981](https://issues.apache.org/jira/browse/HADOOP-8981) | TestMetricsSystemImpl fails on Windows | Major | metrics | Chris Nauroth | Xuan Gong | +| [HADOOP-8962](https://issues.apache.org/jira/browse/HADOOP-8962) | RawLocalFileSystem.listStatus fails when a child filename contains a colon | Critical | fs | Jason Lowe | Jason Lowe | +| [HADOOP-8948](https://issues.apache.org/jira/browse/HADOOP-8948) | TestFileUtil.testGetDU fails on Windows due to incorrect assumption of line separator | Major | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-8913](https://issues.apache.org/jira/browse/HADOOP-8913) | hadoop-metrics2.properties should give units in comment for sampling period | Minor | metrics | Sandy Ryza | Sandy Ryza | +| [HADOOP-8912](https://issues.apache.org/jira/browse/HADOOP-8912) | adding .gitattributes file to prevent CRLF and LF mismatches for source and text files | Major | build | Raja Aluri | Raja Aluri | +| [HADOOP-8911](https://issues.apache.org/jira/browse/HADOOP-8911) | CRLF characters in source and text files | Major | build | Raja Aluri | Raja Aluri | +| [HADOOP-8906](https://issues.apache.org/jira/browse/HADOOP-8906) | paths with multiple globs are unreliable | Critical | fs | Daryn Sharp | Daryn Sharp | +| [HADOOP-8901](https://issues.apache.org/jira/browse/HADOOP-8901) | GZip and Snappy support may not work without unversioned libraries | Minor | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8900](https://issues.apache.org/jira/browse/HADOOP-8900) | BuiltInGzipDecompressor throws IOException - stored gzip size doesn't match decompressed size | Major | . | Slavik Krassovsky | Andy Isaacson | +| [HADOOP-8883](https://issues.apache.org/jira/browse/HADOOP-8883) | Anonymous fallback in KerberosAuthenticator is broken | Major | . | Robert Kanter | Robert Kanter | +| [HADOOP-8881](https://issues.apache.org/jira/browse/HADOOP-8881) | FileBasedKeyStoresFactory initialization logging should be debug not info | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8878](https://issues.apache.org/jira/browse/HADOOP-8878) | uppercase namenode hostname causes hadoop dfs calls with webhdfs filesystem and fsck to fail when security is on | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8855](https://issues.apache.org/jira/browse/HADOOP-8855) | SSL-based image transfer does not work when Kerberos is disabled | Minor | security | Todd Lipcon | Todd Lipcon | +| [HADOOP-8843](https://issues.apache.org/jira/browse/HADOOP-8843) | Old trash directories are never deleted on upgrade from 1.x | Critical | . | Robert Joseph Evans | Jason Lowe | +| [HADOOP-8833](https://issues.apache.org/jira/browse/HADOOP-8833) | fs -text should make sure to call inputstream.seek(0) before using input stream | Major | fs | Harsh J | Harsh J | +| [HADOOP-8822](https://issues.apache.org/jira/browse/HADOOP-8822) | relnotes.py was deleted post mavenization | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-8819](https://issues.apache.org/jira/browse/HADOOP-8819) | Should use && instead of & in a few places in FTPFileSystem,FTPInputStream,S3InputStream,ViewFileSystem,ViewFs | Major | fs | Brandon Li | Brandon Li | +| [HADOOP-8816](https://issues.apache.org/jira/browse/HADOOP-8816) | HTTP Error 413 full HEAD if using kerberos authentication | Major | net | Moritz Moeller | Moritz Moeller | +| [HADOOP-8811](https://issues.apache.org/jira/browse/HADOOP-8811) | Compile hadoop native library in FreeBSD | Critical | native | Radim Kolar | Radim Kolar | +| [HADOOP-8795](https://issues.apache.org/jira/browse/HADOOP-8795) | BASH tab completion doesn't look in PATH, assumes path to executable is specified | Minor | scripts | Sean Mackrory | Sean Mackrory | +| [HADOOP-8791](https://issues.apache.org/jira/browse/HADOOP-8791) | rm "Only deletes non empty directory and files." | Major | documentation | Bertrand Dechoux | Jing Zhao | +| [HADOOP-8786](https://issues.apache.org/jira/browse/HADOOP-8786) | HttpServer continues to start even if AuthenticationFilter fails to init | Major | . | Todd Lipcon | Todd Lipcon | +| [HADOOP-8780](https://issues.apache.org/jira/browse/HADOOP-8780) | Update DeprecatedProperties apt file | Major | . | Ahmed Radwan | Ahmed Radwan | +| [HADOOP-8756](https://issues.apache.org/jira/browse/HADOOP-8756) | Fix SEGV when libsnappy is in java.library.path but not LD\_LIBRARY\_PATH | Minor | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8713](https://issues.apache.org/jira/browse/HADOOP-8713) | TestRPCCompatibility fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HADOOP-8684](https://issues.apache.org/jira/browse/HADOOP-8684) | Deadlock between WritableComparator and WritableComparable | Minor | io | Hiroshi Ikeda | Jing Zhao | +| [HADOOP-8616](https://issues.apache.org/jira/browse/HADOOP-8616) | ViewFS configuration requires a trailing slash | Major | viewfs | Eli Collins | Sandy Ryza | +| [HADOOP-8589](https://issues.apache.org/jira/browse/HADOOP-8589) | ViewFs tests fail when tests and home dirs are nested | Major | fs, test | Andrey Klochkov | Sanjay Radia | +| [HADOOP-8418](https://issues.apache.org/jira/browse/HADOOP-8418) | Fix UGI for IBM JDK running on Windows | Major | security | Luke Lu | Yu Gao | +| [HADOOP-7294](https://issues.apache.org/jira/browse/HADOOP-7294) | FileUtil uses wrong stat command for FreeBSD | Major | fs | Vitalii Tymchyshyn | | +| [HADOOP-7115](https://issues.apache.org/jira/browse/HADOOP-7115) | Add a cache for getpwuid\_r and getpwgid\_r calls | Major | . | Arun C Murthy | Alejandro Abdelnur | +| [HADOOP-6762](https://issues.apache.org/jira/browse/HADOOP-6762) | exception while doing RPC I/O closes channel | Critical | . | sam rash | sam rash | +| [HADOOP-6607](https://issues.apache.org/jira/browse/HADOOP-6607) | Add different variants of non caching HTTP headers | Minor | io | Steve Loughran | Alejandro Abdelnur | +| [HDFS-4468](https://issues.apache.org/jira/browse/HDFS-4468) | Fix TestHDFSCLI and TestQuota for HADOOP-9252 | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4462](https://issues.apache.org/jira/browse/HDFS-4462) | 2NN will fail to checkpoint after an HDFS upgrade from a pre-federation version of HDFS | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4458](https://issues.apache.org/jira/browse/HDFS-4458) | start balancer failed with "Failed to create file [/system/balancer.id]" if configure IP on fs.defaultFS | Major | balancer & mover | Wenwu Peng | Binglin Chang | +| [HDFS-4452](https://issues.apache.org/jira/browse/HDFS-4452) | getAdditionalBlock() can create multiple blocks if the client times out and retries. | Critical | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-4444](https://issues.apache.org/jira/browse/HDFS-4444) | Add space between total transaction time and number of transactions in FSEditLog#printStatistics | Trivial | . | Stephen Chu | Stephen Chu | +| [HDFS-4443](https://issues.apache.org/jira/browse/HDFS-4443) | Remove trailing '`' character from HDFS nodelist jsp | Trivial | namenode | Christian Rohling | Christian Rohling | +| [HDFS-4428](https://issues.apache.org/jira/browse/HDFS-4428) | FsDatasetImpl should disclose what the error is when a rename fails | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4426](https://issues.apache.org/jira/browse/HDFS-4426) | Secondary namenode shuts down immediately after startup | Blocker | namenode | Jason Lowe | Arpit Agarwal | +| [HDFS-4415](https://issues.apache.org/jira/browse/HDFS-4415) | HostnameFilter should handle hostname resolution failures and continue processing | Major | . | Robert Kanter | Robert Kanter | +| [HDFS-4404](https://issues.apache.org/jira/browse/HDFS-4404) | Create file failure when the machine of first attempted NameNode is down | Critical | ha, hdfs-client | liaowenrui | Todd Lipcon | +| [HDFS-4403](https://issues.apache.org/jira/browse/HDFS-4403) | DFSClient can infer checksum type when not provided by reading first byte | Minor | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-4385](https://issues.apache.org/jira/browse/HDFS-4385) | Maven RAT plugin is not checking all source files | Critical | build | Thomas Graves | Thomas Graves | +| [HDFS-4384](https://issues.apache.org/jira/browse/HDFS-4384) | test\_libhdfs\_threaded gets SEGV if JNIEnv cannot be initialized | Minor | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4377](https://issues.apache.org/jira/browse/HDFS-4377) | Some trivial DN comment cleanup | Trivial | . | Eli Collins | Eli Collins | +| [HDFS-4363](https://issues.apache.org/jira/browse/HDFS-4363) | Combine PBHelper and HdfsProtoUtil and remove redundant methods | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4359](https://issues.apache.org/jira/browse/HDFS-4359) | remove an unnecessary synchronized keyword in BPOfferService.java | Major | datanode | Liang Xie | Liang Xie | +| [HDFS-4351](https://issues.apache.org/jira/browse/HDFS-4351) | Fix BlockPlacementPolicyDefault#chooseTarget when avoiding stale nodes | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4347](https://issues.apache.org/jira/browse/HDFS-4347) | TestBackupNode can go into infinite loop "Waiting checkpoint to complete." | Major | namenode, test | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-4344](https://issues.apache.org/jira/browse/HDFS-4344) | dfshealth.jsp throws NumberFormatException when dfs.hosts/dfs.hosts.exclude includes port number | Major | namenode | tamtam180 | Andy Isaacson | +| [HDFS-4315](https://issues.apache.org/jira/browse/HDFS-4315) | DNs with multiple BPs can have BPOfferServices fail to start due to unsynchronized map access | Major | datanode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4308](https://issues.apache.org/jira/browse/HDFS-4308) | addBlock() should persist file blocks once | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-4307](https://issues.apache.org/jira/browse/HDFS-4307) | SocketCache should use monotonic time | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4306](https://issues.apache.org/jira/browse/HDFS-4306) | PBHelper.convertLocatedBlock miss convert BlockToken | Major | . | Binglin Chang | Binglin Chang | +| [HDFS-4302](https://issues.apache.org/jira/browse/HDFS-4302) | Precondition in EditLogFileInputStream's length() method is checked too early in NameNode startup, causing fatal exception | Major | ha, namenode | Eugene Koontz | Eugene Koontz | +| [HDFS-4295](https://issues.apache.org/jira/browse/HDFS-4295) | Using port 1023 should be valid when starting Secure DataNode | Major | security | Stephen Chu | Stephen Chu | +| [HDFS-4294](https://issues.apache.org/jira/browse/HDFS-4294) | Backwards compatibility is not maintained for TestVolumeId | Major | . | Robert Parker | Robert Parker | +| [HDFS-4292](https://issues.apache.org/jira/browse/HDFS-4292) | Sanity check not correct in RemoteBlockReader2.newBlockReader | Minor | . | Binglin Chang | Binglin Chang | +| [HDFS-4291](https://issues.apache.org/jira/browse/HDFS-4291) | edit log unit tests leave stray test\_edit\_log\_file around | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4288](https://issues.apache.org/jira/browse/HDFS-4288) | NN accepts incremental BR as IBR in safemode | Critical | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4282](https://issues.apache.org/jira/browse/HDFS-4282) | TestEditLog.testFuzzSequences FAILED in all pre-commit test | Major | namenode, test | Junping Du | Todd Lipcon | +| [HDFS-4279](https://issues.apache.org/jira/browse/HDFS-4279) | NameNode does not initialize generic conf keys when started with -recover | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4274](https://issues.apache.org/jira/browse/HDFS-4274) | BlockPoolSliceScanner does not close verification log during shutdown | Minor | datanode | Chris Nauroth | Chris Nauroth | +| [HDFS-4270](https://issues.apache.org/jira/browse/HDFS-4270) | Replications of the highest priority should be allowed to choose a source datanode that has reached its max replication limit | Minor | namenode | Derek Dagit | Derek Dagit | +| [HDFS-4268](https://issues.apache.org/jira/browse/HDFS-4268) | Remove redundant enum NNHAStatusHeartbeat.State | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-4242](https://issues.apache.org/jira/browse/HDFS-4242) | Map.Entry is incorrectly used in LeaseManager | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4238](https://issues.apache.org/jira/browse/HDFS-4238) | [HA] Standby namenode should not do purging of shared storage edits. | Major | ha | Vinayakumar B | Todd Lipcon | +| [HDFS-4236](https://issues.apache.org/jira/browse/HDFS-4236) | Regression: HDFS-4171 puts artificial limit on username length | Blocker | . | Allen Wittenauer | Alejandro Abdelnur | +| [HDFS-4232](https://issues.apache.org/jira/browse/HDFS-4232) | NN fails to write a fsimage with stale leases | Blocker | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4216](https://issues.apache.org/jira/browse/HDFS-4216) | Adding symlink should not ignore QuotaExceededException | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4186](https://issues.apache.org/jira/browse/HDFS-4186) | logSync() is called with the write lock held while releasing lease | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4182](https://issues.apache.org/jira/browse/HDFS-4182) | SecondaryNameNode leaks NameCache entries | Critical | namenode | Todd Lipcon | Robert Joseph Evans | +| [HDFS-4181](https://issues.apache.org/jira/browse/HDFS-4181) | LeaseManager tries to double remove and prints extra messages | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4179](https://issues.apache.org/jira/browse/HDFS-4179) | BackupNode: allow reads, fix checkpointing, safeMode | Major | namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-4178](https://issues.apache.org/jira/browse/HDFS-4178) | shell scripts should not close stderr | Major | scripts | Andy Isaacson | Andy Isaacson | +| [HDFS-4172](https://issues.apache.org/jira/browse/HDFS-4172) | namenode does not URI-encode parameters when building URI for datanode request | Minor | namenode | Derek Dagit | Derek Dagit | +| [HDFS-4171](https://issues.apache.org/jira/browse/HDFS-4171) | WebHDFS and HttpFs should accept only valid Unix user names | Major | . | Harsh J | Alejandro Abdelnur | +| [HDFS-4164](https://issues.apache.org/jira/browse/HDFS-4164) | fuse\_dfs: add -lrt to the compiler command line on Linux | Minor | fuse-dfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4162](https://issues.apache.org/jira/browse/HDFS-4162) | Some malformed and unquoted HTML strings are returned from datanode web ui | Minor | datanode | Derek Dagit | Derek Dagit | +| [HDFS-4156](https://issues.apache.org/jira/browse/HDFS-4156) | Seeking to a negative position should throw an IOE | Major | . | Eli Collins | Eli Reisman | +| [HDFS-4140](https://issues.apache.org/jira/browse/HDFS-4140) | fuse-dfs handles open(O\_TRUNC) poorly | Major | fuse-dfs | Andy Isaacson | Colin Patrick McCabe | +| [HDFS-4139](https://issues.apache.org/jira/browse/HDFS-4139) | fuse-dfs RO mode still allows file truncation | Major | fuse-dfs | Andy Isaacson | Colin Patrick McCabe | +| [HDFS-4132](https://issues.apache.org/jira/browse/HDFS-4132) | when libwebhdfs is not enabled, nativeMiniDfsClient frees uninitialized memory | Major | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4127](https://issues.apache.org/jira/browse/HDFS-4127) | Log message is not correct in case of short of replica | Minor | namenode | Junping Du | Junping Du | +| [HDFS-4112](https://issues.apache.org/jira/browse/HDFS-4112) | A few improvements on INodeDirectory | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4107](https://issues.apache.org/jira/browse/HDFS-4107) | Add utility methods to cast INode to INodeFile and INodeFileUnderConstruction | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4106](https://issues.apache.org/jira/browse/HDFS-4106) | BPServiceActor#lastHeartbeat, lastBlockReport and lastDeletedReport should be declared as volatile | Minor | namenode, test | Jing Zhao | Jing Zhao | +| [HDFS-4105](https://issues.apache.org/jira/browse/HDFS-4105) | the SPNEGO user for secondary namenode should use the web keytab | Major | . | Arpit Gupta | Arpit Gupta | +| [HDFS-4104](https://issues.apache.org/jira/browse/HDFS-4104) | dfs -test -d prints inappropriate error on nonexistent directory | Minor | . | Andy Isaacson | Andy Isaacson | +| [HDFS-4099](https://issues.apache.org/jira/browse/HDFS-4099) | Clean up replication code and add more javadoc | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4090](https://issues.apache.org/jira/browse/HDFS-4090) | getFileChecksum() result incompatible when called against zero-byte files. | Critical | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-4075](https://issues.apache.org/jira/browse/HDFS-4075) | Reduce recommissioning overhead | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4072](https://issues.apache.org/jira/browse/HDFS-4072) | On file deletion remove corresponding blocks pending replication | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-4061](https://issues.apache.org/jira/browse/HDFS-4061) | TestBalancer and TestUnderReplicatedBlocks need timeouts | Major | . | Eli Collins | Eli Collins | +| [HDFS-4055](https://issues.apache.org/jira/browse/HDFS-4055) | TestAuditLogs is flaky | Major | . | Binglin Chang | Binglin Chang | +| [HDFS-4049](https://issues.apache.org/jira/browse/HDFS-4049) | hflush performance regression due to nagling delays | Critical | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-4046](https://issues.apache.org/jira/browse/HDFS-4046) | ChecksumTypeProto use NULL as enum value which is illegal in C/C++ | Minor | datanode, namenode | Binglin Chang | Binglin Chang | +| [HDFS-4044](https://issues.apache.org/jira/browse/HDFS-4044) | Duplicate ChecksumType definition in HDFS .proto files | Major | datanode | Binglin Chang | Binglin Chang | +| [HDFS-4036](https://issues.apache.org/jira/browse/HDFS-4036) | FSDirectory.unprotectedAddFile(..) should not throw UnresolvedLinkException | Major | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4022](https://issues.apache.org/jira/browse/HDFS-4022) | Replication not happening for appended block | Blocker | . | suja s | Vinayakumar B | +| [HDFS-4021](https://issues.apache.org/jira/browse/HDFS-4021) | Misleading error message when resources are low on the NameNode | Minor | namenode | Colin Patrick McCabe | Christopher Conner | +| [HDFS-4020](https://issues.apache.org/jira/browse/HDFS-4020) | TestRBWBlockInvalidation may time out | Major | . | Eli Collins | Eli Collins | +| [HDFS-4018](https://issues.apache.org/jira/browse/HDFS-4018) | TestDataNodeMultipleRegistrations#testMiniDFSClusterWithMultipleNN is missing some cluster cleanup | Minor | . | Eli Collins | Eli Collins | +| [HDFS-4006](https://issues.apache.org/jira/browse/HDFS-4006) | TestCheckpoint#testSecondaryHasVeryOutOfDateImage occasionally fails due to unexpected exit | Major | namenode | Eli Collins | Todd Lipcon | +| [HDFS-3999](https://issues.apache.org/jira/browse/HDFS-3999) | HttpFS OPEN operation expects len parameter, it should be length | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3997](https://issues.apache.org/jira/browse/HDFS-3997) | OfflineImageViewer incorrectly passes value of imageVersion when visiting IS\_COMPRESSED element | Trivial | namenode | Mithun Radhakrishnan | Mithun Radhakrishnan | +| [HDFS-3996](https://issues.apache.org/jira/browse/HDFS-3996) | Add debug log removed in HDFS-3873 back | Minor | . | Eli Collins | Eli Collins | +| [HDFS-3992](https://issues.apache.org/jira/browse/HDFS-3992) | Method org.apache.hadoop.hdfs.TestHftpFileSystem.tearDown() sometimes throws NPEs | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HDFS-3990](https://issues.apache.org/jira/browse/HDFS-3990) | NN's health report has severe performance problems | Critical | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-3985](https://issues.apache.org/jira/browse/HDFS-3985) | Add timeouts to TestMulitipleNNDataBlockScanner | Major | test | Eli Collins | | +| [HDFS-3979](https://issues.apache.org/jira/browse/HDFS-3979) | Fix hsync semantics | Major | datanode | Lars Hofhansl | Lars Hofhansl | +| [HDFS-3970](https://issues.apache.org/jira/browse/HDFS-3970) | BlockPoolSliceStorage#doRollback(..) should use BlockPoolSliceStorage instead of DataStorage to read prev version file. | Major | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-3964](https://issues.apache.org/jira/browse/HDFS-3964) | Make NN log of fs.defaultFS debug rather than info | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-3951](https://issues.apache.org/jira/browse/HDFS-3951) | datanode web ui does not work over HTTPS when datanode is started in secure mode | Major | datanode, security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3949](https://issues.apache.org/jira/browse/HDFS-3949) | NameNodeRpcServer#join should join on both client and server RPC servers | Minor | namenode | Eli Collins | Eli Collins | +| [HDFS-3948](https://issues.apache.org/jira/browse/HDFS-3948) | TestWebHDFS#testNamenodeRestart occasionally fails | Minor | test | Eli Collins | Jing Zhao | +| [HDFS-3938](https://issues.apache.org/jira/browse/HDFS-3938) | remove current limitations from HttpFS docs | Major | documentation | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3936](https://issues.apache.org/jira/browse/HDFS-3936) | MiniDFSCluster shutdown races with BlocksMap usage | Major | . | Eli Collins | Eli Collins | +| [HDFS-3932](https://issues.apache.org/jira/browse/HDFS-3932) | NameNode Web UI broken if the rpc-address is set to the wildcard | Major | . | Eli Collins | Eli Collins | +| [HDFS-3931](https://issues.apache.org/jira/browse/HDFS-3931) | TestDatanodeBlockScanner#testBlockCorruptionPolicy2 is broken | Minor | test | Eli Collins | Andy Isaacson | +| [HDFS-3924](https://issues.apache.org/jira/browse/HDFS-3924) | Multi-byte id in HdfsVolumeId | Major | hdfs-client | Andrew Wang | Andrew Wang | +| [HDFS-3921](https://issues.apache.org/jira/browse/HDFS-3921) | NN will prematurely consider blocks missing when entering active state while still in safe mode | Major | . | Stephen Chu | Aaron T. Myers | +| [HDFS-3919](https://issues.apache.org/jira/browse/HDFS-3919) | MiniDFSCluster:waitClusterUp can hang forever | Minor | test | Andy Isaacson | Andy Isaacson | +| [HDFS-3916](https://issues.apache.org/jira/browse/HDFS-3916) | libwebhdfs (C client) code cleanups | Major | webhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3831](https://issues.apache.org/jira/browse/HDFS-3831) | Failure to renew tokens due to test-sources left in classpath | Critical | security | Jason Lowe | Jason Lowe | +| [HDFS-3829](https://issues.apache.org/jira/browse/HDFS-3829) | TestHftpURLTimeouts fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HDFS-3824](https://issues.apache.org/jira/browse/HDFS-3824) | TestHftpDelegationToken fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HDFS-3804](https://issues.apache.org/jira/browse/HDFS-3804) | TestHftpFileSystem fails intermittently with JDK7 | Major | test | Trevor Robinson | Trevor Robinson | +| [HDFS-3753](https://issues.apache.org/jira/browse/HDFS-3753) | Tests don't run with native libraries | Major | build, test | Eli Collins | Colin Patrick McCabe | +| [HDFS-3678](https://issues.apache.org/jira/browse/HDFS-3678) | Edit log files are never being purged from 2NN | Critical | namenode | Todd Lipcon | Aaron T. Myers | +| [HDFS-3626](https://issues.apache.org/jira/browse/HDFS-3626) | Creating file with invalid path can corrupt edit log | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3625](https://issues.apache.org/jira/browse/HDFS-3625) | Fix TestBackupNode by properly initializing edit log | Blocker | ha | Eli Collins | Junping Du | +| [HDFS-3616](https://issues.apache.org/jira/browse/HDFS-3616) | TestWebHdfsWithMultipleNameNodes fails with ConcurrentModificationException in DN shutdown | Major | datanode | Uma Maheswara Rao G | Jing Zhao | +| [HDFS-3553](https://issues.apache.org/jira/browse/HDFS-3553) | Hftp proxy tokens are broken | Blocker | . | Daryn Sharp | Daryn Sharp | +| [HDFS-3510](https://issues.apache.org/jira/browse/HDFS-3510) | Improve FSEditLog pre-allocation | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-3507](https://issues.apache.org/jira/browse/HDFS-3507) | DFS#isInSafeMode needs to execute only on Active NameNode | Critical | ha | Vinayakumar B | Vinayakumar B | +| [HDFS-3429](https://issues.apache.org/jira/browse/HDFS-3429) | DataNode reads checksums even if client does not need them | Major | datanode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-3373](https://issues.apache.org/jira/browse/HDFS-3373) | FileContext HDFS implementation can leak socket caches | Major | hdfs-client | Todd Lipcon | John George | +| [HDFS-3224](https://issues.apache.org/jira/browse/HDFS-3224) | Bug in check for DN re-registration with different storage ID | Minor | . | Eli Collins | Jason Lowe | +| [HDFS-2264](https://issues.apache.org/jira/browse/HDFS-2264) | NamenodeProtocol has the wrong value for clientPrincipal in KerberosInfo annotation | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-1322](https://issues.apache.org/jira/browse/HDFS-1322) | Document umask in DistributedFileSystem#mkdirs javadocs | Major | . | Ravi Gummadi | Colin Patrick McCabe | +| [MAPREDUCE-4969](https://issues.apache.org/jira/browse/MAPREDUCE-4969) | TestKeyValueTextInputFormat test fails with Open JDK 7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [MAPREDUCE-4953](https://issues.apache.org/jira/browse/MAPREDUCE-4953) | HadoopPipes misuses fprintf | Major | pipes | Andy Isaacson | Andy Isaacson | +| [MAPREDUCE-4948](https://issues.apache.org/jira/browse/MAPREDUCE-4948) | TestYARNRunner.testHistoryServerToken failed on trunk | Critical | client | Junping Du | Junping Du | +| [MAPREDUCE-4946](https://issues.apache.org/jira/browse/MAPREDUCE-4946) | Type conversion of map completion events leads to performance problems with large jobs | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4936](https://issues.apache.org/jira/browse/MAPREDUCE-4936) | JobImpl uber checks for cpu are wrong | Critical | mrv2 | Daryn Sharp | Arun C Murthy | +| [MAPREDUCE-4934](https://issues.apache.org/jira/browse/MAPREDUCE-4934) | Maven RAT plugin is not checking all source files | Critical | build | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4925](https://issues.apache.org/jira/browse/MAPREDUCE-4925) | The pentomino option parser may be buggy | Major | examples | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4924](https://issues.apache.org/jira/browse/MAPREDUCE-4924) | flakey test: org.apache.hadoop.mapred.TestClusterMRNotification.testMR | Trivial | mrv1 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4923](https://issues.apache.org/jira/browse/MAPREDUCE-4923) | Add toString method to TaggedInputSplit | Minor | mrv1, mrv2, task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4921](https://issues.apache.org/jira/browse/MAPREDUCE-4921) | JobClient should acquire HS token with RM principal | Blocker | client | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-4920](https://issues.apache.org/jira/browse/MAPREDUCE-4920) | Use security token protobuf definition from hadoop common | Major | . | Vinod Kumar Vavilapalli | Suresh Srinivas | +| [MAPREDUCE-4913](https://issues.apache.org/jira/browse/MAPREDUCE-4913) | TestMRAppMaster#testMRAppMasterMissingStaging occasionally exits | Major | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4902](https://issues.apache.org/jira/browse/MAPREDUCE-4902) | Fix typo "receievd" should be "received" in log output | Trivial | . | Albert Chu | Albert Chu | +| [MAPREDUCE-4895](https://issues.apache.org/jira/browse/MAPREDUCE-4895) | Fix compilation failure of org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulators | Major | . | Dennis Y | Dennis Y | +| [MAPREDUCE-4894](https://issues.apache.org/jira/browse/MAPREDUCE-4894) | Renewal / cancellation of JobHistory tokens | Blocker | jobhistoryserver, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4893](https://issues.apache.org/jira/browse/MAPREDUCE-4893) | MR AppMaster can do sub-optimal assignment of containers to map tasks leading to poor node locality | Major | applicationmaster | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4890](https://issues.apache.org/jira/browse/MAPREDUCE-4890) | Invalid TaskImpl state transitions when task fails while speculating | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4884](https://issues.apache.org/jira/browse/MAPREDUCE-4884) | streaming tests fail to start MiniMRCluster due to "Queue configuration missing child queue names for root" | Major | contrib/streaming, test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-4861](https://issues.apache.org/jira/browse/MAPREDUCE-4861) | Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4856](https://issues.apache.org/jira/browse/MAPREDUCE-4856) | TestJobOutputCommitter uses same directory as TestJobCleanup | Major | test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4848](https://issues.apache.org/jira/browse/MAPREDUCE-4848) | TaskAttemptContext cast error during AM recovery | Major | mr-am | Jason Lowe | Jerry Chen | +| [MAPREDUCE-4842](https://issues.apache.org/jira/browse/MAPREDUCE-4842) | Shuffle race can hang reducer | Blocker | mrv2 | Jason Lowe | Mariappan Asokan | +| [MAPREDUCE-4836](https://issues.apache.org/jira/browse/MAPREDUCE-4836) | Elapsed time for running tasks on AM web UI tasks page is 0 | Major | . | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4833](https://issues.apache.org/jira/browse/MAPREDUCE-4833) | Task can get stuck in FAIL\_CONTAINER\_CLEANUP | Critical | applicationmaster, mrv2 | Robert Joseph Evans | Robert Parker | +| [MAPREDUCE-4832](https://issues.apache.org/jira/browse/MAPREDUCE-4832) | MR AM can get in a split brain situation | Critical | applicationmaster | Robert Joseph Evans | Jason Lowe | +| [MAPREDUCE-4825](https://issues.apache.org/jira/browse/MAPREDUCE-4825) | JobImpl.finished doesn't expect ERROR as a final job state | Major | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4819](https://issues.apache.org/jira/browse/MAPREDUCE-4819) | AM can rerun job after reporting final job status to the client | Blocker | mr-am | Jason Lowe | Bikas Saha | +| [MAPREDUCE-4817](https://issues.apache.org/jira/browse/MAPREDUCE-4817) | Hardcoded task ping timeout kills tasks localizing large amounts of data | Critical | applicationmaster, mr-am | Jason Lowe | Thomas Graves | +| [MAPREDUCE-4813](https://issues.apache.org/jira/browse/MAPREDUCE-4813) | AM timing out during job commit | Critical | applicationmaster | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4801](https://issues.apache.org/jira/browse/MAPREDUCE-4801) | ShuffleHandler can generate large logs due to prematurely closed channels | Critical | . | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4797](https://issues.apache.org/jira/browse/MAPREDUCE-4797) | LocalContainerAllocator can loop forever trying to contact the RM | Major | applicationmaster | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4787](https://issues.apache.org/jira/browse/MAPREDUCE-4787) | TestJobMonitorAndPrint is broken | Major | test | Ravi Prakash | Robert Parker | +| [MAPREDUCE-4786](https://issues.apache.org/jira/browse/MAPREDUCE-4786) | Job End Notification retry interval is 5 milliseconds by default | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4782](https://issues.apache.org/jira/browse/MAPREDUCE-4782) | NLineInputFormat skips first line of last InputSplit | Blocker | client | Mark Fuhs | Mark Fuhs | +| [MAPREDUCE-4778](https://issues.apache.org/jira/browse/MAPREDUCE-4778) | Fair scheduler event log is only written if directory exists on HDFS | Major | jobtracker, scheduler | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4774](https://issues.apache.org/jira/browse/MAPREDUCE-4774) | JobImpl does not handle asynchronous task events in FAILED state | Major | applicationmaster, mrv2 | Ivan A. Veselovsky | Jason Lowe | +| [MAPREDUCE-4772](https://issues.apache.org/jira/browse/MAPREDUCE-4772) | Fetch failures can take way too long for a map to be restarted | Critical | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4771](https://issues.apache.org/jira/browse/MAPREDUCE-4771) | KeyFieldBasedPartitioner not partitioning properly when configured | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4751](https://issues.apache.org/jira/browse/MAPREDUCE-4751) | AM stuck in KILL\_WAIT for days | Major | . | Ravi Prakash | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4748](https://issues.apache.org/jira/browse/MAPREDUCE-4748) | Invalid event: T\_ATTEMPT\_SUCCEEDED at SUCCEEDED | Blocker | mrv2 | Robert Joseph Evans | Jason Lowe | +| [MAPREDUCE-4746](https://issues.apache.org/jira/browse/MAPREDUCE-4746) | The MR Application Master does not have a config to set environment variables | Major | applicationmaster | Robert Parker | Robert Parker | +| [MAPREDUCE-4741](https://issues.apache.org/jira/browse/MAPREDUCE-4741) | WARN and ERROR messages logged during normal AM shutdown | Minor | applicationmaster, mrv2 | Jason Lowe | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4740](https://issues.apache.org/jira/browse/MAPREDUCE-4740) | only .jars can be added to the Distributed Cache classpath | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-4733](https://issues.apache.org/jira/browse/MAPREDUCE-4733) | Reducer can fail to make progress during shuffle if too many reducers complete consecutively | Major | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4730](https://issues.apache.org/jira/browse/MAPREDUCE-4730) | AM crashes due to OOM while serving up map task completion events | Blocker | applicationmaster, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4729](https://issues.apache.org/jira/browse/MAPREDUCE-4729) | job history UI not showing all job attempts | Major | jobhistoryserver | Thomas Graves | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4724](https://issues.apache.org/jira/browse/MAPREDUCE-4724) | job history web ui applications page should be sorted to display last app first | Major | jobhistoryserver | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4721](https://issues.apache.org/jira/browse/MAPREDUCE-4721) | Task startup time in JHS is same as job startup time. | Major | jobhistoryserver | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4720](https://issues.apache.org/jira/browse/MAPREDUCE-4720) | Browser thinks History Server main page JS is taking too long | Major | . | Robert Joseph Evans | Ravi Prakash | +| [MAPREDUCE-4712](https://issues.apache.org/jira/browse/MAPREDUCE-4712) | mr-jobhistory-daemon.sh doesn't accept --config | Major | jobhistoryserver | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-4705](https://issues.apache.org/jira/browse/MAPREDUCE-4705) | Historyserver links expire before the history data does | Critical | jobhistoryserver, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4681](https://issues.apache.org/jira/browse/MAPREDUCE-4681) | HDFS-3910 broke MR tests | Major | . | Arun C Murthy | Arun C Murthy | +| [MAPREDUCE-4678](https://issues.apache.org/jira/browse/MAPREDUCE-4678) | Running the Pentomino example with defaults throws java.lang.NegativeArraySizeException | Minor | examples | Chris McConnell | Chris McConnell | +| [MAPREDUCE-4674](https://issues.apache.org/jira/browse/MAPREDUCE-4674) | Hadoop examples secondarysort has a typo "secondarysrot" in the usage | Minor | . | Robert Justice | Robert Justice | +| [MAPREDUCE-4654](https://issues.apache.org/jira/browse/MAPREDUCE-4654) | TestDistCp is @ignored | Critical | test | Colin Patrick McCabe | Sandy Ryza | +| [MAPREDUCE-4637](https://issues.apache.org/jira/browse/MAPREDUCE-4637) | Killing an unassigned task attempt causes the job to fail | Major | mrv2 | Tom White | Mayank Bansal | +| [MAPREDUCE-4607](https://issues.apache.org/jira/browse/MAPREDUCE-4607) | Race condition in ReduceTask completion can result in Task being incorrectly failed | Major | . | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4554](https://issues.apache.org/jira/browse/MAPREDUCE-4554) | Job Credentials are not transmitted if security is turned off | Major | job submission, security | Benoy Antony | Benoy Antony | +| [MAPREDUCE-4521](https://issues.apache.org/jira/browse/MAPREDUCE-4521) | mapreduce.user.classpath.first incompatibility with 0.20/1.x | Major | mrv2 | Jason Lowe | Ravi Prakash | +| [MAPREDUCE-4479](https://issues.apache.org/jira/browse/MAPREDUCE-4479) | Fix parameter order in assertEquals() in TestCombineInputFileFormat.java | Major | test | Mariappan Asokan | Mariappan Asokan | +| [MAPREDUCE-4425](https://issues.apache.org/jira/browse/MAPREDUCE-4425) | Speculation + Fetch failures can lead to a hung job | Critical | mrv2 | Siddharth Seth | Jason Lowe | +| [MAPREDUCE-4279](https://issues.apache.org/jira/browse/MAPREDUCE-4279) | getClusterStatus() fails with null pointer exception when running jobs in local mode | Major | jobtracker | Rahul Jain | Devaraj K | +| [MAPREDUCE-4278](https://issues.apache.org/jira/browse/MAPREDUCE-4278) | cannot run two local jobs in parallel from the same gateway. | Major | . | Araceli Henley | Sandy Ryza | +| [MAPREDUCE-4272](https://issues.apache.org/jira/browse/MAPREDUCE-4272) | SortedRanges.Range#compareTo is not spec compliant | Major | task | Luke Lu | Yu Gao | +| [MAPREDUCE-2264](https://issues.apache.org/jira/browse/MAPREDUCE-2264) | Job status exceeds 100% in some cases | Major | jobtracker | Adam Kramer | Devaraj K | +| [MAPREDUCE-1806](https://issues.apache.org/jira/browse/MAPREDUCE-1806) | CombineFileInputFormat does not work with paths not on default FS | Major | harchive | Paul Yang | Gera Shegalov | +| [MAPREDUCE-1700](https://issues.apache.org/jira/browse/MAPREDUCE-1700) | User supplied dependencies may conflict with MapReduce system JARs | Major | task | Tom White | Tom White | +| [YARN-364](https://issues.apache.org/jira/browse/YARN-364) | AggregatedLogDeletionService can take too long to delete logs | Major | . | Jason Lowe | Jason Lowe | +| [YARN-360](https://issues.apache.org/jira/browse/YARN-360) | Allow apps to concurrently register tokens for renewal | Critical | . | Daryn Sharp | Daryn Sharp | +| [YARN-357](https://issues.apache.org/jira/browse/YARN-357) | App submission should not be synchronized | Major | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-355](https://issues.apache.org/jira/browse/YARN-355) | RM app submission jams under load | Blocker | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-354](https://issues.apache.org/jira/browse/YARN-354) | WebAppProxyServer exits immediately after startup | Blocker | . | Liang Xie | Liang Xie | +| [YARN-343](https://issues.apache.org/jira/browse/YARN-343) | Capacity Scheduler maximum-capacity value -1 is invalid | Major | capacityscheduler | Thomas Graves | Xuan Gong | +| [YARN-336](https://issues.apache.org/jira/browse/YARN-336) | Fair scheduler FIFO scheduling within a queue only allows 1 app at a time | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-334](https://issues.apache.org/jira/browse/YARN-334) | Maven RAT plugin is not checking all source files | Critical | . | Thomas Graves | Thomas Graves | +| [YARN-330](https://issues.apache.org/jira/browse/YARN-330) | Flakey test: TestNodeManagerShutdown#testKillContainersOnShutdown | Major | nodemanager | Hitesh Shah | Sandy Ryza | +| [YARN-325](https://issues.apache.org/jira/browse/YARN-325) | RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completing | Blocker | capacityscheduler | Jason Lowe | Arun C Murthy | +| [YARN-320](https://issues.apache.org/jira/browse/YARN-320) | RM should always be able to renew its own tokens | Blocker | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-302](https://issues.apache.org/jira/browse/YARN-302) | Fair scheduler assignmultiple should default to false | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-301](https://issues.apache.org/jira/browse/YARN-301) | Fair scheduler throws ConcurrentModificationException when iterating over app's priorities | Major | resourcemanager, scheduler | Hong Shen | Hong Shen | +| [YARN-300](https://issues.apache.org/jira/browse/YARN-300) | After YARN-271, fair scheduler can infinite loop and not schedule any application. | Major | resourcemanager, scheduler | Hong Shen | Sandy Ryza | +| [YARN-293](https://issues.apache.org/jira/browse/YARN-293) | Node Manager leaks LocalizerRunner object for every Container | Critical | nodemanager | Devaraj K | Robert Joseph Evans | +| [YARN-288](https://issues.apache.org/jira/browse/YARN-288) | Fair scheduler queue doesn't accept any jobs when ACLs are configured. | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-283](https://issues.apache.org/jira/browse/YARN-283) | Fair scheduler fails to get queue info without root prefix | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-282](https://issues.apache.org/jira/browse/YARN-282) | Fair scheduler web UI double counts Apps Submitted | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-278](https://issues.apache.org/jira/browse/YARN-278) | Fair scheduler maxRunningApps config causes no apps to make progress | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-272](https://issues.apache.org/jira/browse/YARN-272) | Fair scheduler log messages try to print objects without overridden toString methods | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-271](https://issues.apache.org/jira/browse/YARN-271) | Fair scheduler hits IllegalStateException trying to reserve different apps on same node | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-267](https://issues.apache.org/jira/browse/YARN-267) | Fix fair scheduler web UI | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-266](https://issues.apache.org/jira/browse/YARN-266) | RM and JHS Web UIs are blank because AppsBlock is not escaping string properly | Critical | resourcemanager | Ravi Prakash | Ravi Prakash | +| [YARN-264](https://issues.apache.org/jira/browse/YARN-264) | y.s.rm.DelegationTokenRenewer attempts to renew token even after removing an app | Major | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-258](https://issues.apache.org/jira/browse/YARN-258) | RM web page UI shows Invalid Date for start and finish times | Major | resourcemanager | Ravi Prakash | Ravi Prakash | +| [YARN-253](https://issues.apache.org/jira/browse/YARN-253) | Container launch may fail if no files were localized | Critical | nodemanager | Tom White | Tom White | +| [YARN-251](https://issues.apache.org/jira/browse/YARN-251) | Proxy URI generation fails for blank tracking URIs | Major | resourcemanager | Tom White | Tom White | +| [YARN-225](https://issues.apache.org/jira/browse/YARN-225) | Proxy Link in RM UI thows NPE in Secure mode | Critical | resourcemanager | Devaraj K | Devaraj K | +| [YARN-224](https://issues.apache.org/jira/browse/YARN-224) | Fair scheduler logs too many nodeUpdate INFO messages | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-223](https://issues.apache.org/jira/browse/YARN-223) | Change processTree interface to work better with native code | Critical | . | Radim Kolar | Radim Kolar | +| [YARN-217](https://issues.apache.org/jira/browse/YARN-217) | yarn rmadmin commands fail in secure cluster | Blocker | resourcemanager | Devaraj K | Devaraj K | +| [YARN-214](https://issues.apache.org/jira/browse/YARN-214) | RMContainerImpl does not handle event EXPIRE at state RUNNING | Major | resourcemanager | Jason Lowe | Jonathan Eagles | +| [YARN-212](https://issues.apache.org/jira/browse/YARN-212) | NM state machine ignores an APPLICATION\_CONTAINER\_FINISHED event when it shouldn't | Blocker | nodemanager | Nathan Roberts | Nathan Roberts | +| [YARN-206](https://issues.apache.org/jira/browse/YARN-206) | TestApplicationCleanup.testContainerCleanup occasionally fails | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-201](https://issues.apache.org/jira/browse/YARN-201) | CapacityScheduler can take a very long time to schedule containers if requests are off cluster | Critical | capacityscheduler | Jason Lowe | Jason Lowe | +| [YARN-189](https://issues.apache.org/jira/browse/YARN-189) | deadlock in RM - AMResponse object | Blocker | resourcemanager | Thomas Graves | Thomas Graves | +| [YARN-181](https://issues.apache.org/jira/browse/YARN-181) | capacity-scheduler.xml move breaks Eclipse import | Critical | resourcemanager | Siddharth Seth | Siddharth Seth | +| [YARN-180](https://issues.apache.org/jira/browse/YARN-180) | Capacity scheduler - containers that get reserved create container token to early | Critical | capacityscheduler | Thomas Graves | Arun C Murthy | +| [YARN-179](https://issues.apache.org/jira/browse/YARN-179) | Bunch of test failures on trunk | Blocker | capacityscheduler | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-178](https://issues.apache.org/jira/browse/YARN-178) | Fix custom ProcessTree instance creation | Critical | . | Radim Kolar | Radim Kolar | +| [YARN-177](https://issues.apache.org/jira/browse/YARN-177) | CapacityScheduler - adding a queue while the RM is running has wacky results | Critical | capacityscheduler | Thomas Graves | Arun C Murthy | +| [YARN-170](https://issues.apache.org/jira/browse/YARN-170) | NodeManager stop() gets called twice on shutdown | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-166](https://issues.apache.org/jira/browse/YARN-166) | capacity scheduler doesn't allow capacity \< 1.0 | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-163](https://issues.apache.org/jira/browse/YARN-163) | Retrieving container log via NM webapp can hang with multibyte characters in log | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-161](https://issues.apache.org/jira/browse/YARN-161) | Yarn Common has multiple compiler warnings for unchecked operations | Major | api | Chris Nauroth | Chris Nauroth | +| [YARN-159](https://issues.apache.org/jira/browse/YARN-159) | RM web ui applications page should be sorted to display last app first | Major | resourcemanager | Thomas Graves | Thomas Graves | +| [YARN-151](https://issues.apache.org/jira/browse/YARN-151) | Browser thinks RM main page JS is taking too long | Major | . | Robert Joseph Evans | Ravi Prakash | +| [YARN-150](https://issues.apache.org/jira/browse/YARN-150) | AppRejectedTransition does not unregister app from master service and scheduler | Major | . | Bikas Saha | Bikas Saha | +| [YARN-140](https://issues.apache.org/jira/browse/YARN-140) | Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler. | Major | capacityscheduler | Ahmed Radwan | Ahmed Radwan | +| [YARN-139](https://issues.apache.org/jira/browse/YARN-139) | Interrupted Exception within AsyncDispatcher leads to user confusion | Major | api | Nathan Roberts | Vinod Kumar Vavilapalli | +| [YARN-136](https://issues.apache.org/jira/browse/YARN-136) | Make ClientTokenSecretManager part of RMContext | Major | resourcemanager | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-133](https://issues.apache.org/jira/browse/YARN-133) | update web services docs for RM clusterMetrics | Major | resourcemanager | Thomas Graves | Ravi Prakash | +| [YARN-131](https://issues.apache.org/jira/browse/YARN-131) | Incorrect ACL properties in capacity scheduler documentation | Major | capacityscheduler | Ahmed Radwan | Ahmed Radwan | +| [YARN-127](https://issues.apache.org/jira/browse/YARN-127) | Move RMAdmin tool to the client package | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-116](https://issues.apache.org/jira/browse/YARN-116) | RM is missing ability to add include/exclude files without a restart | Major | resourcemanager | xieguiming | xieguiming | +| [YARN-102](https://issues.apache.org/jira/browse/YARN-102) | Move the apache licence header to the top of the file in MemStore.java | Trivial | resourcemanager | Devaraj K | Devaraj K | +| [YARN-94](https://issues.apache.org/jira/browse/YARN-94) | DistributedShell jar should point to Client as the main class by default | Major | applications/distributed-shell | Vinod Kumar Vavilapalli | Hitesh Shah | +| [YARN-93](https://issues.apache.org/jira/browse/YARN-93) | Diagnostics missing from applications that have finished but failed | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-82](https://issues.apache.org/jira/browse/YARN-82) | YARN local-dirs defaults to /tmp/nm-local-dir | Minor | nodemanager | Andy Isaacson | Hemanth Yamijala | +| [YARN-78](https://issues.apache.org/jira/browse/YARN-78) | Change UnmanagedAMLauncher to use YarnClientImpl | Major | applications | Bikas Saha | Bikas Saha | +| [YARN-72](https://issues.apache.org/jira/browse/YARN-72) | NM should handle cleaning up containers when it shuts down | Major | nodemanager | Hitesh Shah | Sandy Ryza | +| [YARN-53](https://issues.apache.org/jira/browse/YARN-53) | Add protocol to YARN to support GetGroups | Major | resourcemanager | Alejandro Abdelnur | Bo Wang | +| [YARN-43](https://issues.apache.org/jira/browse/YARN-43) | TestResourceTrackerService fail intermittently on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [YARN-40](https://issues.apache.org/jira/browse/YARN-40) | Provide support for missing yarn commands | Major | client | Devaraj K | Devaraj K | +| [YARN-33](https://issues.apache.org/jira/browse/YARN-33) | LocalDirsHandler should validate the configured local and log dirs | Major | nodemanager | Mayank Bansal | Mayank Bansal | +| [YARN-32](https://issues.apache.org/jira/browse/YARN-32) | TestApplicationTokens fails intermintently on jdk7 | Major | . | Thomas Graves | Vinod Kumar Vavilapalli | +| [YARN-30](https://issues.apache.org/jira/browse/YARN-30) | TestNMWebServicesApps, TestRMWebServicesApps and TestRMWebServicesNodes fail on jdk7 | Major | . | Thomas Graves | Thomas Graves | +| [YARN-28](https://issues.apache.org/jira/browse/YARN-28) | TestCompositeService fails on jdk7 | Major | . | Thomas Graves | Thomas Graves | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9217](https://issues.apache.org/jira/browse/HADOOP-9217) | Print thread dumps when hadoop-common tests fail | Major | test | Andrey Klochkov | Andrey Klochkov | +| [HADOOP-9119](https://issues.apache.org/jira/browse/HADOOP-9119) | Add test to FileSystemContractBaseTest to verify integrity of overwritten files | Minor | fs, test | Steve Loughran | Steve Loughran | +| [HADOOP-9067](https://issues.apache.org/jira/browse/HADOOP-9067) | provide test for method org.apache.hadoop.fs.LocalFileSystem.reportChecksumFailure(Path, FSDataInputStream, long, FSDataInputStream, long) | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9042](https://issues.apache.org/jira/browse/HADOOP-9042) | Add a test for umask in FileSystemContractBaseTest | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9038](https://issues.apache.org/jira/browse/HADOOP-9038) | provide unit-test coverage of class org.apache.hadoop.fs.LocalDirAllocator.AllocatorPerContext.PathIterator | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HDFS-4349](https://issues.apache.org/jira/browse/HDFS-4349) | Test reading files from BackupNode | Major | namenode, test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-4199](https://issues.apache.org/jira/browse/HDFS-4199) | Provide test for HdfsVolumeId | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HDFS-4007](https://issues.apache.org/jira/browse/HDFS-4007) | Rehabilitate bit-rotted unit tests under hadoop-hdfs-project/hadoop-hdfs/src/test/unit/ | Minor | test | Colin Patrick McCabe | Colin Patrick McCabe | +| [MAPREDUCE-4905](https://issues.apache.org/jira/browse/MAPREDUCE-4905) | test org.apache.hadoop.mapred.pipes | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4803](https://issues.apache.org/jira/browse/MAPREDUCE-4803) | Duplicate copies of TestIndexCache.java | Minor | test | Mariappan Asokan | Mariappan Asokan | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9035](https://issues.apache.org/jira/browse/HADOOP-9035) | Generalize setup of LoginContext | Major | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9021](https://issues.apache.org/jira/browse/HADOOP-9021) | Enforce configured SASL method on the server | Major | ipc, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9020](https://issues.apache.org/jira/browse/HADOOP-9020) | Add a SASL PLAIN server | Major | ipc, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9015](https://issues.apache.org/jira/browse/HADOOP-9015) | Standardize creation of SaslRpcServers | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9014](https://issues.apache.org/jira/browse/HADOOP-9014) | Standardize creation of SaslRpcClients | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9013](https://issues.apache.org/jira/browse/HADOOP-9013) | UGI should not hardcode loginUser's authenticationType | Major | fs, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9012](https://issues.apache.org/jira/browse/HADOOP-9012) | IPC Client sends wrong connection context | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9010](https://issues.apache.org/jira/browse/HADOOP-9010) | Map UGI authenticationMethod to RPC authMethod | Major | fs, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9009](https://issues.apache.org/jira/browse/HADOOP-9009) | Add SecurityUtil methods to get/set authentication method | Major | fs, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-8784](https://issues.apache.org/jira/browse/HADOOP-8784) | Improve IPC.Client's token use | Major | ipc, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-8783](https://issues.apache.org/jira/browse/HADOOP-8783) | Improve RPC.Server's digest auth | Major | ipc, security | Daryn Sharp | Daryn Sharp | +| [HDFS-4445](https://issues.apache.org/jira/browse/HDFS-4445) | All BKJM ledgers are not checked while tailing, So failover will fail. | Blocker | . | Vinayakumar B | Vinayakumar B | +| [HDFS-4248](https://issues.apache.org/jira/browse/HDFS-4248) | Renames may remove file leases | Blocker | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4247](https://issues.apache.org/jira/browse/HDFS-4247) | saveNamespace should be tolerant of dangling lease | Blocker | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4130](https://issues.apache.org/jira/browse/HDFS-4130) | BKJM: The reading for editlog at NN starting using bkjm is not efficient | Major | ha, performance | Han Xiao | Han Xiao | +| [HDFS-4100](https://issues.apache.org/jira/browse/HDFS-4100) | Fix all findbug security warings | Major | datanode, journal-node, security | Liang Xie | Liang Xie | +| [HDFS-4059](https://issues.apache.org/jira/browse/HDFS-4059) | Add number of stale DataNodes to metrics | Minor | datanode, namenode | Jing Zhao | Jing Zhao | +| [HDFS-4038](https://issues.apache.org/jira/browse/HDFS-4038) | Override toString() for BookKeeperEditLogInputStream | Minor | ha | Vinayakumar B | Vinayakumar B | +| [HDFS-4035](https://issues.apache.org/jira/browse/HDFS-4035) | LightWeightGSet and LightWeightHashSet increment a volatile without synchronization | Major | . | Eli Collins | Eli Collins | +| [HDFS-4034](https://issues.apache.org/jira/browse/HDFS-4034) | Remove redundant null checks | Major | . | Eli Collins | Eli Collins | +| [HDFS-4033](https://issues.apache.org/jira/browse/HDFS-4033) | Miscellaneous findbugs 2 fixes | Major | . | Eli Collins | Eli Collins | +| [HDFS-4032](https://issues.apache.org/jira/browse/HDFS-4032) | Specify the charset explicitly rather than rely on the default | Major | . | Eli Collins | Eli Collins | +| [HDFS-4031](https://issues.apache.org/jira/browse/HDFS-4031) | Update findbugsExcludeFile.xml to include findbugs 2 exclusions | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-4030](https://issues.apache.org/jira/browse/HDFS-4030) | BlockManager excessBlocksCount and postponedMisreplicatedBlocksCount should be AtomicLongs | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-4029](https://issues.apache.org/jira/browse/HDFS-4029) | GenerationStamp should use an AtomicLong | Major | namenode | Eli Collins | Eli Collins | +| [HDFS-3935](https://issues.apache.org/jira/browse/HDFS-3935) | QJM: Add JournalNode to the start / stop scripts | Major | . | Eli Collins | Andy Isaacson | +| [HDFS-3923](https://issues.apache.org/jira/browse/HDFS-3923) | libwebhdfs testing code cleanup | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-3920](https://issues.apache.org/jira/browse/HDFS-3920) | libwebdhfs code cleanup: string processing and using strerror consistently to handle all errors | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-3912](https://issues.apache.org/jira/browse/HDFS-3912) | Detecting and avoiding stale datanodes for writing | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-3810](https://issues.apache.org/jira/browse/HDFS-3810) | Implement format() for BKJM | Major | . | Ivan Kelly | Ivan Kelly | +| [HDFS-3809](https://issues.apache.org/jira/browse/HDFS-3809) | Make BKJM use protobufs for all serialization with ZK | Major | namenode | Ivan Kelly | Ivan Kelly | +| [HDFS-3789](https://issues.apache.org/jira/browse/HDFS-3789) | JournalManager#format() should be able to throw IOException | Major | ha, namenode | Ivan Kelly | Ivan Kelly | +| [HDFS-3695](https://issues.apache.org/jira/browse/HDFS-3695) | Genericize format() to non-file JournalManagers | Major | ha, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3623](https://issues.apache.org/jira/browse/HDFS-3623) | BKJM: zkLatchWaitTimeout hard coded to 6000. Make use of ZKSessionTimeout instead. | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-3573](https://issues.apache.org/jira/browse/HDFS-3573) | Supply NamespaceInfo when instantiating JournalManagers | Minor | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3571](https://issues.apache.org/jira/browse/HDFS-3571) | Allow EditLogFileInputStream to read from a remote URL | Major | ha, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-2908](https://issues.apache.org/jira/browse/HDFS-2908) | Add apache license header for StorageReport.java | Minor | . | Suresh Srinivas | Brandon Li | +| [MAPREDUCE-4809](https://issues.apache.org/jira/browse/MAPREDUCE-4809) | Change visibility of classes for pluggable sort changes | Major | . | Arun C Murthy | Mariappan Asokan | +| [MAPREDUCE-4807](https://issues.apache.org/jira/browse/MAPREDUCE-4807) | Allow MapOutputBuffer to be pluggable | Major | . | Arun C Murthy | Mariappan Asokan | +| [MAPREDUCE-4049](https://issues.apache.org/jira/browse/MAPREDUCE-4049) | plugin for generic shuffle service | Major | performance, task, tasktracker | Avner BenHanoch | Avner BenHanoch | +| [YARN-280](https://issues.apache.org/jira/browse/YARN-280) | RM does not reject app submission with invalid tokens | Major | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-231](https://issues.apache.org/jira/browse/YARN-231) | Add FS-based persistent store implementation for RMStateStore | Major | resourcemanager | Bikas Saha | Bikas Saha | +| [YARN-230](https://issues.apache.org/jira/browse/YARN-230) | Make changes for RM restart phase 1 | Major | resourcemanager | Bikas Saha | Bikas Saha | +| [YARN-229](https://issues.apache.org/jira/browse/YARN-229) | Remove old code for restart | Major | resourcemanager | Bikas Saha | Bikas Saha | +| [YARN-219](https://issues.apache.org/jira/browse/YARN-219) | NM should aggregate logs when application finishes. | Critical | nodemanager | Robert Joseph Evans | Robert Joseph Evans | +| [YARN-204](https://issues.apache.org/jira/browse/YARN-204) | test coverage for org.apache.hadoop.tools | Major | applications | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-202](https://issues.apache.org/jira/browse/YARN-202) | Log Aggregation generates a storm of fsync() for namenode | Critical | . | Kihwal Lee | Kihwal Lee | +| [YARN-188](https://issues.apache.org/jira/browse/YARN-188) | Coverage fixing for CapacityScheduler | Major | capacityscheduler | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-186](https://issues.apache.org/jira/browse/YARN-186) | Coverage fixing LinuxContainerExecutor | Major | resourcemanager, scheduler | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-135](https://issues.apache.org/jira/browse/YARN-135) | ClientTokens should be per app-attempt and be unregistered on App-finish. | Major | resourcemanager | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-134](https://issues.apache.org/jira/browse/YARN-134) | ClientToAMSecretManager creates keys without checking for validity of the appID | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-103](https://issues.apache.org/jira/browse/YARN-103) | Add a yarn AM - RM client module | Major | . | Bikas Saha | Bikas Saha | +| [YARN-50](https://issues.apache.org/jira/browse/YARN-50) | Implement renewal / cancellation of Delegation Tokens | Blocker | . | Siddharth Seth | Siddharth Seth | +| [YARN-3](https://issues.apache.org/jira/browse/YARN-3) | Add support for CPU isolation/monitoring of containers | Major | . | Arun C Murthy | Andrew Ferguson | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8860](https://issues.apache.org/jira/browse/HADOOP-8860) | Split MapReduce and YARN sections in documentation navigation | Major | documentation | Tom White | Tom White | +| [HADOOP-8427](https://issues.apache.org/jira/browse/HADOOP-8427) | Convert Forrest docs to APT, incremental | Major | documentation | Eli Collins | Andy Isaacson | +| [HDFS-4326](https://issues.apache.org/jira/browse/HDFS-4326) | bump up Tomcat version for HttpFS to 6.0.36 | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3944](https://issues.apache.org/jira/browse/HDFS-3944) | Httpfs resolveAuthority() is not resolving host correctly | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-4596](https://issues.apache.org/jira/browse/MAPREDUCE-4596) | Split StateMachine state from states seen by MRClientProtocol (for Job, Task, TaskAttempt) | Major | applicationmaster, mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-4266](https://issues.apache.org/jira/browse/MAPREDUCE-4266) | remove Ant remnants from MR | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-372](https://issues.apache.org/jira/browse/YARN-372) | Move InlineDispatcher from hadoop-yarn-server-resourcemanager to hadoop-yarn-common | Minor | . | Siddharth Seth | Siddharth Seth | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/RELEASENOTES.2.0.3-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/RELEASENOTES.2.0.3-alpha.md new file mode 100644 index 00000000000..e8aaca661de --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.3-alpha/RELEASENOTES.2.0.3-alpha.md @@ -0,0 +1,222 @@ + + +# Apache Hadoop 2.0.3-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9147](https://issues.apache.org/jira/browse/HADOOP-9147) | *Trivial* | **Add missing fields to FIleStatus.toString** + +Update FileStatus.toString to include missing fields + + +--- + +* [HADOOP-9119](https://issues.apache.org/jira/browse/HADOOP-9119) | *Minor* | **Add test to FileSystemContractBaseTest to verify integrity of overwritten files** + +Patches adds more tests to verify overwritten and more complex operations -write-delete-overwrite. By using differently sized datasets and different data inside, these tests verify that the overwrite really did take place. While HDFS meets all these requirements directly, eventually consistent object stores may not -hence these tests. + + +--- + +* [HADOOP-9118](https://issues.apache.org/jira/browse/HADOOP-9118) | *Trivial* | **FileSystemContractBaseTest test data for read/write isn't rigorous enough** + +Resolved as part of HADOOP-9119 -it's test data generator creates more bits in every test byte + + +--- + +* [HADOOP-9106](https://issues.apache.org/jira/browse/HADOOP-9106) | *Major* | **Allow configuration of IPC connect timeout** + +This jira introduces a new configuration parameter "ipc.client.connect.timeout". This configuration defines the Hadoop RPC connection timeout in milliseconds for a client to connect to a server. For details see the description associated with this configuration in core-default.xml. + + +--- + +* [HADOOP-9070](https://issues.apache.org/jira/browse/HADOOP-9070) | *Blocker* | **Kerberos SASL server cannot find kerberos key** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8999](https://issues.apache.org/jira/browse/HADOOP-8999) | *Major* | **SASL negotiation is flawed** + +The RPC SASL negotiation now always ends with final response. If the SASL mechanism does not have a final response (GSSAPI, PLAIN), then an empty success response is sent to the client. The client will now always expect a final response to definitively know if negotiation is complete/successful. + + +--- + +* [HADOOP-8926](https://issues.apache.org/jira/browse/HADOOP-8926) | *Major* | **hadoop.util.PureJavaCrc32 cache hit-ratio is low for static data** + +Speed up Crc32 by improving the cache hit-ratio of hadoop.util.PureJavaCrc32 + + +--- + +* [HADOOP-8922](https://issues.apache.org/jira/browse/HADOOP-8922) | *Trivial* | **Provide alternate JSONP output for JMXJsonServlet to allow javascript in browser dashboard** + +Add a JSONP alternative outpout for /jmx HTTP interface to provide a Javascript polling ability in browsers. + + +--- + +* [HADOOP-8712](https://issues.apache.org/jira/browse/HADOOP-8712) | *Minor* | **Change default hadoop.security.group.mapping** + +The default group mapping policy has been changed to JniBasedUnixGroupsNetgroupMappingWithFallback. This should maintain the same semantics as the prior default for most users. + + +--- + +* [HDFS-4451](https://issues.apache.org/jira/browse/HDFS-4451) | *Major* | **hdfs balancer command returns exit code 1 on success instead of 0** + +This is an incompatible change from release 2.0.2-alpha and prior releases. Balancer tool exited with exit code 1 on success. It is changed to exit with exit code 0 on success. Non 0 exit code indicates failure. + + +--- + +* [HDFS-4403](https://issues.apache.org/jira/browse/HDFS-4403) | *Minor* | **DFSClient can infer checksum type when not provided by reading first byte** + +The HDFS implementation of getFileChecksum() can now operate correctly against earlier-version datanodes which do not include the checksum type information in their checksum response. The checksum type is automatically inferred by issuing a read of the first byte of each block. + + +--- + +* [HDFS-4369](https://issues.apache.org/jira/browse/HDFS-4369) | *Blocker* | **GetBlockKeysResponseProto does not handle null response** + +Protobuf message GetBlockKeysResponseProto member keys is made optional from required so that null values can be passed over the wire. This is an incompatible wire protocol change and does not affect the API backward compatibility. + + +--- + +* [HDFS-4367](https://issues.apache.org/jira/browse/HDFS-4367) | *Blocker* | **GetDataEncryptionKeyResponseProto does not handle null response** + +Member dataEncryptionKey of the protobuf message GetDataEncryptionKeyResponseProto is made optional instead of required. This is incompatible change is not likely to affect the existing users (that are using HDFS FileSystem and other public APIs). + + +--- + +* [HDFS-4364](https://issues.apache.org/jira/browse/HDFS-4364) | *Blocker* | **GetLinkTargetResponseProto does not handle null path** + +Protobuf message GetLinkTargetResponseProto member targetPath is made optional from required so that null values can be passed over the wire. This is an incompatible wire protocol change and does not affect the API backward compatibility. + + +--- + +* [HDFS-4362](https://issues.apache.org/jira/browse/HDFS-4362) | *Critical* | **GetDelegationTokenResponseProto does not handle null token** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-4350](https://issues.apache.org/jira/browse/HDFS-4350) | *Major* | **Make enabling of stale marking on read and write paths independent** + +This patch makes an incompatible configuration change, as described below: +In releases 1.1.0 and other point releases 1.1.x, the configuration parameter "dfs.namenode.check.stale.datanode" could be used to turn on checking for the stale nodes. This configuration is no longer supported in release 1.2.0 onwards and is renamed as "dfs.namenode.avoid.read.stale.datanode". + +How feature works and configuring this feature: +As described in HDFS-3703 release notes, datanode stale period can be configured using parameter "dfs.namenode.stale.datanode.interval" in seconds (default value is 30 seconds). NameNode can be configured to use this staleness information for reads using configuration "dfs.namenode.avoid.read.stale.datanode". When this parameter is set to true, namenode picks a stale datanode as the last target to read from when returning block locations for reads. Using staleness information for writes is as described in the releases notes of HDFS-3912. + + +--- + +* [HDFS-4122](https://issues.apache.org/jira/browse/HDFS-4122) | *Major* | **Cleanup HDFS logs and reduce the size of logged messages** + +The change from this jira changes the content of some of the log messages. No log message are removed. Only the content of the log messages is changed to reduce the size. If you have a tool that depends on the exact content of the log, please look at the patch and make appropriate updates to the tool. + + +--- + +* [HDFS-4080](https://issues.apache.org/jira/browse/HDFS-4080) | *Major* | **Add a separate logger for block state change logs to enable turning off those logs** + +Add a separate logger "BlockStateChange" for block state change logs. + + +--- + +* [HDFS-4059](https://issues.apache.org/jira/browse/HDFS-4059) | *Minor* | **Add number of stale DataNodes to metrics** + +This jira adds a new metric with name "StaleDataNodes" under metrics context "dfs" of type Gauge. This tracks the number of DataNodes marked as stale. A DataNode is marked stale when the heartbeat message from the DataNode is not received within the configured time ""dfs.namenode.stale.datanode.interval". + + +Please see hdfs-default.xml documentation corresponding to ""dfs.namenode.stale.datanode.interval" for more details on how to configure this feature. When this feature is not configured, this metrics would return zero. + + +--- + +* [HDFS-3703](https://issues.apache.org/jira/browse/HDFS-3703) | *Major* | **Decrease the datanode failure detection time** + +This jira adds a new DataNode state called "stale" at the NameNode. DataNodes are marked as stale if it does not send heartbeat message to NameNode within the timeout configured using the configuration parameter "dfs.namenode.stale.datanode.interval" in seconds (default value is 30 seconds). NameNode picks a stale datanode as the last target to read from when returning block locations for reads. + +This feature is by default turned * off *. To turn on the feature, set the HDFS configuration "dfs.namenode.check.stale.datanode" to true. + + +--- + +* [HDFS-1331](https://issues.apache.org/jira/browse/HDFS-1331) | *Minor* | **dfs -test should work like /bin/test** + +"test" will not print a warning for non-existent paths when testing for existence + + +--- + +* [MAPREDUCE-4928](https://issues.apache.org/jira/browse/MAPREDUCE-4928) | *Major* | **Use token request messages defined in hadoop common** + +Protobuf message GetDelegationTokenRequestProto field renewer is made requried from optional. This change is not wire compatible with the older releases. + + +--- + +* [MAPREDUCE-4637](https://issues.apache.org/jira/browse/MAPREDUCE-4637) | *Major* | **Killing an unassigned task attempt causes the job to fail** + +Handle TaskAttempt diagnostic updates while in the NEW and UNASSIGNED states. + + +--- + +* [MAPREDUCE-4123](https://issues.apache.org/jira/browse/MAPREDUCE-4123) | *Critical* | **./mapred groups gives NoClassDefFoundError** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4049](https://issues.apache.org/jira/browse/MAPREDUCE-4049) | *Major* | **plugin for generic shuffle service** + +Allow ReduceTask loading a third party plugin for shuffle (and merge) instead of the default shuffle. + + +--- + +* [MAPREDUCE-3678](https://issues.apache.org/jira/browse/MAPREDUCE-3678) | *Major* | **The Map tasks logs should have the value of input split it processed** + +A map-task's syslogs now carries basic info on the InputSplit it processed. + + +--- + +* [MAPREDUCE-2454](https://issues.apache.org/jira/browse/MAPREDUCE-2454) | *Minor* | **Allow external sorter plugin for MR** + +MAPREDUCE-4807 Allow external implementations of the sort phase in a Map task + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/CHANGES.2.0.4-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/CHANGES.2.0.4-alpha.md new file mode 100644 index 00000000000..ff53af93708 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/CHANGES.2.0.4-alpha.md @@ -0,0 +1,98 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.4-alpha - 2013-04-25 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9379](https://issues.apache.org/jira/browse/HADOOP-9379) | capture the ulimit info after printing the log to the console | Trivial | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9374](https://issues.apache.org/jira/browse/HADOOP-9374) | Add tokens from -tokenCacheFile into UGI | Major | security | Daryn Sharp | Daryn Sharp | +| [YARN-443](https://issues.apache.org/jira/browse/YARN-443) | allow OS scheduling priority of NM to be different than the containers it launches | Major | nodemanager | Thomas Graves | Thomas Graves | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9471](https://issues.apache.org/jira/browse/HADOOP-9471) | hadoop-client wrongfully excludes jetty-util JAR, breaking webhdfs | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-9467](https://issues.apache.org/jira/browse/HADOOP-9467) | Metrics2 record filtering (.record.filter.include/exclude) does not filter by name | Major | metrics | Chris Nauroth | Chris Nauroth | +| [HADOOP-9444](https://issues.apache.org/jira/browse/HADOOP-9444) | $var shell substitution in properties are not expanded in hadoop-policy.xml | Blocker | conf | Konstantin Boudnik | Roman Shaposhnik | +| [HADOOP-9406](https://issues.apache.org/jira/browse/HADOOP-9406) | hadoop-client leaks dependency on JDK tools jar | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-9405](https://issues.apache.org/jira/browse/HADOOP-9405) | TestGridmixSummary#testExecutionSummarizer is broken | Minor | test, tools | Andrew Wang | Andrew Wang | +| [HADOOP-9399](https://issues.apache.org/jira/browse/HADOOP-9399) | protoc maven plugin doesn't work on mvn 3.0.2 | Minor | build | Todd Lipcon | Konstantin Boudnik | +| [HADOOP-9301](https://issues.apache.org/jira/browse/HADOOP-9301) | hadoop client servlet/jsp/jetty/tomcat JARs creating conflicts in Oozie & HttpFS | Blocker | build | Roman Shaposhnik | Alejandro Abdelnur | +| [HADOOP-9299](https://issues.apache.org/jira/browse/HADOOP-9299) | kerberos name resolution is kicking in even when kerberos is not configured | Blocker | security | Roman Shaposhnik | Daryn Sharp | +| [HDFS-4649](https://issues.apache.org/jira/browse/HDFS-4649) | Webhdfs cannot list large directories | Blocker | namenode, security, webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4646](https://issues.apache.org/jira/browse/HDFS-4646) | createNNProxyWithClientProtocol ignores configured timeout value | Minor | namenode | Jagane Sundar | Jagane Sundar | +| [HDFS-4581](https://issues.apache.org/jira/browse/HDFS-4581) | DataNode#checkDiskError should not be called on network errors | Major | datanode | Rohit Kochar | Rohit Kochar | +| [HDFS-4576](https://issues.apache.org/jira/browse/HDFS-4576) | Webhdfs authentication issues | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4571](https://issues.apache.org/jira/browse/HDFS-4571) | WebHDFS should not set the service hostname on the server side | Major | webhdfs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-3344](https://issues.apache.org/jira/browse/HDFS-3344) | Unreliable corrupt blocks counting in TestProcessCorruptBlocks | Major | namenode | Tsz Wo Nicholas Sze | Kihwal Lee | +| [MAPREDUCE-5117](https://issues.apache.org/jira/browse/MAPREDUCE-5117) | With security enabled HS delegation token renewer fails | Blocker | security | Roman Shaposhnik | Siddharth Seth | +| [MAPREDUCE-5094](https://issues.apache.org/jira/browse/MAPREDUCE-5094) | Disable mem monitoring by default in MiniMRYarnCluster | Major | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-5088](https://issues.apache.org/jira/browse/MAPREDUCE-5088) | MR Client gets an renewer token exception while Oozie is submitting a job | Blocker | . | Roman Shaposhnik | Daryn Sharp | +| [MAPREDUCE-5083](https://issues.apache.org/jira/browse/MAPREDUCE-5083) | MiniMRCluster should use a random component when creating an actual cluster | Major | mrv2 | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-5053](https://issues.apache.org/jira/browse/MAPREDUCE-5053) | java.lang.InternalError from decompression codec cause reducer to fail | Major | . | Robert Parker | Robert Parker | +| [MAPREDUCE-5023](https://issues.apache.org/jira/browse/MAPREDUCE-5023) | History Server Web Services missing Job Counters | Critical | jobhistoryserver, webapps | Kendall Thrapp | Ravi Prakash | +| [MAPREDUCE-5006](https://issues.apache.org/jira/browse/MAPREDUCE-5006) | streaming tests failing | Major | contrib/streaming | Alejandro Abdelnur | Sandy Ryza | +| [MAPREDUCE-4549](https://issues.apache.org/jira/browse/MAPREDUCE-4549) | Distributed cache conflicts breaks backwards compatability | Blocker | mrv2 | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-3685](https://issues.apache.org/jira/browse/MAPREDUCE-3685) | There are some bugs in implementation of MergeManager | Critical | mrv2 | anty.rao | anty | +| [YARN-470](https://issues.apache.org/jira/browse/YARN-470) | Support a way to disable resource monitoring on the NodeManager | Major | nodemanager | Hitesh Shah | Siddharth Seth | +| [YARN-449](https://issues.apache.org/jira/browse/YARN-449) | HBase test failures when running against Hadoop 2 | Blocker | . | Siddharth Seth | | +| [YARN-429](https://issues.apache.org/jira/browse/YARN-429) | capacity-scheduler config missing from yarn-test artifact | Blocker | resourcemanager | Siddharth Seth | Siddharth Seth | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-4577](https://issues.apache.org/jira/browse/HDFS-4577) | Webhdfs operations should declare if authentication is required | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4567](https://issues.apache.org/jira/browse/HDFS-4567) | Webhdfs does not need a token for token operations | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4566](https://issues.apache.org/jira/browse/HDFS-4566) | Webdhfs token cancelation should use authentication | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4560](https://issues.apache.org/jira/browse/HDFS-4560) | Webhdfs cannot use tokens obtained by another user | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4548](https://issues.apache.org/jira/browse/HDFS-4548) | Webhdfs doesn't renegotiate SPNEGO token | Blocker | . | Daryn Sharp | Daryn Sharp | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/RELEASENOTES.2.0.4-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/RELEASENOTES.2.0.4-alpha.md new file mode 100644 index 00000000000..453d2a1b4ae --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.4-alpha/RELEASENOTES.2.0.4-alpha.md @@ -0,0 +1,38 @@ + + +# Apache Hadoop 2.0.4-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9399](https://issues.apache.org/jira/browse/HADOOP-9399) | *Minor* | **protoc maven plugin doesn't work on mvn 3.0.2** + +Committed to 2.0.4-alpha branch + + +--- + +* [MAPREDUCE-5083](https://issues.apache.org/jira/browse/MAPREDUCE-5083) | *Major* | **MiniMRCluster should use a random component when creating an actual cluster** + +Committed to branch-2.0.4. Modified changes.txt in trunk, branch-2 and branch-2.0.4 accordingly. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/CHANGES.2.0.5-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/CHANGES.2.0.5-alpha.md new file mode 100644 index 00000000000..09e191b0b82 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/CHANGES.2.0.5-alpha.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.5-alpha - 2013-06-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9614](https://issues.apache.org/jira/browse/HADOOP-9614) | smart-test-patch.sh hangs for new version of patch (2.7.1) | Major | . | Ravi Prakash | Ravi Prakash | +| [HADOOP-9407](https://issues.apache.org/jira/browse/HADOOP-9407) | commons-daemon 1.0.3 dependency has bad group id causing build issues | Major | build | Sangjin Lee | Sangjin Lee | +| [HADOOP-8419](https://issues.apache.org/jira/browse/HADOOP-8419) | GzipCodec NPE upon reset with IBM JDK | Major | io | Luke Lu | Yu Li | +| [HDFS-4482](https://issues.apache.org/jira/browse/HDFS-4482) | ReplicationMonitor thread can exit with NPE due to the race between delete and replication of same file. | Blocker | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [MAPREDUCE-5240](https://issues.apache.org/jira/browse/MAPREDUCE-5240) | inside of FileOutputCommitter the initialized Credentials cache appears to be empty | Blocker | mrv2 | Roman Shaposhnik | Vinod Kumar Vavilapalli | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/RELEASENOTES.2.0.5-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/RELEASENOTES.2.0.5-alpha.md new file mode 100644 index 00000000000..5ba1cbc597a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.5-alpha/RELEASENOTES.2.0.5-alpha.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 2.0.5-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/CHANGES.2.0.6-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/CHANGES.2.0.6-alpha.md new file mode 100644 index 00000000000..5047be88939 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/CHANGES.2.0.6-alpha.md @@ -0,0 +1,65 @@ + + +# Apache Hadoop Changelog + +## Release 2.0.6-alpha - 2013-08-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-854](https://issues.apache.org/jira/browse/YARN-854) | App submission fails on secure deploy | Blocker | . | Ramya Sunil | Omkar Vinit Joshi | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/RELEASENOTES.2.0.6-alpha.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/RELEASENOTES.2.0.6-alpha.md new file mode 100644 index 00000000000..6ecfaa99213 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.0.6-alpha/RELEASENOTES.2.0.6-alpha.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 2.0.6-alpha Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/CHANGES.2.1.0-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/CHANGES.2.1.0-beta.md new file mode 100644 index 00000000000..13c97f8334a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/CHANGES.2.1.0-beta.md @@ -0,0 +1,912 @@ + + +# Apache Hadoop Changelog + +## Release 2.1.0-beta - 2013-08-22 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9832](https://issues.apache.org/jira/browse/HADOOP-9832) | Add RPC header to client ping | Blocker | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9820](https://issues.apache.org/jira/browse/HADOOP-9820) | RPCv9 wire protocol is insufficient to support multiplexing | Blocker | ipc, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9698](https://issues.apache.org/jira/browse/HADOOP-9698) | RPCv9 client must honor server's SASL negotiate response | Blocker | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9683](https://issues.apache.org/jira/browse/HADOOP-9683) | Wrap IpcConnectionContext in RPC headers | Blocker | ipc | Luke Lu | Daryn Sharp | +| [HADOOP-9649](https://issues.apache.org/jira/browse/HADOOP-9649) | Promote YARN service life-cycle libraries into Hadoop Common | Blocker | . | Zhijie Shen | Zhijie Shen | +| [HADOOP-9630](https://issues.apache.org/jira/browse/HADOOP-9630) | Remove IpcSerializationType | Major | ipc | Luke Lu | Junping Du | +| [HADOOP-9425](https://issues.apache.org/jira/browse/HADOOP-9425) | Add error codes to rpc-response | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-9421](https://issues.apache.org/jira/browse/HADOOP-9421) | Convert SASL to use ProtoBuf and provide negotiation capabilities | Blocker | . | Sanjay Radia | Daryn Sharp | +| [HADOOP-9380](https://issues.apache.org/jira/browse/HADOOP-9380) | Add totalLength to rpc response | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-9194](https://issues.apache.org/jira/browse/HADOOP-9194) | RPC Support for QoS | Major | ipc | Luke Lu | Junping Du | +| [HADOOP-9163](https://issues.apache.org/jira/browse/HADOOP-9163) | The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra copy | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-9151](https://issues.apache.org/jira/browse/HADOOP-9151) | Include RPC error info in RpcResponseHeader instead of sending it separately | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-8886](https://issues.apache.org/jira/browse/HADOOP-8886) | Remove KFS support | Major | fs | Eli Collins | Eli Collins | +| [HDFS-5083](https://issues.apache.org/jira/browse/HDFS-5083) | Update the HDFS compatibility version range | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-4866](https://issues.apache.org/jira/browse/HDFS-4866) | Protocol buffer support cannot compile under C | Blocker | namenode | Ralph Castain | Arpit Agarwal | +| [HDFS-4659](https://issues.apache.org/jira/browse/HDFS-4659) | Support setting execution bit for regular files | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-4434](https://issues.apache.org/jira/browse/HDFS-4434) | Provide a mapping from INodeId to INode | Major | namenode | Brandon Li | Suresh Srinivas | +| [HDFS-4305](https://issues.apache.org/jira/browse/HDFS-4305) | Add a configurable limit on number of blocks per file, and min block size | Minor | namenode | Todd Lipcon | Andrew Wang | +| [HDFS-2802](https://issues.apache.org/jira/browse/HDFS-2802) | Support for RW/RO snapshots in HDFS | Major | namenode | Hari Mankude | Tsz Wo Nicholas Sze | +| [MAPREDUCE-5304](https://issues.apache.org/jira/browse/MAPREDUCE-5304) | mapreduce.Job killTask/failTask/getTaskCompletionEvents methods have incompatible signature changes | Blocker | . | Alejandro Abdelnur | Karthik Kambatla | +| [MAPREDUCE-5300](https://issues.apache.org/jira/browse/MAPREDUCE-5300) | Two function signature changes in filecache.DistributedCache | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5237](https://issues.apache.org/jira/browse/MAPREDUCE-5237) | ClusterStatus incompatiblity issues with MR1 | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5234](https://issues.apache.org/jira/browse/MAPREDUCE-5234) | Signature changes for getTaskId of TaskReport in mapred | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5233](https://issues.apache.org/jira/browse/MAPREDUCE-5233) | Functions are changed or removed from Job in jobcontrol | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5176](https://issues.apache.org/jira/browse/MAPREDUCE-5176) | Preemptable annotations (to support preemption in MR) | Major | mrv2 | Carlo Curino | Carlo Curino | +| [MAPREDUCE-5156](https://issues.apache.org/jira/browse/MAPREDUCE-5156) | Hadoop-examples-1.x.x.jar cannot run on Yarn | Blocker | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-4942](https://issues.apache.org/jira/browse/MAPREDUCE-4942) | mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1 | Major | mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4737](https://issues.apache.org/jira/browse/MAPREDUCE-4737) | Hadoop does not close output file / does not call Mapper.cleanup if exception in map | Major | . | Daniel Dai | Arun C Murthy | +| [MAPREDUCE-4067](https://issues.apache.org/jira/browse/MAPREDUCE-4067) | Replace YarnRemoteException with IOException in MRv2 APIs | Critical | . | Jitendra Nath Pandey | Xuan Gong | +| [YARN-1056](https://issues.apache.org/jira/browse/YARN-1056) | Fix configs yarn.resourcemanager.resourcemanager.connect.{max.wait.secs\|retry\_interval.secs} | Trivial | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-926](https://issues.apache.org/jira/browse/YARN-926) | ContainerManagerProtcol APIs should take in requests for multiple containers | Blocker | . | Vinod Kumar Vavilapalli | Jian He | +| [YARN-918](https://issues.apache.org/jira/browse/YARN-918) | ApplicationMasterProtocol doesn't need ApplicationAttemptId in the payload after YARN-701 | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-869](https://issues.apache.org/jira/browse/YARN-869) | ResourceManagerAdministrationProtocol should neither be public(yet) nor in yarn.api | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-851](https://issues.apache.org/jira/browse/YARN-851) | Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently. | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-841](https://issues.apache.org/jira/browse/YARN-841) | Annotate and document AuxService APIs | Major | . | Siddharth Seth | Vinod Kumar Vavilapalli | +| [YARN-840](https://issues.apache.org/jira/browse/YARN-840) | Move ProtoUtils to yarn.api.records.pb.impl | Major | . | Jian He | Jian He | +| [YARN-837](https://issues.apache.org/jira/browse/YARN-837) | ClusterInfo.java doesn't seem to belong to org.apache.hadoop.yarn | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-834](https://issues.apache.org/jira/browse/YARN-834) | Review/fix annotations for yarn-client module and clearly differentiate \*Async apis | Blocker | . | Arun C Murthy | Zhijie Shen | +| [YARN-831](https://issues.apache.org/jira/browse/YARN-831) | Remove resource min from GetNewApplicationResponse | Blocker | . | Jian He | Jian He | +| [YARN-829](https://issues.apache.org/jira/browse/YARN-829) | Rename RMTokenSelector to be RMDelegationTokenSelector | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-828](https://issues.apache.org/jira/browse/YARN-828) | Remove YarnVersionAnnotation | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-826](https://issues.apache.org/jira/browse/YARN-826) | Move Clock/SystemClock to util package | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-824](https://issues.apache.org/jira/browse/YARN-824) | Add static factory to yarn client lib interface and change it to abstract class | Major | . | Jian He | Jian He | +| [YARN-823](https://issues.apache.org/jira/browse/YARN-823) | Move RMAdmin from yarn.client to yarn.client.cli and rename as RMAdminCLI | Major | . | Jian He | Jian He | +| [YARN-822](https://issues.apache.org/jira/browse/YARN-822) | Rename ApplicationToken to AMRMToken | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-821](https://issues.apache.org/jira/browse/YARN-821) | Rename FinishApplicationMasterRequest.setFinishApplicationStatus to setFinalApplicationStatus to be consistent with getter | Major | . | Jian He | Jian He | +| [YARN-806](https://issues.apache.org/jira/browse/YARN-806) | Move ContainerExitStatus from yarn.api to yarn.api.records | Major | . | Jian He | Jian He | +| [YARN-792](https://issues.apache.org/jira/browse/YARN-792) | Move NodeHealthStatus from yarn.api.record to yarn.server.api.record | Major | . | Jian He | Jian He | +| [YARN-791](https://issues.apache.org/jira/browse/YARN-791) | Ensure that RM RPC APIs that return nodes are consistent with /nodes REST API | Blocker | api, resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-787](https://issues.apache.org/jira/browse/YARN-787) | Remove resource min from Yarn client API | Blocker | api | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-777](https://issues.apache.org/jira/browse/YARN-777) | Remove unreferenced objects from proto | Major | . | Jian He | Jian He | +| [YARN-756](https://issues.apache.org/jira/browse/YARN-756) | Move PreemptionContainer/PremptionContract/PreemptionMessage/StrictPreemptionContract/PreemptionResourceRequest to api.records | Major | . | Jian He | Jian He | +| [YARN-755](https://issues.apache.org/jira/browse/YARN-755) | Rename AllocateResponse.reboot to AllocateResponse.resync | Major | . | Bikas Saha | Bikas Saha | +| [YARN-753](https://issues.apache.org/jira/browse/YARN-753) | Add individual factory method for api protocol records | Major | . | Jian He | Jian He | +| [YARN-749](https://issues.apache.org/jira/browse/YARN-749) | Rename ResourceRequest (get,set)HostName to (get,set)ResourceName | Major | . | Arun C Murthy | Arun C Murthy | +| [YARN-748](https://issues.apache.org/jira/browse/YARN-748) | Move BuilderUtils from yarn-common to yarn-server-common | Major | . | Jian He | Jian He | +| [YARN-746](https://issues.apache.org/jira/browse/YARN-746) | rename Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively | Major | . | Steve Loughran | Steve Loughran | +| [YARN-735](https://issues.apache.org/jira/browse/YARN-735) | Make ApplicationAttemptID, ContainerID, NodeID immutable | Major | . | Jian He | Jian He | +| [YARN-724](https://issues.apache.org/jira/browse/YARN-724) | Move ProtoBase from api.records to api.records.impl.pb | Major | . | Jian He | Jian He | +| [YARN-720](https://issues.apache.org/jira/browse/YARN-720) | container-log4j.properties should not refer to mapreduce properties | Major | . | Siddharth Seth | Zhijie Shen | +| [YARN-716](https://issues.apache.org/jira/browse/YARN-716) | Make ApplicationID immutable | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-701](https://issues.apache.org/jira/browse/YARN-701) | ApplicationTokens should be used irrespective of kerberos | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-694](https://issues.apache.org/jira/browse/YARN-694) | Start using NMTokens to authenticate all communication with NM | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-684](https://issues.apache.org/jira/browse/YARN-684) | ContainerManager.startContainer needs to only have ContainerTokenIdentifier instead of the whole Container | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-642](https://issues.apache.org/jira/browse/YARN-642) | Fix up /nodes REST API to have 1 param and be consistent with the Java API | Major | api, resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-635](https://issues.apache.org/jira/browse/YARN-635) | Rename YarnRemoteException to YarnException | Major | . | Xuan Gong | Siddharth Seth | +| [YARN-633](https://issues.apache.org/jira/browse/YARN-633) | Change RMAdminProtocol api to throw IOException and YarnRemoteException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-632](https://issues.apache.org/jira/browse/YARN-632) | Change ContainerManager api to throw IOException and YarnRemoteException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-631](https://issues.apache.org/jira/browse/YARN-631) | Change ClientRMProtocol api to throw IOException and YarnRemoteException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-630](https://issues.apache.org/jira/browse/YARN-630) | Change AMRMProtocol api to throw IOException and YarnRemoteException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-629](https://issues.apache.org/jira/browse/YARN-629) | Make YarnRemoteException not be rooted at IOException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-615](https://issues.apache.org/jira/browse/YARN-615) | ContainerLaunchContext.containerTokens should simply be called tokens | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-610](https://issues.apache.org/jira/browse/YARN-610) | ClientToken (ClientToAMToken) should not be set in the environment | Blocker | . | Siddharth Seth | Omkar Vinit Joshi | +| [YARN-579](https://issues.apache.org/jira/browse/YARN-579) | Make ApplicationToken part of Container's token list to help RM-restart | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-571](https://issues.apache.org/jira/browse/YARN-571) | User should not be part of ContainerLaunchContext | Major | . | Hitesh Shah | Omkar Vinit Joshi | +| [YARN-561](https://issues.apache.org/jira/browse/YARN-561) | Nodemanager should set some key information into the environment of every container that it launches. | Major | . | Hitesh Shah | Xuan Gong | +| [YARN-553](https://issues.apache.org/jira/browse/YARN-553) | Have YarnClient generate a directly usable ApplicationSubmissionContext | Minor | client | Harsh J | Karthik Kambatla | +| [YARN-536](https://issues.apache.org/jira/browse/YARN-536) | Remove ContainerStatus, ContainerState from Container api interface as they will not be called by the container object | Major | . | Xuan Gong | Xuan Gong | +| [YARN-530](https://issues.apache.org/jira/browse/YARN-530) | Define Service model strictly, implement AbstractService for robust subclassing, migrate yarn-common services | Major | . | Steve Loughran | Steve Loughran | +| [YARN-440](https://issues.apache.org/jira/browse/YARN-440) | Flatten RegisterNodeManagerResponse | Major | . | Siddharth Seth | Xuan Gong | +| [YARN-439](https://issues.apache.org/jira/browse/YARN-439) | Flatten NodeHeartbeatResponse | Major | . | Siddharth Seth | Xuan Gong | +| [YARN-396](https://issues.apache.org/jira/browse/YARN-396) | Rationalize AllocateResponse in RM scheduler API | Major | . | Bikas Saha | Zhijie Shen | +| [YARN-387](https://issues.apache.org/jira/browse/YARN-387) | Fix inconsistent protocol naming | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9763](https://issues.apache.org/jira/browse/HADOOP-9763) | Extends LightWeightGSet to support eviction of expired elements | Major | util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9515](https://issues.apache.org/jira/browse/HADOOP-9515) | Add general interface for NFS and Mount | Major | . | Brandon Li | Brandon Li | +| [HADOOP-9509](https://issues.apache.org/jira/browse/HADOOP-9509) | Implement ONCRPC and XDR | Major | . | Brandon Li | Brandon Li | +| [HADOOP-9283](https://issues.apache.org/jira/browse/HADOOP-9283) | Add support for running the Hadoop client on AIX | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-9209](https://issues.apache.org/jira/browse/HADOOP-9209) | Add shell command to dump file checksums | Major | fs, tools | Todd Lipcon | Todd Lipcon | +| [HADOOP-8562](https://issues.apache.org/jira/browse/HADOOP-8562) | Enhancements to support Hadoop on Windows Server and Windows Azure environments | Major | . | Bikas Saha | Bikas Saha | +| [HADOOP-8040](https://issues.apache.org/jira/browse/HADOOP-8040) | Add symlink support to FileSystem | Major | fs | Eli Collins | Andrew Wang | +| [HDFS-4249](https://issues.apache.org/jira/browse/HDFS-4249) | Add status NameNode startup to webUI | Major | namenode | Suresh Srinivas | Chris Nauroth | +| [HDFS-4124](https://issues.apache.org/jira/browse/HDFS-4124) | Refactor INodeDirectory#getExistingPathINodes() to enable returning more than INode array | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-3601](https://issues.apache.org/jira/browse/HDFS-3601) | Implementation of ReplicaPlacementPolicyNodeGroup to support 4-layer network topology | Major | namenode | Junping Du | Junping Du | +| [HDFS-3495](https://issues.apache.org/jira/browse/HDFS-3495) | Update Balancer to support new NetworkTopology with NodeGroup | Major | balancer & mover | Junping Du | Junping Du | +| [HDFS-3278](https://issues.apache.org/jira/browse/HDFS-3278) | Umbrella Jira for HDFS-HA Phase 2 | Major | . | Sanjay Radia | Todd Lipcon | +| [HDFS-2576](https://issues.apache.org/jira/browse/HDFS-2576) | Namenode should have a favored nodes hint to enable clients to have control over block placement. | Major | hdfs-client, namenode | Pritam Damania | Devaraj Das | +| [HDFS-1804](https://issues.apache.org/jira/browse/HDFS-1804) | Add a new block-volume device choosing policy that looks at free space | Minor | datanode | Harsh J | Aaron T. Myers | +| [MAPREDUCE-5298](https://issues.apache.org/jira/browse/MAPREDUCE-5298) | Move MapReduce services to YARN-117 stricter lifecycle | Major | applicationmaster | Steve Loughran | Steve Loughran | +| [MAPREDUCE-5129](https://issues.apache.org/jira/browse/MAPREDUCE-5129) | Add tag info to JH files | Minor | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-326](https://issues.apache.org/jira/browse/YARN-326) | Add multi-resource scheduling to the fair scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9845](https://issues.apache.org/jira/browse/HADOOP-9845) | Update protobuf to 2.5 from 2.4.x | Blocker | performance | stack | Alejandro Abdelnur | +| [HADOOP-9792](https://issues.apache.org/jira/browse/HADOOP-9792) | Retry the methods that are tagged @AtMostOnce along with @Idempotent | Major | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9770](https://issues.apache.org/jira/browse/HADOOP-9770) | Make RetryCache#state non volatile | Minor | util | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9762](https://issues.apache.org/jira/browse/HADOOP-9762) | RetryCache utility for implementing RPC retries | Major | util | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9760](https://issues.apache.org/jira/browse/HADOOP-9760) | Move GSet and LightWeightGSet to hadoop-common | Major | util | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9756](https://issues.apache.org/jira/browse/HADOOP-9756) | Additional cleanup RPC code | Minor | ipc | Junping Du | Junping Du | +| [HADOOP-9754](https://issues.apache.org/jira/browse/HADOOP-9754) | Clean up RPC code | Minor | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9751](https://issues.apache.org/jira/browse/HADOOP-9751) | Add clientId and retryCount to RpcResponseHeaderProto | Major | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9734](https://issues.apache.org/jira/browse/HADOOP-9734) | Common protobuf definitions for GetUserMappingsProtocol, RefreshAuthorizationPolicyProtocol and RefreshUserMappingsProtocol | Minor | ipc | Jason Lowe | Jason Lowe | +| [HADOOP-9717](https://issues.apache.org/jira/browse/HADOOP-9717) | Add retry attempt count to the RPC requests | Major | ipc | Suresh Srinivas | Jing Zhao | +| [HADOOP-9716](https://issues.apache.org/jira/browse/HADOOP-9716) | Move the Rpc request call ID generation to client side InvocationHandler | Major | ipc | Suresh Srinivas | Tsz Wo Nicholas Sze | +| [HADOOP-9691](https://issues.apache.org/jira/browse/HADOOP-9691) | RPC clients can generate call ID using AtomicInteger instead of synchronizing on the Client instance. | Minor | ipc | Chris Nauroth | Chris Nauroth | +| [HADOOP-9688](https://issues.apache.org/jira/browse/HADOOP-9688) | Add globally unique Client ID to RPC requests | Blocker | ipc | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9676](https://issues.apache.org/jira/browse/HADOOP-9676) | make maximum RPC buffer size configurable | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9673](https://issues.apache.org/jira/browse/HADOOP-9673) | NetworkTopology: when a node can't be added, print out its location for diagnostic purposes | Trivial | net | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9661](https://issues.apache.org/jira/browse/HADOOP-9661) | Allow metrics sources to be extended | Major | metrics | Sandy Ryza | Sandy Ryza | +| [HADOOP-9625](https://issues.apache.org/jira/browse/HADOOP-9625) | HADOOP\_OPTS not picked up by hadoop command | Minor | bin, conf | Paul Han | Paul Han | +| [HADOOP-9605](https://issues.apache.org/jira/browse/HADOOP-9605) | Update junit dependency | Major | build | Timothy St. Clair | | +| [HADOOP-9604](https://issues.apache.org/jira/browse/HADOOP-9604) | Wrong Javadoc of FSDataOutputStream | Minor | fs | Jingguo Yao | Jingguo Yao | +| [HADOOP-9560](https://issues.apache.org/jira/browse/HADOOP-9560) | metrics2#JvmMetrics should have max memory size of JVM | Minor | metrics | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-9523](https://issues.apache.org/jira/browse/HADOOP-9523) | Provide a generic IBM java vendor flag in PlatformName.java to support non-Sun JREs | Major | . | Tian Hong Wang | Tian Hong Wang | +| [HADOOP-9511](https://issues.apache.org/jira/browse/HADOOP-9511) | Adding support for additional input streams (FSDataInputStream and RandomAccessFile) in SecureIOUtils. | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [HADOOP-9503](https://issues.apache.org/jira/browse/HADOOP-9503) | Remove sleep between IPC client connect timeouts | Minor | ipc | Varun Sharma | Varun Sharma | +| [HADOOP-9483](https://issues.apache.org/jira/browse/HADOOP-9483) | winutils support for readlink command | Major | util | Chris Nauroth | Arpit Agarwal | +| [HADOOP-9450](https://issues.apache.org/jira/browse/HADOOP-9450) | HADOOP\_USER\_CLASSPATH\_FIRST is not honored; CLASSPATH is PREpended instead of APpended | Major | scripts | Mitch Wyle | Harsh J | +| [HADOOP-9401](https://issues.apache.org/jira/browse/HADOOP-9401) | CodecPool: Add counters for number of (de)compressors leased out | Major | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9379](https://issues.apache.org/jira/browse/HADOOP-9379) | capture the ulimit info after printing the log to the console | Trivial | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9352](https://issues.apache.org/jira/browse/HADOOP-9352) | Expose UGI.setLoginUser for tests | Major | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9343](https://issues.apache.org/jira/browse/HADOOP-9343) | Allow additional exceptions through the RPC layer | Major | . | Siddharth Seth | Siddharth Seth | +| [HADOOP-9338](https://issues.apache.org/jira/browse/HADOOP-9338) | FsShell Copy Commands Should Optionally Preserve File Attributes | Major | fs | Nick White | Nick White | +| [HADOOP-9336](https://issues.apache.org/jira/browse/HADOOP-9336) | Allow UGI of current connection to be queried | Critical | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9334](https://issues.apache.org/jira/browse/HADOOP-9334) | Update netty version | Minor | build | Nicolas Liochon | Nicolas Liochon | +| [HADOOP-9322](https://issues.apache.org/jira/browse/HADOOP-9322) | LdapGroupsMapping doesn't seem to set a timeout for its directory search | Minor | security | Harsh J | Harsh J | +| [HADOOP-9318](https://issues.apache.org/jira/browse/HADOOP-9318) | when exiting on a signal, print the signal name first | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9279](https://issues.apache.org/jira/browse/HADOOP-9279) | Document the need to build hadoop-maven-plugins for eclipse and separate project builds | Major | build, documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-9253](https://issues.apache.org/jira/browse/HADOOP-9253) | Capture ulimit info in the logs at service start time | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9227](https://issues.apache.org/jira/browse/HADOOP-9227) | FileSystemContractBaseTest doesn't test filesystem's mkdir/isDirectory() logic rigorously enough | Trivial | fs | Steve Loughran | Steve Loughran | +| [HADOOP-9164](https://issues.apache.org/jira/browse/HADOOP-9164) | Print paths of loaded native libraries in NativeLibraryChecker | Minor | native | Binglin Chang | Binglin Chang | +| [HADOOP-9117](https://issues.apache.org/jira/browse/HADOOP-9117) | replace protoc ant plugin exec with a maven plugin | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-8924](https://issues.apache.org/jira/browse/HADOOP-8924) | Add maven plugin alternative to shell script to save package-info.java | Major | build | Chris Nauroth | Chris Nauroth | +| [HADOOP-8711](https://issues.apache.org/jira/browse/HADOOP-8711) | provide an option for IPC server users to avoid printing stack information for certain exceptions | Major | ipc | Brandon Li | Brandon Li | +| [HADOOP-8608](https://issues.apache.org/jira/browse/HADOOP-8608) | Add Configuration API for parsing time durations | Minor | conf | Todd Lipcon | Chris Douglas | +| [HADOOP-8462](https://issues.apache.org/jira/browse/HADOOP-8462) | Native-code implementation of bzip2 codec | Major | io | Govind Kamat | Govind Kamat | +| [HADOOP-8415](https://issues.apache.org/jira/browse/HADOOP-8415) | getDouble() and setDouble() in org.apache.hadoop.conf.Configuration | Minor | conf | Jan van der Lugt | Jan van der Lugt | +| [HDFS-5027](https://issues.apache.org/jira/browse/HDFS-5027) | On startup, DN should scan volumes in parallel | Major | datanode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5020](https://issues.apache.org/jira/browse/HDFS-5020) | Make DatanodeProtocol#blockReceivedAndDeleted idempotent | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5010](https://issues.apache.org/jira/browse/HDFS-5010) | Reduce the frequency of getCurrentUser() calls from namenode | Major | namenode, performance | Kihwal Lee | Kihwal Lee | +| [HDFS-5008](https://issues.apache.org/jira/browse/HDFS-5008) | Make ClientProtocol#abandonBlock() idempotent | Major | namenode | Suresh Srinivas | Jing Zhao | +| [HDFS-5007](https://issues.apache.org/jira/browse/HDFS-5007) | Replace hard-coded property keys with DFSConfigKeys fields | Minor | . | Kousuke Saruta | Kousuke Saruta | +| [HDFS-4996](https://issues.apache.org/jira/browse/HDFS-4996) | ClientProtocol#metaSave can be made idempotent by overwriting the output file instead of appending to it | Minor | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-4992](https://issues.apache.org/jira/browse/HDFS-4992) | Make balancer's thread count configurable | Major | balancer & mover | Max Lapan | Max Lapan | +| [HDFS-4978](https://issues.apache.org/jira/browse/HDFS-4978) | Make disallowSnapshot idempotent | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-4942](https://issues.apache.org/jira/browse/HDFS-4942) | Add retry cache support in Namenode | Major | ha, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4932](https://issues.apache.org/jira/browse/HDFS-4932) | Avoid a wide line on the name node webUI if we have more Journal nodes | Minor | ha, namenode | Fengdong Yu | Fengdong Yu | +| [HDFS-4914](https://issues.apache.org/jira/browse/HDFS-4914) | When possible, Use DFSClient.Conf instead of Configuration | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4912](https://issues.apache.org/jira/browse/HDFS-4912) | Cleanup FSNamesystem#startFileInternal | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4903](https://issues.apache.org/jira/browse/HDFS-4903) | Print trash configuration and trash emptier state in namenode log | Minor | namenode | Suresh Srinivas | Arpit Agarwal | +| [HDFS-4848](https://issues.apache.org/jira/browse/HDFS-4848) | copyFromLocal and renaming a file to ".snapshot" should output that ".snapshot" is a reserved name | Minor | snapshots | Stephen Chu | Jing Zhao | +| [HDFS-4804](https://issues.apache.org/jira/browse/HDFS-4804) | WARN when users set the block balanced preference percent below 0.5 or above 1.0 | Minor | . | Stephen Chu | Stephen Chu | +| [HDFS-4787](https://issues.apache.org/jira/browse/HDFS-4787) | Create a new HdfsConfiguration before each TestDFSClientRetries testcases | Major | . | Tian Hong Wang | Tian Hong Wang | +| [HDFS-4772](https://issues.apache.org/jira/browse/HDFS-4772) | Add number of children in HdfsFileStatus | Minor | namenode | Brandon Li | Brandon Li | +| [HDFS-4721](https://issues.apache.org/jira/browse/HDFS-4721) | Speed up lease/block recovery when DN fails and a block goes into recovery | Major | namenode | Varun Sharma | Varun Sharma | +| [HDFS-4698](https://issues.apache.org/jira/browse/HDFS-4698) | provide client-side metrics for remote reads, local reads, and short-circuit reads | Minor | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4679](https://issues.apache.org/jira/browse/HDFS-4679) | Namenode operation checks should be done in a consistent manner | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4645](https://issues.apache.org/jira/browse/HDFS-4645) | Move from randomly generated block ID to sequentially generated block ID | Major | namenode | Suresh Srinivas | Arpit Agarwal | +| [HDFS-4635](https://issues.apache.org/jira/browse/HDFS-4635) | Move BlockManager#computeCapacity to LightWeightGSet | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4569](https://issues.apache.org/jira/browse/HDFS-4569) | Small image transfer related cleanups. | Trivial | . | Andrew Wang | Andrew Wang | +| [HDFS-4565](https://issues.apache.org/jira/browse/HDFS-4565) | use DFSUtil.getSpnegoKeytabKey() to get the spnego keytab key in secondary namenode and namenode http server | Minor | security | Arpit Gupta | Arpit Gupta | +| [HDFS-4521](https://issues.apache.org/jira/browse/HDFS-4521) | invalid network topologies should not be cached | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4465](https://issues.apache.org/jira/browse/HDFS-4465) | Optimize datanode ReplicasMap and ReplicaInfo | Major | datanode | Suresh Srinivas | Aaron T. Myers | +| [HDFS-4461](https://issues.apache.org/jira/browse/HDFS-4461) | DirectoryScanner: volume path prefix takes up memory for every block that is scanned | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4304](https://issues.apache.org/jira/browse/HDFS-4304) | Make FSEditLogOp.MAX\_OP\_SIZE configurable | Major | namenode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-4246](https://issues.apache.org/jira/browse/HDFS-4246) | The exclude node list should be more forgiving, for each output stream | Minor | hdfs-client | Harsh J | Harsh J | +| [HDFS-4234](https://issues.apache.org/jira/browse/HDFS-4234) | Use the generic code for choosing datanode in Balancer | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4215](https://issues.apache.org/jira/browse/HDFS-4215) | Improvements on INode and image loading | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4206](https://issues.apache.org/jira/browse/HDFS-4206) | Change the fields in INode and its subclasses to private | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4152](https://issues.apache.org/jira/browse/HDFS-4152) | Add a new class for the parameter in INode.collectSubtreeBlocksAndClear(..) | Minor | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4151](https://issues.apache.org/jira/browse/HDFS-4151) | Passing INodesInPath instead of INode[] in FSDirectory | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4053](https://issues.apache.org/jira/browse/HDFS-4053) | Increase the default block size | Major | . | Eli Collins | Eli Collins | +| [HDFS-3940](https://issues.apache.org/jira/browse/HDFS-3940) | Add Gset#clear method and clear the block map when namenode is shutdown | Minor | . | Eli Collins | Suresh Srinivas | +| [HDFS-3880](https://issues.apache.org/jira/browse/HDFS-3880) | Use Builder to get RPC server in HDFS | Minor | datanode, ha, namenode, security | Brandon Li | Brandon Li | +| [HDFS-3817](https://issues.apache.org/jira/browse/HDFS-3817) | avoid printing stack information for SafeModeException | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-3498](https://issues.apache.org/jira/browse/HDFS-3498) | Make Replica Removal Policy pluggable and ReplicaPlacementPolicyDefault extensible for reusing code in subclass | Major | namenode | Junping Du | Junping Du | +| [HDFS-3163](https://issues.apache.org/jira/browse/HDFS-3163) | TestHDFSCLI.testAll fails if the user name is not all lowercase | Trivial | test | Brandon Li | Brandon Li | +| [HDFS-2857](https://issues.apache.org/jira/browse/HDFS-2857) | Cleanup BlockInfo class | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-2572](https://issues.apache.org/jira/browse/HDFS-2572) | Unnecessary double-check in DN#getHostName | Trivial | datanode | Harsh J | Harsh J | +| [HDFS-2042](https://issues.apache.org/jira/browse/HDFS-2042) | Require c99 when building libhdfs | Minor | libhdfs | Eli Collins | | +| [HDFS-347](https://issues.apache.org/jira/browse/HDFS-347) | DFS read performance suboptimal when client co-located on nodes with data | Major | datanode, hdfs-client, performance | George Porter | Colin Patrick McCabe | +| [MAPREDUCE-5398](https://issues.apache.org/jira/browse/MAPREDUCE-5398) | MR changes for YARN-513 | Major | . | Bikas Saha | Jian He | +| [MAPREDUCE-5352](https://issues.apache.org/jira/browse/MAPREDUCE-5352) | Optimize node local splits generated by CombineFileInputFormat | Major | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-5283](https://issues.apache.org/jira/browse/MAPREDUCE-5283) | Over 10 different tests have near identical implementations of AppContext | Major | applicationmaster, test | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5268](https://issues.apache.org/jira/browse/MAPREDUCE-5268) | Improve history server startup performance | Major | jobhistoryserver | Jason Lowe | Karthik Kambatla | +| [MAPREDUCE-5246](https://issues.apache.org/jira/browse/MAPREDUCE-5246) | Adding application type to submission context | Major | . | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-5171](https://issues.apache.org/jira/browse/MAPREDUCE-5171) | Expose blacklisted nodes from the MR AM REST API | Major | applicationmaster | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5128](https://issues.apache.org/jira/browse/MAPREDUCE-5128) | mapred-default.xml is missing a bunch of history server configs | Major | documentation, jobhistoryserver | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5079](https://issues.apache.org/jira/browse/MAPREDUCE-5079) | Recovery should restore task state from job history info directly | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5069](https://issues.apache.org/jira/browse/MAPREDUCE-5069) | add concrete common implementations of CombineFileInputFormat | Minor | mrv1, mrv2 | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-5033](https://issues.apache.org/jira/browse/MAPREDUCE-5033) | mapred shell script should respect usage flags (--help -help -h) | Minor | . | Andrew Wang | Andrew Wang | +| [MAPREDUCE-4990](https://issues.apache.org/jira/browse/MAPREDUCE-4990) | Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput() | Trivial | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4989](https://issues.apache.org/jira/browse/MAPREDUCE-4989) | JSONify DataTables input data for Attempts page | Major | jobhistoryserver, mr-am | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-4974](https://issues.apache.org/jira/browse/MAPREDUCE-4974) | Optimising the LineRecordReader initialize() method | Major | mrv1, mrv2, performance | Arun A K | Gelesh | +| [MAPREDUCE-4846](https://issues.apache.org/jira/browse/MAPREDUCE-4846) | Some JobQueueInfo methods are public in MR1 but protected in MR2 | Major | client | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-3787](https://issues.apache.org/jira/browse/MAPREDUCE-3787) | [Gridmix] Improve STRESS mode | Major | contrib/gridmix | Amar Kamat | Amar Kamat | +| [MAPREDUCE-3533](https://issues.apache.org/jira/browse/MAPREDUCE-3533) | have the service interface extend Closeable and use close() as its shutdown operation | Minor | mrv2 | Steve Loughran | | +| [YARN-1045](https://issues.apache.org/jira/browse/YARN-1045) | Improve toString implementation for PBImpls | Major | . | Siddharth Seth | Jian He | +| [YARN-883](https://issues.apache.org/jira/browse/YARN-883) | Expose Fair Scheduler-specific queue metrics | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-865](https://issues.apache.org/jira/browse/YARN-865) | RM webservices can't query based on application Types | Major | . | Xuan Gong | Xuan Gong | +| [YARN-803](https://issues.apache.org/jira/browse/YARN-803) | factor out scheduler config validation from the ResourceManager to each scheduler implementation | Major | resourcemanager, scheduler | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-789](https://issues.apache.org/jira/browse/YARN-789) | Enable zero capabilities resource requests in fair scheduler | Major | scheduler | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-782](https://issues.apache.org/jira/browse/YARN-782) | vcores-pcores ratio functions differently from vmem-pmem ratio in misleading way | Critical | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-752](https://issues.apache.org/jira/browse/YARN-752) | In AMRMClient, automatically add corresponding rack requests for requested nodes | Major | api, applications | Sandy Ryza | Sandy Ryza | +| [YARN-736](https://issues.apache.org/jira/browse/YARN-736) | Add a multi-resource fair sharing metric | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-600](https://issues.apache.org/jira/browse/YARN-600) | Hook up cgroups CPU settings to the number of virtual cores allocated | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-598](https://issues.apache.org/jira/browse/YARN-598) | Add virtual cores to queue metrics | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-590](https://issues.apache.org/jira/browse/YARN-590) | Add an optional mesage to RegisterNodeManagerResponse as to why NM is being asked to resync or shutdown | Major | . | Vinod Kumar Vavilapalli | Mayank Bansal | +| [YARN-538](https://issues.apache.org/jira/browse/YARN-538) | RM address DNS lookup can cause unnecessary slowness on every JHS page load | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-525](https://issues.apache.org/jira/browse/YARN-525) | make CS node-locality-delay refreshable | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-518](https://issues.apache.org/jira/browse/YARN-518) | Fair Scheduler's document link could be added to the hadoop 2.x main doc page | Major | documentation | Dapeng Sun | Sandy Ryza | +| [YARN-447](https://issues.apache.org/jira/browse/YARN-447) | applicationComparator improvement for CS | Minor | scheduler | Nemon Lou | Nemon Lou | +| [YARN-406](https://issues.apache.org/jira/browse/YARN-406) | TestRackResolver fails when local network resolves "host1" to a valid host | Minor | . | Hitesh Shah | Hitesh Shah | +| [YARN-391](https://issues.apache.org/jira/browse/YARN-391) | detabify LCEResourcesHandler classes | Trivial | nodemanager | Steve Loughran | Steve Loughran | +| [YARN-385](https://issues.apache.org/jira/browse/YARN-385) | ResourceRequestPBImpl's toString() is missing location and # containers | Major | api | Sandy Ryza | Sandy Ryza | +| [YARN-382](https://issues.apache.org/jira/browse/YARN-382) | SchedulerUtils improve way normalizeRequest sets the resource capabilities | Major | scheduler | Thomas Graves | Zhijie Shen | +| [YARN-381](https://issues.apache.org/jira/browse/YARN-381) | Improve FS docs | Minor | documentation | Eli Collins | Sandy Ryza | +| [YARN-347](https://issues.apache.org/jira/browse/YARN-347) | YARN CLI should show CPU info besides memory info in node status | Major | client | Junping Du | Junping Du | +| [YARN-297](https://issues.apache.org/jira/browse/YARN-297) | Improve hashCode implementations for PB records | Major | . | Arun C Murthy | Xuan Gong | +| [YARN-249](https://issues.apache.org/jira/browse/YARN-249) | Capacity Scheduler web page should show list of active users per queue like it used to (in 1.x) | Major | capacityscheduler | Ravi Prakash | Ravi Prakash | +| [YARN-237](https://issues.apache.org/jira/browse/YARN-237) | Refreshing the RM page forgets how many rows I had in my Datatables | Major | resourcemanager | Ravi Prakash | Jian He | +| [YARN-198](https://issues.apache.org/jira/browse/YARN-198) | If we are navigating to Nodemanager UI from Resourcemanager,then there is not link to navigate back to Resource manager | Minor | nodemanager | Ramgopal N | Jian He | +| [YARN-117](https://issues.apache.org/jira/browse/YARN-117) | Enhance YARN service model | Major | . | Steve Loughran | Steve Loughran | +| [YARN-84](https://issues.apache.org/jira/browse/YARN-84) | Use Builder to get RPC server in YARN | Minor | . | Brandon Li | Brandon Li | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9872](https://issues.apache.org/jira/browse/HADOOP-9872) | Improve protoc version handling and detection | Blocker | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-9850](https://issues.apache.org/jira/browse/HADOOP-9850) | RPC kerberos errors don't trigger relogin | Blocker | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9816](https://issues.apache.org/jira/browse/HADOOP-9816) | RPC Sasl QOP is broken | Blocker | ipc, security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9787](https://issues.apache.org/jira/browse/HADOOP-9787) | ShutdownHelper util to shutdown threads and threadpools | Major | util | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9786](https://issues.apache.org/jira/browse/HADOOP-9786) | RetryInvocationHandler#isRpcInvocation should support ProtocolTranslator | Major | . | Jing Zhao | Jing Zhao | +| [HADOOP-9773](https://issues.apache.org/jira/browse/HADOOP-9773) | TestLightWeightCache fails | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9759](https://issues.apache.org/jira/browse/HADOOP-9759) | Add support for NativeCodeLoader#getLibraryName on Windows | Critical | . | Chuan Liu | Chuan Liu | +| [HADOOP-9738](https://issues.apache.org/jira/browse/HADOOP-9738) | TestDistCh fails | Major | tools | Kihwal Lee | Jing Zhao | +| [HADOOP-9707](https://issues.apache.org/jira/browse/HADOOP-9707) | Fix register lists for crc32c inline assembly | Minor | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-9701](https://issues.apache.org/jira/browse/HADOOP-9701) | mvn site ambiguous links in hadoop-common | Minor | documentation | Steve Loughran | Karthik Kambatla | +| [HADOOP-9681](https://issues.apache.org/jira/browse/HADOOP-9681) | FileUtil.unTarUsingJava() should close the InputStream upon finishing | Minor | . | Chuan Liu | Chuan Liu | +| [HADOOP-9678](https://issues.apache.org/jira/browse/HADOOP-9678) | TestRPC#testStopsAllThreads intermittently fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9665](https://issues.apache.org/jira/browse/HADOOP-9665) | BlockDecompressorStream#decompress will throw EOFException instead of return -1 when EOF | Critical | . | Zhijie Shen | Zhijie Shen | +| [HADOOP-9656](https://issues.apache.org/jira/browse/HADOOP-9656) | Gridmix unit tests fail on Windows and Linux | Minor | test, tools | Chuan Liu | Chuan Liu | +| [HADOOP-9643](https://issues.apache.org/jira/browse/HADOOP-9643) | org.apache.hadoop.security.SecurityUtil calls toUpperCase(Locale.getDefault()) as well as toLowerCase(Locale.getDefault()) on hadoop.security.authentication value. | Minor | security | Mark Miller | Mark Miller | +| [HADOOP-9638](https://issues.apache.org/jira/browse/HADOOP-9638) | parallel test changes caused invalid test path for several HDFS tests on Windows | Major | test | Chris Nauroth | Andrey Klochkov | +| [HADOOP-9637](https://issues.apache.org/jira/browse/HADOOP-9637) | Adding Native Fstat for Windows as needed by YARN | Major | . | Chuan Liu | Chuan Liu | +| [HADOOP-9632](https://issues.apache.org/jira/browse/HADOOP-9632) | TestShellCommandFencer will fail if there is a 'host' machine in the network | Minor | . | Chuan Liu | Chuan Liu | +| [HADOOP-9607](https://issues.apache.org/jira/browse/HADOOP-9607) | Fixes in Javadoc build | Minor | documentation | Timothy St. Clair | | +| [HADOOP-9599](https://issues.apache.org/jira/browse/HADOOP-9599) | hadoop-config.cmd doesn't set JAVA\_LIBRARY\_PATH correctly | Major | . | Mostafa Elhemali | Mostafa Elhemali | +| [HADOOP-9593](https://issues.apache.org/jira/browse/HADOOP-9593) | stack trace printed at ERROR for all yarn clients without hadoop.home set | Major | util | Steve Loughran | Steve Loughran | +| [HADOOP-9581](https://issues.apache.org/jira/browse/HADOOP-9581) | hadoop --config non-existent directory should result in error | Major | scripts | Ashwin Shankar | Ashwin Shankar | +| [HADOOP-9574](https://issues.apache.org/jira/browse/HADOOP-9574) | Add new methods in AbstractDelegationTokenSecretManager for restoring RMDelegationTokens on RMRestart | Major | . | Jian He | Jian He | +| [HADOOP-9566](https://issues.apache.org/jira/browse/HADOOP-9566) | Performing direct read using libhdfs sometimes raises SIGPIPE (which in turn throws SIGABRT) causing client crashes | Major | native | Lenni Kuff | Colin Patrick McCabe | +| [HADOOP-9563](https://issues.apache.org/jira/browse/HADOOP-9563) | Fix incompatibility introduced by HADOOP-9523 | Major | util | Kihwal Lee | Tian Hong Wang | +| [HADOOP-9556](https://issues.apache.org/jira/browse/HADOOP-9556) | disable HA tests on Windows that fail due to ZooKeeper client connection management bug | Major | ha, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9553](https://issues.apache.org/jira/browse/HADOOP-9553) | TestAuthenticationToken fails on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9550](https://issues.apache.org/jira/browse/HADOOP-9550) | Remove aspectj dependency | Major | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9549](https://issues.apache.org/jira/browse/HADOOP-9549) | WebHdfsFileSystem hangs on close() | Blocker | security | Kihwal Lee | Daryn Sharp | +| [HADOOP-9532](https://issues.apache.org/jira/browse/HADOOP-9532) | HADOOP\_CLIENT\_OPTS is appended twice by Windows cmd scripts | Minor | bin | Chris Nauroth | Chris Nauroth | +| [HADOOP-9527](https://issues.apache.org/jira/browse/HADOOP-9527) | Add symlink support to LocalFileSystem on Windows | Major | fs, test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9526](https://issues.apache.org/jira/browse/HADOOP-9526) | TestShellCommandFencer and TestShell fail on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9524](https://issues.apache.org/jira/browse/HADOOP-9524) | Fix ShellCommandFencer to work on Windows | Major | ha | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9517](https://issues.apache.org/jira/browse/HADOOP-9517) | Document Hadoop Compatibility | Blocker | documentation | Arun C Murthy | Karthik Kambatla | +| [HADOOP-9507](https://issues.apache.org/jira/browse/HADOOP-9507) | LocalFileSystem rename() is broken in some cases when destination exists | Minor | fs | Mostafa Elhemali | Chris Nauroth | +| [HADOOP-9504](https://issues.apache.org/jira/browse/HADOOP-9504) | MetricsDynamicMBeanBase has concurrency issues in createMBeanInfo | Critical | metrics | Liang Xie | Liang Xie | +| [HADOOP-9500](https://issues.apache.org/jira/browse/HADOOP-9500) | TestUserGroupInformation#testGetServerSideGroups fails on Windows due to failure to find winutils.exe | Major | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9496](https://issues.apache.org/jira/browse/HADOOP-9496) | Bad merge of HADOOP-9450 on branch-2 breaks all bin/hadoop calls that need HADOOP\_CLASSPATH | Critical | bin | Gopal V | Harsh J | +| [HADOOP-9490](https://issues.apache.org/jira/browse/HADOOP-9490) | LocalFileSystem#reportChecksumFailure not closing the checksum file handle before rename | Major | fs | Ivan Mitic | Ivan Mitic | +| [HADOOP-9488](https://issues.apache.org/jira/browse/HADOOP-9488) | FileUtil#createJarWithClassPath only substitutes environment variables from current process environment/does not support overriding when launching new process | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9486](https://issues.apache.org/jira/browse/HADOOP-9486) | Promote Windows and Shell related utils from YARN to Hadoop Common | Major | . | Vinod Kumar Vavilapalli | Chris Nauroth | +| [HADOOP-9485](https://issues.apache.org/jira/browse/HADOOP-9485) | No default value in the code for hadoop.rpc.socket.factory.class.default | Minor | net | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9481](https://issues.apache.org/jira/browse/HADOOP-9481) | Broken conditional logic with HADOOP\_SNAPPY\_LIBRARY | Minor | . | Vadim Bondarev | Vadim Bondarev | +| [HADOOP-9473](https://issues.apache.org/jira/browse/HADOOP-9473) | typo in FileUtil copy() method | Trivial | fs | Glen Mazza | | +| [HADOOP-9469](https://issues.apache.org/jira/browse/HADOOP-9469) | mapreduce/yarn source jars not included in dist tarball | Major | . | Thomas Graves | Robert Parker | +| [HADOOP-9459](https://issues.apache.org/jira/browse/HADOOP-9459) | ActiveStandbyElector can join election even before Service HEALTHY, and results in null data at ActiveBreadCrumb | Critical | ha | Vinayakumar B | Vinayakumar B | +| [HADOOP-9455](https://issues.apache.org/jira/browse/HADOOP-9455) | HADOOP\_CLIENT\_OPTS appended twice causes JVM failures | Minor | bin | Sangjin Lee | Chris Nauroth | +| [HADOOP-9451](https://issues.apache.org/jira/browse/HADOOP-9451) | Node with one topology layer should be handled as fault topology when NodeGroup layer is enabled | Major | net | Junping Du | Junping Du | +| [HADOOP-9443](https://issues.apache.org/jira/browse/HADOOP-9443) | Port winutils static code analysis change to trunk | Major | . | Chuan Liu | Chuan Liu | +| [HADOOP-9439](https://issues.apache.org/jira/browse/HADOOP-9439) | JniBasedUnixGroupsMapping: fix some crash bugs | Minor | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9437](https://issues.apache.org/jira/browse/HADOOP-9437) | TestNativeIO#testRenameTo fails on Windows due to assumption that POSIX errno is embedded in NativeIOException | Major | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9430](https://issues.apache.org/jira/browse/HADOOP-9430) | TestSSLFactory fails on IBM JVM | Major | security | Amir Sanjar | | +| [HADOOP-9429](https://issues.apache.org/jira/browse/HADOOP-9429) | TestConfiguration fails with IBM JAVA | Major | test | Amir Sanjar | | +| [HADOOP-9413](https://issues.apache.org/jira/browse/HADOOP-9413) | Introduce common utils for File#setReadable/Writable/Executable and File#canRead/Write/Execute that work cross-platform | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9408](https://issues.apache.org/jira/browse/HADOOP-9408) | misleading description for net.topology.table.file.name property in core-default.xml | Minor | conf | rajeshbabu | rajeshbabu | +| [HADOOP-9407](https://issues.apache.org/jira/browse/HADOOP-9407) | commons-daemon 1.0.3 dependency has bad group id causing build issues | Major | build | Sangjin Lee | Sangjin Lee | +| [HADOOP-9397](https://issues.apache.org/jira/browse/HADOOP-9397) | Incremental dist tar build fails | Major | build | Jason Lowe | Chris Nauroth | +| [HADOOP-9388](https://issues.apache.org/jira/browse/HADOOP-9388) | TestFsShellCopy fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9387](https://issues.apache.org/jira/browse/HADOOP-9387) | TestDFVariations fails on Windows after the merge | Minor | fs | Ivan Mitic | Ivan Mitic | +| [HADOOP-9376](https://issues.apache.org/jira/browse/HADOOP-9376) | TestProxyUserFromEnv fails on a Windows domain joined machine | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9373](https://issues.apache.org/jira/browse/HADOOP-9373) | Merge CHANGES.branch-trunk-win.txt to CHANGES.txt | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-9372](https://issues.apache.org/jira/browse/HADOOP-9372) | Fix bad timeout annotations on tests | Major | . | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9369](https://issues.apache.org/jira/browse/HADOOP-9369) | DNS#reverseDns() can return hostname with . appended at the end | Major | net | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9365](https://issues.apache.org/jira/browse/HADOOP-9365) | TestHAZKUtil fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9364](https://issues.apache.org/jira/browse/HADOOP-9364) | PathData#expandAsGlob does not return correct results for absolute paths on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9358](https://issues.apache.org/jira/browse/HADOOP-9358) | "Auth failed" log should include exception string | Major | ipc, security | Todd Lipcon | Todd Lipcon | +| [HADOOP-9353](https://issues.apache.org/jira/browse/HADOOP-9353) | Activate native-win profile by default on Windows | Major | build | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-9349](https://issues.apache.org/jira/browse/HADOOP-9349) | Confusing output when running hadoop version from one hadoop installation when HADOOP\_HOME points to another | Major | tools | Sandy Ryza | Sandy Ryza | +| [HADOOP-9342](https://issues.apache.org/jira/browse/HADOOP-9342) | Remove jline from distribution | Major | build | Thomas Weise | Thomas Weise | +| [HADOOP-9339](https://issues.apache.org/jira/browse/HADOOP-9339) | IPC.Server incorrectly sets UGI auth type | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9337](https://issues.apache.org/jira/browse/HADOOP-9337) | org.apache.hadoop.fs.DF.getMount() does not work on Mac OS | Major | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9323](https://issues.apache.org/jira/browse/HADOOP-9323) | Typos in API documentation | Minor | documentation, fs, io, record | Hao Zhong | Suresh Srinivas | +| [HADOOP-9307](https://issues.apache.org/jira/browse/HADOOP-9307) | BufferedFSInputStream.read returns wrong results after certain seeks | Major | fs | Todd Lipcon | Todd Lipcon | +| [HADOOP-9305](https://issues.apache.org/jira/browse/HADOOP-9305) | Add support for running the Hadoop client on 64-bit AIX | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-9304](https://issues.apache.org/jira/browse/HADOOP-9304) | remove addition of avro genreated-sources dirs to build | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-9303](https://issues.apache.org/jira/browse/HADOOP-9303) | command manual dfsadmin missing entry for restoreFailedStorage option | Major | . | Thomas Graves | Andy Isaacson | +| [HADOOP-9302](https://issues.apache.org/jira/browse/HADOOP-9302) | HDFS docs not linked from top level | Major | documentation | Thomas Graves | Andy Isaacson | +| [HADOOP-9297](https://issues.apache.org/jira/browse/HADOOP-9297) | remove old record IO generation and tests | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-9294](https://issues.apache.org/jira/browse/HADOOP-9294) | GetGroupsTestBase fails on Windows | Major | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9290](https://issues.apache.org/jira/browse/HADOOP-9290) | Some tests cannot load native library | Major | build, native | Arpit Agarwal | Chris Nauroth | +| [HADOOP-9267](https://issues.apache.org/jira/browse/HADOOP-9267) | hadoop -help, -h, --help should show usage instructions | Minor | . | Andrew Wang | Andrew Wang | +| [HADOOP-9264](https://issues.apache.org/jira/browse/HADOOP-9264) | port change to use Java untar API on Windows from branch-1-win to trunk | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9246](https://issues.apache.org/jira/browse/HADOOP-9246) | Execution phase for hadoop-maven-plugin should be process-resources | Major | build | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9245](https://issues.apache.org/jira/browse/HADOOP-9245) | mvn clean without running mvn install before fails | Major | build | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9230](https://issues.apache.org/jira/browse/HADOOP-9230) | TestUniformSizeInputFormat fails intermittently | Major | test | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9228](https://issues.apache.org/jira/browse/HADOOP-9228) | FileSystemContractTestBase never verifies that files are files | Minor | fs | Steve Loughran | Steve Loughran | +| [HADOOP-9220](https://issues.apache.org/jira/browse/HADOOP-9220) | Unnecessary transition to standby in ActiveStandbyElector | Critical | ha | Tom White | Tom White | +| [HADOOP-9211](https://issues.apache.org/jira/browse/HADOOP-9211) | HADOOP\_CLIENT\_OPTS default setting fixes max heap size at 128m, disregards HADOOP\_HEAPSIZE | Major | conf | Sarah Weissman | Plamen Jeliazkov | +| [HADOOP-9154](https://issues.apache.org/jira/browse/HADOOP-9154) | SortedMapWritable#putAll() doesn't add key/value classes to the map | Major | io | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9150](https://issues.apache.org/jira/browse/HADOOP-9150) | Unnecessary DNS resolution attempts for logical URIs | Critical | fs/s3, ha, performance, viewfs | Todd Lipcon | Todd Lipcon | +| [HADOOP-9131](https://issues.apache.org/jira/browse/HADOOP-9131) | TestLocalFileSystem#testListStatusWithColons cannot run on Windows | Major | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9125](https://issues.apache.org/jira/browse/HADOOP-9125) | LdapGroupsMapping threw CommunicationException after some idle time | Major | security | Kai Zheng | Kai Zheng | +| [HADOOP-9043](https://issues.apache.org/jira/browse/HADOOP-9043) | disallow in winutils creating symlinks with forwards slashes | Major | util | Chris Nauroth | Chris Nauroth | +| [HADOOP-8982](https://issues.apache.org/jira/browse/HADOOP-8982) | TestSocketIOWithTimeout fails on Windows | Major | net | Chris Nauroth | Chris Nauroth | +| [HADOOP-8973](https://issues.apache.org/jira/browse/HADOOP-8973) | DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs | Major | util | Chris Nauroth | Chris Nauroth | +| [HADOOP-8958](https://issues.apache.org/jira/browse/HADOOP-8958) | ViewFs:Non absolute mount name failures when running multiple tests on Windows | Major | viewfs | Chris Nauroth | Chris Nauroth | +| [HADOOP-8957](https://issues.apache.org/jira/browse/HADOOP-8957) | AbstractFileSystem#IsValidName should be overridden for embedded file systems like ViewFs | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-8917](https://issues.apache.org/jira/browse/HADOOP-8917) | add LOCALE.US to toLowerCase in SecurityUtil.replacePattern | Major | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-8569](https://issues.apache.org/jira/browse/HADOOP-8569) | CMakeLists.txt: define \_GNU\_SOURCE and \_LARGEFILE\_SOURCE | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-8440](https://issues.apache.org/jira/browse/HADOOP-8440) | HarFileSystem.decodeHarURI fails for URIs whose host contains numbers | Minor | fs | Ivan Mitic | Ivan Mitic | +| [HADOOP-7487](https://issues.apache.org/jira/browse/HADOOP-7487) | DF should throw a more reasonable exception when mount cannot be determined | Major | fs | Todd Lipcon | Andrew Wang | +| [HADOOP-7391](https://issues.apache.org/jira/browse/HADOOP-7391) | Document Interface Classification from HADOOP-5073 | Major | . | Sanjay Radia | Sanjay Radia | +| [HDFS-5024](https://issues.apache.org/jira/browse/HDFS-5024) | Make DatanodeProtocol#commitBlockSynchronization idempotent | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-5018](https://issues.apache.org/jira/browse/HDFS-5018) | Misspelled DFSConfigKeys#DFS\_NAMENODE\_STALE\_DATANODE\_INTERVAL\_DEFAULT in javadoc of DatanodeInfo#isStale() | Minor | . | Ted Yu | Ted Yu | +| [HDFS-5016](https://issues.apache.org/jira/browse/HDFS-5016) | Deadlock in pipeline recovery causes Datanode to be marked dead | Blocker | . | Devaraj Das | Suresh Srinivas | +| [HDFS-5005](https://issues.apache.org/jira/browse/HDFS-5005) | Move SnapshotException and SnapshotAccessControlException to o.a.h.hdfs.protocol | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-5003](https://issues.apache.org/jira/browse/HDFS-5003) | TestNNThroughputBenchmark failed caused by existing directories | Minor | test | Xi Fang | Xi Fang | +| [HDFS-4999](https://issues.apache.org/jira/browse/HDFS-4999) | fix TestShortCircuitLocalRead on branch-2 | Major | . | Kihwal Lee | Colin Patrick McCabe | +| [HDFS-4998](https://issues.apache.org/jira/browse/HDFS-4998) | TestUnderReplicatedBlocks fails intermittently | Major | test | Kihwal Lee | Kihwal Lee | +| [HDFS-4982](https://issues.apache.org/jira/browse/HDFS-4982) | JournalNode should relogin from keytab before fetching logs from other JNs | Major | journal-node, security | Todd Lipcon | Todd Lipcon | +| [HDFS-4980](https://issues.apache.org/jira/browse/HDFS-4980) | Incorrect logging.properties file for hadoop-httpfs | Major | build | Mark Grover | Mark Grover | +| [HDFS-4969](https://issues.apache.org/jira/browse/HDFS-4969) | WebhdfsFileSystem expects non-standard WEBHDFS Json element | Blocker | test, webhdfs | Robert Kanter | Robert Kanter | +| [HDFS-4954](https://issues.apache.org/jira/browse/HDFS-4954) | compile failure in branch-2: getFlushedOffset should catch or rethrow IOException | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-4951](https://issues.apache.org/jira/browse/HDFS-4951) | FsShell commands using secure httpfs throw exceptions due to missing TokenRenewer | Major | security | Robert Kanter | Robert Kanter | +| [HDFS-4948](https://issues.apache.org/jira/browse/HDFS-4948) | mvn site for hadoop-hdfs-nfs fails | Major | . | Robert Joseph Evans | Brandon Li | +| [HDFS-4944](https://issues.apache.org/jira/browse/HDFS-4944) | WebHDFS cannot create a file path containing characters that must be URI-encoded, such as space. | Major | webhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-4943](https://issues.apache.org/jira/browse/HDFS-4943) | WebHdfsFileSystem does not work when original file path has encoded chars | Minor | webhdfs | Jerry He | Jerry He | +| [HDFS-4927](https://issues.apache.org/jira/browse/HDFS-4927) | CreateEditsLog creates inodes with an invalid inode ID, which then cannot be loaded by a namenode. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4917](https://issues.apache.org/jira/browse/HDFS-4917) | Start-dfs.sh cannot pass the parameters correctly | Major | datanode, namenode | Fengdong Yu | Fengdong Yu | +| [HDFS-4910](https://issues.apache.org/jira/browse/HDFS-4910) | TestPermission failed in branch-2 | Major | . | Chuan Liu | Chuan Liu | +| [HDFS-4906](https://issues.apache.org/jira/browse/HDFS-4906) | HDFS Output streams should not accept writes after being closed | Major | hdfs-client | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4902](https://issues.apache.org/jira/browse/HDFS-4902) | DFSClient.getSnapshotDiffReport should use string path rather than o.a.h.fs.Path | Major | snapshots | Binglin Chang | Binglin Chang | +| [HDFS-4888](https://issues.apache.org/jira/browse/HDFS-4888) | Refactor and fix FSNamesystem.getTurnOffTip to sanity | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-4887](https://issues.apache.org/jira/browse/HDFS-4887) | TestNNThroughputBenchmark exits abruptly | Blocker | benchmarks, test | Kihwal Lee | Kihwal Lee | +| [HDFS-4883](https://issues.apache.org/jira/browse/HDFS-4883) | complete() should verify fileId | Major | namenode | Konstantin Shvachko | Tao Luo | +| [HDFS-4880](https://issues.apache.org/jira/browse/HDFS-4880) | Diagnostic logging while loading name/edits files | Major | namenode | Arpit Agarwal | Suresh Srinivas | +| [HDFS-4878](https://issues.apache.org/jira/browse/HDFS-4878) | On Remove Block, Block is not Removed from neededReplications queue | Major | namenode | Tao Luo | Tao Luo | +| [HDFS-4877](https://issues.apache.org/jira/browse/HDFS-4877) | Snapshot: fix the scenario where a directory is renamed under its prior descendant | Blocker | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-4873](https://issues.apache.org/jira/browse/HDFS-4873) | callGetBlockLocations returns incorrect number of blocks for snapshotted files | Major | snapshots | Hari Mankude | Jing Zhao | +| [HDFS-4867](https://issues.apache.org/jira/browse/HDFS-4867) | metaSave NPEs when there are invalid blocks in repl queue. | Major | namenode | Kihwal Lee | Plamen Jeliazkov | +| [HDFS-4865](https://issues.apache.org/jira/browse/HDFS-4865) | Remove sub resource warning from httpfs log at startup time | Major | . | Wei Yan | Wei Yan | +| [HDFS-4863](https://issues.apache.org/jira/browse/HDFS-4863) | The root directory should be added to the snapshottable directory list while loading fsimage | Major | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-4862](https://issues.apache.org/jira/browse/HDFS-4862) | SafeModeInfo.isManual() returns true when resources are low even if it wasn't entered into manually | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-4857](https://issues.apache.org/jira/browse/HDFS-4857) | Snapshot.Root and AbstractINodeDiff#snapshotINode should not be put into INodeMap when loading FSImage | Major | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-4850](https://issues.apache.org/jira/browse/HDFS-4850) | fix OfflineImageViewer to work on fsimages with empty files or snapshots | Major | tools | Stephen Chu | Jing Zhao | +| [HDFS-4846](https://issues.apache.org/jira/browse/HDFS-4846) | Clean up snapshot CLI commands output stacktrace for invalid arguments | Minor | snapshots | Stephen Chu | Jing Zhao | +| [HDFS-4845](https://issues.apache.org/jira/browse/HDFS-4845) | FSEditLogLoader gets NPE while accessing INodeMap in TestEditLogRace | Critical | namenode | Kihwal Lee | Arpit Agarwal | +| [HDFS-4841](https://issues.apache.org/jira/browse/HDFS-4841) | FsShell commands using secure webhfds fail ClientFinalizer shutdown hook | Major | security, webhdfs | Stephen Chu | Robert Kanter | +| [HDFS-4840](https://issues.apache.org/jira/browse/HDFS-4840) | ReplicationMonitor gets NPE during shutdown | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4832](https://issues.apache.org/jira/browse/HDFS-4832) | Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave | Critical | . | Ravi Prakash | Ravi Prakash | +| [HDFS-4830](https://issues.apache.org/jira/browse/HDFS-4830) | Typo in config settings for AvailableSpaceVolumeChoosingPolicy in hdfs-default.xml | Minor | . | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4827](https://issues.apache.org/jira/browse/HDFS-4827) | Slight update to the implementation of API for handling favored nodes in DFSClient | Major | . | Devaraj Das | Devaraj Das | +| [HDFS-4826](https://issues.apache.org/jira/browse/HDFS-4826) | TestNestedSnapshots times out due to repeated slow edit log flushes when running on virtualized disk | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4825](https://issues.apache.org/jira/browse/HDFS-4825) | webhdfs / httpfs tests broken because of min block size change | Major | webhdfs | Andrew Wang | Andrew Wang | +| [HDFS-4824](https://issues.apache.org/jira/browse/HDFS-4824) | FileInputStreamCache.close leaves dangling reference to FileInputStreamCache.cacheCleaner | Major | hdfs-client | Henry Robinson | Colin Patrick McCabe | +| [HDFS-4818](https://issues.apache.org/jira/browse/HDFS-4818) | several HDFS tests that attempt to make directories unusable do not work correctly on Windows | Minor | namenode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-4815](https://issues.apache.org/jira/browse/HDFS-4815) | TestRBWBlockInvalidation#testBlockInvalidationWhenRBWReplicaMissedInDN: Double call countReplicas() to fetch corruptReplicas and liveReplicas is not needed | Major | datanode, test | Tian Hong Wang | Tian Hong Wang | +| [HDFS-4813](https://issues.apache.org/jira/browse/HDFS-4813) | BlocksMap may throw NullPointerException during shutdown | Minor | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4810](https://issues.apache.org/jira/browse/HDFS-4810) | several HDFS HA tests have timeouts that are too short | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4807](https://issues.apache.org/jira/browse/HDFS-4807) | DFSOutputStream.createSocketForPipeline() should not include timeout extension on connect | Major | . | Kihwal Lee | Cristina L. Abad | +| [HDFS-4805](https://issues.apache.org/jira/browse/HDFS-4805) | Webhdfs client is fragile to token renewal errors | Critical | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4799](https://issues.apache.org/jira/browse/HDFS-4799) | Corrupt replica can be prematurely removed from corruptReplicas map | Blocker | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-4797](https://issues.apache.org/jira/browse/HDFS-4797) | BlockScanInfo does not override equals(..) and hashCode() consistently | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4783](https://issues.apache.org/jira/browse/HDFS-4783) | TestDelegationTokensWithHA#testHAUtilClonesDelegationTokens fails on Windows | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4780](https://issues.apache.org/jira/browse/HDFS-4780) | Use the correct relogin method for services | Minor | namenode | Kihwal Lee | Robert Parker | +| [HDFS-4778](https://issues.apache.org/jira/browse/HDFS-4778) | Invoke getPipeline in the chooseTarget implementation that has favoredNodes | Major | namenode | Devaraj Das | Devaraj Das | +| [HDFS-4768](https://issues.apache.org/jira/browse/HDFS-4768) | File handle leak in datanode when a block pool is removed | Major | datanode | Chris Nauroth | Chris Nauroth | +| [HDFS-4765](https://issues.apache.org/jira/browse/HDFS-4765) | Permission check of symlink deletion incorrectly throws UnresolvedLinkException | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4751](https://issues.apache.org/jira/browse/HDFS-4751) | TestLeaseRenewer#testThreadName flakes | Minor | test | Andrew Wang | Andrew Wang | +| [HDFS-4748](https://issues.apache.org/jira/browse/HDFS-4748) | MiniJournalCluster#restartJournalNode leaks resources, which causes sporadic test failures | Major | qjm, test | Chris Nauroth | Chris Nauroth | +| [HDFS-4745](https://issues.apache.org/jira/browse/HDFS-4745) | TestDataTransferKeepalive#testSlowReader has race condition that causes sporadic failure | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4743](https://issues.apache.org/jira/browse/HDFS-4743) | TestNNStorageRetentionManager fails on Windows | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4741](https://issues.apache.org/jira/browse/HDFS-4741) | TestStorageRestore#testStorageRestoreFailure fails on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4740](https://issues.apache.org/jira/browse/HDFS-4740) | Fixes for a few test failures on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4739](https://issues.apache.org/jira/browse/HDFS-4739) | NN can miscalculate the number of extra edit log segments to retain | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4737](https://issues.apache.org/jira/browse/HDFS-4737) | JVM path embedded in fuse binaries | Major | . | Sean Mackrory | Sean Mackrory | +| [HDFS-4734](https://issues.apache.org/jira/browse/HDFS-4734) | HDFS Tests that use ShellCommandFencer are broken on Windows | Major | . | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4733](https://issues.apache.org/jira/browse/HDFS-4733) | Make HttpFS username pattern configurable | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-4732](https://issues.apache.org/jira/browse/HDFS-4732) | TestDFSUpgradeFromImage fails on Windows due to failure to unpack old image tarball that contains hard links | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4725](https://issues.apache.org/jira/browse/HDFS-4725) | fix HDFS file handle leaks | Major | namenode, test, tools | Chris Nauroth | Chris Nauroth | +| [HDFS-4722](https://issues.apache.org/jira/browse/HDFS-4722) | TestGetConf#testFederation times out on Windows | Major | test | Ivan Mitic | Ivan Mitic | +| [HDFS-4714](https://issues.apache.org/jira/browse/HDFS-4714) | Log short messages in Namenode RPC server for exceptions meant for clients | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4705](https://issues.apache.org/jira/browse/HDFS-4705) | Address HDFS test failures on Windows because of invalid dfs.namenode.name.dir | Minor | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4699](https://issues.apache.org/jira/browse/HDFS-4699) | TestPipelinesFailover#testPipelineRecoveryStress fails sporadically | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4695](https://issues.apache.org/jira/browse/HDFS-4695) | TestEditLog leaks open file handles between tests | Major | test | Ivan Mitic | Ivan Mitic | +| [HDFS-4693](https://issues.apache.org/jira/browse/HDFS-4693) | Some test cases in TestCheckpoint do not clean up after themselves | Minor | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4687](https://issues.apache.org/jira/browse/HDFS-4687) | TestDelegationTokenForProxyUser#testWebHdfsDoAs is flaky with JDK7 | Minor | test | Andrew Wang | Andrew Wang | +| [HDFS-4677](https://issues.apache.org/jira/browse/HDFS-4677) | Editlog should support synchronous writes | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4676](https://issues.apache.org/jira/browse/HDFS-4676) | TestHDFSFileSystemContract should set MiniDFSCluster variable to null to free up memory | Minor | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4674](https://issues.apache.org/jira/browse/HDFS-4674) | TestBPOfferService fails on Windows due to failure parsing datanode data directory as URI | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-4669](https://issues.apache.org/jira/browse/HDFS-4669) | TestBlockPoolManager fails using IBM java | Major | test | Tian Hong Wang | Tian Hong Wang | +| [HDFS-4658](https://issues.apache.org/jira/browse/HDFS-4658) | Standby NN will log that it has received a block report "after becoming active" | Trivial | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4655](https://issues.apache.org/jira/browse/HDFS-4655) | DNA\_FINALIZE is logged as being an unknown command by the DN when received from the standby NN | Minor | datanode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4643](https://issues.apache.org/jira/browse/HDFS-4643) | Fix flakiness in TestQuorumJournalManager | Trivial | qjm, test | Todd Lipcon | Todd Lipcon | +| [HDFS-4639](https://issues.apache.org/jira/browse/HDFS-4639) | startFileInternal() should not increment generation stamp | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-4625](https://issues.apache.org/jira/browse/HDFS-4625) | Make TestNNWithQJM#testNewNamenodeTakesOverWriter work on Windows | Minor | test | Arpit Agarwal | Ivan Mitic | +| [HDFS-4621](https://issues.apache.org/jira/browse/HDFS-4621) | additional logging to help diagnose slow QJM logSync | Minor | ha, qjm | Todd Lipcon | Todd Lipcon | +| [HDFS-4620](https://issues.apache.org/jira/browse/HDFS-4620) | Documentation for dfs.namenode.rpc-address specifies wrong format | Major | documentation | Sandy Ryza | Sandy Ryza | +| [HDFS-4618](https://issues.apache.org/jira/browse/HDFS-4618) | default for checkpoint txn interval is too low | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-4615](https://issues.apache.org/jira/browse/HDFS-4615) | Fix TestDFSShell failures on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4614](https://issues.apache.org/jira/browse/HDFS-4614) | FSNamesystem#getContentSummary should use getPermissionChecker helper method | Trivial | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4610](https://issues.apache.org/jira/browse/HDFS-4610) | Move to using common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4609](https://issues.apache.org/jira/browse/HDFS-4609) | TestAuditLogs should release log handles between tests | Minor | test | Ivan Mitic | Ivan Mitic | +| [HDFS-4607](https://issues.apache.org/jira/browse/HDFS-4607) | TestGetConf#testGetSpecificKey fails on Windows | Minor | test | Ivan Mitic | Ivan Mitic | +| [HDFS-4604](https://issues.apache.org/jira/browse/HDFS-4604) | TestJournalNode fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4603](https://issues.apache.org/jira/browse/HDFS-4603) | TestMiniDFSCluster fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4598](https://issues.apache.org/jira/browse/HDFS-4598) | WebHDFS concat: the default value of sources in the code does not match the doc | Minor | webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4596](https://issues.apache.org/jira/browse/HDFS-4596) | Shutting down namenode during checkpointing can lead to md5sum error | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4595](https://issues.apache.org/jira/browse/HDFS-4595) | When short circuit read is fails, DFSClient does not fallback to regular reads | Major | hdfs-client | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4593](https://issues.apache.org/jira/browse/HDFS-4593) | TestSaveNamespace fails on Windows | Major | . | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4592](https://issues.apache.org/jira/browse/HDFS-4592) | Default values for access time precision are out of sync between hdfs-default.xml and the code | Minor | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4591](https://issues.apache.org/jira/browse/HDFS-4591) | HA clients can fail to fail over while Standby NN is performing long checkpoint | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4586](https://issues.apache.org/jira/browse/HDFS-4586) | TestDataDirs.testGetDataDirsFromURIs fails with all directories in dfs.datanode.data.dir are invalid | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4584](https://issues.apache.org/jira/browse/HDFS-4584) | Fix TestNNWithQJM failures on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4583](https://issues.apache.org/jira/browse/HDFS-4583) | TestNodeCount fails | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4582](https://issues.apache.org/jira/browse/HDFS-4582) | TestHostsFiles fails on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [HDFS-4573](https://issues.apache.org/jira/browse/HDFS-4573) | Fix TestINodeFile on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4572](https://issues.apache.org/jira/browse/HDFS-4572) | Fix TestJournal failures on Windows | Major | namenode, test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4544](https://issues.apache.org/jira/browse/HDFS-4544) | Error in deleting blocks should not do check disk, for all types of errors | Major | . | Amareshwari Sriramadasu | Arpit Agarwal | +| [HDFS-4541](https://issues.apache.org/jira/browse/HDFS-4541) | set hadoop.log.dir and hadoop.id.str when starting secure datanode so it writes the logs to the correct dir by default | Major | datanode, security | Arpit Gupta | Arpit Gupta | +| [HDFS-4540](https://issues.apache.org/jira/browse/HDFS-4540) | namenode http server should use the web authentication keytab for spnego principal | Major | security | Arpit Gupta | Arpit Gupta | +| [HDFS-4533](https://issues.apache.org/jira/browse/HDFS-4533) | start-dfs.sh ignored additional parameters besides -upgrade | Major | datanode, namenode | Fengdong Yu | Fengdong Yu | +| [HDFS-4532](https://issues.apache.org/jira/browse/HDFS-4532) | RPC call queue may fill due to current user lookup | Critical | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-4522](https://issues.apache.org/jira/browse/HDFS-4522) | LightWeightGSet expects incrementing a volatile to be atomic | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4519](https://issues.apache.org/jira/browse/HDFS-4519) | Support override of jsvc binary and log file locations when launching secure datanode. | Major | datanode, scripts | Chris Nauroth | Chris Nauroth | +| [HDFS-4518](https://issues.apache.org/jira/browse/HDFS-4518) | Finer grained metrics for HDFS capacity | Major | . | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4495](https://issues.apache.org/jira/browse/HDFS-4495) | Allow client-side lease renewal to be retried beyond soft-limit | Major | hdfs-client | Kihwal Lee | Kihwal Lee | +| [HDFS-4489](https://issues.apache.org/jira/browse/HDFS-4489) | Use InodeID as as an identifier of a file in HDFS protocols and APIs | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-4484](https://issues.apache.org/jira/browse/HDFS-4484) | libwebhdfs compilation broken with gcc 4.6.2 | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4477](https://issues.apache.org/jira/browse/HDFS-4477) | Secondary namenode may retain old tokens | Critical | security | Kihwal Lee | Daryn Sharp | +| [HDFS-4471](https://issues.apache.org/jira/browse/HDFS-4471) | Namenode WebUI file browsing does not work with wildcard addresses configured | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4470](https://issues.apache.org/jira/browse/HDFS-4470) | several HDFS tests attempt file operations on invalid HDFS paths when running on Windows | Major | . | Chris Nauroth | Chris Nauroth | +| [HDFS-4382](https://issues.apache.org/jira/browse/HDFS-4382) | Fix typo MAX\_NOT\_CHANGED\_INTERATIONS | Major | . | Ted Yu | Ted Yu | +| [HDFS-4342](https://issues.apache.org/jira/browse/HDFS-4342) | Edits dir in dfs.namenode.edits.dir.required will be silently ignored if it is not in dfs.namenode.edits.dir | Major | namenode | Mark Yang | Arpit Agarwal | +| [HDFS-4300](https://issues.apache.org/jira/browse/HDFS-4300) | TransferFsImage.downloadEditsToStorage should use a tmp file for destination | Critical | . | Todd Lipcon | Andrew Wang | +| [HDFS-4298](https://issues.apache.org/jira/browse/HDFS-4298) | StorageRetentionManager spews warnings when used with QJM | Major | namenode | Todd Lipcon | Aaron T. Myers | +| [HDFS-4296](https://issues.apache.org/jira/browse/HDFS-4296) | Add layout version for HDFS-4256 for release 1.2.0 | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4287](https://issues.apache.org/jira/browse/HDFS-4287) | HTTPFS tests fail on Windows | Major | webhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-4269](https://issues.apache.org/jira/browse/HDFS-4269) | DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-4261](https://issues.apache.org/jira/browse/HDFS-4261) | TestBalancerWithNodeGroup times out | Major | balancer & mover | Tsz Wo Nicholas Sze | Junping Du | +| [HDFS-4243](https://issues.apache.org/jira/browse/HDFS-4243) | INodeDirectory.replaceChild(..) does not update parent | Major | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-4240](https://issues.apache.org/jira/browse/HDFS-4240) | In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup | Major | namenode | Junping Du | Junping Du | +| [HDFS-4235](https://issues.apache.org/jira/browse/HDFS-4235) | when outputting XML, OfflineEditsViewer can't handle some edits containing non-ASCII strings | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4222](https://issues.apache.org/jira/browse/HDFS-4222) | NN is unresponsive and loses heartbeats of DNs when Hadoop is configured to use LDAP and LDAP has issues | Minor | namenode | Xiaobo Peng | Xiaobo Peng | +| [HDFS-4209](https://issues.apache.org/jira/browse/HDFS-4209) | Clean up the addNode/addChild/addChildNoQuotaCheck methods in FSDirectory | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4205](https://issues.apache.org/jira/browse/HDFS-4205) | fsck fails with symlinks | Major | hdfs-client | Andy Isaacson | Jason Lowe | +| [HDFS-4128](https://issues.apache.org/jira/browse/HDFS-4128) | 2NN gets stuck in inconsistent state if edit log replay fails in the middle | Major | namenode | Todd Lipcon | Kihwal Lee | +| [HDFS-4013](https://issues.apache.org/jira/browse/HDFS-4013) | TestHftpURLTimeouts throws NPE | Trivial | hdfs-client | Chao Shi | Chao Shi | +| [HDFS-3934](https://issues.apache.org/jira/browse/HDFS-3934) | duplicative dfs\_hosts entries handled wrong | Minor | . | Andy Isaacson | Colin Patrick McCabe | +| [HDFS-3875](https://issues.apache.org/jira/browse/HDFS-3875) | Issue handling checksum errors in write pipeline | Critical | datanode, hdfs-client | Todd Lipcon | Kihwal Lee | +| [HDFS-3792](https://issues.apache.org/jira/browse/HDFS-3792) | Fix two findbugs introduced by HDFS-3695 | Trivial | build, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-3499](https://issues.apache.org/jira/browse/HDFS-3499) | Make NetworkTopology support user specified topology class | Major | datanode | Junping Du | Junping Du | +| [HDFS-3277](https://issues.apache.org/jira/browse/HDFS-3277) | fail over to loading a different FSImage if the first one we try to load is corrupt | Major | . | Colin Patrick McCabe | Andrew Wang | +| [HDFS-3180](https://issues.apache.org/jira/browse/HDFS-3180) | Add socket timeouts to webhdfs | Major | webhdfs | Daryn Sharp | Chris Nauroth | +| [HDFS-3009](https://issues.apache.org/jira/browse/HDFS-3009) | DFSClient islocaladdress() can use similar routine in netutils | Trivial | hdfs-client | Hari Mankude | Hari Mankude | +| [MAPREDUCE-5421](https://issues.apache.org/jira/browse/MAPREDUCE-5421) | TestNonExistentJob is failed due to recent changes in YARN | Blocker | test | Junping Du | Junping Du | +| [MAPREDUCE-5419](https://issues.apache.org/jira/browse/MAPREDUCE-5419) | TestSlive is getting FileNotFound Exception | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5412](https://issues.apache.org/jira/browse/MAPREDUCE-5412) | Change MR to use multiple containers API of ContainerManager after YARN-926 | Major | . | Jian He | Jian He | +| [MAPREDUCE-5399](https://issues.apache.org/jira/browse/MAPREDUCE-5399) | Unnecessary Configuration instantiation in IFileInputStream slows down merge | Blocker | mrv1, mrv2 | Stanislav Barton | Stanislav Barton | +| [MAPREDUCE-5366](https://issues.apache.org/jira/browse/MAPREDUCE-5366) | TestMRAsyncDiskService fails on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5360](https://issues.apache.org/jira/browse/MAPREDUCE-5360) | TestMRJobClient fails on Windows due to path format | Minor | test | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5359](https://issues.apache.org/jira/browse/MAPREDUCE-5359) | JobHistory should not use File.separator to match timestamp in path | Minor | . | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5357](https://issues.apache.org/jira/browse/MAPREDUCE-5357) | Job staging directory owner checking could fail on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5355](https://issues.apache.org/jira/browse/MAPREDUCE-5355) | MiniMRYarnCluster with localFs does not work on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5349](https://issues.apache.org/jira/browse/MAPREDUCE-5349) | TestClusterMapReduceTestCase and TestJobName fail on Windows in branch-2 | Minor | . | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5334](https://issues.apache.org/jira/browse/MAPREDUCE-5334) | TestContainerLauncherImpl is failing | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5328](https://issues.apache.org/jira/browse/MAPREDUCE-5328) | ClientToken should not be set in the environment | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5326](https://issues.apache.org/jira/browse/MAPREDUCE-5326) | Add version to shuffle header | Blocker | . | Arun C Murthy | Zhijie Shen | +| [MAPREDUCE-5325](https://issues.apache.org/jira/browse/MAPREDUCE-5325) | ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter---MR changes | Major | . | Xuan Gong | Xuan Gong | +| [MAPREDUCE-5319](https://issues.apache.org/jira/browse/MAPREDUCE-5319) | Job.xml file does not has 'user.name' property for Hadoop2 | Major | . | Yesha Vora | Xuan Gong | +| [MAPREDUCE-5315](https://issues.apache.org/jira/browse/MAPREDUCE-5315) | DistCp reports success even on failure. | Critical | distcp | Mithun Radhakrishnan | Mithun Radhakrishnan | +| [MAPREDUCE-5312](https://issues.apache.org/jira/browse/MAPREDUCE-5312) | TestRMNMInfo is failing | Major | . | Alejandro Abdelnur | Sandy Ryza | +| [MAPREDUCE-5310](https://issues.apache.org/jira/browse/MAPREDUCE-5310) | MRAM should not normalize allocation request capabilities | Major | applicationmaster | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-5308](https://issues.apache.org/jira/browse/MAPREDUCE-5308) | Shuffling to memory can get out-of-sync when fetching multiple compressed map outputs | Major | . | Nathan Roberts | Nathan Roberts | +| [MAPREDUCE-5303](https://issues.apache.org/jira/browse/MAPREDUCE-5303) | Changes on MR after moving ProtoBase to package impl.pb on YARN-724 | Major | . | Jian He | Jian He | +| [MAPREDUCE-5301](https://issues.apache.org/jira/browse/MAPREDUCE-5301) | Update MR code to work with YARN-635 changes | Major | . | Siddharth Seth | Siddharth Seth | +| [MAPREDUCE-5297](https://issues.apache.org/jira/browse/MAPREDUCE-5297) | Update MR App since BuilderUtils is moved to yarn-server-common after YARN-748 | Major | . | Jian He | Jian He | +| [MAPREDUCE-5291](https://issues.apache.org/jira/browse/MAPREDUCE-5291) | Change MR App to use update property names in container-log4j.properties | Major | . | Siddharth Seth | Zhijie Shen | +| [MAPREDUCE-5289](https://issues.apache.org/jira/browse/MAPREDUCE-5289) | Update MR App to use Token directly after YARN-717 | Major | . | Vinod Kumar Vavilapalli | Jian He | +| [MAPREDUCE-5285](https://issues.apache.org/jira/browse/MAPREDUCE-5285) | Update MR App to use immutable ApplicationAttemptID, ContainerID, NodeID after YARN-735 | Major | . | Jian He | | +| [MAPREDUCE-5282](https://issues.apache.org/jira/browse/MAPREDUCE-5282) | Update MR App to use immutable ApplicationID after YARN-716 | Major | . | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-5270](https://issues.apache.org/jira/browse/MAPREDUCE-5270) | Migrate from using BuilderUtil factory methods to individual record factory method on MapReduce side | Major | . | Jian He | Jian He | +| [MAPREDUCE-5259](https://issues.apache.org/jira/browse/MAPREDUCE-5259) | TestTaskLog fails on Windows because of path separators missmatch | Major | test | Ivan Mitic | Ivan Mitic | +| [MAPREDUCE-5257](https://issues.apache.org/jira/browse/MAPREDUCE-5257) | TestContainerLauncherImpl fails | Major | mr-am, mrv2 | Jason Lowe | Omkar Vinit Joshi | +| [MAPREDUCE-5240](https://issues.apache.org/jira/browse/MAPREDUCE-5240) | inside of FileOutputCommitter the initialized Credentials cache appears to be empty | Blocker | mrv2 | Roman Shaposhnik | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5239](https://issues.apache.org/jira/browse/MAPREDUCE-5239) | Update MR App to reflect YarnRemoteException changes after YARN-634 | Major | . | Vinod Kumar Vavilapalli | Siddharth Seth | +| [MAPREDUCE-5226](https://issues.apache.org/jira/browse/MAPREDUCE-5226) | Handle exception related changes in YARN's AMRMProtocol api after YARN-630 | Major | . | Xuan Gong | Xuan Gong | +| [MAPREDUCE-5213](https://issues.apache.org/jira/browse/MAPREDUCE-5213) | Re-assess TokenCache methods marked @Private | Minor | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5212](https://issues.apache.org/jira/browse/MAPREDUCE-5212) | Handle exception related changes in YARN's ClientRMProtocol api after YARN-631 | Major | . | Xuan Gong | Xuan Gong | +| [MAPREDUCE-5209](https://issues.apache.org/jira/browse/MAPREDUCE-5209) | ShuffleScheduler log message incorrect | Minor | mrv2 | Radim Kolar | Tsuyoshi Ozawa | +| [MAPREDUCE-5208](https://issues.apache.org/jira/browse/MAPREDUCE-5208) | SpillRecord and ShuffleHandler should use SecureIOUtils for reading index file and map output | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5205](https://issues.apache.org/jira/browse/MAPREDUCE-5205) | Apps fail in secure cluster setup | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5204](https://issues.apache.org/jira/browse/MAPREDUCE-5204) | Handle YarnRemoteException separately from IOException in MR api | Major | . | Xuan Gong | Xuan Gong | +| [MAPREDUCE-5193](https://issues.apache.org/jira/browse/MAPREDUCE-5193) | A few MR tests use block sizes which are smaller than the default minimum block size | Major | test | Aaron T. Myers | Andrew Wang | +| [MAPREDUCE-5191](https://issues.apache.org/jira/browse/MAPREDUCE-5191) | TestQueue#testQueue fails with timeout on Windows | Major | . | Ivan Mitic | Ivan Mitic | +| [MAPREDUCE-5187](https://issues.apache.org/jira/browse/MAPREDUCE-5187) | Create mapreduce command scripts on Windows | Major | mrv2 | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5181](https://issues.apache.org/jira/browse/MAPREDUCE-5181) | RMCommunicator should not use AMToken from the env | Major | applicationmaster | Siddharth Seth | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5179](https://issues.apache.org/jira/browse/MAPREDUCE-5179) | Change TestHSWebServices to do string equal check on hadoop build version similar to YARN-605 | Major | . | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-5178](https://issues.apache.org/jira/browse/MAPREDUCE-5178) | Fix use of BuilderUtils#newApplicationReport as a result of YARN-577. | Major | . | Hitesh Shah | Hitesh Shah | +| [MAPREDUCE-5177](https://issues.apache.org/jira/browse/MAPREDUCE-5177) | Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute | Major | . | Ivan Mitic | Ivan Mitic | +| [MAPREDUCE-5175](https://issues.apache.org/jira/browse/MAPREDUCE-5175) | Update MR App to not set envs that will be set by NMs anyways after YARN-561 | Major | . | Vinod Kumar Vavilapalli | Xuan Gong | +| [MAPREDUCE-5167](https://issues.apache.org/jira/browse/MAPREDUCE-5167) | Update MR App after YARN-562 | Major | . | Vinod Kumar Vavilapalli | Jian He | +| [MAPREDUCE-5166](https://issues.apache.org/jira/browse/MAPREDUCE-5166) | ConcurrentModificationException in LocalJobRunner | Blocker | . | Gunther Hagleitner | Sandy Ryza | +| [MAPREDUCE-5163](https://issues.apache.org/jira/browse/MAPREDUCE-5163) | Update MR App after YARN-441 | Major | . | Vinod Kumar Vavilapalli | Xuan Gong | +| [MAPREDUCE-5152](https://issues.apache.org/jira/browse/MAPREDUCE-5152) | MR App is not using Container from RM | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5151](https://issues.apache.org/jira/browse/MAPREDUCE-5151) | Update MR App after YARN-444 | Major | . | Vinod Kumar Vavilapalli | Sandy Ryza | +| [MAPREDUCE-5147](https://issues.apache.org/jira/browse/MAPREDUCE-5147) | Maven build should create hadoop-mapreduce-client-app-VERSION.jar directly | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5146](https://issues.apache.org/jira/browse/MAPREDUCE-5146) | application classloader may be used too early to load classes | Minor | task | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-5145](https://issues.apache.org/jira/browse/MAPREDUCE-5145) | Change default max-attempts to be more than one for MR jobs as well | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5140](https://issues.apache.org/jira/browse/MAPREDUCE-5140) | MR part of YARN-514 | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5139](https://issues.apache.org/jira/browse/MAPREDUCE-5139) | Update MR App after YARN-486 | Major | . | Vinod Kumar Vavilapalli | Xuan Gong | +| [MAPREDUCE-5138](https://issues.apache.org/jira/browse/MAPREDUCE-5138) | Fix LocalDistributedCacheManager after YARN-112 | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [MAPREDUCE-5137](https://issues.apache.org/jira/browse/MAPREDUCE-5137) | AM web UI: clicking on Map Task results in 500 error | Major | applicationmaster | Thomas Graves | Thomas Graves | +| [MAPREDUCE-5136](https://issues.apache.org/jira/browse/MAPREDUCE-5136) | TestJobImpl-\>testJobNoTasks fails with IBM JAVA | Major | . | Amir Sanjar | Amir Sanjar | +| [MAPREDUCE-5113](https://issues.apache.org/jira/browse/MAPREDUCE-5113) | Streaming input/output types are ignored with java mapper/reducer | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5098](https://issues.apache.org/jira/browse/MAPREDUCE-5098) | Fix findbugs warnings in gridmix | Major | contrib/gridmix | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5078](https://issues.apache.org/jira/browse/MAPREDUCE-5078) | TestMRAppMaster fails on Windows due to mismatched path separators | Major | client | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5077](https://issues.apache.org/jira/browse/MAPREDUCE-5077) | Cleanup: mapreduce.util.ResourceCalculatorPlugin and related code should be removed | Minor | mrv2 | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5075](https://issues.apache.org/jira/browse/MAPREDUCE-5075) | DistCp leaks input file handles | Major | distcp | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5066](https://issues.apache.org/jira/browse/MAPREDUCE-5066) | JobTracker should set a timeout when calling into job.end.notification.url | Major | . | Ivan Mitic | Ivan Mitic | +| [MAPREDUCE-5065](https://issues.apache.org/jira/browse/MAPREDUCE-5065) | DistCp should skip checksum comparisons if block-sizes are different on source/target. | Major | distcp | Mithun Radhakrishnan | Mithun Radhakrishnan | +| [MAPREDUCE-5062](https://issues.apache.org/jira/browse/MAPREDUCE-5062) | MR AM should read max-retries information from the RM | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [MAPREDUCE-5060](https://issues.apache.org/jira/browse/MAPREDUCE-5060) | Fetch failures that time out only count against the first map task | Critical | . | Robert Joseph Evans | Robert Joseph Evans | +| [MAPREDUCE-5059](https://issues.apache.org/jira/browse/MAPREDUCE-5059) | Job overview shows average merge time larger than for any reduce attempt | Major | jobhistoryserver, webapps | Jason Lowe | Omkar Vinit Joshi | +| [MAPREDUCE-5043](https://issues.apache.org/jira/browse/MAPREDUCE-5043) | Fetch failure processing can cause AM event queue to backup and eventually OOM | Blocker | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5042](https://issues.apache.org/jira/browse/MAPREDUCE-5042) | Reducer unable to fetch for a map task that was recovered | Blocker | mr-am, security | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5027](https://issues.apache.org/jira/browse/MAPREDUCE-5027) | Shuffle does not limit number of outstanding connections | Major | . | Jason Lowe | Robert Parker | +| [MAPREDUCE-5013](https://issues.apache.org/jira/browse/MAPREDUCE-5013) | mapred.JobStatus compatibility: MR2 missing constructors from MR1 | Major | client | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5009](https://issues.apache.org/jira/browse/MAPREDUCE-5009) | Killing the Task Attempt slated for commit does not clear the value from the Task commitAttempt member | Critical | mrv1 | Robert Parker | Robert Parker | +| [MAPREDUCE-5008](https://issues.apache.org/jira/browse/MAPREDUCE-5008) | Merger progress miscounts with respect to EOF\_MARKER | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5000](https://issues.apache.org/jira/browse/MAPREDUCE-5000) | TaskImpl.getCounters() can return the counters for the wrong task attempt when task is speculating | Critical | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4994](https://issues.apache.org/jira/browse/MAPREDUCE-4994) | -jt generic command line option does not work | Major | client | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4992](https://issues.apache.org/jira/browse/MAPREDUCE-4992) | AM hangs in RecoveryService when recovering tasks with speculative attempts | Critical | mr-am | Robert Parker | Robert Parker | +| [MAPREDUCE-4987](https://issues.apache.org/jira/browse/MAPREDUCE-4987) | TestMRJobs#testDistributedCache fails on Windows due to classpath problems and unexpected behavior of symlinks | Major | distributed-cache, nodemanager | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-4985](https://issues.apache.org/jira/browse/MAPREDUCE-4985) | TestDFSIO supports compression but usages doesn't reflect | Trivial | . | Plamen Jeliazkov | Plamen Jeliazkov | +| [MAPREDUCE-4981](https://issues.apache.org/jira/browse/MAPREDUCE-4981) | WordMean, WordMedian, WordStandardDeviation missing from ExamplesDriver | Minor | . | Plamen Jeliazkov | Plamen Jeliazkov | +| [MAPREDUCE-4932](https://issues.apache.org/jira/browse/MAPREDUCE-4932) | mapreduce.job#getTaskCompletionEvents incompatible with Hadoop 1 | Major | mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4927](https://issues.apache.org/jira/browse/MAPREDUCE-4927) | Historyserver 500 error due to NPE when accessing specific counters page for failed job | Major | jobhistoryserver | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-4898](https://issues.apache.org/jira/browse/MAPREDUCE-4898) | FileOutputFormat.checkOutputSpecs and FileOutputFormat.setOutputPath incompatible with MR1 | Major | mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-4896](https://issues.apache.org/jira/browse/MAPREDUCE-4896) | "mapred queue -info" spits out ugly exception when queue does not exist | Major | client, scheduler | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-4892](https://issues.apache.org/jira/browse/MAPREDUCE-4892) | CombineFileInputFormat node input split can be skewed on small clusters | Major | . | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4885](https://issues.apache.org/jira/browse/MAPREDUCE-4885) | Streaming tests have multiple failures on Windows | Major | contrib/streaming, test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-4871](https://issues.apache.org/jira/browse/MAPREDUCE-4871) | AM uses mapreduce.jobtracker.split.metainfo.maxsize but mapred-default has mapreduce.job.split.metainfo.maxsize | Major | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4794](https://issues.apache.org/jira/browse/MAPREDUCE-4794) | DefaultSpeculator generates error messages on normal shutdown | Major | applicationmaster | Jason Lowe | Jason Lowe | +| [MAPREDUCE-4716](https://issues.apache.org/jira/browse/MAPREDUCE-4716) | TestHsWebServicesJobsQuery.testJobsQueryStateInvalid fails with jdk7 | Major | jobhistoryserver | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4693](https://issues.apache.org/jira/browse/MAPREDUCE-4693) | Historyserver should provide counters for failed tasks | Major | jobhistoryserver, mrv2 | Jason Lowe | Xuan Gong | +| [MAPREDUCE-4671](https://issues.apache.org/jira/browse/MAPREDUCE-4671) | AM does not tell the RM about container requests that are no longer needed | Major | . | Bikas Saha | Bikas Saha | +| [MAPREDUCE-4571](https://issues.apache.org/jira/browse/MAPREDUCE-4571) | TestHsWebServicesJobs fails on jdk7 | Major | webapps | Thomas Graves | Thomas Graves | +| [MAPREDUCE-4374](https://issues.apache.org/jira/browse/MAPREDUCE-4374) | Fix child task environment variable config and add support for Windows | Minor | mrv2 | Chuan Liu | Chuan Liu | +| [MAPREDUCE-4356](https://issues.apache.org/jira/browse/MAPREDUCE-4356) | Provide access to ParsedTask.obtainTaskAttempts() | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-4149](https://issues.apache.org/jira/browse/MAPREDUCE-4149) | Rumen fails to parse certain counter strings | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-4100](https://issues.apache.org/jira/browse/MAPREDUCE-4100) | Sometimes gridmix emulates data larger much larger then acutal counter for map only jobs | Minor | contrib/gridmix | Karam Singh | Amar Kamat | +| [MAPREDUCE-4083](https://issues.apache.org/jira/browse/MAPREDUCE-4083) | GridMix emulated job tasks.resource-usage emulator for CPU usage throws NPE when Trace contains cumulativeCpuUsage value of 0 at attempt level | Major | contrib/gridmix | Karam Singh | Amar Kamat | +| [MAPREDUCE-4019](https://issues.apache.org/jira/browse/MAPREDUCE-4019) | -list-attempt-ids is not working | Minor | client | B Anil Kumar | Ashwin Shankar | +| [MAPREDUCE-3953](https://issues.apache.org/jira/browse/MAPREDUCE-3953) | Gridmix throws NPE and does not simulate a job if the trace contains null taskStatus for a task | Major | . | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3872](https://issues.apache.org/jira/browse/MAPREDUCE-3872) | event handling races in ContainerLauncherImpl and TestContainerLauncher | Major | client, mrv2 | Patrick Hunt | Robert Kanter | +| [MAPREDUCE-3829](https://issues.apache.org/jira/browse/MAPREDUCE-3829) | [Gridmix] Gridmix should give better error message when input-data directory already exists and -generate option is given | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3757](https://issues.apache.org/jira/browse/MAPREDUCE-3757) | Rumen Folder is not adjusting the shuffleFinished and sortFinished times of reduce task attempts | Major | tools/rumen | Ravi Gummadi | Ravi Gummadi | +| [MAPREDUCE-3685](https://issues.apache.org/jira/browse/MAPREDUCE-3685) | There are some bugs in implementation of MergeManager | Critical | mrv2 | anty.rao | anty | +| [MAPREDUCE-2722](https://issues.apache.org/jira/browse/MAPREDUCE-2722) | Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used | Major | contrib/gridmix | Ravi Gummadi | Ravi Gummadi | +| [YARN-1046](https://issues.apache.org/jira/browse/YARN-1046) | Disable mem monitoring by default in MiniYARNCluster | Major | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-1043](https://issues.apache.org/jira/browse/YARN-1043) | YARN Queue metrics are getting pushed to neither file nor Ganglia | Major | . | Yusaku Sako | Jian He | +| [YARN-968](https://issues.apache.org/jira/browse/YARN-968) | RM admin commands don't work | Blocker | . | Kihwal Lee | Vinod Kumar Vavilapalli | +| [YARN-960](https://issues.apache.org/jira/browse/YARN-960) | TestMRCredentials and TestBinaryTokenFile are failing on trunk | Blocker | . | Alejandro Abdelnur | Daryn Sharp | +| [YARN-945](https://issues.apache.org/jira/browse/YARN-945) | AM register failing after AMRMToken | Blocker | . | Bikas Saha | Vinod Kumar Vavilapalli | +| [YARN-937](https://issues.apache.org/jira/browse/YARN-937) | Fix unmanaged AM in non-secure/secure setup post YARN-701 | Blocker | applications/unmanaged-AM-launcher | Arun C Murthy | Alejandro Abdelnur | +| [YARN-932](https://issues.apache.org/jira/browse/YARN-932) | TestResourceLocalizationService.testLocalizationInit can fail on JDK7 | Major | . | Sandy Ryza | Karthik Kambatla | +| [YARN-919](https://issues.apache.org/jira/browse/YARN-919) | Document setting default heap sizes in yarn env | Minor | . | Mayank Bansal | Mayank Bansal | +| [YARN-912](https://issues.apache.org/jira/browse/YARN-912) | Create exceptions package in common/api for yarn and move client facing exceptions to them | Major | . | Bikas Saha | Mayank Bansal | +| [YARN-909](https://issues.apache.org/jira/browse/YARN-909) | Disable TestLinuxContainerExecutorWithMocks on Windows | Minor | nodemanager | Chuan Liu | Chuan Liu | +| [YARN-897](https://issues.apache.org/jira/browse/YARN-897) | CapacityScheduler wrongly sorted queues | Blocker | capacityscheduler | Djellel Eddine Difallah | Djellel Eddine Difallah | +| [YARN-894](https://issues.apache.org/jira/browse/YARN-894) | NodeHealthScriptRunner timeout checking is inaccurate on Windows | Minor | nodemanager | Chuan Liu | Chuan Liu | +| [YARN-875](https://issues.apache.org/jira/browse/YARN-875) | Application can hang if AMRMClientAsync callback thread has exception | Major | . | Bikas Saha | Xuan Gong | +| [YARN-874](https://issues.apache.org/jira/browse/YARN-874) | Tracking YARN/MR test failures after HADOOP-9421 and YARN-827 | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-861](https://issues.apache.org/jira/browse/YARN-861) | TestContainerManager is failing | Critical | nodemanager | Devaraj K | Vinod Kumar Vavilapalli | +| [YARN-854](https://issues.apache.org/jira/browse/YARN-854) | App submission fails on secure deploy | Blocker | . | Ramya Sunil | Omkar Vinit Joshi | +| [YARN-853](https://issues.apache.org/jira/browse/YARN-853) | maximum-am-resource-percent doesn't work after refreshQueues command | Major | capacityscheduler | Devaraj K | Devaraj K | +| [YARN-852](https://issues.apache.org/jira/browse/YARN-852) | TestAggregatedLogFormat.testContainerLogsFileAccess fails on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-848](https://issues.apache.org/jira/browse/YARN-848) | Nodemanager does not register with RM using the fully qualified hostname | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-839](https://issues.apache.org/jira/browse/YARN-839) | TestContainerLaunch.testContainerEnvVariables fails on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-833](https://issues.apache.org/jira/browse/YARN-833) | Move Graph and VisualizeStateMachine into yarn.state package | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-812](https://issues.apache.org/jira/browse/YARN-812) | Enabling app summary logs causes 'FileNotFound' errors | Major | . | Ramya Sunil | Siddharth Seth | +| [YARN-799](https://issues.apache.org/jira/browse/YARN-799) | CgroupsLCEResourcesHandler tries to write to cgroup.procs | Major | nodemanager | Chris Riccomini | Chris Riccomini | +| [YARN-795](https://issues.apache.org/jira/browse/YARN-795) | Fair scheduler queue metrics should subtract allocated vCores from available vCores | Major | scheduler | Wei Yan | Wei Yan | +| [YARN-767](https://issues.apache.org/jira/browse/YARN-767) | Initialize Application status metrics when QueueMetrics is initialized | Major | . | Jian He | Jian He | +| [YARN-764](https://issues.apache.org/jira/browse/YARN-764) | blank Used Resources on Capacity Scheduler page | Major | resourcemanager | Nemon Lou | Nemon Lou | +| [YARN-763](https://issues.apache.org/jira/browse/YARN-763) | AMRMClientAsync should stop heartbeating after receiving shutdown from RM | Major | . | Bikas Saha | Xuan Gong | +| [YARN-761](https://issues.apache.org/jira/browse/YARN-761) | TestNMClientAsync fails sometimes | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-760](https://issues.apache.org/jira/browse/YARN-760) | NodeManager throws AvroRuntimeException on failed start | Major | nodemanager | Sandy Ryza | Niranjan Singh | +| [YARN-757](https://issues.apache.org/jira/browse/YARN-757) | TestRMRestart failing/stuck on trunk | Blocker | . | Bikas Saha | Bikas Saha | +| [YARN-742](https://issues.apache.org/jira/browse/YARN-742) | Log aggregation causes a lot of redundant setPermission calls | Major | nodemanager | Kihwal Lee | Jason Lowe | +| [YARN-733](https://issues.apache.org/jira/browse/YARN-733) | TestNMClient fails occasionally | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-726](https://issues.apache.org/jira/browse/YARN-726) | Queue, FinishTime fields broken on RM UI | Critical | . | Siddharth Seth | Mayank Bansal | +| [YARN-715](https://issues.apache.org/jira/browse/YARN-715) | TestDistributedShell and TestUnmanagedAMLauncher are failing | Major | . | Siddharth Seth | Vinod Kumar Vavilapalli | +| [YARN-706](https://issues.apache.org/jira/browse/YARN-706) | Race Condition in TestFSDownload | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-700](https://issues.apache.org/jira/browse/YARN-700) | TestInfoBlock fails on Windows because of line ending missmatch | Major | . | Ivan Mitic | Ivan Mitic | +| [YARN-690](https://issues.apache.org/jira/browse/YARN-690) | RM exits on token cancel/renew problems | Blocker | resourcemanager | Daryn Sharp | Daryn Sharp | +| [YARN-688](https://issues.apache.org/jira/browse/YARN-688) | Containers not cleaned up when NM received SHUTDOWN event from NodeStatusUpdater | Major | . | Jian He | Jian He | +| [YARN-661](https://issues.apache.org/jira/browse/YARN-661) | NM fails to cleanup local directories for users | Major | nodemanager | Jason Lowe | Omkar Vinit Joshi | +| [YARN-656](https://issues.apache.org/jira/browse/YARN-656) | In scheduler UI, including reserved memory in "Memory Total" can make it exceed cluster capacity. | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-655](https://issues.apache.org/jira/browse/YARN-655) | Fair scheduler metrics should subtract allocated memory from available memory | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-654](https://issues.apache.org/jira/browse/YARN-654) | AMRMClient: Perform sanity checks for parameters of public methods | Major | . | Bikas Saha | Xuan Gong | +| [YARN-648](https://issues.apache.org/jira/browse/YARN-648) | FS: Add documentation for pluggable policy | Major | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-646](https://issues.apache.org/jira/browse/YARN-646) | Some issues in Fair Scheduler's document | Major | documentation | Dapeng Sun | Dapeng Sun | +| [YARN-645](https://issues.apache.org/jira/browse/YARN-645) | Move RMDelegationTokenSecretManager from yarn-server-common to yarn-server-resourcemanager | Major | . | Jian He | Jian He | +| [YARN-639](https://issues.apache.org/jira/browse/YARN-639) | Make AM of Distributed Shell Use NMClient | Major | applications/distributed-shell | Zhijie Shen | Zhijie Shen | +| [YARN-637](https://issues.apache.org/jira/browse/YARN-637) | FS: maxAssign is not honored | Major | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-618](https://issues.apache.org/jira/browse/YARN-618) | Modify RM\_INVALID\_IDENTIFIER to a -ve number | Major | . | Jian He | Jian He | +| [YARN-605](https://issues.apache.org/jira/browse/YARN-605) | Failing unit test in TestNMWebServices when using git for source control | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-599](https://issues.apache.org/jira/browse/YARN-599) | Refactoring submitApplication in ClientRMService and RMAppManager | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-597](https://issues.apache.org/jira/browse/YARN-597) | TestFSDownload fails on Windows because of dependencies on tar/gzip/jar tools | Major | . | Ivan Mitic | Ivan Mitic | +| [YARN-594](https://issues.apache.org/jira/browse/YARN-594) | Update test and add comments in YARN-534 | Major | . | Jian He | Jian He | +| [YARN-593](https://issues.apache.org/jira/browse/YARN-593) | container launch on Windows does not correctly populate classpath with new process's environment variables and localized resources | Major | nodemanager | Chris Nauroth | Chris Nauroth | +| [YARN-586](https://issues.apache.org/jira/browse/YARN-586) | Typo in ApplicationSubmissionContext#setApplicationId | Trivial | . | Zhijie Shen | Zhijie Shen | +| [YARN-585](https://issues.apache.org/jira/browse/YARN-585) | TestFairScheduler#testNotAllowSubmitApplication is broken due to YARN-514 | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-576](https://issues.apache.org/jira/browse/YARN-576) | RM should not allow registrations from NMs that do not satisfy minimum scheduler allocations | Major | . | Hitesh Shah | Kenji Kikushima | +| [YARN-557](https://issues.apache.org/jira/browse/YARN-557) | TestUnmanagedAMLauncher fails on Windows | Major | applications | Chris Nauroth | Chris Nauroth | +| [YARN-542](https://issues.apache.org/jira/browse/YARN-542) | Change the default global AM max-attempts value to be not one | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-541](https://issues.apache.org/jira/browse/YARN-541) | getAllocatedContainers() is not returning all the allocated containers | Blocker | resourcemanager | Krishna Kishore Bonagiri | Bikas Saha | +| [YARN-532](https://issues.apache.org/jira/browse/YARN-532) | RMAdminProtocolPBClientImpl should implement Closeable | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-515](https://issues.apache.org/jira/browse/YARN-515) | Node Manager not getting the master key | Blocker | . | Robert Joseph Evans | Robert Joseph Evans | +| [YARN-512](https://issues.apache.org/jira/browse/YARN-512) | Log aggregation root directory check is more expensive than it needs to be | Minor | nodemanager | Jason Lowe | Maysam Yabandeh | +| [YARN-507](https://issues.apache.org/jira/browse/YARN-507) | Add interface visibility and stability annotations to FS interfaces/classes | Minor | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-506](https://issues.apache.org/jira/browse/YARN-506) | Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute | Major | . | Ivan Mitic | Ivan Mitic | +| [YARN-500](https://issues.apache.org/jira/browse/YARN-500) | ResourceManager webapp is using next port if configured port is already in use | Major | resourcemanager | Nishan Shetty | Kenji Kikushima | +| [YARN-496](https://issues.apache.org/jira/browse/YARN-496) | Fair scheduler configs are refreshed inconsistently in reinitialize | Minor | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-495](https://issues.apache.org/jira/browse/YARN-495) | Change NM behavior of reboot to resync | Major | . | Jian He | Jian He | +| [YARN-493](https://issues.apache.org/jira/browse/YARN-493) | NodeManager job control logic flaws on Windows | Major | nodemanager | Chris Nauroth | Chris Nauroth | +| [YARN-491](https://issues.apache.org/jira/browse/YARN-491) | TestContainerLogsPage fails on Windows | Major | nodemanager | Chris Nauroth | Chris Nauroth | +| [YARN-490](https://issues.apache.org/jira/browse/YARN-490) | TestDistributedShell fails on Windows | Major | applications/distributed-shell | Chris Nauroth | Chris Nauroth | +| [YARN-488](https://issues.apache.org/jira/browse/YARN-488) | TestContainerManagerSecurity fails on Windows | Major | nodemanager | Chris Nauroth | Chris Nauroth | +| [YARN-487](https://issues.apache.org/jira/browse/YARN-487) | TestDiskFailures fails on Windows due to path mishandling | Major | nodemanager | Chris Nauroth | Chris Nauroth | +| [YARN-485](https://issues.apache.org/jira/browse/YARN-485) | TestProcfsProcessTree#testProcessTree() doesn't wait long enough for the process to die | Major | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-481](https://issues.apache.org/jira/browse/YARN-481) | Add AM Host and RPC Port to ApplicationCLI Status Output | Major | client | Chris Riccomini | Chris Riccomini | +| [YARN-479](https://issues.apache.org/jira/browse/YARN-479) | NM retry behavior for connection to RM should be similar for lost heartbeats | Major | . | Hitesh Shah | Jian He | +| [YARN-476](https://issues.apache.org/jira/browse/YARN-476) | ProcfsBasedProcessTree info message confuses users | Minor | . | Jason Lowe | Sandy Ryza | +| [YARN-474](https://issues.apache.org/jira/browse/YARN-474) | CapacityScheduler does not activate applications when maximum-am-resource-percent configuration is refreshed | Major | capacityscheduler | Hitesh Shah | Zhijie Shen | +| [YARN-460](https://issues.apache.org/jira/browse/YARN-460) | CS user left in list of active users for the queue even when application finished | Blocker | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-458](https://issues.apache.org/jira/browse/YARN-458) | YARN daemon addresses must be placed in many different configs | Major | nodemanager, resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-448](https://issues.apache.org/jira/browse/YARN-448) | Remove unnecessary hflush from log aggregation | Major | nodemanager | Kihwal Lee | Kihwal Lee | +| [YARN-426](https://issues.apache.org/jira/browse/YARN-426) | Failure to download a public resource on a node prevents further downloads of the resource from that node | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-412](https://issues.apache.org/jira/browse/YARN-412) | FifoScheduler incorrectly checking for node locality | Minor | scheduler | Roger Hoover | Roger Hoover | +| [YARN-410](https://issues.apache.org/jira/browse/YARN-410) | New lines in diagnostics for a failed app on the per-application page make it hard to read | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-400](https://issues.apache.org/jira/browse/YARN-400) | RM can return null application resource usage report leading to NPE in client | Critical | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-390](https://issues.apache.org/jira/browse/YARN-390) | ApplicationCLI and NodeCLI use hard-coded platform-specific line separator, which causes test failures on Windows | Major | client | Chris Nauroth | Chris Nauroth | +| [YARN-383](https://issues.apache.org/jira/browse/YARN-383) | AMRMClientImpl should handle null rmClient in stop() | Minor | . | Hitesh Shah | Hitesh Shah | +| [YARN-380](https://issues.apache.org/jira/browse/YARN-380) | yarn node -status prints Last-Last-Health-Update | Major | client | Thomas Graves | Omkar Vinit Joshi | +| [YARN-377](https://issues.apache.org/jira/browse/YARN-377) | Fix TestContainersMonitor for HADOOP-9252 | Minor | . | Tsz Wo Nicholas Sze | Chris Nauroth | +| [YARN-376](https://issues.apache.org/jira/browse/YARN-376) | Apps that have completed can appear as RUNNING on the NM UI | Blocker | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-368](https://issues.apache.org/jira/browse/YARN-368) | Fix typo "defiend" should be "defined" in error output | Trivial | . | Albert Chu | Albert Chu | +| [YARN-363](https://issues.apache.org/jira/browse/YARN-363) | yarn proxyserver fails to find webapps/proxy directory on startup | Major | . | Jason Lowe | Kenji Kikushima | +| [YARN-362](https://issues.apache.org/jira/browse/YARN-362) | Unexpected extra results when using webUI table search | Minor | . | Jason Lowe | Ravi Prakash | +| [YARN-345](https://issues.apache.org/jira/browse/YARN-345) | Many InvalidStateTransitonException errors for ApplicationImpl in Node Manager | Critical | nodemanager | Devaraj K | Robert Parker | +| [YARN-333](https://issues.apache.org/jira/browse/YARN-333) | Schedulers cannot control the queue-name of an application | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-319](https://issues.apache.org/jira/browse/YARN-319) | Submit a job to a queue that not allowed in fairScheduler, client will hold forever. | Major | resourcemanager, scheduler | Hong Shen | Hong Shen | +| [YARN-289](https://issues.apache.org/jira/browse/YARN-289) | Fair scheduler allows reservations that won't fit on node | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-269](https://issues.apache.org/jira/browse/YARN-269) | Resource Manager not logging the health\_check\_script result when taking it out | Major | resourcemanager | Thomas Graves | Jason Lowe | +| [YARN-236](https://issues.apache.org/jira/browse/YARN-236) | RM should point tracking URL to RM web page when app fails to start | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-227](https://issues.apache.org/jira/browse/YARN-227) | Application expiration difficult to debug for end-users | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-209](https://issues.apache.org/jira/browse/YARN-209) | Capacity scheduler doesn't trigger app-activation after adding nodes | Major | capacityscheduler | Bikas Saha | Zhijie Shen | +| [YARN-196](https://issues.apache.org/jira/browse/YARN-196) | Nodemanager should be more robust in handling connection failure to ResourceManager when a cluster is started | Major | nodemanager | Ramgopal N | Xuan Gong | +| [YARN-193](https://issues.apache.org/jira/browse/YARN-193) | Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits | Major | resourcemanager | Hitesh Shah | Zhijie Shen | +| [YARN-109](https://issues.apache.org/jira/browse/YARN-109) | .tmp file is not deleted for localized archives | Major | nodemanager | Jason Lowe | Mayank Bansal | +| [YARN-101](https://issues.apache.org/jira/browse/YARN-101) | If the heartbeat message loss, the nodestatus info of complete container will loss too. | Minor | nodemanager | xieguiming | Xuan Gong | +| [YARN-71](https://issues.apache.org/jira/browse/YARN-71) | Ensure/confirm that the NodeManager cleans up local-dirs on restart | Critical | nodemanager | Vinod Kumar Vavilapalli | Xuan Gong | +| [YARN-45](https://issues.apache.org/jira/browse/YARN-45) | [Preemption] Scheduler feedback to AM to release containers | Major | resourcemanager | Chris Douglas | Carlo Curino | +| [YARN-24](https://issues.apache.org/jira/browse/YARN-24) | Nodemanager fails to start if log aggregation enabled and namenode unavailable | Major | nodemanager | Jason Lowe | Sandy Ryza | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9624](https://issues.apache.org/jira/browse/HADOOP-9624) | TestFSMainOperationsLocalFileSystem failed when the Hadoop test root path has "X" in its name | Minor | test | Xi Fang | Xi Fang | +| [HADOOP-9287](https://issues.apache.org/jira/browse/HADOOP-9287) | Parallel testing hadoop-common | Major | test | Tsuyoshi Ozawa | Andrey Klochkov | +| [HADOOP-9233](https://issues.apache.org/jira/browse/HADOOP-9233) | Cover package org.apache.hadoop.io.compress.zlib with unit tests | Major | . | Vadim Bondarev | Vadim Bondarev | +| [HADOOP-9222](https://issues.apache.org/jira/browse/HADOOP-9222) | Cover package with org.apache.hadoop.io.lz4 unit tests | Major | . | Vadim Bondarev | Vadim Bondarev | +| [HDFS-4129](https://issues.apache.org/jira/browse/HDFS-4129) | Add utility methods to dump NameNode in memory tree for testing | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-5333](https://issues.apache.org/jira/browse/MAPREDUCE-5333) | Add test that verifies MRAM works correctly when sending requests with non-normalized capabilities | Major | mr-am | Alejandro Abdelnur | Wei Yan | +| [MAPREDUCE-5015](https://issues.apache.org/jira/browse/MAPREDUCE-5015) | Coverage fix for org.apache.hadoop.mapreduce.tools.CLI | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-5007](https://issues.apache.org/jira/browse/MAPREDUCE-5007) | fix coverage org.apache.hadoop.mapreduce.v2.hs | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4991](https://issues.apache.org/jira/browse/MAPREDUCE-4991) | coverage for gridmix | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4972](https://issues.apache.org/jira/browse/MAPREDUCE-4972) | Coverage fixing for org.apache.hadoop.mapreduce.jobhistory | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [MAPREDUCE-4875](https://issues.apache.org/jira/browse/MAPREDUCE-4875) | coverage fixing for org.apache.hadoop.mapred | Major | test | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-866](https://issues.apache.org/jira/browse/YARN-866) | Add test for class ResourceWeights | Major | . | Wei Yan | Wei Yan | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9720](https://issues.apache.org/jira/browse/HADOOP-9720) | Rename Client#uuid to Client#clientId | Major | . | Suresh Srinivas | Arpit Agarwal | +| [HADOOP-9619](https://issues.apache.org/jira/browse/HADOOP-9619) | Mark stability of .proto files | Major | documentation | Sanjay Radia | Sanjay Radia | +| [HADOOP-9418](https://issues.apache.org/jira/browse/HADOOP-9418) | Add symlink resolution support to DistributedFileSystem | Major | fs | Andrew Wang | Andrew Wang | +| [HADOOP-9416](https://issues.apache.org/jira/browse/HADOOP-9416) | Add new symlink resolution methods in FileSystem and FileSystemLinkResolver | Major | fs | Andrew Wang | Andrew Wang | +| [HADOOP-9414](https://issues.apache.org/jira/browse/HADOOP-9414) | Refactor out FSLinkResolver and relevant helper methods | Major | fs | Andrew Wang | Andrew Wang | +| [HADOOP-9355](https://issues.apache.org/jira/browse/HADOOP-9355) | Abstract symlink tests to use either FileContext or FileSystem | Major | fs | Andrew Wang | Andrew Wang | +| [HADOOP-9258](https://issues.apache.org/jira/browse/HADOOP-9258) | Add stricter tests to FileSystemContractTestBase | Major | test | Steve Loughran | Steve Loughran | +| [HADOOP-9218](https://issues.apache.org/jira/browse/HADOOP-9218) | Document the Rpc-wrappers used internally | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-9140](https://issues.apache.org/jira/browse/HADOOP-9140) | Cleanup rpc PB protos | Major | ipc | Sanjay Radia | Sanjay Radia | +| [HADOOP-8470](https://issues.apache.org/jira/browse/HADOOP-8470) | Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup) | Major | . | Junping Du | Junping Du | +| [HADOOP-8469](https://issues.apache.org/jira/browse/HADOOP-8469) | Make NetworkTopology class pluggable | Major | . | Junping Du | Junping Du | +| [HDFS-5025](https://issues.apache.org/jira/browse/HDFS-5025) | Record ClientId and CallId in EditLog to enable rebuilding retry cache in case of HA failover | Major | ha, namenode | Jing Zhao | Jing Zhao | +| [HDFS-4979](https://issues.apache.org/jira/browse/HDFS-4979) | Implement retry cache on the namenode | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4974](https://issues.apache.org/jira/browse/HDFS-4974) | Analyze and add annotations to Namenode protocol methods and enable retry | Major | ha, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4908](https://issues.apache.org/jira/browse/HDFS-4908) | Reduce snapshot inode memory usage | Major | namenode, snapshots | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4876](https://issues.apache.org/jira/browse/HDFS-4876) | The javadoc of FileWithSnapshot is incorrect | Minor | snapshots | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4875](https://issues.apache.org/jira/browse/HDFS-4875) | Add a test for testing snapshot file length | Minor | snapshots, test | Tsz Wo Nicholas Sze | Arpit Agarwal | +| [HDFS-4842](https://issues.apache.org/jira/browse/HDFS-4842) | Snapshot: identify the correct prior snapshot when deleting a snapshot under a renamed subtree | Major | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-4819](https://issues.apache.org/jira/browse/HDFS-4819) | Update Snapshot doc for HDFS-4758 | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4785](https://issues.apache.org/jira/browse/HDFS-4785) | Concat operation does not remove concatenated files from InodeMap | Major | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-4784](https://issues.apache.org/jira/browse/HDFS-4784) | NPE in FSDirectory.resolvePath() | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-4762](https://issues.apache.org/jira/browse/HDFS-4762) | Provide HDFS based NFSv3 and Mountd implementation | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-4661](https://issues.apache.org/jira/browse/HDFS-4661) | fix various bugs in short circuit read | Major | datanode, hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4602](https://issues.apache.org/jira/browse/HDFS-4602) | TestBookKeeperHACheckpoints fails | Major | . | Suresh Srinivas | Uma Maheswara Rao G | +| [HDFS-4542](https://issues.apache.org/jira/browse/HDFS-4542) | Webhdfs doesn't support secure proxy users | Blocker | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-4525](https://issues.apache.org/jira/browse/HDFS-4525) | Provide an API for knowing that whether file is closed or not. | Major | namenode | Uma Maheswara Rao G | SreeHari | +| [HDFS-4502](https://issues.apache.org/jira/browse/HDFS-4502) | WebHdfsFileSystem handling of fileld breaks compatibility | Blocker | webhdfs | Alejandro Abdelnur | Brandon Li | +| [HDFS-4485](https://issues.apache.org/jira/browse/HDFS-4485) | HDFS-347: DN should chmod socket path a+w | Critical | datanode | Todd Lipcon | Colin Patrick McCabe | +| [HDFS-4374](https://issues.apache.org/jira/browse/HDFS-4374) | Display NameNode startup progress in UI | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-4373](https://issues.apache.org/jira/browse/HDFS-4373) | Add HTTP API for querying NameNode startup progress | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-4372](https://issues.apache.org/jira/browse/HDFS-4372) | Track NameNode startup progress | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-4346](https://issues.apache.org/jira/browse/HDFS-4346) | Refactor INodeId and GenerationStamp | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4340](https://issues.apache.org/jira/browse/HDFS-4340) | Update addBlock() to inculde inode id as additional argument | Major | hdfs-client, namenode | Brandon Li | Brandon Li | +| [HDFS-4339](https://issues.apache.org/jira/browse/HDFS-4339) | Persist inode id in fsimage and editlog | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-4334](https://issues.apache.org/jira/browse/HDFS-4334) | Add a unique id to each INode | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-3769](https://issues.apache.org/jira/browse/HDFS-3769) | standby namenode become active fails because starting log segment fail on shared storage | Critical | ha | liaowenrui | | +| [MAPREDUCE-5299](https://issues.apache.org/jira/browse/MAPREDUCE-5299) | Mapred API: void setTaskID(TaskAttemptID) is missing in TaskCompletionEvent | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5296](https://issues.apache.org/jira/browse/MAPREDUCE-5296) | Mapred API: Function signature change in JobControl | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5280](https://issues.apache.org/jira/browse/MAPREDUCE-5280) | Mapreduce API: ClusterMetrics incompatibility issues with MR1 | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5275](https://issues.apache.org/jira/browse/MAPREDUCE-5275) | Mapreduce API: TokenCache incompatibility issues with MR1 | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5274](https://issues.apache.org/jira/browse/MAPREDUCE-5274) | Mapreduce API: String toHex(byte[]) is removed from SecureShuffleUtils | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5273](https://issues.apache.org/jira/browse/MAPREDUCE-5273) | Protected variables are removed from CombineFileRecordReader in both mapred and mapreduce | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5263](https://issues.apache.org/jira/browse/MAPREDUCE-5263) | filecache.DistributedCache incompatiblity issues with MR1 | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5245](https://issues.apache.org/jira/browse/MAPREDUCE-5245) | A number of public static variables are removed from JobConf | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5244](https://issues.apache.org/jira/browse/MAPREDUCE-5244) | Two functions changed their visibility in JobStatus | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5235](https://issues.apache.org/jira/browse/MAPREDUCE-5235) | mapred.Counters incompatiblity issues with MR1 | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5231](https://issues.apache.org/jira/browse/MAPREDUCE-5231) | Constructor of DBInputFormat.DBRecordReader in mapred is changed | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5230](https://issues.apache.org/jira/browse/MAPREDUCE-5230) | createFileSplit is removed from NLineInputFormat of mapred | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5229](https://issues.apache.org/jira/browse/MAPREDUCE-5229) | TEMP\_DIR\_NAME is removed from of FileOutputCommitter of mapreduce | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5228](https://issues.apache.org/jira/browse/MAPREDUCE-5228) | Enum Counter is removed from FileInputFormat and FileOutputFormat of both mapred and mapreduce | Major | . | Zhijie Shen | Mayank Bansal | +| [MAPREDUCE-5222](https://issues.apache.org/jira/browse/MAPREDUCE-5222) | Fix JobClient incompatibilities with MR1 | Major | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5220](https://issues.apache.org/jira/browse/MAPREDUCE-5220) | Mapred API: TaskCompletionEvent incompatibility issues with MR1 | Major | client | Sandy Ryza | Zhijie Shen | +| [MAPREDUCE-5199](https://issues.apache.org/jira/browse/MAPREDUCE-5199) | AppTokens file can/should be removed | Blocker | security | Vinod Kumar Vavilapalli | Daryn Sharp | +| [MAPREDUCE-5184](https://issues.apache.org/jira/browse/MAPREDUCE-5184) | Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2 | Major | documentation | Arun C Murthy | Zhijie Shen | +| [MAPREDUCE-5159](https://issues.apache.org/jira/browse/MAPREDUCE-5159) | Aggregatewordcount and aggregatewordhist in hadoop-1 examples are not binary compatible with hadoop-2 mapred.lib.aggregate | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5157](https://issues.apache.org/jira/browse/MAPREDUCE-5157) | Sort in hadoop-1 examples is not binary compatible with hadoop-2 mapred.lib | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5086](https://issues.apache.org/jira/browse/MAPREDUCE-5086) | MR app master deletes staging dir when sent a reboot command from the RM | Major | . | Jian He | Jian He | +| [MAPREDUCE-4951](https://issues.apache.org/jira/browse/MAPREDUCE-4951) | Container preemption interpreted as task failure | Major | applicationmaster, mr-am, mrv2 | Sandy Ryza | Sandy Ryza | +| [YARN-961](https://issues.apache.org/jira/browse/YARN-961) | ContainerManagerImpl should enforce token on server. Today it is [TOKEN, SIMPLE] | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-922](https://issues.apache.org/jira/browse/YARN-922) | Change FileSystemRMStateStore to use directories | Major | resourcemanager | Jian He | Jian He | +| [YARN-877](https://issues.apache.org/jira/browse/YARN-877) | Allow for black-listing resources in FifoScheduler | Major | scheduler | Junping Du | Junping Du | +| [YARN-873](https://issues.apache.org/jira/browse/YARN-873) | YARNClient.getApplicationReport(unknownAppId) returns a null report | Major | . | Bikas Saha | Xuan Gong | +| [YARN-850](https://issues.apache.org/jira/browse/YARN-850) | Rename getClusterAvailableResources to getAvailableResources in AMRMClients | Major | . | Jian He | Jian He | +| [YARN-846](https://issues.apache.org/jira/browse/YARN-846) | Move pb Impl from yarn-api to yarn-common | Major | . | Jian He | Jian He | +| [YARN-845](https://issues.apache.org/jira/browse/YARN-845) | RM crash with NPE on NODE\_UPDATE | Major | resourcemanager | Arpit Gupta | Mayank Bansal | +| [YARN-827](https://issues.apache.org/jira/browse/YARN-827) | Need to make Resource arithmetic methods accessible | Critical | . | Bikas Saha | Jian He | +| [YARN-825](https://issues.apache.org/jira/browse/YARN-825) | Fix yarn-common javadoc annotations | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-820](https://issues.apache.org/jira/browse/YARN-820) | NodeManager has invalid state transition after error in resource localization | Major | . | Bikas Saha | Mayank Bansal | +| [YARN-814](https://issues.apache.org/jira/browse/YARN-814) | Difficult to diagnose a failed container launch when error due to invalid environment variable | Major | . | Hitesh Shah | Jian He | +| [YARN-805](https://issues.apache.org/jira/browse/YARN-805) | Fix yarn-api javadoc annotations | Blocker | . | Jian He | Jian He | +| [YARN-781](https://issues.apache.org/jira/browse/YARN-781) | Expose LOGDIR that containers should use for logging | Major | . | Devaraj Das | Jian He | +| [YARN-773](https://issues.apache.org/jira/browse/YARN-773) | Move YarnRuntimeException from package api.yarn to api.yarn.exceptions | Major | . | Jian He | Jian He | +| [YARN-759](https://issues.apache.org/jira/browse/YARN-759) | Create Command enum in AllocateResponse | Major | . | Bikas Saha | Bikas Saha | +| [YARN-750](https://issues.apache.org/jira/browse/YARN-750) | Allow for black-listing resources in YARN API and Impl in CS | Major | . | Arun C Murthy | Arun C Murthy | +| [YARN-739](https://issues.apache.org/jira/browse/YARN-739) | NM startContainer should validate the NodeId | Major | . | Siddharth Seth | Omkar Vinit Joshi | +| [YARN-737](https://issues.apache.org/jira/browse/YARN-737) | Some Exceptions no longer need to be wrapped by YarnException and can be directly thrown out after YARN-142 | Major | . | Jian He | Jian He | +| [YARN-731](https://issues.apache.org/jira/browse/YARN-731) | RPCUtil.unwrapAndThrowException should unwrap remote RuntimeExceptions | Major | . | Siddharth Seth | Zhijie Shen | +| [YARN-727](https://issues.apache.org/jira/browse/YARN-727) | ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter | Blocker | . | Siddharth Seth | Xuan Gong | +| [YARN-719](https://issues.apache.org/jira/browse/YARN-719) | Move RMIdentifier from Container to ContainerTokenIdentifier | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-717](https://issues.apache.org/jira/browse/YARN-717) | Copy BuilderUtil methods into token-related records | Major | . | Jian He | Jian He | +| [YARN-714](https://issues.apache.org/jira/browse/YARN-714) | AMRM protocol changes for sending NMToken list | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-711](https://issues.apache.org/jira/browse/YARN-711) | Copy BuilderUtil methods into individual records | Major | . | Vinod Kumar Vavilapalli | Jian He | +| [YARN-695](https://issues.apache.org/jira/browse/YARN-695) | masterContainer and status are in ApplicationReportProto but not in ApplicationReport | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-693](https://issues.apache.org/jira/browse/YARN-693) | Sending NMToken to AM on allocate call | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-692](https://issues.apache.org/jira/browse/YARN-692) | Creating NMToken master key on RM and sharing it with NM as a part of RM-NM heartbeat. | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-686](https://issues.apache.org/jira/browse/YARN-686) | Flatten NodeReport | Major | api | Sandy Ryza | Sandy Ryza | +| [YARN-663](https://issues.apache.org/jira/browse/YARN-663) | Change ResourceTracker API and LocalizationProtocol API to throw YarnRemoteException and IOException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-660](https://issues.apache.org/jira/browse/YARN-660) | Improve AMRMClient with matching requests | Major | . | Bikas Saha | Bikas Saha | +| [YARN-651](https://issues.apache.org/jira/browse/YARN-651) | Change ContainerManagerPBClientImpl and RMAdminProtocolPBClientImpl to throw IOException and YarnRemoteException | Major | . | Xuan Gong | Xuan Gong | +| [YARN-638](https://issues.apache.org/jira/browse/YARN-638) | Restore RMDelegationTokens after RM Restart | Major | resourcemanager | Jian He | Jian He | +| [YARN-634](https://issues.apache.org/jira/browse/YARN-634) | Make YarnRemoteException not backed by PB and introduce a SerializedException | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-628](https://issues.apache.org/jira/browse/YARN-628) | Fix YarnException unwrapping | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-625](https://issues.apache.org/jira/browse/YARN-625) | Move unwrapAndThrowException from YarnRemoteExceptionPBImpl to RPCUtil | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-617](https://issues.apache.org/jira/browse/YARN-617) | In unsercure mode, AM can fake resource requirements | Minor | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-613](https://issues.apache.org/jira/browse/YARN-613) | Create NM proxy per NM instead of per container | Major | . | Bikas Saha | Omkar Vinit Joshi | +| [YARN-595](https://issues.apache.org/jira/browse/YARN-595) | Refactor fair scheduler to use common Resources | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-591](https://issues.apache.org/jira/browse/YARN-591) | RM recovery related records do not belong to the API | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-583](https://issues.apache.org/jira/browse/YARN-583) | Application cache files should be localized under local-dir/usercache/userid/appcache/appid/filecache | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-582](https://issues.apache.org/jira/browse/YARN-582) | Restore appToken and clientToken for app attempt after RM restart | Major | resourcemanager | Bikas Saha | Jian He | +| [YARN-581](https://issues.apache.org/jira/browse/YARN-581) | Test and verify that app delegation tokens are added to tokenRenewer after RM restart | Major | resourcemanager | Bikas Saha | Jian He | +| [YARN-578](https://issues.apache.org/jira/browse/YARN-578) | NodeManager should use SecureIOUtils for serving and aggregating logs | Major | nodemanager | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-577](https://issues.apache.org/jira/browse/YARN-577) | ApplicationReport does not provide progress value of application | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-569](https://issues.apache.org/jira/browse/YARN-569) | CapacityScheduler: support for preemption (using a capacity monitor) | Major | capacityscheduler | Carlo Curino | Carlo Curino | +| [YARN-568](https://issues.apache.org/jira/browse/YARN-568) | FairScheduler: support for work-preserving preemption | Major | scheduler | Carlo Curino | Carlo Curino | +| [YARN-567](https://issues.apache.org/jira/browse/YARN-567) | RM changes to support preemption for FairScheduler and CapacityScheduler | Major | resourcemanager | Carlo Curino | Carlo Curino | +| [YARN-563](https://issues.apache.org/jira/browse/YARN-563) | Add application type to ApplicationReport | Major | . | Thomas Weise | Mayank Bansal | +| [YARN-562](https://issues.apache.org/jira/browse/YARN-562) | NM should reject containers allocated by previous RM | Major | resourcemanager | Jian He | Jian He | +| [YARN-549](https://issues.apache.org/jira/browse/YARN-549) | YarnClient.submitApplication should wait for application to be accepted by the RM | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-548](https://issues.apache.org/jira/browse/YARN-548) | Add tests for YarnUncaughtExceptionHandler | Major | . | Vadim Bondarev | Vadim Bondarev | +| [YARN-547](https://issues.apache.org/jira/browse/YARN-547) | Race condition in Public / Private Localizer may result into resource getting downloaded again | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-539](https://issues.apache.org/jira/browse/YARN-539) | LocalizedResources are leaked in memory in case resource localization fails | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-534](https://issues.apache.org/jira/browse/YARN-534) | AM max attempts is not checked when RM restart and try to recover attempts | Major | resourcemanager | Jian He | Jian He | +| [YARN-523](https://issues.apache.org/jira/browse/YARN-523) | Container localization failures aren't reported from NM to RM | Major | . | Vinod Kumar Vavilapalli | Jian He | +| [YARN-521](https://issues.apache.org/jira/browse/YARN-521) | Augment AM - RM client module to be able to request containers only at specific locations | Major | api | Sandy Ryza | Sandy Ryza | +| [YARN-514](https://issues.apache.org/jira/browse/YARN-514) | Delayed store operations should not result in RM unavailability for app submission | Major | resourcemanager | Bikas Saha | Zhijie Shen | +| [YARN-513](https://issues.apache.org/jira/browse/YARN-513) | Create common proxy client for communicating with RM | Major | resourcemanager | Bikas Saha | Jian He | +| [YARN-486](https://issues.apache.org/jira/browse/YARN-486) | Change startContainer NM API to accept Container as a parameter and make ContainerLaunchContext user land | Major | . | Bikas Saha | Xuan Gong | +| [YARN-482](https://issues.apache.org/jira/browse/YARN-482) | FS: Extend SchedulingMode to intermediate queues | Major | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-475](https://issues.apache.org/jira/browse/YARN-475) | Remove ApplicationConstants.AM\_APP\_ATTEMPT\_ID\_ENV as it is no longer set in an AM's environment | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-469](https://issues.apache.org/jira/browse/YARN-469) | Make scheduling mode in FS pluggable | Major | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-468](https://issues.apache.org/jira/browse/YARN-468) | coverage fix for org.apache.hadoop.yarn.server.webproxy.amfilter | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-467](https://issues.apache.org/jira/browse/YARN-467) | Jobs fail during resource localization when public distributed-cache hits unix directory limits | Major | nodemanager | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-450](https://issues.apache.org/jira/browse/YARN-450) | Define value for \* in the scheduling protocol | Major | . | Bikas Saha | Zhijie Shen | +| [YARN-444](https://issues.apache.org/jira/browse/YARN-444) | Move special container exit codes from YarnConfiguration to API | Major | api, applications/distributed-shell | Sandy Ryza | Sandy Ryza | +| [YARN-441](https://issues.apache.org/jira/browse/YARN-441) | Clean up unused collection methods in various APIs | Major | . | Siddharth Seth | Xuan Gong | +| [YARN-422](https://issues.apache.org/jira/browse/YARN-422) | Add NM client library | Major | . | Bikas Saha | Zhijie Shen | +| [YARN-417](https://issues.apache.org/jira/browse/YARN-417) | Create AMRMClient wrapper that provides asynchronous callbacks | Major | api, applications | Sandy Ryza | Sandy Ryza | +| [YARN-398](https://issues.apache.org/jira/browse/YARN-398) | Enhance CS to allow for white-list of resources | Major | . | Arun C Murthy | Arun C Murthy | +| [YARN-392](https://issues.apache.org/jira/browse/YARN-392) | Make it possible to specify hard locality constraints in resource requests | Major | resourcemanager | Bikas Saha | Sandy Ryza | +| [YARN-378](https://issues.apache.org/jira/browse/YARN-378) | ApplicationMaster retry times should be set by Client | Major | client, resourcemanager | xieguiming | Zhijie Shen | +| [YARN-369](https://issues.apache.org/jira/browse/YARN-369) | Handle ( or throw a proper error when receiving) status updates from application masters that have not registered | Major | resourcemanager | Hitesh Shah | Mayank Bansal | +| [YARN-365](https://issues.apache.org/jira/browse/YARN-365) | Each NM heartbeat should not generate an event for the Scheduler | Major | resourcemanager, scheduler | Siddharth Seth | Xuan Gong | +| [YARN-309](https://issues.apache.org/jira/browse/YARN-309) | Make RM provide heartbeat interval to NM | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-295](https://issues.apache.org/jira/browse/YARN-295) | Resource Manager throws InvalidStateTransitonException: Invalid event: CONTAINER\_FINISHED at ALLOCATED for RMAppAttemptImpl | Major | resourcemanager | Devaraj K | Mayank Bansal | +| [YARN-200](https://issues.apache.org/jira/browse/YARN-200) | yarn log does not output all needed information, and is in a binary format | Major | . | Robert Joseph Evans | Ravi Prakash | +| [YARN-125](https://issues.apache.org/jira/browse/YARN-125) | Make Yarn Client service shutdown operations robust | Minor | . | Steve Loughran | Steve Loughran | +| [YARN-124](https://issues.apache.org/jira/browse/YARN-124) | Make Yarn Node Manager services robust against shutdown | Minor | . | Steve Loughran | Steve Loughran | +| [YARN-123](https://issues.apache.org/jira/browse/YARN-123) | Make yarn Resource Manager services robust against shutdown | Minor | . | Steve Loughran | Steve Loughran | +| [YARN-112](https://issues.apache.org/jira/browse/YARN-112) | Race in localization can cause containers to fail | Major | nodemanager | Jason Lowe | Omkar Vinit Joshi | +| [YARN-99](https://issues.apache.org/jira/browse/YARN-99) | Jobs fail during resource localization when private distributed-cache hits unix directory limits | Major | nodemanager | Devaraj K | Omkar Vinit Joshi | +| [YARN-62](https://issues.apache.org/jira/browse/YARN-62) | AM should not be able to abuse container tokens for repetitive container launches | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5286](https://issues.apache.org/jira/browse/MAPREDUCE-5286) | startContainer call should use the ContainerToken instead of Container [YARN-684] | Major | . | Siddharth Seth | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5194](https://issues.apache.org/jira/browse/MAPREDUCE-5194) | Heed interrupts during Fetcher shutdown | Minor | task | Chris Douglas | Chris Douglas | +| [MAPREDUCE-5192](https://issues.apache.org/jira/browse/MAPREDUCE-5192) | Separate TCE resolution from fetch | Minor | task | Chris Douglas | Chris Douglas | +| [MAPREDUCE-3502](https://issues.apache.org/jira/browse/MAPREDUCE-3502) | Review all Service.stop() operations and make sure that they work before a service is started | Major | mrv2 | Steve Loughran | Steve Loughran | +| [YARN-927](https://issues.apache.org/jira/browse/YARN-927) | Change ContainerRequest to not have more than 1 container count and remove StoreContainerRequest | Major | . | Bikas Saha | Bikas Saha | +| [YARN-708](https://issues.apache.org/jira/browse/YARN-708) | Move RecordFactory classes to hadoop-yarn-api, miscellaneous fixes to the interfaces | Major | . | Siddharth Seth | Siddharth Seth | +| [YARN-142](https://issues.apache.org/jira/browse/YARN-142) | [Umbrella] Cleanup YARN APIs w.r.t exceptions | Blocker | . | Siddharth Seth | | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md new file mode 100644 index 00000000000..0dbd6bdfeae --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.0-beta/RELEASENOTES.2.1.0-beta.md @@ -0,0 +1,757 @@ + + +# Apache Hadoop 2.1.0-beta Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9832](https://issues.apache.org/jira/browse/HADOOP-9832) | *Blocker* | **Add RPC header to client ping** + +Client ping will be sent as a RPC header with a reserved callId instead of as a sentinel RPC packet length. + + +--- + +* [HADOOP-9820](https://issues.apache.org/jira/browse/HADOOP-9820) | *Blocker* | **RPCv9 wire protocol is insufficient to support multiplexing** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9698](https://issues.apache.org/jira/browse/HADOOP-9698) | *Blocker* | **RPCv9 client must honor server's SASL negotiate response** + +The RPC client now waits for the Server's SASL negotiate response before instantiating its SASL client. + + +--- + +* [HADOOP-9683](https://issues.apache.org/jira/browse/HADOOP-9683) | *Blocker* | **Wrap IpcConnectionContext in RPC headers** + +Connection context is now sent as a rpc header wrapped protobuf. + + +--- + +* [HADOOP-9649](https://issues.apache.org/jira/browse/HADOOP-9649) | *Blocker* | **Promote YARN service life-cycle libraries into Hadoop Common** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9630](https://issues.apache.org/jira/browse/HADOOP-9630) | *Major* | **Remove IpcSerializationType** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9425](https://issues.apache.org/jira/browse/HADOOP-9425) | *Major* | **Add error codes to rpc-response** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9421](https://issues.apache.org/jira/browse/HADOOP-9421) | *Blocker* | **Convert SASL to use ProtoBuf and provide negotiation capabilities** + +Raw SASL protocol now uses protobufs wrapped with RPC headers. +The negotiation sequence incorporates the state of the exchange. +The server now has the ability to advertise its supported auth types. + + +--- + +* [HADOOP-9380](https://issues.apache.org/jira/browse/HADOOP-9380) | *Major* | **Add totalLength to rpc response** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9228](https://issues.apache.org/jira/browse/HADOOP-9228) | *Minor* | **FileSystemContractTestBase never verifies that files are files** + +fixed in HADOOP-9258 + + +--- + +* [HADOOP-9227](https://issues.apache.org/jira/browse/HADOOP-9227) | *Trivial* | **FileSystemContractBaseTest doesn't test filesystem's mkdir/isDirectory() logic rigorously enough** + +fixed in HADOOP-9258 + + +--- + +* [HADOOP-9194](https://issues.apache.org/jira/browse/HADOOP-9194) | *Major* | **RPC Support for QoS** + +Part of the RPC version 9 change. A service class byte is added after the version byte. + + +--- + +* [HADOOP-9163](https://issues.apache.org/jira/browse/HADOOP-9163) | *Major* | **The rpc msg in ProtobufRpcEngine.proto should be moved out to avoid an extra copy** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-9151](https://issues.apache.org/jira/browse/HADOOP-9151) | *Major* | **Include RPC error info in RpcResponseHeader instead of sending it separately** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-8886](https://issues.apache.org/jira/browse/HADOOP-8886) | *Major* | **Remove KFS support** + +Kosmos FS (KFS) is no longer maintained and Hadoop support has been removed. KFS has been replaced by QFS (HADOOP-8885). + + +--- + +* [HADOOP-8562](https://issues.apache.org/jira/browse/HADOOP-8562) | *Major* | **Enhancements to support Hadoop on Windows Server and Windows Azure environments** + +This umbrella jira makes enhancements to support Hadoop natively on Windows Server and Windows Azure environments. + + +--- + +* [HADOOP-8470](https://issues.apache.org/jira/browse/HADOOP-8470) | *Major* | **Implementation of 4-layer subclass of NetworkTopology (NetworkTopologyWithNodeGroup)** + +This patch should be checked in together (or after) with JIRA Hadoop-8469: https://issues.apache.org/jira/browse/HADOOP-8469 + + +--- + +* [HDFS-5083](https://issues.apache.org/jira/browse/HDFS-5083) | *Blocker* | **Update the HDFS compatibility version range** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-4996](https://issues.apache.org/jira/browse/HDFS-4996) | *Minor* | **ClientProtocol#metaSave can be made idempotent by overwriting the output file instead of appending to it** + +The dfsadmin -metasave command has been changed to overwrite the output file. Previously, this command would append to the output file if it already existed. + + +--- + +* [HDFS-4866](https://issues.apache.org/jira/browse/HDFS-4866) | *Blocker* | **Protocol buffer support cannot compile under C** + +The Protocol Buffers definition of the inter-namenode protocol required a change for compatibility with compiled C clients. This is a backwards-incompatible change. A namenode prior to this change will not be able to communicate with a namenode after this change. + + +--- + +* [HDFS-4832](https://issues.apache.org/jira/browse/HDFS-4832) | *Critical* | **Namenode doesn't change the number of missing blocks in safemode when DNs rejoin or leave** + +This change makes name node keep its internal replication queues and data node state updated in manual safe mode. This allows metrics and UI to present up-to-date information while in safe mode. The behavior during start-up safe mode is unchanged. + + +--- + +* [HDFS-4659](https://issues.apache.org/jira/browse/HDFS-4659) | *Major* | **Support setting execution bit for regular files** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-4519](https://issues.apache.org/jira/browse/HDFS-4519) | *Major* | **Support override of jsvc binary and log file locations when launching secure datanode.** + +With this improvement the following options are available in release 1.2.0 and later on 1.x release stream: +1. jsvc location can be overridden by setting environment variable JSVC\_HOME. Defaults to jsvc binary packaged within the Hadoop distro. +2. jsvc log output is directed to the file defined by JSVC\_OUTFILE. Defaults to $HADOOP\_LOG\_DIR/jsvc.out. +3. jsvc error output is directed to the file defined by JSVC\_ERRFILE file. Defaults to $HADOOP\_LOG\_DIR/jsvc.err. + +With this improvement the following options are available in release 2.0.4 and later on 2.x release stream: +1. jsvc log output is directed to the file defined by JSVC\_OUTFILE. Defaults to $HADOOP\_LOG\_DIR/jsvc.out. +2. jsvc error output is directed to the file defined by JSVC\_ERRFILE file. Defaults to $HADOOP\_LOG\_DIR/jsvc.err. + +For overriding jsvc location on 2.x releases, here is the release notes from HDFS-2303: +To run secure Datanodes users must install jsvc for their platform and set JSVC\_HOME to point to the location of jsvc in their environment. + + +--- + +* [HDFS-4434](https://issues.apache.org/jira/browse/HDFS-4434) | *Major* | **Provide a mapping from INodeId to INode** + +This change adds support for referencing files and directories based on fileID/inodeID using a path /.reserved/.inodes/\. With this change creating a file or directory /.reserved is not longer allowed. Before upgrading to a release with this change, files /.reserved needs to be renamed to another name. + + +--- + +* [HDFS-4305](https://issues.apache.org/jira/browse/HDFS-4305) | *Minor* | **Add a configurable limit on number of blocks per file, and min block size** + +This change introduces a maximum number of blocks per file, by default one million, and a minimum block size, by default 1MB. These can optionally be changed via the configuration settings "dfs.namenode.fs-limits.max-blocks-per-file" and "dfs.namenode.fs-limits.min-block-size", respectively. + + +--- + +* [HDFS-4053](https://issues.apache.org/jira/browse/HDFS-4053) | *Major* | **Increase the default block size** + +The default blocks size prior to this change was 64MB. This jira changes the default block size to 128MB. To go back to previous behavior, please configure the in hdfs-site.xml, the configuration parameter "dfs.blocksize" to 67108864. + + +--- + +* [HDFS-2802](https://issues.apache.org/jira/browse/HDFS-2802) | *Major* | **Support for RW/RO snapshots in HDFS** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-1804](https://issues.apache.org/jira/browse/HDFS-1804) | *Minor* | **Add a new block-volume device choosing policy that looks at free space** + +There is now a new option to have the DN take into account available disk space on each volume when choosing where to place a replica when performing an HDFS write. This can be enabled by setting the config "dfs.datanode.fsdataset.volume.choosing.policy" to the value "org.apache.hadoop.hdfs.server.datanode.fsdataset.AvailableSpaceVolumeChoosingPolicy". + + +--- + +* [MAPREDUCE-5399](https://issues.apache.org/jira/browse/MAPREDUCE-5399) | *Blocker* | **Unnecessary Configuration instantiation in IFileInputStream slows down merge** + +Fixes blank Configuration object creation overhead by reusing the Job configuration in InMemoryReader. + + +--- + +* [MAPREDUCE-5304](https://issues.apache.org/jira/browse/MAPREDUCE-5304) | *Blocker* | **mapreduce.Job killTask/failTask/getTaskCompletionEvents methods have incompatible signature changes** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5300](https://issues.apache.org/jira/browse/MAPREDUCE-5300) | *Major* | **Two function signature changes in filecache.DistributedCache** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5237](https://issues.apache.org/jira/browse/MAPREDUCE-5237) | *Major* | **ClusterStatus incompatiblity issues with MR1** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5234](https://issues.apache.org/jira/browse/MAPREDUCE-5234) | *Major* | **Signature changes for getTaskId of TaskReport in mapred** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5233](https://issues.apache.org/jira/browse/MAPREDUCE-5233) | *Major* | **Functions are changed or removed from Job in jobcontrol** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5184](https://issues.apache.org/jira/browse/MAPREDUCE-5184) | *Major* | **Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2** + +Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2 for end-users. + + +--- + +* [MAPREDUCE-5176](https://issues.apache.org/jira/browse/MAPREDUCE-5176) | *Major* | **Preemptable annotations (to support preemption in MR)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-5156](https://issues.apache.org/jira/browse/MAPREDUCE-5156) | *Blocker* | **Hadoop-examples-1.x.x.jar cannot run on Yarn** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4942](https://issues.apache.org/jira/browse/MAPREDUCE-4942) | *Major* | **mapreduce.Job has a bunch of methods that throw InterruptedException so its incompatible with MR1** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-4737](https://issues.apache.org/jira/browse/MAPREDUCE-4737) | *Major* | ** Hadoop does not close output file / does not call Mapper.cleanup if exception in map** + +Ensure that mapreduce APIs are semantically consistent with mapred API w.r.t Mapper.cleanup and Reducer.cleanup; in the sense that cleanup is now called even if there is an error. The old mapred API already ensures that Mapper.close and Reducer.close are invoked during error handling. Note that it is an incompatible change, however end-users can override Mapper.run and Reducer.run to get the old (inconsistent) behaviour. + + +--- + +* [MAPREDUCE-4356](https://issues.apache.org/jira/browse/MAPREDUCE-4356) | *Major* | **Provide access to ParsedTask.obtainTaskAttempts()** + +Made the method ParsedTask.obtainTaskAttempts() public. + + +--- + +* [MAPREDUCE-4149](https://issues.apache.org/jira/browse/MAPREDUCE-4149) | *Major* | **Rumen fails to parse certain counter strings** + +Fixes Rumen to parse counter strings containing the special characters "{" and "}". + + +--- + +* [MAPREDUCE-4100](https://issues.apache.org/jira/browse/MAPREDUCE-4100) | *Minor* | **Sometimes gridmix emulates data larger much larger then acutal counter for map only jobs** + +Bug fixed in compression emulation feature for map only jobs. + + +--- + +* [MAPREDUCE-4083](https://issues.apache.org/jira/browse/MAPREDUCE-4083) | *Major* | **GridMix emulated job tasks.resource-usage emulator for CPU usage throws NPE when Trace contains cumulativeCpuUsage value of 0 at attempt level** + +Fixes NPE in cpu emulation in Gridmix + + +--- + +* [MAPREDUCE-4067](https://issues.apache.org/jira/browse/MAPREDUCE-4067) | *Critical* | **Replace YarnRemoteException with IOException in MRv2 APIs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [MAPREDUCE-3953](https://issues.apache.org/jira/browse/MAPREDUCE-3953) | *Major* | **Gridmix throws NPE and does not simulate a job if the trace contains null taskStatus for a task** + +Fixes NPE and makes Gridmix simulate succeeded-jobs-with-failed-tasks. All tasks of such simulated jobs(including the failed ones of original job) will succeed. + + +--- + +* [MAPREDUCE-3829](https://issues.apache.org/jira/browse/MAPREDUCE-3829) | *Major* | **[Gridmix] Gridmix should give better error message when input-data directory already exists and -generate option is given** + +Makes Gridmix emit out correct error message when the input data directory already exists and -generate option is used. Makes Gridmix exit with proper exit codes when Gridmix fails in args-processing, startup/setup. + + +--- + +* [MAPREDUCE-3787](https://issues.apache.org/jira/browse/MAPREDUCE-3787) | *Major* | **[Gridmix] Improve STRESS mode** + +JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary. + + +--- + +* [MAPREDUCE-3757](https://issues.apache.org/jira/browse/MAPREDUCE-3757) | *Major* | **Rumen Folder is not adjusting the shuffleFinished and sortFinished times of reduce task attempts** + +Fixed the sortFinishTime and shuffleFinishTime adjustments in Rumen Folder. + + +--- + +* [MAPREDUCE-2722](https://issues.apache.org/jira/browse/MAPREDUCE-2722) | *Major* | **Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used** + +Makes Gridmix use the uncompressed input data size while simulating map tasks in the case where compressed input data was used in original job. + + +--- + +* [YARN-1056](https://issues.apache.org/jira/browse/YARN-1056) | *Trivial* | **Fix configs yarn.resourcemanager.resourcemanager.connect.{max.wait.secs\|retry\_interval.secs}** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-926](https://issues.apache.org/jira/browse/YARN-926) | *Blocker* | **ContainerManagerProtcol APIs should take in requests for multiple containers** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-918](https://issues.apache.org/jira/browse/YARN-918) | *Blocker* | **ApplicationMasterProtocol doesn't need ApplicationAttemptId in the payload after YARN-701** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-869](https://issues.apache.org/jira/browse/YARN-869) | *Blocker* | **ResourceManagerAdministrationProtocol should neither be public(yet) nor in yarn.api** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-851](https://issues.apache.org/jira/browse/YARN-851) | *Major* | **Share NMTokens using NMTokenCache (api-based) instead of memory based approach which is used currently.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-841](https://issues.apache.org/jira/browse/YARN-841) | *Major* | **Annotate and document AuxService APIs** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-840](https://issues.apache.org/jira/browse/YARN-840) | *Major* | **Move ProtoUtils to yarn.api.records.pb.impl** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-837](https://issues.apache.org/jira/browse/YARN-837) | *Major* | **ClusterInfo.java doesn't seem to belong to org.apache.hadoop.yarn** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-834](https://issues.apache.org/jira/browse/YARN-834) | *Blocker* | **Review/fix annotations for yarn-client module and clearly differentiate \*Async apis** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-831](https://issues.apache.org/jira/browse/YARN-831) | *Blocker* | **Remove resource min from GetNewApplicationResponse** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-829](https://issues.apache.org/jira/browse/YARN-829) | *Major* | **Rename RMTokenSelector to be RMDelegationTokenSelector** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-828](https://issues.apache.org/jira/browse/YARN-828) | *Major* | **Remove YarnVersionAnnotation** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-826](https://issues.apache.org/jira/browse/YARN-826) | *Major* | **Move Clock/SystemClock to util package** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-824](https://issues.apache.org/jira/browse/YARN-824) | *Major* | **Add static factory to yarn client lib interface and change it to abstract class** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-823](https://issues.apache.org/jira/browse/YARN-823) | *Major* | **Move RMAdmin from yarn.client to yarn.client.cli and rename as RMAdminCLI** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-822](https://issues.apache.org/jira/browse/YARN-822) | *Major* | **Rename ApplicationToken to AMRMToken** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-821](https://issues.apache.org/jira/browse/YARN-821) | *Major* | **Rename FinishApplicationMasterRequest.setFinishApplicationStatus to setFinalApplicationStatus to be consistent with getter** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-806](https://issues.apache.org/jira/browse/YARN-806) | *Major* | **Move ContainerExitStatus from yarn.api to yarn.api.records** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-792](https://issues.apache.org/jira/browse/YARN-792) | *Major* | **Move NodeHealthStatus from yarn.api.record to yarn.server.api.record** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-791](https://issues.apache.org/jira/browse/YARN-791) | *Blocker* | **Ensure that RM RPC APIs that return nodes are consistent with /nodes REST API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-787](https://issues.apache.org/jira/browse/YARN-787) | *Blocker* | **Remove resource min from Yarn client API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-777](https://issues.apache.org/jira/browse/YARN-777) | *Major* | **Remove unreferenced objects from proto** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-756](https://issues.apache.org/jira/browse/YARN-756) | *Major* | **Move PreemptionContainer/PremptionContract/PreemptionMessage/StrictPreemptionContract/PreemptionResourceRequest to api.records** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-755](https://issues.apache.org/jira/browse/YARN-755) | *Major* | **Rename AllocateResponse.reboot to AllocateResponse.resync** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-753](https://issues.apache.org/jira/browse/YARN-753) | *Major* | **Add individual factory method for api protocol records** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-749](https://issues.apache.org/jira/browse/YARN-749) | *Major* | **Rename ResourceRequest (get,set)HostName to (get,set)ResourceName** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-748](https://issues.apache.org/jira/browse/YARN-748) | *Major* | **Move BuilderUtils from yarn-common to yarn-server-common** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-746](https://issues.apache.org/jira/browse/YARN-746) | *Major* | **rename Service.register() and Service.unregister() to registerServiceListener() & unregisterServiceListener() respectively** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-735](https://issues.apache.org/jira/browse/YARN-735) | *Major* | **Make ApplicationAttemptID, ContainerID, NodeID immutable** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-724](https://issues.apache.org/jira/browse/YARN-724) | *Major* | **Move ProtoBase from api.records to api.records.impl.pb** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-720](https://issues.apache.org/jira/browse/YARN-720) | *Major* | **container-log4j.properties should not refer to mapreduce properties** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-716](https://issues.apache.org/jira/browse/YARN-716) | *Major* | **Make ApplicationID immutable** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-701](https://issues.apache.org/jira/browse/YARN-701) | *Blocker* | **ApplicationTokens should be used irrespective of kerberos** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-694](https://issues.apache.org/jira/browse/YARN-694) | *Major* | **Start using NMTokens to authenticate all communication with NM** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-684](https://issues.apache.org/jira/browse/YARN-684) | *Major* | **ContainerManager.startContainer needs to only have ContainerTokenIdentifier instead of the whole Container** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-642](https://issues.apache.org/jira/browse/YARN-642) | *Major* | **Fix up /nodes REST API to have 1 param and be consistent with the Java API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-635](https://issues.apache.org/jira/browse/YARN-635) | *Major* | **Rename YarnRemoteException to YarnException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-633](https://issues.apache.org/jira/browse/YARN-633) | *Major* | **Change RMAdminProtocol api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-632](https://issues.apache.org/jira/browse/YARN-632) | *Major* | **Change ContainerManager api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-631](https://issues.apache.org/jira/browse/YARN-631) | *Major* | **Change ClientRMProtocol api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-630](https://issues.apache.org/jira/browse/YARN-630) | *Major* | **Change AMRMProtocol api to throw IOException and YarnRemoteException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-629](https://issues.apache.org/jira/browse/YARN-629) | *Major* | **Make YarnRemoteException not be rooted at IOException** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-615](https://issues.apache.org/jira/browse/YARN-615) | *Major* | **ContainerLaunchContext.containerTokens should simply be called tokens** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-610](https://issues.apache.org/jira/browse/YARN-610) | *Blocker* | **ClientToken (ClientToAMToken) should not be set in the environment** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-579](https://issues.apache.org/jira/browse/YARN-579) | *Major* | **Make ApplicationToken part of Container's token list to help RM-restart** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-571](https://issues.apache.org/jira/browse/YARN-571) | *Major* | **User should not be part of ContainerLaunchContext** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-561](https://issues.apache.org/jira/browse/YARN-561) | *Major* | **Nodemanager should set some key information into the environment of every container that it launches.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-553](https://issues.apache.org/jira/browse/YARN-553) | *Minor* | **Have YarnClient generate a directly usable ApplicationSubmissionContext** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-536](https://issues.apache.org/jira/browse/YARN-536) | *Major* | **Remove ContainerStatus, ContainerState from Container api interface as they will not be called by the container object** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-530](https://issues.apache.org/jira/browse/YARN-530) | *Major* | **Define Service model strictly, implement AbstractService for robust subclassing, migrate yarn-common services** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-440](https://issues.apache.org/jira/browse/YARN-440) | *Major* | **Flatten RegisterNodeManagerResponse** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-439](https://issues.apache.org/jira/browse/YARN-439) | *Major* | **Flatten NodeHeartbeatResponse** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-396](https://issues.apache.org/jira/browse/YARN-396) | *Major* | **Rationalize AllocateResponse in RM scheduler API** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-387](https://issues.apache.org/jira/browse/YARN-387) | *Blocker* | **Fix inconsistent protocol naming** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md new file mode 100644 index 00000000000..245bdd46cf7 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/CHANGES.2.1.1-beta.md @@ -0,0 +1,239 @@ + + +# Apache Hadoop Changelog + +## Release 2.1.1-beta - 2013-09-16 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9944](https://issues.apache.org/jira/browse/HADOOP-9944) | RpcRequestHeaderProto defines callId as uint32 while ipc.Client.CONNECTION\_CONTEXT\_CALL\_ID is signed (-3) | Blocker | . | Arun C Murthy | Arun C Murthy | +| [YARN-1170](https://issues.apache.org/jira/browse/YARN-1170) | yarn proto definitions should specify package as 'hadoop.yarn' | Blocker | . | Arun C Murthy | Binglin Chang | +| [YARN-707](https://issues.apache.org/jira/browse/YARN-707) | Add user info in the YARN ClientToken | Blocker | . | Bikas Saha | Jason Lowe | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9789](https://issues.apache.org/jira/browse/HADOOP-9789) | Support server advertised kerberos principals | Critical | ipc, security | Daryn Sharp | Daryn Sharp | +| [HDFS-5118](https://issues.apache.org/jira/browse/HDFS-5118) | Provide testing support for DFSClient to drop RPC responses | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-5076](https://issues.apache.org/jira/browse/HDFS-5076) | Add MXBean methods to query NN's transaction information and JournalNode's journal status | Minor | . | Jing Zhao | Jing Zhao | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9962](https://issues.apache.org/jira/browse/HADOOP-9962) | in order to avoid dependency divergence within Hadoop itself lets enable DependencyConvergence | Major | build | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-9945](https://issues.apache.org/jira/browse/HADOOP-9945) | HAServiceState should have a state for stopped services | Minor | ha | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9918](https://issues.apache.org/jira/browse/HADOOP-9918) | Add addIfService() to CompositeService | Minor | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9886](https://issues.apache.org/jira/browse/HADOOP-9886) | Turn warning message in RetryInvocationHandler to debug | Minor | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-9879](https://issues.apache.org/jira/browse/HADOOP-9879) | Move the version info of zookeeper dependencies to hadoop-project/pom | Minor | build | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9833](https://issues.apache.org/jira/browse/HADOOP-9833) | move slf4j to version 1.7.5 | Minor | build | Steve Loughran | Kousuke Saruta | +| [HADOOP-9831](https://issues.apache.org/jira/browse/HADOOP-9831) | Make checknative shell command accessible on Windows. | Minor | bin | Chris Nauroth | Chris Nauroth | +| [HADOOP-9821](https://issues.apache.org/jira/browse/HADOOP-9821) | ClientId should have getMsb/getLsb methods | Minor | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-9803](https://issues.apache.org/jira/browse/HADOOP-9803) | Add generic type parameter to RetryInvocationHandler | Minor | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9802](https://issues.apache.org/jira/browse/HADOOP-9802) | Support Snappy codec on Windows. | Major | io | Chris Nauroth | Chris Nauroth | +| [HADOOP-9686](https://issues.apache.org/jira/browse/HADOOP-9686) | Easy access to final parameters in Configuration | Major | conf | Jason Lowe | Jason Lowe | +| [HADOOP-9672](https://issues.apache.org/jira/browse/HADOOP-9672) | Upgrade Avro dependency to 1.7.4 | Major | . | Sandy Ryza | Sandy Ryza | +| [HADOOP-9669](https://issues.apache.org/jira/browse/HADOOP-9669) | Reduce the number of byte array creations and copies in XDR data manipulation | Major | nfs | Tsz Wo Nicholas Sze | Haohui Mai | +| [HADOOP-9446](https://issues.apache.org/jira/browse/HADOOP-9446) | Support Kerberos HTTP SPNEGO authentication for non-SUN JDK | Major | security | Yu Gao | Yu Gao | +| [HADOOP-8814](https://issues.apache.org/jira/browse/HADOOP-8814) | Inefficient comparison with the empty string. Use isEmpty() instead | Minor | conf, fs, fs/s3, ha, io, metrics, performance, record, security, util | Brandon Li | Brandon Li | +| [HDFS-5128](https://issues.apache.org/jira/browse/HDFS-5128) | Allow multiple net interfaces to be used with HA namenode RPC server | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5061](https://issues.apache.org/jira/browse/HDFS-5061) | Make FSNameSystem#auditLoggers an unmodifiable list | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-5045](https://issues.apache.org/jira/browse/HDFS-5045) | Add more unit tests for retry cache to cover all AtMostOnce methods | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-4926](https://issues.apache.org/jira/browse/HDFS-4926) | namenode webserver's page has a tooltip that is inconsistent with the datanode HTML link | Trivial | namenode | Joseph Lorenzini | Vivek Ganesan | +| [HDFS-4905](https://issues.apache.org/jira/browse/HDFS-4905) | Add appendToFile command to "hdfs dfs" | Minor | tools | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4513](https://issues.apache.org/jira/browse/HDFS-4513) | Clarify WebHDFS REST API that all JSON respsonses may contain additional properties | Minor | documentation, webhdfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-3245](https://issues.apache.org/jira/browse/HDFS-3245) | Add metrics and web UI for cluster version summary | Major | namenode | Todd Lipcon | Ravi Prakash | +| [MAPREDUCE-5478](https://issues.apache.org/jira/browse/MAPREDUCE-5478) | TeraInputFormat unnecessarily defines its own FileSplit subclass | Minor | examples | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5379](https://issues.apache.org/jira/browse/MAPREDUCE-5379) | Include token tracking ids in jobconf | Major | job submission, security | Sandy Ryza | Karthik Kambatla | +| [MAPREDUCE-5367](https://issues.apache.org/jira/browse/MAPREDUCE-5367) | Local jobs all use same local working directory | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-1981](https://issues.apache.org/jira/browse/MAPREDUCE-1981) | Improve getSplits performance by using listLocatedStatus | Major | job submission | Hairong Kuang | Hairong Kuang | +| [YARN-1137](https://issues.apache.org/jira/browse/YARN-1137) | Add support whitelist for system users to Yarn container-executor.c | Major | nodemanager | Alejandro Abdelnur | Roman Shaposhnik | +| [YARN-1117](https://issues.apache.org/jira/browse/YARN-1117) | Improve help message for $ yarn applications and $yarn node | Major | client | Tassapol Athiapinya | Xuan Gong | +| [YARN-1081](https://issues.apache.org/jira/browse/YARN-1081) | Minor improvement to output header for $ yarn node -list | Minor | client | Tassapol Athiapinya | Akira AJISAKA | +| [YARN-1080](https://issues.apache.org/jira/browse/YARN-1080) | Improve help message for $ yarn logs | Major | client | Tassapol Athiapinya | Xuan Gong | +| [YARN-1074](https://issues.apache.org/jira/browse/YARN-1074) | Clean up YARN CLI app list to show only running apps. | Major | client | Tassapol Athiapinya | Xuan Gong | +| [YARN-910](https://issues.apache.org/jira/browse/YARN-910) | Allow auxiliary services to listen for container starts and completions | Major | nodemanager | Sandy Ryza | Alejandro Abdelnur | +| [YARN-758](https://issues.apache.org/jira/browse/YARN-758) | Augment MockNM to use multiple cores | Minor | . | Bikas Saha | Karthik Kambatla | +| [YARN-696](https://issues.apache.org/jira/browse/YARN-696) | Enable multiple states to to be specified in Resource Manager apps REST call | Major | resourcemanager | Trevor Lorimer | Trevor Lorimer | +| [YARN-589](https://issues.apache.org/jira/browse/YARN-589) | Expose a REST API for monitoring the fair scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9977](https://issues.apache.org/jira/browse/HADOOP-9977) | Hadoop services won't start with different keypass and keystorepass when https is enabled | Major | security | Yesha Vora | Chris Nauroth | +| [HADOOP-9961](https://issues.apache.org/jira/browse/HADOOP-9961) | versions of a few transitive dependencies diverged between hadoop subprojects | Minor | build | Roman Shaposhnik | Roman Shaposhnik | +| [HADOOP-9960](https://issues.apache.org/jira/browse/HADOOP-9960) | Upgrade Jersey version to 1.9 | Blocker | . | Brock Noland | Karthik Kambatla | +| [HADOOP-9958](https://issues.apache.org/jira/browse/HADOOP-9958) | Add old constructor back to DelegationTokenInformation to unbreak downstream builds | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-9932](https://issues.apache.org/jira/browse/HADOOP-9932) | Improper synchronization in RetryCache | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-9924](https://issues.apache.org/jira/browse/HADOOP-9924) | FileUtil.createJarWithClassPath() does not generate relative classpath correctly | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-9916](https://issues.apache.org/jira/browse/HADOOP-9916) | Race condition in ipc.Client causes TestIPC timeout | Minor | . | Binglin Chang | Binglin Chang | +| [HADOOP-9910](https://issues.apache.org/jira/browse/HADOOP-9910) | proxy server start and stop documentation wrong | Minor | . | AndrĂ© Kelpe | | +| [HADOOP-9906](https://issues.apache.org/jira/browse/HADOOP-9906) | Move HAZKUtil to o.a.h.util.ZKUtil and make inner-classes public | Minor | ha | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9899](https://issues.apache.org/jira/browse/HADOOP-9899) | Remove the debug message added by HADOOP-8855 | Minor | security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-9880](https://issues.apache.org/jira/browse/HADOOP-9880) | SASL changes from HADOOP-9421 breaks Secure HA NN | Blocker | . | Kihwal Lee | Daryn Sharp | +| [HADOOP-9868](https://issues.apache.org/jira/browse/HADOOP-9868) | Server must not advertise kerberos realm | Blocker | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9858](https://issues.apache.org/jira/browse/HADOOP-9858) | Remove unused private RawLocalFileSystem#execCommand method from branch-2. | Trivial | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9857](https://issues.apache.org/jira/browse/HADOOP-9857) | Tests block and sometimes timeout on Windows due to invalid entropy source. | Major | build, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-9806](https://issues.apache.org/jira/browse/HADOOP-9806) | PortmapInterface should check if the procedure is out-of-range | Major | nfs | Brandon Li | Brandon Li | +| [HADOOP-9801](https://issues.apache.org/jira/browse/HADOOP-9801) | Configuration#writeXml uses platform defaulting encoding, which may mishandle multi-byte characters. | Major | conf | Chris Nauroth | Chris Nauroth | +| [HADOOP-9774](https://issues.apache.org/jira/browse/HADOOP-9774) | RawLocalFileSystem.listStatus() return absolute paths when input path is relative on Windows | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-9768](https://issues.apache.org/jira/browse/HADOOP-9768) | chown and chgrp reject users and groups with spaces on platforms where spaces are otherwise acceptable | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-9757](https://issues.apache.org/jira/browse/HADOOP-9757) | Har metadata cache can grow without limit | Major | fs | Jason Lowe | Cristina L. Abad | +| [HADOOP-9557](https://issues.apache.org/jira/browse/HADOOP-9557) | hadoop-client excludes commons-httpclient | Major | build | Lohit Vijayarenu | Lohit Vijayarenu | +| [HADOOP-9435](https://issues.apache.org/jira/browse/HADOOP-9435) | Support building the JNI code against the IBM JVM | Major | build | Tian Hong Wang | Tian Hong Wang | +| [HADOOP-9381](https://issues.apache.org/jira/browse/HADOOP-9381) | Document dfs cp -f option | Trivial | . | Keegan Witt | Keegan Witt | +| [HADOOP-9315](https://issues.apache.org/jira/browse/HADOOP-9315) | Port HADOOP-9249 hadoop-maven-plugins Clover fix to branch-2 to fix build failures | Major | build | Dennis Y | Chris Nauroth | +| [HDFS-5231](https://issues.apache.org/jira/browse/HDFS-5231) | Fix broken links in the document of HDFS Federation | Minor | . | Haohui Mai | Haohui Mai | +| [HDFS-5219](https://issues.apache.org/jira/browse/HDFS-5219) | Add configuration keys for retry policy in WebHDFSFileSystem | Major | webhdfs | Haohui Mai | Haohui Mai | +| [HDFS-5192](https://issues.apache.org/jira/browse/HDFS-5192) | NameNode may fail to start when dfs.client.test.drop.namenode.response.number is set | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-5159](https://issues.apache.org/jira/browse/HDFS-5159) | Secondary NameNode fails to checkpoint if error occurs downloading edits on first checkpoint | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5150](https://issues.apache.org/jira/browse/HDFS-5150) | Allow per NN SPN for internal SPNEGO. | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5140](https://issues.apache.org/jira/browse/HDFS-5140) | Too many safemode monitor threads being created in the standby namenode causing it to fail with out of memory error | Blocker | ha | Arpit Gupta | Jing Zhao | +| [HDFS-5132](https://issues.apache.org/jira/browse/HDFS-5132) | Deadlock in NameNode between SafeModeMonitor#run and DatanodeManager#handleHeartbeat | Blocker | namenode | Arpit Gupta | Kihwal Lee | +| [HDFS-5124](https://issues.apache.org/jira/browse/HDFS-5124) | DelegationTokenSecretManager#retrievePassword can cause deadlock in NameNode | Blocker | namenode | Deepesh Khandelwal | Daryn Sharp | +| [HDFS-5111](https://issues.apache.org/jira/browse/HDFS-5111) | Remove duplicated error message for snapshot commands when processing invalid arguments | Minor | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-5106](https://issues.apache.org/jira/browse/HDFS-5106) | TestDatanodeBlockScanner fails on Windows due to incorrect path format | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5105](https://issues.apache.org/jira/browse/HDFS-5105) | TestFsck fails on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [HDFS-5103](https://issues.apache.org/jira/browse/HDFS-5103) | TestDirectoryScanner fails on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5102](https://issues.apache.org/jira/browse/HDFS-5102) | Snapshot names should not be allowed to contain slash characters | Major | snapshots | Aaron T. Myers | Jing Zhao | +| [HDFS-5100](https://issues.apache.org/jira/browse/HDFS-5100) | TestNamenodeRetryCache fails on Windows due to incorrect cleanup | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5099](https://issues.apache.org/jira/browse/HDFS-5099) | Namenode#copyEditLogSegmentsToSharedDir should close EditLogInputStreams upon finishing | Major | namenode | Chuan Liu | Chuan Liu | +| [HDFS-5091](https://issues.apache.org/jira/browse/HDFS-5091) | Support for spnego keytab separate from the JournalNode keytab for secure HA | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-5080](https://issues.apache.org/jira/browse/HDFS-5080) | BootstrapStandby not working with QJM when the existing NN is active | Major | ha, qjm | Jing Zhao | Jing Zhao | +| [HDFS-5077](https://issues.apache.org/jira/browse/HDFS-5077) | NPE in FSNamesystem.commitBlockSynchronization() | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-5055](https://issues.apache.org/jira/browse/HDFS-5055) | nn fails to download checkpointed image from snn in some setups | Blocker | namenode | Allen Wittenauer | Vinayakumar B | +| [HDFS-5047](https://issues.apache.org/jira/browse/HDFS-5047) | Supress logging of full stack trace of quota and lease exceptions | Major | namenode | Kihwal Lee | Robert Parker | +| [HDFS-5043](https://issues.apache.org/jira/browse/HDFS-5043) | For HdfsFileStatus, set default value of childrenNum to -1 instead of 0 to avoid confusing applications | Major | . | Brandon Li | Brandon Li | +| [HDFS-5028](https://issues.apache.org/jira/browse/HDFS-5028) | LeaseRenewer throw java.util.ConcurrentModificationException when timeout | Major | . | zhaoyunjiong | zhaoyunjiong | +| [HDFS-4993](https://issues.apache.org/jira/browse/HDFS-4993) | fsck can fail if a file is renamed or deleted | Major | . | Kihwal Lee | Robert Parker | +| [HDFS-4898](https://issues.apache.org/jira/browse/HDFS-4898) | BlockPlacementPolicyWithNodeGroup.chooseRemoteRack() fails to properly fallback to local rack | Minor | namenode | Eric Sirianni | Tsz Wo Nicholas Sze | +| [HDFS-4680](https://issues.apache.org/jira/browse/HDFS-4680) | Audit logging of delegation tokens for MR tracing | Major | namenode, security | Andrew Wang | Andrew Wang | +| [HDFS-4632](https://issues.apache.org/jira/browse/HDFS-4632) | globStatus using backslash for escaping does not work on Windows | Major | test | Chris Nauroth | Chuan Liu | +| [HDFS-4594](https://issues.apache.org/jira/browse/HDFS-4594) | WebHDFS open sets Content-Length header to what is specified by length parameter rather than how much data is actually returned. | Minor | webhdfs | Arpit Gupta | Chris Nauroth | +| [HDFS-2994](https://issues.apache.org/jira/browse/HDFS-2994) | If lease soft limit is recovered successfully the append can fail | Major | . | Todd Lipcon | Tao Luo | +| [MAPREDUCE-5497](https://issues.apache.org/jira/browse/MAPREDUCE-5497) | '5s sleep' in MRAppMaster.shutDownJob is only needed before stopping ClientService | Major | . | Jian He | Jian He | +| [MAPREDUCE-5493](https://issues.apache.org/jira/browse/MAPREDUCE-5493) | In-memory map outputs can be leaked after shuffle completes | Blocker | mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5483](https://issues.apache.org/jira/browse/MAPREDUCE-5483) | revert MAPREDUCE-5357 | Major | distcp | Alejandro Abdelnur | Robert Kanter | +| [MAPREDUCE-5475](https://issues.apache.org/jira/browse/MAPREDUCE-5475) | MRClientService does not verify ACLs properly | Blocker | mr-am, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5470](https://issues.apache.org/jira/browse/MAPREDUCE-5470) | LocalJobRunner does not work on Windows. | Major | . | Chris Nauroth | Sandy Ryza | +| [MAPREDUCE-5468](https://issues.apache.org/jira/browse/MAPREDUCE-5468) | AM recovery does not work for map only jobs | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5454](https://issues.apache.org/jira/browse/MAPREDUCE-5454) | TestDFSIO fails intermittently on JDK7 | Major | test | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5446](https://issues.apache.org/jira/browse/MAPREDUCE-5446) | TestJobHistoryEvents and TestJobHistoryParsing have race conditions | Major | mrv2, test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5440](https://issues.apache.org/jira/browse/MAPREDUCE-5440) | TestCopyCommitter Fails on JDK7 | Major | mrv2 | Robert Parker | Robert Parker | +| [MAPREDUCE-5428](https://issues.apache.org/jira/browse/MAPREDUCE-5428) | HistoryFileManager doesn't stop threads when service is stopped | Major | jobhistoryserver, mrv2 | Jason Lowe | Karthik Kambatla | +| [MAPREDUCE-5425](https://issues.apache.org/jira/browse/MAPREDUCE-5425) | Junit in TestJobHistoryServer failing in jdk 7 | Major | jobhistoryserver | Ashwin Shankar | Robert Parker | +| [MAPREDUCE-5414](https://issues.apache.org/jira/browse/MAPREDUCE-5414) | TestTaskAttempt fails jdk7 with NullPointerException | Major | test | Nemon Lou | Nemon Lou | +| [MAPREDUCE-5385](https://issues.apache.org/jira/browse/MAPREDUCE-5385) | JobContext cache files api are broken | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5358](https://issues.apache.org/jira/browse/MAPREDUCE-5358) | MRAppMaster throws invalid transitions for JobImpl | Major | mr-am | Devaraj K | Devaraj K | +| [MAPREDUCE-5317](https://issues.apache.org/jira/browse/MAPREDUCE-5317) | Stale files left behind for failed jobs | Major | mrv2 | Ravi Prakash | Ravi Prakash | +| [MAPREDUCE-5251](https://issues.apache.org/jira/browse/MAPREDUCE-5251) | Reducer should not implicate map attempt if it has insufficient space to fetch map output | Major | mrv2 | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-5164](https://issues.apache.org/jira/browse/MAPREDUCE-5164) | command "mapred job" and "mapred queue" omit HADOOP\_CLIENT\_OPTS | Major | . | Nemon Lou | Nemon Lou | +| [MAPREDUCE-5020](https://issues.apache.org/jira/browse/MAPREDUCE-5020) | Compile failure with JDK8 | Major | client | Trevor Robinson | Trevor Robinson | +| [MAPREDUCE-5001](https://issues.apache.org/jira/browse/MAPREDUCE-5001) | LocalJobRunner has race condition resulting in job failures | Major | . | Brock Noland | Sandy Ryza | +| [MAPREDUCE-3193](https://issues.apache.org/jira/browse/MAPREDUCE-3193) | FileInputFormat doesn't read files recursively in the input path dir | Major | mrv1, mrv2 | Ramgopal N | Devaraj K | +| [YARN-1194](https://issues.apache.org/jira/browse/YARN-1194) | TestContainerLogsPage fails with native builds | Minor | nodemanager | Roman Shaposhnik | Roman Shaposhnik | +| [YARN-1189](https://issues.apache.org/jira/browse/YARN-1189) | NMTokenSecretManagerInNM is not being told when applications have finished | Blocker | . | Jason Lowe | Omkar Vinit Joshi | +| [YARN-1176](https://issues.apache.org/jira/browse/YARN-1176) | RM web services ClusterMetricsInfo total nodes doesn't include unhealthy nodes | Critical | resourcemanager | Thomas Graves | Jonathan Eagles | +| [YARN-1152](https://issues.apache.org/jira/browse/YARN-1152) | Invalid key to HMAC computation error when getting application report for completed app attempt | Blocker | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-1144](https://issues.apache.org/jira/browse/YARN-1144) | Unmanaged AMs registering a tracking URI should not be proxy-fied | Critical | resourcemanager | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1124](https://issues.apache.org/jira/browse/YARN-1124) | By default yarn application -list should display all the applications in a state other than FINISHED / FAILED | Blocker | . | Omkar Vinit Joshi | Xuan Gong | +| [YARN-1120](https://issues.apache.org/jira/browse/YARN-1120) | Make ApplicationConstants.Environment.USER definition OS neutral | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-1107](https://issues.apache.org/jira/browse/YARN-1107) | Job submitted with Delegation token in secured environment causes RM to fail during RM restart | Blocker | resourcemanager | Arpit Gupta | Omkar Vinit Joshi | +| [YARN-1101](https://issues.apache.org/jira/browse/YARN-1101) | Active nodes can be decremented below 0 | Major | resourcemanager | Robert Parker | Robert Parker | +| [YARN-1094](https://issues.apache.org/jira/browse/YARN-1094) | RM restart throws Null pointer Exception in Secure Env | Blocker | . | Yesha Vora | Vinod Kumar Vavilapalli | +| [YARN-1093](https://issues.apache.org/jira/browse/YARN-1093) | Corrections to Fair Scheduler documentation | Major | documentation | Wing Yew Poon | | +| [YARN-1083](https://issues.apache.org/jira/browse/YARN-1083) | ResourceManager should fail when yarn.nm.liveness-monitor.expiry-interval-ms is set less than heartbeat interval | Major | resourcemanager | Yesha Vora | Zhijie Shen | +| [YARN-1082](https://issues.apache.org/jira/browse/YARN-1082) | Secure RM with recovery enabled and rm state store on hdfs fails with gss exception | Blocker | resourcemanager | Arpit Gupta | Vinod Kumar Vavilapalli | +| [YARN-1078](https://issues.apache.org/jira/browse/YARN-1078) | TestNodeManagerResync, TestNodeManagerShutdown, and TestNodeStatusUpdater fail on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-1077](https://issues.apache.org/jira/browse/YARN-1077) | TestContainerLaunch fails on Windows | Minor | . | Chuan Liu | Chuan Liu | +| [YARN-1025](https://issues.apache.org/jira/browse/YARN-1025) | ResourceManager and NodeManager do not load native libraries on Windows. | Major | nodemanager, resourcemanager | Chris Nauroth | Chris Nauroth | +| [YARN-1008](https://issues.apache.org/jira/browse/YARN-1008) | MiniYARNCluster with multiple nodemanagers, all nodes have same key for allocations | Major | nodemanager | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1006](https://issues.apache.org/jira/browse/YARN-1006) | Nodes list web page on the RM web UI is broken | Major | . | Jian He | Xuan Gong | +| [YARN-994](https://issues.apache.org/jira/browse/YARN-994) | HeartBeat thread in AMRMClientAsync does not handle runtime exception correctly | Major | . | Xuan Gong | Xuan Gong | +| [YARN-981](https://issues.apache.org/jira/browse/YARN-981) | YARN/MR2/Job-history /logs link does not have correct content | Major | . | Xuan Gong | Jian He | +| [YARN-966](https://issues.apache.org/jira/browse/YARN-966) | The thread of ContainerLaunch#call will fail without any signal if getLocalizedResources() is called when the container is not at LOCALIZED | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-957](https://issues.apache.org/jira/browse/YARN-957) | Capacity Scheduler tries to reserve the memory more than what node manager reports. | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-948](https://issues.apache.org/jira/browse/YARN-948) | RM should validate the release container list before actually releasing them | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-942](https://issues.apache.org/jira/browse/YARN-942) | In Fair Scheduler documentation, inconsistency on which properties have prefix | Major | scheduler | Sandy Ryza | Akira AJISAKA | +| [YARN-903](https://issues.apache.org/jira/browse/YARN-903) | DistributedShell throwing Errors in logs after successfull completion | Major | applications/distributed-shell | Abhishek Kapoor | Omkar Vinit Joshi | +| [YARN-881](https://issues.apache.org/jira/browse/YARN-881) | Priority#compareTo method seems to be wrong. | Major | . | Jian He | Jian He | +| [YARN-643](https://issues.apache.org/jira/browse/YARN-643) | WHY appToken is removed both in BaseFinalTransition and AMUnregisteredTransition AND clientToken is removed in FinalTransition and not BaseFinalTransition | Major | . | Jian He | Xuan Gong | +| [YARN-602](https://issues.apache.org/jira/browse/YARN-602) | NodeManager should mandatorily set some Environment variables into every containers that it launches | Major | . | Xuan Gong | Kenji Kikushima | +| [YARN-337](https://issues.apache.org/jira/browse/YARN-337) | RM handles killed application tracking URL poorly | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-107](https://issues.apache.org/jira/browse/YARN-107) | ClientRMService.forceKillApplication() should handle the non-RUNNING applications properly | Major | resourcemanager | Devaraj K | Xuan Gong | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-5249](https://issues.apache.org/jira/browse/HDFS-5249) | Fix dumper thread which may die silently | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5234](https://issues.apache.org/jira/browse/HDFS-5234) | Move RpcFrameDecoder out of the public API | Minor | nfs | Haohui Mai | Haohui Mai | +| [HDFS-5212](https://issues.apache.org/jira/browse/HDFS-5212) | Refactor RpcMessage and NFS3Response to support different types of authentication information | Major | nfs | Jing Zhao | Jing Zhao | +| [HDFS-5199](https://issues.apache.org/jira/browse/HDFS-5199) | Add more debug trace for NFS READ and WRITE | Trivial | nfs | Brandon Li | Brandon Li | +| [HDFS-5136](https://issues.apache.org/jira/browse/HDFS-5136) | MNT EXPORT should give the full group list which can mount the exports | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5110](https://issues.apache.org/jira/browse/HDFS-5110) | Change FSDataOutputStream to HdfsDataOutputStream for opened streams to fix type cast error | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5107](https://issues.apache.org/jira/browse/HDFS-5107) | Fix array copy error in Readdir and Readdirplus responses | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5104](https://issues.apache.org/jira/browse/HDFS-5104) | Support dotdot name in NFS LOOKUP operation | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5085](https://issues.apache.org/jira/browse/HDFS-5085) | Refactor o.a.h.nfs to support different types of authentications | Major | nfs | Brandon Li | Jing Zhao | +| [HDFS-5078](https://issues.apache.org/jira/browse/HDFS-5078) | Support file append in NFSv3 gateway to enable data streaming to HDFS | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5071](https://issues.apache.org/jira/browse/HDFS-5071) | Change hdfs-nfs parent project to hadoop-project | Major | nfs | Kihwal Lee | Brandon Li | +| [HDFS-5069](https://issues.apache.org/jira/browse/HDFS-5069) | Include hadoop-nfs and hadoop-hdfs-nfs into hadoop dist for NFS deployment | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5067](https://issues.apache.org/jira/browse/HDFS-5067) | Support symlink operations | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-4971](https://issues.apache.org/jira/browse/HDFS-4971) | Move IO operations out of locking in OpenFileCtx | Major | nfs | Jing Zhao | Jing Zhao | +| [HDFS-4962](https://issues.apache.org/jira/browse/HDFS-4962) | Use enum for nfs constants | Minor | nfs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4947](https://issues.apache.org/jira/browse/HDFS-4947) | Add NFS server export table to control export by hostname or IP range | Major | nfs | Brandon Li | Jing Zhao | +| [HDFS-4763](https://issues.apache.org/jira/browse/HDFS-4763) | Add script changes/utility for starting NFS gateway | Major | nfs | Brandon Li | Brandon Li | +| [MAPREDUCE-5476](https://issues.apache.org/jira/browse/MAPREDUCE-5476) | Job can fail when RM restarts after staging dir is cleaned but before MR successfully unregister with RM | Blocker | . | Jian He | Jian He | +| [MAPREDUCE-5466](https://issues.apache.org/jira/browse/MAPREDUCE-5466) | Historyserver does not refresh the result of restarted jobs after RM restart | Blocker | . | Yesha Vora | Jian He | +| [MAPREDUCE-5462](https://issues.apache.org/jira/browse/MAPREDUCE-5462) | In map-side sort, swap entire meta entries instead of indexes for better cache performance | Major | performance, task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5441](https://issues.apache.org/jira/browse/MAPREDUCE-5441) | JobClient exit whenever RM issue Reboot command to 1st attempt App Master. | Major | applicationmaster, client | Rohith Sharma K S | Jian He | +| [YARN-1184](https://issues.apache.org/jira/browse/YARN-1184) | ClassCastException is thrown during preemption When a huge job is submitted to a queue B whose resources is used by a job in queueA | Major | capacityscheduler, resourcemanager | J.Andreina | Chris Douglas | +| [YARN-1116](https://issues.apache.org/jira/browse/YARN-1116) | Populate AMRMTokens back to AMRMTokenSecretManager after RM restarts | Major | resourcemanager | Jian He | Jian He | +| [YARN-1085](https://issues.apache.org/jira/browse/YARN-1085) | Yarn and MRv2 should do HTTP client authentication in kerberos setup. | Blocker | nodemanager, resourcemanager | Jaimin D Jetly | Omkar Vinit Joshi | +| [YARN-1049](https://issues.apache.org/jira/browse/YARN-1049) | ContainerExistStatus should define a status for preempted containers | Blocker | api | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-906](https://issues.apache.org/jira/browse/YARN-906) | Cancelling ContainerLaunch#call at KILLING causes that the container cannot be completed | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-771](https://issues.apache.org/jira/browse/YARN-771) | AMRMClient support for resource blacklisting | Major | . | Bikas Saha | Junping Du | +| [YARN-573](https://issues.apache.org/jira/browse/YARN-573) | Shared data structures in Public Localizer and Private Localizer are not Thread safe. | Critical | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-540](https://issues.apache.org/jira/browse/YARN-540) | Race condition causing RM to potentially relaunch already unregistered AMs on RM restart | Major | resourcemanager | Jian He | Jian He | +| [YARN-502](https://issues.apache.org/jira/browse/YARN-502) | RM crash with NPE on NODE\_REMOVED event with FairScheduler | Major | . | Lohit Vijayarenu | Mayank Bansal | +| [YARN-292](https://issues.apache.org/jira/browse/YARN-292) | ResourceManager throws ArrayIndexOutOfBoundsException while handling CONTAINER\_ALLOCATED for application attempt | Major | resourcemanager | Devaraj K | Zhijie Shen | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-1034](https://issues.apache.org/jira/browse/YARN-1034) | Remove "experimental" in the Fair Scheduler documentation | Trivial | documentation, scheduler | Sandy Ryza | Karthik Kambatla | +| [YARN-1001](https://issues.apache.org/jira/browse/YARN-1001) | YARN should provide per application-type and state statistics | Blocker | api | Srimanth Gunturi | Zhijie Shen | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md new file mode 100644 index 00000000000..db2f17012c0 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.1.1-beta/RELEASENOTES.2.1.1-beta.md @@ -0,0 +1,52 @@ + + +# Apache Hadoop 2.1.1-beta Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-9944](https://issues.apache.org/jira/browse/HADOOP-9944) | *Blocker* | **RpcRequestHeaderProto defines callId as uint32 while ipc.Client.CONNECTION\_CONTEXT\_CALL\_ID is signed (-3)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-5118](https://issues.apache.org/jira/browse/HDFS-5118) | *Major* | **Provide testing support for DFSClient to drop RPC responses** + +Used for testing when NameNode HA is enabled. Users can use a new configuration property "dfs.client.test.drop.namenode.response.number" to specify the number of responses that DFSClient will drop in each RPC call. This feature can help testing functionalities such as NameNode retry cache. + + +--- + +* [YARN-1170](https://issues.apache.org/jira/browse/YARN-1170) | *Blocker* | **yarn proto definitions should specify package as 'hadoop.yarn'** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-707](https://issues.apache.org/jira/browse/YARN-707) | *Blocker* | **Add user info in the YARN ClientToken** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md new file mode 100644 index 00000000000..e70a79a3239 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/CHANGES.2.2.0.md @@ -0,0 +1,152 @@ + + +# Apache Hadoop Changelog + +## Release 2.2.0 - 2013-10-15 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10020](https://issues.apache.org/jira/browse/HADOOP-10020) | disable symlinks temporarily | Blocker | fs | Colin Patrick McCabe | Sanjay Radia | +| [YARN-1229](https://issues.apache.org/jira/browse/YARN-1229) | Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce\_shuffle. | Blocker | nodemanager | Tassapol Athiapinya | Xuan Gong | +| [YARN-1228](https://issues.apache.org/jira/browse/YARN-1228) | Clean up Fair Scheduler configuration loading | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9758](https://issues.apache.org/jira/browse/HADOOP-9758) | Provide configuration option for FileSystem/FileContext symlink resolution | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-8315](https://issues.apache.org/jira/browse/HADOOP-8315) | Support SASL-authenticated ZooKeeper in ActiveStandbyElector | Major | auto-failover, ha | Todd Lipcon | Todd Lipcon | +| [HDFS-5308](https://issues.apache.org/jira/browse/HDFS-5308) | Replace HttpConfig#getSchemePrefix with implicit schemes in HDFS JSP | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5256](https://issues.apache.org/jira/browse/HDFS-5256) | Use guava LoadingCache to implement DFSClientCache | Major | nfs | Haohui Mai | Haohui Mai | +| [HDFS-5139](https://issues.apache.org/jira/browse/HDFS-5139) | Remove redundant -R option from setrep | Major | tools | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4817](https://issues.apache.org/jira/browse/HDFS-4817) | make HDFS advisory caching configurable on a per-file basis | Minor | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [YARN-1246](https://issues.apache.org/jira/browse/YARN-1246) | Log application status in the rm log when app is done running | Minor | . | Arpit Gupta | Arpit Gupta | +| [YARN-1213](https://issues.apache.org/jira/browse/YARN-1213) | Restore config to ban submitting to undeclared pools in the Fair Scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10012](https://issues.apache.org/jira/browse/HADOOP-10012) | Secure Oozie jobs fail with delegation token renewal exception in Namenode HA setup | Blocker | ha | Arpit Gupta | Suresh Srinivas | +| [HADOOP-10003](https://issues.apache.org/jira/browse/HADOOP-10003) | HarFileSystem.listLocatedStatus() fails | Major | fs | Jason Dere | | +| [HADOOP-9976](https://issues.apache.org/jira/browse/HADOOP-9976) | Different versions of avro and avro-maven-plugin | Major | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9776](https://issues.apache.org/jira/browse/HADOOP-9776) | HarFileSystem.listStatus() returns invalid authority if port number is empty | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-9761](https://issues.apache.org/jira/browse/HADOOP-9761) | ViewFileSystem#rename fails when using DistributedFileSystem | Blocker | viewfs | Andrew Wang | Andrew Wang | +| [HDFS-5300](https://issues.apache.org/jira/browse/HDFS-5300) | FSNameSystem#deleteSnapshot() should not check owner in case of permissions disabled | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5299](https://issues.apache.org/jira/browse/HDFS-5299) | DFS client hangs in updatePipeline RPC when failover happened | Blocker | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5289](https://issues.apache.org/jira/browse/HDFS-5289) | Race condition in TestRetryCacheWithHA#testCreateSymlink causes spurious test failure | Major | test | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5279](https://issues.apache.org/jira/browse/HDFS-5279) | Guard against NullPointerException in NameNode JSP pages before initialization of FSNamesystem. | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-5268](https://issues.apache.org/jira/browse/HDFS-5268) | NFS write commit verifier is not set in a few places | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5265](https://issues.apache.org/jira/browse/HDFS-5265) | Namenode fails to start when dfs.https.port is unspecified | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5258](https://issues.apache.org/jira/browse/HDFS-5258) | Skip tests in TestHDFSCLI that are not applicable on Windows. | Minor | test | Chris Nauroth | Chuan Liu | +| [HDFS-5255](https://issues.apache.org/jira/browse/HDFS-5255) | Distcp job fails with hsftp when https is enabled in insecure cluster | Major | . | Yesha Vora | Arpit Agarwal | +| [HDFS-5251](https://issues.apache.org/jira/browse/HDFS-5251) | Race between the initialization of NameNode and the http server | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5228](https://issues.apache.org/jira/browse/HDFS-5228) | The RemoteIterator returned by DistributedFileSystem.listFiles(..) may throw NPE | Blocker | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5031](https://issues.apache.org/jira/browse/HDFS-5031) | BlockScanner scans the block multiple times and on restart scans everything | Blocker | datanode | Vinayakumar B | Vinayakumar B | +| [MAPREDUCE-5554](https://issues.apache.org/jira/browse/MAPREDUCE-5554) | hdfs-site.xml included in hadoop-mapreduce-client-jobclient tests jar is breaking tests for downstream components | Minor | test | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5545](https://issues.apache.org/jira/browse/MAPREDUCE-5545) | org.apache.hadoop.mapred.TestTaskAttemptListenerImpl.testCommitWindow times out | Major | . | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5544](https://issues.apache.org/jira/browse/MAPREDUCE-5544) | JobClient#getJob loads job conf twice | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5536](https://issues.apache.org/jira/browse/MAPREDUCE-5536) | mapreduce.jobhistory.webapp.https.address property is not respected | Blocker | . | Yesha Vora | Omkar Vinit Joshi | +| [MAPREDUCE-5533](https://issues.apache.org/jira/browse/MAPREDUCE-5533) | Speculative execution does not function for reduce | Major | applicationmaster | Tassapol Athiapinya | Xuan Gong | +| [MAPREDUCE-5523](https://issues.apache.org/jira/browse/MAPREDUCE-5523) | Need to add https port related property in Job history server | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5515](https://issues.apache.org/jira/browse/MAPREDUCE-5515) | Application Manager UI does not appear with Https enabled | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [MAPREDUCE-5513](https://issues.apache.org/jira/browse/MAPREDUCE-5513) | ConcurrentModificationException in JobControl | Major | . | Jason Lowe | Robert Parker | +| [MAPREDUCE-5503](https://issues.apache.org/jira/browse/MAPREDUCE-5503) | TestMRJobClient.testJobClient is failing | Blocker | mrv2 | Jason Lowe | Jian He | +| [MAPREDUCE-5489](https://issues.apache.org/jira/browse/MAPREDUCE-5489) | MR jobs hangs as it does not use the node-blacklisting feature in RM requests | Critical | . | Yesha Vora | Zhijie Shen | +| [MAPREDUCE-5488](https://issues.apache.org/jira/browse/MAPREDUCE-5488) | Job recovery fails after killing all the running containers for the app | Major | . | Arpit Gupta | Jian He | +| [MAPREDUCE-5459](https://issues.apache.org/jira/browse/MAPREDUCE-5459) | Update the doc of running MRv1 examples jar on YARN | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5442](https://issues.apache.org/jira/browse/MAPREDUCE-5442) | $HADOOP\_MAPRED\_HOME/$HADOOP\_CONF\_DIR setting not working on Windows | Major | client | Yingda Chen | Yingda Chen | +| [MAPREDUCE-5170](https://issues.apache.org/jira/browse/MAPREDUCE-5170) | incorrect exception message if min node size \> min rack size | Trivial | mrv2 | Sangjin Lee | Sangjin Lee | +| [YARN-1278](https://issues.apache.org/jira/browse/YARN-1278) | New AM does not start after rm restart | Blocker | . | Yesha Vora | Hitesh Shah | +| [YARN-1274](https://issues.apache.org/jira/browse/YARN-1274) | LCE fails to run containers that don't have resources to localize | Blocker | nodemanager | Alejandro Abdelnur | Siddharth Seth | +| [YARN-1273](https://issues.apache.org/jira/browse/YARN-1273) | Distributed shell does not account for start container failures reported asynchronously. | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-1271](https://issues.apache.org/jira/browse/YARN-1271) | "Text file busy" errors launching containers again | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1262](https://issues.apache.org/jira/browse/YARN-1262) | TestApplicationCleanup relies on all containers assigned in a single heartbeat | Major | . | Sandy Ryza | Karthik Kambatla | +| [YARN-1251](https://issues.apache.org/jira/browse/YARN-1251) | TestDistributedShell#TestDSShell failed with timeout | Major | applications/distributed-shell | Junping Du | Xuan Gong | +| [YARN-1247](https://issues.apache.org/jira/browse/YARN-1247) | test-container-executor has gotten out of sync with the changes to container-executor | Major | nodemanager | Roman Shaposhnik | Roman Shaposhnik | +| [YARN-1236](https://issues.apache.org/jira/browse/YARN-1236) | FairScheduler setting queue name in RMApp is not working | Major | resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1221](https://issues.apache.org/jira/browse/YARN-1221) | With Fair Scheduler, reserved MB reported in RM web UI increases indefinitely | Major | resourcemanager, scheduler | Sandy Ryza | Siqi Li | +| [YARN-1219](https://issues.apache.org/jira/browse/YARN-1219) | FSDownload changes file suffix making FileUtil.unTar() throw exception | Major | nodemanager | shanyu zhao | shanyu zhao | +| [YARN-1215](https://issues.apache.org/jira/browse/YARN-1215) | Yarn URL should include userinfo | Major | api | Chuan Liu | Chuan Liu | +| [YARN-1167](https://issues.apache.org/jira/browse/YARN-1167) | Submitted distributed shell application shows appMasterHost = empty | Major | applications/distributed-shell | Tassapol Athiapinya | Xuan Gong | +| [YARN-1157](https://issues.apache.org/jira/browse/YARN-1157) | ResourceManager UI has invalid tracking URL link for distributed shell application | Major | resourcemanager | Tassapol Athiapinya | Xuan Gong | +| [YARN-1149](https://issues.apache.org/jira/browse/YARN-1149) | NM throws InvalidStateTransitonException: Invalid event: APPLICATION\_LOG\_HANDLING\_FINISHED at RUNNING | Major | . | Ramya Sunil | Xuan Gong | +| [YARN-1141](https://issues.apache.org/jira/browse/YARN-1141) | Updating resource requests should be decoupled with updating blacklist | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1128](https://issues.apache.org/jira/browse/YARN-1128) | FifoPolicy.computeShares throws NPE on empty list of Schedulables | Major | scheduler | Sandy Ryza | Karthik Kambatla | +| [YARN-1090](https://issues.apache.org/jira/browse/YARN-1090) | Job does not get into Pending State | Major | . | Yesha Vora | Jian He | +| [YARN-1032](https://issues.apache.org/jira/browse/YARN-1032) | NPE in RackResolve | Critical | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [YARN-890](https://issues.apache.org/jira/browse/YARN-890) | The roundup for memory values on resource manager UI is misleading | Major | resourcemanager | Trupti Dhavle | Xuan Gong | +| [YARN-876](https://issues.apache.org/jira/browse/YARN-876) | Node resource is added twice when node comes back from unhealthy to healthy | Major | resourcemanager | Peng Zhang | Peng Zhang | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9948](https://issues.apache.org/jira/browse/HADOOP-9948) | Add a config value to CLITestHelper to skip tests on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5186](https://issues.apache.org/jira/browse/HDFS-5186) | TestFileJournalManager fails on Windows due to file handle leaks | Minor | namenode, test | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5525](https://issues.apache.org/jira/browse/MAPREDUCE-5525) | Increase timeout of TestDFSIO.testAppend and TestMRJobsWithHistoryService.testJobHistoryData | Minor | mrv2, test | Chuan Liu | Chuan Liu | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10017](https://issues.apache.org/jira/browse/HADOOP-10017) | Fix NPE in DFSClient#getDelegationToken when doing Distcp from a secured cluster to an insecured cluster | Major | . | Jing Zhao | Haohui Mai | +| [HDFS-5306](https://issues.apache.org/jira/browse/HDFS-5306) | Datanode https port is not available at the namenode | Major | datanode, namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-5259](https://issues.apache.org/jira/browse/HDFS-5259) | Support client which combines appended data with old data before sends it to NFS server | Major | nfs | Yesha Vora | Brandon Li | +| [HDFS-5246](https://issues.apache.org/jira/browse/HDFS-5246) | Make Hadoop nfs server port and mount daemon port configurable | Major | nfs | Jinghui Wang | Jinghui Wang | +| [HDFS-5230](https://issues.apache.org/jira/browse/HDFS-5230) | Introduce RpcInfo to decouple XDR classes from the RPC API | Major | nfs | Haohui Mai | Haohui Mai | +| [MAPREDUCE-5562](https://issues.apache.org/jira/browse/MAPREDUCE-5562) | MR AM should exit when unregister() throws exception | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5551](https://issues.apache.org/jira/browse/MAPREDUCE-5551) | Binary Incompatibility of O.A.H.U.mapred.SequenceFileAsBinaryOutputFormat.WritableValueBytes | Blocker | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5538](https://issues.apache.org/jira/browse/MAPREDUCE-5538) | MRAppMaster#shutDownJob shouldn't send job end notification before checking isLastRetry | Blocker | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5531](https://issues.apache.org/jira/browse/MAPREDUCE-5531) | Binary and source incompatibility in mapreduce.TaskID and mapreduce.TaskAttemptID between branch-1 and branch-2 | Blocker | mrv1, mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5530](https://issues.apache.org/jira/browse/MAPREDUCE-5530) | Binary and source incompatibility in mapred.lib.CombineFileInputFormat between branch-1 and branch-2 | Blocker | mrv1, mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5529](https://issues.apache.org/jira/browse/MAPREDUCE-5529) | Binary incompatibilities in mapred.lib.TotalOrderPartitioner between branch-1 and branch-2 | Blocker | mrv1, mrv2 | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5505](https://issues.apache.org/jira/browse/MAPREDUCE-5505) | Clients should be notified job finished only after job successfully unregistered | Critical | . | Jian He | Zhijie Shen | +| [YARN-1277](https://issues.apache.org/jira/browse/YARN-1277) | Add http policy support for YARN daemons | Major | . | Suresh Srinivas | Omkar Vinit Joshi | +| [YARN-1260](https://issues.apache.org/jira/browse/YARN-1260) | RM\_HOME link breaks when webapp.https.address related properties are not specified | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1256](https://issues.apache.org/jira/browse/YARN-1256) | NM silently ignores non-existent service in StartContainerRequest | Critical | . | Bikas Saha | Xuan Gong | +| [YARN-1254](https://issues.apache.org/jira/browse/YARN-1254) | NM is polluting container's credentials | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-1214](https://issues.apache.org/jira/browse/YARN-1214) | Register ClientToken MasterKey in SecretManager after it is saved | Critical | resourcemanager | Jian He | Jian He | +| [YARN-1204](https://issues.apache.org/jira/browse/YARN-1204) | Need to add https port related property in Yarn | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1203](https://issues.apache.org/jira/browse/YARN-1203) | Application Manager UI does not appear with Https enabled | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1131](https://issues.apache.org/jira/browse/YARN-1131) | $yarn logs command should return an appropriate error message if YARN application is still running | Minor | client | Tassapol Athiapinya | Siddharth Seth | +| [YARN-1070](https://issues.apache.org/jira/browse/YARN-1070) | ContainerImpl State Machine: Invalid event: CONTAINER\_KILLED\_ON\_REQUEST at CONTAINER\_CLEANEDUP\_AFTER\_KILL | Major | nodemanager | Hitesh Shah | Zhijie Shen | +| [YARN-899](https://issues.apache.org/jira/browse/YARN-899) | Get queue administration ACLs working | Major | scheduler | Sandy Ryza | Xuan Gong | +| [YARN-621](https://issues.apache.org/jira/browse/YARN-621) | RM triggers web auth failure before first job | Critical | resourcemanager | Allen Wittenauer | Omkar Vinit Joshi | +| [YARN-49](https://issues.apache.org/jira/browse/YARN-49) | Improve distributed shell application to work on a secure cluster | Major | applications/distributed-shell | Hitesh Shah | Vinod Kumar Vavilapalli | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md new file mode 100644 index 00000000000..301e0c6324b --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.0/RELEASENOTES.2.2.0.md @@ -0,0 +1,45 @@ + + +# Apache Hadoop 2.2.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10020](https://issues.apache.org/jira/browse/HADOOP-10020) | *Blocker* | **disable symlinks temporarily** + +During review of symbolic links, many issues were found related to the impact on semantics of existing APIs such FileSystem#listStatus, FileSystem#globStatus etc. There were also many issues brought up about symbolic links, and the impact on security and functionality of HDFS. All these issues will be addressed in the upcoming release 2.3. Until then the feature is temporarily disabled. + + +--- + +* [YARN-1229](https://issues.apache.org/jira/browse/YARN-1229) | *Blocker* | **Define constraints on Auxiliary Service names. Change ShuffleHandler service name from mapreduce.shuffle to mapreduce\_shuffle.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-1228](https://issues.apache.org/jira/browse/YARN-1228) | *Major* | **Clean up Fair Scheduler configuration loading** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/CHANGES.2.2.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/CHANGES.2.2.1.md new file mode 100644 index 00000000000..1f7eb2c927a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/CHANGES.2.2.1.md @@ -0,0 +1,66 @@ + + +# Apache Hadoop Changelog + +## Release 2.2.1 - Unreleased + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-5587](https://issues.apache.org/jira/browse/HDFS-5587) | add debug information when NFS fails to start with duplicate user or group names | Minor | nfs | Brandon Li | Brandon Li | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10110](https://issues.apache.org/jira/browse/HADOOP-10110) | hadoop-auth has a build break due to missing dependency | Blocker | build | Chuan Liu | Chuan Liu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/RELEASENOTES.2.2.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/RELEASENOTES.2.2.1.md new file mode 100644 index 00000000000..b4d1b73c925 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.2.1/RELEASENOTES.2.2.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 2.2.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/CHANGES.2.3.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/CHANGES.2.3.0.md new file mode 100644 index 00000000000..bf50e3224fc --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/CHANGES.2.3.0.md @@ -0,0 +1,664 @@ + + +# Apache Hadoop Changelog + +## Release 2.3.0 - 2014-02-20 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-4997](https://issues.apache.org/jira/browse/HDFS-4997) | libhdfs doesn't return correct error codes in most cases | Major | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10047](https://issues.apache.org/jira/browse/HADOOP-10047) | Add a directbuffer Decompressor API to hadoop | Major | io | Gopal V | Gopal V | +| [HADOOP-9848](https://issues.apache.org/jira/browse/HADOOP-9848) | Create a MiniKDC for use with security testing | Major | security, test | Wei Yan | Wei Yan | +| [HADOOP-9618](https://issues.apache.org/jira/browse/HADOOP-9618) | Add thread which detects JVM pauses | Major | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-9432](https://issues.apache.org/jira/browse/HADOOP-9432) | Add support for markdown .md files in site documentation | Minor | build, documentation | Steve Loughran | | +| [HADOOP-8545](https://issues.apache.org/jira/browse/HADOOP-8545) | Filesystem Implementation for OpenStack Swift | Major | fs | Tim Miller | Dmitry Mezhensky | +| [HDFS-5703](https://issues.apache.org/jira/browse/HDFS-5703) | Add support for HTTPS and swebhdfs to HttpFS | Major | webhdfs | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-5260](https://issues.apache.org/jira/browse/HDFS-5260) | Merge zero-copy memory-mapped HDFS client reads to trunk and branch-2. | Major | hdfs-client, libhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-4949](https://issues.apache.org/jira/browse/HDFS-4949) | Centralized cache management in HDFS | Major | datanode, namenode | Andrew Wang | Andrew Wang | +| [HDFS-2832](https://issues.apache.org/jira/browse/HDFS-2832) | Enable support for heterogeneous storages in HDFS - DN as a collection of storages | Major | datanode, namenode | Suresh Srinivas | Arpit Agarwal | +| [MAPREDUCE-5332](https://issues.apache.org/jira/browse/MAPREDUCE-5332) | Support token-preserving restart of history server | Major | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5266](https://issues.apache.org/jira/browse/MAPREDUCE-5266) | Ability to refresh retention settings on history server | Major | jobhistoryserver | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-5265](https://issues.apache.org/jira/browse/MAPREDUCE-5265) | History server admin service to refresh user and superuser group mappings | Major | jobhistoryserver | Jason Lowe | Ashwin Shankar | +| [MAPREDUCE-1176](https://issues.apache.org/jira/browse/MAPREDUCE-1176) | FixedLengthInputFormat and FixedLengthRecordReader | Major | . | BitsOfInfo | Mariappan Asokan | +| [YARN-1392](https://issues.apache.org/jira/browse/YARN-1392) | Allow sophisticated app-to-queue placement policies in the Fair Scheduler | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1253](https://issues.apache.org/jira/browse/YARN-1253) | Changes to LinuxContainerExecutor to run containers as a single dedicated user in non-secure mode | Blocker | nodemanager | Alejandro Abdelnur | Roman Shaposhnik | +| [YARN-1021](https://issues.apache.org/jira/browse/YARN-1021) | Yarn Scheduler Load Simulator | Major | scheduler | Wei Yan | Wei Yan | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10274](https://issues.apache.org/jira/browse/HADOOP-10274) | Lower the logging level from ERROR to WARN for UGI.doAs method | Minor | security | takeshi.miao | takeshi.miao | +| [HADOOP-10248](https://issues.apache.org/jira/browse/HADOOP-10248) | Property name should be included in the exception where property value is null | Major | . | Ted Yu | Akira AJISAKA | +| [HADOOP-10228](https://issues.apache.org/jira/browse/HADOOP-10228) | FsPermission#fromShort() should cache FsAction.values() | Minor | fs | Haohui Mai | Haohui Mai | +| [HADOOP-10208](https://issues.apache.org/jira/browse/HADOOP-10208) | Remove duplicate initialization in StringUtils.getStringCollection | Trivial | . | Benoy Antony | Benoy Antony | +| [HADOOP-10198](https://issues.apache.org/jira/browse/HADOOP-10198) | DomainSocket: add support for socketpair | Minor | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-10173](https://issues.apache.org/jira/browse/HADOOP-10173) | Remove UGI from DIGEST-MD5 SASL server creation | Critical | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-10172](https://issues.apache.org/jira/browse/HADOOP-10172) | Cache SASL server factories | Critical | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-10169](https://issues.apache.org/jira/browse/HADOOP-10169) | remove the unnecessary synchronized in JvmMetrics class | Minor | metrics | Liang Xie | Liang Xie | +| [HADOOP-10167](https://issues.apache.org/jira/browse/HADOOP-10167) | Mark hadoop-common source as UTF-8 in Maven pom files / refactoring | Major | build | Mikhail Antonov | Mikhail Antonov | +| [HADOOP-10164](https://issues.apache.org/jira/browse/HADOOP-10164) | Allow UGI to login with a known Subject | Major | . | Robert Joseph Evans | Robert Joseph Evans | +| [HADOOP-10143](https://issues.apache.org/jira/browse/HADOOP-10143) | replace WritableFactories's hashmap with ConcurrentHashMap | Major | io | Liang Xie | Liang Xie | +| [HADOOP-10132](https://issues.apache.org/jira/browse/HADOOP-10132) | RPC#stopProxy() should log the class of proxy when IllegalArgumentException is encountered | Minor | . | Ted Yu | Ted Yu | +| [HADOOP-10111](https://issues.apache.org/jira/browse/HADOOP-10111) | Allow DU to be initialized with an initial value | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10095](https://issues.apache.org/jira/browse/HADOOP-10095) | Performance improvement in CodecPool | Minor | io | Nicolas Liochon | Nicolas Liochon | +| [HADOOP-10086](https://issues.apache.org/jira/browse/HADOOP-10086) | User document for authentication in secure cluster | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-10079](https://issues.apache.org/jira/browse/HADOOP-10079) | log a warning message if group resolution takes too long. | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-10067](https://issues.apache.org/jira/browse/HADOOP-10067) | Missing POM dependency on jsr305 | Minor | . | Robert Rati | Robert Rati | +| [HADOOP-10064](https://issues.apache.org/jira/browse/HADOOP-10064) | Upgrade to maven antrun plugin version 1.7 | Major | build | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-10046](https://issues.apache.org/jira/browse/HADOOP-10046) | Print a log message when SSL is enabled | Trivial | . | David S. Wang | David S. Wang | +| [HADOOP-10005](https://issues.apache.org/jira/browse/HADOOP-10005) | No need to check INFO severity level is enabled or not | Trivial | . | Jackie Chang | Jackie Chang | +| [HADOOP-9998](https://issues.apache.org/jira/browse/HADOOP-9998) | Provide methods to clear only part of the DNSToSwitchMapping | Major | net | Junping Du | Junping Du | +| [HADOOP-9915](https://issues.apache.org/jira/browse/HADOOP-9915) | o.a.h.fs.Stat support on Macosx | Trivial | . | Binglin Chang | Binglin Chang | +| [HADOOP-9909](https://issues.apache.org/jira/browse/HADOOP-9909) | org.apache.hadoop.fs.Stat should permit other LANG | Major | fs | Shinichi Yamashita | | +| [HADOOP-9897](https://issues.apache.org/jira/browse/HADOOP-9897) | Add method to get path start position without drive specifier in o.a.h.fs.Path | Trivial | fs | Binglin Chang | Binglin Chang | +| [HADOOP-9860](https://issues.apache.org/jira/browse/HADOOP-9860) | Remove class HackedKeytab and HackedKeytabEncoder from hadoop-minikdc once jira DIRSERVER-1882 solved | Major | . | Wei Yan | Wei Yan | +| [HADOOP-9784](https://issues.apache.org/jira/browse/HADOOP-9784) | Add a builder for HttpServer | Major | . | Junping Du | Junping Du | +| [HADOOP-9693](https://issues.apache.org/jira/browse/HADOOP-9693) | Shell should add a probe for OSX | Trivial | . | Steve Loughran | | +| [HADOOP-9652](https://issues.apache.org/jira/browse/HADOOP-9652) | Allow RawLocalFs#getFileLinkStatus to fill in the link owner and mode if requested | Major | . | Colin Patrick McCabe | Andrew Wang | +| [HADOOP-9623](https://issues.apache.org/jira/browse/HADOOP-9623) | Update jets3t dependency to 0.9.0 | Major | fs/s3 | Timothy St. Clair | Amandeep Khurana | +| [HADOOP-9611](https://issues.apache.org/jira/browse/HADOOP-9611) | mvn-rpmbuild against google-guice \> 3.0 yields missing cglib dependency | Major | build | Timothy St. Clair | Timothy St. Clair | +| [HADOOP-9594](https://issues.apache.org/jira/browse/HADOOP-9594) | Update apache commons math dependency | Major | build | Timothy St. Clair | Timothy St. Clair | +| [HADOOP-9494](https://issues.apache.org/jira/browse/HADOOP-9494) | Excluded auto-generated and examples code from clover reports | Major | . | Dennis Y | Andrey Klochkov | +| [HADOOP-9487](https://issues.apache.org/jira/browse/HADOOP-9487) | Deprecation warnings in Configuration should go to their own log or otherwise be suppressible | Major | conf | Steve Loughran | | +| [HADOOP-9470](https://issues.apache.org/jira/browse/HADOOP-9470) | eliminate duplicate FQN tests in different Hadoop modules | Major | test | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9319](https://issues.apache.org/jira/browse/HADOOP-9319) | Update bundled lz4 source to latest version | Major | . | Arpit Agarwal | Binglin Chang | +| [HADOOP-9241](https://issues.apache.org/jira/browse/HADOOP-9241) | DU refresh interval is not configurable | Trivial | . | Harsh J | Harsh J | +| [HADOOP-8704](https://issues.apache.org/jira/browse/HADOOP-8704) | add request logging to jetty/httpserver | Major | . | Thomas Graves | Jonathan Eagles | +| [HDFS-5841](https://issues.apache.org/jira/browse/HDFS-5841) | Update HDFS caching documentation with new changes | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-5833](https://issues.apache.org/jira/browse/HDFS-5833) | SecondaryNameNode have an incorrect java doc | Trivial | namenode | Bangtao Zhou | | +| [HDFS-5825](https://issues.apache.org/jira/browse/HDFS-5825) | Use FileUtils.copyFile() to implement DFSTestUtils.copyFile() | Minor | . | Haohui Mai | Haohui Mai | +| [HDFS-5788](https://issues.apache.org/jira/browse/HDFS-5788) | listLocatedStatus response can be very large | Major | namenode | Nathan Roberts | Nathan Roberts | +| [HDFS-5748](https://issues.apache.org/jira/browse/HDFS-5748) | Too much information shown in the dfs health page. | Major | . | Kihwal Lee | Haohui Mai | +| [HDFS-5721](https://issues.apache.org/jira/browse/HDFS-5721) | sharedEditsImage in Namenode#initializeSharedEdits() should be closed before method returns | Minor | . | Ted Yu | Ted Yu | +| [HDFS-5695](https://issues.apache.org/jira/browse/HDFS-5695) | Clean up TestOfflineEditsViewer and OfflineEditsViewerHelper | Major | test | Haohui Mai | Haohui Mai | +| [HDFS-5677](https://issues.apache.org/jira/browse/HDFS-5677) | Need error checking for HA cluster configuration | Minor | datanode, ha | Vincent Sheffer | Vincent Sheffer | +| [HDFS-5676](https://issues.apache.org/jira/browse/HDFS-5676) | fix inconsistent synchronization of CachingStrategy | Minor | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5674](https://issues.apache.org/jira/browse/HDFS-5674) | Editlog code cleanup | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5665](https://issues.apache.org/jira/browse/HDFS-5665) | Remove the unnecessary writeLock while initializing CacheManager in FsNameSystem Ctor | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-5663](https://issues.apache.org/jira/browse/HDFS-5663) | make the retry time and interval value configurable in openInfo() | Major | hdfs-client | Liang Xie | Liang Xie | +| [HDFS-5662](https://issues.apache.org/jira/browse/HDFS-5662) | Can't decommission a DataNode due to file's replication factor larger than the rest of the cluster size | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-5652](https://issues.apache.org/jira/browse/HDFS-5652) | refactoring/uniforming invalid block token exception handling in DFSInputStream | Minor | hdfs-client | Liang Xie | Liang Xie | +| [HDFS-5637](https://issues.apache.org/jira/browse/HDFS-5637) | try to refeatchToken while local read InvalidToken occurred | Major | hdfs-client, security | Liang Xie | Liang Xie | +| [HDFS-5633](https://issues.apache.org/jira/browse/HDFS-5633) | Improve OfflineImageViewer to use less memory | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-5587](https://issues.apache.org/jira/browse/HDFS-5587) | add debug information when NFS fails to start with duplicate user or group names | Minor | nfs | Brandon Li | Brandon Li | +| [HDFS-5577](https://issues.apache.org/jira/browse/HDFS-5577) | NFS user guide update | Trivial | documentation | Brandon Li | Brandon Li | +| [HDFS-5568](https://issues.apache.org/jira/browse/HDFS-5568) | Support inclusion of snapshot paths in Namenode fsck | Major | snapshots | Vinayakumar B | Vinayakumar B | +| [HDFS-5563](https://issues.apache.org/jira/browse/HDFS-5563) | NFS gateway should commit the buffered data when read request comes after write to the same file | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5561](https://issues.apache.org/jira/browse/HDFS-5561) | FSNameSystem#getNameJournalStatus() in JMX should return plain text instead of HTML | Minor | namenode | Fengdong Yu | Haohui Mai | +| [HDFS-5548](https://issues.apache.org/jira/browse/HDFS-5548) | Use ConcurrentHashMap in portmap | Major | nfs | Haohui Mai | Haohui Mai | +| [HDFS-5532](https://issues.apache.org/jira/browse/HDFS-5532) | Enable the webhdfs by default to support new HDFS web UI | Major | webhdfs | Vinayakumar B | Vinayakumar B | +| [HDFS-5495](https://issues.apache.org/jira/browse/HDFS-5495) | Remove further JUnit3 usages from HDFS | Major | . | Andrew Wang | Jarek Jarcec Cecho | +| [HDFS-5467](https://issues.apache.org/jira/browse/HDFS-5467) | Remove tab characters in hdfs-default.xml | Trivial | . | Andrew Wang | Shinichi Yamashita | +| [HDFS-5399](https://issues.apache.org/jira/browse/HDFS-5399) | Revisit SafeModeException and corresponding retry policies | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-5371](https://issues.apache.org/jira/browse/HDFS-5371) | Let client retry the same NN when "dfs.client.test.drop.namenode.response.number" is enabled | Minor | ha, test | Jing Zhao | Jing Zhao | +| [HDFS-5360](https://issues.apache.org/jira/browse/HDFS-5360) | Improvement of usage message of renameSnapshot and deleteSnapshot | Minor | snapshots | Shinichi Yamashita | Shinichi Yamashita | +| [HDFS-5350](https://issues.apache.org/jira/browse/HDFS-5350) | Name Node should report fsimage transfer time as a metric | Minor | namenode | Rob Weltman | Jimmy Xiang | +| [HDFS-5344](https://issues.apache.org/jira/browse/HDFS-5344) | Make LsSnapshottableDir as Tool interface implementation | Minor | snapshots, tools | sathish | sathish | +| [HDFS-5338](https://issues.apache.org/jira/browse/HDFS-5338) | Add a conf to disable hostname check in DN registration | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5333](https://issues.apache.org/jira/browse/HDFS-5333) | Improvement of current HDFS Web UI | Major | . | Jing Zhao | Haohui Mai | +| [HDFS-5331](https://issues.apache.org/jira/browse/HDFS-5331) | make SnapshotDiff.java to a o.a.h.util.Tool interface implementation | Major | snapshots | Vinayakumar B | Vinayakumar B | +| [HDFS-5323](https://issues.apache.org/jira/browse/HDFS-5323) | Remove some deadcode in BlockManager | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5267](https://issues.apache.org/jira/browse/HDFS-5267) | Remove volatile from LightWeightHashSet | Minor | . | Junping Du | Junping Du | +| [HDFS-5220](https://issues.apache.org/jira/browse/HDFS-5220) | Expose group resolution time as metric | Major | namenode | Rob Weltman | Jimmy Xiang | +| [HDFS-5207](https://issues.apache.org/jira/browse/HDFS-5207) | In BlockPlacementPolicy, update 2 parameters of chooseTarget() | Major | namenode | Junping Du | Junping Du | +| [HDFS-5188](https://issues.apache.org/jira/browse/HDFS-5188) | Clean up BlockPlacementPolicy and its implementations | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5144](https://issues.apache.org/jira/browse/HDFS-5144) | Document time unit to NameNodeMetrics.java | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5068](https://issues.apache.org/jira/browse/HDFS-5068) | Convert NNThroughputBenchmark to a Tool to allow generic options. | Major | benchmarks | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-5037](https://issues.apache.org/jira/browse/HDFS-5037) | Active NN should trigger its own edit log rolls | Critical | ha, namenode | Todd Lipcon | Andrew Wang | +| [HDFS-5034](https://issues.apache.org/jira/browse/HDFS-5034) | Remove debug prints from getFileLinkInfo | Trivial | namenode | Andrew Wang | Andrew Wang | +| [HDFS-5004](https://issues.apache.org/jira/browse/HDFS-5004) | Add additional JMX bean for NameNode status data | Major | namenode | Trevor Lorimer | Trevor Lorimer | +| [HDFS-4983](https://issues.apache.org/jira/browse/HDFS-4983) | Numeric usernames do not work with WebHDFS FS | Major | webhdfs | Harsh J | Yongjun Zhang | +| [HDFS-4879](https://issues.apache.org/jira/browse/HDFS-4879) | Add "blocked ArrayList" collection to avoid CMS full GCs | Major | namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-4860](https://issues.apache.org/jira/browse/HDFS-4860) | Add additional attributes to JMX beans | Major | namenode | Trevor Lorimer | Trevor Lorimer | +| [HDFS-4278](https://issues.apache.org/jira/browse/HDFS-4278) | Log an ERROR when DFS\_BLOCK\_ACCESS\_TOKEN\_ENABLE config is disabled but security is turned on. | Major | datanode, namenode | Harsh J | Kousuke Saruta | +| [HDFS-2933](https://issues.apache.org/jira/browse/HDFS-2933) | Improve DataNode Web UI Index Page | Major | datanode | Philip Zeyliger | Vivek Ganesan | +| [MAPREDUCE-5692](https://issues.apache.org/jira/browse/MAPREDUCE-5692) | Add explicit diagnostics when a task attempt is killed due to speculative execution | Major | mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5672](https://issues.apache.org/jira/browse/MAPREDUCE-5672) | Provide optional RollingFileAppender for container log4j (syslog) | Major | mr-am, mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5640](https://issues.apache.org/jira/browse/MAPREDUCE-5640) | Rename TestLineRecordReader in jobclient module | Trivial | test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5601](https://issues.apache.org/jira/browse/MAPREDUCE-5601) | ShuffleHandler fadvises file regions as DONTNEED even when fetch fails | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5596](https://issues.apache.org/jira/browse/MAPREDUCE-5596) | Allow configuring the number of threads used to serve shuffle connections | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5487](https://issues.apache.org/jira/browse/MAPREDUCE-5487) | In task processes, JobConf is unnecessarily loaded again in Limits | Major | performance, task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5484](https://issues.apache.org/jira/browse/MAPREDUCE-5484) | YarnChild unnecessarily loads job conf twice | Major | task | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5457](https://issues.apache.org/jira/browse/MAPREDUCE-5457) | Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-3310](https://issues.apache.org/jira/browse/MAPREDUCE-3310) | Custom grouping comparator cannot be set for Combiners | Major | client | Mathias Herberts | Alejandro Abdelnur | +| [MAPREDUCE-2734](https://issues.apache.org/jira/browse/MAPREDUCE-2734) | DistCp with FairScheduler assign all map tasks in one TT | Trivial | jobtracker | Bochun Bai | Bochun Bai | +| [MAPREDUCE-434](https://issues.apache.org/jira/browse/MAPREDUCE-434) | LocalJobRunner limited to single reducer | Minor | . | Yoram Arnon | Aaron Kimball | +| [YARN-1623](https://issues.apache.org/jira/browse/YARN-1623) | Include queue name in RegisterApplicationMasterResponse | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1616](https://issues.apache.org/jira/browse/YARN-1616) | RMFatalEventDispatcher should log the cause of the event | Trivial | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1567](https://issues.apache.org/jira/browse/YARN-1567) | In Fair Scheduler, allow empty queues to change between leaf and parent on allocation file reload | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1423](https://issues.apache.org/jira/browse/YARN-1423) | Support queue placement by secondary group in the Fair Scheduler | Major | scheduler | Sandy Ryza | Ted Malaska | +| [YARN-1403](https://issues.apache.org/jira/browse/YARN-1403) | Separate out configuration loading from QueueManager in the Fair Scheduler | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-1387](https://issues.apache.org/jira/browse/YARN-1387) | RMWebServices should use ClientRMService for filtering applications | Major | api | Karthik Kambatla | Karthik Kambatla | +| [YARN-1335](https://issues.apache.org/jira/browse/YARN-1335) | Move duplicate code from FSSchedulerApp and FiCaSchedulerApp into SchedulerApplication | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1333](https://issues.apache.org/jira/browse/YARN-1333) | Support blacklisting in the Fair Scheduler | Major | scheduler | Sandy Ryza | Tsuyoshi Ozawa | +| [YARN-1332](https://issues.apache.org/jira/browse/YARN-1332) | In TestAMRMClient, replace assertTrue with assertEquals where possible | Minor | . | Sandy Ryza | Sebastian Wong | +| [YARN-1303](https://issues.apache.org/jira/browse/YARN-1303) | Allow multiple commands separating with ";" in distributed-shell | Major | applications/distributed-shell | Tassapol Athiapinya | Xuan Gong | +| [YARN-1290](https://issues.apache.org/jira/browse/YARN-1290) | Let continuous scheduling achieve more balanced task assignment | Major | . | Wei Yan | Wei Yan | +| [YARN-1258](https://issues.apache.org/jira/browse/YARN-1258) | Allow configuring the Fair Scheduler root queue | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1199](https://issues.apache.org/jira/browse/YARN-1199) | Make NM/RM Versions Available | Major | . | Mit Desai | Mit Desai | +| [YARN-1109](https://issues.apache.org/jira/browse/YARN-1109) | Demote NodeManager "Sending out status for container" logs to debug | Major | nodemanager | Sandy Ryza | haosdent | +| [YARN-1010](https://issues.apache.org/jira/browse/YARN-1010) | FairScheduler: decouple container scheduling from nodemanager heartbeats | Critical | scheduler | Alejandro Abdelnur | Wei Yan | +| [YARN-985](https://issues.apache.org/jira/browse/YARN-985) | Nodemanager should log where a resource was localized | Major | nodemanager | Ravi Prakash | Ravi Prakash | +| [YARN-905](https://issues.apache.org/jira/browse/YARN-905) | Add state filters to nodes CLI | Major | . | Sandy Ryza | Wei Yan | +| [YARN-807](https://issues.apache.org/jira/browse/YARN-807) | When querying apps by queue, iterating over all apps is inefficient and limiting | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-786](https://issues.apache.org/jira/browse/YARN-786) | Expose application resource usage in RM REST API | Major | . | Sandy Ryza | Sandy Ryza | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10317](https://issues.apache.org/jira/browse/HADOOP-10317) | Rename branch-2.3 release version from 2.4.0-SNAPSHOT to 2.3.0-SNAPSHOT | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-10313](https://issues.apache.org/jira/browse/HADOOP-10313) | Script and jenkins job to produce Hadoop release artifacts | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10311](https://issues.apache.org/jira/browse/HADOOP-10311) | Cleanup vendor names from the code base | Blocker | . | Suresh Srinivas | Alejandro Abdelnur | +| [HADOOP-10310](https://issues.apache.org/jira/browse/HADOOP-10310) | SaslRpcServer should be initialized even when no secret manager present | Blocker | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-10305](https://issues.apache.org/jira/browse/HADOOP-10305) | Add "rpc.metrics.quantile.enable" and "rpc.metrics.percentiles.intervals" to core-default.xml | Major | metrics | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10292](https://issues.apache.org/jira/browse/HADOOP-10292) | Restore HttpServer from branch-2.2 in branch-2 | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-10291](https://issues.apache.org/jira/browse/HADOOP-10291) | TestSecurityUtil#testSocketAddrWithIP fails | Major | . | Mit Desai | Mit Desai | +| [HADOOP-10288](https://issues.apache.org/jira/browse/HADOOP-10288) | Explicit reference to Log4JLogger breaks non-log4j users | Major | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-10273](https://issues.apache.org/jira/browse/HADOOP-10273) | Fix 'mvn site' | Major | build | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-10255](https://issues.apache.org/jira/browse/HADOOP-10255) | Rename HttpServer to HttpServer2 to retain older HttpServer in branch-2 for compatibility | Blocker | . | Haohui Mai | Haohui Mai | +| [HADOOP-10252](https://issues.apache.org/jira/browse/HADOOP-10252) | HttpServer can't start if hostname is not specified | Major | . | Jimmy Xiang | Jimmy Xiang | +| [HADOOP-10250](https://issues.apache.org/jira/browse/HADOOP-10250) | VersionUtil returns wrong value when comparing two versions | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-10240](https://issues.apache.org/jira/browse/HADOOP-10240) | Windows build instructions incorrectly state requirement of protoc 2.4.1 instead of 2.5.0 | Trivial | documentation | Chris Nauroth | Chris Nauroth | +| [HADOOP-10236](https://issues.apache.org/jira/browse/HADOOP-10236) | Fix typo in o.a.h.ipc.Client#checkResponse | Trivial | . | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10235](https://issues.apache.org/jira/browse/HADOOP-10235) | Hadoop tarball has 2 versions of stax-api JARs | Major | build | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10234](https://issues.apache.org/jira/browse/HADOOP-10234) | "hadoop.cmd jar" does not propagate exit code. | Major | scripts | Chris Nauroth | Chris Nauroth | +| [HADOOP-10223](https://issues.apache.org/jira/browse/HADOOP-10223) | MiniKdc#main() should close the FileReader it creates | Minor | . | Ted Yu | Ted Yu | +| [HADOOP-10214](https://issues.apache.org/jira/browse/HADOOP-10214) | Fix multithreaded correctness warnings in ActiveStandbyElector | Major | ha | Liang Xie | Liang Xie | +| [HADOOP-10212](https://issues.apache.org/jira/browse/HADOOP-10212) | Incorrect compile command in Native Library document | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10203](https://issues.apache.org/jira/browse/HADOOP-10203) | Connection leak in Jets3tNativeFileSystemStore#retrieveMetadata | Major | fs/s3 | Andrei Savu | Andrei Savu | +| [HADOOP-10193](https://issues.apache.org/jira/browse/HADOOP-10193) | hadoop-auth's PseudoAuthenticationHandler can consume getInputStream | Minor | security | Gregory Chanan | Gregory Chanan | +| [HADOOP-10178](https://issues.apache.org/jira/browse/HADOOP-10178) | Configuration deprecation always emit "deprecated" warnings when a new key is used | Major | conf | shanyu zhao | shanyu zhao | +| [HADOOP-10175](https://issues.apache.org/jira/browse/HADOOP-10175) | Har files system authority should preserve userinfo | Major | fs | Chuan Liu | Chuan Liu | +| [HADOOP-10171](https://issues.apache.org/jira/browse/HADOOP-10171) | TestRPC fails intermittently on jkd7 | Major | . | Mit Desai | Mit Desai | +| [HADOOP-10168](https://issues.apache.org/jira/browse/HADOOP-10168) | fix javadoc of ReflectionUtils.copy | Major | . | Thejas M Nair | Thejas M Nair | +| [HADOOP-10162](https://issues.apache.org/jira/browse/HADOOP-10162) | Fix symlink-related test failures in TestFileContextResolveAfs and TestStat in branch-2 | Major | . | Mit Desai | Mit Desai | +| [HADOOP-10147](https://issues.apache.org/jira/browse/HADOOP-10147) | Upgrade to commons-logging 1.1.3 to avoid potential deadlock in MiniDFSCluster | Minor | build | Eric Sirianni | Steve Loughran | +| [HADOOP-10146](https://issues.apache.org/jira/browse/HADOOP-10146) | Workaround JDK7 Process fd close bug | Critical | util | Daryn Sharp | Daryn Sharp | +| [HADOOP-10142](https://issues.apache.org/jira/browse/HADOOP-10142) | Avoid groups lookup for unprivileged users such as "dr.who" | Major | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-10135](https://issues.apache.org/jira/browse/HADOOP-10135) | writes to swift fs over partition size leave temp files and empty output file | Major | fs | David Dobbins | David Dobbins | +| [HADOOP-10130](https://issues.apache.org/jira/browse/HADOOP-10130) | RawLocalFS::LocalFSFileInputStream.pread does not track FS::Statistics | Minor | . | Binglin Chang | Binglin Chang | +| [HADOOP-10129](https://issues.apache.org/jira/browse/HADOOP-10129) | Distcp may succeed when it fails | Critical | tools/distcp | Daryn Sharp | Daryn Sharp | +| [HADOOP-10127](https://issues.apache.org/jira/browse/HADOOP-10127) | Add ipc.client.connect.retry.interval to control the frequency of connection retries | Major | ipc | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-10126](https://issues.apache.org/jira/browse/HADOOP-10126) | LightWeightGSet log message is confusing : "2.0% max memory = 2.0 GB" | Minor | util | Vinayakumar B | Vinayakumar B | +| [HADOOP-10125](https://issues.apache.org/jira/browse/HADOOP-10125) | no need to process RPC request if the client connection has been dropped | Major | ipc | Ming Ma | Ming Ma | +| [HADOOP-10112](https://issues.apache.org/jira/browse/HADOOP-10112) | har file listing doesn't work with wild card | Major | tools | Brandon Li | Brandon Li | +| [HADOOP-10110](https://issues.apache.org/jira/browse/HADOOP-10110) | hadoop-auth has a build break due to missing dependency | Blocker | build | Chuan Liu | Chuan Liu | +| [HADOOP-10106](https://issues.apache.org/jira/browse/HADOOP-10106) | Incorrect thread name in RPC log messages | Minor | . | Ming Ma | Ming Ma | +| [HADOOP-10100](https://issues.apache.org/jira/browse/HADOOP-10100) | MiniKDC shouldn't use apacheds-all artifact | Major | . | Robert Kanter | Robert Kanter | +| [HADOOP-10094](https://issues.apache.org/jira/browse/HADOOP-10094) | NPE in GenericOptionsParser#preProcessForWindows() | Trivial | util | Enis Soztutar | Enis Soztutar | +| [HADOOP-10093](https://issues.apache.org/jira/browse/HADOOP-10093) | hadoop-env.cmd sets HADOOP\_CLIENT\_OPTS with a max heap size that is too small. | Major | conf | shanyu zhao | shanyu zhao | +| [HADOOP-10090](https://issues.apache.org/jira/browse/HADOOP-10090) | Jobtracker metrics not updated properly after execution of a mapreduce job | Major | metrics | Ivan Mitic | Ivan Mitic | +| [HADOOP-10088](https://issues.apache.org/jira/browse/HADOOP-10088) | copy-nativedistlibs.sh needs to quote snappy lib dir | Major | build | Raja Aluri | Raja Aluri | +| [HADOOP-10087](https://issues.apache.org/jira/browse/HADOOP-10087) | UserGroupInformation.getGroupNames() fails to return primary group first when JniBasedUnixGroupsMappingWithFallback is used | Major | security | Yu Gao | Colin Patrick McCabe | +| [HADOOP-10081](https://issues.apache.org/jira/browse/HADOOP-10081) | Client.setupIOStreams can leak socket resources on exception or error | Critical | ipc | Jason Lowe | Tsuyoshi Ozawa | +| [HADOOP-10078](https://issues.apache.org/jira/browse/HADOOP-10078) | KerberosAuthenticator always does SPNEGO | Minor | security | Robert Kanter | Robert Kanter | +| [HADOOP-10072](https://issues.apache.org/jira/browse/HADOOP-10072) | TestNfsExports#testMultiMatchers fails due to non-deterministic timing around cache expiry check. | Trivial | nfs, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-10058](https://issues.apache.org/jira/browse/HADOOP-10058) | TestMetricsSystemImpl#testInitFirstVerifyStopInvokedImmediately fails on trunk | Minor | metrics | Akira AJISAKA | Chen He | +| [HADOOP-10055](https://issues.apache.org/jira/browse/HADOOP-10055) | FileSystemShell.apt.vm doc has typo "numRepicas" | Trivial | documentation | Eli Collins | Akira AJISAKA | +| [HADOOP-10040](https://issues.apache.org/jira/browse/HADOOP-10040) | hadoop.cmd in UNIX format and would not run by default on Windows | Major | . | Yingda Chen | Chris Nauroth | +| [HADOOP-10039](https://issues.apache.org/jira/browse/HADOOP-10039) | Add Hive to the list of projects using AbstractDelegationTokenSecretManager | Major | security | Suresh Srinivas | Haohui Mai | +| [HADOOP-10031](https://issues.apache.org/jira/browse/HADOOP-10031) | FsShell -get/copyToLocal/moveFromLocal should support Windows local path | Major | fs | Chuan Liu | Chuan Liu | +| [HADOOP-10030](https://issues.apache.org/jira/browse/HADOOP-10030) | FsShell -put/copyFromLocal should support Windows local path | Major | . | Chuan Liu | Chuan Liu | +| [HADOOP-10029](https://issues.apache.org/jira/browse/HADOOP-10029) | Specifying har file to MR job fails in secure cluster | Major | fs | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-10028](https://issues.apache.org/jira/browse/HADOOP-10028) | Malformed ssl-server.xml.example | Minor | . | Jing Zhao | Haohui Mai | +| [HADOOP-10006](https://issues.apache.org/jira/browse/HADOOP-10006) | Compilation failure in trunk for o.a.h.fs.swift.util.JSONUtil | Blocker | fs, util | Junping Du | Junping Du | +| [HADOOP-9982](https://issues.apache.org/jira/browse/HADOOP-9982) | Fix dead links in hadoop site docs | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-9981](https://issues.apache.org/jira/browse/HADOOP-9981) | globStatus should minimize its listStatus and getFileStatus calls | Critical | . | Kihwal Lee | Colin Patrick McCabe | +| [HADOOP-9964](https://issues.apache.org/jira/browse/HADOOP-9964) | O.A.H.U.ReflectionUtils.printThreadInfo() is not thread-safe which cause TestHttpServer pending 10 minutes or longer. | Major | util | Junping Du | Junping Du | +| [HADOOP-9929](https://issues.apache.org/jira/browse/HADOOP-9929) | Insufficient permissions for a path reported as file not found | Major | fs | Jason Lowe | Colin Patrick McCabe | +| [HADOOP-9908](https://issues.apache.org/jira/browse/HADOOP-9908) | Fix NPE when versioninfo properties file is missing | Major | util | Todd Lipcon | Todd Lipcon | +| [HADOOP-9898](https://issues.apache.org/jira/browse/HADOOP-9898) | Set SO\_KEEPALIVE on all our sockets | Minor | ipc, net | Todd Lipcon | Todd Lipcon | +| [HADOOP-9889](https://issues.apache.org/jira/browse/HADOOP-9889) | Refresh the Krb5 configuration when creating a new kdc in Hadoop-MiniKDC | Major | . | Wei Yan | Wei Yan | +| [HADOOP-9887](https://issues.apache.org/jira/browse/HADOOP-9887) | globStatus does not correctly handle paths starting with a drive spec on Windows | Major | fs | Chris Nauroth | Chuan Liu | +| [HADOOP-9875](https://issues.apache.org/jira/browse/HADOOP-9875) | TestDoAsEffectiveUser can fail on JDK 7 | Minor | test | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-9871](https://issues.apache.org/jira/browse/HADOOP-9871) | Fix intermittent findbug warnings in DefaultMetricsSystem | Minor | . | Luke Lu | Junping Du | +| [HADOOP-9865](https://issues.apache.org/jira/browse/HADOOP-9865) | FileContext.globStatus() has a regression with respect to relative path | Major | . | Chuan Liu | Chuan Liu | +| [HADOOP-9847](https://issues.apache.org/jira/browse/HADOOP-9847) | TestGlobPath symlink tests fail to cleanup properly | Minor | . | Andrew Wang | Colin Patrick McCabe | +| [HADOOP-9830](https://issues.apache.org/jira/browse/HADOOP-9830) | Typo at http://hadoop.apache.org/docs/current/ | Trivial | documentation | Dmitry Lysnichenko | Kousuke Saruta | +| [HADOOP-9817](https://issues.apache.org/jira/browse/HADOOP-9817) | FileSystem#globStatus and FileContext#globStatus need to work with symlinks | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-9791](https://issues.apache.org/jira/browse/HADOOP-9791) | Add a test case covering long paths for new FileUtil access check methods | Major | . | Ivan Mitic | Ivan Mitic | +| [HADOOP-9787](https://issues.apache.org/jira/browse/HADOOP-9787) | ShutdownHelper util to shutdown threads and threadpools | Major | util | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-9703](https://issues.apache.org/jira/browse/HADOOP-9703) | org.apache.hadoop.ipc.Client leaks threads on stop. | Minor | . | Mark Miller | Tsuyoshi Ozawa | +| [HADOOP-9660](https://issues.apache.org/jira/browse/HADOOP-9660) | [WINDOWS] Powershell / cmd parses -Dkey=value from command line as [-Dkey, value] which breaks GenericsOptionParser | Major | scripts, util | Enis Soztutar | Enis Soztutar | +| [HADOOP-9635](https://issues.apache.org/jira/browse/HADOOP-9635) | Fix Potential Stack Overflow in DomainSocket.c | Major | native | V. Karthik Kumar | | +| [HADOOP-9582](https://issues.apache.org/jira/browse/HADOOP-9582) | Non-existent file to "hadoop fs -conf" doesn't throw error | Major | conf | Ashwin Shankar | Ashwin Shankar | +| [HADOOP-9478](https://issues.apache.org/jira/browse/HADOOP-9478) | Fix race conditions during the initialization of Configuration related to deprecatedKeyMap | Major | conf | Dongyong Wang | Colin Patrick McCabe | +| [HADOOP-9420](https://issues.apache.org/jira/browse/HADOOP-9420) | Add percentile or max metric for rpcQueueTime, processing time | Major | ipc, metrics | Todd Lipcon | Liang Xie | +| [HADOOP-9350](https://issues.apache.org/jira/browse/HADOOP-9350) | Hadoop not building against Java7 on OSX | Minor | build | Steve Loughran | Robert Kanter | +| [HADOOP-9114](https://issues.apache.org/jira/browse/HADOOP-9114) | After defined the dfs.checksum.type as the NULL, write file and hflush will through java.lang.ArrayIndexOutOfBoundsException | Minor | . | liuyang | sathish | +| [HADOOP-9016](https://issues.apache.org/jira/browse/HADOOP-9016) | org.apache.hadoop.fs.HarFileSystem.HarFSDataInputStream.HarFsInputStream.skip(long) must never return negative value. | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-8753](https://issues.apache.org/jira/browse/HADOOP-8753) | LocalDirAllocator throws "ArithmeticException: / by zero" when there is no available space on configured local dir | Minor | . | Nishan Shetty | Benoy Antony | +| [HADOOP-7344](https://issues.apache.org/jira/browse/HADOOP-7344) | globStatus doesn't grok groupings with a slash | Major | fs | Daryn Sharp | Colin Patrick McCabe | +| [HDFS-5921](https://issues.apache.org/jira/browse/HDFS-5921) | Cannot browse file system via NN web UI if any directory has the sticky bit set | Critical | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5876](https://issues.apache.org/jira/browse/HDFS-5876) | SecureDataNodeStarter does not pick up configuration in hdfs-site.xml | Major | datanode | Haohui Mai | Haohui Mai | +| [HDFS-5873](https://issues.apache.org/jira/browse/HDFS-5873) | dfs.http.policy should have higher precedence over dfs.https.enable | Major | . | Yesha Vora | Haohui Mai | +| [HDFS-5845](https://issues.apache.org/jira/browse/HDFS-5845) | SecondaryNameNode dies when checkpointing with cache pools | Blocker | namenode | Andrew Wang | Andrew Wang | +| [HDFS-5844](https://issues.apache.org/jira/browse/HDFS-5844) | Fix broken link in WebHDFS.apt.vm | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5842](https://issues.apache.org/jira/browse/HDFS-5842) | Cannot create hftp filesystem when using a proxy user ugi and a doAs on a secure cluster | Major | security | Arpit Gupta | Jing Zhao | +| [HDFS-5837](https://issues.apache.org/jira/browse/HDFS-5837) | dfs.namenode.replication.considerLoad does not consider decommissioned nodes | Major | namenode | Bryan Beaudreault | Tao Luo | +| [HDFS-5830](https://issues.apache.org/jira/browse/HDFS-5830) | WebHdfsFileSystem.getFileBlockLocations throws IllegalArgumentException when accessing another cluster. | Blocker | caching, hdfs-client | Yongjun Zhang | Yongjun Zhang | +| [HDFS-5806](https://issues.apache.org/jira/browse/HDFS-5806) | balancer should set SoTimeout to avoid indefinite hangs | Major | balancer & mover | Nathan Roberts | Nathan Roberts | +| [HDFS-5800](https://issues.apache.org/jira/browse/HDFS-5800) | Typo: soft-limit for hard-limit in DFSClient | Trivial | hdfs-client | Kousuke Saruta | Kousuke Saruta | +| [HDFS-5789](https://issues.apache.org/jira/browse/HDFS-5789) | Some of snapshot APIs missing checkOperation double check in fsn | Major | namenode | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-5777](https://issues.apache.org/jira/browse/HDFS-5777) | Update LayoutVersion for the new editlog op OP\_ADD\_BLOCK | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5766](https://issues.apache.org/jira/browse/HDFS-5766) | In DFSInputStream, do not add datanode to deadNodes after InvalidEncryptionKeyException in fetchBlockByteRange | Major | hdfs-client | Liang Xie | Liang Xie | +| [HDFS-5762](https://issues.apache.org/jira/browse/HDFS-5762) | BlockReaderLocal doesn't return -1 on EOF when doing zero-length reads | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5756](https://issues.apache.org/jira/browse/HDFS-5756) | hadoopRzOptionsSetByteBufferPool does not accept NULL argument, contrary to docs | Major | libhdfs | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5747](https://issues.apache.org/jira/browse/HDFS-5747) | BlocksMap.getStoredBlock(..) and BlockInfoUnderConstruction.addReplicaIfNotPresent(..) may throw NullPointerException | Minor | namenode | Tsz Wo Nicholas Sze | Arpit Agarwal | +| [HDFS-5728](https://issues.apache.org/jira/browse/HDFS-5728) | [Diskfull] Block recovery will fail if the metafile does not have crc for all chunks of the block | Critical | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-5719](https://issues.apache.org/jira/browse/HDFS-5719) | FSImage#doRollback() should close prevState before return | Minor | namenode | Ted Yu | Ted Yu | +| [HDFS-5710](https://issues.apache.org/jira/browse/HDFS-5710) | FSDirectory#getFullPathName should check inodes against null | Major | . | Ted Yu | Uma Maheswara Rao G | +| [HDFS-5704](https://issues.apache.org/jira/browse/HDFS-5704) | Change OP\_UPDATE\_BLOCKS with a new OP\_ADD\_BLOCK | Major | namenode | Suresh Srinivas | Jing Zhao | +| [HDFS-5701](https://issues.apache.org/jira/browse/HDFS-5701) | Fix the CacheAdmin -addPool -maxTtl option name | Minor | caching, tools | Stephen Chu | Stephen Chu | +| [HDFS-5691](https://issues.apache.org/jira/browse/HDFS-5691) | Fix typo in ShortCircuitLocalRead document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5690](https://issues.apache.org/jira/browse/HDFS-5690) | DataNode fails to start in secure mode when dfs.http.policy equals to HTTP\_ONLY | Blocker | . | Haohui Mai | Haohui Mai | +| [HDFS-5681](https://issues.apache.org/jira/browse/HDFS-5681) | renewLease should not hold fsn write lock | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-5675](https://issues.apache.org/jira/browse/HDFS-5675) | Add Mkdirs operation to NNThroughputBenchmark | Minor | benchmarks | Plamen Jeliazkov | Plamen Jeliazkov | +| [HDFS-5671](https://issues.apache.org/jira/browse/HDFS-5671) | Fix socket leak in DFSInputStream#getBlockReader | Critical | hdfs-client | JamesLi | JamesLi | +| [HDFS-5666](https://issues.apache.org/jira/browse/HDFS-5666) | Fix inconsistent synchronization in BPOfferService | Minor | namenode | Colin Patrick McCabe | Jimmy Xiang | +| [HDFS-5661](https://issues.apache.org/jira/browse/HDFS-5661) | Browsing FileSystem via web ui, should use datanode's fqdn instead of ip address | Major | security | Benoy Antony | Benoy Antony | +| [HDFS-5659](https://issues.apache.org/jira/browse/HDFS-5659) | dfsadmin -report doesn't output cache information properly | Major | caching | Akira AJISAKA | Andrew Wang | +| [HDFS-5657](https://issues.apache.org/jira/browse/HDFS-5657) | race condition causes writeback state error in NFS gateway | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5649](https://issues.apache.org/jira/browse/HDFS-5649) | Unregister NFS and Mount service when NFS gateway is shutting down | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5592](https://issues.apache.org/jira/browse/HDFS-5592) | "DIR\* completeFile: /file is closed by DFSClient\_" should be logged only for successful closure of the file. | Major | . | Vinayakumar B | Vinayakumar B | +| [HDFS-5590](https://issues.apache.org/jira/browse/HDFS-5590) | Block ID and generation stamp may be reused when persistBlocks is set to false | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-5582](https://issues.apache.org/jira/browse/HDFS-5582) | hdfs getconf -excludeFile or -includeFile always failed | Minor | . | Henry Hung | sathish | +| [HDFS-5581](https://issues.apache.org/jira/browse/HDFS-5581) | NameNodeFsck should use only one instance of BlockPlacementPolicy | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5580](https://issues.apache.org/jira/browse/HDFS-5580) | Infinite loop in Balancer.waitForMoveCompletion | Major | . | Binglin Chang | Binglin Chang | +| [HDFS-5579](https://issues.apache.org/jira/browse/HDFS-5579) | Under construction files make DataNode decommission take very long hours | Major | namenode | zhaoyunjiong | zhaoyunjiong | +| [HDFS-5562](https://issues.apache.org/jira/browse/HDFS-5562) | TestCacheDirectives and TestFsDatasetCache should stub out native mlock | Major | test | Akira AJISAKA | Colin Patrick McCabe | +| [HDFS-5560](https://issues.apache.org/jira/browse/HDFS-5560) | Trash configuration log statements prints incorrect units | Major | . | Josh Elser | Josh Elser | +| [HDFS-5558](https://issues.apache.org/jira/browse/HDFS-5558) | LeaseManager monitor thread can crash if the last block is complete but another block is not. | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5557](https://issues.apache.org/jira/browse/HDFS-5557) | Write pipeline recovery for the last packet in the block may cause rejection of valid replicas | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5555](https://issues.apache.org/jira/browse/HDFS-5555) | CacheAdmin commands fail when first listed NameNode is in Standby | Major | caching | Stephen Chu | Jimmy Xiang | +| [HDFS-5552](https://issues.apache.org/jira/browse/HDFS-5552) | Fix wrong information of "Cluster summay" in dfshealth.html | Major | namenode | Shinichi Yamashita | Haohui Mai | +| [HDFS-5544](https://issues.apache.org/jira/browse/HDFS-5544) | Adding Test case For Checking dfs.checksum type as NULL value | Minor | hdfs-client | sathish | sathish | +| [HDFS-5540](https://issues.apache.org/jira/browse/HDFS-5540) | Fix intermittent failure in TestBlocksWithNotEnoughRacks | Minor | . | Binglin Chang | Binglin Chang | +| [HDFS-5533](https://issues.apache.org/jira/browse/HDFS-5533) | Symlink delete/create should be treated as DELETE/CREATE in snapshot diff report | Minor | snapshots | Binglin Chang | Binglin Chang | +| [HDFS-5526](https://issues.apache.org/jira/browse/HDFS-5526) | Datanode cannot roll back to previous layout version | Blocker | datanode | Tsz Wo Nicholas Sze | Kihwal Lee | +| [HDFS-5513](https://issues.apache.org/jira/browse/HDFS-5513) | CacheAdmin commands fail when using . as the path | Minor | caching, tools | Stephen Chu | Andrew Wang | +| [HDFS-5512](https://issues.apache.org/jira/browse/HDFS-5512) | CacheAdmin -listPools fails with NPE when user lacks permissions to view all pools | Major | caching, tools | Stephen Chu | Andrew Wang | +| [HDFS-5504](https://issues.apache.org/jira/browse/HDFS-5504) | In HA mode, OP\_DELETE\_SNAPSHOT is not decrementing the safemode threshold, leads to NN safemode. | Major | snapshots | Vinayakumar B | Vinayakumar B | +| [HDFS-5482](https://issues.apache.org/jira/browse/HDFS-5482) | DistributedFileSystem#listPathBasedCacheDirectives must support relative paths | Major | tools | Stephen Chu | Colin Patrick McCabe | +| [HDFS-5476](https://issues.apache.org/jira/browse/HDFS-5476) | Snapshot: clean the blocks/files/directories under a renamed file/directory while deletion | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-5474](https://issues.apache.org/jira/browse/HDFS-5474) | Deletesnapshot can make Namenode in safemode on NN restarts. | Major | snapshots | Uma Maheswara Rao G | sathish | +| [HDFS-5471](https://issues.apache.org/jira/browse/HDFS-5471) | CacheAdmin -listPools fails when user lacks permissions to view all pools | Major | tools | Stephen Chu | Andrew Wang | +| [HDFS-5468](https://issues.apache.org/jira/browse/HDFS-5468) | CacheAdmin help command does not recognize commands | Minor | tools | Stephen Chu | Stephen Chu | +| [HDFS-5458](https://issues.apache.org/jira/browse/HDFS-5458) | Datanode failed volume threshold ignored if exception is thrown in getDataDirsFromURIs | Major | datanode | Andrew Wang | Mike Mellenthin | +| [HDFS-5456](https://issues.apache.org/jira/browse/HDFS-5456) | NameNode startup progress creates new steps if caller attempts to create a counter for a step that doesn't already exist. | Critical | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-5449](https://issues.apache.org/jira/browse/HDFS-5449) | WebHdfs compatibility broken between 2.2 and 1.x / 23.x | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5443](https://issues.apache.org/jira/browse/HDFS-5443) | Delete 0-sized block when deleting an under-construction file that is included in snapshot | Major | snapshots | Uma Maheswara Rao G | Jing Zhao | +| [HDFS-5438](https://issues.apache.org/jira/browse/HDFS-5438) | Flaws in block report processing can cause data loss | Critical | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-5434](https://issues.apache.org/jira/browse/HDFS-5434) | Write resiliency for replica count 1 | Minor | namenode | Taylor, Buddy | | +| [HDFS-5433](https://issues.apache.org/jira/browse/HDFS-5433) | When reloading fsimage during checkpointing, we should clear existing snapshottable directories | Critical | snapshots | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5432](https://issues.apache.org/jira/browse/HDFS-5432) | TestDatanodeJsp fails on Windows due to assumption that loopback address resolves to host name localhost. | Trivial | datanode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-5428](https://issues.apache.org/jira/browse/HDFS-5428) | under construction files deletion after snapshot+checkpoint+nn restart leads nn safemode | Major | snapshots | Vinayakumar B | Jing Zhao | +| [HDFS-5427](https://issues.apache.org/jira/browse/HDFS-5427) | not able to read deleted files from snapshot directly under snapshottable dir after checkpoint and NN restart | Major | snapshots | Vinayakumar B | Vinayakumar B | +| [HDFS-5425](https://issues.apache.org/jira/browse/HDFS-5425) | Renaming underconstruction file with snapshots can make NN failure on restart | Major | namenode, snapshots | sathish | Jing Zhao | +| [HDFS-5413](https://issues.apache.org/jira/browse/HDFS-5413) | hdfs.cmd does not support passthrough to any arbitrary class. | Major | scripts | Chris Nauroth | Chris Nauroth | +| [HDFS-5407](https://issues.apache.org/jira/browse/HDFS-5407) | Fix typos in DFSClientCache | Trivial | . | Haohui Mai | Haohui Mai | +| [HDFS-5403](https://issues.apache.org/jira/browse/HDFS-5403) | WebHdfs client cannot communicate with older WebHdfs servers post HDFS-5306 | Major | webhdfs | Aaron T. Myers | Aaron T. Myers | +| [HDFS-5400](https://issues.apache.org/jira/browse/HDFS-5400) | DFS\_CLIENT\_MMAP\_CACHE\_THREAD\_RUNS\_PER\_TIMEOUT constant is set to the wrong value | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5375](https://issues.apache.org/jira/browse/HDFS-5375) | hdfs.cmd does not expose several snapshot commands. | Minor | tools | Chris Nauroth | Chris Nauroth | +| [HDFS-5374](https://issues.apache.org/jira/browse/HDFS-5374) | Remove deadcode in DFSOutputStream | Trivial | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-5372](https://issues.apache.org/jira/browse/HDFS-5372) | In FSNamesystem, hasReadLock() returns false if the current thread holds the write lock | Major | namenode | Tsz Wo Nicholas Sze | Vinayakumar B | +| [HDFS-5370](https://issues.apache.org/jira/browse/HDFS-5370) | Typo in Error Message: different between range in condition and range in error message | Trivial | hdfs-client | Kousuke Saruta | Kousuke Saruta | +| [HDFS-5365](https://issues.apache.org/jira/browse/HDFS-5365) | Fix libhdfs compile error on FreeBSD9 | Major | build, libhdfs | Radim Kolar | Radim Kolar | +| [HDFS-5353](https://issues.apache.org/jira/browse/HDFS-5353) | Short circuit reads fail when dfs.encrypt.data.transfer is enabled | Blocker | . | Haohui Mai | Colin Patrick McCabe | +| [HDFS-5352](https://issues.apache.org/jira/browse/HDFS-5352) | Server#initLog() doesn't close InputStream in httpfs | Minor | . | Ted Yu | Ted Yu | +| [HDFS-5346](https://issues.apache.org/jira/browse/HDFS-5346) | Avoid unnecessary call to getNumLiveDataNodes() for each block during IBR processing | Major | namenode, performance | Kihwal Lee | Ravi Prakash | +| [HDFS-5343](https://issues.apache.org/jira/browse/HDFS-5343) | When cat command is issued on snapshot files getting unexpected result | Major | hdfs-client | sathish | sathish | +| [HDFS-5341](https://issues.apache.org/jira/browse/HDFS-5341) | Reduce fsdataset lock duration during directory scanning. | Major | datanode | qus-jiawei | qus-jiawei | +| [HDFS-5336](https://issues.apache.org/jira/browse/HDFS-5336) | DataNode should not output 'StartupProgress' metrics | Minor | namenode | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5335](https://issues.apache.org/jira/browse/HDFS-5335) | DFSOutputStream#close() keeps throwing exceptions when it is called multiple times | Major | . | Arpit Gupta | Haohui Mai | +| [HDFS-5329](https://issues.apache.org/jira/browse/HDFS-5329) | Update FSNamesystem#getListing() to handle inode path in startAfter token | Major | namenode, nfs | Brandon Li | Brandon Li | +| [HDFS-5322](https://issues.apache.org/jira/browse/HDFS-5322) | HDFS delegation token not found in cache errors seen on secure HA clusters | Major | ha | Arpit Gupta | Jing Zhao | +| [HDFS-5305](https://issues.apache.org/jira/browse/HDFS-5305) | Add https support in HDFS | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-5297](https://issues.apache.org/jira/browse/HDFS-5297) | Fix dead links in HDFS site documents | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5291](https://issues.apache.org/jira/browse/HDFS-5291) | Clients need to retry when Active NN is in SafeMode | Critical | ha | Arpit Gupta | Jing Zhao | +| [HDFS-5283](https://issues.apache.org/jira/browse/HDFS-5283) | NN not coming out of startup safemode due to under construction blocks only inside snapshots also counted in safemode threshhold | Critical | snapshots | Vinayakumar B | Vinayakumar B | +| [HDFS-5276](https://issues.apache.org/jira/browse/HDFS-5276) | FileSystem.Statistics got performance issue on multi-thread read/write. | Major | . | Chengxiang Li | Colin Patrick McCabe | +| [HDFS-5257](https://issues.apache.org/jira/browse/HDFS-5257) | addBlock() retry should return LocatedBlock with locations else client will get AIOBE | Major | hdfs-client, namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5170](https://issues.apache.org/jira/browse/HDFS-5170) | BlockPlacementPolicyDefault uses the wrong classname when alerting to enable debug logging | Trivial | . | Andrew Wang | Andrew Wang | +| [HDFS-5164](https://issues.apache.org/jira/browse/HDFS-5164) | deleteSnapshot should check if OperationCategory.WRITE is possible before taking write lock | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5122](https://issues.apache.org/jira/browse/HDFS-5122) | Support failover and retry in WebHdfsFileSystem for NN HA | Major | ha, webhdfs | Arpit Gupta | Haohui Mai | +| [HDFS-5093](https://issues.apache.org/jira/browse/HDFS-5093) | TestGlobPaths should re-use the MiniDFSCluster to avoid failure on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [HDFS-5075](https://issues.apache.org/jira/browse/HDFS-5075) | httpfs-config.sh calls out incorrect env script name | Major | . | Timothy St. Clair | Timothy St. Clair | +| [HDFS-5074](https://issues.apache.org/jira/browse/HDFS-5074) | Allow starting up from an fsimage checkpoint in the middle of a segment | Major | ha, namenode | Todd Lipcon | Todd Lipcon | +| [HDFS-5073](https://issues.apache.org/jira/browse/HDFS-5073) | TestListCorruptFileBlocks fails intermittently | Minor | test | Kihwal Lee | Arpit Agarwal | +| [HDFS-5065](https://issues.apache.org/jira/browse/HDFS-5065) | TestSymlinkHdfsDisable fails on Windows | Major | hdfs-client, test | Ivan Mitic | Ivan Mitic | +| [HDFS-5035](https://issues.apache.org/jira/browse/HDFS-5035) | getFileLinkStatus and rename do not correctly check permissions of symlinks | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-5023](https://issues.apache.org/jira/browse/HDFS-5023) | TestSnapshotPathINodes.testAllowSnapshot is failing with jdk7 | Major | snapshots, test | Ravi Prakash | Mit Desai | +| [HDFS-5014](https://issues.apache.org/jira/browse/HDFS-5014) | BPOfferService#processCommandFromActor() synchronization on namenode RPC call delays IBR to Active NN, if Stanby NN is unstable | Major | datanode, ha | Vinayakumar B | Vinayakumar B | +| [HDFS-4995](https://issues.apache.org/jira/browse/HDFS-4995) | Make getContentSummary() less expensive | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-4994](https://issues.apache.org/jira/browse/HDFS-4994) | Audit log getContentSummary() calls | Minor | namenode | Kihwal Lee | Robert Parker | +| [HDFS-4816](https://issues.apache.org/jira/browse/HDFS-4816) | transitionToActive blocks if the SBN is doing checkpoint image transfer | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-4657](https://issues.apache.org/jira/browse/HDFS-4657) | Limit the number of blocks logged by the NN after a block report to a configurable value. | Major | namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4633](https://issues.apache.org/jira/browse/HDFS-4633) | TestDFSClientExcludedNodes fails sporadically if excluded nodes cache expires too quickly | Major | hdfs-client, test | Chris Nauroth | Chris Nauroth | +| [HDFS-4516](https://issues.apache.org/jira/browse/HDFS-4516) | Client crash after block allocation and NN switch before lease recovery for the same file can cause readers to fail forever | Critical | namenode | Uma Maheswara Rao G | Vinayakumar B | +| [HDFS-4376](https://issues.apache.org/jira/browse/HDFS-4376) | Fix several race conditions in Balancer and resolve intermittent timeout of TestBalancerWithNodeGroup | Major | balancer & mover | Aaron T. Myers | Junping Du | +| [HDFS-4329](https://issues.apache.org/jira/browse/HDFS-4329) | DFSShell issues with directories with spaces in name | Major | hdfs-client | Andy Isaacson | Cristina L. Abad | +| [HDFS-4201](https://issues.apache.org/jira/browse/HDFS-4201) | NPE in BPServiceActor#sendHeartBeat | Critical | namenode | Eli Collins | Jimmy Xiang | +| [HDFS-3981](https://issues.apache.org/jira/browse/HDFS-3981) | access time is set without holding FSNamesystem write lock | Major | namenode | Xiaobo Peng | Xiaobo Peng | +| [MAPREDUCE-5744](https://issues.apache.org/jira/browse/MAPREDUCE-5744) | Job hangs because RMContainerAllocator$AssignedRequests.preemptReduce() violates the comparator contract | Blocker | . | Sangjin Lee | Gera Shegalov | +| [MAPREDUCE-5743](https://issues.apache.org/jira/browse/MAPREDUCE-5743) | TestRMContainerAllocator is failing | Major | . | Ted Yu | Ted Yu | +| [MAPREDUCE-5729](https://issues.apache.org/jira/browse/MAPREDUCE-5729) | mapred job -list throws NPE | Critical | mrv2 | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5725](https://issues.apache.org/jira/browse/MAPREDUCE-5725) | TestNetworkedJob relies on the Capacity Scheduler | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5724](https://issues.apache.org/jira/browse/MAPREDUCE-5724) | JobHistoryServer does not start if HDFS is not running | Critical | jobhistoryserver | Alejandro Abdelnur | Alejandro Abdelnur | +| [MAPREDUCE-5723](https://issues.apache.org/jira/browse/MAPREDUCE-5723) | MR AM container log can be truncated or empty | Blocker | applicationmaster | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [MAPREDUCE-5694](https://issues.apache.org/jira/browse/MAPREDUCE-5694) | MR AM container syslog is empty | Major | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [MAPREDUCE-5693](https://issues.apache.org/jira/browse/MAPREDUCE-5693) | Restore MRv1 behavior for log flush | Major | mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5689](https://issues.apache.org/jira/browse/MAPREDUCE-5689) | MRAppMaster does not preempt reducers when scheduled maps cannot be fulfilled | Critical | . | Lohit Vijayarenu | Lohit Vijayarenu | +| [MAPREDUCE-5685](https://issues.apache.org/jira/browse/MAPREDUCE-5685) | getCacheFiles() api doesn't work in WrappedReducer.java due to typo | Blocker | client | Yi Song | Yi Song | +| [MAPREDUCE-5679](https://issues.apache.org/jira/browse/MAPREDUCE-5679) | TestJobHistoryParsing has race condition | Major | . | Liyin Liang | Liyin Liang | +| [MAPREDUCE-5674](https://issues.apache.org/jira/browse/MAPREDUCE-5674) | Missing start and finish time in mapred.JobStatus | Major | client | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5656](https://issues.apache.org/jira/browse/MAPREDUCE-5656) | bzip2 codec can drop records when reading data in splits | Critical | . | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5650](https://issues.apache.org/jira/browse/MAPREDUCE-5650) | Job fails when hprof mapreduce.task.profile.map/reduce.params is specified | Major | mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5645](https://issues.apache.org/jira/browse/MAPREDUCE-5645) | TestFixedLengthInputFormat fails with native libs | Major | . | Jonathan Eagles | Mit Desai | +| [MAPREDUCE-5631](https://issues.apache.org/jira/browse/MAPREDUCE-5631) | TestJobEndNotifier.testNotifyRetries fails with Should have taken more than 5 seconds in jdk7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5623](https://issues.apache.org/jira/browse/MAPREDUCE-5623) | TestJobCleanup fails because of RejectedExecutionException and NPE. | Major | . | Tsuyoshi Ozawa | Jason Lowe | +| [MAPREDUCE-5616](https://issues.apache.org/jira/browse/MAPREDUCE-5616) | MR Client-AppMaster RPC max retries on socket timeout is too high. | Major | client | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5613](https://issues.apache.org/jira/browse/MAPREDUCE-5613) | DefaultSpeculator holds and checks hashmap that is always empty | Major | applicationmaster | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5604](https://issues.apache.org/jira/browse/MAPREDUCE-5604) | TestMRAMWithNonNormalizedCapabilities fails on Windows due to exceeding max path length | Minor | test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5598](https://issues.apache.org/jira/browse/MAPREDUCE-5598) | TestUserDefinedCounters.testMapReduceJob is flakey | Major | test | Robert Kanter | Robert Kanter | +| [MAPREDUCE-5587](https://issues.apache.org/jira/browse/MAPREDUCE-5587) | TestTextOutputFormat fails on JDK7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5586](https://issues.apache.org/jira/browse/MAPREDUCE-5586) | TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5585](https://issues.apache.org/jira/browse/MAPREDUCE-5585) | TestCopyCommitter#testNoCommitAction Fails on JDK7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5569](https://issues.apache.org/jira/browse/MAPREDUCE-5569) | FloatSplitter is not generating correct splits | Major | . | Nathan Roberts | Nathan Roberts | +| [MAPREDUCE-5561](https://issues.apache.org/jira/browse/MAPREDUCE-5561) | org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk | Critical | . | Cindy Li | Karthik Kambatla | +| [MAPREDUCE-5550](https://issues.apache.org/jira/browse/MAPREDUCE-5550) | Task Status message (reporter.setStatus) not shown in UI with Hadoop 2.0 | Major | . | Vrushali C | Gera Shegalov | +| [MAPREDUCE-5546](https://issues.apache.org/jira/browse/MAPREDUCE-5546) | mapred.cmd on Windows set HADOOP\_OPTS incorrectly | Major | . | Chuan Liu | Chuan Liu | +| [MAPREDUCE-5522](https://issues.apache.org/jira/browse/MAPREDUCE-5522) | Incorrectly expect the array of JobQueueInfo returned by o.a.h.mapred.QueueManager#getJobQueueInfos to have a specific order. | Minor | test | Jinghui Wang | Jinghui Wang | +| [MAPREDUCE-5518](https://issues.apache.org/jira/browse/MAPREDUCE-5518) | Fix typo "can't read paritions file" | Trivial | examples | Albert Chu | Albert Chu | +| [MAPREDUCE-5514](https://issues.apache.org/jira/browse/MAPREDUCE-5514) | TestRMContainerAllocator fails on trunk | Blocker | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5504](https://issues.apache.org/jira/browse/MAPREDUCE-5504) | mapred queue -info inconsistent with types | Major | client | Thomas Graves | Kousuke Saruta | +| [MAPREDUCE-5481](https://issues.apache.org/jira/browse/MAPREDUCE-5481) | Enable uber jobs to have multiple reducers | Blocker | mrv2, test | Jason Lowe | Sandy Ryza | +| [MAPREDUCE-5451](https://issues.apache.org/jira/browse/MAPREDUCE-5451) | MR uses LD\_LIBRARY\_PATH which doesn't mean anything in Windows | Major | . | Mostafa Elhemali | Yingda Chen | +| [MAPREDUCE-5431](https://issues.apache.org/jira/browse/MAPREDUCE-5431) | Missing pom dependency in MR-client | Major | build | Timothy St. Clair | Timothy St. Clair | +| [MAPREDUCE-5404](https://issues.apache.org/jira/browse/MAPREDUCE-5404) | HSAdminServer does not use ephemeral ports in minicluster mode | Major | jobhistoryserver | Ted Yu | Ted Yu | +| [MAPREDUCE-5380](https://issues.apache.org/jira/browse/MAPREDUCE-5380) | Invalid mapred command should return non-zero exit code | Major | . | Stephen Chu | Stephen Chu | +| [MAPREDUCE-5373](https://issues.apache.org/jira/browse/MAPREDUCE-5373) | TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently | Major | . | Chuan Liu | Jonathan Eagles | +| [MAPREDUCE-5329](https://issues.apache.org/jira/browse/MAPREDUCE-5329) | APPLICATION\_INIT is never sent to AuxServices other than the builtin ShuffleHandler | Major | mr-am | Avner BenHanoch | Avner BenHanoch | +| [MAPREDUCE-5316](https://issues.apache.org/jira/browse/MAPREDUCE-5316) | job -list-attempt-ids command does not handle illegal task-state | Major | client | Ashwin Shankar | Ashwin Shankar | +| [MAPREDUCE-5186](https://issues.apache.org/jira/browse/MAPREDUCE-5186) | mapreduce.job.max.split.locations causes some splits created by CombineFileInputFormat to fail | Critical | job submission | Sangjin Lee | Robert Parker | +| [MAPREDUCE-5052](https://issues.apache.org/jira/browse/MAPREDUCE-5052) | Job History UI and web services confusing job start time and job submit time | Critical | jobhistoryserver, webapps | Kendall Thrapp | Chen He | +| [MAPREDUCE-4680](https://issues.apache.org/jira/browse/MAPREDUCE-4680) | Job history cleaner should only check timestamps of files in old enough directories | Major | jobhistoryserver | Sandy Ryza | Robert Kanter | +| [YARN-1630](https://issues.apache.org/jira/browse/YARN-1630) | Introduce timeout for async polling operations in YarnClientImpl | Major | client | Aditya Acharya | Aditya Acharya | +| [YARN-1629](https://issues.apache.org/jira/browse/YARN-1629) | IndexOutOfBoundsException in Fair Scheduler MaxRunningAppsEnforcer | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1628](https://issues.apache.org/jira/browse/YARN-1628) | TestContainerManagerSecurity fails on trunk | Major | . | Mit Desai | Vinod Kumar Vavilapalli | +| [YARN-1624](https://issues.apache.org/jira/browse/YARN-1624) | QueuePlacementPolicy format is not easily readable via a JAXB parser | Major | scheduler | Aditya Acharya | Aditya Acharya | +| [YARN-1608](https://issues.apache.org/jira/browse/YARN-1608) | LinuxContainerExecutor has a few DEBUG messages at INFO level | Trivial | nodemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1607](https://issues.apache.org/jira/browse/YARN-1607) | TestRM expects the capacity scheduler | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-1603](https://issues.apache.org/jira/browse/YARN-1603) | Remove two \*.orig files which were unexpectedly committed | Trivial | . | Zhijie Shen | Zhijie Shen | +| [YARN-1601](https://issues.apache.org/jira/browse/YARN-1601) | 3rd party JARs are missing from hadoop-dist output | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1600](https://issues.apache.org/jira/browse/YARN-1600) | RM does not startup when security is enabled without spnego configured | Blocker | resourcemanager | Jason Lowe | Haohui Mai | +| [YARN-1541](https://issues.apache.org/jira/browse/YARN-1541) | Invalidate AM Host/Port when app attempt is done so that in the mean-while client doesn’t get wrong information. | Major | . | Jian He | Jian He | +| [YARN-1527](https://issues.apache.org/jira/browse/YARN-1527) | yarn rmadmin command prints wrong usage info: | Trivial | . | Jian He | Akira AJISAKA | +| [YARN-1522](https://issues.apache.org/jira/browse/YARN-1522) | TestApplicationCleanup.testAppCleanup occasionally fails | Major | . | Liyin Liang | Liyin Liang | +| [YARN-1505](https://issues.apache.org/jira/browse/YARN-1505) | WebAppProxyServer should not set localhost as YarnConfiguration.PROXY\_ADDRESS by itself | Blocker | . | Xuan Gong | Xuan Gong | +| [YARN-1491](https://issues.apache.org/jira/browse/YARN-1491) | Upgrade JUnit3 TestCase to JUnit 4 | Trivial | . | Jonathan Eagles | Chen He | +| [YARN-1454](https://issues.apache.org/jira/browse/YARN-1454) | TestRMRestart.testRMDelegationTokenRestoredOnRMRestart is failing intermittently | Critical | . | Jian He | Karthik Kambatla | +| [YARN-1451](https://issues.apache.org/jira/browse/YARN-1451) | TestResourceManager relies on the scheduler assigning multiple containers in a single node update | Minor | . | Sandy Ryza | Sandy Ryza | +| [YARN-1450](https://issues.apache.org/jira/browse/YARN-1450) | TestUnmanagedAMLauncher#testDSShell fails on trunk | Major | applications/distributed-shell | Akira AJISAKA | Binglin Chang | +| [YARN-1435](https://issues.apache.org/jira/browse/YARN-1435) | Distributed Shell should not run other commands except "sh", and run the custom script at the same time. | Major | applications/distributed-shell | Tassapol Athiapinya | Xuan Gong | +| [YARN-1425](https://issues.apache.org/jira/browse/YARN-1425) | TestRMRestart fails because MockRM.waitForState(AttemptId) uses current attempt instead of the attempt passed as argument | Major | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-1419](https://issues.apache.org/jira/browse/YARN-1419) | TestFifoScheduler.testAppAttemptMetrics fails intermittently under jdk7 | Minor | scheduler | Jonathan Eagles | Jonathan Eagles | +| [YARN-1416](https://issues.apache.org/jira/browse/YARN-1416) | InvalidStateTransitions getting reported in multiple test cases even though they pass | Major | . | Omkar Vinit Joshi | Jian He | +| [YARN-1409](https://issues.apache.org/jira/browse/YARN-1409) | NonAggregatingLogHandler can throw RejectedExecutionException | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-1407](https://issues.apache.org/jira/browse/YARN-1407) | RM Web UI and REST APIs should uniformly use YarnApplicationState | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-1401](https://issues.apache.org/jira/browse/YARN-1401) | With zero sleep-delay-before-sigkill.ms, no signal is ever sent | Major | nodemanager | Gera Shegalov | Gera Shegalov | +| [YARN-1400](https://issues.apache.org/jira/browse/YARN-1400) | yarn.cmd uses HADOOP\_RESOURCEMANAGER\_OPTS. Should be YARN\_RESOURCEMANAGER\_OPTS. | Trivial | resourcemanager | Raja Aluri | Raja Aluri | +| [YARN-1395](https://issues.apache.org/jira/browse/YARN-1395) | Distributed shell application master launched with debug flag can hang waiting for external ls process. | Major | applications/distributed-shell | Chris Nauroth | Chris Nauroth | +| [YARN-1388](https://issues.apache.org/jira/browse/YARN-1388) | Fair Scheduler page always displays blank fair share | Trivial | resourcemanager | Liyin Liang | Liyin Liang | +| [YARN-1386](https://issues.apache.org/jira/browse/YARN-1386) | NodeManager mistakenly loses resources and relocalizes them | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1381](https://issues.apache.org/jira/browse/YARN-1381) | Same relaxLocality appears twice in exception message of AMRMClientImpl#checkLocalityRelaxationConflict() | Minor | . | Ted Yu | Ted Yu | +| [YARN-1374](https://issues.apache.org/jira/browse/YARN-1374) | Resource Manager fails to start due to ConcurrentModificationException | Blocker | resourcemanager | Devaraj K | Karthik Kambatla | +| [YARN-1351](https://issues.apache.org/jira/browse/YARN-1351) | Invalid string format in Fair Scheduler log warn message | Trivial | resourcemanager | Konstantin Weitz | Konstantin Weitz | +| [YARN-1349](https://issues.apache.org/jira/browse/YARN-1349) | yarn.cmd does not support passthrough to any arbitrary class. | Major | client | Chris Nauroth | Chris Nauroth | +| [YARN-1343](https://issues.apache.org/jira/browse/YARN-1343) | NodeManagers additions/restarts are not reported as node updates in AllocateResponse responses to AMs | Critical | resourcemanager | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1331](https://issues.apache.org/jira/browse/YARN-1331) | yarn.cmd exits with NoClassDefFoundError trying to run rmadmin or logs | Trivial | client | Chris Nauroth | Chris Nauroth | +| [YARN-1330](https://issues.apache.org/jira/browse/YARN-1330) | Fair Scheduler: defaultQueueSchedulingPolicy does not take effect | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1321](https://issues.apache.org/jira/browse/YARN-1321) | NMTokenCache is a singleton, prevents multiple AMs running in a single JVM to work correctly | Blocker | client | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1320](https://issues.apache.org/jira/browse/YARN-1320) | Custom log4j properties in Distributed shell does not work properly. | Major | applications/distributed-shell | Tassapol Athiapinya | Xuan Gong | +| [YARN-1315](https://issues.apache.org/jira/browse/YARN-1315) | TestQueueACLs should also test FairScheduler | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1314](https://issues.apache.org/jira/browse/YARN-1314) | Cannot pass more than 1 argument to shell command | Major | applications/distributed-shell | Tassapol Athiapinya | Xuan Gong | +| [YARN-1306](https://issues.apache.org/jira/browse/YARN-1306) | Clean up hadoop-sls sample-conf according to YARN-1228 | Major | . | Wei Yan | Wei Yan | +| [YARN-1300](https://issues.apache.org/jira/browse/YARN-1300) | SLS tests fail because conf puts yarn properties in fair-scheduler.xml | Major | . | Ted Yu | Ted Yu | +| [YARN-1295](https://issues.apache.org/jira/browse/YARN-1295) | In UnixLocalWrapperScriptBuilder, using bash -c can cause "Text file busy" errors | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1293](https://issues.apache.org/jira/browse/YARN-1293) | TestContainerLaunch.testInvalidEnvSyntaxDiagnostics fails on trunk | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-1288](https://issues.apache.org/jira/browse/YARN-1288) | Make Fair Scheduler ACLs more user friendly | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1284](https://issues.apache.org/jira/browse/YARN-1284) | LCE: Race condition leaves dangling cgroups entries for killed containers | Blocker | nodemanager | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-1268](https://issues.apache.org/jira/browse/YARN-1268) | TestFairScheduler.testContinuousScheduling is flaky | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1265](https://issues.apache.org/jira/browse/YARN-1265) | Fair Scheduler chokes on unhealthy node reconnect | Major | resourcemanager, scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1259](https://issues.apache.org/jira/browse/YARN-1259) | In Fair Scheduler web UI, queue num pending and num active apps switched | Trivial | scheduler | Sandy Ryza | Robert Kanter | +| [YARN-1241](https://issues.apache.org/jira/browse/YARN-1241) | In Fair Scheduler, maxRunningApps does not work for non-leaf queues | Major | . | Sandy Ryza | Sandy Ryza | +| [YARN-1188](https://issues.apache.org/jira/browse/YARN-1188) | The context of QueueMetrics becomes 'default' when using FairScheduler | Trivial | . | Akira AJISAKA | Tsuyoshi Ozawa | +| [YARN-1183](https://issues.apache.org/jira/browse/YARN-1183) | MiniYARNCluster shutdown takes several minutes intermittently | Major | . | Andrey Klochkov | Andrey Klochkov | +| [YARN-1182](https://issues.apache.org/jira/browse/YARN-1182) | MiniYARNCluster creates and inits the RM/NM only on start() | Major | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-1180](https://issues.apache.org/jira/browse/YARN-1180) | Update capacity scheduler docs to include types on the configs | Trivial | capacityscheduler | Thomas Graves | Chen He | +| [YARN-1145](https://issues.apache.org/jira/browse/YARN-1145) | Potential file handle leak in aggregated logs web ui | Major | . | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-1138](https://issues.apache.org/jira/browse/YARN-1138) | yarn.application.classpath is set to point to $HADOOP\_CONF\_DIR etc., which does not work on Windows | Major | api | Yingda Chen | Chuan Liu | +| [YARN-1060](https://issues.apache.org/jira/browse/YARN-1060) | Two tests in TestFairScheduler are missing @Test annotation | Major | scheduler | Sandy Ryza | Niranjan Singh | +| [YARN-1053](https://issues.apache.org/jira/browse/YARN-1053) | Diagnostic message from ContainerExitEvent is ignored in ContainerImpl | Blocker | . | Omkar Vinit Joshi | Omkar Vinit Joshi | +| [YARN-1044](https://issues.apache.org/jira/browse/YARN-1044) | used/min/max resources do not display info in the scheduler page | Critical | resourcemanager, scheduler | Sangjin Lee | Sangjin Lee | +| [YARN-1022](https://issues.apache.org/jira/browse/YARN-1022) | Unnecessary INFO logs in AMRMClientAsync | Trivial | . | Bikas Saha | haosdent | +| [YARN-888](https://issues.apache.org/jira/browse/YARN-888) | clean up POM dependencies | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [YARN-879](https://issues.apache.org/jira/browse/YARN-879) | Fix tests w.r.t o.a.h.y.server.resourcemanager.Application | Major | . | Junping Du | Junping Du | +| [YARN-584](https://issues.apache.org/jira/browse/YARN-584) | In scheduler web UIs, queues unexpand on refresh | Major | scheduler | Sandy Ryza | Harshit Daga | +| [YARN-546](https://issues.apache.org/jira/browse/YARN-546) | Allow disabling the Fair Scheduler event log | Major | scheduler | Lohit Vijayarenu | Sandy Ryza | +| [YARN-461](https://issues.apache.org/jira/browse/YARN-461) | Fair scheduler should not accept apps with empty string queue name | Major | resourcemanager | Sandy Ryza | Wei Yan | +| [YARN-408](https://issues.apache.org/jira/browse/YARN-408) | Capacity Scheduler delay scheduling should not be disabled by default | Minor | scheduler | Mayank Bansal | Mayank Bansal | +| [YARN-305](https://issues.apache.org/jira/browse/YARN-305) | Fair scheduler logs too many "Node offered to app:..." messages | Critical | resourcemanager | Lohit Vijayarenu | Lohit Vijayarenu | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10207](https://issues.apache.org/jira/browse/HADOOP-10207) | TestUserGroupInformation#testLogin is flaky | Minor | . | Jimmy Xiang | Jimmy Xiang | +| [HADOOP-9866](https://issues.apache.org/jira/browse/HADOOP-9866) | convert hadoop-auth testcases requiring kerberos to use minikdc | Major | test | Alejandro Abdelnur | Wei Yan | +| [HADOOP-9598](https://issues.apache.org/jira/browse/HADOOP-9598) | Improve code coverage of RMAdminCLI | Major | . | Aleksey Gorshkov | Andrey Klochkov | +| [HADOOP-9291](https://issues.apache.org/jira/browse/HADOOP-9291) | enhance unit-test coverage of package o.a.h.metrics2 | Major | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9254](https://issues.apache.org/jira/browse/HADOOP-9254) | Cover packages org.apache.hadoop.util.bloom, org.apache.hadoop.util.hash | Major | . | Vadim Bondarev | Vadim Bondarev | +| [HADOOP-9225](https://issues.apache.org/jira/browse/HADOOP-9225) | Cover package org.apache.hadoop.compress.Snappy | Major | . | Vadim Bondarev | Andrey Klochkov | +| [HADOOP-9199](https://issues.apache.org/jira/browse/HADOOP-9199) | Cover package org.apache.hadoop.io with unit tests | Major | . | Vadim Bondarev | Andrey Klochkov | +| [HADOOP-9078](https://issues.apache.org/jira/browse/HADOOP-9078) | enhance unit-test coverage of class org.apache.hadoop.fs.FileContext | Major | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HADOOP-9063](https://issues.apache.org/jira/browse/HADOOP-9063) | enhance unit-test coverage of class org.apache.hadoop.fs.FileUtil | Minor | . | Ivan A. Veselovsky | Ivan A. Veselovsky | +| [HDFS-5679](https://issues.apache.org/jira/browse/HDFS-5679) | TestCacheDirectives should handle the case where native code is not available | Major | . | Ted Yu | Andrew Wang | +| [HDFS-5130](https://issues.apache.org/jira/browse/HDFS-5130) | Add test for snapshot related FsShell and DFSAdmin commands | Minor | test | Binglin Chang | Binglin Chang | +| [HDFS-4517](https://issues.apache.org/jira/browse/HDFS-4517) | Cover class RemoteBlockReader with unit tests | Major | . | Vadim Bondarev | Ivan A. Veselovsky | +| [HDFS-4512](https://issues.apache.org/jira/browse/HDFS-4512) | Cover package org.apache.hadoop.hdfs.server.common with tests | Major | . | Vadim Bondarev | Vadim Bondarev | +| [HDFS-4511](https://issues.apache.org/jira/browse/HDFS-4511) | Cover package org.apache.hadoop.hdfs.tools with unit test | Major | . | Vadim Bondarev | Andrey Klochkov | +| [HDFS-4510](https://issues.apache.org/jira/browse/HDFS-4510) | Cover classes ClusterJspHelper/NamenodeJspHelper with unit tests | Major | . | Vadim Bondarev | Andrey Klochkov | +| [HDFS-4491](https://issues.apache.org/jira/browse/HDFS-4491) | Parallel testing HDFS | Major | test | Tsuyoshi Ozawa | Andrey Klochkov | +| [MAPREDUCE-5687](https://issues.apache.org/jira/browse/MAPREDUCE-5687) | TestYARNRunner#testResourceMgrDelegate fails with NPE after YARN-1446 | Major | . | Ted Yu | Jian He | +| [MAPREDUCE-5632](https://issues.apache.org/jira/browse/MAPREDUCE-5632) | TestRMContainerAllocator#testUpdatedNodes fails | Major | . | Ted Yu | Jonathan Eagles | +| [MAPREDUCE-5625](https://issues.apache.org/jira/browse/MAPREDUCE-5625) | TestFixedLengthInputFormat fails in jdk7 environment | Major | . | Jonathan Eagles | Mariappan Asokan | +| [MAPREDUCE-5610](https://issues.apache.org/jira/browse/MAPREDUCE-5610) | TestSleepJob fails in jdk7 | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-5102](https://issues.apache.org/jira/browse/MAPREDUCE-5102) | fix coverage org.apache.hadoop.mapreduce.lib.db and org.apache.hadoop.mapred.lib.db | Major | . | Aleksey Gorshkov | Andrey Klochkov | +| [MAPREDUCE-5084](https://issues.apache.org/jira/browse/MAPREDUCE-5084) | fix coverage org.apache.hadoop.mapreduce.v2.app.webapp and org.apache.hadoop.mapreduce.v2.hs.webapp | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-1560](https://issues.apache.org/jira/browse/YARN-1560) | TestYarnClient#testAMMRTokens fails with null AMRM token | Major | . | Ted Yu | Ted Yu | +| [YARN-1549](https://issues.apache.org/jira/browse/YARN-1549) | TestUnmanagedAMLauncher#testDSShell fails in trunk | Major | . | Ted Yu | haosdent | +| [YARN-1463](https://issues.apache.org/jira/browse/YARN-1463) | Tests should avoid starting http-server where possible or creates spnego keytab/principals | Major | . | Ted Yu | Vinod Kumar Vavilapalli | +| [YARN-1358](https://issues.apache.org/jira/browse/YARN-1358) | TestYarnCLI fails on Windows due to line endings | Minor | client | Chuan Liu | Chuan Liu | +| [YARN-1357](https://issues.apache.org/jira/browse/YARN-1357) | TestContainerLaunch.testContainerEnvVariables fails on Windows | Minor | nodemanager | Chuan Liu | Chuan Liu | +| [YARN-1119](https://issues.apache.org/jira/browse/YARN-1119) | Add ClusterMetrics checks to tho TestRMNodeTransitions tests | Major | resourcemanager | Robert Parker | Mit Desai | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10109](https://issues.apache.org/jira/browse/HADOOP-10109) | Fix test failure in TestOfflineEditsViewer introduced by HADOOP-10052 | Major | test | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-10107](https://issues.apache.org/jira/browse/HADOOP-10107) | Server.getNumOpenConnections may throw NPE | Major | ipc | Tsz Wo Nicholas Sze | Kihwal Lee | +| [HADOOP-10103](https://issues.apache.org/jira/browse/HADOOP-10103) | update commons-lang to 2.6 | Minor | build | Steve Loughran | Akira AJISAKA | +| [HADOOP-10102](https://issues.apache.org/jira/browse/HADOOP-10102) | update commons IO from 2.1 to 2.4 | Minor | build | Steve Loughran | Akira AJISAKA | +| [HADOOP-10052](https://issues.apache.org/jira/browse/HADOOP-10052) | Temporarily disable client-side symlink resolution | Major | fs | Andrew Wang | Andrew Wang | +| [HADOOP-9956](https://issues.apache.org/jira/browse/HADOOP-9956) | RPC listener inefficiently assigns connections to readers | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9955](https://issues.apache.org/jira/browse/HADOOP-9955) | RPC idle connection closing is extremely inefficient | Major | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-9748](https://issues.apache.org/jira/browse/HADOOP-9748) | Reduce blocking on UGI.ensureInitialized | Critical | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-9417](https://issues.apache.org/jira/browse/HADOOP-9417) | Support for symlink resolution in LocalFileSystem / RawLocalFileSystem | Major | fs | Andrew Wang | Andrew Wang | +| [HDFS-5784](https://issues.apache.org/jira/browse/HDFS-5784) | reserve space in edit log header and fsimage header for feature flag section | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5708](https://issues.apache.org/jira/browse/HDFS-5708) | The CacheManager throws a NPE in the DataNode logs when processing cache reports that refer to a block not known to the BlockManager | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5667](https://issues.apache.org/jira/browse/HDFS-5667) | Include DatanodeStorage in StorageReport | Major | datanode | Eric Sirianni | Arpit Agarwal | +| [HDFS-5651](https://issues.apache.org/jira/browse/HDFS-5651) | Remove dfs.namenode.caching.enabled and improve CRM locking | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5636](https://issues.apache.org/jira/browse/HDFS-5636) | Enforce a max TTL per cache pool | Major | caching, namenode | Andrew Wang | Andrew Wang | +| [HDFS-5634](https://issues.apache.org/jira/browse/HDFS-5634) | allow BlockReaderLocal to switch between checksumming and not | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5630](https://issues.apache.org/jira/browse/HDFS-5630) | Hook up cache directive and pool usage statistics | Major | caching, namenode | Andrew Wang | Andrew Wang | +| [HDFS-5629](https://issues.apache.org/jira/browse/HDFS-5629) | Support HTTPS in JournalNode and SecondaryNameNode | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5589](https://issues.apache.org/jira/browse/HDFS-5589) | Namenode loops caching and uncaching when data should be uncached | Major | caching, namenode | Andrew Wang | Andrew Wang | +| [HDFS-5565](https://issues.apache.org/jira/browse/HDFS-5565) | CacheAdmin help should match against non-dashed commands | Minor | caching | Andrew Wang | Andrew Wang | +| [HDFS-5556](https://issues.apache.org/jira/browse/HDFS-5556) | add some more NameNode cache statistics, cache pool stats | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5545](https://issues.apache.org/jira/browse/HDFS-5545) | Allow specifying endpoints for listeners in HttpServer | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5543](https://issues.apache.org/jira/browse/HDFS-5543) | fix narrow race condition in TestPathBasedCacheRequests | Major | test | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5538](https://issues.apache.org/jira/browse/HDFS-5538) | URLConnectionFactory should pick up the SSL related configuration by default | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5536](https://issues.apache.org/jira/browse/HDFS-5536) | Implement HTTP policy for Namenode and DataNode | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5525](https://issues.apache.org/jira/browse/HDFS-5525) | Inline dust templates | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5520](https://issues.apache.org/jira/browse/HDFS-5520) | loading cache path directives from edit log doesn't update nextEntryId | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5519](https://issues.apache.org/jira/browse/HDFS-5519) | COMMIT handler should update the commit status after sync | Minor | nfs | Brandon Li | Brandon Li | +| [HDFS-5514](https://issues.apache.org/jira/browse/HDFS-5514) | FSNamesystem's fsLock should allow custom implementation | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-5511](https://issues.apache.org/jira/browse/HDFS-5511) | improve CacheManipulator interface to allow better unit testing | Major | datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5509](https://issues.apache.org/jira/browse/HDFS-5509) | TestPathBasedCacheRequests#testReplicationFactor is flaky | Major | datanode, namenode | Andrew Wang | Andrew Wang | +| [HDFS-5506](https://issues.apache.org/jira/browse/HDFS-5506) | Use URLConnectionFactory in DelegationTokenFetcher | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5502](https://issues.apache.org/jira/browse/HDFS-5502) | Fix HTTPS support in HsftpFileSystem | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5489](https://issues.apache.org/jira/browse/HDFS-5489) | Use TokenAspect in WebHDFSFileSystem | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5488](https://issues.apache.org/jira/browse/HDFS-5488) | Clean up TestHftpURLTimeout | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5487](https://issues.apache.org/jira/browse/HDFS-5487) | Introduce unit test for TokenAspect | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5473](https://issues.apache.org/jira/browse/HDFS-5473) | Consistent naming of user-visible caching classes and methods | Major | datanode, namenode | Andrew Wang | Colin Patrick McCabe | +| [HDFS-5469](https://issues.apache.org/jira/browse/HDFS-5469) | Add configuration property for the sub-directroy export path | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5454](https://issues.apache.org/jira/browse/HDFS-5454) | DataNode UUID should be assigned prior to FsDataset initialization | Minor | datanode | Eric Sirianni | Arpit Agarwal | +| [HDFS-5451](https://issues.apache.org/jira/browse/HDFS-5451) | Add byte and file statistics to PathBasedCacheEntry | Major | datanode, namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5450](https://issues.apache.org/jira/browse/HDFS-5450) | better API for getting the cached blocks locations | Minor | hdfs-client | Colin Patrick McCabe | Andrew Wang | +| [HDFS-5444](https://issues.apache.org/jira/browse/HDFS-5444) | Choose default web UI based on browser capabilities | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5440](https://issues.apache.org/jira/browse/HDFS-5440) | Extract the logic of handling delegation tokens in HftpFileSystem to the TokenAspect class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5436](https://issues.apache.org/jira/browse/HDFS-5436) | Move HsFtpFileSystem and HFtpFileSystem into org.apache.hdfs.web | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5431](https://issues.apache.org/jira/browse/HDFS-5431) | support cachepool-based limit management in path-based caching | Major | datanode, namenode | Colin Patrick McCabe | Andrew Wang | +| [HDFS-5430](https://issues.apache.org/jira/browse/HDFS-5430) | Support TTL on CacheDirectives | Minor | datanode, namenode | Colin Patrick McCabe | Andrew Wang | +| [HDFS-5406](https://issues.apache.org/jira/browse/HDFS-5406) | Send incremental block reports for all storages in a single call | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-5394](https://issues.apache.org/jira/browse/HDFS-5394) | fix race conditions in DN caching and uncaching | Major | datanode, namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5393](https://issues.apache.org/jira/browse/HDFS-5393) | Serve bootstrap and jQuery locally | Minor | . | Haohui Mai | Haohui Mai | +| [HDFS-5386](https://issues.apache.org/jira/browse/HDFS-5386) | Add feature documentation for datanode caching. | Major | documentation | Chris Nauroth | Colin Patrick McCabe | +| [HDFS-5382](https://issues.apache.org/jira/browse/HDFS-5382) | Implement the UI of browsing filesystems in HTML 5 page | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5379](https://issues.apache.org/jira/browse/HDFS-5379) | Update links to datanode information in dfshealth.html | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5366](https://issues.apache.org/jira/browse/HDFS-5366) | recaching improvements | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5364](https://issues.apache.org/jira/browse/HDFS-5364) | Add OpenFileCtx cache | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5363](https://issues.apache.org/jira/browse/HDFS-5363) | Refactor WebHdfsFileSystem: move SPENGO-authenticated connection creation to URLConnectionFactory | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5347](https://issues.apache.org/jira/browse/HDFS-5347) | add HDFS NFS user guide | Major | documentation | Brandon Li | Brandon Li | +| [HDFS-5342](https://issues.apache.org/jira/browse/HDFS-5342) | Provide more information in the FSNamesystem JMX interfaces | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5337](https://issues.apache.org/jira/browse/HDFS-5337) | should do hsync for a commit request even there is no pending writes | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5334](https://issues.apache.org/jira/browse/HDFS-5334) | Implement dfshealth.jsp in HTML pages | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5330](https://issues.apache.org/jira/browse/HDFS-5330) | fix readdir and readdirplus for large directories | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5325](https://issues.apache.org/jira/browse/HDFS-5325) | Remove WebHdfsFileSystem#ConnRunner | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5320](https://issues.apache.org/jira/browse/HDFS-5320) | Add datanode caching metrics | Minor | datanode | Andrew Wang | Andrew Wang | +| [HDFS-5317](https://issues.apache.org/jira/browse/HDFS-5317) | Go back to DFS Home link does not work on datanode webUI | Critical | . | Suresh Srinivas | Haohui Mai | +| [HDFS-5316](https://issues.apache.org/jira/browse/HDFS-5316) | Namenode ignores the default https port | Critical | . | Suresh Srinivas | Haohui Mai | +| [HDFS-5312](https://issues.apache.org/jira/browse/HDFS-5312) | Generate HTTP / HTTPS URL in DFSUtil#getInfoServer() based on the configured http policy | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5307](https://issues.apache.org/jira/browse/HDFS-5307) | Support both HTTP and HTTPS in jsp pages | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5288](https://issues.apache.org/jira/browse/HDFS-5288) | Close idle connections in portmap | Major | nfs | Haohui Mai | Haohui Mai | +| [HDFS-5281](https://issues.apache.org/jira/browse/HDFS-5281) | COMMIT request should not block | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5252](https://issues.apache.org/jira/browse/HDFS-5252) | Stable write is not handled correctly in someplace | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-5241](https://issues.apache.org/jira/browse/HDFS-5241) | Provide alternate queuing audit logger to reduce logging contention | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-5240](https://issues.apache.org/jira/browse/HDFS-5240) | Separate formatting from logging in the audit logger API | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-5239](https://issues.apache.org/jira/browse/HDFS-5239) | Allow FSNamesystem lock fairness to be configurable | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-5171](https://issues.apache.org/jira/browse/HDFS-5171) | NFS should create input stream for a file and try to share it with multiple read requests | Major | nfs | Brandon Li | Haohui Mai | +| [HDFS-4885](https://issues.apache.org/jira/browse/HDFS-4885) | Update verifyBlockPlacement() API in BlockPlacementPolicy | Major | . | Junping Du | Junping Du | +| [HDFS-4096](https://issues.apache.org/jira/browse/HDFS-4096) | Add snapshot information to namenode WebUI | Major | datanode, namenode | Jing Zhao | Haohui Mai | +| [HDFS-3987](https://issues.apache.org/jira/browse/HDFS-3987) | Support webhdfs over HTTPS | Major | . | Alejandro Abdelnur | Haohui Mai | +| [MAPREDUCE-5411](https://issues.apache.org/jira/browse/MAPREDUCE-5411) | Refresh size of loaded job cache on history server | Major | jobhistoryserver | Ashwin Shankar | Ashwin Shankar | +| [MAPREDUCE-5409](https://issues.apache.org/jira/browse/MAPREDUCE-5409) | MRAppMaster throws InvalidStateTransitonException: Invalid event: TA\_TOO\_MANY\_FETCH\_FAILURE at KILLED for TaskAttemptImpl | Major | . | Devaraj K | Gera Shegalov | +| [MAPREDUCE-5386](https://issues.apache.org/jira/browse/MAPREDUCE-5386) | Ability to refresh history server job retention and job cleaner settings | Major | jobhistoryserver | Ashwin Shankar | Ashwin Shankar | +| [MAPREDUCE-5356](https://issues.apache.org/jira/browse/MAPREDUCE-5356) | Ability to refresh aggregated log retention period and check interval | Major | jobhistoryserver | Ashwin Shankar | Ashwin Shankar | +| [MAPREDUCE-4421](https://issues.apache.org/jira/browse/MAPREDUCE-4421) | Run MapReduce framework via the distributed cache | Major | . | Arun C Murthy | Jason Lowe | +| [YARN-1642](https://issues.apache.org/jira/browse/YARN-1642) | RMDTRenewer#getRMClient should use ClientRMProxy | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1618](https://issues.apache.org/jira/browse/YARN-1618) | Fix invalid RMApp transition from NEW to FINAL\_SAVING | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1598](https://issues.apache.org/jira/browse/YARN-1598) | HA-related rmadmin commands don't work on a secure cluster | Critical | client, resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1579](https://issues.apache.org/jira/browse/YARN-1579) | ActiveRMInfoProto fields should be optional | Trivial | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1575](https://issues.apache.org/jira/browse/YARN-1575) | Public localizer crashes with "Localized unkown resource" | Critical | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1574](https://issues.apache.org/jira/browse/YARN-1574) | RMDispatcher should be reset on transition to standby | Blocker | . | Xuan Gong | Xuan Gong | +| [YARN-1573](https://issues.apache.org/jira/browse/YARN-1573) | ZK store should use a private password for root-node-acls | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1559](https://issues.apache.org/jira/browse/YARN-1559) | Race between ServerRMProxy and ClientRMProxy setting RMProxy#INSTANCE | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1523](https://issues.apache.org/jira/browse/YARN-1523) | Use StandbyException instead of RMNotYetReadyException | Major | . | Bikas Saha | Karthik Kambatla | +| [YARN-1485](https://issues.apache.org/jira/browse/YARN-1485) | Enabling HA should verify the RM service addresses configurations have been set for every RM Ids defined in RM\_HA\_IDs | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1482](https://issues.apache.org/jira/browse/YARN-1482) | WebApplicationProxy should be always-on w.r.t HA even if it is embedded in the RM | Major | . | Vinod Kumar Vavilapalli | Xuan Gong | +| [YARN-1481](https://issues.apache.org/jira/browse/YARN-1481) | Move internal services logic from AdminService to ResourceManager | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1448](https://issues.apache.org/jira/browse/YARN-1448) | AM-RM protocol changes to support container resizing | Major | api, resourcemanager | Wangda Tan (No longer used) | Wangda Tan | +| [YARN-1447](https://issues.apache.org/jira/browse/YARN-1447) | Common PB type definitions for container resizing | Major | api | Wangda Tan (No longer used) | Wangda Tan | +| [YARN-1446](https://issues.apache.org/jira/browse/YARN-1446) | Change killing application to wait until state store is done | Major | resourcemanager | Jian He | Jian He | +| [YARN-1411](https://issues.apache.org/jira/browse/YARN-1411) | HA config shouldn't affect NodeManager RPC addresses | Critical | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-1405](https://issues.apache.org/jira/browse/YARN-1405) | RM hangs on shutdown if calling system.exit in serviceInit or serviceStart | Major | . | Yesha Vora | Jian He | +| [YARN-1378](https://issues.apache.org/jira/browse/YARN-1378) | Implement a RMStateStore cleaner for deleting application/attempt info | Major | resourcemanager | Jian He | Jian He | +| [YARN-1325](https://issues.apache.org/jira/browse/YARN-1325) | Enabling HA should check Configuration contains multiple RMs | Major | resourcemanager | Tsuyoshi Ozawa | Xuan Gong | +| [YARN-1323](https://issues.apache.org/jira/browse/YARN-1323) | Set HTTPS webapp address along with other RPC addresses in HAUtil | Major | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-1318](https://issues.apache.org/jira/browse/YARN-1318) | Promote AdminService to an Always-On service and merge in RMHAProtocolService | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1311](https://issues.apache.org/jira/browse/YARN-1311) | Fix app specific scheduler-events' names to be app-attempt based | Trivial | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1307](https://issues.apache.org/jira/browse/YARN-1307) | Rethink znode structure for RM HA | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-1305](https://issues.apache.org/jira/browse/YARN-1305) | RMHAProtocolService#serviceInit should handle HAUtil's IllegalArgumentException | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-1283](https://issues.apache.org/jira/browse/YARN-1283) | Invalid 'url of job' mentioned in Job output with yarn.http.policy=HTTPS\_ONLY | Major | . | Yesha Vora | Omkar Vinit Joshi | +| [YARN-1239](https://issues.apache.org/jira/browse/YARN-1239) | Save version information in the state store | Major | resourcemanager | Bikas Saha | Jian He | +| [YARN-1232](https://issues.apache.org/jira/browse/YARN-1232) | Configuration to support multiple RMs | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1222](https://issues.apache.org/jira/browse/YARN-1222) | Make improvements in ZKRMStateStore for fencing | Major | . | Bikas Saha | Karthik Kambatla | +| [YARN-1210](https://issues.apache.org/jira/browse/YARN-1210) | During RM restart, RM should start a new attempt only when previous attempt exits for real | Major | . | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-1185](https://issues.apache.org/jira/browse/YARN-1185) | FileSystemRMStateStore can leave partial files that prevent subsequent recovery | Major | resourcemanager | Jason Lowe | Omkar Vinit Joshi | +| [YARN-1181](https://issues.apache.org/jira/browse/YARN-1181) | Augment MiniYARNCluster to support HA mode | Major | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-1172](https://issues.apache.org/jira/browse/YARN-1172) | Convert \*SecretManagers in the RM to services | Major | resourcemanager | Karthik Kambatla | Tsuyoshi Ozawa | +| [YARN-1121](https://issues.apache.org/jira/browse/YARN-1121) | RMStateStore should flush all pending store events before closing | Major | resourcemanager | Bikas Saha | Jian He | +| [YARN-1098](https://issues.apache.org/jira/browse/YARN-1098) | Separate out RM services into "Always On" and "Active" | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1068](https://issues.apache.org/jira/browse/YARN-1068) | Add admin support for HA operations | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1033](https://issues.apache.org/jira/browse/YARN-1033) | Expose RM active/standby state to Web UI and REST API | Major | . | Nemon Lou | Karthik Kambatla | +| [YARN-1029](https://issues.apache.org/jira/browse/YARN-1029) | Allow embedding leader election into the RM | Major | . | Bikas Saha | Karthik Kambatla | +| [YARN-1028](https://issues.apache.org/jira/browse/YARN-1028) | Add FailoverProxyProvider like capability to RMProxy | Major | . | Bikas Saha | Karthik Kambatla | +| [YARN-1027](https://issues.apache.org/jira/browse/YARN-1027) | Implement RMHAProtocolService | Major | . | Bikas Saha | Karthik Kambatla | +| [YARN-976](https://issues.apache.org/jira/browse/YARN-976) | Document the meaning of a virtual core | Major | documentation | Sandy Ryza | Sandy Ryza | +| [YARN-895](https://issues.apache.org/jira/browse/YARN-895) | RM crashes if it restarts while the state-store is down | Major | resourcemanager | Jian He | Jian He | +| [YARN-891](https://issues.apache.org/jira/browse/YARN-891) | Store completed application information in RM state store | Major | resourcemanager | Bikas Saha | Jian He | +| [YARN-819](https://issues.apache.org/jira/browse/YARN-819) | ResourceManager and NodeManager should check for a minimum allowed version | Major | nodemanager, resourcemanager | Robert Parker | Robert Parker | +| [YARN-709](https://issues.apache.org/jira/browse/YARN-709) | verify that new jobs submitted with old RM delegation tokens after RM restart are accepted | Major | resourcemanager | Jian He | Jian He | +| [YARN-674](https://issues.apache.org/jira/browse/YARN-674) | Slow or failing DelegationToken renewals on submission itself make RM unavailable | Major | resourcemanager | Vinod Kumar Vavilapalli | Omkar Vinit Joshi | +| [YARN-649](https://issues.apache.org/jira/browse/YARN-649) | Make container logs available over HTTP in plain text | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-478](https://issues.apache.org/jira/browse/YARN-478) | fix coverage org.apache.hadoop.yarn.webapp.log | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-465](https://issues.apache.org/jira/browse/YARN-465) | fix coverage org.apache.hadoop.yarn.server.webproxy | Major | . | Aleksey Gorshkov | Andrey Klochkov | +| [YARN-427](https://issues.apache.org/jira/browse/YARN-427) | Coverage fix for org.apache.hadoop.yarn.server.api.\* | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-425](https://issues.apache.org/jira/browse/YARN-425) | coverage fix for yarn api | Major | . | Aleksey Gorshkov | Aleksey Gorshkov | +| [YARN-353](https://issues.apache.org/jira/browse/YARN-353) | Add Zookeeper-based store implementation for RMStateStore | Major | resourcemanager | Hitesh Shah | Karthik Kambatla | +| [YARN-312](https://issues.apache.org/jira/browse/YARN-312) | Add updateNodeResource in ResourceManagerAdministrationProtocol | Major | api | Junping Du | Junping Du | +| [YARN-311](https://issues.apache.org/jira/browse/YARN-311) | Dynamic node resource configuration: core scheduler changes | Major | resourcemanager, scheduler | Junping Du | Junping Du | +| [YARN-7](https://issues.apache.org/jira/browse/YARN-7) | Add support for DistributedShell to ask for CPUs along with memory | Major | . | Arun C Murthy | Junping Du | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [MAPREDUCE-5464](https://issues.apache.org/jira/browse/MAPREDUCE-5464) | Add analogs of the SLOTS\_MILLIS counters that jive with the YARN resource model | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5463](https://issues.apache.org/jira/browse/MAPREDUCE-5463) | Deprecate SLOTS\_MILLIS counters | Major | . | Sandy Ryza | Tsuyoshi Ozawa | +| [YARN-1568](https://issues.apache.org/jira/browse/YARN-1568) | Rename clusterid to clusterId in ActiveRMInfoProto | Trivial | resourcemanager | Karthik Kambatla | Karthik Kambatla | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/RELEASENOTES.2.3.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/RELEASENOTES.2.3.0.md new file mode 100644 index 00000000000..103cf9641c9 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.3.0/RELEASENOTES.2.3.0.md @@ -0,0 +1,138 @@ + + +# Apache Hadoop 2.3.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10047](https://issues.apache.org/jira/browse/HADOOP-10047) | *Major* | **Add a directbuffer Decompressor API to hadoop** + +Direct Bytebuffer decompressors for Zlib (Deflate & Gzip) and Snappy + + +--- + +* [HADOOP-9241](https://issues.apache.org/jira/browse/HADOOP-9241) | *Trivial* | **DU refresh interval is not configurable** + +The 'du' (disk usage command from Unix) script refresh monitor is now configurable in the same way as its 'df' counterpart, via the property 'fs.du.interval', the default of which is 10 minute (in ms). + + +--- + +* [HADOOP-8545](https://issues.apache.org/jira/browse/HADOOP-8545) | *Major* | **Filesystem Implementation for OpenStack Swift** + +Added file system implementation for OpenStack Swift. +There are two implementation: block and native (similar to Amazon S3 integration). +Data locality issue solved by patch in Swift, commit procedure to OpenStack is in progress. + +To use implementation add to core-site.xml following: + +```xml + + fs.swift.impl + com.mirantis.fs.SwiftFileSystem + + + fs.swift.block.impl + com.mirantis.fs.block.SwiftBlockFileSystem + +``` + +In MapReduce job specify following configs for OpenStack Keystone authentication: + +```java +conf.set("swift.auth.url", "http://172.18.66.117:5000/v2.0/tokens"); +conf.set("swift.tenant", "superuser"); +conf.set("swift.username", "admin1"); +conf.set("swift.password", "password"); +conf.setInt("swift.http.port", 8080); +conf.setInt("swift.https.port", 443); +``` + +Additional information specified on github: https://github.com/DmitryMezhensky/Hadoop-and-Swift-integration + + +--- + +* [HDFS-5704](https://issues.apache.org/jira/browse/HDFS-5704) | *Major* | **Change OP\_UPDATE\_BLOCKS with a new OP\_ADD\_BLOCK** + +Add a new editlog record (OP\_ADD\_BLOCK) that only records allocation of the new block instead of the entire block list, on every block allocation. + + +--- + +* [HDFS-5663](https://issues.apache.org/jira/browse/HDFS-5663) | *Major* | **make the retry time and interval value configurable in openInfo()** + +Makes the retries and time between retries getting the length of the last block on file configurable. Below are the new configurations. + +dfs.client.retry.times.get-last-block-length +dfs.client.retry.interval-ms.get-last-block-length + +They are set to the 3 and 4000 respectively, these being what was previously hardcoded. + + +--- + +* [HDFS-5536](https://issues.apache.org/jira/browse/HDFS-5536) | *Major* | **Implement HTTP policy for Namenode and DataNode** + +Add new HTTP policy configuration. Users can use "dfs.http.policy" to control the HTTP endpoints for NameNode and DataNode. Specifically, The following values are supported: +- HTTP\_ONLY : Service is provided only on http +- HTTPS\_ONLY : Service is provided only on https +- HTTP\_AND\_HTTPS : Service is provided both on http and https + +hadoop.ssl.enabled and dfs.https.enabled are deprecated. When the deprecated configuration properties are still configured, currently http policy is decided based on the following rules: +1. If dfs.http.policy is set to HTTPS\_ONLY or HTTP\_AND\_HTTPS. It picks the specified policy, otherwise it proceeds to 2~4. +2. It picks HTTPS\_ONLY if hadoop.ssl.enabled equals to true. +3. It picks HTTP\_AND\_HTTPS if dfs.https.enable equals to true. +4. It picks HTTP\_ONLY for other configurations. + + +--- + +* [HDFS-5502](https://issues.apache.org/jira/browse/HDFS-5502) | *Major* | **Fix HTTPS support in HsftpFileSystem** + +Fix the https support in HsftpFileSystem. With the change the client now verifies the server certificate. In particular, client side will verify the Common Name of the certificate using a strategy specified by the configuration property "hadoop.ssl.hostname.verifier". + + +--- + +* [HDFS-4997](https://issues.apache.org/jira/browse/HDFS-4997) | *Major* | **libhdfs doesn't return correct error codes in most cases** + +libhdfs now returns correct codes in errno. Previously, due to a bug, many functions set errno to 255 instead of the more specific error code. + + +--- + +* [HDFS-4983](https://issues.apache.org/jira/browse/HDFS-4983) | *Major* | **Numeric usernames do not work with WebHDFS FS** + +Add a new configuration property "dfs.webhdfs.user.provider.user.pattern" for specifying user name filters for WebHDFS. + + +--- + +* [MAPREDUCE-1176](https://issues.apache.org/jira/browse/MAPREDUCE-1176) | *Major* | **FixedLengthInputFormat and FixedLengthRecordReader** + +Addition of FixedLengthInputFormat and FixedLengthRecordReader in the org.apache.hadoop.mapreduce.lib.input package. These two classes can be used when you need to read data from files containing fixed length (fixed width) records. Such files have no CR/LF (or any combination thereof), no delimiters etc, but each record is a fixed length, and extra data is padded with spaces. The data is one gigantic line within a file. When creating a job that specifies this input format, the job must have the "mapreduce.input.fixedlengthinputformat.record.length" property set as follows myJobConf.setInt("mapreduce.input.fixedlengthinputformat.record.length",[myFixedRecordLength]); + +Please see javadoc for more details. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/CHANGES.2.4.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/CHANGES.2.4.0.md new file mode 100644 index 00000000000..78bc092b981 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/CHANGES.2.4.0.md @@ -0,0 +1,486 @@ + + +# Apache Hadoop Changelog + +## Release 2.4.0 - 2014-04-07 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8691](https://issues.apache.org/jira/browse/HADOOP-8691) | FsShell can print "Found xxx items" unnecessarily often | Minor | fs | Jason Lowe | Daryn Sharp | +| [HDFS-6102](https://issues.apache.org/jira/browse/HDFS-6102) | Lower the default maximum items per directory to fix PB fsimage loading | Blocker | namenode | Andrew Wang | Andrew Wang | +| [HDFS-6055](https://issues.apache.org/jira/browse/HDFS-6055) | Change default configuration to limit file name length in HDFS | Major | namenode | Suresh Srinivas | Chris Nauroth | +| [HDFS-5804](https://issues.apache.org/jira/browse/HDFS-5804) | HDFS NFS Gateway fails to mount and proxy when using Kerberos | Major | nfs | Abin Shahab | Abin Shahab | +| [HDFS-5321](https://issues.apache.org/jira/browse/HDFS-5321) | Clean up the HTTP-related configuration in HDFS | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5138](https://issues.apache.org/jira/browse/HDFS-5138) | Support HDFS upgrade in HA | Blocker | . | Kihwal Lee | Aaron T. Myers | +| [MAPREDUCE-5036](https://issues.apache.org/jira/browse/MAPREDUCE-5036) | Default shuffle handler port should not be 8080 | Major | . | Sandy Ryza | Sandy Ryza | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10184](https://issues.apache.org/jira/browse/HADOOP-10184) | Hadoop Common changes required to support HDFS ACLs. | Major | fs, security | Chris Nauroth | Chris Nauroth | +| [HDFS-5535](https://issues.apache.org/jira/browse/HDFS-5535) | Umbrella jira for improved HDFS rolling upgrades | Major | datanode, ha, hdfs-client, namenode | Nathan Roberts | Tsz Wo Nicholas Sze | +| [HDFS-4685](https://issues.apache.org/jira/browse/HDFS-4685) | Implementation of ACLs in HDFS | Major | hdfs-client, namenode, security | Sachin Jose | Chris Nauroth | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10423](https://issues.apache.org/jira/browse/HADOOP-10423) | Clarify compatibility policy document for combination of new client and old server. | Minor | documentation | Chris Nauroth | Chris Nauroth | +| [HADOOP-10386](https://issues.apache.org/jira/browse/HADOOP-10386) | Log proxy hostname in various exceptions being thrown in a HA setup | Minor | ha | Arpit Gupta | Haohui Mai | +| [HADOOP-10383](https://issues.apache.org/jira/browse/HADOOP-10383) | InterfaceStability annotations should have RetentionPolicy.RUNTIME | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-10379](https://issues.apache.org/jira/browse/HADOOP-10379) | Protect authentication cookies with the HttpOnly and Secure flags | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-10374](https://issues.apache.org/jira/browse/HADOOP-10374) | InterfaceAudience annotations should have RetentionPolicy.RUNTIME | Major | . | Enis Soztutar | Enis Soztutar | +| [HADOOP-10348](https://issues.apache.org/jira/browse/HADOOP-10348) | Deprecate hadoop.ssl.configuration in branch-2, and remove it in trunk | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-10343](https://issues.apache.org/jira/browse/HADOOP-10343) | Change info to debug log in LossyRetryInvocationHandler | Minor | . | Arpit Gupta | Arpit Gupta | +| [HADOOP-10333](https://issues.apache.org/jira/browse/HADOOP-10333) | Fix grammatical error in overview.html document | Trivial | . | RenĂ© Nyffenegger | RenĂ© Nyffenegger | +| [HADOOP-10295](https://issues.apache.org/jira/browse/HADOOP-10295) | Allow distcp to automatically identify the checksum type of source files and use it for the target | Major | tools/distcp | Jing Zhao | Jing Zhao | +| [HADOOP-10221](https://issues.apache.org/jira/browse/HADOOP-10221) | Add a plugin to specify SaslProperties for RPC protocol based on connection properties | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10211](https://issues.apache.org/jira/browse/HADOOP-10211) | Enable RPC protocol to negotiate SASL-QOP values between clients and servers | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10139](https://issues.apache.org/jira/browse/HADOOP-10139) | Update and improve the Single Cluster Setup document | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-9454](https://issues.apache.org/jira/browse/HADOOP-9454) | Support multipart uploads for s3native | Major | fs/s3 | Jordan Mendelson | Akira AJISAKA | +| [HDFS-6150](https://issues.apache.org/jira/browse/HDFS-6150) | Add inode id information in the logs to make debugging easier | Minor | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6138](https://issues.apache.org/jira/browse/HDFS-6138) | User Guide for how to use viewfs with federation | Minor | documentation | Sanjay Radia | Sanjay Radia | +| [HDFS-6123](https://issues.apache.org/jira/browse/HDFS-6123) | Improve datanode error messages | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6120](https://issues.apache.org/jira/browse/HDFS-6120) | Fix and improve safe mode log messages | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6090](https://issues.apache.org/jira/browse/HDFS-6090) | Use MiniDFSCluster.Builder instead of deprecated constructors | Minor | test | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6085](https://issues.apache.org/jira/browse/HDFS-6085) | Improve CacheReplicationMonitor log messages a bit | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6084](https://issues.apache.org/jira/browse/HDFS-6084) | Namenode UI - "Hadoop" logo link shouldn't go to hadoop homepage | Minor | namenode | Travis Thompson | Travis Thompson | +| [HDFS-6080](https://issues.apache.org/jira/browse/HDFS-6080) | Improve NFS gateway performance by making rtmax and wtmax configurable | Major | nfs, performance | Abin Shahab | Abin Shahab | +| [HDFS-6072](https://issues.apache.org/jira/browse/HDFS-6072) | Clean up dead code of FSImage | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6070](https://issues.apache.org/jira/browse/HDFS-6070) | Cleanup use of ReadStatistics in DFSInputStream | Trivial | . | Andrew Wang | Andrew Wang | +| [HDFS-6069](https://issues.apache.org/jira/browse/HDFS-6069) | Quash stack traces when ACLs are disabled | Trivial | namenode | Andrew Wang | Chris Nauroth | +| [HDFS-6044](https://issues.apache.org/jira/browse/HDFS-6044) | Add property for setting the NFS look up time for users | Minor | nfs | Brandon Li | Brandon Li | +| [HDFS-6043](https://issues.apache.org/jira/browse/HDFS-6043) | Give HDFS daemons NFS3 and Portmap their own OPTS | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-6018](https://issues.apache.org/jira/browse/HDFS-6018) | Exception recorded in LOG when IPCLoggerChannel#close is called | Trivial | . | Jing Zhao | Jing Zhao | +| [HDFS-6006](https://issues.apache.org/jira/browse/HDFS-6006) | Remove duplicate code in FSNameSystem#getFileInfo | Trivial | namenode | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5979](https://issues.apache.org/jira/browse/HDFS-5979) | Typo and logger fix for fsimage PB code | Minor | . | Andrew Wang | Andrew Wang | +| [HDFS-5939](https://issues.apache.org/jira/browse/HDFS-5939) | WebHdfs returns misleading error code and logs nothing if trying to create a file with no DNs in cluster | Major | hdfs-client | Yongjun Zhang | Yongjun Zhang | +| [HDFS-5935](https://issues.apache.org/jira/browse/HDFS-5935) | New Namenode UI FS browser should throw smarter error messages | Minor | namenode | Travis Thompson | Travis Thompson | +| [HDFS-5929](https://issues.apache.org/jira/browse/HDFS-5929) | Add Block pool % usage to HDFS federated nn page | Major | federation | Siqi Li | Siqi Li | +| [HDFS-5910](https://issues.apache.org/jira/browse/HDFS-5910) | Enhance DataTransferProtocol to allow per-connection choice of encryption/plain-text | Major | security | Benoy Antony | Benoy Antony | +| [HDFS-5908](https://issues.apache.org/jira/browse/HDFS-5908) | Change AclFeature to capture list of ACL entries in an ImmutableList. | Minor | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-5781](https://issues.apache.org/jira/browse/HDFS-5781) | Use an array to record the mapping between FSEditLogOpCode and the corresponding byte value | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5776](https://issues.apache.org/jira/browse/HDFS-5776) | Support 'hedged' reads in DFSClient | Major | hdfs-client | Liang Xie | Liang Xie | +| [HDFS-5775](https://issues.apache.org/jira/browse/HDFS-5775) | Consolidate the code for serialization in CacheManager | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-5768](https://issues.apache.org/jira/browse/HDFS-5768) | Consolidate the serialization code in DelegationTokenSecretManager | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-5709](https://issues.apache.org/jira/browse/HDFS-5709) | Improve NameNode upgrade with existing reserved paths and path components | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-5698](https://issues.apache.org/jira/browse/HDFS-5698) | Use protobuf to serialize / deserialize FSImage | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-5318](https://issues.apache.org/jira/browse/HDFS-5318) | Support read-only and read-write paths to shared replicas | Major | namenode | Eric Sirianni | | +| [HDFS-5153](https://issues.apache.org/jira/browse/HDFS-5153) | Datanode should send block reports for each storage in a separate message | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-4911](https://issues.apache.org/jira/browse/HDFS-4911) | Reduce PeerCache timeout to be commensurate with dfs.datanode.socket.reuse.keepalive | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4370](https://issues.apache.org/jira/browse/HDFS-4370) | Fix typo Blanacer in DataNode | Major | datanode | Konstantin Shvachko | Chu Tong | +| [HDFS-4200](https://issues.apache.org/jira/browse/HDFS-4200) | Reduce the size of synchronized sections in PacketResponder | Major | datanode | Suresh Srinivas | Andrew Wang | +| [HDFS-3405](https://issues.apache.org/jira/browse/HDFS-3405) | Checkpointing should use HTTP POST or PUT instead of GET-GET to send merged fsimages | Major | . | Aaron T. Myers | Vinayakumar B | +| [MAPREDUCE-5773](https://issues.apache.org/jira/browse/MAPREDUCE-5773) | Provide dedicated MRAppMaster syslog length limit | Blocker | mr-am | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5761](https://issues.apache.org/jira/browse/MAPREDUCE-5761) | Add a log message like "encrypted shuffle is ON" in nodemanager logs | Trivial | . | Yesha Vora | Jian He | +| [MAPREDUCE-5754](https://issues.apache.org/jira/browse/MAPREDUCE-5754) | Preserve Job diagnostics in history | Major | jobhistoryserver, mr-am | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5732](https://issues.apache.org/jira/browse/MAPREDUCE-5732) | Report proper queue when job has been automatically placed | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5553](https://issues.apache.org/jira/browse/MAPREDUCE-5553) | Add task state filters on Application/MRJob page for MR Application master | Minor | applicationmaster | Paul Han | Paul Han | +| [MAPREDUCE-2349](https://issues.apache.org/jira/browse/MAPREDUCE-2349) | speed up list[located]status calls from input formats | Major | task | Joydeep Sen Sarma | Siddharth Seth | +| [YARN-1789](https://issues.apache.org/jira/browse/YARN-1789) | ApplicationSummary does not escape newlines in the app name | Minor | resourcemanager | Akira AJISAKA | Tsuyoshi Ozawa | +| [YARN-1771](https://issues.apache.org/jira/browse/YARN-1771) | many getFileStatus calls made from node manager for localizing a public distributed cache resource | Critical | nodemanager | Sangjin Lee | Sangjin Lee | +| [YARN-1570](https://issues.apache.org/jira/browse/YARN-1570) | Formatting the lines within 80 chars in YarnCommands.apt.vm | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [YARN-1536](https://issues.apache.org/jira/browse/YARN-1536) | Cleanup: Get rid of ResourceManager#get\*SecretManager() methods and use the RMContext methods instead | Minor | resourcemanager | Karthik Kambatla | Anubhav Dhoot | +| [YARN-1512](https://issues.apache.org/jira/browse/YARN-1512) | Enhance CS to decouple scheduling from node heartbeats | Major | . | Arun C Murthy | Arun C Murthy | +| [YARN-1171](https://issues.apache.org/jira/browse/YARN-1171) | Add default queue properties to Fair Scheduler documentation | Major | documentation, scheduler | Sandy Ryza | Naren Koneru | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10450](https://issues.apache.org/jira/browse/HADOOP-10450) | Build zlib native code bindings in hadoop.dll for Windows. | Major | io, native | Chris Nauroth | Chris Nauroth | +| [HADOOP-10442](https://issues.apache.org/jira/browse/HADOOP-10442) | Group look-up can cause segmentation fault when certain JNI-based mapping module is used. | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10441](https://issues.apache.org/jira/browse/HADOOP-10441) | Namenode metric "rpc.RetryCache/NameNodeRetryCache.CacheHit" can't be correctly processed by Ganglia | Blocker | metrics | Jing Zhao | Jing Zhao | +| [HADOOP-10440](https://issues.apache.org/jira/browse/HADOOP-10440) | HarFsInputStream of HarFileSystem, when reading data, computing the position has bug | Major | fs | guodongdong | guodongdong | +| [HADOOP-10425](https://issues.apache.org/jira/browse/HADOOP-10425) | Incompatible behavior of LocalFileSystem:getContentSummary | Critical | fs | Brandon Li | Tsz Wo Nicholas Sze | +| [HADOOP-10422](https://issues.apache.org/jira/browse/HADOOP-10422) | Remove redundant logging of RPC retry attempts. | Minor | ipc | Chris Nauroth | Chris Nauroth | +| [HADOOP-10395](https://issues.apache.org/jira/browse/HADOOP-10395) | TestCallQueueManager is flaky | Minor | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-10394](https://issues.apache.org/jira/browse/HADOOP-10394) | TestAuthenticationFilter is flaky | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-10368](https://issues.apache.org/jira/browse/HADOOP-10368) | InputStream is not closed in VersionInfo ctor | Minor | util | Ted Yu | Tsuyoshi Ozawa | +| [HADOOP-10361](https://issues.apache.org/jira/browse/HADOOP-10361) | Correct alignment in CLI output for ACLs. | Minor | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-10355](https://issues.apache.org/jira/browse/HADOOP-10355) | TestLoadGenerator#testLoadGenerator fails | Major | . | Akira AJISAKA | Haohui Mai | +| [HADOOP-10354](https://issues.apache.org/jira/browse/HADOOP-10354) | TestWebHDFS fails after merge of HDFS-4685 to trunk | Major | fs | Yongjun Zhang | Chris Nauroth | +| [HADOOP-10353](https://issues.apache.org/jira/browse/HADOOP-10353) | FsUrlStreamHandlerFactory is not thread safe | Major | fs | Tudor Scurtu | Tudor Scurtu | +| [HADOOP-10352](https://issues.apache.org/jira/browse/HADOOP-10352) | Recursive setfacl erroneously attempts to apply default ACL to files. | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-10346](https://issues.apache.org/jira/browse/HADOOP-10346) | Deadlock while logging tokens | Blocker | security | Jason Lowe | Jason Lowe | +| [HADOOP-10338](https://issues.apache.org/jira/browse/HADOOP-10338) | Cannot get the FileStatus of the root inode from the new Globber | Major | . | Andrew Wang | Colin Patrick McCabe | +| [HADOOP-10337](https://issues.apache.org/jira/browse/HADOOP-10337) | ConcurrentModificationException from MetricsDynamicMBeanBase.createMBeanInfo() | Major | metrics | Liang Xie | Liang Xie | +| [HADOOP-10330](https://issues.apache.org/jira/browse/HADOOP-10330) | TestFrameDecoder fails if it cannot bind port 12345 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-10328](https://issues.apache.org/jira/browse/HADOOP-10328) | loadGenerator exit code is not reliable | Major | tools | Arpit Gupta | Haohui Mai | +| [HADOOP-10327](https://issues.apache.org/jira/browse/HADOOP-10327) | Trunk windows build broken after HDFS-5746 | Blocker | native | Vinayakumar B | Vinayakumar B | +| [HADOOP-10326](https://issues.apache.org/jira/browse/HADOOP-10326) | M/R jobs can not access S3 if Kerberos is enabled | Major | security | Manuel DE FERRAN | bc Wong | +| [HADOOP-10320](https://issues.apache.org/jira/browse/HADOOP-10320) | Javadoc in InterfaceStability.java lacks final \ | Trivial | documentation | RenĂ© Nyffenegger | RenĂ© Nyffenegger | +| [HADOOP-10314](https://issues.apache.org/jira/browse/HADOOP-10314) | The ls command help still shows outdated 0.16 format. | Major | . | Kihwal Lee | Rushabh S Shah | +| [HADOOP-10301](https://issues.apache.org/jira/browse/HADOOP-10301) | AuthenticationFilter should return Forbidden for failed authentication | Blocker | security | Daryn Sharp | Daryn Sharp | +| [HADOOP-10249](https://issues.apache.org/jira/browse/HADOOP-10249) | LdapGroupsMapping should trim ldap password read from file | Major | . | Dilli Arumugam | Dilli Arumugam | +| [HADOOP-10191](https://issues.apache.org/jira/browse/HADOOP-10191) | Missing executable permission on viewfs internal dirs | Blocker | viewfs | Gera Shegalov | Gera Shegalov | +| [HADOOP-10085](https://issues.apache.org/jira/browse/HADOOP-10085) | CompositeService should allow adding services while being inited | Blocker | . | Karthik Kambatla | Steve Loughran | +| [HADOOP-10070](https://issues.apache.org/jira/browse/HADOOP-10070) | RPC client doesn't use per-connection conf to determine server's expected Kerberos principal name | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-10015](https://issues.apache.org/jira/browse/HADOOP-10015) | UserGroupInformation prints out excessive ERROR warnings | Minor | security | Haohui Mai | Nicolas Liochon | +| [HDFS-6237](https://issues.apache.org/jira/browse/HDFS-6237) | TestDFSShell#testGet fails on Windows due to invalid file system path. | Trivial | hdfs-client, test | Chris Nauroth | Chris Nauroth | +| [HDFS-6166](https://issues.apache.org/jira/browse/HDFS-6166) | revisit balancer so\_timeout | Blocker | balancer & mover | Nathan Roberts | Nathan Roberts | +| [HDFS-6163](https://issues.apache.org/jira/browse/HDFS-6163) | Fix a minor bug in the HA upgrade document | Minor | documentation | Fengdong Yu | Fengdong Yu | +| [HDFS-6157](https://issues.apache.org/jira/browse/HDFS-6157) | Fix the entry point of OfflineImageViewer for hdfs.cmd | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6140](https://issues.apache.org/jira/browse/HDFS-6140) | WebHDFS cannot create a file with spaces in the name after HA failover changes. | Major | webhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-6135](https://issues.apache.org/jira/browse/HDFS-6135) | In HDFS upgrade with HA setup, JournalNode cannot handle layout version bump when rolling back | Blocker | journal-node | Jing Zhao | Jing Zhao | +| [HDFS-6131](https://issues.apache.org/jira/browse/HDFS-6131) | Move HDFSHighAvailabilityWithNFS.apt.vm and HDFSHighAvailabilityWithQJM.apt.vm from Yarn to HDFS | Major | documentation | Jing Zhao | Jing Zhao | +| [HDFS-6130](https://issues.apache.org/jira/browse/HDFS-6130) | NPE when upgrading namenode from fsimages older than -32 | Blocker | namenode | Fengdong Yu | Haohui Mai | +| [HDFS-6129](https://issues.apache.org/jira/browse/HDFS-6129) | When a replica is not found for deletion, do not throw exception. | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6127](https://issues.apache.org/jira/browse/HDFS-6127) | WebHDFS tokens cannot be renewed in HA setup | Major | ha | Arpit Gupta | Haohui Mai | +| [HDFS-6117](https://issues.apache.org/jira/browse/HDFS-6117) | Print file path information in FileNotFoundException | Minor | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6115](https://issues.apache.org/jira/browse/HDFS-6115) | flush() should be called for every append on block scan verification log | Minor | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-6107](https://issues.apache.org/jira/browse/HDFS-6107) | When a block can't be cached due to limited space on the DataNode, that block becomes uncacheable | Major | datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6106](https://issues.apache.org/jira/browse/HDFS-6106) | Reduce default for dfs.namenode.path.based.cache.refresh.interval.ms | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6105](https://issues.apache.org/jira/browse/HDFS-6105) | NN web UI for DN list loads the same jmx page multiple times. | Major | . | Kihwal Lee | Haohui Mai | +| [HDFS-6100](https://issues.apache.org/jira/browse/HDFS-6100) | DataNodeWebHdfsMethods does not failover in HA mode | Major | ha | Arpit Gupta | Haohui Mai | +| [HDFS-6099](https://issues.apache.org/jira/browse/HDFS-6099) | HDFS file system limits not enforced on renames. | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-6097](https://issues.apache.org/jira/browse/HDFS-6097) | zero-copy reads are incorrectly disabled on file offsets above 2GB | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6096](https://issues.apache.org/jira/browse/HDFS-6096) | TestWebHdfsTokens may timeout | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6094](https://issues.apache.org/jira/browse/HDFS-6094) | The same block can be counted twice towards safe mode threshold | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6089](https://issues.apache.org/jira/browse/HDFS-6089) | Standby NN while transitioning to active throws a connection refused error when the prior active NN process is suspended | Major | ha | Arpit Gupta | Jing Zhao | +| [HDFS-6079](https://issues.apache.org/jira/browse/HDFS-6079) | Timeout for getFileBlockStorageLocations does not work | Major | hdfs-client | Andrew Wang | Andrew Wang | +| [HDFS-6078](https://issues.apache.org/jira/browse/HDFS-6078) | TestIncrementalBlockReports is flaky | Minor | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6077](https://issues.apache.org/jira/browse/HDFS-6077) | running slive with webhdfs on secure HA cluster fails with unkown host exception | Major | . | Arpit Gupta | Jing Zhao | +| [HDFS-6071](https://issues.apache.org/jira/browse/HDFS-6071) | BlockReaderLocal doesn't return -1 on EOF when doing a zero-length read on a short file | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6068](https://issues.apache.org/jira/browse/HDFS-6068) | Disallow snapshot names that are also invalid directory names | Major | snapshots | Andrew Wang | sathish | +| [HDFS-6067](https://issues.apache.org/jira/browse/HDFS-6067) | TestPread.testMaxOutHedgedReadPool is flaky | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6065](https://issues.apache.org/jira/browse/HDFS-6065) | HDFS zero-copy reads should return null on EOF when doing ZCR | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6064](https://issues.apache.org/jira/browse/HDFS-6064) | DFSConfigKeys.DFS\_BLOCKREPORT\_INTERVAL\_MSEC\_DEFAULT is not updated with latest block report interval of 6 hrs | Minor | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-6063](https://issues.apache.org/jira/browse/HDFS-6063) | TestAclCLI fails intermittently when running test 24: copyFromLocal | Minor | test, tools | Colin Patrick McCabe | Chris Nauroth | +| [HDFS-6062](https://issues.apache.org/jira/browse/HDFS-6062) | TestRetryCacheWithHA#testConcat is flaky | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-6059](https://issues.apache.org/jira/browse/HDFS-6059) | TestBlockReaderLocal fails if native library is not available | Major | . | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6058](https://issues.apache.org/jira/browse/HDFS-6058) | Fix TestHDFSCLI failures after HADOOP-8691 change | Major | . | Vinayakumar B | Haohui Mai | +| [HDFS-6057](https://issues.apache.org/jira/browse/HDFS-6057) | DomainSocketWatcher.watcherThread should be marked as daemon thread | Blocker | hdfs-client | Eric Sirianni | Colin Patrick McCabe | +| [HDFS-6053](https://issues.apache.org/jira/browse/HDFS-6053) | Fix TestDecommissioningStatus and TestDecommission in branch-2 | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-6051](https://issues.apache.org/jira/browse/HDFS-6051) | HDFS cannot run on Windows since short-circuit shared memory segment changes. | Blocker | hdfs-client | Chris Nauroth | Colin Patrick McCabe | +| [HDFS-6047](https://issues.apache.org/jira/browse/HDFS-6047) | TestPread NPE inside in DFSInputStream hedgedFetchBlockByteRange | Major | . | stack | stack | +| [HDFS-6039](https://issues.apache.org/jira/browse/HDFS-6039) | Uploading a File under a Dir with default acls throws "Duplicated ACLFeature" | Major | namenode | Yesha Vora | Chris Nauroth | +| [HDFS-6033](https://issues.apache.org/jira/browse/HDFS-6033) | PBImageXmlWriter incorrectly handles processing cache directives | Major | caching | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6028](https://issues.apache.org/jira/browse/HDFS-6028) | Print clearer error message when user attempts to delete required mask entry from ACL. | Trivial | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-6008](https://issues.apache.org/jira/browse/HDFS-6008) | Namenode dead node link is giving HTTP error 500 | Minor | namenode | Benoy Antony | Benoy Antony | +| [HDFS-5988](https://issues.apache.org/jira/browse/HDFS-5988) | Bad fsimage always generated after upgrade | Blocker | namenode | Andrew Wang | Andrew Wang | +| [HDFS-5982](https://issues.apache.org/jira/browse/HDFS-5982) | Need to update snapshot manager when applying editlog for deleting a snapshottable directory | Critical | namenode | Tassapol Athiapinya | Jing Zhao | +| [HDFS-5981](https://issues.apache.org/jira/browse/HDFS-5981) | PBImageXmlWriter generates malformed XML | Minor | tools | Haohui Mai | Haohui Mai | +| [HDFS-5962](https://issues.apache.org/jira/browse/HDFS-5962) | Mtime and atime are not persisted for symbolic links | Critical | . | Kihwal Lee | Akira AJISAKA | +| [HDFS-5961](https://issues.apache.org/jira/browse/HDFS-5961) | OIV cannot load fsimages containing a symbolic link | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5959](https://issues.apache.org/jira/browse/HDFS-5959) | Fix typo at section name in FSImageFormatProtobuf.java | Minor | . | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5949](https://issues.apache.org/jira/browse/HDFS-5949) | New Namenode UI when trying to download a file, the browser doesn't know the file name | Minor | namenode | Travis Thompson | Travis Thompson | +| [HDFS-5948](https://issues.apache.org/jira/browse/HDFS-5948) | TestBackupNode flakes with port in use error | Major | . | Andrew Wang | Haohui Mai | +| [HDFS-5944](https://issues.apache.org/jira/browse/HDFS-5944) | LeaseManager:findLeaseWithPrefixPath can't handle path like /a/b/ right and cause SecondaryNameNode failed do checkpoint | Major | namenode | zhaoyunjiong | zhaoyunjiong | +| [HDFS-5943](https://issues.apache.org/jira/browse/HDFS-5943) | 'dfs.namenode.https-address.ns1' property is not used in federation setup | Major | . | Yesha Vora | Suresh Srinivas | +| [HDFS-5941](https://issues.apache.org/jira/browse/HDFS-5941) | add dfs.namenode.secondary.https-address and dfs.namenode.secondary.https-address in hdfs-default.xml | Major | documentation, namenode | Haohui Mai | Haohui Mai | +| [HDFS-5934](https://issues.apache.org/jira/browse/HDFS-5934) | New Namenode UI back button doesn't work as expected | Minor | namenode | Travis Thompson | Travis Thompson | +| [HDFS-5922](https://issues.apache.org/jira/browse/HDFS-5922) | DN heartbeat thread can get stuck in tight loop | Major | datanode | Aaron T. Myers | Arpit Agarwal | +| [HDFS-5915](https://issues.apache.org/jira/browse/HDFS-5915) | Refactor FSImageFormatProtobuf to simplify cross section reads | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-5913](https://issues.apache.org/jira/browse/HDFS-5913) | Nfs3Utils#getWccAttr() should check attr parameter against null | Minor | nfs | Ted Yu | Brandon Li | +| [HDFS-5904](https://issues.apache.org/jira/browse/HDFS-5904) | TestFileStatus fails intermittently on trunk and branch2 | Major | . | Mit Desai | Mit Desai | +| [HDFS-5901](https://issues.apache.org/jira/browse/HDFS-5901) | NameNode new UI doesn't support IE8 and IE9 on windows 7 | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5900](https://issues.apache.org/jira/browse/HDFS-5900) | Cannot set cache pool limit of "unlimited" via CacheAdmin | Major | caching | Tassapol Athiapinya | Andrew Wang | +| [HDFS-5895](https://issues.apache.org/jira/browse/HDFS-5895) | HDFS cacheadmin -listPools has exit\_code of 1 when the command returns 0 result. | Major | tools | Tassapol Athiapinya | Tassapol Athiapinya | +| [HDFS-5893](https://issues.apache.org/jira/browse/HDFS-5893) | HftpFileSystem.RangeHeaderUrlOpener uses the default URLConnectionFactory which does not import SSL certificates | Major | . | Yesha Vora | Haohui Mai | +| [HDFS-5891](https://issues.apache.org/jira/browse/HDFS-5891) | webhdfs should not try connecting the DN during redirection | Major | namenode, webhdfs | Haohui Mai | Haohui Mai | +| [HDFS-5886](https://issues.apache.org/jira/browse/HDFS-5886) | Potential null pointer deference in RpcProgramNfs3#readlink() | Major | nfs | Ted Yu | Brandon Li | +| [HDFS-5881](https://issues.apache.org/jira/browse/HDFS-5881) | Fix skip() of the short-circuit local reader (legacy). | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-5879](https://issues.apache.org/jira/browse/HDFS-5879) | Some TestHftpFileSystem tests do not close streams | Major | test | Gera Shegalov | Gera Shegalov | +| [HDFS-5859](https://issues.apache.org/jira/browse/HDFS-5859) | DataNode#checkBlockToken should check block tokens even if security is not enabled | Major | datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5857](https://issues.apache.org/jira/browse/HDFS-5857) | TestWebHDFS#testNamenodeRestart fails intermittently with NPE | Major | . | Mit Desai | Mit Desai | +| [HDFS-5856](https://issues.apache.org/jira/browse/HDFS-5856) | DataNode.checkDiskError might throw NPE | Minor | datanode | Josh Elser | Josh Elser | +| [HDFS-5846](https://issues.apache.org/jira/browse/HDFS-5846) | Assigning DEFAULT\_RACK in resolveNetworkLocation method can break data resiliency | Major | namenode | Nikola Vujic | Nikola Vujic | +| [HDFS-5843](https://issues.apache.org/jira/browse/HDFS-5843) | DFSClient.getFileChecksum() throws IOException if checksum is disabled | Major | datanode | Laurent Goujon | Laurent Goujon | +| [HDFS-5840](https://issues.apache.org/jira/browse/HDFS-5840) | Follow-up to HDFS-5138 to improve error handling during partial upgrade failures | Blocker | ha, journal-node, namenode | Aaron T. Myers | Jing Zhao | +| [HDFS-5828](https://issues.apache.org/jira/browse/HDFS-5828) | BlockPlacementPolicyWithNodeGroup can place multiple replicas on the same node group when dfs.namenode.avoid.write.stale.datanode is true | Major | namenode | Taylor, Buddy | Taylor, Buddy | +| [HDFS-5821](https://issues.apache.org/jira/browse/HDFS-5821) | TestHDFSCLI fails for user names with the dash character | Major | test | Gera Shegalov | Gera Shegalov | +| [HDFS-5807](https://issues.apache.org/jira/browse/HDFS-5807) | TestBalancerWithNodeGroup.testBalancerWithNodeGroup fails intermittently on Branch-2 | Major | test | Mit Desai | Chen He | +| [HDFS-5803](https://issues.apache.org/jira/browse/HDFS-5803) | TestBalancer.testBalancer0 fails | Major | . | Mit Desai | Chen He | +| [HDFS-5791](https://issues.apache.org/jira/browse/HDFS-5791) | TestHttpsFileSystem should use a random port to avoid binding error during testing | Major | test | Brandon Li | Haohui Mai | +| [HDFS-5790](https://issues.apache.org/jira/browse/HDFS-5790) | LeaseManager.findPath is very slow when many leases need recovery | Major | namenode, performance | Todd Lipcon | Todd Lipcon | +| [HDFS-5780](https://issues.apache.org/jira/browse/HDFS-5780) | TestRBWBlockInvalidation times out intemittently on branch-2 | Major | . | Mit Desai | Mit Desai | +| [HDFS-5767](https://issues.apache.org/jira/browse/HDFS-5767) | NFS implementation assumes userName userId mapping to be unique, which is not true sometimes | Blocker | nfs | Yongjun Zhang | Yongjun Zhang | +| [HDFS-5759](https://issues.apache.org/jira/browse/HDFS-5759) | Web UI does not show up during the period of loading FSImage | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-5742](https://issues.apache.org/jira/browse/HDFS-5742) | DatanodeCluster (mini cluster of DNs) fails to start | Minor | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-5716](https://issues.apache.org/jira/browse/HDFS-5716) | Allow WebHDFS to use pluggable authentication filter | Major | webhdfs | Haohui Mai | Haohui Mai | +| [HDFS-5705](https://issues.apache.org/jira/browse/HDFS-5705) | TestSecondaryNameNodeUpgrade#testChangeNsIDFails may fail due to ConcurrentModificationException | Major | datanode | Ted Yu | Ted Yu | +| [HDFS-5626](https://issues.apache.org/jira/browse/HDFS-5626) | dfsadmin -report shows incorrect cache values | Major | caching | Stephen Chu | Colin Patrick McCabe | +| [HDFS-5516](https://issues.apache.org/jira/browse/HDFS-5516) | WebHDFS does not require user name when anonymous http requests are disallowed. | Major | webhdfs | Chris Nauroth | Miodrag Radulovic | +| [HDFS-5492](https://issues.apache.org/jira/browse/HDFS-5492) | Port HDFS-2069 (Incorrect default trash interval in the docs) to trunk | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5339](https://issues.apache.org/jira/browse/HDFS-5339) | WebHDFS URI does not accept logical nameservices when security is enabled | Major | webhdfs | Stephen Chu | Haohui Mai | +| [HDFS-5244](https://issues.apache.org/jira/browse/HDFS-5244) | TestNNStorageRetentionManager#testPurgeMultipleDirs fails | Major | test | Jinghui Wang | Jinghui Wang | +| [HDFS-5064](https://issues.apache.org/jira/browse/HDFS-5064) | Standby checkpoints should not block concurrent readers | Major | ha, namenode | Aaron T. Myers | Aaron T. Myers | +| [HDFS-4858](https://issues.apache.org/jira/browse/HDFS-4858) | HDFS DataNode to NameNode RPC should timeout | Minor | datanode | Jagane Sundar | Henry Wang | +| [HDFS-3969](https://issues.apache.org/jira/browse/HDFS-3969) | Small bug fixes and improvements for disk locations API | Major | hdfs-client | Todd Lipcon | Todd Lipcon | +| [HDFS-3128](https://issues.apache.org/jira/browse/HDFS-3128) | Unit tests should not use a test root in /tmp | Minor | test | Eli Collins | Andrew Wang | +| [MAPREDUCE-5813](https://issues.apache.org/jira/browse/MAPREDUCE-5813) | YarnChild does not load job.xml with mapreduce.job.classloader=true | Blocker | mrv2, task | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5810](https://issues.apache.org/jira/browse/MAPREDUCE-5810) | TestStreamingTaskLog#testStreamingTaskLogWithHadoopCmd is failing | Major | contrib/streaming | Mit Desai | Akira AJISAKA | +| [MAPREDUCE-5806](https://issues.apache.org/jira/browse/MAPREDUCE-5806) | Log4j settings in container-log4j.properties cannot be overridden | Major | . | Eugene Koifman | Varun Vasudev | +| [MAPREDUCE-5805](https://issues.apache.org/jira/browse/MAPREDUCE-5805) | Unable to parse launch time from job history file | Major | jobhistoryserver | Fengdong Yu | Akira AJISAKA | +| [MAPREDUCE-5795](https://issues.apache.org/jira/browse/MAPREDUCE-5795) | Job should be marked as Failed if it is recovered from commit. | Major | . | Yesha Vora | Xuan Gong | +| [MAPREDUCE-5794](https://issues.apache.org/jira/browse/MAPREDUCE-5794) | SliveMapper always uses default FileSystem. | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-5791](https://issues.apache.org/jira/browse/MAPREDUCE-5791) | Shuffle phase is slow in Windows - FadviseFileRegion::transferTo does not read disks efficiently | Major | client | Nikola Vujic | Nikola Vujic | +| [MAPREDUCE-5789](https://issues.apache.org/jira/browse/MAPREDUCE-5789) | Average Reduce time is incorrect on Job Overview page | Major | jobhistoryserver, webapps | Rushabh S Shah | Rushabh S Shah | +| [MAPREDUCE-5780](https://issues.apache.org/jira/browse/MAPREDUCE-5780) | SliveTest always uses default FileSystem | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [MAPREDUCE-5778](https://issues.apache.org/jira/browse/MAPREDUCE-5778) | JobSummary does not escape newlines in the job name | Major | jobhistoryserver | Jason Lowe | Akira AJISAKA | +| [MAPREDUCE-5770](https://issues.apache.org/jira/browse/MAPREDUCE-5770) | Redirection from AM-URL is broken with HTTPS\_ONLY policy | Major | . | Yesha Vora | Jian He | +| [MAPREDUCE-5769](https://issues.apache.org/jira/browse/MAPREDUCE-5769) | Unregistration to RM should not be called if AM is crashed before registering with RM | Major | . | Rohith Sharma K S | Rohith Sharma K S | +| [MAPREDUCE-5768](https://issues.apache.org/jira/browse/MAPREDUCE-5768) | TestMRJobs.testContainerRollingLog fails on trunk | Major | . | Zhijie Shen | Gera Shegalov | +| [MAPREDUCE-5766](https://issues.apache.org/jira/browse/MAPREDUCE-5766) | Ping messages from attempts should be moved to DEBUG | Minor | applicationmaster | Ramya Sunil | Jian He | +| [MAPREDUCE-5757](https://issues.apache.org/jira/browse/MAPREDUCE-5757) | ConcurrentModificationException in JobControl.toList | Major | client | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5751](https://issues.apache.org/jira/browse/MAPREDUCE-5751) | MR app master fails to start in some cases if mapreduce.job.classloader is true | Major | . | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-5746](https://issues.apache.org/jira/browse/MAPREDUCE-5746) | Job diagnostics can implicate wrong task for a failed job | Major | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5699](https://issues.apache.org/jira/browse/MAPREDUCE-5699) | Allow setting tags on MR jobs | Major | applicationmaster | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-5688](https://issues.apache.org/jira/browse/MAPREDUCE-5688) | TestStagingCleanup fails intermittently with JDK7 | Major | . | Mit Desai | Mit Desai | +| [MAPREDUCE-5670](https://issues.apache.org/jira/browse/MAPREDUCE-5670) | CombineFileRecordReader should report progress when moving to the next file | Minor | mrv2 | Jason Lowe | Chen He | +| [MAPREDUCE-5570](https://issues.apache.org/jira/browse/MAPREDUCE-5570) | Map task attempt with fetch failure has incorrect attempt finish time | Major | mr-am, mrv2 | Jason Lowe | Rushabh S Shah | +| [MAPREDUCE-5028](https://issues.apache.org/jira/browse/MAPREDUCE-5028) | Maps fail when io.sort.mb is set to high value | Critical | . | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-4052](https://issues.apache.org/jira/browse/MAPREDUCE-4052) | Windows eclipse cannot submit job from Windows client to Linux/Unix Hadoop cluster. | Major | job submission | xieguiming | Jian He | +| [YARN-1873](https://issues.apache.org/jira/browse/YARN-1873) | TestDistributedShell#testDSShell fails when the test cases are out of order | Major | . | Mit Desai | Mit Desai | +| [YARN-1867](https://issues.apache.org/jira/browse/YARN-1867) | NPE while fetching apps via the REST API | Blocker | resourcemanager | Karthik Kambatla | Vinod Kumar Vavilapalli | +| [YARN-1866](https://issues.apache.org/jira/browse/YARN-1866) | YARN RM fails to load state store with delegation token parsing error | Blocker | . | Arpit Gupta | Jian He | +| [YARN-1859](https://issues.apache.org/jira/browse/YARN-1859) | WebAppProxyServlet will throw ApplicationNotFoundException if the app is no longer cached in RM | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1852](https://issues.apache.org/jira/browse/YARN-1852) | Application recovery throws InvalidStateTransitonException for FAILED and KILLED jobs | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-1849](https://issues.apache.org/jira/browse/YARN-1849) | NPE in ResourceTrackerService#registerNodeManager for UAM | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1846](https://issues.apache.org/jira/browse/YARN-1846) | TestRM#testNMTokenSentForNormalContainer assumes CapacityScheduler | Major | . | Robert Kanter | Robert Kanter | +| [YARN-1839](https://issues.apache.org/jira/browse/YARN-1839) | Capacity scheduler preempts an AM out. AM attempt 2 fails to launch task container with SecretManager$InvalidToken: No NMToken sent | Critical | applications, capacityscheduler | Tassapol Athiapinya | Jian He | +| [YARN-1833](https://issues.apache.org/jira/browse/YARN-1833) | TestRMAdminService Fails in trunk and branch-2 : Assert Fails due to different count of UserGroups for currentUser() | Major | . | Mit Desai | Mit Desai | +| [YARN-1830](https://issues.apache.org/jira/browse/YARN-1830) | TestRMRestart.testQueueMetricsOnRMRestart failure | Major | resourcemanager | Karthik Kambatla | Zhijie Shen | +| [YARN-1824](https://issues.apache.org/jira/browse/YARN-1824) | Make Windows client work with Linux/Unix cluster | Major | . | Jian He | Jian He | +| [YARN-1793](https://issues.apache.org/jira/browse/YARN-1793) | yarn application -kill doesn't kill UnmanagedAMs | Critical | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1788](https://issues.apache.org/jira/browse/YARN-1788) | AppsCompleted/AppsKilled metric is incorrect when MR job is killed with yarn application -kill | Critical | resourcemanager | Tassapol Athiapinya | Varun Vasudev | +| [YARN-1785](https://issues.apache.org/jira/browse/YARN-1785) | FairScheduler treats app lookup failures as ERRORs | Major | . | bc Wong | bc Wong | +| [YARN-1783](https://issues.apache.org/jira/browse/YARN-1783) | yarn application does not make any progress even when no other application is running when RM is being restarted in the background | Critical | . | Arpit Gupta | Jian He | +| [YARN-1774](https://issues.apache.org/jira/browse/YARN-1774) | FS: Submitting to non-leaf queue throws NPE | Blocker | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-1768](https://issues.apache.org/jira/browse/YARN-1768) | yarn kill non-existent application is too verbose | Minor | client | Hitesh Shah | Tsuyoshi Ozawa | +| [YARN-1760](https://issues.apache.org/jira/browse/YARN-1760) | TestRMAdminService assumes CapacityScheduler | Trivial | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-1758](https://issues.apache.org/jira/browse/YARN-1758) | MiniYARNCluster broken post YARN-1666 | Blocker | . | Hitesh Shah | Xuan Gong | +| [YARN-1752](https://issues.apache.org/jira/browse/YARN-1752) | Unexpected Unregistered event at Attempt Launched state | Major | . | Jian He | Rohith Sharma K S | +| [YARN-1748](https://issues.apache.org/jira/browse/YARN-1748) | hadoop-yarn-server-tests packages core-site.xml breaking downstream tests | Blocker | . | Sravya Tirukkovalur | Sravya Tirukkovalur | +| [YARN-1742](https://issues.apache.org/jira/browse/YARN-1742) | Fix javadoc of parameter DEFAULT\_NM\_MIN\_HEALTHY\_DISKS\_FRACTION | Trivial | documentation | Akira AJISAKA | Akira AJISAKA | +| [YARN-1724](https://issues.apache.org/jira/browse/YARN-1724) | Race condition in Fair Scheduler when continuous scheduling is turned on | Critical | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1721](https://issues.apache.org/jira/browse/YARN-1721) | When moving app between queues in Fair Scheduler, grab lock on FSSchedulerApp | Critical | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1697](https://issues.apache.org/jira/browse/YARN-1697) | NodeManager reports negative running containers | Major | nodemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1692](https://issues.apache.org/jira/browse/YARN-1692) | ConcurrentModificationException in fair scheduler AppSchedulable | Major | scheduler | Sangjin Lee | Sangjin Lee | +| [YARN-1689](https://issues.apache.org/jira/browse/YARN-1689) | RMAppAttempt is not killed when RMApp is at ACCEPTED | Critical | resourcemanager | Deepesh Khandelwal | Vinod Kumar Vavilapalli | +| [YARN-1686](https://issues.apache.org/jira/browse/YARN-1686) | NodeManager.resyncWithRM() does not handle exception which cause NodeManger to Hang. | Major | nodemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-1673](https://issues.apache.org/jira/browse/YARN-1673) | Valid yarn kill application prints out help message. | Blocker | client | Tassapol Athiapinya | Mayank Bansal | +| [YARN-1672](https://issues.apache.org/jira/browse/YARN-1672) | YarnConfiguration is missing a default for yarn.nodemanager.log.retain-seconds | Trivial | nodemanager | Karthik Kambatla | Naren Koneru | +| [YARN-1670](https://issues.apache.org/jira/browse/YARN-1670) | aggregated log writer can write more log data then it says is the log length | Critical | . | Thomas Graves | Mit Desai | +| [YARN-1661](https://issues.apache.org/jira/browse/YARN-1661) | AppMaster logs says failing even if an application does succeed. | Major | applications/distributed-shell | Tassapol Athiapinya | Vinod Kumar Vavilapalli | +| [YARN-1632](https://issues.apache.org/jira/browse/YARN-1632) | TestApplicationMasterServices should be under org.apache.hadoop.yarn.server.resourcemanager package | Minor | . | Chen He | Chen He | +| [YARN-1617](https://issues.apache.org/jira/browse/YARN-1617) | Remove ancient comment and surround LOG.debug in AppSchedulingInfo.allocate | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1591](https://issues.apache.org/jira/browse/YARN-1591) | TestResourceTrackerService fails randomly on trunk | Major | . | Vinod Kumar Vavilapalli | Tsuyoshi Ozawa | +| [YARN-1590](https://issues.apache.org/jira/browse/YARN-1590) | \_HOST doesn't expand properly for RM, NM, ProxyServer and JHS | Major | resourcemanager | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [YARN-1553](https://issues.apache.org/jira/browse/YARN-1553) | Do not use HttpConfig.isSecure() in YARN | Major | . | Haohui Mai | Haohui Mai | +| [YARN-1531](https://issues.apache.org/jira/browse/YARN-1531) | True up yarn command documentation | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [YARN-1528](https://issues.apache.org/jira/browse/YARN-1528) | Allow setting auth for ZK connections | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1470](https://issues.apache.org/jira/browse/YARN-1470) | Add audience annotation to MiniYARNCluster | Major | . | Sandy Ryza | Anubhav Dhoot | +| [YARN-1444](https://issues.apache.org/jira/browse/YARN-1444) | RM crashes when node resource request sent without corresponding off-switch request | Blocker | client, resourcemanager | Robert Grandl | Wangda Tan | +| [YARN-1417](https://issues.apache.org/jira/browse/YARN-1417) | RM may issue expired container tokens to AM while issuing new containers. | Blocker | . | Omkar Vinit Joshi | Jian He | +| [YARN-1398](https://issues.apache.org/jira/browse/YARN-1398) | Deadlock in capacity scheduler leaf queue and parent queue for getQueueInfo and completedContainer call | Blocker | resourcemanager | Sunil G | Vinod Kumar Vavilapalli | +| [YARN-1301](https://issues.apache.org/jira/browse/YARN-1301) | Need to log the blacklist additions/removals when YarnSchedule#allocate | Minor | . | Zhijie Shen | Tsuyoshi Ozawa | +| [YARN-1285](https://issues.apache.org/jira/browse/YARN-1285) | Inconsistency of default "yarn.acl.enable" value | Major | . | Zhijie Shen | Kenji Kikushima | +| [YARN-1206](https://issues.apache.org/jira/browse/YARN-1206) | AM container log link broken on NM web page | Blocker | . | Jian He | Rohith Sharma K S | +| [YARN-1166](https://issues.apache.org/jira/browse/YARN-1166) | YARN 'appsFailed' metric should be of type 'counter' | Blocker | resourcemanager | Srimanth Gunturi | Zhijie Shen | +| [YARN-1071](https://issues.apache.org/jira/browse/YARN-1071) | ResourceManager's decommissioned and lost node count is 0 after restart | Major | resourcemanager | Srimanth Gunturi | Jian He | +| [YARN-713](https://issues.apache.org/jira/browse/YARN-713) | ResourceManager can exit unexpectedly if DNS is unavailable | Critical | resourcemanager | Jason Lowe | Jian He | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-9525](https://issues.apache.org/jira/browse/HADOOP-9525) | Add tests that validate winutils chmod behavior on folders | Major | test, util | Ivan Mitic | Ivan Mitic | +| [HDFS-5953](https://issues.apache.org/jira/browse/HDFS-5953) | TestBlockReaderFactory fails if libhadoop.so has not been built | Major | . | Ted Yu | Akira AJISAKA | +| [HDFS-5936](https://issues.apache.org/jira/browse/HDFS-5936) | MiniDFSCluster does not clean data left behind by SecondaryNameNode. | Major | namenode, test | Andrew Wang | Binglin Chang | +| [HDFS-5882](https://issues.apache.org/jira/browse/HDFS-5882) | TestAuditLogs is flaky | Minor | . | Jimmy Xiang | Jimmy Xiang | +| [HDFS-5672](https://issues.apache.org/jira/browse/HDFS-5672) | TestHASafeMode#testSafeBlockTracking fails in trunk | Major | namenode | Ted Yu | Jing Zhao | +| [YARN-1863](https://issues.apache.org/jira/browse/YARN-1863) | TestRMFailover fails with 'AssertionError: null' | Blocker | . | Ted Yu | Xuan Gong | +| [YARN-1855](https://issues.apache.org/jira/browse/YARN-1855) | TestRMFailover#testRMWebAppRedirect fails in trunk | Critical | . | Ted Yu | Zhijie Shen | +| [YARN-1854](https://issues.apache.org/jira/browse/YARN-1854) | Race condition in TestRMHA#testStartAndTransitions | Blocker | . | Mit Desai | Rohith Sharma K S | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10449](https://issues.apache.org/jira/browse/HADOOP-10449) | Fix the javac warnings in the security packages. | Minor | security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10437](https://issues.apache.org/jira/browse/HADOOP-10437) | Fix the javac warnings in the conf and the util package | Minor | conf, util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10407](https://issues.apache.org/jira/browse/HADOOP-10407) | Fix the javac warnings in the ipc package. | Minor | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10399](https://issues.apache.org/jira/browse/HADOOP-10399) | FileContext API for ACLs. | Major | fs | Chris Nauroth | Vinayakumar B | +| [HADOOP-10393](https://issues.apache.org/jira/browse/HADOOP-10393) | Fix hadoop-auth javac warnings | Minor | security | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10285](https://issues.apache.org/jira/browse/HADOOP-10285) | Admin interface to swap callqueue at runtime | Major | . | Chris Li | Chris Li | +| [HADOOP-10280](https://issues.apache.org/jira/browse/HADOOP-10280) | Make Schedulables return a configurable identity of user or group | Major | . | Chris Li | Chris Li | +| [HADOOP-10278](https://issues.apache.org/jira/browse/HADOOP-10278) | Refactor to make CallQueue pluggable | Major | ipc | Chris Li | Chris Li | +| [HDFS-6124](https://issues.apache.org/jira/browse/HDFS-6124) | Add final modifier to class members | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6086](https://issues.apache.org/jira/browse/HDFS-6086) | Fix a case where zero-copy or no-checksum reads were not allowed even when the block was cached | Major | datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6076](https://issues.apache.org/jira/browse/HDFS-6076) | SimulatedDataSet should not create DatanodeRegistration with namenode layout version and type | Minor | datanode, test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6061](https://issues.apache.org/jira/browse/HDFS-6061) | Allow dfs.datanode.shared.file.descriptor.path to contain multiple entries and fall back when needed | Major | datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6060](https://issues.apache.org/jira/browse/HDFS-6060) | NameNode should not check DataNode layout version | Major | namenode | Brandon Li | Brandon Li | +| [HDFS-6046](https://issues.apache.org/jira/browse/HDFS-6046) | add dfs.client.mmap.enabled | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6040](https://issues.apache.org/jira/browse/HDFS-6040) | fix DFSClient issue without libhadoop.so and some other ShortCircuitShm cleanups | Blocker | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6038](https://issues.apache.org/jira/browse/HDFS-6038) | Allow JournalNode to handle editlog produced by new release with future layoutversion | Major | journal-node, namenode | Haohui Mai | Jing Zhao | +| [HDFS-5986](https://issues.apache.org/jira/browse/HDFS-5986) | Capture the number of blocks pending deletion on namenode webUI | Major | namenode | Suresh Srinivas | Chris Nauroth | +| [HDFS-5973](https://issues.apache.org/jira/browse/HDFS-5973) | add DomainSocket#shutdown method | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5956](https://issues.apache.org/jira/browse/HDFS-5956) | A file size is multiplied by the replication factor in 'hdfs oiv -p FileDistribution' option | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5950](https://issues.apache.org/jira/browse/HDFS-5950) | The DFSClient and DataNode should use shared memory segments to communicate short-circuit information | Major | datanode, hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5942](https://issues.apache.org/jira/browse/HDFS-5942) | Fix javadoc in OfflineImageViewer | Minor | documentation, tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5940](https://issues.apache.org/jira/browse/HDFS-5940) | Minor cleanups to ShortCircuitReplica, FsDatasetCache, and DomainSocketWatcher | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5938](https://issues.apache.org/jira/browse/HDFS-5938) | Make BlockReaderFactory#BlockReaderPeer a static class | Trivial | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5898](https://issues.apache.org/jira/browse/HDFS-5898) | Allow NFS gateway to login/relogin from its kerberos keytab | Major | nfs | Jing Zhao | Abin Shahab | +| [HDFS-5868](https://issues.apache.org/jira/browse/HDFS-5868) | Make hsync implementation pluggable | Major | datanode | Taylor, Buddy | Taylor, Buddy | +| [HDFS-5866](https://issues.apache.org/jira/browse/HDFS-5866) | '-maxSize' and '-step' option fail in OfflineImageViewer | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5847](https://issues.apache.org/jira/browse/HDFS-5847) | Consolidate INodeReference into a separate section | Major | . | Haohui Mai | Jing Zhao | +| [HDFS-5810](https://issues.apache.org/jira/browse/HDFS-5810) | Unify mmap cache and short-circuit file descriptor cache | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5746](https://issues.apache.org/jira/browse/HDFS-5746) | add ShortCircuitSharedMemorySegment | Major | datanode, hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-5726](https://issues.apache.org/jira/browse/HDFS-5726) | Fix compilation error in AbstractINodeDiff for JDK7 | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5715](https://issues.apache.org/jira/browse/HDFS-5715) | Use Snapshot ID to indicate the corresponding Snapshot for a FileDiff/DirectoryDiff | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5647](https://issues.apache.org/jira/browse/HDFS-5647) | Merge INodeDirectory.Feature and INodeFile.Feature | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-5638](https://issues.apache.org/jira/browse/HDFS-5638) | HDFS implementation of FileContext API for ACLs. | Major | hdfs-client | Chris Nauroth | Vinayakumar B | +| [HDFS-5632](https://issues.apache.org/jira/browse/HDFS-5632) | Add Snapshot feature to INodeDirectory | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5623](https://issues.apache.org/jira/browse/HDFS-5623) | NameNode: add tests for skipping ACL enforcement when permission checks are disabled, user is superuser or user is member of supergroup. | Major | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-5554](https://issues.apache.org/jira/browse/HDFS-5554) | Add Snapshot Feature to INodeFile | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-5537](https://issues.apache.org/jira/browse/HDFS-5537) | Remove FileWithSnapshot interface | Major | namenode, snapshots | Jing Zhao | Jing Zhao | +| [HDFS-5531](https://issues.apache.org/jira/browse/HDFS-5531) | Combine the getNsQuota() and getDsQuota() methods in INode | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5483](https://issues.apache.org/jira/browse/HDFS-5483) | NN should gracefully handle multiple block replicas on same DN | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-5286](https://issues.apache.org/jira/browse/HDFS-5286) | Flatten INodeDirectory hierarchy: add DirectoryWithQuotaFeature | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-5285](https://issues.apache.org/jira/browse/HDFS-5285) | Flatten INodeFile hierarchy: Add UnderContruction Feature | Major | namenode | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-5167](https://issues.apache.org/jira/browse/HDFS-5167) | Add metrics about the NameNode retry cache | Minor | ha, namenode | Jing Zhao | Tsuyoshi Ozawa | +| [HDFS-4564](https://issues.apache.org/jira/browse/HDFS-4564) | Webhdfs returns incorrect http response codes for denied operations | Blocker | webhdfs | Daryn Sharp | Daryn Sharp | +| [MAPREDUCE-5787](https://issues.apache.org/jira/browse/MAPREDUCE-5787) | Modify ShuffleHandler to support Keep-Alive | Critical | nodemanager | Rajesh Balamohan | Rajesh Balamohan | +| [YARN-1893](https://issues.apache.org/jira/browse/YARN-1893) | Make ApplicationMasterProtocol#allocate AtMostOnce | Blocker | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-1850](https://issues.apache.org/jira/browse/YARN-1850) | Make enabling timeline service configurable | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1838](https://issues.apache.org/jira/browse/YARN-1838) | Timeline service getEntities API should provide ability to get entities from given id | Major | . | Srimanth Gunturi | Billie Rinaldi | +| [YARN-1821](https://issues.apache.org/jira/browse/YARN-1821) | NPE on registerNodeManager if the request has containers for UnmanagedAMs | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1816](https://issues.apache.org/jira/browse/YARN-1816) | Succeeded application remains in accepted after RM restart | Major | . | Arpit Gupta | Jian He | +| [YARN-1812](https://issues.apache.org/jira/browse/YARN-1812) | Job stays in PREP state for long time after RM Restarts | Major | . | Yesha Vora | Jian He | +| [YARN-1811](https://issues.apache.org/jira/browse/YARN-1811) | RM HA: AM link broken if the AM is on nodes other than RM | Major | resourcemanager | Robert Kanter | Robert Kanter | +| [YARN-1800](https://issues.apache.org/jira/browse/YARN-1800) | YARN NodeManager with java.util.concurrent.RejectedExecutionException | Critical | nodemanager | Paul Isaychuk | Varun Vasudev | +| [YARN-1787](https://issues.apache.org/jira/browse/YARN-1787) | yarn applicationattempt/container print wrong usage information | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1781](https://issues.apache.org/jira/browse/YARN-1781) | NM should allow users to specify max disk utilization for local disks | Major | nodemanager | Varun Vasudev | Varun Vasudev | +| [YARN-1780](https://issues.apache.org/jira/browse/YARN-1780) | Improve logging in timeline service | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1776](https://issues.apache.org/jira/browse/YARN-1776) | renewDelegationToken should survive RM failover | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1775](https://issues.apache.org/jira/browse/YARN-1775) | Create SMAPBasedProcessTree to get PSS information | Major | nodemanager | Rajesh Balamohan | Rajesh Balamohan | +| [YARN-1766](https://issues.apache.org/jira/browse/YARN-1766) | When RM does the initiation, it should use loaded Configuration instead of bootstrap configuration. | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1765](https://issues.apache.org/jira/browse/YARN-1765) | Write test cases to verify that killApplication API works in RM HA | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1764](https://issues.apache.org/jira/browse/YARN-1764) | Handle RM fail overs after the submitApplication call. | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1761](https://issues.apache.org/jira/browse/YARN-1761) | RMAdminCLI should check whether HA is enabled before executes transitionToActive/transitionToStandby | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1749](https://issues.apache.org/jira/browse/YARN-1749) | Review AHS configs and sync them up with the timeline-service configs | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1734](https://issues.apache.org/jira/browse/YARN-1734) | RM should get the updated Configurations when it transits from Standby to Active | Critical | . | Xuan Gong | Xuan Gong | +| [YARN-1732](https://issues.apache.org/jira/browse/YARN-1732) | Change types of related entities and primary filters in ATSEntity | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1730](https://issues.apache.org/jira/browse/YARN-1730) | Leveldb timeline store needs simple write locking | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1729](https://issues.apache.org/jira/browse/YARN-1729) | TimelineWebServices always passes primary and secondary filters as strings | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1719](https://issues.apache.org/jira/browse/YARN-1719) | ATSWebServices produces jersey warnings | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1717](https://issues.apache.org/jira/browse/YARN-1717) | Enable offline deletion of entries in leveldb timeline store | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1706](https://issues.apache.org/jira/browse/YARN-1706) | Create an utility function to dump timeline records to json | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1704](https://issues.apache.org/jira/browse/YARN-1704) | Review LICENSE and NOTICE to reflect new levelDB releated libraries being used | Blocker | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1698](https://issues.apache.org/jira/browse/YARN-1698) | Replace MemoryApplicationTimelineStore with LeveldbApplicationTimelineStore as default | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1690](https://issues.apache.org/jira/browse/YARN-1690) | Sending timeline entities+events from Distributed shell | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-1687](https://issues.apache.org/jira/browse/YARN-1687) | Refactoring timeline classes to remove "app" related words | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1685](https://issues.apache.org/jira/browse/YARN-1685) | Bugs around log URL | Major | . | Mayank Bansal | Zhijie Shen | +| [YARN-1684](https://issues.apache.org/jira/browse/YARN-1684) | Fix history server heap size in yarn script | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1676](https://issues.apache.org/jira/browse/YARN-1676) | Make admin refreshUserToGroupsMappings of configuration work across RM failover | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1669](https://issues.apache.org/jira/browse/YARN-1669) | Make admin refreshServiceAcls work across RM failover | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1668](https://issues.apache.org/jira/browse/YARN-1668) | Make admin refreshAdminAcls work across RM failover | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1667](https://issues.apache.org/jira/browse/YARN-1667) | Make admin refreshSuperUserGroupsConfiguration work across RM failover | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1666](https://issues.apache.org/jira/browse/YARN-1666) | Make admin refreshNodes work across RM failover | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1665](https://issues.apache.org/jira/browse/YARN-1665) | Set better defaults for HA configs for automatic failover | Major | resourcemanager | Arpit Gupta | Xuan Gong | +| [YARN-1660](https://issues.apache.org/jira/browse/YARN-1660) | add the ability to set yarn.resourcemanager.hostname.rm-id instead of setting all the various host:port properties for RM | Major | resourcemanager | Arpit Gupta | Xuan Gong | +| [YARN-1659](https://issues.apache.org/jira/browse/YARN-1659) | Define the ApplicationTimelineStore store as an abstraction for implementing different storage impls for storing timeline information | Major | . | Billie Rinaldi | Billie Rinaldi | +| [YARN-1658](https://issues.apache.org/jira/browse/YARN-1658) | Webservice should redirect to active RM when HA is enabled. | Major | . | Cindy Li | Cindy Li | +| [YARN-1641](https://issues.apache.org/jira/browse/YARN-1641) | ZK store should attempt a write periodically to ensure it is still Active | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1640](https://issues.apache.org/jira/browse/YARN-1640) | Manual Failover does not work in secure clusters | Blocker | . | Xuan Gong | Xuan Gong | +| [YARN-1639](https://issues.apache.org/jira/browse/YARN-1639) | YARM RM HA requires different configs on different RM hosts | Major | resourcemanager | Arpit Gupta | Xuan Gong | +| [YARN-1637](https://issues.apache.org/jira/browse/YARN-1637) | Implement a client library for java users to post entities+events | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-1636](https://issues.apache.org/jira/browse/YARN-1636) | Implement timeline related web-services inside AHS for storing and retrieving entities+events | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-1635](https://issues.apache.org/jira/browse/YARN-1635) | Implement a Leveldb based ApplicationTimelineStore | Major | . | Vinod Kumar Vavilapalli | Billie Rinaldi | +| [YARN-1634](https://issues.apache.org/jira/browse/YARN-1634) | Define an in-memory implementation of ApplicationTimelineStore | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-1633](https://issues.apache.org/jira/browse/YARN-1633) | Define user-faced entity, entity-info and event objects | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-1625](https://issues.apache.org/jira/browse/YARN-1625) | mvn apache-rat:check outputs warning message in YARN-321 branch | Trivial | . | Shinichi Yamashita | Shinichi Yamashita | +| [YARN-1613](https://issues.apache.org/jira/browse/YARN-1613) | Fix config name YARN\_HISTORY\_SERVICE\_ENABLED | Major | . | Zhijie Shen | Akira AJISAKA | +| [YARN-1611](https://issues.apache.org/jira/browse/YARN-1611) | Make admin refresh of capacity scheduler configuration work across RM failover | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1605](https://issues.apache.org/jira/browse/YARN-1605) | Fix formatting issues with new module in YARN-321 branch | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1597](https://issues.apache.org/jira/browse/YARN-1597) | FindBugs warnings on YARN-321 branch | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1596](https://issues.apache.org/jira/browse/YARN-1596) | Javadoc failures on YARN-321 branch | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1595](https://issues.apache.org/jira/browse/YARN-1595) | Test failures on YARN-321 branch | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1594](https://issues.apache.org/jira/browse/YARN-1594) | YARN-321 branch needs to be updated after YARN-888 pom changes | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1588](https://issues.apache.org/jira/browse/YARN-1588) | Rebind NM tokens for previous attempt's running containers to the new attempt | Major | . | Jian He | Jian He | +| [YARN-1587](https://issues.apache.org/jira/browse/YARN-1587) | [YARN-321] Merge Patch for YARN-321 | Major | . | Mayank Bansal | Vinod Kumar Vavilapalli | +| [YARN-1578](https://issues.apache.org/jira/browse/YARN-1578) | Fix how to read history file in FileSystemApplicationHistoryStore | Major | . | Shinichi Yamashita | Shinichi Yamashita | +| [YARN-1577](https://issues.apache.org/jira/browse/YARN-1577) | Unmanaged AM is broken because of YARN-1493 | Blocker | . | Jian He | Jian He | +| [YARN-1566](https://issues.apache.org/jira/browse/YARN-1566) | Change distributed-shell to retain containers from previous AppAttempt | Major | . | Jian He | Jian He | +| [YARN-1555](https://issues.apache.org/jira/browse/YARN-1555) | [YARN-321] Failing tests in org.apache.hadoop.yarn.server.applicationhistoryservice.\* | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1534](https://issues.apache.org/jira/browse/YARN-1534) | TestAHSWebApp failed in YARN-321 branch | Major | . | Shinichi Yamashita | Shinichi Yamashita | +| [YARN-1525](https://issues.apache.org/jira/browse/YARN-1525) | Web UI should redirect to active RM when HA is enabled. | Major | . | Xuan Gong | Cindy Li | +| [YARN-1521](https://issues.apache.org/jira/browse/YARN-1521) | Mark appropriate protocol methods with the idempotent annotation or AtMostOnce annotation | Blocker | . | Xuan Gong | Xuan Gong | +| [YARN-1504](https://issues.apache.org/jira/browse/YARN-1504) | RM changes for moving apps between queues | Major | resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1499](https://issues.apache.org/jira/browse/YARN-1499) | Fair Scheduler changes for moving apps between queues | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1498](https://issues.apache.org/jira/browse/YARN-1498) | Common scheduler changes for moving apps between queues | Major | resourcemanager | Sandy Ryza | Sandy Ryza | +| [YARN-1497](https://issues.apache.org/jira/browse/YARN-1497) | Expose moving apps between queues on the command line | Major | client | Sandy Ryza | Sandy Ryza | +| [YARN-1496](https://issues.apache.org/jira/browse/YARN-1496) | Protocol additions to allow moving apps between queues | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1493](https://issues.apache.org/jira/browse/YARN-1493) | Schedulers don't recognize apps separately from app-attempts | Major | . | Jian He | Jian He | +| [YARN-1490](https://issues.apache.org/jira/browse/YARN-1490) | RM should optionally not kill all containers when an ApplicationMaster exits | Major | . | Vinod Kumar Vavilapalli | Jian He | +| [YARN-1461](https://issues.apache.org/jira/browse/YARN-1461) | RM API and RM changes to handle tags for running jobs | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-1459](https://issues.apache.org/jira/browse/YARN-1459) | RM services should depend on ConfigurationProvider during startup too | Major | resourcemanager | Karthik Kambatla | Xuan Gong | +| [YARN-1428](https://issues.apache.org/jira/browse/YARN-1428) | RM cannot write the final state of RMApp/RMAppAttempt to the application history store in the transition to the final state | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1410](https://issues.apache.org/jira/browse/YARN-1410) | Handle RM fails over after getApplicationID() and before submitApplication(). | Major | . | Bikas Saha | Xuan Gong | +| [YARN-1389](https://issues.apache.org/jira/browse/YARN-1389) | ApplicationClientProtocol and ApplicationHistoryProtocol should expose analogous APIs | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-1379](https://issues.apache.org/jira/browse/YARN-1379) | [YARN-321] AHS protocols need to be in yarn proto package name after YARN-1170 | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1345](https://issues.apache.org/jira/browse/YARN-1345) | Removing FINAL\_SAVING from YarnApplicationAttemptState | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1266](https://issues.apache.org/jira/browse/YARN-1266) | Implement PB service and client wrappers for ApplicationHistoryProtocol | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-1242](https://issues.apache.org/jira/browse/YARN-1242) | Script changes to start AHS as an individual process | Major | . | Zhijie Shen | Mayank Bansal | +| [YARN-1191](https://issues.apache.org/jira/browse/YARN-1191) | [YARN-321] Update artifact versions for application history service | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-1123](https://issues.apache.org/jira/browse/YARN-1123) | [YARN-321] Adding ContainerReport and Protobuf implementation | Major | . | Zhijie Shen | Mayank Bansal | +| [YARN-1041](https://issues.apache.org/jira/browse/YARN-1041) | Protocol changes for RM to bind and notify a restarted AM of existing containers | Major | resourcemanager | Steve Loughran | Jian He | +| [YARN-1023](https://issues.apache.org/jira/browse/YARN-1023) | [YARN-321] Webservices REST API's support for Application History | Major | . | Devaraj K | Zhijie Shen | +| [YARN-1017](https://issues.apache.org/jira/browse/YARN-1017) | Document RM Restart feature | Blocker | resourcemanager | Jian He | Jian He | +| [YARN-1007](https://issues.apache.org/jira/browse/YARN-1007) | [YARN-321] Enhance History Reader interface for Containers | Major | . | Devaraj K | Mayank Bansal | +| [YARN-987](https://issues.apache.org/jira/browse/YARN-987) | Adding ApplicationHistoryManager responsible for exposing reports to all clients | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-986](https://issues.apache.org/jira/browse/YARN-986) | RM DT token service should have service addresses of both RMs | Blocker | . | Vinod Kumar Vavilapalli | Karthik Kambatla | +| [YARN-984](https://issues.apache.org/jira/browse/YARN-984) | [YARN-321] Move classes from applicationhistoryservice.records.pb.impl package to applicationhistoryservice.records.impl.pb | Major | . | Devaraj K | Devaraj K | +| [YARN-979](https://issues.apache.org/jira/browse/YARN-979) | [YARN-321] Add more APIs related to ApplicationAttempt and Container in ApplicationHistoryProtocol | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-978](https://issues.apache.org/jira/browse/YARN-978) | [YARN-321] Adding ApplicationAttemptReport and Protobuf implementation | Major | . | Mayank Bansal | Mayank Bansal | +| [YARN-975](https://issues.apache.org/jira/browse/YARN-975) | Add a file-system implementation for history-storage | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-974](https://issues.apache.org/jira/browse/YARN-974) | RMContainer should collect more useful information to be recorded in Application-History | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-967](https://issues.apache.org/jira/browse/YARN-967) | [YARN-321] Command Line Interface(CLI) for Reading Application History Storage Data | Major | . | Devaraj K | Mayank Bansal | +| [YARN-962](https://issues.apache.org/jira/browse/YARN-962) | Update application\_history\_service.proto | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-956](https://issues.apache.org/jira/browse/YARN-956) | [YARN-321] Add a testable in-memory HistoryStorage | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-955](https://issues.apache.org/jira/browse/YARN-955) | [YARN-321] Implementation of ApplicationHistoryProtocol | Major | . | Vinod Kumar Vavilapalli | Mayank Bansal | +| [YARN-954](https://issues.apache.org/jira/browse/YARN-954) | [YARN-321] History Service should create the webUI and wire it to HistoryStorage | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-953](https://issues.apache.org/jira/browse/YARN-953) | [YARN-321] Enable ResourceManager to write history data | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-947](https://issues.apache.org/jira/browse/YARN-947) | Defining the history data classes for the implementation of the reading/writing interface | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-935](https://issues.apache.org/jira/browse/YARN-935) | YARN-321 branch is broken due to applicationhistoryserver module's pom.xml | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-934](https://issues.apache.org/jira/browse/YARN-934) | HistoryStorage writer interface for Application History Server | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-930](https://issues.apache.org/jira/browse/YARN-930) | Bootstrap ApplicationHistoryService module | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-925](https://issues.apache.org/jira/browse/YARN-925) | HistoryStorage Reader Interface for Application History Server | Major | . | Mayank Bansal | Mayank Bansal | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-6030](https://issues.apache.org/jira/browse/HDFS-6030) | Remove an unused constructor in INode.java | Trivial | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6025](https://issues.apache.org/jira/browse/HDFS-6025) | Update findbugsExcludeFile.xml | Minor | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [YARN-1891](https://issues.apache.org/jira/browse/YARN-1891) | Document NodeManager health-monitoring | Minor | . | Varun Vasudev | Varun Vasudev | +| [YARN-1452](https://issues.apache.org/jira/browse/YARN-1452) | Document the usage of the generic application history and the timeline data service | Major | . | Zhijie Shen | Zhijie Shen | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md new file mode 100644 index 00000000000..63fbfe4ae54 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.0/RELEASENOTES.2.4.0.md @@ -0,0 +1,145 @@ + + +# Apache Hadoop 2.4.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10295](https://issues.apache.org/jira/browse/HADOOP-10295) | *Major* | **Allow distcp to automatically identify the checksum type of source files and use it for the target** + +Add option for distcp to preserve the checksum type of the source files. Users can use "-pc" as distcp command option to preserve the checksum type. + + +--- + +* [HADOOP-10221](https://issues.apache.org/jira/browse/HADOOP-10221) | *Major* | **Add a plugin to specify SaslProperties for RPC protocol based on connection properties** + +SaslPropertiesResolver or its subclass is used to resolve the QOP used for a connection. The subclass can be specified via "hadoop.security.saslproperties.resolver.class" configuration property. If not specified, the full set of values specified in hadoop.rpc.protection is used while determining the QOP used for the connection. If a class is specified, then the QOP values returned by the class will be used while determining the QOP used for the connection. + +Note that this change, effectively removes SaslRpcServer.SASL\_PROPS which was a public field. Any use of this variable should be replaced with the following code: +SaslPropertiesResolver saslPropsResolver = SaslPropertiesResolver.getInstance(conf); +Map\ sasl\_props = saslPropsResolver.getDefaultProperties(); + + +--- + +* [HADOOP-10211](https://issues.apache.org/jira/browse/HADOOP-10211) | *Major* | **Enable RPC protocol to negotiate SASL-QOP values between clients and servers** + +The hadoop.rpc.protection configuration property previously supported specifying a single value: one of authentication, integrity or privacy. An unrecognized value was silently assumed to mean authentication. This configuration property now accepts a comma-separated list of any of the 3 values, and unrecognized values are rejected with an error. Existing configurations containing an invalid value must be corrected. If the property is empty or not specified, authentication is assumed. + + +--- + +* [HADOOP-8691](https://issues.apache.org/jira/browse/HADOOP-8691) | *Minor* | **FsShell can print "Found xxx items" unnecessarily often** + +The `ls` command only prints "Found foo items" once when listing the directories recursively. + + +--- + +* [HDFS-6102](https://issues.apache.org/jira/browse/HDFS-6102) | *Blocker* | **Lower the default maximum items per directory to fix PB fsimage loading** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-6055](https://issues.apache.org/jira/browse/HDFS-6055) | *Major* | **Change default configuration to limit file name length in HDFS** + +The default configuration of HDFS now sets dfs.namenode.fs-limits.max-component-length to 255 for improved interoperability with other file system implementations. This limits each component of a file system path to a maximum of 255 bytes in UTF-8 encoding. Attempts to create new files that violate this rule will fail with an error. Existing files that violate the rule are not effected. Previously, dfs.namenode.fs-limits.max-component-length was set to 0 (ignored). If necessary, it is possible to set the value back to 0 in the cluster's configuration to restore the old behavior. + + +--- + +* [HDFS-5804](https://issues.apache.org/jira/browse/HDFS-5804) | *Major* | **HDFS NFS Gateway fails to mount and proxy when using Kerberos** + +Fixes NFS on Kerberized cluster. + + +--- + +* [HDFS-5790](https://issues.apache.org/jira/browse/HDFS-5790) | *Major* | **LeaseManager.findPath is very slow when many leases need recovery** + +Committed to branch-2 and trunk. + + +--- + +* [HDFS-5776](https://issues.apache.org/jira/browse/HDFS-5776) | *Major* | **Support 'hedged' reads in DFSClient** + +If a read from a block is slow, start up another parallel, 'hedged' read against a different block replica. We then take the result of which ever read returns first (the outstanding read is cancelled). This 'hedged' read feature will help rein in the outliers, the odd read that takes a long time because it hit a bad patch on the disc, etc. + +This feature is off by default. To enable this feature, set \dfs.client.hedged.read.threadpool.size\ to a positive number. The threadpool size is how many threads to dedicate to the running of these 'hedged', concurrent reads in your client. + +Then set \dfs.client.hedged.read.threshold.millis\ to the number of milliseconds to wait before starting up a 'hedged' read. For example, if you set this property to 10, then if a read has not returned within 10 milliseconds, we will start up a new read against a different block replica. + +This feature emits new metrics: + ++ hedgedReadOps ++ hedgeReadOpsWin -- how many times the hedged read 'beat' the original read ++ hedgedReadOpsInCurThread -- how many times we went to do a hedged read but we had to run it in the current thread because dfs.client.hedged.read.threadpool.size was at a maximum. + + +--- + +* [HDFS-5698](https://issues.apache.org/jira/browse/HDFS-5698) | *Major* | **Use protobuf to serialize / deserialize FSImage** + +Use protobuf to serialize/deserialize the FSImage. + + +--- + +* [HDFS-5321](https://issues.apache.org/jira/browse/HDFS-5321) | *Major* | **Clean up the HTTP-related configuration in HDFS** + +dfs.http.port and dfs.https.port are removed. Filesystem clients, such as WebHdfsFileSystem, now have fixed instead of configurable default ports (i.e., 50070 for http and 50470 for https). + +Users can explicitly specify the port in the URI to access the file system which runs on non-default ports. + + +--- + +* [HDFS-5138](https://issues.apache.org/jira/browse/HDFS-5138) | *Blocker* | **Support HDFS upgrade in HA** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-4685](https://issues.apache.org/jira/browse/HDFS-4685) | *Major* | **Implementation of ACLs in HDFS** + +HDFS now supports ACLs (Access Control Lists). ACLs can specify fine-grained file permissions for specific named users or named groups. + + +--- + +* [HDFS-4370](https://issues.apache.org/jira/browse/HDFS-4370) | *Major* | **Fix typo Blanacer in DataNode** + +I just committed this. Thank you Chu. + + +--- + +* [MAPREDUCE-5036](https://issues.apache.org/jira/browse/MAPREDUCE-5036) | *Major* | **Default shuffle handler port should not be 8080** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md new file mode 100644 index 00000000000..48d94802b98 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/CHANGES.2.4.1.md @@ -0,0 +1,152 @@ + + +# Apache Hadoop Changelog + +## Release 2.4.1 - 2014-06-30 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10466](https://issues.apache.org/jira/browse/HADOOP-10466) | Lower the log level in UserGroupInformation | Minor | security | Nicolas Liochon | Nicolas Liochon | +| [HDFS-4052](https://issues.apache.org/jira/browse/HDFS-4052) | BlockManager#invalidateWork should print logs outside the lock | Minor | . | Jing Zhao | Jing Zhao | +| [YARN-1892](https://issues.apache.org/jira/browse/YARN-1892) | Excessive logging in RM | Minor | scheduler | Siddharth Seth | Jian He | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11273](https://issues.apache.org/jira/browse/HADOOP-11273) | TestMiniKdc failure: login options not compatible with IBM JDK | Major | test | Gao Zhong Liang | Gao Zhong Liang | +| [HADOOP-10612](https://issues.apache.org/jira/browse/HADOOP-10612) | NFS failed to refresh the user group id mapping table | Major | nfs | Brandon Li | Brandon Li | +| [HADOOP-10562](https://issues.apache.org/jira/browse/HADOOP-10562) | Namenode exits on exception without printing stack trace in AbstractDelegationTokenSecretManager | Critical | . | Suresh Srinivas | Suresh Srinivas | +| [HADOOP-10527](https://issues.apache.org/jira/browse/HADOOP-10527) | Fix incorrect return code and allow more retries on EINTR | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10522](https://issues.apache.org/jira/browse/HADOOP-10522) | JniBasedUnixGroupMapping mishandles errors | Critical | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10490](https://issues.apache.org/jira/browse/HADOOP-10490) | TestMapFile and TestBloomMapFile leak file descriptors. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-10473](https://issues.apache.org/jira/browse/HADOOP-10473) | TestCallQueueManager is still flaky | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10456](https://issues.apache.org/jira/browse/HADOOP-10456) | Bug in Configuration.java exposed by Spark (ConcurrentModificationException) | Major | conf | Nishkam Ravi | Nishkam Ravi | +| [HADOOP-10455](https://issues.apache.org/jira/browse/HADOOP-10455) | When there is an exception, ipc.Server should first check whether it is an terse exception | Major | ipc | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-8826](https://issues.apache.org/jira/browse/HADOOP-8826) | Docs still refer to 0.20.205 as stable line | Minor | . | Robert Joseph Evans | Mit Desai | +| [HDFS-6411](https://issues.apache.org/jira/browse/HDFS-6411) | nfs-hdfs-gateway mount raises I/O error and hangs when a unauthorized user attempts to access it | Major | nfs | Zhongyi Xie | Brandon Li | +| [HDFS-6402](https://issues.apache.org/jira/browse/HDFS-6402) | Suppress findbugs warning for failure to override equals and hashCode in FsAclPermission. | Trivial | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-6397](https://issues.apache.org/jira/browse/HDFS-6397) | NN shows inconsistent value in deadnode count | Critical | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [HDFS-6362](https://issues.apache.org/jira/browse/HDFS-6362) | InvalidateBlocks is inconsistent in usage of DatanodeUuid and StorageID | Blocker | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6361](https://issues.apache.org/jira/browse/HDFS-6361) | TestIdUserGroup.testUserUpdateSetting failed due to out of range nfsnobody Id | Major | nfs | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6340](https://issues.apache.org/jira/browse/HDFS-6340) | DN can't finalize upgrade | Blocker | datanode | Rahul Singhal | Rahul Singhal | +| [HDFS-6329](https://issues.apache.org/jira/browse/HDFS-6329) | WebHdfs does not work if HA is enabled on NN but logical URI is not configured. | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6326](https://issues.apache.org/jira/browse/HDFS-6326) | WebHdfs ACL compatibility is broken | Blocker | webhdfs | Daryn Sharp | Chris Nauroth | +| [HDFS-6325](https://issues.apache.org/jira/browse/HDFS-6325) | Append should fail if the last block has insufficient number of replicas | Major | namenode | Konstantin Shvachko | Keith Pak | +| [HDFS-6313](https://issues.apache.org/jira/browse/HDFS-6313) | WebHdfs may use the wrong NN when configured for multiple HA NNs | Blocker | webhdfs | Daryn Sharp | Kihwal Lee | +| [HDFS-6245](https://issues.apache.org/jira/browse/HDFS-6245) | datanode fails to start with a bad disk even when failed volumes is set | Major | . | Arpit Gupta | Arpit Agarwal | +| [HDFS-6236](https://issues.apache.org/jira/browse/HDFS-6236) | ImageServlet should use Time#monotonicNow to measure latency. | Minor | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-6235](https://issues.apache.org/jira/browse/HDFS-6235) | TestFileJournalManager can fail on Windows due to file locking if tests run out of order. | Trivial | namenode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-6234](https://issues.apache.org/jira/browse/HDFS-6234) | TestDatanodeConfig#testMemlockLimit fails on Windows due to invalid file path. | Trivial | datanode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-6232](https://issues.apache.org/jira/browse/HDFS-6232) | OfflineEditsViewer throws a NPE on edits containing ACL modifications | Major | tools | Stephen Chu | Akira AJISAKA | +| [HDFS-6231](https://issues.apache.org/jira/browse/HDFS-6231) | DFSClient hangs infinitely if using hedged reads and all eligible datanodes die. | Major | hdfs-client | Chris Nauroth | Chris Nauroth | +| [HDFS-6229](https://issues.apache.org/jira/browse/HDFS-6229) | Race condition in failover can cause RetryCache fail to work | Major | ha | Jing Zhao | Jing Zhao | +| [HDFS-6215](https://issues.apache.org/jira/browse/HDFS-6215) | Wrong error message for upgrade | Minor | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6209](https://issues.apache.org/jira/browse/HDFS-6209) | Fix flaky test TestValidateConfigurationSettings.testThatDifferentRPCandHttpPortsAreOK | Minor | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6208](https://issues.apache.org/jira/browse/HDFS-6208) | DataNode caching can leak file descriptors. | Major | datanode | Chris Nauroth | Chris Nauroth | +| [HDFS-6206](https://issues.apache.org/jira/browse/HDFS-6206) | DFSUtil.substituteForWildcardAddress may throw NPE | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6204](https://issues.apache.org/jira/browse/HDFS-6204) | TestRBWBlockInvalidation may fail | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6198](https://issues.apache.org/jira/browse/HDFS-6198) | DataNode rolling upgrade does not correctly identify current block pool directory and replace with trash on Windows. | Major | datanode | Chris Nauroth | Chris Nauroth | +| [HDFS-6197](https://issues.apache.org/jira/browse/HDFS-6197) | Rolling upgrade rollback on Windows can fail attempting to rename edit log segment files to a destination that already exists. | Minor | namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-2882](https://issues.apache.org/jira/browse/HDFS-2882) | DN continues to start up, even if block pool fails to initialize | Major | datanode | Todd Lipcon | Vinayakumar B | +| [MAPREDUCE-5841](https://issues.apache.org/jira/browse/MAPREDUCE-5841) | uber job doesn't terminate on getting mapred job kill | Major | mrv2 | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-5835](https://issues.apache.org/jira/browse/MAPREDUCE-5835) | Killing Task might cause the job to go to ERROR state | Critical | . | Ming Ma | Ming Ma | +| [MAPREDUCE-5832](https://issues.apache.org/jira/browse/MAPREDUCE-5832) | Few tests in TestJobClient fail on Windows | Major | . | Jian He | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5830](https://issues.apache.org/jira/browse/MAPREDUCE-5830) | HostUtil.getTaskLogUrl is not backwards binary compatible with 2.3 | Blocker | . | Jason Lowe | Akira AJISAKA | +| [MAPREDUCE-5828](https://issues.apache.org/jira/browse/MAPREDUCE-5828) | TestMapReduceJobControl fails on JDK 7 + Windows | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [MAPREDUCE-5827](https://issues.apache.org/jira/browse/MAPREDUCE-5827) | TestSpeculativeExecutionWithMRApp fails | Major | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-5826](https://issues.apache.org/jira/browse/MAPREDUCE-5826) | TestHistoryServerFileSystemStateStoreService.testTokenStore fails in windows | Major | . | Varun Vasudev | Varun Vasudev | +| [MAPREDUCE-5824](https://issues.apache.org/jira/browse/MAPREDUCE-5824) | TestPipesNonJavaInputFormat.testFormat fails in windows | Major | . | Xuan Gong | Xuan Gong | +| [MAPREDUCE-5821](https://issues.apache.org/jira/browse/MAPREDUCE-5821) | IFile merge allocates new byte array for every value | Major | performance, task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-5818](https://issues.apache.org/jira/browse/MAPREDUCE-5818) | hsadmin cmd is missing in mapred.cmd | Major | . | Jian He | Jian He | +| [MAPREDUCE-5815](https://issues.apache.org/jira/browse/MAPREDUCE-5815) | Fix NPE in TestMRAppMaster | Blocker | client, mrv2 | Gera Shegalov | Akira AJISAKA | +| [MAPREDUCE-5714](https://issues.apache.org/jira/browse/MAPREDUCE-5714) | TestMRAppComponentDependencies causes surefire to exit without saying proper goodbye | Major | test | Jinghui Wang | Jinghui Wang | +| [MAPREDUCE-3191](https://issues.apache.org/jira/browse/MAPREDUCE-3191) | docs for map output compression incorrectly reference SequenceFile | Trivial | . | Todd Lipcon | Chen He | +| [YARN-2081](https://issues.apache.org/jira/browse/YARN-2081) | TestDistributedShell fails after YARN-1962 | Minor | applications/distributed-shell | Hong Zhiguo | Hong Zhiguo | +| [YARN-2066](https://issues.apache.org/jira/browse/YARN-2066) | Wrong field is referenced in GetApplicationsRequestPBImpl#mergeLocalToBuilder() | Minor | . | Ted Yu | Hong Zhiguo | +| [YARN-2016](https://issues.apache.org/jira/browse/YARN-2016) | Yarn getApplicationRequest start time range is not honored | Major | resourcemanager | Venkat Ranganathan | Junping Du | +| [YARN-1986](https://issues.apache.org/jira/browse/YARN-1986) | In Fifo Scheduler, node heartbeat in between creating app and attempt causes NPE | Critical | . | Jon Bringhurst | Hong Zhiguo | +| [YARN-1976](https://issues.apache.org/jira/browse/YARN-1976) | Tracking url missing http protocol for FAILED application | Major | . | Yesha Vora | Junping Du | +| [YARN-1975](https://issues.apache.org/jira/browse/YARN-1975) | Used resources shows escaped html in CapacityScheduler and FairScheduler page | Major | resourcemanager | Nathan Roberts | Mit Desai | +| [YARN-1934](https://issues.apache.org/jira/browse/YARN-1934) | Potential NPE in ZKRMStateStore caused by handling Disconnected event from ZK. | Blocker | resourcemanager | Rohith Sharma K S | Karthik Kambatla | +| [YARN-1933](https://issues.apache.org/jira/browse/YARN-1933) | TestAMRestart and TestNodeHealthService failing sometimes on Windows | Major | . | Jian He | Jian He | +| [YARN-1932](https://issues.apache.org/jira/browse/YARN-1932) | Javascript injection on the job status page | Blocker | . | Mit Desai | Mit Desai | +| [YARN-1931](https://issues.apache.org/jira/browse/YARN-1931) | Private API change in YARN-1824 in 2.4 broke compatibility with previous releases | Blocker | applications | Thomas Graves | Sandy Ryza | +| [YARN-1929](https://issues.apache.org/jira/browse/YARN-1929) | DeadLock in RM when automatic failover is enabled. | Blocker | resourcemanager | Rohith Sharma K S | Karthik Kambatla | +| [YARN-1928](https://issues.apache.org/jira/browse/YARN-1928) | TestAMRMRPCNodeUpdates fails ocassionally | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1926](https://issues.apache.org/jira/browse/YARN-1926) | DistributedShell unit tests fail on Windows | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-1924](https://issues.apache.org/jira/browse/YARN-1924) | STATE\_STORE\_OP\_FAILED happens when ZKRMStateStore tries to update app(attempt) before storing it | Critical | . | Arpit Gupta | Jian He | +| [YARN-1920](https://issues.apache.org/jira/browse/YARN-1920) | TestFileSystemApplicationHistoryStore.testMissingApplicationAttemptHistoryData fails in windows | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1914](https://issues.apache.org/jira/browse/YARN-1914) | Test TestFSDownload.testDownloadPublicWithStatCache fails on Windows | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-1910](https://issues.apache.org/jira/browse/YARN-1910) | TestAMRMTokens fails on windows | Major | . | Xuan Gong | Xuan Gong | +| [YARN-1908](https://issues.apache.org/jira/browse/YARN-1908) | Distributed shell with custom script has permission error. | Major | applications/distributed-shell | Tassapol Athiapinya | Vinod Kumar Vavilapalli | +| [YARN-1907](https://issues.apache.org/jira/browse/YARN-1907) | TestRMApplicationHistoryWriter#testRMWritingMassiveHistory runs slow and intermittently fails | Major | . | Mit Desai | Mit Desai | +| [YARN-1903](https://issues.apache.org/jira/browse/YARN-1903) | Killing Container on NEW and LOCALIZING will result in exitCode and diagnostics not set | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1883](https://issues.apache.org/jira/browse/YARN-1883) | TestRMAdminService fails due to inconsistent entries in UserGroups | Major | . | Mit Desai | Mit Desai | +| [YARN-1837](https://issues.apache.org/jira/browse/YARN-1837) | TestMoveApplication.testMoveRejectedByScheduler randomly fails | Major | . | Tsuyoshi Ozawa | Hong Zhiguo | +| [YARN-1201](https://issues.apache.org/jira/browse/YARN-1201) | TestAMAuthorization fails with local hostname cannot be resolved | Minor | resourcemanager | Nemon Lou | Wangda Tan | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-6189](https://issues.apache.org/jira/browse/HDFS-6189) | Multiple HDFS tests fail on Windows attempting to use a test root path containing a colon. | Major | test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5843](https://issues.apache.org/jira/browse/MAPREDUCE-5843) | TestMRKeyValueTextInputFormat failing on Windows | Major | . | Varun Vasudev | Varun Vasudev | +| [MAPREDUCE-5833](https://issues.apache.org/jira/browse/MAPREDUCE-5833) | TestRMContainerAllocator fails ocassionally | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1947](https://issues.apache.org/jira/browse/YARN-1947) | TestRMDelegationTokens#testRMDTMasterKeyStateOnRollingMasterKey is failing intermittently | Major | . | Jian He | Jian He | +| [YARN-1905](https://issues.apache.org/jira/browse/YARN-1905) | TestProcfsBasedProcessTree must only run on Linux. | Trivial | nodemanager | Chris Nauroth | Chris Nauroth | +| [YARN-1750](https://issues.apache.org/jira/browse/YARN-1750) | TestNodeStatusUpdater#testNMRegistration is incorrect in test case | Major | nodemanager | Ming Ma | Wangda Tan | +| [YARN-1281](https://issues.apache.org/jira/browse/YARN-1281) | TestZKRMStateStoreZKClientConnections fails intermittently | Major | resourcemanager | Karthik Kambatla | Tsuyoshi Ozawa | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [YARN-2053](https://issues.apache.org/jira/browse/YARN-2053) | Slider AM fails to restart: NPE in RegisterApplicationMasterResponseProto$Builder.addAllNmTokensFromPreviousAttempts | Major | resourcemanager | Sumit Mohanty | Wangda Tan | +| [YARN-1962](https://issues.apache.org/jira/browse/YARN-1962) | Timeline server is enabled by default | Major | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [YARN-1957](https://issues.apache.org/jira/browse/YARN-1957) | ProportionalCapacitPreemptionPolicy handling of corner cases... | Major | resourcemanager | Carlo Curino | Carlo Curino | +| [YARN-1898](https://issues.apache.org/jira/browse/YARN-1898) | Standby RM's conf, stacks, logLevel, metrics, jmx and logs links are redirecting to Active RM | Major | resourcemanager | Yesha Vora | Xuan Gong | +| [YARN-1861](https://issues.apache.org/jira/browse/YARN-1861) | Both RM stuck in standby mode when automatic failover is enabled | Blocker | resourcemanager | Arpit Gupta | Karthik Kambatla | +| [YARN-1701](https://issues.apache.org/jira/browse/YARN-1701) | Improve default paths of timeline store and generic history store | Major | . | Gera Shegalov | Tsuyoshi Ozawa | +| [YARN-1696](https://issues.apache.org/jira/browse/YARN-1696) | Document RM HA | Blocker | resourcemanager | Karthik Kambatla | Tsuyoshi Ozawa | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/RELEASENOTES.2.4.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/RELEASENOTES.2.4.1.md new file mode 100644 index 00000000000..0425a58eee9 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.4.1/RELEASENOTES.2.4.1.md @@ -0,0 +1,31 @@ + + +# Apache Hadoop 2.4.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HDFS-6325](https://issues.apache.org/jira/browse/HDFS-6325) | *Major* | **Append should fail if the last block has insufficient number of replicas** + +I have committed the fix to the trunk, branch-2, and branch-2.4 respectively. Thanks Keith! + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/CHANGES.2.5.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/CHANGES.2.5.0.md new file mode 100644 index 00000000000..82f8878cc4c --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/CHANGES.2.5.0.md @@ -0,0 +1,551 @@ + + +# Apache Hadoop Changelog + +## Release 2.5.0 - 2014-08-11 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-6168](https://issues.apache.org/jira/browse/HDFS-6168) | Remove deprecated methods in DistributedFileSystem | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6164](https://issues.apache.org/jira/browse/HDFS-6164) | Remove lsr in OfflineImageViewer | Major | tools | Haohui Mai | Haohui Mai | +| [HDFS-6153](https://issues.apache.org/jira/browse/HDFS-6153) | Document "fileId" and "childrenNum" fields in the FileStatus Json schema | Minor | documentation, webhdfs | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5777](https://issues.apache.org/jira/browse/MAPREDUCE-5777) | Support utf-8 text with BOM (byte order marker) | Major | . | bc Wong | zhihai xu | +| [YARN-2107](https://issues.apache.org/jira/browse/YARN-2107) | Refactor timeline classes into server.timeline package | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10514](https://issues.apache.org/jira/browse/HADOOP-10514) | Common side changes to support HDFS extended attributes (HDFS-2006) | Major | fs | Uma Maheswara Rao G | Yi Liu | +| [HADOOP-10498](https://issues.apache.org/jira/browse/HADOOP-10498) | Add support for proxy server | Major | util | Daryn Sharp | Daryn Sharp | +| [HADOOP-9704](https://issues.apache.org/jira/browse/HADOOP-9704) | Write metrics sink plugin for Hadoop/Graphite | Major | . | Chu Tong | | +| [HDFS-6435](https://issues.apache.org/jira/browse/HDFS-6435) | Add support for specifying a static uid/gid mapping for the NFS gateway | Major | nfs | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6406](https://issues.apache.org/jira/browse/HDFS-6406) | Add capability for NFS gateway to reject connections from unprivileged ports | Major | nfs | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6281](https://issues.apache.org/jira/browse/HDFS-6281) | Provide option to use the NFS Gateway without having to use the Hadoop portmapper | Major | nfs | Aaron T. Myers | Aaron T. Myers | +| [YARN-1864](https://issues.apache.org/jira/browse/YARN-1864) | Fair Scheduler Dynamic Hierarchical User Queues | Major | scheduler | Ashwin Shankar | Ashwin Shankar | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10896](https://issues.apache.org/jira/browse/HADOOP-10896) | Update compatibility doc to capture visibility of un-annotated classes/ methods | Blocker | documentation | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-10782](https://issues.apache.org/jira/browse/HADOOP-10782) | Typo in DataChecksum classs | Trivial | . | Jingguo Yao | Jingguo Yao | +| [HADOOP-10767](https://issues.apache.org/jira/browse/HADOOP-10767) | Clean up unused code in Ls shell command. | Trivial | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-10747](https://issues.apache.org/jira/browse/HADOOP-10747) | Support configurable retries on SASL connection failures in RPC client. | Minor | ipc | Chris Nauroth | Chris Nauroth | +| [HADOOP-10691](https://issues.apache.org/jira/browse/HADOOP-10691) | Improve the readability of 'hadoop fs -help' | Minor | tools | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HADOOP-10688](https://issues.apache.org/jira/browse/HADOOP-10688) | Expose thread-level FileSystem StatisticsData | Major | fs | Sandy Ryza | Sandy Ryza | +| [HADOOP-10674](https://issues.apache.org/jira/browse/HADOOP-10674) | Rewrite the PureJavaCrc32 loop for performance improvement | Major | performance, util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10666](https://issues.apache.org/jira/browse/HADOOP-10666) | Remove Copyright /d/d/d/d Apache Software Foundation from the source files license header | Minor | documentation | Henry Saputra | Henry Saputra | +| [HADOOP-10665](https://issues.apache.org/jira/browse/HADOOP-10665) | Make Hadoop Authentication Handler loads case in-sensitive | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10618](https://issues.apache.org/jira/browse/HADOOP-10618) | Remove SingleNodeSetup.apt.vm | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10614](https://issues.apache.org/jira/browse/HADOOP-10614) | CBZip2InputStream is not threadsafe | Major | . | Xiangrui Meng | Xiangrui Meng | +| [HADOOP-10572](https://issues.apache.org/jira/browse/HADOOP-10572) | Example NFS mount command must pass noacl as it isn't supported by the server yet | Trivial | nfs | Harsh J | Harsh J | +| [HADOOP-10561](https://issues.apache.org/jira/browse/HADOOP-10561) | Copy command with preserve option should handle Xattrs | Major | fs | Uma Maheswara Rao G | Yi Liu | +| [HADOOP-10557](https://issues.apache.org/jira/browse/HADOOP-10557) | FsShell -cp -pa option for preserving extended ACLs | Major | fs | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10556](https://issues.apache.org/jira/browse/HADOOP-10556) | Add toLowerCase support to auth\_to\_local rules for service name | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10549](https://issues.apache.org/jira/browse/HADOOP-10549) | MAX\_SUBST and varPat should be final in Configuration.java | Major | conf | Gera Shegalov | Gera Shegalov | +| [HADOOP-10539](https://issues.apache.org/jira/browse/HADOOP-10539) | Provide backward compatibility for ProxyUsers.authorize() call | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10535](https://issues.apache.org/jira/browse/HADOOP-10535) | Make the retry numbers in ActiveStandbyElector configurable | Minor | . | Jing Zhao | Jing Zhao | +| [HADOOP-10458](https://issues.apache.org/jira/browse/HADOOP-10458) | swifts should throw FileAlreadyExistsException on attempt to overwrite file | Minor | fs | Steve Loughran | Steve Loughran | +| [HADOOP-10454](https://issues.apache.org/jira/browse/HADOOP-10454) | Provide FileContext version of har file system | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10451](https://issues.apache.org/jira/browse/HADOOP-10451) | Remove unused field and imports from SaslRpcServer | Trivial | security | Benoy Antony | Benoy Antony | +| [HADOOP-10376](https://issues.apache.org/jira/browse/HADOOP-10376) | Refactor refresh\*Protocols into a single generic refreshConfigProtocol | Minor | . | Chris Li | Chris Li | +| [HADOOP-10345](https://issues.apache.org/jira/browse/HADOOP-10345) | Sanitize the the inputs (groups and hosts) for the proxyuser configuration | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10322](https://issues.apache.org/jira/browse/HADOOP-10322) | Add ability to read principal names from a keytab | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-9968](https://issues.apache.org/jira/browse/HADOOP-9968) | ProxyUsers does not work with NetGroups | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-9495](https://issues.apache.org/jira/browse/HADOOP-9495) | Define behaviour of Seekable.seek(), write tests, fix all hadoop implementations for compliance | Major | fs | Steve Loughran | Steve Loughran | +| [HADOOP-9361](https://issues.apache.org/jira/browse/HADOOP-9361) | Strictly define the expected behavior of filesystem APIs and write tests to verify compliance | Blocker | fs, test | Steve Loughran | Steve Loughran | +| [HADOOP-8943](https://issues.apache.org/jira/browse/HADOOP-8943) | Support multiple group mapping providers | Major | security | Kai Zheng | Kai Zheng | +| [HADOOP-6350](https://issues.apache.org/jira/browse/HADOOP-6350) | Documenting Hadoop metrics | Major | documentation, metrics | Hong Tang | Akira AJISAKA | +| [HDFS-6620](https://issues.apache.org/jira/browse/HDFS-6620) | Snapshot docs should specify about preserve options with cp command | Major | namenode | Uma Maheswara Rao G | Stephen Chu | +| [HDFS-6603](https://issues.apache.org/jira/browse/HDFS-6603) | Add XAttr with ACL test | Minor | test | Stephen Chu | Stephen Chu | +| [HDFS-6595](https://issues.apache.org/jira/browse/HDFS-6595) | Configure the maximum threads allowed for balancing on datanodes | Minor | balancer & mover, datanode | Benoy Antony | Benoy Antony | +| [HDFS-6593](https://issues.apache.org/jira/browse/HDFS-6593) | Move SnapshotDiffInfo out of INodeDirectorySnapshottable | Minor | namenode, snapshots | Jing Zhao | Jing Zhao | +| [HDFS-6580](https://issues.apache.org/jira/browse/HDFS-6580) | FSNamesystem.mkdirsInt should call the getAuditFileInfo() wrapper | Major | namenode | Zhilei Xu | Zhilei Xu | +| [HDFS-6578](https://issues.apache.org/jira/browse/HDFS-6578) | add toString method to DatanodeStorage for easier debugging | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6558](https://issues.apache.org/jira/browse/HDFS-6558) | Missing '\n' in the description of dfsadmin -rollingUpgrade | Trivial | . | Akira AJISAKA | Chen He | +| [HDFS-6545](https://issues.apache.org/jira/browse/HDFS-6545) | Finalizing rolling upgrade can make NN unavailable for a long duration | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6529](https://issues.apache.org/jira/browse/HDFS-6529) | Trace logging for RemoteBlockReader2 to identify remote datanode and file being read | Minor | hdfs-client | Anubhav Dhoot | Anubhav Dhoot | +| [HDFS-6528](https://issues.apache.org/jira/browse/HDFS-6528) | Add XAttrs to TestOfflineImageViewer | Minor | test | Stephen Chu | Stephen Chu | +| [HDFS-6507](https://issues.apache.org/jira/browse/HDFS-6507) | Improve DFSAdmin to support HA cluster better | Major | tools | Zesheng Wu | Zesheng Wu | +| [HDFS-6503](https://issues.apache.org/jira/browse/HDFS-6503) | Fix typo of DFSAdmin restoreFailedStorage | Minor | tools | Zesheng Wu | Zesheng Wu | +| [HDFS-6499](https://issues.apache.org/jira/browse/HDFS-6499) | Use NativeIO#renameTo instead of File#renameTo in FileJournalManager | Major | namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6492](https://issues.apache.org/jira/browse/HDFS-6492) | Support create-time xattrs and atomically setting multiple xattrs | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-6463](https://issues.apache.org/jira/browse/HDFS-6463) | Clarify behavior of AclStorage#createFsPermissionForExtendedAcl in comments. | Trivial | namenode | Aaron T. Myers | Chris Nauroth | +| [HDFS-6460](https://issues.apache.org/jira/browse/HDFS-6460) | Ignore stale and decommissioned nodes in NetworkTopology#sortByDistance | Minor | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6453](https://issues.apache.org/jira/browse/HDFS-6453) | use Time#monotonicNow to avoid system clock reset | Major | datanode, namenode | Liang Xie | Liang Xie | +| [HDFS-6448](https://issues.apache.org/jira/browse/HDFS-6448) | BlockReaderLocalLegacy should set socket timeout based on conf.socketTimeout | Major | hdfs-client | Liang Xie | Liang Xie | +| [HDFS-6447](https://issues.apache.org/jira/browse/HDFS-6447) | balancer should timestamp the completion message | Trivial | balancer & mover | Allen Wittenauer | Juan Yu | +| [HDFS-6442](https://issues.apache.org/jira/browse/HDFS-6442) | Fix TestEditLogAutoroll and TestStandbyCheckpoints failure caused by port conficts | Minor | test | Zesheng Wu | Zesheng Wu | +| [HDFS-6433](https://issues.apache.org/jira/browse/HDFS-6433) | Replace BytesMoved class with AtomicLong | Major | balancer & mover | Benoy Antony | Benoy Antony | +| [HDFS-6432](https://issues.apache.org/jira/browse/HDFS-6432) | Add snapshot related APIs to webhdfs | Major | namenode, webhdfs | Suresh Srinivas | Jing Zhao | +| [HDFS-6416](https://issues.apache.org/jira/browse/HDFS-6416) | Use Time#monotonicNow in OpenFileCtx and OpenFileCtxCatch to avoid system clock bugs | Minor | nfs | Brandon Li | Abhiraj Butala | +| [HDFS-6403](https://issues.apache.org/jira/browse/HDFS-6403) | Add metrics for log warnings reported by JVM pauses | Major | datanode, namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6396](https://issues.apache.org/jira/browse/HDFS-6396) | Remove support for ACL feature from INodeSymlink | Minor | . | Andrew Wang | Charles Lamb | +| [HDFS-6375](https://issues.apache.org/jira/browse/HDFS-6375) | Listing extended attributes with the search permission | Major | namenode | Andrew Wang | Charles Lamb | +| [HDFS-6369](https://issues.apache.org/jira/browse/HDFS-6369) | Document that BlockReader#available() can return more bytes than are remaining in the block | Trivial | . | Ted Yu | Ted Yu | +| [HDFS-6356](https://issues.apache.org/jira/browse/HDFS-6356) | Fix typo in DatanodeLayoutVersion | Trivial | datanode | Tulasi G | Tulasi G | +| [HDFS-6334](https://issues.apache.org/jira/browse/HDFS-6334) | Client failover proxy provider for IP failover based NN HA | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6304](https://issues.apache.org/jira/browse/HDFS-6304) | Consolidate the logic of path resolution in FSDirectory | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6297](https://issues.apache.org/jira/browse/HDFS-6297) | Add CLI testcases to reflect new features of dfs and dfsadmin | Major | test | Dasha Boudnik | Dasha Boudnik | +| [HDFS-6295](https://issues.apache.org/jira/browse/HDFS-6295) | Add "decommissioning" state and node state filtering to dfsadmin | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-6282](https://issues.apache.org/jira/browse/HDFS-6282) | re-add testIncludeByRegistrationName | Minor | test | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6279](https://issues.apache.org/jira/browse/HDFS-6279) | Create new index page for JN / DN | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6278](https://issues.apache.org/jira/browse/HDFS-6278) | Create HTML5-based UI for SNN | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6273](https://issues.apache.org/jira/browse/HDFS-6273) | Config options to allow wildcard endpoints for namenode HTTP and HTTPS servers | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6269](https://issues.apache.org/jira/browse/HDFS-6269) | NameNode Audit Log should differentiate between webHDFS open and HDFS open. | Major | namenode, webhdfs | Eric Payne | Eric Payne | +| [HDFS-6268](https://issues.apache.org/jira/browse/HDFS-6268) | Better sorting in NetworkTopology#pseudoSortByDistance when no local node is found | Minor | . | Andrew Wang | Andrew Wang | +| [HDFS-6266](https://issues.apache.org/jira/browse/HDFS-6266) | Identify full path for a given INode | Major | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-6256](https://issues.apache.org/jira/browse/HDFS-6256) | Clean up ImageVisitor and SpotCheckImageVisitor | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6225](https://issues.apache.org/jira/browse/HDFS-6225) | Remove the o.a.h.hdfs.server.common.UpgradeStatusReport | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6191](https://issues.apache.org/jira/browse/HDFS-6191) | Disable quota checks when replaying edit log. | Major | namenode | Kihwal Lee | Kihwal Lee | +| [HDFS-6167](https://issues.apache.org/jira/browse/HDFS-6167) | Relocate the non-public API classes in the hdfs.client package | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6158](https://issues.apache.org/jira/browse/HDFS-6158) | Clean up dead code for OfflineImageViewer | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6110](https://issues.apache.org/jira/browse/HDFS-6110) | adding more slow action log in critical write path | Major | datanode | Liang Xie | Liang Xie | +| [HDFS-6109](https://issues.apache.org/jira/browse/HDFS-6109) | let sync\_file\_range() system call run in background | Major | datanode | Liang Xie | Liang Xie | +| [HDFS-6007](https://issues.apache.org/jira/browse/HDFS-6007) | Update documentation about short-circuit local reads | Minor | documentation | Masatake Iwasaki | | +| [HDFS-5693](https://issues.apache.org/jira/browse/HDFS-5693) | Few NN metrics data points were collected via JMX when NN is under heavy load | Major | namenode | Ming Ma | Ming Ma | +| [HDFS-5683](https://issues.apache.org/jira/browse/HDFS-5683) | Better audit log messages for caching operations | Major | namenode | Andrew Wang | Abhiraj Butala | +| [HDFS-5381](https://issues.apache.org/jira/browse/HDFS-5381) | ExtendedBlock#hashCode should use both blockId and block pool ID | Minor | federation | Colin Patrick McCabe | Benoy Antony | +| [HDFS-5196](https://issues.apache.org/jira/browse/HDFS-5196) | Provide more snapshot information in WebUI | Minor | snapshots | Haohui Mai | Shinichi Yamashita | +| [HDFS-5168](https://issues.apache.org/jira/browse/HDFS-5168) | BlockPlacementPolicy does not work for cross node group dependencies | Critical | namenode | Nikola Vujic | Nikola Vujic | +| [HDFS-2949](https://issues.apache.org/jira/browse/HDFS-2949) | HA: Add check to active state transition to prevent operator-induced split brain | Major | ha, namenode | Todd Lipcon | Rushabh S Shah | +| [HDFS-2006](https://issues.apache.org/jira/browse/HDFS-2006) | ability to support storing extended attributes per file | Major | namenode | dhruba borthakur | Yi Liu | +| [MAPREDUCE-5899](https://issues.apache.org/jira/browse/MAPREDUCE-5899) | Support incremental data copy in DistCp | Major | distcp | Jing Zhao | Jing Zhao | +| [MAPREDUCE-5896](https://issues.apache.org/jira/browse/MAPREDUCE-5896) | InputSplits should indicate which locations have the block cached in memory | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5886](https://issues.apache.org/jira/browse/MAPREDUCE-5886) | Allow wordcount example job to accept multiple input paths. | Minor | examples | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5861](https://issues.apache.org/jira/browse/MAPREDUCE-5861) | finishedSubMaps field in LocalContainerLauncher does not need to be volatile | Minor | . | Ted Yu | Tsuyoshi Ozawa | +| [MAPREDUCE-5825](https://issues.apache.org/jira/browse/MAPREDUCE-5825) | Provide diagnostics for reducers killed during ramp down | Major | mr-am | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5812](https://issues.apache.org/jira/browse/MAPREDUCE-5812) | Make job context available to OutputCommitter.isRecoverySupported() | Major | mr-am | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [MAPREDUCE-5809](https://issues.apache.org/jira/browse/MAPREDUCE-5809) | Enhance distcp to support preserving HDFS ACLs. | Major | distcp | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5774](https://issues.apache.org/jira/browse/MAPREDUCE-5774) | Job overview in History UI should list reducer phases in chronological order | Trivial | jobhistoryserver | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5402](https://issues.apache.org/jira/browse/MAPREDUCE-5402) | DynamicInputFormat should allow overriding of MAX\_CHUNKS\_TOLERABLE | Major | distcp, mrv2 | David Rosenstrauch | Tsuyoshi Ozawa | +| [MAPREDUCE-5014](https://issues.apache.org/jira/browse/MAPREDUCE-5014) | Extending DistCp through a custom CopyListing is not possible | Major | distcp | Srikanth Sundarrajan | Srikanth Sundarrajan | +| [YARN-2335](https://issues.apache.org/jira/browse/YARN-2335) | Annotate all hadoop-sls APIs as @Private | Minor | . | Wei Yan | Wei Yan | +| [YARN-2300](https://issues.apache.org/jira/browse/YARN-2300) | Document better sample requests for RM web services for submitting apps | Major | documentation | Varun Vasudev | Varun Vasudev | +| [YARN-2195](https://issues.apache.org/jira/browse/YARN-2195) | Clean a piece of code in ResourceRequest | Trivial | . | Wei Yan | Wei Yan | +| [YARN-2159](https://issues.apache.org/jira/browse/YARN-2159) | Better logging in SchedulerNode#allocateContainer | Trivial | resourcemanager | Ray Chiang | Ray Chiang | +| [YARN-2089](https://issues.apache.org/jira/browse/YARN-2089) | FairScheduler: QueuePlacementPolicy and QueuePlacementRule are missing audience annotations | Major | scheduler | Anubhav Dhoot | zhihai xu | +| [YARN-2072](https://issues.apache.org/jira/browse/YARN-2072) | RM/NM UIs and webservices are missing vcore information | Major | nodemanager, resourcemanager, webapp | Nathan Roberts | Nathan Roberts | +| [YARN-2061](https://issues.apache.org/jira/browse/YARN-2061) | Revisit logging levels in ZKRMStateStore | Minor | resourcemanager | Karthik Kambatla | Ray Chiang | +| [YARN-2030](https://issues.apache.org/jira/browse/YARN-2030) | Use StateMachine to simplify handleStoreEvent() in RMStateStore | Major | . | Junping Du | Binglin Chang | +| [YARN-2012](https://issues.apache.org/jira/browse/YARN-2012) | Fair Scheduler: allow default queue placement rule to take an arbitrary queue | Major | scheduler | Ashwin Shankar | Ashwin Shankar | +| [YARN-1987](https://issues.apache.org/jira/browse/YARN-1987) | Wrapper for leveldb DBIterator to aid in handling database exceptions | Major | . | Jason Lowe | Jason Lowe | +| [YARN-1923](https://issues.apache.org/jira/browse/YARN-1923) | Make FairScheduler resource ratio calculations terminate faster | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-1889](https://issues.apache.org/jira/browse/YARN-1889) | In Fair Scheduler, avoid creating objects on each call to AppSchedulable comparator | Minor | scheduler | Hong Zhiguo | Hong Zhiguo | +| [YARN-1870](https://issues.apache.org/jira/browse/YARN-1870) | FileInputStream is not closed in ProcfsBasedProcessTree#constructProcessSMAPInfo() | Minor | resourcemanager | Ted Yu | Fengdong Yu | +| [YARN-1845](https://issues.apache.org/jira/browse/YARN-1845) | Elapsed time for failed tasks that never started is wrong | Major | . | Rushabh S Shah | Rushabh S Shah | +| [YARN-1751](https://issues.apache.org/jira/browse/YARN-1751) | Improve MiniYarnCluster for log aggregation testing | Major | nodemanager | Ming Ma | Ming Ma | +| [YARN-1561](https://issues.apache.org/jira/browse/YARN-1561) | Fix a generic type warning in FairScheduler | Minor | scheduler | Junping Du | Chen He | +| [YARN-1479](https://issues.apache.org/jira/browse/YARN-1479) | Invalid NaN values in Hadoop REST API JSON response | Major | . | Kendall Thrapp | Chen He | +| [YARN-1424](https://issues.apache.org/jira/browse/YARN-1424) | RMAppAttemptImpl should return the DummyApplicationResourceUsageReport for all invalid accesses | Minor | resourcemanager | Sandy Ryza | Ray Chiang | +| [YARN-614](https://issues.apache.org/jira/browse/YARN-614) | Separate AM failures from hardware failure or YARN error and do not count them to AM retry count | Major | resourcemanager | Bikas Saha | Xuan Gong | +| [YARN-483](https://issues.apache.org/jira/browse/YARN-483) | Improve documentation on log aggregation in yarn-default.xml | Major | documentation | Sandy Ryza | Akira AJISAKA | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10910](https://issues.apache.org/jira/browse/HADOOP-10910) | Increase findbugs maxHeap size | Blocker | . | Andrew Wang | Andrew Wang | +| [HADOOP-10890](https://issues.apache.org/jira/browse/HADOOP-10890) | TestDFVariations.testMount fails intermittently | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-10872](https://issues.apache.org/jira/browse/HADOOP-10872) | TestPathData fails intermittently with "Mkdirs failed to create d1" | Major | fs | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-10801](https://issues.apache.org/jira/browse/HADOOP-10801) | Fix dead link in site.xml | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10746](https://issues.apache.org/jira/browse/HADOOP-10746) | TestSocketIOWithTimeout#testSocketIOWithTimeout fails on Power PC | Major | test | Jinghui Wang | Jinghui Wang | +| [HADOOP-10739](https://issues.apache.org/jira/browse/HADOOP-10739) | Renaming a file into a directory containing the same filename results in a confusing I/O error | Major | fs | Jason Lowe | Chang Li | +| [HADOOP-10737](https://issues.apache.org/jira/browse/HADOOP-10737) | S3n silent failure on copy, data loss on rename | Major | fs/s3 | Gian Merlino | Steve Loughran | +| [HADOOP-10716](https://issues.apache.org/jira/browse/HADOOP-10716) | Cannot use more than 1 har filesystem | Critical | conf, fs | Daryn Sharp | Rushabh S Shah | +| [HADOOP-10711](https://issues.apache.org/jira/browse/HADOOP-10711) | Cleanup some extra dependencies from hadoop-auth | Major | security | Robert Kanter | Robert Kanter | +| [HADOOP-10710](https://issues.apache.org/jira/browse/HADOOP-10710) | hadoop.auth cookie is not properly constructed according to RFC2109 | Major | security | Alejandro Abdelnur | Juan Yu | +| [HADOOP-10702](https://issues.apache.org/jira/browse/HADOOP-10702) | KerberosAuthenticationHandler does not log the principal names correctly | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10701](https://issues.apache.org/jira/browse/HADOOP-10701) | NFS should not validate the access premission only based on the user's primary group | Major | nfs | Premchandra Preetham Kukillaya | Harsh J | +| [HADOOP-10699](https://issues.apache.org/jira/browse/HADOOP-10699) | Fix build native library on mac osx | Major | . | Kirill A. Korinskiy | Binglin Chang | +| [HADOOP-10686](https://issues.apache.org/jira/browse/HADOOP-10686) | Writables are not always configured | Major | . | Abraham Elmahrek | Abraham Elmahrek | +| [HADOOP-10683](https://issues.apache.org/jira/browse/HADOOP-10683) | Users authenticated with KERBEROS are recorded as being authenticated with SIMPLE | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10678](https://issues.apache.org/jira/browse/HADOOP-10678) | SecurityUtil has unnecessary synchronization on collection used for only tests | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10664](https://issues.apache.org/jira/browse/HADOOP-10664) | TestNetUtils.testNormalizeHostName fails | Major | . | Chen He | Aaron T. Myers | +| [HADOOP-10660](https://issues.apache.org/jira/browse/HADOOP-10660) | GraphiteSink should implement Closeable | Major | . | Ted Yu | Chen He | +| [HADOOP-10658](https://issues.apache.org/jira/browse/HADOOP-10658) | SSLFactory expects truststores being configured | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10657](https://issues.apache.org/jira/browse/HADOOP-10657) | Have RetryInvocationHandler log failover attempt at INFO level | Major | . | Ming Ma | Ming Ma | +| [HADOOP-10656](https://issues.apache.org/jira/browse/HADOOP-10656) | The password keystore file is not picked by LDAP group mapping | Major | security | Brandon Li | Brandon Li | +| [HADOOP-10647](https://issues.apache.org/jira/browse/HADOOP-10647) | String Format Exception in SwiftNativeFileSystemStore.java | Minor | fs/swift | Gene Kim | Gene Kim | +| [HADOOP-10639](https://issues.apache.org/jira/browse/HADOOP-10639) | FileBasedKeyStoresFactory initialization is not using default for SSL\_REQUIRE\_CLIENT\_CERT\_KEY | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10638](https://issues.apache.org/jira/browse/HADOOP-10638) | Updating hadoop-daemon.sh to work as expected when nfs is started as a privileged user. | Major | nfs | Manikandan Narayanaswamy | Manikandan Narayanaswamy | +| [HADOOP-10630](https://issues.apache.org/jira/browse/HADOOP-10630) | Possible race condition in RetryInvocationHandler | Major | . | Jing Zhao | Jing Zhao | +| [HADOOP-10625](https://issues.apache.org/jira/browse/HADOOP-10625) | Configuration: names should be trimmed when putting/getting to properties | Major | conf | Wangda Tan | Wangda Tan | +| [HADOOP-10622](https://issues.apache.org/jira/browse/HADOOP-10622) | Shell.runCommand can deadlock | Critical | . | Jason Lowe | Gera Shegalov | +| [HADOOP-10609](https://issues.apache.org/jira/browse/HADOOP-10609) | .gitignore should ignore .orig and .rej files | Major | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-10602](https://issues.apache.org/jira/browse/HADOOP-10602) | Documentation has broken "Go Back" hyperlinks. | Trivial | documentation | Chris Nauroth | Akira AJISAKA | +| [HADOOP-10590](https://issues.apache.org/jira/browse/HADOOP-10590) | ServiceAuthorizationManager is not threadsafe | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10589](https://issues.apache.org/jira/browse/HADOOP-10589) | NativeS3FileSystem throw NullPointerException when the file is empty | Major | fs/s3 | shuisheng wei | Steve Loughran | +| [HADOOP-10588](https://issues.apache.org/jira/browse/HADOOP-10588) | Workaround for jetty6 acceptor startup issue | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10585](https://issues.apache.org/jira/browse/HADOOP-10585) | Retry polices ignore interrupted exceptions | Critical | ipc | Daryn Sharp | Daryn Sharp | +| [HADOOP-10581](https://issues.apache.org/jira/browse/HADOOP-10581) | TestUserGroupInformation#testGetServerSideGroups fails because groups stored in Set and ArrayList are compared | Major | . | Mit Desai | Mit Desai | +| [HADOOP-10568](https://issues.apache.org/jira/browse/HADOOP-10568) | Add s3 server-side encryption | Major | fs/s3 | David S. Wang | David S. Wang | +| [HADOOP-10547](https://issues.apache.org/jira/browse/HADOOP-10547) | Give SaslPropertiesResolver.getDefaultProperties() public scope | Major | security | Jason Dere | Benoy Antony | +| [HADOOP-10543](https://issues.apache.org/jira/browse/HADOOP-10543) | RemoteException's unwrapRemoteException method failed for PathIOException | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-10541](https://issues.apache.org/jira/browse/HADOOP-10541) | InputStream in MiniKdc#initKDCServer for minikdc.ldiff is not closed | Minor | test | Ted Yu | Swarnim Kulkarni | +| [HADOOP-10540](https://issues.apache.org/jira/browse/HADOOP-10540) | Datanode upgrade in Windows fails with hardlink error. | Major | tools | Huan Huang | Arpit Agarwal | +| [HADOOP-10533](https://issues.apache.org/jira/browse/HADOOP-10533) | S3 input stream NPEs in MapReduce job | Minor | fs/s3 | Benjamin Kim | Steve Loughran | +| [HADOOP-10531](https://issues.apache.org/jira/browse/HADOOP-10531) | hadoop-config.sh - bug in --hosts argument | Major | . | Sebastien Barrier | Sebastien Barrier | +| [HADOOP-10526](https://issues.apache.org/jira/browse/HADOOP-10526) | Chance for Stream leakage in CompressorStream | Minor | . | SreeHari | Rushabh S Shah | +| [HADOOP-10517](https://issues.apache.org/jira/browse/HADOOP-10517) | InputStream is not closed in two methods of JarFinder | Minor | test, util | Ted Yu | Ted Yu | +| [HADOOP-10508](https://issues.apache.org/jira/browse/HADOOP-10508) | RefreshCallQueue fails when authorization is enabled | Major | ipc | Chris Li | Chris Li | +| [HADOOP-10500](https://issues.apache.org/jira/browse/HADOOP-10500) | TestDoAsEffectiveUser fails on JDK7 due to failure to reset proxy user configuration. | Trivial | security, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-10496](https://issues.apache.org/jira/browse/HADOOP-10496) | Metrics system FileSink can leak file descriptor. | Major | metrics | Chris Nauroth | Chris Nauroth | +| [HADOOP-10495](https://issues.apache.org/jira/browse/HADOOP-10495) | TestFileUtil fails on Windows due to bad permission assertions. | Trivial | fs, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-10489](https://issues.apache.org/jira/browse/HADOOP-10489) | UserGroupInformation#getTokens and UserGroupInformation#addToken can lead to ConcurrentModificationException | Major | . | Jing Zhao | Robert Kanter | +| [HADOOP-10475](https://issues.apache.org/jira/browse/HADOOP-10475) | ConcurrentModificationException in AbstractDelegationTokenSelector.selectToken() | Major | security | Arpit Gupta | Jing Zhao | +| [HADOOP-10468](https://issues.apache.org/jira/browse/HADOOP-10468) | TestMetricsSystemImpl.testMultiThreadedPublish fails intermediately | Blocker | . | Haohui Mai | Akira AJISAKA | +| [HADOOP-10462](https://issues.apache.org/jira/browse/HADOOP-10462) | DF#getFilesystem is not parsing the command output | Major | . | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10459](https://issues.apache.org/jira/browse/HADOOP-10459) | distcp V2 doesn't preserve root dir's attributes when -p is specified | Major | tools/distcp | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-10419](https://issues.apache.org/jira/browse/HADOOP-10419) | BufferedFSInputStream NPEs on getPos() on a closed stream | Minor | fs | Steve Loughran | Steve Loughran | +| [HADOOP-10418](https://issues.apache.org/jira/browse/HADOOP-10418) | SaslRpcClient should not assume that remote principals are in the default\_realm | Major | security | Aaron T. Myers | Aaron T. Myers | +| [HADOOP-10414](https://issues.apache.org/jira/browse/HADOOP-10414) | Incorrect property name for RefreshUserMappingProtocol in hadoop-policy.xml | Major | conf | Joey Echeverria | Joey Echeverria | +| [HADOOP-10401](https://issues.apache.org/jira/browse/HADOOP-10401) | ShellBasedUnixGroupsMapping#getGroups does not always return primary group first | Major | util | Colin Patrick McCabe | Akira AJISAKA | +| [HADOOP-10378](https://issues.apache.org/jira/browse/HADOOP-10378) | Typo in help printed by hdfs dfs -help | Major | . | Mit Desai | Mit Desai | +| [HADOOP-10350](https://issues.apache.org/jira/browse/HADOOP-10350) | BUILDING.txt should mention openssl dependency required for hadoop-pipes | Major | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-10342](https://issues.apache.org/jira/browse/HADOOP-10342) | Extend UserGroupInformation to return a UGI given a preauthenticated kerberos Subject | Major | security | Larry McCay | Larry McCay | +| [HADOOP-10332](https://issues.apache.org/jira/browse/HADOOP-10332) | HttpServer's jetty audit log always logs 200 OK | Major | . | Daryn Sharp | Jonathan Eagles | +| [HADOOP-10312](https://issues.apache.org/jira/browse/HADOOP-10312) | Shell.ExitCodeException to have more useful toString | Minor | util | Steve Loughran | Steve Loughran | +| [HADOOP-10251](https://issues.apache.org/jira/browse/HADOOP-10251) | Both NameNodes could be in STANDBY State if SNN network is unstable | Critical | ha | Vinayakumar B | Vinayakumar B | +| [HADOOP-10158](https://issues.apache.org/jira/browse/HADOOP-10158) | SPNEGO should work with multiple interfaces/SPNs. | Critical | . | Kihwal Lee | Daryn Sharp | +| [HADOOP-9919](https://issues.apache.org/jira/browse/HADOOP-9919) | Update hadoop-metrics2.properties examples to Yarn | Major | conf | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-9705](https://issues.apache.org/jira/browse/HADOOP-9705) | FsShell cp -p does not preserve directory attibutes | Major | fs | Stephen Chu | Akira AJISAKA | +| [HADOOP-9559](https://issues.apache.org/jira/browse/HADOOP-9559) | When metrics system is restarted MBean names get incorrectly flagged as dupes | Major | metrics | Mostafa Elhemali | Mike Liddell | +| [HADOOP-9555](https://issues.apache.org/jira/browse/HADOOP-9555) | HA functionality that uses ZooKeeper may experience inadvertent TCP RST and miss session expiration event due to bug in client connection management | Major | ha | Chris Nauroth | Chris Nauroth | +| [HADOOP-9099](https://issues.apache.org/jira/browse/HADOOP-9099) | NetUtils.normalizeHostName fails on domains where UnknownHost resolves to an IP address | Minor | test | Ivan Mitic | Ivan Mitic | +| [HDFS-6793](https://issues.apache.org/jira/browse/HDFS-6793) | Missing changes in HftpFileSystem when Reintroduce dfs.http.port / dfs.https.port in branch-2 | Blocker | . | Juan Yu | Juan Yu | +| [HDFS-6752](https://issues.apache.org/jira/browse/HDFS-6752) | Avoid Address bind errors in TestDatanodeConfig#testMemlockLimit | Major | test | Vinayakumar B | Vinayakumar B | +| [HDFS-6723](https://issues.apache.org/jira/browse/HDFS-6723) | New NN webUI no longer displays decommissioned state for dead node | Major | . | Ming Ma | Ming Ma | +| [HDFS-6712](https://issues.apache.org/jira/browse/HDFS-6712) | Document HDFS Multihoming Settings | Major | documentation | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6703](https://issues.apache.org/jira/browse/HDFS-6703) | NFS: Files can be deleted from a read-only mount | Major | nfs | Abhiraj Butala | Srikanth Upputuri | +| [HDFS-6696](https://issues.apache.org/jira/browse/HDFS-6696) | Name node cannot start if the path of a file under construction contains ".snapshot" | Blocker | . | Kihwal Lee | Andrew Wang | +| [HDFS-6680](https://issues.apache.org/jira/browse/HDFS-6680) | BlockPlacementPolicyDefault does not choose favored nodes correctly | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6647](https://issues.apache.org/jira/browse/HDFS-6647) | Edit log corruption when pipeline recovery occurs for deleted file present in snapshot | Blocker | namenode, snapshots | Aaron T. Myers | Kihwal Lee | +| [HDFS-6632](https://issues.apache.org/jira/browse/HDFS-6632) | Reintroduce dfs.http.port / dfs.https.port in branch-2 | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6631](https://issues.apache.org/jira/browse/HDFS-6631) | TestPread#testHedgedReadLoopTooManyTimes fails intermittently. | Major | hdfs-client, test | Chris Nauroth | Liang Xie | +| [HDFS-6622](https://issues.apache.org/jira/browse/HDFS-6622) | Rename and AddBlock may race and produce invalid edits | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6618](https://issues.apache.org/jira/browse/HDFS-6618) | FSNamesystem#delete drops the FSN lock between removing INodes from the tree and deleting them from the inode map | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6612](https://issues.apache.org/jira/browse/HDFS-6612) | MiniDFSNNTopology#simpleFederatedTopology(int) always hardcode nameservice ID | Minor | . | Juan Yu | Juan Yu | +| [HDFS-6610](https://issues.apache.org/jira/browse/HDFS-6610) | TestShortCircuitLocalRead tests sometimes timeout on slow machines | Minor | test | Charles Lamb | Charles Lamb | +| [HDFS-6604](https://issues.apache.org/jira/browse/HDFS-6604) | The short-circuit cache doesn't correctly time out replicas that haven't been used in a while | Critical | hdfs-client | Giuseppe Reina | Colin Patrick McCabe | +| [HDFS-6601](https://issues.apache.org/jira/browse/HDFS-6601) | Issues in finalizing rolling upgrade when there is a layout version change | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6599](https://issues.apache.org/jira/browse/HDFS-6599) | 2.4 addBlock is 10 to 20 times slower compared to 0.23 | Blocker | . | Kihwal Lee | Daryn Sharp | +| [HDFS-6598](https://issues.apache.org/jira/browse/HDFS-6598) | Fix a typo in message issued from explorer.js | Trivial | webhdfs | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6591](https://issues.apache.org/jira/browse/HDFS-6591) | while loop is executed tens of thousands of times in Hedged Read | Major | hdfs-client | LiuLei | Liang Xie | +| [HDFS-6587](https://issues.apache.org/jira/browse/HDFS-6587) | Bug in TestBPOfferService can cause test failure | Major | test | Zhilei Xu | Zhilei Xu | +| [HDFS-6583](https://issues.apache.org/jira/browse/HDFS-6583) | Remove clientNode in FileUnderConstructionFeature | Minor | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6572](https://issues.apache.org/jira/browse/HDFS-6572) | Add an option to the NameNode that prints the software and on-disk image versions | Minor | namenode | Charles Lamb | Charles Lamb | +| [HDFS-6563](https://issues.apache.org/jira/browse/HDFS-6563) | NameNode cannot save fsimage in certain circumstances when snapshots are in use | Critical | namenode, snapshots | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6559](https://issues.apache.org/jira/browse/HDFS-6559) | Fix wrong option "dfsadmin -rollingUpgrade start" in the document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6556](https://issues.apache.org/jira/browse/HDFS-6556) | Refine XAttr permissions | Major | namenode | Yi Liu | Uma Maheswara Rao G | +| [HDFS-6553](https://issues.apache.org/jira/browse/HDFS-6553) | Add missing DeprecationDeltas for NFS Kerberos configurations | Major | nfs | Stephen Chu | Stephen Chu | +| [HDFS-6552](https://issues.apache.org/jira/browse/HDFS-6552) | add DN storage to a BlockInfo will not replace the different storage from same DN | Trivial | namenode | Amir Langer | Amir Langer | +| [HDFS-6551](https://issues.apache.org/jira/browse/HDFS-6551) | Rename with OVERWRITE option may throw NPE when the target file/directory is a reference INode | Major | namenode, snapshots | Jing Zhao | Jing Zhao | +| [HDFS-6549](https://issues.apache.org/jira/browse/HDFS-6549) | Add support for accessing the NFS gateway from the AIX NFS client | Major | nfs | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6539](https://issues.apache.org/jira/browse/HDFS-6539) | test\_native\_mini\_dfs is skipped in hadoop-hdfs/pom.xml | Major | . | Binglin Chang | Binglin Chang | +| [HDFS-6535](https://issues.apache.org/jira/browse/HDFS-6535) | HDFS quota update is wrong when file is appended | Major | namenode | George Wong | George Wong | +| [HDFS-6530](https://issues.apache.org/jira/browse/HDFS-6530) | Fix Balancer documentation | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6527](https://issues.apache.org/jira/browse/HDFS-6527) | Edit log corruption due to defered INode removal | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6518](https://issues.apache.org/jira/browse/HDFS-6518) | TestCacheDirectives#testExceedsCapacity should take FSN read lock when accessing pendingCached list | Major | . | Yongjun Zhang | Andrew Wang | +| [HDFS-6500](https://issues.apache.org/jira/browse/HDFS-6500) | Snapshot shouldn't be removed silently after renaming to an existing snapshot | Blocker | snapshots | Junping Du | Tsz Wo Nicholas Sze | +| [HDFS-6497](https://issues.apache.org/jira/browse/HDFS-6497) | Make TestAvailableSpaceVolumeChoosingPolicy deterministic | Minor | test | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6493](https://issues.apache.org/jira/browse/HDFS-6493) | Change dfs.namenode.startup.delay.block.deletion to second instead of millisecond | Trivial | . | Juan Yu | Juan Yu | +| [HDFS-6487](https://issues.apache.org/jira/browse/HDFS-6487) | TestStandbyCheckpoint#testSBNCheckpoints is racy | Major | . | Mit Desai | Mit Desai | +| [HDFS-6475](https://issues.apache.org/jira/browse/HDFS-6475) | WebHdfs clients fail without retry because incorrect handling of StandbyException | Major | ha, webhdfs | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6472](https://issues.apache.org/jira/browse/HDFS-6472) | fix typo in webapps/hdfs/explorer.js | Trivial | . | Juan Yu | Juan Yu | +| [HDFS-6471](https://issues.apache.org/jira/browse/HDFS-6471) | Make moveFromLocal CLI testcases to be non-disruptive | Major | test | Dasha Boudnik | Dasha Boudnik | +| [HDFS-6470](https://issues.apache.org/jira/browse/HDFS-6470) | TestBPOfferService.testBPInitErrorHandling is flaky | Major | . | Andrew Wang | Ming Ma | +| [HDFS-6464](https://issues.apache.org/jira/browse/HDFS-6464) | Support multiple xattr.name parameters for WebHDFS getXAttrs. | Major | webhdfs | Yi Liu | Yi Liu | +| [HDFS-6462](https://issues.apache.org/jira/browse/HDFS-6462) | NFS: fsstat request fails with the secure hdfs | Major | nfs | Yesha Vora | Brandon Li | +| [HDFS-6461](https://issues.apache.org/jira/browse/HDFS-6461) | Use Time#monotonicNow to compute duration in DataNode#shutDown | Trivial | datanode | James Thomas | James Thomas | +| [HDFS-6443](https://issues.apache.org/jira/browse/HDFS-6443) | Fix MiniQJMHACluster related test failures | Minor | test | Zesheng Wu | Zesheng Wu | +| [HDFS-6439](https://issues.apache.org/jira/browse/HDFS-6439) | NFS should not reject NFS requests to the NULL procedure whether port monitoring is enabled or not | Major | nfs | Brandon Li | Aaron T. Myers | +| [HDFS-6438](https://issues.apache.org/jira/browse/HDFS-6438) | DeleteSnapshot should be a DELETE request in WebHdfs | Major | webhdfs | Jing Zhao | Jing Zhao | +| [HDFS-6424](https://issues.apache.org/jira/browse/HDFS-6424) | blockReport doesn't need to invalidate blocks on SBN | Major | . | Ming Ma | Ming Ma | +| [HDFS-6423](https://issues.apache.org/jira/browse/HDFS-6423) | Diskspace quota usage should be updated when appending data to partial block | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-6422](https://issues.apache.org/jira/browse/HDFS-6422) | getfattr in CLI doesn't throw exception or return non-0 return code when xattr doesn't exist | Blocker | . | Charles Lamb | Charles Lamb | +| [HDFS-6421](https://issues.apache.org/jira/browse/HDFS-6421) | Fix vecsum.c compile on BSD and some other systems | Major | libhdfs | Jason Lowe | Mit Desai | +| [HDFS-6418](https://issues.apache.org/jira/browse/HDFS-6418) | Regression: DFS\_NAMENODE\_USER\_NAME\_KEY missing in trunk | Blocker | hdfs-client | Steve Loughran | Tsz Wo Nicholas Sze | +| [HDFS-6409](https://issues.apache.org/jira/browse/HDFS-6409) | Fix typo in log message about NameNode layout version upgrade. | Trivial | namenode | Chris Nauroth | Chen He | +| [HDFS-6404](https://issues.apache.org/jira/browse/HDFS-6404) | HttpFS should use a 000 umask for mkdir and create operations | Major | . | Alejandro Abdelnur | Mike Yoder | +| [HDFS-6400](https://issues.apache.org/jira/browse/HDFS-6400) | Cannot execute "hdfs oiv\_legacy" | Critical | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6399](https://issues.apache.org/jira/browse/HDFS-6399) | Add note about setfacl in HDFS permissions guide | Minor | documentation, namenode | Charles Lamb | Chris Nauroth | +| [HDFS-6395](https://issues.apache.org/jira/browse/HDFS-6395) | Skip checking xattr limits for non-user-visible namespaces | Major | namenode | Andrew Wang | Yi Liu | +| [HDFS-6381](https://issues.apache.org/jira/browse/HDFS-6381) | Fix a typo in INodeReference.java | Trivial | documentation | Binglin Chang | Binglin Chang | +| [HDFS-6379](https://issues.apache.org/jira/browse/HDFS-6379) | HTTPFS - Implement ACLs support | Major | . | Alejandro Abdelnur | Mike Yoder | +| [HDFS-6378](https://issues.apache.org/jira/browse/HDFS-6378) | NFS registration should timeout instead of hanging when portmap/rpcbind is not available | Major | nfs | Brandon Li | Abhiraj Butala | +| [HDFS-6370](https://issues.apache.org/jira/browse/HDFS-6370) | Web UI fails to display in intranet under IE | Major | datanode, journal-node, namenode | Haohui Mai | Haohui Mai | +| [HDFS-6367](https://issues.apache.org/jira/browse/HDFS-6367) | EnumSetParam$Domain#parse fails for parameter containing more than one enum. | Major | webhdfs | Yi Liu | Yi Liu | +| [HDFS-6364](https://issues.apache.org/jira/browse/HDFS-6364) | Incorrect check for unknown datanode in Balancer | Major | balancer & mover | Benoy Antony | Benoy Antony | +| [HDFS-6355](https://issues.apache.org/jira/browse/HDFS-6355) | Fix divide-by-zero, improper use of wall-clock time in BlockPoolSliceScanner | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6345](https://issues.apache.org/jira/browse/HDFS-6345) | DFS.listCacheDirectives() should allow filtering based on cache directive ID | Major | caching | Lenni Kuff | Andrew Wang | +| [HDFS-6337](https://issues.apache.org/jira/browse/HDFS-6337) | Setfacl testcase is failing due to dash character in username in TestAclCLI | Major | test | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-6312](https://issues.apache.org/jira/browse/HDFS-6312) | WebHdfs HA failover is broken on secure clusters | Blocker | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6305](https://issues.apache.org/jira/browse/HDFS-6305) | WebHdfs response decoding may throw RuntimeExceptions | Critical | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6294](https://issues.apache.org/jira/browse/HDFS-6294) | Use INode IDs to avoid conflicts when a file open for write is renamed | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6293](https://issues.apache.org/jira/browse/HDFS-6293) | Issues with OIV processing PB-based fsimages | Blocker | . | Kihwal Lee | Kihwal Lee | +| [HDFS-6289](https://issues.apache.org/jira/browse/HDFS-6289) | HA failover can fail if there are pending DN messages for DNs which no longer exist | Critical | ha | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6288](https://issues.apache.org/jira/browse/HDFS-6288) | DFSInputStream Pread doesn't update ReadStatistics | Minor | . | Juan Yu | Juan Yu | +| [HDFS-6270](https://issues.apache.org/jira/browse/HDFS-6270) | Secondary namenode status page shows transaction count in bytes | Minor | . | Benoy Antony | Benoy Antony | +| [HDFS-6250](https://issues.apache.org/jira/browse/HDFS-6250) | TestBalancerWithNodeGroup.testBalancerWithRackLocality fails | Major | . | Kihwal Lee | Binglin Chang | +| [HDFS-6243](https://issues.apache.org/jira/browse/HDFS-6243) | HA NameNode transition to active or shutdown may leave lingering image transfer thread. | Major | ha, namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-6238](https://issues.apache.org/jira/browse/HDFS-6238) | TestDirectoryScanner leaks file descriptors. | Minor | datanode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-6230](https://issues.apache.org/jira/browse/HDFS-6230) | Expose upgrade status through NameNode web UI | Major | namenode | Arpit Agarwal | Mit Desai | +| [HDFS-6227](https://issues.apache.org/jira/browse/HDFS-6227) | ShortCircuitCache#unref should purge ShortCircuitReplicas whose streams have been closed by java interrupts | Major | . | Jing Zhao | Colin Patrick McCabe | +| [HDFS-6222](https://issues.apache.org/jira/browse/HDFS-6222) | Remove background token renewer from webhdfs | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6216](https://issues.apache.org/jira/browse/HDFS-6216) | Issues with webhdfs and http proxies | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6214](https://issues.apache.org/jira/browse/HDFS-6214) | Webhdfs has poor throughput for files \>2GB | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6213](https://issues.apache.org/jira/browse/HDFS-6213) | TestDataNodeConfig failing on Jenkins runs due to DN web port in use | Minor | test | Steve Loughran | Andrew Wang | +| [HDFS-6194](https://issues.apache.org/jira/browse/HDFS-6194) | Create new tests for ByteRangeInputStream | Major | . | Haohui Mai | Akira AJISAKA | +| [HDFS-6190](https://issues.apache.org/jira/browse/HDFS-6190) | minor textual fixes in DFSClient | Trivial | tools | Charles Lamb | Charles Lamb | +| [HDFS-6181](https://issues.apache.org/jira/browse/HDFS-6181) | Fix the wrong property names in NFS user guide | Trivial | documentation, nfs | Brandon Li | Brandon Li | +| [HDFS-6180](https://issues.apache.org/jira/browse/HDFS-6180) | dead node count / listing is very broken in JMX and old GUI | Blocker | . | Travis Thompson | Haohui Mai | +| [HDFS-6178](https://issues.apache.org/jira/browse/HDFS-6178) | Decommission on standby NN couldn't finish | Major | namenode | Ming Ma | Ming Ma | +| [HDFS-6160](https://issues.apache.org/jira/browse/HDFS-6160) | TestSafeMode occasionally fails | Major | test | Ted Yu | Arpit Agarwal | +| [HDFS-6159](https://issues.apache.org/jira/browse/HDFS-6159) | TestBalancerWithNodeGroup.testBalancerWithNodeGroup fails if there is block missing after balancer success | Major | test | Chen He | Chen He | +| [HDFS-6156](https://issues.apache.org/jira/browse/HDFS-6156) | Simplify the JMX API that provides snapshot information | Major | . | Haohui Mai | Shinichi Yamashita | +| [HDFS-6143](https://issues.apache.org/jira/browse/HDFS-6143) | WebHdfsFileSystem open should throw FileNotFoundException for non-existing paths | Blocker | . | Gera Shegalov | Gera Shegalov | +| [HDFS-6112](https://issues.apache.org/jira/browse/HDFS-6112) | NFS Gateway docs are incorrect for allowed hosts configuration | Minor | nfs | Aaron T. Myers | Aaron T. Myers | +| [HDFS-6056](https://issues.apache.org/jira/browse/HDFS-6056) | Clean up NFS config settings | Major | nfs | Aaron T. Myers | Brandon Li | +| [HDFS-5669](https://issues.apache.org/jira/browse/HDFS-5669) | Storage#tryLock() should check for null before logging successfull message | Major | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-5591](https://issues.apache.org/jira/browse/HDFS-5591) | Checkpointing should use monotonic time when calculating period | Minor | namenode | Andrew Wang | Charles Lamb | +| [HDFS-5522](https://issues.apache.org/jira/browse/HDFS-5522) | Datanode disk error check may be incorrectly skipped | Major | . | Kihwal Lee | Rushabh S Shah | +| [HDFS-4913](https://issues.apache.org/jira/browse/HDFS-4913) | Deleting file through fuse-dfs when using trash fails requiring root permissions | Major | fuse-dfs | Stephen Chu | Colin Patrick McCabe | +| [HDFS-4909](https://issues.apache.org/jira/browse/HDFS-4909) | Avoid protocol buffer RPC namespace clashes | Blocker | datanode, journal-node, namenode | Ralph Castain | Colin Patrick McCabe | +| [HDFS-3848](https://issues.apache.org/jira/browse/HDFS-3848) | A Bug in recoverLeaseInternal method of FSNameSystem class | Major | namenode | Hooman Peiro Sajjad | Chen He | +| [HDFS-3828](https://issues.apache.org/jira/browse/HDFS-3828) | Block Scanner rescans blocks too frequently | Major | . | Andy Isaacson | Andy Isaacson | +| [HDFS-3493](https://issues.apache.org/jira/browse/HDFS-3493) | Invalidate excess corrupted blocks as long as minimum replication is satisfied | Major | namenode | J.Andreina | Juan Yu | +| [HDFS-3087](https://issues.apache.org/jira/browse/HDFS-3087) | Decomissioning on NN restart can complete without blocks being replicated | Critical | namenode | Kihwal Lee | Rushabh S Shah | +| [MAPREDUCE-6002](https://issues.apache.org/jira/browse/MAPREDUCE-6002) | MR task should prevent report error to AM when process is shutting down | Major | task | Wangda Tan | Wangda Tan | +| [MAPREDUCE-5952](https://issues.apache.org/jira/browse/MAPREDUCE-5952) | LocalContainerLauncher#renameMapOutputForReduce incorrectly assumes a single dir for mapOutIndex | Blocker | mr-am, mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5939](https://issues.apache.org/jira/browse/MAPREDUCE-5939) | StartTime showing up as the epoch time in JHS UI after upgrade | Major | . | Kihwal Lee | Chen He | +| [MAPREDUCE-5924](https://issues.apache.org/jira/browse/MAPREDUCE-5924) | Windows: Sort Job failed due to 'Invalid event: TA\_COMMIT\_PENDING at COMMIT\_PENDING' | Major | . | Yesha Vora | Zhijie Shen | +| [MAPREDUCE-5920](https://issues.apache.org/jira/browse/MAPREDUCE-5920) | Add Xattr option in DistCp docs | Minor | distcp, documentation | Uma Maheswara Rao G | Yi Liu | +| [MAPREDUCE-5898](https://issues.apache.org/jira/browse/MAPREDUCE-5898) | distcp to support preserving HDFS extended attributes(XAttrs) | Major | distcp | Uma Maheswara Rao G | Yi Liu | +| [MAPREDUCE-5895](https://issues.apache.org/jira/browse/MAPREDUCE-5895) | FileAlreadyExistsException was thrown : Temporary Index File can not be cleaned up because OutputStream doesn't close properly | Major | client | Kousuke Saruta | Kousuke Saruta | +| [MAPREDUCE-5888](https://issues.apache.org/jira/browse/MAPREDUCE-5888) | Failed job leaves hung AM after it unregisters | Major | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5884](https://issues.apache.org/jira/browse/MAPREDUCE-5884) | History server uses short user name when canceling tokens | Major | jobhistoryserver, security | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [MAPREDUCE-5874](https://issues.apache.org/jira/browse/MAPREDUCE-5874) | Creating MapReduce REST API section | Major | documentation | Ravi Prakash | Tsuyoshi Ozawa | +| [MAPREDUCE-5868](https://issues.apache.org/jira/browse/MAPREDUCE-5868) | TestPipeApplication causing nightly build to fail | Major | test | Jason Lowe | Akira AJISAKA | +| [MAPREDUCE-5862](https://issues.apache.org/jira/browse/MAPREDUCE-5862) | Line records longer than 2x split size aren't handled correctly | Critical | . | bc Wong | bc Wong | +| [MAPREDUCE-5846](https://issues.apache.org/jira/browse/MAPREDUCE-5846) | Rumen doesn't understand JobQueueChangedEvent | Major | tools/rumen | Nathan Roberts | Nathan Roberts | +| [MAPREDUCE-5844](https://issues.apache.org/jira/browse/MAPREDUCE-5844) | Add a configurable delay to reducer-preemption | Major | . | Maysam Yabandeh | Maysam Yabandeh | +| [MAPREDUCE-5837](https://issues.apache.org/jira/browse/MAPREDUCE-5837) | MRAppMaster fails when checking on uber mode | Critical | . | Haohui Mai | Haohui Mai | +| [MAPREDUCE-5836](https://issues.apache.org/jira/browse/MAPREDUCE-5836) | Fix typo in RandomTextWriter | Trivial | . | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5834](https://issues.apache.org/jira/browse/MAPREDUCE-5834) | TestGridMixClasses tests timesout on branch-2 | Major | . | Mit Desai | Mit Desai | +| [MAPREDUCE-5814](https://issues.apache.org/jira/browse/MAPREDUCE-5814) | fat jar with \*-default.xml may fail when mapreduce.job.classloader=true. | Major | mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5790](https://issues.apache.org/jira/browse/MAPREDUCE-5790) | Default map hprof profile options do not work | Blocker | . | Andrew Wang | Gera Shegalov | +| [MAPREDUCE-5775](https://issues.apache.org/jira/browse/MAPREDUCE-5775) | Remove unnecessary job.setNumReduceTasks in SleepJob.createJob | Minor | . | Liyin Liang | jhanver chand sharma | +| [MAPREDUCE-5765](https://issues.apache.org/jira/browse/MAPREDUCE-5765) | Update hadoop-pipes examples README | Minor | pipes | Jonathan Eagles | Mit Desai | +| [MAPREDUCE-5759](https://issues.apache.org/jira/browse/MAPREDUCE-5759) | Remove unnecessary conf load in Limits | Major | . | Sandy Ryza | Sandy Ryza | +| [MAPREDUCE-5749](https://issues.apache.org/jira/browse/MAPREDUCE-5749) | TestRMContainerAllocator#testReportedAppProgress Failed | Major | . | Hong Shen | Jason Lowe | +| [MAPREDUCE-5713](https://issues.apache.org/jira/browse/MAPREDUCE-5713) | InputFormat and JobConf JavaDoc Fixes | Trivial | documentation | Ben Robie | Chen He | +| [MAPREDUCE-5671](https://issues.apache.org/jira/browse/MAPREDUCE-5671) | NaN can be created by client and assign to Progress | Major | . | Chen He | Chen He | +| [MAPREDUCE-5665](https://issues.apache.org/jira/browse/MAPREDUCE-5665) | Add audience annotations to MiniMRYarnCluster and MiniMRCluster | Major | test | Sandy Ryza | Anubhav Dhoot | +| [MAPREDUCE-5652](https://issues.apache.org/jira/browse/MAPREDUCE-5652) | NM Recovery. ShuffleHandler should handle NM restarts | Major | . | Karthik Kambatla | Jason Lowe | +| [MAPREDUCE-5517](https://issues.apache.org/jira/browse/MAPREDUCE-5517) | enabling uber mode with 0 reducer still requires mapreduce.reduce.memory.mb to be less than yarn.app.mapreduce.am.resource.mb | Minor | . | Siqi Li | Siqi Li | +| [MAPREDUCE-5456](https://issues.apache.org/jira/browse/MAPREDUCE-5456) | TestFetcher.testCopyFromHostExtraBytes is missing | Minor | mrv2, test | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5309](https://issues.apache.org/jira/browse/MAPREDUCE-5309) | 2.0.4 JobHistoryParser can't parse certain failed job history files generated by 2.0.3 history server | Major | jobhistoryserver, mrv2 | Vrushali C | Rushabh S Shah | +| [MAPREDUCE-4937](https://issues.apache.org/jira/browse/MAPREDUCE-4937) | MR AM handles an oversized split metainfo file poorly | Major | mr-am | Jason Lowe | Eric Payne | +| [YARN-2250](https://issues.apache.org/jira/browse/YARN-2250) | FairScheduler.findLowestCommonAncestorQueue returns null when queues not identical | Major | scheduler | Krisztian Horvath | Krisztian Horvath | +| [YARN-2241](https://issues.apache.org/jira/browse/YARN-2241) | ZKRMStateStore: On startup, show nicer messages if znodes already exist | Minor | resourcemanager | Robert Kanter | Robert Kanter | +| [YARN-2232](https://issues.apache.org/jira/browse/YARN-2232) | ClientRMService doesn't allow delegation token owner to cancel their own token in secure mode | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2204](https://issues.apache.org/jira/browse/YARN-2204) | TestAMRestart#testAMRestartWithExistingContainers assumes CapacityScheduler | Trivial | resourcemanager | Robert Kanter | Robert Kanter | +| [YARN-2201](https://issues.apache.org/jira/browse/YARN-2201) | TestRMWebServicesAppsModification dependent on yarn-default.xml | Major | . | Ray Chiang | Varun Vasudev | +| [YARN-2192](https://issues.apache.org/jira/browse/YARN-2192) | TestRMHA fails when run with a mix of Schedulers | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2191](https://issues.apache.org/jira/browse/YARN-2191) | Add a test to make sure NM will do application cleanup even if RM restarting happens before application completed | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2187](https://issues.apache.org/jira/browse/YARN-2187) | FairScheduler: Disable max-AM-share check by default | Major | fairscheduler | Robert Kanter | Robert Kanter | +| [YARN-2171](https://issues.apache.org/jira/browse/YARN-2171) | AMs block on the CapacityScheduler lock during allocate() | Critical | capacityscheduler | Jason Lowe | Jason Lowe | +| [YARN-2167](https://issues.apache.org/jira/browse/YARN-2167) | LeveldbIterator should get closed in NMLeveldbStateStoreService#loadLocalizationState() within finally block | Major | nodemanager | Junping Du | Junping Du | +| [YARN-2163](https://issues.apache.org/jira/browse/YARN-2163) | WebUI: Order of AppId in apps table should be consistent with ApplicationId.compareTo(). | Minor | resourcemanager, webapp | Wangda Tan | Wangda Tan | +| [YARN-2155](https://issues.apache.org/jira/browse/YARN-2155) | FairScheduler: Incorrect threshold check for preemption | Major | . | Wei Yan | Wei Yan | +| [YARN-2148](https://issues.apache.org/jira/browse/YARN-2148) | TestNMClient failed due more exit code values added and passed to AM | Major | client | Wangda Tan | Wangda Tan | +| [YARN-2132](https://issues.apache.org/jira/browse/YARN-2132) | ZKRMStateStore.ZKAction#runWithRetries doesn't log the exception it encounters | Major | resourcemanager | Karthik Kambatla | Vamsee Yarlagadda | +| [YARN-2128](https://issues.apache.org/jira/browse/YARN-2128) | FairScheduler: Incorrect calculation of amResource usage | Major | . | Wei Yan | Wei Yan | +| [YARN-2124](https://issues.apache.org/jira/browse/YARN-2124) | ProportionalCapacityPreemptionPolicy cannot work because it's initialized before scheduler initialized | Critical | resourcemanager, scheduler | Wangda Tan | Wangda Tan | +| [YARN-2122](https://issues.apache.org/jira/browse/YARN-2122) | In AllocationFileLoaderService, the reloadThread should be created in init() and started in start() | Major | scheduler | Karthik Kambatla | Robert Kanter | +| [YARN-2119](https://issues.apache.org/jira/browse/YARN-2119) | DEFAULT\_PROXY\_ADDRESS should use DEFAULT\_PROXY\_PORT | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2112](https://issues.apache.org/jira/browse/YARN-2112) | Hadoop-client is missing jackson libs due to inappropriate configs in pom.xml | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2111](https://issues.apache.org/jira/browse/YARN-2111) | In FairScheduler.attemptScheduling, we don't count containers as assigned if they have 0 memory but non-zero cores | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-2109](https://issues.apache.org/jira/browse/YARN-2109) | Fix TestRM to work with both schedulers | Major | scheduler | Anubhav Dhoot | Karthik Kambatla | +| [YARN-2104](https://issues.apache.org/jira/browse/YARN-2104) | Scheduler queue filter failed to work because index of queue column changed | Major | resourcemanager, webapp | Wangda Tan | Wangda Tan | +| [YARN-2103](https://issues.apache.org/jira/browse/YARN-2103) | Inconsistency between viaProto flag and initial value of SerializedExceptionProto.Builder | Major | . | Binglin Chang | Binglin Chang | +| [YARN-2096](https://issues.apache.org/jira/browse/YARN-2096) | Race in TestRMRestart#testQueueMetricsOnRMRestart | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2075](https://issues.apache.org/jira/browse/YARN-2075) | TestRMAdminCLI consistently fail on trunk and branch-2 | Major | . | Zhijie Shen | Kenji Kikushima | +| [YARN-2073](https://issues.apache.org/jira/browse/YARN-2073) | Fair Scheduler: Add a utilization threshold to prevent preempting resources when cluster is free | Critical | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-2065](https://issues.apache.org/jira/browse/YARN-2065) | AM cannot create new containers after restart-NM token from previous attempt used | Major | . | Steve Loughran | Jian He | +| [YARN-2054](https://issues.apache.org/jira/browse/YARN-2054) | Better defaults for YARN ZK configs for retries and retry-inteval when HA is enabled | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-2050](https://issues.apache.org/jira/browse/YARN-2050) | Fix LogCLIHelpers to create the correct FileContext | Major | . | Ming Ma | Ming Ma | +| [YARN-2036](https://issues.apache.org/jira/browse/YARN-2036) | Document yarn.resourcemanager.hostname in ClusterSetup | Minor | documentation | Karthik Kambatla | Ray Chiang | +| [YARN-1981](https://issues.apache.org/jira/browse/YARN-1981) | Nodemanager version is not updated when a node reconnects | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-1940](https://issues.apache.org/jira/browse/YARN-1940) | deleteAsUser() terminates early without deleting more files on error | Major | . | Kihwal Lee | Rushabh S Shah | +| [YARN-1913](https://issues.apache.org/jira/browse/YARN-1913) | With Fair Scheduler, cluster can logjam when all resources are consumed by AMs | Major | scheduler | bc Wong | Wei Yan | +| [YARN-1885](https://issues.apache.org/jira/browse/YARN-1885) | RM may not send the app-finished signal after RM restart to some nodes where the application ran before RM restarts | Major | . | Arpit Gupta | Wangda Tan | +| [YARN-1868](https://issues.apache.org/jira/browse/YARN-1868) | YARN status web ui does not show correctly in IE 11 | Major | webapp | Chuan Liu | Chuan Liu | +| [YARN-1865](https://issues.apache.org/jira/browse/YARN-1865) | ShellScriptBuilder does not check for some error conditions | Minor | nodemanager | Remus Rusanu | Remus Rusanu | +| [YARN-1790](https://issues.apache.org/jira/browse/YARN-1790) | Fair Scheduler UI not showing apps table | Major | . | bc Wong | bc Wong | +| [YARN-1784](https://issues.apache.org/jira/browse/YARN-1784) | TestContainerAllocation assumes CapacityScheduler | Minor | resourcemanager | Karthik Kambatla | Robert Kanter | +| [YARN-1736](https://issues.apache.org/jira/browse/YARN-1736) | FS: AppSchedulable.assignContainer's priority argument is redundant | Minor | scheduler | Sandy Ryza | Naren Koneru | +| [YARN-1726](https://issues.apache.org/jira/browse/YARN-1726) | ResourceSchedulerWrapper broken due to AbstractYarnScheduler | Blocker | . | Wei Yan | Wei Yan | +| [YARN-1718](https://issues.apache.org/jira/browse/YARN-1718) | Fix a couple isTerminals in Fair Scheduler queue placement rules | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1678](https://issues.apache.org/jira/browse/YARN-1678) | Fair scheduler gabs incessantly about reservations | Major | scheduler | Sandy Ryza | Sandy Ryza | +| [YARN-1550](https://issues.apache.org/jira/browse/YARN-1550) | NPE in FairSchedulerAppsBlock#render | Critical | fairscheduler | caolong | Anubhav Dhoot | +| [YARN-1520](https://issues.apache.org/jira/browse/YARN-1520) | update capacity scheduler docs to include necessary parameters | Major | . | Chen He | Chen He | +| [YARN-1429](https://issues.apache.org/jira/browse/YARN-1429) | \*nix: Allow a way for users to augment classpath of YARN daemons | Trivial | client | Sandy Ryza | Jarek Jarcec Cecho | +| [YARN-1136](https://issues.apache.org/jira/browse/YARN-1136) | Replace junit.framework.Assert with org.junit.Assert | Major | . | Karthik Kambatla | Chen He | +| [YARN-738](https://issues.apache.org/jira/browse/YARN-738) | TestClientRMTokens is failing irregularly while running all yarn tests | Major | . | Omkar Vinit Joshi | Ming Ma | +| [YARN-596](https://issues.apache.org/jira/browse/YARN-596) | Use scheduling policies throughout the queue hierarchy to decide which containers to preempt | Major | scheduler | Sandy Ryza | Wei Yan | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10754](https://issues.apache.org/jira/browse/HADOOP-10754) | Reenable several HA ZooKeeper-related tests on Windows. | Trivial | ha, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-3679](https://issues.apache.org/jira/browse/HADOOP-3679) | calls to junit Assert::assertEquals invert arguments, causing misleading error messages, other minor improvements. | Minor | test | Chris Douglas | jay vyas | +| [HDFS-6614](https://issues.apache.org/jira/browse/HDFS-6614) | shorten TestPread run time with a smaller retry timeout setting | Minor | test | Liang Xie | Liang Xie | +| [HDFS-6419](https://issues.apache.org/jira/browse/HDFS-6419) | TestBookKeeperHACheckpoints#TestSBNCheckpoints fails on trunk | Major | . | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6287](https://issues.apache.org/jira/browse/HDFS-6287) | Add vecsum test of libhdfs read access times | Minor | libhdfs, test | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6265](https://issues.apache.org/jira/browse/HDFS-6265) | Prepare HDFS codebase for JUnit 4.11. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-6257](https://issues.apache.org/jira/browse/HDFS-6257) | TestCacheDirectives#testExceedsCapacity fails occasionally | Minor | caching | Ted Yu | Colin Patrick McCabe | +| [HDFS-6224](https://issues.apache.org/jira/browse/HDFS-6224) | Add a unit test to TestAuditLogger for file permissions passed to logAuditEvent | Minor | test | Charles Lamb | Charles Lamb | +| [HDFS-5892](https://issues.apache.org/jira/browse/HDFS-5892) | TestDeleteBlockPool fails in branch-2 | Minor | . | Ted Yu | Ted Yu | +| [HDFS-5409](https://issues.apache.org/jira/browse/HDFS-5409) | TestOfflineEditsViewer#testStored fails on Windows due to CRLF line endings in editsStored.xml from git checkout | Minor | test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5852](https://issues.apache.org/jira/browse/MAPREDUCE-5852) | Prepare MapReduce codebase for JUnit 4.11. | Minor | test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-5804](https://issues.apache.org/jira/browse/MAPREDUCE-5804) | TestMRJobsWithProfiler#testProfiler timesout | Major | . | Mit Desai | Mit Desai | +| [MAPREDUCE-5642](https://issues.apache.org/jira/browse/MAPREDUCE-5642) | TestMiniMRChildTask fails on Windows | Minor | test | Chuan Liu | Chuan Liu | +| [YARN-2319](https://issues.apache.org/jira/browse/YARN-2319) | Fix MiniKdc not close in TestRMWebServicesDelegationTokens.java | Major | resourcemanager | Wenwu Peng | Wenwu Peng | +| [YARN-2270](https://issues.apache.org/jira/browse/YARN-2270) | TestFSDownload#testDownloadPublicWithStatCache fails in trunk | Minor | . | Ted Yu | Akira AJISAKA | +| [YARN-2224](https://issues.apache.org/jira/browse/YARN-2224) | Explicitly enable vmem check in TestContainersMonitor#testContainerKillOnMemoryOverflow | Trivial | nodemanager | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2216](https://issues.apache.org/jira/browse/YARN-2216) | TestRMApplicationHistoryWriter sometimes fails in trunk | Minor | . | Ted Yu | Zhijie Shen | +| [YARN-2105](https://issues.apache.org/jira/browse/YARN-2105) | Fix TestFairScheduler after YARN-2012 | Major | . | Ted Yu | Ashwin Shankar | +| [YARN-2011](https://issues.apache.org/jira/browse/YARN-2011) | Fix typo and warning in TestLeafQueue | Trivial | . | Chen He | Chen He | +| [YARN-1977](https://issues.apache.org/jira/browse/YARN-1977) | Add tests on getApplicationRequest with filtering start time range | Minor | . | Junping Du | Junping Du | +| [YARN-1970](https://issues.apache.org/jira/browse/YARN-1970) | Prepare YARN codebase for JUnit 4.11. | Minor | . | Chris Nauroth | Chris Nauroth | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10894](https://issues.apache.org/jira/browse/HADOOP-10894) | Fix dead link in ToolRunner documentation | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10864](https://issues.apache.org/jira/browse/HADOOP-10864) | Tool documentenation is broken | Minor | documentation | Allen Wittenauer | Akira AJISAKA | +| [HADOOP-10659](https://issues.apache.org/jira/browse/HADOOP-10659) | Refactor AccessControlList to reuse utility functions and to improve performance | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10652](https://issues.apache.org/jira/browse/HADOOP-10652) | Refactor Proxyusers to use AccessControlList | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10649](https://issues.apache.org/jira/browse/HADOOP-10649) | Allow overriding the default ACL for service authorization | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10566](https://issues.apache.org/jira/browse/HADOOP-10566) | Refactor proxyservers out of ProxyUsers | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10565](https://issues.apache.org/jira/browse/HADOOP-10565) | Support IP ranges (CIDR) in proxyuser.hosts | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10503](https://issues.apache.org/jira/browse/HADOOP-10503) | Move junit up to v 4.11 | Minor | build | Steve Loughran | Chris Nauroth | +| [HADOOP-10499](https://issues.apache.org/jira/browse/HADOOP-10499) | Remove unused parameter from ProxyUsers.authorize() | Minor | security | Benoy Antony | Benoy Antony | +| [HADOOP-10479](https://issues.apache.org/jira/browse/HADOOP-10479) | Fix new findbugs warnings in hadoop-minikdc | Major | . | Haohui Mai | Swarnim Kulkarni | +| [HADOOP-10471](https://issues.apache.org/jira/browse/HADOOP-10471) | Reduce the visibility of constants in ProxyUsers | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10467](https://issues.apache.org/jira/browse/HADOOP-10467) | Enable proxyuser specification to support list of users in addition to list of groups. | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10448](https://issues.apache.org/jira/browse/HADOOP-10448) | Support pluggable mechanism to specify proxy user settings | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10439](https://issues.apache.org/jira/browse/HADOOP-10439) | Fix compilation error in branch-2 after HADOOP-10426 | Major | build | Haohui Mai | Haohui Mai | +| [HADOOP-10426](https://issues.apache.org/jira/browse/HADOOP-10426) | CreateOpts.getOpt(..) should declare with generic type argument | Minor | fs | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HADOOP-10279](https://issues.apache.org/jira/browse/HADOOP-10279) | Create multiplexer, a requirement for the fair queue | Major | . | Chris Li | Chris Li | +| [HADOOP-10104](https://issues.apache.org/jira/browse/HADOOP-10104) | Update jackson to 1.9.13 | Minor | build | Steve Loughran | Akira AJISAKA | +| [HADOOP-9712](https://issues.apache.org/jira/browse/HADOOP-9712) | Write contract tests for FTP filesystem, fix places where it breaks | Minor | fs/s3 | Steve Loughran | | +| [HADOOP-9711](https://issues.apache.org/jira/browse/HADOOP-9711) | Write contract tests for S3Native; fix places where it breaks | Minor | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-9371](https://issues.apache.org/jira/browse/HADOOP-9371) | Define Semantics of FileSystem more rigorously | Major | fs | Steve Loughran | Steve Loughran | +| [HDFS-6562](https://issues.apache.org/jira/browse/HDFS-6562) | Refactor rename() in FSDirectory | Minor | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6557](https://issues.apache.org/jira/browse/HDFS-6557) | Move the reference of fsimage to FSNamesystem | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6480](https://issues.apache.org/jira/browse/HDFS-6480) | Move waitForReady() from FSDirectory to FSNamesystem | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6351](https://issues.apache.org/jira/browse/HDFS-6351) | Command "hdfs dfs -rm -r" can't remove empty directory | Major | hdfs-client | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6330](https://issues.apache.org/jira/browse/HDFS-6330) | Move mkdirs() to FSNamesystem | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6328](https://issues.apache.org/jira/browse/HDFS-6328) | Clean up dead code in FSDirectory | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-6315](https://issues.apache.org/jira/browse/HDFS-6315) | Decouple recording edit logs from FSDirectory | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-6276](https://issues.apache.org/jira/browse/HDFS-6276) | Remove unnecessary conditions and null check | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6275](https://issues.apache.org/jira/browse/HDFS-6275) | Fix warnings - type arguments can be inferred and redudant local variable | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6274](https://issues.apache.org/jira/browse/HDFS-6274) | Cleanup javadoc warnings in HDFS code | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6240](https://issues.apache.org/jira/browse/HDFS-6240) | WebImageViewer returns 404 if LISTSTATUS to an empty directory | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6219](https://issues.apache.org/jira/browse/HDFS-6219) | Proxy superuser configuration should use true client IP for address checks | Major | namenode, webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6218](https://issues.apache.org/jira/browse/HDFS-6218) | Audit log should use true client IP for proxied webhdfs operations | Major | namenode, webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6217](https://issues.apache.org/jira/browse/HDFS-6217) | Webhdfs PUT operations may not work via a http proxy | Major | webhdfs | Daryn Sharp | Daryn Sharp | +| [HDFS-6210](https://issues.apache.org/jira/browse/HDFS-6210) | Support GETACLSTATUS operation in WebImageViewer | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6186](https://issues.apache.org/jira/browse/HDFS-6186) | Pause deletion of blocks when the namenode starts up | Major | namenode | Suresh Srinivas | Jing Zhao | +| [HDFS-6173](https://issues.apache.org/jira/browse/HDFS-6173) | Move the default processor from Ls to Web in OfflineImageViewer | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6170](https://issues.apache.org/jira/browse/HDFS-6170) | Support GETFILESTATUS operation in WebImageViewer | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6169](https://issues.apache.org/jira/browse/HDFS-6169) | Move the address in WebImageViewer | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6162](https://issues.apache.org/jira/browse/HDFS-6162) | Format strings should use platform independent line separator | Minor | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6155](https://issues.apache.org/jira/browse/HDFS-6155) | Fix Boxing/unboxing to parse a primitive findbugs warnings | Major | . | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6125](https://issues.apache.org/jira/browse/HDFS-6125) | Cleanup unnecessary cast in HDFS code base | Major | test | Suresh Srinivas | Suresh Srinivas | +| [HDFS-6119](https://issues.apache.org/jira/browse/HDFS-6119) | FSNamesystem code cleanup | Minor | namenode | Suresh Srinivas | Suresh Srinivas | +| [HDFS-5978](https://issues.apache.org/jira/browse/HDFS-5978) | Create a tool to take fsimage and expose read-only WebHDFS API | Major | tools | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5865](https://issues.apache.org/jira/browse/HDFS-5865) | Update OfflineImageViewer document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-5411](https://issues.apache.org/jira/browse/HDFS-5411) | Update Bookkeeper dependency to 4.2.3 | Minor | . | Robert Rati | Rakesh R | +| [HDFS-4667](https://issues.apache.org/jira/browse/HDFS-4667) | Capture renamed files/directories in snapshot diff report | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-4286](https://issues.apache.org/jira/browse/HDFS-4286) | Changes from BOOKKEEPER-203 broken capability of including bookkeeper-server jar in hidden package of BKJM | Major | . | Vinayakumar B | Rakesh R | +| [HDFS-4221](https://issues.apache.org/jira/browse/HDFS-4221) | Remove the format limitation point from BKJM documentation as HDFS-3810 closed | Major | ha | Uma Maheswara Rao G | Rakesh R | +| [MAPREDUCE-5900](https://issues.apache.org/jira/browse/MAPREDUCE-5900) | Container preemption interpreted as task failures and eventually job failures | Major | applicationmaster, mr-am, mrv2 | Mayank Bansal | Mayank Bansal | +| [MAPREDUCE-5639](https://issues.apache.org/jira/browse/MAPREDUCE-5639) | Port DistCp2 document to trunk | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5638](https://issues.apache.org/jira/browse/MAPREDUCE-5638) | Port Hadoop Archives document to trunk | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5637](https://issues.apache.org/jira/browse/MAPREDUCE-5637) | Convert Hadoop Streaming document to APT | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5636](https://issues.apache.org/jira/browse/MAPREDUCE-5636) | Convert MapReduce Tutorial document to APT | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [YARN-2247](https://issues.apache.org/jira/browse/YARN-2247) | Allow RM web services users to authenticate using delegation tokens | Blocker | . | Varun Vasudev | Varun Vasudev | +| [YARN-2233](https://issues.apache.org/jira/browse/YARN-2233) | Implement web services to create, renew and cancel delegation tokens | Blocker | resourcemanager | Varun Vasudev | Varun Vasudev | +| [YARN-2152](https://issues.apache.org/jira/browse/YARN-2152) | Recover missing container information | Major | resourcemanager | Jian He | Jian He | +| [YARN-2121](https://issues.apache.org/jira/browse/YARN-2121) | TimelineAuthenticator#hasDelegationToken may throw NPE | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2118](https://issues.apache.org/jira/browse/YARN-2118) | Type mismatch in contains() check of TimelineWebServices#injectOwnerInfo() | Major | . | Ted Yu | Ted Yu | +| [YARN-2117](https://issues.apache.org/jira/browse/YARN-2117) | Close of Reader in TimelineAuthenticationFilterInitializer#initFilter() should be enclosed in finally block | Minor | . | Ted Yu | Chen He | +| [YARN-2115](https://issues.apache.org/jira/browse/YARN-2115) | Replace RegisterNodeManagerRequest#ContainerStatus with a new NMContainerStatus | Major | . | Jian He | Jian He | +| [YARN-2074](https://issues.apache.org/jira/browse/YARN-2074) | Preemption of AM containers shouldn't count towards AM failures | Major | resourcemanager | Vinod Kumar Vavilapalli | Jian He | +| [YARN-2071](https://issues.apache.org/jira/browse/YARN-2071) | Enforce more restricted permissions for the directory of Leveldb store | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2059](https://issues.apache.org/jira/browse/YARN-2059) | Extend access control for admin acls | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2052](https://issues.apache.org/jira/browse/YARN-2052) | ContainerId creation after work preserving restart is broken | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2049](https://issues.apache.org/jira/browse/YARN-2049) | Delegation token stuff for the timeline sever | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2022](https://issues.apache.org/jira/browse/YARN-2022) | Preempting an Application Master container can be kept as least priority when multiple applications are marked for preemption by ProportionalCapacityPreemptionPolicy | Major | resourcemanager | Sunil G | Sunil G | +| [YARN-2017](https://issues.apache.org/jira/browse/YARN-2017) | Merge some of the common lib code in schedulers | Major | resourcemanager | Jian He | Jian He | +| [YARN-1982](https://issues.apache.org/jira/browse/YARN-1982) | Rename the daemon name to timelineserver | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1938](https://issues.apache.org/jira/browse/YARN-1938) | Kerberos authentication for the timeline server | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1937](https://issues.apache.org/jira/browse/YARN-1937) | Add entity-level access control of the timeline data for owners only | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1936](https://issues.apache.org/jira/browse/YARN-1936) | Secured timeline client | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1877](https://issues.apache.org/jira/browse/YARN-1877) | Document yarn.resourcemanager.zk-auth and its scope | Critical | resourcemanager | Karthik Kambatla | Robert Kanter | +| [YARN-1757](https://issues.apache.org/jira/browse/YARN-1757) | NM Recovery. Auxiliary service support. | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1713](https://issues.apache.org/jira/browse/YARN-1713) | Implement getnewapplication and submitapp as part of RM web service | Blocker | . | Varun Vasudev | Varun Vasudev | +| [YARN-1702](https://issues.apache.org/jira/browse/YARN-1702) | Expose kill app functionality as part of RM web services | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-1474](https://issues.apache.org/jira/browse/YARN-1474) | Make schedulers services | Major | scheduler | Sandy Ryza | Tsuyoshi Ozawa | +| [YARN-1408](https://issues.apache.org/jira/browse/YARN-1408) | Preemption caused Invalid State Event: ACQUIRED at KILLED and caused a task timeout for 30mins | Major | resourcemanager | Sunil G | Sunil G | +| [YARN-1368](https://issues.apache.org/jira/browse/YARN-1368) | Common work to re-populate containers’ state into scheduler | Major | . | Bikas Saha | Jian He | +| [YARN-1366](https://issues.apache.org/jira/browse/YARN-1366) | AM should implement Resync with the ApplicationMasterService instead of shutting down | Major | resourcemanager | Bikas Saha | Rohith Sharma K S | +| [YARN-1365](https://issues.apache.org/jira/browse/YARN-1365) | ApplicationMasterService to allow Register of an app that was running before restart | Major | resourcemanager | Bikas Saha | Anubhav Dhoot | +| [YARN-1362](https://issues.apache.org/jira/browse/YARN-1362) | Distinguish between nodemanager shutdown for decommission vs shutdown for restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1339](https://issues.apache.org/jira/browse/YARN-1339) | Recover DeletionService state upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1338](https://issues.apache.org/jira/browse/YARN-1338) | Recover localized resource cache state upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10821](https://issues.apache.org/jira/browse/HADOOP-10821) | Prepare the release notes for Hadoop 2.5.0 | Blocker | . | Akira AJISAKA | Andrew Wang | +| [HADOOP-10715](https://issues.apache.org/jira/browse/HADOOP-10715) | Remove public GraphiteSink#setWriter() | Minor | . | Ted Yu | | +| [HDFS-6486](https://issues.apache.org/jira/browse/HDFS-6486) | Add user doc for XAttrs via WebHDFS. | Minor | webhdfs | Yi Liu | Yi Liu | +| [HDFS-6430](https://issues.apache.org/jira/browse/HDFS-6430) | HTTPFS - Implement XAttr support | Major | . | Yi Liu | Yi Liu | +| [MAPREDUCE-4282](https://issues.apache.org/jira/browse/MAPREDUCE-4282) | Convert Forrest docs to APT | Major | documentation | Eli Collins | Akira AJISAKA | +| [YARN-2125](https://issues.apache.org/jira/browse/YARN-2125) | ProportionalCapacityPreemptionPolicy should only log CSV when debug enabled | Minor | resourcemanager, scheduler | Wangda Tan | Wangda Tan | +| [YARN-2091](https://issues.apache.org/jira/browse/YARN-2091) | Add more values to ContainerExitStatus and pass it from NM to RM and then to app masters | Major | . | Bikas Saha | Tsuyoshi Ozawa | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/RELEASENOTES.2.5.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/RELEASENOTES.2.5.0.md new file mode 100644 index 00000000000..6f243580c1c --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.0/RELEASENOTES.2.5.0.md @@ -0,0 +1,143 @@ + + +# Apache Hadoop 2.5.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10568](https://issues.apache.org/jira/browse/HADOOP-10568) | *Major* | **Add s3 server-side encryption** + +s3 server-side encryption is now supported. + +To enable this feature, specify the following in your client-side configuration: + +name: fs.s3n.server-side-encryption-algorithm +value: AES256 + + +--- + +* [HADOOP-10451](https://issues.apache.org/jira/browse/HADOOP-10451) | *Trivial* | **Remove unused field and imports from SaslRpcServer** + +SaslRpcServer.SASL\_PROPS is removed. +Any use of this variable should be replaced with the following code: +SaslPropertiesResolver saslPropsResolver = SaslPropertiesResolver.getInstance(conf); +Map\ sasl\_props = saslPropsResolver.getDefaultProperties(); + + +--- + +* [HADOOP-10342](https://issues.apache.org/jira/browse/HADOOP-10342) | *Major* | **Extend UserGroupInformation to return a UGI given a preauthenticated kerberos Subject** + +Add getUGIFromSubject to leverage an external kerberos authentication + + +--- + +* [HADOOP-9919](https://issues.apache.org/jira/browse/HADOOP-9919) | *Major* | **Update hadoop-metrics2.properties examples to Yarn** + +Remove MRv1 settings from hadoop-metrics2.properties, add YARN settings instead. + + +--- + +* [HDFS-6471](https://issues.apache.org/jira/browse/HDFS-6471) | *Major* | **Make moveFromLocal CLI testcases to be non-disruptive** + +Committed to trunk and merged into branch-2. Thanks Dasha! + + +--- + +* [HDFS-6297](https://issues.apache.org/jira/browse/HDFS-6297) | *Major* | **Add CLI testcases to reflect new features of dfs and dfsadmin** + +Committed to the trunk and branch-2. Thanks Dasha! + + +--- + +* [HDFS-6293](https://issues.apache.org/jira/browse/HDFS-6293) | *Blocker* | **Issues with OIV processing PB-based fsimages** + +Set "dfs.namenode.legacy-oiv-image.dir" to an appropriate directory to make standby name node or secondary name node save its file system state in the old fsimage format during checkpointing. This image can be used for offline analysis using the OfflineImageViewer. Use the "hdfs oiv\_legacy" command to process the old fsimage format. + + +--- + +* [HDFS-6273](https://issues.apache.org/jira/browse/HDFS-6273) | *Major* | **Config options to allow wildcard endpoints for namenode HTTP and HTTPS servers** + +HDFS-6273 introduces two new HDFS configuration keys: +- dfs.namenode.http-bind-host +- dfs.namenode.https-bind-host + +The most common use case for these keys is to have the NameNode HTTP (or HTTPS) endpoints listen on all interfaces on multi-homed systems by setting the keys to 0.0.0.0 i.e. INADDR\_ANY. + +For the systems background on this usage of INADDR\_ANY please refer to ip(7) in the Linux Programmer's Manual (web link: http://man7.org/linux/man-pages/man7/ip.7.html). + +These keys complement the existing NameNode options: +- dfs.namenode.rpc-bind-host +- dfs.namenode.servicerpc-bind-host + + +--- + +* [HDFS-6168](https://issues.apache.org/jira/browse/HDFS-6168) | *Major* | **Remove deprecated methods in DistributedFileSystem** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-6164](https://issues.apache.org/jira/browse/HDFS-6164) | *Major* | **Remove lsr in OfflineImageViewer** + +The offlineimageviewer no longer generates lsr-style outputs. The functionality has been superseded by a tool that takes the fsimage and exposes WebHDFS-like API for user queries. + + +--- + +* [HDFS-6153](https://issues.apache.org/jira/browse/HDFS-6153) | *Minor* | **Document "fileId" and "childrenNum" fields in the FileStatus Json schema** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-6110](https://issues.apache.org/jira/browse/HDFS-6110) | *Major* | **adding more slow action log in critical write path** + +Log slow i/o. Set log thresholds in dfsclient and datanode via the below new configs: + +dfs.client.slow.io.warning.threshold.ms (Default 30 seconds) +dfs.datanode.slow.io.warning.threshold.ms (Default 300ms) + + +--- + +* [MAPREDUCE-5777](https://issues.apache.org/jira/browse/MAPREDUCE-5777) | *Major* | **Support utf-8 text with BOM (byte order marker)** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-2107](https://issues.apache.org/jira/browse/YARN-2107) | *Major* | **Refactor timeline classes into server.timeline package** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/CHANGES.2.5.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/CHANGES.2.5.1.md new file mode 100644 index 00000000000..9c8e4b72c1a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/CHANGES.2.5.1.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 2.5.1 - 2014-09-05 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11065](https://issues.apache.org/jira/browse/HADOOP-11065) | Rat check should exclude \*\*/build/\*\* | Blocker | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-11001](https://issues.apache.org/jira/browse/HADOOP-11001) | Fix test-patch to work with the git repo | Blocker | scripts | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-10957](https://issues.apache.org/jira/browse/HADOOP-10957) | The globber will sometimes erroneously return a permission denied exception when there is a non-terminal wildcard | Blocker | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-10956](https://issues.apache.org/jira/browse/HADOOP-10956) | Fix create-release script to include docs and necessary txt files | Blocker | scripts | Karthik Kambatla | Karthik Kambatla | +| [MAPREDUCE-6033](https://issues.apache.org/jira/browse/MAPREDUCE-6033) | Users are not allowed to view their own jobs, denied by JobACLsManager | Major | . | Yu Gao | Yu Gao | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/RELEASENOTES.2.5.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/RELEASENOTES.2.5.1.md new file mode 100644 index 00000000000..ea5e7506355 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.1/RELEASENOTES.2.5.1.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 2.5.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/CHANGES.2.5.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/CHANGES.2.5.2.md new file mode 100644 index 00000000000..34063598e29 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/CHANGES.2.5.2.md @@ -0,0 +1,69 @@ + + +# Apache Hadoop Changelog + +## Release 2.5.2 - 2014-11-19 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11307](https://issues.apache.org/jira/browse/HADOOP-11307) | create-release script should run git clean first | Blocker | scripts | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-11260](https://issues.apache.org/jira/browse/HADOOP-11260) | Patch up Jetty to disable SSLv3 | Blocker | security | Karthik Kambatla | Mike Yoder | +| [HADOOP-11243](https://issues.apache.org/jira/browse/HADOOP-11243) | SSLFactory shouldn't allow SSLv3 | Blocker | . | Wei Yan | Wei Yan | +| [HDFS-7391](https://issues.apache.org/jira/browse/HDFS-7391) | Renable SSLv2Hello in HttpFS | Blocker | webhdfs | Robert Kanter | Robert Kanter | +| [HDFS-7274](https://issues.apache.org/jira/browse/HDFS-7274) | Disable SSLv3 in HttpFS | Blocker | webhdfs | Robert Kanter | Robert Kanter | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/RELEASENOTES.2.5.2.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/RELEASENOTES.2.5.2.md new file mode 100644 index 00000000000..7348c5298bb --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.5.2/RELEASENOTES.2.5.2.md @@ -0,0 +1,24 @@ + + +# Apache Hadoop 2.5.2 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/CHANGES.2.6.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/CHANGES.2.6.0.md new file mode 100644 index 00000000000..85098fec9c9 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/CHANGES.2.6.0.md @@ -0,0 +1,964 @@ + + +# Apache Hadoop Changelog + +## Release 2.6.0 - 2014-11-18 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-8944](https://issues.apache.org/jira/browse/HADOOP-8944) | Shell command fs -count should include human readable option | Trivial | . | Jonathan Allen | Jonathan Allen | +| [YARN-2615](https://issues.apache.org/jira/browse/YARN-2615) | ClientToAMTokenIdentifier and DelegationTokenIdentifier should allow extended fields | Blocker | . | Junping Du | Junping Du | +| [YARN-1051](https://issues.apache.org/jira/browse/YARN-1051) | YARN Admission Control/Planner: enhancing the resource allocation model with time. | Major | capacityscheduler, resourcemanager, scheduler | Carlo Curino | Carlo Curino | +| [YARN-668](https://issues.apache.org/jira/browse/YARN-668) | TokenIdentifier serialization should consider Unknown fields | Blocker | . | Siddharth Seth | Junping Du | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10893](https://issues.apache.org/jira/browse/HADOOP-10893) | isolated classloader on the client side | Major | util | Sangjin Lee | Sangjin Lee | +| [HADOOP-10719](https://issues.apache.org/jira/browse/HADOOP-10719) | Add generateEncryptedKey and decryptEncryptedKey methods to KeyProvider | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10607](https://issues.apache.org/jira/browse/HADOOP-10607) | Create an API to Separate Credentials/Password Storage from Applications | Major | security | Larry McCay | Larry McCay | +| [HADOOP-10400](https://issues.apache.org/jira/browse/HADOOP-10400) | Incorporate new S3A FileSystem implementation | Major | fs, fs/s3 | Jordan Mendelson | Jordan Mendelson | +| [HADOOP-10150](https://issues.apache.org/jira/browse/HADOOP-10150) | Hadoop cryptographic file system | Major | security | Yi Liu | Yi Liu | +| [HDFS-6959](https://issues.apache.org/jira/browse/HDFS-6959) | Make the HDFS home directory location customizable. | Minor | . | Kevin Odell | Yongjun Zhang | +| [HDFS-6634](https://issues.apache.org/jira/browse/HDFS-6634) | inotify in HDFS | Major | hdfs-client, namenode, qjm | James Thomas | James Thomas | +| [HDFS-6584](https://issues.apache.org/jira/browse/HDFS-6584) | Support Archival Storage | Major | balancer & mover, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6581](https://issues.apache.org/jira/browse/HDFS-6581) | Write to single replica in memory | Major | datanode, hdfs-client, namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6134](https://issues.apache.org/jira/browse/HDFS-6134) | Transparent data at rest encryption | Major | security | Alejandro Abdelnur | Charles Lamb | +| [HDFS-4257](https://issues.apache.org/jira/browse/HDFS-4257) | The ReplaceDatanodeOnFailure policies could have a forgiving option | Minor | hdfs-client | Harsh J | Tsz Wo Nicholas Sze | +| [MAPREDUCE-6007](https://issues.apache.org/jira/browse/MAPREDUCE-6007) | Add support to distcp to preserve raw.\* namespace extended attributes | Major | distcp | Charles Lamb | Charles Lamb | +| [MAPREDUCE-5890](https://issues.apache.org/jira/browse/MAPREDUCE-5890) | Support for encrypting Intermediate data and spills in local filesystem | Major | security | Alejandro Abdelnur | Arun Suresh | +| [YARN-2395](https://issues.apache.org/jira/browse/YARN-2395) | FairScheduler: Preemption timeout should be configurable per queue | Major | fairscheduler | Ashwin Shankar | Wei Yan | +| [YARN-2394](https://issues.apache.org/jira/browse/YARN-2394) | FairScheduler: Configure fairSharePreemptionThreshold per queue | Major | fairscheduler | Ashwin Shankar | Wei Yan | +| [YARN-2393](https://issues.apache.org/jira/browse/YARN-2393) | FairScheduler: Add the notion of steady fair share | Major | fairscheduler | Ashwin Shankar | Wei Yan | +| [YARN-2131](https://issues.apache.org/jira/browse/YARN-2131) | Add a way to format the RMStateStore | Major | resourcemanager | Karthik Kambatla | Robert Kanter | +| [YARN-1964](https://issues.apache.org/jira/browse/YARN-1964) | Create Docker analog of the LinuxContainerExecutor in YARN | Major | . | Arun C Murthy | Abin Shahab | +| [YARN-415](https://issues.apache.org/jira/browse/YARN-415) | Capture aggregate memory allocation at the app-level for chargeback | Major | resourcemanager | Kendall Thrapp | Eric Payne | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11216](https://issues.apache.org/jira/browse/HADOOP-11216) | Improve Openssl library finding | Major | security | Yi Liu | Colin Patrick McCabe | +| [HADOOP-11195](https://issues.apache.org/jira/browse/HADOOP-11195) | Move Id-Name mapping in NFS to the hadoop-common area for better maintenance | Major | nfs, security | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-11184](https://issues.apache.org/jira/browse/HADOOP-11184) | Update Hadoop's lz4 to r123 | Major | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11153](https://issues.apache.org/jira/browse/HADOOP-11153) | Make number of KMS threads configurable | Major | kms | Andrew Wang | Andrew Wang | +| [HADOOP-11111](https://issues.apache.org/jira/browse/HADOOP-11111) | MiniKDC to use locale EN\_US for case conversions | Minor | tools | Steve Loughran | Steve Loughran | +| [HADOOP-11106](https://issues.apache.org/jira/browse/HADOOP-11106) | Document considerations of HAR and Encryption | Minor | documentation | Andrew Wang | Charles Lamb | +| [HADOOP-11101](https://issues.apache.org/jira/browse/HADOOP-11101) | How about inputstream close statement from catch block to finally block in FileContext#copy() ? | Minor | . | skrho | skrho | +| [HADOOP-11068](https://issues.apache.org/jira/browse/HADOOP-11068) | Match hadoop.auth cookie format to jetty output | Major | security | Gregory Chanan | Gregory Chanan | +| [HADOOP-11057](https://issues.apache.org/jira/browse/HADOOP-11057) | checknative command to probe for winutils.exe on windows | Minor | native | Steve Loughran | Xiaoyu Yao | +| [HADOOP-11054](https://issues.apache.org/jira/browse/HADOOP-11054) | Add a KeyProvider instantiation based on a URI | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11030](https://issues.apache.org/jira/browse/HADOOP-11030) | Define a variable jackson.version instead of using constant at multiple places | Minor | . | Juan Yu | Juan Yu | +| [HADOOP-11021](https://issues.apache.org/jira/browse/HADOOP-11021) | Configurable replication factor in the hadoop archive command | Minor | . | Zhe Zhang | Zhe Zhang | +| [HADOOP-11017](https://issues.apache.org/jira/browse/HADOOP-11017) | KMS delegation token secret manager should be able to use zookeeper as store | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-11016](https://issues.apache.org/jira/browse/HADOOP-11016) | KMS should support signing cookies with zookeeper secret manager | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11015](https://issues.apache.org/jira/browse/HADOOP-11015) | Http server/client utils to propagate and recreate Exceptions from server to client | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11009](https://issues.apache.org/jira/browse/HADOOP-11009) | Add Timestamp Preservation to DistCp | Major | tools/distcp | Gary Steelman | Gary Steelman | +| [HADOOP-11007](https://issues.apache.org/jira/browse/HADOOP-11007) | Reinstate building of ant tasks support | Major | build, fs | Jason Lowe | Jason Lowe | +| [HADOOP-10998](https://issues.apache.org/jira/browse/HADOOP-10998) | Fix bash tab completion code to work | Trivial | scripts | Jim Hester | Jim Hester | +| [HADOOP-10990](https://issues.apache.org/jira/browse/HADOOP-10990) | Add missed NFSv3 request and response classes | Major | nfs | Brandon Li | Brandon Li | +| [HADOOP-10982](https://issues.apache.org/jira/browse/HADOOP-10982) | KMS: Support for multiple Kerberos principals | Major | . | Andrew Wang | Alejandro Abdelnur | +| [HADOOP-10975](https://issues.apache.org/jira/browse/HADOOP-10975) | org.apache.hadoop.util.DataChecksum should support native checksum calculation | Major | performance | James Thomas | James Thomas | +| [HADOOP-10970](https://issues.apache.org/jira/browse/HADOOP-10970) | Cleanup KMS configuration keys | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-10967](https://issues.apache.org/jira/browse/HADOOP-10967) | Improve DefaultCryptoExtension#generateEncryptedKey performance | Major | security | Yi Liu | Yi Liu | +| [HADOOP-10954](https://issues.apache.org/jira/browse/HADOOP-10954) | Adding site documents of hadoop-tools | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-10936](https://issues.apache.org/jira/browse/HADOOP-10936) | Change default KeyProvider bitlength to 128 | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-10922](https://issues.apache.org/jira/browse/HADOOP-10922) | User documentation for CredentialShell | Major | . | Andrew Wang | Larry McCay | +| [HADOOP-10903](https://issues.apache.org/jira/browse/HADOOP-10903) | Enhance hadoop classpath command to expand wildcards or write classpath into jar manifest. | Major | scripts, util | Chris Nauroth | Chris Nauroth | +| [HADOOP-10902](https://issues.apache.org/jira/browse/HADOOP-10902) | Deletion of directories with snapshots will not output reason for trash move failure | Minor | . | Stephen Chu | Stephen Chu | +| [HADOOP-10891](https://issues.apache.org/jira/browse/HADOOP-10891) | Add EncryptedKeyVersion factory method to KeyProviderCryptoExtension | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-10881](https://issues.apache.org/jira/browse/HADOOP-10881) | Clarify usage of encryption and encrypted encryption key in KeyProviderCryptoExtension | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-10863](https://issues.apache.org/jira/browse/HADOOP-10863) | KMS should have a blacklist for decrypting EEKs | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10855](https://issues.apache.org/jira/browse/HADOOP-10855) | Allow Text to be read with a known length | Minor | io | Todd Lipcon | Todd Lipcon | +| [HADOOP-10845](https://issues.apache.org/jira/browse/HADOOP-10845) | Add common tests for ACLs in combination with viewfs. | Major | fs, test | Chris Nauroth | Stephen Chu | +| [HADOOP-10841](https://issues.apache.org/jira/browse/HADOOP-10841) | EncryptedKeyVersion should have a key name property | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10839](https://issues.apache.org/jira/browse/HADOOP-10839) | Add unregisterSource() to MetricsSystem API | Major | metrics | shanyu zhao | shanyu zhao | +| [HADOOP-10838](https://issues.apache.org/jira/browse/HADOOP-10838) | Byte array native checksumming | Major | performance | James Thomas | James Thomas | +| [HADOOP-10835](https://issues.apache.org/jira/browse/HADOOP-10835) | Implement HTTP proxyuser support in HTTP authentication client/server libraries | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10833](https://issues.apache.org/jira/browse/HADOOP-10833) | Remove unused cache in UserProvider | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10826](https://issues.apache.org/jira/browse/HADOOP-10826) | Iteration on KeyProviderFactory.serviceLoader is thread-unsafe | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10824](https://issues.apache.org/jira/browse/HADOOP-10824) | Refactor KMSACLs to avoid locking | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10817](https://issues.apache.org/jira/browse/HADOOP-10817) | ProxyUsers configuration should support configurable prefixes | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10815](https://issues.apache.org/jira/browse/HADOOP-10815) | Implement Windows equivalent of mlock. | Major | native | Chris Nauroth | Chris Nauroth | +| [HADOOP-10812](https://issues.apache.org/jira/browse/HADOOP-10812) | Delegate KeyProviderExtension#toString to underlying KeyProvider | Trivial | . | Andrew Wang | Andrew Wang | +| [HADOOP-10808](https://issues.apache.org/jira/browse/HADOOP-10808) | Remove unused native code for munlock. | Minor | native | Chris Nauroth | Chris Nauroth | +| [HADOOP-10793](https://issues.apache.org/jira/browse/HADOOP-10793) | KeyShell args should use single-dash style | Major | security | Mike Yoder | Andrew Wang | +| [HADOOP-10791](https://issues.apache.org/jira/browse/HADOOP-10791) | AuthenticationFilter should support externalizing the secret for signing and provide rotation support | Major | security | Alejandro Abdelnur | Robert Kanter | +| [HADOOP-10771](https://issues.apache.org/jira/browse/HADOOP-10771) | Refactor HTTP delegation support out of httpfs to common | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10770](https://issues.apache.org/jira/browse/HADOOP-10770) | KMS add delegation token support | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10769](https://issues.apache.org/jira/browse/HADOOP-10769) | Create KeyProvider extension to handle delegation tokens | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10758](https://issues.apache.org/jira/browse/HADOOP-10758) | KMS: add ACLs on per key basis. | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10757](https://issues.apache.org/jira/browse/HADOOP-10757) | KeyProvider KeyVersion should provide the key name | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10756](https://issues.apache.org/jira/browse/HADOOP-10756) | KMS audit log should consolidate successful similar requests | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10755](https://issues.apache.org/jira/browse/HADOOP-10755) | Support negative caching of user-group mapping | Major | security | Andrew Wang | Lei (Eddy) Xu | +| [HADOOP-10750](https://issues.apache.org/jira/browse/HADOOP-10750) | KMSKeyProviderCache should be in hadoop-common | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10736](https://issues.apache.org/jira/browse/HADOOP-10736) | Add key attributes to the key shell | Major | security | Mike Yoder | Mike Yoder | +| [HADOOP-10731](https://issues.apache.org/jira/browse/HADOOP-10731) | Remove @date JavaDoc comment in ProgramDriver class | Trivial | documentation | Henry Saputra | Henry Saputra | +| [HADOOP-10720](https://issues.apache.org/jira/browse/HADOOP-10720) | KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10698](https://issues.apache.org/jira/browse/HADOOP-10698) | KMS, add proxyuser support | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10696](https://issues.apache.org/jira/browse/HADOOP-10696) | Add optional attributes to KeyProvider Options and Metadata | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10695](https://issues.apache.org/jira/browse/HADOOP-10695) | KMSClientProvider should respect a configurable timeout. | Major | . | Andrew Wang | Mike Yoder | +| [HADOOP-10675](https://issues.apache.org/jira/browse/HADOOP-10675) | Add server-side encryption functionality to s3a | Major | fs/s3 | David S. Wang | David S. Wang | +| [HADOOP-10620](https://issues.apache.org/jira/browse/HADOOP-10620) | /docs/current doesn't point to the latest version 2.4.0 | Major | documentation | Jacek Laskowski | | +| [HADOOP-10610](https://issues.apache.org/jira/browse/HADOOP-10610) | Upgrade S3n fs.s3.buffer.dir to support multi directories | Minor | fs/s3 | Ted Malaska | Ted Malaska | +| [HADOOP-10433](https://issues.apache.org/jira/browse/HADOOP-10433) | Key Management Server based on KeyProvider API | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10432](https://issues.apache.org/jira/browse/HADOOP-10432) | Refactor SSLFactory to expose static method to determine HostnameVerifier | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10431](https://issues.apache.org/jira/browse/HADOOP-10431) | Change visibility of KeyStore.Options getter methods to public | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10430](https://issues.apache.org/jira/browse/HADOOP-10430) | KeyProvider Metadata should have an optional description, there should be a method to retrieve the metadata from all keys | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10429](https://issues.apache.org/jira/browse/HADOOP-10429) | KeyStores should have methods to generate the materials themselves, KeyShell should use them | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10428](https://issues.apache.org/jira/browse/HADOOP-10428) | JavaKeyStoreProvider should accept keystore password via configuration falling back to ENV VAR | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10427](https://issues.apache.org/jira/browse/HADOOP-10427) | KeyProvider implementations should be thread safe | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10373](https://issues.apache.org/jira/browse/HADOOP-10373) | create tools/hadoop-amazon for aws/EMR support | Major | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-10335](https://issues.apache.org/jira/browse/HADOOP-10335) | An ip whilelist based implementation to resolve Sasl properties per connection | Major | . | Benoy Antony | Benoy Antony | +| [HADOOP-10231](https://issues.apache.org/jira/browse/HADOOP-10231) | Add some components in Native Libraries document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-9540](https://issues.apache.org/jira/browse/HADOOP-9540) | Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing. | Minor | fs/s3, test | Hari | | +| [HADOOP-9457](https://issues.apache.org/jira/browse/HADOOP-9457) | add an SCM-ignored XML filename to keep secrets in (auth-keys.xml?) | Minor | build | Steve Loughran | | +| [HADOOP-8896](https://issues.apache.org/jira/browse/HADOOP-8896) | Javadoc points to Wrong Reader and Writer classes in SequenceFile | Trivial | documentation, io | Timothy Mann | Ray Chiang | +| [HADOOP-8815](https://issues.apache.org/jira/browse/HADOOP-8815) | RandomDatum overrides equals(Object) but no hashCode() | Minor | test | Brandon Li | Brandon Li | +| [HADOOP-8069](https://issues.apache.org/jira/browse/HADOOP-8069) | Enable TCP\_NODELAY by default for IPC | Major | ipc | Todd Lipcon | Todd Lipcon | +| [HADOOP-7664](https://issues.apache.org/jira/browse/HADOOP-7664) | o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same. | Minor | conf | Ravi Prakash | Ravi Prakash | +| [HDFS-7313](https://issues.apache.org/jira/browse/HDFS-7313) | Support optional configuration of AES cipher suite on DataTransferProtocol. | Major | datanode, hdfs-client, security | Chris Nauroth | Chris Nauroth | +| [HDFS-7276](https://issues.apache.org/jira/browse/HDFS-7276) | Limit the number of byte arrays used by DFSOutputStream | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7260](https://issues.apache.org/jira/browse/HDFS-7260) | Make DFSOutputStream.MAX\_PACKETS configurable | Minor | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7233](https://issues.apache.org/jira/browse/HDFS-7233) | NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException | Major | namenode | Rushabh S Shah | Rushabh S Shah | +| [HDFS-7230](https://issues.apache.org/jira/browse/HDFS-7230) | Add rolling downgrade documentation | Major | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7228](https://issues.apache.org/jira/browse/HDFS-7228) | Add an SSD policy into the default BlockStoragePolicySuite | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-7217](https://issues.apache.org/jira/browse/HDFS-7217) | Better batching of IBRs | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7215](https://issues.apache.org/jira/browse/HDFS-7215) | Add JvmPauseMonitor to NFS gateway | Minor | nfs | Brandon Li | Brandon Li | +| [HDFS-7195](https://issues.apache.org/jira/browse/HDFS-7195) | Update user doc of secure mode about Datanodes don't require root or jsvc | Major | documentation, security | Yi Liu | Chris Nauroth | +| [HDFS-7158](https://issues.apache.org/jira/browse/HDFS-7158) | Reduce the memory usage of WebImageViewer | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7153](https://issues.apache.org/jira/browse/HDFS-7153) | Add storagePolicy to NN edit log during file creation | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7128](https://issues.apache.org/jira/browse/HDFS-7128) | Decommission slows way down when it gets towards the end | Major | namenode | Ming Ma | Ming Ma | +| [HDFS-7093](https://issues.apache.org/jira/browse/HDFS-7093) | Add config key to restrict setStoragePolicy | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7059](https://issues.apache.org/jira/browse/HDFS-7059) | HAadmin transtionToActive with forceActive option can show confusing message. | Minor | . | Rushabh S Shah | Rushabh S Shah | +| [HDFS-6943](https://issues.apache.org/jira/browse/HDFS-6943) | Improve NN allocateBlock log to include replicas' datanode IPs | Minor | namenode | Ming Ma | Ming Ma | +| [HDFS-6899](https://issues.apache.org/jira/browse/HDFS-6899) | Allow changing MiniDFSCluster volumes per DN and capacity per volume | Major | datanode, test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6886](https://issues.apache.org/jira/browse/HDFS-6886) | Use single editlog record for creating file + overwrite. | Critical | namenode | Yi Liu | Yi Liu | +| [HDFS-6858](https://issues.apache.org/jira/browse/HDFS-6858) | Allow dfs.data.transfer.saslproperties.resolver.class default to hadoop.security.saslproperties.resolver.class | Minor | security | Benoy Antony | Benoy Antony | +| [HDFS-6850](https://issues.apache.org/jira/browse/HDFS-6850) | Move NFS out of order write unit tests into TestWrites class | Minor | nfs | Zhe Zhang | Zhe Zhang | +| [HDFS-6849](https://issues.apache.org/jira/browse/HDFS-6849) | Replace HttpFS custom proxyuser handling with common implementation | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-6837](https://issues.apache.org/jira/browse/HDFS-6837) | Code cleanup for Balancer and Dispatcher | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6836](https://issues.apache.org/jira/browse/HDFS-6836) | HDFS INFO logging is verbose & uses file appenders | Major | datanode | Gopal V | Nathan Yao | +| [HDFS-6828](https://issues.apache.org/jira/browse/HDFS-6828) | Separate block replica dispatching from Balancer | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6812](https://issues.apache.org/jira/browse/HDFS-6812) | Remove addBlock and replaceBlock from DatanodeDescriptor | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6809](https://issues.apache.org/jira/browse/HDFS-6809) | Move some Balancer's inner classes to standalone classes | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6800](https://issues.apache.org/jira/browse/HDFS-6800) | Support Datanode layout changes with rolling upgrade | Major | datanode | Colin Patrick McCabe | James Thomas | +| [HDFS-6798](https://issues.apache.org/jira/browse/HDFS-6798) | Add test case for incorrect data node condition during balancing | Major | balancer & mover | Benoy Antony | Benoy Antony | +| [HDFS-6796](https://issues.apache.org/jira/browse/HDFS-6796) | Improving the argument check during balancer command line parsing | Minor | balancer & mover | Benoy Antony | Benoy Antony | +| [HDFS-6794](https://issues.apache.org/jira/browse/HDFS-6794) | Update BlockManager methods to use DatanodeStorageInfo where possible | Minor | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6781](https://issues.apache.org/jira/browse/HDFS-6781) | Separate HDFS commands from CommandsManual.apt.vm | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6779](https://issues.apache.org/jira/browse/HDFS-6779) | Add missing version subcommand for hdfs | Minor | scripts | Allen Wittenauer | Sasaki Toru | +| [HDFS-6773](https://issues.apache.org/jira/browse/HDFS-6773) | MiniDFSCluster should skip edit log fsync by default | Major | namenode | Daryn Sharp | Stephen Chu | +| [HDFS-6772](https://issues.apache.org/jira/browse/HDFS-6772) | Get DN storages out of blockContentsStale state faster after NN restarts | Major | . | Ming Ma | Ming Ma | +| [HDFS-6758](https://issues.apache.org/jira/browse/HDFS-6758) | block writer should pass the expected block size to DataXceiverServer | Major | datanode, hdfs-client | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6755](https://issues.apache.org/jira/browse/HDFS-6755) | There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode | Major | . | Mit Desai | Mit Desai | +| [HDFS-6739](https://issues.apache.org/jira/browse/HDFS-6739) | Add getDatanodeStorageReport to ClientProtocol | Major | hdfs-client, namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6701](https://issues.apache.org/jira/browse/HDFS-6701) | Make seed optional in NetworkTopology#sortByDistance | Major | namenode | Ashwin Shankar | Ashwin Shankar | +| [HDFS-6700](https://issues.apache.org/jira/browse/HDFS-6700) | BlockPlacementPolicy shoud choose storage but not datanode for deletion | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6690](https://issues.apache.org/jira/browse/HDFS-6690) | Deduplicate xattr names in memory | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-6685](https://issues.apache.org/jira/browse/HDFS-6685) | Balancer should preserve storage type of replicas | Major | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6655](https://issues.apache.org/jira/browse/HDFS-6655) | Add 'header banner' to 'explorer.html' also in Namenode UI | Major | . | Vinayakumar B | Vinayakumar B | +| [HDFS-6643](https://issues.apache.org/jira/browse/HDFS-6643) | Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6627](https://issues.apache.org/jira/browse/HDFS-6627) | Rename DataNode#checkWriteAccess to checkReadAccess. | Major | datanode | Liang Xie | Liang Xie | +| [HDFS-6613](https://issues.apache.org/jira/browse/HDFS-6613) | Improve logging in caching classes | Minor | caching | Andrew Wang | Andrew Wang | +| [HDFS-6606](https://issues.apache.org/jira/browse/HDFS-6606) | Optimize HDFS Encrypted Transport performance | Major | datanode, hdfs-client, security | Yi Liu | Yi Liu | +| [HDFS-6597](https://issues.apache.org/jira/browse/HDFS-6597) | Add a new option to NN upgrade to terminate the process after upgrade on NN is completed | Major | namenode | Danilo Vunjak | Danilo Vunjak | +| [HDFS-6570](https://issues.apache.org/jira/browse/HDFS-6570) | add api that enables checking if a user has certain permissions on a file | Major | hdfs-client, namenode, webhdfs | Thejas M Nair | Jitendra Nath Pandey | +| [HDFS-6519](https://issues.apache.org/jira/browse/HDFS-6519) | Document oiv\_legacy command | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6511](https://issues.apache.org/jira/browse/HDFS-6511) | BlockManager#computeInvalidateWork() could do nothing | Minor | . | Juan Yu | Juan Yu | +| [HDFS-6482](https://issues.apache.org/jira/browse/HDFS-6482) | Use block ID-based block layout on datanodes | Major | datanode | James Thomas | James Thomas | +| [HDFS-6441](https://issues.apache.org/jira/browse/HDFS-6441) | Add ability to exclude/include specific datanodes while balancing | Major | balancer & mover | Benoy Antony | Benoy Antony | +| [HDFS-6188](https://issues.apache.org/jira/browse/HDFS-6188) | An ip whitelist based implementation of TrustedChannelResolver | Major | security | Benoy Antony | Benoy Antony | +| [HDFS-5202](https://issues.apache.org/jira/browse/HDFS-5202) | Support Centralized Cache Management on Windows. | Major | datanode | Colin Patrick McCabe | Chris Nauroth | +| [HDFS-5182](https://issues.apache.org/jira/browse/HDFS-5182) | BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-4486](https://issues.apache.org/jira/browse/HDFS-4486) | Add log category for long-running DFSClient notices | Minor | . | Todd Lipcon | Zhe Zhang | +| [HDFS-4120](https://issues.apache.org/jira/browse/HDFS-4120) | Add a new "-skipSharedEditsCheck" option for BootstrapStandby | Minor | ha, namenode | Liang Xie | Rakesh R | +| [HDFS-3851](https://issues.apache.org/jira/browse/HDFS-3851) | Make DFSOuputSteram$Packet default constructor reuse the other constructor | Trivial | hdfs-client | Jing Zhao | Jing Zhao | +| [HDFS-3528](https://issues.apache.org/jira/browse/HDFS-3528) | Use native CRC32 in DFS write path | Major | datanode, hdfs-client, performance | Todd Lipcon | James Thomas | +| [HDFS-2856](https://issues.apache.org/jira/browse/HDFS-2856) | Fix block protocol so that Datanodes don't require root or jsvc | Major | datanode, security | Owen O'Malley | Chris Nauroth | +| [HDFS-573](https://issues.apache.org/jira/browse/HDFS-573) | Porting libhdfs to Windows | Major | libhdfs | Ziliang Guo | Chris Nauroth | +| [MAPREDUCE-6086](https://issues.apache.org/jira/browse/MAPREDUCE-6086) | mapreduce.job.credentials.binary should allow all URIs | Major | security | zhihai xu | zhihai xu | +| [MAPREDUCE-6072](https://issues.apache.org/jira/browse/MAPREDUCE-6072) | Remove INSTALL document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-6071](https://issues.apache.org/jira/browse/MAPREDUCE-6071) | JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs | Trivial | client | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [MAPREDUCE-6070](https://issues.apache.org/jira/browse/MAPREDUCE-6070) | yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented | Trivial | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [MAPREDUCE-6024](https://issues.apache.org/jira/browse/MAPREDUCE-6024) | java.net.SocketTimeoutException in Fetcher caused jobs stuck for more than 1 hour | Critical | mr-am, task | zhaoyunjiong | zhaoyunjiong | +| [MAPREDUCE-5971](https://issues.apache.org/jira/browse/MAPREDUCE-5971) | Move the default options for distcp -p to DistCpOptionSwitch | Trivial | distcp | Charles Lamb | Charles Lamb | +| [MAPREDUCE-5970](https://issues.apache.org/jira/browse/MAPREDUCE-5970) | Provide a boolean switch to enable MR-AM profiling | Minor | applicationmaster, client | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5943](https://issues.apache.org/jira/browse/MAPREDUCE-5943) | Separate mapred commands from CommandsManual.apt.vm | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5906](https://issues.apache.org/jira/browse/MAPREDUCE-5906) | Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent" | Minor | . | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5130](https://issues.apache.org/jira/browse/MAPREDUCE-5130) | Add missing job config options to mapred-default.xml | Major | documentation | Sandy Ryza | Ray Chiang | +| [MAPREDUCE-4791](https://issues.apache.org/jira/browse/MAPREDUCE-4791) | Javadoc for KeyValueTextInputFormat should include default separator and how to change it | Minor | documentation | Matt Lavin | Akira AJISAKA | +| [MAPREDUCE-883](https://issues.apache.org/jira/browse/MAPREDUCE-883) | harchive: Document how to unarchive | Minor | documentation, harchive | Koji Noguchi | Akira AJISAKA | +| [YARN-2627](https://issues.apache.org/jira/browse/YARN-2627) | Add logs when attemptFailuresValidityInterval is enabled | Major | . | Xuan Gong | Xuan Gong | +| [YARN-2577](https://issues.apache.org/jira/browse/YARN-2577) | Clarify ACL delimiter and how to configure ACL groups only | Trivial | documentation, fairscheduler | Miklos Christine | Miklos Christine | +| [YARN-2539](https://issues.apache.org/jira/browse/YARN-2539) | FairScheduler: Set the default value for maxAMShare to 0.5 | Minor | . | Wei Yan | Wei Yan | +| [YARN-2531](https://issues.apache.org/jira/browse/YARN-2531) | CGroups - Admins should be allowed to enforce strict cpu limits | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2448](https://issues.apache.org/jira/browse/YARN-2448) | RM should expose the resource types considered during scheduling when AMs register | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2411](https://issues.apache.org/jira/browse/YARN-2411) | [Capacity Scheduler] support simple user and group mappings to queues | Major | capacityscheduler | Ram Venkatesh | Ram Venkatesh | +| [YARN-2399](https://issues.apache.org/jira/browse/YARN-2399) | FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-2377](https://issues.apache.org/jira/browse/YARN-2377) | Localization exception stack traces are not passed as diagnostic info | Major | nodemanager | Gera Shegalov | Gera Shegalov | +| [YARN-2372](https://issues.apache.org/jira/browse/YARN-2372) | There are Chinese Characters in the FairScheduler's document | Minor | documentation | Fengdong Yu | Fengdong Yu | +| [YARN-2361](https://issues.apache.org/jira/browse/YARN-2361) | RMAppAttempt state machine entries for KILLED state has duplicate event entries | Trivial | resourcemanager | zhihai xu | zhihai xu | +| [YARN-2352](https://issues.apache.org/jira/browse/YARN-2352) | FairScheduler: Collect metrics on duration of critical methods that affect performance | Major | scheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-2343](https://issues.apache.org/jira/browse/YARN-2343) | Improve error message on token expire exception | Trivial | . | Li Lu | Li Lu | +| [YARN-2337](https://issues.apache.org/jira/browse/YARN-2337) | ResourceManager sets ClientRMService in RMContext multiple times | Trivial | resourcemanager | zhihai xu | zhihai xu | +| [YARN-2323](https://issues.apache.org/jira/browse/YARN-2323) | FairShareComparator creates too many Resource objects | Minor | fairscheduler | Hong Zhiguo | Hong Zhiguo | +| [YARN-2294](https://issues.apache.org/jira/browse/YARN-2294) | Update sample program and documentations for writing YARN Application | Major | . | Li Lu | | +| [YARN-2274](https://issues.apache.org/jira/browse/YARN-2274) | FairScheduler: Add debug information about cluster capacity, availability and reservations | Trivial | fairscheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-2214](https://issues.apache.org/jira/browse/YARN-2214) | FairScheduler: preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness | Major | scheduler | Ashwin Shankar | Ashwin Shankar | +| [YARN-2209](https://issues.apache.org/jira/browse/YARN-2209) | Replace AM resync/shutdown command with corresponding exceptions | Major | . | Jian He | Jian He | +| [YARN-2198](https://issues.apache.org/jira/browse/YARN-2198) | Remove the need to run NodeManager as privileged account for Windows Secure Container Executor | Major | . | Remus Rusanu | Remus Rusanu | +| [YARN-2197](https://issues.apache.org/jira/browse/YARN-2197) | Add a link to YARN CHANGES.txt in the left side of doc | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [YARN-1994](https://issues.apache.org/jira/browse/YARN-1994) | Expose YARN/MR endpoints on multiple interfaces | Major | nodemanager, resourcemanager, webapp | Arpit Agarwal | Craig Welch | +| [YARN-1954](https://issues.apache.org/jira/browse/YARN-1954) | Add waitFor to AMRMClient(Async) | Major | client | Zhijie Shen | Tsuyoshi Ozawa | +| [YARN-1918](https://issues.apache.org/jira/browse/YARN-1918) | Typo in description and error message for 'yarn.resourcemanager.cluster-id' | Trivial | . | Devaraj K | Anandha L Ranganathan | +| [YARN-1769](https://issues.apache.org/jira/browse/YARN-1769) | CapacityScheduler: Improve reservations | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-666](https://issues.apache.org/jira/browse/YARN-666) | [Umbrella] Support rolling upgrades in YARN | Major | . | Siddharth Seth | | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11286](https://issues.apache.org/jira/browse/HADOOP-11286) | Map/Reduce dangerously adds Guava @Beta class to CryptoUtils | Blocker | . | Christopher Tubbs | | +| [HADOOP-11282](https://issues.apache.org/jira/browse/HADOOP-11282) | Skip NFS TestShellBasedIdMapping tests that are irrelevant on Windows. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [HADOOP-11280](https://issues.apache.org/jira/browse/HADOOP-11280) | TestWinUtils#testChmod fails after removal of NO\_PROPAGATE\_INHERIT\_ACE. | Trivial | native, test | Chris Nauroth | Chris Nauroth | +| [HADOOP-11265](https://issues.apache.org/jira/browse/HADOOP-11265) | Credential and Key Shell Commands not available on Windows | Major | scripts | Larry McCay | Larry McCay | +| [HADOOP-11260](https://issues.apache.org/jira/browse/HADOOP-11260) | Patch up Jetty to disable SSLv3 | Blocker | security | Karthik Kambatla | Mike Yoder | +| [HADOOP-11254](https://issues.apache.org/jira/browse/HADOOP-11254) | Promoting AccessControlList to be public | Major | . | Zhijie Shen | Zhijie Shen | +| [HADOOP-11250](https://issues.apache.org/jira/browse/HADOOP-11250) | fix endmacro of set\_find\_shared\_library\_without\_version in CMakeLists | Minor | build | Yi Liu | Yi Liu | +| [HADOOP-11247](https://issues.apache.org/jira/browse/HADOOP-11247) | Fix a couple javac warnings in NFS | Major | nfs | Brandon Li | Brandon Li | +| [HADOOP-11243](https://issues.apache.org/jira/browse/HADOOP-11243) | SSLFactory shouldn't allow SSLv3 | Blocker | . | Wei Yan | Wei Yan | +| [HADOOP-11233](https://issues.apache.org/jira/browse/HADOOP-11233) | hadoop.security.kms.client.encrypted.key.cache.expiry property spelled wrong in core-default | Minor | conf | Steve Loughran | Stephen Chu | +| [HADOOP-11228](https://issues.apache.org/jira/browse/HADOOP-11228) | winutils task: unsecure path should not call AddNodeManagerAndUserACEsToObject | Major | . | Remus Rusanu | Remus Rusanu | +| [HADOOP-11221](https://issues.apache.org/jira/browse/HADOOP-11221) | JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative | Major | util | Jinghui Wang | Jinghui Wang | +| [HADOOP-11217](https://issues.apache.org/jira/browse/HADOOP-11217) | Disable SSLv3 in KMS | Blocker | kms | Robert Kanter | Robert Kanter | +| [HADOOP-11207](https://issues.apache.org/jira/browse/HADOOP-11207) | DelegationTokenAuthenticationHandler needs to support DT operations for proxy user | Major | security | Zhijie Shen | Zhijie Shen | +| [HADOOP-11198](https://issues.apache.org/jira/browse/HADOOP-11198) | Fix typo in javadoc for FileSystem#listStatus() | Minor | . | Ted Yu | Li Lu | +| [HADOOP-11194](https://issues.apache.org/jira/browse/HADOOP-11194) | Ignore .keep files | Major | . | Karthik Kambatla | Karthik Kambatla | +| [HADOOP-11193](https://issues.apache.org/jira/browse/HADOOP-11193) | Fix uninitialized variables in NativeIO.c | Major | native | Xiaoyu Yao | Xiaoyu Yao | +| [HADOOP-11182](https://issues.apache.org/jira/browse/HADOOP-11182) | GraphiteSink emits wrong timestamps | Major | . | Sascha Coenen | Ravi Prakash | +| [HADOOP-11181](https://issues.apache.org/jira/browse/HADOOP-11181) | o.a.h.security.token.delegation.DelegationTokenManager should be more generalized to handle other DelegationTokenIdentifier | Major | security | Zhijie Shen | Zhijie Shen | +| [HADOOP-11179](https://issues.apache.org/jira/browse/HADOOP-11179) | Tarball as local resource type archive fails to localize on Windows | Major | . | Hitesh Shah | Craig Welch | +| [HADOOP-11178](https://issues.apache.org/jira/browse/HADOOP-11178) | Fix findbugs exclude file | Minor | build | Arun Suresh | Arun Suresh | +| [HADOOP-11176](https://issues.apache.org/jira/browse/HADOOP-11176) | KMSClientProvider authentication fails when both currentUgi and loginUgi are a proxied user | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11175](https://issues.apache.org/jira/browse/HADOOP-11175) | Fix several issues of hadoop security configuration in user doc. | Trivial | documentation, security | Yi Liu | Yi Liu | +| [HADOOP-11174](https://issues.apache.org/jira/browse/HADOOP-11174) | Delegation token for KMS should only be got once if it already exists | Major | kms, security | Yi Liu | Yi Liu | +| [HADOOP-11170](https://issues.apache.org/jira/browse/HADOOP-11170) | ZKDelegationTokenSecretManager fails to renewToken created by a peer | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11169](https://issues.apache.org/jira/browse/HADOOP-11169) | Fix DelegationTokenAuthenticatedURL to pass the connection Configurator to the authenticator | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11168](https://issues.apache.org/jira/browse/HADOOP-11168) | Remove duplicated entry "dfs.webhdfs.enabled" in the user doc | Trivial | documentation | Yi Liu | Yi Liu | +| [HADOOP-11163](https://issues.apache.org/jira/browse/HADOOP-11163) | MetricsSystemImpl may miss a registered source | Minor | metrics | Chuan Liu | Chuan Liu | +| [HADOOP-11161](https://issues.apache.org/jira/browse/HADOOP-11161) | Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11160](https://issues.apache.org/jira/browse/HADOOP-11160) | Fix typo in nfs3 server duplicate entry reporting | Trivial | nfs | Charles Lamb | Charles Lamb | +| [HADOOP-11154](https://issues.apache.org/jira/browse/HADOOP-11154) | Update BUILDING.txt to state that CMake 3.0 or newer is required on Mac. | Trivial | documentation, native | Chris Nauroth | Chris Nauroth | +| [HADOOP-11151](https://issues.apache.org/jira/browse/HADOOP-11151) | Automatically refresh auth token and retry on auth failure | Major | security | zhubin | Arun Suresh | +| [HADOOP-11145](https://issues.apache.org/jira/browse/HADOOP-11145) | TestFairCallQueue fails | Major | test | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-11143](https://issues.apache.org/jira/browse/HADOOP-11143) | NetUtils.wrapException loses inner stack trace on BindException | Minor | net | Steve Loughran | Steve Loughran | +| [HADOOP-11140](https://issues.apache.org/jira/browse/HADOOP-11140) | hadoop-aws only need test-scoped dependency on hadoop-common's tests jar | Major | . | Juan Yu | Juan Yu | +| [HADOOP-11133](https://issues.apache.org/jira/browse/HADOOP-11133) | Should trim the content of keystore password file for JavaKeyStoreProvider | Minor | security | zhubin | Yi Liu | +| [HADOOP-11130](https://issues.apache.org/jira/browse/HADOOP-11130) | NFS updateMaps OS check is reversed | Major | nfs | Allen Wittenauer | Brandon Li | +| [HADOOP-11122](https://issues.apache.org/jira/browse/HADOOP-11122) | Fix findbugs in ZK DelegationTokenSecretManagers | Blocker | . | Karthik Kambatla | Arun Suresh | +| [HADOOP-11113](https://issues.apache.org/jira/browse/HADOOP-11113) | Namenode not able to reconnect to KMS after KMS restart | Major | security | Arun Suresh | Arun Suresh | +| [HADOOP-11112](https://issues.apache.org/jira/browse/HADOOP-11112) | TestKMSWithZK does not use KEY\_PROVIDER\_URI | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11110](https://issues.apache.org/jira/browse/HADOOP-11110) | JavaKeystoreProvider should not report a key as created if it was not flushed to the backing file | Major | . | Andrew Wang | Arun Suresh | +| [HADOOP-11109](https://issues.apache.org/jira/browse/HADOOP-11109) | Site build is broken | Major | . | Jian He | Jian He | +| [HADOOP-11105](https://issues.apache.org/jira/browse/HADOOP-11105) | MetricsSystemImpl could leak memory in registered callbacks | Major | metrics | Chuan Liu | Chuan Liu | +| [HADOOP-11099](https://issues.apache.org/jira/browse/HADOOP-11099) | KMS return HTTP UNAUTHORIZED 401 on ACL failure | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11097](https://issues.apache.org/jira/browse/HADOOP-11097) | kms docs say proxyusers, not proxyuser for config params | Trivial | documentation | Charles Lamb | Charles Lamb | +| [HADOOP-11096](https://issues.apache.org/jira/browse/HADOOP-11096) | KMS: KeyAuthorizationKeyProvider should verify the keyversion belongs to the keyname on decrypt | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11091](https://issues.apache.org/jira/browse/HADOOP-11091) | Eliminate old configuration parameter names from s3a | Minor | fs/s3 | David S. Wang | David S. Wang | +| [HADOOP-11085](https://issues.apache.org/jira/browse/HADOOP-11085) | Excessive logging by org.apache.hadoop.util.Progress when value is NaN | Major | . | Mit Desai | Mit Desai | +| [HADOOP-11083](https://issues.apache.org/jira/browse/HADOOP-11083) | After refactoring of HTTP proxyuser to common, doAs param is case sensitive | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11077](https://issues.apache.org/jira/browse/HADOOP-11077) | NPE if hosts not specified in ProxyUsers | Major | security | Gregory Chanan | Gregory Chanan | +| [HADOOP-11069](https://issues.apache.org/jira/browse/HADOOP-11069) | KMSClientProvider should use getAuthenticationMethod() to determine if in proxyuser mode or not | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11067](https://issues.apache.org/jira/browse/HADOOP-11067) | warning message 'ssl.client.truststore.location has not been set' gets printed for hftp command | Major | . | Yesha Vora | Xiaoyu Yao | +| [HADOOP-11064](https://issues.apache.org/jira/browse/HADOOP-11064) | UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due to NativeCRC32 method changes | Blocker | native | Steve Loughran | Chris Nauroth | +| [HADOOP-11063](https://issues.apache.org/jira/browse/HADOOP-11063) | KMS cannot deploy on Windows, because class names are too long. | Blocker | . | Chris Nauroth | Chris Nauroth | +| [HADOOP-11062](https://issues.apache.org/jira/browse/HADOOP-11062) | CryptoCodec testcases requiring OpenSSL should be run only if -Pnative is used | Major | security, test | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-11056](https://issues.apache.org/jira/browse/HADOOP-11056) | OsSecureRandom.setConf() might leak file descriptors. | Major | security | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-11049](https://issues.apache.org/jira/browse/HADOOP-11049) | javax package system class default is too broad | Minor | util | Sangjin Lee | Sangjin Lee | +| [HADOOP-11048](https://issues.apache.org/jira/browse/HADOOP-11048) | user/custom LogManager fails to load if the client classloader is enabled | Minor | util | Sangjin Lee | Sangjin Lee | +| [HADOOP-11040](https://issues.apache.org/jira/browse/HADOOP-11040) | Return value of read(ByteBuffer buf) in CryptoInputStream is incorrect in some cases | Major | security | Yi Liu | Yi Liu | +| [HADOOP-11036](https://issues.apache.org/jira/browse/HADOOP-11036) | Add build directory to .gitignore | Minor | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-11012](https://issues.apache.org/jira/browse/HADOOP-11012) | hadoop fs -text of zero-length file causes EOFException | Major | fs | Eric Payne | Eric Payne | +| [HADOOP-11005](https://issues.apache.org/jira/browse/HADOOP-11005) | Fix HTTP content type for ReconfigurationServlet | Minor | conf | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HADOOP-10989](https://issues.apache.org/jira/browse/HADOOP-10989) | Work around buggy getgrouplist() implementations on Linux that return 0 on failure | Major | native | Chris Nauroth | Chris Nauroth | +| [HADOOP-10973](https://issues.apache.org/jira/browse/HADOOP-10973) | Native Libraries Guide contains format error | Minor | documentation | Peter Klavins | Peter Klavins | +| [HADOOP-10972](https://issues.apache.org/jira/browse/HADOOP-10972) | Native Libraries Guide contains mis-spelt build line | Major | documentation | Peter Klavins | Peter Klavins | +| [HADOOP-10968](https://issues.apache.org/jira/browse/HADOOP-10968) | hadoop native build fails to detect java\_libarch on ppc64le | Major | build | Dinar Valeev | | +| [HADOOP-10966](https://issues.apache.org/jira/browse/HADOOP-10966) | Hadoop Common native compilation broken in windows | Blocker | native | Vinayakumar B | David Villegas | +| [HADOOP-10964](https://issues.apache.org/jira/browse/HADOOP-10964) | Small fix for NetworkTopologyWithNodeGroup#sortByDistance | Minor | . | Yi Liu | Yi Liu | +| [HADOOP-10962](https://issues.apache.org/jira/browse/HADOOP-10962) | Flags for posix\_fadvise are not valid in some architectures | Major | native | David Villegas | David Villegas | +| [HADOOP-10946](https://issues.apache.org/jira/browse/HADOOP-10946) | Fix a bunch of typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | +| [HADOOP-10939](https://issues.apache.org/jira/browse/HADOOP-10939) | Fix TestKeyProviderFactory testcases to use default 128 bit length keys | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-10937](https://issues.apache.org/jira/browse/HADOOP-10937) | Need to set version name correctly before decrypting EEK | Major | security | Arun Suresh | Arun Suresh | +| [HADOOP-10931](https://issues.apache.org/jira/browse/HADOOP-10931) | compile error on project "Apache Hadoop OpenStack support" | Minor | build, fs/swift | xukun | | +| [HADOOP-10929](https://issues.apache.org/jira/browse/HADOOP-10929) | Typo in Configuration.getPasswordFromCredentialProviders | Trivial | security | Larry McCay | Larry McCay | +| [HADOOP-10928](https://issues.apache.org/jira/browse/HADOOP-10928) | Incorrect usage on 'hadoop credential list' | Trivial | security | Josh Elser | Josh Elser | +| [HADOOP-10927](https://issues.apache.org/jira/browse/HADOOP-10927) | Fix CredentialShell help behavior and error codes | Minor | security | Josh Elser | Josh Elser | +| [HADOOP-10925](https://issues.apache.org/jira/browse/HADOOP-10925) | Compilation fails in native link0 function on Windows. | Blocker | native | Chris Nauroth | Chris Nauroth | +| [HADOOP-10920](https://issues.apache.org/jira/browse/HADOOP-10920) | site plugin couldn't parse hadoop-kms index.apt.vm | Minor | documentation | Ted Yu | Akira AJISAKA | +| [HADOOP-10918](https://issues.apache.org/jira/browse/HADOOP-10918) | JMXJsonServlet fails when used within Tomcat | Major | . | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10911](https://issues.apache.org/jira/browse/HADOOP-10911) | hadoop.auth cookie after HADOOP-10710 still not proper according to RFC2109 | Major | security | Gregory Chanan | | +| [HADOOP-10900](https://issues.apache.org/jira/browse/HADOOP-10900) | CredentialShell args should use single-dash style | Minor | . | Andrew Wang | Andrew Wang | +| [HADOOP-10887](https://issues.apache.org/jira/browse/HADOOP-10887) | Add XAttrs to ViewFs and make XAttrs + ViewFileSystem internal dir behavior consistent | Major | fs, test | Stephen Chu | Stephen Chu | +| [HADOOP-10880](https://issues.apache.org/jira/browse/HADOOP-10880) | Move HTTP delegation tokens out of URL querystring to a header | Blocker | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10866](https://issues.apache.org/jira/browse/HADOOP-10866) | RawLocalFileSystem fails to read symlink targets via the stat command when the format of stat command uses non-curly quotes | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-10862](https://issues.apache.org/jira/browse/HADOOP-10862) | Miscellaneous trivial corrections to KMS classes | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10857](https://issues.apache.org/jira/browse/HADOOP-10857) | Native Libraries Guide doen't mention a dependency on openssl-development package | Major | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-10851](https://issues.apache.org/jira/browse/HADOOP-10851) | NetgroupCache does not remove group memberships | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10843](https://issues.apache.org/jira/browse/HADOOP-10843) | TestGridmixRecord unit tests failure on PowerPC | Major | test, tools | Jinghui Wang | Jinghui Wang | +| [HADOOP-10842](https://issues.apache.org/jira/browse/HADOOP-10842) | CryptoExtension generateEncryptedKey method should receive the key name | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HADOOP-10830](https://issues.apache.org/jira/browse/HADOOP-10830) | Missing lock in JavaKeyStoreProvider.createCredentialEntry | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10820](https://issues.apache.org/jira/browse/HADOOP-10820) | Throw an exception in GenericOptionsParser when passed an empty Path | Minor | . | Alex Holmes | zhihai xu | +| [HADOOP-10816](https://issues.apache.org/jira/browse/HADOOP-10816) | KeyShell returns -1 on error to the shell, should be 1 | Major | security | Mike Yoder | Mike Yoder | +| [HADOOP-10814](https://issues.apache.org/jira/browse/HADOOP-10814) | Update Tomcat version used by HttpFS and KMS to latest 6.x version | Major | . | Alejandro Abdelnur | Robert Kanter | +| [HADOOP-10810](https://issues.apache.org/jira/browse/HADOOP-10810) | Clean up native code compilation warnings. | Minor | native | Chris Nauroth | Chris Nauroth | +| [HADOOP-10781](https://issues.apache.org/jira/browse/HADOOP-10781) | Unportable getgrouplist() usage breaks FreeBSD | Major | . | Dmitry Sivachenko | Dmitry Sivachenko | +| [HADOOP-10780](https://issues.apache.org/jira/browse/HADOOP-10780) | hadoop\_user\_info\_alloc fails on FreeBSD due to incorrect sysconf use | Major | . | Dmitry Sivachenko | Dmitry Sivachenko | +| [HADOOP-10744](https://issues.apache.org/jira/browse/HADOOP-10744) | LZ4 Compression fails to recognize PowerPC Little Endian Architecture | Major | io, native | Ayappan | Bert Sanders | +| [HADOOP-10733](https://issues.apache.org/jira/browse/HADOOP-10733) | Potential null dereference in CredentialShell#promptForCredential() | Minor | . | Ted Yu | Ted Yu | +| [HADOOP-10732](https://issues.apache.org/jira/browse/HADOOP-10732) | Update without holding write lock in JavaKeyStoreProvider#innerSetCredential() | Minor | . | Ted Yu | Ted Yu | +| [HADOOP-10681](https://issues.apache.org/jira/browse/HADOOP-10681) | Remove synchronized blocks from SnappyCodec and ZlibCodec buffering inner loop | Major | performance | Gopal V | Gopal V | +| [HADOOP-10677](https://issues.apache.org/jira/browse/HADOOP-10677) | ExportSnapshot fails on kerberized cluster using s3a | Major | fs/s3 | David S. Wang | David S. Wang | +| [HADOOP-10676](https://issues.apache.org/jira/browse/HADOOP-10676) | S3AOutputStream not reading new config knobs for multipart configs | Major | fs/s3 | David S. Wang | David S. Wang | +| [HADOOP-10673](https://issues.apache.org/jira/browse/HADOOP-10673) | Update rpc metrics when the call throws an exception | Major | . | Ming Ma | Ming Ma | +| [HADOOP-10645](https://issues.apache.org/jira/browse/HADOOP-10645) | TestKMS fails because race condition writing acl files | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10611](https://issues.apache.org/jira/browse/HADOOP-10611) | KMS, keyVersion name should not be assumed to be keyName@versionNumber | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10591](https://issues.apache.org/jira/browse/HADOOP-10591) | Compression codecs must used pooled direct buffers or deallocate direct buffers when stream is closed | Major | . | Hari Shreedharan | Colin Patrick McCabe | +| [HADOOP-10586](https://issues.apache.org/jira/browse/HADOOP-10586) | KeyShell doesn't allow setting Options via CLI | Minor | bin | Charles Lamb | Charles Lamb | +| [HADOOP-10583](https://issues.apache.org/jira/browse/HADOOP-10583) | bin/hadoop key throws NPE with no args and assorted other fixups | Minor | bin | Charles Lamb | Charles Lamb | +| [HADOOP-10552](https://issues.apache.org/jira/browse/HADOOP-10552) | Fix usage and example at FileSystemShell.apt.vm | Trivial | documentation | Kenji Kikushima | Kenji Kikushima | +| [HADOOP-10534](https://issues.apache.org/jira/browse/HADOOP-10534) | KeyProvider API should using windowing for retrieving metadata | Major | . | Owen O'Malley | Owen O'Malley | +| [HADOOP-10507](https://issues.apache.org/jira/browse/HADOOP-10507) | FsShell setfacl can throw ArrayIndexOutOfBoundsException when no perm is specified | Minor | fs | Stephen Chu | sathish | +| [HADOOP-10488](https://issues.apache.org/jira/browse/HADOOP-10488) | TestKeyProviderFactory fails randomly | Major | test | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10404](https://issues.apache.org/jira/browse/HADOOP-10404) | Some accesses to DomainSocketWatcher#closed are not protected by lock | Minor | . | Ted Yu | Colin Patrick McCabe | +| [HADOOP-10402](https://issues.apache.org/jira/browse/HADOOP-10402) | Configuration.getValByRegex does not substitute for variables | Major | . | Robert Kanter | Robert Kanter | +| [HADOOP-10244](https://issues.apache.org/jira/browse/HADOOP-10244) | TestKeyShell improperly tests the results of a Delete | Major | security | Larry McCay | Larry McCay | +| [HADOOP-10237](https://issues.apache.org/jira/browse/HADOOP-10237) | JavaKeyStoreProvider needs to set keystore permissions properly | Major | security | Larry McCay | Larry McCay | +| [HADOOP-10224](https://issues.apache.org/jira/browse/HADOOP-10224) | JavaKeyStoreProvider has to protect against corrupting underlying store | Major | security | Larry McCay | Arun Suresh | +| [HADOOP-10177](https://issues.apache.org/jira/browse/HADOOP-10177) | Create CLI tools for managing keys via the KeyProvider API | Major | security | Owen O'Malley | Larry McCay | +| [HADOOP-10141](https://issues.apache.org/jira/browse/HADOOP-10141) | Create an API to separate encryption key storage from applications | Major | security | Owen O'Malley | Owen O'Malley | +| [HADOOP-10131](https://issues.apache.org/jira/browse/HADOOP-10131) | NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if excluded nodes passed are not part of the cluster tree | Major | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-10121](https://issues.apache.org/jira/browse/HADOOP-10121) | Fix javadoc spelling for HadoopArchives#writeTopLevelDirs | Trivial | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10059](https://issues.apache.org/jira/browse/HADOOP-10059) | RPC authentication and authorization metrics overflow to negative values on busy clusters | Minor | metrics | Jason Lowe | Tsuyoshi Ozawa | +| [HADOOP-10037](https://issues.apache.org/jira/browse/HADOOP-10037) | s3n read truncated, but doesn't throw exception | Major | fs/s3 | David Rosenstrauch | | +| [HADOOP-9989](https://issues.apache.org/jira/browse/HADOOP-9989) | Bug introduced in HADOOP-9374, which parses the -tokenCacheFile as binary file but set it to the configuration as JSON file. | Major | security, util | Jinghui Wang | zhihai xu | +| [HADOOP-9921](https://issues.apache.org/jira/browse/HADOOP-9921) | daemon scripts should remove pid file on stop call after stop or process is found not running | Major | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-9740](https://issues.apache.org/jira/browse/HADOOP-9740) | FsShell's Text command does not read avro data files stored on HDFS | Major | fs | Allan Yan | Allan Yan | +| [HADOOP-9576](https://issues.apache.org/jira/browse/HADOOP-9576) | Make NetUtils.wrapException throw EOFException instead of wrapping it as IOException | Major | . | Jian He | Steve Loughran | +| [HADOOP-8808](https://issues.apache.org/jira/browse/HADOOP-8808) | Update FsShell documentation to mention deprecation of some of the commands, and mention alternatives | Major | documentation, fs | Hemanth Yamijala | Akira AJISAKA | +| [HADOOP-8158](https://issues.apache.org/jira/browse/HADOOP-8158) | Interrupting hadoop fs -put from the command line causes a LeaseExpiredException | Major | . | Todd Lipcon | Daryn Sharp | +| [HDFS-7391](https://issues.apache.org/jira/browse/HDFS-7391) | Renable SSLv2Hello in HttpFS | Blocker | webhdfs | Robert Kanter | Robert Kanter | +| [HDFS-7387](https://issues.apache.org/jira/browse/HDFS-7387) | NFS may only do partial commit due to a race between COMMIT and write | Critical | nfs | Brandon Li | Brandon Li | +| [HDFS-7385](https://issues.apache.org/jira/browse/HDFS-7385) | ThreadLocal used in FSEditLog class causes FSImage permission mess up | Blocker | namenode | jiangyu | jiangyu | +| [HDFS-7383](https://issues.apache.org/jira/browse/HDFS-7383) | DataNode.requestShortCircuitFdsForRead may throw NullPointerException | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7382](https://issues.apache.org/jira/browse/HDFS-7382) | DataNode in secure mode may throw NullPointerException if client connects before DataNode registers itself with NameNode. | Minor | datanode, security | Chris Nauroth | Chris Nauroth | +| [HDFS-7379](https://issues.apache.org/jira/browse/HDFS-7379) | TestBalancer#testBalancerWithRamDisk creates test files incorrectly | Minor | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7367](https://issues.apache.org/jira/browse/HDFS-7367) | HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol. | Major | hdfs-client | Chris Nauroth | Chris Nauroth | +| [HDFS-7364](https://issues.apache.org/jira/browse/HDFS-7364) | Balancer always shows zero Bytes Already Moved | Minor | balancer & mover | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7359](https://issues.apache.org/jira/browse/HDFS-7359) | NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address. | Major | journal-node, namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-7340](https://issues.apache.org/jira/browse/HDFS-7340) | make rollingUpgrade start/finalize idempotent | Major | ha | Arpit Gupta | Jing Zhao | +| [HDFS-7334](https://issues.apache.org/jira/browse/HDFS-7334) | Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures | Minor | test | Charles Lamb | Charles Lamb | +| [HDFS-7328](https://issues.apache.org/jira/browse/HDFS-7328) | TestTraceAdmin assumes Unix line endings. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7309](https://issues.apache.org/jira/browse/HDFS-7309) | XMLUtils.mangleXmlString doesn't seem to handle less than sign | Minor | . | Ravi Prakash | Colin Patrick McCabe | +| [HDFS-7305](https://issues.apache.org/jira/browse/HDFS-7305) | NPE seen in wbhdfs FS while running SLive | Minor | webhdfs | Arpit Gupta | Jing Zhao | +| [HDFS-7300](https://issues.apache.org/jira/browse/HDFS-7300) | The getMaxNodesPerRack() method in BlockPlacementPolicyDefault is flawed | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7296](https://issues.apache.org/jira/browse/HDFS-7296) | HdfsConstants#MEMORY\_STORAGE\_POLICY\_ID and HdfsConstants#MEMORY\_STORAGE\_POLICY\_ID are missing in branch-2 | Minor | . | Jing Zhao | Jing Zhao | +| [HDFS-7287](https://issues.apache.org/jira/browse/HDFS-7287) | The OfflineImageViewer (OIV) can output invalid XML depending on the filename | Major | . | Ravi Prakash | Ravi Prakash | +| [HDFS-7274](https://issues.apache.org/jira/browse/HDFS-7274) | Disable SSLv3 in HttpFS | Blocker | webhdfs | Robert Kanter | Robert Kanter | +| [HDFS-7259](https://issues.apache.org/jira/browse/HDFS-7259) | Unresponseive NFS mount point due to deferred COMMIT response | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7243](https://issues.apache.org/jira/browse/HDFS-7243) | HDFS concat operation should not be allowed in Encryption Zone | Major | encryption, namenode | Yi Liu | Charles Lamb | +| [HDFS-7237](https://issues.apache.org/jira/browse/HDFS-7237) | namenode -rollingUpgrade throws ArrayIndexOutOfBoundsException | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7236](https://issues.apache.org/jira/browse/HDFS-7236) | Fix TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7226](https://issues.apache.org/jira/browse/HDFS-7226) | TestDNFencing.testQueueingWithAppend failed often in latest test | Major | ha | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7221](https://issues.apache.org/jira/browse/HDFS-7221) | TestDNFencingWithReplication fails consistently | Minor | test | Charles Lamb | Charles Lamb | +| [HDFS-7218](https://issues.apache.org/jira/browse/HDFS-7218) | FSNamesystem ACL operations should write to audit log on failure | Minor | namenode | Charles Lamb | Charles Lamb | +| [HDFS-7208](https://issues.apache.org/jira/browse/HDFS-7208) | NN doesn't schedule replication when a DN storage fails | Major | namenode | Ming Ma | Ming Ma | +| [HDFS-7199](https://issues.apache.org/jira/browse/HDFS-7199) | DFSOutputStream should not silently drop data if DataStreamer crashes with an unchecked exception | Critical | hdfs-client | Jason Lowe | Rushabh S Shah | +| [HDFS-7185](https://issues.apache.org/jira/browse/HDFS-7185) | The active NameNode will not accept an fsimage sent from the standby during rolling upgrade | Major | namenode | Colin Patrick McCabe | Jing Zhao | +| [HDFS-7180](https://issues.apache.org/jira/browse/HDFS-7180) | NFSv3 gateway frequently gets stuck due to GC | Critical | nfs | Eric Zhiqiang Ma | Brandon Li | +| [HDFS-7178](https://issues.apache.org/jira/browse/HDFS-7178) | Additional unit test for replica write with full disk | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7176](https://issues.apache.org/jira/browse/HDFS-7176) | The namenode usage message doesn't include "-rollingupgrade started" | Minor | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7172](https://issues.apache.org/jira/browse/HDFS-7172) | Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7169](https://issues.apache.org/jira/browse/HDFS-7169) | Fix a findbugs warning in ReplaceDatanodeOnFailure | Minor | build | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7167](https://issues.apache.org/jira/browse/HDFS-7167) | NPE while running Mover if the given path is for a file | Major | balancer & mover | Prabushankar Chinnasamy | Jing Zhao | +| [HDFS-7162](https://issues.apache.org/jira/browse/HDFS-7162) | Wrong path when deleting through fuse-dfs a file which already exists in trash | Major | fuse-dfs | Chengbing Liu | Chengbing Liu | +| [HDFS-7156](https://issues.apache.org/jira/browse/HDFS-7156) | Fsck documentation is outdated. | Major | documentation | Konstantin Shvachko | Masahiro Yamaguchi | +| [HDFS-7148](https://issues.apache.org/jira/browse/HDFS-7148) | TestEncryptionZones#testIsEncryptedMethod fails on branch-2 after archival storage merge | Major | encryption | Andrew Wang | Andrew Wang | +| [HDFS-7147](https://issues.apache.org/jira/browse/HDFS-7147) | Update archival storage user documentation | Blocker | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7132](https://issues.apache.org/jira/browse/HDFS-7132) | hdfs namenode -metadataVersion command does not honor configured name dirs | Minor | namenode | Charles Lamb | Charles Lamb | +| [HDFS-7131](https://issues.apache.org/jira/browse/HDFS-7131) | During HA upgrade, JournalNode should create a new committedTxnId file in the current directory | Major | qjm | Jing Zhao | Jing Zhao | +| [HDFS-7130](https://issues.apache.org/jira/browse/HDFS-7130) | TestDataTransferKeepalive fails intermittently on Windows. | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7127](https://issues.apache.org/jira/browse/HDFS-7127) | TestLeaseRecovery leaks MiniDFSCluster instances. | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7122](https://issues.apache.org/jira/browse/HDFS-7122) | Use of ThreadLocal\ results in poor block placement | Blocker | namenode | Jeff Buell | Andrew Wang | +| [HDFS-7111](https://issues.apache.org/jira/browse/HDFS-7111) | TestSafeMode assumes Unix line endings in safe mode tip. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7110](https://issues.apache.org/jira/browse/HDFS-7110) | Skip tests related to short-circuit read on platforms that do not currently implement short-circuit read. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7109](https://issues.apache.org/jira/browse/HDFS-7109) | TestDataStorage does not release file locks between tests. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7107](https://issues.apache.org/jira/browse/HDFS-7107) | Avoid Findbugs warning for synchronization on AbstractNNFailoverProxyProvider#fallbackToSimpleAuth. | Trivial | ha | Chris Nauroth | Chris Nauroth | +| [HDFS-7106](https://issues.apache.org/jira/browse/HDFS-7106) | Reconfiguring DataNode volumes does not release the lock files in removed volumes. | Major | datanode | Chris Nauroth | Chris Nauroth | +| [HDFS-7105](https://issues.apache.org/jira/browse/HDFS-7105) | Fix TestJournalNode#testFailToStartWithBadConfig to match log output change | Minor | test | Ray Chiang | Ray Chiang | +| [HDFS-7104](https://issues.apache.org/jira/browse/HDFS-7104) | Fix and clarify INodeInPath getter functions | Minor | . | Zhe Zhang | Zhe Zhang | +| [HDFS-7096](https://issues.apache.org/jira/browse/HDFS-7096) | Fix TestRpcProgramNfs3 to use DFS\_ENCRYPTION\_KEY\_PROVIDER\_URI | Minor | test | Charles Lamb | Charles Lamb | +| [HDFS-7075](https://issues.apache.org/jira/browse/HDFS-7075) | hadoop-fuse-dfs fails because it cannot find JavaKeyStoreProvider$Factory | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7073](https://issues.apache.org/jira/browse/HDFS-7073) | Allow falling back to a non-SASL connection on DataTransferProtocol in several edge cases. | Major | datanode, hdfs-client, security | Chris Nauroth | Chris Nauroth | +| [HDFS-7065](https://issues.apache.org/jira/browse/HDFS-7065) | Pipeline close recovery race can cause block corruption | Critical | datanode | Kihwal Lee | Kihwal Lee | +| [HDFS-7049](https://issues.apache.org/jira/browse/HDFS-7049) | TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2 | Minor | test | Juan Yu | Juan Yu | +| [HDFS-7046](https://issues.apache.org/jira/browse/HDFS-7046) | HA NN can NPE upon transition to active | Critical | namenode | Daryn Sharp | Kihwal Lee | +| [HDFS-7045](https://issues.apache.org/jira/browse/HDFS-7045) | Fix NameNode deadlock when opening file under /.reserved path | Critical | namenode | Yi Liu | Yi Liu | +| [HDFS-7042](https://issues.apache.org/jira/browse/HDFS-7042) | Upgrade fails for Windows HA cluster due to file locks held during rename in JournalNode. | Blocker | journal-node | Chris Nauroth | Chris Nauroth | +| [HDFS-7005](https://issues.apache.org/jira/browse/HDFS-7005) | DFS input streams do not timeout | Critical | hdfs-client | Daryn Sharp | Daryn Sharp | +| [HDFS-7001](https://issues.apache.org/jira/browse/HDFS-7001) | Tests in TestTracing should not depend on the order of execution | Minor | . | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-6996](https://issues.apache.org/jira/browse/HDFS-6996) | SnapshotDiff report can hit IndexOutOfBoundsException when there are nested renamed directory/file | Major | snapshots | Jing Zhao | Jing Zhao | +| [HDFS-6995](https://issues.apache.org/jira/browse/HDFS-6995) | Block should be placed in the client's 'rack-local' node if 'client-local' node is not available | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-6981](https://issues.apache.org/jira/browse/HDFS-6981) | Fix DN upgrade with layout version change | Major | datanode | James Thomas | Arpit Agarwal | +| [HDFS-6979](https://issues.apache.org/jira/browse/HDFS-6979) | hdfs.dll does not produce .pdb files | Minor | hdfs-client | Remus Rusanu | Chris Nauroth | +| [HDFS-6972](https://issues.apache.org/jira/browse/HDFS-6972) | TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6965](https://issues.apache.org/jira/browse/HDFS-6965) | NN continues to issue block locations for DNs with full disks | Major | namenode | Daryn Sharp | Rushabh S Shah | +| [HDFS-6954](https://issues.apache.org/jira/browse/HDFS-6954) | With crypto, no native lib systems are too verbose | Major | encryption | Allen Wittenauer | Charles Lamb | +| [HDFS-6948](https://issues.apache.org/jira/browse/HDFS-6948) | DN rejects blocks if it has older UC block | Major | . | Daryn Sharp | Eric Payne | +| [HDFS-6942](https://issues.apache.org/jira/browse/HDFS-6942) | Fix typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | +| [HDFS-6912](https://issues.apache.org/jira/browse/HDFS-6912) | SharedFileDescriptorFactory should not allocate sparse files | Minor | caching | Gopal V | Colin Patrick McCabe | +| [HDFS-6908](https://issues.apache.org/jira/browse/HDFS-6908) | incorrect snapshot directory diff generated by snapshot deletion | Critical | snapshots | Juan Yu | Juan Yu | +| [HDFS-6905](https://issues.apache.org/jira/browse/HDFS-6905) | fs-encryption merge triggered release audit failures | Blocker | . | Allen Wittenauer | Charles Lamb | +| [HDFS-6904](https://issues.apache.org/jira/browse/HDFS-6904) | YARN unable to renew delegation token fetched via webhdfs due to incorrect service port | Critical | webhdfs | Varun Vasudev | Jitendra Nath Pandey | +| [HDFS-6902](https://issues.apache.org/jira/browse/HDFS-6902) | FileWriter should be closed in finally block in BlockReceiver#receiveBlock() | Minor | . | Ted Yu | Tsuyoshi Ozawa | +| [HDFS-6898](https://issues.apache.org/jira/browse/HDFS-6898) | DN must reserve space for a full block when an RBW block is created | Major | datanode | Gopal V | Arpit Agarwal | +| [HDFS-6890](https://issues.apache.org/jira/browse/HDFS-6890) | NFS readdirplus doesn't return dotdot attributes | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-6870](https://issues.apache.org/jira/browse/HDFS-6870) | Blocks and INodes could leak for Rename with overwrite flag | Major | namenode | Yi Liu | Yi Liu | +| [HDFS-6868](https://issues.apache.org/jira/browse/HDFS-6868) | portmap and nfs3 are documented as hadoop commands instead of hdfs | Major | documentation, nfs | Allen Wittenauer | Brandon Li | +| [HDFS-6862](https://issues.apache.org/jira/browse/HDFS-6862) | Add missing timeout annotations to tests | Major | test | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-6848](https://issues.apache.org/jira/browse/HDFS-6848) | Lack of synchronization on access to datanodeUuid in DataStorage#format() | Minor | . | Ted Yu | Xiaoyu Yao | +| [HDFS-6840](https://issues.apache.org/jira/browse/HDFS-6840) | Clients are always sent to the same datanode when read is off rack | Critical | . | Jason Lowe | Andrew Wang | +| [HDFS-6838](https://issues.apache.org/jira/browse/HDFS-6838) | Code cleanup for unnecessary INode replacement | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-6831](https://issues.apache.org/jira/browse/HDFS-6831) | Inconsistency between 'hdfs dfsadmin' and 'hdfs dfsadmin -help' | Minor | . | Akira AJISAKA | Xiaoyu Yao | +| [HDFS-6830](https://issues.apache.org/jira/browse/HDFS-6830) | BlockInfo.addStorage fails when DN changes the storage for a block replica | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6829](https://issues.apache.org/jira/browse/HDFS-6829) | DFSAdmin refreshSuperUserGroupsConfiguration failed in security cluster | Minor | tools | zhaoyunjiong | zhaoyunjiong | +| [HDFS-6825](https://issues.apache.org/jira/browse/HDFS-6825) | Edit log corruption due to delayed block removal | Major | namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6823](https://issues.apache.org/jira/browse/HDFS-6823) | dfs.web.authentication.kerberos.principal shows up in logs for insecure HDFS | Minor | namenode | Allen Wittenauer | Allen Wittenauer | +| [HDFS-6810](https://issues.apache.org/jira/browse/HDFS-6810) | StorageReport array is initialized with wrong size in DatanodeDescriptor#getStorageReports | Minor | namenode | Ted Yu | Tsz Wo Nicholas Sze | +| [HDFS-6802](https://issues.apache.org/jira/browse/HDFS-6802) | Some tests in TestDFSClientFailover are missing @Test annotation | Major | test | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6799](https://issues.apache.org/jira/browse/HDFS-6799) | The invalidate method in SimulatedFSDataset.java failed to remove (invalidate) blocks from the file system. | Minor | datanode, test | Megasthenis Asteris | Megasthenis Asteris | +| [HDFS-6797](https://issues.apache.org/jira/browse/HDFS-6797) | DataNode logs wrong layoutversion during upgrade | Major | datanode | Benoy Antony | Benoy Antony | +| [HDFS-6791](https://issues.apache.org/jira/browse/HDFS-6791) | A block could remain under replicated if all of its replicas are on decommissioned nodes | Major | . | Ming Ma | Ming Ma | +| [HDFS-6790](https://issues.apache.org/jira/browse/HDFS-6790) | DFSUtil Should Use configuration.getPassword for SSL passwords | Major | . | Larry McCay | Larry McCay | +| [HDFS-6789](https://issues.apache.org/jira/browse/HDFS-6789) | TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7 | Major | test | Rushabh S Shah | Akira AJISAKA | +| [HDFS-6788](https://issues.apache.org/jira/browse/HDFS-6788) | Improve synchronization in BPOfferService with read write lock | Major | datanode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6787](https://issues.apache.org/jira/browse/HDFS-6787) | Remove duplicate code in FSDirectory#unprotectedConcat | Major | namenode | Yi Liu | Yi Liu | +| [HDFS-6783](https://issues.apache.org/jira/browse/HDFS-6783) | Fix HDFS CacheReplicationMonitor rescan logic | Major | caching | Yi Liu | Yi Liu | +| [HDFS-6778](https://issues.apache.org/jira/browse/HDFS-6778) | The extended attributes javadoc should simply refer to the user docs | Major | . | Charles Lamb | Charles Lamb | +| [HDFS-6776](https://issues.apache.org/jira/browse/HDFS-6776) | Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6768](https://issues.apache.org/jira/browse/HDFS-6768) | Fix a few unit tests that use hard-coded port numbers | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6754](https://issues.apache.org/jira/browse/HDFS-6754) | TestNamenodeCapacityReport.testXceiverCount may sometimes fail due to lack of retry | Major | . | Mit Desai | Mit Desai | +| [HDFS-6749](https://issues.apache.org/jira/browse/HDFS-6749) | FSNamesystem methods should call resolvePath | Major | namenode | Charles Lamb | Charles Lamb | +| [HDFS-6731](https://issues.apache.org/jira/browse/HDFS-6731) | Run "hdfs zkfc-formatZK" on a server in a non-namenode will cause a null pointer exception. | Major | auto-failover, ha | WenJin Ma | Masatake Iwasaki | +| [HDFS-6715](https://issues.apache.org/jira/browse/HDFS-6715) | webhdfs wont fail over when it gets java.io.IOException: Namenode is in startup mode | Major | ha, webhdfs | Arpit Gupta | Jing Zhao | +| [HDFS-6714](https://issues.apache.org/jira/browse/HDFS-6714) | TestBlocksScheduledCounter#testBlocksScheduledCounter should shutdown cluster | Minor | test | Vinayakumar B | Vinayakumar B | +| [HDFS-6704](https://issues.apache.org/jira/browse/HDFS-6704) | Fix the command to launch JournalNode in HDFS-HA document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6702](https://issues.apache.org/jira/browse/HDFS-6702) | DFSClient should create blocks using StorageType | Major | datanode, hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-6693](https://issues.apache.org/jira/browse/HDFS-6693) | TestDFSAdminWithHA fails on windows | Major | test, tools | Vinayakumar B | Vinayakumar B | +| [HDFS-6689](https://issues.apache.org/jira/browse/HDFS-6689) | NFS doesn't return correct lookup access for directories | Major | nfs | Yesha Vora | Brandon Li | +| [HDFS-6678](https://issues.apache.org/jira/browse/HDFS-6678) | MiniDFSCluster may still be partially running after initialization fails. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-6667](https://issues.apache.org/jira/browse/HDFS-6667) | In HDFS HA mode, Distcp/SLive with webhdfs on secure cluster fails with Client cannot authenticate via:[TOKEN, KERBEROS] error | Major | security | Jian He | Jing Zhao | +| [HDFS-6664](https://issues.apache.org/jira/browse/HDFS-6664) | HDFS permissions guide documentation states incorrect default group mapping class. | Trivial | documentation | Chris Nauroth | Ray Chiang | +| [HDFS-6646](https://issues.apache.org/jira/browse/HDFS-6646) | [ HDFS Rolling Upgrade - Shell ] shutdownDatanode and getDatanodeInfo usage is missed | Major | tools | Brahma Reddy Battula | Brahma Reddy Battula | +| [HDFS-6640](https://issues.apache.org/jira/browse/HDFS-6640) | [ Web HDFS ] Syntax for MKDIRS, CREATESYMLINK, and SETXATTR are given wrongly(missed webhdfs/v1).). | Major | documentation, webhdfs | Brahma Reddy Battula | Stephen Chu | +| [HDFS-6630](https://issues.apache.org/jira/browse/HDFS-6630) | Unable to fetch the block information by Browsing the file system on Namenode UI through IE9 | Major | namenode | J.Andreina | Haohui Mai | +| [HDFS-6621](https://issues.apache.org/jira/browse/HDFS-6621) | Hadoop Balancer prematurely exits iterations | Major | balancer & mover | Benjamin Bowman | Rafal Wojdyla | +| [HDFS-6616](https://issues.apache.org/jira/browse/HDFS-6616) | bestNode shouldn't always return the first DataNode | Minor | webhdfs | zhaoyunjiong | zhaoyunjiong | +| [HDFS-6582](https://issues.apache.org/jira/browse/HDFS-6582) | Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler) | Minor | nfs | Ted Yu | Abhiraj Butala | +| [HDFS-6569](https://issues.apache.org/jira/browse/HDFS-6569) | OOB message can't be sent to the client when DataNode shuts down for upgrade | Major | datanode | Brandon Li | Brandon Li | +| [HDFS-6567](https://issues.apache.org/jira/browse/HDFS-6567) | Normalize the order of public final in HdfsFileStatus | Major | . | Haohui Mai | Tassapol Athiapinya | +| [HDFS-6544](https://issues.apache.org/jira/browse/HDFS-6544) | Broken Link for GFS in package.html | Minor | . | Suraj Nayak M | Suraj Nayak M | +| [HDFS-6534](https://issues.apache.org/jira/browse/HDFS-6534) | Fix build on macosx: HDFS parts | Minor | . | Binglin Chang | Binglin Chang | +| [HDFS-6517](https://issues.apache.org/jira/browse/HDFS-6517) | Remove hadoop-metrics2.properties from hdfs project | Major | . | Akira AJISAKA | Akira AJISAKA | +| [HDFS-6506](https://issues.apache.org/jira/browse/HDFS-6506) | Newly moved block replica been invalidated and deleted in TestBalancer | Major | balancer & mover, test | Binglin Chang | Binglin Chang | +| [HDFS-6478](https://issues.apache.org/jira/browse/HDFS-6478) | RemoteException can't be retried properly for non-HA scenario | Major | . | Ming Ma | Ming Ma | +| [HDFS-6456](https://issues.apache.org/jira/browse/HDFS-6456) | NFS should throw error for invalid entry in dfs.nfs.exports.allowed.hosts | Major | nfs | Yesha Vora | Abhiraj Butala | +| [HDFS-6455](https://issues.apache.org/jira/browse/HDFS-6455) | NFS: Exception should be added in NFS log for invalid separator in nfs.exports.allowed.hosts | Major | nfs | Yesha Vora | Abhiraj Butala | +| [HDFS-6451](https://issues.apache.org/jira/browse/HDFS-6451) | NFS should not return NFS3ERR\_IO for AccessControlException | Major | nfs | Brandon Li | Abhiraj Butala | +| [HDFS-6376](https://issues.apache.org/jira/browse/HDFS-6376) | Distcp data between two HA clusters requires another configuration | Major | datanode, federation, hdfs-client | Dave Marion | Dave Marion | +| [HDFS-6247](https://issues.apache.org/jira/browse/HDFS-6247) | Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer | Major | balancer & mover, datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-6114](https://issues.apache.org/jira/browse/HDFS-6114) | Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp\_block\_verification.log.curr | Critical | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-5919](https://issues.apache.org/jira/browse/HDFS-5919) | FileJournalManager doesn't purge empty and corrupt inprogress edits files | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5809](https://issues.apache.org/jira/browse/HDFS-5809) | BlockPoolSliceScanner and high speed hdfs appending make datanode to drop into infinite loop | Critical | datanode | ikweesung | Colin Patrick McCabe | +| [HDFS-5723](https://issues.apache.org/jira/browse/HDFS-5723) | Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-5185](https://issues.apache.org/jira/browse/HDFS-5185) | DN fails to startup if one of the data dir is full | Critical | datanode | Vinayakumar B | Vinayakumar B | +| [HDFS-5089](https://issues.apache.org/jira/browse/HDFS-5089) | When a LayoutVersion support SNAPSHOT, it must support FSIMAGE\_NAME\_OPTIMIZATION. | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-4852](https://issues.apache.org/jira/browse/HDFS-4852) | libhdfs documentation is out of date | Minor | . | Andrew Wang | Chris Nauroth | +| [HDFS-4629](https://issues.apache.org/jira/browse/HDFS-4629) | Using com.sun.org.apache.xml.internal.serialize.\* in XmlEditsVisitor.java is JVM vendor specific. Breaks IBM JAVA | Major | tools | Amir Sanjar | | +| [HDFS-4227](https://issues.apache.org/jira/browse/HDFS-4227) | Document dfs.namenode.resource.\* | Major | documentation | Eli Collins | Daisuke Kobayashi | +| [HDFS-4165](https://issues.apache.org/jira/browse/HDFS-4165) | Faulty sanity check in FsDirectory.unprotectedSetQuota | Trivial | namenode | Binglin Chang | Binglin Chang | +| [HDFS-3482](https://issues.apache.org/jira/browse/HDFS-3482) | hdfs balancer throws ArrayIndexOutOfBoundsException if option is specified without arguments | Minor | balancer & mover | Stephen Chu | madhukara phatak | +| [HDFS-2976](https://issues.apache.org/jira/browse/HDFS-2976) | Remove unnecessary method (tokenRefetchNeeded) in DFSClient | Trivial | hdfs-client | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HDFS-2975](https://issues.apache.org/jira/browse/HDFS-2975) | Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart). | Major | namenode | Uma Maheswara Rao G | Yi Liu | +| [MAPREDUCE-6156](https://issues.apache.org/jira/browse/MAPREDUCE-6156) | Fetcher - connect() doesn't handle connection refused correctly | Blocker | . | Sidharta Seethana | Junping Du | +| [MAPREDUCE-6126](https://issues.apache.org/jira/browse/MAPREDUCE-6126) | (Rumen) Rumen tool returns error "ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type" | Major | . | Junping Du | Junping Du | +| [MAPREDUCE-6125](https://issues.apache.org/jira/browse/MAPREDUCE-6125) | TestContainerLauncherImpl sometimes fails | Major | test | Mit Desai | Mit Desai | +| [MAPREDUCE-6123](https://issues.apache.org/jira/browse/MAPREDUCE-6123) | TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-6122](https://issues.apache.org/jira/browse/MAPREDUCE-6122) | TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [MAPREDUCE-6109](https://issues.apache.org/jira/browse/MAPREDUCE-6109) | Fix minor typo in distcp -p usage text | Trivial | distcp | Charles Lamb | Charles Lamb | +| [MAPREDUCE-6104](https://issues.apache.org/jira/browse/MAPREDUCE-6104) | TestJobHistoryParsing.testPartialJob fails in branch-2 | Major | . | Mit Desai | Mit Desai | +| [MAPREDUCE-6095](https://issues.apache.org/jira/browse/MAPREDUCE-6095) | Enable DistributedCache for uber-mode Jobs | Major | applicationmaster, distributed-cache | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-6094](https://issues.apache.org/jira/browse/MAPREDUCE-6094) | TestMRCJCFileInputFormat.testAddInputPath() fails on trunk | Minor | test | Sangjin Lee | Akira AJISAKA | +| [MAPREDUCE-6093](https://issues.apache.org/jira/browse/MAPREDUCE-6093) | minor distcp doc edits | Trivial | distcp, documentation | Charles Lamb | Charles Lamb | +| [MAPREDUCE-6091](https://issues.apache.org/jira/browse/MAPREDUCE-6091) | YARNRunner.getJobStatus() fails with ApplicationNotFoundException if the job rolled off the RM view | Major | client | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-6090](https://issues.apache.org/jira/browse/MAPREDUCE-6090) | mapred hsadmin getGroups fails to connect in some cases | Major | client | Robert Kanter | Robert Kanter | +| [MAPREDUCE-6087](https://issues.apache.org/jira/browse/MAPREDUCE-6087) | MRJobConfig#MR\_CLIENT\_TO\_AM\_IPC\_MAX\_RETRIES\_ON\_TIMEOUTS config name is wrong | Major | . | Jian He | Akira AJISAKA | +| [MAPREDUCE-6075](https://issues.apache.org/jira/browse/MAPREDUCE-6075) | HistoryServerFileSystemStateStore can create zero-length files | Major | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6073](https://issues.apache.org/jira/browse/MAPREDUCE-6073) | Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags | Trivial | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [MAPREDUCE-6063](https://issues.apache.org/jira/browse/MAPREDUCE-6063) | In sortAndSpill of MapTask.java, size is calculated wrongly when bufend \< bufstart. | Major | mrv1, mrv2 | zhihai xu | zhihai xu | +| [MAPREDUCE-6052](https://issues.apache.org/jira/browse/MAPREDUCE-6052) | Support overriding log4j.properties per job | Major | . | Junping Du | Junping Du | +| [MAPREDUCE-6051](https://issues.apache.org/jira/browse/MAPREDUCE-6051) | Fix typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | +| [MAPREDUCE-6044](https://issues.apache.org/jira/browse/MAPREDUCE-6044) | Fully qualified intermediate done directory will break per-user dir creation on Windows | Major | jobhistoryserver | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-6041](https://issues.apache.org/jira/browse/MAPREDUCE-6041) | Fix TestOptionsParser | Major | security | Charles Lamb | Charles Lamb | +| [MAPREDUCE-6036](https://issues.apache.org/jira/browse/MAPREDUCE-6036) | TestJobEndNotifier fails intermittently in branch-2 | Major | . | Mit Desai | Chang Li | +| [MAPREDUCE-6032](https://issues.apache.org/jira/browse/MAPREDUCE-6032) | Unable to check mapreduce job status if submitted using a non-default namenode | Major | jobhistoryserver | Benjamin Zhitomirsky | Benjamin Zhitomirsky | +| [MAPREDUCE-6029](https://issues.apache.org/jira/browse/MAPREDUCE-6029) | TestCommitterEventHandler fails in trunk | Major | . | Ted Yu | Mit Desai | +| [MAPREDUCE-6022](https://issues.apache.org/jira/browse/MAPREDUCE-6022) | map\_input\_file is missing from streaming job environment | Major | . | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6021](https://issues.apache.org/jira/browse/MAPREDUCE-6021) | MR AM should have working directory in LD\_LIBRARY\_PATH | Major | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6019](https://issues.apache.org/jira/browse/MAPREDUCE-6019) | MapReduce changes for exposing YARN/MR endpoints on multiple interfaces. | Major | . | Xuan Gong | Craig Welch | +| [MAPREDUCE-6014](https://issues.apache.org/jira/browse/MAPREDUCE-6014) | New task status field in task attempts table can lead to an empty web page | Major | . | Mit Desai | Mit Desai | +| [MAPREDUCE-6012](https://issues.apache.org/jira/browse/MAPREDUCE-6012) | DBInputSplit creates invalid ranges on Oracle | Major | . | Julien Serdaru | Wei Yan | +| [MAPREDUCE-6010](https://issues.apache.org/jira/browse/MAPREDUCE-6010) | HistoryServerFileSystemStateStore fails to update tokens | Major | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5999](https://issues.apache.org/jira/browse/MAPREDUCE-5999) | Fix dead link in InputFormat javadoc | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5998](https://issues.apache.org/jira/browse/MAPREDUCE-5998) | CompositeInputFormat javadoc is broken | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5960](https://issues.apache.org/jira/browse/MAPREDUCE-5960) | JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path. | Major | client | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5958](https://issues.apache.org/jira/browse/MAPREDUCE-5958) | Wrong reduce task progress if map output is compressed | Minor | . | Emilio Coppa | Emilio Coppa | +| [MAPREDUCE-5957](https://issues.apache.org/jira/browse/MAPREDUCE-5957) | AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used | Major | . | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-5950](https://issues.apache.org/jira/browse/MAPREDUCE-5950) | incorrect description in distcp2 document | Major | documentation | Yongjun Zhang | Akira AJISAKA | +| [MAPREDUCE-5931](https://issues.apache.org/jira/browse/MAPREDUCE-5931) | Validate SleepJob command line parameters | Minor | test | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5885](https://issues.apache.org/jira/browse/MAPREDUCE-5885) | build/test/test.mapred.spill causes release audit warnings | Major | test | Jason Lowe | Chen He | +| [MAPREDUCE-5878](https://issues.apache.org/jira/browse/MAPREDUCE-5878) | some standard JDK APIs are not part of system classes defaults | Major | mrv2 | Sangjin Lee | Sangjin Lee | +| [MAPREDUCE-5873](https://issues.apache.org/jira/browse/MAPREDUCE-5873) | Shuffle bandwidth computation includes time spent waiting for maps | Major | . | Siqi Li | Siqi Li | +| [MAPREDUCE-5831](https://issues.apache.org/jira/browse/MAPREDUCE-5831) | Old MR client is not compatible with new MR application | Blocker | client, mr-am | Zhijie Shen | Junping Du | +| [MAPREDUCE-5796](https://issues.apache.org/jira/browse/MAPREDUCE-5796) | Use current version of the archive name in DistributedCacheDeploy document | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5756](https://issues.apache.org/jira/browse/MAPREDUCE-5756) | CombineFileInputFormat.getSplits() including directories in its results | Major | . | Jason Dere | Jason Dere | +| [MAPREDUCE-5597](https://issues.apache.org/jira/browse/MAPREDUCE-5597) | Missing alternatives in javadocs for deprecated constructors in mapreduce.Job | Minor | client, documentation, job submission | Christopher Tubbs | Akira AJISAKA | +| [MAPREDUCE-5595](https://issues.apache.org/jira/browse/MAPREDUCE-5595) | Typo in MergeManagerImpl.java | Trivial | . | Efe Gencer | Akira AJISAKA | +| [MAPREDUCE-5542](https://issues.apache.org/jira/browse/MAPREDUCE-5542) | Killing a job just as it finishes can generate an NPE in client | Major | client, mrv2 | Jason Lowe | Rohith Sharma K S | +| [MAPREDUCE-5363](https://issues.apache.org/jira/browse/MAPREDUCE-5363) | Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus | Minor | mrv1, mrv2 | Sandy Ryza | Akira AJISAKA | +| [MAPREDUCE-5279](https://issues.apache.org/jira/browse/MAPREDUCE-5279) | Jobs can deadlock if headroom is limited by cpu instead of memory | Critical | mrv2, scheduler | Peng Zhang | Peng Zhang | +| [YARN-2853](https://issues.apache.org/jira/browse/YARN-2853) | Killing app may hang while AM is unregistering | Major | . | Jian He | Jian He | +| [YARN-2846](https://issues.apache.org/jira/browse/YARN-2846) | Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager restart. | Blocker | nodemanager | Junping Du | Junping Du | +| [YARN-2834](https://issues.apache.org/jira/browse/YARN-2834) | Resource manager crashed with Null Pointer Exception | Blocker | . | Yesha Vora | Jian He | +| [YARN-2830](https://issues.apache.org/jira/browse/YARN-2830) | Add backwords compatible ContainerId.newInstance constructor for use within Tez Local Mode | Blocker | . | Jonathan Eagles | Jonathan Eagles | +| [YARN-2827](https://issues.apache.org/jira/browse/YARN-2827) | Fix bugs of yarn queue CLI | Critical | client | Wangda Tan | Wangda Tan | +| [YARN-2826](https://issues.apache.org/jira/browse/YARN-2826) | User-Group mappings not updated by RM when a user is removed from a group. | Critical | . | Sidharta Seethana | Wangda Tan | +| [YARN-2825](https://issues.apache.org/jira/browse/YARN-2825) | Container leak on NM | Critical | . | Jian He | Jian He | +| [YARN-2823](https://issues.apache.org/jira/browse/YARN-2823) | NullPointerException in RM HA enabled 3-node cluster | Critical | resourcemanager | Gour Saha | Jian He | +| [YARN-2819](https://issues.apache.org/jira/browse/YARN-2819) | NPE in ATS Timeline Domains when upgrading from 2.4 to 2.6 | Critical | timelineserver | Gopal V | Zhijie Shen | +| [YARN-2818](https://issues.apache.org/jira/browse/YARN-2818) | Remove the logic to inject entity owner as the primary filter | Critical | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2813](https://issues.apache.org/jira/browse/YARN-2813) | NPE from MemoryTimelineStore.getDomains | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2805](https://issues.apache.org/jira/browse/YARN-2805) | RM2 in HA setup tries to login using the RM1's kerberos principal | Blocker | resourcemanager | Arpit Gupta | Wangda Tan | +| [YARN-2804](https://issues.apache.org/jira/browse/YARN-2804) | Timeline server .out log have JAXB binding exceptions and warnings. | Critical | . | Zhijie Shen | Zhijie Shen | +| [YARN-2803](https://issues.apache.org/jira/browse/YARN-2803) | MR distributed cache not working correctly on Windows after NodeManager privileged account changes. | Critical | nodemanager | Chris Nauroth | Craig Welch | +| [YARN-2798](https://issues.apache.org/jira/browse/YARN-2798) | YarnClient doesn't need to translate Kerberos name of timeline DT renewer | Blocker | timelineserver | Arpit Gupta | Zhijie Shen | +| [YARN-2794](https://issues.apache.org/jira/browse/YARN-2794) | Fix log msgs about distributing system-credentials | Major | . | Jian He | Jian He | +| [YARN-2790](https://issues.apache.org/jira/browse/YARN-2790) | NM can't aggregate logs past HDFS delegation token expiry. | Critical | nodemanager | Tassapol Athiapinya | Jian He | +| [YARN-2788](https://issues.apache.org/jira/browse/YARN-2788) | yarn logs -applicationId on 2.6.0 should support logs written by 2.4.0 | Blocker | log-aggregation | Gopal V | Xuan Gong | +| [YARN-2779](https://issues.apache.org/jira/browse/YARN-2779) | SystemMetricsPublisher can use Kerberos directly instead of timeline DT | Critical | resourcemanager, timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2769](https://issues.apache.org/jira/browse/YARN-2769) | Timeline server domain not set correctly when using shell\_command on Windows | Major | applications/distributed-shell | Varun Vasudev | Varun Vasudev | +| [YARN-2760](https://issues.apache.org/jira/browse/YARN-2760) | Completely remove word 'experimental' from FairScheduler docs | Trivial | documentation | Harsh J | Harsh J | +| [YARN-2755](https://issues.apache.org/jira/browse/YARN-2755) | NM fails to clean up usercache\_DEL\_\ dirs after YARN-661 | Critical | . | Siqi Li | Siqi Li | +| [YARN-2752](https://issues.apache.org/jira/browse/YARN-2752) | ContainerExecutor always append "nice -n" in command on branch-2 | Critical | . | Xuan Gong | Xuan Gong | +| [YARN-2743](https://issues.apache.org/jira/browse/YARN-2743) | Yarn jobs via oozie fail with failed to renew token (secure) or digest mismatch (unsecure) errors when RM is being killed | Blocker | resourcemanager | Arpit Gupta | Jian He | +| [YARN-2741](https://issues.apache.org/jira/browse/YARN-2741) | Windows: Node manager cannot serve up log files via the web user interface when yarn.nodemanager.log-dirs to any drive letter other than C: (or, the drive that nodemanager is running on) | Major | nodemanager | Craig Welch | Craig Welch | +| [YARN-2734](https://issues.apache.org/jira/browse/YARN-2734) | If a sub-folder is encountered by log aggregator it results in invalid aggregated file | Major | log-aggregation | Sumit Mohanty | Xuan Gong | +| [YARN-2732](https://issues.apache.org/jira/browse/YARN-2732) | Fix syntax error in SecureContainer.apt.vm | Major | . | Jian He | Jian He | +| [YARN-2730](https://issues.apache.org/jira/browse/YARN-2730) | DefaultContainerExecutor runs only one localizer at a time | Critical | . | Siqi Li | Siqi Li | +| [YARN-2724](https://issues.apache.org/jira/browse/YARN-2724) | If an unreadable file is encountered during log aggregation then aggregated file in HDFS badly formed | Major | log-aggregation | Sumit Mohanty | Xuan Gong | +| [YARN-2721](https://issues.apache.org/jira/browse/YARN-2721) | Race condition: ZKRMStateStore retry logic may throw NodeExist exception | Blocker | . | Jian He | Jian He | +| [YARN-2720](https://issues.apache.org/jira/browse/YARN-2720) | Windows: Wildcard classpath variables not expanded against resources contained in archives | Major | nodemanager | Craig Welch | Craig Welch | +| [YARN-2715](https://issues.apache.org/jira/browse/YARN-2715) | Proxy user is problem for RPC interface if yarn.resourcemanager.webapp.proxyuser is not set. | Blocker | resourcemanager | Zhijie Shen | Zhijie Shen | +| [YARN-2707](https://issues.apache.org/jira/browse/YARN-2707) | Potential null dereference in FSDownload | Minor | . | Ted Yu | Gera Shegalov | +| [YARN-2701](https://issues.apache.org/jira/browse/YARN-2701) | Potential race condition in startLocalizer when using LinuxContainerExecutor | Blocker | . | Xuan Gong | Xuan Gong | +| [YARN-2682](https://issues.apache.org/jira/browse/YARN-2682) | WindowsSecureContainerExecutor should not depend on DefaultContainerExecutor#getFirstApplicationDir. | Minor | nodemanager | zhihai xu | zhihai xu | +| [YARN-2671](https://issues.apache.org/jira/browse/YARN-2671) | ApplicationSubmissionContext change breaks the existing app submission | Blocker | resourcemanager | Zhijie Shen | Wangda Tan | +| [YARN-2667](https://issues.apache.org/jira/browse/YARN-2667) | Fix the release audit warning caused by hadoop-yarn-registry | Minor | . | Yi Liu | Yi Liu | +| [YARN-2662](https://issues.apache.org/jira/browse/YARN-2662) | TestCgroupsLCEResourcesHandler leaks file descriptors. | Minor | test | Chris Nauroth | Chris Nauroth | +| [YARN-2656](https://issues.apache.org/jira/browse/YARN-2656) | RM web services authentication filter should add support for proxy user | Major | resourcemanager | Varun Vasudev | Zhijie Shen | +| [YARN-2649](https://issues.apache.org/jira/browse/YARN-2649) | Flaky test TestAMRMRPCNodeUpdates | Major | . | Ming Ma | Ming Ma | +| [YARN-2635](https://issues.apache.org/jira/browse/YARN-2635) | TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS | Major | . | Wei Yan | Wei Yan | +| [YARN-2630](https://issues.apache.org/jira/browse/YARN-2630) | TestDistributedShell#testDSRestartWithPreviousRunningContainers fails | Major | . | Jian He | Jian He | +| [YARN-2628](https://issues.apache.org/jira/browse/YARN-2628) | Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free | Major | capacityscheduler | Varun Vasudev | Varun Vasudev | +| [YARN-2624](https://issues.apache.org/jira/browse/YARN-2624) | Resource Localization fails on a cluster due to existing cache directories | Blocker | nodemanager | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2617](https://issues.apache.org/jira/browse/YARN-2617) | NM does not need to send finished container whose APP is not running to RM | Major | nodemanager | Jun Gong | Jun Gong | +| [YARN-2610](https://issues.apache.org/jira/browse/YARN-2610) | Hamlet should close table tags | Major | . | Ray Chiang | Ray Chiang | +| [YARN-2608](https://issues.apache.org/jira/browse/YARN-2608) | FairScheduler: Potential deadlocks in loading alloc files and clock access | Major | . | Wei Yan | Wei Yan | +| [YARN-2606](https://issues.apache.org/jira/browse/YARN-2606) | Application History Server tries to access hdfs before doing secure login | Major | timelineserver | Mit Desai | Mit Desai | +| [YARN-2602](https://issues.apache.org/jira/browse/YARN-2602) | Generic History Service of TimelineServer sometimes not able to handle NPE | Major | timelineserver | Karam Singh | Zhijie Shen | +| [YARN-2594](https://issues.apache.org/jira/browse/YARN-2594) | Potential deadlock in RM when querying ApplicationResourceUsageReport | Blocker | resourcemanager | Karam Singh | Wangda Tan | +| [YARN-2588](https://issues.apache.org/jira/browse/YARN-2588) | Standby RM does not transitionToActive if previous transitionToActive is failed with ZK exception. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2579](https://issues.apache.org/jira/browse/YARN-2579) | Deadlock when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2568](https://issues.apache.org/jira/browse/YARN-2568) | TestAMRMClientOnRMRestart test fails | Major | . | Jian He | Jian He | +| [YARN-2565](https://issues.apache.org/jira/browse/YARN-2565) | RM shouldn't use the old RMApplicationHistoryWriter unless explicitly setting FileSystemApplicationHistoryStore | Major | resourcemanager, timelineserver | Karam Singh | Zhijie Shen | +| [YARN-2563](https://issues.apache.org/jira/browse/YARN-2563) | On secure clusters call to timeline server fails with authentication errors when running a job via oozie | Blocker | timelineserver | Arpit Gupta | Zhijie Shen | +| [YARN-2562](https://issues.apache.org/jira/browse/YARN-2562) | ContainerId@toString() is unreadable for epoch \>0 after YARN-2182 | Critical | . | Vinod Kumar Vavilapalli | Tsuyoshi Ozawa | +| [YARN-2559](https://issues.apache.org/jira/browse/YARN-2559) | ResourceManager sometime become un-responsive due to NPE in SystemMetricsPublisher | Major | resourcemanager, timelineserver | Karam Singh | Zhijie Shen | +| [YARN-2557](https://issues.apache.org/jira/browse/YARN-2557) | Add a parameter "attempt\_Failures\_Validity\_Interval" in DistributedShell | Major | applications/distributed-shell | Xuan Gong | Xuan Gong | +| [YARN-2546](https://issues.apache.org/jira/browse/YARN-2546) | REST API for application creation/submission is using strings for numeric & boolean values | Major | api | Doug Haigh | Varun Vasudev | +| [YARN-2541](https://issues.apache.org/jira/browse/YARN-2541) | Fix ResourceManagerRest.apt.vm syntax error | Major | . | Jian He | Jian He | +| [YARN-2540](https://issues.apache.org/jira/browse/YARN-2540) | FairScheduler: Queue filters not working on scheduler page in RM UI | Major | scheduler | Ashwin Shankar | Ashwin Shankar | +| [YARN-2534](https://issues.apache.org/jira/browse/YARN-2534) | FairScheduler: Potential integer overflow calculating totalMaxShare | Major | scheduler | zhihai xu | zhihai xu | +| [YARN-2527](https://issues.apache.org/jira/browse/YARN-2527) | NPE in ApplicationACLsManager | Major | resourcemanager | Benoy Antony | Benoy Antony | +| [YARN-2526](https://issues.apache.org/jira/browse/YARN-2526) | SLS can deadlock when all the threads are taken by AMSimulators | Critical | scheduler-load-simulator | Wei Yan | Wei Yan | +| [YARN-2523](https://issues.apache.org/jira/browse/YARN-2523) | ResourceManager UI showing negative value for "Decommissioned Nodes" field | Major | resourcemanager, webapp | Nishan Shetty | Rohith Sharma K S | +| [YARN-2484](https://issues.apache.org/jira/browse/YARN-2484) | FileSystemRMStateStore#readFile/writeFile should close FSData(In\|Out)putStream in final block | Trivial | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2462](https://issues.apache.org/jira/browse/YARN-2462) | TestNodeManagerResync#testBlockNewContainerRequestsOnStartAndResync should have a test timeout | Major | . | Jason Lowe | Eric Payne | +| [YARN-2460](https://issues.apache.org/jira/browse/YARN-2460) | Remove obsolete entries from yarn-default.xml | Minor | . | Ray Chiang | Ray Chiang | +| [YARN-2459](https://issues.apache.org/jira/browse/YARN-2459) | RM crashes if App gets rejected for any reason and HA is enabled | Major | resourcemanager | Mayank Bansal | Mayank Bansal | +| [YARN-2453](https://issues.apache.org/jira/browse/YARN-2453) | TestProportionalCapacityPreemptionPolicy fails with FairScheduler | Major | . | zhihai xu | zhihai xu | +| [YARN-2452](https://issues.apache.org/jira/browse/YARN-2452) | TestRMApplicationHistoryWriter fails with FairScheduler | Major | . | zhihai xu | zhihai xu | +| [YARN-2450](https://issues.apache.org/jira/browse/YARN-2450) | Fix typos in log messages | Trivial | . | Ray Chiang | Ray Chiang | +| [YARN-2449](https://issues.apache.org/jira/browse/YARN-2449) | Timelineserver returns invalid Delegation token in secure kerberos enabled cluster when hadoop.http.filter.initializers are not set | Critical | timelineserver | Karam Singh | Varun Vasudev | +| [YARN-2447](https://issues.apache.org/jira/browse/YARN-2447) | RM web services app submission doesn't pass secrets correctly | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2440](https://issues.apache.org/jira/browse/YARN-2440) | Cgroups should allow YARN containers to be limited to allocated cores | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2426](https://issues.apache.org/jira/browse/YARN-2426) | ResourceManger is not able renew WebHDFS token when application submitted by Yarn WebService | Major | nodemanager, resourcemanager, webapp | Karam Singh | Varun Vasudev | +| [YARN-2424](https://issues.apache.org/jira/browse/YARN-2424) | LCE should support non-cgroups, non-secure mode | Blocker | nodemanager | Allen Wittenauer | Allen Wittenauer | +| [YARN-2409](https://issues.apache.org/jira/browse/YARN-2409) | Active to StandBy transition does not stop rmDispatcher that causes 1 AsyncDispatcher thread leak. | Critical | resourcemanager | Nishan Shetty | Rohith Sharma K S | +| [YARN-2405](https://issues.apache.org/jira/browse/YARN-2405) | NPE in FairSchedulerAppsBlock | Major | . | Maysam Yabandeh | Tsuyoshi Ozawa | +| [YARN-2400](https://issues.apache.org/jira/browse/YARN-2400) | TestAMRestart fails intermittently | Major | . | Jian He | Jian He | +| [YARN-2397](https://issues.apache.org/jira/browse/YARN-2397) | RM and TS web interfaces sometimes return request is a replay error in secure mode | Critical | . | Varun Vasudev | Varun Vasudev | +| [YARN-2387](https://issues.apache.org/jira/browse/YARN-2387) | Resource Manager crashes with NPE due to lack of synchronization | Blocker | . | Mit Desai | Mit Desai | +| [YARN-2374](https://issues.apache.org/jira/browse/YARN-2374) | YARN trunk build failing TestDistributedShell.testDSShell | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2373](https://issues.apache.org/jira/browse/YARN-2373) | WebAppUtils Should Use configuration.getPassword for Accessing SSL Passwords | Major | . | Larry McCay | Larry McCay | +| [YARN-2370](https://issues.apache.org/jira/browse/YARN-2370) | Fix comment in o.a.h.y.server.resourcemanager.schedulerAppSchedulingInfo | Trivial | resourcemanager | Wenwu Peng | Wenwu Peng | +| [YARN-2363](https://issues.apache.org/jira/browse/YARN-2363) | Submitted applications occasionally lack a tracking URL | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-2359](https://issues.apache.org/jira/browse/YARN-2359) | Application hangs when it fails to launch AM container | Critical | resourcemanager | zhihai xu | zhihai xu | +| [YARN-2321](https://issues.apache.org/jira/browse/YARN-2321) | NodeManager web UI can incorrectly report Pmem enforcement | Major | nodemanager | Leitao Guo | Leitao Guo | +| [YARN-2314](https://issues.apache.org/jira/browse/YARN-2314) | ContainerManagementProtocolProxy can create thousands of threads for a large cluster | Critical | client | Jason Lowe | Jason Lowe | +| [YARN-2313](https://issues.apache.org/jira/browse/YARN-2313) | Livelock can occur in FairScheduler when there are lots of running apps | Major | fairscheduler | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2308](https://issues.apache.org/jira/browse/YARN-2308) | NPE happened when RM restart after CapacityScheduler queue configuration changed | Critical | resourcemanager, scheduler | Wangda Tan | Chang Li | +| [YARN-2273](https://issues.apache.org/jira/browse/YARN-2273) | NPE in ContinuousScheduling thread when we lose a node | Major | fairscheduler, resourcemanager | Andy Skelton | Wei Yan | +| [YARN-2269](https://issues.apache.org/jira/browse/YARN-2269) | External links need to be removed from YARN UI | Major | . | Yesha Vora | Craig Welch | +| [YARN-2264](https://issues.apache.org/jira/browse/YARN-2264) | Race in DrainDispatcher can cause random test failures | Major | . | Siddharth Seth | Li Lu | +| [YARN-2252](https://issues.apache.org/jira/browse/YARN-2252) | Intermittent failure of TestFairScheduler.testContinuousScheduling | Major | scheduler | Ratandeep Ratti | | +| [YARN-2251](https://issues.apache.org/jira/browse/YARN-2251) | Avoid negative elapsed time in JHS/MRAM web UI and services | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2244](https://issues.apache.org/jira/browse/YARN-2244) | FairScheduler missing handling of containers for unknown application attempts | Critical | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2219](https://issues.apache.org/jira/browse/YARN-2219) | AMs and NMs can get exceptions after recovery but before scheduler knowns apps and app-attempts | Major | resourcemanager | Ashwin Shankar | Jian He | +| [YARN-2181](https://issues.apache.org/jira/browse/YARN-2181) | Add preemption info to RM Web UI and add logs when preemption occurs | Major | resourcemanager, webapp | Wangda Tan | Wangda Tan | +| [YARN-2161](https://issues.apache.org/jira/browse/YARN-2161) | Fix build on macosx: YARN parts | Major | . | Binglin Chang | Binglin Chang | +| [YARN-2147](https://issues.apache.org/jira/browse/YARN-2147) | client lacks delegation token exception details when application submit fails | Minor | resourcemanager | Jason Lowe | Chen He | +| [YARN-2138](https://issues.apache.org/jira/browse/YARN-2138) | Cleanup notifyDone\* methods in RMStateStore | Major | . | Jian He | Varun Saxena | +| [YARN-2088](https://issues.apache.org/jira/browse/YARN-2088) | Fix code bug in GetApplicationsRequestPBImpl#mergeLocalToBuilder | Major | . | Binglin Chang | Binglin Chang | +| [YARN-2034](https://issues.apache.org/jira/browse/YARN-2034) | Description for yarn.nodemanager.localizer.cache.target-size-mb is incorrect | Minor | nodemanager | Jason Lowe | Chen He | +| [YARN-2026](https://issues.apache.org/jira/browse/YARN-2026) | Fair scheduler: Consider only active queues for computing fairshare | Major | scheduler | Ashwin Shankar | Ashwin Shankar | +| [YARN-2010](https://issues.apache.org/jira/browse/YARN-2010) | Handle app-recovery failures gracefully | Blocker | resourcemanager | bc Wong | Karthik Kambatla | +| [YARN-1959](https://issues.apache.org/jira/browse/YARN-1959) | Fix headroom calculation in FairScheduler | Major | . | Sandy Ryza | Anubhav Dhoot | +| [YARN-1922](https://issues.apache.org/jira/browse/YARN-1922) | Process group remains alive after container process is killed externally | Major | nodemanager | Billie Rinaldi | Billie Rinaldi | +| [YARN-1919](https://issues.apache.org/jira/browse/YARN-1919) | Potential NPE in EmbeddedElectorService#stop | Minor | resourcemanager | Devaraj K | Tsuyoshi Ozawa | +| [YARN-1796](https://issues.apache.org/jira/browse/YARN-1796) | container-executor shouldn't require o-r permissions | Minor | nodemanager | Aaron T. Myers | Aaron T. Myers | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11253](https://issues.apache.org/jira/browse/HADOOP-11253) | Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows | Major | tools | Varun Vasudev | Varun Vasudev | +| [HADOOP-11241](https://issues.apache.org/jira/browse/HADOOP-11241) | TestNMSimulator fails sometimes due to timing issue | Major | . | Varun Vasudev | Varun Vasudev | +| [HADOOP-11088](https://issues.apache.org/jira/browse/HADOOP-11088) | Unittest TestKeyShell, TestCredShell and TestKMS assume UNIX path separator for JECKS key store path | Major | security | Xiaoyu Yao | Xiaoyu Yao | +| [HADOOP-11073](https://issues.apache.org/jira/browse/HADOOP-11073) | Credential Provider related Unit Tests Failure on Windows | Major | security | Xiaoyu Yao | Xiaoyu Yao | +| [HADOOP-11071](https://issues.apache.org/jira/browse/HADOOP-11071) | KMSClientProvider should drain the local generated EEK cache on key rollover | Minor | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11070](https://issues.apache.org/jira/browse/HADOOP-11070) | Create MiniKMS for testing | Major | security, test | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-11060](https://issues.apache.org/jira/browse/HADOOP-11060) | Create a CryptoCodec test that verifies interoperability between the JCE and OpenSSL implementations | Major | security | Alejandro Abdelnur | Yi Liu | +| [HDFS-7355](https://issues.apache.org/jira/browse/HDFS-7355) | TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner. | Trivial | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7126](https://issues.apache.org/jira/browse/HDFS-7126) | TestEncryptionZonesWithHA assumes Unix path separator for KMS key store path | Minor | security, test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7115](https://issues.apache.org/jira/browse/HDFS-7115) | TestEncryptionZones assumes Unix path separator for KMS key store path | Major | encryption | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7051](https://issues.apache.org/jira/browse/HDFS-7051) | TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator. | Minor | datanode, test | Chris Nauroth | Chris Nauroth | +| [HDFS-7025](https://issues.apache.org/jira/browse/HDFS-7025) | HDFS Credential Provider related Unit Test Failure | Major | encryption | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7006](https://issues.apache.org/jira/browse/HDFS-7006) | Test encryption zones with KMS | Major | security, test | Alejandro Abdelnur | Alejandro Abdelnur | +| [HDFS-6878](https://issues.apache.org/jira/browse/HDFS-6878) | Change MiniDFSCluster to support StorageType configuration for individual directories | Minor | test | Tsz Wo Nicholas Sze | Arpit Agarwal | +| [HDFS-6665](https://issues.apache.org/jira/browse/HDFS-6665) | Add tests for XAttrs in combination with viewfs | Major | hdfs-client | Stephen Chu | Stephen Chu | +| [HDFS-6645](https://issues.apache.org/jira/browse/HDFS-6645) | Add test for successive Snapshots between XAttr modifications | Minor | snapshots, test | Stephen Chu | Stephen Chu | +| [HDFS-6638](https://issues.apache.org/jira/browse/HDFS-6638) | shorten test run time with a smaller retry timeout setting | Major | test | Liang Xie | Liang Xie | +| [HDFS-6617](https://issues.apache.org/jira/browse/HDFS-6617) | Flake TestDFSZKFailoverController.testManualFailoverWithDFSHAAdmin due to a long edit log sync op | Minor | auto-failover, test | Liang Xie | Liang Xie | +| [HDFS-5624](https://issues.apache.org/jira/browse/HDFS-5624) | Add HDFS tests for ACLs in combination with viewfs. | Major | hdfs-client, test | Chris Nauroth | Stephen Chu | +| [MAPREDUCE-6115](https://issues.apache.org/jira/browse/MAPREDUCE-6115) | TestPipeApplication#testSubmitter fails in trunk | Minor | . | Ted Yu | Binglin Chang | +| [MAPREDUCE-6048](https://issues.apache.org/jira/browse/MAPREDUCE-6048) | TestJavaSerialization fails in trunk build | Minor | . | Ted Yu | Varun Vasudev | +| [MAPREDUCE-5866](https://issues.apache.org/jira/browse/MAPREDUCE-5866) | TestFixedLengthInputFormat fails in windows | Major | client, test | Varun Vasudev | Varun Vasudev | +| [YARN-2812](https://issues.apache.org/jira/browse/YARN-2812) | TestApplicationHistoryServer is likely to fail on less powerful machine | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2810](https://issues.apache.org/jira/browse/YARN-2810) | TestRMProxyUsersConf fails on Windows VMs | Major | resourcemanager | Varun Vasudev | Varun Vasudev | +| [YARN-2785](https://issues.apache.org/jira/browse/YARN-2785) | TestContainerResourceUsage fails intermittently | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2767](https://issues.apache.org/jira/browse/YARN-2767) | RM web services - add test case to ensure the http static user cannot kill or submit apps in secure mode | Major | resourcemanager | Varun Vasudev | Varun Vasudev | +| [YARN-2758](https://issues.apache.org/jira/browse/YARN-2758) | Update TestApplicationHistoryClientService to use the new generic history store | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2747](https://issues.apache.org/jira/browse/YARN-2747) | TestAggregatedLogFormat fails in trunk | Major | . | Xuan Gong | Xuan Gong | +| [YARN-2711](https://issues.apache.org/jira/browse/YARN-2711) | TestDefaultContainerExecutor#testContainerLaunchError fails on Windows | Major | . | Varun Vasudev | Varun Vasudev | +| [YARN-2607](https://issues.apache.org/jira/browse/YARN-2607) | TestDistributedShell fails in trunk | Major | . | Ted Yu | Wangda Tan | +| [YARN-2596](https://issues.apache.org/jira/browse/YARN-2596) | TestWorkPreservingRMRestart fails with FairScheduler | Major | . | Junping Du | Karthik Kambatla | +| [YARN-2584](https://issues.apache.org/jira/browse/YARN-2584) | TestContainerManagerSecurity fails on trunk | Major | . | Zhijie Shen | Jian He | +| [YARN-2549](https://issues.apache.org/jira/browse/YARN-2549) | TestContainerLaunch fails due to classpath problem with hamcrest classes. | Minor | nodemanager, test | Chris Nauroth | Chris Nauroth | +| [YARN-2519](https://issues.apache.org/jira/browse/YARN-2519) | Credential Provider related unit tests failed on Windows | Major | webapp | Xiaoyu Yao | Xiaoyu Yao | +| [YARN-2388](https://issues.apache.org/jira/browse/YARN-2388) | TestTimelineWebServices fails on trunk after HADOOP-10791 | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2158](https://issues.apache.org/jira/browse/YARN-2158) | TestRMWebServicesAppsModification sometimes fails in trunk | Minor | . | Ted Yu | Varun Vasudev | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11074](https://issues.apache.org/jira/browse/HADOOP-11074) | Move s3-related FS connector code to hadoop-aws | Major | fs/s3 | David S. Wang | David S. Wang | +| [HADOOP-10933](https://issues.apache.org/jira/browse/HADOOP-10933) | FileBasedKeyStoresFactory Should use Configuration.getPassword for SSL Passwords | Major | security | Larry McCay | Larry McCay | +| [HADOOP-10905](https://issues.apache.org/jira/browse/HADOOP-10905) | LdapGroupsMapping Should use configuration.getPassword for SSL and LDAP Passwords | Major | security | Larry McCay | Larry McCay | +| [HADOOP-10886](https://issues.apache.org/jira/browse/HADOOP-10886) | CryptoCodec#getCodecclasses throws NPE when configurations not loaded. | Major | fs | Uma Maheswara Rao G | Uma Maheswara Rao G | +| [HADOOP-10884](https://issues.apache.org/jira/browse/HADOOP-10884) | Fix dead link in Configuration javadoc | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10868](https://issues.apache.org/jira/browse/HADOOP-10868) | Create a ZooKeeper-backed secret provider | Major | security | Robert Kanter | Robert Kanter | +| [HADOOP-10650](https://issues.apache.org/jira/browse/HADOOP-10650) | Add ability to specify a reverse ACL (black list) of users and groups | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10635](https://issues.apache.org/jira/browse/HADOOP-10635) | Add a method to CryptoCodec to generate SRNs for IV | Major | security | Alejandro Abdelnur | Yi Liu | +| [HADOOP-10632](https://issues.apache.org/jira/browse/HADOOP-10632) | Minor improvements to Crypto input and output streams | Major | security | Alejandro Abdelnur | Yi Liu | +| [HADOOP-10282](https://issues.apache.org/jira/browse/HADOOP-10282) | Create a FairCallQueue: a multi-level call queue which schedules incoming calls and multiplexes outgoing calls | Major | . | Chris Li | Chris Li | +| [HADOOP-10281](https://issues.apache.org/jira/browse/HADOOP-10281) | Create a scheduler, which assigns schedulables a priority level | Major | . | Chris Li | Chris Li | +| [HADOOP-10201](https://issues.apache.org/jira/browse/HADOOP-10201) | Add Listing Support to Key Management APIs | Major | security | Larry McCay | Larry McCay | +| [HDFS-7291](https://issues.apache.org/jira/browse/HDFS-7291) | Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7181](https://issues.apache.org/jira/browse/HDFS-7181) | Remove incorrect precondition check on key length in FileEncryptionInfo | Critical | encryption | Andrew Wang | Andrew Wang | +| [HDFS-7179](https://issues.apache.org/jira/browse/HDFS-7179) | DFSClient should instantiate a KeyProvider, not a KeyProviderCryptoExtension | Critical | encryption | Andrew Wang | Andrew Wang | +| [HDFS-7171](https://issues.apache.org/jira/browse/HDFS-7171) | Fix Jenkins failures in HDFS-6581 branch | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7159](https://issues.apache.org/jira/browse/HDFS-7159) | Use block storage policy to set lazy persist preference | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7157](https://issues.apache.org/jira/browse/HDFS-7157) | Using Time.now() for recording start/end time of reconfiguration tasks | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7155](https://issues.apache.org/jira/browse/HDFS-7155) | Bugfix in createLocatedFileStatus caused by bad merge | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7154](https://issues.apache.org/jira/browse/HDFS-7154) | Fix returning value of starting reconfiguration task | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7144](https://issues.apache.org/jira/browse/HDFS-7144) | Fix findbugs warnings in RamDiskReplicaTracker | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7143](https://issues.apache.org/jira/browse/HDFS-7143) | Fix findbugs warnings in HDFS-6581 branch | Major | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7140](https://issues.apache.org/jira/browse/HDFS-7140) | Add a tool to list all the existing block storage policies | Minor | hdfs-client, namenode | Jing Zhao | Jing Zhao | +| [HDFS-7139](https://issues.apache.org/jira/browse/HDFS-7139) | Unit test for creating encryption zone on root path | Minor | . | Zhe Zhang | Zhe Zhang | +| [HDFS-7138](https://issues.apache.org/jira/browse/HDFS-7138) | Fix hftp to work with encryption | Major | namenode | Charles Lamb | Charles Lamb | +| [HDFS-7129](https://issues.apache.org/jira/browse/HDFS-7129) | Metrics to track usage of memory for writes | Major | datanode | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-7124](https://issues.apache.org/jira/browse/HDFS-7124) | Remove EncryptionZoneManager.NULL\_EZ | Minor | namenode | Charles Lamb | Charles Lamb | +| [HDFS-7119](https://issues.apache.org/jira/browse/HDFS-7119) | Split error checks in AtomicFileOutputStream#close into separate conditions to improve diagnostics. | Minor | journal-node | Chris Nauroth | Chris Nauroth | +| [HDFS-7118](https://issues.apache.org/jira/browse/HDFS-7118) | Improve diagnostics on storage directory rename operations by using NativeIO#renameTo in Storage#rename. | Major | journal-node, namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-7112](https://issues.apache.org/jira/browse/HDFS-7112) | LazyWriter should use either async IO or one thread per physical disk | Major | datanode | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-7108](https://issues.apache.org/jira/browse/HDFS-7108) | Fix unit test failures in SimulatedFsDataset | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7100](https://issues.apache.org/jira/browse/HDFS-7100) | Make eviction scheme pluggable | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7095](https://issues.apache.org/jira/browse/HDFS-7095) | TestStorageMover often fails in Jenkins | Minor | test | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-7091](https://issues.apache.org/jira/browse/HDFS-7091) | Add forwarding constructor for INodeFile for existing callers | Minor | namenode, test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7090](https://issues.apache.org/jira/browse/HDFS-7090) | Use unbuffered writes when persisting in-memory replicas | Major | datanode | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-7084](https://issues.apache.org/jira/browse/HDFS-7084) | FsDatasetImpl#copyBlockFiles debug log can be improved | Minor | datanode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7081](https://issues.apache.org/jira/browse/HDFS-7081) | Add new DistributedFileSystem API for getting all the existing storage policies | Major | balancer & mover, namenode | Jing Zhao | Jing Zhao | +| [HDFS-7080](https://issues.apache.org/jira/browse/HDFS-7080) | Fix finalize and upgrade unit test failures | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7079](https://issues.apache.org/jira/browse/HDFS-7079) | Few more unit test fixes for HDFS-6581 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7078](https://issues.apache.org/jira/browse/HDFS-7078) | Fix listEZs to work correctly with snapshots | Major | encryption | Andrew Wang | Andrew Wang | +| [HDFS-7077](https://issues.apache.org/jira/browse/HDFS-7077) | Separate CipherSuite from crypto protocol version | Major | encryption | Andrew Wang | Andrew Wang | +| [HDFS-7066](https://issues.apache.org/jira/browse/HDFS-7066) | LazyWriter#evictBlocks misses a null check for replicaState | Minor | datanode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7064](https://issues.apache.org/jira/browse/HDFS-7064) | Fix unit test failures in HDFS-6581 branch | Major | test | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-7061](https://issues.apache.org/jira/browse/HDFS-7061) | Add test to verify encryption zone creation after NameNode restart without saving namespace | Minor | encryption, test | Stephen Chu | Stephen Chu | +| [HDFS-7047](https://issues.apache.org/jira/browse/HDFS-7047) | Expose FileStatus#isEncrypted in libhdfs | Major | encryption | Andrew Wang | Colin Patrick McCabe | +| [HDFS-7032](https://issues.apache.org/jira/browse/HDFS-7032) | Add WebHDFS support for reading and writing to encryption zones | Major | encryption, webhdfs | Stephen Chu | Charles Lamb | +| [HDFS-7004](https://issues.apache.org/jira/browse/HDFS-7004) | Update KeyProvider instantiation to create by URI | Major | encryption | Andrew Wang | Andrew Wang | +| [HDFS-7003](https://issues.apache.org/jira/browse/HDFS-7003) | Add NFS Gateway support for reading and writing to encryption zones | Major | encryption, nfs | Stephen Chu | Charles Lamb | +| [HDFS-6991](https://issues.apache.org/jira/browse/HDFS-6991) | Notify NN of evicted block before deleting it from RAM disk | Major | datanode, namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6990](https://issues.apache.org/jira/browse/HDFS-6990) | Add unit test for evict/delete RAM\_DISK block with open handle | Major | datanode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-6988](https://issues.apache.org/jira/browse/HDFS-6988) | Improve HDFS-6581 eviction configuration | Major | datanode | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-6987](https://issues.apache.org/jira/browse/HDFS-6987) | Move CipherSuite xattr information up to the encryption zone root | Major | encryption | Andrew Wang | Zhe Zhang | +| [HDFS-6986](https://issues.apache.org/jira/browse/HDFS-6986) | DistributedFileSystem must get delegation tokens from configured KeyProvider | Major | security | Alejandro Abdelnur | Zhe Zhang | +| [HDFS-6978](https://issues.apache.org/jira/browse/HDFS-6978) | Directory scanner should correctly reconcile blocks on RAM disk | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6977](https://issues.apache.org/jira/browse/HDFS-6977) | Delete all copies when a block is deleted from the block space | Major | datanode | Nathan Yao | Arpit Agarwal | +| [HDFS-6970](https://issues.apache.org/jira/browse/HDFS-6970) | Move startFile EDEK retries to the DFSClient | Major | encryption | Andrew Wang | Andrew Wang | +| [HDFS-6966](https://issues.apache.org/jira/browse/HDFS-6966) | Add additional unit tests for encryption zones | Major | encryption | Stephen Chu | Stephen Chu | +| [HDFS-6960](https://issues.apache.org/jira/browse/HDFS-6960) | Bugfix in LazyWriter, fix test case and some refactoring | Major | datanode, test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6956](https://issues.apache.org/jira/browse/HDFS-6956) | Allow dynamically changing the tracing level in Hadoop servers | Major | datanode, namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6951](https://issues.apache.org/jira/browse/HDFS-6951) | Correctly persist raw namespace xattrs to edit log and fsimage | Major | encryption | Stephen Chu | Charles Lamb | +| [HDFS-6950](https://issues.apache.org/jira/browse/HDFS-6950) | Add Additional unit tests for HDFS-6581 | Major | . | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-6934](https://issues.apache.org/jira/browse/HDFS-6934) | Move checksum computation off the hot path when writing to RAM disk | Major | datanode, hdfs-client | Arpit Agarwal | Chris Nauroth | +| [HDFS-6932](https://issues.apache.org/jira/browse/HDFS-6932) | Balancer and Mover tools should ignore replicas on RAM\_DISK | Major | datanode | Arpit Agarwal | Xiaoyu Yao | +| [HDFS-6931](https://issues.apache.org/jira/browse/HDFS-6931) | Move lazily persisted replicas to finalized directory on DN startup | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6930](https://issues.apache.org/jira/browse/HDFS-6930) | Improve replica eviction from RAM disk | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6929](https://issues.apache.org/jira/browse/HDFS-6929) | NN periodically unlinks lazy persist files with missing replicas from namespace | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6928](https://issues.apache.org/jira/browse/HDFS-6928) | 'hdfs put' command should accept lazyPersist flag for testing | Major | datanode | Tassapol Athiapinya | Arpit Agarwal | +| [HDFS-6927](https://issues.apache.org/jira/browse/HDFS-6927) | Add unit tests | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6926](https://issues.apache.org/jira/browse/HDFS-6926) | DN support for saving replicas to persistent storage and evicting in-memory replicas | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6925](https://issues.apache.org/jira/browse/HDFS-6925) | DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6924](https://issues.apache.org/jira/browse/HDFS-6924) | Add new RAM\_DISK storage type | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6923](https://issues.apache.org/jira/browse/HDFS-6923) | Propagate LazyPersist flag to DNs via DataTransferProtocol | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6921](https://issues.apache.org/jira/browse/HDFS-6921) | Add LazyPersist flag to FileStatus | Major | . | Arpit Agarwal | Arpit Agarwal | +| [HDFS-6894](https://issues.apache.org/jira/browse/HDFS-6894) | Add XDR parser method for each NFS response | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-6892](https://issues.apache.org/jira/browse/HDFS-6892) | Add XDR packaging method for each NFS request | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-6880](https://issues.apache.org/jira/browse/HDFS-6880) | Adding tracing to DataNode data transfer protocol | Major | . | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-6879](https://issues.apache.org/jira/browse/HDFS-6879) | Adding tracing to Hadoop RPC | Major | . | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-6865](https://issues.apache.org/jira/browse/HDFS-6865) | Byte array native checksumming on client side (HDFS changes) | Major | hdfs-client, performance | James Thomas | James Thomas | +| [HDFS-6851](https://issues.apache.org/jira/browse/HDFS-6851) | Refactor EncryptionZoneWithId and EncryptionZone | Major | namenode, security | Charles Lamb | Charles Lamb | +| [HDFS-6843](https://issues.apache.org/jira/browse/HDFS-6843) | Create FileStatus isEncrypted() method | Major | namenode, security | Charles Lamb | Charles Lamb | +| [HDFS-6808](https://issues.apache.org/jira/browse/HDFS-6808) | Add command line option to ask DataNode reload configuration. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6777](https://issues.apache.org/jira/browse/HDFS-6777) | Supporting consistent edit log reads when in-progress edit log segments are included | Major | qjm | James Thomas | James Thomas | +| [HDFS-6774](https://issues.apache.org/jira/browse/HDFS-6774) | Make FsDataset and DataStore support removing volumes. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6750](https://issues.apache.org/jira/browse/HDFS-6750) | The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale | Major | datanode, hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6740](https://issues.apache.org/jira/browse/HDFS-6740) | Make FSDataset support adding data volumes dynamically | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6728](https://issues.apache.org/jira/browse/HDFS-6728) | Dynamically add new volumes to DataStorage, formatted if necessary. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6727](https://issues.apache.org/jira/browse/HDFS-6727) | Refresh data volumes on DataNode based on configuration changes | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6722](https://issues.apache.org/jira/browse/HDFS-6722) | Display readable last contact time for dead nodes on NN webUI | Major | . | Ming Ma | Ming Ma | +| [HDFS-6717](https://issues.apache.org/jira/browse/HDFS-6717) | Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config | Minor | nfs | Jeff Hansen | Brandon Li | +| [HDFS-6705](https://issues.apache.org/jira/browse/HDFS-6705) | Create an XAttr that disallows the HDFS admin from accessing a file | Major | namenode, security | Charles Lamb | Charles Lamb | +| [HDFS-6609](https://issues.apache.org/jira/browse/HDFS-6609) | Use DirectorySnapshottableFeature to represent a snapshottable directory | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-6385](https://issues.apache.org/jira/browse/HDFS-6385) | Show when block deletion will start after NameNode startup in WebUI | Major | . | Jing Zhao | Chris Nauroth | +| [HDFS-6036](https://issues.apache.org/jira/browse/HDFS-6036) | Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long | Major | caching, datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [MAPREDUCE-6142](https://issues.apache.org/jira/browse/MAPREDUCE-6142) | Test failure in TestJobHistoryEventHandler and TestMRTimelineEventHandling | Critical | . | Zhijie Shen | Zhijie Shen | +| [MAPREDUCE-6018](https://issues.apache.org/jira/browse/MAPREDUCE-6018) | Create a framework specific config to enable timeline server | Major | . | Jonathan Eagles | Robert Kanter | +| [MAPREDUCE-5974](https://issues.apache.org/jira/browse/MAPREDUCE-5974) | Allow specifying multiple MapOutputCollectors with fallback | Major | task | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-5963](https://issues.apache.org/jira/browse/MAPREDUCE-5963) | ShuffleHandler DB schema should be versioned with compatible/incompatible changes | Major | . | Junping Du | Junping Du | +| [MAPREDUCE-5956](https://issues.apache.org/jira/browse/MAPREDUCE-5956) | MapReduce AM should not use maxAttempts to determine if this is the last retry | Blocker | applicationmaster, mrv2 | Vinod Kumar Vavilapalli | Wangda Tan | +| [MAPREDUCE-5945](https://issues.apache.org/jira/browse/MAPREDUCE-5945) | Update the description of GenericOptionsParser -jt option | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5942](https://issues.apache.org/jira/browse/MAPREDUCE-5942) | Remove MRv1 commands from CommandsManual.apt.vm | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5933](https://issues.apache.org/jira/browse/MAPREDUCE-5933) | Enable MR AM to post history events to the timeline server | Major | mr-am | Zhijie Shen | Robert Kanter | +| [MAPREDUCE-5891](https://issues.apache.org/jira/browse/MAPREDUCE-5891) | Improved shuffle error handling across NM restarts | Major | . | Jason Lowe | Junping Du | +| [YARN-2843](https://issues.apache.org/jira/browse/YARN-2843) | NodeLabels manager should trim all inputs for hosts and labels | Major | resourcemanager | Sushmitha Sreenivasan | Wangda Tan | +| [YARN-2841](https://issues.apache.org/jira/browse/YARN-2841) | RMProxy should retry EOFException | Critical | resourcemanager | Jian He | Jian He | +| [YARN-2824](https://issues.apache.org/jira/browse/YARN-2824) | Capacity of labels should be zero by default | Critical | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2795](https://issues.apache.org/jira/browse/YARN-2795) | Resource Manager fails startup with HDFS label storage and secure cluster | Major | resourcemanager | Phil D'Amore | Wangda Tan | +| [YARN-2778](https://issues.apache.org/jira/browse/YARN-2778) | YARN node CLI should display labels on returned node reports | Major | client | Wangda Tan | Wangda Tan | +| [YARN-2770](https://issues.apache.org/jira/browse/YARN-2770) | Timeline delegation tokens need to be automatically renewed by the RM | Critical | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2753](https://issues.apache.org/jira/browse/YARN-2753) | Fix potential issues and code clean up for \*NodeLabelsManager | Major | . | zhihai xu | zhihai xu | +| [YARN-2744](https://issues.apache.org/jira/browse/YARN-2744) | Under some scenario, it is possible to end up with capacity scheduler configuration that uses labels that no longer exist | Critical | capacityscheduler | Sumit Mohanty | Wangda Tan | +| [YARN-2726](https://issues.apache.org/jira/browse/YARN-2726) | CapacityScheduler should explicitly log when an accessible label has no capacity | Minor | capacityscheduler | Phil D'Amore | Wangda Tan | +| [YARN-2723](https://issues.apache.org/jira/browse/YARN-2723) | rmadmin -replaceLabelsOnNode does not correctly parse port | Major | client | Phil D'Amore | Naganarasimha G R | +| [YARN-2717](https://issues.apache.org/jira/browse/YARN-2717) | containerLogNotFound log shows multiple time for the same container | Major | log-aggregation | Xuan Gong | Xuan Gong | +| [YARN-2709](https://issues.apache.org/jira/browse/YARN-2709) | Add retry for timeline client getDelegationToken method | Major | . | Li Lu | Li Lu | +| [YARN-2705](https://issues.apache.org/jira/browse/YARN-2705) | Changes of RM node label manager default configuration | Critical | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2704](https://issues.apache.org/jira/browse/YARN-2704) | Localization and log-aggregation will fail if hdfs delegation token expired after token-max-life-time | Critical | . | Jian He | Jian He | +| [YARN-2703](https://issues.apache.org/jira/browse/YARN-2703) | Add logUploadedTime into LogValue for better display | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2700](https://issues.apache.org/jira/browse/YARN-2700) | TestSecureRMRegistryOperations failing on windows: auth problems | Major | api, resourcemanager | Steve Loughran | Steve Loughran | +| [YARN-2699](https://issues.apache.org/jira/browse/YARN-2699) | Fix test timeout in TestResourceTrackerOnHA#testResourceTrackerOnHA | Blocker | client | Wangda Tan | Wangda Tan | +| [YARN-2698](https://issues.apache.org/jira/browse/YARN-2698) | Move getClusterNodeLabels and getNodeToLabels to YarnClient instead of AdminService | Critical | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2692](https://issues.apache.org/jira/browse/YARN-2692) | ktutil test hanging on some machines/ktutil versions | Major | . | Steve Loughran | Steve Loughran | +| [YARN-2689](https://issues.apache.org/jira/browse/YARN-2689) | TestSecureRMRegistryOperations failing on windows: secure ZK won't start | Major | api, resourcemanager | Steve Loughran | Steve Loughran | +| [YARN-2685](https://issues.apache.org/jira/browse/YARN-2685) | Resource on each label not correct when multiple NMs in a same host and some has label some not | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2678](https://issues.apache.org/jira/browse/YARN-2678) | Improved Yarn Registry service record structure | Major | api, resourcemanager | Gour Saha | Steve Loughran | +| [YARN-2677](https://issues.apache.org/jira/browse/YARN-2677) | registry punycoding of usernames doesn't fix all usernames to be DNS-valid | Major | api, resourcemanager | Steve Loughran | Steve Loughran | +| [YARN-2676](https://issues.apache.org/jira/browse/YARN-2676) | Timeline authentication filter should add support for proxy user | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2673](https://issues.apache.org/jira/browse/YARN-2673) | Add retry for timeline client put APIs | Major | . | Li Lu | Li Lu | +| [YARN-2668](https://issues.apache.org/jira/browse/YARN-2668) | yarn-registry JAR won't link against ZK 3.4.5 | Major | client | Steve Loughran | Steve Loughran | +| [YARN-2652](https://issues.apache.org/jira/browse/YARN-2652) | add hadoop-yarn-registry package under hadoop-yarn | Major | api | Steve Loughran | Steve Loughran | +| [YARN-2651](https://issues.apache.org/jira/browse/YARN-2651) | Spin off the LogRollingInterval from LogAggregationContext | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2647](https://issues.apache.org/jira/browse/YARN-2647) | Add yarn queue CLI to get queue infos | Major | client | Wangda Tan | Sunil G | +| [YARN-2644](https://issues.apache.org/jira/browse/YARN-2644) | Recalculate headroom more frequently to keep it accurate | Major | . | Craig Welch | Craig Welch | +| [YARN-2632](https://issues.apache.org/jira/browse/YARN-2632) | Document NM Restart feature | Blocker | nodemanager | Junping Du | Junping Du | +| [YARN-2629](https://issues.apache.org/jira/browse/YARN-2629) | Make distributed shell use the domain-based timeline ACLs | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2621](https://issues.apache.org/jira/browse/YARN-2621) | Simplify the output when the user doesn't have the access for getDomain(s) | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2611](https://issues.apache.org/jira/browse/YARN-2611) | Fix jenkins findbugs warning and test case failures for trunk merge patch | Major | capacityscheduler, resourcemanager, scheduler | Subru Krishnan | Subru Krishnan | +| [YARN-2591](https://issues.apache.org/jira/browse/YARN-2591) | AHSWebServices should return FORBIDDEN(403) if the request user doesn't have access to the history data | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2583](https://issues.apache.org/jira/browse/YARN-2583) | Modify the LogDeletionService to support Log aggregation for LRS | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2582](https://issues.apache.org/jira/browse/YARN-2582) | Log related CLI and Web UI changes for Aggregated Logs in LRS | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2581](https://issues.apache.org/jira/browse/YARN-2581) | NMs need to find a way to get LogAggregationContext | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2576](https://issues.apache.org/jira/browse/YARN-2576) | Prepare yarn-1051 branch for merging with trunk | Major | capacityscheduler, resourcemanager, scheduler | Subru Krishnan | Subru Krishnan | +| [YARN-2569](https://issues.apache.org/jira/browse/YARN-2569) | API changes for handling logs of long-running services | Major | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2566](https://issues.apache.org/jira/browse/YARN-2566) | DefaultContainerExecutor should pick a working directory randomly | Critical | nodemanager | zhihai xu | zhihai xu | +| [YARN-2561](https://issues.apache.org/jira/browse/YARN-2561) | MR job client cannot reconnect to AM after NM restart. | Blocker | . | Tassapol Athiapinya | Junping Du | +| [YARN-2558](https://issues.apache.org/jira/browse/YARN-2558) | Updating ContainerTokenIdentifier#read/write to use ContainerId#getContainerId | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2547](https://issues.apache.org/jira/browse/YARN-2547) | Cross Origin Filter throws UnsupportedOperationException upon destroy | Major | timelineserver | Jonathan Eagles | Mit Desai | +| [YARN-2544](https://issues.apache.org/jira/browse/YARN-2544) | Common server side PB changes (not include user API PB changes) | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2542](https://issues.apache.org/jira/browse/YARN-2542) | "yarn application -status \" throws NPE when retrieving the app from the timelineserver | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2538](https://issues.apache.org/jira/browse/YARN-2538) | Add logs when RM send new AMRMToken to ApplicationMaster | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2529](https://issues.apache.org/jira/browse/YARN-2529) | Generic history service RPC interface doesn't work when service authorization is enabled | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2528](https://issues.apache.org/jira/browse/YARN-2528) | Cross Origin Filter Http response split vulnerability protection rejects valid origins | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | +| [YARN-2515](https://issues.apache.org/jira/browse/YARN-2515) | Update ConverterUtils#toContainerId to parse epoch | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2512](https://issues.apache.org/jira/browse/YARN-2512) | Allow for origin pattern matching in cross origin filter | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | +| [YARN-2511](https://issues.apache.org/jira/browse/YARN-2511) | Allow All Origins by default when Cross Origin Filter is enabled | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | +| [YARN-2509](https://issues.apache.org/jira/browse/YARN-2509) | Enable Cross Origin Filter for timeline server only and not all Yarn servers | Major | timelineserver | Jonathan Eagles | Mit Desai | +| [YARN-2508](https://issues.apache.org/jira/browse/YARN-2508) | Cross Origin configuration parameters prefix are not honored | Major | timelineserver | Jonathan Eagles | Mit Desai | +| [YARN-2507](https://issues.apache.org/jira/browse/YARN-2507) | Document Cross Origin Filter Configuration for ATS | Major | documentation, timelineserver | Jonathan Eagles | Jonathan Eagles | +| [YARN-2505](https://issues.apache.org/jira/browse/YARN-2505) | Support get/add/remove/change labels in RM REST API | Major | resourcemanager | Wangda Tan | Craig Welch | +| [YARN-2504](https://issues.apache.org/jira/browse/YARN-2504) | Support get/add/remove/change labels in RM admin CLI | Critical | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2503](https://issues.apache.org/jira/browse/YARN-2503) | Changes in RM Web UI to better show labels to end users | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2502](https://issues.apache.org/jira/browse/YARN-2502) | Changes in distributed shell to support specify labels | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2501](https://issues.apache.org/jira/browse/YARN-2501) | Changes in AMRMClient to support labels | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2500](https://issues.apache.org/jira/browse/YARN-2500) | Miscellaneous changes in ResourceManager to support labels | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2496](https://issues.apache.org/jira/browse/YARN-2496) | Changes for capacity scheduler to support allocate resource respect labels | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2494](https://issues.apache.org/jira/browse/YARN-2494) | Node label manager API and storage implementations | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2493](https://issues.apache.org/jira/browse/YARN-2493) | API changes for users | Major | api | Wangda Tan | Wangda Tan | +| [YARN-2475](https://issues.apache.org/jira/browse/YARN-2475) | ReservationSystem: replan upon capacity reduction | Major | resourcemanager | Carlo Curino | Carlo Curino | +| [YARN-2468](https://issues.apache.org/jira/browse/YARN-2468) | Log handling for LRS | Major | log-aggregation, nodemanager, resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2456](https://issues.apache.org/jira/browse/YARN-2456) | Possible livelock in CapacityScheduler when RM is recovering apps | Major | resourcemanager | Jian He | Jian He | +| [YARN-2446](https://issues.apache.org/jira/browse/YARN-2446) | Using TimelineNamespace to shield the entities of a user | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2434](https://issues.apache.org/jira/browse/YARN-2434) | RM should not recover containers from previously failed attempt when AM restart is not enabled | Major | . | Jian He | Jian He | +| [YARN-2431](https://issues.apache.org/jira/browse/YARN-2431) | NM restart: cgroup is not removed for reacquired containers | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-2406](https://issues.apache.org/jira/browse/YARN-2406) | Move RM recovery related proto to yarn\_server\_resourcemanager\_recovery.proto | Major | . | Jian He | Tsuyoshi Ozawa | +| [YARN-2389](https://issues.apache.org/jira/browse/YARN-2389) | Adding support for drainig a queue, ie killing all apps in the queue | Major | capacityscheduler, fairscheduler | Subru Krishnan | Subru Krishnan | +| [YARN-2378](https://issues.apache.org/jira/browse/YARN-2378) | Adding support for moving apps between queues in Capacity Scheduler | Major | capacityscheduler | Subru Krishnan | Subru Krishnan | +| [YARN-2354](https://issues.apache.org/jira/browse/YARN-2354) | DistributedShell may allocate more containers than client specified after it restarts | Major | . | Jian He | Li Lu | +| [YARN-2347](https://issues.apache.org/jira/browse/YARN-2347) | Consolidate RMStateVersion and NMDBSchemaVersion into StateVersion in yarn-server-common | Major | . | Junping Du | Junping Du | +| [YARN-2317](https://issues.apache.org/jira/browse/YARN-2317) | Update documentation about how to write YARN applications | Major | documentation | Li Lu | Li Lu | +| [YARN-2312](https://issues.apache.org/jira/browse/YARN-2312) | Marking ContainerId#getId as deprecated | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2302](https://issues.apache.org/jira/browse/YARN-2302) | Refactor TimelineWebServices | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2298](https://issues.apache.org/jira/browse/YARN-2298) | Move TimelineClient to yarn-common project | Major | client | Zhijie Shen | Zhijie Shen | +| [YARN-2295](https://issues.apache.org/jira/browse/YARN-2295) | Refactor YARN distributed shell with existing public stable API | Major | . | Li Lu | Li Lu | +| [YARN-2288](https://issues.apache.org/jira/browse/YARN-2288) | Data persistent in timelinestore should be versioned | Major | timelineserver | Junping Du | Junping Du | +| [YARN-2279](https://issues.apache.org/jira/browse/YARN-2279) | Add UTs to cover timeline server authentication | Major | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2277](https://issues.apache.org/jira/browse/YARN-2277) | Add Cross-Origin support to the ATS REST API | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | +| [YARN-2260](https://issues.apache.org/jira/browse/YARN-2260) | Add containers to launchedContainers list in RMNode on container recovery | Major | resourcemanager | Jian He | Jian He | +| [YARN-2249](https://issues.apache.org/jira/browse/YARN-2249) | AM release request may be lost on RM restart | Major | resourcemanager | Jian He | Jian He | +| [YARN-2248](https://issues.apache.org/jira/browse/YARN-2248) | Capacity Scheduler changes for moving apps between queues | Major | capacityscheduler | Janos Matyas | Janos Matyas | +| [YARN-2242](https://issues.apache.org/jira/browse/YARN-2242) | Improve exception information on AM launch crashes | Major | . | Li Lu | Li Lu | +| [YARN-2237](https://issues.apache.org/jira/browse/YARN-2237) | MRAppMaster changes for AMRMToken roll-up | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2229](https://issues.apache.org/jira/browse/YARN-2229) | ContainerId can overflow with RM restart | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2228](https://issues.apache.org/jira/browse/YARN-2228) | TimelineServer should load pseudo authentication filter when authentication = simple | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2212](https://issues.apache.org/jira/browse/YARN-2212) | ApplicationMaster needs to find a way to update the AMRMToken periodically | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2211](https://issues.apache.org/jira/browse/YARN-2211) | RMStateStore needs to save AMRMToken master key for recovery when RM restart/failover happens | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2208](https://issues.apache.org/jira/browse/YARN-2208) | AMRMTokenManager need to have a way to roll over AMRMToken | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-2182](https://issues.apache.org/jira/browse/YARN-2182) | Update ContainerId#toString() to avoid conflicts before and after RM restart | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2174](https://issues.apache.org/jira/browse/YARN-2174) | Enabling HTTPs for the writer REST API of TimelineServer | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2153](https://issues.apache.org/jira/browse/YARN-2153) | Ensure distributed shell work with RM work-preserving recovery | Major | resourcemanager | Jian He | Jian He | +| [YARN-2102](https://issues.apache.org/jira/browse/YARN-2102) | More generalized timeline ACLs | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2080](https://issues.apache.org/jira/browse/YARN-2080) | Admission Control: Integrate Reservation subsystem with ResourceManager | Major | resourcemanager | Subru Krishnan | Subru Krishnan | +| [YARN-2070](https://issues.apache.org/jira/browse/YARN-2070) | DistributedShell publishes unfriendly user information to the timeline server | Minor | . | Zhijie Shen | Robert Kanter | +| [YARN-2045](https://issues.apache.org/jira/browse/YARN-2045) | Data persisted in NM should be versioned | Major | nodemanager | Junping Du | Junping Du | +| [YARN-2035](https://issues.apache.org/jira/browse/YARN-2035) | FileSystemApplicationHistoryStore blocks RM and AHS while NN is in safemode | Major | . | Jonathan Eagles | Jonathan Eagles | +| [YARN-2033](https://issues.apache.org/jira/browse/YARN-2033) | Merging generic-history into the Timeline Store | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-2013](https://issues.apache.org/jira/browse/YARN-2013) | The diagnostics is always the ExitCodeException stack when the container crashes | Major | nodemanager | Zhijie Shen | Tsuyoshi Ozawa | +| [YARN-2008](https://issues.apache.org/jira/browse/YARN-2008) | CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure | Major | . | Chen He | Craig Welch | +| [YARN-2001](https://issues.apache.org/jira/browse/YARN-2001) | Threshold for RM to accept requests from AM after failover | Major | resourcemanager | Jian He | Jian He | +| [YARN-1972](https://issues.apache.org/jira/browse/YARN-1972) | Implement secure Windows Container Executor | Major | nodemanager | Remus Rusanu | Remus Rusanu | +| [YARN-1915](https://issues.apache.org/jira/browse/YARN-1915) | ClientToAMTokenMasterKey should be provided to AM at launch time | Blocker | . | Hitesh Shah | Jason Lowe | +| [YARN-1879](https://issues.apache.org/jira/browse/YARN-1879) | Mark Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over | Critical | resourcemanager | Jian He | Tsuyoshi Ozawa | +| [YARN-1857](https://issues.apache.org/jira/browse/YARN-1857) | CapacityScheduler headroom doesn't account for other AM's running | Critical | capacityscheduler | Thomas Graves | Chen He | +| [YARN-1779](https://issues.apache.org/jira/browse/YARN-1779) | Handle AMRMTokens across RM failover | Blocker | resourcemanager | Karthik Kambatla | Jian He | +| [YARN-1712](https://issues.apache.org/jira/browse/YARN-1712) | Admission Control: plan follower | Major | capacityscheduler, resourcemanager | Carlo Curino | Carlo Curino | +| [YARN-1711](https://issues.apache.org/jira/browse/YARN-1711) | CapacityOverTimePolicy: a policy to enforce quotas over time for YARN-1709 | Major | . | Carlo Curino | Carlo Curino | +| [YARN-1710](https://issues.apache.org/jira/browse/YARN-1710) | Admission Control: agents to allocate reservation | Major | resourcemanager | Carlo Curino | Carlo Curino | +| [YARN-1709](https://issues.apache.org/jira/browse/YARN-1709) | Admission Control: Reservation subsystem | Major | resourcemanager | Carlo Curino | Subru Krishnan | +| [YARN-1708](https://issues.apache.org/jira/browse/YARN-1708) | Add a public API to reserve resources (part of YARN-1051) | Major | resourcemanager | Carlo Curino | Subru Krishnan | +| [YARN-1707](https://issues.apache.org/jira/browse/YARN-1707) | Making the CapacityScheduler more dynamic | Major | capacityscheduler | Carlo Curino | Carlo Curino | +| [YARN-1372](https://issues.apache.org/jira/browse/YARN-1372) | Ensure all completed containers are reported to the AMs across RM restart | Major | resourcemanager | Bikas Saha | Anubhav Dhoot | +| [YARN-1370](https://issues.apache.org/jira/browse/YARN-1370) | Fair scheduler to re-populate container allocation state | Major | fairscheduler | Bikas Saha | Anubhav Dhoot | +| [YARN-1367](https://issues.apache.org/jira/browse/YARN-1367) | After restart NM should resync with the RM without killing containers | Major | resourcemanager | Bikas Saha | Anubhav Dhoot | +| [YARN-1354](https://issues.apache.org/jira/browse/YARN-1354) | Recover applications upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1342](https://issues.apache.org/jira/browse/YARN-1342) | Recover container tokens upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1341](https://issues.apache.org/jira/browse/YARN-1341) | Recover NMTokens upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1337](https://issues.apache.org/jira/browse/YARN-1337) | Recover containers upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-1326](https://issues.apache.org/jira/browse/YARN-1326) | RM should log using RMStore at startup time | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-1250](https://issues.apache.org/jira/browse/YARN-1250) | Generic history service should support application-acls | Major | . | Vinod Kumar Vavilapalli | Zhijie Shen | +| [YARN-1063](https://issues.apache.org/jira/browse/YARN-1063) | Winutils needs ability to create task as domain user | Major | nodemanager | Kyle Leckie | Remus Rusanu | +| [YARN-611](https://issues.apache.org/jira/browse/YARN-611) | Add an AM retry count reset window to YARN RM | Major | resourcemanager | Chris Riccomini | Xuan Gong | +| [YARN-90](https://issues.apache.org/jira/browse/YARN-90) | NodeManager should identify failed disks becoming good again | Major | nodemanager | Ravi Gummadi | Varun Vasudev | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-10994](https://issues.apache.org/jira/browse/HADOOP-10994) | KeyProviderCryptoExtension should use CryptoCodec for generation/decryption of keys | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10992](https://issues.apache.org/jira/browse/HADOOP-10992) | Merge KMS to branch-2 | Major | security | Alejandro Abdelnur | Alejandro Abdelnur | +| [HADOOP-10882](https://issues.apache.org/jira/browse/HADOOP-10882) | Move DirectBufferPool into common util | Minor | util | Todd Lipcon | Todd Lipcon | +| [MAPREDUCE-5910](https://issues.apache.org/jira/browse/MAPREDUCE-5910) | MRAppMaster should handle Resync from RM instead of shutting down. | Major | applicationmaster | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2789](https://issues.apache.org/jira/browse/YARN-2789) | Re-instate the NodeReport.newInstance API modified in YARN-2698 | Critical | . | Siddharth Seth | Wangda Tan | +| [YARN-2207](https://issues.apache.org/jira/browse/YARN-2207) | Add ability to roll over AMRMToken | Major | resourcemanager | Xuan Gong | Xuan Gong | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/RELEASENOTES.2.6.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/RELEASENOTES.2.6.0.md new file mode 100644 index 00000000000..756938e5445 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.6.0/RELEASENOTES.2.6.0.md @@ -0,0 +1,184 @@ + + +# Apache Hadoop 2.6.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-10903](https://issues.apache.org/jira/browse/HADOOP-10903) | *Major* | **Enhance hadoop classpath command to expand wildcards or write classpath into jar manifest.** + +The "hadoop classpath" command has been enhanced to support options for automatic expansion of wildcards in classpath elements and writing the classpath to a jar file manifest. These options make it easier to construct a correct classpath for libhdfs applications. + + +--- + +* [HADOOP-10839](https://issues.apache.org/jira/browse/HADOOP-10839) | *Major* | **Add unregisterSource() to MetricsSystem API** + +The MetricsSystem abstract class has added a new abstract method, unregisterSource, for unregistering a previously registered metrics source. Custom subclasses of MetricsSystem must be updated to provide an implementation of this method. + + +--- + +* [HADOOP-10681](https://issues.apache.org/jira/browse/HADOOP-10681) | *Major* | **Remove synchronized blocks from SnappyCodec and ZlibCodec buffering inner loop** + +Remove unnecessary synchronized blocks from Snappy/Zlib codecs. + + +--- + +* [HADOOP-10620](https://issues.apache.org/jira/browse/HADOOP-10620) | *Major* | **/docs/current doesn't point to the latest version 2.4.0** + +Verified http://hadoop.apache.org/docs/current/ link now point to current release (v2.6.0). + + +--- + +* [HADOOP-10583](https://issues.apache.org/jira/browse/HADOOP-10583) | *Minor* | **bin/hadoop key throws NPE with no args and assorted other fixups** + +bin/hadoop key +with no args would throw an NPE. + + +--- + +* [HADOOP-10244](https://issues.apache.org/jira/browse/HADOOP-10244) | *Major* | **TestKeyShell improperly tests the results of a Delete** + +Fix of inappropriate test of delete functionality. + + +--- + +* [HADOOP-10201](https://issues.apache.org/jira/browse/HADOOP-10201) | *Major* | **Add Listing Support to Key Management APIs** + +I just committed this. Thanks, Larry! + + +--- + +* [HADOOP-8944](https://issues.apache.org/jira/browse/HADOOP-8944) | *Trivial* | **Shell command fs -count should include human readable option** + +Implements -h option for fs -count to show file sizes in human readable format. Additionally, ContentSummary.getHeader() now returns a different string that is incompatible with previous releases. + + +--- + +* [HADOOP-8069](https://issues.apache.org/jira/browse/HADOOP-8069) | *Major* | **Enable TCP\_NODELAY by default for IPC** + +This change enables the TCP\_NODELAY flag for all Hadoop IPC connections, hence bypassing TCP Nagling. Nagling interacts poorly with TCP delayed ACKs especially for request-response protocols. + + +--- + +* [HDFS-7276](https://issues.apache.org/jira/browse/HDFS-7276) | *Major* | **Limit the number of byte arrays used by DFSOutputStream** + +The following configuration properties are added. + +- dfs.client.write.byte-array-manager.enabled: +for enabling/disabling byte array manger. Default is false. + +- dfs.client.write.byte-array-manager.count-threshold: +The count threshold for each array length so that a manager is created only after the allocation count exceeds the threshold. In other words, the particular array length is not managed until the allocation count exceeds the threshold. Default is 128. + +- dfs.client.write.byte-array-manager.count-limit: +The maximum number of arrays allowed for each array length. Default is 2048. + +- dfs.client.write.byte-array-manager.count-reset-time-period-ms: +The time period in milliseconds that the allocation count for each array length is reset to zero if there is no increment. Default is 10,000ms, i.e. 10 seconds. + + +--- + +* [HDFS-7091](https://issues.apache.org/jira/browse/HDFS-7091) | *Minor* | **Add forwarding constructor for INodeFile for existing callers** + +Thanks Nicholas! Revised title and committed to the feature branch. + + +--- + +* [HDFS-7046](https://issues.apache.org/jira/browse/HDFS-7046) | *Critical* | **HA NN can NPE upon transition to active** + +Thanks for the reviews, gentlemen. It's been committed to trunk and branch-2. + + +--- + +* [HDFS-6606](https://issues.apache.org/jira/browse/HDFS-6606) | *Major* | **Optimize HDFS Encrypted Transport performance** + +HDFS now supports the option to configure AES encryption for block data transfer. AES offers improved cryptographic strength and performance over the prior options of 3DES and RC4. + + +--- + +* [HDFS-6482](https://issues.apache.org/jira/browse/HDFS-6482) | *Major* | **Use block ID-based block layout on datanodes** + +The directory structure for finalized replicas on DNs has been changed. Now, the directory that a finalized replica goes in is determined uniquely by its ID. Specifically, we use a two-level directory structure, with the 24th through 17th bits identifying the correct directory at the first level and the 16th through 8th bits identifying the correct directory at the second level. + + +--- + +* [HDFS-6376](https://issues.apache.org/jira/browse/HDFS-6376) | *Major* | **Distcp data between two HA clusters requires another configuration** + +Allow distcp to copy data between HA clusters. Users can use a new configuration property "dfs.internal.nameservices" to explicitly specify the name services belonging to the local cluster, while continue using the configuration property "dfs.nameservices" to specify all the name services in the local and remote clusters. + + +--- + +* [HDFS-2856](https://issues.apache.org/jira/browse/HDFS-2856) | *Major* | **Fix block protocol so that Datanodes don't require root or jsvc** + +SASL now can be used to secure the DataTransferProtocol, which transfers file block content between HDFS clients and DataNodes. In this configuration, it is no longer required for secured clusters to start the DataNode as root and bind to privileged ports. + + +--- + +* [HDFS-573](https://issues.apache.org/jira/browse/HDFS-573) | *Major* | **Porting libhdfs to Windows** + +The libhdfs C API is now supported on Windows. + + +--- + +* [YARN-2830](https://issues.apache.org/jira/browse/YARN-2830) | *Blocker* | **Add backwords compatible ContainerId.newInstance constructor for use within Tez Local Mode** + +I just committed this. Thanks [~jeagles] for the patch and [~ozawa] for the reviews! + + +--- + +* [YARN-2615](https://issues.apache.org/jira/browse/YARN-2615) | *Blocker* | **ClientToAMTokenIdentifier and DelegationTokenIdentifier should allow extended fields** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-1051](https://issues.apache.org/jira/browse/YARN-1051) | *Major* | **YARN Admission Control/Planner: enhancing the resource allocation model with time.** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [YARN-668](https://issues.apache.org/jira/browse/YARN-668) | *Blocker* | **TokenIdentifier serialization should consider Unknown fields** + +**WARNING: No release note provided for this incompatible change.** + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/CHANGES.2.7.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/CHANGES.2.7.0.md new file mode 100644 index 00000000000..81ff61e1e22 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/CHANGES.2.7.0.md @@ -0,0 +1,988 @@ + + +# Apache Hadoop Changelog + +## Release 2.7.0 - 2015-04-20 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11498](https://issues.apache.org/jira/browse/HADOOP-11498) | Bump the version of HTrace to 3.1.0-incubating | Major | . | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11492](https://issues.apache.org/jira/browse/HADOOP-11492) | Bump up curator version to 2.7.1 | Major | . | Karthik Kambatla | Arun Suresh | +| [HADOOP-11385](https://issues.apache.org/jira/browse/HADOOP-11385) | Prevent cross site scripting attack on JMXJSONServlet | Critical | . | Haohui Mai | Haohui Mai | +| [HADOOP-11311](https://issues.apache.org/jira/browse/HADOOP-11311) | Restrict uppercase key names from being created with JCEKS | Major | security | Andrew Wang | Andrew Wang | +| [HADOOP-10530](https://issues.apache.org/jira/browse/HADOOP-10530) | Make hadoop trunk build on Java7+ only | Blocker | build | Steve Loughran | Steve Loughran | +| [HDFS-6651](https://issues.apache.org/jira/browse/HDFS-6651) | Deletion failure can leak inodes permanently | Critical | . | Kihwal Lee | Jing Zhao | +| [HDFS-6252](https://issues.apache.org/jira/browse/HDFS-6252) | Phase out the old web UI in HDFS | Minor | namenode | Fengdong Yu | Haohui Mai | +| [YARN-3217](https://issues.apache.org/jira/browse/YARN-3217) | Remove httpclient dependency from hadoop-yarn-server-web-proxy | Major | . | Akira AJISAKA | Brahma Reddy Battula | +| [YARN-3154](https://issues.apache.org/jira/browse/YARN-3154) | Should not upload partial logs for MR jobs or other "short-running' applications | Blocker | nodemanager, resourcemanager | Xuan Gong | Xuan Gong | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11510](https://issues.apache.org/jira/browse/HADOOP-11510) | Expose truncate API via FileContext | Major | fs | Yi Liu | Yi Liu | +| [HADOOP-11490](https://issues.apache.org/jira/browse/HADOOP-11490) | Expose truncate API via FileSystem and shell command | Major | fs | Konstantin Shvachko | Milan Desai | +| [HADOOP-11341](https://issues.apache.org/jira/browse/HADOOP-11341) | KMS support for whitelist key ACLs | Major | kms, security | Arun Suresh | Arun Suresh | +| [HADOOP-10728](https://issues.apache.org/jira/browse/HADOOP-10728) | Metrics system for Windows Azure Storage Filesystem | Major | tools | Mike Liddell | Mike Liddell | +| [HADOOP-9629](https://issues.apache.org/jira/browse/HADOOP-9629) | Support Windows Azure Storage - Blob as a file system in Hadoop | Major | tools | Mostafa Elhemali | Chris Nauroth | +| [HADOOP-8989](https://issues.apache.org/jira/browse/HADOOP-8989) | hadoop fs -find feature | Major | . | Marco Nicosia | Jonathan Allen | +| [HADOOP-7984](https://issues.apache.org/jira/browse/HADOOP-7984) | Add hadoop --loglevel option to change log level | Minor | scripts | Eli Collins | Akira AJISAKA | +| [HDFS-7584](https://issues.apache.org/jira/browse/HDFS-7584) | Enable Quota Support for Storage Types | Major | datanode, namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7449](https://issues.apache.org/jira/browse/HDFS-7449) | Add metrics to NFS gateway | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7424](https://issues.apache.org/jira/browse/HDFS-7424) | Add web UI for NFS gateway | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7222](https://issues.apache.org/jira/browse/HDFS-7222) | Expose DataNode network errors as a metric | Minor | datanode | Charles Lamb | Charles Lamb | +| [HDFS-6982](https://issues.apache.org/jira/browse/HDFS-6982) | nntop: top­-like tool for name node users | Major | . | Maysam Yabandeh | Maysam Yabandeh | +| [HDFS-6826](https://issues.apache.org/jira/browse/HDFS-6826) | Plugin interface to enable delegation of HDFS authorization assertions | Major | security | Alejandro Abdelnur | Arun Suresh | +| [HDFS-6663](https://issues.apache.org/jira/browse/HDFS-6663) | Admin command to track file and locations from block id | Major | . | Kihwal Lee | Chen He | +| [HDFS-6488](https://issues.apache.org/jira/browse/HDFS-6488) | Support HDFS superuser in NFSv3 gateway | Major | nfs | Stephen Chu | Brandon Li | +| [HDFS-3689](https://issues.apache.org/jira/browse/HDFS-3689) | Add support for variable length block | Major | datanode, hdfs-client, namenode | Suresh Srinivas | Jing Zhao | +| [HDFS-3107](https://issues.apache.org/jira/browse/HDFS-3107) | HDFS truncate | Major | datanode, namenode | Lei Chang | Plamen Jeliazkov | +| [HDFS-1362](https://issues.apache.org/jira/browse/HDFS-1362) | Provide volume management functionality for DataNode | Major | datanode | Wang Xu | Wang Xu | +| [MAPREDUCE-6228](https://issues.apache.org/jira/browse/MAPREDUCE-6228) | Add truncate operation to SLive | Major | benchmarks, test | Konstantin Shvachko | Plamen Jeliazkov | +| [MAPREDUCE-6227](https://issues.apache.org/jira/browse/MAPREDUCE-6227) | DFSIO for truncate | Major | benchmarks, test | Konstantin Shvachko | Konstantin Shvachko | +| [YARN-2837](https://issues.apache.org/jira/browse/YARN-2837) | Timeline server needs to recover the timeline DT when restarting | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-2574](https://issues.apache.org/jira/browse/YARN-2574) | Add support for FairScheduler to the ReservationSystem | Major | fairscheduler | Subru Krishnan | Anubhav Dhoot | +| [YARN-2427](https://issues.apache.org/jira/browse/YARN-2427) | Add support for moving apps between queues in RM web services | Major | resourcemanager | Varun Vasudev | Varun Vasudev | +| [YARN-2360](https://issues.apache.org/jira/browse/YARN-2360) | Fair Scheduler: Display dynamic fair share for queues on the scheduler page | Major | fairscheduler | Ashwin Shankar | Ashwin Shankar | +| [YARN-2190](https://issues.apache.org/jira/browse/YARN-2190) | Add CPU and memory limit options to the default container executor for Windows containers | Major | nodemanager | Chuan Liu | Chuan Liu | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11801](https://issues.apache.org/jira/browse/HADOOP-11801) | Update BUILDING.txt for Ubuntu | Minor | documentation | Gabor Liptak | Gabor Liptak | +| [HADOOP-11714](https://issues.apache.org/jira/browse/HADOOP-11714) | Add more trace log4j messages to SpanReceiverHost | Minor | tracing | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11658](https://issues.apache.org/jira/browse/HADOOP-11658) | Externalize io.compression.codecs property | Minor | . | Kai Zheng | Kai Zheng | +| [HADOOP-11648](https://issues.apache.org/jira/browse/HADOOP-11648) | Set DomainSocketWatcher thread name explicitly | Major | net | Liang Xie | Liang Xie | +| [HADOOP-11642](https://issues.apache.org/jira/browse/HADOOP-11642) | Upgrade azure sdk version from 0.6.0 to 2.0.0 | Major | tools | shashank | shashank | +| [HADOOP-11633](https://issues.apache.org/jira/browse/HADOOP-11633) | Convert remaining branch-2 .apt.vm files to markdown | Major | documentation | Colin Patrick McCabe | Masatake Iwasaki | +| [HADOOP-11632](https://issues.apache.org/jira/browse/HADOOP-11632) | Cleanup Find.java to remove SupressWarnings annotations | Minor | . | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-11620](https://issues.apache.org/jira/browse/HADOOP-11620) | Add support for load balancing across a group of KMS for HA | Major | kms | Arun Suresh | Arun Suresh | +| [HADOOP-11607](https://issues.apache.org/jira/browse/HADOOP-11607) | Reduce log spew in S3AFileSystem | Trivial | fs/s3 | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HADOOP-11599](https://issues.apache.org/jira/browse/HADOOP-11599) | Client#getTimeout should use IPC\_CLIENT\_PING\_DEFAULT when IPC\_CLIENT\_PING\_KEY is not configured. | Minor | ipc | zhihai xu | zhihai xu | +| [HADOOP-11589](https://issues.apache.org/jira/browse/HADOOP-11589) | NetUtils.createSocketAddr should trim the input URI | Minor | net | Akira AJISAKA | Rakesh R | +| [HADOOP-11586](https://issues.apache.org/jira/browse/HADOOP-11586) | Update use of Iterator to Iterable in AbstractMetricsContext.java | Minor | metrics | Ray Chiang | Ray Chiang | +| [HADOOP-11579](https://issues.apache.org/jira/browse/HADOOP-11579) | Documentation for truncate | Major | documentation | Steve Loughran | Konstantin Shvachko | +| [HADOOP-11569](https://issues.apache.org/jira/browse/HADOOP-11569) | Provide Merge API for MapFile to merge multiple similar MapFiles to one MapFile | Major | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-11544](https://issues.apache.org/jira/browse/HADOOP-11544) | Remove unused configuration keys for tracing | Trivial | . | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11543](https://issues.apache.org/jira/browse/HADOOP-11543) | Improve help message for hadoop/yarn command | Trivial | scripts | Jagadesh Kiran N | Brahma Reddy Battula | +| [HADOOP-11520](https://issues.apache.org/jira/browse/HADOOP-11520) | Clean incomplete multi-part uploads in S3A tests | Minor | fs/s3 | Thomas Demoor | Thomas Demoor | +| [HADOOP-11506](https://issues.apache.org/jira/browse/HADOOP-11506) | Configuration variable expansion regex expensive for long values | Major | conf | Dmitriy V. Ryaboy | Gera Shegalov | +| [HADOOP-11495](https://issues.apache.org/jira/browse/HADOOP-11495) | Convert site documentation from apt to markdown | Major | documentation | Allen Wittenauer | Masatake Iwasaki | +| [HADOOP-11483](https://issues.apache.org/jira/browse/HADOOP-11483) | HardLink.java should use the jdk7 createLink method | Major | . | Colin Patrick McCabe | Akira AJISAKA | +| [HADOOP-11481](https://issues.apache.org/jira/browse/HADOOP-11481) | ClassCastException while using a key created by keytool to create encryption zone. | Minor | . | Yi Yao | Charles Lamb | +| [HADOOP-11464](https://issues.apache.org/jira/browse/HADOOP-11464) | Reinstate support for launching Hadoop processes on Windows using Cygwin. | Major | bin | Chris Nauroth | Chris Nauroth | +| [HADOOP-11455](https://issues.apache.org/jira/browse/HADOOP-11455) | KMS and Credential CLI should request confirmation for deletion by default | Minor | security | Charles Lamb | Charles Lamb | +| [HADOOP-11448](https://issues.apache.org/jira/browse/HADOOP-11448) | Fix findbugs warnings in FileBasedIPList | Minor | . | Akira AJISAKA | Tsuyoshi Ozawa | +| [HADOOP-11442](https://issues.apache.org/jira/browse/HADOOP-11442) | hadoop-azure: Create test jar | Major | tools | shashank | shashank | +| [HADOOP-11441](https://issues.apache.org/jira/browse/HADOOP-11441) | Hadoop-azure: Change few methods scope to public | Minor | tools | shashank | shashank | +| [HADOOP-11440](https://issues.apache.org/jira/browse/HADOOP-11440) | Use "test.build.data" instead of "build.test.dir" for testing in ClientBaseWithFixes | Minor | . | Akira AJISAKA | Kengo Seki | +| [HADOOP-11430](https://issues.apache.org/jira/browse/HADOOP-11430) | Add GenericTestUtils#disableLog, GenericTestUtils#setLogLevel | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11427](https://issues.apache.org/jira/browse/HADOOP-11427) | ChunkedArrayList: fix removal via iterator and implement get | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11422](https://issues.apache.org/jira/browse/HADOOP-11422) | Check CryptoCodec is AES-CTR for Crypto input/output stream | Minor | . | Yi Liu | Yi Liu | +| [HADOOP-11421](https://issues.apache.org/jira/browse/HADOOP-11421) | Add IOUtils#listDirectory | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11419](https://issues.apache.org/jira/browse/HADOOP-11419) | improve hadoop-maven-plugins | Minor | build | HervĂ© Boutemy | HervĂ© Boutemy | +| [HADOOP-11416](https://issues.apache.org/jira/browse/HADOOP-11416) | Move ChunkedArrayList into hadoop-common | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11410](https://issues.apache.org/jira/browse/HADOOP-11410) | make the rpath of libhadoop.so configurable | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11399](https://issues.apache.org/jira/browse/HADOOP-11399) | Java Configuration file and .xml files should be automatically cross-compared | Minor | . | Ray Chiang | Ray Chiang | +| [HADOOP-11396](https://issues.apache.org/jira/browse/HADOOP-11396) | Provide navigation in the site documentation linking to the Hadoop Compatible File Systems. | Major | documentation | Chris Nauroth | Chris Nauroth | +| [HADOOP-11395](https://issues.apache.org/jira/browse/HADOOP-11395) | Add site documentation for Azure Storage FileSystem integration. | Major | documentation | Chris Nauroth | Chris Nauroth | +| [HADOOP-11390](https://issues.apache.org/jira/browse/HADOOP-11390) | Metrics 2 ganglia provider to include hostname in unresolved address problems | Minor | metrics | Steve Loughran | Varun Saxena | +| [HADOOP-11323](https://issues.apache.org/jira/browse/HADOOP-11323) | WritableComparator#compare keeps reference to byte array | Major | performance | Wilfred Spiegelenburg | Wilfred Spiegelenburg | +| [HADOOP-11313](https://issues.apache.org/jira/browse/HADOOP-11313) | Adding a document about NativeLibraryChecker | Major | documentation | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-11301](https://issues.apache.org/jira/browse/HADOOP-11301) | [optionally] update jmx cache to drop old metrics | Major | . | Maysam Yabandeh | Maysam Yabandeh | +| [HADOOP-11291](https://issues.apache.org/jira/browse/HADOOP-11291) | Log the cause of SASL connection failures | Minor | security | Stephen Chu | Stephen Chu | +| [HADOOP-11261](https://issues.apache.org/jira/browse/HADOOP-11261) | Set custom endpoint for S3A | Major | fs/s3 | Thomas Demoor | Thomas Demoor | +| [HADOOP-11257](https://issues.apache.org/jira/browse/HADOOP-11257) | Update "hadoop jar" documentation to warn against using it for launching yarn jars | Blocker | . | Allen Wittenauer | Masatake Iwasaki | +| [HADOOP-11231](https://issues.apache.org/jira/browse/HADOOP-11231) | Remove dead code in ServletUtil | Minor | . | Haohui Mai | Li Lu | +| [HADOOP-11188](https://issues.apache.org/jira/browse/HADOOP-11188) | hadoop-azure: automatically expand page blobs when they become full | Major | fs | Eric Hanson | Eric Hanson | +| [HADOOP-11173](https://issues.apache.org/jira/browse/HADOOP-11173) | Improve error messages for some KeyShell commands | Minor | . | Andrew Wang | Andrew Wang | +| [HADOOP-11172](https://issues.apache.org/jira/browse/HADOOP-11172) | Improve error message in Shell#runCommand on OutOfMemoryError | Major | . | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-11171](https://issues.apache.org/jira/browse/HADOOP-11171) | Enable using a proxy server to connect to S3a. | Major | fs/s3 | Thomas Demoor | Thomas Demoor | +| [HADOOP-11045](https://issues.apache.org/jira/browse/HADOOP-11045) | Introducing a tool to detect flaky tests of hadoop jenkins test job | Major | build, tools | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-11032](https://issues.apache.org/jira/browse/HADOOP-11032) | Replace use of Guava's Stopwatch with Hadoop's StopWatch | Major | . | Gary Steelman | Tsuyoshi Ozawa | +| [HADOOP-10987](https://issues.apache.org/jira/browse/HADOOP-10987) | Provide an iterator-based listing API for FileSystem | Major | . | Kihwal Lee | Kihwal Lee | +| [HADOOP-10976](https://issues.apache.org/jira/browse/HADOOP-10976) | moving the source code of hadoop-tools docs to the directory under hadoop-tools | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-10847](https://issues.apache.org/jira/browse/HADOOP-10847) | Remove the usage of sun.security.x509.\* in testing code | Minor | security | Kai Zheng | pascal oliva | +| [HADOOP-10809](https://issues.apache.org/jira/browse/HADOOP-10809) | hadoop-azure: page blob support | Major | tools | Mike Liddell | Eric Hanson | +| [HADOOP-10786](https://issues.apache.org/jira/browse/HADOOP-10786) | Fix UGI#reloginFromKeytab on Java 8 | Major | security | Tobi Vollebregt | Stephen Chu | +| [HADOOP-10670](https://issues.apache.org/jira/browse/HADOOP-10670) | Allow AuthenticationFilters to load secret from signature secret files | Minor | security | Kai Zheng | Kai Zheng | +| [HADOOP-10626](https://issues.apache.org/jira/browse/HADOOP-10626) | Limit Returning Attributes for LDAP search | Major | security | Jason Hubbard | Jason Hubbard | +| [HADOOP-10563](https://issues.apache.org/jira/browse/HADOOP-10563) | Remove the dependency of jsp in trunk | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-10525](https://issues.apache.org/jira/browse/HADOOP-10525) | Remove DRFA.MaxBackupIndex config from log4j.properties | Minor | . | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-10140](https://issues.apache.org/jira/browse/HADOOP-10140) | Specification of HADOOP\_CONF\_DIR via the environment in hadoop\_config.cmd | Minor | scripts | Ian Jackson | Kiran Kumar M R | +| [HADOOP-9992](https://issues.apache.org/jira/browse/HADOOP-9992) | Modify the NN loadGenerator to optionally run as a MapReduce job | Major | test | Akshay Radia | Akshay Radia | +| [HADOOP-9869](https://issues.apache.org/jira/browse/HADOOP-9869) | Configuration.getSocketAddr()/getEnum() should use getTrimmed() | Minor | conf | Steve Loughran | Tsuyoshi Ozawa | +| [HADOOP-8757](https://issues.apache.org/jira/browse/HADOOP-8757) | Metrics should disallow names with invalid characters | Minor | metrics | Todd Lipcon | Ray Chiang | +| [HADOOP-8059](https://issues.apache.org/jira/browse/HADOOP-8059) | Add javadoc to InterfaceAudience and InterfaceStability | Major | documentation | Suresh Srinivas | Brandon Li | +| [HADOOP-4297](https://issues.apache.org/jira/browse/HADOOP-4297) | Enable Java assertions when running tests | Major | build | Yoram Kulbak | Tsz Wo Nicholas Sze | +| [HDFS-8071](https://issues.apache.org/jira/browse/HDFS-8071) | Redundant checkFileProgress() in PART II of getAdditionalBlock() | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-7976](https://issues.apache.org/jira/browse/HDFS-7976) | Update NFS user guide for mount option "sync" to minimize or avoid reordered writes | Major | documentation, nfs | Brandon Li | Brandon Li | +| [HDFS-7962](https://issues.apache.org/jira/browse/HDFS-7962) | Remove duplicated logs in BlockManager | Minor | . | Yi Liu | Yi Liu | +| [HDFS-7917](https://issues.apache.org/jira/browse/HDFS-7917) | Use file to replace data dirs in test to simulate a disk failure. | Minor | test | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7898](https://issues.apache.org/jira/browse/HDFS-7898) | Change TestAppendSnapshotTruncate to fail-fast | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7849](https://issues.apache.org/jira/browse/HDFS-7849) | Update documentation for enabling a new feature in rolling upgrade | Minor | documentation | Tsz Wo Nicholas Sze | J.Andreina | +| [HDFS-7832](https://issues.apache.org/jira/browse/HDFS-7832) | Show 'Last Modified' in Namenode's 'Browse Filesystem' | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7797](https://issues.apache.org/jira/browse/HDFS-7797) | Add audit log for setQuota operation | Major | namenode | Rakesh R | Rakesh R | +| [HDFS-7795](https://issues.apache.org/jira/browse/HDFS-7795) | Show warning if not all favored nodes were chosen by namenode | Minor | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7790](https://issues.apache.org/jira/browse/HDFS-7790) | Do not create optional fields in DFSInputStream unless they are needed | Minor | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7789](https://issues.apache.org/jira/browse/HDFS-7789) | DFSck should resolve the path to support cross-FS symlinks | Major | tools | Gera Shegalov | Gera Shegalov | +| [HDFS-7780](https://issues.apache.org/jira/browse/HDFS-7780) | Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo | Minor | . | Ray Chiang | Ray Chiang | +| [HDFS-7773](https://issues.apache.org/jira/browse/HDFS-7773) | Additional metrics in HDFS to be accessed via jmx. | Major | datanode, namenode | Anu Engineer | Anu Engineer | +| [HDFS-7772](https://issues.apache.org/jira/browse/HDFS-7772) | Document hdfs balancer -exclude/-include option in HDFSCommands.html | Trivial | documentation | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7771](https://issues.apache.org/jira/browse/HDFS-7771) | fuse\_dfs should permit FILE: on the front of KRB5CCNAME | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7761](https://issues.apache.org/jira/browse/HDFS-7761) | cleanup unnecssary code logic in LocatedBlock | Minor | . | Yi Liu | Yi Liu | +| [HDFS-7757](https://issues.apache.org/jira/browse/HDFS-7757) | Misleading error messages in FSImage.java | Major | namenode | Arpit Agarwal | Brahma Reddy Battula | +| [HDFS-7752](https://issues.apache.org/jira/browse/HDFS-7752) | Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml | Minor | documentation | Wellington Chevreuil | Wellington Chevreuil | +| [HDFS-7743](https://issues.apache.org/jira/browse/HDFS-7743) | Code cleanup of BlockInfo and rename BlockInfo to BlockInfoContiguous | Minor | namenode | Jing Zhao | Jing Zhao | +| [HDFS-7732](https://issues.apache.org/jira/browse/HDFS-7732) | Fix the order of the parameters in DFSConfigKeys | Trivial | . | Akira AJISAKA | Brahma Reddy Battula | +| [HDFS-7710](https://issues.apache.org/jira/browse/HDFS-7710) | Remove dead code in BackupImage.java | Minor | . | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7706](https://issues.apache.org/jira/browse/HDFS-7706) | Switch BlockManager logging to use slf4j | Minor | namenode | Andrew Wang | Andrew Wang | +| [HDFS-7703](https://issues.apache.org/jira/browse/HDFS-7703) | Support favouredNodes for the append for new blocks | Major | . | Vinayakumar B | Vinayakumar B | +| [HDFS-7694](https://issues.apache.org/jira/browse/HDFS-7694) | FSDataInputStream should support "unbuffer" | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7685](https://issues.apache.org/jira/browse/HDFS-7685) | Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml | Minor | documentation | Frank Lanitz | Kai Sasaki | +| [HDFS-7684](https://issues.apache.org/jira/browse/HDFS-7684) | The host:port settings of the daemons should be trimmed before use | Major | . | Tianyin Xu | Anu Engineer | +| [HDFS-7683](https://issues.apache.org/jira/browse/HDFS-7683) | Combine usages and percent stats in NameNode UI | Minor | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7675](https://issues.apache.org/jira/browse/HDFS-7675) | Remove unused member DFSClient#spanReceiverHost | Trivial | hdfs-client | Konstantin Shvachko | Colin Patrick McCabe | +| [HDFS-7668](https://issues.apache.org/jira/browse/HDFS-7668) | Convert site documentation from apt to markdown | Major | documentation | Allen Wittenauer | Masatake Iwasaki | +| [HDFS-7640](https://issues.apache.org/jira/browse/HDFS-7640) | print NFS Client in the NFS log | Trivial | nfs | Brandon Li | Brandon Li | +| [HDFS-7604](https://issues.apache.org/jira/browse/HDFS-7604) | Track and display failed DataNode storage locations in NameNode. | Major | datanode, namenode | Chris Nauroth | Chris Nauroth | +| [HDFS-7600](https://issues.apache.org/jira/browse/HDFS-7600) | Refine hdfs admin classes to reuse common code | Major | tools | Yi Liu | Jing Zhao | +| [HDFS-7598](https://issues.apache.org/jira/browse/HDFS-7598) | Remove dependency on old version of Guava in TestDFSClientCache#testEviction | Minor | test | Sangjin Lee | Sangjin Lee | +| [HDFS-7591](https://issues.apache.org/jira/browse/HDFS-7591) | hdfs classpath command should support same options as hadoop classpath. | Minor | scripts | Chris Nauroth | Varun Saxena | +| [HDFS-7579](https://issues.apache.org/jira/browse/HDFS-7579) | Improve log reporting during block report rpc failure | Minor | datanode | Charles Lamb | Charles Lamb | +| [HDFS-7564](https://issues.apache.org/jira/browse/HDFS-7564) | NFS gateway dynamically reload UID/GID mapping file /etc/nfs.map | Minor | nfs | Hari Sekhon | Yongjun Zhang | +| [HDFS-7557](https://issues.apache.org/jira/browse/HDFS-7557) | Fix spacing for a few keys in DFSConfigKeys.java | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7555](https://issues.apache.org/jira/browse/HDFS-7555) | Remove the support of unmanaged connectors in HttpServer2 | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7537](https://issues.apache.org/jira/browse/HDFS-7537) | fsck is confusing when dfs.namenode.replication.min \> 1 && missing replicas && NN restart | Major | namenode | Allen Wittenauer | GAO Rui | +| [HDFS-7535](https://issues.apache.org/jira/browse/HDFS-7535) | Utilize Snapshot diff report for distcp | Major | distcp, snapshots | Jing Zhao | Jing Zhao | +| [HDFS-7531](https://issues.apache.org/jira/browse/HDFS-7531) | Improve the concurrent access on FsVolumeList | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7513](https://issues.apache.org/jira/browse/HDFS-7513) | HDFS inotify: add defaultBlockSize to CreateEvent | Major | namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7491](https://issues.apache.org/jira/browse/HDFS-7491) | Add incremental blockreport latency to DN metrics | Minor | datanode | Ming Ma | Ming Ma | +| [HDFS-7484](https://issues.apache.org/jira/browse/HDFS-7484) | Make FSDirectory#addINode take existing INodes as its parameter | Major | . | Haohui Mai | Jing Zhao | +| [HDFS-7478](https://issues.apache.org/jira/browse/HDFS-7478) | Move org.apache.hadoop.hdfs.server.namenode.NNConf to FSNamesystem | Major | . | Li Lu | Li Lu | +| [HDFS-7463](https://issues.apache.org/jira/browse/HDFS-7463) | Simplify FSNamesystem#getBlockLocationsUpdateTimes | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7458](https://issues.apache.org/jira/browse/HDFS-7458) | Add description to the nfs ports in core-site.xml used by nfs test to avoid confusion | Minor | nfs, test | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7454](https://issues.apache.org/jira/browse/HDFS-7454) | Reduce memory footprint for AclEntries in NameNode | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7446](https://issues.apache.org/jira/browse/HDFS-7446) | HDFS inotify should have the ability to determine what txid it has read up to | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7439](https://issues.apache.org/jira/browse/HDFS-7439) | Add BlockOpResponseProto's message to DFSClient's exception message | Minor | balancer & mover, datanode, hdfs-client | Ming Ma | Takanobu Asanuma | +| [HDFS-7435](https://issues.apache.org/jira/browse/HDFS-7435) | PB encoding of block reports is very inefficient | Critical | datanode, namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-7434](https://issues.apache.org/jira/browse/HDFS-7434) | DatanodeID hashCode should not be mutable | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-7430](https://issues.apache.org/jira/browse/HDFS-7430) | Rewrite the BlockScanner to use O(1) memory and use multiple threads | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7426](https://issues.apache.org/jira/browse/HDFS-7426) | Change nntop JMX format to be a JSON blob | Major | namenode | Andrew Wang | Andrew Wang | +| [HDFS-7419](https://issues.apache.org/jira/browse/HDFS-7419) | Improve error messages for DataNode hot swap drive feature | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7411](https://issues.apache.org/jira/browse/HDFS-7411) | Refactor and improve decommissioning logic into DecommissionManager | Major | . | Andrew Wang | Andrew Wang | +| [HDFS-7410](https://issues.apache.org/jira/browse/HDFS-7410) | Support CreateFlags with append() to support hsync() for appending streams | Major | hdfs-client | Vinayakumar B | Vinayakumar B | +| [HDFS-7409](https://issues.apache.org/jira/browse/HDFS-7409) | Allow dead nodes to finish decommissioning if all files are fully replicated | Minor | . | Andrew Wang | Andrew Wang | +| [HDFS-7404](https://issues.apache.org/jira/browse/HDFS-7404) | Remove o.a.h.hdfs.server.datanode.web.resources | Major | . | Haohui Mai | Li Lu | +| [HDFS-7398](https://issues.apache.org/jira/browse/HDFS-7398) | Reset cached thread-local FSEditLogOp's on every FSEditLog#logEdit | Major | namenode | Gera Shegalov | Gera Shegalov | +| [HDFS-7386](https://issues.apache.org/jira/browse/HDFS-7386) | Replace check "port number \< 1024" with shared isPrivilegedPort method | Trivial | datanode, security | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7384](https://issues.apache.org/jira/browse/HDFS-7384) | 'getfacl' command and 'getAclStatus' output should be in sync | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7381](https://issues.apache.org/jira/browse/HDFS-7381) | Decouple the management of block id and gen stamps from FSNamesystem | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7375](https://issues.apache.org/jira/browse/HDFS-7375) | Move FSClusterStats to o.a.h.h.hdfs.server.blockmanagement | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7365](https://issues.apache.org/jira/browse/HDFS-7365) | Remove hdfs.server.blockmanagement.MutableBlockCollection | Minor | . | Li Lu | Li Lu | +| [HDFS-7357](https://issues.apache.org/jira/browse/HDFS-7357) | FSNamesystem.checkFileProgress should log file path | Minor | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7356](https://issues.apache.org/jira/browse/HDFS-7356) | Use DirectoryListing.hasMore() directly in nfs | Minor | nfs | Haohui Mai | Li Lu | +| [HDFS-7336](https://issues.apache.org/jira/browse/HDFS-7336) | Unused member DFSInputStream.buffersize | Major | hdfs-client | Konstantin Shvachko | Milan Desai | +| [HDFS-7335](https://issues.apache.org/jira/browse/HDFS-7335) | Redundant checkOperation() in FSN.analyzeFileState() | Major | namenode | Konstantin Shvachko | Milan Desai | +| [HDFS-7333](https://issues.apache.org/jira/browse/HDFS-7333) | Improve log message in Storage.tryLock() | Major | datanode, namenode | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-7331](https://issues.apache.org/jira/browse/HDFS-7331) | Add Datanode network counts to datanode jmx page | Minor | datanode | Charles Lamb | Charles Lamb | +| [HDFS-7329](https://issues.apache.org/jira/browse/HDFS-7329) | MiniDFSCluster should log the exception when createNameNodesAndSetConf() fails. | Major | test | Konstantin Shvachko | Byron Wong | +| [HDFS-7326](https://issues.apache.org/jira/browse/HDFS-7326) | Add documentation for hdfs debug commands | Minor | documentation | Colin Patrick McCabe | Vijay Bhat | +| [HDFS-7323](https://issues.apache.org/jira/browse/HDFS-7323) | Move the get/setStoragePolicy commands out from dfsadmin | Major | hdfs-client | Tsz Wo Nicholas Sze | Jing Zhao | +| [HDFS-7308](https://issues.apache.org/jira/browse/HDFS-7308) | DFSClient write packet size may \> 64kB | Minor | hdfs-client | Tsz Wo Nicholas Sze | Takuya Fukudome | +| [HDFS-7283](https://issues.apache.org/jira/browse/HDFS-7283) | Bump DataNode OOM log from WARN to ERROR | Trivial | datanode | Stephen Chu | Stephen Chu | +| [HDFS-7280](https://issues.apache.org/jira/browse/HDFS-7280) | Use netty 4 in WebImageViewer | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7279](https://issues.apache.org/jira/browse/HDFS-7279) | Use netty to implement DatanodeWebHdfsMethods | Major | datanode, webhdfs | Haohui Mai | Haohui Mai | +| [HDFS-7278](https://issues.apache.org/jira/browse/HDFS-7278) | Add a command that allows sysadmins to manually trigger full block reports from a DN | Major | datanode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7270](https://issues.apache.org/jira/browse/HDFS-7270) | Add congestion signaling capability to DataNode write protocol | Major | datanode | Haohui Mai | Haohui Mai | +| [HDFS-7266](https://issues.apache.org/jira/browse/HDFS-7266) | HDFS Peercache enabled check should not lock on object | Minor | hdfs-client | Gopal V | Andrew Wang | +| [HDFS-7257](https://issues.apache.org/jira/browse/HDFS-7257) | Add the time of last HA state transition to NN's /jmx page | Minor | namenode | Charles Lamb | Charles Lamb | +| [HDFS-7252](https://issues.apache.org/jira/browse/HDFS-7252) | small refinement to the use of isInAnEZ in FSNamesystem | Trivial | . | Yi Liu | Yi Liu | +| [HDFS-7242](https://issues.apache.org/jira/browse/HDFS-7242) | Code improvement for FSN#checkUnreadableBySuperuser | Minor | namenode | Yi Liu | Yi Liu | +| [HDFS-7223](https://issues.apache.org/jira/browse/HDFS-7223) | Tracing span description of IPC client is too long | Minor | . | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-7210](https://issues.apache.org/jira/browse/HDFS-7210) | Avoid two separate RPC's namenode.append() and namenode.getFileInfo() for an append call from DFSClient | Major | hdfs-client, namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7190](https://issues.apache.org/jira/browse/HDFS-7190) | Bad use of Preconditions in startFileInternal() | Major | namenode | Konstantin Shvachko | Dawson Choong | +| [HDFS-7186](https://issues.apache.org/jira/browse/HDFS-7186) | Document the "hadoop trace" command. | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-7182](https://issues.apache.org/jira/browse/HDFS-7182) | JMX metrics aren't accessible when NN is busy | Major | . | Ming Ma | Ming Ma | +| [HDFS-7165](https://issues.apache.org/jira/browse/HDFS-7165) | Separate block metrics for files with replication count 1 | Major | namenode | Andrew Wang | Zhe Zhang | +| [HDFS-7026](https://issues.apache.org/jira/browse/HDFS-7026) | Introduce a string constant for "Failed to obtain user group info..." | Trivial | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-6806](https://issues.apache.org/jira/browse/HDFS-6806) | HDFS Rolling upgrade document should mention the versions available | Minor | documentation | Akira AJISAKA | J.Andreina | +| [HDFS-6741](https://issues.apache.org/jira/browse/HDFS-6741) | Improve permission denied message when FSPermissionChecker#checkOwner fails | Trivial | . | Stephen Chu | Harsh J | +| [HDFS-6735](https://issues.apache.org/jira/browse/HDFS-6735) | A minor optimization to avoid pread() be blocked by read() inside the same DFSInputStream | Major | hdfs-client | Liang Xie | Lars Hofhansl | +| [HDFS-6565](https://issues.apache.org/jira/browse/HDFS-6565) | Use jackson instead jetty json in hdfs-client | Major | . | Haohui Mai | Akira AJISAKA | +| [HDFS-6133](https://issues.apache.org/jira/browse/HDFS-6133) | Make Balancer support exclude specified path | Major | balancer & mover, datanode | zhaoyunjiong | zhaoyunjiong | +| [HDFS-5853](https://issues.apache.org/jira/browse/HDFS-5853) | Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml | Minor | documentation, namenode | Akira AJISAKA | Akira AJISAKA | +| [HDFS-3342](https://issues.apache.org/jira/browse/HDFS-3342) | SocketTimeoutException in BlockSender.sendChunks could have a better error message | Minor | datanode | Todd Lipcon | Yongjun Zhang | +| [HDFS-2219](https://issues.apache.org/jira/browse/HDFS-2219) | Fsck should work with fully qualified file paths. | Minor | tools | Jitendra Nath Pandey | Tsz Wo Nicholas Sze | +| [HDFS-316](https://issues.apache.org/jira/browse/HDFS-316) | Balancer should run for a configurable # of iterations | Minor | balancer & mover | Brian Bockelman | Xiaoyu Yao | +| [MAPREDUCE-6292](https://issues.apache.org/jira/browse/MAPREDUCE-6292) | Use org.junit package instead of junit.framework in TestCombineFileInputFormat | Minor | test | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-6267](https://issues.apache.org/jira/browse/MAPREDUCE-6267) | Refactor JobSubmitter#copyAndConfigureFiles into it's own class | Minor | . | Chris Trezzo | Chris Trezzo | +| [MAPREDUCE-6265](https://issues.apache.org/jira/browse/MAPREDUCE-6265) | Make ContainerLauncherImpl.INITIAL\_POOL\_SIZE configurable to better control to launch/kill containers | Major | mrv2 | zhihai xu | zhihai xu | +| [MAPREDUCE-6256](https://issues.apache.org/jira/browse/MAPREDUCE-6256) | Removed unused private methods in o.a.h.mapreduce.Job.java | Minor | . | Devaraj K | Naganarasimha G R | +| [MAPREDUCE-6255](https://issues.apache.org/jira/browse/MAPREDUCE-6255) | Fix JobCounter's format to use grouping separator | Minor | client | Ryu Kobayashi | Ryu Kobayashi | +| [MAPREDUCE-6253](https://issues.apache.org/jira/browse/MAPREDUCE-6253) | Update use of Iterator to Iterable | Minor | . | Ray Chiang | Ray Chiang | +| [MAPREDUCE-6248](https://issues.apache.org/jira/browse/MAPREDUCE-6248) | Allow users to get the MR job information for distcp | Major | distcp | Jing Zhao | Jing Zhao | +| [MAPREDUCE-6194](https://issues.apache.org/jira/browse/MAPREDUCE-6194) | Bubble up final exception in failures during creation of output collectors | Minor | task | Harsh J | Varun Saxena | +| [MAPREDUCE-6173](https://issues.apache.org/jira/browse/MAPREDUCE-6173) | Document the configuration of deploying MR over distributed cache with enabling wired encryption at the same time | Major | distributed-cache, documentation | Junping Du | Junping Du | +| [MAPREDUCE-6169](https://issues.apache.org/jira/browse/MAPREDUCE-6169) | MergeQueue should release reference to the current item from key and value at the end of the iteration to save memory. | Major | mrv2 | zhihai xu | zhihai xu | +| [MAPREDUCE-6151](https://issues.apache.org/jira/browse/MAPREDUCE-6151) | Update document of GridMix | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [MAPREDUCE-6150](https://issues.apache.org/jira/browse/MAPREDUCE-6150) | Update document of Rumen | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [MAPREDUCE-6149](https://issues.apache.org/jira/browse/MAPREDUCE-6149) | Document override log4j.properties in MR job | Major | documentation | Junping Du | Junping Du | +| [MAPREDUCE-6143](https://issues.apache.org/jira/browse/MAPREDUCE-6143) | add configuration for mapreduce speculative execution in MR2 | Major | mrv2 | zhihai xu | zhihai xu | +| [MAPREDUCE-6141](https://issues.apache.org/jira/browse/MAPREDUCE-6141) | History server leveldb recovery store | Major | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6059](https://issues.apache.org/jira/browse/MAPREDUCE-6059) | Speed up history server startup time | Major | . | Siqi Li | Siqi Li | +| [MAPREDUCE-6046](https://issues.apache.org/jira/browse/MAPREDUCE-6046) | Change the class name for logs in RMCommunicator.java | Minor | mr-am | Devaraj K | Sahil Takiar | +| [MAPREDUCE-5932](https://issues.apache.org/jira/browse/MAPREDUCE-5932) | Provide an option to use a dedicated reduce-side shuffle log | Major | mrv2 | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-5800](https://issues.apache.org/jira/browse/MAPREDUCE-5800) | Use Job#getInstance instead of deprecated constructors | Minor | . | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5612](https://issues.apache.org/jira/browse/MAPREDUCE-5612) | Add javadoc for TaskCompletionEvent.Status | Minor | documentation | Sandy Ryza | Chris Palmer | +| [MAPREDUCE-5583](https://issues.apache.org/jira/browse/MAPREDUCE-5583) | Ability to limit running map and reduce tasks | Major | mr-am, mrv2 | Jason Lowe | Jason Lowe | +| [MAPREDUCE-5335](https://issues.apache.org/jira/browse/MAPREDUCE-5335) | Rename Job Tracker terminology in ShuffleSchedulerImpl | Major | applicationmaster | Devaraj K | Devaraj K | +| [MAPREDUCE-4815](https://issues.apache.org/jira/browse/MAPREDUCE-4815) | Speed up FileOutputCommitter#commitJob for many output files | Major | mrv2 | Jason Lowe | Siqi Li | +| [MAPREDUCE-4431](https://issues.apache.org/jira/browse/MAPREDUCE-4431) | mapred command should print the reason on killing already completed jobs | Minor | mrv2 | Nishan Shetty | Devaraj K | +| [YARN-3349](https://issues.apache.org/jira/browse/YARN-3349) | Treat all exceptions as failure in TestFSRMStateStore#testFSRMStateStoreClientRetry | Minor | test | zhihai xu | zhihai xu | +| [YARN-3285](https://issues.apache.org/jira/browse/YARN-3285) | Convert branch-2 .apt.vm files of YARN to markdown | Major | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [YARN-3273](https://issues.apache.org/jira/browse/YARN-3273) | Improve web UI to facilitate scheduling analysis and debugging | Major | . | Jian He | Rohith Sharma K S | +| [YARN-3272](https://issues.apache.org/jira/browse/YARN-3272) | Surface container locality info in RM web UI | Major | . | Jian He | Jian He | +| [YARN-3262](https://issues.apache.org/jira/browse/YARN-3262) | Surface application outstanding resource requests table in RM web UI | Major | yarn | Jian He | Jian He | +| [YARN-3249](https://issues.apache.org/jira/browse/YARN-3249) | Add a "kill application" button to Resource Manager's Web UI | Minor | resourcemanager | Ryu Kobayashi | Ryu Kobayashi | +| [YARN-3236](https://issues.apache.org/jira/browse/YARN-3236) | cleanup RMAuthenticationFilter#AUTH\_HANDLER\_PROPERTY. | Trivial | resourcemanager | zhihai xu | zhihai xu | +| [YARN-3230](https://issues.apache.org/jira/browse/YARN-3230) | Clarify application states on the web UI | Major | . | Jian He | Jian He | +| [YARN-3203](https://issues.apache.org/jira/browse/YARN-3203) | Correct a log message in AuxServices | Minor | . | Brahma Reddy Battula | Brahma Reddy Battula | +| [YARN-3195](https://issues.apache.org/jira/browse/YARN-3195) | Add -help to yarn logs and nodes CLI command | Minor | client | Jagadesh Kiran N | Jagadesh Kiran N | +| [YARN-3187](https://issues.apache.org/jira/browse/YARN-3187) | Documentation of Capacity Scheduler Queue mapping based on user or group | Major | capacityscheduler, documentation | Naganarasimha G R | Gururaj Shetty | +| [YARN-3182](https://issues.apache.org/jira/browse/YARN-3182) | Cleanup switch statement in ApplicationMasterLauncher#handle() | Minor | . | Ray Chiang | Ray Chiang | +| [YARN-3179](https://issues.apache.org/jira/browse/YARN-3179) | Update use of Iterator to Iterable | Minor | . | Ray Chiang | Ray Chiang | +| [YARN-3158](https://issues.apache.org/jira/browse/YARN-3158) | Correct log messages in ResourceTrackerService | Major | . | Devaraj K | Varun Saxena | +| [YARN-3157](https://issues.apache.org/jira/browse/YARN-3157) | Refactor the exception handling in ConverterUtils#to\*Id | Minor | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | +| [YARN-3147](https://issues.apache.org/jira/browse/YARN-3147) | Clean up RM web proxy code | Major | webapp | Steve Loughran | Steve Loughran | +| [YARN-3144](https://issues.apache.org/jira/browse/YARN-3144) | Configuration for making delegation token failures to timeline server not-fatal | Major | . | Jonathan Eagles | Jonathan Eagles | +| [YARN-3123](https://issues.apache.org/jira/browse/YARN-3123) | Make YARN CLI show a single completed container even if the app is running | Major | client | Zhijie Shen | Naganarasimha G R | +| [YARN-3108](https://issues.apache.org/jira/browse/YARN-3108) | ApplicationHistoryServer doesn't process -D arguments | Major | . | Chang Li | Chang Li | +| [YARN-3100](https://issues.apache.org/jira/browse/YARN-3100) | Make YARN authorization pluggable | Major | . | Jian He | Jian He | +| [YARN-3086](https://issues.apache.org/jira/browse/YARN-3086) | Make NodeManager memory configurable in MiniYARNCluster | Minor | test | Robert Metzger | Robert Metzger | +| [YARN-3085](https://issues.apache.org/jira/browse/YARN-3085) | Application summary should include the application type | Major | resourcemanager | Jason Lowe | Rohith Sharma K S | +| [YARN-3077](https://issues.apache.org/jira/browse/YARN-3077) | RM should create yarn.resourcemanager.zk-state-store.parent-path recursively | Major | resourcemanager | Chun Chen | Chun Chen | +| [YARN-3056](https://issues.apache.org/jira/browse/YARN-3056) | add verification for containerLaunchDuration in TestNodeManagerMetrics. | Trivial | test | zhihai xu | zhihai xu | +| [YARN-3022](https://issues.apache.org/jira/browse/YARN-3022) | Expose Container resource information from NodeManager for monitoring | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3005](https://issues.apache.org/jira/browse/YARN-3005) | [JDK7] Use switch statement for String instead of if-else statement in RegistrySecurity.java | Trivial | . | Akira AJISAKA | Kengo Seki | +| [YARN-2996](https://issues.apache.org/jira/browse/YARN-2996) | Refine fs operations in FileSystemRMStateStore and few fixes | Major | resourcemanager | Yi Liu | Yi Liu | +| [YARN-2957](https://issues.apache.org/jira/browse/YARN-2957) | Create unit test to automatically compare YarnConfiguration and yarn-default.xml | Minor | . | Ray Chiang | Ray Chiang | +| [YARN-2950](https://issues.apache.org/jira/browse/YARN-2950) | Change message to mandate, not suggest JS requirement on UI | Minor | webapp | Harsh J | Dustin Cote | +| [YARN-2940](https://issues.apache.org/jira/browse/YARN-2940) | Fix new findbugs warnings in rest of the hadoop-yarn components | Major | . | Varun Saxena | Li Lu | +| [YARN-2939](https://issues.apache.org/jira/browse/YARN-2939) | Fix new findbugs warnings in hadoop-yarn-common | Major | . | Varun Saxena | Li Lu | +| [YARN-2938](https://issues.apache.org/jira/browse/YARN-2938) | Fix new findbugs warnings in hadoop-yarn-resourcemanager and hadoop-yarn-applicationhistoryservice | Major | . | Varun Saxena | Varun Saxena | +| [YARN-2937](https://issues.apache.org/jira/browse/YARN-2937) | Fix new findbugs warnings in hadoop-yarn-nodemanager | Major | . | Varun Saxena | Varun Saxena | +| [YARN-2891](https://issues.apache.org/jira/browse/YARN-2891) | Failed Container Executor does not provide a clear error message | Minor | nodemanager | Dustin Cote | Dustin Cote | +| [YARN-2878](https://issues.apache.org/jira/browse/YARN-2878) | Fix DockerContainerExecutor.apt.vm formatting | Major | documentation | Abin Shahab | Abin Shahab | +| [YARN-2854](https://issues.apache.org/jira/browse/YARN-2854) | The document about timeline service and generic service needs to be updated | Critical | timelineserver | Zhijie Shen | Naganarasimha G R | +| [YARN-2820](https://issues.apache.org/jira/browse/YARN-2820) | Retry in FileSystemRMStateStore when FS's operations fail due to IOException. | Major | resourcemanager | zhihai xu | zhihai xu | +| [YARN-2802](https://issues.apache.org/jira/browse/YARN-2802) | ClusterMetrics to include AM launch and register delays | Major | resourcemanager | zhihai xu | zhihai xu | +| [YARN-2799](https://issues.apache.org/jira/browse/YARN-2799) | cleanup TestLogAggregationService based on the change in YARN-90 | Minor | test | zhihai xu | zhihai xu | +| [YARN-2797](https://issues.apache.org/jira/browse/YARN-2797) | TestWorkPreservingRMRestart should use ParametrizedSchedulerTestBase | Minor | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-2780](https://issues.apache.org/jira/browse/YARN-2780) | Log aggregated resource allocation in rm-appsummary.log | Minor | resourcemanager | Koji Noguchi | Eric Payne | +| [YARN-2777](https://issues.apache.org/jira/browse/YARN-2777) | Mark the end of individual log in aggregated log | Major | . | Ted Yu | Varun Saxena | +| [YARN-2679](https://issues.apache.org/jira/browse/YARN-2679) | Add metric for container launch duration | Major | nodemanager | zhihai xu | zhihai xu | +| [YARN-2669](https://issues.apache.org/jira/browse/YARN-2669) | FairScheduler: queue names shouldn't allow periods | Major | . | Wei Yan | Wei Yan | +| [YARN-2643](https://issues.apache.org/jira/browse/YARN-2643) | Don't create a new DominantResourceCalculator on every FairScheduler.allocate call | Trivial | . | Sandy Ryza | Karthik Kambatla | +| [YARN-2641](https://issues.apache.org/jira/browse/YARN-2641) | Decommission nodes on -refreshNodes instead of next NM-RM heartbeat | Major | resourcemanager | zhihai xu | zhihai xu | +| [YARN-2604](https://issues.apache.org/jira/browse/YARN-2604) | Scheduler should consider max-allocation-\* in conjunction with the largest node | Major | scheduler | Karthik Kambatla | Robert Kanter | +| [YARN-2301](https://issues.apache.org/jira/browse/YARN-2301) | Improve yarn container command | Major | . | Jian He | Naganarasimha G R | +| [YARN-2254](https://issues.apache.org/jira/browse/YARN-2254) | TestRMWebServicesAppsModification should run against both CS and FS | Minor | . | zhihai xu | zhihai xu | +| [YARN-2157](https://issues.apache.org/jira/browse/YARN-2157) | Document YARN metrics | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [YARN-1582](https://issues.apache.org/jira/browse/YARN-1582) | Capacity Scheduler: add a maximum-allocation-mb setting per queue | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-1393](https://issues.apache.org/jira/browse/YARN-1393) | SLS: Add how-to-use instructions | Major | . | Wei Yan | Wei Yan | +| [YARN-1299](https://issues.apache.org/jira/browse/YARN-1299) | Improve a log message in AppSchedulingInfo by adding application id | Major | resourcemanager | Devaraj K | | +| [YARN-1156](https://issues.apache.org/jira/browse/YARN-1156) | Enhance NodeManager AllocatedGB and AvailableGB metrics for aggregation of decimal values | Major | . | Akira AJISAKA | Tsuyoshi Ozawa | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11837](https://issues.apache.org/jira/browse/HADOOP-11837) | AuthenticationFilter should destroy SignerSecretProvider in Tomcat deployments | Blocker | security | Venkat Ranganathan | Bowen Zhang | +| [HADOOP-11815](https://issues.apache.org/jira/browse/HADOOP-11815) | HttpServer2 should destroy SignerSecretProvider when it stops | Blocker | . | Rohith Sharma K S | Rohith Sharma K S | +| [HADOOP-11787](https://issues.apache.org/jira/browse/HADOOP-11787) | OpensslSecureRandom.c pthread\_threadid\_np usage signature is wrong on 32-bit Mac | Critical | native | Colin Patrick McCabe | Kiran Kumar M R | +| [HADOOP-11776](https://issues.apache.org/jira/browse/HADOOP-11776) | jdiff is broken in Hadoop 2 | Blocker | . | Li Lu | Li Lu | +| [HADOOP-11761](https://issues.apache.org/jira/browse/HADOOP-11761) | Fix findbugs warnings in org.apache.hadoop.security.authentication | Minor | . | Li Lu | Li Lu | +| [HADOOP-11757](https://issues.apache.org/jira/browse/HADOOP-11757) | NFS gateway should shutdown when it can't start UDP or TCP server | Major | nfs | Brandon Li | Brandon Li | +| [HADOOP-11754](https://issues.apache.org/jira/browse/HADOOP-11754) | RM fails to start in non-secure mode due to authentication filter failure | Blocker | . | Sangjin Lee | Haohui Mai | +| [HADOOP-11748](https://issues.apache.org/jira/browse/HADOOP-11748) | The secrets of auth cookies should not be specified in configuration in clear text | Critical | . | Haohui Mai | Li Lu | +| [HADOOP-11738](https://issues.apache.org/jira/browse/HADOOP-11738) | Fix a link of Protocol Buffers 2.5 for download in BUILDING.txt | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-11729](https://issues.apache.org/jira/browse/HADOOP-11729) | Fix link to cgroups doc in site.xml | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11722](https://issues.apache.org/jira/browse/HADOOP-11722) | Some Instances of Services using ZKDelegationTokenSecretManager go down when old token cannot be deleted | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11720](https://issues.apache.org/jira/browse/HADOOP-11720) | [JDK8] Fix javadoc errors caused by incorrect or illegal tags in hadoop-tools | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-11693](https://issues.apache.org/jira/browse/HADOOP-11693) | Azure Storage FileSystem rename operations are throttled too aggressively to complete HBase WAL archiving. | Major | tools | Duo Xu | Duo Xu | +| [HADOOP-11691](https://issues.apache.org/jira/browse/HADOOP-11691) | X86 build of libwinutils is broken | Critical | build, native | Remus Rusanu | Kiran Kumar M R | +| [HADOOP-11686](https://issues.apache.org/jira/browse/HADOOP-11686) | MiniKDC cannot change ORG\_NAME or ORG\_DOMAIN | Major | security, test | Duo Zhang | Duo Zhang | +| [HADOOP-11674](https://issues.apache.org/jira/browse/HADOOP-11674) | oneByteBuf in CryptoInputStream and CryptoOutputStream should be non static | Critical | io | Sean Busbey | Sean Busbey | +| [HADOOP-11666](https://issues.apache.org/jira/browse/HADOOP-11666) | Revert the format change of du output introduced by HADOOP-6857 | Major | . | Akira AJISAKA | Byron Wong | +| [HADOOP-11639](https://issues.apache.org/jira/browse/HADOOP-11639) | Clean up Windows native code compilation warnings related to Windows Secure Container Executor. | Major | native | Chris Nauroth | Remus Rusanu | +| [HADOOP-11638](https://issues.apache.org/jira/browse/HADOOP-11638) | OpensslSecureRandom.c pthreads\_thread\_id should support FreeBSD and Solaris in addition to Linux | Major | native | Dmitry Sivachenko | Kiran Kumar M R | +| [HADOOP-11634](https://issues.apache.org/jira/browse/HADOOP-11634) | Description of webhdfs' principal/keytab should switch places each other | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | +| [HADOOP-11629](https://issues.apache.org/jira/browse/HADOOP-11629) | WASB filesystem should not start BandwidthGaugeUpdater if fs.azure.skip.metrics set to true | Major | tools | shanyu zhao | shanyu zhao | +| [HADOOP-11619](https://issues.apache.org/jira/browse/HADOOP-11619) | FTPFileSystem should override getDefaultPort | Major | fs | Gera Shegalov | Brahma Reddy Battula | +| [HADOOP-11618](https://issues.apache.org/jira/browse/HADOOP-11618) | DelegateToFileSystem erroneously uses default FS's port in constructor | Major | fs | Gera Shegalov | Brahma Reddy Battula | +| [HADOOP-11615](https://issues.apache.org/jira/browse/HADOOP-11615) | Update ServiceLevelAuth.md for YARN | Minor | documentation | Akira AJISAKA | Brahma Reddy Battula | +| [HADOOP-11609](https://issues.apache.org/jira/browse/HADOOP-11609) | Correct credential commands info in CommandsManual.html#credential | Major | documentation, security | Brahma Reddy Battula | Varun Saxena | +| [HADOOP-11605](https://issues.apache.org/jira/browse/HADOOP-11605) | FilterFileSystem#create with ChecksumOpt should propagate it to wrapped FS | Minor | fs | Gera Shegalov | Gera Shegalov | +| [HADOOP-11604](https://issues.apache.org/jira/browse/HADOOP-11604) | Prevent ConcurrentModificationException while closing domain sockets during shutdown of DomainSocketWatcher thread. | Critical | net | Liang Xie | Chris Nauroth | +| [HADOOP-11602](https://issues.apache.org/jira/browse/HADOOP-11602) | Fix toUpperCase/toLowerCase to use Locale.ENGLISH | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-11595](https://issues.apache.org/jira/browse/HADOOP-11595) | Add default implementation for AbstractFileSystem#truncate | Major | fs | Yi Liu | Yi Liu | +| [HADOOP-11587](https://issues.apache.org/jira/browse/HADOOP-11587) | TestMapFile#testMainMethodMapFile creates test files in hadoop-common project root | Trivial | test | Xiaoyu Yao | Xiaoyu Yao | +| [HADOOP-11571](https://issues.apache.org/jira/browse/HADOOP-11571) | Über-jira: S3a stabilisation phase I | Blocker | fs/s3 | Steve Loughran | Steve Loughran | +| [HADOOP-11558](https://issues.apache.org/jira/browse/HADOOP-11558) | Fix dead links to doc of hadoop-tools | Minor | documentation | Masatake Iwasaki | Jean-Pierre Matsumoto | +| [HADOOP-11549](https://issues.apache.org/jira/browse/HADOOP-11549) | flaky test detection tool failed to handle special control characters in test result | Major | tools | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-11548](https://issues.apache.org/jira/browse/HADOOP-11548) | checknative should display a nicer error message when openssl support is not compiled in | Major | build, native | Colin Patrick McCabe | Anu Engineer | +| [HADOOP-11547](https://issues.apache.org/jira/browse/HADOOP-11547) | hadoop-common native compilation fails on Windows due to missing support for \_\_attribute\_\_ declaration. | Major | native | Chris Nauroth | Chris Nauroth | +| [HADOOP-11546](https://issues.apache.org/jira/browse/HADOOP-11546) | Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck | Major | build | Steve Loughran | Tsuyoshi Ozawa | +| [HADOOP-11545](https://issues.apache.org/jira/browse/HADOOP-11545) | ArrayIndexOutOfBoundsException is thrown with "hadoop credential list -provider" | Minor | security | Brahma Reddy Battula | Brahma Reddy Battula | +| [HADOOP-11529](https://issues.apache.org/jira/browse/HADOOP-11529) | Fix findbugs warnings in hadoop-archives | Minor | tools | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11526](https://issues.apache.org/jira/browse/HADOOP-11526) | Memory leak in Bzip2Compressor and Bzip2Decompressor | Major | io, native | Ian Rogers | Anu Engineer | +| [HADOOP-11523](https://issues.apache.org/jira/browse/HADOOP-11523) | StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB | Major | tools | Duo Xu | Duo Xu | +| [HADOOP-11512](https://issues.apache.org/jira/browse/HADOOP-11512) | Use getTrimmedStrings when reading serialization keys | Minor | io | Harsh J | Ryan P | +| [HADOOP-11509](https://issues.apache.org/jira/browse/HADOOP-11509) | change parsing sequence in GenericOptionsParser to parse -D parameters first | Major | . | Xuan Gong | Xuan Gong | +| [HADOOP-11507](https://issues.apache.org/jira/browse/HADOOP-11507) | Hadoop RPC Authentication problem with different user locale | Minor | . | Talat UYARER | Talat UYARER | +| [HADOOP-11500](https://issues.apache.org/jira/browse/HADOOP-11500) | InputStream is left unclosed in ApplicationClassLoader | Major | . | Ted Yu | Ted Yu | +| [HADOOP-11499](https://issues.apache.org/jira/browse/HADOOP-11499) | Check of executorThreadsStarted in ValueQueue#submitRefillTask() evades lock acquisition | Minor | . | Ted Yu | Ted Yu | +| [HADOOP-11497](https://issues.apache.org/jira/browse/HADOOP-11497) | Fix typo in ClusterSetup.html#Hadoop\_Startup | Major | documentation | Christian Winkler | Christian Winkler | +| [HADOOP-11494](https://issues.apache.org/jira/browse/HADOOP-11494) | Lock acquisition on WrappedInputStream#unwrappedRpcBuffer may race with another thread | Minor | . | Ted Yu | Ted Yu | +| [HADOOP-11493](https://issues.apache.org/jira/browse/HADOOP-11493) | Fix some typos in kms-acls.xml description | Trivial | kms | Charles Lamb | Charles Lamb | +| [HADOOP-11488](https://issues.apache.org/jira/browse/HADOOP-11488) | Difference in default connection timeout for S3A FS | Minor | fs/s3 | Harsh J | Daisuke Kobayashi | +| [HADOOP-11482](https://issues.apache.org/jira/browse/HADOOP-11482) | Use correct UGI when KMSClientProvider is called by a proxy user | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11480](https://issues.apache.org/jira/browse/HADOOP-11480) | Typo in hadoop-aws/index.md uses wrong scheme for test.fs.s3.name | Minor | documentation | Ted Yu | Ted Yu | +| [HADOOP-11470](https://issues.apache.org/jira/browse/HADOOP-11470) | Remove some uses of obsolete guava APIs from the hadoop codebase | Major | . | Sangjin Lee | Sangjin Lee | +| [HADOOP-11469](https://issues.apache.org/jira/browse/HADOOP-11469) | KMS should skip default.key.acl and whitelist.key.acl when loading key acl | Minor | kms | Dian Fu | Dian Fu | +| [HADOOP-11467](https://issues.apache.org/jira/browse/HADOOP-11467) | KerberosAuthenticator can connect to a non-secure cluster | Critical | security | Robert Kanter | Yongjun Zhang | +| [HADOOP-11462](https://issues.apache.org/jira/browse/HADOOP-11462) | TestSocketIOWithTimeout needs change for PowerPC platform | Major | test | Ayappan | Ayappan | +| [HADOOP-11459](https://issues.apache.org/jira/browse/HADOOP-11459) | Fix recent findbugs in ActiveStandbyElector, NetUtils and ShellBasedIdMapping | Minor | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-11450](https://issues.apache.org/jira/browse/HADOOP-11450) | Cleanup DistCpV1 not to use deprecated methods and fix javadocs | Minor | tools/distcp | Tsuyoshi Ozawa | Varun Saxena | +| [HADOOP-11449](https://issues.apache.org/jira/browse/HADOOP-11449) | [JDK8] Cannot build on Windows: error: unexpected end tag: \ | Major | build | Alec Taylor | Chris Nauroth | +| [HADOOP-11446](https://issues.apache.org/jira/browse/HADOOP-11446) | S3AOutputStream should use shared thread pool to avoid OutOfMemoryError | Major | fs/s3 | Ted Yu | Ted Yu | +| [HADOOP-11445](https://issues.apache.org/jira/browse/HADOOP-11445) | Bzip2Codec: Data block is skipped when position of newly created stream is equal to start of split | Critical | . | Ankit Kamboj | Ankit Kamboj | +| [HADOOP-11431](https://issues.apache.org/jira/browse/HADOOP-11431) | clean up redundant maven-site-plugin configuration | Major | . | HervĂ© Boutemy | HervĂ© Boutemy | +| [HADOOP-11428](https://issues.apache.org/jira/browse/HADOOP-11428) | Remove obsolete reference to Cygwin in BUILDING.txt | Major | documentation | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-11420](https://issues.apache.org/jira/browse/HADOOP-11420) | Use latest maven-site-plugin and replace link to svn with link to git | Trivial | site | HervĂ© Boutemy | HervĂ© Boutemy | +| [HADOOP-11414](https://issues.apache.org/jira/browse/HADOOP-11414) | FileBasedIPList#readLines() can leak file descriptors | Minor | . | Ted Yu | Tsuyoshi Ozawa | +| [HADOOP-11412](https://issues.apache.org/jira/browse/HADOOP-11412) | POMs mention "The Apache Software License" rather than "Apache License" | Trivial | . | HervĂ© Boutemy | HervĂ© Boutemy | +| [HADOOP-11411](https://issues.apache.org/jira/browse/HADOOP-11411) | Hive build failure on hadoop-2.7 due to HADOOP-11356 | Major | . | Jason Dere | | +| [HADOOP-11409](https://issues.apache.org/jira/browse/HADOOP-11409) | FileContext.getFileContext can stack overflow if default fs misconfigured | Major | . | Jason Lowe | Gera Shegalov | +| [HADOOP-11403](https://issues.apache.org/jira/browse/HADOOP-11403) | Avoid using sys\_errlist on Solaris, which lacks support for it | Major | . | Malcolm Kavalsky | Malcolm Kavalsky | +| [HADOOP-11402](https://issues.apache.org/jira/browse/HADOOP-11402) | Negative user-to-group cache entries are never cleared for never-again-accessed users | Major | . | Colin Patrick McCabe | Varun Saxena | +| [HADOOP-11400](https://issues.apache.org/jira/browse/HADOOP-11400) | GraphiteSink does not reconnect to Graphite after 'broken pipe' | Major | metrics | Kamil Gorlo | Kamil Gorlo | +| [HADOOP-11394](https://issues.apache.org/jira/browse/HADOOP-11394) | hadoop-aws documentation missing. | Major | documentation | Chris Nauroth | Chris Nauroth | +| [HADOOP-11388](https://issues.apache.org/jira/browse/HADOOP-11388) | Remove deprecated o.a.h.metrics.file.FileContext | Minor | . | Haohui Mai | Li Lu | +| [HADOOP-11386](https://issues.apache.org/jira/browse/HADOOP-11386) | Replace \n by %n in format hadoop-common format strings | Major | . | Li Lu | Li Lu | +| [HADOOP-11368](https://issues.apache.org/jira/browse/HADOOP-11368) | Fix SSLFactory truststore reloader thread leak in KMSClientProvider | Major | kms | Arun Suresh | Arun Suresh | +| [HADOOP-11363](https://issues.apache.org/jira/browse/HADOOP-11363) | Hadoop maven surefire-plugin uses must set heap size | Major | build | Steve Loughran | Steve Loughran | +| [HADOOP-11355](https://issues.apache.org/jira/browse/HADOOP-11355) | When accessing data in HDFS and the key has been deleted, a Null Pointer Exception is shown. | Minor | . | Arun Suresh | Arun Suresh | +| [HADOOP-11354](https://issues.apache.org/jira/browse/HADOOP-11354) | ThrottledInputStream doesn't perform effective throttling | Major | . | Ted Yu | Ted Yu | +| [HADOOP-11350](https://issues.apache.org/jira/browse/HADOOP-11350) | The size of header buffer of HttpServer is too small when HTTPS is enabled | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-11349](https://issues.apache.org/jira/browse/HADOOP-11349) | RawLocalFileSystem leaks file descriptor while creating a file if creat succeeds but chmod fails. | Minor | fs | Chris Nauroth | Varun Saxena | +| [HADOOP-11348](https://issues.apache.org/jira/browse/HADOOP-11348) | Remove unused variable from CMake error message for finding openssl | Minor | . | Dian Fu | Dian Fu | +| [HADOOP-11344](https://issues.apache.org/jira/browse/HADOOP-11344) | KMS kms-config.sh sets a default value for the keystore password even in non-ssl setup | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11343](https://issues.apache.org/jira/browse/HADOOP-11343) | Overflow is not properly handled in caclulating final iv for AES CTR | Blocker | security | Jerry Chen | Jerry Chen | +| [HADOOP-11342](https://issues.apache.org/jira/browse/HADOOP-11342) | KMS key ACL should ignore ALL operation for default key ACL and whitelist key ACL | Major | kms, security | Dian Fu | Dian Fu | +| [HADOOP-11337](https://issues.apache.org/jira/browse/HADOOP-11337) | KeyAuthorizationKeyProvider access checks need to be done atomically | Major | . | Dian Fu | Dian Fu | +| [HADOOP-11333](https://issues.apache.org/jira/browse/HADOOP-11333) | Fix deadlock in DomainSocketWatcher when the notification pipe is full | Major | . | zhaoyunjiong | zhaoyunjiong | +| [HADOOP-11332](https://issues.apache.org/jira/browse/HADOOP-11332) | KerberosAuthenticator#doSpnegoSequence should check if kerberos TGT is available in the subject | Major | security | Dian Fu | Dian Fu | +| [HADOOP-11329](https://issues.apache.org/jira/browse/HADOOP-11329) | Add JAVA\_LIBRARY\_PATH to KMS startup options | Major | kms, security | Dian Fu | Arun Suresh | +| [HADOOP-11327](https://issues.apache.org/jira/browse/HADOOP-11327) | BloomFilter#not() omits the last bit, resulting in an incorrect filter | Minor | util | Tim Luo | Eric Payne | +| [HADOOP-11322](https://issues.apache.org/jira/browse/HADOOP-11322) | key based ACL check in KMS always check KeyOpType.MANAGEMENT even actual KeyOpType is not MANAGEMENT | Major | security | Dian Fu | Dian Fu | +| [HADOOP-11321](https://issues.apache.org/jira/browse/HADOOP-11321) | copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions. | Major | fs | Chris Nauroth | Chris Nauroth | +| [HADOOP-11318](https://issues.apache.org/jira/browse/HADOOP-11318) | Update the document for hadoop fs -stat | Major | documentation | Akira AJISAKA | Akira AJISAKA | +| [HADOOP-11316](https://issues.apache.org/jira/browse/HADOOP-11316) | "mvn package -Pdist,docs -DskipTests -Dtar" fails because of non-ascii characters | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-11312](https://issues.apache.org/jira/browse/HADOOP-11312) | Fix unit tests to not use uppercase key names | Major | security | Andrew Wang | Andrew Wang | +| [HADOOP-11309](https://issues.apache.org/jira/browse/HADOOP-11309) | System class pattern package.Foo should match package.Foo$Bar, too | Blocker | . | Gera Shegalov | Gera Shegalov | +| [HADOOP-11300](https://issues.apache.org/jira/browse/HADOOP-11300) | KMS startup scripts must not display the keystore / truststore passwords | Major | kms | Arun Suresh | Arun Suresh | +| [HADOOP-11295](https://issues.apache.org/jira/browse/HADOOP-11295) | RPC Server Reader thread can't shutdown if RPCCallQueue is full | Major | . | Ming Ma | Ming Ma | +| [HADOOP-11294](https://issues.apache.org/jira/browse/HADOOP-11294) | Nfs3FileAttributes should not change the values of rdev, nlink and size in the constructor | Minor | nfs | Brandon Li | Brandon Li | +| [HADOOP-11289](https://issues.apache.org/jira/browse/HADOOP-11289) | Fix typo in RpcUtil log message | Trivial | net | Charles Lamb | Charles Lamb | +| [HADOOP-11287](https://issues.apache.org/jira/browse/HADOOP-11287) | Simplify UGI#reloginFromKeytab for Java 7+ | Major | . | Haohui Mai | Li Lu | +| [HADOOP-11283](https://issues.apache.org/jira/browse/HADOOP-11283) | Potentially unclosed SequenceFile.Writer in DistCpV1#setup() | Minor | . | Ted Yu | Varun Saxena | +| [HADOOP-11273](https://issues.apache.org/jira/browse/HADOOP-11273) | TestMiniKdc failure: login options not compatible with IBM JDK | Major | test | Gao Zhong Liang | Gao Zhong Liang | +| [HADOOP-11272](https://issues.apache.org/jira/browse/HADOOP-11272) | Allow ZKSignerSecretProvider and ZKDelegationTokenSecretManager to use the same curator client | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11271](https://issues.apache.org/jira/browse/HADOOP-11271) | Use Time.monotonicNow() in Shell.java instead of Time.now() | Minor | . | Vinayakumar B | Vinayakumar B | +| [HADOOP-11269](https://issues.apache.org/jira/browse/HADOOP-11269) | Add java 8 profile for hadoop-annotations | Major | . | Haohui Mai | Li Lu | +| [HADOOP-11268](https://issues.apache.org/jira/browse/HADOOP-11268) | Update BUILDING.txt to remove the workaround for tools.jar | Minor | . | Haohui Mai | Li Lu | +| [HADOOP-11267](https://issues.apache.org/jira/browse/HADOOP-11267) | TestSecurityUtil fails when run with JDK8 because of empty principal names | Minor | security, test | Stephen Chu | Stephen Chu | +| [HADOOP-11266](https://issues.apache.org/jira/browse/HADOOP-11266) | Remove no longer supported activation properties for packaging from pom | Trivial | build | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11256](https://issues.apache.org/jira/browse/HADOOP-11256) | Some site docs have inconsistent appearance | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11248](https://issues.apache.org/jira/browse/HADOOP-11248) | Add hadoop configuration to disable Azure Filesystem metrics collection | Major | fs | shanyu zhao | shanyu zhao | +| [HADOOP-11246](https://issues.apache.org/jira/browse/HADOOP-11246) | Move jenkins to Java 7 | Major | . | Haohui Mai | Steve Loughran | +| [HADOOP-11238](https://issues.apache.org/jira/browse/HADOOP-11238) | Update the NameNode's Group Cache in the background when possible | Minor | . | Chris Li | Chris Li | +| [HADOOP-11236](https://issues.apache.org/jira/browse/HADOOP-11236) | NFS: Fix javadoc warning in RpcProgram.java | Trivial | documentation | Abhiraj Butala | Abhiraj Butala | +| [HADOOP-11230](https://issues.apache.org/jira/browse/HADOOP-11230) | Add missing dependency of bouncycastle for kms, httpfs, hdfs, MR and YARN | Major | test | Robert Kanter | Robert Kanter | +| [HADOOP-11213](https://issues.apache.org/jira/browse/HADOOP-11213) | Typos in html pages: SecureMode and EncryptedShuffle | Minor | . | Wei Yan | Wei Yan | +| [HADOOP-11211](https://issues.apache.org/jira/browse/HADOOP-11211) | mapreduce.job.classloader.system.classes semantics should be order-independent | Major | . | Yitong Zhou | Yitong Zhou | +| [HADOOP-11209](https://issues.apache.org/jira/browse/HADOOP-11209) | Configuration#updatingResource/finalParameters are not thread-safe | Major | conf | Josh Rosen | Varun Saxena | +| [HADOOP-11201](https://issues.apache.org/jira/browse/HADOOP-11201) | Hadoop Archives should support globs resolving to files | Blocker | tools | Gera Shegalov | Gera Shegalov | +| [HADOOP-11187](https://issues.apache.org/jira/browse/HADOOP-11187) | NameNode - KMS communication fails after a long period of inactivity | Major | . | Arun Suresh | Arun Suresh | +| [HADOOP-11186](https://issues.apache.org/jira/browse/HADOOP-11186) | documentation should talk about hadoop.htrace.spanreceiver.classes, not hadoop.trace.spanreceiver.classes | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HADOOP-11166](https://issues.apache.org/jira/browse/HADOOP-11166) | Remove ulimit from test-patch.sh | Major | . | Andrew Wang | Andrew Wang | +| [HADOOP-11157](https://issues.apache.org/jira/browse/HADOOP-11157) | ZKDelegationTokenSecretManager never shuts down listenerThreadPool | Major | security | Gregory Chanan | Arun Suresh | +| [HADOOP-11156](https://issues.apache.org/jira/browse/HADOOP-11156) | DelegateToFileSystem should implement getFsStatus(final Path f). | Major | fs | zhihai xu | zhihai xu | +| [HADOOP-11039](https://issues.apache.org/jira/browse/HADOOP-11039) | ByteBufferReadable API doc is inconsistent with the implementations. | Minor | documentation | Yi Liu | Yi Liu | +| [HADOOP-11014](https://issues.apache.org/jira/browse/HADOOP-11014) | Potential resource leak in JavaKeyStoreProvider due to unclosed stream | Minor | security | Ted Yu | Tsuyoshi Ozawa | +| [HADOOP-11008](https://issues.apache.org/jira/browse/HADOOP-11008) | Remove duplicated description about proxy-user in site documents | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-11000](https://issues.apache.org/jira/browse/HADOOP-11000) | HAServiceProtocol's health state is incorrectly transitioned to SERVICE\_NOT\_RESPONDING | Major | . | Ming Ma | Ming Ma | +| [HADOOP-10953](https://issues.apache.org/jira/browse/HADOOP-10953) | NetworkTopology#add calls NetworkTopology#toString without holding the netlock | Minor | net | Liang Xie | Liang Xie | +| [HADOOP-10852](https://issues.apache.org/jira/browse/HADOOP-10852) | NetgroupCache is not thread-safe | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10840](https://issues.apache.org/jira/browse/HADOOP-10840) | Fix OutOfMemoryError caused by metrics system in Azure File System | Major | metrics | shanyu zhao | shanyu zhao | +| [HADOOP-10748](https://issues.apache.org/jira/browse/HADOOP-10748) | HttpServer2 should not load JspServlet | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-10717](https://issues.apache.org/jira/browse/HADOOP-10717) | HttpServer2 should load jsp DTD from local jars instead of going remote | Blocker | . | Dapeng Sun | Dapeng Sun | +| [HADOOP-10714](https://issues.apache.org/jira/browse/HADOOP-10714) | AmazonS3Client.deleteObjects() need to be limited to 1000 entries per call | Critical | fs/s3 | David S. Wang | Juan Yu | +| [HADOOP-10703](https://issues.apache.org/jira/browse/HADOOP-10703) | HttpServer2 creates multiple authentication filters | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10690](https://issues.apache.org/jira/browse/HADOOP-10690) | Lack of synchronization on access to InputStream in NativeAzureFileSystem#NativeAzureFsInputStream#close() | Minor | tools | Ted Yu | Chen He | +| [HADOOP-10689](https://issues.apache.org/jira/browse/HADOOP-10689) | InputStream is not closed in AzureNativeFileSystemStore#retrieve() | Minor | tools | Ted Yu | Chen He | +| [HADOOP-10542](https://issues.apache.org/jira/browse/HADOOP-10542) | Potential null pointer dereference in Jets3tFileSystemStore#retrieveBlock() | Minor | fs/s3 | Ted Yu | Ted Yu | +| [HADOOP-10181](https://issues.apache.org/jira/browse/HADOOP-10181) | GangliaContext does not work with multicast ganglia setup | Minor | metrics | Andrew Otto | Andrew Johnson | +| [HADOOP-10134](https://issues.apache.org/jira/browse/HADOOP-10134) | [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments | Minor | . | Andrew Purtell | Andrew Purtell | +| [HADOOP-10062](https://issues.apache.org/jira/browse/HADOOP-10062) | race condition in MetricsSystemImpl#publishMetricsNow that causes incorrect results | Major | metrics | Shinichi Yamashita | Sangjin Lee | +| [HADOOP-9922](https://issues.apache.org/jira/browse/HADOOP-9922) | hadoop windows native build will fail in 32 bit machine | Major | build, native | Vinayakumar B | Kiran Kumar M R | +| [HADOOP-9907](https://issues.apache.org/jira/browse/HADOOP-9907) | Webapp http://hostname:port/metrics link is not working | Critical | . | Jian He | Akira AJISAKA | +| [HADOOP-9137](https://issues.apache.org/jira/browse/HADOOP-9137) | Support connection limiting in IPC server | Major | . | Sanjay Radia | Kihwal Lee | +| [HADOOP-9087](https://issues.apache.org/jira/browse/HADOOP-9087) | Queue size metric for metric sinks isn't actually maintained | Minor | metrics | Mostafa Elhemali | Akira AJISAKA | +| [HADOOP-8642](https://issues.apache.org/jira/browse/HADOOP-8642) | Document that io.native.lib.available only controls native bz2 and zlib compression codecs | Major | documentation, native | Eli Collins | Akira AJISAKA | +| [HADOOP-6221](https://issues.apache.org/jira/browse/HADOOP-6221) | RPC Client operations cannot be interrupted | Minor | ipc | Steve Loughran | Steve Loughran | +| [HDFS-8072](https://issues.apache.org/jira/browse/HDFS-8072) | Reserved RBW space is not released if client terminates while writing block | Major | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-8063](https://issues.apache.org/jira/browse/HDFS-8063) | Fix intermittent test failures in TestTracing | Minor | test | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-8051](https://issues.apache.org/jira/browse/HDFS-8051) | FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-8036](https://issues.apache.org/jira/browse/HDFS-8036) | Use snapshot path as source when using snapshot diff report in DistCp | Major | distcp | Sushmitha Sreenivasan | Jing Zhao | +| [HDFS-8025](https://issues.apache.org/jira/browse/HDFS-8025) | Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated | Major | . | Ming Ma | Ming Ma | +| [HDFS-8001](https://issues.apache.org/jira/browse/HDFS-8001) | RpcProgramNfs3 : wrong parsing of dfs.blocksize | Trivial | nfs | Remi Catherinot | Remi Catherinot | +| [HDFS-7999](https://issues.apache.org/jira/browse/HDFS-7999) | FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time | Major | . | zhouyingchao | zhouyingchao | +| [HDFS-7996](https://issues.apache.org/jira/browse/HDFS-7996) | After swapping a volume, BlockReceiver reports ReplicaNotFoundException | Critical | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7977](https://issues.apache.org/jira/browse/HDFS-7977) | NFS couldn't take percentile intervals | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7963](https://issues.apache.org/jira/browse/HDFS-7963) | Fix expected tracing spans in TestTracing along with HDFS-7054 | Critical | test | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-7961](https://issues.apache.org/jira/browse/HDFS-7961) | Trigger full block report after hot swapping disk | Major | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7960](https://issues.apache.org/jira/browse/HDFS-7960) | The full block report should prune zombie storages even if they're not empty | Critical | . | Lei (Eddy) Xu | Colin Patrick McCabe | +| [HDFS-7957](https://issues.apache.org/jira/browse/HDFS-7957) | Truncate should verify quota before making changes | Critical | namenode | Jing Zhao | Jing Zhao | +| [HDFS-7956](https://issues.apache.org/jira/browse/HDFS-7956) | Improve logging for DatanodeRegistration. | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-7953](https://issues.apache.org/jira/browse/HDFS-7953) | NN Web UI fails to navigate to paths that contain # | Minor | namenode | kanaka kumar avvaru | kanaka kumar avvaru | +| [HDFS-7945](https://issues.apache.org/jira/browse/HDFS-7945) | The WebHdfs system on DN does not honor the length parameter | Blocker | . | Haohui Mai | Haohui Mai | +| [HDFS-7943](https://issues.apache.org/jira/browse/HDFS-7943) | Append cannot handle the last block with length greater than the preferred block size | Blocker | . | Jing Zhao | Jing Zhao | +| [HDFS-7942](https://issues.apache.org/jira/browse/HDFS-7942) | NFS: support regexp grouping in nfs.exports.allowed.hosts | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7932](https://issues.apache.org/jira/browse/HDFS-7932) | Speed up the shutdown of datanode during rolling upgrade | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7930](https://issues.apache.org/jira/browse/HDFS-7930) | commitBlockSynchronization() does not remove locations | Blocker | namenode | Konstantin Shvachko | Yi Liu | +| [HDFS-7929](https://issues.apache.org/jira/browse/HDFS-7929) | inotify unable fetch pre-upgrade edit log segments once upgrade starts | Major | . | Zhe Zhang | Zhe Zhang | +| [HDFS-7926](https://issues.apache.org/jira/browse/HDFS-7926) | NameNode implementation of ClientProtocol.truncate(..) is not idempotent | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7915](https://issues.apache.org/jira/browse/HDFS-7915) | The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7886](https://issues.apache.org/jira/browse/HDFS-7886) | TestFileTruncate#testTruncateWithDataNodesRestart runs timeout sometimes | Minor | test | Yi Liu | Plamen Jeliazkov | +| [HDFS-7885](https://issues.apache.org/jira/browse/HDFS-7885) | Datanode should not trust the generation stamp provided by client | Critical | datanode | vitthal (Suhas) Gogate | Tsz Wo Nicholas Sze | +| [HDFS-7884](https://issues.apache.org/jira/browse/HDFS-7884) | NullPointerException in BlockSender | Blocker | datanode | Tsz Wo Nicholas Sze | Brahma Reddy Battula | +| [HDFS-7881](https://issues.apache.org/jira/browse/HDFS-7881) | TestHftpFileSystem#testSeek fails in branch-2 | Blocker | . | Akira AJISAKA | Brahma Reddy Battula | +| [HDFS-7880](https://issues.apache.org/jira/browse/HDFS-7880) | Remove the tests for legacy Web UI in branch-2 | Blocker | test | Akira AJISAKA | Brahma Reddy Battula | +| [HDFS-7879](https://issues.apache.org/jira/browse/HDFS-7879) | hdfs.dll does not export functions of the public libhdfs API | Major | build, libhdfs | Chris Nauroth | Chris Nauroth | +| [HDFS-7871](https://issues.apache.org/jira/browse/HDFS-7871) | NameNodeEditLogRoller can keep printing "Swallowing exception" message | Critical | . | Jing Zhao | Jing Zhao | +| [HDFS-7869](https://issues.apache.org/jira/browse/HDFS-7869) | Inconsistency in the return information while performing rolling upgrade | Major | . | J.Andreina | J.Andreina | +| [HDFS-7831](https://issues.apache.org/jira/browse/HDFS-7831) | Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks() | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-7830](https://issues.apache.org/jira/browse/HDFS-7830) | DataNode does not release the volume lock when adding a volume fails. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7818](https://issues.apache.org/jira/browse/HDFS-7818) | OffsetParam should return the default value instead of throwing NPE when the value is unspecified | Blocker | webhdfs | Eric Payne | Eric Payne | +| [HDFS-7816](https://issues.apache.org/jira/browse/HDFS-7816) | Unable to open webhdfs paths with "+" | Blocker | webhdfs | Jason Lowe | Haohui Mai | +| [HDFS-7813](https://issues.apache.org/jira/browse/HDFS-7813) | TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently | Major | ha, test | Rakesh R | Rakesh R | +| [HDFS-7807](https://issues.apache.org/jira/browse/HDFS-7807) | libhdfs htable.c: fix htable resizing, add unit test | Major | native | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7805](https://issues.apache.org/jira/browse/HDFS-7805) | NameNode recovery prompt should be printed on console | Major | namenode | Surendra Singh Lilhore | Surendra Singh Lilhore | +| [HDFS-7798](https://issues.apache.org/jira/browse/HDFS-7798) | Checkpointing failure caused by shared KerberosAuthenticator | Critical | security | Chengbing Liu | Chengbing Liu | +| [HDFS-7788](https://issues.apache.org/jira/browse/HDFS-7788) | Post-2.6 namenode may not start up with an image containing inodes created with an old release. | Blocker | . | Kihwal Lee | Rushabh S Shah | +| [HDFS-7785](https://issues.apache.org/jira/browse/HDFS-7785) | Improve diagnostics information for HttpPutFailedException | Major | namenode | Chengbing Liu | Chengbing Liu | +| [HDFS-7778](https://issues.apache.org/jira/browse/HDFS-7778) | Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2 | Major | datanode, test | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7774](https://issues.apache.org/jira/browse/HDFS-7774) | Unresolved symbols error while compiling HDFS on Windows 7/32 bit | Critical | build, native | Venkatasubramaniam Ramakrishnan | Kiran Kumar M R | +| [HDFS-7769](https://issues.apache.org/jira/browse/HDFS-7769) | TestHDFSCLI create files in hdfs project root dir | Trivial | test | Tsz Wo Nicholas Sze | | +| [HDFS-7763](https://issues.apache.org/jira/browse/HDFS-7763) | fix zkfc hung issue due to not catching exception in a corner case | Major | ha | Liang Xie | Liang Xie | +| [HDFS-7756](https://issues.apache.org/jira/browse/HDFS-7756) | Restore method signature for LocatedBlock#getLocations() | Major | . | Ted Yu | Ted Yu | +| [HDFS-7753](https://issues.apache.org/jira/browse/HDFS-7753) | Fix Multithreaded correctness Warnings in BackupImage.java | Major | . | Rakesh R | Konstantin Shvachko | +| [HDFS-7748](https://issues.apache.org/jira/browse/HDFS-7748) | Separate ECN flags from the Status in the DataTransferPipelineAck | Blocker | . | Haohui Mai | Anu Engineer | +| [HDFS-7744](https://issues.apache.org/jira/browse/HDFS-7744) | Fix potential NPE in DFSInputStream after setDropBehind or setReadahead is called | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7742](https://issues.apache.org/jira/browse/HDFS-7742) | favoring decommissioning node for replication can cause a block to stay underreplicated for long periods | Major | namenode | Nathan Roberts | Nathan Roberts | +| [HDFS-7741](https://issues.apache.org/jira/browse/HDFS-7741) | Remove unnecessary synchronized in FSDataInputStream and HdfsDataInputStream | Minor | . | Yi Liu | Yi Liu | +| [HDFS-7734](https://issues.apache.org/jira/browse/HDFS-7734) | Class cast exception in NameNode#main | Blocker | namenode | Arpit Agarwal | Yi Liu | +| [HDFS-7722](https://issues.apache.org/jira/browse/HDFS-7722) | DataNode#checkDiskError should also remove Storage when error is found. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7721](https://issues.apache.org/jira/browse/HDFS-7721) | The HDFS BlockScanner may run fast during the first hour | Major | datanode | Tsz Wo Nicholas Sze | Colin Patrick McCabe | +| [HDFS-7719](https://issues.apache.org/jira/browse/HDFS-7719) | BlockPoolSliceStorage#removeVolumes fails to remove some in-memory state associated with volumes | Major | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7718](https://issues.apache.org/jira/browse/HDFS-7718) | Store KeyProvider in ClientContext to avoid leaking key provider threads when using FileContext | Major | . | Arun Suresh | Arun Suresh | +| [HDFS-7714](https://issues.apache.org/jira/browse/HDFS-7714) | Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode. | Major | datanode | Chris Nauroth | Vinayakumar B | +| [HDFS-7709](https://issues.apache.org/jira/browse/HDFS-7709) | Fix findbug warnings in httpfs | Major | . | Rakesh R | Rakesh R | +| [HDFS-7707](https://issues.apache.org/jira/browse/HDFS-7707) | Edit log corruption due to delayed block removal again | Major | namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7704](https://issues.apache.org/jira/browse/HDFS-7704) | DN heartbeat to Active NN may be blocked and expire if connection to Standby NN continues to time out. | Major | datanode, namenode | Rushabh S Shah | Rushabh S Shah | +| [HDFS-7698](https://issues.apache.org/jira/browse/HDFS-7698) | Fix locking on HDFS read statistics and add a method for clearing them. | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7697](https://issues.apache.org/jira/browse/HDFS-7697) | Mark the PB OIV tool as experimental | Major | . | Haohui Mai | Lei (Eddy) Xu | +| [HDFS-7696](https://issues.apache.org/jira/browse/HDFS-7696) | FsDatasetImpl.getTmpInputStreams(..) may leak file descriptors | Minor | datanode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7686](https://issues.apache.org/jira/browse/HDFS-7686) | Re-add rapid rescan of possibly corrupt block feature to the block scanner | Blocker | . | Rushabh S Shah | Colin Patrick McCabe | +| [HDFS-7682](https://issues.apache.org/jira/browse/HDFS-7682) | {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content | Major | . | Charles Lamb | Charles Lamb | +| [HDFS-7660](https://issues.apache.org/jira/browse/HDFS-7660) | BlockReceiver#close() might be called multiple times, which causes the fsvolume reference being released incorrectly. | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7647](https://issues.apache.org/jira/browse/HDFS-7647) | DatanodeManager.sortLocatedBlocks sorts DatanodeInfos but not StorageIDs | Major | . | Milan Desai | Milan Desai | +| [HDFS-7644](https://issues.apache.org/jira/browse/HDFS-7644) | minor typo in HttpFS doc | Trivial | documentation | Charles Lamb | Charles Lamb | +| [HDFS-7641](https://issues.apache.org/jira/browse/HDFS-7641) | Update archival storage user doc for list/set/get block storage policies | Minor | documentation | Yi Liu | Yi Liu | +| [HDFS-7637](https://issues.apache.org/jira/browse/HDFS-7637) | Fix the check condition for reserved path | Minor | . | Yi Liu | Yi Liu | +| [HDFS-7635](https://issues.apache.org/jira/browse/HDFS-7635) | Remove TestCorruptFilesJsp from branch-2. | Minor | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7632](https://issues.apache.org/jira/browse/HDFS-7632) | MiniDFSCluster configures DataNode data directories incorrectly if using more than 1 DataNode and more than 2 storage locations per DataNode. | Major | test | Chris Nauroth | Chris Nauroth | +| [HDFS-7615](https://issues.apache.org/jira/browse/HDFS-7615) | Remove longReadLock | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7611](https://issues.apache.org/jira/browse/HDFS-7611) | deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. | Critical | namenode | Konstantin Shvachko | Jing Zhao | +| [HDFS-7610](https://issues.apache.org/jira/browse/HDFS-7610) | Fix removal of dynamically added DN volumes | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7606](https://issues.apache.org/jira/browse/HDFS-7606) | Missing null check in INodeFile#getBlocks() | Minor | . | Ted Yu | Byron Wong | +| [HDFS-7603](https://issues.apache.org/jira/browse/HDFS-7603) | The background replication queue initialization may not let others run | Critical | rolling upgrades | Kihwal Lee | Kihwal Lee | +| [HDFS-7596](https://issues.apache.org/jira/browse/HDFS-7596) | NameNode should prune dead storages from storageMap | Major | namenode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7587](https://issues.apache.org/jira/browse/HDFS-7587) | Edit log corruption can happen if append fails with a quota violation | Blocker | namenode | Kihwal Lee | Jing Zhao | +| [HDFS-7583](https://issues.apache.org/jira/browse/HDFS-7583) | Fix findbug in TransferFsImage.java | Minor | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7575](https://issues.apache.org/jira/browse/HDFS-7575) | Upgrade should generate a unique storage ID for each volume | Critical | . | Lars Francke | Arpit Agarwal | +| [HDFS-7572](https://issues.apache.org/jira/browse/HDFS-7572) | TestLazyPersistFiles#testDnRestartWithSavedReplicas is flaky on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7566](https://issues.apache.org/jira/browse/HDFS-7566) | Remove obsolete entries from hdfs-default.xml | Major | . | Ray Chiang | Ray Chiang | +| [HDFS-7563](https://issues.apache.org/jira/browse/HDFS-7563) | NFS gateway parseStaticMap NumberFormatException | Major | nfs | Hari Sekhon | Yongjun Zhang | +| [HDFS-7561](https://issues.apache.org/jira/browse/HDFS-7561) | TestFetchImage should write fetched-image-dir under target. | Major | . | Konstantin Shvachko | Liang Xie | +| [HDFS-7560](https://issues.apache.org/jira/browse/HDFS-7560) | ACLs removed by removeDefaultAcl() will be back after NameNode restart/failover | Critical | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7552](https://issues.apache.org/jira/browse/HDFS-7552) | change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration | Major | datanode, test | Liang Xie | Liang Xie | +| [HDFS-7548](https://issues.apache.org/jira/browse/HDFS-7548) | Corrupt block reporting delayed until datablock scanner thread detects it | Major | . | Rushabh S Shah | Rushabh S Shah | +| [HDFS-7536](https://issues.apache.org/jira/browse/HDFS-7536) | Remove unused CryptoCodec in org.apache.hadoop.fs.Hdfs | Minor | security | Yi Liu | Yi Liu | +| [HDFS-7533](https://issues.apache.org/jira/browse/HDFS-7533) | Datanode sometimes does not shutdown on receiving upgrade shutdown command | Major | . | Kihwal Lee | Eric Payne | +| [HDFS-7530](https://issues.apache.org/jira/browse/HDFS-7530) | Allow renaming of encryption zone roots | Minor | namenode | Charles Lamb | Charles Lamb | +| [HDFS-7517](https://issues.apache.org/jira/browse/HDFS-7517) | Remove redundant non-null checks in FSNamesystem#getBlockLocations | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7516](https://issues.apache.org/jira/browse/HDFS-7516) | Fix findbugs warnings in hadoop-nfs project | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7515](https://issues.apache.org/jira/browse/HDFS-7515) | Fix new findbugs warnings in hadoop-hdfs | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7514](https://issues.apache.org/jira/browse/HDFS-7514) | TestTextCommand fails on Windows | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-7502](https://issues.apache.org/jira/browse/HDFS-7502) | Fix findbugs warning in hdfs-nfs project | Major | nfs | Brandon Li | Brandon Li | +| [HDFS-7497](https://issues.apache.org/jira/browse/HDFS-7497) | Inconsistent report of decommissioning DataNodes between dfsadmin and NameNode webui | Major | datanode, namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7496](https://issues.apache.org/jira/browse/HDFS-7496) | Fix FsVolume removal race conditions on the DataNode by reference-counting the volume instances | Major | . | Colin Patrick McCabe | Lei (Eddy) Xu | +| [HDFS-7495](https://issues.apache.org/jira/browse/HDFS-7495) | Remove updatePosition argument from DFSInputStream#getBlockAt() | Minor | . | Ted Yu | Colin Patrick McCabe | +| [HDFS-7494](https://issues.apache.org/jira/browse/HDFS-7494) | Checking of closed in DFSInputStream#pread() should be protected by synchronization | Minor | . | Ted Yu | Ted Yu | +| [HDFS-7490](https://issues.apache.org/jira/browse/HDFS-7490) | HDFS tests OOM on Java7+ | Major | build, test | Steve Loughran | Steve Loughran | +| [HDFS-7481](https://issues.apache.org/jira/browse/HDFS-7481) | Add ACL indicator to the "Permission Denied" exception. | Minor | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7473](https://issues.apache.org/jira/browse/HDFS-7473) | Document setting dfs.namenode.fs-limits.max-directory-items to 0 is invalid | Major | documentation | Jason Keller | Akira AJISAKA | +| [HDFS-7472](https://issues.apache.org/jira/browse/HDFS-7472) | Fix typo in message of ReplicaNotFoundException | Trivial | . | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-7470](https://issues.apache.org/jira/browse/HDFS-7470) | SecondaryNameNode need twice memory when calling reloadFromImageFile | Major | namenode | zhaoyunjiong | zhaoyunjiong | +| [HDFS-7457](https://issues.apache.org/jira/browse/HDFS-7457) | DatanodeID generates excessive garbage | Major | namenode | Daryn Sharp | Daryn Sharp | +| [HDFS-7456](https://issues.apache.org/jira/browse/HDFS-7456) | De-duplicate AclFeature instances with same AclEntries do reduce memory footprint of NameNode | Major | namenode | Vinayakumar B | Vinayakumar B | +| [HDFS-7444](https://issues.apache.org/jira/browse/HDFS-7444) | convertToBlockUnderConstruction should preserve BlockCollection | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7431](https://issues.apache.org/jira/browse/HDFS-7431) | log message for InvalidMagicNumberException may be incorrect | Major | security | Yi Liu | Yi Liu | +| [HDFS-7423](https://issues.apache.org/jira/browse/HDFS-7423) | various typos and message formatting fixes in nfs daemon and doc | Trivial | nfs | Charles Lamb | Charles Lamb | +| [HDFS-7406](https://issues.apache.org/jira/browse/HDFS-7406) | SimpleHttpProxyHandler puts incorrect "Connection: Close" header | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7403](https://issues.apache.org/jira/browse/HDFS-7403) | Inaccurate javadoc of BlockUCState#COMPLETE state | Trivial | namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7399](https://issues.apache.org/jira/browse/HDFS-7399) | Lack of synchronization in DFSOutputStream#Packet#getLastByteOffsetBlock() | Minor | . | Ted Yu | Vinayakumar B | +| [HDFS-7395](https://issues.apache.org/jira/browse/HDFS-7395) | BlockIdManager#clear() bails out when resetting the GenerationStampV1Limit | Major | namenode | Yongjun Zhang | Haohui Mai | +| [HDFS-7394](https://issues.apache.org/jira/browse/HDFS-7394) | Log at INFO level, not WARN level, when InvalidToken is seen in ShortCircuitCache | Minor | . | Kihwal Lee | Keith Pak | +| [HDFS-7389](https://issues.apache.org/jira/browse/HDFS-7389) | Named user ACL cannot stop the user from accessing the FS entity. | Major | namenode | Chunjun Xiao | Vinayakumar B | +| [HDFS-7374](https://issues.apache.org/jira/browse/HDFS-7374) | Allow decommissioning of dead DataNodes | Major | . | Zhe Zhang | Zhe Zhang | +| [HDFS-7373](https://issues.apache.org/jira/browse/HDFS-7373) | Clean up temporary files after fsimage transfer failures | Major | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7366](https://issues.apache.org/jira/browse/HDFS-7366) | BlockInfo should take replication as an short in the constructor | Minor | . | Haohui Mai | Li Lu | +| [HDFS-7361](https://issues.apache.org/jira/browse/HDFS-7361) | TestCheckpoint#testStorageAlreadyLockedErrorMessage fails after change of log message related to locking violation. | Minor | datanode, namenode, test | Chris Nauroth | Konstantin Shvachko | +| [HDFS-7358](https://issues.apache.org/jira/browse/HDFS-7358) | Clients may get stuck waiting when using ByteArrayManager | Major | hdfs-client | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7315](https://issues.apache.org/jira/browse/HDFS-7315) | DFSTestUtil.readFileBuffer opens extra FSDataInputStream | Trivial | . | Plamen Jeliazkov | Plamen Jeliazkov | +| [HDFS-7303](https://issues.apache.org/jira/browse/HDFS-7303) | NN UI fails to distinguish datanodes on the same host | Minor | . | Benoy Antony | Benoy Antony | +| [HDFS-7301](https://issues.apache.org/jira/browse/HDFS-7301) | TestMissingBlocksAlert should use MXBeans instead of old web UI | Minor | . | Zhe Zhang | Zhe Zhang | +| [HDFS-7282](https://issues.apache.org/jira/browse/HDFS-7282) | Fix intermittent TestShortCircuitCache and TestBlockReaderFactory failures resulting from TemporarySocketDirectory GC | Major | test | Jinghui Wang | Jinghui Wang | +| [HDFS-7277](https://issues.apache.org/jira/browse/HDFS-7277) | Remove explicit dependency on netty 3.2 in BKJournal | Minor | build | Haohui Mai | Haohui Mai | +| [HDFS-7263](https://issues.apache.org/jira/browse/HDFS-7263) | Snapshot read can reveal future bytes for appended files. | Major | hdfs-client | Konstantin Shvachko | Tao Luo | +| [HDFS-7258](https://issues.apache.org/jira/browse/HDFS-7258) | CacheReplicationMonitor rescan schedule log should use DEBUG level instead of INFO level | Minor | namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7235](https://issues.apache.org/jira/browse/HDFS-7235) | DataNode#transferBlock should report blocks that don't exist using reportBadBlock | Major | datanode, namenode | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7232](https://issues.apache.org/jira/browse/HDFS-7232) | Populate hostname in httpfs audit log | Trivial | . | Zoran Dimitrijevic | Zoran Dimitrijevic | +| [HDFS-7227](https://issues.apache.org/jira/browse/HDFS-7227) | Fix findbugs warning about NP\_DEREFERENCE\_OF\_READLINE\_VALUE in SpanReceiverHost | Minor | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7225](https://issues.apache.org/jira/browse/HDFS-7225) | Remove stale block invalidation work when DN re-registers with different UUID | Major | namenode | Zhe Zhang | Zhe Zhang | +| [HDFS-7224](https://issues.apache.org/jira/browse/HDFS-7224) | Allow reuse of NN connections via webhdfs | Major | webhdfs | Eric Payne | Eric Payne | +| [HDFS-7213](https://issues.apache.org/jira/browse/HDFS-7213) | processIncrementalBlockReport performance degradation | Critical | namenode | Daryn Sharp | Eric Payne | +| [HDFS-7202](https://issues.apache.org/jira/browse/HDFS-7202) | Should be able to omit package name of SpanReceiver on "hadoop trace -add" | Minor | . | Masatake Iwasaki | Masatake Iwasaki | +| [HDFS-7201](https://issues.apache.org/jira/browse/HDFS-7201) | Fix typos in hdfs-default.xml | Major | . | Konstantin Shvachko | Dawson Choong | +| [HDFS-7198](https://issues.apache.org/jira/browse/HDFS-7198) | Fix findbugs "unchecked conversion" warning in DFSClient#getPathTraceScope | Trivial | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7194](https://issues.apache.org/jira/browse/HDFS-7194) | Fix findbugs "inefficient new String constructor" warning in DFSClient#PATH | Trivial | . | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7146](https://issues.apache.org/jira/browse/HDFS-7146) | NFS ID/Group lookup requires SSSD enumeration on the server | Major | nfs | Yongjun Zhang | Yongjun Zhang | +| [HDFS-7097](https://issues.apache.org/jira/browse/HDFS-7097) | Allow block reports to be processed during checkpointing on standby name node | Critical | . | Kihwal Lee | Kihwal Lee | +| [HDFS-7009](https://issues.apache.org/jira/browse/HDFS-7009) | Active NN and standby NN have different live nodes | Major | datanode | Ming Ma | Ming Ma | +| [HDFS-7008](https://issues.apache.org/jira/browse/HDFS-7008) | xlator should be closed upon exit from DFSAdmin#genericRefresh() | Minor | . | Ted Yu | Tsuyoshi Ozawa | +| [HDFS-6938](https://issues.apache.org/jira/browse/HDFS-6938) | Cleanup javac warnings in FSNamesystem | Trivial | namenode | Charles Lamb | Charles Lamb | +| [HDFS-6917](https://issues.apache.org/jira/browse/HDFS-6917) | Add an hdfs debug command to validate blocks, call recoverlease, etc. | Major | hdfs-client | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-6841](https://issues.apache.org/jira/browse/HDFS-6841) | Use Time.monotonicNow() wherever applicable instead of Time.now() | Major | . | Vinayakumar B | Vinayakumar B | +| [HDFS-6833](https://issues.apache.org/jira/browse/HDFS-6833) | DirectoryScanner should not register a deleting block with memory of DataNode | Critical | datanode | Shinichi Yamashita | Shinichi Yamashita | +| [HDFS-6753](https://issues.apache.org/jira/browse/HDFS-6753) | Initialize checkDisk when DirectoryScanner not able to get files list for scanning | Major | . | J.Andreina | J.Andreina | +| [HDFS-6672](https://issues.apache.org/jira/browse/HDFS-6672) | Regression with hdfs oiv tool | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6662](https://issues.apache.org/jira/browse/HDFS-6662) | WebHDFS cannot open a file if its path contains "%" | Critical | namenode | Brahma Reddy Battula | Gerson Carlos | +| [HDFS-6657](https://issues.apache.org/jira/browse/HDFS-6657) | Remove link to 'Legacy UI' in Namenode UI | Minor | . | Vinayakumar B | Vinayakumar B | +| [HDFS-6538](https://issues.apache.org/jira/browse/HDFS-6538) | Comment format error in ShortCircuitRegistry javadoc | Trivial | datanode | debugging | David Luo | +| [HDFS-6425](https://issues.apache.org/jira/browse/HDFS-6425) | Large postponedMisreplicatedBlocks has impact on blockReport latency | Major | . | Ming Ma | Ming Ma | +| [HDFS-5578](https://issues.apache.org/jira/browse/HDFS-5578) | [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments | Minor | . | Andrew Purtell | Andrew Purtell | +| [HDFS-5445](https://issues.apache.org/jira/browse/HDFS-5445) | PacketReceiver populates the packetLen field in PacketHeader incorrectly | Minor | datanode | Jonathan Mace | Jonathan Mace | +| [HDFS-3519](https://issues.apache.org/jira/browse/HDFS-3519) | Checkpoint upload may interfere with a concurrent saveNamespace | Critical | namenode | Todd Lipcon | Ming Ma | +| [HDFS-2605](https://issues.apache.org/jira/browse/HDFS-2605) | CHANGES.txt has two "Release 0.21.1" sections | Major | documentation | Konstantin Shvachko | Allen Wittenauer | +| [HDFS-1522](https://issues.apache.org/jira/browse/HDFS-1522) | Merge Block.BLOCK\_FILE\_PREFIX and DataStorage.BLOCK\_FILE\_PREFIX into one constant | Major | datanode | Konstantin Shvachko | Dongming Liang | +| [HDFS-49](https://issues.apache.org/jira/browse/HDFS-49) | MiniDFSCluster.stopDataNode will always shut down a node in the cluster if a matching name is not found | Minor | test | Steve Loughran | Steve Loughran | +| [MAPREDUCE-6303](https://issues.apache.org/jira/browse/MAPREDUCE-6303) | Read timeout when retrying a fetch error can be fatal to a reducer | Blocker | . | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6285](https://issues.apache.org/jira/browse/MAPREDUCE-6285) | ClientServiceDelegate should not retry upon AuthenticationException | Major | . | Jonathan Eagles | Jonathan Eagles | +| [MAPREDUCE-6277](https://issues.apache.org/jira/browse/MAPREDUCE-6277) | Job can post multiple history files if attempt loses connection to the RM | Major | mr-am | Chang Li | Chang Li | +| [MAPREDUCE-6275](https://issues.apache.org/jira/browse/MAPREDUCE-6275) | Race condition in FileOutputCommitter v2 for user-specified task output subdirs | Critical | . | Siqi Li | Gera Shegalov | +| [MAPREDUCE-6268](https://issues.apache.org/jira/browse/MAPREDUCE-6268) | Fix typo in Task Attempt API's URL | Minor | . | Ryu Kobayashi | Ryu Kobayashi | +| [MAPREDUCE-6263](https://issues.apache.org/jira/browse/MAPREDUCE-6263) | Configurable timeout between YARNRunner terminate the application and forcefully kill. | Major | client | Jason Lowe | Eric Payne | +| [MAPREDUCE-6261](https://issues.apache.org/jira/browse/MAPREDUCE-6261) | NullPointerException if MapOutputBuffer.flush invoked twice | Major | mrv2 | Jason Lowe | Tsuyoshi Ozawa | +| [MAPREDUCE-6243](https://issues.apache.org/jira/browse/MAPREDUCE-6243) | Fix findbugs warnings in hadoop-rumen | Minor | tools/rumen | Akira AJISAKA | Masatake Iwasaki | +| [MAPREDUCE-6233](https://issues.apache.org/jira/browse/MAPREDUCE-6233) | org.apache.hadoop.mapreduce.TestLargeSort.testLargeSort failed in trunk | Major | test | Yongjun Zhang | zhihai xu | +| [MAPREDUCE-6231](https://issues.apache.org/jira/browse/MAPREDUCE-6231) | Grep example job is not working on a fully-distributed cluster | Major | examples | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-6230](https://issues.apache.org/jira/browse/MAPREDUCE-6230) | MR AM does not survive RM restart if RM activated a new AMRM secret key | Blocker | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6225](https://issues.apache.org/jira/browse/MAPREDUCE-6225) | Fix new findbug warnings in hadoop-mapreduce-client-core | Major | . | Jason Lowe | Varun Saxena | +| [MAPREDUCE-6221](https://issues.apache.org/jira/browse/MAPREDUCE-6221) | Stringifier is left unclosed in Chain#getChainElementConf() | Minor | . | Ted Yu | Ted Yu | +| [MAPREDUCE-6210](https://issues.apache.org/jira/browse/MAPREDUCE-6210) | Use getApplicationAttemptId() instead of getApplicationID() for logging AttemptId in RMContainerAllocator.java | Minor | applicationmaster | Leitao Guo | Leitao Guo | +| [MAPREDUCE-6206](https://issues.apache.org/jira/browse/MAPREDUCE-6206) | TestAggregatedTransferRate fails on non-US systems | Critical | . | Jens Rabe | Jens Rabe | +| [MAPREDUCE-6186](https://issues.apache.org/jira/browse/MAPREDUCE-6186) | Redundant call to requireJob() while displaying the conf page | Minor | jobhistoryserver | Rohit Agarwal | Rohit Agarwal | +| [MAPREDUCE-6177](https://issues.apache.org/jira/browse/MAPREDUCE-6177) | Minor typo in the EncryptedShuffle document about ssl-client.xml | Trivial | documentation | yangping wu | yangping wu | +| [MAPREDUCE-6172](https://issues.apache.org/jira/browse/MAPREDUCE-6172) | TestDbClasses timeouts are too aggressive | Minor | test | Jason Lowe | Varun Saxena | +| [MAPREDUCE-6166](https://issues.apache.org/jira/browse/MAPREDUCE-6166) | Reducers do not validate checksum of map outputs when fetching directly to disk | Major | mrv2 | Eric Payne | Eric Payne | +| [MAPREDUCE-6162](https://issues.apache.org/jira/browse/MAPREDUCE-6162) | mapred hsadmin fails on a secure cluster | Blocker | jobhistoryserver | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6160](https://issues.apache.org/jira/browse/MAPREDUCE-6160) | Potential NullPointerException in MRClientProtocol interface implementation. | Major | . | Rohith Sharma K S | Rohith Sharma K S | +| [MAPREDUCE-6136](https://issues.apache.org/jira/browse/MAPREDUCE-6136) | MRAppMaster doesn't shutdown file systems | Major | applicationmaster | Noah Watkins | Brahma Reddy Battula | +| [MAPREDUCE-6049](https://issues.apache.org/jira/browse/MAPREDUCE-6049) | AM JVM does not exit if MRClientService gracefull shutdown fails | Major | applicationmaster, resourcemanager | Nishan Shetty | Rohith Sharma K S | +| [MAPREDUCE-6045](https://issues.apache.org/jira/browse/MAPREDUCE-6045) | need close the DataInputStream after open it in TestMapReduce.java | Minor | test | zhihai xu | zhihai xu | +| [MAPREDUCE-5988](https://issues.apache.org/jira/browse/MAPREDUCE-5988) | Fix dead links to the javadocs in mapreduce project | Minor | documentation | Akira AJISAKA | Akira AJISAKA | +| [MAPREDUCE-5918](https://issues.apache.org/jira/browse/MAPREDUCE-5918) | LineRecordReader can return the same decompressor to CodecPool multiple times | Major | . | Sergey Murylev | Sergey Murylev | +| [MAPREDUCE-5657](https://issues.apache.org/jira/browse/MAPREDUCE-5657) | [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments | Minor | documentation | Andrew Purtell | Akira AJISAKA | +| [MAPREDUCE-5568](https://issues.apache.org/jira/browse/MAPREDUCE-5568) | JHS returns invalid string for reducer completion percentage if AM restarts with 0 reducer. | Major | . | Jian He | MinJi Kim | +| [MAPREDUCE-4879](https://issues.apache.org/jira/browse/MAPREDUCE-4879) | TeraOutputFormat may overwrite an existing output directory | Major | examples | Gera Shegalov | Gera Shegalov | +| [MAPREDUCE-4742](https://issues.apache.org/jira/browse/MAPREDUCE-4742) | Fix typo in nnbench#displayUsage | Trivial | test | Liang Xie | Liang Xie | +| [MAPREDUCE-4286](https://issues.apache.org/jira/browse/MAPREDUCE-4286) | TestClientProtocolProviderImpls passes on failure conditions | Major | . | Devaraj K | Devaraj K | +| [MAPREDUCE-3283](https://issues.apache.org/jira/browse/MAPREDUCE-3283) | mapred classpath CLI does not display the complete classpath | Minor | scripts | Ramya Sunil | Varun Saxena | +| [MAPREDUCE-2815](https://issues.apache.org/jira/browse/MAPREDUCE-2815) | JavaDoc does not generate correctly for MultithreadedMapRunner | Minor | documentation | Shane Butler | Chris Palmer | +| [YARN-3393](https://issues.apache.org/jira/browse/YARN-3393) | Getting application(s) goes wrong when app finishes before starting the attempt | Critical | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-3384](https://issues.apache.org/jira/browse/YARN-3384) | TestLogAggregationService.verifyContainerLogs fails after YARN-2777 | Minor | . | Naganarasimha G R | Naganarasimha G R | +| [YARN-3369](https://issues.apache.org/jira/browse/YARN-3369) | Missing NullPointer check in AppSchedulingInfo causes RM to die | Blocker | resourcemanager | Giovanni Matteo Fumarola | Brahma Reddy Battula | +| [YARN-3338](https://issues.apache.org/jira/browse/YARN-3338) | Exclude jline dependency from YARN | Blocker | build | Zhijie Shen | Zhijie Shen | +| [YARN-3336](https://issues.apache.org/jira/browse/YARN-3336) | FileSystem memory leak in DelegationTokenRenewer | Critical | resourcemanager | zhihai xu | zhihai xu | +| [YARN-3304](https://issues.apache.org/jira/browse/YARN-3304) | ResourceCalculatorProcessTree#getCpuUsagePercent default return value is inconsistent with other getters | Blocker | nodemanager | Junping Du | Junping Du | +| [YARN-3296](https://issues.apache.org/jira/browse/YARN-3296) | yarn.nodemanager.container-monitor.process-tree.class is configurable but ResourceCalculatorProcessTree class is marked Private | Major | . | Hitesh Shah | Hitesh Shah | +| [YARN-3295](https://issues.apache.org/jira/browse/YARN-3295) | Fix documentation nits found in markdown conversion | Trivial | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [YARN-3287](https://issues.apache.org/jira/browse/YARN-3287) | TimelineClient kerberos authentication failure uses wrong login context. | Major | . | Jonathan Eagles | Daryn Sharp | +| [YARN-3281](https://issues.apache.org/jira/browse/YARN-3281) | Add RMStateStore to StateMachine visualization list | Minor | scripts | Chengbing Liu | Chengbing Liu | +| [YARN-3275](https://issues.apache.org/jira/browse/YARN-3275) | CapacityScheduler: Preemption happening on non-preemptable queues | Major | . | Eric Payne | Eric Payne | +| [YARN-3270](https://issues.apache.org/jira/browse/YARN-3270) | node label expression not getting set in ApplicationSubmissionContext | Minor | . | Rohit Agarwal | Rohit Agarwal | +| [YARN-3267](https://issues.apache.org/jira/browse/YARN-3267) | Timelineserver applies the ACL rules after applying the limit on the number of records | Major | . | Prakash Ramachandran | Chang Li | +| [YARN-3256](https://issues.apache.org/jira/browse/YARN-3256) | TestClientToAMTokens#testClientTokenRace is not running against all Schedulers even when using ParameterizedSchedulerTestBase | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3255](https://issues.apache.org/jira/browse/YARN-3255) | RM, NM, JobHistoryServer, and WebAppProxyServer's main() should support generic options | Major | nodemanager, resourcemanager | Konstantin Shvachko | Konstantin Shvachko | +| [YARN-3242](https://issues.apache.org/jira/browse/YARN-3242) | Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client | Critical | resourcemanager | zhihai xu | zhihai xu | +| [YARN-3239](https://issues.apache.org/jira/browse/YARN-3239) | WebAppProxy does not support a final tracking url which has query fragments and params | Major | . | Hitesh Shah | Jian He | +| [YARN-3238](https://issues.apache.org/jira/browse/YARN-3238) | Connection timeouts to nodemanagers are retried at multiple levels | Blocker | . | Jason Lowe | Jason Lowe | +| [YARN-3237](https://issues.apache.org/jira/browse/YARN-3237) | AppLogAggregatorImpl fails to log error cause | Major | . | Rushabh S Shah | Rushabh S Shah | +| [YARN-3231](https://issues.apache.org/jira/browse/YARN-3231) | FairScheduler: Changing queueMaxRunningApps interferes with pending jobs | Critical | . | Siqi Li | Siqi Li | +| [YARN-3227](https://issues.apache.org/jira/browse/YARN-3227) | Timeline renew delegation token fails when RM user's TGT is expired | Critical | . | Jonathan Eagles | Zhijie Shen | +| [YARN-3222](https://issues.apache.org/jira/browse/YARN-3222) | RMNodeImpl#ReconnectNodeTransition should send scheduler events in sequential order | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-3207](https://issues.apache.org/jira/browse/YARN-3207) | secondary filter matches entites which do not have the key being filtered for. | Major | timelineserver | Prakash Ramachandran | Zhijie Shen | +| [YARN-3194](https://issues.apache.org/jira/browse/YARN-3194) | RM should handle NMContainerStatuses sent by NM while registering if NM is Reconnected node | Blocker | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-3191](https://issues.apache.org/jira/browse/YARN-3191) | Log object should be initialized with its own class | Trivial | nodemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-3164](https://issues.apache.org/jira/browse/YARN-3164) | rmadmin command usage prints incorrect command name | Minor | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | +| [YARN-3160](https://issues.apache.org/jira/browse/YARN-3160) | Non-atomic operation on nodeUpdateQueue in RMNodeImpl | Major | resourcemanager | Chengbing Liu | Chengbing Liu | +| [YARN-3155](https://issues.apache.org/jira/browse/YARN-3155) | Refactor the exception handling code for TimelineClientImpl's retryOn method | Minor | . | Li Lu | Li Lu | +| [YARN-3151](https://issues.apache.org/jira/browse/YARN-3151) | On Failover tracking url wrong in application cli for KILLED application | Minor | client, resourcemanager | Bibin A Chundatt | Rohith Sharma K S | +| [YARN-3149](https://issues.apache.org/jira/browse/YARN-3149) | Typo in message for invalid application id | Trivial | resourcemanager | Bibin A Chundatt | Bibin A Chundatt | +| [YARN-3145](https://issues.apache.org/jira/browse/YARN-3145) | ConcurrentModificationException on CapacityScheduler ParentQueue#getQueueUserAclInfo | Major | . | Jian He | Tsuyoshi Ozawa | +| [YARN-3143](https://issues.apache.org/jira/browse/YARN-3143) | RM Apps REST API can return NPE or entries missing id and other fields | Major | webapp | Kendall Thrapp | Jason Lowe | +| [YARN-3131](https://issues.apache.org/jira/browse/YARN-3131) | YarnClientImpl should check FAILED and KILLED state in submitApplication | Major | . | Chang Li | Chang Li | +| [YARN-3113](https://issues.apache.org/jira/browse/YARN-3113) | Release audit warning for "Sorting icons.psd" | Major | . | Chang Li | Steve Loughran | +| [YARN-3104](https://issues.apache.org/jira/browse/YARN-3104) | RM generates new AMRM tokens every heartbeat between rolling and activation | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-3103](https://issues.apache.org/jira/browse/YARN-3103) | AMRMClientImpl does not update AMRM token properly | Blocker | client | Jason Lowe | Jason Lowe | +| [YARN-3101](https://issues.apache.org/jira/browse/YARN-3101) | In Fair Scheduler, fix canceling of reservations for exceeding max share | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3094](https://issues.apache.org/jira/browse/YARN-3094) | reset timer for liveness monitors after RM recovery | Major | resourcemanager | Jun Gong | Jun Gong | +| [YARN-3090](https://issues.apache.org/jira/browse/YARN-3090) | DeletionService can silently ignore deletion task failures | Major | nodemanager | Jason Lowe | Varun Saxena | +| [YARN-3089](https://issues.apache.org/jira/browse/YARN-3089) | LinuxContainerExecutor does not handle file arguments to deleteAsUser | Blocker | . | Jason Lowe | Eric Payne | +| [YARN-3088](https://issues.apache.org/jira/browse/YARN-3088) | LinuxContainerExecutor.deleteAsUser can throw NPE if native executor returns an error | Major | nodemanager | Jason Lowe | Eric Payne | +| [YARN-3082](https://issues.apache.org/jira/browse/YARN-3082) | Non thread safe access to systemCredentials in NodeHeartbeatResponse processing | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3079](https://issues.apache.org/jira/browse/YARN-3079) | Scheduler should also update maximumAllocation when updateNodeResource. | Major | . | zhihai xu | zhihai xu | +| [YARN-3078](https://issues.apache.org/jira/browse/YARN-3078) | LogCLIHelpers lacks of a blank space before string 'does not exist' | Minor | log-aggregation | sam liu | | +| [YARN-3074](https://issues.apache.org/jira/browse/YARN-3074) | Nodemanager dies when localizer runner tries to write to a full disk | Major | nodemanager | Jason Lowe | Varun Saxena | +| [YARN-3071](https://issues.apache.org/jira/browse/YARN-3071) | Remove invalid char from sample conf in doc of FairScheduler | Trivial | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [YARN-3064](https://issues.apache.org/jira/browse/YARN-3064) | TestRMRestart/TestContainerResourceUsage/TestNodeManagerResync failure with allocation timeout | Critical | scheduler | Wangda Tan | Jian He | +| [YARN-3058](https://issues.apache.org/jira/browse/YARN-3058) | Fix error message of tokens' activation delay configuration | Minor | . | Yi Liu | Yi Liu | +| [YARN-3055](https://issues.apache.org/jira/browse/YARN-3055) | The token is not renewed properly if it's shared by jobs (oozie) in DelegationTokenRenewer | Blocker | security | Yi Liu | Daryn Sharp | +| [YARN-3029](https://issues.apache.org/jira/browse/YARN-3029) | FSDownload.unpack() uses local locale for FS case conversion, may not work everywhere | Major | nodemanager | Steve Loughran | Varun Saxena | +| [YARN-3027](https://issues.apache.org/jira/browse/YARN-3027) | Scheduler should use totalAvailable resource from node instead of availableResource for maxAllocation | Major | . | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3024](https://issues.apache.org/jira/browse/YARN-3024) | LocalizerRunner should give DIE action when all resources are localized | Major | nodemanager | Chengbing Liu | Chengbing Liu | +| [YARN-3015](https://issues.apache.org/jira/browse/YARN-3015) | yarn classpath command should support same options as hadoop classpath. | Minor | scripts | Chris Nauroth | Varun Saxena | +| [YARN-3010](https://issues.apache.org/jira/browse/YARN-3010) | Fix recent findbug issue in AbstractYarnScheduler | Minor | . | Yi Liu | Yi Liu | +| [YARN-2997](https://issues.apache.org/jira/browse/YARN-2997) | NM keeps sending already-sent completed containers to RM until containers are removed from context | Major | nodemanager | Chengbing Liu | Chengbing Liu | +| [YARN-2993](https://issues.apache.org/jira/browse/YARN-2993) | Several fixes (missing acl check, error log msg ...) and some refinement in AdminService | Major | resourcemanager | Yi Liu | Yi Liu | +| [YARN-2992](https://issues.apache.org/jira/browse/YARN-2992) | ZKRMStateStore crashes due to session expiry | Blocker | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-2991](https://issues.apache.org/jira/browse/YARN-2991) | TestRMRestart.testDecomissionedNMsMetricsOnRMRestart intermittently fails on trunk | Blocker | . | Zhijie Shen | Rohith Sharma K S | +| [YARN-2990](https://issues.apache.org/jira/browse/YARN-2990) | FairScheduler's delay-scheduling always waits for node-local and rack-local delays, even for off-rack-only requests | Major | fairscheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-2988](https://issues.apache.org/jira/browse/YARN-2988) | Graph#save() may leak file descriptors | Minor | . | Ted Yu | Ted Yu | +| [YARN-2987](https://issues.apache.org/jira/browse/YARN-2987) | ClientRMService#getQueueInfo doesn't check app ACLs | Major | . | Jian He | Varun Saxena | +| [YARN-2983](https://issues.apache.org/jira/browse/YARN-2983) | NPE possible in ClientRMService#getQueueInfo | Major | resourcemanager | Varun Saxena | Varun Saxena | +| [YARN-2978](https://issues.apache.org/jira/browse/YARN-2978) | ResourceManager crashes with NPE while getting queue info | Critical | . | Jason Tufo | Varun Saxena | +| [YARN-2977](https://issues.apache.org/jira/browse/YARN-2977) | TestNMClient get failed intermittently | Major | . | Junping Du | Junping Du | +| [YARN-2975](https://issues.apache.org/jira/browse/YARN-2975) | FSLeafQueue app lists are accessed without required locks | Blocker | . | Karthik Kambatla | Karthik Kambatla | +| [YARN-2972](https://issues.apache.org/jira/browse/YARN-2972) | DelegationTokenRenewer thread pool never expands | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-2964](https://issues.apache.org/jira/browse/YARN-2964) | RM prematurely cancels tokens for jobs that submit jobs (oozie) | Blocker | resourcemanager | Daryn Sharp | Jian He | +| [YARN-2958](https://issues.apache.org/jira/browse/YARN-2958) | RMStateStore seems to unnecessarily and wrongly store sequence number separately | Blocker | resourcemanager | Zhijie Shen | Varun Saxena | +| [YARN-2956](https://issues.apache.org/jira/browse/YARN-2956) | Some yarn-site index linked pages are difficult to discover because are not in the side bar | Minor | documentation | Remus Rusanu | Masatake Iwasaki | +| [YARN-2952](https://issues.apache.org/jira/browse/YARN-2952) | Incorrect version check in RMStateStore | Major | . | Jian He | Rohith Sharma K S | +| [YARN-2945](https://issues.apache.org/jira/browse/YARN-2945) | FSLeafQueue#assignContainer - document the reason for using both write and read locks | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2936](https://issues.apache.org/jira/browse/YARN-2936) | YARNDelegationTokenIdentifier doesn't set proto.builder now | Major | . | Zhijie Shen | Varun Saxena | +| [YARN-2932](https://issues.apache.org/jira/browse/YARN-2932) | Add entry for "preemptable" status (enabled/disabled) to scheduler web UI and queue initialize/refresh logging | Major | . | Eric Payne | Eric Payne | +| [YARN-2931](https://issues.apache.org/jira/browse/YARN-2931) | PublicLocalizer may fail until directory is initialized by LocalizeRunner | Critical | nodemanager | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2922](https://issues.apache.org/jira/browse/YARN-2922) | ConcurrentModificationException in CapacityScheduler's LeafQueue | Major | capacityscheduler, resourcemanager, scheduler | Jason Tufo | Rohith Sharma K S | +| [YARN-2917](https://issues.apache.org/jira/browse/YARN-2917) | Potential deadlock in AsyncDispatcher when system.exit called in AsyncDispatcher#dispatch and AsyscDispatcher#serviceStop from shutdown hook | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2912](https://issues.apache.org/jira/browse/YARN-2912) | Jersey Tests failing with port in use | Major | test | Steve Loughran | Varun Saxena | +| [YARN-2910](https://issues.apache.org/jira/browse/YARN-2910) | FSLeafQueue can throw ConcurrentModificationException | Major | fairscheduler | Wilfred Spiegelenburg | Wilfred Spiegelenburg | +| [YARN-2907](https://issues.apache.org/jira/browse/YARN-2907) | SchedulerNode#toString should print all resource detail instead of only memory. | Trivial | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2906](https://issues.apache.org/jira/browse/YARN-2906) | CapacitySchedulerPage shows HTML tags for a queue's Active Users | Major | capacityscheduler | Jason Lowe | Jason Lowe | +| [YARN-2905](https://issues.apache.org/jira/browse/YARN-2905) | AggregatedLogsBlock page can infinitely loop if the aggregated log file is corrupted | Blocker | . | Jason Lowe | Varun Saxena | +| [YARN-2899](https://issues.apache.org/jira/browse/YARN-2899) | Run TestDockerContainerExecutorWithMocks on Linux only | Minor | nodemanager, test | Ming Ma | Ming Ma | +| [YARN-2897](https://issues.apache.org/jira/browse/YARN-2897) | CrossOriginFilter needs more log statements | Major | . | Mit Desai | Mit Desai | +| [YARN-2894](https://issues.apache.org/jira/browse/YARN-2894) | When ACL's are enabled, if RM switches then application can not be viewed from web. | Major | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2874](https://issues.apache.org/jira/browse/YARN-2874) | Dead lock in "DelegationTokenRenewer" which blocks RM to execute any further apps | Blocker | resourcemanager | Naganarasimha G R | Naganarasimha G R | +| [YARN-2870](https://issues.apache.org/jira/browse/YARN-2870) | Update examples in document of Timeline Server | Trivial | documentation, timelineserver | Masatake Iwasaki | Masatake Iwasaki | +| [YARN-2869](https://issues.apache.org/jira/browse/YARN-2869) | CapacityScheduler should trim sub queue names when parse configuration | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2865](https://issues.apache.org/jira/browse/YARN-2865) | Application recovery continuously fails with "Application with id already present. Cannot duplicate" | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2861](https://issues.apache.org/jira/browse/YARN-2861) | Timeline DT secret manager should not reuse the RM's configs. | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-2857](https://issues.apache.org/jira/browse/YARN-2857) | ConcurrentModificationException in ContainerLogAppender | Critical | . | Mohammad Kamrul Islam | Mohammad Kamrul Islam | +| [YARN-2856](https://issues.apache.org/jira/browse/YARN-2856) | Application recovery throw InvalidStateTransitonException: Invalid event: ATTEMPT\_KILLED at ACCEPTED | Critical | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2847](https://issues.apache.org/jira/browse/YARN-2847) | Linux native container executor segfaults if default banned user detected | Major | nodemanager | Jason Lowe | Olaf Flebbe | +| [YARN-2816](https://issues.apache.org/jira/browse/YARN-2816) | NM fail to start with NPE during container recovery | Major | nodemanager | zhihai xu | zhihai xu | +| [YARN-2815](https://issues.apache.org/jira/browse/YARN-2815) | Remove jline from hadoop-yarn-server-common | Major | . | Ferdinand Xu | Ferdinand Xu | +| [YARN-2811](https://issues.apache.org/jira/browse/YARN-2811) | In Fair Scheduler, reservation fulfillments shouldn't ignore max share | Major | . | Siqi Li | Siqi Li | +| [YARN-2809](https://issues.apache.org/jira/browse/YARN-2809) | Implement workaround for linux kernel panic when removing cgroup | Major | nodemanager | Nathan Roberts | Nathan Roberts | +| [YARN-2808](https://issues.apache.org/jira/browse/YARN-2808) | yarn client tool can not list app\_attempt's container info correctly | Major | client | Gordon Wang | Naganarasimha G R | +| [YARN-2749](https://issues.apache.org/jira/browse/YARN-2749) | Some testcases from TestLogAggregationService fails in trunk | Major | . | Xuan Gong | Xuan Gong | +| [YARN-2742](https://issues.apache.org/jira/browse/YARN-2742) | FairSchedulerConfiguration should allow extra spaces between value and unit | Minor | fairscheduler | Sangjin Lee | Wei Yan | +| [YARN-2735](https://issues.apache.org/jira/browse/YARN-2735) | diskUtilizationPercentageCutoff and diskUtilizationSpaceCutoff are initialized twice in DirectoryCollection | Trivial | nodemanager | zhihai xu | zhihai xu | +| [YARN-2731](https://issues.apache.org/jira/browse/YARN-2731) | Fixed RegisterApplicationMasterResponsePBImpl to properly invoke maybeInitBuilder | Major | . | Carlo Curino | Carlo Curino | +| [YARN-2713](https://issues.apache.org/jira/browse/YARN-2713) | "RM Home" link in NM should point to one of the RMs in an HA setup | Major | resourcemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-2697](https://issues.apache.org/jira/browse/YARN-2697) | RMAuthenticationHandler is no longer useful | Major | resourcemanager | Zhijie Shen | haosdent | +| [YARN-2675](https://issues.apache.org/jira/browse/YARN-2675) | containersKilled metrics is not updated when the container is killed during localization | Major | nodemanager | zhihai xu | zhihai xu | +| [YARN-2637](https://issues.apache.org/jira/browse/YARN-2637) | maximum-am-resource-percent could be respected for both LeafQueue/User when trying to activate applications. | Critical | resourcemanager | Wangda Tan | Craig Welch | +| [YARN-2461](https://issues.apache.org/jira/browse/YARN-2461) | Fix PROCFS\_USE\_SMAPS\_BASED\_RSS\_ENABLED property in YarnConfiguration | Minor | . | Ray Chiang | Ray Chiang | +| [YARN-2432](https://issues.apache.org/jira/browse/YARN-2432) | RMStateStore should process the pending events before close | Major | resourcemanager | Varun Saxena | Varun Saxena | +| [YARN-2414](https://issues.apache.org/jira/browse/YARN-2414) | RM web UI: app page will crash if app is failed before any attempt has been created | Major | webapp | Zhijie Shen | Wangda Tan | +| [YARN-2356](https://issues.apache.org/jira/browse/YARN-2356) | yarn status command for non-existent application/application attempt/container is too verbose | Minor | client | Sunil G | Sunil G | +| [YARN-2340](https://issues.apache.org/jira/browse/YARN-2340) | NPE thrown when RM restart after queue is STOPPED. There after RM can not recovery application's and remain in standby | Critical | resourcemanager, scheduler | Nishan Shetty | Rohith Sharma K S | +| [YARN-2315](https://issues.apache.org/jira/browse/YARN-2315) | FairScheduler: Set current capacity in addition to capacity | Major | . | zhihai xu | zhihai xu | +| [YARN-2246](https://issues.apache.org/jira/browse/YARN-2246) | Job History Link in RM UI is redirecting to the URL which contains Job Id twice | Major | webapp | Devaraj K | Devaraj K | +| [YARN-2243](https://issues.apache.org/jira/browse/YARN-2243) | Order of arguments for Preconditions.checkNotNull() is wrong in SchedulerApplicationAttempt ctor | Minor | . | Ted Yu | Devaraj K | +| [YARN-2230](https://issues.apache.org/jira/browse/YARN-2230) | Fix description of yarn.scheduler.maximum-allocation-vcores in yarn-default.xml (or code) | Minor | client, documentation, scheduler | Adam Kawa | Vijay Bhat | +| [YARN-2136](https://issues.apache.org/jira/browse/YARN-2136) | RMStateStore can explicitly handle store/update events when fenced | Major | . | Jian He | Varun Saxena | +| [YARN-1703](https://issues.apache.org/jira/browse/YARN-1703) | Too many connections are opened for proxy server when applicationMaster UI is accessed. | Critical | . | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-1615](https://issues.apache.org/jira/browse/YARN-1615) | Fix typos in description about delay scheduling | Trivial | documentation, scheduler | Akira AJISAKA | Akira AJISAKA | +| [YARN-1580](https://issues.apache.org/jira/browse/YARN-1580) | Documentation error regarding "container-allocation.expiry-interval-ms" | Trivial | documentation | German Florez-Larrahondo | Brahma Reddy Battula | +| [YARN-1453](https://issues.apache.org/jira/browse/YARN-1453) | [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments | Minor | . | Andrew Purtell | Akira AJISAKA | +| [YARN-1237](https://issues.apache.org/jira/browse/YARN-1237) | Description for yarn.nodemanager.aux-services in yarn-default.xml is misleading | Minor | documentation | Hitesh Shah | Brahma Reddy Battula | +| [YARN-933](https://issues.apache.org/jira/browse/YARN-933) | Potential InvalidStateTransitonException: Invalid event: LAUNCHED at FINAL\_SAVING | Major | resourcemanager | J.Andreina | Rohith Sharma K S | +| [YARN-570](https://issues.apache.org/jira/browse/YARN-570) | Time strings are formated in different timezone | Major | webapp | Peng Zhang | Akira AJISAKA | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11535](https://issues.apache.org/jira/browse/HADOOP-11535) | TableMapping related tests failed due to 'successful' resolving of invalid test hostname | Minor | . | Kai Zheng | Kai Zheng | +| [HADOOP-11432](https://issues.apache.org/jira/browse/HADOOP-11432) | Fix SymlinkBaseTest#testCreateLinkUsingPartQualPath2 | Major | fs | Liang Xie | Liang Xie | +| [HADOOP-11358](https://issues.apache.org/jira/browse/HADOOP-11358) | Tests for encryption/decryption with IV calculation overflow | Major | security, test | Yi Liu | Yi Liu | +| [HADOOP-11165](https://issues.apache.org/jira/browse/HADOOP-11165) | TestUTF8 fails when run against java 8 | Minor | test | Ted Yu | Stephen Chu | +| [HADOOP-11125](https://issues.apache.org/jira/browse/HADOOP-11125) | Remove redundant tests in TestOsSecureRandom | Major | . | Ted Yu | Masanori Oyama | +| [HADOOP-10668](https://issues.apache.org/jira/browse/HADOOP-10668) | TestZKFailoverControllerStress#testExpireBackAndForth occasionally fails | Major | test | Ted Yu | Ming Ma | +| [HDFS-7914](https://issues.apache.org/jira/browse/HDFS-7914) | TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use | Major | . | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7585](https://issues.apache.org/jira/browse/HDFS-7585) | Get TestEnhancedByteBufferAccess working on CPU architectures with page sizes other than 4096 | Major | test | sam liu | sam liu | +| [HDFS-7475](https://issues.apache.org/jira/browse/HDFS-7475) | Make TestLazyPersistFiles#testLazyPersistBlocksAreSaved deterministic | Major | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7448](https://issues.apache.org/jira/browse/HDFS-7448) | TestBookKeeperHACheckpoints fails in trunk build | Minor | . | Ted Yu | Akira AJISAKA | +| [YARN-3070](https://issues.apache.org/jira/browse/YARN-3070) | TestRMAdminCLI#testHelp fails for transitionToActive command | Minor | . | Ted Yu | Junping Du | +| [YARN-2930](https://issues.apache.org/jira/browse/YARN-2930) | TestRMRestart#testRMRestartRecoveringNodeLabelManager sometimes fails against Java 7 & 8 | Minor | . | Ted Yu | Wangda Tan | +| [YARN-1979](https://issues.apache.org/jira/browse/YARN-1979) | TestDirectoryCollection fails when the umask is unusual | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli | +| [YARN-1537](https://issues.apache.org/jira/browse/YARN-1537) | TestLocalResourcesTrackerImpl.testLocalResourceCache often failed | Major | nodemanager | Hong Shen | Xuan Gong | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11796](https://issues.apache.org/jira/browse/HADOOP-11796) | Skip TestShellBasedIdMapping.testStaticMapUpdate on Windows | Minor | test | Xiaoyu Yao | Xiaoyu Yao | +| [HADOOP-11710](https://issues.apache.org/jira/browse/HADOOP-11710) | Make CryptoOutputStream behave like DFSOutputStream wrt synchronization | Critical | fs | Sean Busbey | Sean Busbey | +| [HADOOP-11670](https://issues.apache.org/jira/browse/HADOOP-11670) | Regression: s3a auth setup broken | Blocker | fs/s3 | Adam Budde | Adam Budde | +| [HADOOP-11600](https://issues.apache.org/jira/browse/HADOOP-11600) | Fix up source codes to be compiled with Guava 17.0 | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [HADOOP-11584](https://issues.apache.org/jira/browse/HADOOP-11584) | s3a file block size set to 0 in getFileStatus | Blocker | fs/s3 | Dan Hecht | Brahma Reddy Battula | +| [HADOOP-11570](https://issues.apache.org/jira/browse/HADOOP-11570) | S3AInputStream.close() downloads the remaining bytes of the object from S3 | Major | fs/s3 | Dan Hecht | Dan Hecht | +| [HADOOP-11522](https://issues.apache.org/jira/browse/HADOOP-11522) | Update S3A Documentation | Major | fs/s3 | Thomas Demoor | Thomas Demoor | +| [HADOOP-11521](https://issues.apache.org/jira/browse/HADOOP-11521) | Make connection timeout configurable in s3a | Minor | fs/s3 | Thomas Demoor | Thomas Demoor | +| [HADOOP-11465](https://issues.apache.org/jira/browse/HADOOP-11465) | Fix findbugs warnings in hadoop-gridmix | Major | . | Varun Saxena | Varun Saxena | +| [HADOOP-11429](https://issues.apache.org/jira/browse/HADOOP-11429) | Findbugs warnings in hadoop extras | Major | . | Varun Saxena | Varun Saxena | +| [HADOOP-11389](https://issues.apache.org/jira/browse/HADOOP-11389) | Clean up byte to string encoding issues in hadoop-common | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-11381](https://issues.apache.org/jira/browse/HADOOP-11381) | Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack | Major | . | Li Lu | Li Lu | +| [HADOOP-11379](https://issues.apache.org/jira/browse/HADOOP-11379) | Fix new findbugs warnings in hadoop-auth\* | Major | . | Li Lu | Li Lu | +| [HADOOP-11378](https://issues.apache.org/jira/browse/HADOOP-11378) | Fix new findbugs warnings in hadoop-kms | Major | . | Li Lu | Li Lu | +| [HADOOP-11377](https://issues.apache.org/jira/browse/HADOOP-11377) | jdiff failing on java 7 and java 8, "Null.java" not found | Major | build | Steve Loughran | Tsuyoshi Ozawa | +| [HADOOP-11372](https://issues.apache.org/jira/browse/HADOOP-11372) | Fix new findbugs warnings in mapreduce-examples | Major | . | Li Lu | Li Lu | +| [HADOOP-11370](https://issues.apache.org/jira/browse/HADOOP-11370) | Fix new findbug warnings hadoop-yarn | Major | . | Zhijie Shen | | +| [HADOOP-11369](https://issues.apache.org/jira/browse/HADOOP-11369) | Fix new findbugs warnings in hadoop-mapreduce-client, non-core directories | Major | . | Li Lu | Li Lu | +| [HADOOP-11367](https://issues.apache.org/jira/browse/HADOOP-11367) | Fix warnings from findbugs 3.0 in hadoop-streaming | Major | . | Li Lu | Li Lu | +| [HADOOP-11317](https://issues.apache.org/jira/browse/HADOOP-11317) | Increment SLF4J version to 1.7.10 | Major | build | Steve Loughran | Tim Robertson | +| [HADOOP-11183](https://issues.apache.org/jira/browse/HADOOP-11183) | Memory-based S3AOutputstream | Major | fs/s3 | Thomas Demoor | Thomas Demoor | +| [HADOOP-10651](https://issues.apache.org/jira/browse/HADOOP-10651) | Add ability to restrict service access using IP addresses and hostnames | Major | security | Benoy Antony | Benoy Antony | +| [HADOOP-10574](https://issues.apache.org/jira/browse/HADOOP-10574) | Bump the maven plugin versions too -moving the numbers into properties | Major | build | Steve Loughran | Akira AJISAKA | +| [HADOOP-10482](https://issues.apache.org/jira/browse/HADOOP-10482) | Fix various findbugs warnings in hadoop-common | Major | . | Haohui Mai | Haohui Mai | +| [HADOOP-10478](https://issues.apache.org/jira/browse/HADOOP-10478) | Fix new findbugs warnings in hadoop-maven-plugins | Major | . | Haohui Mai | Li Lu | +| [HADOOP-10476](https://issues.apache.org/jira/browse/HADOOP-10476) | Bumping the findbugs version to 3.0.0 | Major | build | Haohui Mai | Haohui Mai | +| [HDFS-8039](https://issues.apache.org/jira/browse/HDFS-8039) | Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows | Minor | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-8038](https://issues.apache.org/jira/browse/HDFS-8038) | PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. | Minor | tools | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7954](https://issues.apache.org/jira/browse/HDFS-7954) | TestBalancer#testBalancerWithPinnedBlocks failed on Windows | Minor | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7951](https://issues.apache.org/jira/browse/HDFS-7951) | Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux | Major | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7950](https://issues.apache.org/jira/browse/HDFS-7950) | Fix TestFsDatasetImpl#testAddVolumes failure on Windows | Major | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7948](https://issues.apache.org/jira/browse/HDFS-7948) | TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows | Major | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7946](https://issues.apache.org/jira/browse/HDFS-7946) | TestDataNodeVolumeFailureReporting NPE on Windows | Major | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7940](https://issues.apache.org/jira/browse/HDFS-7940) | Add tracing to DFSClient#setQuotaByStorageType | Major | hdfs-client | Rakesh R | Rakesh R | +| [HDFS-7903](https://issues.apache.org/jira/browse/HDFS-7903) | Cannot recover block after truncate and delete snapshot | Blocker | datanode, namenode | Tsz Wo Nicholas Sze | Plamen Jeliazkov | +| [HDFS-7855](https://issues.apache.org/jira/browse/HDFS-7855) | Separate class Packet from DFSOutputStream | Major | hdfs-client | Li Bo | Li Bo | +| [HDFS-7843](https://issues.apache.org/jira/browse/HDFS-7843) | A truncated file is corrupted after rollback from a rolling upgrade | Blocker | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7838](https://issues.apache.org/jira/browse/HDFS-7838) | Expose truncate API for libhdfs | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7824](https://issues.apache.org/jira/browse/HDFS-7824) | GetContentSummary API and its namenode implementation for Storage Type Quota/Usage | Major | datanode, namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7819](https://issues.apache.org/jira/browse/HDFS-7819) | Log WARN message for the blocks which are not in Block ID based layout | Major | datanode | Rakesh R | Rakesh R | +| [HDFS-7814](https://issues.apache.org/jira/browse/HDFS-7814) | Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota" | Minor | datanode, namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7811](https://issues.apache.org/jira/browse/HDFS-7811) | Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage | Minor | namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7806](https://issues.apache.org/jira/browse/HDFS-7806) | Refactor: move StorageType from hadoop-hdfs to hadoop-common | Minor | datanode, namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7776](https://issues.apache.org/jira/browse/HDFS-7776) | Adding additional unit tests for Quota By Storage Type | Major | test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7775](https://issues.apache.org/jira/browse/HDFS-7775) | Use consistent naming for NN-internal quota related types and functions | Minor | datanode, namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7760](https://issues.apache.org/jira/browse/HDFS-7760) | Document truncate for WebHDFS. | Minor | documentation | Yi Liu | Konstantin Shvachko | +| [HDFS-7746](https://issues.apache.org/jira/browse/HDFS-7746) | Add a test randomly mixing append, truncate and snapshot | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7740](https://issues.apache.org/jira/browse/HDFS-7740) | Test truncate with DataNodes restarting | Major | test | Konstantin Shvachko | Yi Liu | +| [HDFS-7738](https://issues.apache.org/jira/browse/HDFS-7738) | Add more tests for truncate | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-7723](https://issues.apache.org/jira/browse/HDFS-7723) | Quota By Storage Type namenode implemenation | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7720](https://issues.apache.org/jira/browse/HDFS-7720) | Quota by Storage Type API, tools and ClientNameNode Protocol changes | Major | datanode, namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7700](https://issues.apache.org/jira/browse/HDFS-7700) | Document quota support for storage types | Major | documentation | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7681](https://issues.apache.org/jira/browse/HDFS-7681) | Fix ReplicaInputStream constructor to take InputStreams | Major | datanode | Joe Pallas | Joe Pallas | +| [HDFS-7677](https://issues.apache.org/jira/browse/HDFS-7677) | DistributedFileSystem#truncate should resolve symlinks | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7676](https://issues.apache.org/jira/browse/HDFS-7676) | Fix TestFileTruncate to avoid bug of HDFS-7611 | Major | test | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-7659](https://issues.apache.org/jira/browse/HDFS-7659) | We should check the new length of truncate can't be a negative value. | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7656](https://issues.apache.org/jira/browse/HDFS-7656) | Expose truncate API for HDFS httpfs | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7655](https://issues.apache.org/jira/browse/HDFS-7655) | Expose truncate API for Web HDFS | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7643](https://issues.apache.org/jira/browse/HDFS-7643) | Test case to ensure lazy persist files cannot be truncated | Major | test | Arpit Agarwal | Yi Liu | +| [HDFS-7638](https://issues.apache.org/jira/browse/HDFS-7638) | Small fix and few refinements for FSN#truncate | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7634](https://issues.apache.org/jira/browse/HDFS-7634) | Disallow truncation of Lazy persist files | Major | datanode, namenode | Yi Liu | Yi Liu | +| [HDFS-7623](https://issues.apache.org/jira/browse/HDFS-7623) | Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace | Major | . | Yi Liu | Yi Liu | +| [HDFS-7589](https://issues.apache.org/jira/browse/HDFS-7589) | Break the dependency between libnative\_mini\_dfs and libhdfs | Major | libhdfs | Zhanwei Wang | Zhanwei Wang | +| [HDFS-7573](https://issues.apache.org/jira/browse/HDFS-7573) | Consolidate the implementation of delete() into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7543](https://issues.apache.org/jira/browse/HDFS-7543) | Avoid path resolution when getting FileStatus for audit logs | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7528](https://issues.apache.org/jira/browse/HDFS-7528) | Consolidate symlink-related implementation into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7509](https://issues.apache.org/jira/browse/HDFS-7509) | Avoid resolving path multiple times | Major | . | Jing Zhao | Jing Zhao | +| [HDFS-7506](https://issues.apache.org/jira/browse/HDFS-7506) | Consolidate implementation of setting inode attributes into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7498](https://issues.apache.org/jira/browse/HDFS-7498) | Simplify the logic in INodesInPath | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-7486](https://issues.apache.org/jira/browse/HDFS-7486) | Consolidate XAttr-related implementation into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7476](https://issues.apache.org/jira/browse/HDFS-7476) | Consolidate ACL-related operations to a single class | Major | namenode | Haohui Mai | Haohui Mai | +| [HDFS-7474](https://issues.apache.org/jira/browse/HDFS-7474) | Avoid resolving path in FSPermissionChecker | Major | namenode | Jing Zhao | Jing Zhao | +| [HDFS-7468](https://issues.apache.org/jira/browse/HDFS-7468) | Moving verify\* functions to corresponding classes | Major | . | Li Lu | Li Lu | +| [HDFS-7467](https://issues.apache.org/jira/browse/HDFS-7467) | Provide storage tier information for a directory via fsck | Major | balancer & mover | Benoy Antony | Benoy Antony | +| [HDFS-7462](https://issues.apache.org/jira/browse/HDFS-7462) | Consolidate implementation of mkdirs() into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7459](https://issues.apache.org/jira/browse/HDFS-7459) | Consolidate cache-related implementation in FSNamesystem into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7450](https://issues.apache.org/jira/browse/HDFS-7450) | Consolidate the implementation of GetFileInfo(), GetListings() and GetContentSummary() into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7440](https://issues.apache.org/jira/browse/HDFS-7440) | Consolidate snapshot related operations in a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7438](https://issues.apache.org/jira/browse/HDFS-7438) | Consolidate the implementation of rename() into a single class | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7420](https://issues.apache.org/jira/browse/HDFS-7420) | Delegate permission checks to FSDirectory | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7415](https://issues.apache.org/jira/browse/HDFS-7415) | Move FSNameSystem.resolvePath() to FSDirectory | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7413](https://issues.apache.org/jira/browse/HDFS-7413) | Some unit tests should use NameNodeProtocols instead of FSNameSystem | Major | test | Haohui Mai | Haohui Mai | +| [HDFS-7412](https://issues.apache.org/jira/browse/HDFS-7412) | Move RetryCache to NameNodeRpcServer | Major | . | Haohui Mai | Haohui Mai | +| [HDFS-7310](https://issues.apache.org/jira/browse/HDFS-7310) | Mover can give first priority to local DN if it has target storage type available in local DN | Major | balancer & mover | Uma Maheswara Rao G | Vinayakumar B | +| [HDFS-7254](https://issues.apache.org/jira/browse/HDFS-7254) | Add documentation for hot swaping DataNode drives | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-7209](https://issues.apache.org/jira/browse/HDFS-7209) | Populate EDEK cache when creating encryption zone | Major | encryption, performance | Yi Liu | Yi Liu | +| [HDFS-7189](https://issues.apache.org/jira/browse/HDFS-7189) | Add trace spans for DFSClient metadata operations | Major | datanode, namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7058](https://issues.apache.org/jira/browse/HDFS-7058) | Tests for truncate CLI | Major | test | Konstantin Shvachko | Dasha Boudnik | +| [HDFS-7056](https://issues.apache.org/jira/browse/HDFS-7056) | Snapshot support for truncate | Major | namenode | Konstantin Shvachko | Plamen Jeliazkov | +| [HDFS-7055](https://issues.apache.org/jira/browse/HDFS-7055) | Add tracing to DFSInputStream | Major | datanode, namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7054](https://issues.apache.org/jira/browse/HDFS-7054) | Make DFSOutputStream tracing more fine-grained | Major | datanode, namenode | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-7035](https://issues.apache.org/jira/browse/HDFS-7035) | Make adding a new data directory to the DataNode an atomic operation and improve error handling | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6877](https://issues.apache.org/jira/browse/HDFS-6877) | Avoid calling checkDisk when an HDFS volume is removed during a write. | Major | datanode | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-6824](https://issues.apache.org/jira/browse/HDFS-6824) | Additional user documentation for HDFS encryption. | Minor | documentation | Andrew Wang | Andrew Wang | +| [HDFS-6803](https://issues.apache.org/jira/browse/HDFS-6803) | Documenting DFSClient#DFSInputStream expectations reading and preading in concurrent context | Major | hdfs-client | stack | stack | +| [HDFS-6673](https://issues.apache.org/jira/browse/HDFS-6673) | Add delimited format support to PB OIV tool | Minor | . | Lei (Eddy) Xu | Lei (Eddy) Xu | +| [HDFS-5928](https://issues.apache.org/jira/browse/HDFS-5928) | show namespace and namenode ID on NN dfshealth page | Major | . | Siqi Li | Siqi Li | +| [HDFS-5782](https://issues.apache.org/jira/browse/HDFS-5782) | BlockListAsLongs should take lists of Replicas rather than concrete classes | Minor | datanode | David Powell | Joe Pallas | +| [HDFS-5631](https://issues.apache.org/jira/browse/HDFS-5631) | Expose interfaces required by FsDatasetSpi implementations | Minor | datanode | David Powell | Joe Pallas | +| [HDFS-4266](https://issues.apache.org/jira/browse/HDFS-4266) | BKJM: Separate write and ack quorum | Major | ha | Ivan Kelly | Rakesh R | +| [HDFS-4265](https://issues.apache.org/jira/browse/HDFS-4265) | BKJM doesn't take advantage of speculative reads | Major | ha | Ivan Kelly | Rakesh R | +| [YARN-3430](https://issues.apache.org/jira/browse/YARN-3430) | RMAppAttempt headroom data is missing in RM Web UI | Blocker | resourcemanager, webapp, yarn | Xuan Gong | Xuan Gong | +| [YARN-3379](https://issues.apache.org/jira/browse/YARN-3379) | Missing data in localityTable and ResourceRequests table in RM WebUI | Major | resourcemanager, webapp, yarn | Xuan Gong | Xuan Gong | +| [YARN-3300](https://issues.apache.org/jira/browse/YARN-3300) | outstanding\_resource\_requests table should not be shown in AHS | Major | resourcemanager | Xuan Gong | Xuan Gong | +| [YARN-3265](https://issues.apache.org/jira/browse/YARN-3265) | CapacityScheduler deadlock when computing absolute max avail capacity (fix for trunk/branch-2) | Blocker | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-3171](https://issues.apache.org/jira/browse/YARN-3171) | Sort by Application id, AppAttempt & ContainerID doesn't work in ATS / RM web ui | Minor | timelineserver | Jeff Zhang | Naganarasimha G R | +| [YARN-3132](https://issues.apache.org/jira/browse/YARN-3132) | RMNodeLabelsManager should remove node from node-to-label mapping when node becomes deactivated | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-3124](https://issues.apache.org/jira/browse/YARN-3124) | Capacity Scheduler LeafQueue/ParentQueue should use QueueCapacities to track capacities-by-label | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-3122](https://issues.apache.org/jira/browse/YARN-3122) | Metrics for container's actual CPU usage | Major | nodemanager | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3099](https://issues.apache.org/jira/browse/YARN-3099) | Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage to track used-resources-by-label. | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-3098](https://issues.apache.org/jira/browse/YARN-3098) | Create common QueueCapacities class in Capacity Scheduler to track capacities-by-labels of queues | Major | capacityscheduler | Wangda Tan | Wangda Tan | +| [YARN-3092](https://issues.apache.org/jira/browse/YARN-3092) | Create common ResourceUsage class to track labeled resource usages in Capacity Scheduler | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-3076](https://issues.apache.org/jira/browse/YARN-3076) | Add API/Implementation to YarnClient to retrieve label-to-node mapping | Major | client | Varun Saxena | Varun Saxena | +| [YARN-3075](https://issues.apache.org/jira/browse/YARN-3075) | NodeLabelsManager implementation to retrieve label to node mapping | Major | resourcemanager | Varun Saxena | Varun Saxena | +| [YARN-3028](https://issues.apache.org/jira/browse/YARN-3028) | Better syntax for replaceLabelsOnNode in RMAdmin CLI | Major | api, client, resourcemanager | Jian He | Rohith Sharma K S | +| [YARN-3019](https://issues.apache.org/jira/browse/YARN-3019) | Make work-preserving-recovery the default mechanism for RM recovery | Major | resourcemanager | Jian He | Jian He | +| [YARN-3014](https://issues.apache.org/jira/browse/YARN-3014) | Replaces labels on a host should update all NM's labels on that host | Major | . | Wangda Tan | Wangda Tan | +| [YARN-3011](https://issues.apache.org/jira/browse/YARN-3011) | NM dies because of the failure of resource localization | Major | nodemanager | Wang Hao | Varun Saxena | +| [YARN-2998](https://issues.apache.org/jira/browse/YARN-2998) | Abstract out scheduler independent PlanFollower components | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2994](https://issues.apache.org/jira/browse/YARN-2994) | Document work-preserving RM restart | Major | resourcemanager | Jian He | Jian He | +| [YARN-2984](https://issues.apache.org/jira/browse/YARN-2984) | Metrics for container's actual memory usage | Major | nodemanager | Karthik Kambatla | Karthik Kambatla | +| [YARN-2971](https://issues.apache.org/jira/browse/YARN-2971) | RM uses conf instead of token service address to renew timeline delegation tokens | Major | timelineserver | Jonathan Eagles | Jonathan Eagles | +| [YARN-2970](https://issues.apache.org/jira/browse/YARN-2970) | NodeLabel operations in RMAdmin CLI get missing in help command. | Minor | api, client, resourcemanager | Junping Du | Varun Saxena | +| [YARN-2944](https://issues.apache.org/jira/browse/YARN-2944) | InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance | Minor | . | Chris Trezzo | Chris Trezzo | +| [YARN-2943](https://issues.apache.org/jira/browse/YARN-2943) | Add a node-labels page in RM web UI | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2933](https://issues.apache.org/jira/browse/YARN-2933) | Capacity Scheduler preemption policy should only consider capacity without labels temporarily | Major | capacityscheduler | Wangda Tan | Mayank Bansal | +| [YARN-2927](https://issues.apache.org/jira/browse/YARN-2927) | InMemorySCMStore properties are inconsistent | Major | . | Ray Chiang | Ray Chiang | +| [YARN-2924](https://issues.apache.org/jira/browse/YARN-2924) | Node to labels mapping should not transfer to lowercase when adding from RMAdminCLI | Major | client | Wangda Tan | Wangda Tan | +| [YARN-2920](https://issues.apache.org/jira/browse/YARN-2920) | CapacityScheduler should be notified when labels on nodes changed | Major | . | Wangda Tan | Wangda Tan | +| [YARN-2914](https://issues.apache.org/jira/browse/YARN-2914) | Potential race condition in Singleton implementation of SharedCacheUploaderMetrics, CleanerMetrics, ClientSCMMetrics | Minor | . | Ted Yu | Varun Saxena | +| [YARN-2881](https://issues.apache.org/jira/browse/YARN-2881) | Implement PlanFollower for FairScheduler | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2880](https://issues.apache.org/jira/browse/YARN-2880) | Add a test in TestRMRestart to make sure node labels will be recovered if it is enabled | Major | resourcemanager | Wangda Tan | Rohith Sharma K S | +| [YARN-2807](https://issues.apache.org/jira/browse/YARN-2807) | Option "--forceactive" not works as described in usage of "yarn rmadmin -transitionToActive" | Minor | documentation, resourcemanager | Wangda Tan | Masatake Iwasaki | +| [YARN-2800](https://issues.apache.org/jira/browse/YARN-2800) | Remove MemoryNodeLabelsStore and add a way to enable/disable node labels feature | Major | client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2786](https://issues.apache.org/jira/browse/YARN-2786) | Create yarn cluster CLI to enable list node labels collection | Major | api, client, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2766](https://issues.apache.org/jira/browse/YARN-2766) | ApplicationHistoryManager is expected to return a sorted list of apps/attempts/containers | Major | timelineserver | Robert Kanter | Robert Kanter | +| [YARN-2765](https://issues.apache.org/jira/browse/YARN-2765) | Add leveldb-based implementation for RMStateStore | Major | resourcemanager | Jason Lowe | Jason Lowe | +| [YARN-2762](https://issues.apache.org/jira/browse/YARN-2762) | RMAdminCLI node-labels-related args should be trimmed and checked before sending to RM | Minor | resourcemanager | Rohith Sharma K S | Rohith Sharma K S | +| [YARN-2738](https://issues.apache.org/jira/browse/YARN-2738) | Add FairReservationSystem for FairScheduler | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2712](https://issues.apache.org/jira/browse/YARN-2712) | TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks | Major | resourcemanager | Tsuyoshi Ozawa | Tsuyoshi Ozawa | +| [YARN-2694](https://issues.apache.org/jira/browse/YARN-2694) | Ensure only single node labels specified in resource request / host, and node label expression only specified when resourceName=ANY | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2690](https://issues.apache.org/jira/browse/YARN-2690) | Make ReservationSystem and its dependent classes independent of Scheduler type | Major | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-2683](https://issues.apache.org/jira/browse/YARN-2683) | registry config options: document and move to core-default | Major | api, resourcemanager | Steve Loughran | Steve Loughran | +| [YARN-2616](https://issues.apache.org/jira/browse/YARN-2616) | Add CLI client to the registry to list, view and manipulate entries | Major | client | Steve Loughran | Akshay Radia | +| [YARN-2543](https://issues.apache.org/jira/browse/YARN-2543) | Resource usage should be published to the timeline server as well | Major | timelineserver | Zhijie Shen | Naganarasimha G R | +| [YARN-2404](https://issues.apache.org/jira/browse/YARN-2404) | Remove ApplicationAttemptState and ApplicationState class in RMStateStore class | Major | . | Jian He | Tsuyoshi Ozawa | +| [YARN-2375](https://issues.apache.org/jira/browse/YARN-2375) | Allow enabling/disabling timeline server per framework | Major | . | Jonathan Eagles | Mit Desai | +| [YARN-2236](https://issues.apache.org/jira/browse/YARN-2236) | Shared Cache uploader service on the Node Manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2217](https://issues.apache.org/jira/browse/YARN-2217) | Shared cache client side changes | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2203](https://issues.apache.org/jira/browse/YARN-2203) | Web UI for cache manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2189](https://issues.apache.org/jira/browse/YARN-2189) | Admin service for cache manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2188](https://issues.apache.org/jira/browse/YARN-2188) | Client service for cache manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2186](https://issues.apache.org/jira/browse/YARN-2186) | Node Manager uploader service for cache manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2183](https://issues.apache.org/jira/browse/YARN-2183) | Cleaner service for cache manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2180](https://issues.apache.org/jira/browse/YARN-2180) | In-memory backing store for cache manager | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2179](https://issues.apache.org/jira/browse/YARN-2179) | Initial cache manager structure and context | Major | . | Chris Trezzo | Chris Trezzo | +| [YARN-2165](https://issues.apache.org/jira/browse/YARN-2165) | Timeline server should validate the numeric configuration values | Major | timelineserver | Karam Singh | Vasanth kumar RJ | +| [YARN-2079](https://issues.apache.org/jira/browse/YARN-2079) | Recover NonAggregatingLogHandler state upon nodemanager restart | Major | nodemanager | Jason Lowe | Jason Lowe | +| [YARN-2056](https://issues.apache.org/jira/browse/YARN-2056) | Disable preemption at Queue level | Major | resourcemanager | Mayank Bansal | Eric Payne | +| [YARN-1984](https://issues.apache.org/jira/browse/YARN-1984) | LeveldbTimelineStore does not handle db exceptions properly | Major | . | Jason Lowe | Varun Saxena | +| [YARN-1904](https://issues.apache.org/jira/browse/YARN-1904) | Uniform the XXXXNotFound messages from ClientRMService and ApplicationHistoryClientService | Major | . | Zhijie Shen | Zhijie Shen | +| [YARN-1884](https://issues.apache.org/jira/browse/YARN-1884) | ContainerReport should have nodeHttpAddress | Major | . | Zhijie Shen | Xuan Gong | +| [YARN-1809](https://issues.apache.org/jira/browse/YARN-1809) | Synchronize RM and Generic History Service Web-UIs | Major | . | Zhijie Shen | Xuan Gong | +| [YARN-1723](https://issues.apache.org/jira/browse/YARN-1723) | AMRMClientAsync missing blacklist addition and removal functionality | Major | . | Bikas Saha | Bartosz Ɓugowski | +| [YARN-1514](https://issues.apache.org/jira/browse/YARN-1514) | Utility to benchmark ZKRMStateStore#loadState for ResourceManager-HA | Major | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11612](https://issues.apache.org/jira/browse/HADOOP-11612) | Workaround for Curator's ChildReaper requiring Guava 15+ | Major | . | Robert Kanter | Robert Kanter | +| [HADOOP-11489](https://issues.apache.org/jira/browse/HADOOP-11489) | Dropping dependency on io.netty from hadoop-nfs' pom.xml | Minor | nfs | Ted Yu | Ted Yu | +| [HADOOP-11463](https://issues.apache.org/jira/browse/HADOOP-11463) | Replace method-local TransferManager object with S3AFileSystem#transfers | Major | fs/s3 | Ted Yu | Ted Yu | +| [HADOOP-9329](https://issues.apache.org/jira/browse/HADOOP-9329) | document native build dependencies in BUILDING.txt | Trivial | documentation | Colin Patrick McCabe | Vijay Bhat | +| [HDFS-2486](https://issues.apache.org/jira/browse/HDFS-2486) | Review issues with UnderReplicatedBlocks | Minor | namenode | Steve Loughran | Uma Maheswara Rao G | +| [MAPREDUCE-6264](https://issues.apache.org/jira/browse/MAPREDUCE-6264) | Remove httpclient dependency from hadoop-mapreduce-client | Major | . | Akira AJISAKA | Brahma Reddy Battula | +| [MAPREDUCE-5420](https://issues.apache.org/jira/browse/MAPREDUCE-5420) | Remove mapreduce.task.tmp.dir from mapred-default.xml | Major | . | Sandy Ryza | James Carman | +| [YARN-2949](https://issues.apache.org/jira/browse/YARN-2949) | Add documentation for CGroups | Major | documentation, nodemanager | Varun Vasudev | Varun Vasudev | +| [YARN-2213](https://issues.apache.org/jira/browse/YARN-2213) | Change proxy-user cookie log in AmIpFilter to DEBUG | Minor | . | Ted Yu | Varun Saxena | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/RELEASENOTES.2.7.0.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/RELEASENOTES.2.7.0.md new file mode 100644 index 00000000000..e429b224a54 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.0/RELEASENOTES.2.7.0.md @@ -0,0 +1,295 @@ + + +# Apache Hadoop 2.7.0 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HADOOP-11801](https://issues.apache.org/jira/browse/HADOOP-11801) | *Minor* | **Update BUILDING.txt for Ubuntu** + +ProtocolBuffer is packaged in Ubuntu + + +--- + +* [HADOOP-11729](https://issues.apache.org/jira/browse/HADOOP-11729) | *Minor* | **Fix link to cgroups doc in site.xml** + +Committed this to trunk, branch-2, and branch-2.7. Thanks Masatake for your contribution! + + +--- + +* [HADOOP-11498](https://issues.apache.org/jira/browse/HADOOP-11498) | *Major* | **Bump the version of HTrace to 3.1.0-incubating** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-11497](https://issues.apache.org/jira/browse/HADOOP-11497) | *Major* | **Fix typo in ClusterSetup.html#Hadoop\_Startup** + +Correct startup command for cluster data nodes + + +--- + +* [HADOOP-11492](https://issues.apache.org/jira/browse/HADOOP-11492) | *Major* | **Bump up curator version to 2.7.1** + +Apache Curator version change: Apache Hadoop has updated the version of Apache Curator used from 2.6.0 to 2.7.1. This change should be binary and source compatible for the majority of downstream users. Notable exceptions: +# Binary incompatible change: org.apache.curator.utils.PathUtils.validatePath(String) changed return types. Downstream users of this method will need to recompile. +# Source incompatible change: org.apache.curator.framework.recipes.shared.SharedCountReader added a method to its interface definition. Downstream users with custom implementations of this interface can continue without binary compatibility problems but will need to modify their source code to recompile. +# Source incompatible change: org.apache.curator.framework.recipes.shared.SharedValueReader added a method to its interface definition. Downstream users with custom implementations of this interface can continue without binary compatibility problems but will need to modify their source code to recompile. + +Downstream users are reminded that while the Hadoop community will attempt to avoid egregious incompatible dependency changes, there is currently no policy around when Hadoop's exposed dependencies will change across versions (ref http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java\_Classpath). + + +--- + +* [HADOOP-11464](https://issues.apache.org/jira/browse/HADOOP-11464) | *Major* | **Reinstate support for launching Hadoop processes on Windows using Cygwin.** + +We have reinstated support for launching Hadoop processes on Windows by using Cygwin to run the shell scripts. All processes still must have access to the native components: hadoop.dll and winutils.exe. + + +--- + +* [HADOOP-11446](https://issues.apache.org/jira/browse/HADOOP-11446) | *Major* | **S3AOutputStream should use shared thread pool to avoid OutOfMemoryError** + +The following parameters are introduced in this JIRA: +fs.s3a.threads.max: the maximum number of threads to allow in the pool used by TransferManager +fs.s3a.threads.core: the number of threads to keep in the pool used by TransferManager +fs.s3a.threads.keepalivetime: when the number of threads is greater than the core, this is the maximum time that excess idle threads will wait for new tasks before terminating +fs.s3a.max.total.tasks: the maximum number of tasks that the LinkedBlockingQueue can hold + + +--- + +* [HADOOP-11385](https://issues.apache.org/jira/browse/HADOOP-11385) | *Critical* | **Prevent cross site scripting attack on JMXJSONServlet** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-11348](https://issues.apache.org/jira/browse/HADOOP-11348) | *Minor* | **Remove unused variable from CMake error message for finding openssl** + +Test failure is unrelated. Committed to 2.7. Thanks, Dian. + + +--- + +* [HADOOP-11311](https://issues.apache.org/jira/browse/HADOOP-11311) | *Major* | **Restrict uppercase key names from being created with JCEKS** + +Keys with uppercase names can no longer be created when using the JavaKeyStoreProvider to resolve ambiguity about case-sensitivity in the KeyStore spec. + + +--- + +* [HADOOP-10530](https://issues.apache.org/jira/browse/HADOOP-10530) | *Blocker* | **Make hadoop trunk build on Java7+ only** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HADOOP-10181](https://issues.apache.org/jira/browse/HADOOP-10181) | *Minor* | **GangliaContext does not work with multicast ganglia setup** + +Hadoop metrics sent to Ganglia over multicast now support optional configuration of socket TTL. The default TTL is 1, which preserves the behavior of prior Hadoop versions. Clusters that span multiple subnets/VLANs will likely want to increase this. + + +--- + +* [HADOOP-9922](https://issues.apache.org/jira/browse/HADOOP-9922) | *Major* | **hadoop windows native build will fail in 32 bit machine** + +The Hadoop Common native components now support 32-bit build targets on Windows. + + +--- + +* [HADOOP-9629](https://issues.apache.org/jira/browse/HADOOP-9629) | *Major* | **Support Windows Azure Storage - Blob as a file system in Hadoop** + +Hadoop now supports integration with Azure Storage as an alternative Hadoop Compatible File System. + + +--- + +* [HADOOP-9329](https://issues.apache.org/jira/browse/HADOOP-9329) | *Trivial* | **document native build dependencies in BUILDING.txt** + +Added a section to BUILDING.txt on how to install required / optional packages on a clean install of Ubuntu 14.04 LTS Desktop. + +Went through the CMakeLists.txt files in the repo and added the following optional library dependencies - Snappy, Bzip2, Linux FUSE and Jansson. + +Updated the required packages / version numbers from the trunk branch version of BUILDING.txt. + + +--- + +* [HADOOP-8989](https://issues.apache.org/jira/browse/HADOOP-8989) | *Major* | **hadoop fs -find feature** + +New fs -find command + + +--- + +* [HDFS-8001](https://issues.apache.org/jira/browse/HDFS-8001) | *Trivial* | **RpcProgramNfs3 : wrong parsing of dfs.blocksize** + +patch is fully backward compatible. + + +--- + +* [HDFS-7806](https://issues.apache.org/jira/browse/HDFS-7806) | *Minor* | **Refactor: move StorageType from hadoop-hdfs to hadoop-common** + +This fix moves the public class StorageType from the package org.apache.hadoop.hdfs to org.apache.hadoop.fs. + + +--- + +* [HDFS-7774](https://issues.apache.org/jira/browse/HDFS-7774) | *Critical* | **Unresolved symbols error while compiling HDFS on Windows 7/32 bit** + +LibHDFS now supports 32-bit build targets on Windows. + + +--- + +* [HDFS-7584](https://issues.apache.org/jira/browse/HDFS-7584) | *Major* | **Enable Quota Support for Storage Types** + +1. Introduced quota by storage type as a hard limit on the amount of space usage allowed for different storage types (SSD, DISK, ARCHIVE) under the target directory. +2. Added {{SetQuotaByStorageType}} API and {{-storagetype}} option for {{hdfs dfsadmin -setSpaceQuota/-clrSpaceQuota}} commands to allow set/clear quota by storage type under the target directory. + + +--- + +* [HDFS-7457](https://issues.apache.org/jira/browse/HDFS-7457) | *Major* | **DatanodeID generates excessive garbage** + +Thanks for the reviews, gentlemen. I've committed this to trunk and branch-2. Thanks for identifying and working on the issue, Daryn. + + +--- + +* [HDFS-7411](https://issues.apache.org/jira/browse/HDFS-7411) | *Major* | **Refactor and improve decommissioning logic into DecommissionManager** + +This change introduces a new configuration key used to throttle decommissioning work, "dfs.namenode.decommission.blocks.per.interval". This new key overrides and deprecates the previous related configuration key "dfs.namenode.decommission.nodes.per.interval". The new key is intended to result in more predictable pause times while scanning decommissioning nodes. + + +--- + +* [HDFS-7326](https://issues.apache.org/jira/browse/HDFS-7326) | *Minor* | **Add documentation for hdfs debug commands** + +Added documentation for the hdfs debug commands to the following URL in the documentation website. + +hadoop-project-dist/hadoop-hdfs/HDFSCommands.html + +In order to view the new documentation, build the website in a staging area: +$ mvn clean site; mvn site:stage -DstagingDirectory=/tmp/hadoop-site + +Point your browser to +file:///tmp/hadoop-site/hadoop-project/hadoop-project-dist/hadoop-hdfs/HDFSCommands.html + + +--- + +* [HDFS-6651](https://issues.apache.org/jira/browse/HDFS-6651) | *Critical* | **Deletion failure can leak inodes permanently** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-6252](https://issues.apache.org/jira/browse/HDFS-6252) | *Minor* | **Phase out the old web UI in HDFS** + +**WARNING: No release note provided for this incompatible change.** + + +--- + +* [HDFS-6133](https://issues.apache.org/jira/browse/HDFS-6133) | *Major* | **Make Balancer support exclude specified path** + +Add a feature for replica pinning so that when a replica is pinned in a datanode, it will not be moved by Balancer/Mover. The replica pinning feature can be enabled/disabled by "dfs.datanode.block-pinning.enabled", where the default is false. + + +--- + +* [HDFS-3689](https://issues.apache.org/jira/browse/HDFS-3689) | *Major* | **Add support for variable length block** + +1. HDFS now can choose to append data to a new block instead of end of the last partial block. Users can pass {{CreateFlag.APPEND}} and {{CreateFlag.NEW\_BLOCK}} to the {{append}} API to indicate this requirement. +2. HDFS now allows users to pass {{SyncFlag.END\_BLOCK}} to the {{hsync}} API to finish the current block and write remaining data to a new block. + + +--- + +* [HDFS-1522](https://issues.apache.org/jira/browse/HDFS-1522) | *Major* | **Merge Block.BLOCK\_FILE\_PREFIX and DataStorage.BLOCK\_FILE\_PREFIX into one constant** + +This merges Block.BLOCK\_FILE\_PREFIX and DataStorage.BLOCK\_FILE\_PREFIX into one constant. Hard-coded +literals of "blk\_" in various files are also updated to use the same constant. + + +--- + +* [HDFS-1362](https://issues.apache.org/jira/browse/HDFS-1362) | *Major* | **Provide volume management functionality for DataNode** + +Based on the reconfiguration framework provided by HADOOP-7001, enable reconfigure the dfs.datanode.data.dir and add new volumes into service. + + +--- + +* [MAPREDUCE-5583](https://issues.apache.org/jira/browse/MAPREDUCE-5583) | *Major* | **Ability to limit running map and reduce tasks** + +This introduces two new MR2 job configs, mentioned below, which allow users to control the maximum simultaneously-running tasks of the submitted job, across the cluster: + +* mapreduce.job.running.map.limit (default: 0, for no limit) +* mapreduce.job.running.reduce.limit (default: 0, for no limit) + +This is controllable at a per-job level. + + +--- + +* [YARN-3217](https://issues.apache.org/jira/browse/YARN-3217) | *Major* | **Remove httpclient dependency from hadoop-yarn-server-web-proxy** + +Removed commons-httpclient dependency from hadoop-yarn-server-web-proxy module. + + +--- + +* [YARN-3154](https://issues.apache.org/jira/browse/YARN-3154) | *Blocker* | **Should not upload partial logs for MR jobs or other "short-running' applications** + +Applications which made use of the LogAggregationContext in their application will need to revisit this code in order to make sure that their logs continue to get rolled out. + + +--- + +* [YARN-2230](https://issues.apache.org/jira/browse/YARN-2230) | *Minor* | **Fix description of yarn.scheduler.maximum-allocation-vcores in yarn-default.xml (or code)** + +I have modified the description of the yarn.scheduler.maximum-allocation-vcores setting in yarn-default.xml to be reflective of the actual behavior (throw InvalidRequestException when the limit is crossed). + +Since this is a documentation change, I have not added any test cases. + +Please review the patch, thanks! + + +--- + +* [YARN-1904](https://issues.apache.org/jira/browse/YARN-1904) | *Major* | **Uniform the XXXXNotFound messages from ClientRMService and ApplicationHistoryClientService** + +I just committed this. Thanks Zhijie! + + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/CHANGES.2.7.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/CHANGES.2.7.1.md new file mode 100644 index 00000000000..ed9100e6fa4 --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/CHANGES.2.7.1.md @@ -0,0 +1,195 @@ + + +# Apache Hadoop Changelog + +## Release 2.7.1 - 2015-07-06 + +### INCOMPATIBLE CHANGES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### NEW FEATURES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### IMPROVEMENTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-11851](https://issues.apache.org/jira/browse/HADOOP-11851) | s3n to swallow IOEs on inner stream close | Minor | fs/s3 | Steve Loughran | Takenori Sato | +| [HDFS-8521](https://issues.apache.org/jira/browse/HDFS-8521) | Add @VisibleForTesting annotation to {{BlockPoolSlice#selectReplicaToDelete}} | Trivial | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-8361](https://issues.apache.org/jira/browse/HDFS-8361) | Choose SSD over DISK in block placement | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze | +| [HDFS-8204](https://issues.apache.org/jira/browse/HDFS-8204) | Mover/Balancer should not schedule two replicas to the same DN | Minor | balancer & mover | Walter Su | Walter Su | +| [HDFS-7770](https://issues.apache.org/jira/browse/HDFS-7770) | Need document for storage type label of data node storage locations under dfs.datanode.data.dir | Major | documentation | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7546](https://issues.apache.org/jira/browse/HDFS-7546) | Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern | Minor | security | Harsh J | Harsh J | +| [YARN-3539](https://issues.apache.org/jira/browse/YARN-3539) | Compatibility doc to state that ATS v1 is a stable REST API | Major | documentation | Steve Loughran | Steve Loughran | +| [YARN-3489](https://issues.apache.org/jira/browse/YARN-3489) | RMServerUtils.validateResourceRequests should only obtain queue info once | Major | resourcemanager | Jason Lowe | Varun Saxena | +| [YARN-3469](https://issues.apache.org/jira/browse/YARN-3469) | ZKRMStateStore: Avoid setting watches that are not required | Minor | . | Jun Gong | Jun Gong | +| [YARN-3193](https://issues.apache.org/jira/browse/YARN-3193) | When visit standby RM webui, it will redirect to the active RM webui slowly. | Minor | webapp | Japs\_123 | Steve Loughran | + + +### BUG FIXES: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HADOOP-12103](https://issues.apache.org/jira/browse/HADOOP-12103) | Small refactoring of DelegationTokenAuthenticationFilter to allow code sharing | Minor | security | Yongjun Zhang | Yongjun Zhang | +| [HADOOP-12100](https://issues.apache.org/jira/browse/HADOOP-12100) | ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission | Major | . | Robert Kanter | Bibin A Chundatt | +| [HADOOP-12078](https://issues.apache.org/jira/browse/HADOOP-12078) | The default retry policy does not handle RetriableException correctly | Critical | ipc | Arpit Agarwal | Arpit Agarwal | +| [HADOOP-12058](https://issues.apache.org/jira/browse/HADOOP-12058) | Fix dead links to DistCp and Hadoop Archives pages. | Minor | documentation, site | Kazuho Fujii | Kazuho Fujii | +| [HADOOP-11973](https://issues.apache.org/jira/browse/HADOOP-11973) | Ensure ZkDelegationTokenSecretManager namespace znodes get created with ACLs | Major | security | Gregory Chanan | Gregory Chanan | +| [HADOOP-11966](https://issues.apache.org/jira/browse/HADOOP-11966) | Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. | Critical | scripts | Chris Nauroth | Chris Nauroth | +| [HADOOP-11934](https://issues.apache.org/jira/browse/HADOOP-11934) | Use of JavaKeyStoreProvider in LdapGroupsMapping causes infinite loop | Blocker | security | Mike Yoder | Larry McCay | +| [HADOOP-11891](https://issues.apache.org/jira/browse/HADOOP-11891) | OsSecureRandom should lazily fill its reservoir | Major | security | Arun Suresh | Arun Suresh | +| [HADOOP-11872](https://issues.apache.org/jira/browse/HADOOP-11872) | "hadoop dfs" command prints message about using "yarn jar" on Windows(branch-2 only) | Minor | scripts | Varun Vasudev | Varun Vasudev | +| [HADOOP-11868](https://issues.apache.org/jira/browse/HADOOP-11868) | Invalid user logins trigger large backtraces in server log | Major | . | Chang Li | Chang Li | +| [HADOOP-11802](https://issues.apache.org/jira/browse/HADOOP-11802) | DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm | Major | . | Eric Payne | Colin Patrick McCabe | +| [HADOOP-11730](https://issues.apache.org/jira/browse/HADOOP-11730) | Regression: s3n read failure recovery broken | Major | fs/s3 | Takenori Sato | Takenori Sato | +| [HADOOP-11663](https://issues.apache.org/jira/browse/HADOOP-11663) | Remove description about Java 6 from docs | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [HADOOP-9658](https://issues.apache.org/jira/browse/HADOOP-9658) | SnappyCodec#checkNativeCodeLoaded may unexpectedly fail when native code is not loaded | Major | . | Zhijie Shen | Zhijie Shen | +| [HDFS-8681](https://issues.apache.org/jira/browse/HDFS-8681) | BlockScanner is incorrectly disabled by default | Blocker | datanode | Andrew Wang | Arpit Agarwal | +| [HDFS-8633](https://issues.apache.org/jira/browse/HDFS-8633) | Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys | Minor | HDFS | Ray Chiang | Ray Chiang | +| [HDFS-8626](https://issues.apache.org/jira/browse/HDFS-8626) | Reserved RBW space is not released if creation of RBW File fails | Blocker | . | kanaka kumar avvaru | kanaka kumar avvaru | +| [HDFS-8600](https://issues.apache.org/jira/browse/HDFS-8600) | TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7 | Major | webhdfs | Arpit Agarwal | Arpit Agarwal | +| [HDFS-8596](https://issues.apache.org/jira/browse/HDFS-8596) | TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute | Blocker | HDFS | Yongjun Zhang | Yongjun Zhang | +| [HDFS-8595](https://issues.apache.org/jira/browse/HDFS-8595) | TestCommitBlockSynchronization fails in branch-2.7 | Major | test | Arpit Agarwal | Arpit Agarwal | +| [HDFS-8583](https://issues.apache.org/jira/browse/HDFS-8583) | Document that NFS gateway does not work with rpcbind on SLES 11 | Major | documentation | Arpit Agarwal | Arpit Agarwal | +| [HDFS-8576](https://issues.apache.org/jira/browse/HDFS-8576) | Lease recovery should return true if the lease can be released and the file can be closed | Major | namenode | J.Andreina | J.Andreina | +| [HDFS-8572](https://issues.apache.org/jira/browse/HDFS-8572) | DN always uses HTTP/localhost@REALM principals in SPNEGO | Blocker | . | Haohui Mai | Haohui Mai | +| [HDFS-8566](https://issues.apache.org/jira/browse/HDFS-8566) | HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands | Major | documentation | Surendra Singh Lilhore | Surendra Singh Lilhore | +| [HDFS-8544](https://issues.apache.org/jira/browse/HDFS-8544) | Incorrect port specified in HFTP Guide document in branch-2 | Major | documentation | Brahma Reddy Battula | Brahma Reddy Battula | +| [HDFS-8523](https://issues.apache.org/jira/browse/HDFS-8523) | Remove usage information on unsupported operation "fsck -showprogress" from branch-2 | Major | documentation | J.Andreina | J.Andreina | +| [HDFS-8522](https://issues.apache.org/jira/browse/HDFS-8522) | Change heavily recorded NN logs from INFO to DEBUG level | Major | namenode | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-8486](https://issues.apache.org/jira/browse/HDFS-8486) | DN startup may cause severe data loss | Blocker | datanode | Daryn Sharp | Daryn Sharp | +| [HDFS-8480](https://issues.apache.org/jira/browse/HDFS-8480) | Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs instead of copying them | Critical | . | Zhe Zhang | Zhe Zhang | +| [HDFS-8451](https://issues.apache.org/jira/browse/HDFS-8451) | DFSClient probe for encryption testing interprets empty URI property for "enabled" | Blocker | encryption | Steve Loughran | Steve Loughran | +| [HDFS-8405](https://issues.apache.org/jira/browse/HDFS-8405) | Fix a typo in NamenodeFsck | Minor | namenode | Tsz Wo Nicholas Sze | Takanobu Asanuma | +| [HDFS-8404](https://issues.apache.org/jira/browse/HDFS-8404) | Pending block replication can get stuck using older genstamp | Major | namenode | Nathan Roberts | Nathan Roberts | +| [HDFS-8305](https://issues.apache.org/jira/browse/HDFS-8305) | HDFS INotify: the destination field of RenameOp should always end with the file name | Major | . | Colin Patrick McCabe | Colin Patrick McCabe | +| [HDFS-8273](https://issues.apache.org/jira/browse/HDFS-8273) | FSNamesystem#Delete() should not call logSync() when holding the lock | Blocker | namenode | Jing Zhao | Haohui Mai | +| [HDFS-8270](https://issues.apache.org/jira/browse/HDFS-8270) | create() always retried with hardcoded timeout when file already exists with open lease | Major | hdfs-client | Andrey Stepachev | J.Andreina | +| [HDFS-8269](https://issues.apache.org/jira/browse/HDFS-8269) | getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime | Blocker | . | Yesha Vora | Haohui Mai | +| [HDFS-8245](https://issues.apache.org/jira/browse/HDFS-8245) | Standby namenode doesn't process DELETED\_BLOCK if the add block request is in edit log. | Major | . | Rushabh S Shah | Rushabh S Shah | +| [HDFS-8226](https://issues.apache.org/jira/browse/HDFS-8226) | Non-HA rollback compatibility broken | Blocker | . | J.Andreina | J.Andreina | +| [HDFS-8213](https://issues.apache.org/jira/browse/HDFS-8213) | DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace | Critical | . | Billie Rinaldi | Colin Patrick McCabe | +| [HDFS-8179](https://issues.apache.org/jira/browse/HDFS-8179) | DFSClient#getServerDefaults returns null within 1 hour of system start | Blocker | . | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-8163](https://issues.apache.org/jira/browse/HDFS-8163) | Using monotonicNow for block report scheduling causes test failures on recently restarted systems | Blocker | datanode | Arpit Agarwal | Arpit Agarwal | +| [HDFS-8153](https://issues.apache.org/jira/browse/HDFS-8153) | Error Message points to wrong parent directory in case of path component name length error | Major | namenode | Anu Engineer | Anu Engineer | +| [HDFS-8151](https://issues.apache.org/jira/browse/HDFS-8151) | Always use snapshot path as source when invalid snapshot names are used for diff based distcp | Minor | distcp | Sushmitha Sreenivasan | Jing Zhao | +| [HDFS-8149](https://issues.apache.org/jira/browse/HDFS-8149) | The footer of the Web UI "Hadoop, 2014" is old | Major | . | Akira AJISAKA | Brahma Reddy Battula | +| [HDFS-8147](https://issues.apache.org/jira/browse/HDFS-8147) | Mover should not schedule two replicas to the same DN storage | Major | balancer & mover | Surendra Singh Lilhore | Surendra Singh Lilhore | +| [HDFS-8127](https://issues.apache.org/jira/browse/HDFS-8127) | NameNode Failover during HA upgrade can cause DataNode to finalize upgrade | Blocker | ha | Jing Zhao | Jing Zhao | +| [HDFS-8091](https://issues.apache.org/jira/browse/HDFS-8091) | ACLStatus and XAttributes not properly presented to INodeAttributesProvider before returning to client | Major | HDFS | Arun Suresh | Arun Suresh | +| [HDFS-8081](https://issues.apache.org/jira/browse/HDFS-8081) | Split getAdditionalBlock() into two methods. | Major | . | Konstantin Shvachko | Konstantin Shvachko | +| [HDFS-8070](https://issues.apache.org/jira/browse/HDFS-8070) | Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode | Blocker | caching | Gopal V | Colin Patrick McCabe | +| [HDFS-7980](https://issues.apache.org/jira/browse/HDFS-7980) | Incremental BlockReport will dramatically slow down the startup of a namenode | Major | . | Hui Zheng | Walter Su | +| [HDFS-7934](https://issues.apache.org/jira/browse/HDFS-7934) | Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN | Critical | documentation | J.Andreina | J.Andreina | +| [HDFS-7931](https://issues.apache.org/jira/browse/HDFS-7931) | DistributedFIleSystem should not look for keyProvider in cache if Encryption is disabled | Minor | hdfs-client | Arun Suresh | Arun Suresh | +| [HDFS-7916](https://issues.apache.org/jira/browse/HDFS-7916) | 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop | Critical | datanode | Vinayakumar B | Rushabh S Shah | +| [HDFS-7894](https://issues.apache.org/jira/browse/HDFS-7894) | Rolling upgrade readiness is not updated in jmx until query command is issued. | Critical | . | Kihwal Lee | Brahma Reddy Battula | +| [HDFS-6300](https://issues.apache.org/jira/browse/HDFS-6300) | Prevent multiple balancers from running simultaneously | Critical | balancer & mover | Rakesh R | Rakesh R | +| [HDFS-5215](https://issues.apache.org/jira/browse/HDFS-5215) | dfs.datanode.du.reserved is not considered while computing available space | Major | datanode | Brahma Reddy Battula | Brahma Reddy Battula | +| [HDFS-4660](https://issues.apache.org/jira/browse/HDFS-4660) | Block corruption can happen during pipeline recovery | Blocker | datanode | Peng Zhang | Kihwal Lee | +| [MAPREDUCE-6410](https://issues.apache.org/jira/browse/MAPREDUCE-6410) | Aggregated Logs Deletion doesnt work after refreshing Log Retention Settings in secure cluster | Critical | . | Zhang Wei | Varun Saxena | +| [MAPREDUCE-6387](https://issues.apache.org/jira/browse/MAPREDUCE-6387) | Serialize the recently added Task#encryptedSpillKey field at the end | Minor | . | Arun Suresh | Arun Suresh | +| [MAPREDUCE-6361](https://issues.apache.org/jira/browse/MAPREDUCE-6361) | NPE issue in shuffle caused by concurrent issue between copySucceeded() in one thread and copyFailed() in another thread on the same host | Critical | . | Junping Du | Junping Du | +| [MAPREDUCE-6339](https://issues.apache.org/jira/browse/MAPREDUCE-6339) | Job history file is not flushed correctly because isTimerActive flag is not set true when flushTimerTask is scheduled. | Critical | mrv2 | zhihai xu | zhihai xu | +| [MAPREDUCE-6334](https://issues.apache.org/jira/browse/MAPREDUCE-6334) | Fetcher#copyMapOutput is leaking usedMemory upon IOException during InMemoryMapOutput shuffle handler | Blocker | . | Eric Payne | Eric Payne | +| [MAPREDUCE-6324](https://issues.apache.org/jira/browse/MAPREDUCE-6324) | Uber jobs fail to update AMRM token when it rolls over | Blocker | mr-am | Jason Lowe | Jason Lowe | +| [MAPREDUCE-6300](https://issues.apache.org/jira/browse/MAPREDUCE-6300) | Task list sort by task id broken | Minor | . | Siqi Li | Siqi Li | +| [MAPREDUCE-6259](https://issues.apache.org/jira/browse/MAPREDUCE-6259) | IllegalArgumentException due to missing job submit time | Major | jobhistoryserver | zhihai xu | zhihai xu | +| [MAPREDUCE-6252](https://issues.apache.org/jira/browse/MAPREDUCE-6252) | JobHistoryServer should not fail when encountering a missing directory | Major | jobhistoryserver | Craig Welch | Craig Welch | +| [MAPREDUCE-6251](https://issues.apache.org/jira/browse/MAPREDUCE-6251) | JobClient needs additional retries at a higher level to address not-immediately-consistent dfs corner cases | Major | jobhistoryserver, mrv2 | Craig Welch | Craig Welch | +| [MAPREDUCE-6238](https://issues.apache.org/jira/browse/MAPREDUCE-6238) | MR2 can't run local jobs with -libjars command options which is a regression from MR1 | Critical | mrv2 | zhihai xu | zhihai xu | +| [YARN-3850](https://issues.apache.org/jira/browse/YARN-3850) | NM fails to read files from full disks which can lead to container logs being lost and other issues | Blocker | log-aggregation, nodemanager | Varun Saxena | Varun Saxena | +| [YARN-3842](https://issues.apache.org/jira/browse/YARN-3842) | NMProxy should retry on NMNotYetReadyException | Critical | . | Karthik Kambatla | Robert Kanter | +| [YARN-3832](https://issues.apache.org/jira/browse/YARN-3832) | Resource Localization fails on a cluster due to existing cache directories | Critical | nodemanager | Ranga Swamy | Brahma Reddy Battula | +| [YARN-3809](https://issues.apache.org/jira/browse/YARN-3809) | Failed to launch new attempts because ApplicationMasterLauncher's threads all hang | Major | resourcemanager | Jun Gong | Jun Gong | +| [YARN-3804](https://issues.apache.org/jira/browse/YARN-3804) | Both RM are on standBy state when kerberos user not in yarn.admin.acl | Critical | resourcemanager | Bibin A Chundatt | Varun Saxena | +| [YARN-3764](https://issues.apache.org/jira/browse/YARN-3764) | CapacityScheduler should forbid moving LeafQueue from one parent to another | Blocker | . | Wangda Tan | Wangda Tan | +| [YARN-3753](https://issues.apache.org/jira/browse/YARN-3753) | RM failed to come up with "java.io.IOException: Wait for ZKClient creation timed out" | Critical | yarn | Sumana Sathish | Jian He | +| [YARN-3733](https://issues.apache.org/jira/browse/YARN-3733) | Fix DominantRC#compare() does not work as expected if cluster resource is empty | Blocker | resourcemanager | Bibin A Chundatt | Rohith Sharma K S | +| [YARN-3725](https://issues.apache.org/jira/browse/YARN-3725) | App submission via REST API is broken in secure mode due to Timeline DT service address is empty | Blocker | resourcemanager, timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-3723](https://issues.apache.org/jira/browse/YARN-3723) | Need to clearly document primaryFilter and otherInfo value type | Critical | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-3701](https://issues.apache.org/jira/browse/YARN-3701) | Isolating the error of generating a single app report when getting all apps from generic history service | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-3694](https://issues.apache.org/jira/browse/YARN-3694) | Fix dead link for TimelineServer REST API | Minor | documentation | Akira AJISAKA | Jagadesh Kiran N | +| [YARN-3681](https://issues.apache.org/jira/browse/YARN-3681) | yarn cmd says "could not find main class 'queue'" in windows | Blocker | yarn | Sumana Sathish | Varun Saxena | +| [YARN-3677](https://issues.apache.org/jira/browse/YARN-3677) | Fix findbugs warnings in yarn-server-resourcemanager | Minor | resourcemanager | Akira AJISAKA | Vinod Kumar Vavilapalli | +| [YARN-3675](https://issues.apache.org/jira/browse/YARN-3675) | FairScheduler: RM quits when node removal races with continousscheduling on the same node | Critical | fairscheduler | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3646](https://issues.apache.org/jira/browse/YARN-3646) | Applications are getting stuck some times in case of retry policy forever | Major | client | Raju Bairishetti | Raju Bairishetti | +| [YARN-3626](https://issues.apache.org/jira/browse/YARN-3626) | On Windows localized resources are not moved to the front of the classpath when they should be | Major | yarn | Craig Welch | Craig Welch | +| [YARN-3614](https://issues.apache.org/jira/browse/YARN-3614) | FileSystemRMStateStore throw exception when failed to remove application, that cause resourcemanager to crash | Critical | resourcemanager | lachisis | | +| [YARN-3601](https://issues.apache.org/jira/browse/YARN-3601) | Fix UT TestRMFailover.testRMWebAppRedirect | Critical | resourcemanager, webapp | Weiwei Yang | Weiwei Yang | +| [YARN-3585](https://issues.apache.org/jira/browse/YARN-3585) | NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled | Critical | . | Peng Zhang | Rohith Sharma K S | +| [YARN-3554](https://issues.apache.org/jira/browse/YARN-3554) | Default value for maximum nodemanager connect wait time is too high | Major | . | Jason Lowe | Naganarasimha G R | +| [YARN-3537](https://issues.apache.org/jira/browse/YARN-3537) | NPE when NodeManager.serviceInit fails and stopRecoveryStore invoked | Major | nodemanager | Brahma Reddy Battula | Brahma Reddy Battula | +| [YARN-3526](https://issues.apache.org/jira/browse/YARN-3526) | ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster | Major | resourcemanager, webapp | Weiwei Yang | Weiwei Yang | +| [YARN-3522](https://issues.apache.org/jira/browse/YARN-3522) | DistributedShell uses the wrong user to put timeline data | Blocker | timelineserver | Zhijie Shen | Zhijie Shen | +| [YARN-3516](https://issues.apache.org/jira/browse/YARN-3516) | killing ContainerLocalizer action doesn't take effect when private localizer receives FETCH\_FAILURE status. | Minor | nodemanager | zhihai xu | zhihai xu | +| [YARN-3497](https://issues.apache.org/jira/browse/YARN-3497) | ContainerManagementProtocolProxy modifies IPC timeout conf without making a copy | Major | client | Jason Lowe | Jason Lowe | +| [YARN-3493](https://issues.apache.org/jira/browse/YARN-3493) | RM fails to come up with error "Failed to load/recover state" when mem settings are changed | Critical | yarn | Sumana Sathish | Jian He | +| [YARN-3485](https://issues.apache.org/jira/browse/YARN-3485) | FairScheduler headroom calculation doesn't consider maxResources for Fifo and FairShare policies | Critical | fairscheduler | Karthik Kambatla | Karthik Kambatla | +| [YARN-3476](https://issues.apache.org/jira/browse/YARN-3476) | Nodemanager can fail to delete local logs if log aggregation fails | Major | log-aggregation, nodemanager | Jason Lowe | Rohith Sharma K S | +| [YARN-3472](https://issues.apache.org/jira/browse/YARN-3472) | Possible leak in DelegationTokenRenewer#allTokens | Major | . | Jian He | Rohith Sharma K S | +| [YARN-3466](https://issues.apache.org/jira/browse/YARN-3466) | Fix RM nodes web page to sort by node HTTP-address, #containers and node-label column | Major | resourcemanager, webapp | Jason Lowe | Jason Lowe | +| [YARN-3465](https://issues.apache.org/jira/browse/YARN-3465) | Use LinkedHashMap to preserve order of resource requests | Major | nodemanager | zhihai xu | zhihai xu | +| [YARN-3464](https://issues.apache.org/jira/browse/YARN-3464) | Race condition in LocalizerRunner kills localizer before localizing all resources | Critical | nodemanager | zhihai xu | zhihai xu | +| [YARN-3462](https://issues.apache.org/jira/browse/YARN-3462) | Patches applied for YARN-2424 are inconsistent between trunk and branch-2 | Major | . | Sidharta Seethana | Naganarasimha G R | +| [YARN-3457](https://issues.apache.org/jira/browse/YARN-3457) | NPE when NodeManager.serviceInit fails and stopRecoveryStore called | Minor | nodemanager | Bibin A Chundatt | Bibin A Chundatt | +| [YARN-3434](https://issues.apache.org/jira/browse/YARN-3434) | Interaction between reservations and userlimit can result in significant ULF violation | Major | capacityscheduler | Thomas Graves | Thomas Graves | +| [YARN-3385](https://issues.apache.org/jira/browse/YARN-3385) | Race condition: KeeperException$NoNodeException will cause RM shutdown during ZK node deletion. | Critical | resourcemanager | zhihai xu | zhihai xu | +| [YARN-3382](https://issues.apache.org/jira/browse/YARN-3382) | Some of UserMetricsInfo metrics are incorrectly set to root queue metrics | Major | webapp | Rohit Agarwal | Rohit Agarwal | +| [YARN-3358](https://issues.apache.org/jira/browse/YARN-3358) | Audit log not present while refreshing Service ACLs | Minor | resourcemanager | Varun Saxena | Varun Saxena | +| [YARN-3351](https://issues.apache.org/jira/browse/YARN-3351) | AppMaster tracking URL is broken in HA | Major | webapp | Anubhav Dhoot | Anubhav Dhoot | +| [YARN-3243](https://issues.apache.org/jira/browse/YARN-3243) | CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits. | Major | capacityscheduler, resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-2238](https://issues.apache.org/jira/browse/YARN-2238) | filtering on UI sticks even if I move away from the page | Major | webapp | Sangjin Lee | Jian He | + + +### TESTS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + +### SUB-TASKS: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | +| [HDFS-8597](https://issues.apache.org/jira/browse/HDFS-8597) | Fix TestFSImage#testZeroBlockSize on Windows | Major | datanode, test | Xiaoyu Yao | Xiaoyu Yao | +| [HDFS-7164](https://issues.apache.org/jira/browse/HDFS-7164) | Feature documentation for HDFS-6581 | Major | documentation | Arpit Agarwal | Arpit Agarwal | +| [YARN-3711](https://issues.apache.org/jira/browse/YARN-3711) | Documentation of ResourceManager HA should explain configurations about listen addresses | Minor | documentation | Masatake Iwasaki | Masatake Iwasaki | +| [YARN-3686](https://issues.apache.org/jira/browse/YARN-3686) | CapacityScheduler should trim default\_node\_label\_expression | Critical | api, client, resourcemanager | Wangda Tan | Sunil G | +| [YARN-3609](https://issues.apache.org/jira/browse/YARN-3609) | Move load labels from storage from serviceInit to serviceStart to make it works with RM HA case. | Major | resourcemanager | Wangda Tan | Wangda Tan | +| [YARN-3544](https://issues.apache.org/jira/browse/YARN-3544) | AM logs link missing in the RM UI for a completed app | Blocker | . | Hitesh Shah | Xuan Gong | +| [YARN-3487](https://issues.apache.org/jira/browse/YARN-3487) | CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue | Critical | capacityscheduler | Jason Lowe | Jason Lowe | +| [YARN-3006](https://issues.apache.org/jira/browse/YARN-3006) | Improve the error message when attempting manual failover with auto-failover enabled | Minor | . | Akira AJISAKA | Akira AJISAKA | +| [YARN-2918](https://issues.apache.org/jira/browse/YARN-2918) | Don't fail RM if queue's configured labels are not existed in cluster-node-labels | Major | resourcemanager | Rohith Sharma K S | Wangda Tan | +| [YARN-2900](https://issues.apache.org/jira/browse/YARN-2900) | Application (Attempt and Container) Not Found in AHS results in Internal Server Error (500) | Major | timelineserver | Jonathan Eagles | Mit Desai | +| [YARN-2605](https://issues.apache.org/jira/browse/YARN-2605) | [RM HA] Rest api endpoints doing redirect incorrectly | Major | resourcemanager | bc Wong | Xuan Gong | + + +### OTHER: + +| JIRA | Summary | Priority | Component | Reporter | Contributor | +|:---- |:---- | :--- |:---- |:---- |:---- | + + diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/RELEASENOTES.2.7.1.md b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/RELEASENOTES.2.7.1.md new file mode 100644 index 00000000000..2b62c13002a --- /dev/null +++ b/hadoop-common-project/hadoop-common/src/site/markdown/release/2.7.1/RELEASENOTES.2.7.1.md @@ -0,0 +1,47 @@ + + +# Apache Hadoop 2.7.1 Release Notes + +These release notes cover new developer and user-facing incompatibilities, features, and major improvements. + + +--- + +* [HDFS-8486](https://issues.apache.org/jira/browse/HDFS-8486) | *Blocker* | **DN startup may cause severe data loss** + +Public service notice: +- Every restart of a 2.6.x or 2.7.0 DN incurs a risk of unwanted block deletion. +- Apply this patch if you are running a pre-2.7.1 release. + + +--- + +* [HDFS-8270](https://issues.apache.org/jira/browse/HDFS-8270) | *Major* | **create() always retried with hardcoded timeout when file already exists with open lease** + +Proxy level retries will not be done on AlreadyBeingCreatedExeption for create() op. + + +--- + +* [HDFS-8226](https://issues.apache.org/jira/browse/HDFS-8226) | *Blocker* | **Non-HA rollback compatibility broken** + +Non-HA rollback steps have been changed. Run the rollback command on the namenode (`bin/hdfs namenode -rollback`) before starting cluster with '-rollback' option using (sbin/start-dfs.sh -rollback). + + +