Hadoop YARN Change Log Release 2.0.3-alpha - Unreleased INCOMPATIBLE CHANGES NEW FEATURES YARN-145. Add a Web UI to the fair share scheduler. (Sandy Ryza via tomwhite) IMPROVEMENTS YARN-78. Changed UnManagedAM application to use YarnClient. (Bikas Saha via vinodkv) YARN-93. Fixed RM to propagate diagnostics from applications that have finished but failed (Jason Lowe via vinodkv). YARN-28. Fixed TestCompositeService to not depend on test-order and thus made it pass on JDK7 (Thomas Graves via vinodkv). YARN-82. Change the default local and log dirs to be based on hadoop.tmp.dir and yarn.log.dir. (Hemanth Yamijala via sseth) YARN-53. Added the missing getGroups API to ResourceManager. (Bo Wang via vinodkv) YARN-116. Add the ability to change the RM include/exclude file without a restart. (xieguiming and Harsh J via sseth) YARN-23. FairScheduler: FSQueueSchedulable#updateDemand() - potential redundant aggregation. (kkambatl via tucu) YARN-127. Move RMAdmin tool to its correct location - the client module. (vinodkv) YARN-40. Provided support for missing YARN commands (Devaraj K and Vinod Kumar Vavilapalli via vinodkv) YARN-33. Change LocalDirsHandlerService to validate the configured local and log dirs. (Mayank Bansal via sseth) YARN-94. Modify DistributedShell to point to main-class by default, clean up the help message, and hard-code the AM class. (Hitesh Shah via vinodkv) YARN-146. Add unit tests for computing fair share in the fair scheduler. (Sandy Ryza via tomwhite) HADOOP-8911. CRLF characters in source and text files. (Raja Aluri via suresh) YARN-136. Make ClientToAMTokenSecretManager part of RMContext (Vinod Kumar Vavilapalli via sseth) YARN-183. Clean up fair scheduler code. (Sandy Ryza via tomwhite) OPTIMIZATIONS BUG FIXES YARN-131. Fix incorrect ACL properties in capacity scheduler documentation. (Ahmed Radwan via sseth) YARN-102. Move the apache header to the top of the file in MemStore.java. (Devaraj K via sseth) YARN-134. ClientToAMSecretManager creates keys without checking for validity of the appID. (Vinod Kumar Vavilapalli via sseth) YARN-30. Fixed tests verifying web-services to work on JDK7. (Thomas Graves via vinodkv) YARN-150. Fixes AppRejectedTransition does not unregister a rejected app-attempt from the ApplicationMasterService (Bikas Saha via sseth) YARN-140. Add capacity-scheduler-default.xml to provide a default set of configurations for the capacity scheduler. (ahmed via tucu) YARN-179. Fix some unit test failures. (Vinod Kumar Vavilapalli via sseth) YARN-181. Fixed eclipse settings broken by capacity-scheduler.xml move via YARN-140. (Siddharth Seth via vinodkv) YARN-169. Update log4j.appender.EventCounter to use org.apache.hadoop.log.metrics.EventCounter (Anthony Rojas via tomwhite) Release 2.0.2-alpha - 2012-09-07 YARN-9. Rename YARN_HOME to HADOOP_YARN_HOME. (vinodkv via acmurthy) NEW FEATURES YARN-1. Promote YARN to be a sub-project of Apache Hadoop. (acmurthy) IMPROVEMENTS YARN-29. Add a yarn-client module. (Vinod Kumar Vavilapalli via sseth) YARN-10. Fix DistributedShell module to not have a dependency on hadoop-mapreduce-client-core. (Hitesh Shah via vinodkv) YARN-80. Add support for delaying rack-local containers in CapacityScheduler. (acmurthy) YARN-137. Change the default YARN scheduler to be the CapacityScheduler. (sseth via acmurthy) OPTIMIZATIONS BUG FIXES YARN-13. Fix pom versions for YARN in branch-2 (todd) MAPREDUCE-2374. "Text File Busy" errors launching MR tasks. (Andy Isaacson via atm) YARN-12. Fix findbugs warnings in FairScheduler. (Junping Du via acmurthy) YARN-22. Fix ContainerLogs to work if the log-dir is specified as a URI. (Mayank Bansal via sseth) YARN-37. Change TestRMAppTransitions to use the DrainDispatcher. (Mayank Bansal via sseth) YARN-79. Implement close on all clients to YARN so that RPC clients don't throw exceptions on shut-down. (Vinod Kumar Vavilapalli) YARN-42. Modify NM's non-aggregating logs' handler to stop properly so that NMs don't get NPEs on startup errors. (Devaraj K via vinodkv) YARN-15. Updated default classpath for YARN applications to reflect split of YARN into a sub-project. (Arun C Murthy via vinodkv) YARN-75. Modified ResourceManager's RMContainer to handle a valid RELEASE event at RUNNING state. (Siddharth Seth via vinodkv) YARN-138. Ensure default values for minimum/maximum container sizes is sane. (harsh & sseth via acmurthy) Release 0.23.6 - UNRELEASED INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS OPTIMIZATIONS BUG FIXES YARN-188. Coverage fixing for CapacityScheduler (Aleksey Gorshkov via bobby) YARN-214. RMContainerImpl does not handle event EXPIRE at state RUNNING (jeagles via bobby) YARN-151. Browser thinks RM main page JS is taking too long (Ravi Prakash via bobby) Release 0.23.5 - UNRELEASED INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS YARN-161. Fix multiple compiler warnings for unchecked operations in YARN common. (Chris Nauroth via vinodkv) YARN-43. Fix TestResourceTrackerService to not depend on test order and thus pass on JDK7. (Thomas Graves via vinodkv) YARN-32. Fix TestApplicationTokens to not depend on test order and thus pass on JDK7. (vinodkv) YARN-186. Coverage fixing LinuxContainerExecutor (Aleksey Gorshkov via bobby) YARN-216. Remove jquery theming support. (Robert Joseph Evans via jlowe) OPTIMIZATIONS BUG FIXES YARN-163. Retrieving container log via NM webapp can hang with multibyte characters in log (jlowe via bobby) YARN-174. Modify NodeManager to pass the user's configuration even when rebooting. (vinodkv) YARN-177. CapacityScheduler - adding a queue while the RM is running has wacky results (acmurthy vai tgraves) YARN-178. Fix custom ProcessTree instance creation (Radim Kolar via bobby) YARN-180. Capacity scheduler - containers that get reserved create container token to early (acmurthy and bobby) YARN-139. Interrupted Exception within AsyncDispatcher leads to user confusion. (Vinod Kumar Vavilapalli via jlowe) YARN-165. RM should point tracking URL to RM web page for app when AM fails (jlowe via bobby) YARN-159. RM web ui applications page should be sorted to display last app first (tgraves via bobby) YARN-166. capacity scheduler doesn't allow capacity < 1.0 (tgraves via bobby) YARN-189. Fixed a deadlock between RM's ApplicationMasterService and the dispatcher. (Thomas Graves via vinodkv) YARN-202. Log Aggregation generates a storm of fsync() for namenode (Kihwal Lee via bobby) YARN-201. Fix CapacityScheduler to be less conservative for starved off-switch requests. (jlowe via acmurthy) YARN-206. TestApplicationCleanup.testContainerCleanup occasionally fails. (jlowe via jeagles) YARN-212. NM state machine ignores an APPLICATION_CONTAINER_FINISHED event when it shouldn't (Nathan Roberts via jlowe) Release 0.23.4 INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS Change package of YarnClient to org.apache.hadoop. (Bikas Saha via vinodkv) YARN-108. FSDownload can create cache directories with the wrong permissions (Jason Lowe via bobby) YARN-57. Allow process-tree based resource calculation et al. to be pluggable to support it on multiple platforms. (Radim Kolar via acmurthy) OPTIMIZATIONS BUG FIXES YARN-88. DefaultContainerExecutor can fail to set proper permissions. (Jason Lowe via sseth) YARN-106. Nodemanager needs to set permissions of local directories (jlowe via bobby) Release 0.23.3 INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS OPTIMAZATIONS BUG FIXES YARN-14. Symlinks to peer distributed cache files no longer work (Jason Lowe via bobby) YARN-25. remove old aggregated logs (Robert Evans via tgraves) YARN-27. Failed refreshQueues due to misconfiguration prevents further refreshing of queues (Arun Murthy via tgraves) MAPREDUCE-4323. NM leaks filesystems (Jason Lowe via jeagles) YARN-39. RM-NM secret-keys should be randomly generated and rolled every so often. (vinodkv and sseth via sseth) YARN-31. Fix TestDelegationTokenRenewer to not depend on test order so as to pass tests on jdk7. (Thomas Graves via vinodkv) YARN-63. RMNodeImpl is missing valid transitions from the UNHEALTHY state (Jason Lowe via bobby) YARN-60. Fixed a bug in ResourceManager which causes all NMs to get NPEs and thus causes all containers to be rejected. (vinodkv) YARN-66. aggregated logs permissions not set properly (tgraves via bobby) YARN-68. NodeManager will refuse to shutdown indefinitely due to container log aggregation (daryn via bobby) YARN-87. NM ResourceLocalizationService does not set permissions of local cache directories (Jason Lowe via tgraves)