diff --git a/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html b/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
index acd0c568c9a..29da38394e0 100644
--- a/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
+++ b/hadoop-common-project/hadoop-common/src/main/docs/releasenotes.html
@@ -1,5 +1,5 @@
-Hadoop 2.6.0 Release Notes
+Hadoop 2.6.1 Release Notes
-Hadoop 2.6.0 Release Notes
+Hadoop 2.6.1 Release Notes
These release notes include new developer and user-facing incompatibilities, features, and major improvements.
-Changes since Hadoop 2.5.1
+Changes since Hadoop 2.6.0
-- YARN-2853.
+
- YARN-4047.
+ Major bug reported by Jason Lowe and fixed by Jason Lowe (resourcemanager)
+ ClientRMService getApplications has high scheduler lock contention
+
+- YARN-3999.
Major bug reported by Jian He and fixed by Jian He
- Killing app may hang while AM is unregistering
+ RM hangs on draining events
-- YARN-2846.
- Blocker bug reported by Junping Du and fixed by Junping Du (nodemanager)
- Incorrect persist exit code for running containers in reacquireContainer() that interrupted by NodeManager restart.
+ - YARN-3990.
+ Critical bug reported by Rohith Sharma K S and fixed by Bibin A Chundatt (resourcemanager)
+ AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected
-- YARN-2843.
- Major sub-task reported by Sushmitha Sreenivasan and fixed by Wangda Tan (resourcemanager)
- NodeLabels manager should trim all inputs for hosts and labels
+ - YARN-3978.
+ Major improvement reported by Eric Payne and fixed by Eric Payne (timelineserver , yarn)
+ Configurably turn off the saving of container info in Generic AHS
-- YARN-2841.
- Critical sub-task reported by Jian He and fixed by Jian He (resourcemanager)
- RMProxy should retry EOFException
+ - YARN-3850.
+ Blocker bug reported by Varun Saxena and fixed by Varun Saxena (log-aggregation , nodemanager)
+ NM fails to read files from full disks which can lead to container logs being lost and other issues
-- YARN-2834.
- Blocker bug reported by Yesha Vora and fixed by Jian He
- Resource manager crashed with Null Pointer Exception
-
-- YARN-2830.
- Blocker bug reported by Jonathan Eagles and fixed by Jonathan Eagles
- Add backwords compatible ContainerId.newInstance constructor for use within Tez Local Mode
- I just committed this. Thanks [~jeagles] for the patch and [~ozawa] for the reviews!
-- YARN-2827.
- Critical bug reported by Wangda Tan and fixed by Wangda Tan (client)
- Fix bugs of yarn queue CLI
-
-- YARN-2826.
- Critical bug reported by Sidharta Seethana and fixed by Wangda Tan
- User-Group mappings not updated by RM when a user is removed from a group.
-
-- YARN-2825.
- Critical bug reported by Jian He and fixed by Jian He
- Container leak on NM
-
-- YARN-2824.
- Critical sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Capacity of labels should be zero by default
-
-- YARN-2823.
- Critical bug reported by Gour Saha and fixed by Jian He (resourcemanager)
- NullPointerException in RM HA enabled 3-node cluster
-
-- YARN-2819.
- Critical bug reported by Gopal V and fixed by Zhijie Shen (timelineserver)
- NPE in ATS Timeline Domains when upgrading from 2.4 to 2.6
-
-- YARN-2818.
- Critical bug reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Remove the logic to inject entity owner as the primary filter
-
-- YARN-2813.
- Major bug reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- NPE from MemoryTimelineStore.getDomains
-
-- YARN-2812.
- Major test reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- TestApplicationHistoryServer is likely to fail on less powerful machine
-
-- YARN-2810.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev (resourcemanager)
- TestRMProxyUsersConf fails on Windows VMs
-
-- YARN-2805.
- Blocker bug reported by Arpit Gupta and fixed by Wangda Tan (resourcemanager)
- RM2 in HA setup tries to login using the RM1's kerberos principal
-
-- YARN-2804.
- Critical bug reported by Zhijie Shen and fixed by Zhijie Shen
- Timeline server .out log have JAXB binding exceptions and warnings.
-
-- YARN-2803.
- Critical bug reported by Chris Nauroth and fixed by Craig Welch (nodemanager)
- MR distributed cache not working correctly on Windows after NodeManager privileged account changes.
-
-- YARN-2798.
- Blocker bug reported by Arpit Gupta and fixed by Zhijie Shen (timelineserver)
- YarnClient doesn't need to translate Kerberos name of timeline DT renewer
-
-- YARN-2795.
- Major sub-task reported by Phil D'Amore and fixed by Wangda Tan (resourcemanager)
- Resource Manager fails startup with HDFS label storage and secure cluster
-
-- YARN-2794.
- Major bug reported by Jian He and fixed by Jian He
- Fix log msgs about distributing system-credentials
-
-- YARN-2790.
- Critical bug reported by Tassapol Athiapinya and fixed by Jian He (nodemanager)
- NM can't aggregate logs past HDFS delegation token expiry.
-
-- YARN-2789.
- Critical task reported by Siddharth Seth and fixed by Wangda Tan
- Re-instate the NodeReport.newInstance API modified in YARN-2698
-
-- YARN-2788.
- Blocker bug reported by Gopal V and fixed by Xuan Gong (log-aggregation)
- yarn logs -applicationId on 2.6.0 should support logs written by 2.4.0
-
-- YARN-2785.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev
- TestContainerResourceUsage fails intermittently
-
-- YARN-2779.
- Critical bug reported by Zhijie Shen and fixed by Zhijie Shen (resourcemanager , timelineserver)
- SystemMetricsPublisher can use Kerberos directly instead of timeline DT
-
-- YARN-2778.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (client)
- YARN node CLI should display labels on returned node reports
-
-- YARN-2770.
- Critical sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Timeline delegation tokens need to be automatically renewed by the RM
-
-- YARN-2769.
- Major bug reported by Varun Vasudev and fixed by Varun Vasudev (applications/distributed-shell)
- Timeline server domain not set correctly when using shell_command on Windows
-
-- YARN-2767.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev (resourcemanager)
- RM web services - add test case to ensure the http static user cannot kill or submit apps in secure mode
-
-- YARN-2760.
- Trivial bug reported by Harsh J and fixed by Harsh J (documentation)
- Completely remove word 'experimental' from FairScheduler docs
-
-- YARN-2758.
- Major test reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Update TestApplicationHistoryClientService to use the new generic history store
-
-- YARN-2755.
- Critical bug reported by Siqi Li and fixed by Siqi Li
- NM fails to clean up usercache_DEL_<timestamp> dirs after YARN-661
-
-- YARN-2753.
- Major sub-task reported by zhihai xu and fixed by zhihai xu
- Fix potential issues and code clean up for *NodeLabelsManager
-
-- YARN-2752.
- Critical bug reported by Xuan Gong and fixed by Xuan Gong
- ContainerExecutor always append "nice -n" in command on branch-2
-
-- YARN-2747.
- Major test reported by Xuan Gong and fixed by Xuan Gong
- TestAggregatedLogFormat fails in trunk
-
-- YARN-2744.
- Critical sub-task reported by Sumit Mohanty and fixed by Wangda Tan (capacityscheduler)
- Under some scenario, it is possible to end up with capacity scheduler configuration that uses labels that no longer exist
-
-- YARN-2743.
- Blocker bug reported by Arpit Gupta and fixed by Jian He (resourcemanager)
- Yarn jobs via oozie fail with failed to renew token (secure) or digest mismatch (unsecure) errors when RM is being killed
-
-- YARN-2741.
- Major bug reported by Craig Welch and fixed by Craig Welch (nodemanager)
- Windows: Node manager cannot serve up log files via the web user interface when yarn.nodemanager.log-dirs to any drive letter other than C: (or, the drive that nodemanager is running on)
-
-- YARN-2734.
- Major bug reported by Sumit Mohanty and fixed by Xuan Gong (log-aggregation)
- If a sub-folder is encountered by log aggregator it results in invalid aggregated file
-
-- YARN-2732.
- Major bug reported by Jian He and fixed by Jian He
- Fix syntax error in SecureContainer.apt.vm
-
-- YARN-2730.
- Critical bug reported by Siqi Li and fixed by Siqi Li
- DefaultContainerExecutor runs only one localizer at a time
-
-- YARN-2726.
- Minor sub-task reported by Phil D'Amore and fixed by Wangda Tan (capacityscheduler)
- CapacityScheduler should explicitly log when an accessible label has no capacity
-
-- YARN-2724.
- Major bug reported by Sumit Mohanty and fixed by Xuan Gong (log-aggregation)
- If an unreadable file is encountered during log aggregation then aggregated file in HDFS badly formed
-
-- YARN-2723.
- Major sub-task reported by Phil D'Amore and fixed by Naganarasimha G R (client)
- rmadmin -replaceLabelsOnNode does not correctly parse port
-
-- YARN-2721.
- Blocker bug reported by Jian He and fixed by Jian He
- Race condition: ZKRMStateStore retry logic may throw NodeExist exception
-
-- YARN-2720.
- Major bug reported by Craig Welch and fixed by Craig Welch (nodemanager)
- Windows: Wildcard classpath variables not expanded against resources contained in archives
-
-- YARN-2717.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (log-aggregation)
- containerLogNotFound log shows multiple time for the same container
-
-- YARN-2715.
- Blocker bug reported by Zhijie Shen and fixed by Zhijie Shen (resourcemanager)
- Proxy user is problem for RPC interface if yarn.resourcemanager.webapp.proxyuser is not set.
-
-- YARN-2711.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev
- TestDefaultContainerExecutor#testContainerLaunchError fails on Windows
-
-- YARN-2709.
- Major sub-task reported by Li Lu and fixed by Li Lu
- Add retry for timeline client getDelegationToken method
-
-- YARN-2707.
- Minor bug reported by Ted Yu and fixed by Gera Shegalov
- Potential null dereference in FSDownload
-
-- YARN-2705.
- Critical sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
- Changes of RM node label manager default configuration
-
-- YARN-2704.
- Critical sub-task reported by Jian He and fixed by Jian He
- Localization and log-aggregation will fail if hdfs delegation token expired after token-max-life-time
-
-- YARN-2703.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
- Add logUploadedTime into LogValue for better display
-
-- YARN-2701.
- Blocker bug reported by Xuan Gong and fixed by Xuan Gong
- Potential race condition in startLocalizer when using LinuxContainerExecutor
-
-- YARN-2700.
- Major sub-task reported by Steve Loughran and fixed by Steve Loughran (api , resourcemanager)
- TestSecureRMRegistryOperations failing on windows: auth problems
-
-- YARN-2699.
- Blocker sub-task reported by Wangda Tan and fixed by Wangda Tan (client)
- Fix test timeout in TestResourceTrackerOnHA#testResourceTrackerOnHA
-
-- YARN-2698.
- Critical sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
- Move getClusterNodeLabels and getNodeToLabels to YarnClient instead of AdminService
-
-- YARN-2692.
- Major sub-task reported by Steve Loughran and fixed by Steve Loughran
- ktutil test hanging on some machines/ktutil versions
-
-- YARN-2689.
- Major sub-task reported by Steve Loughran and fixed by Steve Loughran (api , resourcemanager)
- TestSecureRMRegistryOperations failing on windows: secure ZK won't start
-
-- YARN-2685.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
- Resource on each label not correct when multiple NMs in a same host and some has label some not
-
-- YARN-2682.
- Minor bug reported by zhihai xu and fixed by zhihai xu (nodemanager)
- WindowsSecureContainerExecutor should not depend on DefaultContainerExecutor#getFirstApplicationDir.
-
-- YARN-2678.
- Major sub-task reported by Gour Saha and fixed by Steve Loughran (api , resourcemanager)
- Improved Yarn Registry service record structure
-
-- YARN-2677.
- Major sub-task reported by Steve Loughran and fixed by Steve Loughran (api , resourcemanager)
- registry punycoding of usernames doesn't fix all usernames to be DNS-valid
-
-- YARN-2676.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Timeline authentication filter should add support for proxy user
-
-- YARN-2673.
- Major sub-task reported by Li Lu and fixed by Li Lu
- Add retry for timeline client put APIs
-
-- YARN-2671.
- Blocker bug reported by Zhijie Shen and fixed by Wangda Tan (resourcemanager)
- ApplicationSubmissionContext change breaks the existing app submission
-
-- YARN-2668.
- Major sub-task reported by Steve Loughran and fixed by Steve Loughran (client)
- yarn-registry JAR won't link against ZK 3.4.5
-
-- YARN-2667.
- Minor bug reported by Yi Liu and fixed by Yi Liu
- Fix the release audit warning caused by hadoop-yarn-registry
-
-- YARN-2662.
- Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestCgroupsLCEResourcesHandler leaks file descriptors.
-
-- YARN-2656.
- Major bug reported by Varun Vasudev and fixed by Zhijie Shen (resourcemanager)
- RM web services authentication filter should add support for proxy user
-
-- YARN-2652.
- Major sub-task reported by Steve Loughran and fixed by Steve Loughran (api)
- add hadoop-yarn-registry package under hadoop-yarn
-
-- YARN-2651.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
- Spin off the LogRollingInterval from LogAggregationContext
-
-- YARN-2649.
- Major bug reported by Ming Ma and fixed by Ming Ma
- Flaky test TestAMRMRPCNodeUpdates
-
-- YARN-2647.
- Major sub-task reported by Wangda Tan and fixed by Sunil G (client)
- Add yarn queue CLI to get queue infos
-
-- YARN-2644.
- Major sub-task reported by Craig Welch and fixed by Craig Welch
- Recalculate headroom more frequently to keep it accurate
-
-- YARN-2635.
- Major bug reported by Wei Yan and fixed by Wei Yan
- TestRM, TestRMRestart, TestClientToAMTokens should run with both CS and FS
-
-- YARN-2632.
- Blocker sub-task reported by Junping Du and fixed by Junping Du (nodemanager)
- Document NM Restart feature
-
-- YARN-2630.
- Major bug reported by Jian He and fixed by Jian He
- TestDistributedShell#testDSRestartWithPreviousRunningContainers fails
-
-- YARN-2629.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Make distributed shell use the domain-based timeline ACLs
-
-- YARN-2628.
- Major bug reported by Varun Vasudev and fixed by Varun Vasudev (capacityscheduler)
- Capacity scheduler with DominantResourceCalculator carries out reservation even though slots are free
-
-- YARN-2627.
- Major improvement reported by Xuan Gong and fixed by Xuan Gong
- Add logs when attemptFailuresValidityInterval is enabled
-
-- YARN-2624.
- Blocker bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot (nodemanager)
+ - YARN-3832.
+ Critical bug reported by Ranga Swamy and fixed by Brahma Reddy Battula (nodemanager)
Resource Localization fails on a cluster due to existing cache directories
-- YARN-2621.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Simplify the output when the user doesn't have the access for getDomain(s)
+ - YARN-3733.
+ Blocker bug reported by Bibin A Chundatt and fixed by Rohith Sharma K S (resourcemanager)
+ Fix DominantRC#compare() does not work as expected if cluster resource is empty
-- YARN-2617.
- Major bug reported by Jun Gong and fixed by Jun Gong (nodemanager)
- NM does not need to send finished container whose APP is not running to RM
+ - YARN-3725.
+ Blocker bug reported by Zhijie Shen and fixed by Zhijie Shen (resourcemanager , timelineserver)
+ App submission via REST API is broken in secure mode due to Timeline DT service address is empty
-- YARN-2615.
- Blocker sub-task reported by Junping Du and fixed by Junping Du
- ClientToAMTokenIdentifier and DelegationTokenIdentifier should allow extended fields
+ - YARN-3700.
+ Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager , webapp , yarn)
+ ATS Web Performance issue at load time when large number of jobs
-- YARN-2611.
- Major sub-task reported by Subru Krishnan and fixed by Subru Krishnan (capacityscheduler , resourcemanager , scheduler)
- Fix jenkins findbugs warning and test case failures for trunk merge patch
+ - YARN-3585.
+ Critical bug reported by Peng Zhang and fixed by Rohith Sharma K S
+ NodeManager cannot exit on SHUTDOWN event triggered and NM recovery is enabled
-- YARN-2610.
- Major bug reported by Ray Chiang and fixed by Ray Chiang
- Hamlet should close table tags
+ - YARN-3526.
+ Major bug reported by Weiwei Yang and fixed by Weiwei Yang (resourcemanager , webapp)
+ ApplicationMaster tracking URL is incorrectly redirected on a QJM cluster
-- YARN-2608.
- Major bug reported by Wei Yan and fixed by Wei Yan
- FairScheduler: Potential deadlocks in loading alloc files and clock access
+ - YARN-3493.
+ Critical bug reported by Sumana Sathish and fixed by Jian He (yarn)
+ RM fails to come up with error "Failed to load/recover state" when mem settings are changed
-- YARN-2607.
- Major test reported by Ted Yu and fixed by Wangda Tan
- TestDistributedShell fails in trunk
+ - YARN-3487.
+ Critical sub-task reported by Jason Lowe and fixed by Jason Lowe (capacityscheduler)
+ CapacityScheduler scheduler lock obtained unnecessarily when calling getQueue
-- YARN-2606.
- Major bug reported by Mit Desai and fixed by Mit Desai (timelineserver)
- Application History Server tries to access hdfs before doing secure login
+ - YARN-3464.
+ Critical bug reported by zhihai xu and fixed by zhihai xu (nodemanager)
+ Race condition in LocalizerRunner kills localizer before localizing all resources
-- YARN-2602.
- Major bug reported by Karam Singh and fixed by Zhijie Shen (timelineserver)
- Generic History Service of TimelineServer sometimes not able to handle NPE
+ - YARN-3393.
+ Critical bug reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
+ Getting application(s) goes wrong when app finishes before starting the attempt
-- YARN-2596.
- Major test reported by Junping Du and fixed by Karthik Kambatla
- TestWorkPreservingRMRestart fails with FairScheduler
+ - YARN-3369.
+ Blocker bug reported by Giovanni Matteo Fumarola and fixed by Brahma Reddy Battula (resourcemanager)
+ Missing NullPointer check in AppSchedulingInfo causes RM to die
-- YARN-2594.
- Blocker bug reported by Karam Singh and fixed by Wangda Tan (resourcemanager)
- Potential deadlock in RM when querying ApplicationResourceUsageReport
+ - YARN-3287.
+ Major bug reported by Jonathan Eagles and fixed by Daryn Sharp
+ TimelineClient kerberos authentication failure uses wrong login context.
-- YARN-2591.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- AHSWebServices should return FORBIDDEN(403) if the request user doesn't have access to the history data
+ - YARN-3267.
+ Major bug reported by Prakash Ramachandran and fixed by Chang Li
+ Timelineserver applies the ACL rules after applying the limit on the number of records
-- YARN-2588.
- Major bug reported by Rohith and fixed by Rohith (resourcemanager)
- Standby RM does not transitionToActive if previous transitionToActive is failed with ZK exception.
+ - YARN-3251.
+ Blocker bug reported by Jason Lowe and fixed by Craig Welch
+ Fix CapacityScheduler deadlock when computing absolute max avail capacity (short term fix for 2.6.1)
-- YARN-2584.
- Major test reported by Zhijie Shen and fixed by Jian He
- TestContainerManagerSecurity fails on trunk
+ - YARN-3249.
+ Minor improvement reported by Ryu Kobayashi and fixed by Ryu Kobayashi (resourcemanager)
+ Add a "kill application" button to Resource Manager's Web UI
-- YARN-2583.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
- Modify the LogDeletionService to support Log aggregation for LRS
-
-- YARN-2582.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
- Log related CLI and Web UI changes for Aggregated Logs in LRS
-
-- YARN-2581.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
- NMs need to find a way to get LogAggregationContext
-
-- YARN-2579.
- Blocker bug reported by Rohith and fixed by Rohith (resourcemanager)
- Deadlock when EmbeddedElectorService and FatalEventDispatcher try to transition RM to StandBy at the same time
-
-- YARN-2577.
- Trivial improvement reported by Miklos Christine and fixed by Miklos Christine (documentation , fairscheduler)
- Clarify ACL delimiter and how to configure ACL groups only
-
-- YARN-2576.
- Major sub-task reported by Subru Krishnan and fixed by Subru Krishnan (capacityscheduler , resourcemanager , scheduler)
- Prepare yarn-1051 branch for merging with trunk
-
-- YARN-2569.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (nodemanager , resourcemanager)
- API changes for handling logs of long-running services
-
-- YARN-2568.
- Major bug reported by Jian He and fixed by Jian He
- TestAMRMClientOnRMRestart test fails
-
-- YARN-2566.
- Critical sub-task reported by zhihai xu and fixed by zhihai xu (nodemanager)
- DefaultContainerExecutor should pick a working directory randomly
-
-- YARN-2565.
- Major bug reported by Karam Singh and fixed by Zhijie Shen (resourcemanager , timelineserver)
- RM shouldn't use the old RMApplicationHistoryWriter unless explicitly setting FileSystemApplicationHistoryStore
-
-- YARN-2563.
- Blocker bug reported by Arpit Gupta and fixed by Zhijie Shen (timelineserver)
- On secure clusters call to timeline server fails with authentication errors when running a job via oozie
-
-- YARN-2562.
- Critical bug reported by Vinod Kumar Vavilapalli and fixed by Tsuyoshi OZAWA
- ContainerId@toString() is unreadable for epoch >0 after YARN-2182
-
-- YARN-2561.
- Blocker sub-task reported by Tassapol Athiapinya and fixed by Junping Du
- MR job client cannot reconnect to AM after NM restart.
-
-- YARN-2559.
- Major bug reported by Karam Singh and fixed by Zhijie Shen (resourcemanager , timelineserver)
- ResourceManager sometime become un-responsive due to NPE in SystemMetricsPublisher
-
-- YARN-2558.
- Blocker sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA
- Updating ContainerTokenIdentifier#read/write to use ContainerId#getContainerId
-
-- YARN-2557.
- Major bug reported by Xuan Gong and fixed by Xuan Gong (applications/distributed-shell)
- Add a parameter "attempt_Failures_Validity_Interval" in DistributedShell
-
-- YARN-2549.
- Minor test reported by Chris Nauroth and fixed by Chris Nauroth (nodemanager , test)
- TestContainerLaunch fails due to classpath problem with hamcrest classes.
-
-- YARN-2547.
- Major sub-task reported by Jonathan Eagles and fixed by Mit Desai (timelineserver)
- Cross Origin Filter throws UnsupportedOperationException upon destroy
-
-- YARN-2546.
- Major bug reported by Doug Haigh and fixed by Varun Vasudev (api)
- REST API for application creation/submission is using strings for numeric & boolean values
-
-- YARN-2544.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api , client , resourcemanager)
- Common server side PB changes (not include user API PB changes)
-
-- YARN-2542.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen
- "yarn application -status <appId>" throws NPE when retrieving the app from the timelineserver
-
-- YARN-2541.
- Major bug reported by Jian He and fixed by Jian He
- Fix ResourceManagerRest.apt.vm syntax error
-
-- YARN-2540.
- Major bug reported by Ashwin Shankar and fixed by Ashwin Shankar (scheduler)
- FairScheduler: Queue filters not working on scheduler page in RM UI
-
-- YARN-2539.
- Minor improvement reported by Wei Yan and fixed by Wei Yan
- FairScheduler: Set the default value for maxAMShare to 0.5
-
-- YARN-2538.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
- Add logs when RM send new AMRMToken to ApplicationMaster
-
-- YARN-2534.
- Major bug reported by zhihai xu and fixed by zhihai xu (scheduler)
- FairScheduler: Potential integer overflow calculating totalMaxShare
-
-- YARN-2531.
- Major improvement reported by Varun Vasudev and fixed by Varun Vasudev
- CGroups - Admins should be allowed to enforce strict cpu limits
-
-- YARN-2529.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Generic history service RPC interface doesn't work when service authorization is enabled
-
-- YARN-2528.
- Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (timelineserver)
- Cross Origin Filter Http response split vulnerability protection rejects valid origins
-
-- YARN-2527.
- Major bug reported by Benoy Antony and fixed by Benoy Antony (resourcemanager)
- NPE in ApplicationACLsManager
-
-- YARN-2526.
- Critical bug reported by Wei Yan and fixed by Wei Yan (scheduler-load-simulator)
- SLS can deadlock when all the threads are taken by AMSimulators
-
-- YARN-2523.
- Major bug reported by Nishan Shetty and fixed by Rohith (resourcemanager , webapp)
- ResourceManager UI showing negative value for "Decommissioned Nodes" field
-
-- YARN-2519.
- Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (webapp)
- Credential Provider related unit tests failed on Windows
-
-- YARN-2515.
- Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (resourcemanager)
- Update ConverterUtils#toContainerId to parse epoch
-
-- YARN-2512.
- Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (timelineserver)
- Allow for origin pattern matching in cross origin filter
-
-- YARN-2511.
- Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (timelineserver)
- Allow All Origins by default when Cross Origin Filter is enabled
-
-- YARN-2509.
- Major sub-task reported by Jonathan Eagles and fixed by Mit Desai (timelineserver)
- Enable Cross Origin Filter for timeline server only and not all Yarn servers
-
-- YARN-2508.
- Major sub-task reported by Jonathan Eagles and fixed by Mit Desai (timelineserver)
- Cross Origin configuration parameters prefix are not honored
-
-- YARN-2507.
- Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (documentation , timelineserver)
- Document Cross Origin Filter Configuration for ATS
-
-- YARN-2505.
- Major sub-task reported by Wangda Tan and fixed by Craig Welch (resourcemanager)
- Support get/add/remove/change labels in RM REST API
-
-- YARN-2504.
- Critical sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Support get/add/remove/change labels in RM admin CLI
-
-- YARN-2503.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Changes in RM Web UI to better show labels to end users
-
-- YARN-2502.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Changes in distributed shell to support specify labels
-
-- YARN-2501.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Changes in AMRMClient to support labels
-
-- YARN-2500.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Miscellaneous changes in ResourceManager to support labels
-
-- YARN-2496.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Changes for capacity scheduler to support allocate resource respect labels
-
-- YARN-2494.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (resourcemanager)
- Node label manager API and storage implementations
-
-- YARN-2493.
- Major sub-task reported by Wangda Tan and fixed by Wangda Tan (api)
- API changes for users
-
-- YARN-2484.
- Trivial bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA
- FileSystemRMStateStore#readFile/writeFile should close FSData(In|Out)putStream in final block
-
-- YARN-2475.
- Major sub-task reported by Carlo Curino and fixed by Carlo Curino (resourcemanager)
- ReservationSystem: replan upon capacity reduction
-
-- YARN-2468.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (log-aggregation , nodemanager , resourcemanager)
- Log handling for LRS
-
-- YARN-2462.
- Major bug reported by Jason Lowe and fixed by Eric Payne
- TestNodeManagerResync#testBlockNewContainerRequestsOnStartAndResync should have a test timeout
-
-- YARN-2460.
- Minor bug reported by Ray Chiang and fixed by Ray Chiang
- Remove obsolete entries from yarn-default.xml
-
-- YARN-2459.
- Major bug reported by Mayank Bansal and fixed by Mayank Bansal (resourcemanager)
- RM crashes if App gets rejected for any reason and HA is enabled
-
-- YARN-2456.
- Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
- Possible livelock in CapacityScheduler when RM is recovering apps
-
-- YARN-2453.
- Major bug reported by zhihai xu and fixed by zhihai xu
- TestProportionalCapacityPreemptionPolicy fails with FairScheduler
-
-- YARN-2452.
- Major bug reported by zhihai xu and fixed by zhihai xu
- TestRMApplicationHistoryWriter fails with FairScheduler
-
-- YARN-2450.
- Trivial bug reported by Ray Chiang and fixed by Ray Chiang
- Fix typos in log messages
-
-- YARN-2449.
- Critical bug reported by Karam Singh and fixed by Varun Vasudev (timelineserver)
- Timelineserver returns invalid Delegation token in secure kerberos enabled cluster when hadoop.http.filter.initializers are not set
-
-- YARN-2448.
- Major improvement reported by Varun Vasudev and fixed by Varun Vasudev
- RM should expose the resource types considered during scheduling when AMs register
-
-- YARN-2447.
- Major bug reported by Varun Vasudev and fixed by Varun Vasudev
- RM web services app submission doesn't pass secrets correctly
-
-- YARN-2446.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Using TimelineNamespace to shield the entities of a user
-
-- YARN-2440.
- Major bug reported by Varun Vasudev and fixed by Varun Vasudev
- Cgroups should allow YARN containers to be limited to allocated cores
-
-- YARN-2434.
- Major sub-task reported by Jian He and fixed by Jian He
- RM should not recover containers from previously failed attempt when AM restart is not enabled
-
-- YARN-2431.
- Major sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)
- NM restart: cgroup is not removed for reacquired containers
-
-- YARN-2424.
- Blocker bug reported by Allen Wittenauer and fixed by Allen Wittenauer (nodemanager)
- LCE should support non-cgroups, non-secure mode
-
-- YARN-2411.
- Major improvement reported by Ram Venkatesh and fixed by Ram Venkatesh (capacityscheduler)
- [Capacity Scheduler] support simple user and group mappings to queues
-
-- YARN-2409.
- Critical bug reported by Nishan Shetty and fixed by Rohith (resourcemanager)
- Active to StandBy transition does not stop rmDispatcher that causes 1 AsyncDispatcher thread leak.
-
-- YARN-2406.
- Major sub-task reported by Jian He and fixed by Tsuyoshi OZAWA
- Move RM recovery related proto to yarn_server_resourcemanager_recovery.proto
-
-- YARN-2405.
- Major bug reported by Maysam Yabandeh and fixed by Tsuyoshi OZAWA
- NPE in FairSchedulerAppsBlock
-
-- YARN-2400.
- Major bug reported by Jian He and fixed by Jian He
- TestAMRestart fails intermittently
-
-- YARN-2399.
- Major improvement reported by Karthik Kambatla and fixed by Karthik Kambatla (fairscheduler)
- FairScheduler: Merge AppSchedulable and FSSchedulerApp into FSAppAttempt
-
-- YARN-2397.
- Critical bug reported by Varun Vasudev and fixed by Varun Vasudev
- RM and TS web interfaces sometimes return request is a replay error in secure mode
-
-- YARN-2395.
- Major new feature reported by Ashwin Shankar and fixed by Wei Yan (fairscheduler)
- FairScheduler: Preemption timeout should be configurable per queue
-
-- YARN-2394.
- Major new feature reported by Ashwin Shankar and fixed by Wei Yan (fairscheduler)
- FairScheduler: Configure fairSharePreemptionThreshold per queue
-
-- YARN-2393.
- Major new feature reported by Ashwin Shankar and fixed by Wei Yan (fairscheduler)
- FairScheduler: Add the notion of steady fair share
-
-- YARN-2389.
- Major sub-task reported by Subru Krishnan and fixed by Subru Krishnan (capacityscheduler , fairscheduler)
- Adding support for drainig a queue, ie killing all apps in the queue
-
-- YARN-2388.
- Major test reported by Zhijie Shen and fixed by Zhijie Shen
- TestTimelineWebServices fails on trunk after HADOOP-10791
-
-- YARN-2387.
- Blocker bug reported by Mit Desai and fixed by Mit Desai
- Resource Manager crashes with NPE due to lack of synchronization
-
-- YARN-2378.
- Major sub-task reported by Subru Krishnan and fixed by Subru Krishnan (capacityscheduler)
- Adding support for moving apps between queues in Capacity Scheduler
-
-- YARN-2377.
- Major improvement reported by Gera Shegalov and fixed by Gera Shegalov (nodemanager)
- Localization exception stack traces are not passed as diagnostic info
-
-- YARN-2374.
- Major bug reported by Varun Vasudev and fixed by Varun Vasudev
- YARN trunk build failing TestDistributedShell.testDSShell
-
-- YARN-2373.
- Major bug reported by Larry McCay and fixed by Larry McCay
- WebAppUtils Should Use configuration.getPassword for Accessing SSL Passwords
-
-- YARN-2372.
- Minor improvement reported by Fengdong Yu and fixed by Fengdong Yu (documentation)
- There are Chinese Characters in the FairScheduler's document
-
-- YARN-2370.
- Trivial bug reported by Wenwu Peng and fixed by Wenwu Peng (resourcemanager)
- Fix comment in o.a.h.y.server.resourcemanager.schedulerAppSchedulingInfo
-
-- YARN-2363.
- Major bug reported by Jason Lowe and fixed by Jason Lowe (resourcemanager)
- Submitted applications occasionally lack a tracking URL
-
-- YARN-2361.
- Trivial improvement reported by zhihai xu and fixed by zhihai xu (resourcemanager)
- RMAppAttempt state machine entries for KILLED state has duplicate event entries
-
-- YARN-2360.
- Major new feature reported by Ashwin Shankar and fixed by Ashwin Shankar (fairscheduler)
- Fair Scheduler: Display dynamic fair share for queues on the scheduler page
-
-- YARN-2359.
+
- YARN-3242.
Critical bug reported by zhihai xu and fixed by zhihai xu (resourcemanager)
- Application hangs when it fails to launch AM container
+ Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving events for old client
-- YARN-2354.
- Major sub-task reported by Jian He and fixed by Li Lu
- DistributedShell may allocate more containers than client specified after it restarts
-
-- YARN-2352.
- Major improvement reported by Karthik Kambatla and fixed by Karthik Kambatla (scheduler)
- FairScheduler: Collect metrics on duration of critical methods that affect performance
-
-- YARN-2347.
- Major sub-task reported by Junping Du and fixed by Junping Du
- Consolidate RMStateVersion and NMDBSchemaVersion into StateVersion in yarn-server-common
-
-- YARN-2343.
- Trivial improvement reported by Li Lu and fixed by Li Lu
- Improve error message on token expire exception
-
-- YARN-2337.
- Trivial improvement reported by zhihai xu and fixed by zhihai xu (resourcemanager)
- ResourceManager sets ClientRMService in RMContext multiple times
-
-- YARN-2323.
- Minor improvement reported by Hong Zhiguo and fixed by Hong Zhiguo (fairscheduler)
- FairShareComparator creates too many Resource objects
-
-- YARN-2321.
- Major bug reported by Leitao Guo and fixed by Leitao Guo (nodemanager)
- NodeManager web UI can incorrectly report Pmem enforcement
-
-- YARN-2317.
- Major sub-task reported by Li Lu and fixed by Li Lu (documentation)
- Update documentation about how to write YARN applications
-
-- YARN-2314.
- Critical bug reported by Jason Lowe and fixed by Jason Lowe (client)
- ContainerManagementProtocolProxy can create thousands of threads for a large cluster
-
-- YARN-2313.
- Major bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (fairscheduler)
- Livelock can occur in FairScheduler when there are lots of running apps
-
-- YARN-2312.
- Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (resourcemanager)
- Marking ContainerId#getId as deprecated
-
-- YARN-2308.
- Critical bug reported by Wangda Tan and fixed by chang li (resourcemanager , scheduler)
- NPE happened when RM restart after CapacityScheduler queue configuration changed
-
-- YARN-2302.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Refactor TimelineWebServices
-
-- YARN-2298.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (client)
- Move TimelineClient to yarn-common project
-
-- YARN-2295.
- Major sub-task reported by Li Lu and fixed by Li Lu
- Refactor YARN distributed shell with existing public stable API
-
-- YARN-2288.
- Major sub-task reported by Junping Du and fixed by Junping Du (timelineserver)
- Data persistent in timelinestore should be versioned
-
-- YARN-2279.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen (timelineserver)
- Add UTs to cover timeline server authentication
-
-- YARN-2277.
- Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles (timelineserver)
- Add Cross-Origin support to the ATS REST API
-
-- YARN-2274.
- Trivial improvement reported by Karthik Kambatla and fixed by Karthik Kambatla (fairscheduler)
- FairScheduler: Add debug information about cluster capacity, availability and reservations
-
-- YARN-2273.
- Major bug reported by Andy Skelton and fixed by Wei Yan (fairscheduler , resourcemanager)
- NPE in ContinuousScheduling thread when we lose a node
-
-- YARN-2269.
- Major bug reported by Yesha Vora and fixed by Craig Welch
- External links need to be removed from YARN UI
-
-- YARN-2264.
- Major bug reported by Siddharth Seth and fixed by Li Lu
- Race in DrainDispatcher can cause random test failures
-
-- YARN-2260.
- Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
- Add containers to launchedContainers list in RMNode on container recovery
-
-- YARN-2252.
- Major bug reported by Ratandeep Ratti and fixed by (scheduler)
- Intermittent failure of TestFairScheduler.testContinuousScheduling
-
-- YARN-2251.
- Major bug reported by Zhijie Shen and fixed by Zhijie Shen
- Avoid negative elapsed time in JHS/MRAM web UI and services
-
-- YARN-2249.
- Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
- AM release request may be lost on RM restart
-
-- YARN-2248.
- Major sub-task reported by Janos Matyas and fixed by Janos Matyas (capacityscheduler)
- Capacity Scheduler changes for moving apps between queues
-
-- YARN-2244.
- Critical bug reported by Anubhav Dhoot and fixed by Anubhav Dhoot (fairscheduler)
- FairScheduler missing handling of containers for unknown application attempts
-
-- YARN-2242.
- Major sub-task reported by Li Lu and fixed by Li Lu
- Improve exception information on AM launch crashes
-
-- YARN-2237.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
- MRAppMaster changes for AMRMToken roll-up
-
-- YARN-2229.
- Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (resourcemanager)
- ContainerId can overflow with RM restart
-
-- YARN-2228.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen
- TimelineServer should load pseudo authentication filter when authentication = simple
-
-- YARN-2219.
- Major bug reported by Ashwin Shankar and fixed by Jian He (resourcemanager)
- AMs and NMs can get exceptions after recovery but before scheduler knowns apps and app-attempts
-
-- YARN-2214.
- Major improvement reported by Ashwin Shankar and fixed by Ashwin Shankar (scheduler)
- FairScheduler: preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness
-
-- YARN-2212.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
- ApplicationMaster needs to find a way to update the AMRMToken periodically
-
-- YARN-2211.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
- RMStateStore needs to save AMRMToken master key for recovery when RM restart/failover happens
-
-- YARN-2209.
- Major improvement reported by Jian He and fixed by Jian He
- Replace AM resync/shutdown command with corresponding exceptions
-
-- YARN-2208.
- Major sub-task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
- AMRMTokenManager need to have a way to roll over AMRMToken
-
-- YARN-2207.
- Major task reported by Xuan Gong and fixed by Xuan Gong (resourcemanager)
- Add ability to roll over AMRMToken
-
-- YARN-2198.
- Major improvement reported by Remus Rusanu and fixed by Remus Rusanu
- Remove the need to run NodeManager as privileged account for Windows Secure Container Executor
-
-- YARN-2197.
- Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Add a link to YARN CHANGES.txt in the left side of doc
-
-- YARN-2182.
- Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA
- Update ContainerId#toString() to avoid conflicts before and after RM restart
-
-- YARN-2181.
- Major bug reported by Wangda Tan and fixed by Wangda Tan (resourcemanager , webapp)
- Add preemption info to RM Web UI and add logs when preemption occurs
-
-- YARN-2174.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen
- Enabling HTTPs for the writer REST API of TimelineServer
-
-- YARN-2161.
- Major bug reported by Binglin Chang and fixed by Binglin Chang
- Fix build on macosx: YARN parts
-
-- YARN-2158.
- Minor test reported by Ted Yu and fixed by Varun Vasudev
- TestRMWebServicesAppsModification sometimes fails in trunk
-
-- YARN-2153.
- Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
- Ensure distributed shell work with RM work-preserving recovery
-
-- YARN-2147.
- Minor bug reported by Jason Lowe and fixed by Chen He (resourcemanager)
- client lacks delegation token exception details when application submit fails
-
-- YARN-2138.
- Major bug reported by Jian He and fixed by Varun Saxena
- Cleanup notifyDone* methods in RMStateStore
-
-- YARN-2131.
- Major new feature reported by Karthik Kambatla and fixed by Robert Kanter (resourcemanager)
- Add a way to format the RMStateStore
-
-- YARN-2102.
- Major sub-task reported by Zhijie Shen and fixed by Zhijie Shen
- More generalized timeline ACLs
-
-- YARN-2088.
- Major bug reported by Binglin Chang and fixed by Binglin Chang
- Fix code bug in GetApplicationsRequestPBImpl#mergeLocalToBuilder
-
-- YARN-2080.
- Major sub-task reported by Subru Krishnan and fixed by Subru Krishnan (resourcemanager)
- Admission Control: Integrate Reservation subsystem with ResourceManager
-
-- YARN-2070.
- Minor sub-task reported by Zhijie Shen and fixed by Robert Kanter
- DistributedShell publishes unfriendly user information to the timeline server
-
-- YARN-2045.
- Major sub-task reported by Junping Du and fixed by Junping Du (nodemanager)
- Data persisted in NM should be versioned
-
-- YARN-2035.
- Major sub-task reported by Jonathan Eagles and fixed by Jonathan Eagles
- FileSystemApplicationHistoryStore blocks RM and AHS while NN is in safemode
-
-- YARN-2034.
- Minor bug reported by Jason Lowe and fixed by Chen He (nodemanager)
- Description for yarn.nodemanager.localizer.cache.target-size-mb is incorrect
-
-- YARN-2033.
- Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Zhijie Shen
- Merging generic-history into the Timeline Store
-
-- YARN-2026.
- Major bug reported by Ashwin Shankar and fixed by Ashwin Shankar (scheduler)
- Fair scheduler: Consider only active queues for computing fairshare
-
-- YARN-2013.
- Major sub-task reported by Zhijie Shen and fixed by Tsuyoshi OZAWA (nodemanager)
- The diagnostics is always the ExitCodeException stack when the container crashes
-
-- YARN-2010.
- Blocker bug reported by bc Wong and fixed by Karthik Kambatla (resourcemanager)
- Handle app-recovery failures gracefully
-
-- YARN-2008.
- Major sub-task reported by Chen He and fixed by Craig Welch
- CapacityScheduler may report incorrect queueMaxCap if there is hierarchy queue structure
-
-- YARN-2001.
- Major sub-task reported by Jian He and fixed by Jian He (resourcemanager)
- Threshold for RM to accept requests from AM after failover
-
-- YARN-1994.
- Major improvement reported by Arpit Agarwal and fixed by Craig Welch (nodemanager , resourcemanager , webapp)
- Expose YARN/MR endpoints on multiple interfaces
-
-- YARN-1972.
- Major sub-task reported by Remus Rusanu and fixed by Remus Rusanu (nodemanager)
- Implement secure Windows Container Executor
-
-- YARN-1964.
- Major new feature reported by Arun C Murthy and fixed by Abin Shahab
- Create Docker analog of the LinuxContainerExecutor in YARN
-
-- YARN-1959.
- Major bug reported by Sandy Ryza and fixed by Anubhav Dhoot
- Fix headroom calculation in FairScheduler
-
-- YARN-1954.
- Major improvement reported by Zhijie Shen and fixed by Tsuyoshi OZAWA (client)
- Add waitFor to AMRMClient(Async)
-
-- YARN-1922.
- Major bug reported by Billie Rinaldi and fixed by Billie Rinaldi (nodemanager)
- Process group remains alive after container process is killed externally
-
-- YARN-1919.
- Minor bug reported by Devaraj K and fixed by Tsuyoshi OZAWA (resourcemanager)
- Potential NPE in EmbeddedElectorService#stop
-
-- YARN-1918.
- Trivial improvement reported by Devaraj K and fixed by Anandha L Ranganathan
- Typo in description and error message for 'yarn.resourcemanager.cluster-id'
-
-- YARN-1915.
- Blocker sub-task reported by Hitesh Shah and fixed by Jason Lowe
- ClientToAMTokenMasterKey should be provided to AM at launch time
-
-- YARN-1879.
- Critical sub-task reported by Jian He and fixed by Tsuyoshi OZAWA (resourcemanager)
- Mark Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over
-
-- YARN-1857.
- Critical sub-task reported by Thomas Graves and fixed by Chen He (capacityscheduler)
- CapacityScheduler headroom doesn't account for other AM's running
-
-- YARN-1796.
- Minor bug reported by Aaron T. Myers and fixed by Aaron T. Myers (nodemanager)
- container-executor shouldn't require o-r permissions
-
-- YARN-1779.
- Blocker sub-task reported by Karthik Kambatla and fixed by Jian He (resourcemanager)
- Handle AMRMTokens across RM failover
-
-- YARN-1769.
- Major improvement reported by Thomas Graves and fixed by Thomas Graves (capacityscheduler)
- CapacityScheduler: Improve reservations
-
-- YARN-1712.
- Major sub-task reported by Carlo Curino and fixed by Carlo Curino (capacityscheduler , resourcemanager)
- Admission Control: plan follower
-
-- YARN-1711.
- Major sub-task reported by Carlo Curino and fixed by Carlo Curino
- CapacityOverTimePolicy: a policy to enforce quotas over time for YARN-1709
-
-- YARN-1710.
- Major sub-task reported by Carlo Curino and fixed by Carlo Curino (resourcemanager)
- Admission Control: agents to allocate reservation
-
-- YARN-1709.
- Major sub-task reported by Carlo Curino and fixed by Subru Krishnan (resourcemanager)
- Admission Control: Reservation subsystem
-
-- YARN-1708.
- Major sub-task reported by Carlo Curino and fixed by Subru Krishnan (resourcemanager)
- Add a public API to reserve resources (part of YARN-1051)
-
-- YARN-1707.
- Major sub-task reported by Carlo Curino and fixed by Carlo Curino (capacityscheduler)
- Making the CapacityScheduler more dynamic
-
-- YARN-1372.
- Major sub-task reported by Bikas Saha and fixed by Anubhav Dhoot (resourcemanager)
- Ensure all completed containers are reported to the AMs across RM restart
-
-- YARN-1370.
- Major sub-task reported by Bikas Saha and fixed by Anubhav Dhoot (resourcemanager)
- Fair scheduler to re-populate container allocation state
-
-- YARN-1367.
- Major sub-task reported by Bikas Saha and fixed by Anubhav Dhoot (resourcemanager)
- After restart NM should resync with the RM without killing containers
-
-- YARN-1354.
- Major sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)
- Recover applications upon nodemanager restart
-
-- YARN-1342.
- Major sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)
- Recover container tokens upon nodemanager restart
-
-- YARN-1341.
- Major sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)
- Recover NMTokens upon nodemanager restart
-
-- YARN-1337.
- Major sub-task reported by Jason Lowe and fixed by Jason Lowe (nodemanager)
- Recover containers upon nodemanager restart
-
-- YARN-1326.
- Major sub-task reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA
- RM should log using RMStore at startup time
-
-- YARN-1250.
- Major sub-task reported by Vinod Kumar Vavilapalli and fixed by Zhijie Shen
- Generic history service should support application-acls
-
-- YARN-1063.
- Major sub-task reported by Kyle Leckie and fixed by Remus Rusanu (nodemanager)
- Winutils needs ability to create task as domain user
-
-- YARN-1051.
- Major improvement reported by Carlo Curino and fixed by Carlo Curino (capacityscheduler , resourcemanager , scheduler)
- YARN Admission Control/Planner: enhancing the resource allocation model with time.
-
-- YARN-668.
- Blocker sub-task reported by Siddharth Seth and fixed by Junping Du
- TokenIdentifier serialization should consider Unknown fields
-
-- YARN-611.
- Major sub-task reported by Chris Riccomini and fixed by Xuan Gong (resourcemanager)
- Add an AM retry count reset window to YARN RM
-
-- YARN-415.
- Major new feature reported by Kendall Thrapp and fixed by Eric Payne (resourcemanager)
- Capture aggregate memory allocation at the app-level for chargeback
-
-- YARN-90.
- Major sub-task reported by Ravi Gummadi and fixed by Varun Vasudev (nodemanager)
- NodeManager should identify failed disks becoming good again
-
-- MAPREDUCE-6156.
- Blocker bug reported by Sidharta Seethana and fixed by Junping Du
- Fetcher - connect() doesn't handle connection refused correctly
-
-- MAPREDUCE-6142.
- Critical sub-task reported by Zhijie Shen and fixed by Zhijie Shen
- Test failure in TestJobHistoryEventHandler and TestMRTimelineEventHandling
-
-- MAPREDUCE-6126.
- Major bug reported by Junping Du and fixed by Junping Du
- (Rumen) Rumen tool returns error "ava.lang.IllegalArgumentException: JobBuilder.process(HistoryEvent): unknown event type"
-
-- MAPREDUCE-6125.
- Major bug reported by Mit Desai and fixed by Mit Desai (test)
- TestContainerLauncherImpl sometimes fails
-
-- MAPREDUCE-6123.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestCombineFileInputFormat incorrectly starts 2 MiniDFSCluster instances.
-
-- MAPREDUCE-6122.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestLineRecordReader may fail due to test data files checked out of git with incorrect line endings.
-
-- MAPREDUCE-6115.
- Minor test reported by Ted Yu and fixed by Binglin Chang
- TestPipeApplication#testSubmitter fails in trunk
-
-- MAPREDUCE-6109.
- Trivial bug reported by Charles Lamb and fixed by Charles Lamb (distcp)
- Fix minor typo in distcp -p usage text
-
-- MAPREDUCE-6104.
- Major bug reported by Mit Desai and fixed by Mit Desai
- TestJobHistoryParsing.testPartialJob fails in branch-2
-
-- MAPREDUCE-6095.
- Major bug reported by Gera Shegalov and fixed by Gera Shegalov (applicationmaster , distributed-cache)
- Enable DistributedCache for uber-mode Jobs
-
-- MAPREDUCE-6094.
- Minor bug reported by Sangjin Lee and fixed by Akira AJISAKA (test)
- TestMRCJCFileInputFormat.testAddInputPath() fails on trunk
-
-- MAPREDUCE-6093.
- Trivial bug reported by Charles Lamb and fixed by Charles Lamb (distcp , documentation)
- minor distcp doc edits
-
-- MAPREDUCE-6091.
- Major bug reported by Sangjin Lee and fixed by Sangjin Lee (client)
- YARNRunner.getJobStatus() fails with ApplicationNotFoundException if the job rolled off the RM view
-
-- MAPREDUCE-6090.
- Major bug reported by Robert Kanter and fixed by Robert Kanter (client)
- mapred hsadmin getGroups fails to connect in some cases
-
-- MAPREDUCE-6086.
- Major improvement reported by zhihai xu and fixed by zhihai xu (security)
- mapreduce.job.credentials.binary should allow all URIs
-
-- MAPREDUCE-6075.
- Major bug reported by Jason Lowe and fixed by Jason Lowe (jobhistoryserver)
- HistoryServerFileSystemStateStore can create zero-length files
-
-- MAPREDUCE-6073.
- Trivial bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (documentation)
- Description of mapreduce.job.speculative.slowtaskthreshold in mapred-default should be moved into description tags
-
-- MAPREDUCE-6072.
- Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Remove INSTALL document
-
-- MAPREDUCE-6071.
- Trivial improvement reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (client)
- JobImpl#makeUberDecision doesn't log that Uber mode is disabled because of too much CPUs
-
-- MAPREDUCE-6070.
- Trivial improvement reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (documentation)
- yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented
-
-- MAPREDUCE-6063.
- Major bug reported by zhihai xu and fixed by zhihai xu (mrv1 , mrv2)
- In sortAndSpill of MapTask.java, size is calculated wrongly when bufend < bufstart.
-
-- MAPREDUCE-6052.
- Major bug reported by Junping Du and fixed by Junping Du
- Support overriding log4j.properties per job
-
-- MAPREDUCE-6051.
- Trivial bug reported by Ray Chiang and fixed by Ray Chiang
- Fix typos in log messages
-
-- MAPREDUCE-6048.
- Minor test reported by Ted Yu and fixed by Varun Vasudev
- TestJavaSerialization fails in trunk build
-
-- MAPREDUCE-6044.
- Major bug reported by Zhijie Shen and fixed by Zhijie Shen (jobhistoryserver)
- Fully qualified intermediate done directory will break per-user dir creation on Windows
-
-- MAPREDUCE-6036.
- Major bug reported by Mit Desai and fixed by chang li
- TestJobEndNotifier fails intermittently in branch-2
-
-- MAPREDUCE-6032.
- Major bug reported by Benjamin Zhitomirsky and fixed by Benjamin Zhitomirsky (jobhistoryserver)
- Unable to check mapreduce job status if submitted using a non-default namenode
-
-- MAPREDUCE-6029.
- Major bug reported by Ted Yu and fixed by Mit Desai
- TestCommitterEventHandler fails in trunk
-
-- MAPREDUCE-6024.
- Critical improvement reported by zhaoyunjiong and fixed by zhaoyunjiong (mr-am , task)
- java.net.SocketTimeoutException in Fetcher caused jobs stuck for more than 1 hour
-
-- MAPREDUCE-6022.
- Major bug reported by Jason Lowe and fixed by Jason Lowe
- map_input_file is missing from streaming job environment
-
-- MAPREDUCE-6021.
- Major bug reported by Jason Lowe and fixed by Jason Lowe (mr-am)
- MR AM should have working directory in LD_LIBRARY_PATH
-
-- MAPREDUCE-6019.
- Major bug reported by Xuan Gong and fixed by Craig Welch
- MapReduce changes for exposing YARN/MR endpoints on multiple interfaces.
-
-- MAPREDUCE-6018.
- Major sub-task reported by Jonathan Eagles and fixed by Robert Kanter
- Create a framework specific config to enable timeline server
-
-- MAPREDUCE-6014.
- Major bug reported by Mit Desai and fixed by Mit Desai
- New task status field in task attempts table can lead to an empty web page
-
-- MAPREDUCE-6012.
- Major bug reported by Julien Serdaru and fixed by Wei Yan
- DBInputSplit creates invalid ranges on Oracle
-
-- MAPREDUCE-6010.
- Major bug reported by Jason Lowe and fixed by Jason Lowe (jobhistoryserver)
- HistoryServerFileSystemStateStore fails to update tokens
-
-- MAPREDUCE-5999.
- Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Fix dead link in InputFormat javadoc
-
-- MAPREDUCE-5998.
- Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- CompositeInputFormat javadoc is broken
-
-- MAPREDUCE-5974.
- Major sub-task reported by Todd Lipcon and fixed by Todd Lipcon (task)
- Allow specifying multiple MapOutputCollectors with fallback
-
-- MAPREDUCE-5971.
- Trivial improvement reported by Charles Lamb and fixed by Charles Lamb (distcp)
- Move the default options for distcp -p to DistCpOptionSwitch
-
-- MAPREDUCE-5970.
- Minor improvement reported by Gera Shegalov and fixed by Gera Shegalov (applicationmaster , client)
- Provide a boolean switch to enable MR-AM profiling
-
-- MAPREDUCE-5963.
- Major sub-task reported by Junping Du and fixed by Junping Du
- ShuffleHandler DB schema should be versioned with compatible/incompatible changes
-
-- MAPREDUCE-5960.
- Major bug reported by Gera Shegalov and fixed by Gera Shegalov (client)
- JobSubmitter's check whether job.jar is local is incorrect with no authority in job jar path.
-
-- MAPREDUCE-5958.
- Minor bug reported by Emilio Coppa and fixed by Emilio Coppa
- Wrong reduce task progress if map output is compressed
-
-- MAPREDUCE-5957.
- Major bug reported by Sangjin Lee and fixed by Sangjin Lee
- AM throws ClassNotFoundException with job classloader enabled if custom output format/committer is used
-
-- MAPREDUCE-5956.
- Blocker sub-task reported by Vinod Kumar Vavilapalli and fixed by Wangda Tan (applicationmaster , mrv2)
- MapReduce AM should not use maxAttempts to determine if this is the last retry
-
-- MAPREDUCE-5950.
- Major bug reported by Yongjun Zhang and fixed by Akira AJISAKA (documentation)
- incorrect description in distcp2 document
-
-- MAPREDUCE-5945.
- Minor sub-task reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Update the description of GenericOptionsParser -jt option
-
-- MAPREDUCE-5943.
- Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Separate mapred commands from CommandsManual.apt.vm
-
-- MAPREDUCE-5942.
- Minor sub-task reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Remove MRv1 commands from CommandsManual.apt.vm
-
-- MAPREDUCE-5933.
- Major sub-task reported by Zhijie Shen and fixed by Robert Kanter (mr-am)
- Enable MR AM to post history events to the timeline server
-
-- MAPREDUCE-5931.
- Minor bug reported by Gera Shegalov and fixed by Gera Shegalov (test)
- Validate SleepJob command line parameters
-
-- MAPREDUCE-5910.
- Major task reported by Rohith and fixed by Rohith (applicationmaster)
- MRAppMaster should handle Resync from RM instead of shutting down.
-
-- MAPREDUCE-5906.
- Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA
- Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent"
-
-- MAPREDUCE-5891.
- Major sub-task reported by Jason Lowe and fixed by Junping Du
- Improved shuffle error handling across NM restarts
-
-- MAPREDUCE-5885.
- Major bug reported by Jason Lowe and fixed by Chen He (test)
- build/test/test.mapred.spill causes release audit warnings
-
-- MAPREDUCE-5878.
- Major bug reported by Sangjin Lee and fixed by Sangjin Lee (mrv2)
- some standard JDK APIs are not part of system classes defaults
-
-- MAPREDUCE-5873.
- Major bug reported by Siqi Li and fixed by Siqi Li
- Shuffle bandwidth computation includes time spent waiting for maps
-
-- MAPREDUCE-5866.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev (client , test)
- TestFixedLengthInputFormat fails in windows
-
-- MAPREDUCE-5831.
- Blocker bug reported by Zhijie Shen and fixed by Junping Du (client , mr-am)
- Old MR client is not compatible with new MR application
-
-- MAPREDUCE-5796.
- Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Use current version of the archive name in DistributedCacheDeploy document
-
-- MAPREDUCE-5756.
- Major bug reported by Jason Dere and fixed by Jason Dere
- CombineFileInputFormat.getSplits() including directories in its results
-
-- MAPREDUCE-5597.
- Minor bug reported by Christopher Tubbs and fixed by Akira AJISAKA (client , documentation , job submission)
- Missing alternatives in javadocs for deprecated constructors in mapreduce.Job
-
-- MAPREDUCE-5595.
- Trivial bug reported by Efe Gencer and fixed by Akira AJISAKA
- Typo in MergeManagerImpl.java
-
-- MAPREDUCE-5542.
- Major bug reported by Jason Lowe and fixed by Rohith (client , mrv2)
- Killing a job just as it finishes can generate an NPE in client
-
-- MAPREDUCE-5363.
- Minor bug reported by Sandy Ryza and fixed by Akira AJISAKA (mrv1 , mrv2)
- Fix doc and spelling for TaskCompletionEvent#getTaskStatus and getStatus
-
-- MAPREDUCE-5279.
- Critical bug reported by Peng Zhang and fixed by Peng Zhang (mrv2 , scheduler)
- Jobs can deadlock if headroom is limited by cpu instead of memory
-
-- MAPREDUCE-5130.
- Major improvement reported by Sandy Ryza and fixed by Ray Chiang (documentation)
- Add missing job config options to mapred-default.xml
-
-- MAPREDUCE-4791.
- Minor improvement reported by Matt Lavin and fixed by Akira AJISAKA (documentation)
- Javadoc for KeyValueTextInputFormat should include default separator and how to change it
-
-- MAPREDUCE-883.
- Minor improvement reported by Koji Noguchi and fixed by Akira AJISAKA (documentation , harchive)
- harchive: Document how to unarchive
-
-- HDFS-7391.
- Blocker bug reported by Robert Kanter and fixed by Robert Kanter (webhdfs)
- Renable SSLv2Hello in HttpFS
-
-- HDFS-7387.
- Critical bug reported by Brandon Li and fixed by Brandon Li (nfs)
- NFS may only do partial commit due to a race between COMMIT and write
-
-- HDFS-7385.
- Blocker bug reported by jiangyu and fixed by jiangyu (namenode)
- ThreadLocal used in FSEditLog class causes FSImage permission mess up
-
-- HDFS-7383.
- Major bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (datanode)
- DataNode.requestShortCircuitFdsForRead may throw NullPointerException
-
-- HDFS-7382.
- Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (datanode , security)
- DataNode in secure mode may throw NullPointerException if client connects before DataNode registers itself with NameNode.
-
-- HDFS-7379.
- Minor bug reported by Xiaoyu Yao and fixed by Xiaoyu Yao (test)
- TestBalancer#testBalancerWithRamDisk creates test files incorrectly
-
-- HDFS-7367.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (hdfs-client)
- HDFS short-circuit read cannot negotiate shared memory slot and file descriptors when SASL is enabled on DataTransferProtocol.
-
-- HDFS-7364.
- Minor bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (balancer & mover)
- Balancer always shows zero Bytes Already Moved
-
-- HDFS-7359.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (journal-node , namenode)
- NameNode in secured HA cluster fails to start if dfs.namenode.secondary.http-address cannot be interpreted as a network address.
-
-- HDFS-7355.
- Trivial test reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails on Windows, because we cannot deny access to the file owner.
-
-- HDFS-7340.
- Major bug reported by Arpit Gupta and fixed by Jing Zhao (ha)
- make rollingUpgrade start/finalize idempotent
-
-- HDFS-7334.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (test)
- Fix periodic failures of TestCheckpoint#testTooManyEditReplayFailures
-
-- HDFS-7328.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestTraceAdmin assumes Unix line endings.
-
-- HDFS-7313.
- Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (datanode , hdfs-client , security)
- Support optional configuration of AES cipher suite on DataTransferProtocol.
-
-- HDFS-7309.
- Minor bug reported by Ravi Prakash and fixed by Colin Patrick McCabe
- XMLUtils.mangleXmlString doesn't seem to handle less than sign
-
-- HDFS-7305.
- Minor bug reported by Arpit Gupta and fixed by Jing Zhao (webhdfs)
- NPE seen in wbhdfs FS while running SLive
-
-- HDFS-7300.
- Critical bug reported by Kihwal Lee and fixed by Kihwal Lee
- The getMaxNodesPerRack() method in BlockPlacementPolicyDefault is flawed
-
-- HDFS-7296.
- Minor bug reported by Jing Zhao and fixed by Jing Zhao
- HdfsConstants#MEMORY_STORAGE_POLICY_ID and HdfsConstants#MEMORY_STORAGE_POLICY_ID are missing in branch-2
-
-- HDFS-7291.
- Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode)
- Persist in-memory replicas with appropriate unbuffered copy API on POSIX and Windows
-
-- HDFS-7287.
- Major bug reported by Ravi Prakash and fixed by Ravi Prakash
- The OfflineImageViewer (OIV) can output invalid XML depending on the filename
-
-- HDFS-7276.
- Major improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (hdfs-client)
- Limit the number of byte arrays used by DFSOutputStream
-
-- HDFS-7274.
- Blocker bug reported by Robert Kanter and fixed by Robert Kanter (webhdfs)
- Disable SSLv3 in HttpFS
-
-- HDFS-7260.
- Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (hdfs-client)
- Make DFSOutputStream.MAX_PACKETS configurable
-
-- HDFS-7259.
- Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
- Unresponseive NFS mount point due to deferred COMMIT response
-
-- HDFS-7243.
- Major bug reported by Yi Liu and fixed by Charles Lamb (encryption , namenode)
- HDFS concat operation should not be allowed in Encryption Zone
-
-- HDFS-7237.
- Minor bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
- namenode -rollingUpgrade throws ArrayIndexOutOfBoundsException
-
-- HDFS-7236.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang
- Fix TestOpenFilesWithSnapshot#testOpenFilesWithMultipleSnapshots
-
-- HDFS-7233.
- Major improvement reported by Rushabh S Shah and fixed by Rushabh S Shah (namenode)
- NN logs unnecessary org.apache.hadoop.hdfs.protocol.UnresolvedPathException
-
-- HDFS-7230.
- Major improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (documentation)
- Add rolling downgrade documentation
-
-- HDFS-7228.
- Major improvement reported by Jing Zhao and fixed by Jing Zhao
- Add an SSD policy into the default BlockStoragePolicySuite
-
-- HDFS-7226.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (ha)
- TestDNFencing.testQueueingWithAppend failed often in latest test
-
-- HDFS-7221.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (test)
- TestDNFencingWithReplication fails consistently
-
-- HDFS-7218.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (namenode)
- FSNamesystem ACL operations should write to audit log on failure
-
-- HDFS-7217.
- Major improvement reported by Kihwal Lee and fixed by Kihwal Lee
- Better batching of IBRs
-
-- HDFS-7215.
- Minor improvement reported by Brandon Li and fixed by Brandon Li (nfs)
- Add JvmPauseMonitor to NFS gateway
-
-- HDFS-7208.
- Major bug reported by Ming Ma and fixed by Ming Ma (namenode)
- NN doesn't schedule replication when a DN storage fails
-
-- HDFS-7199.
- Critical bug reported by Jason Lowe and fixed by Rushabh S Shah (hdfs-client)
- DFSOutputStream should not silently drop data if DataStreamer crashes with an unchecked exception
-
-- HDFS-7195.
- Major improvement reported by Yi Liu and fixed by Chris Nauroth (documentation , security)
- Update user doc of secure mode about Datanodes don't require root or jsvc
-
-- HDFS-7185.
- Major bug reported by Colin Patrick McCabe and fixed by Jing Zhao (namenode)
- The active NameNode will not accept an fsimage sent from the standby during rolling upgrade
-
-- HDFS-7181.
- Critical sub-task reported by Andrew Wang and fixed by Andrew Wang (encryption)
- Remove incorrect precondition check on key length in FileEncryptionInfo
-
-- HDFS-7180.
- Critical bug reported by Eric Zhiqiang Ma and fixed by Brandon Li (nfs)
- NFSv3 gateway frequently gets stuck due to GC
-
-- HDFS-7179.
- Critical sub-task reported by Andrew Wang and fixed by Andrew Wang (encryption)
- DFSClient should instantiate a KeyProvider, not a KeyProviderCryptoExtension
-
-- HDFS-7178.
- Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
- Additional unit test for replica write with full disk
-
-- HDFS-7176.
- Minor bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (namenode)
- The namenode usage message doesn't include "-rollingupgrade started"
-
-- HDFS-7172.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- Test data files may be checked out of git with incorrect line endings, causing test failures in TestHDFSCLI.
-
-- HDFS-7171.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Fix Jenkins failures in HDFS-6581 branch
-
-- HDFS-7169.
- Minor bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (build)
- Fix a findbugs warning in ReplaceDatanodeOnFailure
-
-- HDFS-7167.
- Major bug reported by Prabushankar Chinnasamy and fixed by Jing Zhao (balancer & mover)
- NPE while running Mover if the given path is for a file
-
-- HDFS-7162.
- Major bug reported by Chengbing Liu and fixed by Chengbing Liu (fuse-dfs)
- Wrong path when deleting through fuse-dfs a file which already exists in trash
-
-- HDFS-7159.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode)
- Use block storage policy to set lazy persist preference
-
-- HDFS-7158.
- Major improvement reported by Haohui Mai and fixed by Haohui Mai
- Reduce the memory usage of WebImageViewer
-
-- HDFS-7157.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Using Time.now() for recording start/end time of reconfiguration tasks
-
-- HDFS-7156.
- Major bug reported by Konstantin Shvachko and fixed by Masahiro Yamaguchi (documentation)
- Fsck documentation is outdated.
-
-- HDFS-7155.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Bugfix in createLocatedFileStatus caused by bad merge
-
-- HDFS-7154.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Fix returning value of starting reconfiguration task
-
-- HDFS-7153.
- Major improvement reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode)
- Add storagePolicy to NN edit log during file creation
-
-- HDFS-7148.
- Major bug reported by Andrew Wang and fixed by Andrew Wang (encryption)
- TestEncryptionZones#testIsEncryptedMethod fails on branch-2 after archival storage merge
-
-- HDFS-7147.
- Blocker bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (documentation)
- Update archival storage user documentation
-
-- HDFS-7144.
- Minor sub-task reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (datanode)
- Fix findbugs warnings in RamDiskReplicaTracker
-
-- HDFS-7143.
- Major sub-task reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (datanode)
- Fix findbugs warnings in HDFS-6581 branch
-
-- HDFS-7140.
- Minor sub-task reported by Jing Zhao and fixed by Jing Zhao (hdfs-client , namenode)
- Add a tool to list all the existing block storage policies
-
-- HDFS-7139.
- Minor sub-task reported by Zhe Zhang and fixed by Zhe Zhang
- Unit test for creating encryption zone on root path
-
-- HDFS-7138.
- Major sub-task reported by Charles Lamb and fixed by Charles Lamb (namenode)
- Fix hftp to work with encryption
-
-- HDFS-7132.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (namenode)
- hdfs namenode -metadataVersion command does not honor configured name dirs
-
-- HDFS-7131.
- Major bug reported by Jing Zhao and fixed by Jing Zhao (qjm)
- During HA upgrade, JournalNode should create a new committedTxnId file in the current directory
-
-- HDFS-7130.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestDataTransferKeepalive fails intermittently on Windows.
-
-- HDFS-7129.
- Major sub-task reported by Arpit Agarwal and fixed by Xiaoyu Yao (datanode)
- Metrics to track usage of memory for writes
-
-- HDFS-7128.
- Major improvement reported by Ming Ma and fixed by Ming Ma (namenode)
- Decommission slows way down when it gets towards the end
-
-- HDFS-7127.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestLeaseRecovery leaks MiniDFSCluster instances.
-
-- HDFS-7126.
- Minor test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (security , test)
- TestEncryptionZonesWithHA assumes Unix path separator for KMS key store path
-
-- HDFS-7124.
- Minor sub-task reported by Charles Lamb and fixed by Charles Lamb (namenode)
- Remove EncryptionZoneManager.NULL_EZ
-
-- HDFS-7122.
- Blocker bug reported by Jeff Buell and fixed by Andrew Wang (namenode)
- Use of ThreadLocal<Random> results in poor block placement
-
-- HDFS-7119.
- Minor sub-task reported by Chris Nauroth and fixed by Chris Nauroth (journal-node)
- Split error checks in AtomicFileOutputStream#close into separate conditions to improve diagnostics.
-
-- HDFS-7118.
- Major sub-task reported by Chris Nauroth and fixed by Chris Nauroth (journal-node , namenode)
- Improve diagnostics on storage directory rename operations by using NativeIO#renameTo in Storage#rename.
-
-- HDFS-7115.
- Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (encryption)
- TestEncryptionZones assumes Unix path separator for KMS key store path
-
-- HDFS-7112.
- Major sub-task reported by Arpit Agarwal and fixed by Xiaoyu Yao (datanode)
- LazyWriter should use either async IO or one thread per physical disk
-
-- HDFS-7111.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestSafeMode assumes Unix line endings in safe mode tip.
-
-- HDFS-7110.
- Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- Skip tests related to short-circuit read on platforms that do not currently implement short-circuit read.
-
-- HDFS-7109.
- Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- TestDataStorage does not release file locks between tests.
-
-- HDFS-7108.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
- Fix unit test failures in SimulatedFsDataset
-
-- HDFS-7107.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (ha)
- Avoid Findbugs warning for synchronization on AbstractNNFailoverProxyProvider#fallbackToSimpleAuth.
-
-- HDFS-7106.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (datanode)
- Reconfiguring DataNode volumes does not release the lock files in removed volumes.
-
-- HDFS-7105.
- Minor bug reported by Ray Chiang and fixed by Ray Chiang (test)
- Fix TestJournalNode#testFailToStartWithBadConfig to match log output change
-
-- HDFS-7104.
- Minor bug reported by Zhe Zhang and fixed by Zhe Zhang
- Fix and clarify INodeInPath getter functions
-
-- HDFS-7100.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Make eviction scheme pluggable
-
-- HDFS-7096.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (test)
- Fix TestRpcProgramNfs3 to use DFS_ENCRYPTION_KEY_PROVIDER_URI
-
-- HDFS-7095.
- Minor sub-task reported by Tsz Wo Nicholas Sze and fixed by Jing Zhao (test)
- TestStorageMover often fails in Jenkins
-
-- HDFS-7093.
- Major improvement reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode)
- Add config key to restrict setStoragePolicy
-
-- HDFS-7091.
- Minor sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode , test)
- Add forwarding constructor for INodeFile for existing callers
- Thanks Nicholas! Revised title and committed to the feature branch.
-- HDFS-7090.
- Major sub-task reported by Arpit Agarwal and fixed by Xiaoyu Yao (datanode)
- Use unbuffered writes when persisting in-memory replicas
-
-- HDFS-7084.
- Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode)
- FsDatasetImpl#copyBlockFiles debug log can be improved
-
-- HDFS-7081.
- Major sub-task reported by Jing Zhao and fixed by Jing Zhao (balancer & mover , namenode)
- Add new DistributedFileSystem API for getting all the existing storage policies
-
-- HDFS-7080.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
- Fix finalize and upgrade unit test failures
-
-- HDFS-7079.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
- Few more unit test fixes for HDFS-6581
-
-- HDFS-7078.
- Major sub-task reported by Andrew Wang and fixed by Andrew Wang (encryption)
- Fix listEZs to work correctly with snapshots
-
-- HDFS-7077.
- Major sub-task reported by Andrew Wang and fixed by Andrew Wang (encryption)
- Separate CipherSuite from crypto protocol version
-
-- HDFS-7075.
- Major bug reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe
- hadoop-fuse-dfs fails because it cannot find JavaKeyStoreProvider$Factory
-
-- HDFS-7073.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (datanode , hdfs-client , security)
- Allow falling back to a non-SASL connection on DataTransferProtocol in several edge cases.
-
-- HDFS-7066.
- Minor sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode)
- LazyWriter#evictBlocks misses a null check for replicaState
-
-- HDFS-7065.
- Critical bug reported by Kihwal Lee and fixed by Kihwal Lee (datanode)
- Pipeline close recovery race can cause block corruption
-
-- HDFS-7064.
- Major sub-task reported by Arpit Agarwal and fixed by Xiaoyu Yao (test)
- Fix unit test failures in HDFS-6581 branch
-
-- HDFS-7061.
- Minor sub-task reported by Stephen Chu and fixed by Stephen Chu (encryption , test)
- Add test to verify encryption zone creation after NameNode restart without saving namespace
-
-- HDFS-7059.
- Minor improvement reported by Rushabh S Shah and fixed by Rushabh S Shah
- HAadmin transtionToActive with forceActive option can show confusing message.
-
-- HDFS-7051.
- Minor test reported by Chris Nauroth and fixed by Chris Nauroth (datanode , test)
- TestDataNodeRollingUpgrade#isBlockFileInPrevious assumes Unix file path separator.
-
-- HDFS-7049.
- Minor bug reported by Juan Yu and fixed by Juan Yu (test)
- TestByteRangeInputStream.testPropagatedClose fails and throw NPE on branch-2
-
-- HDFS-7047.
- Major sub-task reported by Andrew Wang and fixed by Colin Patrick McCabe (encryption)
- Expose FileStatus#isEncrypted in libhdfs
-
-- HDFS-7046.
- Critical bug reported by Daryn Sharp and fixed by Kihwal Lee (namenode)
- HA NN can NPE upon transition to active
- Thanks for the reviews, gentlemen. It's been committed to trunk and branch-2.
-- HDFS-7045.
- Critical bug reported by Yi Liu and fixed by Yi Liu (namenode)
- Fix NameNode deadlock when opening file under /.reserved path
-
-- HDFS-7042.
- Blocker bug reported by Chris Nauroth and fixed by Chris Nauroth (journal-node)
- Upgrade fails for Windows HA cluster due to file locks held during rename in JournalNode.
-
-- HDFS-7032.
- Major sub-task reported by Stephen Chu and fixed by Charles Lamb (encryption , webhdfs)
- Add WebHDFS support for reading and writing to encryption zones
-
-- HDFS-7025.
- Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (encryption)
- HDFS Credential Provider related Unit Test Failure
-
-- HDFS-7006.
- Major test reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security , test)
- Test encryption zones with KMS
-
-- HDFS-7005.
- Critical bug reported by Daryn Sharp and fixed by Daryn Sharp (hdfs-client)
- DFS input streams do not timeout
-
-- HDFS-7004.
- Major sub-task reported by Andrew Wang and fixed by Andrew Wang (encryption)
- Update KeyProvider instantiation to create by URI
-
-- HDFS-7003.
- Major sub-task reported by Stephen Chu and fixed by Charles Lamb (encryption , nfs)
- Add NFS Gateway support for reading and writing to encryption zones
-
-- HDFS-7001.
- Minor bug reported by Masatake Iwasaki and fixed by Masatake Iwasaki
- Tests in TestTracing should not depend on the order of execution
-
-- HDFS-6996.
- Major bug reported by Jing Zhao and fixed by Jing Zhao (snapshots)
- SnapshotDiff report can hit IndexOutOfBoundsException when there are nested renamed directory/file
-
-- HDFS-6995.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
- Block should be placed in the client's 'rack-local' node if 'client-local' node is not available
-
-- HDFS-6991.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode , namenode)
- Notify NN of evicted block before deleting it from RAM disk
-
-- HDFS-6990.
- Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao (datanode)
- Add unit test for evict/delete RAM_DISK block with open handle
-
-- HDFS-6988.
- Major sub-task reported by Arpit Agarwal and fixed by Xiaoyu Yao (datanode)
- Improve HDFS-6581 eviction configuration
-
-- HDFS-6987.
- Major sub-task reported by Andrew Wang and fixed by Zhe Zhang (encryption)
- Move CipherSuite xattr information up to the encryption zone root
-
-- HDFS-6986.
- Major sub-task reported by Alejandro Abdelnur and fixed by Zhe Zhang (security)
- DistributedFileSystem must get delegation tokens from configured KeyProvider
-
-- HDFS-6981.
- Major bug reported by James Thomas and fixed by Arpit Agarwal (datanode)
- Fix DN upgrade with layout version change
-
-- HDFS-6979.
- Minor bug reported by Remus Rusanu and fixed by Chris Nauroth (hdfs-client)
- hdfs.dll does not produce .pdb files
-
-- HDFS-6978.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Directory scanner should correctly reconcile blocks on RAM disk
-
-- HDFS-6977.
- Major sub-task reported by Nathan Yao and fixed by Arpit Agarwal (datanode)
- Delete all copies when a block is deleted from the block space
-
-- HDFS-6972.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang
- TestRefreshUserMappings.testRefreshSuperUserGroupsConfiguration doesn't decode url correctly
-
-- HDFS-6970.
- Major sub-task reported by Andrew Wang and fixed by Andrew Wang (encryption)
- Move startFile EDEK retries to the DFSClient
-
-- HDFS-6966.
- Major sub-task reported by Stephen Chu and fixed by Stephen Chu (encryption)
- Add additional unit tests for encryption zones
-
-- HDFS-6965.
- Major bug reported by Daryn Sharp and fixed by Rushabh S Shah (namenode)
- NN continues to issue block locations for DNs with full disks
-
-- HDFS-6960.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode , test)
- Bugfix in LazyWriter, fix test case and some refactoring
-
-- HDFS-6959.
- Minor new feature reported by Kevin Odell and fixed by Yongjun Zhang
- Make the HDFS home directory location customizable.
-
-- HDFS-6956.
- Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode , namenode)
- Allow dynamically changing the tracing level in Hadoop servers
-
-- HDFS-6954.
- Major bug reported by Allen Wittenauer and fixed by Charles Lamb (encryption)
- With crypto, no native lib systems are too verbose
-
-- HDFS-6951.
- Major sub-task reported by Stephen Chu and fixed by Charles Lamb (encryption)
- Correctly persist raw namespace xattrs to edit log and fsimage
-
-- HDFS-6950.
- Major sub-task reported by Xiaoyu Yao and fixed by Xiaoyu Yao
- Add Additional unit tests for HDFS-6581
-
-- HDFS-6948.
- Major bug reported by Daryn Sharp and fixed by Eric Payne
- DN rejects blocks if it has older UC block
-
-- HDFS-6943.
- Minor improvement reported by Ming Ma and fixed by Ming Ma (namenode)
- Improve NN allocateBlock log to include replicas' datanode IPs
-
-- HDFS-6942.
- Trivial bug reported by Ray Chiang and fixed by Ray Chiang
- Fix typos in log messages
-
-- HDFS-6934.
- Major sub-task reported by Arpit Agarwal and fixed by Chris Nauroth (datanode , hdfs-client)
- Move checksum computation off the hot path when writing to RAM disk
-
-- HDFS-6932.
- Major sub-task reported by Arpit Agarwal and fixed by Xiaoyu Yao (datanode)
- Balancer and Mover tools should ignore replicas on RAM_DISK
-
-- HDFS-6931.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Move lazily persisted replicas to finalized directory on DN startup
-
-- HDFS-6930.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Improve replica eviction from RAM disk
-
-- HDFS-6929.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- NN periodically unlinks lazy persist files with missing replicas from namespace
-
-- HDFS-6928.
- Major sub-task reported by Tassapol Athiapinya and fixed by Arpit Agarwal (datanode)
- 'hdfs put' command should accept lazyPersist flag for testing
-
-- HDFS-6927.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Add unit tests
-
-- HDFS-6926.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- DN support for saving replicas to persistent storage and evicting in-memory replicas
-
-- HDFS-6925.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- DataNode should attempt to place replicas on transient storage first if lazyPersist flag is received
-
-- HDFS-6924.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Add new RAM_DISK storage type
-
-- HDFS-6923.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode)
- Propagate LazyPersist flag to DNs via DataTransferProtocol
-
-- HDFS-6921.
- Major sub-task reported by Arpit Agarwal and fixed by Arpit Agarwal
- Add LazyPersist flag to FileStatus
-
-- HDFS-6912.
- Minor bug reported by Gopal V and fixed by Colin Patrick McCabe (caching)
- SharedFileDescriptorFactory should not allocate sparse files
-
-- HDFS-6908.
- Critical bug reported by Juan Yu and fixed by Juan Yu (snapshots)
- incorrect snapshot directory diff generated by snapshot deletion
-
-- HDFS-6905.
- Blocker bug reported by Allen Wittenauer and fixed by Charles Lamb
- fs-encryption merge triggered release audit failures
-
-- HDFS-6904.
- Critical bug reported by Varun Vasudev and fixed by Jitendra Nath Pandey (webhdfs)
- YARN unable to renew delegation token fetched via webhdfs due to incorrect service port
-
-- HDFS-6902.
- Minor bug reported by Ted Yu and fixed by Tsuyoshi OZAWA
- FileWriter should be closed in finally block in BlockReceiver#receiveBlock()
-
-- HDFS-6899.
- Major improvement reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode , test)
- Allow changing MiniDFSCluster volumes per DN and capacity per volume
-
-- HDFS-6894.
- Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)
- Add XDR parser method for each NFS response
-
-- HDFS-6892.
- Major sub-task reported by Brandon Li and fixed by Brandon Li (nfs)
- Add XDR packaging method for each NFS request
-
-- HDFS-6890.
- Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
- NFS readdirplus doesn't return dotdot attributes
-
-- HDFS-6886.
- Critical improvement reported by Yi Liu and fixed by Yi Liu (namenode)
- Use single editlog record for creating file + overwrite.
-
-- HDFS-6880.
- Major sub-task reported by Masatake Iwasaki and fixed by Masatake Iwasaki
- Adding tracing to DataNode data transfer protocol
-
-- HDFS-6879.
- Major sub-task reported by Masatake Iwasaki and fixed by Masatake Iwasaki
- Adding tracing to Hadoop RPC
-
-- HDFS-6878.
- Minor test reported by Tsz Wo Nicholas Sze and fixed by Arpit Agarwal (test)
- Change MiniDFSCluster to support StorageType configuration for individual directories
-
-- HDFS-6870.
- Major bug reported by Yi Liu and fixed by Yi Liu (namenode)
- Blocks and INodes could leak for Rename with overwrite flag
-
-- HDFS-6868.
- Major bug reported by Allen Wittenauer and fixed by Brandon Li (documentation , nfs)
- portmap and nfs3 are documented as hadoop commands instead of hdfs
-
-- HDFS-6865.
- Major sub-task reported by James Thomas and fixed by James Thomas (hdfs-client , performance)
- Byte array native checksumming on client side (HDFS changes)
-
-- HDFS-6862.
- Major bug reported by Arpit Agarwal and fixed by Xiaoyu Yao (test)
- Add missing timeout annotations to tests
-
-- HDFS-6858.
- Minor improvement reported by Benoy Antony and fixed by Benoy Antony (security)
- Allow dfs.data.transfer.saslproperties.resolver.class default to hadoop.security.saslproperties.resolver.class
-
-- HDFS-6851.
- Major sub-task reported by Charles Lamb and fixed by Charles Lamb (namenode , security)
- Refactor EncryptionZoneWithId and EncryptionZone
-
-- HDFS-6850.
- Minor improvement reported by Zhe Zhang and fixed by Zhe Zhang (nfs)
- Move NFS out of order write unit tests into TestWrites class
-
-- HDFS-6849.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Replace HttpFS custom proxyuser handling with common implementation
-
-- HDFS-6848.
- Minor bug reported by Ted Yu and fixed by Xiaoyu Yao
- Lack of synchronization on access to datanodeUuid in DataStorage#format()
-
-- HDFS-6843.
- Major sub-task reported by Charles Lamb and fixed by Charles Lamb (namenode , security)
- Create FileStatus isEncrypted() method
-
-- HDFS-6840.
- Critical bug reported by Jason Lowe and fixed by Andrew Wang
- Clients are always sent to the same datanode when read is off rack
-
-- HDFS-6838.
- Minor bug reported by Jing Zhao and fixed by Jing Zhao (namenode)
- Code cleanup for unnecessary INode replacement
-
-- HDFS-6837.
- Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (balancer & mover)
- Code cleanup for Balancer and Dispatcher
-
-- HDFS-6836.
- Major improvement reported by Gopal V and fixed by Nathan Yao (datanode)
- HDFS INFO logging is verbose & uses file appenders
-
-- HDFS-6831.
- Minor bug reported by Akira AJISAKA and fixed by Xiaoyu Yao
- Inconsistency between 'hdfs dfsadmin' and 'hdfs dfsadmin -help'
-
-- HDFS-6830.
- Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode)
- BlockInfo.addStorage fails when DN changes the storage for a block replica
-
-- HDFS-6829.
- Minor bug reported by zhaoyunjiong and fixed by zhaoyunjiong (tools)
- DFSAdmin refreshSuperUserGroupsConfiguration failed in security cluster
-
-- HDFS-6828.
- Major improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (balancer & mover)
- Separate block replica dispatching from Balancer
-
-- HDFS-6825.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (namenode)
- Edit log corruption due to delayed block removal
-
-- HDFS-6823.
- Minor bug reported by Allen Wittenauer and fixed by Allen Wittenauer (namenode)
- dfs.web.authentication.kerberos.principal shows up in logs for insecure HDFS
-
-- HDFS-6812.
- Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
- Remove addBlock and replaceBlock from DatanodeDescriptor
-
-- HDFS-6810.
- Minor bug reported by Ted Yu and fixed by Tsz Wo Nicholas Sze (namenode)
- StorageReport array is initialized with wrong size in DatanodeDescriptor#getStorageReports
-
-- HDFS-6809.
- Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (balancer & mover)
- Move some Balancer's inner classes to standalone classes
-
-- HDFS-6808.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Add command line option to ask DataNode reload configuration.
-
-- HDFS-6802.
- Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (test)
- Some tests in TestDFSClientFailover are missing @Test annotation
-
-- HDFS-6800.
- Major improvement reported by Colin Patrick McCabe and fixed by James Thomas (datanode)
- Support Datanode layout changes with rolling upgrade
-
-- HDFS-6799.
- Minor bug reported by Megasthenis Asteris and fixed by Megasthenis Asteris (datanode , test)
- The invalidate method in SimulatedFSDataset.java failed to remove (invalidate) blocks from the file system.
-
-- HDFS-6798.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony (balancer & mover)
- Add test case for incorrect data node condition during balancing
-
-- HDFS-6797.
- Major bug reported by Benoy Antony and fixed by Benoy Antony (datanode)
- DataNode logs wrong layoutversion during upgrade
-
-- HDFS-6796.
- Minor improvement reported by Benoy Antony and fixed by Benoy Antony (balancer & mover)
- Improving the argument check during balancer command line parsing
-
-- HDFS-6794.
- Minor improvement reported by Arpit Agarwal and fixed by Arpit Agarwal (namenode)
- Update BlockManager methods to use DatanodeStorageInfo where possible
-
-- HDFS-6791.
- Major bug reported by Ming Ma and fixed by Ming Ma
- A block could remain under replicated if all of its replicas are on decommissioned nodes
-
-- HDFS-6790.
- Major bug reported by Larry McCay and fixed by Larry McCay
- DFSUtil Should Use configuration.getPassword for SSL passwords
-
-- HDFS-6789.
- Major bug reported by Rushabh S Shah and fixed by Akira AJISAKA (test)
- TestDFSClientFailover.testFileContextDoesntDnsResolveLogicalURI and TestDFSClientFailover.testDoesntDnsResolveLogicalURI failing on jdk7
-
-- HDFS-6788.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (datanode)
- Improve synchronization in BPOfferService with read write lock
-
-- HDFS-6787.
- Major bug reported by Yi Liu and fixed by Yi Liu (namenode)
- Remove duplicate code in FSDirectory#unprotectedConcat
-
-- HDFS-6783.
- Major bug reported by Yi Liu and fixed by Yi Liu (caching)
- Fix HDFS CacheReplicationMonitor rescan logic
-
-- HDFS-6781.
- Major improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Separate HDFS commands from CommandsManual.apt.vm
-
-- HDFS-6779.
- Minor improvement reported by Allen Wittenauer and fixed by Sasaki Toru (scripts)
- Add missing version subcommand for hdfs
-
-- HDFS-6778.
- Major bug reported by Charles Lamb and fixed by Charles Lamb
- The extended attributes javadoc should simply refer to the user docs
-
-- HDFS-6777.
- Major sub-task reported by James Thomas and fixed by James Thomas (qjm)
- Supporting consistent edit log reads when in-progress edit log segments are included
-
-- HDFS-6776.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang
- Using distcp to copy data between insecure and secure cluster via webdhfs doesn't work
-
-- HDFS-6774.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Make FsDataset and DataStore support removing volumes.
-
-- HDFS-6773.
- Major improvement reported by Daryn Sharp and fixed by Stephen Chu (namenode)
- MiniDFSCluster should skip edit log fsync by default
-
-- HDFS-6772.
- Major improvement reported by Ming Ma and fixed by Ming Ma
- Get DN storages out of blockContentsStale state faster after NN restarts
-
-- HDFS-6768.
- Major bug reported by Arpit Agarwal and fixed by Arpit Agarwal (test)
- Fix a few unit tests that use hard-coded port numbers
-
-- HDFS-6758.
- Major improvement reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode , hdfs-client)
- block writer should pass the expected block size to DataXceiverServer
-
-- HDFS-6755.
- Major improvement reported by Mit Desai and fixed by Mit Desai
- There is an unnecessary sleep in the code path where DFSOutputStream#close gives up its attempt to contact the namenode
-
-- HDFS-6754.
- Major bug reported by Mit Desai and fixed by Mit Desai
- TestNamenodeCapacityReport.testXceiverCount may sometimes fail due to lack of retry
-
-- HDFS-6750.
- Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (datanode , hdfs-client)
- The DataNode should use its shared memory segment to mark short-circuit replicas that have been unlinked as stale
-
-- HDFS-6749.
- Major bug reported by Charles Lamb and fixed by Charles Lamb (namenode)
- FSNamesystem methods should call resolvePath
-
-- HDFS-6740.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Make FSDataset support adding data volumes dynamically
-
-- HDFS-6739.
- Major improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (hdfs-client , namenode)
- Add getDatanodeStorageReport to ClientProtocol
-
-- HDFS-6731.
- Major bug reported by WenJin Ma and fixed by Masatake Iwasaki (auto-failover , ha)
- Run "hdfs zkfc-formatZK" on a server in a non-namenode will cause a null pointer exception.
-
-- HDFS-6728.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Dynamically add new volumes to DataStorage, formatted if necessary.
-
-- HDFS-6727.
- Major sub-task reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (datanode)
- Refresh data volumes on DataNode based on configuration changes
-
-- HDFS-6722.
- Major sub-task reported by Ming Ma and fixed by Ming Ma
- Display readable last contact time for dead nodes on NN webUI
-
-- HDFS-6717.
- Minor sub-task reported by Jeff Hansen and fixed by Brandon Li (nfs)
- Jira HDFS-5804 breaks default nfs-gateway behavior for unsecured config
-
-- HDFS-6715.
- Major bug reported by Arpit Gupta and fixed by Jing Zhao (ha , webhdfs)
- webhdfs wont fail over when it gets java.io.IOException: Namenode is in startup mode
-
-- HDFS-6714.
- Minor bug reported by Vinayakumar B and fixed by Vinayakumar B (test)
- TestBlocksScheduledCounter#testBlocksScheduledCounter should shutdown cluster
-
-- HDFS-6705.
- Major sub-task reported by Charles Lamb and fixed by Charles Lamb (namenode , security)
- Create an XAttr that disallows the HDFS admin from accessing a file
-
-- HDFS-6704.
- Minor bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Fix the command to launch JournalNode in HDFS-HA document
-
-- HDFS-6702.
- Major bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (datanode , hdfs-client)
- DFSClient should create blocks using StorageType
-
-- HDFS-6701.
- Major improvement reported by Ashwin Shankar and fixed by Ashwin Shankar (namenode)
- Make seed optional in NetworkTopology#sortByDistance
-
-- HDFS-6700.
- Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
- BlockPlacementPolicy shoud choose storage but not datanode for deletion
-
-- HDFS-6693.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B (test , tools)
- TestDFSAdminWithHA fails on windows
-
-- HDFS-6690.
- Major improvement reported by Andrew Wang and fixed by Andrew Wang (namenode)
- Deduplicate xattr names in memory
-
-- HDFS-6689.
- Major bug reported by Yesha Vora and fixed by Brandon Li (nfs)
- NFS doesn't return correct lookup access for directories
-
-- HDFS-6685.
- Major improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (balancer & mover)
- Balancer should preserve storage type of replicas
-
-- HDFS-6678.
- Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- MiniDFSCluster may still be partially running after initialization fails.
-
-- HDFS-6667.
- Major bug reported by Jian He and fixed by Jing Zhao (security)
- In HDFS HA mode, Distcp/SLive with webhdfs on secure cluster fails with Client cannot authenticate via:[TOKEN, KERBEROS] error
-
-- HDFS-6665.
- Major test reported by Stephen Chu and fixed by Stephen Chu (hdfs-client)
- Add tests for XAttrs in combination with viewfs
-
-- HDFS-6664.
- Trivial bug reported by Chris Nauroth and fixed by Ray Chiang (documentation)
- HDFS permissions guide documentation states incorrect default group mapping class.
-
-- HDFS-6655.
- Major improvement reported by Vinayakumar B and fixed by Vinayakumar B
- Add 'header banner' to 'explorer.html' also in Namenode UI
-
-- HDFS-6646.
- Major bug reported by Brahma Reddy Battula and fixed by Brahma Reddy Battula (tools)
- [ HDFS Rolling Upgrade - Shell ] shutdownDatanode and getDatanodeInfo usage is missed
-
-- HDFS-6645.
- Minor test reported by Stephen Chu and fixed by Stephen Chu (snapshots , test)
- Add test for successive Snapshots between XAttr modifications
-
-- HDFS-6643.
- Minor improvement reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
- Refactor INodeFile.HeaderFormat and INodeWithAdditionalFields.PermissionStatusFormat
-
-- HDFS-6640.
- Major bug reported by Brahma Reddy Battula and fixed by Stephen Chu (documentation , webhdfs)
- [ Web HDFS ] Syntax for MKDIRS, CREATESYMLINK, and SETXATTR are given wrongly(missed webhdfs/v1).).
-
-- HDFS-6638.
- Major test reported by Liang Xie and fixed by Liang Xie (test)
- shorten test run time with a smaller retry timeout setting
-
-- HDFS-6634.
- Major new feature reported by James Thomas and fixed by James Thomas (hdfs-client , namenode , qjm)
- inotify in HDFS
-
-- HDFS-6630.
- Major bug reported by J.Andreina and fixed by Haohui Mai (namenode)
- Unable to fetch the block information by Browsing the file system on Namenode UI through IE9
-
-- HDFS-6627.
- Major improvement reported by Liang Xie and fixed by Liang Xie (datanode)
- Rename DataNode#checkWriteAccess to checkReadAccess.
-
-- HDFS-6621.
- Major bug reported by Benjamin Bowman and fixed by Rafal Wojdyla (balancer & mover)
- Hadoop Balancer prematurely exits iterations
-
-- HDFS-6617.
- Minor test reported by Liang Xie and fixed by Liang Xie (auto-failover , test)
- Flake TestDFSZKFailoverController.testManualFailoverWithDFSHAAdmin due to a long edit log sync op
-
-- HDFS-6616.
- Minor bug reported by zhaoyunjiong and fixed by zhaoyunjiong (webhdfs)
- bestNode shouldn't always return the first DataNode
-
-- HDFS-6613.
- Minor improvement reported by Andrew Wang and fixed by Andrew Wang (caching)
- Improve logging in caching classes
-
-- HDFS-6609.
- Major sub-task reported by Jing Zhao and fixed by Jing Zhao (namenode)
- Use DirectorySnapshottableFeature to represent a snapshottable directory
-
-- HDFS-6606.
- Major improvement reported by Yi Liu and fixed by Yi Liu (datanode , hdfs-client , security)
- Optimize HDFS Encrypted Transport performance
- HDFS now supports the option to configure AES encryption for block data transfer. AES offers improved cryptographic strength and performance over the prior options of 3DES and RC4.
-- HDFS-6597.
- Major improvement reported by Danilo Vunjak and fixed by Danilo Vunjak (namenode)
- Add a new option to NN upgrade to terminate the process after upgrade on NN is completed
-
-- HDFS-6584.
- Major new feature reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (balancer & mover , namenode)
- Support Archival Storage
-
-- HDFS-6582.
- Minor bug reported by Ted Yu and fixed by Abhiraj Butala (nfs)
- Missing null check in RpcProgramNfs3#read(XDR, SecurityHandler)
-
-- HDFS-6581.
- Major new feature reported by Arpit Agarwal and fixed by Arpit Agarwal (datanode , hdfs-client , namenode)
- Write to single replica in memory
-
-- HDFS-6570.
- Major improvement reported by Thejas M Nair and fixed by Jitendra Nath Pandey (hdfs-client , namenode , webhdfs)
- add api that enables checking if a user has certain permissions on a file
-
-- HDFS-6569.
- Major bug reported by Brandon Li and fixed by Brandon Li (datanode)
- OOB message can't be sent to the client when DataNode shuts down for upgrade
-
-- HDFS-6567.
- Major bug reported by Haohui Mai and fixed by Tassapol Athiapinya
- Normalize the order of public final in HdfsFileStatus
-
-- HDFS-6544.
- Minor bug reported by Suraj Nayak M and fixed by Suraj Nayak M
- Broken Link for GFS in package.html
-
-- HDFS-6534.
- Minor bug reported by Binglin Chang and fixed by Binglin Chang
- Fix build on macosx: HDFS parts
-
-- HDFS-6519.
- Major improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Document oiv_legacy command
-
-- HDFS-6517.
- Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA
- Remove hadoop-metrics2.properties from hdfs project
-
-- HDFS-6511.
- Minor improvement reported by Juan Yu and fixed by Juan Yu
- BlockManager#computeInvalidateWork() could do nothing
-
-- HDFS-6506.
- Major bug reported by Binglin Chang and fixed by Binglin Chang (balancer & mover , test)
- Newly moved block replica been invalidated and deleted in TestBalancer
-
-- HDFS-6482.
- Major improvement reported by James Thomas and fixed by James Thomas (datanode)
- Use block ID-based block layout on datanodes
- The directory structure for finalized replicas on DNs has been changed. Now, the directory that a finalized replica goes in is determined uniquely by its ID. Specifically, we use a two-level directory structure, with the 24th through 17th bits identifying the correct directory at the first level and the 16th through 8th bits identifying the correct directory at the second level.
-- HDFS-6478.
- Major bug reported by Ming Ma and fixed by Ming Ma
- RemoteException can't be retried properly for non-HA scenario
-
-- HDFS-6456.
- Major bug reported by Yesha Vora and fixed by Abhiraj Butala (nfs)
- NFS should throw error for invalid entry in dfs.nfs.exports.allowed.hosts
-
-- HDFS-6455.
- Major bug reported by Yesha Vora and fixed by Abhiraj Butala (nfs)
- NFS: Exception should be added in NFS log for invalid separator in nfs.exports.allowed.hosts
-
-- HDFS-6451.
- Major bug reported by Brandon Li and fixed by Abhiraj Butala (nfs)
- NFS should not return NFS3ERR_IO for AccessControlException
-
-- HDFS-6441.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony (balancer & mover)
- Add ability to exclude/include specific datanodes while balancing
-
-- HDFS-6385.
- Major sub-task reported by Jing Zhao and fixed by Chris Nauroth
- Show when block deletion will start after NameNode startup in WebUI
-
-- HDFS-6376.
- Major bug reported by Dave Marion and fixed by Dave Marion (datanode , federation , hdfs-client)
- Distcp data between two HA clusters requires another configuration
- Allow distcp to copy data between HA clusters. Users can use a new configuration property "dfs.internal.nameservices" to explicitly specify the name services belonging to the local cluster, while continue using the configuration property "dfs.nameservices" to specify all the name services in the local and remote clusters.
-- HDFS-6247.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B (balancer & mover , datanode)
- Avoid timeouts for replaceBlock() call by sending intermediate responses to Balancer
-
-- HDFS-6188.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony (security)
- An ip whitelist based implementation of TrustedChannelResolver
-
-- HDFS-6134.
- Major new feature reported by Alejandro Abdelnur and fixed by Charles Lamb (security)
- Transparent data at rest encryption
-
-- HDFS-6114.
- Critical bug reported by Vinayakumar B and fixed by Vinayakumar B (datanode)
- Block Scan log rolling will never happen if blocks written continuously leading to huge size of dncp_block_verification.log.curr
-
-- HDFS-6036.
- Major sub-task reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (caching , datanode)
- Forcibly timeout misbehaving DFSClients that try to do no-checksum reads that extend too long
-
-- HDFS-5919.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
- FileJournalManager doesn't purge empty and corrupt inprogress edits files
-
-- HDFS-5809.
- Critical bug reported by ikweesung and fixed by Colin Patrick McCabe (datanode)
- BlockPoolSliceScanner and high speed hdfs appending make datanode to drop into infinite loop
-
-- HDFS-5723.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B (namenode)
- Append failed FINALIZED replica should not be accepted as valid when that block is underconstruction
-
-- HDFS-5624.
- Major test reported by Chris Nauroth and fixed by Stephen Chu (hdfs-client , test)
- Add HDFS tests for ACLs in combination with viewfs.
-
-- HDFS-5202.
- Major improvement reported by Colin Patrick McCabe and fixed by Chris Nauroth (datanode)
- Support Centralized Cache Management on Windows.
-
-- HDFS-5185.
- Critical bug reported by Vinayakumar B and fixed by Vinayakumar B (datanode)
- DN fails to startup if one of the data dir is full
-
-- HDFS-5182.
- Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (hdfs-client)
- BlockReaderLocal must allow zero-copy reads only when the DN believes it's valid
-
-- HDFS-5089.
- Major bug reported by Tsz Wo Nicholas Sze and fixed by Tsz Wo Nicholas Sze (namenode)
- When a LayoutVersion support SNAPSHOT, it must support FSIMAGE_NAME_OPTIMIZATION.
-
-- HDFS-4852.
- Minor bug reported by Andrew Wang and fixed by Chris Nauroth
- libhdfs documentation is out of date
-
-- HDFS-4629.
- Major bug reported by Amir Sanjar and fixed by (tools)
- Using com.sun.org.apache.xml.internal.serialize.* in XmlEditsVisitor.java is JVM vendor specific. Breaks IBM JAVA
-
-- HDFS-4486.
- Minor improvement reported by Todd Lipcon and fixed by Zhe Zhang
- Add log category for long-running DFSClient notices
-
-- HDFS-4257.
- Minor new feature reported by Harsh J and fixed by Tsz Wo Nicholas Sze (hdfs-client)
- The ReplaceDatanodeOnFailure policies could have a forgiving option
-
-- HDFS-4227.
- Major bug reported by Eli Collins and fixed by Daisuke Kobayashi (documentation)
- Document dfs.namenode.resource.*
-
-- HDFS-4165.
- Trivial bug reported by Binglin Chang and fixed by Binglin Chang (namenode)
- Faulty sanity check in FsDirectory.unprotectedSetQuota
-
-- HDFS-4120.
- Minor improvement reported by Liang Xie and fixed by Rakesh R (ha , namenode)
- Add a new "-skipSharedEditsCheck" option for BootstrapStandby
-
-- HDFS-3851.
- Trivial improvement reported by Jing Zhao and fixed by Jing Zhao (hdfs-client)
- Make DFSOuputSteram$Packet default constructor reuse the other constructor
-
-- HDFS-3528.
- Major improvement reported by Todd Lipcon and fixed by James Thomas (datanode , hdfs-client , performance)
- Use native CRC32 in DFS write path
-
-- HDFS-3482.
- Minor bug reported by Stephen Chu and fixed by madhukara phatak (balancer & mover)
- hdfs balancer throws ArrayIndexOutOfBoundsException if option is specified without arguments
-
-- HDFS-2976.
- Trivial bug reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (hdfs-client)
- Remove unnecessary method (tokenRefetchNeeded) in DFSClient
-
-- HDFS-2975.
- Major bug reported by Uma Maheswara Rao G and fixed by Yi Liu (namenode)
- Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart).
-
-- HDFS-2856.
- Major improvement reported by Owen O'Malley and fixed by Chris Nauroth (datanode , security)
- Fix block protocol so that Datanodes don't require root or jsvc
- SASL now can be used to secure the DataTransferProtocol, which transfers file block content between HDFS clients and DataNodes. In this configuration, it is no longer required for secured clusters to start the DataNode as root and bind to privileged ports.
-- HDFS-573.
- Major improvement reported by Ziliang Guo and fixed by Chris Nauroth (libhdfs)
- Porting libhdfs to Windows
- The libhdfs C API is now supported on Windows.
-- HADOOP-11286.
- Blocker bug reported by Christopher Tubbs and fixed by
- Map/Reduce dangerously adds Guava @Beta class to CryptoUtils
-
-- HADOOP-11282.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (test)
- Skip NFS TestShellBasedIdMapping tests that are irrelevant on Windows.
-
-- HADOOP-11280.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (native , test)
- TestWinUtils#testChmod fails after removal of NO_PROPAGATE_INHERIT_ACE.
-
-- HADOOP-11265.
- Major bug reported by Larry McCay and fixed by Larry McCay (scripts)
- Credential and Key Shell Commands not available on Windows
-
-- HADOOP-11260.
- Blocker bug reported by Karthik Kambatla and fixed by Mike Yoder (security)
- Patch up Jetty to disable SSLv3
-
-- HADOOP-11254.
- Major bug reported by Zhijie Shen and fixed by Zhijie Shen
- Promoting AccessControlList to be public
-
-- HADOOP-11253.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev (tools)
- Hadoop streaming test TestStreamXmlMultipleRecords fails on Windows
-
-- HADOOP-11250.
- Minor bug reported by Yi Liu and fixed by Yi Liu (build)
- fix endmacro of set_find_shared_library_without_version in CMakeLists
-
-- HADOOP-11247.
- Major bug reported by Brandon Li and fixed by Brandon Li (nfs)
- Fix a couple javac warnings in NFS
-
-- HADOOP-11243.
- Blocker bug reported by Wei Yan and fixed by Wei Yan
- SSLFactory shouldn't allow SSLv3
-
-- HADOOP-11241.
- Major test reported by Varun Vasudev and fixed by Varun Vasudev
- TestNMSimulator fails sometimes due to timing issue
-
-- HADOOP-11233.
- Minor bug reported by Steve Loughran and fixed by Stephen Chu (conf)
- hadoop.security.kms.client.encrypted.key.cache.expiry property spelled wrong in core-default
-
-- HADOOP-11228.
- Major bug reported by Remus Rusanu and fixed by Remus Rusanu
- winutils task: unsecure path should not call AddNodeManagerAndUserACEsToObject
-
-- HADOOP-11221.
- Major bug reported by Jinghui Wang and fixed by Jinghui Wang (util)
- JAVA specification for hashcode does not enforce it to be non-negative, but IdentityHashStore assumes System.identityHashCode() is non-negative
-
-- HADOOP-11217.
- Blocker bug reported by Robert Kanter and fixed by Robert Kanter (kms)
- Disable SSLv3 in KMS
-
-- HADOOP-11216.
- Major improvement reported by Yi Liu and fixed by Colin Patrick McCabe (security)
- Improve Openssl library finding
-
-- HADOOP-11207.
- Major bug reported by Zhijie Shen and fixed by Zhijie Shen (security)
- DelegationTokenAuthenticationHandler needs to support DT operations for proxy user
-
-- HADOOP-11198.
- Minor bug reported by Ted Yu and fixed by Li Lu
- Fix typo in javadoc for FileSystem#listStatus()
-
-- HADOOP-11195.
- Major improvement reported by Yongjun Zhang and fixed by Yongjun Zhang (nfs , security)
- Move Id-Name mapping in NFS to the hadoop-common area for better maintenance
-
-- HADOOP-11194.
- Major bug reported by Karthik Kambatla and fixed by Karthik Kambatla
- Ignore .keep files
-
-- HADOOP-11193.
- Major bug reported by Xiaoyu Yao and fixed by Xiaoyu Yao (native)
- Fix uninitialized variables in NativeIO.c
-
-- HADOOP-11184.
- Major improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (native)
- Update Hadoop's lz4 to r123
-
-- HADOOP-11182.
- Major bug reported by Sascha Coenen and fixed by Ravi Prakash
- GraphiteSink emits wrong timestamps
-
-- HADOOP-11181.
- Major bug reported by Zhijie Shen and fixed by Zhijie Shen (security)
- o.a.h.security.token.delegation.DelegationTokenManager should be more generalized to handle other DelegationTokenIdentifier
-
-- HADOOP-11179.
- Major bug reported by Hitesh Shah and fixed by Craig Welch
- Tarball as local resource type archive fails to localize on Windows
-
-- HADOOP-11178.
- Minor bug reported by Arun Suresh and fixed by Arun Suresh (build)
- Fix findbugs exclude file
-
-- HADOOP-11176.
- Major bug reported by Arun Suresh and fixed by Arun Suresh
- KMSClientProvider authentication fails when both currentUgi and loginUgi are a proxied user
-
-- HADOOP-11175.
- Trivial bug reported by Yi Liu and fixed by Yi Liu (documentation , security)
- Fix several issues of hadoop security configuration in user doc.
-
-- HADOOP-11174.
- Major bug reported by Yi Liu and fixed by Yi Liu (kms , security)
- Delegation token for KMS should only be got once if it already exists
-
-- HADOOP-11170.
- Major bug reported by Arun Suresh and fixed by Arun Suresh
- ZKDelegationTokenSecretManager fails to renewToken created by a peer
-
-- HADOOP-11169.
- Major bug reported by Arun Suresh and fixed by Arun Suresh
- Fix DelegationTokenAuthenticatedURL to pass the connection Configurator to the authenticator
-
-- HADOOP-11168.
- Trivial bug reported by Yi Liu and fixed by Yi Liu (documentation)
- Remove duplicated entry "dfs.webhdfs.enabled" in the user doc
-
-- HADOOP-11163.
- Minor bug reported by Chuan Liu and fixed by Chuan Liu (metrics)
- MetricsSystemImpl may miss a registered source
-
-- HADOOP-11161.
- Major bug reported by Arun Suresh and fixed by Arun Suresh
- Expose close method in KeyProvider to give clients of Provider implementations a hook to release resources
-
-- HADOOP-11160.
- Trivial bug reported by Charles Lamb and fixed by Charles Lamb (nfs)
- Fix typo in nfs3 server duplicate entry reporting
-
-- HADOOP-11154.
- Trivial bug reported by Chris Nauroth and fixed by Chris Nauroth (documentation , native)
- Update BUILDING.txt to state that CMake 3.0 or newer is required on Mac.
-
-- HADOOP-11153.
- Major improvement reported by Andrew Wang and fixed by Andrew Wang (kms)
- Make number of KMS threads configurable
-
-- HADOOP-11151.
- Major bug reported by zhubin and fixed by Arun Suresh (security)
- Automatically refresh auth token and retry on auth failure
-
-- HADOOP-11145.
- Major bug reported by Akira AJISAKA and fixed by Akira AJISAKA (test)
- TestFairCallQueue fails
-
-- HADOOP-11143.
- Minor bug reported by Steve Loughran and fixed by Steve Loughran (net)
- NetUtils.wrapException loses inner stack trace on BindException
-
-- HADOOP-11140.
- Major bug reported by Juan Yu and fixed by Juan Yu
- hadoop-aws only need test-scoped dependency on hadoop-common's tests jar
-
-- HADOOP-11133.
- Minor bug reported by zhubin and fixed by Yi Liu (security)
- Should trim the content of keystore password file for JavaKeyStoreProvider
-
-- HADOOP-11130.
- Major bug reported by Allen Wittenauer and fixed by Brandon Li (nfs)
- NFS updateMaps OS check is reversed
-
-- HADOOP-11122.
- Blocker bug reported by Karthik Kambatla and fixed by Arun Suresh
- Fix findbugs in ZK DelegationTokenSecretManagers
-
-- HADOOP-11113.
- Major bug reported by Arun Suresh and fixed by Arun Suresh (security)
- Namenode not able to reconnect to KMS after KMS restart
-
-- HADOOP-11112.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- TestKMSWithZK does not use KEY_PROVIDER_URI
-
-- HADOOP-11111.
- Minor improvement reported by Steve Loughran and fixed by Steve Loughran (tools)
- MiniKDC to use locale EN_US for case conversions
-
-- HADOOP-11110.
- Major bug reported by Andrew Wang and fixed by Arun Suresh
- JavaKeystoreProvider should not report a key as created if it was not flushed to the backing file
-
-- HADOOP-11109.
- Major bug reported by Jian He and fixed by Jian He
- Site build is broken
-
-- HADOOP-11106.
- Minor improvement reported by Andrew Wang and fixed by Charles Lamb (documentation)
- Document considerations of HAR and Encryption
-
-- HADOOP-11105.
- Major bug reported by Chuan Liu and fixed by Chuan Liu (metrics)
- MetricsSystemImpl could leak memory in registered callbacks
-
-- HADOOP-11101.
- Minor improvement reported by skrho and fixed by skrho
- How about inputstream close statement from catch block to finally block in FileContext#copy() ?
-
-- HADOOP-11099.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMS return HTTP UNAUTHORIZED 401 on ACL failure
-
-- HADOOP-11097.
- Trivial bug reported by Charles Lamb and fixed by Charles Lamb (documentation)
- kms docs say proxyusers, not proxyuser for config params
-
-- HADOOP-11096.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMS: KeyAuthorizationKeyProvider should verify the keyversion belongs to the keyname on decrypt
-
-- HADOOP-11091.
- Minor bug reported by David S. Wang and fixed by David S. Wang (fs/s3)
- Eliminate old configuration parameter names from s3a
-
-- HADOOP-11088.
- Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (security)
- Unittest TestKeyShell, TestCredShell and TestKMS assume UNIX path separator for JECKS key store path
-
-- HADOOP-11085.
- Major bug reported by Mit Desai and fixed by Mit Desai
- Excessive logging by org.apache.hadoop.util.Progress when value is NaN
-
-- HADOOP-11083.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- After refactoring of HTTP proxyuser to common, doAs param is case sensitive
-
-- HADOOP-11077.
- Major bug reported by Gregory Chanan and fixed by Gregory Chanan (security)
- NPE if hosts not specified in ProxyUsers
-
-- HADOOP-11074.
- Major sub-task reported by David S. Wang and fixed by David S. Wang (fs/s3)
- Move s3-related FS connector code to hadoop-aws
-
-- HADOOP-11073.
- Major test reported by Xiaoyu Yao and fixed by Xiaoyu Yao (security)
- Credential Provider related Unit Tests Failure on Windows
-
-- HADOOP-11071.
- Minor test reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMSClientProvider should drain the local generated EEK cache on key rollover
-
-- HADOOP-11070.
- Major test reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security , test)
- Create MiniKMS for testing
-
-- HADOOP-11069.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMSClientProvider should use getAuthenticationMethod() to determine if in proxyuser mode or not
-
-- HADOOP-11068.
- Major improvement reported by Gregory Chanan and fixed by Gregory Chanan (security)
- Match hadoop.auth cookie format to jetty output
-
-- HADOOP-11067.
- Major bug reported by Yesha Vora and fixed by Xiaoyu Yao
- warning message 'ssl.client.truststore.location has not been set' gets printed for hftp command
-
-- HADOOP-11064.
- Blocker bug reported by Steve Loughran and fixed by Chris Nauroth (native)
- UnsatisifedLinkError with hadoop 2.4 JARs on hadoop-2.6 due to NativeCRC32 method changes
-
-- HADOOP-11063.
- Blocker bug reported by Chris Nauroth and fixed by Chris Nauroth
- KMS cannot deploy on Windows, because class names are too long.
-
-- HADOOP-11062.
- Major bug reported by Alejandro Abdelnur and fixed by Arun Suresh (security , test)
- CryptoCodec testcases requiring OpenSSL should be run only if -Pnative is used
-
-- HADOOP-11060.
- Major test reported by Alejandro Abdelnur and fixed by Yi Liu (security)
- Create a CryptoCodec test that verifies interoperability between the JCE and OpenSSL implementations
-
-- HADOOP-11057.
- Minor improvement reported by Steve Loughran and fixed by Xiaoyu Yao (native)
- checknative command to probe for winutils.exe on windows
-
-- HADOOP-11056.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang (security)
- OsSecureRandom.setConf() might leak file descriptors.
-
-- HADOOP-11054.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Add a KeyProvider instantiation based on a URI
-
-- HADOOP-11049.
- Minor bug reported by Sangjin Lee and fixed by Sangjin Lee (util)
- javax package system class default is too broad
-
-- HADOOP-11048.
- Minor bug reported by Sangjin Lee and fixed by Sangjin Lee (util)
- user/custom LogManager fails to load if the client classloader is enabled
-
-- HADOOP-11040.
- Major bug reported by Yi Liu and fixed by Yi Liu (security)
- Return value of read(ByteBuffer buf) in CryptoInputStream is incorrect in some cases
-
-- HADOOP-11036.
- Minor bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA
- Add build directory to .gitignore
-
-- HADOOP-11030.
- Minor improvement reported by Juan Yu and fixed by Juan Yu
- Define a variable jackson.version instead of using constant at multiple places
-
-- HADOOP-11021.
- Minor improvement reported by Zhe Zhang and fixed by Zhe Zhang
- Configurable replication factor in the hadoop archive command
-
-- HADOOP-11017.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KMS delegation token secret manager should be able to use zookeeper as store
-
-- HADOOP-11016.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMS should support signing cookies with zookeeper secret manager
-
-- HADOOP-11015.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur
- Http server/client utils to propagate and recreate Exceptions from server to client
-
-- HADOOP-11012.
- Major bug reported by Eric Payne and fixed by Eric Payne (fs)
- hadoop fs -text of zero-length file causes EOFException
-
-- HADOOP-11009.
- Major improvement reported by Gary Steelman and fixed by Gary Steelman (tools/distcp)
- Add Timestamp Preservation to DistCp
-
-- HADOOP-11007.
- Major improvement reported by Jason Lowe and fixed by Jason Lowe (build , fs)
- Reinstate building of ant tasks support
-
-- HADOOP-11005.
- Minor bug reported by Lei (Eddy) Xu and fixed by Lei (Eddy) Xu (conf)
- Fix HTTP content type for ReconfigurationServlet
-
-- HADOOP-10998.
- Trivial improvement reported by Jim Hester and fixed by Jim Hester (scripts)
- Fix bash tab completion code to work
-
-- HADOOP-10994.
- Major task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KeyProviderCryptoExtension should use CryptoCodec for generation/decryption of keys
-
-- HADOOP-10992.
- Major task reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Merge KMS to branch-2
-
-- HADOOP-10990.
- Major improvement reported by Brandon Li and fixed by Brandon Li (nfs)
- Add missed NFSv3 request and response classes
-
-- HADOOP-10989.
- Major bug reported by Chris Nauroth and fixed by Chris Nauroth (native)
- Work around buggy getgrouplist() implementations on Linux that return 0 on failure
-
-- HADOOP-10982.
- Major improvement reported by Andrew Wang and fixed by Alejandro Abdelnur
- KMS: Support for multiple Kerberos principals
-
-- HADOOP-10975.
- Major improvement reported by James Thomas and fixed by James Thomas (performance)
- org.apache.hadoop.util.DataChecksum should support native checksum calculation
-
-- HADOOP-10973.
- Minor bug reported by Peter Klavins and fixed by Peter Klavins (documentation)
- Native Libraries Guide contains format error
-
-- HADOOP-10972.
- Major bug reported by Peter Klavins and fixed by Peter Klavins (documentation)
- Native Libraries Guide contains mis-spelt build line
-
-- HADOOP-10970.
- Major improvement reported by Andrew Wang and fixed by Andrew Wang
- Cleanup KMS configuration keys
-
-- HADOOP-10968.
- Major bug reported by Dinar Valeev and fixed by (build)
- hadoop native build fails to detect java_libarch on ppc64le
-
-- HADOOP-10967.
- Major improvement reported by Yi Liu and fixed by Yi Liu (security)
- Improve DefaultCryptoExtension#generateEncryptedKey performance
-
-- HADOOP-10966.
- Blocker bug reported by Vinayakumar B and fixed by David Villegas (native)
- Hadoop Common native compilation broken in windows
-
-- HADOOP-10964.
- Minor bug reported by Yi Liu and fixed by Yi Liu
- Small fix for NetworkTopologyWithNodeGroup#sortByDistance
-
-- HADOOP-10962.
- Major bug reported by David Villegas and fixed by David Villegas (native)
- Flags for posix_fadvise are not valid in some architectures
-
-- HADOOP-10954.
- Minor improvement reported by Masatake Iwasaki and fixed by Masatake Iwasaki (documentation)
- Adding site documents of hadoop-tools
-
-- HADOOP-10946.
- Trivial bug reported by Ray Chiang and fixed by Ray Chiang
- Fix a bunch of typos in log messages
-
-- HADOOP-10939.
- Major bug reported by Arun Suresh and fixed by Arun Suresh
- Fix TestKeyProviderFactory testcases to use default 128 bit length keys
-
-- HADOOP-10937.
- Major bug reported by Arun Suresh and fixed by Arun Suresh (security)
- Need to set version name correctly before decrypting EEK
-
-- HADOOP-10936.
- Major improvement reported by Andrew Wang and fixed by Andrew Wang
- Change default KeyProvider bitlength to 128
-
-- HADOOP-10933.
- Major sub-task reported by Larry McCay and fixed by Larry McCay (security)
- FileBasedKeyStoresFactory Should use Configuration.getPassword for SSL Passwords
-
-- HADOOP-10931.
- Minor bug reported by xukun and fixed by (build , fs/swift)
- compile error on project "Apache Hadoop OpenStack support"
-
-- HADOOP-10929.
- Trivial bug reported by Larry McCay and fixed by Larry McCay (security)
- Typo in Configuration.getPasswordFromCredentialProviders
-
-- HADOOP-10928.
- Trivial bug reported by Josh Elser and fixed by Josh Elser (security)
- Incorrect usage on 'hadoop credential list'
-
-- HADOOP-10927.
- Minor bug reported by Josh Elser and fixed by Josh Elser (security)
- Fix CredentialShell help behavior and error codes
-
-- HADOOP-10925.
- Blocker bug reported by Chris Nauroth and fixed by Chris Nauroth (native)
- Compilation fails in native link0 function on Windows.
-
-- HADOOP-10922.
- Major improvement reported by Andrew Wang and fixed by Larry McCay
- User documentation for CredentialShell
-
-- HADOOP-10920.
- Minor bug reported by Ted Yu and fixed by Akira AJISAKA (documentation)
- site plugin couldn't parse hadoop-kms index.apt.vm
-
-- HADOOP-10918.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur
- JMXJsonServlet fails when used within Tomcat
-
-- HADOOP-10911.
- Major bug reported by Gregory Chanan and fixed by (security)
- hadoop.auth cookie after HADOOP-10710 still not proper according to RFC2109
-
-- HADOOP-10905.
- Major sub-task reported by Larry McCay and fixed by Larry McCay (security)
- LdapGroupsMapping Should use configuration.getPassword for SSL and LDAP Passwords
-
-- HADOOP-10903.
- Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (scripts , util)
- Enhance hadoop classpath command to expand wildcards or write classpath into jar manifest.
- The "hadoop classpath" command has been enhanced to support options for automatic expansion of wildcards in classpath elements and writing the classpath to a jar file manifest. These options make it easier to construct a correct classpath for libhdfs applications.
-- HADOOP-10902.
- Minor improvement reported by Stephen Chu and fixed by Stephen Chu
- Deletion of directories with snapshots will not output reason for trash move failure
-
-- HADOOP-10900.
- Minor bug reported by Andrew Wang and fixed by Andrew Wang
- CredentialShell args should use single-dash style
-
-- HADOOP-10893.
- Major new feature reported by Sangjin Lee and fixed by Sangjin Lee (util)
- isolated classloader on the client side
-
-- HADOOP-10891.
- Major improvement reported by Andrew Wang and fixed by Andrew Wang
- Add EncryptedKeyVersion factory method to KeyProviderCryptoExtension
-
-- HADOOP-10887.
- Major bug reported by Stephen Chu and fixed by Stephen Chu (fs , test)
- Add XAttrs to ViewFs and make XAttrs + ViewFileSystem internal dir behavior consistent
-
-- HADOOP-10886.
- Major sub-task reported by Uma Maheswara Rao G and fixed by Uma Maheswara Rao G (fs)
- CryptoCodec#getCodecclasses throws NPE when configurations not loaded.
-
-- HADOOP-10884.
- Minor sub-task reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Fix dead link in Configuration javadoc
-
-- HADOOP-10882.
- Minor task reported by Todd Lipcon and fixed by Todd Lipcon (util)
- Move DirectBufferPool into common util
-
-- HADOOP-10881.
- Major improvement reported by Andrew Wang and fixed by Andrew Wang
- Clarify usage of encryption and encrypted encryption key in KeyProviderCryptoExtension
-
-- HADOOP-10880.
- Blocker bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Move HTTP delegation tokens out of URL querystring to a header
-
-- HADOOP-10868.
- Major sub-task reported by Robert Kanter and fixed by Robert Kanter (security)
- Create a ZooKeeper-backed secret provider
-
-- HADOOP-10866.
- Major bug reported by Yongjun Zhang and fixed by Yongjun Zhang
- RawLocalFileSystem fails to read symlink targets via the stat command when the format of stat command uses non-curly quotes
-
-- HADOOP-10863.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KMS should have a blacklist for decrypting EEKs
-
-- HADOOP-10862.
- Major bug reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- Miscellaneous trivial corrections to KMS classes
-
-- HADOOP-10857.
- Major bug reported by Tsuyoshi OZAWA and fixed by Tsuyoshi OZAWA (documentation)
- Native Libraries Guide doen't mention a dependency on openssl-development package
-
-- HADOOP-10855.
- Minor improvement reported by Todd Lipcon and fixed by Todd Lipcon (io)
- Allow Text to be read with a known length
-
-- HADOOP-10851.
- Major bug reported by Benoy Antony and fixed by Benoy Antony (security)
- NetgroupCache does not remove group memberships
-
-- HADOOP-10845.
- Major improvement reported by Chris Nauroth and fixed by Stephen Chu (fs , test)
- Add common tests for ACLs in combination with viewfs.
-
-- HADOOP-10843.
- Major bug reported by Jinghui Wang and fixed by Jinghui Wang (test , tools)
- TestGridmixRecord unit tests failure on PowerPC
-
-- HADOOP-10842.
- Major bug reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- CryptoExtension generateEncryptedKey method should receive the key name
-
-- HADOOP-10841.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- EncryptedKeyVersion should have a key name property
-
-- HADOOP-10839.
- Major improvement reported by shanyu zhao and fixed by shanyu zhao (metrics)
- Add unregisterSource() to MetricsSystem API
- The MetricsSystem abstract class has added a new abstract method, unregisterSource, for unregistering a previously registered metrics source. Custom subclasses of MetricsSystem must be updated to provide an implementation of this method.
-- HADOOP-10838.
- Major improvement reported by James Thomas and fixed by James Thomas (performance)
- Byte array native checksumming
-
-- HADOOP-10835.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Implement HTTP proxyuser support in HTTP authentication client/server libraries
-
-- HADOOP-10833.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony (security)
- Remove unused cache in UserProvider
-
-- HADOOP-10830.
- Major bug reported by Benoy Antony and fixed by Benoy Antony (security)
- Missing lock in JavaKeyStoreProvider.createCredentialEntry
-
-- HADOOP-10826.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony (security)
- Iteration on KeyProviderFactory.serviceLoader is thread-unsafe
-
-- HADOOP-10824.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony (security)
- Refactor KMSACLs to avoid locking
-
-- HADOOP-10820.
- Minor bug reported by Alex Holmes and fixed by zhihai xu
- Throw an exception in GenericOptionsParser when passed an empty Path
-
-- HADOOP-10817.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- ProxyUsers configuration should support configurable prefixes
-
-- HADOOP-10816.
- Major bug reported by Mike Yoder and fixed by Mike Yoder (security)
- KeyShell returns -1 on error to the shell, should be 1
-
-- HADOOP-10815.
- Major improvement reported by Chris Nauroth and fixed by Chris Nauroth (native)
- Implement Windows equivalent of mlock.
-
-- HADOOP-10814.
- Major bug reported by Alejandro Abdelnur and fixed by Robert Kanter
- Update Tomcat version used by HttpFS and KMS to latest 6.x version
-
-- HADOOP-10812.
- Trivial improvement reported by Andrew Wang and fixed by Andrew Wang
- Delegate KeyProviderExtension#toString to underlying KeyProvider
-
-- HADOOP-10810.
- Minor bug reported by Chris Nauroth and fixed by Chris Nauroth (native)
- Clean up native code compilation warnings.
-
-- HADOOP-10808.
- Minor improvement reported by Chris Nauroth and fixed by Chris Nauroth (native)
- Remove unused native code for munlock.
-
-- HADOOP-10793.
- Major improvement reported by Mike Yoder and fixed by Andrew Wang (security)
- KeyShell args should use single-dash style
-
-- HADOOP-10791.
- Major improvement reported by Alejandro Abdelnur and fixed by Robert Kanter (security)
- AuthenticationFilter should support externalizing the secret for signing and provide rotation support
-
-- HADOOP-10781.
- Major bug reported by Dmitry Sivachenko and fixed by Dmitry Sivachenko
- Unportable getgrouplist() usage breaks FreeBSD
-
-- HADOOP-10780.
- Major bug reported by Dmitry Sivachenko and fixed by Dmitry Sivachenko
- hadoop_user_info_alloc fails on FreeBSD due to incorrect sysconf use
-
-- HADOOP-10771.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Refactor HTTP delegation support out of httpfs to common
-
-- HADOOP-10770.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMS add delegation token support
-
-- HADOOP-10769.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- Create KeyProvider extension to handle delegation tokens
-
-- HADOOP-10758.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KMS: add ACLs on per key basis.
-
-- HADOOP-10757.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KeyProvider KeyVersion should provide the key name
-
-- HADOOP-10756.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KMS audit log should consolidate successful similar requests
-
-- HADOOP-10755.
- Major improvement reported by Andrew Wang and fixed by Lei (Eddy) Xu (security)
- Support negative caching of user-group mapping
-
-- HADOOP-10750.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KMSKeyProviderCache should be in hadoop-common
-
-- HADOOP-10736.
- Major improvement reported by Mike Yoder and fixed by Mike Yoder (security)
- Add key attributes to the key shell
-
-- HADOOP-10733.
- Minor bug reported by Ted Yu and fixed by Ted Yu
- Potential null dereference in CredentialShell#promptForCredential()
-
-- HADOOP-10732.
- Minor bug reported by Ted Yu and fixed by Ted Yu
- Update without holding write lock in JavaKeyStoreProvider#innerSetCredential()
-
-- HADOOP-10731.
- Trivial improvement reported by Henry Saputra and fixed by Henry Saputra (documentation)
- Remove @date JavaDoc comment in ProgramDriver class
-
-- HADOOP-10720.
- Major improvement reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API
-
-- HADOOP-10719.
- Major new feature reported by Alejandro Abdelnur and fixed by Arun Suresh (security)
- Add generateEncryptedKey and decryptEncryptedKey methods to KeyProvider
-
-- HADOOP-10698.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMS, add proxyuser support
-
-- HADOOP-10696.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Add optional attributes to KeyProvider Options and Metadata
-
-- HADOOP-10695.
- Major improvement reported by Andrew Wang and fixed by Mike Yoder
- KMSClientProvider should respect a configurable timeout.
-
-- HADOOP-10681.
- Major bug reported by Gopal V and fixed by Gopal V (performance)
- Remove synchronized blocks from SnappyCodec and ZlibCodec buffering inner loop
- Remove unnecessary synchronized blocks from Snappy/Zlib codecs.
-- HADOOP-10677.
- Major bug reported by David S. Wang and fixed by David S. Wang (fs/s3)
- ExportSnapshot fails on kerberized cluster using s3a
-
-- HADOOP-10676.
- Major bug reported by David S. Wang and fixed by David S. Wang (fs/s3)
- S3AOutputStream not reading new config knobs for multipart configs
-
-- HADOOP-10675.
- Major improvement reported by David S. Wang and fixed by David S. Wang (fs/s3)
- Add server-side encryption functionality to s3a
-
-- HADOOP-10673.
- Major bug reported by Ming Ma and fixed by Ming Ma
- Update rpc metrics when the call throws an exception
-
-- HADOOP-10650.
- Major sub-task reported by Benoy Antony and fixed by Benoy Antony (security)
- Add ability to specify a reverse ACL (black list) of users and groups
-
-- HADOOP-10645.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- TestKMS fails because race condition writing acl files
-
-- HADOOP-10635.
- Major sub-task reported by Alejandro Abdelnur and fixed by Yi Liu (security)
- Add a method to CryptoCodec to generate SRNs for IV
-
-- HADOOP-10632.
- Major sub-task reported by Alejandro Abdelnur and fixed by Yi Liu (security)
- Minor improvements to Crypto input and output streams
-
-- HADOOP-10611.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KMS, keyVersion name should not be assumed to be keyName@versionNumber
-
-- HADOOP-10610.
- Minor improvement reported by Ted Malaska and fixed by Ted Malaska (fs/s3)
- Upgrade S3n s3.fs.buffer.dir to support multi directories
-
-- HADOOP-10607.
- Major new feature reported by Larry McCay and fixed by Larry McCay (security)
- Create an API to Separate Credentials/Password Storage from Applications
-
-- HADOOP-10591.
- Major bug reported by Hari Shreedharan and fixed by Colin Patrick McCabe
- Compression codecs must used pooled direct buffers or deallocate direct buffers when stream is closed
-
-- HADOOP-10586.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (bin)
- KeyShell doesn't allow setting Options via CLI
-
-- HADOOP-10583.
- Minor bug reported by Charles Lamb and fixed by Charles Lamb (bin)
- bin/hadoop key throws NPE with no args and assorted other fixups
- bin/hadoop key
-with no args would throw an NPE.
-- HADOOP-10552.
- Trivial bug reported by Kenji Kikushima and fixed by Kenji Kikushima (documentation)
- Fix usage and example at FileSystemShell.apt.vm
-
-- HADOOP-10534.
- Major bug reported by Owen O'Malley and fixed by Owen O'Malley
- KeyProvider API should using windowing for retrieving metadata
-
-- HADOOP-10507.
- Minor bug reported by Stephen Chu and fixed by sathish (fs)
- FsShell setfacl can throw ArrayIndexOutOfBoundsException when no perm is specified
-
-- HADOOP-10488.
- Major bug reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (test)
- TestKeyProviderFactory fails randomly
-
-- HADOOP-10433.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Key Management Server based on KeyProvider API
-
-- HADOOP-10432.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Refactor SSLFactory to expose static method to determine HostnameVerifier
-
-- HADOOP-10431.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- Change visibility of KeyStore.Options getter methods to public
-
-- HADOOP-10430.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KeyProvider Metadata should have an optional description, there should be a method to retrieve the metadata from all keys
-
-- HADOOP-10429.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KeyStores should have methods to generate the materials themselves, KeyShell should use them
-
-- HADOOP-10428.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- JavaKeyStoreProvider should accept keystore password via configuration falling back to ENV VAR
-
-- HADOOP-10427.
- Major improvement reported by Alejandro Abdelnur and fixed by Alejandro Abdelnur (security)
- KeyProvider implementations should be thread safe
-
-- HADOOP-10404.
- Minor bug reported by Ted Yu and fixed by Colin Patrick McCabe
- Some accesses to DomainSocketWatcher#closed are not protected by lock
-
-- HADOOP-10402.
- Major bug reported by Robert Kanter and fixed by Robert Kanter
- Configuration.getValByRegex does not substitute for variables
-
-- HADOOP-10400.
- Major new feature reported by Jordan Mendelson and fixed by Jordan Mendelson (fs , fs/s3)
- Incorporate new S3A FileSystem implementation
-
-- HADOOP-10373.
- Major improvement reported by Steve Loughran and fixed by Steve Loughran (fs/s3)
- create tools/hadoop-amazon for aws/EMR support
-
-- HADOOP-10335.
- Major improvement reported by Benoy Antony and fixed by Benoy Antony
- An ip whilelist based implementation to resolve Sasl properties per connection
-
-- HADOOP-10281.
- Major sub-task reported by Chris Li and fixed by Chris Li
- Create a scheduler, which assigns schedulables a priority level
-
-- HADOOP-10244.
- Major bug reported by Larry McCay and fixed by Larry McCay (security)
- TestKeyShell improperly tests the results of a Delete
- Fix of inappropriate test of delete functionality.
-- HADOOP-10237.
- Major bug reported by Larry McCay and fixed by Larry McCay (security)
- JavaKeyStoreProvider needs to set keystore permissions properly
-
-- HADOOP-10231.
- Minor improvement reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Add some components in Native Libraries document
-
-- HADOOP-10224.
- Major bug reported by Larry McCay and fixed by Arun Suresh (security)
- JavaKeyStoreProvider has to protect against corrupting underlying store
-
-- HADOOP-10201.
- Major sub-task reported by Larry McCay and fixed by Larry McCay (security)
- Add Listing Support to Key Management APIs
- I just committed this. Thanks, Larry!
-- HADOOP-10177.
- Major bug reported by Owen O'Malley and fixed by Larry McCay (security)
- Create CLI tools for managing keys via the KeyProvider API
-
-- HADOOP-10150.
- Major new feature reported by Yi Liu and fixed by Yi Liu (security)
- Hadoop cryptographic file system
-
-- HADOOP-10141.
- Major bug reported by Owen O'Malley and fixed by Owen O'Malley (security)
- Create an API to separate encryption key storage from applications
-
-- HADOOP-10131.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B
- NetWorkTopology#countNumOfAvailableNodes() is returning wrong value if excluded nodes passed are not part of the cluster tree
-
-- HADOOP-10121.
- Trivial bug reported by Akira AJISAKA and fixed by Akira AJISAKA (documentation)
- Fix javadoc spelling for HadoopArchives#writeTopLevelDirs
-
-- HADOOP-10059.
- Minor bug reported by Jason Lowe and fixed by Tsuyoshi OZAWA (metrics)
- RPC authentication and authorization metrics overflow to negative values on busy clusters
-
-- HADOOP-9989.
- Major bug reported by Jinghui Wang and fixed by zhihai xu (security , util)
- Bug introduced in HADOOP-9374, which parses the -tokenCacheFile as binary file but set it to the configuration as JSON file.
-
-- HADOOP-9921.
- Major bug reported by Vinayakumar B and fixed by Vinayakumar B
- daemon scripts should remove pid file on stop call after stop or process is found not running
-
-- HADOOP-9740.
- Major bug reported by Allan Yan and fixed by Allan Yan (fs)
- FsShell's Text command does not read avro data files stored on HDFS
-
-- HADOOP-9576.
- Major bug reported by Jian He and fixed by Steve Loughran
- Make NetUtils.wrapException throw EOFException instead of wrapping it as IOException
-
-- HADOOP-9540.
- Minor improvement reported by Hari and fixed by (fs/s3 , test)
- Expose the InMemoryS3 and S3N FilesystemStores implementations for Unit testing.
-
-- HADOOP-8944.
- Trivial improvement reported by Jonathan Allen and fixed by Jonathan Allen
- Shell command fs -count should include human readable option
- Implements -h option for fs -count to show file sizes in human readable format. Additionally, ContentSummary.getHeader() now returns a different string that is incompatible with previous releases.
-- HADOOP-8896.
- Trivial improvement reported by Timothy Mann and fixed by Ray Chiang (documentation , io)
- Javadoc points to Wrong Reader and Writer classes in SequenceFile
-
-- HADOOP-8815.
- Minor improvement reported by Brandon Li and fixed by Brandon Li (test)
- RandomDatum overrides equals(Object) but no hashCode()
-
-- HADOOP-8808.
- Major bug reported by Hemanth Yamijala and fixed by Akira AJISAKA (documentation , fs)
- Update FsShell documentation to mention deprecation of some of the commands, and mention alternatives
-
-- HADOOP-8158.
- Major bug reported by Todd Lipcon and fixed by Daryn Sharp
- Interrupting hadoop fs -put from the command line causes a LeaseExpiredException
-
-- HADOOP-8069.
- Major improvement reported by Todd Lipcon and fixed by Todd Lipcon (ipc)
- Enable TCP_NODELAY by default for IPC
- This change enables the TCP_NODELAY flag for all Hadoop IPC connections, hence bypassing TCP Nagling. Nagling interacts poorly with TCP delayed ACKs especially for request-response protocols.
-- HADOOP-7664.
- Minor improvement reported by Ravi Prakash and fixed by Ravi Prakash (conf)
- o.a.h.conf.Configuration complains of overriding final parameter even if the value with which its attempting to override is the same.
-
-
-