Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,678 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

12678 Commits

Author SHA1 Message Date
Xiaoyu Yao 420e4db98a HDFS-11209. SNN can't checkpoint when rolling upgrade is not finalized. Contributed by Xiaoyu Yao. 
(cherry picked from commit b1fce2b8b1)
 2017-01-17 14:36:03 -08:00
Kihwal Lee 3017069a76 HADOOP-13976. Path globbing does not match newlines. Contributed by Eric Badger. 
(cherry picked from commit 706d630eb9)
 2017-01-17 15:12:50 -06:00
Xiaoyu Yao eae5bde007 HDFS-10913. Introduce fault injectors to simulate slow mirrors. Contributed by Xiaobing Zhou. 2017-01-17 10:30:34 -08:00
Jason Lowe bec7c56ce5 MAPREDUCE-6831. Flaky test TestJobImpl.testKilledDuringKillAbort. Contributed by Peter Bacsko 
(cherry picked from commit 78b487bde1)
 2017-01-17 16:03:20 +00:00
bibinchundatt 580275628b YARN-6057. yarn.scheduler.minimum-allocation-* descriptions are incorrect about behavior when a request is out of bounds. Contributed by Julia Sommer. 
(cherry picked from commit a9d946d4df)
 2017-01-17 14:18:41 +05:30
Arpit Agarwal be8e001f1f HDFS-11339. Support File IO sampling for Datanode IO profiling hooks. Contributed by Hanisha Koneru. 2017-01-16 22:49:14 -08:00
Arpit Agarwal 52fdd387ec HDFS-11299. Support multiple Datanode File IO hooks. Contributed by Hanisha Koneru. 2017-01-16 22:42:13 -08:00
Arpit Agarwal ae0082a3fc HDFS-11282. Document the missing metrics of DataNode Volume IO operations. Contributed by Yiqun Lin. 2017-01-16 22:24:49 -08:00
Arpit Agarwal 8ec9dca2e1 HDFS-11279. Cleanup unused DataNode#checkDiskErrorAsync(). Contributed by Hanisha Koneru. 2017-01-16 22:14:06 -08:00
Xiaoyu Yao eafaddca1a HDFS-11274. Datanode should only check the failed volume upon IO errors. Contributed by Xiaoyu Yao. 2017-01-16 22:14:04 -08:00
Arpit Agarwal ec80de3ccc HDFS-11182. Update DataNode to use DatasetVolumeChecker. 2017-01-16 22:14:00 -08:00
Arpit Agarwal 413dccaf78 HDFS-10959. Update DataNode to use DatasetVolumeChecker. Contributed by Xiaoyu Yao. 2017-01-16 22:13:58 -08:00
Akira Ajisaka 2d6be7ea23 HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb)
 2017-01-17 10:22:29 +09:00
Junping Du 861e275646 YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong. 
(cherry picked from commit cf695577aa)
 2017-01-16 16:41:16 -08:00
Jitendra Pandey ffb4c22f6e HDFS-11307. The rpc to portmap service for NFS has hardcoded timeout. Contributed by Mukul Kumar Singh. 2017-01-16 15:17:29 -08:00
Arpit Agarwal 954dae26cd HDFS-11337. (HDFS-10958 backport). Add instrumentation hooks around Datanode disk IO. 2017-01-16 13:53:30 -08:00
Lei Xu 00dec84cc2 HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error. (John Zhuge via lei) 
(cherry picked from commit 15066a3e8ddacd36bf7be9fba0b10c350ae45668)
 2017-01-15 14:36:21 +08:00
Wei-Chiu Chuang 914eeb997b HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 1cde954a4f)
 2017-01-13 11:46:57 -08:00
Sunil G f166bb8f09 YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan. 
(cherry picked from commit d3170f9eba)
 2017-01-13 18:30:31 +05:30
Arun Suresh 963ef1e31f YARN-5646. Addendum patch to include missing changes to site.xml. 
(cherry picked from commit 1f344e0579)
 2017-01-12 07:38:32 -08:00
Naganarasimha 4bd05840fc YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 
(cherry picked from commit a6b06f7179)
 2017-01-12 06:42:18 +05:30
Daniel Templeton 718ae2d477 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 
(cherry picked from commit 0fffebe51a91023c4949ab560f5c828f1b568072)
 2017-01-11 15:36:18 -08:00
Andrew Wang b089e122b1 HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 
(cherry picked from commit d51f8ba808)
 2017-01-11 15:13:41 -08:00
Steve Loughran ec2f5b18ab HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran 2017-01-11 17:24:25 +00:00
Jason Lowe cc09b1a49a YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du 
(cherry picked from commit 357eab9566)
 2017-01-11 17:15:14 +00:00
Arun Suresh 8e5de45e08 HADOOP-13903. Improvements to KMS logging to help debug authorization errors. (Tristan Stevens via asuresh) 
(cherry picked from commit be529dade1)
 2017-01-11 00:27:14 -08:00
Ming Ma b6258e2b15 HDFS-9391. Update webUI/JMX to display maintenance state info. (Manoj Govindassamy via mingma) 2017-01-10 20:20:13 -08:00
Junping Du ba6a013341 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 
(cherry picked from commit 4db119b7b5)
 2017-01-10 18:22:45 -08:00
Mingliang Liu ce5ad0e88f HDFS-11253. FileInputStream leak on failure path in BlockSender. Contributed by Arpit Agarwal 2017-01-10 14:48:28 -08:00
Xiaoyu Yao 48fb796fcb HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao. 2017-01-10 14:48:23 -08:00
Wangda Tan c00a468b0b YARN-6022. Revert changes of AbstractResourceRequest. (Contributed by Wangda Tan / Daniel Templeton) 2017-01-10 10:17:33 -08:00
Naganarasimha a7cdd39234 YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a6940)
 2017-01-10 15:55:46 +05:30
Naganarasimha ea38256fab YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-10 15:10:54 +05:30
Sunil G 3190a4ba44 YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang. 
(cherry picked from commit 9594c35dcb)
 2017-01-10 14:10:25 +05:30
Daniel Templeton 74bf9c6e4b YARN-6073. Misuse of format specifier in Preconditions.checkArgument (Contributed by Yuanbo Liu via Daniel Templeton) 
(cherry picked from commit 6332a318bc)
 2017-01-09 22:14:46 -08:00
Junping Du b600577095 YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 
(cherry picked from commit 945db55f2e)
 2017-01-09 18:15:21 -08:00
Yongjun Zhang 137547579b HDFS-11292. log lastWrittenTxId etc info in logSyncAll. Contributed by Yongjun Zhang. 
(cherry picked from commit 603cbcd513)
 2017-01-09 16:16:42 -08:00
Wei-Chiu Chuang 76d62b0632 HADOOP-13953. Make FTPFileSystem's data connection mode and transfer mode configurable. Contributed by Xiao Chen. 
(cherry picked from commit 0a212a40fc)

Conflicts:
   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-01-09 15:59:36 -08:00
Wangda Tan 447f71326b YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 
(cherry picked from commit 287d3d6804)
 2017-01-09 08:42:38 -08:00
Varun Saxena 4fab250fc0 YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce)
 2017-01-09 11:31:39 +05:30
Junping Du 7b5b57e5b2 HADOOP-13958. Bump up release year to 2017. Contributed by Junping Du. 
(cherry picked from commit 94f1d06f20)
 2017-01-08 15:48:35 -08:00
Arun Suresh 4985217de4 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 2017-01-08 08:27:10 -08:00
Junping Du bf87c523ab MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c6)
 2017-01-07 22:05:16 -08:00
Junping Du f615c1fbbc Addendum patch to YARN-3866: fix incompatible API change. Contributed by Wangda Tan. 2017-01-07 19:58:42 -08:00
Jian He ba5d482366 YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S 
(cherry picked from commit 020316458d)
 2017-01-06 18:27:36 -08:00
Arun Suresh 81da7d1d30 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 
(cherry picked from commit ac1e5d4f77)
 2017-01-06 11:25:49 -08:00
Arun Suresh 2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Arun Suresh 3d3bb30a25 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit b0aace21b1)
(cherry picked from commit 3faa0b0d682687099b253feee9b41340edef7111)
 2017-01-06 11:15:09 -08:00
Arun Suresh 44774eb21c YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh) 
(cherry picked from commit 005850b28f)
(cherry picked from commit cbff10b4147f98a89b393519b17e16385294af07)
 2017-01-06 11:15:09 -08:00
Arun Suresh 368565f0ce YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 
(cherry picked from commit 3219b7b4ac)
(cherry picked from commit 49921678858a17ca58ca2ae704ced1ff584e0859)
 2017-01-06 11:15:09 -08:00