Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,720 Commits 326 Branches 371 Tags
Commit Graph

12720 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Arpit Agarwal
8ec9dca2e1 HDFS-11279. Cleanup unused DataNode#checkDiskErrorAsync(). Contributed by Hanisha Koneru. 2017-01-16 22:14:06 -08:00
Xiaoyu Yao
eafaddca1a HDFS-11274. Datanode should only check the failed volume upon IO errors. Contributed by Xiaoyu Yao. 2017-01-16 22:14:04 -08:00
Arpit Agarwal
ec80de3ccc HDFS-11182. Update DataNode to use DatasetVolumeChecker. 2017-01-16 22:14:00 -08:00
Arpit Agarwal
413dccaf78 HDFS-10959. Update DataNode to use DatasetVolumeChecker. Contributed by Xiaoyu Yao. 2017-01-16 22:13:58 -08:00
Akira Ajisaka
2d6be7ea23 HADOOP-13933. Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit e407449ddb52c36b74358f22db498424e82ec4a7)
 2017-01-17 10:22:29 +09:00
Junping Du
861e275646 YARN-6011. Add a new web service to list the files on a container in AHSWebService. Contributed by Xuan Gong. 
(cherry picked from commit cf695577aa6d9715b77ab7309ecd792ef226c439)
 2017-01-16 16:41:16 -08:00
Jitendra Pandey
ffb4c22f6e HDFS-11307. The rpc to portmap service for NFS has hardcoded timeout. Contributed by Mukul Kumar Singh. 2017-01-16 15:17:29 -08:00
Arpit Agarwal
954dae26cd HDFS-11337. (HDFS-10958 backport). Add instrumentation hooks around Datanode disk IO. 2017-01-16 13:53:30 -08:00
Lei Xu
00dec84cc2 HADOOP-13928. TestAdlFileContextMainOperationsLive.testGetFileContext1 runtime error. (John Zhuge via lei) 
(cherry picked from commit 15066a3e8ddacd36bf7be9fba0b10c350ae45668)
 2017-01-15 14:36:21 +08:00
Wei-Chiu Chuang
914eeb997b HDFS-11306. Print remaining edit logs from buffer if edit log can't be rolled. Contributed by Wei-Chiu Chuang. 
(cherry picked from commit 1cde954a4fe7760a09b680413ad763cbe4a8feb3)
 2017-01-13 11:46:57 -08:00
Sunil G
f166bb8f09 YARN-6081. LeafQueue#getTotalPendingResourcesConsideringUserLimit should deduct reserved from pending to avoid unnecessary preemption of reserved container. Contributed by Wangda Tan. 
(cherry picked from commit d3170f9eba9bc5c38b5fa50c24e37ca2bd5636c2)
 2017-01-13 18:30:31 +05:30
Arun Suresh
963ef1e31f YARN-5646. Addendum patch to include missing changes to site.xml. 
(cherry picked from commit 1f344e057977cdc7765fe3af0f8c48ebed1d664e)
 2017-01-12 07:38:32 -08:00
Naganarasimha
4bd05840fc YARN-6072. RM unable to start in secure mode. Contributed by Ajith S. 
(cherry picked from commit a6b06f71797ad1ed9edbcef279bcf7d9e569f955)
 2017-01-12 06:42:18 +05:30
Daniel Templeton
718ae2d477 YARN-5849. Automatically create YARN control group for pre-mounted cgroups (Contributed by Miklos Szegedi via Daniel Templeton) 
(cherry picked from commit 0fffebe51a91023c4949ab560f5c828f1b568072)
 2017-01-11 15:36:18 -08:00
Andrew Wang
b089e122b1 HDFS-11312. Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto. Contributed by Sean Mackrory. 
(cherry picked from commit d51f8ba80816c20cd97ba5aa839a84b0bb6a7291)
 2017-01-11 15:13:41 -08:00
Steve Loughran
ec2f5b18ab HADOOP-13336 S3A to support per-bucket configuration. Contributed by Steve Loughran 2017-01-11 17:24:25 +00:00
Jason Lowe
cc09b1a49a YARN-5416. TestRMRestart#testRMRestartWaitForPreviousAMToFinish failed intermittently due to not wait SchedulerApplicationAttempt to be stopped. Contributed by Junping Du 
(cherry picked from commit 357eab95668dbc419239857ac5ce763d76fd40e7)
 2017-01-11 17:15:14 +00:00
Arun Suresh
8e5de45e08 HADOOP-13903. Improvements to KMS logging to help debug authorization errors. (Tristan Stevens via asuresh) 
(cherry picked from commit be529dade182dd2f3718fc52133f43e83dce191f)
 2017-01-11 00:27:14 -08:00
Ming Ma
b6258e2b15 HDFS-9391. Update webUI/JMX to display maintenance state info. (Manoj Govindassamy via mingma) 2017-01-10 20:20:13 -08:00
Junping Du
ba6a013341 YARN-6079. Fix simple spelling errors in yarn test code. Contributed by vijay. 
(cherry picked from commit 4db119b7b55213929e5b86f2abb0ed84a21719b5)
 2017-01-10 18:22:45 -08:00
Mingliang Liu
ce5ad0e88f HDFS-11253. FileInputStream leak on failure path in BlockSender. Contributed by Arpit Agarwal 2017-01-10 14:48:28 -08:00
Xiaoyu Yao
48fb796fcb HDFS-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao. 2017-01-10 14:48:23 -08:00
Wangda Tan
c00a468b0b YARN-6022. Revert changes of AbstractResourceRequest. (Contributed by Wangda Tan / Daniel Templeton) 2017-01-10 10:17:33 -08:00
Naganarasimha
a7cdd39234 YARN-6054. TimelineServer fails to start when some LevelDb state files are missing. Contributed by Ravi Prakash. 
(cherry picked from commit 4c431a694059e40e78365b02a1497a6c7e479a70)
 2017-01-10 15:55:46 +05:30
Naganarasimha
ea38256fab YARN-6015. AsyncDispatcher thread name can be set to improved debugging. Contributed by Ajith S. 2017-01-10 15:10:54 +05:30
Sunil G
3190a4ba44 YARN-5899. Debug log in AbstractCSQueue#canAssignToThisQueue needs improvement. Contributed by Ying Zhang. 
(cherry picked from commit 9594c35dcb655add1991d8fd15897b40c4ad6205)
 2017-01-10 14:10:25 +05:30
Daniel Templeton
74bf9c6e4b YARN-6073. Misuse of format specifier in Preconditions.checkArgument (Contributed by Yuanbo Liu via Daniel Templeton) 
(cherry picked from commit 6332a318bc1e2e9d73d7159eab26347bb3f1f9b3)
 2017-01-09 22:14:46 -08:00
Junping Du
b600577095 YARN-4148. When killing app, RM releases app's resource before they are released by NM. Contributed by Jason Lowe. 
(cherry picked from commit 945db55f2e6521d33d4f90bbb09179b0feba5e7a)
 2017-01-09 18:15:21 -08:00
Yongjun Zhang
137547579b HDFS-11292. log lastWrittenTxId etc info in logSyncAll. Contributed by Yongjun Zhang. 
(cherry picked from commit 603cbcd513a74c29e0e4ec9dc181ff08887d64a4)
 2017-01-09 16:16:42 -08:00
Wei-Chiu Chuang
76d62b0632 HADOOP-13953. Make FTPFileSystem's data connection mode and transfer mode configurable. Contributed by Xiao Chen. 
(cherry picked from commit 0a212a40fcbd12a11294bff7a31e7433111733c9)

Conflicts:
   hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
 2017-01-09 15:59:36 -08:00
Wangda Tan
447f71326b YARN-3955. Support for application priority ACLs in queues of CapacityScheduler. (Sunil G via wangda) 
(cherry picked from commit 287d3d6804a869723ae36605a3c2d2b3eae3941e)
 2017-01-09 08:42:38 -08:00
Varun Saxena
4fab250fc0 YARN-6068. Log aggregation get failed when NM restart even with recovery (Junping Du via Varun Saxena) 
(cherry picked from commit f59e36b4ce71d3019ab91b136b6d7646316954e7)
 2017-01-09 11:31:39 +05:30
Junping Du
7b5b57e5b2 HADOOP-13958. Bump up release year to 2017. Contributed by Junping Du. 
(cherry picked from commit 94f1d06f205eb5679475569b932eb2792f53d891)
 2017-01-08 15:48:35 -08:00
Arun Suresh
4985217de4 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 2017-01-08 08:27:10 -08:00
Junping Du
bf87c523ab MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c643defb3759a2ecd3c704315edee124)
 2017-01-07 22:05:16 -08:00
Junping Du
f615c1fbbc Addendum patch to YARN-3866: fix incompatible API change. Contributed by Wangda Tan. 2017-01-07 19:58:42 -08:00
Jian He
ba5d482366 YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S 
(cherry picked from commit 020316458dfe6059b700f8d93a9791e4cb817b3f)
 2017-01-06 18:27:36 -08:00
Arun Suresh
81da7d1d30 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 
(cherry picked from commit ac1e5d4f77e3b9df8dcacb0b1f72eecc27931eb8)
 2017-01-06 11:25:49 -08:00
Arun Suresh
2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f3895163046cca09ff5e983df301d22)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Arun Suresh
3d3bb30a25 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit b0aace21b1ef3436ba9d516186208fee9a9ceef2)
(cherry picked from commit 3faa0b0d682687099b253feee9b41340edef7111)
 2017-01-06 11:15:09 -08:00
Arun Suresh
44774eb21c YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh) 
(cherry picked from commit 005850b28feb2f7bb8c2844d11e3f9d21b45d754)
(cherry picked from commit cbff10b4147f98a89b393519b17e16385294af07)
 2017-01-06 11:15:09 -08:00
Arun Suresh
368565f0ce YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 
(cherry picked from commit 3219b7b4ac7d12aee343f6ab2980b3357fc618b6)
(cherry picked from commit 49921678858a17ca58ca2ae704ced1ff584e0859)
 2017-01-06 11:15:09 -08:00
Arun Suresh
047772f15f YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 283fa33febe043bd7b4fa87546be26c9c5a8f8b5)
(cherry picked from commit 1c4cc88a754ac9f557cdc8c859b8aadec19a5067)
 2017-01-06 11:15:09 -08:00
Arun Suresh
0e60c7cd33 YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f38a6d03a11ca6de93a225563ddf55ec99d5063c)
(cherry picked from commit 83ccab8a653a4ef943276651c9604bd434675980)
 2017-01-06 11:15:09 -08:00
Arun Suresh
7437252102 YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 0aafc122d401a2f49d0f9ca556343f9ba871e979)
(cherry picked from commit 25598b6e7118bacd33a167fb496e9cda5e9f9ad0)
 2017-01-06 11:15:09 -08:00
Arun Suresh
28f3bb38bf YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh) 
(cherry picked from commit aa3cab1eb29c56368d15882d7260a994e615e8d8)
(cherry picked from commit ce10e98826884245e733431297d0275177965f17)
 2017-01-06 11:15:09 -08:00
Arun Suresh
f9ea753414 YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 10be45986cdf86a89055065b752959bd6369d54f)
(cherry picked from commit e3baa0988b4dfb592afe426ffeba3c7091b271c5)
 2017-01-06 11:15:09 -08:00
Jason Lowe
937f739804 MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec2451568f463ab68f7eb6db734ab14e)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:46:28 +00:00
Sunil G
65e7ae5dcf YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan. 2017-01-06 21:30:52 +05:30
Arpit Agarwal
6f35700f04 HDFS-11149. Support for parallel checking of FsVolumes. 
Change-Id: I6e7ada81562f380a260f8f561d0fa6a4cb102560
 2017-01-06 07:55:19 -08:00