Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,587 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

12587 Commits

Author SHA1 Message Date
Arun Suresh 4985217de4 YARN-6066. Opportunistic containers Minor fixes : API annotations, parameter name changes, checkstyles. (asuresh) 2017-01-08 08:27:10 -08:00
Junping Du bf87c523ab MAPREDUCE-6711. JobImpl fails to handle preemption events on state COMMITTING. Contributed by Prabhu Joseph. 
(cherry picked from commit 679478d0c6)
 2017-01-07 22:05:16 -08:00
Junping Du f615c1fbbc Addendum patch to YARN-3866: fix incompatible API change. Contributed by Wangda Tan. 2017-01-07 19:58:42 -08:00
Jian He ba5d482366 YARN-6009. Skip validating app timeout value on recovery. Contributed by Rohith Sharma K S 
(cherry picked from commit 020316458d)
 2017-01-06 18:27:36 -08:00
Arun Suresh 81da7d1d30 YARN-5938. Refactoring OpportunisticContainerAllocator to use SchedulerRequestKey instead of Priority and other misc fixes (asuresh) 
(cherry picked from commit ac1e5d4f77)
 2017-01-06 11:25:49 -08:00
Arun Suresh 2b4d3e8506 YARN-5646. Add documentation and update config parameter names for scheduling of OPPORTUNISTIC containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 2273a74c1f)
(cherry picked from commit 9e17ffe599e12c59c48f4355de645c31a4735d04)
 2017-01-06 11:15:10 -08:00
Arun Suresh 3d3bb30a25 YARN-5982. Simplify opportunistic container parameters and metrics. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit b0aace21b1)
(cherry picked from commit 3faa0b0d682687099b253feee9b41340edef7111)
 2017-01-06 11:15:09 -08:00
Arun Suresh 44774eb21c YARN-5918. Handle Opportunistic scheduling allocate request failure when NM is lost. (Bibin A Chundatt via asuresh) 
(cherry picked from commit 005850b28f)
(cherry picked from commit cbff10b4147f98a89b393519b17e16385294af07)
 2017-01-06 11:15:09 -08:00
Arun Suresh 368565f0ce YARN-4597. Introduce ContainerScheduler and a SCHEDULED state to NodeManager container lifecycle. (asuresh) 
(cherry picked from commit 3219b7b4ac)
(cherry picked from commit 49921678858a17ca58ca2ae704ced1ff584e0859)
 2017-01-06 11:15:09 -08:00
Arun Suresh 047772f15f YARN-5823. Update NMTokens in case of requests with only opportunistic containers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 283fa33feb)
(cherry picked from commit 1c4cc88a754ac9f557cdc8c859b8aadec19a5067)
 2017-01-06 11:15:09 -08:00
Arun Suresh 0e60c7cd33 YARN-5377. Fix TestQueuingContainerManager.testKillMultipleOpportunisticContainers. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit f38a6d03a1)
(cherry picked from commit 83ccab8a653a4ef943276651c9604bd434675980)
 2017-01-06 11:15:09 -08:00
Arun Suresh 7437252102 YARN-2995. Enhance UI to show cluster resource utilization of various container Execution types. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 0aafc122d4)
(cherry picked from commit 25598b6e7118bacd33a167fb496e9cda5e9f9ad0)
 2017-01-06 11:15:09 -08:00
Arun Suresh 28f3bb38bf YARN-5799. Fix Opportunistic Allocation to set the correct value of Node Http Address. (asuresh) 
(cherry picked from commit aa3cab1eb2)
(cherry picked from commit ce10e98826884245e733431297d0275177965f17)
 2017-01-06 11:15:09 -08:00
Arun Suresh f9ea753414 YARN-5486. Update OpportunisticContainerAllocatorAMService::allocate method to handle OPPORTUNISTIC container requests. (Konstantinos Karanasos via asuresh) 
(cherry picked from commit 10be45986c)
(cherry picked from commit e3baa0988b4dfb592afe426ffeba3c7091b271c5)
 2017-01-06 11:15:09 -08:00
Jason Lowe 937f739804 MAPREDUCE-6682. TestMRCJCFileOutputCommitter fails intermittently Contributed by Akira Ajisaka. 
(cherry picked from commit 8f1c374bec)

Conflicts:

	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestMRTimelineEventHandling.java
 2017-01-06 18:46:28 +00:00
Sunil G 65e7ae5dcf YARN-5906. Update AppSchedulingInfo to use SchedulingPlacementSet. Contributed by Wangda Tan. 2017-01-06 21:30:52 +05:30
Arpit Agarwal 6f35700f04 HDFS-11149. Support for parallel checking of FsVolumes. 
Change-Id: I6e7ada81562f380a260f8f561d0fa6a4cb102560
 2017-01-06 07:55:19 -08:00
Arpit Agarwal fd3b1ca26c HDFS-11148. Update DataNode to use StorageLocationChecker at startup. 
Change-Id: I61dfae840e767f9a33c0bb45a058f2b866cbffa4
 2017-01-05 20:49:34 -08:00
Arpit Agarwal 76f1ab524c HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup. 
Change-Id: Iddedbeefd056af165be3557ccad199fe2878b591
 2017-01-05 20:49:30 -08:00
Daniel Templeton d1aa844dc6 MAPREDUCE-6715. Fix Several Unsafe Practices (Contributed by Yufei Gu via Daniel Templeton) 
(cherry picked from commit 0b8a7c18dd)
 2017-01-05 17:57:14 -08:00
Chris Nauroth 9c61858ee7 HDFS-9483. Documentation does not cover use of "swebhdfs" as URL scheme for SSL-secured WebHDFS. Contributed by Surendra Singh Lilhore. 
(cherry picked from commit 4c8f9e1302)
 2017-01-05 15:05:01 -08:00
Haohui Mai 20486730e8 HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2017-01-04 21:03:05 -08:00
Andrew Wang d4ca1c5226 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 2017-01-04 15:10:15 -08:00
Jason Lowe db947fb870 HADOOP-13578. Add Codec for ZStandard Compression. Contributed by churro morales 2017-01-04 14:46:44 +00:00
Rohith Sharma K S 3207762192 YARN-5988. RM unable to start in secure setup. Contributed by Ajith S. 
(cherry picked from commit e49e0a6e37)
 2017-01-04 12:10:05 +05:30
Akira Ajisaka 3e2d26a97f HADOOP-12733. Remove references to obsolete io.seqfile configuration variables. Contributed by Ray Chiang. 
(cherry picked from commit 01d31fe938)
 2017-01-04 14:15:00 +09:00
Junping Du dc2e44e8f8 YARN-5923. Unable to access logs for a running application if YARN_ACL_ENABLE is enabled. Contributed by Xuan Gong. 
(cherry picked from commit 8fadd69047)
 2017-01-03 15:19:36 -08:00
Wangda Tan 23f6a33156 YARN-6025. Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312)
(cherry picked from commit 554e6656bc)
 2017-01-03 14:52:31 -08:00
Wangda Tan cb64e8eb19 Revert "Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda)" because forgot to add JIRA Number 
This reverts commit 554e6656bc.
 2017-01-03 14:51:59 -08:00
Wangda Tan 554e6656bc Fix synchronization issues of AbstractYarnScheduler#nodeUpdate and its implementations. (Naganarasimha G R via wangda) 
(cherry picked from commit e0f2379312)
 2017-01-03 14:50:19 -08:00
Mingliang Liu 1b8cb0b9c9 Revert "HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran" 
This reverts commit a447efa2ab.
 2017-01-03 14:45:19 -08:00
Mingliang Liu a447efa2ab HADOOP-13946. Document how HDFS updates timestamps in the FS spec; compare with object stores. Contributed by Steve Loughran 
(cherry picked from commit 451efb08fe)
 2017-01-03 13:09:29 -08:00
Robert Kanter 2f32a6aea2 YARN-5529. Create new DiskValidator class with metrics (yufeigu via rkanter) 
(cherry picked from commit 591fb15944)
 2017-01-03 12:14:13 -08:00
Andrew Wang 2b5b62f9cc Revert "HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang." 
This reverts commit 80e6640848.
 2017-01-03 11:12:53 -08:00
Andrew Wang 80e6640848 HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang. 
(cherry picked from commit 7fcc73fc0d)

 Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
 2017-01-03 09:58:45 -08:00
Brahma Reddy Battula 029b6fbe71 Revert "HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao." 
This reverts commit 1bc9b316ba.
 2017-01-03 19:26:44 +05:30
Brahma Reddy Battula d6e5e4288c HADOOP-13883. Addendum patch to correct the message and alphabetized with the earlier patch. Contributed by Yiqun Lin. 
(cherry picked from commit 6938b67711)
 2016-12-31 16:17:15 +05:30
Haohui Mai 1bc9b316ba HDFS-11280. Allow WebHDFS to reuse HTTP connections to NN. Contributed by Zheng Shao. 2016-12-30 22:59:24 -08:00
Daniel Templeton 8cb7aa2b52 YARN-5931. Document timeout interfaces CLI and REST APIs (Contributed by Rohith Sharma K S via Daniel Templeton) 
(cherry picked from commit 165d01a73e)
 2016-12-30 08:56:03 -08:00
Mingliang Liu ec646b1a9d HADOOP-13942. Build failure due to errors of javadoc build in hadoop-azure. Contributed by Kai Sasaki 
(cherry picked from commit c6a5b689db)
 2016-12-29 16:24:38 -08:00
Lei Xu db81cc3add HDFS-11267. Avoid redefinition of storageDirs in NNStorage and cleanup its accessors in Storage. (Manoj Govindassamy via lei) 
(cherry picked from commit a4f66655ec)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
 2016-12-29 17:06:21 +08:00
Lei Xu f349c7692d HDFS-11251. ConcurrentModificationException during DataNode#refreshVolumes. (Manoj Govindassamy via lei) 
(cherry picked from commit e9f1396834)
 2016-12-29 15:35:17 +08:00
Daniel Templeton af266c8c85 HDFS-11278. Add missing @Test annotation for TestSafeMode.testSafeModeUtils() (Contributed by Lukas Majercak via Daniel Templeton) 
(cherry picked from commit ce3613c965)
 2016-12-28 20:11:09 -08:00
Robert Kanter f9b48206dd YARN-4882. Change the log level to DEBUG for recovering completed applications (templedf via rkanter) 
(cherry picked from commit f216276d21)
 2016-12-28 15:29:06 -08:00
Daniel Templeton 47f0a0e15b HADOOP-13932. Fix indefinite article in comments (Contributed by LiXin Ge via Daniel Templeton) 2016-12-28 07:03:51 -08:00
Sunil G b3231b480d YARN-6024. Capacity Scheduler 'continuous reservation looking' doesn't work when sum of queue's used and reserved resources is equal to max. Contributed by Wangda Tan. 
(cherry picked from commit f6715b26b6)
 2016-12-28 16:00:52 +05:30
Rohith Sharma K S cb40e3cffa YARN-6001. Improve moveApplicationQueues command line. Contributed by Sunil G. 
(cherry picked from commit e297be74ed)
 2016-12-28 12:17:38 +05:30
Wangda Tan 42f571728c YARN-5756. Add state-machine implementation for scheduler queues. (Xuan Gong via wangda) 
(cherry picked from commit 0840b4329b)
 2016-12-27 21:19:10 -08:00
Brahma Reddy Battula 5d46d5d4c6 HDFS-11252. TestFileTruncate#testTruncateWithDataNodesRestartImmediately can fail with BindException. Contributed by Yiqun Lin. 
(cherry picked from commit 0ddb8defad)
 2016-12-28 10:37:21 +05:30
Robert Kanter 88597991b0 YARN-5257. Fix unreleased resources and null dereferences (yufeigu via rkanter) 
(cherry picked from commit 9262797e86)
 2016-12-27 14:14:41 -08:00