Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,709 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

13709 Commits

Author SHA1 Message Date
Jason Lowe cd6cf0caf6 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 
(cherry picked from commit a4f9c7c924)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-18 10:34:53 -05:00
Arpit Agarwal f42705b957 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:14:39 -07:00
Junping Du c34c823d26 YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 
(cherry picked from commit 1a84c24b0c)
 2017-09-15 15:34:00 -07:00
Erik Krogen 791e20c244 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262)
 2017-09-15 14:03:10 -07:00
Nathan Roberts bd14e8d8b0 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc156)
 2017-09-15 14:47:07 -05:00
Robert Kanter 4e36452d02 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:53:55 -07:00
Kihwal Lee 07dee7fd63 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae4)
 2017-09-15 09:55:59 -05:00
Sunil G 317af87add Addendum patch for YARN-7163 in branch-2 2017-09-15 11:34:14 +05:30
John Zhuge eaef8446f6 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065)
 2017-09-14 11:29:34 -07:00
Arun Suresh b93a23f411 YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh) 
(cherry picked from commit 66ca0a6540)
 2017-09-14 10:30:30 -07:00
Arun Suresh 52bf458616 YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh) 
(cherry picked from commit 4f8194430f)
 2017-09-14 10:29:54 -07:00
Arun Suresh 1af9f962c9 YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh) 
(cherry picked from commit 864fbacd45)
 2017-09-14 10:26:34 -07:00
Junping Du 7e6cc17133 YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong. 
(cherry picked from commit c92c1d521e)
 2017-09-13 22:18:39 -07:00
Jason Lowe f525dfc5ec MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)
 2017-09-13 17:22:58 -05:00
Eric Payne fb3d53e001 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 
(cherry picked from commit 3860be7961)
 2017-09-13 15:07:27 -05:00
Arpit Agarwal ecf352d645 HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:21 -07:00
Nathan Roberts 1d33290a0a YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts) 
(cherry picked from commit d265459024)
 2017-09-13 09:47:55 -07:00
Varun Vasudev 4647761341 YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas. 
(cherry picked from commit 972da46cb4)
 2017-09-13 18:15:02 +05:30
Robert Kanter 055bde9806
YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter) 
(cherry picked from commit 44fdf00964)
 2017-09-13 16:48:43 +09:00
Xiao Chen 0a8ed025e4 Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 5337c2511d.
 2017-09-12 19:53:14 -07:00
Arun Suresh 0d32a38ac3 YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix. 
(cherry picked from commit a1c398e0dc)
 2017-09-12 17:05:01 -07:00
Arun Suresh d735c1f26b YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh) 
(cherry picked from commit 2ae72692fc)
 2017-09-12 16:33:05 -07:00
Arun Suresh 7803a93fff YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh) 
(cherry picked from commit 5b007921cd)
 2017-09-12 16:33:05 -07:00
Arpit Agarwal ac20d7e246 HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:18:56 -07:00
Yufei Gu 1d40bc225a YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d50)
 2017-09-12 16:14:31 -07:00
Jason Lowe c314a66353 HADOOP-14843. Improve FsPermission symbolic parsing unit test coverage. Contributed by Bharat Viswanadham 
(cherry picked from commit 86f4d1c66c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/permission/TestFsPermission.java
 2017-09-12 17:00:22 -05:00
Wangda Tan 10803bf08d YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
 2017-09-11 20:56:17 -07:00
Daniel Templeton fac4172e02 YARN-6622. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 6651cbcc72)
 2017-09-11 16:16:15 -07:00
Daniel Templeton 2e5a48afe3 Revert "YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)" 
This reverts commit 1acd004727.
 2017-09-11 16:15:06 -07:00
Daniel Templeton 1acd004727 YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 66be6c848b)
 2017-09-11 15:48:21 -07:00
Steve Loughran a46204437b
HADOOP-14851 LambdaTestUtils.eventually() doesn't spin on Assertion failures. 
Contributed by Steve Loughran

(cherry picked from commit 180e814b08)
 2017-09-11 21:33:44 +01:00
Xiaoyu Yao b669aca518 HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin. 2017-09-11 11:40:48 -07:00
Wangda Tan c093e4622c YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) 
Change-Id: I8fa7d70526505d4082ac9b51cd6066376a599108
 2017-09-11 11:34:51 -07:00
Steve Loughran 87af3f4991
HADOOP-14520. WASB: Block compaction for Azure Block Blobs. 
Contributed by Georgi Chalakov
 2017-09-11 14:15:58 +01:00
Yufei Gu 416a44004f YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c)
 2017-09-10 23:24:14 -07:00
Xuan 02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
Sunil G 23dc6ef8b0 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:34:59 +05:30
Subru Krishnan 2c051dbe9e YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c)
 2017-09-08 14:44:46 -07:00
Chris Douglas 2f8825588e YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 
(cherry picked from commit d383bfdcd4)
 2017-09-08 14:44:34 -07:00
John Zhuge 1421196d20 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 14:15:51 -07:00
Nathan Roberts b3a478832a YARN-6219. Fix conf usage in NMWebServer unit tests. Contributed by Jason Lowe. 2017-09-08 16:01:06 -05:00
Robert Kanter 72823b2738 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter) 
(cherry picked from commit a323f73bae)
(cherry picked from commit f5f343a665)
 2017-09-08 13:18:59 -07:00
Junping Du eaf5c66f7e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc)
 2017-09-08 13:14:39 -07:00
Arun Suresh 2b09cec218 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 1f53ae7972)
 2017-09-08 09:26:50 -07:00
Sunil G a39829c456 YARN-6600. Introduce default and max lifetime of application at LeafQueue level. Contributed by Rohith Sharma K S. 2017-09-08 19:20:52 +05:30
Manoj Govindassamy 796404c3ed HDFS-12404. Rename hdfs config authorization.provider.bypass.users to attributes.provider.bypass.users. 
(cherry picked from commit 3b3be355b3)
 2017-09-07 17:21:33 -07:00
Xiao Chen c6a1e3facb HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)
 2017-09-07 16:30:38 -07:00
Jason Lowe c004d57d47 YARN-6930. Admins should be able to explicitly enable specific LinuxContainerRuntime in the NodeManager. Contributed by Shane Kumpf 2017-09-07 16:24:55 -05:00
Sunil G ef342455cb YARN-6248. user is not removed from UsersManager’s when app is killed with pending container requests. Contributed by Eric Payne. 
(cherry picked from commit c1386a90d7)
 2017-09-07 14:47:07 -05:00
Steve Loughran 6e20ae954c
HADOOP-14774. S3A case "testRandomReadOverBuffer" failed due to improper range parameter. 
Contributed by Yonger.

(cherry picked from commit 83449abb40)
 2017-09-07 19:58:40 +01:00