Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,765 Commits 326 Branches 371 Tags
Commit Graph

13765 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Daniel Templeton
1944b0b286 MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88f698950a2f326030261001aa51b35e)
 2017-09-19 09:05:39 -05:00
Steve Loughran
12a026badb
HADOOP-14583. wasb throws an exception if you try to create a file and there's no parent directory. 
Contributed by Esfandiar Manii.
 2017-09-19 11:02:10 +01:00
Akira Ajisaka
e61baf9496
MAPREDUCE-6947. Moving logging APIs over to slf4j in hadoop-mapreduce-examples. Contributed by Gergery Novák. 
(cherry picked from commit 2018538fdba1a95a6556187569e872fce7f9e1c3)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
 2017-09-19 11:25:29 +09:00
Arun Suresh
8a12a914fd YARN-7203. Add container ExecutionType into ContainerReport. (Botong Huang via asuresh) 
(cherry picked from commit 56ef5279c1db93d03b2f1e04badbfe804f548918)
 2017-09-18 15:51:19 -07:00
Xuan
6aaca3d08e YARN-6570. No logs were found for running application, running 
container. Contributed by Junping Du

(cherry picked from commit 7c732924a889cd280e972882619a1827877fbafa)
(cherry picked from commit e91447ccc9ff438dd300e2564c251b2c4b0f369a)
 2017-09-18 14:05:36 -07:00
Wangda Tan
74708d3817 YARN-7149. Cross-queue preemption sometimes starves an underserved queue. (Eric Payne via wangda) 
Change-Id: Ib269991dbebce160378e8372ee6d24849c4a5ed6
(cherry picked from commit 3dfa937a1fadfc62947755872515f549b3b15e6a)
(cherry picked from commit 38c14ef8d8a094a7101917eb77d90f5e62324f61)
 2017-09-18 13:58:51 -05:00
Jason Lowe
cd6cf0caf6 YARN-7192. Add a pluggable StateMachine Listener that is notified of NM Container State changes. Contributed by Arun Suresh 
(cherry picked from commit a4f9c7c9247801dd37beec6fc195622af1b884ad)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java
 2017-09-18 10:34:53 -05:00
Arpit Agarwal
f42705b957 HDFS-12472. Add JUNIT timeout to TestBlockStatsMXBean. Contributed by Bharat Viswanadham. 2017-09-16 10:14:39 -07:00
Junping Du
c34c823d26 YARN-7174. Add retry logic in LogsCLI when fetch running application logs. Contributed by Xuan Gong. 
(cherry picked from commit 1a84c24b0cf6674fa755403971fa57d8e412b320)
 2017-09-15 15:34:00 -07:00
Erik Krogen
791e20c244 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-15 14:03:10 -07:00
Nathan Roberts
bd14e8d8b0 HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 14:47:07 -05:00
Robert Kanter
4e36452d02 YARN-7162. Remove XML excludes file format (rkanter) 2017-09-15 09:53:55 -07:00
Kihwal Lee
07dee7fd63 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 
(cherry picked from commit 78bdf10ae40300a5f2f3b95491d280802edc1df2)
 2017-09-15 09:55:59 -05:00
Sunil G
317af87add Addendum patch for YARN-7163 in branch-2 2017-09-15 11:34:14 +05:30
John Zhuge
eaef8446f6 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:29:34 -07:00
Arun Suresh
b93a23f411 YARN-6059. Update paused container state in the NM state store. (Hitesh Sharma via asuresh) 
(cherry picked from commit 66ca0a65408521d5f9b080dd16b353b49fb8eaea)
 2017-09-14 10:30:30 -07:00
Arun Suresh
52bf458616 YARN-5216. Expose configurable preemption policy for OPPORTUNISTIC containers running on the NM. (Hitesh Sharma via asuresh) 
(cherry picked from commit 4f8194430fc6a69d9cc99b78828fd7045d5683e8)
 2017-09-14 10:29:54 -07:00
Arun Suresh
1af9f962c9 YARN-5292. NM Container lifecycle and state transitions to support for PAUSED container state. (Hitesh Sharma via asuresh) 
(cherry picked from commit 864fbacd4548004b1de8b0812627976acd22aff5)
 2017-09-14 10:26:34 -07:00
Junping Du
7e6cc17133 YARN-7194. Log aggregation status is always Failed with the newly added log aggregation IndexedFileFormat. Contributed by Xuan Gong. 
(cherry picked from commit c92c1d521eadfd8a4cd8205cc6aee74816f353f4)
 2017-09-13 22:18:39 -07:00
Jason Lowe
f525dfc5ec MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec1b5d196053426c70d455cf8f83f84f)
 2017-09-13 17:22:58 -05:00
Eric Payne
fb3d53e001 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 
(cherry picked from commit 3860be7961580ac20dd505d665b580f0a04ac4f8)
 2017-09-13 15:07:27 -05:00
Arpit Agarwal
ecf352d645 HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:54:21 -07:00
Nathan Roberts
1d33290a0a YARN-7014. Fix off-by-one error causing heap corruption (Jason Lowe via nroberts) 
(cherry picked from commit d265459024b8e5f5eccf421627f684ca8f162112)
 2017-09-13 09:47:55 -07:00
Varun Vasudev
4647761341 YARN-5719. Enforce a C standard for native container-executor. Contributed by Chris Douglas. 
(cherry picked from commit 972da46cb48725ad49d3e0a033742bd1a8228f51)
 2017-09-13 18:15:02 +05:30
Robert Kanter
055bde9806
YARN-5753. fix NPE in AMRMClientImpl.getMatchingRequests() (haibochen via rkanter) 
(cherry picked from commit 44fdf009642ae4e99b15f89ec0ca43834f991ef3)
 2017-09-13 16:48:43 +09:00
Xiao Chen
0a8ed025e4 Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 5337c2511df5f75d950b713740c4d61bffc6034d.
 2017-09-12 19:53:14 -07:00
Arun Suresh
0d32a38ac3 YARN-7185. [Addendum patch] Minor javadoc and checkstyle fix. 
(cherry picked from commit a1c398e0dc85e424b45c50a3c3a0a784c04ed737)
 2017-09-12 17:05:01 -07:00
Arun Suresh
d735c1f26b YARN-7185. ContainerScheduler should only look at availableResource for GUARANTEED containers when OPPORTUNISTIC container queuing is enabled. (Wangda Tan via asuresh) 
(cherry picked from commit 2ae72692fc370267141a1ee55ef372ff62302b54)
 2017-09-12 16:33:05 -07:00
Arun Suresh
7803a93fff YARN-6706. Refactor ContainerScheduler to make oversubscription change easier. (Haibo Chen via asuresh) 
(cherry picked from commit 5b007921cdf01ecc8ed97c164b7d327b8304c529)
 2017-09-12 16:33:05 -07:00
Arpit Agarwal
ac20d7e246 HDFS-12407. Journal node fails to shutdown cleanly if JournalNodeHttpServer or JournalNodeRpcServer fails to start. Contributed by Ajay Kumar. 2017-09-12 16:18:56 -07:00
Yufei Gu
1d40bc225a YARN-7057. FSAppAttempt#getResourceUsage doesn't need to consider resources queued for preemption. (Contributed by Karthik via Yufei Gu) 
(cherry picked from commit 82c5dd1d508292ed88eda0f5356776437ba67d4c)
 2017-09-12 16:14:31 -07:00
Jason Lowe
c314a66353 HADOOP-14843. Improve FsPermission symbolic parsing unit test coverage. Contributed by Bharat Viswanadham 
(cherry picked from commit 86f4d1c66c8b541465ff769e5d951305c41c715c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/permission/TestFsPermission.java
 2017-09-12 17:00:22 -05:00
Wangda Tan
10803bf08d YARN-7173. Container update RM-NM communication fix for backward compatibility. (Arun Suresh via wangda) 
Change-Id: Ia7d61e0d9df1e703bc983a31e6856e84a5a0521c
 2017-09-11 20:56:17 -07:00
Daniel Templeton
fac4172e02 YARN-6622. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 6651cbcc72d92caf86b744fa76fba513b36b12c7)
 2017-09-11 16:16:15 -07:00
Daniel Templeton
2e5a48afe3 Revert "YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev)" 
This reverts commit 1acd0047275ec8a15ee18f76dbb392d31e8f607a.
 2017-09-11 16:15:06 -07:00
Daniel Templeton
1acd004727 YARN-6022. Document Docker work as experimental (Contributed by Varun Vasudev) 
(cherry picked from commit 66be6c848b47ae7248b0658e3bb2149d2387cebd)
 2017-09-11 15:48:21 -07:00
Steve Loughran
a46204437b
HADOOP-14851 LambdaTestUtils.eventually() doesn't spin on Assertion failures. 
Contributed by Steve Loughran

(cherry picked from commit 180e814b081d3707c95641171d649b547db41a04)
 2017-09-11 21:33:44 +01:00
Xiaoyu Yao
b669aca518 HADOOP-14839. DistCp log output should contain copied and deleted files and directories. Contributed by Yiqun Lin. 2017-09-11 11:40:48 -07:00
Wangda Tan
c093e4622c YARN-6726. Fix issues with docker commands executed by container-executor. (Shane Kumpf via wangda) 
Change-Id: I8fa7d70526505d4082ac9b51cd6066376a599108
 2017-09-11 11:34:51 -07:00
Steve Loughran
87af3f4991
HADOOP-14520. WASB: Block compaction for Azure Block Blobs. 
Contributed by Georgi Chalakov
 2017-09-11 14:15:58 +01:00
Yufei Gu
416a44004f YARN-6799. Remove the duplicated code in CGroupsHandlerImp.java. (Contributed by Weiyuan via Yufei Gu) 
(cherry picked from commit 5ab21dfe9c8e14585db71e02bcb021a526507f98)
 2017-09-10 23:24:14 -07:00
Xuan
02ae3f8202 YARN-7072. Add a new log aggregation file format controller (xgong) 2017-09-10 23:18:31 -07:00
Sunil G
23dc6ef8b0 YARN-7163. RMContext need not to be injected to webapp and other Always Running services. Contributed by Rohith Sharma K S. 2017-09-11 10:34:59 +05:30
Subru Krishnan
2c051dbe9e YARN-5330. SharingPolicy enhancements required to support recurring reservations in ReservationSystem. (Carlo Curino via Subru). 
(cherry picked from commit fa6137501c1499ae33f6e0e2adc31671a7e782dc)
 2017-09-08 14:44:46 -07:00
Chris Douglas
2f8825588e YARN-5164. Use plan RLE to improve CapacityOverTimePolicy efficiency 
(cherry picked from commit d383bfdcd40c2315197fa5f85c3e5cb3bb83167f)
 2017-09-08 14:44:34 -07:00
John Zhuge
1421196d20 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1e0794baf493a468191e12681d68ab4)
 2017-09-08 14:15:51 -07:00
Nathan Roberts
b3a478832a YARN-6219. Fix conf usage in NMWebServer unit tests. Contributed by Jason Lowe. 2017-09-08 16:01:06 -05:00
Robert Kanter
72823b2738 MAPREDUCE-6953. Skip the testcase testJobWithChangePriority if FairScheduler is used (pbacsko via rkanter) 
(cherry picked from commit a323f73bae65483652de290ef819268850803a17)
(cherry picked from commit f5f343a6654647721120380556fdc2e401a39275)
 2017-09-08 13:18:59 -07:00
Junping Du
eaf5c66f7e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc42957c885d5076f8c8e7945e074ba8)
 2017-09-08 13:14:39 -07:00
Arun Suresh
2b09cec218 YARN-6849. NMContainerStatus should have the Container ExecutionType. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 1f53ae79728065417c6a99eb6fcc8d3a080ab4cc)
 2017-09-08 09:26:50 -07:00