Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
13,820 Commits 326 Branches 371 Tags
Commit Graph

13820 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Yufei Gu
221c1bdf7e YARN-7099. ResourceHandlerModule.parseConfiguredCGroupPath only works for privileged yarn users. (Contributed by Miklos Szegedi via Yufei Gu) 
(cherry picked from commit d5bc9582793de6ab2d2e489cde293bf076229a1c)
 2017-08-28 10:11:01 -07:00
Yiqun Lin
d63e98ad5f HDFS-11530. Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault. Contributed by Chen Liang. 2017-08-28 20:47:38 +08:00
Sunil G
80570737a8 YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:27:29 +05:30
Li Lu
b7f319c53c YARN-6999. Add log about how to solve Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster. (Linlin Zhou via gtcarrera9) 
(cherry picked from commit 3bee7046c52bff681a5aecc956ba9be6c3543397)
 2017-08-25 22:45:20 -07:00
Lei Xu
c22b5108df HDFS-12215. DataNode.transferBlock does not create its daemon in the xceiver thread group. (Lei Xu) 
Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
 2017-08-25 16:06:34 -07:00
Eric Payne
1617744801 YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
 2017-08-25 15:52:29 -05:00
Jason Lowe
ea5bb5898f YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:32:29 -05:00
Kihwal Lee
5a83ffa396 HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 2017-08-25 14:53:53 -05:00
Arpit Agarwal
2cae387402 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 10:41:59 -07:00
Jason Lowe
82957ce0d0 YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)
 2017-08-25 09:20:50 -05:00
Yufei Gu
389b659c94 YARN-7049. FSAppAttempt preemption related fields have confusing names. (Contributed by Karthik Kambatla via Yufei Gu) 
(cherry picked from commit 9e2699ac2c99d8df85191dd7fbf9468b00f5b5aa)
 2017-08-24 17:54:00 -07:00
Junping Du
e12358c5ce YARN-6876. Create an abstract log writer for extendability. Contributed by Xuan Gong. 
(cherry picked from commit c2cb7ea1ef6532020b69031dbd18b0f9b8369f0f)
 2017-08-24 15:27:40 -07:00
Akira Ajisaka
bf903396a0
YARN-7047. Moving logging APIs over to slf4j in hadoop-yarn-server-nodemanager. Contributed by Yeliang Cang. 2017-08-25 03:28:02 +09:00
Arpit Agarwal
ae69c99068 HDFS-11577. Combine the old and the new chooseRandom for better performance. Contributed by Chen Liang. 2017-08-24 11:17:36 -07:00
bibinchundatt
cc50ca0722 YARN-7074. Fix NM state store update comment. Contributed by Botong Huang 
(cherry picked from commit de0cba700bcf4276726c0aa9df8d738787debc17)
 2017-08-24 16:40:18 +05:30
Junping Du
c98917f798 YARN-7090. testRMRestartAfterNodeLabelDisabled get failed when CapacityScheduler is configured. Contributed by Wangda Tan. 
(cherry picked from commit 652dd434d96cd1a1fe25cd8c636b1859c29f462b)
 2017-08-23 18:07:07 -07:00
Arpit Agarwal
8b2d5a4faa HDFS-11514. DFSTopologyNodeImpl#chooseRandom optimizations. Contributed by Chen Liang. 2017-08-23 12:49:13 -07:00
John Zhuge
6e8c696651 HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 
(cherry picked from commit 7e6463d2fb5f9383d88baec290461868cf476e4c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-23 11:59:47 -07:00
Wangda Tan
245b49332d YARN-6251. Do async container release to prevent deadlock during container updates. (Arun Suresh via wangda) 
Change-Id: I6c67d20c5dd4d22752830ebf0ed2340824976ecb
(cherry picked from commit f49843a9888ad8fe5c1bb4c16bfb5217d693009d)
 2017-08-23 10:35:37 -07:00
John Zhuge
c54c3500ed HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 
(cherry picked from commit b6bfb2fcb2391d51b8de97c01c1290880779132e)
 2017-08-23 19:44:52 +05:30
Subru Krishnan
a45ffdcddc YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru). 
(cherry picked from commit 4249172e1419acdb2b69ae3db43dc59da2aa2e03)
 2017-08-22 19:21:35 -07:00
Ray Chiang
2299c8dffa YARN-6798. Fix NM startup failure with old state store due to version mismatch. (Botong Huang via rchiang) 
(cherry picked from commit f5f14a2ad67f91064a73685b44369c6314f0e1cd)
 2017-08-22 15:24:02 -07:00
Jason Lowe
45c1f8ac8e HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 
(cherry picked from commit c3793102121767c46091805eae65ef3919a5f368)
 2017-08-22 16:53:51 -05:00
Jason Lowe
b15a393440 YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-22 16:28:08 -05:00
Jason Lowe
a7893033fe YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)
 2017-08-22 13:17:31 -05:00
Jason Lowe
fc7df25a11 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)
 2017-08-22 12:59:47 -05:00
Arpit Agarwal
b3ea11dfdb HDFS-11482. Add storage type demand to into DFSNetworkTopology#chooseRandom. Contributed by Chen Liang. 2017-08-21 14:07:59 -07:00
Manoj Govindassamy
89fc7fe670 HDFS-11988. Verify HDFS Snapshots with open files captured are consistent across truncates and appends to current version file. 2017-08-21 11:14:44 -07:00
Arpit Agarwal
d5379cc564 HDFS-12325. SFTPFileSystem operations should restore cwd. Contributed by Chen Liang. 2017-08-20 23:41:16 -07:00
Arun Suresh
e0ba8a9ed8 YARN-6979. [Addendum patch] Fixed classname and added javadocs. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 7a82d7bcea8124e1b65c275fac15bf2047d17471)
 2017-08-20 10:28:42 -07:00
Arun Suresh
f8f193c8e8 YARN-6979. Add flag to notify all types of container updates to NM via NodeHeartbeatResponse. (Kartheek Muthyala via asuresh) 
(cherry picked from commit 8410d862d3a72740f461ef91dddb5325955e1ca5)
 2017-08-20 08:31:17 -07:00
John Zhuge
76fbeed997 HDFS-12117. HttpFS does not seem to support SNAPSHOT related methods for WebHDFS REST Interface. Contributed by Wellington Chevreuil. 2017-08-18 22:06:02 -07:00
Yufei Gu
4bd417a056 YARN-6969. Clean up unused code in class FairSchedulerQueueInfo. (Larry Lo via Yufei Gu) 
(cherry picked from commit 8991f0baec62625c45144e2544066195800ab95b)
 2017-08-18 14:45:13 -07:00
Daniel Templeton
78f9dca2cb YARN-6952. Enable scheduling monitor in FS. (Yufei Gu via Daniel Templeton) 2017-08-18 14:40:30 -07:00
Arpit Agarwal
d8d2a221f8 HDFS-11450. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang. 2017-08-18 11:43:17 -07:00
Arpit Agarwal
850593133b Revert "HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru." 
This reverts commit bdc425057ebedb8f0988ce4dae18fdcd67c8523f.
 2017-08-18 10:16:13 -07:00
bibinchundatt
ecf1a029bb YARN-7007. NPE in RM while using YarnClient.getApplications(). Contributed by Lingfeng Su. 
(cherry picked from commit e05fa3451db343c0d22496b332910874b6be5b7f)
 2017-08-18 20:31:26 +05:30
Steve Loughran
c42e694ee5
HADOOP-14769. WASB: delete recursive should not fail if a file is deleted. 
Contributed by Thomas Marquardt

(cherry picked from commit c6b4e656b76b68cc1d0dbcc15a5aa5ea23335b7b)
 2017-08-18 14:38:04 +01:00
Manoj Govindassamy
a5bc30170c HDFS-12316. Verify HDFS snapshot deletion doesn't crash the ongoing file writes. 
(cherry picked from commit 4230872dd66d748172903b1522885b03f34bbf9b)
 2017-08-17 16:25:15 -07:00
Sunil G
5511cea86b YARN-3254. HealthReport should include disk full information. Contributed by Suma Shivaprasad. 
(cherry picked from commit f9a0e2338150f1bd3ba2c29f76979183fd3ed80c)
 2017-08-17 15:10:07 +05:30
Naganarasimha
fa3094e436 YARN-6327. Removing queues from CapacitySchedulerQueueManager and ParentQueue should be done with iterator. Contributed by Jonathan Hung. 
(cherry picked from commit 0a3aa40fe7878c939dbf4e6b43466595159ff930)
 2017-08-17 10:42:00 +05:30
Robert Kanter
9de332d502 MAPREDUCE-6936. Remove unnecessary dependency of hadoop-yarn-server-common from hadoop-mapreduce-client-common (haibochen via rkanter) 
(cherry picked from commit ab051bd42ee1d7c4d3b7cc71e6b2734a0955e767)
 2017-08-16 16:15:48 -07:00
Mingliang Liu
03d15fa58e HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze 2017-08-16 16:06:13 -07:00
Jason Lowe
107e1722d4 MAPREDUCE-6940. Copy-paste error in the TaskAttemptUnsuccessfulCompletionEvent constructor. Contributed by Oleg Danilov 2017-08-16 16:35:41 -05:00
Jason Lowe
cf30380d46 YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be0a341df3e628dcaf06717b4630b05e)
 2017-08-16 13:06:13 -05:00
Steve Loughran
d5680c08ed
HADOOP-14660. wasb: improve throughput by 34% when account limit exceeded. 
Contributed by Thomas Marquardt.
 2017-08-16 13:49:47 +01:00
bibinchundatt
67f9968407 YARN-6741. Deleting all children of a Parent Queue on refresh throws exception. Contributed by Naganarasimha G R. 2017-08-16 14:52:09 +05:30
Akira Ajisaka
a28a3dc8f2
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
 2017-08-16 14:07:08 +09:00
Subru Krishnan
ad46a90872 HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru). 
(cherry picked from commit 75dd866bfb8b63cb9f13179d4365b05c48e0907d)
 2017-08-15 16:54:48 -07:00
Ravi Prakash
895a35eac0 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
 2017-08-15 15:45:37 -07:00