Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,378 Commits 326 Branches 371 Tags
Commit Graph

12378 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
John Zhuge
584cd134e8 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:35:25 -07:00
Jason Lowe
e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec1b5d196053426c70d455cf8f83f84f)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Eric Payne
a171cb3d81 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 2017-09-13 15:19:41 -05:00
Arpit Agarwal
2cddf1cc0f HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:55:30 -07:00
Xiao Chen
9d4c4b2f6e Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 8b7d1df766002ae033d5fca8017cfe7141803c58.
 2017-09-12 19:53:26 -07:00
John Zhuge
6afbe47b03 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1e0794baf493a468191e12681d68ab4)
 2017-09-08 14:29:05 -07:00
Junping Du
c2d96dd50e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc42957c885d5076f8c8e7945e074ba8)
 2017-09-08 13:14:08 -07:00
Xiao Chen
69d7b1ae50 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-09-07 16:30:53 -07:00
Jason Lowe
c6db50503a HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen 
(cherry picked from commit dd814946f68d52a9b1627ac4dd61f9ab093423ae)
 2017-09-06 16:14:24 -05:00
Jason Lowe
2071f8338c YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb4909bf9968dd30067617f74543e07ffb)
 2017-09-06 15:26:13 -05:00
Oleg Danilov
867a4b7d78
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
(cherry picked from commit b616462eef5ee06a0d7f7d5f78b34a8069b80507)
 2017-09-01 21:36:45 +02:00
Jason Lowe
798ff2f52b YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451fb91ef956c926f891283f70f0ada1c)
 2017-09-01 11:11:28 -05:00
John Zhuge
b2f496bdcc HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed62c80397f71949bee41dcd03065755c)
 2017-08-31 21:16:58 -07:00
Junping Du
d665d85682 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 2017-08-31 18:10:33 -07:00
Haibo Chen
2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko
7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc28d48a7909822416478aff7487bfe6)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Junping Du
f9ebcebe74 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9cfb0bcfb69cc0f1594120e7656f031)
(cherry picked from commit 7dd5bb949206fb635d875cd651292b451267a951)
 2017-08-30 13:33:42 -07:00
Xiao Chen
98436a2eb7 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 441c472552eadf66c748ab9493fd02d2c914994a)
 2017-08-30 09:43:27 -07:00
Junping Du
8724e36562 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:21:56 -07:00
Junping Du
79294b5f32 YARN-7083. Log aggregation deletes/renames while file is open. Contributed by Jason Lowe. 2017-08-29 13:18:49 -07:00
Jason Lowe
fe4aff7330 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6042a837edb17bc1ae98a651d969a43)
 2017-08-29 12:54:32 -05:00
Steve Loughran
37da90b17a
HADOOP-13743. error message in AzureNativeFileSystemStore.connectUsingAnonymousCredentials has too many spaces. 
Contributed by Mingliang Liu and Steve Loughran
 2017-08-29 15:06:00 +01:00
Junping Du
1dfb3850c8 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
(cherry picked from commit f252e1fe2887cc961cf3ad532bfa22d646775fbf)
 2017-08-28 18:31:36 -07:00
Wangda Tan
f40f4ce0e9 YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I26d719ad164062b688b49142dc7db40d888b882b
 2017-08-28 15:05:30 -07:00
Sunil G
bc7307788c YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:43:47 +05:30
Eric Payne
8eb39f775c YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
 2017-08-25 15:58:28 -05:00
Jason Lowe
dfdda969ad YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:33:00 -05:00
Kihwal Lee
b0e951725f HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396089972e23c533eca33c9cba231c45a)
 2017-08-25 15:04:37 -05:00
Arpit Agarwal
b07eef43ae HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:02:59 -07:00
Arpit Agarwal
22fb9bb843 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 11:10:25 -07:00
Colin Patrick Mccabe
945c916eec HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe) 
(cherry picked from commit f67149ab08bb49381def6c535ab4c4610e0a4221)
 2017-08-25 11:02:05 -07:00
Jason Lowe
2d548759ad YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169dc3da9df0158ba6f44a9bc8576e217)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java
 2017-08-25 10:16:58 -05:00
John Zhuge
832b0b82e2 HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 
(cherry picked from commit 7e6463d2fb5f9383d88baec290461868cf476e4c)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-23 12:18:36 -07:00
John Zhuge
bc1c8f3e5a HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 
(cherry picked from commit b6bfb2fcb2391d51b8de97c01c1290880779132e)
(cherry picked from commit c54c3500ed720e39926f4cc7ca3db0cb4854ff55)
 2017-08-23 20:05:54 +05:30
Jason Lowe
f6892f5821 HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 2017-08-22 16:55:14 -05:00
Jason Lowe
ff9cbcbbf4 YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-22 16:32:45 -05:00
Jason Lowe
5cb8a92bdb YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc840bce3426170baf68f9d46842f4f53)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
 2017-08-22 13:28:44 -05:00
Jason Lowe
0b64773678 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b3eb667d83e08b500bb7a7ea559fabe)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
 2017-08-22 13:02:52 -05:00
Jian He
3f735ad640 YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-18 13:08:54 -07:00
Jason Lowe
2810e6ab9c YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be0a341df3e628dcaf06717b4630b05e)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java
 2017-08-16 13:14:25 -05:00
Akira Ajisaka
2b92c1be04
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd49bdbd5708f7805bf6c68f09fee142)
(cherry picked from commit a28a3dc8f2709738c28f70fc394f51d3b1266296)
 2017-08-16 14:07:23 +09:00
Ravi Prakash
a611922de4 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652310442cb5339aabbbb269f10dfa838)
(cherry picked from commit 895a35eac0422c1845ebe3c5bed1ac2bda1ac807)
 2017-08-15 15:46:22 -07:00
Jason Lowe
87c36d8b8b YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894169425bff054364a1ab4c48b347fb9)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:41:54 -05:00
Jason Lowe
00e51a9cd3 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d6f364a82dd32485d9ef6d987380ce3)
 2017-08-14 14:48:48 -05:00
John Zhuge
de50bb6365 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e9614956283a86eda9e4e69aaa592c0ca960)
 2017-08-13 00:32:25 -07:00
John Zhuge
e967744558 HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 
(cherry picked from commit 582648befaf9908159f937d2cc8f549583a3483e)
 2017-08-11 11:02:18 -07:00
John Zhuge
407bbe39fa HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang 
(cherry picked from commit 00160f71b6d98244fcb1cb58b2db9fc24f1cd672)
 2017-08-11 11:02:14 -07:00
John Zhuge
8fdcf86e64 HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang 
(cherry picked from commit 59d59667a8b1d3fb4a744a41774b2397fd91cbb3)
 2017-08-11 11:02:09 -07:00
Kihwal Lee
a54c3437af HDFS-12278. LeaseManager operations are inefficient in 2.8. Contributed by Rushabh S Shah. 2017-08-09 16:51:19 -05:00
Kihwal Lee
639380efff HDFS-12157. Do fsyncDirectory(..) outside of FSDataset lock. Contributed by Vinayakumar B. 
(cherry picked from commit 74bcc8d1f213ffd18056b7d0590d0cd346abaff9)
 2017-08-09 09:33:07 -05:00