Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,432 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

12432 Commits

Author SHA1 Message Date
Daniel Templeton b1e1f7546c MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88)
 2017-09-19 09:08:00 -05:00
Erik Krogen b8d86bb535 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262)
 2017-09-15 14:06:14 -07:00
Nathan Roberts 11b325d17d HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc156)
 2017-09-15 14:54:37 -05:00
Kihwal Lee 9e9e36fa70 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 13:16:43 -05:00
John Zhuge 584cd134e8 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065)
 2017-09-14 11:35:25 -07:00
Jason Lowe e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Eric Payne a171cb3d81 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 2017-09-13 15:19:41 -05:00
Arpit Agarwal 2cddf1cc0f HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:55:30 -07:00
Xiao Chen 9d4c4b2f6e Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 8b7d1df766.
 2017-09-12 19:53:26 -07:00
John Zhuge 6afbe47b03 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 14:29:05 -07:00
Junping Du c2d96dd50e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc)
 2017-09-08 13:14:08 -07:00
Xiao Chen 69d7b1ae50 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-09-07 16:30:53 -07:00
Jason Lowe c6db50503a HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen 
(cherry picked from commit dd814946f6)
 2017-09-06 16:14:24 -05:00
Jason Lowe 2071f8338c YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb49)
 2017-09-06 15:26:13 -05:00
Oleg Danilov 867a4b7d78
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e1)
(cherry picked from commit b616462eef)
 2017-09-01 21:36:45 +02:00
Jason Lowe 798ff2f52b YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451)
 2017-09-01 11:11:28 -05:00
John Zhuge b2f496bdcc HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed6)
 2017-08-31 21:16:58 -07:00
Junping Du d665d85682 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 2017-08-31 18:10:33 -07:00
Haibo Chen 2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko 7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Junping Du f9ebcebe74 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9)
(cherry picked from commit 7dd5bb9492)
 2017-08-30 13:33:42 -07:00
Xiao Chen 98436a2eb7 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 441c472552eadf66c748ab9493fd02d2c914994a)
 2017-08-30 09:43:27 -07:00
Junping Du 8724e36562 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:21:56 -07:00
Junping Du 79294b5f32 YARN-7083. Log aggregation deletes/renames while file is open. Contributed by Jason Lowe. 2017-08-29 13:18:49 -07:00
Jason Lowe fe4aff7330 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6)
 2017-08-29 12:54:32 -05:00
Steve Loughran 37da90b17a
HADOOP-13743. error message in AzureNativeFileSystemStore.connectUsingAnonymousCredentials has too many spaces. 
Contributed by Mingliang Liu and Steve Loughran
 2017-08-29 15:06:00 +01:00
Junping Du 1dfb3850c8 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da)
(cherry picked from commit f252e1fe28)
 2017-08-28 18:31:36 -07:00
Wangda Tan f40f4ce0e9 YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I26d719ad164062b688b49142dc7db40d888b882b
 2017-08-28 15:05:30 -07:00
Sunil G bc7307788c YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04)
 2017-08-28 12:43:47 +05:30
Eric Payne 8eb39f775c YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471)
 2017-08-25 15:58:28 -05:00
Jason Lowe dfdda969ad YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a)
 2017-08-25 15:33:00 -05:00
Kihwal Lee b0e951725f HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396)
 2017-08-25 15:04:37 -05:00
Arpit Agarwal b07eef43ae HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:02:59 -07:00
Arpit Agarwal 22fb9bb843 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 11:10:25 -07:00
Colin Patrick Mccabe 945c916eec HDFS-7764. DirectoryScanner shouldn't abort the scan if one directory had an error (Rakesh R via cmccabe) 
(cherry picked from commit f67149ab08)
 2017-08-25 11:02:05 -07:00
Jason Lowe 2d548759ad YARN-6640. AM heartbeat stuck when responseId overflows MAX_INT. Contributed by Botong Huang 
(cherry picked from commit 3a4e861169)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java
 2017-08-25 10:16:58 -05:00
John Zhuge 832b0b82e2 HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge. 
(cherry picked from commit 7e6463d2fb)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
 2017-08-23 12:18:36 -07:00
John Zhuge bc1c8f3e5a HDFS-11738. Hedged pread takes more time when block moved from initial locations. Contributed by Vinayakumar B. 
(cherry picked from commit b6bfb2fcb2)
(cherry picked from commit c54c3500ed)
 2017-08-23 20:05:54 +05:30
Jason Lowe f6892f5821 HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp 2017-08-22 16:55:14 -05:00
Jason Lowe ff9cbcbbf4 YARN-6988. container-executor fails for docker when command length > 4096 B. Contributed by Eric Badger 2017-08-22 16:32:45 -05:00
Jason Lowe 5cb8a92bdb YARN-7048. Fix tests faking kerberos to explicitly set ugi auth type. Contributed by Daryn Sharp 
(cherry picked from commit 657dd59cc8)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/security/TestRMDelegationTokens.java
 2017-08-22 13:28:44 -05:00
Jason Lowe 0b64773678 YARN-2416. InvalidStateTransitonException in ResourceManager if AMLauncher does not receive response for startContainers() call in time. Contributed by Jonathan Eagles 
(cherry picked from commit 3efcd51c3b)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/attempt/TestRMAppAttemptTransitions.java
 2017-08-22 13:02:52 -05:00
Jian He 3f735ad640 YARN-6959. RM may allocate wrong AM Container for new attempt. Contributed by Yuqi Wang 2017-08-18 13:08:54 -07:00
Jason Lowe 2810e6ab9c YARN-7020. TestAMRMProxy#testAMRMProxyTokenRenewal is flakey. Contributed by Robert Kanter 
(cherry picked from commit 14553061be)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestAMRMProxy.java
 2017-08-16 13:14:25 -05:00
Akira Ajisaka 2b92c1be04
YARN-6965. Duplicate instantiation in FairSchedulerQueueInfo. Contributed by Masahiro Tanaka. 
(cherry picked from commit 588c190afd)
(cherry picked from commit a28a3dc8f2)
 2017-08-16 14:07:23 +09:00
Ravi Prakash a611922de4 HDFS-12301. NN File Browser UI: Navigate to a path when enter is pressed 
(cherry picked from commit f34646d652)
(cherry picked from commit 895a35eac0)
 2017-08-15 15:46:22 -07:00
Jason Lowe 87c36d8b8b YARN-6917. Queue path is recomputed from scratch on every allocation. Contributed by Eric Payne 
(cherry picked from commit 5558792894)

Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
 2017-08-14 15:41:54 -05:00
Jason Lowe 00e51a9cd3 YARN-6987. Log app attempt during InvalidStateTransition. Contributed by Jonathan Eagles 
(cherry picked from commit 3325ef653d)
 2017-08-14 14:48:48 -05:00
John Zhuge de50bb6365 HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria. 
(cherry picked from commit 7769e96149)
 2017-08-13 00:32:25 -07:00
John Zhuge e967744558 HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge. 
(cherry picked from commit 582648befa)
 2017-08-11 11:02:18 -07:00