Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,449 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

12449 Commits

Author SHA1 Message Date
Eric Payne f794adde3b YARN-7249. Fix CapacityScheduler NPE issue when a container preempted while the node is being removed. Contributed by Wangda Tan. 2017-09-26 09:07:04 -05:00
Akira Ajisaka 75aed26511
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari. 
(cherry picked from commit d08b8c801a)
(cherry picked from commit b017af2391)
 2017-09-26 16:48:28 +09:00
Akira Ajisaka ea7e655897
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit 3a10367a17)
(cherry picked from commit 6004fb4676)
 2017-09-26 01:48:54 +09:00
Jason Lowe 44b51b0047 Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit e103acc8f4.
 2017-09-25 11:13:50 -05:00
Anu Engineer 3ecc1431fc
HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 
(cherry picked from commit de5744c903)
 2017-09-25 13:12:27 +09:00
Xuan e103acc8f4 YARN-6570. No logs were found for running application, running 
container. (Contributed by Junping Du)
 2017-09-22 14:48:12 -07:00
Sunil G 1132f48253 YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 
(cherry picked from commit f9e0cc8cdc)
 2017-09-22 11:52:30 -05:00
Akira Ajisaka dade7396a2
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d137960)
(cherry picked from commit d80e2f2eae)
 2017-09-22 15:10:42 +09:00
Junping Du 1b3503e4b9 YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi. 2017-09-21 14:18:50 -07:00
Jason Lowe b0b10fc94a YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by sandflee and Jonathan Hung 2017-09-21 15:17:03 -05:00
Kihwal Lee c0bb242360 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3)
 2017-09-21 09:00:26 -05:00
Akira Ajisaka 4ac3fc10c2
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb)
(cherry picked from commit 3527327f34)
 2017-09-21 19:29:18 +09:00
Sangjin Lee 6d6758f751 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632ac)
 2017-09-20 22:50:55 -07:00
Ming Ma c54310a638 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:21:32 -07:00
Brahma Reddy Battula a81167e2ec HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55)
 2017-09-19 11:34:52 +05:30
Eric Payne 67ada5d485 MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408)
 2017-09-19 11:00:20 -05:00
Jason Lowe 03a380daec MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 2017-09-19 10:27:04 -05:00
Daniel Templeton b1e1f7546c MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88)
 2017-09-19 09:08:00 -05:00
Erik Krogen b8d86bb535 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262)
 2017-09-15 14:06:14 -07:00
Nathan Roberts 11b325d17d HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc156)
 2017-09-15 14:54:37 -05:00
Kihwal Lee 9e9e36fa70 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 13:16:43 -05:00
John Zhuge 584cd134e8 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065)
 2017-09-14 11:35:25 -07:00
Jason Lowe e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Eric Payne a171cb3d81 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 2017-09-13 15:19:41 -05:00
Arpit Agarwal 2cddf1cc0f HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:55:30 -07:00
Xiao Chen 9d4c4b2f6e Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 8b7d1df766.
 2017-09-12 19:53:26 -07:00
John Zhuge 6afbe47b03 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1)
 2017-09-08 14:29:05 -07:00
Junping Du c2d96dd50e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc)
 2017-09-08 13:14:08 -07:00
Xiao Chen 69d7b1ae50 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-09-07 16:30:53 -07:00
Jason Lowe c6db50503a HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen 
(cherry picked from commit dd814946f6)
 2017-09-06 16:14:24 -05:00
Jason Lowe 2071f8338c YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb49)
 2017-09-06 15:26:13 -05:00
Oleg Danilov 867a4b7d78
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e1)
(cherry picked from commit b616462eef)
 2017-09-01 21:36:45 +02:00
Jason Lowe 798ff2f52b YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451)
 2017-09-01 11:11:28 -05:00
John Zhuge b2f496bdcc HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed6)
 2017-08-31 21:16:58 -07:00
Junping Du d665d85682 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 2017-08-31 18:10:33 -07:00
Haibo Chen 2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko 7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Junping Du f9ebcebe74 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9)
(cherry picked from commit 7dd5bb9492)
 2017-08-30 13:33:42 -07:00
Xiao Chen 98436a2eb7 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 441c472552eadf66c748ab9493fd02d2c914994a)
 2017-08-30 09:43:27 -07:00
Junping Du 8724e36562 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:21:56 -07:00
Junping Du 79294b5f32 YARN-7083. Log aggregation deletes/renames while file is open. Contributed by Jason Lowe. 2017-08-29 13:18:49 -07:00
Jason Lowe fe4aff7330 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6)
 2017-08-29 12:54:32 -05:00
Steve Loughran 37da90b17a
HADOOP-13743. error message in AzureNativeFileSystemStore.connectUsingAnonymousCredentials has too many spaces. 
Contributed by Mingliang Liu and Steve Loughran
 2017-08-29 15:06:00 +01:00
Junping Du 1dfb3850c8 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da)
(cherry picked from commit f252e1fe28)
 2017-08-28 18:31:36 -07:00
Wangda Tan f40f4ce0e9 YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I26d719ad164062b688b49142dc7db40d888b882b
 2017-08-28 15:05:30 -07:00
Sunil G bc7307788c YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04)
 2017-08-28 12:43:47 +05:30
Eric Payne 8eb39f775c YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471)
 2017-08-25 15:58:28 -05:00
Jason Lowe dfdda969ad YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a)
 2017-08-25 15:33:00 -05:00
Kihwal Lee b0e951725f HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396)
 2017-08-25 15:04:37 -05:00
Arpit Agarwal b07eef43ae HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:02:59 -07:00