Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,398 Commits 326 Branches 371 Tags
Commit Graph

12398 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Akira Ajisaka
75aed26511
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari. 
(cherry picked from commit d08b8c801a908b4242e7b21a54f3b1e4072f1eae)
(cherry picked from commit b017af23911c08d1f7e928007ebd42dbdbd82507)
 2017-09-26 16:48:28 +09:00
Akira Ajisaka
ea7e655897
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit 3a10367a17b99e2c3035ec54048f94f334d93227)
(cherry picked from commit 6004fb46761810ef87367c5207fafcdbb6507c39)
 2017-09-26 01:48:54 +09:00
Jason Lowe
44b51b0047 Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit e103acc8f495135e52ec1e700dc33747e735967e.
 2017-09-25 11:13:50 -05:00
Anu Engineer
3ecc1431fc
HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 
(cherry picked from commit de5744c9035308654c7096bf5bd04e769a3a19e3)
 2017-09-25 13:12:27 +09:00
Xuan
e103acc8f4 YARN-6570. No logs were found for running application, running 
container. (Contributed by Junping Du)
 2017-09-22 14:48:12 -07:00
Sunil G
1132f48253 YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 
(cherry picked from commit f9e0cc8cdca387bf2811c07beccae4af07362952)
 2017-09-22 11:52:30 -05:00
Akira Ajisaka
dade7396a2
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d13796039815350d3c73c51f9fe48531af0fa)
(cherry picked from commit d80e2f2eae1d584b99da305d715a3f5e958114e9)
 2017-09-22 15:10:42 +09:00
Junping Du
1b3503e4b9 YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi. 2017-09-21 14:18:50 -07:00
Jason Lowe
b0b10fc94a YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by sandflee and Jonathan Hung 2017-09-21 15:17:03 -05:00
Kihwal Lee
c0bb242360 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 09:00:26 -05:00
Akira Ajisaka
4ac3fc10c2
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
(cherry picked from commit 3527327f349264ae62f3673ab368505a16cc0a34)
 2017-09-21 19:29:18 +09:00
Sangjin Lee
6d6758f751 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:50:55 -07:00
Ming Ma
c54310a638 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:21:32 -07:00
Brahma Reddy Battula
a81167e2ec HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:34:52 +05:30
Eric Payne
67ada5d485 MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408104bdfe1f08efd79930e18862fafbb)
 2017-09-19 11:00:20 -05:00
Jason Lowe
03a380daec MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 2017-09-19 10:27:04 -05:00
Daniel Templeton
b1e1f7546c MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88f698950a2f326030261001aa51b35e)
 2017-09-19 09:08:00 -05:00
Erik Krogen
b8d86bb535 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-15 14:06:14 -07:00
Nathan Roberts
11b325d17d HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 14:54:37 -05:00
Kihwal Lee
9e9e36fa70 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 13:16:43 -05:00
John Zhuge
584cd134e8 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:35:25 -07:00
Jason Lowe
e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec1b5d196053426c70d455cf8f83f84f)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Eric Payne
a171cb3d81 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 2017-09-13 15:19:41 -05:00
Arpit Agarwal
2cddf1cc0f HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:55:30 -07:00
Xiao Chen
9d4c4b2f6e Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 8b7d1df766002ae033d5fca8017cfe7141803c58.
 2017-09-12 19:53:26 -07:00
John Zhuge
6afbe47b03 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1e0794baf493a468191e12681d68ab4)
 2017-09-08 14:29:05 -07:00
Junping Du
c2d96dd50e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc42957c885d5076f8c8e7945e074ba8)
 2017-09-08 13:14:08 -07:00
Xiao Chen
69d7b1ae50 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-09-07 16:30:53 -07:00
Jason Lowe
c6db50503a HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen 
(cherry picked from commit dd814946f68d52a9b1627ac4dd61f9ab093423ae)
 2017-09-06 16:14:24 -05:00
Jason Lowe
2071f8338c YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb4909bf9968dd30067617f74543e07ffb)
 2017-09-06 15:26:13 -05:00
Oleg Danilov
867a4b7d78
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
(cherry picked from commit b616462eef5ee06a0d7f7d5f78b34a8069b80507)
 2017-09-01 21:36:45 +02:00
Jason Lowe
798ff2f52b YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451fb91ef956c926f891283f70f0ada1c)
 2017-09-01 11:11:28 -05:00
John Zhuge
b2f496bdcc HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed62c80397f71949bee41dcd03065755c)
 2017-08-31 21:16:58 -07:00
Junping Du
d665d85682 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 2017-08-31 18:10:33 -07:00
Haibo Chen
2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko
7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc28d48a7909822416478aff7487bfe6)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Junping Du
f9ebcebe74 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9cfb0bcfb69cc0f1594120e7656f031)
(cherry picked from commit 7dd5bb949206fb635d875cd651292b451267a951)
 2017-08-30 13:33:42 -07:00
Xiao Chen
98436a2eb7 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 441c472552eadf66c748ab9493fd02d2c914994a)
 2017-08-30 09:43:27 -07:00
Junping Du
8724e36562 YARN-7037. Optimize data transfer with zero-copy approach for containerlogs REST API in NMWebServices. Contributed by Tao Yang. 2017-08-29 15:21:56 -07:00
Junping Du
79294b5f32 YARN-7083. Log aggregation deletes/renames while file is open. Contributed by Jason Lowe. 2017-08-29 13:18:49 -07:00
Jason Lowe
fe4aff7330 MAPREDUCE-6641. TestTaskAttempt fails in trunk. Contributed by Haibo Chen. 
(cherry picked from commit cd08aa72a6042a837edb17bc1ae98a651d969a43)
 2017-08-29 12:54:32 -05:00
Steve Loughran
37da90b17a
HADOOP-13743. error message in AzureNativeFileSystemStore.connectUsingAnonymousCredentials has too many spaces. 
Contributed by Mingliang Liu and Steve Loughran
 2017-08-29 15:06:00 +01:00
Junping Du
1dfb3850c8 YARN-7076. yarn application -list -appTypes is not working. Contributed by Jian He. 
(cherry picked from commit 312b1fd9da2781da97f8c76fe1262c4d99b9c37f)
(cherry picked from commit f252e1fe2887cc961cf3ad532bfa22d646775fbf)
 2017-08-28 18:31:36 -07:00
Wangda Tan
f40f4ce0e9 YARN-7112. TestAMRMProxy is failing with invalid request. (Jason Lowe via wangda) 
Change-Id: I26d719ad164062b688b49142dc7db40d888b882b
 2017-08-28 15:05:30 -07:00
Sunil G
bc7307788c YARN-7051. Avoid concurrent modification exception in FifoIntraQueuePreemptionPlugin. Contributed by Eric Payne. 
(cherry picked from commit 02599bda04e0ef46f4628b006f2430ad63cac97e)
 2017-08-28 12:43:47 +05:30
Eric Payne
8eb39f775c YARN-7087. NM failed to perform log aggregation due to absent container. Contributed by Jason Lowe. 
(cherry picked from commit e864f81471407a384395fefe1ceb3b66fc7f87f2)
 2017-08-25 15:58:28 -05:00
Jason Lowe
dfdda969ad YARN-7052. RM SchedulingMonitor gives no indication why the spawned thread crashed. Contributed by Eric Payne 
(cherry picked from commit 39a9dc8e4a6e1d13658867ad756878d3dd6352b0)
 2017-08-25 15:33:00 -05:00
Kihwal Lee
b0e951725f HDFS-12299. Race Between update pipeline and DN Re-Registration. Contributed by Brahma Reddy Battula. 
(cherry picked from commit 5a83ffa396089972e23c533eca33c9cba231c45a)
 2017-08-25 15:04:37 -05:00
Arpit Agarwal
b07eef43ae HDFS-12319. Addendum commit for missed changes. 2017-08-25 12:02:59 -07:00
Arpit Agarwal
22fb9bb843 HDFS-12319. DirectoryScanner will throw IllegalStateException when Multiple BP's are present. Contributed by Brahma Reddy Battula. 2017-08-25 11:10:25 -07:00