Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,410 Commits 326 Branches 371 Tags
Commit Graph

12410 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Sunil G
b41a702751 YARN-7245. Max AM Resource column in Active Users Info section of Capacity Scheduler UI page should be updated per-user. Contributed by Eric Payne. 
(cherry picked from commit 8137254a1f4367974bf1883f1982e31ab2865107)
 2017-10-09 14:58:42 +05:30
Xiao Chen
99b7ced3bf HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)

Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/LoadBalancingKMSClientProvider.java
 2017-10-05 19:47:26 -07:00
Naganarasimha
d4cd8ccb5d YARN-7285. ContainerExecutor always launches with priorities due to yarn-default property. Contributed by Jason Lowe. 
(cherry picked from commit d0046bb5c6919c9620ffa5330eb37b39484151ad)
 2017-10-06 00:31:24 +05:30
Sidharta S
935ef1c24e YARN-7226. Whitelisted variables do not support delayed variable expansion. Contributed by Jason Lowe 2017-10-03 18:44:36 -07:00
Jason Lowe
3629c95c51 YARN-5444. Fix failing unit tests in TestLinuxContainerExecutorWithMocks. Contributed by Yufei Gu. 
(cherry picked from commit 95694b70cd38dfc591dc4f0e9413809cf3c91dbe)
 2017-10-03 14:44:04 -05:00
Eric Payne
4540cbb3a0 YARN-7084. TestSchedulingMonitor#testRMStarts fails sporadically. Contributed by Jason Lowe. 
(cherry picked from commit 7f6118f918e0fbe81cb64e8b9f6959b8d66679f6)
 2017-09-29 10:56:56 -05:00
Jason Lowe
949525b81a HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru 
(cherry picked from commit 6f789fe05766a61b12ca10df3f26ee354eac84aa)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/loadGenerator/LoadGenerator.java
 2017-09-28 16:45:02 -05:00
Jason Lowe
ba5656d5e0 HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure durations. Contributed by Bharat Viswanadham 
(cherry picked from commit ac05a51bbb2a3fad4e85f9334a3408571967900a)
 2017-09-28 16:27:08 -05:00
Yiqun Lin
fb5de93c46 HDFS-12495. TestPendingInvalidateBlock#testPendingDeleteUnknownBlocks fails intermittently. Contributed by Eric Badger. 
(cherry picked from commit e9b790db9de7c3fbe451bcdc3fd928a27a0d8b4e)
 2017-09-27 10:51:00 +08:00
Daryn Sharp
438291787a [HDFS-12386] Add fsserver defaults call to WebhdfsFileSystem. (Rushabh Shah via daryn) 2017-09-26 15:25:12 -05:00
Jason Lowe
86e092029d HADOOP-14891. Remove references to Guava Objects.toStringHelper. Contributed by Jonathan Eagles 
(cherry picked from commit 0aac5af6e1a2b265cc8faa99b386276ed14faeda)
 2017-09-26 10:13:24 -05:00
Eric Payne
f794adde3b YARN-7249. Fix CapacityScheduler NPE issue when a container preempted while the node is being removed. Contributed by Wangda Tan. 2017-09-26 09:07:04 -05:00
Akira Ajisaka
75aed26511
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari. 
(cherry picked from commit d08b8c801a908b4242e7b21a54f3b1e4072f1eae)
(cherry picked from commit b017af23911c08d1f7e928007ebd42dbdbd82507)
 2017-09-26 16:48:28 +09:00
Akira Ajisaka
ea7e655897
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit 3a10367a17b99e2c3035ec54048f94f334d93227)
(cherry picked from commit 6004fb46761810ef87367c5207fafcdbb6507c39)
 2017-09-26 01:48:54 +09:00
Jason Lowe
44b51b0047 Revert "YARN-6570. No logs were found for running application, running" 
This reverts commit e103acc8f495135e52ec1e700dc33747e735967e.
 2017-09-25 11:13:50 -05:00
Anu Engineer
3ecc1431fc
HDFS-12531. Fix conflict in the javadoc of UnderReplicatedBlocks.java in branch-2. Contributed by Bharat Viswanadham. 
(cherry picked from commit de5744c9035308654c7096bf5bd04e769a3a19e3)
 2017-09-25 13:12:27 +09:00
Xuan
e103acc8f4 YARN-6570. No logs were found for running application, running 
container. (Contributed by Junping Du)
 2017-09-22 14:48:12 -07:00
Sunil G
1132f48253 YARN-7116. CapacityScheduler Web UI: Queue's AM usage is always show on per-user's AM usage. Contributed by Wangda Tan. 
(cherry picked from commit f9e0cc8cdca387bf2811c07beccae4af07362952)
 2017-09-22 11:52:30 -05:00
Akira Ajisaka
dade7396a2
MAPREDUCE-6966. DistSum should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari. 
(cherry picked from commit c71d13796039815350d3c73c51f9fe48531af0fa)
(cherry picked from commit d80e2f2eae1d584b99da305d715a3f5e958114e9)
 2017-09-22 15:10:42 +09:00
Junping Du
1b3503e4b9 YARN-7034. DefaultLinuxContainerRuntime and DockerLinuxContainerRuntime sends client environment variables to container-executor. Contributed by Miklos Szegedi. 2017-09-21 14:18:50 -07:00
Jason Lowe
b0b10fc94a YARN-5195. RM intermittently crashed with NPE while handling APP_ATTEMPT_REMOVED event when async-scheduling enabled in CapacityScheduler. Contributed by sandflee and Jonathan Hung 2017-09-21 15:17:03 -05:00
Kihwal Lee
c0bb242360 HDFS-12371. BlockVerificationFailures and BlocksVerified show up as 0 in Datanode JMX. Contributed by Hanisha Koneru. 
(cherry picked from commit 6bf921a5c3152a307b5c0903056d73ce07775a08)
 2017-09-21 09:00:26 -05:00
Akira Ajisaka
4ac3fc10c2
HDFS-12526. FSDirectory should use Time.monotonicNow for durations. Contributed by Bharat Viswanadham. 
(cherry picked from commit 50849ec9eb9bad90586a95d2b2380ee6e8724d6b)
(cherry picked from commit 3527327f349264ae62f3673ab368505a16cc0a34)
 2017-09-21 19:29:18 +09:00
Sangjin Lee
6d6758f751 YARN-6771. Use classloader inside configuration class to make new 
classes. Contributed by Jongyoul Lee.

(cherry picked from commit 8b336632acad10e45d029596c5e3196e1857d891)
 2017-09-20 22:50:55 -07:00
Ming Ma
c54310a638 HDFS-12473. Change hosts JSON file format. 2017-09-20 09:21:32 -07:00
Brahma Reddy Battula
a81167e2ec HDFS-11799. Introduce a config to allow setting up write pipeline with fewer nodes than replication factor. Contributed by Brahma Reddy Battula 
(cherry picked from commit fda1221c55101d97ac62e1ee4e3ddf9a915d5363)
 2017-09-19 11:34:52 +05:30
Eric Payne
67ada5d485 MAPREDUCE-6960. Shuffle Handler prints disk error stack traces for every read failure. 
(cherry picked from commit 595d478408104bdfe1f08efd79930e18862fafbb)
 2017-09-19 11:00:20 -05:00
Jason Lowe
03a380daec MAPREDUCE-6958. Shuffle audit logger should log size of shuffle transfer. Contributed by Jason Lowe 2017-09-19 10:27:04 -05:00
Daniel Templeton
b1e1f7546c MAPREDUCE-6808. Log map attempts as part of shuffle handler audit log (Contributed by Gergő Pásztor via Daniel Templeton) 
(cherry picked from commit a7463b6c88f698950a2f326030261001aa51b35e)
 2017-09-19 09:08:00 -05:00
Erik Krogen
b8d86bb535 HDFS-12323. NameNode terminates after full GC thinking QJM unresponsive if full GC is much longer than timeout. Contributed by Erik Krogen. 
(cherry picked from commit 90894c7262df0243e795b675f3ac9f7b322ccd11)
 2017-09-15 14:06:14 -07:00
Nathan Roberts
11b325d17d HDFS-12441. Supress UnresolvedPathException in namenode log. Contributed by Kihwal Lee. 
(cherry picked from commit de197fc1562c4d1c39d24738c7cf8a8869dcec9c)
 2017-09-15 14:54:37 -05:00
Kihwal Lee
9e9e36fa70 HDFS-12424. Datatable sorting on the Datanode Information page in the Namenode UI is broken. Contributed by Shawna Martell. 2017-09-15 13:16:43 -05:00
John Zhuge
584cd134e8 HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham. 
(cherry picked from commit 7ee02d1065268511ec10a8b8852d0d6327419b9c)
 2017-09-14 11:35:25 -07:00
Jason Lowe
e63a2d4d33 MAPREDUCE-6957. shuffle hangs after a node manager connection timeout. Contributed by Jooseong Kim 
(cherry picked from commit 4d98936eec1b5d196053426c70d455cf8f83f84f)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
 2017-09-13 17:24:19 -05:00
Eric Payne
a171cb3d81 YARN-4727. Unable to override the $HADOOP_CONF_DIR env variable for container. Contributed by Jason Lowe. 2017-09-13 15:19:41 -05:00
Arpit Agarwal
2cddf1cc0f HADOOP-14867. Update HDFS Federation setup document, for incorrect property name for secondary name node http address. Contributed by Bharat Viswanadham. 2017-09-13 12:55:30 -07:00
Xiao Chen
9d4c4b2f6e Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah." 
This reverts commit 8b7d1df766002ae033d5fca8017cfe7141803c58.
 2017-09-12 19:53:26 -07:00
John Zhuge
6afbe47b03 HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria. 
(cherry picked from commit a4661850c1e0794baf493a468191e12681d68ab4)
 2017-09-08 14:29:05 -07:00
Junping Du
c2d96dd50e HADOOP-14842. Hadoop 2.8.2 release build process get stuck due to java issue. Contributed by Junping Du. 
(cherry picked from commit d0a0f24abc42957c885d5076f8c8e7945e074ba8)
 2017-09-08 13:14:08 -07:00
Xiao Chen
69d7b1ae50 HDFS-12369. Edit log corruption due to hard lease recovery of not-closed file which has snapshots. 
(cherry picked from commit 3964b131f3f0131a663377685950d1b7bd3fe63d)
(cherry picked from commit 8b7d597d3a8d728aa05302ccd769af84a1c9ec0f)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
 2017-09-07 16:30:53 -07:00
Jason Lowe
c6db50503a HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen 
(cherry picked from commit dd814946f68d52a9b1627ac4dd61f9ab093423ae)
 2017-09-06 16:14:24 -05:00
Jason Lowe
2071f8338c YARN-7164. TestAMRMClientOnRMRestart fails sporadically with bind address in use. Contributed by Jason Lowe 
(cherry picked from commit 704267cb4909bf9968dd30067617f74543e07ffb)
 2017-09-06 15:26:13 -05:00
Oleg Danilov
867a4b7d78
YARN-7023. Incorrect ReservationId.compareTo() implementation. 
This closes #262

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit c5281a85e108c7cab6e84dfe351843483d720b9b)
(cherry picked from commit b616462eef5ee06a0d7f7d5f78b34a8069b80507)
 2017-09-01 21:36:45 +02:00
Jason Lowe
798ff2f52b YARN-7120. CapacitySchedulerPage NPE in "Aggregate scheduler counts" section. Contributed by Eric Payne 
(cherry picked from commit 99a7f5d451fb91ef956c926f891283f70f0ada1c)
 2017-09-01 11:11:28 -05:00
John Zhuge
b2f496bdcc HADOOP-14824. Update ADLS SDK to 2.2.2 for MSI fix. Contributed by Atul Sikaria. 
(cherry picked from commit 7ecc6dbed62c80397f71949bee41dcd03065755c)
 2017-08-31 21:16:58 -07:00
Junping Du
d665d85682 YARN-5547. NMLeveldbStateStore should be more tolerant of unknown keys. Contributed by Ajith S. 2017-08-31 18:10:33 -07:00
Haibo Chen
2fce0fa8d7 MAPREDUCE-6937. Backport MAPREDUCE-6870 to branch-2 while preserving compatibility. (Peter Bacsko via Haibo Chen) 2017-08-31 09:21:09 -07:00
Konstantin V Shvachko
7ab03ffbf0 MAPREDUCE-6931. Remove TestDFSIO "Total Throughput" calculation. Contributed by Dennis Huo. 
(cherry picked from commit 3e0e2033cc28d48a7909822416478aff7487bfe6)

Conflicts:
	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/fs/TestDFSIO.java
 2017-08-30 14:46:39 -07:00
Junping Du
f9ebcebe74 HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du. 
(cherry picked from commit 41480233a9cfb0bcfb69cc0f1594120e7656f031)
(cherry picked from commit 7dd5bb949206fb635d875cd651292b451267a951)
 2017-08-30 13:33:42 -07:00
Xiao Chen
98436a2eb7 HDFS-12336. Listing encryption zones still fails when deleted EZ is not a direct child of snapshottable directory. Contributed by Wellington Chevreuil. 
(cherry picked from commit 441c472552eadf66c748ab9493fd02d2c914994a)
 2017-08-30 09:43:27 -07:00