Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,242 Commits 326 Branches 371 Tags
Commit Graph

21242 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Babneet Singh
44fdee351d HDFS-15270. Account for *env == NULL in hdfsThreadDestructor (#1951) 
OpenJ9 JVM properly terminates the thread before hdfsThreadDestructor is
invoked. JNIEnv is a mirror of J9VMThread in OpenJ9. After proper thread
termination, accessing JNIEnv in hdfsThreadDestructor (*env)->GetJavaVM,
yields a SIGSEGV since *env is NULL after thread cleanup is performed.

The main purpose of hdfsThreadDestructor is to invoke
DetachCurrentThread, which performs thread cleanup in OpenJ9. Since
OpenJ9 performs thread cleanup before hdfsThreadDestructor is invoked,
hdfsThreadDestructor should account for *env == NULL and skip
DetachCurrentThread.

Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>
(cherry picked from commit 1996351b0b7be6866eda73223ab6ef1ec78d30cd)
(cherry picked from commit d1e5e393c30ef25212be365651404c1034532338)
 2020-05-11 10:46:10 -07:00
Ayush Saxena
06f43fc628 HDFS-15250. Setting dfs.client.use.datanode.hostname to true can crash the system because of unhandled UnresolvedAddressException. Contributed by Ctest. 2020-05-10 11:44:43 +05:30
Ayush Saxena
c3fffa8451 HDFS-15339. TestHDFSCLI fails for user names with the dot/dash character. Contributed by Yan Xiaole. 2020-05-08 00:57:24 +05:30
Szilard Nemeth
d345994468 YARN-9444. YARN API ResourceUtils's getRequestedResourcesFromConfig doesn't recognize yarn.io/gpu as a valid resource. Contributed by Gergely Pollak 
(cherry picked from commit 52e9ee39a12ce91b3a545603dcf1103518ad2920)
 2020-05-07 18:17:06 +00:00
Masatake Iwasaki
89696b66e7 HADOOP-17025. Fix invalid metastore configuration in S3GuardTool tests. (#1994) 
(cherry picked from commit 99840aaba662e7bb6187206b74e35132102a3b38)

 Conflicts:
	hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/s3guard/AbstractS3GuardToolTestBase.java
 2020-05-07 12:11:37 +09:00
Ahmed Hussein
7740b88ee9 YARN-8959. TestContainerResizing fails randomly (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit 92e3ebb40199aec0890b868b8d6bf2d7fe90abbf)
 2020-05-06 12:32:36 -05:00
Inigo Goiri
53f6d26500 HDFS-14599. HDFS-12487 breaks test TestDiskBalancer.testDiskBalancerWithFedClusterWithOneNameServiceEmpty. Contributed by He Xiaoqiao. 
(cherry picked from commit 4a212242d99071d028b40c2f5d40afb7f337e471)
 2020-05-04 18:04:07 -07:00
Konstantin V Shvachko
e95a4ddbb1 HDFS-15323. StandbyNode fails transition to active due to insufficient transaction tailing. Contributed by Konstantin V Shvachko. 
(cherry picked from commit ebb878bab991c242b5089a18881aa10abf318ea0)
 2020-05-04 10:47:07 -07:00
Ahmed Hussein
b23a585cb1 YARN-10256. Refactor TestContainerSchedulerQueuing.testContainerUpdateExecTypeGuaranteedToOpportunistic (Ahmed Hussein via jeagles) 
Signed-off-by: Jonathan Eagles <jeagles@gmail.com>
(cherry picked from commit f5081a9a5d51fe781da97f07ac4dca8dd49c98e2)
 2020-05-04 10:49:45 -05:00
Ayush Saxena
d451c9e2a4 Erasure Coding: metrics xmitsInProgress become to negative. Contributed by maobaolong and Toshihiko Uchida. 2020-05-03 19:17:44 +05:30
Eric Yang
9db6e96db3
SPNEGO TLS verification 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit ba66f3b454a5f6ea84f2cf7ac0082c555e2954a7)
 2020-05-02 21:51:41 +09:00
Ayush Saxena
50ba1949ee HADOOP-16886. Add hadoop.http.idle_timeout.ms to core-default.xml. Contributed by Lisheng Sun. 2020-05-01 14:27:24 +05:30
Szilard Nemeth
1dabbd5006 YARN-10194. YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Contributed by Prabhu Joseph 2020-04-28 17:52:14 +02:00
Akira Ajisaka
5154f6eef4
HDFS-15286. Concat on a same file deleting the file. Contributed by hemanthboyina. 
(cherry picked from commit 5e0eda5d5f696aba7fc209874d232baf2a50d547)
 2020-04-28 06:19:17 +09:00
Siyao Meng
9aef65cdb2
HADOOP-17014. Upgrade jackson-databind to 2.9.10.4 (#1981) 2020-04-27 13:34:33 -07:00
Eric E Payne
6bf255b4ea MAPREDUCE-7277. IndexCache totalMemoryUsed differs from cache contents. Contributed by Jon Eagles (jeagles). 
(cherry picked from commit e2322e1117a2a3435aabecd49de0ad3c8d2f64b9)
 2020-04-27 19:56:48 +00:00
Szilard Nemeth
f445487d50 YARN-10189. Code cleanup in LeveldbRMStateStore. Contributed by Benjamin Teke 2020-04-27 09:59:13 +02:00
Mingliang Liu
57c1e7ad9b
HDFS-15297. TestNNHandlesBlockReportPerStorage::blockReport_02 fails intermittently. Contributed by Ayush Saxena 2020-04-25 15:44:27 -07:00
Dhiraj
6225fce50d
HDFS-15281. Make sure ZKFC uses dfs.namenode.rpc-address to bind to host address (#1964) 
Contributed by Dhiraj Hegde.

Signed-off-by: Mingliang Liu <liuml07@apache.org>
Signed-off-by: Inigo Goiri <inigoiri@apache.org>
 2020-04-25 13:08:32 -07:00
Wei-Chiu Chuang
9e3b4c9ebd HDFS-15295. AvailableSpaceBlockPlacementPolicy should use chooseRandomWithStorageTypeTwoTrial() for better performance. Contributed by Jinglun. 
(cherry picked from commit 453771f170a062265b7687b31b2a3735827958fb)
(cherry picked from commit d6c48f81009eaf9462cb6ad1ec1cbe609b223cb2)
 2020-04-24 11:54:20 -07:00
Steve Loughran
204d54005a HADOOP-16117. Update AWS SDK to 1.11.563. 
Contributed by Steve Loughran.

Change-Id: I7c46ed2a6378e1370f567acf4cdcfeb93e43fa13
 2020-04-24 10:46:24 +01:00
Szilard Nemeth
0dbb02a76c YARN-9999. TestFSSchedulerConfigurationStore: Extend from ConfigurationStoreBaseTest, general code cleanup. Contributed by Benjamin Teke 2020-04-24 11:31:33 +02:00
Szilard Nemeth
3b67dc24aa YARN-9998. Code cleanup in LeveldbConfigurationStore. Contributed by Benjamin Teke 2020-04-24 11:15:53 +02:00
Masatake Iwasaki
5ebfb7203a HADOOP-16647. Support OpenSSL 1.1.1 LTS. Contributed by Rakesh Radhakrishnan. 
(cherry picked from commit 8f8be6b92a32982c2fd5be4647ebe1610a533dc3)
 2020-04-24 15:38:11 +09:00
Masatake Iwasaki
83c4f8b9a0 HADOOP-16739. Fix native build failure of hadoop-pipes on CentOS 8. 2020-04-24 15:38:11 +09:00
Akira Ajisaka
7b036c512f
YARN-10223. Remove jersey-test-framework-core dependency from yarn-server-common. (#1939) 
(cherry picked from commit 9827ff2961e249e8be9023f39d91e1c0fbd23d61)
 2020-04-24 10:47:36 +09:00
Wei-Chiu Chuang
48f1c8ffb6 Revert "YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja" 
This reverts commit a2067aafa941344c2c1713ba835a107f7231c2cc.
 2020-04-23 12:37:21 -07:00
Szilard Nemeth
c81844d8a5 YARN-9996. Code cleanup in QueueAdminConfigurationMutationACLPolicy. Contributed by Siddharth Ahuja 2020-04-23 14:57:18 +02:00
Szilard Nemeth
764fa92c9f YARN-9997. Code cleanup in ZKConfigurationStore. Contributed by Andras Gyori 2020-04-23 14:51:07 +02:00
Mingliang Liu
e154d9d9f7
HDFS-15276. Concat on INodeRefernce fails with illegal state exception. Contributed by hemanthboyina 2020-04-22 13:56:15 -07:00
Mingliang Liu
36bcbce2cd
HADOOP-17001. The suffix name of the unified compression class. Contributed by bianqi 2020-04-22 13:07:18 -07:00
Inigo Goiri
7d64d44515 HDFS-14004. TestLeaseRecovery2#testCloseWhileRecoverLease fails intermittently in trunk. Contributed by Ayush Saxena. 
(cherry picked from commit efdfe679d64ce9de4ba6aaf2afa34e180f68d969)
 2020-04-21 09:50:20 -07:00
Akira Ajisaka
ce87784771
HADOOP-16944. Use Yetus 0.12.0 in GitHub PR (#1917) 
(cherry picked from commit 5576915236aba172cb5ab49b43111661590058af)
 2020-04-21 21:12:31 +09:00
Szilard Nemeth
73cb3d3cb3 YARN-10001. Add explanation of unimplemented methods in InMemoryConfigurationStore. Contributed by Siddharth Ahuja 2020-04-18 09:38:22 +02:00
Jonathan Hung
d1af4e0fae YARN-9954. Configurable max application tags and max tag length. Contributed by Bilwa S T 
(cherry picked from commit 49ae9b2137dddaff1b579586e593a24da21a9e7d)
 2020-04-17 10:36:16 -07:00
Stephen O'Donnell
041c93a26c HDFS-15283. Cache pool MAXTTL is not persisted and restored on cluster restart. Contributed by Stephen O'Donnell. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit 3481895f8a9ea9f6e217a0ba158c48da89b3faf2)
(cherry picked from commit aaad947c740c91c36139f3f0569ae78b53bca682)
 2020-04-16 20:19:14 -07:00
Masatake Iwasaki
9d66a0e28e HADOOP-16982. Update Netty to 4.1.48.Final. Contributed by Lisheng Sun. 2020-04-16 11:08:20 +09:00
Szilard Nemeth
2f01a91428 YARN-10002. Code cleanup and improvements in ConfigurationStoreBaseTest. Contributed by Benjamin Teke 2020-04-15 08:24:15 +02:00
Szilard Nemeth
58e559b5ac YARN-9354. Resources should be created with ResourceTypesTestHelper instead of TestUtils. Contributed by Andras Gyori 2020-04-15 08:16:15 +02:00
Szilard Nemeth
a7eb95f114 YARN-9995. Code cleanup in TestSchedConfCLI. Contributed by Bilwa S T. 2020-04-15 08:05:07 +02:00
Szilard Nemeth
9af7d905f9 YARN-5277. When localizers fail due to resource timestamps being out, provide more diagnostics. Contributed by Siddharth Ahuja 2020-04-15 07:55:49 +02:00
Eric E Payne
9c53616190 MAPREDUCE-7272. TaskAttemptListenerImpl excessive log messages. Contributed by Ahmed Hussein (ahussein) 
(cherry picked from commit 11d17417ceba0f1a2944e0c8b4286515c4883889)
 2020-04-13 19:07:24 +00:00
Jonathan Hung
54599b177c YARN-10212. Create separate configuration for max global AM attempts. Contributed by Bilwa S T 
(cherry picked from commit 57659422abbf6d9bf52e6e27fca775254bb77a56)
(cherry picked from commit e3a52804b03d646f15048c078f8c5292d5cbecfa)
 2020-04-09 10:37:36 -07:00
Szilard Nemeth
d2853d1bb0 YARN-10003. YarnConfigurationStore#checkVersion throws exception that belongs to RMStateStore. Contributed by Benjamin Teke 2020-04-09 17:40:26 +02:00
Ayush Saxena
4c63a81b00 HADOOP-16952. Add .diff to gitignore. Contributed by Ayush Saxena. 2020-04-09 19:26:46 +05:30
Szilard Nemeth
8daa8556d6 MAPREDUCE-7266. historyContext doesn't need to be a class attribute inside JobHistoryServer. Contributed by Siddharth Ahuja 
(cherry picked from commit e2c87b79170f18b316957cb935fbc254d448cd08)
 2020-04-09 12:42:21 +02:00
Szilard Nemeth
9a66477e4d YARN-10207. CLOSE_WAIT socket connection leaks during rendering of (corrupted) aggregated logs on the JobHistoryServer Web UI. Contributed by Siddharth Ahuja 2020-04-09 12:22:12 +02:00
Akira Ajisaka
2a2c3a4094
HADOOP-14836. Upgrade maven-clean-plugin to 3.1.0 (#1933) 
(cherry picked from commit e53d472bb07fc25d6cc5a13af1bd5ffdbf671164)
 2020-04-09 01:49:28 +09:00
Wilfred Spiegelenburg
a2067aafa9
YARN-10063. Add container-executor arguments --http/--https to usage. Contributed by Siddharth Ahuja 
Conflicts:
	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/main.c

(cherry picked from commit 2214005c0f11955b2c50c4d2d4bd14947dd797ba)
 2020-04-08 13:12:31 +10:00
Akira Ajisaka
11aff6c269
HADOOP-16949. pylint fails in the build environment (#1926) 
(cherry picked from commit e6455cc864d932469610e86644162fa324589067)

Conflicts:
	dev-support/docker/Dockerfile_aarch64
 2020-04-04 16:55:10 +09:00