Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
15,348 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

3813 Commits

Author SHA1 Message Date
Masatake Iwasaki 62fdfaf093 HADOOP-16386. FindBugs warning in branch-2: GlobalStorageStatistics defines non-transient non-serializable instance field map. 
Signed-off-by: Masatake Iwasaki <iwasakims@apache.org>
 2019-07-17 07:50:24 +09:00
stack a7c6d25dc3 HDFS-14483 Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9"" 
This is a revert of a revert, i.e. a reapplication of the HDFS-14483
patch. Original commit was missing the JIRA number.

Revert "Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9""

This reverts commit b26ad8a4c3.
 2019-07-09 08:58:16 -07:00
stack b26ad8a4c3 Revert "Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9" 
This reverts commit 4f09a6bcfd.
 2019-07-09 08:57:57 -07:00
sunlisheng 4f09a6bcfd Backport HDFS-3246,HDFS-14111 ByteBuffer pread interface to branch-2.9 
Signed-off-by: stack <stack@apache.org>
 2019-07-09 08:54:37 -07:00
stack bb7e143d34 HDFS-14585 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9 
Reapply but include JIRA # in the commit message this time.

Revert "Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9""

This reverts commit 1dd54163fc.
 2019-07-01 11:08:05 -07:00
stack 1dd54163fc Revert "Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9" 
This reverts commit bccb0fd846.
 2019-07-01 11:07:43 -07:00
sunlisheng bccb0fd846 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch-2.9 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
Signed-off-by: stack <stack@apache.org>
 2019-07-01 10:59:58 -07:00
Takanobu Asanuma 64102e865f HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles. 
(cherry picked from commit 54f9f75a44)
 2019-06-14 10:21:30 +09:00
Erik Krogen e9eaf21874 HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen. 
(cherry-picked from 76b94c274f)
(cherry-picked from f607efd195)
(cherry-picked from cf3c1b9ffc)
(cherry-picked from d43219c21d)
 2019-06-07 14:23:23 -07:00
Takanobu Asanuma e3d4f6bfed HADOOP-16331. Fix ASF License check in pom.xml. Contributed by Akira Ajisaka. 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:41:21 +09:00
Akira Ajisaka cea1b0000c
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:28:54 +09:00
Alexis Daboville b70fdc64d3
HADOOP-16248. MutableQuantiles leak memory under heavy load. 
Contributed by Alexis Daboville,

(cherry picked from commit 4cb3da6ac7)
 2019-05-17 15:27:47 +01:00
Yuan Gao 18f66a05cf
HADOOP-16205 Backport ABFS driver from trunk to branch 2.0: Fix build and test failures. 
Contributed by Yuan Gao.
 2019-04-25 18:36:14 +01:00
Steve Loughran 94d0de2eab HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch. 
Contributed by Steve Loughran.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt 3bd089ddca HADOOP-15663. ABFS: Simplify configuration. 
Contributed by Da Zhou.
 2019-04-24 21:46:25 +00:00
Thomas Marquardt b537d8c442 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests. 
Contributed by Steve Loughran and Da Zhou.
 2019-04-24 21:46:25 +00:00
Steve Loughran 1b64b0f924 HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit. 
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
 2019-04-24 21:46:25 +00:00
Chen Liang 7935485eec HDFS-14205. Backport HDFS-6440 to branch-2. Contributed by Chao Sun. 2019-03-26 12:12:14 -07:00
Erik Krogen c3973c9ae6 HADOOP-16192. Fix CallQueue backoff bugs: perform backoff when add() is used and update backoff when refreshed. 
(cherry-picked from 8c95cb9d6b)
(cherry-picked from 0de8b55a09)
(cherry-picked from d4fbbc83ad)
(cherry-picked from e172fc62ce)
 2019-03-18 09:00:40 -07:00
Steve Loughran 9b8044d00b
HADOOP-16109. Parquet reading S3AFileSystem causes EOF (#589) 
Nobody gets seek right. No matter how many times they think they have.

Reproducible test from: Dave Christianson
Fixed seek() logic: Steve Loughran

Change-Id: I39b87f3d5daa98f65de2c0a44e348821a4930573
 2019-03-13 15:33:58 +00:00
Tsz Wo Nicholas Sze 5bed24ad5f HADOOP-16126. ipc.Client.stop() may sleep too long to wait for all connections. 
(cherry picked from commit 0edb0c51dc)
 2019-03-01 17:52:53 +09:00
Eric Yang bae607f734 Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit 439422fff9.
 2019-02-21 17:36:59 -05:00
Eric Yang 4a45a9b57e Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit a4356bb7a5.
 2019-02-21 17:36:02 -05:00
Eric Yang da854234a2 Revert "HADOOP-13707. Skip authorization for anonymous user to access Hadoop" 
This reverts commit 7993fb5b81.
 2019-02-21 17:35:19 -05:00
Daryn Sharp 665cad03f3 HADOOP-15813. Enable more reliable SSL connection reuse. Contributed by Daryn Sharp. 
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
(cherry picked from commit a87e458432)
(cherry picked from commit ae8839e6e8)
(cherry picked from commit 704330a616)
(cherry picked from commit 4eccf2a3cc)
 2019-02-20 18:22:48 -08:00
Chen Liang d3b06d179d HADOOP-15481. Emit FairCallQueue stats as metrics. Contributed by Christopher Gregorian. 2019-01-17 13:30:42 -08:00
Chen Liang 1dbe6c9780 HADOOP-16013. DecayRpcScheduler decay thread should run as a daemon. Contributed by Erik Krogen. 2019-01-11 12:57:30 -08:00
Akira Ajisaka 1684301485
HADOOP-16016. TestSSLFactory#testServerWeakCiphers fails on Java 1.8.0_191 or upper 
(cherry picked from commit d4ca907da6)
 2019-01-11 13:05:58 +09:00
Jason Lowe d62bfaf1a4 HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne 2018-12-20 10:02:18 -06:00
shwetayakkali@cloudera.com 500b2a0ca6
HADOOP-16008. Fix typo in CommandsManual.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
 2018-12-18 13:27:14 +09:00
Inigo Goiri 5cef413a8e HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 13:23:10 -08:00
Inigo Goiri a29eb49f85 Revert "HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak." 
This reverts commit b98ffbe3f2.
 2018-12-03 13:22:14 -08:00
Giovanni Matteo Fumarola b98ffbe3f2 HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 12:46:30 -08:00
Steve Loughran a3470c65d8
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple" 2018-11-30 11:33:19 +00:00
Akira Ajisaka 6cae964f6c
Add 2.9.2 release notes and changes documents. 
(cherry picked from commit 1a00b4e325)
 2018-11-20 14:30:09 +09:00
Steve Loughran f23f339a6c
HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility. 
Contributed by Wei-Chiu Chuang.

(cherry picked from commit 52cb766ad0)
 2018-10-30 09:58:59 +09:00
Akira Ajisaka ca699774a2
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 
(cherry picked from commit 8f97d6f2cd)

Conflicts:
	dev-support/docker/Dockerfile
 2018-10-24 14:00:34 +09:00
Jason Lowe e412d8f6cd HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 
(cherry picked from commit 9abda83947)
 2018-10-17 14:45:56 -05:00
Steve Loughran 5da49631b6
HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended. 2018-10-11 12:43:09 +01:00
Vidura Mudalige fa4a111037
HADOOP-15818. Fix deprecated maven-surefire-plugin configuration in hadoop-kms module 
This closes #425

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit a30b1d1824)
 2018-10-09 12:57:42 +09:00
Jason Lowe d7e9ea37bf HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 
(cherry picked from commit f13e231025)
 2018-10-05 09:30:50 -05:00
Yiqun Lin 7b88a57c37 HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 
(cherry picked from commit 81f635f47f)
 2018-10-04 10:47:22 +08:00
Chen Liang 3a6ad9cd3d HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-20 10:00:13 -07:00
Steve Loughran 5b77991548
HADOOP-15748. S3 listing inconsistency can raise NPE in globber. 
Contributed by Steve Loughran.

(cherry picked from commit 646874c326)
 2018-09-20 13:07:07 +01:00
Jason Lowe 88e2ca405a HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 
(cherry picked from commit e71f61ecb8)

Conflicts:
	hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestStringUtils.java
 2018-09-18 16:05:47 -05:00
Inigo Goiri 1b12925d69 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 
(cherry picked from commit 527288ef89)
 2018-09-06 16:50:09 -07:00
Vrushali C f6e4b13d5f HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks 2018-09-05 21:09:16 -07:00
Steve Loughran 5902c0658e HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns. 
Contributed by Kihwal Lee and Lukas Majercak.

(cherry picked from commit 9e96ac666d)
 2018-09-04 11:58:56 -07:00
Konstantin V Shvachko 64681a362d HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 
(cherry picked from commit a4121c71c2)
 2018-08-24 19:00:21 -07:00
Takanobu Asanuma 947b9a27e2 HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao. 
(cherry picked from commit bed8cb6979)
 2018-08-23 21:56:30 +09:00
Xiao Chen 8056472879 HADOOP-15674. Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard Nemeth. 
(cherry picked from commit 8d7c93186e)
 2018-08-17 10:20:29 -07:00
Chen Liang d2ec117435 HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen. 2018-07-31 11:44:08 -07:00
Kihwal Lee cf255dc01f HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 
(cherry picked from commit ccf2db7fc2)
 2018-07-19 11:29:58 -05:00
Hanisha Koneru 4c54ddd1c2 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 
(cherry picked from commit 3fde0f1db5)
 2018-07-09 15:23:33 -05:00
Eric E Payne f951d92d46 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 20:18:55 +00:00
Inigo Goiri 1d457673c8 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit abc3e4bad9)
 2018-06-25 09:52:15 -07:00
Eric E Payne d084c1a4bc YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 
(cherry picked from commit 6432128622)
 2018-06-22 17:53:07 +00:00
Surendra Singh Lilhore 59686179ae HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 
(cherry picked from commit 3905fdb793)
 2018-06-17 12:20:11 +05:30
Inigo Goiri b3f4c27661 HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola. 
(cherry picked from commit aeaf9fec62)
 2018-06-12 14:21:45 -07:00
Rohith Sharma K S 0af3bea05d YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 
(cherry picked from commit 2df73dace0)
 2018-06-12 17:30:02 +05:30
Kihwal Lee 6d7d192e47 Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 
(cherry picked from commit 65e55097da)
 2018-05-29 14:33:31 -05:00
Inigo Goiri 09fbbff695 HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 3c75f8e493)
 2018-05-29 09:12:47 -07:00
Karthik Palanisamy 961c0461fe
HADOOP-15449. Increase default timeout of ZK session to avoid frequent NameNode failover 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 61df174e8b)
 2018-05-28 19:43:03 +09:00
Xiao Chen fe32d65211 HADOOP-15473. Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997. Contributed by Gabor Bota. 
(cherry picked from commit 02322de3f9)
 2018-05-25 09:42:16 -07:00
Miklos Szegedi 81ee9938c5 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-24 15:50:17 -07:00
Arpit Agarwal 50c39db1ad HADOOP-15486. Make NetworkTopology#netLock fair. Contributed by Nanda kumar. 2018-05-23 10:30:48 -07:00
Akira Ajisaka cedc28d4ab
Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka. 2018-05-23 17:20:20 +09:00
Kihwal Lee 5dcd57cbea HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal. 
(cherry picked from commit bcc8e76bad)
 2018-05-22 11:29:54 -05:00
Rushabh Shah 734cbdd6b2 Skip the proxy user check if the ugi has not been initialized. Contributed by Daryn Sharp 
(cherry picked from commit 73e9120ad7)
 2018-05-21 12:53:54 -05:00
Inigo Goiri c4d4ae62cb HDFS-13586. Fsync fails on directories on Windows. Contributed by Lukas Majercack. 
(cherry picked from commit 8783613696)
 2018-05-17 19:28:28 -07:00
Inigo Goiri a5ec44c655 HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 
(cherry picked from commit 53b807a6a8)
 2018-05-17 17:04:59 -07:00
Sammi Chen eabc588d22 Add 2.9.1 release notes and changes documents 2018-05-14 15:36:30 +08:00
Rushabh Shah 1bc12310d8 HADOOP-15441. Log kms url and token service at debug level. Contributed by Gabor Bota 
(cherry picked from commit a5449d3631)
 2018-05-12 12:30:59 -05:00
Inigo Goiri ed592a210a HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit 1da8d4190d)
 2018-05-10 09:42:39 -07:00
Xiao Chen 723afa0978 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah." 
This reverts commit 95cedc5587.

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java
 2018-05-07 14:11:13 -07:00
Robert Kanter 312462532e HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 
(cherry picked from commit 7ab08a9c37)
(cherry picked from commit 74e5b4b437)
 2018-05-07 12:39:57 -07:00
Rushabh Shah d261645177 HADOOP-14841 Kms client should disconnect if unable to get output stream from connection. Contributed by Rushabh S Shah 
(cherry picked from commit 4cdbdce752)
 2018-05-04 15:55:12 -05:00
Brahma Reddy Battula cbcd16e6b7 HADOOP-15180. branch-2 : daemon processes' sysout overwrites 'ulimit -a' in daemon's out file. Contributed by Ranith Sardar 2018-04-18 23:01:50 +05:30
Inigo Goiri 7a2edb588c HADOOP-15394. Backport PowerShell NodeFencer HADOOP-14309 to branch-2. Contributed by Inigo Goiri. 2018-04-17 11:33:09 -07:00
Virajith Jalaparti b9dc0e597e HDFS-13311. RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows. Contributed by Inigo Goiri 2018-04-16 16:08:36 -07:00
Xiao Chen 95cedc5587 HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah. 2018-04-10 15:46:46 -07:00
Jason Lowe 0fb1457d86 HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan 2018-04-10 16:59:31 -05:00
Wei Yan 689a8252aa HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 
(cherry picked from commit 3121e8c293)
 2018-04-05 13:23:07 -07:00
Xiao Chen b8d8c25a18 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 
(cherry picked from commit 59d78a5088)
 2018-04-03 11:14:26 -07:00
Xiao Chen 117a8d6573 HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge. 
(cherry picked from commit f614b44fe583033947cc78d8f04cfe6fb64fc892)
 2018-04-02 23:10:35 -07:00
Xiao Chen 251fc1fb40 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 
(cherry picked from commit 57374c4737)
 2018-04-02 22:55:03 -07:00
Konstantin V Shvachko de419e2777 HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 
(cherry picked from commit acfd764fcc)
 2018-03-30 19:08:12 -07:00
Konstantin V Shvachko c138682e04 HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko. 
(cherry picked from commit 2216bde322)
 2018-03-29 18:03:32 -07:00
Wei-Chiu Chuang 827d2015c4 HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge. 
(cherry picked from commit 3fe41c65d8)
(cherry picked from commit 1176a128d6)
(cherry picked from commit c6939e9330)
 2018-03-27 13:38:06 -07:00
Inigo Goiri d6df90f7b5 HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 
(cherry picked from commit fe224ff972)
 2018-03-20 11:21:44 -07:00
Inigo Goiri 30d9a5db29 HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 
(cherry picked from commit e65ff1c8be)
 2018-03-19 17:21:55 -07:00
Inigo Goiri 36451f2d54 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 
(cherry picked from commit 427fd027a3)
 2018-03-13 20:06:07 -07:00
Chris Douglas 340cd5f1b1 HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 
(cherry picked from commit 45d1b0fdcc)
(cherry picked from commit 09940b1eb3)
(cherry picked from commit a03c8ea61f)
 2018-03-12 20:08:46 -07:00
Ajay Kumar d5d38918d7
HADOOP-15286. Remove unused imports from TestKMSWithZK.java 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 3dc30bc24e)
 2018-03-04 07:20:40 -08:00
Nanda kumar 45ec3e18e3
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 55d04a6db1)
 2018-03-01 19:04:57 +09:00
Arpit Agarwal 0945f0eb23 HADOOP-15266. [branch-2] Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-26 13:45:51 -08:00
Ray Chiang eeb731307f HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179)
 2018-02-22 18:08:52 +05:30
Robert Kanter 14f5797eff HADOOP-15235. Authentication Tokens should use HMAC instead of MAC (rkanter) 
(cherry picked from commit 324e5a7cf2)
 2018-02-20 17:25:12 -08:00
Jason Lowe b1d5e73c1f HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 
(cherry picked from commit 0898ff42e9)
 2018-02-16 15:00:10 -06:00
Kihwal Lee 56b950d777 Revert "HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang)" 
This reverts commit 577a267198.
 2018-02-14 15:20:45 -06:00
Ray Chiang 577a267198 HADOOP-14799. Update nimbus-jose-jwt to 4.41.1. (rchiang) 
(cherry picked from commit 556812c179)
 2018-02-14 00:07:09 +05:30
Kihwal Lee 34466c6b16 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 
(cherry picked from commit 2dd960de98)
 2018-02-06 15:36:32 -06:00
Xiao Chen 0922ea0aaf HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 
(cherry picked from commit b63dcd583f)
(cherry picked from commit 75be15cf02)

Conflicts:
	hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/EncryptionFaultInjector.java
 2018-01-28 22:30:14 -08:00
Steve Loughran 312c571694 HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to hadoop-common 
Contributed by Genmao Yu
 2018-01-27 20:31:21 -08:00
Chris Douglas 9bd439e2c5 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 
(cherry picked from commit 91db424c4360d7556660e8c57ac9a266e6688e01)
(cherry picked from commit 7f35487788)
 2018-01-26 09:15:53 -08:00
Wei Yan cefe4b7bb6 HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 12:58:16 -08:00
Masahiro Tanaka ab482ebe73
HADOOP-15181. Typo in SecureMode.md 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 6e27b20b4d)
 2018-01-22 17:15:31 +09:00
Brahma Reddy Battula 8181367c62 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 
(cherry picked from commit 08332e12d0)
 2018-01-18 10:56:58 +05:30
Miklos Szegedi 679ad2ced0 HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 
(cherry picked from commit 12d0645)
 2018-01-10 11:15:58 -08:00
Jitendra Pandey cb60c05c3a HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 2018-01-01 21:51:48 -08:00
Xiao Chen e20173d557 HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 
(cherry picked from commit 81127616c5)
(cherry picked from commit 8c275c63ff)
 2017-12-29 13:57:39 -08:00
Steve Loughran 1ef906e29e HADOOP-15123. KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG. 
Contributed by Vipin Rathor.
 2017-12-18 18:48:32 +00:00
Jason Lowe 93fdc3d935 HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 2017-12-15 17:17:25 -06:00
Yiqun Lin 22bc62058b HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 
(cherry picked from commit 91c96bdf8e)
 2017-12-14 16:03:34 +08:00
John Zhuge 5e0f4f212d HADOOP-14872. CryptoInputStream should implement unbuffer. Contributed by John Zhuge. 
(cherry picked from commit 6c32ddad30)
(cherry picked from commit fc9e156484)
 2017-12-07 21:26:12 -08:00
Xiao Chen 31436a12d5 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 
(cherry picked from commit 19e0894209)
(cherry picked from commit 1d1401ad38)
 2017-12-07 21:20:33 -08:00
Xiao Chen c9d48352c2 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-12-07 21:20:33 -08:00
Chris Douglas afc606c3aa HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-07 00:43:21 -08:00
Xiao Chen 274bbb193f HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:22 -08:00
Xiao Chen a4674c3e40 HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 
(cherry picked from commit d162252d7a)
 2017-11-23 09:02:23 -08:00
Robert Kanter 8db49df4a2 HADOOP-14982. Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env (pbacsko via rkanter) 
(cherry picked from commit f2efaf013f)
 2017-11-16 11:12:01 -08:00
Xiao Chen ba2eab9122 HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 
(cherry picked from commit b1941b200d)
 2017-11-15 16:45:06 -08:00
Arun Suresh 4e847d63a3 Addendum patch for Configuration fix. (Jason Lowe via asuresh) 2017-11-13 13:49:08 -08:00
Xiao Chen 39fb402da4 HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 
(cherry picked from commit 3c6adda291)
 2017-11-09 21:06:55 -08:00
Arun Suresh 7af9b8ad1e Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:10:21 -08:00
Brahma Reddy Battula 135abcac4b HDFS-12783. [branch-2] dfsrouter should use hdfsScript. Contributed by Brahma Reddy Battula 2017-11-08 01:45:53 +05:30
Arun Suresh 71a55bebfb Preparing for 2.10.0 development 2017-10-31 12:51:14 -07:00
Jason Lowe 347997c959 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 
(cherry picked from commit 2fae63aa60)
 2017-10-31 09:32:45 -05:00
Chris Douglas 9e721775a4 HADOOP-14986. Enforce JDK limitations. 
(cherry picked from commit b50def3697)
(cherry picked from commit 1660a638f0)
 2017-10-27 21:42:42 -07:00
Steve Loughran 3fbcf76324 HADOOP-14553. Add (parallelized) integration tests to hadoop-azure. 
Contributed by Steve Loughran.

(backport JIRA HADOOP-14870)
 2017-10-27 12:55:12 +01:00
Junping Du 3793d2b9cf Add CHANGES.md and RELEASENOTES for 2.8.2 release. 2017-10-26 23:23:39 -07:00
Junping Du 1f2a2ba2be Add several jdiff xml files for 2.8.2 release. 
(cherry picked from commit a25b5aa0cf)
 2017-10-26 23:20:24 -07:00
Varun Saxena 0c8a2c2609 HADOOP-9657. NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports. Contributed by Varun Saxena. 
(cherry picked from commit 67e7673750)
 2017-10-25 03:09:07 +05:30
Naganarasimha 2b04ae918c HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 
(cherry picked from commit 6b8122458e)
 2017-10-23 23:47:33 +05:30
Arpit Agarwal a373094bf9 HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:35 -07:00
Xiao Chen a6370dde90 HADOOP-14944. Add JvmMetrics to KMS. 2017-10-19 22:37:35 -07:00
Wei-Chiu Chuang 9eb33d0e87 HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 
(cherry picked from commit 97c70c7ac6)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2017-10-19 06:06:01 -07:00
Junping Du e7e2b8158d HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 
(cherry picked from commit b016f08f67)
(cherry picked from commit 9433f9eb09)
 2017-10-18 15:07:59 -07:00
Wangda Tan d894f910d0 YARN-6608. Backport all SLS improvements from trunk to branch-2. (Carlo Curino via wangda) 2017-10-17 19:45:09 -07:00
Rohith Sharma K S 4e1af68ae1 YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena. 
(cherry picked from commit 387dd3da87759860dd18bd8aa95c60d962b1e6a9)
 2017-10-17 23:36:39 +05:30
Sangjin Lee e212e562ec YARN-5355. Backported YARN-2928 into our branch-2 feature branch. 2017-10-17 23:36:33 +05:30
Xiao Chen a103b323e6 Revert "HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar." 
This reverts commit 1452ba139e.
 2017-10-17 10:11:38 -07:00
Robert Kanter 8a1ed84c77 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-16 16:51:28 -07:00
Wei-Chiu Chuang 1452ba139e HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 
(cherry picked from commit e906108fc9)
 2017-10-16 16:44:35 -07:00
Xiao Chen 0bddcf10ef HADOOP-14949. TestKMS#testACLs fails intermittently. 
(cherry picked from commit 152317d40ece8abfa76fb5bd29f04878c1b0d3d8)

Conflicts:
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
 2017-10-16 14:20:05 -07:00
Anu Engineer 5933068e70 HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin. 
(cherry picked from commit 075358eb6f)
 2017-10-11 16:15:59 -07:00
Arun Suresh fb8bd4d2d7 HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh) 
(cherry picked from commit b6c2c9058e)
 2017-10-11 15:22:51 -07:00
Steve Loughran ccfd1a4a56 HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB. 
Contributed by Varada Hemeswari.

Cherry-picked from commit ref 3de574413c
 2017-10-11 18:09:13 +01:00
Jason Lowe 63d86ee3a5 HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 
(cherry picked from commit 1123f8f0b6)
 2017-10-10 14:50:15 -05:00
Xiaoyu Yao 65b5e81752 HADOOP-14920. KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. Contributed by Xiaoyu Yao. 
(cherry picked from commit 2b08a1fc64)
 2017-10-06 10:39:37 -07:00