Commit Graph

5378 Commits

Author SHA1 Message Date
Chao Sun fa8550337d HDFS-14149. [SBN read] Fix annotations on new interfaces/classes for SBN reads. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Erik Krogen ef3e1929dd HDFS-14146. [SBN read] Handle exceptions from and prevent handler threads from blocking within internalQueueCall. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chen Liang cae279610b HDFS-14138. [SBN read] Description errors in the comparison logic of transaction ID. Contributed by xiangheng. 2018-12-24 09:34:01 -08:00
Erik Krogen b73fb70f97 HDFS-13873. [SBN read] ObserverNode should reject read requests when it is too far behind. Contributed by Konstantin Shvachko. 2018-12-24 09:34:01 -08:00
Chao Sun cbc2f932eb HDFS-14067. [SBN read] Allow manual failover between standby and observer. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chao Sun 8c49135078 HDFS-13924. [SBN read] Handle BlockMissingException when reading from observer. Contributed by Chao Sun. 2018-12-24 09:34:01 -08:00
Chen Liang a65bb97f5d HDFS-13791. Limit logging frequency of edit tail related statements. Contributed by Erik Krogen. 2018-12-24 09:34:00 -08:00
Chen Liang d2e85b0b6d HDFS-13880. Add mechanism to allow certain RPC calls to bypass sync. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Chen Liang 66ec3a67e8 HDFS-13767. Add msync server implementation. Contributed by Chen Liang. 2018-12-24 09:34:00 -08:00
Plamen Jeliazkov e27708c2da HDFS-13665. [SBN read] Move RPC response serialization into Server.doResponse(). Contributed by Plamen Jeliazkov. 2018-12-24 09:34:00 -08:00
Chao Sun 64b7cf59bd HDFS-12976. [SBN read] Introduce ObserverReadProxyProvider. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Erik Krogen 151c8ddbe4 HDFS-13608. [SBN read] Edit Tail Fast Path Part 2: Add ability for JournalNode to serve edits via RPC. Contributed by Erik Krogen. 2018-12-24 09:33:59 -08:00
Plamen Jeliazkov e880660a20 HDFS-13399. [SBN read] Make Client field AlignmentContext non-static. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Erik Krogen 091ad974cd HDFS-13286. [SBN read] Add haadmin commands to transition between standby and observer. Contributed by Chao Sun. 2018-12-24 09:33:59 -08:00
Erik Krogen 9bf0696c73 HDFS-13331. [SBN read] Add lastSeenStateId to RpcRequestHeader. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Plamen Jeliazkov c9d73437e8 HDFS-12977. [SBN read] Add stateId to RPC headers. Contributed by Plamen Jeliazkov. 2018-12-24 09:33:59 -08:00
Yiqun Lin feb2664ac4 HDFS-13946. Log longest FSN write/read lock held stack trace. 2018-12-22 23:09:59 +08:00
Surendra Singh Lilhore 763e96ea2b HDFS-14166. Ls with -e option not giving the result in proper format. Contributed by Shubham Dewan. 2018-12-22 12:22:24 +05:30
Bharat Viswanadham 2499435d9d HADOOP-16014. Fix test, checkstyle and javadoc issues in TestKerberosAuthenticationHandler. Contributed by Dinesh Chitlangia. 2018-12-21 13:30:48 -08:00
Jason Lowe 3961690037 HADOOP-15973. Configuration: Included properties are not cached if resource is a stream. Contributed by Eric Payne 2018-12-20 09:29:48 -06:00
Takanobu Asanuma d963575918 HDFS-13661. Ls command with e option fails when the filesystem is not HDFS. 2018-12-20 10:03:33 +09:00
Surendra Singh Lilhore 788e7473a4 HDFS-14096. [SPS] : Add Support for Storage Policy Satisfier in ViewFs. Contributed by Ayush Saxena. 2018-12-17 11:24:57 +05:30
Gabor Bota 1ea29b7385
HADOOP-16000. Remove TLSv1 and SSLv2Hello from the default value of hadoop.ssl.enabled.protocols
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-12-15 10:53:52 +09:00
Giovanni Matteo Fumarola 76efeacd5f HADOOP-15995. Add ldap.bind.password.alias in LdapGroupsMapping to distinguish aliases when using multiple providers through CompositeGroupsMapping. Contributed by Lukas Majercak. 2018-12-12 11:10:24 -08:00
Steve Loughran 202926ac33
HADOOP-15808. Harden Token service loader use.
Contributed by Steve Loughran.
2018-12-11 17:33:49 +00:00
Giovanni Matteo Fumarola aa89492f29 Revert "HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr."
This reverts commit fb9deed41d.
2018-12-04 12:57:28 -08:00
Steve Loughran db2d8b01c6
HADOOP-15966. Hadoop Kerberos broken on macos as java.security.krb5.realm is reset.
Contributed by Steve Loughran
2018-12-04 15:35:43 +00:00
Giovanni Matteo Fumarola fb9deed41d HADOOP-15852. Refactor QuotaUsage. Contributed by Beluga Behr. 2018-12-03 13:25:02 -08:00
Giovanni Matteo Fumarola c9a3aa64dc HADOOP-15950. Failover for LdapGroupsMapping. Contributed by Lukas Majercak. 2018-12-03 12:10:05 -08:00
Eric Yang 38ea3814bd HADOOP-15922. Revert patch 004. 2018-11-30 12:49:46 -05:00
Akira Ajisaka c9bfca217f
MAPREDUCE-7165. mapred-site.xml is misformatted in single node setup document. Contributed by Zhaohui Xin. 2018-11-30 13:13:23 +09:00
Steve Loughran d0edd37269
HADOOP-15959. Revert "HADOOP-12751. While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple"
This reverts commit 829a2e4d27.
2018-11-29 17:52:11 +00:00
Ewan Higgs c1d24f8483
HDFS-13713. Add specification of Multipart Upload API to FS specification, with contract tests.
Contributed by Ewan Higgs and Steve Loughran.
2018-11-29 15:12:17 +00:00
Eric Yang a9d96948ed HADOOP-15922. Fixed doAsUser decoding for DelegationTokenAuthenticationFilter.
Contributed by He Xiaoqiao
2018-11-26 13:49:19 -05:00
Vinayakumar B 23b441c225 HADOOP-15948. Inconsistency in get and put syntax if filename/dirname contains space. Contributed by Ayush Saxena. 2018-11-26 18:11:01 +05:30
Márton Elek 6934a65402 HADOOP-15358. SFTPConnectionPool connections leakage. Contributed by Mikhail Pryakhin. 2018-11-23 09:48:26 +01:00
Akira Ajisaka 1a00b4e325
Add 2.9.2 release notes and changes documents. 2018-11-20 14:25:18 +09:00
Inigo Goiri 460a94a10f HDFS-14045. Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN. Contributed by Jiandan Yang. 2018-11-15 10:58:57 -08:00
Surendra Singh Lilhore 75291e6d53 HADOOP-15869. BlockDecompressorStream#decompress should not return -1 in case of IOException. Contributed by Surendra Singh Lilhore 2018-11-13 20:22:58 +05:30
Steve Loughran 3c9d97b8f7
HADOOP-15110. Gauges are getting logged in exceptions from AutoRenewalThreadForUserCreds.
Contributed by LiXin Ge.
2018-11-12 12:58:05 +00:00
Giovanni Matteo Fumarola 89b49167a5 HADOOP-15903. Allow HttpServer2 to discover resources in /static when symlinks are used. Contributed by Inigo Goiri. 2018-11-08 14:52:24 -08:00
Jason Lowe 0b6625a973 MAPREDUCE-7148. Fast fail jobs when exceeds dfs quota limitation. Contributed by Wang Yan 2018-11-07 08:20:49 -06:00
Jason Lowe ab6aa4c726 YARN-8865. RMStateStore contains large number of expired RMDelegationToken. Contributed by Wilfred Spiegelenburg 2018-11-06 08:40:59 -06:00
Lars Francke cb8d679c95
HADOOP-15687. Credentials class should allow access to aliases.
Author:    Lars Francke <lars.francke@gmail.com>
2018-11-03 16:21:29 +00:00
Giovanni Matteo Fumarola 44e37b4fd9 HADOOP-15885. Add base64 (urlString) support to DTUtil. Contributed by Inigo Goiri. 2018-11-02 10:54:12 -07:00
Akira Ajisaka f747f5b06c
HADOOP-15886. Fix findbugs warnings in RegistryDNS.java. 2018-10-31 10:01:31 +09:00
Steve Loughran 62d98ca92a
HADOOP-15855. Review hadoop credential doc, including object store details.
Contributed by Steve Loughran.
2018-10-30 15:58:04 +00:00
Wei-Chiu Chuang 63e7134df6 Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao."
This reverts commit fb2b72e6fc.
2018-10-28 17:04:59 -07:00
Hrishikesh Gadre bfb9adc2b9 HADOOP-9567. Provide auto-renewal for keytab based logins. Contributed by Hrishikesh Gadre, Gary Helmling and Harsh J.
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2018-10-27 08:59:47 -07:00
Wei-Chiu Chuang fb2b72e6fc HADOOP-15864. Job submitter / executor fail when SBN domain name can not resolved. Contributed by He Xiaoqiao. 2018-10-25 09:33:31 -07:00
Akira Ajisaka 8f97d6f2cd
HADOOP-15822. zstd compressor can fail with a small output buffer. Contributed by Jason Lowe. 2018-10-24 13:49:28 +09:00
Haibo Chen 823bb5dda8 MAPREDUCE-4669. MRAM web UI does not work with HTTPS. (Contributed by Robert Kanter) 2018-10-23 15:28:13 -07:00
Chen Liang 635786a511 HDFS-13566. Add configurable additional RPC listener to NameNode. Contributed by Chen Liang. 2018-10-23 14:53:45 -07:00
Inigo Goiri dd268a64d3 Revert "HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR."
This reverts commit 00254d7b8c.
2018-10-23 09:23:03 -07:00
Steve Loughran 52cb766ad0 HADOOP-15866. Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility.
Contributed by Wei-Chiu Chuang.
2018-10-22 11:21:12 +01:00
Eric Yang a043dfa216 HADOOP-15821. Addendum patch to fix Hadoop Registry parent project.
Contributed by Íñigo Goiri
2018-10-21 12:50:10 -04:00
Eric Yang e2a9fa8448 HADOOP-15821. Move YARN Registry to Hadoop Registry.
Contributed by Íñigo Goiri
2018-10-19 19:46:48 -04:00
Inigo Goiri 00254d7b8c HADOOP-15836. Review of AccessControlList. Contributed by BELUGA BEHR. 2018-10-19 13:57:43 -07:00
Wei-Chiu Chuang cd2158456d HADOOP-15418. Hadoop KMSAuthenticationFilter needs to use getPropsByPrefix instead of iterator to avoid ConcurrentModificationException. Contributed by lqjack and Suma Shivaprasad 2018-10-18 17:28:28 -07:00
Jitendra Pandey d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Jason Lowe 9abda83947 HADOOP-15859. ZStandardDecompressor.c mistakes a class for an instance. Contributed by Jason Lowe 2018-10-17 14:38:42 -05:00
Adam Antal 24dc068a36 HADOOP-11100. Support to configure ftpClient.setControlKeepAliveTimeout.
Contributed by Adam Antal.

Signed-off-by: Xiao Chen <xiao@apache.org>
2018-10-17 11:34:50 -07:00
Steve Loughran 41b3603b5b
HADOOP-15861. Move DelegationTokenIssuer to the right path.
Contributed by Wei-Chiu Chuang.
2018-10-17 11:01:53 +01:00
Steve Loughran b738cb148c
HADOOP-15854. AuthToken Use StringBuilder instead of StringBuffer.
Contributed by Beluga Behr.
2018-10-17 10:29:09 +01:00
Haibo Chen c2288ac45b YARN-8448. AM HTTPS Support for AM communication with RMWeb proxy. (Contributed by Robert Kanter) 2018-10-16 13:36:26 -07:00
Robert Kanter 7fe1a40a6b HADOOP-15853. TestConfigurationDeprecation leaves behind a temp file, resulting in a license issue (ayushtkn via rkanter) 2018-10-15 15:02:37 -07:00
Xiao Chen b6fc72a025 HADOOP-14445. Addendum: Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances. 2018-10-15 10:51:55 -07:00
Anu Engineer 3c1fe073d2 HDDS-445. Create a logger to print out all of the incoming requests.
Contributed by Bharat Viswanadham.
2018-10-12 16:27:54 -07:00
Xiao Chen 5ec86b445c HADOOP-14445. Use DelegationTokenIssuer to create KMS delegation tokens that can authenticate to all KMS instances.
Contributed by Daryn Sharp, Xiao Chen, Rushabh S Shah.
2018-10-12 09:35:52 -07:00
Robert Kanter d91d47bc73 HADOOP-15717. TGT renewal thread does not log IOException (snemeth via rkanter) 2018-10-11 15:35:44 -07:00
Xiao Chen 64f2b32d57 HADOOP-15676. Cleanup TestSSLHttpServer. Contributed by Szilard Nemeth. 2018-10-11 15:12:36 -07:00
Takanobu Asanuma 7b57f2f71f HADOOP-15785. [JDK10] Javadoc build fails on JDK 10 in hadoop-common. Contributed by Dinesh Chitlangia. 2018-10-11 13:51:51 +09:00
Robert Kanter f261c31937 HADOOP-15708. Reading values from Configuration before adding deprecations make it impossible to read value with deprecated key (zsiegl via rkanter) 2018-10-10 18:51:37 -07:00
Steve Loughran cdc4350718
HADOOP-15839. Review + update cloud store sensitive keys in hadoop.security.sensitive-config-keys.
Contributed by Steve Loughran.
2018-10-10 17:00:37 +01:00
Akira Ajisaka 6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Eric Yang bf04f19456 YARN-8845. Removed unused hadoop.registry.rm reference.
Contributed by Íñigo Goiri
2018-10-09 15:54:45 -04:00
Takanobu Asanuma 9bbeb52486 HADOOP-15775. [JDK9] Add missing javax.activation-api dependency. Contributed by Akira Ajisaka. 2018-10-09 13:06:14 +09:00
Vidura Mudalige a30b1d1824
HADOOP-15818. Fix deprecated maven-surefire-plugin configuration in hadoop-kms module
This closes #425

Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-10-09 12:55:34 +09:00
Jason Lowe f13e231025 HADOOP-15820. ZStandardDecompressor native code sets an integer field as a long. Contributed by Jason Lowe 2018-10-05 09:06:02 -05:00
Yiqun Lin 81f635f47f HADOOP-15817. Reuse Object Mapper in KMSJSONReader. Contributed by Jonathan Eagles. 2018-10-04 10:30:30 +08:00
Aaron Fabbri 046b8768af
HADOOP-15621 S3Guard: Implement time-based (TTL) expiry for Authoritative Directory Listing. Contributed by Gabor Bota 2018-10-02 21:22:49 -07:00
Akira Ajisaka 31fa9bb799
HADOOP-15767. [JDK10] Building native package on JDK10 fails due to missing javah. Contributed by Takanobu Asanuma. 2018-10-03 11:29:18 +09:00
Sunil G 58fa96b697 Changed version in trunk to 3.3.0-SNAPSHOT. 2018-10-02 22:41:41 +05:30
Ewan Higgs 6fab6886f6 HDFS-13936. Multipart upload to HDFS to support 0 byte upload. Contributed by Ewan Higgs. 2018-10-02 14:05:35 +02:00
Akira Ajisaka 5c8d90763c
HADOOP-15764. Addendum patch: Fix NPE in SecurityUtil. 2018-09-28 10:03:22 +09:00
Akira Ajisaka c62b440421
HADOOP-15693. Typo in Secure Mode documentation. Contributed by Thomas Levine. 2018-09-26 10:30:43 +09:00
Akira Ajisaka 93b0f540ed
HADOOP-15783. [JDK10] TestSFTPFileSystem.testGetModifyTime fails. 2018-09-25 16:03:28 +09:00
Sunil G d060cbea48 HDFS-13937. Multipart Uploader APIs to be marked as private/unstable in 3.2.0. Contributed by Steve Loughran. 2018-09-24 21:19:47 +05:30
Sean Mackrory 0def61482b Merge branch 'HADOOP-15407' into trunk 2018-09-22 21:19:12 -06:00
Ewan Higgs 429a07e08c HADOOP-15764. [JDK10] Migrate from sun.net.dns.ResolverConfiguration to the replacement. Contributed by Akira Ajisaka. 2018-09-20 15:13:55 +02:00
Ewan Higgs 3da94a36e2 HADOOP-15756. [JDK10] Migrate from sun.net.util.IPAddressUtil to the replacement. Contributed by Akira Ajisaka. 2018-09-20 14:53:21 +02:00
Steve Loughran 646874c326
HADOOP-15748. S3 listing inconsistency can raise NPE in globber.
Contributed by Steve Loughran.
2018-09-20 13:04:52 +01:00
Vinayakumar B 7ad27e97f0 HADOOP-15736. Trash : Negative Value For Deletion Interval Leads To Abnormal Behaviour. Contributed by Ayush Saxena. 2018-09-20 09:31:35 +05:30
Chen Liang a30b4f9e71 HADOOP-15726. Create utility to limit frequency of log statements. Contributed by Erik Krogen. 2018-09-19 13:22:37 -07:00
Jason Lowe e71f61ecb8 HADOOP-15755. StringUtils#createStartupShutdownMessage throws NPE when args is null. Contributed by Lokesh Jain and Dinesh Chitlangia 2018-09-18 15:57:33 -05:00
Steve Loughran 1cf38a38da
HADOOP-15744. AbstractContractAppendTest fails against HDFS on HADOOP-15407 branch.
Contributed by Steve Loughran.
2018-09-18 10:56:56 +01:00
Takanobu Asanuma f796cfde76 HADOOP-15304. [JDK10] Migrate from com.sun.tools.doclets to the replacement. Contributed by Akira Ajisaka. 2018-09-18 18:48:25 +09:00
Yiqun Lin ee051ef9fe HADOOP-15742. Log if ipc backoff is enabled in CallQueueManager. Contributed by Ryan Wu. 2018-09-18 11:10:33 +08:00
Sean Mackrory 9475fd902a HADOOP-15745. ABFS: Add ABFS configuration to ConfigRedactor. 2018-09-17 19:54:01 +00:00
Thomas Marquardt 81dc4a995c HADOOP-15663. ABFS: Simplify configuration.
Contributed by Da Zhou.
2018-09-17 19:54:01 +00:00
Thomas Marquardt ce03a93f78 HADOOP-15446. ABFS: tune imports & javadocs; stabilise tests.
Contributed by Steve Loughran and Da Zhou.
2018-09-17 19:54:01 +00:00
Steve Loughran f044deedbb HADOOP-15407. HADOOP-15540. Support Windows Azure Storage - Blob file system "ABFS" in Hadoop: Core Commit.
Contributed by Shane Mainali, Thomas Marquardt, Zichen Sun, Georgi Chalakov, Esfandiar Manii, Amit Singh, Dana Kaban, Da Zhou, Junhua Gu, Saher Ahwal, Saurabh Pant, James Baker, Shaoyu Zhang, Lawrence Chen, Kevin Chen and Steve Loughran
2018-09-17 19:54:01 +00:00
Brahma Reddy Battula ef5c776a42 HADOOP-15733. Correct the log when Invalid emptier Interval configured. Contributed by Ayush Saxena 2018-09-14 07:32:27 +05:30
Hanisha Koneru 291dcf2247 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton. 2018-09-13 16:37:44 -07:00
Takanobu Asanuma 5d084d7eca HADOOP-12760. sun.misc.Cleaner has moved to a new location in OpenJDK 9. Contributed by Akira Ajisaka. 2018-09-13 20:42:09 +09:00
Steve Loughran d7c0a08a1c
HADOOP-15426 Make S3guard client resilient to DDB throttle events and network failures (Contributed by Steve Loughran) 2018-09-12 21:04:49 -07:00
bibinchundatt c44088ac19 YARN-8739. Fix jenkins issues for Node Attributes branch. Contributed by Sunil Govindan. 2018-09-12 16:01:01 +05:30
Naganarasimha 2475fb0a1e YARN-6856. [YARN-3409] Support CLI for Node Attributes Mapping. Contributed by Naganarasimha G R. 2018-09-12 16:00:59 +05:30
Mingliang Liu 87f63b6479 HADOOP-14833. Remove s3a user:secret authentication. Contributed by Steve Loughran 2018-09-11 17:18:42 -07:00
Allen Wittenauer 4b8c2b1c1c
HADOOP-14671. Upgrade Apache Yetus to 0.8.0
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-09-11 08:55:34 -07:00
Inigo Goiri 527288ef89 HDFS-13836. RBF: Handle mount table znode with null value. Contributed by yanghuafeng. 2018-09-06 16:47:54 -07:00
Vrushali C 962089ab59 HADOOP-15657 Registering MutableQuantiles via Metric annotation. Contributed by Sushil Ks 2018-09-05 20:53:21 -07:00
Giovanni Matteo Fumarola 9af96d4ed4 HADOOP-15707. Add IsActiveServlet to be used for Load Balancers. Contributed by Lukas Majercak. 2018-09-05 10:50:25 -07:00
Wei-Chiu Chuang e780556ae9 HADOOP-15696. KMS performance regression due to too many open file descriptors after Jetty migration. Contributed by Wei-Chiu Chuang. 2018-09-05 09:52:35 -07:00
Steve Loughran 9e96ac666d
HADOOP-10219. ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns.
Contributed by Kihwal Lee and Lukas Majercak.
2018-09-04 16:46:12 +01:00
Daniel Templeton f2c2a68ec2 HADOOP-15706. Typo in compatibility doc: SHOUD -> SHOULD
(Contributed by Laszlo Kollar via Daniel Templeton)

Change-Id: I6e2459d0700df7f3bad4eac8297a11690191c3ba
2018-08-30 09:12:36 -07:00
Steve Loughran 2e6c1109dc
HADOOP-15667. FileSystemMultipartUploader should verify that UploadHandle has non-0 length.
Contributed by Ewan Higgs
2018-08-30 14:33:16 +01:00
Kitti Nanasi 781437c219 HADOOP-15698. KMS log4j is not initialized properly at startup. Contributed by Kitti Nanasi. 2018-08-29 22:07:49 -07:00
Daniel Templeton d53a10b0a5 HADOOP-15705. Typo in the definition of "stable" in the interface classification
Change-Id: I3eae2143400a534903db4f186400561fc8d2bd56
2018-08-29 13:59:32 -07:00
John Zhuge f9c6fd9471
HADOOP-15633. fs.TrashPolicyDefault: Can't create trash directory. Contributed by Fei Hui. 2018-08-27 09:22:59 -07:00
Konstantin V Shvachko a4121c71c2 HDFS-13848. Refactor NameNode failover proxy providers. Contributed by Konstantin Shvachko. 2018-08-24 18:27:30 -07:00
Takanobu Asanuma bed8cb6979 HADOOP-14314. The OpenSolaris taxonomy link is dead in InterfaceClassification.md. Contributed by Rui Gao. 2018-08-23 21:54:38 +09:00
Steve Loughran 34577d2c21
HADOOP-15679. ShutdownHookManager shutdown time needs to be configurable & extended.
Contributed by Steve Loughran.
2018-08-20 18:36:24 -07:00
Giovanni Matteo Fumarola 79c97f6a0b HADOOP-14624. Add GenericTestUtils.DelayAnswer that accept slf4j logger API. Contributed by Ian Pickering and Wenxin He. 2018-08-17 14:40:00 -07:00
Xiao Chen 60ffec9f79 HADOOP-9214. Create a new touch command to allow modifying atime and mtime. Contributed by Hrishikesh Gadre. 2018-08-17 11:18:09 -07:00
Jason Lowe a17eed1b87 HDFS-13822. speedup libhdfs++ build (enable parallel build). Contributed by Allen Wittenauer and Pradeep Ambati 2018-08-17 12:25:36 -05:00
Xiao Chen 8d7c93186e HADOOP-15674. Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite. Contributed by Szilard Nemeth. 2018-08-17 10:09:23 -07:00
Márton Elek 77b015000a HADOOP-8807. Update README and website to reflect HADOOP-8662. Contributed by Andras Bokor. 2018-08-17 11:10:29 +02:00
Xiao Chen edeb2a356a HADOOP-15655. Enhance KMS client retry behavior. Contributed by Kitti Nanasi. 2018-08-16 22:42:03 -07:00
Akira Ajisaka 3e3963b035
HADOOP-15552. Move logging APIs over to slf4j in hadoop-tools - Part2. Contributed by Ian Pickering. 2018-08-16 00:31:59 +09:00
Márton Elek 7e822ec246 HDDS-324. Addendum: remove the q letter which is accidentally added to the hadoop-functions.sh. 2018-08-14 20:54:33 +02:00
Xiaoyu Yao b4031a8f1b HDDS-324. Use pipeline name as Ratis groupID to allow datanode to report pipeline info. Contributed by Mukul Kumar Singh. 2018-08-13 12:47:49 -07:00
Xiao Chen b94c8874e2 HADOOP-15638. KMS Accept Queue Size default changed from 500 to 128 in Hadoop 3.x. Contributed by Wei-Chiu Chuang. 2018-08-13 10:40:31 -07:00
Steve Loughran da9a39eed1
HADOOP-15583. Stabilize S3A Assumed Role support.
Contributed by Steve Loughran.
2018-08-08 22:57:24 -07:00
Wangda Tan 49c687608b Make 3.1.1 awared by other branches 2018-08-08 13:02:12 -07:00
Xiao Chen 9499df7b81 HDFS-13658. Expose HighestPriorityLowRedundancy blocks statistics. Contributed by Kitti Nanasi. 2018-08-08 10:40:20 -07:00
Ewan Higgs 2ec97abb2e HADOOP-15576. S3A Multipart Uploader to work with S3Guard and encryption Originally contributed by Ewan Higgs with refinements by Steve Loughran. 2018-08-08 13:50:23 +02:00
Mukul Kumar Singh 6ed8593d18 HDDS-301. ozone command shell does not contain subcommand to run ozoneFS commands. Contributed by Nilotpal Nandi. 2018-08-07 16:09:53 +05:30
Yiqun Lin 2e4e02b4df HDFS-13792. Fix FSN read/write lock metrics name. Contributed by Chao Sun. 2018-08-07 09:32:51 +08:00
Arpit Agarwal f2e29acbfa HADOOP-15476. fix logging for split-dns multihome . Contributed by Ajay Kumar. 2018-08-01 12:32:01 -07:00
Chen Liang e8f952ef06 HADOOP-15637. LocalFs#listLocatedStatus does not filter out hidden .crc files. Contributed by Erik Krogen. 2018-07-30 10:25:07 -07:00
Eric Yang 77721f39e2 HADOOP-15593. Fixed NPE in UGI spawnAutoRenewalThreadForUserCreds.
Contributed by Gabor Bota
2018-07-26 18:35:36 -04:00
Xiao Chen be150a17b1 HDFS-13622. mkdir should print the parent directory in the error message when parent directories do not exist. Contributed by Shweta. 2018-07-26 10:24:32 -07:00
Yiqun Lin 9089790cab HADOOP-15611. Log more details for FairCallQueue. Contributed by Ryan Wu. 2018-07-26 18:08:28 +08:00
Mukul Kumar Singh 5f0b924360 HADOOP-15395. DefaultImpersonationProvider fails to parse proxy user config if username has . in it. Contributed by Ajay Kumar. 2018-07-25 21:09:11 +05:30
Xiao Chen 81d59506e5 HADOOP-15609. Retry KMS calls when SSLHandshakeException occurs. Contributed by Kitti Nanasi. 2018-07-24 21:45:43 -07:00
Daniel Templeton 849c45db18 HDFS-13448. HDFS Block Placement - Ignore Locality for First Block Replica
(Contributed by BELUGA BEHR via Daniel Templeton)

Change-Id: I965d1cfa642ad24296038b83e3d5c9983545267d
2018-07-24 16:05:27 -07:00
Gera Shegalov 6bec03cfc8 HADOOP-15612. Improve exception when tfile fails to load LzoCodec. (gera) 2018-07-24 14:32:30 -07:00
bibinchundatt 8461278833 YARN-8544. [DS] AM registration fails when hadoop authorization is enabled. Contributed by Bibin A Chundatt. 2018-07-24 13:09:17 +05:30
Haibo Chen 17e26163ec HADOOP-15586. Fix wrong log statement in AbstractService. (Szilard Nemeth via Haibo Chen) 2018-07-23 11:18:25 -07:00
Xiao Chen 993ec026d1 HADOOP-15596. Stack trace should not be printed out when running hadoop key commands. Contributed by Kitti Nanasi. 2018-07-20 19:46:46 -07:00
Haibo Chen e6873dfde0 YARN-7300. DiskValidator is not used in LocalDirAllocator. (Szilard Nemeth via Haibo Chen) 2018-07-19 16:27:11 -07:00
Kihwal Lee ccf2db7fc2 HADOOP-15614. TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails. Contributed by Weiwei Yang. 2018-07-19 11:13:37 -05:00
Wei-Chiu Chuang d215357718 HDFS-13485. DataNode WebHDFS endpoint throws NPE. Contributed by Siyao Meng. 2018-07-16 15:45:55 -07:00
Xiao Chen d2874e0417 HDFS-13690. Improve error message when creating encryption zone while KMS is unreachable. Contributed by Kitti Nanasi. 2018-07-16 13:19:53 -07:00
Wei-Chiu Chuang 0c7a578927 HADOOP-15598. DataChecksum calculate checksum is contented on hashtable synchronization. Contributed by Prasanth Jayachandran. 2018-07-16 11:32:45 -07:00
Akira Ajisaka 88625f5cd9
HADOOP-15531. Use commons-text instead of commons-lang in some classes to fix deprecation warnings. Contributed by Takanobu Asanuma. 2018-07-13 11:42:12 -04:00
Sean Mackrory 4f3f9391b0 HADOOP-15316. GenericTestUtils can exceed maxSleepTime. Contributed by Adam Antal. 2018-07-12 17:24:01 +02:00
Wei-Chiu Chuang 162228e8db HDFS-13723. Occasional "Should be different group" error in TestRefreshUserMappings#testGroupMappingRefresh. Contributed by Siyao Meng. 2018-07-11 10:02:08 -07:00
Akira Ajisaka 4a08ddfa68
HADOOP-15568. fix some typos in the .sh comments. Contributed by Steve Loughran. 2018-07-09 15:43:38 -04:00
Xiao Chen 895845e9b0 HADOOP-15581. Set default jetty log level to INFO in KMS. Contributed by Kitti Nanasi. 2018-07-09 12:06:50 -07:00
Xiao Chen def9d94a40 HADOOP-15591. KMSClientProvider should log KMS DT acquisition at INFO level. Contributed by Kitti Nanasi. 2018-07-09 12:01:52 -07:00
Yiqun Lin 7a68ac607c HDFS-13674. Improve documentation on Metrics. Contributed by Chao Sun. 2018-07-09 14:27:34 +08:00
Arpit Agarwal 061b168529 HDDS-167. Rename KeySpaceManager to OzoneManager. Contributed by Arpit Agarwal. 2018-07-06 12:09:05 -07:00
Xiaoyu Yao 498e3bfb6b HADOOP-15571. Multiple FileContexts created with the same configuration object should be allowed to have different umask. Contributed by Vinod Kumar Vavilapalli. 2018-07-06 11:56:09 -07:00
Andrew Wang f51da9c4d1 HADOOP-15554. Improve JIT performance for Configuration parsing. Contributed by Todd Lipcon. 2018-07-02 18:31:21 +02:00
Nanda kumar d40121845e HADOOP-15574: Suppress build error if there are no docs after excluding private annotations. Contributed by Takanobu Asanuma. 2018-07-01 17:11:46 +05:30
Eric E Payne d36f6b9e93 HADOOP-15548: Randomize local dirs. Contributed by Jim Brennan. 2018-06-29 18:18:32 +00:00
Eric Yang 384764cdea YARN-8409. Fixed NPE in ActiveStandbyElectorBasedElectorService.
Contributed by Chandni Singh
2018-06-28 12:39:49 -04:00
Bharat Viswanadham d3fa83a44b HDFS-13705:The native ISA-L library loading failure should be made warning rather than an error message. Contributed by Shashikant Banerjee. 2018-06-28 09:21:56 -07:00
Akira Ajisaka ddbff7c8d3
HADOOP-14313. Replace/improve Hadoop's byte[] comparator. Contributed by Vikas Vishwakarma. 2018-06-28 14:58:40 +09:00
Akira Ajisaka 2b2399d623
HADOOP-15495. Upgrade commons-lang version to 3.7 in hadoop-common-project and hadoop-tools. Contributed by Takanobu Asanuma. 2018-06-28 14:37:22 +09:00
Todd Lipcon 7a3c6e9c3c HADOOP-15550. Avoid static initialization of ObjectMappers 2018-06-25 15:36:45 -07:00
Inigo Goiri abc3e4bad9 HADOOP-15458. TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows. Contributed by Xiao Liang. 2018-06-25 09:50:27 -07:00
Steve Loughran 1ba4e62304
HADOOP-14396. Add builder interface to FileContext.
Contributed by  Lei (Eddy) Xu.
2018-06-25 14:38:57 +01:00
Eric E Payne 6432128622 YARN-8444: NodeResourceMonitor crashes on bad swapFree value. Contributed by Jim Brennan. 2018-06-22 17:15:29 +00:00
Todd Lipcon 59de967954 HADOOP-15549. Upgrade to commons-configuration 2.1 regresses task CPU consumption 2018-06-21 10:32:52 -07:00
Todd Lipcon 43541a1890 HADOOP-15551. Avoid use of Arrays.stream in Configuration.addTags 2018-06-20 16:43:10 -07:00
Xiao Chen 32f867a6a9 HDFS-13682. Cannot create encryption zone after KMS auth token expires. 2018-06-20 15:58:26 -07:00
Eric Yang 2d87592fc6 HADOOP-15527. Improve delay check for stopping processes.
Fixed script location.
               Contributed by Vinod Kumar Vavilapalli
2018-06-19 13:38:13 -04:00
Chris Douglas 980031bb04 HADOOP-13186. Multipart Uploader API. Contributed by Ewan Higgs 2018-06-17 11:54:26 -07:00
Surendra Singh Lilhore 3905fdb793 HADOOP-15523. Shell command timeout given is in seconds whereas it is taken as millisec while scheduling. Contributed by Bilwa S T. 2018-06-17 12:12:01 +05:30
Yiqun Lin 8e7548d33b HDFS-13641. Add metrics for edit log tailing. Contributed by Chao Sun. 2018-06-13 20:05:55 +08:00
Eric Yang 108da85320 HADOOP-15527. Improve delay check for stopping processes.
Contributed by Vinod Kumar Vavilapalli
2018-06-12 20:40:32 -04:00
Inigo Goiri aeaf9fec62 HADOOP-15532. TestBasicDiskValidator fails with NoSuchFileException. Contributed by Giovanni Matteo Fumarola. 2018-06-12 14:16:14 -07:00
Xiao Chen a16623df21 HADOOP-15307. NFS: flavor AUTH_SYS should use VerifierNone. Contributed by Gabor Bota. 2018-06-12 11:16:24 -07:00
Rohith Sharma K S 2df73dace0 YARN-8405. RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773. Contributed by Íñigo Goiri. 2018-06-12 17:13:09 +05:30
Yongjun Zhang 7c3dc39083 Update CHANGES, RELEASENOTES, and jdiff for 3.0.3 release.
(cherry picked from commit 5a82f10e32)
2018-06-11 15:13:18 -07:00
Sammi Chen 18201b882a HADOOP-15499. Performance severe drops when running RawErasureCoderBenchmark with NativeRSRawErasureCoder. Contributed by Sammi Chen. 2018-06-11 13:53:37 +08:00
Steve Loughran ef0118b91e
HADOOP-15520. Add tests for various org.apache.hadoop.util classes.
Contributed by Arash Nabili
2018-06-09 15:33:38 +01:00
Inigo Goiri 12be8bad7d HADOOP-15516. Add test cases to cover FileUtil#readLink. Contributed by Giovanni Matteo Fumarola. 2018-06-07 13:34:52 -07:00
Steve Loughran f494f0b896
HADOOP-15512. Clean up Shell from JDK7 workarounds.
Contributed by Zsolt Venczel.
2018-06-07 15:55:49 +01:00
Inigo Goiri db81f85e84 HADOOP-15513. Add additional test cases to cover some corner cases for FileUtil#symlink. Contributed by Giovanni Matteo Fumarola. 2018-06-06 11:44:17 -07:00
Xiao Chen ba4011d64f HADOOP-15217. FsUrlConnection does not handle paths with spaces. Contributed by Joseph Fourny and Zsolt Venczel. 2018-06-05 21:18:31 -07:00
Chen Liang 1b0d4f4606 HDFS-13547. Add ingress port based sasl resolver. Contributed by Chen Liang. 2018-06-05 11:51:29 -07:00
Xiao Chen 6d5e87aec2 HADOOP-15507. Add MapReduce counters about EC bytes read. 2018-06-04 21:13:47 -07:00
Haibo Chen 04cf699dd5 YARN-8388. TestCGroupElasticMemoryController.testNormalExit() hangs on Linux. (Miklos Szegedi via Haibo Chen) 2018-06-04 14:23:08 -07:00
Haibo Chen 4880d890ee YARN-8375. TestCGroupElasticMemoryController fails surefire build. (Miklos Szegedi via Haibo Chen) 2018-06-01 16:29:13 -07:00
Daniel Templeton cba3194998 HDFS-13636. Cross-Site Scripting vulnerability in HttpServer2
(Contributed by Haibo Yan via Daniel Templeton)

Change-Id: I28edde8125dd20d8d270f0e609d1c04d8173c8b7
2018-06-01 14:42:39 -07:00
Wei-Chiu Chuang 32671d8713 HADOOP-14783. [KMS] Add missing configuration properties into kms-default.xml. Contributed by Chetna Chaudhari. 2018-05-31 16:09:33 -07:00
Takanobu Asanuma d1e2b80980
MAPREDUCE-7098. Upgrade common-langs version to 3.7 in hadoop-mapreduce-project
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-31 18:31:10 +09:00
Kihwal Lee e3236a9680 Additional check when unpacking archives. Contributed by Wilfred Spiegelenburg. 2018-05-29 14:15:12 -05:00
Inigo Goiri 3c75f8e493 HADOOP-15497. TestTrash should use proper test path to avoid failing on Windows. Contributed by Anbang Hu. 2018-05-29 09:11:08 -07:00
Wei-Chiu Chuang 438ef4951a HADOOP-15455. Incorrect debug message in KMSACL#hasAccess. Contributed by Yuen-Kuei Hsueh. 2018-05-28 17:32:32 -07:00
Karthik Palanisamy 61df174e8b
HADOOP-15449. Increase default timeout of ZK session to avoid frequent NameNode failover
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-28 19:41:07 +09:00
Johan Gustavsson d14e26b31f
HADOOP-15477. Make unjar in RunJar overrideable
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-28 17:29:59 +09:00
Xiao Chen 02322de3f9 HADOOP-15473. Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997. Contributed by Gabor Bota. 2018-05-25 09:10:51 -07:00
Steve Loughran bddfe796f2 HADOOP-15494. TestRawLocalFileSystemContract fails on Windows.
Contributed by Anbang Hu.
2018-05-25 11:12:47 +01:00
Arpit Agarwal 51ce02bb54 HADOOP-15486. Make NetworkTopology#netLock fair. Contributed by Nanda kumar. 2018-05-23 10:30:12 -07:00
Robert Kanter aa23d49fc8 HADOOP-15457. Add Security-Related HTTP Response Header in WEBUIs. (kanwaljeets via rkanter) 2018-05-23 10:24:09 -07:00
Sammi Chen 34e8b9f9a8 HDFS-13540. DFSStripedInputStream should only allocate new buffers when reading. Contributed by Xiao Chen. 2018-05-23 19:10:09 +08:00
Akira Ajisaka 745f203e57
Additional check when unpacking archives. Contributed by Jason Lowe and Akira Ajisaka. 2018-05-23 17:16:23 +09:00
Miklos Szegedi 3e5f7ea986 YARN-8310. Handle old NMTokenIdentifier, AMRMTokenIdentifier, and ContainerTokenIdentifier formats. Contributed by Robert Kanter. 2018-05-22 18:10:33 -07:00
Kihwal Lee bcc8e76bad HADOOP-15450. Avoid fsync storm triggered by DiskChecker and handle disk full situation. Contributed by Arpit Agarwal. 2018-05-22 11:20:51 -05:00
Mukul Kumar Singh 57c2feb0d3 HADOOP-15474. Rename properties introduced for <tags>. Contributed by Zsolt Venczel. 2018-05-22 13:33:31 +05:30
Rushabh Shah 73e9120ad7 Skip the proxy user check if the ugi has not been initialized. Contributed by Daryn Sharp 2018-05-21 12:33:00 -05:00
Xiao Chen 89f5911392 HADOOP-15154. Abstract new method assertCapability for StreamCapabilities testing. Contributed by Zsolt Venczel. 2018-05-18 11:29:20 -07:00
Inigo Goiri 8783613696 HDFS-13586. Fsync fails on directories on Windows. Contributed by Lukas Majercack. 2018-05-17 19:26:44 -07:00
Inigo Goiri 53b807a6a8 HDFS-13560. Insufficient system resources exist to complete the requested service for some tests on Windows. Contributed by Anbang Hu. 2018-05-17 17:03:23 -07:00
Steve Loughran 07d8505f75 HADOOP-15466. Correct units in adl.http.timeout.
Contributed by Sean Mackrory,
2018-05-15 16:19:03 +01:00
Sammi Chen f3f544b004 Add 2.9.1 release notes and changes documents 2018-05-14 15:14:02 +08:00
sammichen 66c9905b57 Revert "Add 2.9.1 release notes and changes documents"
This reverts commit e4dc346d65.
2018-05-14 14:38:40 +08:00
littlezhou e4dc346d65 Add 2.9.1 release notes and changes documents 2018-05-14 14:24:01 +08:00
Rushabh Shah a5449d3631 HADOOP-15441. Log kms url and token service at debug level. Contributed by Gabor Bota 2018-05-12 12:19:13 -05:00
Haibo Chen 274eee3284 MAPREDUCE-7095. Race conditions in closing FadvisedChunkedFile. (Miklos Szegedi via Haibo Chen) 2018-05-10 10:46:55 -07:00
Inigo Goiri 1da8d4190d HADOOP-15454. TestRollingFileSystemSinkWithLocal fails on Windows. Contributed by Xiao Liang. 2018-05-10 09:41:16 -07:00
Sean Mackrory 1cfe7506f7 HADOOP-15356. Make HTTP timeout configurable in ADLS connector. Contributed by Atul Sikaria and Sean Mackrory. 2018-05-09 15:15:51 -06:00
Anu Engineer 3a43ac2851 HDDS-1. Remove SCM Block DB. Contributed by Xiaoyu Yao. 2018-05-07 14:58:52 -07:00
Xiao Chen a3a1552c33 Revert "HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah."
This reverts commit 583fa6ed48.

 Conflicts:
	hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSTokenRenewer.java
	hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
2018-05-07 13:32:27 -07:00
Rushabh Shah 4cdbdce752 HADOOP-14841 Kms client should disconnect if unable to get output stream from connection. Contributed by Rushabh S Shah 2018-05-04 15:36:13 -05:00
Daniel Templeton 87c23ef643 HDFS-13481. TestRollingFileSystemSinkWithHdfs#testFlushThread: test failed intermittently
(Contributed by Gabor Bota via Daniel Templeton)

Change-Id: I9921981dfa69669fe7912dd2a31ae8b638283204
2018-05-02 17:13:40 -07:00
Steve Loughran fe649bb305 HADOOP-12071. conftest is not documented.
Contributed by Kengo Seki.
2018-05-02 13:33:56 +01:00
Jason Lowe e07156e8b0
HADOOP-15406. hadoop-nfs dependencies for mockito and junit are not test scope
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-02 17:30:10 +09:00
BELUGA BEHR 33768724ff
HADOOP-15377. Improve debug messages in MetricsConfig.java
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-05-02 17:09:22 +09:00
Steve Loughran 8f42dafcf8 HADOOP-15250. Split-DNS MultiHomed Server Network Cluster Network IPC Client Bind Addr Wrong
Contributed by Ajay Kumar
2018-05-01 22:32:40 +01:00
Wei-Chiu Chuang bff3d7b0cf HADOOP-15382. Log kinit output in credential renewal thread. Contributed by Gabor Bota. 2018-04-27 10:05:55 -07:00
Ewan Higgs 84ecfe3ceb
HADOOP-14188. Remove the usage of org.mockito.internal.util.reflection.Whitebox
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-27 19:21:37 +09:00
Wangda Tan 13d389bf51 HADOOP-15411. AuthenticationFilter should use Configuration.getPropsWithPrefix instead of iterator. (Suma Shivaprasad via wangda)
Change-Id: Ia19458a43aceb4ffdb1e8eccc519655235d92ba9
2018-04-26 13:54:40 -07:00
Daryn Sharp bbc8dfcb8b HADOOP-15402. Prevent double logout of UGI's LoginContext
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-26 13:54:40 -07:00
Owen O'Malley 2adda92de1
HDFS-7240 git commit fixup
Signed-off-by: Owen O'Malley <omalley@apache.org>
2018-04-26 13:52:17 -07:00
Mukul Kumar Singh 8cc13ff962 HDFS-13425. Ozone: Clean-up of ozone related change from hadoop-common-project. Contributed by Lokesh Jain. 2018-04-26 05:36:04 -07:00
Mukul Kumar Singh ec6c8742e5 HDFS-13415. Ozone: Remove cblock code from HDFS-7240. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer 8b832f3c35 HDFS-13405. Ozone: Rename HDSL to HDDS.
Contributed by Ajay Kumar, Elek Marton, Mukul Kumar Singh, Shashikant Banerjee and Anu Engineer.
2018-04-26 05:36:04 -07:00
Anu Engineer 4f37ff7616 HDFS-13341. Ozone: Move ozone specific ServiceRuntimeInfo utility to the framework. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer 4e61bc431e HDFS-13258. Ozone: restructure Hdsl/Ozone code to separated maven subprojects.
Contributed by Elek Marton, Mukul Kumar Singh, Xiaoyu Yao, Ajay Kumar, Anu Engineer, Lokesh Jain, Nanda Kumar.
2018-04-26 05:36:04 -07:00
Anu Engineer 740a06cdd7 HDFS-12719. Ozone: Fix checkstyle, javac, whitespace issues in HDFS-7240 branch. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Anu Engineer 06c6582694 HDFS-12585. Ozone: KSM UI: Add description for configs in UI. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Anu Engineer 0b3270a8de HDFS-12513. Ozone: Create UI page to show Ozone configs by tag. Contributed by Ajay Kumar. 2018-04-26 05:36:04 -07:00
Chris Douglas 6d3d01d7ad HADOOP-12077. Provide a multi-URI replication Inode for ViewFs. Contributed by Gera Shegalov 2018-04-26 05:36:04 -07:00
Anu Engineer dcc21a4f93 HDFS-12005. Ozone: Web interface for SCM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Anu Engineer acfae2716a HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2018-04-26 05:36:04 -07:00
Yiqun Lin 8cf0c864c4 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2018-04-26 05:36:04 -07:00
Rakesh Radhakrishnan 665f7c8b69 HDFS-11794. Add ec sub command -listCodec to show currently supported ec codecs. Contributed by SammiChen. 2018-04-26 05:36:04 -07:00
Robert Kanter 7ab08a9c37 HADOOP-15390. Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens (xiaochen via rkanter) 2018-04-23 15:44:15 -07:00
Lei Xu 42e82f0281 Update CHANGES, RELEASENOTES, and jdiff for 3.0.2 release.
(cherry picked from commit f6ecb76d0b)

Conflicts:
	hadoop-project-dist/pom.xml
2018-04-23 14:14:05 -07:00
Virajith Jalaparti 827d8593a5 HDFS-13311. RBF: TestRouterAdminCLI#testCreateInvalidEntry fails on Windows. Contributed by Inigo Goiri 2018-04-16 15:55:00 -07:00
Bharat Viswanadham ec1e8c1a8c HADOOP-15379. Make IrqHandler.bind() public. Contributed by Ajay Kumar 2018-04-12 21:51:20 -07:00
Aaron Fabbri 0d898b7bb8
HADOOP-12502 SetReplication OutOfMemoryError. Contributed by Vinayakumar B. 2018-04-11 17:19:56 -07:00
Xiao Chen 7c9cdad6d0 HDFS-13056. Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts. Contributed by Dennis Huo. 2018-04-10 21:31:48 -07:00
Xiao Chen 583fa6ed48 HADOOP-14445. Delegation tokens are not shared between KMS instances. Contributed by Xiao Chen and Rushabh S Shah. 2018-04-10 15:38:25 -07:00
Jason Lowe e81397545a HADOOP-15357. Configuration.getPropsWithPrefix no longer does variable substitution. Contributed by Jim Brennan 2018-04-10 16:44:55 -05:00
Xiaoyu Yao 8ab776d61e HADOOP-15340. Provide meaningful RPC server name for RpcMetrics. Contributed by Elek Marton. 2018-04-10 11:42:54 -07:00
Ewan Higgs cef8eb7981
HADOOP-15376. Remove double semi colons on imports that make Clover fall over.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-04-10 23:58:26 +09:00
Bharat 0006346abe HADOOP-15328. Fix the typo in HttpAuthentication.md. Contributed by fang zhenyi 2018-04-09 16:37:49 -07:00
Mukul Kumar Singh 0b345b7653 HADOOP-15366. Add a helper shutdown routine in HadoopExecutor to ensure clean shutdown. Contributed by Shashikant Banerjee. 2018-04-07 16:29:01 +05:30
Wangda Tan 6cf023f9b7 Added CHANGES/RELEASES/Jdiff for 3.1.0 release
Change-Id: Ied5067a996151c04d15cad46c46ac98b60c37b39
(cherry picked from commit 2d96570452a72569befdf9cfe9b90c9fa2e0e261)
2018-04-05 15:52:39 -07:00
Wei Yan 3121e8c293 HDFS-13353. RBF: TestRouterWebHDFSContractCreate failed. Contributed by Takanobu Asanuma. 2018-04-05 12:00:52 -07:00
Miklos Szegedi e52539b46f HADOOP-14855. Hadoop scripts may errantly believe a daemon is still running, preventing it from starting. Contributed by Robert Kanter. 2018-04-04 15:35:58 -07:00
Xiao Chen 1077392eaa HADOOP-15355. TestCommonConfigurationFields is broken by HADOOP-15312. Contributed by LiXin Ge. 2018-04-02 23:10:08 -07:00
Xiao Chen 57374c4737 HADOOP-15317. Improve NetworkTopology chooseRandom's loop. 2018-04-02 22:48:26 -07:00
Steve Loughran dde1579096 HADOOP-15146. Remove DataOutputByteBuffer.
Contributed by BELUGA BEHR.
2018-04-02 15:52:40 +01:00
Konstantin V Shvachko acfd764fcc HADOOP-15253. Should update maxQueueSize when refresh call queue. Contributed by Tao Jie. 2018-03-30 18:55:35 -07:00
Konstantin V Shvachko 2216bde322 HADOOP-12862. LDAP Group Mapping over SSL can not specify trust store. Contributed by Wei-Chiu Chuang and Konstantin Shvachko. 2018-03-29 17:16:23 -07:00
Wei-Chiu Chuang 3fe41c65d8 HADOOP-15312. Undocumented KeyProvider configuration keys. Contributed by LiXin Ge. 2018-03-27 13:33:41 -07:00
Xiao Chen c22d62b338 HADOOP-15313. TestKMS should close providers. 2018-03-26 15:59:32 -07:00
Xiaoyu Yao 22194f3d21 HADOOP-15339. Support additional key/value propereties in JMX bean registration. Contributed by Elek Marton. 2018-03-26 10:45:29 -07:00
Anu Engineer 28790b81ec HADOOP-15295. Remove redundant logging related to tags from Configuration. Contributed by Ajay Kumar. 2018-03-23 21:17:18 -07:00
Lei Xu f9427b73a2 Update releasenotes and changelogs for 3.0.1 release 2018-03-23 11:46:41 -07:00
Jitendra Pandey 4bea96f9a8 HADOOP-14067. VersionInfo should load version-info.properties from its own classloader. Contributed by Thejas M Nair. 2018-03-22 14:09:06 -07:00
Yufei Gu 268c29a5f5 HADOOP-15331. Fix a race condition causing parsing error of java.io.BufferedInputStream in class org.apache.hadoop.conf.Configuration. Contributed by Miklos Szegedi. 2018-03-22 11:04:37 -07:00
fang zhenyi a3ed6f40e1
HADOOP-15330. Remove jdk1.7 profile from hadoop-annotations module
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-21 14:01:26 +09:00
Yufei Gu 138c1ed566 HADOOP-15062. TestCryptoStreamsWithOpensslAesCtrCryptoCodec fails on Debian 9. Contributed by Miklos Szegedi. 2018-03-20 15:19:18 -07:00
Inigo Goiri fe224ff972 HDFS-13268. TestWebHdfsFileContextMainOperations fails on Windows. Contributed by Xiao Liang. 2018-03-20 11:19:46 -07:00
Inigo Goiri e65ff1c8be HDFS-13296. GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases. Contributed by Xiao Liang. 2018-03-19 17:19:30 -07:00
Chris Douglas 3fc3fa9711 HADOOP-14667. Flexible Visual Studio support. Contributed by Allen Wittenauer 2018-03-19 16:05:55 -07:00
Xiao Chen 21c6661461 HADOOP-15234. Throw meaningful message on null when initializing KMSWebApp. Contributed by fang zhenyi. 2018-03-15 20:17:16 -07:00
Steve Loughran 1976e0066e HADOOP-15209. DistCp to eliminate needless deletion of files under already-deleted directories.
Contributed by Steve Loughran.
2018-03-15 18:05:14 +00:00
Aaron Fabbri e840b4adf4
HADOOP-15278 log s3a at info. Contributed by Steve Loughran. 2018-03-14 15:47:56 -07:00
Wangda Tan 252c2b4d52 Revert "HADOOP-13707. If kerberos is enabled while HTTP SPNEGO is not configured, some links cannot be accessed. Contributed by Yuanbo Liu"
Change-Id: I946a466a43d56c73bb0135384e73cb8513595461
(cherry picked from commit 80ee5248b2dda1cb8d122d4f362f2f8cf02b9467)
2018-03-14 13:36:53 -07:00
Takanobu Asanuma fea16a440d
HADOOP-15294. TestUGILoginFromKeytab fails on Java9
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-14 13:38:07 +09:00
Inigo Goiri 427fd027a3 HADOOP-15308. TestConfiguration fails on Windows because of paths. Contributed by Xiao Liang. 2018-03-13 20:01:07 -07:00
Chris Douglas 9d6994da19 HADOOP-15311. HttpServer2 needs a way to configure the acceptor/selector count. Contributed by Erik Krogen 2018-03-13 13:55:18 -07:00
Chris Douglas 45d1b0fdcc HADOOP-14696. parallel tests don't work for Windows. Contributed by Allen Wittenauer 2018-03-12 20:05:39 -07:00
Steve Loughran dd05871b8b HADOOP-15297. Make S3A etag => checksum feature optional.
Contributed by Steve Loughran.
2018-03-12 14:01:42 +00:00
Owen O'Malley fa6a8b78d4 Revert "HADOOP-13119. Add ability to secure log servlet using proxy users. Contribute by Yuanbo Liu."
This reverts commit a847903b6e.

Change-Id: I3122a2142f5bdf8507dece930e447556a43cd9ae
(cherry picked from commit 8fad3ec76070ccfcd3ed80feaba4355077bc6f5c)
2018-03-09 22:46:41 -08:00
Owen O'Malley 3a8dade9b1 Revert "HADOOP-14077. Add ability to access jmx via proxy. Contributed by Yuanbo Liu."
This reverts commit 172b23af33.

(cherry picked from commit d0d2d4c51e9534e08893ae14cf3fff7b2ee70b1d)
2018-03-09 22:46:30 -08:00
Takanobu Asanuma 99ab511cba
HADOOP-15293. TestLogLevel fails on Java 9
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-09 10:20:35 -08:00
Steve Loughran 3f7bd46797 HADOOP-15277. Remove .FluentPropertyBeanIntrospector from CLI operation log output.
Contributed by Steve Loughran.
2018-03-09 10:44:07 +00:00
Xiao Chen a906a22645 HADOOP-15280. TestKMS.testWebHDFSProxyUserKerb and TestKMS.testWebHDFSProxyUserSimple fail in trunk. Contributed by Bharat Viswanadham. 2018-03-08 10:17:02 -08:00
Takanobu Asanuma e6f99e205b
HADOOP-15291. TestMiniKdc fails on Java 9
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-06 06:01:33 -08:00
Steve Loughran 12ecb55ffe HADOOP-15289. FileStatus.readFields() assertion incorrect.
Contributed by Steve Loughran.
2018-03-06 10:05:12 +00:00
Takanobu Asanuma 4971276631
HADOOP-15271. Remove unicode multibyte characters from JavaDoc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-05 15:07:58 -08:00
Ajay Kumar 3dc30bc24e
HADOOP-15286. Remove unused imports from TestKMSWithZK.java
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-04 07:12:52 -08:00
Ray Chiang 923e1773e1 HADOOP-14652. Update metrics-core version to 3.2.4. (rchiang) 2018-03-01 15:36:10 -08:00
Nanda kumar 55d04a6db1
HADOOP-15275. Incorrect javadoc for return type of RetryPolicy#shouldRetry
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-03-01 19:03:47 +09:00
Xiaoyu Yao f20e10b2dd HDFS-13081. Datanode#checkSecureConfig should allow SASL and privileged HTTP. Contributed by Ajay Kumar. 2018-02-28 09:44:39 -08:00
Arpit Agarwal ac42dfc888 MAPREDUCE-7061. SingleCluster setup document needs to be updated. Contributed by Bharat Viswanadham. 2018-02-27 10:27:18 -08:00
Arpit Agarwal 137f0d324a HADOOP-14959. DelegationTokenAuthenticator.authenticate() to wrap network exceptions. Contributed by Ajay Kumar. 2018-02-27 10:15:43 -08:00
Arpit Agarwal 28f644bf25 HADOOP-15178. Generalize NetUtils#wrapException to handle other subclasses with String Constructor. Contributed by Ajay Kumar. 2018-02-27 07:38:29 -08:00
Arpit Agarwal 78a10029ec HADOOP-15265. Exclude json-smart explicitly in hadoop-auth avoid being pulled in transitively. Contributed by Nishant Bangarwa. 2018-02-26 13:56:53 -08:00
Arpit Agarwal 2fa7963c3d HADOOP-15254. Correct the wrong word spelling 'intialize'. Contributed by fang zhenyi. 2018-02-24 14:41:55 -08:00
Arpit Agarwal 033f9c68ea HADOOP-15255. Upper/Lower case conversion support for group names in LdapGroupsMapping. Contributed by Nanda kumar. 2018-02-23 15:37:17 -08:00
Kihwal Lee 59cf758877 HADOOP-9747. Reduce unnecessary UGI synchronization. Contributed by Daryn Sharp. 2018-02-23 13:10:56 -06:00
Anu Engineer 3688e491d5 HADOOP-15007. Stabilize and document Configuration <tag> element. Contributed by Ajay Kumar. 2018-02-23 10:26:22 -08:00
Sean Mackrory 2bc3351eaf HADOOP-6852. apparent bug in concatenated-bzip2 support (decoding). Contributed by Zsolt Venczel. 2018-02-21 12:57:14 -07:00
Arpit Agarwal b0d3c877e3 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-20 18:18:58 -08:00
Robert Kanter 324e5a7cf2 HADOOP-15235. Authentication Tokens should use HMAC instead of MAC (rkanter) 2018-02-20 17:24:37 -08:00
Steve Loughran 1d37cf675c HADOOP-15070. add test to verify FileSystem and paths differentiate on user info.
Contributed by Steve Loughran.
2018-02-19 20:43:40 +00:00
fang zhenyi 4d4dde5112
HADOOP-15223. Replace Collections.EMPTY* with empty* when available
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-02-18 22:19:39 +09:00
Wangda Tan 31db977cb3 HADOOP-14875. Create end user documentation from the compatibility guidelines. (Daniel Templeton via wangda)
Change-Id: I5d66d0f034874623af22aecae0a0b858b65396ed
2018-02-17 12:28:55 +08:00
Jason Lowe 0898ff42e9 HADOOP-15206. BZip2 drops and duplicates records when input split size is small. Contributed by Aki Tanaka 2018-02-16 14:49:00 -06:00
Steve Loughran 9a013b255f HADOOP-15176. Enhance IAM Assumed Role support in S3A client.
Contributed by Steve Loughran

(cherry picked from commit 96c047fbb9)
2018-02-15 15:57:10 +00:00
Anu Engineer 8f66affd62 HADOOP-15204. Add Configuration API for parsing storage sizes. Contributed by Anu Engineer. 2018-02-14 13:11:37 -08:00
Xiao Chen 1f20f432d2 Revert "HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar."
This reverts commit 332269de06.
2018-02-14 10:25:05 -08:00
Steve Loughran f20dc0d577 HADOOP-10571. Use Log.*(Object, Throwable) overload to log exceptions.
Contributed by Andras Bokor.
2018-02-14 16:20:14 +00:00
Arpit Agarwal 332269de06 HADOOP-12897. KerberosAuthenticator.authenticate to include URL on IO failures. Contributed by Ajay Kumar. 2018-02-13 10:14:16 -08:00
Chris Douglas 0c5d7d71a8 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:07:15 -08:00
Chris Douglas 9cc6d1dfb3 Revert "HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine"
This reverts commit 5b88cb3398.
2018-02-12 21:06:10 -08:00
Chris Douglas 5b88cb3398 HADOOP-15195. With SELinux enabled, directories mounted with start-build-env.sh may not be accessible. Contributed by Grigori Rybkine 2018-02-12 21:00:47 -08:00
Wangda Tan 60f9e60b3b Preparing for 3.2.0 development
Change-Id: I6d0e01f3d665d26573ef2b957add1cf0cddf7938
2018-02-11 11:17:38 +08:00
Steve Loughran 996796f104 HADOOP-15214. Make Hadoop compatible with Guava 21.0.
Contributed by Igor Dvorzhak
2018-02-08 10:55:54 -08:00
Anu Engineer 4304fcd5bd HDFS-12990. Change default NameNode RPC port back to 8020. Contributed by Xiao Chen. 2018-02-06 13:43:45 -08:00
Kihwal Lee 2dd960de98 HADOOP-15212. Add independent secret manager method for logging expired tokens. Contributed by Daryn Sharp. 2018-02-06 15:33:18 -06:00
Jason Lowe 33e6cdbd16 YARN-7876. Localized jars that are expanded after localization are not fully copied. Contributed by Miklos Szegedi 2018-02-05 12:30:20 -06:00
Hanisha Koneru 51cb6c5380 HADOOP-15168. Add kdiag tool to hadoop command. Contributed by Bharat Viswanadham. 2018-02-02 12:51:27 -08:00
Jason Lowe 460d77bd64 HADOOP-15170. Add symlink support to FileUtil#unTarUsingJava. Contributed by Ajay Kumar 2018-02-02 11:33:26 -06:00
Xiao Chen 09dd709d6e HADOOP-15197. Remove tomcat from the Hadoop-auth test bundle. 2018-02-01 15:33:52 -08:00
Xiaoyu Yao af015c0b23 HDFS-13060. Adding a BlacklistBasedTrustedChannelResolver for TrustedChannelResolver. Contributed by Ajay Kumar. 2018-01-31 22:39:51 -08:00
Eric Yang 0bee3849e3 YARN-7816. Allow same application name submitted by multiple users. (Contributed by Gour Saha) 2018-01-31 20:51:40 -05:00
Jason Lowe 7288b08330 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-31 09:39:43 -06:00
Jason Lowe 901d15a30b Revert "YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi"
This reverts commit 1b0f265db1.
2018-01-30 08:34:39 -06:00
Xiao Chen b63dcd583f HDFS-12974. Exception message is not printed when creating an encryption zone fails with AuthorizationException. Contributed by fang zhenyi. 2018-01-28 22:19:49 -08:00
Haibo Chen 649ef7ac33 YARN-7064. Use cgroup to get container resource utilization. (Miklos Szegedi via Haibo Chen) 2018-01-26 16:27:31 -08:00
Jason Lowe 1b0f265db1 YARN-2185. Use pipes when localizing archives. Contributed by Miklos Szegedi 2018-01-26 13:25:20 -06:00
Chris Douglas 56872cff92 MapFile.fix creates a wrong index file in case of block-compressed data file. Contributed by Grigori Rybkine 2018-01-26 09:18:30 -08:00
Wei Yan 0c139d5bcf HDFS-13049. RBF: Inconsistent Router OPTS config in branch-2 and branch-3. Contributed by Wei Yan. 2018-01-25 11:36:40 -08:00
Billie Rinaldi e307edcb47 YARN-7540 and YARN-7605. Convert yarn app cli to call yarn api services and implement doAs for Api Service REST API. Contributed by Eric Yang and Jian He 2018-01-23 17:54:39 -08:00
Hanisha Koneru 6347b2253d HDFS-13023. Journal Sync does not work on a secure cluster. Contributed by Bharat Viswanadham. 2018-01-22 16:02:32 -08:00
Hanisha Koneru 3fde0f1db5 HADOOP-15121. Encounter NullPointerException when using DecayRpcScheduler. Contributed by Tao Jie. 2018-01-22 15:54:44 -08:00
Masahiro Tanaka 6e27b20b4d
HADOOP-15181. Typo in SecureMode.md
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2018-01-22 17:12:36 +09:00
Vrushali C c191538ed1 HADOOP-15166 CLI MiniCluster fails with ClassNotFoundException o.a.h.yarn.server.timelineservice.collector.TimelineCollectorManager. Contributed by Gera Shegalov 2018-01-19 16:15:55 -08:00
Steve Loughran d689b2d99c HADOOP-15114. Add closeStreams(...) to IOUtils (addendum).
Contributed by Ajay Kumar.
2018-01-19 14:54:13 +00:00
Steve Loughran e5a1ad6e24 HADOOP-14788. Credentials readTokenStorageFile to stop wrapping IOEs in IOEs.
Contributed by Ajay Kumar.
2018-01-19 14:48:45 +00:00
Steve Loughran de630708d1 HADOOP-15123. KDiag tries to load krb5.conf from KRB5CCNAME instead of KRB5_CONFIG.
Contributed by Vipin Rathor.

(cherry picked from commit 1ef906e29e)
(cherry picked from commit f61edab1d0)
2018-01-18 12:35:57 +00:00
Steve Loughran f274fe33ea Revert "HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits."
This reverts commit 35ad9b1dd2.
2018-01-18 12:35:57 +00:00
Brahma Reddy Battula 08332e12d0 HADOOP-15150. in FsShell, UGI params should be overidden through env vars(-D arg). Contributed by Brahma Reddy Battula. 2018-01-18 10:54:32 +05:30
Aaron Fabbri 268ab4e027
HADOOP-15141 Support IAM Assumed roles in S3A. Contributed by Steve Loughran. 2018-01-17 00:05:24 -08:00
Yufei Gu b202935353 HADOOP-15157. Zookeeper authentication related properties to support CredentialProviders. (Contributed by Gergo Repas) 2018-01-16 10:15:22 -08:00
Arpit Agarwal 7016dd44e0 HDFS-13016. globStatus javadoc refers to glob pattern as "regular expression". Contributed by Mukul Kumar Singh. 2018-01-13 07:19:30 -08:00
Arpit Agarwal addbcd8cd4 HADOOP-15114. Add closeStreams(...) to IOUtils. Contributed by Ajay Kumar. 2018-01-11 16:50:21 -08:00
Steve Loughran 2e0a451a8d HADOOP-15033. Use java.util.zip.CRC32C for Java 9 and above
Contributed by Dmitry Chuyko,
2018-01-11 10:47:50 +00:00
Miklos Szegedi 12d0645990 HADOOP-15060. TestShellBasedUnixGroupsMapping.testFiniteGroupResolutionTime flaky. Contributed by Miklos Szegedi. 2018-01-10 10:52:26 -08:00
Haibo Chen 2f6c038be6 YARN-7602. NM should reference the singleton JvmMetrics instance. 2018-01-03 09:41:26 -08:00
Arpit Agarwal 4ad39ec308 Revert "HADOOP-10054. ViewFsFileStatus.toString() is broken. Contributed by Hanisha Koneru."
This reverts commit 37efa67e37.
2018-01-02 17:34:29 -08:00
Arpit Agarwal d82874851e HADOOP-15155. Error in javadoc of ReconfigurableBase#reconfigureProperty. Contributed by Ajay Kumar. 2018-01-02 17:22:55 -08:00
Arpit Agarwal dfe0cd8655 HADOOP-15152. Typo in javadoc of ReconfigurableBase#reconfigurePropertyImpl. Contributed by Nanda kumar. 2018-01-02 10:50:13 -08:00
Chris Douglas 7fe6f83c8f HADOOP-15117. open(PathHandle) contract test should be exhaustive for default options 2017-12-30 17:58:35 -08:00
Xiao Chen 81127616c5 HADOOP-15149. CryptoOutputStream should implement StreamCapabilities. 2017-12-29 13:41:15 -08:00
Jitendra Pandey d31c9d8c49 HADOOP-15143. NPE due to Invalid KerberosTicket in UGI. Contributed by Mukul Kumar Singh. 2017-12-27 23:17:07 -08:00
Steve Loughran c8ff0cc304 HADOOP-13282. S3 blob etags to be made visible in S3A status/getFileChecksum() calls.
Contributed by Steve Loughran
2017-12-21 14:58:58 +00:00
Junping Du a7f8caf58e Add 2.8.3 release jdiff files.
(cherry picked from commit c89f99aade)
2017-12-18 22:39:01 -08:00
Steve Loughran 35ad9b1dd2 HADOOP-13974. S3Guard CLI to support list/purge of pending multipart commits.
Contributed by Aaron Fabbri
2017-12-18 21:19:06 +00:00
Chris Douglas 5e81f32d11 HADOOP-15106. FileSystem::open(PathHandle) should throw a specific exception on validation failure 2017-12-16 10:53:10 -08:00
Virajith Jalaparti b668eb9155 HDFS-10675. Datanode support to read from external stores. 2017-12-15 17:51:37 -08:00
Xiaoyu Yao 37efa67e37 HADOOP-10054. ViewFsFileStatus.toString() is broken. Contributed by Hanisha Koneru. 2017-12-14 10:58:16 -08:00
Jason Lowe f8af0e2feb HADOOP-15085. Output streams closed with IOUtils suppressing write errors. Contributed by Jim Brennan 2017-12-14 10:18:08 -06:00
Yiqun Lin 91c96bdf8e HDFS-12883. RBF: Document Router and State Store metrics. Contributed by Yiqun Lin. 2017-12-14 16:00:53 +08:00
Kai Zheng f86c81d923 HADOOP-15039. Move SemaphoredDelegatingExecutor to hadoop-common. Contributed by Genmao Yu 2017-12-14 11:14:52 +08:00
Andrew Wang d447152d49 Update CHANGES, RELEASENOTES, jdiff for 3.0.0 release. 2017-12-13 13:05:32 -08:00
Chris Douglas 693169ef34 HDFS-12882. Support full open(PathHandle) contract in HDFS 2017-12-11 20:14:15 -08:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.) f19638333b HADOOP-15059. Undoing the switch of Credentials to PB format as default - done via HADOOP-12563 for supporting 2.x to 3.x upgrades. 2017-12-08 08:00:21 -08:00
Xiao Chen 19e0894209 HADOOP-15056. Fix TestUnbuffer#testUnbufferException failure. Contributed by Jack Bearden. 2017-12-07 21:05:55 -08:00
Sunil G daa1cdd062 YARN-7564. Cleanup to fix checkstyle issues of YARN-5881 branch. Contributed by Sunil G. 2017-12-07 18:56:54 -08:00
Wangda Tan 5e798b1a0d YARN-6471. Support to add min/max resource configuration for a queue. (Sunil G via wangda)
Change-Id: I9213f5297a6841fab5c573e85ee4c4e5f4a0b7ff
2017-12-07 18:56:54 -08:00
Sean Mackrory acb92904d0 HADOOP-15098. TestClusterTopology#testChooseRandom fails intermittently. Contributed by Zsolt Venczel. 2017-12-07 12:30:58 -07:00
Lei Xu 67662e2ac9 HDFS-12840. Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog. Contributed by Lei (Eddy) Xu. 2017-12-07 11:15:40 -08:00
Arpit Agarwal e00c7f78c1 HADOOP-14976. Set HADOOP_SHELL_EXECNAME explicitly in scripts. 2017-12-04 21:02:04 -08:00
Xiao Chen 404eab4dc0 HDFS-12396. Webhdfs file system should get delegation token from kms provider. Contributed by Rushabh S Shah. 2017-12-04 10:40:11 -08:00
Chris Douglas 42307e3c3a HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 22:34:30 -08:00
Chris Douglas f9d195dfe9 HADOOP-14600. LocatedFileStatus constructor forces RawLocalFS to exec a process to get the permissions. Contributed by Ping Liu 2017-12-01 22:12:26 -08:00
Chris Douglas 53bbef3802 Revert "HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak"
This reverts commit 5304698dc8.
2017-12-01 11:19:38 -08:00
Chris Douglas 5304698dc8 HDFS-11576. Block recovery will fail indefinitely if recovery time > heartbeat interval. Contributed by Lukas Majercak 2017-12-01 10:29:30 -08:00
Chris Douglas 0780fdb1eb HDFS-12877. Add open(PathHandle) with default buffersize 2017-11-30 15:13:16 -08:00
Robert Kanter 75a3ab88f5 HADOOP-13493. Compatibility Docs should clarify the policy for what takes precedence when a conflict is found (templedf via rkanter) 2017-11-30 07:39:15 -08:00
Chris Douglas 0e560f3b8d HDFS-12681. Make HdfsLocatedFileStatus a subtype of LocatedFileStatus 2017-11-29 20:28:06 -08:00
Xiao Chen d162252d7a HADOOP-15067. GC time percentage reported in JvmMetrics should be a gauge, not counter. Contributed by Misha Dmitriev. 2017-11-23 09:01:28 -08:00
Steve Loughran de8b6ca5ef HADOOP-13786 Add S3A committer for zero-rename commits to S3 endpoints.
Contributed by Steve Loughran and Ryan Blue.
2017-11-22 15:28:12 +00:00
Robert Kanter f2efaf013f HADOOP-14982. Clients using FailoverOnNetworkExceptionRetry can go into a loop if they're used without authenticating with kerberos in HA env (pbacsko via rkanter) 2017-11-16 11:11:19 -08:00
Chris Douglas 675e9a8f57 Revert "HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus."
This reverts commit b85603e3f8.
2017-11-15 19:20:37 -08:00
Xiao Chen b1941b200d HADOOP-15023. ValueQueue should also validate (int) (lowWatermark * numValues) > 0 on construction. 2017-11-15 16:44:06 -08:00
Daniel Templeton 69043ba8b5 HADOOP-14876. Create downstream developer docs from the compatibility guidelines 2017-11-15 10:03:29 -08:00
Arun Suresh b07e68b02a Addendum patch for Configuration fix. (Jason Lowe via asuresh) 2017-11-13 14:03:50 -08:00
Eric Yang 1b68b8ff2c HADOOP-15008. Fixed period unit calculation for Hadoop Metrics V2. (Contribute by Erik Krogen) 2017-11-13 12:42:43 -05:00
Akira Ajisaka 975a57a688
HADOOP-15031. Fix javadoc issues in Hadoop Common. Contributed by Mukul Kumar Singh. 2017-11-13 23:12:23 +09:00
Chris Douglas 796a0d3a5c HADOOP-8522. ResetableGzipOutputStream creates invalid gzip files when finish() and resetState() are used. Contributed by Mike Percy 2017-11-10 17:41:29 -08:00
Xiao Chen 3c6adda291 HADOOP-14960. Add GC time percentage monitor/alerter. Contributed by Misha Dmitriev. 2017-11-09 21:06:06 -08:00
Arun Suresh 5eb7dbe9b3 Fixing Job History Server Configuration parsing. (Jason Lowe via asuresh) 2017-11-09 15:15:51 -08:00
John Zhuge 6c32ddad30 HADOOP-14872. CryptoInputStream should implement unbuffer. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
John Zhuge bf6a660232 HADOOP-15012. Add readahead, dropbehind, and unbuffer to StreamCapabilities. Contributed by John Zhuge. 2017-11-09 10:16:12 -08:00
Mingliang Liu b6d8f87b8e HADOOP-15015. TestConfigurationFieldsBase to use SLF4J for logging. Contributed by Steve Loughran 2017-11-03 21:01:56 -07:00
Xiaoyu Yao 59d78a5088 HADOOP-14987. Improve KMSClientProvider log around delegation token checking. Contributed by Xiaoyu Yao and Xiao Chen. 2017-11-03 16:10:37 -07:00
Chris Douglas b85603e3f8 HDFS-12681. Fold HdfsLocatedFileStatus into HdfsFileStatus. 2017-11-03 14:30:57 -07:00
Chris Douglas d015e0bbd5 HDFS-7878. API - expose a unique file identifier. 2017-10-31 09:44:01 -07:00
Sean Mackrory 81225430d0 HDFS-206. Support for head in FSShell. Contributed by Gabor Bota. 2017-10-31 08:34:01 -06:00
Jason Lowe 2fae63aa60 HADOOP-14919. BZip2 drops records when reading data in splits. Contributed by Jason Lowe 2017-10-31 09:30:13 -05:00
Junping Du a25b5aa0cf Add several jdiff xml files for 2.8.2 release. 2017-10-26 19:30:48 -07:00
Junping Du 1ad863e220 Update CHANGES.md and RELEASENOTES for 2.8.2 release. 2017-10-26 19:13:58 -07:00
Xiao Chen f66ad1fab9 HADOOP-14957. ReconfigurationTaskStatus is exposing guava Optional in its public api. Contributed by Haibo Chen, Xiao Chen. 2017-10-25 14:11:30 -07:00
Akira Ajisaka 4c970088af
HADOOP-14030. PreCommit TestKDiag failure. Contributed by Wei-Chiu Chuang. 2017-10-25 17:17:21 +09:00
Varun Saxena 67e7673750 HADOOP-9657. NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports. Contributed by Varun Saxena. 2017-10-25 03:06:12 +05:30
John Zhuge c904d60c38 Revert "HADOOP-14954. MetricsSystemImpl#init should increment refCount when already initialized. Contributed by John Zhuge."
This reverts commit 14b3c2695b.
2017-10-23 15:32:25 -07:00
Naganarasimha 6b8122458e HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt. 2017-10-23 23:42:06 +05:30
Arpit Agarwal 4b00c9a47f HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham. 2017-10-23 10:47:16 -07:00
John Zhuge 14b3c2695b HADOOP-14954. MetricsSystemImpl#init should increment refCount when already initialized. Contributed by John Zhuge. 2017-10-20 16:13:41 -07:00
Kai Zheng ce7cf66e5e HDFS-12448. Make sure user defined erasure coding policy ID will not overflow. Contributed by Huafeng Wang 2017-10-20 09:42:04 +08:00
Wei-Chiu Chuang 97c70c7ac6 HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota. 2017-10-19 06:02:13 -07:00
Junping Du b016f08f67 HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du. 2017-10-18 15:06:30 -07:00
Xiao Chen 86ee0c5e4e HADOOP-14944. Add JvmMetrics to KMS. 2017-10-17 19:06:45 -07:00
Lei Xu 31ebccc962 HDFS-12613. Native EC coder should implement release() as idempotent function. (Lei (Eddy) Xu) 2017-10-16 19:44:30 -07:00
Wei-Chiu Chuang e906108fc9 HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar. 2017-10-16 16:42:59 -07:00
Xiao Chen b7ff624c76 HADOOP-14949. TestKMS#testACLs fails intermittently. 2017-10-16 14:19:31 -07:00
Manoj Govindassamy 133d7ca76e HADOOP-13055. Implement linkMergeSlash and linkFallback for ViewFileSystem 2017-10-13 17:43:21 -07:00
Robert Kanter e163f41850 HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter) 2017-10-13 13:52:58 -07:00
Anu Engineer 075358eb6f HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin. 2017-10-11 15:58:20 -07:00
Arun Suresh b6c2c9058e HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh) 2017-10-11 15:21:21 -07:00
Steve Loughran 3de574413c HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB.
Contributed by Varada Hemeswari.
2017-10-11 18:06:43 +01:00
Jason Lowe 1123f8f0b6 HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp 2017-10-10 14:47:25 -05:00
Jitendra Pandey d6602b5f39 HDFS-11575. Supporting HDFS NFS gateway with Federated HDFS. Contributed by Mukul Kumar Singh. 2017-10-10 10:38:05 -07:00
John Zhuge 9676ed19be HADOOP-14184. Remove service loader config entry for ftp fs. Contributed by Sen Zhao. 2017-10-09 00:33:17 -07:00
Xiaoyu Yao 2b08a1fc64 HADOOP-14920. KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. Contributed by Xiaoyu Yao. 2017-10-06 10:12:24 -07:00
Xiao Chen 25f31d9fc4 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-10-05 19:46:47 -07:00
Akira Ajisaka fe84cdc9da
HADOOP-14616. Client fails to read a block with erasure code (XOR, native) when one of the data block is lost. Contributed by Huafeng Wang. 2017-10-05 11:06:09 +09:00
Daniel Templeton 20e9ce3ab3 HADOOP-14459. SerializationFactory shouldn't throw a NullPointerException if the serializations list is not defined
(Contributed by Nandor Kollar via Daniel Templeton)
2017-10-04 15:04:02 +02:00
Andrew Wang b34b3ff94b Add 3.0.0-beta1 CHANGES and RELEASENOTES. 2017-10-03 17:14:40 -07:00
Akira Ajisaka c9b525bd94
YARN-6943. Update Yarn to YARN in documentation. Contributed by Chetna Chaudhari. 2017-10-04 07:01:54 +09:00
Xiaoyu Yao 107c177782 HDFS-12455. WebHDFS - Adding "snapshot enabled" status to ListStatus query result. Contributed by Ajay Kumar. 2017-10-03 13:02:12 -07:00
Arpit Agarwal b91305119b HDFS-11968. ViewFS: StoragePolicies commands fail with HDFS federation. Contributed by Mukul Kumar Singh. 2017-10-03 11:23:40 -07:00
Allen Wittenauer 4d5dd75b60
HADOOP-14908. CrossOriginFilter should trigger regex on more input (Johannes Alberti via aw) 2017-10-03 10:58:28 -07:00
Anu Engineer 442ea85545 HADOOP-14752. TestCopyFromLocal#testCopyFromLocalWithThreads is fleaky. Contributed by Andras Bokor. 2017-10-02 13:11:47 -07:00
Chen Liang 563dcdfc1d HADOOP-14915. method name is incorrect in ConfServlet. Contributed by Bharat Viswanadham. 2017-10-02 10:58:19 -07:00