Varun Saxena
0c8a2c2609
HADOOP-9657. NetUtils.wrapException to have special handling for 0.0.0.0 addresses and :0 ports. Contributed by Varun Saxena.
...
(cherry picked from commit 67e7673750
)
2017-10-25 03:09:07 +05:30
Naganarasimha
2b04ae918c
HADOOP-14966. Handle JDK-8071638 for hadoop-common. Contributed by Bibin A Chundatt.
...
(cherry picked from commit 6b8122458e
)
2017-10-23 23:47:33 +05:30
Arpit Agarwal
a373094bf9
HDFS-12683. DFSZKFailOverController re-order logic for logging Exception. Contributed by Bharat Viswanadham.
2017-10-23 10:47:35 -07:00
Xiao Chen
a6370dde90
HADOOP-14944. Add JvmMetrics to KMS.
2017-10-19 22:37:35 -07:00
Wei-Chiu Chuang
9eb33d0e87
HADOOP-14880. [KMS] Document&test missing KMS client side configs. Contributed by Gabor Bota.
...
(cherry picked from commit 97c70c7ac6
)
Conflicts:
hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
2017-10-19 06:06:01 -07:00
Junping Du
e7e2b8158d
HADOOP-14958. Fix source-level compatibility after HADOOP-11252. Contributed by Junping Du.
...
(cherry picked from commit b016f08f67
)
(cherry picked from commit 9433f9eb09
)
2017-10-18 15:07:59 -07:00
Wangda Tan
d894f910d0
YARN-6608. Backport all SLS improvements from trunk to branch-2. (Carlo Curino via wangda)
2017-10-17 19:45:09 -07:00
Rohith Sharma K S
4e1af68ae1
YARN-7038. [Atsv2 Security] CollectorNodemanagerProtocol RPC interface doesn't work when service authorization is enabled. Contributed by Varun Saxena.
...
(cherry picked from commit 387dd3da87759860dd18bd8aa95c60d962b1e6a9)
2017-10-17 23:36:39 +05:30
Sangjin Lee
e212e562ec
YARN-5355. Backported YARN-2928 into our branch-2 feature branch.
2017-10-17 23:36:33 +05:30
Xiao Chen
a103b323e6
Revert "HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar."
...
This reverts commit 1452ba139e
.
2017-10-17 10:11:38 -07:00
Robert Kanter
8a1ed84c77
HADOOP-14938. Configuration.updatingResource map should be initialized lazily (misha@cloudera.com via rkanter)
2017-10-16 16:51:28 -07:00
Wei-Chiu Chuang
1452ba139e
HADOOP-14948. Document missing config key hadoop.treat.subject.external. Contributed by Ajay Kumar.
...
(cherry picked from commit e906108fc9
)
2017-10-16 16:44:35 -07:00
Anu Engineer
5933068e70
HADOOP-13102. Update GroupsMapping documentation to reflect the new changes. Contributed by Esther Kundin.
...
(cherry picked from commit 075358eb6f
)
2017-10-11 16:15:59 -07:00
Arun Suresh
fb8bd4d2d7
HADOOP-13556. Change Configuration.getPropsWithPrefix to use getProps instead of iterator. (Larry McCay via asuresh)
...
(cherry picked from commit b6c2c9058e
)
2017-10-11 15:22:51 -07:00
Steve Loughran
ccfd1a4a56
HADOOP-14913. Sticky bit implementation for rename() operation in Azure WASB.
...
Contributed by Varada Hemeswari.
Cherry-picked from commit ref 3de574413c
2017-10-11 18:09:13 +01:00
Jason Lowe
63d86ee3a5
HADOOP-14912. FairCallQueue may defer servicing calls. Contributed by Daryn Sharp
...
(cherry picked from commit 1123f8f0b6
)
2017-10-10 14:50:15 -05:00
Xiaoyu Yao
65b5e81752
HADOOP-14920. KMSClientProvider won't work with KMS delegation token retrieved from non-Java client. Contributed by Xiaoyu Yao.
...
(cherry picked from commit 2b08a1fc64
)
2017-10-06 10:39:37 -07:00
Xiao Chen
dd0113d777
HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah.
...
(cherry picked from commit 212342d6db14673f8f5652a5209e1c4fcc0f0ede)
2017-10-05 19:47:04 -07:00
Wangda Tan
73d1b02d5a
HADOOP-13835. Move Google Test Framework code from mapreduce to hadoop-common. (Varun Vasudev via wangda)
...
Change-Id: I1031a4fa4e0c5c76928d03c5e61971978ec7e6ab
2017-10-05 17:02:00 -07:00
Steve Loughran
a1afc6aa59
HADOOP-13345 S3Guard: Improved Consistency for S3A.
...
Contributed by: Chris Nauroth, Aaron Fabbri, Mingliang Liu, Lei (Eddy) Xu,
Sean Mackrory, Steve Loughran and others.
(cherry-picked from/based on commit 621b43e254
)
2017-09-29 11:29:22 +01:00
Jason Lowe
11ac10fe1d
HADOOP-14902. LoadGenerator#genFile write close timing is incorrectly calculated. Contributed by Hanisha Koneru
...
(cherry picked from commit 6f789fe057
)
2017-09-28 16:40:18 -05:00
Jason Lowe
0aac5af6e1
HADOOP-14891. Remove references to Guava Objects.toStringHelper. Contributed by Jonathan Eagles
2017-09-26 10:08:06 -05:00
Akira Ajisaka
b017af2391
HADOOP-14893. WritableRpcEngine should use Time.monotonicNow. Contributed by Chetna Chaudhari.
...
(cherry picked from commit d08b8c801a
)
2017-09-26 16:47:50 +09:00
Anu Engineer
17b17aed0f
HADOOP-14901. ReuseObjectMapper in Hadoop Common. Contributed by Hanisha Koneru.
2017-09-25 13:44:51 -07:00
Jason Lowe
09be0acae0
HADOOP-14881. LoadGenerator should use Time.monotonicNow() to measure durations. Contributed by Bharat Viswanadham
...
(cherry picked from commit ac05a51bbb
)
2017-09-25 15:38:52 -05:00
Akira Ajisaka
6004fb4676
HADOOP-14892. MetricsSystemImpl should use Time.monotonicNow for measuring durations. Contributed by Chetna Chaudhari.
...
(cherry picked from commit 3a10367a17
)
2017-09-26 01:48:24 +09:00
Akira Ajisaka
6581f2dea3
HADOOP-7308. Remove unused TaskLogAppender configurations from log4j.properties. Contributed by Todd Lipcon and J.Andreina.
...
(cherry picked from commit 7e58b2478c
)
2017-09-20 21:08:54 +09:00
Robert Kanter
4e36452d02
YARN-7162. Remove XML excludes file format (rkanter)
2017-09-15 09:53:55 -07:00
John Zhuge
eaef8446f6
HADOOP-14864. FSDataInputStream#unbuffer UOE should include stream class name. Contributed by Bharat Viswanadham.
...
(cherry picked from commit 7ee02d1065
)
2017-09-14 11:29:34 -07:00
Xiao Chen
0a8ed025e4
Revert "HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah."
...
This reverts commit 5337c2511d
.
2017-09-12 19:53:14 -07:00
Jason Lowe
c314a66353
HADOOP-14843. Improve FsPermission symbolic parsing unit test coverage. Contributed by Bharat Viswanadham
...
(cherry picked from commit 86f4d1c66c
)
Conflicts:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/permission/TestFsPermission.java
2017-09-12 17:00:22 -05:00
Steve Loughran
a46204437b
HADOOP-14851 LambdaTestUtils.eventually() doesn't spin on Assertion failures.
...
Contributed by Steve Loughran
(cherry picked from commit 180e814b08
)
2017-09-11 21:33:44 +01:00
Xuan
02ae3f8202
YARN-7072. Add a new log aggregation file format controller (xgong)
2017-09-10 23:18:31 -07:00
John Zhuge
1421196d20
HADOOP-14844. Remove requirement to specify TenantGuid for MSI Token Provider. Contributed by Atul Sikaria.
...
(cherry picked from commit a4661850c1
)
2017-09-08 14:15:51 -07:00
Jason Lowe
f39a444120
HADOOP-14827. Allow StopWatch to accept a Timer parameter for tests. Contributed by Erik Krogen
...
(cherry picked from commit dd814946f6
)
2017-09-06 16:05:58 -05:00
Wei-Chiu Chuang
aada99a376
HADOOP-14688. Intern strings in KeyVersion and EncryptedKeyVersion. Contributed by Xiao Chen.
...
(cherry picked from commit ad32759fd9
)
(cherry picked from commit c16ecf0fe8
)
2017-09-05 15:14:13 -07:00
Yiqun Lin
41d8e4e9b3
HDFS-12317. HDFS metrics render error in the page of Github. Contributed by Yiqun Lin.
2017-09-01 10:13:01 +08:00
Steve Loughran
25e9c29608
HADOOP-14802. Add support for using container saskeys for all accesses.
...
Contributed by Sivaguru Sankaridurg
(cherry picked from commit 021974f4cb
)
Conflicts:
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
2017-08-30 22:40:50 +01:00
Junping Du
7dd5bb9492
HADOOP-14814. Fix incompatible API change on FsServerDefaults to HADOOP-14104. Contributed by Junping Du.
...
(cherry picked from commit 41480233a9
)
2017-08-30 13:30:48 -07:00
Arpit Agarwal
ae69c99068
HDFS-11577. Combine the old and the new chooseRandom for better performance. Contributed by Chen Liang.
2017-08-24 11:17:36 -07:00
John Zhuge
6e8c696651
HADOOP-14251. Credential provider should handle property key deprecation. Contributed by John Zhuge.
...
(cherry picked from commit 7e6463d2fb
)
Conflicts:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
2017-08-23 11:59:47 -07:00
Subru Krishnan
a45ffdcddc
YARN-7053. Move curator transaction support to ZKCuratorManager. (Jonathan Hung via Subru).
...
(cherry picked from commit 4249172e14
)
2017-08-22 19:21:35 -07:00
Jason Lowe
45c1f8ac8e
HADOOP-14687. AuthenticatedURL will reuse bad/expired session cookies. Contributed by Daryn Sharp
...
(cherry picked from commit c379310212
)
2017-08-22 16:53:51 -05:00
Arpit Agarwal
b3ea11dfdb
HDFS-11482. Add storage type demand to into DFSNetworkTopology#chooseRandom. Contributed by Chen Liang.
2017-08-21 14:07:59 -07:00
Arpit Agarwal
d5379cc564
HDFS-12325. SFTPFileSystem operations should restore cwd. Contributed by Chen Liang.
2017-08-20 23:41:16 -07:00
Arpit Agarwal
d8d2a221f8
HDFS-11450. HDFS specific network topology classes with storage type info included. Contributed by Chen Liang.
2017-08-18 11:43:17 -07:00
Arpit Agarwal
850593133b
Revert "HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru."
...
This reverts commit bdc425057e
.
2017-08-18 10:16:13 -07:00
Mingliang Liu
03d15fa58e
HDFS-11430. Separate class InnerNode from class NetworkTopology and make it extendable. Contributed by Tsz Wo Nicholas Sze
2017-08-16 16:06:13 -07:00
Steve Loughran
d5680c08ed
HADOOP-14660. wasb: improve throughput by 34% when account limit exceeded.
...
Contributed by Thomas Marquardt.
2017-08-16 13:49:47 +01:00
Subru Krishnan
ad46a90872
HADOOP-14773. Extend ZKCuratorManager API for more reusability. (Íñigo Goiri via Subru).
...
(cherry picked from commit 75dd866bfb
)
2017-08-15 16:54:48 -07:00
Arpit Agarwal
bdc425057e
HADOOP-14732. ProtobufRpcEngine should use Time.monotonicNow to measure durations. Contributed by Hanisha Koneru.
2017-08-14 15:53:43 -07:00
Subru Krishnan
a70efb6138
HADOOP-14741. Refactor curator based ZooKeeper communication into common library. (Íñigo Goiri via Subru).
2017-08-14 11:03:50 -07:00
John Zhuge
0efc590e6d
HADOOP-14627. Support MSI and DeviceCode token provider in ADLS. Contributed by Atul Sikaria.
...
(cherry picked from commit 7769e96149
)
2017-08-13 00:27:33 -07:00
John Zhuge
e208a4e7ee
HADOOP-14260. Configuration.dumpConfiguration should redact sensitive information. Contributed by John Zhuge.
...
(cherry picked from commit 582648befa
)
2017-08-11 10:54:41 -07:00
John Zhuge
76c4aee2e3
HADOOP-13628. Support to retrieve specific property from configuration via REST API. Contributed by Weiwei Yang
...
(cherry picked from commit 00160f71b6
)
2017-08-11 10:54:30 -07:00
John Zhuge
9ba3268fa3
HADOOP-13588. ConfServlet should respect Accept request header. Contributed by Weiwei Yang
...
(cherry picked from commit 59d59667a8
)
2017-08-11 10:54:25 -07:00
Steve Loughran
9c6e8ece98
HADOOP-14754. TestCommonConfigurationFields failed: core-default.xml has 2 wasb properties missing in classes.
...
Contributed by John Zhuge.
(cherry picked from commit d964062f66
)
2017-08-11 10:58:26 +01:00
Wei-Chiu Chuang
304cc1a856
HADOOP-14743. CompositeGroupsMapping should not swallow exceptions. Contributed by Wei-Chiu Chuang.
...
(cherry picked from commit a8b75466b2
)
2017-08-10 09:36:21 -07:00
Steve Loughran
96befde480
HADOOP-14183. Remove service loader config file for wasb fs.
...
Contributed by Esfandiar Manii.
(cherry picked from commit 54356b1e83
)
2017-08-10 16:49:51 +01:00
Haohui Mai
3e2753daa1
HADOOP-14598. Blacklist Http/HttpsFileSystem in FsUrlStreamHandlerFactory. Contributed by Steve Loughran.
2017-08-08 16:33:54 -07:00
Jian He
ad1567c487
YARN-6890. Not display killApp button on UI if UI is unsecured but cluster is secured. Contributed by Junping Du
2017-08-08 11:10:29 -07:00
Yiqun Lin
817c0f4564
HDFS-12198. Document missing namenode metrics that were added recently. Contributed by Yiqun Lin.
2017-08-08 19:30:03 +08:00
Arpit Agarwal
2906c494c5
HDFS-12264. DataNode uses a deprecated method IoUtils#cleanup. Contributed by Ajay Yadav.
2017-08-07 15:33:36 -07:00
Xiao Chen
050c50e3c5
HADOOP-14727. Socket not closed properly when reading Configurations with BlockReaderRemote. Contributed by Jonathan Eagles.
...
(cherry picked from commit 9a755e6da7158783789f55796644578330653efb)
Conflicts:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java
2017-08-07 10:26:10 -07:00
Lei Xu
d87a92bf55
HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu)
...
(cherry picked from commit fe3341786a
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md
2017-08-04 11:25:21 -07:00
wenxinhe
11e1e0c275
HADOOP-14706. Adding a helper method to determine whether a log is Log4j implement.
...
This closes #258
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-08-04 14:42:35 +09:00
Andrew Wang
2da20df1dd
HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen.
2017-08-03 15:45:21 -07:00
Steve Loughran
cb46792610
HADOOP-14709. Fix checkstyle warnings in ContractTestUtils.
...
Contributed by Thomas Marquardt.
2017-08-02 12:46:30 +01:00
Masatake Iwasaki
d9a07b3904
HADOOP-14702. Fix formatting issue and regression caused by conversion from APT to Markdown. Contributed by Doris Gu.
2017-08-02 06:21:37 +00:00
Andrew Wang
5467d1a4d1
HADOOP-14701. Configuration can log misleading warnings about an attempt to override final parameter. Contributed by Andrew Sherman.
...
(cherry picked from commit a11c230236
)
Conflicts:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestConfiguration.java
2017-08-01 15:18:18 -07:00
Arpit Agarwal
d19504866c
HDFS-14676. Wrong default value for fs.df.interval. Contributed by xiangguang zheng.
2017-08-01 13:31:55 -07:00
Lei Xu
f09d20cffb
HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu)
...
(cherry picked from commit 667ee003bf47e44beb3fdff8d06a7264a13dd22c)
2017-07-31 20:07:13 -07:00
Akira Ajisaka
9c8a296169
HADOOP-14690. RetryInvocationHandler should override toString(). Contributed by Yeliang Cang.
...
(cherry picked from commit f14be0d241
)
2017-07-31 14:10:18 +09:00
Brahma Reddy Battula
8bfb9971ca
HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula.
2017-07-30 14:01:41 +08:00
Yongjun Zhang
c3e8c7d56e
HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang.
...
(cherry picked from commit c6330f22a5
)
2017-07-27 17:05:30 -07:00
Kihwal Lee
4edb849611
HADOOP-14578. Bind IPC connections to kerberos UPN host for proxy users. Contributed by Daryn Sharp.
...
(cherry picked from commit 27a1a5fde9
)
2017-07-26 13:16:34 -05:00
Jitendra Pandey
ab95086ed1
HADOOP-14518. Customize User-Agent header sent in HTTP/HTTPS requests by WASB. Contributed by Georgi Chalakov.
2017-07-25 08:31:35 -07:00
Akira Ajisaka
22f237c5d9
HADOOP-14681. Remove MockitoMaker class. Contributed by Andras Bokor.
...
(cherry picked from commit cca51e916b
)
2017-07-25 15:25:36 +09:00
Jason Lowe
db7c5636b6
YARN-6768. Improve performance of yarn api record toString and fromString. Contributed by Jonathan Eagles
...
(cherry picked from commit 24853bf32a
)
2017-07-24 10:22:31 -05:00
Allen Wittenauer
daa7685344
HADOOP-13178. TestShellBasedIdMapping.testStaticMapUpdate doesn't work on OS X (Kai Sasaki via aw)
...
(cherry picked from commit d8c1fd1944
)
2017-07-24 11:57:10 +09:00
wenxinhe
7e583a3824
HADOOP-14539. Move commons logging APIs over to slf4j in hadoop-common.
...
This closes #251
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2017-07-24 11:14:08 +09:00
Kihwal Lee
a735916b2c
HADOOP-14659. UGI getShortUserName does not need to search the Subject. Contributed by Daryn Sharp.
...
(cherry picked from commit b3269f7cc1
)
2017-07-20 16:15:06 -05:00
Daniel Templeton
1d629c95af
HADOOP-14666. Tests use assertTrue(....equals(...)) instead of assertEquals()
...
(cherry picked from commit c21c260392
)
2017-07-19 14:01:10 -07:00
Jitendra Pandey
23b920cd7a
HADOOP-14642. wasb: add support for caching Authorization and SASKeys. Contributed by Sivaguru Sankaridurg.
2017-07-19 11:37:36 -07:00
Jason Lowe
057631c031
HADOOP-14669. GenericTestUtils.waitFor should use monotonic time. Contributed by Daniel Templeton
...
(cherry picked from commit df180259b0
)
2017-07-19 09:42:38 -05:00
Jason Lowe
9b961db569
HADOOP-14637. GenericTestUtils.waitFor needs to check condition again after max wait time. Contributed by Daniel Templeton
...
(cherry picked from commit 5aa2bf231f
)
2017-07-18 16:27:34 -05:00
Xiao Chen
5337c2511d
HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah.
...
(cherry picked from commit aa395c1ec9a170f2da732c6e6114c60412161d40)
2017-07-14 22:23:07 -07:00
Masatake Iwasaki
956e83c100
HADOOP-14646. FileContextMainOperationsBaseTest#testListStatusFilterWithSomeMatches never runs. Contributed by Andras Bokor.
...
(cherry picked from commit b61ab8573e
)
2017-07-13 21:43:10 +09:00
Akira Ajisaka
2653cf4466
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor.
...
(cherry picked from commit 9144fd9e9b
)
2017-07-12 11:40:48 +09:00
Steve Loughran
41e83b2ca2
HADOOP-14535 wasb: implement high-performance random access and seek of block blobs.
...
Contributed by Thomas Marquardt
(cherry picked from commit d670c3a4da
)
2017-07-11 21:45:24 +01:00
wenxinhe
e494a169ed
HADOOP-14638. Replace commons-logging APIs with slf4j in StreamPumper.
...
This closes #247
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit fce7951014
)
2017-07-11 13:36:56 +09:00
Jitendra Pandey
b82485d6fe
HADOOP-10829. Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe. Contributed by Benoy Antony and Rakesh R.
2017-07-10 17:55:02 -07:00
Akira Ajisaka
1c40df5bcd
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He.
2017-07-08 03:01:18 +09:00
Wei-Chiu Chuang
730b21e9f9
HADOOP-14563. LoadBalancingKMSClientProvider#warmUpEncryptedKeys swallows IOException. Contributed by Rushabh S Shah.
...
(cherry picked from commit 8153fe2bd3
)
2017-07-07 06:14:07 -07:00
Vinayakumar B
1e8301b215
HADOOP-13414. Hide Jetty Server version header in HTTP responses. Contributed by Surendra Singth Lilhore.
2017-07-05 16:06:19 +05:30
Steve Loughran
3404824854
HADOOP-14617. Add ReflectionUtils.logThreadInfo that accept slf4j logger API.
...
Contributed by Wenxin He.
(cherry picked from commit 128d8bc3e9d792fde9cacd9e1d786db8ef682b0b)
2017-07-04 11:40:46 +01:00
Wenxin He
ca8102962f
HADOOP-14571. Deprecate public APIs relate to log4j1
...
This closes #244
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit f2aba1da30
)
2017-07-04 18:56:41 +09:00
Steve Loughran
4ceb3bd4b3
HADOOP-14615. Add ServiceOperations.stopQuietly that accept slf4j logger API.
...
Contributed by Wenxin He.
(cherry picked from commit 1aaa7f1eac
)
2017-07-04 10:49:20 +01:00
Daniel Templeton
367f525e81
HADOOP-14611. NetworkTopology.DEFAULT_HOST_LEVEL is unused
...
(Contributed by Chen Liang via Daniel Templeton)
(cherry picked from commit 5a75f73893
)
2017-06-29 16:56:55 -07:00
Akira Ajisaka
b59b96deee
HDFS-11992. Replace commons-logging APIs with slf4j in FsDatasetImpl. Contributed by hu xiaodong.
2017-06-27 03:28:09 +09:00
Akira Ajisaka
239b382153
HADOOP-14549. Use GenericTestUtils.setLogLevel when available in hadoop-tools. Contributed by wenxin he.
...
(cherry picked from commit a594f96fb1
)
2017-06-26 16:41:44 +09:00
Arpit Agarwal
4b420e0678
HADOOP-14543. ZKFC should use getAversion() while setting the zkacl. Contributed by Brahma Reddy Battula.
2017-06-23 16:50:56 -07:00
Steve Loughran
5858bc70a2
HADOOP-14568. GenericTestUtils#waitFor missing parameter verification.
...
Contributed by Yiqun Lin
(cherry picked from commit e5db9af8a0
)
2017-06-23 14:41:08 +01:00
Arpit Agarwal
f9f6ef83eb
HDFS-12010. TestCopyPreserveFlag fails consistently because of mismatch in access time. Contributed by Mukul Kumar Singh.
2017-06-22 15:43:00 -07:00
Akira Ajisaka
897d5a4d24
HADOOP-14542. Add IOUtils.cleanupWithLogger that accepts slf4j logger API. Contributed by Chen Liang.
...
(cherry picked from commit b64951905e
)
2017-06-22 17:43:33 +09:00
Yongjun Zhang
ba487ead42
HADOOP-14429. FTPFileSystem#getFsAction always returns FsAction.NONE. (Hongyuan Li via Yongjun Zhang)
...
(cherry picked from commit 5157f6c46e
)
2017-06-20 17:07:01 -07:00
Brahma Reddy Battula
933fad2e17
HADOOP-14533. Size of args cannot be less than zero in TraceAdmin#run as its linkedlist. Contributed by Weisen Han.
...
(cherry picked from commit 2e9daa2e27
)
2017-06-20 20:45:33 +08:00
Akira Ajisaka
6d82db40d5
HADOOP-14540. Replace MRv1 specific terms in HostsFileReader. Contributed by hu xiaodong.
...
(cherry picked from commit 300804595e
)
2017-06-19 19:40:19 +09:00
Akira Ajisaka
2d0b6ab65d
HADOOP-14538. Fix TestFilterFileSystem and TestHarFileSystem failures after DistributedFileSystem.append API. Contributed by Lei (Eddy) Xu.
...
(cherry picked from commit 7ade5124b8
)
2017-06-19 19:10:13 +09:00
Masatake Iwasaki
023dcdc73d
HADOOP-14424. Add CRC32C performance test. Contributed by LiXin Ge.
...
(cherry picked from commit 75043d3429
)
2017-06-19 09:18:02 +09:00
Lei Xu
33afa1fdca
HADOOP-14395. Provide Builder pattern for DistributedFileSystem.append. Contributed by Lei (Eddy) Xu.
...
(cherry picked from commit 6460df21a0
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
2017-06-16 17:32:29 -07:00
Arpit Agarwal
9529513f18
HADOOP-14503. Make RollingAverages a mutable metric. Contributed by Hanisha Koneru.
2017-06-16 16:41:08 -07:00
Xiao Chen
2afe9722af
HADOOP-14524. Make CryptoCodec Closeable so it can be cleaned up proactively.
2017-06-16 09:49:41 -07:00
Steve Loughran
4e6348f346
HADOOP-14486 TestSFTPFileSystem#testGetAccessTime test failure using openJDK 1.8.0.
...
Contributed by Hongyuan Li.
(cherry picked from commit 942ecbbc98
)
2017-06-16 09:46:34 +01:00
Akira Ajisaka
12cb3456d7
HADOOP-14289. Move log4j APIs over to slf4j in hadoop-common.
...
(cherry picked from commit e855cc4ed4
)
2017-06-16 13:32:16 +09:00
Lei Xu
abac844c90
HADOOP-14394. Provide Builder pattern for DistributedFileSystem.create. (lei)
...
(cherry picked from commit 5fbec46525d6d49837d934556b59ba77bd2301a8)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/NameNodeConnector.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDistributedFileSystem.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestErasureCodingPolicies.java
2017-06-15 10:59:11 -07:00
Xiao Chen
7d81b0beab
HADOOP-14523. OpensslAesCtrCryptoCodec.finalize() holds excessive amounts of memory. Contributed by Misha Dmitriev.
...
(cherry picked from commit ef8edab930
)
2017-06-15 10:41:34 -07:00
Steve Loughran
31446f8e10
HADOOP-14506. Add create() contract test that verifies ancestor dir creation.
...
Contributed by Sean Mackrory.
(cherry picked from commit d780a67864
)
2017-06-15 14:51:50 +01:00
Jonathan Eagles
f901132a8a
HADOOP-14501. Switch from aalto-xml to woodstox to handle odd XML features (jeagles)
2017-06-12 17:19:46 -05:00
Mingliang Liu
ae2457e1dd
HADOOP-14465. LdapGroupsMapping - support user and group search base. Contributed by Shwetha G S and Mingliang Liu
...
(cherry picked from commit a2121cb0d9
)
2017-06-09 16:26:10 -07:00
Xiao Chen
ffaf24e308
HADOOP-13174. Add more debug logs for delegation tokens and authentication.
...
(cherry picked from commit 4a56bde6ba1f72588a25cd96acc76089706cb786)
Conflicts:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java
2017-06-08 21:34:09 -07:00
John Zhuge
87f4428de0
HADOOP-14511. WritableRpcEngine.Invocation#toString NPE on null parameters. Contributed by John Zhuge.
2017-06-08 21:26:02 -07:00
John Zhuge
2448d84572
HDFS-11861. ipc.Client.Connection#sendRpcRequest should log request name. Contributed by John Zhuge.
...
(cherry picked from commit 5672ae7b37
)
2017-06-07 15:54:08 -07:00
Kihwal Lee
f7e597b56d
HADOOP-14035. Reduce fair call queue backoff's impact on clients. Contributed by Daryn Sharp.
2017-06-06 12:19:44 -05:00
Brahma Reddy Battula
e889c826d7
HADOOP-14431. ModifyTime of FileStatus returned by SFTPFileSystem's getFileStatus method is wrong. Contributed by Hongyuan Li.
...
(cherry picked from commit 66c6fd8314
)
2017-06-06 12:32:37 +08:00
Mingliang Liu
867026d94e
HADOOP-14428. s3a: mkdir appears to be broken. Contributed by Mingliang Liu
...
(cherry picked from commit 6aeda55bb8
)
2017-06-05 11:31:13 -07:00
Brahma Reddy Battula
ca76378aad
HADOOP-14440. Add metrics for connections dropped. Contributed by Eric Badger.
...
(cherry picked from commit abdd609e51
)
2017-06-06 00:21:59 +08:00
Brahma Reddy Battula
ac2af66e77
HDFS-11905. Fix license header inconsistency in hdfs. Contributed by Yeliang Cang.
2017-06-02 22:22:43 +08:00
Andrew Wang
d0a4e7e95d
HDFS-11383. Intern strings in BlockLocation and ExtendedBlock. Contributed by Misha Dmitriev.
...
(cherry picked from commit 7101477e47
)
2017-06-01 15:20:22 -07:00
Kihwal Lee
5b0baeab5e
HDFS-5042. Completed files lost after power failure. Contributed by Vinayakumar B.
2017-05-31 11:02:34 -05:00
Arpit Agarwal
f200399496
HDFS-11891. DU#refresh should print the path of the directory when an exception is caught. Contributed by Chen Liang.
2017-05-26 16:02:53 -07:00
Steve Loughran
1efba62fd7
HADOOP-14430 the accessTime of FileStatus returned by SFTPFileSystem's
...
getFileStatus method is always 0.
Contributed by Hongyuan Li.
2017-05-25 15:17:43 +01:00
Steve Loughran
07c14b35ec
HADOOP-14399. Configuration does not correctly XInclude absolute file URIs.
...
Contributed by Jonathan Eagles
(cherry picked from commit 1ba9704eec
)
2017-05-25 15:03:01 +01:00
Amar Kamat
3a8f3bcccb
HADOOP-7851. Configuration.getClasses() never returns the default value. (amarrk)
...
(cherry picked from commit 82d57ee7fe
)
2017-05-24 17:01:49 -07:00
Brahma Reddy Battula
6f55be197c
HADOOP-14166. Reset the DecayRpcScheduler AvgResponseTime metric to zero when queue is not used. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit dcf4559ebd
)
2017-05-25 01:27:52 +08:00
Brahma Reddy Battula
e3a65b24bf
HADOOP-14449. The ASF Header in ComparableVersion.java and SSLHostnameVerifier.java is not correct. Contributed by ZhangBing Lin.
...
(cherry picked from commit b7a0c0e701
)
2017-05-25 01:04:06 +08:00
Brahma Reddy Battula
d61418f34a
HDFS-11864. Document Metrics to track usage of memory for writes. Contributed by Yiqun Lin.
...
(cherry picked from commit 52661e0912
)
2017-05-23 23:53:27 +08:00
Sunil G
ec346d6377
YARN-6584. Correct license headers in hadoop-common, hdfs, yarn and mapreduce. Contributed by Yeliang Cang.
2017-05-23 07:57:36 +05:30
Brahma Reddy Battula
2755255744
HDFS-11863. Document missing metrics for blocks count in pending IBR. Contributed By Yiqun Lin.
...
(cherry picked from commit 2a8fcf0c9a
)
2017-05-23 02:23:54 +08:00
Chris Douglas
4bb0565443
HADOOP-14434. Use MoveFileEx to allow renaming a file when the destination exists. Contributed by Lukas Majercak
...
(cherry picked from commit ef9e536a71
)
2017-05-17 17:24:36 -07:00
Mingliang Liu
94755e2d6d
HADOOP-14427. Avoid reloading of Configuration in ViewFileSystem creation. Contributed by Vinayakumar B
...
(cherry picked from commit eb7791b474
)
2017-05-17 13:21:34 -07:00
Rohith Sharma K S
379e5bd26f
HADOOP-14412. HostsFileReader#getHostDetails is very expensive on large clusters. Contributed by Jason Lowe.
2017-05-17 08:27:50 +05:30
Andrew Wang
18b89e902b
HDFS-11644. Support for querying outputstream capabilities. Contributed by Manoj Govindassamy.
2017-05-13 21:41:11 -07:00
Jason Lowe
955e8316d5
HADOOP-14376. Memory leak when reading a compressed file using the native library. Contributed by Eli Acherkan
...
(cherry picked from commit 7bc2172248
)
2017-05-12 16:55:49 -05:00
Daniel Templeton
7213c935c6
HADOOP-14413. Add Javadoc comment for jitter parameter on CachingGetSpaceUsed
...
(Contributed by Erik Krogen via Daniel Templeton)
(cherry picked from commit 09f28da2d2
)
2017-05-11 12:06:39 -07:00
Brahma Reddy Battula
3766e54893
HADOOP-14410. Correct spelling of 'beginning' and variants. Contributed By Dongtao Zhang
...
(cherry picked from commit 4aae2d40a3
)
2017-05-11 23:38:18 +05:30
Mingliang Liu
214f3abe23
HDFS-11800. Document output of 'hdfs count -u' should contain PATHNAME. Contributed by Xiaobing Zhou
...
(cherry picked from commit 3873904a94
)
2017-05-10 14:34:11 -07:00
Akira Ajisaka
d9c3c78b78
HADOOP-14373. License error in org.apache.hadoop.metrics2.util.Servers. Contributed by hu xiaodong.
...
(cherry picked from commit a5fa954684
)
2017-05-10 05:30:37 -05:00
Masatake Iwasaki
05dda3063b
HADOOP-14405. Fix performance regression due to incorrect use of DataChecksum. Contributed by LiXin Ge.
...
(cherry picked from commit 06ffeb8d4d
)
2017-05-10 14:19:40 +09:00
Jason Lowe
f9ba4ef7d5
HADOOP-14377. Increase Common test timeouts from 1 second to 10 seconds. Contributed by Eric Badger
...
(cherry picked from commit f7faac8e90
)
Conflicts:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestGlobPattern.java
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
2017-05-09 16:16:49 -05:00
Lei Xu
4a95cff95d
HADOOP-14384. Reduce the visibility of FileSystem.newFSDataOutputStreamBuilder before the API becomes stable. (lei)
...
(cherry picked from commit 149b903be3
)
2017-05-09 10:28:21 -07:00
Haohui Mai
b8870d8159
HADOOP-14383. Implement FileSystem that reads from HTTP / HTTPS endpoints.
2017-05-08 15:28:30 -07:00
Arpit Agarwal
89baea96d5
HDFS-11722. Change Datanode file IO profiling sampling to percentage. Contributed by Hanisha Koneru.
2017-05-03 16:30:01 -07:00
Arpit Agarwal
4b7d1ff326
Revert "HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru."
...
This reverts commit dfc6ef42c4
.
2017-05-03 16:29:44 -07:00
Arpit Agarwal
dfc6ef42c4
HDFS-11488. JN log segment syncing should support HA upgrade. Contributed by Hanisha Koneru.
2017-05-03 15:53:43 -07:00
Mingliang Liu
616413ecb2
HDFS-11739. Fix regression in tests caused by YARN-679. Contributed by Steve Loughran
...
(cherry picked from commit 83dded556d
)
2017-05-03 11:23:15 -07:00
Kihwal Lee
f843c8ccf5
HADOOP-14372. TestSymlinkLocalFS timeouts are too low. Contributed by Eric Badger.
...
(cherry picked from commit d4631e466b
)
2017-05-03 09:49:17 -05:00
Jason Lowe
9e9c3a511c
HADOOP-14306. TestLocalFileSystem tests have very low timeouts. Contributed by Eric Badger
...
(cherry picked from commit cedaf4cab9
)
2017-05-02 16:52:52 -05:00
Andrew Wang
484bab3944
HADOOP-14369. NetworkTopology calls expensive toString() when logging. Contributed by Inigo Goiri.
...
(cherry picked from commit dcc292d7de
)
2017-05-02 10:53:57 -07:00
Akira Ajisaka
036624900b
HADOOP-14371. License error in TestLoadBalancingKMSClientProvider.java. Contributed by hu xiaodong.
...
(cherry picked from commit 20cde55274
)
2017-05-02 23:39:00 +09:00
Mingliang Liu
9b8940366e
Revert "HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang"
...
This reverts commit 18a5925256
.
2017-04-29 23:51:30 -07:00
Mingliang Liu
18a5925256
HADOOP-14363. Inconsistent default block location in FileSystem javadoc. Contributed by Chen Liang
...
(cherry picked from commit c735362df7
)
2017-04-29 23:50:04 -07:00
Eric Payne
2bf29ca2f9
HADOOP-14320. TestIPC.testIpcWithReaderQueuing fails intermittently. Contributed by Eric Badger.
...
(cherry picked from commit 6fe11f6155
)
2017-04-28 14:49:02 -05:00
Junping Du
5d81b6b29a
YARN-679. Add an entry point that can start any Yarn service. Contributed by Steve Loughran.
...
(cherry picked from commit 373bb4931f
)
2017-04-28 11:10:00 -07:00
Steve Loughran
b500714e98
HADOOP-14346. CryptoOutputStream throws IOException if stream is closed.
...
Contributed by Pierre Lacave
(cherry picked from commit 40b9c30d64
)
2017-04-27 12:18:30 +01:00
John Zhuge
1e60cfed7e
HADOOP-14341. Support multi-line value for ssl.server.exclude.cipher.list. Contributed by John Zhuge.
...
(cherry picked from commit 9ccb849eb6
)
Conflicts:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/http/HttpServer2.java
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLFactory.java
2017-04-25 22:36:21 -07:00
Xiaoyu Yao
fc0e6de929
HADOOP-14207. "dfsadmin -refreshCallQueue" fails with DecayRpcScheduler. Contributed by Surendra Singh Lihore.
...
(cherry picked from commit 918dee10f56861d15bb6644edcefb4246a3e00a4)
2017-04-25 18:49:44 -07:00
Mingliang Liu
6fd072f924
HADOOP-12856. FileUtil.checkDest() and RawLocalFileSystem.mkdirs() to throw stricter IOEs; RawLocalFS contract tests to verify. Contributed by Steve Loughran
...
(cherry picked from commit d11d280b8b
)
2017-04-25 18:34:26 -07:00
Mingliang Liu
3b7bb7b94b
HADOOP-14324. Refine S3 server-side-encryption key as encryption secret; improve error reporting and diagnostics. Contributed by Steve Loughran
2017-04-20 17:07:57 -07:00
Daniel Templeton
d77137a03f
YARN-6453. fairscheduler-statedump.log gets generated regardless of service
...
(Contributed by Yufei Gu via Daniel Templeton)
(cherry picked from commit e74ca7216aecc998d277e5ca1f4d0abae43c7e31)
2017-04-20 14:14:38 -07:00
John Zhuge
6865746ea4
HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge.
2017-04-19 22:22:36 -07:00
John Zhuge
ef74a5661b
Revert "HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge."
...
This reverts commit 3a39dd5391
.
2017-04-19 13:33:38 -07:00
John Zhuge
3a39dd5391
HADOOP-14241. Add ADLS sensitive config keys to default list. Contributed by John Zhuge.
...
(cherry picked from commit 0344bea3fd
)
2017-04-19 12:04:03 -07:00
Daniel Templeton
5225ae50b7
HADOOP-13997. Typo in metrics docs (Contributed by Ana Krasteva via Daniel Templeton)
...
(cherry picked from commit d4483a2fe4
)
2017-04-19 09:42:30 -07:00
Kengo Seki
6cfceee865
HADOOP-14315. Python example in the rack awareness document doesn't work due to bad indentation
...
This closes #214
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
(cherry picked from commit 654372db85
)
2017-04-18 17:15:35 +09:00
Akira Ajisaka
a17bd70892
HADOOP-14318. Remove non-existent setfattr command option from FileSystemShell.md. Contributed by Doris Gu.
...
(cherry picked from commit 84a8848aae
)
2017-04-18 16:45:14 +09:00
Mingliang Liu
508389ee54
HADOOP-14255. S3A to delete unnecessary fake directory objects in mkdirs(). Contributed by Mingliang Liu
...
(cherry picked from commit 507845074970b90330db86bc1ff0124def550bb5)
2017-04-12 14:30:54 -07:00
Chris Nauroth
d5e2bd4096
HDFS-11163. Mover should move the file blocks to default storage once policy is unset. Contributed by Surendra Singh Lilhore.
...
(cherry picked from commit 00ed21a6fedb45a7c8992b8d45adaa83f14af34c)
2017-04-11 21:18:47 -07:00
Steve Loughran
5efb3ade1f
HADOOP-14066 VersionInfo should be marked as public API
...
(cherry picked from commit 2ed65540a3
)
2017-04-11 15:06:40 -07:00
Jason Lowe
0cb22a0f15
YARN-6195. Export UsedCapacity and AbsoluteUsedCapacity to JMX. Contributed by Benson Qiu
...
(cherry picked from commit 0e065f2ede
)
2017-04-11 08:45:16 -05:00
Andrew Wang
dab1deb9a0
HADOOP-14293. Initialize FakeTimer with a less trivial value.
...
(cherry picked from commit be144117a8
)
2017-04-10 11:37:31 -07:00
John Zhuge
ff2870310e
HADOOP-14174. Set default ADLS access token provider type to ClientCredential. Contributed by John Zhuge.
...
Signed-off-by: John Zhuge <jzhuge@apache.org>
(cherry picked from commit 56e81f2a20
)
Conflicts:
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/TestCommonConfigurationFields.java
2017-04-07 15:01:29 -07:00
Zhe Zhang
c85026038a
HADOOP-14276. Add a nanosecond API to Time/Timer/FakeTimer. Contributed by Erik Krogen.
...
(cherry picked from commit 95b7f1d29a
)
2017-04-07 10:39:40 -07:00
Andrew Wang
ed8f738be5
HADOOP-14104. Client should always ask namenode for kms provider path. Contributed by Rushabh S Shah.
2017-04-06 14:38:22 -07:00
Xiaoyu Yao
9016614eb1
HDFS-11302. Improve Logging for SSLHostnameVerifier. Contributed by Chen Liang.
...
(cherry picked from commit 32bb36b750
)
2017-04-05 18:30:33 -07:00
Jonathan Eagles
60e65a3e55
HADOOP-14216. Addendum to Improve Configuration XML Parsing Performance (jeagles)
2017-03-30 14:15:37 -05:00
Mingliang Liu
3d5e9e61d5
HADOOP-14247. FileContextMainOperationsBaseTest should clean up test root path. Contributed by Mingliang Liu
...
(cherry picked from commit 13c766b62c
)
2017-03-29 11:19:28 -07:00
Jonathan Eagles
719ae5f2e8
HADOOP-14216. Improve Configuration XML Parsing Performance (jeagles)
2017-03-29 10:14:49 -05:00
John Zhuge
ef43a62482
HADOOP-14038. Rename ADLS credential properties. Contributed by John Zhuge.
...
(cherry picked from commit 253e3e78ab
)
2017-03-27 23:07:08 -07:00
Akira Ajisaka
643298b282
HADOOP-14218. Replace assertThat with assertTrue in MetricsAsserts.
...
(cherry picked from commit 448ec81fd7
)
2017-03-28 14:33:36 +09:00
Andrew Wang
10deafee82
HDFS-11170. Add builder-based create API to FileSystem. Contributed by SammiChen and Wei Zhou.
2017-03-27 14:35:03 -07:00
Jonathan Eagles
e41aac0cd4
Configuration#get return value optimization (jeagles)
2017-03-27 12:52:59 -05:00
Jonathan Eagles
675bc97f18
Delay construction of PreCondition.check failure message in Configuration#set (jeagles)
...
(cherry picked from commit 858d597be0
)
2017-03-27 11:02:41 -05:00
Junping Du
3ad4d37363
Add COMMON/MAPREDUCE/YARN jdiff of 2.8.0 release.
...
(cherry picked from commit 6df029db36
)
2017-03-25 14:33:20 -07:00
Junping Du
e796105a6f
Add 2.8.0 RELEASENOTEs.md and CHANGES.md.
...
(cherry picked from commit d8e6d6df4c
)
2017-03-25 14:32:47 -07:00
Daniel Templeton
bbd08bb7a4
YARN-6360. Prevent FS state dump logger from cramming other log files
...
(Contributed by Yufei Gu via Daniel Templeton)
(cherry picked from commit d49d1862a6
)
2017-03-24 11:31:58 -07:00
Andrew Wang
96fe940e59
HADOOP-14211. FilterFs and ChRootedFs are too aggressive about enforcing 'authorityNeeded'. Contributed by Erik Krogen.
...
(cherry picked from commit 0e556a5ba6
)
2017-03-24 11:12:31 -07:00
John Zhuge
d946dfcbe1
HADOOP-14195. CredentialProviderFactory$getProviders is not thread-safe. Contributed by Vihang Karajgaonkar.
...
(cherry picked from commit 128015584d
)
2017-03-23 13:57:41 -07:00
Mingliang Liu
1e64c582f5
HADOOP-14214. DomainSocketWatcher::add()/delete() should not self interrupt while looping await(). Contributed by Mingliang Liu
...
(cherry picked from commit d35e79abc2
)
2017-03-23 13:34:53 -07:00
Ravi Prakash
87339d7691
HADOOP-14213. Move Configuration runtime check for hadoop-site.xml to initialization. Contributed by Jonathan Eagles
...
(cherry picked from commit 595f62e362
)
2017-03-23 09:30:22 -07:00