71aad60e51
HDFS-14691. Wrong usage hint for hadoop fs command "test".
...
Contributed by Jianfei Jiang.
Change-Id: I9f5e89721ff210641375fbf42a70043f0d74458e
2019-08-05 13:08:47 +01:00
61180f4656
HADOOP-15942. Change the logging level form DEBUG to ERROR for RuntimeErrorException in JMXJsonServlet. Contributed by Anuhan Torgonshar.
2019-08-02 14:58:24 -07:00
797d14e816
Revert "HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm."
...
This reverts commit 076618677d
2019-08-02 08:25:41 -07:00
e872ceb810
HADOOP-15865. ConcurrentModificationException in Configuration.overlay() method. Contributed by Oleksandr Shevchenko.
2019-08-01 19:56:51 -07:00
e20b19543b
HADOOP-15681. AuthenticationFilter should generate valid date format for Set-Cookie header regardless of default Locale. Contributed by Cao Manh Dat.
2019-08-01 17:35:31 -07:00
d086d058d8
HDFS-14652. HealthMonitor connection retry times should be configurable. Contributed by Chen Zhang.
2019-08-01 16:13:10 -07:00
b94eba9f11
HADOOP-12282. Connection thread's name should be updated after address changing is detected. Contributed by Lisheng Sun.
2019-08-01 15:50:43 -07:00
f86de6f76a
HDFS-13529. Fix default trash policy emptier trigger time correctly. Contributed by He Xiaoqiao.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-08-01 14:54:39 -07:00
8bda91d20a
HADOOP-16398. Exports Hadoop metrics to Prometheus ( #1170 )
2019-07-31 10:11:36 -07:00
0f2dad6679
HDFS-14569. Result of crypto -listZones is not formatted properly. Contributed by hemanthboyina.
2019-07-30 16:52:42 -07:00
c75f16db79
HADOOP-16452. Increase ipc.maximum.data.length default from 64MB to 128MB. Contributed by Siyao Meng.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-07-30 11:22:45 -07:00
204a977f55
HADOOP-15910. Fix Javadoc for LdapAuthenticationHandler#ENABLE_START_TLS
...
Contributed by Don Jeba.
Change-Id: I2755bfb1263fc659078a1af8f0bdfd739fd1ae40
2019-07-30 12:39:48 +01:00
cbfa3f3e98
HADOOP-16435. RpcMetrics should not retained forever. Contributed by Zoltan Haindrich.
2019-07-29 17:37:26 -07:00
62efb63006
HADOOP-16245. Restrict the effect of LdapGroupsMapping SSL configurations to avoid interfering with other SSL connections. Contributed by Erik Krogen.
2019-07-26 11:16:58 -07:00
aebac6d2d2
HADOOP-16461. Regression: FileSystem cache lock parses XML within the lock.
...
Contributed by Gopal V.
Change-Id: If6654f850e9c24ee0d9519a46fd6269b18e1a7a4
2019-07-26 11:32:13 +01:00
07530314c2
HADOOP-9844. NPE when trying to create an error message response of SASL RPC
...
This closes #55
Change-Id: I10a20380565fa89762f4aa564b2f1c83b9aeecdc
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-26 17:53:18 +09:00
a1251addff
HADOOP-16431. Remove useless log in IOUtils.java and ExceptionDiags.java.
...
This closes #1091
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-24 10:04:39 +09:00
4317d33232
HADOOP-16380. S3Guard to determine empty directory status for all non-root directories.
...
Contributed by Steve Loughran and Gabor Bota.
This
* Asks S3Guard to determine the empty directory status.
* Has S3A's root directory rm("/") command to always return false (as abfs does)
* Documents that object stores MAY do this
* Overloads ContractTestUtils.assertDeleted to let assertions declare that the source directory does not need to exist. This stops inconsistencies in directory listings failing a root test.
It avoids a recent regression (HADOOP-16279) where if there was a tombstone above the first element found in a directory listing, the directory would be considered empty, when in fact there were child entries. That could downgrade an rm(path, recursive) to a no-op, while also confusing rename(src, dest), as dest could be mistaken for an empty directory and so permit the copy above it, rather than reject it "destination path exists and is not empty".
Change-Id: I136a3d1a5a48a67e6155d790a40ff558d0d2c108
2019-07-23 14:52:03 +01:00
eb36b09cb7
HADOOP-16443. Improve help text for setfacl --set option.
...
Contributed by S O'Donnell.
Change-Id: I1da46c4c414a5d2b07ee15867508f0799440a413
2019-07-23 10:24:07 +01:00
7f1b76ca35
HADOOP-13868. [s3a] New default for S3A multi-part configuration ( #1125 )
2019-07-19 09:49:59 -06:00
d545f9c290
HADOOP-16437 documentation typo fix: fs.s3a.experimental.input.fadvise
...
Fix fs.s3a.experimental.fadvise to fs.s3a.experimental.input.fadvise
Contributed by: Josh Rosen
2019-07-18 23:19:38 +01:00
b4466a3b0a
HADOOP-16341. ShutDownHookManager: Regressed performance on Hook removals after HADOOP-15679
...
Contributed by Gopal V and Atilla Magyar.
Change-Id: I066d5eece332a1673594de0f9b484443f95530ec
2019-07-17 13:50:02 +01:00
b15ef7dc3d
HADOOP-16384: S3A: Avoid inconsistencies between DDB and S3.
...
Contributed by Steve Loughran
Contains
- HADOOP-16397. Hadoop S3Guard Prune command to support a -tombstone option.
- HADOOP-16406. ITestDynamoDBMetadataStore.testProvisionTable times out intermittently
This patch doesn't fix the underlying problem but it
* changes some tests to clean up better
* does a lot more in logging operations in against DDB, if enabled
* adds an entry point to dump the state of the metastore and s3 tables (precursor to fsck)
* adds a purge entry point to help clean up after a test run has got a store into a mess
* s3guard prune command adds -tombstone option to only clear tombstones
The outcome is that tests should pass consistently and if problems occur we have better diagnostics.
Change-Id: I3eca3f5529d7f6fec398c0ff0472919f08f054eb
2019-07-12 13:02:25 +01:00
8fb5ca3f40
HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang.
2019-07-11 13:23:25 -07:00
6a3433bffd
HADOOP-16357. TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists.
...
Contributed by Steve Loughran.
This patch
* changes the default for the staging committer to append, as we get for the classic FileOutputFormat committer
* adds a check for the dest path being a file not a dir
* adds tests for this
* Changes AbstractCommitTerasortIT. to not use the simple parser, so fails if the file is present.
Change-Id: Id53742958ed1cf321ff96c9063505d64f3254f53
2019-07-11 18:15:34 +01:00
aa9f0850e8
HADOOP-16385. Namenode crashes with 'RedundancyMonitor thread received Runtime exception'. Contributed by Ayush Saxena.
2019-07-02 13:22:30 -07:00
129576f628
HDFS-14403. Cost-based extension to the RPC Fair Call Queue. Contributed by Christopher Gregorian.
2019-06-24 12:09:17 -07:00
95c94dcca7
HADOOP-16350. Ability to tell HDFS client not to request KMS Information from NameNode. Ccontributed by Greg Senia, Ajay Kumar.
2019-06-24 11:59:07 -07:00
b52fd05d42
HDFS-13404. Addendum: RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fail. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
ba221dc50e
HDFS-14526. RBF: Update the document of RBF related metrics. Contributed by Takanobu Asanuma.
2019-06-24 22:03:04 +05:30
559cb11551
HDFS-13404. RBF: TestRouterWebHDFSContractAppend.testRenameFileBeingAppended fails.
2019-06-24 22:03:03 +05:30
f40e10b349
HDFS-14129. RBF: Create new policy provider for router. Contributed by Ranith Sardar.
2019-06-24 22:03:03 +05:30
371452e260
HADOOP-15989. Synchronized at CompositeService#removeService is not required. Contributed by Prabhu Joseph.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-21 17:05:13 -07:00
840d02ca5b
HADOOP-14385. HttpExceptionUtils#validateResponse swallows exceptions. Contributed by Wei-Chiu Chuang.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-20 19:57:48 -07:00
e02eb24e0a
HADOOP-15183. S3Guard store becomes inconsistent after partial failure of rename.
...
Contributed by Steve Loughran.
Change-Id: I825b0bc36be960475d2d259b1cdab45ae1bb78eb
2019-06-20 09:56:40 +01:00
3ab77d9bc9
HDFS-14201. Ability to disallow safemode NN to become active. Contributed by Xiao Liang and He Xiaoqiao.
2019-06-18 09:58:29 -07:00
dd4a7633ec
HADOOP-16156. [Clean-up] Remove NULL check before instanceof and fix checkstyle in InnerNodeImpl. Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 23:47:57 -07:00
56c621c68f
HADOOP-15914. hadoop jar command has no help argument. Contributed by Adam Antal.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-17 22:56:00 -07:00
f1c239c6a4
HADOOP-9157. Better option for curl in hadoop-auth-examples. Contributed by Andras Bokor.
2019-06-17 21:51:33 -07:00
10311c30b0
HADOOP-14807. should prevent the possibility of NPE about ReconfigurableBase.java. Contributed by hu xiaodong.
2019-06-17 19:55:56 -07:00
1e92db5a1e
HDFS-11949. Add testcase for ensuring that FsShell cann't move file to the target directory that file exists. Contributed by legend.
2019-06-17 18:29:34 -07:00
f9cc9e1621
HADOOP-16279. S3Guard: Implement time-based (TTL) expiry for entries (and tombstones).
...
Contributed by Gabor Bota.
Change-Id: I73a2d2861901dedfe7a0e783b310fbb95e7c1af9
2019-06-16 17:05:01 +01:00
076618677d
HADOOP-16336. finish variable is unused in ZStandardCompressor. Contributed by cxorm.
2019-06-14 20:05:20 -07:00
c7554ffd5c
HADOOP-16373. Fix typo in FileSystemShell#test documentation ( #968 )
2019-06-14 14:36:04 -07:00
54f9f75a44
HADOOP-16369. Fix zstandard shortname misspelled as zts. Contributed by Jonathan Eagles.
2019-06-14 10:17:25 +09:00
4ea6c2f457
HADOOP-16354. Enable AuthFilter as default for WebHDFS.
...
Contributed by Prabhu Joseph
2019-06-11 18:41:08 -04:00
b417a4c854
MAPREDUCE-7214. Remove unused pieces related to `mapreduce.job.userlog.retain.hours`
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-06-11 18:40:35 +09:00
d6d95d2686
HDFS-14494. Move Server logging of StatedId inside receiveRequestState(). Contributed by Shweta Yakkali.
...
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-06-10 10:23:18 -07:00
76b94c274f
HADOOP-16345. Fix a potential NPE when instantiating FairCallQueue metrics. Contributed by Erik Krogen.
2019-06-07 14:20:44 -07:00
4e38dafde4
HADOOP-15563. S3Guard to support creating on-demand DDB tables.
...
Contributed by Steve Loughran
Change-Id: I2262b5b9f52e42ded8ed6f50fd39756f96e77087
2019-06-07 18:26:10 +01:00
Jianfei Jiang
Wei-Chiu Chuang
He Xiaoqiao
Akira Ajisaka
Siyao Meng
Don Jeba
Erik Krogen
Gopal V
Steve Loughran
sunlisheng
Steve Loughran
S O'Donnell
Sean Mackrory
Josh Rosen
Chen Liang
Inigo Goiri
Christopher Gregorian
Ajay Kumar
Ayush Saxena
Takanobu Asanuma
Surendra Singh Lilhore
Prabhu Joseph
Shweta Yakkali
Adam Antal
Gabor Bota
cxorm
dineshchitlangia
Eric Yang
Wanqiang Ji
Erik Krogen