Commit Graph

22477 Commits

Author SHA1 Message Date
Shen Yinjie ee3115f488
HDFS-9913. DistCp to add -useTrash to move deleted files to Trash.
Contributed by Shen Yinjie.

Change-Id: I03ac7d22ab1054f8e5de4aa7552909c734438f4a
2019-07-17 11:50:46 +01:00
Weiwei Yang 85d9111a88 YARN-9682. Wrong log message when finalizing the upgrade. Contributed by kyungwan nam. 2019-07-17 10:28:20 +08:00
Haibo Chen 5915c902aa YARN-9646. DistributedShell tests failed to bind to a local host name. (Contributed by Ray Yang) 2019-07-16 17:36:49 -07:00
Sean Mackrory 5672efa5c7
HADOOP-15729. [s3a] Allow core threads to time out. (#1075) 2019-07-16 18:14:23 -06:00
Hanisha Koneru 7adc4a327a
HDDS-1775. Make OM KeyDeletingService compatible with HA model (#1063) 2019-07-16 16:59:57 -07:00
Xiaoyu Yao 71c21f0a9b
HDDS-1813. Fix false warning from ozones3 acceptance test. Contributed by Xiaoyu Yao. (#1107) 2019-07-16 13:47:51 -07:00
Xiaoyu Yao d429d742f0 HDDS-1544. Support default Acls for volume, bucket, keys and prefix. Contributed by Ajay Kumar, Xiaoyu Yao.
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-16 12:58:59 -07:00
Sammi Chen 493b0b5760 HDDS-1787. NPE thrown while trying to find DN closest to client. Contributed by Sammi Chen. (#1094) 2019-07-16 12:52:29 -07:00
Doroszlai, Attila c5e3ab5a4d
HDDS-1793. Acceptance test of ozone-topology cluster is failing
Closes #1096
2019-07-16 16:52:14 +02:00
Shashikant Banerjee d21eccf8ba HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084) 2019-07-16 18:01:13 +05:30
Nanda kumar 1502e6a22f HDDS-1756. DeleteContainerCommandHandler fails with NPE. Contributed by Nanda kumar(#1095). 2019-07-16 15:06:41 +05:30
bibinchundatt 7a93be0f60 YARN-9645. Fix Invalid event FINISHED_CONTAINERS_PULLED_BY_AM at NEW on NM restart. Contributed by Bilwa S T. 2019-07-16 14:03:22 +05:30
Bharat Viswanadham 395cb3cfd7
HDDS-1736. Cleanup 2phase old HA code for Key requests. (#1038) 2019-07-15 21:51:59 -07:00
Ayush Saxena f77d54c243 HDFS-14642. processMisReplicatedBlocks does not return correct processed count. Contributed by Stephen O'Donnell. 2019-07-16 08:14:27 +05:30
Bharat Viswanadham ef66e4999f HDDS-1666. Issue in openKey when allocating block. (#943) 2019-07-15 17:54:41 -07:00
Bharat Viswanadham 585f4d5c64
HDDS-1761. Fix class hierarchy for KeyRequest and FileRequest classes. (#1052) 2019-07-15 17:53:19 -07:00
Anu Engineer 141151325b
Merge pull request #944 from elek/HDDS-1668
HDDS-1668. Add liveness probe to the example k8s resources files
2019-07-15 13:19:32 -07:00
Anu Engineer 47345e598e
Merge pull request #1058 from elek/HDDS-1763
HDDS-1763. Use vendor neutral s3 logo in ozone doc
2019-07-15 13:17:21 -07:00
Ayush Saxena 64d4abf489 HDFS-14593. RBF: Implement deletion feature for expired records in State Store. Contributed by Takanobu Asanuma. 2019-07-15 22:38:00 +05:30
Doroszlai, Attila 61bbdeee19
HDDS-1800. Result of author check is inverted
Closes #1092
2019-07-15 18:00:10 +02:00
Szilard Nemeth 5446308360 YARN-9326. Fair Scheduler configuration defaults are not documented in case of min and maxResources. Contributed by Adam Antal 2019-07-15 13:28:01 +02:00
Szilard Nemeth 18ee1092b4 YARN-9127. Create more tests to verify GpuDeviceInformationParser. Contributed by Peter Bacsko 2019-07-15 11:59:11 +02:00
Szilard Nemeth be784de2d4 SUBMARINE-62. PS_LAUNCH_CMD CLI description is wrong in RunJobCli. Contributed by Adam Antal 2019-07-15 11:17:16 +02:00
Szilard Nemeth 91ce09e706 YARN-9360. Do not expose innards of QueueMetrics object into FSLeafQueue#computeMaxAMResource. Contributed by Peter Bacsko 2019-07-15 10:47:20 +02:00
Rakesh Radhakrishnan 30a8f840f1 HDFS-14357. Update documentation for HDFS cache on SCM support. Contributed by Feilong He. 2019-07-15 13:18:23 +05:30
Rakesh Radhakrishnan e98adb00b7 HDFS-14458. Report pmem stats to namenode. Contributed by Feilong He. 2019-07-15 13:02:37 +05:30
Mukul Kumar Singh 0976f6fc30
HDDS-1766. ContainerStateMachine is unable to increment lastAppliedTermIndex. Contributed by Mukul Kumar Singh. (#1072) 2019-07-14 10:53:51 +05:30
Anu Engineer 4a70a0d816
HDDS-1735. Create separate unit and integration test executor dev-support script. Contributed by Elek, Marton. 2019-07-12 09:25:00 -07:00
Szilard Nemeth 557056e18e YARN-9626. UI2 - Fair scheduler queue apps page issues. Contributed by Zoltan Siegl 2019-07-12 17:40:14 +02:00
Szilard Nemeth 61b0c2bb7c YARN-9337. GPU auto-discovery script runs even when the resource is given by hand. Contributed by Adam Antal 2019-07-12 17:28:14 +02:00
Szilard Nemeth 8b3c6791b1 YARN-9135. NM State store ResourceMappings serialization are tested with Strings instead of real Device objects. Contributed by Peter Bacsko 2019-07-12 17:20:42 +02:00
Szilard Nemeth c416284bb7 YARN-9235. If linux container executor is not set for a GPU cluster GpuResourceHandlerImpl is not initialized and NPE is thrown. Contributed by Antal Balint Steinbach, Adam Antal 2019-07-12 16:51:58 +02:00
Nanda kumar 190e4349d7
HDDS-1790. Fix checkstyle issues in TestDataScrubber
Closes #1082
2019-07-12 14:42:29 +02:00
Nanda kumar 7b8177ba0f
HDDS-1791. Update network-tests/src/test/blockade/README.md file
Closes #1083
2019-07-12 14:38:22 +02:00
Steve Loughran b15ef7dc3d
HADOOP-16384: S3A: Avoid inconsistencies between DDB and S3.
Contributed by Steve Loughran

Contains

- HADOOP-16397. Hadoop S3Guard Prune command to support a -tombstone option.
- HADOOP-16406. ITestDynamoDBMetadataStore.testProvisionTable times out intermittently

This patch doesn't fix the underlying problem but it

* changes some tests to clean up better
* does a lot more in logging operations in against DDB, if enabled
* adds an entry point to dump the state of the metastore and s3 tables (precursor to fsck)
* adds a purge entry point to help clean up after a test run has got a store into a mess
* s3guard prune command adds -tombstone option to only clear tombstones

The outcome is that tests should pass consistently and if problems occur we have better diagnostics.

Change-Id: I3eca3f5529d7f6fec398c0ff0472919f08f054eb
2019-07-12 13:02:25 +01:00
Shashikant Banerjee f9fab9f22a HDFS-14499. Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory. Contributed by Shashikant Banerjee. 2019-07-12 15:42:06 +05:30
Márton Elek 51e7dc3221
make author check yetus safe 2019-07-12 11:59:24 +02:00
Márton Elek ca7b3e8310
reenable unit test execution 2019-07-12 11:59:20 +02:00
Márton Elek 317f2c896f
fix rat problem 2019-07-12 11:30:56 +02:00
Márton Elek 987bf3f696
fix print0 consequences 2019-07-12 11:30:56 +02:00
Márton Elek 6f25420f57
fixing ALL the checkstule issues under hadoop-ozone/dev-support/checks 2019-07-12 11:30:56 +02:00
Márton Elek 0bae9e8ec8
HDDS-1735. Create separate unit and integration test executor dev-support script 2019-07-12 11:30:55 +02:00
Márton Elek 9119ed07ff
HDDS-1384. TestBlockOutputStreamWithFailures is failing
Closes #1029
2019-07-12 11:00:03 +02:00
Hrishikesh Gadre 14c43f85de HDDS-1752. ConcurrentModificationException while handling DeadNodeHandler event. (#1080) 2019-07-12 14:15:48 +05:30
supratimdeka 738fab3bff HDDS-1754. getContainerWithPipeline fails with PipelineNotFoundException. Contributed by Supratim Deka (#1081) 2019-07-12 10:31:48 +05:30
Masatake Iwasaki 00dd843a1a
HDFS-14466. Add a regression test for HDFS-14323.
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-07-12 10:04:55 +09:00
Anu Engineer cdcc205490
HDDS-1784. Missing HostName and IpAddress in the response of register command. (#1078) 2019-07-11 17:17:05 -07:00
Haibo Chen 9b54dd7186 YARN-9668. UGI conf doesn't read user overridden configurations on RM and NM startup. (Contributed by Jonathan Hung) 2019-07-11 13:57:08 -07:00
Chen Liang 8fb5ca3f40 HDFS-14611. Move handshake secret field from Token to BlockAccessToken. Contributed by Chen Liang. 2019-07-11 13:23:25 -07:00
Steve Loughran 6a3433bffd
HADOOP-16357. TeraSort Job failing on S3 DirectoryStagingCommitter: destination path exists.
Contributed by Steve Loughran.

This patch

* changes the default for the staging committer to append, as we get for the classic FileOutputFormat committer
* adds a check for the dest path being a file not a dir
* adds tests for this
* Changes AbstractCommitTerasortIT. to not use the simple parser, so fails if the file is present.

Change-Id: Id53742958ed1cf321ff96c9063505d64f3254f53
2019-07-11 18:15:34 +01:00