Commit Graph

306 Commits

Author SHA1 Message Date
Clebert Suconic 873c2bcc18 ARTEMIS-3084 Avoiding Race condition on async close 2021-01-29 10:44:55 -05:00
Clebert Suconic 4b5c29939c ARTEMIS-3084 Fixing intermittent failures on testsuite. file.close() should block
This fixed a failure I saw once at GroupingFailoverReplicationTest.testGroupingLocalHandlerFailsMultipleGroups
2021-01-28 14:24:12 -05:00
Clebert Suconic 03f8f83935 ARTEMIS-3084 Small tweak. Log a Thread dump if a server.stop cannot get a completion of pending closes 2021-01-28 11:31:58 -05:00
Clebert Suconic c019218c4e ARTEMIS-3084 Eliminate Block on moving to next file on libaio 2021-01-28 11:10:40 -05:00
Tomas Hofman b1835fa2d8 ARTEMIS-3037 JournalImpl#checkKnownRecordID() implementation can leave a thread hanging in WAITING state 2020-12-18 12:44:18 -05:00
Clebert Suconic 4e7bb97df7 [maven-release-plugin] prepare for next development iteration 2020-11-02 17:45:51 -05:00
Clebert Suconic 9768017530 [maven-release-plugin] prepare release 2.16.0 2020-11-02 17:45:38 -05:00
Clebert Suconic 28919b6ad8 [maven-release-plugin] prepare for next development iteration 2020-10-30 10:16:29 -04:00
Clebert Suconic af5ca9f1e6 [maven-release-plugin] prepare release 2.16.0 2020-10-30 10:16:17 -04:00
Emmanuel Hugonnet fdfc58171b [ARTEMIS-2939]: Artemis should not delete corrupt log files.
* Moving corrupted journal files to the attic folder.

Jira: https://issues.apache.org/jira/browse/ARTEMIS-2939
2020-10-20 18:39:06 -04:00
franz1981 a680f7d52e ARTEMIS-2928 blocking CallbackCache can be replaced with a JCTools lock-free queue 2020-10-06 07:14:24 +02:00
Domenico Francesco Bruscino 32bf9680f2 [maven-release-plugin] prepare for next development iteration 2020-08-24 16:03:24 +02:00
Domenico Francesco Bruscino a549fcedde [maven-release-plugin] prepare release 2.15.0 2020-08-24 16:03:12 +02:00
franz1981 a6bf7d0e04 ARTEMIS-2837 Bursts of open files under high load
This would prevent to push a new opened file if there
is already one available to be consumed
2020-08-17 13:01:41 -04:00
franz1981 851aef1172 Revert "ARTEMIS-2837 Avoiding bursts on writes and pending callbacks"
This reverts commit 1761f763
2020-08-17 13:01:41 -04:00
franz1981 7902dd1757 ARTEMIS-2849 Optimize zeroing of buffers while writing the ASYNCIO journal 2020-07-23 15:44:52 -04:00
Clebert Suconic 6690ba1d24 [maven-release-plugin] prepare for next development iteration 2020-07-09 12:49:08 -04:00
Clebert Suconic a76f41a7ed [maven-release-plugin] prepare release 2.14.0 2020-07-09 12:48:54 -04:00
Clebert Suconic 1761f76308 ARTEMIS-2837 Avoiding bursts on writes and pending callbacks 2020-07-08 23:08:27 -04:00
Robbie Gemmell d2c46762e8 ARTEMIS-2109: fix javadoc errors, remove doclint config for JDK8, remove superfluous plugin versions, update to 3.0.1 javadoc plugin
Using 3.0.1 like the current apache parent pom [not being used], and also as newer ones break build.
2020-06-11 18:50:01 +01:00
Emmanuel Hugonnet efe0f468de ARTEMIS-2109: Updating the build to be able to execute it on Java 11 and Java 8. 2020-06-11 18:50:01 +01:00
Clebert Suconic faa83b2ba6 [maven-release-plugin] prepare for next development iteration 2020-05-16 18:38:47 -04:00
Clebert Suconic 5f49d89264 [maven-release-plugin] prepare release 2.13.0 2020-05-16 18:38:34 -04:00
Clebert Suconic c99fcd501b [maven-release-plugin] prepare for next development iteration 2020-05-15 16:42:54 -04:00
Clebert Suconic 47fafac760 [maven-release-plugin] prepare release 2.13.0 2020-05-15 16:42:41 -04:00
Francesco Nigro 4e1af3aea3 ARTEMIS-2762 JournalRecord is using too much memory to track JournalFile updates 2020-05-15 09:21:33 -04:00
Clebert Suconic 3541ebf218 ARTEMIS-2732 Cleanup on testsuite logging 2020-04-22 23:08:46 -04:00
Clebert Suconic 11a3e810bb [maven-release-plugin] prepare for next development iteration 2020-04-21 17:06:21 -04:00
Clebert Suconic 30272e0c2f [maven-release-plugin] prepare release 2.12.0 2020-04-21 17:06:08 -04:00
Clebert Suconic d231e2ac63 [maven-release-plugin] prepare for next development iteration 2020-04-16 16:16:41 -04:00
Clebert Suconic 9636f4a3b0 [maven-release-plugin] prepare release 2.12.0 2020-04-16 16:16:28 -04:00
Clebert Suconic af796d5ce4 ARTEMIS-2701 Improving DLQ/check over previously removed records 2020-04-06 12:25:35 -04:00
Clebert Suconic 03fb630f73 ARTEMIS-1975 Fixing LargeMessage encoding for replication 2020-03-25 16:16:05 -04:00
Clebert Suconic 31c945f8b0 ARTEMIS-1975 Fixing encodesize cached on AMQP Large Message
Encoding ahead is broken with AMQP Large Message
This is still part of ARTEMIS-1975
2020-03-24 15:51:46 -04:00
Clebert Suconic ddd8ed4402 ARTEMIS-1975 Real Large Message support into AMQP
This is a Large commit where I am refactoring largeMessage Body out of CoreMessage
which is now reused with AMQP.

I had also to fix Reference Counting to fix how Large Messages are Acked

And I also had to make sure Large Messages are transversing correctly when in cluster.
2020-02-25 15:25:01 +01:00
Francesco Nigro 5897909dc9 ARTEMIS-2617 use core pools to reduce GC on journal loading 2020-02-12 13:29:51 -05:00
Francesco Nigro fa0c187ae6 ARTEMIS-2602 Reduce number of copies for non JDBC Journal 2020-01-18 18:45:12 +01:00
Francesco Nigro b10d765139 ARTEMIS-2602 load surviving records into SparseArrayLinkedList 2020-01-17 14:31:52 -05:00
Francesco Nigro 4cc6464ddd ARTEMIS-2602 Improve Journal loading heap usage 2020-01-17 14:31:52 -05:00
Justin Bertram 97735ca72c [maven-release-plugin] prepare for next development iteration 2020-01-10 09:00:58 -06:00
Justin Bertram 25a947f6cb [maven-release-plugin] prepare release 2.11.0 2020-01-10 08:57:46 -06:00
Clebert Suconic 91f4d0a633 [maven-release-plugin] prepare for next development iteration 2019-09-23 11:19:54 -04:00
Clebert Suconic 86a975fdad [maven-release-plugin] prepare release 2.10.1 2019-09-23 11:19:42 -04:00
Francesco Nigro f51c799ac0 ARTEMIS-1811 NIO Seq File should use RandomAccessFile with heap buffers
It use RandomAccessFile to allow using heap buffers without additional
copies and/or leaks of direct buffers, as performed by FileChannel JDK
implementation (see https://bugs.openjdk.java.net/browse/JDK-8147468)
2019-09-23 10:55:20 -04:00
Clebert Suconic 5016f69daa [maven-release-plugin] prepare for next development iteration 2019-08-26 10:42:13 -04:00
Clebert Suconic f092d5396a [maven-release-plugin] prepare release 2.10.0 2019-08-26 10:42:01 -04:00
Clebert Suconic aa5d76e1bb ARTEMIS-2414 AIOSequentialFile was ignoring sync and leaking files 2019-07-30 11:03:06 -04:00
Wei Yang edace8845e ARTEMIS-2414 Sync before closing file in case data loss 2019-07-29 14:44:26 -04:00
Clebert Suconic ebbc500b82 NO-JIRA Duplicate Call on getEncodeSize 2019-07-16 15:15:43 -04:00
yang wei dd62be1602 ARTEMIS-2414 Sync before closing file in case data loss 2019-07-15 23:53:44 +01:00
Francesco Nigro 4445261f34 ARTEMIS-2382 Reclaimer doesn't need to be instantiatable 2019-06-17 22:31:30 -04:00
brusdev 9b52547ff7 ARTEMIS-2371 Message with huge header shuts broker down
Add max record size check before adding a record to prevent that the
broker shuts down, when there is one really large header sent with the
message. Add message size check before allocating large message resource
if it can't be stored.
2019-06-10 17:36:18 -04:00
Justin Bertram dbb2b54fa7 [maven-release-plugin] prepare for next development iteration 2019-05-29 22:02:00 -05:00
Justin Bertram 88c686cc99 [maven-release-plugin] prepare release 2.9.0 2019-05-29 21:59:18 -05:00
Clebert Suconic 2ae732cfc8 [maven-release-plugin] prepare for next development iteration 2019-05-16 09:58:27 -04:00
Clebert Suconic 481b9824a8 [maven-release-plugin] prepare release 2.8.1 2019-05-16 09:58:15 -04:00
Clebert Suconic 56152a9e7c [maven-release-plugin] prepare for next development iteration 2019-05-02 16:39:10 -04:00
Clebert Suconic a8e7f9d07a [maven-release-plugin] prepare release 2.8.0 2019-05-02 16:38:58 -04:00
Jiri Danek f7a36300ef ARTEMIS-2320 Fix IntLongMath errorprone warning
Expression of type int may overflow before being assigned to a long
    (see https://errorprone.info/bugpattern/IntLongMath)
2019-04-29 11:27:29 +01:00
Clebert Suconic 228f6af4ca [maven-release-plugin] prepare for next development iteration 2019-04-25 16:52:09 -04:00
Clebert Suconic 09d2ba18a7 [maven-release-plugin] prepare release 2.8.0 2019-04-25 16:51:57 -04:00
Clebert Suconic 516b1a1b9f [maven-release-plugin] prepare for next development iteration 2019-03-14 12:41:04 -04:00
Clebert Suconic c3eb6c12ad [maven-release-plugin] prepare release 2.7.0 2019-03-14 12:40:52 -04:00
Clebert Suconic 828a4856da ARTEMIS-2274 Fix on Journal buffer overflow with almost large messages 2019-03-13 14:56:46 -04:00
Francesco Nigro 89c02f1cc0 ARTEMIS-2211 Refactor ByteBuffer pooling, alignment and zeroing
Refactored thread local ByteBuffer pooling, alignment
and zeroing in order to avoid duplicate code and
improve code coverage with tests.
In addition are being provided faster branchless
alignment operations and optional zeroing of
pooled ByteBuffers for both ASYNCIO and
NIO/MAPPED journal types.
2019-03-12 18:32:53 -04:00
Clebert Suconic 061fb2787e ARTEMIS-1977 Stripping activemq-artemis as a separated proejct 2019-03-07 11:34:15 -05:00
Francesco Nigro 4da9d84311 ARTEMIS-2239 Zero-copy NIO/MAPPED TimedBuffer
NIO/MAPPED journal types can use directly the buffer of TimedBuffer
to perform file writes, avoiding an expensive copy + zeroing.
2019-02-14 00:20:51 +00:00
Clebert Suconic 05d893036e ARTEMIS-2200 NPE fix
This was a typo on my last commit around ARTEMIS-2200
2019-01-17 21:51:30 -05:00
Justin Bertram 57aacf784c NO-JIRA fix lgtm.com warnings
Warnings enumerated at
https://lgtm.com/projects/g/apache/activemq-artemis/alerts/?mode=tree&severity=warning
2019-01-17 22:30:28 +00:00
Clebert Suconic b3f0a87e0d ARTEMIS-2200 NPE when calling journal.delete from Multiple Threads 2019-01-17 15:38:51 -05:00
Francesco Nigro 702f445205 ARTEMIS-2200 NPE while dropping/failing large messages on paging
Large messages pendingRecordID is not accessed atomically, leading
to races that would lead to records that cannot been found on the
journal for deletion: it would lead to cause NPE that won't clean
the pending tasks on the current OperationContextImpl.
Adding a cleanup on error of those tasks and avoiding the race
to happen by adding proper synchronization will both enforce
correct clean up when something bad happen and avoid NPE.
2019-01-17 14:35:25 -05:00
Francesco Nigro 1c0ef5d7f7 ARTEMIS-2196 Avoid creating RandomAccessFile when FileChannel is needed 2018-12-14 15:10:30 -06:00
feuillemorte a3001fd9bd ARTEMIS-2201 Added tests on DEFAULT_JOURNAL_FILE_OPEN_TIMEOUT value 2018-12-14 11:32:13 +01:00
Francesco Nigro 2967df6a99 ARTEMIS-1996 MappedSequentialFileFactory may cause DirectByteBuffer off-heap memory leaks
Compaction is now reusing direct ByteBuffers on both
reading and writing with explicit and deterministic
release to avoid high peak of native memory utilisation
after compaction.
2018-10-22 09:16:15 +02:00
Francesco Nigro cae253d136 ARTEMIS-2131 Error compacting journal
Compaction cannot free a sliced view of a ByteBuffer on Java >=9:
the fix is using the original ByteBuffer instead of the slice
to perform a file write and allow it to be correctly released by
the Cleaner.
2018-10-17 20:49:50 -04:00
Clebert Suconic 6b1abd1aad ARTEMIS-2053 avoiding data loss after compacting 2018-08-26 18:17:29 -04:00
Clebert Suconic 63e6cd98f8 ARTEMIS-2046 Fixing issues with JournalStorageManager.stop in replication, JDBC and shared storage 2018-08-20 21:25:57 -04:00
Justin Bertram 0ae7d32532 ARTEMIS-2014 Treat inability to create directory for paging as critial 2018-08-07 11:09:46 -05:00
Clebert Suconic 56be281aaf ARTEMIS-1989 Replication catch up leaking files
Related commit that broke this at 837694e705
2018-07-25 15:03:57 +01:00
yang wei 1d5128839f ARTEMIS-1923 fix constructing binding journalimpl bug
Clebert Suconic ammended this commit by adding a test and getter properties to satisfy the test

This closes #2133
2018-06-12 12:30:25 -04:00
Clebert Suconic af56b67e38 [maven-release-plugin] prepare for next development iteration 2018-05-16 11:03:26 -04:00
Clebert Suconic bcaa11c2ee [maven-release-plugin] prepare release 2.6.0 2018-05-16 11:03:14 -04:00
Kui LIU be6705afac ARTEMIS-1822: Change the method name 'lookupRecord' to 'containsRecord'.
The method is named "lookupRecord".
"lookupRecord" seems to find a related record.
But the method is checking whether recordsSnapshot contains the id or not.
Thus, the method name "containsRecord" is more intuitive than "lookupRecord".
2018-04-26 21:25:37 -04:00
Clebert Suconic 4dd594f38b ARTEMIS-618 Using proper block size on Native One Mega Buffer
This buffer is used on fill
2018-04-05 10:49:57 -05:00
Clebert Suconic 65db317176 NO-JIRA Providing information about failure on test
Travis CI has been reporting test failures.
Looking on logs I could see a critical failure happening but not much information on why.
This will help identify further issues.
2018-04-04 16:05:48 -04:00
Clebert Suconic 920af19adb [maven-release-plugin] prepare for next development iteration 2018-03-12 12:46:27 -04:00
Clebert Suconic 5f5b1dcf8f [maven-release-plugin] prepare release 2.5.0 2018-03-12 12:46:16 -04:00
Clebert Suconic c77fa6c897 NO-JIRA Test tweaks
small fix and System.out
2018-02-20 23:44:16 -05:00
Clebert Suconic 0273e3e4ff NO-JIRA Fixing a deadlock during #tearDown and TimedBuffer.stop 2018-02-20 15:42:27 -05:00
Christopher L. Shannon (cshannon) ea70af15a3 ARTEMIS-1663 - Add new message count and size metrics
Adding new metrics for tracking message counts and sizes on a Queue.
This includes tracking metrics for pending, delivering and scheduled
messages.  The paging store also tracks message size now.
2018-02-08 11:35:12 -05:00
Francesco Nigro 78a2e3a8f0 ARTEMIS-1643 Compaction must check against NULL records while replaying
JournalCompactor.UpdateCompactCommand::execute is checking if updateRecord is null to avoid on replay under huge load that will be thrown AMQ142028.
2018-01-31 10:34:16 +08:00
Clebert Suconic 61a1123ee1 ARTEMIS-1613 Integrating JDBC into CLI (create print-data and exp) 2018-01-18 20:55:00 -06:00
shoukun c4bfb9521f ARTEMIS-1570 Flush appendExecutor before take journal snapshot
When live start replication, it must make sure there is
no pending write in message & bindings journal, or we may
lost journal records during initial replication.

So we need flush append executor after acquire StorageManager's
write lock, before Journal's write lock.
Also we set a 10 seconds timeout when flush, the same as
Journal::flushExecutor. If we failed to flush in 10 seconds,
we abort replication, backup will try again later.

Use OrderedExecutorFactory::flushExecutor to flush executor
2018-01-18 13:16:27 -05:00
Justin Bertram 94b53c67c0 ARTEMIS-1406 removing impossible instanceof 2018-01-03 10:50:13 +08:00
Clebert Suconic f9d101d0a1 ARTEMIS-1531 Adding timedbuffer operations on critical analyzer
Also, making TimedBuffer.stop() synchronized to avoid issues during device outages
2017-11-29 18:09:00 -05:00
Justin Bertram 55d7260a07 ARTEMIS-1510 refactor Maven poms
Clean up unused declared dependencies and undeclared dependencies which
are pulled in transitively.
2017-11-13 17:03:35 -05:00
Francesco Nigro 3950169c21 ARTEMIS-1471 Needs Bounds Checking on writes for MappedSequentialFile
The MappedSequentialFile relies on the assumption that any writers
won't exceed the maximum capacity of the file, leaving the JVM to crash otherwise.
This commit adds proper bounds checking on write operations (and position changes too)
in order to provide recoverable effects if such scenario should occour.
In addition are provided minor fixes on Mapped and Nio SequentialFile::fill behaviour
to match the original contract.
2017-11-06 13:52:57 -05:00
Justin Bertram 8703d9d51d [maven-release-plugin] prepare for next development iteration 2017-11-01 00:38:57 -05:00
Justin Bertram ec63189a0a [maven-release-plugin] prepare release 2.4.0 2017-11-01 00:38:56 -05:00
Justin Bertram a44b9d5edf [maven-release-plugin] prepare for next development iteration 2017-10-31 12:09:43 -05:00