Commit Graph

280 Commits

Author SHA1 Message Date
Clebert Suconic 3541ebf218 ARTEMIS-2732 Cleanup on testsuite logging 2020-04-22 23:08:46 -04:00
Clebert Suconic 11a3e810bb [maven-release-plugin] prepare for next development iteration 2020-04-21 17:06:21 -04:00
Clebert Suconic 30272e0c2f [maven-release-plugin] prepare release 2.12.0 2020-04-21 17:06:08 -04:00
Clebert Suconic d231e2ac63 [maven-release-plugin] prepare for next development iteration 2020-04-16 16:16:41 -04:00
Clebert Suconic 9636f4a3b0 [maven-release-plugin] prepare release 2.12.0 2020-04-16 16:16:28 -04:00
Clebert Suconic af796d5ce4 ARTEMIS-2701 Improving DLQ/check over previously removed records 2020-04-06 12:25:35 -04:00
Clebert Suconic 03fb630f73 ARTEMIS-1975 Fixing LargeMessage encoding for replication 2020-03-25 16:16:05 -04:00
Clebert Suconic 31c945f8b0 ARTEMIS-1975 Fixing encodesize cached on AMQP Large Message
Encoding ahead is broken with AMQP Large Message
This is still part of ARTEMIS-1975
2020-03-24 15:51:46 -04:00
Clebert Suconic ddd8ed4402 ARTEMIS-1975 Real Large Message support into AMQP
This is a Large commit where I am refactoring largeMessage Body out of CoreMessage
which is now reused with AMQP.

I had also to fix Reference Counting to fix how Large Messages are Acked

And I also had to make sure Large Messages are transversing correctly when in cluster.
2020-02-25 15:25:01 +01:00
Francesco Nigro 5897909dc9 ARTEMIS-2617 use core pools to reduce GC on journal loading 2020-02-12 13:29:51 -05:00
Francesco Nigro fa0c187ae6 ARTEMIS-2602 Reduce number of copies for non JDBC Journal 2020-01-18 18:45:12 +01:00
Francesco Nigro b10d765139 ARTEMIS-2602 load surviving records into SparseArrayLinkedList 2020-01-17 14:31:52 -05:00
Francesco Nigro 4cc6464ddd ARTEMIS-2602 Improve Journal loading heap usage 2020-01-17 14:31:52 -05:00
Justin Bertram 97735ca72c [maven-release-plugin] prepare for next development iteration 2020-01-10 09:00:58 -06:00
Justin Bertram 25a947f6cb [maven-release-plugin] prepare release 2.11.0 2020-01-10 08:57:46 -06:00
Clebert Suconic 91f4d0a633 [maven-release-plugin] prepare for next development iteration 2019-09-23 11:19:54 -04:00
Clebert Suconic 86a975fdad [maven-release-plugin] prepare release 2.10.1 2019-09-23 11:19:42 -04:00
Francesco Nigro f51c799ac0 ARTEMIS-1811 NIO Seq File should use RandomAccessFile with heap buffers
It use RandomAccessFile to allow using heap buffers without additional
copies and/or leaks of direct buffers, as performed by FileChannel JDK
implementation (see https://bugs.openjdk.java.net/browse/JDK-8147468)
2019-09-23 10:55:20 -04:00
Clebert Suconic 5016f69daa [maven-release-plugin] prepare for next development iteration 2019-08-26 10:42:13 -04:00
Clebert Suconic f092d5396a [maven-release-plugin] prepare release 2.10.0 2019-08-26 10:42:01 -04:00
Clebert Suconic aa5d76e1bb ARTEMIS-2414 AIOSequentialFile was ignoring sync and leaking files 2019-07-30 11:03:06 -04:00
Wei Yang edace8845e ARTEMIS-2414 Sync before closing file in case data loss 2019-07-29 14:44:26 -04:00
Clebert Suconic ebbc500b82 NO-JIRA Duplicate Call on getEncodeSize 2019-07-16 15:15:43 -04:00
yang wei dd62be1602 ARTEMIS-2414 Sync before closing file in case data loss 2019-07-15 23:53:44 +01:00
Francesco Nigro 4445261f34 ARTEMIS-2382 Reclaimer doesn't need to be instantiatable 2019-06-17 22:31:30 -04:00
brusdev 9b52547ff7 ARTEMIS-2371 Message with huge header shuts broker down
Add max record size check before adding a record to prevent that the
broker shuts down, when there is one really large header sent with the
message. Add message size check before allocating large message resource
if it can't be stored.
2019-06-10 17:36:18 -04:00
Justin Bertram dbb2b54fa7 [maven-release-plugin] prepare for next development iteration 2019-05-29 22:02:00 -05:00
Justin Bertram 88c686cc99 [maven-release-plugin] prepare release 2.9.0 2019-05-29 21:59:18 -05:00
Clebert Suconic 2ae732cfc8 [maven-release-plugin] prepare for next development iteration 2019-05-16 09:58:27 -04:00
Clebert Suconic 481b9824a8 [maven-release-plugin] prepare release 2.8.1 2019-05-16 09:58:15 -04:00
Clebert Suconic 56152a9e7c [maven-release-plugin] prepare for next development iteration 2019-05-02 16:39:10 -04:00
Clebert Suconic a8e7f9d07a [maven-release-plugin] prepare release 2.8.0 2019-05-02 16:38:58 -04:00
Jiri Danek f7a36300ef ARTEMIS-2320 Fix IntLongMath errorprone warning
Expression of type int may overflow before being assigned to a long
    (see https://errorprone.info/bugpattern/IntLongMath)
2019-04-29 11:27:29 +01:00
Clebert Suconic 228f6af4ca [maven-release-plugin] prepare for next development iteration 2019-04-25 16:52:09 -04:00
Clebert Suconic 09d2ba18a7 [maven-release-plugin] prepare release 2.8.0 2019-04-25 16:51:57 -04:00
Clebert Suconic 516b1a1b9f [maven-release-plugin] prepare for next development iteration 2019-03-14 12:41:04 -04:00
Clebert Suconic c3eb6c12ad [maven-release-plugin] prepare release 2.7.0 2019-03-14 12:40:52 -04:00
Clebert Suconic 828a4856da ARTEMIS-2274 Fix on Journal buffer overflow with almost large messages 2019-03-13 14:56:46 -04:00
Francesco Nigro 89c02f1cc0 ARTEMIS-2211 Refactor ByteBuffer pooling, alignment and zeroing
Refactored thread local ByteBuffer pooling, alignment
and zeroing in order to avoid duplicate code and
improve code coverage with tests.
In addition are being provided faster branchless
alignment operations and optional zeroing of
pooled ByteBuffers for both ASYNCIO and
NIO/MAPPED journal types.
2019-03-12 18:32:53 -04:00
Clebert Suconic 061fb2787e ARTEMIS-1977 Stripping activemq-artemis as a separated proejct 2019-03-07 11:34:15 -05:00
Francesco Nigro 4da9d84311 ARTEMIS-2239 Zero-copy NIO/MAPPED TimedBuffer
NIO/MAPPED journal types can use directly the buffer of TimedBuffer
to perform file writes, avoiding an expensive copy + zeroing.
2019-02-14 00:20:51 +00:00
Clebert Suconic 05d893036e ARTEMIS-2200 NPE fix
This was a typo on my last commit around ARTEMIS-2200
2019-01-17 21:51:30 -05:00
Justin Bertram 57aacf784c NO-JIRA fix lgtm.com warnings
Warnings enumerated at
https://lgtm.com/projects/g/apache/activemq-artemis/alerts/?mode=tree&severity=warning
2019-01-17 22:30:28 +00:00
Clebert Suconic b3f0a87e0d ARTEMIS-2200 NPE when calling journal.delete from Multiple Threads 2019-01-17 15:38:51 -05:00
Francesco Nigro 702f445205 ARTEMIS-2200 NPE while dropping/failing large messages on paging
Large messages pendingRecordID is not accessed atomically, leading
to races that would lead to records that cannot been found on the
journal for deletion: it would lead to cause NPE that won't clean
the pending tasks on the current OperationContextImpl.
Adding a cleanup on error of those tasks and avoiding the race
to happen by adding proper synchronization will both enforce
correct clean up when something bad happen and avoid NPE.
2019-01-17 14:35:25 -05:00
Francesco Nigro 1c0ef5d7f7 ARTEMIS-2196 Avoid creating RandomAccessFile when FileChannel is needed 2018-12-14 15:10:30 -06:00
feuillemorte a3001fd9bd ARTEMIS-2201 Added tests on DEFAULT_JOURNAL_FILE_OPEN_TIMEOUT value 2018-12-14 11:32:13 +01:00
Francesco Nigro 2967df6a99 ARTEMIS-1996 MappedSequentialFileFactory may cause DirectByteBuffer off-heap memory leaks
Compaction is now reusing direct ByteBuffers on both
reading and writing with explicit and deterministic
release to avoid high peak of native memory utilisation
after compaction.
2018-10-22 09:16:15 +02:00
Francesco Nigro cae253d136 ARTEMIS-2131 Error compacting journal
Compaction cannot free a sliced view of a ByteBuffer on Java >=9:
the fix is using the original ByteBuffer instead of the slice
to perform a file write and allow it to be correctly released by
the Cleaner.
2018-10-17 20:49:50 -04:00
Clebert Suconic 6b1abd1aad ARTEMIS-2053 avoiding data loss after compacting 2018-08-26 18:17:29 -04:00
Clebert Suconic 63e6cd98f8 ARTEMIS-2046 Fixing issues with JournalStorageManager.stop in replication, JDBC and shared storage 2018-08-20 21:25:57 -04:00
Justin Bertram 0ae7d32532 ARTEMIS-2014 Treat inability to create directory for paging as critial 2018-08-07 11:09:46 -05:00
Clebert Suconic 56be281aaf ARTEMIS-1989 Replication catch up leaking files
Related commit that broke this at 837694e705
2018-07-25 15:03:57 +01:00
yang wei 1d5128839f ARTEMIS-1923 fix constructing binding journalimpl bug
Clebert Suconic ammended this commit by adding a test and getter properties to satisfy the test

This closes #2133
2018-06-12 12:30:25 -04:00
Clebert Suconic af56b67e38 [maven-release-plugin] prepare for next development iteration 2018-05-16 11:03:26 -04:00
Clebert Suconic bcaa11c2ee [maven-release-plugin] prepare release 2.6.0 2018-05-16 11:03:14 -04:00
Kui LIU be6705afac ARTEMIS-1822: Change the method name 'lookupRecord' to 'containsRecord'.
The method is named "lookupRecord".
"lookupRecord" seems to find a related record.
But the method is checking whether recordsSnapshot contains the id or not.
Thus, the method name "containsRecord" is more intuitive than "lookupRecord".
2018-04-26 21:25:37 -04:00
Clebert Suconic 4dd594f38b ARTEMIS-618 Using proper block size on Native One Mega Buffer
This buffer is used on fill
2018-04-05 10:49:57 -05:00
Clebert Suconic 65db317176 NO-JIRA Providing information about failure on test
Travis CI has been reporting test failures.
Looking on logs I could see a critical failure happening but not much information on why.
This will help identify further issues.
2018-04-04 16:05:48 -04:00
Clebert Suconic 920af19adb [maven-release-plugin] prepare for next development iteration 2018-03-12 12:46:27 -04:00
Clebert Suconic 5f5b1dcf8f [maven-release-plugin] prepare release 2.5.0 2018-03-12 12:46:16 -04:00
Clebert Suconic c77fa6c897 NO-JIRA Test tweaks
small fix and System.out
2018-02-20 23:44:16 -05:00
Clebert Suconic 0273e3e4ff NO-JIRA Fixing a deadlock during #tearDown and TimedBuffer.stop 2018-02-20 15:42:27 -05:00
Christopher L. Shannon (cshannon) ea70af15a3 ARTEMIS-1663 - Add new message count and size metrics
Adding new metrics for tracking message counts and sizes on a Queue.
This includes tracking metrics for pending, delivering and scheduled
messages.  The paging store also tracks message size now.
2018-02-08 11:35:12 -05:00
Francesco Nigro 78a2e3a8f0 ARTEMIS-1643 Compaction must check against NULL records while replaying
JournalCompactor.UpdateCompactCommand::execute is checking if updateRecord is null to avoid on replay under huge load that will be thrown AMQ142028.
2018-01-31 10:34:16 +08:00
Clebert Suconic 61a1123ee1 ARTEMIS-1613 Integrating JDBC into CLI (create print-data and exp) 2018-01-18 20:55:00 -06:00
shoukun c4bfb9521f ARTEMIS-1570 Flush appendExecutor before take journal snapshot
When live start replication, it must make sure there is
no pending write in message & bindings journal, or we may
lost journal records during initial replication.

So we need flush append executor after acquire StorageManager's
write lock, before Journal's write lock.
Also we set a 10 seconds timeout when flush, the same as
Journal::flushExecutor. If we failed to flush in 10 seconds,
we abort replication, backup will try again later.

Use OrderedExecutorFactory::flushExecutor to flush executor
2018-01-18 13:16:27 -05:00
Justin Bertram 94b53c67c0 ARTEMIS-1406 removing impossible instanceof 2018-01-03 10:50:13 +08:00
Clebert Suconic f9d101d0a1 ARTEMIS-1531 Adding timedbuffer operations on critical analyzer
Also, making TimedBuffer.stop() synchronized to avoid issues during device outages
2017-11-29 18:09:00 -05:00
Justin Bertram 55d7260a07 ARTEMIS-1510 refactor Maven poms
Clean up unused declared dependencies and undeclared dependencies which
are pulled in transitively.
2017-11-13 17:03:35 -05:00
Francesco Nigro 3950169c21 ARTEMIS-1471 Needs Bounds Checking on writes for MappedSequentialFile
The MappedSequentialFile relies on the assumption that any writers
won't exceed the maximum capacity of the file, leaving the JVM to crash otherwise.
This commit adds proper bounds checking on write operations (and position changes too)
in order to provide recoverable effects if such scenario should occour.
In addition are provided minor fixes on Mapped and Nio SequentialFile::fill behaviour
to match the original contract.
2017-11-06 13:52:57 -05:00
Justin Bertram 8703d9d51d [maven-release-plugin] prepare for next development iteration 2017-11-01 00:38:57 -05:00
Justin Bertram ec63189a0a [maven-release-plugin] prepare release 2.4.0 2017-11-01 00:38:56 -05:00
Justin Bertram a44b9d5edf [maven-release-plugin] prepare for next development iteration 2017-10-31 12:09:43 -05:00
Justin Bertram 34a7431d1b [maven-release-plugin] prepare release 2.4.0 2017-10-31 12:07:55 -05:00
Clebert Suconic 2bf690e21b ARTEMIS-1489 Adding Timed Buffer into Critical Analyzer 2017-10-31 08:33:44 -05:00
Clebert Suconic febd1f509c NO-JIRA Individualizing Loggers on TimedBuffer 2017-10-23 14:53:02 -04:00
Francesco Nigro f5dfbf7f12 ARTEMIS-1474 TimedBuffer improved doc and refactored dead brenches on methods 2017-10-23 14:44:09 -04:00
Clebert Suconic d190b611be ARTEMIS-1452 Improvements to IO parameters and options
- it is now possible to disable the TimedBuffer
- this is increasing the default on libaio maxAIO to 4k
- The Auto Tuning on the journal will use asynchronous writes to simulate what would happen on faster disks
- If you set datasync=false on the CLI, the system will suggest mapped and disable the buffer timeout

This closes #1436

This commit superseeds #1436 since it's now disabling the timed buffer through the CLI
2017-10-09 14:32:04 -04:00
Clebert Suconic ba1323c8b2 ARTEMIS-1455 Fixing issues on Large Message conversion 2017-10-09 14:32:03 -04:00
Martyn Taylor 988c91557d ARTEMIS-1444 Support Messages > JournalBufferSize in all Protocols 2017-10-09 12:50:38 -04:00
Clebert Suconic 30ba65a082 NO-JIRA: Fixing the testsuite on Page.finalize() 2017-10-07 12:05:55 -04:00
Clebert Suconic bb554e5264 ARTEMIS-1397 Removing Netty Copied classes
instead of duplicating a buffer from Netty, this will use an existing Wrapped Unpooled Buffer
Which will in turn use Unsafe Properly.
2017-10-02 17:06:06 -04:00
Ryan Yeats 420421c90b ARTEMIS-1397 remove package private access to netty 2017-10-02 09:33:47 -04:00
Clebert Suconic aa3e8941d1 ARTEMIS-1442 Shutdown server if can't move file on journal 2017-09-29 11:35:47 -04:00
Martyn Taylor 520a40b1a1 ARTEMIS-1418 AIO Shutdown on IOError and logging 2017-09-13 16:09:21 +01:00
Francesco Nigro 35c3475092 ARTEMIS-1401 Numerical overflow fix when using System::nanoTime 2017-09-07 10:40:14 -04:00
Clebert Suconic c8982d775b [maven-release-plugin] prepare for next development iteration 2017-09-05 17:03:48 -04:00
Clebert Suconic 84d5ac65b4 [maven-release-plugin] prepare release 2.3.0 2017-09-05 17:03:37 -04:00
Clebert Suconic ed1b268d42 [maven-release-plugin] prepare for next development iteration 2017-08-31 13:48:09 -04:00
Clebert Suconic 21f2a4a52c [maven-release-plugin] prepare release 2.3.0 2017-08-31 13:47:57 -04:00
Martyn Taylor 652573a5a9 ARTEMIS-1354 Critical IO Error on AIO write failure 2017-08-17 12:29:13 -04:00
Michael André Pearce 684dd24cc7 ARTEMIS-1335 Update Netty to 4.1.14
Update Netty to 4.1.14
2017-08-09 15:20:49 -04:00
Clebert Suconic 1ace306121 ARTEMIS-1328 Improving direct delivery check
Instead of wait to flush an executor,
I have added a method isFlushed() which will just translate to the
state on the OrderedExecutor.

In the case another executor is provided (for tests) there's a delegate
into normal executors.
2017-08-08 14:00:58 -04:00
Francesco Nigro 74f243cc4d ARTEMIS-1312 TimedBuffer doubled timeout with blocking flush 2017-08-01 11:52:28 -04:00
Francesco Nigro 567bfe3b9b ARTEMIS-1312 TimedBuffer doubled timeout with blocking flush 2017-08-01 06:34:25 -04:00
Clebert Suconic fdad83be22 [maven-release-plugin] prepare for next development iteration 2017-07-24 21:21:18 -04:00
Clebert Suconic 71b1cc2a20 [maven-release-plugin] prepare release 2.2.0 2017-07-24 21:21:06 -04:00
Clebert Suconic ad372ec98e ARTEMIS-1294 Using older sleep on TimedBuffer
And also adding test
2017-07-18 16:01:51 -04:00
Francesco Nigro 3dc9566fb6 ARTEMIS-1294 Reverted TimedBuffer timeout policy 2017-07-18 10:23:02 -04:00