Commit Graph

167 Commits

Author SHA1 Message Date
Christopher L. Shannon (cshannon) ea70af15a3 ARTEMIS-1663 - Add new message count and size metrics
Adding new metrics for tracking message counts and sizes on a Queue.
This includes tracking metrics for pending, delivering and scheduled
messages.  The paging store also tracks message size now.
2018-02-08 11:35:12 -05:00
Francesco Nigro 78a2e3a8f0 ARTEMIS-1643 Compaction must check against NULL records while replaying
JournalCompactor.UpdateCompactCommand::execute is checking if updateRecord is null to avoid on replay under huge load that will be thrown AMQ142028.
2018-01-31 10:34:16 +08:00
Clebert Suconic 61a1123ee1 ARTEMIS-1613 Integrating JDBC into CLI (create print-data and exp) 2018-01-18 20:55:00 -06:00
shoukun c4bfb9521f ARTEMIS-1570 Flush appendExecutor before take journal snapshot
When live start replication, it must make sure there is
no pending write in message & bindings journal, or we may
lost journal records during initial replication.

So we need flush append executor after acquire StorageManager's
write lock, before Journal's write lock.
Also we set a 10 seconds timeout when flush, the same as
Journal::flushExecutor. If we failed to flush in 10 seconds,
we abort replication, backup will try again later.

Use OrderedExecutorFactory::flushExecutor to flush executor
2018-01-18 13:16:27 -05:00
Justin Bertram 94b53c67c0 ARTEMIS-1406 removing impossible instanceof 2018-01-03 10:50:13 +08:00
Clebert Suconic f9d101d0a1 ARTEMIS-1531 Adding timedbuffer operations on critical analyzer
Also, making TimedBuffer.stop() synchronized to avoid issues during device outages
2017-11-29 18:09:00 -05:00
Justin Bertram 55d7260a07 ARTEMIS-1510 refactor Maven poms
Clean up unused declared dependencies and undeclared dependencies which
are pulled in transitively.
2017-11-13 17:03:35 -05:00
Francesco Nigro 3950169c21 ARTEMIS-1471 Needs Bounds Checking on writes for MappedSequentialFile
The MappedSequentialFile relies on the assumption that any writers
won't exceed the maximum capacity of the file, leaving the JVM to crash otherwise.
This commit adds proper bounds checking on write operations (and position changes too)
in order to provide recoverable effects if such scenario should occour.
In addition are provided minor fixes on Mapped and Nio SequentialFile::fill behaviour
to match the original contract.
2017-11-06 13:52:57 -05:00
Justin Bertram 8703d9d51d [maven-release-plugin] prepare for next development iteration 2017-11-01 00:38:57 -05:00
Justin Bertram ec63189a0a [maven-release-plugin] prepare release 2.4.0 2017-11-01 00:38:56 -05:00
Justin Bertram a44b9d5edf [maven-release-plugin] prepare for next development iteration 2017-10-31 12:09:43 -05:00
Justin Bertram 34a7431d1b [maven-release-plugin] prepare release 2.4.0 2017-10-31 12:07:55 -05:00
Clebert Suconic 2bf690e21b ARTEMIS-1489 Adding Timed Buffer into Critical Analyzer 2017-10-31 08:33:44 -05:00
Clebert Suconic febd1f509c NO-JIRA Individualizing Loggers on TimedBuffer 2017-10-23 14:53:02 -04:00
Francesco Nigro f5dfbf7f12 ARTEMIS-1474 TimedBuffer improved doc and refactored dead brenches on methods 2017-10-23 14:44:09 -04:00
Clebert Suconic d190b611be ARTEMIS-1452 Improvements to IO parameters and options
- it is now possible to disable the TimedBuffer
- this is increasing the default on libaio maxAIO to 4k
- The Auto Tuning on the journal will use asynchronous writes to simulate what would happen on faster disks
- If you set datasync=false on the CLI, the system will suggest mapped and disable the buffer timeout

This closes #1436

This commit superseeds #1436 since it's now disabling the timed buffer through the CLI
2017-10-09 14:32:04 -04:00
Clebert Suconic ba1323c8b2 ARTEMIS-1455 Fixing issues on Large Message conversion 2017-10-09 14:32:03 -04:00
Martyn Taylor 988c91557d ARTEMIS-1444 Support Messages > JournalBufferSize in all Protocols 2017-10-09 12:50:38 -04:00
Clebert Suconic 30ba65a082 NO-JIRA: Fixing the testsuite on Page.finalize() 2017-10-07 12:05:55 -04:00
Clebert Suconic bb554e5264 ARTEMIS-1397 Removing Netty Copied classes
instead of duplicating a buffer from Netty, this will use an existing Wrapped Unpooled Buffer
Which will in turn use Unsafe Properly.
2017-10-02 17:06:06 -04:00
Ryan Yeats 420421c90b ARTEMIS-1397 remove package private access to netty 2017-10-02 09:33:47 -04:00
Clebert Suconic aa3e8941d1 ARTEMIS-1442 Shutdown server if can't move file on journal 2017-09-29 11:35:47 -04:00
Martyn Taylor 520a40b1a1 ARTEMIS-1418 AIO Shutdown on IOError and logging 2017-09-13 16:09:21 +01:00
Francesco Nigro 35c3475092 ARTEMIS-1401 Numerical overflow fix when using System::nanoTime 2017-09-07 10:40:14 -04:00
Clebert Suconic c8982d775b [maven-release-plugin] prepare for next development iteration 2017-09-05 17:03:48 -04:00
Clebert Suconic 84d5ac65b4 [maven-release-plugin] prepare release 2.3.0 2017-09-05 17:03:37 -04:00
Clebert Suconic ed1b268d42 [maven-release-plugin] prepare for next development iteration 2017-08-31 13:48:09 -04:00
Clebert Suconic 21f2a4a52c [maven-release-plugin] prepare release 2.3.0 2017-08-31 13:47:57 -04:00
Martyn Taylor 652573a5a9 ARTEMIS-1354 Critical IO Error on AIO write failure 2017-08-17 12:29:13 -04:00
Michael André Pearce 684dd24cc7 ARTEMIS-1335 Update Netty to 4.1.14
Update Netty to 4.1.14
2017-08-09 15:20:49 -04:00
Clebert Suconic 1ace306121 ARTEMIS-1328 Improving direct delivery check
Instead of wait to flush an executor,
I have added a method isFlushed() which will just translate to the
state on the OrderedExecutor.

In the case another executor is provided (for tests) there's a delegate
into normal executors.
2017-08-08 14:00:58 -04:00
Francesco Nigro 74f243cc4d ARTEMIS-1312 TimedBuffer doubled timeout with blocking flush 2017-08-01 11:52:28 -04:00
Francesco Nigro 567bfe3b9b ARTEMIS-1312 TimedBuffer doubled timeout with blocking flush 2017-08-01 06:34:25 -04:00
Clebert Suconic fdad83be22 [maven-release-plugin] prepare for next development iteration 2017-07-24 21:21:18 -04:00
Clebert Suconic 71b1cc2a20 [maven-release-plugin] prepare release 2.2.0 2017-07-24 21:21:06 -04:00
Clebert Suconic ad372ec98e ARTEMIS-1294 Using older sleep on TimedBuffer
And also adding test
2017-07-18 16:01:51 -04:00
Francesco Nigro 3dc9566fb6 ARTEMIS-1294 Reverted TimedBuffer timeout policy 2017-07-18 10:23:02 -04:00
Clebert Suconic 31d5758885 ARTEMIS-1288 crash during compact control files issues
The system would become irresponsive if crashed right at the
control file writing.
2017-07-13 17:12:58 -04:00
Clebert Suconic 7fd17f407f ARTEMIS-1269 Simple Actor to replace certain executions
This is replacing an executor on ServerSessionPacketHandler
by a this actor.

This is to avoid creating a new runnable per packet received.

Instead of creating new Runnable, this will use a single static runnable
and the packet will be send by a message, which will be treated by a listener.

Look at ServerSessionPacketHandler on this commit for more information on how it works.
2017-07-11 14:22:36 -04:00
Clebert Suconic 89e84e1320 ARTEMIS-1269 Fixing blocked replication
If replication blocked anything on the journal
the processing from clients would be blocked
and nothing would work.

As part of this fix I am using an executor on ServerSessionPacketHandler
which will also scale better as the reader from Netty would be feed immediately.
2017-07-06 19:04:48 -04:00
Francesco Nigro 7075e2e457 ARTEMIS-1266 Mapped Journal refactoring
The MAPPED journal refactoring include:
 - simplified lifecycle and logic (eg fixed file size with single mmap memory region)
 - supports for the TimedBuffer to coalesce msyncs (via Decorator pattern)
 - TLAB pooling of direct ByteBuffer like the NIO journal
 - remove of old benchmarks and benchmark dependencies
2017-06-30 16:17:19 +02:00
Francesco Nigro 9f8a14440b ARTEMIS-1223 OutOfDirectMemoryError raised from TimedBuffer
Force allocation of direct ByteBuffer with Cleaner to help GC on tests.
2017-06-27 12:45:26 -04:00
Andy Taylor fbe27682a1 ARTEMIS-1237 - Hard coded timeout on Journal file open
Make timeout configurable

https://issues.apache.org/jira/browse/ARTEMIS-1237
2017-06-22 10:48:09 -04:00
Jiri Danek 3d4e907a1b NO-JIRA fix typos 2017-06-21 11:16:29 -04:00
Michael Andre Pearce c65ea783ea ARTEMIS-1189 - Fix checkstyle violations post checkstyle upgrade
After upgrade of checkstyle, resolve violations

remove checkstyle override added as temp measure at point of upgrade forced by sevntu
2017-05-30 13:40:00 -04:00
Clebert Suconic dc26ac96b4 ARTEMIS-1156: moving our collections on its own package 2017-05-12 10:06:05 -04:00
Michael André Pearce c1d55aa84f ARTEMIS-1156: FIX: Long Autoboxing occurring on Hot Path
Building on ARTEMIS-905 JCtools ConcurrentMap replacement  first proposed but currently parked by @franz1981, replace the collections with primitive key concurrent collections to avoid auto boxing.

The goal of this is to reduce/remove autoboxing on the hot path.
We are just adding jctools to the broker (should not be in client dependencies)
Like wise targeting specific use case with specific implementation rather than a blanket replace all.

Using collections from Bookkeeper, reduces outside tlab allocation, on resizing compared to JCTools, which occurs frequently on testing.
2017-05-12 10:05:51 -04:00
Clebert Suconic ce61d20f5a [maven-release-plugin] prepare for next development iteration 2017-05-08 15:20:12 -04:00
Clebert Suconic 64e8f015ee [maven-release-plugin] prepare release 2.1.0 2017-05-08 15:20:01 -04:00
Francesco Nigro 21c9ed85cf ARTEMIS-1151 Adapting TimedBuffer and NIO Buffer Pooling
- NIO/ASYNCIO new TimedBuffer with adapting batch window heuristic
- NIO/ASYNCIO improved TimedBuffer write monitoring with
  lightweight concurrent performance counters
- NIO/ASYNCIO journal/paging operations benefit from less buffer copy
- NIO/ASYNCIO any buffer copy is always performed with raw batch copy
  using SIMD instrinsics (System::arrayCopy) or memcpy under the hood
- NIO improved clear buffers using SIMD instrinsics (Arrays::fill) and/or memset
- NIO journal operation perform by default TLABs allocation pooling (off heap)
  retaining only the last max sized buffer
- NIO improved file copy operations using zero-copy FileChannel::transfertTo
- NIO improved zeroing using pooled single OS page buffer to clean the file
  + pwrite (on Linux)
- NIO deterministic release of unpooled direct buffers to avoid OOM errors
  due to slow GC
- Exposed OS PAGE SIZE value using Env class
2017-05-08 11:55:28 -04:00