This commit introduces support for configuring a specific Duplicate ID cache size per address in the Artemis server. Previously, there was only a global setting for the ID cache size, but now each address can have its own cache size.
The changes include the addition of a new configuration property id-cache-size in the Artemis server configuration file. This property can now be specified under each address setting in the configuration file, and its value will determine the Duplicate ID cache size for that particular address. If the id-cache-size property is not specified for an address, it will use the global setting.
The test cases have been updated to cover this new functionality, and integration test have been added to verify that address-specific cache sizes work as expected.
Documentation has been added to address-settings.adoc, configuration-index.adoc and duplicate-detection.adoc
Markdown, which is currently used for user-facing documentation, is good
for a lot of things. However, it's not great for the kind of complex
documentation we have and our need to produce both multi-page HTML and
single-page PDF output via Maven.
Markdown lacks features which would make the documentation easier to
read, easier to navigate, and just look better overall.
The current tool-chain uses honkit and a tool called Calibre. Honkit is
written in TypeScript and is installed via NPM. Calibre is a native tool
so it must be installed via an OS-specific package manager. All this
complexity makes building, releasing, uploading, etc. a pain.
AsciiDoc is relatively simple like Markdown, but it has more features
for presentation and navigation not to mention Java-based Maven tooling
to generate both HTML and PDF. Migrating will improve both the
appearance of the documentation as well as the processes to generate and
upload it.
This commit contains the following changes:
- Convert all the Markdown for the User Manual, Migration Guide, and
Hacking guide to AsciiDoc via kramdown [1].
- Update the `artemis-website` build to use AsciiDoctor Maven tooling.
- Update `RELEASING.md` with simplified instructions.
- Update Hacking Guide with simplified instructions.
- Use AsciiDoc link syntax in Artemis Maven doc plugin.
- Drop EPUB & MOBI docs for User Manual as well as PDF for the Hacking
Guide. All docs will be HTML only except for the User Manual which
will have PDF.
- Move all docs up out of their respective "en" directory. This was a
hold-over from when we had docs in different languages.
- Migration & Hacking Guides are now single-page HTML since they are
relatively short.
- Refactor README.md to simplify and remove redundant content.
Benefits of the change:
- Much simplified tooling. No more NPM packages or native tools.
- Auto-generated table of contents for every chapter.
- Auto-generated anchor links for every sub-section.
- Overall more appealing presentation.
- All docs will use the ActiveMQ favicon.
- No more manual line-wrapping! AsciiDoc recommends one sentence per
line and paragraphs are separated by a blank line.
- AsciiDoctor plugins for IDEA are quite good.
- Resulting HTML is less than *half* of the previous size.
All previous links/bookmarks should continue to work.
[1] https://github.com/asciidoctor/kramdown-asciidoc
ARTEMIS-4375 Implement artemis shell using JLine3 integrated with auto-completion from picocli
This commit involves two JIRAs. One is adding PicoCLI and the next is Using JLine3 and implement a shell.
I have tried to keep these commits separate but these changes became interdependent hence the two JIRAs are squashed in this commit.
The "Airline" library we're currently using is deprecated according the
GitHub project - https://github.com/airlift/airline. It recommends using
either Airline 2 or Picocli. The former offers the simplest migration
path as it's almost completely compatible with the current code. This
commit implements that migration.
Improve the CORE client failover connecting to other live servers when all
reconnect attempts fails, i.e. in a cluster composed of 2 live servers,
when the server to which the CORE client is connected goes down the CORE
client should reconnect its sessions to the other liver broker.
When skipping the authentication cache details for the original
exception are not logged.
This commit ensures these details are logged and adopts the
ExceptionUtils class from Apache Commons Lang in lieu of the previous
custom implementation.
This commit fixes the following things:
- Moves connection audit logging to the resource audit logger instead
of using a dedicated logger as that would adversely impact upgrading
users, and arguably didn't make sense in the first place.
- Mitigates an potential NPE w.r.t. connection ID.
- Updates the "dummy" management connection to return a valid
connection ID.
There are certain use-cases where addresses will be auto-created and
never have a direct binding created on them. Because of this they will
never be auto-deleted. If a large number of these addresses build up
they will consume a problematic amount of heap space.
One specific example of this use-case is an MQTT subscriber with a
wild-card subscription and a large number of MQTT producers sending one
or two messages a large number of different MQTT topics covered by the
wild-card. Since no bindings are ever created on any of these individual
addresses (e.g. from a subscription queue) they will never be
auto-deleted, but they will eventually consume a large amount of heap.
The only way to deal with these addresses is to manually delete them.
There are also situations where queues may be created and never have
any messages sent to them or never have a consumer connect. These
queues will never be auto-deleted so they must be deleted manually.
This commit adds the ability to configure the broker to skip the usage
check so that these kinds of addresses and queues can be deleted
automatically.
I am adding three attributes to Address-settings:
* page-limit-bytes: Number of bytes. We will convert this metric into max number of pages internally by dividing max-bytes / page-size. It will allow a max based on an estimate.
* page-limit-messages: Number of messages
* page-full-message-policy: fail or drop
We will now allow paging, until these max values and then fail or drop messages.
Once these values are retracted, the address will remain full until a period where cleanup is kicked in by paging. So these values may have a certain delay on being applied, but they should always be cleared once cleanup happened.
I am adding an option sync=true or false on mirror. if sync, any client blocking operation will wait a roundtrip to the mirror
acting like a sync replica.
Some LDAP servers (e.g. OpenLDAP) do not support the "persistent search"
feature and therefore the existing "listener" feature does not actually
fetch updates. This commit implements a "pull" feature controlled by a
configurable interval equivalent to what is implemented in the cached
LDAP authorization module from ActiveMQ "Classic."
Adds a new module 'artemis-junit-5' which adds JUnit 5 Extensions for
unit testing. For backwards compability, 'artemis-junit' still uses
JUnit 4. Common stuff has been moved to 'artemis-junit-commons'. Work is
based on the initial PR
https://github.com/apache/activemq-artemis/pull/3436 by @luisalves00
Sometimes users want to perform custom client ID validation, and in the
case of an invalid client ID the proper reason code should be returned
in the CONNACK packet.
Currently the broker detects the presence of the web console by looking
for the name of a file (i.e. console.war). This is fragile because if
the file is renamed for any reason then the broker won't print the
status of the web console when it starts.
This commit improves web console detection by inspecting the
<display-name> tag in the war file's WEB-INF/web.xml. By default it
looks for "hawtio", but this can be customized using the system property
"org.apache.activemq.artemis.webConsoleDisplayName".
MQTT 3.1 and 3.1.1 clients using a clean session should have a
*non-durable* subscription queue. If the broker restarts the queue
should be removed. This is due to [MQTT-3.1.2-6] which states that the
session (and any state) must last only as long as the network
connection.
It would be useful to be able to cycle the embedded web server if, for
example, one needed to renew the SSL certificates. To support
functionality I made a handful of changes, e.g.:
- Refactoring WebServerComponent so that all the necessary
configuration would happen in the start() method.
- Refactoring WebServerComponentTest to re-use code.
Allow replication only certain addresses with mirror controller.
The configuration is similar to cluster address configuration.
Co-authored-by: Robbie Gemmell <robbie@apache.org>
The commit includes the following changes:
- Don't drop the connection on subscribe or publish authorization
failures for 3.1 clients.
- Don't drop the connection on subscribe authorization failures for
3.1.1 clients.
- Add configuration parameter to control behavior on publish
authorization failures for 3.1.1 clients (either disconnect or not).
Mainly refactoring the address docs. This commit has the following
changes:
- Remove examples for discouraged use-cases (e.g. using anycast and
multicast on the same address).
- Reword to use configuration terms wherever possible. For example,
instead of saying "point-to-point" (which is not a configuration term)
say "anycast". References to things like "point-to-point" and
"publish-subscribe" are still there since users are familiar with these
terms. They're just used much less often.
- Remove duplicate explanation of exclusive queues.
- Remove duplicate explanation of auto-create and auto-delete elements.
- Re-create graphics and include the master SVGs for potential updates
later.
- Give non-destructive queues its own chapter.
- Add details about specifying routing type using a message property.
- Update the styling on the user manual's cover page to look better.
- Lots of re-wording for clarity's sake.
- Re-order sub-sections for clarity's sake.
- Break up the address model and the settings documentation. The
settings documentation is large and deserves its own chapter. The
original anchor link is still available with a link to the new chapter.
In general the address-specific documentation should be much more clear,
concise, and consistent now.
JGroups 3.x hasn't been updated in some time now. The last release was
in April 2020 almost 2 years ago. Lots of protocols have been updated
and added and users are wanting to use them. There is also increasing
concern about using older components triggered mainly by other
recently-discovered high-profile vulnerabilities in the wider Open
Source Java community.
This commit bumps JGroups up to the latest release - 5.2.0.Final.
However, there is a cost associated with upgrading.
The old-style properties configuration is no longer supported. I think
it's unlikely that end-users are leveraging this because it is not
exposed via broker.xml. The JGroups XML configuration has been around
for a long time, is widely adopted, and is still supported. I expect
most (if not all) users are using this. However, a handful of tests
needed to be updated and/or removed to deal with this absence.
Some protocols and/or protocol properties are no longer supported. This
means that users may have to change their JGroups stack configurations
when they upgrade. For example, our own clustered-jgroups example had to
be updated or it wouldn't run properly.
MQTT 5 is an OASIS standard which debuted in March 2019. It boasts
numerous improvments over its predecessor (i.e. MQTT 3.1.1) which will
benefit users. These improvements are summarized in the specification
at:
https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901293
The specification describes all the behavior necessary for a client or
server to conform. The spec is highlighted with special "normative"
conformance statements which distill the descriptions into concise
terms. The specification provides a helpful summary of all these
statements. See:
https://docs.oasis-open.org/mqtt/mqtt/v5.0/os/mqtt-v5.0-os.html#_Toc3901292
This commit implements all of the mandatory elements from the
specification and provides tests which are identified using the
corresponding normative conformance statement. All normative
conformance statements either have an explicit test or are noted in
comments with an explanation of why an explicit test doesn't exist. See
org.apache.activemq.artemis.tests.integration.mqtt5 for all those
details.
This commit also includes documentation about how to configure
everything related to the new MQTT 5 features.
Clebert Suconic
Justin Bertram
Mike Artz
Robbie Gemmell
iamgd67
Domenico Francesco Bruscino
Gary Tully
a181321
Timothy Bish
Paul Wright
ALX
Ryan Highley
iliya
Vilius Šumskas
waldi5001
eidottermihi
ruromero
Jan Šmucr
Gerd Klingler
Marlon Müller
Stephen E. Baker
Tiago Bueno
franz1981
Matteo Baccan
Luis De Bello
Gergő Rubint
Paul Wright