The filter and the view use different convection for field names, ie the
connection view uses the `sessionID` field name while the connection filter
uses the `SESSION_ID` field name. This commit replace the field names used
by the filter with the field names used by the view preserving the backward
compatibility.
The provider of an SSL key/trust store is different from that store's
type. However, the broker currently doesn't differentiate these and uses
the provider for both. Changing this *may* potentially break existing
users who are setting the provider, but I don't see any way to avoid
that. This is a bug that needs to be fixed in order to support use-cases
like PKCS#11.
Change summary:
- Added documentation.
- Consolidated several 2-way SSL tests classes into a single
parameterized test class. All these classes were essentially the same
except for a few key test parameters. Consolidating them avoided
having to update the same code in multiple places.
- Expanded tests to include different providers & types.
- Regenerated all SSL artifacts to allow tests to pass with new
constraints.
- Improved logging for when SSL handler initialization fails.
Previously, when a session was reattached, all the close/failure listeners
were removed from the old connection and set onto the new connection.
This only worked when at most 1 session of the old connection was
transferred: When the second session was transferred, the old
connection already didn't contain any close/failure listeners anymore,
and therefore the list of close/failure listeners was overwritten by
an empty list for the new connection.
Now, when a session is being transferred, it only transfers the
close/failure listeners that belong to it, which are the session itself
+ the TempQueueCleanerUppers.
Modified a test to check whether the sessions are failure listeners of
the new connection after reattachment.
During session reattachment, also set the new connection on the
"session" member of the ServerSessionPacketHandler. Until now,
the connected ServerSessionImpl instance still referenced the old
connection although it had already been transferred on the new connection.
Change summary:
- Remove the existing Xalan-based XPath evaluator since Xalan appears
to be no longer maintained.
- Implement a JAXP XPath evaluator (from the ActiveMQ 5.x code-base).
- Pull in the changes from https://issues.apache.org/jira/browse/AMQ-5333
to enable configurable XML parser features.
- Add a method to the base Message interface to make it easier to get
the message body as a string. This relieves the filter from having
to deal with message implementation details.
- Update the Qpid JMS client to get the jms.validateSelector parameter.
WARNING: the eclipse static analyser is pretty limited and it cannot cope with cases like: correlated variables, exception paths, .... 70% of eclipse warnings on artemis codebase is a false alarm.
Anyway some variable correlations can be eliminated and code becomes more readable for humans too.
For "never-throws", the assumption is made explicit. If you disagree with the reverse-engineered assumption then it is likely an indication of a true potential NPE.
Last but not least, copy&paste is a common source of bugs. I suspect eclipse indirectly detected one such case.
Hope it helps
The fallback consumer authorization implemented in ARTEMIS-592 needs to
check for an *exact* security-settings match otherwise in certain
configurations a more general and more permissive setting might
be used instead of the intended more specific and more restrictive
setting.
The merge method in AddressSettings should *not* use any getters. It
should reference the relevant variables directly. Using any getters will
return default values in the underlying value is null. This can cause
problems for hierarchical settings.
Also fixed a few potential NPEs exposed by the test-case.
The existing deactivation callback happens *after* several important
services are shutdown (e.g. the remoting service which allows client
connectivity). This commit adds a new callback which is invoked *before*
any services are stopped. This is useful for embedded use-cases where
applications want to stop gracefully before any part of the broker is
stopped.
A default, empty method implementation is provided so that existing
callback implementations don't need to change.
This is to avoid shutting down the server on a critical failure in case the message is a few bytes shy
from beyond the max buffer size.
This will prevent the issue.
When deleting a durable scheduled message via the management API the
message would be removed from memory but it wouldn't be removed from
storage so when the broker restarted the message would reappear.
This commit fixes that by acking the message during the delete
operation.
Using a ThreadLocal for the audit user information works in most cases,
but it can fail when dispatching messages to consumers because threads
are taken out of a pool to do the dispatching and those threads may not
be associated with the proper credentials. This commit fixes that
problem with the following changes:
- Passes the Subject explicitly when logging audit info during dispatch
- Relocates security audit logging from the SecurityManager
implementation(s) to the SecurityStore implementation
- Associates the Subject with the connection properly with the new
security caching
- Adding a paragraph about addressing and distinct queue names
- Renaming match on peers, senders and receivers as "address-match"
- Changing qpid dispatch test to use a single listener
- Fixing reconnect attemps message
This commit is fixing:
- a missing commit that can make leak a connection
- restricting default specific commons-dbcp2 to the default data source
- setting poolPreparedStatements true by default
- configured embedded Derby to be in-memory to speedup tests
It add additional required fixes:
- Fixed uncommitted deleted tx records
- Fixed JDBC authorization on test
- Using property-based version for commons-dbcp2
- stopping thread pool after activation to allow JDBC lease locks to release the lock
- centralize JDBC network timeout configuration and save repeating it
- adding dbcp2 as the default pooled DataSource to be used
Replaces direct jdbc connections with dbcp2 datasource. Adds
configuration options to use alternative datasources and to alter the
parameters. While adding slight overhead, this vastly improves the
management and pooling capabilities with db connections.
This reverts commit dbb3a90fe6.
The org.apache.activemq.artemis.core.server.Queue#getRate method is for
slow-consumer detection and is designed for internal use only.
Furthermore, it's too opaque to be trusted by a remote user as it only
returns the number of message added to the queue since *the last time
it was called*. The problem here is that the user calling it doesn't
know when it was invoked last. Therefore, they could be getting the
rate of messages added for the last 5 minutes or the last 5
milliseconds. This can lead to inconsistent and misleading results.
There are three main ways for users to track rates of message
production and consumption:
1. Use a metrics plugin. This is the most feature-rich and flexible
way to track broker metrics, although it requires tools (e.g.
Prometheus) to store the metrics and display them (e.g. Grafana).
2. Invoke the getMessageCount() and getMessagesAdded() management
methods and store the returned values along with the time they were
retrieved. A time-series database is a great tool for this job. This is
exactly what tools like Prometheus do. That data can then be used to
create informative graphs, etc. using tools like Grafana. Of course, one
can skip all the tools and just do some simple math to calculate rates
based on the last time the counts were retrieved.
3. Use the broker's message counters. Message counters are the broker's
simple way of providing historical information about the queue. They
provide similar results to the previous solutions, but with less
flexibility since they only track data while the broker is up and
there's not really any good options for graphing.
When performing concurrent user admin actions (e.g. resetUser, addUser,
removeUser on ActiveMQServerControl) when using the
PropertiesLoginModule with reload=true the underlying user and role
properties files can get corrupted.
This commit fixes the issue via the following changes:
- Add synchronization to the management commands
- Add concurrency controls to underlying file access
- Change CLI user commands to use remote methods instead of modifying
the files directly. This avoids potential concurrent changes. This
change forced me to modify the names of some of the commands'
parameters to disambiguate them from connection-related parameters.
In a cluster scenario where non durable subscribers fail over to
backup while another live node forwarding messages to it,
there is a chance that the the live node keeps the old remote
binding for the subs and messages go to those
old remote bindings will result in "binding not found".
The default JAAS security manager doesn't need the address/FQQN for
authorization, but I'm putting it back into the interface because there
are other use cases which *do* need it.
Both authentication and authorization will hit the underlying security
repository (e.g. files, LDAP, etc.). For example, creating a JMS
connection and a consumer will result in 2 hits with the *same*
authentication request. This can cause unwanted (and unnecessary)
resource utilization, especially in the case of networked configuration
like LDAP.
There is already a rudimentary cache for authorization, but it is
cleared *totally* every 10 seconds by default (controlled via the
security-invalidation-interval setting), and it must be populated
initially which still results in duplicate auth requests.
This commit optimizes authentication and authorization via the following
changes:
- Replace our home-grown cache with Google Guava's cache. This provides
simple caching with both time-based and size-based LRU eviction. See more
at https://github.com/google/guava/wiki/CachesExplained. I also thought
about using Caffeine, but we already have a dependency on Guava and the
cache implementions look to be negligibly different for this use-case.
- Add caching for authentication. Both successful and unsuccessful
authentication attempts will be cached to spare the underlying security
repository as much as possible. Authenticated Subjects will be cached
and re-used whenever possible.
- Authorization will used Subjects cached during authentication. If the
required Subject is not in the cache it will be fetched from the
underlying security repo.
- Caching can be disabled by setting the security-invalidation-interval
to 0.
- Cache sizes are configurable.
- Management operations exist to inspect cache sizes at runtime.
This is allowing journal appends to happen in burst
during replication, by batching replication response
into the network at the end of the append burst.
I couldn't reproduce this with a test, but static code analysis led me
to this solution which is similar to the fix done for ARTEMIS-2592 via
e397a17796.
Due to a lack of concurrency protections it's possible to create a
consumer on a closed session. I've not been able to reproduce this with
a test, but I've seen it in the wild. Static code analysis points to a
need for better concurrency controls around closing the session and
creating consumers.
This is almost a NO-JIRA, but since I'm fixing ManagementServiceImplTest.testGetResources broken after this
I'm commiting this change associated with ARTEMIS-2828
Since getDiskStoreUsage on the ActiveMQServerControl is converting a
double to a long the value is always 0 in the management API. It should
return a double instead.
Adding this metric required moving the meter registration code from the
AddressInfo class to the ManagementService in order to get clean access
to both the AddressInfo and AddressControl classes.
The calculation used by
ActiveMQServerControlImpl.getDiskStoreUsagePercentage() is incorrect. It
uses disk space info with global-max-size which is for address memory.
Also, the existing getDiskStoreUsage() method *already* returns a
percentage of total disk store usage so this method seems redundant.
Now it is possible to reset queue parameters to their defaults by removing them
from broker.xml and redeploying the configuration.
Originally this PR covered the "filter" parameter only.
ORIG message propertes like _AMQ_ORIG_ADDRESS are added to messages
during various broker operations (e.g. diverting a message, expiring a
message, etc.). However, if multiple operations try to set these
properties on the same message (e.g. administratively moving a message
which eventually gets sent to a dead-letter address) then important
details can be lost. This is particularly problematic when using
auto-created dead-letter or expiry resources which use filters based on
_AMQ_ORIG_ADDRESS and can lead to message loss.
This commit simply over-writes the existing ORIG properties rather than
preserving them so that the most recent information is available.
- when sending messages to DLQ or Expiry we now use x-opt legal names
- we now support filtering thorugh annotations if using m. as a prefix.
- enabling hyphenated_props: to allow m. as a prefix
DivertBindings are now properly cleaned up when a queue binding is
removed that matches the divert. The correct key is now used to remove
the queue address from the set and the correct address is now used to
remove the remote consumer.
This one should improve eventual failures on ClusterConnectionControlTest
to validate this, run ClusterConnectionControlTest::testNotifications in a loop
you will see eventually the test taking longer to shutdown the executor as the call could be blocked.
This won't be an issue on a real server (Production system)
however, on the testsuite or while embedded this could cause issues,
if a same instance is stopped then started.
This is the reason why BackupAuthenticationTest was intermittently failing.
I also adapted the test since I would need to stop the server and reactivate it in order to change the configuration.
The previous test wasn't acting like a real server.
Currently when the broker hits the common 'Address already in use' issue
when starting the process terminates with a vague exception. The user is
left to guess which acceptor actually failed. If the broker has lots of
acceptors it is a tedious process to identify the problematic
configuration. This commit adds details to the exception message about
which acceptor failed and which host:port it was attempting to bind to.
This commit does the following:
- Deprecates existing overloaded createQueue, createSharedQueue,
createTemporaryQueue, & updateQueue methods for ClientSession,
ServerSession, ActiveMQServer, & ActiveMQServerControl where
applicable.
- Deprecates QueueAttributes, QueueConfig, & CoreQueueConfiguration.
- Deprecates existing overloaded constructors for QueueImpl.
- Implements QueueConfiguration with JavaDoc to be the single,
centralized configuration object for both client-side and broker-side
queue creation including methods to convert to & from JSON for use in
the management API.
- Implements new createQueue, createSharedQueue & updateQueue methods
with JavaDoc for ClientSession, ServerSession, ActiveMQServer, &
ActiveMQServerControl as well as a new constructor for QueueImpl all
using the new QueueConfiguration object.
- Changes all internal broker code to use the new methods.
Due to the changes in 6b5fff40cb the
config parameter message-expiry-thread-priority is no longer needed. The
code now uses a ScheduledExecutorService and a thread pool rather than
dedicating a thread 100% to the expiry scanner. The pool's size can be
controlled via scheduled-thread-pool-max-size.
This appears to have been added to the code-base by mistake over 10
years ago. It seems related to debugging and I can't see anywhere where
it is actually used so I'm removing it.
Using a property on AMQPLargeMessage instead of a ThreadLocal
This was causing issues on the journal as the message may transverse different threads on the journal.
There is no guarantee that the encodeSize size is the same in AMQP right after read.
As the protocol may add additional bytes right after decoded such as header, extra properties.. etc.
Christoph Läubrich
franz1981
Domenico Francesco Bruscino
AntonRoskvist
gtully
Andy Taylor
Justin Bertram
Clebert Suconic
Sebastian Thomschke
Markus Meierhofer
Christopher L. Shannon (cshannon)
Petr Kužel
岳豹
Marcos Singermann
Erwin Dondorp
Tomas Hofman
Ryan Yeats
Tadayoshi Sato
Luis De Bello
Emmanuel Hugonnet
Luis Alves
Mikko Uoti
Bernd Gutjahr
Howard Gao
Michael Pearce
brusdev
Jan Šmucr
qihongxu
Bas Elzinga
Keith Wall
Havret
Robbie Gemmell
Emmanuel Hugonnet
Atri Sharma
Benjamin Graf