* In ServletChannel, in COMPLETE state, there was a check comparing the number of bytes written with that declared by Content-Length.
Unfortunately the check was wrong in case of gzip, because it was comparing the application length with the gzipped length, resulting in a late sendErrorOrAbort() that was an abort() because the response was already fully sent.
Furthermore, if sendErrorOrAbort() was actually an abort(), there was an unnecessary attempt to complete the output.
* In GzipHandlerResponseAndCallback there were 2 last writes: one when the application calls output.close(), and one when the callback is succeeded.
Furthermore, when no content needs to be written, it was still compressed, causing an IOException in ChannelResponse (again when double-checking the bytes written, since Content-Length was 0, but the gzip header and trailer were written).
* Updates semantic of when to add the "Vary" header.
Now it is always only added by GzipHandler, when it would be possible to generate a response that might be compressed (even if it will not).
* Added *.bz2 mime-type.
Updated GzipHandler with the correct mime-type for *.bz2: application/x-bzip2.
* Improved checks for the number of bytes written against declared content-length.
This is necessary because welcome files are written by DefaultServlet (most of the times) bypassing the Servlet classes, using ServletContextResponse directly.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Changed `JettyWebSocketFrameHandlerFactory` to use an application MethodHandle.Lookup (rather than a server one) when creating listener metadata.
This fixes the JPMS problem so that now JPMS applications do not need any additional configuration to invoke endpoints.
The (acceptable) downside is that anonymous inner classes (that are not public) cannot be used as listener endpoints.
This change affects core and EE10 WebSocket; EE9 and EE8 WebSocket have not been fixed (so they allow anonymous inner classes but are broken in JPMS).
Renamed "connectLatch" fields to "openLatch" in test classes.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Olivier Lamy <olamy@apache.org>
+ Introduce new events on `ComplianceViolation.Listener`
+ Introduce new `ComplianceViolation.Listener.initialize()` to allow for a new Listener at the appropriate time (to support per-request listeners)
+ Introduce new `ComplianceViolation.CapturingListener`
+ Introduce new `HttpConfiguration.(add/remove/get)ComplianceViolationListener()` methods.
+ Deprecate/Remove handling of `recordComplianceViolations` in `HttpConnection` and `HttpConnectionFactory` classes.
+ Produce warnings if use of `ComplianceViolation.Listener` as beans is still present.
+ Add `ComplianceViolation.Listener` support to `UriCompliance` locations.
+ Add `ComplianceViolation.Listener` support to `MultiPartCompliance` locations.
+ Add `ComplianceViolation.Listener` support to `CookieCompliance` locations.
+ Add `ComplianceViolation.Listener` support to `HttpCompliance` locations in HTTP/2 and HTTP/3
---------
Co-authored-by: gregw <gregw@webtide.com>
Introduced AutoLock.tryLock() to use it in the toString() implementations that lock in order to retrieve a consistent state to produce the string.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Using @foo.version@ technique and proper
lib lists to allow arbitrary version
updates from properties
* Fix spotless
* Remove duplicate entries
* Only replace System property "jetty.version" with manifest-ver if blank/null
* Fixing bad module definitions
Now upon the second idle timeout, the connection is forcibly closed.
Fixed also similar problem in HTTP/3.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed parsing of form parameters in FormFields.parse().
Added more tests for valid edge cases, and invalid cases.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Reorganized ArrayByteBufferPool.evict() code to avoid NPE.
* Fixed acquire logic to take into account that a reserved entry may fail to be enabled because it may be concurrently removed.
* Small change to ConcurrentPool: no need to null check the final ConcurrentEntry.holder field.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #11009 - add test for bad Jakarta endpoint
* Issue #11009 - ensure endpoint deployable before adding ServerEndpointConfig
* add same test and fix for ee9
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Most often failures come from the read side, so failure listeners are now serialized in the _readInvoker.
This avoids that a failure while parsing a request (e.g. an early EOF) results in concurrent executions of the invokeOnContentAvailable task and the invokeOnFailureListeners task.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Do not use HttpStream.Wrapper in SizeLimitHandler
* Updated the SizeLimitHandlerServletTest
* Udpated documentation and javadocs.
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Made ServletChannel error handling more robust.
A failure in error handling is now remembered so that the Handler callback can be failed later.
* Avoid failing the Handler callback from ServletChannel.abort(), as it is too early: should be failed when processing the TERMINATED state, similarly to when it is succeeded.
* Removed dead code from HttpConnection.SendCallback.reset(), since response is always non-null.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Introduced IteratingCallback.abort(Throwable) to serialize calls to onCompleteFailure().
* HttpSender now uses IC.abort() to serialize the abort of the request.
* Fixed HTTP/2 and HTTP/3 to dispatch the sending of the request to another thread, to free the reader thread that read the server preface.
* HTTP/1.1 does not need this, because just created connections do not need to read.
* Improved handling of request abort, as it can be from two sides: external and internal.
* Calling abort() instead of failed() for WebSocket flushers.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Call ServletChannelState.asyncFailure from error listener. Fix#10933
* Separate invokers for read side and write side
* document async error issues
* updates from review
* updates from review
We special-cased this test on macOS to support NFD form normalization.
That may no longer be necessary, and therefore the test currently fails.
Run the default expectations, and, in case of failure on macOS, test
against the old workaround.
Now trying to use $JETTY_BASE/work, otherwise throwing IllegalArgumentException with a descriptive message.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Removed duplicate handling from servlet. Now only handled in core in HttpChannelState and protected it from concurrent writes.
Added additional tests.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Separated read failures from write failures.
* In this way it is possible to read even if the write side is failed and write even if the read side is failed.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
The commented out (and optional) configurations
* `jetty.session.jdbc.schema.schemaName`
* `jetty.session.jdbc.schema.catalogName`
Are not represented as an empty key=value pair
so it wasn't being found by tooling.
* #10926 - skip existence check if only one Normalizer
* fix for AttributeNormalizer
---------
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Co-authored-by: gregw <gregw@webtide.com>
Setting MultiPart.Parser.crContent=false when the boundary is matched.
This is necessary when a read chunk ends with \r\n, and then matches the boundary on the next read chunk, and avoids to emit the CR as part content.
Fixed case of a chunk ending with \r.
Fixed case where a CR in a previous chunk is followed by a boundary at index 0 in the next chunk.
The CR should not be emitted as content, and crContent reset to false.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Fixes#10912 - Document Request listeners
* Documented Request listeners and updated javadocs.
* Removed code in HttpChannelState.onIdleTimeout() that was automatically complete the Handler callback.
* Invoking failure listeners only once (although HttpChannelState.onFailure() may be called multiple times).
* Made sure that in ChannelCallback.succeeded() the last stream send uses the ChannelResponse as Callback, like it is done in Response.write().
* Moved Request listeners tests from various test classes into RequestListenersTest.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* #10386 Align core, ee9 and ee10 AbstractTest:
- use the same temporary folder technique
- configure the SSL contexts the same way
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Fixes#10234
* Introduced transient failures in reads where a failure chunk has last=false.
* Transient failure now do not fail the handler callback.
* Improve eeN ContentProducer to more carefully assert transient and terminal errors + enable HttpInputIntegrationTest
* Do not add connection: close to the response when the error is transient
* Rework ChunksContentSource to support null chunks
* Added tests to verify the new transient failure cases
* Review all code that handles failure, and handling correctly transient failure, either by making them fatal, and/or by failing Content.Source.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Signed-off-by: Olivier Lamy <olamy@apache.org>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Co-authored-by: Chad Wilson <chadw@thoughtworks.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Make timed waits more reliable
* Remove all timed assertions to make this test non-flaky
* Synchronize the threads on their state to make sure blocking happens + add timeout annotation
* Use assertThrows where exceptions are expected
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* Allow UpgradeListener to see Response in case of non-successful upgrade
* Using a boolean to track whether request was upgraded
* Improved exception handling in HttpUpgraderOverHTTP.
* Avoid using log warnings in CoreClientUpgradeRequest.
* Delayed setting of this.upgraded after the last throw statement.
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* WebSocketUpgradeHandler should not require a ContextHandler
* Exposing the ServerWebSocketContainer as attribute also when creating the WebSocketUpgradeHandler with only the Server, to allow dynamic configuration of WebSocket endpoints.
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Fixes#10879 - Improve redirect handling with reproducible content
Now both the redirect and the authentication ProtocolHandlers will abort the request on response success.
If the request is already completed, the abort attempt will be a no-op, proceeding as usual.
Otherwise, the request upload will be aborted and a new request sent with the reproducible content.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Olivier Lamy <olamy@apache.org>
* Don't track mounts for newResource() that doesn't exist.
* Cannot dereference if not tracking both resources that use the JAR.
* Only mount the root. Check if root already mounted in mountIfNeeded
---------
Co-authored-by: gregw <gregw@webtide.com>
* Various cleanups of Handler.insertHandler
* Added missing call to relinkHandlers() in setHandler() after calling super.
* Moved call to relinkHandlers() in insertHandler(), as the various setSession|Security|ServletHandler() already call relinkHandlers().
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
---------
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
The local/remote SocketAddress is cached within the ConnectionMetaData or Connection instance, so that any changes are not visible during the request lifetime.
Ensure that all server Connection types respect HttpConfiguration#getLocalAddress and that it is not implemented only in servlet layer
Avoid DNS resolution.
---------
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Fix bug with #10801 in that completion listeners were not well sequenced with stream wrappers.
---------
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* The Pool is now a list of Holder instances, each with a WeakReference and a strong reference to an Entry.
* Removed thread-local cache from Pool.
* Avoid using deprecated ConcurrentPool constructors.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Improve handling of SSL metadata in ConnectionMetaData, proxied connections and forwarded requests
+ Added an Attributes.Synthetic mechanism to better (and mor commonly) handle additional attributes, whilst meeting the new TCK requirement that they can be deleted or replaced.
+ Made SslSessionData a reusable type provided by EndPoint
+ Added the EndPoint.Securable subtype that can provide a SslSessionData instance for secure EndPoints
+ Updated ForwardedRequestCustomizer, SecureRequestCustomizer to understand Securable and SslSessionData
+ Javadocs improvements.
+ Removed unnecessary attribute constants in SecureRequestCustomizer.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
---------
Signed-off-by: Olivier Lamy <olamy@apache.org>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Recycle ServletChannel in ServletContextHandler rather than in ServletHandler, so that completion events on servlet API request/response can be handled.
* Fix overlays with CombinedResource
* Improvements to CombinedResource for AnnotationParser
* Added `contains` and `getPathTo` methods to the Resource API, so they can be used by the AnnotationParser
* Fixed numerous bugs in CombinedResource list and getAllResources
* handles cross file system copy as well.
* Reduced fallback to URI string manipulation.
---------
Co-authored-by: gregw <gregw@webtide.com>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* Send attributes as Map<String,String>
* JMX-annotated ErrorHandler, fixed usage of showMessageInTitle and removed showServlet.
* JMX-annotated Request.Handler.
* Added JMX test module to test improvements to JMX.
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Updated base MBean class for o.e.j.server.Handler.
Also updated and modernized existing explicit MBean classes.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* #10726 fix NPE in ResponseListeners when demultiplexing is used and the original source spuriously calls demand()
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* #10703 fix race condition in clear() and make sure non-reusable released entries are removed from the pool
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* Avoid a copy in MutableHttpFields.asImmutable if the mutable is never mutated again.
* reduce instance creations needed for iterations
---------
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
* Bumped the rate control rate from 50 events/s to 128.
* Added rate control for all CONTINUATION frames.
* Added rate control for invalid PUSH_PROMISE frames.
* Added rate control for RST_STREAM frames.
* Added rate control for all SETTINGS frames.
* Fixed growth of header block accumulation buffer.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Changed default implementation of Session.Listener.onNewStream() and Stream.Listener.onDataAvailable() to auto-discard DATA frames.
For normal cases, these methods are overridden and the application is in full control.
For test cases, these methods may not be overridden and the default implementation conveniently avoids buffer leaks.
Fixed flaky test RawHTTP2ProxyTest.testRawHTTP2Proxy() due to the bad assumption that the first DATA frame ends the stream (so an aggregator is needed), and a copy/paste error in ServerToProxyToClient while processing DATA frames.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed test case that was racy.
When the DATA frames arrived at the server before the call to consumeAvailable(), they were read and the client flow control window re-opened.
If it happened that the DATA frames arrived at the server after the call to consumeAvailable(), the client flow control window was not re-opened, making the test flaky.
Fixed by avoiding the race in the test.
Added over-release buffer tracking, add leak tracking to H2 tests, fix client leaks in tests.
Also reviewed the places that required re-opening of the flow control window in case the DATA frames are not read.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
* Changes the H2 semantic of `Stream.readData()` so that it is `readData()` that enlarges the flow control window, and not anymore the release of the `Stream.Data`.
This allows applications to buffer in memory by retaining the `Stream.Data` instances more than the H2 flow control window.
* Updated `FlowControlStrategyTest` after `Stream.readData()` semantic changes.
* Updated `DataDemandTest` after `Stream.readData()` semantic changes.
* Updated documentation.
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: gregw <gregw@webtide.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Web functions are currently supported with servlets. These changes add/move utility classes to core to better support direct usage of core APIs
* increase usage of Charset in request
* Added flush mechanism to BufferedContentSink
* Changed default buffer size from 2 GB to 16 KB.
* Make max buffer size configurable.
* Introduce `BufferedContentSink` with all the buffering logic, doing only one buffer copy instead of two, starting with a small buffer and growing it if needed.
* Refactor `BufferedResponseHandler` to delegate all buffering work to `BufferedContentSink`
* Introduced `ByteBufferAggregator` to aggregate ByteBuffers into a single ByteBuffer.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: gregw <gregw@webtide.com>
Reviewed the implementation of RewriteHandler, that was broken for those rules that were overriding `Rule.Handler.handle()`.
The problem was that the handling was not forwarded along the chain of rules, so only the last one was applied.
Now the wrapping at the constructor produces RH3(RH2(RH1(Request))), but the handling is performed from the innermost towards the outermost.
In this way, the order of rules is respected, both in the wrapping at rule application, and in the `Rule.Handler` handling.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #10466 review session documentation.
Also fix session config context init param names and add missing code to
configure SessionHandler via context init params, and added test for
config.
Introduce `Utf8CharacterCodingException` and `Utf8IllegalArgumentException` as a substitutes for the removed `Utf8Appendable.NotUtf8Exception`.
* Updates from review
* Issue #10328 - Review ResourceFactory.newSystemResource
+ Create a new ResourceFactory.newClassLoaderResource(String, boolean)
+ Make .newSystemResource(String) use it
+ Make .newClassPathResource(String) use it
+ Deprecate .newSystemResource(String)
+ Deprecate .newClassPathResource(String)
+ Adjust own codebase to not use deprecated methods
* Using request.getLength() instead of looking up the Content-Length header.
* Jetty Handler vs Servlet pros.
* Using CompletableFuture (not Promise) in examples.
* Removed unused cruft from tests.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Address #10513 ContentSourceInputStream close by making it do a single read looking for EOF
If any content is skipped, then it is an abnormal close.
use Chunk.next in read
* Thread T1 may initialize HttpTester.Message that extends MutableHttpFields, so grabs the lock for the initialization of class MutableHttpFields.
* Thread T2 may initialize HttpFields, so grabs the lock for HttpFields and initializes field EMPTY, which calls new MutableHttpFields.
* To initialize MutableHttpFields, T1 must initialize HttpFields, but sees that its lock is taken and waits.
* To initialize HttpFields, T2 must create an instance and therefore initialize MutableHttpFields, but sees that its lock is taken and waits.
* Deadlock.
The solution is to use another class, EmptyHttpFields, to initialize HttpFields.EMPTY, so that there is no deadlock.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Improved parsing of JSESSIONID cookies and jsessionid parameters.
Better handling of invalid and duplicate session IDs
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Co-authored-by: Jan Bartel <janb@webtide.com>
* Issue #10500 - preserve request header quoting when accessed through JettyHttpExchangeDelegate
* improve test cases with quoted-pair feature in RFC
* add skip of value lists on specific headers known to not have value lists
* Remove URL usage
* Changes from review
* Refactored domain checks into overridable method.
* Added support for IPv6, and clarified domain checks.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Added SetCookieParser interface and RFC6265SetCookieParser implementation to properly parse Set-Cookie values.
* Removed hacky implementation in HttpClient.
* Removed unused methods in HttpCookieUtils.
* Using SetCookieParser for the implementation of newPushBuilder in ee9,ee10.
* Reworked HttpCookieStore.Default implementation.
* Implemented properly cookie path resolution.
* Using URI.getRawPath() to resolve cookie paths.
* Removed secure vs. non-secure scheme distinction when storing cookies.
* Refactored common code in HttpCookieStore.Default to avoid duplications.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Introduced QoSHandler.
* Updated documentation, and removed old documentation.
* Now using dynamic max priority.
The current implementation retains priorities and queue as they are created, but hopefully the priority range is constrained.
* Better algorithm for resuming: rather than releasing a permit and try to acquire it again, keep the permit and resume a request.
This also removes the issue that a request could have been suspended again, therefore remaining suspended much more than maxSuspend.
* Deprecated QoSFilters in ee10 and ee9.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced CompletableResponseListener, a replacement for FutureResponseListener that uses better APIs based on CompletableFuture.
Deprecated FutureResponseListener and replaced its usages.
Updated documentation.
Added tests for zipped request content.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now the log statement is only emitted when there is no matching setter method, but a mismatched one exists.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Implemented a simpler default environment algorithm where an application that does not specify an environment is always attempted in the default.
Updated documentation.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Only fail request callback if a failure has not been otherwise notified.
Slight optimisation for failing idle timeouts by avoiding double lock.
Always create a failure if failing the callback.
Use the core response HttpFields directly as the ee9 response headers to avoid copy and retain persistent field behaviour.
Fix#10416 EE9 Response headers
Added EE9 test to show that Persistent fields can be modified
Updated fix for #10339 so that persistent fields revert to original values after a clear operation
* Added direct WebSocket upgrade in the Jetty core WebSocket APIs.
* Updated the WebSocket documentation.
* Optimized WebSocketMappings.getMatchedNegotiator() to avoid allocating a lambda for every invocation.
* Cleaned up core.server.WebSocketUpgradeHandler.
* Expanded websocket docs to mention how the demand mechanism works.
* Fixed code examples with correct demand handling.
* Javadocs for api.Callback.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Lachlan Roberts
Ludovic Orban
Simone Bordet
Greg Wilkins
Joakim Erdfelt
Jan Bartel
Christian Kohlschütter
Olivier Lamy
Olivier Lamy