* Fixes#11763 - Race condition in QoSHandler.
Now using a read-write lock to atomically execute expire().
This guarantees that there are no races with resume().
The concurrency between handle() and resume(), which should be the most common case, is handled by atomic data structures.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Refactored length checks so that they are mostly performed once, rather than multiple times, during parsing.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Fixes#11679 - Jetty 12.0.8 seems to leak connection when it encounters earlyEOF.
* Changed HttpConnection.RequestHandler.earlyEOF() to produce EofException instead of BadMessageException, as it is more appropriate.
* Changed handling of HttpChannelState.onFailure() to not fail the write side unless there is a pending write callback.
* Early EOF events now produce a EofException that is also an HttpException.
* Now failures only impact pending writes, so that it would be possible to write an HTTP error response.
---------
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* Addendum to #11566.
Restored methods that were removed in WebAppClassLoader.Context.
Fixed method signatures for deprecated method -- they must take the deprecated ClassMatcher, not the newly introduced one.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #11514 - Cleanup `jetty.webapp.addServerClasses` property behavior for ee10/ee9/ee8
* Merging patterns (default -> env -> config)
* Moved ClassMatcher to util
* System Server Classes
* Renaming to Hidden/Protected
* Updated for loss of dump.war
* Update jetty-core/jetty-ee/src/main/java/org/eclipse/jetty/ee/WebappClassLoading.java
* fixed OSGi tests
* Ensure default hidden server and system classes.
* Fixed ee9 OSGi to load jetty-ee
* Adding more deprecations
* Changing XML demos/tests to use new getter names
* rollback xml changes in ee9/ee8
* Fix ee8 maven plugin
* Make jetty-maven dependency optional in ee9 to match ee10
---------
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Co-authored-by: gregw <gregw@webtide.com>
* #11687 make HttpFields.Mutable.Wrapper.computeField() call onRemoveField() and remove the field when null is returned by computeFn
* #11687 replace IAE with NPE
* #11687 replace collect(Collectors.toList()) with toList()
---------
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* #11631 abort instead of rethrow when the exception handler in handle() throws an exception
* fix ee10 state machine error when client aborts
* H3: wire client-sent reset packet on abort
* H3: only fail finished streams when fill interested is false
---------
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Fix buffer leak in HTTP and FCGI when the server is being shut down while there are in-flight requests
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* implemented changes to CACHE and parseFields to handle OWS properly for all UNMATCHED_VALUE headers.
* added 3 new OWS test cases (that fail in 12.0.x HEAD btw) to handle this OWS case.
* Issue #11514 - Cleanup `jetty.webapp.addServerClasses` property behavior for ee10/ee9/ee8
* Fix test
* Merging patterns (default -> env -> config)
* Moved ClassMatcher to util
* Adding more deprecations
* Changing XML demos/tests to use new getter names
* rollback xml changes in ee9/ee8
---------
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Co-authored-by: Jan Bartel <janb@webtide.com>
Allows to suppress "caused by" in error message, not only the stacks traces.
ErrorHandler.showCause = false by default.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Dennis Hoersch <dennis.hoersch@springernature.com>
Co-authored-by: Dennis Hoersch <dhs3000+ghtu@posteo.de>
* Issue #11567 - fix relative path for resourceBase set in DefaultServlet
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
* Remove the ResourceFactory adaption from between baseResource and a HttpContent.Factory
* Remove toURI usage
* ensure ee9 DefaultServlet encodes the pathInContext
* Moved toURI functionality to ResourceFactory
* use context baseResource if baseResource not set
* removed usages of URIUtil.split
* fixed javadoc
* updates from review
* updates from review
* updates from review
* updates from review
* fixed resource leak in test
* fixed resource leak in test
* updates from review
* Fixes to URIUtil for Windows (#11585)
* Fixes for Windows
* Remove test that is not needed
* inlined resolveOrNew
---------
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Co-authored-by: Lachlan Roberts <lachlan@webtide.com>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Refactorings to rationalize and simplify how we do IO with resources internally by introducing the IOResources helper.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Initialized PEM directories for both client and server.
Added [lib] section that was missing due to #11263.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #11495 - Add UriCompliance rules that follow the HTTP / URI / Servlet specs for illegal & suspicious characters
* more illegalPathCharacterData test cases
* Correcting SUSPICIOUS_PATH_CHARACTERS semantic (encoded and decoded)
* Check for illegal and suspicious characters as we are parsing the path.
* Only look for ambiguous paths if we know there are dots or encodings.
---------
Co-authored-by: gregw <gregw@webtide.com>
* Issue #11539 restore old behavior of Resource.copyTo()
* Do not delete destination resource
if it exists before copy to restore
the behavior to what we had in
Jetty 9/10/11.
* Formalizing IO.resolvePath()
Introduced NetworkConnector.Listener to notify of open/close events.
Applications can register a listener on one NetworkConnector, be notified when it opens, and configure other components (for example using the NetworkConnector.localPort).
Cleaned up ManagedSelector.CloseConnections, which had code that was not used, and fixed the close of non-connection elements such as the ServerSocketChannel and DatagramChannel (when acceptors=0).
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Defaulted showStacks to false, to reduce false positives reported by penetration testing tools.
Deprecated ErrorHandler.badMessageError(), as it was not invoked anymore.
Implemented HttpStreamOverHTTP[2|3].onBadMessage() that was left as TODO.
Simplified the same code for HTTP/1.
Moved invocation of ComplianceViolation.Listener.onRequestEnd() to HandlerInvoker.completeStream().
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Deprecated idle(), acquired(), released() for removal.
Renamed removed(), now deprecated, to onRemoved() to match with onCreated().
Kept onCreated() and onRemoved() as they are the only methods that are not racy.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Introduced ThreadIdPool and replaced ThreadLocal with it
* Modified ReservedThreadExecutor to be backed by a ThreadIdPool of semaphores
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: gregw <gregw@webtide.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Fix#11411 by allowing non-existent resources to be returned from resolve
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
#11482 introduce new onComplete event in EventsHandler and use it to record status in StatisticsHandler
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Use ByteBuffer.getLong to look for entire request (GET / HTTP/1.1) or response (HTTP/1.1 200 OK) line with 2 long lookups. Failing that, a single long lookup is sufficient to determine the common methods and/or HttpVersion.
* Cleanup generator also
* Added a fallback int lookup
* HttpURI toURI passes all info
Fix#11465 and #7750
HttpURI.toURI user and fragment are retained.
Use to URI(String) constructor, as all URI constructors will parse the URI anyway.
* HttpURI toURI passes all info
Fix#11465 and #7750
HttpURI.toURI user and fragment are retained.
Use to URI(String) constructor, as all URI constructors will parse the URI anyway.
This makes the default configuration more secure and explicitly requires configuration from users.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Fix#10805 zero dynamic table (#11445)
* Added test for #10805 Zero Dynamic Table
* fixed file header
* Added test for #10805 Zero Dynamic Table
* Fix for #10805 Zero Dynamic Table
Set the correct default size for the table.
Always send the max table size on the first encode
* updated file header
Signed-off-by: gregw <gregw@webtide.com>
---------
Signed-off-by: gregw <gregw@webtide.com>
* Issue #11387: Reintroduce MultiPartCompliance.LEGACY in ee9/ee8
* Correcting javadoc
* Updating MultiPartCaptureTest to ...
* Test with MultiPartFormData.Parser and MultiPart.Parser
* Enable all test cases
* base64 behaviors modified to not auto-decode base64 content
* forms submitted without `_charset_` part (some using a different
charset than UTF-8, like `Shift_JIS`)
* Fixing checkstyle warning
* Re-enable Part-ContainsContents expectations
* Rename MultiPartCompliance.NO_CRLF_AFTER_PREAMBLE to WHITESPACE_BEFORE_BOUNDARY to fit spec better
* Make ee9/ee8 legacy parser use legacy tokenization
* Testing ee9/ee8 legacy parser base64 auto-decoding behaviors
* Cleanup jetty-test-multipart class naming
* Adding ee10 tests against raw multipart examples
* Adding shorter whitespace multipart test
* Adding jetty-core version of failing ee10 tests
* Fixed missed notification for CR content in case of 1 chunk ending with CR and the next chunk ending with LF.
* Removed internal unused class MultiPartParser.
* Adding MultiPartCompliance.Violation events
+ in MultiPart.Parser
+ in MultiPartFormData.Parser
* lenient mode behavior
* new name fits violation better
+ adding violation to MultiPart.Parser.parseHeaderStart
* some simple cleanup of new ee9 code
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Now catching and rethrowing XmlConfigurationException with details about the XML file location.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Added null guard.
* Avoid executing the event actions in case the response is either complete or (new change) terminated.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Improve jetty-util on Windows
* Enable ATOMIC_MOVE on Resource.copyTo()
* Add reference to Resource impl in exception.
* Attempting to address sneaky Windows path strings that look like URIs
* Deprecate URIUtil.correctFileURI in favor of new URIUtil.correctURI method
Removed the call to `ServletChannel.abort()` from the write callback.
As the write was issued from `ServletChannel.handle()` case COMPLETE, it was eventually calling `ServletChannelState.completed(Throwable)`, which is expecting the requestState to be COMPLETING.
However, calling `abort()` would set the requestState to COMPLETED, causing the IllegalStateException.
There should be no need to call `abort()` from the callback of failed writes, since failing the various callbacks should be enough, eventually failing the `HttpStream`, which would take care of tearing down the connection (HTTP/1) or the stream (HTTP/2+).
Now aborting the response from ServletChannelState.completed(Throwable).
Fixed SizeLimitHandler exception message.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Introduced oej.io.Transport as the abstraction for the low-level transport of high-level protocols.
Now protocols such as HTTP/1.1 or HTTP/2 can be transported over TCP, QUIC, Unix-Domain, memory, and possibly over other low-level custom protocols too.
* Introduced oej.client.Request.transport(Transport) to specify Transport for each request.
* Introduced Transport to [HTTP2Client|HTTP3Client].connect(...) methods.
* Introduced [Client|Server]QuicConfiguration so that it can be used in other Connectors such as MemoryConnector.
* Introduced oej.server.MemoryConnector and EndPoint.Pipe for memory communication between peers, along with a MemoryTransport.
* Introduced QuicTransport as a wrapper for other Transports, so that QUIC can now also be transported over memory.
* Improved javadocs and documentation.
* Removed usage of ClientConnector.forUnixDomain() from FastCGIProxyServlet (ee10 and ee9).
* Replaced usage of HTTP3ServerConnector with QuicServerConnector in jetty-http3.xml.
* Fixed handling of Instruction notifications in case of re-entrance.
Now first clear the list, then notify to avoid that when re-entering the same instruction is notified multiple times.
* Introduced ContentSourceRequestContent, and updated ProxyHandler to use it.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
#10226
- fix HttpClientIdleTimeoutTest to wait for server's idle timeout before checking for leaks
- improve HttpClientIdleTimeoutTest by making it upload some content
- fix FCGI server leak caused by idle timeout
- fix H3 server leak caused by idle timeout
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* Experiment with ArrayByteBufferPool
No overall size accounting
reserved buffer release always checks max memory
released buffers check max memory 1% of the time.
only a single thread can check memory at once.
single pass through buckets so no looping forever.
* Experiment with ArrayByteBufferPool
updates from review
* JMH updates
* updates from review
* Fixed comments.
Fixed call to recordEvict().
Removed unused methods.
Method getAvailable*Memory() no longer JMX-enabled, as they are the same as get*Memory().
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
---------
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Now the calls to the upper layer produce tasks that are fed to the ExecutionFactory in HTTP2Connection.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed case in MultiPart.Parser where a small chunk contains part of the boundary.
Added and fixed related tests.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #11414 - use HttpURI instead of URIUtil to have a single point of spec behavior
* Issue #11414 - enforce lowercase scheme in HttpConfiguration.secureScheme
* Issue #11414 - Scheme produced on `Location` header is lowercase
* Issue #11414 - Scheme to lowercase
* Issue #11414 - Scheme to lowercase
* Revert change to HttpClient
* Added schema port knowledge to URIUtil
* Fixed tests for normalized URIs
* updates from review
* updates from review
* Fix tests
* Restored methods as deprecated
* More testing
---------
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* Fixes#11371 - Review ArrayByteBufferPool eviction.
* Eviction is now performed on release(), rather than acquire().
* Memory accounting is done on release(), rather than acquire().
This is because we were always exceeding the memory usage on acquire(), by returning a non-pooled buffer.
We only need to account for what is idle in the pool, and that is done more efficiently on release(), and it is leak-resistant (i.e. if the buffer is not returned, the memory is already non accounted for, keeping the pool consistent).
* Released entries now give precedence to Concurrent.Entry, rather than Queued.Entry, so the queued pool is always kept at minimum size.
* Changed eviction algorithm to be simpler: one pass through the buckets excluding the current, trying to remove idle buffers until enough memory is recovered.
If successful, the buffer being released is pooled, otherwise it is also discarded.
* Added detailed statistics to ArrayByteBufferPool.RetainedBuckets.
* Added statisticsEnabled property in Jetty module bytebufferpool.mod.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Removed toString() override that was not calling super, to restore printing the total number of keys in dumps.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed regression introduced with #9897.
After onExpire() returns false, the entity is asked again the expirationNanoTime, but there was no check whether it was Long.MAX_VALUE, indicating that the entity should not be rescheduled.
This was causing scheduling of timeouts far in the future (about Long.MAX_VALUE nanos away), but each at a slightly different time, causing the leak.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* In ServletChannel, in COMPLETE state, there was a check comparing the number of bytes written with that declared by Content-Length.
Unfortunately the check was wrong in case of gzip, because it was comparing the application length with the gzipped length, resulting in a late sendErrorOrAbort() that was an abort() because the response was already fully sent.
Furthermore, if sendErrorOrAbort() was actually an abort(), there was an unnecessary attempt to complete the output.
* In GzipHandlerResponseAndCallback there were 2 last writes: one when the application calls output.close(), and one when the callback is succeeded.
Furthermore, when no content needs to be written, it was still compressed, causing an IOException in ChannelResponse (again when double-checking the bytes written, since Content-Length was 0, but the gzip header and trailer were written).
* Updates semantic of when to add the "Vary" header.
Now it is always only added by GzipHandler, when it would be possible to generate a response that might be compressed (even if it will not).
* Added *.bz2 mime-type.
Updated GzipHandler with the correct mime-type for *.bz2: application/x-bzip2.
* Improved checks for the number of bytes written against declared content-length.
This is necessary because welcome files are written by DefaultServlet (most of the times) bypassing the Servlet classes, using ServletContextResponse directly.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Changed `JettyWebSocketFrameHandlerFactory` to use an application MethodHandle.Lookup (rather than a server one) when creating listener metadata.
This fixes the JPMS problem so that now JPMS applications do not need any additional configuration to invoke endpoints.
The (acceptable) downside is that anonymous inner classes (that are not public) cannot be used as listener endpoints.
This change affects core and EE10 WebSocket; EE9 and EE8 WebSocket have not been fixed (so they allow anonymous inner classes but are broken in JPMS).
Renamed "connectLatch" fields to "openLatch" in test classes.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Olivier Lamy <olamy@apache.org>
+ Introduce new events on `ComplianceViolation.Listener`
+ Introduce new `ComplianceViolation.Listener.initialize()` to allow for a new Listener at the appropriate time (to support per-request listeners)
+ Introduce new `ComplianceViolation.CapturingListener`
+ Introduce new `HttpConfiguration.(add/remove/get)ComplianceViolationListener()` methods.
+ Deprecate/Remove handling of `recordComplianceViolations` in `HttpConnection` and `HttpConnectionFactory` classes.
+ Produce warnings if use of `ComplianceViolation.Listener` as beans is still present.
+ Add `ComplianceViolation.Listener` support to `UriCompliance` locations.
+ Add `ComplianceViolation.Listener` support to `MultiPartCompliance` locations.
+ Add `ComplianceViolation.Listener` support to `CookieCompliance` locations.
+ Add `ComplianceViolation.Listener` support to `HttpCompliance` locations in HTTP/2 and HTTP/3
---------
Co-authored-by: gregw <gregw@webtide.com>
Introduced AutoLock.tryLock() to use it in the toString() implementations that lock in order to retrieve a consistent state to produce the string.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Using @foo.version@ technique and proper
lib lists to allow arbitrary version
updates from properties
* Fix spotless
* Remove duplicate entries
* Only replace System property "jetty.version" with manifest-ver if blank/null
* Fixing bad module definitions
Now upon the second idle timeout, the connection is forcibly closed.
Fixed also similar problem in HTTP/3.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed parsing of form parameters in FormFields.parse().
Added more tests for valid edge cases, and invalid cases.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Reorganized ArrayByteBufferPool.evict() code to avoid NPE.
* Fixed acquire logic to take into account that a reserved entry may fail to be enabled because it may be concurrently removed.
* Small change to ConcurrentPool: no need to null check the final ConcurrentEntry.holder field.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #11009 - add test for bad Jakarta endpoint
* Issue #11009 - ensure endpoint deployable before adding ServerEndpointConfig
* add same test and fix for ee9
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Most often failures come from the read side, so failure listeners are now serialized in the _readInvoker.
This avoids that a failure while parsing a request (e.g. an early EOF) results in concurrent executions of the invokeOnContentAvailable task and the invokeOnFailureListeners task.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Do not use HttpStream.Wrapper in SizeLimitHandler
* Updated the SizeLimitHandlerServletTest
* Udpated documentation and javadocs.
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Ludovic Orban
Simone Bordet
Joakim Erdfelt
Greg Wilkins
Lachlan
Olivier Lamy
Jan Bartel
Dennis Hoersch
Danish Nawab
Hervé Boutemy