Fix buffer leak in HTTP and FCGI when the server is being shut down while there are in-flight requests
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* Issue #11567 - fix relative path for resourceBase set in DefaultServlet
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
* Remove the ResourceFactory adaption from between baseResource and a HttpContent.Factory
* Remove toURI usage
* ensure ee9 DefaultServlet encodes the pathInContext
* Moved toURI functionality to ResourceFactory
* use context baseResource if baseResource not set
* removed usages of URIUtil.split
* fixed javadoc
* updates from review
* updates from review
* updates from review
* updates from review
* fixed resource leak in test
* fixed resource leak in test
* updates from review
* Fixes to URIUtil for Windows (#11585)
* Fixes for Windows
* Remove test that is not needed
* inlined resolveOrNew
---------
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Co-authored-by: Lachlan Roberts <lachlan@webtide.com>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Refactorings to rationalize and simplify how we do IO with resources internally by introducing the IOResources helper.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Introduced NetworkConnector.Listener to notify of open/close events.
Applications can register a listener on one NetworkConnector, be notified when it opens, and configure other components (for example using the NetworkConnector.localPort).
Cleaned up ManagedSelector.CloseConnections, which had code that was not used, and fixed the close of non-connection elements such as the ServerSocketChannel and DatagramChannel (when acceptors=0).
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Introduced ThreadIdPool and replaced ThreadLocal with it
* Modified ReservedThreadExecutor to be backed by a ThreadIdPool of semaphores
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: gregw <gregw@webtide.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
* Introduced oej.io.Transport as the abstraction for the low-level transport of high-level protocols.
Now protocols such as HTTP/1.1 or HTTP/2 can be transported over TCP, QUIC, Unix-Domain, memory, and possibly over other low-level custom protocols too.
* Introduced oej.client.Request.transport(Transport) to specify Transport for each request.
* Introduced Transport to [HTTP2Client|HTTP3Client].connect(...) methods.
* Introduced [Client|Server]QuicConfiguration so that it can be used in other Connectors such as MemoryConnector.
* Introduced oej.server.MemoryConnector and EndPoint.Pipe for memory communication between peers, along with a MemoryTransport.
* Introduced QuicTransport as a wrapper for other Transports, so that QUIC can now also be transported over memory.
* Improved javadocs and documentation.
* Removed usage of ClientConnector.forUnixDomain() from FastCGIProxyServlet (ee10 and ee9).
* Replaced usage of HTTP3ServerConnector with QuicServerConnector in jetty-http3.xml.
* Fixed handling of Instruction notifications in case of re-entrance.
Now first clear the list, then notify to avoid that when re-entering the same instruction is notified multiple times.
* Introduced ContentSourceRequestContent, and updated ProxyHandler to use it.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Experiment with ArrayByteBufferPool
No overall size accounting
reserved buffer release always checks max memory
released buffers check max memory 1% of the time.
only a single thread can check memory at once.
single pass through buckets so no looping forever.
* Experiment with ArrayByteBufferPool
updates from review
* JMH updates
* updates from review
* Fixed comments.
Fixed call to recordEvict().
Removed unused methods.
Method getAvailable*Memory() no longer JMX-enabled, as they are the same as get*Memory().
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
---------
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* Fixes#11371 - Review ArrayByteBufferPool eviction.
* Eviction is now performed on release(), rather than acquire().
* Memory accounting is done on release(), rather than acquire().
This is because we were always exceeding the memory usage on acquire(), by returning a non-pooled buffer.
We only need to account for what is idle in the pool, and that is done more efficiently on release(), and it is leak-resistant (i.e. if the buffer is not returned, the memory is already non accounted for, keeping the pool consistent).
* Released entries now give precedence to Concurrent.Entry, rather than Queued.Entry, so the queued pool is always kept at minimum size.
* Changed eviction algorithm to be simpler: one pass through the buckets excluding the current, trying to remove idle buffers until enough memory is recovered.
If successful, the buffer being released is pooled, otherwise it is also discarded.
* Added detailed statistics to ArrayByteBufferPool.RetainedBuckets.
* Added statisticsEnabled property in Jetty module bytebufferpool.mod.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed regression introduced with #9897.
After onExpire() returns false, the entity is asked again the expirationNanoTime, but there was no check whether it was Long.MAX_VALUE, indicating that the entity should not be rescheduled.
This was causing scheduling of timeouts far in the future (about Long.MAX_VALUE nanos away), but each at a slightly different time, causing the leak.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced AutoLock.tryLock() to use it in the toString() implementations that lock in order to retrieve a consistent state to produce the string.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Reorganized ArrayByteBufferPool.evict() code to avoid NPE.
* Fixed acquire logic to take into account that a reserved entry may fail to be enabled because it may be concurrently removed.
* Small change to ConcurrentPool: no need to null check the final ConcurrentEntry.holder field.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixes#10234
* Introduced transient failures in reads where a failure chunk has last=false.
* Transient failure now do not fail the handler callback.
* Improve eeN ContentProducer to more carefully assert transient and terminal errors + enable HttpInputIntegrationTest
* Do not add connection: close to the response when the error is transient
* Rework ChunksContentSource to support null chunks
* Added tests to verify the new transient failure cases
* Review all code that handles failure, and handling correctly transient failure, either by making them fatal, and/or by failing Content.Source.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Signed-off-by: Olivier Lamy <olamy@apache.org>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Co-authored-by: Chad Wilson <chadw@thoughtworks.com>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
The local/remote SocketAddress is cached within the ConnectionMetaData or Connection instance, so that any changes are not visible during the request lifetime.
Ensure that all server Connection types respect HttpConfiguration#getLocalAddress and that it is not implemented only in servlet layer
Avoid DNS resolution.
---------
Co-authored-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* The Pool is now a list of Holder instances, each with a WeakReference and a strong reference to an Entry.
* Removed thread-local cache from Pool.
* Avoid using deprecated ConcurrentPool constructors.
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
Improve handling of SSL metadata in ConnectionMetaData, proxied connections and forwarded requests
+ Added an Attributes.Synthetic mechanism to better (and mor commonly) handle additional attributes, whilst meeting the new TCK requirement that they can be deleted or replaced.
+ Made SslSessionData a reusable type provided by EndPoint
+ Added the EndPoint.Securable subtype that can provide a SslSessionData instance for secure EndPoints
+ Updated ForwardedRequestCustomizer, SecureRequestCustomizer to understand Securable and SslSessionData
+ Javadocs improvements.
+ Removed unnecessary attribute constants in SecureRequestCustomizer.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
---------
Signed-off-by: Olivier Lamy <olamy@apache.org>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Co-authored-by: Olivier Lamy <olamy@apache.org>
Co-authored-by: Simone Bordet <simone.bordet@gmail.com>
* #10703 fix race condition in clear() and make sure non-reusable released entries are removed from the pool
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
* Avoid a copy in MutableHttpFields.asImmutable if the mutable is never mutated again.
* reduce instance creations needed for iterations
---------
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Fixed test case that was racy.
When the DATA frames arrived at the server before the call to consumeAvailable(), they were read and the client flow control window re-opened.
If it happened that the DATA frames arrived at the server after the call to consumeAvailable(), the client flow control window was not re-opened, making the test flaky.
Fixed by avoiding the race in the test.
Added over-release buffer tracking, add leak tracking to H2 tests, fix client leaks in tests.
Also reviewed the places that required re-opening of the flow control window in case the DATA frames are not read.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Ludovic Orban <lorban@bitronix.be>
Co-authored-by: Ludovic Orban <lorban@bitronix.be>
Web functions are currently supported with servlets. These changes add/move utility classes to core to better support direct usage of core APIs
* increase usage of Charset in request
* Added flush mechanism to BufferedContentSink
gregw
Ludovic Orban
Joakim Erdfelt
Simone Bordet
Lachlan
Olivier Lamy