Updating SslContextFactory configuration for tests, since
the change in the default endPointIdentificationAlgorithm
makes the test failing as the certificates are not valid
for the local host, which is different depending on where
the tests are run (locally, jenkins).
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced PlainOrSslConnectionFactory, to "sniff" the first bytes
on a connection and upgrade to SSL (if the bytes are TLS bytes), or
upgrade to a specific, configured, protocol.
Added also the ability to fail the upgrade in case of a `http`
request to a `https` port and write a minimal response to the client.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Using system property "org.eclipse.jetty.io.forceSelectNow"
to force a selectNow() call; if absent, forcing selectNow()
only in the Windows OS.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Move RetainableByteBuffer to jetty-io=
use RetainableByteBuffer
use RetainableByteBuffer - changes from review.
Reviewed and applied small changes.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced safeInterestOps() and safeReadyOps() to catch exceptions
they may throw and using them in relevant places to fix the issue.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Issue #3038 - SSL connection leak.
Fixed SSL spin caused when fill had NEED_WRAP, but a flush/wrap
produced 0 bytes and stayed in NEED_WRAP
Removed check of isInputShutdown prior to filling that allowed EOF to
overtake data already read.
Fix for leak by shutting down output in HttpConnection if
filled -1 and the HttpChannelState was no longer processing
current request.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* Cleanup the dump implementation
* improved the clarity of utility methods for dump and updated most dump methods
* fixed upgrade filter dump
* Improved dump after review
* Moved dumpObjects to Dumpable
* implemented dumpBeans with dumpObjects
* less verbose dump
* Dump streams
* fixed dump test
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Issue #2954 Report badmessage cause
Pass BadMessageException from parser to HttpReceiverOVerHTTP
This change has already mostly been made in 9.4, so essentially this is a back port. However the
primary signature of HttpParser.Handler for badMessage has not been changed and a default method
used to handle the cause. This avoids breaking any usages of the interface.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
+ Changes needed for new Junit 5
+ Migrating from Vintage junit API to Jupiter junit API
+ Relies on SNAPSHOT jetty-test-helper
- this will be a formal release once this issue has been
resolved satisfactory
+ Have jenkins always pull latest SNAPSHOT for each build
+ Adding jetty.snapshots repository
+ Using surefire 2.22.0 per advice from junit
+ Ensuring <reuseForks>true</reuseForks> to work around issue junit-team/junit5#801
+ Disabling <forkMode>always</forkMode> in maven-surefire-plugin
due to bug https://github.com/junit-team/junit5/issues/801
+ OSGi tests must remain at vintage due to PaxExam
+ Moving from vintage TestingDir to jupiter WorkDir
+ Fixing imports to use jupiter, not vintage
+ Migrating vintage ExpectedException to jupiter assertThrows
+ Migrating vintage TestName to jupiter TestInfo
+ Migrating @RunWith(Parameterized.class)
to @ParameterizedTest with Argument Sources
+ Migrating assertTrue(val.contains(needle))
to assertThat(val, containsString(needle))
+ Aligning junit versions per recommendations from @sormuras
+ Adjusting parameter order change for assertEquals()
+ Test LifeCycle Annotation Migration
junit 4 | junit 5 / jupiter
------------ | -----------
@Before | @BeforeEach
@After | @AfterEach
@BeforeClass | @BeforeAll
@AfterClass | @AfterAll
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Signed-off-by: olivier lamy <oliver.lamy@gmail.com>
Issue #2711 - TLS 1.3 compliance.
Disabled few tests that are TLS 1.2 specific.
Renegotiation in SslConnection is now skipped for TLS 1.3.
Replaced SNI keystore DSA certificate with RSA certificate.
First full build achieved with JDK 11+28.
Small changes after review.
Modified the test case to pass in JDK 8, where the implementation
throws SSLException, while in later JDKs throws SSLHandshakeException.
Minor cleanup
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
WIP.
Tentative fix to avoid that onFillable() is called after the exchange
is completed.
Now DecryptedEndPoint.fillInterest.fillable() is called before other
methods that may advance the exchange processing.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now IllegalStateExceptions are thrown with a message.
Methods needsFillInterest() and onIncompleteFlush() are now wrapped in
try/catch to make sure the connection is closed in case of exceptions.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Isolated SSLEngine method calls and wrapped them into try/catch
blocks to ignore the NullPointerException thrown by Conscrypt.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now checking for thread interrupted only when select() reports 0 selected.
This will avoid the check in case of legit I/O wakeups.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Added check for thread interrupted.
If interrupted and ManagedSelector is not running, bail out.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Major refactor of SslConnection to address #2233 and to simplify in preparation for java-11 support.
Made the `needFillInterest` and `onIncompleteFlush` methods the primary stateful methods with state for fill and flush side that does not reproduce state already held by the SslEngine itself.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
After discussion on openjdk/nio-dev, we now wakeup the selector
after closing a socket, so that the SelectionKey can be removed
from the Selector and the TCP stack notified that the socket
has been really closed, so that it can send RST to clients.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #2468 - Remove SoLinger.
For non-blocking sockets, StandardSocketOptions#SO_LINGER javadocs
report that the behavior is undefined. In JDK 11 setting SoLinger
for non-blocking sockets will be ignored.
As such, there is no point in allowing SoLinger to be configured
in Jetty that only uses non-blocking sockets.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
WriteFlusher.write() now requires the callback it is given is not null
the FAILED state of WriteFlusher is now a terminal state
any failure will now result in the callback being failed and a transition to the FAILED state
the WriteFlusher documentation and WriteFlusherTests have also been altered to reflect these changes
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
removed HttpOutput.close(Closeable) method as IO.close(Closeable) should be used instead
added isFailed() method to WriteFlusher and used it to fix WriteFlusherTest.testFailWhileBlocking()
surrounded usage of onError() in HttpOutput.run() with try-finally so that IO.close(this) is executed if onError throws
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
changed write flusher to go from pending state to failed state
reverted previous HttpOutput changes
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
* Fixes#2366 - Review HTTP/2 interleaving.
Now using a fairer algorithm that performs multiple generation passes
but without generating all frames for the current entries - it stops
after N passes over all the entries when the number of bytes to
write is large enough.
Using HttpConfiguration.outputBufferSize as the writeThreshold on
the server and a default 32 KiB on the client to stop the generation
after a pass over all the entries.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
+ New SocketChannelEndPointTest replacement class
* Is Parameterized for both Normal/Ssl scenarios
* Builds a new set of connections/selector manager etc
for each test
* TestConnection moved to inner static class to not rely
on test class fields/methods
* Adding SafeInteger to help track down bad test behavior.
Blocking Read behavior doesn't do what we think, see Exception on CI.
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
+ Renamed old SocketChannelEndPointTest to
SocketChannelEndPointOpenCloseTest
+ Moved SelectChannelEndPointSslTest.checkSslEngineBehavior() to
new SslEngineBehaviorTest as its unrelated to the rest
of the SelectChannelEndPointTest logic
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
+ Renamed SelectChannelEndPointInterestsTest to
SocketChannelEndPointInterestsTest
+ Merged EndPointTest abstract class into SocketChannelEndPointTest
as it's only used by that testcase
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Hoping that this will reveal the specific circumstances (test order)
that lead to the frequent test failures on CI
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* Only close if parser closed and output is shutdown
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* a better possible fix
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* after review
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Protecton from recursion in SSL flush try again #2233
This would not be needed if we could make flush iterate when necessary.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Objects which inherit or implement an `equals()` method should not be compared with == or !=
When the comparison of references is intentional `@SuppressWarnings("ReferenceEquality")` can be used
Signed-off-by: Lachlan Roberts <lachlan@webtide.com>
Merging this PR for #2199 as we currently think it is not worse than previous and the code base is definitely simpler. While JMH has shown significant benefits for this approach, we have yet been able to demonstrate them in full scale integration tests - however that is likely due to GC and thread pool issues dominating.
A `org.eclipse.jetty.http2.PEC_MODE` System property has been added to allow the EWYK scheduling to be disabled for HTTP/2 if need be.
* Implementation of #1803 proposal 2 - EITHER dispatch type for EWYK
* made code more readable
* increase small threadpools in tests for extra reserved thread
* clean up
* minor code simplifications
* Work in progress to simplify reserved thread pool
* use a single ReservedThreadPool built into the QueuedThreadPool
* fixed javadoc
* removed more old reserved thread references
* disable EWYK for h2
* fixes from review
* use EWYK for HTTP2
* Fixed javadocs, imports and QueuedThreadPool constructors.
* fix javadoc
* EWYK avoid unnecessary pendings
* after review
* fixed merge with jetty-threadpool.xml
* alternate EWYK implementations
* added jetty copyright headers
* Simplified EWYK code after review
* fixed bad merge
* Code cleanups.
* Improved Javadocs for deprecated property "reservedThreads".
* Improved Javadocs for deprecated property "reservedThreads".
* added a system property to enable only PEC for HTTP/2
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Issue #1803 - Review usage of scheduling strategies
Use a single ReservedThreadExecutor built into the QueuedThreadPool
via new interface TryExecutor.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
There are still problems with this impl (some client tests ignored) and there is still a work around for the JNR bug 50, however this impl is already much better than the unix socket support that is already in the release. So will merge for now and put more effort in once there is a JNR fix.
* WIP add unix domain sockets support in HttpClient
* move unix socket client part to unix socket module #2014
* some cleanup #2014
* add missing headers #2014
* add TODO
* UnixSocket client refactor
* cleanup test and pom
* minor changes, use LOG.isDebugEnabled() before using debug method
* add UNIX SOCKET http client test with all other tests, push this to see what happen on Jenkins
* fix some unit tests
* fix more tests
* fix load test
* UnixSocket client
* Demonstrate JNR bug
* Worked around JNR bug 50
* close channel on client side as well
* more details in log
* log file path as well
* #2014 disable test per default as doesn't work on some environement
* Revert "#2014 disable test per default as doesn't work on some environement"
* test only on unix
* Allow test of specific transport(s)
* Move unix socket to /tmp
* move test socket to /tmp
* move test socket to /tmp
* ignore failing tests for now
* fix bean name and possible to use sys prop org.eclipse.jetty.http.client.AbstractTest.Transports with mvn cli
* test isBlank as surefire props is not null
* correctly create tmp file with @Before
* do not delete file
* use /tmp as build directory doesn't seem to work within docker...
* do not delete sock file on client as it is own by the server
* file must not exist when binding unix socket
* #2014 fix license header
* network specific tests assumed
* Fixed to handle null selector keys
* add assume for tests that assume a network connector
Signed-off-by: olivier lamy <olamy@webtide.com>
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* Scalable scheduler changes for #1918
* Added HttpChannel.destroy to destroy CyclicTimer
* fixed rebase with HttpConnectionOverFCGI
* renamed to acquire
* Destroying the HttpChannel consistently in all transports.
* updated headers
* cleanup after final review
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Clean up of actions (now updates) prior to #2046 fix
* prevent exceptions from termincating lifecycle doStop or destroy
* Refactored ManagedSelector stop to always close endpoints
* Fixed NPE if SelectorManager is already stopped
* refactored after review
* further simplifications after review
* Wait only for oshut endpoints
* Cleanup from review
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Issue #2081 No idle timeout exception when dispatch is delayed
* Delegate the readtimeout handling to HttpChannel so that a delayed dispatch can be ended.
* Added unit test for delayed dispatch idle
* Now using HttpInput.onIdleTimeout() to fail the HttpInput, and then dispatching the request in case it has not been dispatched yet. This ensure consistent behavior independently of the value of HttpConfiguration.delayDispatchUntilContent.
* Fixed for both HTTP/1.1 and HTTP/2.
* Added tests for non-blocking reads.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* ManagedSelector dump improvements from #1970
ManagedSelector dump improvements from #1970:
+ DumpKeys is now prepended to actions list so it is less likely to be delayed by a stuck/busy selector
+ Timestamps are included for actions and keys which may be separated by time
+ Race removed race for updating dumpKey list while it is being added to.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* moved utility timestamp format method to Log
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* updates after review
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* changes after review
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* Issue #1970 - ManagedSelector dump improvements.
Code cleanups.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Revert "Issue #1970 - ManagedSelector dump improvements."
This reverts commit 4febaf10dc.
* Fixed imports and other review feedback
Signed-off-by: Greg Wilkins <gregw@webtide.com>
* Code cleanups.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Improved test case handler.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Improved exception message.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #1973 - Implement minimum response data rate.
Implemented response content data rate control in HttpOutput.
Introduced a WriteFlusher.Listener interface that produces events
for every flush(). These events are forwarded to the Connection
and from there to the HttpOutput so that the data rate control can
be enforced.
Both HTTP/1.1 and HTTP/2 are implemented.
Data rate control for HTTP/1.1 is approximate because it will count
also headers bytes and the chunk bytes, while for HTTP/2 is precise.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #1973 - Implement minimum response data rate.
Addressed review comments.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
CreateEndPoint and DestroyEndPoint are now submitted directly to
the Executor, rather than being submitted as selector actions.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Tag for release: jetty-9.4.8-SNAPSHOT
+ 212 HttpClient should support pluggable AuthenticationStore
+ 215 Add Conscrypt for native ALPN/TLS/SSL
+ 272 WebSocket hangs in blockingWrite
+ 487 JDK 9 build compatibility
+ 901 Overriding SSL context KeyStoreType requires explicit override of
TrustStoreType
+ 922 Implements methods Connection.getBytes[In|Out]()
+ 1209 IllegalStateException when HTTP/2 push is disabled
+ 1213 Upgrade to ASM Version 6.0 for JDK9
+ 1509 Improve GZIPContentDecoder buffer pooling sizing
+ 1550 Resolve inconsistent Shutdown configuration with Jetty Runner
+ 1640 Introduce :run-distro goal for local jetty distribution deployment
+ 1692 Annotation scanning should ignore `module-info.class` files
+ 1696 Missing stacktraces on debug of WriteFlusher onFail
+ 1705 Rejected executions in QueuedThreadPool can lead to memory leaks
+ 1760 Update to apache jasper 8.5.20
+ 1768 Allow jetty properties to be set for the jetty:run-forked goal
+ 1782 Using assembly.tarLongFileMode=posix for jetty-home and
jetty-distribution assembly
+ 1797 JEP 238 - Multi-Release JAR files break bytecode scanning
+ 1806 Improved ReservedThreads idle timeout
+ 1807 Add new HttpChannel listener and events for metrics libraries
+ 1814 Move JavaVersion to jetty-util for future Java 9 support requirements
+ 1818 Improve Infinispan support on JDK 9
+ 1819 Race condition during annotation parsing
+ 1823 ResourceHandler with ranged requests does not return Content-Type
response header
+ 1829 OSGi webbundle classes scanned twice
+ 1833 Request.startAsync requires context path
+ 1835 Locker is not reentrant on ServerConnector#setConnectionFactories
+ 1836 Migrate Locker implementation to JVM ReentrantLock implementation
+ 1841 Reduce contention on ServletHolder
+ 1845 Allow null User-Agent in HttpClient
+ 1849 Refactoring of SelectorManager.defaultSchedulers()
+ 1851 Improve insufficient thread warnings/errors
+ 1854 Consistent IOException and timeout handling when extracting form
parameters
+ 1856 ResourceHandler without ServletContext throws NPE for welcome files if
used directly
+ 1857 GZIPContentTransformer fails to send entire message if used with
BufferedContentTransformer
+ 1865 Improve Exception on invalid redirect usage
+ 1867 Improve Exception thrown during Expect 100 Continue
+ 1868 Need a way to randomly select ports for tests
+ 1871 JMXify SslContextFactory
+ 1878 Handle 100 Continue response without Expect header
+ 1879 'Bad tld url' seen during :jetty-run when running integration tests
+ 1881 Improve support of WebSocket over Unix Domain Socket
+ 1885 SessionHandler get/set maxInactiveInterval is not symmetric with
negative values
+ 1888 Implement cookie matching on Path attribute per RFC 6265
+ 1891 Make HTTP/2 async error notifications configurable
+ 1892 NPE resulting from bad JEP 238 MultiReleaseJarFile structure
+ 1893 Add ability to set HttpClient Connection TTL
+ 1897 Introduce a round-robin connection pool for HttpClient
+ 1900 Update to CDI 2.0 for cdi module
+ 1901 Reimplement PathWatcher as scanner
+ 1909 Update to Apache Jasper 8.5.23
+ 1910 Remove unused jetty-jsp module
+ 1912 AbstractConnector EndPoint leak for failed SSL connections
+ 1914 HttpClient fails to parse Content-Type response header with RFC 2045
charset="utf-8" syntax
+ 1919 Review LowResourceMonitor
+ 1920 Connect Timeouts with NonBlocking CreateEndPoint
+ 1924 ManagedSelector can livelock under high load
+ 1931 Expose RolloverOutputStream for pluggable behaviour
+ 1933 Use CLASSPATH for scanning java9 system classes
+ 1956 Store and report build information of Jetty
+ 1958 Blocking Timeout has different behavior in HttpInput vs HttpOutput
+ 1970 ManagedSelector can lose selector thread under high concurrent load
+ 1980 PushCacheFilter does not push TLS offloaded HTTP/2 requests
+ 1981 Loading resource content failed
+ 1984 Remove jetty-client dependency in jetty-rewrite
* Issue #1970 - ManagedSelector loses selector thread.
Removed broken data structure ConcurrentStack (ABA problem).
Made ReservedThreadExecutor use a ConcurrentLinkedDeque
instead of ConcurrentStack.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #1924 - ManagedSelector livelock.
Alternate implementation that is count based rather than time based.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Made CreateEndPoint a blocking task, because it calls into application
code via Connection.Listener, and for safety we assume it may be
blocking code, avoiding to stall the processing of NIO selected keys.
Only for the client having a non-blocking CreateEndPoint was a problem,
because it was sending requests from onOpen().
For a busy client with always queued requests, CreateEndPoint.run()
did never return, causing the NIO selection to stall.
Now upgrading the connection also in cased of filled=-1, so that the
new nested connection attempts to read/write and will see a failure
(although not the SslHandshakeException).
Simone Bordet
Olivier Lamy
Greg Wilkins
Joakim Erdfelt
Jesse McConnell
Kazuhiro Sera
Lachlan Roberts
olivier lamy