Code cleanups and reformatting.
Fixed logic for SETTINGS frame replies: they are not subject to rate control.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Renamed onDataRequested() to onDataDemanded() to
avoid using the "request" word which is overloaded.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Reduce the number of RateControl fields, instead using common field in
HeaderParser.
Avoid null checking rateControl by having a NO_RATE_CONTROL static
HPack does not emit field with empty header name.
Apply rate control to any header parsing issue resulting in
session/stream failure
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Fixed race in WindowRateControl by only removing the event that we
just inspected.
Added an AtomicInteger to track the size to avoid iterating over the
linked list.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Implemented rate control for HTTP/2 frames using a single RateControl
object to avoid that each individual vulnerability is within limits,
but combined they still overload the server.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixes#250 - Implement HTTP CONNECT for HTTP/2.
Modified HTTP/2 implementation to support the CONNECT method.
Implemented semantic defined by RFC 8441.
Implemented section 8.3 of RFC 7540.
Introduced HTTP2Client.streamIdleTimeout.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Updated server-side to use direct/heap ByteBuffers based on
getters and setters in the relevant components.
Made HTTP/1.1, HTTP/2, and WebSocket use the same mechanism.
Removed unused obsoleted methods:
* EndPoint.isOptimizedForDirectBuffers()
* HttpTransport.isOptimizedForDirectBuffers()
* HttpOutput.Interceptor.isOptimizedForDirectBuffers()
* HttpChannel.useDirectBuffers()
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
After review, introduced WriteFlusher.isPending() and now using that
in the test case to test for TCP congestion.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
The client reset wakes up threads blocked in
writes, but these may again attempt to write,
therefore blocking again.
Now we detect that the stream is not writable
and mark the transport as failed, so that
writes fail immediately without blocking.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Removing Legacy Method Separators
* Restyling branch `jetty-9.4.x`
* Applying changes highlighted by checkstyle
* Applying XML restyling
* Fixing XML codestyle for IntelliJ
* Fixing XML style mistakes
* Revert "Applying XML restyling"
* Updating checkstyle for XML codestyle
* Reformatting pom.xml files
* Fixed empty string from line wraps
* Update intellij style to not do expression relative formatting. Reformatted code based on that.
* Increasing line split on Eclipse IDE Formatter to 512
* Restoring setting on internal default value.
+ IntelliJ will not export settings on things that set to their
internal default values.
We want to keep those values as a hedge against future default
value changes in future releases of IntelliJ.
* Fixing intellij codestyle
* do not allow single line simple methods
* misc checkstyle fixes
* re-exported with correct name and all values
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Modified the sender logic to allow specific subclasses to decide
when to send the trailers, if any.
This allows HTTP/2 to correctly compute the end_stream flag and avoid
sending empty trailers frames with end_stream=true.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
+ Deprecated B64Code
+ All code that isn't B64CodeTest is now using java.util.Base64
+ B64CodeTest is updated to confirm change to java.util.Base64
is possible without change in behavior. Just have to make
sure you use the appropriate Encoder / Decoder for the task
at hand (default vs mime vs url)
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Now properly resetting HttpReceiverOverHTTP2.ContentNotifier by overriding
reset() and making sure fields are cleared from previous values.
Also, disable channel recycling by default - needs to be benchmarked
whether it's beneficial or not.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced SslContextFactory subclasses Client and Server.
Replaced all usages of SslContextFactory with either Client or Server
as required.
Refactored configuration checking so that warnings are not emitted
when non necessary.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced `HttpClientTransportDynamic` to be able to switch transport dynamically.
Refactored other transports and HttpDestination, removing now unnecessary classes.
Introduced ProxyProtocolClientConnectionFactory and used it for testing.
Updated OSGi tests now that jetty-client depends on jetty-alpn-client.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
+ Changing usages of JVM deprecated classes / methods as well
+ Cleaning up test cases and javadoc
+ Removing test methods that rely on deprecated and now removed concepts
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Updated the push filters to use only the Servlet 4.0 APIs,
removing the usage of Jetty specific APIs.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Reviewed module-info.java files.
For those that have a "requires static" added a comment about
why the dependency is optional.
Rearranged directives in alphabetical order.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced ClientConnector and refactored HttpClient transports,
removing duplicated code that was connect() to a remote host.
Refactored also HTTP2Client to reference ClientConnector.
Refactored tests accordingly to the changes introduced in the
implementations.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Updating SslContextFactory configuration for tests, since
the change in the default endPointIdentificationAlgorithm
makes the test failing as the certificates are not valid
for the local host, which is different depending on where
the tests are run (locally, jenkins).
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced ClientConnector and refactored HttpClient transports,
removing duplicated code that was connect() to a remote host.
Refactored also HTTP2Client to reference ClientConnector.
Refactored tests accordingly to the changes introduced in the
implementations.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Client and server need to to treat an incoming RST_STREAM frame
differently, testing whether the stream is closed for respectively
local and remote streams.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Updated module-info.java to reference the "jetty.servlet.api" module.
* Updated POMs to reference the o.e.j.toolchain:jetty-servlet-api artifact.
* Removed references to jetty-schemas.jar.
* Updated attribute "org.eclipse.jetty.server.webapp.ContainerIncludeJarPattern"
to match the new Jetty Servlet API jar.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixes#2978 - Add module-info to relevant Jetty modules.
Added module-info.java for Jetty modules that are not test modules.
Moved jetty-http test utility classes to new module "jetty-http-tools".
Removed generation of test-jar from websocket-core.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Move RetainableByteBuffer to jetty-io=
use RetainableByteBuffer
use RetainableByteBuffer - changes from review.
Reviewed and applied small changes.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
+ Can get HttpCompliance from HttpConfiguration now
+ Signature change to HttpConnection to avoid duplicate arguments
on constructor.
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
* Cleanup the dump implementation
* improved the clarity of utility methods for dump and updated most dump methods
* fixed upgrade filter dump
* Improved dump after review
* Moved dumpObjects to Dumpable
* implemented dumpBeans with dumpObjects
* less verbose dump
* Dump streams
* fixed dump test
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Reviewed other possible places where max local stream count may
overflow.
Fixed handling of HTTP/2 stream idle timeouts.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
First pass at making Jetty 10.x require JDK 11.
* Removed JDK 8 modules (in particular ALPN and ALPN *.mod files).
* Removed profiles targeting JDK 8, 9 and 10.
* Updated dependencies to newer versions that support JDK 11.
* Temporarily commented out the Jetty Maven Plugins due to
maven-plugin-plugin not working with JDK 11.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
HPack as specified allows for values to be opaque bytes, even though HTTP only allows ISO-8859-1 within a header.
This updates HPack, so that strings are converted to UTF-8 and then encoded as bytes iff non ISO-8859-1 characters are discovered.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
+ Changes needed for new Junit 5
+ Migrating from Vintage junit API to Jupiter junit API
+ Relies on SNAPSHOT jetty-test-helper
- this will be a formal release once this issue has been
resolved satisfactory
+ Have jenkins always pull latest SNAPSHOT for each build
+ Adding jetty.snapshots repository
+ Using surefire 2.22.0 per advice from junit
+ Ensuring <reuseForks>true</reuseForks> to work around issue junit-team/junit5#801
+ Disabling <forkMode>always</forkMode> in maven-surefire-plugin
due to bug https://github.com/junit-team/junit5/issues/801
+ OSGi tests must remain at vintage due to PaxExam
+ Moving from vintage TestingDir to jupiter WorkDir
+ Fixing imports to use jupiter, not vintage
+ Migrating vintage ExpectedException to jupiter assertThrows
+ Migrating vintage TestName to jupiter TestInfo
+ Migrating @RunWith(Parameterized.class)
to @ParameterizedTest with Argument Sources
+ Migrating assertTrue(val.contains(needle))
to assertThat(val, containsString(needle))
+ Aligning junit versions per recommendations from @sormuras
+ Adjusting parameter order change for assertEquals()
+ Test LifeCycle Annotation Migration
junit 4 | junit 5 / jupiter
------------ | -----------
@Before | @BeforeEach
@After | @AfterEach
@BeforeClass | @BeforeAll
@AfterClass | @AfterAll
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Signed-off-by: olivier lamy <oliver.lamy@gmail.com>
+ Changes needed for new Junit 5
+ Migrating from Vintage junit API to Jupiter junit API
+ Relies on SNAPSHOT jetty-test-helper
- this will be a formal release once this issue has been
resolved satisfactory
+ Have jenkins always pull latest SNAPSHOT for each build
+ Adding jetty.snapshots repository
+ Using surefire 2.22.0 per advice from junit
+ Ensuring <reuseForks>true</reuseForks> to work around issue junit-team/junit5#801
+ Disabling <forkMode>always</forkMode> in maven-surefire-plugin
due to bug https://github.com/junit-team/junit5/issues/801
+ OSGi tests must remain at vintage due to PaxExam
+ Moving from vintage TestingDir to jupiter WorkDir
+ Fixing imports to use jupiter, not vintage
+ Migrating vintage ExpectedException to jupiter assertThrows
+ Migrating vintage TestName to jupiter TestInfo
+ Migrating @RunWith(Parameterized.class)
to @ParameterizedTest with Argument Sources
+ Migrating assertTrue(val.contains(needle))
to assertThat(val, containsString(needle))
+ Aligning junit versions per recommendations from @sormuras
+ Adjusting parameter order change for assertEquals()
+ Test LifeCycle Annotation Migration
junit 4 | junit 5 / jupiter
------------ | -----------
@Before | @BeforeEach
@After | @AfterEach
@BeforeClass | @BeforeAll
@AfterClass | @AfterAll
Signed-off-by: Joakim Erdfelt <joakim.erdfelt@gmail.com>
Signed-off-by: olivier lamy <oliver.lamy@gmail.com>
HttpInput.consume() now checks if the state is already failed,
and if so it does not change it when consuming the input.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now HTTP/2 sessions are not added to the Jetty component tree,
but rather just held by HTTP2SessionContainer that is added to
the Jetty container tree at startup.
HTTP2SessionContainer uses a concurrent Set to hold HTTP/2 sessions
to have good add/remove performance.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now releasing the connection only after the stream has been reset, so
we are sure that the stream has been closed and its count decremented.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
now handling padding over 7 bits with Exception
throwing CompressionExceptions instead of StreamExceptions
fixed issue with non terminal encoded strings
Signed-off-by: lachan-roberts <lachlan@webtide.com>
* Fixes#2672 - Max local stream count exceeded for HttpClient with HTTP/2 transport.
Now waiting for the server preface to arrive to the client before making
the connection available to the ConnectionPool.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Fixes#2616 - Trailers preventing client from processing all the data.
Trailer handling was erroneously firing the response success event
before all the response content events happened.
Now the trailer handling uses a poison-pill DATA frame to make sure that
all response content events happen before the response success event.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
When an entry that is too large for the dynamic table is added, the entire table should be evicted as per the RFC. Previously we were throwing a 431 Bad Message. This will require that the OP should retest #1134.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Moved the initialization of the SETTINGS frame to a common place
shared by both the low-level HTTP2Client and the higher level
HttpClientTransportOverHTTP2.
Did the same with HTTP2 server ConnectionFactories.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #2439 - Remove HTTP/2 data copy.
Implemented reference counting for the network buffer, with the
semantic that calling succeeded() on callbacks decrements the
reference count.
Introduced interface Retainable, used by the client when notifying
multiple application content listeners.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Fixed by start reading from the server after the session
promise has been notified, which may send the first request.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Updated the logic to avoid sending an empty data frame
when only sending the trailers in HttpTransportOverHTTP2.send().
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Fixes#2366 - Review HTTP/2 interleaving.
Now using a fairer algorithm that performs multiple generation passes
but without generating all frames for the current entries - it stops
after N passes over all the entries when the number of bytes to
write is large enough.
Using HttpConfiguration.outputBufferSize as the writeThreshold on
the server and a default 32 KiB on the client to stop the generation
after a pass over all the entries.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Removed usage of ternary expressions in favor of if/else statements
to improve readability of the logic for the send() method.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
The problem was caused by the reset arriving to the server
_before_ the commit callback was invoked.
Now waiting for the commit callback to complete before
sending the reset to the server.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #2349 - Review HTTP/2 max streams enforcement.
Changed the max concurrent remote streams enforcement algorithm.
It is now based on the stream count and the closing stream count,
updated atomically in a state machine in HTTP2Stream.
Fixed Javadoc.
Fixed close() method.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now exiting the parse loop when the response is complete; if there
are bytes remaining in the buffer, then it's cleared out.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Changed the signature of HttpParser.Listener.onBadMessage()
to take a BadMessageException and updated dependent code.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Dumping also HttpChannelOverHTTP2 when dumping the HTTP2Stream.
The channel is now stored as a field in HTTP2Stream rather than
as an attribute to save the attributes map allocation.
Consequently, IStream has now getter/setter for the property
"attachment" that is used to store the channel without having
a type dependency on HttpChannel.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Introduced property connectBlocking in HTTP2Client so that it can
be forwarded by HttpClient and then used for HTTP/2 connects.
Also introduced HTTP2Client.bindAddress, again forwarded from HttpClient.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Issue #2293 Pending Multiplexed Connections
Added a AtomicBiInteger to allow both total connections and pending connections to be encoded in the
same atomic int.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Merging this PR for #2199 as we currently think it is not worse than previous and the code base is definitely simpler. While JMH has shown significant benefits for this approach, we have yet been able to demonstrate them in full scale integration tests - however that is likely due to GC and thread pool issues dominating.
A `org.eclipse.jetty.http2.PEC_MODE` System property has been added to allow the EWYK scheduling to be disabled for HTTP/2 if need be.
* Implementation of #1803 proposal 2 - EITHER dispatch type for EWYK
* made code more readable
* increase small threadpools in tests for extra reserved thread
* clean up
* minor code simplifications
* Work in progress to simplify reserved thread pool
* use a single ReservedThreadPool built into the QueuedThreadPool
* fixed javadoc
* removed more old reserved thread references
* disable EWYK for h2
* fixes from review
* use EWYK for HTTP2
* Fixed javadocs, imports and QueuedThreadPool constructors.
* fix javadoc
* EWYK avoid unnecessary pendings
* after review
* fixed merge with jetty-threadpool.xml
* alternate EWYK implementations
* added jetty copyright headers
* Simplified EWYK code after review
* fixed bad merge
* Code cleanups.
* Improved Javadocs for deprecated property "reservedThreads".
* Improved Javadocs for deprecated property "reservedThreads".
* added a system property to enable only PEC for HTTP/2
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Issue #1803 - Review usage of scheduling strategies
Use a single ReservedThreadExecutor built into the QueuedThreadPool
via new interface TryExecutor.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Scalable scheduler changes for #1918
* Added HttpChannel.destroy to destroy CyclicTimer
* fixed rebase with HttpConnectionOverFCGI
* renamed to acquire
* Destroying the HttpChannel consistently in all transports.
* updated headers
* cleanup after final review
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Removed the distinction between pushed and non-pushed channels; only
non-pushed channels are released and recycled if they're not failed.
Properly resetting HttpReceiverOverHTTP2.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Issue #2081 No idle timeout exception when dispatch is delayed
* Delegate the readtimeout handling to HttpChannel so that a delayed dispatch can be ended.
* Added unit test for delayed dispatch idle
* Now using HttpInput.onIdleTimeout() to fail the HttpInput, and then dispatching the request in case it has not been dispatched yet. This ensure consistent behavior independently of the value of HttpConfiguration.delayDispatchUntilContent.
* Fixed for both HTTP/1.1 and HTTP/2.
* Added tests for non-blocking reads.
Signed-off-by: Greg Wilkins <gregw@webtide.com>
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Code cleanups.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Improved test case handler.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Improved exception message.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #1973 - Implement minimum response data rate.
Implemented response content data rate control in HttpOutput.
Introduced a WriteFlusher.Listener interface that produces events
for every flush(). These events are forwarded to the Connection
and from there to the HttpOutput so that the data rate control can
be enforced.
Both HTTP/1.1 and HTTP/2 are implemented.
Data rate control for HTTP/1.1 is approximate because it will count
also headers bytes and the chunk bytes, while for HTTP/2 is precise.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
* Issue #1973 - Implement minimum response data rate.
Addressed review comments.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Added missing @ManagedObject annotation to AbstractConnectorHttpClientTransport.
Also exported to JMX the "multiplexed" attribute for the FCGI transport
and the "useALPN" attribute for the HTTP/2 transport.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Now waking up the flusher via iterate() after a reset has been received.
This ensures that frames that may have stalled are removed from the
flusher queue.
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
Because now the failures are asynchronous, code that was executed after
invoking the failure listener must be now executed after the
asynchronous processing done by the listener and therefore Callbacks
are introduced.
Idle timeout have a special meaning in that they become a no-operation
if the application is dispatched but idle (neither reading nor writing).
HttpChannelOverHTTP2 now forwards the idle timeout to HttpInput, which
will only change its state if it is interested in reading.
HttpInput.consumeAll() has been modified to consume all input even if
it's already failed.
Failures caused by the other peer (e.g. I/O failures or stream resets)
are now retained and will eagerly consumed any queued data to free up
the flow control windows.
Introduced ConnectionPool.Factory and HttpClientTransport.connectionPoolFactory.
This allows applications to create a ConnectionPool given the HttpDestination.
Introduced ConnectionPool.Factory and HttpClientTransport.connectionPoolFactory.
This allows applications to create a ConnectionPool given the HttpDestination.
Simone Bordet
Chris Walker
康智冬
Greg Wilkins
olivier lamy
Joakim Erdfelt
Jan Bartel
olivier lamy
Jesse McConnell
Kazuhiro Sera
WalkerWatch
lachan-roberts
Matthias Perktold
olivier lamy
stephane martin