Improved documentation about virtual threads. (#8900)
Added programming guide section about Jetty threading model. Updated operations guide with new sections about virtual threads. Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
This commit is contained in:
Simone Bordet
GitHub
parent
44666573e2
commit
e33c9a1284
|
|
@ -19,11 +19,11 @@
|
|||
|
||||
Jetty is an HTTP server and Servlet Container, and supports deployments of web applications.
|
||||
|
||||
The Jetty _server_ listens on one or more network ports using one or more __connector__s.
|
||||
The xref:og-server[Jetty server] listens on one or more network ports using one or more xref:og-protocols[protocol connectors].
|
||||
|
||||
Clients send HTTP requests for specific URIs, such as `+https://host/store/cart+`.
|
||||
|
||||
The HTTP requests arrive to the connectors through the network; the Jetty server processes the requests and, based on their URIs, forwards them to the appropriate web application.
|
||||
The HTTP requests arrive to the connectors through the network; the Jetty server processes the requests and, based on their URIs, forwards them to the appropriate xref:og-deploy[deployed web application].
|
||||
|
||||
[plantuml]
|
||||
----
|
||||
|
|
|
|||
|
|
@ -33,11 +33,5 @@ Clustering::
|
|||
* xref:og-sessions[HTTP Session Caching and Clustering]
|
||||
|
||||
Performance::
|
||||
* xref:og-server-threadpool-virtual[Virtual Threads]
|
||||
* xref:og-quickstart[Faster Web Application Deployment]
|
||||
|
||||
TODO
|
||||
|
||||
* Jetty Overview
|
||||
* Jetty Modules
|
||||
* Rewrite Modules
|
||||
|
||||
|
|
|
|||
|
|
@ -20,5 +20,6 @@
|
|||
* xref:og-protocols-http2s[Configure Secure HTTP/2]
|
||||
* xref:og-protocols-http3[Configure HTTP/3]
|
||||
* xref:og-protocols-proxy[Configure Jetty Behind a Load Balancer or Reverse Proxy]
|
||||
* xref:og-logging[Configure Jetty Logging]
|
||||
* xref:og-server-logging[Configure Jetty Logging]
|
||||
* xref:og-server-threadpool[Configure Jetty Thread Pool and Virtual Threads]
|
||||
* xref:og-troubleshooting[Troubleshooting]
|
||||
|
|
|
|||
|
|
@ -20,12 +20,13 @@ include::../config.adoc[]
|
|||
include::.asciidoctorconfig[]
|
||||
include::introduction.adoc[]
|
||||
include::begin/chapter.adoc[]
|
||||
include::architecture/chapter.adoc[]
|
||||
include::features.adoc[]
|
||||
include::howtos.adoc[]
|
||||
include::architecture/chapter.adoc[]
|
||||
include::start/chapter.adoc[]
|
||||
include::modules/chapter.adoc[]
|
||||
include::deploy/chapter.adoc[]
|
||||
include::server/chapter.adoc[]
|
||||
include::protocols/chapter.adoc[]
|
||||
include::keystore/chapter.adoc[]
|
||||
include::sessions/chapter.adoc[]
|
||||
|
|
@ -36,6 +37,5 @@ include::jndi/chapter.adoc[]
|
|||
include::jaas/chapter.adoc[]
|
||||
include::jaspi/chapter.adoc[]
|
||||
include::jmx/chapter.adoc[]
|
||||
include::logging/chapter.adoc[]
|
||||
include::troubleshooting/chapter.adoc[]
|
||||
include::xml/chapter.adoc[]
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
//
|
||||
|
||||
[[og-jmx]]
|
||||
=== Eclipse Jetty Monitoring & Management
|
||||
=== JMX Monitoring & Management
|
||||
|
||||
Monitoring and management of a Jetty server is important because it allows you to monitor the status of the server (_"Is the server processing requests?"_) and to manage -- i.e. read and possibly change -- its configuration.
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,38 @@
|
|||
//
|
||||
// ========================================================================
|
||||
// Copyright (c) 1995-2022 Mort Bay Consulting Pty Ltd and others.
|
||||
//
|
||||
// This program and the accompanying materials are made available under the
|
||||
// terms of the Eclipse Public License v. 2.0 which is available at
|
||||
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
|
||||
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
|
||||
//
|
||||
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
|
||||
// ========================================================================
|
||||
//
|
||||
|
||||
[[og-module-threadpool-virtual-preview]]
|
||||
===== Module `threadpool-virtual-preview`
|
||||
|
||||
The `threadpool-virtual-preview` module allows you to configure the server-wide thread pool, similarly to what you can do with the xref:og-module-threadpool[`threadpool`] Jetty module, but also specify to use virtual threads, introduced as a preview feature in Java 19.
|
||||
|
||||
NOTE: To enable preview features, this module needs to specify the `+--enable-preview+` command line option using the xref:og-modules-directive-exec[[exec\] directive], and as such it will fork another JVM.
|
||||
|
||||
Refer to the xref:og-module-threadpool[`threadpool`] Jetty module for the general features provided by that Jetty module that also this Jetty module provides.
|
||||
|
||||
The module properties to configure the thread pool are:
|
||||
|
||||
----
|
||||
include::{JETTY_HOME}/modules/threadpool-virtual-preview.mod[tags=documentation]
|
||||
----
|
||||
|
||||
The specific properties to configure virtual threads are:
|
||||
|
||||
`jetty.threadPool.virtual.namePrefix`::
|
||||
The name prefix to use for the virtual thread names.
|
||||
|
||||
`jetty.threadPool.virtual.allowSetThreadLocals`::
|
||||
Whether virtual threads are allowed to set thread locals.
|
||||
|
||||
`jetty.threadPool.useVirtualThreads`::
|
||||
Whether virtual threads inherit the values of `InheritableThreadLocal` variables.
|
||||
|
|
@ -16,19 +16,22 @@
|
|||
|
||||
The `threadpool` module allows you to configure the server-wide thread pool.
|
||||
|
||||
The thread pool creates threads on demand up to `maxThreads`, and idle them out if they are not used.
|
||||
The thread pool creates threads on demand up to `maxThreads`, and idles them out if they are not used.
|
||||
|
||||
Since Jetty uses the thread pool internally to execute critical tasks, it is not recommended to constrain the thread pool to small values of `maxThreads` with the purpose of limiting HTTP request concurrency, as this could very likely cause a server lockup when Jetty needs to run a critical task but there are no threads available.
|
||||
Start with the default value of `maxThreads`, and tune for larger values if needed.
|
||||
|
||||
The module file is `$JETTY_HOME/modules/threadpool.mod`:
|
||||
The module properties to configure the thread pool are:
|
||||
|
||||
----
|
||||
include::{JETTY_HOME}/modules/threadpool.mod[]
|
||||
include::{JETTY_HOME}/modules/threadpool.mod[tags=documentation]
|
||||
----
|
||||
|
||||
Among the configurable properties, the most relevant are:
|
||||
|
||||
`jetty.threadPool.namePrefix`::
|
||||
The name prefix to use for the thread names.
|
||||
|
||||
`jetty.threadPool.detailedDump`::
|
||||
Whether the thread pool should dump the whole stack trace of each thread, or just the topmost stack frame -- defaults to `false`.
|
||||
|
||||
|
|
@ -37,3 +40,5 @@ The time, in milliseconds, after which an idle thread is released from the pool
|
|||
|
||||
`jetty.threadPool.maxThreads`::
|
||||
The max number of threads pooled by the thread pool -- defaults to 200.
|
||||
|
||||
If you want to use virtual threads, introduced as a preview feature in Java 19, use the xref:og-module-threadpool-virtual-preview[`threadpool-virtual-preview`] Jetty module instead (see also the xref:og-server-threadpool[section about configuring the thread pool]).
|
||||
|
|
|
|||
|
|
@ -31,4 +31,5 @@ include::module-ssl.adoc[]
|
|||
include::module-ssl-reload.adoc[]
|
||||
include::module-test-keystore.adoc[]
|
||||
include::module-threadpool.adoc[]
|
||||
include::module-threadpool-virtual-preview.adoc[]
|
||||
include::module-well-known.adoc[]
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
//
|
||||
// ========================================================================
|
||||
// Copyright (c) 1995-2022 Mort Bay Consulting Pty Ltd and others.
|
||||
//
|
||||
// This program and the accompanying materials are made available under the
|
||||
// terms of the Eclipse Public License v. 2.0 which is available at
|
||||
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
|
||||
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
|
||||
//
|
||||
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
|
||||
// ========================================================================
|
||||
//
|
||||
|
||||
[[og-server]]
|
||||
=== Jetty Server
|
||||
|
||||
The Jetty `Server` object is the central component that links protocol connectors to web applications.
|
||||
|
||||
The `Server` component is defined by the xref:og-module-server[`server` Jetty module], that in turn depends on other Jetty modules that provide key functionalities, in particular:
|
||||
|
||||
* xref:og-server-logging[Logging]
|
||||
* xref:og-module-bytebufferpool[`ByteBuffer` pooling]
|
||||
* xref:og-server-threadpool[`Thread` pooling]
|
||||
|
||||
include::server-logging.adoc[]
|
||||
include::server-threadpool.adoc[]
|
||||
|
|
@ -12,7 +12,7 @@
|
|||
//
|
||||
|
||||
[[og-logging-request]]
|
||||
==== Request Logging
|
||||
===== Request Logging
|
||||
|
||||
HTTP requests and responses can be logged to provide data that can be later analyzed with other tools, that can provide information such as the most frequently accessed request URIs, the response status codes, the request/response content lengths, geographical information about the clients, etc.
|
||||
|
||||
|
|
@ -11,8 +11,8 @@
|
|||
// ========================================================================
|
||||
//
|
||||
|
||||
[[og-logging-server]]
|
||||
==== Server Logging
|
||||
[[og-server-logging-server]]
|
||||
===== Server Logging
|
||||
|
||||
The Jetty code uses the link:http://slf4j.org/[SLF4J] API for its logging.
|
||||
|
||||
|
|
@ -24,8 +24,8 @@ The logging of the Jetty server itself is enabled by default with the `logging`
|
|||
|
||||
The `logging` Jetty module is a _virtual_ module (see xref:og-modules-names[this section]) and its default implementation is provided by the `logging-jetty` Jetty module, which uses the Jetty SLF4J binding.
|
||||
|
||||
[[og-logging-server-default]]
|
||||
===== Default Configuration
|
||||
[[og-server-logging-server-default]]
|
||||
====== Default Configuration
|
||||
|
||||
The Jetty SLF4J binding is configured with an appender (`org.eclipse.jetty.logging.StdErrAppender`) that directs the logging to `System.err`, and reads its configuration from a file named `jetty-logging.properties` that must be found in the class-path.
|
||||
|
||||
|
|
@ -93,7 +93,7 @@ The logging levels that you can specify in the `jetty-logging.properties` file a
|
|||
|
||||
When using the Jetty SLF4J binding, the logging levels can be dynamically changed via JMX, see xref:og-troubleshooting-logging[the troubleshooting section] for more information.
|
||||
|
||||
[[og-logging-server-default-rolling]]
|
||||
[[og-server-logging-server-default-rolling]]
|
||||
====== Capturing Logs to a Rolling File
|
||||
|
||||
Having the logging output on `System.err` may be fine at development time, but you typically want the logs to be captured in a file so that they can be looked at even if you don't have a terminal (for example, you started Jetty as a service).
|
||||
|
|
@ -108,8 +108,8 @@ See the xref:og-module-console-capture[`console-capture` module] for more inform
|
|||
The `console-capture` Jetty module should be used only in conjunction with the `logging-jetty` module, as other SLF4J bindings such as LogBack or Log4j2 have their own, more sophisticated, rolling file appenders.
|
||||
====
|
||||
|
||||
[[og-logging-server-custom]]
|
||||
===== Custom Configuration
|
||||
[[og-server-logging-server-custom]]
|
||||
====== Custom Configuration
|
||||
|
||||
You can use a different SLF4J binding if you are more familiar with other logging libraries, or if you need custom logging appenders.
|
||||
There are a number of out-of-the-box Jetty modules that you can use:
|
||||
|
|
@ -120,7 +120,7 @@ There are a number of out-of-the-box Jetty modules that you can use:
|
|||
* `logging-jul`, to use the `java.util.logging` binding
|
||||
* `logging-noop`, to use the SLF4J no-operation binding (discards all logging)
|
||||
|
||||
[[og-logging-server-custom-logback]]
|
||||
[[og-server-logging-server-custom-logback]]
|
||||
====== Logging with LogBack
|
||||
|
||||
You can enable, for example, the `logging-logback` Jetty module in this way (from the `$JETTY_BASE` directory):
|
||||
|
|
@ -148,10 +148,10 @@ $JETTY_BASE
|
|||
As you can see, the Jetty module system downloaded the required LogBack `+*.jar+` files, and created a `$JETTY_BASE/resources/logback.xml` file that you can configure to customize your LogBack logging.
|
||||
Please refer to the link:http://logback.qos.ch/manual/configuration.html[LogBack configuration manual] for more information about how to configure LogBack.
|
||||
|
||||
[[og-logging-server-custom-log4j2]]
|
||||
[[og-server-logging-server-custom-log4j2]]
|
||||
====== Logging with Log4j2
|
||||
|
||||
Similarly to xref:og-logging-server-custom-logback[logging with LogBack], you can enable the `logging-log4j2` Jetty module in this way (from the `$JETTY_BASE` directory):
|
||||
Similarly to xref:og-server-logging-server-custom-logback[logging with LogBack], you can enable the `logging-log4j2` Jetty module in this way (from the `$JETTY_BASE` directory):
|
||||
|
||||
----
|
||||
$ java -jar $JETTY_HOME/start.jar --add-modules=logging-log4j2,http
|
||||
|
|
@ -175,17 +175,17 @@ $JETTY_BASE
|
|||
|
||||
The Jetty module system downloaded the required Log4j2 `+*.jar+` files, and created a `$JETTY_BASE/resources/log4j2.xml` file that you can configure to customize your Log4j2 logging.
|
||||
|
||||
[[og-logging-server-bridges]]
|
||||
===== Bridging Logging to SLF4J
|
||||
[[og-server-logging-server-bridges]]
|
||||
====== Bridging Logging to SLF4J
|
||||
|
||||
When you use libraries that provide the features you need (for example, JDBC drivers), it may be possible that those libraries use a different logging framework than SLF4J.
|
||||
|
||||
SLF4J provides link:http://www.slf4j.org/legacy.html[bridges for legacy logging APIs] that allows you to bridge logging from one of these legacy logging frameworks to SLF4J.
|
||||
Once the logging is bridged to SLF4J, you can use the xref:og-logging-server-default[default configuration] or the xref:og-logging-server-custom[custom configuration] so that your logging is centralized in one place only.
|
||||
Once the logging is bridged to SLF4J, you can use the xref:og-server-logging-server-default[default configuration] or the xref:og-server-logging-server-custom[custom configuration] so that your logging is centralized in one place only.
|
||||
|
||||
Jetty provides out-of-the-box modules that you can enable to bridge logging from other logging frameworks to SLF4J.
|
||||
|
||||
[[og-logging-server-bridge-jul]]
|
||||
[[og-server-logging-server-bridge-jul]]
|
||||
====== Bridging `java.util.logging`
|
||||
|
||||
For libraries that use `java.util.logging` as their logging framework you can enable the `logging-jul-capture` Jetty module.
|
||||
|
|
@ -11,13 +11,13 @@
|
|||
// ========================================================================
|
||||
//
|
||||
|
||||
[[og-logging]]
|
||||
=== Logging
|
||||
[[og-server-logging]]
|
||||
==== Logging
|
||||
|
||||
There are two types of logging that can be configured in Jetty:
|
||||
|
||||
* The logging of Jetty itself, that logs the server activity
|
||||
* The HTTP request logging, that logs information about HTTP requests and responses processed by Jetty
|
||||
|
||||
include::logging-server.adoc[]
|
||||
include::logging-request.adoc[]
|
||||
include::server-logging-server.adoc[]
|
||||
include::server-logging-request.adoc[]
|
||||
|
|
@ -0,0 +1,79 @@
|
|||
//
|
||||
// ========================================================================
|
||||
// Copyright (c) 1995-2022 Mort Bay Consulting Pty Ltd and others.
|
||||
//
|
||||
// This program and the accompanying materials are made available under the
|
||||
// terms of the Eclipse Public License v. 2.0 which is available at
|
||||
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
|
||||
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
|
||||
//
|
||||
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
|
||||
// ========================================================================
|
||||
//
|
||||
|
||||
[[og-server-threadpool]]
|
||||
==== Thread Pooling
|
||||
|
||||
Jetty uses thread pooling to efficiently execute tasks that provide Jetty functionalities.
|
||||
|
||||
Like any other component, the Jetty thread pool is configured and enabled via the xref:og-module-threadpool[`threadpool` Jetty module], that is transitively enabled by the xref:og-module-server[`server` Jetty module] which, in turn, is transitively enabled by a protocol module such as the xref:og-protocols-http[`http` Jetty module]:
|
||||
|
||||
----
|
||||
$ java -jar $JETTY_HOME/start.jar --add-modules=http
|
||||
----
|
||||
|
||||
The command above gives you the default configuration for the thread pool.
|
||||
|
||||
If you want to explicitly configure the thread pool, it is enough to explicitly specify the xref:og-module-threadpool[`threadpool`] module:
|
||||
|
||||
----
|
||||
$ java -jar $JETTY_HOME/start.jar --add-modules=threadpool,http
|
||||
----
|
||||
|
||||
After the command above, the `$JETTY_BASE` directory looks like this:
|
||||
|
||||
[source,subs=verbatim]
|
||||
----
|
||||
$JETTY_BASE
|
||||
├── resources
|
||||
│ └── jetty-logging.properties
|
||||
└── start.d
|
||||
├── http.ini
|
||||
└── threadpool.ini
|
||||
----
|
||||
|
||||
Now you can customize the `threadpool.ini` file to explicitly configure the thread pool.
|
||||
|
||||
[[og-server-threadpool-virtual]]
|
||||
===== Virtual Threads Support
|
||||
|
||||
Virtual threads have been introduced as a preview feature in Java 19.
|
||||
|
||||
The xref:og-module-threadpool-virtual-preview[`threadpool-virtual-preview`] Jetty module provides support for virtual threads and it is mutually exclusive with the `threadpool` Jetty module.
|
||||
|
||||
If you have already enabled the `threadpool` Jetty module, it is sufficient to remove it by removing the `$JETTY_BASE/start.d/threadpool.ini` file.
|
||||
|
||||
Then, you can enable the xref:og-module-threadpool-virtual-preview[`threadpool-virtual-preview`] module:
|
||||
|
||||
----
|
||||
$ java -jar $JETTY_HOME/start.jar --add-modules=threadpool-virtual-preview,http
|
||||
----
|
||||
|
||||
After the command above, the `$JETTY_BASE` directory looks like this:
|
||||
|
||||
[source,subs=verbatim]
|
||||
----
|
||||
$JETTY_BASE
|
||||
├── resources
|
||||
│ └── jetty-logging.properties
|
||||
└── start.d
|
||||
├── http.ini
|
||||
└── threadpool-virtual-preview.ini
|
||||
----
|
||||
|
||||
Now you can customize the `threadpool-virtual-preview.ini` file to explicitly configure the thread pool and the virtual threads and then start Jetty:
|
||||
|
||||
[source,subs=quotes,options=nowrap]
|
||||
----
|
||||
include::jetty[setupArgs="--add-modules=threadpool-virtual-preview,http"]
|
||||
----
|
||||
|
|
@ -63,7 +63,7 @@ The `session-store-gcloud` module provides GCloud support for storing session da
|
|||
Because the Google Cloud DataStore is not a technology provided by the Eclipse Foundation, when enabling the module you will be prompted to assent to the licenses of the external vendor.
|
||||
|
||||
As GCloud requires certain Java Commons Logging features to work correctly, Jetty routes these through SLF4J.
|
||||
By default Jetty implements the SLF4J api, but you can choose a different logging implementation by following the instructions xref:og-logging[here]
|
||||
By default Jetty implements the SLF4J api, but you can choose a different logging implementation by following the instructions xref:og-server-logging[here]
|
||||
|
||||
IMPORTANT: If you want to use updated versions of the jar files automatically downloaded during the module enablement, you can place them in the associated `$JETTY_BASE/lib/` directory and use the `--skip-file-validation=<module name>` command line option to prevent errors when starting your server.
|
||||
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ This is necessary to avoid that the Jetty start mechanism depend on logging libr
|
|||
[NOTE]
|
||||
====
|
||||
This section is about the logging performed by the Jetty start mechanism _before_ it configures and starts Jetty.
|
||||
See the xref:og-logging[logging section] for information about logging when Jetty starts.
|
||||
See the xref:og-server-logging[logging section] for information about logging when Jetty starts.
|
||||
====
|
||||
|
||||
You can enable DEBUG level logging with the `--debug` command line option, for both the _tool_ and _start_ modes:
|
||||
|
|
|
|||
|
|
@ -170,6 +170,7 @@ This is useful if you need to start Jetty and want to specify JVM options such a
|
|||
* `+-Xlog:gc+`, to specify the GC log file and options
|
||||
* `+-javaagent+`, to specify Java agents
|
||||
* `+-XX:+` options, for example to specify the GC implementation
|
||||
* `+--enable-preview+`, to enable Java preview features
|
||||
|
||||
Start by creating `$JETTY_BASE/modules/jvm.mod`:
|
||||
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@
|
|||
|
||||
Enabling DEBUG level logging for the `org.eclipse.jetty` logger name provides the maximum amount of information to troubleshoot Jetty issues.
|
||||
|
||||
Refer to the xref:og-logging[logging section] for more information about how to configure logging in Jetty.
|
||||
Refer to the xref:og-server-logging[logging section] for more information about how to configure logging in Jetty.
|
||||
|
||||
[CAUTION]
|
||||
====
|
||||
|
|
|
|||
|
|
@ -0,0 +1,217 @@
|
|||
//
|
||||
// ========================================================================
|
||||
// Copyright (c) 1995-2022 Mort Bay Consulting Pty Ltd and others.
|
||||
//
|
||||
// This program and the accompanying materials are made available under the
|
||||
// terms of the Eclipse Public License v. 2.0 which is available at
|
||||
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
|
||||
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
|
||||
//
|
||||
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
|
||||
// ========================================================================
|
||||
//
|
||||
|
||||
[[pg-arch-threads]]
|
||||
=== Jetty Threading Architecture
|
||||
|
||||
Writing a performant client or server is difficult, because it should:
|
||||
|
||||
* Scale well with the number of processors.
|
||||
* Be efficient at using processor caches to avoid link:https://en.wikipedia.org/wiki/Parallel_slowdown[parallel slowdown].
|
||||
* Support multiple network protocols that may have very different requirements; for example, multiplexed protocols such as HTTP/2 introduce new challenges that are not present in non-multiplexed protocols such as HTTP/1.1.
|
||||
* Support different application threading models; for example, if a Jetty server invokes server-side application code that is allowed to call blocking APIs, then the Jetty server should not be affected by how long the blocking API call takes, and should be able to process other connections or other requests in a timely fashion.
|
||||
|
||||
[[pg-arch-threads-execution-strategy]]
|
||||
==== Execution Strategies
|
||||
|
||||
The Jetty threading architecture can be modeled with a producer/consumer pattern, where produced tasks needs to be consumed efficiently.
|
||||
|
||||
For example, Jetty produces (among others) these tasks:
|
||||
|
||||
* A task that wraps a NIO selection event, see the xref:pg-arch-io[Jetty I/O architecture].
|
||||
* A task that wraps the invocation of application code that may block (for example, the invocation of a Servlet to handle an HTTP request).
|
||||
|
||||
A task is typically a `Runnable` object that may implement `org.eclipse.jetty.util.thread.Invocable` to indicate the behavior of the task (in particular, whether the task may block or not).
|
||||
|
||||
Once a task has been produced, it may be consumed using these modes:
|
||||
|
||||
* xref:pg-arch-threads-execution-strategy-pc[`Produce-Consume`]
|
||||
* xref:pg-arch-threads-execution-strategy-pec[`Produce-Execute-Consume`]
|
||||
* xref:pg-arch-threads-execution-strategy-epc[`Execute-Produce-Consume`]
|
||||
|
||||
[[pg-arch-threads-execution-strategy-pc]]
|
||||
===== Produce-Consume
|
||||
In the `Produce-Consume` mode, the producer thread loops to produce a task that is run directly by the `Producer Thread`.
|
||||
|
||||
[plantuml]
|
||||
----
|
||||
skinparam backgroundColor transparent
|
||||
|
||||
compact concise "Producer Thread" as PT
|
||||
hide time-axis
|
||||
|
||||
@PT
|
||||
0 is T1 #lightgreen
|
||||
1 is "Run T1" #dodgerblue
|
||||
5 is T2 #lightgreen
|
||||
6 is "Run T2" #dodgerblue
|
||||
8 is T3 #lightgreen
|
||||
9 is "Run T3" #dodgerblue
|
||||
12 is {hidden}
|
||||
----
|
||||
|
||||
If the task is a NIO selection event, then this mode is the thread-per-selector mode which is very CPU core cache efficient, but suffers from the link:http://en.wikipedia.org/wiki/Head-of-line_blocking[head-of-line blocking]: if one of the tasks blocks or runs slowly, then subsequent tasks cannot be produced (and therefore cannot be consumed either) and will pay in latency the cost of running previous, possibly unrelated, tasks.
|
||||
|
||||
This mode should only be used if the produced task is known to never block, or if the system tolerates well (or does not care about) head-of-line blocking.
|
||||
|
||||
[[pg-arch-threads-execution-strategy-pec]]
|
||||
===== Produce-Execute-Consume
|
||||
In the `Produce-Execute-Consume` mode, the `Producer Thread` loops to produce tasks that are submitted to a `java.util.concurrent.Executor` to be run by ``Worker Thread``s different from the `Producer Thread`.
|
||||
|
||||
[plantuml]
|
||||
----
|
||||
skinparam backgroundColor transparent
|
||||
|
||||
compact concise "Producer Thread" as PT
|
||||
compact concise "Worker Thread 1" as WT1
|
||||
compact concise "Worker Thread 2" as WT2
|
||||
compact concise "Worker Thread 3" as WT3
|
||||
hide time-axis
|
||||
|
||||
@PT
|
||||
0 is T1 #lightgreen
|
||||
1 is T2 #lightgreen
|
||||
2 is T3 #lightgreen
|
||||
3 is T4 #lightgreen
|
||||
4 is {hidden}
|
||||
|
||||
@WT1
|
||||
1 is "Run T1" #dodgerblue
|
||||
5 is {hidden}
|
||||
|
||||
@WT2
|
||||
2 is "Run T2" #dodgerblue
|
||||
4 is "Run T4" #dodgerblue
|
||||
8 is {hidden}
|
||||
|
||||
@WT3
|
||||
3 is "Run T3" #dodgerblue
|
||||
6 is {hidden}
|
||||
----
|
||||
|
||||
The `Executor` implementation typically adds the task to a queue, and dequeues the task when there is a worker thread available to run it.
|
||||
|
||||
This mode solves the head-of-line blocking discussed in the xref:pg-arch-threads-execution-strategy-pc[`Produce-Consume` section], but suffers from other issues:
|
||||
|
||||
* It is not CPU core cache efficient, as the data available to the producer thread will need to be accessed by another thread that likely is going to run on a CPU core that will not have that data in its caches.
|
||||
* If the tasks take time to be run, the `Executor` queue may grow indefinitely.
|
||||
* A small latency is added to every task: the time it waits in the `Executor` queue.
|
||||
|
||||
[[pg-arch-threads-execution-strategy-epc]]
|
||||
===== Execute-Produce-Consume
|
||||
In the `Execute-Produce-Consume` mode, the producer thread `Thread 1` loops to produce a task, then submits one internal task to an `Executor` to take over production on thread `Thread 2`, and then runs the task in `Thread 1`, and so on.
|
||||
|
||||
[plantuml]
|
||||
----
|
||||
skinparam backgroundColor transparent
|
||||
|
||||
compact concise "Thread 1" as WT1
|
||||
compact concise "Thread 2" as WT2
|
||||
compact concise "Thread 3" as WT3
|
||||
compact concise "Thread 4" as WT4
|
||||
hide time-axis
|
||||
|
||||
@WT1
|
||||
0 is T1 #lightgreen
|
||||
1 is "Run T1" #dodgerblue
|
||||
5 is {hidden}
|
||||
|
||||
@WT2
|
||||
1 is T2 #lightgreen
|
||||
2 is "Run T2" #dodgerblue
|
||||
4 is T5 #lightgreen
|
||||
5 is "Run T5" #dodgerblue
|
||||
10 is {hidden}
|
||||
|
||||
@WT3
|
||||
2 is T3 #lightgreen
|
||||
3 is "Run T3" #dodgerblue
|
||||
6 is {hidden}
|
||||
|
||||
@WT4
|
||||
3 is T4 #lightgreen
|
||||
4 is "Run T4" #dodgerblue
|
||||
8 is {hidden}
|
||||
|
||||
----
|
||||
|
||||
This mode may operate like xref:pg-arch-threads-execution-strategy-pc[`Produce-Consume`] when the take over production task run, for example, by thread `Thread 3` takes time to be executed (for example, in a busy server): then thread `Thread 2` will produce one task and run it, then produce another task and run it, etc. -- `Thread 2` behaves exactly like the `Produce-Consume` mode.
|
||||
By the time thread `Thread 3` takes over task production from `Thread 2`, all the work might already be done.
|
||||
|
||||
This mode may also operate similarly to xref:pg-arch-threads-execution-strategy-pec[`Produce-Execute-Consume`] when the take over production task always finds a free CPU core immediately (for example, in a mostly idle server): thread `Thread 1` will produce a task, yield production to `Thread 2` while `Thread 1` is running the task; `Thread 2` will produce a task, yield production to `Thread 3` while `Thread 2` is running the task, etc.
|
||||
|
||||
Differently from `Produce-Execute-Consume`, here production happens on different threads, but the advantage is that the task is run by the same thread that produced it (which is CPU core cache efficient).
|
||||
|
||||
[[pg-arch-threads-execution-strategy-adaptive]]
|
||||
===== Adaptive Execution Strategy
|
||||
The modes of task consumption discussed above are captured by the `org.eclipse.jetty.util.thread.ExecutionStrategy` interface, with an additional implementation that also takes into account the behavior of the task when the task implements `Invocable`.
|
||||
|
||||
For example, a task that declares itself as non-blocking can be consumed using the `Produce-Consume` mode, since there is no risk to stop production because the task will not block.
|
||||
|
||||
Conversely, a task that declares itself as blocking will stop production, and therefore must be consumed using either the `Produce-Execute-Consume` mode or the `Execute-Produce-Consume` mode.
|
||||
Deciding between these two modes depends on whether there is a free thread immediately available to take over production, and this is captured by the `org.eclipse.jetty.util.thread.TryExecutor` interface.
|
||||
|
||||
An implementation of `TryExecutor` can be asked whether a thread can be immediately and exclusively allocated to run a task, as opposed to a normal `Executor` that can only queue the task in the expectation that there will be a thread available in the near future to run the task.
|
||||
|
||||
The concept of task consumption modes, coupled with `Invocable` tasks that expose their own behavior, coupled with a `TryExecutor` that guarantees whether production can be immediately taken over are captured by the default Jetty execution strategy, named `org.eclipse.jetty.util.thread.AdaptiveExecutionStrategy`.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
`AdaptiveExecutionStrategy` was previously named `EatWhatYouKill`, named after a hunting proverb in the sense that one should produce (kill) only what it consumes (eats).
|
||||
====
|
||||
|
||||
[[pg-arch-threads-thread-pool]]
|
||||
==== Thread Pool
|
||||
Jetty's xref:pg-arch-threads[threading architecture] requires a more sophisticated thread pool than what offered by Java's `java.util.concurrent.ExecutorService`.
|
||||
|
||||
Jetty's default thread pool implementation is link:{javadoc-url}/org/eclipse/jetty/util/thread/QueuedThreadPool.html[`QueuedThreadPool`].
|
||||
|
||||
`QueuedThreadPool` integrates with the xref:pg-arch-bean[Jetty component model], implements `Executor`, provides a `TryExecutor` implementation (discussed in the xref:pg-arch-threads-execution-strategy-adaptive[adaptive execution strategy section]), and supports xref:pg-arch-threads-thread-pool-virtual-threads[virtual threads] (introduced as a preview feature in Java 19).
|
||||
|
||||
`QueuedThreadPool` can be configured with a `maxThreads` value.
|
||||
|
||||
However, some of the Jetty components (such as the xref:pg-arch-io-selector-manager[selectors]) permanently steal threads for their internal use, or rather `QueuedThreadPool` leases some threads to these components.
|
||||
These threads are reported by `QueuedThreadPool.leasedThreads` and are not available to run application code.
|
||||
|
||||
`QueuedThreadPool` can be configured with a `reservedThreads` value.
|
||||
This value represents the maximum number of threads that can be reserved and used by the `TryExecutor` implementation.
|
||||
A negative value for `QueuedThreadPool.reservedThreads` means that the actual value will be heuristically derived from the number of CPU cores and `QueuedThreadPool.maxThreads`.
|
||||
A value of zero for `QueuedThreadPool.reservedThreads` means that reserved threads are disabled, and therefore the xref:pg-arch-threads-execution-strategy-epc[`Execute-Produce-Consume` mode] is never used -- the xref:pg-arch-threads-execution-strategy-pec[`Produce-Execute-Consume` mode] is always used instead.
|
||||
|
||||
[[pg-arch-threads-thread-pool-virtual-threads]]
|
||||
===== Virtual Threads
|
||||
Virtual threads have been introduced in Java 19 as a preview feature.
|
||||
|
||||
NOTE: In Java versions where virtual threads are a preview feature, remember to add `+--enable-preview+` to the command line options to use virtual threads.
|
||||
|
||||
`QueuedThreadPool` can be configured to use virtual threads by specifying the virtual threads `Executor`:
|
||||
|
||||
[source,java]
|
||||
----
|
||||
QueuedThreadPool threadPool = new QueuedThreadPool();
|
||||
threadPool.setVirtualThreadsExecutor(Executors.newVirtualThreadPerTaskExecutor());
|
||||
----
|
||||
|
||||
[CAUTION]
|
||||
====
|
||||
Jetty cannot enforce that the `Executor` passed to `setVirtualThreadsExecutor(Executor)` uses virtual threads, so make sure to specify a _virtual_ threads `Executor` and not a normal `Executor` that uses platform threads.
|
||||
====
|
||||
|
||||
`AdaptiveExecutionStrategy` makes use of this setting when it determines that a task should be run with the xref:pg-arch-threads-execution-strategy-pec[`Produce-Execute-Consume` mode]: rather than submitting the task to `QueuedThreadPool` to be run in a platform thread, it submits the task to the virtual threads `Executor`.
|
||||
|
||||
[NOTE]
|
||||
====
|
||||
Enabling virtual threads in `QueuedThreadPool` will default the number of reserved threads to zero, unless the number of reserved threads is explicitly configured to a positive value.
|
||||
|
||||
Defaulting the number of reserved threads to zero ensures that the xref:pg-arch-threads-execution-strategy-pec[Produce-Execute-Consume mode] is always used, which means that virtual threads will always be used for blocking tasks.
|
||||
====
|
||||
|
|
@ -16,6 +16,7 @@
|
|||
== Jetty Architecture
|
||||
|
||||
include::arch-bean.adoc[]
|
||||
include::arch-threads.adoc[]
|
||||
include::arch-io.adoc[]
|
||||
include::arch-listener.adoc[]
|
||||
include::arch-jmx.adoc[]
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@
|
|||
<Set name="minThreads" type="int"><Property name="jetty.threadPool.minThreads" deprecated="threads.min" default="10"/></Set>
|
||||
<Set name="maxThreads" type="int"><Property name="jetty.threadPool.maxThreads" deprecated="threads.max" default="200"/></Set>
|
||||
<Set name="reservedThreads" type="int"><Property name="jetty.threadPool.reservedThreads" default="-1"/></Set>
|
||||
<Set name="useVirtualThreads" type="boolean"><Property deprecated="jetty.threadPool.useVirtualThreads" default="false"/></Set>
|
||||
<Set name="useVirtualThreads" property="jetty.threadPool.useVirtualThreads" />
|
||||
<Set name="idleTimeout" type="int"><Property name="jetty.threadPool.idleTimeout" deprecated="threads.timeout" default="60000"/></Set>
|
||||
<Set name="detailedDump" type="boolean"><Property name="jetty.threadPool.detailedDump" default="false"/></Set>
|
||||
</New>
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ threadpool
|
|||
etc/jetty-threadpool-virtual-preview.xml
|
||||
|
||||
[ini-template]
|
||||
# tag::documentation[]
|
||||
## Platform threads name prefix.
|
||||
#jetty.threadPool.namePrefix=qtp<hashCode>
|
||||
|
||||
|
|
@ -39,4 +40,5 @@ etc/jetty-threadpool-virtual-preview.xml
|
|||
#jetty.threadPool.virtual.allowSetThreadLocals=true
|
||||
|
||||
## Whether virtual threads inherits the values of inheritable thread locals.
|
||||
#jetty.threadPool.virtual.allowSetThreadLocals=true
|
||||
#jetty.threadPool.virtual.inheritInheritableThreadLocals=true
|
||||
# end::documentation[]
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ threadpool|default
|
|||
etc/jetty-threadpool.xml
|
||||
|
||||
[ini-template]
|
||||
# tag::documentation[]
|
||||
## Thread name prefix.
|
||||
#jetty.threadPool.namePrefix=qtp<hashCode>
|
||||
|
||||
|
|
@ -32,3 +33,4 @@ etc/jetty-threadpool.xml
|
|||
|
||||
## Whether to output a detailed dump.
|
||||
#jetty.threadPool.detailedDump=false
|
||||
# end::documentation[]
|
||||
|
|
|
|||
Loading…
Reference in New Issue