Merged branch 'jetty-10.0.x' into 'jetty-11.0.x'.

Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
This commit is contained in:
Simone Bordet 2021-01-11 19:05:04 +01:00
commit c79cdec600
27 changed files with 327 additions and 2179 deletions

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 431 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 329 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 441 KiB

View File

@ -21,4 +21,3 @@ include::default-logging-with-stderrlog.adoc[]
include::configuring-jetty-request-logs.adoc[]
include::configuring-logging-modules.adoc[]
include::example-logback-centralized-logging.adoc[]
include::jetty-server-dump.adoc[]

View File

@ -1,211 +0,0 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[jetty-server-dump]]
=== Jetty Server Dump
The dump feature in Jetty provides a snapshot of the bean containment tree of the main Jetty components together with a summary of their configuration.This includes threadpool, connectors, contexts, webapplications, servlets and so forth.
[[configuring-dump-feature]]
==== Configuring the Jetty Server Dump
You can request that Jetty do a dump immediately after starting and just before stopping by calling the appropriate setters on the `Server` instance.
This functionality can be enabled using two properties to control this behaviour which are referenced in `jetty.xml` to call these setters.
```java
jetty.server.dumpAfterStart=true
jetty.server.dumpBeforeStop=true
```
The server dump can be temporarily enabled by supplying either of these properties as command line arguments,
or they can be enabled via the `server.ini` file (see xref:quick-start-configure[]).
``` shell
$ java -jar $JETTY_HOME/start.jar jetty.server.dumpAfterStart=true jetty.server.dumpBeforeStop=true
```
[[extra-threadpool-info]]
==== Extra ThreadPool Information
To get maximum detail from the `QueuedThreadPool` in the dump, you need to enable a detailed dump on any instances of `QueuedThreadPool` you are using.
This extra detail in the detailed dump consists of full stack traces for every running thread, and a list of queued jobs waiting to be run.
This can be enabled using the `threadpool` module and configuring the `jetty.threadPool.detailedDump` property.
See xref:startup-modules[] for information on how to enable a module.
This same property can also be set via the command line the same as the server dump property.
[[dump-tool-via-jmx]]
==== Using the Dump Feature via JMX
The `dump` method is on the Server instance and many of its nested components (Handlers, Connectors, and so forth).
Dumps may be obtained by calling these methods either in code or via JMX (see xref:using-jmx[]).
The Server MBean has a `dump()` method, which dumps everything, plus a `dumpStdErr()` operation that dumps to StdErr rather than replying to JConsole.
[[examing-jetty-distro-dump]]
==== Explanation of the Dump Key
- `+- bean` is a java POJO that is contained by the parent object as a bean added with the addBean method.
- `+= managed` is a bean that is also a LifeCycle that is started and stopped with the parent object.
- `+~ unmanaged` is a bean that is also a LifeCycle that is started and stopped with the parent object. It is typically shared with other objects (hence its children are not dumped).
- `+? auto` is a bean that has been added to an unstarted parent. If it is a LifeCycle that is not started when the parent is started, then it is started and becomes a managed bean, otherwise it becomes either unmanaged or just a bean.
- `+: iterable` is an object that is contained within an iterable field of the parent (eg a list, set etc).
- `+] array` is an object that is contained within an array field of the parent.
- `+@ map` is an object that is contained within an map field of the parent.
- `+> undefined` is an object that is contained within the parent by an undefined relationship.
==== Jetty Server Dump Example
This is a dump of the OneServletContext embedded example with extra threadpool information:
....
Server@59906517{STARTED}[9.4.32-SNAPSHOT] - STARTED
+= QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}] - STARTED
| += ReservedThreadExecutor@16267862{s=2/4,p=0} - STARTED
| +> threads size=8
| | +> qtp488044861-13 RUNNABLE tid=13 prio=5 SELECTING
| | +> qtp488044861-15-acceptor-0@296e0338-ServerConnector@1e6d1014{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} RUNNABLE tid=15 prio=3 ACCEPTING
| | +> qtp488044861-17 TIMED_WAITING tid=17 prio=5 RESERVED
| | +> qtp488044861-19 RUNNABLE tid=19 prio=5
| | | +> app//org.eclipse.jetty.http.pathmap.PathMappings.getMatch(PathMappings.java:130)
| | | +> app//org.eclipse.jetty.servlet.ServletHandler.getMappedServlet(ServletHandler.java:591)
| | | +> app//org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:474)
| | | +> app//org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)
| | | +> app//org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
| | | +> app//org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
| | | +> app//org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
| | | +> app//org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
| | | +> app//org.eclipse.jetty.server.Server.handle(Server.java:516)
| | | +> app//org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
| | | +> app//org.eclipse.jetty.server.HttpChannel$$Lambda$102/0x000000010016d440.dispatch(Unknown Source)
| | | +> app//org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
| | | +> app//org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
| | | +> app//org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
| | | +> app//org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
| | | +> app//org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
| | | +> app//org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
| | | +> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
| | | +> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
| | | +> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
| | | +> app//org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
| | | +> app//org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
| | | +> app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
| | | +> app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
| | | +> java.base@11.0.5/java.lang.Thread.run(Thread.java:834)
| | +> qtp488044861-16 TIMED_WAITING tid=16 prio=5 RESERVED
| | +> qtp488044861-21 RUNNABLE tid=21 prio=5 SELECTING
| | +> qtp488044861-18 TIMED_WAITING tid=18 prio=5 IDLE
| | +> qtp488044861-14 TIMED_WAITING tid=14 prio=5 IDLE
| +> jobs size=0
+= ServerConnector@1e6d1014{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} - STARTED
| +~ Server@59906517{STARTED}[9.4.32-SNAPSHOT] - STARTED
| +~ QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}] - STARTED
| += ScheduledExecutorScheduler@453da22c{STARTED} - STARTED
| | +> java.base@11.0.5/jdk.internal.misc.Unsafe.park(Native Method)
| | +> java.base@11.0.5/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
| | +> java.base@11.0.5/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
| | +> java.base@11.0.5/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
| | +> java.base@11.0.5/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
| | +> java.base@11.0.5/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
| | +> java.base@11.0.5/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
| | +> java.base@11.0.5/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
| | +> java.base@11.0.5/java.lang.Thread.run(Thread.java:834)
| +- org.eclipse.jetty.io.ArrayByteBufferPool@71248c21
| += HttpConnectionFactory@3fd7a715[HTTP/1.1] - STARTED
| | +- HttpConfiguration@442675e1{32768/8192,8192/8192,https://:0,[]}
| | +> customizers size=0
| | +> formEncodedMethods size=2
| | | +> POST
| | | +> PUT
| | +> outputBufferSize=32768
| | +> outputAggregationSize=8192
| | +> requestHeaderSize=8192
| | +> responseHeaderSize=8192
| | +> headerCacheSize=1024
| | +> secureScheme=https
| | +> securePort=0
| | +> idleTimeout=-1
| | +> blockingTimeout=-1
| | +> sendDateHeader=true
| | +> sendServerVersion=true
| | +> sendXPoweredBy=false
| | +> delayDispatchUntilContent=true
| | +> persistentConnectionsEnabled=true
| | +> maxErrorDispatches=10
| | +> minRequestDataRate=0
| | +> minResponseDataRate=0
| | +> cookieCompliance=RFC6265
| | +> setRequestCookieCompliance=RFC6265
| | +> notifyRemoteAsyncErrors=true
| | +> relativeRedirectAllowed=false
| += SelectorManager@ServerConnector@1e6d1014{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} - STARTED
| | += ManagedSelector@38364841{STARTED} id=0 keys=0 selected=0 updates=0 - STARTED
| | | += EatWhatYouKill@28c4711c/SelectorProducer@59717824/PRODUCING/p=false/QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}][pc=0,pic=0,pec=0,epc=0]@2020-09-04T10:57:20.077669+10:00 - STARTED
| | | | +- SelectorProducer@59717824
| | | | +~ QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}] - STARTED
| | | +> updates @ 2020-09-04T10:57:20.058489+10:00 size=0
| | | +> keys @ 2020-09-04T10:57:20.061714+10:00 size=0
| | += ManagedSelector@146044d7{STARTED} id=1 keys=1 selected=0 updates=0 - STARTED
| | += EatWhatYouKill@1e9e725a/SelectorProducer@15d9bc04/PRODUCING/p=false/QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}][pc=0,pic=0,pec=1,epc=14]@2020-09-04T10:57:20.082696+10:00 - STARTED
| | | +- SelectorProducer@15d9bc04
| | | +~ QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}] - STARTED
| | +> updates @ 2020-09-04T10:57:20.078661+10:00 size=0
| | +> keys @ 2020-09-04T10:57:20.082035+10:00 size=1
| | +> SelectionKey@74bb45ed{i=0}->SocketChannelEndPoint@569ef11f{l=/127.0.0.1:8080,r=/127.0.0.1:58702,OPEN,fill=-,flush=-,to=3/30000}{io=0/0,kio=0,kro=1}->HttpConnection@25b03990[p=HttpParser{s=CONTENT,0 of -1},g=HttpGenerator@218fb9fe{s=START}]=>HttpChannelOverHttp@648d33ab{s=HttpChannelState@717b7e16{s=HANDLING rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=54,c=false/false,a=HANDLING,uri=//localhost:8080/,age=4}
| +- sun.nio.ch.ServerSocketChannelImpl[/0:0:0:0:0:0:0:0:8080]
| +- qtp488044861-15-acceptor-0@296e0338-ServerConnector@1e6d1014{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
+= AttributeContainerMap@473b46c3{size=0} - STARTED
+= o.e.j.s.ServletContextHandler@3ffcd140{/,file:///tmp/,AVAILABLE} - STARTED
| += org.eclipse.jetty.server.session.SessionHandler1089504328==dftMaxIdleSec=-1 - STARTED
| | += ServletHandler@516be40f{STARTED} - STARTED
| | | +> listeners ServletHandler@516be40f{STARTED} size=2
| | | | +> ListenerHolder@3c0a50da{STARTED}: org.eclipse.jetty.demos.OneServletContext$InitListener - STARTED
| | | | +> ListenerHolder@646be2c3{STARTED}: org.eclipse.jetty.demos.OneServletContext$RequestListener - STARTED
| | | +> filters ServletHandler@516be40f{STARTED} size=2
| | | | +> org.eclipse.jetty.demos.OneServletContext$TestFilter-e874448@e874448==org.eclipse.jetty.demos.OneServletContext$TestFilter,inst=true,async=true - STARTED
| | | | | +> org.eclipse.jetty.demos.OneServletContext$TestFilter@797badd3
| | | | +> org.eclipse.jetty.demos.OneServletContext$TestFilter-60285225@60285225==org.eclipse.jetty.demos.OneServletContext$TestFilter,inst=true,async=true - STARTED
| | | | +> org.eclipse.jetty.demos.OneServletContext$TestFilter@77be656f
| | | +> filterMappings ServletHandler@516be40f{STARTED} size=2
| | | | +> [/test/*]/[]/[REQUEST]=>org.eclipse.jetty.demos.OneServletContext$TestFilter-e874448
| | | | +> [*.test]/[]/[ASYNC, REQUEST]=>org.eclipse.jetty.demos.OneServletContext$TestFilter-60285225
| | | +> servlets ServletHandler@516be40f{STARTED} size=3
| | | | +> org.eclipse.jetty.demos.HelloServlet-58c1670b@d20bf05b==org.eclipse.jetty.demos.HelloServlet,jsp=null,order=-1,inst=false,async=true - STARTED
| | | | | +> class org.eclipse.jetty.demos.HelloServlet
| | | | +> debug@5b09653==org.eclipse.jetty.demos.DumpServlet,jsp=null,order=-1,inst=false,async=true - STARTED
| | | | | +> class org.eclipse.jetty.demos.DumpServlet
| | | | +> org.eclipse.jetty.servlet.DefaultServlet-6b9651f3@8eb381d1==org.eclipse.jetty.servlet.DefaultServlet,jsp=null,order=-1,inst=true,async=true - STARTED
| | | | +> org.eclipse.jetty.servlet.DefaultServlet@78a2da20
| | | +> servletMappings ServletHandler@516be40f{STARTED} size=4
| | | +> [/hello/*]=>org.eclipse.jetty.demos.HelloServlet-58c1670b
| | | +> [/dump/*]=>debug
| | | +> [*.dump]=>debug
| | | +> [/]=>org.eclipse.jetty.servlet.DefaultServlet-6b9651f3
| | += org.eclipse.jetty.server.session.DefaultSessionCache@dd3b207[evict=-1,removeUnloadable=false,saveOnCreate=false,saveOnInactiveEvict=false] - STARTED
| | | += org.eclipse.jetty.server.session.NullSessionDataStore@551bdc27[passivating=false,graceSec=3600] - STARTED
| | +~ DefaultSessionIdManager@58fdd99{STARTED}[worker=node0] - STARTED
| +> No ClassLoader
| +> eventListeners o.e.j.s.ServletContextHandler@3ffcd140{/,file:///tmp/,AVAILABLE} size=2
| | +> org.eclipse.jetty.demos.OneServletContext$InitListener@6b1274d2
| | +> org.eclipse.jetty.demos.OneServletContext$RequestListener@7bc1a03d
| +> handler attributes o.e.j.s.ServletContextHandler@3ffcd140{/,file:///tmp/,AVAILABLE} size=1
| | +> org.eclipse.jetty.server.Executor=QueuedThreadPool[qtp488044861]@1d16f93d{STARTED,8<=8<=200,i=2,r=4,q=0}[ReservedThreadExecutor@16267862{s=2/4,p=0}]
| +> context attributes o.e.j.s.ServletContextHandler@3ffcd140{/,file:///tmp/,AVAILABLE} size=2
| | +> org.eclipse.jetty.util.DecoratedObjectFactory=org.eclipse.jetty.util.DecoratedObjectFactory[decorators=1]
| | +> X-Init=true
| +> initparams o.e.j.s.ServletContextHandler@3ffcd140{/,file:///tmp/,AVAILABLE} size=0
+= ErrorHandler@ba8d91c{STARTED} - STARTED
+= DefaultSessionIdManager@58fdd99{STARTED}[worker=node0] - STARTED
| += HouseKeeper@60438a68{STARTED}[interval=660000, ownscheduler=true] - STARTED
+> jdk.internal.loader.ClassLoaders$AppClassLoader@2c13da15
+> jdk.internal.loader.ClassLoaders$PlatformClassLoader@7364985f
key: +- bean, += managed, +~ unmanaged, +? auto, +: iterable, +] array, +@ map, +> undefined
....

View File

@ -33,6 +33,6 @@ include::annotations/chapter.adoc[]
include::jsp/chapter.adoc[]
include::jndi/chapter.adoc[]
include::jaas/chapter.adoc[]
include::websocket/chapter.adoc[]
include::jmx/chapter.adoc[]
include::logging/chapter.adoc[]
include::troubleshooting/chapter.adoc[]

View File

@ -16,7 +16,7 @@
The Eclipse Jetty Operations Guide targets sysops, devops, and developers who want to install Eclipse Jetty as a standalone server to deploy web applications.
=== Introduction
=== Getting Started
If you are new to Eclipse Jetty, read xref:og-begin[here] to download, install, start and deploy web applications to Jetty.
@ -55,4 +55,5 @@ TODO
* xref:og-protocols-http2c[Configure Clear-Text HTTP/2]
* xref:og-protocols-http2s[Configure Secure HTTP/2]
* xref:og-protocols-proxy[Configure Jetty Behind a Load Balancer or Reverse Proxy]
* xref:og-logging[Configure Jetty Logging]
* xref:og-troubleshooting[Troubleshooting]

View File

@ -0,0 +1,17 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[og-logging]]
=== Eclipse Jetty Logging
TODO

View File

@ -15,8 +15,15 @@
==== Module `bytebufferpool`
The `bytebufferpool` module allows you to configure the server-wide `ByteBuffer` pool.
Pooling ``ByteBuffer``s results in less memory usage and less pressure on the Garbage Collector.
// TODO: expand
``ByteBuffer``s are pooled in _buckets_; each bucket as a capacity that is a multiple of a capacity factor that you can configure.
For example, if a request for a `ByteBuffer` of capacity 2000 is requested, and the capacity factor is 1024, then the pool will allocate a buffer from the second bucket, of capacity 2048 (1024 * 2).
Applications that need to sustain many concurrent requests -- or load spikes -- may require many buffers during peak load. These buffers will remain pooled once the system transitions to a lighter load (or becomes idle), and it may be undesirable to retain a lot of memory for an idle system.
It is possible to configure the max heap memory and the max direct memory that the pool retains.
Excess buffers will not be pooled and will be eventually garbage collected.
The module file is `$JETTY_HOME/modules/bytebufferpool.mod`:
@ -26,4 +33,8 @@ include::{JETTY_HOME}/modules/bytebufferpool.mod[]
Among the configurable properties, the most relevant are:
TODO
`jetty.byteBufferPool.maxHeapMemory`::
This property allows you to cap the max heap memory retained by the pool.
`jetty.byteBufferPool.maxDirectMemory`::
This property allows you to cap the max direct memory retained by the pool.

View File

@ -26,6 +26,8 @@ include::{JETTY_HOME}/modules/deploy.mod[]
Among the configurable properties, the most relevant are:
* `jetty.deploy.monitoredDir`, to change the name of the monitored directory.
* `jetty.deploy.scanInterval`, to change the scan period, that is how frequently the `DeploymentManager` wakes up to scan the monitored directory for changes.
`jetty.deploy.monitoredDir`::
The name of the monitored directory.
`jetty.deploy.scanInterval`::
The scan period in seconds, that is how frequently the `DeploymentManager` wakes up to scan the monitored directory for changes.
Setting `jetty.deploy.scanInterval=0` disabled _hot_ deployment so that only static deployment will be possible (see also xref:og-deploy-hot-static[here] for more information).

View File

@ -24,10 +24,14 @@ include::{JETTY_HOME}/modules/http.mod[tags=documentation]
Among the configurable properties, the most relevant are:
* `jetty.http.port`, default `8080`, is the network port that Jetty listens to for clear-text HTTP/1.1 connections.
* `jetty.http.idleTimeout`, default `30` seconds, is the amount of time a connection can be idle (i.e. no bytes received and no bytes sent) until the server decides to close it to save resources.
* `jetty.http.acceptors`, default -1 (i.e. an accept heuristic decides the value based on the number of cores), is the number of threads that compete to accept connections.
* `jetty.http.selectors`, default -1 (i.e. a select heuristic decides the value based on the number of cores), is the number of NIO selectors (with an associated thread) that manage connections.
`jetty.http.port`::
The network port that Jetty listens to for clear-text HTTP/1.1 connections -- default `8080`.
`jetty.http.idleTimeout`::
The amount of time a connection can be idle (i.e. no bytes received and no bytes sent) until the server decides to close it to save resources -- default `30` seconds.
`jetty.http.acceptors`::
The number of threads that compete to accept connections -- default -1 (i.e. an accept heuristic decides the value based on the number of cores).
`jetty.http.selectors`::
The number of NIO selectors (with an associated thread) that manage connections -- default -1 (i.e. a select heuristic decides the value based on the number of cores).
[[og-module-http-acceptors]]
===== Configuration of Acceptors

View File

@ -26,6 +26,14 @@ include::{JETTY_HOME}/modules/server.mod[]
Among the configurable properties, the most relevant are:
TODO
// TODO: consider extracting the httpConfig and scheduler properties into separate files.
`jetty.server.dumpAfterStart`::
Whether to perform a `Server.dump()` operation after the `Server` has started.
The output of the dump operation is sent to `System.err`.
See also the xref:og-troubleshooting-dump[Jetty Server Dump] section for more information.
`jetty.server.dumpBeforeStop`::
Whether to perform a `Server.dump()` operation before the `Server` stops.
The output of the dump operation is sent to `System.err`.
See also the xref:og-troubleshooting-dump[Jetty Server Dump] section for more information.

View File

@ -28,14 +28,14 @@ include::{JETTY_HOME}/modules/ssl.mod[tags=documentation-connector]
Among the configurable properties, the most relevant are:
`jetty.ssl.port`::
the network port that Jetty listens to for secure connections -- default `8443`.
The network port that Jetty listens to for secure connections -- default `8443`.
`jetty.http.idleTimeout`::
the amount of time a connection can be idle (i.e. no bytes received and no bytes sent) until the server decides to close it to save resources -- default `30000` milliseconds.
The amount of time a connection can be idle (i.e. no bytes received and no bytes sent) until the server decides to close it to save resources -- default `30000` milliseconds.
`jetty.http.acceptors`::
the number of threads that compete to accept connections -- default -1 (i.e. an accept heuristic decides the value based on the number of cores).
The number of threads that compete to accept connections -- default -1 (i.e. an accept heuristic decides the value based on the number of cores).
Refer to xref:og-module-http-acceptors[this section] for more information about acceptor threads.
`jetty.http.selectors`::
the number of NIO selectors (with an associated thread) that manage connections -- default -1 (i.e. a select heuristic decides the value based on the number of cores).
The number of NIO selectors (with an associated thread) that manage connections -- default -1 (i.e. a select heuristic decides the value based on the number of cores).
Refer to xref:og-module-http-selectors[this section] for more information about selector threads.
The module properties to configure the KeyStore and TLS parameters are:
@ -49,19 +49,17 @@ include::{JETTY_HOME}/modules/ssl.mod[tags=documentation-ssl-context]
Among the configurable properties, the most relevant are:
jetty.sslContext.keyStorePath::
the KeyStore path on the file system.
If it is a relative path, it is relative to `$JETTY_BASE`.
Defaults to `$JETTY_BASE/etc/keystore.p12`.
`jetty.sslContext.keyStorePath`::
The KeyStore path on the file system relative to `$JETTY_BASE` -- defaults to `$JETTY_BASE/etc/keystore.p12`.
`jetty.sslContext.keyStorePassword`::
the KeyStore password, which you want to explicitly configure.
The KeyStore password, which you want to explicitly configure.
The password may be obfuscated with the xref:og-password[Jetty Password Tool].
If you need to configure client certificate authentication, you want to configure one of these properties (they are mutually exclusive):
`jetty.sslContext.needClientAuth`::
whether client certificate authentication should be required.
Whether client certificate authentication should be required.
`jetty.sslContext.wantClientAuth`::
whether client certificate authentication should be requested.
Whether client certificate authentication should be requested.
If you configure client certificate authentication, you need to configure and distribute a client KeyStore as explained in xref:og-keystore-client-authn[this section].

View File

@ -16,7 +16,10 @@
The `threadpool` module allows you to configure the server-wide thread pool.
// TODO: thread pool per connector should be documented here?
The thread pool creates threads on demand up to `maxThreads`, and idle them out if they are not used.
Since Jetty uses the thread pool internally to execute critical tasks, it is not recommended to constrain the thread pool to small values of `maxThreads` with the purpose of limiting HTTP request concurrency, as this could very likely cause a server lockup when Jetty needs to run a critical task but there are no threads available.
Start with the default value of `maxThreads`, and tune for larger values if needed.
The module file is `$JETTY_HOME/modules/threadpool.mod`:
@ -26,4 +29,11 @@ include::{JETTY_HOME}/modules/threadpool.mod[]
Among the configurable properties, the most relevant are:
TODO
`jetty.threadPool.detailedDump`::
Whether the thread pool should dump the whole stack trace of each thread, or just the topmost stack frame -- defaults to `false`.
`jetty.threadPool.idleTimeout`::
The time, in milliseconds, after which an idle thread is released from the pool -- defaults to 60000, i.e. 60 seconds.
`jetty.threadPool.maxThreads`::
The max number of threads pooled by the thread pool -- defaults to 200.

View File

@ -14,5 +14,24 @@
[[og-troubleshooting]]
=== Eclipse Jetty Troubleshooting
// TODO: see old docs under troubleshooting/*
TODO
To troubleshoot Jetty when used as a production server, there are two main tools: the Jetty Server Dump and enabling DEBUG level logging.
Jetty is based on components organized as a tree, with the `Server` instance at the root of the tree.
As explained in the xref:og-jmx[JMX section], these components can be exported as JMX MBeans and therefore be accessible from JMX Consoles such as Java Missions Control (JMC).
Being able to take a snapshot of the state of Jetty while it is running is the most useful information that can be attached when reporting an issue.
Such state includes:
* The thread pool configuration and its current state, including how many threads are in use, and their stack trace.
* The TLS configuration.
* The I/O configuration and its current state, including the ports Jetty listens to, how many connections are currently open, and he state of each connection, and the state of the request/response handling for each connection.
* The `Handler` structure and its configuration.
* The web applications deployed and their configurations, including the class loader information.
The prerequisite for troubleshooting is to enable JMX, so that Jetty -- possibly a production server -- can be accessed from a remote location to obtain the information exported via JMX, and possibly be able to reconfigure Jetty to solve the issue.
IMPORTANT: Make sure you read about how to secure the access to Jetty when using xref:og-jmx-remote[remote JMX].
include::troubleshooting-dump.adoc[]
include::troubleshooting-logging.adoc[]

Binary file not shown.

After

Width:  |  Height:  |  Size: 262 KiB

View File

@ -0,0 +1,139 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[og-troubleshooting-dump]]
==== Troubleshooting: Jetty Server Dump
The Jetty Server Dump is obtained by invoking, via JMX, the `Server.dump()` operation, as shown below.
image::jmc-server-dump.png[]
Find the `Server` MBean in the MBean Tree, under `org.eclipse.jetty.server:type=server,id=0`.
Then click on the "Operations" tab, select the `dump()` operation, and then click the `Execute` button.
In the bottom panel you will see the result of the invocation, that you can copy into a text editor and save to your file system.
[CAUTION]
====
Taking a Jetty Server Dump is a relatively expensive operation, as it dumps the state of all connections (which can be thousands), and the state of all threads.
The result of the invocation may produce a large string, possibly few MiB, that may impact the server memory usage.
Furthermore, dumping the state of the I/O Jetty components takes a little CPU time off the handling of the actual I/O, possibly slowing it down temporarily.
While the slow-down caused by taking the Jetty Server Dump may be noticeable on highly loaded systems, it is typically a very small price to pay to obtain the information about the Jetty state that may be critical to the resolution of an issue.
====
[NOTE]
====
The format of the Jetty Server Dump output is subject to change at any time, as Jetty developers modify the Jetty code and decide to include more state, or remove state that is no longer relevant.
The Jetty Server Dump is organized in a tree whose structure is similar to the runtime Jetty component tree.
At the end of the dump output there is a legend that explains the type of tree node: whether it is a node that represent a _managed_ component, or an _array_ node (or a _map_ node) that represent some component state, etc.
====
[[og-troubleshooting-dump-start-stop]]
===== Jetty Server Dump at Server Start/Stop
The `Server.dump()` operation may also be invoked just after the `Server` starts (to log the state of the freshly started server), and just before the `Server` stops (which may be useful to log the state of server that is not working properly).
You can temporarily enable the Jetty Server Dump at start time by overriding the `jetty.server.dumpAfterStart` property on the command line:
[source,subs=quotes]
----
$ java -jar $JETTY_HOME/start.jar *jetty.server.dumpAfterStart=true*
----
To make this change persistent across server restarts, see the xref:og-module-server[`server` module] configuration for more information about how to configure the server to dump at start/stop time.
[[og-troubleshooting-dump-detailed]]
===== Detailed ThreadPool Information
By default, the dump of the thread pool will only dump the topmost stack frame of each thread.
It is possible to configure the thread pool to dump the whole stack trace for each thread; while this may be a little more expensive, it provides complete information about the state of each thread, which may be important to diagnose the issue.
See the xref:og-module-threadpool[`threadpool` module] configuration for more information about how to configure the thread pool to dump detailed thread information.
Detailed thread pool information can also be turned on/off on-the-fly via JMX, by finding the `ThreadPool` MBean under `org.eclipse.jetty.util.thread:type=queuedthreadpool,id=0`, then selecting the `detailedDump` attribute and setting it to `true`. You can now perform the `Server.dump()` operation as explained above, and then set `detailedDump` back to `false`.
[[og-troubleshooting-dump-example]]
===== Jetty Server Dump Example
Below you can find a simple example of a Jetty Server Dump, with annotations for the principal components:
[source,subs=verbatim,role=small,options=nowrap]
----
Server@3ee0fea4{STARTING}[10.0.0,sto=5000] - STARTED <1>
+= QueuedThreadPool[qtp1924582348]@72b6cbcc{STARTED,4<=4<=200,i=2,r=12,q=0}[ReservedThreadExecutor@64cd705f{s=0/12,p=0}] - STARTED <2>
| += ReservedThreadExecutor@64cd705f{s=0/12,p=0} - STARTED
| +> threads size=10
| +> qtp1924582348-14-acceptor-0@65a7a043-ServerConnector@5c909414{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} RUNNABLE tid=14 prio=3 ACCEPTING <3>
| +> qtp1924582348-13 RUNNABLE tid=13 prio=5 SELECTING <4>
| +> qtp1924582348-15 TIMED_WAITING tid=15 prio=5 IDLE
| +> qtp1924582348-16 TIMED_WAITING tid=16 prio=5 IDLE
+- org.eclipse.jetty.io.ArrayByteBufferPool@6a28ffa4
+= ScheduledExecutorScheduler@31f924f5{STARTED} - STARTED
+= HandlerList@48ae9b55{STARTED} - STARTED <5>
| += ContextHandlerCollection@1700915{STARTED} - STARTED
| += DefaultHandler@21de60b4{STARTED} - STARTED
+= ServerConnector@5c909414{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} - STARTED <6>
| +~ QueuedThreadPool[qtp1924582348]@72b6cbcc{STARTED,10<=10<=200,i=8,r=12,q=0}[ReservedThreadExecutor@64cd705f{s=0/12,p=0}] - STARTED
| +~ ScheduledExecutorScheduler@31f924f5{STARTED} - STARTED
| +- org.eclipse.jetty.io.ArrayByteBufferPool@6a28ffa4
| +- org.eclipse.jetty.server.AbstractConnector$1@c267ef4
| += HttpConnectionFactory@29ba4338[HTTP/1.1] - STARTED
| | +- HttpConfiguration@1139b2f3{32768/8192,8192/8192,https://:0,[]}
| += SelectorManager@ServerConnector@5c909414{HTTP/1.1, (http/1.1)}{0.0.0.0:8080} - STARTED
| | += ManagedSelector@636be97c{STARTED} id=0 keys=0 selected=0 updates=0 - STARTED <7>
| | += EatWhatYouKill@50a638b5/SelectorProducer@1817d444/PRODUCING/p=false/QueuedThreadPool[qtp1924582348]@72b6cbcc{STARTED,10<=10<=200,i=8,r=12,q=0}[ReservedThreadExecutor@64cd705f{s=0/12,p=0}][pc=0,pic=0,pec=0,epc=0]@2021-01-11T13:08:00.333168316+01:00 - STARTED
| | | +- SelectorProducer@1817d444
| | | +~ QueuedThreadPool[qtp1924582348]@72b6cbcc{STARTED,10<=10<=200,i=8,r=12,q=0}[ReservedThreadExecutor@64cd705f{s=0/12,p=0}] - STARTED
| | +> updates @ 2021-01-11T13:08:00.331898257+01:00 size=0
| | +> keys @ 2021-01-11T13:08:00.332436637+01:00 size=1 <8>
| | +> SelectionKey@306e0284{i=1}->SocketChannelEndPoint@330eef1{l=/[0:0:0:0:0:0:0:1]:8080,r=/[0:0:0:0:0:0:0:1]:36402,OPEN,fill=FI,flush=-,to=17713/30000}{io=1/1,kio=1,kro=1}->HttpConnection@198220f9[p=HttpParser{s=START,0 of -1},g=HttpGenerator@7f14183e{s=START}]=>HttpChannelOverHttp@61ec57fb{s=HttpChannelState@2ebfced8{s=IDLE rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=2,c=false/false,a=IDLE,uri=null,age=0}
| +- sun.nio.ch.ServerSocketChannelImpl[/[0:0:0:0:0:0:0:0]:8080]
| +- qtp1924582348-14-acceptor-0@65a7a043-ServerConnector@5c909414{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
+= ErrorHandler@6ca8564a{STARTED} - STARTED
+> startJarLoader@41975e01 <9>
+> URLs size=9
| +> file:/tmp/jetty.base/resources/
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/logging/slf4j-api-2.0.0-alpha1.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/logging/jetty-slf4j-impl-10.0.1-SNAPSHOT.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/jetty-servlet-api-4.0.5.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/jetty-http-10.0.1-SNAPSHOT.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/jetty-server-10.0.1-SNAPSHOT.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/jetty-xml-10.0.1-SNAPSHOT.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/jetty-util-10.0.1-SNAPSHOT.jar
| +> file:/home/simon/opensource/jetty/jetty10.0/jetty-home/target/jetty-home/lib/jetty-io-10.0.1-SNAPSHOT.jar
+> jdk.internal.loader.ClassLoaders$AppClassLoader@5bc2b487
+> packages size=4
| +> package org.eclipse.jetty.start.config
| +> package org.eclipse.jetty.start.builders
| +> package org.eclipse.jetty.start.shaded.util
| +> package org.eclipse.jetty.start
+> jdk.internal.loader.ClassLoaders$PlatformClassLoader@3d51f06e
+> packages size=2
+> package sun.util.resources.provider
+> package sun.util.resources.cldr.provider
key: +- bean, += managed, +~ unmanaged, +? auto, +: iterable, +] array, +@ map, +> undefined <10>
----
<1> The `Server` instance at the root of the tree
<2> The thread pool component
<3> The thread accepting connections
<4> The thread selecting connections
<5> The root of the `Handler` structure
<6> The connector listening on port `8080` for the HTTP/1.1 protocol
<7> A selector component that manages connections
<8> The connections currently managed by the selector component
<9> The server `ClassLoader` and its classpath
<10> The legend for the dump nodes

View File

@ -0,0 +1,67 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[og-troubleshooting-logging]]
==== Troubleshooting: DEBUG Logging
Enabling DEBUG level logging for the `org.eclipse.jetty` logger name provides the maximum amount of information to troubleshoot Jetty issues.
Refer to the xref:og-logging[logging section] for more information about how to configure logging in Jetty.
[CAUTION]
====
Enabling DEBUG level logging for `org.eclipse.jetty` is very, *very* expensive.
Your server could be slowed down to almost a halt, especially if it is under heavy load.
Furthermore, the log file could quickly fill up the entire filesystem (unless configured to roll over), so you want to be really careful using DEBUG logging.
For production servers, consider using the xref:og-troubleshooting-dump[Jetty Server Dump] first, and enable DEBUG logging only as a last resort.
====
However, sometimes issues are such that only DEBUG logging can really tell what's going on in the system, and enabling DEBUG logging is your best chance to figure the issue out.
Below you can find few suggestions that can help you reduce the impact when you have to enable DEBUG logging.
[[og-troubleshooting-logging-backend]]
===== Jetty Behind a Load Balancer
If Jetty instances are behind a load balancer, you may configure the load balancer to send less load to a particular Jetty instance, and enable DEBUG logging in that instance only.
[[og-troubleshooting-logging-jmx]]
===== Enabling DEBUG Logging for a Short Time
In certain cases the issue can be reproduced reliably, but only in the production environment.
You can use JMX to temporarily enable DEBUG logging, reproduce the issue, and then disable DEBUG logging.
Alternatively, if you cannot reliably reproduce the issue, but you _know_ it is happening, you can temporarily enable DEBUG logging for a small period of time, let's say 10-60 seconds, and then disable DEBUG logging.
Changing the log level at runtime is a feature of the logging implementation that you are using.
The Jetty SLF4J implementation, used by default, exposes via JMX method `boolean JettyLoggerFactoryMBean.setLoggerLevel(String loggerName, String levelName)` that you can invoke via a JMX console to change the level for the specified logger name.
The method returns `true` if the logger level was successfully changed.
For example, you can pass the string `org.eclipse.jetty` as the first parameter, and the string `DEBUG` (upper case) as the second parameter.
You can then use the string `INFO` or `WARN` (upper case) to restore the logging level to its previous value.
[[og-troubleshooting-logging-subpackages]]
===== Enabling DEBUG Logging for SubPackages
Enabling DEBUG logging for the `org.eclipse.jetty` logger name implies that all children logger names, recursively, inherit the DEBUG level.
Processing a single HTTP request involves many Jetty components: the I/O subsystem (under `org.eclipse.jetty.io`), the thread pool (under `org.eclipse.jetty.util`), the HTTP/1.1 parsing (under `org.eclipse.jetty.http`), etc.
If you can cut the amount of DEBUG logging to just what you need to troubleshoot the issue, the impact of enabling DEBUG logging will be much less than enabling it for all Jetty components.
For example, if you need to troubleshoot a client that sends bad HTTP/1.1 requests, it may be enough to enable only the `org.eclipse.jetty.http` logger name, therefore saving the large amount of DEBUG logging produced by the I/O subsystem and by the thread pool.
In another case, you may need to troubleshoot only HTTP/2 requests, and therefore enabling only the `org.eclipse.jetty.http2` logger name could be enough.

View File

@ -286,7 +286,7 @@ You can only specify the key value via the `name` attribute, so the key can only
[[og-xml-syntax-array]]
===== `<Array>` and `<Item>`
Element `<Array>` creates a new array, whose component type may be specified by the `type` attribute.
Element `<Array>` creates a new array, whose component type may be specified by the `type` attribute, or by a `Type` child element.
[source,xml]
----

View File

@ -1,5 +1,3 @@
# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html
[description]
Configures the ByteBufferPool used by ServerConnectors.
@ -10,21 +8,24 @@ logging
etc/jetty-bytebufferpool.xml
[ini-template]
### Server ByteBufferPool Configuration
## Minimum capacity to pool ByteBuffers
## Minimum capacity of a single ByteBuffer.
#jetty.byteBufferPool.minCapacity=0
## Maximum capacity to pool ByteBuffers
## Maximum capacity of a single ByteBuffer.
## Requests for ByteBuffers larger than this value results
## in the ByteBuffer being allocated but not pooled.
#jetty.byteBufferPool.maxCapacity=65536
## Capacity factor
## Bucket capacity factor.
## ByteBuffers are allocated out of buckets that have
## a capacity that is multiple of this factor.
#jetty.byteBufferPool.factor=1024
## Maximum queue length for each bucket (-1 for unbounded)
## Maximum queue length for each bucket (-1 for unbounded).
#jetty.byteBufferPool.maxQueueLength=-1
## Maximum heap memory retainable by the pool (-1 for unlimited)
## Maximum heap memory retainable by the pool (-1 for unlimited).
#jetty.byteBufferPool.maxHeapMemory=-1
## Maximum direct memory retainable by the pool (-1 for unlimited)
## Maximum direct memory retainable by the pool (-1 for unlimited).
#jetty.byteBufferPool.maxDirectMemory=-1

View File

@ -1,7 +1,7 @@
# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html
[description]
Enables the core Jetty server on the classpath.
Enables and configures the Jetty server.
This module does not enable any network protocol support.
To enable a specific network protocol such as HTTP/1.1, you must enable the correspondent Jetty module.
[optional]
jvm

View File

@ -1,7 +1,5 @@
# DO NOT EDIT - See: https://www.eclipse.org/jetty/documentation/current/startup-modules.html
[description]
Enables and configures the Server thread pool.
Enables and configures the Server ThreadPool.
[depends]
logging
@ -10,19 +8,17 @@ logging
etc/jetty-threadpool.xml
[ini-template]
### Server Thread Pool Configuration
## Minimum Number of Threads
## Minimum number of pooled threads.
#jetty.threadPool.minThreads=10
## Maximum Number of Threads
## Maximum number of pooled threads.
#jetty.threadPool.maxThreads=200
## Number of reserved threads (-1 for heuristic)
## Number of reserved threads (-1 for heuristic).
#jetty.threadPool.reservedThreads=-1
## Thread Idle Timeout (in milliseconds)
## Thread idle timeout (in milliseconds).
#jetty.threadPool.idleTimeout=60000
## Whether to Output a Detailed Dump
## Whether to output a detailed dump.
#jetty.threadPool.detailedDump=false

View File

@ -100,7 +100,8 @@
<Call name="callStatic" class="org.eclipse.jetty.xml.TestConfiguration"/>
<Call name="call">
<Arg><Array type="java.lang.Object">
<Arg><Array>
<Type>java.lang.Object</Type>
<Item>Blah</Item>
<Item type="String">1.2.3.4:5678</Item>
<Item><New class="java.lang.Double"><Arg>1.2345</Arg></New></Item>
@ -109,7 +110,8 @@
</Call>
<Call name="call">
<Arg><Array type="int">
<Arg><Array>
<Type>int</Type>
<Item type="int">1</Item>
<Item type="int">2</Item>
<Item type="int">3</Item>