Fixes #7369 - Document CustomRequestLog

Added documentation to the programming guide.
Removed related old documentation.

Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
This commit is contained in:
Simone Bordet 2022-01-12 17:12:01 +01:00
parent 251649f0a9
commit 51123c7cd1
No known key found for this signature in database
GPG Key ID: 1677D141BCF3584D
6 changed files with 101 additions and 144 deletions

View File

@ -168,7 +168,6 @@
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-servlets</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>

View File

@ -18,6 +18,5 @@ This chapter discusses various options for configuring logging.
include::configuring-jetty-logging.adoc[]
include::default-logging-with-stderrlog.adoc[]
include::configuring-jetty-request-logs.adoc[]
include::configuring-logging-modules.adoc[]
include::example-logback-centralized-logging.adoc[]

View File

@ -1,140 +0,0 @@
//
// ========================================================================
// Copyright (c) 1995-2021 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under the
// terms of the Eclipse Public License v. 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
// which is available at https://www.apache.org/licenses/LICENSE-2.0.
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[configuring-jetty-request-logs]]
=== Configuring Jetty Request Logs
Request logs are a record of the requests that the server has processed.
There is one entry per request received, and commonly in the standard NCSA format, so you can use tools like http://en.wikipedia.org/wiki/Webalizer[Webalizer] to analyze them conveniently.
[[constructing-request-log-entry]]
==== Constructing a Request Log Entry
A standard request log entry includes the client IP address, date, method, URL, result, size, referrer, user agent and latency.
For example:
....
123.4.5.6 - - [20/Jul/2016:10:16:17 +0000]
"GET /jetty/tut/XmlConfiguration.html HTTP/1.1"
200 76793 "http://localhost:8080/jetty/tut/logging.html"
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8" 342
....
[[implementing-request-log]]
==== Implementing a Request Log
Jetty provides an implementation called `NCSARequestLog` which supports the NCSA format in files that will roll over on a daily basis.
The http://logback.qos.ch/[Logback Project] offers http://logback.qos.ch/access.html[another implementation] of a `RequestLog` interface, providing rich and powerful HTTP-access log functionality.
If neither of these options meets your needs, you can implement a custom request logger by implementing Jetty's link:{JDURL}/org/eclipse/jetty/server/RequestLog.html[`RequestLog.java`] interface and plugging it in similar to the `NCSARequestLog`, as shown below.
[[configuring-request-log]]
==== Configuring the Request Log module
To enable the Request Log module for the entire server via the Jetty distribution, it first needs to be enabled on the command line:
[source,screen,subs="{sub-order}"]
----
[my-base]$ java -jar /path/to/jetty-home/start.jar --add-to-start=requestlog
INFO: requestlog initialised in ${jetty.base}/start.d/requestlog.ini
MKDIR: ${jetty.base}/logs
INFO: Base directory was modified
----
The above command will add a new `requestlog.ini` file to your link:#start-vs-startd[`{$jetty.base}/start.d` directory].
____
[NOTE]
By default, request logs are not set to be appended, meaning a the log file is wiped clean upon sever restart.
You can change this setting by editing the `requestlog.ini` and un-commenting the line that reads `jetty.requestlog.append=true`.
____
The equivalent code for embedded usages of Jetty is:
[source, java, subs="{sub-order}"]
----
NCSARequestLog requestLog = new NCSARequestLog("/var/logs/jetty/jetty-yyyy_mm_dd.request.log");
requestLog.setAppend(true);
requestLog.setExtended(false);
requestLog.setLogTimeZone("GMT");
requestLog.setLogLatency(true);
requestLog.setRetainDays("90");
server.setRequestLog(requestLog);
----
This configures a request log in `{$jetty.home}/logs` with filenames including the date.
Existing log files are appended to and the extended NCSA format is used in the GMT time zone.
The above configuration enables Log Latency, which is the amount of time it took the server to handle the request.
This value is measured in milliseconds and is appended to the the log file for each request.
You can also customize the number of days you wish to keep request logs.
By default, log files are kept for 90 days before being deleted.
The value for `retainDays` (xml) or `setRetainDays` (Java) should be configured as _1 + n_ days.
For example, if you wanted to keep the logs for the current day and the day prior you would set the `retainDays` (or `setRetainDays`) value to 2.
[[request-log-custom-writer]]
==== Introducing RequestLog.Writer
The concept of a `RequestLog.Writer`, introduced in Jetty 9.4.15, manages the writing to a log the string generated by the `RequestLog`.
This allows the `CustomRequestLog` to match the functionality of other `RequestLogger` implementations by plugging in any `RequestLog.Writer` and a custom format string.
Jetty currently has implementations of `RequestLog.Writer`, `RequestLogWriter`, `AsyncRequestLogWriter`, and `Slf4jRequestLogWriter`.
So, the way to create an asynchronous `RequestLog` using the extended NCSA format has been changed from:
`new AsyncNcsaRequestLog(filename)`
to:
`new CustomRequestLog(new AsyncRequestLogWriter(filename), CustomRequestLog.EXTENDED_NCSA_FORMAT)`
Additionally, there are now two settings for the log timezone to be configured.
There is the configuration for logging the request time, which is set in the `timeZone` parameter in the `%t` format code of the string, given in the format `%{format|timeZone|locale}t`.
The other `timeZone` parameter relates to the generation of the log file name (both at creation and roll over).
This is configured in the `requestlog` module file, or can be used as a setter on `RequestLogWriter` via XML.
Both timezones are set to GMT by default.
[[configuring-separate-request-log-for-web-application]]
==== Configuring a Separate Request Log For a Web Application
To configure a separate request log for specific a web application, add the following to the context XML file.
[source, xml, subs="{sub-order}"]
----
<Configure class="org.eclipse.jetty.webapp.WebAppContext">
...
<Call name="insertHandler">
<Arg>
<New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler">
<Set name="requestLog">
<New id="RequestLogImpl" class="org.eclipse.jetty.server.NCSARequestLog">
<Set name="filename"><Property name="jetty.logs" default="./logs"/>/test-yyyy_mm_dd.request.log</Set>
<Set name="filenameDateFormat">yyyy_MM_dd</Set>
<Set name="LogTimeZone">GMT</Set>
<Set name="retainDays">90</Set>
<Set name="append">true</Set>
<Set name="LogLatency">true</Set>
</New>
</Set>
</New>
</Arg>
</Call>
...
</Configure>
----

View File

@ -24,7 +24,7 @@ An `org.eclipse.jetty.server.Handler` is the component that processes incoming H
* nested, where one `Handler` invokes the next, nested, `Handler`
** `HandlerWrapper` implements this behavior
The `HandlerCollection` behavior (invoking _all_ handlers) is useful when for example the last `Handler` is a logging `Handler` that logs the request(that may have been modified by previous handlers).
The `HandlerCollection` behavior (invoking _all_ handlers) is useful when for example the last `Handler` is a logging `Handler` that logs the request (that may have been modified by previous handlers).
The `HandlerList` behavior (invoking handlers up to the first that calls `Request.setHandled(true)`) is useful when each handler processes a different URIs or a different virtual hosts: ``Handler``s are invoked one after the other until one matches the URI or virtual host.

View File

@ -71,7 +71,7 @@ Read xref:pg-server-http-handler[this section] for further details about ``Handl
If you are interested in writing your server application based on the Servlet APIs, jump to xref:pg-server-http-handler-use-servlet[this section].
[[pg-server-http-request-processing]]
==== Server Request Processing
==== Request Processing
The Jetty HTTP request processing is outlined below in the diagram below.
You may want to refer to the xref:pg-arch-io[Jetty I/O architecture] for additional information about the classes mentioned below.
@ -153,6 +153,49 @@ Server applications can register `HttpChannel.Listener` by adding them as xref:p
include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=httpChannelListener]
----
[[pg-server-http-request-logging]]
==== Request Logging
HTTP requests and responses can be logged to provide data that can be later analyzed with other tools.
These tools can provide information such as the most frequently accessed request URIs, the response status codes, the request/response content lengths, geographical information about the clients, etc.
The default request/response log line format is the link:https://en.wikipedia.org/wiki/Common_Log_Format[NCSA Format] extended with referrer data and user-agent data.
[NOTE]
====
Typically, the extended NCSA format is the is enough and it's the standard used and understood by most log parsing tools and monitoring tools.
To customize the request/response log line format see the link:{javadoc-url}/org/eclipse/jetty/server/CustomRequestLog.html[`CustomRequestLog` javadocs].
====
Request logging can be enabled at the server level, or at the web application context level.
The request logging output can be directed to an SLF4J logger named `"org.eclipse.jetty.server.RequestLog"` at `INFO` level, and therefore to any logging library implementation of your choice (see also xref:pg-troubleshooting-logging[this section] about logging).
[source,java,indent=0]
----
include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=serverRequestLogSLF4J]
----
Alternatively, the request logging output can be directed to a daily rolling file of your choice, and the file name must contain `yyyy_MM_dd` so that rolled over files retain their date:
[source,java,indent=0]
----
include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=serverRequestLogFile]
----
For maximum flexibility, you can log to multiple ``RequestLog``s using class `RequestLog.Collection`, for example by logging with different formats or to different outputs.
You can use `CustomRequestLog` with a custom `RequestLog.Writer` to direct the request logging output to your custom targets (for example, an RDBMS).
You can implement your own `RequestLog` if you want to have functionalities that are not implemented by `CustomRequestLog`.
Request logging can also be enabled at the web application context level, using `RequestLogHandler` (see xref:pg-server-http-handler[this section] about how to organize Jetty ``Handler``s) to wrap a web application `Handler`:
[source,java,indent=0]
----
include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=contextRequestLog]
----
include::server-http-connector.adoc[]
include::server-http-handler.adoc[]
include::server-http-security.adoc[]

View File

@ -16,6 +16,7 @@ package org.eclipse.jetty.docs.programming.server.http;
import java.io.IOException;
import java.nio.file.Path;
import java.util.EnumSet;
import java.util.TimeZone;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
import javax.servlet.DispatcherType;
@ -40,14 +41,17 @@ import org.eclipse.jetty.rewrite.handler.RedirectRegexRule;
import org.eclipse.jetty.rewrite.handler.RewriteHandler;
import org.eclipse.jetty.rewrite.handler.RewriteRegexRule;
import org.eclipse.jetty.server.Connector;
import org.eclipse.jetty.server.CustomRequestLog;
import org.eclipse.jetty.server.HttpChannel;
import org.eclipse.jetty.server.HttpConfiguration;
import org.eclipse.jetty.server.HttpConnectionFactory;
import org.eclipse.jetty.server.ProxyConnectionFactory;
import org.eclipse.jetty.server.Request;
import org.eclipse.jetty.server.RequestLogWriter;
import org.eclipse.jetty.server.SecureRequestCustomizer;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.server.ServerConnector;
import org.eclipse.jetty.server.Slf4jRequestLogWriter;
import org.eclipse.jetty.server.SslConnectionFactory;
import org.eclipse.jetty.server.handler.AbstractHandler;
import org.eclipse.jetty.server.handler.ContextHandler;
@ -56,6 +60,7 @@ import org.eclipse.jetty.server.handler.DefaultHandler;
import org.eclipse.jetty.server.handler.HandlerCollection;
import org.eclipse.jetty.server.handler.HandlerList;
import org.eclipse.jetty.server.handler.HandlerWrapper;
import org.eclipse.jetty.server.handler.RequestLogHandler;
import org.eclipse.jetty.server.handler.ResourceHandler;
import org.eclipse.jetty.server.handler.SecuredRedirectHandler;
import org.eclipse.jetty.server.handler.StatisticsHandler;
@ -155,6 +160,57 @@ public class HTTPServerDocs
// end::httpChannelListener[]
}
public void serverRequestLogSLF4J()
{
// tag::serverRequestLogSLF4J[]
Server server = new Server();
// Sets the RequestLog to log to an SLF4J logger named "org.eclipse.jetty.server.RequestLog" at INFO level.
server.setRequestLog(new CustomRequestLog(new Slf4jRequestLogWriter(), CustomRequestLog.EXTENDED_NCSA_FORMAT));
// end::serverRequestLogSLF4J[]
}
public void serverRequestLogFile()
{
// tag::serverRequestLogFile[]
Server server = new Server();
// Use a file name with the pattern 'yyyy_MM_dd' so rolled over files retain their date.
RequestLogWriter logWriter = new RequestLogWriter("/var/log/yyyy_MM_dd.jetty.request.log");
// Retain rolled over files for 2 weeks.
logWriter.setRetainDays(14);
// Log times are in the current time zone.
logWriter.setTimeZone(TimeZone.getDefault().getID());
// Set the RequestLog to log to the given file, rolling over at midnight.
server.setRequestLog(new CustomRequestLog(logWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT));
// end::serverRequestLogFile[]
}
public void contextRequestLog()
{
// tag::contextRequestLog[]
Server server = new Server();
// Create a first ServletContextHandler for your main application.
ServletContextHandler mainContext = new ServletContextHandler();
mainContext.setContextPath("/main");
// Create a RequestLogHandler to log requests for your main application.
RequestLogHandler requestLogHandler = new RequestLogHandler();
requestLogHandler.setRequestLog(new CustomRequestLog());
// Wrap the main application with the request log handler.
requestLogHandler.setHandler(mainContext);
// Create a second ServletContextHandler for your other application.
// No request logging for this application.
ServletContextHandler otherContext = new ServletContextHandler();
mainContext.setContextPath("/other");
server.setHandler(new HandlerList(requestLogHandler, otherContext));
// end::contextRequestLog[]
}
public void configureConnector() throws Exception
{
// tag::configureConnector[]