Fixes #7369 - Document CustomRequestLog

Added documentation to the programming guide.
Removed related old documentation.

Signed-off-by: Simone Bordet <simone.bordet@gmail.com>

documentation/jetty-documentation/pom.xml

@@ -168,7 +168,6 @@
     <dependency>
       <groupId>org.eclipse.jetty</groupId>
       <artifactId>jetty-servlets</artifactId>
-      <version>${project.version}</version>
     </dependency>
     <dependency>
       <groupId>org.eclipse.jetty</groupId>

documentation/jetty-documentation/src/main/asciidoc/old_docs/logging/chapter.adoc

@@ -18,6 +18,5 @@ This chapter discusses various options for configuring logging.
 
 include::configuring-jetty-logging.adoc[]
 include::default-logging-with-stderrlog.adoc[]
-include::configuring-jetty-request-logs.adoc[]
 include::configuring-logging-modules.adoc[]
 include::example-logback-centralized-logging.adoc[]

documentation/jetty-documentation/src/main/asciidoc/old_docs/logging/configuring-jetty-request-logs.adoc

@@ -1,140 +0,0 @@
-//
-// ========================================================================
-// Copyright (c) 1995-2021 Mort Bay Consulting Pty Ltd and others.
-//
-// This program and the accompanying materials are made available under the
-// terms of the Eclipse Public License v. 2.0 which is available at
-// https://www.eclipse.org/legal/epl-2.0, or the Apache License, Version 2.0
-// which is available at https://www.apache.org/licenses/LICENSE-2.0.
-//
-// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
-// ========================================================================
-//
-
-[[configuring-jetty-request-logs]]
-=== Configuring Jetty Request Logs
-
-Request logs are a record of the requests that the server has processed.
-There is one entry per request received, and commonly in the standard NCSA format, so you can use tools like http://en.wikipedia.org/wiki/Webalizer[Webalizer] to analyze them conveniently.
-
-[[constructing-request-log-entry]]
-==== Constructing a Request Log Entry
-
-A standard request log entry includes the client IP address, date, method, URL, result, size, referrer, user agent and latency.
-For example:
-
-....
-123.4.5.6 - - [20/Jul/2016:10:16:17 +0000]
-  "GET /jetty/tut/XmlConfiguration.html HTTP/1.1"
-  200 76793 "http://localhost:8080/jetty/tut/logging.html"
-  "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8" 342
-....
-
-[[implementing-request-log]]
-==== Implementing a Request Log
-
-Jetty provides an implementation called `NCSARequestLog` which supports the NCSA format in files that will roll over on a daily basis.
-
-The http://logback.qos.ch/[Logback Project] offers http://logback.qos.ch/access.html[another implementation] of a `RequestLog` interface, providing rich and powerful HTTP-access log functionality.
-
-If neither of these options meets your needs, you can implement a custom request logger by implementing Jetty's link:{JDURL}/org/eclipse/jetty/server/RequestLog.html[`RequestLog.java`] interface and plugging it in similar to the `NCSARequestLog`, as shown below.
-
-[[configuring-request-log]]
-==== Configuring the Request Log module
-
-To enable the Request Log module for the entire server via the Jetty distribution, it first needs to be enabled on the command line:
-
-[source,screen,subs="{sub-order}"]
-----
-[my-base]$ java -jar /path/to/jetty-home/start.jar --add-to-start=requestlog
-
-INFO: requestlog      initialised in ${jetty.base}/start.d/requestlog.ini
-MKDIR: ${jetty.base}/logs
-INFO: Base directory was modified
-----
-
-The above command will add a new `requestlog.ini` file to your link:#start-vs-startd[`{$jetty.base}/start.d` directory].
-
-____
-[NOTE]
-By default, request logs are not set to be appended, meaning a the log file is wiped clean upon sever restart.
-You can change this setting by editing the `requestlog.ini` and un-commenting the line that reads `jetty.requestlog.append=true`.
-____
-
-The equivalent code for embedded usages of Jetty is:
-
-[source, java, subs="{sub-order}"]
-----
-NCSARequestLog requestLog = new NCSARequestLog("/var/logs/jetty/jetty-yyyy_mm_dd.request.log");
-requestLog.setAppend(true);
-requestLog.setExtended(false);
-requestLog.setLogTimeZone("GMT");
-requestLog.setLogLatency(true);
-requestLog.setRetainDays("90");
-
-server.setRequestLog(requestLog);
-----
-
-This configures a request log in `{$jetty.home}/logs` with filenames including the date.
-Existing log files are appended to and the extended NCSA format is used in the GMT time zone.
-
-The above configuration enables Log Latency, which is the amount of time it took the server to handle the request.
-This value is measured in milliseconds and is appended to the the log file for each request.
-
-You can also customize the number of days you wish to keep request logs.
-By default, log files are kept for 90 days before being deleted.
-The value for `retainDays` (xml) or `setRetainDays` (Java) should be configured as _1 + n_ days.
-For example, if you wanted to keep the logs for the current day and the day prior you would set the `retainDays` (or `setRetainDays`) value to 2.
-
-
-[[request-log-custom-writer]]
-==== Introducing RequestLog.Writer
-
-The concept of a `RequestLog.Writer`, introduced in Jetty 9.4.15, manages the writing to a log the string generated by the `RequestLog`.
-This allows the `CustomRequestLog` to match the functionality of other `RequestLogger` implementations by plugging in any `RequestLog.Writer` and a custom format string.
-Jetty currently has implementations of `RequestLog.Writer`, `RequestLogWriter`, `AsyncRequestLogWriter`, and `Slf4jRequestLogWriter`.
-
-So, the way to create an asynchronous `RequestLog` using the extended NCSA format has been changed from:
-
-`new AsyncNcsaRequestLog(filename)`
-
-to:
-
-`new CustomRequestLog(new AsyncRequestLogWriter(filename), CustomRequestLog.EXTENDED_NCSA_FORMAT)`
-
-Additionally, there are now two settings for the log timezone to be configured.
-There is the configuration for logging the request time, which is set in the `timeZone` parameter in the `%t` format code of the string, given in the format `%{format|timeZone|locale}t`.
-
-The other `timeZone` parameter relates to the generation of the log file name (both at creation and roll over).
-This is configured in the `requestlog` module file, or can be used as a setter on `RequestLogWriter` via XML.
-
-Both timezones are set to GMT by default.
-
-[[configuring-separate-request-log-for-web-application]]
-==== Configuring a Separate Request Log For a Web Application
-
-To configure a separate request log for specific a web application, add the following to the context XML file.
-
-[source, xml, subs="{sub-order}"]
-----
-<Configure class="org.eclipse.jetty.webapp.WebAppContext">
-  ...
-  <Call name="insertHandler">
-    <Arg>
-      <New id="RequestLog" class="org.eclipse.jetty.server.handler.RequestLogHandler">
-        <Set name="requestLog">
-          <New id="RequestLogImpl" class="org.eclipse.jetty.server.NCSARequestLog">
-            <Set name="filename"><Property name="jetty.logs" default="./logs"/>/test-yyyy_mm_dd.request.log</Set>
-            <Set name="filenameDateFormat">yyyy_MM_dd</Set>
-            <Set name="LogTimeZone">GMT</Set>
-            <Set name="retainDays">90</Set>
-            <Set name="append">true</Set>
-            <Set name="LogLatency">true</Set>
-          </New>
-        </Set>
-      </New>
-    </Arg>
-  </Call>
-  ...
-</Configure>
-----

documentation/jetty-documentation/src/main/asciidoc/programming-guide/server/http/server-http.adoc

@@ -71,7 +71,7 @@ Read xref:pg-server-http-handler[this section] for further details about ``Handl
 If you are interested in writing your server application based on the Servlet APIs, jump to xref:pg-server-http-handler-use-servlet[this section].
 
 [[pg-server-http-request-processing]]
-==== Server Request Processing
+==== Request Processing
 
 The Jetty HTTP request processing is outlined below in the diagram below.
 You may want to refer to the xref:pg-arch-io[Jetty I/O architecture] for additional information about the classes mentioned below.
@@ -153,6 +153,49 @@ Server applications can register `HttpChannel.Listener` by adding them as xref:p
 include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=httpChannelListener]
 ----
 
+[[pg-server-http-request-logging]]
+==== Request Logging
+
+HTTP requests and responses can be logged to provide data that can be later analyzed with other tools.
+These tools can provide information such as the most frequently accessed request URIs, the response status codes, the request/response content lengths, geographical information about the clients, etc.
+
+The default request/response log line format is the link:https://en.wikipedia.org/wiki/Common_Log_Format[NCSA Format] extended with referrer data and user-agent data.
+
+[NOTE]
+====
+Typically, the extended NCSA format is the is enough and it's the standard used and understood by most log parsing tools and monitoring tools.
+
+To customize the request/response log line format see the link:{javadoc-url}/org/eclipse/jetty/server/CustomRequestLog.html[`CustomRequestLog` javadocs].
+====
+
+Request logging can be enabled at the server level, or at the web application context level.
+
+The request logging output can be directed to an SLF4J logger named `"org.eclipse.jetty.server.RequestLog"` at `INFO` level, and therefore to any logging library implementation of your choice (see also xref:pg-troubleshooting-logging[this section] about logging).
+
+[source,java,indent=0]
+----
+include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=serverRequestLogSLF4J]
+----
+
+Alternatively, the request logging output can be directed to a daily rolling file of your choice, and the file name must contain `yyyy_MM_dd` so that rolled over files retain their date:
+
+[source,java,indent=0]
+----
+include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=serverRequestLogFile]
+----
+
+For maximum flexibility, you can log to multiple ``RequestLog``s using class `RequestLog.Collection`, for example by logging with different formats or to different outputs.
+
+You can use `CustomRequestLog` with a custom `RequestLog.Writer` to direct the request logging output to your custom targets (for example, an RDBMS).
+You can implement your own `RequestLog` if you want to have functionalities that are not implemented by `CustomRequestLog`.
+
+Request logging can also be enabled at the web application context level, using `RequestLogHandler` (see xref:pg-server-http-handler[this section] about how to organize Jetty ``Handler``s) to wrap a web application `Handler`:
+
+[source,java,indent=0]
+----
+include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=contextRequestLog]
+----
+
 include::server-http-connector.adoc[]
 include::server-http-handler.adoc[]
 include::server-http-security.adoc[]

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java

@@ -16,6 +16,7 @@ package org.eclipse.jetty.docs.programming.server.http;
 import java.io.IOException;
 import java.nio.file.Path;
 import java.util.EnumSet;
+import java.util.TimeZone;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.ConcurrentMap;
 import javax.servlet.DispatcherType;
@@ -40,14 +41,17 @@ import org.eclipse.jetty.rewrite.handler.RedirectRegexRule;
 import org.eclipse.jetty.rewrite.handler.RewriteHandler;
 import org.eclipse.jetty.rewrite.handler.RewriteRegexRule;
 import org.eclipse.jetty.server.Connector;
+import org.eclipse.jetty.server.CustomRequestLog;
 import org.eclipse.jetty.server.HttpChannel;
 import org.eclipse.jetty.server.HttpConfiguration;
 import org.eclipse.jetty.server.HttpConnectionFactory;
 import org.eclipse.jetty.server.ProxyConnectionFactory;
 import org.eclipse.jetty.server.Request;
+import org.eclipse.jetty.server.RequestLogWriter;
 import org.eclipse.jetty.server.SecureRequestCustomizer;
 import org.eclipse.jetty.server.Server;
 import org.eclipse.jetty.server.ServerConnector;
+import org.eclipse.jetty.server.Slf4jRequestLogWriter;
 import org.eclipse.jetty.server.SslConnectionFactory;
 import org.eclipse.jetty.server.handler.AbstractHandler;
 import org.eclipse.jetty.server.handler.ContextHandler;
@@ -56,6 +60,7 @@ import org.eclipse.jetty.server.handler.DefaultHandler;
 import org.eclipse.jetty.server.handler.HandlerCollection;
 import org.eclipse.jetty.server.handler.HandlerList;
 import org.eclipse.jetty.server.handler.HandlerWrapper;
+import org.eclipse.jetty.server.handler.RequestLogHandler;
 import org.eclipse.jetty.server.handler.ResourceHandler;
 import org.eclipse.jetty.server.handler.SecuredRedirectHandler;
 import org.eclipse.jetty.server.handler.StatisticsHandler;
@@ -155,6 +160,57 @@ public class HTTPServerDocs
         // end::httpChannelListener[]
     }
 
+    public void serverRequestLogSLF4J()
+    {
+        // tag::serverRequestLogSLF4J[]
+        Server server = new Server();
+
+        // Sets the RequestLog to log to an SLF4J logger named "org.eclipse.jetty.server.RequestLog" at INFO level.
+        server.setRequestLog(new CustomRequestLog(new Slf4jRequestLogWriter(), CustomRequestLog.EXTENDED_NCSA_FORMAT));
+        // end::serverRequestLogSLF4J[]
+    }
+
+    public void serverRequestLogFile()
+    {
+        // tag::serverRequestLogFile[]
+        Server server = new Server();
+
+        // Use a file name with the pattern 'yyyy_MM_dd' so rolled over files retain their date.
+        RequestLogWriter logWriter = new RequestLogWriter("/var/log/yyyy_MM_dd.jetty.request.log");
+        // Retain rolled over files for 2 weeks.
+        logWriter.setRetainDays(14);
+        // Log times are in the current time zone.
+        logWriter.setTimeZone(TimeZone.getDefault().getID());
+
+        // Set the RequestLog to log to the given file, rolling over at midnight.
+        server.setRequestLog(new CustomRequestLog(logWriter, CustomRequestLog.EXTENDED_NCSA_FORMAT));
+        // end::serverRequestLogFile[]
+    }
+
+    public void contextRequestLog()
+    {
+        // tag::contextRequestLog[]
+        Server server = new Server();
+
+        // Create a first ServletContextHandler for your main application.
+        ServletContextHandler mainContext = new ServletContextHandler();
+        mainContext.setContextPath("/main");
+
+        // Create a RequestLogHandler to log requests for your main application.
+        RequestLogHandler requestLogHandler = new RequestLogHandler();
+        requestLogHandler.setRequestLog(new CustomRequestLog());
+        // Wrap the main application with the request log handler.
+        requestLogHandler.setHandler(mainContext);
+
+        // Create a second ServletContextHandler for your other application.
+        // No request logging for this application.
+        ServletContextHandler otherContext = new ServletContextHandler();
+        mainContext.setContextPath("/other");
+
+        server.setHandler(new HandlerList(requestLogHandler, otherContext));
+        // end::contextRequestLog[]
+    }
+
     public void configureConnector() throws Exception
     {
         // tag::configureConnector[]