Eclipse/jetty.project
1
0
Fork 0
mirror of https://github.com/jetty/jetty.project.git synced 2025-03-02 20:09:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improvements to the Jetty client documentation.

Browse Source 
Signed-off-by: Simone Bordet <simone.bordet@gmail.com>
This commit is contained in:
Simone Bordet 2020-03-31 17:53:28 +02:00
parent 414a1dd396
commit 9a6ad8af62
19 changed files with 781 additions and 203 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
jetty-documentation/pom.xml
View File

@ -49,6 +49,11 @@
</properties>
<dependencies>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-client</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-io</artifactId>
@ -111,6 +116,13 @@
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctor-maven-plugin</artifactId>
<version>${asciidoctor.maven.plugin.version}</version>
<dependencies>
<dependency>
<groupId>org.asciidoctor</groupId>
<artifactId>asciidoctorj-diagram</artifactId>
<version>2.0.1</version>
</dependency>
</dependencies>
<configuration>
<attributes>
<JDURL>http://www.eclipse.org/jetty/javadoc/${javadoc.version}</JDURL>
@ -183,6 +195,10 @@
<sourceDirectory>${basedir}/src/main/asciidoc/embedded-guide</sourceDirectory>
<sourceDocumentName>index.adoc</sourceDocumentName>
<outputDirectory>${project.build.directory}/html/embedded-guide</outputDirectory>
<sourceHighlighter>coderay</sourceHighlighter>
<requires>
<require>asciidoctor-diagram</require>
</requires>
</configuration>
</execution>
</executions>

123
jetty-documentation/src/main/asciidoc/embedded-guide/client/client-concepts.adoc → jetty-documentation/src/main/asciidoc/embedded-guide/client/client-io-arch.adoc
View File

@ -16,32 +16,25 @@
// ========================================================================
//
[[client-concepts]]
=== Client Libraries Concepts
[[client-io-arch]]
=== Client Libraries Architecture
The Jetty client libraries implement a network client speaking different protocols
such as HTTP/1.1, HTTP/2, WebSocket and FastCGI.
The Jetty client libraries provide the basic components and APIs to implement
a network client.
It is possible to implement your own custom protocol on top of the Jetty client
libraries.
They build on the common link:#io-arch[Jetty I/O Architecture] and provide client
specific concepts (such as establishing a connection to a server).
NOTE: TODO: perhaps add a section about this.
There are conceptually two layers that compose the Jetty client libraries:
There are conceptually three layers that compose the Jetty client libraries, from
more abstract to more concrete:
. link:#client-io-arch-network[The network layer], that handles the low level
I/O and deals with buffers, threads, etc.
. link:#client-io-arch-protocol[The protocol layer], that handles the parsing
of bytes read from the network and the generation of bytes to write to the
network.
. The API layer, that exposes semantic APIs to applications so that they can write
code such as "GET me the resource at this URI".
. The protocol layer, where the API request is converted into the appropriate
protocol bytes, for example encrypted HTTP/2.
. The infrastructure layer, that handles the low level I/O and deals with network,
buffer, threads, etc.
Let's look at these layers starting from the more concrete (and low level) one
and build up to the more abstract layer.
[[client-concepts-infrastructure]]
==== Client Libraries Infrastructure Layer
[[client-io-arch-network]]
==== Client Libraries Network Layer
The Jetty client libraries use the common I/O design described in
link:#io-arch[this section].
@ -78,7 +71,7 @@ include::../{doc_code}/embedded/client/ClientConnectorDocs.java[tags=typical]
----
A more advanced example that customizes the `ClientConnector` by overriding
factory methods:
some of its methods:
[source,java,indent=0]
----
@ -86,7 +79,7 @@ include::../{doc_code}/embedded/client/ClientConnectorDocs.java[tags=advanced]
----
Since `ClientConnector` is the component that handles the low-level network, it
is also the component where you want to configure the low-leven network configuration.
is also the component where you want to configure the low-level network configuration.
The most common parameters are:
@ -122,28 +115,82 @@ Please refer to the `ClientConnector`
link:{JDURL}/org/eclipse/jetty/io/ClientConnector.html[javadocs]
for the complete list of configurable parameters.
Once the `ClientConnector` is configured and started, it can be used to connect
to the server via `ClientConnector.connect(SocketAddress, Map<String, Object>)`
which in turn will call `SocketChannel.connect(SocketAddress)`.
[[client-io-arch-protocol]]
==== Client Libraries Protocol Layer
When establishing a TCP connection to a server, applications need to tell
The protocol layer builds on top of the network layer to generate the
bytes to be written to the network and to parse the bytes read from the
network.
Recall from link:#io-arch-connection[this section] that Jetty uses the
`Connection` abstraction to produce and interpret the network bytes.
On the client side, a `ClientConnectionFactory` implementation is the
component that creates `Connection` instances based on the protocol that
the client wants to "speak" with the server.
Applications use `ClientConnector.connect(SocketAddress, Map<String, Object>)`
to establish a TCP connection to the server, and must tell
`ClientConnector` how to create the `Connection` for that particular
TCP connection.
This is done via a
link:{JDURL}/org/eclipse/jetty/io/ClientConnectionFactory.html[`ClientConnectionFactory`].
that must be passed in the context `Map` as follows:
TCP connection, and how to notify back the application when the connection
creation succeeds or fails.
This is done by passing a
link:{JDURL}/org/eclipse/jetty/io/ClientConnectionFactory.html[`ClientConnectionFactory`]
(that creates `Connection` instances) and a
link:{JDURL}/org/eclipse/jetty/util/Promise.html[`Promise`] (that is notified
of connection creation success or failure) in the context `Map` as follows:
[source,java,indent=0]
----
include::../{doc_code}/embedded/client/ClientConnectorDocs.java[tags=connect]
----
TODO: expand on what is the API to use, what parameters the context Map must
have, and basically how we can write a generic network client with it.
When a `Connection` is created successfully, its `onOpen()` method is invoked,
and then the promise is completed successfully.
[[client-concepts-protocol]]
==== Client Libraries Protocol Layer
It is now possible to write a super-simple `telnet` client that reads and writes
string lines:
The protocol layer builds on top of the infrastructure layer to generate the
bytes to be written to the network and to parse the bytes received from the
network.
[source,java,indent=0]
----
include::../{doc_code}/embedded/client/ClientConnectorDocs.java[tags=telnet]
----
Note how a very basic "telnet" API that applications could use is implemented
in the form of the `onLine(Consumer<String>)` for the non-blocking receiving
side and `writeLine(String, Callback)` for the non-blocking sending side.
Note also how the `onFillable()` method implements some basic "parsing"
by looking up the `\n` character in the buffer.
NOTE: The "telnet" client above looks like a super-simple HTTP client because
HTTP/1.0 can be seen as a line-based protocol. HTTP/1.0 was used just as an
example, but we could have used any other line-based protocol such as
link:https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol[SMTP],
provided that the server was able to understand it.
This is very similar to what the Jetty client implementation does for real
network protocols.
Real network protocols are of course more complicated and so is the implementation
code that handles them, but the general ideas are similar.
The Jetty client implementation provides a number of `ClientConnectionFactory`
implementations that can be composed to produce and interpret the network bytes.
For example, it is simple to modify the above example to use the TLS protocol
so that you will be able to connect to the server on port `443`, typically
reserved for the encrypted HTTP protocol.
The differences between the clear-text version and the TLS encrypted version
are minimal:
[source,java,indent=0]
----
include::../{doc_code}/embedded/client/ClientConnectorDocs.java[tags=tlsTelnet]
----
The differences with the clear-text version are only:
* Change the port from `80` to `443`.
* Wrap the `ClientConnectionFactory` with `SslClientConnectionFactory`.
* Unwrap the `SslConnection` to access `TelnetConnection`.

20
jetty-documentation/src/main/asciidoc/embedded-guide/client/client.adoc
View File

@ -17,12 +17,15 @@
//
[[client]]
== Jetty Client Libraries
== Client Libraries
The Eclipse Jetty Project provides not only server-side libraries so that you
can embed a server in your code, but it also provides client-side libraries
that allow you to embed a client - for example a HTTP client invoking a third
party HTTP service - in your application.
The Eclipse Jetty Project provides also provides client-side libraries
that allow you to embed a client in your applications.
A typical example is an application that needs to contact a third party
service via HTTP (for example a REST service).
Another example is a proxy application that receives HTTP requests and
forwards them as FCGI requests to a PHP application such as WordPress,
or receives HTTP/1.1 requests and converts them to HTTP/2.
The client libraries are designed to be non-blocking and offer both synchronous
and asynchronous APIs and come with a large number of configuration options.
@ -32,4 +35,9 @@ There are primarily two client libraries:
* link:#client-http[The HTTP client library]
* link:#client-websocket[The WebSocket client library]
include::client-concepts.adoc[]
If you are interested in the low-level details of how the Eclipse Jetty
client libraries work, or are interested in writing a custom protocol,
look at the link:#client-io-arch[Client I/O Architecture].
include::http/client-http.adoc[]
include::client-io-arch.adoc[]

0
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/http-client-api.adoc → jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-api.adoc
View File

4
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/http-client-authentication.adoc → jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-authentication.adoc
View File

@ -16,7 +16,7 @@
// ========================================================================
//
[[http-client-authentication]]
[[client-http-authentication]]
=== Authentication Support
Jetty's HTTP client supports the `BASIC` and `DIGEST` authentication mechanisms defined by link:https://tools.ietf.org/html/rfc7235[RFC 7235].
@ -88,4 +88,4 @@ authn.apply(request);
request.send();
----
See also the link:#http-client-proxy-authentication[proxy authentication section] for further information about how authentication works with HTTP proxies.
See also the link:#client-http-proxy-authentication[proxy authentication section] for further information about how authentication works with HTTP proxies.

86
jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-configuration.adoc Normal file
View File

@ -0,0 +1,86 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under
// the terms of the Eclipse Public License 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0
//
// This Source Code may also be made available under the following
// Secondary Licenses when the conditions for such availability set
// forth in the Eclipse Public License, v. 2.0 are satisfied:
// the Apache License v2.0 which is available at
// https://www.apache.org/licenses/LICENSE-2.0
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[client-http-configuration]]
=== HttpClient Configuration
`HttpClient` has a quite large number of configuration parameters.
Please refer to the `HttpClient`
link:{JDURL}/org/eclipse/jetty/client/HttpClient.html[javadocs]
for the complete list of configurable parameters.
The most common parameters are:
* `HttpClient.idleTimeout`: same as `ClientConnector.idleTimeout`
described in link:#client-io-arch-network[this section].
* `HttpClient.connectBlocking`: same as `ClientConnector.connectBlocking`
described in link:#client-io-arch-network[this section].
* `HttpClient.connectTimeout`: same as `ClientConnector.connectTimeout`
described in link:#client-io-arch-network[this section].
* `HttpClient.maxConnectionsPerDestination`: the max number of TCP
connections that are opened for a particular destination (defaults to 64).
* `HttpClient.maxRequestsQueuedPerDestination`: the max number of requests
queued (defaults to 1024).
[[client-http-configuration-tls]]
==== HttpClient TLS Configuration
`HttpClient` supports HTTPS requests out-of-the-box like a browser does.
The support for HTTPS request is provided by a `SslContextFactory.Client`,
typically configured in the `ClientConnector`.
If not explicitly configured, the `ClientConnector` will allocate a default
one when started.
[source,java,indent=0]
----
include::../../{doc_code}/embedded/client/http/HTTPClientDocs.java[tags=tlsExplicit]
----
The default `SslContextFactory.Client` verifies the certificate sent by the
server by verifying the certificate chain.
This means that requests to public websites that have a valid certificates
(such as ``https://google.com``) will work out-of-the-box.
However, requests made to sites (typically ``localhost``) that have invalid
(for example, expired or with a wrong host) or self-signed certificates will
fail (like they will in a browser).
Certificate validation is performed at two levels: at the TLS implementation
level (in the JDK) and - optionally - at the application level.
By default, certificate validation at the TLS level is enabled, while
certificate validation at the application level is disabled.
You can configure the `SslContextFactory.Client` to skip certificate validation
at the TLS level:
[source,java,indent=0]
----
include::../../{doc_code}/embedded/client/http/HTTPClientDocs.java[tags=tlsNoValidation]
----
You can enable certificate validation at the application level:
[source,java,indent=0]
----
include::../../{doc_code}/embedded/client/http/HTTPClientDocs.java[tags=tlsAppValidation]
----
Please refer to the `SslContextFactory.Client`
link:{JDURL}/org/eclipse/jetty/util/ssl/SslContextFactory.Client.html[javadocs]
for the complete list of configurable parameters.

2
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/http-client-cookie.adoc → jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-cookie.adoc
View File

@ -16,7 +16,7 @@
// ========================================================================
//
[[http-client-cookie]]
[[client-http-cookie]]
=== Cookies Support
Jetty HTTP client supports cookies out of the box.

175
jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-intro.adoc Normal file
View File

@ -0,0 +1,175 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under
// the terms of the Eclipse Public License 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0
//
// This Source Code may also be made available under the following
// Secondary Licenses when the conditions for such availability set
// forth in the Eclipse Public License, v. 2.0 are satisfied:
// the Apache License v2.0 which is available at
// https://www.apache.org/licenses/LICENSE-2.0
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[client-http-intro]]
=== HttpClient Introduction
The Jetty HTTP client module provides easy-to-use APIs and utility classes to perform HTTP (or HTTPS) requests.
Jetty's HTTP client is non-blocking and asynchronous.
It offers an asynchronous API that never blocks for I/O, making it very efficient in thread utilization and well suited for high performance scenarios such as load testing or parallel computation.
However, when all you need to do is to perform a `GET` request to a resource, Jetty's HTTP client offers also a synchronous API; a programming interface
where the thread that issued the request blocks until the request/response conversation is complete.
Jetty's HTTP client supports link:#http-client-transport[different transports]: HTTP/1.1, FastCGI and HTTP/2.
This means that the semantic of a HTTP request (that is, " `GET` me the resource `/index.html` ") can be carried over the network in different formats.
The most common and default format is HTTP/1.1.
That said, Jetty's HTTP client can carry the same request using the FastCGI format or the HTTP/2 format.
The FastCGI transport is heavily used in Jetty's link:#fastcgi[FastCGI support] that allows Jetty to work as a reverse proxy to PHP (exactly like Apache or Nginx do) and therefore be able to serve - for example - WordPress websites.
The HTTP/2 transport allows Jetty's HTTP client to perform requests using HTTP/2 to HTTP/2 enabled web sites, see also Jetty's link:#http2[HTTP/2 support].
Out of the box features that you get with the Jetty HTTP client include:
* Redirect support - redirect codes such as 302 or 303 are automatically followed.
* Cookies support - cookies sent by servers are stored and sent back to servers in matching requests.
* Authentication support - HTTP "Basic" and "Digest" authentications are supported, others are pluggable.
* Forward proxy support - HTTP proxying and SOCKS4 proxying.
[[client-http-start]]
==== Starting HttpClient
The main class is named `org.eclipse.jetty.client.HttpClient`.
You can think of a `HttpClient` instance as a browser instance.
Like a browser it can make requests to different domains, it manages redirects, cookies and authentication, you can configure it with a proxy, and
it provides you with the responses to the requests you make.
In order to use `HttpClient`, you must instantiate it, configure it, and then start it:
[source,java,indent=0]
----
include::../../{doc_code}/embedded/client/http/HTTPClientDocs.java[tags=start]
----
You may create multiple instances of `HttpClient`, but typically one instance is enough for an application.
There are several reasons for having multiple `HttpClient` instances including, but not limited to:
* You want to specify different configuration parameters (for example, one instance is configured with a forward proxy while another is not).
* You want the two instances to behave like two different browsers and hence have different cookies, different authentication credentials, etc.
* You want to use link:#http-client-transport[different transports].
Like browsers, HTTPS requests are supported out-of-the-box, as long as the server
provides a valid certificate.
In case the server does not provide a valid certificate (or in case it is self-signed)
you want to customize ``HttpClient``'s TLS configuration as described in
link:#client-http-configuration-tls[this section].
[[client-http-stop]]
==== Stopping HttpClient
It is recommended that when your application stops, you also stop the `HttpClient` instance (or instances) that you are using.
[source,java,indent=0]
----
include::../../{doc_code}/embedded/client/http/HTTPClientDocs.java[tags=stop]
----
Stopping `HttpClient` makes sure that the memory it holds (for example, authentication credentials, cookies, etc.) is released, and that the thread pool and scheduler are properly stopped allowing all threads used by `HttpClient` to exit.
[[client-http-arch]]
==== HttpClient Architecture
A `HttpClient` instance can be thought as a browser instance, and it manages the
following components:
* a `CookieStore` (see link:#client-http-cookie[this section]).
* a `AuthenticationStore` (see link:#client-http-authentication[this section]).
* a `ProxyConfiguration` (see link:#client-http-proxy[this section]).
* a set of _destinations_.
A _destination_ is the client-side component that represent an _origin_ on
a server, and manages a queue of requests for that origin, and a pool of
connections to that origin.
An _origin_ may be simply thought as the tuple `(scheme, host, port)` and it
is where the client connects to in order to communicate with the server.
However, this is not enough.
If you use `HttpClient` to write a proxy you may have different clients that
want to contact the same server.
In this case, you may not want to use the same proxy-to-server connection to
proxy requests for both clients, for example for authentication reasons: the
server may associate the connection with authentication credentials and you
do not want to use the same connection for two different users that have
different credentials.
Instead, you want to use different connections for different clients and
this can be achieved by "tagging" a destination with a tag object that
represents the remote client (for example, it could be the remote client IP
address).
Two origins with the same `(scheme, host, port)` but different `tag`
create two different destinations and therefore two different connection pools.
However, also this is not enough.
It is possible that a server speaks different protocols on the same `port`.
A connection may start by speaking one protocol, for example HTTP/1.1, but
then be upgraded to speak a different protocol, for example HTTP/2.
After a connection has been upgraded to a second protocol, it cannot speak
the first protocol anymore, so it can only be used to communicate using
the second protocol.
Two origins with the same `(scheme, host, port)` but different
`protocol` create two different destinations and therefore two different
connection pools.
Therefore an origin is identified by the tuple
`(scheme, host, port, tag, protocol)`.
[[client-http-request-processing]]
==== HttpClient Request Processing
When a request is sent, an origin is computed from the request; `HttpClient`
uses that origin to find (or create if it does not exist) the correspondent
destination.
The request is then queued onto the destination, and this causes the
destination to ask its connection pool for a free connection.
If a connection is available, it is returned, otherwise a new connection is
created.
Once the destination has obtained the connection, it dequeues the request
and sends it over the connection.
The first request to a destination triggers the opening of the first
connection.
A second request with the same origin sent _after_ the first will reuse the
same connection.
A second request with the same origin sent _concurrently_ with the first
request will cause the opening of a second connection.
The configuration parameter `HttpClient.maxConnectionsPerDestination`
(see also the link:#client-http-configuration[configuration section]) controls
the max number of connections that can be opened for a destination.
NOTE: If opening connections to a given origin takes a long time, then
requests for that origin will queue up in the corresponding destination.
Each connection can handle a limited number of requests.
For HTTP/1.1, this number is always `1`: there can only be one outstanding
request for each connection.
For HTTP/2 this number is determined by the server `max_concurrent_stream`
setting (typically around `100`, i.e. there can be up to `100` outstanding
requests for every connection).
When a destination has maxed out its number of connections, and all
connections have maxed out their number of outstanding requests, more requests
sent to that destination will be queued.
When the request queue is full, the request will be failed.
The configuration parameter `HttpClient.maxRequestsQueuedPerDestination`
(see also the link:#client-http-configuration[configuration section]) controls
the max number of requests that can be queued for a destination.

8
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/http-client-proxy.adoc → jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-proxy.adoc
View File

@ -16,7 +16,7 @@
// ========================================================================
//
[[http-client-proxy]]
[[client-http-proxy]]
=== Proxy Support
Jetty's HTTP client can be configured to use proxies to connect to destinations.
@ -44,10 +44,10 @@ You specify the proxy host and port, and optionally also the addresses that you
Configured in this way, `HttpClient` makes requests to the HTTP proxy (for plain-text HTTP requests) or establishes a tunnel via `HTTP CONNECT` (for encrypted HTTPS requests).
[[http-client-proxy-authentication]]
[[client-http-proxy-authentication]]
==== Proxy Authentication Support
Jetty's HTTP client support proxy authentication in the same way it supports link:#http-client-authentication[server authentication].
Jetty's HTTP client support proxy authentication in the same way it supports link:#client-http-authentication[server authentication].
In the example below, the proxy requires Basic authentication, but the server requires Digest authentication, and therefore:
@ -100,4 +100,4 @@ Application HttpClient Proxy Server
The application does not receive events related to the responses with code 407 and 401 since they are handled internally by `HttpClient`.
Similarly to the link:#http-client-authentication[authentication section], the proxy authentication result and the server authentication result can be preempted to avoid, respectively, the 407 and 401 roundtrips.
Similarly to the link:#client-http-authentication[authentication section], the proxy authentication result and the server authentication result can be preempted to avoid, respectively, the 407 and 401 roundtrips.

0
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/http-client-transport.adoc → jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http-transport.adoc
View File

9
jetty-documentation/src/main/asciidoc/embedded-guide/client/http/client-http.adoc
View File

@ -18,3 +18,12 @@
[[client-http]]
=== HTTP Client
include::client-http-intro.adoc[]
include::client-http-configuration.adoc[]
include::client-http-api.adoc[]
include::client-http-cookie.adoc[]
include::client-http-authentication.adoc[]
include::client-http-proxy.adoc[]
include::client-http-transport.adoc[]

2
jetty-documentation/src/main/asciidoc/embedded-guide/io-arch.adoc
View File

@ -261,3 +261,5 @@ Otherwise, the write is now `PENDING` and waiting for the callback to be
notified of the completion at a later time.
When the callback is notified of the `write()` completion, it checks whether
the `write()` was `PENDING`, and if it was it resumes reading.
NOTE: TODO: Introduce IteratingCallback?

27
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/chapter.adoc
View File

@ -1,27 +0,0 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under
// the terms of the Eclipse Public License 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0
//
// This Source Code may also be made available under the following
// Secondary Licenses when the conditions for such availability set
// forth in the Eclipse Public License, v. 2.0 are satisfied:
// the Apache License v2.0 which is available at
// https://www.apache.org/licenses/LICENSE-2.0
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[http-client]]
== HTTP Client
include::http-client-intro.adoc[]
include::http-client-api.adoc[]
include::http-client-cookie.adoc[]
include::http-client-authentication.adoc[]
include::http-client-proxy.adoc[]
include::http-client-transport.adoc[]

105
jetty-documentation/src/main/asciidoc/embedded-guide/server/clients/http/http-client-intro.adoc
View File

@ -1,105 +0,0 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under
// the terms of the Eclipse Public License 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0
//
// This Source Code may also be made available under the following
// Secondary Licenses when the conditions for such availability set
// forth in the Eclipse Public License, v. 2.0 are satisfied:
// the Apache License v2.0 which is available at
// https://www.apache.org/licenses/LICENSE-2.0
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
[[http-client-intro]]
=== Introduction
The Jetty HTTP client module provides easy-to-use APIs and utility classes to perform HTTP (or HTTPS) requests.
Jetty's HTTP client is non-blocking and asynchronous.
It offers an asynchronous API that never blocks for I/O, making it very efficient in thread utilization and well suited for high performance scenarios such as load testing or parallel computation.
However, when all you need to do is to perform a `GET` request to a resource, Jetty's HTTP client offers also a synchronous API; a programming interface
where the thread that issued the request blocks until the request/response conversation is complete.
Jetty's HTTP client supports different link:#http-client-transport[transports]: HTTP/1.1, FastCGI and HTTP/2.
This means that the semantic of a HTTP request (that is, " `GET` me the resource `/index.html` ") can be carried over the network in different formats.
The most common and default format is HTTP/1.1.
That said, Jetty's HTTP client can carry the same request using the FastCGI format or the new HTTP/2 format.
The FastCGI transport is heavily used in Jetty's link:#fastcgi[FastCGI support] that allows Jetty to work as a reverse proxy to PHP (exactly like Apache or Nginx do) and therefore be able to serve - for example - WordPress websites.
The HTTP/2 transport allows Jetty's HTTP client to perform requests using HTTP/2 to HTTP/2 enabled web sites, see also Jetty's link:#http2[HTTP/2 support].
Out of the box features that you get with the Jetty HTTP client include:
* Redirect support - redirect codes such as 302 or 303 are automatically followed.
* Cookies support - cookies sent by servers are stored and sent back to servers in matching requests.
* Authentication support - HTTP "Basic" and "Digest" authentications are supported, others are pluggable.
* Forward proxy support - HTTP proxying and SOCKS4 proxying.
[[http-client-init]]
==== Starting HttpClient
The main class is named `org.eclipse.jetty.client.HttpClient`.
You can think of a `HttpClient` instance as a browser instance.
Like a browser it can make requests to different domains, it manages redirects, cookies and authentication, you can configure it with a proxy, and
it provides you with the responses to the requests you make.
In order to use `HttpClient`, you must instantiate it, configure it, and then start it:
[source, java, subs="{sub-order}"]
----
// Instantiate HttpClient
HttpClient httpClient = new HttpClient();
// Configure HttpClient, for example:
httpClient.setFollowRedirects(false);
// Start HttpClient
httpClient.start();
----
You may create multiple instances of `HttpClient`, but typically one instance is enough for an application.
There are several reasons for having multiple `HttpClient` instances including, but not limited to:
* You want to specify different configuration parameters (for example, one instance is configured with a forward proxy while another is not)
* You want the two instances to behave like two different browsers and hence have different cookies, different authentication credentials...etc.
* You want to use different transports
When you create a `HttpClient` instance using the parameterless constructor, you will only be able to perform plain HTTP requests and you will not be able to perform HTTPS requests.
In order to perform HTTPS requests, you should create first a link:{JDURL}/org/eclipse/jetty/util/ssl/SslContextFactory.Client.html[`SslContextFactory.Client`], configure it, and pass it to the `HttpClient` constructor.
When created with a `SslContextFactory`, the `HttpClient` will be able to perform both HTTP and HTTPS requests to any domain.
[source, java, subs="{sub-order}"]
----
// Instantiate and configure the SslContextFactory
SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
// Instantiate HttpClient with the SslContextFactory
HttpClient httpClient = new HttpClient(sslContextFactory);
// Configure HttpClient, for example:
httpClient.setFollowRedirects(false);
// Start HttpClient
httpClient.start();
----
==== Stopping HttpClient
It is recommended that when your application stops, you also stop the `HttpClient` instance (or instances) that you are using.
[source, java, subs="{sub-order}"]
----
httpClient.stop();
----
Stopping `HttpClient` makes sure that the memory it holds (for example, authentication credentials, cookies, etc.) is released, and that the thread pool and scheduler are properly stopped allowing all threads used by `HttpClient` to exit.

293
jetty-documentation/src/main/java/embedded/client/ClientConnectorDocs.java
View File

@ -18,17 +18,27 @@
package embedded.client;
import java.io.ByteArrayOutputStream;
import java.net.InetSocketAddress;
import java.net.SocketAddress;
import java.nio.ByteBuffer;
import java.nio.charset.StandardCharsets;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.Executor;
import java.util.function.Consumer;
import org.eclipse.jetty.io.AbstractConnection;
import org.eclipse.jetty.io.ClientConnectionFactory;
import org.eclipse.jetty.io.ClientConnector;
import org.eclipse.jetty.io.EndPoint;
import org.eclipse.jetty.io.SelectorManager;
import org.eclipse.jetty.io.ssl.SslClientConnectionFactory;
import org.eclipse.jetty.io.ssl.SslConnection;
import org.eclipse.jetty.util.BufferUtil;
import org.eclipse.jetty.util.Callback;
import org.eclipse.jetty.util.Promise;
import org.eclipse.jetty.util.ssl.SslContextFactory;
import org.eclipse.jetty.util.thread.QueuedThreadPool;
import org.eclipse.jetty.util.thread.ScheduledExecutorScheduler;
@ -108,9 +118,9 @@ public class ClientConnectorDocs
public void connect() throws Exception
{
// tag::connect[]
class CustomHTTPConnection extends AbstractConnection
class CustomConnection extends AbstractConnection
{
public CustomHTTPConnection(EndPoint endPoint, Executor executor)
public CustomConnection(EndPoint endPoint, Executor executor)
{
super(endPoint, executor);
}
@ -119,6 +129,7 @@ public class ClientConnectorDocs
public void onOpen()
{
super.onOpen();
System.getLogger("connection").log(INFO, "Opened connection {0}", this);
}
@Override
@ -130,23 +141,289 @@ public class ClientConnectorDocs
ClientConnector clientConnector = new ClientConnector();
clientConnector.start();
String host = "serverHost";
int port = 8080;
SocketAddress address = new InetSocketAddress(host, port);
// The ClientConnectionFactory that creates CustomConnection instances.
ClientConnectionFactory connectionFactory = (endPoint, context) ->
{
System.getLogger("connection").log(INFO, "Creating connection for {0}", endPoint);
return new CustomConnection(endPoint, clientConnector.getExecutor());
};
// The Promise to notify of connection creation success or failure.
CompletableFuture<CustomConnection> connectionPromise = new Promise.Completable<>();
// Populate the context with the mandatory keys to create and obtain connections.
Map<String, Object> context = new HashMap<>();
context.put(ClientConnector.CLIENT_CONNECTION_FACTORY_CONTEXT_KEY, connectionFactory);
context.put(ClientConnector.CONNECTION_PROMISE_CONTEXT_KEY, connectionPromise);
clientConnector.connect(address, context);
// Use the Connection when it's available.
// Use it in a non-blocking way via CompletableFuture APIs.
connectionPromise.whenComplete((connection, failure) ->
{
System.getLogger("connection").log(INFO, "Created connection for {0}", connection);
});
// Alternatively, you can block waiting for the connection (or a failure).
// CustomConnection connection = connectionPromise.get();
// end::connect[]
}
public void telnet() throws Exception
{
// tag::telnet[]
class TelnetConnection extends AbstractConnection
{
private final ByteArrayOutputStream bytes = new ByteArrayOutputStream();
private Consumer<String> consumer;
public TelnetConnection(EndPoint endPoint, Executor executor)
{
super(endPoint, executor);
}
@Override
public void onOpen()
{
super.onOpen();
// Declare interest for fill events.
fillInterested();
}
@Override
public void onFillable()
{
try
{
ByteBuffer buffer = BufferUtil.allocate(1024);
while (true)
{
int filled = getEndPoint().fill(buffer);
if (filled > 0)
{
while (buffer.hasRemaining())
{
// Search for newline.
byte read = buffer.get();
if (read == '\n')
{
// Notify the consumer of the line.
consumer.accept(bytes.toString(StandardCharsets.UTF_8));
bytes.reset();
}
else
{
bytes.write(read);
}
}
}
else if (filled == 0)
{
// No more bytes to fill, declare
// again interest for fill events.
fillInterested();
return;
}
else
{
// The other peer closed the
// connection, close it back.
getEndPoint().close();
return;
}
}
}
catch (Exception x)
{
getEndPoint().close(x);
}
}
public void onLine(Consumer<String> consumer)
{
this.consumer = consumer;
}
public void writeLine(String line, Callback callback)
{
line = line + "\r\n";
getEndPoint().write(callback, ByteBuffer.wrap(line.getBytes(StandardCharsets.UTF_8)));
}
}
ClientConnector clientConnector = new ClientConnector();
clientConnector.start();
String host = "wikipedia.org";
int port = 80;
SocketAddress address = new InetSocketAddress(host, port);
ClientConnectionFactory connectionFactory = (endPoint, context) ->
{
System.getLogger("connection").log(INFO, "Creating connection for {0}", endPoint);
return new CustomHTTPConnection(endPoint, clientConnector.getExecutor());
};
new TelnetConnection(endPoint, clientConnector.getExecutor());
CompletableFuture<TelnetConnection> connectionPromise = new Promise.Completable<>();
Map<String, Object> context = new HashMap<>();
context.put(ClientConnector.CLIENT_CONNECTION_FACTORY_CONTEXT_KEY, connectionFactory);
context.put(ClientConnector.CONNECTION_PROMISE_CONTEXT_KEY, connectionPromise);
clientConnector.connect(address, context);
// end::connect[]
connectionPromise.whenComplete((connection, failure) ->
{
if (failure == null)
{
// Register a listener that receives string lines.
connection.onLine(line -> System.getLogger("app").log(INFO, "line: {0}", line));
// Write a line.
connection.writeLine("" +
"GET / HTTP/1.0\r\n" +
"", Callback.NOOP);
}
else
{
failure.printStackTrace();
}
});
// end::telnet[]
}
public void tlsTelnet() throws Exception
{
// tag::tlsTelnet[]
class TelnetConnection extends AbstractConnection
{
private final ByteArrayOutputStream bytes = new ByteArrayOutputStream();
private Consumer<String> consumer;
public TelnetConnection(EndPoint endPoint, Executor executor)
{
super(endPoint, executor);
}
@Override
public void onOpen()
{
super.onOpen();
// Declare interest for fill events.
fillInterested();
}
@Override
public void onFillable()
{
try
{
ByteBuffer buffer = BufferUtil.allocate(1024);
while (true)
{
int filled = getEndPoint().fill(buffer);
if (filled > 0)
{
while (buffer.hasRemaining())
{
// Search for newline.
byte read = buffer.get();
if (read == '\n')
{
// Notify the consumer of the line.
consumer.accept(bytes.toString(StandardCharsets.UTF_8));
bytes.reset();
}
else
{
bytes.write(read);
}
}
}
else if (filled == 0)
{
// No more bytes to fill, declare
// again interest for fill events.
fillInterested();
return;
}
else
{
// The other peer closed the
// connection, close it back.
getEndPoint().close();
return;
}
}
}
catch (Exception x)
{
getEndPoint().close(x);
}
}
public void onLine(Consumer<String> consumer)
{
this.consumer = consumer;
}
public void writeLine(String line, Callback callback)
{
line = line + "\r\n";
getEndPoint().write(callback, ByteBuffer.wrap(line.getBytes(StandardCharsets.UTF_8)));
}
}
ClientConnector clientConnector = new ClientConnector();
clientConnector.start();
// Use port 443 to contact the server using encrypted HTTP.
String host = "wikipedia.org";
int port = 443;
SocketAddress address = new InetSocketAddress(host, port);
ClientConnectionFactory connectionFactory = (endPoint, context) ->
new TelnetConnection(endPoint, clientConnector.getExecutor());
// Wrap the "telnet" ClientConnectionFactory with the SslClientConnectionFactory.
connectionFactory = new SslClientConnectionFactory(clientConnector.getSslContextFactory(),
clientConnector.getByteBufferPool(), clientConnector.getExecutor(), connectionFactory);
// We will obtain a SslConnection now.
CompletableFuture<SslConnection> connectionPromise = new Promise.Completable<>();
Map<String, Object> context = new HashMap<>();
context.put(ClientConnector.CLIENT_CONNECTION_FACTORY_CONTEXT_KEY, connectionFactory);
context.put(ClientConnector.CONNECTION_PROMISE_CONTEXT_KEY, connectionPromise);
clientConnector.connect(address, context);
connectionPromise.whenComplete((sslConnection, failure) ->
{
if (failure == null)
{
// Unwrap the SslConnection to access the "line" APIs in TelnetConnection.
TelnetConnection connection = (TelnetConnection)sslConnection.getDecryptedEndPoint().getConnection();
// Register a listener that receives string lines.
connection.onLine(line -> System.getLogger("app").log(INFO, "line: {0}", line));
// Write a line.
connection.writeLine("" +
"GET / HTTP/1.0\r\n" +
"", Callback.NOOP);
}
else
{
failure.printStackTrace();
}
});
// end::tlsTelnet[]
}
public static void main(String[] args) throws Exception
{
new ClientConnectorDocs().connect();
new ClientConnectorDocs().tlsTelnet();
}
}

82
jetty-documentation/src/main/java/embedded/client/http/HTTPClientDocs.java Normal file
View File

@ -0,0 +1,82 @@
//
// ========================================================================
// Copyright (c) 1995-2020 Mort Bay Consulting Pty Ltd and others.
//
// This program and the accompanying materials are made available under
// the terms of the Eclipse Public License 2.0 which is available at
// https://www.eclipse.org/legal/epl-2.0
//
// This Source Code may also be made available under the following
// Secondary Licenses when the conditions for such availability set
// forth in the Eclipse Public License, v. 2.0 are satisfied:
// the Apache License v2.0 which is available at
// https://www.apache.org/licenses/LICENSE-2.0
//
// SPDX-License-Identifier: EPL-2.0 OR Apache-2.0
// ========================================================================
//
package embedded.client.http;
import org.eclipse.jetty.client.HttpClient;
import org.eclipse.jetty.client.dynamic.HttpClientTransportDynamic;
import org.eclipse.jetty.io.ClientConnector;
import org.eclipse.jetty.util.ssl.SslContextFactory;
public class HTTPClientDocs
{
public void start() throws Exception
{
// tag::start[]
// Instantiate HttpClient.
HttpClient httpClient = new HttpClient();
// Configure HttpClient, for example:
httpClient.setFollowRedirects(false);
// Start HttpClient.
httpClient.start();
// end::start[]
}
public void stop() throws Exception
{
HttpClient httpClient = new HttpClient();
httpClient.start();
// tag::stop[]
// Stop HttpClient.
httpClient.stop();
// end::stop[]
}
public void tlsExplicit() throws Exception
{
// tag::tlsExplicit[]
SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
ClientConnector clientConnector = new ClientConnector();
clientConnector.setSslContextFactory(sslContextFactory);
HttpClient httpClient = new HttpClient(new HttpClientTransportDynamic(clientConnector));
httpClient.start();
// end::tlsExplicit[]
}
public void tlsNoValidation() throws Exception
{
// tag::tlsNoValidation[]
SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
// Disable certificate validation at the TLS level.
sslContextFactory.setEndpointIdentificationAlgorithm(null);
// end::tlsNoValidation[]
}
public void tlsAppValidation() throws Exception
{
// tag::tlsAppValidation[]
SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
// Only allow subdomains of domain.com.
sslContextFactory.setHostnameVerifier((hostName, session) -> hostName.endsWith(".domain.com"));
// end::tlsAppValidation[]
}
}

23
jetty-io/src/main/java/org/eclipse/jetty/io/ClientConnector.java
View File

@ -30,6 +30,7 @@ import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.Executor;
import org.eclipse.jetty.util.IO;
import org.eclipse.jetty.util.Promise;
import org.eclipse.jetty.util.component.ContainerLifeCycle;
import org.eclipse.jetty.util.ssl.SslContextFactory;
@ -281,15 +282,7 @@ public class ClientConnector extends ContainerLifeCycle
protected void safeClose(Closeable closeable)
{
try
{
if (closeable != null)
closeable.close();
}
catch (Throwable x)
{
LOG.trace("IGNORED", x);
}
IO.close(closeable);
}
protected void configure(SocketChannel channel) throws IOException
@ -330,6 +323,18 @@ public class ClientConnector extends ContainerLifeCycle
return factory.newConnection(endPoint, context);
}
@Override
public void connectionOpened(Connection connection, Object context)
{
super.connectionOpened(connection, context);
@SuppressWarnings("unchecked")
Map<String, Object> contextMap = (Map<String, Object>)context;
@SuppressWarnings("unchecked")
Promise<Connection> promise = (Promise<Connection>)contextMap.get(CONNECTION_PROMISE_CONTEXT_KEY);
if (promise != null)
promise.succeeded(connection);
}
@Override
protected void connectionFailed(SelectableChannel channel, Throwable failure, Object attachment)
{

5
jetty-io/src/main/java/org/eclipse/jetty/io/ManagedSelector.java
View File

@ -268,12 +268,13 @@ public class ManagedSelector extends ContainerLifeCycle implements Dumpable
private void createEndPoint(SelectableChannel channel, SelectionKey selectionKey) throws IOException
{
EndPoint endPoint = _selectorManager.newEndPoint(channel, this, selectionKey);
Connection connection = _selectorManager.newConnection(channel, endPoint, selectionKey.attachment());
Object context = selectionKey.attachment();
Connection connection = _selectorManager.newConnection(channel, endPoint, context);
endPoint.setConnection(connection);
selectionKey.attach(endPoint);
endPoint.onOpen();
endPointOpened(endPoint);
_selectorManager.connectionOpened(connection);
_selectorManager.connectionOpened(connection, context);
if (LOG.isDebugEnabled())
LOG.debug("Created {}", endPoint);
}

4
jetty-io/src/main/java/org/eclipse/jetty/io/SelectorManager.java
View File

@ -302,8 +302,10 @@ public abstract class SelectorManager extends ContainerLifeCycle implements Dump
* <p>Callback method invoked when a connection is opened.</p>
*
* @param connection the connection just opened
* @param context the attachment associated with the creation of the connection
* @see #newConnection(SelectableChannel, EndPoint, Object)
*/
public void connectionOpened(Connection connection)
public void connectionOpened(Connection connection, Object context)
{
try
{