Eclipse
/
jetty.project
mirror of https://github.com/jetty/jetty.project.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Issue #684 - HttpClient proxies (HttpProxy and Socks4Proxy) do not support authentication. 

The feature was already present, improved documentation.
This commit is contained in:
Simone Bordet 2016-07-12 23:22:24 +02:00
parent d08d5b949c
commit c77562bc95
7 changed files with 521 additions and 313 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
jetty-documentation/src/main/asciidoc/development/clients/http/chapter.adoc
View File

@ -19,4 +19,6 @@
include::http-client-intro.adoc[]
include::http-client-api.adoc[]
include::http-client-other.adoc[]
include::http-client-cookie.adoc[]
include::http-client-authentication.adoc[]
include::http-client-proxy.adoc[]

276
jetty-documentation/src/main/asciidoc/development/clients/http/http-client-api.adoc
View File

@ -25,16 +25,20 @@ The simpler way to perform a HTTP request is the following:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.GET("http://domain.com/path?query");
----
Method `HttpClient.GET(...)` performs a HTTP GET request to the given URI and returns a `ContentResponse` when the request/response conversation completes successfully.
Method `HttpClient.GET(...)` performs a HTTP GET request to the given URI and
returns a `ContentResponse` when the request/response conversation completes
successfully.
The `ContentResponse` object contains the HTTP response information: status code, headers and possibly a content.
The content length is limited by default to 2 MiB; for larger content see xref:http-client-response-content[].
The `ContentResponse` object contains the HTTP response information: status
code, headers and possibly a content.
The content length is limited by default to 2 MiB; for larger content see
xref:http-client-response-content[].
If you want to customize the request, for example by issuing a HEAD request instead of a GET, and simulating a browser user agent, you can do it in this way:
If you want to customize the request, for example by issuing a HEAD request
instead of a GET, and simulating a browser user agent, you can do it in this
way:
[source, java, subs="{sub-order}"]
----
@ -42,102 +46,128 @@ ContentResponse response = httpClient.newRequest("http://domain.com/path?query")
.method(HttpMethod.HEAD)
.agent("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0")
.send();
----
This is a shorthand for:
[source, java, subs="{sub-order}"]
----
Request request = httpClient.newRequest("http://domain.com/path?query");
request.method(HttpMethod.HEAD);
request.agent("Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0");
ContentResponse response = request.send();
----
You first create a request object using `httpClient.newRequest(...)`, and then you customize it using the fluent API style (that is, chained invocation of methods on the request object).
When the request object is customized, you call `Request.send()` that produces the `ContentResponse` when the request/response conversation is complete.
You first create a request object using `httpClient.newRequest(...)`, and then
you customize it using the fluent API style (that is, chained invocation of
methods on the request object).
When the request object is customized, you call `Request.send()` that produces
the `ContentResponse` when the request/response conversation is complete.
Simple POST requests also have a shortcut method:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.POST("http://domain.com/entity/1")
.param("p", "value")
.send();
----
The POST parameter values are automatically URL-encoded.
The POST parameter values added via the `param()` method are automatically
URL-encoded.
Jetty HTTP client automatically follows redirects, so automatically handles the typical web pattern http://en.wikipedia.org/wiki/Post/Redirect/Get[POST/Redirect/GET], and the response object contains the content of the response of the GET request.
Following redirects is a feature that you can enable/disable on a per-request basis or globally.
Jetty's HTTP client automatically follows redirects, so automatically handles
the typical web pattern http://en.wikipedia.org/wiki/Post/Redirect/Get[POST/Redirect/GET],
and the response object contains the content of the response of the GET request.
Following redirects is a feature that you can enable/disable on a per-request
basis or globally.
File uploads also require one line, and make use of JDK 7′s `java.nio.file` classes:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.newRequest("http://domain.com/upload")
.method(HttpMethod.POST)
.file(Paths.get("file_to_upload.txt"), "text/plain")
.send();
----
It is possible to impose a total timeout for the request/response conversation using the `Request.timeout(...)` method, in this way:
It is possible to impose a total timeout for the request/response conversation
using the `Request.timeout(...)` method, in this way:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.newRequest("http://domain.com/path?query")
.timeout(5, TimeUnit.SECONDS)
.send();
----
In the example above, when the 5 seconds expire, the request is aborted and a `java.util.concurrent.TimeoutException` is thrown.
In the example above, when the 5 seconds expire, the request is aborted and a
`java.util.concurrent.TimeoutException` is thrown.
[[http-client-async]]
==== Asynchronous APIs
==== Non-Blocking APIs
So far we have shown how to use Jetty HTTP client in a blocking style, that is the thread that issues the request blocks until the request/response conversation is complete.
In this section we will look at Jetty HTTP client asynchronous, non-blocking, APIs that are perfectly suited for large content downloads, for parallel processing of requests/responses and in all those cases where performance and efficient thread and resource utilization is a key factor.
So far we have shown how to use Jetty HTTP client in a blocking style, that is
the thread that issues the request blocks until the request/response conversation
is complete.
The asynchronous APIs rely heavily on listeners that are invoked at various staged of request and response processing.
In this section we will look at Jetty's HTTP client non-blocking, asynchronous,
APIs that are perfectly suited for large content downloads, for parallel
processing of requests/responses and in all those cases where performance and
efficient thread and resource utilization is a key factor.
The asynchronous APIs rely heavily on listeners that are invoked at various stages
of request and response processing.
These listeners are implemented by applications and may perform any kind of logic.
The implementation invokes these listeners in the same thread that is used to process the request or response.
Therefore, if the application code in these listeners takes a long time to execute, the request or response processing is delayed until the listener returns.
The implementation invokes these listeners in the same thread that is used to
process the request or response.
Therefore, if the application code in these listeners takes a long time to execute,
the request or response processing is delayed until the listener returns.
If you need to execute application code that takes long time inside a listener, you must spawn your own thread, and remember to deep copy any data provided by the listener that you will need in your code, because when the listener returns the data it provides may be recycled/cleared/destroyed.
If you need to execute application code that takes long time inside a listener,
you must spawn your own thread, and remember to deep copy any data provided by
the listener that you will need in your code, because when the listener returns
the data it provides may be recycled/cleared/destroyed.
Request and response processing are executed by two different threads and therefore may happen concurrently.
A typical example of this concurrent processing is an echo server, where a large upload may be concurrent with the large download echoed back.
As a side note, remember that responses may be processed and completed _before_ requests; a typical example is a large upload that triggers a quick response - for example an error - by the server: the response may arrive and be completed while the request content is still being uploaded.
Request and response processing are executed by two different threads and
therefore may happen concurrently.
A typical example of this concurrent processing is an echo server, where a
large upload may be concurrent with the large download echoed back.
As a side note, remember that responses may be processed and completed _before_
requests; a typical example is a large upload that triggers a quick response -
for example an error - by the server: the response may arrive and be completed
while the request content is still being uploaded.
The application thread that calls `Request.send(CompleteListener)` performs the processing of the request until either the request is fully processed or until it would block on I/O, then it returns (and therefore never blocks).
If it would block on I/O, the thread asks the I/O system to emit an event when the I/O will be ready to continue, then returns.
When such an event is fired, a thread taken from the `HttpClient` thread pool will resume the processing of the request.
The application thread that calls `Request.send(Response.CompleteListener)`
performs the processing of the request until either the request is fully
processed or until it would block on I/O, then it returns (and therefore never
blocks).
If it would block on I/O, the thread asks the I/O system to emit an event when
the I/O will be ready to continue, then returns.
When such an event is fired, a thread taken from the `HttpClient` thread pool
will resume the processing of the request.
Response are processed either from the I/O system thread that fires the event that bytes are ready to be read or by a thread taken from the `HttpClient` thread pool (this is controlled by the `HttpClient.isDispatchIO()` property).
Response processing continues until either the response is fully processed or until it would block for I/O.
If it would block for I/O, the thread asks the I/O system to emit an event when the I/O will be ready to continue, then returns.
When such an event is fired, a thread taken from the `HttpClient` thread pool will resume the processing of the response.
Response are processed from the I/O thread that fires the event that bytes are
ready to be read.
Response processing continues until either the response is fully processed or
until it would block for I/O.
If it would block for I/O, the thread asks the I/O system to emit an event when
the I/O will be ready to continue, then returns.
When such an event is fired, a thread taken from the `HttpClient` thread pool
will resume the processing of the response.
When the request and the response are both fully processed, the thread that finished the last processing (usually the thread that processes the response, but may also be the thread that processes the request - if the request takes more time than the response to be processed) is used to dequeue the next request for the same destination and processes it.
When the request and the response are both fully processed, the thread that
finished the last processing (usually the thread that processes the response,
but may also be the thread that processes the request - if the request takes
more time than the response to be processed) is used to dequeue the next
request for the same destination and processes it.
A simple asynchronous GET request that discards the response content can be written in this way:
A simple asynchronous GET request that discards the response content can be
written in this way:
[source, java, subs="{sub-order}"]
----
httpClient.newRequest("http://domain.com/path")
.send(new Response.CompleteListener()
{
@ -147,69 +177,63 @@ httpClient.newRequest("http://domain.com/path")
// Your logic here
}
});
----
Method `Request.send(Response.CompleteListener)` returns void and does not block; the `Response.CompleteListener` provided as a parameter is notified when the request/response conversation is complete, and the `Result` parameter allows you to access the response object.
Method `Request.send(Response.CompleteListener)` returns `void` and does not
block; the `Response.CompleteListener` provided as a parameter is notified when
the request/response conversation is complete, and the `Result` parameter
allows you to access the response object.
You can write the same code using JDK 8′s lambda expressions:
[source, java, subs="{sub-order}"]
----
httpClient.newRequest("http://domain.com/path")
.send((result) -> { /* Your logic here */ });
.send(result -> { /* Your logic here */ });
----
You can impose a total timeout for the request/response conversation in the same way used by the synchronous API:
You can impose a total timeout for the request/response conversation in the
same way used by the synchronous API:
[source, java, subs="{sub-order}"]
----
Request request = httpClient.newRequest("http://domain.com/path")
.timeout(3, TimeUnit.SECONDS)
.send(new Response.CompleteListener()
{
@Override
public void onComplete(Result result)
{
// Your logic here
}
});
.send(result -> { /* Your logic here */ });
----
The example above will impose a total timeout of 3 seconds on the request/response conversation.
The example above will impose a total timeout of 3 seconds on the request/response
conversation.
The HTTP client APIs use listeners extensively to provide hooks for all possible request and response events, and with JDK 8′s lambda expressions they’re even more fun to use:
The HTTP client APIs use listeners extensively to provide hooks for all possible
request and response events, and with JDK 8′s lambda expressions they’re even
more fun to use:
[source, java, subs="{sub-order}"]
----
httpClient.newRequest("http://domain.com/path")
// Add request hooks
.onRequestQueued((request) -> { ... })
.onRequestBegin((request) -> { ... })
.onRequestQueued(request -> { ... })
.onRequestBegin(request -> { ... })
... // More request hooks available
// Add response hooks
.onResponseBegin((response) -> { ... })
.onResponseHeaders((response) -> { ... })
.onResponseBegin(response -> { ... })
.onResponseHeaders(response -> { ... })
.onResponseContent((response, buffer) -> { ... })
... // More response hooks available
.send((result) -> { ... });
.send(result -> { ... });
----
This makes Jetty HTTP client suitable for HTTP load testing because, for example, you can accurately time every step of the request/response conversation (thus knowing where the request/response time is really spent).
This makes Jetty HTTP client suitable for HTTP load testing because, for example,
you can accurately time every step of the request/response conversation (thus
knowing where the request/response time is really spent).
Have a look at the link:{JDURL}/org/eclipse/jetty/client/api/Request.Listener.html[`Request.Listener`] class to know about request events, and to the link:{JDURL}/org/eclipse/jetty/client/api/Response.Listener.html[`Response.Listener`] class to know about response events.
Have a look at the link:{JDURL}/org/eclipse/jetty/client/api/Request.Listener.html[`Request.Listener`]
class to know about request events, and to the
link:{JDURL}/org/eclipse/jetty/client/api/Response.Listener.html[`Response.Listener`]
class to know about response events.
[[http-client-content]]
==== Content Handling
@ -217,67 +241,67 @@ Have a look at the link:{JDURL}/org/eclipse/jetty/client/api/Request.Listener.ht
[[http-client-request-content]]
===== Request Content Handling
Jetty HTTP client provides a number of utility classes off the shelf to handle request content.
Jetty's HTTP client provides a number of utility classes off the shelf to handle
request content.
You can provide request content as `String`, `byte[]`, `ByteBuffer`, `java.nio.file.Path`, `InputStream`, and provide your own implementation of `org.eclipse.jetty.client.api.ContentProvider`.
You can provide request content as `String`, `byte[]`, `ByteBuffer`,
`java.nio.file.Path`, `InputStream`, and provide your own implementation of
`org.eclipse.jetty.client.api.ContentProvider`.
Here’s an example that provides the request content using `java.nio.file.Paths`:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.newRequest("http://domain.com/upload")
.method(HttpMethod.POST)
.file(Paths.get("file_to_upload.txt"), "text/plain")
.send();
----
This is equivalent to using the `PathContentProvider` utility class:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.newRequest("http://domain.com/upload")
.method(HttpMethod.POST)
.content(new PathContentProvider(Paths.get("file_to_upload.txt")), "text/plain")
.send();
----
Alternatively, you can use `FileInputStream` via the `InputStreamContentProvider` utility class:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.newRequest("http://domain.com/upload")
.method(HttpMethod.POST)
.content(new InputStreamContentProvider(new FileInputStream("file_to_upload.txt")), "text/plain")
.send();
----
Since `InputStream` is blocking, then also the send of the request will block if the input stream blocks, even in case of usage of the asynchronous `HttpClient` APIs.
Since `InputStream` is blocking, then also the send of the request will block
if the input stream blocks, even in case of usage of the asynchronous
`HttpClient` APIs.
If you have already read the content in memory, you can pass it as a `byte[]` using the `BytesContentProvider` utility class:
If you have already read the content in memory, you can pass it as a `byte[]`
using the `BytesContentProvider` utility class:
[source, java, subs="{sub-order}"]
----
byte[] bytes = ...;
ContentResponse response = httpClient.newRequest("http://domain.com/upload")
.method(HttpMethod.POST)
.content(new BytesContentProvider(bytes), "text/plain")
.send();
----
If the request content is not immediately available, but your application will be notified of the content to send, you can use `DeferredContentProvider` in this way:
If the request content is not immediately available, but your application will
be notified of the content to send, you can use `DeferredContentProvider` in
this way:
[source, java, subs="{sub-order}"]
----
DeferredContentProvider content = new DeferredContentProvider();
httpClient.newRequest("http://domain.com/upload")
.method(HttpMethod.POST)
.content(content)
.send(new Response.CompleteListener()
{
@ -300,24 +324,29 @@ content.offer(ByteBuffer.wrap(bytes));
// All content has arrived
content.close();
----
While the request content is awaited and consequently uploaded by the client application, the server may be able to respond (at least with the response headers) completely asynchronously. In this case, `Response.Listener` callbacks will be invoked before the request is fully sent.
This allows fine-grained control of the request/response conversation: for example the server may reject contents that are too big, send a response to the client, which in turn may stop the content upload.
While the request content is awaited and consequently uploaded by the client
application, the server may be able to respond (at least with the response
headers) completely asynchronously. In this case, `Response.Listener` callbacks
will be invoked before the request is fully sent.
This allows fine-grained control of the request/response conversation: for
example the server may reject contents that are too big, send a response to the
client, which in turn may stop the content upload.
Another way to provide request content is by using an `OutputStreamContentProvider`, which allows applications to write request content when it is available to the `OutputStream` provided by `OutputStreamContentProvider`:
Another way to provide request content is by using an `OutputStreamContentProvider`,
which allows applications to write request content when it is available to the
`OutputStream` provided by `OutputStreamContentProvider`:
[source, java, subs="{sub-order}"]
----
OutputStreamContentProvider content = new OutputStreamContentProvider();
// Use try-with-resources to close the OutputStream when all content is written
try (OutputStream output = content.getOutputStream())
{
client.newRequest("localhost", 8080)
.method(HttpMethod.POST)
.content(content)
.send(new Response.CompleteListener()
{
@ -334,8 +363,6 @@ try (OutputStream output = content.getOutputStream())
writeContent(output);
}
// End of try-with-resource, output.close() called automatically to signal end of content
----
[[http-client-response-content]]
@ -343,13 +370,16 @@ try (OutputStream output = content.getOutputStream())
Jetty HTTP client allows applications to handle response content in different ways.
The first way is to buffer the response content in memory; this is done when using the blocking APIs (see xref:http-client-blocking[]) and the content is buffered within a `ContentResponse` up to 2 MiB.
The first way is to buffer the response content in memory; this is done when
using the blocking APIs (see xref:http-client-blocking[]) and the content is
buffered within a `ContentResponse` up to 2 MiB.
If you want to control the length of the response content (for example limiting to values smaller than the default of 2 MiB), then you can use a `org.eclipse.jetty.client.util.FutureResponseListener`in this way:
If you want to control the length of the response content (for example limiting
to values smaller than the default of 2 MiB), then you can use a
`org.eclipse.jetty.client.util.FutureResponseListener`in this way:
[source, java, subs="{sub-order}"]
----
Request request = httpClient.newRequest("http://domain.com/path");
// Limit response content buffer to 512 KiB
@ -358,17 +388,16 @@ FutureResponseListener listener = new FutureResponseListener(request, 512 * 1024
request.send(listener);
ContentResponse response = listener.get(5, TimeUnit.SECONDS);
----
If the response content length is exceeded, the response will be aborted, and an exception will be thrown by method `get()`.
If the response content length is exceeded, the response will be aborted, and
an exception will be thrown by method `get()`.
If you are using the asynchronous APIs (see xref:http-client-async[]), you can use the `BufferingResponseListener` utility class:
If you are using the asynchronous APIs (see xref:http-client-async[]), you can
use the `BufferingResponseListener` utility class:
[source, java, subs="{sub-order}"]
----
httpClient.newRequest("http://domain.com/path")
// Buffer response content up to 8 MiB
.send(new BufferingResponseListener(8 * 1024 * 1024)
@ -383,18 +412,22 @@ httpClient.newRequest("http://domain.com/path")
}
}
});
----
The second way is the most efficient (because it avoids content copies) and allows you to specify a `Response.ContentListener`, or a subclass, to handle the content as soon as it arrives:
The second way is the most efficient (because it avoids content copies) and
allows you to specify a `Response.ContentListener`, or a subclass, to handle
the content as soon as it arrives.
In the example below, `Response.Listener.Adapter` is a class that implements
both `Response.ContentListener` and `Response.CompleteListener` and can
be passed to `Request.send()`.
Jetty's HTTP client will invoke the `onContent()` method zero or more times
(until there is content), and finally invoke the `onComplete()` method.
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient
.newRequest("http://domain.com/path")
.send(new Response.Listener.Empty()
.send(new Response.Listener.Adapter()
{
@Override
public void onContent(Response response, ByteBuffer buffer)
@ -402,11 +435,10 @@ ContentResponse response = httpClient
// Your logic here
}
});
----
The third way allows you to wait for the response and then stream the content using the `InputStreamResponseListener` utility class:
The third way allows you to wait for the response and then stream the content
using the `InputStreamResponseListener` utility class:
[source, java, subs="{sub-order}"]
----
@ -419,7 +451,7 @@ httpClient.newRequest("http://domain.com/path")
Response response = listener.get(5, TimeUnit.SECONDS);
// Look at the response
if (response.getStatus() == 200)
if (response.getStatus() == HttpStatus.OK_200)
{
// Use try-with-resources to close input stream.
try (InputStream responseContent = listener.getInputStream())
@ -427,6 +459,4 @@ if (response.getStatus() == 200)
// Your logic here
}
}
----

90
jetty-documentation/src/main/asciidoc/development/clients/http/http-client-authentication.adoc Normal file
View File

@ -0,0 +1,90 @@
// ========================================================================
// Copyright (c) 1995-2016 Mort Bay Consulting Pty. Ltd.
// ========================================================================
// All rights reserved. This program and the accompanying materials
// are made available under the terms of the Eclipse Public License v1.0
// and Apache License v2.0 which accompanies this distribution.
//
// The Eclipse Public License is available at
// http://www.eclipse.org/legal/epl-v10.html
//
// The Apache License v2.0 is available at
// http://www.opensource.org/licenses/apache2.0.php
//
// You may elect to redistribute this code under either of these licenses.
// ========================================================================
[[http-client-authentication]]
==== Authentication Support
Jetty's HTTP client supports the "Basic" and "Digest" authentication mechanisms
defined by https://tools.ietf.org/html/rfc7235[RFC 7235].
You can configure authentication credentials in the HTTP client instance as follows:
[source, java, subs="{sub-order}"]
----
URI uri = new URI("http://domain.com/secure");
String realm = "MyRealm";
String user = "username";
String pass = "password";
// Add authentication credentials
AuthenticationStore auth = httpClient.getAuthenticationStore();
auth.addAuthentication(new BasicAuthentication(uri, realm, user, pass));
ContentResponse response = httpClient
.newRequest(uri)
.send()
.get(5, TimeUnit.SECONDS);
----
Jetty's HTTP client tests authentication credentials against the challenge(s)
the server issues, and if they match it automatically sends the right
authentication headers to the server for authentication.
If the authentication is successful, it caches the result and reuses it for
subsequent requests for the same domain and matching URIs.
The HTTP conversation for a successful match is the following:
----
Application HttpClient Server
| | |
|--- GET ---|------------ GET ----------->|
| | |
| |<-- 401 + WWW-Authenticate --|
| | |
| |--- GET + Authentication --->|
| | |
|<-- 200 ---|------------ 200 ------------|
----
The application does not receive events related to the response with code 401,
they are handled internally by `HttpClient` which produces a request similar
to the original but with the correct `Authorization` header, and then relays
the response with code 200 to the application.
Successful authentications are cached, but it is possible to clear them in
order to force authentication again:
[source, java, subs="{sub-order}"]
----
httpClient.getAuthenticationStore().clearAuthenticationResults();
----
Authentications may be preempted to avoid the additional roundtrip due to the
server challenge in this way:
[source, java, subs="{sub-order}"]
----
AuthenticationStore auth = httpClient.getAuthenticationStore();
URI uri = URI.create("http://domain.com/secure");
auth.addAuthenticationResult(new BasicAuthentication.BasicResult(uri, "username", "password"));
----
In this way, the original request is enriched by `HttpClient` immediately with
the `Authorization` header, and the server should respond with a 200 and the
resource content rather than with the 401 and the challenge.
See also the <<http-client-proxy-authentication,proxy authentication section>>
for further information about how authentication works with HTTP proxies.

95
jetty-documentation/src/main/asciidoc/development/clients/http/http-client-cookie.adoc Normal file
View File

@ -0,0 +1,95 @@
// ========================================================================
// Copyright (c) 1995-2016 Mort Bay Consulting Pty. Ltd.
// ========================================================================
// All rights reserved. This program and the accompanying materials
// are made available under the terms of the Eclipse Public License v1.0
// and Apache License v2.0 which accompanies this distribution.
//
// The Eclipse Public License is available at
// http://www.eclipse.org/legal/epl-v10.html
//
// The Apache License v2.0 is available at
// http://www.opensource.org/licenses/apache2.0.php
//
// You may elect to redistribute this code under either of these licenses.
// ========================================================================
[[http-client-cookie]]
==== Cookies Support
Jetty HTTP client supports cookies out of the box.
The `HttpClient` instance receives cookies from HTTP responses and stores them
in a `java.net.CookieStore`, a class that is part of the JDK.
When new requests are made, the cookie store is consulted and if there are
matching cookies (that is, cookies that are not expired and that match domain
and path of the request) then they are added to the requests.
Applications can programmatically access the cookie store to find the cookies
that have been set:
[source, java, subs="{sub-order}"]
----
CookieStore cookieStore = httpClient.getCookieStore();
List<HttpCookie> cookies = cookieStore.get(URI.create("http://domain.com/path"));
----
Applications can also programmatically set cookies as if they were returned from a HTTP response:
[source, java, subs="{sub-order}"]
----
CookieStore cookieStore = httpClient.getCookieStore();
HttpCookie cookie = new HttpCookie("foo", "bar");
cookie.setDomain("domain.com");
cookie.setPath("/");
cookie.setMaxAge(TimeUnit.DAYS.toSeconds(1));
cookieStore.add(URI.create("http://domain.com"), cookie);
----
Cookies may be added only for a particular request:
[source, java, subs="{sub-order}"]
----
ContentResponse response = httpClient.newRequest("http://domain.com/path")
.cookie(new HttpCookie("foo", "bar"))
.send();
----
You can remove cookies that you do not want to be sent in future HTTP requests:
[source, java, subs="{sub-order}"]
----
CookieStore cookieStore = httpClient.getCookieStore();
URI uri = URI.create("http://domain.com");
List<HttpCookie> cookies = cookieStore.get(uri);
for (HttpCookie cookie : cookies)
cookieStore.remove(uri, cookie);
----
If you want to totally disable cookie handling, you can install a
`HttpCookieStore.Empty` instance in this way:
[source, java, subs="{sub-order}"]
----
httpClient.setCookieStore(new HttpCookieStore.Empty());
----
You can enable cookie filtering by installing a cookie store that performs the
filtering logic in this way:
[source, java, subs="{sub-order}"]
----
httpClient.setCookieStore(new GoogleOnlyCookieStore());
public class GoogleOnlyCookieStore extends HttpCookieStore
{
@Override
public void add(URI uri, HttpCookie cookie)
{
if (uri.getHost().endsWith("google.com"))
super.add(uri, cookie);
}
}
----
The example above will retain only cookies that come from the `google.com`
domain or sub-domains.

89
jetty-documentation/src/main/asciidoc/development/clients/http/http-client-intro.adoc
View File

@ -17,34 +17,57 @@
[[http-client-intro]]
=== Introduction
The Jetty HTTP client module provides easy-to-use APIs, utility classes and a high performance, asynchronous implementation to perform HTTP and HTTPS requests.
The Jetty HTTP client module provides easy-to-use APIs and utility classes to
perform HTTP (or HTTPS) requests.
The Jetty HTTP client module requires Java version 1.7 or superior and it is Java 1.8 lambda compliant, that is, Java 1.8 applications can use lambda expressions in many of the HTTP client APIs.
Jetty's HTTP client is non-blocking and asynchronous. It offers an asynchronous
API that never blocks for I/O, making it very efficient in thread utilization
and well suited for high performance scenarios such as load testing or parallel
computation.
Jetty HTTP client is implemented and offers an asynchronous API, that is a programming interface that never blocks for I/O events, thus making it very efficient in thread utilization and well suited for load testing and parallel computation.
However, when all you need to do is to perform a GET request to a resource,
Jetty's HTTP client offers also a synchronous API, that is a programming interface
where the thread that issued the request blocks until the request/response
conversation is complete.
However, sometimes all you need to do is to perform a GET request to a resource, and Jetty HTTP client offers also a synchronous API, that is a programming interface where the thread that issued the request blocks until the request/response conversation is complete.
Jetty's HTTP client supports different transports: HTTP/1.1, FastCGI and HTTP/2.
This means that the semantic of a HTTP request (that is, "GET me the resource
`/index.html`") can be carried over the network in different formats.
The most common and default format is HTTP/1.1.
However, Jetty's HTTP client can carry the same request using the FastCGI format
or the new HTTP/2 format.
The FastCGI transport is heavily used in Jetty's <<fastcgi,FastCGI support>>
that allows Jetty to work as a reverse proxy to PHP (exactly like Apache or
Nginx do) and therefore be able to serve - for example - WordPress websites.
The HTTP/2 transport allows Jetty's HTTP client to perform requests using HTTP/2
to HTTP/2 enabled web sites, see also Jetty's <<http2,HTTP/2 support>>.
Out of the box features that you get with the Jetty HTTP client are:
* Redirect support; redirect codes such as 302 or 303 are automatically followed
* Cookies support; cookies sent by servers are stored and sent back to servers in matching requests
* Authentication support; HTTP "Basic" and "Digest" authentications are supported, others are pluggable
* Forward proxy support
* Cookies support; cookies sent by servers are stored and sent back to servers
in matching requests
* Authentication support; HTTP "Basic" and "Digest" authentications are supported,
others are pluggable
* Forward proxy support; HTTP proxying and SOCKS4 proxying.
[[http-client-init]]
==== Initialization
==== Starting HttpClient
The main class is named, as in Jetty 7 and Jetty 8, `org.eclipse.jetty.client.HttpClient` (although it is not backward compatible with the same class in Jetty 7 and Jetty 8).
The main class is named `org.eclipse.jetty.client.HttpClient`.
You can think of an `HttpClient` instance as a browser instance.
Like a browser, it can make requests to different domains, it manages redirects, cookies and authentication, you can configure it with a proxy, and it provides you with the responses to the requests you make.
You can think of a `HttpClient` instance as a browser instance.
Like a browser, it can make requests to different domains, it manages
redirects, cookies and authentication, you can configure it with a proxy, and
it provides you with the responses to the requests you make.
In order to use `HttpClient`, you must instantiate it, configure it, and then start it:
In order to use `HttpClient`, you must instantiate it, configure it, and then
start it:
[source, java, subs="{sub-order}"]
----
// Instantiate HttpClient
HttpClient httpClient = new HttpClient();
@ -53,20 +76,29 @@ httpClient.setFollowRedirects(false);
// Start HttpClient
httpClient.start();
----
You can create multiple instances of `HttpClient`; the reason to do this is that you want to specify different configuration parameters (for example, one instance is configured with a forward proxy while another is not), or because you want the two instances to behave like two different browsers and hence have different cookies, different authentication credentials and so on.
You may create multiple instances of `HttpClient`, but typically one instance
is enough for an application.
The reason to create multiple instances of `HttpClient` is that you want to
specify different configuration parameters (for example, one instance is
configured with a forward proxy while another is not), or because you want the
two instances to behave like two different browsers and hence have different
cookies, different authentication credentials and so on, or because you want to
use different transports.
When you create a `HttpClient` instance using the parameterless constructor, you will only be able to perform plain HTTP requests, and you will not be able to perform HTTPS requests.
When you create a `HttpClient` instance using the parameterless constructor,
you will only be able to perform plain HTTP requests, and you will not be able
to perform HTTPS requests.
In order to perform HTTPS requests, you should create first a link:{JDURL}/org/eclipse/jetty/util/ssl/SslContextFactory.html[`SslContextFactory`], configure it, and pass it to `HttpClient`'s constructor.
When created with a `SslContextFactory`, the `HttpClient` will be able to perform both HTTP and HTTPS requests to any domain.
In order to perform HTTPS requests, you should create first a
link:{JDURL}/org/eclipse/jetty/util/ssl/SslContextFactory.html[`SslContextFactory`],
configure it, and pass it to `HttpClient`'s constructor.
When created with a `SslContextFactory`, the `HttpClient` will be able to perform
both HTTP and HTTPS requests to any domain.
[source, java, subs="{sub-order}"]
----
// Instantiate and configure the SslContextFactory
SslContextFactory sslContextFactory = new SslContextFactory();
@ -78,6 +110,19 @@ httpClient.setFollowRedirects(false);
// Start HttpClient
httpClient.start();
----
==== Stopping HttpClient
It is recommended that when your application stops, you also stop the `HttpClient`
instance (or instances) that you are using.
[source, java, subs="{sub-order}"]
----
httpClient.stop();
----
Stopping `HttpClient` makes sure that the memory it holds (for example,
authentication credentials, cookies, etc.) is released, and that the thread
pool and scheduler are properly stopped allowing all threads used by
`HttpClient` to exit.

167
jetty-documentation/src/main/asciidoc/development/clients/http/http-client-other.adoc
View File

@ -1,167 +0,0 @@
// ========================================================================
// Copyright (c) 1995-2016 Mort Bay Consulting Pty. Ltd.
// ========================================================================
// All rights reserved. This program and the accompanying materials
// are made available under the terms of the Eclipse Public License v1.0
// and Apache License v2.0 which accompanies this distribution.
//
// The Eclipse Public License is available at
// http://www.eclipse.org/legal/epl-v10.html
//
// The Apache License v2.0 is available at
// http://www.opensource.org/licenses/apache2.0.php
//
// You may elect to redistribute this code under either of these licenses.
// ========================================================================
[[http-client-other]]
=== Other Features
[[http-client-cookie]]
==== Cookies Support
Jetty HTTP client supports cookies out of the box.
The `HttpClient` instance receives cookies from HTTP responses and stores them in a `java.net.CookieStore`, a class that is part of the JDK.
When new requests are made, the cookie store is consulted and if there are matching cookies (that is, cookies that are not expired and that match domain and path of the request) then they are added to the requests.
Applications can programmatically access the cookie store to find the cookies that have been set:
[source, java, subs="{sub-order}"]
----
CookieStore cookieStore = httpClient.getCookieStore();
List<HttpCookie> cookies = cookieStore.get(URI.create("http://domain.com/path"));
----
Applications can also programmatically set cookies as if they were returned from a HTTP response:
[source, java, subs="{sub-order}"]
----
CookieStore cookieStore = httpClient.getCookieStore();
HttpCookie cookie = new HttpCookie("foo", "bar");
cookie.setDomain("domain.com");
cookie.setPath("/");
cookie.setMaxAge(TimeUnit.DAYS.toSeconds(1));
cookieStore.add(URI.create("http://domain.com"), cookie);
----
You can remove cookies that you do not want to be sent in future HTTP requests:
[source, java, subs="{sub-order}"]
----
CookieStore cookieStore = httpClient.getCookieStore();
URI uri = URI.create("http://domain.com");
List<HttpCookie> cookies = cookieStore.get(uri);
for (HttpCookie cookie : cookies)
cookieStore.remove(uri, cookie);
----
If you want to totally disable cookie handling, you can install a `HttpCookieStore.Empty` instance in this way:
[source, java, subs="{sub-order}"]
----
httpClient.setCookieStore(new HttpCookieStore.Empty());
----
You can enable cookie filtering by installing a cookie store that performs the filtering logic in this way:
[source, java, subs="{sub-order}"]
----
httpClient.setCookieStore(new GoogleOnlyCookieStore());
public class GoogleOnlyCookieStore extends HttpCookieStore
{
@Override
public void add(URI uri, HttpCookie cookie)
{
if (uri.getHost().endsWith("google.com"))
super.add(uri, cookie);
}
}
----
The example above will retain only cookies that come from the `google.com` domain or sub-domains.
[[http-client-authentication]]
==== Authentication Support
Jetty HTTP client supports the "Basic" and "Digest" authentication mechanisms defined by http://www.ietf.org/rfc/rfc2617.txt[RFC 2617].
You can configure authentication credentials in the HTTP client instance as follows:
[source, java, subs="{sub-order}"]
----
URI uri = new URI("http://domain.com/secure");
String realm = "MyRealm";
String user = "username";
String pass = "password";
// Add authentication credentials
AuthenticationStore auth = httpClient.getAuthenticationStore();
auth.addAuthentication(new BasicAuthentication(uri, realm, user, pass));
ContentResponse response = httpClient
.newRequest(uri)
.send()
.get(5, TimeUnit.SECONDS);
----
Jetty HTTP client tests authentication credentials against the challenge(s) the server issues, and if they match it automatically sends the right authentication headers to the server for authentication.
If the authentication is successful, it caches the result and reuses it for subsequent requests for the same domain and matching URIs.
Successful authentications are cached, but it is possible to clear them in order to force authentication again:
[source, java, subs="{sub-order}"]
----
httpClient.getAuthenticationStore().clearAuthenticationResults();
----
[[http-client-proxy]]
==== Proxy Support
Jetty's HTTP client can be configured to use proxies to connect to destinations.
Two types of proxies are available out of the box: a HTTP proxy (provided by class `org.eclipse.jetty.client.HttpProxy`) and a SOCKS 4 proxy (provided by class `org.eclipse.jetty.client.Socks4Proxy`).
Other implementations may be written by subclassing `ProxyConfiguration.Proxy`.
A typical configuration is the following:
[source, java, subs="{sub-order}"]
----
ProxyConfiguration proxyConfig = httpClient.getProxyConfiguration();
HttpProxy proxy = new HttpProxy("proxyHost", proxyPort);
// Do not proxy requests for localhost:8080
proxy.getExcludedAddresses().add("localhost:8080");
// add the new proxy to the list of proxies already registered
proxyConfig.getProxies().add(proxy);
ContentResponse response = httpClient.GET(uri);
----
You specify the proxy host and port, and optionally also the addresses that you do not want to be proxied, and then add the proxy configuration on the `ProxyConfiguration` instance.
Configured in this way, `HttpClient` makes requests to the HTTP proxy (for plain-text HTTP requests) or establishes a tunnel via HTTP CONNECT (for encrypted HTTPS requests).

113
jetty-documentation/src/main/asciidoc/development/clients/http/http-client-proxy.adoc Normal file
View File

@ -0,0 +1,113 @@
// ========================================================================
// Copyright (c) 1995-2016 Mort Bay Consulting Pty. Ltd.
// ========================================================================
// All rights reserved. This program and the accompanying materials
// are made available under the terms of the Eclipse Public License v1.0
// and Apache License v2.0 which accompanies this distribution.
//
// The Eclipse Public License is available at
// http://www.eclipse.org/legal/epl-v10.html
//
// The Apache License v2.0 is available at
// http://www.opensource.org/licenses/apache2.0.php
//
// You may elect to redistribute this code under either of these licenses.
// ========================================================================
[[http-client-proxy]]
==== Proxy Support
Jetty's HTTP client can be configured to use proxies to connect to destinations.
Two types of proxies are available out of the box: a HTTP proxy (provided by
class `org.eclipse.jetty.client.HttpProxy`) and a SOCKS 4 proxy (provided by
class `org.eclipse.jetty.client.Socks4Proxy`).
Other implementations may be written by subclassing `ProxyConfiguration.Proxy`.
A typical configuration is the following:
[source, java, subs="{sub-order}"]
----
ProxyConfiguration proxyConfig = httpClient.getProxyConfiguration();
HttpProxy proxy = new HttpProxy("proxyHost", proxyPort);
// Do not proxy requests for localhost:8080
proxy.getExcludedAddresses().add("localhost:8080");
// add the new proxy to the list of proxies already registered
proxyConfig.getProxies().add(proxy);
ContentResponse response = httpClient.GET(uri);
----
You specify the proxy host and port, and optionally also the addresses that
you do not want to be proxied, and then add the proxy configuration on the
`ProxyConfiguration` instance.
Configured in this way, `HttpClient` makes requests to the HTTP proxy
(for plain-text HTTP requests) or establishes a tunnel via HTTP CONNECT
(for encrypted HTTPS requests).
[[http-client-proxy-authentication]]
==== Proxy Authentication Support
Jetty's HTTP client support proxy authentication in the same way it supports
<<http-client-authentication,server authentication>>.
In the example below, the proxy requires Basic authentication, but the server
requires Digest authentication, and therefore:
[source, java, subs="{sub-order}"]
----
URI proxyURI = new URI("http://proxy.net:8080");
URI serverURI = new URI("http://domain.com/secure");
AuthenticationStore auth = httpClient.getAuthenticationStore();
// Proxy credentials.
auth.addAuthentication(new BasicAuthentication(proxyURI, "ProxyRealm", "proxyUser", "proxyPass"));
// Server credentials.
auth.addAuthentication(new DigestAuthentication(serverURI, "ServerRealm", "serverUser", "serverPass"));
// Proxy configuration.
ProxyConfiguration proxyConfig = httpClient.getProxyConfiguration();
HttpProxy proxy = new HttpProxy("proxy.net", 8080);
proxyConfig.getProxies().add(proxy);
ContentResponse response = httpClient.newRequest(serverURI)
.send()
.get(5, TimeUnit.SECONDS);
----
The HTTP conversation for successful authentications on both the proxy and the
server is the following:
----
Application HttpClient Proxy Server
| | | |
|--- GET -->|------------- GET ------------->| |
| | | |
| |<----- 407 + Proxy-Authn -------| |
| | | |
| |------ GET + Proxy-Authz ------>| |
| | | |
| | |---------- GET --------->|
| | | |
| | |<--- 401 + WWW-Authn ----|
| | | |
| |<------ 401 + WWW-Authn --------| |
| | | |
| |-- GET + Proxy-Authz + Authz -->| |
| | | |
| | |------ GET + Authz ----->|
| | | |
|<-- 200 ---|<------------ 200 --------------|<--------- 200 ----------|
----
The application does not receive events related to the responses with code 407
and 401 since they are handled internally by `HttpClient`.
Similarly to the <<http-client-authentication,authentication section>>, the
proxy authentication result and the server authentication result can be
preempted to avoid, respectively, the 407 and 401 roundtrips.