Commit Graph

279 Commits

Author SHA1 Message Date
Oleg Kalnichevski 1d9d6d70c9 Reverted changes to handling of cached responses with variants and different ETag values (f3f07a30); test code cleanups 2023-12-13 11:54:41 +01:00
Arturo Bernal e26896596f HTTPCLIENT-2277: Improve Age Header Handling and Calculation in Accordance with RFC9111.
This commit enhances the processing and calculation of the "Age" response header field to comply fully with Sections 5.1 and 4.2.3 of RFC9111.
2023-12-13 11:54:41 +01:00
Oleg Kalnichevski f8eb716c11 Corrected matchers for cache unit tests 2023-12-13 11:54:41 +01:00
Oleg Kalnichevski 6faef71798 Support methods for parsing a message header into a sequence of tokens 2023-12-13 11:54:41 +01:00
Arturo Bernal 5ba0701535 HTTPCLIENT-2277: Update Freshness Lifetime Calculation (RFC 9111 4.2.1)
This commit enhances the getFreshnessLifetime() method in the CacheValidityPolicy class to better comply with RFC 9111 4.2.1. The method now accounts for a negative Duration between the Date and Expires header fields.
2023-12-13 11:54:41 +01:00
Oleg Kalnichevski 65ec9a8e7a HTTPCLIENT-2277: do not store hop-by-hop and connection specific headers in cache (RFC 9111 3.1) 2023-12-13 11:54:40 +01:00
Oleg Kalnichevski ec6b6630a2 Test code cleanup (no functional changes) 2023-12-13 11:54:40 +01:00
Oleg Kalnichevski 65ef10f9f6 HTTPCLIENT-2277: optimized response header merge upon SC_NOT_MODIFIED 2023-12-13 11:54:40 +01:00
Oleg Kalnichevski d58320b4c2 HTTPCLIENT-2277: Redesign of the internal cache serialization format 2023-12-13 11:54:40 +01:00
Arturo Bernal b7a39b3040 Enhancements to HTTP Conditional Request Handling:
1. Added support for HTTP 412 (Precondition Failed) status code. This status code is returned when one or more conditions given in the request header fields evaluated to false when tested on the server.
2. Ensured all instances of `If-Match` and `If-None-Match` headers are considered as required by the specification.
3. Added handling for cases where a weak ETag is used with a range request.
2023-12-13 11:54:40 +01:00
Oleg Kalnichevski 38b8398a20 Removed HttpCacheSupport from public API 2023-12-13 11:54:40 +01:00
Oleg Kalnichevski fbed77880b Parse Cache-Control request and response headers only once 2023-12-13 11:54:40 +01:00
Oleg Kalnichevski cd2930af1f * Caching protocol classes to use CacheControlHeaderParser to parse request and response cache control directives
* Several protocol recommendations related to Cache-Control header composition no longer mentioned by RFC 7234 have been removed
* Cleanup of header constants
2023-12-13 11:54:40 +01:00
Oleg Kalnichevski 3f81f21cab Request / response Cache-Control APIs 2023-12-13 11:54:39 +01:00
Arturo Bernal f0d76de66d Add heuristic expiration warning to cached responses"
Implemented an enhancement to add a heuristic expiration warning to cached responses when the freshness lifetime and current age are both greater than one day. This warning, indicated by the '113' code, helps in identifying potential staleness in cached data and aligns with the HTTP caching specifications.
2023-12-13 11:54:39 +01:00
Arturo Bernal 46fe5a6a81 Refactor CacheControl parser to handle multiple headers.
This commit refactors the CacheControl parsing logic to handle multiple "Cache-Control" headers. The previous implementation treated each header independently, returning an array of CacheControl objects. This caused issues when headers had directives that should be combined into a single CacheControl object.
The updated implementation combines all directives from all headers into a single CacheControl object, ensuring accurate representation of the caching directives.
2023-12-13 11:54:39 +01:00
Arturo Bernal cf7b582d6e Handle "no-cache" directive with specific header fields.
This commit enhances the handling of the "no-cache" directive when used with specified header fields. The changes include:

* Updated the caching module to correctly identify and handle "no-cache" directives with specific header fields. This was done by modifying the `responseContainsNoCacheDirective` method.
* Modified the `handleCacheHit` method to ensure that a cached entry is revalidated with the origin server when a "no-cache" directive with specific header fields is present in the response.
* Ensured that the rest of the response is still cacheable when the specified header fields are present, as long as the response complies with other caching requirements.
2023-12-13 11:54:39 +01:00
Arturo Bernal 0db4f4fa9e Handling for 304 Not Modified responses in CachingHttpAsyncClient and c. When a 304 response is received, the cache entry is updated and the updated entry is used to generate the response. 2023-12-13 11:54:39 +01:00
Arturo Bernal 5f6d370ccd HTTPCLIENT-1920: Add a check in CachedResponseSuitabilityChecker to ensure that cache entries created by HEAD requests are not used to serve GET requests. 2023-12-13 11:54:39 +01:00
Arturo Bernal b9e2bbc778 Add Last-Modified header to 304 response when ETag is not present
.

This commit adds the Last-Modified header to the 304 Not Modified response when the ETag header is not present in the cache entry. This aligns the behavior with the recommendations in RFC 7232 and helps clients that rely on the Last-Modified header for cache updates when
2023-12-13 11:54:39 +01:00
Arturo Bernal e1cfb2add6 Complete the implementation of stale-if-error support as per RFC 5861
- Updated handling of IOExceptions to serve stale responses when allowed by stale-if-error
2023-12-13 11:54:39 +01:00
Arturo Bernal 7bf84b71d4 Extend stale-if-error to apply to non-revalidatable cache entries.
The stale-if-error Cache-Control directive is used to indicate that a cached response can be used to satisfy a request even when an error occurs, as long as the response is still fresh or within the specified staleness limit. However, in the current implementation, this directive is only applied to cache entries that are revalidatable, meaning they have an ETag or Last-Modified header and can be refreshed with a conditional request.

This commit extends the stale-if-error directive to apply to any stale cache entry, whether revalidatable or not. This ensures that clients will continue to receive a cached response even if the original request resulted in an error, and helps to reduce the load on origin servers.
2023-12-13 11:54:38 +01:00
Arturo Bernal b915a3ab33 Improve HttpByteArrayCacheEntrySerializer class by adding new methods and enhancing performance.
This commit adds several new constant values to the HttpByteArrayCacheEntrySerializer class to manage cache entry headers. The HttpByteArrayCacheEntrySerializer class was also updated with a new constructor to set the buffer size and a new method to serialize HTTP cache storage entry objects. Additionally, the code was refactored to enhance performance and remove unnecessary variables.
2023-12-13 11:54:38 +01:00
Arturo Bernal f190e4f52e Avoid using deprecated DateUtils. 2023-12-13 11:54:38 +01:00
Arturo Bernal 4784fdfed4 Fix issue with duplicate parsing of Cache-Control header
Previously, the same Cache-Control header was being parsed twice, once by isExplicitlyNonCacheable and again by calculateFreshnessLifetime. The parsing code was extracted from calculateFreshnessLifetime and enhanced to include the main cache control directive that isExplicitlyNonCacheable could use to make its decision. This improves the efficiency and accuracy of the caching logic.
2023-12-13 11:54:38 +01:00
Arturo Bernal cfcdd11cb6 Allow certain responses from a POST to be cacheable.
Update ResponseCachingPolicy to allow caching of responses to POST requests under certain circumstances, as specified in RFC 2616 and explained in more detail in draft-ietf-httpbis-p2-semantics-20#section-2.3.4. This change extends the cacheability of responses beyond GET and HEAD methods, improving the cache's efficiency and reducing network traffic.
2023-12-13 11:54:38 +01:00
Arturo Bernal 019cf460ec Fix handling of cached responses with variants and different ETag values.
Previously, the getCacheEntry method was not correctly selecting the matching variant for a given request, which led to incorrect behavior when serving cached responses.
This commit improves the method's logic to correctly identify the cache entry using the request's cache key, and then select the variant with the matching ETag value. If no matching variant is found, the cache entry is considered stale and a new response is fetched from the origin server. The fix includes a new test case to ensure the correct behavior of the method in this scenario
2023-12-13 11:54:38 +01:00
Arturo Bernal d1f46e0c66 Fix Heuristic caching for URIs with query strings by adhering to RFC 2616 Section 13.9. The change is based on the fact that caches MUST NOT treat responses to such URIs as fresh unless the server provides an explicit expiration time. However, the updated RFC 7234 note indicates that it would be ok to introduce a configuration option that relaxed this rule. Therefore, a new configuration option is added to allow HTTP/1.1 responses with query strings to be cached. 2023-12-13 11:54:38 +01:00
Oleg Kalnichevski f7de67ff25 Upgraded project version to 5.4-alpha1 2023-12-13 11:54:30 +01:00
Oleg Kalnichevski f6a37780cf Upgraded HttpClient version to 5.3.1-SNAPSHOT 2023-12-06 16:54:47 +01:00
Oleg Kalnichevski 19ab1111c7 More consistent handling of OperationTimeoutException in MemcachedHttpCacheStorage 2023-09-29 15:24:37 +02:00
Oleg Kalnichevski 9d23ddbea7 Upgraded HttpClient version to 5.3-alpha2-SNAPSHOT 2023-08-19 18:39:58 +02:00
Oleg Kalnichevski 83c6079e65 NTLM scheme deprecated and disabled by default 2023-08-12 17:14:53 +02:00
Oleg Kalnichevski c0194331c3 Upgraded project version to 5.3-alpha1-SNAPSHOT 2023-08-12 17:14:16 +02:00
Gary Gregory 09ae5f212a Mostly missing Javadoc in org.apache.hc.client5.http.entity and minor
clean ups
2023-07-02 16:31:07 -04:00
Oleg Kalnichevski 3e38a5e5ec HTTPCLIENT-2271: Do not optimize the path component of the resolved URI by default. 2023-04-27 17:13:53 +02:00
Michael Lee ec5b1f1d60 Hygiene - remove unused private fields 2023-03-13 15:43:31 +01:00
MartinWitt f198fc0e62 Make inner classes static (#405) 2023-03-13 15:43:24 +01:00
Oleg Kalnichevski 22e5a0be6b Upgraded HttpClient version to 5.2.2-SNAPSHOT 2022-12-07 14:09:11 +01:00
Gary Gregory 290ec22022 Fix compiler warnings in tests
Use try-with-resources, also avoids possible leaks on test failures
2022-11-14 22:13:14 +01:00
Gary Gregory 230a2caf67 Replace deprecated use of LangUtils#equals() with Objects.equals() 2022-11-14 09:57:05 -05:00
Oleg Kalnichevski e52ff02f74 Upgraded HttpClient version to 5.2.1-SNAPSHOT 2022-11-13 12:28:46 +01:00
Gary Gregory dca9108352 Use try-with-resources 2022-07-06 10:56:04 -04:00
Gary Gregory 1cd12fc1dc Cleaning up:
- Super interface already implements FutureCallback
- No need to override methods to only call super
- Add missing Javadoc tag
- Access static field directly
2022-07-06 10:13:23 -04:00
jkmcl 15951d8094 Use HTTP header name constants
Use HTTP header name constants instead of string literals.
2022-06-20 19:25:00 +02:00
Oleg Kalnichevski 8dbaf131f5 Upgraded HttpClient version to 5.2-beta2-SNAPSHOT 2022-06-07 14:42:03 +02:00
j3graham 26dcc6f914 HTTPCLIENT-2218: Use Java 8 Base64 utility (#370) 2022-06-01 23:04:29 +02:00
Oleg Kalnichevski b9a6b5ed89 HTTPCLIENT-2202: MemcachedHttpCacheStorage to support MemcachedClientIF interface 2022-02-10 20:33:59 +01:00
Arturo Bernal 4c464b2432 Avoid redundant validation of '\"'. Its check it before. 2021-12-12 20:50:25 +01:00
Arturo Bernal c091c05b72 Expose method to check the cache is still active. 2021-12-11 12:08:17 +01:00
Arturo Bernal d323e0d684 HTTPCLIENT-2189 - Cookie and Cache APIs to use Java time primitives 2021-11-29 14:21:01 +01:00
Arturo Bernal 0a42d173ef HTTPCLIENT-2186 - Migrate Test to Unit 5. 2021-11-21 19:44:13 +01:00
Arturo Bernal 4d0caa4f42 Fix typo method and class. Deprecate the old ones. 2021-11-12 22:16:52 +01:00
Oleg Kalnichevski 3dd37952bc Use Java 8 Time APIs for date / time operations in the HTTP cache tests 2021-11-09 22:50:00 +01:00
Oleg Kalnichevski 1eb2d40df1 Upgraded HttpClient version to 5.2-alpha2-SNAPSHOT 2021-11-01 18:04:02 +01:00
Oleg Kalnichevski dfc2086d24 Replaced SimpleDateFormat and Calendar with Java 8 Time APIs; removed thread-local from DateUtils 2021-10-24 16:50:07 +02:00
Arturo Bernal 277c7228c3 Change loop that don't loop for "if" conditions. 2021-10-23 18:10:15 +02:00
Gary Gregory b6ae693fe5 Don't initialize AtomicReference to its default value. 2021-10-13 10:36:49 +02:00
Oleg Kalnichevski 5390aef223 Always bind the exchange ID to the execution context 2021-09-29 18:01:46 +02:00
Oleg Kalnichevski aff1d2024c Better exception asserts in unit tests 2021-06-19 21:10:43 +02:00
Oleg Kalnichevski 760795b6df HTTPCLIENT-1244: Replaced EasyMock with Mockito in the HTTP cache unit tests 2021-06-19 16:49:12 +02:00
Arturo Bernal d77112f608 Replace assert calls by simpler but equivalent calls. 2021-06-19 16:47:44 +02:00
Arturo Bernal fde3fca687 Simplify conditions and avoid extra checks.
Inline return variables.
2021-06-19 16:47:41 +02:00
Oleg Kalnichevski 0805cfe582 Updated test cases with exception asserts 2021-05-24 18:53:58 +02:00
Oleg Kalnichevski 82432f50d9 Removed references to deprecated Assert#assertThat 2021-05-24 16:58:33 +02:00
Oleg Kalnichevski 8580d7fddf Redundant type declarations 2021-05-24 14:43:18 +02:00
Oleg Kalnichevski 646257e107 Upgraded project version to 5.2-alpha1-SNAPSHOT 2021-05-24 11:30:08 +02:00
Oleg Kalnichevski bb04d078ad Java 1.8 upgrade 2021-05-11 21:34:49 +02:00
Oleg Kalnichevski 6693c186f4 Upgraded HttpClient version to 5.1.1-SNAPSHOT 2021-05-11 21:03:27 +02:00
Oleg Kalnichevski b151df7e8c HTTPCLIENT-2157: response object generated by the classic caching backend is missing the original content encoding 2021-05-01 17:27:12 +02:00
Oleg Kalnichevski 13137eb6c7 Async clients to support scheduled (delayed) re-execution of requests 2021-04-27 12:57:15 +02:00
Arturo Bernal e0c049060b Fix javadoc 2021-03-28 12:36:14 +02:00
Arturo Bernal 879a063b57 Reuse org.apache.hc.core5.http.Method HTTP spec enum 2021-03-16 12:53:31 +01:00
Arturo Bernal 118e7359a1 Fixed typos javadocs and var names 2021-03-14 16:34:18 +01:00
Oleg Kalnichevski 3de88293fe Deprecated message copiers in favor of generic message builders 2021-03-11 22:47:29 +01:00
Arturo Bernal f2e9ad3b11 Fix Typo 2021-02-10 22:09:20 +01:00
Oleg Kalnichevski 744c9d8020 Upgraded HttpClient version to 5.1-beta2-SNAPSHOT 2021-02-10 22:06:51 +01:00
Oleg Kalnichevski f9ea1b6238 Upgraded project version to 5.1-beta1-SNAPSHOT 2021-02-06 13:28:08 +01:00
Arturo Bernal 0940d35602 Minor Improvement:
* Avoid duplicate code
* Simplify if else
* Inline variables
2021-02-05 22:18:45 +01:00
Arturo Bernal 3c9f1f85e1 Minor Improvements:
* Use Empty collections
* Unnecessary toString
2021-01-26 09:14:46 +01:00
Oleg Kalnichevski b71e2fcb83 No need to copy example sources as resources when generating site content 2021-01-17 11:12:33 +01:00
Hervé Boutemy 3b0a7aeee8 improve documentation format 2021-01-12 09:33:48 +01:00
Arturo Bernal 6a02e818ff Minor Improvements:
* Remove redundant initialization
2021-01-05 20:01:19 +01:00
Arturo Bernal 36e1bde6ff Minor Improvements:
* Add final to variable
* Unnecessary semicolon
* Use Standard Charset object
* Unnecessary conversion to String
* Simplifiable conditional expression
* Replace 'Arrays.asList()' with Collections.singletonList
* Redundant local variable. Simplify
2020-12-31 13:50:06 +01:00
Gary Gregory 400771a1a7 Add missing @Override. 2020-11-24 09:22:25 -05:00
Gary Gregory b7f851104a Use Arrays.fill(). 2020-11-24 09:22:25 -05:00
Gary Gregory 460abd7474 Use try-with-resources. 2020-11-24 09:22:25 -05:00
Oleg Kalnichevski c9489606d5 HTTPCLIENT-2126: `AsyncCachingExec` throws NPE when response body is null 2020-11-05 15:14:07 +01:00
Oleg Kalnichevski 918ac1535f RFC 3986 conformance: corrected handling of path segments by `URIUtils#normalizeSyntax`; optimized path segment operations 2020-09-27 12:36:29 +02:00
Oleg Kalnichevski 9bc49cc439 Fixed 'JdbcRowSetImpl is internal proprietary API and may be removed in a future release' warning 2020-09-27 12:20:32 +02:00
Oleg Kalnichevski 594e5fcefb Upgraded HttpClient version to 5.0.3-SNAPSHOT 2020-09-27 12:12:59 +02:00
Michael Osipov e249e56c14 Add project modules to dependency management 2020-09-22 09:25:43 +02:00
Michael Osipov e13e4ff548 Unify JXR and Surefire report config 2020-09-22 09:25:43 +02:00
Michael Osipov 37c1a9da1d Unify Javadoc report config 2020-09-22 09:25:43 +02:00
Michael Osipov aec050315e Remove default config 2020-09-22 09:25:43 +02:00
Michael Osipov 4b5a2f44a1 Replace HTTP with HTTPS 2020-09-22 09:25:43 +02:00
dirkhenselin 2520590437 Avoid updating Content-Length header in a 304 response.
I observed the following problem: `Transfer-Encoding` and
`Content-Length` headers should be mutually exclusive and because I use
chunked transfer, the `Transfer-Encoding` header is set in the response
while the `Content-Length` header is not. In case of a 304 during a
revalidation, the header contains Content-Length=0. Probably a proxy is
responsible for this, just like the comment "Some well-known proxies
respond with Content-Length=0, when returning 304" in the method
CachedHttpResponseGenerator::addMissingContentLengthHeader is saying. In
CacheEntryUpdater::mergeHeaders the Content-Length=0 is merged into the
cached entry, but the cached entry contains also a `Transfer-Encoding`
header, so in the cached entry these headers aren't mutually exclusive
anymore. Because of the `Transfer-Encoding` header the method
CachedHttpResponseGenerator::addMissingContentLengthHeader isn't fixing
the `Content-Length` header and Content-Length=0 causes returning null
instead of the cached content. IMHO the `Content-Length` header should
not be merged into the cached response in case of a 304, at least if the
cached entry contains a `Transfer-Encoding` header.
2020-08-18 14:54:36 +02:00
Carter Kozak 9866865357 HTTPCLIENT-2096: Migrate instance loggers to static fields
Note that this may change the origin of logging when classes
have been subclassed, as the logger origin will use the class
name that defined the logger where previously the subclass type
would be used. In scenarios where external libraries subclass
httpclient utilities this allows httpclient logging to maintain
the `org.apache.hc` prefix instead of inheriting arbitrary
subclass names.

Using some logging frameworks this may result in reduced churn
when httpclient components are created (new connections, for example)
because loggers are looked up once per class.
2020-07-09 11:34:48 +02:00
Niels Basjes 9a967de60e Add Automatic-Module-Name in manifest so Java9 modular applications can depend on this library 2020-07-07 15:48:49 +02:00
Carter Kozak 84bd290954
HTTPCLIENT-2095: Use slf4j interpolation instead of string concatenation where possible (#232) 2020-07-02 17:51:43 +02:00