When constructing the query path, S3 does not properly handle encoded
paths. For example, if a blob named %20 is to be placed into the blob
store, S3 would end up placing blob named " " (what %20 represents).
This occurs because the S3 provider examines the URI's path portion
(which is presented in a decoded fasion to the caller). After
examining the path, it is not encoded again. Instead, we should call
getRawPath() to avoid this issue.
There are two issues on the decoding path:
1. Given a blob named " ", S3 API will throw a RuntimeException due to
a NULL check -- the key that it uses is NULL to represent the XML
content " " corresponding to the blob name.
2. Given a blob named "%20 ", S3 API will generate a URI for a blob
named "%20%20", which is also incorrect. The correct URI would be
"%2520%20" (escaping the first "%" and " " characters).
The first issue is due to the currentOrNull() helper, which calls
trim() on the string and then compares the string to an empty string.
This means that a blob named " " will be parsed as "" and then
converted to NULL as the result of that method. Passing "null" as the
key then fails in a number of places (notably, appendPath()).
The second issue is due to the appendPath() method in the jclouds Uris
class. The issue here is that appendPath() calls urlDecode() and
passes the result to path(). The path() method, in turn, also calls
urlDecode(). After these transformations, a properly encoded blob of
the form %2520%20 turns into "%20 " and then " " (two spaces). After
these transformations the path is encoded again, resulting in "%20%20"
(which is wrong).
jclouds should not check if the string is encoded, but rather expect
that all strings would be encoded prior to transmission. As part of
that change, we must make sure that no code relies on such behavior
within jclouds. This commit adds a blobstore test to check encoding
pattern for blobs. It also removes the encoding check in the Strings2
class and the related test.
refactor AccountApiExpectedTest to AccountApiMockTest
refactor DatacenterApiExpectTest to DatacenterApiMockTest
refactor SoftwareDescriptionApiExpectTest to SoftwareDescriptionApiMockTest
refactor VirtualGuestApiExpectTest to VirtualGuestApiMockTest
refactor VirtualGuestBlockDeviceTemplateGroupApiExpectTest to VirtualGuestBlockDeviceTemplateGroupApiMockTest
ideally we shouldn't need this function and instead never double
encode strings, but auditing for that is beyond what I have time
for. currently, putBlob(" ") and putBlob("%20") behave the same
way which is arguably incorrect
If an IOException is thrown during the execution of an HttpCommand retry only if the HTTP method is idempotent (i.e. GET, DELETE, PUT). Otherwise the retry could cause unwanted side effects (i.e. creating and leaking multiple new nodes).
* Improves extension namespace configuration to use Guice multibindings
so each provider can cleanly provide their own namespaces.
* Fixes the HPCloud Compute volume attachment namespace and adds the
corresponding live tests.
* Fixes the Rackspace CloudServers UK volume attachment namespace.
- If get timeout of 0 (or negative), then still try once.
- Remove (unlikely) race in retry’s apply(T) where context-switching
delays could cause `before(end)` to return false the first time, even
though the timeout was positive.
- Ensure retries at end of the timeout (e.g. if timeout is 30 secs
and last sleep takes us up to the 30 secs mark, then test again
rather than returning immediately after the sleep!)
- Use `long` for time, rather than `java.util.Date`, for internal
calculations. Deprecates old protected methods that use Date.
ByteSourcePayloadIterator avoids buffering the entire slice like
InputStreamPayloadIterator does. Also rename PayloadIterator to
better reflect its intent and reduce its visibility.
Prefer ByteSourcePayload which offers a superset of its functionality.
Note that ByteArrayPayload implicitly set the contentLength while
users of ByteSourcePayload must do so explicitly.
Some providers, notably Azure, include a byte-order mark in their XML
responses. ParseSax.apply buffers these responses in a String when
users enable trace-level logging to include the response in any thrown
exceptions. InputSource(InputStream) skips these byte-order marks
while InputSource(Reader) does not, yielding a SAXParseException.
This method is dangerous since all ByteSource should provide a new
InputStream on every call to openStream while the method returns the
same InputStream for non-repeatable Payloads.
This avoids unneeded garbage, especially during XML parsing. Replaced
with:
find -name \*.java | xargs sed -i 's/^\( *[^ ]*\) = new StringBuilder();$/\1.setLength(0);/'
Timur Alperovich
Oliver Gondža
Andrew Gaul
Arvind Nadendla
Andrea Turli
Ka-Hing Cheung
Zack Shoylev
Ignasi Barrera
Svetoslav Neykov
Aled Sage
Jeremy Daggett
Adrian Cole
Adrian Cole
Andrew Phillips