Commit Graph

197 Commits

Author SHA1 Message Date
Niraj Tolia e8ef5c0665 [JCLOUDS-301] Reduce reflection overhead of Invokable.getParameters()
By caching the results from Invokable.getParameters(), this commit
improves request signing performance (GETs and PUTs) for S3 by >
3X. These performance problems were seen in production and diagnosed
using the YourKit profiler.
2013-09-27 07:49:09 -07:00
Andrew Gaul 5f8961723f Remove BlobStore Map abstractions
These provide little end-user benefit and make evolving the core API
more difficult.  We deprecated these in 1.6.0.
2013-09-25 20:46:31 -07:00
Andrew Gaul 15d9abead0 Skip testDirectory on S3
This test cannot succeed due to the way directories are implemented as
prefix pseudo-files in S3.
2013-08-27 23:01:04 -07:00
Andrew Phillips f6b5b67b5b JCLOUDS-200: Consolidate all feature-specific in the AWS provider
Follow-up to c64c7423
2013-07-26 15:03:12 -04:00
Diwaker Gupta c64c7423cd AWS-S3 configurable temporary signed URL support
Introduces AWSS3BlobRequestSigner, which reuses the
RequestAuthorizeSignature filter for most of the heavy lifting.
Other implementation details based on [1].

Tested with AWSS3BlobSignerLiveTest, in particular,
testSign(Get|Put)UrlWithTime.

Closes JCLOUDS-200

[1] http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html
2013-07-24 15:22:45 -07:00
Andrew Bayer 75385ee57d JCLOUDS-100. Add AWSServerErrorRetryHandler, with an increased back
off time.
2013-06-17 16:48:15 -07:00
Andrew Gaul 64e9a4e4c6 Remove Strings2.replaceAll(String, char, String)
String.replace(String, String) serves this purpose better.
2013-06-03 14:41:18 -07:00
adriancole ed2f18a1af JCLOUDS-31. updated to ASF headers via mvn com.mycila.maven-license-plugin:maven-license-plugin:format 2013-05-16 21:30:37 -07:00
Andrew Gaul fff2fa6e05 Address Checkstyle violations
This commit requires temporarily disabling some checks.
2013-04-17 16:21:21 -07:00
Andrew Phillips 282f3e2a37 Trying to get the test to *pass* now 2013-04-10 13:04:44 +02:00
Andrew Phillips 52301e0a86 Fixing S3ClientMockTest hanging on Java 7 since 843ed57 2013-04-10 12:03:34 +02:00
adriancole f56da43db1 deprecated blobstore interfaces that imply complex internal code 2013-04-09 14:37:34 -07:00
adriancole 053197fe61 deprecated all top-level Async apis and clients 2013-04-09 10:18:31 -07:00
adriancole 36dd831b84 moved all code off use of RestContext except CloudStackContext, which will use it until 1.7 2013-04-09 09:23:31 -07:00
adriancole 88871c9fe9 issue #1501: add Closeable annotation to all top-level apis 2013-04-08 16:13:09 -07:00
Andrew Gaul 371db8d18f Address Xlint warnings 2013-03-13 22:23:19 -07:00
adriancole 1c2090fd08 s3: explicitly set Expect header on methods that need it 2013-03-03 21:09:49 -08:00
Shri Javadekar 7bfa6649ce Make AWS-S3 use the right directory suffix.
Fixes #1336.
2013-02-26 10:18:57 -08:00
Adrian Cole 37f07f98d2 rename TemporaryCredentials to SessionCredentials per aws sdk 2013-02-25 20:02:48 -08:00
Andrew Gaul ff42fdfa13 Address FindBugs warnings 2013-02-24 20:17:50 -08:00
Andrew Gaul b51ce5994a Expose resource creation date 2013-02-18 11:26:44 -08:00
adriancole 6c658f6450 fix issue #1297 put with zero length payload 2013-02-04 10:32:03 -08:00
Adrian Cole 5aa41d07aa Merge pull request #1261 from jclouds/aws-action-names
changed aws apis to use Action key for config instead of more verbose iam names
2013-01-29 05:53:06 -08:00
adriancole b3c3cb0a88 issue #1184 changed aws apis to use Action key for config instead of more verbose iam names 2013-01-29 04:45:42 -08:00
adriancole a51e6a8323 issue #1089 removed default timeout values so that those not using async will have simpler debugging 2013-01-29 04:19:00 -08:00
adriancole 40d9f8bfe7 changes that facilitate synchronous fallbacks and http invocation 2013-01-29 04:09:45 -08:00
Andrew Gaul af963df92d Use Map.entrySet where possible
Improves run-time efficiency and addresses FindBugs warnings.
2013-01-21 23:59:42 -08:00
adriancole be12dc3293 integrated s3 request signing with temporary credentials 2013-01-21 10:16:40 -08:00
Adrian Cole 37e587ee51 changes related to centralizing construction of invokables 2013-01-19 17:17:30 -08:00
Adrian Cole 4980367991 centralize TypeToken creation 2013-01-19 11:31:55 -08:00
adriancole 04fe89b75a update to guava 14.0-rc2 2013-01-18 16:02:25 -08:00
adriancole b3901deeb9 rewrote aws config to use standard op names 2013-01-16 10:54:17 -08:00
Adrian Cole 1ab3f0bb44 accomodate runtime credential changes 2013-01-15 22:26:50 -08:00
Adrian Cole a49a060152 There can be only one retryable predicate; use Predicates2.retry decorator 2013-01-13 20:30:34 -08:00
Adrian Cole 3962b30808 updated to use standard guava Futures and ListenableFuture 2013-01-13 09:31:10 -08:00
Adrian Cole 547f574eea got rid of custom MoreExecutors.sameThreadExecutor 2013-01-12 15:47:58 -08:00
Adrian Cole f1819fe8b9 decorating cacheloader only used in s3 2013-01-12 15:47:51 -08:00
Adrian Cole 015fbf22a2 removed redundant retrying assertion code 2013-01-12 14:04:47 -08:00
Adrian Cole 47c215c63c removed custom guava code for scoped binding and type params 2013-01-08 22:48:10 -08:00
Andrew Gaul 6da0c84101 Ignore ETags with non-hex suffixes
Amazon S3 blobs created via multi-part uploads have these suffixes.
Reference:

https://forums.aws.amazon.com/thread.jspa?messageID=203510
2013-01-08 17:08:06 -08:00
Adrian Cole 2480481835 dependent changes on tests and internal classes 2013-01-08 00:32:45 -08:00
Adrian Cole 3089111621 refactored internal code and tests to use FunctionalReflection 2013-01-07 00:36:31 -08:00
Adrian Cole 74c633f90a moved all providers and test cases from j.l.r.Method -> Invokable 2013-01-05 18:51:38 -08:00
Adrian Cole 42c7037a59 removed unused code or duplicated code; moved code only used once to where its used 2013-01-01 00:08:34 -08:00
Adrian Cole 9745e9dc2f fixed covariance and warnings on ApiMetadata subclasses 2012-12-31 19:40:49 -08:00
Adrian Cole bb953c981e fix issue #1101: Move to Guava 14 BaseEncoding and Hashing 2012-12-31 14:48:31 -08:00
Adrian Cole cddd35bc51 removed unnecessary generic types and made RestAnnotationProcessor immutable 2012-12-30 23:18:59 -08:00
Adrian Cole 13d8b9c402 replaced @Timeout w/explicit properties 2012-12-30 12:31:43 -08:00
Adrian Cole 34156ca090 issue #1107: remove special casing of bucketExists relating to HEAD vs GET 2012-12-29 10:33:02 -08:00
Adrian Cole 9b1872c6ff removed custom code that duplicates functionality of guava fallbacks 2012-12-27 16:54:31 -08:00
Adrian Cole 20c3426fc8 replaced jersey w/Uris & UriTemplates; removed complexity in url encoding and edge cases like matrix params 2012-12-24 14:32:59 -08:00
Adrian Cole 98a36eb3f6 added named annotations corresponding to IAM actions 2012-12-21 12:15:21 -08:00
Andrei Savu 9a085e51a0 Issue 773. Support S3 Multi-Object Delete (API) 2012-11-26 16:11:55 +02:00
Adrian Cole 878ec1bcab undid Serializable and added checkstyle InterfaceIsType:allowMarkerInterfaces=false 2012-11-18 09:23:44 -08:00
Andrew Gaul cca02ae02d Improve use of Splitter.omitEmptyStrings 2012-11-07 22:30:52 -08:00
Andrew Gaul de82496aae Ensure simple boolean expressions via Checkstyle 2012-11-05 11:41:16 -08:00
Andrew Gaul 3d191253ec Enforce use of Collection helpers via Checkstyle
This ensures that we do not repeat the type name.  Some instances remain
due to auto-generated code in dmtf/cim/Cim*.
2012-11-04 20:45:49 -08:00
Andrew Gaul bfa3458b97 Prefer specific asserts where possible 2012-11-04 14:40:24 -08:00
Andrew Gaul 08b9982384 Enforce modifier order via Checkstyle
Also fix violations, via:

find -name \*.java | xargs sed -i 's/final static/static final/'
find -name \*.java | xargs sed -i 's/static private/private static/'
find -name \*.java | xargs sed -i 's/static abstract/abstract static/'
2012-11-01 20:01:30 -07:00
Andrew Gaul 93d69ece2b Enforce newline at end of file via Checkstyle
Also address all warnings, via:

find -name \*.java | while read i; do if [ x`tail -c 1 $i` != x"" ]; then echo >> $i; fi; done
2012-10-29 22:12:44 -07:00
Andrew Gaul 709ca69a2d Enforce that all imports are used via Checkstyle
Also remove all unused imports, via:

mvn checkstyle:checkstyle -Dcheckstyle.output.file=/dev/stdout -Dcheckstyle.output.format=plain | tac | awk -v FS=: '/warning/{print "sed -i " $2 "d " $1}' | while read i; do $i; done
2012-10-29 17:11:08 -07:00
Andrew Gaul 28d7a6f143 Prefer immutable over empty collections
The former does not create an object when calling iterator(), as
discussed here:

https://groups.google.com/d/msg/jclouds-dev/l7BviKDXreA/mP6Ow_RYxIgJ
2012-10-20 15:35:29 -07:00
Adrian Cole 081046d4d6 added S3 PagedIterable adapter 2012-10-18 17:27:32 -07:00
Adrian Cole ffa9cd3bb0 corrected guice bindings for swift-based blob signatures and backfilled expect tests 2012-10-10 12:03:23 +02:00
Adrian Cole a77a80f030 Issue 1092: fix hostname when path-based buckets are used in non-default location 2012-09-25 21:22:17 -07:00
Andrew Gaul 0c440c9c8a Fix deleteAndVerifyContainerGone return value
Return true if the container does not exist -- this matches the
behavior of Atmos and Swift.  This allows deleteAndEnsurePathGone to
terminate correctly with S3.
2012-09-19 13:39:02 -07:00
Andrei Savu 49bde19fa0 Add support for Swift TempURL Middleware 2012-09-10 13:45:55 +03:00
Adrian Cole 1abcffb612 Issue 1052:Path incorrect when S3 in path-mode and servicePath is default 2012-08-01 11:35:10 -07:00
Andrew Gaul 562f8277d3 Introduce LocalAsyncBlobStore
This commit unifies the filesystem and transient blobstores.
2012-07-23 09:20:41 -07:00
Adrian Cole d5f9bb3172 Merge pull request #739 from andrewgaul/prefer-valueof
Prefer valueOf over explicit object creation
2012-07-22 22:00:09 -07:00
Andrew Gaul 985cccff9a Prefer valueOf over explicit object creation
This allows use of cached values.  Patched with:

find -name \*.java | xargs sed -i 's/new Boolean(false)/Boolean.FALSE/g'
find -name \*.java | xargs sed -i 's/new Boolean(true)/Boolean.TRUE/g'
find -name \*.java | xargs sed -i 's/new Boolean(/Boolean.valueOf(/g'
find -name \*.java | xargs sed -i 's/new Integer(/Integer.valueOf(/g'
find -name \*.java | xargs sed -i 's/new Long(/Long.valueOf(/g'
2012-07-22 21:01:46 -07:00
Andrew Gaul 0bc5c69a4f Move copy(MutableBlobMetadata) to BlobStoreUtils 2012-07-22 17:37:18 -07:00
Andrew Gaul a40983a39d Prefer InputSupplier helpers
These ensure that inputs are closed properly.

Updated with: find -name \*.java | xargs sed -i
's/toStringAndClose(\(.*\)\.getInput())/toString(\1)/'
2012-07-18 17:42:48 -07:00
Adrian Cole 05fd64af44 cleaned up http builders and expect tests 2012-07-15 01:50:38 -07:00
vijaykiran 41fe8b9e0e Issue-1010 - AWS ACL LogDelivery URI is updated 2012-07-10 01:03:16 +02:00
Adrian Cole 74b404eab8 Issue 1008:aws-s3 could not get location for region eu-west-1 error 2012-07-07 17:28:44 -07:00
Jeremy Whitlock 2702e942d2 Issue 992: Fix AWS S3 to work with non-DNS, but still valid, named buckets.
Prior to this commit, jclouds wouldn't allow you to interact with any buckets
in S3 that were named with uppercase characters.  Per AWS S3 docs, this
non-standard naming is valid in US regions only.  This update fixes jclouds so
that it can interact with, and even attempt to create, buckets with uppercase
characters for AWS S3 without actually impacting other S3 implementations.  This
fix also will not have any impact in non-US regions other than instead of a
bucket name validation error you'll get an InvalidBucketName error back from
AWS S3 when you attempt to create a bucket with an uppercase character in a
non-US region.  To summarize, nothing changes other than US regions now allow
creation of bucket names with upper case characters and jclouds now can
interact with these non-standard named buckets without failure.
2012-06-21 17:05:20 -06:00
vijaykiran 6d74805dd9 Issue-953 Make sure that port is set when modifying headers 2012-06-01 23:02:56 +02:00
Aled Sage cd9c830c5a Issue 647: store Expires as Date; added ContentMetadataCodec for converting to/from HTTP headers 2012-05-17 11:24:50 +01:00
Aled Sage 9aedf7b6f6 Issue 647: added "Expires" header for ContentMetadata 2012-05-17 11:24:45 +01:00
Andrew Gaul 74440a7a87 Eliminate Throwables2.propagateOrNull
Throwables.propagate always propagates the Throwable; there is no need
to return null.
2012-05-02 13:51:36 -07:00
Adrian Cole df7d2acbb4 wrapper -> view 2012-04-23 19:14:36 -07:00
Adrian Cole 886d2cafb9 created base Context class with ProviderMetadata and Identity 2012-04-23 16:16:51 -07:00
Adrian Cole dab1dd69fc fix covariance/assignability in RestContext<? extends Client> subclasses 2012-04-18 15:05:39 -07:00
Adrian Cole 0b59dea073 removed ContextBuilder subclasses; added unwrap; undid generic params 2012-04-16 00:44:20 -07:00
Adrian Cole 23cb01f977 refactor PropertiesBuilder and related types into ApiMetadata and ProviderMetadata, and update all usage 2012-04-11 19:47:09 -06:00
Andrew Gaul b47860afc1 Use inference to elide duplicated type parameters
Found with:
grep 'Builder.*<.*>.*Immutable.*<.*>.*builder();'
2012-04-08 22:32:04 -07:00
Adrian Cole 764e0907f7 Issue 657:ApiMetadata 2012-04-01 18:43:31 -07:00
Adrian Cole 5c6ce9f081 organize imports 2012-03-28 03:37:48 -04:00
Andrew Gaul e2417605a2 Prefer Guava.nullToEmpty and HttpUtils.nullToEmpty
Make HttpUtils.valueOrEmpty static and change method name to
nullToEmpty to mimic Guava.
2012-03-19 20:48:52 -07:00
Andrew Gaul 74862d7321 Prefer ImmutableCollection over mutable array
Also static.
2012-03-19 19:04:42 -07:00
Adrian Cole 3e9c566edc aws related changes for issue 826 2012-02-03 00:29:41 -08:00
Adrian Cole 55975af60d Issue 820:Move off PropertiesBuilder for rocoto 2012-01-29 23:11:37 +08:00
Adrian Cole f93c495c43 refactored to use consistent setup*Properties methods like expect tt tests 2012-01-29 20:36:24 +08:00
andreisavu f27bed524c Putting basic blocks in place for writing Expect tests for aws-s3 2012-01-14 14:05:33 +02:00
andreisavu bf330f6348 Use listBucket to query StorageClass for a blob 2012-01-11 23:02:36 +02:00
andreisavu f90a2bd039 Issue 681. Enhance jcloud to support aws-s3 Reduce Redundancy Storage (RRS) 2012-01-11 22:18:28 +02:00
Adrian Cole 5000270090 grammar 2012-01-10 12:33:45 -08:00
Adrian Cole da96b4637b fixed method name 2012-01-10 12:28:50 -08:00
Adrian Cole bdf8725b7a first s3 expect tests 2012-01-10 12:21:26 -08:00