Commit Graph

2885 Commits

Author SHA1 Message Date
Richard Downer 98c289721c Recognise CentOS images on AWS Marketplace
CentOS’ officially-supported AMIs are hosted on the AWS Marketplace.
This adds support for those images, recognising the AMI naming
convention and ensuring the OS metadata is parsed correctly and the
correct SSH login name is used.

There is no change to the default jclouds configuration and the
official CentOS images will not be detected by default. To use these
images, you must alter the ami-query properties to include searching
the “AWS Marketplace”, which has an owner ID of 679593333241. You must
also manually log on to the AWS Marketplace, select your chosen CentOS
image, and “subscribe” to it (you can do this by proceeding as if to
launch an image, but stopping after you have agreed to the subscription
and before launching).
2017-11-03 10:23:37 +01:00
Andrew Gaul c0a7938b69 Address Checkstyle violation 2017-10-30 18:31:40 -07:00
Andrew Gaul af05e24ae9 Implement partial prefix support for Atmos
Atmos only supports listing by directories while other blobstores
allow listing via arbitrary prefixes.  Allow requests which list
directories via both prefix and delimiter = "/" to succeed instead of
failing all requests.  Also change a test which specified recursive to
instead be delimiter = "/".  Fixes gaul/s3proxy#244.
2017-10-30 17:51:35 -07:00
Andrew Gaul 0960ea4969 Add mtime support to Atmos directory listing 2017-10-29 17:40:25 -07:00
Andrew Gaul 4b10bcc054 Enable passing Atmos tests
Also clarify why we skip certain tests with SkipException.
2017-10-29 13:43:10 -07:00
Andrew Gaul 29397b516b Add parent to system metadata
Previously this caused spurious test failures.
2017-10-28 11:06:38 -07:00
Andrew Gaul cf2c077836 Handle empty prefix on Atmos 2017-10-28 11:06:25 -07:00
Ignasi Barrera f7b74d95c9 Exclude tier tests on OSX 2017-10-25 08:30:49 +02:00
Andrew Gaul c901bf3a0f JCLOUDS-1337: Swift putBlob portable storage tiers 2017-10-23 21:31:49 -07:00
Andrew Gaul f513bf7c40 JCLOUDS-1337: Atmos putBlob portable storage tiers 2017-10-23 21:31:48 -07:00
Andrew Gaul b93270aef2 JCLOUDS-1337: only set fs tier when non-null
Fixes regression from 61d4be8746.
2017-10-20 20:51:50 -07:00
Andrew Gaul 5448b92ee8 JCLOUDS-1337: only set fs tier when non-null
Fixes regression from 61d4be8746.
2017-10-17 23:14:09 -07:00
Andrew Gaul 89053d9a8b JCLOUDS-1337: S3 putBlob portable storage tiers
Also promote hacky and limited storage class support from aws-s3
provider to s3 api.
2017-10-12 11:16:48 -07:00
Andrew Gaul 61d4be8746 JCLOUDS-1337: filesystem putBlob portable storage tiers 2017-10-12 11:15:14 -07:00
Andrew Gaul 6cf0a63186 Remove stray Javadoc /** 2017-09-27 09:45:35 -07:00
Andrew Gaul 9628a9064e Upgrade to EasyMock 3.4
Also switch from unmaintained easymockclassextension to easymock.
Release notes:

https://github.com/easymock/easymock/releases
2017-09-17 15:45:46 -07:00
Andrew Gaul 6046fc925c Avoid hard-coding rackspace region in swift test 2017-09-09 13:42:59 -07:00
Andrew Gaul 7704893650 Remove Inject on constructors for abstract classes
This is not meaningful since these classes cannot be instantiated.
Found via error-prone.
2017-09-07 23:37:24 -07:00
Ignasi Barrera 1c1f81c373 JCLOUDS-1258: Allow China endpoints in FormSignerV4 2017-09-08 08:12:50 +02:00
Timur Alperovich 6feacb1ee5 Fixes for jclouds openstack swift tests.
The ContainerApiLiveTest class has tests that mostly assume that a
single test container always exists. They collide on changes to the
state of the container. Some of the tests, though, assume the
container does _not_ exist and attempt to create it and subsequently
delete it. The change in this patch makes their behavior uniform with
respect to that test container: assume it exists and if the test tries
to delete it or mutate it, delete the container and create it at the
end of the test.

The fix in ObjectApi is for the Object GET requests. Currently,
jclouds passes "format=json", which is not a supported query parameter
for object GET.

Lastly, ObjectApiLiveTest that test server-side copy should ignore the
X-Openstack-Request-ID header, as its value will change between the
two requests.
2017-09-07 13:25:50 -07:00
Andrew Gaul 97c77541ec Allow empty delimiters with openstack-swift 2017-09-07 12:28:41 -07:00
Andrew Gaul 3a1710d15c Default Swift signer timeout to 15 minutes
This matches other providers.
2017-09-07 11:53:54 -07:00
Andrew Gaul b424a6c164 Alphabetize modules 2017-09-07 07:27:56 -07:00
Andrew Gaul fb584d844d Address BindingToUnqualifiedCommonType warnings
Found via error-prone.
2017-09-07 07:08:28 -07:00
Andrew Gaul d7d28fe7d7 JCLOUDS-1333: Correct JDK 1.8 method overloading
Newer JDK have a different resolution process, likely due to enhanced
target type inference.  Found via:

mvn test -Dmaven.compile.source=1.8 -Dmaven.compile.target=1.8
2017-09-07 01:41:52 -07:00
Andrew Gaul c20afbeda6 Make auto service optional
This makes dependencies consistent and eliminates warnings of the
form:

$M2_HOME/repository/org/apache/jclouds/driver/jclouds-slf4j/2.1.0-SNAPSHOT/jclouds-slf4j-2.1.0-SNAPSHOT.jar(org/jclouds/logging/slf4j/config/SLF4JLoggingModule.class): warning: Cannot find annotation method 'value()' in type 'AutoService': class file for com.google.auto.service.AutoService not found

Reference:

https://github.com/google/auto/tree/master/service#download
2017-08-30 12:27:08 -07:00
Andrew Gaul bfd54049b7 Make auto service optional
This makes dependencies consistent and eliminates warnings of the
form:

$M2_HOME/repository/org/apache/jclouds/driver/jclouds-slf4j/2.1.0-SNAPSHOT/jclouds-slf4j-2.1.0-SNAPSHOT.jar(org/jclouds/logging/slf4j/config/SLF4JLoggingModule.class): warning: Cannot find annotation method 'value()' in type 'AutoService': class file for com.google.auto.service.AutoService not found

Reference:

https://github.com/google/auto/tree/master/service#download
2017-08-30 12:23:49 -07:00
Andrew Gaul a4e38f7c84 Prefer Guava BaseEncoding over JAXB 2017-08-30 11:26:15 -07:00
Andrew Gaul 6bb9c927bb JCLOUDS-1101: Use correct HttpException
Importing javax.xml.ws.http.HTTPException causes Java 9 issues.
2017-08-30 09:50:58 -07:00
Andrew Gaul 8dae432b78 Avoid injecting on final fields
This can cause thread visibility issues.  Found via error-prone.
2017-08-28 21:13:54 -07:00
Andrew Gaul df43b36487 Avoid injecting on final fields
This can cause thread visibility issues.  Found via error-prone.
2017-08-28 21:08:59 -07:00
Andrew Gaul 6b8a81335d JCLOUDS-1225: Address Guava 18 Objects changes
Fixed with:

find -name \*.java | xargs sed -i 's/Objects.[Tt]oStringHelper/More&/g'
find -name \*.java | xargs sed -i 's/Objects.firstNonNull/More&/g'
find -name \*.java | xargs sed -i 's/^\(import com.google.common.base.\)\(Objects.*\)/\1More\2\n\1\2/g'
find -name \*.java | xargs java -jar google-java-format-1.3-all-deps.jar -i --fix-imports-only --skip-sorting-imports
2017-08-24 18:31:07 -07:00
Andrew Gaul 517323fbd5 JCLOUDS-1225: Address Guava 18 MoreExecutors changes
Fixed with:

find -name \*.java | xargs sed -i 's/sameThreadExecutor/newDirectExecutorService/g'
2017-08-22 17:14:22 -07:00
Andrew Gaul a493e2ca2f JCLOUDS-1225: Address Guava 18 Objects changes
Fixed with:

find -name \*.java | xargs sed -i 's/Objects.[Tt]oStringHelper/More&/g'
find -name \*.java | xargs sed -i 's/Objects.firstNonNull/More&/g'
find -name \*.java | xargs sed -i 's/^\(import com.google.common.base.\)\(Objects.*\)/\1More\2\n\1\2/g'
find -name \*.java | xargs java -jar google-java-format-1.3-all-deps.jar -i --fix-imports-only --skip-sorting-imports
2017-08-22 17:12:53 -07:00
Chaithanya Ganta e331a000d1 JCLOUDS-1322: Remove Expect header for requests with empty body
Signed-off-by: Chaithanya Ganta <ganta@adobe.com>
2017-08-21 21:05:02 -07:00
Andrea Turli 7c58f9d77d JCLOUDS-1323: use security group names in openstack nova options
- fix NovaComputeServiceExpectTest
2017-08-09 12:25:53 +02:00
Andrea Turli c61fa9d02b fix NovaComputeServiceExpectTest 2017-08-09 11:15:59 +02:00
Andrew Gaul 3e1e9cbab6 Remove duplicate Nullable import 2017-08-06 18:38:52 -07:00
Andrew Gaul 3ed99ad74b Allow lastModified to be null
Follow on to ec932321bd
2017-08-06 17:38:36 -07:00
Andrea Turli aa11765bee [JCLOUDS-1318] fix based on nodeTerminatePredicate
- wait for serer deletion, before deleting the security group
- rename cleanupServer to cleanupResources
- remove keyPairCache
- better usage of tags to remove securityGroups created by jclouds
- remove keyPair after the creation of a group
- remove test for create unique keypair
- openstack nova re-adding support for existing keypair
- fix securityGroupApi check
- fix other unit tests
- remove ServerPredicates as it is now duplicated
- remove TemplateOptions.securityGroupNames as deprecated
- address commits for ApplyNovaTemplateOptionsCreatesNodesWithGroupEncodedIntoNameAndAddToSet
- fix testCreateNodeWhileUserSpecifiesKeyPairAndUserSpecifiedGroups
2017-07-17 11:32:24 +02:00
Andrew Gaul 0bc935dd57 Remove clojure bindings
These have not seen any development in many years.
2017-07-10 11:39:11 -07:00
Andrew Gaul e446b5b8b4 JCLOUDS-1111: Force overwriting Atmos objects
This avoids a racy delete then write in the portable abstraction.
2017-07-08 15:50:04 -07:00
Svetoslav Neykov 624e4cb6af
JCLOUDS-1307: Invalidate SG on removal even if already externally deleted 2017-06-27 13:32:38 +03:00
Svetoslav Neykov 1d4cb6c392 [JCLOUDS-1306] Fix SG cache invalidation when deleting 2017-06-09 12:04:29 +02:00
Geoff Macartney 28c3c33bf0 Temporarily use a custom annotation instead of @SinceApiVersion.
The intention is to use @SinceApiVersion for this purpose, but that
would affect a number of APIs, and we would want to have good test
coverage before merging that change (in
FormSignerUtils#getAnnotatedApiVersion). However, there is some issue
wth certain tests at resent that means we cannot successfully test
all APIs that make use of @SinceApiVersion in order to assure
ourselves that FormSignerUtils will not introduce some problem.

See https://github.com/jclouds/jclouds/pull/1102#issuecomment-302682049
for details.

This annotation is introduced as a temporary measure in order to
decouple the functionality of FormSignerUtils#getAnnotatedApiVersion
from @SinceApiVersion and the tests in question. It can be removed and
replaced by @SinceApiVersion when those tests are fixed.

Designates that a method overrides the {@link ApiVersion} on the class
with a specific value.
2017-05-29 10:24:52 +02:00
Geoff Macartney ce0a0ad213 Add ModifySubnetAttribute 2017-05-29 10:24:52 +02:00
Andrew Gaul 2b49bf3618 JCLOUDS-1299: Handle % in names during Swift list
uriBuilder.appendPath assumes an encoded path.
2017-05-16 17:29:02 -07:00
ui4j a299a6c1fd JCLOUDS-1297: Fix uploadId format for non-English locale
Add Locale.ENGLISH parameter to String.format(...) to generate same uploadId for different locales.
2017-05-15 20:01:40 +03:00
Stephen Tomkinson 89ae3b4fa6 Add logic from JCLOUDS-1261 to the other 2 types of Aws4 signer. 2017-05-10 10:09:22 -07:00
Tomas Tulka 6452960c72 fix for deleting an invalid file
If the file for delete is invalid (typically a wrong filename) an IOException will be thrown.
2017-05-10 01:02:09 -07:00
Stephen Tomkinson 15d27da739 Resolves JCLOUDS-1261 by ensuring non-standard port numbers are in the host header that's used for the AWSv4 auth calculations. 2017-05-10 00:00:14 -07:00
Andrea Turli 1c6c36b81a JCLOUDS-1269; JCLOUDS-1120: ec2 subnet/securityGroups fixes 2017-05-04 17:40:50 +02:00
Aled Sage fe134cf612 Fix logging in nova: don’t assume logback 2017-05-04 16:48:15 +02:00
Andrew Gaul 2e7de9f850 JCLOUDS-1271: Deprecate signRemoveBlob
URL signing should always use authentication parameters instead of
headers yet no provider supports DELETEs with the former.
2017-04-19 18:08:52 -07:00
Andrew Gaul db00f679ef JCLOUDS-255: Consistently use query string auth
Fixes regressions introduced in
7a110b31ba.
2017-04-19 00:54:33 -07:00
Andrew Gaul 7a110b31ba JCLOUDS-255: Support S3 signed URLs with expiry
Mostly code motion from AWSS3BlobRequestSigner to S3BlobRequestSigner
with some additional cleanups.
2017-04-18 18:51:22 -07:00
John McDonnell 24f961eac2 [JCLOUDS-1128] Added page and pagesize fields to ListProjectOptions.
Cloudstack's listProjects request supports additional fields for page
and pagesize.  These were missing but have been added in as part of this
commit.
2017-04-18 15:05:41 +02:00
John McDonnell 723a732521 [JCLOUDS-1130] Cloudstack fixed projectid parameter in listUsageRecords.
Cloudstack's listUsageRecords response doesn't use camelcase so the projectid
parameter was incorrect.  Updated from projectId to projectid.
2017-04-18 15:05:34 +02:00
Andrew Gaul 5a1c95525f JCLOUDS-1262: Correct US-OH region
Follow on to 04ab255d9f.
2017-04-10 13:11:38 -07:00
Andrew Gaul 93a8ce433b Allow null S3 list MPU lastModified 2017-04-10 12:09:02 -07:00
Mat Mannion d41101df59 JCLOUDS-1264: Swift Unicode multipart manifests
This fixes a bug where previously BindManifestToJsonPayload used the
character length as the ContentLength, instead of the byte length,
which caused issues if the JSON contained multi-byte Unicode
characters.
2017-04-04 05:05:53 -07:00
Ignasi Barrera 04ab255d9f JCLOUDS-1262: Add the Ohio region to AWS 2017-03-29 09:17:41 +02:00
Michelle Zhang 96a1c4427e JCLOUDS-1217/JCLOUDS-1258: Add Canada, London and China regions to AWS 2017-03-29 08:53:07 +02:00
Andrew Gaul 01780db4db Remove erroneous static final from BIG_FILE
Also use preferred bigFile capitalization for member.  Fixes
regression from e058973abc.
2017-03-17 17:41:32 -07:00
Andrew Gaul e058973abc Ensure SOME_CONSTANTS are static final
Found via error-prone.
2017-03-17 16:59:41 -07:00
Andrew Gaul 7767acbec9 Correct invalid comparison
Found via error-prone.
2017-03-17 16:42:38 -07:00
Andrew Donald Kennedy c3c86de9e5 Fix error creating list of volume bindings for Docker 2017-03-17 22:25:45 +01:00
Andrew Gaul 87eda15085 Return more metadata from listMultipartUpload
Include Content-Length and Last-Modified.
2017-03-08 20:35:24 -08:00
Andrew Gaul 49f4617afa JCLOUDS-1101: Upgrade to auto-value 1.4-rc2
This resolves issues using Java 9.  Release notes:

https://github.com/google/auto/releases/tag/auto-value-1.4-rc2
2017-03-07 00:18:21 -08:00
Andrew Gaul ec932321bd Return lastModified and size when listing S3 parts 2017-03-06 18:29:45 -08:00
Ignasi Barrera 459d437af2 JCLOUDS-1249: Fix OpenStack server rollback 2017-03-06 11:24:35 +01:00
Ignasi Barrera 2f70f85c2b Remove all sonatype snapshot repositories 2017-03-03 10:49:48 +01:00
Geert Vanheusden fbf62437c7 JCLOUDS-1218: filesystem get blob without xattrs
Some filesystems, notably Docker and Mac OS X, do not support xattr
which causes an IOException when getting a blob.
2017-03-01 13:41:30 -08:00
Archana Chinnaiah 8649002e03 JCLOUDS-1246: Allow extending SwiftBlobStoreContextModule 2017-03-01 11:45:01 -08:00
Fritz Elfert e26146c6c1 JCLOUDS-1239: Handle gzipped userdata in logging 2017-02-16 10:50:18 +01:00
Geoff Macartney 717b75a34e Fix O(n^2) response time for "list-security-groups" on openstack-nova.
For https://issues.apache.org/jira/browse/JCLOUDS-1235.

This change takes the approach of storing the information about the
overall list of groups within the `SecurityGroupInRegion` when it is
created, so that any subsequent conversion operation has access to all
the groups in the same region as the one to be converted.

It also collapses the functionality of `NovaSecurityGroupToSecurityGroup`,
`SecurityGroupRuleToIpPermission` and `FindSecurityGroupWithNameAndReturnTrue`
all into `NovaSecurityGroupInRegionToSecurityGroup`, and deletes the
now unused-classes SecurityGroupRuleToIpPermission,
NovaSecurityGroupToSecurityGroup and associated tests.
2017-02-13 12:30:18 +01:00
Ignasi Barrera e450670d86 Fix Chef live tests 2017-02-03 00:21:08 +01:00
Ignasi Barrera 4509cdecbb Fix container presence check in filesystem provider 2017-02-01 17:42:28 +01:00
Andrew Gaul 32bb2db06e Propagate error on non-existent container or key 2017-01-18 11:20:47 -08:00
Timur Alperovich 98ea917477 GetBucketLocation should use path-style requests.
When making a GetBucketLocation request, Amazon may route the request
to the bucket region. When making it with v4 signer, the request may
fail because of the region mismatch. Concretely, a request to
test.s3.amazonaws.com may resolve to s3-us-west-2-w.amazonaws.com. The
request itself is prepared for the us-east-1 region (s3.amazonaws.com
endpoint), but then fails when the DNS resolution points to a
us-west-2 endpoint.

Bucket-in-path works around this for the GetBucketLocation requests.
That means that every GetBucketLocation request will be of the form:
https://s3.amazonaws.com/{bucket}?location. This ensures that jclouds
requests will not be subjected to Amazon's routing/DNS pointers.

Fixes: JCLOUDS-1213
2017-01-11 18:14:57 -08:00
mildis 61d0920f19 WindowsEveryonePrincipal : update regex for FR
"Everyone" in a french Windows is "Tout le monde".
Thus, the line.split(" ") returns only "Tout" and putBlob() throws an exception.
The modified regex search for the first two consecutive white char to return the name.
Windows's whoami returns a bunch of 0x20 between the name and the type.
2017-01-10 13:11:33 -06:00
Etienne CARRIERE ac2f746e64 JCLOUDS-1220: Possibility to change the name of Header of Identity v1 protocol 2017-01-05 15:55:19 +01:00
Andrea Turli c2cda931a5 update Docker README
improve description for osx
2017-01-05 09:20:36 +01:00
Zack Shoylev 6bc82ee262 Change BaseSwiftApiLiveTest to inherit from BaseBlobStoreIntegrationTest and use its methods 2016-12-20 14:10:22 -06:00
Zack Shoylev 301f6812af Updates the default template so live tests pass again. 2016-11-21 15:47:27 -06:00
Andrew Gaul fc14dce5ce Avoid creating temporary files via relative paths
This avoids creating junk in the source directory.
2016-11-15 20:28:31 -08:00
Zack Shoylev ee89c253b3 Missing cleanup was causing testReplaceManifest to fail 2016-11-15 11:21:25 -06:00
Ignasi Barrera f516fea2af Next development version 2.1.0-SNAPSHOT 2016-11-15 01:13:00 +01:00
Ignasi Barrera d03f061a68 Next development version 2.1.0-SNAPSHOT 2016-11-15 01:13:00 +01:00
Ignasi Barrera d290705419 Next development version 2.1.0-SNAPSHOT 2016-11-15 01:12:59 +01:00
Ignasi Barrera 66a82b4f60 Apache jclouds 2.0.0-rc3 release 2016-11-09 22:47:20 +01:00
Ignasi Barrera d6a16b8487 Apache jclouds 2.0.0-rc3 release 2016-11-09 22:43:56 +01:00
Ignasi Barrera 6a3a37f4e0 Apache jclouds 2.0.0-rc3 release 2016-11-09 22:30:45 +01:00
Andrea Turli 9b3e6d91bf Fix retryOnRenew classes
These classes should not close the release the payload as they are not
reading it.

- fix swift
- fix openstack-swift v1 and v2
- fix RetryOnRenewTest for v1 and v2
2016-10-28 17:29:05 +02:00
Zack Shoylev 78b3120e5e filesystem: Modify the backslash only on Windows 2016-10-26 19:53:55 -05:00
Erich Duda dff16876ee Change types of variables in docker ImageSummary
The int type is not sufficient for size and virtualSize
parameters, if the image is bigger than 2gb.

This fixes https://issues.apache.org/jira/browse/JCLOUDS-1190
2016-10-26 21:31:13 +02:00
Zack Shoylev fc91f64290 Ensure that jclouds consistently uses /, works with \\, and uses File.separator on the filesystem level. 2016-10-25 17:23:32 -05:00
Andrea Turli a1f6b7ebf2 update CloudstackTemplateOptions pattern 2016-10-25 15:49:46 +02:00
Josef Cacek c10540b04d [JCLOUDS-1162] DockerComputeServiceAdapter shouldn't remove container if it stops with non-zero exit code quickly after start. The commit also adds feature which prints container logs into JClouds Compute Log during Node destroy if the TRACE log level is enabled 2016-10-25 15:34:09 +02:00
Andrew Gaul 3c9f66b8e9 Avoid lower-case l literal suffix
Readers can confuse this with 1.  Found via error-prone.  Fixed via:

find -name \*.java | xargs sed -i 's/\( [0-9][0-9]*\)l/\1L/g'
find -name \*.java | xargs sed -i 's/\(([0-9][0-9]*\)l/\1L/g'
2016-10-23 14:24:35 -07:00
Andrew Gaul 8117574c83 Avoid C-style array declarations
Found via error-prone.
2016-10-23 13:51:59 -07:00
Andrew Gaul d1508b2d23 Make inner class static
Found via error-prone.
2016-10-23 06:39:46 -07:00
Andrew Gaul 2a04a52d56 Remove unused imports
Found via error-prone.
2016-10-23 06:39:46 -07:00
Ignasi Barrera 7e21cf0a4a Fixed infinite recursion in EC2 image live test 2016-10-20 12:21:51 +02:00
Shri Javadekar 7826d22d30 JCLOUDS-1161: Make AWSS3BlobRequestSignerV4 the default signer.
Added new live tests and fixed some unit tests accordingly.
2016-10-18 20:06:34 -07:00
Zack Shoylev 00f7ee0738 jclouds was not properly retrying on an expect: 100-continue PUT request that requires re-athentication because of java protocol code
JCLOUDS-1179 This fix also addresses the same problem with other providers

No currently working tests because of https://github.com/square/okhttp/issues/675

This is a common problem in other tools as well https://curl.haxx.se/mail/lib-2004-08/0002.html
2016-10-13 12:17:19 -05:00
Ignasi Barrera b06795ebe4 JCLOUDS-1182: Added Seoul and Mumbai regions to AWS 2016-10-13 18:14:20 +02:00
Ignasi Barrera cce4495872 Proper template config in the image extension live tests 2016-10-13 15:06:50 +02:00
Andrew Gaul 50e6d44433 JCLOUDS-945: fix local blobstore marker handling
Previously using prefix markers would not correctly find the next key.
Add test for this behavior.
2016-10-12 22:22:28 -07:00
Ignasi Barrera d98348d503 Better predicate signature for node operations 2016-09-23 10:22:23 +02:00
Arvind Nadendla 34b54ad163 Remove hardcoded limitation of content size in Openstack Nova CreateServerOptions API
Fix issue JCLOUDS-1175 by removing hardcoded content sizes in Openstack
Nova API
2016-09-23 10:20:01 +02:00
Andrew Gaul 76995a363d Do not fail AWS S3 signer tests with expiry
Regression from 41bdd9c5a9.
References #1008.
2016-09-17 12:13:28 -07:00
Zack Shoylev 4bbca9edf9 More fixes to parallel download resource cleanup 2016-09-14 14:36:44 -05:00
Zack Shoylev de68c2a1b0 Some handles were not properly closed 2016-09-07 16:28:00 -05:00
Andrew Gaul 7cde28a4d2 JCLOUDS-1074: Guava 20 compatibility
* dynamically call TypeToken.isSupertypeOf with Guava 19 and later and
  TypeToken.isAssignableFrom with Guava 18 and earlier
* consume or ignore values from methods with CheckReturnValue
* replace usage of removed Iterators.emptyIterator
2016-09-06 21:49:20 -07:00
Zack Shoylev 05c05e3de2 Allows users to download large files efficiently and directly to disk. 2016-09-02 14:05:26 -05:00
Francois Rigault dacd9825f0 Allow overriding getUserDefinedFileAttributeView
Today the filesystem blobstore provider use the extended attributes of the
filesystem to store metadata. This is not always desirable as some filesystems
(such as NFSv3) do not support extended metadata.  The current source code does
not allow to change this easily. getUserDefinedFileAttributeView could easily
be overriden for this purpose and allow extensions that customize where the
metadata should be stored.

See also JCLOUDS-658
2016-09-01 11:19:38 -05:00
Ignasi Barrera 0fd013da06 Fixed OAuth instructions for Azure 2016-09-01 09:56:16 +02:00
Daniel Haeser Rech 913fdeb168 JCLOUDS-1148: Fix token caches in OAuth flows 2016-09-01 09:55:23 +02:00
Ignasi Barrera 9662edf903 Update poms to include Docker and fixed groupId after promotion 2016-08-30 11:14:33 +02:00
Ignasi Barrera c493e2aa49 Promoted Docker from jclouds-labs 2016-08-30 11:14:20 +02:00
Josef Cacek a4e2110a63 JCLOUDS-1158 fix Docker find image Predicate for cases where repoTags field contains 'docker.io/' registry host prefix 2016-08-29 22:31:44 +02:00
Josef Cacek 71dd766227 Docker live test fix - SshToCustomPortLiveTest didn't work for remote docker endpoints due to wrong port number used 2016-08-29 22:27:32 +02:00
Josef Cacek c4605a32d4 [JCLOUDS-1157] fix networks in DockerComputeServiceAdapterLiveTest 2016-08-19 17:51:42 +02:00
Josef Cacek 58e11d42d1 Update Docker README.md 2016-08-19 17:49:25 +02:00
Iván Lomba 79e95c5a4c JCLOUDS-482: Add support for arbitrary CPU and RAM
This is a combination of 16 commits:

* First approach to ArbitraryCpuRamTemplateBuilderImpl
* Several fixes: refactoring some names, format, identation problems, some missing license headers and generateId method
* Refactored parse utility
* Added GoogleComputeEngineArbitraryCpuRamTemplateBuilderImpl to support GCE custom machine URI
* extracted hardware creation to automaticHardwareForCpuAndRam method
* Fixed ide automatic asterisk imports
* correcting WIP base case PR according to comments
* added machineTypeUriToHardware to set custom hardware in nodes
* fix checkstyle violations and other PR comments
* Set the providerId to custom machineType URI and fix adding node log
* Arbitrary hardware tests added to BaseTemplateBuilderLiveTest and GoogleComputeEngineTemplateBuilderLiveTest
* Added two more tests to BaseTemplateBuilderLiveTest
* Move repeated constants to TestUtils to reuse code
* Fix full path in the Hardware id and URI
* Add custom hardware tests to BaseComputeServiceLiveTest and GCEServiceLiveTest
* Change customHardware test to use buildTemplate and fix identation
2016-08-16 02:44:04 +02:00
Josef Cacek 9af449651d JCLOUDS-1153 Fix empty Docker repoTags field in ImageToImage function 2016-08-12 09:27:08 +02:00
Ignasi Barrera 449c6d8d84 Always take into account the configured template builder spec when building a template in live tests 2016-08-12 00:28:01 +02:00
Josef Cacek 0616b865ed [JCLOUDS-1147] upgrade snakeyaml version in byon and resolve removed deprecated Loader class 2016-08-02 15:28:41 +02:00
Andrew Donald Kennedy 6408c3a835 Add Network autovalue builder and tests 2016-08-01 11:42:31 +02:00
Andrew Donald Kennedy f2ce5679cc Connect container to list of networks in options 2016-08-01 11:42:26 +02:00
neonbunny e7bf9b1298 Ensure the canonicalRequest is logged properly
Resolves JCLOUDS-1143
2016-07-19 14:50:03 +01:00
Zack Shoylev 42079e1392 Parallel upload for BaseBlobStore 2016-07-15 04:15:17 -05:00
Andrea Turli a515ce2f22 Change line endings on files recursively 2016-07-14 00:02:13 +02:00
Andrew Gaul 4874a1eb18 JCLOUDS-1137: Handle TOCTOU during blobMetadata
A similar issue exists when getting a blob payload when a caller
simultaneously removes the blob.
2016-07-11 22:02:15 -07:00
Andrew Gaul 08e78c979e Add S3Proxy profile to skip failing tests 2016-07-11 16:42:09 -07:00
Timur Alperovich f61be9aa94 Add a system setting for Glacier's initial wait.
Glacier has an initial wait parameter, which is used to check on the
progress of a job. By default, it is set to 3 hours. This may work
well against existing AWS Glacier interface, however, against a
Glacier emulator the timeout should be overriden. The change allows
setting the timeout to an arbitrary shorter value with
-Dtest.glacier.initial-wait.
2016-07-11 11:23:08 +02:00
Timur Alperovich af7f4eeb51 Do not assume an endpoint in tests.
jclouds should not assume a particular endpoint when running glacier
tests, as it's possible that a glacier emulator is used (e.g.
glacier-proxy).
2016-07-11 11:23:05 +02:00
Zack Shoylev 6bff97b6d3 Changes the upload behavior to parallel, a TODO 2016-07-01 11:26:56 -05:00
Duncan Grant 7a979ba87b ImageId can be image name or id and will pull from hub 2016-06-28 09:47:43 +02:00
Andrew Gaul fff12293f9 JCLOUDS-1125: Add missing expected exception 2016-06-23 23:06:13 -07:00
Andrew Gaul 6df0472ab5 JCLOUDS-1125: local blobstore list MPUs 2016-06-23 21:45:04 -07:00
Arvind Nadendla 112c64e92a Make links nullable 2016-06-20 18:11:12 -05:00
Ignasi Barrera bde70457d0 JCLOUDS-1105: Do not create the default security group if custom groups are specified 2016-06-17 22:18:35 +02:00
urban 71b3a2b631 JCLOUDS-1129:Add more interfaces of the same network 2016-06-17 22:16:59 +02:00
Andrew Donald Kennedy 496fbae646 Fix incorrect key format check in Docker 2016-06-17 00:01:35 +02:00
Andrew Gaul 0a75446ae7 JCLOUDS-1028: Configure idempotent methods
Enable for Glacier.
2016-06-15 16:37:53 -07:00
Andrew Gaul 5fec2346a6 JCLOUDS-1028: Configure idempotent methods
Enable POST for Atmos, S3, and Swift.
2016-06-15 16:35:22 -07:00
Andrew Gaul ee5386258b JCLOUDS-1125: Glacier list multipart uploads
Not yet implemented.
2016-06-13 16:33:36 -07:00
Andrew Gaul 0bd2959410 JCLOUDS-1125: portable list multipart uploads
Only Azure, B2, and S3 support this operation.  Some MultipartUpload
fields become nullable.
2016-06-13 16:26:43 -07:00
Andrew Gaul 445664c9f1 JCLOUDS-1125: S3 list multipart uploads 2016-06-13 16:26:43 -07:00
Aled Sage 730a4cfe87 Docker.ContainerToNodeMetadata: remove needless mocking
Previously it mocked the container and the StateToStatus. It just
asserted that getter methods were called, rather than confirming the
resulting NodeMetadata had the right values.

By removing some of the mocks, it not only simplifies the code but
improves code-coverage. (e.g. if StateToStatus was being passed the
wrong value, the previous test would not have noticed).
2016-06-13 17:18:01 +02:00
Aled Sage fc88756d5a docker: get container’s IPs from all networks 2016-06-13 17:17:56 +02:00
Duncan Grant 91339b200e Node json should be optional
The Node element of the container json is optional
as well as nullable.
2016-06-13 12:56:05 +02:00
Zack Shoylev c1ce819f61 updateTemporaryUrlKey test 2016-06-07 17:37:39 -05:00
Duncan Grant ebc8f7568f Use container host ip not manager ip
When providing node ip in metadata jclouds was using the
ip of the docker manager which was fine for docker machine
but fails when using a swarm cluster.  This uses the ip returned
from docker.
2016-06-06 16:16:44 +02:00
Andrew Gaul a67ae3f27b Make inner classes static where possible
Found via error-prone.
2016-06-01 20:03:54 -07:00
Josef Cacek c0469cc9b7 [JCLOUDS-1118] fix configuration options location; mark some fields Nullable 2016-05-26 12:07:39 +02:00
Josef Cacek d4cf774389 [JCLOUDS-1117] fix for advanced Docker configuration - the Config.Builder should stay unchanged if used 2016-05-26 12:07:33 +02:00
Andrew Gaul 4ef28251c5 Add missing import
Regression from c5d845d368adf01909fd8ab5eeb824f7c44fb69e.
2016-05-23 20:26:01 -07:00
Andrew Donald Kennedy 61fb3ec145 Support supplying key and certificate to Docker as data 2016-05-23 17:52:24 +02:00
Jim Spring f46b38dd89 client credentials JWT support 2016-05-20 00:10:37 +02:00
Zack Shoylev c96cfb6176 Fix cleanup when the security group extension is not available. 2016-05-19 12:08:36 -05:00
Andrew Gaul 41bdd9c5a9 Skip unimplemented signed URL tests in S3 provider
jclouds aws-s3 provider supports these but s3 does not.  Also expect
failure in atmos and azureblob.  Follow on to
a4c40e15cd.
2016-05-12 12:12:52 -07:00
Andrew Gaul e423bc949e Avoid use of primitive wrapper constructors
Deprecated in Java 9.
2016-05-09 22:47:27 -07:00
Andrew Gaul 62410d31ae Handle missing containers in LocalBlobStore.list
Also fix FilesystemStorageStrategyImpl.getContainerMetadata to return
null on missing container as jdbc and transient already do.
2016-05-06 21:36:28 -07:00
Andrew Gaul a4c40e15cd Handle unimplemented signed URL tests in providers
This commit makes it evident in source code which providers do not
support this feature.
2016-05-06 10:31:37 -07:00
Andrew Gaul 0af3380a60 Address error-prone CheckReturnValue errors
Found via Guava 20.0.
2016-04-28 14:16:22 -07:00
Andrew Gaul da386ad0e2 Correct FindBugs warnings 2016-04-27 20:28:37 -07:00
Ignasi Barrera 34114dea00 JCLOUDS-1104: Extension namespaces are deprecated and can be null 2016-04-26 23:15:51 +02:00
Ignasi Barrera 04f1bb2b49 JCLOUDS-1104: Extension namespaces are deprecated and can be null 2016-04-26 22:38:45 +02:00
Andrew Donald Kennedy 42d0576aa9 Add openStdin option to Docker template options and set all port bindings explicitly 2016-04-26 10:33:17 +02:00
Andrew Gaul 6446627aad Correct equals with incompatible types
Found via error-prone.
2016-04-24 23:28:55 -07:00
Andrew Gaul 66cda12b50 Avoid eager evaluation of Preconditions messages
Found via error-prone.
2016-04-24 23:28:55 -07:00
Andrew Donald Kennedy 49f7bc37af Make DockerTemplateOptions values null safe 2016-04-19 17:01:18 +02:00
Andrew Gaul 3cf4e3d79c JCLOUDS-1101: Correct generics wildcard errors
Always incorrect but previously broken on Java 9.  Reference:

https://bugs.openjdk.java.net/browse/JDK-8075793
2016-04-18 09:49:58 -07:00
Ignasi Barrera 819141a608 Fixed OAuth authentication flow injections 2016-04-07 10:10:10 +02:00
Andrew Donald Kennedy 047595c7bb Added volumesFrom to Docker template options 2016-04-05 10:22:56 +02:00
Jim Spring 0420741690 Add support for Azure AD authentication using Service Principal and Password 2016-04-01 14:50:09 +02:00
Oleg 74a2a2683f JCLOUDS-1102: Fix. Rackspace returns a new structure for volume types. 2016-04-01 14:48:12 +02:00
Fernando Ribeiro 80beb9b397 JCLOUDS-1098: Fix OpenStack Marconi Unit Tests 2016-03-23 16:28:34 +01:00
Fernando Ribeiro 92e6319cd7 JCLOUDS-1095: Update OpenStack KeyStone Support for Zaqar 2016-03-23 10:29:07 +01:00
Zack Shoylev 760995a46e Use non-privileged mode for compute 2016-03-21 12:49:07 -05:00
Andrew Gaul 6e7801ad58 JCLOUDS-766: Fix issues with chunked upload signer
Previously we created a new builder instead of using the one the
method modified and did not preserve content encoding.  Addresses
AWSS3BlobIntegrationLiveTest.testPutInputStream test failures.
2016-03-20 16:06:51 -07:00
Andrew Gaul 8053abb530 JCLOUDS-766: Check metadata and headers for MD5
Addresses AWSS3ClientLiveTest.testMultipartSynchronously test
failures.
2016-03-18 22:56:54 -07:00
Andrew Gaul 35c5d7ffc0 JCLOUDS-766: Include Content-MD5 in v4 signature
Addresses AWSS3BlobIntegrationLiveTest.deleteMultipleObjects test
failures.
2016-03-18 16:58:03 -07:00
Andrew Gaul feae011c4d JCLOUDS-766: URL encode path for v4 signatures
Addresses AWSS3BlobIntegrationLiveTest.deleteObject test failures.
2016-03-18 15:05:05 -07:00
IgnacioCalvo 7d2f1e8b4d JCLOUDS-1080: Force JSON format for requests
Also make all fields optional.
2016-03-17 20:40:51 -07:00
Josef Cacek f3ee898c13 [JCLOUDS-1089] Add image history support to Docker ImageApi 2016-03-08 23:06:21 +01:00
Josef Cacek 831cdc67c3 [JCLOUDS-1002] provide access to Docker container Config object in the Node template options 2016-03-08 23:01:52 +01:00
Zack Shoylev f8ebbfdf8a Fixes an intermittently failing swift live test. 2016-03-08 12:38:13 -06:00
Andrew Gaul c18371a793 JCLOUDS-766: Add eu-central-1 to AWS S3 regions
Also clean up some missing and duplicated properties.
2016-03-07 21:21:08 -08:00
Andrew Gaul dd670f3f78 JCLOUDS-1064: More helpful error for null length 2016-03-06 19:08:26 -08:00
Zack Shoylev 243afed278 Add alias fallback info to keystone and nova extensions. 2016-02-24 11:22:10 -06:00
Josef Cacek 3f1fe271ed [JCLOUDS-1084] Docker live tests fixed and made more robust 2016-02-22 17:16:40 +01:00
Josef Cacek e0ab5d848a [JCLOUDS-1079] Make Floating IPs public in the NodeMetadata in openstack-nova provider 2016-02-19 11:59:52 +01:00
Zack Shoylev 8bbcfed312 Ensures the Swift container list call requests JSON as per spec 2016-02-18 18:48:55 -06:00
Zack Shoylev 738121d934 Renames extension resolver 2016-02-16 22:50:41 -06:00
Andrew Gaul e0a7ea7fdf Only quote ETag if it does not already have quotes 2016-02-16 16:29:54 -08:00
Andrew Gaul 7eb46cce36 JCLOUDS-651: Swift support for conditional copies 2016-02-16 16:29:54 -08:00