diff --git a/its/ITProblems.txt b/its/ITProblems.txt deleted file mode 100644 index e9ecea828b..0000000000 --- a/its/ITProblems.txt +++ /dev/null @@ -1,149 +0,0 @@ -* Instructions - -Running the following from the top-level directory builds a distributable: - ----- -mvn install ----- - -If you wish to also run the integration tests themselves: - ----- -mvn -Prun-its install ----- - -This command can also be run from the core-integration-tests directory if the prerequisite artifacts have already been -installed or deployed. - -* Reusing the suite - -The suite is bundled up as a JAR containing all the tests and resources. All artifacts are now versioned according to the -version of Maven they are testing (currently, 2.1-SNAPSHOT), though it can be run against older versions as the verifier -will detect and omit tests that require the newer version of Maven to pass. - -* Outstanding Issues - -The top things that could be done: - -1) The issues that would be most helpful that could be tackled on a piecewise basis by many would be to take plugin - specific ITs out of the ITs. There are many in there for the surefire plugin so while you're doing that you can look - at it. Piecewise but probably totally simple because you have to replace it with an IT that actually tests what it - was testing. A lot of time I have had to make a new IT plugin flavour. - -2) The next issue of importance would be to collect all the in IT plugin plugins, invokers and verifiers and align all - these. - -3) Once 2) is done then we wire the embedder option into the resulting invoker. - -4) Proper isolation. The tests currently pick up your Maven settings file and current local repository. A clean local - repo can be used but it would result in a lot of unnecessary artifact retrieval. This fails in a locked down - environment (as additional repositories declared in IT 92, 94 and 120 are ignored). It also causes other risks. - A possible solution is to construct a repository artifact (using a POM with the dependency plugin to create an - assembly) that represents everything used by the ITs, then to have the setUp lay that out as a remote repo and use a - controlled settings.xml that uses a clean local repository, and mirrors central to that remote repository. - -5) The support artifacts are currently deployed to the central repository, meaning they can get out of sync with those - in SVN (unless they are assumed to have been 'released'). After solving (4), these could be injected into the - constructed repository instead. - -6) It's not obvious how to run them from and IDE and that's where I've found it to be most convenient to run them. - In particular, the resources directory is treated as test sources, which is not the case as each are projects - -7) it0119 places artifacts in the o.a.m.plugins and o.c.mojo group IDs to test plugin prefix order instead of using a - replacement settings file and subgroups of o.a.m.its as it should - -Other issues: - - - [ ] An IT should be completely self-contained so that the problem - can be understood by looking in one place, in one Maven - project. - - [ ] We should be able to create an Archetype so that users can - easily create ITs for us. The is not completed but we do have the - core-integration-test-sample directory with has a sample I - offered to users. - - [ ] We should be able to easily integrate the IT into a larger - run where we can use forked or embedded execution. - - [ ] We should create Archetypes for all categories of problems so - that anyone can generate tests cases for us. Then there is so - much that we can do in terms of automating this process of - checking tests for quality along with the patches. - - [ ] automate the testing of ITs submitted by users - - [ ] Each IT should have its own repository if it needs resources - from repository. We can't mess with a users repository when - testing. - - [ ] We need to have a file system based remote repository for - testing - - [ ] We need a primary run that can done entirely offline to simply test - the guts of Maven from a baseline, then a secondary run possibly using - the exact same repository except served via different means like http, - ftp, scp which would allow us to find all the holes in the transport - mechanisms. - - [ ] We need to standardize on integration testing in general. We - have people going all over the place and it's a disaster. - - [ ] We have too many IT plugins (3) - - [ ] We have too many invokers (5) - - [ ] We have too many verifiers (3) - - [ ] The ITs should run nicely from an IDE. Solution: this does - work but requires that you run mvn clean - resources:testResources first as the IDE doesn't know how to - set that up. Needs to be fully fixed. But it is much nicer - running this stuff in your IDE. -- [-] Problems with ITs - - [ ] it0006 is an integration test for the verifier plugin. - - [ ] it0014 uses the compiler plugin to test plugin configuration. - - [ ] it0016 is an integration test for the WAR plugin. - - [ ] it0017 is an integration test for the EJB plugin. - - [ ] it0018 uses real dependencies to test artifact resolution. - - [ ] it0020 tests beanshell mojo support. - - [ ] it0024 uses the compiler plugin to test mojo configuration. - - [ ] it0028 uses the compiler plugin to test mojo configuration. - - [ ] it0029 uses the compiler plugin to test for pluginManagement - injection of plugin configuration. - - [ ] it0033 is an integration test for the EAR plugin. - - [ ] it0043 uses the help plugin to look at the state of the final - POM, we should have this built into Maven for testing. - - [ ] it0047 uses plexus-container-default for testing. - - [ ] it0048 uses the surefire plugin to test default value - population for mojo parameters. - - [ ] it0050 is an integration test for the surefire plugin. - - [ ] it0051 is an integration test for the release plugin. - - [ ] it0052 is an integration test for the release plugin. - - [ ] it0054 is an integration test for the resources plugin. - - [ ] it0055 is an integration test for the compiler plugin. - - [ ] it0056 uses the compiler plugin to test multiple executions. - - [ ] it0060 uses the compiler plugin to test aggregation of list - configuration items when using 'combine.children=append' - attribute. - - [ ] it0063 tests JDK 1.4.2 specifics. - - [ ] it0068 uses the modello plugin to test repository - accumulation (i'm not sure what that means, John?) - - [ ] it0069 uses classworlds dependency to test offline mode. - - [ ] it0070 is an integration test for the RAR plugin. - - [ ] it0074 uses the compiler plugin to test mojo configuration. - - [ ] it0076 uses the WAR plugin to test pluginManagement. - - [ ] it0078 uses the compiler plugin to test pluginManagement. - - [ ] it0079 uses the source plugin to test that attached artifacts - have the same build number as the primary artifact. - - [ ] it0080 uses the WAR plugin to test an artifact handler that - specifies no contribution to transitive dependencies. if the - behavior of this handler changed the test would break. - - [ ] it0081 uses production dependency of wagon as a dependency - - [ ] it0083 uses the WAR plugin to test an artifact handler that - the WAR artifact handler currently behaves like - - [ ] it0086 uses production dependency of bsh to test classloading - - [ ] it0087 uses prodcution dependency of commons-logging to test - classloading - - [ ] it0089 is an integration test for the checkstyle plugin, - and places an artifact in the wrong group ID - - [x] it0094 has a test that attempts to access a protected field - in a ClassRealm - - [ ] it0095 uses the help and verifier plugins to test URL - calculation - - [ ] it0100 uses the antrun plugin to make sure - ${parent.artifactId} resolves correctly - - [ ] it0102 uses the help and antrun plugins to test profile - activation - - [ ] it0104 uses the surefire plugin to test interpolation - - [ ] it0105 is an integration test for the resources plugin - - [ ] artifactIds should be aligned with directories -