Commit Graph

342 Commits

Author SHA1 Message Date
ianmarshall f7db280099 Merge remote-tracking branch 'remotes/origin/master' into im_20200131_remove_circular_dependency_forcedid
# Conflicts:
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java
2020-02-18 12:13:00 -05:00
jamesagnew ddc7b601ee Version bump to 4.3.0-SNAPSHOT 2020-02-16 19:23:22 -05:00
jamesagnew 4fd80cfca7 Bump to 4.2.0 2020-02-16 19:00:15 -05:00
Ken Stevens da11d3d89e
remove distinct from search (#1712)
* it works now

* oops

* add successful migration message

* add successful migration message

* Add fix for #1300

* Remove FIXME

* Test fix

* Test fix

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2020-02-15 12:45:59 -05:00
Ken Stevens 5e8eec8bf6 it works now 2020-02-09 18:29:55 -05:00
James Agnew b2d2346228 Work on multitenancy 2020-02-07 15:18:06 -05:00
Ken Stevens 5f30bdb27b not sure this is going to work yet 2020-02-04 17:00:56 -05:00
Ken Stevens 318b306020
fix migration (#1699) 2020-02-03 17:13:35 -05:00
Ken Stevens 3c46595e81 Merge remote-tracking branch 'remotes/origin/master' into ks-20200203-remove-search-lastused 2020-02-03 11:33:15 -05:00
Ken Stevens b4325199cf review feedback 2020-02-03 11:20:18 -05:00
James Agnew 709aeb4ae8
Drop unnecessary index (#1696) 2020-02-03 10:29:39 -05:00
Ken Stevens cda906d7b8 Column removed. Time to run tests. 2020-02-03 09:44:10 -05:00
ianmarshall 5d5b3d7639 Changes to schema to loosen dependencies between Forced ID table and the Resource and Resource History tables. 2020-01-31 11:16:01 -05:00
James Agnew 3edeb03edc
Rework hashcode generation (#1684)
* Rework hashcode generation

* Compile fix

* Hashcoode refactoring

* FIx compile errors
2020-01-26 22:23:06 -05:00
Jafer Khan 4a4fb0fd9d Remove transient fields from migration task 2020-01-24 08:28:01 -05:00
James Agnew 0718f6b52a
Allow schemalocation in XML validation (#1677)
* Allow schemaLocation declaration in XML when validating

* Add fix for #1676
2020-01-21 07:14:16 +09:00
James Agnew a19d246432 License header version bump, and add the ability for "add column"
migration tasks to fail gracefully
2020-01-12 11:01:21 +08:00
James Agnew a61527d4cb Allow multiple PK column in addTable migrator 2019-12-21 14:05:35 -05:00
James Agnew a94b684be5 Addres TODO 2019-12-18 14:43:57 -05:00
James Agnew 0a21ab6941 Use datasource for migrator 2019-12-18 14:23:46 -05:00
Ken Stevens 68a7613f9e Merge remote-tracking branch 'remotes/origin/master' into ks-20191119-scheduler
# Conflicts:
#	hapi-fhir-cli/hapi-fhir-cli-api/src/main/java/ca/uhn/fhir/cli/BaseFlywayMigrateDatabaseCommand.java
2019-12-16 08:58:26 -05:00
jamesagnew 4592decbfa Add license headers 2019-12-15 19:51:49 -05:00
Ken Stevens 9f422efd7a handle case where version list is quoted 2019-12-14 21:02:31 -05:00
Ken Stevens 7c3514fecc Skip versions tested 2019-12-14 19:00:58 -05:00
Ken Stevens 4946fffac2 pass test 2019-12-14 17:27:17 -05:00
Ken Stevens 11e55cebf5 begin with failing test 2019-12-14 17:26:56 -05:00
Ken Stevens 0dde239696 migration task don't compare length if no length is specified in the task 2019-12-13 15:12:59 -05:00
Ken Stevens 931a41938b added dry-run output to both Flyway and TaskOnly migrator
dry-run only calls TaskOnly migrator now
2019-12-12 14:42:43 -05:00
Ken Stevens 83106c92c1 move quartz stuff out 2019-12-11 17:00:31 -05:00
Ken Stevens 91f4e62148 Oracle works yay! 2019-12-11 15:15:47 -05:00
Ken Stevens 7cc690ffd3 Merge remote-tracking branch 'remotes/origin/master' into ks-20191119-scheduler 2019-12-11 14:17:54 -05:00
Ken Stevens 7c1544d8f3 strip comments from sql before splitting ; (to avoid ; in comments) 2019-12-11 14:02:24 -05:00
Ken Stevens da5034959b start working on quartz fix 2019-12-11 13:41:27 -05:00
Ken Stevens ae5f00accc add support to stub out renameindex 2019-12-11 12:07:43 -05:00
Ken Stevens 176bf657a7 Merge remote-tracking branch 'remotes/origin/master' into ks-20191119-scheduler
# Conflicts:
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java
2019-12-10 10:10:41 -05:00
Ken Stevens 198779a25f fix MSSQL index rename 2019-12-09 18:36:41 -05:00
Ken Stevens 3657a3ad5a pre-review cleanup 2019-12-09 16:51:42 -05:00
Ken Stevens bd1b41ff82 change out-of-order-permitted from environment variablet to flag to work like dont-use-flyway 2019-12-09 16:44:24 -05:00
Ken Stevens 77373605bd add 420 migration 2019-12-09 15:58:08 -05:00
Ken Stevens b0b410384f add renameIndex task 2019-12-09 15:42:40 -05:00
Ken Stevens deb5fd40a7 Added stubbed tasks for add and drop index 2019-12-09 14:42:37 -05:00
Ken Stevens ec37138138 add environment variable to allow flyway out of order migration 2019-12-09 11:32:25 -05:00
Ken Stevens 0b30033b0a upgrade flyway to latest version
automatically repair failed migrations
2019-12-09 11:19:18 -05:00
Ken Stevens 45bab9879f migration failed when no-column-shrink was enabled. 2019-12-06 19:21:30 -05:00
Ken Stevens bfa89483e7 Merge remote-tracking branch 'origin/master' into ks-20191119-scheduler 2019-12-06 18:13:38 -05:00
James Agnew 438ef70fdb Reuse connections in brute force migrator 2019-12-06 16:52:26 -05:00
Ken Stevens 6c432733d5 Merge remote-tracking branch 'remotes/origin/master' into ks-20191119-scheduler 2019-12-05 18:03:02 -05:00
James Agnew d91617e923
Account for review changes (#1623) 2019-12-05 18:01:45 -05:00
Ken Stevens 66cc83464f Merge remote-tracking branch 'remotes/origin/master' into ks-20191119-scheduler 2019-12-05 13:06:44 -05:00
James Agnew 69a5266127 Allow non-flyway migration 2019-12-05 10:46:37 -05:00
Ken Stevens e0a2b9a179 support schema initialize naming in a backwards compatible way 2019-12-02 19:02:04 -05:00
Ken Stevens c27dfdff51 back out schema initialize naming. there may be migrated schemas in the wild this would interfere with. 2019-12-02 18:48:40 -05:00
Ken Stevens f29118fa9c move clustered quartz from cm db to pers db 2019-12-02 14:32:34 -05:00
Ken Stevens 63218e560d Merge remote-tracking branch 'remotes/origin/master' into ks-20191119-scheduler 2019-12-02 08:59:17 -05:00
James Agnew 86bd9a28fe
Fix to valueset expansion logic (#1613)
* Fox to valueset expansion logic

* Work on docs

* Docs tweaks

* Docs tweaks

* Fix test

* Remove unused class

* Add changelog
2019-12-01 21:21:19 -05:00
Ken Stevens a58da3b535 name schema initializer to distinguish between different schemas 2019-11-28 12:10:34 -05:00
Ken Stevens bf003f0d16 cleaned up cleanup 2019-11-27 21:54:13 -05:00
Ken Stevens 9693b1736e fixed quartz sql with a hack. need a better fix. 2019-11-27 17:10:27 -05:00
jamesagnew df57d24333 Add license headers 2019-11-27 04:49:45 -05:00
Ken Stevens 7593b7f8f7 fix test 2019-11-26 16:57:59 -05:00
Ken Stevens 82990f3e97 fix test 2019-11-26 10:39:21 -05:00
Ken Stevens ef77490c51 fix test 2019-11-26 09:50:40 -05:00
Ken Stevens 3d00634b16 make migration table name configurable 2019-11-26 09:07:53 -05:00
Ken Stevens 98d21c4ee3 review feedback 2019-11-23 07:34:49 -05:00
Ken Stevens 6543da1470 Merge remote-tracking branch 'remotes/origin/master' into ks-flyway 2019-11-15 09:28:30 -05:00
James Agnew 497a96ae68 Version bump for next round of development 2019-11-12 22:09:21 -05:00
jamesagnew f99ad981b5 Version bump to 4.1.0 2019-11-12 16:08:20 -05:00
Ken Stevens 509799095c don't hash on init script length since it will change. Hash based on class name. 2019-11-04 10:45:02 -05:00
Ken Stevens 53cedddcab rename BaseTask constructor parameters 2019-11-04 10:27:34 -05:00
Ken Stevens 4e2806bea5 add task descriptions 2019-11-04 10:20:54 -05:00
Ken Stevens 30865588e3 pre-review cleanup 2019-11-04 08:36:24 -05:00
Ken Stevens 947ac6f096 fix compile error from merge 2019-11-03 21:11:36 -05:00
Ken Stevens 4c732ce977 Merge remote-tracking branch 'remotes/origin/master' into ks-flyway 2019-11-03 21:07:59 -05:00
James Agnew f481e11ad5 Update migration tasks 2019-11-03 18:49:47 -05:00
Ken Stevens 779366ccad final hashcode equals fix 2019-11-03 15:52:32 -05:00
Ken Stevens ea21b59fde generalize initialize schema interface 2019-11-03 14:02:28 -05:00
Ken Stevens 5890059362 moar integration tests 2019-11-03 13:46:09 -05:00
Ken Stevens de4f909eda optimize imports 2019-11-02 21:52:35 -04:00
Ken Stevens becd6790e8 fix test 2019-11-02 21:51:17 -04:00
Ken Stevens 8ee06855d7 FIXME cleanup 2019-11-02 21:36:03 -04:00
Ken Stevens 4cdde60917 FIXME cleanup 2019-11-02 21:32:42 -04:00
Ken Stevens 8887d41283 added proxy indicator for hash indexes migration having been run (used TRM_CONCEPT_DESIG table existence) 2019-11-02 21:30:40 -04:00
Ken Stevens 6356205768 fix final task hashcode. Multiple runs now work. 2019-11-02 17:53:21 -04:00
Ken Stevens 3bfcb42012 fix enum hashcode 2019-11-02 17:48:30 -04:00
Ken Stevens 0404454167 abort schema initialization if HFJ_RESOURCE table already exists 2019-11-02 17:03:45 -04:00
Ken Stevens 5577680a85 test hashcodes are consistent (fixed 3 that weren't) 2019-11-02 16:56:39 -04:00
Ken Stevens f20fee297a add hashcode to initializationProvider 2019-11-01 22:02:07 -04:00
Ken Stevens 071b14cbcc add hashcode to initializationProvider 2019-11-01 22:00:59 -04:00
Ken Stevens 7855ce070c typo 2019-11-01 21:46:15 -04:00
Ken Stevens 6049ee6f1c init schema test 2019-11-01 21:40:24 -04:00
Ken Stevens 3dc2770962 consolidate providers 2019-11-01 21:35:11 -04:00
Ken Stevens 7066f35e49 persistence migrate from zero works. 2019-11-01 20:39:11 -04:00
Ken Stevens 49265dea32 added schema initialization but not used yet 2019-11-01 17:18:27 -04:00
Ken Stevens 5f294f335d merge 2019-11-01 13:38:18 -04:00
Ken Stevens bee993a0ec Merge remote-tracking branch 'remotes/origin/master' into ks-flyway
# Conflicts:
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/taskdef/DropIndexTask.java
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/taskdef/DropTableTask.java
2019-11-01 13:37:00 -04:00
James Agnew a5dceca075 Work on migrator task support for dropping tables in postgres 2019-11-01 11:04:20 -04:00
Ken Stevens d1814947b3 include version number in log message 2019-10-31 17:12:02 -04:00
Ken Stevens 0cfea8a871 changed to use hibernate auto ddl flag as a check 2019-10-31 15:47:52 -04:00
Ken Stevens f95e4d387d added schema migration validation 2019-10-31 14:56:55 -04:00
Ken Stevens 29752465c8 add boolean method FliwayMigrator.migrationRequired() so we can test if we have the current version 2019-10-30 20:27:25 -04:00
Ken Stevens 7bd29d69fd all done but removing the old migrator 2019-10-29 21:23:55 -04:00
Ken Stevens 6cd14122cd moved version number up to root component 2019-10-29 21:21:54 -04:00
Ken Stevens b420578cc5 nudged dates within release date ranges 2019-10-29 20:43:56 -04:00
Ken Stevens e8673ecb6f finished reindexing 2019-10-29 20:40:40 -04:00
Ken Stevens 28e628ec0e all done but reversioning 2019-10-29 18:03:13 -04:00
Ken Stevens 6ba3afb489 All tests pass except noShrink case 2019-10-29 15:30:25 -04:00
Ken Stevens c7bd11ed0d migration version validation tested 2019-10-29 14:54:35 -04:00
Ken Stevens 788f386ce4 added version order validation 2019-10-29 14:30:17 -04:00
Ken Stevens c8c89d976f Flyway integration test passes 2019-10-29 14:18:41 -04:00
Ken Stevens 94c8f96856 Add equals and hashcode to tasks for flyway 2019-10-29 13:34:11 -04:00
Ken Stevens dd2cb39fd1 tests pass but now need to renumber 2019-10-28 17:45:36 -04:00
Ken Stevens a123bfe564 All tests pass except createTableByColumns 2019-10-28 17:28:26 -04:00
Ken Stevens f8e15ea69e added final String release field to migration BaseTask 2019-10-28 16:09:18 -04:00
Ken Stevens 1aabe9bf40 added final String version field to migration BaseTask 2019-10-28 15:34:21 -04:00
Ken Stevens 18e8604494 Merge remote-tracking branch 'remotes/origin/master' into ks-flyway
# Conflicts:
#	hapi-fhir-cli/hapi-fhir-cli-api/pom.xml
2019-10-27 08:29:52 -04:00
James Agnew 464c6c5b45
Upgrade dependencies and addess Semmle issues (#1554)
* Version bumps

* FIx build issues

* Address a few Semmle issues

* Work on tests

* Some test fixes

* Test fix

* Fix deletions

* Test fix

* Fix intermittent test failure

* One more dependency bump

* Add some test logging

* Remove bad import

* Add some tests

* Test updates
2019-10-22 17:10:58 -04:00
James Agnew 13b80a294a Test fix 2019-10-18 07:13:11 -04:00
James Agnew 5dcbed0438 Fix foreign key handling when dropping tables 2019-10-17 15:52:35 -04:00
James Agnew 839f935401 remove unnecessary index 2019-10-15 19:07:37 -04:00
James Agnew e727769988 FIx a migration task 2019-10-15 18:16:22 -04:00
James Agnew 2725797610
Implement Terminology Delta Operations (#1536)
* Start refactoring terminology delta operations

* Work on delta operations

* Work on concept saving

* Split term services into smaller services

* Work on term delta operations

* Work on term svcs

* Work on term operations

* More work on delta uploader

* Add a test

* Wrk on term service

* Fix compile error

* Some refactoring

* Test fix

* Test fix

* Test fixes

* Test fix

* Test fixes

* Test fixes

* Work on delta

* Work on tests#

* Test fixes

* Improve resequencing logic

* Build test

* More testing

* More build testing

* More work on tests

* CHange test logging

* Fix term service PID issue

* Update src/changes/changes.xml

Co-Authored-By: Diederik Muylwyk <diederik.muylwyk@gmail.com>

* Address review comment

* Some cleanup

* Test fix

* Fix some tests

* Test fixes
2019-10-14 19:14:41 -04:00
Diederik Muylwyk 3c24e14a71 Fixed test based on wrong assumption. 2019-10-08 13:39:13 -04:00
Diederik Muylwyk 23264f497f Added test. 2019-10-08 13:19:35 -04:00
Diederik Muylwyk 175c0781eb Add case for handling JDBC type CLOB in JdbcUtils. 2019-10-08 11:55:03 -04:00
Ken Stevens 790f655a95 flyway initial implementation (with FIXMEs) 2019-10-06 17:55:10 -04:00
James Agnew 2193a30b45
CodeSystem Delta Uploader (#1520)
* Terminology Delta Uploading

* Add delta operations

* Fix up tests

* Improve test coverage a bit

* Improve test coverage reporting

* fix yaml

* A couple of test enhancements

* FIx intermittent test failure
2019-10-03 07:19:00 -04:00
James Agnew 260cee023a Account for longer resource type names 2019-10-02 10:00:45 -04:00
Diederik Muylwyk 826cb0ff10 Fix migration task for TermConceptProperty; should be BLOB, not CLOB. Good job! 2019-09-30 20:18:20 -04:00
Diederik Muylwyk 396f358f2d Support for long concept property values has been added. 2019-09-30 14:40:31 -04:00
James Agnew 8657afb01e
Fix intermittent test failures regarding search (#1509)
* Add some logging

* Add more test logging

* Work on some test logging

* Fix compile error

* Work on codecov

* Work on codecov

* Remove test debug messages
2019-09-25 20:15:21 -04:00
Ken Stevens 945cc6419f fixed npe 2019-09-24 18:13:53 -04:00
James Agnew f5788341f2
Add no-shrink-columns option to migrator (#1500)
* Add no-shrink-columns option to migrator

* Address review comments
2019-09-23 18:44:51 -04:00
James Agnew f4252cdeaa Merge branch 'master' of github.com:jamesagnew/hapi-fhir 2019-09-21 08:54:59 -04:00
James Agnew 0cbb33eded Fix up migratiom tasks for 4.1.0 2019-09-21 08:54:33 -04:00
Diederik Muylwyk e1e8ec981b Replaced migration task column length constants with literals because they are not necessarily constant across schema versions. 2019-09-20 12:31:54 -04:00
James Agnew 6b21e8cd90 Add some testing to the bulk export service 2019-09-18 14:43:00 -04:00
James Agnew 4a751cbfc5
Integrate Bulk Export (#1487)
* Start working on subscription processor

* Work on new scheduler

* Test fixes

* Scheduler refactoring

* Fix test failure

* One more test fix

* Updates to scheduler

* More scheduler work

* Tests now all passing

* Ongoing work on export

* Ongoing scheduler work

* Ongoing testing

* Work on export task

* Sync master

* Ongoing work

* Bump xml patch version

* Work on provider

* Work on bulk

* Work on export scheduler

* More test fies

* More test fixes

* Compile fix

* Reduce logging

* Improve logging

* Reuse bulk export jobs

* Export provider

* Improve logging in bulk export

* Work on bulk export service

* One more bugfix

* Ongoing work on Bulk Data

* Add changelog
2019-09-17 16:01:35 -04:00
Diederik Muylwyk 67d24eedff Merging master into working branch.
# Conflicts:
#	hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/TermValueSetConceptDesignation.java
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java
2019-09-09 19:01:29 -04:00
Ken Stevens dded46d220 fixed bug in drop fk migration task 2019-09-06 10:27:20 -04:00
Ken Stevens c88e6a6fa7
Merge pull request #1442 from jamesagnew/ja_20190822_1440_infinispan_query_cache
Externalize query cache into its own service
2019-09-06 08:28:12 -04:00
James Agnew 785d40a777 Accuont for a few schema changes 2019-09-05 21:29:47 -04:00
Ken Stevens b1421b1d60 @NotNull (hope I picked the right one!) 2019-09-05 16:20:24 -04:00
Ken Stevens 2a08808935 final PR cleanup 2019-09-05 16:11:58 -04:00
Ken Stevens 14fadd3004 fix drop foreign key task 2019-09-05 15:51:38 -04:00
Ken Stevens fa62ab9c9e Merge remote-tracking branch 'remotes/origin/master' into ja_20190822_1440_infinispan_query_cache
# Conflicts:
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java
#	src/changes/changes.xml
2019-09-04 13:31:00 -04:00
Diederik Muylwyk ea19b46f60 Added an indexed order column to TermValueSetConcept for improved fetching. 2019-09-03 20:36:16 -04:00
Diederik Muylwyk b42b598d01 Added pre-fetch for concept designations, improved paging for expanded ValueSets, and columns on TermValueSet for number of concepts and designations. 2019-09-03 16:02:28 -04:00
Ken Stevens 259426b0dd Merge remote-tracking branch 'remotes/origin/master' into ja_20190822_1440_infinispan_query_cache
# Conflicts:
#	hapi-fhir-jpaserver-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java
#	src/changes/changes.xml
2019-08-28 10:37:17 -04:00
Diederik Muylwyk 890555a77d Incremental work on large ValueSet expansion support; transaction boundaries need to be broken up. 2019-08-23 11:46:32 -04:00
James Agnew 6fa27934a8 Tests all seem to be working 2019-08-23 10:14:13 -04:00