Commit Graph

313 Commits

Author SHA1 Message Date
Ken Stevens 346e29920a
Issue 2927 convert mdm clear to spring batch (#2929)
* initial roughout of mdm clear batch job

* still roughing out classes

* finished first draft of reader

* most tests passing now

* all tests pass.  now FIXMEs

* FIXMEs done.  Time for regression.

* fix test

* changelog and docs

* fix test

* pre-review cleanup

* version bump

* move spring autowire deps for cdr

* move spring autowire deps for cdr

* finally got beans working phew

* rearrange method calls so persistence module can perform clear operation on its own

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* review feedback

* review feedback

* Remove inheritance

* fix beans

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2021-09-03 14:58:19 -04:00
Tadgh 37ad73a422 Bump PRE version so we don't conflict downstream 2021-08-23 15:42:18 -04:00
Tadgh 4fadc62066 Bump Hapi Version 2021-08-06 13:44:04 -04:00
Ken Stevens f5dfc3a5ed
New reindex spring batch operation (#2845)
* first sweep to add new $reindex job

* first cut done.  next up integration test.

* start roughing out reindex everything

* Failing test

* test passes

* add BundleUtil helper method

* rough in processEverything

* rough in processEverything

* fix mock test

* merge master

* Make BaseJpaTest.mySrd lenient.

* fix test

* reindex everything works with FIXMEs

* reindex everything more tests

* moar tests

* moar tests

* fix test

* fix test

* fix regression and fix test

* fix test cycle

* fixme

* fix test

* test provider layer

* reindex everything partition aware

* yay last fixme!

* run reindex in partitioned threads

* add fixmes for areas that require extra testing

* added transaction and log message

* manual integration testing

* changelog

* reindex all partitions

* bump hapi version

* fix test

* moar test

* pre-review cleanup

* fix javadoc

* review feedback

* review feedback

* review feedback

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_5_0/2845-reindex-operation.yaml

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* review feedback

* review feedback

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/delete/job/ReindexJobTest.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/delete/job/ReindexJobTest.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantBatchOperationR4Test.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantBatchOperationR4Test.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/ResourceSearch.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/api/server/storage/IDeleteExpungeJobSubmitter.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2021-07-29 19:18:49 -04:00
James Agnew f957661dda
Patient ID Partition Mode should support conditional create (#2828)
* Fixes

* Test fix

* Add changelog

* Test fixes

* Test fixes

* Docs tweak

* Partitioning fixes

* Import fix

* Fix test data
2021-07-26 14:18:58 -04:00
jamesagnew a5e6674e84 Version bump to 5.5.0-PRE7-SNAPSHOT 2021-07-22 08:41:16 -04:00
James Agnew 6d37749be8
Add non unique combo search params (#2809)
* Start work on nonunique combo search params

* Work on SQL

* Version bump

* A fix

* Test fixes

* Fixes

* Undo version bump

* Test fixes

* Resolve fixme
2021-07-21 19:14:14 -04:00
Michael Buckley cbb9a4355a Add details when we fail to load an element of a package definition. 2021-07-19 17:59:54 -04:00
Tadgh a7e6813510 Update string length. Add migration 2021-07-13 15:46:47 -04:00
James Agnew d762a07817
Partition patient interceptor (#2766)
* Start work on interceptor

* Implemented

* Add interceptor and docs

* Add docs

* Compile fix

* Test fixes

* Test cleanup

* Test fixes

* Test fixes

* Improve error message

* Add broken tests

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_5_0/2766-add-patient-id-compartment-interceptor.yaml

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/interceptor/PatientIdPartitionInterceptor.java

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Address review comments

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
2021-07-05 09:15:20 -04:00
James Agnew 4bb495e9d7
Fix NPE (#2774) 2021-07-02 05:29:49 -04:00
Tadgh f66f9b992e Bump pre-version as this will break downstream implementations 2021-06-30 15:37:19 -04:00
jamesagnew 9bc4859761 Add test 2021-06-23 20:55:22 -04:00
James Agnew 8379392e16
Add inline tag mode (#2748)
* Start work on tag mode 3

* Add changelog

* Test fix
2021-06-22 23:08:09 -04:00
James Agnew 8b205b23d0
Fix history paging (#2741)
* Fix history paging

* Add changelog

* Add license header
2021-06-18 17:47:51 -04:00
Ken Stevens 134631fdee
Convert delete expunge to use Spring Batch (#2697)
* prepare to add $delete-expunge operation that will create a spring batch job

* Add operation

* Wire up jpa provider.  Begin with failing test.

* Copy/paste bulk import job as a starting point.
FIXME with proposed design

* delete expunge job parameter validation with test

* implemented reader
stubbed processor, writer

* wip for master merge

* started implementing reader

* started implementing reader

* working with stubs

* happy path batch delete expunge is done

* Provider done but test not passing.  Guessing batch infrastructure not running in that test.

* IT test works now

* add reader test

* Converted delete _expunge=true to use new batch job

* DeleteExpungeDaoTest passes

* Fix test

* Change batch size to integer

* rename search count to batch size

* Make delete expunge partition aware

* updated docs

* pre-review cleanup

* change log

* add partition id to SystemRequestDetails

* Make RequestPartitionId serializable

* Change delete expunge provider to use partition id instead of tenant name

* fix tests

* test pointcut gets called

* assert on pointcut calls

* Add resource type to STORAGE_PARTITION_SELECTED pointcut

* bump hapi-fhir version
move expunge provider parameters from JpaConstants to ProviderConstants

* bump hapi-fhir version

* copyrights

* restore deleteexpungeservice for mdm

* restore deleteexpungeservice for mdm

* fix test

* public constants

* convert instant to date

* Moved expunge constants to ProviderConstants

* final review

* disabling InMemoryResourceMatcherR5Test.testNowNextMinute() to see if I can get a clean test run

* fix tests

* fix tests

* fix tests

* fix tests

* review feedback

* review feedback

* review feedback

* review feedback

* review feedback

* review feedback

* improve logging

* bump version

* version bump

* recovering from failed merge

* unzip RequestListJson per Gary's suggestion.  I didn't want to do it at first, but as usual Gary was right.

* fix serialization
2021-06-15 10:36:05 -04:00
jamesagnew 376a84d213 Version bump 2021-06-14 13:12:34 -04:00
James Agnew b934abb297
Impropve transaction Performance (#2717)
* Work on changes

* Work on perf

* Work on testing

* Work on perf

* Work on perf

* Work on fix

* Work on perf

* Ongoing work

* Add changelog

* Additional docs

* Test fixes

* Address review comments

* Test fix
2021-06-14 13:08:19 -04:00
James Agnew dc627dc019
Add bulk export authorization layer (#2712)
* Add bulk export authorization layer

* Add docs

* Version bump

* Authorize any

* Add bulk export - all

* Address lgtm issue
2021-06-06 20:47:43 -04:00
James Agnew 24b3f0f30d
Enable mass ingestion mode (#2681)
* Work on fixes

* Work on counts

* Enable mass ingestion mode

* Add changelog

* Test fix

* Test fix

* Test fixes

* Fixes

* Test fix

* Test fix
2021-05-27 18:43:51 -04:00
James Agnew 7a83c76e65
Add validation suppression mode (#2675)
* Add validation suppression mode

* Add changelog

* Compile fix
2021-05-26 17:41:48 -04:00
jamesagnew 926bf393d3 Merge branch 'rel_5_4' 2021-05-17 09:10:58 -04:00
James Agnew 40ecafcd31
Don't unescape plus in match URLs (#2658)
* Don't unescape plus in match URLs

* Perf improvements

* Add changelog

* Test fix

* Work on tests

* Fixes

* Test fix
2021-05-16 21:51:23 -04:00
Tadgh d6252526c3 Move member variable 2021-05-11 10:53:10 -04:00
jamesagnew 2068372998 Version bump 2021-05-07 17:43:54 -04:00
jamesagnew 63c4b22cd3 Undo bump 2021-05-07 17:42:50 -04:00
jamesagnew 17e41520a4 Version bump 2021-05-07 17:41:39 -04:00
Tadgh e15825b336 Increase length, add test 2021-05-05 15:29:02 -04:00
James Agnew 970a9884f0
Avoid tag definition constraint (#2620)
* Avoid tag definition constraint failure

* Version bumps

* Add changelog

* Avoid guava issue

* Test fix

* Test fix

* Bump pom
2021-05-01 17:16:15 -04:00
James Agnew 95d7ba6fa8
Streamline Search SQLs (#2576)
* Streamline SQL queries

* SQL query tidying

* Add changelog

* Test fixes

* Tests passing

* Ongoing work

* Test fixes

* Test fixes

* Build fix

* Cleanup

* Fixes
2021-04-28 10:29:09 -04:00
James Agnew 2ba1005762
OpenAPI Support (#2560)
* Start work on OpenAPI

* Fixes

* Work on OpenAPI

* Cleanup

* Cleanup

* More swagger work

* Build fix

* More work

* More work

* Add documentation

* Docs fixes

* Add changelog

* License updates

* Add API

* Cleanup

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_plain/openapi.md

Co-authored-by: patrick-vachon-smilecdr <81274188+patrick-vachon-smilecdr@users.noreply.github.com>

* Work on scripts

* Add docs

* Compile fix

* Work on fixes

* Fix tests

* Test fix

* Test fix

* Build fix

* Tests

* Build fix

* Work on pipeline

* Version bump

* Test fix

* Version bump

* Test fix attempts

* Test fix

* Test fix

* Remove accidentally committed files

* Fixes

* Tets fixes

* Test fix

* Test fix

* Test fix

* Test fixes

* Test fixes

* Test fixes

* License header updates

* test fix

* Test fixes

* Test fix

* Test fixes

* Test fix

* Checkstyle bump

Co-authored-by: patrick-vachon-smilecdr <81274188+patrick-vachon-smilecdr@users.noreply.github.com>
2021-04-25 15:40:50 -04:00
Nick Goupinets 4e266462ec Bumped up version due to breaking changes 2021-04-20 16:17:27 -04:00
Tadgh 7cabcbd772 Fix partition selection for system request details 2021-04-16 11:12:40 -04:00
James Agnew ca2088f3ad
Add framework for Bulk Import (#2538)
* Start work on bul;k import

* Work on bulk import

* Have batch working

* Working

* Working

* More work

* More work on bulk export

* Address fixmes

* License header updates

* Test fixes

* License header updates

* Test fix

* Test fix

* Version bumps

* Work on config

* Test cleanup

* One more version bump

* Version bump

* CLeanup

* A few additions

* Test fixes

* Test fix

* Test fix

* Migration fix

* Test fix

* Test fix
2021-04-14 17:41:32 -04:00
Kevin Dougan SmileCDR b45ddcc3da
2543 - Fix issue where versionned references are not being returned properly. (#2544)
* 2543 - Fix issue where versionned references are not being returned properly.

* 2543 - Added changelog entries for this fix plus a previous fix for 2533.
2021-04-14 08:45:49 -04:00
Tadgh 99e10de5fb
Merge pull request #2487 from hapifhir/mdm-expansion-interceptor
MDM Expansion Interceptor (poc)
2021-04-09 11:23:04 -04:00
James Agnew a6cbf7eebd
Add support for hierarchycal valueset expansion (#2525)
* Add support for hierarchycal valueset expansion

* Add changelog

* Add tests

* Cleanup

* Test fix

* Test fixes
2021-04-06 17:36:54 -04:00
James Agnew b617c7690d
Search Parameter Infrastructure Cleanup (#2522)
* Search param service cleanup

* Work on collapsing search param

* Search param cleanup

* Work on build

* Test fix

* Test fixes

* Ongoing work

* Test fix

* Compile fix

* Test fixes

* Test fix

* Test fix

* License header updates

* Remove fixme

* Cleanup

* Cleanup
2021-04-05 17:40:10 -04:00
Tadgh 74be6761b1 Merge remote-tracking branch 'origin/master' into mdm-expansion-interceptor 2021-03-31 09:47:57 -04:00
Tadgh a044e8c4eb Add a delegator in daoconfig 2021-03-30 12:59:29 -04:00
Tadgh 02e6514176 Add modelconfig toggle 2021-03-30 12:57:09 -04:00
Diederik Muylwyk b720c32c1d Version bump 2021-03-29 18:23:16 -04:00
James Agnew dc41d5ad61
Add terminology mapping and display population interceptors (#2488)
* Start work on code mapping interceptors

* Work on generation

* Add terminology mapping and display population interceptors

* Add changelog

* License header updates

* Test fixes

* Fix typo

* Interceptor fixes

* License header update

* Test fixes

* Mapping fixes

* Version bump

* License header updates

* Add null tests

* Add null guard

* Intermittent fix
2021-03-21 12:03:57 -04:00
Frank Tao 85824efa51
Added index to improve the performance (#2385)
* Added index to improve the performance

* Added index for FORCED_ID

* Added comments for the index of while 'missing' is supported.

* Updated comments

* Moved the migration script to V5_4_0
2021-03-10 16:58:10 -05:00
James Agnew 807d9d53f0
Improve performance for large transactions (#2462)
* Performance tweaks

* Work on processing speed

* Disable no longer relevant tests

* Work on performance

* Work on tests

* Work on large transaction performance

* Add changelog

* Perf tweaks

* Test fixes

* Resolve FIXME

* Fixes

* Test cleanup

* Fix broken changelog entry
2021-03-09 18:16:02 -05:00
James Agnew be50a46d76
Tweaks to contained searches (#2461)
* Tweaks to contained searches

* Add changelog

* Build fix
2021-03-09 11:51:30 -05:00
Frank Tao bf8e890a0c
Supported contained resource search (#2441)
* POC for indexing on contained resource - test case may failed

* Test contained url

* Add a spt to handle contained flag

* Added search option for contained resource

* Impl contained resource search

* fixed typo

* Reworked on creating index based on the review comments

* Added changelog

* Added more test cases

Co-authored-by: jamesagnew <jamesagnew@gmail.com>
2021-03-09 06:50:23 -05:00
Tadgh de09c7e2da Bit of logic tidying 2021-03-03 19:03:31 -05:00
Tadgh 7b663bfe99 Add constants, fix up some tests 2021-03-03 18:51:03 -05:00
Tadgh 74066a81b6
Merge pull request #2420 from hapifhir/issue-2401-add-support-for-group-id-export
Issue 2401 add support for group id export
2021-02-26 10:23:42 -05:00