Commit Graph

106 Commits

Author SHA1 Message Date
longma1 01d6e15f90
2579 partitioned support for mdm (#3485)
* committed changes discussed at tasking

* Changed the subscription message to store the request partition

* modified mdm subscription loader to create a cross partition subscription in the default partition if multitenancy is enabled

* Fix errors when running Multitenant IT for mdm

* Added partition id to mdm golden resource search and creating/updating golden resource

* added partition id column to the mdm link table

* Added partition id of the source resource to partition id column of the mdm link table

* broken build

* working -links, missing hapi tests

* Fixed the mdm link db table due to feedback

* added IT for create link mdm operation, also added changes to pass the IT

* Fixed test for create mdm links, added test for update mdm link on partitioned server

* changed mdmLink search from searchbyExample to criteriaBuilder, added partitionHelperSvc to determine partition

* added test for merge golden resource and search golden resource on partitioned server

* added unit test for not duplicate on partitioned and non-partitioned server

* added criteriabuilder search and tests

* code cleanup

* added partition support to -merge-golden-resources operation

* test cleanup

* fixed  operations with multitenancy

* added test for mdm clear operation

* added waiting for batch job to finish in mdm clear operation test

* added query to dao to support partitioned server for mdm clear operation

* resolve mistake from merge conflict

* added mdm-submit operations, also fixed paging for -query

* added partitionIds constant

* fixed build issue where there was no Code.msg in mdm partition exceptions

* code review fixes

* fix broken test, also fix a bug where query link could not query by linkSource properly

* fix test stubbing

* bump hapi version to 6.0.0-PRE10-SNAPSHOT

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
2022-04-06 15:48:30 -04:00
Johnson Lu c42ecc5c08
Add error codes to all exceptions (#3278)
* Initial design of adding exception codes to exceptions

* Initial design of adding exception codes to exceptions

* change int module name to string

* add comment

* shorten message

* shorten message

* added checkstyle

* developing checker

* completed checkstyle plugin and unit test for it

* fix NPE

* fix checker so it ignores throws on exceptions that aren't new

* change parameter order in InvalidResponseException

* accept Msg.code() in any parameter since some exceptions like MessageException have the message string as the second parameter

* exclude generated files

* BIG COMMIT added error codes to all exception throws

* example test fix

* Modified tests such that the assert statements would better match the message responses.

* Modified additional tests such that the assert statements would better match the message responses.

* fix test

* merge master

* merge master

* Fourth commit of modified tests such that the assert statements would better match the message response

* Changes made to modify assert statements such that they match the new errors which now use error codes. Also changed ResourceNotFoundException to include error codes in its messages.

* Modified assert messages to better match new error codes

* Time Zone changes DRAFT. Certain tests pass in intellij and fail during mvn install

* Modified assert messages to better match new error codes

* Added comments for time zone fixes. Previous commit comment on time zone fixes was incorrect. By invaliding caches, all tests on both intellij and mvn install passed.

* Reverted changes on azure-pipelines.yml

* updated hapi-fhir version for checkstyle module

* changes to azure-pipelines.yml

* changes to azure-pipelines.yml

* change to pom.xml to include hapi-fhir-checkstyle in ALLMODULES

* changes to azure-piplines (these changes are to be reverted after testing)

* reverted changes to azure-piplines

* reverted change where hapi-fhir-checkstyle was added to pom.xml (hapi-fhir)

* Attempted fix on checkstyle build error

* modified azure-piplines.yml to change maven repository for checkstyle build

* Attempted fix on checkstyle build error

* checkstyle config location change for hapi-depoloyable-pom. (will be reverted if it does not work)

* attempted dependency add for hapi-depoloyable-pom. (will be reverted if it does not work)

* reverted changes for hapi-deployable-pom

* add javadoc

* update last code

* Attempted fix on checkstyle build error

* Finished adding in missing error codes to pass Checkstyle checks

* Test pipelining

* Fixing errors and bumping version (as suggested by Gary)

* Merge

* Fixing Checkstyle error

* Attempted fix for date error

* Add error code onto assert statement

* Add in a couple more error codes

* Fixing assert statements

* Bump to static version for dependency

* Adding error code to message

* Adding error codes to exceptions

* Adding error codes to exceptions

* Change to work with cdr branch

* Update to faulty error message

* Merge changes

* allow checkstyle pom to deploy

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Simon Zuccherato <simon.zuccherato@smilecdr.com>
Co-authored-by: Predap <simon.zuccherato@gmail.com>
Co-authored-by: Predap <46201929+Predap@users.noreply.github.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-01-31 13:10:15 -08:00
TipzCM 41193c60db
3335 allow alternative character encoding length in searchparam (#3336)
* added fixme

* 2714 added support for specifying max code lengths for supported phonetic encoders

* 3335git status

* 3335 updated changelog

* 3335 review fixes

* review fixes

* fixing test

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-27 17:29:07 +00:00
jamesagnew aafc42fdd8 License header updates 2022-01-04 13:37:10 -05:00
James Agnew d610d33ac3
Dependency bumps (#3264)
* Bump a number of versions

* Fixes

* Work on version bump

* Bump

* Test fix

* More version bumps

* Build fixes

* Test fixes

* Fixes

* Fixes

* Test fix

* Bump jacoco

* Test fix

* Test fix

* Test fix

* Build fix

* Build fix

* Remove versioned H2

* Build fix

* Avoid memory issue in validation tests

* JDK bump

* Restore compile with errorprone

* Force coloured output

* Force jansi

* Try again to force jansi

* Fix for hardcoded date that just passed

* Fixes to JPA

* Improve changelog

* Work on failing test

* Test fix

* Compile fix

* One more version bump

* Test fixes
2022-01-04 13:21:14 -05:00
Ken Stevens 9753b66610
2564 contained resource validation flag (#3225)
* First cut at multithreading bundle validation

* javadoc

* change test to a red-green test

* add daoconfig

* move bundle concurrency config from validationsupport to request validator

* Moved concurrent details from ValidationOptions to FhirValidator

* Remove bundleValidationThreadCount (it's set by creating an appropriate executor)

* validate threadpool prefixes

* FIXME cleanup

* reassign FIXMEs to JB

* only create the threadpool if it will be used

* Add the MdcTaskDectorator to all threadpools

* Add bundle path to concurrently validated bundle resource validation messages.

* Add test to check bundle entry path being added to validation messages

* Threaded validation of contained resources initial checkin

* FHIR validator refactored to maintain correct bundle path and async validation task

* Refactored to use validation complete invoke callback to support raw string resource or resource object

* switch from XML to JSON

* # Conflicts:
#	hapi-fhir-base/src/main/java/ca/uhn/fhir/validation/FhirValidator.java
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Updated to use new HAPI-FHIR version

* merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation

* Updated to use new HAPI-FHIR version

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit 910b7f676b.

* Revert "merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation"

This reverts commit c7d60fc3e6.

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit aa8a7af455.

# Conflicts:
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Fix Test dependencies after reverting changes.

* merge 'validate_contains_change' to 2564-contained-resource-validation-flag

* Disable threadpool prefix name validate check.

* merge master

* fix thread regexp

* pre-review cleanup

* change log

* fix test

* revert to XML as default validation serialization

* fix test

* fix test

Co-authored-by: Jaison B <jaisonb@gmail.com>
2021-12-08 15:18:54 -05:00
Tadgh 41ef7cf961
Support multiple EID systems for MDM. (#3178)
* Add implementation, changelog, docs, and tests

* Fix potential NPE

* Use constant instead of *
2021-11-24 06:05:13 +00:00
TipzCM 7e291bb469
3163 refactor some base mdm stuff (#3166)
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-11-16 10:47:44 -05:00
Tadgh e29ad5179a
Head off potential MDM bug (#3140)
* wip

* fix up error message

* wip

* Fix all tests

* Update old tests for failure messages

* Add another test case

* Remove dead file

* Fix log message
2021-11-04 22:09:46 +00:00
jamesagnew a54ebd1d30 License header updates 2021-10-11 17:22:54 -04:00
katiesmilecdr 3c80dba90e
[3040] finish implementation (#3049)
* [3040] finish implementation

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* [3040] throw exception if link exists

* [3040] add matchResult parameter/update documentation

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
2021-10-07 17:42:21 +00:00
Ken Stevens 060791aeb4
hapi-fhir-storage-api (#3006)
* rename hapi-fhir-jpaserver-migrate to hapi-fhir-sql-migrate and move hapi-tasks into jpaserver-persistence

* rename hapi-fhir-jpaserver-api to hapi-fhir-storage-api

* move bulk import apis

* create hapi-fhir-jpa

* create hapi-fhir-jpa

* move CircularQueueCaptureQueriesListener to japi-fhir-jpa

* move mdm logs to storage-api

* move gziputil to storage-api

* move quartz scheduling to hapi-fhir-jpa

* move default subscription channel factory to storage-api

* consolidated batch constants

* correct accidental Logs change

* remove dependency of cdr-api on cdr-persistence

* fix javadoc

* pom descriptions

* review feedback
2021-09-21 15:36:08 +00:00
Jaison B 8bb9aca8df Merge branch 'master' into 2849_add_new_mdm_param 2021-09-17 10:18:24 -06:00
Ken Stevens 661ca02a6a revert clobbered code 2021-09-09 09:28:01 -04:00
Nick Goupinets 35aaa98431 Merge branch 'master' into 2849_add_new_mdm_param 2021-09-07 11:27:49 -04:00
Nick Goupinets 010628fa86 Added asserts 2021-09-07 11:24:37 -04:00
Nick Goupinets c63a503841 Updated link processing for MDM events 2021-09-03 16:29:26 -04:00
Ken Stevens 346e29920a
Issue 2927 convert mdm clear to spring batch (#2929)
* initial roughout of mdm clear batch job

* still roughing out classes

* finished first draft of reader

* most tests passing now

* all tests pass.  now FIXMEs

* FIXMEs done.  Time for regression.

* fix test

* changelog and docs

* fix test

* pre-review cleanup

* version bump

* move spring autowire deps for cdr

* move spring autowire deps for cdr

* finally got beans working phew

* rearrange method calls so persistence module can perform clear operation on its own

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* review feedback

* review feedback

* Remove inheritance

* fix beans

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2021-09-03 14:58:19 -04:00
Nick Goupinets 4e012236c2 Updated comments and added misc refactorings 2021-08-27 16:31:23 -04:00
Nick Goupinets 6ea58f74d1 Code review 2021-08-23 10:55:09 -04:00
Nick Goupinets 8670f107b5 Updated link expansion 2021-08-17 13:17:44 -04:00
Nick Goupinets 4cfe11a221 Merge branch 'master' into 2849_add_new_mdm_param 2021-08-16 13:33:57 -04:00
Tadgh d5fe1eeef7 Move test out of IT 2021-08-09 16:13:47 -04:00
Nick Goupinets 891a6304d0 WIP 2021-08-04 11:06:14 -04:00
Nick Goupinets 0d34fe61c8 Adding tests 2021-07-30 11:17:56 -04:00
Nick Goupinets a2aa2ca1c3 2849 Added new parameter to MDM processing 2021-07-29 15:52:27 -04:00
James Agnew 6d37749be8
Add non unique combo search params (#2809)
* Start work on nonunique combo search params

* Work on SQL

* Version bump

* A fix

* Test fixes

* Fixes

* Undo version bump

* Test fixes

* Resolve fixme
2021-07-21 19:14:14 -04:00
Tadgh a4fb72975d
Merge pull request #2767 from hapifhir/2765-mdm-paging
Paging implementation for MDM query links operation
2021-07-01 23:16:47 -04:00
Tadgh cfcc939caa Fix broken tests, add license 2021-07-01 18:28:29 -04:00
Tadgh 86519a892a Add tests for link builder 2021-07-01 09:43:53 -04:00
Tadgh b963520c98 Fix bug in link builder 2021-07-01 09:33:19 -04:00
Tadgh 0893ba0039 Fix up operation param types, fix bug in link builder 2021-06-30 23:17:04 -04:00
Tadgh 1eb82d633d Rip out pagingprovider requirement 2021-06-30 18:41:18 -04:00
Tadgh 950096d42f remove demand for paging provider 2021-06-30 18:21:08 -04:00
Tadgh 9f3b262d6d Add alternate caller methods 2021-06-30 17:54:09 -04:00
Tadgh d407501071 More docs 2021-06-30 15:04:05 -04:00
Tadgh 883337a9f4 Tidying 2021-06-30 14:55:57 -04:00
Michael Buckley fa750d209e $mdm-submit operation was only submitting 100 resources and then stopping.
It now correctly submits all requested resources.
2021-06-30 14:48:28 -04:00
Tadgh 8f3cbb0a7f Refactor docs 2021-06-30 14:48:13 -04:00
Tadgh 61260dd1b1 Slightly more portable solution for paging 2021-06-30 11:54:59 -04:00
Tadgh a770d577cb Initial quick implementation of paging with tests 2021-06-29 17:46:03 -04:00
Nick Goupinets 5c9f2617da Code review 2021-06-22 15:23:03 -04:00
Nick Goupinets 95012623f6 Added null checks 2021-06-22 15:12:10 -04:00
Ken Stevens 32ac16e2aa
add a limit to mdm candidate searches (#2742)
* done

* verbal review feedback

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_5_0/2739-max-mdm-matches.yaml

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* review feedback

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
2021-06-18 20:48:34 -04:00
Nick Goupinets dc89a4abd7 Updated identifier matcher to only match on non-empty values 2021-05-26 15:13:43 -04:00
Tadgh ac7c684437 Refactor to support all new tests 2021-04-29 17:01:19 -04:00
Nick Goupinets 1f10a15c35 Merge branch 'master' into 2515_MDM_survivorship_rules 2021-04-19 17:33:53 -04:00
jamesagnew b294f7d208 License headers 2021-04-14 18:00:52 -04:00
Ken Stevens 550602b2f1
added numeric matcher (#2547)
* added numeric matcher

* changelog

* performance optimization

* fix test
2021-04-14 13:15:30 -04:00
James Agnew b617c7690d
Search Parameter Infrastructure Cleanup (#2522)
* Search param service cleanup

* Work on collapsing search param

* Search param cleanup

* Work on build

* Test fix

* Test fixes

* Ongoing work

* Test fix

* Compile fix

* Test fixes

* Test fix

* Test fix

* License header updates

* Remove fixme

* Cleanup

* Cleanup
2021-04-05 17:40:10 -04:00