Commit Graph

189 Commits

Author SHA1 Message Date
Ken Stevens 04c28b6c08
Schedule jobs via IJobScheduler instead of in @PostConstruct (#4415)
* parameterizing BinaryStorageInterceptor

* parameterizing BinaryStorageInterceptor

* changed jobs scheduling from @PostConstruct to coordinated by HapiLifecycleService events

* fix test

* remove events for now

* fix test

* simplify design

* FIXME

* changelog

* fix compile issue

* review feedback.  partially implemented

* review feedback

* review feedback

* bump hapi version to 6.3.8-SNAPSHOT

* fix test

* mirror contextclosedevent handlers with contextrefreshedevent

* version bump

* licenses

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-14 23:51:04 -05:00
James Agnew 7a31860376
Improve query count accuracy (#4424)
* Improve query count accuracy

* Add changelog

* Test fix
2023-01-11 17:53:04 -05:00
Justin McKelvy 8c48a6fb91
Clinical reasoning module (#4347)
* Add new Clinical Reasoning module

* Add config to get test up and running

* Removed FhirClient from tests

* First passing test

* Remove unused utilities

* Build updates

* exclude xxp3 and substitute with unbundled version

* Refactor to use provider factory and loader pattern

* First passing dstu3 test

* CR r4 test config and immunization tests

* Add base test classes

* Exclude transitive dependency xmlpull

* More dependency cleanup

* Refactor to remove JPA dependencies, rename module to hapi-fhir-storage-cr

* Fixes for duplicate xpp3 classes

* Remove more references to jpa

* Add mock FHIR server for terminology endpoint tests

* fix cql file for making providerr4 tests pass

* fix pom

* fix generic

* Adding r4 test cases for measure operations.

* Fix endpoint tests

* javadoc comment, temporarily comment out test results for practitioner sync

* javadoc typo fix

* adding java docs for cr module WIP

* Updates to clinical reasoning documentation

* More documentation updates

* docs now compile

* fix test

* DqmR4IT now successfully starts up, but the test does not pass

* Fix additionalData test

* Make measure service bean overridable

* adding clinical reasoning javadoc information

* Rename interfaces to start with I

* More styleguide updates

* Reformat all code in the storage-cr module

* More coding convention updates

* Remove bunches of unused code, reogranize so that fhir version in package name is consistent

* Clean up some static array initializations

* add practitioner patient list

* refactor product-line constant

* test dependency update

* add change log for cr module

* Add hapi-validation to the pom

* Exclude dependencies that are causing duplicates

* More dependencies and exclusions

* Make some base MeasureService components protected

* Update to latest cql compiler release

* fix measure service for null subject & practitioner and open up practitioner test

* Rev to release versions

* Add remaining Dstu3 tests

* update hedis bundle from policyHolder to beneficiary

* update snapshot

* Delete old CQL module

* Post-master-merge recovery

* Fix Java 17 feature usage issue

* update changelog with more information about new functionality

* Remove support chain wrapper usage

* Add javadocs to IResourceLoader

* added the property versioning for antlr jar.

* addressing code review comments.

* addressing code review comments.

* Fix typos in changelog

* Remove azure pipeline for cql module, add it for cr module

* Address exception comments, add Msg.code calls to exceptions

* Fix self-assignment bug

* Add storage-cr module to bom

* addressing coding comments.

* Addressed the variable naming convention to camelCase.

* Snapshot version bump

Co-authored-by: Brenin Rhodes <brenin@alphora.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: Chalma Maadaadi <chalma@alphora.com>
Co-authored-by: mdnazmulkarim <nazmul.karim@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-01-10 19:49:57 +00:00
Tadgh f2f29a1a32 Add forgotten license changes 2023-01-05 17:31:43 -08:00
karneet1212 cfa6c13262
Add resource type as a parameter for $mdm-query-links (#4372)
* Added Solution + test

* Removed old methods

* fixed issues

* Added refactoring

* Deprecated methods on the interfaces

* Remove todo

* Documentation

* Changelog

* Changelog

* Changelog

* Changes made based on comments

* Changes made based on comments

* Clean up

* hapi bumping to version 6.3.6-SNAPSHOT

* hapi bumping to version 6.3.6-SNAPSHOT
2022-12-22 09:54:32 -05:00
James Agnew 0d5f8da93f
API improvements for MegaScale (#4112)
* Start work on sharded DB

* Work on migrating

* Work

* Ongoing work

* Ongoing work

* Work

* Work

* Ongoing work

* Work

* Work

* Test fix

* Test fix

* Work

* Compile fix

* Test cleanup

* Test fix

* Resolve fixmes

* Resolve fixme

* Resolve compile errors

* Compile fix

* Fixes

* Work

* Add note for later

* Test fix

* Test fixes

* Build fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Fixes

* Fixes

* Remove dead code in pom

* More test cleanup

* Docs fix

* Changes

* Fix

* Fixes

* Resolve merge conflicts

* Build update

* Merge build fixes

* Version bump

* Address review comments

* Review comments

* Review comments

* Test fixes

* Test fix

* Cleanup
2022-12-20 12:05:15 -05:00
Nathan Doef 7b86eda8a9
Refactor Common SearchParameter Functionality (#4321)
* refactor checkpoint

* clean up

* add method to find runtime search param by id

* new search param cache tests

* extract SearchParameter validation functionality

* create common interface for combo searchparameters

* Remove ResourceTable from extractor interface

* clean up

* move SearchParameterDaoValidator

* resolve code review comments

* remove reflection

* change log

* bump version to 6.3.4-SNAPSHOT

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-12-15 06:46:45 -05:00
Nathan Doef 360f32f3e4
fix index migration error (#4365)
* fix index migration error

* add exception to duplicateNameValidationExceptionList

* - change index name to match convention
- drop old index h2 automatically created

* Revert "- change index name to match convention"

This reverts commit 59f35a1891.

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-12-14 12:40:53 -05:00
kateryna-mironova 017cc4e29c
4244 - Deprecate usages of ResourcePersitentId in jpa (#4245)
* init rev

* id to jpa and mongo id

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* fixed conflicts, pulled from master

* fixed code for passing tests successfully

* changed PersistentId to Jpa in MdmSearchInterceptor

* added the todo comments with questions

* added one more question

* renamed TODO questions

* first pass at converting to generic ResourcePersistentId

* fix test

* fix test

* fix test

* make ResourcePersistentId abstract

* Generic type MdmLink

* Final TODOs

* Final TODOs

* cleanup

* fix test

* fix id to string

* fix cyclic package dep

* add <?> to Base so we can extract interface

* Revert "add <?> to Base so we can extract interface"

This reverts commit a806f3d2bc.

* switch to interface

* javadoc

* replace base with interface

* fix test

* add generic template to mdmdao

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary MongoPid casts

* fix test

* fix regression

* fix compile issues arising from merging master

* fix beans

* bump version to 6.3.3-SNAPSHOT

* change logs

* review feedback

* review feedback

* fixme

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-12-09 09:30:51 -05:00
James Agnew 50ca94eded
Improve JPA response messages (#4293)
* Improve JPA response messages

* FIxes

* Test fixes

* Test fixes

* Ongoing testing

* Work on messages

* Work on messages

* Add valueset

* Add response code enum

* Version bump

* Undo bump

* Improve changelog

* Test fixes

* Add javadocs

* Version bump HAPI

* Test fixes

* Test fix

* Test fixes

* Test fixes

* Account for review changes

* Test fix

* Docs fix

* Work on API

* Improve SqlQuery api
2022-11-28 07:52:12 -05:00
James Agnew c3697a5f87
JPA Cleanup (#4278)
* JPA cleanup

* Test fixes

* Cleanup

* Resolve fixme

* Fix bad changelog entry

* A bit of documentation reworking

* Version bump

* POM bump

* Avoid timeout

* Build fix

* HTMLUnit bump
2022-11-24 14:32:56 -05:00
Tadgh b1e3468dcd
Rel 6 2 1 mergeback (#4288)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix up checkstyle

* log to console

* Remove double import

* Version bump

* licenses

* Fix up version

* fix build

* wip

* wip

* Fix up search expansion to use new behaviour of idhelper service

* Fix up MDM Search Expansion

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-11-20 08:14:28 -08:00
michaelabuckley 6c4283e930
Start moving forced-id inline to ResourceTable (#4260)
New fhir_id column in HFJ_RESOURCE.

Migration, changelog, and insert hook for inline fhir_id column.
2022-11-16 13:38:52 -05:00
James Agnew b3ebbe7933
ValueSet cleanup (#4227)
* POST release deferred cleanup

* Fixes

* One more test fix

* Test fixes

* Test fixes

* Test cleanup

* Test fixes

* Fixes

* ValueSet cleanup

* Test fix

* Test fixes

* Fixes

* Test fixes

* Fixed

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixc

* Build fix

* Fix merge artifact

* Build fix

* Work on tests

* Test fixes

* Work

* Fixes

* Changelog fix

* Add changelog

* Test fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Tests

* Bumps

* Fixes

* Add errorprone

* Drop bz2 bins

* POM fix

* Build fix

* Update

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-08 22:18:36 -05:00
Tadgh d3367cfede
Rel 6 2 mergeback (#4257)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix

* Bump HAPI version

* Remove jetbrains

* Fix broken test

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-11-07 23:57:57 +00:00
samuelwlee2 1b55f49a60
4153 extend $member match operation to store consent resource if matched (#4165)
* added senstive and regular filter

* add UUID to Consent

* blah

* fix merge with master

* fixing merge

* blah

* fhir tests

* use constructor injection

* put all tests together

* blah

* blah

* finalize

* add consent update patient and performer reference

* fixes after code review

* blah

* refactor to save consent once

* blah

* fix performer and add validation

* code clean up

* add changelog

* add hrex member match flag

* fix PatientMemberMatchOperationR4Test

* adding code fixes after code review

* move changelog file

* bump version

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2022-10-31 13:33:54 -04:00
Tadgh 31a193fbd3
Bump to 6.3.0 (#4211) 2022-10-28 02:40:59 +00:00
James Agnew 7a39f8cbaf
Make reference extractor more reusable (#4205)
* Refactor

* Make reference extractor more reusable

* Test fix

* Test fix

* Headers

* Small refactor
2022-10-27 21:31:39 -04:00
James Agnew a71b8fad56
Add R4B Support (#4150)
* Add r4b stuff

* Work on R4B

* Core version bump

* Revert some changes

* Add R4B

* Docs

* Test fixes

* Work on R4B

* Tests passgin

* Add config

* Work on merging

* Work

* Work

* Work

* Test fix

* Fixes

* Test fixes

* Test fix

* Work

* Work

* Fixes

* Test fixes

* Fixes

* Test fix

* Test fix

* Test fix

* Version bump

* Test fix

* Test fix

* ompile fixes

* Fix

* Resolve build warnings

* Updates

* Test fixes

* Fix docs
2022-10-20 13:57:06 -07:00
KGJ-software 69d55b7184
3052 Changed IBinaryStorageSvc MaximumBinarySize from int to long. (#4123)
* 3052 Changed IBinaryStorageSvc MaximumBinarySize from int to long.

* 3052 Added changelog

* 3052 Bumped version

* 3052 Remove mysterious +1 on maximum binary size causing overflow.

* Bumping version to 6.2.0-PRE16-SNAPSHOT

* Adding + 1L as original implementation.

* Getting all tests to pass dealing with + 1L.

* Getting all tests to pass dealing with + 1L.

* Adding migration definition and removing unnecessary comment.

* Preventing register overflow when setting maxBinaryStorage.

* Mering relevant test.

* Fixing typo in migration script

Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-10-19 06:09:02 -04:00
James Agnew 422ef87a5d
Standardize on one Transactional annotation (#4130)
* Standardize on one Transactional annotation

* Add changelog

* Disable checkstyle rule

* Fix up transactions

* Test fixes

* Test cleanup

* License headers

* One more dep bump

* Resolve commit issue

* Test fixes

* Test fix

* Dep bump

* Fix

* Fix test

* Version bump

* Test fixes
2022-10-13 16:26:11 -04:00
Luke deGruchy 5831cbedf5
Concurrent cascade delete operations leading to confusing errors (#4126)
* Add unit tests and supporting classes from gitlab issue.

* Commit with changes thus far to be reviewed.   Namely, add savepoint capabilities to HapiTransactionService, inject HapiTransactionService into CascadingDeleteInterceptor, add savepoint capabilities to the TransactionDetails, and catch an Exception in BaseHapiFhirResourceDao.delete() then rollback to the checkpoint.

* Fix compile error.

* hapi transaction service test

* hapi transaction service test

* hapi transaction service test

* First draft of RetryTemplate and flush() logic.  Try unit testing with PointcutLatch.

* hapi transaction service test

* Latest changes including moving RetryTemplate, cleaning up unit tests.

* First attempt at correct design.  Fix compile errors.

* Add failed rollback savepoint test.

* fix propogation

* interesting test failure in SafeDeleterTest

* Latest code

* move delete

* test passing

* all tests pass

* yay we're done just cleanup

* First round of cleanup:  Get rid of all exploratory logging, remove unneeded dependencies, move creation of RetryTemplate to SafeDeleter.

* More cleanup.  Add changelog.

* Bump to 6.2.0-PRE13-SNAPSHOT.

* Code review feedback:  Refactor SafeDeleter to a service and bean.  Add comments.

* Code review feedback:  Comments on unit test assertions.

* Code review feedback:  Rename SafeDeleterSvc to ThreadSafeResourceDeleterSvc.  Add javadoc.  Add logger to DelayListener.   Remove useless commented out code.

* Fix unit test failure in CascadingDeleteInterceptorTest.

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-10-11 16:48:01 -04:00
Ken Stevens 40163f73d2
Ks 20220916 remove flyway (#4040)
* removed flyway, started coding persistence

* started wiring up new dao service

* replace jpa with sqlbuilder

* down to 16 failed tests

* remove outOfOrder test.  We no longer support validating order.

* Removed noFlyway mode.  It is no longer a thing.

* merge migrator

* down to 1 failing test

* cleanup

* removed unused noflyway cli arguments

* cleanup

* cleanup

* cleanup

* cleanup

* create MigrationTaskList to replace the List<BaseTask> that gets passed around a lot.  This allows us to add a bunch of behaviour to that as a first-order thing

* fix test

* fix test

* improve migration table detection to use the catalogue

* Msg.code()

* cleanup

* save version in same format as flyway

* cleanup

* add migration result to migration result class

* failed migrations stay in the table and don't really affect anything.

* change log

* docs

* ugh Flyway requires lower-case column names

* integration test fixes

* log

* fix oracle issue

* lowercase tablenames for postgres.  fussy fussy!

* Revert "lowercase tablenames for postgres.  fussy fussy!"

This reverts commit 92c9cb263b.

* revert to capital table names.  surround table name by quotes for Postgres.

* added log message before executing find all query

* changed log message from info to debug

* javadoc

* pom cleanup

* review feedback

* Update hapi-fhir-sql-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/MigrationTaskList.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* review feedback

* review feedback.  Use prepared statement.

* Change installedOn insert from timestamp to date

* Change installedOn column type from timestamp to date

* bump hapi version

* fix poms

* fix poms and cdr compile

* add exclusions to clean up build

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2022-10-03 22:40:09 -04:00
weiping202209 66baa8263d
Fix bug in history search (#4009)
* add failing test

* Fix history search bug and add tests for two more scenarios

* Fix history search bug and add tests for two more scenarios

* - added failing test condition

* Fix history search bug and add tests for two more scenarios

* Fix build test failures

* Fix build test failures

* Add extra verifications for scenarios of _since

* Refactor dao.history interface to add overload method and shall come back to remove old method once cdr is refactored.

* Update with new hapi interface

* Add depreciate comment

* Add depreciate comment

* Version bump to mvn 6.2.0-PRE11-SNAPSHOT

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-10-03 09:00:41 -04:00
Ken Stevens e9a855d3f6
remove spring batch (#4064)
* remove spring batch
it compiles
expect test failures

* fix test transaction manager bean name (wow that was quite the hunt the wumpus!)

* fix test and fix build

* fix test and fix build

* fix test

* remove spring batch schema initialization

* bump hapi version

* fix schema migrator

* bump to PRE-10

* merge master

* change logs

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-30 13:18:48 -04:00
Tadgh b472c4bb54
4083 batch permissions (#4095)
* Move threadpool factory into a bean

* Move threadpool factory into a bean

* change return type

* Add license files

* Version bump

* Fix test
2022-09-29 17:24:05 +00:00
michaelabuckley 96b92153f6
Implement composite search parameters in HibernateSearch for JPA (#3839)
Implement composite search parameters in Lucene/Elasticsearch Hibernate Search indexing

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-09-27 00:20:06 +00:00
michaelabuckley ee1cb4e392
Cleanup and utilities from composite work (#4066)
Utilities and cleanup from composite work #4066
2022-09-24 00:47:25 +00:00
jdar8 c647154777
Everything operation with type param code review fixes (#3926)
* check for valid resource type

* add test and refactor test class

* refactor auto get method name

* introduce parameter object to patient everything methods

* introduce parameter object to patient everything methods

* change record to class

* add hapi error code

* refactor

* refactor

* change code num

* refactor getmethodname to util method

* add docs and rename fields to conform to fhir standard

* typo

* version bump

* resolving merge conflicts.

* Bumping version

* Bumping version to 6.2.0-PRE5-SNAPSHOT

* Encapsulating parameters in a PatientEverythingParameters object.

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-09-07 06:44:36 -04:00
Ken Stevens 100b8f9190
batch2 rework (#3964)
* consolidated Batch2Helper
check status in assert before running maintenance
ignore illegal job instance state transition

* Fix mock tests

* rename intermittent to IT

* extra logging

* into to debug

* review feedback

* added more logging

* removed fasttracking.  replaced with support for triggering a maintenance run

* fixed tests

* fixed tests

* add asserts

* licenses

* snakeyaml cve version bump

* data migration

* merge origin/master

* merge origin/master

* fix test (support triggering jobs when scheduling is disabled)

* add counter to abort potential infinite loop after 100000 calls

* compile error

* move multipartition mdm tests out to a separate IT
update mongo job instance creation with new logic

* fix race condition

* back out infinite loop circuit breaker.  it breaks terminology tests that depend on this loop running > 100000 times.

* fix test (triggered job key was missing group)

* revert saveAllDeferred

* Almost finished getting JpaPersistenceR4IT to pass

* testBulkExportDeleteForOnGoingJob still not passing

* testBulkExportDeleteForOnGoingJob still not passing

* Removed method `fetchInstanceAndMarkInProgress`.  It was clobbering other state transitions like CANCELLED.

* undo rename of cancelled to cancelRequested.  Thymeleaf depends on the old name

* ExpandResourcesStep requires a synchronous search

* Tests are passing but man testBulkExportDeleteForOnGoingJob is a flakey test

* fix test

* fix test

* change log

* allow ERRORED -> FAILED state transition

* add a timeout to saveAllDeferred()

* improve error message on timeout message

* fix tests

* added more logging

* added more logging

* fix semaphore issue

* Msg.code

* fix test

* javadoc

* switch all batch loggers to batch troubleshooting log

* add unit test, improve logging

* move TimeoutManager to its own class.  Add tests.

* fix tests

* comment cleanup

* test race condition

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-05 00:04:54 -04:00
Tadgh e7a4c49aac
Rel 6 1 mergeback (#3927)
* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Bump Version for non-breaking purposes

* Fix broken test

* bump error code

* revert internal logic

* Revert offset mode change

* wip

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-08-18 17:21:27 -07:00
Tadgh dcfaf42e73 add new version enum, bump version, new folder 2022-08-02 21:46:20 -07:00
jdar8 5f1c50a6b8
everything operation with type param (#3842)
* wip+test

* wip, rough implementation

* clean up

* clean up

* clean up unused methods

* Completing implementation.

* Removing TODO.

* Replacing toList() with Collectors.toList()

* support other fhir versions and everything instance

* tests wip

* tests for instance

* Adding default methods in interface

* add testsss

* Removing @Deprecated.

* fix spacing

* changelog

* reword changelog

* Removing default methods from interface.

* Bumping version to 6.1.0-PRE28-SNAPSHOT

* Adding parameter to method call following interface modification.

* Fixing documentation and parameter order for consistency.

* Bumping version to PRE30.

* Providing backward compatibility.

* fix pom

* licences

* Fixing pom dependencies.

* Fixing more pom dependencies.

* Fixing dependencies for test-utilities.

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-08-02 20:51:37 -07:00
Nathan Doef e2bed96d74
3776 add https support for hapi fhir cli using tls (#3777)
* wip functional
TODO
- add tests
- look at adding TLS to other RestfulClientFactory classes (FhirwebRestfulClientFactory, JaxRsRestfulClientFactory, OkHttpRestfulClientFactory)

* move BaseRestServerHelper to Hapi-Fhir

* - add TLS support for JaxRsClient and OkHttpClient
- add tests

* -  to use resources and files

* -  ci fixes

* -  ci fixes

* -  ci fixes

* -  ci fixes

* - refactor and tests

* - ci fixes

* - tests

* - tests

* - tests / refactor

* - clean up

* - changelog

* - remove OkHttpClient HTTPS support

* - ci

* - Ken's code review changes

* - Ken's code review: method renaming

* - Ken's code review: javadoc

* - Code coverage

* - Create HapiFhirCliRestfulClientFactory to encapsulate new HTTPS functionality. Remove modifications to the FhirContext API.

* - version bump

* - fix unable to bind port errors by using Jetty functionality to assign random ports

* Revert "- version bump"

This reverts commit cfce82576a.

* - version bump

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-08-02 09:27:02 -04:00
David be0b6f1a0b
3840 add sort parameter to fetching jobs (#3861)
* added a paging api to ijobcoordinator

* adding changelog

* moved location

* added a comment

* Added Sort parameter to JobInstanceFetchRequest

* Bump HAPI-FHIR version

* Updated JPA api to include sort

* Add changelog

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: David Chen <david.chen@smilecdr.com>
2022-07-29 09:12:53 -06:00
longma1 f933ff3d9c
2878 mdm mongo support (#3748)
* small refactor

* refactored MDM module to use iterface IMDMLink, refactored MDMLink to use ResourcePersistenceId wrapper rather than pid as long

* removing IJpaIdHelperService and refactor IIdHelperService

* fixed bugs for GoldenResourceMergerSvcImpl and FindCandidateByExampleSvc

* changes to IIdHelperService

* move MdmSearchParamSvc and MdmSubmitSvcImpl out of jpaserver-mdm

* moved JpaResourceLoader to hapi-fhir-storage, renamed to ResourceLoaderImpl

* added error Msg code, removed Fixme

* fixed error code, update last used code in Msg.java to 2083

* fixed accidentally throwing unprocessableentityException

* added MdmLinkExpandSvc interface

* build adding license to files

* MdmLinkdao refactor

* fix build

* mdm config refactoring

* MdmStorageInterceptor and IExpungeEverythingService changes

* fix bug where a pid is converted to ResourcePersistenceId twice

* minor refactor to IMdmLink, change id from long to ResourcePersistenceId

* changed PageImpl to Page interface

* refactor MdmSearchExpandingInterceptor to use interface

* fixed missed refactor where the resourcePersistenceId is still assumed to be long

* resolve test failures resulting from merging master

* changed conflicting exception code

* fixed issue with bulk export wrapping resourcePid inside resourcePersistenceId

* added IMdmLinkImpl to MdmSettings

* fixed missing import in IdHelperService from merging master

* fixed MdmLinkDaoSvcTest importing wrong MdmLinkTuple

* restored JpaIdHelperService and added changelog

* modified msg.code to non-duplicate

* code review changes

* refactored some bulk export batch 2 files to accomodate change in beans being exported

* fixed jpa bulk export failing tests

* removed duplicate bean declaration of MdmLinkImplFactory in BaseMdmR4Test

* merged master and bump version to pre25

* extracted expungeEverythingByType to IExpungeEverythingService

* fixed wiring issue

* messed up merging pom with master

* fixed broken changelog

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Samuel Lee <samuel.lee@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-29 08:58:40 -06:00
TipzCM 59540ce803
3813 paging job coordinator (#3814)
* added a paging api to ijobcoordinator

* adding changelog

* moved location

* added a comment

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-07-26 11:29:37 -06:00
Ken Stevens 533019f051
Response Terminology Wiring (#3807)
* cleaned up response terminology wiring

* jpa is failing atm

* bump pom versions

* double build RAM

* bump hapi version to PRE-22

* license headers

* re-bump to PRE-22

* tag intermittent

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-25 18:41:01 -04:00
Ken Stevens f01e4a7ad2
Remote Terminology support refactor (#3822)
* Revert "test passes with a lot of copy/paste"

This reverts commit f2f28e22ef.

* make translateOutcomeToResults public

* double RAM

* final consolidation.  move shared code out to static utility methods

* revert azure memory
disable jacoco on jpaserver test prokect

* bump hapi version

* rebump hapi to PRE-21

* licenses

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-25 11:47:31 -04:00
Etienne Poirier d2fd0989c3
3820 Providing an exception for search request with invalid or incomplete tokenParam (#3825)
* Creating an exception to be raised when searching for _tag/_provide/_security where the associated tokenParam is not properly formatted.

* Adding changelogs.

* Adding integration test.

* Rewording ChangeLog.

* Rewording exception message.

* Addressing code review comments.

* Addressing code review comments.

* Bumping hapi-fhir version to 6.1.0-PRE20-SNAPSHOT

* Modifications to pass checkstyle.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-07-22 11:55:44 -04:00
Qingyixia 592190701c
Implement patch in mongo db (#3781)
* Not the final version

* Support PATCH operation

* minor modifications

* minor modifications

* Final version

* Final version

* Version Bump

* bump version again

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-17 21:26:34 -04:00
TipzCM 8ed0a18ef4
3531 implement bulk processors (#3566)
* base batch2 bulk export steps - unimplemented

* added the basic steps

* adding tests

* we have added the tests

* added tests for bulk export stages

* implementing jpa

* 3531 adding the jpabukexportprocessorimplementation

* 3531 first step for mongo bulk export

* jpa stuff updated

* 3531 hapi fhir stuff

* trying to fix up tests

* fixing test

* fixed most tests in batch export

* deprecate

* need some more guidance here

* fixing some tests

* some steps taken

* blah

* stashing

* test

* fixing tests round 1

* many a changes

* updating version

* small updates

* merging in mster again

* fixing up serialization

* added the permission checkers

* cleanup and sonarkube crap

* some more tests

* updating adding tests

* crimes

* blah

* updatingmsg number again

* removal of commented code

* cleanup

* merge confflicts

* fixing msg

* review fixes round 1

* more review fixes

* merging in master

* fixing poms

* review points

* test fix

* test fix

* fix test race condition

* truncate error

* add stack trace to error

* rename to IT

* fix race condition in test

* finally fixed race condition in test.  this time I'm optimistic hapi ci will finally pass

* fix mock test.  not a fan of mock tests like this.

* any job definition with a reducer step must be gated.

* fixed common batch bean wiring

* fix test

* lazy init ResponseTerminologyTranslationSvc

* change the way we lazy init ResponseTerminologyTranslationSvc

* change the way we lazy init ResponseTerminologyTranslationSvc (for the third time.  this is super tricky!)

* disabling testGoldenResourceWithCircularReferenceCanBeCleared for now

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-17 17:49:03 -04:00
TipzCM 0c644271ce
3770 fix batch2 failure loop (#3779)
* 3770 fixing error handling

* some updates to error handling

* changelog

* blah

* bumping version

* fix test failures

* code review points

* review points

* add retry test

* assert counts in the test

* test fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-15 12:42:46 -04:00
StevenXLi 2bc6c91153
3719 enable map response terminology for bulk export (#3735)
* added failing test for Map Response Terminology for bulk export

* moved logic for terminology translation out of the interceptor to svc, added translation logic to bulk export file writer, added more tests

* added changelog

* Wip refactor dependency fix

* wip terminology translation code refactor

* terminology translation code refactor

* refactor clean up

* Remove circular dependency

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_1_0/3719-enable-map-response-terminology-for-bulk-export.yaml

* bump version of hapi to 6.1.0-PRE7-SNAPSHOT

* doc update

* fix issue when search param registry doesn't contain search param for the resource type

* fix beans

* hapi version bump from PRE7 to PRE10

* doc update

* add ResponseTerminologyTranslationSvc as a constructor variable to the interceptor

* fixed ResponseTerminologyInterceptorLoaderTest, added IT for cdr side

* version bump to PRE13

* removed unused import

Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-12 12:24:15 -04:00
Ken Stevens 0526da080b
3757 move delete expunge to batch2 (#3759)
* begin with failing test

* begin with failing test

* remove unused Spring Batch classes.
convert service classes

* move provider

* just one test left to fix

* fix test

* msg.code

* change log

* bumping hapi to PRE12

* fix test

* review feedback

* license update

* fix intermittent and status update atomicity

* restore url-level partition selection

* fix tests

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-09 17:25:55 -04:00
Nathan Doef b68ccb373a
3693 Search by id is returning a deleted resource with a client-generated id (#3694)
- Added functionality to optionally filter out deleted resources when resolving forced ids to persistent ids
- Bumps version

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-05 12:37:25 -04:00
Ken Stevens 3a28920ea7
Add error-handler callback and ensure batch2 completion handler can't be called twice (#3752)
* rename field that drives everything includes

* discovered design flaw in completion

* fix intermittent

* ensure batch2 completion handler can't be called twice

* changelog

* changelog

* javadoc

* bump hapi version

* licences

* Batch2 job definitions can now optionally provide an error handler callback that will be called when a job
instance fails, errors or is cancelled.

* Changed error handler and completion handler to take a "read-only" copy of an instance instead of just certain fields.

* comment

* javadoc

* fix test

* change step execution context from taking instance id to instance

* review feedback

* fix test

* replace 20 or so jetbrains annotations

* fixed a few incorrect @NotNull annotations

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-04 23:27:58 -04:00
Ken Stevens 069f20a5f7
batch2 management operations api (#3742)
* first test passes.  onto the next one

* two down two to go

* three down one to go

* licenses

* basic admin-json plumbing confirmed with IT

* FIXME

* FIXME

* FIXME

* FIXME

* changelog

* bump hapi version to 6.1.0-PRE6-SNAPSHOT

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 23:02:17 -04:00
jmarchionatto 84fe58cc86
Hsearch namespace consolidation (#3733)
* Initial implementation

* Implement job cancellation

* Works for date and token. Cleanup required.

* Works for string and quantity.Cleanup pending.

* Works for reference and perform cleanup.

* Cleanup configuration and add missing tests

* Eliminate sort property registry (no go for a cluster) for a static property map.

* Missed in previous commit

* Implement HSearch number parameter

* Implement HSearch number sorting

* Unify different hibernate search names (HibernateSearch, Freetext and Lucene) to HSearch

* Move analysis configures for lucene and elastic to same file to enforce the need to keep them in synch

* Use new defined constant

* Rename package too

* Rename additional elastic references

* bump version to 6.1.0-PRE5-SNAPSHOT

* Restore Hap prefix to hapi configurers

* fixing hapi version of new maven module

* fix pom

* fix pom

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 18:38:05 -04:00
Ken Stevens 49d298dba1
mdm clear move to batch2 (#3671)
* begin with failing test

* continue building out

* continue building out

* reworking chunk api

* replace ChunkId zippers with proper iterator

* progressing test

* yay test passes!

* switch to idchunk interface

* fixmes

* compiles

* fixme

* fixme

* rename classes

* rename json classes with json suffix

* work towards homogeneous chunks

* fixme

* fixme

* unit test

* fixme

* fixme

* pull out builder

* moar tests

* moar tests

* preserve order

* moar test

* moar test

* fixme

* fixme -> wip to start ci

* msg code

* msg code fix test

* fixme

* change provider implementation

* final WIPs

* IT failing

* IT passes

* java 11

* fix test

* fix test

* fix test

* changelog

* use batch size if provided

* last WIP

* fix test

* fix test

* fix test

* whitespace change to trigger new build

* fix test

* bumping hapi version

* bumping hapi version

* fix test

* review feedback

Co-authored-by: Ken Stevens <ken@Kens-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-25 20:26:44 -04:00
TipzCM 48c28997e6
3662 batch2 reduction step (#3681)
* adding a bunch of stuff

* most of the steps are working now

* writing more tests

* adding more tests

* updated the batch2 coordinators

* updating version

* some minor test tweaks

* added a new test

* adding more tests

* blah

* blah

* review fixes 1

* fixing tests and refactoring

* review points

* updated for code review

* review fixes

* updated

* step 1 on review step 5

* final touches

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-20 18:59:01 +00:00
JasonRoberts-smile b9579313ef
Jr 20220519 expunge config params (#3627)
* fix config settings for system-level expunge

* clean up expunge everything interface

* extract common system provider

* changelog

* version bump

* touch changelog to force a fresh pipeline

* re-enable publishing of checkstyle

* back out Gary's Sonatype change entirely
2022-05-23 18:55:42 -04:00
Mark Iantorno 544ef3de1b
Update to 6 1 (#3582)
* added changelog folder, upped version

* version enum
2022-05-03 20:34:25 +00:00
Tadgh 581f40f373
Upgrade Hibernate Search (#3572)
* wip

* updated rest high level for lucene mismatch

* Remove custom JNA

* Bump to 13

* License files

* Changelog
2022-05-02 13:38:56 -04:00
JasonRoberts-smile 1114c72f03
Further expunge refactoring (#3550)
* convert expunge public interfaces to use ResourcePersistentId instead of Long

* changelog

* version bump
2022-04-21 09:51:19 -04:00
longma1 01d6e15f90
2579 partitioned support for mdm (#3485)
* committed changes discussed at tasking

* Changed the subscription message to store the request partition

* modified mdm subscription loader to create a cross partition subscription in the default partition if multitenancy is enabled

* Fix errors when running Multitenant IT for mdm

* Added partition id to mdm golden resource search and creating/updating golden resource

* added partition id column to the mdm link table

* Added partition id of the source resource to partition id column of the mdm link table

* broken build

* working -links, missing hapi tests

* Fixed the mdm link db table due to feedback

* added IT for create link mdm operation, also added changes to pass the IT

* Fixed test for create mdm links, added test for update mdm link on partitioned server

* changed mdmLink search from searchbyExample to criteriaBuilder, added partitionHelperSvc to determine partition

* added test for merge golden resource and search golden resource on partitioned server

* added unit test for not duplicate on partitioned and non-partitioned server

* added criteriabuilder search and tests

* code cleanup

* added partition support to -merge-golden-resources operation

* test cleanup

* fixed  operations with multitenancy

* added test for mdm clear operation

* added waiting for batch job to finish in mdm clear operation test

* added query to dao to support partitioned server for mdm clear operation

* resolve mistake from merge conflict

* added mdm-submit operations, also fixed paging for -query

* added partitionIds constant

* fixed build issue where there was no Code.msg in mdm partition exceptions

* code review fixes

* fix broken test, also fix a bug where query link could not query by linkSource properly

* fix test stubbing

* bump hapi version to 6.0.0-PRE10-SNAPSHOT

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
2022-04-06 15:48:30 -04:00
Tadgh 5b413a9135
Package move-around (#3488)
* moved classes

* Bump version

* licensing

* Repackage, fix messages

* reject empty streams

* Remove dead import

* Version bump

* Fix up test to actually return a data stream

* Move null binary svc to storage
2022-03-26 22:43:41 +00:00
James Agnew f82534ca51
Convert Reindex into Batch2 Job (#3458)
* Start working on reindex job

* Updates

* Ongoing work

* Ongoing work

* Work on reindex

* Reindex work

* Add logging

* Fix typo

* Test fix

* Test fix

* Address fixme

* Liocense header

* Resolve fixme

* Add logging

* Update logs

* Address review comments

* License header

* Test fixes

* Test fix

* Test fixes

* Test fix

* Version bump

* Version bump
2022-03-19 16:07:58 -04:00
michaelabuckley 03b6c589c8
New indexing for number and quantity search parameters (#3464)
* New Add Index forced-INCLUDE columns, and Online option

* Remove forced-include.  Simpler just to add all columns to index.

* New date index definitions.

* Oops.  Remove test for deleted feature.

* Cleanup

* Drop IDX_SP_DATE_UPDATED

* Add online option to drop index

* Update annotations to match and redefine FK

* Push sp->resource FK down to control name

* remove dead end

* Cleanup date tests

* New token query test cases for sql extraction

* New token search param indexing.

* Continue to allow the legacy hibernate names while we update the indexing.

* Fix jpa annotations with overloaded field

* Decide on token sorting - we do it.

* review comments

* fixme

* disable dead data migration too

* review fixups

* Test cases for numeric search

* add ne and ap

* Test for Numeric SP

* start

* finish number

* new indexing for quantity

* Move config

* cleanup
2022-03-11 16:23:06 +00:00
michaelabuckley 80d1a5a6f8
Mb new token search index (#3386)
* New token search param indexing.
* New token query test cases for sql extraction
* Retire JPA fk ugly nams
2022-03-11 00:51:53 +00:00
Ken Stevens dfd99c5471
improve search parameter registry logging (#3452)
* Startup log cleanup.  Use cached fhir contexts.

* improve logging

* improve logging

* add full comparator

* wrap resource search param map

* rename wrapped map methods

* add fixmes

* further searchparam encapsulation

* fix regression found by test

* improve logging

* pre-review cleanup

* pre-review cleanup

* bump hapi version

* license

* since I'm bumping hapi-fhir now, improve the api dependency

* licenses
2022-03-09 19:57:38 -05:00
James Agnew 2cba62b4e8
Enable search narrowing on large ValueSets (#3405)
* Enable search narrowing on large ValueSets

* ValueSet improvements

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Add test

* Work on narrowing interceptor

* Work on narrowing

* License headers

* Refactor code narrowing

* Add docs

* Version bump

* Test fix

* Test fixes

* Build fix

* Fixes

* Version bump

* Test fix

* License header updates

* Build fix

* Test fixes

* Test fix

* Test fix

* Docs fix

* Test fix

* Test fix

* Resolve fixme

* Bump

* Force a CI build

* Make CI happen again
2022-03-01 09:16:31 -05:00
James Agnew b7d1ae217d
New batch framework / Bulk Import (#3387)
* Work on new batch framework

* Work on new batch framework

* Work on new batch framework

* Work on batch

* Compile working

* Work on bulk import

* Adjust import

* Work

* Work on batch

* Bump version

* WOrk on new batch processes

* Work on bath

* Bump to PRE4

* Build fixes

* CLeanup

* Small tweak

* Add exception code

* Test fixes

* Test fixes

* Test fix

* Additional synchronization

* Add license headers

* Test fixes

* Test fixes

* Add changelogs

* Address PG

* Test fix

* Test fix

* Test fixes

* Review notes

* Work on tests

* Test fixes

* Test fix

* Work on tests

* Tets fix

* Test fixes

* Test fixes

* Add missing exception codes

* Test fix

* Test fixes

* More test fixing

* License headers

* Test fix

* Add new test logging

* Work on tests

* Test fixes

* Test fix

* Resolve fixme

* Try to avoid test failure

* Add import command

* Work on storage

* Fix error codes

* Fixes

* License header

* Build fix

* Build fixes

* Fix dep
2022-02-27 16:04:49 -05:00
jmarchionatto 53c5a9bbeb
Add FK index to avoid timeout when deleting large ValueSet (#3416)
* Add FK index to avoid timeout when deleting large ValueSet

* Add exception to allow same name for index and FK

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-02-23 13:54:30 -05:00
Tadgh 3561672fe7
Rel 5 7 mergeback (#3401)
* fix handling of common search parameters

* Revert "fix handling of common search parameters"

This reverts commit 89c45eebdc.

* Fix implementation, add test (#3378)

* Fix implementation, add test

* Tighten test

* Rip out dead modules

* Add changelog

* Jr 20220210 handle common search params in contained searches (#3377)

* fix handling of common search parameters

* add support for reference search parameters with multiple paths

* Issue 3357

* Version bump

* Fixed null pointer exception for re-loading subscription on cdr restart and when there's no partition id in the request, and added tests

* added changelogs for this fix

* Fix broken changelog file

* Can't specify specific resource type permissions for bulk export (#3376)

* deny user from exporting without perms

* add unit tests

* add changelog

Co-authored-by: olivia-you <olivia.you@smilecdr.com>

* Make migration donothing as it was added in error

* Remove ehcache

* Add version.yaml:

* Add to sources/javadocs for dist

* Fix typo

* fix up pom

* wip test removing checkstyle plugin from deployable pom

* remove test pom changes, instead just dont deploy to sonatype

* typo

* Remove ehcache

Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>
Co-authored-by: olivia-you <olivia.you@smilecdr.com>
2022-02-18 19:22:44 +00:00
Ken Stevens 631391819b
remove hapi jpa spring config hierarchy (#3390)
* initial refactor to remove hapi jpa spring config hierarchy

* fix poms

* fix batch config

* dependency rearrangement arising from further testing

* fix searchDao bean name

* fixed dstu2 tests

* consolidate validation config

* dstu2 validation regression (man dstu2 support is expensive)

* fix test (with WIP comment)

* restore poms

* fix demo projects

* fix build

* bumping version

* rename generated config class
2022-02-16 20:44:03 -05:00
michaelabuckley 1c82d0933c
Mb new date search index (#3368)
* New date index definitions.
* Dropped unused indexes.
* Add online option to drop index and add index
* Push sp->resource FK down to control name
* Update annotations to match and redefine FK
* Continue to allow the legacy hibernate names while we update the indexing.
2022-02-16 22:12:20 +00:00
James Agnew 0f2fc7a882
Allow search narrowing and auth interceptor by `token:in` (#3360)
* Optmize valueset expansion

* Working

* Version bump

* Add documentation

* Add test

* Checkstyle message cleanup

* Add reverse rule

* Test ficx

* Test fix

* Test fix

* Test fixes

* Test fixes

* Test fix

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/search_narrowing_interceptor.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/search_narrowing_interceptor.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Test fixes

* Fix conflict

* Add setter

* Test fixes

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-02-09 14:27:14 -05:00
Tadgh 6982c4d680
bump version to 6.0.0-PRE1-SNAPSHOT (#3353)
* bump version

* Add new version enum

* Bumped to 6.0.0
2022-02-02 12:11:31 -08:00
Johnson Lu c42ecc5c08
Add error codes to all exceptions (#3278)
* Initial design of adding exception codes to exceptions

* Initial design of adding exception codes to exceptions

* change int module name to string

* add comment

* shorten message

* shorten message

* added checkstyle

* developing checker

* completed checkstyle plugin and unit test for it

* fix NPE

* fix checker so it ignores throws on exceptions that aren't new

* change parameter order in InvalidResponseException

* accept Msg.code() in any parameter since some exceptions like MessageException have the message string as the second parameter

* exclude generated files

* BIG COMMIT added error codes to all exception throws

* example test fix

* Modified tests such that the assert statements would better match the message responses.

* Modified additional tests such that the assert statements would better match the message responses.

* fix test

* merge master

* merge master

* Fourth commit of modified tests such that the assert statements would better match the message response

* Changes made to modify assert statements such that they match the new errors which now use error codes. Also changed ResourceNotFoundException to include error codes in its messages.

* Modified assert messages to better match new error codes

* Time Zone changes DRAFT. Certain tests pass in intellij and fail during mvn install

* Modified assert messages to better match new error codes

* Added comments for time zone fixes. Previous commit comment on time zone fixes was incorrect. By invaliding caches, all tests on both intellij and mvn install passed.

* Reverted changes on azure-pipelines.yml

* updated hapi-fhir version for checkstyle module

* changes to azure-pipelines.yml

* changes to azure-pipelines.yml

* change to pom.xml to include hapi-fhir-checkstyle in ALLMODULES

* changes to azure-piplines (these changes are to be reverted after testing)

* reverted changes to azure-piplines

* reverted change where hapi-fhir-checkstyle was added to pom.xml (hapi-fhir)

* Attempted fix on checkstyle build error

* modified azure-piplines.yml to change maven repository for checkstyle build

* Attempted fix on checkstyle build error

* checkstyle config location change for hapi-depoloyable-pom. (will be reverted if it does not work)

* attempted dependency add for hapi-depoloyable-pom. (will be reverted if it does not work)

* reverted changes for hapi-deployable-pom

* add javadoc

* update last code

* Attempted fix on checkstyle build error

* Finished adding in missing error codes to pass Checkstyle checks

* Test pipelining

* Fixing errors and bumping version (as suggested by Gary)

* Merge

* Fixing Checkstyle error

* Attempted fix for date error

* Add error code onto assert statement

* Add in a couple more error codes

* Fixing assert statements

* Bump to static version for dependency

* Adding error code to message

* Adding error codes to exceptions

* Adding error codes to exceptions

* Change to work with cdr branch

* Update to faulty error message

* Merge changes

* allow checkstyle pom to deploy

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Simon Zuccherato <simon.zuccherato@smilecdr.com>
Co-authored-by: Predap <simon.zuccherato@gmail.com>
Co-authored-by: Predap <46201929+Predap@users.noreply.github.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-01-31 13:10:15 -08:00
jamesagnew aafc42fdd8 License header updates 2022-01-04 13:37:10 -05:00
James Agnew d610d33ac3
Dependency bumps (#3264)
* Bump a number of versions

* Fixes

* Work on version bump

* Bump

* Test fix

* More version bumps

* Build fixes

* Test fixes

* Fixes

* Fixes

* Test fix

* Bump jacoco

* Test fix

* Test fix

* Test fix

* Build fix

* Build fix

* Remove versioned H2

* Build fix

* Avoid memory issue in validation tests

* JDK bump

* Restore compile with errorprone

* Force coloured output

* Force jansi

* Try again to force jansi

* Fix for hardcoded date that just passed

* Fixes to JPA

* Improve changelog

* Work on failing test

* Test fix

* Compile fix

* One more version bump

* Test fixes
2022-01-04 13:21:14 -05:00
Tadgh 50a3005c23
Fix flag name (#3247)
* Fix flag name

* Bump hapi version
2021-12-14 19:20:03 +00:00
longma1 a56603daa2
2480 partition aware subscription (#3218)
* some fixmes to start

* Added some FIXMEs

* added RequestPartitionId to reosurceDelivryMessage and ResourceModifiedMessage

* ResourceDeliveryMessage and ResourceModifiedMessage tests

* fixed issue with test missing partitionHelperSvc mocked bean

* Added tests and implemented SubscriptionMatchingSubscriber for partition aware subscription

* modified implementation of partitionId in CanonicalSubscription

* Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)

* Revert "Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)"

This reverts commit fe40fb9733.

* Got added partitionId to subscriptions, added changes to make SubscriptionMatchingSubscriberTest work

* added SubscriptionTriggering test, also added partition support to subscriptionLoader

* Changed implementation for storing partition id of subscriptions from messages, refactored tests to new implementation

* added all subscription systemRequestDetails with all partition to subscription reader

* refactored a generic system request details with default all partition request

* Added test for dao subscriptions, fixes to get the test working

* added partition support for latest version delivery

* added doc changes and changelog for multitenancy subscription

* cleanup and added partitioned subscription manually trigger test

* fixed mocked subscriptionDao

* added package-info for subscription module

* some code review changes

* removed AllPartitionSystemRequestDetail, added new text for multitenant subscription

* renamed method for code review

* version bump to 5.7.0PRE7

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
2021-12-13 08:45:30 -07:00
VK-SMILECDR bd89cf022b
Improve mail service (#3230)
* move mail config to mail service constructor

* update subscription email sender

* added license text

* [2669] bump HAPI version to 5.7.0-PRE6-SNAPSHOT

* add changelog
2021-12-09 17:05:42 -05:00
Ken Stevens 9753b66610
2564 contained resource validation flag (#3225)
* First cut at multithreading bundle validation

* javadoc

* change test to a red-green test

* add daoconfig

* move bundle concurrency config from validationsupport to request validator

* Moved concurrent details from ValidationOptions to FhirValidator

* Remove bundleValidationThreadCount (it's set by creating an appropriate executor)

* validate threadpool prefixes

* FIXME cleanup

* reassign FIXMEs to JB

* only create the threadpool if it will be used

* Add the MdcTaskDectorator to all threadpools

* Add bundle path to concurrently validated bundle resource validation messages.

* Add test to check bundle entry path being added to validation messages

* Threaded validation of contained resources initial checkin

* FHIR validator refactored to maintain correct bundle path and async validation task

* Refactored to use validation complete invoke callback to support raw string resource or resource object

* switch from XML to JSON

* # Conflicts:
#	hapi-fhir-base/src/main/java/ca/uhn/fhir/validation/FhirValidator.java
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Updated to use new HAPI-FHIR version

* merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation

* Updated to use new HAPI-FHIR version

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit 910b7f676b.

* Revert "merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation"

This reverts commit c7d60fc3e6.

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit aa8a7af455.

# Conflicts:
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Fix Test dependencies after reverting changes.

* merge 'validate_contains_change' to 2564-contained-resource-validation-flag

* Disable threadpool prefix name validate check.

* merge master

* fix thread regexp

* pre-review cleanup

* change log

* fix test

* revert to XML as default validation serialization

* fix test

* fix test

Co-authored-by: Jaison B <jaisonb@gmail.com>
2021-12-08 15:18:54 -05:00
jamesagnew 90d05a9ed4 Fix accidental commit 2021-12-07 14:55:16 -05:00
James Agnew d072357c2a
Move graphql provider (#3214)
* Move graphql provider

* Version bump
2021-12-05 17:22:20 -05:00
jmarchionatto 175958de5e
Issue 3108 terminology large codesystem deletion takes days to complete (#3160)
* Add indexes to foreign keys

* Add new version enum

* Add TermCodeSystem and TermCodeSystemVersion deletion job artifacts

* Add Propagation.NOT_SUPPORTED to allow starting jobs from transactional code

* New jobs artifacts

* Improve comments

* Restructure job configurations to avoid bean name collisions

* Restructure job configurations to avoid bean name collisions

* Use new offline deletion by job

* Test using multiple versions

* Retrieve only Pids

* Revert to master

* Maintain deferred functions synchronization as similar as possible to the way it was

* Bypass validations to allow calling an index same as a foreign key to make SchemaMigrationTest happy as H2 adds FK indexes automatically

* Adjust tests

* Add copyrights

* Add test

* Revert to delete CodeSystemVersions using deferred ITermDeferredStorageSvc

* Allow to wait for jobs tentatively

* Organize imports

* Fix tests

* Increase timeout to cover for possible additional heavy DB activity

* Add indexes to two remaining non-indexed foreign key fields to avoid much longer transactions which produce locks on heavy DB load

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_7_0/3108-terminology-large-code-system-deletion-performance.yaml

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Remove opening stars from logging messages

* Relax index name checking logic to allow for indexes on FK columns to be called same as the FKs, to make SchemaMigrationTest happy even when H2 adds indexes to FKs automatically.

* Use static array to enumerate platforms which doesn't add indexes to foreign keys automatically.

* Cleanup also job executions when performing test artifact cleanup

* Adjust to property name change

* Add ABANDONED status to the list of waited job statuses to avoid test failures due to wait timeouts

* Add also UNKNOWN status to the list of waited job statuses to avoid test failures due to wait timeouts

* Filter job executions to be stopped

* Set trace to debug test problem

* Add STOPPED status to the list of waited job statuses to avoid test failures due to wait timeouts

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2021-11-18 09:49:16 -05:00
TipzCM 25c025cdee
3153 update unknowncodesystemwarningvallidationsupport to have configurable severities (#3155)
* 3153 updating the unknowncodesystemwarningvalidationsupport class

* 3153 updating version for smile

* 3153 update broken tests

* 3153 adding log

* 3153 cleanup

* 3153 cleanup

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-11-10 16:06:09 -05:00
TipzCM 59bf8b836f
Issue 3120 support adding retry extension (#3137)
* issue-1134 initial work on adding retry handler

* issue-1134 subdscription creation fixes

* issue-1134 update the test

* issue-1134 retry policy off canonicalsubscription

* issue 1134 base subscription changes

* add failing test

* issue 1134 retry configs on base parameters

* issue 1134 passing forward params

* issue-1134 added more tests for making sure subscriptions create their channels correctly

* issue-1134 updates to get the retry to channels

* issue-1134 updating channel factory

* issue-1134 remove the dlq since it's not defineable

* issue-3120 changelog added

* issue-3120 cleaning up

* issue-3120 test fix

* issue-3120 review fixes

* issue-3120 fixed bad master merge

* issue-3120 updates for new release

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-11-03 18:21:10 -04:00
Tadgh a10c58e839
Bump version, add version to versionenum (#3135) 2021-11-02 09:40:56 -04:00
Tadgh 7c63441663
Revert "Issue 1134 retry mechanic on subscriptions (#3121)" (#3128)
This reverts commit 9b23e9e825.
2021-10-30 19:46:42 +00:00
TipzCM 9b23e9e825
Issue 1134 retry mechanic on subscriptions (#3121)
* issue-1134 initial work on adding retry handler

* issue-1134 subdscription creation fixes

* issue-1134 update the test

* issue-1134 retry policy off canonicalsubscription

* issue 1134 base subscription changes

* add failing test

* issue 1134 retry configs on base parameters

* issue 1134 passing forward params

* issue-1134 added more tests for making sure subscriptions create their channels correctly

* issue-1134 updates to get the retry to channels

* issue-1134 updating channel factory

* issue-1134 remove the dlq since it's not defineable

* issue-3120 changelog added

* issue-3120 cleaning up

* issue-3120 test fix

* issue-3120 review fixes

* issue-3120 fixed bad master merge

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-10-29 11:21:43 -04:00
Tadgh b427135b04
Patient/$everything _id parameter (#3077)
* Add tests. Add implementation in R4. Add changelog

* Wip partial fixup of other STU versions

* Updated usages of new patient type everything api

* Partial commit of incomplete documentation

* Remove docs

* Cut to token instead of string

* Bump hapi version due to breaking api changes
2021-10-17 14:34:33 +00:00
James Agnew 310a4cdb58
Migrate CLOB columns to LOB (SMILE-3072) (#3016)
* Migration finished

* Work on LOBS

* Add to ClobMigrated annotation

* Deal with lobs

* Work on new approach

* HAPI FHIR version bump

* Add changelog

* Add license header

* Fix intermittent

* CLeanup

* Cleanup
2021-10-11 16:47:37 -04:00
Ken Stevens 53c7476dae
hapi storage (#3024)
* first pass moving core storage classes to storage-api

* move term service apis

* nearly done

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* create new SearchConstants class to store platform independent search constants

* move a couple of subscription services

* move transaction processor adapter to storage

* version bump

* move searchparam

* fix test

* fix compile includes

* fix text

* fix docs
2021-09-23 22:04:46 -04:00
Jason Roberts fa96c5b8be Merge branch 'master' into jr-20210921-mssql-data-migration 2021-09-21 12:22:56 -04:00
Ken Stevens 060791aeb4
hapi-fhir-storage-api (#3006)
* rename hapi-fhir-jpaserver-migrate to hapi-fhir-sql-migrate and move hapi-tasks into jpaserver-persistence

* rename hapi-fhir-jpaserver-api to hapi-fhir-storage-api

* move bulk import apis

* create hapi-fhir-jpa

* create hapi-fhir-jpa

* move CircularQueueCaptureQueriesListener to japi-fhir-jpa

* move mdm logs to storage-api

* move gziputil to storage-api

* move quartz scheduling to hapi-fhir-jpa

* move default subscription channel factory to storage-api

* consolidated batch constants

* correct accidental Logs change

* remove dependency of cdr-api on cdr-persistence

* fix javadoc

* pom descriptions

* review feedback
2021-09-21 15:36:08 +00:00