Commit Graph

158 Commits

Author SHA1 Message Date
Ken Stevens 5dda16d467
Reduce postconstruct use (#4445) 2023-01-21 00:37:58 -05:00
Ken Stevens bf53a08b83
Batch2 fastracking configurable (#4453)
* licenses

* hapi-fhir side done

* cdr side

* add assert

* add test

* bump hapi version

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-20 19:03:13 -05:00
Qingyixia 663834ddfe
Added resourceType as a parameter to $mdm-duplicate-golden-resources operation (#4434)
* Solution + changelog

* Addressing suggestions

* Minor changes

* Version Bump
2023-01-19 08:43:44 -05:00
TipzCM 23b593aa17
breaking up package installer and package config (#4319)
* breaking up package installer and package config

* breaking out parsing of resources

* wrong nullable

* added an integration test

* adding a changelog

* added a few minor tests

* review fixes

* unit test fixes

* fixing code checkstyle

* moving fetching local package data to package loader svc

* updating the method name

* splitting the fhirversion parsing

* updating table creation task for pretty printing and use in RTE

* adding foreign keys

* Adding mroe info

* minor prettiness changes

* fixing fk generation

* minor changes

* update version

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2023-01-17 01:58:14 +00:00
Ken Stevens 04c28b6c08
Schedule jobs via IJobScheduler instead of in @PostConstruct (#4415)
* parameterizing BinaryStorageInterceptor

* parameterizing BinaryStorageInterceptor

* changed jobs scheduling from @PostConstruct to coordinated by HapiLifecycleService events

* fix test

* remove events for now

* fix test

* simplify design

* FIXME

* changelog

* fix compile issue

* review feedback.  partially implemented

* review feedback

* review feedback

* bump hapi version to 6.3.8-SNAPSHOT

* fix test

* mirror contextclosedevent handlers with contextrefreshedevent

* version bump

* licenses

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-14 23:51:04 -05:00
michaelabuckley 21b1820294
Demote PHI logging to debug or lower (#4430)
Ensure resource bodies are only logged at debug or lower.

* Drop possible-phi logs to debug.
* Demote request/response log bodies to debug since they may contain PHI.
* Demote job-param logging to debug.
* Add startup hook to App for test injection.
* Change terminology cli logging to avoid logging whole resource.
2023-01-13 15:34:29 +00:00
Justin McKelvy 8c48a6fb91
Clinical reasoning module (#4347)
* Add new Clinical Reasoning module

* Add config to get test up and running

* Removed FhirClient from tests

* First passing test

* Remove unused utilities

* Build updates

* exclude xxp3 and substitute with unbundled version

* Refactor to use provider factory and loader pattern

* First passing dstu3 test

* CR r4 test config and immunization tests

* Add base test classes

* Exclude transitive dependency xmlpull

* More dependency cleanup

* Refactor to remove JPA dependencies, rename module to hapi-fhir-storage-cr

* Fixes for duplicate xpp3 classes

* Remove more references to jpa

* Add mock FHIR server for terminology endpoint tests

* fix cql file for making providerr4 tests pass

* fix pom

* fix generic

* Adding r4 test cases for measure operations.

* Fix endpoint tests

* javadoc comment, temporarily comment out test results for practitioner sync

* javadoc typo fix

* adding java docs for cr module WIP

* Updates to clinical reasoning documentation

* More documentation updates

* docs now compile

* fix test

* DqmR4IT now successfully starts up, but the test does not pass

* Fix additionalData test

* Make measure service bean overridable

* adding clinical reasoning javadoc information

* Rename interfaces to start with I

* More styleguide updates

* Reformat all code in the storage-cr module

* More coding convention updates

* Remove bunches of unused code, reogranize so that fhir version in package name is consistent

* Clean up some static array initializations

* add practitioner patient list

* refactor product-line constant

* test dependency update

* add change log for cr module

* Add hapi-validation to the pom

* Exclude dependencies that are causing duplicates

* More dependencies and exclusions

* Make some base MeasureService components protected

* Update to latest cql compiler release

* fix measure service for null subject & practitioner and open up practitioner test

* Rev to release versions

* Add remaining Dstu3 tests

* update hedis bundle from policyHolder to beneficiary

* update snapshot

* Delete old CQL module

* Post-master-merge recovery

* Fix Java 17 feature usage issue

* update changelog with more information about new functionality

* Remove support chain wrapper usage

* Add javadocs to IResourceLoader

* added the property versioning for antlr jar.

* addressing code review comments.

* addressing code review comments.

* Fix typos in changelog

* Remove azure pipeline for cql module, add it for cr module

* Address exception comments, add Msg.code calls to exceptions

* Fix self-assignment bug

* Add storage-cr module to bom

* addressing coding comments.

* Addressed the variable naming convention to camelCase.

* Snapshot version bump

Co-authored-by: Brenin Rhodes <brenin@alphora.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: Chalma Maadaadi <chalma@alphora.com>
Co-authored-by: mdnazmulkarim <nazmul.karim@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-01-10 19:49:57 +00:00
karneet1212 cfa6c13262
Add resource type as a parameter for $mdm-query-links (#4372)
* Added Solution + test

* Removed old methods

* fixed issues

* Added refactoring

* Deprecated methods on the interfaces

* Remove todo

* Documentation

* Changelog

* Changelog

* Changelog

* Changes made based on comments

* Changes made based on comments

* Clean up

* hapi bumping to version 6.3.6-SNAPSHOT

* hapi bumping to version 6.3.6-SNAPSHOT
2022-12-22 09:54:32 -05:00
James Agnew 0d5f8da93f
API improvements for MegaScale (#4112)
* Start work on sharded DB

* Work on migrating

* Work

* Ongoing work

* Ongoing work

* Work

* Work

* Ongoing work

* Work

* Work

* Test fix

* Test fix

* Work

* Compile fix

* Test cleanup

* Test fix

* Resolve fixmes

* Resolve fixme

* Resolve compile errors

* Compile fix

* Fixes

* Work

* Add note for later

* Test fix

* Test fixes

* Build fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Fixes

* Fixes

* Remove dead code in pom

* More test cleanup

* Docs fix

* Changes

* Fix

* Fixes

* Resolve merge conflicts

* Build update

* Merge build fixes

* Version bump

* Address review comments

* Review comments

* Review comments

* Test fixes

* Test fix

* Cleanup
2022-12-20 12:05:15 -05:00
Nathan Doef 7b86eda8a9
Refactor Common SearchParameter Functionality (#4321)
* refactor checkpoint

* clean up

* add method to find runtime search param by id

* new search param cache tests

* extract SearchParameter validation functionality

* create common interface for combo searchparameters

* Remove ResourceTable from extractor interface

* clean up

* move SearchParameterDaoValidator

* resolve code review comments

* remove reflection

* change log

* bump version to 6.3.4-SNAPSHOT

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-12-15 06:46:45 -05:00
kateryna-mironova 017cc4e29c
4244 - Deprecate usages of ResourcePersitentId in jpa (#4245)
* init rev

* id to jpa and mongo id

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* fixed conflicts, pulled from master

* fixed code for passing tests successfully

* changed PersistentId to Jpa in MdmSearchInterceptor

* added the todo comments with questions

* added one more question

* renamed TODO questions

* first pass at converting to generic ResourcePersistentId

* fix test

* fix test

* fix test

* make ResourcePersistentId abstract

* Generic type MdmLink

* Final TODOs

* Final TODOs

* cleanup

* fix test

* fix id to string

* fix cyclic package dep

* add <?> to Base so we can extract interface

* Revert "add <?> to Base so we can extract interface"

This reverts commit a806f3d2bc.

* switch to interface

* javadoc

* replace base with interface

* fix test

* add generic template to mdmdao

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary MongoPid casts

* fix test

* fix regression

* fix compile issues arising from merging master

* fix beans

* bump version to 6.3.3-SNAPSHOT

* change logs

* review feedback

* review feedback

* fixme

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-12-09 09:30:51 -05:00
James Agnew 50ca94eded
Improve JPA response messages (#4293)
* Improve JPA response messages

* FIxes

* Test fixes

* Test fixes

* Ongoing testing

* Work on messages

* Work on messages

* Add valueset

* Add response code enum

* Version bump

* Undo bump

* Improve changelog

* Test fixes

* Add javadocs

* Version bump HAPI

* Test fixes

* Test fix

* Test fixes

* Test fixes

* Account for review changes

* Test fix

* Docs fix

* Work on API

* Improve SqlQuery api
2022-11-28 07:52:12 -05:00
James Agnew c3697a5f87
JPA Cleanup (#4278)
* JPA cleanup

* Test fixes

* Cleanup

* Resolve fixme

* Fix bad changelog entry

* A bit of documentation reworking

* Version bump

* POM bump

* Avoid timeout

* Build fix

* HTMLUnit bump
2022-11-24 14:32:56 -05:00
Tadgh b1e3468dcd
Rel 6 2 1 mergeback (#4288)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix up checkstyle

* log to console

* Remove double import

* Version bump

* licenses

* Fix up version

* fix build

* wip

* wip

* Fix up search expansion to use new behaviour of idhelper service

* Fix up MDM Search Expansion

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-11-20 08:14:28 -08:00
Ken Stevens d8d8c5fa44
Consolidate all System Properties into a single class (#4252)
* started converting System.setProperty to HapiSystem

* added a couple more.  renamed class

* all the setProperty calls done

* finished System.getProperty()

* Split HapiSystemProperties into HapiSystemProperties and HapiTestSystemProperties

* Javadoc

* Javadoc

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-11-14 18:51:03 +00:00
jmarchionatto 2df09c0cf3
Issue 4214 bulk export group observations missed when search parameter indexing enabled (#4241)
* Add failing test

* Add failing test for hsearch reference without type

* Modify cache to store forcedid type

* Disable failing test until finding out if it is a bug and strategy for fix in that case

* Remove fixmes and commented lines

* Update test. Add changelog.

* Added test that passes on this branch.

* Added Encounters to test case.

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-11-11 20:08:41 +00:00
James Agnew 9a45576793
Cache service loaders (#4196)
* Extracting a Cache interface and building service loaders for Caffeine and Guava

* Fix estimated size interface

* Use LoadingCache when needed.

* Removing Caffeine from dependency lists.

* Adding hapi-fhir-cache-caffeine as a test dependency

* Putting caching solutions in a single module.

* Fixing the spacing

* Standardizing the use of TimeUnits

* Making a new module to simplify the switch of the cache library in tests.

* Making sure the Guava design matches the behavior of Caffeine.

* Making sure the Cache structure also does not throw InvalidCacheLoading exception to match the LoadingCache.

* Renaming module names for the caching group.

* Better error handing that informs devs what to do.

* Improving documentation

* Typo

* Matching error message design with Caffeine.

* Matching the behavior of Caffeine with Guava

* Final adjustments for the test dependencies on the cache modules.

* Fixing relative pom path.

* Adding caffeine as a testing requirement for the new modules.

* Add changelog and set JPA server to use caffeine cache

* POM fixes

* Build fix

* Buid fix

* Fixes

* Address review comment

* One more cache

* Move changelog to next release

* Update pom versions

* Build fix

* Build fixes

* Build fix

* Try to get build working

* Experiment with failing build

* Rever change

* Fix POM version

* Build fix

* Build fix

* Add Msg.code to new exceptions

Co-authored-by: Vitor Pamplona <vitor@vitorpamplona.com>
2022-11-09 19:47:23 +00:00
James Agnew f50d350f1f
Post release HAPI FHIR cleanup (#4266)
* POST release deferred cleanup

* Fixes

* One more test fix

* Test fixes

* Test fixes

* Test cleanup

* Test fixes

* Fixes

* ValueSet cleanup

* Test fix

* Test fixes

* Fixes

* Test fixes

* Fixed

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixc

* Build fix

* Fix merge artifact

* Build fix

* Work on tests

* Test fixes

* Work

* Fixes

* Changelog fix

* Add changelog

* Test fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Tests

* Bumps

* Fixes

* Add errorprone

* Drop bz2 bins

* POM fix

* Build fix

* Update

* Test fix

* Fix tests

* Add changelog

* Test fix

* Avoid intermittent

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-09 13:28:00 -05:00
James Agnew b3ebbe7933
ValueSet cleanup (#4227)
* POST release deferred cleanup

* Fixes

* One more test fix

* Test fixes

* Test fixes

* Test cleanup

* Test fixes

* Fixes

* ValueSet cleanup

* Test fix

* Test fixes

* Fixes

* Test fixes

* Fixed

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixc

* Build fix

* Fix merge artifact

* Build fix

* Work on tests

* Test fixes

* Work

* Fixes

* Changelog fix

* Add changelog

* Test fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Tests

* Bumps

* Fixes

* Add errorprone

* Drop bz2 bins

* POM fix

* Build fix

* Update

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-08 22:18:36 -05:00
Tadgh d3367cfede
Rel 6 2 mergeback (#4257)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix

* Bump HAPI version

* Remove jetbrains

* Fix broken test

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-11-07 23:57:57 +00:00
jmarchionatto 6cf5989d6c
Issue 4106 process map to properties when uploading loinc terminology (#4231)
* Add loinc upload handler to process MAP_TO properties.

* Move test to test folder

* Fix mapto file location property name

* Allow override as full loinc upload tests take (much) longer than defined timeout

* Move handler later to make sure all concepts were added to map

* Update test to check all project enhancements

* Restore project unrelated changes

* Rename base test class properly and move to right package

* Condition MapTo.csv file processing to file presence for backwards as file is optional

* Document and annotate test-only usage for disabling deferred tasks timeout

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-11-03 08:48:02 -04:00
samuelwlee2 1b55f49a60
4153 extend $member match operation to store consent resource if matched (#4165)
* added senstive and regular filter

* add UUID to Consent

* blah

* fix merge with master

* fixing merge

* blah

* fhir tests

* use constructor injection

* put all tests together

* blah

* blah

* finalize

* add consent update patient and performer reference

* fixes after code review

* blah

* refactor to save consent once

* blah

* fix performer and add validation

* code clean up

* add changelog

* add hrex member match flag

* fix PatientMemberMatchOperationR4Test

* adding code fixes after code review

* move changelog file

* bump version

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2022-10-31 13:33:54 -04:00
Tadgh 31a193fbd3
Bump to 6.3.0 (#4211) 2022-10-28 02:40:59 +00:00
James Agnew 7a39f8cbaf
Make reference extractor more reusable (#4205)
* Refactor

* Make reference extractor more reusable

* Test fix

* Test fix

* Headers

* Small refactor
2022-10-27 21:31:39 -04:00
michaelabuckley 929767a09b
Change to a minimal first pre-fetch to avoid latency on the async path. (#4174)
* Change to a minimal first pre-fetch to avoid latency on the async path.

The "async" path is now synchronous, and was causing latency for first page by loading 500 results.
We avoid pre-fetch on the first page, and only pre-fetch after users fetch the second page.

* Fix a nasty bug continuing a pre-fetched search.

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_2_0/3664-shorten-first-prefetch.yaml

* Patch up tests that assumed 500 pre-fetch first step
2022-10-21 20:45:30 +00:00
jmarchionatto 2830792c4b
Issue 3992 elastic search mapper parsing exception when uploading loinc terminology (#4141)
* Revert TermConceptProperty indexing to not-nested type to avoid hitting elastic default maximums and
fix regexp issue which was main reason to change indexing way in the first place.

* Add changelog

* Document that terminology reindexing is required.

* Implement revision suggestions. Fix file name typo.

* Adjust and enable test created to reproduce bug

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-10-21 16:19:45 -04:00
James Agnew a71b8fad56
Add R4B Support (#4150)
* Add r4b stuff

* Work on R4B

* Core version bump

* Revert some changes

* Add R4B

* Docs

* Test fixes

* Work on R4B

* Tests passgin

* Add config

* Work on merging

* Work

* Work

* Work

* Test fix

* Fixes

* Test fixes

* Test fix

* Work

* Work

* Fixes

* Test fixes

* Fixes

* Test fix

* Test fix

* Test fix

* Version bump

* Test fix

* Test fix

* ompile fixes

* Fix

* Resolve build warnings

* Updates

* Test fixes

* Fix docs
2022-10-20 13:57:06 -07:00
KGJ-software 69d55b7184
3052 Changed IBinaryStorageSvc MaximumBinarySize from int to long. (#4123)
* 3052 Changed IBinaryStorageSvc MaximumBinarySize from int to long.

* 3052 Added changelog

* 3052 Bumped version

* 3052 Remove mysterious +1 on maximum binary size causing overflow.

* Bumping version to 6.2.0-PRE16-SNAPSHOT

* Adding + 1L as original implementation.

* Getting all tests to pass dealing with + 1L.

* Getting all tests to pass dealing with + 1L.

* Adding migration definition and removing unnecessary comment.

* Preventing register overflow when setting maxBinaryStorage.

* Mering relevant test.

* Fixing typo in migration script

Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-10-19 06:09:02 -04:00
James Agnew 422ef87a5d
Standardize on one Transactional annotation (#4130)
* Standardize on one Transactional annotation

* Add changelog

* Disable checkstyle rule

* Fix up transactions

* Test fixes

* Test cleanup

* License headers

* One more dep bump

* Resolve commit issue

* Test fixes

* Test fix

* Dep bump

* Fix

* Fix test

* Version bump

* Test fixes
2022-10-13 16:26:11 -04:00
Luke deGruchy 5831cbedf5
Concurrent cascade delete operations leading to confusing errors (#4126)
* Add unit tests and supporting classes from gitlab issue.

* Commit with changes thus far to be reviewed.   Namely, add savepoint capabilities to HapiTransactionService, inject HapiTransactionService into CascadingDeleteInterceptor, add savepoint capabilities to the TransactionDetails, and catch an Exception in BaseHapiFhirResourceDao.delete() then rollback to the checkpoint.

* Fix compile error.

* hapi transaction service test

* hapi transaction service test

* hapi transaction service test

* First draft of RetryTemplate and flush() logic.  Try unit testing with PointcutLatch.

* hapi transaction service test

* Latest changes including moving RetryTemplate, cleaning up unit tests.

* First attempt at correct design.  Fix compile errors.

* Add failed rollback savepoint test.

* fix propogation

* interesting test failure in SafeDeleterTest

* Latest code

* move delete

* test passing

* all tests pass

* yay we're done just cleanup

* First round of cleanup:  Get rid of all exploratory logging, remove unneeded dependencies, move creation of RetryTemplate to SafeDeleter.

* More cleanup.  Add changelog.

* Bump to 6.2.0-PRE13-SNAPSHOT.

* Code review feedback:  Refactor SafeDeleter to a service and bean.  Add comments.

* Code review feedback:  Comments on unit test assertions.

* Code review feedback:  Rename SafeDeleterSvc to ThreadSafeResourceDeleterSvc.  Add javadoc.  Add logger to DelayListener.   Remove useless commented out code.

* Fix unit test failure in CascadingDeleteInterceptorTest.

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-10-11 16:48:01 -04:00
Ken Stevens 40163f73d2
Ks 20220916 remove flyway (#4040)
* removed flyway, started coding persistence

* started wiring up new dao service

* replace jpa with sqlbuilder

* down to 16 failed tests

* remove outOfOrder test.  We no longer support validating order.

* Removed noFlyway mode.  It is no longer a thing.

* merge migrator

* down to 1 failing test

* cleanup

* removed unused noflyway cli arguments

* cleanup

* cleanup

* cleanup

* cleanup

* create MigrationTaskList to replace the List<BaseTask> that gets passed around a lot.  This allows us to add a bunch of behaviour to that as a first-order thing

* fix test

* fix test

* improve migration table detection to use the catalogue

* Msg.code()

* cleanup

* save version in same format as flyway

* cleanup

* add migration result to migration result class

* failed migrations stay in the table and don't really affect anything.

* change log

* docs

* ugh Flyway requires lower-case column names

* integration test fixes

* log

* fix oracle issue

* lowercase tablenames for postgres.  fussy fussy!

* Revert "lowercase tablenames for postgres.  fussy fussy!"

This reverts commit 92c9cb263b.

* revert to capital table names.  surround table name by quotes for Postgres.

* added log message before executing find all query

* changed log message from info to debug

* javadoc

* pom cleanup

* review feedback

* Update hapi-fhir-sql-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/MigrationTaskList.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* review feedback

* review feedback.  Use prepared statement.

* Change installedOn insert from timestamp to date

* Change installedOn column type from timestamp to date

* bump hapi version

* fix poms

* fix poms and cdr compile

* add exclusions to clean up build

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2022-10-03 22:40:09 -04:00
weiping202209 66baa8263d
Fix bug in history search (#4009)
* add failing test

* Fix history search bug and add tests for two more scenarios

* Fix history search bug and add tests for two more scenarios

* - added failing test condition

* Fix history search bug and add tests for two more scenarios

* Fix build test failures

* Fix build test failures

* Add extra verifications for scenarios of _since

* Refactor dao.history interface to add overload method and shall come back to remove old method once cdr is refactored.

* Update with new hapi interface

* Add depreciate comment

* Add depreciate comment

* Version bump to mvn 6.2.0-PRE11-SNAPSHOT

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-10-03 09:00:41 -04:00
Ken Stevens e9a855d3f6
remove spring batch (#4064)
* remove spring batch
it compiles
expect test failures

* fix test transaction manager bean name (wow that was quite the hunt the wumpus!)

* fix test and fix build

* fix test and fix build

* fix test

* remove spring batch schema initialization

* bump hapi version

* fix schema migrator

* bump to PRE-10

* merge master

* change logs

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-30 13:18:48 -04:00
Tadgh b472c4bb54
4083 batch permissions (#4095)
* Move threadpool factory into a bean

* Move threadpool factory into a bean

* change return type

* Add license files

* Version bump

* Fix test
2022-09-29 17:24:05 +00:00
michaelabuckley 96b92153f6
Implement composite search parameters in HibernateSearch for JPA (#3839)
Implement composite search parameters in Lucene/Elasticsearch Hibernate Search indexing

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-09-27 00:20:06 +00:00
michaelabuckley ee1cb4e392
Cleanup and utilities from composite work (#4066)
Utilities and cleanup from composite work #4066
2022-09-24 00:47:25 +00:00
jmarchionatto 27ecba796b
Change hsearch to consider unquoted parameter strings as prefix match (#4045)
* Change hsearch to consider unquoted parameter strings as prefix match

* Adjust test according to spec (code fixed before).

* Add :text handling to StringParam

* Adjust StringParam tokenization to spec.
Enable StringParam tests.
Add changelog.

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_2_0/4034-align-text-query-syntax-with-hapi-string-search.yaml

Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>

* Avoid changing simple query parameters.
Implement suggested tests.

* Consider a few Lucene Simple Query Syntax special cases

* Add search syntax changes to documentation and test to make sure samples are good.

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
2022-09-21 20:09:09 +00:00
jmarchionatto ad57ebf148
Add disabled test reproducing open bug (#3994)
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-09-13 17:56:31 +00:00
Tadgh cb3cc1f434
Iam opensearch auth (#4011)
* Bump HS version, add support for credential provider

* License files

* Use index settings file instead of injecting a template at boot

* Changelog

* Typo

* fix logs

* Add javadoc
2022-09-09 23:24:06 +00:00
jmarchionatto 002a7dc670
Issue 3419 elastic search exception while pre expanding valueset with more than 10 000 concepts (#3991)
* Add test reproducing bug

* Refactor ValueSet expansion hsearch queries to use scroll.

* Use void method for never used return value.

* Execute action when transaction is active

* Update tests query counts as the CodeSystemVersion.pid is always queried now

* Enable test after reproduced bug fix.
Add changelog.

* Implement suggestions

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-09-07 15:52:38 -04:00
jdar8 c647154777
Everything operation with type param code review fixes (#3926)
* check for valid resource type

* add test and refactor test class

* refactor auto get method name

* introduce parameter object to patient everything methods

* introduce parameter object to patient everything methods

* change record to class

* add hapi error code

* refactor

* refactor

* change code num

* refactor getmethodname to util method

* add docs and rename fields to conform to fhir standard

* typo

* version bump

* resolving merge conflicts.

* Bumping version

* Bumping version to 6.2.0-PRE5-SNAPSHOT

* Encapsulating parameters in a PatientEverythingParameters object.

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-09-07 06:44:36 -04:00
Ken Stevens 100b8f9190
batch2 rework (#3964)
* consolidated Batch2Helper
check status in assert before running maintenance
ignore illegal job instance state transition

* Fix mock tests

* rename intermittent to IT

* extra logging

* into to debug

* review feedback

* added more logging

* removed fasttracking.  replaced with support for triggering a maintenance run

* fixed tests

* fixed tests

* add asserts

* licenses

* snakeyaml cve version bump

* data migration

* merge origin/master

* merge origin/master

* fix test (support triggering jobs when scheduling is disabled)

* add counter to abort potential infinite loop after 100000 calls

* compile error

* move multipartition mdm tests out to a separate IT
update mongo job instance creation with new logic

* fix race condition

* back out infinite loop circuit breaker.  it breaks terminology tests that depend on this loop running > 100000 times.

* fix test (triggered job key was missing group)

* revert saveAllDeferred

* Almost finished getting JpaPersistenceR4IT to pass

* testBulkExportDeleteForOnGoingJob still not passing

* testBulkExportDeleteForOnGoingJob still not passing

* Removed method `fetchInstanceAndMarkInProgress`.  It was clobbering other state transitions like CANCELLED.

* undo rename of cancelled to cancelRequested.  Thymeleaf depends on the old name

* ExpandResourcesStep requires a synchronous search

* Tests are passing but man testBulkExportDeleteForOnGoingJob is a flakey test

* fix test

* fix test

* change log

* allow ERRORED -> FAILED state transition

* add a timeout to saveAllDeferred()

* improve error message on timeout message

* fix tests

* added more logging

* added more logging

* fix semaphore issue

* Msg.code

* fix test

* javadoc

* switch all batch loggers to batch troubleshooting log

* add unit test, improve logging

* move TimeoutManager to its own class.  Add tests.

* fix tests

* comment cleanup

* test race condition

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-05 00:04:54 -04:00
Tadgh b9caec1c42
Rel 6 0 mb 3 (#3933)
* adding version.yaml, updating version in pom.xml

* It is possible to write to a resource in a partition the user is not authorized to. (#3397)

* fixed

* remove sout

* add msg.code

* fix failed tests

* fix equal sign

* update msg code

* extract method

* Fix up code numbers

* Clean changelog

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Revert final artifact version

* add graphql test (#3585)

* added graphql birthdate test

* fix variable name

* typo

* 3506 mdm log enhancement (#3543)

* Providing Fixme's to be reworked at a later time.

* Adding // FIXME Anna to assist our Austrian friend.

* Adding logging test as a first step in addressing issue #2822.

* hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

* Update MdmMatchLinkSvc.java

#3506 move "narrowed down" log to different place

* #3506 added scores and tests

#3506 added scores and tests

* #3506 formatting

#3506 formatting

* #3506 create changelog file

#3506 create changelog file

* #3506 fix typo

#3506 fix typo

* #3506 fix part 3, minor formatting

#3506 fix part 3, minor formatting

* #3506 fix tests

#3506 fix tests

Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>

* mdm matching (#3579)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* begin with failing test

* fixed

* changelog

* add jira tag

* Update to 6 1 (#3582)

* added changelog folder, upped version

* version enum

* add a few more unit tests to assert proper NO_MATCH exclusion

* revert merge master doh! bad reflexes

* revert merge origin master

* unrevert revert.  ugh what a pain

* merge recovery.  fix poms.

* merge recovery.  more reverting

* merge recovery.  more reverting

* merge recovery.  more reverting

* Revert "merge recovery.  fix poms."

This reverts commit ae6e0ddb06.

* more revert revert reversions

* more revert revert reversions

* pre-review cleanup

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>

* Fix regression of 3411 - _lastUpdated gets clobbered during $reindex job (#3586)

* When chunking for the reindex job, don't clobber the lastUpdated if provided by the caller.

* License

* Ks 20220508 log colour (#3592)

* don't use colours when output is redirected to a file

* change log

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551) (#3594)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* 3584 case sensitive string elasticsearch (#3596)

* Fix for case-sensitive search with newer elasticsearch

* Test and fix ascii normalization too

* more test cases

* comments

* Docs update empi usecase (#3598)

* Update diagram for use case 5

* Add new page for database support

* Licnese

* Mb norm fix (#3604)

* Normalize query since wildcard searches aren't normalized in elastic

* Handle batch2 job cancellation (#3603)

* Handle batch2 job cancellation

* Add CANCELLED status

* Remove unuseful test

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Add batch2 interfaces to obtain recent instances (#3601)

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* changelog folder

* Documentation change, added warning log and change log for ticket (#3609)

Co-authored-by: Steven Li <steven@smilecdr.com>

* fix ne for lastUpdated search param (#3589)

* init/wip

* add implementation, add tests

* add checks and some refactoring

* refactor and simplify tests

* refactor

* get rid of some of the warnings

* add changelog

* remove my todos

* redo ne logic to account for less precise dates

* add tests

* refactoring

* rename to follow convention

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>

* Version bump

* Add skip of checkstyle to upload

* Typo

* upping version enum, upping version number, 6.0.1

* Group Bulk Export rules exiting too early.  (#3643)

* Implementation, test, changelog

* Add jira link

* adding backport in yaml for 3642

* adding folder for 6.0.1 release

* pom changes to publish checkstyle to sonatype

* bad version in pom file

* Make FulltextSearchSvcImpl optional bean for ValuesetOperationProvider

* AUtomcplete error code

* updating comment indicating final erorr code

* Bump error code

* fixing version enum so release docs generate correctly, adding additional changelog for 3650

* 3684 subscription for delivering delete events formatted wrong (#3685)

* Fixed issue with delivering delete events Subscriptions

* Extracted deliverer class name to a constant, now shared across tests and interceptor class

* Added change log and doc changes

* moved the constant to a more appropriate place

Co-authored-by: Steven Li <steven@smilecdr.com>

* Version bump

* Add version enum

* Version enum and folder

* adding release pipeline yml file fo azure

* Adding autoclose for sonatype publishing (#3797)

* Add impl test and changelog (#3713)

* fix up changelog, new version

* Version bump

* Updating version to: 6.0.4 post release.

* reverting 6.0.4 commit

* Updating version to: 6.0.4 post release.

* removing double entry from preious release

* Add support for DSTU3 delete subscrion events, add test

* Add changelogP

* Add version enum

* Remove notnull

* Add test for DSTU2

* Updating version to: 6.0.5 post release.

* Fix up

* wip

* Adding solution and changeLog file.

* Adding failing test.

* fix up

* Version bump

Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: katiesmilecdr <88786813+katiesmilecdr@users.noreply.github.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: alackerbauer <33912849+alackerbauer@users.noreply.github.com>
Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
2022-08-30 02:29:02 +00:00
Mark Iantorno caf370da31
Azure Pipelines Overhaul (#3948)
This PR changes the way HAPI is built by Azure pipelines. Instead of a FIFO queue for all tests in the project, the entire project is quickly built and cached without testing, then the individual modules are built and tested in separate jobs which can be run in parallel from each other. Each of these jobs then uploads the test coverage for that module as a build artifact. Finally, a last stage downloads all the archived tests results and aggregates them using jacoco so they can be uploaded to codecov.

This has two key benefits:

    If there is a failing test because of an intermittent, the individual failing module can be re-run instead of having to re-run the whole project build.
    The build time is decreased substantially (90min -> 45min).

This is the first run at this, I will be iterating to further reduce build times going forward.
2022-08-24 12:55:00 -04:00
Tadgh e7a4c49aac
Rel 6 1 mergeback (#3927)
* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Bump Version for non-breaking purposes

* Fix broken test

* bump error code

* revert internal logic

* Revert offset mode change

* wip

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-08-18 17:21:27 -07:00
michaelabuckley 60a879c842
Renable elasticsearch tests (#3850)
* Renable elasticsearch tests

Limit elasticsearch container to 512MB instead of the default 4GB.
2022-08-07 07:09:12 +00:00
jmarchionatto a5c4b0756b
Fallback direct resource HSearch to JPA when resources are not found … (#3823)
* Fallback direct resource HSearch to JPA when resources are not found in index

* Use specific exception instead of identifying it by the message

* Add Message code to exception

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-08-05 16:39:00 +00:00
jmarchionatto ebac65cb31
Add simple local implementation of UCUM temperature canonicalization … (#3872)
* Add simple local implementation of UCUM temperature canonicalization for Celsius and Fahrenheit

* Adjust conversion to not increase precision, as it matter when finding ranges

* Simplify test variables

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-08-05 14:48:37 +00:00
jmarchionatto 2e1f4c25f5
Make sure that direct resource load path is only executed for synchro… (#3833)
* Make sure that direct resource load path is only executed for synchronous searches

* Remove unneeded mockito extension

* Only allow for resource loading from fulltext index when advanced search (indexed parameters) is also enabled

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-08-04 22:19:06 +00:00
jmarchionatto 7010ddd715
Allow hsearch sorted offset searches (#3864)
* Start direct HSearch path

* Support no HSearch

* Spike out the direct resource query

* Implement hsearch fast load

* Fix last master merge in issues

* Implement revision requests

* Test direct resources (no IDs query) sorting

* Use mock to count freetext searches to avoid implementing interface in test

* Remove fixme

* Make listener optional as it is used only for tests

* Provide new dependency

* Widen fast path test scope and fix previously untested configurations

* Make method transactional as it can be called from outside a TX (at least testObservationLastNAllParamsPopulated does)

* Update test validation

* Allow hsearch sorted offset searches

* Add changelog

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-08-04 12:30:20 -04:00
jmarchionatto a6c2e58c2c
Fix range calculation for number or quantity search parameters (#3871)
* Fix range calculation for number or quantity search parameters

* Unify clause building for prefixed numeric parameters.
Fix range tests for quantity.

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-08-03 15:44:10 -04:00
jmarchionatto 71a3fa949c
Fix range calculation for number or quantity search parameters (#3870)
by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-08-03 11:51:34 -04:00
David be0b6f1a0b
3840 add sort parameter to fetching jobs (#3861)
* added a paging api to ijobcoordinator

* adding changelog

* moved location

* added a comment

* Added Sort parameter to JobInstanceFetchRequest

* Bump HAPI-FHIR version

* Updated JPA api to include sort

* Add changelog

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: David Chen <david.chen@smilecdr.com>
2022-07-29 09:12:53 -06:00
jmarchionatto 5586b325ce
Issue 3857 elastic lucene search returns only 50 results when offset0 and count100 (#3858)
* Add logback definition to project to stop debugging all output

* Fix limit definition to include _offset=0 case

* Move test required resources from previous test module

* Add changelog

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-07-28 19:09:31 -04:00
TipzCM 59540ce803
3813 paging job coordinator (#3814)
* added a paging api to ijobcoordinator

* adding changelog

* moved location

* added a comment

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-07-26 11:29:37 -06:00
Ken Stevens 533019f051
Response Terminology Wiring (#3807)
* cleaned up response terminology wiring

* jpa is failing atm

* bump pom versions

* double build RAM

* bump hapi version to PRE-22

* license headers

* re-bump to PRE-22

* tag intermittent

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-25 18:41:01 -04:00
Ken Stevens f01e4a7ad2
Remote Terminology support refactor (#3822)
* Revert "test passes with a lot of copy/paste"

This reverts commit f2f28e22ef.

* make translateOutcomeToResults public

* double RAM

* final consolidation.  move shared code out to static utility methods

* revert azure memory
disable jacoco on jpaserver test prokect

* bump hapi version

* rebump hapi to PRE-21

* licenses

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-25 11:47:31 -04:00
Ken Stevens 85384feac7
separate elastic tests into their own project (#3827)
* move elastic tests out into their own project in the hopes it solves ci memory issues

* move elastic tests out into their own project in the hopes it solves ci memory issues

* disable elastic tests temporarily

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-21 17:12:19 +00:00