Commit Graph

393 Commits

Author SHA1 Message Date
StevenXLi 83411e585b
4570 add support for warning messages to the batch framework (#4571)
* added failing test

* added support for warning messages to batch2

* added change log

* fixed errors from merge, made warning work with new batch2

* fixed message parse, changed interface method to default, formatting changes

* added changelog

* code review changes

* code review change

* added new implementation for warning message processing, fixed migration

* changed column to lob, as text is not well supported

* code review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-06-01 13:16:44 -04:00
volodymyr-korzh 31ae8d8f27
4934 validation doesn't consider loaded ig immediately (#4935)
* Unit tests for validation after install and uninstall IG

* Fix validation result cached after install and uninstall IG

* Fix validation result cached after install and uninstall IG - updated changelog

* Fix validation result cached after uninstalling IG - updated changelog

* Fix validation result cached after uninstalling IG - fixed unit tests

* Fix validation result cached after uninstalling IG - fixed changelog

* Fix validation result cached after uninstalling IG - test fixes
2023-05-30 14:02:14 -04:00
dotasek 34f4c90eb9
Bump org.hl7.fhir.core dependency to 6.0.8 (#4942)
* Bump core, fix compilation errors

* Match pattern in kindling for use of version

Match the pattern used by Grahame in this update of kindling: f765642470

* Bump core, fix compilation errors

* Match pattern in kindling for use of version

Match the pattern used by Grahame in this update of kindling: f765642470

* Bump core to 6.0.5, fix compilation errors

* Bump to core version 6.0.8

* Individual message code for exception

* Bump core to 6.0.10

* Fix test message
2023-05-30 09:36:40 -04:00
Etienne Poirier e068a2f32b
4799 searching on paramname profile with and missing modifier does not match resources (#4840)
* initial test

* Adding changelog

* Updated to remove the discarding of search params starting with an underscore

* test commit

* Updated handling of search parameters with underscores

* Revert work in progress

* Updated storage settings with new property for inline tag storage mode

* Updated storage settings to tell when inline tag storage mode is set

* simplified test failure

* Added tests for _tag and _system

* Add PreviousVersion service with tests (#4902)

* started writing PreviousVersionReader

* started writing PreviousVersionReader

* moar tests

* add partitioning to the test

* switch subscription to use previous version reader

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Updated logic to be less confusing

* Bumping version to 6.7.4

* Reverted hapi bump, added improve logic for handling missing search params

* updates

* Updated failing tests

* remove redundant tests

* more updates

* Adding Jira number.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: David Raeside <davidraeside@smilecdr.com>
Co-authored-by: volodymyr <volodymyr.korzh@smilecdr.com>
Co-authored-by: David Raeside <david.raeside@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-05-30 07:12:53 -04:00
Ken Stevens b84e8c0bcc
Canonicalize R4 topic subscriptions (#4913)
* WIP

* add canonicalizer for R4 topic subscriptions

* test R4 topic subscription registration

* Add support for R4 Subscription Topic Backport

* FIXME

* add filter matching support

* add simpler signature

* add resource type matching to filter

* review feedback

* fixed

* changelog

* test both ways

* add bundle test

* change how toplevel is detected since in some contexts the parent state can be a non-null pre-resource state

* added logs

* changelog

* fix v2 issues

* Clean up WIP comments

* Switch to in-memory event numbers (instead of always 1)

* Final WIP cleanup

* Msg.code

* review feedback

* review feedback

* review feedback

* review feedback

* back out import changes

* back out import changes

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-05-29 05:32:22 +00:00
James Agnew 46857711c9
Delete expunge with cascade (#4931)
* Delete expunge with cascade

* Work

* Workgin

* Version bump hibernate

* Start working on delete cascade

* Work on delete expunge

* Test fixes

* Test fixes

* Add changelog

* Work on cascade

* Fixes

* Test work

* Test fixes
2023-05-27 11:07:59 -04:00
James Agnew 82ab61f796
Expunge performance tweak (#4920)
* Expunge performance tweak

* Add changelog

* Address review comment
2023-05-21 17:56:31 +00:00
Tadgh 805e80e61f
6.6.0 Mergeback (#4924)
* Force Verify tests

* fix ITs (#4809)

* fix RestHookTestR5IT

* fix intermittent

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix migrator error on Oracle (#4814)

* Fix Oracle SQL error

* Add changelog

* Update clinical reasoning version (#4816)

* Update clinical reasoning version

* Update version

* Update version

* Clean-up and more wireup of evaluationSettings

* Add changelog

---------

Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>

* Opening the care-gaps endpoint for GET. (#4823)

Co-authored-by: Chalma Maadaadi <chalma@alphora.com>

* added version to mdm golden resource tag (#4820)

Co-authored-by: Long Ma <long@smilecdr.com>

* Update the changelog for 4697 to be more descriptive (#4827)

* Update the changelog for 4697 to be more descriptive

* Futher tweaks of the changelog

* Fixes a bug with tags.  (#4813)

* Test, fix

* Drop constraint, add migration

* Add changelog

* Fix userSelected null vs false

* Fix merge

* Fix up checkstyle whining

* One more failure

* Fix test

* wip

* changelog clarity

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* change index

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>

* fix migration issue (#4830)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Create correct version enum

* Remove superfluous migration

* fixing test (#4835)

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* email subscription, throw NullPointerException (#4790)

* fix bug

* Favoring constructor initialization to autowiring.

* enhancing test.

* Making class LoggingEmailSender available outside of the hapi-fhir-japserver-uhnfhirtest module.

* Passing all tests.

* adding changelog.

* Bumping version to 6.5.20-SNAPSHOT

* addressing code review comment.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Add docs for CR operations (#4855)

* Add docs for CR operations

* Correct changelog and javadoc for $package

* Add documentation for $apply parameters

* Add additional documentation for $package

* Cleanup

* Cleanup

* Cleanup

* Address review comments

* Add documentation for $care-gaps operation. (#4862)

* Add documentation for -gaps.

* addressing the comments.

---------

Co-authored-by: Chalma Maadaadi <chalma@alphora.com>

* 4853 validation does not error when display is not the same as the display defined in the codesystem 2 (#4854)

* added failing test

* implemented the solution

* changed test name

* added change log

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_6_0/4853-validation-does-not-error-when-display-is-not-the-same-as-the-display-defined-in-the-codesystem-2.yaml

Co-authored-by: James Agnew <jamesagnew@gmail.com>

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>

* fixing patient everything operator (#4845)

* fixing patient everything operator

* review fix

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* fix link

* Move image file

* Bundle resources containing over 100 references to the same Organization will fail with HAPI-2207 (#4871)

* Add failing unit test.

* Fix JpaId Long equality comparison to use ! equals() instead of !=, which fails for different instances of the same Long value.

* Add changelog.

* added warn message and test (#4848)

* added warn message and test

* code review fixes

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* Issue 4804 full table scan on mpi link during mdm clear (#4805)

* version bump for next release  (#4793)

* version bump

* Bump to correctnumber

* Version Enum and folder

* Remove interim from list

* wip

* Fix operation on nested type-choices in FhirPatch implementation (#4783)

* Fix operation on nested type-choices in FhirPatch implementation

* Add credit for #4783

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* #4468 fix previous link offset no cache pagination (#4489)

* #4468 Add test reproducing the issue

* #4468 Fix previous link for no cache offset pagination

* #4468 Use unchecked URI parsing

* Credit for #4489

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Changelog and data generating test

* Add MdmLink index

* Avoid double link deletion

* Use ThreadLocal safely

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Zach Smith <85943952+zachdoctolib@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Aleksej Parovysnik <100864000+alparodev@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Fix erroneous batch 2 $export 75% complete count when the job is COMPLETE (#4859)

* Add failing unit test.

* Add conditional logic to the InstanceProgress progress percentage to disregard the incomplete count if this is called from the reduction step.  This is to get around a race condition in which a work chunk is QUEUED and not yet complete when the reduction step calculates the progress.

* Add final.

* Add changelog.

* disable wars (#4877)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4868 fix paging hapi (#4870)

* fixing some offset and adding a test

* fixing the offset paging

* Removing duplicate

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Aleksej Parovysnik <100864000+alparodev@users.noreply.github.com>

* 4875-binary-access-write-doest-trigger-STORAGE-BINARY-ASSIGN-BLOB-ID-PREFIX-pointcut (#4876)

* Add failing test

* Add failing test

* Fix and changelog

* Pass content type parameter

* Back to auto wiring the context

* Invoke interceptor only when getting blobId, not also when storing it

* Avoid breaking implementers

* Address review comment

* Add new exception Msg code

* Fix broken test

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Fix batch job (bulk export) processed record count (#4879)

* Remove racy stats recalc.

* Throw 404 when requesting $export of non-existent Group or Patient (#4890)

* Remove default implementation intended only for interim backwards compatibility (#4894)

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Rule apply patient export (#4893)

* Test, fix, and changelog

* Better partition resolution

* Add checks based on rule applier

* Fix ordering failure due to hash set

* Allow empty auth interceptor

* Fix up operation type on invocation

* Add more tests, make hack implementation for patient instance level operation

* Tighten test name

* Changelog

* Default method

* remove dead method

* Remove dead autowire

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>

* cve pom changes (#4898)

Co-authored-by: Long Ma <long@smilecdr.com>

* backport subscription topic bean cleanup (#4904)

* 4891 bulk export do not recurse unasked for resources (#4895)

* updating tests

* fixing bulk export to not fetch resources not requested

* cleanup

* cleanup

* more warning suppressing

* adding error code

* blah

* fix test

* review fixes

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* lowers log level to remove bootup noise (#4908)

* CVE rel 6 6 (#4907)

* cve pom changes

* bump javax.el to jakarta.el

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* Issue 4905 post binary failure invoking interceptor for pointcuts storage preshow resources (#4906)

* Initial failing test

* Avoid applying binary blob id prefix multiple times

* Remove recently introduced method not needed anymore

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Enhance LogbackCaptureTestExtension (#4869)

* repro bug with test, fix bug

* ken informed me he resolved this bug on master, so i'm switching to use his solution

* disable wars

* review feedback

* review feedback

* review feedback again

---------

Co-authored-by: josie <josie.vandewetering@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Resolve 4863 from release branch searchparametercanonicalizer does not account for search parameters for custom resources types when converting dstu23 into runtimesearchparam (#4887)

* Modified canonicalizeSearchParameterDstu2 and 3, now correctly detect search parameters for custom resources

* Canonicalizers now correctly handle search parameters for custom resources

* created changelog

* Modification based on comments:
- remove Resource from target field when there are custom resource types
- fixed changelog typo
- removed unnecessary variable providesMembershipInCompartments

* Added tests for the SearchParameterCanonicalizer to test if base and target of RuntimeSearchParam is set as expected for DSTU2, DSTU3, R4, R4B, and R5 resources

* Fixed typo and removed commented code

* re-ordered init methods

* Update changelog

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* modifications following first code review.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* License

* Remove _lastUpdated filtering of _revincludes. (#4899)

Remove _lastUpdated filtering of _revincludes.

* 4910-dm-migration-error-for-oracle-19c (#4916)

* Remove all_constraints references which break in oracle 19c

* Add changelog

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* 4873 empty fhirid causes missing resource (#4874)

* add check for empty fhirid string and add test

* add test for populateid

* changelog

* version bump

* version bump

* reverse version bump

* Back to 6.5.21-SNAPSHOT.

---------

Co-authored-by: justindar <justin.dar@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>

* Fix include canonical url performance (#4919)

Use hash_identity for canonical join

* License

* Version bump

* Fix failure in test

* Licenses

* Review comments for pipeline

* Dead entry

* other typo

---------

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Brenin Rhodes <brenin@alphora.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: chalmarm <44471040+chalmarm@users.noreply.github.com>
Co-authored-by: Chalma Maadaadi <chalma@alphora.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Sam Gunter <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: Zach Smith <85943952+zachdoctolib@users.noreply.github.com>
Co-authored-by: Aleksej Parovysnik <100864000+alparodev@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Josie <80289977+pepsiofficial@users.noreply.github.com>
Co-authored-by: josie <josie.vandewetering@smilecdr.com>
Co-authored-by: TynerGjs <132295567+TynerGjs@users.noreply.github.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: justindar <justin.dar@smilecdr.com>
2023-05-20 20:38:35 -07:00
David d94627c382
4847 allow filtering batch jobs by status (#4849)
* Add method to get all jobs of a certain status

* add queries

* add tests

* Refactor so extra parameter is not added

* Version bump

* Fix broken test

* Revert accidental change to pom unrelated to version

---------

Co-authored-by: David Chen <david.chen@smilecdr.com>
2023-05-18 14:46:44 -04:00
James Agnew 1c66e57465
Update resource provenance indexing strategy (#4883)
* Add migration

* Update indexes

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Address review comments

* Test fixes

* Test fixes

* Test fix

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-05-18 13:23:57 -04:00
Kevin Dougan SmileCDR 2fd2d95c88
Fixed broken Unit Tester. (#4889)
* Fixed broken Unit Tester.

* Fixed issue with compilation against CDR.

* Fixed an overlay id value in the kotlin pom.xml.
2023-05-15 21:48:29 -04:00
James Agnew 05defb58c5
Server fixes from Connectathon 32 (#4838)
* NPM test cleanup

* fix imports

* Ongoing cleanup

* Rework memory queue

* Reindex cleanup

* Cleanup

* Clean up

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Test fix
2023-05-15 08:28:55 -04:00
James Agnew 483ddca3be
Transaction with delete then update should not fail (#4831)
* Fixed

* Test fixes

* Add test

* Ongoing work

* Work on xactx

* Cleanup

* Changelog cleanup

* Resolve fixme

* Rework broken APIs

* Version bump

* Add license headers

* License header update

* License

* rk on fixes

* Test fixes

* Address review comments

* Test fixes

* Add license headers

* License header
2023-05-15 07:41:40 -04:00
James Agnew 8cf14c8c0e
Dont crash indexing contained references on Bundle (#4881)
* Dont crash indexing contained references on Bundle

* Add changelog
2023-05-14 21:33:32 -04:00
Ken Stevens 4313dc9958
pointcut exceptions (#4822)
* enforce no exceptions on the invoking thread in PointCutLatch

* comment

* Msg.code

* Catch exceptions outside of await block

* fix migration issue

* Refactor PointcutLatch and add tests

* fix tests

* fix tests

* fix tests

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test!

* fix test!

* fix test

* fix test

* clean up latch error formatting

* fix test

* fix test

* fix test

* fix test

* fix test

* fix tests

* fix test

* fix test

* Remove hapi-fhir-jpaserver-uhnfhirtest from build as it is crashing CI

* fix test

* fix intermittent

* A SNAPSHOT dependency on an upstream module was added to HAPI FHIR, which changed and caused some downstream tests to break.

* fix intermittent

* Add Maven enforcer plugin for CR dependencies

* Make maven enforcer conditional on CI builds

* Remove hapi-fhir-jpaserver-uhnfhirtest from build as it is crashing CI

* improve test logging

* pre-review cleanup

* review feedback

* remove hapi-fhir-base-test-jaxrsserver-kotlin from the build

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
2023-05-11 22:31:38 -04:00
JasonRoberts-smile 0475cb682f
Jr 20230502 fhir patch test coverage (#4841)
* add tests for fhir patch

* extract constants

* extract some methods

* isolate add operation

* isolate delete operation

* isolate replace operation

* isolate move operation

* clean up integer handling

* enumeration creation

* improve variable naming

* code review feedback
2023-05-11 16:40:57 -04:00
Ken Stevens 224b7f6206
SubscriptionTopic part3 (#4817)
* Force Verify tests

* wip

* merge troubleshooting rel_6_6 troubleshooting changes

* fix ITs (#4809)

* fix RestHookTestR5IT

* fix intermittent

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* post-merge cleanup

* fix test

* fix mock test

* fix wiring

* fix mock test

* fix test

* use IBaseResource.isDeleted()

* fixmes

* cleanup

* change log

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-05-09 16:29:21 -04:00
Tadgh f0726a32d5
Fixes a bug with tags. (#4813)
* Test, fix

* Drop constraint, add migration

* Add changelog

* Fix userSelected null vs false

* Fix merge

* Fix up checkstyle whining

* One more failure

* Fix test

* wip

* changelog clarity

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* change index

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-05-05 19:06:03 +00:00
James Agnew a3c33d2a53
Add BasicAuditEventLogging (BALP) Interceptor (#4787)
* Begin work on BALP interceptor

* Work on BALP

* Basic profile

* Work on BALP

* Add BALP

* Work on balp

* Add update and delete

* Work on BALP

* Add logging

* Tests

* Modify test server

* Work on docs

* Add documentation

* Add changelog

* Fix #4728 - Typos in docs

* Test fixes

* Move changelog

* Update hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/interceptor/balp/BalpAuditCaptureInterceptor.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Resolve PR comments

* Test fixes

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-05-03 15:46:22 +00:00
Zach Smith 9c1d235f90
Fix operation on nested type-choices in FhirPatch implementation (#4783)
* Fix operation on nested type-choices in FhirPatch implementation

* Add credit for #4783

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-05-01 09:52:34 +00:00
Tadgh 9a095cc2d7
version bump for next release (#4793)
* version bump

* Bump to correctnumber

* Version Enum and folder

* Remove interim from list

* wip
2023-04-30 15:01:28 -04:00
Luke deGruchy 79c96dc1bb
Port changes to master from release branch 6_4 (#4785)
* Start porting changes to master that were lost from the release branch.

* Add remainder of tag changes from release branch.

* Changelog changes.

* Fix schema migration.

* Remove duplicate lines.

* Update core to 6.0.1.

* Fixes for new core.

* Bump to snapshot 19 to mirror James' changes.

* Fix migration tasks.

* Fix some of the tests due to the core upgrade.

* Fix rest of tests.
2023-04-28 19:56:02 +00:00
michaelabuckley 81854baa02
Batch2 cleanup (#4779)
Post review batch2 cleanup.

Review fixes and cleanup from #4647

Deleted methods in IJobPersistence unused in prod paths, and replaced their usage in tests.
Lots of docs.
Replace copy-on-write pattern in JobDefinitionRegistry with simple ConcurrentHashMap
Fixed bad mappings from job ERRORED state to external APIs. People think ERRORED is a failed state. ¯\_(ツ)_/¯
Added some spec tests for chunk purging
Deprecated ERRORED. For deleting in 6.8
Lots of plans for 6.8. Too risky for 6.6
2023-04-28 15:44:30 -04:00
James Agnew e2e5ff6bb8
Version bump HAPI FHIR (#4788) 2023-04-28 19:30:51 +00:00
James Agnew 69a79e39a1
Subscription retriggering enhancements (#4758)
* Work on forward porting reindex optimize storage

* Subscription retriggering performance enhancements

* Adjust readmes

* Add a test

* Add test

* Cleanup

* Test cleanup

* Add comment

* Add changelogs

* Test fixes

* Test fixes

* Bump size

* Fixes

* Test logging

* Restore forkcount

* Test fix

* Test fix

* Version bump

* Clean up changelogs

* Address review comment
2023-04-28 12:06:38 -04:00
James Agnew e2717bd63a
Bump corelib to FHIR R5 Final (#4727)
* Core bump

* Build build issues

* Bump core

* Cleanup

* Cleanup

* Resolve intermittent

* Work on core

* Add failing test for a check

* Test fixes

* Fixes

* DOn't mangle subscriptions

* Fix

* License headers

* Build fixes

* Version bump

* Compile fixes

* Fix compile issues
2023-04-28 06:28:22 -04:00
longma1 2171ad04a2
3924 mdm match resources across partitions and storing golden resources on a specific partition (#4770)
* added mdm search all partition for golden resource functionality

* added mdm partition setting

* build header fix

* test fixes and added missing side effect of allowing cross partition links on opeartions

* added changelog, also test fixes

* fixed changelog to an actual yaml file

* review fixes,  also simplified tests

* reverted one test because the original test schenario was correct

* added setting clearing in afterEach method so test dont fail while run in maven

* bump to pre-15

---------

Co-authored-by: Long Ma <long@smilecdr.com>
2023-04-28 02:15:35 -06:00
David 457b39bac1
Fix $mdm clear doesn't expunge golden record resources (#4767)
* Failing test

* Add method to expunge collection of deleted resources

* Failing test passes

* Add changelog

* Version bump

* Forgot to commit change

* Update test

* Check type validity

* Fix broken tests

---------

Co-authored-by: David Chen <david.chen@smilecdr.com>
2023-04-27 21:35:27 -06:00
JasonRoberts-smile ae1e7400dd
delete operation removes element from list (#4782)
* delete operation removes element from list

* changelog
2023-04-28 00:01:28 +00:00
Tadgh 78b3b148ba
Support `_exportId` for bulk exports. (#4781)
* Full implementation, test, changelogs

* Add changelogs

* Add default method

* Update hapi-fhir-storage-batch2-jobs/src/main/java/ca/uhn/fhir/batch2/jobs/export/WriteBinaryStep.java

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Code Review Comments

* Compilation failures

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-04-27 21:46:09 +00:00
James Agnew 855e10a62d
Add post-fetch filtering to bulk export (#4772)
* Add post-fetch filtering to bulk export

* Add changelog

* Add validator

* Test fixes

* Test fix
2023-04-26 17:05:03 -04:00
michaelabuckley 8813d9beda
Batch stabilization (#4647)
* Use java event names for work chunk transitions.

* Cherry-pick d5ebd1f667 from rel_6_4

Avoid fetching work-chunk data (#4622)

* add end time to reduction step (#4640)

* add end time to reduction step

* add changelog

---------

Co-authored-by: Long Ma <long@smilecdr.com>
(cherry picked from commit 37f5e59ffc)

* Cancel processing

Provide error message in cancelled jobs, and avoid transitions in final states.

* Apply tx boundary to starting job and first chunk.

* cleanup

* Apply tx boundary to work chunk processing

* Delete BatchWorkChunk

* Introduce events for job create, and chunk dequeue

* Apply tx boundary to chunk handler

* Move instance cancellation to database

* tx boundary around stats collection and completion

* tx boundary around stats collection and completion

* Extend tx boundary to error, fail, and cancel

* Move failure into status calc

* ERROR is not an "ended" state.

* Revert generics cleanup to avoid noise

* Avoid sending gated chunks twice.

* Make no-data path safer.  Cleanup

* Fix mock test for step advance.

* Delete unsafe updateInstace() call

* Cleanup

* Changelog and notes

* Fix cancel boundary.  Cleanups

* Cleanup

* Sort mongo chunks for stable paging.

Other cleanup

* Document error handling

* Cleanup

* Update hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/Batch2JobHelper.java

Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>

---------

Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
2023-04-25 15:47:23 -07:00
Tadgh d6d2ff531f
4757 externalized binary npe (#4759)
* Basic test for reading from provider

* Basic test for reading from provider

* Failing test

* Add changelog

* Update hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/BinaryStorageInterceptorR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

---------

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2023-04-25 18:34:27 +00:00
TipzCM 1c0addafeb
transaction bundle multi-threading issues (#4739)
* a solution for trying to avoid multi-thread issues for transaction bundle processing

* cleanup

* using retries

* mnor tweaks

* checking in

* changes

* updating changelog

* some minor tweaks

* cleanup

* flip if

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-04-20 18:43:37 +00:00
Ken Stevens 58850fed82
try removing fork count from surefire (#4752)
* try removing fork count from test-r4

* try removing fork count from the rest of the projects

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-04-20 13:58:35 -04:00
Ken Stevens 9e741cb145
remove enabled flag and instead only create the subscription topic beans in the fhir versions that use them (#4737)
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-04-18 10:20:43 -04:00
Etienne Poirier 88459d0d00
4657 method name is inconsistent with the method behavior (#4659)
* Test implementation

* providing method getMessageKeyOrDefault();

* providing changelog

* preping codereview.

* Deprecating method getMessageKeyOrNull in favor of getMessagekey()

* updating comments following code review.

* Bumping version to 6.5.13-SNAPSHOT

* passing all tests.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-04-17 10:41:08 -04:00
James Agnew 7bdbda9ef0
Improve Subscription Retriggering Efficiency (#4742)
* Improve subscription efficiency

* Reduce number of queries

* Add changelog

* Fix ITs

* Review comments

* Test fix
2023-04-17 08:18:57 -04:00
Luke deGruchy 35bd1c400a
Fix bundle auto-create by checking for IBaseReference.hasIdentifier() before validating URL-based references (#4731)
* First commit:  New tests, logs, and comments.

* Add hacky fix to run pipelines on.   Add another unit test.

* Disable baseUrl checking if hasIdentifier() is true.

* Add changelog.  Tweak unit tests.

* Remove junk.

* Add more assertions to unit tests.  Make new production fix code clearer.

* Add more clarity to unit tests.

* Another unit test fix.
2023-04-14 17:55:31 +00:00
James Agnew 869b6c306a
Allow transaction with conditional create and patch in same bundle (#4735)
* Allow transaction with conditional create and patch in same bundle

* Add changelog

* Cleanup
2023-04-13 15:51:46 -04:00
Ken Stevens b88ccbc7b0
R4B SubscriptionTopic support (#4724)
* add tests for r4b subscriptions

* begin with failing test

* prepare for SubscriptionTopicLoader

* backwards compatibility

* subscription topic registry done

* topic matching is working

* all but delivery is working now

* yay test passes with FIXMEs

* FIXME -> WIP

* switch notification to bundle

* message codes

* fixme

* disable services for fhir versions below R4B

* fix regression

* fix intermittent

* this change will likely break some other tests

* try a safer option

* fix tests

* fix intermittent (I hope)

* unit test

* improve logic around topic subscription categorization

* moar test

* moar test

* changed to support both r4b and r5

* moar test

* cleanup for test

* moar test

* moar test

* moar test

* changelog

* comment

* Msg.code

* fix mock

* add update test

* fix test cleanup

* tracking link for version converter issue

* review

* fix test

* fix test

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-04-12 00:44:08 +00:00
James Agnew e500b23fe6
Instance level reindex operations (#4699)
* Reindex improvements

* Work

* Work on reindex

* Test fixes

* Generator now working

* Add instance reindex

* Work

* Build fixes

* Build fix

* Fixes

* Add changelog

* Test fixes

* Resolve checkstyle fix

* Version bump

* Address review comments

* Resolve imports

* Address review comments

* Test fix
2023-04-06 17:00:41 -04:00
James Agnew a2bc9a7212
Partition aware search cleanup (#4706)
* Partition aware search cleanup

* Compile fixes

* Build fixes

* HAPI FHIR version bump

* License

* License header

* Tests
2023-04-02 11:50:20 -04:00
Ken Stevens 4fbeeccda4
enable some disabled tests (#4662)
* enabled test

* added explanations to disabled tests

* added explanations to disabled tests

* yay enabled another one

* yay enabled another one

* testEverythingWithLargeSet2 failing intermittently

* fix intermittent regression introduced by splitting test class

* test passes

* need to document

* updated documentation

* prereview cleanup

* change log

* cleanup

* cleanup

* added breaking changes

* update changelog and breaking changes upgrade document to reverse the change to always return 200 per the spec.

* back out documentation change

* update documentation

* fix tests

* dstu2

* dstu3

* fix r4 validation warning test

* changelog

* hapi-fhir version bump to 6.5.10-SNAPSHOT

* fix cdr test

* fix tests

* remove accidental commit

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-04-02 08:47:16 -04:00
Justin McKelvy 9066ad618e
Bug cql engine paging (#4664)
* WIP paging

* iterable HapifhirRetrieveProvider

* Bundle iterating action

* cr config update

* adding paging functionality for search in cr

* updating BundleIterable logic and removing paging references

* add measureeval threaded config into cr

* update properties

* adding hapi change log 4663

* version bump for hapi

* update exception with msg.code, move copyright

* exorcism of paging provider from cr

* updating IT tests for terminology evaluatemeasure

* unit test, javadocs, removed reflection code

* added comments for explanation

* fix javadoc, iterable.size instead of method

* update from master

---------

Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
2023-03-31 11:09:50 -04:00
StevenXLi 3fb9a16975
4693 enable partitioning in bulk import (#4694)
* created failing tests

* implemented feature, added more tests

* added documentation and changelog

* fixed duplicate error code

* code review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-03-31 09:43:40 -04:00
James Agnew e69fe05e96
Re-enable "Flush SP cache after SP change" (#4635)
* Revert "Revert "Flush SP cache after SP change (#4566)" (#4614)"

This reverts commit 9492999ed6.

* Additions for performance

* HAPI version bump

* Test logging improvement

* Test fixes

* Test fix

* Test fix

* Compile fix

* Fix

* License headers

* Test fix
2023-03-29 19:57:57 -04:00
James Agnew 05b3bff89f
Order Location searches using near in closest distance to center of the box (#4687)
* All working

* CHangelog tweak

* Address review comments
2023-03-29 03:19:34 +00:00
Luke deGruchy 479e412786
Ensure that source resource type of Provenance is also excluded from the $everything linkage. (#4692) 2023-03-28 14:41:24 +00:00
IanMMarshall b2c0a48915
4610 enable partitioning in bulk export (#4682)
* Initial commit adding support for partition awareness to Bulk Export.

* Additional cleanup and remove references in tests to "all partitions".

* Additional cleanup and remove references in tests to "all partitions".

* More cleanup, test fixes and additional checking to validate partition accesses.

* Additional changes to ensure checking of partition during polling operations.

* Add change log.

* More cleanup.

* Change recommended in code review.

---------

Co-authored-by: ianmarshall <ian@simpatico.ai>
2023-03-27 14:22:30 -04:00
Ken Stevens 47262f5bd5
Ks 20230324 include prov (#4685)
* fix test

* fixed

* changelog

* add another test

* improve docs

* fix change log

* fix test

* fix test

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-03-25 17:50:12 -04:00
michaelabuckley 7d26a7a38d
V2 index for string (#4629)
v2 indexing for String for when sorting.
2023-03-24 19:45:14 +00:00
James Agnew bcc1ca7593
Transaction SQL Optimization (#4679)
* Start work optimizing transaction

* Tons of test cleanup

* Cleanup

* More optimization

* Optimize

* Many tests fixed

* Work on test fixes

* Optimization done, now doing cleanup

* Cleanup

* Add docs

* Test fixes

* Test fix

* License headers

* Test fix

* Test cleanup

* Test fix
2023-03-24 10:29:56 -04:00
Jens Kristian Villadsen 90895aab57
Added exposure of uninstall (#4567)
* Added exposure of uninstall

* Add changelog

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-03-23 21:08:11 +00:00
Etienne Poirier fb057d4fb1
4671 Searching with _source parameter will return matches for all resource types (#4673)
* adding solution with new test.

* adding changelog

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-03-23 14:04:01 -04:00
michaelabuckley 4b3405a456
Use java event names for work chunk transitions. (#4658)
Change work-chunk creation to be first event in state machine.
2023-03-21 13:53:16 -04:00
Ken Stevens 8b80da6441
re-enabled some disabled tests (#4644)
* re-enabled some disabled tests

* add logging

* improve logging

* removed nonsense test

* fix regression

* fix %now

* licenses

* disable searchparam reindexing on intermittent failing test

* refining fix to RetryingMessageHandlerWrapper

* licenses

* remove hapi-fhir-cli-jpaserver from azure pipelines

* undo unneeded change

* change log

* back out unneccessary changes

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-03-17 13:04:29 -04:00
samguntersmilecdr 32d224b7bc
Bring support of conditional updates into alignment with the R4 spec (#4602)
* first draft changes

* small fixes

* rename config

* allow placeholders

* fix tests

* fix concurrency test (messy fix)

* add temp comment

* test pipeline

* add placeholder check

* remove configurability

* cleanup comment

* code review suggestions

* condition changes for test

* extract conditional expression into method

* fix conditional expression
2023-03-15 18:35:25 -04:00
StevenXLi 733259a13b
4618 refactor searchparamwithinlinereferencesextractor to create a interface and a base class (#4619)
* Introduced new interface, refactored SearchParamWithInlineReferencesExtractor, added new test

* clean up, remove unused param

* added change log

* moved error message to properties file, made base class abstract

* fixed issue with test

* code review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-03-15 16:27:52 +00:00
Luke deGruchy e4bb254754
Reverse disabled tests. (#4646) 2023-03-15 10:58:08 -04:00
James Agnew cf5470ae58
Search and Sort on Uplifted Refchains (#4633)
* Start work on refchains

* Semi working

* Add tests for transactions

* Add docs and lots of tests

* Add changelog

* Add tests

* Work on cleanup

* Add document operations

* Test fix

* Test fix

* Fixes

* Fix typo

* Test fix

* Test update

* Test updates

* Test fix

* Test fixes

* Test additions

* Test fix

* Add some javadocs

* Test fixes

* Intermittent test fix

* Doc tweak

* Test fixes

* Merge master in

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Review comments

* Version bump

* Add license

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-03-15 07:04:32 -04:00
Aditya Dave 8956b273f0
Concurrent Transaction Conditonal Creates (#4639)
* push broken test for testing placeholder create in transaction retry

* reproduce error for transaction retry

* cleanup test

* fix test testPlaceholderCreateTransactionRetry

* revert temp fix

* revert spacing

* Add tests

* Remove fixme

* Test fix

* test fix

* Test fix

* Remove test line

* Remove unneeded change

* Review comments

* Add docs

* Test logging

* Add test logging

* Add logging for intermittents

---------

Co-authored-by: aditya_dave <aditya@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-03-14 16:39:55 -04:00
Luke deGruchy ea8b68f5e5
Introduce hibernate envers, use it only for MdmLink, disable it for now, add to MdmLink Dao and unit test revisions. (#4615)
* First commit with new maven deps for envers and stubbed changelist.

* Add repositoryFactoryBeanClass = EnversRevisionRepositoryFactoryBean.class to BaseAppCtxPersistence.  Comment it out in JpaConfig.  Add constant for hibernate.integration.envers.enabled.  Hard-code envers to false.  App starts without error.  Basic smoke testing for MpmLink update works.

* Add mdmLink code to retrieve envers history and unit test it.

* Initial code to handle envers from HapiFhirEnversRevision along with a custom extension of the revision/REVINFO entity.

* Add changelist and tweak table/sequence/field names and clean up the HapiFhirJpaMigrationTasks method.

* Fix some unit test failures and address TODOs.

* Ensure hard-coded deactivation of envers is overridden by unit tests.

* Remove commented-out code.

* Address code review comments.

* Remove sysout traces.

* Tweaks to poms with comments explaining why I left the versions in the plugin dependencies.  Get rid of a duplicate envers dependency.

* Bump hapi-fhir version to 6.5.5-SNAPSHOT.

* Disable intermittently failing test: PartitionedSubscriptionTriggeringR4Test.testCreateSubscriptionInPartitionAndResourceInDifferentPartition

* Try disabling Batch2CoordinatorIT failing test and see if branch pipeline passes.

* Add back hapi-fhir-validation-resources-r4b to hapi-fhir-jpaserver-base pom.xml.

* Bump hapi-fhir to 6.5.6-SNAPSHOT.

* Bump rest of hapi-fhir to 6.5.6-SNAPSHOT.

* Fix compile error on Batch2CoordinatorIT.

* Change revision ID column from integer to long in both the revision entity and the migration tasks.  Fix a bug with the migration tasks having the wrong name for the timestamp column.

* Fix interface in MongoDB to return Long for the revision ID instead of Int.   Implement TINYINT for all supported database products.  Keep TARGET_TYPE column at 40 instead of 100 for now since otherwise the schema migration test will fail.  Also, don't try to rename the foreign key or the schema migration test will fail.
2023-03-14 15:32:00 -04:00
Etienne Poirier fdf7450096
4598: Possible creation of resource duplicates when ingesting large quantity of resources. (#4609)
* Creating test to reproduce the issue.

* rebasing development branch from release to master.

* adding migration task.

* providing solution to duplicate resource creation and passing initial test.

* making initial test more readable.

* Implementing test for scheduled deletion of stale searchUrl entries.

* Implementing test for scheduled deletion of stale searchUrl entries.

* Removing interface declaration since it was not needed.

* Reverting changes.

* Adding changelog.

* Adding comments for clarity.

* Adding purgeDatabase operation in IT to delete all ResourceSearchUrlEntity [before|after}Each tests.

* Adding pom.xml exclusion to get all tests passing.

* Adjusting query counts and adding pointcut name to log in interceptor.

* Fixing typos.

* WIP

* implementing edge case tests and solutions.

* wip: passing all tests.

* wip: passing more tests .

* wip: passing tests.

* wip: passing tests.

* enhancing the test.

* changes following code review.

* modifications to tests.

* minor modifications following second code review.

* passing all tests.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-03-14 08:01:06 -04:00
James Agnew 59123cf0f0
Fix intermittent in batch2 tests (#4643)
* Fix intermittent in batch2 tests

* Add changelog

* Test fix

* Resolve intermittent

* Reducer changes

* Test fix

* Test fixes

* Resolve fixme

* Fix changelog
2023-03-13 19:59:05 +00:00
michaelabuckley d1f9c94038
Make WorkChunk handling transactional at state transitions. (#4621)
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
2023-03-12 19:19:21 -04:00
Nathan Doef 2f3108c2b6
fix tests (#4631) 2023-03-07 14:28:31 -05:00
Tadgh bfa8ca9571
Add chaained near search test (#4611)
* Add chaained near search test

* robustify test, remove gate checking

* Add changelog

* Correct issue number

* Tidy
2023-03-03 08:30:28 -08:00
James Agnew 9492999ed6
Revert "Flush SP cache after SP change (#4566)" (#4614)
This reverts commit 2083b7ca74.
2023-03-03 02:22:42 +00:00
James Agnew ae1d249d99
Bulk export reducer step fix mergeback (#4606)
* One more fix for #4467

* Enabling massIngestionMode causes incomplete resource deletion (#4476)

* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)

* Adding initial test.

* Adding initial solution implementation.

* Adding change log and code clean up.

* addressing comments from 1st code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Change visibility of migration method (#4471)

* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)

* First commit:  Make SubscriptionValidatingInterceptor aware of which Pointcut is being called.  In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED.   Fix resulting compile errors in various unit tests.

* Fix/enhance unit tests.  Mark methods as deprecated instead of deleting them.  Add proper error code.  Complete changelog.

* Remove erroneous TODOs and tweak the validation logic.

* Enhance unit tests and fix changelog.

* Reindex batch job fails when processing deleted resources. (#4482)

* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* cleaning up checkstyle files (#4470)

* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Bump core to 5.6.881 (#4496)

* Bump core to 5.6.881-SNAPSHOT

* Work on fixing tests

* Work on fixing tests 2

* Bump to core release

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>

* Issue 4486 mdm inconsistent possible match score values (#4487)

* Extract method for readability

* Save always normalized score values in POSSIBLE_MATCH links.

* Avoid setting properties to null values. Adjust test.

* Simplify fix

* Fix test. Add RangeTestHelper.

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Revert "cleaning up checkstyle files (#4470)"

This reverts commit efae3b5d5f.

* core version fix

* Loosen rules for id helper

* License

* fix batch2 reduction step (#4499)

* fix bug where FINALIZE jobs are not cancellable

* moved reduction step to message hander

* moving reduction step to queue

* addingchangelog

* cleaning up

* review fixes

* review fix'

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Scheduled batch 2 bulk export job and binary delete (#4492)

* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.

* Change bulk import test for valueUri type (#4503)

* change tests

* suggested test change

* CVE resolutions (#4513)

* Bump Postgres for CVE

* Bump jetty

* Verison bump

* Remove comments

* Revrt bump

* Add check in scanner (#4518)

* 4516 create hapi fhir cli command to clear stale lock entries (#4517)

* Initial implementation

* better tests

* Add changelog and docs

* Forgotten files

* Code review comments

* Fix checkstyle

* Unable to Expunge CodeSystem (#4507)

* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>

* New line::

* Update to documentation regarding narrative generation; (#4521)

Providing changelog;

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* changed what score is set for mdmlinks that created new golden resource (#4514)

* changed what score is set for mdmlinks that created new golden resource

* fix test

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* REVERT: change to operationoutcome.html

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* added change log

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fix build (#4530)

* Making narrative_generation.md reference an html snippet (#4531)

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* 4441 rel 6 4 bad references creation bug (#4519)

* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fixed channel import null pointer exception from null header (#4534)

* retryCount 0 on null header + test + changelog

* suggested changes

* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)

This reverts commit 53252b8d15.

* Better error handling for when channel type is not supported (#4538)

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Avoid logging message payloads that contain sensitive data (#4537)

Don't log payloads - they may contain sensitive data.

* Bulk Export Bug With Many Resources and Low Max File Size (#4506)

* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd953.

* Revert "maintenance pass"

This reverts commit bbc7418d51.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149.

* testing

* Revert "testing"

This reverts commit aafc95c2f3.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* bump ver

* License updates'

* Downgrade dep'

* Updating version to: 6.4.1 post release.

* Add javadocs and sources to our serviceloaders

* Reset version

* Change parent

* Remove bumped version

* License fixes, new parent

* Updating version to: 6.4.1 post release.

* Fix bad creation of versionenum

* Improve performance on bulk export

* Add changelog

* Start working on cleaned up reducer

* Clean up batch calls

* Work on issues

* Build fixes

* typedbundleprovider getallresources override (#4552)

* typedbundleprovider getallresources override

* added test to immunization tests that validates pagination does not result in null pointer if over default queryCount

* moved HapiFhirDal test to its own test class

* removed unused imports

* Update to use JpaStorageSettings

* adding changelog for issue 4551

* fix changelog spacing

* changelog type  to fix

---------

Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>

* Add backport info

* Upgrade core to 5.6.97, make adjustments in hapi-fhir, and ensure that all tests pass (#4579)

* First commit:  Create new branch from the release branch with changes from James' branch.  This probably won't compile as the work is incomplete.

* Second round of changes from integrating James' branch.

* Mark most test failures with TODOs.

* Add whitespace

* Add changes to FhirPathR4 to set FHIRPathEngine to non-strict FP evaluation.

* Fix CreatePackageCommandTest to assert null instead of empty string.  Comments on tests that fail due to the double-quote encoding problem.

* Downgrade to core 5.6.97.

* Fix another test and remove TODOs.

* Fix changelog.

* Clean up some erroneous changes and TODOs.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Fix up dal test

* Address leftover code review feedback from the upgrade to core 5.6.97. (#4585)

* Exclude pinned core deps

* Force pin structs

* Add model changes to IBaseCoding and related changes (#4587)

* Add IbaseCoding changes, and tinder changes

* Fix up tag definition

* converter addition

* Fix unit test and add changelog.

* Add jira to changelog.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Fix changelog

* Tidy metadata

* Test fixing

* Test fixes

* Work on progress

* Add changelog

* Build tweak

* Fixes

* Disable intermittently failing tests. (#4593)

* rename tests to IT

* Disable more intermittently failing tests (#4595)

* Disable more intermittently failing tests.

* Disable another intermittently failing tests.

* ITify

* Disable yet another intermittently failing tests. (#4596)

* disable

* disables

* Fixes

* disables

* Fix compile

* Test fixes

* Updating version to: 6.4.2 post release.

* Bump to 6.4.2-SNAPSHOT

* Fix compile

* fix version

* Address review comments

* Review comments

* Version bump

* Compile fix

* Test fixes

* Compile fixes

* One more compile fix

* Test fixes

* Version bump

* Resolve fixme

---------

Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
2023-03-01 14:52:36 -05:00
James Agnew 2083b7ca74
Flush SP cache after SP change (#4566)
* Flush SP cache after SP change

* Add changelog

* Test fix
2023-02-28 13:42:08 -05:00
michaelabuckley ec13b751fe
Support retry of $reindex version conflict (#4603)
* Support retry of $reindex conflict
* Notes and cleanup of Phaser
2023-02-28 17:53:06 +00:00
James Agnew bf495e2d92
Don't merge yet: MegaScale API Changes 2 (#4544)
* Megascale changes 2

* Cleanup

* CLeanup

* Sync master

* Cleanup

* Ongoing work

* Move service

* Fixes

* Ongoing megascaler work

* Work

* Work

* Fixes

* Work

* Test fixes

* Work on remote services

* Interceptor rework

* Work on config refactor

* Fixes

* Docs tweaks

* Address review comments

* Test tweak

* Test fixes

* Try to fix tests

* Test tweaks

* Fix changelogs

* Work on merging

* Changes

* Megascale fixes

* Remove accidental commit

* Address fixmes

* Cleanup

* Test fixes

* Compile fix

* Test fixes

* Account for review comments

* Bump HAPI FHIR version
2023-02-27 22:19:28 -05:00
michaelabuckley cedf69516b
Simultaneous $reindex and DELETE can orphan index rows (#4584)
Add optimistic write lock to $reindex operation to prevent indexing stale data.

Introduce Lockstep Phaser to make testing easier
Add 1 query to every resource reindex :-(
2023-02-26 22:12:12 -05:00
Qingyixia 3554e9cf91
Fix the issue of missing meta.tags in subscription message (#4576)
* Fix + Test

* Addressing comments

* Minor fix
2023-02-23 15:07:10 -08:00
Tadgh 607f27b686
4523 subscription sources (#4543)
* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* Revert changes, add new test clas, new test consumer for queus

* Finish implementation

* fix compilation

* Remove unused method and test from the previous solution

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* Revert changes, add new test clas, new test consumer for queus

* Finish implementation

* fix compilation

* Remove unused method and test from the previous solution

* More Renaming

* fix compilation failure

* Revert compilation fix and merge master

* Fix to pipeline test failures

* Addressing suggestions

---------

Co-authored-by: qingyixia <cherry.xia@smilecdr.com>
2023-02-17 13:09:06 -05:00
James Agnew eea6193df9
JPA config refactoring (#4545)
* Interceptor rework

* Work on config refactor

* Fixes

* Docs tweaks

* Address review comments

* Test tweak

* Test fixes

* Try to fix tests

* Test tweaks
2023-02-15 08:43:01 -05:00
Ken Stevens 794e4510c6
6_4 mergeback (#4547)
* One more fix for #4467

* Enabling massIngestionMode causes incomplete resource deletion (#4476)

* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)

* Adding initial test.

* Adding initial solution implementation.

* Adding change log and code clean up.

* addressing comments from 1st code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Change visibility of migration method (#4471)

* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)

* First commit:  Make SubscriptionValidatingInterceptor aware of which Pointcut is being called.  In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED.   Fix resulting compile errors in various unit tests.

* Fix/enhance unit tests.  Mark methods as deprecated instead of deleting them.  Add proper error code.  Complete changelog.

* Remove erroneous TODOs and tweak the validation logic.

* Enhance unit tests and fix changelog.

* Reindex batch job fails when processing deleted resources. (#4482)

* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* cleaning up checkstyle files (#4470)

* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Bump core to 5.6.881 (#4496)

* Bump core to 5.6.881-SNAPSHOT

* Work on fixing tests

* Work on fixing tests 2

* Bump to core release

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>

* Issue 4486 mdm inconsistent possible match score values (#4487)

* Extract method for readability

* Save always normalized score values in POSSIBLE_MATCH links.

* Avoid setting properties to null values. Adjust test.

* Simplify fix

* Fix test. Add RangeTestHelper.

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Revert "cleaning up checkstyle files (#4470)"

This reverts commit efae3b5d5f.

* core version fix

* Loosen rules for id helper

* License

* fix batch2 reduction step (#4499)

* fix bug where FINALIZE jobs are not cancellable

* moved reduction step to message hander

* moving reduction step to queue

* addingchangelog

* cleaning up

* review fixes

* review fix'

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Scheduled batch 2 bulk export job and binary delete (#4492)

* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.

* Change bulk import test for valueUri type (#4503)

* change tests

* suggested test change

* CVE resolutions (#4513)

* Bump Postgres for CVE

* Bump jetty

* Verison bump

* Remove comments

* Revrt bump

* Add check in scanner (#4518)

* 4516 create hapi fhir cli command to clear stale lock entries (#4517)

* Initial implementation

* better tests

* Add changelog and docs

* Forgotten files

* Code review comments

* Fix checkstyle

* Unable to Expunge CodeSystem (#4507)

* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>

* New line::

* Update to documentation regarding narrative generation; (#4521)

Providing changelog;

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* changed what score is set for mdmlinks that created new golden resource (#4514)

* changed what score is set for mdmlinks that created new golden resource

* fix test

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* REVERT: change to operationoutcome.html

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* added change log

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fix build (#4530)

* Making narrative_generation.md reference an html snippet (#4531)

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* 4441 rel 6 4 bad references creation bug (#4519)

* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fixed channel import null pointer exception from null header (#4534)

* retryCount 0 on null header + test + changelog

* suggested changes

* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)

This reverts commit 53252b8d15.

* Better error handling for when channel type is not supported (#4538)

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Avoid logging message payloads that contain sensitive data (#4537)

Don't log payloads - they may contain sensitive data.

* Bulk Export Bug With Many Resources and Low Max File Size (#4506)

* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd953.

* Revert "maintenance pass"

This reverts commit bbc7418d51.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149.

* testing

* Revert "testing"

This reverts commit aafc95c2f3.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* bump ver

* License updates'

* Remove checkstyle

* Verison bump

* Move migrations

* semicolon

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-02-13 13:23:25 -08:00
JorisHeadease 9665476c00
The `OverridePathBasedReferentialIntegrityForDeletesInterceptor` now … (#4483)
* The `OverridePathBasedReferentialIntegrityForDeletesInterceptor` now works for a multitenant enabled server. The `RequestDetails` weren't set while requesting the conflicting resource via the `DaoRegistry`.

* Add credit

---------

Co-authored-by: Joris Scharp <joris@headease.nl>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-01-31 13:41:41 +00:00
Tadgh 54c9b9422a
Bump to 6.5 post-release-branch (#4472)
* Create version enum and bump

* Version bump

* fix typo
2023-01-28 09:28:15 -08:00
michaelabuckley 10060bef7f
Batch option processing cleanup (#4466)
* BulkDataExportOptions null safety
2023-01-26 11:50:45 -05:00
James Agnew d46f6d635e
Web Admin Console Improvements (#4456)
* Testpage improvements

* Work on tests

* Split fragment out

* Reformat command buttons

* Work on web admin console

* Work on testpage

* Fixes

* Fix

* Test fixes

* Improve examples

* Test fixes

* Address review comment
2023-01-25 09:27:12 -05:00
Tadgh 456cc81b32
Rel 6 2 5 mergeback (#4459)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix wrong pinned version

* Updating version to: 6.2.2 post release.

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https… (#4291)

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https://github.com/hapifhir/hapi-fhir/issues/4276

* Credit for #4291

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* backport and changelog for 6.2.2

* Updating version to: 6.2.3 post release.

* fix https://simpaticois.atlassian.net/browse/SMILE-5781

* Version bump to 6.2.3-SNAPSHOT

* Auto retry on MDM Clear conflicts (#4398)

* Auto-retry mdm-clear on conflict

* Add changelog

* Build fix

* Disable failing test

* Update to 6.2.3 again

* Update license dates

* Dont fail on batch2 double delivery (#4400)

* Don't fail on Batch2 double delivery

* Add changelog

* Update docker for release ppipeline

* Updating version to: 6.2.4 post release.

* Add test and implementation to fix potential NPE in pre-show resources (#4388)

* Add test and implementation to fix potential NPE in pre-show resources

* add test

* WIP getting identical test scenario

* More robust solution

* Finalize Code

* Add changelog, move a bunch of changelogs

* Remove not needed test

* Minor refactor and reporting

* Fix up megeback

* update backport info

* update backport info

* Updating version to: 6.2.5 post release.

* Prevent chunk from returning to in-progress unless it is errorred, in-progress, or queued

* changelog

* Update logger

* Indicate backport

* Add version and upgrade info. add backport

* hapi-fhir side done

* cdr side

* add assert

* add test

* Fix import

* prevent multiple reduction step runs in maintenance run (#4423)

* setting job to inprog to prevent multiple maintenance passes from running reduction steps

* added a new status

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Add backport info

* Fix up tests

* Fix discrepancies

* Fix test

* Version bump

* bump ips

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-01-24 17:38:35 -08:00
KGJ-software 527a948285
Added SearchRequest.requester to Patient/everything (#4462)
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2023-01-24 13:28:19 -05:00
James Agnew d584e14048
Add IPS Generator (#4438)
* Begin IPS refactor

* Commit work so far

* Fix test bug

* Fix typo

* Fix typo

* Narrative generator cleanup

* Narrative generator working

* Add test for bad reference in narrative

* Tests passing

* Start docs

* Tests passing

* Cleanup

* Update cyangelog

* Doc tweaks

* Version bump

* Address review comments

* Address review comments

* Build fix

* Cleanup

* Compile fix

* Test fix

* Test fix

* Version bump

* Build update

* Test fix

* Test fix

* Add one utility method

* Add doc
2023-01-24 12:07:36 -05:00
Nathan Doef 320a0d895f
Fix ConceptMap processing in transaction (#4442)
* failing test

* fix

* only store ConceptMap once

* remove new myPreliminaryTransactionWrite flag and use thePerformIndexing

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2023-01-24 10:58:11 -05:00
ivagulin 758c8f12dd
fix ValueSet$expand?includeHierarchy=true with fts disabled (#4449)
* fix ValueSet$expand?includeHierarchy=true

* add test for valuset$expand with hierarchy without hibernate search

* move testExpandValueSet_PreExpandedWithHierarchyNoHibernateSearch to FhirResourceDaoR4ValueSetTest, add javadoc

* Add credit

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-01-23 10:25:24 -05:00
Ken Stevens 5dda16d467
Reduce postconstruct use (#4445) 2023-01-21 00:37:58 -05:00
Ken Stevens bf53a08b83
Batch2 fastracking configurable (#4453)
* licenses

* hapi-fhir side done

* cdr side

* add assert

* add test

* bump hapi version

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-20 19:03:13 -05:00
Qingyixia 663834ddfe
Added resourceType as a parameter to $mdm-duplicate-golden-resources operation (#4434)
* Solution + changelog

* Addressing suggestions

* Minor changes

* Version Bump
2023-01-19 08:43:44 -05:00
James Agnew 6eeeb7068b
Remove duplicate field in JPA bundle provider (#4435)
* Remove duplicate field in JPA bundle provider

* Add changelog

* Work on tests

* Test fixes

* Test fix

* Add logging to detect an intermittent

* Test fix

* Address review comment
2023-01-18 18:42:04 +00:00
TipzCM 23b593aa17
breaking up package installer and package config (#4319)
* breaking up package installer and package config

* breaking out parsing of resources

* wrong nullable

* added an integration test

* adding a changelog

* added a few minor tests

* review fixes

* unit test fixes

* fixing code checkstyle

* moving fetching local package data to package loader svc

* updating the method name

* splitting the fhirversion parsing

* updating table creation task for pretty printing and use in RTE

* adding foreign keys

* Adding mroe info

* minor prettiness changes

* fixing fk generation

* minor changes

* update version

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2023-01-17 01:58:14 +00:00
Ken Stevens 04c28b6c08
Schedule jobs via IJobScheduler instead of in @PostConstruct (#4415)
* parameterizing BinaryStorageInterceptor

* parameterizing BinaryStorageInterceptor

* changed jobs scheduling from @PostConstruct to coordinated by HapiLifecycleService events

* fix test

* remove events for now

* fix test

* simplify design

* FIXME

* changelog

* fix compile issue

* review feedback.  partially implemented

* review feedback

* review feedback

* bump hapi version to 6.3.8-SNAPSHOT

* fix test

* mirror contextclosedevent handlers with contextrefreshedevent

* version bump

* licenses

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-14 23:51:04 -05:00
michaelabuckley 21b1820294
Demote PHI logging to debug or lower (#4430)
Ensure resource bodies are only logged at debug or lower.

* Drop possible-phi logs to debug.
* Demote request/response log bodies to debug since they may contain PHI.
* Demote job-param logging to debug.
* Add startup hook to App for test injection.
* Change terminology cli logging to avoid logging whole resource.
2023-01-13 15:34:29 +00:00
Qingyixia a9ecc18986
Fixed $export-poll-status null point exception issue (#4428)
* Tests + Fix

* Addressing suggestions

* Addressing suggestions

* Minor fix
2023-01-13 08:59:44 -05:00
James Agnew d0a77f4322
Clean up ResourceMetadataKeyEnum (#4425)
* Clean up ResourceMetadataKeyEnum

* Compile fix

* Test fix

* Test fix

* Change

* Test fixes
2023-01-12 11:22:40 -05:00
James Agnew 7a31860376
Improve query count accuracy (#4424)
* Improve query count accuracy

* Add changelog

* Test fix
2023-01-11 17:53:04 -05:00
StevenXLi 920755427b
4420 tags are not included in search results after using $meta delete on one tag (#4421)
* added failing test

* added a check for when the tag storage mode is non-versioned in meta delete to only check latest version of resource, added more test

* added change log

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-01-11 08:03:51 -05:00
Justin McKelvy 8c48a6fb91
Clinical reasoning module (#4347)
* Add new Clinical Reasoning module

* Add config to get test up and running

* Removed FhirClient from tests

* First passing test

* Remove unused utilities

* Build updates

* exclude xxp3 and substitute with unbundled version

* Refactor to use provider factory and loader pattern

* First passing dstu3 test

* CR r4 test config and immunization tests

* Add base test classes

* Exclude transitive dependency xmlpull

* More dependency cleanup

* Refactor to remove JPA dependencies, rename module to hapi-fhir-storage-cr

* Fixes for duplicate xpp3 classes

* Remove more references to jpa

* Add mock FHIR server for terminology endpoint tests

* fix cql file for making providerr4 tests pass

* fix pom

* fix generic

* Adding r4 test cases for measure operations.

* Fix endpoint tests

* javadoc comment, temporarily comment out test results for practitioner sync

* javadoc typo fix

* adding java docs for cr module WIP

* Updates to clinical reasoning documentation

* More documentation updates

* docs now compile

* fix test

* DqmR4IT now successfully starts up, but the test does not pass

* Fix additionalData test

* Make measure service bean overridable

* adding clinical reasoning javadoc information

* Rename interfaces to start with I

* More styleguide updates

* Reformat all code in the storage-cr module

* More coding convention updates

* Remove bunches of unused code, reogranize so that fhir version in package name is consistent

* Clean up some static array initializations

* add practitioner patient list

* refactor product-line constant

* test dependency update

* add change log for cr module

* Add hapi-validation to the pom

* Exclude dependencies that are causing duplicates

* More dependencies and exclusions

* Make some base MeasureService components protected

* Update to latest cql compiler release

* fix measure service for null subject & practitioner and open up practitioner test

* Rev to release versions

* Add remaining Dstu3 tests

* update hedis bundle from policyHolder to beneficiary

* update snapshot

* Delete old CQL module

* Post-master-merge recovery

* Fix Java 17 feature usage issue

* update changelog with more information about new functionality

* Remove support chain wrapper usage

* Add javadocs to IResourceLoader

* added the property versioning for antlr jar.

* addressing code review comments.

* addressing code review comments.

* Fix typos in changelog

* Remove azure pipeline for cql module, add it for cr module

* Address exception comments, add Msg.code calls to exceptions

* Fix self-assignment bug

* Add storage-cr module to bom

* addressing coding comments.

* Addressed the variable naming convention to camelCase.

* Snapshot version bump

Co-authored-by: Brenin Rhodes <brenin@alphora.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: Chalma Maadaadi <chalma@alphora.com>
Co-authored-by: mdnazmulkarim <nazmul.karim@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-01-10 19:49:57 +00:00
Tadgh 65bf8d47ce
Prevent chunk from returning to in-progress unless it is errorred, in… (#4417)
* Prevent chunk from returning to in-progress unless it is errorred, in-progress, or queued

* changelog

* Update logger

* Add new test

* Add test
2023-01-10 03:36:06 +00:00
Kevin Dougan SmileCDR 84d83f6230
Restored a couple of the Unit Tests in ca.uhn.fhir.jpa.stresstest.StressTestR4Test.java (#4413)
* Restored the Unit Tests in ca.uhn.fhir.jpa.stresstest.StressTestR4Test.java.

* 4412 - Disabled some long running Unit Tests.
2023-01-09 12:55:46 -05:00
James Agnew 10615eb64c
Fix contention aware patch in transaction (#4416)
* Fix contention aware patch in transaction

* Add changelog
2023-01-08 17:51:34 -05:00
William Denton 02de3e49e3
Fix some issues with 'mvn test' on windows (#4223)
* fix handling of Windows line ends

* some tests need to wait a little longer for things to get done

* Windows path separator fix

* simpler lineend fix

* more timeouts and another path separator issue

* redo some timeout issues

* Issue in Windows: URI is not hierarchical

* another timeout issue
2023-01-07 14:49:35 -05:00
James Agnew 31e4f039ff
Don't log broken pipe exception when client disconnects (#4399)
* Avoid error on broken pipe

* Cleanup

* Add changelog

* Test fix

* Remove fixme

* Fix tests

* Fix build

* ddress warning

* Troubleshoot code

* More test logging

* Add debug logging

* Add logging

* Roll back changes

* Test fix

* Address review comments
2023-01-06 11:32:29 -05:00
Tadgh d70d813249
Rel 6 2 4 mergeback (#4406)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix wrong pinned version

* Updating version to: 6.2.2 post release.

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https… (#4291)

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https://github.com/hapifhir/hapi-fhir/issues/4276

* Credit for #4291

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* backport and changelog for 6.2.2

* Updating version to: 6.2.3 post release.

* fix https://simpaticois.atlassian.net/browse/SMILE-5781

* Version bump to 6.2.3-SNAPSHOT

* Auto retry on MDM Clear conflicts (#4398)

* Auto-retry mdm-clear on conflict

* Add changelog

* Build fix

* Disable failing test

* Update to 6.2.3 again

* Update license dates

* Dont fail on batch2 double delivery (#4400)

* Don't fail on Batch2 double delivery

* Add changelog

* Update docker for release ppipeline

* Updating version to: 6.2.4 post release.

* Add test and implementation to fix potential NPE in pre-show resources (#4388)

* Add test and implementation to fix potential NPE in pre-show resources

* add test

* WIP getting identical test scenario

* More robust solution

* Finalize Code

* Add changelog, move a bunch of changelogs

* Remove not needed test

* Minor refactor and reporting

* Fix up megeback

* update backport info

* update backport info

* Updating version to: 6.2.5 post release.

* please

* fix test

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
2023-01-06 03:45:22 +00:00
Tadgh f32bc8f67a
Add test and implementation to fix potential NPE in pre-show resources (#4388)
* Add test and implementation to fix potential NPE in pre-show resources

* add test

* WIP getting identical test scenario

* More robust solution

* Finalize Code

* Add changelog, move a bunch of changelogs

* Remove not needed test

* Minor refactor and reporting
2022-12-22 22:53:07 +00:00
Qingyixia 51797c3c1e
Group bulk export _since parameter issue (#4389)
* Test + fix

* javadoc
2022-12-22 13:19:07 -05:00
karneet1212 cfa6c13262
Add resource type as a parameter for $mdm-query-links (#4372)
* Added Solution + test

* Removed old methods

* fixed issues

* Added refactoring

* Deprecated methods on the interfaces

* Remove todo

* Documentation

* Changelog

* Changelog

* Changelog

* Changes made based on comments

* Changes made based on comments

* Clean up

* hapi bumping to version 6.3.6-SNAPSHOT

* hapi bumping to version 6.3.6-SNAPSHOT
2022-12-22 09:54:32 -05:00
Luke deGruchy 6e8db3626d
Add Device to the list of resource types considered for group bulk export async (#4384)
* Add Device to the list of resource types considered for group bulk export async. New unit test for bulk export.

* Add changelog.
2022-12-20 16:33:25 -05:00
James Agnew 0d5f8da93f
API improvements for MegaScale (#4112)
* Start work on sharded DB

* Work on migrating

* Work

* Ongoing work

* Ongoing work

* Work

* Work

* Ongoing work

* Work

* Work

* Test fix

* Test fix

* Work

* Compile fix

* Test cleanup

* Test fix

* Resolve fixmes

* Resolve fixme

* Resolve compile errors

* Compile fix

* Fixes

* Work

* Add note for later

* Test fix

* Test fixes

* Build fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Fixes

* Fixes

* Remove dead code in pom

* More test cleanup

* Docs fix

* Changes

* Fix

* Fixes

* Resolve merge conflicts

* Build update

* Merge build fixes

* Version bump

* Address review comments

* Review comments

* Review comments

* Test fixes

* Test fix

* Cleanup
2022-12-20 12:05:15 -05:00
KGJ-software 58c6c86d56
3877 more tests (#4368)
* add tests

* add tests

* clean up test file

* device test

* Added more tests

* Clean up tests

Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-12-15 14:08:39 -05:00
Nathan Doef 7b86eda8a9
Refactor Common SearchParameter Functionality (#4321)
* refactor checkpoint

* clean up

* add method to find runtime search param by id

* new search param cache tests

* extract SearchParameter validation functionality

* create common interface for combo searchparameters

* Remove ResourceTable from extractor interface

* clean up

* move SearchParameterDaoValidator

* resolve code review comments

* remove reflection

* change log

* bump version to 6.3.4-SNAPSHOT

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-12-15 06:46:45 -05:00
StevenXLi 29ebb950e8
4360 bulk export questionnaireresponses should get picked when author is not empty (#4361)
* added failing test

* implemented solution

* added more tests

* added changelog

* changed the implementation, now extended to get all the patient based search params for a given resource instead of 2 in a fixed list of resources

* added test for patient bulk export for resources not in patient compartment, fixed implementation to pass test

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-12-14 18:43:11 -05:00
Tadgh daaa92fdbc
4350 ap sa and eb prefixes are not handled correctly (#4358)
* Add test and implementation

* Add changelog
2022-12-11 18:19:54 -05:00
James Agnew 7747ebae41
Combine postconstruct methods (#4330)
* Combine postconstruct methods

* Add changelog
2022-12-11 18:02:04 +00:00
kateryna-mironova 017cc4e29c
4244 - Deprecate usages of ResourcePersitentId in jpa (#4245)
* init rev

* id to jpa and mongo id

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* fixed conflicts, pulled from master

* fixed code for passing tests successfully

* changed PersistentId to Jpa in MdmSearchInterceptor

* added the todo comments with questions

* added one more question

* renamed TODO questions

* first pass at converting to generic ResourcePersistentId

* fix test

* fix test

* fix test

* make ResourcePersistentId abstract

* Generic type MdmLink

* Final TODOs

* Final TODOs

* cleanup

* fix test

* fix id to string

* fix cyclic package dep

* add <?> to Base so we can extract interface

* Revert "add <?> to Base so we can extract interface"

This reverts commit a806f3d2bc.

* switch to interface

* javadoc

* replace base with interface

* fix test

* add generic template to mdmdao

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary MongoPid casts

* fix test

* fix regression

* fix compile issues arising from merging master

* fix beans

* bump version to 6.3.3-SNAPSHOT

* change logs

* review feedback

* review feedback

* fixme

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-12-09 09:30:51 -05:00
Nathan Doef d639a2a86d
meta.versionId Incremented After Update Operation With No Changes (DSTU 3) (#4349)
* Tests for R4 (passing) and DSTU3 (failing)

* Fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-12-08 20:26:49 -08:00
KGJ-software b8760cc022
add tests (#4340)
* add tests

* add tests

* clean up test file

* device test

Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-12-08 16:06:37 -08:00
Ken Stevens f4ce765122
IndexedSearchParamExtractor regression returned wrong number of links pre-persistence (#4341)
* fixed the issue

* change log

* fix regression

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-12-08 05:22:51 +00:00
Etienne Poirier f13b247c74
4324 excessive amount of time required to update large group resources (#4332)
* Initial test.

* Solution implementation with test adjustment.

* Adding changelog.

* Modifications following first code review.

* Moving changelog to 6.4.0

* Further reductions in query counts

* Fix for broken test

* Test fix

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2022-12-07 15:27:33 +00:00
James Agnew 312128754b
Allow forced id for conceptmap translation (#4304)
* Allow forced id for conceptmap translation

* Test fixes

* Test fix

* Conversion cleanup

* Test fixes

* Test fixes

* Build fixes

* Build fix

* Test fixes

* Test fixes
2022-11-30 05:31:26 -05:00
James Agnew 50ca94eded
Improve JPA response messages (#4293)
* Improve JPA response messages

* FIxes

* Test fixes

* Test fixes

* Ongoing testing

* Work on messages

* Work on messages

* Add valueset

* Add response code enum

* Version bump

* Undo bump

* Improve changelog

* Test fixes

* Add javadocs

* Version bump HAPI

* Test fixes

* Test fix

* Test fixes

* Test fixes

* Account for review changes

* Test fix

* Docs fix

* Work on API

* Improve SqlQuery api
2022-11-28 07:52:12 -05:00
James Agnew c3697a5f87
JPA Cleanup (#4278)
* JPA cleanup

* Test fixes

* Cleanup

* Resolve fixme

* Fix bad changelog entry

* A bit of documentation reworking

* Version bump

* POM bump

* Avoid timeout

* Build fix

* HTMLUnit bump
2022-11-24 14:32:56 -05:00
Nathan Doef 028c3ed5c1
Fix Expunge Deleted Resources Race Condition (#4285)
* failing test

* fix race condition

* change log

* refactor

* clean up

* remove ExpungeOptions hard limit enforcement

* nitpick

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-11-22 12:48:06 -05:00
karneet1212 62c630c546
bundle not throwing an error when multiple matches were found (#4283)
* Added test + condition to throw error

* changelog

* changes made based on comments
2022-11-21 10:08:10 -05:00
Tadgh b1e3468dcd
Rel 6 2 1 mergeback (#4288)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix up checkstyle

* log to console

* Remove double import

* Version bump

* licenses

* Fix up version

* fix build

* wip

* wip

* Fix up search expansion to use new behaviour of idhelper service

* Fix up MDM Search Expansion

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-11-20 08:14:28 -08:00
Ken Stevens d8d8c5fa44
Consolidate all System Properties into a single class (#4252)
* started converting System.setProperty to HapiSystem

* added a couple more.  renamed class

* all the setProperty calls done

* finished System.getProperty()

* Split HapiSystemProperties into HapiSystemProperties and HapiTestSystemProperties

* Javadoc

* Javadoc

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-11-14 18:51:03 +00:00
jmarchionatto 2df09c0cf3
Issue 4214 bulk export group observations missed when search parameter indexing enabled (#4241)
* Add failing test

* Add failing test for hsearch reference without type

* Modify cache to store forcedid type

* Disable failing test until finding out if it is a bug and strategy for fix in that case

* Remove fixmes and commented lines

* Update test. Add changelog.

* Added test that passes on this branch.

* Added Encounters to test case.

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-11-11 20:08:41 +00:00
James Agnew 9a45576793
Cache service loaders (#4196)
* Extracting a Cache interface and building service loaders for Caffeine and Guava

* Fix estimated size interface

* Use LoadingCache when needed.

* Removing Caffeine from dependency lists.

* Adding hapi-fhir-cache-caffeine as a test dependency

* Putting caching solutions in a single module.

* Fixing the spacing

* Standardizing the use of TimeUnits

* Making a new module to simplify the switch of the cache library in tests.

* Making sure the Guava design matches the behavior of Caffeine.

* Making sure the Cache structure also does not throw InvalidCacheLoading exception to match the LoadingCache.

* Renaming module names for the caching group.

* Better error handing that informs devs what to do.

* Improving documentation

* Typo

* Matching error message design with Caffeine.

* Matching the behavior of Caffeine with Guava

* Final adjustments for the test dependencies on the cache modules.

* Fixing relative pom path.

* Adding caffeine as a testing requirement for the new modules.

* Add changelog and set JPA server to use caffeine cache

* POM fixes

* Build fix

* Buid fix

* Fixes

* Address review comment

* One more cache

* Move changelog to next release

* Update pom versions

* Build fix

* Build fixes

* Build fix

* Try to get build working

* Experiment with failing build

* Rever change

* Fix POM version

* Build fix

* Build fix

* Add Msg.code to new exceptions

Co-authored-by: Vitor Pamplona <vitor@vitorpamplona.com>
2022-11-09 19:47:23 +00:00
James Agnew f50d350f1f
Post release HAPI FHIR cleanup (#4266)
* POST release deferred cleanup

* Fixes

* One more test fix

* Test fixes

* Test fixes

* Test cleanup

* Test fixes

* Fixes

* ValueSet cleanup

* Test fix

* Test fixes

* Fixes

* Test fixes

* Fixed

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixc

* Build fix

* Fix merge artifact

* Build fix

* Work on tests

* Test fixes

* Work

* Fixes

* Changelog fix

* Add changelog

* Test fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Tests

* Bumps

* Fixes

* Add errorprone

* Drop bz2 bins

* POM fix

* Build fix

* Update

* Test fix

* Fix tests

* Add changelog

* Test fix

* Avoid intermittent

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-09 13:28:00 -05:00
James Agnew b3ebbe7933
ValueSet cleanup (#4227)
* POST release deferred cleanup

* Fixes

* One more test fix

* Test fixes

* Test fixes

* Test cleanup

* Test fixes

* Fixes

* ValueSet cleanup

* Test fix

* Test fixes

* Fixes

* Test fixes

* Fixed

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixc

* Build fix

* Fix merge artifact

* Build fix

* Work on tests

* Test fixes

* Work

* Fixes

* Changelog fix

* Add changelog

* Test fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Tests

* Bumps

* Fixes

* Add errorprone

* Drop bz2 bins

* POM fix

* Build fix

* Update

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-08 22:18:36 -05:00
Tadgh d3367cfede
Rel 6 2 mergeback (#4257)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix

* Bump HAPI version

* Remove jetbrains

* Fix broken test

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-11-07 23:57:57 +00:00
jmarchionatto 6c9fe710ee
Issue 4180 process slows down to a crawl while loading uscore ig (#4201)
* Make query synchronous

* Use outer joins only for sorting.

* Adjust test

* Add changelog

* Fix changelog typo

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-11-07 19:20:28 +00:00
jmarchionatto 6cf5989d6c
Issue 4106 process map to properties when uploading loinc terminology (#4231)
* Add loinc upload handler to process MAP_TO properties.

* Move test to test folder

* Fix mapto file location property name

* Allow override as full loinc upload tests take (much) longer than defined timeout

* Move handler later to make sure all concepts were added to map

* Update test to check all project enhancements

* Restore project unrelated changes

* Rename base test class properly and move to right package

* Condition MapTo.csv file processing to file presence for backwards as file is optional

* Document and annotate test-only usage for disabling deferred tasks timeout

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-11-03 08:48:02 -04:00
samuelwlee2 1b55f49a60
4153 extend $member match operation to store consent resource if matched (#4165)
* added senstive and regular filter

* add UUID to Consent

* blah

* fix merge with master

* fixing merge

* blah

* fhir tests

* use constructor injection

* put all tests together

* blah

* blah

* finalize

* add consent update patient and performer reference

* fixes after code review

* blah

* refactor to save consent once

* blah

* fix performer and add validation

* code clean up

* add changelog

* add hrex member match flag

* fix PatientMemberMatchOperationR4Test

* adding code fixes after code review

* move changelog file

* bump version

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2022-10-31 13:33:54 -04:00
Tadgh 31a193fbd3
Bump to 6.3.0 (#4211) 2022-10-28 02:40:59 +00:00
James Agnew 7a39f8cbaf
Make reference extractor more reusable (#4205)
* Refactor

* Make reference extractor more reusable

* Test fix

* Test fix

* Headers

* Small refactor
2022-10-27 21:31:39 -04:00
karneet1212 0a160a35ba
4175-$validate-operation-on-a-resource-update-results-in-an-error-stating-resource-has-no-id (#4181)
* Added unit test + solution

* Changelog

* Cleanup

* changelog filename
2022-10-26 10:29:28 -07:00
jmarchionatto 3d9dfd4f08
Fix ForcedId criteria query (#4195)
* Fix ForcedId criteria query

* wip parameterizing tests

* Refactor almost all APIs to use ResourcePersistentId. Convert to fhir resource id when necessary (at search time)

* run tests in ANY id mode

* Test fix

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: jamesagnew <jamesagnew@gmail.com>
2022-10-25 13:56:08 +00:00
StevenXLi e1e33df38c
4190 offset count and total are wrong for some queries offset produces short pages and repeats results on different pages (#4193)
* created failing test

* implemented the fix, added more tests

* added change log

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-10-25 09:42:54 -04:00
James Agnew 5328951f15
Version Bump a few dependencies (#4191)
* Version bumps

* Test fixes

* Test fix

* Test fix
2022-10-24 15:20:39 -07:00
Ken Stevens a824ba263b
package loader npe (#4186)
* begin with failing test

* fixed

* change log

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-10-22 16:16:27 +00:00
michaelabuckley 929767a09b
Change to a minimal first pre-fetch to avoid latency on the async path. (#4174)
* Change to a minimal first pre-fetch to avoid latency on the async path.

The "async" path is now synchronous, and was causing latency for first page by loading 500 results.
We avoid pre-fetch on the first page, and only pre-fetch after users fetch the second page.

* Fix a nasty bug continuing a pre-fetched search.

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_2_0/3664-shorten-first-prefetch.yaml

* Patch up tests that assumed 500 pre-fetch first step
2022-10-21 20:45:30 +00:00
Tue Toft Nørgård 1dba9392ef
#4078 Fix BinaryAccessProvider empty stream check (#4170)
* Fix #4078 by using theRequestDetails.loadRequestContents() to check if the input stream is empty or not

* Remove unused imports
2022-10-21 13:08:21 -07:00
James Agnew a71b8fad56
Add R4B Support (#4150)
* Add r4b stuff

* Work on R4B

* Core version bump

* Revert some changes

* Add R4B

* Docs

* Test fixes

* Work on R4B

* Tests passgin

* Add config

* Work on merging

* Work

* Work

* Work

* Test fix

* Fixes

* Test fixes

* Test fix

* Work

* Work

* Fixes

* Test fixes

* Fixes

* Test fix

* Test fix

* Test fix

* Version bump

* Test fix

* Test fix

* ompile fixes

* Fix

* Resolve build warnings

* Updates

* Test fixes

* Fix docs
2022-10-20 13:57:06 -07:00
Luke deGruchy c8312662d7
Set batch export job results count to number of resources processed instead of 0. (#4172)
* Add count to batch export admin page results and new add new unit test.

* Fix failing unit test in WriteBinaryStepTest.

* Add changelog.

* Small changelog fix.
2022-10-20 14:27:01 -04:00
JasonRoberts-smile 5a31ce0386
disable batch job reuse (#4155) 2022-10-19 11:43:51 -04:00
karneet1212 a6ed393969
4140-upload-terminology-does-not-allow-snomed-ct-canada-edition-zip-file-to-load-due-to-a-different-file-name (#4144)
* Test + Implementation of the Solution

* Changes made based on comments
2022-10-19 08:33:21 -07:00
KGJ-software 69d55b7184
3052 Changed IBinaryStorageSvc MaximumBinarySize from int to long. (#4123)
* 3052 Changed IBinaryStorageSvc MaximumBinarySize from int to long.

* 3052 Added changelog

* 3052 Bumped version

* 3052 Remove mysterious +1 on maximum binary size causing overflow.

* Bumping version to 6.2.0-PRE16-SNAPSHOT

* Adding + 1L as original implementation.

* Getting all tests to pass dealing with + 1L.

* Getting all tests to pass dealing with + 1L.

* Adding migration definition and removing unnecessary comment.

* Preventing register overflow when setting maxBinaryStorage.

* Mering relevant test.

* Fixing typo in migration script

Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-10-19 06:09:02 -04:00
samuelwlee2 dac266b719
Extend $member match to validate matched patient (#4136)
* added validation for family name and birthdate

* making changes to tests

* update patient identifer return and validatePatientMember

* update patient identifer type for test

* update method name

* update test comment
2022-10-18 12:19:03 -06:00
James Agnew 26ef32c1dd
Fix 2 issues with has reverse chaining (#4139)
* Fix 2 issues with chains

* Add changelog

* Cleanup

* Add comment

* Cleanup
2022-10-14 16:46:18 -04:00
James Agnew 422ef87a5d
Standardize on one Transactional annotation (#4130)
* Standardize on one Transactional annotation

* Add changelog

* Disable checkstyle rule

* Fix up transactions

* Test fixes

* Test cleanup

* License headers

* One more dep bump

* Resolve commit issue

* Test fixes

* Test fix

* Dep bump

* Fix

* Fix test

* Version bump

* Test fixes
2022-10-13 16:26:11 -04:00
MykolaMedynskyiSCDR 2e6b3d16d6
4017 creating a searchparameter with an expression referencing metasecurity will return error hapi 1118 (#4037)
* Adding initial test reproducing the issue.

* some comments and small refactoring to run pipeline to see the extent of the damage that this can cause.

* Parsing dstu3 expression with FHIRpath parser

* fixed code.

* omitting searchParameter expression(xpath) validation for DSTU2

* Added test for search parameter with many expressions

* changed deprecated method.

* removed and modified custom search parameters tests for DSTU3 and DSTU2.

* fixed code

* Added change log

* Test testIndexFailsIfInvalidSearchParameterExists() is back.

* Update createResourceSearchParameter_withExpressionMetaSecurity_succeeds() test

* Update createResourceSearchParameter_withExpressionMetaSecurity_succeeds() test

* Refactoring if statement and exception message.

* Refactoring tests and rewording changelog.

* Fixing tests.

* Fixing more tests.

* Addressing comments from first code review.

* Fixing issue number.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-10-13 11:05:04 -04:00
Etienne Poirier 4cebcfb766
4115 Combining 'indexMissingFields' and 'advancedHSearchIndexing' functionalities will prevent creating a resource (#4122)
* Added Test

* Added Integration Test

* Made changes based on comments

* Implemented solution

* Implemented solution

* Debug

* Changelog

* Changes made based on comments

* Changelog

* Made changes based on comments

* Made changes based on comments

Co-authored-by: Karneet Kaur <karneet.kaur@smilecdr.com>
2022-10-12 16:16:44 +00:00
Tadgh b74e12172e
4098 Resultset missing resources when querying with _sort result parameter (#4099)
* Add broken test

* Finished writing test

* Add in parameterized test to show it works with one

* Adding extra test asserting sort results when providing short chain

* Providing fix to search with sorting.4098-sort-missing-parameter

* Adding ChangeLog.

* Improving, cleaning up and commenting tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-10-12 08:06:46 -04:00
James Agnew 108e357ca7
Preserve tags on delete (#4128)
* Preserve tags on delete

* Add changelog

* Add changelog

* Test fix

* Build fix

* Address review comments
2022-10-12 11:01:47 +00:00
Luke deGruchy 5831cbedf5
Concurrent cascade delete operations leading to confusing errors (#4126)
* Add unit tests and supporting classes from gitlab issue.

* Commit with changes thus far to be reviewed.   Namely, add savepoint capabilities to HapiTransactionService, inject HapiTransactionService into CascadingDeleteInterceptor, add savepoint capabilities to the TransactionDetails, and catch an Exception in BaseHapiFhirResourceDao.delete() then rollback to the checkpoint.

* Fix compile error.

* hapi transaction service test

* hapi transaction service test

* hapi transaction service test

* First draft of RetryTemplate and flush() logic.  Try unit testing with PointcutLatch.

* hapi transaction service test

* Latest changes including moving RetryTemplate, cleaning up unit tests.

* First attempt at correct design.  Fix compile errors.

* Add failed rollback savepoint test.

* fix propogation

* interesting test failure in SafeDeleterTest

* Latest code

* move delete

* test passing

* all tests pass

* yay we're done just cleanup

* First round of cleanup:  Get rid of all exploratory logging, remove unneeded dependencies, move creation of RetryTemplate to SafeDeleter.

* More cleanup.  Add changelog.

* Bump to 6.2.0-PRE13-SNAPSHOT.

* Code review feedback:  Refactor SafeDeleter to a service and bean.  Add comments.

* Code review feedback:  Comments on unit test assertions.

* Code review feedback:  Rename SafeDeleterSvc to ThreadSafeResourceDeleterSvc.  Add javadoc.  Add logger to DelayListener.   Remove useless commented out code.

* Fix unit test failure in CascadingDeleteInterceptorTest.

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-10-11 16:48:01 -04:00
James Agnew 26fe5e6b8e
Interceptor improvements (#4125)
* Interceptor improvements

* Add changelogs
2022-10-09 15:02:12 -04:00
KGJ-software c5e1effe5b
3240 Added test case. Currently does not reproduce bug. (#4108)
* 3240 Added test case. Currently does not reproduce bug.

* 3552 Fixed test case.

* 3552 Added StrictErrorHandler validation before applying patch.

* 3552 Use logger instead of sysout.

* 3552 Use try blocks.

* 3552 Better test name.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-10-07 09:50:49 -04:00
Ken Stevens 40163f73d2
Ks 20220916 remove flyway (#4040)
* removed flyway, started coding persistence

* started wiring up new dao service

* replace jpa with sqlbuilder

* down to 16 failed tests

* remove outOfOrder test.  We no longer support validating order.

* Removed noFlyway mode.  It is no longer a thing.

* merge migrator

* down to 1 failing test

* cleanup

* removed unused noflyway cli arguments

* cleanup

* cleanup

* cleanup

* cleanup

* create MigrationTaskList to replace the List<BaseTask> that gets passed around a lot.  This allows us to add a bunch of behaviour to that as a first-order thing

* fix test

* fix test

* improve migration table detection to use the catalogue

* Msg.code()

* cleanup

* save version in same format as flyway

* cleanup

* add migration result to migration result class

* failed migrations stay in the table and don't really affect anything.

* change log

* docs

* ugh Flyway requires lower-case column names

* integration test fixes

* log

* fix oracle issue

* lowercase tablenames for postgres.  fussy fussy!

* Revert "lowercase tablenames for postgres.  fussy fussy!"

This reverts commit 92c9cb263b.

* revert to capital table names.  surround table name by quotes for Postgres.

* added log message before executing find all query

* changed log message from info to debug

* javadoc

* pom cleanup

* review feedback

* Update hapi-fhir-sql-migrate/src/main/java/ca/uhn/fhir/jpa/migrate/MigrationTaskList.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* review feedback

* review feedback.  Use prepared statement.

* Change installedOn insert from timestamp to date

* Change installedOn column type from timestamp to date

* bump hapi version

* fix poms

* fix poms and cdr compile

* add exclusions to clean up build

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2022-10-03 22:40:09 -04:00
weiping202209 66baa8263d
Fix bug in history search (#4009)
* add failing test

* Fix history search bug and add tests for two more scenarios

* Fix history search bug and add tests for two more scenarios

* - added failing test condition

* Fix history search bug and add tests for two more scenarios

* Fix build test failures

* Fix build test failures

* Add extra verifications for scenarios of _since

* Refactor dao.history interface to add overload method and shall come back to remove old method once cdr is refactored.

* Update with new hapi interface

* Add depreciate comment

* Add depreciate comment

* Version bump to mvn 6.2.0-PRE11-SNAPSHOT

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-10-03 09:00:41 -04:00
Ken Stevens 26756a40bb
add message key to all message types (#4101)
* add message key to all message types

* move test

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-10-01 15:36:04 +00:00
Ken Stevens e9a855d3f6
remove spring batch (#4064)
* remove spring batch
it compiles
expect test failures

* fix test transaction manager bean name (wow that was quite the hunt the wumpus!)

* fix test and fix build

* fix test and fix build

* fix test

* remove spring batch schema initialization

* bump hapi version

* fix schema migrator

* bump to PRE-10

* merge master

* change logs

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-30 13:18:48 -04:00
Tadgh b472c4bb54
4083 batch permissions (#4095)
* Move threadpool factory into a bean

* Move threadpool factory into a bean

* change return type

* Add license files

* Version bump

* Fix test
2022-09-29 17:24:05 +00:00
James Agnew fc82ed4e44
One more tweak to #4093 (#4094)
* One more tweak to #4093

* Add changelog
2022-09-28 16:46:00 +00:00
James Agnew e4a2285f30
Make sure to not mix resource types in bulk export (#4093)
* Make sure to not mix resource types in bulk export

* Avoid dupes

* Add test
2022-09-28 15:16:15 +00:00
StevenXLi 9e1120a47a
4080 support onc requirements for group bulk export (#4081)
* added failing tests

* added include to bulk export processor, added support for different resource types in bulk export fetch id step

* added changelog

* fixed duplication issue, and added null check

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-09-27 14:51:43 +00:00
Tadgh f06f9dc90a
4084 error segment of bulk export job poll status is missing (#4085)
* changelog

* Change serialization rules

* Always populate error field even if no errors
2022-09-27 01:16:33 +00:00
michaelabuckley 96b92153f6
Implement composite search parameters in HibernateSearch for JPA (#3839)
Implement composite search parameters in Lucene/Elasticsearch Hibernate Search indexing

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-09-27 00:20:06 +00:00
Luke deGruchy 3435806df5
4058: Fix JpaPersistedResourceValidationSupport to search for resources that it currently doesn't handle (#4062)
* Add failing and passing tests with bundle JSONs for each scenario based on pared down versions of the following JSON bundle: https://smilecdr.com/docs/fhir_repository/clinical_quality_language/measure-bundle-example.json.html

* Add experimental code to handle cases of "Library" and "Measure". Add a measure report to measure unit test with new JSON with a MeasureReport to Measure resource (can't make it pass yet).

* Refine case statements for JpaPersistedResourceValidationSupport.  MeasureReport test passes if -ea is omitted from run configuration.

* Add changelog.  Remove TODOs from validator.

* Remove erroneous whitespace chance.

* Ken feedback:  Handle SearchParameters from the default case casting a wider net than just Library and Resource.  Assume all CanonicalType inputs to the switch statement have a URL.

* Get rid of TODO since it's not relevant.

* Disable test with only MeasureReport and Measure.  Fix case issue with filenames that was only discovered by running the github pipeline.

* Add more context to the Disabled text, including a reference to a new issue:  https://github.com/hapifhir/org.hl7.fhir.core/issues/930.

* Add more context to the Disabled text, including a reference to a new issue:  https://github.com/hapifhir/org.hl7.fhir.core/issues/930.
2022-09-26 10:08:04 -04:00
TipzCM 389bdce372
fixed bug (#4069)
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-09-26 13:43:11 +00:00
James Agnew 917cf8d062
Remove ActionRequestDetails (#3527)
* Add test

* Add test

* Start removing unneeded things

* Fixes

* Add changelog

* Test fix

* Address LGTM issues

* Resolve fixme

* Address review comment

* Add changelog

* Update doc

* Test fixes
2022-09-25 21:29:08 +00:00
James Agnew 3bab3544ec
Reduce SQL queries for _include (#4007)
* Rework includes processing for reuse

* Optimize include

* Test fixes

* Bump version down

* Fixed

* Update

* Test fixes

* Start to add javadoc

* Test fixes
2022-09-25 15:22:06 -04:00
Tadgh 6f80206776
Fix transactional issue in everything operation (#4063)
* Fix transactional issue in everything operation

* Add license files

* Add changelog
2022-09-23 08:33:08 -07:00
Tadgh 5a978646c5
Add changelog, test, and implementation (#4055) 2022-09-21 21:18:12 -07:00
Tadgh 392bfb790f
Fix content negotiation on binary streaming (#4053)
* Fix content negotiation on binary streaming

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_2_0/4051-content-type-negotiation-binaries.yaml

Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>

* Tidy logic

* Tidy logic

* Naming

* Wip

Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
2022-09-21 19:05:35 +00:00
jdar8 dbe52b429c
fix-matchers-not-contains-in-asserts (#3913)
* test

* replace not(contains())

* make a util class to replace not(hasItems()) and not(containsInAnyOrder())

* replace not(hasItems()) and not(containsInAnyOrder())

* add javadocs to new util class

* test for util class

* Addressing comments from code review.

* Adding static import.

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-09-21 11:54:27 -04:00
StevenXLi 3b781022c1
4030 bulk export to support fhir basepatientid$export endpoint (#4032)
* Refactored test helper method to be more generic, added new failing test for feature, and changed BulkDataExportOptions to fix compile error

* added field to all objects needed in the bulk export pipeline for adding patient id, added new operation for the feature

* added changelog

* added support for patient parameter to patient export, changed patient id to a list to match the requirements for the patient parameter, added test for multiple id patient export

* refactored methods, added new changes to changelog

* code review changes

* fix test error

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-09-20 10:42:55 -07:00
Tadgh d1f988f98a
Skip resource updating completely if found in package load. (#4036)
* add ability to skip loading package resources that already exists, to avoid reindex when loading IGs

* Add a test for validating no updates or creates are called on anything but binaries

* Add changelog

Co-authored-by: Craig McClendon <craig.mcclendon@accenture.com>
2022-09-20 08:45:37 -07:00
MykolaMedynskyiSCDR 14253dd0e3
3960 when creating a resource patient where patientmetasource is provided with a hash the hash value is overwritten (#3965)
* Adding failing test.

* What was done:
-Meta source id is not replaced if provided.
-Added change log.

* Fixed code

* More tests for meta source

* Overwriting meta source can be configured.

* Test naming convention uplift and renaming of overwrite flag along with setter/getter

* Fixed when the source did not return if there was no resourceID

* Test naming convention uplift and renaming of overwrite flag along with setter/getter

* WIP in getting search on _source to be returning the correct number of patients.

* Custom requestId now stored on its own column

* Simplified getRequestId method in BaseHapiFhirDao

* Simplified getRequestId method in BaseHapiFhirDao

* An unused variable has been removed

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-09-19 10:17:09 -04:00
Tadgh c3339f38c3
Rel 6 1 2 mergeback (#4033)
* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Tidy versionenum

* Fix import

* Remove files that were repackaged

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
2022-09-15 22:18:32 +00:00
TipzCM ae3faafc9a
change search method for missing parameter names (#3982)
* handle changing in missing parameters

* added changelog

* refactor step1

* refactor step 2

* refactor step 3

* refactor step 3

* more code review fixes

* tra lala

* tra lala

* blah

* more changes

* review fixes

* review point

* more review points

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-09-14 09:29:16 -04:00
James Agnew 6c0c730ebc
ForcedOffsetSearchModeInterceptor should not modify the search when it is already manually marked as synchronous (#4016)
* ForcedOffsetSearchModeInterceptor should not modify the search when it
is already manually marked as synchronous

* Add changelog
2022-09-12 10:48:56 +00:00
Tadgh 4226cae752
Broken bulk export (#4012)
* Implement test

* wip

* Fixed a bug in bulk export

* add changelog

* typo

* Switch out to debug logs
2022-09-09 23:18:30 +00:00
Tadgh 6d9a19997f
Rel 6 1 1 mergeback (#4004)
* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Fix bad mergback

* Added a couple of missing imports to a unit test.

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Kevin SmileCDR <kevinsmilecdr@Kevins-MacBook-Pro-2.local>
2022-09-08 10:38:42 -04:00
jmarchionatto 002a7dc670
Issue 3419 elastic search exception while pre expanding valueset with more than 10 000 concepts (#3991)
* Add test reproducing bug

* Refactor ValueSet expansion hsearch queries to use scroll.

* Use void method for never used return value.

* Execute action when transaction is active

* Update tests query counts as the CodeSystemVersion.pid is always queried now

* Enable test after reproduced bug fix.
Add changelog.

* Implement suggestions

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-09-07 15:52:38 -04:00
Nathan Doef 9b50b332a4
:nickname Qualifier Support for Custom SearchParameters (#3969)
* - add failing test

* 4977 Allow any search parameter to have a nickname qualifier.

* IT test to ensure nickname expansion is working on custom SearchParameters

* capture log output in tests

* changelog

* code review changes

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-09-07 08:52:37 -07:00
jdar8 c647154777
Everything operation with type param code review fixes (#3926)
* check for valid resource type

* add test and refactor test class

* refactor auto get method name

* introduce parameter object to patient everything methods

* introduce parameter object to patient everything methods

* change record to class

* add hapi error code

* refactor

* refactor

* change code num

* refactor getmethodname to util method

* add docs and rename fields to conform to fhir standard

* typo

* version bump

* resolving merge conflicts.

* Bumping version

* Bumping version to 6.2.0-PRE5-SNAPSHOT

* Encapsulating parameters in a PatientEverythingParameters object.

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-09-07 06:44:36 -04:00
Qingyixia f3e6837fce
Fixed the issue of bulk export returning the wrong path if "Fixed Value for Endpoint Base URL" is set (#3958)
* Fixed the issue of bulk export returning the wrong path if "Fixed Value for Endpoint Base URL" is set

* Minor modifications according to the suggestions

* Modify the msg code number

* Modify the tests to be parameterized test

* Modifications according to review suggestions

* Update the msg code
2022-09-06 08:40:03 -07:00
Ken Stevens 100b8f9190
batch2 rework (#3964)
* consolidated Batch2Helper
check status in assert before running maintenance
ignore illegal job instance state transition

* Fix mock tests

* rename intermittent to IT

* extra logging

* into to debug

* review feedback

* added more logging

* removed fasttracking.  replaced with support for triggering a maintenance run

* fixed tests

* fixed tests

* add asserts

* licenses

* snakeyaml cve version bump

* data migration

* merge origin/master

* merge origin/master

* fix test (support triggering jobs when scheduling is disabled)

* add counter to abort potential infinite loop after 100000 calls

* compile error

* move multipartition mdm tests out to a separate IT
update mongo job instance creation with new logic

* fix race condition

* back out infinite loop circuit breaker.  it breaks terminology tests that depend on this loop running > 100000 times.

* fix test (triggered job key was missing group)

* revert saveAllDeferred

* Almost finished getting JpaPersistenceR4IT to pass

* testBulkExportDeleteForOnGoingJob still not passing

* testBulkExportDeleteForOnGoingJob still not passing

* Removed method `fetchInstanceAndMarkInProgress`.  It was clobbering other state transitions like CANCELLED.

* undo rename of cancelled to cancelRequested.  Thymeleaf depends on the old name

* ExpandResourcesStep requires a synchronous search

* Tests are passing but man testBulkExportDeleteForOnGoingJob is a flakey test

* fix test

* fix test

* change log

* allow ERRORED -> FAILED state transition

* add a timeout to saveAllDeferred()

* improve error message on timeout message

* fix tests

* added more logging

* added more logging

* fix semaphore issue

* Msg.code

* fix test

* javadoc

* switch all batch loggers to batch troubleshooting log

* add unit test, improve logging

* move TimeoutManager to its own class.  Add tests.

* fix tests

* comment cleanup

* test race condition

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-05 00:04:54 -04:00
StevenXLi a4a7fd9344
3987 group bulk export does not apply typefilter searches to members of the group (#3988)
* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-09-02 13:53:52 -07:00
Tadgh 9fd0addec6
Add test, changelog, and implementation (#3981) 2022-09-01 18:00:48 +00:00
Ken Stevens 68c343902a
jpa test split (#3959)
* create fhir-version specific hapi-fhir-jpaserver-test projects

* moved dstu2 tests out to dstu2 test project.

* moved dstu2 tests out to dstu2 test project.

* finished moving dstu2 tests out to dstu2 test project.

* finished moving dstu2 tests out to dstu2 test project.

* dstu3 test project compiles

* start moving dstu3 resources

* letting azure pileine know about the new modules

* finished moving dstu3 resources.  this might pass

* OK this one should pass for sure

* OK this one should pass for sure

* change test poms to test scope

* move the rest of R4 test classes

* move the rest of R4 test classes

* moar r4 moves

* moar r4 moves

* moar r4 moves

* finished with r4

* finished with r4

* started r5

* started r5

* Cache busting based on source branch name

* Fix other cache key

* cache pegged to SHa1

* test cleanup

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-09-01 02:46:17 +00:00