Commit Graph

66 Commits

Author SHA1 Message Date
michaelabuckley 8813d9beda
Batch stabilization (#4647)
* Use java event names for work chunk transitions.

* Cherry-pick d5ebd1f667 from rel_6_4

Avoid fetching work-chunk data (#4622)

* add end time to reduction step (#4640)

* add end time to reduction step

* add changelog

---------

Co-authored-by: Long Ma <long@smilecdr.com>
(cherry picked from commit 37f5e59ffc)

* Cancel processing

Provide error message in cancelled jobs, and avoid transitions in final states.

* Apply tx boundary to starting job and first chunk.

* cleanup

* Apply tx boundary to work chunk processing

* Delete BatchWorkChunk

* Introduce events for job create, and chunk dequeue

* Apply tx boundary to chunk handler

* Move instance cancellation to database

* tx boundary around stats collection and completion

* tx boundary around stats collection and completion

* Extend tx boundary to error, fail, and cancel

* Move failure into status calc

* ERROR is not an "ended" state.

* Revert generics cleanup to avoid noise

* Avoid sending gated chunks twice.

* Make no-data path safer.  Cleanup

* Fix mock test for step advance.

* Delete unsafe updateInstace() call

* Cleanup

* Changelog and notes

* Fix cancel boundary.  Cleanups

* Cleanup

* Sort mongo chunks for stable paging.

Other cleanup

* Document error handling

* Cleanup

* Update hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/Batch2JobHelper.java

Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>

---------

Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
2023-04-25 15:47:23 -07:00
StevenXLi 3fb9a16975
4693 enable partitioning in bulk import (#4694)
* created failing tests

* implemented feature, added more tests

* added documentation and changelog

* fixed duplicate error code

* code review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-03-31 09:43:40 -04:00
James Agnew bcc1ca7593
Transaction SQL Optimization (#4679)
* Start work optimizing transaction

* Tons of test cleanup

* Cleanup

* More optimization

* Optimize

* Many tests fixed

* Work on test fixes

* Optimization done, now doing cleanup

* Cleanup

* Add docs

* Test fixes

* Test fix

* License headers

* Test fix

* Test cleanup

* Test fix
2023-03-24 10:29:56 -04:00
michaelabuckley 4b3405a456
Use java event names for work chunk transitions. (#4658)
Change work-chunk creation to be first event in state machine.
2023-03-21 13:53:16 -04:00
Isaac Wen 14be0553d0
Moving all license comments to the first line (#4656)
* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
2023-03-20 10:04:26 -04:00
Ken Stevens 8b80da6441
re-enabled some disabled tests (#4644)
* re-enabled some disabled tests

* add logging

* improve logging

* removed nonsense test

* fix regression

* fix %now

* licenses

* disable searchparam reindexing on intermittent failing test

* refining fix to RetryingMessageHandlerWrapper

* licenses

* remove hapi-fhir-cli-jpaserver from azure pipelines

* undo unneeded change

* change log

* back out unneccessary changes

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-03-17 13:04:29 -04:00
James Agnew cf5470ae58
Search and Sort on Uplifted Refchains (#4633)
* Start work on refchains

* Semi working

* Add tests for transactions

* Add docs and lots of tests

* Add changelog

* Add tests

* Work on cleanup

* Add document operations

* Test fix

* Test fix

* Fixes

* Fix typo

* Test fix

* Test update

* Test updates

* Test fix

* Test fixes

* Test additions

* Test fix

* Add some javadocs

* Test fixes

* Intermittent test fix

* Doc tweak

* Test fixes

* Merge master in

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Review comments

* Version bump

* Add license

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-03-15 07:04:32 -04:00
Aditya Dave 8956b273f0
Concurrent Transaction Conditonal Creates (#4639)
* push broken test for testing placeholder create in transaction retry

* reproduce error for transaction retry

* cleanup test

* fix test testPlaceholderCreateTransactionRetry

* revert temp fix

* revert spacing

* Add tests

* Remove fixme

* Test fix

* test fix

* Test fix

* Remove test line

* Remove unneeded change

* Review comments

* Add docs

* Test logging

* Add test logging

* Add logging for intermittents

---------

Co-authored-by: aditya_dave <aditya@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-03-14 16:39:55 -04:00
James Agnew 59123cf0f0
Fix intermittent in batch2 tests (#4643)
* Fix intermittent in batch2 tests

* Add changelog

* Test fix

* Resolve intermittent

* Reducer changes

* Test fix

* Test fixes

* Resolve fixme

* Fix changelog
2023-03-13 19:59:05 +00:00
michaelabuckley d1f9c94038
Make WorkChunk handling transactional at state transitions. (#4621)
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
2023-03-12 19:19:21 -04:00
James Agnew ae1d249d99
Bulk export reducer step fix mergeback (#4606)
* One more fix for #4467

* Enabling massIngestionMode causes incomplete resource deletion (#4476)

* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)

* Adding initial test.

* Adding initial solution implementation.

* Adding change log and code clean up.

* addressing comments from 1st code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Change visibility of migration method (#4471)

* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)

* First commit:  Make SubscriptionValidatingInterceptor aware of which Pointcut is being called.  In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED.   Fix resulting compile errors in various unit tests.

* Fix/enhance unit tests.  Mark methods as deprecated instead of deleting them.  Add proper error code.  Complete changelog.

* Remove erroneous TODOs and tweak the validation logic.

* Enhance unit tests and fix changelog.

* Reindex batch job fails when processing deleted resources. (#4482)

* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* cleaning up checkstyle files (#4470)

* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Bump core to 5.6.881 (#4496)

* Bump core to 5.6.881-SNAPSHOT

* Work on fixing tests

* Work on fixing tests 2

* Bump to core release

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>

* Issue 4486 mdm inconsistent possible match score values (#4487)

* Extract method for readability

* Save always normalized score values in POSSIBLE_MATCH links.

* Avoid setting properties to null values. Adjust test.

* Simplify fix

* Fix test. Add RangeTestHelper.

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Revert "cleaning up checkstyle files (#4470)"

This reverts commit efae3b5d5f.

* core version fix

* Loosen rules for id helper

* License

* fix batch2 reduction step (#4499)

* fix bug where FINALIZE jobs are not cancellable

* moved reduction step to message hander

* moving reduction step to queue

* addingchangelog

* cleaning up

* review fixes

* review fix'

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Scheduled batch 2 bulk export job and binary delete (#4492)

* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.

* Change bulk import test for valueUri type (#4503)

* change tests

* suggested test change

* CVE resolutions (#4513)

* Bump Postgres for CVE

* Bump jetty

* Verison bump

* Remove comments

* Revrt bump

* Add check in scanner (#4518)

* 4516 create hapi fhir cli command to clear stale lock entries (#4517)

* Initial implementation

* better tests

* Add changelog and docs

* Forgotten files

* Code review comments

* Fix checkstyle

* Unable to Expunge CodeSystem (#4507)

* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>

* New line::

* Update to documentation regarding narrative generation; (#4521)

Providing changelog;

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* changed what score is set for mdmlinks that created new golden resource (#4514)

* changed what score is set for mdmlinks that created new golden resource

* fix test

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* REVERT: change to operationoutcome.html

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* added change log

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fix build (#4530)

* Making narrative_generation.md reference an html snippet (#4531)

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* 4441 rel 6 4 bad references creation bug (#4519)

* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fixed channel import null pointer exception from null header (#4534)

* retryCount 0 on null header + test + changelog

* suggested changes

* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)

This reverts commit 53252b8d15.

* Better error handling for when channel type is not supported (#4538)

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Avoid logging message payloads that contain sensitive data (#4537)

Don't log payloads - they may contain sensitive data.

* Bulk Export Bug With Many Resources and Low Max File Size (#4506)

* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd953.

* Revert "maintenance pass"

This reverts commit bbc7418d51.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149.

* testing

* Revert "testing"

This reverts commit aafc95c2f3.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* bump ver

* License updates'

* Downgrade dep'

* Updating version to: 6.4.1 post release.

* Add javadocs and sources to our serviceloaders

* Reset version

* Change parent

* Remove bumped version

* License fixes, new parent

* Updating version to: 6.4.1 post release.

* Fix bad creation of versionenum

* Improve performance on bulk export

* Add changelog

* Start working on cleaned up reducer

* Clean up batch calls

* Work on issues

* Build fixes

* typedbundleprovider getallresources override (#4552)

* typedbundleprovider getallresources override

* added test to immunization tests that validates pagination does not result in null pointer if over default queryCount

* moved HapiFhirDal test to its own test class

* removed unused imports

* Update to use JpaStorageSettings

* adding changelog for issue 4551

* fix changelog spacing

* changelog type  to fix

---------

Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>

* Add backport info

* Upgrade core to 5.6.97, make adjustments in hapi-fhir, and ensure that all tests pass (#4579)

* First commit:  Create new branch from the release branch with changes from James' branch.  This probably won't compile as the work is incomplete.

* Second round of changes from integrating James' branch.

* Mark most test failures with TODOs.

* Add whitespace

* Add changes to FhirPathR4 to set FHIRPathEngine to non-strict FP evaluation.

* Fix CreatePackageCommandTest to assert null instead of empty string.  Comments on tests that fail due to the double-quote encoding problem.

* Downgrade to core 5.6.97.

* Fix another test and remove TODOs.

* Fix changelog.

* Clean up some erroneous changes and TODOs.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Fix up dal test

* Address leftover code review feedback from the upgrade to core 5.6.97. (#4585)

* Exclude pinned core deps

* Force pin structs

* Add model changes to IBaseCoding and related changes (#4587)

* Add IbaseCoding changes, and tinder changes

* Fix up tag definition

* converter addition

* Fix unit test and add changelog.

* Add jira to changelog.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Fix changelog

* Tidy metadata

* Test fixing

* Test fixes

* Work on progress

* Add changelog

* Build tweak

* Fixes

* Disable intermittently failing tests. (#4593)

* rename tests to IT

* Disable more intermittently failing tests (#4595)

* Disable more intermittently failing tests.

* Disable another intermittently failing tests.

* ITify

* Disable yet another intermittently failing tests. (#4596)

* disable

* disables

* Fixes

* disables

* Fix compile

* Test fixes

* Updating version to: 6.4.2 post release.

* Bump to 6.4.2-SNAPSHOT

* Fix compile

* fix version

* Address review comments

* Review comments

* Version bump

* Compile fix

* Test fixes

* Compile fixes

* One more compile fix

* Test fixes

* Version bump

* Resolve fixme

---------

Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
2023-03-01 14:52:36 -05:00
James Agnew eea6193df9
JPA config refactoring (#4545)
* Interceptor rework

* Work on config refactor

* Fixes

* Docs tweaks

* Address review comments

* Test tweak

* Test fixes

* Try to fix tests

* Test tweaks
2023-02-15 08:43:01 -05:00
Ken Stevens 794e4510c6
6_4 mergeback (#4547)
* One more fix for #4467

* Enabling massIngestionMode causes incomplete resource deletion (#4476)

* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)

* Adding initial test.

* Adding initial solution implementation.

* Adding change log and code clean up.

* addressing comments from 1st code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Change visibility of migration method (#4471)

* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)

* First commit:  Make SubscriptionValidatingInterceptor aware of which Pointcut is being called.  In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED.   Fix resulting compile errors in various unit tests.

* Fix/enhance unit tests.  Mark methods as deprecated instead of deleting them.  Add proper error code.  Complete changelog.

* Remove erroneous TODOs and tweak the validation logic.

* Enhance unit tests and fix changelog.

* Reindex batch job fails when processing deleted resources. (#4482)

* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* cleaning up checkstyle files (#4470)

* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Bump core to 5.6.881 (#4496)

* Bump core to 5.6.881-SNAPSHOT

* Work on fixing tests

* Work on fixing tests 2

* Bump to core release

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>

* Issue 4486 mdm inconsistent possible match score values (#4487)

* Extract method for readability

* Save always normalized score values in POSSIBLE_MATCH links.

* Avoid setting properties to null values. Adjust test.

* Simplify fix

* Fix test. Add RangeTestHelper.

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Revert "cleaning up checkstyle files (#4470)"

This reverts commit efae3b5d5f.

* core version fix

* Loosen rules for id helper

* License

* fix batch2 reduction step (#4499)

* fix bug where FINALIZE jobs are not cancellable

* moved reduction step to message hander

* moving reduction step to queue

* addingchangelog

* cleaning up

* review fixes

* review fix'

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Scheduled batch 2 bulk export job and binary delete (#4492)

* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.

* Change bulk import test for valueUri type (#4503)

* change tests

* suggested test change

* CVE resolutions (#4513)

* Bump Postgres for CVE

* Bump jetty

* Verison bump

* Remove comments

* Revrt bump

* Add check in scanner (#4518)

* 4516 create hapi fhir cli command to clear stale lock entries (#4517)

* Initial implementation

* better tests

* Add changelog and docs

* Forgotten files

* Code review comments

* Fix checkstyle

* Unable to Expunge CodeSystem (#4507)

* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>

* New line::

* Update to documentation regarding narrative generation; (#4521)

Providing changelog;

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* changed what score is set for mdmlinks that created new golden resource (#4514)

* changed what score is set for mdmlinks that created new golden resource

* fix test

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* REVERT: change to operationoutcome.html

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* added change log

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fix build (#4530)

* Making narrative_generation.md reference an html snippet (#4531)

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* 4441 rel 6 4 bad references creation bug (#4519)

* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fixed channel import null pointer exception from null header (#4534)

* retryCount 0 on null header + test + changelog

* suggested changes

* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)

This reverts commit 53252b8d15.

* Better error handling for when channel type is not supported (#4538)

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Avoid logging message payloads that contain sensitive data (#4537)

Don't log payloads - they may contain sensitive data.

* Bulk Export Bug With Many Resources and Low Max File Size (#4506)

* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd953.

* Revert "maintenance pass"

This reverts commit bbc7418d51.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149.

* testing

* Revert "testing"

This reverts commit aafc95c2f3.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* bump ver

* License updates'

* Remove checkstyle

* Verison bump

* Move migrations

* semicolon

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-02-13 13:23:25 -08:00
Tadgh 456cc81b32
Rel 6 2 5 mergeback (#4459)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix wrong pinned version

* Updating version to: 6.2.2 post release.

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https… (#4291)

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https://github.com/hapifhir/hapi-fhir/issues/4276

* Credit for #4291

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* backport and changelog for 6.2.2

* Updating version to: 6.2.3 post release.

* fix https://simpaticois.atlassian.net/browse/SMILE-5781

* Version bump to 6.2.3-SNAPSHOT

* Auto retry on MDM Clear conflicts (#4398)

* Auto-retry mdm-clear on conflict

* Add changelog

* Build fix

* Disable failing test

* Update to 6.2.3 again

* Update license dates

* Dont fail on batch2 double delivery (#4400)

* Don't fail on Batch2 double delivery

* Add changelog

* Update docker for release ppipeline

* Updating version to: 6.2.4 post release.

* Add test and implementation to fix potential NPE in pre-show resources (#4388)

* Add test and implementation to fix potential NPE in pre-show resources

* add test

* WIP getting identical test scenario

* More robust solution

* Finalize Code

* Add changelog, move a bunch of changelogs

* Remove not needed test

* Minor refactor and reporting

* Fix up megeback

* update backport info

* update backport info

* Updating version to: 6.2.5 post release.

* Prevent chunk from returning to in-progress unless it is errorred, in-progress, or queued

* changelog

* Update logger

* Indicate backport

* Add version and upgrade info. add backport

* hapi-fhir side done

* cdr side

* add assert

* add test

* Fix import

* prevent multiple reduction step runs in maintenance run (#4423)

* setting job to inprog to prevent multiple maintenance passes from running reduction steps

* added a new status

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Add backport info

* Fix up tests

* Fix discrepancies

* Fix test

* Version bump

* bump ips

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-01-24 17:38:35 -08:00
James Agnew d584e14048
Add IPS Generator (#4438)
* Begin IPS refactor

* Commit work so far

* Fix test bug

* Fix typo

* Fix typo

* Narrative generator cleanup

* Narrative generator working

* Add test for bad reference in narrative

* Tests passing

* Start docs

* Tests passing

* Cleanup

* Update cyangelog

* Doc tweaks

* Version bump

* Address review comments

* Address review comments

* Build fix

* Cleanup

* Compile fix

* Test fix

* Test fix

* Version bump

* Build update

* Test fix

* Test fix

* Add one utility method

* Add doc
2023-01-24 12:07:36 -05:00
Ken Stevens 5dda16d467
Reduce postconstruct use (#4445) 2023-01-21 00:37:58 -05:00
Ken Stevens bf53a08b83
Batch2 fastracking configurable (#4453)
* licenses

* hapi-fhir side done

* cdr side

* add assert

* add test

* bump hapi version

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-20 19:03:13 -05:00
Ken Stevens 04c28b6c08
Schedule jobs via IJobScheduler instead of in @PostConstruct (#4415)
* parameterizing BinaryStorageInterceptor

* parameterizing BinaryStorageInterceptor

* changed jobs scheduling from @PostConstruct to coordinated by HapiLifecycleService events

* fix test

* remove events for now

* fix test

* simplify design

* FIXME

* changelog

* fix compile issue

* review feedback.  partially implemented

* review feedback

* review feedback

* bump hapi version to 6.3.8-SNAPSHOT

* fix test

* mirror contextclosedevent handlers with contextrefreshedevent

* version bump

* licenses

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-01-14 23:51:04 -05:00
michaelabuckley 21b1820294
Demote PHI logging to debug or lower (#4430)
Ensure resource bodies are only logged at debug or lower.

* Drop possible-phi logs to debug.
* Demote request/response log bodies to debug since they may contain PHI.
* Demote job-param logging to debug.
* Add startup hook to App for test injection.
* Change terminology cli logging to avoid logging whole resource.
2023-01-13 15:34:29 +00:00
Qingyixia a9ecc18986
Fixed $export-poll-status null point exception issue (#4428)
* Tests + Fix

* Addressing suggestions

* Addressing suggestions

* Minor fix
2023-01-13 08:59:44 -05:00
Tadgh 65bf8d47ce
Prevent chunk from returning to in-progress unless it is errorred, in… (#4417)
* Prevent chunk from returning to in-progress unless it is errorred, in-progress, or queued

* changelog

* Update logger

* Add new test

* Add test
2023-01-10 03:36:06 +00:00
Tadgh d70d813249
Rel 6 2 4 mergeback (#4406)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix wrong pinned version

* Updating version to: 6.2.2 post release.

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https… (#4291)

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https://github.com/hapifhir/hapi-fhir/issues/4276

* Credit for #4291

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* backport and changelog for 6.2.2

* Updating version to: 6.2.3 post release.

* fix https://simpaticois.atlassian.net/browse/SMILE-5781

* Version bump to 6.2.3-SNAPSHOT

* Auto retry on MDM Clear conflicts (#4398)

* Auto-retry mdm-clear on conflict

* Add changelog

* Build fix

* Disable failing test

* Update to 6.2.3 again

* Update license dates

* Dont fail on batch2 double delivery (#4400)

* Don't fail on Batch2 double delivery

* Add changelog

* Update docker for release ppipeline

* Updating version to: 6.2.4 post release.

* Add test and implementation to fix potential NPE in pre-show resources (#4388)

* Add test and implementation to fix potential NPE in pre-show resources

* add test

* WIP getting identical test scenario

* More robust solution

* Finalize Code

* Add changelog, move a bunch of changelogs

* Remove not needed test

* Minor refactor and reporting

* Fix up megeback

* update backport info

* update backport info

* Updating version to: 6.2.5 post release.

* please

* fix test

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
2023-01-06 03:45:22 +00:00
Tadgh f2f29a1a32 Add forgotten license changes 2023-01-05 17:31:43 -08:00
kateryna-mironova 017cc4e29c
4244 - Deprecate usages of ResourcePersitentId in jpa (#4245)
* init rev

* id to jpa and mongo id

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* 3444 - replaced methods and constructors of ResourcePersistentId with child constructors and methods

* fixed conflicts, pulled from master

* fixed code for passing tests successfully

* changed PersistentId to Jpa in MdmSearchInterceptor

* added the todo comments with questions

* added one more question

* renamed TODO questions

* first pass at converting to generic ResourcePersistentId

* fix test

* fix test

* fix test

* make ResourcePersistentId abstract

* Generic type MdmLink

* Final TODOs

* Final TODOs

* cleanup

* fix test

* fix id to string

* fix cyclic package dep

* add <?> to Base so we can extract interface

* Revert "add <?> to Base so we can extract interface"

This reverts commit a806f3d2bc.

* switch to interface

* javadoc

* replace base with interface

* fix test

* add generic template to mdmdao

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary JpaPid casts

* remove unneccessary MongoPid casts

* fix test

* fix regression

* fix compile issues arising from merging master

* fix beans

* bump version to 6.3.3-SNAPSHOT

* change logs

* review feedback

* review feedback

* fixme

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-12-09 09:30:51 -05:00
Tadgh b1e3468dcd
Rel 6 2 1 mergeback (#4288)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix up checkstyle

* log to console

* Remove double import

* Version bump

* licenses

* Fix up version

* fix build

* wip

* wip

* Fix up search expansion to use new behaviour of idhelper service

* Fix up MDM Search Expansion

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2022-11-20 08:14:28 -08:00
Ken Stevens d8d8c5fa44
Consolidate all System Properties into a single class (#4252)
* started converting System.setProperty to HapiSystem

* added a couple more.  renamed class

* all the setProperty calls done

* finished System.getProperty()

* Split HapiSystemProperties into HapiSystemProperties and HapiTestSystemProperties

* Javadoc

* Javadoc

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-11-14 18:51:03 +00:00
James Agnew b3ebbe7933
ValueSet cleanup (#4227)
* POST release deferred cleanup

* Fixes

* One more test fix

* Test fixes

* Test fixes

* Test cleanup

* Test fixes

* Fixes

* ValueSet cleanup

* Test fix

* Test fixes

* Fixes

* Test fixes

* Fixed

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixc

* Build fix

* Fix merge artifact

* Build fix

* Work on tests

* Test fixes

* Work

* Fixes

* Changelog fix

* Add changelog

* Test fix

* Test fixes

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Tests

* Bumps

* Fixes

* Add errorprone

* Drop bz2 bins

* POM fix

* Build fix

* Update

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-08 22:18:36 -05:00
Tadgh d3367cfede
Rel 6 2 mergeback (#4257)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix

* Bump HAPI version

* Remove jetbrains

* Fix broken test

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
2022-11-07 23:57:57 +00:00
James Agnew 5533afb9f5
Allow Batch2 transition from ERRORED to COMPLETE (#4242) (#4243)
* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

Co-authored-by: James Agnew <james@jamess-mbp.lan>
2022-11-04 09:06:22 -04:00
Tadgh 78ce2a6344
4182 convert $mdm submit to a batch job (#4188)
* Changelog

* Wip implementation

* Wip

* wip

* wip

* Fix validator

* Fix up implementation

* tidfy

* TIdy up tests

* Add some docs

* Add changelog info:

* Tidy up self review

* License files

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Review changes

* fixes

* add new method

* Update tests

* Add batch size parameter

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-10-25 12:02:04 -07:00
James Agnew cbd65c794d
Add remove method to jobregistry (#4187)
* Add remove method to jobregistry

* Add changelog
2022-10-24 17:07:32 -04:00
James Agnew 422ef87a5d
Standardize on one Transactional annotation (#4130)
* Standardize on one Transactional annotation

* Add changelog

* Disable checkstyle rule

* Fix up transactions

* Test fixes

* Test cleanup

* License headers

* One more dep bump

* Resolve commit issue

* Test fixes

* Test fix

* Dep bump

* Fix

* Fix test

* Version bump

* Test fixes
2022-10-13 16:26:11 -04:00
Tadgh c3339f38c3
Rel 6 1 2 mergeback (#4033)
* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Tidy versionenum

* Fix import

* Remove files that were repackaged

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
2022-09-15 22:18:32 +00:00
Tadgh 4226cae752
Broken bulk export (#4012)
* Implement test

* wip

* Fixed a bug in bulk export

* add changelog

* typo

* Switch out to debug logs
2022-09-09 23:18:30 +00:00
Ken Stevens 100b8f9190
batch2 rework (#3964)
* consolidated Batch2Helper
check status in assert before running maintenance
ignore illegal job instance state transition

* Fix mock tests

* rename intermittent to IT

* extra logging

* into to debug

* review feedback

* added more logging

* removed fasttracking.  replaced with support for triggering a maintenance run

* fixed tests

* fixed tests

* add asserts

* licenses

* snakeyaml cve version bump

* data migration

* merge origin/master

* merge origin/master

* fix test (support triggering jobs when scheduling is disabled)

* add counter to abort potential infinite loop after 100000 calls

* compile error

* move multipartition mdm tests out to a separate IT
update mongo job instance creation with new logic

* fix race condition

* back out infinite loop circuit breaker.  it breaks terminology tests that depend on this loop running > 100000 times.

* fix test (triggered job key was missing group)

* revert saveAllDeferred

* Almost finished getting JpaPersistenceR4IT to pass

* testBulkExportDeleteForOnGoingJob still not passing

* testBulkExportDeleteForOnGoingJob still not passing

* Removed method `fetchInstanceAndMarkInProgress`.  It was clobbering other state transitions like CANCELLED.

* undo rename of cancelled to cancelRequested.  Thymeleaf depends on the old name

* ExpandResourcesStep requires a synchronous search

* Tests are passing but man testBulkExportDeleteForOnGoingJob is a flakey test

* fix test

* fix test

* change log

* allow ERRORED -> FAILED state transition

* add a timeout to saveAllDeferred()

* improve error message on timeout message

* fix tests

* added more logging

* added more logging

* fix semaphore issue

* Msg.code

* fix test

* javadoc

* switch all batch loggers to batch troubleshooting log

* add unit test, improve logging

* move TimeoutManager to its own class.  Add tests.

* fix tests

* comment cleanup

* test race condition

* review feedback

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-09-05 00:04:54 -04:00
Tadgh b9caec1c42
Rel 6 0 mb 3 (#3933)
* adding version.yaml, updating version in pom.xml

* It is possible to write to a resource in a partition the user is not authorized to. (#3397)

* fixed

* remove sout

* add msg.code

* fix failed tests

* fix equal sign

* update msg code

* extract method

* Fix up code numbers

* Clean changelog

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Revert final artifact version

* add graphql test (#3585)

* added graphql birthdate test

* fix variable name

* typo

* 3506 mdm log enhancement (#3543)

* Providing Fixme's to be reworked at a later time.

* Adding // FIXME Anna to assist our Austrian friend.

* Adding logging test as a first step in addressing issue #2822.

* hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

* Update MdmMatchLinkSvc.java

#3506 move "narrowed down" log to different place

* #3506 added scores and tests

#3506 added scores and tests

* #3506 formatting

#3506 formatting

* #3506 create changelog file

#3506 create changelog file

* #3506 fix typo

#3506 fix typo

* #3506 fix part 3, minor formatting

#3506 fix part 3, minor formatting

* #3506 fix tests

#3506 fix tests

Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>

* mdm matching (#3579)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* begin with failing test

* fixed

* changelog

* add jira tag

* Update to 6 1 (#3582)

* added changelog folder, upped version

* version enum

* add a few more unit tests to assert proper NO_MATCH exclusion

* revert merge master doh! bad reflexes

* revert merge origin master

* unrevert revert.  ugh what a pain

* merge recovery.  fix poms.

* merge recovery.  more reverting

* merge recovery.  more reverting

* merge recovery.  more reverting

* Revert "merge recovery.  fix poms."

This reverts commit ae6e0ddb06.

* more revert revert reversions

* more revert revert reversions

* pre-review cleanup

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>

* Fix regression of 3411 - _lastUpdated gets clobbered during $reindex job (#3586)

* When chunking for the reindex job, don't clobber the lastUpdated if provided by the caller.

* License

* Ks 20220508 log colour (#3592)

* don't use colours when output is redirected to a file

* change log

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551) (#3594)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* 3584 case sensitive string elasticsearch (#3596)

* Fix for case-sensitive search with newer elasticsearch

* Test and fix ascii normalization too

* more test cases

* comments

* Docs update empi usecase (#3598)

* Update diagram for use case 5

* Add new page for database support

* Licnese

* Mb norm fix (#3604)

* Normalize query since wildcard searches aren't normalized in elastic

* Handle batch2 job cancellation (#3603)

* Handle batch2 job cancellation

* Add CANCELLED status

* Remove unuseful test

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Add batch2 interfaces to obtain recent instances (#3601)

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* changelog folder

* Documentation change, added warning log and change log for ticket (#3609)

Co-authored-by: Steven Li <steven@smilecdr.com>

* fix ne for lastUpdated search param (#3589)

* init/wip

* add implementation, add tests

* add checks and some refactoring

* refactor and simplify tests

* refactor

* get rid of some of the warnings

* add changelog

* remove my todos

* redo ne logic to account for less precise dates

* add tests

* refactoring

* rename to follow convention

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>

* Version bump

* Add skip of checkstyle to upload

* Typo

* upping version enum, upping version number, 6.0.1

* Group Bulk Export rules exiting too early.  (#3643)

* Implementation, test, changelog

* Add jira link

* adding backport in yaml for 3642

* adding folder for 6.0.1 release

* pom changes to publish checkstyle to sonatype

* bad version in pom file

* Make FulltextSearchSvcImpl optional bean for ValuesetOperationProvider

* AUtomcplete error code

* updating comment indicating final erorr code

* Bump error code

* fixing version enum so release docs generate correctly, adding additional changelog for 3650

* 3684 subscription for delivering delete events formatted wrong (#3685)

* Fixed issue with delivering delete events Subscriptions

* Extracted deliverer class name to a constant, now shared across tests and interceptor class

* Added change log and doc changes

* moved the constant to a more appropriate place

Co-authored-by: Steven Li <steven@smilecdr.com>

* Version bump

* Add version enum

* Version enum and folder

* adding release pipeline yml file fo azure

* Adding autoclose for sonatype publishing (#3797)

* Add impl test and changelog (#3713)

* fix up changelog, new version

* Version bump

* Updating version to: 6.0.4 post release.

* reverting 6.0.4 commit

* Updating version to: 6.0.4 post release.

* removing double entry from preious release

* Add support for DSTU3 delete subscrion events, add test

* Add changelogP

* Add version enum

* Remove notnull

* Add test for DSTU2

* Updating version to: 6.0.5 post release.

* Fix up

* wip

* Adding solution and changeLog file.

* Adding failing test.

* fix up

* Version bump

Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: katiesmilecdr <88786813+katiesmilecdr@users.noreply.github.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: alackerbauer <33912849+alackerbauer@users.noreply.github.com>
Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
2022-08-30 02:29:02 +00:00
Tadgh e7a4c49aac
Rel 6 1 mergeback (#3927)
* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Bump Version for non-breaking purposes

* Fix broken test

* bump error code

* revert internal logic

* Revert offset mode change

* wip

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-08-18 17:21:27 -07:00
michaelabuckley 60a879c842
Renable elasticsearch tests (#3850)
* Renable elasticsearch tests

Limit elasticsearch container to 512MB instead of the default 4GB.
2022-08-07 07:09:12 +00:00
TipzCM eae1c0ffb8
3846 migrate bulk import pull to batch2 (#3853)
* still fixing tests

* migrated a bulk import pull job t obatch 2

* changelog

* cleanup

* review fixes

* minor tweaks

* cleaningup

* tra la la

* blah

* blah

* updating msg

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-08-03 10:50:51 -04:00
longma1 f933ff3d9c
2878 mdm mongo support (#3748)
* small refactor

* refactored MDM module to use iterface IMDMLink, refactored MDMLink to use ResourcePersistenceId wrapper rather than pid as long

* removing IJpaIdHelperService and refactor IIdHelperService

* fixed bugs for GoldenResourceMergerSvcImpl and FindCandidateByExampleSvc

* changes to IIdHelperService

* move MdmSearchParamSvc and MdmSubmitSvcImpl out of jpaserver-mdm

* moved JpaResourceLoader to hapi-fhir-storage, renamed to ResourceLoaderImpl

* added error Msg code, removed Fixme

* fixed error code, update last used code in Msg.java to 2083

* fixed accidentally throwing unprocessableentityException

* added MdmLinkExpandSvc interface

* build adding license to files

* MdmLinkdao refactor

* fix build

* mdm config refactoring

* MdmStorageInterceptor and IExpungeEverythingService changes

* fix bug where a pid is converted to ResourcePersistenceId twice

* minor refactor to IMdmLink, change id from long to ResourcePersistenceId

* changed PageImpl to Page interface

* refactor MdmSearchExpandingInterceptor to use interface

* fixed missed refactor where the resourcePersistenceId is still assumed to be long

* resolve test failures resulting from merging master

* changed conflicting exception code

* fixed issue with bulk export wrapping resourcePid inside resourcePersistenceId

* added IMdmLinkImpl to MdmSettings

* fixed missing import in IdHelperService from merging master

* fixed MdmLinkDaoSvcTest importing wrong MdmLinkTuple

* restored JpaIdHelperService and added changelog

* modified msg.code to non-duplicate

* code review changes

* refactored some bulk export batch 2 files to accomodate change in beans being exported

* fixed jpa bulk export failing tests

* removed duplicate bean declaration of MdmLinkImplFactory in BaseMdmR4Test

* merged master and bump version to pre25

* extracted expungeEverythingByType to IExpungeEverythingService

* fixed wiring issue

* messed up merging pom with master

* fixed broken changelog

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Samuel Lee <samuel.lee@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-29 08:58:40 -06:00
TipzCM 9a1d6f3172
3821 term code system batch jobs (#3826)
* step one

* blah

* no working test

* updated existing tests

* delete code system version and code system jobs

* trying to get terminology fixed

* updating tests

* need to have some movement

* added completion handler

* added clearing code for failsafe

* review points

* fixing tests

* fixing tests

* more test fixes

* fixing more tests

* review fixes

* todo

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-07-27 14:50:42 -04:00
TipzCM 59540ce803
3813 paging job coordinator (#3814)
* added a paging api to ijobcoordinator

* adding changelog

* moved location

* added a comment

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-07-26 11:29:37 -06:00
TipzCM 8ed0a18ef4
3531 implement bulk processors (#3566)
* base batch2 bulk export steps - unimplemented

* added the basic steps

* adding tests

* we have added the tests

* added tests for bulk export stages

* implementing jpa

* 3531 adding the jpabukexportprocessorimplementation

* 3531 first step for mongo bulk export

* jpa stuff updated

* 3531 hapi fhir stuff

* trying to fix up tests

* fixing test

* fixed most tests in batch export

* deprecate

* need some more guidance here

* fixing some tests

* some steps taken

* blah

* stashing

* test

* fixing tests round 1

* many a changes

* updating version

* small updates

* merging in mster again

* fixing up serialization

* added the permission checkers

* cleanup and sonarkube crap

* some more tests

* updating adding tests

* crimes

* blah

* updatingmsg number again

* removal of commented code

* cleanup

* merge confflicts

* fixing msg

* review fixes round 1

* more review fixes

* merging in master

* fixing poms

* review points

* test fix

* test fix

* fix test race condition

* truncate error

* add stack trace to error

* rename to IT

* fix race condition in test

* finally fixed race condition in test.  this time I'm optimistic hapi ci will finally pass

* fix mock test.  not a fan of mock tests like this.

* any job definition with a reducer step must be gated.

* fixed common batch bean wiring

* fix test

* lazy init ResponseTerminologyTranslationSvc

* change the way we lazy init ResponseTerminologyTranslationSvc

* change the way we lazy init ResponseTerminologyTranslationSvc (for the third time.  this is super tricky!)

* disabling testGoldenResourceWithCircularReferenceCanBeCleared for now

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-17 17:49:03 -04:00
TipzCM 0c644271ce
3770 fix batch2 failure loop (#3779)
* 3770 fixing error handling

* some updates to error handling

* changelog

* blah

* bumping version

* fix test failures

* code review points

* review points

* add retry test

* assert counts in the test

* test fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-15 12:42:46 -04:00
Ken Stevens 0526da080b
3757 move delete expunge to batch2 (#3759)
* begin with failing test

* begin with failing test

* remove unused Spring Batch classes.
convert service classes

* move provider

* just one test left to fix

* fix test

* msg.code

* change log

* bumping hapi to PRE12

* fix test

* review feedback

* license update

* fix intermittent and status update atomicity

* restore url-level partition selection

* fix tests

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-09 17:25:55 -04:00
James Agnew 61d34ab695
Add license headers (#3765) 2022-07-07 17:22:38 -04:00
Ken Stevens 3a28920ea7
Add error-handler callback and ensure batch2 completion handler can't be called twice (#3752)
* rename field that drives everything includes

* discovered design flaw in completion

* fix intermittent

* ensure batch2 completion handler can't be called twice

* changelog

* changelog

* javadoc

* bump hapi version

* licences

* Batch2 job definitions can now optionally provide an error handler callback that will be called when a job
instance fails, errors or is cancelled.

* Changed error handler and completion handler to take a "read-only" copy of an instance instead of just certain fields.

* comment

* javadoc

* fix test

* change step execution context from taking instance id to instance

* review feedback

* fix test

* replace 20 or so jetbrains annotations

* fixed a few incorrect @NotNull annotations

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-04 23:27:58 -04:00
Ken Stevens 069f20a5f7
batch2 management operations api (#3742)
* first test passes.  onto the next one

* two down two to go

* three down one to go

* licenses

* basic admin-json plumbing confirmed with IT

* FIXME

* FIXME

* FIXME

* FIXME

* changelog

* bump hapi version to 6.1.0-PRE6-SNAPSHOT

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 23:02:17 -04:00
Ken Stevens c3f34099fb
only register mdm clear job definition if it is not registered yet (#3740)
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-26 14:10:42 -04:00
Ken Stevens 49d298dba1
mdm clear move to batch2 (#3671)
* begin with failing test

* continue building out

* continue building out

* reworking chunk api

* replace ChunkId zippers with proper iterator

* progressing test

* yay test passes!

* switch to idchunk interface

* fixmes

* compiles

* fixme

* fixme

* rename classes

* rename json classes with json suffix

* work towards homogeneous chunks

* fixme

* fixme

* unit test

* fixme

* fixme

* pull out builder

* moar tests

* moar tests

* preserve order

* moar test

* moar test

* fixme

* fixme -> wip to start ci

* msg code

* msg code fix test

* fixme

* change provider implementation

* final WIPs

* IT failing

* IT passes

* java 11

* fix test

* fix test

* fix test

* changelog

* use batch size if provided

* last WIP

* fix test

* fix test

* fix test

* whitespace change to trigger new build

* fix test

* bumping hapi version

* bumping hapi version

* fix test

* review feedback

Co-authored-by: Ken Stevens <ken@Kens-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-25 20:26:44 -04:00