Commit Graph

102 Commits

Author SHA1 Message Date
Ken Stevens 7e9faf84b8 change is unit tested 2024-07-10 17:52:40 -04:00
Martha Mitran ecef72729f
Add support for running the reindex job across resources in multiple partitions and all partitions (#6009)
* Cleanup and fix compile warnings

* Cleanup and enhance batch2 jobs to support running them across multiple partitions.

* Add some documentation and fix build in the pipeline.

* Fix tests

* Add support for RequestPartitionId.allPartitions as input to the IJobPartitionProvider. Add tests and fix one test. Add missing changelog.

* Separate basic partition provider logic so it can be reused by Mongo. Update documentation.

* Revert change to apply certain validation only for creates.

* Remove added logs

* spotless fix

* Address code review comments
2024-07-05 22:23:07 -07:00
Emre Dincturk 48c387ecb0
added batch job related attributes to OpenTelemetry span (#5972) 2024-06-06 11:07:03 -07:00
TipzCM 5302a7da52
updating field to make it nullable (#5982)
* updating field to make it nullable

* review fixes and test fixes

* review fixes

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2024-06-06 09:35:29 -04:00
Ken Stevens cdf5b602c1
Replace hamcrest with assertj (#5911)
* optimize imports

* optimize imports

* Hamcrest replacement

* Hamcrest replacement

* Test failures

* migrate hapi-fhir-base

* docs

* Few more cutovers

* Full rewrite of all assert cases

* Full rewrite of all assert cases

* Partial cutover

* assert library conversion

* wip

* fix imports

* remove equalto

* once-over again

* minor cutover

* wip

* wip

* wip

* wip

* wip

* Remove hamcrest asserts replace with assertj

* wip

* Partial hamcrest rip-out

* More hamcrest stripping

* wip

* Partial completion

* spotless

* it compiles now

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fixing compile issues

* fix compile issues

* fix compile issues

* hapi-fhir now compiles

* fix test

* fix test

* fix test

* fix tests

* fix tests

* fix tests

* fix test

* fix test

* fix tests

* fix tests

* fix tests

* yay all tests pass now!

* revert assertEquals change

* revert assertEquals change

* organize imports

* organize imports

* organize imports

* post merge cleanup

* organize imports

* more cleanup

* more cleanup

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* removing hamcrest stragglers

* organize imports

* organize imports

* remove final hamcrest stuff

* remove final hamcrest usages

* remove final hamcrest usages

* Revert "remove final hamcrest usages"

This reverts commit eaf5107396.

* fix compile issues from a hacky search/replace

* compiles with fixmes

* fix tests

* fix test

* fix test

* fix post merge compile issues

* fix post merge compile issues

* fix post merge compile issues

* fix post merge compile issues

* fix post merge compile issues

* fix post merge test issues

* fix test

* fix test

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* replace hamcrest with assertj

* all but mdm tests

* final hamcrest removed from hapi-fhir yay!

* revert mdm changes. it's too hard for now

* revert cdr breaking change

* revert cdr breaking changes

* revert cdr breaking changes

* revert cdr breaking changes

* revert cdr breaking changes

* revert changes that break cdr

* merge master with compile errors

* review feedback

* review feedback

* review feedback: revert bad regexp change

* fix test

* fix test

* convert new hamcrest entries

* convert new hamcrest entries

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2024-06-05 16:02:46 -04:00
TipzCM f199a3ee4e
added workchunk to step execution details (#5932)
* added workchunk to step execution details

* versino bump

* added deprecated constructor

* spotless

* review fixes

* spotless

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2024-05-26 21:59:17 -04:00
TipzCM ae67e7b55e
Batch2 workchunk states hapi (#5851)
* step 1

* updated batch 2 framework with READY state

* spotless

* remove entity manager

* spotless

* fixing up more tests for batch2

* updating documentation

* cleanup

* removing checkstyle violation

* code review points

* review points continued

* review poitns finished

* updating tests

* updates

* spotless

* updated

* step 1

* updated

* sketch out test cases

* basic state transition shell work

* typos

* spotless

* adding spy override

* fixing tests

* spotless

* changing comment to complete build

* fixing some tests and adding a view

* adding different paging mechanism

* spotless

* waiting step 1

* commit changes

* remove text

* review fixes

* spotless

* some tweaks

* updating documentation and adding change log

* spotless

* added documentation

* review comments 1

* more review fixes

* spotless

* fixing bug

* fixing path

* spotless

* update state diagram

* review points round 1

* revert

* updating diag

* review fixes round 2

* spotless

* - Implemented GATE_WAITING state for the batch2 state machine.
- This will be the initial status for all workchunks of a gated job.
- made compatible with the equivalent "fake QUEUED" state in the Old batch2 implementation.
- Updated corresponding docs.
- added corresponding tests and changelog

* Revert "- Implemented GATE_WAITING state for the batch2 state machine."

This reverts commit 32a00f4b81.

* - Implemented GATE_WAITING state for the batch2 state machine.
- This will be the initial status for all workchunks of a gated job.
- made compatible with the equivalent "fake QUEUED" state in the Old batch2 implementation.
- Updated corresponding docs.
- added corresponding tests and changelog

* fixing a bug

* spotless

* fixing

* - fix merges conflicts
- set first chunk to be always created in READY

* - have only one path through the equeueReady method
- fixed tests

* - hid the over-powered transition function behind a proper state action

* spotless

* resolved review comments

* fixing tests

* resolved review comments

* resolved review comments

* resolved review comments

* resolved review comments

* resolved review comments

* updating migration script number

* fixed bugs

* spotless

* fix test high concurrency

* fixing a test

* code fix

* fixing tests in bulkexportit

* fixing tests

* fixing tests

* cleanup

* completed instance will not be sent to the reduction step service

* Revert "completed instance will not be sent to the reduction step service"

This reverts commit aa149b6691.

* Revert "Revert "completed instance will not be sent to the reduction step service""

This reverts commit e18f5796a1.

* removing dead code

* changed db query for step advance to take statuses as parameter instead

* test fixes

* spotless

* test fix

* spotless

* fixing tests

* migration fix

* fixing test

* testing pipeline with `testGroupBulkExportNotInGroup_DoesNotShowUp` disabled

* fixing some tests

* Add new race test for simultaneous queue/dequeue

* re-enabling `testGroupBulkExportNotInGroup_DoesNotShowUp`

* cascade tag deletes

* test fixes

* some logging

* a test case

* adding job id

* more test code

* marking purge checks

* test fix

* testing

* pausing schedulers on cleanup

* adding a wait

* max thread count guarantee

* fixing the tests again

* removing dead code

* spotless

* checking

* msg codes:

* Fixing a test

* review points

* spotless

* required pom values

* step 1 of reduction ready

* update

* reductoin ready

* annother test

* spotless

* cleanup

* cleanup

* simplifying check in reduction step

* review fixes

* updating version

* using 7.3.1

* adding check

* test finessing

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: tyner <tyner.guo@smilecdr.com>
2024-05-07 18:23:03 -04:00
TipzCM 107de2cfba
reduction step failing fails jobs (#5831)
* adding a catch for failing reduction step

* spotless

* added changelog

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2024-04-08 13:51:41 -04:00
TipzCM 5d55594a73
Batch2 test refactor only (#5823)
* refactoring

* test refactor only

* spotless

* bumping version

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2024-04-05 21:39:03 -04:00
TynerGjs df047a9dea
Resolve 5758-completion-handler-for-reduction-steps-are-not-called-with-the-up-to-date-job-instance-status (#5759)
* updated instance status after run is called.

* added changelog

* version bump
2024-03-06 11:01:42 -05:00
TipzCM 370d2c16b1
5746 fixing reducer step (#5753)
* fixing reduction step to fire job completion handler

* unwinding

* fixing changelog

* spotless

* review fixes

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2024-03-04 09:12:31 -05:00
Martha Mitran e5d410d10b
Refactor and fix functionality dealing with ReadPartitionIdRequestDetails (#5691)
* Throw exception when resourceType of parameter value is invalid when performing FHIR search with non-chained reference

* Fix changelog issue id and spotless warning

* Cleanup unnecessary test

* Validate resourceType only when using relative reference

* Fix test for search using _type parameter

* Refactor and fix functionality dealing with ReadPartitionIdRequestDetails

* Spotless fixes

* Revert some of the refactoring changes to BaseRequestPartitionHelperSvc and fix tests

* Small cleanup

* Missing change

* Small changes

* Fix tests and add a few more

* Spotless fix

* Address code review comments

* Some polishing on the interface annotations and documentation

* Spotless fix

* Fix test

* Fix tests and test method rename

* Ensure SearchParameterMap is not null

---------

Co-authored-by: Martha Mitran <martha.mitran@smilecdr.com>
2024-02-17 16:16:14 -05:00
longma1 993cfcce99
7 0 0 mb (#5678)
* pom bump to 7.0.0 (#5615)

* pom bump to 7.0.0

* add version enum

* fixed feb release name

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* Check index existence on raw SQL creation (#5624)

* Add conditional check on index existence before we try again

* Add conditional check on index existence before we try again

* Changelog

* remit

* remit

* debug statements

* 5621 deadlock on caffeine cache when creating a resource with conditional create (#5622)

* Modifying the CacheProvider to avoid doing db I/O within the cache miss value supplier callback.

* Setting the initial capacity of instantiated caches to the cache max size to avoid resizing during operations.

* adding changelog and spotless.

* Fixing typo.

* Addressing comments from code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Searching with more than one chained Bundle SearchParameter returns incorrect results (#5614)

* Failing test

* fix

* changelog

* Avoiding Exception being thrown on @EventListener invocation (#5628)

* replaced EventListener annotation with @PreDestroy

* adding changelog

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* simple fix (#5630)

Co-authored-by: Long Ma <long@smilecdr.com>

* Incorrect version of auto versioned reference for conditional update with urn id placeholder (#5625)

* Incorrect version from versioned_references.auto_at_paths for conditional update - implementation

* Oracle:  Ensure migrated database still takes large resource text updates (#5629)

* First pass at fix to Oracle HFJ_RES_VER.RES_TEXT_VC migration.

* First stab at agreed upon solution.

* Fix error with 4001 by removing unnecessary annotation.

* Spotless and TODO.

* Remove annotation for good and set length to LONG32.

* Fix copyright year.

* Finalize changelog.

* Remove migration changes.  Fix unit test.

* Fix compile error.

* Log output.

* Refactor resource history code into new ResourceHistoryCalculator.

* Spotless.

* Convert record to POJO.

* Restore pre-17 switch statement.

* Finalize new resource history calculator code and tests.

* Spotless.

* Remove logging.

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/7_0_0/5633-oracle-hfj-res-ver-clob-migration.yaml

Apply code reviewer suggestion

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>

* Code review feedback.

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>

* Fix expansion for `ValueSet` with no concepts based on CodeSystem `urn:ietf:bcp:13` (#5638)

* When fetching the mimetype code system, return empty CodeSystem with NOTSUPPORTED content. Update expansion logic to handle this case. Add some test cases.

* Minor change to fix test

* Rename changelog file

* Remove TODOs as they're replaced by reported issue

* Revert accidental change added with TODO removal

* $expunge operation ignoring ExpungeThreadCount setting in certain cases (#5637)

* $expunge operation ignoring ExpungeThreadCount setting in certain cases - implementation

* Fix Measure group id null pointer exception (#5620)

* Update to new version of clinical reasoning, includes a fix for null pointers on Measure group id

* add changelog

---------

Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>

* Rule evaluation:  Allow Bundles with PATCH Parameters (#5641)

* Remove parameters clause.

* Finalize changelog.  Add tests.  Finalize implementation.

* Undo changes to this test.

* Revert all changes to FhirQueryRuleImplTest.  Add new RuleImplOpTest.  Ensure that proper nested Bundle is created for test and other fixes.

* Tweak test.

* Use real rule applier in test and remove all mocks.

* Prevent batch2 job execution to stop for empty chunk when last job st… (#5635)

* Prevent batch2 job execution to stop for empty chunk when last job step is a reduction.
Add output to bulk export result even when empty.

* Fix test

* Unimportant change to force fresh build

* Implement review suggestions

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Index review fixes (#5649)

* Don't hold locks while adding indices during upgrade.
* concurrent indexing is non-transactional in PG.

* Fix conditional creates without leading '?' (#5646)

* First commit with failing test.

* More tests and logs.

* More logs

* Try new solution for BaseTransactionProcessor.performIdSubstitutionsInMatchUrl().

* Simplify solution.  Add more tests.

* Changelog.

* javadoc

* Searching for Bundles with read all Bundles permission returns 403 (#5644)

* failing test

* another failing test case

* fix

* changelog

* fix bug

* spotless

* cr

* Fix NullPointerException when performing a system bulk export in the presence of PatientIdPartitionInterceptor. (#5660)

* Serializing changes for sensitive data (#5655)

* Add new senstiive data serializer

* Add new senstiive data serializer

* Add new senstiive data serializer

* Remove dead comments

* Change up the test

* review comments

* wip

* Tighten tests and push annotation down

* Tighten tests and push annotation down

* Changelog

* Add test

* 7.0.1-SNAPSHOT bump

* Error code

* Add method used by CDR

* add version enum

* Fix test

* Rel 7 0 CVE (#5663)

* Bump json-path

* Pin parrson

* Bump elastic

* Bump spring version

* Exclude JDBC

* Descendent fix (#5669)

* Fix "is-a" ValueSet expansion, add "descendent-of" support

* Fixed tests in DSTU3 and R5

* Trigger new build

* Revert "Trigger new build"

This reverts commit 46c672b338.

* fix default partition setting on resource (#5617)

* fix default partition setting on resource

* changelog

* Handle DEFAULT partition in rule checker.

* Fix spotless

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>

* pom bump, doc add, version enum add (#5616)

Co-authored-by: Long Ma <long@smilecdr.com>

* fix default partition setting on resource (#5618)

* fix default partition setting on resource
* Handle DEFAULT partition in rule checker.

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Add setting to make broker not use JacksonMessageConverter (#5611)

* Add setting to make broker not use JacksonMessageConverter

* Add changelog

* Implement suggestions

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Fix version

* add changelog, add attribution, remove accidental bring-overs

* add changelog, add attribution, remove accidental bring-overs

* Finish jira section

---------

Co-authored-by: Ole Hedegaard <ohe@trifork.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* fix conflict mistake

* removed test assert that is no longer true:

* version bumpb to 7.1.1

* fix version Enum test

---------

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: volodymyr-korzh <132366313+volodymyr-korzh@users.noreply.github.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: Martha Mitran <martha.mitran@smilecdr.com>
Co-authored-by: JP <jonathan.i.percival@gmail.com>
Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ole Hedegaard <ohe@trifork.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2024-02-07 22:16:34 -07:00
Luke deGruchy 644c68872f
Update copyright headers for 2024. (#5575)
* Update copyright headers for 2024.

* Fix more copyright headers.

* Add <name> to hapi-fhir-jpa pom.
2024-01-03 14:47:05 +00:00
Luke deGruchy 78a50f92bc
Replace all nullable/nonnull imports of javax.annotation with jakarta.annotation (#5540)
* Replace all nullable/nonnull imports of javax.annotation with jakarta.annotation.

* Fix checkstyle.  Update checkstyle version.
2023-12-07 09:16:48 -05:00
Etienne Poirier fdeaf8384f
Request for exposing client id and username in batch job status api (#5526)
* initial tests and implementation of the feature persistence layer.

* initial tests and implementation pointcut to modify JobInstance before persistence.

* spotless happy.

* Addressing comments from first code review.

* happy spotless

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-12-04 11:44:48 -05:00
James Agnew 6d58df0f44
Jakarta/Javax Servlet Migration (#5429)
* Servlet migrsation changes

* Apply spotless

* Work on migration

* Start working on JPA migration

* Test fixes

* Test fixes

* Test fixes

* Work

* Work on servlet migration

* Work on migration

* Work on fixes

* Imports fix

* Work on tests

* Test fix

* All tests passing

* Work on migration

* Schema generator

* Refactor ID strategy

* Some test fixes

* Test fixes

* API additions

* Work on websockets

* Test fixe

* Version bump

* Work on migration

* All tests have passed!

* Work on test fixes

* Fixes

* Work on fixes

* CDR fix

* Work on migration

* Fixes

* Spotless

* Disable onetoone ban

* Adjust migratyions

* Transaction service fix

* Add tinyint to migrator

* Work

* Migration

* Compile fix

* Add license

* Test fix

* Test fix

* Cleanup

* Cleanup attempt

* Test fix

* Reslve fixmes

* Remove legacy lastn implementation

* Test cleanup

* Cleanup

* Remove dead import

* Bad import setups

* Review comments

* Review fixes

* Build fix

* Test fix

* Spotless

* Account for CR changes

* Compile fixes

* Rollback one change

* Test tweak

* License header

* Test fix

* Test fix part 2:

* Clean up tx isolation

* Spotless

* Spotless apply

* Drop min SQL Server level

* Attempted test fix

* Test fix

* Test fix

* Test fix

* Bump hibernate

* Test fix

* Test fixes

* Test fixes

* Test fixes

* Test cleanup

* One more test fix

* Test fix

* Try to fix test

* Clean up expunge

* Test fix

* Remove unneeded task

* Spotless

* Bump compiler plugin

* Spotless

* Compile fix

* Add license headers

* License header

* Test fixes

* Bump animal sniffer

* Spotless

* Bump Jetty version

* Test fix and version bump

* Cleanup

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-12-03 11:15:37 -05:00
Michael Buckley 83bfa817ae
Batch - ensure we target the default partition for non-gated jobs (#5496)
* Target the default partition for batch2 storage.
2023-11-27 17:32:58 -05:00
Michael Buckley 9aa173a981
Target the default partition for batch2 storage. (#5475) 2023-11-22 00:12:38 +00:00
Michael Buckley 6abfed603e
Use streams to avoid multiple queries during batch job id chunking. (#5444)
Use stream for chunking instead of repeated sorted query pages.
2023-11-20 22:39:36 +00:00
Ken Stevens b9155721a7
fix batch2 npe when no results returned (#5219)
* fix npe error

* spotless

* spotless
2023-08-20 00:37:16 +00:00
Tadgh a10856e091
GM1 Mergeback (#5203)
* version bump

* Bump to core release 6.0.22 (#5028)

* Bump to core release 6.0.16

* Bump to core version 6.0.20

* Fix errors thrown as a result of VersionSpecificWorkerContextWrapper

* Bump to core 6.0.22

* Resolve 5126 hfj res ver prov might cause migration error on db that automatically indexes the primary key (#5127)

* dropped old index FK_RESVERPROV_RES_PID on RES_PID column before adding IDX_RESVERPROV_RES_PID

* added changelog

* changed to valid version number

* changed to valid version number, need to be ordered by version number...

* 5123 - Use DEFAULT partition for server-based requests if none specified (#5124)

5123 - Use DEFAULT partition for server-based requests if none specified

* consent remove all suppresses next link in bundle (#5119)

* added FIXME with source of issue

* added FIXME with root cause

* added FIXME with root cause

* Providing solution to the issue and removing fixmes.

* Providing changelog

* auto-formatting.

* Adding new test.

* Adding a new test for standard paging

* let's try this and see if it works...?

* fix tests

* cleanup to trigger a new run

* fixing tests

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 5117 MDM Score for No Match Fields Should Not Be Included in Total Score  (#5118)

* fix, test, changelog

* fix, test, changelog

---------

Co-authored-by: justindar <justin.dar@smilecdr.com>

* Rename file to force IT mode

* _source search parameter needs to support modifiers (#5095)

_source search parameter needs to support modifiers - added support form :contains, :missing, :above modifiers

* Fix HFQL docs (#5151)

* Expunge operation on codesystem may throw 500 internal error with precondition fail message. (#5156)

* Initial failing test.

* Solution with changelog.

* fixing format.

* Addressing comment from code review.

* fixing failing test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* documentation update (#5154)

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* Fix hsql jdbc driver deps (#5168)

Avoid non-included classes in jdbc driver dependencies.

* $delete-expunge over 10k resources will now delete all resources (#5144)

* First commit with very rough fix and unit test.

* Refinements to ResourceIdListStep and Batch2DaoSvcImpl.  Make LoadIdsStepTest pass.   Enhance Batch2DaoSvcImplTest.

* Spotless

* Fix checkstyle errors.

* Fix test failures.

* Minor refactoring.  New unit test.  Finalize changelist.

* Spotless fix.

* Delete now useless code from unit test.

* Delete more useless code.

* Test pre-commit hook

* More spotless fixes.

* Address most code review feedback.

* Remove use of pageSize parameter and see if this breaks the pipeline.

* Remove use of pageSize parameter and see if this breaks the pipeline.

* Fix the noUrl case by passing an unlimited Pegeable instead.  Effectively stop using page size for most databases.

* Deprecate the old method and have it call the new one by default.

* updating documentation (#5170)

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* _source search parameter modifiers for Subscription matching (#5159)

* _source search parameter modifiers for Subscription matching - test, implementation and changelog

* Removal of meta tags during updates do not trigger subscription (#5181)

* Initial failing test.

* adding solution;
fixing documentation;

* spotless apply

* adding changelog

* modifying current test

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Issue 5173 get gateway everything doesnt return all patients (#5174)

* Failing test

* Also set offset and count in base DAO override

* Changelog

* Fix for specific case where count has been set in parameters

* spotless

* Improve checks

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Do not 500 and continue IG ingestion when different IGs try to save different ValueSets with colliding FHIR IDs (#5175)

* First commit with failing unit test and small tweaks.

* Swallow resource version exceptions from colliding ValueSet OIDs and log a descriptive error instead.  Add more detailed unit testing.

* Tweaks to logic and update the changelog.  Reverse all changes to TermReadSvcImpl.

* Revert PackageResourceParsingSvc to release branch baseline.

* Accept code reviewer suggestion to change changelog description.

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Fix link

* Remove target slf4j version

* dont use new API for a bit (#5191)

* Return DropIdGeneratorTask from the Builder to permit clients to mutate the (#5193)

DropIdGeneratorTask.

* Dqm performance bug update and provider loader fix (#5180)

* update tests, move properties, update operation loader

* update wip

* remove test

* fixing tests, adding config

* update config and provider loader

* fix bundles

* fix cache settings on tests

* version bump and change log

* version bump

* fix formatting

* CVE-2022-45868

* wip cve change

* cve h2 add back in

---------

Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>

* bulkExportReuse with POST and GET (#5161)

* string manipulation

* Code to ensure bulkExportReuse works with POST and GET requests

* Added formatting changes

* Fixed tests that were not working

* Formatting

* Code clean up

* fixing test failures

* fixing test failures

* Removed arrOfParams to now utilize ObjectMapper

* Removing stack trace and adding an exception

* Fixed test issue

* formatting

* formatting

* Resolving code review comments

* Reduce size of subscription max results (#5194)

* Reduce MAX_SUBSCRIPTION_RESULTS to 10000

* Add changelog

* 5037 goldenresource remains when target resource deleted (#5038)

* draft test and fix

* remove unused fields

* remove unused fields

* remove unused fields

* draft test + solution for possible match case

* combine sql statement + more error checking

* add test case for possible duplicate

* add config for autodeleting grs

* refactoring, adding support for mongo, docs

* refactoring + fixing mongo queries

* add changelogs

* fix both way link removal

* clean up test comments

* rename method

* remove unnecessary bean

* merge master/resolve conflicts

* mvn spotless

* address comment

* changes to avoid version bumping

* spotless

* change error code

---------

Co-authored-by: justindar <justin.dar@smilecdr.com>

* dont use new API for a bit (#5190)

* licenses

* wip

* Fix API usage

* wip

* Version bump

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: TynerGjs <132295567+TynerGjs@users.noreply.github.com>
Co-authored-by: Steve Corbett <137920358+steve-corbett-smilecdr@users.noreply.github.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: justindar <justin.dar@smilecdr.com>
Co-authored-by: volodymyr-korzh <132366313+volodymyr-korzh@users.noreply.github.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: LalithE <132382565+LalithE@users.noreply.github.com>
2023-08-15 12:32:10 -07:00
LalithE 2fbd3fa272
Lowering the logging severity level from info to debug for specific services (#4948)
* Changed the two files to reduce info level to debug level for select services.

* Adding change log

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-07-20 10:52:11 -04:00
Tadgh 52f3ef5992
New formatting rules and pre-commit hooks (#5027)
* pre-apply spotless

* New pre-format checkpoint

* Once-over the repository

* Add pre-commit hook

* New github action

* excludes

* Revert "Once-over the repository"

This reverts commit 89334ba8d1.

* Once-over the repo

* do not inherit

* Revert "Once-over the repo"

This reverts commit 31ed8731ad.

* Add baisc license support to spotless

* Move license

* Fix inclusions

* Fix inclusions

* Fix licensing

* license fixes

* Remove dead file that is only a license

* Remove dead class

* remove dead file

* Tighten licensing

* Tighten licensing

* Tighten licensing

* Tighten licensing

* Remove specific package

* Make the precommit hook executable

* Fix ordering

* rollback license

* wip

* Add to deployable pom

* remove file

* Fix location

* Remove dead comment

* wipP

* Fix ratchet

* Fix up error

* Add pre-commit

* precommit fixes

* wip

* wip

* Autoformat

* remove dead pom parts

* replace internal hook with pre-commit

* Fix fetch-depth for checker action

* Remove ratchet and filter

* fix up

* rename

* Add comment check

* rework message

* First once-over of the repo

* wip

* format
2023-07-11 17:36:44 -07:00
Steve Corbett 10af1434fc
Delete expunge fails with identical update timestamps (#5047)
* 4759 - WIP unit test for this issue

* WIP updating unit test for ResourceIdListStep

* WIP candidate fix for ResourceIdListStep batch size problem

* Enhance unit test with ArgumentCaptor.

* WIP Parameterized the ResourceIdListStepTest and added assertions

* Adding changelog for this issue

* Updating changelog message for 5055

* Code formatting and import cleanup

---------

Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
2023-07-06 11:59:56 -04:00
volodymyr-korzh 93a41559b5
Reindex Batch Jobs stuck in QUEUED status after SearchParameter update (#5043)
* Reindex Batch Jobs stuck in QUEUED status after SearchParameter update - test added

* Reindex Batch Jobs stuck in QUEUED status after SearchParameter update - fix

* Reindex Batch Jobs stuck in QUEUED status after SearchParameter update - fix test

* Reindex Batch Jobs stuck in QUEUED status after SearchParameter update - send BatchJobWorkNotification after commit

* Reindex Batch Jobs stuck in QUEUED status after SearchParameter update - changelog fix

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-06-29 14:29:37 -04:00
James Agnew 6e0651e261
Bulk export partitioning fixes (#5039)
* Work

* Bulk export partitioning fixes

* Changelog fixes

* cleanup

* Compile fixes

* Test fix

* Add JIRA note

* Version bump

* Drop unneeded tag
2023-06-29 08:34:32 -04:00
StevenXLi 73d6997d21
fixed batch jobs on Postgres (#5022)
* fixed issue

* fixed pipeline issue

* review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-06-23 09:06:38 -04:00
Luke deGruchy e5699bcbfc
Add auto-generated headers to Java files. (#4999) 2023-06-16 13:53:20 -04:00
TipzCM 65ae1785e8
mdm-clear will now use batch-size parameter (#4985)
* using delete expunge service

* test fixing

* update version

* review points

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-06-15 20:32:21 -04:00
Luke deGruchy e37edfcf84
Ensure StepExecutor only logs a stack trace and error if the retries have been exhausted. Otherwise, just log a debug. (#4961)
* Ensure StepExecutor only logs a stack trace and error if the retries have been exhausted.  Otherwise, just log a debug.

* Accept code reviewer suggestion to amend log message

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-06-07 09:05:08 -04:00
michaelabuckley 9838f8a19a
Delete dead code now that Hapi has merged. (#4946) 2023-06-01 17:27:14 -07:00
StevenXLi 83411e585b
4570 add support for warning messages to the batch framework (#4571)
* added failing test

* added support for warning messages to batch2

* added change log

* fixed errors from merge, made warning work with new batch2

* fixed message parse, changed interface method to default, formatting changes

* added changelog

* code review changes

* code review change

* added new implementation for warning message processing, fixed migration

* changed column to lob, as text is not well supported

* code review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-06-01 13:16:44 -04:00
James Agnew 46857711c9
Delete expunge with cascade (#4931)
* Delete expunge with cascade

* Work

* Workgin

* Version bump hibernate

* Start working on delete cascade

* Work on delete expunge

* Test fixes

* Test fixes

* Add changelog

* Work on cascade

* Fixes

* Test work

* Test fixes
2023-05-27 11:07:59 -04:00
Tadgh 805e80e61f
6.6.0 Mergeback (#4924)
* Force Verify tests

* fix ITs (#4809)

* fix RestHookTestR5IT

* fix intermittent

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix migrator error on Oracle (#4814)

* Fix Oracle SQL error

* Add changelog

* Update clinical reasoning version (#4816)

* Update clinical reasoning version

* Update version

* Update version

* Clean-up and more wireup of evaluationSettings

* Add changelog

---------

Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>

* Opening the care-gaps endpoint for GET. (#4823)

Co-authored-by: Chalma Maadaadi <chalma@alphora.com>

* added version to mdm golden resource tag (#4820)

Co-authored-by: Long Ma <long@smilecdr.com>

* Update the changelog for 4697 to be more descriptive (#4827)

* Update the changelog for 4697 to be more descriptive

* Futher tweaks of the changelog

* Fixes a bug with tags.  (#4813)

* Test, fix

* Drop constraint, add migration

* Add changelog

* Fix userSelected null vs false

* Fix merge

* Fix up checkstyle whining

* One more failure

* Fix test

* wip

* changelog clarity

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* change index

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>

* fix migration issue (#4830)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Create correct version enum

* Remove superfluous migration

* fixing test (#4835)

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* email subscription, throw NullPointerException (#4790)

* fix bug

* Favoring constructor initialization to autowiring.

* enhancing test.

* Making class LoggingEmailSender available outside of the hapi-fhir-japserver-uhnfhirtest module.

* Passing all tests.

* adding changelog.

* Bumping version to 6.5.20-SNAPSHOT

* addressing code review comment.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Add docs for CR operations (#4855)

* Add docs for CR operations

* Correct changelog and javadoc for $package

* Add documentation for $apply parameters

* Add additional documentation for $package

* Cleanup

* Cleanup

* Cleanup

* Address review comments

* Add documentation for $care-gaps operation. (#4862)

* Add documentation for -gaps.

* addressing the comments.

---------

Co-authored-by: Chalma Maadaadi <chalma@alphora.com>

* 4853 validation does not error when display is not the same as the display defined in the codesystem 2 (#4854)

* added failing test

* implemented the solution

* changed test name

* added change log

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_6_0/4853-validation-does-not-error-when-display-is-not-the-same-as-the-display-defined-in-the-codesystem-2.yaml

Co-authored-by: James Agnew <jamesagnew@gmail.com>

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>

* fixing patient everything operator (#4845)

* fixing patient everything operator

* review fix

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* fix link

* Move image file

* Bundle resources containing over 100 references to the same Organization will fail with HAPI-2207 (#4871)

* Add failing unit test.

* Fix JpaId Long equality comparison to use ! equals() instead of !=, which fails for different instances of the same Long value.

* Add changelog.

* added warn message and test (#4848)

* added warn message and test

* code review fixes

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* Issue 4804 full table scan on mpi link during mdm clear (#4805)

* version bump for next release  (#4793)

* version bump

* Bump to correctnumber

* Version Enum and folder

* Remove interim from list

* wip

* Fix operation on nested type-choices in FhirPatch implementation (#4783)

* Fix operation on nested type-choices in FhirPatch implementation

* Add credit for #4783

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* #4468 fix previous link offset no cache pagination (#4489)

* #4468 Add test reproducing the issue

* #4468 Fix previous link for no cache offset pagination

* #4468 Use unchecked URI parsing

* Credit for #4489

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Changelog and data generating test

* Add MdmLink index

* Avoid double link deletion

* Use ThreadLocal safely

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Zach Smith <85943952+zachdoctolib@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Aleksej Parovysnik <100864000+alparodev@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Fix erroneous batch 2 $export 75% complete count when the job is COMPLETE (#4859)

* Add failing unit test.

* Add conditional logic to the InstanceProgress progress percentage to disregard the incomplete count if this is called from the reduction step.  This is to get around a race condition in which a work chunk is QUEUED and not yet complete when the reduction step calculates the progress.

* Add final.

* Add changelog.

* disable wars (#4877)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4868 fix paging hapi (#4870)

* fixing some offset and adding a test

* fixing the offset paging

* Removing duplicate

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Aleksej Parovysnik <100864000+alparodev@users.noreply.github.com>

* 4875-binary-access-write-doest-trigger-STORAGE-BINARY-ASSIGN-BLOB-ID-PREFIX-pointcut (#4876)

* Add failing test

* Add failing test

* Fix and changelog

* Pass content type parameter

* Back to auto wiring the context

* Invoke interceptor only when getting blobId, not also when storing it

* Avoid breaking implementers

* Address review comment

* Add new exception Msg code

* Fix broken test

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Fix batch job (bulk export) processed record count (#4879)

* Remove racy stats recalc.

* Throw 404 when requesting $export of non-existent Group or Patient (#4890)

* Remove default implementation intended only for interim backwards compatibility (#4894)

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Rule apply patient export (#4893)

* Test, fix, and changelog

* Better partition resolution

* Add checks based on rule applier

* Fix ordering failure due to hash set

* Allow empty auth interceptor

* Fix up operation type on invocation

* Add more tests, make hack implementation for patient instance level operation

* Tighten test name

* Changelog

* Default method

* remove dead method

* Remove dead autowire

---------

Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>

* cve pom changes (#4898)

Co-authored-by: Long Ma <long@smilecdr.com>

* backport subscription topic bean cleanup (#4904)

* 4891 bulk export do not recurse unasked for resources (#4895)

* updating tests

* fixing bulk export to not fetch resources not requested

* cleanup

* cleanup

* more warning suppressing

* adding error code

* blah

* fix test

* review fixes

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* lowers log level to remove bootup noise (#4908)

* CVE rel 6 6 (#4907)

* cve pom changes

* bump javax.el to jakarta.el

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* Issue 4905 post binary failure invoking interceptor for pointcuts storage preshow resources (#4906)

* Initial failing test

* Avoid applying binary blob id prefix multiple times

* Remove recently introduced method not needed anymore

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Enhance LogbackCaptureTestExtension (#4869)

* repro bug with test, fix bug

* ken informed me he resolved this bug on master, so i'm switching to use his solution

* disable wars

* review feedback

* review feedback

* review feedback again

---------

Co-authored-by: josie <josie.vandewetering@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Resolve 4863 from release branch searchparametercanonicalizer does not account for search parameters for custom resources types when converting dstu23 into runtimesearchparam (#4887)

* Modified canonicalizeSearchParameterDstu2 and 3, now correctly detect search parameters for custom resources

* Canonicalizers now correctly handle search parameters for custom resources

* created changelog

* Modification based on comments:
- remove Resource from target field when there are custom resource types
- fixed changelog typo
- removed unnecessary variable providesMembershipInCompartments

* Added tests for the SearchParameterCanonicalizer to test if base and target of RuntimeSearchParam is set as expected for DSTU2, DSTU3, R4, R4B, and R5 resources

* Fixed typo and removed commented code

* re-ordered init methods

* Update changelog

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* modifications following first code review.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* License

* Remove _lastUpdated filtering of _revincludes. (#4899)

Remove _lastUpdated filtering of _revincludes.

* 4910-dm-migration-error-for-oracle-19c (#4916)

* Remove all_constraints references which break in oracle 19c

* Add changelog

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* 4873 empty fhirid causes missing resource (#4874)

* add check for empty fhirid string and add test

* add test for populateid

* changelog

* version bump

* version bump

* reverse version bump

* Back to 6.5.21-SNAPSHOT.

---------

Co-authored-by: justindar <justin.dar@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>

* Fix include canonical url performance (#4919)

Use hash_identity for canonical join

* License

* Version bump

* Fix failure in test

* Licenses

* Review comments for pipeline

* Dead entry

* other typo

---------

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Brenin Rhodes <brenin@alphora.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: chalmarm <44471040+chalmarm@users.noreply.github.com>
Co-authored-by: Chalma Maadaadi <chalma@alphora.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Michael Buckley <michaelabuckley@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Sam Gunter <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: Zach Smith <85943952+zachdoctolib@users.noreply.github.com>
Co-authored-by: Aleksej Parovysnik <100864000+alparodev@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Josie <80289977+pepsiofficial@users.noreply.github.com>
Co-authored-by: josie <josie.vandewetering@smilecdr.com>
Co-authored-by: TynerGjs <132295567+TynerGjs@users.noreply.github.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: justindar <justin.dar@smilecdr.com>
2023-05-20 20:38:35 -07:00
michaelabuckley 81854baa02
Batch2 cleanup (#4779)
Post review batch2 cleanup.

Review fixes and cleanup from #4647

Deleted methods in IJobPersistence unused in prod paths, and replaced their usage in tests.
Lots of docs.
Replace copy-on-write pattern in JobDefinitionRegistry with simple ConcurrentHashMap
Fixed bad mappings from job ERRORED state to external APIs. People think ERRORED is a failed state. ¯\_(ツ)_/¯
Added some spec tests for chunk purging
Deprecated ERRORED. For deleting in 6.8
Lots of plans for 6.8. Too risky for 6.6
2023-04-28 15:44:30 -04:00
michaelabuckley 8813d9beda
Batch stabilization (#4647)
* Use java event names for work chunk transitions.

* Cherry-pick d5ebd1f667 from rel_6_4

Avoid fetching work-chunk data (#4622)

* add end time to reduction step (#4640)

* add end time to reduction step

* add changelog

---------

Co-authored-by: Long Ma <long@smilecdr.com>
(cherry picked from commit 37f5e59ffc)

* Cancel processing

Provide error message in cancelled jobs, and avoid transitions in final states.

* Apply tx boundary to starting job and first chunk.

* cleanup

* Apply tx boundary to work chunk processing

* Delete BatchWorkChunk

* Introduce events for job create, and chunk dequeue

* Apply tx boundary to chunk handler

* Move instance cancellation to database

* tx boundary around stats collection and completion

* tx boundary around stats collection and completion

* Extend tx boundary to error, fail, and cancel

* Move failure into status calc

* ERROR is not an "ended" state.

* Revert generics cleanup to avoid noise

* Avoid sending gated chunks twice.

* Make no-data path safer.  Cleanup

* Fix mock test for step advance.

* Delete unsafe updateInstace() call

* Cleanup

* Changelog and notes

* Fix cancel boundary.  Cleanups

* Cleanup

* Sort mongo chunks for stable paging.

Other cleanup

* Document error handling

* Cleanup

* Update hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/Batch2JobHelper.java

Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>

---------

Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
2023-04-25 15:47:23 -07:00
StevenXLi 3fb9a16975
4693 enable partitioning in bulk import (#4694)
* created failing tests

* implemented feature, added more tests

* added documentation and changelog

* fixed duplicate error code

* code review changes

---------

Co-authored-by: Steven Li <steven@smilecdr.com>
2023-03-31 09:43:40 -04:00
James Agnew bcc1ca7593
Transaction SQL Optimization (#4679)
* Start work optimizing transaction

* Tons of test cleanup

* Cleanup

* More optimization

* Optimize

* Many tests fixed

* Work on test fixes

* Optimization done, now doing cleanup

* Cleanup

* Add docs

* Test fixes

* Test fix

* License headers

* Test fix

* Test cleanup

* Test fix
2023-03-24 10:29:56 -04:00
michaelabuckley 4b3405a456
Use java event names for work chunk transitions. (#4658)
Change work-chunk creation to be first event in state machine.
2023-03-21 13:53:16 -04:00
Isaac Wen 14be0553d0
Moving all license comments to the first line (#4656)
* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

* changes for GL-4023

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
2023-03-20 10:04:26 -04:00
Ken Stevens 8b80da6441
re-enabled some disabled tests (#4644)
* re-enabled some disabled tests

* add logging

* improve logging

* removed nonsense test

* fix regression

* fix %now

* licenses

* disable searchparam reindexing on intermittent failing test

* refining fix to RetryingMessageHandlerWrapper

* licenses

* remove hapi-fhir-cli-jpaserver from azure pipelines

* undo unneeded change

* change log

* back out unneccessary changes

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-03-17 13:04:29 -04:00
James Agnew cf5470ae58
Search and Sort on Uplifted Refchains (#4633)
* Start work on refchains

* Semi working

* Add tests for transactions

* Add docs and lots of tests

* Add changelog

* Add tests

* Work on cleanup

* Add document operations

* Test fix

* Test fix

* Fixes

* Fix typo

* Test fix

* Test update

* Test updates

* Test fix

* Test fixes

* Test additions

* Test fix

* Add some javadocs

* Test fixes

* Intermittent test fix

* Doc tweak

* Test fixes

* Merge master in

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Review comments

* Version bump

* Add license

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-03-15 07:04:32 -04:00
Aditya Dave 8956b273f0
Concurrent Transaction Conditonal Creates (#4639)
* push broken test for testing placeholder create in transaction retry

* reproduce error for transaction retry

* cleanup test

* fix test testPlaceholderCreateTransactionRetry

* revert temp fix

* revert spacing

* Add tests

* Remove fixme

* Test fix

* test fix

* Test fix

* Remove test line

* Remove unneeded change

* Review comments

* Add docs

* Test logging

* Add test logging

* Add logging for intermittents

---------

Co-authored-by: aditya_dave <aditya@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-03-14 16:39:55 -04:00
James Agnew 59123cf0f0
Fix intermittent in batch2 tests (#4643)
* Fix intermittent in batch2 tests

* Add changelog

* Test fix

* Resolve intermittent

* Reducer changes

* Test fix

* Test fixes

* Resolve fixme

* Fix changelog
2023-03-13 19:59:05 +00:00
michaelabuckley d1f9c94038
Make WorkChunk handling transactional at state transitions. (#4621)
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
2023-03-12 19:19:21 -04:00
James Agnew ae1d249d99
Bulk export reducer step fix mergeback (#4606)
* One more fix for #4467

* Enabling massIngestionMode causes incomplete resource deletion (#4476)

* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)

* Adding initial test.

* Adding initial solution implementation.

* Adding change log and code clean up.

* addressing comments from 1st code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Change visibility of migration method (#4471)

* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)

* First commit:  Make SubscriptionValidatingInterceptor aware of which Pointcut is being called.  In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED.   Fix resulting compile errors in various unit tests.

* Fix/enhance unit tests.  Mark methods as deprecated instead of deleting them.  Add proper error code.  Complete changelog.

* Remove erroneous TODOs and tweak the validation logic.

* Enhance unit tests and fix changelog.

* Reindex batch job fails when processing deleted resources. (#4482)

* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* cleaning up checkstyle files (#4470)

* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Bump core to 5.6.881 (#4496)

* Bump core to 5.6.881-SNAPSHOT

* Work on fixing tests

* Work on fixing tests 2

* Bump to core release

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>

* Issue 4486 mdm inconsistent possible match score values (#4487)

* Extract method for readability

* Save always normalized score values in POSSIBLE_MATCH links.

* Avoid setting properties to null values. Adjust test.

* Simplify fix

* Fix test. Add RangeTestHelper.

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Revert "cleaning up checkstyle files (#4470)"

This reverts commit efae3b5d5f.

* core version fix

* Loosen rules for id helper

* License

* fix batch2 reduction step (#4499)

* fix bug where FINALIZE jobs are not cancellable

* moved reduction step to message hander

* moving reduction step to queue

* addingchangelog

* cleaning up

* review fixes

* review fix'

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Scheduled batch 2 bulk export job and binary delete (#4492)

* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.

* Change bulk import test for valueUri type (#4503)

* change tests

* suggested test change

* CVE resolutions (#4513)

* Bump Postgres for CVE

* Bump jetty

* Verison bump

* Remove comments

* Revrt bump

* Add check in scanner (#4518)

* 4516 create hapi fhir cli command to clear stale lock entries (#4517)

* Initial implementation

* better tests

* Add changelog and docs

* Forgotten files

* Code review comments

* Fix checkstyle

* Unable to Expunge CodeSystem (#4507)

* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>

* New line::

* Update to documentation regarding narrative generation; (#4521)

Providing changelog;

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* changed what score is set for mdmlinks that created new golden resource (#4514)

* changed what score is set for mdmlinks that created new golden resource

* fix test

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* REVERT: change to operationoutcome.html

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* added change log

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fix build (#4530)

* Making narrative_generation.md reference an html snippet (#4531)

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* 4441 rel 6 4 bad references creation bug (#4519)

* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fixed channel import null pointer exception from null header (#4534)

* retryCount 0 on null header + test + changelog

* suggested changes

* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)

This reverts commit 53252b8d15.

* Better error handling for when channel type is not supported (#4538)

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Avoid logging message payloads that contain sensitive data (#4537)

Don't log payloads - they may contain sensitive data.

* Bulk Export Bug With Many Resources and Low Max File Size (#4506)

* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd953.

* Revert "maintenance pass"

This reverts commit bbc7418d51.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149.

* testing

* Revert "testing"

This reverts commit aafc95c2f3.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* bump ver

* License updates'

* Downgrade dep'

* Updating version to: 6.4.1 post release.

* Add javadocs and sources to our serviceloaders

* Reset version

* Change parent

* Remove bumped version

* License fixes, new parent

* Updating version to: 6.4.1 post release.

* Fix bad creation of versionenum

* Improve performance on bulk export

* Add changelog

* Start working on cleaned up reducer

* Clean up batch calls

* Work on issues

* Build fixes

* typedbundleprovider getallresources override (#4552)

* typedbundleprovider getallresources override

* added test to immunization tests that validates pagination does not result in null pointer if over default queryCount

* moved HapiFhirDal test to its own test class

* removed unused imports

* Update to use JpaStorageSettings

* adding changelog for issue 4551

* fix changelog spacing

* changelog type  to fix

---------

Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>

* Add backport info

* Upgrade core to 5.6.97, make adjustments in hapi-fhir, and ensure that all tests pass (#4579)

* First commit:  Create new branch from the release branch with changes from James' branch.  This probably won't compile as the work is incomplete.

* Second round of changes from integrating James' branch.

* Mark most test failures with TODOs.

* Add whitespace

* Add changes to FhirPathR4 to set FHIRPathEngine to non-strict FP evaluation.

* Fix CreatePackageCommandTest to assert null instead of empty string.  Comments on tests that fail due to the double-quote encoding problem.

* Downgrade to core 5.6.97.

* Fix another test and remove TODOs.

* Fix changelog.

* Clean up some erroneous changes and TODOs.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Fix up dal test

* Address leftover code review feedback from the upgrade to core 5.6.97. (#4585)

* Exclude pinned core deps

* Force pin structs

* Add model changes to IBaseCoding and related changes (#4587)

* Add IbaseCoding changes, and tinder changes

* Fix up tag definition

* converter addition

* Fix unit test and add changelog.

* Add jira to changelog.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Fix changelog

* Tidy metadata

* Test fixing

* Test fixes

* Work on progress

* Add changelog

* Build tweak

* Fixes

* Disable intermittently failing tests. (#4593)

* rename tests to IT

* Disable more intermittently failing tests (#4595)

* Disable more intermittently failing tests.

* Disable another intermittently failing tests.

* ITify

* Disable yet another intermittently failing tests. (#4596)

* disable

* disables

* Fixes

* disables

* Fix compile

* Test fixes

* Updating version to: 6.4.2 post release.

* Bump to 6.4.2-SNAPSHOT

* Fix compile

* fix version

* Address review comments

* Review comments

* Version bump

* Compile fix

* Test fixes

* Compile fixes

* One more compile fix

* Test fixes

* Version bump

* Resolve fixme

---------

Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
2023-03-01 14:52:36 -05:00
James Agnew eea6193df9
JPA config refactoring (#4545)
* Interceptor rework

* Work on config refactor

* Fixes

* Docs tweaks

* Address review comments

* Test tweak

* Test fixes

* Try to fix tests

* Test tweaks
2023-02-15 08:43:01 -05:00
Ken Stevens 794e4510c6
6_4 mergeback (#4547)
* One more fix for #4467

* Enabling massIngestionMode causes incomplete resource deletion (#4476)

* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)

* Adding initial test.

* Adding initial solution implementation.

* Adding change log and code clean up.

* addressing comments from 1st code review.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Change visibility of migration method (#4471)

* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)

* First commit:  Make SubscriptionValidatingInterceptor aware of which Pointcut is being called.  In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED.   Fix resulting compile errors in various unit tests.

* Fix/enhance unit tests.  Mark methods as deprecated instead of deleting them.  Add proper error code.  Complete changelog.

* Remove erroneous TODOs and tweak the validation logic.

* Enhance unit tests and fix changelog.

* Reindex batch job fails when processing deleted resources. (#4482)

* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* cleaning up checkstyle files (#4470)

* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Bump core to 5.6.881 (#4496)

* Bump core to 5.6.881-SNAPSHOT

* Work on fixing tests

* Work on fixing tests 2

* Bump to core release

---------

Co-authored-by: dotasek <david.otasek@smilecdr.com>

* Issue 4486 mdm inconsistent possible match score values (#4487)

* Extract method for readability

* Save always normalized score values in POSSIBLE_MATCH links.

* Avoid setting properties to null values. Adjust test.

* Simplify fix

* Fix test. Add RangeTestHelper.

---------

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Revert "cleaning up checkstyle files (#4470)"

This reverts commit efae3b5d5f.

* core version fix

* Loosen rules for id helper

* License

* fix batch2 reduction step (#4499)

* fix bug where FINALIZE jobs are not cancellable

* moved reduction step to message hander

* moving reduction step to queue

* addingchangelog

* cleaning up

* review fixes

* review fix'

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Scheduled batch 2 bulk export job and binary delete (#4492)

* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.

* Change bulk import test for valueUri type (#4503)

* change tests

* suggested test change

* CVE resolutions (#4513)

* Bump Postgres for CVE

* Bump jetty

* Verison bump

* Remove comments

* Revrt bump

* Add check in scanner (#4518)

* 4516 create hapi fhir cli command to clear stale lock entries (#4517)

* Initial implementation

* better tests

* Add changelog and docs

* Forgotten files

* Code review comments

* Fix checkstyle

* Unable to Expunge CodeSystem (#4507)

* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>

* New line::

* Update to documentation regarding narrative generation; (#4521)

Providing changelog;

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* changed what score is set for mdmlinks that created new golden resource (#4514)

* changed what score is set for mdmlinks that created new golden resource

* fix test

---------

Co-authored-by: Long Ma <long@smilecdr.com>

* REVERT: change to operationoutcome.html

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* added change log

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fix build (#4530)

* Making narrative_generation.md reference an html snippet (#4531)

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)

* fix + test

* minor fix

* Addressing suggestion

* Minor changes

* 4441 rel 6 4 bad references creation bug (#4519)

* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* fixed channel import null pointer exception from null header (#4534)

* retryCount 0 on null header + test + changelog

* suggested changes

* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)

This reverts commit 53252b8d15.

* Better error handling for when channel type is not supported (#4538)

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Avoid logging message payloads that contain sensitive data (#4537)

Don't log payloads - they may contain sensitive data.

* Bulk Export Bug With Many Resources and Low Max File Size (#4506)

* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd953.

* Revert "maintenance pass"

This reverts commit bbc7418d51.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149.

* testing

* Revert "testing"

This reverts commit aafc95c2f3.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* bump ver

* License updates'

* Remove checkstyle

* Verison bump

* Move migrations

* semicolon

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-02-13 13:23:25 -08:00
Tadgh 456cc81b32
Rel 6 2 5 mergeback (#4459)
* jm wrong bundle entry url (#4213)

* Bug test

* here you go

* Generate relative URIs for bundle entry.request.url, as specified

* Point jira issue in changelog

* Adjust tests to fixes

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* improved logging (#4217)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Rel 6 1 3 mergeback (#4215)

* Bump for CVE (#3856)

* Bump for CVE

* Bump spring-data version

* Fix compile

* Cut over to spring bom

* Bump to RC1

* remove RC

* do not contrain reindex for common SP updates (#3876)

* only fast-track jobs with exactly one chunk (#3879)

* Fix illegalstateexception when an exception is thrown during stream response (#3882)

* Finish up changelog, minor refactor

* reset buffer only

* Hack for some replacements

* Failure handling

* wip

* Fixed the issue (#3845)

* Fixed the issue

* Changelog modification

* Changelog modification

* Implemented seventh character extended code and the corresponding dis… (#3709)

* Implemented seventh character extended code and the corresponding display

* Modifications

* Changes on previous test according to modifications made in ICD10-CM XML file

* Subscription sending delete events being skipped (#3888)

* fixed bug and added test

* refactor

* Update for CVE (#3895)

* updated pointcuts to work as intended (#3903)

* updated pointcuts to work as intended

* added changelog

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* 3904 during $delete expunge job hibernate search indexed documents are left orphaned (#3905)

* Add test and implementation

* Add changelog

* 3899 code in limits (#3901)

* Add implementation, changelog, test

* Update hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/provider/r4/ResourceProviderR4Test.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* 3884 overlapping searchparameter undetected rel 6 1 (#3909)

* Applying all changes from previous dev branch to current one pointing to rel_6_1

* Fixing merge conflict related to Msg.code value.

* Fixing Msg.code value.

* Making checkstyle happy.

* Making sure that all tests are passing.

* Passing all tests after fixing Msg.code

* Passing all tests.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* 3745 - fixed NPE for bundle with duplicate conditional create resourc… (#3746)

* 3745 - fixed NPE for bundle with duplicate conditional create resources and a conditional delete

* created unit test for skip of delete operation while processing duplicating create entries

* moved unit test to FhirSystemDaoR4Test

* 3379 mdm fixes (#3906)

* added MdmLinkCreateSvcimplTest

* fixed creating mdm-link not setting the resource type correctly

* fixed a bug where ResourcePersistenceId was being duplicated instead of passed on

* Update hapi-fhir-jpaserver-mdm/src/test/java/ca/uhn/fhir/jpa/mdm/svc/MdmLinkCreateSvcImplTest.java

Change order of tests such that assertEquals takes expected value then actual value

Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* added changelog, also changed a setup function in test to beforeeach

Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>

* Fix to the issue (#3855)

* Fix to the issue

* Progress

* fixed the issue

* Addressing suggestions

* add response status code to MethodOutcome

* Addressing suggestions

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Fix for caching appearing broken in batch2 for bulkexport jobs (#3912)

* Respect caching in bullk export, fix bug with completed date on empty jobs

* add changelog

* Add impl

* Add breaking test

* Complete failing test

* more broken tests

* Fix more tests'

* Fix paging bug

* Fix another brittle test

* 3915 do not collapse rules with filters (#3916)

* do not attempt to merge compartment permissions with filters

* changelog

* Rename to IT for concurrency problems

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Version bump

* fix $mdm-submit output (#3917)

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Gl3407 bundle offset size (#3918)

* begin with failing test

* fixed

* change log

* rollback default count change and corresponding comments

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* Offset interceptor now only works for external calls

* Initialize some beans (esp interceptors) later in the boot process so they don't slow down startup.

* do not reindex searchparam jobs on startup

* Fix oracle non-enterprise attempting online index add (#3925)

* 3922 delete expunge large dataset (#3923)

* lower batchsize of delete requests so that we do not get sql exceptions

* blah

* fix test

* updated tests to not fail

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* add index

* Fix up colun grab

* Revert offset mode change

* Revert fix for null/system request details checks for reindex purposes

* Fix bug and add test for SP Validating Interceptor (#3930)

* wip

* Fix uptests

* Fix index online test

* Fix SP validating interceptor logic

* Updating version to: 6.1.1 post release.

* fix compile error

* Deploy to sonatype (#3934)

* adding sonatype profile to checkstyle module

* adding sonatype profile to tinder module

* adding sonatype profile to base pom

* adding final deployToSonatype profile

* wip

* Revert version enum

* Updating version to: 6.1.1 post release.

* Add test, changelog, and implementation

* Add backport info

* Create failing test

* Implemented the fix, fixed existing unit tests

* added changelog

* added test case for no filter, exclude 1 patient

* wip

* Add backport info

* Add info of new version

* Updating version to: 6.1.2 post release.

* bump info and backport for 6.1.2

* Bump for hapi

* Implement bug fixes, add new tests (#4022)

* Implement bug fixes, add new tests

* tidy

* Tidy

* refactor for cleaning

* More tidying

* Lower logging

* Split into nested tests, rename, add todos

* Typo

* Code review

* add backport info

* Updating version to: 6.1.3 post release.

* Updating version to: 6.1.3 post release.

* removed duplicate mention of ver 6.1.3 in versionEnum

* backport pr 4101

* mdm message key (#4111)

* begin with failing test

* fixed 2 tests

* fix tests

* fix tests

* change log

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* backport 6.1.3 docs changes

* fixed typo on doc backport message

* fix test breaking

* Updating version to: 6.1.4 post release.

* wip

Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>

* pin okio-jvm for kotlin vuln (#4216)

* Fix UrlUtil.unescape() by not escaping "+" to " " if this is an "application/..." _outputFormat. (#4220)

* First commit:  Failing unit test and a TODO with a vague idea of where the bug happens.

* Don't escape "+" in a URL GET parameter if it starts with "application".

* Remove unnecessary TODO.

* Add changelog.

* Code review feedback on naming.  Also, make logic more robust by putting plus and should escape boolean && in parens.

* Ks 20221031 migration lock (#4224)

* started design

* complete with tests

* changelog

* cleanup

* tyop

Co-authored-by: Ken Stevens <ken@smilecdr.com>

* 4207-getpagesoffset-set-to-total-number-of-resources-results-in-inconsistent-amount-of-entries-when-requests-are-sent-consecutively (#4209)

* Added test

* Added solution

* Changelog

* Changes made based on comments

* Fix bug with MDM submit

* fix

* Version bump

* 4234 consent in conjunction with versionedapiconverterinterceptor fails (#4236)

* Add constant for interceptor

* add test, changelog

* Allow Batch2 transition from ERRORED to COMPLETE (#4242)

* Allow Batch2 transition from ERRORED to COMPLETE

* Add changelog

* Test fix

Co-authored-by: James Agnew <james@jamess-mbp.lan>

* 3685 When bulk exporting, if no resource type param is provided, defa… (#4233)

* 3685 When bulk exporting, if no resource type param is provided, default to all registered types.

* Update test case.

* Cleaned up changelog.

* Added test case for multiple resource types.

* Added failing test case for not returning Binary resource.

* Refactor solution.

Co-authored-by: kylejule <kyle.jule@smilecdr.com>

* Add next version

* bulk export permanently reusing cached results (#4249)

* Add test, fix bug, add changelog

* minor refactor

* Fix broken test

* Smile 4892 DocumentReference Attachment url (#4237)

* failing test

* fix

* increase test Attachment url size to new max

* decrease limit to 500

* ci fix

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Overlapping SearchParameter with the same code and base are not allowed (#4253)

* Overlapping SearchParameter with the same code and base are not allowed

* Fix existing tests according to changes

* Cleanup dead code and remove related tests

* Version Bump

* ignore misfires in quartz

* Allowing Failures On Index Drops (#4272)

* Allowing failure on index drops.

* Adding changeLog

* Modification to changelog following code review.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>

* Revert "ignore misfires in quartz"

This reverts commit 15c74a46bc.

* Ignore misfires in quartz (#4273)

* Reindex Behaviour Issues (#4261)

* fixmes for ND

* address FIXME comments

* fix tests

* increase max retries

* fix resource id chunking logic

* fix test

* add modular patient

* change log

* version bump

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>

* Set official Version

* license

* Fix up numbers

* Fix up numbers

* Update numbers

* wip

* fix numbers

* Fix test:

* Fix more tests

* TEMP FIX FOR BUILD

* wip

* Updating version to: 6.2.1 post release.

* Add a whack of logging

* wip

* add implementation

* wip and test

* wip

* last-second-fetch

* expose useful method

* remove 10000 limit

* Strip some logging

* Fix up logging

* Unpublicize method

* Fix version

* Make minor changes

* once again on 6.2.1

* re-add version enum

* add folder

* fix test

* DIsable busted test

* Disable more broken tests

* Only submit queued chunks

* Quiet log

* Fix wrong pinned version

* Updating version to: 6.2.2 post release.

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https… (#4291)

* fixes for https://github.com/hapifhir/hapi-fhir/issues/4277 and https://github.com/hapifhir/hapi-fhir/issues/4276

* Credit for #4291

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* backport and changelog for 6.2.2

* Updating version to: 6.2.3 post release.

* fix https://simpaticois.atlassian.net/browse/SMILE-5781

* Version bump to 6.2.3-SNAPSHOT

* Auto retry on MDM Clear conflicts (#4398)

* Auto-retry mdm-clear on conflict

* Add changelog

* Build fix

* Disable failing test

* Update to 6.2.3 again

* Update license dates

* Dont fail on batch2 double delivery (#4400)

* Don't fail on Batch2 double delivery

* Add changelog

* Update docker for release ppipeline

* Updating version to: 6.2.4 post release.

* Add test and implementation to fix potential NPE in pre-show resources (#4388)

* Add test and implementation to fix potential NPE in pre-show resources

* add test

* WIP getting identical test scenario

* More robust solution

* Finalize Code

* Add changelog, move a bunch of changelogs

* Remove not needed test

* Minor refactor and reporting

* Fix up megeback

* update backport info

* update backport info

* Updating version to: 6.2.5 post release.

* Prevent chunk from returning to in-progress unless it is errorred, in-progress, or queued

* changelog

* Update logger

* Indicate backport

* Add version and upgrade info. add backport

* hapi-fhir side done

* cdr side

* add assert

* add test

* Fix import

* prevent multiple reduction step runs in maintenance run (#4423)

* setting job to inprog to prevent multiple maintenance passes from running reduction steps

* added a new status

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>

* Add backport info

* Fix up tests

* Fix discrepancies

* Fix test

* Version bump

* bump ips

Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: kateryna-mironova <107507153+kateryna-mironova@users.noreply.github.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: karneet1212 <112980019+karneet1212@users.noreply.github.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
Co-authored-by: James Agnew <james@jamess-mbp.lan>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-01-24 17:38:35 -08:00