Commit Graph

76 Commits

Author SHA1 Message Date
James Agnew 6e0651e261
Bulk export partitioning fixes (#5039)
* Work

* Bulk export partitioning fixes

* Changelog fixes

* cleanup

* Compile fixes

* Test fix

* Add JIRA note

* Version bump

* Drop unneeded tag
2023-06-29 08:34:32 -04:00
volodymyr-korzh 1f44ac1092
GenericClient POST paging support (#5013)
* GenericClient POST paging support - implementation and test

* GenericClient POST paging support - fixes

* GenericClient POST paging support - fixes

* GenericClient POST paging support - failed tests fix

* GenericClient POST paging support - HAPI version bump
2023-06-22 15:28:47 -04:00
TipzCM d7bdabb91b
nickname svc refatoring (#5006)
* nickname svc refatoring

* moving things around

* cleanup

* gitignore

* removing factory

* added changelog

* version bump

* fixing test

* fix test

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-06-22 08:26:47 -04:00
TipzCM 65ae1785e8
mdm-clear will now use batch-size parameter (#4985)
* using delete expunge service

* test fixing

* update version

* review points

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-06-15 20:32:21 -04:00
TipzCM 0a7f97adb7
4917 refactor nicknaming hapi with version bump (#4955)
* refactoring 1

* refactoring step 2

* refactoring matches

* cleanup

* fixing some tests

* fixing more tests

* remove dbugcode

* adding the nickname factory

* code review fixes

* review fixes

* put default to prevent breaking

* refactor review fixes

* more review points

* more review points

* review points

* review points

* more review changes

* review points

* review points

* adding bean

* updating version

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-06-06 09:00:14 -04:00
Tadgh 6c03a512f5
Revert "refactoring nickname matching (#4918)" (#4954)
This reverts commit 7a18f17a01.
2023-06-02 19:40:50 +00:00
TipzCM 7a18f17a01
refactoring nickname matching (#4918)
* refactoring 1

* refactoring step 2

* refactoring matches

* cleanup

* fixing some tests

* fixing more tests

* remove dbugcode

* adding the nickname factory

* code review fixes

* review fixes

* put default to prevent breaking

* refactor review fixes

* more review points

* more review points

* review points

* review points

* more review changes

* review points

* review points

* adding bean

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-06-02 14:04:30 -04:00
David d94627c382
4847 allow filtering batch jobs by status (#4849)
* Add method to get all jobs of a certain status

* add queries

* add tests

* Refactor so extra parameter is not added

* Version bump

* Fix broken test

* Revert accidental change to pom unrelated to version

---------

Co-authored-by: David Chen <david.chen@smilecdr.com>
2023-05-18 14:46:44 -04:00
James Agnew 483ddca3be
Transaction with delete then update should not fail (#4831)
* Fixed

* Test fixes

* Add test

* Ongoing work

* Work on xactx

* Cleanup

* Changelog cleanup

* Resolve fixme

* Rework broken APIs

* Version bump

* Add license headers

* License header update

* License

* rk on fixes

* Test fixes

* Address review comments

* Test fixes

* Add license headers

* License header
2023-05-15 07:41:40 -04:00
Tadgh 9a095cc2d7
version bump for next release (#4793)
* version bump

* Bump to correctnumber

* Version Enum and folder

* Remove interim from list

* wip
2023-04-30 15:01:28 -04:00
michaelabuckley 81854baa02
Batch2 cleanup (#4779)
Post review batch2 cleanup.

Review fixes and cleanup from #4647

Deleted methods in IJobPersistence unused in prod paths, and replaced their usage in tests.
Lots of docs.
Replace copy-on-write pattern in JobDefinitionRegistry with simple ConcurrentHashMap
Fixed bad mappings from job ERRORED state to external APIs. People think ERRORED is a failed state. ¯\_(ツ)_/¯
Added some spec tests for chunk purging
Deprecated ERRORED. For deleting in 6.8
Lots of plans for 6.8. Too risky for 6.6
2023-04-28 15:44:30 -04:00
James Agnew e2e5ff6bb8
Version bump HAPI FHIR (#4788) 2023-04-28 19:30:51 +00:00
James Agnew e2717bd63a
Bump corelib to FHIR R5 Final (#4727)
* Core bump

* Build build issues

* Bump core

* Cleanup

* Cleanup

* Resolve intermittent

* Work on core

* Add failing test for a check

* Test fixes

* Fixes

* DOn't mangle subscriptions

* Fix

* License headers

* Build fixes

* Version bump

* Compile fixes

* Fix compile issues
2023-04-28 06:28:22 -04:00
longma1 2171ad04a2
3924 mdm match resources across partitions and storing golden resources on a specific partition (#4770)
* added mdm search all partition for golden resource functionality

* added mdm partition setting

* build header fix

* test fixes and added missing side effect of allowing cross partition links on opeartions

* added changelog, also test fixes

* fixed changelog to an actual yaml file

* review fixes,  also simplified tests

* reverted one test because the original test schenario was correct

* added setting clearing in afterEach method so test dont fail while run in maven

* bump to pre-15

---------

Co-authored-by: Long Ma <long@smilecdr.com>
2023-04-28 02:15:35 -06:00
David 457b39bac1
Fix $mdm clear doesn't expunge golden record resources (#4767)
* Failing test

* Add method to expunge collection of deleted resources

* Failing test passes

* Add changelog

* Version bump

* Forgot to commit change

* Update test

* Check type validity

* Fix broken tests

---------

Co-authored-by: David Chen <david.chen@smilecdr.com>
2023-04-27 21:35:27 -06:00
michaelabuckley 8813d9beda
Batch stabilization (#4647)
* Use java event names for work chunk transitions.

* Cherry-pick d5ebd1f667 from rel_6_4

Avoid fetching work-chunk data (#4622)

* add end time to reduction step (#4640)

* add end time to reduction step

* add changelog

---------

Co-authored-by: Long Ma <long@smilecdr.com>
(cherry picked from commit 37f5e59ffc)

* Cancel processing

Provide error message in cancelled jobs, and avoid transitions in final states.

* Apply tx boundary to starting job and first chunk.

* cleanup

* Apply tx boundary to work chunk processing

* Delete BatchWorkChunk

* Introduce events for job create, and chunk dequeue

* Apply tx boundary to chunk handler

* Move instance cancellation to database

* tx boundary around stats collection and completion

* tx boundary around stats collection and completion

* Extend tx boundary to error, fail, and cancel

* Move failure into status calc

* ERROR is not an "ended" state.

* Revert generics cleanup to avoid noise

* Avoid sending gated chunks twice.

* Make no-data path safer.  Cleanup

* Fix mock test for step advance.

* Delete unsafe updateInstace() call

* Cleanup

* Changelog and notes

* Fix cancel boundary.  Cleanups

* Cleanup

* Sort mongo chunks for stable paging.

Other cleanup

* Document error handling

* Cleanup

* Update hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/Batch2JobHelper.java

Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>

---------

Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
2023-04-25 15:47:23 -07:00
Etienne Poirier 88459d0d00
4657 method name is inconsistent with the method behavior (#4659)
* Test implementation

* providing method getMessageKeyOrDefault();

* providing changelog

* preping codereview.

* Deprecating method getMessageKeyOrNull in favor of getMessagekey()

* updating comments following code review.

* Bumping version to 6.5.13-SNAPSHOT

* passing all tests.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-04-17 10:41:08 -04:00
James Agnew e500b23fe6
Instance level reindex operations (#4699)
* Reindex improvements

* Work

* Work on reindex

* Test fixes

* Generator now working

* Add instance reindex

* Work

* Build fixes

* Build fix

* Fixes

* Add changelog

* Test fixes

* Resolve checkstyle fix

* Version bump

* Address review comments

* Resolve imports

* Address review comments

* Test fix
2023-04-06 17:00:41 -04:00
James Agnew a2bc9a7212
Partition aware search cleanup (#4706)
* Partition aware search cleanup

* Compile fixes

* Build fixes

* HAPI FHIR version bump

* License

* License header

* Tests
2023-04-02 11:50:20 -04:00
Ken Stevens 4fbeeccda4
enable some disabled tests (#4662)
* enabled test

* added explanations to disabled tests

* added explanations to disabled tests

* yay enabled another one

* yay enabled another one

* testEverythingWithLargeSet2 failing intermittently

* fix intermittent regression introduced by splitting test class

* test passes

* need to document

* updated documentation

* prereview cleanup

* change log

* cleanup

* cleanup

* added breaking changes

* update changelog and breaking changes upgrade document to reverse the change to always return 200 per the spec.

* back out documentation change

* update documentation

* fix tests

* dstu2

* dstu3

* fix r4 validation warning test

* changelog

* hapi-fhir version bump to 6.5.10-SNAPSHOT

* fix cdr test

* fix tests

* remove accidental commit

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2023-04-02 08:47:16 -04:00
Justin McKelvy 9066ad618e
Bug cql engine paging (#4664)
* WIP paging

* iterable HapifhirRetrieveProvider

* Bundle iterating action

* cr config update

* adding paging functionality for search in cr

* updating BundleIterable logic and removing paging references

* add measureeval threaded config into cr

* update properties

* adding hapi change log 4663

* version bump for hapi

* update exception with msg.code, move copyright

* exorcism of paging provider from cr

* updating IT tests for terminology evaluatemeasure

* unit test, javadocs, removed reflection code

* added comments for explanation

* fix javadoc, iterable.size instead of method

* update from master

---------

Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
2023-03-31 11:09:50 -04:00
James Agnew e69fe05e96
Re-enable "Flush SP cache after SP change" (#4635)
* Revert "Revert "Flush SP cache after SP change (#4566)" (#4614)"

This reverts commit 9492999ed6.

* Additions for performance

* HAPI version bump

* Test logging improvement

* Test fixes

* Test fix

* Test fix

* Compile fix

* Fix

* License headers

* Test fix
2023-03-29 19:57:57 -04:00
michaelabuckley 4b3405a456
Use java event names for work chunk transitions. (#4658)
Change work-chunk creation to be first event in state machine.
2023-03-21 13:53:16 -04:00
James Agnew cf5470ae58
Search and Sort on Uplifted Refchains (#4633)
* Start work on refchains

* Semi working

* Add tests for transactions

* Add docs and lots of tests

* Add changelog

* Add tests

* Work on cleanup

* Add document operations

* Test fix

* Test fix

* Fixes

* Fix typo

* Test fix

* Test update

* Test updates

* Test fix

* Test fixes

* Test additions

* Test fix

* Add some javadocs

* Test fixes

* Intermittent test fix

* Doc tweak

* Test fixes

* Merge master in

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Review comments

* Version bump

* Add license

---------

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2023-03-15 07:04:32 -04:00
Luke deGruchy ea8b68f5e5
Introduce hibernate envers, use it only for MdmLink, disable it for now, add to MdmLink Dao and unit test revisions. (#4615)
* First commit with new maven deps for envers and stubbed changelist.

* Add repositoryFactoryBeanClass = EnversRevisionRepositoryFactoryBean.class to BaseAppCtxPersistence.  Comment it out in JpaConfig.  Add constant for hibernate.integration.envers.enabled.  Hard-code envers to false.  App starts without error.  Basic smoke testing for MpmLink update works.

* Add mdmLink code to retrieve envers history and unit test it.

* Initial code to handle envers from HapiFhirEnversRevision along with a custom extension of the revision/REVINFO entity.

* Add changelist and tweak table/sequence/field names and clean up the HapiFhirJpaMigrationTasks method.

* Fix some unit test failures and address TODOs.

* Ensure hard-coded deactivation of envers is overridden by unit tests.

* Remove commented-out code.

* Address code review comments.

* Remove sysout traces.

* Tweaks to poms with comments explaining why I left the versions in the plugin dependencies.  Get rid of a duplicate envers dependency.

* Bump hapi-fhir version to 6.5.5-SNAPSHOT.

* Disable intermittently failing test: PartitionedSubscriptionTriggeringR4Test.testCreateSubscriptionInPartitionAndResourceInDifferentPartition

* Try disabling Batch2CoordinatorIT failing test and see if branch pipeline passes.

* Add back hapi-fhir-validation-resources-r4b to hapi-fhir-jpaserver-base pom.xml.

* Bump hapi-fhir to 6.5.6-SNAPSHOT.

* Bump rest of hapi-fhir to 6.5.6-SNAPSHOT.

* Fix compile error on Batch2CoordinatorIT.

* Change revision ID column from integer to long in both the revision entity and the migration tasks.  Fix a bug with the migration tasks having the wrong name for the timestamp column.

* Fix interface in MongoDB to return Long for the revision ID instead of Int.   Implement TINYINT for all supported database products.  Keep TARGET_TYPE column at 40 instead of 100 for now since otherwise the schema migration test will fail.  Also, don't try to rename the foreign key or the schema migration test will fail.
2023-03-14 15:32:00 -04:00
michaelabuckley d1f9c94038
Make WorkChunk handling transactional at state transitions. (#4621)
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
2023-03-12 19:19:21 -04:00