* Failing test
* Add method to expunge collection of deleted resources
* Failing test passes
* Add changelog
* Version bump
* Forgot to commit change
* Update test
* Check type validity
* Fix broken tests
---------
Co-authored-by: David Chen <david.chen@smilecdr.com>
* Use java event names for work chunk transitions.
* Cherry-pick d5ebd1f667 from rel_6_4
Avoid fetching work-chunk data (#4622)
* add end time to reduction step (#4640)
* add end time to reduction step
* add changelog
---------
Co-authored-by: Long Ma <long@smilecdr.com>
(cherry picked from commit 37f5e59ffc)
* Cancel processing
Provide error message in cancelled jobs, and avoid transitions in final states.
* Apply tx boundary to starting job and first chunk.
* cleanup
* Apply tx boundary to work chunk processing
* Delete BatchWorkChunk
* Introduce events for job create, and chunk dequeue
* Apply tx boundary to chunk handler
* Move instance cancellation to database
* tx boundary around stats collection and completion
* tx boundary around stats collection and completion
* Extend tx boundary to error, fail, and cancel
* Move failure into status calc
* ERROR is not an "ended" state.
* Revert generics cleanup to avoid noise
* Avoid sending gated chunks twice.
* Make no-data path safer. Cleanup
* Fix mock test for step advance.
* Delete unsafe updateInstace() call
* Cleanup
* Changelog and notes
* Fix cancel boundary. Cleanups
* Cleanup
* Sort mongo chunks for stable paging.
Other cleanup
* Document error handling
* Cleanup
* Update hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/Batch2JobHelper.java
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
---------
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
* Basic test for reading from provider
* Basic test for reading from provider
* Failing test
* Add changelog
* Update hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/BinaryStorageInterceptorR4Test.java
Co-authored-by: Ken Stevens <khstevens@gmail.com>
---------
Co-authored-by: Ken Stevens <khstevens@gmail.com>
* a solution for trying to avoid multi-thread issues for transaction bundle processing
* cleanup
* using retries
* mnor tweaks
* checking in
* changes
* updating changelog
* some minor tweaks
* cleanup
* flip if
---------
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* try removing fork count from test-r4
* try removing fork count from the rest of the projects
---------
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* First commit: New tests, logs, and comments.
* Add hacky fix to run pipelines on. Add another unit test.
* Disable baseUrl checking if hasIdentifier() is true.
* Add changelog. Tweak unit tests.
* Remove junk.
* Add more assertions to unit tests. Make new production fix code clearer.
* Add more clarity to unit tests.
* Another unit test fix.
* add tests for r4b subscriptions
* begin with failing test
* prepare for SubscriptionTopicLoader
* backwards compatibility
* subscription topic registry done
* topic matching is working
* all but delivery is working now
* yay test passes with FIXMEs
* FIXME -> WIP
* switch notification to bundle
* message codes
* fixme
* disable services for fhir versions below R4B
* fix regression
* fix intermittent
* this change will likely break some other tests
* try a safer option
* fix tests
* fix intermittent (I hope)
* unit test
* improve logic around topic subscription categorization
* moar test
* moar test
* changed to support both r4b and r5
* moar test
* cleanup for test
* moar test
* moar test
* moar test
* changelog
* comment
* Msg.code
* fix mock
* add update test
* fix test cleanup
* tracking link for version converter issue
* review
* fix test
* fix test
---------
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* enabled test
* added explanations to disabled tests
* added explanations to disabled tests
* yay enabled another one
* yay enabled another one
* testEverythingWithLargeSet2 failing intermittently
* fix intermittent regression introduced by splitting test class
* test passes
* need to document
* updated documentation
* prereview cleanup
* change log
* cleanup
* cleanup
* added breaking changes
* update changelog and breaking changes upgrade document to reverse the change to always return 200 per the spec.
* back out documentation change
* update documentation
* fix tests
* dstu2
* dstu3
* fix r4 validation warning test
* changelog
* hapi-fhir version bump to 6.5.10-SNAPSHOT
* fix cdr test
* fix tests
* remove accidental commit
---------
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* Initial commit adding support for partition awareness to Bulk Export.
* Additional cleanup and remove references in tests to "all partitions".
* Additional cleanup and remove references in tests to "all partitions".
* More cleanup, test fixes and additional checking to validate partition accesses.
* Additional changes to ensure checking of partition during polling operations.
* Add change log.
* More cleanup.
* Change recommended in code review.
---------
Co-authored-by: ianmarshall <ian@simpatico.ai>
* fix test
* fixed
* changelog
* add another test
* improve docs
* fix change log
* fix test
* fix test
---------
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* Start work optimizing transaction
* Tons of test cleanup
* Cleanup
* More optimization
* Optimize
* Many tests fixed
* Work on test fixes
* Optimization done, now doing cleanup
* Cleanup
* Add docs
* Test fixes
* Test fix
* License headers
* Test fix
* Test cleanup
* Test fix
* Introduced new interface, refactored SearchParamWithInlineReferencesExtractor, added new test
* clean up, remove unused param
* added change log
* moved error message to properties file, made base class abstract
* fixed issue with test
* code review changes
---------
Co-authored-by: Steven Li <steven@smilecdr.com>
* Start work on refchains
* Semi working
* Add tests for transactions
* Add docs and lots of tests
* Add changelog
* Add tests
* Work on cleanup
* Add document operations
* Test fix
* Test fix
* Fixes
* Fix typo
* Test fix
* Test update
* Test updates
* Test fix
* Test fixes
* Test additions
* Test fix
* Add some javadocs
* Test fixes
* Intermittent test fix
* Doc tweak
* Test fixes
* Merge master in
* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
* Review comments
* Version bump
* Add license
---------
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
* First commit with new maven deps for envers and stubbed changelist.
* Add repositoryFactoryBeanClass = EnversRevisionRepositoryFactoryBean.class to BaseAppCtxPersistence. Comment it out in JpaConfig. Add constant for hibernate.integration.envers.enabled. Hard-code envers to false. App starts without error. Basic smoke testing for MpmLink update works.
* Add mdmLink code to retrieve envers history and unit test it.
* Initial code to handle envers from HapiFhirEnversRevision along with a custom extension of the revision/REVINFO entity.
* Add changelist and tweak table/sequence/field names and clean up the HapiFhirJpaMigrationTasks method.
* Fix some unit test failures and address TODOs.
* Ensure hard-coded deactivation of envers is overridden by unit tests.
* Remove commented-out code.
* Address code review comments.
* Remove sysout traces.
* Tweaks to poms with comments explaining why I left the versions in the plugin dependencies. Get rid of a duplicate envers dependency.
* Bump hapi-fhir version to 6.5.5-SNAPSHOT.
* Disable intermittently failing test: PartitionedSubscriptionTriggeringR4Test.testCreateSubscriptionInPartitionAndResourceInDifferentPartition
* Try disabling Batch2CoordinatorIT failing test and see if branch pipeline passes.
* Add back hapi-fhir-validation-resources-r4b to hapi-fhir-jpaserver-base pom.xml.
* Bump hapi-fhir to 6.5.6-SNAPSHOT.
* Bump rest of hapi-fhir to 6.5.6-SNAPSHOT.
* Fix compile error on Batch2CoordinatorIT.
* Change revision ID column from integer to long in both the revision entity and the migration tasks. Fix a bug with the migration tasks having the wrong name for the timestamp column.
* Fix interface in MongoDB to return Long for the revision ID instead of Int. Implement TINYINT for all supported database products. Keep TARGET_TYPE column at 40 instead of 100 for now since otherwise the schema migration test will fail. Also, don't try to rename the foreign key or the schema migration test will fail.
* Creating test to reproduce the issue.
* rebasing development branch from release to master.
* adding migration task.
* providing solution to duplicate resource creation and passing initial test.
* making initial test more readable.
* Implementing test for scheduled deletion of stale searchUrl entries.
* Implementing test for scheduled deletion of stale searchUrl entries.
* Removing interface declaration since it was not needed.
* Reverting changes.
* Adding changelog.
* Adding comments for clarity.
* Adding purgeDatabase operation in IT to delete all ResourceSearchUrlEntity [before|after}Each tests.
* Adding pom.xml exclusion to get all tests passing.
* Adjusting query counts and adding pointcut name to log in interceptor.
* Fixing typos.
* WIP
* implementing edge case tests and solutions.
* wip: passing all tests.
* wip: passing more tests .
* wip: passing tests.
* wip: passing tests.
* enhancing the test.
* changes following code review.
* modifications to tests.
* minor modifications following second code review.
* passing all tests.
---------
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
* One more fix for #4467
* Enabling massIngestionMode causes incomplete resource deletion (#4476)
* Adding initial test.
* Adding fix and subsequent test.
* Adding changelog.
---------
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
* Provide the capability to request that the name of the subscription matching channel be unqualified (#4464)
* Adding initial test.
* Adding initial solution implementation.
* Adding change log and code clean up.
* addressing comments from 1st code review.
---------
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
* Change visibility of migration method (#4471)
* change migration visibility
* add empty migration method for 640
---------
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
* Fix subscription validation not to validate partition ID when invoked from an update pointcut (#4484)
* First commit: Make SubscriptionValidatingInterceptor aware of which Pointcut is being called. In validatePermissions(), skip determinePartition() if the Pointcut is STORAGE_PRESTORAGE_RESOURCE_UPDATED. Fix resulting compile errors in various unit tests.
* Fix/enhance unit tests. Mark methods as deprecated instead of deleting them. Add proper error code. Complete changelog.
* Remove erroneous TODOs and tweak the validation logic.
* Enhance unit tests and fix changelog.
* Reindex batch job fails when processing deleted resources. (#4482)
* adding changelog.
* Providing solution and adding changelog.
* Adding new test.
---------
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
* cleaning up checkstyle files (#4470)
* cleaning up checkstyle files
* One more fix for #4467 (#4469)
* added exlusions for files at base project level so checkstyle doesn't error out
* duplicate error code from merge
* changing lifecycle goal for all module checkstyle check
* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code
* wip
* trying to figure out why pipeline cannot copy files
* removing modules that don't actually need to be built.
* I messed up the version
---------
Co-authored-by: James Agnew <jamesagnew@gmail.com>
* Bump core to 5.6.881 (#4496)
* Bump core to 5.6.881-SNAPSHOT
* Work on fixing tests
* Work on fixing tests 2
* Bump to core release
---------
Co-authored-by: dotasek <david.otasek@smilecdr.com>
* Issue 4486 mdm inconsistent possible match score values (#4487)
* Extract method for readability
* Save always normalized score values in POSSIBLE_MATCH links.
* Avoid setting properties to null values. Adjust test.
* Simplify fix
* Fix test. Add RangeTestHelper.
---------
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
* Revert "cleaning up checkstyle files (#4470)"
This reverts commit efae3b5d5f.
* core version fix
* Loosen rules for id helper
* License
* fix batch2 reduction step (#4499)
* fix bug where FINALIZE jobs are not cancellable
* moved reduction step to message hander
* moving reduction step to queue
* addingchangelog
* cleaning up
* review fixes
* review fix'
---------
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
* Scheduled batch 2 bulk export job and binary delete (#4492)
* First commit: Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.
* Refine solution and add a concrete unit test but still work to do.
* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.
* Unit tests complete. New Msg code for new IJobPersistence.fetchInstances() method. Cleanup TODOs and add others.
* Finish final touches on implementation.
* Add changelog.
* Various cleanup.
* Code review feedback.
* Small tweak to changelog.
* Last code review tweak.
* Address more code review comments.
* Reverse changes to consider work chunks. Add a constant for write-to-binary.
* Change bulk import test for valueUri type (#4503)
* change tests
* suggested test change
* CVE resolutions (#4513)
* Bump Postgres for CVE
* Bump jetty
* Verison bump
* Remove comments
* Revrt bump
* Add check in scanner (#4518)
* 4516 create hapi fhir cli command to clear stale lock entries (#4517)
* Initial implementation
* better tests
* Add changelog and docs
* Forgotten files
* Code review comments
* Fix checkstyle
* Unable to Expunge CodeSystem (#4507)
* changes for GL-3943
* changes for GL-3943
---------
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
* New line::
* Update to documentation regarding narrative generation; (#4521)
Providing changelog;
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
* changed what score is set for mdmlinks that created new golden resource (#4514)
* changed what score is set for mdmlinks that created new golden resource
* fix test
---------
Co-authored-by: Long Ma <long@smilecdr.com>
* REVERT: change to operationoutcome.html
* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNo… (#4527)
* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp
* added change log
---------
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
* fix build (#4530)
* Making narrative_generation.md reference an html snippet (#4531)
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
* fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)
* fix + test
* minor fix
* Addressing suggestion
* Minor changes
* 4441 rel 6 4 bad references creation bug (#4519)
* adding a test
* fail in the case of ref enforce on type and on write and autocreate are all true
* update to code
* removing a line
* cleanup
* removing check on urn
* changing just to trigger a build
* adding a comment to the pom
* updating test for better information
---------
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
* fixed channel import null pointer exception from null header (#4534)
* retryCount 0 on null header + test + changelog
* suggested changes
* Revert "fixed the issue of meta.source field inconsistently populated in subscription messages for different requests (#4524)" (#4535)
This reverts commit 53252b8d15.
* Better error handling for when channel type is not supported (#4538)
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
* Avoid logging message payloads that contain sensitive data (#4537)
Don't log payloads - they may contain sensitive data.
* Bulk Export Bug With Many Resources and Low Max File Size (#4506)
* failing test
* fix + changelog
* tweak
* add method to IJobPersistence to use a Stream
* tweak
* tweak
* decrease test time
* clean up
* code review comments
* version bump
* Increase timeout limit to match BulkExportUseCaseTest
* shorten test
* maintenance pass
* add logging
* Revert "add logging"
This reverts commit b0453fd953.
* Revert "maintenance pass"
This reverts commit bbc7418d51.
* test
* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp
* shorten tests
* logging
* move test location
* fixes a regression caused my change in hapi-fhir
* timeout
* Revert "fixes a regression caused my change in hapi-fhir"
This reverts commit 4b58013149.
* testing
* Revert "testing"
This reverts commit aafc95c2f3.
---------
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
* bump ver
* License updates'
* Downgrade dep'
* Updating version to: 6.4.1 post release.
* Add javadocs and sources to our serviceloaders
* Reset version
* Change parent
* Remove bumped version
* License fixes, new parent
* Updating version to: 6.4.1 post release.
* Fix bad creation of versionenum
* Improve performance on bulk export
* Add changelog
* Start working on cleaned up reducer
* Clean up batch calls
* Work on issues
* Build fixes
* typedbundleprovider getallresources override (#4552)
* typedbundleprovider getallresources override
* added test to immunization tests that validates pagination does not result in null pointer if over default queryCount
* moved HapiFhirDal test to its own test class
* removed unused imports
* Update to use JpaStorageSettings
* adding changelog for issue 4551
* fix changelog spacing
* changelog type to fix
---------
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
* Add backport info
* Upgrade core to 5.6.97, make adjustments in hapi-fhir, and ensure that all tests pass (#4579)
* First commit: Create new branch from the release branch with changes from James' branch. This probably won't compile as the work is incomplete.
* Second round of changes from integrating James' branch.
* Mark most test failures with TODOs.
* Add whitespace
* Add changes to FhirPathR4 to set FHIRPathEngine to non-strict FP evaluation.
* Fix CreatePackageCommandTest to assert null instead of empty string. Comments on tests that fail due to the double-quote encoding problem.
* Downgrade to core 5.6.97.
* Fix another test and remove TODOs.
* Fix changelog.
* Clean up some erroneous changes and TODOs.
---------
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
* Fix up dal test
* Address leftover code review feedback from the upgrade to core 5.6.97. (#4585)
* Exclude pinned core deps
* Force pin structs
* Add model changes to IBaseCoding and related changes (#4587)
* Add IbaseCoding changes, and tinder changes
* Fix up tag definition
* converter addition
* Fix unit test and add changelog.
* Add jira to changelog.
---------
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
* Fix changelog
* Tidy metadata
* Test fixing
* Test fixes
* Work on progress
* Add changelog
* Build tweak
* Fixes
* Disable intermittently failing tests. (#4593)
* rename tests to IT
* Disable more intermittently failing tests (#4595)
* Disable more intermittently failing tests.
* Disable another intermittently failing tests.
* ITify
* Disable yet another intermittently failing tests. (#4596)
* disable
* disables
* Fixes
* disables
* Fix compile
* Test fixes
* Updating version to: 6.4.2 post release.
* Bump to 6.4.2-SNAPSHOT
* Fix compile
* fix version
* Address review comments
* Review comments
* Version bump
* Compile fix
* Test fixes
* Compile fixes
* One more compile fix
* Test fixes
* Version bump
* Resolve fixme
---------
Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com>
Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: Nathan Doef <n.doef@protonmail.com>
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Luke deGruchy <luke.degruchy@smilecdr.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: dotasek <dotasek.dev@gmail.com>
Co-authored-by: dotasek <david.otasek@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
Co-authored-by: samguntersmilecdr <123124187+samguntersmilecdr@users.noreply.github.com>
Co-authored-by: Isaac Wen <76772867+isaacwen@users.noreply.github.com>
Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Qingyixia <106992634+Qingyixia@users.noreply.github.com>
Co-authored-by: KGJ-software <39975592+KGJ-software@users.noreply.github.com>
Co-authored-by: kylejule <kyle.jule@smilecdr.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com>
Co-authored-by: justin.mckelvy <justin.mckelvy@smilecdr.com>
Co-authored-by: Jonathan Percival <jonathan.i.percival@gmail.com>
Add optimistic write lock to $reindex operation to prevent indexing stale data.
Introduce Lockstep Phaser to make testing easier
Add 1 query to every resource reindex :-(
* fix + test
* minor fix
* Addressing suggestion
* Minor changes
* Revert changes, add new test clas, new test consumer for queus
* Finish implementation
* fix compilation
* Remove unused method and test from the previous solution
* fix + test
* minor fix
* Addressing suggestion
* Minor changes
* Revert changes, add new test clas, new test consumer for queus
* Finish implementation
* fix compilation
* Remove unused method and test from the previous solution
* More Renaming
* fix compilation failure
* Revert compilation fix and merge master
* Fix to pipeline test failures
* Addressing suggestions
---------
Co-authored-by: qingyixia <cherry.xia@smilecdr.com>
David
JasonRoberts-smile
Tadgh
James Agnew
michaelabuckley
TipzCM
Ken Stevens
Etienne Poirier
Luke deGruchy
Justin McKelvy
StevenXLi
IanMMarshall
Jens Kristian Villadsen
samguntersmilecdr
Aditya Dave
Nathan Doef
Qingyixia