4020 Commits

Author SHA1 Message Date
michaelabuckley
428fb14f0b Make WorkChunk handling transactional at state transitions. (#4621)
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
2023-04-25 17:13:26 -07:00
James Agnew
e3a1c4eb1c Test fixes 2023-04-25 14:32:01 -04:00
James Agnew
42241a6625 Reduce log level 2023-04-25 05:29:06 -04:00
James Agnew
4013c08650 Query updates 2023-04-24 17:15:52 -04:00
James Agnew
fdbc0fd0c6 Increase fetch size 2023-04-24 16:10:06 -04:00
James Agnew
71a61489f4 Add temporary distinct clause 2023-04-24 06:59:57 -04:00
James Agnew
6975e26814 Reduce load on DB for bulk export 2023-04-22 16:11:17 -04:00
James Agnew
d952c87d9b Rework reindex 2023-04-19 15:09:12 -04:00
James Agnew
fd11cb0777 Cleanup 2023-04-19 13:31:18 -04:00
Tadgh
c10d356273 New method on DAO. bad design, but lets try it 2023-04-18 17:54:33 -07:00
Tadgh
ff1edd586c Shorten schedule 2023-04-18 14:38:49 -07:00
James Agnew
cae75767c5 Begin working on parallel processing 2023-04-18 16:01:08 -04:00
James Agnew
88d16de4f2 Fixes 2023-04-17 18:57:59 -04:00
James Agnew
89cf484dcb Merge tests 2023-04-17 08:57:12 -04:00
James Agnew
1db4d5f2e0 Improve Subscription Retriggering Efficiency (#4742)
* Improve subscription efficiency

* Reduce number of queries

* Add changelog

* Fix ITs

* Review comments

* Test fix
2023-04-17 08:57:06 -04:00
markiantorno
df8d2cd797 Updating version to: 6.4.5 post release. 2023-03-21 20:59:13 +00:00
Kevin Dougan SmileCDR
107a1bd073
Fixed migration step for MS SQL Server (#4652)
* Fixed migration step for MS SQL Server.

* Implemented the fix the RIGHT way this time.
2023-03-17 08:45:42 -06:00
longma1
17cb2e106b fix mistakes when cherry-picking 2023-03-15 12:03:59 -06:00
longma1
4ac490e14c Merge branch 'rel_6_4' of https://github.com/hapifhir/hapi-fhir into rel_6_4 2023-03-15 11:19:34 -06:00
Etienne Poirier
5f7ccb3ac1 backported pr 4609, fixed merge conflicts 2023-03-15 11:19:28 -06:00
markiantorno
47d7809037 Updating version to: 6.4.4 post release. 2023-03-15 16:07:25 +00:00
Kevin Dougan SmileCDR
5eca889296
4636 - Fixed Oracle syntax problem with fix provided via PR 4630. (#4637)
* 4636 - Fixed Oracle syntax problem with fix provided via PR 4630.

* 4636 - Added a changelog file for this fix.
2023-03-10 08:45:48 -05:00
longma1
6fa127073b Merge branch 'rel_6_4' of https://github.com/hapifhir/hapi-fhir into rel_6_4 2023-03-07 12:42:49 -07:00
michaelabuckley
efe1be424a
Provide default update time to batch2 entities (#4630)
* Data migration - default value for batch2 UPDATE_TIME
2023-03-07 13:35:09 -05:00
longma1
c83dc290b4 set version back to 6.4.3 2023-03-07 09:48:32 -07:00
michaelabuckley
02169f9a6b
Bump to 6.4.3-SNAPSHOT 2023-03-04 12:58:21 -05:00
michaelabuckley
d5ebd1f667
Avoid fetching work-chunk data (#4622)
* Replace work-chunk maintenance query with projection to avoid fetching clob data.
* Add processing logging
2023-03-04 12:12:04 -05:00
Tadgh
c51aa39d8f Set back to 6.4.2 2023-02-27 16:27:59 -08:00
michaelabuckley
7a4a8c97a5 Simultaneous $reindex and DELETE can orphan index rows (#4584)
Add optimistic write lock to $reindex operation to prevent indexing stale data.

Introduce Lockstep Phaser to make testing easier
Add 1 query to every resource reindex :-(
2023-02-27 16:17:34 -08:00
Luke deGruchy
9d70dbd08f
Add UserSelected and Version to resource tag (#4591)
* First commit:  Add userSelected and version to ResourceTag and ResourceHistoryTag.  Add migration step to migration tasks.  For some reason SchemaMigrationTest is failing.

* Fix nullability, update constructors, truncate versions over 30 chars.

* yay we're done just cleanup

* Checkpoint for recent changes.

* Last commit before pausing work.

* Fix bad code to find the version, tweak unit test pid ID to versionless and add TODO about why is update part of the test not passing?

* Fix bad code in extractTagsRi():  Make it leverage the IBaseCoding variable already in the for loop.  Fix bad code in populateResourceMetadataRi().   Leverage the existing BaseTag and use it to populate the IBaseCoding tag instance instead.  Add more and better assertions in the unit test.

* Merge rel_6_4 in

* Current state for pairing

* Fix CodingDt.getUserSelected method return code in tinder

* Complete migration task and cleanup

* Remove missed fixme

* Bump version temporarily to use it for building core lib

* Use primitive for compatibility with tinder generated CodingDt

* Copyright updates

* Hack to bypass NPE generated by bogus tinder-generated code

* Simplify by removing temporary deprecations as version bump is needed anyway because of TagDefinition entity changes

* Adjust tests

* Fix migration task

* Add test to show problem with tinder generated BooleanDt

* More test adjustments

* Improve BooleanDt NPE avoidance

* Update core dep

* Reverse Juan CVE VersionEnumTest changes.

* Fix unit test.

---------

Co-authored-by: Ken Stevens <ken@smilecdr.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-02-27 09:58:27 -08:00
James Agnew
3a8b42dc39
Improve bulk export performance (#4569)
* Improve performance on bulk export

* Add changelog

* Start working on cleaned up reducer

* Clean up batch calls

* Work on issues

* Build fixes

* Test fixing

* Test fixes

* Work on progress

* Add changelog

* Build tweak

* Fixes

* Fixes

* Fix compile

* Test fixes

* Bump to 6.4.2-SNAPSHOT

* Fix compile

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-02-27 07:32:39 -05:00
markiantorno
31610cd8fb Updating version to: 6.4.2 post release. 2023-02-26 22:29:48 +00:00
Luke deGruchy
2c34bf074f
Upgrade core to 5.6.97, make adjustments in hapi-fhir, and ensure that all tests pass (#4579)
* First commit:  Create new branch from the release branch with changes from James' branch.  This probably won't compile as the work is incomplete.

* Second round of changes from integrating James' branch.

* Mark most test failures with TODOs.

* Add whitespace

* Add changes to FhirPathR4 to set FHIRPathEngine to non-strict FP evaluation.

* Fix CreatePackageCommandTest to assert null instead of empty string.  Comments on tests that fail due to the double-quote encoding problem.

* Downgrade to core 5.6.97.

* Fix another test and remove TODOs.

* Fix changelog.

* Clean up some erroneous changes and TODOs.

---------

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2023-02-23 15:32:13 -08:00
markiantorno
827bbba1e2 Updating version to: 6.4.1 post release. 2023-02-17 04:29:53 +00:00
Tadgh
96855fb943 Reset version 2023-02-16 16:33:56 -08:00
markiantorno
b2644c1c60 Updating version to: 6.4.1 post release. 2023-02-16 23:33:58 +00:00
Tadgh
a0e4c2949e License updates' 2023-02-10 20:51:17 -08:00
Tadgh
9578610e15 bump ver 2023-02-10 20:42:38 -08:00
Nathan Doef
9754a72046
Bulk Export Bug With Many Resources and Low Max File Size (#4506)
* failing test

* fix + changelog

* tweak

* add method to IJobPersistence to use a Stream

* tweak

* tweak

* decrease test time

* clean up

* code review comments

* version bump

* Increase timeout limit to match BulkExportUseCaseTest

* shorten test

* maintenance pass

* add logging

* Revert "add logging"

This reverts commit b0453fd9538d6858e6bd212f402d4b04c6306136.

* Revert "maintenance pass"

This reverts commit bbc7418d519260c4b9827546a0bf822db1c78b1b.

* test

* trying to fix BulkDataExportTest testGroupBulkExportNotInGroup_DoesNotShowUp

* shorten tests

* logging

* move test location

* fixes a regression caused my change in hapi-fhir

* timeout

* Revert "fixes a regression caused my change in hapi-fhir"

This reverts commit 4b58013149836eededc568d295c5baf8fb3df989.

* testing

* Revert "testing"

This reverts commit aafc95c2f36bf89ae935c6c342017ec300fe4289.

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-02-10 13:14:29 -05:00
TipzCM
fe078c2884
4441 rel 6 4 bad references creation bug (#4519)
* adding a test

* fail in the case of ref enforce on type and on write and autocreate are all true

* update to code

* removing a line

* cleanup

* removing check on urn

* changing just to trigger a build

* adding a comment to the pom

* updating test for better information

---------

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-mbp.home>
2023-02-09 14:48:59 -05:00
Isaac Wen
85ecbf181e
Unable to Expunge CodeSystem (#4507)
* changes for GL-3943

* changes for GL-3943

---------

Co-authored-by: isaacwen <isaac.wen@smilecdr.com>
2023-02-07 15:26:10 -05:00
Luke deGruchy
2a963acde2
Scheduled batch 2 bulk export job and binary delete (#4492)
* First commit:  Scheduled batch 2 bulk export job delete and binary, incomplete mock-based unit test, and a mess of TODOs and code that needs to be deleted.

* Refine solution and add a concrete unit test but still work to do.

* Comment out code in cancelAndPurgeAllJobs() and see if it breaks the pipeline.

* Unit tests complete.  New Msg code for new IJobPersistence.fetchInstances() method.  Cleanup TODOs and add others.

* Finish final touches on implementation.

* Add changelog.

* Various cleanup.

* Code review feedback.

* Small tweak to changelog.

* Last code review tweak.

* Address more code review comments.

* Reverse changes to consider work chunks.  Add a constant for write-to-binary.
2023-02-06 09:04:04 -05:00
Tadgh
df83d05fbb Merge branch 'rel_6_4' of github.com:hapifhir/hapi-fhir into rel_6_4 2023-02-02 15:22:28 -08:00
Tadgh
a855af576d Loosen rules for id helper 2023-02-02 15:11:32 -08:00
Mark Iantorno
740fec9554 Revert "cleaning up checkstyle files (#4470)"
This reverts commit efae3b5d5f397ab8fd92d7aaa82b484cfdfb3c33.
2023-02-02 17:24:34 -05:00
Mark Iantorno
efae3b5d5f
cleaning up checkstyle files (#4470)
* cleaning up checkstyle files

* One more fix for #4467 (#4469)

* added exlusions for files at base project level so checkstyle doesn't error out

* duplicate error code from merge

* changing lifecycle goal for all module checkstyle check

* moving checkstyle to base pom file, changing exectution phase on base check, cleaning dependency, resolving duplicate error code

* wip

* trying to figure out why pipeline cannot copy files

* removing modules that don't actually need to be built.

* I messed up the version

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-02-01 18:00:45 -05:00
Etienne Poirier
dd8c8a3b24
Reindex batch job fails when processing deleted resources. (#4482)
* adding changelog.

* Providing solution and adding changelog.

* Adding new test.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-02-01 08:15:33 -05:00
Nathan Doef
882f22fdb0
Change visibility of migration method (#4471)
* change migration visibility

* add empty migration method for 640

---------

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
2023-01-31 10:19:25 -05:00
Etienne Poirier
1a9b749b5a
Enabling massIngestionMode causes incomplete resource deletion (#4476)
* Adding initial test.

* Adding fix and subsequent test.

* Adding changelog.

---------

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2023-01-30 07:34:50 -05:00
michaelabuckley
10060bef7f
Batch option processing cleanup (#4466)
* BulkDataExportOptions null safety
2023-01-26 11:50:45 -05:00