Commit Graph

9444 Commits

Author SHA1 Message Date
Tadgh 0f720e00b1 update version 2023-05-04 09:23:12 -07:00
Tadgh d35708b18c add import 2023-05-01 08:25:58 -07:00
Tadgh a06b6225e1 Support `_exportId` for bulk exports. (#4781)
* Full implementation, test, changelogs

* Add changelogs

* Add default method

* Update hapi-fhir-storage-batch2-jobs/src/main/java/ca/uhn/fhir/batch2/jobs/export/WriteBinaryStep.java

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Code Review Comments

* Compilation failures

---------

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2023-05-01 08:18:53 -07:00
James Agnew 263f62b9fc Resolve fixme 2023-04-25 21:35:35 -04:00
James Agnew 310019588e Fix intermittent in batch2 tests (#4643)
* Fix intermittent in batch2 tests

* Add changelog

* Test fix

* Resolve intermittent

* Reducer changes

* Test fix

* Test fixes

* Resolve fixme

* Fix changelog
2023-04-25 17:54:48 -07:00
Tadgh 70e295cb5d Fix up processor 2023-04-25 17:46:18 -07:00
Tadgh a581656bd3 Fix up processor 2023-04-25 17:43:09 -07:00
Tadgh bce71da2b0 Fix up processor 2023-04-25 17:42:13 -07:00
Tadgh f20206ad29 Replace system request with a null request 2023-04-25 17:37:07 -07:00
michaelabuckley 9e214154a2 Batch stabilization (#4647)
* Use java event names for work chunk transitions.

* Cherry-pick d5ebd1f667 from rel_6_4

Avoid fetching work-chunk data (#4622)

* add end time to reduction step (#4640)

* add end time to reduction step

* add changelog

---------

Co-authored-by: Long Ma <long@smilecdr.com>
(cherry picked from commit 37f5e59ffc)

* Cancel processing

Provide error message in cancelled jobs, and avoid transitions in final states.

* Apply tx boundary to starting job and first chunk.

* cleanup

* Apply tx boundary to work chunk processing

* Delete BatchWorkChunk

* Introduce events for job create, and chunk dequeue

* Apply tx boundary to chunk handler

* Move instance cancellation to database

* tx boundary around stats collection and completion

* tx boundary around stats collection and completion

* Extend tx boundary to error, fail, and cancel

* Move failure into status calc

* ERROR is not an "ended" state.

* Revert generics cleanup to avoid noise

* Avoid sending gated chunks twice.

* Make no-data path safer.  Cleanup

* Fix mock test for step advance.

* Delete unsafe updateInstace() call

* Cleanup

* Changelog and notes

* Fix cancel boundary.  Cleanups

* Cleanup

* Sort mongo chunks for stable paging.

Other cleanup

* Document error handling

* Cleanup

* Update hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/Batch2JobHelper.java

Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>

---------

Co-authored-by: longma1 <32119004+longma1@users.noreply.github.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
2023-04-25 17:36:30 -07:00
michaelabuckley cc2d6a3ad9 Use java event names for work chunk transitions. (#4658)
Change work-chunk creation to be first event in state machine.
2023-04-25 17:19:54 -07:00
Tadgh 670dd74f02 Cherry pick some pom fixes 2023-04-25 17:17:17 -07:00
michaelabuckley 428fb14f0b Make WorkChunk handling transactional at state transitions. (#4621)
- use a separate enum for the states - chunks have different transitions than instances.
- use transactional update events for work-chunk state transitions
- introduce spec-test to define behaviour of batch2 storage
- replace synchronized facade with simpler ProxyUtil handler.
- change job cancellation to db update query
2023-04-25 17:13:26 -07:00
James Agnew e3a1c4eb1c Test fixes 2023-04-25 14:32:01 -04:00
James Agnew 93ba9af99c Resolve merge conflict 2023-04-25 09:33:17 -04:00
James Agnew 576a6b7cc9 Add post-fetch filtering to bulk export 2023-04-25 09:33:12 -04:00
James Agnew 42241a6625 Reduce log level 2023-04-25 05:29:06 -04:00
James Agnew 4013c08650 Query updates 2023-04-24 17:15:52 -04:00
James Agnew fdbc0fd0c6 Increase fetch size 2023-04-24 16:10:06 -04:00
James Agnew 71a61489f4 Add temporary distinct clause 2023-04-24 06:59:57 -04:00
James Agnew 6975e26814 Reduce load on DB for bulk export 2023-04-22 16:11:17 -04:00
James Agnew 748e86cb04 Updates 2023-04-20 15:26:40 -04:00
James Agnew d952c87d9b Rework reindex 2023-04-19 15:09:12 -04:00
James Agnew 27e5c41498 Add param to operation 2023-04-19 13:43:51 -04:00
James Agnew da0ce64a74 Merge branch 'hacked-operation-flag' into ja_2030417_backport_subscription_opt_to_6_4 2023-04-19 13:32:07 -04:00
James Agnew fd11cb0777 Cleanup 2023-04-19 13:31:18 -04:00
James Agnew 445a86157b License header 2023-04-19 05:36:30 -04:00
Tadgh 8b751aa30f wip 2023-04-18 18:11:59 -07:00
Tadgh c10d356273 New method on DAO. bad design, but lets try it 2023-04-18 17:54:33 -07:00
Tadgh ff1edd586c Shorten schedule 2023-04-18 14:38:49 -07:00
James Agnew ffedab5524 Parallize subscription triggering 2023-04-18 16:39:45 -04:00
James Agnew cae75767c5 Begin working on parallel processing 2023-04-18 16:01:08 -04:00
James Agnew 88d16de4f2 Fixes 2023-04-17 18:57:59 -04:00
James Agnew aea88b0839 Subscription fix 2023-04-17 10:54:06 -04:00
James Agnew 89cf484dcb Merge tests 2023-04-17 08:57:12 -04:00
James Agnew 1db4d5f2e0 Improve Subscription Retriggering Efficiency (#4742)
* Improve subscription efficiency

* Reduce number of queries

* Add changelog

* Fix ITs

* Review comments

* Test fix
2023-04-17 08:57:06 -04:00
markiantorno df8d2cd797 Updating version to: 6.4.5 post release. 2023-03-21 20:59:13 +00:00
Kevin Dougan SmileCDR 107a1bd073
Fixed migration step for MS SQL Server (#4652)
* Fixed migration step for MS SQL Server.

* Implemented the fix the RIGHT way this time.
2023-03-17 08:45:42 -06:00
longma1 1bd8f0a492 added new version changelogs 2023-03-15 12:16:50 -06:00
longma1 17cb2e106b fix mistakes when cherry-picking 2023-03-15 12:03:59 -06:00
longma1 4ac490e14c Merge branch 'rel_6_4' of https://github.com/hapifhir/hapi-fhir into rel_6_4 2023-03-15 11:19:34 -06:00
Etienne Poirier 5f7ccb3ac1 backported pr 4609, fixed merge conflicts 2023-03-15 11:19:28 -06:00
markiantorno 47d7809037 Updating version to: 6.4.4 post release. 2023-03-15 16:07:25 +00:00
Ken Stevens cfc9af8148 fix regression 2023-03-14 09:53:24 -06:00
longma1 37f5e59ffc
add end time to reduction step (#4640)
* add end time to reduction step

* add changelog

---------

Co-authored-by: Long Ma <long@smilecdr.com>
2023-03-10 15:01:28 -07:00
Kevin Dougan SmileCDR 5eca889296
4636 - Fixed Oracle syntax problem with fix provided via PR 4630. (#4637)
* 4636 - Fixed Oracle syntax problem with fix provided via PR 4630.

* 4636 - Added a changelog file for this fix.
2023-03-10 08:45:48 -05:00
longma1 b450729ff9 fixed changelog, added 6.4.3 changelog folder 2023-03-08 09:56:27 -07:00
longma1 6fa127073b Merge branch 'rel_6_4' of https://github.com/hapifhir/hapi-fhir into rel_6_4 2023-03-07 12:42:49 -07:00
michaelabuckley efe1be424a
Provide default update time to batch2 entities (#4630)
* Data migration - default value for batch2 UPDATE_TIME
2023-03-07 13:35:09 -05:00
longma1 c83dc290b4 set version back to 6.4.3 2023-03-07 09:48:32 -07:00