Commit Graph

28 Commits

Author SHA1 Message Date
Ken Stevens 134631fdee
Convert delete expunge to use Spring Batch (#2697)
* prepare to add $delete-expunge operation that will create a spring batch job

* Add operation

* Wire up jpa provider.  Begin with failing test.

* Copy/paste bulk import job as a starting point.
FIXME with proposed design

* delete expunge job parameter validation with test

* implemented reader
stubbed processor, writer

* wip for master merge

* started implementing reader

* started implementing reader

* working with stubs

* happy path batch delete expunge is done

* Provider done but test not passing.  Guessing batch infrastructure not running in that test.

* IT test works now

* add reader test

* Converted delete _expunge=true to use new batch job

* DeleteExpungeDaoTest passes

* Fix test

* Change batch size to integer

* rename search count to batch size

* Make delete expunge partition aware

* updated docs

* pre-review cleanup

* change log

* add partition id to SystemRequestDetails

* Make RequestPartitionId serializable

* Change delete expunge provider to use partition id instead of tenant name

* fix tests

* test pointcut gets called

* assert on pointcut calls

* Add resource type to STORAGE_PARTITION_SELECTED pointcut

* bump hapi-fhir version
move expunge provider parameters from JpaConstants to ProviderConstants

* bump hapi-fhir version

* copyrights

* restore deleteexpungeservice for mdm

* restore deleteexpungeservice for mdm

* fix test

* public constants

* convert instant to date

* Moved expunge constants to ProviderConstants

* final review

* disabling InMemoryResourceMatcherR5Test.testNowNextMinute() to see if I can get a clean test run

* fix tests

* fix tests

* fix tests

* fix tests

* review feedback

* review feedback

* review feedback

* review feedback

* review feedback

* review feedback

* improve logging

* bump version

* version bump

* recovering from failed merge

* unzip RequestListJson per Gary's suggestion.  I didn't want to do it at first, but as usual Gary was right.

* fix serialization
2021-06-15 10:36:05 -04:00
James Agnew b64346ba1e
Add shcheduled task enablement config (#2652)
* Add shcheduled task enablement config

* Add changelog

* Test fix

* Cleanup
2021-05-12 14:38:30 -04:00
James Agnew ca2088f3ad
Add framework for Bulk Import (#2538)
* Start work on bul;k import

* Work on bulk import

* Have batch working

* Working

* Working

* More work

* More work on bulk export

* Address fixmes

* License header updates

* Test fixes

* License header updates

* Test fix

* Test fix

* Version bumps

* Work on config

* Test cleanup

* One more version bump

* Version bump

* CLeanup

* A few additions

* Test fixes

* Test fix

* Test fix

* Migration fix

* Test fix

* Test fix
2021-04-14 17:41:32 -04:00
Tadgh 4d3b021f12 Add annotating processor and composite processor 2021-03-25 13:53:41 -04:00
jamesagnew d20b07c78e Update license headers 2021-01-01 15:50:24 -05:00
jamesagnew f30999b053 License header updates 2021-01-01 15:07:23 -05:00
James Agnew 72294fc0a0
Suport typeFilter parameter for bulk export (#2147)
* Cleanup

* Fix tests

* Work on bulk export

* Work on export

* Test fixes

* Add changelog

* Address review comments

* Address review comments
2020-10-26 12:28:07 -04:00
Tadgh 8f6aee64b1 Fix batch project to now use an async task launcher 2020-06-26 12:22:23 -07:00
Tadgh b31b6652a9 Add async to the job launcher, fix test to reflect this 2020-06-26 11:51:01 -07:00
Tadgh 484f390db0 Fix batch test 2020-06-25 13:00:59 -07:00
Ken Stevens c11b6eeab6 back out Tx wrap
use qualifier
2020-06-25 15:27:31 -04:00
Tadgh 4cccc9804b nop for CI 2020-06-23 16:29:36 -07:00
Tadgh d32ad74402 some renaming 2020-06-15 21:49:34 -07:00
Tadgh 0f469c1a56 Rework to allow for job creation inside of batch 2020-06-12 14:20:50 -07:00
Tadgh 4e42a59ad4 Add licenses 2020-06-10 11:52:45 -07:00
Tadgh 6fc5b753e5 Readding imports 2020-06-09 19:20:43 -07:00
Tadgh 942fc313ec Code review comments 2020-06-09 19:00:41 -07:00
Ken Stevens 57f2002b84 organize imports 2020-06-09 15:49:11 -04:00
Tadgh b1d2ab7619 Add invalid parameter validator 2020-06-09 11:02:37 -07:00
Tadgh 44aa688a20 Final config layout of Bulk Export Batch job 2020-06-08 15:59:08 -07:00
Tadgh 7d46b3cc10 Partitioning Based on resource type complete. Still no threading 2020-06-03 14:58:42 -07:00
Tadgh f8d699e13b Initial working commit passing jobs to spring batch 2020-06-01 15:51:09 -07:00
Tadgh 0fc2e04e65 wip 2020-06-01 12:13:55 -07:00
Tadgh aa7d1cbcb7 Wip getting tasks built 2020-06-01 11:32:36 -07:00
Tadgh a306825625 moving dependencies around. Get non-persisted batch working 2020-05-28 12:10:52 -07:00
Tadgh 81d5ff8f5a Add schema initialization 2020-05-27 17:06:52 -07:00
Tadgh 48fcf8a39d wip 2020-05-26 12:08:00 -07:00
Tadgh 35b1533db1 Add new project for spring batch 2020-05-25 22:32:00 -07:00