* base batch2 bulk export steps - unimplemented
* added the basic steps
* adding tests
* we have added the tests
* added tests for bulk export stages
* implementing jpa
* 3531 adding the jpabukexportprocessorimplementation
* 3531 first step for mongo bulk export
* jpa stuff updated
* 3531 hapi fhir stuff
* trying to fix up tests
* fixing test
* fixed most tests in batch export
* deprecate
* need some more guidance here
* fixing some tests
* some steps taken
* blah
* stashing
* test
* fixing tests round 1
* many a changes
* updating version
* small updates
* merging in mster again
* fixing up serialization
* added the permission checkers
* cleanup and sonarkube crap
* some more tests
* updating adding tests
* crimes
* blah
* updatingmsg number again
* removal of commented code
* cleanup
* merge confflicts
* fixing msg
* review fixes round 1
* more review fixes
* merging in master
* fixing poms
* review points
* test fix
* test fix
* fix test race condition
* truncate error
* add stack trace to error
* rename to IT
* fix race condition in test
* finally fixed race condition in test. this time I'm optimistic hapi ci will finally pass
* fix mock test. not a fan of mock tests like this.
* any job definition with a reducer step must be gated.
* fixed common batch bean wiring
* fix test
* lazy init ResponseTerminologyTranslationSvc
* change the way we lazy init ResponseTerminologyTranslationSvc
* change the way we lazy init ResponseTerminologyTranslationSvc (for the third time. this is super tricky!)
* disabling testGoldenResourceWithCircularReferenceCanBeCleared for now
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* 3770 fixing error handling
* some updates to error handling
* changelog
* blah
* bumping version
* fix test failures
* code review points
* review points
* add retry test
* assert counts in the test
* test fixes
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* added failing test for Map Response Terminology for bulk export
* moved logic for terminology translation out of the interceptor to svc, added translation logic to bulk export file writer, added more tests
* added changelog
* Wip refactor dependency fix
* wip terminology translation code refactor
* terminology translation code refactor
* refactor clean up
* Remove circular dependency
* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_1_0/3719-enable-map-response-terminology-for-bulk-export.yaml
* bump version of hapi to 6.1.0-PRE7-SNAPSHOT
* doc update
* fix issue when search param registry doesn't contain search param for the resource type
* fix beans
* hapi version bump from PRE7 to PRE10
* doc update
* add ResponseTerminologyTranslationSvc as a constructor variable to the interceptor
* fixed ResponseTerminologyInterceptorLoaderTest, added IT for cdr side
* version bump to PRE13
* removed unused import
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* begin with failing test
* begin with failing test
* remove unused Spring Batch classes.
convert service classes
* move provider
* just one test left to fix
* fix test
* msg.code
* change log
* bumping hapi to PRE12
* fix test
* review feedback
* license update
* fix intermittent and status update atomicity
* restore url-level partition selection
* fix tests
Co-authored-by: Ken Stevens <ken@smilecdr.com>
- Added functionality to optionally filter out deleted resources when resolving forced ids to persistent ids
- Bumps version
Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* rename field that drives everything includes
* discovered design flaw in completion
* fix intermittent
* ensure batch2 completion handler can't be called twice
* changelog
* changelog
* javadoc
* bump hapi version
* licences
* Batch2 job definitions can now optionally provide an error handler callback that will be called when a job
instance fails, errors or is cancelled.
* Changed error handler and completion handler to take a "read-only" copy of an instance instead of just certain fields.
* comment
* javadoc
* fix test
* change step execution context from taking instance id to instance
* review feedback
* fix test
* replace 20 or so jetbrains annotations
* fixed a few incorrect @NotNull annotations
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* first test passes. onto the next one
* two down two to go
* three down one to go
* licenses
* basic admin-json plumbing confirmed with IT
* FIXME
* FIXME
* FIXME
* FIXME
* changelog
* bump hapi version to 6.1.0-PRE6-SNAPSHOT
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* Initial implementation
* Implement job cancellation
* Works for date and token. Cleanup required.
* Works for string and quantity.Cleanup pending.
* Works for reference and perform cleanup.
* Cleanup configuration and add missing tests
* Eliminate sort property registry (no go for a cluster) for a static property map.
* Missed in previous commit
* Implement HSearch number parameter
* Implement HSearch number sorting
* Unify different hibernate search names (HibernateSearch, Freetext and Lucene) to HSearch
* Move analysis configures for lucene and elastic to same file to enforce the need to keep them in synch
* Use new defined constant
* Rename package too
* Rename additional elastic references
* bump version to 6.1.0-PRE5-SNAPSHOT
* Restore Hap prefix to hapi configurers
* fixing hapi version of new maven module
* fix pom
* fix pom
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* begin with failing test
* continue building out
* continue building out
* reworking chunk api
* replace ChunkId zippers with proper iterator
* progressing test
* yay test passes!
* switch to idchunk interface
* fixmes
* compiles
* fixme
* fixme
* rename classes
* rename json classes with json suffix
* work towards homogeneous chunks
* fixme
* fixme
* unit test
* fixme
* fixme
* pull out builder
* moar tests
* moar tests
* preserve order
* moar test
* moar test
* fixme
* fixme -> wip to start ci
* msg code
* msg code fix test
* fixme
* change provider implementation
* final WIPs
* IT failing
* IT passes
* java 11
* fix test
* fix test
* fix test
* changelog
* use batch size if provided
* last WIP
* fix test
* fix test
* fix test
* whitespace change to trigger new build
* fix test
* bumping hapi version
* bumping hapi version
* fix test
* review feedback
Co-authored-by: Ken Stevens <ken@Kens-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* adding a bunch of stuff
* most of the steps are working now
* writing more tests
* adding more tests
* updated the batch2 coordinators
* updating version
* some minor test tweaks
* added a new test
* adding more tests
* blah
* blah
* review fixes 1
* fixing tests and refactoring
* review points
* updated for code review
* review fixes
* updated
* step 1 on review step 5
* final touches
* blah
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
* Updating With History Rewrite implementation
* Missed new files
* Updated changelog and documentation
* Fixed test failures and doc
* minor fix
* lgtm code refactor
* Fixed test to clear DaoConfig
* Removed unused class
* Code review changes
* Simplified updateHistoryRewrite's fluent, combining with normal update's, also changed logic on custom header error checking, updated tests
* refactored method to extracted common code
* removed `updateHistoryRewrite()` and replaced with `update()`, updated tests accordingly
Co-authored-by: Steven Li <steven@smilecdr.com>
* SRP refactor maintenance
* SRP refactor maintenance
* cache jobDefinition in jobInstance when processing (rather than looking it up in a bunch of different places)
* begin with failing test
* test passes
* tests pass. still need to add a few more tests
* add comment to trigger new ci
* added positive and negative tests for new behaviour
* fix IJ warnings
* change log
* change log
* fix test race condition
* resolved feedback
* review feedback
* fix config settings for system-level expunge
* clean up expunge everything interface
* extract common system provider
* changelog
* version bump
* touch changelog to force a fresh pipeline
* re-enable publishing of checkstyle
* back out Gary's Sonatype change entirely
* adding version.yaml, updating version in pom.xml
* It is possible to write to a resource in a partition the user is not authorized to. (#3397)
* fixed
* remove sout
* add msg.code
* fix failed tests
* fix equal sign
* update msg code
* extract method
* Fix up code numbers
* Clean changelog
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
* Revert final artifact version
* add graphql test (#3585)
* added graphql birthdate test
* fix variable name
* typo
* 3506 mdm log enhancement (#3543)
* Providing Fixme's to be reworked at a later time.
* Adding // FIXME Anna to assist our Austrian friend.
* Adding logging test as a first step in addressing issue #2822.
* hapifhir#3506 part 2: enhance logging for (un)successful MDM matching
hapifhir#3506 part 2: enhance logging for (un)successful MDM matching
* Update MdmMatchLinkSvc.java
#3506 move "narrowed down" log to different place
* #3506 added scores and tests
#3506 added scores and tests
* #3506 formatting
#3506 formatting
* #3506 create changelog file
#3506 create changelog file
* #3506 fix typo
#3506 fix typo
* #3506 fix part 3, minor formatting
#3506 fix part 3, minor formatting
* #3506 fix tests
#3506 fix tests
Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>
* mdm matching (#3579)
* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551)
* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328
* Update HapiFhirJpaMigrationTasks.java
Corrected ordering
* Update HapiFhirJpaMigrationTasks.java
Moving index status to be last operation
* Revert "Update HapiFhirJpaMigrationTasks.java"
This reverts commit 37bfd3e66e.
* Moved to bottom
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
* begin with failing test
* fixed
* changelog
* add jira tag
* Update to 6 1 (#3582)
* added changelog folder, upped version
* version enum
* add a few more unit tests to assert proper NO_MATCH exclusion
* revert merge master doh! bad reflexes
* revert merge origin master
* unrevert revert. ugh what a pain
* merge recovery. fix poms.
* merge recovery. more reverting
* merge recovery. more reverting
* merge recovery. more reverting
* Revert "merge recovery. fix poms."
This reverts commit ae6e0ddb06.
* more revert revert reversions
* more revert revert reversions
* pre-review cleanup
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
* Fix regression of 3411 - _lastUpdated gets clobbered during $reindex job (#3586)
* When chunking for the reindex job, don't clobber the lastUpdated if provided by the caller.
* License
* Ks 20220508 log colour (#3592)
* don't use colours when output is redirected to a file
* change log
* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551) (#3594)
* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328
* Update HapiFhirJpaMigrationTasks.java
Corrected ordering
* Update HapiFhirJpaMigrationTasks.java
Moving index status to be last operation
* Revert "Update HapiFhirJpaMigrationTasks.java"
This reverts commit 37bfd3e66e.
* Moved to bottom
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
* 3584 case sensitive string elasticsearch (#3596)
* Fix for case-sensitive search with newer elasticsearch
* Test and fix ascii normalization too
* more test cases
* comments
* Docs update empi usecase (#3598)
* Update diagram for use case 5
* Add new page for database support
* Licnese
* Mb norm fix (#3604)
* Normalize query since wildcard searches aren't normalized in elastic
* Handle batch2 job cancellation (#3603)
* Handle batch2 job cancellation
* Add CANCELLED status
* Remove unuseful test
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
* Add batch2 interfaces to obtain recent instances (#3601)
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
* changelog folder
* Documentation change, added warning log and change log for ticket (#3609)
Co-authored-by: Steven Li <steven@smilecdr.com>
* fix ne for lastUpdated search param (#3589)
* init/wip
* add implementation, add tests
* add checks and some refactoring
* refactor and simplify tests
* refactor
* get rid of some of the warnings
* add changelog
* remove my todos
* redo ne logic to account for less precise dates
* add tests
* refactoring
* rename to follow convention
Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
* Version bump
Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: katiesmilecdr <88786813+katiesmilecdr@users.noreply.github.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: alackerbauer <33912849+alackerbauer@users.noreply.github.com>
Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
* Implementation, parameterize test, changelog
* Bump spring framework for a vuln while i'm in here
* Fix number of changelog
* Add IS NOT NULL to query
* committed changes discussed at tasking
* Changed the subscription message to store the request partition
* modified mdm subscription loader to create a cross partition subscription in the default partition if multitenancy is enabled
* Fix errors when running Multitenant IT for mdm
* Added partition id to mdm golden resource search and creating/updating golden resource
* added partition id column to the mdm link table
* Added partition id of the source resource to partition id column of the mdm link table
* broken build
* working -links, missing hapi tests
* Fixed the mdm link db table due to feedback
* added IT for create link mdm operation, also added changes to pass the IT
* Fixed test for create mdm links, added test for update mdm link on partitioned server
* changed mdmLink search from searchbyExample to criteriaBuilder, added partitionHelperSvc to determine partition
* added test for merge golden resource and search golden resource on partitioned server
* added unit test for not duplicate on partitioned and non-partitioned server
* added criteriabuilder search and tests
* code cleanup
* added partition support to -merge-golden-resources operation
* test cleanup
* fixed operations with multitenancy
* added test for mdm clear operation
* added waiting for batch job to finish in mdm clear operation test
* added query to dao to support partitioned server for mdm clear operation
* resolve mistake from merge conflict
* added mdm-submit operations, also fixed paging for -query
* added partitionIds constant
* fixed build issue where there was no Code.msg in mdm partition exceptions
* code review fixes
* fix broken test, also fix a bug where query link could not query by linkSource properly
* fix test stubbing
* bump hapi version to 6.0.0-PRE10-SNAPSHOT
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
* moved classes
* Bump version
* licensing
* Repackage, fix messages
* reject empty streams
* Remove dead import
* Version bump
* Fix up test to actually return a data stream
* Move null binary svc to storage
* Improve batch2 error handling
* Build fixes
* Work on reindexing
* Test fixes
* Account for gzipped contents
* License header updates
* Register VS operations on public server
* Test fix
* Build fix
* Remove animal sniffer plugin
* Compile fix
* Start working on reindex job
* Updates
* Ongoing work
* Ongoing work
* Work on reindex
* Reindex work
* Add logging
* Fix typo
* Test fix
* Test fix
* Address fixme
* Liocense header
* Resolve fixme
* Add logging
* Update logs
* Address review comments
* License header
* Test fixes
* Test fix
* Test fixes
* Test fix
* Version bump
* Version bump
* Enable search narrowing on large ValueSets
* ValueSet improvements
* Work on narrowing
* Work on narrowing
* Work on narrowing
* Work on narrowing
* Add test
* Work on narrowing interceptor
* Work on narrowing
* License headers
* Refactor code narrowing
* Add docs
* Version bump
* Test fix
* Test fixes
* Build fix
* Fixes
* Version bump
* Test fix
* License header updates
* Build fix
* Test fixes
* Test fix
* Test fix
* Docs fix
* Test fix
* Test fix
* Resolve fixme
* Bump
* Force a CI build
* Make CI happen again
* Work on new batch framework
* Work on new batch framework
* Work on new batch framework
* Work on batch
* Compile working
* Work on bulk import
* Adjust import
* Work
* Work on batch
* Bump version
* WOrk on new batch processes
* Work on bath
* Bump to PRE4
* Build fixes
* CLeanup
* Small tweak
* Add exception code
* Test fixes
* Test fixes
* Test fix
* Additional synchronization
* Add license headers
* Test fixes
* Test fixes
* Add changelogs
* Address PG
* Test fix
* Test fix
* Test fixes
* Review notes
* Work on tests
* Test fixes
* Test fix
* Work on tests
* Tets fix
* Test fixes
* Test fixes
* Add missing exception codes
* Test fix
* Test fixes
* More test fixing
* License headers
* Test fix
* Add new test logging
* Work on tests
* Test fixes
* Test fix
* Resolve fixme
* Try to avoid test failure
* Add import command
* Work on storage
* Fix error codes
* Fixes
* License header
* Build fix
* Build fixes
* Fix dep
TipzCM
StevenXLi
Ken Stevens
James Agnew
Nathan Doef
jmarchionatto
JasonRoberts-smile
Tadgh
Mark Iantorno
longma1