Commit Graph

2583 Commits

Author SHA1 Message Date
Qingyixia 3d4556b70b
Move PATCH to hapi-fhir-storage (#3763)
* Not the final version

* Support PATCH operation

* minor modifications

* minor modifications

* Final version

* Final version
2022-07-11 14:56:22 +00:00
Ken Stevens 0526da080b
3757 move delete expunge to batch2 (#3759)
* begin with failing test

* begin with failing test

* remove unused Spring Batch classes.
convert service classes

* move provider

* just one test left to fix

* fix test

* msg.code

* change log

* bumping hapi to PRE12

* fix test

* review feedback

* license update

* fix intermittent and status update atomicity

* restore url-level partition selection

* fix tests

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-09 17:25:55 -04:00
Nathan Doef b68ccb373a
3693 Search by id is returning a deleted resource with a client-generated id (#3694)
- Added functionality to optionally filter out deleted resources when resolving forced ids to persistent ids
- Bumps version

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-05 12:37:25 -04:00
Ken Stevens 3a28920ea7
Add error-handler callback and ensure batch2 completion handler can't be called twice (#3752)
* rename field that drives everything includes

* discovered design flaw in completion

* fix intermittent

* ensure batch2 completion handler can't be called twice

* changelog

* changelog

* javadoc

* bump hapi version

* licences

* Batch2 job definitions can now optionally provide an error handler callback that will be called when a job
instance fails, errors or is cancelled.

* Changed error handler and completion handler to take a "read-only" copy of an instance instead of just certain fields.

* comment

* javadoc

* fix test

* change step execution context from taking instance id to instance

* review feedback

* fix test

* replace 20 or so jetbrains annotations

* fixed a few incorrect @NotNull annotations

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-04 23:27:58 -04:00
Tadgh 3068eb38c4
Rel 6 0 mb 2 (#3747)
* adding version.yaml, updating version in pom.xml

* It is possible to write to a resource in a partition the user is not authorized to. (#3397)

* fixed

* remove sout

* add msg.code

* fix failed tests

* fix equal sign

* update msg code

* extract method

* Fix up code numbers

* Clean changelog

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Revert final artifact version

* add graphql test (#3585)

* added graphql birthdate test

* fix variable name

* typo

* 3506 mdm log enhancement (#3543)

* Providing Fixme's to be reworked at a later time.

* Adding // FIXME Anna to assist our Austrian friend.

* Adding logging test as a first step in addressing issue #2822.

* hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

* Update MdmMatchLinkSvc.java

#3506 move "narrowed down" log to different place

* #3506 added scores and tests

#3506 added scores and tests

* #3506 formatting

#3506 formatting

* #3506 create changelog file

#3506 create changelog file

* #3506 fix typo

#3506 fix typo

* #3506 fix part 3, minor formatting

#3506 fix part 3, minor formatting

* #3506 fix tests

#3506 fix tests

Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>

* mdm matching (#3579)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* begin with failing test

* fixed

* changelog

* add jira tag

* Update to 6 1 (#3582)

* added changelog folder, upped version

* version enum

* add a few more unit tests to assert proper NO_MATCH exclusion

* revert merge master doh! bad reflexes

* revert merge origin master

* unrevert revert.  ugh what a pain

* merge recovery.  fix poms.

* merge recovery.  more reverting

* merge recovery.  more reverting

* merge recovery.  more reverting

* Revert "merge recovery.  fix poms."

This reverts commit ae6e0ddb06.

* more revert revert reversions

* more revert revert reversions

* pre-review cleanup

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>

* Fix regression of 3411 - _lastUpdated gets clobbered during $reindex job (#3586)

* When chunking for the reindex job, don't clobber the lastUpdated if provided by the caller.

* License

* Ks 20220508 log colour (#3592)

* don't use colours when output is redirected to a file

* change log

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551) (#3594)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* 3584 case sensitive string elasticsearch (#3596)

* Fix for case-sensitive search with newer elasticsearch

* Test and fix ascii normalization too

* more test cases

* comments

* Docs update empi usecase (#3598)

* Update diagram for use case 5

* Add new page for database support

* Licnese

* Mb norm fix (#3604)

* Normalize query since wildcard searches aren't normalized in elastic

* Handle batch2 job cancellation (#3603)

* Handle batch2 job cancellation

* Add CANCELLED status

* Remove unuseful test

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Add batch2 interfaces to obtain recent instances (#3601)

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* changelog folder

* Documentation change, added warning log and change log for ticket (#3609)

Co-authored-by: Steven Li <steven@smilecdr.com>

* fix ne for lastUpdated search param (#3589)

* init/wip

* add implementation, add tests

* add checks and some refactoring

* refactor and simplify tests

* refactor

* get rid of some of the warnings

* add changelog

* remove my todos

* redo ne logic to account for less precise dates

* add tests

* refactoring

* rename to follow convention

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>

* Version bump

* Add skip of checkstyle to upload

* Typo

* upping version enum, upping version number, 6.0.1

* Group Bulk Export rules exiting too early.  (#3643)

* Implementation, test, changelog

* Add jira link

* adding backport in yaml for 3642

* adding folder for 6.0.1 release

* pom changes to publish checkstyle to sonatype

* bad version in pom file

* Make FulltextSearchSvcImpl optional bean for ValuesetOperationProvider

* AUtomcplete error code

* updating comment indicating final erorr code

* Bump error code

* fixing version enum so release docs generate correctly, adding additional changelog for 3650

* 3684 subscription for delivering delete events formatted wrong (#3685)

* Fixed issue with delivering delete events Subscriptions

* Extracted deliverer class name to a constant, now shared across tests and interceptor class

* Added change log and doc changes

* moved the constant to a more appropriate place

Co-authored-by: Steven Li <steven@smilecdr.com>

* Version bump

* Add version enum

* Version enum and folder

* duplicate import

* duplicate import

Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: katiesmilecdr <88786813+katiesmilecdr@users.noreply.github.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: alackerbauer <33912849+alackerbauer@users.noreply.github.com>
Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
2022-06-29 21:03:52 +00:00
Ken Stevens 069f20a5f7
batch2 management operations api (#3742)
* first test passes.  onto the next one

* two down two to go

* three down one to go

* licenses

* basic admin-json plumbing confirmed with IT

* FIXME

* FIXME

* FIXME

* FIXME

* changelog

* bump hapi version to 6.1.0-PRE6-SNAPSHOT

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 23:02:17 -04:00
jmarchionatto 84fe58cc86
Hsearch namespace consolidation (#3733)
* Initial implementation

* Implement job cancellation

* Works for date and token. Cleanup required.

* Works for string and quantity.Cleanup pending.

* Works for reference and perform cleanup.

* Cleanup configuration and add missing tests

* Eliminate sort property registry (no go for a cluster) for a static property map.

* Missed in previous commit

* Implement HSearch number parameter

* Implement HSearch number sorting

* Unify different hibernate search names (HibernateSearch, Freetext and Lucene) to HSearch

* Move analysis configures for lucene and elastic to same file to enforce the need to keep them in synch

* Use new defined constant

* Rename package too

* Rename additional elastic references

* bump version to 6.1.0-PRE5-SNAPSHOT

* Restore Hap prefix to hapi configurers

* fixing hapi version of new maven module

* fix pom

* fix pom

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 18:38:05 -04:00
Ken Stevens 49d298dba1
mdm clear move to batch2 (#3671)
* begin with failing test

* continue building out

* continue building out

* reworking chunk api

* replace ChunkId zippers with proper iterator

* progressing test

* yay test passes!

* switch to idchunk interface

* fixmes

* compiles

* fixme

* fixme

* rename classes

* rename json classes with json suffix

* work towards homogeneous chunks

* fixme

* fixme

* unit test

* fixme

* fixme

* pull out builder

* moar tests

* moar tests

* preserve order

* moar test

* moar test

* fixme

* fixme -> wip to start ci

* msg code

* msg code fix test

* fixme

* change provider implementation

* final WIPs

* IT failing

* IT passes

* java 11

* fix test

* fix test

* fix test

* changelog

* use batch size if provided

* last WIP

* fix test

* fix test

* fix test

* whitespace change to trigger new build

* fix test

* bumping hapi version

* bumping hapi version

* fix test

* review feedback

Co-authored-by: Ken Stevens <ken@Kens-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-25 20:26:44 -04:00
TipzCM 48c28997e6
3662 batch2 reduction step (#3681)
* adding a bunch of stuff

* most of the steps are working now

* writing more tests

* adding more tests

* updated the batch2 coordinators

* updating version

* some minor test tweaks

* added a new test

* adding more tests

* blah

* blah

* review fixes 1

* fixing tests and refactoring

* review points

* updated for code review

* review fixes

* updated

* step 1 on review step 5

* final touches

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-20 18:59:01 +00:00
JasonRoberts-smile cedeaa04f1
add support for in and not-in qualifiers (#3680)
* add support for in and not-in qualifiers

* fix failing test

* fix broken test

* fix broken tests

* code review comments

* Make dependency on IValidationSupport optional

* create IValidationSupport dependency only if needed

* small design improvements

* changelog

* rework validation support initialization

* Update hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/matcher/InMemoryResourceMatcher.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-06-16 20:17:57 +00:00
StevenXLi 3da39aeda3
3658 adding updating with history rewrite feature (#3659)
* Updating With History Rewrite implementation

* Missed new files

* Updated changelog and documentation

* Fixed test failures and doc

* minor fix

* lgtm code refactor

* Fixed test to clear DaoConfig

* Removed unused class

* Code review changes

* Simplified updateHistoryRewrite's fluent, combining with normal update's, also changed logic on custom header error checking, updated tests

* refactored method to extracted common code

* removed `updateHistoryRewrite()` and replaced with `update()`, updated tests accordingly

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-06-15 12:40:58 -04:00
Tadgh 1eeb6112f6
Fix Patch in bundle to return Response Entry (#3678)
* Refactor test to cause failure

* Handle patch for entry response building

* add changelog

* Update hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/dao/BaseTransactionProcessor.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-06-13 19:11:18 +00:00
StevenXLi 9336af4b0f
3684 subscription for delivering delete events formatted wrong (#3685)
* Fixed issue with delivering delete events Subscriptions

* Extracted deliverer class name to a constant, now shared across tests and interceptor class

* Added change log and doc changes

* moved the constant to a more appropriate place

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-06-13 18:04:49 +00:00
michaelabuckley 92acd9b001
Avoid unsupported features during migrations on MSSQL (#3652)
Avoid unsupported ONLINE index operations on MSSQL Standard
2022-05-31 18:44:42 +00:00
Tadgh 6f5eea81b7
Jpaserver changes (#3632)
* Make FulltextSearchSvcImpl optional bean for ValuesetOperationProvider

* wip

* wip

* wip fix test

* move to override in test class

* Test fixes

* Add failsafe to other projects which have ITs

* Fix test

* Fix test to use transaction

* Move subscription loading

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-05-26 02:52:05 +00:00
Ken Stevens fb6e5cf6ce
Batch 2 Maintenance refactor (#3631)
* SRP refactor maintenance

* SRP refactor maintenance

* cache jobDefinition in jobInstance when processing (rather than looking it up in a bunch of different places)

* begin with failing test

* test passes

* tests pass.  still need to add a few more tests

* add comment to trigger new ci

* added positive and negative tests for new behaviour

* fix IJ warnings

* change log

* change log

* fix test race condition

* resolved feedback

* review feedback
2022-05-25 04:16:58 +00:00
JasonRoberts-smile b9579313ef
Jr 20220519 expunge config params (#3627)
* fix config settings for system-level expunge

* clean up expunge everything interface

* extract common system provider

* changelog

* version bump

* touch changelog to force a fresh pipeline

* re-enable publishing of checkstyle

* back out Gary's Sonatype change entirely
2022-05-23 18:55:42 -04:00
Tadgh 54f578c8b1
Rel 6 0 mergeback (#3597)
* adding version.yaml, updating version in pom.xml

* It is possible to write to a resource in a partition the user is not authorized to. (#3397)

* fixed

* remove sout

* add msg.code

* fix failed tests

* fix equal sign

* update msg code

* extract method

* Fix up code numbers

* Clean changelog

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* Revert final artifact version

* add graphql test (#3585)

* added graphql birthdate test

* fix variable name

* typo

* 3506 mdm log enhancement (#3543)

* Providing Fixme's to be reworked at a later time.

* Adding // FIXME Anna to assist our Austrian friend.

* Adding logging test as a first step in addressing issue #2822.

* hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

hapifhir#3506 part 2: enhance logging for (un)successful MDM matching

* Update MdmMatchLinkSvc.java

#3506 move "narrowed down" log to different place

* #3506 added scores and tests

#3506 added scores and tests

* #3506 formatting

#3506 formatting

* #3506 create changelog file

#3506 create changelog file

* #3506 fix typo

#3506 fix typo

* #3506 fix part 3, minor formatting

#3506 fix part 3, minor formatting

* #3506 fix tests

#3506 fix tests

Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>

* mdm matching (#3579)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* begin with failing test

* fixed

* changelog

* add jira tag

* Update to 6 1 (#3582)

* added changelog folder, upped version

* version enum

* add a few more unit tests to assert proper NO_MATCH exclusion

* revert merge master doh! bad reflexes

* revert merge origin master

* unrevert revert.  ugh what a pain

* merge recovery.  fix poms.

* merge recovery.  more reverting

* merge recovery.  more reverting

* merge recovery.  more reverting

* Revert "merge recovery.  fix poms."

This reverts commit ae6e0ddb06.

* more revert revert reversions

* more revert revert reversions

* pre-review cleanup

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>

* Fix regression of 3411 - _lastUpdated gets clobbered during $reindex job (#3586)

* When chunking for the reindex job, don't clobber the lastUpdated if provided by the caller.

* License

* Ks 20220508 log colour (#3592)

* don't use colours when output is redirected to a file

* change log

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter… (#3551) (#3594)

* Added fix for https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/328

* Update HapiFhirJpaMigrationTasks.java

Corrected ordering

* Update HapiFhirJpaMigrationTasks.java

Moving index status to be last operation

* Revert "Update HapiFhirJpaMigrationTasks.java"

This reverts commit 37bfd3e66e.

* Moved to bottom

Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>

* 3584 case sensitive string elasticsearch (#3596)

* Fix for case-sensitive search with newer elasticsearch

* Test and fix ascii normalization too

* more test cases

* comments

* Docs update empi usecase (#3598)

* Update diagram for use case 5

* Add new page for database support

* Licnese

* Mb norm fix (#3604)

* Normalize query since wildcard searches aren't normalized in elastic

* Handle batch2 job cancellation (#3603)

* Handle batch2 job cancellation

* Add CANCELLED status

* Remove unuseful test

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* Add batch2 interfaces to obtain recent instances (#3601)

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>

* changelog folder

* Documentation change, added warning log and change log for ticket (#3609)

Co-authored-by: Steven Li <steven@smilecdr.com>

* fix ne for lastUpdated search param (#3589)

* init/wip

* add implementation, add tests

* add checks and some refactoring

* refactor and simplify tests

* refactor

* get rid of some of the warnings

* add changelog

* remove my todos

* redo ne logic to account for less precise dates

* add tests

* refactoring

* rename to follow convention

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>

* Version bump

Co-authored-by: markiantorno <markiantorno@gmail.com>
Co-authored-by: katiesmilecdr <88786813+katiesmilecdr@users.noreply.github.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: alackerbauer <33912849+alackerbauer@users.noreply.github.com>
Co-authored-by: Etienne Poirier <etienne.poirier@smilecdr.com>
Co-authored-by: Anna <anna@MacBook-Pro.local>
Co-authored-by: Jens Kristian Villadsen <jenskristianvilladsen@gmail.com>
Co-authored-by: Jens Kristian Villadsen <jvi@trifork.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: StevenXLi <stevenli_8118@hotmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com>
Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
2022-05-16 22:08:27 +00:00
jmarchionatto 231c2659b8
Implement :below qualifier for tag parameters (#3613)
* Initial implementation

* Implement job cancellation

* Implement :below qualifier for tag parameters

* Add changelog

* Add tests

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-05-16 14:06:48 -04:00
Mark Iantorno 544ef3de1b
Update to 6 1 (#3582)
* added changelog folder, upped version

* version enum
2022-05-03 20:34:25 +00:00
Tadgh 581f40f373
Upgrade Hibernate Search (#3572)
* wip

* updated rest high level for lucene mismatch

* Remove custom JNA

* Bump to 13

* License files

* Changelog
2022-05-02 13:38:56 -04:00
Ken Stevens 10ebb082d0
Issue 3563 mdm nickname support (#3565)
* added NicknameMap

* added NicknameSvc

* create NicknameInterceptor

* test interceptor integration

* add nickname matcher

* test nickname matcher

* unit test

* documentation

* unit test

* cleanup

* cleanup

* Msg.code()

* Msg.code() in test

* Msg.code()

* review feedback
2022-04-28 18:02:56 -04:00
Jaison Baskaran e0f1b913b7
Support _total and _count works for hibernate search (#3567)
Add support for _total, _count, and _offset to Lucene backend.

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-04-28 16:35:39 -04:00
seanmikalson-smilecdr 68c8523046
$translate operation implementation for remote terminology (#3552) 2022-04-25 12:37:15 -07:00
jdar8 5ccb4effb0
3547 Interceptor hook for client-assigned IDs (#3559)
* add pointcut

* draft test

* fix test, change name and location of pointcut

* add changelog

* change check condition, improve changelog wording

* improve changelog wording

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_0_0/3547-interceptor-hook-for-client-assigned-ids.yaml

Co-authored-by: Tadgh <garygrantgraham@gmail.com>

* change pointcut name

* update test

* fix test

* fix changelog

* renaming variables in test

Co-authored-by: Justin_Dar <justin.dar@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-04-22 18:09:25 -04:00
JasonRoberts-smile 1114c72f03
Further expunge refactoring (#3550)
* convert expunge public interfaces to use ResourcePersistentId instead of Long

* changelog

* version bump
2022-04-21 09:51:19 -04:00
Ibrohim Kholilul Islam d20c6a7d25
Implement GraphQL Connection Approach (#1987)
* implement GraphQLQueryBodyParameter and GraphQLQueryBody annotation

* GraphQLQueryBodyParameter to recognize application/graphql

* fix 500 error on method without @GraphQLQueryBody

* refactor to processGraphQlGetRequest and processGraphQlPostRequest

* add testGraphPostContentTypeJson and testGraphPostContentTypeGraphql to GraphQLR4RawTest

* fix imports

* implement GraphQL Connection

* add tests

* add cacheControlDirective and requestDetails to registerSearch

* fix tests and fix bugs

* set error code
2022-04-19 17:43:41 +00:00
James Agnew 7dec2e334d
AuthorizationInterceptor concurrency failure (#3528)
* Fix #3256 - AuthorizationInterceptor concurrency failure

* Test fixes
2022-04-15 15:27:57 -04:00
Ken Stevens 9d2a317446
Issue 3530 terserutil contained failure (#3532)
* License headers

* begin with failing test

* fixed

* changelog

* add jira tag

* review feedback
2022-04-12 20:16:23 -04:00
michaelabuckley 1a8678cb1c
Implement support for Quantity search parameters in the Hibernate Search index (#3477)
* markers for new quantity hibernate search support

* Add quantity search

* Add normalized (canonical) quantity search

* Add comment

* Use double values instead of BigDecimal to avoid scaling

* Remove multiple case labels which are not supported until source version 14

* Remove switch rules which are not supported until source version 14

* Use constant for error message used in tests

* A failing test for multiple indexed quantity paths

* Disallow dirtying context for nested classes

* failing test for normalized units

* Make sp.value-quantity.quantity a nested structure to handle correlated queries for code and value.
Save units for canonical case and consider them when querying.
Collapse normalized sub-structure under *.quantity.

* Nest sp.value-quantity instead of sp.value-quantity.quantity for compatibility with other nested types (IE. token)

* Comment out debug statements

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/search/ExtendedLuceneClauseBuilder.java

Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>

* Add test for nested correlated quantity query

* Address MR comments

* Add proper handling of or query predicates

* Use positive conditions for better readability

* Add tests to validate behavior of combined and and or clauses, including some making no sense but still syntactically correct.

* Only consider normalized quantity searching if ModelConfig search level is NORMALIZED_QUANTITY_SEARCH_SUPPORTED

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>
2022-04-08 08:25:27 -06:00
longma1 01d6e15f90
2579 partitioned support for mdm (#3485)
* committed changes discussed at tasking

* Changed the subscription message to store the request partition

* modified mdm subscription loader to create a cross partition subscription in the default partition if multitenancy is enabled

* Fix errors when running Multitenant IT for mdm

* Added partition id to mdm golden resource search and creating/updating golden resource

* added partition id column to the mdm link table

* Added partition id of the source resource to partition id column of the mdm link table

* broken build

* working -links, missing hapi tests

* Fixed the mdm link db table due to feedback

* added IT for create link mdm operation, also added changes to pass the IT

* Fixed test for create mdm links, added test for update mdm link on partitioned server

* changed mdmLink search from searchbyExample to criteriaBuilder, added partitionHelperSvc to determine partition

* added test for merge golden resource and search golden resource on partitioned server

* added unit test for not duplicate on partitioned and non-partitioned server

* added criteriabuilder search and tests

* code cleanup

* added partition support to -merge-golden-resources operation

* test cleanup

* fixed  operations with multitenancy

* added test for mdm clear operation

* added waiting for batch job to finish in mdm clear operation test

* added query to dao to support partitioned server for mdm clear operation

* resolve mistake from merge conflict

* added mdm-submit operations, also fixed paging for -query

* added partitionIds constant

* fixed build issue where there was no Code.msg in mdm partition exceptions

* code review fixes

* fix broken test, also fix a bug where query link could not query by linkSource properly

* fix test stubbing

* bump hapi version to 6.0.0-PRE10-SNAPSHOT

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
2022-04-06 15:48:30 -04:00
Tadgh 523fe95820
Rel 5 7 2 mergeback (#3518)
* fix handling of common search parameters

* Revert "fix handling of common search parameters"

This reverts commit 89c45eebdc.

* Fix implementation, add test (#3378)

* Fix implementation, add test

* Tighten test

* Rip out dead modules

* Add changelog

* Jr 20220210 handle common search params in contained searches (#3377)

* fix handling of common search parameters

* add support for reference search parameters with multiple paths

* Issue 3357

* Version bump

* Fixed null pointer exception for re-loading subscription on cdr restart and when there's no partition id in the request, and added tests

* added changelogs for this fix

* Fix broken changelog file

* Can't specify specific resource type permissions for bulk export (#3376)

* deny user from exporting without perms

* add unit tests

* add changelog

Co-authored-by: olivia-you <olivia.you@smilecdr.com>

* Make migration donothing as it was added in error

* Remove ehcache

* Add version.yaml:

* Add to sources/javadocs for dist

* Fix typo

* fix up pom

* wip test removing checkstyle plugin from deployable pom

* remove test pom changes, instead just dont deploy to sonatype

* typo

* Version bump

* Add implementation, testing, and changelog

* Add backport info, make aware of 6_0_0

* add jira ref

* Fix test

* Add versionenum

* Bump dep for spring, add changelog, add versionenum, add changelog folder

* Actually do the version bump

* remove doubleimport

Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>
Co-authored-by: olivia-you <olivia.you@smilecdr.com>
2022-04-05 22:39:09 +00:00
Tadgh fe53bcb63d
Rel 5 5 7 mergeback (#3520)
* Add versionenum, bump maven version

* failing tests

* more logs

* Add more test cases, not passing

* fixed the double chain with contained resource second case

* clean up a bit

* add tests for qualifiers

* force the resource table to be the root of the query if we might be traversing a contained reference

* tidy up

* tests for longer chains

* changelog

* code review feedback

* backport changelog

* Add version.yaml

* [2935] Escape "%" in like expression

* Add backport to changelog

* Bumping version number. Update VersionEnum to support 5.5.3

* Jr 20211013 chained references (#3079)

* Create index entries for outbound references of contained resources

* build query for chained reference

* fix case where the contained reference is an explicit id rather than a continued chain

* fix contained index to use path names not search param names

* make qualified search work

* cleanup and changelog

* code review

* fix broken tests

* add imports from bad merge

* Add forgotten import from merge

* Jr 20211018 chained references 2 (#3099)

* Create index entries for outbound references of contained resources

* build query for chained reference

* fix case where the contained reference is an explicit id rather than a continued chain

* fix contained index to use path names not search param names

* make qualified search work

* cleanup and changelog

* recurse while creating indexes on contained resources

* double link both contained

* longer contained subchains

* adding some failing test cases to illustrate the limitations of qualified searches

* clean up merge cruft

* changelog

* Jr 20211021 chained references 3 (#3107)

* Create index entries for outbound references of contained resources

* build query for chained reference

* fix case where the contained reference is an explicit id rather than a continued chain

* fix contained index to use path names not search param names

* make qualified search work

* cleanup and changelog

* recurse while creating indexes on contained resources

* double link both contained

* longer contained subchains

* adding some failing test cases to illustrate the limitations of qualified searches

* clean up merge cruft

* changelog

* create recursive resource links

* add test coverage for a more complicated case

* changelog

* remove unnecessary check for _contained flag

* fix broken tests

* Add backport information

* Bump version, add version enum

* begin with failing test

* fix bug

* change log

* change log

* Add backport information

* Improve performance of chained queries into contained resources (#3312)

* base restructuring of query

* fix unit tests

* suppress unnecessary resource type parameter

* pass the resource type used to fetch the search param as part of the chain, so later we do not need to guess what it was

* add query structure tests

* changelog

* fix test failures

* got one of the branches wrong in the 3-reference case

* Add backport info

* Bump version

* Fix errors due to merge

* Bump license year

* Add folder for release version

* Version bump

* Jr 20220210 handle common search params in contained searches (#3377)

* fix handling of common search parameters

* add support for reference search parameters with multiple paths

* Add backport info

* add new folder for version

* Add missing message

* New version enum, and folder

* Version bump

* Bump spring lib version

* Tidy versions

* Remove test

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
Co-authored-by: katie_smilecdr <katie@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
2022-04-05 17:15:18 -04:00
samuelwlee2 7dc302b7ef
add addBundleEntry method (#3522)
* add addBundleEntry method

* change method to createNewBundleEntryWithSingleField

* changes to changelog
2022-04-05 10:40:09 -04:00
Tadgh c2f30502b0
Rel 5 6 3 mergeback (#3519)
* 3138 externalized binary packages (#3139)

* Add test and impl

* Add changelog

* Fix test

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3138-support-externalized-binaries.yaml

* add beans to test configs

* Typo

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/binstore/IBinaryStorageSvc.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/binstore/NullBinaryStorageSvcImpl.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

Co-authored-by: Kevin Dougan SmileCDR <72025369+KevinDougan-SmileCDR@users.noreply.github.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* 3131 - Added support for the lookup operation in the Remote Terminology code (#3134)

* Remove leading underscores from identifiers (#3146)

* Version bump

* License files

* version.yaml

* Add executeRawSql() method to Migrator (#3183)

* Add implementation, add test

* Add javadoc

* Add another helper function

* Bump version

* 3153 update unknowncodesystemwarningvallidationsupport to have configurable severities (#3155)

* 3153 updating the unknowncodesystemwarningvalidationsupport class

* 3153 updating version for smile

* 3153 update broken tests

* 3153 adding log

* 3153 cleanup

* 3153 cleanup

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* Add backport info

* Avoid creating ResourcePersistentId for placeholder resources with null ID (#3158)

* Add check before mapping storage ID to resource ID in TransactionDetails.

* Add change log.

* Changed to instead prevent creation of ResourcePersistentId with null ID value.

* Changed to instead prevent ResourcePersistentId being created with null resource ID.

Co-authored-by: ianmarshall <ian@simpatico.ai>

* Fix bug loading packages in non-database mode (#3199)

* Add implementation

* Add changelog

* Add backports

* 3164 updating code review points (#3165)

* 3164 updating code review points

* 3164 updating code review points

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* Add executeRawSqlStub (#3203)

Co-authored-by: Joanne Mendoza <joanne@smilecdr.com>

* Add new version

* begin with failing test

* fix bug

* change log

* change log

* Add backport informationm

* begin with failing test

* fix issue

* change log

* code review

* Add backport info. Add folder

* fixed

* Add test

* Update test

* Add backport info

* Bump minor version

* 3170 language portion of language code is case insensitive (#3171)

* 3170 language portion of language code is case insensitive

* 3170 adding changelog

* 3170 house keeping

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>

* Add backport info:

* Add version enum

* Bump hapi version to 5.6.3

* Add version enum

* Add changelog folder and version.yaml

* Patch vulnerability, add changelog

* Add new version

* Remove test

Co-authored-by: Kevin Dougan SmileCDR <72025369+KevinDougan-SmileCDR@users.noreply.github.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: TipzCM <leif.stawnyczy@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: IanMMarshall <49525404+IanMMarshall@users.noreply.github.com>
Co-authored-by: ianmarshall <ian@simpatico.ai>
Co-authored-by: Joanne Mendoza <joannemmendoza@gmail.com>
Co-authored-by: Joanne Mendoza <joanne@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: katie_smilecdr <katie@smilecdr.com>
2022-04-04 11:29:42 -04:00
Tadgh b1e2d7c104
Add test, add changelog, fix bug (#3494)
* Add test, add changelog, fix bug

* Test fix

* Fix some more test artifacts
2022-03-26 20:34:43 -04:00
Tadgh 5b413a9135
Package move-around (#3488)
* moved classes

* Bump version

* licensing

* Repackage, fix messages

* reject empty streams

* Remove dead import

* Version bump

* Fix up test to actually return a data stream

* Move null binary svc to storage
2022-03-26 22:43:41 +00:00
James Agnew b833c13a3c
Improve batch2 error handling (#3482)
* Improve batch2 error handling

* Build fixes

* Work on reindexing

* Test fixes

* Account for gzipped contents

* License header updates

* Register VS operations on public server

* Test fix

* Build fix

* Remove animal sniffer plugin

* Compile fix
2022-03-26 17:40:12 -04:00
Olivia You 625ed936e9
3470 ne date search (#3471)
* fix mongo date search with the NOT_EQUALS prefix

* add test & changelog

* add some unit tests

* rename unit test names to be more clear

Co-authored-by: olivia-you <olivia.you@smilecdr.com>
2022-03-25 17:51:17 -04:00
James Agnew f82534ca51
Convert Reindex into Batch2 Job (#3458)
* Start working on reindex job

* Updates

* Ongoing work

* Ongoing work

* Work on reindex

* Reindex work

* Add logging

* Fix typo

* Test fix

* Test fix

* Address fixme

* Liocense header

* Resolve fixme

* Add logging

* Update logs

* Address review comments

* License header

* Test fixes

* Test fix

* Test fixes

* Test fix

* Version bump

* Version bump
2022-03-19 16:07:58 -04:00
jmarchionatto e45f7ee9cb
Issue 3450 there is no way to recreate freetext indexes for terminology entities (#3463)
* Add reindex-terminology batch command

* Handle number of thread throttling and concurrency with other terminology batch operations

* Add required dbcp2 dependency

* Fix test

* Improve ConnectionPoolInfoProvider setup. Handle maximum connections.

* Remove java higher version construction

* Remove unused config

* Add reindex terminology integration test.
Reset termConcept counters before pre-expanding, which otherwise accumulate if it was pre-expanded before.

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-03-18 08:39:12 -04:00
Tadgh 17cf1ed9ce
Rel 5 7 1 mergeback (#3480)
* fix handling of common search parameters

* Revert "fix handling of common search parameters"

This reverts commit 89c45eebdc.

* Fix implementation, add test (#3378)

* Fix implementation, add test

* Tighten test

* Rip out dead modules

* Add changelog

* Jr 20220210 handle common search params in contained searches (#3377)

* fix handling of common search parameters

* add support for reference search parameters with multiple paths

* Issue 3357

* Version bump

* Fixed null pointer exception for re-loading subscription on cdr restart and when there's no partition id in the request, and added tests

* added changelogs for this fix

* Fix broken changelog file

* Can't specify specific resource type permissions for bulk export (#3376)

* deny user from exporting without perms

* add unit tests

* add changelog

Co-authored-by: olivia-you <olivia.you@smilecdr.com>

* Make migration donothing as it was added in error

* Remove ehcache

* Add version.yaml:

* Add to sources/javadocs for dist

* Fix typo

* fix up pom

* wip test removing checkstyle plugin from deployable pom

* remove test pom changes, instead just dont deploy to sonatype

* typo

* Version bump

* Add implementation, testing, and changelog

* Add backport info, make aware of 6_0_0

* add jira ref

* Fix test

* Add versionenum

* Remove dups

* Remove unnecessary stuff

Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>
Co-authored-by: olivia-you <olivia.you@smilecdr.com>
2022-03-17 22:31:40 -04:00
Ken Stevens dfd99c5471
improve search parameter registry logging (#3452)
* Startup log cleanup.  Use cached fhir contexts.

* improve logging

* improve logging

* add full comparator

* wrap resource search param map

* rename wrapped map methods

* add fixmes

* further searchparam encapsulation

* fix regression found by test

* improve logging

* pre-review cleanup

* pre-review cleanup

* bump hapi version

* license

* since I'm bumping hapi-fhir now, improve the api dependency

* licenses
2022-03-09 19:57:38 -05:00
James Agnew ac80c7ffaa
Search Narrowing by Code Tweaks (#3453)
* Search narrowing tweaks

* Search narrowing tweaks

* Test

* Test fixes

* Try to fix LGTM

* Add license headers

* Test fixes

* Test fixes
2022-03-07 20:27:21 -05:00
michaelabuckley 20e092ba6d
Better error messages for ValueSet $expand contextDirection=existing (#3446)
* Better error messages for ValueSet $expand contextDirection=existing
* Apply to $lastn too - it uses aggregations
2022-03-03 21:11:08 -07:00
Tadgh a2133f6940
Break a couple Circular Bean Dependencies (#3437)
* Rely on context refresh event instead of postconstruct. Load bean lazily.

* Remove parent autowire

* Break termcodesystem circular dependency with new bean

* Break cycle between BaseTermReadSvcImpl and TermCodeSystemStorageSvcImpl

* move back into helper method

* Rip out scheduling and submission to spring batch away from BulkDataExportSvcImpl

* wip

* Fix bean naming

* Revert

* re-add autowiring

* Reformat

* docs and reformat

* Back out change

* feedback from review

* merge conflicts
2022-03-03 09:37:40 -05:00
James Agnew 2cba62b4e8
Enable search narrowing on large ValueSets (#3405)
* Enable search narrowing on large ValueSets

* ValueSet improvements

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Add test

* Work on narrowing interceptor

* Work on narrowing

* License headers

* Refactor code narrowing

* Add docs

* Version bump

* Test fix

* Test fixes

* Build fix

* Fixes

* Version bump

* Test fix

* License header updates

* Build fix

* Test fixes

* Test fix

* Test fix

* Docs fix

* Test fix

* Test fix

* Resolve fixme

* Bump

* Force a CI build

* Make CI happen again
2022-03-01 09:16:31 -05:00
kailiu-smile 1bc6e0e3ae
Smile 2775 package install logs ignore resource validation (#3381)
* Added log to the create() and valiForUpload() methods in the PackageInstallerSvcImpl class

* Added helper methods to extract URL from SearchParameter.Tested and fixed log entries.

* Changelog entry

* Update 3384-Package-Install-added-logs-for-resources-validation.yaml

Co-authored-by: Kai Liu <kliu@Kais-MacBook-Pro.local>
2022-02-28 19:52:43 +00:00
James Agnew b7d1ae217d
New batch framework / Bulk Import (#3387)
* Work on new batch framework

* Work on new batch framework

* Work on new batch framework

* Work on batch

* Compile working

* Work on bulk import

* Adjust import

* Work

* Work on batch

* Bump version

* WOrk on new batch processes

* Work on bath

* Bump to PRE4

* Build fixes

* CLeanup

* Small tweak

* Add exception code

* Test fixes

* Test fixes

* Test fix

* Additional synchronization

* Add license headers

* Test fixes

* Test fixes

* Add changelogs

* Address PG

* Test fix

* Test fix

* Test fixes

* Review notes

* Work on tests

* Test fixes

* Test fix

* Work on tests

* Tets fix

* Test fixes

* Test fixes

* Add missing exception codes

* Test fix

* Test fixes

* More test fixing

* License headers

* Test fix

* Add new test logging

* Work on tests

* Test fixes

* Test fix

* Resolve fixme

* Try to avoid test failure

* Add import command

* Work on storage

* Fix error codes

* Fixes

* License header

* Build fix

* Build fixes

* Fix dep
2022-02-27 16:04:49 -05:00
jmarchionatto dbc9427a05
Issue 3391 2588 expand valueset regex include not working (#3410)
* Refactor ValueSet expansion code to simplify property filtering by using new Hibernate search V6 nested classes capabilities; use generic hibernate search queries where possible (mostly everywhere but regex queries) and add R4 integration tests  for both elastic and lucene.

* Refactor ValueSet expansion code to simplify property filtering by using new Hibernate search V6 nested classes capabilities; use generic hibernate search queries where possible (mostly everywhere but regex queries) and add R4 integration tests  for both elastic and lucene.

* Add comment for trick

* Add codes to exception messages

* Add test specific for the fix

* Move changelog to current release. Add upgrade notice.

* Improve fail-fast method

* Update upgrade instructions

* Try luck with new exception codes

* Update hapi-fhir-storage/src/test/java/ca/uhn/fhir/jpa/search/ElasticsearchNestedQueryBuilderUtilTest.java

Add missed quote

Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
2022-02-24 21:32:11 +00:00