Commit Graph

258 Commits

Author SHA1 Message Date
TipzCM 8ed0a18ef4
3531 implement bulk processors (#3566)
* base batch2 bulk export steps - unimplemented

* added the basic steps

* adding tests

* we have added the tests

* added tests for bulk export stages

* implementing jpa

* 3531 adding the jpabukexportprocessorimplementation

* 3531 first step for mongo bulk export

* jpa stuff updated

* 3531 hapi fhir stuff

* trying to fix up tests

* fixing test

* fixed most tests in batch export

* deprecate

* need some more guidance here

* fixing some tests

* some steps taken

* blah

* stashing

* test

* fixing tests round 1

* many a changes

* updating version

* small updates

* merging in mster again

* fixing up serialization

* added the permission checkers

* cleanup and sonarkube crap

* some more tests

* updating adding tests

* crimes

* blah

* updatingmsg number again

* removal of commented code

* cleanup

* merge confflicts

* fixing msg

* review fixes round 1

* more review fixes

* merging in master

* fixing poms

* review points

* test fix

* test fix

* fix test race condition

* truncate error

* add stack trace to error

* rename to IT

* fix race condition in test

* finally fixed race condition in test.  this time I'm optimistic hapi ci will finally pass

* fix mock test.  not a fan of mock tests like this.

* any job definition with a reducer step must be gated.

* fixed common batch bean wiring

* fix test

* lazy init ResponseTerminologyTranslationSvc

* change the way we lazy init ResponseTerminologyTranslationSvc

* change the way we lazy init ResponseTerminologyTranslationSvc (for the third time.  this is super tricky!)

* disabling testGoldenResourceWithCircularReferenceCanBeCleared for now

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-17 17:49:03 -04:00
TipzCM 0c644271ce
3770 fix batch2 failure loop (#3779)
* 3770 fixing error handling

* some updates to error handling

* changelog

* blah

* bumping version

* fix test failures

* code review points

* review points

* add retry test

* assert counts in the test

* test fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-15 12:42:46 -04:00
Tadgh ab4d9578e7
Make patch `add` actually add instead of replace. (#3800)
* Fix some patch problems

* Changelog
2022-07-15 16:29:16 +00:00
Tadgh 05ebf0286d
3790 remove legacy search builder (#3791)
* Remove legacy SB

* Add changelog

* Fix more messages

* Fix missed hapi message

* Fix up more messages
2022-07-14 16:01:44 +00:00
StevenXLi 2bc6c91153
3719 enable map response terminology for bulk export (#3735)
* added failing test for Map Response Terminology for bulk export

* moved logic for terminology translation out of the interceptor to svc, added translation logic to bulk export file writer, added more tests

* added changelog

* Wip refactor dependency fix

* wip terminology translation code refactor

* terminology translation code refactor

* refactor clean up

* Remove circular dependency

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_1_0/3719-enable-map-response-terminology-for-bulk-export.yaml

* bump version of hapi to 6.1.0-PRE7-SNAPSHOT

* doc update

* fix issue when search param registry doesn't contain search param for the resource type

* fix beans

* hapi version bump from PRE7 to PRE10

* doc update

* add ResponseTerminologyTranslationSvc as a constructor variable to the interceptor

* fixed ResponseTerminologyInterceptorLoaderTest, added IT for cdr side

* version bump to PRE13

* removed unused import

Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-12 12:24:15 -04:00
Qingyixia 3d4556b70b
Move PATCH to hapi-fhir-storage (#3763)
* Not the final version

* Support PATCH operation

* minor modifications

* minor modifications

* Final version

* Final version
2022-07-11 14:56:22 +00:00
Ken Stevens 0526da080b
3757 move delete expunge to batch2 (#3759)
* begin with failing test

* begin with failing test

* remove unused Spring Batch classes.
convert service classes

* move provider

* just one test left to fix

* fix test

* msg.code

* change log

* bumping hapi to PRE12

* fix test

* review feedback

* license update

* fix intermittent and status update atomicity

* restore url-level partition selection

* fix tests

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-09 17:25:55 -04:00
James Agnew 61d34ab695
Add license headers (#3765) 2022-07-07 17:22:38 -04:00
Nathan Doef b68ccb373a
3693 Search by id is returning a deleted resource with a client-generated id (#3694)
- Added functionality to optionally filter out deleted resources when resolving forced ids to persistent ids
- Bumps version

Co-authored-by: nathaniel.doef <nathaniel.doef@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-05 12:37:25 -04:00
Ken Stevens 3a28920ea7
Add error-handler callback and ensure batch2 completion handler can't be called twice (#3752)
* rename field that drives everything includes

* discovered design flaw in completion

* fix intermittent

* ensure batch2 completion handler can't be called twice

* changelog

* changelog

* javadoc

* bump hapi version

* licences

* Batch2 job definitions can now optionally provide an error handler callback that will be called when a job
instance fails, errors or is cancelled.

* Changed error handler and completion handler to take a "read-only" copy of an instance instead of just certain fields.

* comment

* javadoc

* fix test

* change step execution context from taking instance id to instance

* review feedback

* fix test

* replace 20 or so jetbrains annotations

* fixed a few incorrect @NotNull annotations

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-07-04 23:27:58 -04:00
michaelabuckley 9aac4f9ba8
Introduce memory-matcher based rules (#3677)
Extend rules with filters.

New rule extension, and consent service for post-query filtering.

Co-authored-by: josie <josie.vandewetering@smilecdr.com>
Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-06-29 18:23:12 +00:00
Ken Stevens 069f20a5f7
batch2 management operations api (#3742)
* first test passes.  onto the next one

* two down two to go

* three down one to go

* licenses

* basic admin-json plumbing confirmed with IT

* FIXME

* FIXME

* FIXME

* FIXME

* changelog

* bump hapi version to 6.1.0-PRE6-SNAPSHOT

Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 23:02:17 -04:00
jmarchionatto 84fe58cc86
Hsearch namespace consolidation (#3733)
* Initial implementation

* Implement job cancellation

* Works for date and token. Cleanup required.

* Works for string and quantity.Cleanup pending.

* Works for reference and perform cleanup.

* Cleanup configuration and add missing tests

* Eliminate sort property registry (no go for a cluster) for a static property map.

* Missed in previous commit

* Implement HSearch number parameter

* Implement HSearch number sorting

* Unify different hibernate search names (HibernateSearch, Freetext and Lucene) to HSearch

* Move analysis configures for lucene and elastic to same file to enforce the need to keep them in synch

* Use new defined constant

* Rename package too

* Rename additional elastic references

* bump version to 6.1.0-PRE5-SNAPSHOT

* Restore Hap prefix to hapi configurers

* fixing hapi version of new maven module

* fix pom

* fix pom

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-28 18:38:05 -04:00
Ken Stevens 49d298dba1
mdm clear move to batch2 (#3671)
* begin with failing test

* continue building out

* continue building out

* reworking chunk api

* replace ChunkId zippers with proper iterator

* progressing test

* yay test passes!

* switch to idchunk interface

* fixmes

* compiles

* fixme

* fixme

* rename classes

* rename json classes with json suffix

* work towards homogeneous chunks

* fixme

* fixme

* unit test

* fixme

* fixme

* pull out builder

* moar tests

* moar tests

* preserve order

* moar test

* moar test

* fixme

* fixme -> wip to start ci

* msg code

* msg code fix test

* fixme

* change provider implementation

* final WIPs

* IT failing

* IT passes

* java 11

* fix test

* fix test

* fix test

* changelog

* use batch size if provided

* last WIP

* fix test

* fix test

* fix test

* whitespace change to trigger new build

* fix test

* bumping hapi version

* bumping hapi version

* fix test

* review feedback

Co-authored-by: Ken Stevens <ken@Kens-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-25 20:26:44 -04:00
Qingyixia dce330eb01
Fix to the issue (#3723) 2022-06-21 16:29:35 +00:00
TipzCM 48c28997e6
3662 batch2 reduction step (#3681)
* adding a bunch of stuff

* most of the steps are working now

* writing more tests

* adding more tests

* updated the batch2 coordinators

* updating version

* some minor test tweaks

* added a new test

* adding more tests

* blah

* blah

* review fixes 1

* fixing tests and refactoring

* review points

* updated for code review

* review fixes

* updated

* step 1 on review step 5

* final touches

* blah

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <ken@smilecdr.com>
2022-06-20 18:59:01 +00:00
StevenXLi 3da39aeda3
3658 adding updating with history rewrite feature (#3659)
* Updating With History Rewrite implementation

* Missed new files

* Updated changelog and documentation

* Fixed test failures and doc

* minor fix

* lgtm code refactor

* Fixed test to clear DaoConfig

* Removed unused class

* Code review changes

* Simplified updateHistoryRewrite's fluent, combining with normal update's, also changed logic on custom header error checking, updated tests

* refactored method to extracted common code

* removed `updateHistoryRewrite()` and replaced with `update()`, updated tests accordingly

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-06-15 12:40:58 -04:00
Tadgh 1eeb6112f6
Fix Patch in bundle to return Response Entry (#3678)
* Refactor test to cause failure

* Handle patch for entry response building

* add changelog

* Update hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/dao/BaseTransactionProcessor.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-06-13 19:11:18 +00:00
StevenXLi 9336af4b0f
3684 subscription for delivering delete events formatted wrong (#3685)
* Fixed issue with delivering delete events Subscriptions

* Extracted deliverer class name to a constant, now shared across tests and interceptor class

* Added change log and doc changes

* moved the constant to a more appropriate place

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-06-13 18:04:49 +00:00
Etienne Poirier 05cb6c241c
3668 fhir patch operation returning 500 when patching in a transaction with a resource query in property request.url (#3675)
* Initial test exerting the issue.

* Providing fix and changelog

* Providing fix and changelog

* Providing a searchUrl or matchUrl based on the requestUrl.

* Providing fix and changelog

* Modification on test

* Modification on test

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
Co-authored-by: qingyixia <cherry.xia@smilecdr.com>
2022-06-13 09:15:09 -04:00
Etienne Poirier 868a7e5c40
3637 encapsulation of synchronous query execution capabilities into a new service for reuse (#3638)
* Adding test reproducing the issue.

* Providing implementation fixing the issue and strengthening integrated test.

* Cleaning up commended code.

* Preparing code for review.

* Adding test class SynchronousSearchSvcImplTest.

* Addressing first code review comments.

* Addressing first code review comments.

* Forcing one extra processing step to ensure exit condition is met.

* Adding tests

* Clean up in preparation of second code review.

* Addition of setting page size if required.

* Grouping synch test in nested test class.

Co-authored-by: peartree <etienne.poirier@smilecdr.com>
2022-06-05 20:38:53 -04:00
jamesagnew 5b7add9b81 License header 2022-05-24 15:56:31 -04:00
JasonRoberts-smile b9579313ef
Jr 20220519 expunge config params (#3627)
* fix config settings for system-level expunge

* clean up expunge everything interface

* extract common system provider

* changelog

* version bump

* touch changelog to force a fresh pipeline

* re-enable publishing of checkstyle

* back out Gary's Sonatype change entirely
2022-05-23 18:55:42 -04:00
Ken Stevens 27024067b0
Batch 2 documentation and SRP cleanup (#3616)
* begin writing documentation.  minor refactor.

* document submitting a job

* single responsibility principle

* single responsibility principle

* single responsibility principle

* reduce method parameter count

* reduce method parameter count

* fix test mocks

* reduce method arguments

* update documentation

* add tests
resolve warnings

* resolve warnings

* moar test

* moar test

* Add @Nonnull and @Nullable annotation to construction parameters

* pre-review cleanup

* pre-review cleanup

* review feedback

* review feedback.  added FIXME

* started writing IT with FIXMES

* started writing IT with FIXMES

* added batch 2 state transition integration tests

* FIXME

* improve readability of test

* improve readability of test

* fix race condition

* pre-merge cleanup
2022-05-17 21:25:43 +00:00
Mark Iantorno 544ef3de1b
Update to 6 1 (#3582)
* added changelog folder, upped version

* version enum
2022-05-03 20:34:25 +00:00
Tadgh 581f40f373
Upgrade Hibernate Search (#3572)
* wip

* updated rest high level for lucene mismatch

* Remove custom JNA

* Bump to 13

* License files

* Changelog
2022-05-02 13:38:56 -04:00
seanmikalson-smilecdr 68c8523046
$translate operation implementation for remote terminology (#3552) 2022-04-25 12:37:15 -07:00
Tadgh f7a31287b6
3558 bulk export premature deletion (#3562)
* Implementation, parameterize test, changelog

* Bump spring framework for a vuln while i'm in here

* Fix number of changelog

* Add IS NOT NULL to query
2022-04-24 01:46:44 +00:00
samuelwlee2 4c9667c92a
3524 export poll status returning the wrong partition in the url (#3526)
* test added and bulk data export provider fixed

* add test and change jira number

* change test

* modify tests
2022-04-23 16:47:04 +00:00
Tadgh 53c8b067d5
Support forward references in Group Bulk Export (#3558)
* Preliminary test work to generate forward-reference data

* Complete implementation

* Permit Practitioner/Organization as forward references in bulk export group params

* Wip

* Fix test

* Add error number

* Address review comments

* Update assert
2022-04-22 05:33:10 +00:00
JasonRoberts-smile 1114c72f03
Further expunge refactoring (#3550)
* convert expunge public interfaces to use ResourcePersistentId instead of Long

* changelog

* version bump
2022-04-21 09:51:19 -04:00
Tadgh 578d42c955
Add implementation and changelog, and test. (#3554)
* Add implementation and changelog, and test.

* Add daoconfig settings

* Updatae test

* Rename deexternalize -> autoinflate
2022-04-20 13:47:02 -04:00
Ibrohim Kholilul Islam d20c6a7d25
Implement GraphQL Connection Approach (#1987)
* implement GraphQLQueryBodyParameter and GraphQLQueryBody annotation

* GraphQLQueryBodyParameter to recognize application/graphql

* fix 500 error on method without @GraphQLQueryBody

* refactor to processGraphQlGetRequest and processGraphQlPostRequest

* add testGraphPostContentTypeJson and testGraphPostContentTypeGraphql to GraphQLR4RawTest

* fix imports

* implement GraphQL Connection

* add tests

* add cacheControlDirective and requestDetails to registerSearch

* fix tests and fix bugs

* set error code
2022-04-19 17:43:41 +00:00
Ken Stevens 9d2a317446
Issue 3530 terserutil contained failure (#3532)
* License headers

* begin with failing test

* fixed

* changelog

* add jira tag

* review feedback
2022-04-12 20:16:23 -04:00
JasonRoberts-smile f361aa5998
move expunge core classes and interfaces to storage project (#3529)
* move expunge core classes and interfaces to storage project

* post-merge clean-up

* changelog
2022-04-12 14:12:10 -04:00
James Agnew 8d116b932e
Refactor to restore LGTM functionality (#3455)
* Refactor to restore LGTM functionality

* Compile fix

* Work on cleanup

* Attempt to restore LGTM

* Work on tests

* Fixes

* Build fixes

* Build fix

* Test fix

* Test fixes

* Work on build

* License header updates

* Test fixes

* Test fixes

* License header

* Test fix

* Test fix

* Test fixes

* Restore LGTM

* Fix azure

* Disable test

* Test fixes

* Test cleanup

* Build fix

* Build fix

* Test fix

* Test fix

* Test fixes
2022-04-10 15:54:27 -04:00
michaelabuckley 1a8678cb1c
Implement support for Quantity search parameters in the Hibernate Search index (#3477)
* markers for new quantity hibernate search support

* Add quantity search

* Add normalized (canonical) quantity search

* Add comment

* Use double values instead of BigDecimal to avoid scaling

* Remove multiple case labels which are not supported until source version 14

* Remove switch rules which are not supported until source version 14

* Use constant for error message used in tests

* A failing test for multiple indexed quantity paths

* Disallow dirtying context for nested classes

* failing test for normalized units

* Make sp.value-quantity.quantity a nested structure to handle correlated queries for code and value.
Save units for canonical case and consider them when querying.
Collapse normalized sub-structure under *.quantity.

* Nest sp.value-quantity instead of sp.value-quantity.quantity for compatibility with other nested types (IE. token)

* Comment out debug statements

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/search/ExtendedLuceneClauseBuilder.java

Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>

* Add test for nested correlated quantity query

* Address MR comments

* Add proper handling of or query predicates

* Use positive conditions for better readability

* Add tests to validate behavior of combined and and or clauses, including some making no sense but still syntactically correct.

* Only consider normalized quantity searching if ModelConfig search level is NORMALIZED_QUANTITY_SEARCH_SUPPORTED

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com>
Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>
2022-04-08 08:25:27 -06:00
longma1 01d6e15f90
2579 partitioned support for mdm (#3485)
* committed changes discussed at tasking

* Changed the subscription message to store the request partition

* modified mdm subscription loader to create a cross partition subscription in the default partition if multitenancy is enabled

* Fix errors when running Multitenant IT for mdm

* Added partition id to mdm golden resource search and creating/updating golden resource

* added partition id column to the mdm link table

* Added partition id of the source resource to partition id column of the mdm link table

* broken build

* working -links, missing hapi tests

* Fixed the mdm link db table due to feedback

* added IT for create link mdm operation, also added changes to pass the IT

* Fixed test for create mdm links, added test for update mdm link on partitioned server

* changed mdmLink search from searchbyExample to criteriaBuilder, added partitionHelperSvc to determine partition

* added test for merge golden resource and search golden resource on partitioned server

* added unit test for not duplicate on partitioned and non-partitioned server

* added criteriabuilder search and tests

* code cleanup

* added partition support to -merge-golden-resources operation

* test cleanup

* fixed  operations with multitenancy

* added test for mdm clear operation

* added waiting for batch job to finish in mdm clear operation test

* added query to dao to support partitioned server for mdm clear operation

* resolve mistake from merge conflict

* added mdm-submit operations, also fixed paging for -query

* added partitionIds constant

* fixed build issue where there was no Code.msg in mdm partition exceptions

* code review fixes

* fix broken test, also fix a bug where query link could not query by linkSource properly

* fix test stubbing

* bump hapi version to 6.0.0-PRE10-SNAPSHOT

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
2022-04-06 15:48:30 -04:00
Tadgh f87f468521 Move non-DB modes to storage 2022-03-27 16:51:35 -07:00
Tadgh 5b413a9135
Package move-around (#3488)
* moved classes

* Bump version

* licensing

* Repackage, fix messages

* reject empty streams

* Remove dead import

* Version bump

* Fix up test to actually return a data stream

* Move null binary svc to storage
2022-03-26 22:43:41 +00:00
Jaison Baskaran 3eb3014d25
Revert change to treat Canonical URLs as Locals (#3483)
* Revert change added in MR: https://github.com/hapifhir/hapi-fhir/pull/3050 to add support for treating Canonical Urls as locals. This is no longer needed as the issue reported in #2843 (_include not working for Canonicals) has been addressed in MR: https://github.com/hapifhir/hapi-fhir/pull/3440
2022-03-22 17:46:10 +00:00
James Agnew f82534ca51
Convert Reindex into Batch2 Job (#3458)
* Start working on reindex job

* Updates

* Ongoing work

* Ongoing work

* Work on reindex

* Reindex work

* Add logging

* Fix typo

* Test fix

* Test fix

* Address fixme

* Liocense header

* Resolve fixme

* Add logging

* Update logs

* Address review comments

* License header

* Test fixes

* Test fix

* Test fixes

* Test fix

* Version bump

* Version bump
2022-03-19 16:07:58 -04:00
michaelabuckley 03b6c589c8
New indexing for number and quantity search parameters (#3464)
* New Add Index forced-INCLUDE columns, and Online option

* Remove forced-include.  Simpler just to add all columns to index.

* New date index definitions.

* Oops.  Remove test for deleted feature.

* Cleanup

* Drop IDX_SP_DATE_UPDATED

* Add online option to drop index

* Update annotations to match and redefine FK

* Push sp->resource FK down to control name

* remove dead end

* Cleanup date tests

* New token query test cases for sql extraction

* New token search param indexing.

* Continue to allow the legacy hibernate names while we update the indexing.

* Fix jpa annotations with overloaded field

* Decide on token sorting - we do it.

* review comments

* fixme

* disable dead data migration too

* review fixups

* Test cases for numeric search

* add ne and ap

* Test for Numeric SP

* start

* finish number

* new indexing for quantity

* Move config

* cleanup
2022-03-11 16:23:06 +00:00
michaelabuckley 80d1a5a6f8
Mb new token search index (#3386)
* New token search param indexing.
* New token query test cases for sql extraction
* Retire JPA fk ugly nams
2022-03-11 00:51:53 +00:00
Ken Stevens dfd99c5471
improve search parameter registry logging (#3452)
* Startup log cleanup.  Use cached fhir contexts.

* improve logging

* improve logging

* add full comparator

* wrap resource search param map

* rename wrapped map methods

* add fixmes

* further searchparam encapsulation

* fix regression found by test

* improve logging

* pre-review cleanup

* pre-review cleanup

* bump hapi version

* license

* since I'm bumping hapi-fhir now, improve the api dependency

* licenses
2022-03-09 19:57:38 -05:00
Tadgh 7eb1bd14d1
Do not validate auto-created placeholder resources (#3461)
* Add implementation, testing, and changelog

* add jira ref
2022-03-08 22:22:04 +00:00
James Agnew ac80c7ffaa
Search Narrowing by Code Tweaks (#3453)
* Search narrowing tweaks

* Search narrowing tweaks

* Test

* Test fixes

* Try to fix LGTM

* Add license headers

* Test fixes

* Test fixes
2022-03-07 20:27:21 -05:00
Tadgh a2133f6940
Break a couple Circular Bean Dependencies (#3437)
* Rely on context refresh event instead of postconstruct. Load bean lazily.

* Remove parent autowire

* Break termcodesystem circular dependency with new bean

* Break cycle between BaseTermReadSvcImpl and TermCodeSystemStorageSvcImpl

* move back into helper method

* Rip out scheduling and submission to spring batch away from BulkDataExportSvcImpl

* wip

* Fix bean naming

* Revert

* re-add autowiring

* Reformat

* docs and reformat

* Back out change

* feedback from review

* merge conflicts
2022-03-03 09:37:40 -05:00
James Agnew 2cba62b4e8
Enable search narrowing on large ValueSets (#3405)
* Enable search narrowing on large ValueSets

* ValueSet improvements

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Add test

* Work on narrowing interceptor

* Work on narrowing

* License headers

* Refactor code narrowing

* Add docs

* Version bump

* Test fix

* Test fixes

* Build fix

* Fixes

* Version bump

* Test fix

* License header updates

* Build fix

* Test fixes

* Test fix

* Test fix

* Docs fix

* Test fix

* Test fix

* Resolve fixme

* Bump

* Force a CI build

* Make CI happen again
2022-03-01 09:16:31 -05:00
James Agnew bf04add6ef
Add missing fields to canonical subscription compare (#3431) 2022-02-28 06:48:59 -05:00
James Agnew b7d1ae217d
New batch framework / Bulk Import (#3387)
* Work on new batch framework

* Work on new batch framework

* Work on new batch framework

* Work on batch

* Compile working

* Work on bulk import

* Adjust import

* Work

* Work on batch

* Bump version

* WOrk on new batch processes

* Work on bath

* Bump to PRE4

* Build fixes

* CLeanup

* Small tweak

* Add exception code

* Test fixes

* Test fixes

* Test fix

* Additional synchronization

* Add license headers

* Test fixes

* Test fixes

* Add changelogs

* Address PG

* Test fix

* Test fix

* Test fixes

* Review notes

* Work on tests

* Test fixes

* Test fix

* Work on tests

* Tets fix

* Test fixes

* Test fixes

* Add missing exception codes

* Test fix

* Test fixes

* More test fixing

* License headers

* Test fix

* Add new test logging

* Work on tests

* Test fixes

* Test fix

* Resolve fixme

* Try to avoid test failure

* Add import command

* Work on storage

* Fix error codes

* Fixes

* License header

* Build fix

* Build fixes

* Fix dep
2022-02-27 16:04:49 -05:00
Ben Li-Sauerwine 65776bbab3
Adds a NDJSON-capable Bulk Data Import Provider (#3039)
* NDJsonParser with one example test.

* Add test for empty NDJson, and fix bug for empty NDJson.

* Adds multi-Patient test, and fixes bug whereby all multi-line NDJSON would be put into the same line.

* Adds test for NDJson with newlines in it.

* Adds test for converting non-Bundle types to NDJSON failing.

* Confirm that we can only extract to Bundle types in test.

* Update hapi-fhir-base/src/main/java/ca/uhn/fhir/context/FhirContext.java

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Documents behavior of the NDJsonParser in FhirContext.

* Attempt to fix failing build by using TestUtil to clear context in the manner of r4 parser tests instead of dstu

Also clean up indentation.

* Adds a BulkDataImportProvider, with a single test.  More tests are forthcoming.

* Adds an additional test.

* Enhance tests to include content of job files.  Fix a bug whereby the content was never actually being added.

* Adds several tests for BulkImportProvider's polling operation.

* Adds requred Msg.code to errors.

* Apparently I can't duplicate Msg.code even if it's the same issue.

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2022-02-27 11:38:20 -05:00
jmarchionatto dbc9427a05
Issue 3391 2588 expand valueset regex include not working (#3410)
* Refactor ValueSet expansion code to simplify property filtering by using new Hibernate search V6 nested classes capabilities; use generic hibernate search queries where possible (mostly everywhere but regex queries) and add R4 integration tests  for both elastic and lucene.

* Refactor ValueSet expansion code to simplify property filtering by using new Hibernate search V6 nested classes capabilities; use generic hibernate search queries where possible (mostly everywhere but regex queries) and add R4 integration tests  for both elastic and lucene.

* Add comment for trick

* Add codes to exception messages

* Add test specific for the fix

* Move changelog to current release. Add upgrade notice.

* Improve fail-fast method

* Update upgrade instructions

* Try luck with new exception codes

* Update hapi-fhir-storage/src/test/java/ca/uhn/fhir/jpa/search/ElasticsearchNestedQueryBuilderUtilTest.java

Add missed quote

Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
2022-02-24 21:32:11 +00:00
James Agnew cc44deee1a
GraphQL Introspection Support (#3348)
* Start working on graphql Schema

* Start testing

* Work on introspection

* Work on introspection

* Use integers for date ordinals

* Add changelog

* GraphQL updates

* Ongoing wrk

* Cleanup

* Bump core lib

* Add changelog

* Clean up dependencies

* CLeanup

* Add missing message

* Test fix

* Change to force CI
2022-02-22 09:48:57 -05:00
Olivia You e55f9d510a
3400 bulk export rules incorrectly applied to group and patient exports (#3403)
* export rules are applied now

* add changelog and test

* add changelog

* Delete 2797-bulk-export-getting-incorrect-group-data.yml

* update test

* add test

Co-authored-by: olivia-you <olivia.you@smilecdr.com>
2022-02-20 11:39:21 -05:00
Tadgh c6122bcb1b Add license files 2022-02-18 19:11:29 -08:00
Ken Stevens 631391819b
remove hapi jpa spring config hierarchy (#3390)
* initial refactor to remove hapi jpa spring config hierarchy

* fix poms

* fix batch config

* dependency rearrangement arising from further testing

* fix searchDao bean name

* fixed dstu2 tests

* consolidate validation config

* dstu2 validation regression (man dstu2 support is expensive)

* fix test (with WIP comment)

* restore poms

* fix demo projects

* fix build

* bumping version

* rename generated config class
2022-02-16 20:44:03 -05:00
Jaison B ecdb8c2ce6
Fix datesearch issue to resolve CI build failures (#3375) 2022-02-10 19:55:31 +00:00
James Agnew 0f2fc7a882
Allow search narrowing and auth interceptor by `token:in` (#3360)
* Optmize valueset expansion

* Working

* Version bump

* Add documentation

* Add test

* Checkstyle message cleanup

* Add reverse rule

* Test ficx

* Test fix

* Test fix

* Test fixes

* Test fixes

* Test fix

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/search_narrowing_interceptor.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/search_narrowing_interceptor.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Test fixes

* Fix conflict

* Add setter

* Test fixes

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-02-09 14:27:14 -05:00
Tadgh 6982c4d680
bump version to 6.0.0-PRE1-SNAPSHOT (#3353)
* bump version

* Add new version enum

* Bumped to 6.0.0
2022-02-02 12:11:31 -08:00
Johnson Lu c42ecc5c08
Add error codes to all exceptions (#3278)
* Initial design of adding exception codes to exceptions

* Initial design of adding exception codes to exceptions

* change int module name to string

* add comment

* shorten message

* shorten message

* added checkstyle

* developing checker

* completed checkstyle plugin and unit test for it

* fix NPE

* fix checker so it ignores throws on exceptions that aren't new

* change parameter order in InvalidResponseException

* accept Msg.code() in any parameter since some exceptions like MessageException have the message string as the second parameter

* exclude generated files

* BIG COMMIT added error codes to all exception throws

* example test fix

* Modified tests such that the assert statements would better match the message responses.

* Modified additional tests such that the assert statements would better match the message responses.

* fix test

* merge master

* merge master

* Fourth commit of modified tests such that the assert statements would better match the message response

* Changes made to modify assert statements such that they match the new errors which now use error codes. Also changed ResourceNotFoundException to include error codes in its messages.

* Modified assert messages to better match new error codes

* Time Zone changes DRAFT. Certain tests pass in intellij and fail during mvn install

* Modified assert messages to better match new error codes

* Added comments for time zone fixes. Previous commit comment on time zone fixes was incorrect. By invaliding caches, all tests on both intellij and mvn install passed.

* Reverted changes on azure-pipelines.yml

* updated hapi-fhir version for checkstyle module

* changes to azure-pipelines.yml

* changes to azure-pipelines.yml

* change to pom.xml to include hapi-fhir-checkstyle in ALLMODULES

* changes to azure-piplines (these changes are to be reverted after testing)

* reverted changes to azure-piplines

* reverted change where hapi-fhir-checkstyle was added to pom.xml (hapi-fhir)

* Attempted fix on checkstyle build error

* modified azure-piplines.yml to change maven repository for checkstyle build

* Attempted fix on checkstyle build error

* checkstyle config location change for hapi-depoloyable-pom. (will be reverted if it does not work)

* attempted dependency add for hapi-depoloyable-pom. (will be reverted if it does not work)

* reverted changes for hapi-deployable-pom

* add javadoc

* update last code

* Attempted fix on checkstyle build error

* Finished adding in missing error codes to pass Checkstyle checks

* Test pipelining

* Fixing errors and bumping version (as suggested by Gary)

* Merge

* Fixing Checkstyle error

* Attempted fix for date error

* Add error code onto assert statement

* Add in a couple more error codes

* Fixing assert statements

* Bump to static version for dependency

* Adding error code to message

* Adding error codes to exceptions

* Adding error codes to exceptions

* Change to work with cdr branch

* Update to faulty error message

* Merge changes

* allow checkstyle pom to deploy

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Simon Zuccherato <simon.zuccherato@smilecdr.com>
Co-authored-by: Predap <simon.zuccherato@gmail.com>
Co-authored-by: Predap <46201929+Predap@users.noreply.github.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-01-31 13:10:15 -08:00
katiesmilecdr a11d72ba51
3322 graph ql searches cannot be executed when number of matching results exceeds default page size (#3323)
* fix NPE

* check if size is null
2022-01-24 11:53:21 -05:00
James Agnew 715a4c4602
Fix resource version regression caused by Hibernate 5.6 (#3325)
* failing test

* failing test

* Fixes

* Updated note to self

* Improve pre-caching of resource IDs

* Work on tests

* Test fixes

* Add changelog

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-01-23 19:05:02 -05:00
longma1 ab22fd6e82
2718 cross partition support for subscription (#3298)
* added settings for cross-partition to Dao and Model config

* modified subscription validator to validate cross partition subscriptions

* modify subscription matcher to allow matching on all partitions when subscription is set to cross partition

* added cross partition to canonicalSubscription

* end of day commit, changed how we are checking if default partition in subscription validator

* fixed issue with i memory matching for cross partition subscriptions

* added test for parsing legacy CanonicalSubscription

* added changelog and doc changes

* addressed comments in pr

* added assertdoesnotthrow

* test fixes, added new mocks

* added check for if resource can have extension before checking for extensions

* removed unnecessary semicolon

Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
2022-01-21 15:04:39 -07:00
James Agnew 3fa76a9027
Add support for of-type token modifier (#3309)
* Add support for of-type token modifier

* Add changelog

* Clean up hash calculation logic

* Test fix

* Rework hash generation

* Work on hash generation

* Test fix

* Avoid redundant hash calculations

* Revert some useless changes

* Cleanup

* Test fixes
2022-01-21 16:10:51 +00:00
TipzCM 221d7ed5b7
3274 transactions with full request url should be rejected (#3280)
* 3274 throw if url is full url

* 3274 will throw on invalid urls

* 3274 fixing the tests

* 3274 cleanup

* anonymize the data

* review fixes

* 3274 fixed for backward compatibility

* added review fixes

* fixing tests

* some more test fixes

* test fixes

* 3274 fixing the check to only look for resource at begining

* 3274 fixed test

* test fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-17 08:46:32 -08:00
jamesagnew aafc42fdd8 License header updates 2022-01-04 13:37:10 -05:00
James Agnew d610d33ac3
Dependency bumps (#3264)
* Bump a number of versions

* Fixes

* Work on version bump

* Bump

* Test fix

* More version bumps

* Build fixes

* Test fixes

* Fixes

* Fixes

* Test fix

* Bump jacoco

* Test fix

* Test fix

* Test fix

* Build fix

* Build fix

* Remove versioned H2

* Build fix

* Avoid memory issue in validation tests

* JDK bump

* Restore compile with errorprone

* Force coloured output

* Force jansi

* Try again to force jansi

* Fix for hardcoded date that just passed

* Fixes to JPA

* Improve changelog

* Work on failing test

* Test fix

* Compile fix

* One more version bump

* Test fixes
2022-01-04 13:21:14 -05:00
James Agnew d9820bfb89
Add inline resource mode (#3267)
* Start work on inline resource text storage mode

* Work on inlining

* Add changelog

* Test fixes

* Fix for hardcoded date that just passed
2022-01-03 11:43:45 -05:00
michaelabuckley 1c318aac1e
Bypass jpa for lastn results (#3226)
* Add flag in DaoConfig to enable storing resources in lucene index

* Update loadResourceByPid to use ElasticSearch for loading observation resource and test added to check it

* Added parsing to elastic search to parse inlined observation resources json to resource object
2021-12-17 22:38:43 +00:00
Ken Stevens a078306a7d merge master 2021-12-16 14:58:50 -05:00
TipzCM f3c17865d7
2521 delete subscription resource (#3235)
* 2521 will set subscriptionst hat cant be initialized to error state to prevent infinite retries

* 2521 add changelog

* cleanup

* fix issue in merge

* fix bad merge

* bad merge

* 2521 review fixes and test fixes

* 2521 review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-12-15 16:17:03 +00:00
jmarchionatto ef83777115
Allow Subscription to be configured to send delete messages (#3245)
* Allow Subscription to be configured to send delete messages

* Add tests and implement MR suggestions

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2021-12-14 16:32:56 -05:00
Tadgh 50a3005c23
Fix flag name (#3247)
* Fix flag name

* Bump hapi version
2021-12-14 19:20:03 +00:00
longma1 a56603daa2
2480 partition aware subscription (#3218)
* some fixmes to start

* Added some FIXMEs

* added RequestPartitionId to reosurceDelivryMessage and ResourceModifiedMessage

* ResourceDeliveryMessage and ResourceModifiedMessage tests

* fixed issue with test missing partitionHelperSvc mocked bean

* Added tests and implemented SubscriptionMatchingSubscriber for partition aware subscription

* modified implementation of partitionId in CanonicalSubscription

* Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)

* Revert "Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)"

This reverts commit fe40fb9733.

* Got added partitionId to subscriptions, added changes to make SubscriptionMatchingSubscriberTest work

* added SubscriptionTriggering test, also added partition support to subscriptionLoader

* Changed implementation for storing partition id of subscriptions from messages, refactored tests to new implementation

* added all subscription systemRequestDetails with all partition to subscription reader

* refactored a generic system request details with default all partition request

* Added test for dao subscriptions, fixes to get the test working

* added partition support for latest version delivery

* added doc changes and changelog for multitenancy subscription

* cleanup and added partitioned subscription manually trigger test

* fixed mocked subscriptionDao

* added package-info for subscription module

* some code review changes

* removed AllPartitionSystemRequestDetail, added new text for multitenant subscription

* renamed method for code review

* version bump to 5.7.0PRE7

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
2021-12-13 08:45:30 -07:00
VK-SMILECDR bd89cf022b
Improve mail service (#3230)
* move mail config to mail service constructor

* update subscription email sender

* added license text

* [2669] bump HAPI version to 5.7.0-PRE6-SNAPSHOT

* add changelog
2021-12-09 17:05:42 -05:00
Ken Stevens 9753b66610
2564 contained resource validation flag (#3225)
* First cut at multithreading bundle validation

* javadoc

* change test to a red-green test

* add daoconfig

* move bundle concurrency config from validationsupport to request validator

* Moved concurrent details from ValidationOptions to FhirValidator

* Remove bundleValidationThreadCount (it's set by creating an appropriate executor)

* validate threadpool prefixes

* FIXME cleanup

* reassign FIXMEs to JB

* only create the threadpool if it will be used

* Add the MdcTaskDectorator to all threadpools

* Add bundle path to concurrently validated bundle resource validation messages.

* Add test to check bundle entry path being added to validation messages

* Threaded validation of contained resources initial checkin

* FHIR validator refactored to maintain correct bundle path and async validation task

* Refactored to use validation complete invoke callback to support raw string resource or resource object

* switch from XML to JSON

* # Conflicts:
#	hapi-fhir-base/src/main/java/ca/uhn/fhir/validation/FhirValidator.java
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Updated to use new HAPI-FHIR version

* merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation

* Updated to use new HAPI-FHIR version

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit 910b7f676b.

* Revert "merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation"

This reverts commit c7d60fc3e6.

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit aa8a7af455.

# Conflicts:
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Fix Test dependencies after reverting changes.

* merge 'validate_contains_change' to 2564-contained-resource-validation-flag

* Disable threadpool prefix name validate check.

* merge master

* fix thread regexp

* pre-review cleanup

* change log

* fix test

* revert to XML as default validation serialization

* fix test

* fix test

Co-authored-by: Jaison B <jaisonb@gmail.com>
2021-12-08 15:18:54 -05:00
jamesagnew d481896d9b License header updates 2021-12-07 15:38:36 -05:00
jamesagnew 8aef580aed Bump core library 2021-12-07 10:57:43 -05:00
Mark Iantorno d60d07b697
Validate contains change (#3209)
* wip

* The updated changes to the validator that were pulled in from the core dependencies are not backwards compat. Need to bump HAPI version.

* pointing to stable core version

* review feedback

* Add changelog

Co-authored-by: markiantorno <miantorno@pop-os.localdomain>
2021-12-06 11:25:00 -05:00
James Agnew b7bf10ced4
Missed commits from #3214 (#3222)
* Move graphql provider

* Version bump

* Moving things around

* More reworking

* Fix
2021-12-06 06:49:52 -05:00
James Agnew d072357c2a
Move graphql provider (#3214)
* Move graphql provider

* Version bump
2021-12-05 17:22:20 -05:00
jamesagnew 1a1243db34 Missing fix from #3197 2021-11-30 16:15:09 -05:00
James Agnew aae53d6ec7
Improve loading performance (#3197)
* Synthea perf work

* Cleanup

* Work on performance

* Ongoing perf work

* More perf work

* Perf work

* Test fixes

* Updates

* Test fix

* Test fixes

* Clean up tests

* Test cleanup

* Cleanup

* Add test

* One more perf tweak

* Fixes

* Add changelog

* Add changelog

* License header update

* License header updates
2021-11-28 10:47:30 -05:00
TipzCM 3743fbc62f
test fix (#3193)
* test fix

* review fixes

* minor tweak

* little cleareR

* Update hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/match/registry/SubscriptionCanonicalizer.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-11-25 20:09:00 +00:00
Tadgh 41ef7cf961
Support multiple EID systems for MDM. (#3178)
* Add implementation, changelog, docs, and tests

* Fix potential NPE

* Use constant instead of *
2021-11-24 06:05:13 +00:00
michaelabuckley 7d7865ab17
Mb share jpa date tests (#3154)
* publish test-jar from hapi-fhir-storage

* Extract BaseDateSearchDaoTests up to hapi-fhir-storage.

Share a date search tests between JPA and Mongo.

* Use ITestDataBuilder for setup

* Move Mongo date cases into common set

* cleanup

* More cleanup

* naming
2021-11-10 21:07:36 +00:00
TipzCM 25c025cdee
3153 update unknowncodesystemwarningvallidationsupport to have configurable severities (#3155)
* 3153 updating the unknowncodesystemwarningvalidationsupport class

* 3153 updating version for smile

* 3153 update broken tests

* 3153 adding log

* 3153 cleanup

* 3153 cleanup

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-11-10 16:06:09 -05:00
TipzCM 59bf8b836f
Issue 3120 support adding retry extension (#3137)
* issue-1134 initial work on adding retry handler

* issue-1134 subdscription creation fixes

* issue-1134 update the test

* issue-1134 retry policy off canonicalsubscription

* issue 1134 base subscription changes

* add failing test

* issue 1134 retry configs on base parameters

* issue 1134 passing forward params

* issue-1134 added more tests for making sure subscriptions create their channels correctly

* issue-1134 updates to get the retry to channels

* issue-1134 updating channel factory

* issue-1134 remove the dlq since it's not defineable

* issue-3120 changelog added

* issue-3120 cleaning up

* issue-3120 test fix

* issue-3120 review fixes

* issue-3120 fixed bad master merge

* issue-3120 updates for new release

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-11-03 18:21:10 -04:00
Tadgh a10c58e839
Bump version, add version to versionenum (#3135) 2021-11-02 09:40:56 -04:00
Tadgh 7c63441663
Revert "Issue 1134 retry mechanic on subscriptions (#3121)" (#3128)
This reverts commit 9b23e9e825.
2021-10-30 19:46:42 +00:00
TipzCM 9b23e9e825
Issue 1134 retry mechanic on subscriptions (#3121)
* issue-1134 initial work on adding retry handler

* issue-1134 subdscription creation fixes

* issue-1134 update the test

* issue-1134 retry policy off canonicalsubscription

* issue 1134 base subscription changes

* add failing test

* issue 1134 retry configs on base parameters

* issue 1134 passing forward params

* issue-1134 added more tests for making sure subscriptions create their channels correctly

* issue-1134 updates to get the retry to channels

* issue-1134 updating channel factory

* issue-1134 remove the dlq since it's not defineable

* issue-3120 changelog added

* issue-3120 cleaning up

* issue-3120 test fix

* issue-3120 review fixes

* issue-3120 fixed bad master merge

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-10-29 11:21:43 -04:00
Tadgh 8a7b5c1b2f
Multiple improvements (#2997)
* Add failing test

* Tidy test for new tokenparam style

* Add removeByQualifier to SearchParameterMap

* Add new property binder for search terms

* Member move, add test for new parser

* whoops missed an add

* Passing test.

Introduced SearchParamTextWrapper to give a type for the binder.
Enhance FulltextSearchSvcImpl to understand TokenParams as well as StringParams.

* Add more to test, change analyzer for template

* Optimize imports

* Minor refactoring

* added eq and gt test cases

* added missing gt search param tests

* Search all CodeableConcepts

* Extract new LuceneRuntimeSearchParam to mediate indexing and searching.

* Fix up broken test with mock

* Add support for identifier type

* Bump for special maven package

* fix-me to todo

* Prevent version check

* FIx version for test

* DSTU2 does not support fluent path

* Don't accidentally strip SPs in DSTU2

* added lt tests

* Add some high-bound test cases

* More test cases and new compressed format

* More test cases

* More test cases

* Re-order columns for easier reading

* Comments

* Cleanup and comments

* Make support :text for all token SPs

* Make support :text for all token SPs

* Fixed incomplete date issue

* Set to the last millisecond 23:59:59.999

* Disabled 4 failed JVM/TZ related test cases

* add test for failing batch bug, remove bounding on queue

* remove parallelism from non-get batch operations

* Fix count for test

* New index layout

* Notes and cleanup

* Notes and cleanup

* remove the jetbrains

* Demote FIXME to WIP

* Change :text search to be lucene simple search and use standardAnalyzer.

* Change :text to require * for prefix match.

* add prefix negative test

* Bump version

* Added changelog

* Add more search parameters to lucene indexing (#3000)

* dirty the branch

* Add oversight to changelog

* Start using jpa indexing for es

* Correct negative search test

* Update templates, index tokens, write new test. TODO gotta modify SB

* Add test for code token search

* add comment

* Flesh out token search

* minor cleanup and comments

* Extract index handling from FulltextSearchSvcImpl

* D'oh.  Actually use elastic to search.

* Move supported query check closer to query builder

* String search test before activating lucene

* Add string:exact to hibernate search.

* Add string:contains search

* cleanup

* demote fixmes to allow build to run.

* Add unmodified string search.

* empty stubs for quantity and reference

* Ignore magic search params in lucene index

* Support full-text on all FHIR releases

* Include reference lookups in ES

* Fix and/or logic in _text

* Test for string with space

* Cherry-pick the NoFT fix

* Disable advanced indexing in a bunch of tests of emitted sql.

* Stub changelog

* Fix :missing flag

* Move DaoConfig up to BaseJpaTest to share teardown

* Disable new indexing for partition test

* Add a different analyzer and field for default string search vs text

* checkstyle for a pre-build

* Index full prefix for string search when using Lucene

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3000-Extend-lucene-indexing.yaml

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/HibernateSearchQueryBuilder.java

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Review feedback.

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Remove double-check

* Version bump

* Fix typo

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Long Ma <longma@Longs-MacBook-Pro.local>
Co-authored-by: Frank Tao <frankjtao@gmail.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2021-10-28 15:13:16 -04:00
longma1 2067ba83ee
minor refactor to extract static string variables into a singel constants class in bulk export (#3117)
Co-authored-by: Long Ma <longma@Longs-MBP.hitronhub.home>
2021-10-28 16:04:31 +00:00
James Agnew 5f672bc9e8
Deprecate redundant BaseResourceModifiedMessage myId (#3116)
* Deprecate redundant BaseResourceModifiedMessage myId

* Add changelog

* Test fix

* Test fix
2021-10-28 07:55:46 -04:00
StevenXLi 34010fa78d
Batch job refactor (#3087)
* added AbstractResourceToFileWriter

* Added AbstractBaseBulkItemReader

* added bulkExportStatusEnum fromBatchStatus test

* Code review name change

Co-authored-by: Long Ma <longma@Longs-MBP.hitronhub.home>
Co-authored-by: Steven Li <steven@smilecdr.com>
2021-10-19 09:07:05 -04:00
JasonRoberts-smile b5e31abf9b
add authorization check to inline matches (#3048)
* add authorization check to inline matches

* code review feedback: cached results case

* Add query count test

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3047-inline-match-security.yaml

Accept code review suggestion

Co-authored-by: James Agnew <jamesagnew@gmail.com>

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2021-10-18 12:21:26 +00:00
Tadgh b427135b04
Patient/$everything _id parameter (#3077)
* Add tests. Add implementation in R4. Add changelog

* Wip partial fixup of other STU versions

* Updated usages of new patient type everything api

* Partial commit of incomplete documentation

* Remove docs

* Cut to token instead of string

* Bump hapi version due to breaking api changes
2021-10-17 14:34:33 +00:00
James Agnew 310a4cdb58
Migrate CLOB columns to LOB (SMILE-3072) (#3016)
* Migration finished

* Work on LOBS

* Add to ClobMigrated annotation

* Deal with lobs

* Work on new approach

* HAPI FHIR version bump

* Add changelog

* Add license header

* Fix intermittent

* CLeanup

* Cleanup
2021-10-11 16:47:37 -04:00
James Agnew 756b0fdc7b
Fix transaction scoping (#2799)
* Work on transaction boundaries

* Test fixes

* Test fixes

* Test fix

* Resolve FIXME

* Test fix

* Test fix

* Test fix

* Test fixes

* Test fix

* Adjust changelog

* Remove unneeded changelog

* Test fix

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Fixes

* Test fix
2021-10-11 16:45:22 -04:00
Tadgh bc2cd7cafc
premature optimization is the root of all evil (#3053) 2021-10-06 14:44:39 +00:00
Tadgh 59afbe086c
Allow canonical references to support treating given base URLs as local. (#3050)
* Add failling test, works on #2843

* Partial hack to get this working

* Move implementation into pre-processing

* Add changelog

* Move implementation

* Minify Test

* Remove fixmes

* Tidying

* Ensure this only operates on STU versions which support canonical
2021-10-05 11:09:49 -04:00
James Agnew d320b78ae1
Improve synthea ingest performance (#3033)
* Improve synthea ingest performance

* Tweaks

* Fix param splitting

* Update to test

* Add changelog

* Test fix

* Add some runtime checks

* Remove useless test

* Test fix
2021-10-04 05:46:06 -04:00
jamesagnew d4f01ac4e7 License header updates 2021-10-03 20:20:04 -04:00
Tadgh 040d09cfa6 Remvoe error in moving code over 2021-09-28 16:43:29 -04:00
Tadgh 7dfe628551 remove parallelism from non-get batch operations 2021-09-27 15:32:03 -04:00
Ken Stevens 0db40c6118 support non-jpa batch 2021-09-24 00:39:15 -04:00
Ken Stevens 53c7476dae
hapi storage (#3024)
* first pass moving core storage classes to storage-api

* move term service apis

* nearly done

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* create new SearchConstants class to store platform independent search constants

* move a couple of subscription services

* move transaction processor adapter to storage

* version bump

* move searchparam

* fix test

* fix compile includes

* fix text

* fix docs
2021-09-23 22:04:46 -04:00