Commit Graph

3789 Commits

Author SHA1 Message Date
Tadgh d11a312abf This bean should not be lazy 2022-03-03 18:59:14 -08:00
Tadgh a2133f6940
Break a couple Circular Bean Dependencies (#3437)
* Rely on context refresh event instead of postconstruct. Load bean lazily.

* Remove parent autowire

* Break termcodesystem circular dependency with new bean

* Break cycle between BaseTermReadSvcImpl and TermCodeSystemStorageSvcImpl

* move back into helper method

* Rip out scheduling and submission to spring batch away from BulkDataExportSvcImpl

* wip

* Fix bean naming

* Revert

* re-add autowiring

* Reformat

* docs and reformat

* Back out change

* feedback from review

* merge conflicts
2022-03-03 09:37:40 -05:00
jmarchionatto 4b5f75c069
Issue 3421 pre expansion fails for valuesets with more than 1024 concepts (#3434)
* Build search queries involving many predicates avoiding crashing when number is larger than configured BooleanQuery.maxClauseCount

* Add changelog

* Restore temp directory config

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-03-02 14:11:55 -05:00
James Agnew f614eaf6c8
Avoid error when unknown code system behaviour set to warning (#3436)
* Avoid error when unknown code system behaviour set to warning

* Add changelog

* Test fix

* Build fix

* Fixes

* Address review comment
2022-03-01 17:30:59 +00:00
James Agnew 2cba62b4e8
Enable search narrowing on large ValueSets (#3405)
* Enable search narrowing on large ValueSets

* ValueSet improvements

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Work on narrowing

* Add test

* Work on narrowing interceptor

* Work on narrowing

* License headers

* Refactor code narrowing

* Add docs

* Version bump

* Test fix

* Test fixes

* Build fix

* Fixes

* Version bump

* Test fix

* License header updates

* Build fix

* Test fixes

* Test fix

* Test fix

* Docs fix

* Test fix

* Test fix

* Resolve fixme

* Bump

* Force a CI build

* Make CI happen again
2022-03-01 09:16:31 -05:00
Tadgh 68c1015552
Bump core to 5.6.36 (#3433)
* Bump version

* Add changelog, make use of theNoInactive flag

* add test

* remove magic strings
2022-03-01 08:23:48 -05:00
michaelabuckley 86f7373f16
Distinguish multi-coded items during token autocomplete (#3429)
Add nested indexing to support autocomplete of multi-coded items.
2022-03-01 00:59:09 +00:00
kailiu-smile 1bc6e0e3ae
Smile 2775 package install logs ignore resource validation (#3381)
* Added log to the create() and valiForUpload() methods in the PackageInstallerSvcImpl class

* Added helper methods to extract URL from SearchParameter.Tested and fixed log entries.

* Changelog entry

* Update 3384-Package-Install-added-logs-for-resources-validation.yaml

Co-authored-by: Kai Liu <kliu@Kais-MacBook-Pro.local>
2022-02-28 19:52:43 +00:00
James Agnew f2afb0844d
Fix migration (#3428) 2022-02-27 21:25:14 -05:00
James Agnew b7d1ae217d
New batch framework / Bulk Import (#3387)
* Work on new batch framework

* Work on new batch framework

* Work on new batch framework

* Work on batch

* Compile working

* Work on bulk import

* Adjust import

* Work

* Work on batch

* Bump version

* WOrk on new batch processes

* Work on bath

* Bump to PRE4

* Build fixes

* CLeanup

* Small tweak

* Add exception code

* Test fixes

* Test fixes

* Test fix

* Additional synchronization

* Add license headers

* Test fixes

* Test fixes

* Add changelogs

* Address PG

* Test fix

* Test fix

* Test fixes

* Review notes

* Work on tests

* Test fixes

* Test fix

* Work on tests

* Tets fix

* Test fixes

* Test fixes

* Add missing exception codes

* Test fix

* Test fixes

* More test fixing

* License headers

* Test fix

* Add new test logging

* Work on tests

* Test fixes

* Test fix

* Resolve fixme

* Try to avoid test failure

* Add import command

* Work on storage

* Fix error codes

* Fixes

* License header

* Build fix

* Build fixes

* Fix dep
2022-02-27 16:04:49 -05:00
Ben Li-Sauerwine 65776bbab3
Adds a NDJSON-capable Bulk Data Import Provider (#3039)
* NDJsonParser with one example test.

* Add test for empty NDJson, and fix bug for empty NDJson.

* Adds multi-Patient test, and fixes bug whereby all multi-line NDJSON would be put into the same line.

* Adds test for NDJson with newlines in it.

* Adds test for converting non-Bundle types to NDJSON failing.

* Confirm that we can only extract to Bundle types in test.

* Update hapi-fhir-base/src/main/java/ca/uhn/fhir/context/FhirContext.java

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Documents behavior of the NDJsonParser in FhirContext.

* Attempt to fix failing build by using TestUtil to clear context in the manner of r4 parser tests instead of dstu

Also clean up indentation.

* Adds a BulkDataImportProvider, with a single test.  More tests are forthcoming.

* Adds an additional test.

* Enhance tests to include content of job files.  Fix a bug whereby the content was never actually being added.

* Adds several tests for BulkImportProvider's polling operation.

* Adds requred Msg.code to errors.

* Apparently I can't duplicate Msg.code even if it's the same issue.

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2022-02-27 11:38:20 -05:00
Ken Stevens c7be674b84
small performance optimization (#3426) 2022-02-26 02:35:23 +00:00
jmarchionatto dbc9427a05
Issue 3391 2588 expand valueset regex include not working (#3410)
* Refactor ValueSet expansion code to simplify property filtering by using new Hibernate search V6 nested classes capabilities; use generic hibernate search queries where possible (mostly everywhere but regex queries) and add R4 integration tests  for both elastic and lucene.

* Refactor ValueSet expansion code to simplify property filtering by using new Hibernate search V6 nested classes capabilities; use generic hibernate search queries where possible (mostly everywhere but regex queries) and add R4 integration tests  for both elastic and lucene.

* Add comment for trick

* Add codes to exception messages

* Add test specific for the fix

* Move changelog to current release. Add upgrade notice.

* Improve fail-fast method

* Update upgrade instructions

* Try luck with new exception codes

* Update hapi-fhir-storage/src/test/java/ca/uhn/fhir/jpa/search/ElasticsearchNestedQueryBuilderUtilTest.java

Add missed quote

Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: michaelabuckley <michael.buckley@smilecdr.com>
2022-02-24 21:32:11 +00:00
jmarchionatto 53c5a9bbeb
Add FK index to avoid timeout when deleting large ValueSet (#3416)
* Add FK index to avoid timeout when deleting large ValueSet

* Add exception to allow same name for index and FK

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-02-23 13:54:30 -05:00
James Agnew cc44deee1a
GraphQL Introspection Support (#3348)
* Start working on graphql Schema

* Start testing

* Work on introspection

* Work on introspection

* Use integers for date ordinals

* Add changelog

* GraphQL updates

* Ongoing wrk

* Cleanup

* Bump core lib

* Add changelog

* Clean up dependencies

* CLeanup

* Add missing message

* Test fix

* Change to force CI
2022-02-22 09:48:57 -05:00
James Agnew 1e2e17784a
Fix parameter count limitation on MSSQL/Oracle large transactions (#3075)
* Fix parameter count limitation on MSSQL/Oracle large transactions

* Changelog fix

* Move changelog

* Force a change to trigger CI

* Test refactoring
2022-02-20 11:52:42 -05:00
James Agnew 058e53616f
Use integers for date ordinals (#3346)
* Use integers for date ordinals

* Add changelog

* Adjust changelog
2022-02-19 18:50:18 -05:00
Tadgh c6122bcb1b Add license files 2022-02-18 19:11:29 -08:00
Tadgh 3561672fe7
Rel 5 7 mergeback (#3401)
* fix handling of common search parameters

* Revert "fix handling of common search parameters"

This reverts commit 89c45eebdc.

* Fix implementation, add test (#3378)

* Fix implementation, add test

* Tighten test

* Rip out dead modules

* Add changelog

* Jr 20220210 handle common search params in contained searches (#3377)

* fix handling of common search parameters

* add support for reference search parameters with multiple paths

* Issue 3357

* Version bump

* Fixed null pointer exception for re-loading subscription on cdr restart and when there's no partition id in the request, and added tests

* added changelogs for this fix

* Fix broken changelog file

* Can't specify specific resource type permissions for bulk export (#3376)

* deny user from exporting without perms

* add unit tests

* add changelog

Co-authored-by: olivia-you <olivia.you@smilecdr.com>

* Make migration donothing as it was added in error

* Remove ehcache

* Add version.yaml:

* Add to sources/javadocs for dist

* Fix typo

* fix up pom

* wip test removing checkstyle plugin from deployable pom

* remove test pom changes, instead just dont deploy to sonatype

* typo

* Remove ehcache

Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
Co-authored-by: JasonRoberts-smile <85363818+JasonRoberts-smile@users.noreply.github.com>
Co-authored-by: Mark Iantorno <markiantorno@gmail.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Olivia You <46392181+oliviayou@users.noreply.github.com>
Co-authored-by: olivia-you <olivia.you@smilecdr.com>
2022-02-18 19:22:44 +00:00
Ken Stevens 631391819b
remove hapi jpa spring config hierarchy (#3390)
* initial refactor to remove hapi jpa spring config hierarchy

* fix poms

* fix batch config

* dependency rearrangement arising from further testing

* fix searchDao bean name

* fixed dstu2 tests

* consolidate validation config

* dstu2 validation regression (man dstu2 support is expensive)

* fix test (with WIP comment)

* restore poms

* fix demo projects

* fix build

* bumping version

* rename generated config class
2022-02-16 20:44:03 -05:00
michaelabuckley 1c82d0933c
Mb new date search index (#3368)
* New date index definitions.
* Dropped unused indexes.
* Add online option to drop index and add index
* Push sp->resource FK down to control name
* Update annotations to match and redefine FK
* Continue to allow the legacy hibernate names while we update the indexing.
2022-02-16 22:12:20 +00:00
michaelabuckley ae33cf825b
Handle token search by partial code, and various nulls (#3399) 2022-02-16 08:23:16 -07:00
Tadgh 700221ad07
return concretions (#3373) 2022-02-10 21:07:47 -05:00
Tadgh 38912423c1
Remove dead modules (#3380)
* fix handling of common search parameters

* Revert "fix handling of common search parameters"

This reverts commit 89c45eebdc.

* Fix implementation, add test (#3378)

* Fix implementation, add test

* Tighten test

* Rip out dead modules

* Add changelog

Co-authored-by: Jason Roberts <jason.roberts@smilecdr.com>
2022-02-10 16:44:26 -05:00
Jaison B ecdb8c2ce6
Fix datesearch issue to resolve CI build failures (#3375) 2022-02-10 19:55:31 +00:00
StevenXLi 7841e578ab
2789 nullpointerexception rest hook subscription is not being triggered when cross partition is (#3370)
* Fixed null pointer exception for re-loading subscription on cdr restart and when there's no partition id in the request, and added tests

* added changelogs for this fix

Co-authored-by: Steven Li <steven@smilecdr.com>
2022-02-10 14:14:41 -05:00
Tadgh 43f570259e Merge branch 'rel_5_7' 2022-02-10 08:44:07 -08:00
Jaison B 7794113455
2838 lastn refactor to use extended lucene index (#3339)
* Change token and reference search params to 'keyword' field type

* remove normalizer for keyword field

* WIP - refactor lastN operation to use hibernate search generated ES index.

* Add ES native aggregation builder for lastN

* added date search param support for searching resourcetable ES index

* * Added DateSearch parameterized tests to ElasticSearch test suite

* Added more tests cases for Date searches

* WIP refactor LastN test data to central generator class and use it for both hibernate-search and elastic search implementations.

* WIP fix lastNDataGenerator to avoid assertion errors.

* Fixed tests to run test suite on both existing lastN search implementation and newer extended lucene index based search.

* Cleanup json string to avoid carriage return noise

* Fix checkstyle issues

* Fix checkstyle Msg code issues

* Fix checkstyle Msg code issues

* Add more test case for GT, SA precision modifier

* Fix date precision test cases

* Fix Msg Code conflicts

* Add more logging

* Add way more loggging

* even more logging

* fix msg code conflicts

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-02-10 11:05:31 +00:00
michaelabuckley 3531d9b4fc
Fix exception when contained indexing met hibernate search. (#3372)
* Fix exception when contained indexing met hibernate search.
2022-02-09 20:27:40 -05:00
James Agnew 0f2fc7a882
Allow search narrowing and auth interceptor by `token:in` (#3360)
* Optmize valueset expansion

* Working

* Version bump

* Add documentation

* Add test

* Checkstyle message cleanup

* Add reverse rule

* Test ficx

* Test fix

* Test fix

* Test fixes

* Test fixes

* Test fix

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/search_narrowing_interceptor.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/search_narrowing_interceptor.md

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Test fixes

* Fix conflict

* Add setter

* Test fixes

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-02-09 14:27:14 -05:00
Tadgh 0ccf1bfa52 Merge branch 'rel_5_7' 2022-02-07 12:11:35 -08:00
Patrick Werner 2531e6e57d
feat: added support for package.tgz uri in the "file:/" format (#3347)
* feat: added support for package.zgz uri in the "file:/" format

* added changelog entry

* add error code to thrown message

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-02-05 13:25:50 -05:00
Tadgh 1af4912603 Merge branch 'rel_5_7' 2022-02-04 22:22:02 -08:00
James Agnew 1d5714aec2
Fix #3261 - Incorrect revinclude selector (#3358) 2022-02-04 07:50:13 -05:00
TipzCM f6181fb6ff
3354 fixing flakey tests (#3355)
* 3354 fixing flakey tests

* must fix code

* updating number again

* fixing test

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-02-03 18:30:58 -05:00
michaelabuckley 6b15754b4a
Mb auto complete code (#3330)
New autocomplete extension to ValueSet $expand with contextDirection=existing
2022-02-02 21:03:53 +00:00
Tadgh 6982c4d680
bump version to 6.0.0-PRE1-SNAPSHOT (#3353)
* bump version

* Add new version enum

* Bumped to 6.0.0
2022-02-02 12:11:31 -08:00
michaelabuckley 717ff71695
Simplify JPA Elasticsearch Test Configuration (#3345)
* Delete TestR4WithLuceneDisabledConfig

* Split off the ES and ESClient test configs

* move hibernate search config out to add-ins

* Use addIn for all R4

* Finish R4

* Finish R3

* Finish dstu2

* R5

* Remove property override, and docs

* Apply error code to error

* Fix test to match new error code

* Lighten test

* Disabling broken test.
2022-02-02 16:09:19 +00:00
Johnson Lu c42ecc5c08
Add error codes to all exceptions (#3278)
* Initial design of adding exception codes to exceptions

* Initial design of adding exception codes to exceptions

* change int module name to string

* add comment

* shorten message

* shorten message

* added checkstyle

* developing checker

* completed checkstyle plugin and unit test for it

* fix NPE

* fix checker so it ignores throws on exceptions that aren't new

* change parameter order in InvalidResponseException

* accept Msg.code() in any parameter since some exceptions like MessageException have the message string as the second parameter

* exclude generated files

* BIG COMMIT added error codes to all exception throws

* example test fix

* Modified tests such that the assert statements would better match the message responses.

* Modified additional tests such that the assert statements would better match the message responses.

* fix test

* merge master

* merge master

* Fourth commit of modified tests such that the assert statements would better match the message response

* Changes made to modify assert statements such that they match the new errors which now use error codes. Also changed ResourceNotFoundException to include error codes in its messages.

* Modified assert messages to better match new error codes

* Time Zone changes DRAFT. Certain tests pass in intellij and fail during mvn install

* Modified assert messages to better match new error codes

* Added comments for time zone fixes. Previous commit comment on time zone fixes was incorrect. By invaliding caches, all tests on both intellij and mvn install passed.

* Reverted changes on azure-pipelines.yml

* updated hapi-fhir version for checkstyle module

* changes to azure-pipelines.yml

* changes to azure-pipelines.yml

* change to pom.xml to include hapi-fhir-checkstyle in ALLMODULES

* changes to azure-piplines (these changes are to be reverted after testing)

* reverted changes to azure-piplines

* reverted change where hapi-fhir-checkstyle was added to pom.xml (hapi-fhir)

* Attempted fix on checkstyle build error

* modified azure-piplines.yml to change maven repository for checkstyle build

* Attempted fix on checkstyle build error

* checkstyle config location change for hapi-depoloyable-pom. (will be reverted if it does not work)

* attempted dependency add for hapi-depoloyable-pom. (will be reverted if it does not work)

* reverted changes for hapi-deployable-pom

* add javadoc

* update last code

* Attempted fix on checkstyle build error

* Finished adding in missing error codes to pass Checkstyle checks

* Test pipelining

* Fixing errors and bumping version (as suggested by Gary)

* Merge

* Fixing Checkstyle error

* Attempted fix for date error

* Add error code onto assert statement

* Add in a couple more error codes

* Fixing assert statements

* Bump to static version for dependency

* Adding error code to message

* Adding error codes to exceptions

* Adding error codes to exceptions

* Change to work with cdr branch

* Update to faulty error message

* Merge changes

* allow checkstyle pom to deploy

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Simon Zuccherato <simon.zuccherato@smilecdr.com>
Co-authored-by: Predap <simon.zuccherato@gmail.com>
Co-authored-by: Predap <46201929+Predap@users.noreply.github.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2022-01-31 13:10:15 -08:00
TipzCM 65398ca14f
3156 multiple tag race condition (#3297)
* 3156 fix race condition that results in multiple tags being created

* 3156 added changelog

* cleanup

* added smile jira ticket

* 3156 review fixes

* review fixes

* 3156 added an additional integratoin test

* cleanup

* review fix and master merge

* threadsafe stuff

* small change

* upping min threads for tests to pass

* changing test

* test check

* fix

* test testing

* one more update

* using threadpoolutil

* temporary measure

* git push

* all builds

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-31 08:49:12 -08:00
TipzCM 41193c60db
3335 allow alternative character encoding length in searchparam (#3336)
* added fixme

* 2714 added support for specifying max code lengths for supported phonetic encoders

* 3335git status

* 3335 updated changelog

* 3335 review fixes

* review fixes

* fixing test

Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-27 17:29:07 +00:00
Tadgh 6506527dd6
Fix mdm config for single application context environment. (#3338)
* wip

* remove beans

* Remove javax.mail from the classpath as we use simplemail instead, which uses jakarta.
2022-01-27 06:18:44 -05:00
James Agnew 3f42414287
Validate include statements for JPA (#3331)
* Validate include statements for JPA

* Add changelog

* Optmize test fix
2022-01-26 09:16:28 -05:00
James Agnew 31bb1d9e9a
Bump RestEasy to 5.0.2 (#3329)
* Bump deps

* Test fix

* Add test logging

* Add test logging

* Add logging
2022-01-24 20:15:41 -05:00
katiesmilecdr a11d72ba51
3322 graph ql searches cannot be executed when number of matching results exceeds default page size (#3323)
* fix NPE

* check if size is null
2022-01-24 11:53:21 -05:00
Patrick Werner 37e7d6eea5
populate ValueSet.url in $expand Operation (#3241) 2022-01-24 08:12:56 -08:00
James Agnew 715a4c4602
Fix resource version regression caused by Hibernate 5.6 (#3325)
* failing test

* failing test

* Fixes

* Updated note to self

* Improve pre-caching of resource IDs

* Work on tests

* Test fixes

* Add changelog

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2022-01-23 19:05:02 -05:00
longma1 ab22fd6e82
2718 cross partition support for subscription (#3298)
* added settings for cross-partition to Dao and Model config

* modified subscription validator to validate cross partition subscriptions

* modify subscription matcher to allow matching on all partitions when subscription is set to cross partition

* added cross partition to canonicalSubscription

* end of day commit, changed how we are checking if default partition in subscription validator

* fixed issue with i memory matching for cross partition subscriptions

* added test for parsing legacy CanonicalSubscription

* added changelog and doc changes

* addressed comments in pr

* added assertdoesnotthrow

* test fixes, added new mocks

* added check for if resource can have extension before checking for extensions

* removed unnecessary semicolon

Co-authored-by: Steven Li <steven@smilecdr.com>
Co-authored-by: Long Ma <long@smilecdr.com>
2022-01-21 15:04:39 -07:00
jmarchionatto 62966de58b
Issue 3299 conformance validation is happening once per client endpoint thread instead as just once per client endpoint (#3319)
* Make RestfulClientFactory.myValidatedServerBaseUrls usage thread safe so capability validation request is sent only once per client-endpoint

* Can't use @GuardedBy for fields which are not always locked by annotation object

* Use same variable when checking and setting

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-01-21 14:29:24 -05:00
James Agnew 3fa76a9027
Add support for of-type token modifier (#3309)
* Add support for of-type token modifier

* Add changelog

* Clean up hash calculation logic

* Test fix

* Rework hash generation

* Work on hash generation

* Test fix

* Avoid redundant hash calculations

* Revert some useless changes

* Cleanup

* Test fixes
2022-01-21 16:10:51 +00:00
JasonRoberts-smile 48caff30d9
Improve performance of chained queries into contained resources (#3312)
* base restructuring of query

* fix unit tests

* suppress unnecessary resource type parameter

* pass the resource type used to fetch the search param as part of the chain, so later we do not need to guess what it was

* add query structure tests

* changelog

* fix test failures

* got one of the branches wrong in the 3-reference case
2022-01-20 21:22:02 +00:00
michaelabuckley 283ff19375
Re-index lucene when reindexing hibernate, even if resource unchanged. (#3311) 2022-01-19 23:02:39 +00:00
Tadgh 71e959c1be
SERVER_OUTGOING_RESPONSE invocation on manualResponse operations (#3315)
* Fix implementation so we can make the call anyhow

* modify test to ensure that pointcut is invoked

* Add changelog

* Tighten tests

* Add comment
2022-01-19 08:24:59 -08:00
James Agnew 80e4c3dd92
Bump HIbernate to 5.6.x (#3305)
* Bump HIbernate to 5.6.x

* Test fixes

* Test fix

* Dependency bump

* Rollback change

* Fix tests

* One more fix

* Avoid bump
2022-01-19 11:14:14 -05:00
TipzCM f0a9b33671
3283 delete will not return a 404 if resource is not found (#3284)
* 3283 delete will not return a 404 if resource is not found

* minor tweaks

* checking if tests fail due to asynchronicity

* 3283 added integration test

* cleanup

* review fixes

* changed id names

* cleanup

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-18 13:07:26 -05:00
Tadgh 9f7c454c6b
Issue 2377 missing entry full url in document (#3304)
* start with failing test

* wip

* add changelog

* Add implementation

* Fix DSTU3 impl

* Fix DSTU2 and R5
2022-01-18 10:40:53 -05:00
TipzCM 221d7ed5b7
3274 transactions with full request url should be rejected (#3280)
* 3274 throw if url is full url

* 3274 will throw on invalid urls

* 3274 fixing the tests

* 3274 cleanup

* anonymize the data

* review fixes

* 3274 fixed for backward compatibility

* added review fixes

* fixing tests

* some more test fixes

* test fixes

* 3274 fixing the check to only look for resource at begining

* 3274 fixed test

* test fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-17 08:46:32 -08:00
Mark Iantorno 10dea0796a
JUnit jupiter fix for @Disabled tag order. (#3300) 2022-01-14 01:43:56 +00:00
IanMMarshall e6fd5c9807
Fix batching of mdm-clear tasks (#3295)
* Fix batching of mdm-clear tasks.

* Fix batching of mdm-clear tasks.

Co-authored-by: ianmarshall <ian@simpatico.ai>
2022-01-13 12:06:27 -05:00
jmarchionatto 737409955a
Issue 3251 terminology reindexing job is never triggered (#3271)
* Run intended job

* Add test ti validate right method is invoked by inner job

* Add changelog

* Make test fail with proper message instead of crash

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-01-06 13:29:01 -05:00
IanMMarshall 095b959917
Fix PatientIdPartitionInterceptor where patient ID has hashCode equal to Integer.MIN_VALUE. (#3275)
* Fix PatientIdPartitionInterceptor for case where patient ID has hashCode equal to Integer.MIN_VALUE

* Fix PatientIdPartitionInterceptor for case where patient ID has hashCode equal to Integer.MIN_VALUE

Co-authored-by: ianmarshall <ian@simpatico.ai>
2022-01-06 10:00:33 -05:00
James Agnew c38fdc4c42
Fix inline resource migration (#3277) 2022-01-06 08:40:51 -05:00
jmarchionatto 2106631718
Disallow operation invocation by GET request (#3272)
Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2022-01-05 16:53:46 -05:00
TipzCM b148121ab3
3242 search content param removed not readded (#3244)
* 3242 initial changes and nonsense tests

* 3242 added changelog

* 3242 cleanup

* cleanup

* 3242 updated for review

* cleanup

* 3242 review fixes

* review fixes

* fix changelog file

* review fixes

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2022-01-04 19:38:52 +00:00
jamesagnew aafc42fdd8 License header updates 2022-01-04 13:37:10 -05:00
James Agnew d610d33ac3
Dependency bumps (#3264)
* Bump a number of versions

* Fixes

* Work on version bump

* Bump

* Test fix

* More version bumps

* Build fixes

* Test fixes

* Fixes

* Fixes

* Test fix

* Bump jacoco

* Test fix

* Test fix

* Test fix

* Build fix

* Build fix

* Remove versioned H2

* Build fix

* Avoid memory issue in validation tests

* JDK bump

* Restore compile with errorprone

* Force coloured output

* Force jansi

* Try again to force jansi

* Fix for hardcoded date that just passed

* Fixes to JPA

* Improve changelog

* Work on failing test

* Test fix

* Compile fix

* One more version bump

* Test fixes
2022-01-04 13:21:14 -05:00
James Agnew d9820bfb89
Add inline resource mode (#3267)
* Start work on inline resource text storage mode

* Work on inlining

* Add changelog

* Test fixes

* Fix for hardcoded date that just passed
2022-01-03 11:43:45 -05:00
michaelabuckley 1c318aac1e
Bypass jpa for lastn results (#3226)
* Add flag in DaoConfig to enable storing resources in lucene index

* Update loadResourceByPid to use ElasticSearch for loading observation resource and test added to check it

* Added parsing to elastic search to parse inlined observation resources json to resource object
2021-12-17 22:38:43 +00:00
Ken Stevens a41d79ed22
call delete expunge hook for pid list within delete expunge batch job (#3252)
* call delete expunge hook for pid list within delete expunge batch job

* review feedback
2021-12-16 21:35:23 -05:00
Tadgh 6a333c08bc Fix badly built migrations and indexes 2021-12-16 01:49:28 -08:00
Tadgh 700b614610 Allow system request details to go through MDM 2021-12-15 22:47:33 -08:00
Tadgh 9e20d62380
Issue 3237 delete expunge performance (#3239)
* Initial test looking for bad deletes

* Add indexes, add migration, add test'

* Fix npe

* Add provider test

* fix up comments, tighten test

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/delete/job/DeleteExpungeProcessor.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Add changelog

* Remove unnecessary migration

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2021-12-15 06:53:26 +00:00
jmarchionatto ef83777115
Allow Subscription to be configured to send delete messages (#3245)
* Allow Subscription to be configured to send delete messages

* Add tests and implement MR suggestions

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2021-12-14 16:32:56 -05:00
Tadgh 50a3005c23
Fix flag name (#3247)
* Fix flag name

* Bump hapi version
2021-12-14 19:20:03 +00:00
longma1 a56603daa2
2480 partition aware subscription (#3218)
* some fixmes to start

* Added some FIXMEs

* added RequestPartitionId to reosurceDelivryMessage and ResourceModifiedMessage

* ResourceDeliveryMessage and ResourceModifiedMessage tests

* fixed issue with test missing partitionHelperSvc mocked bean

* Added tests and implemented SubscriptionMatchingSubscriber for partition aware subscription

* modified implementation of partitionId in CanonicalSubscription

* Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)

* Revert "Moved PartitionablePartitionId, and refactored all calls to getUserData(Constants.RESOURCE_PARTITION_ID) and setUserData(Constants.RESOURCE_PARTITION_ID)"

This reverts commit fe40fb9733.

* Got added partitionId to subscriptions, added changes to make SubscriptionMatchingSubscriberTest work

* added SubscriptionTriggering test, also added partition support to subscriptionLoader

* Changed implementation for storing partition id of subscriptions from messages, refactored tests to new implementation

* added all subscription systemRequestDetails with all partition to subscription reader

* refactored a generic system request details with default all partition request

* Added test for dao subscriptions, fixes to get the test working

* added partition support for latest version delivery

* added doc changes and changelog for multitenancy subscription

* cleanup and added partitioned subscription manually trigger test

* fixed mocked subscriptionDao

* added package-info for subscription module

* some code review changes

* removed AllPartitionSystemRequestDetail, added new text for multitenant subscription

* renamed method for code review

* version bump to 5.7.0PRE7

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
Co-authored-by: Long Ma <long@smilecdr.com>
Co-authored-by: Steven Li <steven@smilecdr.com>
2021-12-13 08:45:30 -07:00
Ken Stevens d561577a9e
Ks 20211209 xml to java (#3236)
* try xml -> json

* Change from xml serialization to json serialization.  Also fix a deadlock in how the validation cache is used.

* changelog

* back out logback
change

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_7_0/3236-change-validation-from-xml-to-java.yaml

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
2021-12-12 00:29:52 +00:00
VK-SMILECDR bd89cf022b
Improve mail service (#3230)
* move mail config to mail service constructor

* update subscription email sender

* added license text

* [2669] bump HAPI version to 5.7.0-PRE6-SNAPSHOT

* add changelog
2021-12-09 17:05:42 -05:00
Ken Stevens 9753b66610
2564 contained resource validation flag (#3225)
* First cut at multithreading bundle validation

* javadoc

* change test to a red-green test

* add daoconfig

* move bundle concurrency config from validationsupport to request validator

* Moved concurrent details from ValidationOptions to FhirValidator

* Remove bundleValidationThreadCount (it's set by creating an appropriate executor)

* validate threadpool prefixes

* FIXME cleanup

* reassign FIXMEs to JB

* only create the threadpool if it will be used

* Add the MdcTaskDectorator to all threadpools

* Add bundle path to concurrently validated bundle resource validation messages.

* Add test to check bundle entry path being added to validation messages

* Threaded validation of contained resources initial checkin

* FHIR validator refactored to maintain correct bundle path and async validation task

* Refactored to use validation complete invoke callback to support raw string resource or resource object

* switch from XML to JSON

* # Conflicts:
#	hapi-fhir-base/src/main/java/ca/uhn/fhir/validation/FhirValidator.java
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Updated to use new HAPI-FHIR version

* merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation

* Updated to use new HAPI-FHIR version

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit 910b7f676b.

* Revert "merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation"

This reverts commit c7d60fc3e6.

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit aa8a7af455.

# Conflicts:
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Fix Test dependencies after reverting changes.

* merge 'validate_contains_change' to 2564-contained-resource-validation-flag

* Disable threadpool prefix name validate check.

* merge master

* fix thread regexp

* pre-review cleanup

* change log

* fix test

* revert to XML as default validation serialization

* fix test

* fix test

Co-authored-by: Jaison B <jaisonb@gmail.com>
2021-12-08 15:18:54 -05:00
jamesagnew 90d05a9ed4 Fix accidental commit 2021-12-07 14:55:16 -05:00
jamesagnew 8aef580aed Bump core library 2021-12-07 10:57:43 -05:00
Mark Iantorno d60d07b697
Validate contains change (#3209)
* wip

* The updated changes to the validator that were pulled in from the core dependencies are not backwards compat. Need to bump HAPI version.

* pointing to stable core version

* review feedback

* Add changelog

Co-authored-by: markiantorno <miantorno@pop-os.localdomain>
2021-12-06 11:25:00 -05:00
James Agnew b7bf10ced4
Missed commits from #3214 (#3222)
* Move graphql provider

* Version bump

* Moving things around

* More reworking

* Fix
2021-12-06 06:49:52 -05:00
James Agnew d072357c2a
Move graphql provider (#3214)
* Move graphql provider

* Version bump
2021-12-05 17:22:20 -05:00
katiesmilecdr 2681560598
fixed (#3217)
* fixed

* Add test

* Update test

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2021-12-04 00:38:51 +00:00
Ken Stevens 0656037c8d
Ks 20211203 mdm possible match (#3216)
* begin with failing test

* fix issue

* change log

* code review
2021-12-03 19:44:25 +00:00
Ken Stevens 2fbbf31431
Ks 20211202 fix major reindex bug (#3211)
* begin with failing test

* fix bug

* change log

* change log
2021-12-02 21:38:54 +00:00
James Agnew aa09e59255
Migrate to OpenAPI (#3206)
* Migrate to OpenAPI

* Update spring

* Test fix
2021-12-02 13:01:50 -05:00
James Agnew aae53d6ec7
Improve loading performance (#3197)
* Synthea perf work

* Cleanup

* Work on performance

* Ongoing perf work

* More perf work

* Perf work

* Test fixes

* Updates

* Test fix

* Test fixes

* Clean up tests

* Test cleanup

* Cleanup

* Add test

* One more perf tweak

* Fixes

* Add changelog

* Add changelog

* License header update

* License header updates
2021-11-28 10:47:30 -05:00
Tadgh 7ef1d159ee
Fix bug loading packages in non-database mode (#3199)
* Add implementation

* Add changelog
2021-11-26 20:17:51 -05:00
Patrick 3363807931
Support https with elastic (#3192)
* 3138 externalized binary packages (#3139)

* Add test and impl

* Add changelog

* Fix test

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3138-support-externalized-binaries.yaml

* add beans to test configs

* Typo

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/binstore/IBinaryStorageSvc.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/binstore/NullBinaryStorageSvcImpl.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

Co-authored-by: Kevin Dougan SmileCDR <72025369+KevinDougan-SmileCDR@users.noreply.github.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* 3131 - Added support for the lookup operation in the Remote Terminology code (#3134)

* Remove leading underscores from identifiers (#3146)

* Version bump

* License files

* version.yaml

* Support HTTPs

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
Co-authored-by: Kevin Dougan SmileCDR <72025369+KevinDougan-SmileCDR@users.noreply.github.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: ppalacin <patrick.palacin@i-atros.com>
2021-11-26 15:08:38 -05:00
jmarchionatto d496ca98ad
Issue 3172 support member match operation (#3187)
* Suppress unresolved maven properties errors to avoid IntelliJ complains

* Support for $member-match operation

* Add more reference cases

* Adjust tests

* Add test and handle missed condition

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
2021-11-26 14:20:14 -05:00
Kevin Dougan SmileCDR 10cad65534
Add NULL-safe check on the myValidationSupportChain as well as a setter method in case it needs to be injected. (#3190) 2021-11-25 07:59:35 -05:00
Kevin Dougan SmileCDR 3c2b4bb397
3136 - Provide RemoteTerminologyServiceValidationSupport Implementation For Operation $validate-code (#3142)
* 3136 - Send validate-code requests to an Instance of IValidationSupport instead of directly to the DAO.

* 3136 - Added support for R4 validate-code calls to a Remote Terminology Server.

* Added more Unit Tests.

* Added NULL checks based on code analysis.

* Adding some cleanup code to the Unit Tests since some other unrelated Unit Tests were failing when running in an Azure Pipeline.

* Adding @Ignore to both of my Unit Test classes to try and see if this is the cause of other unrelated Unit Test failures in the Azure Pipeline.

* Commented out ALL CODE in my Unit Tests to see if this fixes the other Unit Tests.

* Added 1 Unit Tester called RemoteTerminologyServiceResourceProviderR4Test to see if it causes other Unit Test failures.

* Added a Unit Tester called ResourceProviderR4RemoteTerminologyTest with the code that creates a RemoteTerminologyServiceValidationSupport instance commented out for now, just to see if this allows all other Unit Tests to pass.

* Added a Unit test back in to verify how the RemoteTerminologyServiceValidationSupport works when inserted into the front of the IValidation chain. Lets see if this will mess up any other Unit Tests in the Azure Pipeline...

* Added an @AfterEach method to remove the RemoteTerminologyServiceValidationSupport instance from the ValidationSupportChain to see if it allows all the other Unit Tests to work as before.

* Added more Unit tests.

* Added more Unit Tests to increase code coverage.

* Added some NULL checks.

* Added a changelog for this feature add.

* Removed the decision logic in the REST APIs and also the isRemoteTerminologyServiceConfigured() method, and letting the ValidationSupportChain figure out how to perform the validate-code Operation.

* Removed a NULL check that was not needed.

* Reverting the previous change where I tried using the IValidationSupportChain for each and every incoming Operation request. Splitting the implementation between local and remote seems to be required at this time.
2021-11-23 10:39:56 -05:00
Tadgh 0a64294467
Release 5.6.0 (#3174)
* 3138 externalized binary packages (#3139)

* Add test and impl

* Add changelog

* Fix test

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3138-support-externalized-binaries.yaml

* add beans to test configs

* Typo

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/binstore/IBinaryStorageSvc.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/binstore/NullBinaryStorageSvcImpl.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

Co-authored-by: Kevin Dougan SmileCDR <72025369+KevinDougan-SmileCDR@users.noreply.github.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* 3131 - Added support for the lookup operation in the Remote Terminology code (#3134)

* Remove leading underscores from identifiers (#3146)

* Version bump

* License files

* version.yaml

Co-authored-by: Kevin Dougan SmileCDR <72025369+KevinDougan-SmileCDR@users.noreply.github.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2021-11-19 15:24:44 +00:00
jmarchionatto 175958de5e
Issue 3108 terminology large codesystem deletion takes days to complete (#3160)
* Add indexes to foreign keys

* Add new version enum

* Add TermCodeSystem and TermCodeSystemVersion deletion job artifacts

* Add Propagation.NOT_SUPPORTED to allow starting jobs from transactional code

* New jobs artifacts

* Improve comments

* Restructure job configurations to avoid bean name collisions

* Restructure job configurations to avoid bean name collisions

* Use new offline deletion by job

* Test using multiple versions

* Retrieve only Pids

* Revert to master

* Maintain deferred functions synchronization as similar as possible to the way it was

* Bypass validations to allow calling an index same as a foreign key to make SchemaMigrationTest happy as H2 adds FK indexes automatically

* Adjust tests

* Add copyrights

* Add test

* Revert to delete CodeSystemVersions using deferred ITermDeferredStorageSvc

* Allow to wait for jobs tentatively

* Organize imports

* Fix tests

* Increase timeout to cover for possible additional heavy DB activity

* Add indexes to two remaining non-indexed foreign key fields to avoid much longer transactions which produce locks on heavy DB load

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_7_0/3108-terminology-large-code-system-deletion-performance.yaml

Co-authored-by: James Agnew <jamesagnew@gmail.com>

* Remove opening stars from logging messages

* Relax index name checking logic to allow for indexes on FK columns to be called same as the FKs, to make SchemaMigrationTest happy even when H2 adds indexes to FKs automatically.

* Use static array to enumerate platforms which doesn't add indexes to foreign keys automatically.

* Cleanup also job executions when performing test artifact cleanup

* Adjust to property name change

* Add ABANDONED status to the list of waited job statuses to avoid test failures due to wait timeouts

* Add also UNKNOWN status to the list of waited job statuses to avoid test failures due to wait timeouts

* Filter job executions to be stopped

* Set trace to debug test problem

* Add STOPPED status to the list of waited job statuses to avoid test failures due to wait timeouts

Co-authored-by: juan.marchionatto <juan.marchionatto@smilecdr.com>
Co-authored-by: James Agnew <jamesagnew@gmail.com>
2021-11-18 09:49:16 -05:00
IanMMarshall 843517f7ba
Avoid creating ResourcePersistentId for placeholder resources with null ID (#3158)
* Add check before mapping storage ID to resource ID in TransactionDetails.

* Add change log.

* Changed to instead prevent creation of ResourcePersistentId with null ID value.

* Changed to instead prevent ResourcePersistentId being created with null resource ID.

Co-authored-by: ianmarshall <ian@simpatico.ai>
2021-11-15 16:36:30 -05:00
TipzCM 1478abac3a
3164 updating code review points (#3165)
* 3164 updating code review points

* 3164 updating code review points

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-11-12 20:13:42 +00:00
jamesagnew 2e6fb10df3 Merge branch 'master' of github.com:hapifhir/hapi-fhir 2021-11-11 15:05:37 -05:00
michaelabuckley 7d7865ab17
Mb share jpa date tests (#3154)
* publish test-jar from hapi-fhir-storage

* Extract BaseDateSearchDaoTests up to hapi-fhir-storage.

Share a date search tests between JPA and Mongo.

* Use ITestDataBuilder for setup

* Move Mongo date cases into common set

* cleanup

* More cleanup

* naming
2021-11-10 21:07:36 +00:00
TipzCM 25c025cdee
3153 update unknowncodesystemwarningvallidationsupport to have configurable severities (#3155)
* 3153 updating the unknowncodesystemwarningvalidationsupport class

* 3153 updating version for smile

* 3153 update broken tests

* 3153 adding log

* 3153 cleanup

* 3153 cleanup

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-11-10 16:06:09 -05:00