Commit Graph

382 Commits

Author SHA1 Message Date
Ken Stevens 9753b66610
2564 contained resource validation flag (#3225)
* First cut at multithreading bundle validation

* javadoc

* change test to a red-green test

* add daoconfig

* move bundle concurrency config from validationsupport to request validator

* Moved concurrent details from ValidationOptions to FhirValidator

* Remove bundleValidationThreadCount (it's set by creating an appropriate executor)

* validate threadpool prefixes

* FIXME cleanup

* reassign FIXMEs to JB

* only create the threadpool if it will be used

* Add the MdcTaskDectorator to all threadpools

* Add bundle path to concurrently validated bundle resource validation messages.

* Add test to check bundle entry path being added to validation messages

* Threaded validation of contained resources initial checkin

* FHIR validator refactored to maintain correct bundle path and async validation task

* Refactored to use validation complete invoke callback to support raw string resource or resource object

* switch from XML to JSON

* # Conflicts:
#	hapi-fhir-base/src/main/java/ca/uhn/fhir/validation/FhirValidator.java
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Updated to use new HAPI-FHIR version

* merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation

* Updated to use new HAPI-FHIR version

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit 910b7f676b.

* Revert "merge 'validate_contains_change' into issue-3144-multi-threaded-bundle-validation"

This reverts commit c7d60fc3e6.

* Revert "Updated to use new HAPI-FHIR version"

This reverts commit aa8a7af455.

# Conflicts:
#	hapi-fhir-validation/src/test/java/org/hl7/fhir/r4/validation/FhirInstanceValidatorR4Test.java

* Fix Test dependencies after reverting changes.

* merge 'validate_contains_change' to 2564-contained-resource-validation-flag

* Disable threadpool prefix name validate check.

* merge master

* fix thread regexp

* pre-review cleanup

* change log

* fix test

* revert to XML as default validation serialization

* fix test

* fix test

Co-authored-by: Jaison B <jaisonb@gmail.com>
2021-12-08 15:18:54 -05:00
jamesagnew d481896d9b License header updates 2021-12-07 15:38:36 -05:00
jamesagnew 8aef580aed Bump core library 2021-12-07 10:57:43 -05:00
Mark Iantorno d60d07b697
Validate contains change (#3209)
* wip

* The updated changes to the validator that were pulled in from the core dependencies are not backwards compat. Need to bump HAPI version.

* pointing to stable core version

* review feedback

* Add changelog

Co-authored-by: markiantorno <miantorno@pop-os.localdomain>
2021-12-06 11:25:00 -05:00
James Agnew b7bf10ced4
Missed commits from #3214 (#3222)
* Move graphql provider

* Version bump

* Moving things around

* More reworking

* Fix
2021-12-06 06:49:52 -05:00
James Agnew d072357c2a
Move graphql provider (#3214)
* Move graphql provider

* Version bump
2021-12-05 17:22:20 -05:00
jamesagnew 1a1243db34 Missing fix from #3197 2021-11-30 16:15:09 -05:00
James Agnew aae53d6ec7
Improve loading performance (#3197)
* Synthea perf work

* Cleanup

* Work on performance

* Ongoing perf work

* More perf work

* Perf work

* Test fixes

* Updates

* Test fix

* Test fixes

* Clean up tests

* Test cleanup

* Cleanup

* Add test

* One more perf tweak

* Fixes

* Add changelog

* Add changelog

* License header update

* License header updates
2021-11-28 10:47:30 -05:00
TipzCM 3743fbc62f
test fix (#3193)
* test fix

* review fixes

* minor tweak

* little cleareR

* Update hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/match/registry/SubscriptionCanonicalizer.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-11-25 20:09:00 +00:00
Tadgh 41ef7cf961
Support multiple EID systems for MDM. (#3178)
* Add implementation, changelog, docs, and tests

* Fix potential NPE

* Use constant instead of *
2021-11-24 06:05:13 +00:00
michaelabuckley 7d7865ab17
Mb share jpa date tests (#3154)
* publish test-jar from hapi-fhir-storage

* Extract BaseDateSearchDaoTests up to hapi-fhir-storage.

Share a date search tests between JPA and Mongo.

* Use ITestDataBuilder for setup

* Move Mongo date cases into common set

* cleanup

* More cleanup

* naming
2021-11-10 21:07:36 +00:00
TipzCM 25c025cdee
3153 update unknowncodesystemwarningvallidationsupport to have configurable severities (#3155)
* 3153 updating the unknowncodesystemwarningvalidationsupport class

* 3153 updating version for smile

* 3153 update broken tests

* 3153 adding log

* 3153 cleanup

* 3153 cleanup

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
2021-11-10 16:06:09 -05:00
TipzCM 59bf8b836f
Issue 3120 support adding retry extension (#3137)
* issue-1134 initial work on adding retry handler

* issue-1134 subdscription creation fixes

* issue-1134 update the test

* issue-1134 retry policy off canonicalsubscription

* issue 1134 base subscription changes

* add failing test

* issue 1134 retry configs on base parameters

* issue 1134 passing forward params

* issue-1134 added more tests for making sure subscriptions create their channels correctly

* issue-1134 updates to get the retry to channels

* issue-1134 updating channel factory

* issue-1134 remove the dlq since it's not defineable

* issue-3120 changelog added

* issue-3120 cleaning up

* issue-3120 test fix

* issue-3120 review fixes

* issue-3120 fixed bad master merge

* issue-3120 updates for new release

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-11-03 18:21:10 -04:00
Tadgh a10c58e839
Bump version, add version to versionenum (#3135) 2021-11-02 09:40:56 -04:00
Tadgh 7c63441663
Revert "Issue 1134 retry mechanic on subscriptions (#3121)" (#3128)
This reverts commit 9b23e9e825.
2021-10-30 19:46:42 +00:00
TipzCM 9b23e9e825
Issue 1134 retry mechanic on subscriptions (#3121)
* issue-1134 initial work on adding retry handler

* issue-1134 subdscription creation fixes

* issue-1134 update the test

* issue-1134 retry policy off canonicalsubscription

* issue 1134 base subscription changes

* add failing test

* issue 1134 retry configs on base parameters

* issue 1134 passing forward params

* issue-1134 added more tests for making sure subscriptions create their channels correctly

* issue-1134 updates to get the retry to channels

* issue-1134 updating channel factory

* issue-1134 remove the dlq since it's not defineable

* issue-3120 changelog added

* issue-3120 cleaning up

* issue-3120 test fix

* issue-3120 review fixes

* issue-3120 fixed bad master merge

Co-authored-by: leif stawnyczy <leifstawnyczy@leifs-MacBook-Pro.local>
Co-authored-by: Ken Stevens <khstevens@gmail.com>
2021-10-29 11:21:43 -04:00
Tadgh 8a7b5c1b2f
Multiple improvements (#2997)
* Add failing test

* Tidy test for new tokenparam style

* Add removeByQualifier to SearchParameterMap

* Add new property binder for search terms

* Member move, add test for new parser

* whoops missed an add

* Passing test.

Introduced SearchParamTextWrapper to give a type for the binder.
Enhance FulltextSearchSvcImpl to understand TokenParams as well as StringParams.

* Add more to test, change analyzer for template

* Optimize imports

* Minor refactoring

* added eq and gt test cases

* added missing gt search param tests

* Search all CodeableConcepts

* Extract new LuceneRuntimeSearchParam to mediate indexing and searching.

* Fix up broken test with mock

* Add support for identifier type

* Bump for special maven package

* fix-me to todo

* Prevent version check

* FIx version for test

* DSTU2 does not support fluent path

* Don't accidentally strip SPs in DSTU2

* added lt tests

* Add some high-bound test cases

* More test cases and new compressed format

* More test cases

* More test cases

* Re-order columns for easier reading

* Comments

* Cleanup and comments

* Make support :text for all token SPs

* Make support :text for all token SPs

* Fixed incomplete date issue

* Set to the last millisecond 23:59:59.999

* Disabled 4 failed JVM/TZ related test cases

* add test for failing batch bug, remove bounding on queue

* remove parallelism from non-get batch operations

* Fix count for test

* New index layout

* Notes and cleanup

* Notes and cleanup

* remove the jetbrains

* Demote FIXME to WIP

* Change :text search to be lucene simple search and use standardAnalyzer.

* Change :text to require * for prefix match.

* add prefix negative test

* Bump version

* Added changelog

* Add more search parameters to lucene indexing (#3000)

* dirty the branch

* Add oversight to changelog

* Start using jpa indexing for es

* Correct negative search test

* Update templates, index tokens, write new test. TODO gotta modify SB

* Add test for code token search

* add comment

* Flesh out token search

* minor cleanup and comments

* Extract index handling from FulltextSearchSvcImpl

* D'oh.  Actually use elastic to search.

* Move supported query check closer to query builder

* String search test before activating lucene

* Add string:exact to hibernate search.

* Add string:contains search

* cleanup

* demote fixmes to allow build to run.

* Add unmodified string search.

* empty stubs for quantity and reference

* Ignore magic search params in lucene index

* Support full-text on all FHIR releases

* Include reference lookups in ES

* Fix and/or logic in _text

* Test for string with space

* Cherry-pick the NoFT fix

* Disable advanced indexing in a bunch of tests of emitted sql.

* Stub changelog

* Fix :missing flag

* Move DaoConfig up to BaseJpaTest to share teardown

* Disable new indexing for partition test

* Add a different analyzer and field for default string search vs text

* checkstyle for a pre-build

* Index full prefix for string search when using Lucene

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3000-Extend-lucene-indexing.yaml

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/HibernateSearchQueryBuilder.java

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Review feedback.

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Remove double-check

* Version bump

* Fix typo

Co-authored-by: Michael Buckley <michael.buckley@smilecdr.com>
Co-authored-by: Long Ma <longma@Longs-MacBook-Pro.local>
Co-authored-by: Frank Tao <frankjtao@gmail.com>
Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2021-10-28 15:13:16 -04:00
longma1 2067ba83ee
minor refactor to extract static string variables into a singel constants class in bulk export (#3117)
Co-authored-by: Long Ma <longma@Longs-MBP.hitronhub.home>
2021-10-28 16:04:31 +00:00
James Agnew 5f672bc9e8
Deprecate redundant BaseResourceModifiedMessage myId (#3116)
* Deprecate redundant BaseResourceModifiedMessage myId

* Add changelog

* Test fix

* Test fix
2021-10-28 07:55:46 -04:00
StevenXLi 34010fa78d
Batch job refactor (#3087)
* added AbstractResourceToFileWriter

* Added AbstractBaseBulkItemReader

* added bulkExportStatusEnum fromBatchStatus test

* Code review name change

Co-authored-by: Long Ma <longma@Longs-MBP.hitronhub.home>
Co-authored-by: Steven Li <steven@smilecdr.com>
2021-10-19 09:07:05 -04:00
JasonRoberts-smile b5e31abf9b
add authorization check to inline matches (#3048)
* add authorization check to inline matches

* code review feedback: cached results case

* Add query count test

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_6_0/3047-inline-match-security.yaml

Accept code review suggestion

Co-authored-by: James Agnew <jamesagnew@gmail.com>

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2021-10-18 12:21:26 +00:00
Tadgh b427135b04
Patient/$everything _id parameter (#3077)
* Add tests. Add implementation in R4. Add changelog

* Wip partial fixup of other STU versions

* Updated usages of new patient type everything api

* Partial commit of incomplete documentation

* Remove docs

* Cut to token instead of string

* Bump hapi version due to breaking api changes
2021-10-17 14:34:33 +00:00
James Agnew 310a4cdb58
Migrate CLOB columns to LOB (SMILE-3072) (#3016)
* Migration finished

* Work on LOBS

* Add to ClobMigrated annotation

* Deal with lobs

* Work on new approach

* HAPI FHIR version bump

* Add changelog

* Add license header

* Fix intermittent

* CLeanup

* Cleanup
2021-10-11 16:47:37 -04:00
James Agnew 756b0fdc7b
Fix transaction scoping (#2799)
* Work on transaction boundaries

* Test fixes

* Test fixes

* Test fix

* Resolve FIXME

* Test fix

* Test fix

* Test fix

* Test fixes

* Test fix

* Adjust changelog

* Remove unneeded changelog

* Test fix

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Fixes

* Test fix
2021-10-11 16:45:22 -04:00
Tadgh bc2cd7cafc
premature optimization is the root of all evil (#3053) 2021-10-06 14:44:39 +00:00
Tadgh 59afbe086c
Allow canonical references to support treating given base URLs as local. (#3050)
* Add failling test, works on #2843

* Partial hack to get this working

* Move implementation into pre-processing

* Add changelog

* Move implementation

* Minify Test

* Remove fixmes

* Tidying

* Ensure this only operates on STU versions which support canonical
2021-10-05 11:09:49 -04:00
James Agnew d320b78ae1
Improve synthea ingest performance (#3033)
* Improve synthea ingest performance

* Tweaks

* Fix param splitting

* Update to test

* Add changelog

* Test fix

* Add some runtime checks

* Remove useless test

* Test fix
2021-10-04 05:46:06 -04:00
jamesagnew d4f01ac4e7 License header updates 2021-10-03 20:20:04 -04:00
Tadgh 040d09cfa6 Remvoe error in moving code over 2021-09-28 16:43:29 -04:00
Tadgh 7dfe628551 remove parallelism from non-get batch operations 2021-09-27 15:32:03 -04:00
Ken Stevens 0db40c6118 support non-jpa batch 2021-09-24 00:39:15 -04:00
Ken Stevens 53c7476dae
hapi storage (#3024)
* first pass moving core storage classes to storage-api

* move term service apis

* nearly done

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* create new SearchConstants class to store platform independent search constants

* move a couple of subscription services

* move transaction processor adapter to storage

* version bump

* move searchparam

* fix test

* fix compile includes

* fix text

* fix docs
2021-09-23 22:04:46 -04:00