Commit Graph

389 Commits

Author SHA1 Message Date
James Agnew d320b78ae1
Improve synthea ingest performance (#3033)
* Improve synthea ingest performance

* Tweaks

* Fix param splitting

* Update to test

* Add changelog

* Test fix

* Add some runtime checks

* Remove useless test

* Test fix
2021-10-04 05:46:06 -04:00
James Agnew eca725afa4
Improve versioned URL support for ValueSet expansion (#2994)
* Add test

* Test fixes

* Work on expansion errors

* Fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fixes

* Work on fix

* Test fixes

* Test fixes

* Test fix

* Cleanup

* Test fixes

* Remove accidental commit

* Test fixes

* Version bump

* Add changelog

* Fixes

* Test fix

* Fix conflicts
2021-10-03 19:47:23 -04:00
Ken Stevens b1dd40fdba
Try commenting out clearAllStaticFieldsForUnitTest to see what happens (#3034)
* Try commenting out clearAllStaticFieldsForUnitTest to see what happens

* Rename clearAllStaticFieldsForUnitTest to randomizeLocaleAndTimezone and only do that
2021-09-24 16:01:28 -04:00
Ken Stevens 53c7476dae
hapi storage (#3024)
* first pass moving core storage classes to storage-api

* move term service apis

* nearly done

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* rename hapi-fhir-storage-api to hapi-fhir-storage and move transaction processor

* create new SearchConstants class to store platform independent search constants

* move a couple of subscription services

* move transaction processor adapter to storage

* version bump

* move searchparam

* fix test

* fix compile includes

* fix text

* fix docs
2021-09-23 22:04:46 -04:00
Ken Stevens 060791aeb4
hapi-fhir-storage-api (#3006)
* rename hapi-fhir-jpaserver-migrate to hapi-fhir-sql-migrate and move hapi-tasks into jpaserver-persistence

* rename hapi-fhir-jpaserver-api to hapi-fhir-storage-api

* move bulk import apis

* create hapi-fhir-jpa

* create hapi-fhir-jpa

* move CircularQueueCaptureQueriesListener to japi-fhir-jpa

* move mdm logs to storage-api

* move gziputil to storage-api

* move quartz scheduling to hapi-fhir-jpa

* move default subscription channel factory to storage-api

* consolidated batch constants

* correct accidental Logs change

* remove dependency of cdr-api on cdr-persistence

* fix javadoc

* pom descriptions

* review feedback
2021-09-21 15:36:08 +00:00
Tadgh 9d982c5e08
Merge branch 'master' into conditional-bug 2021-09-14 08:51:58 -04:00
Tadgh 3ccadd636e
Merge branch 'master' into 2958-consistent-error-handling 2021-09-13 19:38:57 -04:00
Tadgh 5f86370c98 Merge branch 'master' into conditional-bug 2021-09-13 13:04:10 -04:00
Tadgh 59c8765e6a Add implementation to validate conditional URLs post-transaction 2021-09-13 13:03:44 -04:00
Ken Stevens eee7d7e455
code cleanup (#2986)
* failed attempt to get it to work

* Use simplified FhirContext.forCached methods

* Update hapi-fhir-base/src/main/java/ca/uhn/fhir/context/FhirContext.java

Co-authored-by: James Agnew <jamesagnew@gmail.com>

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2021-09-13 03:09:18 +00:00
James Agnew 5554431146
Avoid dead SearchParameter links in CapabilityStatement (#2790)
* Avoid dead SearchParameter links in CapabilityStatement

* Resolve fixmes

* Build fix

* Test fix

* Test fix

* Test fix

* Work on metadata

* Fixes

* Test fixes

* Test fix

* Test fix

* Test fixes

* Test fix

* Work on test fixes

* Test fixes

* Test fixes

* Test fixes

* Test fix

* Fix params

* Resolve fixme

* Adjust changelogs

* Version bump to 5.6.0-PRE7

* Add license header

* Bump version to 5.6.0-PRE5-SNAPSHOT
2021-09-11 18:27:18 -04:00
jdar8 a02a9e0254
Merge branch 'master' into 2958-consistent-error-handling 2021-09-10 17:02:31 -07:00
Justin Dar 77845d66f7 Improved code formatting 2021-09-10 10:54:33 -07:00
Justin Dar 836948010f Fixes issue (#2958) where Procedure?patient caused inconsistent results and ensured that such requests gave http 400 2021-09-09 16:10:09 -07:00
Ken Stevens 661ca02a6a revert clobbered code 2021-09-09 09:28:01 -04:00
Ken Stevens 5bc571d97d fix test 2021-09-09 01:00:12 -04:00
Ken Stevens 67c8216d73 cleanup 2021-09-08 20:17:14 -04:00
Ken Stevens 81abe26cda improve api 2021-09-08 20:08:06 -04:00
Ken Stevens 519244a2ea unit test passes 2021-09-08 19:14:50 -04:00
Tadgh 3ec47cb0ba Bump version as we break the localcontainerentitymanagerfactory bean constructors args 2021-09-07 13:58:22 -04:00
Ken Stevens 346e29920a
Issue 2927 convert mdm clear to spring batch (#2929)
* initial roughout of mdm clear batch job

* still roughing out classes

* finished first draft of reader

* most tests passing now

* all tests pass.  now FIXMEs

* FIXMEs done.  Time for regression.

* fix test

* changelog and docs

* fix test

* pre-review cleanup

* version bump

* move spring autowire deps for cdr

* move spring autowire deps for cdr

* finally got beans working phew

* rearrange method calls so persistence module can perform clear operation on its own

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_mdm/mdm_operations.md

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* review feedback

* review feedback

* Remove inheritance

* fix beans

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
Co-authored-by: Tadgh <garygrantgraham@gmail.com>
2021-09-03 14:58:19 -04:00
Tadgh 37ad73a422 Bump PRE version so we don't conflict downstream 2021-08-23 15:42:18 -04:00
Tadgh 1acaf325c5 Merge branch 'master' into rel_5_5_0 2021-08-23 14:50:29 -04:00
Tadgh cf917d0f82 Version Bump 2021-08-15 16:54:38 -04:00
Tadgh 3fd5f7d6d4 Omit if set to false 2021-08-12 19:12:09 -04:00
Tadgh 669cf05b30 Modify SearchParameterMap toNormalizedQueryString to include value of _contained parameter 2021-08-12 14:25:12 -04:00
Tadgh 6f3006f2ba Fix NPE 2021-08-10 20:56:54 -04:00
Tadgh daa3bc773e remove dummy test 2021-08-10 16:46:03 -04:00
Tadgh fe98162616 Add changelog 2021-08-10 14:59:03 -04:00
Tadgh 4fadc62066 Bump Hapi Version 2021-08-06 13:44:04 -04:00
Ken Stevens f5dfc3a5ed
New reindex spring batch operation (#2845)
* first sweep to add new $reindex job

* first cut done.  next up integration test.

* start roughing out reindex everything

* Failing test

* test passes

* add BundleUtil helper method

* rough in processEverything

* rough in processEverything

* fix mock test

* merge master

* Make BaseJpaTest.mySrd lenient.

* fix test

* reindex everything works with FIXMEs

* reindex everything more tests

* moar tests

* moar tests

* fix test

* fix test

* fix regression and fix test

* fix test cycle

* fixme

* fix test

* test provider layer

* reindex everything partition aware

* yay last fixme!

* run reindex in partitioned threads

* add fixmes for areas that require extra testing

* added transaction and log message

* manual integration testing

* changelog

* reindex all partitions

* bump hapi version

* fix test

* moar test

* pre-review cleanup

* fix javadoc

* review feedback

* review feedback

* review feedback

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_5_0/2845-reindex-operation.yaml

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* review feedback

* review feedback

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/delete/job/ReindexJobTest.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/delete/job/ReindexJobTest.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantBatchOperationR4Test.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantBatchOperationR4Test.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/ResourceSearch.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

* Update hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/api/server/storage/IDeleteExpungeJobSubmitter.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
2021-07-29 19:18:49 -04:00
jamesagnew a5e6674e84 Version bump to 5.5.0-PRE7-SNAPSHOT 2021-07-22 08:41:16 -04:00
James Agnew 6d37749be8
Add non unique combo search params (#2809)
* Start work on nonunique combo search params

* Work on SQL

* Version bump

* A fix

* Test fixes

* Fixes

* Undo version bump

* Test fixes

* Resolve fixme
2021-07-21 19:14:14 -04:00
James Agnew 6af022062f
Fix issue with partitioning in patient ID compartment mode (#2810)
* Fix issue with partitioning in patient ID compartment mode

* Add changelog
2021-07-21 17:21:23 -04:00
James Agnew d762a07817
Partition patient interceptor (#2766)
* Start work on interceptor

* Implemented

* Add interceptor and docs

* Add docs

* Compile fix

* Test fixes

* Test cleanup

* Test fixes

* Test fixes

* Improve error message

* Add broken tests

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_5_0/2766-add-patient-id-compartment-interceptor.yaml

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/interceptor/PatientIdPartitionInterceptor.java

Co-authored-by: Tadgh <tadgh@cs.toronto.edu>

* Address review comments

Co-authored-by: Tadgh <garygrantgraham@gmail.com>
Co-authored-by: Tadgh <tadgh@cs.toronto.edu>
2021-07-05 09:15:20 -04:00
Tadgh f66f9b992e Bump pre-version as this will break downstream implementations 2021-06-30 15:37:19 -04:00
James Agnew 7185089c9d
Add forced offset search JPA interceptor (#2756)
* Add support for offset searches

* Tests working

* Add changelog

* Test fixes

* Test fix

* More test fixes

* Add commit counter
2021-06-25 10:40:39 -04:00
Ken Stevens 0b05c730fb
don't create a channel if subscriptions are disabled. (#2733)
* don't create a channel if subscriptions are disabled.

* review feedback

* fix test

* fix intermittent

* fix intermittent
2021-06-19 00:00:16 -04:00
James Agnew 8b205b23d0
Fix history paging (#2741)
* Fix history paging

* Add changelog

* Add license header
2021-06-18 17:47:51 -04:00
Ken Stevens 134631fdee
Convert delete expunge to use Spring Batch (#2697)
* prepare to add $delete-expunge operation that will create a spring batch job

* Add operation

* Wire up jpa provider.  Begin with failing test.

* Copy/paste bulk import job as a starting point.
FIXME with proposed design

* delete expunge job parameter validation with test

* implemented reader
stubbed processor, writer

* wip for master merge

* started implementing reader

* started implementing reader

* working with stubs

* happy path batch delete expunge is done

* Provider done but test not passing.  Guessing batch infrastructure not running in that test.

* IT test works now

* add reader test

* Converted delete _expunge=true to use new batch job

* DeleteExpungeDaoTest passes

* Fix test

* Change batch size to integer

* rename search count to batch size

* Make delete expunge partition aware

* updated docs

* pre-review cleanup

* change log

* add partition id to SystemRequestDetails

* Make RequestPartitionId serializable

* Change delete expunge provider to use partition id instead of tenant name

* fix tests

* test pointcut gets called

* assert on pointcut calls

* Add resource type to STORAGE_PARTITION_SELECTED pointcut

* bump hapi-fhir version
move expunge provider parameters from JpaConstants to ProviderConstants

* bump hapi-fhir version

* copyrights

* restore deleteexpungeservice for mdm

* restore deleteexpungeservice for mdm

* fix test

* public constants

* convert instant to date

* Moved expunge constants to ProviderConstants

* final review

* disabling InMemoryResourceMatcherR5Test.testNowNextMinute() to see if I can get a clean test run

* fix tests

* fix tests

* fix tests

* fix tests

* review feedback

* review feedback

* review feedback

* review feedback

* review feedback

* review feedback

* improve logging

* bump version

* version bump

* recovering from failed merge

* unzip RequestListJson per Gary's suggestion.  I didn't want to do it at first, but as usual Gary was right.

* fix serialization
2021-06-15 10:36:05 -04:00
jamesagnew 376a84d213 Version bump 2021-06-14 13:12:34 -04:00
James Agnew dc627dc019
Add bulk export authorization layer (#2712)
* Add bulk export authorization layer

* Add docs

* Version bump

* Authorize any

* Add bulk export - all

* Address lgtm issue
2021-06-06 20:47:43 -04:00
James Agnew a2950324ef
Avoid SQL based transaction dupe check (#2688)
* Avoid SQL based transaction dupe check

* Add changelog

* Test fixes

* Test fixes
2021-05-30 20:26:36 -04:00
James Agnew 24b3f0f30d
Enable mass ingestion mode (#2681)
* Work on fixes

* Work on counts

* Enable mass ingestion mode

* Add changelog

* Test fix

* Test fix

* Test fixes

* Fixes

* Test fix

* Test fix
2021-05-27 18:43:51 -04:00
jamesagnew d70bbad6f1 Expand access 2021-05-27 12:15:37 -04:00
James Agnew c8c596aff4
Fix date search on period with no end (#2676)
* Add test

* Working test

* Update test

* Account for date fixes

* Add changelog

* Add setting

* Test fixes

* Test fix
2021-05-26 21:03:53 -04:00
James Agnew 7a83c76e65
Add validation suppression mode (#2675)
* Add validation suppression mode

* Add changelog

* Compile fix
2021-05-26 17:41:48 -04:00
James Agnew 2c722e64c2
Add support for including qualified star syntax (#2672)
* Add support for including qualified star syntax

* Add changelog

* Add docs

* Build fix
2021-05-23 16:50:37 -04:00
jamesagnew 2068372998 Version bump 2021-05-07 17:43:54 -04:00
James Agnew 970a9884f0
Avoid tag definition constraint (#2620)
* Avoid tag definition constraint failure

* Version bumps

* Add changelog

* Avoid guava issue

* Test fix

* Test fix

* Bump pom
2021-05-01 17:16:15 -04:00
James Agnew 95d7ba6fa8
Streamline Search SQLs (#2576)
* Streamline SQL queries

* SQL query tidying

* Add changelog

* Test fixes

* Tests passing

* Ongoing work

* Test fixes

* Test fixes

* Build fix

* Cleanup

* Fixes
2021-04-28 10:29:09 -04:00
James Agnew 2ba1005762
OpenAPI Support (#2560)
* Start work on OpenAPI

* Fixes

* Work on OpenAPI

* Cleanup

* Cleanup

* More swagger work

* Build fix

* More work

* More work

* Add documentation

* Docs fixes

* Add changelog

* License updates

* Add API

* Cleanup

* Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_plain/openapi.md

Co-authored-by: patrick-vachon-smilecdr <81274188+patrick-vachon-smilecdr@users.noreply.github.com>

* Work on scripts

* Add docs

* Compile fix

* Work on fixes

* Fix tests

* Test fix

* Test fix

* Build fix

* Tests

* Build fix

* Work on pipeline

* Version bump

* Test fix

* Version bump

* Test fix attempts

* Test fix

* Test fix

* Remove accidentally committed files

* Fixes

* Tets fixes

* Test fix

* Test fix

* Test fix

* Test fixes

* Test fixes

* Test fixes

* License header updates

* test fix

* Test fixes

* Test fix

* Test fixes

* Test fix

* Checkstyle bump

Co-authored-by: patrick-vachon-smilecdr <81274188+patrick-vachon-smilecdr@users.noreply.github.com>
2021-04-25 15:40:50 -04:00
Nick Goupinets 4e266462ec Bumped up version due to breaking changes 2021-04-20 16:17:27 -04:00
James Agnew ca2088f3ad
Add framework for Bulk Import (#2538)
* Start work on bul;k import

* Work on bulk import

* Have batch working

* Working

* Working

* More work

* More work on bulk export

* Address fixmes

* License header updates

* Test fixes

* License header updates

* Test fix

* Test fix

* Version bumps

* Work on config

* Test cleanup

* One more version bump

* Version bump

* CLeanup

* A few additions

* Test fixes

* Test fix

* Test fix

* Migration fix

* Test fix

* Test fix
2021-04-14 17:41:32 -04:00
James Agnew a6cbf7eebd
Add support for hierarchycal valueset expansion (#2525)
* Add support for hierarchycal valueset expansion

* Add changelog

* Add tests

* Cleanup

* Test fix

* Test fixes
2021-04-06 17:36:54 -04:00
James Agnew b617c7690d
Search Parameter Infrastructure Cleanup (#2522)
* Search param service cleanup

* Work on collapsing search param

* Search param cleanup

* Work on build

* Test fix

* Test fixes

* Ongoing work

* Test fix

* Compile fix

* Test fixes

* Test fix

* Test fix

* License header updates

* Remove fixme

* Cleanup

* Cleanup
2021-04-05 17:40:10 -04:00
James Agnew 0bf746f23b
Add support for revincludes to CapabilityStatement generator (#2521)
* Add support for revincludes to CapabilotyStatement generator

* Add changelog

* Fixes

* Test fix

* Test fixes

* Test fix

* Test fix

* More fixes to CS cache

* Test fix

* Test fix

* Test fix
2021-04-05 08:29:04 -04:00
James Agnew 049097e5b4
Improve CapabilityStatement Generation (#2511)
* Start work

* Work on testing

* Version bump

* Test fix

* Compile fix

* Test Fixes

* Test fix

* Test fix

* Compile fix

* Tst fix

* Test fix

* Test fix

* Test fix
2021-03-30 05:48:03 -04:00
Diederik Muylwyk b720c32c1d Version bump 2021-03-29 18:23:16 -04:00
James Agnew dc41d5ad61
Add terminology mapping and display population interceptors (#2488)
* Start work on code mapping interceptors

* Work on generation

* Add terminology mapping and display population interceptors

* Add changelog

* License header updates

* Test fixes

* Fix typo

* Interceptor fixes

* License header update

* Test fixes

* Mapping fixes

* Version bump

* License header updates

* Add null tests

* Add null guard

* Intermittent fix
2021-03-21 12:03:57 -04:00
James Agnew d857048207
Preserve meta values in stored resources (#2481)
* Fix #1731

* Test fix

* Test fixes

* Avoid intermittent failure
2021-03-17 09:23:22 -04:00
James Agnew 807d9d53f0
Improve performance for large transactions (#2462)
* Performance tweaks

* Work on processing speed

* Disable no longer relevant tests

* Work on performance

* Work on tests

* Work on large transaction performance

* Add changelog

* Perf tweaks

* Test fixes

* Resolve FIXME

* Fixes

* Test cleanup

* Fix broken changelog entry
2021-03-09 18:16:02 -05:00
James Agnew be50a46d76
Tweaks to contained searches (#2461)
* Tweaks to contained searches

* Add changelog

* Build fix
2021-03-09 11:51:30 -05:00
Frank Tao bf8e890a0c
Supported contained resource search (#2441)
* POC for indexing on contained resource - test case may failed

* Test contained url

* Add a spt to handle contained flag

* Added search option for contained resource

* Impl contained resource search

* fixed typo

* Reworked on creating index based on the review comments

* Added changelog

* Added more test cases

Co-authored-by: jamesagnew <jamesagnew@gmail.com>
2021-03-09 06:50:23 -05:00
Diederik Muylwyk 6cc8a5ce4d
Fix HasParam#doGetQueryParameterQualifier() (#2460)
* Fix HasParam#doGetQueryParameterQualifier(); add changelog entry

* Fix bad test in SearchParameterMapTest
2021-03-09 06:28:40 -05:00
Ken Stevens 7d769e3b05
Issue 2332 create via put for implementationguide that references some placeholders (#2423)
* begin with failing test

* fixed.  test passes.

* documentation and changelog

* added ConceptMap

* review feedback
2021-02-26 09:44:48 -05:00
Kevin Hartmann 26a385a595 Merge remote-tracking branch 'origin/master' into kh-20200222-search-list 2021-02-24 10:29:38 -05:00
Kevin Hartmann 1eab2caf9f Added _list as a valid parameter in MatchUrlService
Wrote tests for and/or/both processing in FhirResourceDaoSearchListTest.
2021-02-24 10:27:22 -05:00
Patrick Werner cf1f2f27db
added nullchecks to prevent NPE (#2417)
* added nullchecks to prevent NPE
fixes #2171

* Add test from #2171 and changelog

Co-authored-by: jamesagnew <jamesagnew@gmail.com>
2021-02-24 10:09:04 -05:00
Ken Stevens 0e314a9382
fix searchparam not found failure at startup (#2408)
* reproduced and fixed issue reported by Adi

* undoing rearrange
2021-02-22 15:09:59 -05:00
James Agnew 3eda19c65a
Version bump (#2400)
* Version bump

* Docs changes

* Docs tweaks

* Docs tweak

* Spelling fixes

* Docs tweaks
2021-02-18 11:07:49 -05:00
jamesagnew 919c1dbddc Version bump to 5.3.0 2021-02-17 07:10:13 -05:00
Jari Maijenburg 0ec64cfab1
Fixed bug where AND resource search-params were matched as OR params (#2365)
* Fixed bug where AND resource search-params were matched as OR params

* Added InMemoryResourceMatcher test to test searching in a period
2021-02-08 10:23:07 -05:00
Mike G f61cb163ae refactor and modify for prefix and suffix 2021-02-01 12:11:11 -05:00
James Agnew 909d9f742c
Enable Auto-reference versions in JPA (#2323)
* Automatic reference versions

* Auto reference versions

* resolve fixmes

* Add changelog

* Test fixes

* Test fixes

* Resolve fixmes

* Work on versioning

* Add support for _includes

* Update headers

* Change visibility

* One more tweak

* One more API tweak

* Fixes

* Account for review comments

* Fix errorprone reported issue

* Test fix

* Test fixes

* Test fixes
2021-01-27 14:49:10 -05:00
Ken Stevens 749934d5f1
fixed name text indexing (#2309)
* fixed name text indexing

* fixed name text indexing
2021-01-22 15:39:26 -05:00
jamesagnew 4913c33fdf Test fix 2021-01-17 07:24:00 -05:00
James Agnew 1dc4fb2206
Make quantity indexes use double datatype (#2290)
* Make quantity indexes use double datatype

* Add changelog

* Test fixes

* Test fixes

* Disable test
2021-01-16 18:44:15 -05:00
Frank Tao f6d4a40388
UCUM Service Support (#2261)
* Added UcumServiceUtil

* Removed ucum-essense.xml

* Added ucum service existing test cases passed

* Added more test cases

* Merge branch 'master' into ft-ucum-support

# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.

* Added back extractReferenceParamsAsQueryTokens

* Added changelog and migration script

* Added length for the string type

* Moved UCUM util to hapi-fhir-jpaserver-model

* Renamed UCUM support to Normalized Quantity Search Support

* Moved setNormalizedQuantitySearchNotSupported to AfterEach

* Changed migrate task to 530

* Removed comments

* Fixed in memory search issue and added more test cases

* Fixed the version order

* License header updates

* Updated based on review comments

* Updated the migration task based on review comments

* Fixed RES_TYPE with type String and length 100

* Changed myParamsQuantityNormalizedPopulated type to Boolean class
2021-01-10 21:05:24 -05:00
jamesagnew 12d366c86e License header updates 2021-01-05 18:13:14 -05:00
Tadgh efc6809ea6
Replacing Hibernate Search 5 with Hibernate Search 6 (#2190)
* Beginning work on converting to hibernate search 6
* replace all out of date properties
* replace all POM references to old versions
* remove the shaded ES jars as HS6 supports 7.9
* Convert "Bridges" to RoutingBinder and RoutingBridge
* Modernize all indexing annotations
* Begin refactoring of search queries (wip)

* Fix spatial API changes, work on BaseTermReadSvcImpl

* Most of the way through the various filter property conversions

* Finished i think with BaseTermReadSvcImpl

* Compiling, but definitely broken

* Start test compilation failures

* All tests compiling (i think)

* remove suggest keywords operation

* Fix bootstrap errors, still have to deal with transient fields

* Rollback CR

* More refactoring, got IDs baked back into docs

* Fix coord util, fix a few more fields which require projection

* merge fixes

* begin refactor of valuesetexpansion tests

* fix another test

* Update tests

* remove todos, add TestContainers

* Fix descendant in

* Add testcontainers, fix another test

* fix more tests, replace embedded es

* Rip out embedded elastic, replace with TestContainer

* merge conflicts for new analyzers

* Bump to CR2 for hibernate search 6. Add forgotten CLI dep. Add backwards compatible codecs

* Remove dead deps, update testcontainer

* Remove java-hamcrest

* Pull asserts up, refactor tests to pass in RP tests

* Update V2 check

* Remove suggest keyword tests

* Split line for testing

* Fix perhaps one of the dumbest programming mistakes in my career so far

* Refactor tests which no longer rely on a strict known order

* Fix up test config, disable log test temporarily

* Remove log4j from being transitively pulled in. re-enable test

* Update log4j exclusions with new bridge. Update test to check for core class

* Move dependency to root pom in dep management section

* Update changelog, add changelog for HS6 with ES requirements. Remove dead comment

* update testst, remove todos

* Add hibernate props provider method, add elastic regexp query

* Add todo for high level client builder

* Modify ElasticsearchRestClientFactory to support HTTPS hosts

* Remove protocol extraction from rest url

* do we even need to index this...?

* remove purge of non-indexed resources

* Add longer timeout to testcontainers

* Add task to add docker CLI to azure pipeline via task, for testcontainers

* WIP remove this

* Dont update version

* Add test for duplicate termconcepts

* Use real hibernate ORM methods

* merge issues

* Add partitionsettigns bean to match master

* Trying to debug testcontainers on azure...

* Trying to debug testcontainers on azure...

* Run even on previous failure

* Fix testcontainer port binds

* Add full text logs attaching, and test reporting

* I love yaml

* Remove comment

* Refactor delta remove to not do a once-over traversal and flatten all children pre-delete. Resolves transaction commit boundary issue

* Modify TX beheaviour

* wip

* Rework to use config and mocked beans for partition-aware lastN

* rework to not use beans and just inject vars i need for test, keps test envs cleaner

* update azure pipeline to only copy test results

* Remove erroneous publish

* Always get test logs

* Make it so indexing works,

* Add todos

* revert deleteByPid

* Work on test fixes

* Test fixes

* Test fixes

* Another test fix

* Test fix

* Work on tests

* Test fixes

* All tests passing locally

* Fix test failure

* FIx build error

* Rename usages of HibernateDialectProvider -> HibernatePropertiesProvider

* Update ngram of token filters

* Tidying

* subvert the purpose of hibernate elastic props builder to inject template

* dead space

* Rename changed class

Co-authored-by: jamesagnew <jamesagnew@gmail.com>
2021-01-05 17:56:59 -05:00
jamesagnew d20b07c78e Update license headers 2021-01-01 15:50:24 -05:00
jamesagnew f30999b053 License header updates 2021-01-01 15:07:23 -05:00
James Agnew e4d6c51290
Allow plus symbol in email address match url (#2224)
* Allow plus symbol in email address match url

* Add changelog
2020-12-09 21:52:03 -05:00
James Agnew 30b6244988
Avoid blocking on shutdown (#2206) 2020-12-02 08:44:06 -05:00
jamesagnew 74aed5d107 License headers 2020-11-29 20:53:19 -05:00
Ken Stevens 3d3242cf9a
Resource change listener (#2191)
* init rev

* fix build

* Tweaked the POM config settings and also added some dependency exclusions (commented out for now).

* More re-factoring of the CQL Unit Tests.

* Removed a LogMessages.html file and a minor Unit Test change.

* Unit Tests.

* added debug logging to troubleshoot the dao that has no name

* added debug logging to troubleshoot the dao that has no name

* workaround to get past null dao resourceName issue

* fix jsons to get test to pass.  Test still fails with library id problem

* gitignore

* gitignore

* test passes!  Woohoo!

* undo troubleshooting logging

* added timer and logging.

* added asserts and time multiple evaluations and measure the average

* readme

* adding explanations

* added more explanatory notes

* measure 2 patients

* move pom to use cqf snapshot

* roughed out cache

* roughed out cache

* Added code to VersionChangeCache class.

* added tests

* added polling test

* wrote init version

* wrote init version

* optimized versioncache

* worked on getting tests to pass

* redesigned interfaces

* all tests pass

* fixmes

* fixmes

* rename param

* Added Unit Tests.

* javadoc

* Fixed the 2-Patient Unit Test.

* More Unit Test work.

* make ResourceVersionMap immutable

* Fixed a Unit Test that was failing intermittently by adding a new way to refresh the cache.

* Use a new method called doRefreshAllCaches(0) to force a refresh and have all Listeners called immediately.

* Cleaned up IVersionChangeListenerRegistry interface to make methods more clear and resolved all Unit Tests.

* disabled tests

* disabled tests

* removed unused test method

* fixed refresh logic and added asserts

* moved cache so it can be used by searchparamregistry

* Updated the Cql Unit Tests to be properly configured for Dstu3 or R4.

* started rewriting SearchParamRegistryImpl to use new cache
added init method to listener interface

* added fixmes

* adding tests

* tests pass

* added tests

* Fixed the way CqlProviderFactory Autowires Beans so it can work with both Dstu3 and R4 contexts.

* moar tests

* fix test

* work tests

* reverting unneccessary refactors

* undo unneccessary import changes to reduce MR size

* undo unneccessary import changes to reduce MR size

* Unit Test fixes...more to come...

* add unregister

* fix tests

* Changed ResourceVersionCache to use a Map of Maps.

* searchparam test

* test passes

* resolved fixme

* fixmies

* strengthen test asserts

* More Unit Test changes and added some FIMXME items.

* changed from long to changeresult

* renamed VersionChange -> ResourceChange

* fixed delete bug

* organize imports

* fix test

* add update test

* add test reset function

* fix stack overflow

* fix startup race condition (might still be intermittent)

* found the problem.  delete doesn't work because we can't look up the deleted resource to find out what its name is

* fixed regression

* abandoned idea of incrementally updating searchparam registry.  Rebuilding every time--it doesn't change that often.

* fix test

* begin with failing test

* test passes

* fixmes and javadoc

* fix test

* fixme

* fix test

* whack-a-mole.  Either subs pass or cql passes.  Something's fishy with the FhirContext

* fix subscription test initialization

* fix method name

* Re-factored the CqlProvider Unit Tests.

* changed ResourceChange API

* add interface

* add interface

* fix test

* add schedule test

* add doc

* init rev

* FIXME

* modify FhirContext change

* change fhirContext.getResourceTypes to lazy load

* converted subscriptions

* converted subscriptions

* begin with failing test

* test passes

* fix test

* test coverage

* test coverage

* test coverage

* test coverage

* good coverage now

* pre-review cleanup.  I think I found a bug.

* moved cache into listener entry
tests pass with fixmes

* fix test

* fix test

* fix test

* fixme

* FIXMEs

* merge cache and registry

* method reorg

* javadoc

* javadoc done.  all FIXMEs resolved.

* change log

* changes needed by cdr

* spring config cleanup

* james feedback

* james feedback

* might not work. Try moving resourcechangeconfig into searchparam config

* merge ResourceChangeListenerRegistryConfig.java into SearchParamConfig

* fix test

* fix SubscriptionLoader

* fix SubscriptionLoader

* create ResourceVersionMap from resources

* added cache handle interface

* fix test

* javadoc

* fix test

* fix test

* James feedback: clone searchparametermap

* fix startup

* fix test

* fix test

* fix intermittent

* pre-review cleanup

* FIXME

* final FIXME yay!

* Address a couple of my own reviw comments

Co-authored-by: Kevin Dougan <kevin.dougan@smilecdr.com>
Co-authored-by: jamesagnew <jamesagnew@gmail.com>
2020-11-29 19:42:40 -05:00
jamesagnew 0c31741eec Version bump to 5.3.0-SNAPSHOT 2020-11-18 14:52:43 -05:00
jamesagnew 4d441ad7f2 Version bump to 5.2.0 2020-11-18 06:30:20 -05:00
James Agnew 228e75b501
Bump core to latest versions (#2154)
* Bump core to latest versions

* Compile fix

* Bump to non snapshot core library
2020-11-03 15:43:37 -05:00
James Agnew 5e8ec8d6f3
Resolve crash in InMemory SP resolver for empty source (#2150)
* Resolve crash in InMemory SP resolver for empty source

* Add changelog
2020-10-28 14:30:32 -04:00
James Agnew 8000d2d0cf
SQL Join Rework (#2086)
* switched to adding annotations.  Just did a test with Token, but the sql looks clean.

* switched to adding annotations.  Just did a test with Token, but the sql looks clean.

* added the other six index links

* trying different annotations

* Start testing

* Update resources on package install

* Add changelog

* Join rework

* CLean up SQL builder

* Add docs

* SP rework

* Join work

* Work on params

* Work on refactor

* Work on chains

* Work on joins

* Rework queries

* Work on queries

* Many more tests passing

* Refs test

* Work on sorting

* Work on tests

* More joins work

* Work on tests

* Work on queries

* Tests passing

* More test fixes

* Test fixes

* Work on SQL

* Tests passing

* Add some tests

* Add some tests

* License headers

* Use entity manager to get datasourcd

* One more fix

* Model cleanup

* Ongoing work

* Fixes

* Fixes

* Work on joins

* Ongoing fixes

* Merge conflict

* Cleanup

* clean up unused fields

* Work on join

* COmpile fix

* Rework querying

* Update hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/HibernateDialectProvider.java

Co-authored-by: Ken Stevens <khstevens@gmail.com>

* Address review comments

* Resolve fixmes

* Test fix

* Test fixes

* Test fix

Co-authored-by: Ken Stevens <khstevens@gmail.com>
2020-10-26 05:24:26 -04:00
Ken Stevens 3b91873b7c
delete expunge (#2131)
Added delete _expunge=true
2020-10-20 17:21:50 -04:00
Tuomo Ala-Vannesluoma 1435540320
Improve synchronous searching by providing offset & limit support (#2059)
* Improve synchronous searching by providing offset & limit support

Add support for offset querying which leverages paging at the query level
Add configuration for search default page size and search maximum page size
If using offset, always use synchronous searching to avoid extra database insert/update etc.
When using offset, only calculate count if it's wanted
Validate params closer to provider and handle size if search returned "all" (or last)

* Review and test fixes

Comment _offset as nonstandard parameter
Make synchronous search always count the total value (for now)
Fix issue with FulltextSearchSvcImpl mutating param map
Dirty fix for BaseJpaTest (not sure how to fix the including resources issue)

* Remove temporary count querying fix for synchronous loads

* Fix offsetting of everything operations, do not drop zero offset

* Fix jpa test default and maximum page size, add some fixmes to tests before resolved

* Ignore one failing test, fix others

* Fix Dereferenced variable may be null

* Fix everything paging in R4 by adding DISTINCT if synchronous load is used

Also fix assertion of size when hitting fetchSizeDefaultMaximum

* Add documentation about offset annotation and paging

Co-authored-by: James Agnew <jamesagnew@gmail.com>
2020-10-12 17:42:03 -04:00
James Agnew 2134d40f1c
Account for timezones in lastUpdated search (#2113)
* Add test

* Fix

* Add changelog

* Test fixes

* Test fix

* Test fixes

* Resolve fixme

* Test fixes

* Remove debug logs

* Try to get CI tests passing again
2020-10-09 07:52:46 -04:00
James Agnew ef4f3df945
Don't install draft package contents (#2099)
* Don't install draft package contents

* Add changelog

* Add changelog
2020-09-22 09:44:36 -04:00
James Agnew 437e81fc65
Support multiple updates of one resource in a transaction (#2050)
* Support multiple updates of one resource in a transaction

* Test update

* Add changelog

* Test fix
2020-08-25 05:55:54 -04:00
jamesagnew 92718c7ef2 Version bump to 5.2.0-SNAPSHOT 2020-08-12 10:23:46 -04:00
jamesagnew 5fb7bd5156 Version bump to 5.1.0 2020-08-11 17:43:24 -04:00
James Agnew e61b39fc30
Force a flush of the SearchParameter cache sooner when SPs change (#2030)
* Force a flush of the SearchParameter cache sooner when SPs change

* Add changelog

* Test fixes
2020-08-09 19:30:59 -04:00