From 777859ad00f11b6b62e3084fa8e49a40a387b229 Mon Sep 17 00:00:00 2001 From: Tadgh Date: Fri, 10 Nov 2023 14:00:08 -0800 Subject: [PATCH] Rel 6 10 mb (#5441) * Allow cached search with consent active when safe (#5387) Allow the search cache when using consent if safe * Change package installation behaviour such that it updates the existing SearchParameter base with remaining resources (#5376) * Change package installation behavior such that it updates the existing SearchParameter base with remaining resources * Change package installation behavior such that it updates the existing SearchParameter base with remaining resources * Use resourceType in the package installer output to fix tests. Minor change with resourceType condition. Update changelog description to make it more readable. * Use resourceType in the package installer output to fix tests. Minor change with resourceType condition. Update changelog description to make it more readable. * Transaction with conditional update fails if SearchNarrowingInterceptor is registered and Enabled Partitioning (#5389) * Transaction with conditional update fails if SearchNarrowingInterceptor is registered and Enabled Partitioning - Implementation * Reverse Chaining searches returns an error when invoked with parameter _lastUpdated. (#5177) * version bump * Bump to core release 6.0.22 (#5028) * Bump to core release 6.0.16 * Bump to core version 6.0.20 * Fix errors thrown as a result of VersionSpecificWorkerContextWrapper * Bump to core 6.0.22 * Resolve 5126 hfj res ver prov might cause migration error on db that automatically indexes the primary key (#5127) * dropped old index FK_RESVERPROV_RES_PID on RES_PID column before adding IDX_RESVERPROV_RES_PID * added changelog * changed to valid version number * changed to valid version number, need to be ordered by version number... * 5123 - Use DEFAULT partition for server-based requests if none specified (#5124) 5123 - Use DEFAULT partition for server-based requests if none specified * consent remove all suppresses next link in bundle (#5119) * added FIXME with source of issue * added FIXME with root cause * added FIXME with root cause * Providing solution to the issue and removing fixmes. * Providing changelog * auto-formatting. * Adding new test. * Adding a new test for standard paging * let's try this and see if it works...? * fix tests * cleanup to trigger a new run * fixing tests --------- Co-authored-by: Ken Stevens Co-authored-by: peartree * 5117 MDM Score for No Match Fields Should Not Be Included in Total Score (#5118) * fix, test, changelog * fix, test, changelog --------- Co-authored-by: justindar * _source search parameter needs to support modifiers (#5095) _source search parameter needs to support modifiers - added support form :contains, :missing, :above modifiers * Fix HFQL docs (#5151) * Expunge operation on codesystem may throw 500 internal error with precondition fail message. (#5156) * Initial failing test. * Solution with changelog. * fixing format. * Addressing comment from code review. * fixing failing test. --------- Co-authored-by: peartree * documentation update (#5154) Co-authored-by: leif stawnyczy * Fix hsql jdbc driver deps (#5168) Avoid non-included classes in jdbc driver dependencies. * $delete-expunge over 10k resources will now delete all resources (#5144) * First commit with very rough fix and unit test. * Refinements to ResourceIdListStep and Batch2DaoSvcImpl. Make LoadIdsStepTest pass. Enhance Batch2DaoSvcImplTest. * Spotless * Fix checkstyle errors. * Fix test failures. * Minor refactoring. New unit test. Finalize changelist. * Spotless fix. * Delete now useless code from unit test. * Delete more useless code. * Test pre-commit hook * More spotless fixes. * Address most code review feedback. * Remove use of pageSize parameter and see if this breaks the pipeline. * Remove use of pageSize parameter and see if this breaks the pipeline. * Fix the noUrl case by passing an unlimited Pegeable instead. Effectively stop using page size for most databases. * Deprecate the old method and have it call the new one by default. * updating documentation (#5170) Co-authored-by: leif stawnyczy * _source search parameter modifiers for Subscription matching (#5159) * _source search parameter modifiers for Subscription matching - test, implementation and changelog * first fix * tests and preliminary fixes * wip, commit before switching to release branch. * adding capability to handle _lastUpdated in reverse search (_has) * adding changelog * applying spotless. * addressing code review comments. --------- Co-authored-by: tadgh Co-authored-by: dotasek Co-authored-by: Steve Corbett <137920358+steve-corbett-smilecdr@users.noreply.github.com> Co-authored-by: Ken Stevens Co-authored-by: Ken Stevens Co-authored-by: peartree Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com> Co-authored-by: justindar Co-authored-by: volodymyr-korzh <132366313+volodymyr-korzh@users.noreply.github.com> Co-authored-by: Nathan Doef Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com> Co-authored-by: TipzCM Co-authored-by: leif stawnyczy Co-authored-by: michaelabuckley Co-authored-by: Luke deGruchy * Br 20231019 add cr settings for cds hooks (#5394) * Add settings used in CR CDS Services. Remove config dependency on Spring Boot. * Add changelog * Use String.format rather than concat strings * spotless apply * Add javadoc * Upgrade notes for the forced-id change (#5400) Add upgrade notes for forced-id * Clean stale search results more aggressively. (#5396) Use bulk DMA statements when cleaning the search cache. The cleaner job now works as long as possible until a deadline based on the scheduling frequency. * bump version of clinical reasoning (#5406) * Transaction fails if SearchNarrowingInterceptor is registered and Partitioning Enabled - fix cross-tenant requests failure (#5408) * Transaction with conditional update fails if SearchNarrowingInterceptor is registered and Enabled Partitioning - fix and tests added * removed unused alias from SQL query of mdm-clear (#5416) * Issue 5418 support Boolean class return type in BaseInterceptorService (#5421) * Enable child classes to use Boolean class return type * spotless --------- Co-authored-by: juan.marchionatto * If AutoInflateBinaries is enabled, binaries are created on the disk only for the first resource entry of the bundle (#5420) * If AutoInflateBinaries is enabled, binaries created on disk by bundled requests are created only for the first resource entry - fix * Revert "Issue 5418 support Boolean class return type in BaseInterceptorService (#5421)" (#5423) This reverts commit 4e295a59fb143a2ba54bc44305474096e2acd578. Co-authored-by: Nathan Doef * Use new FHIR_ID column for sorting (#5405) * Sort `_id` using new FHIR_ID column. * Fix old tests that put client-assigned ids first. * Better indexing for sort * Bump core to 6.1.2.2 (#5425) * Bump core to 6.1.2.1 Patch release that uses https for primary org.hl7.fhir.core package server * Bump core to 6.1.2.2 * Make sure to return always a value for Boolean class return type. (#5424) Implement change in a non-disruptive way for overriders Co-authored-by: juan.marchionatto * Add non-standard __pid SP for breaking ties cheaply during sorts. (#5428) Add a non-standard __pid SP. * Review changes for new _pid SP. (#5430) Change name to _pid to match our standard and add warning. * Fix VersionCanonicalizer conversion from R5 into DSTU2 for CapabilityStatement, Parameters and StructuredDefinition (#5432) * Fix VersionCanonicalizer conversion from R5 into DSTU2 for CapabilityStatement, Parameters and StructuredDefinition. * Fix spotless issue * CVEs for 6.10.0 (#5433) * Bump jetty * Bump okio-jvm * 8.2.0 mysql connector * Jena and elastic bumps * Fix test * 5412 post bundle on partition incorrect response.link shown (#5413) * Initial fix and unit test provided * spottless check * Made relevant changes to make solution version agnostic * relevant logic changes made * spotless changes made * New logic added to fix failing test cases * formatting * New logic to make the function more robust * spotless checks * Left a trailing slash in the tests * Made relevant test changes and changed logic * spotless changes * Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5412-during-partition-fullUrl-not-shown-in-response.yaml changing changelog Co-authored-by: volodymyr-korzh <132366313+volodymyr-korzh@users.noreply.github.com> * Formatting requirements --------- Co-authored-by: volodymyr-korzh <132366313+volodymyr-korzh@users.noreply.github.com> * Resolve We don't have guaranteed subscription delivery if a resource is too large (#5414) * first fix * - added the ability to handle null payload to SubscriptionDeliveringMessageSubscriber and SubscriptionDeliveringEmailSubscriber - refactored code to reduce repeated code - cleaned unnecessary comments and reformatted files * Changed myResourceModifiedMessagePersistenceSvc to be autowired * removed unused import * added error handling when inflating the message to email and message subscriber * reformatted code * Fixing subscription tests with mocked IResourceModifiedMessagePersistenceSvc * Changes by gary * Reformatted file * fixed failed tests * implemented test for message and email delivery subscriber. Fixed logical error. Reformatted File. * - implemented IT - fixed logical error - added changelog * fix for cdr tests, NOTE: this makes the assumption that we will always succeed for inflating the database in the tests that uses SynchronousSubscriptionMatcherInterceptor * fix for cdr tests, NOTE: this makes the assumption that we will always succeed for inflating the database in the tests that uses SynchronousSubscriptionMatcherInterceptor * resolve code review comments * reformatted files * fixed tests * Fix for failing IT test in jpaserver-starter (#5435) Co-authored-by: dotasek * wip --------- Co-authored-by: michaelabuckley Co-authored-by: Martha Mitran Co-authored-by: volodymyr-korzh <132366313+volodymyr-korzh@users.noreply.github.com> Co-authored-by: TynerGjs <132295567+TynerGjs@users.noreply.github.com> Co-authored-by: dotasek Co-authored-by: Steve Corbett <137920358+steve-corbett-smilecdr@users.noreply.github.com> Co-authored-by: Ken Stevens Co-authored-by: Ken Stevens Co-authored-by: peartree Co-authored-by: jdar8 <69840459+jdar8@users.noreply.github.com> Co-authored-by: justindar Co-authored-by: Nathan Doef Co-authored-by: Etienne Poirier <33007955+epeartree@users.noreply.github.com> Co-authored-by: TipzCM Co-authored-by: leif stawnyczy Co-authored-by: Luke deGruchy Co-authored-by: Brenin Rhodes Co-authored-by: Justin McKelvy <60718638+Capt-Mac@users.noreply.github.com> Co-authored-by: jmarchionatto <60409882+jmarchionatto@users.noreply.github.com> Co-authored-by: juan.marchionatto Co-authored-by: Nathan Doef Co-authored-by: LalithE <132382565+LalithE@users.noreply.github.com> Co-authored-by: dotasek --- .../executor/BaseInterceptorService.java | 18 +- .../java/ca/uhn/fhir/rest/api/Constants.java | 2 + .../canonical/VersionCanonicalizer.java | 6 +- .../canonical/VersionCanonicalizerTest.java | 164 ++++---- .../4803-forced-id-step-2.yaml | 0 ...-lastUpdated-sp-with-reverse-chaining.yaml | 6 + ...rameters-with-multiple-base-resources.yaml | 7 + .../5375-add-cr-settings-for-cds-hooks.yaml | 4 + ...5387-allow-cached-search-with-consent.yaml | 6 + ...s-registered-and-partitioning-enabled.yaml | 6 + .../6_10_0/5395-search-cleaner-faster.yaml | 5 + .../5404-cql-translator-fhirhelpers-bug.yaml | 4 + .../6_10_0/5405-use-new-fhir-id-for-sort.yaml | 4 + ...n-delivery-if-a-resource-is-too-large.yaml | 7 + ...-partition-response-link-is-incorrect.yaml | 4 + .../6_10_0/5415-mdm-clear-fails-on-mssql.yaml | 4 + ...he-first-resource-entry-of-the-bundle.yaml | 6 + .../fhir/changelog/6_10_0/5428-pid-sp.yaml | 5 + ...lizer_fails_capabilitystatement_dstu2.yaml | 5 + .../hapi/fhir/changelog/6_10_0/changes.yaml | 2 +- .../uhn/hapi/fhir/changelog/6_10_0/upgrade.md | 7 + .../fhir/docs/security/consent_interceptor.md | 10 + .../uhn/hapi/fhir/docs/server_jpa/search.md | 6 + .../config/HapiFhirHibernateJpaDialect.java | 2 +- .../ca/uhn/fhir/jpa/config/JpaConfig.java | 7 - .../jpa/dao/data/IMdmLinkJpaRepository.java | 2 +- .../ca/uhn/fhir/jpa/dao/data/ISearchDao.java | 22 +- .../fhir/jpa/dao/data/ISearchIncludeDao.java | 8 +- .../fhir/jpa/dao/data/ISearchResultDao.java | 17 +- .../jpa/dao/data/SearchIdAndResultSize.java | 37 ++ .../ca/uhn/fhir/jpa/entity/SearchResult.java | 9 +- .../tasks/HapiFhirJpaMigrationTasks.java | 13 +- .../jpa/packages/PackageInstallerSvcImpl.java | 268 +++++++------ .../search/StaleSearchDeletingSvcImpl.java | 16 +- .../fhir/jpa/search/builder/QueryStack.java | 256 ++++++++++--- .../jpa/search/builder/SearchBuilder.java | 18 +- .../predicate/ForcedIdPredicateBuilder.java | 51 --- .../ResourceLinkPredicateBuilder.java | 17 +- .../ResourceTablePredicateBuilder.java | 9 +- .../builder/sql/SearchQueryBuilder.java | 38 +- .../search/builder/sql/SqlObjectFactory.java | 5 - .../cache/DatabaseSearchCacheSvcImpl.java | 359 +++++++++++++----- .../jpa/search/cache/ISearchCacheSvc.java | 8 +- ...urceModifiedMessagePersistenceSvcImpl.java | 56 ++- .../ca/uhn/fhir/jpa/term/TermReadSvcImpl.java | 2 +- .../packages/PackageInstallerSvcImplTest.java | 147 +++++-- .../FhirResourceDaoR4SearchLastNAsyncIT.java | 7 +- .../fhir/jpa/model/entity/ResourceTable.java | 12 +- .../jpa/searchparam/ResourceMetaParams.java | 2 + .../BaseSubscriptionDeliverySubscriber.java | 19 + ...SubscriptionDeliveringEmailSubscriber.java | 24 +- ...bscriptionDeliveringMessageSubscriber.java | 16 +- .../SubscriptionActivatingSubscriber.java | 16 +- .../SubscriptionMatchDeliverer.java | 18 +- .../SubscriptionMatchingSubscriber.java | 15 + ...hronousSubscriptionMatcherInterceptor.java | 26 +- .../svc/ResourceModifiedSubmitterSvc.java | 40 +- .../SubscriptionTopicMatchingSubscriber.java | 15 + ...aseSubscriptionDeliverySubscriberTest.java | 53 +++ .../matching/DaoSubscriptionMatcherTest.java | 6 + .../cache/SubscriptionRegistrySharedTest.java | 5 + .../config/TestSubscriptionDstu3Config.java | 6 + ...kingQueueSubscribableChannelDstu3Test.java | 9 + .../SubscriptionMatchingSubscriberTest.java | 8 + .../WebsocketConnectionValidatorTest.java | 6 + .../dao/dstu2/FhirResourceDaoDstu2Test.java | 6 +- .../FhirResourceDaoDstu3SearchNoFtTest.java | 2 +- .../dao/dstu3/FhirResourceDaoDstu3Test.java | 6 +- .../jpa/dao/r4/BasePartitioningR4Test.java | 2 +- .../dao/r4/FhirResourceDaoR4QuerySandbox.java | 150 ++++++++ .../r4/FhirResourceDaoR4SearchNoFtTest.java | 52 +++ .../jpa/dao/r4/FhirResourceDaoR4SortTest.java | 4 +- ...rResourceDaoR4StandardQueriesNoFTTest.java | 60 ++- .../jpa/dao/r4/FhirResourceDaoR4Test.java | 57 --- .../dao/r4/SearchCoordinatorSvcImplTest.java | 31 +- .../PackageInstallerSvcImplCreateTest.java | 9 +- ...ageInstallerSvcImplRewriteHistoryTest.java | 2 +- .../r4/BinaryStorageInterceptorR4Test.java | 92 ++++- ...onsentInterceptorResourceProviderR4IT.java | 155 ++++---- .../provider/r4/MultitenantServerR4Test.java | 54 +++ .../r4/StaleSearchDeletingSvcR4Test.java | 31 +- .../message/MessageSubscriptionR4Test.java | 20 +- .../svc/ResourceModifiedSubmitterSvcTest.java | 10 +- .../ca/uhn/fhir/jpa/dao/TestDaoSearch.java | 28 ++ .../jpa/search/IIdSearchTestTemplate.java | 74 ++++ .../ca/uhn/fhir/jpa/test/BaseJpaTest.java | 5 + .../jpa/test/config/ConnectionWrapper.java | 1 + .../fhir/jpa/config/ConnectionWrapper.java | 1 + .../HapiFhirHibernateJpaDialectTest.java | 4 +- .../fhir/cdshooks/api/ICdsConfigService.java | 4 + .../fhir/cdshooks/config/CdsCrConfig.java | 37 ++ .../fhir/cdshooks/config/CdsHooksConfig.java | 13 +- .../cdshooks/svc/CdsConfigServiceImpl.java | 10 + .../cdshooks/svc/cr/CdsCrServiceDstu3.java | 22 +- .../fhir/cdshooks/svc/cr/CdsCrServiceR4.java | 24 +- .../fhir/cdshooks/svc/cr/CdsCrServiceR5.java | 23 +- .../fhir/cdshooks/svc/cr/CdsCrSettings.java | 44 +++ .../cdshooks/config/TestCdsHooksConfig.java | 4 + .../hapi/fhir/cdshooks/svc/cr/BaseCrTest.java | 2 + .../fhir/cdshooks/svc/cr/TestCrConfig.java | 24 ++ .../svc/cr/resolution/CdsCrServiceR4Test.java | 14 +- .../test/resources/ASLPCrdServiceRequest.json | 7 + .../uhn/fhir/rest/server/RestfulServer.java | 18 +- .../fhir/rest/server/RestfulServerUtils.java | 21 +- .../consent/ConsentInterceptor.java | 80 ++-- .../consent/DelegatingConsentService.java | 6 + .../BaseResourceModifiedMessage.java | 12 +- .../tenant/ITenantIdentificationStrategy.java | 11 +- .../UrlBaseTenantIdentificationStrategy.java | 31 +- .../rest/server/util/ServletRequestUtil.java | 1 + .../rest/server/RestfulServerUtilsTest.java | 44 ++- hapi-fhir-storage-cr/pom.xml | 6 - .../uhn/fhir/cr/config/CrConfigCondition.java | 60 +++ .../uhn/fhir/cr/config/RepositoryConfig.java | 2 - .../cr/config/RepositoryConfigCondition.java | 47 +++ .../cr/config/dstu3/ApplyOperationConfig.java | 20 +- .../cr/config/dstu3/CrProcessorConfig.java | 56 +++ .../config/dstu3/ExtractOperationConfig.java | 13 +- .../config/dstu3/PackageOperationConfig.java | 20 +- .../config/dstu3/PopulateOperationConfig.java | 13 +- .../cr/config/r4/ApplyOperationConfig.java | 20 +- .../fhir/cr/config/r4/CrProcessorConfig.java | 56 +++ .../cr/config/r4/ExtractOperationConfig.java | 13 +- .../cr/config/r4/PackageOperationConfig.java | 20 +- .../cr/config/r4/PopulateOperationConfig.java | 13 +- .../fhir/storage/test/DaoTestDataBuilder.java | 7 +- .../interceptor/BinaryStorageInterceptor.java | 24 +- .../jpa/dao/tx/HapiTransactionService.java | 3 +- .../channel/api/PayloadTooLargeException.java | 35 ++ .../model/ResourceDeliveryMessage.java | 4 + .../model/ResourceModifiedMessage.java | 10 + ...ResourceModifiedMessagePersistenceSvc.java | 24 +- .../ca/uhn/fhir/parser/RDFParserR4Test.java | 6 +- .../interceptor/ConsentInterceptorTest.java | 62 ++- ...lBaseTenantIdentificationStrategyTest.java | 26 ++ pom.xml | 14 +- 136 files changed, 2823 insertions(+), 1017 deletions(-) rename hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/{6_8_0 => 6_10_0}/4803-forced-id-step-2.yaml (100%) create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5176-supporting-lastUpdated-sp-with-reverse-chaining.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5366-package-installer-overrides-build-in-search-parameters-with-multiple-base-resources.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5375-add-cr-settings-for-cds-hooks.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5387-allow-cached-search-with-consent.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5388-fhir-transaction-fails-if-searchnarrowinginterceptor-is-registered-and-partitioning-enabled.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5395-search-cleaner-faster.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5404-cql-translator-fhirhelpers-bug.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5405-use-new-fhir-id-for-sort.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5407-we-dont-have-guaranteed-subscription-delivery-if-a-resource-is-too-large.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5412-during-partition-response-link-is-incorrect.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5415-mdm-clear-fails-on-mssql.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5419-binaries-created-only-for-the-first-resource-entry-of-the-bundle.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5428-pid-sp.yaml create mode 100644 hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5431-version_canonicalizer_fails_capabilitystatement_dstu2.yaml create mode 100644 hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/SearchIdAndResultSize.java delete mode 100644 hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ForcedIdPredicateBuilder.java create mode 100644 hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4QuerySandbox.java create mode 100644 hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/search/IIdSearchTestTemplate.java create mode 100644 hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsCrConfig.java create mode 100644 hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrSettings.java create mode 100644 hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/CrConfigCondition.java create mode 100644 hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfigCondition.java create mode 100644 hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/CrProcessorConfig.java create mode 100644 hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/CrProcessorConfig.java create mode 100644 hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/channel/api/PayloadTooLargeException.java diff --git a/hapi-fhir-base/src/main/java/ca/uhn/fhir/interceptor/executor/BaseInterceptorService.java b/hapi-fhir-base/src/main/java/ca/uhn/fhir/interceptor/executor/BaseInterceptorService.java index 07ac7df658b..9a592bdd6e9 100644 --- a/hapi-fhir-base/src/main/java/ca/uhn/fhir/interceptor/executor/BaseInterceptorService.java +++ b/hapi-fhir-base/src/main/java/ca/uhn/fhir/interceptor/executor/BaseInterceptorService.java @@ -263,10 +263,14 @@ public abstract class BaseInterceptorService & I return myRegisteredPointcuts.contains(thePointcut); } + protected Class getBooleanReturnType() { + return boolean.class; + } + @Override public boolean callHooks(POINTCUT thePointcut, HookParams theParams) { assert haveAppropriateParams(thePointcut, theParams); - assert thePointcut.getReturnType() == void.class || thePointcut.getReturnType() == boolean.class; + assert thePointcut.getReturnType() == void.class || thePointcut.getReturnType() == getBooleanReturnType(); Object retValObj = doCallHooks(thePointcut, theParams, true); return (Boolean) retValObj; @@ -282,14 +286,16 @@ public abstract class BaseInterceptorService & I for (BaseInvoker nextInvoker : invokers) { Object nextOutcome = nextInvoker.invoke(theParams); Class pointcutReturnType = thePointcut.getReturnType(); - if (pointcutReturnType.equals(boolean.class)) { + if (pointcutReturnType.equals(getBooleanReturnType())) { Boolean nextOutcomeAsBoolean = (Boolean) nextOutcome; if (Boolean.FALSE.equals(nextOutcomeAsBoolean)) { ourLog.trace("callHooks({}) for invoker({}) returned false", thePointcut, nextInvoker); theRetVal = false; break; + } else { + theRetVal = true; } - } else if (pointcutReturnType.equals(void.class) == false) { + } else if (!pointcutReturnType.equals(void.class)) { if (nextOutcome != null) { theRetVal = nextOutcome; break; @@ -349,7 +355,7 @@ public abstract class BaseInterceptorService & I List retVal; - if (haveMultiple == false) { + if (!haveMultiple) { // The global list doesn't need to be sorted every time since it's sorted on // insertion each time. Doing so is a waste of cycles.. @@ -485,9 +491,9 @@ public abstract class BaseInterceptorService & I myMethod = theHookMethod; Class returnType = theHookMethod.getReturnType(); - if (myPointcut.getReturnType().equals(boolean.class)) { + if (myPointcut.getReturnType().equals(getBooleanReturnType())) { Validate.isTrue( - boolean.class.equals(returnType) || void.class.equals(returnType), + getBooleanReturnType().equals(returnType) || void.class.equals(returnType), "Method does not return boolean or void: %s", theHookMethod); } else if (myPointcut.getReturnType().equals(void.class)) { diff --git a/hapi-fhir-base/src/main/java/ca/uhn/fhir/rest/api/Constants.java b/hapi-fhir-base/src/main/java/ca/uhn/fhir/rest/api/Constants.java index 780e528759e..e6301929bfa 100644 --- a/hapi-fhir-base/src/main/java/ca/uhn/fhir/rest/api/Constants.java +++ b/hapi-fhir-base/src/main/java/ca/uhn/fhir/rest/api/Constants.java @@ -199,6 +199,8 @@ public class Constants { public static final String PARAM_PRETTY_VALUE_FALSE = "false"; public static final String PARAM_PRETTY_VALUE_TRUE = "true"; public static final String PARAM_PROFILE = "_profile"; + public static final String PARAM_PID = "_pid"; + public static final String PARAM_QUERY = "_query"; public static final String PARAM_RESPONSE_URL = "response-url"; // Used in messaging public static final String PARAM_REVINCLUDE = "_revinclude"; diff --git a/hapi-fhir-converter/src/main/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizer.java b/hapi-fhir-converter/src/main/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizer.java index 16c0af9310e..916322631f9 100644 --- a/hapi-fhir-converter/src/main/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizer.java +++ b/hapi-fhir-converter/src/main/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizer.java @@ -458,7 +458,7 @@ public class VersionCanonicalizer { @Override public IBaseParameters parametersFromCanonical(Parameters theParameters) { Resource converted = VersionConvertorFactory_10_40.convertResource(theParameters, ADVISOR_10_40); - return (IBaseParameters) reencodeToHl7Org(converted); + return (IBaseParameters) reencodeFromHl7Org(converted); } @Override @@ -470,7 +470,7 @@ public class VersionCanonicalizer { @Override public IBaseResource structureDefinitionFromCanonical(StructureDefinition theResource) { Resource converted = VersionConvertorFactory_10_50.convertResource(theResource, ADVISOR_10_50); - return reencodeToHl7Org(converted); + return reencodeFromHl7Org(converted); } @Override @@ -514,7 +514,7 @@ public class VersionCanonicalizer { @Override public IBaseConformance capabilityStatementFromCanonical(CapabilityStatement theResource) { Resource converted = VersionConvertorFactory_10_50.convertResource(theResource, ADVISOR_10_50); - return (IBaseConformance) reencodeToHl7Org(converted); + return (IBaseConformance) reencodeFromHl7Org(converted); } private Resource reencodeToHl7Org(IBaseResource theInput) { diff --git a/hapi-fhir-converter/src/test/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizerTest.java b/hapi-fhir-converter/src/test/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizerTest.java index bd1e53cba51..c9ff28b1c64 100644 --- a/hapi-fhir-converter/src/test/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizerTest.java +++ b/hapi-fhir-converter/src/test/java/ca/uhn/hapi/converters/canonical/VersionCanonicalizerTest.java @@ -2,22 +2,19 @@ package ca.uhn.hapi.converters.canonical; import ca.uhn.fhir.context.FhirVersionEnum; import ca.uhn.fhir.model.dstu2.composite.CodingDt; +import ca.uhn.fhir.model.dstu2.resource.Conformance; import ca.uhn.fhir.util.HapiExtensions; -import org.apache.commons.lang3.StringUtils; -import org.hl7.fhir.convertors.factory.VersionConvertorFactory_40_50; import org.hl7.fhir.instance.model.api.IBaseCoding; -import org.hl7.fhir.instance.model.api.IBaseHasExtensions; -import org.hl7.fhir.instance.model.api.IBaseResource; -import org.hl7.fhir.instance.model.api.IPrimitiveType; import org.hl7.fhir.r4.model.CodeType; import org.hl7.fhir.r4.model.Coding; -import org.hl7.fhir.r5.model.Base; +import org.hl7.fhir.r4.model.Parameters; +import org.hl7.fhir.r5.model.CapabilityStatement; import org.hl7.fhir.r5.model.Enumeration; import org.hl7.fhir.r5.model.SearchParameter; +import org.hl7.fhir.r5.model.StructureDefinition; +import org.junit.jupiter.api.Nested; import org.junit.jupiter.api.Test; -import javax.annotation.Nonnull; -import java.util.List; import java.util.stream.Collectors; import static ca.uhn.fhir.util.ExtensionUtil.getExtensionPrimitiveValues; @@ -25,73 +22,100 @@ import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.contains; import static org.hamcrest.Matchers.empty; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; class VersionCanonicalizerTest { + @Nested + class VersionCanonicalizerR4 { - @Test - public void testToCanonicalCoding() { - VersionCanonicalizer canonicalizer = new VersionCanonicalizer(FhirVersionEnum.DSTU2); - IBaseCoding coding = new CodingDt("dstuSystem", "dstuCode"); - Coding convertedCoding = canonicalizer.codingToCanonical(coding); - assertEquals("dstuCode", convertedCoding.getCode()); - assertEquals("dstuSystem", convertedCoding.getSystem()); + private static final FhirVersionEnum FHIR_VERSION = FhirVersionEnum.R4; + private static final VersionCanonicalizer ourCanonicalizer = new VersionCanonicalizer(FHIR_VERSION); + @Test + public void testToCanonical_SearchParameterNoCustomResourceType_ConvertedCorrectly() { + org.hl7.fhir.r4.model.SearchParameter input = new org.hl7.fhir.r4.model.SearchParameter(); + input.addBase("Patient"); + input.addBase("Observation"); + input.addTarget("Organization"); + + // Test + org.hl7.fhir.r5.model.SearchParameter actual = ourCanonicalizer.searchParameterToCanonical(input); + + // Verify + assertThat(actual.getBase().stream().map(Enumeration::getCode).collect(Collectors.toList()), contains("Patient", "Observation")); + assertThat(actual.getTarget().stream().map(Enumeration::getCode).collect(Collectors.toList()), contains("Organization")); + assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_BASE_RESOURCE), empty()); + assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_TARGET_RESOURCE), empty()); + + } + + @Test + public void testToCanonical_SearchParameterWithCustomResourceType__ConvertedCorrectly() { + // Setup + org.hl7.fhir.r4.model.SearchParameter input = new org.hl7.fhir.r4.model.SearchParameter(); + input.addBase("Base1"); + input.addBase("Base2"); + input.addTarget("Target1"); + input.addTarget("Target2"); + + // Test + org.hl7.fhir.r5.model.SearchParameter actual = ourCanonicalizer.searchParameterToCanonical(input); + + // Verify + assertThat(actual.getBase().stream().map(Enumeration::getCode).collect(Collectors.toList()), empty()); + assertThat(actual.getTarget().stream().map(Enumeration::getCode).collect(Collectors.toList()), empty()); + assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_BASE_RESOURCE), contains("Base1", "Base2")); + assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_TARGET_RESOURCE), contains("Target1", "Target2")); + // Original shouldn't be modified + assertThat(input.getBase().stream().map(CodeType::getCode).toList(), contains("Base1", "Base2")); + assertThat(input.getTarget().stream().map(CodeType::getCode).toList(), contains("Target1", "Target2")); + + } } - @Test - public void testFromCanonicalSearchParameter() { - VersionCanonicalizer canonicalizer = new VersionCanonicalizer(FhirVersionEnum.DSTU2); + @Nested + class VersionCanonicalizerDstu2 { + private static final FhirVersionEnum FHIR_VERSION = FhirVersionEnum.DSTU2; + private static final VersionCanonicalizer ourCanonicalizer = new VersionCanonicalizer(FHIR_VERSION); - SearchParameter inputR5 = new SearchParameter(); - inputR5.setUrl("http://foo"); - ca.uhn.fhir.model.dstu2.resource.SearchParameter outputDstu2 = (ca.uhn.fhir.model.dstu2.resource.SearchParameter) canonicalizer.searchParameterFromCanonical(inputR5); - assertEquals("http://foo", outputDstu2.getUrl()); + @Test + public void testToCanonical_Coding_ConvertSuccessful() { + IBaseCoding coding = new CodingDt("dstuSystem", "dstuCode"); + Coding convertedCoding = ourCanonicalizer.codingToCanonical(coding); + assertEquals("dstuCode", convertedCoding.getCode()); + assertEquals("dstuSystem", convertedCoding.getSystem()); + } + + @Test + public void testFromCanonical_SearchParameter_ConvertSuccessful() { + SearchParameter inputR5 = new SearchParameter(); + inputR5.setUrl("http://foo"); + ca.uhn.fhir.model.dstu2.resource.SearchParameter outputDstu2 = (ca.uhn.fhir.model.dstu2.resource.SearchParameter) ourCanonicalizer.searchParameterFromCanonical(inputR5); + assertEquals("http://foo", outputDstu2.getUrl()); + } + + @Test + public void testFromCanonical_CapabilityStatement_ConvertSuccessful() { + CapabilityStatement inputR5 = new CapabilityStatement(); + inputR5.setUrl("http://foo"); + Conformance conformance = (Conformance) ourCanonicalizer.capabilityStatementFromCanonical(inputR5); + assertEquals("http://foo", conformance.getUrl()); + } + + @Test + public void testFromCanonical_StructureDefinition_ConvertSuccessful() { + StructureDefinition inputR5 = new StructureDefinition(); + inputR5.setId("123"); + ca.uhn.fhir.model.dstu2.resource.StructureDefinition structureDefinition = (ca.uhn.fhir.model.dstu2.resource.StructureDefinition) ourCanonicalizer.structureDefinitionFromCanonical(inputR5); + assertEquals("StructureDefinition/123", structureDefinition.getId().getValue()); + } + + @Test + public void testFromCanonical_Parameters_ConvertSuccessful() { + org.hl7.fhir.r4.model.Parameters inputR4 = new Parameters(); + inputR4.setParameter("paramA", "1"); + ca.uhn.fhir.model.dstu2.resource.Parameters parameters = (ca.uhn.fhir.model.dstu2.resource.Parameters) ourCanonicalizer.parametersFromCanonical(inputR4); + assertNotNull(parameters.getParameter()); + assertEquals("paramA", parameters.getParameter().get(0).getName()); + } } - - @Test - public void testToCanonicalSearchParameter_NoCustomResourceType() { - // Setup - VersionCanonicalizer canonicalizer = new VersionCanonicalizer(FhirVersionEnum.R4); - - org.hl7.fhir.r4.model.SearchParameter input = new org.hl7.fhir.r4.model.SearchParameter(); - input.addBase("Patient"); - input.addBase("Observation"); - input.addTarget("Organization"); - - // Test - org.hl7.fhir.r5.model.SearchParameter actual = canonicalizer.searchParameterToCanonical(input); - - // Verify - assertThat(actual.getBase().stream().map(Enumeration::getCode).collect(Collectors.toList()), contains("Patient", "Observation")); - assertThat(actual.getTarget().stream().map(Enumeration::getCode).collect(Collectors.toList()), contains("Organization")); - assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_BASE_RESOURCE), empty()); - assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_TARGET_RESOURCE), empty()); - - } - - @Test - public void testToCanonicalSearchParameter_WithCustomResourceType() { - // Setup - VersionCanonicalizer canonicalizer = new VersionCanonicalizer(FhirVersionEnum.R4); - - org.hl7.fhir.r4.model.SearchParameter input = new org.hl7.fhir.r4.model.SearchParameter(); - input.addBase("Base1"); - input.addBase("Base2"); - input.addTarget("Target1"); - input.addTarget("Target2"); - - // Test - org.hl7.fhir.r5.model.SearchParameter actual = canonicalizer.searchParameterToCanonical(input); - - // Verify - assertThat(actual.getBase().stream().map(Enumeration::getCode).collect(Collectors.toList()), empty()); - assertThat(actual.getTarget().stream().map(Enumeration::getCode).collect(Collectors.toList()), empty()); - assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_BASE_RESOURCE), contains("Base1", "Base2")); - assertThat(getExtensionPrimitiveValues(actual, HapiExtensions.EXTENSION_SEARCHPARAM_CUSTOM_TARGET_RESOURCE), contains("Target1", "Target2")); - // Original shouldn't be modified - assertThat(input.getBase().stream().map(CodeType::getCode).toList(), contains("Base1", "Base2")); - assertThat(input.getTarget().stream().map(CodeType::getCode).toList(), contains("Target1", "Target2")); - - } - - } diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_8_0/4803-forced-id-step-2.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/4803-forced-id-step-2.yaml similarity index 100% rename from hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_8_0/4803-forced-id-step-2.yaml rename to hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/4803-forced-id-step-2.yaml diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5176-supporting-lastUpdated-sp-with-reverse-chaining.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5176-supporting-lastUpdated-sp-with-reverse-chaining.yaml new file mode 100644 index 00000000000..3c050843501 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5176-supporting-lastUpdated-sp-with-reverse-chaining.yaml @@ -0,0 +1,6 @@ +--- +type: fix +issue: 5176 +jira: SMILE-6333 +title: "Previously, the use of search parameter _lastUpdated as part of a reverse chaining search would return an error + message to the client. This issue has been fixed" diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5366-package-installer-overrides-build-in-search-parameters-with-multiple-base-resources.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5366-package-installer-overrides-build-in-search-parameters-with-multiple-base-resources.yaml new file mode 100644 index 00000000000..ba042540a99 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5366-package-installer-overrides-build-in-search-parameters-with-multiple-base-resources.yaml @@ -0,0 +1,7 @@ +--- +type: add +issue: 5366 +jira: SMILE-5184 +title: "The package installer overrides existing (built-in) SearchParameter with multiple base resources. + This is happening when installing US Core package for Practitioner.given as an example. + This change allows the existing SearchParameter to continue to exist with the remaining base resources." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5375-add-cr-settings-for-cds-hooks.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5375-add-cr-settings-for-cds-hooks.yaml new file mode 100644 index 00000000000..5ca9d560973 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5375-add-cr-settings-for-cds-hooks.yaml @@ -0,0 +1,4 @@ +--- +type: add +issue: 5375 +title: "Add settings for CDS Services using CDS on FHIR. Also removed the dependency on Spring Boot from the CR configs used by CDS Hooks." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5387-allow-cached-search-with-consent.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5387-allow-cached-search-with-consent.yaml new file mode 100644 index 00000000000..543467f2dc4 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5387-allow-cached-search-with-consent.yaml @@ -0,0 +1,6 @@ +--- +type: perf +issue: 5387 +title: "Enable the search cache for some requests even when a consent interceptor is active. + If no consent service uses canSeeResource (i.e. shouldProcessCanSeeResource() returns false); + or startOperation() returns AUTHORIZED; then the search cache is enabled." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5388-fhir-transaction-fails-if-searchnarrowinginterceptor-is-registered-and-partitioning-enabled.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5388-fhir-transaction-fails-if-searchnarrowinginterceptor-is-registered-and-partitioning-enabled.yaml new file mode 100644 index 00000000000..dcfc89bcc74 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5388-fhir-transaction-fails-if-searchnarrowinginterceptor-is-registered-and-partitioning-enabled.yaml @@ -0,0 +1,6 @@ +--- +type: fix +issue: 5388 +title: "Previously, with partitioning enabled and `UrlBaseTenantIdentificationStrategy` used, registering +`SearchNarrowingInterceptor` would cause to incorrect resolution of `entry.request.url` parameter during +transaction bundle processing. This has been fixed." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5395-search-cleaner-faster.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5395-search-cleaner-faster.yaml new file mode 100644 index 00000000000..871a0b64218 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5395-search-cleaner-faster.yaml @@ -0,0 +1,5 @@ +--- +type: perf +issue: 5395 +title: "The background activity that clears stale search results now has higher throughput. + Busy servers should no longer accumulate dead stale search results." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5404-cql-translator-fhirhelpers-bug.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5404-cql-translator-fhirhelpers-bug.yaml new file mode 100644 index 00000000000..a9ee0f693cc --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5404-cql-translator-fhirhelpers-bug.yaml @@ -0,0 +1,4 @@ +--- +type: fix +issue: 5404 +title: "Cql translating bug where FHIRHelpers library function was erroring and blocking clinical reasoning content functionality" diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5405-use-new-fhir-id-for-sort.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5405-use-new-fhir-id-for-sort.yaml new file mode 100644 index 00000000000..85c322066d1 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5405-use-new-fhir-id-for-sort.yaml @@ -0,0 +1,4 @@ +--- +type: perf +issue: 5405 +title: "Sorting by _id now uses the FHIR_ID column on HFJ_RESOURCE and avoid joins." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5407-we-dont-have-guaranteed-subscription-delivery-if-a-resource-is-too-large.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5407-we-dont-have-guaranteed-subscription-delivery-if-a-resource-is-too-large.yaml new file mode 100644 index 00000000000..e7f8d7ef0be --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5407-we-dont-have-guaranteed-subscription-delivery-if-a-resource-is-too-large.yaml @@ -0,0 +1,7 @@ +--- +type: add +issue: 5407 +title: "Previously, when the payload of a subscription message exceeds the broker maximum message size, exception would +be thrown and retry will be performed indefinitely until the maximum message size is adjusted. Now, the message will be +successfully delivered for rest-hook and email subscriptions, while message subscriptions remains the same behavior as +before." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5412-during-partition-response-link-is-incorrect.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5412-during-partition-response-link-is-incorrect.yaml new file mode 100644 index 00000000000..03f4b2c8443 --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5412-during-partition-response-link-is-incorrect.yaml @@ -0,0 +1,4 @@ +--- +type: fix +issue: 5412 +title: "Previously, with Partitioning enabled, submitting a bundle request would return a response with the partition name displayed twice in `response.link` property. This has been fixed." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5415-mdm-clear-fails-on-mssql.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5415-mdm-clear-fails-on-mssql.yaml new file mode 100644 index 00000000000..d6cfc8cc9da --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5415-mdm-clear-fails-on-mssql.yaml @@ -0,0 +1,4 @@ +--- +type: fix +issue: 5415 +title: "Previously, `$mdm-clear` jobs would fail on MSSQL. This is now fixed." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5419-binaries-created-only-for-the-first-resource-entry-of-the-bundle.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5419-binaries-created-only-for-the-first-resource-entry-of-the-bundle.yaml new file mode 100644 index 00000000000..e1394edddcc --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5419-binaries-created-only-for-the-first-resource-entry-of-the-bundle.yaml @@ -0,0 +1,6 @@ +--- +type: fix +issue: 5419 +title: "Previously, when `AllowAutoInflateBinaries` was enabled in `JpaStorageSettings` and bundles with multiple +resources were submitted, binaries were created on the disk only for the first resource entry of the bundle. +This has been fixed." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5428-pid-sp.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5428-pid-sp.yaml new file mode 100644 index 00000000000..2c91eef4d4e --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5428-pid-sp.yaml @@ -0,0 +1,5 @@ +--- +type: add +issue: 5428 +title: "Add support for non-standard _pid SearchParameter to the the JPA engine. + This new SP provides an efficient tie-breaking sort key." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5431-version_canonicalizer_fails_capabilitystatement_dstu2.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5431-version_canonicalizer_fails_capabilitystatement_dstu2.yaml new file mode 100644 index 00000000000..771291f525e --- /dev/null +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/5431-version_canonicalizer_fails_capabilitystatement_dstu2.yaml @@ -0,0 +1,5 @@ +--- +type: fix +issue: 5431 +jira: SMILE-5306 +title: "Previously, using VersionCanonicalizer to convert a CapabilityStatement from R5 to DSTU2 would fail. This is now fixed." diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/changes.yaml b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/changes.yaml index 86eb2ac6c22..b6ce96f25a1 100644 --- a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/changes.yaml +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/changes.yaml @@ -11,5 +11,5 @@
  • Thymeleaf (Testpage Overlay): 3.0.14.RELEASE -> 3.1.2.RELEASE
  • xpp3 (All): 1.1.4c.0 -> 1.1.6
  • HtmlUnit (All): 2.67.0 -> 2.70.0
    • -
  • org.hl7.fhir.core (All): 6.0.22.2 -> 6.1.2
    • +
  • org.hl7.fhir.core (All): 6.0.22.2 -> 6.1.2.2
    • " diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/upgrade.md b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/upgrade.md index 258d2440c2f..16559ea2e17 100644 --- a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/upgrade.md +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/6_10_0/upgrade.md @@ -1,3 +1,9 @@ +### Major Database Change + +This release makes performance changes to the database definition in a way that is incompatible with releases before 6.4. +Attempting to run version 6.2 or older simultaneously with this release may experience errors when saving new resources. + +### Change Tracking and Subscriptions This release introduces significant a change to the mechanism performing submission of resource modification events to the message broker. Previously, an event would be submitted as part of the synchronous transaction modifying a resource. Synchronous submission yielded responsive publishing with the caveat that events would be dropped @@ -8,6 +14,7 @@ database upon completion of the transaction and subsequently submitted to the br This new asynchronous submission mechanism will introduce a slight delay in event publishing. It is our view that such delay is largely compensated by the capability to retry submission upon failure which will eliminate event losses. +### Tag, Security Label, and Profile changes There are some potentially breaking changes: * On resource retrieval and before storage, tags, security label and profile collections in resource meta will be diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/consent_interceptor.md b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/consent_interceptor.md index 96b2dc481d2..900baa3566d 100644 --- a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/consent_interceptor.md +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/security/consent_interceptor.md @@ -24,3 +24,13 @@ The ConsentInterceptor requires a user-supplied instance of the [IConsentService ```java {{snippet:classpath:/ca/uhn/hapi/fhir/docs/ConsentInterceptors.java|service}} ``` + +## Performance and Privacy + +Filtering search results in `canSeeResource()` requires inspecting every resource during a search and editing the results. +This is slower than the normal path, and will prevent the reuse of the results from the search cache. +The `willSeeResource()` operation supports reusing cached search results, but removed resources may be 'visible' as holes in returned bundles. +Disabling `canSeeResource()` by returning `false` from `processCanSeeResource()` will enable the search cache. + + + diff --git a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md index 8699d76d5bf..e760b4a3dab 100644 --- a/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md +++ b/hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa/search.md @@ -22,6 +22,12 @@ Searching on Location.Position using `near` currently uses a box search, not a r The special `_filter` is only partially implemented. +### _pid + +The JPA server implements a non-standard special `_pid` which matches/sorts on the raw internal database id. +This sort is useful for imposing tie-breaking sort order in an efficient way. + +Note that this is an internal feature that may change or be removed in the future. Use with caution. diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialect.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialect.java index 0339ea69546..463e3a471cb 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialect.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialect.java @@ -96,7 +96,7 @@ public class HapiFhirHibernateJpaDialect extends HibernateJpaDialect { + makeErrorMessage( messageToPrepend, "resourceIndexedCompositeStringUniqueConstraintFailure")); } - if (constraintName.contains(ResourceTable.IDX_RES_FHIR_ID)) { + if (constraintName.contains(ResourceTable.IDX_RES_TYPE_FHIR_ID)) { throw new ResourceVersionConflictException( Msg.code(825) + makeErrorMessage(messageToPrepend, "forcedIdConstraintFailure")); } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/JpaConfig.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/JpaConfig.java index 3a44a5f8ed2..c6a27252a95 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/JpaConfig.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/JpaConfig.java @@ -114,7 +114,6 @@ import ca.uhn.fhir.jpa.search.builder.predicate.ComboNonUniqueSearchParameterPre import ca.uhn.fhir.jpa.search.builder.predicate.ComboUniqueSearchParameterPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.CoordsPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.DatePredicateBuilder; -import ca.uhn.fhir.jpa.search.builder.predicate.ForcedIdPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.NumberPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.QuantityNormalizedPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.QuantityPredicateBuilder; @@ -613,12 +612,6 @@ public class JpaConfig { return new DatePredicateBuilder(theSearchBuilder); } - @Bean - @Scope("prototype") - public ForcedIdPredicateBuilder newForcedIdPredicateBuilder(SearchQueryBuilder theSearchBuilder) { - return new ForcedIdPredicateBuilder(theSearchBuilder); - } - @Bean @Scope("prototype") public NumberPredicateBuilder newNumberPredicateBuilder(SearchQueryBuilder theSearchBuilder) { diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/IMdmLinkJpaRepository.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/IMdmLinkJpaRepository.java index 363f2ae9b28..1962fa63176 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/IMdmLinkJpaRepository.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/IMdmLinkJpaRepository.java @@ -53,7 +53,7 @@ public interface IMdmLinkJpaRepository @Modifying @Query( value = - "DELETE FROM MPI_LINK_AUD f WHERE GOLDEN_RESOURCE_PID IN (:goldenPids) OR TARGET_PID IN (:goldenPids)", + "DELETE FROM MPI_LINK_AUD WHERE GOLDEN_RESOURCE_PID IN (:goldenPids) OR TARGET_PID IN (:goldenPids)", nativeQuery = true) void deleteLinksHistoryWithAnyReferenceToPids(@Param("goldenPids") List theResourcePids); diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchDao.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchDao.java index 6a9fa7fd5ab..35bb509b69a 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchDao.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchDao.java @@ -20,8 +20,7 @@ package ca.uhn.fhir.jpa.dao.data; import ca.uhn.fhir.jpa.entity.Search; -import org.springframework.data.domain.Pageable; -import org.springframework.data.domain.Slice; +import com.google.errorprone.annotations.CanIgnoreReturnValue; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.data.jpa.repository.Modifying; import org.springframework.data.jpa.repository.Query; @@ -30,6 +29,8 @@ import org.springframework.data.repository.query.Param; import java.util.Collection; import java.util.Date; import java.util.Optional; +import java.util.Set; +import java.util.stream.Stream; public interface ISearchDao extends JpaRepository, IHapiFhirJpaRepository { @@ -38,10 +39,12 @@ public interface ISearchDao extends JpaRepository, IHapiFhirJpaRep @Query( "SELECT s.myId FROM Search s WHERE (s.myCreated < :cutoff) AND (s.myExpiryOrNull IS NULL OR s.myExpiryOrNull < :now) AND (s.myDeleted IS NULL OR s.myDeleted = FALSE)") - Slice findWhereCreatedBefore(@Param("cutoff") Date theCutoff, @Param("now") Date theNow, Pageable thePage); + Stream findWhereCreatedBefore(@Param("cutoff") Date theCutoff, @Param("now") Date theNow); - @Query("SELECT s.myId FROM Search s WHERE s.myDeleted = TRUE") - Slice findDeleted(Pageable thePage); + @Query("SELECT new ca.uhn.fhir.jpa.dao.data.SearchIdAndResultSize(" + "s.myId, " + + "(select max(sr.myOrder) as maxOrder from SearchResult sr where sr.mySearchPid = s.myId)) " + + "FROM Search s WHERE s.myDeleted = TRUE") + Stream findDeleted(); @Query( "SELECT s FROM Search s WHERE s.myResourceType = :type AND s.mySearchQueryStringHash = :hash AND (s.myCreated > :cutoff) AND s.myDeleted = FALSE AND s.myStatus <> 'FAILED'") @@ -54,10 +57,15 @@ public interface ISearchDao extends JpaRepository, IHapiFhirJpaRep int countDeleted(); @Modifying - @Query("UPDATE Search s SET s.myDeleted = :deleted WHERE s.myId = :pid") - void updateDeleted(@Param("pid") Long thePid, @Param("deleted") boolean theDeleted); + @Query("UPDATE Search s SET s.myDeleted = :deleted WHERE s.myId in (:pids)") + @CanIgnoreReturnValue + int updateDeleted(@Param("pids") Set thePid, @Param("deleted") boolean theDeleted); @Modifying @Query("DELETE FROM Search s WHERE s.myId = :pid") void deleteByPid(@Param("pid") Long theId); + + @Modifying + @Query("DELETE FROM Search s WHERE s.myId in (:pids)") + void deleteByPids(@Param("pids") Collection theSearchToDelete); } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchIncludeDao.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchIncludeDao.java index 9312d300f0a..776b8a94faf 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchIncludeDao.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchIncludeDao.java @@ -20,14 +20,18 @@ package ca.uhn.fhir.jpa.dao.data; import ca.uhn.fhir.jpa.entity.SearchInclude; +import com.google.errorprone.annotations.CanIgnoreReturnValue; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.data.jpa.repository.Modifying; import org.springframework.data.jpa.repository.Query; import org.springframework.data.repository.query.Param; +import java.util.Collection; + public interface ISearchIncludeDao extends JpaRepository, IHapiFhirJpaRepository { @Modifying - @Query(value = "DELETE FROM SearchInclude r WHERE r.mySearchPid = :search") - void deleteForSearch(@Param("search") Long theSearchPid); + @Query(value = "DELETE FROM SearchInclude r WHERE r.mySearchPid in (:search)") + @CanIgnoreReturnValue + int deleteForSearch(@Param("search") Collection theSearchPid); } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchResultDao.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchResultDao.java index b16a1d99dbf..eb6a4f89474 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchResultDao.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/ISearchResultDao.java @@ -20,6 +20,7 @@ package ca.uhn.fhir.jpa.dao.data; import ca.uhn.fhir.jpa.entity.SearchResult; +import com.google.errorprone.annotations.CanIgnoreReturnValue; import org.springframework.data.domain.Pageable; import org.springframework.data.domain.Slice; import org.springframework.data.jpa.repository.JpaRepository; @@ -27,6 +28,7 @@ import org.springframework.data.jpa.repository.Modifying; import org.springframework.data.jpa.repository.Query; import org.springframework.data.repository.query.Param; +import java.util.Collection; import java.util.List; public interface ISearchResultDao extends JpaRepository, IHapiFhirJpaRepository { @@ -37,12 +39,19 @@ public interface ISearchResultDao extends JpaRepository, IHa @Query(value = "SELECT r.myResourcePid FROM SearchResult r WHERE r.mySearchPid = :search") List findWithSearchPidOrderIndependent(@Param("search") Long theSearchPid); - @Query(value = "SELECT r.myId FROM SearchResult r WHERE r.mySearchPid = :search") - Slice findForSearch(Pageable thePage, @Param("search") Long theSearchPid); + @Modifying + @Query("DELETE FROM SearchResult s WHERE s.mySearchPid IN :searchIds") + @CanIgnoreReturnValue + int deleteBySearchIds(@Param("searchIds") Collection theSearchIds); @Modifying - @Query("DELETE FROM SearchResult s WHERE s.myId IN :ids") - void deleteByIds(@Param("ids") List theContent); + @Query( + "DELETE FROM SearchResult s WHERE s.mySearchPid = :searchId and s.myOrder >= :rangeStart and s.myOrder <= :rangeEnd") + @CanIgnoreReturnValue + int deleteBySearchIdInRange( + @Param("searchId") Long theSearchId, + @Param("rangeStart") int theRangeStart, + @Param("rangeEnd") int theRangeEnd); @Query("SELECT count(r) FROM SearchResult r WHERE r.mySearchPid = :search") int countForSearch(@Param("search") Long theSearchPid); diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/SearchIdAndResultSize.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/SearchIdAndResultSize.java new file mode 100644 index 00000000000..8c6d822de7f --- /dev/null +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/data/SearchIdAndResultSize.java @@ -0,0 +1,37 @@ +/*- + * #%L + * HAPI FHIR JPA Server + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.jpa.dao.data; + +import java.util.Objects; + +/** + * Record for search result returning the PK of a Search, and the number of associated SearchResults + */ +public class SearchIdAndResultSize { + /** Search PK */ + public final long searchId; + /** Number of SearchResults attached */ + public final int size; + + public SearchIdAndResultSize(long theSearchId, Integer theSize) { + searchId = theSearchId; + size = Objects.requireNonNullElse(theSize, 0); + } +} diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/SearchResult.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/SearchResult.java index 5dc807554eb..7a559a05988 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/SearchResult.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/SearchResult.java @@ -37,21 +37,22 @@ public class SearchResult implements Serializable { private static final long serialVersionUID = 1L; + @Deprecated(since = "6.10", forRemoval = true) // migrating to composite PK on searchPid,Order @GeneratedValue(strategy = GenerationType.AUTO, generator = "SEQ_SEARCH_RES") @SequenceGenerator(name = "SEQ_SEARCH_RES", sequenceName = "SEQ_SEARCH_RES") @Id @Column(name = "PID") private Long myId; - @Column(name = "SEARCH_ORDER", nullable = false, insertable = true, updatable = false) + @Column(name = "SEARCH_PID", insertable = true, updatable = false, nullable = false) + private Long mySearchPid; + + @Column(name = "SEARCH_ORDER", insertable = true, updatable = false, nullable = false) private int myOrder; @Column(name = "RESOURCE_PID", insertable = true, updatable = false, nullable = false) private Long myResourcePid; - @Column(name = "SEARCH_PID", insertable = true, updatable = false, nullable = false) - private Long mySearchPid; - /** * Constructor */ diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java index 444549e8968..a0f718dceb8 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/migrate/tasks/HapiFhirJpaMigrationTasks.java @@ -118,12 +118,19 @@ public class HapiFhirJpaMigrationTasks extends BaseMigrationTasks { Builder.BuilderWithTableName hfjResource = version.onTable("HFJ_RESOURCE"); hfjResource.modifyColumn("20231018.2", "FHIR_ID").nonNullable(); + + hfjResource.dropIndex("20231027.1", "IDX_RES_FHIR_ID"); hfjResource - .addIndex("20231018.3", "IDX_RES_FHIR_ID") + .addIndex("20231027.2", "IDX_RES_TYPE_FHIR_ID") .unique(true) .online(true) - .includeColumns("RES_ID") - .withColumns("FHIR_ID", "RES_TYPE"); + // include res_id and our deleted flag so we can satisfy Observation?_sort=_id from the index on + // platforms that support it. + .includeColumns("RES_ID, RES_DELETED_AT") + .withColumns("RES_TYPE", "FHIR_ID"); + + // For resolving references that don't supply the type. + hfjResource.addIndex("20231027.3", "IDX_RES_FHIR_ID").unique(false).withColumns("FHIR_ID"); } protected void init680() { diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImpl.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImpl.java index 723450da4d0..bc7fe8f5f34 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImpl.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImpl.java @@ -40,6 +40,7 @@ import ca.uhn.fhir.jpa.searchparam.SearchParameterMap; import ca.uhn.fhir.jpa.searchparam.registry.ISearchParamRegistryController; import ca.uhn.fhir.jpa.searchparam.util.SearchParameterHelper; import ca.uhn.fhir.rest.api.server.IBundleProvider; +import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.api.server.SystemRequestDetails; import ca.uhn.fhir.rest.param.StringParam; import ca.uhn.fhir.rest.param.TokenParam; @@ -64,12 +65,13 @@ import org.springframework.beans.factory.annotation.Autowired; import java.io.IOException; import java.util.Collection; +import java.util.HashSet; import java.util.List; import java.util.Optional; -import javax.annotation.Nonnull; import javax.annotation.PostConstruct; import static ca.uhn.fhir.jpa.packages.util.PackageUtils.DEFAULT_INSTALL_TYPES; +import static ca.uhn.fhir.util.SearchParameterUtil.getBaseAsStrings; import static org.apache.commons.lang3.StringUtils.defaultString; import static org.apache.commons.lang3.StringUtils.isBlank; @@ -251,7 +253,7 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { for (IBaseResource next : resources) { try { next = isStructureDefinitionWithoutSnapshot(next) ? generateSnapshot(next) : next; - create(next, theInstallationSpec, theOutcome); + install(next, theInstallationSpec, theOutcome); } catch (Exception e) { ourLog.warn( "Failed to upload resource of type {} with ID {} - Error: {}", @@ -345,83 +347,42 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { * ============================= Utility methods =============================== */ @VisibleForTesting - void create( + void install( IBaseResource theResource, PackageInstallationSpec theInstallationSpec, PackageInstallOutcomeJson theOutcome) { - IFhirResourceDao dao = myDaoRegistry.getResourceDao(theResource.getClass()); - SearchParameterMap map = createSearchParameterMapFor(theResource); - IBundleProvider searchResult = searchResource(dao, map); - if (validForUpload(theResource)) { - if (searchResult.isEmpty()) { - ourLog.info("Creating new resource matching {}", map.toNormalizedQueryString(myFhirContext)); - theOutcome.incrementResourcesInstalled(myFhirContext.getResourceType(theResource)); - - IIdType id = theResource.getIdElement(); - - if (id.isEmpty()) { - createResource(dao, theResource); - ourLog.info("Created resource with new id"); - } else { - if (id.isIdPartValidLong()) { - String newIdPart = "npm-" + id.getIdPart(); - id.setParts(id.getBaseUrl(), id.getResourceType(), newIdPart, id.getVersionIdPart()); - } - - try { - updateResource(dao, theResource); - - ourLog.info("Created resource with existing id"); - } catch (ResourceVersionConflictException exception) { - final Optional optResource = readResourceById(dao, id); - - final String existingResourceUrlOrNull = optResource - .filter(MetadataResource.class::isInstance) - .map(MetadataResource.class::cast) - .map(MetadataResource::getUrl) - .orElse(null); - final String newResourceUrlOrNull = (theResource instanceof MetadataResource) - ? ((MetadataResource) theResource).getUrl() - : null; - - ourLog.error( - "Version conflict error: This is possibly due to a collision between ValueSets from different IGs that are coincidentally using the same resource ID: [{}] and new resource URL: [{}], with the exisitng resource having URL: [{}]. Ignoring this update and continuing: The first IG wins. ", - id.getIdPart(), - newResourceUrlOrNull, - existingResourceUrlOrNull, - exception); - } - } - } else { - if (theInstallationSpec.isReloadExisting()) { - ourLog.info("Updating existing resource matching {}", map.toNormalizedQueryString(myFhirContext)); - theResource.setId(searchResult - .getResources(0, 1) - .get(0) - .getIdElement() - .toUnqualifiedVersionless()); - DaoMethodOutcome outcome = updateResource(dao, theResource); - if (!outcome.isNop()) { - theOutcome.incrementResourcesInstalled(myFhirContext.getResourceType(theResource)); - } - } else { - ourLog.info( - "Skipping update of existing resource matching {}", - map.toNormalizedQueryString(myFhirContext)); - } - } - } else { + if (!validForUpload(theResource)) { ourLog.warn( "Failed to upload resource of type {} with ID {} - Error: Resource failed validation", theResource.fhirType(), theResource.getIdElement().getValue()); + return; + } + + IFhirResourceDao dao = myDaoRegistry.getResourceDao(theResource.getClass()); + SearchParameterMap map = createSearchParameterMapFor(theResource); + IBundleProvider searchResult = searchResource(dao, map); + + String resourceQuery = map.toNormalizedQueryString(myFhirContext); + if (!searchResult.isEmpty() && !theInstallationSpec.isReloadExisting()) { + ourLog.info("Skipping update of existing resource matching {}", resourceQuery); + return; + } + if (!searchResult.isEmpty()) { + ourLog.info("Updating existing resource matching {}", resourceQuery); + } + IBaseResource existingResource = + !searchResult.isEmpty() ? searchResult.getResources(0, 1).get(0) : null; + boolean isInstalled = createOrUpdateResource(dao, theResource, existingResource); + if (isInstalled) { + theOutcome.incrementResourcesInstalled(myFhirContext.getResourceType(theResource)); } } private Optional readResourceById(IFhirResourceDao dao, IIdType id) { try { - return Optional.ofNullable(dao.read(id.toUnqualifiedVersionless(), newSystemRequestDetails())); + return Optional.ofNullable(dao.read(id.toUnqualifiedVersionless(), createRequestDetails())); } catch (Exception exception) { // ignore because we're running this query to help build the log @@ -432,30 +393,112 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { } private IBundleProvider searchResource(IFhirResourceDao theDao, SearchParameterMap theMap) { - return theDao.search(theMap, newSystemRequestDetails()); + return theDao.search(theMap, createRequestDetails()); } - @Nonnull - private SystemRequestDetails newSystemRequestDetails() { - return new SystemRequestDetails().setRequestPartitionId(RequestPartitionId.defaultPartition()); - } + protected boolean createOrUpdateResource( + IFhirResourceDao theDao, IBaseResource theResource, IBaseResource theExistingResource) { + final IIdType id = theResource.getIdElement(); - private void createResource(IFhirResourceDao theDao, IBaseResource theResource) { - if (myPartitionSettings.isPartitioningEnabled()) { - SystemRequestDetails requestDetails = newSystemRequestDetails(); - theDao.create(theResource, requestDetails); - } else { - theDao.create(theResource); + if (theExistingResource == null && id.isEmpty()) { + ourLog.debug("Install resource without id will be created"); + theDao.create(theResource, createRequestDetails()); + return true; } + + if (theExistingResource == null && !id.isEmpty() && id.isIdPartValidLong()) { + String newIdPart = "npm-" + id.getIdPart(); + id.setParts(id.getBaseUrl(), id.getResourceType(), newIdPart, id.getVersionIdPart()); + } + + boolean isExistingUpdated = updateExistingResourceIfNecessary(theDao, theResource, theExistingResource); + boolean shouldOverrideId = theExistingResource != null && !isExistingUpdated; + + if (shouldOverrideId) { + ourLog.debug( + "Existing resource {} will be overridden with installed resource {}", + theExistingResource.getIdElement(), + id); + theResource.setId(theExistingResource.getIdElement().toUnqualifiedVersionless()); + } else { + ourLog.debug("Install resource {} will be created", id); + } + + DaoMethodOutcome outcome = updateResource(theDao, theResource); + return outcome != null && !outcome.isNop(); } - DaoMethodOutcome updateResource(IFhirResourceDao theDao, IBaseResource theResource) { - if (myPartitionSettings.isPartitioningEnabled()) { - SystemRequestDetails requestDetails = newSystemRequestDetails(); - return theDao.update(theResource, requestDetails); - } else { - return theDao.update(theResource, new SystemRequestDetails()); + private boolean updateExistingResourceIfNecessary( + IFhirResourceDao theDao, IBaseResource theResource, IBaseResource theExistingResource) { + if (!"SearchParameter".equals(theResource.getClass().getSimpleName())) { + return false; } + if (theExistingResource == null) { + return false; + } + if (theExistingResource + .getIdElement() + .getIdPart() + .equals(theResource.getIdElement().getIdPart())) { + return false; + } + Collection remainingBaseList = new HashSet<>(getBaseAsStrings(myFhirContext, theExistingResource)); + remainingBaseList.removeAll(getBaseAsStrings(myFhirContext, theResource)); + if (remainingBaseList.isEmpty()) { + return false; + } + myFhirContext + .getResourceDefinition(theExistingResource) + .getChildByName("base") + .getMutator() + .setValue(theExistingResource, null); + + for (String baseResourceName : remainingBaseList) { + myFhirContext.newTerser().addElement(theExistingResource, "base", baseResourceName); + } + ourLog.info( + "Existing SearchParameter {} will be updated with base {}", + theExistingResource.getIdElement().getIdPart(), + remainingBaseList); + updateResource(theDao, theExistingResource); + return true; + } + + private DaoMethodOutcome updateResource(IFhirResourceDao theDao, IBaseResource theResource) { + DaoMethodOutcome outcome = null; + + IIdType id = theResource.getIdElement(); + RequestDetails requestDetails = createRequestDetails(); + + try { + outcome = theDao.update(theResource, requestDetails); + } catch (ResourceVersionConflictException exception) { + final Optional optResource = readResourceById(theDao, id); + + final String existingResourceUrlOrNull = optResource + .filter(MetadataResource.class::isInstance) + .map(MetadataResource.class::cast) + .map(MetadataResource::getUrl) + .orElse(null); + final String newResourceUrlOrNull = + (theResource instanceof MetadataResource) ? ((MetadataResource) theResource).getUrl() : null; + + ourLog.error( + "Version conflict error: This is possibly due to a collision between ValueSets from different IGs that are coincidentally using the same resource ID: [{}] and new resource URL: [{}], with the exisitng resource having URL: [{}]. Ignoring this update and continuing: The first IG wins. ", + id.getIdPart(), + newResourceUrlOrNull, + existingResourceUrlOrNull, + exception); + } + return outcome; + } + + private RequestDetails createRequestDetails() { + SystemRequestDetails requestDetails = new SystemRequestDetails(); + if (myPartitionSettings.isPartitioningEnabled()) { + requestDetails.setRequestPartitionId(RequestPartitionId.defaultPartition()); + } + return requestDetails; } boolean validForUpload(IBaseResource theResource) { @@ -480,7 +523,7 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { return false; } - if (SearchParameterUtil.getBaseAsStrings(myFhirContext, theResource).isEmpty()) { + if (getBaseAsStrings(myFhirContext, theResource).isEmpty()) { ourLog.warn( "Failed to validate resource of type {} with url {} - Error: Resource base is empty", theResource.fhirType(), @@ -560,20 +603,21 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { } } - private SearchParameterMap createSearchParameterMapFor(IBaseResource resource) { - if (resource.getClass().getSimpleName().equals("NamingSystem")) { - String uniqueId = extractUniqeIdFromNamingSystem(resource); + private SearchParameterMap createSearchParameterMapFor(IBaseResource theResource) { + String resourceType = theResource.getClass().getSimpleName(); + if ("NamingSystem".equals(resourceType)) { + String uniqueId = extractUniqeIdFromNamingSystem(theResource); return SearchParameterMap.newSynchronous().add("value", new StringParam(uniqueId).setExact(true)); - } else if (resource.getClass().getSimpleName().equals("Subscription")) { - String id = extractIdFromSubscription(resource); + } else if ("Subscription".equals(resourceType)) { + String id = extractSimpleValue(theResource, "id"); return SearchParameterMap.newSynchronous().add("_id", new TokenParam(id)); - } else if (resource.getClass().getSimpleName().equals("SearchParameter")) { - return buildSearchParameterMapForSearchParameter(resource); - } else if (resourceHasUrlElement(resource)) { - String url = extractUniqueUrlFromMetadataResource(resource); + } else if ("SearchParameter".equals(resourceType)) { + return buildSearchParameterMapForSearchParameter(theResource); + } else if (resourceHasUrlElement(theResource)) { + String url = extractSimpleValue(theResource, "url"); return SearchParameterMap.newSynchronous().add("url", new UriParam(url)); } else { - TokenParam identifierToken = extractIdentifierFromOtherResourceTypes(resource); + TokenParam identifierToken = extractIdentifierFromOtherResourceTypes(theResource); return SearchParameterMap.newSynchronous().add("identifier", identifierToken); } } @@ -593,7 +637,7 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { } if (resourceHasUrlElement(theResource)) { - String url = extractUniqueUrlFromMetadataResource(theResource); + String url = extractSimpleValue(theResource, "url"); return SearchParameterMap.newSynchronous().add("url", new UriParam(url)); } else { TokenParam identifierToken = extractIdentifierFromOtherResourceTypes(theResource); @@ -601,32 +645,17 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { } } - private String extractUniqeIdFromNamingSystem(IBaseResource resource) { - FhirTerser terser = myFhirContext.newTerser(); - IBase uniqueIdComponent = (IBase) terser.getSingleValueOrNull(resource, "uniqueId"); + private String extractUniqeIdFromNamingSystem(IBaseResource theResource) { + IBase uniqueIdComponent = (IBase) extractValue(theResource, "uniqueId"); if (uniqueIdComponent == null) { throw new ImplementationGuideInstallationException( Msg.code(1291) + "NamingSystem does not have uniqueId component."); } - IPrimitiveType asPrimitiveType = (IPrimitiveType) terser.getSingleValueOrNull(uniqueIdComponent, "value"); - return (String) asPrimitiveType.getValue(); + return extractSimpleValue(uniqueIdComponent, "value"); } - private String extractIdFromSubscription(IBaseResource resource) { - FhirTerser terser = myFhirContext.newTerser(); - IPrimitiveType asPrimitiveType = (IPrimitiveType) terser.getSingleValueOrNull(resource, "id"); - return (String) asPrimitiveType.getValue(); - } - - private String extractUniqueUrlFromMetadataResource(IBaseResource resource) { - FhirTerser terser = myFhirContext.newTerser(); - IPrimitiveType asPrimitiveType = (IPrimitiveType) terser.getSingleValueOrNull(resource, "url"); - return (String) asPrimitiveType.getValue(); - } - - private TokenParam extractIdentifierFromOtherResourceTypes(IBaseResource resource) { - FhirTerser terser = myFhirContext.newTerser(); - Identifier identifier = (Identifier) terser.getSingleValueOrNull(resource, "identifier"); + private TokenParam extractIdentifierFromOtherResourceTypes(IBaseResource theResource) { + Identifier identifier = (Identifier) extractValue(theResource, "identifier"); if (identifier != null) { return new TokenParam(identifier.getSystem(), identifier.getValue()); } else { @@ -635,6 +664,15 @@ public class PackageInstallerSvcImpl implements IPackageInstallerSvc { } } + private Object extractValue(IBase theResource, String thePath) { + return myFhirContext.newTerser().getSingleValueOrNull(theResource, thePath); + } + + private String extractSimpleValue(IBase theResource, String thePath) { + IPrimitiveType asPrimitiveType = (IPrimitiveType) extractValue(theResource, thePath); + return (String) asPrimitiveType.getValue(); + } + private boolean resourceHasUrlElement(IBaseResource resource) { BaseRuntimeElementDefinition def = myFhirContext.getElementDefinition(resource.getClass()); if (!(def instanceof BaseRuntimeElementCompositeDefinition)) { diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/StaleSearchDeletingSvcImpl.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/StaleSearchDeletingSvcImpl.java index 700dfa9484e..b37b0be204d 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/StaleSearchDeletingSvcImpl.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/StaleSearchDeletingSvcImpl.java @@ -25,12 +25,16 @@ import ca.uhn.fhir.jpa.model.sched.HapiJob; import ca.uhn.fhir.jpa.model.sched.IHasScheduledJobs; import ca.uhn.fhir.jpa.model.sched.ISchedulerService; import ca.uhn.fhir.jpa.model.sched.ScheduledJobDefinition; +import ca.uhn.fhir.jpa.search.cache.DatabaseSearchCacheSvcImpl; import ca.uhn.fhir.jpa.search.cache.ISearchCacheSvc; import org.quartz.JobExecutionContext; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; +import java.time.Instant; +import java.time.temporal.ChronoUnit; + import static ca.uhn.fhir.jpa.search.cache.DatabaseSearchCacheSvcImpl.SEARCH_CLEANUP_JOB_INTERVAL_MILLIS; /** @@ -42,7 +46,6 @@ import static ca.uhn.fhir.jpa.search.cache.DatabaseSearchCacheSvcImpl.SEARCH_CLE // in Smile. // public class StaleSearchDeletingSvcImpl implements IStaleSearchDeletingSvc, IHasScheduledJobs { - private static final org.slf4j.Logger ourLog = org.slf4j.LoggerFactory.getLogger(StaleSearchDeletingSvcImpl.class); @Autowired private JpaStorageSettings myStorageSettings; @@ -53,7 +56,16 @@ public class StaleSearchDeletingSvcImpl implements IStaleSearchDeletingSvc, IHas @Override @Transactional(propagation = Propagation.NEVER) public void pollForStaleSearchesAndDeleteThem() { - mySearchCacheSvc.pollForStaleSearchesAndDeleteThem(RequestPartitionId.allPartitions()); + mySearchCacheSvc.pollForStaleSearchesAndDeleteThem(RequestPartitionId.allPartitions(), getDeadline()); + } + + /** + * Calculate a deadline to finish before the next scheduled run. + */ + protected Instant getDeadline() { + return Instant.ofEpochMilli(DatabaseSearchCacheSvcImpl.now()) + // target a 90% duty-cycle to avoid confusing quartz + .plus((long) (SEARCH_CLEANUP_JOB_INTERVAL_MILLIS * 0.90), ChronoUnit.MILLIS); } @Override diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/QueryStack.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/QueryStack.java index 16ed33dd153..1075761f352 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/QueryStack.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/QueryStack.java @@ -43,7 +43,6 @@ import ca.uhn.fhir.jpa.search.builder.predicate.ComboNonUniqueSearchParameterPre import ca.uhn.fhir.jpa.search.builder.predicate.ComboUniqueSearchParameterPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.CoordsPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.DatePredicateBuilder; -import ca.uhn.fhir.jpa.search.builder.predicate.ForcedIdPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.ICanMakeMissingParamPredicate; import ca.uhn.fhir.jpa.search.builder.predicate.NumberPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.ParsedLocationParam; @@ -69,10 +68,10 @@ import ca.uhn.fhir.parser.DataFormatException; import ca.uhn.fhir.rest.api.Constants; import ca.uhn.fhir.rest.api.QualifiedParamList; import ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum; -import ca.uhn.fhir.rest.api.SearchContainedModeEnum; import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.param.CompositeParam; import ca.uhn.fhir.rest.param.DateParam; +import ca.uhn.fhir.rest.param.DateRangeParam; import ca.uhn.fhir.rest.param.HasParam; import ca.uhn.fhir.rest.param.NumberParam; import ca.uhn.fhir.rest.param.QuantityParam; @@ -95,7 +94,6 @@ import com.healthmarketscience.sqlbuilder.ComboCondition; import com.healthmarketscience.sqlbuilder.Condition; import com.healthmarketscience.sqlbuilder.Expression; import com.healthmarketscience.sqlbuilder.InCondition; -import com.healthmarketscience.sqlbuilder.OrderObject; import com.healthmarketscience.sqlbuilder.SelectQuery; import com.healthmarketscience.sqlbuilder.SetOperationQuery; import com.healthmarketscience.sqlbuilder.Subquery; @@ -123,6 +121,7 @@ import java.util.function.Supplier; import java.util.stream.Collectors; import javax.annotation.Nullable; +import static ca.uhn.fhir.jpa.search.builder.QueryStack.SearchForIdsParams.with; import static ca.uhn.fhir.jpa.util.QueryParameterUtils.fromOperation; import static ca.uhn.fhir.jpa.util.QueryParameterUtils.getChainedPart; import static ca.uhn.fhir.jpa.util.QueryParameterUtils.getParamNameWithPrefix; @@ -275,16 +274,21 @@ public class QueryStack { } public void addSortOnResourceId(boolean theAscending) { + ResourceTablePredicateBuilder resourceTablePredicateBuilder; BaseJoiningPredicateBuilder firstPredicateBuilder = mySqlBuilder.getOrCreateFirstPredicateBuilder(); - ForcedIdPredicateBuilder sortPredicateBuilder = - mySqlBuilder.addForcedIdPredicateBuilder(firstPredicateBuilder.getResourceIdColumn()); - if (!theAscending) { - mySqlBuilder.addSortString( - sortPredicateBuilder.getColumnForcedId(), false, OrderObject.NullOrder.FIRST, myUseAggregate); + if (firstPredicateBuilder instanceof ResourceTablePredicateBuilder) { + resourceTablePredicateBuilder = (ResourceTablePredicateBuilder) firstPredicateBuilder; } else { - mySqlBuilder.addSortString(sortPredicateBuilder.getColumnForcedId(), true, myUseAggregate); + resourceTablePredicateBuilder = + mySqlBuilder.addResourceTablePredicateBuilder(firstPredicateBuilder.getResourceIdColumn()); } - mySqlBuilder.addSortNumeric(firstPredicateBuilder.getResourceIdColumn(), theAscending, myUseAggregate); + mySqlBuilder.addSortString(resourceTablePredicateBuilder.getColumnFhirId(), theAscending, myUseAggregate); + } + + /** Sort on RES_ID -- used to break ties for reliable sort */ + public void addSortOnResourcePID(boolean theAscending) { + BaseJoiningPredicateBuilder predicateBuilder = mySqlBuilder.getOrCreateFirstPredicateBuilder(); + mySqlBuilder.addSortString(predicateBuilder.getResourceIdColumn(), theAscending); } public void addSortOnResourceLink( @@ -1107,7 +1111,7 @@ public class QueryStack { if (paramName.startsWith("_has:")) { - ourLog.trace("Handing double _has query: {}", paramName); + ourLog.trace("Handling double _has query: {}", paramName); String qualifier = paramName.substring(4); for (IQueryParameterType next : nextOrList) { @@ -1160,26 +1164,30 @@ public class QueryStack { parameterName = parameterName.substring(0, colonIndex); } - ResourceLinkPredicateBuilder join = + ResourceLinkPredicateBuilder resourceLinkTableJoin = mySqlBuilder.addReferencePredicateBuilderReversed(this, theSourceJoinColumn); - Condition partitionPredicate = join.createPartitionIdPredicate(theRequestPartitionId); + Condition partitionPredicate = resourceLinkTableJoin.createPartitionIdPredicate(theRequestPartitionId); - List paths = join.createResourceLinkPaths(targetResourceType, paramReference, new ArrayList<>()); + List paths = resourceLinkTableJoin.createResourceLinkPaths( + targetResourceType, paramReference, new ArrayList<>()); if (CollectionUtils.isEmpty(paths)) { throw new InvalidRequestException(Msg.code(2305) + "Reference field does not exist: " + paramReference); } + Condition typePredicate = BinaryCondition.equalTo( - join.getColumnTargetResourceType(), mySqlBuilder.generatePlaceholder(theResourceType)); - Condition pathPredicate = - toEqualToOrInPredicate(join.getColumnSourcePath(), mySqlBuilder.generatePlaceholders(paths)); - Condition linkedPredicate = searchForIdsWithAndOr( - join.getColumnSrcResourceId(), - targetResourceType, - parameterName, - Collections.singletonList(orValues), - theRequest, - theRequestPartitionId, - SearchContainedModeEnum.FALSE); + resourceLinkTableJoin.getColumnTargetResourceType(), + mySqlBuilder.generatePlaceholder(theResourceType)); + Condition pathPredicate = toEqualToOrInPredicate( + resourceLinkTableJoin.getColumnSourcePath(), mySqlBuilder.generatePlaceholders(paths)); + + Condition linkedPredicate = + searchForIdsWithAndOr(with().setSourceJoinColumn(resourceLinkTableJoin.getColumnSrcResourceId()) + .setResourceName(targetResourceType) + .setParamName(parameterName) + .setAndOrParams(Collections.singletonList(orValues)) + .setRequest(theRequest) + .setRequestPartitionId(theRequestPartitionId)); + andPredicates.add(toAndPredicate(partitionPredicate, pathPredicate, typePredicate, linkedPredicate)); } @@ -2270,57 +2278,125 @@ public class QueryStack { } @Nullable - public Condition searchForIdsWithAndOr( - @Nullable DbColumn theSourceJoinColumn, - String theResourceName, - String theParamName, - List> theAndOrParams, - RequestDetails theRequest, - RequestPartitionId theRequestPartitionId, - SearchContainedModeEnum theSearchContainedMode) { + public Condition searchForIdsWithAndOr(SearchForIdsParams theSearchForIdsParams) { - if (theAndOrParams.isEmpty()) { + if (theSearchForIdsParams.myAndOrParams.isEmpty()) { return null; } - switch (theParamName) { + switch (theSearchForIdsParams.myParamName) { case IAnyResource.SP_RES_ID: return createPredicateResourceId( - theSourceJoinColumn, theAndOrParams, theResourceName, null, theRequestPartitionId); + theSearchForIdsParams.mySourceJoinColumn, + theSearchForIdsParams.myAndOrParams, + theSearchForIdsParams.myResourceName, + null, + theSearchForIdsParams.myRequestPartitionId); + + case Constants.PARAM_PID: + return createPredicateResourcePID( + theSearchForIdsParams.mySourceJoinColumn, theSearchForIdsParams.myAndOrParams); case PARAM_HAS: return createPredicateHas( - theSourceJoinColumn, theResourceName, theAndOrParams, theRequest, theRequestPartitionId); + theSearchForIdsParams.mySourceJoinColumn, + theSearchForIdsParams.myResourceName, + theSearchForIdsParams.myAndOrParams, + theSearchForIdsParams.myRequest, + theSearchForIdsParams.myRequestPartitionId); case Constants.PARAM_TAG: case Constants.PARAM_PROFILE: case Constants.PARAM_SECURITY: if (myStorageSettings.getTagStorageMode() == JpaStorageSettings.TagStorageModeEnum.INLINE) { return createPredicateSearchParameter( - theSourceJoinColumn, - theResourceName, - theParamName, - theAndOrParams, - theRequest, - theRequestPartitionId); + theSearchForIdsParams.mySourceJoinColumn, + theSearchForIdsParams.myResourceName, + theSearchForIdsParams.myParamName, + theSearchForIdsParams.myAndOrParams, + theSearchForIdsParams.myRequest, + theSearchForIdsParams.myRequestPartitionId); } else { - return createPredicateTag(theSourceJoinColumn, theAndOrParams, theParamName, theRequestPartitionId); + return createPredicateTag( + theSearchForIdsParams.mySourceJoinColumn, + theSearchForIdsParams.myAndOrParams, + theSearchForIdsParams.myParamName, + theSearchForIdsParams.myRequestPartitionId); } case Constants.PARAM_SOURCE: - return createPredicateSourceForAndList(theSourceJoinColumn, theAndOrParams); + return createPredicateSourceForAndList( + theSearchForIdsParams.mySourceJoinColumn, theSearchForIdsParams.myAndOrParams); + + case Constants.PARAM_LASTUPDATED: + // this case statement handles a _lastUpdated query as part of a reverse search + // only (/Patient?_has:Encounter:patient:_lastUpdated=ge2023-10-24). + // performing a _lastUpdated query on a resource (/Patient?_lastUpdated=eq2023-10-24) + // is handled in {@link SearchBuilder#createChunkedQuery}. + return createReverseSearchPredicateLastUpdated( + theSearchForIdsParams.myAndOrParams, theSearchForIdsParams.mySourceJoinColumn); default: return createPredicateSearchParameter( - theSourceJoinColumn, - theResourceName, - theParamName, - theAndOrParams, - theRequest, - theRequestPartitionId); + theSearchForIdsParams.mySourceJoinColumn, + theSearchForIdsParams.myResourceName, + theSearchForIdsParams.myParamName, + theSearchForIdsParams.myAndOrParams, + theSearchForIdsParams.myRequest, + theSearchForIdsParams.myRequestPartitionId); } } + /** + * Raw match on RES_ID + */ + private Condition createPredicateResourcePID( + DbColumn theSourceJoinColumn, List> theAndOrParams) { + + DbColumn pidColumn = theSourceJoinColumn; + + if (pidColumn == null) { + BaseJoiningPredicateBuilder predicateBuilder = mySqlBuilder.getOrCreateFirstPredicateBuilder(); + pidColumn = predicateBuilder.getResourceIdColumn(); + } + + // we don't support any modifiers for now + Set pids = theAndOrParams.stream() + .map(orList -> orList.stream() + .map(v -> v.getValueAsQueryToken(myFhirContext)) + .map(Long::valueOf) + .collect(Collectors.toSet())) + .reduce(Sets::intersection) + .orElse(Set.of()); + + if (pids.isEmpty()) { + mySqlBuilder.setMatchNothing(); + return null; + } + + return toEqualToOrInPredicate(pidColumn, mySqlBuilder.generatePlaceholders(pids)); + } + + private Condition createReverseSearchPredicateLastUpdated( + List> theAndOrParams, DbColumn theSourceColumn) { + + ResourceTablePredicateBuilder resourceTableJoin = + mySqlBuilder.addResourceTablePredicateBuilder(theSourceColumn); + + List andPredicates = new ArrayList<>(theAndOrParams.size()); + + for (List aList : theAndOrParams) { + if (!aList.isEmpty()) { + DateParam dateParam = (DateParam) aList.get(0); + DateRangeParam dateRangeParam = new DateRangeParam(dateParam); + Condition aCondition = mySqlBuilder.addPredicateLastUpdated(dateRangeParam, resourceTableJoin); + andPredicates.add(aCondition); + } + } + + return toAndPredicate(andPredicates); + } + @Nullable private Condition createPredicateSearchParameter( @Nullable DbColumn theSourceJoinColumn, @@ -3020,4 +3096,82 @@ public class QueryStack { theParamDefinition, myOrValues, myLeafTarget, myLeafParamName, myLeafPathPrefix, myQualifiers); } } + + public static class SearchForIdsParams { + DbColumn mySourceJoinColumn; + String myResourceName; + String myParamName; + List> myAndOrParams; + RequestDetails myRequest; + RequestPartitionId myRequestPartitionId; + ResourceTablePredicateBuilder myResourceTablePredicateBuilder; + + public static SearchForIdsParams with() { + return new SearchForIdsParams(); + } + + public DbColumn getSourceJoinColumn() { + return mySourceJoinColumn; + } + + public SearchForIdsParams setSourceJoinColumn(DbColumn theSourceJoinColumn) { + mySourceJoinColumn = theSourceJoinColumn; + return this; + } + + public String getResourceName() { + return myResourceName; + } + + public SearchForIdsParams setResourceName(String theResourceName) { + myResourceName = theResourceName; + return this; + } + + public String getParamName() { + return myParamName; + } + + public SearchForIdsParams setParamName(String theParamName) { + myParamName = theParamName; + return this; + } + + public List> getAndOrParams() { + return myAndOrParams; + } + + public SearchForIdsParams setAndOrParams(List> theAndOrParams) { + myAndOrParams = theAndOrParams; + return this; + } + + public RequestDetails getRequest() { + return myRequest; + } + + public SearchForIdsParams setRequest(RequestDetails theRequest) { + myRequest = theRequest; + return this; + } + + public RequestPartitionId getRequestPartitionId() { + return myRequestPartitionId; + } + + public SearchForIdsParams setRequestPartitionId(RequestPartitionId theRequestPartitionId) { + myRequestPartitionId = theRequestPartitionId; + return this; + } + + public ResourceTablePredicateBuilder getResourceTablePredicateBuilder() { + return myResourceTablePredicateBuilder; + } + + public SearchForIdsParams setResourceTablePredicateBuilder( + ResourceTablePredicateBuilder theResourceTablePredicateBuilder) { + myResourceTablePredicateBuilder = theResourceTablePredicateBuilder; + return this; + } + } } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/SearchBuilder.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/SearchBuilder.java index 60d0a986031..a268f5f0d84 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/SearchBuilder.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/SearchBuilder.java @@ -131,6 +131,7 @@ import javax.persistence.criteria.CriteriaBuilder; import static ca.uhn.fhir.jpa.model.util.JpaConstants.UNDESIRED_RESOURCE_LINKAGES_FOR_EVERYTHING_ON_PATIENT_INSTANCE; import static ca.uhn.fhir.jpa.search.builder.QueryStack.LOCATION_POSITION; +import static ca.uhn.fhir.jpa.search.builder.QueryStack.SearchForIdsParams.with; import static org.apache.commons.lang3.StringUtils.defaultString; import static org.apache.commons.lang3.StringUtils.isBlank; import static org.apache.commons.lang3.StringUtils.isNotBlank; @@ -281,14 +282,11 @@ public class SearchBuilder implements ISearchBuilder { continue; } List> andOrParams = myParams.get(nextParamName); - Condition predicate = theQueryStack.searchForIdsWithAndOr( - null, - myResourceName, - nextParamName, - andOrParams, - theRequest, - myRequestPartitionId, - searchContainedMode); + Condition predicate = theQueryStack.searchForIdsWithAndOr(with().setResourceName(myResourceName) + .setParamName(nextParamName) + .setAndOrParams(andOrParams) + .setRequest(theRequest) + .setRequestPartitionId(myRequestPartitionId)); if (predicate != null) { theSearchSqlBuilder.addPredicate(predicate); } @@ -840,6 +838,10 @@ public class SearchBuilder implements ISearchBuilder { theQueryStack.addSortOnResourceId(ascending); + } else if (Constants.PARAM_PID.equals(theSort.getParamName())) { + + theQueryStack.addSortOnResourcePID(ascending); + } else if (Constants.PARAM_LASTUPDATED.equals(theSort.getParamName())) { theQueryStack.addSortOnLastUpdated(ascending); diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ForcedIdPredicateBuilder.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ForcedIdPredicateBuilder.java deleted file mode 100644 index 6f8d1e1b66c..00000000000 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ForcedIdPredicateBuilder.java +++ /dev/null @@ -1,51 +0,0 @@ -/*- - * #%L - * HAPI FHIR JPA Server - * %% - * Copyright (C) 2014 - 2023 Smile CDR, Inc. - * %% - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * #L% - */ -package ca.uhn.fhir.jpa.search.builder.predicate; - -import ca.uhn.fhir.jpa.search.builder.sql.SearchQueryBuilder; -import com.healthmarketscience.sqlbuilder.dbspec.basic.DbColumn; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -public class ForcedIdPredicateBuilder extends BaseJoiningPredicateBuilder { - - private static final Logger ourLog = LoggerFactory.getLogger(ForcedIdPredicateBuilder.class); - private final DbColumn myColumnResourceId; - private final DbColumn myColumnForcedId; - - /** - * Constructor - */ - public ForcedIdPredicateBuilder(SearchQueryBuilder theSearchSqlBuilder) { - super(theSearchSqlBuilder, theSearchSqlBuilder.addTable("HFJ_FORCED_ID")); - - myColumnResourceId = getTable().addColumn("RESOURCE_PID"); - myColumnForcedId = getTable().addColumn("FORCED_ID"); - } - - @Override - public DbColumn getResourceIdColumn() { - return myColumnResourceId; - } - - public DbColumn getColumnForcedId() { - return myColumnForcedId; - } -} diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceLinkPredicateBuilder.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceLinkPredicateBuilder.java index 5952f398c56..81b42f46429 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceLinkPredicateBuilder.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceLinkPredicateBuilder.java @@ -51,7 +51,6 @@ import ca.uhn.fhir.model.primitive.IdDt; import ca.uhn.fhir.parser.DataFormatException; import ca.uhn.fhir.rest.api.Constants; import ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum; -import ca.uhn.fhir.rest.api.SearchContainedModeEnum; import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.param.ReferenceParam; import ca.uhn.fhir.rest.param.TokenParam; @@ -87,6 +86,7 @@ import java.util.stream.Collectors; import javax.annotation.Nonnull; import javax.annotation.Nullable; +import static ca.uhn.fhir.jpa.search.builder.QueryStack.SearchForIdsParams.with; import static org.apache.commons.lang3.StringUtils.isBlank; import static org.apache.commons.lang3.StringUtils.trim; @@ -456,14 +456,13 @@ public class ResourceLinkPredicateBuilder extends BaseJoiningPredicateBuilder im List andPredicates = new ArrayList<>(); List> chainParamValues = Collections.singletonList(orValues); - andPredicates.add(childQueryFactory.searchForIdsWithAndOr( - myColumnTargetResourceId, - subResourceName, - chain, - chainParamValues, - theRequest, - theRequestPartitionId, - SearchContainedModeEnum.FALSE)); + andPredicates.add( + childQueryFactory.searchForIdsWithAndOr(with().setSourceJoinColumn(myColumnTargetResourceId) + .setResourceName(subResourceName) + .setParamName(chain) + .setAndOrParams(chainParamValues) + .setRequest(theRequest) + .setRequestPartitionId(theRequestPartitionId))); orPredicates.add(QueryParameterUtils.toAndPredicate(andPredicates)); } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceTablePredicateBuilder.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceTablePredicateBuilder.java index c9fb8845dd7..75ccddb1170 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceTablePredicateBuilder.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/predicate/ResourceTablePredicateBuilder.java @@ -19,6 +19,7 @@ */ package ca.uhn.fhir.jpa.search.builder.predicate; +import ca.uhn.fhir.jpa.model.entity.ResourceTable; import ca.uhn.fhir.jpa.search.builder.sql.SearchQueryBuilder; import ca.uhn.fhir.jpa.util.QueryParameterUtils; import com.healthmarketscience.sqlbuilder.BinaryCondition; @@ -35,6 +36,7 @@ public class ResourceTablePredicateBuilder extends BaseJoiningPredicateBuilder { private final DbColumn myColumnResType; private final DbColumn myColumnLastUpdated; private final DbColumn myColumnLanguage; + private final DbColumn myColumnFhirId; /** * Constructor @@ -42,10 +44,11 @@ public class ResourceTablePredicateBuilder extends BaseJoiningPredicateBuilder { public ResourceTablePredicateBuilder(SearchQueryBuilder theSearchSqlBuilder) { super(theSearchSqlBuilder, theSearchSqlBuilder.addTable("HFJ_RESOURCE")); myColumnResId = getTable().addColumn("RES_ID"); - myColumnResType = getTable().addColumn("RES_TYPE"); + myColumnResType = getTable().addColumn(ResourceTable.RES_TYPE); myColumnResDeletedAt = getTable().addColumn("RES_DELETED_AT"); myColumnLastUpdated = getTable().addColumn("RES_UPDATED"); myColumnLanguage = getTable().addColumn("RES_LANGUAGE"); + myColumnFhirId = getTable().addColumn(ResourceTable.FHIR_ID); } @Override @@ -77,4 +80,8 @@ public class ResourceTablePredicateBuilder extends BaseJoiningPredicateBuilder { public DbColumn getColumnLastUpdated() { return myColumnLastUpdated; } + + public DbColumn getColumnFhirId() { + return myColumnFhirId; + } } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SearchQueryBuilder.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SearchQueryBuilder.java index cb1d3a74bd4..e840bdec9d3 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SearchQueryBuilder.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SearchQueryBuilder.java @@ -32,7 +32,6 @@ import ca.uhn.fhir.jpa.search.builder.predicate.ComboNonUniqueSearchParameterPre import ca.uhn.fhir.jpa.search.builder.predicate.ComboUniqueSearchParameterPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.CoordsPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.DatePredicateBuilder; -import ca.uhn.fhir.jpa.search.builder.predicate.ForcedIdPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.NumberPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.QuantityNormalizedPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.QuantityPredicateBuilder; @@ -62,7 +61,6 @@ import com.healthmarketscience.sqlbuilder.dbspec.basic.DbJoin; import com.healthmarketscience.sqlbuilder.dbspec.basic.DbSchema; import com.healthmarketscience.sqlbuilder.dbspec.basic.DbSpec; import com.healthmarketscience.sqlbuilder.dbspec.basic.DbTable; -import org.apache.commons.lang3.Validate; import org.hibernate.dialect.Dialect; import org.hibernate.dialect.SQLServerDialect; import org.hibernate.dialect.pagination.AbstractLimitHandler; @@ -222,18 +220,6 @@ public class SearchQueryBuilder { return mySqlBuilderFactory.dateIndexTable(this); } - /** - * Add and return a predicate builder for selecting a forced ID. This is only intended for use with sorts so it can not - * be the root query. - */ - public ForcedIdPredicateBuilder addForcedIdPredicateBuilder(@Nonnull DbColumn theSourceJoinColumn) { - Validate.isTrue(theSourceJoinColumn != null); - - ForcedIdPredicateBuilder retVal = mySqlBuilderFactory.newForcedIdPredicateBuilder(this); - addTableForSorting(retVal, theSourceJoinColumn); - return retVal; - } - /** * Create, add and return a predicate builder (or a root query if no root query exists yet) for selecting on a NUMBER search parameter */ @@ -417,11 +403,6 @@ public class SearchQueryBuilder { addTable(thePredicateBuilder, theSourceJoinColumn, SelectQuery.JoinType.INNER); } - private void addTableForSorting( - BaseJoiningPredicateBuilder thePredicateBuilder, @Nullable DbColumn theSourceJoinColumn) { - addTable(thePredicateBuilder, theSourceJoinColumn, SelectQuery.JoinType.LEFT_OUTER); - } - private void addTable( BaseJoiningPredicateBuilder thePredicateBuilder, @Nullable DbColumn theSourceJoinColumn, @@ -699,15 +680,24 @@ public class SearchQueryBuilder { public ComboCondition addPredicateLastUpdated(DateRangeParam theDateRange) { ResourceTablePredicateBuilder resourceTableRoot = getOrCreateResourceTablePredicateBuilder(false); + return addPredicateLastUpdated(theDateRange, resourceTableRoot); + } + + public ComboCondition addPredicateLastUpdated( + DateRangeParam theDateRange, ResourceTablePredicateBuilder theResourceTablePredicateBuilder) { List conditions = new ArrayList<>(2); BinaryCondition condition; if (isNotEqualsComparator(theDateRange)) { condition = createConditionForValueWithComparator( - LESSTHAN, resourceTableRoot.getLastUpdatedColumn(), theDateRange.getLowerBoundAsInstant()); + LESSTHAN, + theResourceTablePredicateBuilder.getLastUpdatedColumn(), + theDateRange.getLowerBoundAsInstant()); conditions.add(condition); condition = createConditionForValueWithComparator( - GREATERTHAN, resourceTableRoot.getLastUpdatedColumn(), theDateRange.getUpperBoundAsInstant()); + GREATERTHAN, + theResourceTablePredicateBuilder.getLastUpdatedColumn(), + theDateRange.getUpperBoundAsInstant()); conditions.add(condition); return ComboCondition.or(conditions.toArray(new Condition[0])); } @@ -715,7 +705,7 @@ public class SearchQueryBuilder { if (theDateRange.getLowerBoundAsInstant() != null) { condition = createConditionForValueWithComparator( GREATERTHAN_OR_EQUALS, - resourceTableRoot.getLastUpdatedColumn(), + theResourceTablePredicateBuilder.getLastUpdatedColumn(), theDateRange.getLowerBoundAsInstant()); conditions.add(condition); } @@ -723,7 +713,7 @@ public class SearchQueryBuilder { if (theDateRange.getUpperBoundAsInstant() != null) { condition = createConditionForValueWithComparator( LESSTHAN_OR_EQUALS, - resourceTableRoot.getLastUpdatedColumn(), + theResourceTablePredicateBuilder.getLastUpdatedColumn(), theDateRange.getUpperBoundAsInstant()); conditions.add(condition); } @@ -757,7 +747,7 @@ public class SearchQueryBuilder { List excludePids = JpaPid.toLongList(theExistingPidSetToExclude); - ourLog.trace("excludePids = " + excludePids); + ourLog.trace("excludePids = {}", excludePids); DbColumn resourceIdColumn = getOrCreateFirstPredicateBuilder().getResourceIdColumn(); InCondition predicate = new InCondition(resourceIdColumn, generatePlaceholders(excludePids)); diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SqlObjectFactory.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SqlObjectFactory.java index 29e04527f96..621fe211185 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SqlObjectFactory.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/builder/sql/SqlObjectFactory.java @@ -24,7 +24,6 @@ import ca.uhn.fhir.jpa.search.builder.predicate.ComboNonUniqueSearchParameterPre import ca.uhn.fhir.jpa.search.builder.predicate.ComboUniqueSearchParameterPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.CoordsPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.DatePredicateBuilder; -import ca.uhn.fhir.jpa.search.builder.predicate.ForcedIdPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.NumberPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.QuantityNormalizedPredicateBuilder; import ca.uhn.fhir.jpa.search.builder.predicate.QuantityPredicateBuilder; @@ -63,10 +62,6 @@ public class SqlObjectFactory { return myApplicationContext.getBean(DatePredicateBuilder.class, theSearchSqlBuilder); } - public ForcedIdPredicateBuilder newForcedIdPredicateBuilder(SearchQueryBuilder theSearchSqlBuilder) { - return myApplicationContext.getBean(ForcedIdPredicateBuilder.class, theSearchSqlBuilder); - } - public NumberPredicateBuilder numberIndexTable(SearchQueryBuilder theSearchSqlBuilder) { return myApplicationContext.getBean(NumberPredicateBuilder.class, theSearchSqlBuilder); } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/DatabaseSearchCacheSvcImpl.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/DatabaseSearchCacheSvcImpl.java index 7bf60372f0a..0f666f60505 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/DatabaseSearchCacheSvcImpl.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/DatabaseSearchCacheSvcImpl.java @@ -25,29 +25,35 @@ import ca.uhn.fhir.jpa.api.config.JpaStorageSettings; import ca.uhn.fhir.jpa.dao.data.ISearchDao; import ca.uhn.fhir.jpa.dao.data.ISearchIncludeDao; import ca.uhn.fhir.jpa.dao.data.ISearchResultDao; +import ca.uhn.fhir.jpa.dao.data.SearchIdAndResultSize; import ca.uhn.fhir.jpa.dao.tx.HapiTransactionService; import ca.uhn.fhir.jpa.dao.tx.IHapiTransactionService; +import ca.uhn.fhir.jpa.dao.tx.IHapiTransactionService.IExecutionBuilder; import ca.uhn.fhir.jpa.entity.Search; import ca.uhn.fhir.jpa.model.search.SearchStatusEnum; import ca.uhn.fhir.system.HapiSystemProperties; import com.google.common.annotations.VisibleForTesting; -import com.google.common.collect.Lists; import org.apache.commons.lang3.Validate; import org.apache.commons.lang3.time.DateUtils; +import org.hibernate.Session; import org.hl7.fhir.dstu3.model.InstantType; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.data.domain.PageRequest; -import org.springframework.data.domain.Slice; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.annotation.Transactional; +import java.sql.Connection; import java.time.Instant; import java.util.Collection; import java.util.Date; -import java.util.List; +import java.util.HashSet; import java.util.Optional; +import java.util.Set; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.stream.Stream; +import javax.annotation.Nonnull; +import javax.persistence.EntityManager; public class DatabaseSearchCacheSvcImpl implements ISearchCacheSvc { /* @@ -56,13 +62,12 @@ public class DatabaseSearchCacheSvcImpl implements ISearchCacheSvc { * type query and this can fail if we have 1000s of params */ public static final int DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_STMT = 500; - public static final int DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_PAS = 20000; + public static final int DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_PAS = 50000; public static final long SEARCH_CLEANUP_JOB_INTERVAL_MILLIS = DateUtils.MILLIS_PER_MINUTE; public static final int DEFAULT_MAX_DELETE_CANDIDATES_TO_FIND = 2000; private static final Logger ourLog = LoggerFactory.getLogger(DatabaseSearchCacheSvcImpl.class); private static int ourMaximumResultsToDeleteInOneStatement = DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_STMT; - private static int ourMaximumResultsToDeleteInOnePass = DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_PAS; - private static int ourMaximumSearchesToCheckForDeletionCandidacy = DEFAULT_MAX_DELETE_CANDIDATES_TO_FIND; + private static int ourMaximumResultsToDeleteInOneCommit = DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_PAS; private static Long ourNowForUnitTests; /* * We give a bit of extra leeway just to avoid race conditions where a query result @@ -74,6 +79,9 @@ public class DatabaseSearchCacheSvcImpl implements ISearchCacheSvc { @Autowired private ISearchDao mySearchDao; + @Autowired + private EntityManager myEntityManager; + @Autowired private ISearchResultDao mySearchResultDao; @@ -169,14 +177,249 @@ public class DatabaseSearchCacheSvcImpl implements ISearchCacheSvc { return Optional.empty(); } + /** + * A transient worker for a single pass through stale-search deletion. + */ + class DeleteRun { + final RequestPartitionId myRequestPartitionId; + final Instant myDeadline; + final Date myCutoffForDeletion; + final Set myUpdateDeletedFlagBatch = new HashSet<>(); + final Set myDeleteSearchBatch = new HashSet<>(); + /** the Search pids of the SearchResults we plan to delete in a chunk */ + final Set myDeleteSearchResultsBatch = new HashSet<>(); + /** + * Number of results we have queued up in mySearchPidsToDeleteResults to delete. + * We try to keep this to a reasonable size to avoid long transactions that may escalate to a table lock. + */ + private int myDeleteSearchResultsBatchCount = 0; + + DeleteRun(Instant theDeadline, Date theCutoffForDeletion, RequestPartitionId theRequestPartitionId) { + myDeadline = theDeadline; + myCutoffForDeletion = theCutoffForDeletion; + myRequestPartitionId = theRequestPartitionId; + } + + /** + * Mark all ids in the mySearchesToMarkForDeletion buffer as deleted, and clear the buffer. + */ + public void flushDeleteMarks() { + if (myUpdateDeletedFlagBatch.isEmpty()) { + return; + } + ourLog.debug("Marking {} searches as deleted", myUpdateDeletedFlagBatch.size()); + mySearchDao.updateDeleted(myUpdateDeletedFlagBatch, true); + myUpdateDeletedFlagBatch.clear(); + commitOpenChanges(); + } + + /** + * Dig into the guts of our Hibernate session, flush any changes in the session, and commit the underlying connection. + */ + private void commitOpenChanges() { + // flush to force Hibernate to actually get a connection from the pool + myEntityManager.flush(); + // get our connection from the underlying Hibernate session, and commit + //noinspection resource + myEntityManager.unwrap(Session.class).doWork(Connection::commit); + } + + void throwIfDeadlineExpired() { + boolean result = Instant.ofEpochMilli(now()).isAfter(myDeadline); + if (result) { + throw new DeadlineException( + Msg.code(2443) + "Deadline expired while cleaning Search cache - " + myDeadline); + } + } + + private int deleteMarkedSearchesInBatches() { + AtomicInteger deletedCounter = new AtomicInteger(0); + + try (final Stream toDelete = mySearchDao.findDeleted()) { + assert toDelete != null; + + toDelete.forEach(nextSearchToDelete -> { + throwIfDeadlineExpired(); + + deleteSearchAndResults(nextSearchToDelete.searchId, nextSearchToDelete.size); + + deletedCounter.incrementAndGet(); + }); + } + + // flush anything left in the buffers + flushSearchResultDeletes(); + flushSearchAndIncludeDeletes(); + + int deletedCount = deletedCounter.get(); + + ourLog.info("Deleted {} expired searches", deletedCount); + + return deletedCount; + } + + /** + * Schedule theSearchPid for deletion assuming it has theNumberOfResults SearchResults attached. + * + * We accumulate a batch of search pids for deletion, and then do a bulk DML as we reach a threshold number + * of SearchResults. + * + * @param theSearchPid pk of the Search + * @param theNumberOfResults the number of SearchResults attached + */ + private void deleteSearchAndResults(long theSearchPid, int theNumberOfResults) { + ourLog.trace("Buffering deletion of search pid {} and {} results", theSearchPid, theNumberOfResults); + + myDeleteSearchBatch.add(theSearchPid); + + if (theNumberOfResults > ourMaximumResultsToDeleteInOneCommit) { + // don't buffer this one - do it inline + deleteSearchResultsByChunk(theSearchPid, theNumberOfResults); + return; + } + myDeleteSearchResultsBatch.add(theSearchPid); + myDeleteSearchResultsBatchCount += theNumberOfResults; + + if (myDeleteSearchResultsBatchCount > ourMaximumResultsToDeleteInOneCommit) { + flushSearchResultDeletes(); + } + + if (myDeleteSearchBatch.size() > ourMaximumResultsToDeleteInOneStatement) { + // flush the results to make sure we don't have any references. + flushSearchResultDeletes(); + + flushSearchAndIncludeDeletes(); + } + } + + /** + * If this Search has more results than our max delete size, + * delete in by itself in range chunks. + * @param theSearchPid the target Search pid + * @param theNumberOfResults the number of search results present + */ + private void deleteSearchResultsByChunk(long theSearchPid, int theNumberOfResults) { + ourLog.debug( + "Search {} is large: has {} results. Deleting results in chunks.", + theSearchPid, + theNumberOfResults); + for (int rangeEnd = theNumberOfResults; rangeEnd >= 0; rangeEnd -= ourMaximumResultsToDeleteInOneCommit) { + int rangeStart = rangeEnd - ourMaximumResultsToDeleteInOneCommit; + ourLog.trace("Deleting results for search {}: {} - {}", theSearchPid, rangeStart, rangeEnd); + mySearchResultDao.deleteBySearchIdInRange(theSearchPid, rangeStart, rangeEnd); + commitOpenChanges(); + } + } + + private void flushSearchAndIncludeDeletes() { + if (myDeleteSearchBatch.isEmpty()) { + return; + } + ourLog.debug("Deleting {} Search records", myDeleteSearchBatch.size()); + // referential integrity requires we delete includes before the search + mySearchIncludeDao.deleteForSearch(myDeleteSearchBatch); + mySearchDao.deleteByPids(myDeleteSearchBatch); + myDeleteSearchBatch.clear(); + commitOpenChanges(); + } + + private void flushSearchResultDeletes() { + if (myDeleteSearchResultsBatch.isEmpty()) { + return; + } + ourLog.debug( + "Deleting {} Search Results from {} searches", + myDeleteSearchResultsBatchCount, + myDeleteSearchResultsBatch.size()); + mySearchResultDao.deleteBySearchIds(myDeleteSearchResultsBatch); + myDeleteSearchResultsBatch.clear(); + myDeleteSearchResultsBatchCount = 0; + commitOpenChanges(); + } + + IExecutionBuilder getTxBuilder() { + return myTransactionService.withSystemRequest().withRequestPartitionId(myRequestPartitionId); + } + + private void run() { + ourLog.debug("Searching for searches which are before {}", myCutoffForDeletion); + + // this tx builder is not really for tx management. + // Instead, it is used bind a Hibernate session + connection to this thread. + // We will run a streaming query to look for work, and then commit changes in batches during the loops. + getTxBuilder().execute(theStatus -> { + try { + markDeletedInBatches(); + + throwIfDeadlineExpired(); + + // Delete searches that are marked as deleted + int deletedCount = deleteMarkedSearchesInBatches(); + + throwIfDeadlineExpired(); + + if ((ourLog.isDebugEnabled() || HapiSystemProperties.isTestModeEnabled()) && (deletedCount > 0)) { + Long total = mySearchDao.count(); + ourLog.debug("Deleted {} searches, {} remaining", deletedCount, total); + } + } catch (DeadlineException theTimeoutException) { + ourLog.warn(theTimeoutException.getMessage()); + } + + return null; + }); + } + + /** + * Stream through a list of pids before our cutoff, and set myDeleted=true in batches in a DML statement. + */ + private void markDeletedInBatches() { + + try (Stream toMarkDeleted = + mySearchDao.findWhereCreatedBefore(myCutoffForDeletion, new Date(now()))) { + assert toMarkDeleted != null; + + toMarkDeleted.forEach(nextSearchToDelete -> { + throwIfDeadlineExpired(); + + if (myUpdateDeletedFlagBatch.size() >= ourMaximumResultsToDeleteInOneStatement) { + flushDeleteMarks(); + } + ourLog.trace("Marking search with PID {} as ready for deletion", nextSearchToDelete); + myUpdateDeletedFlagBatch.add(nextSearchToDelete); + }); + + flushDeleteMarks(); + } + } + } + + /** + * Marker to abandon our delete run when we are over time. + */ + private static class DeadlineException extends RuntimeException { + public DeadlineException(String message) { + super(message); + } + } + @Override - public void pollForStaleSearchesAndDeleteThem(RequestPartitionId theRequestPartitionId) { + public void pollForStaleSearchesAndDeleteThem(RequestPartitionId theRequestPartitionId, Instant theDeadline) { HapiTransactionService.noTransactionAllowed(); if (!myStorageSettings.isExpireSearchResults()) { return; } + final Date cutoff = getCutoff(); + + final DeleteRun run = new DeleteRun(theDeadline, cutoff, theRequestPartitionId); + + run.run(); + } + + @Nonnull + private Date getCutoff() { long cutoffMillis = myStorageSettings.getExpireSearchResultsAfterMillis(); if (myStorageSettings.getReuseCachedSearchResultsForMillis() != null) { cutoffMillis = cutoffMillis + myStorageSettings.getReuseCachedSearchResultsForMillis(); @@ -189,108 +432,16 @@ public class DatabaseSearchCacheSvcImpl implements ISearchCacheSvc { new InstantType(cutoff), new InstantType(new Date(now()))); } - - ourLog.debug("Searching for searches which are before {}", cutoff); - - // Mark searches as deleted if they should be - final Slice toMarkDeleted = myTransactionService - .withSystemRequestOnPartition(theRequestPartitionId) - .execute(theStatus -> mySearchDao.findWhereCreatedBefore( - cutoff, new Date(), PageRequest.of(0, ourMaximumSearchesToCheckForDeletionCandidacy))); - assert toMarkDeleted != null; - for (final Long nextSearchToDelete : toMarkDeleted) { - ourLog.debug("Deleting search with PID {}", nextSearchToDelete); - myTransactionService - .withSystemRequest() - .withRequestPartitionId(theRequestPartitionId) - .execute(t -> { - mySearchDao.updateDeleted(nextSearchToDelete, true); - return null; - }); - } - - // Delete searches that are marked as deleted - final Slice toDelete = myTransactionService - .withSystemRequestOnPartition(theRequestPartitionId) - .execute(theStatus -> - mySearchDao.findDeleted(PageRequest.of(0, ourMaximumSearchesToCheckForDeletionCandidacy))); - assert toDelete != null; - for (final Long nextSearchToDelete : toDelete) { - ourLog.debug("Deleting search with PID {}", nextSearchToDelete); - myTransactionService - .withSystemRequest() - .withRequestPartitionId(theRequestPartitionId) - .execute(t -> { - deleteSearch(nextSearchToDelete); - return null; - }); - } - - int count = toDelete.getContent().size(); - if (count > 0) { - if (ourLog.isDebugEnabled() || HapiSystemProperties.isTestModeEnabled()) { - Long total = myTransactionService - .withSystemRequest() - .withRequestPartitionId(theRequestPartitionId) - .execute(t -> mySearchDao.count()); - ourLog.debug("Deleted {} searches, {} remaining", count, total); - } - } - } - - private void deleteSearch(final Long theSearchPid) { - mySearchDao.findById(theSearchPid).ifPresent(searchToDelete -> { - mySearchIncludeDao.deleteForSearch(searchToDelete.getId()); - - /* - * Note, we're only deleting up to 500 results in an individual search here. This - * is to prevent really long running transactions in cases where there are - * huge searches with tons of results in them. By the time we've gotten here - * we have marked the parent Search entity as deleted, so it's not such a - * huge deal to be only partially deleting search results. They'll get deleted - * eventually - */ - int max = ourMaximumResultsToDeleteInOnePass; - Slice resultPids = mySearchResultDao.findForSearch(PageRequest.of(0, max), searchToDelete.getId()); - if (resultPids.hasContent()) { - List> partitions = - Lists.partition(resultPids.getContent(), ourMaximumResultsToDeleteInOneStatement); - for (List nextPartition : partitions) { - mySearchResultDao.deleteByIds(nextPartition); - } - } - - // Only delete if we don't have results left in this search - if (resultPids.getNumberOfElements() < max) { - ourLog.debug( - "Deleting search {}/{} - Created[{}]", - searchToDelete.getId(), - searchToDelete.getUuid(), - new InstantType(searchToDelete.getCreated())); - mySearchDao.deleteByPid(searchToDelete.getId()); - } else { - ourLog.debug( - "Purged {} search results for deleted search {}/{}", - resultPids.getSize(), - searchToDelete.getId(), - searchToDelete.getUuid()); - } - }); - } - - @VisibleForTesting - public static void setMaximumSearchesToCheckForDeletionCandidacyForUnitTest( - int theMaximumSearchesToCheckForDeletionCandidacy) { - ourMaximumSearchesToCheckForDeletionCandidacy = theMaximumSearchesToCheckForDeletionCandidacy; + return cutoff; } @VisibleForTesting public static void setMaximumResultsToDeleteInOnePassForUnitTest(int theMaximumResultsToDeleteInOnePass) { - ourMaximumResultsToDeleteInOnePass = theMaximumResultsToDeleteInOnePass; + ourMaximumResultsToDeleteInOneCommit = theMaximumResultsToDeleteInOnePass; } @VisibleForTesting - public static void setMaximumResultsToDeleteForUnitTest(int theMaximumResultsToDelete) { + public static void setMaximumResultsToDeleteInOneStatement(int theMaximumResultsToDelete) { ourMaximumResultsToDeleteInOneStatement = theMaximumResultsToDelete; } @@ -302,7 +453,7 @@ public class DatabaseSearchCacheSvcImpl implements ISearchCacheSvc { ourNowForUnitTests = theNowForUnitTests; } - private static long now() { + public static long now() { if (ourNowForUnitTests != null) { return ourNowForUnitTests; } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/ISearchCacheSvc.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/ISearchCacheSvc.java index 34c662b83f7..8c9ab6f0ec1 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/ISearchCacheSvc.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/search/cache/ISearchCacheSvc.java @@ -23,6 +23,7 @@ import ca.uhn.fhir.interceptor.model.RequestPartitionId; import ca.uhn.fhir.jpa.entity.Search; import java.time.Instant; +import java.time.temporal.ChronoUnit; import java.util.Optional; public interface ISearchCacheSvc { @@ -86,5 +87,10 @@ public interface ISearchCacheSvc { * if they have some other mechanism for expiring stale results other than manually looking for them * and deleting them. */ - void pollForStaleSearchesAndDeleteThem(RequestPartitionId theRequestPartitionId); + void pollForStaleSearchesAndDeleteThem(RequestPartitionId theRequestPartitionId, Instant theDeadline); + + @Deprecated(since = "6.10", forRemoval = true) // wipmb delete once cdr merges + default void pollForStaleSearchesAndDeleteThem(RequestPartitionId theRequestPartitionId) { + pollForStaleSearchesAndDeleteThem(theRequestPartitionId, Instant.now().plus(1, ChronoUnit.MINUTES)); + } } diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/subscription/ResourceModifiedMessagePersistenceSvcImpl.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/subscription/ResourceModifiedMessagePersistenceSvcImpl.java index 86e85a85c9b..893aa6e6744 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/subscription/ResourceModifiedMessagePersistenceSvcImpl.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/subscription/ResourceModifiedMessagePersistenceSvcImpl.java @@ -35,6 +35,7 @@ import ca.uhn.fhir.jpa.subscription.async.AsyncResourceModifiedSubmitterSvc; import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; import ca.uhn.fhir.model.primitive.IdDt; import ca.uhn.fhir.rest.api.server.SystemRequestDetails; +import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException; import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; @@ -45,6 +46,7 @@ import org.slf4j.LoggerFactory; import java.util.Date; import java.util.List; +import java.util.Optional; import static ca.uhn.fhir.jpa.model.entity.PersistedResourceModifiedMessageEntityPK.with; @@ -92,9 +94,43 @@ public class ResourceModifiedMessagePersistenceSvcImpl implements IResourceModif @Override public ResourceModifiedMessage inflatePersistedResourceModifiedMessage( - IPersistedResourceModifiedMessage thePersistedResourceModifiedMessage) { + ResourceModifiedMessage theResourceModifiedMessage) { - return inflateResourceModifiedMessageFromEntity((ResourceModifiedEntity) thePersistedResourceModifiedMessage); + return inflateResourceModifiedMessageFromEntity(createEntityFrom(theResourceModifiedMessage)); + } + + @Override + public Optional inflatePersistedResourceModifiedMessageOrNull( + ResourceModifiedMessage theResourceModifiedMessage) { + ResourceModifiedMessage inflatedResourceModifiedMessage = null; + + try { + inflatedResourceModifiedMessage = inflatePersistedResourceModifiedMessage(theResourceModifiedMessage); + } catch (ResourceNotFoundException e) { + IdDt idDt = new IdDt( + theResourceModifiedMessage.getPayloadType(myFhirContext), + theResourceModifiedMessage.getPayloadId(), + theResourceModifiedMessage.getPayloadVersion()); + + ourLog.warn("Scheduled submission will be ignored since resource {} cannot be found", idDt.getIdPart(), e); + } catch (Exception ex) { + ourLog.error("Unknown error encountered on inflation of resources.", ex); + } + + return Optional.ofNullable(inflatedResourceModifiedMessage); + } + + @Override + public ResourceModifiedMessage createResourceModifiedMessageFromEntityWithoutInflation( + IPersistedResourceModifiedMessage thePersistedResourceModifiedMessage) { + ResourceModifiedMessage resourceModifiedMessage = getPayloadLessMessageFromString( + ((ResourceModifiedEntity) thePersistedResourceModifiedMessage).getSummaryResourceModifiedMessage()); + + IdDt resourceId = + createIdDtFromResourceModifiedEntity((ResourceModifiedEntity) thePersistedResourceModifiedMessage); + resourceModifiedMessage.setPayloadId(resourceId); + + return resourceModifiedMessage; } @Override @@ -112,17 +148,13 @@ public class ResourceModifiedMessagePersistenceSvcImpl implements IResourceModif protected ResourceModifiedMessage inflateResourceModifiedMessageFromEntity( ResourceModifiedEntity theResourceModifiedEntity) { - String resourcePid = - theResourceModifiedEntity.getResourceModifiedEntityPK().getResourcePid(); - String resourceVersion = - theResourceModifiedEntity.getResourceModifiedEntityPK().getResourceVersion(); String resourceType = theResourceModifiedEntity.getResourceType(); ResourceModifiedMessage retVal = getPayloadLessMessageFromString(theResourceModifiedEntity.getSummaryResourceModifiedMessage()); SystemRequestDetails systemRequestDetails = new SystemRequestDetails().setRequestPartitionId(retVal.getPartitionId()); - IdDt resourceIdDt = new IdDt(resourceType, resourcePid, resourceVersion); + IdDt resourceIdDt = createIdDtFromResourceModifiedEntity(theResourceModifiedEntity); IFhirResourceDao dao = myDaoRegistry.getResourceDao(resourceType); IBaseResource iBaseResource = dao.read(resourceIdDt, systemRequestDetails, true); @@ -164,6 +196,16 @@ public class ResourceModifiedMessagePersistenceSvcImpl implements IResourceModif } } + private IdDt createIdDtFromResourceModifiedEntity(ResourceModifiedEntity theResourceModifiedEntity) { + String resourcePid = + theResourceModifiedEntity.getResourceModifiedEntityPK().getResourcePid(); + String resourceVersion = + theResourceModifiedEntity.getResourceModifiedEntityPK().getResourceVersion(); + String resourceType = theResourceModifiedEntity.getResourceType(); + + return new IdDt(resourceType, resourcePid, resourceVersion); + } + private static class PayloadLessResourceModifiedMessage extends ResourceModifiedMessage { public PayloadLessResourceModifiedMessage(ResourceModifiedMessage theMsg) { diff --git a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/term/TermReadSvcImpl.java b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/term/TermReadSvcImpl.java index edc52f2e4a0..447e3058c4d 100644 --- a/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/term/TermReadSvcImpl.java +++ b/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/term/TermReadSvcImpl.java @@ -2979,7 +2979,7 @@ public class TermReadSvcImpl implements ITermReadSvc, IHasScheduledJobs { if (resultList.size() > 1) throw new NonUniqueResultException(Msg.code(911) + "More than one CodeSystem is pointed by forcedId: " - + theForcedId + ". Was constraint " + ResourceTable.IDX_RES_FHIR_ID + " removed?"); + + theForcedId + ". Was constraint " + ResourceTable.IDX_RES_TYPE_FHIR_ID + " removed?"); IFhirResourceDao csDao = myDaoRegistry.getResourceDao("CodeSystem"); IBaseResource cs = myJpaStorageResourceParser.toResource(resultList.get(0), false); diff --git a/hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplTest.java b/hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplTest.java index 2e2528697ce..df28a3f2689 100644 --- a/hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplTest.java +++ b/hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplTest.java @@ -5,7 +5,6 @@ import ca.uhn.fhir.context.FhirVersionEnum; import ca.uhn.fhir.context.support.IValidationSupport; import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.fhir.jpa.api.dao.IFhirResourceDao; -import ca.uhn.fhir.jpa.api.model.DaoMethodOutcome; import ca.uhn.fhir.jpa.dao.data.INpmPackageVersionDao; import ca.uhn.fhir.jpa.dao.tx.IHapiTransactionService; import ca.uhn.fhir.jpa.dao.tx.NonTransactionalHapiTransactionService; @@ -13,18 +12,24 @@ import ca.uhn.fhir.jpa.model.config.PartitionSettings; import ca.uhn.fhir.jpa.packages.loader.PackageResourceParsingSvc; import ca.uhn.fhir.jpa.searchparam.SearchParameterMap; import ca.uhn.fhir.jpa.searchparam.registry.ISearchParamRegistryController; +import ca.uhn.fhir.jpa.searchparam.util.SearchParameterHelper; import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.server.SimpleBundleProvider; +import org.hl7.fhir.instance.model.api.IBaseResource; import org.hl7.fhir.r4.model.CodeSystem; +import org.hl7.fhir.r4.model.CodeType; import org.hl7.fhir.r4.model.Communication; import org.hl7.fhir.r4.model.DocumentReference; import org.hl7.fhir.r4.model.Enumerations; +import org.hl7.fhir.r4.model.IdType; import org.hl7.fhir.r4.model.SearchParameter; import org.hl7.fhir.r4.model.Subscription; import org.hl7.fhir.utilities.npm.NpmPackage; import org.hl7.fhir.utilities.npm.PackageGenerator; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.MethodSource; import org.mockito.ArgumentCaptor; import org.mockito.Captor; import org.mockito.InjectMocks; @@ -36,6 +41,9 @@ import javax.annotation.Nonnull; import java.io.ByteArrayOutputStream; import java.io.IOException; import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Iterator; import java.util.List; import java.util.Optional; @@ -45,12 +53,10 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.verifyNoInteractions; import static org.mockito.Mockito.when; @ExtendWith(MockitoExtension.class) public class PackageInstallerSvcImplTest { - public static final String PACKAGE_VERSION = "1.0"; public static final String PACKAGE_ID_1 = "package1"; @@ -65,7 +71,13 @@ public class PackageInstallerSvcImplTest { @Mock private IFhirResourceDao myCodeSystemDao; @Mock + private IFhirResourceDao mySearchParameterDao; + @Mock private IValidationSupport myIValidationSupport; + @Mock + private SearchParameterHelper mySearchParameterHelper; + @Mock + private SearchParameterMap mySearchParameterMap; @Spy private FhirContext myCtx = FhirContext.forR4Cached(); @Spy @@ -77,6 +89,15 @@ public class PackageInstallerSvcImplTest { @InjectMocks private PackageInstallerSvcImpl mySvc; + @Captor + private ArgumentCaptor mySearchParameterMapCaptor; + @Captor + private ArgumentCaptor myCodeSystemCaptor; + @Captor + private ArgumentCaptor mySearchParameterCaptor; + @Captor + private ArgumentCaptor myRequestDetailsCaptor; + @Test public void testPackageCompatibility() { mySvc.assertFhirVersionsAreCompatible("R4", "R4B"); @@ -206,19 +227,7 @@ public class PackageInstallerSvcImplTest { cs.setUrl("http://my-code-system"); cs.setContent(CodeSystem.CodeSystemContentMode.COMPLETE); - NpmPackage pkg = createPackage(cs, PACKAGE_ID_1); - - when(myPackageVersionDao.findByPackageIdAndVersion(any(), any())).thenReturn(Optional.empty()); - when(myPackageCacheManager.installPackage(any())).thenReturn(pkg); - when(myDaoRegistry.getResourceDao(CodeSystem.class)).thenReturn(myCodeSystemDao); - when(myCodeSystemDao.search(any(), any())).thenReturn(new SimpleBundleProvider(existingCs)); - when(myCodeSystemDao.update(any(),any(RequestDetails.class))).thenReturn(new DaoMethodOutcome()); - - PackageInstallationSpec spec = new PackageInstallationSpec(); - spec.setName(PACKAGE_ID_1); - spec.setVersion(PACKAGE_VERSION); - spec.setInstallMode(PackageInstallationSpec.InstallModeEnum.STORE_AND_INSTALL); - spec.setPackageContents(packageToBytes(pkg)); + PackageInstallationSpec spec = setupResourceInPackage(existingCs, cs, myCodeSystemDao); // Test mySvc.install(spec); @@ -233,34 +242,108 @@ public class PackageInstallerSvcImplTest { assertEquals("existingcs", codeSystem.getIdPart()); } - @Nonnull - private static byte[] packageToBytes(NpmPackage pkg) throws IOException { - ByteArrayOutputStream stream = new ByteArrayOutputStream(); - pkg.save(stream); - byte[] bytes = stream.toByteArray(); - return bytes; + public enum InstallType { + CREATE, UPDATE_WITH_EXISTING, UPDATE, UPDATE_OVERRIDE } - @Captor - private ArgumentCaptor mySearchParameterMapCaptor; - @Captor - private ArgumentCaptor myCodeSystemCaptor; + public static List parameters() { + return List.of( + new Object[]{null, null, null, List.of("Patient"), InstallType.CREATE}, + new Object[]{null, null, "us-core-patient-given", List.of("Patient"), InstallType.UPDATE}, + new Object[]{"individual-given", List.of("Patient", "Practitioner"), "us-core-patient-given", List.of("Patient"), InstallType.UPDATE_WITH_EXISTING}, + new Object[]{"patient-given", List.of("Patient"), "us-core-patient-given", List.of("Patient"), InstallType.UPDATE_OVERRIDE} + ); + } + + @ParameterizedTest + @MethodSource("parameters") + public void testCreateOrUpdate_withSearchParameter(String theExistingId, Collection theExistingBase, + String theInstallId, Collection theInstallBase, + InstallType theInstallType) throws IOException { + // Setup + SearchParameter existingSP = null; + if (theExistingId != null) { + existingSP = createSearchParameter(theExistingId, theExistingBase); + } + SearchParameter installSP = createSearchParameter(theInstallId, theInstallBase); + PackageInstallationSpec spec = setupResourceInPackage(existingSP, installSP, mySearchParameterDao); + + // Test + mySvc.install(spec); + + // Verify + if (theInstallType == InstallType.CREATE) { + verify(mySearchParameterDao, times(1)).create(mySearchParameterCaptor.capture(), myRequestDetailsCaptor.capture()); + } else if (theInstallType == InstallType.UPDATE_WITH_EXISTING){ + verify(mySearchParameterDao, times(2)).update(mySearchParameterCaptor.capture(), myRequestDetailsCaptor.capture()); + } else { + verify(mySearchParameterDao, times(1)).update(mySearchParameterCaptor.capture(), myRequestDetailsCaptor.capture()); + } + + Iterator iteratorSP = mySearchParameterCaptor.getAllValues().iterator(); + if (theInstallType == InstallType.UPDATE_WITH_EXISTING) { + SearchParameter capturedSP = iteratorSP.next(); + assertEquals(theExistingId, capturedSP.getIdPart()); + List expectedBase = new ArrayList<>(theExistingBase); + expectedBase.removeAll(theInstallBase); + assertEquals(expectedBase, capturedSP.getBase().stream().map(CodeType::getCode).toList()); + } + SearchParameter capturedSP = iteratorSP.next(); + if (theInstallType == InstallType.UPDATE_OVERRIDE) { + assertEquals(theExistingId, capturedSP.getIdPart()); + } else { + assertEquals(theInstallId, capturedSP.getIdPart()); + } + assertEquals(theInstallBase, capturedSP.getBase().stream().map(CodeType::getCode).toList()); + } + + private PackageInstallationSpec setupResourceInPackage(IBaseResource myExistingResource, IBaseResource myInstallResource, + IFhirResourceDao myFhirResourceDao) throws IOException { + NpmPackage pkg = createPackage(myInstallResource, myInstallResource.getClass().getSimpleName()); + + when(myPackageVersionDao.findByPackageIdAndVersion(any(), any())).thenReturn(Optional.empty()); + when(myPackageCacheManager.installPackage(any())).thenReturn(pkg); + when(myDaoRegistry.getResourceDao(myInstallResource.getClass())).thenReturn(myFhirResourceDao); + when(myFhirResourceDao.search(any(), any())).thenReturn(myExistingResource != null ? + new SimpleBundleProvider(myExistingResource) : new SimpleBundleProvider()); + if (myInstallResource.getClass().getSimpleName().equals("SearchParameter")) { + when(mySearchParameterHelper.buildSearchParameterMapFromCanonical(any())).thenReturn(Optional.of(mySearchParameterMap)); + } + + PackageInstallationSpec spec = new PackageInstallationSpec(); + spec.setName(PACKAGE_ID_1); + spec.setVersion(PACKAGE_VERSION); + spec.setInstallMode(PackageInstallationSpec.InstallModeEnum.STORE_AND_INSTALL); + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + pkg.save(stream); + spec.setPackageContents(stream.toByteArray()); + + return spec; + } @Nonnull - private NpmPackage createPackage(CodeSystem cs, String packageId) throws IOException { + private NpmPackage createPackage(IBaseResource theResource, String theResourceType) { PackageGenerator manifestGenerator = new PackageGenerator(); - manifestGenerator.name(packageId); + manifestGenerator.name(PACKAGE_ID_1); manifestGenerator.version(PACKAGE_VERSION); manifestGenerator.description("a package"); manifestGenerator.fhirVersions(List.of(FhirVersionEnum.R4.getFhirVersionString())); + String csString = myCtx.newJsonParser().encodeResourceToString(theResource); NpmPackage pkg = NpmPackage.empty(manifestGenerator); - - String csString = myCtx.newJsonParser().encodeResourceToString(cs); - pkg.addFile("package", "cs.json", csString.getBytes(StandardCharsets.UTF_8), "CodeSystem"); + pkg.addFile("package", theResourceType + ".json", csString.getBytes(StandardCharsets.UTF_8), theResourceType); return pkg; } - + private static SearchParameter createSearchParameter(String theId, Collection theBase) { + SearchParameter searchParameter = new SearchParameter(); + if (theId != null) { + searchParameter.setId(new IdType("SearchParameter", theId)); + } + searchParameter.setCode("someCode"); + theBase.forEach(base -> searchParameter.getBase().add(new CodeType(base))); + searchParameter.setExpression("someExpression"); + return searchParameter; + } } diff --git a/hapi-fhir-jpaserver-elastic-test-utilities/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchLastNAsyncIT.java b/hapi-fhir-jpaserver-elastic-test-utilities/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchLastNAsyncIT.java index 1e619d17e99..22274d942d3 100644 --- a/hapi-fhir-jpaserver-elastic-test-utilities/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchLastNAsyncIT.java +++ b/hapi-fhir-jpaserver-elastic-test-utilities/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchLastNAsyncIT.java @@ -22,6 +22,7 @@ import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.Set; import java.util.stream.Collectors; import static org.hamcrest.MatcherAssert.assertThat; @@ -72,9 +73,9 @@ public class FhirResourceDaoR4SearchLastNAsyncIT extends BaseR4SearchLastN { public void testLastNChunking() { runInTransaction(() -> { - for (Search search : mySearchDao.findAll()) { - mySearchDao.updateDeleted(search.getId(), true); - } + Set all = mySearchDao.findAll().stream().map(Search::getId).collect(Collectors.toSet()); + + mySearchDao.updateDeleted(all, true); }); // Set up search parameters that will return 75 Observations. diff --git a/hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceTable.java b/hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceTable.java index f516209a3cd..bf1ca3b56ac 100644 --- a/hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceTable.java +++ b/hapi-fhir-jpaserver-model/src/main/java/ca/uhn/fhir/jpa/model/entity/ResourceTable.java @@ -76,7 +76,7 @@ import javax.persistence.Transient; import javax.persistence.UniqueConstraint; import javax.persistence.Version; -import static ca.uhn.fhir.jpa.model.entity.ResourceTable.IDX_RES_FHIR_ID; +import static ca.uhn.fhir.jpa.model.entity.ResourceTable.IDX_RES_TYPE_FHIR_ID; @Indexed(routingBinder = @RoutingBinderRef(type = ResourceTableRoutingBinder.class)) @Entity @@ -84,12 +84,13 @@ import static ca.uhn.fhir.jpa.model.entity.ResourceTable.IDX_RES_FHIR_ID; name = ResourceTable.HFJ_RESOURCE, uniqueConstraints = { @UniqueConstraint( - name = IDX_RES_FHIR_ID, - columnNames = {"FHIR_ID", "RES_TYPE"}) + name = IDX_RES_TYPE_FHIR_ID, + columnNames = {"RES_TYPE", "FHIR_ID"}) }, indexes = { // Do not reuse previously used index name: IDX_INDEXSTATUS, IDX_RES_TYPE @Index(name = "IDX_RES_DATE", columnList = BaseHasResource.RES_UPDATED), + @Index(name = "IDX_RES_FHIR_ID", columnList = "FHIR_ID"), @Index( name = "IDX_RES_TYPE_DEL_UPDATED", columnList = "RES_TYPE,RES_DELETED_AT,RES_UPDATED,PARTITION_ID,RES_ID"), @@ -100,10 +101,11 @@ public class ResourceTable extends BaseHasResource implements Serializable, IBas public static final int RESTYPE_LEN = 40; public static final String HFJ_RESOURCE = "HFJ_RESOURCE"; public static final String RES_TYPE = "RES_TYPE"; + public static final String FHIR_ID = "FHIR_ID"; private static final int MAX_LANGUAGE_LENGTH = 20; private static final long serialVersionUID = 1L; public static final int MAX_FORCED_ID_LENGTH = 100; - public static final String IDX_RES_FHIR_ID = "IDX_RES_FHIR_ID"; + public static final String IDX_RES_TYPE_FHIR_ID = "IDX_RES_TYPE_FHIR_ID"; /** * Holds the narrative text only - Used for Fulltext searching but not directly stored in the DB @@ -381,7 +383,7 @@ public class ResourceTable extends BaseHasResource implements Serializable, IBas * Will be null during insert time until the first read. */ @Column( - name = "FHIR_ID", + name = FHIR_ID, // [A-Za-z0-9\-\.]{1,64} - https://www.hl7.org/fhir/datatypes.html#id length = 64, // we never update this after insert, and the Generator will otherwise "dirty" the object. diff --git a/hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/ResourceMetaParams.java b/hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/ResourceMetaParams.java index 40a4658aeb5..0114fba0b7a 100644 --- a/hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/ResourceMetaParams.java +++ b/hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/searchparam/ResourceMetaParams.java @@ -51,6 +51,8 @@ public class ResourceMetaParams { Map>> resourceMetaAndParams = new HashMap<>(); resourceMetaParams.put(IAnyResource.SP_RES_ID, StringParam.class); resourceMetaAndParams.put(IAnyResource.SP_RES_ID, StringAndListParam.class); + resourceMetaParams.put(Constants.PARAM_PID, TokenParam.class); + resourceMetaAndParams.put(Constants.PARAM_PID, TokenAndListParam.class); resourceMetaParams.put(Constants.PARAM_TAG, TokenParam.class); resourceMetaAndParams.put(Constants.PARAM_TAG, TokenAndListParam.class); resourceMetaParams.put(Constants.PARAM_PROFILE, UriParam.class); diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriber.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriber.java index 2e4dff6e7e8..641e732b642 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriber.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriber.java @@ -33,7 +33,9 @@ import ca.uhn.fhir.jpa.subscription.match.registry.ActiveSubscription; import ca.uhn.fhir.jpa.subscription.match.registry.SubscriptionRegistry; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscription; import ca.uhn.fhir.jpa.subscription.model.ResourceDeliveryMessage; +import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; import ca.uhn.fhir.rest.api.server.IBundleProvider; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import ca.uhn.fhir.util.BundleBuilder; import com.google.common.annotations.VisibleForTesting; import org.apache.commons.text.StringSubstitutor; @@ -48,6 +50,7 @@ import org.springframework.messaging.MessagingException; import java.util.HashMap; import java.util.Map; +import java.util.Optional; import static ca.uhn.fhir.jpa.subscription.util.SubscriptionUtil.createRequestDetailForPartitionedRequest; @@ -60,6 +63,9 @@ public abstract class BaseSubscriptionDeliverySubscriber implements MessageHandl @Autowired protected SubscriptionRegistry mySubscriptionRegistry; + @Autowired + protected IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; + @Autowired private IInterceptorBroadcaster myInterceptorBroadcaster; @@ -149,6 +155,13 @@ public abstract class BaseSubscriptionDeliverySubscriber implements MessageHandl return builder.getBundle(); } + protected Optional inflateResourceModifiedMessageFromDeliveryMessage( + ResourceDeliveryMessage theMsg) { + ResourceModifiedMessage payloadLess = + new ResourceModifiedMessage(theMsg.getPayloadId(myFhirContext), theMsg.getOperationType()); + return myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(payloadLess); + } + @VisibleForTesting public void setFhirContextForUnitTest(FhirContext theCtx) { myFhirContext = theCtx; @@ -174,6 +187,12 @@ public abstract class BaseSubscriptionDeliverySubscriber implements MessageHandl myMatchUrlService = theMatchUrlService; } + @VisibleForTesting + public void setResourceModifiedMessagePersistenceSvcForUnitTest( + IResourceModifiedMessagePersistenceSvc theResourceModifiedMessagePersistenceSvc) { + myResourceModifiedMessagePersistenceSvc = theResourceModifiedMessagePersistenceSvc; + } + public IInterceptorBroadcaster getInterceptorBroadcaster() { return myInterceptorBroadcaster; } diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/email/SubscriptionDeliveringEmailSubscriber.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/email/SubscriptionDeliveringEmailSubscriber.java index 01ccf19397a..80275a84317 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/email/SubscriptionDeliveringEmailSubscriber.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/email/SubscriptionDeliveringEmailSubscriber.java @@ -24,6 +24,7 @@ import ca.uhn.fhir.jpa.model.entity.StorageSettings; import ca.uhn.fhir.jpa.subscription.match.deliver.BaseSubscriptionDeliverySubscriber; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscription; import ca.uhn.fhir.jpa.subscription.model.ResourceDeliveryMessage; +import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; import ca.uhn.fhir.rest.api.EncodingEnum; import com.google.common.annotations.VisibleForTesting; import org.apache.commons.lang3.StringUtils; @@ -33,6 +34,7 @@ import org.springframework.beans.factory.annotation.Autowired; import java.util.ArrayList; import java.util.List; +import java.util.Optional; import static org.apache.commons.lang3.StringUtils.defaultString; import static org.apache.commons.lang3.StringUtils.isNotBlank; @@ -73,7 +75,7 @@ public class SubscriptionDeliveringEmailSubscriber extends BaseSubscriptionDeliv if (isNotBlank(subscription.getPayloadString())) { EncodingEnum encoding = EncodingEnum.forContentType(subscription.getPayloadString()); if (encoding != null) { - payload = theMessage.getPayloadString(); + payload = getPayloadStringFromMessageOrEmptyString(theMessage); } } @@ -112,4 +114,24 @@ public class SubscriptionDeliveringEmailSubscriber extends BaseSubscriptionDeliv public IEmailSender getEmailSender() { return myEmailSender; } + + /** + * Get the payload string, fetch it from the DB when the payload is null. + */ + private String getPayloadStringFromMessageOrEmptyString(ResourceDeliveryMessage theMessage) { + String payload = theMessage.getPayloadString(); + + if (theMessage.getPayload(myCtx) != null) { + return payload; + } + + Optional inflatedMessage = + inflateResourceModifiedMessageFromDeliveryMessage(theMessage); + if (inflatedMessage.isEmpty()) { + return ""; + } + + payload = inflatedMessage.get().getPayloadString(); + return payload; + } } diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/message/SubscriptionDeliveringMessageSubscriber.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/message/SubscriptionDeliveringMessageSubscriber.java index b801d0e8f95..c15854c143e 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/message/SubscriptionDeliveringMessageSubscriber.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/deliver/message/SubscriptionDeliveringMessageSubscriber.java @@ -39,6 +39,7 @@ import org.springframework.messaging.MessagingException; import java.net.URI; import java.net.URISyntaxException; +import java.util.Optional; import static org.apache.commons.lang3.StringUtils.isNotBlank; @@ -66,7 +67,7 @@ public class SubscriptionDeliveringMessageSubscriber extends BaseSubscriptionDel IBaseResource payloadResource = createDeliveryBundleForPayloadSearchCriteria( theSubscription, theWrappedMessageToSend.getPayload().getPayload(myFhirContext)); ResourceModifiedJsonMessage newWrappedMessageToSend = - convertDeliveryMessageToResourceModifiedMessage(theSourceMessage, payloadResource); + convertDeliveryMessageToResourceModifiedJsonMessage(theSourceMessage, payloadResource); theWrappedMessageToSend.setPayload(newWrappedMessageToSend.getPayload()); payloadId = payloadResource.getIdElement().toUnqualifiedVersionless().getValue(); @@ -82,7 +83,7 @@ public class SubscriptionDeliveringMessageSubscriber extends BaseSubscriptionDel .getValue()); } - private ResourceModifiedJsonMessage convertDeliveryMessageToResourceModifiedMessage( + private ResourceModifiedJsonMessage convertDeliveryMessageToResourceModifiedJsonMessage( ResourceDeliveryMessage theMsg, IBaseResource thePayloadResource) { ResourceModifiedMessage payload = new ResourceModifiedMessage(myFhirContext, thePayloadResource, theMsg.getOperationType()); @@ -96,8 +97,17 @@ public class SubscriptionDeliveringMessageSubscriber extends BaseSubscriptionDel public void handleMessage(ResourceDeliveryMessage theMessage) throws MessagingException, URISyntaxException { CanonicalSubscription subscription = theMessage.getSubscription(); IBaseResource payloadResource = theMessage.getPayload(myFhirContext); + if (payloadResource == null) { + Optional inflatedMsg = + inflateResourceModifiedMessageFromDeliveryMessage(theMessage); + if (inflatedMsg.isEmpty()) { + return; + } + payloadResource = inflatedMsg.get().getPayload(myFhirContext); + } + ResourceModifiedJsonMessage messageWrapperToSend = - convertDeliveryMessageToResourceModifiedMessage(theMessage, payloadResource); + convertDeliveryMessageToResourceModifiedJsonMessage(theMessage, payloadResource); // Interceptor call: SUBSCRIPTION_BEFORE_MESSAGE_DELIVERY HookParams params = new HookParams() diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionActivatingSubscriber.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionActivatingSubscriber.java index 55914efdde5..405cd94796d 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionActivatingSubscriber.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionActivatingSubscriber.java @@ -31,6 +31,7 @@ import ca.uhn.fhir.rest.api.server.SystemRequestDetails; import ca.uhn.fhir.rest.server.exceptions.ResourceGoneException; import ca.uhn.fhir.rest.server.exceptions.UnprocessableEntityException; import ca.uhn.fhir.subscription.SubscriptionConstants; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import ca.uhn.fhir.util.SubscriptionUtil; import org.hl7.fhir.dstu2.model.Subscription; import org.hl7.fhir.instance.model.api.IBaseResource; @@ -41,6 +42,7 @@ import org.springframework.messaging.Message; import org.springframework.messaging.MessageHandler; import org.springframework.messaging.MessagingException; +import java.util.Optional; import javax.annotation.Nonnull; /** @@ -64,6 +66,8 @@ public class SubscriptionActivatingSubscriber implements MessageHandler { @Autowired private StorageSettings myStorageSettings; + @Autowired + private IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; /** * Constructor */ @@ -86,6 +90,16 @@ public class SubscriptionActivatingSubscriber implements MessageHandler { switch (payload.getOperationType()) { case CREATE: case UPDATE: + if (payload.getPayload(myFhirContext) == null) { + Optional inflatedMsg = + myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull( + payload); + if (inflatedMsg.isEmpty()) { + return; + } + payload = inflatedMsg.get(); + } + activateSubscriptionIfRequired(payload.getNewPayload(myFhirContext)); break; case TRANSACTION: @@ -104,7 +118,7 @@ public class SubscriptionActivatingSubscriber implements MessageHandler { */ public synchronized boolean activateSubscriptionIfRequired(final IBaseResource theSubscription) { // Grab the value for "Subscription.channel.type" so we can see if this - // subscriber applies.. + // subscriber applies. CanonicalSubscriptionChannelType subscriptionChannelType = mySubscriptionCanonicalizer.getChannelType(theSubscription); diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchDeliverer.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchDeliverer.java index 488a961c89d..d123f33e98a 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchDeliverer.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchDeliverer.java @@ -25,6 +25,7 @@ import ca.uhn.fhir.interceptor.api.HookParams; import ca.uhn.fhir.interceptor.api.IInterceptorBroadcaster; import ca.uhn.fhir.interceptor.api.Pointcut; import ca.uhn.fhir.jpa.searchparam.matcher.InMemoryMatchResult; +import ca.uhn.fhir.jpa.subscription.channel.api.PayloadTooLargeException; import ca.uhn.fhir.jpa.subscription.channel.subscription.SubscriptionChannelRegistry; import ca.uhn.fhir.jpa.subscription.match.registry.ActiveSubscription; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscription; @@ -156,8 +157,21 @@ public class SubscriptionMatchDeliverer { ourLog.warn("Failed to send message to Delivery Channel."); } } catch (RuntimeException e) { - ourLog.error("Failed to send message to Delivery Channel", e); - throw new RuntimeException(Msg.code(7) + "Failed to send message to Delivery Channel", e); + if (e.getCause() instanceof PayloadTooLargeException) { + ourLog.warn("Failed to send message to Delivery Channel because the payload size is larger than broker " + + "max message size. Retry is about to be performed without payload."); + ResourceDeliveryJsonMessage msgPayloadLess = nullOutPayload(theWrappedMsg); + trySendToDeliveryChannel(msgPayloadLess, theDeliveryChannel); + } else { + ourLog.error("Failed to send message to Delivery Channel", e); + throw new RuntimeException(Msg.code(7) + "Failed to send message to Delivery Channel", e); + } } } + + private ResourceDeliveryJsonMessage nullOutPayload(ResourceDeliveryJsonMessage theWrappedMsg) { + ResourceDeliveryMessage resourceDeliveryMessage = theWrappedMsg.getPayload(); + resourceDeliveryMessage.setPayloadToNull(); + return new ResourceDeliveryJsonMessage(resourceDeliveryMessage); + } } diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchingSubscriber.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchingSubscriber.java index 08623ae0221..f2ef0d1302e 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchingSubscriber.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/match/matcher/subscriber/SubscriptionMatchingSubscriber.java @@ -30,6 +30,7 @@ import ca.uhn.fhir.jpa.subscription.match.registry.SubscriptionRegistry; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscription; import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedJsonMessage; import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import org.hl7.fhir.instance.model.api.IBaseResource; import org.hl7.fhir.instance.model.api.IIdType; import org.slf4j.Logger; @@ -40,6 +41,7 @@ import org.springframework.messaging.MessageHandler; import org.springframework.messaging.MessagingException; import java.util.Collection; +import java.util.Optional; import javax.annotation.Nonnull; import static ca.uhn.fhir.rest.server.messaging.BaseResourceMessage.OperationTypeEnum.DELETE; @@ -64,6 +66,9 @@ public class SubscriptionMatchingSubscriber implements MessageHandler { @Autowired private SubscriptionMatchDeliverer mySubscriptionMatchDeliverer; + @Autowired + private IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; + /** * Constructor */ @@ -97,6 +102,16 @@ public class SubscriptionMatchingSubscriber implements MessageHandler { return; } + if (theMsg.getPayload(myFhirContext) == null) { + // inflate the message and ignore any resource that cannot be found. + Optional inflatedMsg = + myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(theMsg); + if (inflatedMsg.isEmpty()) { + return; + } + theMsg = inflatedMsg.get(); + } + // Interceptor call: SUBSCRIPTION_BEFORE_PERSISTED_RESOURCE_CHECKED HookParams params = new HookParams().add(ResourceModifiedMessage.class, theMsg); if (!myInterceptorBroadcaster.callHooks(Pointcut.SUBSCRIPTION_BEFORE_PERSISTED_RESOURCE_CHECKED, params)) { diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/interceptor/SynchronousSubscriptionMatcherInterceptor.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/interceptor/SynchronousSubscriptionMatcherInterceptor.java index 33d655d6a78..33861b5e205 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/interceptor/SynchronousSubscriptionMatcherInterceptor.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/interceptor/SynchronousSubscriptionMatcherInterceptor.java @@ -20,9 +20,11 @@ package ca.uhn.fhir.jpa.subscription.submit.interceptor; import ca.uhn.fhir.jpa.subscription.async.AsyncResourceModifiedProcessingSchedulerSvc; +import ca.uhn.fhir.jpa.subscription.channel.api.PayloadTooLargeException; import ca.uhn.fhir.jpa.subscription.match.matcher.matching.IResourceModifiedConsumer; import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.messaging.MessageDeliveryException; import org.springframework.transaction.support.TransactionSynchronizationAdapter; import org.springframework.transaction.support.TransactionSynchronizationManager; @@ -49,11 +51,33 @@ public class SynchronousSubscriptionMatcherInterceptor extends SubscriptionMatch @Override public void afterCommit() { - myResourceModifiedConsumer.submitResourceModified(theResourceModifiedMessage); + doSubmitResourceModified(theResourceModifiedMessage); } }); } else { + doSubmitResourceModified(theResourceModifiedMessage); + } + } + + /** + * Submit the message through the broker channel to the matcher. + * + * Note: most of our integrated tests for subscription assume we can successfully inflate the message and therefore + * does not run with an actual database to persist the data. In these cases, submitting the complete message (i.e. + * with payload) is OK. However, there are a few tests that do not assume it and do run with an actual DB. For them, + * we should null out the payload body before submitting. This try-catch block only covers the case where the + * payload is too large, which is enough for now. However, for better practice we might want to consider splitting + * this interceptor into two, each for tests with/without DB connection. + * @param theResourceModifiedMessage + */ + private void doSubmitResourceModified(ResourceModifiedMessage theResourceModifiedMessage) { + try { myResourceModifiedConsumer.submitResourceModified(theResourceModifiedMessage); + } catch (MessageDeliveryException e) { + if (e.getCause() instanceof PayloadTooLargeException) { + theResourceModifiedMessage.setPayloadToNull(); + myResourceModifiedConsumer.submitResourceModified(theResourceModifiedMessage); + } } } } diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/svc/ResourceModifiedSubmitterSvc.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/svc/ResourceModifiedSubmitterSvc.java index 99d291959ad..e57813bbc1a 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/svc/ResourceModifiedSubmitterSvc.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/subscription/submit/svc/ResourceModifiedSubmitterSvc.java @@ -35,7 +35,6 @@ import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException; import ca.uhn.fhir.subscription.api.IResourceModifiedConsumerWithRetries; import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import org.apache.commons.lang3.Validate; -import org.hl7.fhir.r5.model.IdType; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.context.event.ContextRefreshedEvent; @@ -45,8 +44,6 @@ import org.springframework.messaging.MessageDeliveryException; import org.springframework.transaction.annotation.Propagation; import org.springframework.transaction.support.TransactionCallback; -import java.util.Optional; - import static ca.uhn.fhir.jpa.subscription.match.matcher.subscriber.SubscriptionMatchingSubscriber.SUBSCRIPTION_MATCHING_CHANNEL_NAME; /** @@ -151,12 +148,11 @@ public class ResourceModifiedSubmitterSvc implements IResourceModifiedConsumer, boolean wasDeleted = deletePersistedResourceModifiedMessage( thePersistedResourceModifiedMessage.getPersistedResourceModifiedMessagePk()); - Optional optionalResourceModifiedMessage = - inflatePersistedResourceMessage(thePersistedResourceModifiedMessage); + // submit the resource modified message with empty payload, actual inflation is done by the matcher. + resourceModifiedMessage = + createResourceModifiedMessageWithoutInflation(thePersistedResourceModifiedMessage); - if (wasDeleted && optionalResourceModifiedMessage.isPresent()) { - // the PK did exist and we were able to deleted it, ie, we are the only one processing the message - resourceModifiedMessage = optionalResourceModifiedMessage.get(); + if (wasDeleted) { submitResourceModified(resourceModifiedMessage); } } catch (MessageDeliveryException exception) { @@ -186,32 +182,10 @@ public class ResourceModifiedSubmitterSvc implements IResourceModifiedConsumer, }; } - private Optional inflatePersistedResourceMessage( + private ResourceModifiedMessage createResourceModifiedMessageWithoutInflation( IPersistedResourceModifiedMessage thePersistedResourceModifiedMessage) { - ResourceModifiedMessage resourceModifiedMessage = null; - - try { - resourceModifiedMessage = myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage( - thePersistedResourceModifiedMessage); - - } catch (ResourceNotFoundException e) { - IPersistedResourceModifiedMessagePK persistedResourceModifiedMessagePk = - thePersistedResourceModifiedMessage.getPersistedResourceModifiedMessagePk(); - - IdType idType = new IdType( - thePersistedResourceModifiedMessage.getResourceType(), - persistedResourceModifiedMessagePk.getResourcePid(), - persistedResourceModifiedMessagePk.getResourceVersion()); - - ourLog.warn( - "Scheduled submission will be ignored since resource {} cannot be found", - idType.asStringValue(), - e); - } catch (Exception ex) { - ourLog.error("Unknown error encountered on inflation of resources.", ex); - } - - return Optional.ofNullable(resourceModifiedMessage); + return myResourceModifiedMessagePersistenceSvc.createResourceModifiedMessageFromEntityWithoutInflation( + thePersistedResourceModifiedMessage); } private boolean deletePersistedResourceModifiedMessage(IPersistedResourceModifiedMessagePK theResourceModifiedPK) { diff --git a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/topic/SubscriptionTopicMatchingSubscriber.java b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/topic/SubscriptionTopicMatchingSubscriber.java index f8a195ca174..758b2389078 100644 --- a/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/topic/SubscriptionTopicMatchingSubscriber.java +++ b/hapi-fhir-jpaserver-subscription/src/main/java/ca/uhn/fhir/jpa/topic/SubscriptionTopicMatchingSubscriber.java @@ -30,6 +30,7 @@ import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedJsonMessage; import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; import ca.uhn.fhir.jpa.topic.filter.InMemoryTopicFilterMatcher; import ca.uhn.fhir.rest.api.RestOperationTypeEnum; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import ca.uhn.fhir.util.Logs; import org.hl7.fhir.instance.model.api.IBaseResource; import org.hl7.fhir.r5.model.SubscriptionTopic; @@ -42,6 +43,7 @@ import org.springframework.messaging.MessagingException; import java.util.Collection; import java.util.Collections; import java.util.List; +import java.util.Optional; import javax.annotation.Nonnull; public class SubscriptionTopicMatchingSubscriber implements MessageHandler { @@ -73,6 +75,9 @@ public class SubscriptionTopicMatchingSubscriber implements MessageHandler { @Autowired private InMemoryTopicFilterMatcher myInMemoryTopicFilterMatcher; + @Autowired + private IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; + public SubscriptionTopicMatchingSubscriber(FhirContext theFhirContext) { myFhirContext = theFhirContext; } @@ -88,6 +93,16 @@ public class SubscriptionTopicMatchingSubscriber implements MessageHandler { ResourceModifiedMessage msg = ((ResourceModifiedJsonMessage) theMessage).getPayload(); + if (msg.getPayload(myFhirContext) == null) { + // inflate the message and ignore any resource that cannot be found. + Optional inflatedMsg = + myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(msg); + if (inflatedMsg.isEmpty()) { + return; + } + msg = inflatedMsg.get(); + } + // Interceptor call: SUBSCRIPTION_TOPIC_BEFORE_PERSISTED_RESOURCE_CHECKED HookParams params = new HookParams().add(ResourceModifiedMessage.class, msg); if (!myInterceptorBroadcaster.callHooks( diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriberTest.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriberTest.java index 8bfb71cb182..16789be59d3 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriberTest.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/deliver/BaseSubscriptionDeliverySubscriberTest.java @@ -8,10 +8,13 @@ import ca.uhn.fhir.interceptor.api.Pointcut; import ca.uhn.fhir.interceptor.model.RequestPartitionId; import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.fhir.jpa.api.dao.IFhirResourceDao; +import ca.uhn.fhir.jpa.model.entity.StorageSettings; import ca.uhn.fhir.jpa.searchparam.MatchUrlService; import ca.uhn.fhir.jpa.searchparam.SearchParameterMap; import ca.uhn.fhir.jpa.subscription.channel.api.IChannelFactory; import ca.uhn.fhir.jpa.subscription.channel.api.IChannelProducer; +import ca.uhn.fhir.jpa.subscription.match.deliver.email.IEmailSender; +import ca.uhn.fhir.jpa.subscription.match.deliver.email.SubscriptionDeliveringEmailSubscriber; import ca.uhn.fhir.jpa.subscription.match.deliver.message.SubscriptionDeliveringMessageSubscriber; import ca.uhn.fhir.jpa.subscription.match.deliver.resthook.SubscriptionDeliveringRestHookSubscriber; import ca.uhn.fhir.jpa.subscription.match.registry.SubscriptionRegistry; @@ -26,6 +29,7 @@ import ca.uhn.fhir.rest.client.api.IGenericClient; import ca.uhn.fhir.rest.client.api.IRestfulClientFactory; import ca.uhn.fhir.rest.server.SimpleBundleProvider; import ca.uhn.fhir.rest.server.exceptions.InternalErrorException; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import com.fasterxml.jackson.core.JsonProcessingException; import org.hl7.fhir.r4.model.Bundle; import org.hl7.fhir.r4.model.IdType; @@ -33,6 +37,8 @@ import org.hl7.fhir.r4.model.Patient; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; import org.mockito.Answers; import org.mockito.ArgumentCaptor; import org.mockito.ArgumentMatchers; @@ -57,6 +63,7 @@ import static org.hamcrest.Matchers.hasSize; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.fail; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyString; @@ -71,6 +78,7 @@ public class BaseSubscriptionDeliverySubscriberTest { private SubscriptionDeliveringRestHookSubscriber mySubscriber; private SubscriptionDeliveringMessageSubscriber myMessageSubscriber; + private SubscriptionDeliveringEmailSubscriber myEmailSubscriber; private final FhirContext myCtx = FhirContext.forR4(); @Mock @@ -96,6 +104,12 @@ public class BaseSubscriptionDeliverySubscriberTest { @Mock private MatchUrlService myMatchUrlService; + @Mock + private IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; + + @Mock + private IEmailSender myEmailSender; + @BeforeEach public void before() { mySubscriber = new SubscriptionDeliveringRestHookSubscriber(); @@ -109,8 +123,15 @@ public class BaseSubscriptionDeliverySubscriberTest { myMessageSubscriber.setSubscriptionRegistryForUnitTest(mySubscriptionRegistry); myMessageSubscriber.setDaoRegistryForUnitTest(myDaoRegistry); myMessageSubscriber.setMatchUrlServiceForUnitTest(myMatchUrlService); + myMessageSubscriber.setResourceModifiedMessagePersistenceSvcForUnitTest(myResourceModifiedMessagePersistenceSvc); myCtx.setRestfulClientFactory(myRestfulClientFactory); when(myRestfulClientFactory.newGenericClient(any())).thenReturn(myGenericClient); + + myEmailSubscriber = new SubscriptionDeliveringEmailSubscriber(myEmailSender); + myEmailSubscriber.setFhirContextForUnitTest(myCtx); + myEmailSubscriber.setInterceptorBroadcasterForUnitTest(myInterceptorBroadcaster); + myEmailSubscriber.setSubscriptionRegistryForUnitTest(mySubscriptionRegistry); + myEmailSubscriber.setResourceModifiedMessagePersistenceSvcForUnitTest(myResourceModifiedMessagePersistenceSvc); } @Test @@ -400,6 +421,38 @@ public class BaseSubscriptionDeliverySubscriberTest { } } + @ParameterizedTest + @ValueSource(strings = {"message", "email"}) + public void testMessageAndEmailSubscriber_whenPayloadIsNull_shouldTryInflateMessage(String theSubscriber) { + // setup + when(myInterceptorBroadcaster.callHooks(any(), any())).thenReturn(true); + + Patient patient = generatePatient(); + + CanonicalSubscription subscription = generateSubscription(); + + ResourceDeliveryMessage payload = new ResourceDeliveryMessage(); + payload.setSubscription(subscription); + payload.setPayload(myCtx, patient, EncodingEnum.JSON); + payload.setOperationType(ResourceModifiedMessage.OperationTypeEnum.CREATE); + + // mock the inflated message + when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(any())).thenReturn(any()); + + // this will null out the payload but keep the resource id and version. + payload.setPayloadToNull(); + + // execute & verify + switch (theSubscriber) { + case "message" -> + assertThrows(MessagingException.class, () -> myMessageSubscriber.handleMessage(new ResourceDeliveryJsonMessage(payload))); + case "email" -> + assertThrows(MessagingException.class, () -> myEmailSubscriber.handleMessage(new ResourceDeliveryJsonMessage(payload))); + } + + verify(myResourceModifiedMessagePersistenceSvc, times(1)).inflatePersistedResourceModifiedMessageOrNull(any()); + } + @Nonnull private Patient generatePatient() { Patient patient = new Patient(); diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/matcher/matching/DaoSubscriptionMatcherTest.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/matcher/matching/DaoSubscriptionMatcherTest.java index f1933548254..817eca140e6 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/matcher/matching/DaoSubscriptionMatcherTest.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/match/matcher/matching/DaoSubscriptionMatcherTest.java @@ -15,6 +15,7 @@ import ca.uhn.fhir.jpa.subscription.match.config.SubscriptionProcessorConfig; import ca.uhn.fhir.jpa.subscription.match.deliver.email.IEmailSender; import ca.uhn.fhir.jpa.subscription.submit.config.SubscriptionSubmitterConfig; import ca.uhn.fhir.jpa.subscription.submit.interceptor.SubscriptionQueryValidator; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.springframework.beans.factory.annotation.Autowired; @@ -90,6 +91,11 @@ public class DaoSubscriptionMatcherTest { public IEmailSender emailSender(){ return mock(IEmailSender.class); } + + @Bean + public IResourceModifiedMessagePersistenceSvc resourceModifiedMessagePersistenceSvc() { + return mock(IResourceModifiedMessagePersistenceSvc.class); + } } } diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/cache/SubscriptionRegistrySharedTest.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/cache/SubscriptionRegistrySharedTest.java index b7fa0f3df6a..1dbb4cf0721 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/cache/SubscriptionRegistrySharedTest.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/cache/SubscriptionRegistrySharedTest.java @@ -2,8 +2,10 @@ package ca.uhn.fhir.jpa.subscription.module.cache; import ca.uhn.fhir.jpa.subscription.channel.subscription.ISubscriptionDeliveryChannelNamer; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscription; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import org.hl7.fhir.dstu3.model.Subscription; import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Primary; @@ -18,6 +20,9 @@ public class SubscriptionRegistrySharedTest extends BaseSubscriptionRegistryTest private static final String OTHER_ID = "OTHER_ID"; + @Autowired + private IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; + @Configuration public static class SpringConfig { diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/config/TestSubscriptionDstu3Config.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/config/TestSubscriptionDstu3Config.java index b53918743a8..76fd776b5e7 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/config/TestSubscriptionDstu3Config.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/config/TestSubscriptionDstu3Config.java @@ -6,6 +6,7 @@ import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.fhir.jpa.api.dao.IFhirResourceDao; import ca.uhn.fhir.jpa.model.sched.ISchedulerService; import ca.uhn.fhir.jpa.searchparam.registry.ISearchParamProvider; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import com.google.common.collect.Lists; import org.hl7.fhir.dstu3.model.Subscription; import org.slf4j.Logger; @@ -62,4 +63,9 @@ public class TestSubscriptionDstu3Config { return mock; } + @Bean + public IResourceModifiedMessagePersistenceSvc resourceModifiedMessagePersistenceSvc() { + return mock(IResourceModifiedMessagePersistenceSvc.class); + } + } diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/standalone/BaseBlockingQueueSubscribableChannelDstu3Test.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/standalone/BaseBlockingQueueSubscribableChannelDstu3Test.java index e5fa3a5ee8e..b081bbdb37f 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/standalone/BaseBlockingQueueSubscribableChannelDstu3Test.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/standalone/BaseBlockingQueueSubscribableChannelDstu3Test.java @@ -26,6 +26,7 @@ import ca.uhn.fhir.rest.api.Constants; import ca.uhn.fhir.rest.api.MethodOutcome; import ca.uhn.fhir.rest.server.IResourceProvider; import ca.uhn.fhir.rest.server.RestfulServer; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import ca.uhn.fhir.test.utilities.JettyUtil; import ca.uhn.test.concurrency.IPointcutLatch; import ca.uhn.test.concurrency.PointcutLatch; @@ -54,6 +55,10 @@ import javax.servlet.http.HttpServletRequest; import java.util.ArrayList; import java.util.Collections; import java.util.List; +import java.util.Optional; + +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.when; public abstract class BaseBlockingQueueSubscribableChannelDstu3Test extends BaseSubscriptionDstu3Test { public static final ChannelConsumerSettings CONSUMER_OPTIONS = new ChannelConsumerSettings().setConcurrentConsumers(1); @@ -100,6 +105,8 @@ public abstract class BaseBlockingQueueSubscribableChannelDstu3Test extends Base IInterceptorService myInterceptorRegistry; @Autowired private ISubscriptionDeliveryChannelNamer mySubscriptionDeliveryChannelNamer; + @Autowired + private IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; @BeforeEach public void beforeReset() { @@ -140,6 +147,8 @@ public abstract class BaseBlockingQueueSubscribableChannelDstu3Test extends Base public T sendResource(T theResource, RequestPartitionId theRequestPartitionId) throws InterruptedException { ResourceModifiedMessage msg = new ResourceModifiedMessage(myFhirContext, theResource, ResourceModifiedMessage.OperationTypeEnum.CREATE, null, theRequestPartitionId); ResourceModifiedJsonMessage message = new ResourceModifiedJsonMessage(msg); + when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(any())).thenReturn(Optional.of(msg)); + mySubscriptionMatchingPost.setExpectedCount(1); ourSubscribableChannel.send(message); mySubscriptionMatchingPost.awaitExpected(); diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/SubscriptionMatchingSubscriberTest.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/SubscriptionMatchingSubscriberTest.java index 6f7710b042f..7730df2e400 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/SubscriptionMatchingSubscriberTest.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/SubscriptionMatchingSubscriberTest.java @@ -17,6 +17,7 @@ import ca.uhn.fhir.jpa.subscription.module.standalone.BaseBlockingQueueSubscriba import ca.uhn.fhir.model.primitive.IdDt; import ca.uhn.fhir.rest.api.Constants; import ca.uhn.fhir.rest.server.messaging.BaseResourceModifiedMessage; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import ca.uhn.fhir.util.HapiExtensions; import com.google.common.collect.Lists; import org.hl7.fhir.dstu3.model.BooleanType; @@ -33,6 +34,7 @@ import org.mockito.Mockito; import java.util.Collections; import java.util.List; +import java.util.Optional; import static ca.uhn.fhir.jpa.subscription.match.matcher.subscriber.SubscriptionCriteriaParser.TypeEnum.STARTYPE_EXPRESSION; import static org.junit.jupiter.api.Assertions.assertEquals; @@ -434,6 +436,8 @@ public class SubscriptionMatchingSubscriberTest extends BaseBlockingQueueSubscri SubscriptionCriteriaParser.SubscriptionCriteria mySubscriptionCriteria; @Mock SubscriptionMatchDeliverer mySubscriptionMatchDeliverer; + @Mock + IResourceModifiedMessagePersistenceSvc myResourceModifiedMessagePersistenceSvc; @InjectMocks SubscriptionMatchingSubscriber subscriber; @@ -445,6 +449,7 @@ public class SubscriptionMatchingSubscriberTest extends BaseBlockingQueueSubscri when(myInterceptorBroadcaster.callHooks( eq(Pointcut.SUBSCRIPTION_BEFORE_PERSISTED_RESOURCE_CHECKED), any(HookParams.class))).thenReturn(true); when(mySubscriptionRegistry.getAll()).thenReturn(Collections.emptyList()); + when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(any())).thenReturn(Optional.ofNullable(message)); subscriber.matchActiveSubscriptionsAndDeliver(message); @@ -465,6 +470,7 @@ public class SubscriptionMatchingSubscriberTest extends BaseBlockingQueueSubscri when(myActiveSubscription.getCriteria()).thenReturn(mySubscriptionCriteria); when(myActiveSubscription.getId()).thenReturn("Patient/123"); when(mySubscriptionCriteria.getType()).thenReturn(STARTYPE_EXPRESSION); + when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(any())).thenReturn(Optional.ofNullable(message)); subscriber.matchActiveSubscriptionsAndDeliver(message); @@ -486,6 +492,7 @@ public class SubscriptionMatchingSubscriberTest extends BaseBlockingQueueSubscri when(myNonDeleteSubscription.getCriteria()).thenReturn(mySubscriptionCriteria); when(myNonDeleteSubscription.getId()).thenReturn("Patient/123"); when(mySubscriptionCriteria.getType()).thenReturn(STARTYPE_EXPRESSION); + when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(any())).thenReturn(Optional.ofNullable(message)); subscriber.matchActiveSubscriptionsAndDeliver(message); @@ -505,6 +512,7 @@ public class SubscriptionMatchingSubscriberTest extends BaseBlockingQueueSubscri when(myActiveSubscription.getId()).thenReturn("Patient/123"); when(mySubscriptionCriteria.getType()).thenReturn(STARTYPE_EXPRESSION); when(myCanonicalSubscription.getSendDeleteMessages()).thenReturn(true); + when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessageOrNull(any())).thenReturn(Optional.ofNullable(message)); subscriber.matchActiveSubscriptionsAndDeliver(message); diff --git a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/websocket/WebsocketConnectionValidatorTest.java b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/websocket/WebsocketConnectionValidatorTest.java index a14e8794964..4bf3e1fcf3b 100644 --- a/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/websocket/WebsocketConnectionValidatorTest.java +++ b/hapi-fhir-jpaserver-subscription/src/test/java/ca/uhn/fhir/jpa/subscription/module/subscriber/websocket/WebsocketConnectionValidatorTest.java @@ -20,6 +20,7 @@ import ca.uhn.fhir.jpa.subscription.match.registry.SubscriptionRegistry; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscription; import ca.uhn.fhir.jpa.subscription.model.CanonicalSubscriptionChannelType; import ca.uhn.fhir.rest.server.util.ISearchParamRegistry; +import ca.uhn.fhir.subscription.api.IResourceModifiedMessagePersistenceSvc; import org.hl7.fhir.r4.model.IdType; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -146,5 +147,10 @@ public class WebsocketConnectionValidatorTest { return mock(IEmailSender.class); } + @Bean + public IResourceModifiedMessagePersistenceSvc resourceModifiedMessagePersistenceSvc(){ + return mock(IResourceModifiedMessagePersistenceSvc.class); + } + } } diff --git a/hapi-fhir-jpaserver-test-dstu2/src/test/java/ca/uhn/fhir/jpa/dao/dstu2/FhirResourceDaoDstu2Test.java b/hapi-fhir-jpaserver-test-dstu2/src/test/java/ca/uhn/fhir/jpa/dao/dstu2/FhirResourceDaoDstu2Test.java index 7037f682ec5..6fbd1c642c9 100644 --- a/hapi-fhir-jpaserver-test-dstu2/src/test/java/ca/uhn/fhir/jpa/dao/dstu2/FhirResourceDaoDstu2Test.java +++ b/hapi-fhir-jpaserver-test-dstu2/src/test/java/ca/uhn/fhir/jpa/dao/dstu2/FhirResourceDaoDstu2Test.java @@ -2189,21 +2189,21 @@ public class FhirResourceDaoDstu2Test extends BaseJpaDstu2Test { pm.setSort(new SortSpec(BaseResource.SP_RES_ID)); actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); assertEquals(5, actual.size()); - assertThat(actual, contains(idMethodName, id1, id2, id3, id4)); + assertThat(actual, contains(id1, id2, id3, id4, idMethodName)); pm = new SearchParameterMap(); pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); pm.setSort(new SortSpec(BaseResource.SP_RES_ID).setOrder(SortOrderEnum.ASC)); actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); assertEquals(5, actual.size()); - assertThat(actual, contains(idMethodName, id1, id2, id3, id4)); + assertThat(actual, contains(id1, id2, id3, id4, idMethodName)); pm = new SearchParameterMap(); pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); pm.setSort(new SortSpec(BaseResource.SP_RES_ID).setOrder(SortOrderEnum.DESC)); actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); assertEquals(5, actual.size()); - assertThat(actual, contains(id4, id3, id2, id1, idMethodName)); + assertThat(actual, contains(idMethodName, id4, id3, id2, id1)); } @Test diff --git a/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3SearchNoFtTest.java b/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3SearchNoFtTest.java index 7c2d95ac116..db98eb767bd 100644 --- a/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3SearchNoFtTest.java +++ b/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3SearchNoFtTest.java @@ -3323,7 +3323,7 @@ public class FhirResourceDaoDstu3SearchNoFtTest extends BaseJpaDstu3Test { map = new SearchParameterMap(); map.setSort(new SortSpec("_id", SortOrderEnum.ASC)); ids = toUnqualifiedVersionlessIdValues(myPatientDao.search(map)); - assertThat(ids, contains("Patient/AA", "Patient/AB", id1, id2)); + assertThat(ids, contains(id1, id2, "Patient/AA", "Patient/AB")); } diff --git a/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3Test.java b/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3Test.java index 184a18fb8aa..7b2b699d85f 100644 --- a/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3Test.java +++ b/hapi-fhir-jpaserver-test-dstu3/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirResourceDaoDstu3Test.java @@ -2788,21 +2788,21 @@ public class FhirResourceDaoDstu3Test extends BaseJpaDstu3Test { pm.setSort(new SortSpec(IAnyResource.SP_RES_ID)); actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); assertEquals(5, actual.size()); - assertThat(actual.toString(), actual, contains(idMethodName, id1, id2, id3, id4)); + assertThat(actual.toString(), actual, contains(id1, id2, id3, id4, idMethodName)); pm = new SearchParameterMap(); pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); pm.setSort(new SortSpec(IAnyResource.SP_RES_ID).setOrder(SortOrderEnum.ASC)); actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); assertEquals(5, actual.size()); - assertThat(actual.toString(), actual, contains(idMethodName, id1, id2, id3, id4)); + assertThat(actual.toString(), actual, contains(id1, id2, id3, id4, idMethodName)); pm = new SearchParameterMap(); pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); pm.setSort(new SortSpec(IAnyResource.SP_RES_ID).setOrder(SortOrderEnum.DESC)); actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); assertEquals(5, actual.size()); - assertThat(actual.toString(), actual, contains(id4, id3, id2, id1, idMethodName)); + assertThat(actual.toString(), actual, contains(idMethodName, id4, id3, id2, id1)); } @Test diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/BasePartitioningR4Test.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/BasePartitioningR4Test.java index 2700b1b2489..e91e3421bf2 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/BasePartitioningR4Test.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/BasePartitioningR4Test.java @@ -138,7 +138,7 @@ public abstract class BasePartitioningR4Test extends BaseJpaR4SystemTest { protected void dropForcedIdUniqueConstraint() { runInTransaction(() -> { myEntityManager.createNativeQuery("alter table " + ForcedId.HFJ_FORCED_ID + " drop constraint " + ForcedId.IDX_FORCEDID_TYPE_FID).executeUpdate(); - myEntityManager.createNativeQuery("alter table " + ResourceTable.HFJ_RESOURCE + " drop constraint " + ResourceTable.IDX_RES_FHIR_ID).executeUpdate(); + myEntityManager.createNativeQuery("alter table " + ResourceTable.HFJ_RESOURCE + " drop constraint " + ResourceTable.IDX_RES_TYPE_FHIR_ID).executeUpdate(); }); myHaveDroppedForcedIdUniqueConstraint = true; } diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4QuerySandbox.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4QuerySandbox.java new file mode 100644 index 00000000000..7efc920ccaf --- /dev/null +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4QuerySandbox.java @@ -0,0 +1,150 @@ +package ca.uhn.fhir.jpa.dao.r4; + +import ca.uhn.fhir.context.FhirContext; +import ca.uhn.fhir.interceptor.api.Hook; +import ca.uhn.fhir.interceptor.api.Pointcut; +import ca.uhn.fhir.jpa.dao.TestDaoSearch; +import ca.uhn.fhir.jpa.test.BaseJpaTest; +import ca.uhn.fhir.jpa.test.config.TestHSearchAddInConfig; +import ca.uhn.fhir.jpa.test.config.TestR4Config; +import ca.uhn.fhir.jpa.util.SqlQuery; +import ca.uhn.fhir.jpa.util.SqlQueryList; +import ca.uhn.fhir.rest.api.server.RequestDetails; +import ca.uhn.fhir.storage.test.DaoTestDataBuilder; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.api.extension.RegisterExtension; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.annotation.DirtiesContext; +import org.springframework.test.context.ContextConfiguration; +import org.springframework.test.context.TestContext; +import org.springframework.test.context.TestExecutionListeners; +import org.springframework.test.context.junit.jupiter.SpringExtension; +import org.springframework.test.context.support.DependencyInjectionTestExecutionListener; +import org.springframework.test.context.support.DirtiesContextTestExecutionListener; +import org.springframework.transaction.PlatformTransactionManager; + +import java.util.ArrayList; +import java.util.List; + +import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.empty; +import static org.hamcrest.Matchers.not; + +/** + * Sandbox for implementing queries. + * This will NOT run during the build - use this class as a convenient + * place to explore, debug, profile, and optimize. + */ +@ExtendWith(SpringExtension.class) +@ContextConfiguration(classes = { + TestR4Config.class, + TestHSearchAddInConfig.NoFT.class, + DaoTestDataBuilder.Config.class, + TestDaoSearch.Config.class +}) +@DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS) +@TestExecutionListeners(listeners = { + DependencyInjectionTestExecutionListener.class + , FhirResourceDaoR4QuerySandbox.TestDirtiesContextTestExecutionListener.class +}) +public class FhirResourceDaoR4QuerySandbox extends BaseJpaTest { + private static final Logger ourLog = LoggerFactory.getLogger(FhirResourceDaoR4QuerySandbox.class); + + @Autowired + PlatformTransactionManager myTxManager; + @Autowired + FhirContext myFhirCtx; + @RegisterExtension + @Autowired + DaoTestDataBuilder myDataBuilder; + @Autowired + TestDaoSearch myTestDaoSearch; + + @Override + protected PlatformTransactionManager getTxManager() { + return myTxManager; + } + + @Override + protected FhirContext getFhirContext() { + return myFhirCtx; + } + + List myCapturedQueries = new ArrayList<>(); + @BeforeEach + void registerLoggingInterceptor() { + registerInterceptor(new Object(){ + @Hook(Pointcut.JPA_PERFTRACE_RAW_SQL) + public void captureSql(RequestDetails theRequestDetails, SqlQueryList theQueries) { + for (SqlQuery next : theQueries) { + String output = next.getSql(true, true, true); + ourLog.info("Query: {}", output); + myCapturedQueries.add(output); + } + } + }); + + } + + @Test + public void testSearches_logQueries() { + myDataBuilder.createPatient(); + + myTestDaoSearch.searchForIds("Patient?name=smith"); + + assertThat(myCapturedQueries, not(empty())); + } + + @Test + void testQueryByPid() { + + // sentinel for over-match + myDataBuilder.createPatient(); + + String id = myDataBuilder.createPatient( + myDataBuilder.withBirthdate("1971-01-01"), + myDataBuilder.withActiveTrue(), + myDataBuilder.withFamily("Smith")).getIdPart(); + + myTestDaoSearch.assertSearchFindsOnly("search by server assigned id", "Patient?_pid=" + id, id); + } + + @Test + void testQueryByPid_withOtherSPAvoidsResourceTable() { + // sentinel for over-match + myDataBuilder.createPatient(); + + String id = myDataBuilder.createPatient( + myDataBuilder.withBirthdate("1971-01-01"), + myDataBuilder.withActiveTrue(), + myDataBuilder.withFamily("Smith")).getIdPart(); + + myTestDaoSearch.assertSearchFindsOnly("search by server assigned id", "Patient?name=smith&_pid=" + id, id); + } + + @Test + void testSortByPid() { + + String id1 = myDataBuilder.createPatient(myDataBuilder.withFamily("Smithy")).getIdPart(); + String id2 = myDataBuilder.createPatient(myDataBuilder.withFamily("Smithwick")).getIdPart(); + String id3 = myDataBuilder.createPatient(myDataBuilder.withFamily("Smith")).getIdPart(); + + myTestDaoSearch.assertSearchFindsInOrder("sort by server assigned id", "Patient?family=smith&_sort=_pid", id1,id2,id3); + myTestDaoSearch.assertSearchFindsInOrder("reverse sort by server assigned id", "Patient?family=smith&_sort=-_pid", id3,id2,id1); + } + + public static final class TestDirtiesContextTestExecutionListener extends DirtiesContextTestExecutionListener { + + @Override + protected void beforeOrAfterTestClass(TestContext testContext, DirtiesContext.ClassMode requiredClassMode) throws Exception { + if (!testContext.getTestClass().getName().contains("$")) { + super.beforeOrAfterTestClass(testContext, requiredClassMode); + } + } + } + +} diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchNoFtTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchNoFtTest.java index f28fec1fb98..779cd067177 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchNoFtTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SearchNoFtTest.java @@ -176,6 +176,7 @@ import static ca.uhn.fhir.rest.param.ParamPrefixEnum.LESSTHAN; import static ca.uhn.fhir.rest.param.ParamPrefixEnum.LESSTHAN_OR_EQUALS; import static ca.uhn.fhir.rest.param.ParamPrefixEnum.NOT_EQUAL; import static ca.uhn.fhir.test.utilities.CustomMatchersUtil.assertDoesNotContainAnyOf; +import static ca.uhn.fhir.util.DateUtils.convertDateToIso8601String; import static org.apache.commons.lang3.StringUtils.countMatches; import static org.apache.commons.lang3.StringUtils.leftPad; import static org.hamcrest.CoreMatchers.is; @@ -447,6 +448,57 @@ public class FhirResourceDaoR4SearchNoFtTest extends BaseJpaR4Test { assertEquals(0, ids.size()); } + @Test + public void testHasEncounterAndLastUpdated() { + // setup + Patient patientA = new Patient(); + String patientIdA = myPatientDao.create(patientA).getId().toUnqualifiedVersionless().getValue(); + + Patient patientB = new Patient(); + String patientIdB = myPatientDao.create(patientA).getId().toUnqualifiedVersionless().getValue(); + + Encounter encounterA = new Encounter(); + encounterA.getClass_().setSystem("http://snomed.info/sct").setCode("55822004"); + encounterA.getSubject().setReference(patientIdA); + + // record time between encounter A and B + TestUtil.sleepOneClick(); + Date beforeA = new Date(); + TestUtil.sleepOneClick(); + + myEncounterDao.create(encounterA); + + Encounter encounterB = new Encounter(); + encounterB.getClass_().setSystem("http://snomed.info/sct").setCode("55822005"); + encounterB.getSubject().setReference(patientIdB); + + // record time between encounter A and B + TestUtil.sleepOneClick(); + Date beforeB = new Date(); + TestUtil.sleepOneClick(); + + myEncounterDao.create(encounterB); + + // execute + String criteriaA = "_has:Encounter:patient:_lastUpdated=ge" + convertDateToIso8601String(beforeA); + SearchParameterMap mapA = myMatchUrlService.translateMatchUrl(criteriaA, myFhirContext.getResourceDefinition(Patient.class)); + mapA.setLoadSynchronous(true); + myCaptureQueriesListener.clear(); + IBundleProvider resultA = myPatientDao.search(mapA); + myCaptureQueriesListener.logSelectQueries(); + List idsBeforeA = toUnqualifiedVersionlessIdValues(resultA); + + String criteriaB = "_has:Encounter:patient:_lastUpdated=ge" + convertDateToIso8601String(beforeB); + SearchParameterMap mapB = myMatchUrlService.translateMatchUrl(criteriaB, myFhirContext.getResourceDefinition(Patient.class)); + mapB.setLoadSynchronous(true); + IBundleProvider resultB = myPatientDao.search(mapB); + List idsBeforeB = toUnqualifiedVersionlessIdValues(resultB); + + // verify + assertEquals(2, idsBeforeA.size()); + assertEquals(1, idsBeforeB.size()); + } + @Test public void testGenderBirthdateHasCondition() { Patient patient = new Patient(); diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SortTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SortTest.java index ea781b9e4cc..23ab501a227 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SortTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4SortTest.java @@ -89,12 +89,12 @@ public class FhirResourceDaoR4SortTest extends BaseJpaR4Test { map = new SearchParameterMap(); map.setSort(new SortSpec("_id", SortOrderEnum.ASC)); ids = toUnqualifiedVersionlessIdValues(myPatientDao.search(map)); - assertThat(ids, contains("Patient/AA", "Patient/AB", id1, id2)); + assertThat(ids, contains(id1, id2, "Patient/AA", "Patient/AB")); map = new SearchParameterMap(); map.setSort(new SortSpec("_id", SortOrderEnum.DESC)); ids = toUnqualifiedVersionlessIdValues(myPatientDao.search(map)); - assertThat(ids, contains(id2, id1, "Patient/AB", "Patient/AA")); + assertThat(ids, contains("Patient/AB", "Patient/AA", id2, id1)); } @Test diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4StandardQueriesNoFTTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4StandardQueriesNoFTTest.java index 3236f6b60fd..3d23277dc19 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4StandardQueriesNoFTTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4StandardQueriesNoFTTest.java @@ -4,15 +4,17 @@ import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.fhir.jpa.api.dao.IFhirResourceDao; import ca.uhn.fhir.jpa.dao.TestDaoSearch; -import ca.uhn.fhir.jpa.search.CompositeSearchParameterTestCases; -import ca.uhn.fhir.jpa.search.QuantitySearchParameterTestCases; import ca.uhn.fhir.jpa.search.BaseSourceSearchParameterTestCases; +import ca.uhn.fhir.jpa.search.CompositeSearchParameterTestCases; +import ca.uhn.fhir.jpa.search.IIdSearchTestTemplate; +import ca.uhn.fhir.jpa.search.QuantitySearchParameterTestCases; import ca.uhn.fhir.jpa.searchparam.MatchUrlService; import ca.uhn.fhir.jpa.test.BaseJpaTest; import ca.uhn.fhir.jpa.test.config.TestHSearchAddInConfig; import ca.uhn.fhir.jpa.test.config.TestR4Config; import ca.uhn.fhir.storage.test.BaseDateSearchDaoTests; import ca.uhn.fhir.storage.test.DaoTestDataBuilder; +import ca.uhn.fhir.test.utilities.ITestDataBuilder; import ca.uhn.fhir.test.utilities.ITestDataBuilder.ICreationArgument; import org.hl7.fhir.instance.model.api.IIdType; import org.hl7.fhir.r4.model.Observation; @@ -36,9 +38,14 @@ import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.contains; import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.hasItem; -import static org.hamcrest.Matchers.hasItems; import static org.hamcrest.Matchers.not; +/** + * Verify that our query behaviour matches the spec. + * Note: we do not extend BaseJpaR4Test here. + * That does a full purge in @AfterEach which is a bit slow. + * Instead, this test tracks all created resources in DaoTestDataBuilder, and deletes them in teardown. + */ @ExtendWith(SpringExtension.class) @ContextConfiguration(classes = { TestR4Config.class, @@ -256,10 +263,10 @@ public class FhirResourceDaoR4StandardQueriesNoFTTest extends BaseJpaTest { String idExM = withObservation(myDataBuilder.withObservationCode("http://example.org", "MValue")).getIdPart(); List allIds = myTestDaoSearch.searchForIds("/Observation?_sort=code"); - assertThat(allIds, hasItems(idAlphaA, idAlphaM, idAlphaZ, idExA, idExD, idExM)); + assertThat(allIds, contains(idAlphaA, idAlphaM, idAlphaZ, idExA, idExD, idExM)); allIds = myTestDaoSearch.searchForIds("/Observation?_sort=code&code=http://example.org|"); - assertThat(allIds, hasItems(idExA, idExD, idExM)); + assertThat(allIds, contains(idExA, idExD, idExM)); } } } @@ -368,7 +375,7 @@ public class FhirResourceDaoR4StandardQueriesNoFTTest extends BaseJpaTest { String idAlpha5 = withRiskAssessmentWithProbabilty(0.5).getIdPart(); List allIds = myTestDaoSearch.searchForIds("/RiskAssessment?_sort=probability"); - assertThat(allIds, hasItems(idAlpha2, idAlpha5, idAlpha7)); + assertThat(allIds, contains(idAlpha2, idAlpha5, idAlpha7)); } } @@ -491,12 +498,51 @@ public class FhirResourceDaoR4StandardQueriesNoFTTest extends BaseJpaTest { String idAlpha5 = withObservationWithValueQuantity(0.5).getIdPart(); List allIds = myTestDaoSearch.searchForIds("/Observation?_sort=value-quantity"); - assertThat(allIds, hasItems(idAlpha2, idAlpha5, idAlpha7)); + assertThat(allIds, contains(idAlpha2, idAlpha5, idAlpha7)); } } } + @Test + void testQueryByPid() { + + // sentinel for over-match + myDataBuilder.createPatient(); + + String id = myDataBuilder.createPatient( + myDataBuilder.withBirthdate("1971-01-01"), + myDataBuilder.withActiveTrue(), + myDataBuilder.withFamily("Smith")).getIdPart(); + + myTestDaoSearch.assertSearchFindsOnly("search by server assigned id", "Patient?_pid=" + id, id); + myTestDaoSearch.assertSearchFindsOnly("search by server assigned id", "Patient?family=smith&_pid=" + id, id); + } + + @Test + void testSortByPid() { + + String id1 = myDataBuilder.createPatient(myDataBuilder.withFamily("Smithy")).getIdPart(); + String id2 = myDataBuilder.createPatient(myDataBuilder.withFamily("Smithwick")).getIdPart(); + String id3 = myDataBuilder.createPatient(myDataBuilder.withFamily("Smith")).getIdPart(); + + myTestDaoSearch.assertSearchFindsInOrder("sort by server assigned id", "Patient?family=smith&_sort=_pid", id1,id2,id3); + myTestDaoSearch.assertSearchFindsInOrder("reverse sort by server assigned id", "Patient?family=smith&_sort=-_pid", id3,id2,id1); + } + + @Nested + public class IdSearch implements IIdSearchTestTemplate { + @Override + public TestDaoSearch getSearch() { + return myTestDaoSearch; + } + + @Override + public ITestDataBuilder getBuilder() { + return myDataBuilder; + } + } + // todo mb re-enable this. Some of these fail! @Disabled @Nested diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4Test.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4Test.java index d06cba7de2b..ab11fcd27f1 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4Test.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4Test.java @@ -3352,63 +3352,6 @@ public class FhirResourceDaoR4Test extends BaseJpaR4Test { } - @Test - public void testSortById() { - String methodName = "testSortBTyId"; - - Patient p = new Patient(); - p.addIdentifier().setSystem("urn:system").setValue(methodName); - IIdType id1 = myPatientDao.create(p, mySrd).getId().toUnqualifiedVersionless(); - - p = new Patient(); - p.addIdentifier().setSystem("urn:system").setValue(methodName); - IIdType id2 = myPatientDao.create(p, mySrd).getId().toUnqualifiedVersionless(); - - p = new Patient(); - p.setId(methodName + "1"); - p.addIdentifier().setSystem("urn:system").setValue(methodName); - IIdType idMethodName1 = myPatientDao.update(p, mySrd).getId().toUnqualifiedVersionless(); - assertEquals(methodName + "1", idMethodName1.getIdPart()); - - p = new Patient(); - p.addIdentifier().setSystem("urn:system").setValue(methodName); - IIdType id3 = myPatientDao.create(p, mySrd).getId().toUnqualifiedVersionless(); - - p = new Patient(); - p.setId(methodName + "2"); - p.addIdentifier().setSystem("urn:system").setValue(methodName); - IIdType idMethodName2 = myPatientDao.update(p, mySrd).getId().toUnqualifiedVersionless(); - assertEquals(methodName + "2", idMethodName2.getIdPart()); - - p = new Patient(); - p.addIdentifier().setSystem("urn:system").setValue(methodName); - IIdType id4 = myPatientDao.create(p, mySrd).getId().toUnqualifiedVersionless(); - - SearchParameterMap pm; - List actual; - - pm = SearchParameterMap.newSynchronous(); - pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); - pm.setSort(new SortSpec(IAnyResource.SP_RES_ID)); - actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); - assertEquals(6, actual.size()); - assertThat(actual, contains(idMethodName1, idMethodName2, id1, id2, id3, id4)); - - pm = SearchParameterMap.newSynchronous(); - pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); - pm.setSort(new SortSpec(IAnyResource.SP_RES_ID).setOrder(SortOrderEnum.ASC)); - actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); - assertEquals(6, actual.size()); - assertThat(actual, contains(idMethodName1, idMethodName2, id1, id2, id3, id4)); - - pm = SearchParameterMap.newSynchronous(); - pm.add(Patient.SP_IDENTIFIER, new TokenParam("urn:system", methodName)); - pm.setSort(new SortSpec(IAnyResource.SP_RES_ID).setOrder(SortOrderEnum.DESC)); - actual = toUnqualifiedVersionlessIds(myPatientDao.search(pm)); - assertEquals(6, actual.size()); - assertThat(actual, contains(id4, id3, id2, id1, idMethodName2, idMethodName1)); - } - @ParameterizedTest @ValueSource(booleans = {true, false}) public void testSortByMissingAttribute(boolean theIndexMissingData) { diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/SearchCoordinatorSvcImplTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/SearchCoordinatorSvcImplTest.java index 18657d1d5b0..26ff93cc64f 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/SearchCoordinatorSvcImplTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/dao/r4/SearchCoordinatorSvcImplTest.java @@ -1,7 +1,6 @@ package ca.uhn.fhir.jpa.dao.r4; import ca.uhn.fhir.interceptor.model.RequestPartitionId; -import ca.uhn.fhir.jpa.api.svc.ISearchCoordinatorSvc; import ca.uhn.fhir.jpa.dao.data.ISearchDao; import ca.uhn.fhir.jpa.dao.data.ISearchResultDao; import ca.uhn.fhir.jpa.entity.Search; @@ -16,6 +15,8 @@ import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.Test; import org.springframework.beans.factory.annotation.Autowired; +import java.time.Instant; +import java.time.temporal.ChronoUnit; import java.util.Date; import java.util.UUID; @@ -31,22 +32,20 @@ public class SearchCoordinatorSvcImplTest extends BaseJpaR4Test { @Autowired private ISearchResultDao mySearchResultDao; - @Autowired - private ISearchCoordinatorSvc mySearchCoordinator; - @Autowired private ISearchCacheSvc myDatabaseCacheSvc; @AfterEach public void after() { DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOnePassForUnitTest(DatabaseSearchCacheSvcImpl.DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_PAS); - DatabaseSearchCacheSvcImpl.setMaximumSearchesToCheckForDeletionCandidacyForUnitTest(DEFAULT_MAX_DELETE_CANDIDATES_TO_FIND); } + /** + * Semi-obsolete test. This used to test incremental deletion, but we now work until done or a timeout. + */ @Test public void testDeleteDontMarkPreviouslyMarkedSearchesAsDeleted() { DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOnePassForUnitTest(5); - DatabaseSearchCacheSvcImpl.setMaximumSearchesToCheckForDeletionCandidacyForUnitTest(10); runInTransaction(()->{ mySearchResultDao.deleteAll(); @@ -86,28 +85,12 @@ public class SearchCoordinatorSvcImplTest extends BaseJpaR4Test { assertEquals(30, mySearchResultDao.count()); }); - myDatabaseCacheSvc.pollForStaleSearchesAndDeleteThem(RequestPartitionId.allPartitions()); + myDatabaseCacheSvc.pollForStaleSearchesAndDeleteThem(RequestPartitionId.allPartitions(), Instant.now().plus(10, ChronoUnit.SECONDS)); runInTransaction(()->{ // We should delete up to 10, but 3 don't get deleted since they have too many results to delete in one pass - assertEquals(13, mySearchDao.count()); - assertEquals(3, mySearchDao.countDeleted()); - // We delete a max of 5 results per search, so half are gone - assertEquals(15, mySearchResultDao.count()); - }); - - myDatabaseCacheSvc.pollForStaleSearchesAndDeleteThem(RequestPartitionId.allPartitions()); - runInTransaction(()->{ - // Once again we attempt to delete 10, but the first 3 don't get deleted and still remain - // (total is 6 because 3 weren't deleted, and they blocked another 3 that might have been) - assertEquals(6, mySearchDao.count()); - assertEquals(6, mySearchDao.countDeleted()); - assertEquals(0, mySearchResultDao.count()); - }); - - myDatabaseCacheSvc.pollForStaleSearchesAndDeleteThem(RequestPartitionId.allPartitions()); - runInTransaction(()->{ assertEquals(0, mySearchDao.count()); assertEquals(0, mySearchDao.countDeleted()); + // We delete a max of 5 results per search, so half are gone assertEquals(0, mySearchResultDao.count()); }); } diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplCreateTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplCreateTest.java index da9b8ee62e0..25d450771d7 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplCreateTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplCreateTest.java @@ -22,7 +22,6 @@ import java.io.ByteArrayOutputStream; import java.io.IOException; import java.nio.charset.StandardCharsets; import java.util.List; -import java.util.stream.Collectors; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; @@ -53,7 +52,7 @@ public class PackageInstallerSvcImplCreateTest extends BaseJpaR4Test { final NamingSystem namingSystem = new NamingSystem(); namingSystem.getUniqueId().add(new NamingSystem.NamingSystemUniqueIdComponent().setValue("123")); - create(namingSystem); + install(namingSystem); assertEquals(1, myNamingSystemDao.search(SearchParameterMap.newSynchronous(), REQUEST_DETAILS).getAllResources().size()); } @@ -184,7 +183,7 @@ public class PackageInstallerSvcImplCreateTest extends BaseJpaR4Test { } private void createValueSetAndCallCreate(String theOid, String theResourceVersion, String theValueSetVersion, String theUrl, String theCopyright) throws IOException { - create(createValueSet(theOid, theResourceVersion, theValueSetVersion, theUrl, theCopyright)); + install(createValueSet(theOid, theResourceVersion, theValueSetVersion, theUrl, theCopyright)); } @Nonnull @@ -199,8 +198,8 @@ public class PackageInstallerSvcImplCreateTest extends BaseJpaR4Test { return valueSetFromFirstIg; } - private void create(IBaseResource theResource) throws IOException { - mySvc.create(theResource, createInstallationSpec(packageToBytes()), new PackageInstallOutcomeJson()); + private void install(IBaseResource theResource) throws IOException { + mySvc.install(theResource, createInstallationSpec(packageToBytes()), new PackageInstallOutcomeJson()); } @Nonnull diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplRewriteHistoryTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplRewriteHistoryTest.java index b1c88f6f508..aff55ed8ed3 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplRewriteHistoryTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/packages/PackageInstallerSvcImplRewriteHistoryTest.java @@ -38,7 +38,7 @@ public class PackageInstallerSvcImplRewriteHistoryTest extends BaseJpaR4Test { // execute // red-green this threw a NPE before the fix - mySvc.updateResource(myConceptMapDao, conceptMap); + mySvc.createOrUpdateResource(myConceptMapDao, conceptMap, null); // verify ConceptMap readConceptMap = myConceptMapDao.read(CONCEPT_MAP_TEST_ID); diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/BinaryStorageInterceptorR4Test.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/BinaryStorageInterceptorR4Test.java index 91d4a521037..273b382d924 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/BinaryStorageInterceptorR4Test.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/BinaryStorageInterceptorR4Test.java @@ -10,6 +10,7 @@ import ca.uhn.fhir.jpa.binary.interceptor.BinaryStorageInterceptor; import ca.uhn.fhir.jpa.binstore.MemoryBinaryStorageSvcImpl; import ca.uhn.fhir.jpa.model.entity.StorageSettings; import ca.uhn.fhir.jpa.provider.BaseResourceProviderR4Test; +import ca.uhn.fhir.model.primitive.IdDt; import ca.uhn.fhir.rest.api.RestOperationTypeEnum; import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.client.api.IClientInterceptor; @@ -18,13 +19,16 @@ import ca.uhn.fhir.rest.client.api.IHttpResponse; import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException; import ca.uhn.fhir.util.HapiExtensions; import org.hl7.fhir.instance.model.api.IBaseHasExtensions; -import org.hl7.fhir.instance.model.api.IBaseMetaType; import org.hl7.fhir.instance.model.api.IBaseResource; import org.hl7.fhir.instance.model.api.IIdType; import org.hl7.fhir.r4.model.Binary; +import org.hl7.fhir.r4.model.Bundle; import org.hl7.fhir.r4.model.DocumentReference; import org.hl7.fhir.r4.model.Enumerations; import org.hl7.fhir.r4.model.Extension; +import org.hl7.fhir.r4.model.IdType; +import org.hl7.fhir.r4.model.Patient; +import org.hl7.fhir.r4.model.Resource; import org.hl7.fhir.r4.model.StringType; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; @@ -32,7 +36,6 @@ import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.junit.jupiter.params.ParameterizedTest; import org.junit.jupiter.params.provider.EnumSource; -import org.junit.jupiter.params.provider.ValueSource; import org.mockito.junit.jupiter.MockitoExtension; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -62,6 +65,7 @@ public class BinaryStorageInterceptorR4Test extends BaseResourceProviderR4Test { public static final byte[] FEW_BYTES = {4, 3, 2, 1}; public static final byte[] SOME_BYTES = {1, 2, 3, 4, 5, 6, 7, 8, 7, 6, 5, 4, 3, 2, 1, 8, 9, 0, 10, 9}; public static final byte[] SOME_BYTES_2 = {6, 7, 8, 7, 6, 5, 4, 3, 2, 1, 5, 5, 5, 6}; + public static final byte[] SOME_BYTES_3 = {5, 5, 5, 6, 6, 7, 7, 7, 8, 8, 8}; private static final Logger ourLog = LoggerFactory.getLogger(BinaryStorageInterceptorR4Test.class); @Autowired @@ -381,12 +385,8 @@ public class BinaryStorageInterceptorR4Test extends BaseResourceProviderR4Test { // Create a resource with a big enough docRef DocumentReference docRef = new DocumentReference(); - DocumentReference.DocumentReferenceContentComponent content = docRef.addContent(); - content.getAttachment().setContentType("application/octet-stream"); - content.getAttachment().setData(SOME_BYTES); - DocumentReference.DocumentReferenceContentComponent content2 = docRef.addContent(); - content2.getAttachment().setContentType("application/octet-stream"); - content2.getAttachment().setData(SOME_BYTES_2); + addDocumentAttachmentData(docRef, SOME_BYTES); + addDocumentAttachmentData(docRef, SOME_BYTES_2); DaoMethodOutcome outcome = myDocumentReferenceDao.create(docRef, mySrd); // Make sure it was externalized @@ -422,18 +422,73 @@ public class BinaryStorageInterceptorR4Test extends BaseResourceProviderR4Test { } + @Test + public void testCreateBinaryAttachments_bundleWithMultipleDocumentReferences_createdAndReadBackSuccessfully() { + // Create Patient + Patient patient = new Patient(); + patient.addIdentifier().setSystem("urn:system").setValue("001"); + patient.addName().addGiven("Johnny").setFamily("Walker"); + + // Create first DocumentReference with a big enough attachments + DocumentReference docRef = new DocumentReference(); + addDocumentAttachmentData(docRef, SOME_BYTES); + addDocumentAttachmentData(docRef, SOME_BYTES_2); + + // Create second DocumentReference with a big enough attachment + DocumentReference docRef2 = new DocumentReference(); + addDocumentAttachmentData(docRef2, SOME_BYTES_3); + + // Create Bundle + Bundle bundle = new Bundle(); + bundle.setType(Bundle.BundleType.TRANSACTION); + // Patient entry component + addBundleEntry(bundle, patient, "Patient"); + // First DocumentReference entry component + addBundleEntry(bundle, docRef, "DocumentReference"); + // Second DocumentReference entry component + addBundleEntry(bundle, docRef2, "DocumentReference"); + + // Execute transaction + Bundle output = myClient.transaction().withBundle(bundle).execute(); + ourLog.debug(myFhirContext.newXmlParser().setPrettyPrint(true).encodeResourceToString(output)); + + // Verify bundle response + assertEquals(3, output.getEntry().size()); + output.getEntry().forEach(entry -> assertEquals("201 Created", entry.getResponse().getStatus())); + + // Read back and verify first DocumentReference and attachments + IIdType firstDocRef = new IdType(output.getEntry().get(1).getResponse().getLocation()); + DocumentReference firstDoc = myDocumentReferenceDao.read(firstDocRef, mySrd); + assertEquals("application/octet-stream", firstDoc.getContentFirstRep().getAttachment().getContentType()); + assertArrayEquals(SOME_BYTES, firstDoc.getContentFirstRep().getAttachment().getData()); + assertEquals("application/octet-stream", firstDoc.getContent().get(1).getAttachment().getContentType()); + assertArrayEquals(SOME_BYTES_2, firstDoc.getContent().get(1).getAttachment().getData()); + + // Read back and verify second DocumentReference and attachment + IIdType secondDocRef = new IdType(output.getEntry().get(2).getResponse().getLocation()); + DocumentReference secondDoc = myDocumentReferenceDao.read(secondDocRef, mySrd); + assertEquals("application/octet-stream", secondDoc.getContentFirstRep().getAttachment().getContentType()); + assertArrayEquals(SOME_BYTES_3, secondDoc.getContentFirstRep().getAttachment().getData()); + } + + private void addBundleEntry(Bundle theBundle, Resource theResource, String theUrl) { + Bundle.BundleEntryComponent getComponent = new Bundle.BundleEntryComponent(); + Bundle.BundleEntryRequestComponent requestComponent = new Bundle.BundleEntryRequestComponent(); + requestComponent.setMethod(Bundle.HTTPVerb.POST); + requestComponent.setUrl(theUrl); + getComponent.setRequest(requestComponent); + getComponent.setResource(theResource); + getComponent.setFullUrl(IdDt.newRandomUuid().getValue()); + theBundle.addEntry(getComponent); + } @Test public void testUpdateRejectsIncorrectBinary() { // Create a resource with a big enough docRef DocumentReference docRef = new DocumentReference(); - DocumentReference.DocumentReferenceContentComponent content = docRef.addContent(); - content.getAttachment().setContentType("application/octet-stream"); - content.getAttachment().setData(SOME_BYTES); - DocumentReference.DocumentReferenceContentComponent content2 = docRef.addContent(); - content2.getAttachment().setContentType("application/octet-stream"); - content2.getAttachment().setData(SOME_BYTES_2); + addDocumentAttachmentData(docRef, SOME_BYTES); + addDocumentAttachmentData(docRef, SOME_BYTES_2); DaoMethodOutcome outcome = myDocumentReferenceDao.create(docRef, mySrd); // Make sure it was externalized @@ -449,13 +504,13 @@ public class BinaryStorageInterceptorR4Test extends BaseResourceProviderR4Test { docRef = new DocumentReference(); docRef.setId(id.toUnqualifiedVersionless()); docRef.setStatus(Enumerations.DocumentReferenceStatus.CURRENT); - content = docRef.addContent(); + DocumentReference.DocumentReferenceContentComponent content = docRef.addContent(); content.getAttachment().setContentType("application/octet-stream"); content.getAttachment().getDataElement().addExtension( HapiExtensions.EXT_EXTERNALIZED_BINARY_ID, new StringType(binaryId) ); - content2 = docRef.addContent(); + DocumentReference.DocumentReferenceContentComponent content2 = docRef.addContent(); content2.getAttachment().setContentType("application/octet-stream"); content2.getAttachment().getDataElement().addExtension( HapiExtensions.EXT_EXTERNALIZED_BINARY_ID, @@ -497,5 +552,10 @@ public class BinaryStorageInterceptorR4Test extends BaseResourceProviderR4Test { } + private void addDocumentAttachmentData(DocumentReference theDocumentReference, byte[] theData) { + DocumentReference.DocumentReferenceContentComponent content = theDocumentReference.addContent(); + content.getAttachment().setContentType("application/octet-stream"); + content.getAttachment().setData(theData); + } } diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/ConsentInterceptorResourceProviderR4IT.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/ConsentInterceptorResourceProviderR4IT.java index 6fd56b30ee4..52e288f4727 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/ConsentInterceptorResourceProviderR4IT.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/ConsentInterceptorResourceProviderR4IT.java @@ -12,6 +12,7 @@ import ca.uhn.fhir.rest.api.Constants; import ca.uhn.fhir.rest.api.PreferReturnEnum; import ca.uhn.fhir.rest.api.RestOperationTypeEnum; import ca.uhn.fhir.rest.api.server.RequestDetails; +import ca.uhn.fhir.rest.client.api.IHttpResponse; import ca.uhn.fhir.rest.client.interceptor.CapturingInterceptor; import ca.uhn.fhir.rest.gclient.StringClientParam; import ca.uhn.fhir.rest.server.exceptions.BaseServerResponseException; @@ -66,12 +67,15 @@ import java.util.Arrays; import java.util.List; import java.util.stream.Collectors; +import static org.apache.commons.lang3.StringUtils.isEmpty; import static org.apache.commons.lang3.StringUtils.leftPad; import static org.awaitility.Awaitility.await; -import static org.hamcrest.CoreMatchers.containsString; -import static org.hamcrest.CoreMatchers.equalTo; -import static org.hamcrest.CoreMatchers.is; -import static org.hamcrest.CoreMatchers.not; +import static org.hamcrest.Matchers.containsString; +import static org.hamcrest.Matchers.equalTo; +import static org.hamcrest.Matchers.hasItem; +import static org.hamcrest.Matchers.is; +import static org.hamcrest.Matchers.empty; +import static org.hamcrest.Matchers.not; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.blankOrNullString; import static org.hamcrest.Matchers.hasSize; @@ -189,8 +193,64 @@ public class ConsentInterceptorResourceProviderR4IT extends BaseResourceProvider myServer.getRestfulServer().getInterceptorService().registerInterceptor(myConsentInterceptor); // Perform a search and only allow even + String context = "active consent - hide odd"; consentService.setTarget(new ConsentSvcCantSeeOddNumbered()); - Bundle result = myClient + List returnedIdValues = searchForObservations(); + assertEquals(myObservationIdsEvenOnly.subList(0, 15), returnedIdValues); + assertResponseIsNotFromCache(context, capture.getLastResponse()); + + // Perform a search and only allow odd + context = "active consent - hide even"; + consentService.setTarget(new ConsentSvcCantSeeEvenNumbered()); + returnedIdValues = searchForObservations(); + assertEquals(myObservationIdsOddOnly.subList(0, 15), returnedIdValues); + assertResponseIsNotFromCache(context, capture.getLastResponse()); + + // Perform a search and allow all with a PROCEED + context = "active consent - PROCEED on cache"; + consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.PROCEED)); + returnedIdValues = searchForObservations(); + assertEquals(myObservationIds.subList(0, 15), returnedIdValues); + assertResponseIsNotFromCache(context, capture.getLastResponse()); + + // Perform a search and allow all with an AUTHORIZED (no further checking) + context = "active consent - AUTHORIZED after a PROCEED"; + consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.AUTHORIZED)); + returnedIdValues = searchForObservations(); + assertEquals(myObservationIds.subList(0, 15), returnedIdValues); + + // Perform a second search and allow all with an AUTHORIZED (no further checking) + // which means we should finally get one from the cache + context = "active consent - AUTHORIZED after AUTHORIZED"; + consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.AUTHORIZED)); + returnedIdValues = searchForObservations(); + assertEquals(myObservationIds.subList(0, 15), returnedIdValues); + assertResponseIsFromCache(context, capture.getLastResponse()); + + // Perform another search, now with an active consent interceptor that promises not to use canSeeResource. + // Should re-use cache result + context = "active consent - canSeeResource disabled, after AUTHORIZED - should reuse cache"; + consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.PROCEED, false)); + returnedIdValues = searchForObservations(); + assertEquals(myObservationIds.subList(0, 15), returnedIdValues); + assertResponseIsFromCache(context, capture.getLastResponse()); + + myClient.unregisterInterceptor(capture); + } + + private static void assertResponseIsNotFromCache(String theContext, IHttpResponse lastResponse) { + List cacheOutcome= lastResponse.getHeaders(Constants.HEADER_X_CACHE); + assertThat(theContext + " - No cache response headers", cacheOutcome, empty()); + } + + private static void assertResponseIsFromCache(String theContext, IHttpResponse lastResponse) { + List cacheOutcome = lastResponse.getHeaders(Constants.HEADER_X_CACHE); + assertThat(theContext + " - Response came from cache", cacheOutcome, hasItem(matchesPattern("^HIT from .*"))); + } + + private List searchForObservations() { + Bundle result; + result = myClient .search() .forResource("Observation") .sort() @@ -199,77 +259,7 @@ public class ConsentInterceptorResourceProviderR4IT extends BaseResourceProvider .count(15) .execute(); List resources = BundleUtil.toListOfResources(myFhirContext, result); - List returnedIdValues = toUnqualifiedVersionlessIdValues(resources); - assertEquals(myObservationIdsEvenOnly.subList(0, 15), returnedIdValues); - List cacheOutcome = capture.getLastResponse().getHeaders(Constants.HEADER_X_CACHE); - assertEquals(0, cacheOutcome.size()); - - // Perform a search and only allow odd - consentService.setTarget(new ConsentSvcCantSeeEvenNumbered()); - result = myClient - .search() - .forResource("Observation") - .sort() - .ascending(Observation.SP_IDENTIFIER) - .returnBundle(Bundle.class) - .count(15) - .execute(); - resources = BundleUtil.toListOfResources(myFhirContext, result); - returnedIdValues = toUnqualifiedVersionlessIdValues(resources); - assertEquals(myObservationIdsOddOnly.subList(0, 15), returnedIdValues); - cacheOutcome = capture.getLastResponse().getHeaders(Constants.HEADER_X_CACHE); - assertEquals(0, cacheOutcome.size()); - - // Perform a search and allow all with a PROCEED - consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.PROCEED)); - result = myClient - .search() - .forResource("Observation") - .sort() - .ascending(Observation.SP_IDENTIFIER) - .returnBundle(Bundle.class) - .count(15) - .execute(); - resources = BundleUtil.toListOfResources(myFhirContext, result); - returnedIdValues = toUnqualifiedVersionlessIdValues(resources); - assertEquals(myObservationIds.subList(0, 15), returnedIdValues); - cacheOutcome = capture.getLastResponse().getHeaders(Constants.HEADER_X_CACHE); - assertEquals(0, cacheOutcome.size()); - - // Perform a search and allow all with an AUTHORIZED (no further checking) - consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.AUTHORIZED)); - result = myClient - .search() - .forResource("Observation") - .sort() - .ascending(Observation.SP_IDENTIFIER) - .returnBundle(Bundle.class) - .count(15) - .execute(); - resources = BundleUtil.toListOfResources(myFhirContext, result); - returnedIdValues = toUnqualifiedVersionlessIdValues(resources); - assertEquals(myObservationIds.subList(0, 15), returnedIdValues); - cacheOutcome = capture.getLastResponse().getHeaders(Constants.HEADER_X_CACHE); - assertEquals(0, cacheOutcome.size()); - - // Perform a second search and allow all with an AUTHORIZED (no further checking) - // which means we should finally get one from the cache - consentService.setTarget(new ConsentSvcNop(ConsentOperationStatusEnum.AUTHORIZED)); - result = myClient - .search() - .forResource("Observation") - .sort() - .ascending(Observation.SP_IDENTIFIER) - .returnBundle(Bundle.class) - .count(15) - .execute(); - resources = BundleUtil.toListOfResources(myFhirContext, result); - returnedIdValues = toUnqualifiedVersionlessIdValues(resources); - assertEquals(myObservationIds.subList(0, 15), returnedIdValues); - cacheOutcome = capture.getLastResponse().getHeaders(Constants.HEADER_X_CACHE); - assertThat(cacheOutcome.get(0), matchesPattern("^HIT from .*")); - - myClient.unregisterInterceptor(capture); + return toUnqualifiedVersionlessIdValues(resources); } @Test @@ -528,6 +518,7 @@ public class ConsentInterceptorResourceProviderR4IT extends BaseResourceProvider IConsentService svc = mock(IConsentService.class); when(svc.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); + when(svc.shouldProcessCanSeeResource(any(), any())).thenReturn(true); when(svc.canSeeResource(any(), any(), any())).thenReturn(ConsentOutcome.REJECT); consentService.setTarget(svc); @@ -560,6 +551,7 @@ public class ConsentInterceptorResourceProviderR4IT extends BaseResourceProvider IConsentService svc = mock(IConsentService.class); when(svc.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); + when(svc.shouldProcessCanSeeResource(any(), any())).thenReturn(true); when(svc.canSeeResource(any(RequestDetails.class), any(IBaseResource.class), any())).thenAnswer(t -> { IBaseResource resource = t.getArgument(1, IBaseResource.class); if (resource instanceof Organization) { @@ -998,16 +990,27 @@ public class ConsentInterceptorResourceProviderR4IT extends BaseResourceProvider private static class ConsentSvcNop implements IConsentService { private final ConsentOperationStatusEnum myOperationStatus; + private boolean myEnableCanSeeResource = true; private ConsentSvcNop(ConsentOperationStatusEnum theOperationStatus) { myOperationStatus = theOperationStatus; } + private ConsentSvcNop(ConsentOperationStatusEnum theOperationStatus, boolean theEnableCanSeeResource) { + myOperationStatus = theOperationStatus; + myEnableCanSeeResource = theEnableCanSeeResource; + } + @Override public ConsentOutcome startOperation(RequestDetails theRequestDetails, IConsentContextServices theContextServices) { return new ConsentOutcome(myOperationStatus); } + @Override + public boolean shouldProcessCanSeeResource(RequestDetails theRequestDetails, IConsentContextServices theContextServices) { + return myEnableCanSeeResource; + } + @Override public ConsentOutcome canSeeResource(RequestDetails theRequestDetails, IBaseResource theResource, IConsentContextServices theContextServices) { return new ConsentOutcome(ConsentOperationStatusEnum.PROCEED); diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantServerR4Test.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantServerR4Test.java index 5c5e356a283..139f4df1e5a 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantServerR4Test.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/MultitenantServerR4Test.java @@ -23,6 +23,7 @@ import ca.uhn.fhir.rest.server.RestfulServer; import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException; import ca.uhn.fhir.rest.server.exceptions.MethodNotAllowedException; import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException; +import ca.uhn.fhir.rest.server.interceptor.auth.SearchNarrowingInterceptor; import ca.uhn.fhir.rest.server.provider.ProviderConstants; import ca.uhn.fhir.rest.server.servlet.ServletRequestDetails; import ca.uhn.fhir.test.utilities.ITestDataBuilder; @@ -40,11 +41,14 @@ import org.hl7.fhir.r4.model.Condition; import org.hl7.fhir.r4.model.IdType; import org.hl7.fhir.r4.model.Organization; import org.hl7.fhir.r4.model.Patient; +import org.hl7.fhir.r4.model.Resource; import org.hl7.fhir.r4.model.StringType; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Nested; import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; import org.mockito.InjectMocks; import org.mockito.Mock; import org.mockito.Spy; @@ -65,6 +69,7 @@ import static org.hamcrest.Matchers.containsString; import static org.hamcrest.Matchers.hasSize; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.junit.jupiter.api.Assertions.fail; @@ -424,6 +429,55 @@ public class MultitenantServerR4Test extends BaseMultitenantResourceProviderR4Te } + @Test + public void testTransactionPut_withSearchNarrowingInterceptor_createsPatient() { + // setup + IBaseResource patientA = buildPatient(withTenant(TENANT_B), withActiveTrue(), withId("1234a"), + withFamily("Family"), withGiven("Given")); + + Bundle transactioBundle = new Bundle(); + transactioBundle.setType(Bundle.BundleType.TRANSACTION); + transactioBundle.addEntry() + .setFullUrl("http://localhost:8000/TENANT-A/Patient/1234a") + .setResource((Resource) patientA) + .getRequest().setUrl("Patient/1234a").setMethod(Bundle.HTTPVerb.PUT); + + myServer.registerInterceptor(new SearchNarrowingInterceptor()); + + // execute + myClient.transaction().withBundle(transactioBundle).execute(); + + // verify - read back using DAO + SystemRequestDetails requestDetails = new SystemRequestDetails(); + requestDetails.setTenantId(TENANT_B); + Patient patient1 = myPatientDao.read(new IdType("Patient/1234a"), requestDetails); + assertEquals("Family", patient1.getName().get(0).getFamily()); + } + + @ParameterizedTest + @ValueSource(strings = {"Patient/1234a", "TENANT-B/Patient/1234a"}) + public void testTransactionGet_withSearchNarrowingInterceptor_retrievesPatient(String theEntryUrl) { + // setup + createPatient(withTenant(TENANT_B), withActiveTrue(), withId("1234a"), + withFamily("Family"), withGiven("Given")); + + Bundle transactioBundle = new Bundle(); + transactioBundle.setType(Bundle.BundleType.TRANSACTION); + transactioBundle.addEntry() + .getRequest().setUrl(theEntryUrl).setMethod(Bundle.HTTPVerb.GET); + + myServer.registerInterceptor(new SearchNarrowingInterceptor()); + + // execute + Bundle result = myClient.transaction().withBundle(transactioBundle).execute(); + + // verify + assertEquals(1, result.getEntry().size()); + Patient retrievedPatient = (Patient) result.getEntry().get(0).getResource(); + assertNotNull(retrievedPatient); + assertEquals("Family", retrievedPatient.getName().get(0).getFamily()); + } + @Test public void testDirectDaoAccess_PartitionInRequestDetails_Create() { diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/StaleSearchDeletingSvcR4Test.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/StaleSearchDeletingSvcR4Test.java index 3d9f3e0a6e9..1e6432f0af1 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/StaleSearchDeletingSvcR4Test.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/provider/r4/StaleSearchDeletingSvcR4Test.java @@ -48,7 +48,7 @@ public class StaleSearchDeletingSvcR4Test extends BaseResourceProviderR4Test { super.after(); DatabaseSearchCacheSvcImpl staleSearchDeletingSvc = AopTestUtils.getTargetObject(mySearchCacheSvc); staleSearchDeletingSvc.setCutoffSlackForUnitTest(DatabaseSearchCacheSvcImpl.SEARCH_CLEANUP_JOB_INTERVAL_MILLIS); - DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteForUnitTest(DatabaseSearchCacheSvcImpl.DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_STMT); + DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOneStatement(DatabaseSearchCacheSvcImpl.DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_STMT); DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOnePassForUnitTest(DatabaseSearchCacheSvcImpl.DEFAULT_MAX_RESULTS_TO_DELETE_IN_ONE_PAS); } @@ -108,7 +108,7 @@ public class StaleSearchDeletingSvcR4Test extends BaseResourceProviderR4Test { @Test public void testDeleteVeryLargeSearch() { - DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteForUnitTest(10); + DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOneStatement(10); DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOnePassForUnitTest(10); runInTransaction(() -> { @@ -120,24 +120,21 @@ public class StaleSearchDeletingSvcR4Test extends BaseResourceProviderR4Test { search.setResourceType("Patient"); search = mySearchEntityDao.save(search); - for (int i = 0; i < 15; i++) { - ResourceTable resource = new ResourceTable(); - resource.setPublished(new Date()); - resource.setUpdated(new Date()); - resource.setResourceType("Patient"); - resource = myResourceTableDao.saveAndFlush(resource); + ResourceTable resource = new ResourceTable(); + resource.setPublished(new Date()); + resource.setUpdated(new Date()); + resource.setResourceType("Patient"); + resource = myResourceTableDao.saveAndFlush(resource); + for (int i = 0; i < 50; i++) { SearchResult sr = new SearchResult(search); sr.setOrder(i); sr.setResourcePid(resource.getId()); mySearchResultDao.save(sr); } - }); - // It should take two passes to delete the search fully - runInTransaction(() -> assertEquals(1, mySearchEntityDao.count())); - myStaleSearchDeletingSvc.pollForStaleSearchesAndDeleteThem(); + // we are able to delete this in one pass. runInTransaction(() -> assertEquals(1, mySearchEntityDao.count())); myStaleSearchDeletingSvc.pollForStaleSearchesAndDeleteThem(); runInTransaction(() -> assertEquals(0, mySearchEntityDao.count())); @@ -146,9 +143,9 @@ public class StaleSearchDeletingSvcR4Test extends BaseResourceProviderR4Test { @Test public void testDeleteVerySmallSearch() { - DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteForUnitTest(10); + DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOneStatement(10); - runInTransaction(() -> { + runInTransaction(() -> { Search search = new Search(); search.setStatus(SearchStatusEnum.FINISHED); search.setUuid(UUID.randomUUID().toString()); @@ -172,9 +169,9 @@ public class StaleSearchDeletingSvcR4Test extends BaseResourceProviderR4Test { @Test public void testDontDeleteSearchBeforeExpiry() { - DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteForUnitTest(10); + DatabaseSearchCacheSvcImpl.setMaximumResultsToDeleteInOneStatement(10); - runInTransaction(() -> { + runInTransaction(() -> { Search search = new Search(); // Expires in one second, so it should not be deleted right away, @@ -186,7 +183,7 @@ public class StaleSearchDeletingSvcR4Test extends BaseResourceProviderR4Test { search.setCreated(DateUtils.addDays(new Date(), -10000)); search.setSearchType(SearchTypeEnum.SEARCH); search.setResourceType("Patient"); - search = mySearchEntityDao.save(search); + mySearchEntityDao.save(search); }); diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/message/MessageSubscriptionR4Test.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/message/MessageSubscriptionR4Test.java index 2fc3ca8bb20..03b12841055 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/message/MessageSubscriptionR4Test.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/message/MessageSubscriptionR4Test.java @@ -253,26 +253,28 @@ public class MessageSubscriptionR4Test extends BaseSubscriptionsR4Test { } @Test - public void testPersistedResourceModifiedMessage_whenFetchFromDb_willEqualOriginalMessage() throws JsonProcessingException { + public void testMethodInflatePersistedResourceModifiedMessage_whenGivenResourceModifiedMessageWithEmptyPayload_willEqualOriginalMessage() { mySubscriptionTestUtil.unregisterSubscriptionInterceptor(); - // given + // setup TransactionTemplate transactionTemplate = new TransactionTemplate(myTxManager); Observation obs = sendObservation("zoop", "SNOMED-CT", "theExplicitSource", "theRequestId"); ResourceModifiedMessage originalResourceModifiedMessage = createResourceModifiedMessage(obs); + ResourceModifiedMessage resourceModifiedMessageWithEmptyPayload = createResourceModifiedMessage(obs); + resourceModifiedMessageWithEmptyPayload.setPayloadToNull(); transactionTemplate.execute(tx -> { - IPersistedResourceModifiedMessage persistedResourceModifiedMessage = myResourceModifiedMessagePersistenceSvc.persist(originalResourceModifiedMessage); + myResourceModifiedMessagePersistenceSvc.persist(originalResourceModifiedMessage); - // when - ResourceModifiedMessage restoredResourceModifiedMessage = myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(persistedResourceModifiedMessage); + // execute + ResourceModifiedMessage restoredResourceModifiedMessage = myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(resourceModifiedMessageWithEmptyPayload); - // then - assertEquals(toJson(originalResourceModifiedMessage), toJson(restoredResourceModifiedMessage)); - assertEquals(originalResourceModifiedMessage, restoredResourceModifiedMessage); + // verify + assertEquals(toJson(originalResourceModifiedMessage), toJson(restoredResourceModifiedMessage)); + assertEquals(originalResourceModifiedMessage, restoredResourceModifiedMessage); - return null; + return null; }); } diff --git a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/svc/ResourceModifiedSubmitterSvcTest.java b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/svc/ResourceModifiedSubmitterSvcTest.java index 21d1f97c686..f8194204aaa 100644 --- a/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/svc/ResourceModifiedSubmitterSvcTest.java +++ b/hapi-fhir-jpaserver-test-r4/src/test/java/ca/uhn/fhir/jpa/subscription/svc/ResourceModifiedSubmitterSvcTest.java @@ -105,7 +105,7 @@ public class ResourceModifiedSubmitterSvcTest { // given // a successful deletion implies that the message did exist. when(myResourceModifiedMessagePersistenceSvc.deleteByPK(any())).thenReturn(true); - when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(any())).thenReturn(new ResourceModifiedMessage()); + when(myResourceModifiedMessagePersistenceSvc.createResourceModifiedMessageFromEntityWithoutInflation(any())).thenReturn(new ResourceModifiedMessage()); // when boolean wasProcessed = myResourceModifiedSubmitterSvc.submitPersisedResourceModifiedMessage(new ResourceModifiedEntity()); @@ -134,7 +134,7 @@ public class ResourceModifiedSubmitterSvcTest { // when when(myResourceModifiedMessagePersistenceSvc.deleteByPK(any())) .thenThrow(new RuntimeException(deleteExMsg)); - when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(any())) + when(myResourceModifiedMessagePersistenceSvc.createResourceModifiedMessageFromEntityWithoutInflation(any())) .thenThrow(new RuntimeException(inflationExMsg)); // test @@ -180,7 +180,7 @@ public class ResourceModifiedSubmitterSvcTest { // when when(myResourceModifiedMessagePersistenceSvc.deleteByPK(any())) .thenReturn(true); - when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(any())) + when(myResourceModifiedMessagePersistenceSvc.createResourceModifiedMessageFromEntityWithoutInflation(any())) .thenReturn(msg); when(myChannelProducer.send(any())) .thenThrow(new RuntimeException(exceptionString)); @@ -206,7 +206,7 @@ public class ResourceModifiedSubmitterSvcTest { // given // deletion fails, someone else was faster and processed the message when(myResourceModifiedMessagePersistenceSvc.deleteByPK(any())).thenReturn(false); - when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(any())).thenReturn(new ResourceModifiedMessage()); + when(myResourceModifiedMessagePersistenceSvc.createResourceModifiedMessageFromEntityWithoutInflation(any())).thenReturn(new ResourceModifiedMessage()); // when boolean wasProcessed = myResourceModifiedSubmitterSvc.submitPersisedResourceModifiedMessage(new ResourceModifiedEntity()); @@ -223,7 +223,7 @@ public class ResourceModifiedSubmitterSvcTest { public void testSubmitPersistedResourceModifiedMessage_whitErrorOnSending_willRollbackDeletion(){ // given when(myResourceModifiedMessagePersistenceSvc.deleteByPK(any())).thenReturn(true); - when(myResourceModifiedMessagePersistenceSvc.inflatePersistedResourceModifiedMessage(any())).thenReturn(new ResourceModifiedMessage()); + when(myResourceModifiedMessagePersistenceSvc.createResourceModifiedMessageFromEntityWithoutInflation(any())).thenReturn(new ResourceModifiedMessage()); // simulate failure writing to the channel when(myChannelProducer.send(any())).thenThrow(new MessageDeliveryException("sendingError")); diff --git a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/dao/TestDaoSearch.java b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/dao/TestDaoSearch.java index 670b8e07875..b80f19b9e4d 100644 --- a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/dao/TestDaoSearch.java +++ b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/dao/TestDaoSearch.java @@ -46,6 +46,9 @@ import java.util.List; import java.util.stream.Collectors; import javax.annotation.Nonnull; +import static org.apache.commons.lang3.ArrayUtils.EMPTY_STRING_ARRAY; +import static org.hamcrest.Matchers.contains; +import static org.hamcrest.Matchers.containsInAnyOrder; import static org.hamcrest.Matchers.everyItem; import static org.hamcrest.Matchers.hasItems; import static org.hamcrest.Matchers.in; @@ -105,6 +108,10 @@ public class TestDaoSearch { assertSearchResultIds(theQueryUrl, theReason, hasItems(theIds)); } + public void assertSearchFinds(String theReason, String theQueryUrl, List theIds) { + assertSearchFinds(theReason, theQueryUrl, theIds.toArray(EMPTY_STRING_ARRAY)); + } + /** * Assert that the FHIR search has theIds in the search results. * @param theReason junit reason message @@ -117,6 +124,27 @@ public class TestDaoSearch { assertSearchResultIds(theQueryUrl, theReason, hasItems(bareIds)); } + public void assertSearchFindsInOrder(String theReason, String theQueryUrl, String... theIds) { + List ids = searchForIds(theQueryUrl); + + MatcherAssert.assertThat(theReason, ids, contains(theIds)); + } + + public void assertSearchFindsInOrder(String theReason, String theQueryUrl, List theIds) { + assertSearchFindsInOrder(theReason, theQueryUrl, theIds.toArray(EMPTY_STRING_ARRAY)); + } + + public void assertSearchFindsOnly(String theReason, String theQueryUrl, String... theIds) { + assertSearchIdsMatch(theReason, theQueryUrl, containsInAnyOrder(theIds)); + } + + public void assertSearchIdsMatch( + String theReason, String theQueryUrl, Matcher> theMatchers) { + List ids = searchForIds(theQueryUrl); + + MatcherAssert.assertThat(theReason, ids, theMatchers); + } + public void assertSearchResultIds(String theQueryUrl, String theReason, Matcher> matcher) { List ids = searchForIds(theQueryUrl); diff --git a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/search/IIdSearchTestTemplate.java b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/search/IIdSearchTestTemplate.java new file mode 100644 index 00000000000..1319b22b835 --- /dev/null +++ b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/search/IIdSearchTestTemplate.java @@ -0,0 +1,74 @@ +/*- + * #%L + * HAPI FHIR JPA Server Test Utilities + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.jpa.search; + +import ca.uhn.fhir.jpa.dao.TestDaoSearch; +import ca.uhn.fhir.test.utilities.ITestDataBuilder; +import org.hl7.fhir.instance.model.api.IIdType; +import org.junit.jupiter.api.Test; + +import java.util.List; + +public interface IIdSearchTestTemplate { + TestDaoSearch getSearch(); + + ITestDataBuilder getBuilder(); + + @Test + default void testSearchByServerAssignedId_findsResource() { + IIdType id = getBuilder().createPatient(); + + getSearch().assertSearchFinds("search by server assigned id", "Patient?_id=" + id.getIdPart(), id); + } + + @Test + default void testSearchByClientAssignedId_findsResource() { + ITestDataBuilder b = getBuilder(); + b.createPatient(b.withId("client-assigned-id")); + + getSearch() + .assertSearchFinds( + "search by client assigned id", "Patient?_id=client-assigned-id", "client-assigned-id"); + } + + /** + * The _id SP is defined as token, and there is no system. + * So sorting should be string order of the value. + */ + @Test + default void testSortById_treatsIdsAsString() { + ITestDataBuilder b = getBuilder(); + b.createPatient(b.withId("client-assigned-id")); + IIdType serverId = b.createPatient(); + b.createPatient(b.withId("0-sorts-before-other-numbers")); + + getSearch() + .assertSearchFindsInOrder( + "sort by resource id", + "Patient?_sort=_id", + List.of("0-sorts-before-other-numbers", serverId.getIdPart(), "client-assigned-id")); + + getSearch() + .assertSearchFindsInOrder( + "reverse sort by resource id", + "Patient?_sort=-_id", + List.of("client-assigned-id", serverId.getIdPart(), "0-sorts-before-other-numbers")); + } +} diff --git a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/BaseJpaTest.java b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/BaseJpaTest.java index 79019e41f28..9edc49122a3 100644 --- a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/BaseJpaTest.java +++ b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/BaseJpaTest.java @@ -431,6 +431,11 @@ public abstract class BaseJpaTest extends BaseTest { return deliveryLatch; } + protected void registerInterceptor(Object theInterceptor) { + myRegisteredInterceptors.add(theInterceptor); + myInterceptorRegistry.registerInterceptor(theInterceptor); + } + protected void purgeHibernateSearch(EntityManager theEntityManager) { runInTransaction(() -> { if (myFulltestSearchSvc != null && !myFulltestSearchSvc.isDisabled()) { diff --git a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/config/ConnectionWrapper.java b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/config/ConnectionWrapper.java index a628e4a854d..ede47637936 100644 --- a/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/config/ConnectionWrapper.java +++ b/hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/config/ConnectionWrapper.java @@ -65,6 +65,7 @@ public class ConnectionWrapper implements Connection { @Override public void commit() throws SQLException { + if (ourLog.isTraceEnabled()) { ourLog.trace("commit: {}", myWrap.hashCode()); } myWrap.commit(); } diff --git a/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/ConnectionWrapper.java b/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/ConnectionWrapper.java index ce598bed25b..a6e222f9c2c 100644 --- a/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/ConnectionWrapper.java +++ b/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/ConnectionWrapper.java @@ -46,6 +46,7 @@ public class ConnectionWrapper implements Connection { @Override public void commit() throws SQLException { + if (ourLog.isTraceEnabled()) { ourLog.trace("Commit: {}", myWrap.hashCode()); } myWrap.commit(); } diff --git a/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialectTest.java b/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialectTest.java index b0bafdcdf7f..aa12360a223 100644 --- a/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialectTest.java +++ b/hapi-fhir-jpaserver-test-utilities/src/test/java/ca/uhn/fhir/jpa/config/HapiFhirHibernateJpaDialectTest.java @@ -36,7 +36,7 @@ public class HapiFhirHibernateJpaDialectTest { assertThat(outcome.getMessage(), containsString("this is a message")); try { - mySvc.convertHibernateAccessException(new ConstraintViolationException("this is a message", new SQLException("reason"), ResourceTable.IDX_RES_FHIR_ID)); + mySvc.convertHibernateAccessException(new ConstraintViolationException("this is a message", new SQLException("reason"), ResourceTable.IDX_RES_TYPE_FHIR_ID)); fail(); } catch (ResourceVersionConflictException e) { assertThat(e.getMessage(), containsString("The operation has failed with a client-assigned ID constraint failure")); @@ -67,7 +67,7 @@ public class HapiFhirHibernateJpaDialectTest { assertEquals("FOO", outcome.getMessage()); try { - PersistenceException exception = new PersistenceException("a message", new ConstraintViolationException("this is a message", new SQLException("reason"), ResourceTable.IDX_RES_FHIR_ID)); + PersistenceException exception = new PersistenceException("a message", new ConstraintViolationException("this is a message", new SQLException("reason"), ResourceTable.IDX_RES_TYPE_FHIR_ID)); mySvc.translate(exception, "a message"); fail(); } catch (ResourceVersionConflictException e) { diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/api/ICdsConfigService.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/api/ICdsConfigService.java index 705205be473..c3104ab5171 100644 --- a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/api/ICdsConfigService.java +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/api/ICdsConfigService.java @@ -26,6 +26,7 @@ import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.api.server.SystemRequestDetails; import ca.uhn.fhir.rest.api.server.SystemRestfulResponse; import ca.uhn.fhir.rest.server.RestfulServer; +import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsCrSettings; import com.fasterxml.jackson.databind.ObjectMapper; import org.opencds.cqf.fhir.utility.Ids; @@ -39,6 +40,9 @@ public interface ICdsConfigService { @Nonnull ObjectMapper getObjectMapper(); + @Nonnull + CdsCrSettings getCdsCrSettings(); + @Nullable default DaoRegistry getDaoRegistry() { return null; diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsCrConfig.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsCrConfig.java new file mode 100644 index 00000000000..9aa78815ce6 --- /dev/null +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsCrConfig.java @@ -0,0 +1,37 @@ +/*- + * #%L + * HAPI FHIR - CDS Hooks + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.hapi.fhir.cdshooks.config; + +import ca.uhn.fhir.cr.config.CrConfigCondition; +import ca.uhn.fhir.cr.config.RepositoryConfig; +import ca.uhn.fhir.cr.config.r4.ApplyOperationConfig; +import org.springframework.context.annotation.Conditional; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; + +/** + * This class exists as a wrapper for the CR configs required for CDS on FHIR to be loaded only when dependencies are met. + * Adding the condition to the configs themselves causes issues with downstream projects. + * + */ +@Configuration +@Conditional(CrConfigCondition.class) +@Import({RepositoryConfig.class, ApplyOperationConfig.class}) +public class CdsCrConfig {} diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsHooksConfig.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsHooksConfig.java index b0c115807ee..6f679212d65 100644 --- a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsHooksConfig.java +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/config/CdsHooksConfig.java @@ -35,6 +35,7 @@ import ca.uhn.hapi.fhir.cdshooks.svc.CdsConfigServiceImpl; import ca.uhn.hapi.fhir.cdshooks.svc.CdsHooksContextBooter; import ca.uhn.hapi.fhir.cdshooks.svc.CdsServiceRegistryImpl; import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsCrServiceRegistry; +import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsCrSettings; import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsServiceInterceptor; import ca.uhn.hapi.fhir.cdshooks.svc.cr.ICdsCrService; import ca.uhn.hapi.fhir.cdshooks.svc.cr.ICdsCrServiceFactory; @@ -56,12 +57,14 @@ import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.lang.reflect.Constructor; import java.lang.reflect.InvocationTargetException; import java.util.Optional; @Configuration +@Import(CdsCrConfig.class) public class CdsHooksConfig { private static final Logger ourLog = LoggerFactory.getLogger(CdsHooksConfig.class); @@ -128,8 +131,8 @@ public class CdsHooksConfig { } try { Constructor constructor = - clazz.get().getConstructor(RequestDetails.class, Repository.class); - return constructor.newInstance(rd, repository); + clazz.get().getConstructor(RequestDetails.class, Repository.class, ICdsConfigService.class); + return constructor.newInstance(rd, repository, theCdsConfigService); } catch (NoSuchMethodException | InvocationTargetException | InstantiationException @@ -189,9 +192,11 @@ public class CdsHooksConfig { @Bean public ICdsConfigService cdsConfigService( - FhirContext theFhirContext, @Qualifier(CDS_HOOKS_OBJECT_MAPPER_FACTORY) ObjectMapper theObjectMapper) { + FhirContext theFhirContext, + @Qualifier(CDS_HOOKS_OBJECT_MAPPER_FACTORY) ObjectMapper theObjectMapper, + CdsCrSettings theCdsCrSettings) { return new CdsConfigServiceImpl( - theFhirContext, theObjectMapper, myDaoRegistry, myRepositoryFactory, myRestfulServer); + theFhirContext, theObjectMapper, theCdsCrSettings, myDaoRegistry, myRepositoryFactory, myRestfulServer); } @Bean diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/CdsConfigServiceImpl.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/CdsConfigServiceImpl.java index 59343edff07..bf107a08166 100644 --- a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/CdsConfigServiceImpl.java +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/CdsConfigServiceImpl.java @@ -24,6 +24,7 @@ import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.fhir.rest.server.RestfulServer; import ca.uhn.hapi.fhir.cdshooks.api.ICdsConfigService; +import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsCrSettings; import com.fasterxml.jackson.databind.ObjectMapper; import javax.annotation.Nonnull; @@ -32,6 +33,7 @@ import javax.annotation.Nullable; public class CdsConfigServiceImpl implements ICdsConfigService { private final FhirContext myFhirContext; private final ObjectMapper myObjectMapper; + private final CdsCrSettings myCdsCrSettings; private final DaoRegistry myDaoRegistry; private final IRepositoryFactory myRepositoryFactory; private final RestfulServer myRestfulServer; @@ -39,11 +41,13 @@ public class CdsConfigServiceImpl implements ICdsConfigService { public CdsConfigServiceImpl( @Nonnull FhirContext theFhirContext, @Nonnull ObjectMapper theObjectMapper, + @Nonnull CdsCrSettings theCdsCrSettings, @Nullable DaoRegistry theDaoRegistry, @Nullable IRepositoryFactory theRepositoryFactory, @Nullable RestfulServer theRestfulServer) { myFhirContext = theFhirContext; myObjectMapper = theObjectMapper; + myCdsCrSettings = theCdsCrSettings; myDaoRegistry = theDaoRegistry; myRepositoryFactory = theRepositoryFactory; myRestfulServer = theRestfulServer; @@ -61,6 +65,12 @@ public class CdsConfigServiceImpl implements ICdsConfigService { return myObjectMapper; } + @Nonnull + @Override + public CdsCrSettings getCdsCrSettings() { + return myCdsCrSettings; + } + @Nullable @Override public DaoRegistry getDaoRegistry() { diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceDstu3.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceDstu3.java index 17e0ea7afc5..5a6c97375ad 100644 --- a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceDstu3.java +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceDstu3.java @@ -21,7 +21,16 @@ package ca.uhn.hapi.fhir.cdshooks.svc.cr; import ca.uhn.fhir.context.FhirVersionEnum; import ca.uhn.fhir.rest.api.server.RequestDetails; -import ca.uhn.hapi.fhir.cdshooks.api.json.*; +import ca.uhn.hapi.fhir.cdshooks.api.ICdsConfigService; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestAuthorizationJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseCardJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseCardSourceJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseLinkJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSuggestionActionJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSuggestionJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSystemActionJson; import org.hl7.fhir.dstu3.model.Bundle; import org.hl7.fhir.dstu3.model.CarePlan; import org.hl7.fhir.dstu3.model.Endpoint; @@ -60,10 +69,13 @@ import static org.opencds.cqf.fhir.utility.dstu3.Parameters.part; public class CdsCrServiceDstu3 implements ICdsCrService { protected final RequestDetails myRequestDetails; protected final Repository myRepository; + protected final ICdsConfigService myCdsConfigService; protected CarePlan myResponse; protected CdsServiceResponseJson myServiceResponse; - public CdsCrServiceDstu3(RequestDetails theRequestDetails, Repository theRepository) { + public CdsCrServiceDstu3( + RequestDetails theRequestDetails, Repository theRepository, ICdsConfigService theCdsConfigService) { + myCdsConfigService = theCdsConfigService; myRequestDetails = theRequestDetails; myRepository = theRepository; } @@ -108,6 +120,12 @@ public class CdsCrServiceDstu3 implements ICdsCrService { endpoint.addHeader(String.format( "Authorization: %s %s", tokenType, theJson.getServiceRequestAuthorizationJson().getAccessToken())); + if (theJson.getServiceRequestAuthorizationJson().getSubject() != null) { + endpoint.addHeader(String.format( + "%s: %s", + myCdsConfigService.getCdsCrSettings().getClientIdHeaderName(), + theJson.getServiceRequestAuthorizationJson().getSubject())); + } } parameters.addParameter(part(APPLY_PARAMETER_DATA_ENDPOINT, endpoint)); } diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR4.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR4.java index 26b0eaa464d..920f63e319f 100644 --- a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR4.java +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR4.java @@ -22,7 +22,17 @@ package ca.uhn.hapi.fhir.cdshooks.svc.cr; import ca.uhn.fhir.context.FhirVersionEnum; import ca.uhn.fhir.i18n.Msg; import ca.uhn.fhir.rest.api.server.RequestDetails; -import ca.uhn.hapi.fhir.cdshooks.api.json.*; +import ca.uhn.hapi.fhir.cdshooks.api.ICdsConfigService; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceIndicatorEnum; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestAuthorizationJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseCardJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseCardSourceJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseLinkJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSuggestionActionJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSuggestionJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSystemActionJson; import org.hl7.fhir.instance.model.api.IBaseResource; import org.hl7.fhir.r4.model.Bundle; import org.hl7.fhir.r4.model.CanonicalType; @@ -61,10 +71,13 @@ import static org.opencds.cqf.fhir.utility.r4.Parameters.part; public class CdsCrServiceR4 implements ICdsCrService { protected final RequestDetails myRequestDetails; protected final Repository myRepository; + protected final ICdsConfigService myCdsConfigService; protected Bundle myResponseBundle; protected CdsServiceResponseJson myServiceResponse; - public CdsCrServiceR4(RequestDetails theRequestDetails, Repository theRepository) { + public CdsCrServiceR4( + RequestDetails theRequestDetails, Repository theRepository, ICdsConfigService theCdsConfigService) { + myCdsConfigService = theCdsConfigService; myRequestDetails = theRequestDetails; myRepository = theRepository; } @@ -109,8 +122,13 @@ public class CdsCrServiceR4 implements ICdsCrService { endpoint.addHeader(String.format( "Authorization: %s %s", tokenType, theJson.getServiceRequestAuthorizationJson().getAccessToken())); + if (theJson.getServiceRequestAuthorizationJson().getSubject() != null) { + endpoint.addHeader(String.format( + "%s: %s", + myCdsConfigService.getCdsCrSettings().getClientIdHeaderName(), + theJson.getServiceRequestAuthorizationJson().getSubject())); + } } - endpoint.addHeader("Epic-Client-ID: 2cb5af9f-f483-4e2a-aedc-54c3a31cb153"); parameters.addParameter(part(APPLY_PARAMETER_DATA_ENDPOINT, endpoint)); } return parameters; diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR5.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR5.java index 79055a56e1d..8f76db3a2af 100644 --- a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR5.java +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrServiceR5.java @@ -22,7 +22,17 @@ package ca.uhn.hapi.fhir.cdshooks.svc.cr; import ca.uhn.fhir.context.FhirVersionEnum; import ca.uhn.fhir.i18n.Msg; import ca.uhn.fhir.rest.api.server.RequestDetails; -import ca.uhn.hapi.fhir.cdshooks.api.json.*; +import ca.uhn.hapi.fhir.cdshooks.api.ICdsConfigService; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceIndicatorEnum; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestAuthorizationJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseCardJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseCardSourceJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseLinkJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSuggestionActionJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSuggestionJson; +import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseSystemActionJson; import org.hl7.fhir.instance.model.api.IBaseResource; import org.hl7.fhir.r5.model.Bundle; import org.hl7.fhir.r5.model.CanonicalType; @@ -61,10 +71,13 @@ import static org.opencds.cqf.fhir.utility.r5.Parameters.part; public class CdsCrServiceR5 implements ICdsCrService { protected final RequestDetails myRequestDetails; protected final Repository myRepository; + protected final ICdsConfigService myCdsConfigService; protected Bundle myResponseBundle; protected CdsServiceResponseJson myServiceResponse; - public CdsCrServiceR5(RequestDetails theRequestDetails, Repository theRepository) { + public CdsCrServiceR5( + RequestDetails theRequestDetails, Repository theRepository, ICdsConfigService theCdsConfigService) { + myCdsConfigService = theCdsConfigService; myRequestDetails = theRequestDetails; myRepository = theRepository; } @@ -109,6 +122,12 @@ public class CdsCrServiceR5 implements ICdsCrService { endpoint.addHeader(String.format( "Authorization: %s %s", tokenType, theJson.getServiceRequestAuthorizationJson().getAccessToken())); + if (theJson.getServiceRequestAuthorizationJson().getSubject() != null) { + endpoint.addHeader(String.format( + "%s: %s", + myCdsConfigService.getCdsCrSettings().getClientIdHeaderName(), + theJson.getServiceRequestAuthorizationJson().getSubject())); + } } parameters.addParameter(part(APPLY_PARAMETER_DATA_ENDPOINT, endpoint)); } diff --git a/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrSettings.java b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrSettings.java new file mode 100644 index 00000000000..70258940829 --- /dev/null +++ b/hapi-fhir-server-cds-hooks/src/main/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/CdsCrSettings.java @@ -0,0 +1,44 @@ +/*- + * #%L + * HAPI FHIR - CDS Hooks + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.hapi.fhir.cdshooks.svc.cr; + +public class CdsCrSettings { + private final String DEFAULT_CLIENT_ID_HEADER_NAME = "client_id"; + private String myClientIdHeaderName; + + public static CdsCrSettings getDefault() { + CdsCrSettings settings = new CdsCrSettings(); + settings.setClientIdHeaderName(settings.DEFAULT_CLIENT_ID_HEADER_NAME); + return settings; + } + + public void setClientIdHeaderName(String theName) { + myClientIdHeaderName = theName; + } + + public String getClientIdHeaderName() { + return myClientIdHeaderName; + } + + public CdsCrSettings withClientIdHeaderName(String theName) { + myClientIdHeaderName = theName; + return this; + } +} diff --git a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/config/TestCdsHooksConfig.java b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/config/TestCdsHooksConfig.java index 6b55053f357..367d496bb3b 100644 --- a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/config/TestCdsHooksConfig.java +++ b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/config/TestCdsHooksConfig.java @@ -5,6 +5,7 @@ import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.hapi.fhir.cdshooks.api.ICdsHooksDaoAuthorizationSvc; import ca.uhn.hapi.fhir.cdshooks.controller.TestServerAppCtx; import ca.uhn.hapi.fhir.cdshooks.svc.CdsHooksContextBooter; +import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsCrSettings; import org.hl7.fhir.instance.model.api.IBaseResource; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @@ -16,6 +17,9 @@ public class TestCdsHooksConfig { return FhirContext.forR4Cached(); } + @Bean + CdsCrSettings cdsCrSettings() { return CdsCrSettings.getDefault(); } + @Bean public CdsHooksContextBooter cdsHooksContextBooter() { CdsHooksContextBooter retVal = new CdsHooksContextBooter(); diff --git a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/BaseCrTest.java b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/BaseCrTest.java index ca66a5636a1..f00675fdacc 100644 --- a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/BaseCrTest.java +++ b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/BaseCrTest.java @@ -14,4 +14,6 @@ public abstract class BaseCrTest { @Autowired protected FhirContext myFhirContext; + @Autowired + protected CdsCrSettings myCdsCrSettings; } diff --git a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/TestCrConfig.java b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/TestCrConfig.java index 05e0201284b..5b7410c2726 100644 --- a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/TestCrConfig.java +++ b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/TestCrConfig.java @@ -2,13 +2,37 @@ package ca.uhn.hapi.fhir.cdshooks.svc.cr; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.jpa.api.dao.DaoRegistry; +import ca.uhn.hapi.fhir.cdshooks.api.ICdsConfigService; +import ca.uhn.hapi.fhir.cdshooks.module.CdsHooksObjectMapperFactory; +import ca.uhn.hapi.fhir.cdshooks.svc.CdsConfigServiceImpl; +import com.fasterxml.jackson.databind.ObjectMapper; +import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import static ca.uhn.hapi.fhir.cdshooks.config.CdsHooksConfig.CDS_HOOKS_OBJECT_MAPPER_FACTORY; + @Configuration public class TestCrConfig { @Bean FhirContext fhirContext() { return FhirContext.forR4Cached(); } + + @Bean(name = CDS_HOOKS_OBJECT_MAPPER_FACTORY) + public ObjectMapper objectMapper(FhirContext theFhirContext) { + return new CdsHooksObjectMapperFactory(theFhirContext).newMapper(); + } + + @Bean + CdsCrSettings cdsCrSettings() { return CdsCrSettings.getDefault(); } + + @Bean + public ICdsConfigService cdsConfigService( + FhirContext theFhirContext, + @Qualifier(CDS_HOOKS_OBJECT_MAPPER_FACTORY) ObjectMapper theObjectMapper, + CdsCrSettings theCdsCrSettings) { + return new CdsConfigServiceImpl( + theFhirContext, theObjectMapper, theCdsCrSettings, null, null, null); + } } diff --git a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/resolution/CdsCrServiceR4Test.java b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/resolution/CdsCrServiceR4Test.java index e0179374882..6243ab8a8bb 100644 --- a/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/resolution/CdsCrServiceR4Test.java +++ b/hapi-fhir-server-cds-hooks/src/test/java/ca/uhn/hapi/fhir/cdshooks/svc/cr/resolution/CdsCrServiceR4Test.java @@ -3,9 +3,11 @@ package ca.uhn.hapi.fhir.cdshooks.svc.cr.resolution; import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.api.server.SystemRequestDetails; import ca.uhn.fhir.util.ClasspathUtil; +import ca.uhn.hapi.fhir.cdshooks.api.ICdsConfigService; import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceRequestJson; import ca.uhn.hapi.fhir.cdshooks.api.json.CdsServiceResponseJson; import ca.uhn.hapi.fhir.cdshooks.module.CdsHooksObjectMapperFactory; +import ca.uhn.hapi.fhir.cdshooks.svc.CdsConfigServiceImpl; import ca.uhn.hapi.fhir.cdshooks.svc.cr.BaseCrTest; import ca.uhn.hapi.fhir.cdshooks.svc.cr.CdsCrServiceR4; import com.fasterxml.jackson.databind.ObjectMapper; @@ -25,9 +27,13 @@ import static org.junit.jupiter.api.Assertions.assertNotNull; import static org.junit.jupiter.api.Assertions.assertTrue; public class CdsCrServiceR4Test extends BaseCrTest { - private ObjectMapper myObjectMapper;@BeforeEach + private ObjectMapper myObjectMapper; + private ICdsConfigService myCdsConfigService; + + @BeforeEach public void loadJson() throws IOException { myObjectMapper = new CdsHooksObjectMapperFactory(myFhirContext).newMapper(); + myCdsConfigService = new CdsConfigServiceImpl(myFhirContext, myObjectMapper, myCdsCrSettings, null, null, null); } @Test @@ -39,7 +45,7 @@ public class CdsCrServiceR4Test extends BaseCrTest { final RequestDetails requestDetails = new SystemRequestDetails(); final IdType planDefinitionId = new IdType(PLAN_DEFINITION_RESOURCE_NAME, "ASLPCrd"); requestDetails.setId(planDefinitionId); - final Parameters params = new CdsCrServiceR4(requestDetails, repository).encodeParams(cdsServiceRequestJson); + final Parameters params = new CdsCrServiceR4(requestDetails, repository, myCdsConfigService).encodeParams(cdsServiceRequestJson); assertTrue(params.getParameter().size() == 3); assertTrue(params.getParameter("parameters").hasResource()); @@ -53,7 +59,7 @@ public class CdsCrServiceR4Test extends BaseCrTest { final RequestDetails requestDetails = new SystemRequestDetails(); final IdType planDefinitionId = new IdType(PLAN_DEFINITION_RESOURCE_NAME, "ASLPCrd"); requestDetails.setId(planDefinitionId); - final CdsServiceResponseJson cdsServiceResponseJson = new CdsCrServiceR4(requestDetails, repository).encodeResponse(responseBundle); + final CdsServiceResponseJson cdsServiceResponseJson = new CdsCrServiceR4(requestDetails, repository, myCdsConfigService).encodeResponse(responseBundle); assertTrue(cdsServiceResponseJson.getCards().size() == 1); assertTrue(!cdsServiceResponseJson.getCards().get(0).getSummary().isEmpty()); @@ -69,7 +75,7 @@ public class CdsCrServiceR4Test extends BaseCrTest { final RequestDetails requestDetails = new SystemRequestDetails(); final IdType planDefinitionId = new IdType(PLAN_DEFINITION_RESOURCE_NAME, "DischargeInstructionsPlan"); requestDetails.setId(planDefinitionId); - final CdsServiceResponseJson cdsServiceResponseJson = new CdsCrServiceR4(requestDetails, repository).encodeResponse(responseBundle); + final CdsServiceResponseJson cdsServiceResponseJson = new CdsCrServiceR4(requestDetails, repository, myCdsConfigService).encodeResponse(responseBundle); assertTrue(cdsServiceResponseJson.getServiceActions().size() == 1); assertTrue(cdsServiceResponseJson.getServiceActions().get(0).getType().equals(ActionType.CREATE.toCode())); diff --git a/hapi-fhir-server-cds-hooks/src/test/resources/ASLPCrdServiceRequest.json b/hapi-fhir-server-cds-hooks/src/test/resources/ASLPCrdServiceRequest.json index 474e666e302..422a9a05ea2 100644 --- a/hapi-fhir-server-cds-hooks/src/test/resources/ASLPCrdServiceRequest.json +++ b/hapi-fhir-server-cds-hooks/src/test/resources/ASLPCrdServiceRequest.json @@ -2,6 +2,13 @@ "hook" : "order-sign", "hookInstance": "randomGUIDforthehookevent", "fhirServer" : "https://localhost:8000", + "fhirAuthorization": { + "access_token": "sometoken", + "token_type": "Bearer", + "expires_in": 300000, + "scope": "", + "subject": "clientIdHeaderTest" + }, "context" : { "patientId" : "Patient/123", "draftOrders" : { diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/RestfulServer.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/RestfulServer.java index 02dbd0d5ccd..550a9d28fd3 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/RestfulServer.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/RestfulServer.java @@ -1598,9 +1598,16 @@ public class RestfulServer extends HttpServlet implements IRestfulServer excludedParameterNames) { + String tenantId = StringUtils.defaultString(theRequest.getTenantId()); + String requestPath = StringUtils.defaultString(theRequest.getRequestPath()); + StringBuilder b = new StringBuilder(); b.append(theServerBase); + requestPath = StringUtils.substringAfter(requestPath, tenantId); - if (isNotBlank(theRequest.getRequestPath())) { - b.append('/'); - if (isNotBlank(theRequest.getTenantId()) - && theRequest.getRequestPath().startsWith(theRequest.getTenantId() + "/")) { - b.append(theRequest - .getRequestPath() - .substring(theRequest.getTenantId().length() + 1)); - } else { - b.append(theRequest.getRequestPath()); - } + if (isNotBlank(requestPath)) { + requestPath = StringUtils.prependIfMissing(requestPath, "/"); } + + b.append(requestPath); + // For POST the URL parameters get jumbled with the post body parameters so don't include them, they might be // huge if (theRequest.getRequestType() == RequestTypeEnum.GET) { @@ -211,7 +211,6 @@ public class RestfulServerUtils { } } } - return b.toString(); } diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/ConsentInterceptor.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/ConsentInterceptor.java index 95b0481c9bf..512f6b47731 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/ConsentInterceptor.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/ConsentInterceptor.java @@ -53,6 +53,8 @@ import java.util.List; import java.util.Map; import java.util.concurrent.atomic.AtomicInteger; import java.util.stream.Collectors; +import javax.annotation.Nonnull; +import javax.annotation.Nullable; import static ca.uhn.fhir.rest.api.Constants.URL_TOKEN_METADATA; import static ca.uhn.fhir.rest.server.provider.ProviderConstants.OPERATION_META; @@ -178,18 +180,29 @@ public class ConsentInterceptor { } } + /** + * Check if this request is eligible for cached search results. + * We can't use a cached result if consent may use canSeeResource. + * This checks for AUTHORIZED requests, and the responses from shouldProcessCanSeeResource() + * to see if this holds. + * @return may the request be satisfied from cache. + */ @Hook(value = Pointcut.STORAGE_PRECHECK_FOR_CACHED_SEARCH) - public boolean interceptPreCheckForCachedSearch(RequestDetails theRequestDetails) { - if (isRequestAuthorized(theRequestDetails)) { - return true; - } - return false; + public boolean interceptPreCheckForCachedSearch(@Nonnull RequestDetails theRequestDetails) { + return !isProcessCanSeeResource(theRequestDetails, null); } + /** + * Check if the search results from this request might be reused by later searches. + * We can't use a cached result if consent may use canSeeResource. + * This checks for AUTHORIZED requests, and the responses from shouldProcessCanSeeResource() + * to see if this holds. + * If not, marks the result as single-use. + */ @Hook(value = Pointcut.STORAGE_PRESEARCH_REGISTERED) public void interceptPreSearchRegistered( RequestDetails theRequestDetails, ICachedSearchDetails theCachedSearchDetails) { - if (!isRequestAuthorized(theRequestDetails)) { + if (isProcessCanSeeResource(theRequestDetails, null)) { theCachedSearchDetails.setCannotBeReused(); } } @@ -197,28 +210,10 @@ public class ConsentInterceptor { @Hook(value = Pointcut.STORAGE_PREACCESS_RESOURCES) public void interceptPreAccess( RequestDetails theRequestDetails, IPreResourceAccessDetails thePreResourceAccessDetails) { - if (isRequestAuthorized(theRequestDetails)) { - return; - } - if (isSkipServiceForRequest(theRequestDetails)) { - return; - } - if (myConsentService.isEmpty()) { - return; - } - // First check if we should be calling canSeeResource for the individual - // consent services + // Flags for each service boolean[] processConsentSvcs = new boolean[myConsentService.size()]; - boolean processAnyConsentSvcs = false; - for (int consentSvcIdx = 0; consentSvcIdx < myConsentService.size(); consentSvcIdx++) { - IConsentService nextService = myConsentService.get(consentSvcIdx); - - boolean shouldCallCanSeeResource = - nextService.shouldProcessCanSeeResource(theRequestDetails, myContextConsentServices); - processAnyConsentSvcs |= shouldCallCanSeeResource; - processConsentSvcs[consentSvcIdx] = shouldCallCanSeeResource; - } + boolean processAnyConsentSvcs = isProcessCanSeeResource(theRequestDetails, processConsentSvcs); if (!processAnyConsentSvcs) { return; @@ -262,6 +257,39 @@ public class ConsentInterceptor { } } + /** + * Is canSeeResource() active in any services? + * @param theProcessConsentSvcsFlags filled in with the responses from shouldProcessCanSeeResource each service + * @return true of any service responded true to shouldProcessCanSeeResource() + */ + private boolean isProcessCanSeeResource( + @Nonnull RequestDetails theRequestDetails, @Nullable boolean[] theProcessConsentSvcsFlags) { + if (isRequestAuthorized(theRequestDetails)) { + return false; + } + if (isSkipServiceForRequest(theRequestDetails)) { + return false; + } + if (myConsentService.isEmpty()) { + return false; + } + + if (theProcessConsentSvcsFlags == null) { + theProcessConsentSvcsFlags = new boolean[myConsentService.size()]; + } + Validate.isTrue(theProcessConsentSvcsFlags.length == myConsentService.size()); + boolean processAnyConsentSvcs = false; + for (int consentSvcIdx = 0; consentSvcIdx < myConsentService.size(); consentSvcIdx++) { + IConsentService nextService = myConsentService.get(consentSvcIdx); + + boolean shouldCallCanSeeResource = + nextService.shouldProcessCanSeeResource(theRequestDetails, myContextConsentServices); + processAnyConsentSvcs |= shouldCallCanSeeResource; + theProcessConsentSvcsFlags[consentSvcIdx] = shouldCallCanSeeResource; + } + return processAnyConsentSvcs; + } + @Hook(value = Pointcut.STORAGE_PRESHOW_RESOURCES) public void interceptPreShow(RequestDetails theRequestDetails, IPreResourceShowDetails thePreResourceShowDetails) { if (isRequestAuthorized(theRequestDetails)) { diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/DelegatingConsentService.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/DelegatingConsentService.java index f073a436b0a..e44fc3d80b1 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/DelegatingConsentService.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/interceptor/consent/DelegatingConsentService.java @@ -37,6 +37,12 @@ public class DelegatingConsentService implements IConsentService { return myTarget.startOperation(theRequestDetails, theContextServices); } + @Override + public boolean shouldProcessCanSeeResource( + RequestDetails theRequestDetails, IConsentContextServices theContextServices) { + return myTarget.shouldProcessCanSeeResource(theRequestDetails, theContextServices); + } + @Override public ConsentOutcome canSeeResource( RequestDetails theRequestDetails, IBaseResource theResource, IConsentContextServices theContextServices) { diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/messaging/BaseResourceModifiedMessage.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/messaging/BaseResourceModifiedMessage.java index c98030e643a..bda74f798d7 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/messaging/BaseResourceModifiedMessage.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/messaging/BaseResourceModifiedMessage.java @@ -52,15 +52,15 @@ public abstract class BaseResourceModifiedMessage extends BaseResourceMessage im @JsonProperty(value = "partitionId") protected RequestPartitionId myPartitionId; + @JsonProperty(value = "payloadVersion") + protected String myPayloadVersion; + @JsonIgnore protected transient IBaseResource myPayloadDecoded; @JsonIgnore protected transient String myPayloadType; - @JsonIgnore - protected String myPayloadVersion; - /** * Constructor */ @@ -68,6 +68,12 @@ public abstract class BaseResourceModifiedMessage extends BaseResourceMessage im super(); } + public BaseResourceModifiedMessage(IIdType theIdType, OperationTypeEnum theOperationType) { + this(); + setOperationType(theOperationType); + setPayloadId(theIdType); + } + public BaseResourceModifiedMessage( FhirContext theFhirContext, IBaseResource theResource, OperationTypeEnum theOperationType) { this(); diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/ITenantIdentificationStrategy.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/ITenantIdentificationStrategy.java index adac4d18ad9..356fc12cda6 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/ITenantIdentificationStrategy.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/ITenantIdentificationStrategy.java @@ -26,7 +26,7 @@ public interface ITenantIdentificationStrategy { /** * Implementations should use this method to determine the tenant ID - * based on the incoming request andand populate it in the + * based on the incoming request and populate it in the * {@link RequestDetails#setTenantId(String)}. * * @param theUrlPathTokenizer The tokenizer which is used to parse the request path @@ -39,4 +39,13 @@ public interface ITenantIdentificationStrategy { * if necessary based on the tenant ID */ String massageServerBaseUrl(String theFhirServerBase, RequestDetails theRequestDetails); + + /** + * Implementations may use this method to resolve relative URL based on the tenant ID from RequestDetails. + * + * @param theRelativeUrl URL that only includes the path, e.g. "Patient/123" + * @param theRequestDetails The request details object which can be used to access tenant ID + * @return Resolved relative URL that starts with tenant ID (if tenant ID present in RequestDetails). Example: "TENANT-A/Patient/123". + */ + String resolveRelativeUrl(String theRelativeUrl, RequestDetails theRequestDetails); } diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategy.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategy.java index affeaea0a12..344a515a583 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategy.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategy.java @@ -69,7 +69,7 @@ public class UrlBaseTenantIdentificationStrategy implements ITenantIdentificatio tenantId = defaultIfBlank(theUrlPathTokenizer.peek(), null); // If it's "metadata" or starts with "$", use DEFAULT partition and don't consume this token: - if (tenantId != null && (tenantId.equals("metadata") || tenantId.startsWith("$"))) { + if (tenantId != null && (tenantId.equals("metadata") || isOperation(tenantId))) { tenantId = "DEFAULT"; theRequestDetails.setTenantId(tenantId); ourLog.trace("No tenant ID found for metadata or system request; using DEFAULT."); @@ -94,6 +94,10 @@ public class UrlBaseTenantIdentificationStrategy implements ITenantIdentificatio } } + private boolean isOperation(String theToken) { + return theToken.startsWith("$"); + } + @Override public String massageServerBaseUrl(String theFhirServerBase, RequestDetails theRequestDetails) { String result = theFhirServerBase; @@ -102,4 +106,29 @@ public class UrlBaseTenantIdentificationStrategy implements ITenantIdentificatio } return result; } + + @Override + public String resolveRelativeUrl(String theRelativeUrl, RequestDetails theRequestDetails) { + UrlPathTokenizer tokenizer = new UrlPathTokenizer(theRelativeUrl); + // there is no more tokens in the URL - skip url resolution + if (!tokenizer.hasMoreTokens() || tokenizer.peek() == null) { + return theRelativeUrl; + } + String nextToken = tokenizer.peek(); + // there is no tenant ID in parent request details or tenant ID is already present in URL - skip url resolution + if (theRequestDetails.getTenantId() == null || nextToken.equals(theRequestDetails.getTenantId())) { + return theRelativeUrl; + } + + // token is Resource type or operation - adding tenant ID from parent request details + if (isResourceType(nextToken, theRequestDetails) || isOperation(nextToken)) { + return theRequestDetails.getTenantId() + "/" + theRelativeUrl; + } else { + return theRelativeUrl; + } + } + + private boolean isResourceType(String token, RequestDetails theRequestDetails) { + return theRequestDetails.getFhirContext().getResourceTypes().stream().anyMatch(type -> type.equals(token)); + } } diff --git a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/util/ServletRequestUtil.java b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/util/ServletRequestUtil.java index 44ce54a368a..24bafe71c62 100644 --- a/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/util/ServletRequestUtil.java +++ b/hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/util/ServletRequestUtil.java @@ -63,6 +63,7 @@ public class ServletRequestUtil { requestDetails.setRequestPath(url); requestDetails.setFhirServerBase(theRequestDetails.getFhirServerBase()); + requestDetails.setTenantId(theRequestDetails.getTenantId()); theRequestDetails.getServer().populateRequestDetailsFromRequestPath(requestDetails, url); return requestDetails; diff --git a/hapi-fhir-server/src/test/java/ca/uhn/fhir/rest/server/RestfulServerUtilsTest.java b/hapi-fhir-server/src/test/java/ca/uhn/fhir/rest/server/RestfulServerUtilsTest.java index 9b8a159663b..d3399c06bb9 100644 --- a/hapi-fhir-server/src/test/java/ca/uhn/fhir/rest/server/RestfulServerUtilsTest.java +++ b/hapi-fhir-server/src/test/java/ca/uhn/fhir/rest/server/RestfulServerUtilsTest.java @@ -1,13 +1,20 @@ package ca.uhn.fhir.rest.server; -import ca.uhn.fhir.rest.api.*; +import ca.uhn.fhir.rest.api.Constants; +import ca.uhn.fhir.rest.api.DeleteCascadeModeEnum; +import ca.uhn.fhir.rest.api.PreferHandlingEnum; +import ca.uhn.fhir.rest.api.PreferHeader; +import ca.uhn.fhir.rest.api.PreferReturnEnum; import ca.uhn.fhir.rest.api.server.RequestDetails; import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException; import ca.uhn.fhir.rest.server.servlet.ServletRequestDetails; +import org.apache.commons.lang3.StringUtils; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; import org.junit.jupiter.params.provider.CsvSource; +import org.junit.jupiter.params.provider.MethodSource; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; @@ -15,13 +22,19 @@ import java.util.Arrays; import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.stream.Stream; import static ca.uhn.fhir.rest.api.RequestTypeEnum.GET; +import static ca.uhn.fhir.rest.api.RequestTypeEnum.POST; import static org.apache.commons.lang3.StringUtils.isNotBlank; +import static org.apache.http.util.TextUtils.isBlank; import static org.hamcrest.CoreMatchers.is; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.containsString; -import static org.junit.jupiter.api.Assertions.*; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; import static org.mockito.Mockito.when; @ExtendWith(MockitoExtension.class) @@ -149,6 +162,33 @@ public class RestfulServerUtilsTest { //Then assertThat(linkSelfWithoutGivenParameters, is(containsString("http://localhost:8000/$my-operation?"))); assertThat(linkSelfWithoutGivenParameters, is(containsString("_format=json"))); + } + @ParameterizedTest + @MethodSource("testParameters") + public void testCreateSelfLinks_withDifferentResourcePathAndTenantId(String theServerBaseUrl, String theRequestPath, + String theTenantId, String theExpectedUrl) { + //When + ServletRequestDetails servletRequestDetails = new ServletRequestDetails(); + servletRequestDetails.setRequestType(POST); + servletRequestDetails.setTenantId(StringUtils.defaultString(theTenantId)); + servletRequestDetails.setRequestPath(StringUtils.defaultString(theRequestPath)); + + //Then + String linkSelfWithoutGivenParameters = RestfulServerUtils.createLinkSelfWithoutGivenParameters(theServerBaseUrl, servletRequestDetails, null); + //Test + assertEquals(theExpectedUrl, linkSelfWithoutGivenParameters); + } + static Stream testParameters(){ + return Stream.of( + Arguments.of("http://localhost:8000/Partition-B","" ,"Partition-B","http://localhost:8000/Partition-B"), + Arguments.of("http://localhost:8000/Partition-B","Partition-B" ,"Partition-B","http://localhost:8000/Partition-B"), + Arguments.of("http://localhost:8000/Partition-B","Partition-B/Patient" ,"Partition-B","http://localhost:8000/Partition-B/Patient"), + Arguments.of("http://localhost:8000/Partition-B","Partition-B/$my-operation" ,"Partition-B","http://localhost:8000/Partition-B/$my-operation"), + Arguments.of("http://localhost:8000","","","http://localhost:8000"), + Arguments.of("", "","",""), + Arguments.of("http://localhost:8000","Patient","","http://localhost:8000/Patient"), + Arguments.of("http://localhost:8000/Patient","","","http://localhost:8000/Patient") + ); } } diff --git a/hapi-fhir-storage-cr/pom.xml b/hapi-fhir-storage-cr/pom.xml index 2ea85ebb887..afec2b537cb 100644 --- a/hapi-fhir-storage-cr/pom.xml +++ b/hapi-fhir-storage-cr/pom.xml @@ -139,12 +139,6 @@ ${spring-security-core.version} - - org.springframework.boot - spring-boot-autoconfigure - ${spring_boot_version} - - javax.servlet diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/CrConfigCondition.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/CrConfigCondition.java new file mode 100644 index 00000000000..0461701e492 --- /dev/null +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/CrConfigCondition.java @@ -0,0 +1,60 @@ +/*- + * #%L + * HAPI FHIR - Clinical Reasoning + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.cr.config; + +import ca.uhn.fhir.rest.server.RestfulServer; +import org.opencds.cqf.fhir.cql.EvaluationSettings; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.context.annotation.Condition; +import org.springframework.context.annotation.ConditionContext; +import org.springframework.core.type.AnnotatedTypeMetadata; + +/** + * The purpose of this Condition is to verify that the CR dependent beans RestfulServer and EvaluationSettings exist. + */ +public class CrConfigCondition implements Condition { + private static final Logger ourLog = LoggerFactory.getLogger(CrConfigCondition.class); + + @Override + public boolean matches(ConditionContext theConditionContext, AnnotatedTypeMetadata theAnnotatedTypeMetadata) { + ConfigurableListableBeanFactory beanFactory = theConditionContext.getBeanFactory(); + try { + RestfulServer bean = beanFactory.getBean(RestfulServer.class); + if (bean == null) { + return false; + } + } catch (Exception e) { + ourLog.warn("CrConfigCondition not met: Missing RestfulServer bean"); + return false; + } + try { + EvaluationSettings bean = beanFactory.getBean(EvaluationSettings.class); + if (bean == null) { + return false; + } + } catch (Exception e) { + ourLog.warn("CrConfigCondition not met: Missing EvaluationSettings bean"); + return false; + } + return true; + } +} diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfig.java index 5166a5ebaf6..8504c2e2060 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfig.java @@ -23,12 +23,10 @@ import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.repo.HapiFhirRepository; import ca.uhn.fhir.jpa.api.dao.DaoRegistry; import ca.uhn.fhir.rest.server.RestfulServer; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration -@ConditionalOnBean(RestfulServer.class) public class RepositoryConfig { @Bean IRepositoryFactory repositoryFactory(DaoRegistry theDaoRegistry, RestfulServer theRestfulServer) { diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfigCondition.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfigCondition.java new file mode 100644 index 00000000000..caa43d8cd6c --- /dev/null +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/RepositoryConfigCondition.java @@ -0,0 +1,47 @@ +/*- + * #%L + * HAPI FHIR - Clinical Reasoning + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.cr.config; + +import ca.uhn.fhir.rest.server.RestfulServer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; +import org.springframework.context.annotation.Condition; +import org.springframework.context.annotation.ConditionContext; +import org.springframework.core.type.AnnotatedTypeMetadata; + +public class RepositoryConfigCondition implements Condition { + private static final Logger ourLog = LoggerFactory.getLogger(RepositoryConfigCondition.class); + + @Override + public boolean matches(ConditionContext theConditionContext, AnnotatedTypeMetadata theAnnotatedTypeMetadata) { + ConfigurableListableBeanFactory beanFactory = theConditionContext.getBeanFactory(); + try { + RestfulServer bean = beanFactory.getBean(RestfulServer.class); + if (bean == null) { + return false; + } + } catch (Exception e) { + ourLog.warn("Unable to create bean IRepositoryFactory: Missing RestfulServer"); + return false; + } + return true; + } +} diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ApplyOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ApplyOperationConfig.java index d72885a63e6..cbbcd5aa82f 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ApplyOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ApplyOperationConfig.java @@ -21,36 +21,20 @@ package ca.uhn.fhir.cr.config.dstu3; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class ApplyOperationConfig { - @Bean - ca.uhn.fhir.cr.dstu3.IActivityDefinitionProcessorFactory dstu3ActivityDefinitionProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.activitydefinition.dstu3.ActivityDefinitionProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - - @Bean - ca.uhn.fhir.cr.dstu3.IPlanDefinitionProcessorFactory dstu3PlanDefinitionProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.plandefinition.dstu3.PlanDefinitionProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.dstu3.activitydefinition.ActivityDefinitionApplyProvider dstu3ActivityDefinitionApplyProvider() { return new ca.uhn.fhir.cr.dstu3.activitydefinition.ActivityDefinitionApplyProvider(); diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/CrProcessorConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/CrProcessorConfig.java new file mode 100644 index 00000000000..b039eb9514e --- /dev/null +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/CrProcessorConfig.java @@ -0,0 +1,56 @@ +/*- + * #%L + * HAPI FHIR - Clinical Reasoning + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.cr.config.dstu3; + +import ca.uhn.fhir.cr.common.IRepositoryFactory; +import org.opencds.cqf.fhir.cql.EvaluationSettings; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +@Configuration +public class CrProcessorConfig { + @Bean + ca.uhn.fhir.cr.dstu3.IActivityDefinitionProcessorFactory dstu3ActivityDefinitionProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.activitydefinition.dstu3.ActivityDefinitionProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } + + @Bean + ca.uhn.fhir.cr.dstu3.IPlanDefinitionProcessorFactory dstu3PlanDefinitionProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.plandefinition.dstu3.PlanDefinitionProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } + + @Bean + ca.uhn.fhir.cr.dstu3.IQuestionnaireProcessorFactory dstu3QuestionnaireProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.questionnaire.dstu3.QuestionnaireProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } + + @Bean + ca.uhn.fhir.cr.dstu3.IQuestionnaireResponseProcessorFactory dstu3QuestionnaireResponseProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.questionnaireresponse.dstu3.QuestionnaireResponseProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } +} diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ExtractOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ExtractOperationConfig.java index 773e57da71c..6dd20561530 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ExtractOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/ExtractOperationConfig.java @@ -21,29 +21,20 @@ package ca.uhn.fhir.cr.config.dstu3; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class ExtractOperationConfig { - @Bean - ca.uhn.fhir.cr.dstu3.IQuestionnaireResponseProcessorFactory dstu3QuestionnaireResponseProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.questionnaireresponse.dstu3.QuestionnaireResponseProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.dstu3.questionnaireresponse.QuestionnaireResponseExtractProvider dstu3QuestionnaireResponseExtractProvider() { diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PackageOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PackageOperationConfig.java index 1617dfdfcbf..0c6d37040cd 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PackageOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PackageOperationConfig.java @@ -21,41 +21,25 @@ package ca.uhn.fhir.cr.config.dstu3; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class PackageOperationConfig { - @Bean - ca.uhn.fhir.cr.dstu3.IPlanDefinitionProcessorFactory dstu3PlanDefinitionProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.plandefinition.dstu3.PlanDefinitionProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.dstu3.plandefinition.PlanDefinitionPackageProvider dstu3PlanDefinitionPackageProvider() { return new ca.uhn.fhir.cr.dstu3.plandefinition.PlanDefinitionPackageProvider(); } - @Bean - ca.uhn.fhir.cr.dstu3.IQuestionnaireProcessorFactory dstu3QuestionnaireProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.questionnaire.dstu3.QuestionnaireProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.dstu3.questionnaire.QuestionnairePackageProvider dstu3QuestionnairePackageProvider() { return new ca.uhn.fhir.cr.dstu3.questionnaire.QuestionnairePackageProvider(); diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PopulateOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PopulateOperationConfig.java index bccaab2d578..8cb3da6c55b 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PopulateOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/dstu3/PopulateOperationConfig.java @@ -21,29 +21,20 @@ package ca.uhn.fhir.cr.config.dstu3; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class PopulateOperationConfig { - @Bean - ca.uhn.fhir.cr.dstu3.IQuestionnaireProcessorFactory dstu3QuestionnaireProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.questionnaire.dstu3.QuestionnaireProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.dstu3.questionnaire.QuestionnairePopulateProvider dstu3QuestionnairePopulateProvider() { return new ca.uhn.fhir.cr.dstu3.questionnaire.QuestionnairePopulateProvider(); diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ApplyOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ApplyOperationConfig.java index 89d732b54fd..a4cb102e45b 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ApplyOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ApplyOperationConfig.java @@ -21,36 +21,20 @@ package ca.uhn.fhir.cr.config.r4; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class ApplyOperationConfig { - @Bean - ca.uhn.fhir.cr.r4.IActivityDefinitionProcessorFactory r4ActivityDefinitionProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.activitydefinition.r4.ActivityDefinitionProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - - @Bean - ca.uhn.fhir.cr.r4.IPlanDefinitionProcessorFactory r4PlanDefinitionProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.plandefinition.r4.PlanDefinitionProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.r4.activitydefinition.ActivityDefinitionApplyProvider r4ActivityDefinitionApplyProvider() { return new ca.uhn.fhir.cr.r4.activitydefinition.ActivityDefinitionApplyProvider(); diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/CrProcessorConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/CrProcessorConfig.java new file mode 100644 index 00000000000..6632a244c39 --- /dev/null +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/CrProcessorConfig.java @@ -0,0 +1,56 @@ +/*- + * #%L + * HAPI FHIR - Clinical Reasoning + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.cr.config.r4; + +import ca.uhn.fhir.cr.common.IRepositoryFactory; +import org.opencds.cqf.fhir.cql.EvaluationSettings; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +@Configuration +public class CrProcessorConfig { + @Bean + ca.uhn.fhir.cr.r4.IActivityDefinitionProcessorFactory r4ActivityDefinitionProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.activitydefinition.r4.ActivityDefinitionProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } + + @Bean + ca.uhn.fhir.cr.r4.IPlanDefinitionProcessorFactory r4PlanDefinitionProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.plandefinition.r4.PlanDefinitionProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } + + @Bean + ca.uhn.fhir.cr.r4.IQuestionnaireProcessorFactory r4QuestionnaireProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.questionnaire.r4.QuestionnaireProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } + + @Bean + ca.uhn.fhir.cr.r4.IQuestionnaireResponseProcessorFactory r4QuestionnaireResponseProcessorFactory( + IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { + return rd -> new org.opencds.cqf.fhir.cr.questionnaireresponse.r4.QuestionnaireResponseProcessor( + theRepositoryFactory.create(rd), theEvaluationSettings); + } +} diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ExtractOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ExtractOperationConfig.java index a33f1267742..45a031dd910 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ExtractOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/ExtractOperationConfig.java @@ -21,29 +21,20 @@ package ca.uhn.fhir.cr.config.r4; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class ExtractOperationConfig { - @Bean - ca.uhn.fhir.cr.r4.IQuestionnaireResponseProcessorFactory r4QuestionnaireResponseProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.questionnaireresponse.r4.QuestionnaireResponseProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.r4.questionnaireresponse.QuestionnaireResponseExtractProvider r4QuestionnaireResponseExtractProvider() { diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PackageOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PackageOperationConfig.java index 7429a0dfe53..ffbd0c29dfc 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PackageOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PackageOperationConfig.java @@ -21,41 +21,25 @@ package ca.uhn.fhir.cr.config.r4; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class PackageOperationConfig { - @Bean - ca.uhn.fhir.cr.r4.IPlanDefinitionProcessorFactory r4PlanDefinitionProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.plandefinition.r4.PlanDefinitionProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.r4.plandefinition.PlanDefinitionPackageProvider r4PlanDefinitionPackageProvider() { return new ca.uhn.fhir.cr.r4.plandefinition.PlanDefinitionPackageProvider(); } - @Bean - ca.uhn.fhir.cr.r4.IQuestionnaireProcessorFactory r4QuestionnaireProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.questionnaire.r4.QuestionnaireProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.r4.questionnaire.QuestionnairePackageProvider r4QuestionnairePackageProvider() { return new ca.uhn.fhir.cr.r4.questionnaire.QuestionnairePackageProvider(); diff --git a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PopulateOperationConfig.java b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PopulateOperationConfig.java index da10bb376f6..87cdc729be3 100644 --- a/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PopulateOperationConfig.java +++ b/hapi-fhir-storage-cr/src/main/java/ca/uhn/fhir/cr/config/r4/PopulateOperationConfig.java @@ -21,29 +21,20 @@ package ca.uhn.fhir.cr.config.r4; import ca.uhn.fhir.context.FhirContext; import ca.uhn.fhir.context.FhirVersionEnum; -import ca.uhn.fhir.cr.common.IRepositoryFactory; import ca.uhn.fhir.cr.config.ProviderLoader; import ca.uhn.fhir.cr.config.ProviderSelector; import ca.uhn.fhir.rest.server.RestfulServer; -import org.opencds.cqf.fhir.cql.EvaluationSettings; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; import org.springframework.context.ApplicationContext; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Import; import java.util.Arrays; import java.util.Map; @Configuration -@ConditionalOnBean({IRepositoryFactory.class, RestfulServer.class, EvaluationSettings.class}) +@Import(CrProcessorConfig.class) public class PopulateOperationConfig { - @Bean - ca.uhn.fhir.cr.r4.IQuestionnaireProcessorFactory r4QuestionnaireProcessorFactory( - IRepositoryFactory theRepositoryFactory, EvaluationSettings theEvaluationSettings) { - return rd -> new org.opencds.cqf.fhir.cr.questionnaire.r4.QuestionnaireProcessor( - theRepositoryFactory.create(rd), theEvaluationSettings); - } - @Bean ca.uhn.fhir.cr.r4.questionnaire.QuestionnairePopulateProvider r4QuestionnairePopulateProvider() { return new ca.uhn.fhir.cr.r4.questionnaire.QuestionnairePopulateProvider(); diff --git a/hapi-fhir-storage-test-utilities/src/main/java/ca/uhn/fhir/storage/test/DaoTestDataBuilder.java b/hapi-fhir-storage-test-utilities/src/main/java/ca/uhn/fhir/storage/test/DaoTestDataBuilder.java index 625addbd379..012a2856ca9 100644 --- a/hapi-fhir-storage-test-utilities/src/main/java/ca/uhn/fhir/storage/test/DaoTestDataBuilder.java +++ b/hapi-fhir-storage-test-utilities/src/main/java/ca/uhn/fhir/storage/test/DaoTestDataBuilder.java @@ -39,7 +39,8 @@ import org.springframework.context.annotation.Configuration; /** * Implements ITestDataBuilder via a live DaoRegistry. - * + * Note: this implements {@link AfterEachCallback} and will delete any resources created when registered + * via {@link org.junit.jupiter.api.extension.RegisterExtension}. * Add the inner {@link Config} to your spring context to inject this. * For convenience, you can still implement ITestDataBuilder on your test class, and delegate the missing methods to this bean. */ @@ -75,7 +76,9 @@ public class DaoTestDataBuilder implements ITestDataBuilder.WithSupport, ITestDa //noinspection rawtypes IFhirResourceDao dao = myDaoRegistry.getResourceDao(theResource.getClass()); //noinspection unchecked - return dao.update(theResource, mySrd).getId().toUnqualifiedVersionless(); + IIdType id = dao.update(theResource, mySrd).getId().toUnqualifiedVersionless(); + myIds.put(theResource.fhirType(), id); + return id; } @Override diff --git a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/binary/interceptor/BinaryStorageInterceptor.java b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/binary/interceptor/BinaryStorageInterceptor.java index 0ff58ec7cf3..bf32c4a758f 100644 --- a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/binary/interceptor/BinaryStorageInterceptor.java +++ b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/binary/interceptor/BinaryStorageInterceptor.java @@ -251,7 +251,7 @@ public class BinaryStorageInterceptor> { } if (myBinaryStorageSvc.isValidBlobId(newBlobId)) { List deferredBinaryTargets = - getOrCreateDeferredBinaryStorageMap(theTransactionDetails); + getOrCreateDeferredBinaryStorageList(theResource); DeferredBinaryTarget newDeferredBinaryTarget = new DeferredBinaryTarget(newBlobId, nextTarget, data); deferredBinaryTargets.add(newDeferredBinaryTarget); @@ -289,21 +289,29 @@ public class BinaryStorageInterceptor> { } @Nonnull - private List getOrCreateDeferredBinaryStorageMap(TransactionDetails theTransactionDetails) { - return theTransactionDetails.getOrCreateUserData(getDeferredListKey(), ArrayList::new); + @SuppressWarnings("unchecked") + private List getOrCreateDeferredBinaryStorageList(IBaseResource theResource) { + Object deferredBinaryTargetList = theResource.getUserData(getDeferredListKey()); + if (deferredBinaryTargetList == null) { + deferredBinaryTargetList = new ArrayList<>(); + theResource.setUserData(getDeferredListKey(), deferredBinaryTargetList); + } + return (List) deferredBinaryTargetList; } + @SuppressWarnings("unchecked") @Hook(Pointcut.STORAGE_PRECOMMIT_RESOURCE_CREATED) public void storeLargeBinariesBeforeCreatePersistence( - TransactionDetails theTransactionDetails, IBaseResource theResource, Pointcut thePoincut) + TransactionDetails theTransactionDetails, IBaseResource theResource, Pointcut thePointcut) throws IOException { - if (theTransactionDetails == null) { + if (theResource == null) { return; } - List deferredBinaryTargets = theTransactionDetails.getUserData(getDeferredListKey()); - if (deferredBinaryTargets != null) { + Object deferredBinaryTargetList = theResource.getUserData(getDeferredListKey()); + + if (deferredBinaryTargetList != null) { IIdType resourceId = theResource.getIdElement(); - for (DeferredBinaryTarget next : deferredBinaryTargets) { + for (DeferredBinaryTarget next : (List) deferredBinaryTargetList) { String blobId = next.getBlobId(); IBinaryTarget target = next.getBinaryTarget(); InputStream dataStream = next.getDataStream(); diff --git a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/dao/tx/HapiTransactionService.java b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/dao/tx/HapiTransactionService.java index 50d4669be68..56395199c9f 100644 --- a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/dao/tx/HapiTransactionService.java +++ b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/dao/tx/HapiTransactionService.java @@ -514,7 +514,8 @@ public class HapiTransactionService implements IHapiTransactionService { } @Nullable - private static T executeInExistingTransaction(TransactionCallback theCallback) { + private static T executeInExistingTransaction(@Nonnull TransactionCallback theCallback) { + // TODO we could probably track the TransactionStatus we need as a thread local like we do our partition id. return theCallback.doInTransaction(null); } diff --git a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/channel/api/PayloadTooLargeException.java b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/channel/api/PayloadTooLargeException.java new file mode 100644 index 00000000000..daac9c18948 --- /dev/null +++ b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/channel/api/PayloadTooLargeException.java @@ -0,0 +1,35 @@ +/*- + * #%L + * HAPI FHIR Storage api + * %% + * Copyright (C) 2014 - 2023 Smile CDR, Inc. + * %% + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * #L% + */ +package ca.uhn.fhir.jpa.subscription.channel.api; + +/** + * This exception represents the message payload exceeded the maximum message size of the broker. Used as a wrapper of + * similar exceptions specific to different message brokers, e.g. kafka.common.errors.RecordTooLargeException. + */ +public class PayloadTooLargeException extends RuntimeException { + + public PayloadTooLargeException(String theMessage) { + super(theMessage); + } + + public PayloadTooLargeException(String theMessage, Throwable theThrowable) { + super(theMessage, theThrowable); + } +} diff --git a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceDeliveryMessage.java b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceDeliveryMessage.java index 9cd638d2600..04cf84a0cc6 100644 --- a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceDeliveryMessage.java +++ b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceDeliveryMessage.java @@ -108,6 +108,10 @@ public class ResourceDeliveryMessage extends BaseResourceMessage implements IRes myPayloadId = thePayload.getIdElement().toUnqualifiedVersionless().getValue(); } + public void setPayloadToNull() { + myPayloadString = null; + } + @Override public String getPayloadId() { return myPayloadId; diff --git a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceModifiedMessage.java b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceModifiedMessage.java index ab4fe6c7de9..e493035f80c 100644 --- a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceModifiedMessage.java +++ b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/jpa/subscription/model/ResourceModifiedMessage.java @@ -26,6 +26,7 @@ import ca.uhn.fhir.rest.server.messaging.BaseResourceModifiedMessage; import com.fasterxml.jackson.annotation.JsonProperty; import org.apache.commons.lang3.builder.ToStringBuilder; import org.hl7.fhir.instance.model.api.IBaseResource; +import org.hl7.fhir.instance.model.api.IIdType; /** * Most of this class has been moved to ResourceModifiedMessage in the hapi-fhir-server project, for a reusable channel ResourceModifiedMessage @@ -47,6 +48,11 @@ public class ResourceModifiedMessage extends BaseResourceModifiedMessage { super(); } + public ResourceModifiedMessage(IIdType theIdType, OperationTypeEnum theOperationType) { + super(theIdType, theOperationType); + setPartitionId(RequestPartitionId.defaultPartition()); + } + public ResourceModifiedMessage( FhirContext theFhirContext, IBaseResource theResource, OperationTypeEnum theOperationType) { super(theFhirContext, theResource, theOperationType); @@ -79,6 +85,10 @@ public class ResourceModifiedMessage extends BaseResourceModifiedMessage { mySubscriptionId = theSubscriptionId; } + public void setPayloadToNull() { + myPayload = null; + } + @Override public String toString() { return new ToStringBuilder(this) diff --git a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/subscription/api/IResourceModifiedMessagePersistenceSvc.java b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/subscription/api/IResourceModifiedMessagePersistenceSvc.java index 68aad03a48c..5f54b04e544 100644 --- a/hapi-fhir-storage/src/main/java/ca/uhn/fhir/subscription/api/IResourceModifiedMessagePersistenceSvc.java +++ b/hapi-fhir-storage/src/main/java/ca/uhn/fhir/subscription/api/IResourceModifiedMessagePersistenceSvc.java @@ -25,6 +25,7 @@ import ca.uhn.fhir.jpa.model.entity.IPersistedResourceModifiedMessagePK; import ca.uhn.fhir.jpa.subscription.model.ResourceModifiedMessage; import java.util.List; +import java.util.Optional; /** * An implementer of this interface will provide {@link ResourceModifiedMessage} persistence services. @@ -61,10 +62,29 @@ public interface IResourceModifiedMessagePersistenceSvc { /** * Restore a resourceModifiedMessage to its pre persistence representation. * - * @param thePersistedResourceModifiedMessage The message needing restoration. + * @param theResourceModifiedMessage The message needing restoration. * @return The resourceModifiedMessage in its pre persistence form. */ - ResourceModifiedMessage inflatePersistedResourceModifiedMessage( + ResourceModifiedMessage inflatePersistedResourceModifiedMessage(ResourceModifiedMessage theResourceModifiedMessage); + + /** + * Restore a resourceModifiedMessage to its pre persistence representation or null if the resource does not exist. + * + * @param theResourceModifiedMessage + * @return An Optional containing The resourceModifiedMessage in its pre persistence form or null when the resource + * does not exist + */ + Optional inflatePersistedResourceModifiedMessageOrNull( + ResourceModifiedMessage theResourceModifiedMessage); + + /** + * Create a ResourceModifiedMessage without its pre persistence representation, i.e. without the resource body in + * payload + * + * @param thePersistedResourceModifiedMessage The message needing creation + * @return The resourceModifiedMessage without its pre persistence form + */ + ResourceModifiedMessage createResourceModifiedMessageFromEntityWithoutInflation( IPersistedResourceModifiedMessage thePersistedResourceModifiedMessage); /** diff --git a/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/parser/RDFParserR4Test.java b/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/parser/RDFParserR4Test.java index 0d20454ddae..ea370b70b12 100644 --- a/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/parser/RDFParserR4Test.java +++ b/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/parser/RDFParserR4Test.java @@ -55,9 +55,9 @@ public class RDFParserR4Test { @prefix xsd: . - rdf:type fhir:Patient ; - fhir:Patient.active [ fhir:value true ] ; - fhir:Resource.id [ fhir:value "123" ] ; + rdf:type fhir:Patient; + fhir:Patient.active [ fhir:value true ]; + fhir:Resource.id [ fhir:value "123" ]; fhir:nodeRole fhir:treeRoot . """; diff --git a/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/rest/server/interceptor/ConsentInterceptorTest.java b/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/rest/server/interceptor/ConsentInterceptorTest.java index a27b8ae4134..784bd9d68c9 100644 --- a/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/rest/server/interceptor/ConsentInterceptorTest.java +++ b/hapi-fhir-structures-r4/src/test/java/ca/uhn/fhir/rest/server/interceptor/ConsentInterceptorTest.java @@ -18,8 +18,8 @@ import ca.uhn.fhir.rest.server.interceptor.consent.ConsentOutcome; import ca.uhn.fhir.rest.server.interceptor.consent.IConsentService; import ca.uhn.fhir.rest.server.provider.HashMapResourceProvider; import ca.uhn.fhir.rest.server.servlet.ServletRequestDetails; +import ca.uhn.fhir.rest.server.util.ICachedSearchDetails; import ca.uhn.fhir.test.utilities.HttpClientExtension; -import ca.uhn.fhir.test.utilities.LoggingExtension; import ca.uhn.fhir.test.utilities.server.RestfulServerExtension; import com.google.common.base.Charsets; import com.helger.commons.collection.iterate.EmptyEnumeration; @@ -36,6 +36,7 @@ import org.hl7.fhir.r4.model.Parameters; import org.hl7.fhir.r4.model.Patient; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Nested; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.junit.jupiter.api.extension.RegisterExtension; @@ -61,8 +62,11 @@ import static org.hamcrest.CoreMatchers.containsString; import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.not; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.never; import static org.mockito.Mockito.reset; import static org.mockito.Mockito.timeout; import static org.mockito.Mockito.verify; @@ -75,15 +79,13 @@ public class ConsentInterceptorTest { private static final org.slf4j.Logger ourLog = org.slf4j.LoggerFactory.getLogger(ConsentInterceptorTest.class); @RegisterExtension private final HttpClientExtension myClient = new HttpClientExtension(); - @RegisterExtension - private final LoggingExtension myLoggingExtension = new LoggingExtension(); private static final FhirContext ourCtx = FhirContext.forR4Cached(); private int myPort; private static final DummyPatientResourceProvider ourPatientProvider = new DummyPatientResourceProvider(ourCtx); private static final DummySystemProvider ourSystemProvider = new DummySystemProvider(); @RegisterExtension - private static final RestfulServerExtension ourServer = new RestfulServerExtension(ourCtx) + static final RestfulServerExtension ourServer = new RestfulServerExtension(ourCtx) .registerProvider(ourPatientProvider) .registerProvider(ourSystemProvider) .withPagingProvider(new FifoMemoryPagingProvider(10)); @@ -357,9 +359,7 @@ public class ConsentInterceptorTest { when(myConsentSvc.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); when(myConsentSvc.canSeeResource(any(RequestDetails.class), any(IBaseResource.class), any())).thenAnswer(t-> ConsentOutcome.PROCEED); - when(myConsentSvc.willSeeResource(any(RequestDetails.class), any(IBaseResource.class), any())).thenAnswer(t-> { - return ConsentOutcome.REJECT; - }); + when(myConsentSvc.willSeeResource(any(RequestDetails.class), any(IBaseResource.class), any())).thenAnswer(t-> ConsentOutcome.REJECT); HttpGet httpGet = new HttpGet("http://localhost:" + myPort + "/Patient"); @@ -861,7 +861,7 @@ public class ConsentInterceptorTest { @Test - public void testNoServicesRegistered() throws IOException { + public void testNoServicesRegistered() { myInterceptor.unregisterConsentService(myConsentSvc); Patient patientA = new Patient(); @@ -886,6 +886,52 @@ public class ConsentInterceptorTest { assertEquals(2, response.getTotal()); } + @Nested class CacheUsage { + @Mock ICachedSearchDetails myCachedSearchDetails; + ServletRequestDetails myRequestDetails = new ServletRequestDetails(); + + @Test + void testAuthorizedRequestsMayBeCachedAndUseCache() { + when(myConsentSvc.startOperation(any(), any())).thenReturn(ConsentOutcome.AUTHORIZED); + myInterceptor.interceptPreHandled(myRequestDetails); + + assertTrue(myInterceptor.interceptPreCheckForCachedSearch(myRequestDetails), "AUTHORIZED requests can use cache"); + + myInterceptor.interceptPreSearchRegistered(myRequestDetails, myCachedSearchDetails); + verify(myCachedSearchDetails, never()).setCannotBeReused(); + } + + @Test + void testCanSeeResourceFilteredRequestsMayNotBeCachedNorUseCache() { + when(myConsentSvc.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); + when(myConsentSvc.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); + when(myConsentSvc.shouldProcessCanSeeResource(any(), any())).thenReturn(true); + when(myConsentSvc2.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); + when(myConsentSvc2.shouldProcessCanSeeResource(any(), any())).thenReturn(false); + myInterceptor.registerConsentService(myConsentSvc2); + myInterceptor.interceptPreHandled(myRequestDetails); + + assertFalse(myInterceptor.interceptPreCheckForCachedSearch(myRequestDetails), "PROCEED requests can not use cache"); + + myInterceptor.interceptPreSearchRegistered(myRequestDetails, myCachedSearchDetails); + verify(myCachedSearchDetails).setCannotBeReused(); + } + + @Test + void testRequestsWithNoCanSeeFilteringMayBeCachedAndUseCache() { + when(myConsentSvc.startOperation(any(), any())).thenReturn(ConsentOutcome.PROCEED); + when(myConsentSvc.shouldProcessCanSeeResource(any(), any())).thenReturn(false); + myInterceptor.interceptPreHandled(myRequestDetails); + + assertTrue(myInterceptor.interceptPreCheckForCachedSearch(myRequestDetails), "PROCEED requests that promise not to filter can not use cache"); + + myInterceptor.interceptPreSearchRegistered(myRequestDetails, myCachedSearchDetails); + verify(myCachedSearchDetails, never()).setCannotBeReused(); + } + } + + + public static class DummyPatientResourceProvider extends HashMapResourceProvider { public DummyPatientResourceProvider(FhirContext theFhirContext) { diff --git a/hapi-fhir-test-utilities/src/test/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategyTest.java b/hapi-fhir-test-utilities/src/test/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategyTest.java index fcad982df60..4e96c19dc18 100644 --- a/hapi-fhir-test-utilities/src/test/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategyTest.java +++ b/hapi-fhir-test-utilities/src/test/java/ca/uhn/fhir/rest/server/tenant/UrlBaseTenantIdentificationStrategyTest.java @@ -11,12 +11,17 @@ import ca.uhn.fhir.util.UrlPathTokenizer; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; +import java.util.Collections; + import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.Mockito.lenient; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; @@ -69,6 +74,27 @@ public class UrlBaseTenantIdentificationStrategyTest { assertEquals(BASE_URL, actual); } + @CsvSource(value = { + " , , empty input url - empty URL should be returned", + "TENANT1/Patient/123 , TENANT1/Patient/123 , tenant ID already exists - input URL should be returned", + "TENANT1/Patient/$export, TENANT1/Patient/$export , tenant ID already exists - input URL should be returned", + "TENANT2/Patient/123 , TENANT2/Patient/123 , requestDetails contains different tenant ID - input URL should be returned", + "TENANT2/$export , TENANT2/$export , requestDetails contains different tenant ID - input URL should be returned", + "Patient/123 , TENANT1/Patient/123 , url starts with resource type - tenant ID should be added to URL", + "Patient/$export , TENANT1/Patient/$export , url starts with resource type - tenant ID should be added to URL", + "$export , TENANT1/$export , url starts with operation name - tenant ID should be added to URL", + }) + @ParameterizedTest + void resolveRelativeUrl_returnsCorrectlyResolvedUrl(String theInputUrl, String theExpectedResolvedUrl, String theMessage) { + lenient().when(myRequestDetails.getTenantId()).thenReturn("TENANT1"); + lenient().when(myFHIRContext.getResourceTypes()).thenReturn(Collections.singleton("Patient")); + lenient().when(myRequestDetails.getFhirContext()).thenReturn(myFHIRContext); + + String actual = ourTenantStrategy.resolveRelativeUrl(theInputUrl, myRequestDetails); + + assertEquals(theExpectedResolvedUrl, actual, theMessage); + } + @Test void extractTenant_givenNormalRequestAndExplicitTenant_shouldUseTenant() { //given a Patient request on MYTENANT diff --git a/pom.xml b/pom.xml index 41eed268323..d3ed41f8e6e 100644 --- a/pom.xml +++ b/pom.xml @@ -897,7 +897,7 @@ - 6.1.2 + 6.1.2.2 2.37.0 -Dfile.encoding=UTF-8 -Xmx2048m @@ -934,9 +934,9 @@ 2.3.1 2.3.0.1 3.0.0 - 4.8.0 + 4.9.0 3.0.3 - 10.0.14 + 10.0.17 3.0.2 5.9.1 0.64.8 @@ -983,7 +983,7 @@ 1.0.8 - 3.0.0-PRE8 + 3.0.0-PRE9 5.4.1 @@ -1171,7 +1171,7 @@ com.squareup.okio okio-jvm - 3.2.0 + 3.4.0 @@ -1385,7 +1385,7 @@ com.mysql mysql-connector-j - 8.0.32 + 8.2.0 org.springdoc @@ -1871,7 +1871,7 @@ --> org.elasticsearch.client elasticsearch-rest-high-level-client - 7.17.3 + 7.17.13 com.fasterxml.jackson.dataformat