honeymoose
/
hapi-fhir
mirror of https://github.com/hapifhir/hapi-fhir.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'remotes/origin/master' into im_2020601_lastn_code_text_filter

This commit is contained in:
ianmarshall 2020-06-08 09:26:55 -04:00
parent 56cbc5149b dd84846b94
commit 4982e67a0f
32 changed files with 79121 additions and 154 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_1_0/1899-improve-write-xact-perf-with-tags.yaml Normal file
View File

@ -0,0 +1,6 @@
---
type: perf
issue: 1899
title: "When submitting a transaction bundle containing a large number of resources being written, where the
resources had tags or profile definitions, a number of redundant database calls have been optimized out. This should
significantly improve performance for these scenarios."

150
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/docs/server_jpa_empi/empi_operations.md
View File

@ -2,6 +2,8 @@
Several operations exist that can be used to manage EMPI links. These operations are supplied by a [plain provider](/docs/server_plain/resource_providers.html#plain-providers) called [EmpiProvider](/hapi-fhir/apidocs/hapi-fhir-server-empi/ca/uhn/fhir/empi/provider/EmpiProviderR4.html).
In cases where the operation changes data, if a resource id parameter contains a version (e.g. `Person/123/_history/1`), then the operation will fail with a 409 CONFLICT if that is not the latest version of that resource. This could be used to prevent update conflicts in an environment where multiple users are working on the same set of empi links.
## Query links
Ue the `$empi-query-links` operation to view empi links. The results returned are based on the parameters provided. All parameters are optional. This operation takes the following parameters:
@ -74,27 +76,25 @@ The following request body could be used to find all POSSIBLE_MATCH links in the
This operation returns a `Parameters` resource that looks like the following:
```json
<Parameters xmlns="http://hl7.org/fhir">
<parameter>
<name value="link"/>
<part>
<name value="personId"/>
<valueString value="Person/123"/>
</part>
<part>
<name value="targetId"/>
<valueString value="Patient/456"/>
</part>
<part>
<name value="matchResult"/>
<valueString value="MATCH"/>
</part>
<part>
<name value="linkSource"/>
<valueString value="AUTO"/>
</part>
</parameter>
</Parameters>
{
"resourceType": "Parameters",
"parameter": [ {
"name": "link",
"part": [ {
"name": "personId",
"valueString": "Person/123"
}, {
"name": "targetId",
"valueString": "Patient/456"
}, {
"name": "matchResult",
"valueString": "POSSIBLE_MATCH"
}, {
"name": "linkSource",
"valueString": "AUTO"
} ]
} ]
}
```
## Querying links via the Person resource
@ -155,19 +155,93 @@ This operation returns `Parameters` similar to `$empi-query-links`:
```json
<Parameters xmlns="http://hl7.org/fhir">
<parameter>
<name value="link"/>
<part>
<name value="personId"/>
<valueString value="Person/123"/>
</part>
<part>
<name value="targetId"/>
<valueString value="Person/789"/>
</part>
</parameter>
</Parameters>
{
"resourceType": "Parameters",
"parameter": [ {
"name": "link",
"part": [ {
"name": "personId",
"valueString": "Person/123"
}, {
"name": "targetId",
"valueString": "Person/456"
}, {
"name": "matchResult",
"valueString": "POSSIBLE_DUPLICATE"
}, {
"name": "linkSource",
"valueString": "AUTO"
} ]
} ]
}
```
## Unduplicate Persons
Use the `$empi-not-duplicate` operation to mark duplicate persons as not duplicates. This operation takes the following parameters:
<table class="table table-striped table-condensed">
<thead>
<tr>
<th>Name</th>
<th>Type</th>
<th>Cardinality</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>personId</td>
<td>String</td>
<td>1..1</td>
<td>
The id of the Person resource.
</td>
</tr>
<tr>
<td>targetId</td>
<td>String</td>
<td>1..1</td>
<td>
The id of the Person that personId has a possible duplicate link to.
</td>
</tr>
</tbody>
</table>
### Example
Use an HTTP POST to the following URL to invoke this operation:
```url
http://example.com/$empi-not-duplicate
```
The following request body could be used:
```json
{
"resourceType": "Parameters",
"parameter": [ {
"name": "personId",
"valueString": "Person/123"
}, {
"name": "targetId",
"valueString": "Person/456"
} ]
}
```
When the operation is successful, it returns the following `Parameters`:
```json
{
"resourceType": "Parameters",
"parameter": [ {
"name": "success",
"valueBoolean": true
} ]
}
```
## Update Link
@ -243,7 +317,11 @@ The operation returns the updated `Person` resource. Note that this is the only
## Merge Persons
The `$empi-merge-persons` operation can be used to merge one Person resource with another. When doing this, you will need to decide which resource to merge from and which one to merge to. In most cases, fields will be merged (e.g. names, identifiers, and links will be the union of two). However when there is a conflict (e.g. birthday), fields in the toPerson will take precedence over fields in the fromPerson. This operation takes the following parameters:
The `$empi-merge-persons` operation can be used to merge one Person resource with another. When doing this, you will need to decide which resource to merge from and which one to merge to. In most cases, fields will be merged (e.g. names, identifiers, and links will be the union of two). However when there is a conflict (e.g. birthday), fields in the toPerson will take precedence over fields in the fromPerson
After the merge is complete, `fromPerson.active` is set to `false`. Also, a new link with assurance level 4 (MANUAL MATCH) will be added pointing from the fromPerson to the toPerson.
This operation takes the following parameters:
<table class="table table-striped table-condensed">
<thead>
@ -260,7 +338,7 @@ The `$empi-merge-persons` operation can be used to merge one Person resource wit
<td>String</td>
<td>1..1</td>
<td>
The id of the Person resource to merge data from. "active" will be set to "false" on this resource after the merge.
The id of the Person resource to merge data from.
</td>
</tr>
<tr>

28
hapi-fhir-jpaserver-api/src/main/java/ca/uhn/fhir/jpa/api/config/DaoConfig.java
View File

@ -193,6 +193,34 @@ public class DaoConfig {
*/
private boolean myDeleteEnabled = true;
/**
* If set to <code>true</code> (default is <code>false</code>) the <code>$lastn</code> operation will be enabled for
* indexing Observation resources. This operation involves creating a special set of tables in ElasticSearch for
* discovering Observation resources. Enabling this setting increases the amount of storage space required, and can
* slow write operations, but can be very useful for searching for collections of Observations for some applications.
*
* @since 5.1.0
*/
public boolean isLastNEnabled() {
return myLastNEnabled;
}
/**
* If set to <code>true</code> (default is <code>false</code>) the <code>$lastn</code> operation will be enabled for
* indexing Observation resources. This operation involves creating a special set of tables in ElasticSearch for
* discovering Observation resources. Enabling this setting increases the amount of storage space required, and can
* slow write operations, but can be very useful for searching for collections of Observations for some applications.
*
* @since 5.1.0
*/
public void setLastNEnabled(boolean theLastNEnabled) {
myLastNEnabled = theLastNEnabled;
}
/**
* @since 5.1.0
*/
private boolean myLastNEnabled = false;
/**
* Constructor

2
hapi-fhir-jpaserver-api/src/main/java/ca/uhn/fhir/jpa/api/dao/IFhirResourceDao.java
View File

@ -93,8 +93,6 @@ public interface IFhirResourceDao<T extends IBaseResource> extends IDao {
/**
* This method does not throw an exception if there are delete conflicts, but populates them
* in the provided list
*
* @param theRequestDetails TODO
*/
DaoMethodOutcome delete(IIdType theResource, DeleteConflictList theDeleteConflictsListToPopulate, RequestDetails theRequestDetails, TransactionDetails theTransactionDetails);

6
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/config/BaseConfig.java
View File

@ -47,6 +47,7 @@ import ca.uhn.fhir.jpa.search.reindex.IResourceReindexingSvc;
import ca.uhn.fhir.jpa.search.reindex.ResourceReindexingSvcImpl;
import ca.uhn.fhir.jpa.searchparam.config.SearchParamConfig;
import ca.uhn.fhir.jpa.searchparam.extractor.IResourceLinkResolver;
import ca.uhn.fhir.jpa.util.MemoryCacheService;
import ca.uhn.fhir.rest.api.server.RequestDetails;
import ca.uhn.fhir.rest.server.interceptor.consent.IConsentContextServices;
import ca.uhn.fhir.rest.server.interceptor.partition.RequestTenantPartitionInterceptor;
@ -182,6 +183,11 @@ public abstract class BaseConfig {
return new BinaryStorageInterceptor();
}
@Bean
public MemoryCacheService memoryCacheService() {
return new MemoryCacheService();
}
@Bean
@Primary
public IResourceLinkResolver daoResourceLinkResolver() {

14
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/BaseHapiFhirDao.java
View File

@ -47,7 +47,6 @@ import ca.uhn.fhir.jpa.model.entity.TagTypeEnum;
import ca.uhn.fhir.jpa.model.search.SearchStatusEnum;
import ca.uhn.fhir.jpa.model.search.StorageProcessingMessage;
import ca.uhn.fhir.jpa.model.util.JpaConstants;
import ca.uhn.fhir.rest.api.server.storage.TransactionDetails;
import ca.uhn.fhir.jpa.partition.IPartitionLookupSvc;
import ca.uhn.fhir.jpa.partition.RequestPartitionHelperSvc;
import ca.uhn.fhir.jpa.search.PersistedJpaBundleProviderFactory;
@ -59,6 +58,7 @@ import ca.uhn.fhir.jpa.sp.ISearchParamPresenceSvc;
import ca.uhn.fhir.jpa.term.api.ITermReadSvc;
import ca.uhn.fhir.jpa.util.AddRemoveCount;
import ca.uhn.fhir.jpa.util.JpaInterceptorBroadcaster;
import ca.uhn.fhir.jpa.util.MemoryCacheService;
import ca.uhn.fhir.model.api.IResource;
import ca.uhn.fhir.model.api.ResourceMetadataKeyEnum;
import ca.uhn.fhir.model.api.Tag;
@ -76,6 +76,7 @@ import ca.uhn.fhir.rest.api.RestOperationTypeEnum;
import ca.uhn.fhir.rest.api.RestSearchParameterTypeEnum;
import ca.uhn.fhir.rest.api.server.IBundleProvider;
import ca.uhn.fhir.rest.api.server.RequestDetails;
import ca.uhn.fhir.rest.api.server.storage.TransactionDetails;
import ca.uhn.fhir.rest.server.exceptions.InternalErrorException;
import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException;
import ca.uhn.fhir.rest.server.exceptions.UnprocessableEntityException;
@ -91,6 +92,7 @@ import com.google.common.hash.HashFunction;
import com.google.common.hash.Hashing;
import org.apache.commons.lang3.NotImplementedException;
import org.apache.commons.lang3.Validate;
import org.apache.commons.lang3.tuple.Pair;
import org.hl7.fhir.instance.model.api.IAnyResource;
import org.hl7.fhir.instance.model.api.IBase;
import org.hl7.fhir.instance.model.api.IBaseCoding;
@ -233,6 +235,8 @@ public abstract class BaseHapiFhirDao<T extends IBaseResource> extends BaseStora
private PersistedJpaBundleProviderFactory myPersistedJpaBundleProviderFactory;
@Autowired
private IPartitionLookupSvc myPartitionLookupSvc;
@Autowired
private MemoryCacheService myMemoryCacheService;
@Override
protected IInterceptorBroadcaster getInterceptorBroadcaster() {
@ -393,11 +397,17 @@ public abstract class BaseHapiFhirDao<T extends IBaseResource> extends BaseStora
}
}
/**
* <code>null</code> will only be returned if the scheme and tag are both blank
*/
protected TagDefinition getTagOrNull(TagTypeEnum theTagType, String theScheme, String theTerm, String theLabel) {
if (isBlank(theScheme) && isBlank(theTerm) && isBlank(theLabel)) {
return null;
}
Pair<String, String> key = Pair.of(theScheme, theTerm);
return myMemoryCacheService.get(MemoryCacheService.CacheEnum.TAG_DEFINITION, key, k -> {
CriteriaBuilder builder = myEntityManager.getCriteriaBuilder();
CriteriaQuery<TagDefinition> cq = builder.createQuery(TagDefinition.class);
Root<TagDefinition> from = cq.from(TagDefinition.class);
@ -424,6 +434,8 @@ public abstract class BaseHapiFhirDao<T extends IBaseResource> extends BaseStora
myEntityManager.persist(retVal);
return retVal;
}
});
}
protected IBundleProvider history(RequestDetails theRequest, String theResourceType, Long theResourcePid, Date theRangeStartInclusive, Date theRangeEndInclusive) {

2
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/BaseHapiFhirResourceDaoObservation.java
View File

@ -54,6 +54,7 @@ public abstract class BaseHapiFhirResourceDaoObservation<T extends IBaseResource
ResourceTable retVal = super.updateEntity(theRequest, theResource, theEntity, theDeletedTimestampOrNull, thePerformIndexing, theUpdateVersion,
theTransactionDetails, theForceUpdate, theCreateNewHistoryEntry);
if (myDaoConfig.isLastNEnabled()) {
if (!retVal.isUnchangedInCurrentOperation()) {
if (retVal.getDeleted() == null) {
// Update indexes here for LastN operation.
@ -62,6 +63,7 @@ public abstract class BaseHapiFhirResourceDaoObservation<T extends IBaseResource
myObservationLastNIndexPersistSvc.deleteObservationIndex(theEntity);
}
}
}
return retVal;
}

28
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/expunge/ResourceExpungeService.java
View File

@ -23,6 +23,7 @@ package ca.uhn.fhir.jpa.dao.expunge;
import ca.uhn.fhir.interceptor.api.HookParams;
import ca.uhn.fhir.interceptor.api.IInterceptorBroadcaster;
import ca.uhn.fhir.interceptor.api.Pointcut;
import ca.uhn.fhir.jpa.api.config.DaoConfig;
import ca.uhn.fhir.jpa.api.dao.DaoRegistry;
import ca.uhn.fhir.jpa.api.dao.IFhirResourceDao;
import ca.uhn.fhir.jpa.dao.data.IResourceHistoryTableDao;
@ -105,6 +106,8 @@ class ResourceExpungeService implements IResourceExpungeService {
private IResourceProvenanceDao myResourceHistoryProvenanceTableDao;
@Autowired
private ISearchParamPresentDao mySearchParamPresentDao;
@Autowired
private DaoConfig myDaoConfig;
@Override
@Transactional
@ -238,20 +241,43 @@ class ResourceExpungeService implements IResourceExpungeService {
@Override
@Transactional
public void deleteAllSearchParams(Long theResourceId) {
ResourceTable resource = myResourceTableDao.findById(theResourceId).orElse(null);
if (resource == null || resource.isParamsUriPopulated()) {
myResourceIndexedSearchParamUriDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsCoordsPopulated()) {
myResourceIndexedSearchParamCoordsDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsDatePopulated()) {
myResourceIndexedSearchParamDateDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsNumberPopulated()) {
myResourceIndexedSearchParamNumberDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsQuantityPopulated()) {
myResourceIndexedSearchParamQuantityDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsStringPopulated()) {
myResourceIndexedSearchParamStringDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsTokenPopulated()) {
myResourceIndexedSearchParamTokenDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isParamsCompositeStringUniquePresent()) {
myResourceIndexedCompositeStringUniqueDao.deleteByResourceId(theResourceId);
}
if (myDaoConfig.getIndexMissingFields() == DaoConfig.IndexEnabledEnum.ENABLED) {
mySearchParamPresentDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isHasLinks()) {
myResourceLinkDao.deleteByResourceId(theResourceId);
}
if (resource == null || resource.isHasTags()) {
myResourceTagDao.deleteByResourceId(theResourceId);
}
}
private void expungeHistoricalVersionsOfId(RequestDetails theRequestDetails, Long myResourceId, AtomicInteger theRemainingCount) {
ResourceTable resource = myResourceTableDao.findById(myResourceId).orElseThrow(IllegalArgumentException::new);

51
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/dao/index/IdHelperService.java
View File

@ -30,6 +30,7 @@ import ca.uhn.fhir.jpa.model.cross.IResourceLookup;
import ca.uhn.fhir.jpa.model.cross.ResourceLookup;
import ca.uhn.fhir.jpa.model.entity.ForcedId;
import ca.uhn.fhir.jpa.model.entity.ResourceTable;
import ca.uhn.fhir.jpa.util.MemoryCacheService;
import ca.uhn.fhir.jpa.util.QueryChunker;
import ca.uhn.fhir.model.primitive.IdDt;
import ca.uhn.fhir.rest.api.server.RequestDetails;
@ -105,17 +106,6 @@ public class IdHelperService {
@Autowired
private FhirContext myFhirCtx;
private Cache<String, Long> myPersistentIdCache;
private Cache<String, IResourceLookup> myResourceLookupCache;
private Cache<Long, Optional<String>> myForcedIdCache;
@PostConstruct
public void start() {
myPersistentIdCache = newCache();
myResourceLookupCache = newCache();
myForcedIdCache = newCache();
}
public void delete(ForcedId forcedId) {
myForcedIdDao.deleteByPid(forcedId.getId());
}
@ -138,6 +128,9 @@ public class IdHelperService {
return matches.iterator().next();
}
@Autowired
private MemoryCacheService myMemoryCacheService;
/**
* Given a resource type and ID, determines the internal persistent ID for the resource.
*
@ -151,7 +144,7 @@ public class IdHelperService {
retVal = resolveResourceIdentity(theRequestPartitionId, theResourceType, theId);
} else {
String key = RequestPartitionId.stringifyForKey(theRequestPartitionId) + "/" + theResourceType + "/" + theId;
retVal = myPersistentIdCache.get(key, t -> resolveResourceIdentity(theRequestPartitionId, theResourceType, theId));
retVal = myMemoryCacheService.get(MemoryCacheService.CacheEnum.PERSISTENT_ID, key, t -> resolveResourceIdentity(theRequestPartitionId, theResourceType, theId));
}
} else {
@ -201,7 +194,7 @@ public class IdHelperService {
for (Iterator<String> idIterator = nextIds.iterator(); idIterator.hasNext(); ) {
String nextId = idIterator.next();
String key = RequestPartitionId.stringifyForKey(theRequestPartitionId) + "/" + nextResourceType + "/" + nextId;
Long nextCachedPid = myPersistentIdCache.getIfPresent(key);
Long nextCachedPid = myMemoryCacheService.getIfPresent(MemoryCacheService.CacheEnum.PERSISTENT_ID, key);
if (nextCachedPid != null) {
idIterator.remove();
retVal.add(new ResourcePersistentId(nextCachedPid));
@ -226,7 +219,7 @@ public class IdHelperService {
retVal.add(new ResourcePersistentId(pid));
String key = RequestPartitionId.stringifyForKey(theRequestPartitionId) + "/" + nextResourceType + "/" + forcedId;
myPersistentIdCache.put(key, pid);
myMemoryCacheService.put(MemoryCacheService.CacheEnum.PERSISTENT_ID, key, pid);
}
}
@ -255,7 +248,7 @@ public class IdHelperService {
public Optional<String> translatePidIdToForcedId(ResourcePersistentId theId) {
return myForcedIdCache.get(theId.getIdAsLong(), pid -> myForcedIdDao.findByResourcePid(pid).map(t -> t.getForcedId()));
return myMemoryCacheService.get(MemoryCacheService.CacheEnum.FORCED_ID, theId.getIdAsLong(), pid -> myForcedIdDao.findByResourcePid(pid).map(t -> t.getForcedId()));
}
private ListMultimap<String, String> organizeIdsByResourceType(Collection<IIdType> theIds) {
@ -329,7 +322,7 @@ public class IdHelperService {
for (Iterator<String> forcedIdIterator = nextIds.iterator(); forcedIdIterator.hasNext(); ) {
String nextForcedId = forcedIdIterator.next();
String nextKey = nextResourceType + "/" + nextForcedId;
IResourceLookup cachedLookup = myResourceLookupCache.getIfPresent(nextKey);
IResourceLookup cachedLookup = myMemoryCacheService.getIfPresent(MemoryCacheService.CacheEnum.RESOURCE_LOOKUP, nextKey);
if (cachedLookup != null) {
forcedIdIterator.remove();
retVal.add(cachedLookup);
@ -361,7 +354,7 @@ public class IdHelperService {
if (!myDaoConfig.isDeleteEnabled()) {
String key = resourceType + "/" + forcedId;
myResourceLookupCache.put(key, lookup);
myMemoryCacheService.put(MemoryCacheService.CacheEnum.RESOURCE_LOOKUP, key, lookup);
}
}
}
@ -377,7 +370,7 @@ public class IdHelperService {
for (Iterator<Long> forcedIdIterator = thePidsToResolve.iterator(); forcedIdIterator.hasNext(); ) {
Long nextPid = forcedIdIterator.next();
String nextKey = Long.toString(nextPid);
IResourceLookup cachedLookup = myResourceLookupCache.getIfPresent(nextKey);
IResourceLookup cachedLookup = myMemoryCacheService.getIfPresent(MemoryCacheService.CacheEnum.RESOURCE_LOOKUP, nextKey);
if (cachedLookup != null) {
forcedIdIterator.remove();
theTarget.add(cachedLookup);
@ -403,30 +396,16 @@ public class IdHelperService {
theTarget.add(t);
if (!myDaoConfig.isDeleteEnabled()) {
String nextKey = Long.toString(t.getResourceId());
myResourceLookupCache.put(nextKey, t);
myMemoryCacheService.put(MemoryCacheService.CacheEnum.RESOURCE_LOOKUP, nextKey, t);
}
});
}
}
public void clearCache() {
myPersistentIdCache.invalidateAll();
myResourceLookupCache.invalidateAll();
myForcedIdCache.invalidateAll();
}
private <T, V> @NonNull Cache<T, V> newCache() {
return Caffeine
.newBuilder()
.maximumSize(10000)
.expireAfterWrite(10, TimeUnit.MINUTES)
.build();
}
public Map<Long, Optional<String>> translatePidsToForcedIds(Set<Long> thePids) {
Map<Long, Optional<String>> retVal = new HashMap<>(myForcedIdCache.getAllPresent(thePids));
Map<Long, Optional<String>> retVal = new HashMap<>(myMemoryCacheService.getAllPresent(MemoryCacheService.CacheEnum.FORCED_ID, thePids));
List<Long> remainingPids = thePids
.stream()
@ -440,7 +419,7 @@ public class IdHelperService {
Long nextResourcePid = forcedId.getResourceId();
Optional<String> nextForcedId = Optional.of(forcedId.getForcedId());
retVal.put(nextResourcePid, nextForcedId);
myForcedIdCache.put(nextResourcePid, nextForcedId);
myMemoryCacheService.put(MemoryCacheService.CacheEnum.FORCED_ID, nextResourcePid, nextForcedId);
}
});
@ -450,7 +429,7 @@ public class IdHelperService {
.collect(Collectors.toList());
for (Long nextResourcePid : remainingPids) {
retVal.put(nextResourcePid, Optional.empty());
myForcedIdCache.put(nextResourcePid, Optional.empty());
myMemoryCacheService.put(MemoryCacheService.CacheEnum.FORCED_ID, nextResourcePid, Optional.empty());
}
return retVal;

5
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/entity/EmpiLink.java
View File

@ -25,7 +25,6 @@ import ca.uhn.fhir.empi.api.EmpiMatchResultEnum;
import ca.uhn.fhir.jpa.model.entity.ResourceTable;
import org.apache.commons.lang3.builder.ToStringBuilder;
import org.apache.commons.lang3.builder.ToStringStyle;
import org.hibernate.annotations.OptimisticLock;
import javax.persistence.Column;
import javax.persistence.Entity;
@ -157,6 +156,10 @@ public class EmpiLink {
return myMatchResult == EmpiMatchResultEnum.POSSIBLE_MATCH;
}
public boolean isPossibleDuplicate() {
return myMatchResult == EmpiMatchResultEnum.POSSIBLE_DUPLICATE;
}
public EmpiLinkSourceEnum getLinkSource() {
return myLinkSource;
}

4
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/interceptor/CascadingDeleteInterceptor.java
View File

@ -117,6 +117,7 @@ public class CascadingDeleteInterceptor {
String nextSourceId = nextSource.toUnqualifiedVersionless().getValue();
if (!cascadedDeletes.contains(nextSourceId)) {
cascadedDeletes.add(nextSourceId);
IFhirResourceDao dao = myDaoRegistry.getResourceDao(nextSource.getResourceType());
@ -132,9 +133,6 @@ public class CascadingDeleteInterceptor {
// Actually perform the delete
ourLog.info("Have delete conflict {} - Cascading delete", next);
dao.delete(nextSource, theConflictList, theRequest, theTransactionDetails);
cascadedDeletes.add(nextSourceId);
}
}

92
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/util/MemoryCacheService.java Normal file
View File

@ -0,0 +1,92 @@
package ca.uhn.fhir.jpa.util;
/*-
* #%L
* HAPI FHIR JPA Server
* %%
* Copyright (C) 2014 - 2020 University Health Network
* %%
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
* #L%
*/
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import javax.annotation.PostConstruct;
import java.util.EnumMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
/**
* This class acts as a central spot for all of the many Caffeine caches we use in HAPI FHIR.
* <p>
* The API is super simplistic, and caches are all 1-minute, max 10000 entries for starters. We could definitely add nuance to this,
* which will be much easier now that this is being centralized. Some logging/monitoring would be good too.
*/
public class MemoryCacheService {
private EnumMap<CacheEnum, Cache<?, ?>> myCaches;
@PostConstruct
public void start() {
myCaches = new EnumMap<>(CacheEnum.class);
for (CacheEnum next : CacheEnum.values()) {
Cache<Object, Object> nextCache = Caffeine.newBuilder().expireAfterWrite(1, TimeUnit.MINUTES).maximumSize(10000).build();
myCaches.put(next, nextCache);
}
}
public <K, T> T get(CacheEnum theCache, K theKey, Function<K, T> theSupplier) {
Cache<K, T> cache = getCache(theCache);
return cache.get(theKey, theSupplier);
}
public <K, V> V getIfPresent(CacheEnum theCache, K theKey) {
return (V) getCache(theCache).getIfPresent(theKey);
}
public <K, V> void put(CacheEnum theCache, K theKey, V theValue) {
getCache(theCache).put(theKey, theValue);
}
public <K, V> Map<K, V> getAllPresent(CacheEnum theCache, Iterable<K> theKeys) {
return (Map<K, V>) getCache(theCache).getAllPresent(theKeys);
}
public void invalidateAllCaches() {
myCaches.values().forEach(t -> t.invalidateAll());
}
private <K, T> Cache<K, T> getCache(CacheEnum theCache) {
return (Cache<K, T>) myCaches.get(theCache);
}
public enum CacheEnum {
TAG_DEFINITION,
PERSISTENT_ID,
RESOURCE_LOOKUP,
FORCED_ID,
}
}

8
hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/dao/BaseJpaTest.java
View File

@ -21,6 +21,7 @@ import ca.uhn.fhir.jpa.search.cache.ISearchResultCacheSvc;
import ca.uhn.fhir.jpa.search.reindex.IResourceReindexingSvc;
import ca.uhn.fhir.jpa.searchparam.registry.ISearchParamRegistry;
import ca.uhn.fhir.jpa.util.CircularQueueCaptureQueriesListener;
import ca.uhn.fhir.jpa.util.MemoryCacheService;
import ca.uhn.fhir.model.dstu2.resource.Bundle;
import ca.uhn.fhir.model.dstu2.resource.Bundle.Entry;
import ca.uhn.fhir.rest.api.Constants;
@ -122,6 +123,8 @@ public abstract class BaseJpaTest extends BaseTest {
protected IPartitionLookupSvc myPartitionConfigSvc;
@Autowired
private IdHelperService myIdHelperService;
@Autowired
private MemoryCacheService myMemoryCacheService;
@After
public void afterPerformCleanup() {
@ -132,10 +135,9 @@ public abstract class BaseJpaTest extends BaseTest {
if (myPartitionConfigSvc != null) {
myPartitionConfigSvc.clearCaches();
}
if (myIdHelperService != null) {
myIdHelperService.clearCache();
if (myMemoryCacheService != null) {
myMemoryCacheService.invalidateAllCaches();
}
}
@After

57
hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/dao/dstu3/FhirSystemDaoDstu3Test.java
View File

@ -1,9 +1,13 @@
package ca.uhn.fhir.jpa.dao.dstu3;
import ca.uhn.fhir.interceptor.api.IInterceptorBroadcaster;
import ca.uhn.fhir.interceptor.api.IInterceptorService;
import ca.uhn.fhir.jpa.api.config.DaoConfig;
import ca.uhn.fhir.jpa.api.dao.DaoRegistry;
import ca.uhn.fhir.jpa.dao.BaseHapiFhirDao;
import ca.uhn.fhir.jpa.dao.GZipUtil;
import ca.uhn.fhir.jpa.dao.r4.FhirSystemDaoR4;
import ca.uhn.fhir.jpa.interceptor.CascadingDeleteInterceptor;
import ca.uhn.fhir.jpa.model.entity.ResourceTag;
import ca.uhn.fhir.jpa.model.entity.TagTypeEnum;
import ca.uhn.fhir.jpa.provider.SystemProviderDstu2Test;
@ -21,8 +25,10 @@ import ca.uhn.fhir.rest.server.exceptions.ResourceGoneException;
import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException;
import ca.uhn.fhir.rest.server.exceptions.ResourceVersionConflictException;
import ca.uhn.fhir.rest.server.exceptions.UnprocessableEntityException;
import ca.uhn.fhir.util.StopWatch;
import ca.uhn.fhir.util.TestUtil;
import com.google.common.base.Charsets;
import com.google.common.collect.Maps;
import org.apache.commons.io.IOUtils;
import org.hl7.fhir.dstu3.model.Appointment;
import org.hl7.fhir.dstu3.model.Attachment;
@ -60,6 +66,8 @@ import org.junit.AfterClass;
import org.junit.Before;
import org.junit.Ignore;
import org.junit.Test;
import org.mockito.internal.stubbing.answers.CallsRealMethods;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.transaction.TransactionStatus;
import org.springframework.transaction.support.TransactionCallbackWithoutResult;
import org.springframework.transaction.support.TransactionTemplate;
@ -68,8 +76,11 @@ import java.io.IOException;
import java.io.InputStream;
import java.math.BigDecimal;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
@ -90,15 +101,25 @@ import static org.junit.Assert.assertNull;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.Mockito.doAnswer;
import static org.mockito.Mockito.when;
public class FhirSystemDaoDstu3Test extends BaseJpaDstu3SystemTest {
private static final org.slf4j.Logger ourLog = org.slf4j.LoggerFactory.getLogger(FhirSystemDaoDstu3Test.class);
@Autowired
private DaoRegistry myDaoRegistry;
@Autowired
private IInterceptorService myInterceptorBroadcaster;
@After
public void after() {
myDaoConfig.setAllowInlineMatchUrlReferences(false);
myDaoConfig.setAllowMultipleDelete(new DaoConfig().isAllowMultipleDelete());
myDaoConfig.setIndexMissingFields(new DaoConfig().getIndexMissingFields());
myDaoConfig.setMaximumDeleteConflictQueryCount(new DaoConfig().getMaximumDeleteConflictQueryCount());
}
@Before
@ -1826,6 +1847,42 @@ public class FhirSystemDaoDstu3Test extends BaseJpaDstu3SystemTest {
assertEquals("201 Created", resp.getEntry().get(1).getResponse().getStatus());
}
/**
* There is nothing here that isn't tested elsewhere, but it's useful for testing a large transaction followed
* by a large cascading delete
*/
@Test
@Ignore
public void testTransactionFromBundle_Slow() throws Exception {
myDaoConfig.setIndexMissingFields(DaoConfig.IndexEnabledEnum.DISABLED);
myDaoConfig.setMaximumDeleteConflictQueryCount(10000);
StopWatch sw = new StopWatch();
sw.startTask("Parse Bundle");
Bundle bundle = loadBundle("/dstu3/slow_bundle.xml");
sw.startTask("Process transaction");
Bundle resp = mySystemDao.transaction(mySrd, bundle);
ourLog.info("Tasks: {}", sw.formatTaskDurations());
assertEquals("201 Created", resp.getEntry().get(0).getResponse().getStatus());
doAnswer(new CallsRealMethods()).when(mySrd).setParameters(any());
when(mySrd.getParameters()).thenCallRealMethod();
when(mySrd.getUserData()).thenReturn(new HashMap<>());
Map<String, String[]> params = Maps.newHashMap();
params.put(Constants.PARAMETER_CASCADE_DELETE, new String[]{Constants.CASCADE_DELETE});
mySrd.setParameters(params);
CascadingDeleteInterceptor deleteInterceptor = new CascadingDeleteInterceptor(myFhirCtx, myDaoRegistry, myInterceptorBroadcaster);
myInterceptorBroadcaster.registerInterceptor(deleteInterceptor);
myPatientDao.deleteByUrl("Patient?identifier=http://fhir.nl/fhir/NamingSystem/bsn|900197341", mySrd);
}
@Test
public void testTransactionOrdering() {
String methodName = "testTransactionOrdering";

11
hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/dao/r4/BaseR4SearchLastN.java
View File

@ -25,6 +25,7 @@ import org.hl7.fhir.r4.model.DateTimeType;
import org.hl7.fhir.r4.model.Observation;
import org.hl7.fhir.r4.model.Patient;
import org.hl7.fhir.r4.model.StringType;
import org.junit.After;
import org.junit.AfterClass;
import org.junit.Before;
import org.junit.Test;
@ -82,6 +83,16 @@ public class BaseR4SearchLastN extends BaseJpaTest {
return myPlatformTransactionManager;
}
@Before
public void beforeEnableLastN() {
myDaoConfig.setLastNEnabled(true);
}
@After
public void afterDisableLastN() {
myDaoConfig.setLastNEnabled(new DaoConfig().isLastNEnabled());
}
protected final String observationCd0 = "code0";
protected final String observationCd1 = "code1";
protected final String observationCd2 = "code2";

43
hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/dao/r4/FhirResourceDaoR4QueryCountTest.java
View File

@ -1055,6 +1055,49 @@ public class FhirResourceDaoR4QueryCountTest extends BaseJpaR4Test {
}
@Test
public void testTransactionWithMultipleProfiles() {
myDaoConfig.setDeleteEnabled(true);
myDaoConfig.setIndexMissingFields(DaoConfig.IndexEnabledEnum.DISABLED);
// Create transaction
Bundle input = new Bundle();
for (int i = 0; i < 5; i++) {
Patient patient = new Patient();
patient.getMeta().addProfile("http://example.com/profile");
patient.getMeta().addTag().setSystem("http://example.com/tags").setCode("tag-1");
patient.getMeta().addTag().setSystem("http://example.com/tags").setCode("tag-2");
input.addEntry()
.setResource(patient)
.getRequest()
.setMethod(Bundle.HTTPVerb.POST)
.setUrl("Patient");
}
myCaptureQueriesListener.clear();
mySystemDao.transaction(mySrd, input);
myCaptureQueriesListener.logSelectQueriesForCurrentThread();
assertEquals(3, myCaptureQueriesListener.countSelectQueriesForCurrentThread());
assertEquals(8, myCaptureQueriesListener.countInsertQueriesForCurrentThread());
myCaptureQueriesListener.logUpdateQueriesForCurrentThread();
assertEquals(1, myCaptureQueriesListener.countUpdateQueriesForCurrentThread());
assertEquals(0, myCaptureQueriesListener.countDeleteQueriesForCurrentThread());
// Do the same a second time
myCaptureQueriesListener.clear();
mySystemDao.transaction(mySrd, input);
myCaptureQueriesListener.logSelectQueriesForCurrentThread();
assertEquals(0, myCaptureQueriesListener.countSelectQueriesForCurrentThread());
assertEquals(5, myCaptureQueriesListener.countInsertQueriesForCurrentThread());
assertEquals(1, myCaptureQueriesListener.countUpdateQueriesForCurrentThread());
assertEquals(0, myCaptureQueriesListener.countDeleteQueriesForCurrentThread());
}
@AfterClass
public static void afterClassClearContext() {
TestUtil.clearAllStaticFieldsForUnitTest();

17
hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/dao/r4/PersistObservationIndexedSearchParamLastNR4IT.java
View File

@ -1,6 +1,7 @@
package ca.uhn.fhir.jpa.dao.r4;
import ca.uhn.fhir.context.FhirContext;
import ca.uhn.fhir.jpa.api.config.DaoConfig;
import ca.uhn.fhir.jpa.api.dao.IFhirSystemDao;
import ca.uhn.fhir.jpa.config.TestR4ConfigWithElasticsearchClient;
import ca.uhn.fhir.jpa.dao.ObservationLastNIndexPersistSvc;
@ -15,6 +16,7 @@ import ca.uhn.fhir.rest.param.*;
import com.google.common.base.Charsets;
import org.apache.commons.io.IOUtils;
import org.hl7.fhir.r4.model.*;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
@ -56,6 +58,9 @@ public class PersistObservationIndexedSearchParamLastNR4IT {
@Autowired
protected FhirContext myFhirCtx;
@Autowired
private DaoConfig myDaoConfig;
@Before
public void before() throws IOException {
@ -63,8 +68,20 @@ public class PersistObservationIndexedSearchParamLastNR4IT {
elasticsearchSvc.deleteAllDocumentsForTest(ElasticsearchSvcImpl.OBSERVATION_CODE_INDEX);
elasticsearchSvc.refreshIndex(ElasticsearchSvcImpl.OBSERVATION_INDEX);
elasticsearchSvc.refreshIndex(ElasticsearchSvcImpl.OBSERVATION_CODE_INDEX);
}
@Before
public void beforeEnableLastN() {
myDaoConfig.setLastNEnabled(true);
}
@After
public void afterDisableLastN() {
myDaoConfig.setLastNEnabled(new DaoConfig().isLastNEnabled());
}
private final String SINGLE_SUBJECT_ID = "4567";
private final String SINGLE_OBSERVATION_PID = "123";
private final Date SINGLE_EFFECTIVEDTM = new Date();

78383
hapi-fhir-jpaserver-base/src/test/resources/dstu3/slow_bundle.xml Normal file
View File

File diff suppressed because one or more lines are too long

23
hapi-fhir-jpaserver-empi/src/main/java/ca/uhn/fhir/jpa/empi/svc/EmpiLinkSvcImpl.java
View File

@ -63,13 +63,19 @@ public class EmpiLinkSvcImpl implements IEmpiLinkSvc {
@Override
@Transactional
public void updateLink(IAnyResource thePerson, IAnyResource theResource, EmpiMatchResultEnum theMatchResult, EmpiLinkSourceEnum theLinkSource, EmpiTransactionContext theEmpiTransactionContext) {
IIdType resourceId = theResource.getIdElement().toUnqualifiedVersionless();
public void updateLink(IAnyResource thePerson, IAnyResource theTarget, EmpiMatchResultEnum theMatchResult, EmpiLinkSourceEnum theLinkSource, EmpiTransactionContext theEmpiTransactionContext) {
IIdType resourceId = theTarget.getIdElement().toUnqualifiedVersionless();
validateRequestIsLegal(thePerson, theResource, theMatchResult, theLinkSource);
if (theMatchResult == EmpiMatchResultEnum.POSSIBLE_DUPLICATE && personsLinkedAsNoMatch(thePerson, theTarget)) {
log(theEmpiTransactionContext, thePerson.getIdElement().toUnqualifiedVersionless() +
" is linked as NO_MATCH with " +
theTarget.getIdElement().toUnqualifiedVersionless() +
" not linking as POSSIBLE_DUPLICATE.");
return;
}
validateRequestIsLegal(thePerson, theTarget, theMatchResult, theLinkSource);
switch (theMatchResult) {
case MATCH:
//deleteCurrentMatch(theResource);
myPersonHelper.addOrUpdateLink(thePerson, resourceId, AssuranceLevelUtil.getAssuranceLevel(theMatchResult, theLinkSource), theEmpiTransactionContext);
myEmpiResourceDaoSvc.updatePerson(thePerson);
break;
@ -83,8 +89,15 @@ public class EmpiLinkSvcImpl implements IEmpiLinkSvc {
break;
}
myEmpiResourceDaoSvc.updatePerson(thePerson);
createOrUpdateLinkEntity(thePerson, theResource, theMatchResult, theLinkSource, theEmpiTransactionContext);
createOrUpdateLinkEntity(thePerson, theTarget, theMatchResult, theLinkSource, theEmpiTransactionContext);
}
private boolean personsLinkedAsNoMatch(IAnyResource thePerson, IAnyResource theTarget) {
Long personId = myIdHelperService.getPidOrThrowException(thePerson);
Long targetId = myIdHelperService.getPidOrThrowException(theTarget);
// TODO perf collapse into one query
return myEmpiLinkDaoSvc.getEmpiLinksByPersonPidTargetPidAndMatchResult(personId, targetId, EmpiMatchResultEnum.NO_MATCH).isPresent() ||
myEmpiLinkDaoSvc.getEmpiLinksByPersonPidTargetPidAndMatchResult(targetId, personId, EmpiMatchResultEnum.NO_MATCH).isPresent();
}
@Override

50
hapi-fhir-jpaserver-empi/src/main/java/ca/uhn/fhir/jpa/empi/svc/EmpiLinkUpdaterSvcImpl.java
View File

@ -33,7 +33,11 @@ import ca.uhn.fhir.jpa.dao.EmpiLinkDaoSvc;
import ca.uhn.fhir.jpa.dao.index.IdHelperService;
import ca.uhn.fhir.jpa.entity.EmpiLink;
import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException;
import ca.uhn.fhir.rest.server.provider.ProviderConstants;
import ca.uhn.fhir.util.ParametersUtil;
import org.hl7.fhir.instance.model.api.IAnyResource;
import org.hl7.fhir.instance.model.api.IBaseParameters;
import org.hl7.fhir.r4.model.Parameters;
import org.slf4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.transaction.annotation.Transactional;
@ -97,10 +101,10 @@ public class EmpiLinkUpdaterSvcImpl implements IEmpiLinkUpdaterSvc {
}
if (!"Person".equals(personType)) {
throw new InvalidRequestException("First argument to updateLink must be a Person. Was " + personType);
throw new InvalidRequestException("First argument to " + ProviderConstants.EMPI_UPDATE_LINK + " must be a Person. Was " + personType);
}
if (!EmpiUtil.supportedTargetType(theTargetType)) {
throw new InvalidRequestException("Second argument to updateLink must be a Patient or Practitioner. Was " + theTargetType);
throw new InvalidRequestException("Second argument to " + ProviderConstants.EMPI_UPDATE_LINK + " must be a Patient or Practitioner. Was " + theTargetType);
}
if (!EmpiUtil.isEmpiManaged(thePerson)) {
@ -111,4 +115,46 @@ public class EmpiLinkUpdaterSvcImpl implements IEmpiLinkUpdaterSvc {
throw new InvalidRequestException("The target is marked with the " + EmpiConstants.CODE_NO_EMPI_MANAGED + " tag which means it may not be EMPI linked.");
}
}
@Transactional
@Override
public IBaseParameters notDuplicatePerson(IAnyResource thePerson, IAnyResource theTarget, EmpiTransactionContext theEmpiContext) {
validateNotDuplicatePersonRequest(thePerson, theTarget);
Long personId = myIdHelperService.getPidOrThrowException(thePerson);
Long targetId = myIdHelperService.getPidOrThrowException(theTarget);
Optional<EmpiLink> oEmpiLink = myEmpiLinkDaoSvc.getLinkByPersonPidAndTargetPid(personId, targetId);
if (!oEmpiLink.isPresent()) {
throw new InvalidRequestException("No link exists between " + thePerson.getIdElement().toVersionless() + " and " + theTarget.getIdElement().toVersionless());
}
EmpiLink empiLink = oEmpiLink.get();
if (!empiLink.isPossibleDuplicate()) {
throw new InvalidRequestException(thePerson.getIdElement().toVersionless() + " and " + theTarget.getIdElement().toVersionless() + " are not linked as POSSIBLE_DUPLICATE.");
}
empiLink.setMatchResult(EmpiMatchResultEnum.NO_MATCH);
empiLink.setLinkSource(EmpiLinkSourceEnum.MANUAL);
myEmpiLinkDaoSvc.save(empiLink);
Parameters retval = (Parameters) ParametersUtil.newInstance(myFhirContext);
retval.addParameter("success", true);
return retval;
}
private void validateNotDuplicatePersonRequest(IAnyResource thePerson, IAnyResource theTarget) {
String personType = myFhirContext.getResourceType(thePerson);
String targetType = myFhirContext.getResourceType(theTarget);
if (!"Person".equals(personType)) {
throw new InvalidRequestException("First argument to " + ProviderConstants.EMPI_UPDATE_LINK + " must be a Person. Was " + personType);
}
if (!"Person".equals(targetType)) {
throw new InvalidRequestException("Second argument to " + ProviderConstants.EMPI_UPDATE_LINK + " must be a Person . Was " + targetType);
}
if (!EmpiUtil.isEmpiManaged(thePerson) || !EmpiUtil.isEmpiManaged(theTarget)) {
throw new InvalidRequestException("Only EMPI Managed Person resources may be updated via this operation. The Person resource provided is not tagged as managed by hapi-empi");
}
}
}

43
hapi-fhir-jpaserver-empi/src/main/java/ca/uhn/fhir/jpa/empi/svc/EmpiPersonMergerSvcImpl.java
View File

@ -20,6 +20,8 @@ package ca.uhn.fhir.jpa.empi.svc;
* #L%
*/
import ca.uhn.fhir.empi.api.EmpiLinkSourceEnum;
import ca.uhn.fhir.empi.api.EmpiMatchResultEnum;
import ca.uhn.fhir.empi.api.IEmpiLinkSvc;
import ca.uhn.fhir.empi.api.IEmpiPersonMergerSvc;
import ca.uhn.fhir.empi.log.Logs;
@ -56,20 +58,38 @@ public class EmpiPersonMergerSvcImpl implements IEmpiPersonMergerSvc {
@Override
@Transactional
public IAnyResource mergePersons(IAnyResource theFromPerson, IAnyResource theToPerson, EmpiTransactionContext theEmpiTransactionContext) {
// TODO EMPI replace this with a post containing the manually merged fields
myPersonHelper.mergePersonFields(theFromPerson, theToPerson);
mergeLinks(theFromPerson, theToPerson, theEmpiTransactionContext);
myEmpiResourceDaoSvc.updatePerson(theToPerson);
log(theEmpiTransactionContext, "Merged " + theFromPerson.getIdElement().toVersionless() + " into " + theToPerson.getIdElement().toVersionless());
Long toPersonPid = myIdHelperService.getPidOrThrowException(theToPerson);
myPersonHelper.mergePersonFields(theFromPerson, theToPerson);
mergeLinks(theFromPerson, theToPerson, toPersonPid, theEmpiTransactionContext);
refreshLinksAndUpdatePerson(theToPerson, theEmpiTransactionContext);
Long fromPersonPid = myIdHelperService.getPidOrThrowException(theFromPerson);
addMergeLink(fromPersonPid, toPersonPid);
myPersonHelper.deactivatePerson(theFromPerson);
myEmpiResourceDaoSvc.updatePerson(theFromPerson);
log(theEmpiTransactionContext, "Deactivated " + theFromPerson.getIdElement().toVersionless());
refreshLinksAndUpdatePerson(theFromPerson, theEmpiTransactionContext);
log(theEmpiTransactionContext, "Merged " + theFromPerson.getIdElement().toVersionless() + " into " + theToPerson.getIdElement().toVersionless());
return theToPerson;
}
private void mergeLinks(IAnyResource theFromPerson, IAnyResource theToPerson, EmpiTransactionContext theEmpiTransactionContext) {
long toPersonPid = myIdHelperService.getPidOrThrowException(theToPerson);
private void addMergeLink(Long theFromPersonPid, Long theToPersonPid) {
EmpiLink empiLink = new EmpiLink()
.setPersonPid(theFromPersonPid)
.setTargetPid(theToPersonPid)
.setMatchResult(EmpiMatchResultEnum.MATCH)
.setLinkSource(EmpiLinkSourceEnum.MANUAL);
myEmpiLinkDaoSvc.save(empiLink);
}
private void refreshLinksAndUpdatePerson(IAnyResource theToPerson, EmpiTransactionContext theEmpiTransactionContext) {
myEmpiLinkSvc.syncEmpiLinksToPersonLinks(theToPerson, theEmpiTransactionContext);
myEmpiResourceDaoSvc.updatePerson(theToPerson);
}
private void mergeLinks(IAnyResource theFromPerson, IAnyResource theToPerson, Long theToPersonPid, EmpiTransactionContext theEmpiTransactionContext) {
List<EmpiLink> incomingLinks = myEmpiLinkDaoSvc.findEmpiLinksByPersonId(theFromPerson);
List<EmpiLink> origLinks = myEmpiLinkDaoSvc.findEmpiLinksByPersonId(theToPerson);
@ -97,13 +117,10 @@ public class EmpiPersonMergerSvcImpl implements IEmpiPersonMergerSvc {
}
}
// The original links didn't contain this target, so move it over to the toPerson
incomingLink.setPersonPid(toPersonPid);
incomingLink.setPersonPid(theToPersonPid);
ourLog.trace("Saving link {}", incomingLink);
myEmpiLinkDaoSvc.save(incomingLink);
}
myEmpiLinkSvc.syncEmpiLinksToPersonLinks(theFromPerson, theEmpiTransactionContext);
myEmpiLinkSvc.syncEmpiLinksToPersonLinks(theToPerson, theEmpiTransactionContext);
}
private Optional<EmpiLink> findLinkWithMatchingTarget(List<EmpiLink> theEmpiLinks, EmpiLink theLinkWithTargetToMatch) {

1
hapi-fhir-jpaserver-empi/src/test/java/ca/uhn/fhir/jpa/empi/BaseEmpiR4Test.java
View File

@ -102,6 +102,7 @@ abstract public class BaseEmpiR4Test extends BaseJpaR4Test {
protected ServletRequestDetails myRequestDetails = new ServletRequestDetails(null);
@Override
@After
public void after() {
myEmpiLinkDao.deleteAll();

15
hapi-fhir-jpaserver-empi/src/test/java/ca/uhn/fhir/jpa/empi/provider/EmpiProviderMergePersonsR4Test.java
View File

@ -1,5 +1,8 @@
package ca.uhn.fhir.jpa.empi.provider;
import ca.uhn.fhir.empi.api.EmpiLinkSourceEnum;
import ca.uhn.fhir.empi.api.EmpiMatchResultEnum;
import ca.uhn.fhir.empi.util.AssuranceLevelUtil;
import ca.uhn.fhir.rest.server.exceptions.InvalidRequestException;
import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException;
import org.hl7.fhir.r4.model.Bundle;
@ -9,6 +12,9 @@ import org.hl7.fhir.r4.model.StringType;
import org.junit.Before;
import org.junit.Test;
import java.util.List;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertThat;
@ -48,9 +54,16 @@ public class EmpiProviderMergePersonsR4Test extends BaseProviderR4Test {
public void testMerge() {
Person mergedPerson = myEmpiProviderR4.mergePersons(myFromPersonId, myToPersonId, myRequestDetails);
assertEquals(myToPerson.getIdElement(), mergedPerson.getIdElement());
assertThat(mergedPerson, is(samePersonAs(mergedPerson)));
assertThat(mergedPerson, is(samePersonAs(myToPerson)));
assertEquals(2, getAllPersons().size());
assertEquals(1, getAllActivePersons().size());
Person fromPerson = myPersonDao.read(myFromPerson.getIdElement().toUnqualifiedVersionless());
assertThat(fromPerson.getActive(), is(false));
List<Person.PersonLinkComponent> links = fromPerson.getLink();
assertThat(links, hasSize(1));
assertThat(links.get(0).getTarget().getReference(), is (myToPerson.getIdElement().toUnqualifiedVersionless().getValue()));
assertThat(links.get(0).getAssurance(), is (AssuranceLevelUtil.getAssuranceLevel(EmpiMatchResultEnum.MATCH, EmpiLinkSourceEnum.MANUAL).toR4()));
}
@Test

47
hapi-fhir-jpaserver-empi/src/test/java/ca/uhn/fhir/jpa/empi/provider/EmpiProviderQueryLinkR4Test.java
View File

@ -4,6 +4,8 @@ import ca.uhn.fhir.empi.api.EmpiLinkSourceEnum;
import ca.uhn.fhir.empi.api.EmpiMatchResultEnum;
import ca.uhn.fhir.jpa.entity.EmpiLink;
import ca.uhn.fhir.model.primitive.IdDt;
import ca.uhn.fhir.rest.server.exceptions.ResourceNotFoundException;
import org.hl7.fhir.r4.model.BooleanType;
import org.hl7.fhir.r4.model.IdType;
import org.hl7.fhir.r4.model.Parameters;
import org.hl7.fhir.r4.model.Patient;
@ -19,12 +21,15 @@ import java.util.List;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.is;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
public class EmpiProviderQueryLinkR4Test extends BaseLinkR4Test {
private static final Logger ourLog = LoggerFactory.getLogger(EmpiProviderQueryLinkR4Test.class);
private StringType myLinkSource;
private IdType myPerson1Id;
private IdType myPerson2Id;
private StringType myPerson1Id;
private StringType myPerson2Id;
@Before
public void before() {
@ -36,20 +41,20 @@ private static final Logger ourLog = LoggerFactory.getLogger(EmpiProviderQueryLi
// Add a possible duplicate
myLinkSource = new StringType(EmpiLinkSourceEnum.AUTO.name());
Person person1 = createPerson();
myPerson1Id = person1.getIdElement().toVersionless();
myPerson1Id = new StringType(person1.getIdElement().toVersionless().getValue());
Long person1Pid = myIdHelperService.getPidOrNull(person1);
Person person2 = createPerson();
myPerson2Id = person2.getIdElement().toVersionless();
myPerson2Id = new StringType(person2.getIdElement().toVersionless().getValue());
Long person2Pid = myIdHelperService.getPidOrNull(person2);
EmpiLink empiLink = new EmpiLink().setPersonPid(person1Pid).setTargetPid(person2Pid).setMatchResult(EmpiMatchResultEnum.POSSIBLE_DUPLICATE).setLinkSource(EmpiLinkSourceEnum.AUTO);
myEmpiLinkDaoSvc.save(empiLink);
EmpiLink possibleDuplicateEmpiLink = new EmpiLink().setPersonPid(person1Pid).setTargetPid(person2Pid).setMatchResult(EmpiMatchResultEnum.POSSIBLE_DUPLICATE).setLinkSource(EmpiLinkSourceEnum.AUTO);
myEmpiLinkDaoSvc.save(possibleDuplicateEmpiLink);
}
@Test
public void testQueryLinkOneMatch() {
Parameters result = myEmpiProviderR4.queryLinks(myPersonId, myPatientId, null, null, myRequestDetails);
ourLog.info(myFhirContext.newXmlParser().setPrettyPrint(true).encodeResourceToString(result));
ourLog.info(myFhirContext.newJsonParser().setPrettyPrint(true).encodeResourceToString(result));
List<Parameters.ParametersParameterComponent> list = result.getParameter();
assertThat(list, hasSize(1));
List<Parameters.ParametersParameterComponent> part = list.get(0).getPart();
@ -82,6 +87,34 @@ private static final Logger ourLog = LoggerFactory.getLogger(EmpiProviderQueryLi
assertEmpiLink(2, part, myPerson1Id.getValue(), myPerson2Id.getValue(), EmpiMatchResultEnum.POSSIBLE_DUPLICATE);
}
@Test
public void testNotDuplicate() {
{
Parameters result = myEmpiProviderR4.getDuplicatePersons(myRequestDetails);
List<Parameters.ParametersParameterComponent> list = result.getParameter();
assertThat(list, hasSize(1));
}
{
Parameters result = myEmpiProviderR4.notDuplicate(myPerson1Id, myPerson2Id, myRequestDetails);
ourLog.info(myFhirContext.newJsonParser().setPrettyPrint(true).encodeResourceToString(result));
assertEquals("success", result.getParameterFirstRep().getName());
assertTrue(((BooleanType) (result.getParameterFirstRep().getValue())).booleanValue());
}
Parameters result = myEmpiProviderR4.getDuplicatePersons(myRequestDetails);
List<Parameters.ParametersParameterComponent> list = result.getParameter();
assertThat(list, hasSize(0));
}
@Test
public void testNotDuplicateBadId() {
try {
myEmpiProviderR4.notDuplicate(myPerson1Id, new StringType("Person/notAnId123"), myRequestDetails);
fail();
} catch (ResourceNotFoundException e) {
assertEquals("Resource Person/notAnId123 is not known", e.getMessage());
}
}
private void assertEmpiLink(int theExpectedSize, List<Parameters.ParametersParameterComponent> thePart, String thePersonId, String theTargetId, EmpiMatchResultEnum theMatchResult) {
assertThat(thePart, hasSize(theExpectedSize));
assertThat(thePart.get(0).getName(), is("personId"));

59
hapi-fhir-jpaserver-empi/src/test/java/ca/uhn/fhir/jpa/empi/svc/EmpiLinkSvcTest.java
View File

@ -16,6 +16,7 @@ import org.springframework.beans.factory.annotation.Autowired;
import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.fail;
@ -24,11 +25,13 @@ public class EmpiLinkSvcTest extends BaseEmpiR4Test {
@Autowired
IEmpiLinkSvc myEmpiLinkSvc;
@Override
@After
public void after() {
myExpungeEverythingService.expungeEverythingByType(EmpiLink.class);
super.after();
}
@Test
public void compareEmptyPatients() {
Patient patient = new Patient();
@ -60,6 +63,62 @@ public class EmpiLinkSvcTest extends BaseEmpiR4Test {
}
}
@Test
public void testPossibleDuplicate() {
assertLinkCount(0);
Person person = createPerson();
Person target = createPerson();
myEmpiLinkSvc.updateLink(person, target, EmpiMatchResultEnum.POSSIBLE_DUPLICATE, EmpiLinkSourceEnum.AUTO, createContextForCreate());
assertLinkCount(1);
}
@Test
public void testNoMatchBlocksPossibleDuplicate() {
assertLinkCount(0);
Person person = createPerson();
Person target = createPerson();
Long personPid = myIdHelperService.getPidOrNull(person);
Long targetPid = myIdHelperService.getPidOrNull(target);
assertFalse(myEmpiLinkDaoSvc.getLinkByPersonPidAndTargetPid(personPid, targetPid).isPresent());
assertFalse(myEmpiLinkDaoSvc.getLinkByPersonPidAndTargetPid(targetPid, personPid).isPresent());
saveNoMatchLink(personPid, targetPid);
myEmpiLinkSvc.updateLink(person, target, EmpiMatchResultEnum.POSSIBLE_DUPLICATE, EmpiLinkSourceEnum.AUTO, createContextForCreate());
assertFalse(myEmpiLinkDaoSvc.getEmpiLinksByPersonPidTargetPidAndMatchResult(personPid, targetPid, EmpiMatchResultEnum.POSSIBLE_DUPLICATE).isPresent());
assertLinkCount(1);
}
@Test
public void testNoMatchBlocksPossibleDuplicateReversed() {
assertLinkCount(0);
Person person = createPerson();
Person target = createPerson();
Long personPid = myIdHelperService.getPidOrNull(person);
Long targetPid = myIdHelperService.getPidOrNull(target);
assertFalse(myEmpiLinkDaoSvc.getLinkByPersonPidAndTargetPid(personPid, targetPid).isPresent());
assertFalse(myEmpiLinkDaoSvc.getLinkByPersonPidAndTargetPid(targetPid, personPid).isPresent());
saveNoMatchLink(targetPid, personPid);
myEmpiLinkSvc.updateLink(person, target, EmpiMatchResultEnum.POSSIBLE_DUPLICATE, EmpiLinkSourceEnum.AUTO, createContextForCreate());
assertFalse(myEmpiLinkDaoSvc.getEmpiLinksByPersonPidTargetPidAndMatchResult(personPid, targetPid, EmpiMatchResultEnum.POSSIBLE_DUPLICATE).isPresent());
assertLinkCount(1);
}
private void saveNoMatchLink(Long thePersonPid, Long theTargetPid) {
EmpiLink noMatchLink = new EmpiLink()
.setPersonPid(thePersonPid)
.setTargetPid(theTargetPid)
.setLinkSource(EmpiLinkSourceEnum.MANUAL)
.setMatchResult(EmpiMatchResultEnum.NO_MATCH);
myEmpiLinkDaoSvc.save(noMatchLink);
}
@Test
public void testManualEmpiLinksCannotBeModifiedBySystem() {
Person person = createPerson(buildJanePerson());

1
hapi-fhir-jpaserver-empi/src/test/java/ca/uhn/fhir/jpa/empi/svc/EmpiPersonMergerSvcTest.java
View File

@ -78,6 +78,7 @@ public class EmpiPersonMergerSvcTest extends BaseEmpiR4Test {
myInterceptorService.registerInterceptor(myEmpiStorageInterceptor);
}
@Override
@After
public void after() {
myInterceptorService.unregisterInterceptor(myEmpiStorageInterceptor);

5
hapi-fhir-jpaserver-test-utilities/src/main/java/ca/uhn/fhir/jpa/test/BaseJpaTest.java
View File

@ -21,6 +21,7 @@ package ca.uhn.fhir.jpa.test;
*/
import ca.uhn.fhir.jpa.dao.expunge.ExpungeEverythingService;
import ca.uhn.fhir.jpa.util.MemoryCacheService;
import ca.uhn.fhir.rest.api.Constants;
import ca.uhn.fhir.rest.server.exceptions.InternalErrorException;
import ca.uhn.fhir.test.utilities.UnregisterScheduledProcessor;
@ -66,10 +67,14 @@ public abstract class BaseJpaTest {
@Autowired
ApplicationContext myApplicationContext;
@Autowired
MemoryCacheService myMemoryCacheService;
@After
public void after() {
ourLog.info("\n --- @After ---");
myExpungeEverythingService.expungeEverything(null);
myMemoryCacheService.invalidateAllCaches();
}
public TransactionTemplate newTxTemplate() {

3
hapi-fhir-server-empi/src/main/java/ca/uhn/fhir/empi/api/IEmpiLinkUpdaterSvc.java
View File

@ -22,7 +22,10 @@ package ca.uhn.fhir.empi.api;
import ca.uhn.fhir.empi.model.EmpiTransactionContext;
import org.hl7.fhir.instance.model.api.IAnyResource;
import org.hl7.fhir.instance.model.api.IBaseParameters;
public interface IEmpiLinkUpdaterSvc {
IAnyResource updateLink(IAnyResource thePerson, IAnyResource theTarget, EmpiMatchResultEnum theMatchResult, EmpiTransactionContext theEmpiContext);
IBaseParameters notDuplicatePerson(IAnyResource thePerson, IAnyResource theTarget, EmpiTransactionContext theEmpiContext);
}

7
hapi-fhir-server-empi/src/main/java/ca/uhn/fhir/empi/provider/BaseEmpiProvider.java
View File

@ -40,7 +40,7 @@ import org.hl7.fhir.instance.model.api.IPrimitiveType;
public abstract class BaseEmpiProvider {
private final FhirContext myFhirContext;
protected final FhirContext myFhirContext;
private final IResourceLoader myResourceLoader;
public BaseEmpiProvider(FhirContext theFhirContext, IResourceLoader theResourceLoader) {
@ -125,6 +125,11 @@ public abstract class BaseEmpiProvider {
}
}
protected void validateNotDuplicateParameters(IPrimitiveType<String> thePersonId, IPrimitiveType<String> theTargetId) {
validateNotNull(ProviderConstants.EMPI_UPDATE_LINK_PERSON_ID, thePersonId);
validateNotNull(ProviderConstants.EMPI_UPDATE_LINK_TARGET_ID, theTargetId);
}
protected EmpiTransactionContext createEmpiContext(RequestDetails theRequestDetails) {
TransactionLogMessages transactionLogMessages = TransactionLogMessages.createFromTransactionGuid(theRequestDetails.getTransactionGuid());
return new EmpiTransactionContext(transactionLogMessages, EmpiTransactionContext.OperationType.MERGE_PERSONS);

15
hapi-fhir-server-empi/src/main/java/ca/uhn/fhir/empi/provider/EmpiProviderDstu3.java
View File

@ -134,4 +134,19 @@ public class EmpiProviderDstu3 extends BaseEmpiProvider {
public Parameters getDuplicatePersons(ServletRequestDetails theRequestDetails) {
return (Parameters) myEmpiLinkQuerySvc.getPossibleDuplicates(createEmpiContext(theRequestDetails));
}
@Operation(name = ProviderConstants.EMPI_NOT_DUPLICATE)
// TODO KHS can this return void?
public Parameters notDuplicate(@OperationParam(name=ProviderConstants.EMPI_QUERY_LINKS_PERSON_ID, min = 1, max = 1) StringType thePersonId,
@OperationParam(name=ProviderConstants.EMPI_QUERY_LINKS_TARGET_ID, min = 1, max = 1) StringType theTargetId,
ServletRequestDetails theRequestDetails) {
validateNotDuplicateParameters(thePersonId, theTargetId);
IAnyResource person = getLatestPersonFromIdOrThrowException(ProviderConstants.EMPI_UPDATE_LINK_PERSON_ID, thePersonId.getValue());
IAnyResource target = getLatestPersonFromIdOrThrowException(ProviderConstants.EMPI_UPDATE_LINK_TARGET_ID, theTargetId.getValue());
validateSameVersion(person, thePersonId);
validateSameVersion(target, theTargetId);
return (Parameters) myEmpiLinkUpdaterSvc.notDuplicatePerson(person, target, createEmpiContext(theRequestDetails));
}
}

14
hapi-fhir-server-empi/src/main/java/ca/uhn/fhir/empi/provider/EmpiProviderR4.java
View File

@ -135,4 +135,18 @@ public class EmpiProviderR4 extends BaseEmpiProvider {
public Parameters getDuplicatePersons(ServletRequestDetails theRequestDetails) {
return (Parameters) myEmpiLinkQuerySvc.getPossibleDuplicates(createEmpiContext(theRequestDetails));
}
@Operation(name = ProviderConstants.EMPI_NOT_DUPLICATE)
public Parameters notDuplicate(@OperationParam(name=ProviderConstants.EMPI_QUERY_LINKS_PERSON_ID, min = 1, max = 1) StringType thePersonId,
@OperationParam(name=ProviderConstants.EMPI_QUERY_LINKS_TARGET_ID, min = 1, max = 1) StringType theTargetId,
ServletRequestDetails theRequestDetails) {
validateNotDuplicateParameters(thePersonId, theTargetId);
IAnyResource person = getLatestPersonFromIdOrThrowException(ProviderConstants.EMPI_UPDATE_LINK_PERSON_ID, thePersonId.getValue());
IAnyResource target = getLatestPersonFromIdOrThrowException(ProviderConstants.EMPI_UPDATE_LINK_TARGET_ID, theTargetId.getValue());
validateSameVersion(person, thePersonId);
validateSameVersion(target, theTargetId);
return (Parameters) myEmpiLinkUpdaterSvc.notDuplicatePerson(person, target, createEmpiContext(theRequestDetails));
}
}

1
hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/provider/ProviderConstants.java
View File

@ -80,4 +80,5 @@ public class ProviderConstants {
public static final String EMPI_QUERY_LINKS_LINK_SOURCE = "linkSource";
public static final String EMPI_DUPLICATE_PERSONS = "$empi-duplicate-persons";
public static final String EMPI_NOT_DUPLICATE = "$empi-not-duplicate";
}